2023-11-06T02:49:12.197 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-06T02:49:12.198 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738 --name yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/orig.config.yaml 2023-11-06T02:49:12.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T02:49:12.326 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738 branch: reef-release description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} email: yweinste@redhat.com first_in_suite: false job_id: '7448738' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1834 sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgeOL5FtyrcN1GMmQwuIlZ0j86FC3xLAkooaUK1Flxytgx8S3U7T8Dnh0JRIiIgs6RG3WxlxL9T39UpfPzOnXQ= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAx7yMHLyHmaa5GecES0RDKKy6Zo72ZjhkX6pGRAgXz1f3EtApBvRPCC+Lteqqy4ps5/3/yIcm1XXP7qnsP8+jA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-05_23:44:49 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822 2023-11-06T02:49:12.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa; will attempt to use it 2023-11-06T02:49:12.327 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa/tasks 2023-11-06T02:49:12.330 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-06T02:49:12.332 INFO:teuthology.task.internal:Checking packages... 2023-11-06T02:49:12.357 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5' 2023-11-06T02:49:12.358 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-06T02:49:12.358 INFO:teuthology.packaging:ref: None 2023-11-06T02:49:12.359 INFO:teuthology.packaging:tag: None 2023-11-06T02:49:12.359 INFO:teuthology.packaging:branch: reef-release 2023-11-06T02:49:12.359 INFO:teuthology.packaging:sha1: f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T02:49:12.360 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef-release 2023-11-06T02:49:12.580 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T02:49:12.581 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-06T02:49:12.588 INFO:teuthology.task.internal:no buildpackages task found 2023-11-06T02:49:12.589 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-06T02:49:12.637 INFO:teuthology.task.internal:Saving configuration 2023-11-06T02:49:12.655 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-06T02:49:12.663 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T02:49:12.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 02:42:54.301321', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOdvtGIZiDRjBRBk/CqHFyXQVLhmvPhjeVsvAVssuPc6'} 2023-11-06T02:49:12.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 02:42:54.303365', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOZgQPmYYaJ+Md7bdW3wrAvyLc7gB5k0RyilRH/LE/u6'} 2023-11-06T02:49:12.715 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-06T02:49:12.720 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-06T02:49:12.721 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-06T02:49:12.721 INFO:teuthology.run_tasks:Running task console_log... 2023-11-06T02:49:12.820 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3a5b290af0>, signals=[15]) 2023-11-06T02:49:12.822 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-06T02:49:12.827 INFO:teuthology.task.internal:Opening connections... 2023-11-06T02:49:12.828 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T02:49:12.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:49:12.906 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T02:49:12.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:49:12.983 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-06T02:49:12.990 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-11-06T02:49:12.996 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-11-06T02:49:12.997 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-11-06T02:49:13.047 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-06T02:49:13.047 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu" 2023-11-06T02:49:13.048 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="22.04" 2023-11-06T02:49:13.048 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-06T02:49:13.048 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=jammy 2023-11-06T02:49:13.048 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu 2023-11-06T02:49:13.048 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian 2023-11-06T02:49:13.049 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-06T02:49:13.049 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-06T02:49:13.049 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-06T02:49:13.049 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-06T02:49:13.050 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=jammy 2023-11-06T02:49:13.050 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-11-06T02:49:13.075 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2023-11-06T02:49:13.080 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2023-11-06T02:49:13.080 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-11-06T02:49:13.130 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-06T02:49:13.131 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2023-11-06T02:49:13.131 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="22.04" 2023-11-06T02:49:13.131 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-06T02:49:13.131 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=jammy 2023-11-06T02:49:13.132 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2023-11-06T02:49:13.132 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2023-11-06T02:49:13.132 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-06T02:49:13.132 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-06T02:49:13.132 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-06T02:49:13.133 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-06T02:49:13.133 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=jammy 2023-11-06T02:49:13.133 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-11-06T02:49:13.158 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-06T02:49:13.168 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-06T02:49:13.195 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-06T02:49:13.196 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-11-06T02:49:13.198 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2023-11-06T02:49:13.203 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-06T02:49:13.220 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-06T02:49:13.221 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-11-06T02:49:13.245 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2023-11-06T02:49:13.268 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-06T02:49:13.325 INFO:teuthology.run_tasks:Running task kernel... 2023-11-06T02:49:13.340 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-06T02:49:13.341 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-06T02:49:13.342 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T02:49:13.343 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-06T02:49:13.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:49:13.348 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-11-06T02:49:13.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T02:49:13.349 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2023-11-06T02:49:13.396 INFO:teuthology.orchestra.run.smithi157.stdout:5.15.0-56-generic 2023-11-06T02:49:13.397 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.15.0-56-generic 2023-11-06T02:49:13.397 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get clean 2023-11-06T02:49:13.398 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-56-generic 2023-11-06T02:49:13.399 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.15.0-56-generic 2023-11-06T02:49:13.399 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2023-11-06T02:49:13.511 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2023-11-06T02:49:13.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2023-11-06T02:49:13.649 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-11-06T02:49:13.673 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:49:13.674 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-11-06T02:49:13.674 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-11-06T02:49:13.737 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-11-06T02:49:13.791 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:49:13.864 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [360 kB] 2023-11-06T02:49:13.880 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-11-06T02:49:13.916 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [360 kB] 2023-11-06T02:49:13.935 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [940 kB] 2023-11-06T02:49:13.972 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [186 kB] 2023-11-06T02:49:13.976 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-11-06T02:49:13.977 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,085 kB] 2023-11-06T02:49:13.995 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-11-06T02:49:13.996 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [176 kB] 2023-11-06T02:49:13.999 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T02:49:14.000 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,149 kB] 2023-11-06T02:49:14.000 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [563 kB] 2023-11-06T02:49:14.005 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [940 kB] 2023-11-06T02:49:14.008 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [797 kB] 2023-11-06T02:49:14.018 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [146 kB] 2023-11-06T02:49:14.019 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-11-06T02:49:14.020 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-11-06T02:49:14.030 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-11-06T02:49:14.038 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-11-06T02:49:14.038 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-11-06T02:49:14.042 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [186 kB] 2023-11-06T02:49:14.050 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-11-06T02:49:14.050 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,085 kB] 2023-11-06T02:49:14.064 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-11-06T02:49:14.065 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [176 kB] 2023-11-06T02:49:14.068 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T02:49:14.068 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [797 kB] 2023-11-06T02:49:14.073 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [524 kB] 2023-11-06T02:49:14.077 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [563 kB] 2023-11-06T02:49:14.079 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [245 kB] 2023-11-06T02:49:14.117 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-11-06T02:49:14.118 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [146 kB] 2023-11-06T02:49:14.118 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-11-06T02:49:14.118 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,105 kB] 2023-11-06T02:49:14.119 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-11-06T02:49:14.119 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-11-06T02:49:14.120 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-11-06T02:49:14.120 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-11-06T02:49:14.120 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-11-06T02:49:14.130 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [179 kB] 2023-11-06T02:49:14.132 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T02:49:14.132 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [999 kB] 2023-11-06T02:49:14.142 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [661 kB] 2023-11-06T02:49:14.149 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [219 kB] 2023-11-06T02:49:14.151 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-11-06T02:49:14.153 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-11-06T02:49:14.153 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-11-06T02:49:14.154 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-11-06T02:49:14.154 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-11-06T02:49:14.154 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-11-06T02:49:14.155 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-11-06T02:49:14.169 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-11-06T02:49:14.170 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-11-06T02:49:14.170 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-11-06T02:49:14.171 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-11-06T02:49:14.186 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-11-06T02:49:14.187 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-11-06T02:49:14.267 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-11-06T02:49:14.423 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [524 kB] 2023-11-06T02:49:14.630 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,149 kB] 2023-11-06T02:49:14.858 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [245 kB] 2023-11-06T02:49:14.862 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-11-06T02:49:14.863 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,105 kB] 2023-11-06T02:49:14.941 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-11-06T02:49:14.941 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [179 kB] 2023-11-06T02:49:14.945 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-11-06T02:49:15.110 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [661 kB] 2023-11-06T02:49:15.757 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [999 kB] 2023-11-06T02:49:15.869 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [219 kB] 2023-11-06T02:49:15.888 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-11-06T02:49:15.889 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-11-06T02:49:15.892 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-11-06T02:49:15.893 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-11-06T02:49:15.894 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-11-06T02:49:15.894 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-11-06T02:49:15.899 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-11-06T02:49:15.903 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-11-06T02:49:16.095 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-11-06T02:49:16.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-11-06T02:49:16.309 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-11-06T02:49:16.348 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-11-06T02:49:16.413 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-11-06T02:49:19.471 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 10.1 MB in 2s (6,687 kB/s) 2023-11-06T02:49:20.870 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T02:49:20.887 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T02:49:20.891 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 10.1 MB in 3s (3,558 kB/s) 2023-11-06T02:49:20.946 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T02:49:21.133 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2023-11-06T02:49:21.134 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2023-11-06T02:49:21.290 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:49:21.290 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:49:21.291 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:49:21.291 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:49:21.291 INFO:teuthology.orchestra.run.smithi157.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-06T02:49:21.292 INFO:teuthology.orchestra.run.smithi157.stdout: python2.7-minimal 2023-11-06T02:49:21.292 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:49:21.293 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2023-11-06T02:49:21.293 INFO:teuthology.orchestra.run.smithi157.stdout: linux-generic linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T02:49:21.293 INFO:teuthology.orchestra.run.smithi157.stdout: linux-headers-generic linux-image-5.15.0-88-generic 2023-11-06T02:49:21.293 INFO:teuthology.orchestra.run.smithi157.stdout: linux-modules-5.15.0-88-generic linux-modules-extra-5.15.0-88-generic 2023-11-06T02:49:21.294 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2023-11-06T02:49:21.294 INFO:teuthology.orchestra.run.smithi157.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-11-06T02:49:21.294 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2023-11-06T02:49:21.294 INFO:teuthology.orchestra.run.smithi157.stdout: thermald 2023-11-06T02:49:21.343 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2023-11-06T02:49:21.343 INFO:teuthology.orchestra.run.smithi157.stdout: linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T02:49:21.343 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.15.0-88-generic linux-modules-5.15.0-88-generic 2023-11-06T02:49:21.343 INFO:teuthology.orchestra.run.smithi157.stdout: linux-modules-extra-5.15.0-88-generic 2023-11-06T02:49:21.344 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2023-11-06T02:49:21.345 INFO:teuthology.orchestra.run.smithi157.stdout: linux-generic linux-headers-generic linux-image-generic 2023-11-06T02:49:21.394 INFO:teuthology.orchestra.run.smithi157.stdout:3 upgraded, 5 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T02:49:21.395 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 113 MB of archives. 2023-11-06T02:49:21.395 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 582 MB of additional disk space will be used. 2023-11-06T02:49:21.395 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-88-generic amd64 5.15.0-88.98 [22.4 MB] 2023-11-06T02:49:21.867 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-88-generic amd64 5.15.0-88.98 [11.5 MB] 2023-11-06T02:49:21.976 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-88-generic amd64 5.15.0-88.98 [64.0 MB] 2023-11-06T02:49:22.273 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T02:49:22.291 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T02:49:22.348 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T02:49:22.535 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2023-11-06T02:49:22.535 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2023-11-06T02:49:22.684 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:49:22.685 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:49:22.685 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:49:22.685 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:49:22.685 INFO:teuthology.orchestra.run.smithi195.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-06T02:49:22.686 INFO:teuthology.orchestra.run.smithi195.stdout: python2.7-minimal 2023-11-06T02:49:22.686 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:49:22.687 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2023-11-06T02:49:22.687 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T02:49:22.687 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.15.0-88-generic 2023-11-06T02:49:22.687 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.15.0-88-generic linux-modules-extra-5.15.0-88-generic 2023-11-06T02:49:22.688 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2023-11-06T02:49:22.688 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-11-06T02:49:22.688 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2023-11-06T02:49:22.688 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2023-11-06T02:49:22.737 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2023-11-06T02:49:22.737 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.15.0-88 linux-headers-5.15.0-88-generic 2023-11-06T02:49:22.737 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.15.0-88-generic linux-modules-5.15.0-88-generic 2023-11-06T02:49:22.738 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.15.0-88-generic 2023-11-06T02:49:22.738 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2023-11-06T02:49:22.738 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2023-11-06T02:49:22.788 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.88.85 [1,698 B] 2023-11-06T02:49:22.788 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.88.85 [2,488 B] 2023-11-06T02:49:22.788 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88 all 5.15.0-88.98 [12.3 MB] 2023-11-06T02:49:22.913 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88-generic amd64 5.15.0-88.98 [2,878 kB] 2023-11-06T02:49:22.940 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T02:49:22.941 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 113 MB of archives. 2023-11-06T02:49:22.941 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 582 MB of additional disk space will be used. 2023-11-06T02:49:22.941 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-88-generic amd64 5.15.0-88.98 [22.4 MB] 2023-11-06T02:49:22.994 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.88.85 [2,340 B] 2023-11-06T02:49:23.392 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 113 MB in 2s (68.8 MB/s) 2023-11-06T02:49:23.570 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-modules-5.15.0-88-generic. 2023-11-06T02:49:24.552 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-88-generic amd64 5.15.0-88.98 [11.5 MB] 2023-11-06T02:49:25.075 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-88-generic amd64 5.15.0-88.98 [64.0 MB] 2023-11-06T02:49:25.644 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-11-06T02:49:25.648 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../0-linux-modules-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:25.768 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:27.436 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.88.85 [1,698 B] 2023-11-06T02:49:27.437 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.88.85 [2,488 B] 2023-11-06T02:49:27.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88 all 5.15.0-88.98 [12.3 MB] 2023-11-06T02:49:28.102 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-image-5.15.0-88-generic. 2023-11-06T02:49:28.124 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../1-linux-image-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:28.212 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:28.726 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-88-generic amd64 5.15.0-88.98 [2,878 kB] 2023-11-06T02:49:28.943 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.88.85 [2,340 B] 2023-11-06T02:49:29.081 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-88-generic. 2023-11-06T02:49:29.102 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:29.139 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:29.352 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 113 MB in 6s (18.2 MB/s) 2023-11-06T02:49:29.508 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.15.0-88-generic. 2023-11-06T02:49:31.591 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-11-06T02:49:31.597 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:31.772 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:33.823 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.15.0-88-generic. 2023-11-06T02:49:33.844 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:33.943 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:34.426 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-88-generic. 2023-11-06T02:49:34.447 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:34.476 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:36.040 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../3-linux-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:36.167 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:36.567 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:36.712 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:37.032 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-headers-5.15.0-88. 2023-11-06T02:49:37.054 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../5-linux-headers-5.15.0-88_5.15.0-88.98_all.deb ... 2023-11-06T02:49:37.098 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T02:49:40.978 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:41.120 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:41.444 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:41.565 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:41.859 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-88. 2023-11-06T02:49:41.871 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.15.0-88_5.15.0-88.98_all.deb ... 2023-11-06T02:49:41.901 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T02:49:43.078 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-headers-5.15.0-88-generic. 2023-11-06T02:49:43.117 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../6-linux-headers-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:43.153 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:47.228 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:47.271 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-88-generic. 2023-11-06T02:49:47.307 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.15.0-88-generic_5.15.0-88.98_amd64.deb ... 2023-11-06T02:49:47.354 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-headers-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:47.362 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:47.737 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T02:49:47.884 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:48.009 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-headers-generic (5.15.0.88.85) ... 2023-11-06T02:49:48.162 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:49.469 INFO:teuthology.orchestra.run.smithi157.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:49:49.470 INFO:teuthology.orchestra.run.smithi157.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-11-06T02:49:49.470 INFO:teuthology.orchestra.run.smithi157.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:49:49.470 INFO:teuthology.orchestra.run.smithi157.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-88-generic 2023-11-06T02:49:49.657 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:50.548 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.88.85_amd64.deb ... 2023-11-06T02:49:50.673 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.15.0.88.85) over (5.15.0.56.54) ... 2023-11-06T02:49:51.015 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-88 (5.15.0-88.98) ... 2023-11-06T02:49:51.091 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:51.129 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:51.255 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.15.0.88.85) ... 2023-11-06T02:49:51.382 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:52.459 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-generic (5.15.0.88.85) ... 2023-11-06T02:49:52.601 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-generic (5.15.0.88.85) ... 2023-11-06T02:49:52.714 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:49:52.714 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-11-06T02:49:52.714 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:49:52.715 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-88-generic 2023-11-06T02:49:52.728 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:52.803 INFO:teuthology.orchestra.run.smithi157.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-11-06T02:49:52.804 INFO:teuthology.orchestra.run.smithi157.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:49:52.870 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:54.278 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:55.888 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.15.0.88.85) ... 2023-11-06T02:49:56.014 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.15.0.88.85) ... 2023-11-06T02:49:56.165 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.15.0-88-generic (5.15.0-88.98) ... 2023-11-06T02:49:56.242 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-11-06T02:49:56.242 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:50:07.320 INFO:teuthology.orchestra.run.smithi157.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-11-06T02:50:07.321 INFO:teuthology.orchestra.run.smithi157.stdout:Sourcing file `/etc/default/grub' 2023-11-06T02:50:07.338 INFO:teuthology.orchestra.run.smithi157.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T02:50:07.362 INFO:teuthology.orchestra.run.smithi157.stdout:Generating grub configuration file ... 2023-11-06T02:50:07.725 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:50:07.739 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:50:08.101 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:50:08.106 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T02:50:08.179 INFO:teuthology.orchestra.run.smithi157.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T02:50:08.185 INFO:teuthology.orchestra.run.smithi157.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T02:50:08.359 INFO:teuthology.orchestra.run.smithi157.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T02:50:08.359 INFO:teuthology.orchestra.run.smithi157.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T02:50:08.359 INFO:teuthology.orchestra.run.smithi157.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T02:50:08.392 INFO:teuthology.orchestra.run.smithi157.stdout:done 2023-11-06T02:50:08.874 DEBUG:teuthology.orchestra.run.smithi157:> dpkg -s linux-image-generic 2023-11-06T02:50:08.895 INFO:teuthology.orchestra.run.smithi157.stdout:Package: linux-image-generic 2023-11-06T02:50:08.896 INFO:teuthology.orchestra.run.smithi157.stdout:Status: install ok installed 2023-11-06T02:50:08.896 INFO:teuthology.orchestra.run.smithi157.stdout:Priority: optional 2023-11-06T02:50:08.896 INFO:teuthology.orchestra.run.smithi157.stdout:Section: kernel 2023-11-06T02:50:08.896 INFO:teuthology.orchestra.run.smithi157.stdout:Installed-Size: 20 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: amd64 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Source: linux-meta 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Version: 5.15.0.88.85 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T02:50:08.897 INFO:teuthology.orchestra.run.smithi157.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T02:50:08.898 INFO:teuthology.orchestra.run.smithi157.stdout:Recommends: thermald 2023-11-06T02:50:08.898 INFO:teuthology.orchestra.run.smithi157.stdout:Description: Generic Linux kernel image 2023-11-06T02:50:08.898 INFO:teuthology.orchestra.run.smithi157.stdout: This package will always depend on the latest generic kernel image 2023-11-06T02:50:08.898 INFO:teuthology.orchestra.run.smithi157.stdout: available. 2023-11-06T02:50:08.899 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-88-generic 2023-11-06T02:50:08.899 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-06T02:50:08.899 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-06T02:50:08.900 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-06T02:50:08.900 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2023-11-06T02:50:08.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get clean 2023-11-06T02:50:09.045 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2023-11-06T02:50:09.168 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:50:09.179 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T02:50:09.199 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T02:50:09.326 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T02:50:10.311 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-11-06T02:50:10.311 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2023-11-06T02:50:10.320 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T02:50:10.344 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2023-11-06T02:50:10.431 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T02:50:10.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T02:50:10.501 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T02:50:10.590 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:50:10.612 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:50:10.667 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2023-11-06T02:50:10.668 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2023-11-06T02:50:10.811 INFO:teuthology.orchestra.run.smithi157.stdout:linux-image-generic is already the newest version (5.15.0.88.85). 2023-11-06T02:50:10.812 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:50:10.812 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:50:10.812 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:50:10.812 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:50:10.813 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:50:10.813 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:50:10.813 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:50:10.847 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 0 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T02:50:10.849 DEBUG:teuthology.orchestra.run.smithi157:> dpkg -s linux-image-generic 2023-11-06T02:50:10.905 INFO:teuthology.orchestra.run.smithi157.stdout:Package: linux-image-generic 2023-11-06T02:50:10.905 INFO:teuthology.orchestra.run.smithi157.stdout:Status: install ok installed 2023-11-06T02:50:10.906 INFO:teuthology.orchestra.run.smithi157.stdout:Priority: optional 2023-11-06T02:50:10.906 INFO:teuthology.orchestra.run.smithi157.stdout:Section: kernel 2023-11-06T02:50:10.906 INFO:teuthology.orchestra.run.smithi157.stdout:Installed-Size: 20 2023-11-06T02:50:10.906 INFO:teuthology.orchestra.run.smithi157.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T02:50:10.906 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: amd64 2023-11-06T02:50:10.907 INFO:teuthology.orchestra.run.smithi157.stdout:Source: linux-meta 2023-11-06T02:50:10.907 INFO:teuthology.orchestra.run.smithi157.stdout:Version: 5.15.0.88.85 2023-11-06T02:50:10.907 INFO:teuthology.orchestra.run.smithi157.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T02:50:10.907 INFO:teuthology.orchestra.run.smithi157.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T02:50:10.908 INFO:teuthology.orchestra.run.smithi157.stdout:Recommends: thermald 2023-11-06T02:50:10.908 INFO:teuthology.orchestra.run.smithi157.stdout:Description: Generic Linux kernel image 2023-11-06T02:50:10.908 INFO:teuthology.orchestra.run.smithi157.stdout: This package will always depend on the latest generic kernel image 2023-11-06T02:50:10.908 INFO:teuthology.orchestra.run.smithi157.stdout: available. 2023-11-06T02:50:10.909 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-11-06T02:50:10.910 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:50:10.911 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T02:50:10.951 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.xzNeBT2mFr 2023-11-06T02:50:10.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub/grub.cfg /tmp/tmp.xzNeBT2mFr 2023-11-06T02:50:10.975 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T02:50:10.980 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T02:50:11.007 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.xzNeBT2mFr 2023-11-06T02:50:11.118 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.xzNeBT2mFr is 10KB 2023-11-06T02:50:11.163 INFO:teuthology.orchestra.run.smithi195.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T02:50:11.164 INFO:teuthology.orchestra.run.smithi195.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T02:50:11.164 INFO:teuthology.orchestra.run.smithi195.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T02:50:11.172 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.xzNeBT2mFr 2023-11-06T02:50:11.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-11-06T02:50:11.202 INFO:teuthology.orchestra.run.smithi195.stdout:done 2023-11-06T02:50:11.231 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T02:50:11.232 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-11-06T02:50:11.232 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-11-06T02:50:11.314 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-88-generic 2023-11-06T02:50:11.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo update-grub 2023-11-06T02:50:11.636 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2023-11-06T02:50:11.660 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2023-11-06T02:50:11.661 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2023-11-06T02:50:11.661 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2023-11-06T02:50:11.661 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2023-11-06T02:50:11.661 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 20 2023-11-06T02:50:11.662 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T02:50:11.662 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2023-11-06T02:50:11.662 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2023-11-06T02:50:11.662 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.88.85 2023-11-06T02:50:11.663 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T02:50:11.663 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T02:50:11.663 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2023-11-06T02:50:11.663 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2023-11-06T02:50:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2023-11-06T02:50:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2023-11-06T02:50:11.664 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-88-generic 2023-11-06T02:50:11.664 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-06T02:50:11.664 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-06T02:50:11.665 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-06T02:50:11.665 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2023-11-06T02:50:11.665 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2023-11-06T02:50:11.819 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2023-11-06T02:50:11.935 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T02:50:11.944 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T02:50:11.961 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T02:50:12.059 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T02:50:12.599 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2023-11-06T02:50:12.600 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T02:50:12.603 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2023-11-06T02:50:12.792 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:50:12.798 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:50:13.065 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:50:13.069 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T02:50:13.110 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T02:50:13.114 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T02:50:13.181 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T02:50:13.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-11-06T02:50:13.249 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T02:50:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T02:50:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T02:50:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T02:50:13.276 INFO:teuthology.orchestra.run.smithi157.stderr:done 2023-11-06T02:50:13.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2023-11-06T02:50:13.429 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2023-11-06T02:50:13.430 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2023-11-06T02:50:13.573 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.15.0.88.85). 2023-11-06T02:50:13.573 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T02:50:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T02:50:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T02:50:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T02:50:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T02:50:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T02:50:13.575 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T02:50:13.610 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 270 not upgraded. 2023-11-06T02:50:13.612 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2023-11-06T02:50:13.668 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2023-11-06T02:50:13.668 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2023-11-06T02:50:13.668 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2023-11-06T02:50:13.669 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2023-11-06T02:50:13.669 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 20 2023-11-06T02:50:13.669 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2023-11-06T02:50:13.669 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2023-11-06T02:50:13.669 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2023-11-06T02:50:13.670 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.88.85 2023-11-06T02:50:13.670 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-88), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-11-06T02:50:13.670 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-88-generic, linux-modules-extra-5.15.0-88-generic, linux-firmware, intel-microcode, amd64-microcode 2023-11-06T02:50:13.670 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2023-11-06T02:50:13.670 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2023-11-06T02:50:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2023-11-06T02:50:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2023-11-06T02:50:13.671 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2023-11-06T02:50:13.713 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.hwcaU3tR3O 2023-11-06T02:50:13.713 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hwcaU3tR3O 2023-11-06T02:50:13.782 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.hwcaU3tR3O 2023-11-06T02:50:13.906 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.hwcaU3tR3O is 10KB 2023-11-06T02:50:13.919 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.hwcaU3tR3O 2023-11-06T02:50:13.924 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-11-06T02:50:13.977 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T02:50:13.978 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-11-06T02:50:13.978 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-11-06T02:50:14.044 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-88-generic 2023-11-06T02:50:14.044 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2023-11-06T02:50:15.256 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2023-11-06T02:50:15.257 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-06T02:50:15.259 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2023-11-06T02:50:15.453 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-88-generic 2023-11-06T02:50:15.459 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-88-generic 2023-11-06T02:50:15.681 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-06T02:50:15.686 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-06T02:50:15.726 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-06T02:50:15.731 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-06T02:50:15.881 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-06T02:50:15.881 INFO:teuthology.orchestra.run.smithi195.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-06T02:50:15.882 INFO:teuthology.orchestra.run.smithi195.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-06T02:50:15.897 INFO:teuthology.orchestra.run.smithi195.stderr:done 2023-11-06T02:50:15.899 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2023-11-06T02:50:43.306 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-06T02:50:43.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:50:43.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:50:45.902 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-06T02:50:45.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T02:50:45.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:51:17.754 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-06T02:51:26.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:51:26.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:51:29.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-06T02:51:41.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:51:41.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:51:45.906 DEBUG:teuthology.orchestra.remote:timed out 2023-11-06T02:51:54.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T02:51:54.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:51:55.931 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-06T02:52:00.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-06T02:52:07.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T02:52:07.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:52:08.210 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-11-06T02:52:08.843 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T02:52:08.843 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-88-generic"... 2023-11-06T02:52:08.844 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2023-11-06T02:52:08.887 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-88-generic 2023-11-06T02:52:08.887 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-88-generic vs 5.15.0-88-generic 2023-11-06T02:52:08.888 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T02:52:08.888 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-06T02:52:09.889 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-06T02:52:09.889 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-06T02:52:10.252 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2023-11-06T02:52:10.281 DEBUG:teuthology.parallel:result is None 2023-11-06T02:52:15.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:52:15.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:52:16.282 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-11-06T02:52:34.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:52:34.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T02:52:34.487 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-11-06T02:52:35.069 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T02:52:35.069 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-88-generic"... 2023-11-06T02:52:35.069 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-11-06T02:52:35.115 INFO:teuthology.orchestra.run.smithi157.stdout:5.15.0-88-generic 2023-11-06T02:52:35.115 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-88-generic vs 5.15.0-88-generic 2023-11-06T02:52:35.116 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-06T02:52:35.116 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-06T02:52:36.117 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-06T02:52:36.117 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-06T02:52:36.205 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2023-11-06T02:52:36.242 DEBUG:teuthology.parallel:result is None 2023-11-06T02:52:36.243 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-06T02:52:36.249 INFO:teuthology.task.internal:Creating test directory... 2023-11-06T02:52:36.250 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T02:52:36.254 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-06T02:52:36.260 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-06T02:52:36.291 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-06T02:52:36.308 INFO:teuthology.task.internal:Creating archive directory... 2023-11-06T02:52:36.309 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T02:52:36.312 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-06T02:52:36.410 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-06T02:52:36.417 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-06T02:52:36.418 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T02:52:36.420 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-06T02:52:36.442 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T02:52:36.443 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T02:52:36.452 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T02:52:36.452 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-06T02:52:36.454 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-06T02:52:36.461 INFO:teuthology.task.internal:Configuring sudo... 2023-11-06T02:52:36.462 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T02:52:36.497 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-06T02:52:36.521 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-06T02:52:36.531 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-06T02:52:36.531 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T02:52:36.554 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-06T02:52:36.567 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T02:52:36.604 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T02:52:36.651 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T02:52:36.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T02:52:36.708 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-06T02:52:36.714 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-06T02:52:36.763 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T02:52:36.764 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-06T02:52:36.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-11-06T02:52:36.840 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2023-11-06T02:52:36.930 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-06T02:52:36.939 INFO:teuthology.task.internal:Starting timer... 2023-11-06T02:52:36.939 INFO:teuthology.run_tasks:Running task pcp... 2023-11-06T02:52:36.980 INFO:teuthology.run_tasks:Running task selinux... 2023-11-06T02:52:37.005 DEBUG:teuthology.task.selinux:Excluding smithi157: OS 'ubuntu' does not support SELinux 2023-11-06T02:52:37.005 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2023-11-06T02:52:37.006 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-06T02:52:37.006 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-06T02:52:37.007 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-06T02:52:37.007 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-06T02:52:37.020 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-06T02:52:37.021 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-06T02:52:37.041 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-06T02:52:37.043 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi157.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-06T03:00:20.142 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi157.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2023-11-06T03:00:20.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T03:00:20.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T03:00:20.224 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-11-06T03:00:20.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-11-06T03:00:20.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T03:00:20.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-06T03:00:20.417 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-11-06T03:00:20.530 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-06T03:00:20.530 INFO:teuthology.run_tasks:Running task clock... 2023-11-06T03:00:20.541 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-06T03:00:20.542 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T03:00:20.542 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T03:00:20.545 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-06T03:00:20.545 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T03:00:20.576 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-06T03:00:20.576 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Command line: ntpd -gq 2023-11-06T03:00:20.577 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: ---------------------------------------------------- 2023-11-06T03:00:20.577 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: ntp-4 is maintained by Network Time Foundation, 2023-11-06T03:00:20.577 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-06T03:00:20.577 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: corporation. Support and training for ntp-4 are 2023-11-06T03:00:20.577 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: available at https://www.nwtime.org/support 2023-11-06T03:00:20.578 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: ---------------------------------------------------- 2023-11-06T03:00:20.578 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: proto: precision = 0.060 usec (-24) 2023-11-06T03:00:20.578 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: basedate set to 2022-02-04 2023-11-06T03:00:20.579 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: gps base set to 2022-02-06 (week 2196) 2023-11-06T03:00:20.579 INFO:teuthology.orchestra.run.smithi157.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T03:00:20.580 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T03:00:20.580 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: restrict ::: KOD does nothing without LIMITED. 2023-11-06T03:00:20.581 INFO:teuthology.orchestra.run.smithi157.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-06T03:00:20.582 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen and drop on 0 v6wildcard [::]:123 2023-11-06T03:00:20.582 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-06T03:00:20.582 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen normally on 2 lo 127.0.0.1:123 2023-11-06T03:00:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen normally on 3 enp3s0f1 172.21.15.157:123 2023-11-06T03:00:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen normally on 4 lo [::1]:123 2023-11-06T03:00:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea1%5]:123 2023-11-06T03:00:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:20 ntpd[27973]: Listening on routing socket on fd #22 for interface updates 2023-11-06T03:00:20.595 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-06T03:00:20.596 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Command line: ntpd -gq 2023-11-06T03:00:20.596 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: ---------------------------------------------------- 2023-11-06T03:00:20.596 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: ntp-4 is maintained by Network Time Foundation, 2023-11-06T03:00:20.596 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-06T03:00:20.596 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: corporation. Support and training for ntp-4 are 2023-11-06T03:00:20.597 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: available at https://www.nwtime.org/support 2023-11-06T03:00:20.597 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: ---------------------------------------------------- 2023-11-06T03:00:20.597 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: proto: precision = 0.060 usec (-24) 2023-11-06T03:00:20.598 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: basedate set to 2022-02-04 2023-11-06T03:00:20.598 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: gps base set to 2022-02-06 (week 2196) 2023-11-06T03:00:20.598 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T03:00:20.599 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-06T03:00:20.599 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: restrict ::: KOD does nothing without LIMITED. 2023-11-06T03:00:20.599 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-06T03:00:20.600 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen and drop on 0 v6wildcard [::]:123 2023-11-06T03:00:20.600 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-06T03:00:20.601 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen normally on 2 lo 127.0.0.1:123 2023-11-06T03:00:20.601 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2023-11-06T03:00:20.601 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen normally on 4 lo [::1]:123 2023-11-06T03:00:20.601 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2023-11-06T03:00:20.601 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:20 ntpd[27981]: Listening on routing socket on fd #22 for interface updates 2023-11-06T03:00:21.577 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:21.577 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:21.577 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:21.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:21.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:21.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:21.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:21.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:21 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:21.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:21.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:21.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:21.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:21.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:21.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:21 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:23.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:23.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:23.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:23.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:23.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:23.582 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:23 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:23.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:23.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:23.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:23.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:23 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.582 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:25 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.599 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.599 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.599 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.599 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.599 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:25.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:25.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:25.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:25.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:25 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:27.577 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.577 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.578 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:27.579 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:27.580 INFO:teuthology.orchestra.run.smithi157.stdout: 6 Nov 03:00:27 ntpd[27973]: ntpd: time slew -0.001372 s 2023-11-06T03:00:27.580 INFO:teuthology.orchestra.run.smithi157.stdout:ntpd: time slew -0.001372s 2023-11-06T03:00:27.580 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-06T03:00:27.581 INFO:teuthology.orchestra.run.smithi157.stderr: 6 Nov 03:00:27 ntpd[27973]: can't open /var/log/ntpstats/loopstats.20231106: Permission denied 2023-11-06T03:00:27.596 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.597 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-06T03:00:27.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/rawstats.20231106: Permission denied 2023-11-06T03:00:27.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-06T03:00:27.598 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/peerstats.20231106: Permission denied 2023-11-06T03:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout: 6 Nov 03:00:27 ntpd[27981]: ntpd: time slew -0.000304 s 2023-11-06T03:00:27.599 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew -0.000304s 2023-11-06T03:00:27.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-06T03:00:27.600 INFO:teuthology.orchestra.run.smithi195.stderr: 6 Nov 03:00:27 ntpd[27981]: can't open /var/log/ntpstats/loopstats.20231106: Permission denied 2023-11-06T03:00:27.665 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T03:00:27.666 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2023-11-06T03:00:27.666 INFO:teuthology.orchestra.run.smithi157.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.666 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.666 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.667 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T03:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2023-11-06T03:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.673 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.674 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-06T03:00:27.675 INFO:teuthology.run_tasks:Running task install... 2023-11-06T03:00:27.684 DEBUG:teuthology.task.install:project ceph 2023-11-06T03:00:27.685 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}} 2023-11-06T03:00:27.685 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'} 2023-11-06T03:00:27.685 INFO:teuthology.task.install:Using flavor: default 2023-11-06T03:00:27.691 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-06T03:00:27.692 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-06T03:00:27.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-key list | grep Ceph 2023-11-06T03:00:27.693 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2023-11-06T03:00:27.763 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-06T03:00:27.776 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-06T03:00:27.791 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-06T03:00:27.791 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph.com (release key) 2023-11-06T03:00:27.792 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-06T03:00:27.793 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-06T03:00:27.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T03:00:27.804 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-06T03:00:27.805 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2023-11-06T03:00:27.805 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-06T03:00:27.805 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-06T03:00:27.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T03:00:27.935 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default/ 2023-11-06T03:00:27.936 INFO:teuthology.task.install.deb:Package version is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:00:27.944 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default/ 2023-11-06T03:00:27.944 INFO:teuthology.task.install.deb:Package version is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:00:28.052 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:00:28.052 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-06T03:00:28.066 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2023-11-06T03:00:28.105 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:00:28.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-06T03:00:28.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2023-11-06T03:00:28.236 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T03:00:28.280 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T03:00:28.280 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-06T03:00:28.282 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy InRelease 2023-11-06T03:00:28.291 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T03:00:28.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-06T03:00:28.308 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T03:00:28.326 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-06T03:00:28.337 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy InRelease 2023-11-06T03:00:28.347 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-11-06T03:00:28.352 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-06T03:00:28.357 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-06T03:00:28.373 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2023-11-06T03:00:28.384 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-06T03:00:28.405 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-11-06T03:00:28.433 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-06T03:00:28.433 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,418 B] 2023-11-06T03:00:28.514 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-06T03:00:28.641 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 28.5 kB in 0s (83.1 kB/s) 2023-11-06T03:00:28.649 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 28.5 kB in 0s (69.8 kB/s) 2023-11-06T03:00:29.478 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T03:00:29.493 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-1181-gf7e9f9af-1jammy cephadm=18.2.0-1181-gf7e9f9af-1jammy ceph-mds=18.2.0-1181-gf7e9f9af-1jammy ceph-mgr=18.2.0-1181-gf7e9f9af-1jammy ceph-common=18.2.0-1181-gf7e9f9af-1jammy ceph-fuse=18.2.0-1181-gf7e9f9af-1jammy ceph-test=18.2.0-1181-gf7e9f9af-1jammy ceph-volume=18.2.0-1181-gf7e9f9af-1jammy radosgw=18.2.0-1181-gf7e9f9af-1jammy python3-rados=18.2.0-1181-gf7e9f9af-1jammy python3-rgw=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy python3-rbd=18.2.0-1181-gf7e9f9af-1jammy libcephfs2=18.2.0-1181-gf7e9f9af-1jammy libcephfs-dev=18.2.0-1181-gf7e9f9af-1jammy librados2=18.2.0-1181-gf7e9f9af-1jammy librbd1=18.2.0-1181-gf7e9f9af-1jammy rbd-fuse=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy cephfs-shell=18.2.0-1181-gf7e9f9af-1jammy cephfs-top=18.2.0-1181-gf7e9f9af-1jammy cephfs-mirror=18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:00:29.541 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T03:00:29.552 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T03:00:29.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.0-1181-gf7e9f9af-1jammy cephadm=18.2.0-1181-gf7e9f9af-1jammy ceph-mds=18.2.0-1181-gf7e9f9af-1jammy ceph-mgr=18.2.0-1181-gf7e9f9af-1jammy ceph-common=18.2.0-1181-gf7e9f9af-1jammy ceph-fuse=18.2.0-1181-gf7e9f9af-1jammy ceph-test=18.2.0-1181-gf7e9f9af-1jammy ceph-volume=18.2.0-1181-gf7e9f9af-1jammy radosgw=18.2.0-1181-gf7e9f9af-1jammy python3-rados=18.2.0-1181-gf7e9f9af-1jammy python3-rgw=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy python3-rbd=18.2.0-1181-gf7e9f9af-1jammy libcephfs2=18.2.0-1181-gf7e9f9af-1jammy libcephfs-dev=18.2.0-1181-gf7e9f9af-1jammy librados2=18.2.0-1181-gf7e9f9af-1jammy librbd1=18.2.0-1181-gf7e9f9af-1jammy rbd-fuse=18.2.0-1181-gf7e9f9af-1jammy python3-cephfs=18.2.0-1181-gf7e9f9af-1jammy cephfs-shell=18.2.0-1181-gf7e9f9af-1jammy cephfs-top=18.2.0-1181-gf7e9f9af-1jammy cephfs-mirror=18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:00:29.604 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T03:00:29.742 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2023-11-06T03:00:29.743 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2023-11-06T03:00:29.769 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2023-11-06T03:00:29.770 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2023-11-06T03:00:29.907 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T03:00:29.907 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T03:00:29.907 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T03:00:29.908 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T03:00:29.908 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T03:00:29.909 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T03:00:29.909 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T03:00:29.909 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2023-11-06T03:00:29.910 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-06T03:00:29.910 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-06T03:00:29.910 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T03:00:29.910 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-06T03:00:29.910 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-06T03:00:29.911 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-06T03:00:29.911 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-06T03:00:29.911 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-06T03:00:29.911 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-06T03:00:29.911 INFO:teuthology.orchestra.run.smithi195.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-06T03:00:29.912 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-06T03:00:29.912 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-06T03:00:29.912 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-06T03:00:29.912 INFO:teuthology.orchestra.run.smithi195.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-06T03:00:29.912 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-06T03:00:29.913 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-06T03:00:29.913 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-06T03:00:29.913 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-06T03:00:29.913 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-06T03:00:29.914 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-06T03:00:29.914 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-06T03:00:29.914 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-06T03:00:29.914 INFO:teuthology.orchestra.run.smithi195.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-06T03:00:29.915 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2023-11-06T03:00:29.915 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-06T03:00:29.915 INFO:teuthology.orchestra.run.smithi195.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-06T03:00:29.915 INFO:teuthology.orchestra.run.smithi195.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-06T03:00:29.915 INFO:teuthology.orchestra.run.smithi195.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-06T03:00:29.916 INFO:teuthology.orchestra.run.smithi195.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-06T03:00:29.916 INFO:teuthology.orchestra.run.smithi195.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-06T03:00:29.916 INFO:teuthology.orchestra.run.smithi195.stdout: python3-watchdog 2023-11-06T03:00:29.916 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2023-11-06T03:00:29.916 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2023-11-06T03:00:29.917 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T03:00:29.917 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T03:00:29.918 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T03:00:29.918 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T03:00:29.919 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T03:00:29.919 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T03:00:29.919 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T03:00:29.919 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2023-11-06T03:00:29.920 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-06T03:00:29.920 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-06T03:00:29.920 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T03:00:29.920 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-06T03:00:29.921 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-06T03:00:29.921 INFO:teuthology.orchestra.run.smithi157.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-06T03:00:29.921 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-06T03:00:29.922 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-06T03:00:29.922 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-06T03:00:29.922 INFO:teuthology.orchestra.run.smithi157.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-06T03:00:29.922 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-06T03:00:29.923 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-06T03:00:29.924 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-06T03:00:29.924 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-06T03:00:29.924 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-06T03:00:29.924 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-06T03:00:29.924 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-06T03:00:29.925 INFO:teuthology.orchestra.run.smithi157.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-06T03:00:29.925 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2023-11-06T03:00:29.925 INFO:teuthology.orchestra.run.smithi157.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-06T03:00:29.925 INFO:teuthology.orchestra.run.smithi157.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-06T03:00:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-06T03:00:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-06T03:00:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-06T03:00:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-06T03:00:29.926 INFO:teuthology.orchestra.run.smithi157.stdout: python3-watchdog 2023-11-06T03:00:29.927 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2023-11-06T03:00:29.927 INFO:teuthology.orchestra.run.smithi157.stdout: btrfs-tools 2023-11-06T03:00:30.009 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2023-11-06T03:00:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-06T03:00:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-06T03:00:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-06T03:00:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-06T03:00:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T03:00:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-06T03:00:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-11-06T03:00:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-11-06T03:00:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-06T03:00:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-06T03:00:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-06T03:00:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-06T03:00:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-06T03:00:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-06T03:00:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-06T03:00:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-06T03:00:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-06T03:00:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-06T03:00:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-06T03:00:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-06T03:00:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-06T03:00:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-06T03:00:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-06T03:00:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-06T03:00:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-06T03:00:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-06T03:00:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: socat xmlstarlet 2023-11-06T03:00:30.021 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2023-11-06T03:00:30.022 INFO:teuthology.orchestra.run.smithi157.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-06T03:00:30.022 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-06T03:00:30.022 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-06T03:00:30.022 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-06T03:00:30.022 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-11-06T03:00:30.023 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-06T03:00:30.023 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-11-06T03:00:30.024 INFO:teuthology.orchestra.run.smithi157.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-11-06T03:00:30.024 INFO:teuthology.orchestra.run.smithi157.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-06T03:00:30.024 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-06T03:00:30.025 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-06T03:00:30.025 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-06T03:00:30.025 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-06T03:00:30.025 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-06T03:00:30.025 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-06T03:00:30.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-06T03:00:30.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-06T03:00:30.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-06T03:00:30.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-06T03:00:30.026 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-06T03:00:30.027 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-06T03:00:30.027 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-06T03:00:30.027 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-06T03:00:30.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-06T03:00:30.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-06T03:00:30.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-06T03:00:30.028 INFO:teuthology.orchestra.run.smithi157.stdout: socat xmlstarlet 2023-11-06T03:00:30.146 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 115 newly installed, 0 to remove and 258 not upgraded. 2023-11-06T03:00:30.146 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 181 MB of archives. 2023-11-06T03:00:30.146 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 812 MB of additional disk space will be used. 2023-11-06T03:00:30.146 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-06T03:00:30.156 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 115 newly installed, 0 to remove and 258 not upgraded. 2023-11-06T03:00:30.156 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 181 MB of archives. 2023-11-06T03:00:30.157 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 812 MB of additional disk space will be used. 2023-11-06T03:00:30.157 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-06T03:00:30.172 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-06T03:00:30.177 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-06T03:00:30.198 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-06T03:00:30.210 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-06T03:00:30.259 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-11-06T03:00:30.266 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-11-06T03:00:30.270 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-06T03:00:30.278 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-06T03:00:30.283 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,417 kB] 2023-11-06T03:00:30.284 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,417 kB] 2023-11-06T03:00:30.291 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-06T03:00:30.294 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-06T03:00:30.299 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-06T03:00:30.301 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-06T03:00:30.307 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-06T03:00:30.313 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-06T03:00:30.368 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-06T03:00:30.374 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-06T03:00:30.384 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-06T03:00:30.388 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-06T03:00:30.390 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-06T03:00:30.396 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-06T03:00:30.397 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-06T03:00:30.408 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-06T03:00:30.409 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-06T03:00:30.409 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-06T03:00:30.409 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-06T03:00:30.410 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-06T03:00:30.414 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-06T03:00:30.415 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-06T03:00:30.416 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-06T03:00:30.426 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-06T03:00:30.427 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-06T03:00:30.428 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-06T03:00:30.429 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-06T03:00:30.429 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-06T03:00:30.432 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-06T03:00:30.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-06T03:00:30.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-06T03:00:30.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-06T03:00:30.434 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-06T03:00:30.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-06T03:00:30.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-06T03:00:30.439 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-06T03:00:30.440 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-06T03:00:30.443 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-06T03:00:30.444 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-06T03:00:30.444 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-06T03:00:30.445 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-06T03:00:30.456 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-06T03:00:30.457 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-06T03:00:30.457 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-06T03:00:30.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-06T03:00:30.465 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-06T03:00:30.469 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-06T03:00:30.470 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-06T03:00:30.471 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-06T03:00:30.471 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-06T03:00:30.473 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-06T03:00:30.476 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-06T03:00:30.477 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-06T03:00:30.484 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-06T03:00:30.485 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-06T03:00:30.485 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-06T03:00:30.486 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-06T03:00:30.488 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-06T03:00:30.489 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-06T03:00:30.491 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-06T03:00:30.492 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-06T03:00:30.496 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-06T03:00:30.499 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-06T03:00:30.501 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-11-06T03:00:30.502 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-06T03:00:30.510 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-06T03:00:30.518 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-06T03:00:30.519 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-06T03:00:30.519 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-06T03:00:30.520 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-06T03:00:30.522 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-06T03:00:30.523 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-06T03:00:30.525 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-06T03:00:30.526 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-06T03:00:30.530 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-06T03:00:30.545 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-06T03:00:30.546 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-11-06T03:00:30.549 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-06T03:00:30.550 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-06T03:00:30.588 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-06T03:00:30.589 INFO:teuthology.orchestra.run.smithi157.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-06T03:00:30.594 INFO:teuthology.orchestra.run.smithi157.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-06T03:00:30.616 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-06T03:00:30.618 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-06T03:00:30.620 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-06T03:00:30.621 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-06T03:00:30.621 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-06T03:00:30.623 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-06T03:00:30.624 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-06T03:00:30.625 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-06T03:00:30.629 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-06T03:00:30.665 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-06T03:00:30.666 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-06T03:00:30.674 INFO:teuthology.orchestra.run.smithi157.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-06T03:00:30.696 INFO:teuthology.orchestra.run.smithi157.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-06T03:00:30.697 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-06T03:00:30.698 INFO:teuthology.orchestra.run.smithi157.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-06T03:00:30.698 INFO:teuthology.orchestra.run.smithi157.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-06T03:00:30.699 INFO:teuthology.orchestra.run.smithi157.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-06T03:00:30.700 INFO:teuthology.orchestra.run.smithi157.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-06T03:00:30.701 INFO:teuthology.orchestra.run.smithi157.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-06T03:00:30.701 INFO:teuthology.orchestra.run.smithi157.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-06T03:00:30.702 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-06T03:00:30.702 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-06T03:00:30.704 INFO:teuthology.orchestra.run.smithi157.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-06T03:00:30.723 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-06T03:00:30.723 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-06T03:00:30.724 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-06T03:00:30.724 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-06T03:00:30.725 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-06T03:00:30.726 INFO:teuthology.orchestra.run.smithi157.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-06T03:00:30.729 INFO:teuthology.orchestra.run.smithi157.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-06T03:00:30.740 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-06T03:00:30.741 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-06T03:00:30.741 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-06T03:00:30.748 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-06T03:00:30.758 INFO:teuthology.orchestra.run.smithi157.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-06T03:00:30.761 INFO:teuthology.orchestra.run.smithi157.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-06T03:00:30.761 INFO:teuthology.orchestra.run.smithi157.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-06T03:00:30.782 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-06T03:00:30.786 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-06T03:00:30.786 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-06T03:00:30.788 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-06T03:00:30.791 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-06T03:00:30.791 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-06T03:00:30.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-06T03:00:30.797 INFO:teuthology.orchestra.run.smithi157.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-06T03:00:30.798 INFO:teuthology.orchestra.run.smithi157.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-06T03:00:30.799 INFO:teuthology.orchestra.run.smithi157.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-06T03:00:30.799 INFO:teuthology.orchestra.run.smithi157.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-06T03:00:30.803 INFO:teuthology.orchestra.run.smithi157.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-06T03:00:30.803 INFO:teuthology.orchestra.run.smithi157.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-06T03:00:30.804 INFO:teuthology.orchestra.run.smithi157.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-06T03:00:30.809 INFO:teuthology.orchestra.run.smithi157.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-06T03:00:30.810 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-06T03:00:30.813 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-06T03:00:30.814 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-06T03:00:30.816 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-06T03:00:30.831 INFO:teuthology.orchestra.run.smithi157.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-06T03:00:30.833 INFO:teuthology.orchestra.run.smithi157.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-06T03:00:30.834 INFO:teuthology.orchestra.run.smithi157.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-06T03:00:30.840 INFO:teuthology.orchestra.run.smithi157.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-06T03:00:30.843 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-06T03:00:30.843 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-06T03:00:30.844 INFO:teuthology.orchestra.run.smithi157.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-06T03:00:30.844 INFO:teuthology.orchestra.run.smithi157.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-06T03:00:30.845 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-06T03:00:30.870 INFO:teuthology.orchestra.run.smithi157.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-06T03:00:30.874 INFO:teuthology.orchestra.run.smithi157.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-06T03:00:30.875 INFO:teuthology.orchestra.run.smithi157.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-06T03:00:30.877 INFO:teuthology.orchestra.run.smithi157.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-06T03:00:30.891 INFO:teuthology.orchestra.run.smithi157.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-06T03:00:30.892 INFO:teuthology.orchestra.run.smithi157.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-06T03:00:30.893 INFO:teuthology.orchestra.run.smithi157.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-06T03:00:31.475 INFO:teuthology.orchestra.run.smithi157.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,104 kB] 2023-11-06T03:00:31.480 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.0-1181-gf7e9f9af-1jammy [3,104 kB] 2023-11-06T03:00:31.841 INFO:teuthology.orchestra.run.smithi157.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.0-1181-gf7e9f9af-1jammy [669 kB] 2023-11-06T03:00:31.856 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.0-1181-gf7e9f9af-1jammy [669 kB] 2023-11-06T03:00:31.890 INFO:teuthology.orchestra.run.smithi157.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.0-1181-gf7e9f9af-1jammy [356 kB] 2023-11-06T03:00:31.907 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.0-1181-gf7e9f9af-1jammy [356 kB] 2023-11-06T03:00:31.925 INFO:teuthology.orchestra.run.smithi157.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.0-1181-gf7e9f9af-1jammy [32.8 kB] 2023-11-06T03:00:31.926 INFO:teuthology.orchestra.run.smithi157.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.0-1181-gf7e9f9af-1jammy [180 kB] 2023-11-06T03:00:31.938 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.0-1181-gf7e9f9af-1jammy [32.8 kB] 2023-11-06T03:00:31.939 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.0-1181-gf7e9f9af-1jammy [180 kB] 2023-11-06T03:00:31.949 INFO:teuthology.orchestra.run.smithi157.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.0-1181-gf7e9f9af-1jammy [61.5 kB] 2023-11-06T03:00:31.953 INFO:teuthology.orchestra.run.smithi157.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.0-1181-gf7e9f9af-1jammy [330 kB] 2023-11-06T03:00:31.970 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.0-1181-gf7e9f9af-1jammy [61.5 kB] 2023-11-06T03:00:31.972 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.0-1181-gf7e9f9af-1jammy [330 kB] 2023-11-06T03:00:31.984 INFO:teuthology.orchestra.run.smithi157.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.0-1181-gf7e9f9af-1jammy [5,886 kB] 2023-11-06T03:00:32.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.0-1181-gf7e9f9af-1jammy [5,886 kB] 2023-11-06T03:00:32.379 INFO:teuthology.orchestra.run.smithi157.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.0-1181-gf7e9f9af-1jammy [112 kB] 2023-11-06T03:00:32.382 INFO:teuthology.orchestra.run.smithi157.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.0-1181-gf7e9f9af-1jammy [442 kB] 2023-11-06T03:00:32.405 INFO:teuthology.orchestra.run.smithi157.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.0-1181-gf7e9f9af-1jammy [22.0 MB] 2023-11-06T03:00:32.406 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.0-1181-gf7e9f9af-1jammy [112 kB] 2023-11-06T03:00:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.0-1181-gf7e9f9af-1jammy [442 kB] 2023-11-06T03:00:32.424 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.0-1181-gf7e9f9af-1jammy [22.0 MB] 2023-11-06T03:00:33.908 INFO:teuthology.orchestra.run.smithi157.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.0-1181-gf7e9f9af-1jammy [6,838 kB] 2023-11-06T03:00:34.054 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.0-1181-gf7e9f9af-1jammy [6,838 kB] 2023-11-06T03:00:34.265 INFO:teuthology.orchestra.run.smithi157.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.0-1181-gf7e9f9af-1jammy [237 kB] 2023-11-06T03:00:34.280 INFO:teuthology.orchestra.run.smithi157.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [129 kB] 2023-11-06T03:00:34.317 INFO:teuthology.orchestra.run.smithi157.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.0-1181-gf7e9f9af-1jammy [1,483 kB] 2023-11-06T03:00:34.370 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.0-1181-gf7e9f9af-1jammy [237 kB] 2023-11-06T03:00:34.378 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [129 kB] 2023-11-06T03:00:34.383 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.0-1181-gf7e9f9af-1jammy [1,483 kB] 2023-11-06T03:00:34.409 INFO:teuthology.orchestra.run.smithi157.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.0-1181-gf7e9f9af-1jammy [9,877 kB] 2023-11-06T03:00:34.478 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.0-1181-gf7e9f9af-1jammy [9,877 kB] 2023-11-06T03:00:35.114 INFO:teuthology.orchestra.run.smithi157.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.0-1181-gf7e9f9af-1jammy [31.5 MB] 2023-11-06T03:00:35.197 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.0-1181-gf7e9f9af-1jammy [31.5 MB] 2023-11-06T03:00:37.209 INFO:teuthology.orchestra.run.smithi157.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [14.1 kB] 2023-11-06T03:00:37.210 INFO:teuthology.orchestra.run.smithi157.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [824 kB] 2023-11-06T03:00:37.226 INFO:teuthology.orchestra.run.smithi157.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.0-1181-gf7e9f9af-1jammy [2,170 kB] 2023-11-06T03:00:37.485 INFO:teuthology.orchestra.run.smithi157.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.0-1181-gf7e9f9af-1jammy [235 kB] 2023-11-06T03:00:37.491 INFO:teuthology.orchestra.run.smithi157.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.0-1181-gf7e9f9af-1jammy [138 kB] 2023-11-06T03:00:37.508 INFO:teuthology.orchestra.run.smithi157.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.0-1181-gf7e9f9af-1jammy [1,898 kB] 2023-11-06T03:00:37.623 INFO:teuthology.orchestra.run.smithi157.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.0-1181-gf7e9f9af-1jammy [8,625 kB] 2023-11-06T03:00:38.224 INFO:teuthology.orchestra.run.smithi157.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.0-1181-gf7e9f9af-1jammy [14.2 kB] 2023-11-06T03:00:38.225 INFO:teuthology.orchestra.run.smithi157.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.0-1181-gf7e9f9af-1jammy [46.5 MB] 2023-11-06T03:00:38.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.0-1181-gf7e9f9af-1jammy [14.1 kB] 2023-11-06T03:00:38.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [824 kB] 2023-11-06T03:00:38.306 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.0-1181-gf7e9f9af-1jammy [2,170 kB] 2023-11-06T03:00:38.481 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.0-1181-gf7e9f9af-1jammy [235 kB] 2023-11-06T03:00:38.512 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.0-1181-gf7e9f9af-1jammy [138 kB] 2023-11-06T03:00:38.516 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.0-1181-gf7e9f9af-1jammy [1,898 kB] 2023-11-06T03:00:38.691 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.0-1181-gf7e9f9af-1jammy [8,625 kB] 2023-11-06T03:00:39.396 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.0-1181-gf7e9f9af-1jammy [14.2 kB] 2023-11-06T03:00:39.397 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.0-1181-gf7e9f9af-1jammy [46.5 MB] 2023-11-06T03:00:41.089 INFO:teuthology.orchestra.run.smithi157.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.0-1181-gf7e9f9af-1jammy [120 kB] 2023-11-06T03:00:41.091 INFO:teuthology.orchestra.run.smithi157.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.0-1181-gf7e9f9af-1jammy [228 kB] 2023-11-06T03:00:41.159 INFO:teuthology.orchestra.run.smithi157.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.0-1181-gf7e9f9af-1jammy [34.0 kB] 2023-11-06T03:00:41.160 INFO:teuthology.orchestra.run.smithi157.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.0-1181-gf7e9f9af-1jammy [23.3 kB] 2023-11-06T03:00:41.161 INFO:teuthology.orchestra.run.smithi157.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.0-1181-gf7e9f9af-1jammy [38.3 kB] 2023-11-06T03:00:41.162 INFO:teuthology.orchestra.run.smithi157.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.0-1181-gf7e9f9af-1jammy [11.6 MB] 2023-11-06T03:00:41.685 INFO:teuthology.orchestra.run.smithi157.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [92.9 kB] 2023-11-06T03:00:42.243 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 181 MB in 12s (15.5 MB/s) 2023-11-06T03:00:42.354 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-06T03:00:42.442 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165985 files and directories currently installed.) 2023-11-06T03:00:42.447 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-06T03:00:42.486 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-06T03:00:42.756 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-06T03:00:42.782 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-06T03:00:42.822 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T03:00:43.133 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cmd2. 2023-11-06T03:00:43.159 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-06T03:00:43.200 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-06T03:00:43.406 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.0-1181-gf7e9f9af-1jammy [120 kB] 2023-11-06T03:00:43.410 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.0-1181-gf7e9f9af-1jammy [228 kB] 2023-11-06T03:00:43.413 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.0-1181-gf7e9f9af-1jammy [34.0 kB] 2023-11-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.0-1181-gf7e9f9af-1jammy [23.3 kB] 2023-11-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.0-1181-gf7e9f9af-1jammy [38.3 kB] 2023-11-06T03:00:43.477 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.0-1181-gf7e9f9af-1jammy [11.6 MB] 2023-11-06T03:00:43.716 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfmt8:amd64. 2023-11-06T03:00:43.743 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-11-06T03:00:43.870 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T03:00:44.190 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-06T03:00:44.215 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-06T03:00:44.231 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/reef/f7e9f9af51acf98e0ccc3037487018e4ae40caa5/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.0-1181-gf7e9f9af-1jammy [92.9 kB] 2023-11-06T03:00:44.256 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T03:00:44.576 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-06T03:00:44.603 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-06T03:00:44.643 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T03:00:44.776 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 181 MB in 14s (12.7 MB/s) 2023-11-06T03:00:44.914 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-06T03:00:44.962 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-06T03:00:44.989 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-06T03:00:45.013 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165985 files and directories currently installed.) 2023-11-06T03:00:45.018 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-06T03:00:45.029 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T03:00:45.062 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-06T03:00:45.358 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-06T03:00:45.383 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-06T03:00:45.408 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-06T03:00:45.423 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T03:00:45.434 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:45.474 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:45.751 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cmd2. 2023-11-06T03:00:45.778 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-06T03:00:45.817 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-06T03:00:45.844 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-06T03:00:45.859 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:45.894 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:46.128 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfmt8:amd64. 2023-11-06T03:00:46.155 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-11-06T03:00:46.189 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-06T03:00:46.215 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:46.237 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T03:00:46.255 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:46.557 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-06T03:00:46.584 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-06T03:00:46.600 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-06T03:00:46.624 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T03:00:46.626 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-06T03:00:46.667 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T03:00:46.969 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-06T03:00:46.986 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librados2. 2023-11-06T03:00:46.995 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-06T03:00:47.012 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../011-librados2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:47.035 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T03:00:47.053 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:47.396 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-06T03:00:47.424 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-06T03:00:47.463 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T03:00:47.464 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librbd1. 2023-11-06T03:00:47.479 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../012-librbd1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:47.514 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:47.933 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs2. 2023-11-06T03:00:47.960 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../013-libcephfs2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:48.000 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:48.167 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-06T03:00:48.194 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:48.241 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:48.277 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rados. 2023-11-06T03:00:48.293 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../014-python3-rados_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:48.336 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:48.662 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-06T03:00:48.680 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-06T03:00:48.688 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:48.707 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:00:48.728 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:48.747 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:49.040 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-06T03:00:49.067 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-06T03:00:49.115 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:00:49.158 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cephfs. 2023-11-06T03:00:49.174 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../016-python3-cephfs_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:49.208 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:49.468 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-06T03:00:49.477 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-06T03:00:49.494 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-06T03:00:49.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:00:49.534 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T03:00:49.543 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:49.854 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librados2. 2023-11-06T03:00:49.864 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-prettytable. 2023-11-06T03:00:49.881 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-librados2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:49.890 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-11-06T03:00:49.920 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:49.930 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-06T03:00:50.191 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rbd. 2023-11-06T03:00:50.207 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../019-python3-rbd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:50.249 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:50.356 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librbd1. 2023-11-06T03:00:50.383 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-librbd1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:50.423 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:50.577 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-06T03:00:50.604 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-06T03:00:50.635 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T03:00:50.851 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2023-11-06T03:00:50.878 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-libcephfs2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:50.917 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:50.938 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librgw2. 2023-11-06T03:00:50.966 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../021-librgw2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:51.013 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:51.203 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2023-11-06T03:00:51.219 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-rados_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:51.261 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:51.524 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rgw. 2023-11-06T03:00:51.542 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../022-python3-rgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:51.582 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:51.606 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-06T03:00:51.633 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:00:51.672 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:51.916 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2023-11-06T03:00:51.919 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-06T03:00:51.932 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-cephfs_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:51.935 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-06T03:00:51.966 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:51.977 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T03:00:52.236 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-06T03:00:52.255 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libradosstriper1. 2023-11-06T03:00:52.263 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:00:52.281 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../024-libradosstriper1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:52.302 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:52.313 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:52.639 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2023-11-06T03:00:52.666 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-11-06T03:00:52.705 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-06T03:00:52.733 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-common. 2023-11-06T03:00:52.759 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../025-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:52.800 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:52.966 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2023-11-06T03:00:52.993 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-python3-rbd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:53.033 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:53.386 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-06T03:00:53.413 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-06T03:00:53.452 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T03:00:53.747 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2023-11-06T03:00:53.774 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-librgw2_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:53.813 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:54.047 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-base. 2023-11-06T03:00:54.073 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../026-ceph-base_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:54.123 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:54.341 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2023-11-06T03:00:54.368 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-python3-rgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:54.409 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:54.760 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-06T03:00:54.787 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-06T03:00:54.819 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-06T03:00:54.827 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T03:00:54.844 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-06T03:00:54.884 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-06T03:00:55.105 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2023-11-06T03:00:55.131 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-libradosstriper1_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:55.154 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cheroot. 2023-11-06T03:00:55.171 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:55.180 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-06T03:00:55.212 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T03:00:55.515 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-06T03:00:55.531 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-06T03:00:55.565 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-06T03:00:55.566 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2023-11-06T03:00:55.593 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-ceph-common_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:55.633 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:55.851 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-06T03:00:55.878 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-06T03:00:55.918 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-06T03:00:56.221 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-06T03:00:56.247 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-06T03:00:56.287 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-06T03:00:56.532 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempora. 2023-11-06T03:00:56.555 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-11-06T03:00:56.590 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-06T03:00:56.861 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-portend. 2023-11-06T03:00:56.886 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-11-06T03:00:56.889 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2023-11-06T03:00:56.915 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-ceph-base_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:00:56.926 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-06T03:00:56.965 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:00:57.246 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-06T03:00:57.272 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-06T03:00:57.312 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-06T03:00:57.624 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-06T03:00:57.650 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-06T03:00:57.690 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-06T03:00:57.701 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-06T03:00:57.728 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-06T03:00:57.768 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-06T03:00:58.002 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-natsort. 2023-11-06T03:00:58.017 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-11-06T03:00:58.052 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-06T03:00:58.087 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cheroot. 2023-11-06T03:00:58.114 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-06T03:00:58.314 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T03:00:58.372 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-logutils. 2023-11-06T03:00:58.398 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-11-06T03:00:58.447 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-06T03:00:58.649 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-06T03:00:58.675 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-06T03:00:58.715 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-06T03:00:58.750 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-06T03:00:58.776 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-06T03:00:58.816 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-06T03:00:58.968 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-06T03:00:58.994 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-06T03:00:59.035 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-06T03:00:59.102 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-mako. 2023-11-06T03:00:59.118 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-06T03:00:59.338 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-06T03:00:59.364 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-06T03:00:59.377 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T03:00:59.404 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-06T03:00:59.664 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-06T03:00:59.679 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-06T03:00:59.707 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempora. 2023-11-06T03:00:59.714 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-06T03:00:59.733 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-11-06T03:00:59.774 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-06T03:01:00.017 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-06T03:01:00.035 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-portend. 2023-11-06T03:01:00.043 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-06T03:01:00.050 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-11-06T03:01:00.083 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-06T03:01:00.084 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-06T03:01:00.321 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-06T03:01:00.347 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-06T03:01:00.369 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webob. 2023-11-06T03:01:00.387 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-06T03:01:00.395 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-06T03:01:00.502 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-06T03:01:00.707 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-06T03:01:00.734 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-06T03:01:00.773 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-06T03:01:00.856 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-waitress. 2023-11-06T03:01:00.882 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-06T03:01:00.926 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T03:01:01.127 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2023-11-06T03:01:01.154 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-11-06T03:01:01.194 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-06T03:01:01.217 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempita. 2023-11-06T03:01:01.232 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-06T03:01:01.267 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T03:01:01.503 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-paste. 2023-11-06T03:01:01.505 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2023-11-06T03:01:01.529 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-06T03:01:01.532 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-11-06T03:01:01.561 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T03:01:01.580 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-06T03:01:01.866 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-06T03:01:01.873 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-06T03:01:01.888 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-06T03:01:01.893 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-06T03:01:01.931 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T03:01:01.933 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-06T03:01:02.202 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2023-11-06T03:01:02.229 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-06T03:01:02.234 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-06T03:01:02.260 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-06T03:01:02.269 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T03:01:02.300 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-06T03:01:02.603 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webtest. 2023-11-06T03:01:02.614 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-06T03:01:02.630 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-11-06T03:01:02.640 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-06T03:01:02.670 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-06T03:01:02.680 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-06T03:01:02.948 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pecan. 2023-11-06T03:01:02.974 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-06T03:01:02.983 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-06T03:01:03.010 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-06T03:01:03.014 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T03:01:03.050 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-06T03:01:03.319 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2023-11-06T03:01:03.344 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-certifi. 2023-11-06T03:01:03.345 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-06T03:01:03.371 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-11-06T03:01:03.386 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-06T03:01:03.411 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-06T03:01:04.465 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2023-11-06T03:01:04.492 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-06T03:01:04.535 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T03:01:04.649 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-idna. 2023-11-06T03:01:04.676 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-11-06T03:01:04.731 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-06T03:01:04.826 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2023-11-06T03:01:04.852 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-06T03:01:04.892 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T03:01:05.034 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-urllib3. 2023-11-06T03:01:05.060 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-11-06T03:01:05.100 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T03:01:05.195 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2023-11-06T03:01:05.221 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-06T03:01:05.262 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T03:01:05.445 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests. 2023-11-06T03:01:05.472 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-06T03:01:05.511 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T03:01:05.582 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-06T03:01:05.608 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-06T03:01:05.648 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T03:01:05.815 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-06T03:01:05.841 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-06T03:01:05.881 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T03:01:05.951 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-06T03:01:05.977 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-06T03:01:06.017 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-06T03:01:06.167 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-06T03:01:06.183 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:06.226 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:06.345 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2023-11-06T03:01:06.372 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-11-06T03:01:06.412 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-06T03:01:06.589 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-06T03:01:06.615 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:06.655 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:06.698 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2023-11-06T03:01:06.713 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-06T03:01:06.748 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T03:01:06.983 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-openssl. 2023-11-06T03:01:07.010 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-11-06T03:01:07.050 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-06T03:01:07.078 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-certifi. 2023-11-06T03:01:07.104 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-11-06T03:01:07.136 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-06T03:01:07.386 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr. 2023-11-06T03:01:07.413 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../058-ceph-mgr_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:07.452 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:07.497 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-idna. 2023-11-06T03:01:07.524 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-11-06T03:01:07.564 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-06T03:01:07.764 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mon. 2023-11-06T03:01:07.781 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../059-ceph-mon_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:07.822 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:07.875 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-urllib3. 2023-11-06T03:01:07.902 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-11-06T03:01:07.942 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T03:01:08.270 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests. 2023-11-06T03:01:08.296 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-06T03:01:08.339 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T03:01:08.726 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-osd. 2023-11-06T03:01:08.745 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../060-ceph-osd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:08.784 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:08.789 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-06T03:01:08.816 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-06T03:01:08.856 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T03:01:09.150 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-06T03:01:09.166 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:09.209 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:09.555 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-06T03:01:09.582 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:09.622 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:09.933 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2023-11-06T03:01:09.960 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-11-06T03:01:09.999 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-06T03:01:10.344 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2023-11-06T03:01:10.371 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-ceph-mgr_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:10.411 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:10.612 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph. 2023-11-06T03:01:10.627 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../061-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:10.687 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:10.789 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2023-11-06T03:01:10.817 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-ceph-mon_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:10.847 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:11.007 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-fuse. 2023-11-06T03:01:11.033 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../062-ceph-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:11.073 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:11.410 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mds. 2023-11-06T03:01:11.437 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../063-ceph-mds_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:11.476 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:11.776 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2023-11-06T03:01:11.803 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-ceph-osd_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:11.838 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephadm. 2023-11-06T03:01:11.843 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:11.865 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../064-cephadm_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:11.904 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:12.224 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-06T03:01:12.253 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-06T03:01:12.291 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-06T03:01:12.602 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-06T03:01:12.617 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T03:01:12.661 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:13.641 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-babel. 2023-11-06T03:01:13.667 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T03:01:13.707 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:13.796 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2023-11-06T03:01:13.822 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-ceph_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:13.862 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:14.018 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jinja2. 2023-11-06T03:01:14.045 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-11-06T03:01:14.151 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-06T03:01:14.190 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2023-11-06T03:01:14.217 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-ceph-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:14.256 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:14.521 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-06T03:01:14.548 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:14.585 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2023-11-06T03:01:14.588 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:14.612 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-ceph-mds_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:14.643 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:14.927 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-06T03:01:14.954 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-06T03:01:14.993 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-06T03:01:15.029 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2023-11-06T03:01:15.056 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-cephadm_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:15.096 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:15.296 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pluggy. 2023-11-06T03:01:15.323 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-06T03:01:15.362 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-06T03:01:15.424 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-06T03:01:15.451 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-06T03:01:15.490 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-06T03:01:15.624 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-py. 2023-11-06T03:01:15.641 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-11-06T03:01:15.682 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-06T03:01:15.802 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-06T03:01:15.861 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T03:01:15.861 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:15.978 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-06T03:01:16.004 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-06T03:01:16.044 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-06T03:01:16.347 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-toml. 2023-11-06T03:01:16.374 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-11-06T03:01:16.413 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-06T03:01:16.716 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pytest. 2023-11-06T03:01:16.743 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-06T03:01:16.791 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T03:01:16.857 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-babel. 2023-11-06T03:01:16.884 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-06T03:01:16.915 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:17.128 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-06T03:01:17.155 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-11-06T03:01:17.185 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2023-11-06T03:01:17.195 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-06T03:01:17.212 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-11-06T03:01:17.251 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-06T03:01:17.498 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-routes. 2023-11-06T03:01:17.524 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-06T03:01:17.564 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T03:01:17.571 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-06T03:01:17.598 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:17.637 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:17.901 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-06T03:01:17.928 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:17.941 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-06T03:01:17.967 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:17.968 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-06T03:01:18.008 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-06T03:01:18.261 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2023-11-06T03:01:18.288 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-06T03:01:18.319 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-06T03:01:18.396 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-06T03:01:18.420 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-06T03:01:18.462 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T03:01:18.630 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2023-11-06T03:01:18.657 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-11-06T03:01:18.696 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-06T03:01:18.932 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-joblib. 2023-11-06T03:01:18.960 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-06T03:01:18.999 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T03:01:19.027 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-06T03:01:19.055 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-06T03:01:19.094 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-06T03:01:19.319 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-06T03:01:19.346 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-06T03:01:19.371 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-toml. 2023-11-06T03:01:19.377 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-06T03:01:19.387 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-11-06T03:01:19.421 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-06T03:01:19.663 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn. 2023-11-06T03:01:19.690 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-06T03:01:19.708 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2023-11-06T03:01:19.730 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T03:01:19.735 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-06T03:01:19.774 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T03:01:20.136 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-06T03:01:20.164 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-11-06T03:01:20.202 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-06T03:01:20.464 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2023-11-06T03:01:20.480 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-06T03:01:20.490 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-06T03:01:20.514 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T03:01:20.518 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:20.557 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:20.842 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-06T03:01:20.870 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:20.901 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:21.312 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-06T03:01:21.340 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-06T03:01:21.379 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T03:01:21.710 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cachetools. 2023-11-06T03:01:21.737 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-11-06T03:01:21.777 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-06T03:01:21.907 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2023-11-06T03:01:21.935 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-06T03:01:21.974 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T03:01:22.055 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-06T03:01:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-06T03:01:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-06T03:01:22.336 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-06T03:01:22.363 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-06T03:01:22.402 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-06T03:01:22.424 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-06T03:01:22.451 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-06T03:01:22.483 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T03:01:22.713 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2023-11-06T03:01:22.741 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-06T03:01:22.780 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T03:01:22.794 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rsa. 2023-11-06T03:01:22.821 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-11-06T03:01:22.860 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-06T03:01:23.131 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-google-auth. 2023-11-06T03:01:23.158 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-11-06T03:01:23.189 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-06T03:01:23.492 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-06T03:01:23.521 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-06T03:01:23.559 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T03:01:23.565 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-06T03:01:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:23.631 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:23.870 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-websocket. 2023-11-06T03:01:23.897 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-11-06T03:01:23.953 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-06T03:01:24.274 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-06T03:01:24.301 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-06T03:01:24.340 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T03:01:24.827 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2023-11-06T03:01:24.854 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-11-06T03:01:24.893 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-06T03:01:25.172 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-06T03:01:25.196 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-06T03:01:25.200 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:25.223 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-06T03:01:25.246 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:25.271 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-06T03:01:25.515 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-06T03:01:25.530 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-06T03:01:25.565 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T03:01:25.582 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-06T03:01:25.610 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-06T03:01:25.649 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T03:01:25.876 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-06T03:01:25.903 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T03:01:25.936 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2023-11-06T03:01:25.942 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T03:01:25.963 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-11-06T03:01:26.002 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-06T03:01:26.229 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package jq. 2023-11-06T03:01:26.256 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T03:01:26.295 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-06T03:01:26.322 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2023-11-06T03:01:26.350 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-11-06T03:01:26.389 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-06T03:01:26.548 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package socat. 2023-11-06T03:01:26.575 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-06T03:01:26.614 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-06T03:01:26.717 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-06T03:01:26.745 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-06T03:01:26.783 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T03:01:26.893 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package xmlstarlet. 2023-11-06T03:01:26.908 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-06T03:01:26.943 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-06T03:01:27.061 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2023-11-06T03:01:27.077 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-11-06T03:01:27.111 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-06T03:01:27.288 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-test. 2023-11-06T03:01:27.304 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../098-ceph-test_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:27.346 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:27.423 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-06T03:01:27.451 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-06T03:01:27.489 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T03:01:28.203 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-06T03:01:28.232 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:28.278 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:28.623 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-06T03:01:28.650 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-06T03:01:28.689 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T03:01:29.067 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-06T03:01:29.095 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T03:01:29.142 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T03:01:29.453 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2023-11-06T03:01:29.469 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-06T03:01:29.520 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-06T03:01:29.526 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-volume. 2023-11-06T03:01:29.553 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../099-ceph-volume_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:29.584 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:29.806 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2023-11-06T03:01:29.822 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-06T03:01:29.872 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-06T03:01:29.955 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-06T03:01:29.983 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:30.021 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:30.285 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2023-11-06T03:01:30.312 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-06T03:01:30.316 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-colorama. 2023-11-06T03:01:30.343 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-11-06T03:01:30.350 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-06T03:01:30.383 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-06T03:01:30.619 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-shell. 2023-11-06T03:01:30.635 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../102-cephfs-shell_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:30.669 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:30.763 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2023-11-06T03:01:30.791 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-ceph-test_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:30.829 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:30.896 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-top. 2023-11-06T03:01:30.913 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../103-cephfs-top_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:30.955 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:31.191 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-06T03:01:31.206 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:31.249 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:31.619 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package nvme-cli. 2023-11-06T03:01:31.647 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-06T03:01:31.886 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T03:01:32.232 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-06T03:01:32.248 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-06T03:01:32.290 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-06T03:01:32.735 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastescript. 2023-11-06T03:01:32.763 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-11-06T03:01:32.801 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-06T03:01:33.076 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2023-11-06T03:01:33.088 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-psutil. 2023-11-06T03:01:33.103 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-ceph-volume_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:33.116 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-06T03:01:33.142 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:33.146 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-06T03:01:33.474 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pygments. 2023-11-06T03:01:33.502 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-06T03:01:33.513 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-06T03:01:33.540 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T03:01:33.541 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:33.579 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:33.924 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-colorama. 2023-11-06T03:01:33.952 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-11-06T03:01:33.953 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-06T03:01:33.969 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-06T03:01:34.007 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-06T03:01:34.011 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-06T03:01:34.318 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-shell. 2023-11-06T03:01:34.322 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplejson. 2023-11-06T03:01:34.335 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-cephfs-shell_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:34.350 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-06T03:01:34.389 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-06T03:01:34.402 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:34.658 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-06T03:01:34.674 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-06T03:01:34.688 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-top. 2023-11-06T03:01:34.704 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-cephfs-top_18.2.0-1181-gf7e9f9af-1jammy_all.deb ... 2023-11-06T03:01:34.716 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-06T03:01:34.938 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:35.190 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-06T03:01:35.218 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:35.257 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:35.487 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package radosgw. 2023-11-06T03:01:35.515 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../113-radosgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:35.553 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:35.694 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2023-11-06T03:01:35.721 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T03:01:36.131 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-06T03:01:36.159 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-06T03:01:36.198 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-06T03:01:36.265 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package rbd-fuse. 2023-11-06T03:01:36.281 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../114-rbd-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:36.323 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:36.619 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-06T03:01:36.626 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2023-11-06T03:01:36.653 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-11-06T03:01:36.692 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-06T03:01:36.878 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T03:01:36.987 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T03:01:37.021 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2023-11-06T03:01:37.048 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-06T03:01:37.087 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-06T03:01:37.249 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-06T03:01:37.440 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pygments. 2023-11-06T03:01:37.468 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-06T03:01:37.507 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T03:01:37.670 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-06T03:01:37.961 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-06T03:01:37.989 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-06T03:01:38.027 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-06T03:01:38.133 INFO:teuthology.orchestra.run.smithi157.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-06T03:01:38.239 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:38.305 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2023-11-06T03:01:38.321 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-06T03:01:38.355 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-06T03:01:38.607 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-06T03:01:38.623 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-06T03:01:38.657 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-06T03:01:38.799 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user cephadm....done 2023-11-06T03:01:38.953 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T03:01:39.230 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-06T03:01:39.345 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2023-11-06T03:01:39.372 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-radosgw_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:39.489 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-06T03:01:39.607 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-06T03:01:39.867 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-06T03:01:40.118 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T03:01:40.156 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2023-11-06T03:01:40.184 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-rbd-fuse_18.2.0-1181-gf7e9f9af-1jammy_amd64.deb ... 2023-11-06T03:01:40.222 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:40.244 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-06T03:01:40.537 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-06T03:01:40.538 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T03:01:40.810 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-06T03:01:40.897 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-06T03:01:40.936 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-06T03:01:41.157 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-06T03:01:41.194 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-06T03:01:41.400 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-06T03:01:41.516 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-06T03:01:41.642 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-06T03:01:41.894 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-06T03:01:42.002 INFO:teuthology.orchestra.run.smithi195.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-06T03:01:42.096 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:42.281 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-06T03:01:42.581 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2023-11-06T03:01:42.728 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-06T03:01:42.787 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:42.995 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-06T03:01:43.055 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T03:01:43.181 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-06T03:01:43.247 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-06T03:01:43.373 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-06T03:01:43.432 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-06T03:01:43.625 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-06T03:01:43.675 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T03:01:43.785 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-06T03:01:43.858 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-06T03:01:44.018 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-06T03:01:44.136 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:44.237 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-06T03:01:44.328 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-06T03:01:44.496 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T03:01:44.688 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-06T03:01:44.789 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-06T03:01:44.947 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-06T03:01:45.115 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T03:01:45.207 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-06T03:01:45.491 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-06T03:01:45.726 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-06T03:01:45.809 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T03:01:45.977 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-06T03:01:46.060 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T03:01:46.186 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-06T03:01:46.260 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:46.312 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T03:01:46.621 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-06T03:01:46.654 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-06T03:01:46.780 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-06T03:01:47.006 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-06T03:01:47.040 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-06T03:01:47.248 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T03:01:47.307 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-06T03:01:47.409 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-06T03:01:47.617 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T03:01:47.785 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:47.902 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-06T03:01:47.911 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-06T03:01:48.178 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-06T03:01:48.211 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T03:01:48.446 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-06T03:01:48.537 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-06T03:01:48.747 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-06T03:01:48.780 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-06T03:01:48.906 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:49.199 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-idna (3.3-1) ... 2023-11-06T03:01:49.432 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-06T03:01:49.508 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T03:01:49.667 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-06T03:01:49.793 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-06T03:01:49.919 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-06T03:01:50.220 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-06T03:01:50.663 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-06T03:01:50.706 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:01:50.832 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-06T03:01:50.914 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-06T03:01:51.083 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T03:01:51.209 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-06T03:01:51.282 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-06T03:01:51.335 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T03:01:51.559 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-06T03:01:51.636 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-06T03:01:51.852 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-06T03:01:51.887 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-06T03:01:52.122 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:01:52.194 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-06T03:01:52.248 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-06T03:01:52.437 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-06T03:01:52.563 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:01:52.667 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-06T03:01:52.864 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-idna (3.3-1) ... 2023-11-06T03:01:52.943 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-06T03:01:53.182 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-06T03:01:53.211 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-06T03:01:53.445 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-06T03:01:53.679 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-06T03:01:53.930 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T03:01:54.057 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-06T03:01:54.341 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T03:01:54.379 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:01:54.475 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T03:01:54.514 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-06T03:01:54.760 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-06T03:01:54.781 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-06T03:01:54.891 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-06T03:01:54.992 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-11-06T03:01:55.061 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-06T03:01:55.285 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-06T03:01:55.304 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:55.536 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-06T03:01:55.676 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-06T03:01:55.787 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:01:55.913 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-11-06T03:01:55.940 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T03:01:56.039 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-06T03:01:56.066 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T03:01:56.340 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-06T03:01:56.484 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-06T03:01:56.583 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-06T03:01:56.777 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-06T03:01:56.818 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-06T03:01:57.028 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:01:57.069 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-06T03:01:57.154 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-06T03:01:57.320 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-06T03:01:57.472 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T03:01:57.579 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-06T03:01:57.705 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-06T03:01:57.781 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T03:01:57.998 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-06T03:01:58.049 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-06T03:01:58.124 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-06T03:01:58.183 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-06T03:01:58.400 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-06T03:01:58.443 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-06T03:01:58.701 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-06T03:01:58.719 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T03:01:58.952 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-06T03:01:58.970 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-06T03:01:59.330 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-06T03:01:59.351 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-06T03:01:59.589 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-06T03:01:59.614 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T03:01:59.714 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-06T03:01:59.960 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T03:02:00.103 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:00.116 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-06T03:02:00.229 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:00.359 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-06T03:02:00.415 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:00.516 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:00.627 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-06T03:02:00.668 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:00.761 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-06T03:02:00.793 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-06T03:02:01.062 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:01.163 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-06T03:02:01.289 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:01.504 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T03:02:01.505 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T03:02:01.546 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-06T03:02:01.822 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-06T03:02:01.948 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-06T03:02:02.073 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:02.174 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:02.224 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-06T03:02:02.300 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:02.435 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T03:02:02.467 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-06T03:02:02.727 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-06T03:02:03.103 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-06T03:02:03.396 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-06T03:02:03.631 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:03.740 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-06T03:02:03.757 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:03.841 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:03.883 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:03.984 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.009 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.195 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.321 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.456 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.565 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-06T03:02:04.780 INFO:teuthology.orchestra.run.smithi157.stdout:Adding group ceph....done 2023-11-06T03:02:04.827 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:04.953 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:05.079 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:05.139 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user ceph....done 2023-11-06T03:02:05.294 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T03:02:05.294 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-06T03:02:05.316 INFO:teuthology.orchestra.run.smithi157.stdout:Setting system user ceph properties....done 2023-11-06T03:02:05.333 INFO:teuthology.orchestra.run.smithi157.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-06T03:02:05.471 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-06T03:02:05.844 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-06T03:02:05.864 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:05.965 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:06.066 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-top (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:06.184 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-06T03:02:06.388 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:06.598 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T03:02:06.598 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T03:02:07.240 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.365 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.431 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.548 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.649 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.750 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:07.865 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T03:02:07.865 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T03:02:07.867 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-shell (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:08.002 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:08.359 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:08.545 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2023-11-06T03:02:08.716 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-06T03:02:08.988 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2023-11-06T03:02:09.165 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2023-11-06T03:02:09.177 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-06T03:02:09.212 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:09.321 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-06T03:02:09.418 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T03:02:09.419 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T03:02:09.717 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-06T03:02:09.938 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:10.139 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T03:02:10.139 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T03:02:10.270 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-mirror (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:10.485 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T03:02:10.486 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-06T03:02:10.690 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:11.020 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T03:02:11.020 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T03:02:11.021 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:11.147 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:11.558 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:11.625 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T03:02:11.626 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-06T03:02:11.684 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:11.827 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.026 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T03:02:12.026 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T03:02:12.124 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.440 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-06T03:02:12.545 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.671 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.780 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.914 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:12.976 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:13.209 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T03:02:13.209 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-06T03:02:13.260 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T03:02:13.753 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:13.971 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T03:02:13.971 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-06T03:02:14.328 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T03:02:14.537 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:14.778 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T03:02:14.781 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-06T03:02:14.848 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T03:02:14.848 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-06T03:02:14.874 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2023-11-06T03:02:15.043 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2023-11-06T03:02:15.043 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2023-11-06T03:02:15.192 INFO:teuthology.orchestra.run.smithi157.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-06T03:02:15.192 INFO:teuthology.orchestra.run.smithi157.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-06T03:02:15.192 INFO:teuthology.orchestra.run.smithi157.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-06T03:02:15.192 INFO:teuthology.orchestra.run.smithi157.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-06T03:02:15.193 INFO:teuthology.orchestra.run.smithi157.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-06T03:02:15.193 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T03:02:15.193 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T03:02:15.193 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T03:02:15.193 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T03:02:15.194 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T03:02:15.195 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T03:02:15.195 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T03:02:15.195 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2023-11-06T03:02:15.195 INFO:teuthology.orchestra.run.smithi157.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-06T03:02:15.196 INFO:teuthology.orchestra.run.smithi157.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-06T03:02:15.196 INFO:teuthology.orchestra.run.smithi157.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-06T03:02:15.196 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-06T03:02:15.196 INFO:teuthology.orchestra.run.smithi157.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T03:02:15.196 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2023-11-06T03:02:15.197 INFO:teuthology.orchestra.run.smithi157.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-06T03:02:15.197 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-06T03:02:15.197 INFO:teuthology.orchestra.run.smithi157.stdout: libengine-pkcs11-openssl 2023-11-06T03:02:15.229 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2023-11-06T03:02:15.231 INFO:teuthology.orchestra.run.smithi157.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-06T03:02:15.231 INFO:teuthology.orchestra.run.smithi157.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-06T03:02:15.231 INFO:teuthology.orchestra.run.smithi157.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-06T03:02:15.231 INFO:teuthology.orchestra.run.smithi157.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-06T03:02:15.231 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-06T03:02:15.232 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-06T03:02:15.232 INFO:teuthology.orchestra.run.smithi157.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T03:02:15.232 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2023-11-06T03:02:15.233 INFO:teuthology.orchestra.run.smithi157.stdout: libssl3 2023-11-06T03:02:15.397 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:15.410 INFO:teuthology.orchestra.run.smithi157.stdout:1 upgraded, 39 newly installed, 0 to remove and 257 not upgraded. 2023-11-06T03:02:15.410 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 54.6 MB of archives. 2023-11-06T03:02:15.410 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-06T03:02:15.411 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-06T03:02:15.515 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:15.641 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:15.837 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-06T03:02:15.888 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-06T03:02:16.143 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-06T03:02:16.178 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-06T03:02:16.180 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-06T03:02:16.185 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-06T03:02:16.189 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-06T03:02:16.190 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-06T03:02:16.193 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-06T03:02:16.195 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-06T03:02:16.360 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-06T03:02:16.361 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-06T03:02:16.375 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:16.594 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:16.605 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-06T03:02:16.720 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:16.871 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (18.2.0-1181-gf7e9f9af-1jammy) ... 2023-11-06T03:02:17.249 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T03:02:17.731 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-06T03:02:17.735 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-06T03:02:17.736 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-06T03:02:17.742 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-06T03:02:17.743 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-06T03:02:17.754 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-06T03:02:17.758 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-06T03:02:17.774 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-06T03:02:17.939 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-06T03:02:18.350 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T03:02:18.661 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-06T03:02:18.662 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-06T03:02:18.688 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-06T03:02:18.766 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-06T03:02:18.774 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-06T03:02:18.775 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-06T03:02:18.776 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-06T03:02:18.777 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-06T03:02:18.798 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T03:02:18.800 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-06T03:02:18.804 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-06T03:02:18.809 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-06T03:02:18.855 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2023-11-06T03:02:19.043 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2023-11-06T03:02:19.044 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2023-11-06T03:02:19.209 INFO:teuthology.orchestra.run.smithi195.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-06T03:02:19.209 INFO:teuthology.orchestra.run.smithi195.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-06T03:02:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-06T03:02:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-06T03:02:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-06T03:02:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2023-11-06T03:02:19.211 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-06T03:02:19.211 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-06T03:02:19.211 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-06T03:02:19.211 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-06T03:02:19.212 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-06T03:02:19.213 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-06T03:02:19.213 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2023-11-06T03:02:19.213 INFO:teuthology.orchestra.run.smithi195.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-06T03:02:19.213 INFO:teuthology.orchestra.run.smithi195.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-06T03:02:19.214 INFO:teuthology.orchestra.run.smithi195.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-06T03:02:19.214 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-06T03:02:19.214 INFO:teuthology.orchestra.run.smithi195.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T03:02:19.214 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2023-11-06T03:02:19.214 INFO:teuthology.orchestra.run.smithi195.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-06T03:02:19.215 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-06T03:02:19.215 INFO:teuthology.orchestra.run.smithi195.stdout: libengine-pkcs11-openssl 2023-11-06T03:02:19.249 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2023-11-06T03:02:19.250 INFO:teuthology.orchestra.run.smithi195.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-06T03:02:19.250 INFO:teuthology.orchestra.run.smithi195.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-06T03:02:19.251 INFO:teuthology.orchestra.run.smithi195.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-06T03:02:19.251 INFO:teuthology.orchestra.run.smithi195.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-06T03:02:19.251 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-06T03:02:19.251 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-06T03:02:19.252 INFO:teuthology.orchestra.run.smithi195.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-06T03:02:19.252 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2023-11-06T03:02:19.252 INFO:teuthology.orchestra.run.smithi195.stdout: libssl3 2023-11-06T03:02:19.432 INFO:teuthology.orchestra.run.smithi195.stdout:1 upgraded, 39 newly installed, 0 to remove and 257 not upgraded. 2023-11-06T03:02:19.432 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 54.6 MB of archives. 2023-11-06T03:02:19.433 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-06T03:02:19.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-06T03:02:19.455 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-06T03:02:19.538 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-06T03:02:19.538 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-06T03:02:19.628 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-06T03:02:19.633 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-06T03:02:19.909 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-06T03:02:20.090 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-06T03:02:20.092 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-06T03:02:20.166 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-06T03:02:20.200 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-06T03:02:20.202 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-06T03:02:20.207 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-06T03:02:20.211 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-06T03:02:20.212 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-06T03:02:20.215 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-06T03:02:20.217 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-06T03:02:20.383 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-06T03:02:20.383 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-06T03:02:20.494 INFO:teuthology.orchestra.run.smithi157.stdout:Preconfiguring packages ... 2023-11-06T03:02:20.628 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-06T03:02:20.636 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 54.6 MB in 5s (11.2 MB/s) 2023-11-06T03:02:20.761 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package flex. 2023-11-06T03:02:20.857 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172902 files and directories currently installed.) 2023-11-06T03:02:20.863 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:20.909 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-06T03:02:21.468 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T03:02:21.632 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-06T03:02:21.759 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-06T03:02:21.764 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-06T03:02:21.764 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-06T03:02:21.771 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-06T03:02:21.771 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-06T03:02:21.783 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-06T03:02:21.787 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-06T03:02:21.804 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-06T03:02:22.033 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:22.550 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package bison. 2023-11-06T03:02:22.648 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172975 files and directories currently installed.) 2023-11-06T03:02:22.653 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-06T03:02:22.690 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-06T03:02:22.691 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-06T03:02:22.702 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T03:02:22.716 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-06T03:02:22.794 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-06T03:02:22.803 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-06T03:02:22.804 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-06T03:02:22.805 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-06T03:02:22.806 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-06T03:02:22.832 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-06T03:02:22.836 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-06T03:02:23.086 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-06T03:02:23.114 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-06T03:02:23.144 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T03:02:23.464 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package dump. 2023-11-06T03:02:23.492 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-06T03:02:23.530 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking dump (0.4b47-3) ... 2023-11-06T03:02:23.562 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-06T03:02:23.644 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-06T03:02:23.645 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-06T03:02:23.645 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-06T03:02:23.734 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-06T03:02:23.740 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-06T03:02:23.834 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package indent. 2023-11-06T03:02:23.850 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:23.892 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T03:02:24.145 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-06T03:02:24.161 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-06T03:02:24.195 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T03:02:24.196 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-06T03:02:24.198 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-06T03:02:24.473 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-06T03:02:24.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-06T03:02:24.540 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T03:02:24.604 INFO:teuthology.orchestra.run.smithi195.stdout:Preconfiguring packages ... 2023-11-06T03:02:24.751 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 54.6 MB in 5s (11.0 MB/s) 2023-11-06T03:02:24.849 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package flex. 2023-11-06T03:02:24.910 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-06T03:02:24.926 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-06T03:02:24.937 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172902 files and directories currently installed.) 2023-11-06T03:02:24.942 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:24.968 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T03:02:24.981 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-06T03:02:25.264 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-06T03:02:25.292 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:25.330 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:25.532 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-06T03:02:25.549 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:25.581 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T03:02:25.590 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:25.754 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjson-perl. 2023-11-06T03:02:25.851 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-06T03:02:25.893 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-06T03:02:26.180 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:26.204 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-06T03:02:26.233 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-06T03:02:26.271 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T03:02:26.815 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package bison. 2023-11-06T03:02:26.921 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172975 files and directories currently installed.) 2023-11-06T03:02:26.926 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-06T03:02:26.976 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T03:02:27.451 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-06T03:02:27.479 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-06T03:02:27.526 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T03:02:27.929 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package dump. 2023-11-06T03:02:27.957 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-06T03:02:28.004 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking dump (0.4b47-3) ... 2023-11-06T03:02:28.049 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-06T03:02:28.077 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T03:02:28.115 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:28.282 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package indent. 2023-11-06T03:02:28.299 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:28.341 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T03:02:28.343 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-06T03:02:28.359 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T03:02:28.401 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:28.618 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-06T03:02:28.635 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-06T03:02:28.685 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T03:02:28.704 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-06T03:02:28.732 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T03:02:28.770 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:29.089 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-06T03:02:29.115 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-06T03:02:29.116 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-06T03:02:29.142 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-06T03:02:29.163 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T03:02:29.181 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T03:02:29.441 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-06T03:02:29.457 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-06T03:02:29.499 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T03:02:29.500 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-06T03:02:29.528 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T03:02:29.576 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T03:02:29.853 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-06T03:02:29.880 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:29.928 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:29.966 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-06T03:02:29.991 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:30.029 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:30.255 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-06T03:02:30.283 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-06T03:02:30.330 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:30.366 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-06T03:02:30.393 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:30.432 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:30.649 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjson-perl. 2023-11-06T03:02:30.677 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-06T03:02:30.724 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-06T03:02:30.743 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-06T03:02:30.771 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:30.810 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:31.028 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-06T03:02:31.044 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T03:02:31.078 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:31.103 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-06T03:02:31.131 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-06T03:02:31.177 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T03:02:31.682 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-06T03:02:31.710 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T03:02:32.052 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package modemmanager. 2023-11-06T03:02:32.079 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T03:02:32.118 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:32.741 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager. 2023-11-06T03:02:32.768 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T03:02:32.815 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-06T03:02:32.839 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-06T03:02:32.867 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T03:02:32.905 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:33.183 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-06T03:02:33.199 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-06T03:02:33.249 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:33.431 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ppp. 2023-11-06T03:02:33.459 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-06T03:02:33.644 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-06T03:02:33.672 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T03:02:33.719 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:33.741 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T03:02:34.052 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pptp-linux. 2023-11-06T03:02:34.069 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-06T03:02:34.088 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-06T03:02:34.111 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-06T03:02:34.117 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-06T03:02:34.155 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T03:02:34.313 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-06T03:02:34.330 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-06T03:02:34.491 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-06T03:02:34.519 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T03:02:34.566 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T03:02:34.674 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-06T03:02:34.694 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-06T03:02:34.732 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-common (238) ... 2023-11-06T03:02:34.986 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-06T03:02:35.014 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:35.060 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:35.447 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-06T03:02:35.475 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:35.513 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:35.722 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-06T03:02:35.750 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:35.780 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:35.816 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-06T03:02:35.844 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:35.882 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:36.193 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-06T03:02:36.222 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-06T03:02:36.268 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:36.293 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-common. 2023-11-06T03:02:36.321 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-06T03:02:36.378 INFO:teuthology.orchestra.run.smithi157.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-06T03:02:36.408 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-common (238) ... 2023-11-06T03:02:36.763 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-06T03:02:36.788 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-06T03:02:36.821 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T03:02:36.834 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-14. 2023-11-06T03:02:36.862 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:36.904 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:37.141 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package modemmanager. 2023-11-06T03:02:37.169 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-06T03:02:37.232 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:37.822 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager. 2023-11-06T03:02:37.850 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-06T03:02:37.897 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-06T03:02:38.345 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql. 2023-11-06T03:02:38.362 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-06T03:02:38.403 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql (14+238) ... 2023-11-06T03:02:38.521 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ppp. 2023-11-06T03:02:38.549 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-06T03:02:38.647 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client. 2023-11-06T03:02:38.675 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-06T03:02:38.714 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client (14+238) ... 2023-11-06T03:02:38.839 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T03:02:38.941 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-06T03:02:38.957 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-06T03:02:39.008 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-06T03:02:39.184 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pptp-linux. 2023-11-06T03:02:39.212 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-06T03:02:39.251 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-06T03:02:39.252 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl8.6. 2023-11-06T03:02:39.280 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-06T03:02:39.319 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T03:02:39.487 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-06T03:02:39.501 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-06T03:02:39.545 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-06T03:02:39.596 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl. 2023-11-06T03:02:39.625 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-06T03:02:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-06T03:02:40.025 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package wpasupplicant. 2023-11-06T03:02:40.052 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-06T03:02:40.091 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T03:02:40.242 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-06T03:02:40.271 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-06T03:02:40.395 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-06T03:02:40.411 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-06T03:02:40.445 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-06T03:02:40.523 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-common (238) ... 2023-11-06T03:02:40.832 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-06T03:02:40.860 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-06T03:02:40.906 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T03:02:41.009 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-06T03:02:41.037 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:41.123 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:41.201 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-common (238) ... 2023-11-06T03:02:41.379 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-06T03:02:41.489 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T03:02:41.615 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:41.741 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T03:02:41.867 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T03:02:41.952 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-common. 2023-11-06T03:02:41.980 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-06T03:02:42.001 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:42.037 INFO:teuthology.orchestra.run.smithi195.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-06T03:02:42.066 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-common (238) ... 2023-11-06T03:02:42.447 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-06T03:02:42.484 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-14. 2023-11-06T03:02:42.512 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-06T03:02:42.532 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T03:02:42.546 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:42.917 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T03:02:42.917 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T03:02:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T03:02:43.569 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:43.670 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-06T03:02:43.771 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T03:02:43.795 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql. 2023-11-06T03:02:43.825 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-06T03:02:43.881 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:43.887 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql (14+238) ... 2023-11-06T03:02:44.007 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T03:02:44.133 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T03:02:44.181 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client. 2023-11-06T03:02:44.209 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-06T03:02:44.256 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client (14+238) ... 2023-11-06T03:02:44.533 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-06T03:02:44.561 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-06T03:02:44.600 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-06T03:02:44.722 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T03:02:44.840 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up dump (0.4b47-3) ... 2023-11-06T03:02:44.919 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package tcl8.6. 2023-11-06T03:02:44.933 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-06T03:02:44.947 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-06T03:02:44.986 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T03:02:44.999 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:45.126 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T03:02:45.260 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-06T03:02:45.288 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package tcl. 2023-11-06T03:02:45.316 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-06T03:02:45.369 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-06T03:02:45.469 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-06T03:02:45.620 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T03:02:45.711 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-06T03:02:45.780 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T03:02:45.808 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package wpasupplicant. 2023-11-06T03:02:45.836 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-06T03:02:45.883 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T03:02:46.040 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T03:02:46.254 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-06T03:02:46.282 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-06T03:02:46.320 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-06T03:02:46.433 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T03:02:46.559 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:46.698 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-06T03:02:46.712 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-06T03:02:46.726 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-06T03:02:46.774 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T03:02:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client (14+238) ... 2023-11-06T03:02:46.990 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-06T03:02:47.068 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-common (238) ... 2023-11-06T03:02:47.304 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-06T03:02:47.439 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-06T03:02:47.459 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:47.460 INFO:teuthology.orchestra.run.smithi157.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-06T03:02:47.460 INFO:teuthology.orchestra.run.smithi157.stdout:which means they are currently configured by ifupdown: 2023-11-06T03:02:47.460 INFO:teuthology.orchestra.run.smithi157.stdout:- enp3s0f1 2023-11-06T03:02:47.460 INFO:teuthology.orchestra.run.smithi157.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-06T03:02:47.461 INFO:teuthology.orchestra.run.smithi157.stdout:remove their configuration from /etc/network/interfaces. 2023-11-06T03:02:47.461 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:47.591 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:47.591 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-06T03:02:47.741 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-06T03:02:47.892 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-06T03:02:47.973 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-06T03:02:48.043 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:48.342 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-06T03:02:48.575 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-06T03:02:48.673 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-06T03:02:48.933 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:49.068 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:49.104 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T03:02:49.104 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-06T03:02:49.194 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-06T03:02:49.320 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-common (238) ... 2023-11-06T03:02:49.659 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-06T03:02:49.794 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-06T03:02:49.920 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-06T03:02:50.046 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-06T03:02:50.159 INFO:teuthology.orchestra.run.smithi157.stdout:Adding user postgres to group ssl-cert 2023-11-06T03:02:50.172 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:50.323 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-06T03:02:50.374 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:50.375 INFO:teuthology.orchestra.run.smithi157.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-06T03:02:50.474 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-06T03:02:50.570 INFO:teuthology.orchestra.run.smithi157.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-06T03:02:50.571 INFO:teuthology.orchestra.run.smithi157.stdout:Removing obsolete dictionary files: 2023-11-06T03:02:51.148 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-06T03:02:51.564 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:51.630 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-06T03:02:51.732 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up dump (0.4b47-3) ... 2023-11-06T03:02:51.817 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-06T03:02:51.870 INFO:teuthology.orchestra.run.smithi157.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-06T03:02:51.883 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:51.890 INFO:teuthology.orchestra.run.smithi157.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-06T03:02:51.998 INFO:teuthology.orchestra.run.smithi157.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-06T03:02:51.998 INFO:teuthology.orchestra.run.smithi157.stdout:This user must also own the server process. 2023-11-06T03:02:51.998 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:51.999 INFO:teuthology.orchestra.run.smithi157.stdout:The database cluster will be initialized with locale "en_US". 2023-11-06T03:02:51.999 INFO:teuthology.orchestra.run.smithi157.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-06T03:02:51.999 INFO:teuthology.orchestra.run.smithi157.stdout:The default text search configuration will be set to "english". 2023-11-06T03:02:51.999 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:51.999 INFO:teuthology.orchestra.run.smithi157.stdout:Data page checksums are disabled. 2023-11-06T03:02:52.000 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:02:52.000 INFO:teuthology.orchestra.run.smithi157.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-06T03:02:52.000 INFO:teuthology.orchestra.run.smithi157.stdout:creating subdirectories ... ok 2023-11-06T03:02:52.001 INFO:teuthology.orchestra.run.smithi157.stdout:selecting dynamic shared memory implementation ... posix 2023-11-06T03:02:52.015 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default max_connections ... 100 2023-11-06T03:02:52.034 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-06T03:02:52.035 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default shared_buffers ... 128MB 2023-11-06T03:02:52.046 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default time zone ... Etc/UTC 2023-11-06T03:02:52.047 INFO:teuthology.orchestra.run.smithi157.stdout:creating configuration files ... ok 2023-11-06T03:02:52.165 INFO:teuthology.orchestra.run.smithi157.stdout:running bootstrap script ... ok 2023-11-06T03:02:52.185 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-06T03:02:52.320 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-06T03:02:52.446 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-06T03:02:52.536 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-06T03:02:52.597 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-06T03:02:52.740 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-06T03:02:52.950 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-06T03:02:53.101 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:53.231 INFO:teuthology.orchestra.run.smithi157.stdout:performing post-bootstrap initialization ... ok 2023-11-06T03:02:53.252 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-06T03:02:53.438 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client (14+238) ... 2023-11-06T03:02:53.565 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-06T03:02:53.991 INFO:teuthology.orchestra.run.smithi157.stdout:syncing data to disk ... ok 2023-11-06T03:02:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:54.135 INFO:teuthology.orchestra.run.smithi195.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-06T03:02:54.136 INFO:teuthology.orchestra.run.smithi195.stdout:which means they are currently configured by ifupdown: 2023-11-06T03:02:54.136 INFO:teuthology.orchestra.run.smithi195.stdout:- enp3s0f1 2023-11-06T03:02:54.136 INFO:teuthology.orchestra.run.smithi195.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-06T03:02:54.136 INFO:teuthology.orchestra.run.smithi195.stdout:remove their configuration from /etc/network/interfaces. 2023-11-06T03:02:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:54.264 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-06T03:02:54.416 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-06T03:02:54.611 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-06T03:02:54.958 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-06T03:02:55.559 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-06T03:02:55.802 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-06T03:02:56.070 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-06T03:02:56.196 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-common (238) ... 2023-11-06T03:02:57.200 INFO:teuthology.orchestra.run.smithi195.stdout:Adding user postgres to group ssl-cert 2023-11-06T03:02:57.216 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:57.342 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-06T03:02:57.430 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:57.430 INFO:teuthology.orchestra.run.smithi195.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-06T03:02:57.469 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql (14+238) ... 2023-11-06T03:02:57.594 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:02:57.622 INFO:teuthology.orchestra.run.smithi195.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-06T03:02:57.623 INFO:teuthology.orchestra.run.smithi195.stdout:Removing obsolete dictionary files: 2023-11-06T03:02:57.729 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:02:57.963 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T03:02:57.963 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T03:02:58.251 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-06T03:02:58.523 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-06T03:02:58.656 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-06T03:02:58.800 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T03:02:58.943 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T03:02:58.988 INFO:teuthology.orchestra.run.smithi195.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-06T03:02:59.006 INFO:teuthology.orchestra.run.smithi195.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-06T03:02:59.044 INFO:teuthology.orchestra.run.smithi195.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-06T03:02:59.044 INFO:teuthology.orchestra.run.smithi195.stdout:This user must also own the server process. 2023-11-06T03:02:59.045 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:59.045 INFO:teuthology.orchestra.run.smithi195.stdout:The database cluster will be initialized with locale "en_US". 2023-11-06T03:02:59.045 INFO:teuthology.orchestra.run.smithi195.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-06T03:02:59.045 INFO:teuthology.orchestra.run.smithi195.stdout:The default text search configuration will be set to "english". 2023-11-06T03:02:59.045 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:59.046 INFO:teuthology.orchestra.run.smithi195.stdout:Data page checksums are disabled. 2023-11-06T03:02:59.046 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:02:59.046 INFO:teuthology.orchestra.run.smithi195.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-06T03:02:59.046 INFO:teuthology.orchestra.run.smithi195.stdout:creating subdirectories ... ok 2023-11-06T03:02:59.047 INFO:teuthology.orchestra.run.smithi195.stdout:selecting dynamic shared memory implementation ... posix 2023-11-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default max_connections ... 100 2023-11-06T03:02:59.079 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default shared_buffers ... 128MB 2023-11-06T03:02:59.088 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default time zone ... Etc/UTC 2023-11-06T03:02:59.089 INFO:teuthology.orchestra.run.smithi195.stdout:creating configuration files ... ok 2023-11-06T03:02:59.203 INFO:teuthology.orchestra.run.smithi195.stdout:running bootstrap script ... ok 2023-11-06T03:02:59.878 INFO:teuthology.orchestra.run.smithi195.stdout:performing post-bootstrap initialization ... ok 2023-11-06T03:03:00.037 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-06T03:03:00.137 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-06T03:03:00.643 INFO:teuthology.orchestra.run.smithi195.stdout:syncing data to disk ... ok 2023-11-06T03:03:00.750 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T03:03:00.753 DEBUG:teuthology.parallel:result is None 2023-11-06T03:03:01.058 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-06T03:03:03.937 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:03:04.064 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-06T03:03:04.214 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql (14+238) ... 2023-11-06T03:03:04.365 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-06T03:03:04.516 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-06T03:03:04.796 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T03:03:04.796 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-06T03:03:05.394 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-06T03:03:05.730 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-06T03:03:05.881 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-06T03:03:07.218 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-06T03:03:07.326 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-06T03:03:07.962 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-06T03:03:07.965 DEBUG:teuthology.parallel:result is None 2023-11-06T03:03:07.966 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T03:03:08.243 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-query -W -f '${Version}' ceph 2023-11-06T03:03:08.262 INFO:teuthology.orchestra.run.smithi157.stdout:18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:03:08.263 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:03:08.263 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181-gf7e9f9af-1jammy is installed. 2023-11-06T03:03:08.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e9f9af51acf98e0ccc3037487018e4ae40caa5 2023-11-06T03:03:08.408 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2023-11-06T03:03:08.427 INFO:teuthology.orchestra.run.smithi195.stdout:18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:03:08.428 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1181-gf7e9f9af-1jammy 2023-11-06T03:03:08.428 INFO:teuthology.task.install:The correct ceph version 18.2.0-1181-gf7e9f9af-1jammy is installed. 2023-11-06T03:03:08.428 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-06T03:03:08.429 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:08.429 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T03:03:08.443 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:08.444 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-06T03:03:08.483 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-06T03:03:08.484 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:08.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T03:03:08.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T03:03:08.553 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:08.554 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2023-11-06T03:03:08.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-06T03:03:08.621 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-06T03:03:08.622 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:08.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T03:03:08.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T03:03:08.689 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:08.690 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-06T03:03:08.703 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-06T03:03:08.758 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-06T03:03:08.759 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:08.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T03:03:08.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T03:03:08.825 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:08.826 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2023-11-06T03:03:08.839 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-06T03:03:08.893 INFO:teuthology.run_tasks:Running task ceph... 2023-11-06T03:03:08.961 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-06T03:03:08.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2023-11-06T03:03:08.964 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2023-11-06T03:03:08.977 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-06T03:03:08.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T03:03:09.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-06T03:03:09.033 INFO:tasks.ceph:Creating extra log directories... 2023-11-06T03:03:09.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T03:03:09.071 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-06T03:03:09.092 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-06T03:03:09.092 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-06T03:03:09.092 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-06T03:03:09.093 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:09.093 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T03:03:09.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T03:03:09.205 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:09.205 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-06T03:03:09.211 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-06T03:03:09.290 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-06T03:03:09.291 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-06T03:03:09.291 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738', 'branch': 'reef-release', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7448738', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef-release'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef-release', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1834, 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'reef-release', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f7e9f9af51acf98e0ccc3037487018e4ae40caa5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgeOL5FtyrcN1GMmQwuIlZ0j86FC3xLAkooaUK1Flxytgx8S3U7T8Dnh0JRIiIgs6RG3WxlxL9T39UpfPzOnXQ=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAx7yMHLyHmaa5GecES0RDKKy6Zo72ZjhkX6pGRAgXz1f3EtApBvRPCC+Lteqqy4ps5/3/yIcm1XXP7qnsP8+jA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7e9f9af51acf98e0ccc3037487018e4ae40caa5'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-05_23:44:49', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3084822'} 2023-11-06T03:03:09.292 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T03:03:09.295 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-06T03:03:09.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T03:03:09.343 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2023-11-06T03:03:09.390 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:09.391 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T03:03:09.402 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T03:03:09.402 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2023-11-06T03:03:09.450 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T03:03:09.450 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 874 Links: 1 2023-11-06T03:03:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-06 02:57:01.696569320 +0000 2023-11-06T03:03:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-06 02:57:01.320572678 +0000 2023-11-06T03:03:09.451 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-06 02:57:01.320572678 +0000 2023-11-06T03:03:09.452 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-11-06T03:03:09.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T03:03:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:03:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:03:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000248164 s, 2.1 MB/s 2023-11-06T03:03:09.506 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T03:03:09.556 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2023-11-06T03:03:09.602 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T03:03:09.603 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:09.603 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 892 Links: 1 2023-11-06T03:03:09.604 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:09.604 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-06 02:57:02.348563496 +0000 2023-11-06T03:03:09.605 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-06 02:57:01.960566962 +0000 2023-11-06T03:03:09.605 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-06 02:57:01.960566962 +0000 2023-11-06T03:03:09.605 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-11-06T03:03:09.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T03:03:09.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:03:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:03:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000243625 s, 2.1 MB/s 2023-11-06T03:03:09.658 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T03:03:09.707 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2023-11-06T03:03:09.754 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T03:03:09.754 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 911 Links: 1 2023-11-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-06 02:57:02.936558244 +0000 2023-11-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-06 02:57:02.564561567 +0000 2023-11-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-06 02:57:02.564561567 +0000 2023-11-06T03:03:09.756 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-11-06T03:03:09.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T03:03:09.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:03:09.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:03:09.808 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000274655 s, 1.9 MB/s 2023-11-06T03:03:09.809 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T03:03:09.859 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2023-11-06T03:03:09.907 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T03:03:09.907 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:09.908 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 927 Links: 1 2023-11-06T03:03:09.908 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:09.908 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2023-11-06 02:57:03.508553135 +0000 2023-11-06T03:03:09.909 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2023-11-06 02:57:03.148556351 +0000 2023-11-06T03:03:09.909 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2023-11-06 02:57:03.148556351 +0000 2023-11-06T03:03:09.909 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: - 2023-11-06T03:03:09.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T03:03:09.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:03:09.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:03:09.961 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000287421 s, 1.8 MB/s 2023-11-06T03:03:09.962 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T03:03:10.011 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:10.012 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:10.012 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2023-11-06T03:03:10.018 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-06T03:03:10.019 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2023-11-06T03:03:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-06T03:03:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:10.067 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 872 Links: 1 2023-11-06T03:03:10.067 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:10.068 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-06 02:57:01.703152499 +0000 2023-11-06T03:03:10.068 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-06 02:57:01.339155747 +0000 2023-11-06T03:03:10.068 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-06 02:57:01.339155747 +0000 2023-11-06T03:03:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2023-11-06T03:03:10.068 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-06T03:03:10.121 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-06T03:03:10.121 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-06T03:03:10.121 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000253452 s, 2.0 MB/s 2023-11-06T03:03:10.122 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-06T03:03:10.172 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2023-11-06T03:03:10.218 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-06T03:03:10.218 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:10.219 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 894 Links: 1 2023-11-06T03:03:10.219 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:10.219 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-06 02:57:02.311147076 +0000 2023-11-06T03:03:10.219 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-06 02:57:01.915150609 +0000 2023-11-06T03:03:10.219 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-06 02:57:01.915150609 +0000 2023-11-06T03:03:10.220 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2023-11-06T03:03:10.220 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-06T03:03:10.273 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-06T03:03:10.273 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-06T03:03:10.273 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000252038 s, 2.0 MB/s 2023-11-06T03:03:10.274 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-06T03:03:10.323 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2023-11-06T03:03:10.370 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-06T03:03:10.371 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:10.371 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 908 Links: 1 2023-11-06T03:03:10.371 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:10.371 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-06 02:57:02.919141652 +0000 2023-11-06T03:03:10.372 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-06 02:57:02.535145078 +0000 2023-11-06T03:03:10.372 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-06 02:57:02.535145078 +0000 2023-11-06T03:03:10.372 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2023-11-06T03:03:10.372 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-06T03:03:10.424 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-06T03:03:10.425 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-06T03:03:10.425 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000249221 s, 2.1 MB/s 2023-11-06T03:03:10.426 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-06T03:03:10.476 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2023-11-06T03:03:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-06T03:03:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-06T03:03:10.523 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-11-06T03:03:10.523 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-06T03:03:10.524 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-06 02:57:03.143139654 +0000 2023-11-06T03:03:10.524 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-06 02:57:03.139139690 +0000 2023-11-06T03:03:10.524 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-06 02:57:03.139139690 +0000 2023-11-06T03:03:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2023-11-06T03:03:10.524 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-06T03:03:10.576 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-06T03:03:10.577 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-06T03:03:10.577 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000231681 s, 2.2 MB/s 2023-11-06T03:03:10.578 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-06T03:03:10.628 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:10.628 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T03:03:10.628 INFO:tasks.ceph:Generating config... 2023-11-06T03:03:10.630 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-06T03:03:10.630 INFO:tasks.ceph:[client] debug client = 20 2023-11-06T03:03:10.630 INFO:tasks.ceph:[client] debug ms = 1 2023-11-06T03:03:10.631 INFO:tasks.ceph:[client] pid file = 2023-11-06T03:03:10.631 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-06T03:03:10.631 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-06T03:03:10.632 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-06T03:03:10.632 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-06T03:03:10.632 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-06T03:03:10.632 INFO:tasks.ceph:[mds] mds cache max size = 1000 2023-11-06T03:03:10.633 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-06T03:03:10.633 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-06T03:03:10.633 INFO:tasks.ceph:[mds] mds log max segments = 1 2023-11-06T03:03:10.633 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-06T03:03:10.634 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-06T03:03:10.634 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-06T03:03:10.634 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-06T03:03:10.634 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-06T03:03:10.635 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-06T03:03:10.635 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-06T03:03:10.635 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-06T03:03:10.635 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-06T03:03:10.636 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-06T03:03:10.636 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-06T03:03:10.636 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-06T03:03:10.636 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-06T03:03:10.637 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-06T03:03:10.637 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-06T03:03:10.637 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-06T03:03:10.637 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-06T03:03:10.638 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-06T03:03:10.638 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-06T03:03:10.638 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-06T03:03:10.639 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-06T03:03:10.639 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-06T03:03:10.639 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-06T03:03:10.639 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-06T03:03:10.640 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-06T03:03:10.640 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-06T03:03:10.640 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-06T03:03:10.640 INFO:tasks.ceph:Setting up mon.a... 2023-11-06T03:03:10.641 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-06T03:03:10.676 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.keyring 2023-11-06T03:03:10.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-06T03:03:10.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T03:03:10.821 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.157'), ('mon.b', '172.21.15.195'), ('mon.c', '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]')] 2023-11-06T03:03:10.821 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-06T03:03:10.822 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:10.823 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T03:03:10.866 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.157 --add b 172.21.15.195 --addv c '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:10.950 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:10.950 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: generated fsid eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:03:10.950 INFO:teuthology.orchestra.run.smithi157.stdout:setting min_mon_release = pacific 2023-11-06T03:03:10.951 INFO:teuthology.orchestra.run.smithi157.stdout:epoch 0 2023-11-06T03:03:10.951 INFO:teuthology.orchestra.run.smithi157.stdout:fsid eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:03:10.951 INFO:teuthology.orchestra.run.smithi157.stdout:last_changed 2023-11-06T03:03:10.954439+0000 2023-11-06T03:03:10.951 INFO:teuthology.orchestra.run.smithi157.stdout:created 2023-11-06T03:03:10.954439+0000 2023-11-06T03:03:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:min_mon_release 16 (pacific) 2023-11-06T03:03:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:election_strategy: 1 2023-11-06T03:03:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:0: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.a 2023-11-06T03:03:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2023-11-06T03:03:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:2: [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] mon.c 2023-11-06T03:03:10.953 INFO:teuthology.orchestra.run.smithi157.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-06T03:03:10.954 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-06T03:03:11.006 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eca2e009-f15d-40e2-b739-48eaa1ef5309... 2023-11-06T03:03:11.007 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:03:11.051 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:03:11.080 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-06T03:03:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-06T03:03:11.080 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:03:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-06T03:03:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-06T03:03:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.081 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-06T03:03:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-06T03:03:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-06T03:03:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.082 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-06T03:03:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-06T03:03:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:03:11.083 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-06T03:03:11.084 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-06T03:03:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-06T03:03:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:03:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:03:11.085 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-06T03:03:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:03:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-06T03:03:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:03:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:03:11.086 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-06T03:03:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:03:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-06T03:03:11.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-06T03:03:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-06T03:03:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-06T03:03:11.088 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-06T03:03:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:03:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:03:11.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.089 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-06T03:03:11.090 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-06T03:03:11.090 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.090 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-06T03:03:11.090 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-06T03:03:11.090 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-06T03:03:11.091 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-06T03:03:11.092 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-06T03:03:11.092 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.092 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-06T03:03:11.092 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-06T03:03:11.092 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.093 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:03:11.093 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.093 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-06T03:03:11.093 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.093 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-06T03:03:11.094 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.094 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:03:11.094 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-06T03:03:11.094 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-06T03:03:11.095 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-06T03:03:11.096 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-06T03:03:11.096 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-06T03:03:11.096 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-06T03:03:11.097 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:03:11.097 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-06T03:03:11.097 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-06T03:03:11.097 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:03:11.097 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-06T03:03:11.098 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:03:11.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:03:11.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-06T03:03:11.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:03:11.098 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.099 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-06T03:03:11.099 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:03:11.099 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-06T03:03:11.099 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.100 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-06T03:03:11.101 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:03:11.101 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:03:11.101 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-06T03:03:11.101 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:03:11.101 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-06T03:03:11.102 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-06T03:03:11.102 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:03:11.102 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-06T03:03:11.102 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-06T03:03:11.103 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:03:11.103 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.103 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:03:11.103 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:03:11.104 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.105 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-06T03:03:11.105 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:03:11.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.105 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:03:11.105 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-06T03:03:11.106 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:11.106 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-06T03:03:11.106 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-06T03:03:11.106 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-06T03:03:11.106 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-06T03:03:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:03:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:03:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-06T03:03:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-06T03:03:11.107 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-06T03:03:11.108 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-06T03:03:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:03:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:03:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:03:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:03:11.109 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-06T03:03:11.110 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-06T03:03:11.110 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-06T03:03:11.110 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:03:11.110 INFO:teuthology.orchestra.run.smithi157.stdout: mds cache max size = 1000 2023-11-06T03:03:11.110 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-06T03:03:11.111 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2023-11-06T03:03:11.111 INFO:teuthology.orchestra.run.smithi157.stdout: mds log max segments = 1 2023-11-06T03:03:11.111 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-06T03:03:11.111 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2023-11-06T03:03:11.111 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:03:11.112 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:03:11.112 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:03:11.114 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-06T03:03:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-06T03:03:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:03:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-06T03:03:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-06T03:03:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-06T03:03:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-06T03:03:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-06T03:03:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-06T03:03:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-06T03:03:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:03:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-06T03:03:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:03:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:03:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-06T03:03:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-06T03:03:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-06T03:03:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:03:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:03:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-06T03:03:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:03:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-06T03:03:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:03:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-06T03:03:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-06T03:03:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-06T03:03:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-06T03:03:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-06T03:03:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:03:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:03:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.123 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-06T03:03:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-06T03:03:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-06T03:03:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-06T03:03:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:03:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-06T03:03:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-06T03:03:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-06T03:03:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-06T03:03:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-06T03:03:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-06T03:03:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-06T03:03:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:03:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-06T03:03:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-06T03:03:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:03:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-06T03:03:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-06T03:03:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-06T03:03:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-06T03:03:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-06T03:03:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2023-11-06T03:03:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-06T03:03:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-06T03:03:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-06T03:03:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-06T03:03:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:03:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-06T03:03:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-06T03:03:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:03:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-06T03:03:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:03:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:03:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-06T03:03:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:03:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.134 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-06T03:03:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:03:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-06T03:03:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:03:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:03:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:03:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:03:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-06T03:03:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.136 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-06T03:03:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:03:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:03:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-06T03:03:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:03:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-06T03:03:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-06T03:03:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:03:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-06T03:03:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-06T03:03:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:03:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:03:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:03:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:03:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:03:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:03:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:03:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-06T03:03:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:03:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:03:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2023-11-06T03:03:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:03:11.141 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-06T03:03:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-06T03:03:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-06T03:03:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-06T03:03:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:03:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:03:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2023-11-06T03:03:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2023-11-06T03:03:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:03:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2023-11-06T03:03:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:03:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:03:11.144 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-06T03:03:11.144 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-06T03:03:11.145 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-06T03:03:11.145 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:03:11.145 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:03:11.145 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:03:11.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:03:11.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2023-11-06T03:03:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2023-11-06T03:03:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2023-11-06T03:03:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:03:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: mds cache max size = 1000 2023-11-06T03:03:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2023-11-06T03:03:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2023-11-06T03:03:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: mds log max segments = 1 2023-11-06T03:03:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2023-11-06T03:03:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2023-11-06T03:03:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:03:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:03:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:03:11.149 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-06T03:03:11.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-06T03:03:11.209 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-06T03:03:11.210 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:11.210 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-06T03:03:11.258 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:11.259 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-06T03:03:11.308 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:03:11.309 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:11.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T03:03:11.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T03:03:11.372 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:11.372 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:11.418 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:03:11.418 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:11.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-06T03:03:11.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-06T03:03:11.442 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:11.443 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:11.490 INFO:tasks.ceph:Setting up mon nodes... 2023-11-06T03:03:11.491 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-06T03:03:11.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T03:03:11.535 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-06T03:03:11.538 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T03:03:11.583 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-06T03:03:11.609 INFO:tasks.ceph:Setting up mds nodes... 2023-11-06T03:03:11.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T03:03:11.651 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-06T03:03:11.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-06T03:03:11.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T03:03:11.792 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-06T03:03:11.794 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-06T03:03:11.849 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T03:03:11.895 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-06T03:03:11.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-06T03:03:11.911 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T03:03:11.991 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-06T03:03:11.994 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-06T03:03:12.051 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-06T03:03:12.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-06T03:03:12.090 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-06T03:03:12.101 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-06T03:03:12.178 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-06T03:03:12.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-06T03:03:12.223 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-06T03:03:12.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-06T03:03:12.308 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-06T03:03:12.318 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-06T03:03:12.318 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-06T03:03:12.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-06T03:03:12.333 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:12.333 INFO:tasks.ceph:role: osd.0 2023-11-06T03:03:12.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:03:12.334 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T03:03:12.390 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:12.391 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:12.391 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:12.391 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:12.391 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:12.391 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:12.392 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:12.392 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:12.392 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:12.392 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:12.415 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-06T03:03:12.417 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-06T03:03:12.417 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-06T03:03:12.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-06T03:03:12.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:12.527 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:12.528 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-06T03:03:12.579 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:12.579 INFO:tasks.ceph:role: osd.1 2023-11-06T03:03:12.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:03:12.580 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T03:03:12.633 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:12.633 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:12.633 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:12.633 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:12.633 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:12.634 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:12.634 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:12.634 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:12.634 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:12.634 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:12.656 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-06T03:03:12.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-06T03:03:12.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-06T03:03:12.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-06T03:03:12.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:12.767 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:12.767 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-06T03:03:12.821 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:12.821 INFO:tasks.ceph:role: osd.2 2023-11-06T03:03:12.821 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:03:12.822 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T03:03:12.881 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:12.881 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:12.881 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:12.882 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:12.882 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:12.882 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:12.882 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:12.882 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:12.883 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:12.883 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:12.905 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-06T03:03:12.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-06T03:03:12.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-06T03:03:12.971 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-06T03:03:13.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:13.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-06T03:03:13.075 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:13.075 INFO:tasks.ceph:role: osd.3 2023-11-06T03:03:13.075 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:03:13.076 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T03:03:13.132 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:13.132 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:13.133 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:13.133 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:13.133 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:13.133 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:13.133 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:13.134 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:13.134 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:13.134 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:13.156 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2023-11-06T03:03:13.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2023-11-06T03:03:13.158 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-06T03:03:13.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-06T03:03:13.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:13.267 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:13.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-06T03:03:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-06T03:03:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T03:03:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T03:03:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-06T03:03:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:13.379+0000 7f9d234e7640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-06T03:03:16.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-06T03:03:16.400 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:16.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.479+0000 7f5d37f60640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-06T03:03:16.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.479+0000 7f5d37f60640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-06T03:03:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.479+0000 7f5d37f60640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T03:03:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.479+0000 7f5d37f60640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T03:03:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.479+0000 7f5d37f60640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-06T03:03:16.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:16.483+0000 7f5d37f60640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-06T03:03:19.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-06T03:03:19.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:19.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.623+0000 7f8acb055640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-06T03:03:19.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.627+0000 7f8acb055640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-06T03:03:19.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.627+0000 7f8acb055640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T03:03:19.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.627+0000 7f8acb055640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T03:03:19.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.627+0000 7f8acb055640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-06T03:03:19.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:19.627+0000 7f8acb055640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-06T03:03:22.625 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-06T03:03:22.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.767+0000 7fdf992b1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-06T03:03:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.771+0000 7fdf992b1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-06T03:03:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.771+0000 7fdf992b1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T03:03:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.771+0000 7fdf992b1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T03:03:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.771+0000 7fdf992b1640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-06T03:03:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:22.771+0000 7fdf992b1640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-06T03:03:25.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-06T03:03:25.829 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-06T03:03:25.843 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:25.844 INFO:tasks.ceph:role: osd.4 2023-11-06T03:03:25.844 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:03:25.844 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-06T03:03:25.897 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:25.898 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:25.898 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:25.898 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:25.898 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:25.899 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:25.899 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:25.899 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:25.899 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:25.899 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:25.921 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2023-11-06T03:03:25.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2023-11-06T03:03:25.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-06T03:03:25.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-06T03:03:26.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:26.035 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:26.036 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-06T03:03:26.089 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:26.089 INFO:tasks.ceph:role: osd.5 2023-11-06T03:03:26.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:03:26.090 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-06T03:03:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:26.148 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:26.148 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:26.149 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:26.149 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:26.149 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:26.170 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2023-11-06T03:03:26.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2023-11-06T03:03:26.172 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-06T03:03:26.229 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-06T03:03:26.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:26.280 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:26.280 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-06T03:03:26.337 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:26.337 INFO:tasks.ceph:role: osd.6 2023-11-06T03:03:26.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:03:26.338 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-06T03:03:26.390 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:26.391 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:26.391 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:26.391 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:26.392 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:26.413 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2023-11-06T03:03:26.414 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2023-11-06T03:03:26.415 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-06T03:03:26.472 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-06T03:03:26.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:26.524 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:26.524 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-06T03:03:26.577 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-06T03:03:26.577 INFO:tasks.ceph:role: osd.7 2023-11-06T03:03:26.578 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:03:26.578 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-06T03:03:26.628 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-06T03:03:26.629 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-06T03:03:26.629 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-06T03:03:26.629 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-06T03:03:26.629 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-06T03:03:26.630 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2023-11-06T03:03:26.630 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-06T03:03:26.630 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-06T03:03:26.630 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-06T03:03:26.630 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-06T03:03:26.654 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2023-11-06T03:03:26.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2023-11-06T03:03:26.655 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-06T03:03:26.712 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-06T03:03:26.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:03:26.765 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2023-11-06T03:03:26.766 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:26.882 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-06T03:03:26.882 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-06T03:03:26.883 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T03:03:26.883 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T03:03:26.884 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-06T03:03:26.884 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:26.880+0000 7f30e9353640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-06T03:03:29.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-06T03:03:29.895 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:29.977 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-06T03:03:29.977 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-06T03:03:29.977 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T03:03:29.978 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T03:03:29.978 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-06T03:03:29.978 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:29.976+0000 7faa6dac9640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-06T03:03:32.981 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-06T03:03:33.039 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:33.121 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-06T03:03:33.121 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-06T03:03:33.121 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T03:03:33.122 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T03:03:33.122 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-06T03:03:33.122 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:33.120+0000 7fdc6b6f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-06T03:03:36.113 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-06T03:03:36.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:36.251 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-06T03:03:36.252 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-06T03:03:36.252 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T03:03:36.252 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T03:03:36.252 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-06T03:03:36.253 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:36.252+0000 7fca2b6d7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-06T03:03:39.250 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-06T03:03:39.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:03:39.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:03:39.311 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-06T03:03:39.311 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-06T03:03:39.354 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.355 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-06T03:03:39.416 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.417 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-06T03:03:39.472 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-06T03:03:39.525 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.526 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-06T03:03:39.581 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-06T03:03:39.637 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:39.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-06T03:03:39.692 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-06T03:03:39.706 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-06T03:03:39.761 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-06T03:03:39.817 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.817 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-06T03:03:39.869 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.870 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-06T03:03:39.929 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.930 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-06T03:03:39.985 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:39.985 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-06T03:03:40.038 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:40.038 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-06T03:03:40.044 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:40.044 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-06T03:03:40.090 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:40.091 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-06T03:03:40.097 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:40.097 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-06T03:03:40.142 INFO:tasks.ceph:Adding keys to all mons... 2023-11-06T03:03:40.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T03:03:40.146 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-06T03:03:40.188 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2023-11-06T03:03:40.188 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q== 2023-11-06T03:03:40.189 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:03:40.189 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw== 2023-11-06T03:03:40.189 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:03:40.189 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA== 2023-11-06T03:03:40.190 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2023-11-06T03:03:40.190 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg== 2023-11-06T03:03:40.190 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2023-11-06T03:03:40.191 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA== 2023-11-06T03:03:40.191 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2023-11-06T03:03:40.191 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow== 2023-11-06T03:03:40.192 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2023-11-06T03:03:40.192 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ== 2023-11-06T03:03:40.192 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2023-11-06T03:03:40.192 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA== 2023-11-06T03:03:40.192 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:03:40.193 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA== 2023-11-06T03:03:40.193 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:03:40.193 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg== 2023-11-06T03:03:40.193 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2023-11-06T03:03:40.193 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g== 2023-11-06T03:03:40.194 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2023-11-06T03:03:40.194 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg== 2023-11-06T03:03:40.194 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2023-11-06T03:03:40.194 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A== 2023-11-06T03:03:40.194 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw== 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA== 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg== 2023-11-06T03:03:40.195 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2023-11-06T03:03:40.196 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg== 2023-11-06T03:03:40.196 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2023-11-06T03:03:40.196 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w== 2023-11-06T03:03:40.197 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2023-11-06T03:03:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q== 2023-11-06T03:03:40.197 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:03:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw== 2023-11-06T03:03:40.198 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:03:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA== 2023-11-06T03:03:40.198 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2023-11-06T03:03:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg== 2023-11-06T03:03:40.198 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2023-11-06T03:03:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA== 2023-11-06T03:03:40.199 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2023-11-06T03:03:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow== 2023-11-06T03:03:40.199 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2023-11-06T03:03:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ== 2023-11-06T03:03:40.200 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2023-11-06T03:03:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA== 2023-11-06T03:03:40.201 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:03:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA== 2023-11-06T03:03:40.201 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:03:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg== 2023-11-06T03:03:40.202 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2023-11-06T03:03:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g== 2023-11-06T03:03:40.203 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2023-11-06T03:03:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg== 2023-11-06T03:03:40.204 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2023-11-06T03:03:40.205 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A== 2023-11-06T03:03:40.205 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2023-11-06T03:03:40.205 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw== 2023-11-06T03:03:40.206 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2023-11-06T03:03:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA== 2023-11-06T03:03:40.206 INFO:teuthology.orchestra.run.smithi195.stdout:[client.2] 2023-11-06T03:03:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg== 2023-11-06T03:03:40.207 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2023-11-06T03:03:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg== 2023-11-06T03:03:40.208 INFO:teuthology.orchestra.run.smithi195.stdout:[client.3] 2023-11-06T03:03:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w== 2023-11-06T03:03:40.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T03:03:40.255 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T03:03:40.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.305 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.397 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.447 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.527 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.606 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.649 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.695 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:40.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T03:03:40.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-06T03:03:40.818 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.946 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-06T03:03:40.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.031 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.080 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.115 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.199 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.283 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-06T03:03:41.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.455 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.502 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.583 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.586 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-06T03:03:41.632 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-06T03:03:41.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-06T03:03:41.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T03:03:41.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-06T03:03:41.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-06T03:03:41.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T03:03:41.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-06T03:03:41.866 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-06T03:03:41.919 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-06T03:03:42.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-06T03:03:42.028 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:42.030 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-06T03:03:42.073 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-06T03:03:42.074 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-06T03:03:42.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-06T03:03:42.078 INFO:tasks.ceph.mon.a:Started 2023-11-06T03:03:42.079 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-06T03:03:42.079 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-06T03:03:42.119 INFO:tasks.ceph.mon.b:Started 2023-11-06T03:03:42.119 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-06T03:03:42.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-06T03:03:42.122 INFO:tasks.ceph.mon.c:Started 2023-11-06T03:03:42.122 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-06T03:03:42.123 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-06T03:03:42.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-06T03:03:42.126 INFO:tasks.ceph.mgr.y:Started 2023-11-06T03:03:42.126 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-06T03:03:42.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-06T03:03:42.128 INFO:tasks.ceph.mgr.x:Started 2023-11-06T03:03:42.129 INFO:tasks.ceph:Setting crush tunables to default 2023-11-06T03:03:42.129 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd crush tunables default 2023-11-06T03:03:42.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 Processor -- start 2023-11-06T03:03:42.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 -- start start 2023-11-06T03:03:42.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb018066320 con 0x7fb018065e70 2023-11-06T03:03:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180b2840 con 0x7fb01815a700 2023-11-06T03:03:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.247+0000 7fb01d799640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180b2020 con 0x7fb01815aae0 2023-11-06T03:03:42.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56450/0 (socket says 172.21.15.157:56450) 2023-11-06T03:03:42.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 learned_addr learned my addr 172.21.15.157:0/2609690373 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:03:42.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 msgr2=0x7fb01815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-06T03:03:42.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 msgr2=0x7fb01815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T03:03:42.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb016ffd640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 msgr2=0x7fb0180ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T03:03:42.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb016ffd640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 msgr2=0x7fb0180ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T03:03:42.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0167fc640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 msgr2=0x7fb018065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T03:03:42.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0167fc640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 msgr2=0x7fb018065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.251+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-06T03:03:42.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 msgr2=0x7fb01815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-06T03:03:42.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 msgr2=0x7fb01815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T03:03:42.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0167fc640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 msgr2=0x7fb018065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-06T03:03:42.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0167fc640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 msgr2=0x7fb018065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T03:03:42.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb016ffd640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 msgr2=0x7fb0180ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-06T03:03:42.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb016ffd640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 msgr2=0x7fb0180ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-06T03:03:42.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-06T03:03:42.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.455+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-06T03:03:42.692 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:42.691+0000 7f1f862f0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T03:03:42.724 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:42.724+0000 7f2af11b8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-06T03:03:42.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.157:3300/0 conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0167fc640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb016ffd640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 msgr2=0x7fb018065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 msgr2=0x7fb0180ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:42.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.859+0000 7fb0177fe640 1 -- 172.21.15.157:0/2609690373 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180cc220 con 0x7fb018065e70 2023-11-06T03:03:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.863+0000 7fb0177fe640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb018065e70 0x7fb01815d2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0180c9fe0 tx=0x7fb01001cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7794d24db8da935 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.863+0000 7fb015ffb640 1 -- 172.21.15.157:0/2609690373 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100a5cb0 con 0x7fb018065e70 2023-11-06T03:03:42.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.863+0000 7fb015ffb640 1 -- 172.21.15.157:0/2609690373 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0100a9070 con 0x7fb018065e70 2023-11-06T03:03:42.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.863+0000 7fb015ffb640 1 -- 172.21.15.157:0/2609690373 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100af620 con 0x7fb018065e70 2023-11-06T03:03:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- 172.21.15.157:0/2609690373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb018065e70 msgr2=0x7fb01815d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb018065e70 0x7fb01815d2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0180c9fe0 tx=0x7fb01001cd00 comp rx=0 tx=0).stop 2023-11-06T03:03:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- 172.21.15.157:0/2609690373 shutdown_connections 2023-11-06T03:03:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb018065e70 0x7fb01815d2e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 --2- 172.21.15.157:0/2609690373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815aae0 0x7fb018065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 --2- 172.21.15.157:0/2609690373 >> v2:172.21.15.195:3300/0 conn(0x7fb01815a700 0x7fb0180ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- 172.21.15.157:0/2609690373 >> 172.21.15.157:0/2609690373 conn(0x7fb01806a170 msgr2=0x7fb01806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- 172.21.15.157:0/2609690373 shutdown_connections 2023-11-06T03:03:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- 172.21.15.157:0/2609690373 wait complete. 2023-11-06T03:03:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 Processor -- start 2023-11-06T03:03:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.867+0000 7fb01d799640 1 -- start start 2023-11-06T03:03:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018065e70 0x7fb01814d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 0x7fb018157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0181585b0 con 0x7fb01815aae0 2023-11-06T03:03:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018158890 con 0x7fb018065e70 2023-11-06T03:03:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018158b70 con 0x7fb01815a700 2023-11-06T03:03:42.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56510/0 (socket says 172.21.15.157:56510) 2023-11-06T03:03:42.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb016ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018065e70 0x7fb01814d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 -- 172.21.15.157:0/1307239441 learned_addr learned my addr 172.21.15.157:0/1307239441 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:03:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0167fc640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 0x7fb018157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 -- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 msgr2=0x7fb018157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 0x7fb018157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 -- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018065e70 msgr2=0x7fb01814d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018065e70 0x7fb01814d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 -- 172.21.15.157:0/1307239441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100097e0 con 0x7fb01815aae0 2023-11-06T03:03:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0167fc640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 0x7fb018157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:03:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb0177fe640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb01001b3b0 tx=0x7fb01001b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100afbc0 con 0x7fb01815aae0 2023-11-06T03:03:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0100afd20 con 0x7fb01815aae0 2023-11-06T03:03:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb018158e50 con 0x7fb01815aae0 2023-11-06T03:03:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0100ba670 con 0x7fb01815aae0 2023-11-06T03:03:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0180b4110 con 0x7fb01815aae0 2023-11-06T03:03:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb0100baad0 con 0x7fb01815aae0 2023-11-06T03:03:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb0100c59a0 con 0x7fb01815aae0 2023-11-06T03:03:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.871+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018159300 con 0x7fb01815aae0 2023-11-06T03:03:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.875+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fb018159300 con 0x7fb01815aae0 2023-11-06T03:03:42.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.935+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb018158cb0 con 0x7fb01815aae0 2023-11-06T03:03:42.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fafff7fe640 1 -- 172.21.15.157:0/1307239441 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb018158cb0 con 0x7fb01815aae0 2023-11-06T03:03:42.942 INFO:teuthology.orchestra.run.smithi157.stderr:adjusted tunables profile to default 2023-11-06T03:03:42.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 msgr2=0x7fb01815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:42.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb01001b3b0 tx=0x7fb01001b540 comp rx=0 tx=0).stop 2023-11-06T03:03:42.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 shutdown_connections 2023-11-06T03:03:42.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb01815aae0 0x7fb01815c420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb01815a700 0x7fb018157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 --2- 172.21.15.157:0/1307239441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb018065e70 0x7fb01814d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:42.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 >> 172.21.15.157:0/1307239441 conn(0x7fb01806a170 msgr2=0x7fb01806cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:42.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 shutdown_connections 2023-11-06T03:03:42.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:42.943+0000 7fb01d799640 1 -- 172.21.15.157:0/1307239441 wait complete. 2023-11-06T03:03:42.959 INFO:tasks.ceph:check_enable_crimson: False 2023-11-06T03:03:42.960 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-06T03:03:42.960 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:42.960 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-06T03:03:42.972 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:42.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-06T03:03:43.023 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:43.024 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-06T03:03:43.076 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:03:43.077 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-06T03:03:43.129 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:43.129 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-06T03:03:43.143 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:43.144 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-06T03:03:43.197 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:43.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-06T03:03:43.218 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:43.219+0000 7f1f862f0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T03:03:43.239 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:43.240+0000 7f2af11b8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-06T03:03:43.250 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:03:43.250 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-06T03:03:43.304 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 39ef8ea4-08cc-4be6-a7ec-19244f68d5c5 0 2023-11-06T03:03:43.416 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:43.415+0000 7f1f862f0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T03:03:43.429 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:43.428+0000 7f2af11b8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-06T03:03:43.469 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.468+0000 7f1deb88b640 1 Processor -- start 2023-11-06T03:03:43.469 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.468+0000 7f1deb88b640 1 -- start start 2023-11-06T03:03:43.471 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1deb88b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.471 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1deb88b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 0x7f1de415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.471 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1deb88b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de415bff0 con 0x7f1de40cf090 2023-11-06T03:03:43.471 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1deb88b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de415aff0 con 0x7f1de40b1e40 2023-11-06T03:03:43.472 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1deb88b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de406dd80 con 0x7f1de40cc3d0 2023-11-06T03:03:43.472 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.472 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:53948/0 (socket says 172.21.15.195:53948) 2023-11-06T03:03:43.472 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 -- 172.21.15.195:0/1866715222 learned_addr learned my addr 172.21.15.195:0/1866715222 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:43.472 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de9e28640 1 --2- 172.21.15.195:0/1866715222 >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 0x7f1de415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.473 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 -- 172.21.15.195:0/1866715222 >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 msgr2=0x7f1de415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:43.473 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 --2- 172.21.15.195:0/1866715222 >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 0x7f1de415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.473 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 -- 172.21.15.195:0/1866715222 >> v1:172.21.15.157:6789/0 conn(0x7f1de40cf090 legacy=0x7f1de40b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:43.473 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 -- 172.21.15.195:0/1866715222 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de406e060 con 0x7f1de40cc3d0 2023-11-06T03:03:43.473 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de9e28640 1 --2- 172.21.15.195:0/1866715222 >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 0x7f1de415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:43.475 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1de8e26640 1 --2- 172.21.15.195:0/1866715222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009870 tx=0x7f1dd401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62319e264646d5f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.475 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1ddbfff640 1 -- 172.21.15.195:0/1866715222 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40aa670 con 0x7f1de40cc3d0 2023-11-06T03:03:43.475 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1ddbfff640 1 -- 172.21.15.195:0/1866715222 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd40aa7d0 con 0x7f1de40cc3d0 2023-11-06T03:03:43.476 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.472+0000 7f1ddbfff640 1 -- 172.21.15.195:0/1866715222 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40aab60 con 0x7f1de40cc3d0 2023-11-06T03:03:43.476 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/1866715222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 msgr2=0x7f1de4066340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.476 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- 172.21.15.195:0/1866715222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009870 tx=0x7f1dd401c510 comp rx=0 tx=0).stop 2023-11-06T03:03:43.476 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/1866715222 shutdown_connections 2023-11-06T03:03:43.476 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- 172.21.15.195:0/1866715222 >> v2:172.21.15.157:3300/0 conn(0x7f1de40b1e40 0x7f1de415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.477 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- 172.21.15.195:0/1866715222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cc3d0 0x7f1de4066340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.477 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/1866715222 >> 172.21.15.195:0/1866715222 conn(0x7f1de40732f0 msgr2=0x7f1de4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:43.477 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/1866715222 shutdown_connections 2023-11-06T03:03:43.477 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/1866715222 wait complete. 2023-11-06T03:03:43.477 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 Processor -- start 2023-11-06T03:03:43.478 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- start start 2023-11-06T03:03:43.480 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.480 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de40cc3d0 0x7f1de413e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.481 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 0x7f1de406c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.481 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de41547a0 con 0x7f1de40cc3d0 2023-11-06T03:03:43.481 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de4150810 con 0x7f1de40b1e40 2023-11-06T03:03:43.482 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de4150220 con 0x7f1de40cf090 2023-11-06T03:03:43.482 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9627640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.482 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 0x7f1de406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.482 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9627640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57794/0 (socket says 172.21.15.195:57794) 2023-11-06T03:03:43.483 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9627640 1 -- 172.21.15.195:0/21641606 learned_addr learned my addr 172.21.15.195:0/21641606 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:43.484 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de8e26640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de40cc3d0 0x7f1de413e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.484 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 -- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 msgr2=0x7f1de40a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.485 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.485 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 -- 172.21.15.195:0/21641606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de40cc3d0 msgr2=0x7f1de413e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.485 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de40cc3d0 0x7f1de413e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.485 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 -- 172.21.15.195:0/21641606 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40094d0 con 0x7f1de40cf090 2023-11-06T03:03:43.485 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9627640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:43.486 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1de9e28640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 0x7f1de406c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c590 tx=0x7f1de000ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.486 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011800 con 0x7f1de40cf090 2023-11-06T03:03:43.486 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0011980 con 0x7f1de40cf090 2023-11-06T03:03:43.486 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011d30 con 0x7f1de40cf090 2023-11-06T03:03:43.486 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de41549b0 con 0x7f1de40cf090 2023-11-06T03:03:43.487 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de41504a0 con 0x7f1de40cf090 2023-11-06T03:03:43.487 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1de001a450 con 0x7f1de40cf090 2023-11-06T03:03:43.487 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.476+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1de0022800 con 0x7f1de40cf090 2023-11-06T03:03:43.487 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.480+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de4072750 con 0x7f1de40cf090 2023-11-06T03:03:43.495 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.496+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1de001f060 con 0x7f1de40cf090 2023-11-06T03:03:43.562 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.564+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "39ef8ea4-08cc-4be6-a7ec-19244f68d5c5", "id": 0} v 0) v1 -- 0x7f1de4150950 con 0x7f1de40cf090 2023-11-06T03:03:43.569 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.568+0000 7f1dd9ffb640 1 -- 172.21.15.195:0/21641606 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "39ef8ea4-08cc-4be6-a7ec-19244f68d5c5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1de0015070 con 0x7f1de40cf090 2023-11-06T03:03:43.569 INFO:teuthology.orchestra.run.smithi195.stdout:0 2023-11-06T03:03:43.570 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 msgr2=0x7f1de406c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.571 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 0x7f1de406c430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c590 tx=0x7f1de000ca50 comp rx=0 tx=0).stop 2023-11-06T03:03:43.571 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 shutdown_connections 2023-11-06T03:03:43.571 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1de40cf090 0x7f1de406c430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.571 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1de40cc3d0 0x7f1de413e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.571 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 --2- 172.21.15.195:0/21641606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1de40b1e40 0x7f1de40a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.572 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 >> 172.21.15.195:0/21641606 conn(0x7f1de40732f0 msgr2=0x7f1de415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:43.572 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 shutdown_connections 2023-11-06T03:03:43.572 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.572+0000 7f1deb88b640 1 -- 172.21.15.195:0/21641606 wait complete. 2023-11-06T03:03:43.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 720566bd-81fd-4db1-8eba-61ea738b0325 1 2023-11-06T03:03:43.698 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 Processor -- start 2023-11-06T03:03:43.698 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 -- start start 2023-11-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 0x7f98ac0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac15b3d0 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac15b0c0 con 0x7f98ac0b1eb0 2023-11-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98b0f50640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac155610 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98aad9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 0x7f98ac0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a9d9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f98ac0b1eb0 0x7f98ac15b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:40848/0 (socket says 172.21.15.195:40848) 2023-11-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a9d9b640 1 -- 172.21.15.195:0/2526290116 learned_addr learned my addr 172.21.15.195:0/2526290116 (peer_addr_for_me v1:172.21.15.195:0/0) 2023-11-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98aad9d640 1 -- 172.21.15.195:0/2526290116 >> v1:172.21.15.195:6789/0 conn(0x7f98ac0b1eb0 legacy=0x7f98ac15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98aad9d640 1 -- 172.21.15.195:0/2526290116 >> v1:172.21.15.157:6789/0 conn(0x7f98ac0b1ad0 legacy=0x7f98ac0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a959a640 1 -- 172.21.15.195:0/2526290116 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120770495 0 0) 0x7f98ac15b0c0 con 0x7f98ac0b1eb0 2023-11-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98aad9d640 1 -- 172.21.15.195:0/2526290116 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0b50c0 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.702 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98aad9d640 1 --2- 172.21.15.195:0/2526290116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 0x7f98ac0cc180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009dc0 tx=0x7f98a401c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e31b212687db3f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.703 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a959a640 1 -- 172.21.15.195:0/2526290116 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a40a57f0 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.703 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a959a640 1 -- 172.21.15.195:0/2526290116 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a40a5950 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.703 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.700+0000 7f98a959a640 1 -- 172.21.15.195:0/2526290116 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a40a5ce0 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.703 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/2526290116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 msgr2=0x7f98ac0cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.703 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 --2- 172.21.15.195:0/2526290116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 0x7f98ac0cc180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009dc0 tx=0x7f98a401c930 comp rx=0 tx=0).stop 2023-11-06T03:03:43.704 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/2526290116 shutdown_connections 2023-11-06T03:03:43.704 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 --2- 172.21.15.195:0/2526290116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b4ac0 0x7f98ac0cc180 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.704 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/2526290116 >> 172.21.15.195:0/2526290116 conn(0x7f98ac06a170 msgr2=0x7f98ac06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:43.704 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/2526290116 shutdown_connections 2023-11-06T03:03:43.705 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/2526290116 wait complete. 2023-11-06T03:03:43.707 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 Processor -- start 2023-11-06T03:03:43.707 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- start start 2023-11-06T03:03:43.707 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.707 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98ac0b1eb0 0x7f98ac14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.707 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac0b4ac0 0x7f98ac15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14fbf0 con 0x7f98ac0b1eb0 2023-11-06T03:03:43.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14d2c0 con 0x7f98ac0b4ac0 2023-11-06T03:03:43.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14fe20 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:53986/0 (socket says 172.21.15.195:53986) 2023-11-06T03:03:43.709 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aad9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac0b4ac0 0x7f98ac15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.709 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 -- 172.21.15.195:0/587589882 learned_addr learned my addr 172.21.15.195:0/587589882 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:43.709 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 -- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac0b4ac0 msgr2=0x7f98ac15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.709 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac0b4ac0 0x7f98ac15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.709 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 -- 172.21.15.195:0/587589882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98ac0b1eb0 msgr2=0x7f98ac14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.710 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98ac0b1eb0 0x7f98ac14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.710 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 -- 172.21.15.195:0/587589882 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4009a70 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.710 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98aa59c640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c490 tx=0x7f989c001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.710 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003600 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.710 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c003780 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.711 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac150100 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.711 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003b70 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.711 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac1506c0 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.711 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f989c01c440 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.711 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.704+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f989c012910 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.712 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.708+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac072750 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.712 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.708+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f989c017070 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.775 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.776+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "720566bd-81fd-4db1-8eba-61ea738b0325", "id": 1} v 0) v1 -- 0x7f98ac15de70 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.782 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f989affd640 1 -- 172.21.15.195:0/587589882 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "720566bd-81fd-4db1-8eba-61ea738b0325", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f989c012ba0 con 0x7f98ac0b1ad0 2023-11-06T03:03:43.782 INFO:teuthology.orchestra.run.smithi195.stdout:1 2023-11-06T03:03:43.784 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 msgr2=0x7f98ac14a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.784 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c490 tx=0x7f989c001e80 comp rx=0 tx=0).stop 2023-11-06T03:03:43.784 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 shutdown_connections 2023-11-06T03:03:43.784 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98ac0b4ac0 0x7f98ac15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.785 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98ac0b1eb0 0x7f98ac14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.785 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 --2- 172.21.15.195:0/587589882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98ac0b1ad0 0x7f98ac14a7b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.785 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 >> 172.21.15.195:0/587589882 conn(0x7f98ac06a170 msgr2=0x7f98ac15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:43.785 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 shutdown_connections 2023-11-06T03:03:43.786 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.784+0000 7f98b0f50640 1 -- 172.21.15.195:0/587589882 wait complete. 2023-11-06T03:03:43.803 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 530cb9e2-3d91-43f6-87fe-af5789a292e6 2 2023-11-06T03:03:43.921 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.920+0000 7fef345ca640 1 Processor -- start 2023-11-06T03:03:43.921 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.920+0000 7fef345ca640 1 -- start start 2023-11-06T03:03:43.923 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.923 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef2c0713c0 0x7fef2c0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.923 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 0x7fef2c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c06b8d0 con 0x7fef2c0713c0 2023-11-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c06be90 con 0x7fef2c15adc0 2023-11-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef345ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c071880 con 0x7fef2c0ca570 2023-11-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32366640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 0x7fef2c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32366640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57820/0 (socket says 172.21.15.195:57820) 2023-11-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32366640 1 -- 172.21.15.195:0/3096827549 learned_addr learned my addr 172.21.15.195:0/3096827549 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef31b65640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.157:3300/0 conn(0x7fef2c0713c0 0x7fef2c0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 -- 172.21.15.195:0/3096827549 >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 msgr2=0x7fef2c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:43.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 -- 172.21.15.195:0/3096827549 >> v2:172.21.15.157:3300/0 conn(0x7fef2c0713c0 msgr2=0x7fef2c0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:43.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.157:3300/0 conn(0x7fef2c0713c0 0x7fef2c0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 -- 172.21.15.195:0/3096827549 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c071b60 con 0x7fef2c0ca570 2023-11-06T03:03:43.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32366640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:43.930 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef32b67640 1 --2- 172.21.15.195:0/3096827549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 0x7fef2c0b4ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0ced80 tx=0x7fef2401c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfe590cb319c20 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.930 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef31364640 1 -- 172.21.15.195:0/3096827549 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240a5860 con 0x7fef2c0ca570 2023-11-06T03:03:43.930 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef31364640 1 -- 172.21.15.195:0/3096827549 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef240a59c0 con 0x7fef2c0ca570 2023-11-06T03:03:43.930 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.924+0000 7fef31364640 1 -- 172.21.15.195:0/3096827549 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef240a5d50 con 0x7fef2c0ca570 2023-11-06T03:03:43.930 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/3096827549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 msgr2=0x7fef2c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.931 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- 172.21.15.195:0/3096827549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 0x7fef2c0b4ab0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0ced80 tx=0x7fef2401c7c0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.931 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/3096827549 shutdown_connections 2023-11-06T03:03:43.931 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- 172.21.15.195:0/3096827549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0ca570 0x7fef2c0b4ab0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.931 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.157:3300/0 conn(0x7fef2c0713c0 0x7fef2c0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.932 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- 172.21.15.195:0/3096827549 >> v2:172.21.15.195:3300/0 conn(0x7fef2c15adc0 0x7fef2c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.932 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/3096827549 >> 172.21.15.195:0/3096827549 conn(0x7fef2c0cb990 msgr2=0x7fef2c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:43.932 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/3096827549 shutdown_connections 2023-11-06T03:03:43.932 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/3096827549 wait complete. 2023-11-06T03:03:43.932 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 Processor -- start 2023-11-06T03:03:43.933 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- start start 2023-11-06T03:03:43.933 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.933 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 0x7fef2c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.933 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2c15adc0 0x7fef2c0c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:43.934 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c153930 con 0x7fef2c15adc0 2023-11-06T03:03:43.934 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1529a0 con 0x7fef2c0ca570 2023-11-06T03:03:43.934 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0c1d50 con 0x7fef2c0713c0 2023-11-06T03:03:43.934 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 0x7fef2c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.935 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef32366640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.935 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef32366640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:53998/0 (socket says 172.21.15.195:53998) 2023-11-06T03:03:43.935 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef32366640 1 -- 172.21.15.195:0/882547313 learned_addr learned my addr 172.21.15.195:0/882547313 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:43.935 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef32b67640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2c15adc0 0x7fef2c0c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:43.935 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 -- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 msgr2=0x7fef2c145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.936 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.936 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 -- 172.21.15.195:0/882547313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2c15adc0 msgr2=0x7fef2c0c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:43.936 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2c15adc0 0x7fef2c0c0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:43.936 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 -- 172.21.15.195:0/882547313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200094d0 con 0x7fef2c0ca570 2023-11-06T03:03:43.937 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef32366640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:43.937 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef31b65640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 0x7fef2c1533f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2800d5f0 tx=0x7fef2800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:43.937 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28014620 con 0x7fef2c0ca570 2023-11-06T03:03:43.937 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef28014780 con 0x7fef2c0ca570 2023-11-06T03:03:43.937 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef24009a70 con 0x7fef2c0ca570 2023-11-06T03:03:43.938 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28014b10 con 0x7fef2c0ca570 2023-11-06T03:03:43.938 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c152bf0 con 0x7fef2c0ca570 2023-11-06T03:03:43.938 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fef2801d400 con 0x7fef2c0ca570 2023-11-06T03:03:43.938 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.928+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fef28012940 con 0x7fef2c0ca570 2023-11-06T03:03:43.939 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.932+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c072750 con 0x7fef2c0ca570 2023-11-06T03:03:43.939 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:43.932+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fef2803cb10 con 0x7fef2c0ca570 2023-11-06T03:03:44.001 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.000+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "530cb9e2-3d91-43f6-87fe-af5789a292e6", "id": 2} v 0) v1 -- 0x7fef2c06ce30 con 0x7fef2c0ca570 2023-11-06T03:03:44.007 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.007+0000 7f1f862f0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T03:03:44.009 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.008+0000 7fef1effd640 1 -- 172.21.15.195:0/882547313 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "530cb9e2-3d91-43f6-87fe-af5789a292e6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef2802b070 con 0x7fef2c0ca570 2023-11-06T03:03:44.009 INFO:teuthology.orchestra.run.smithi195.stdout:2 2023-11-06T03:03:44.010 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 msgr2=0x7fef2c1533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.011 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 0x7fef2c1533f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2800d5f0 tx=0x7fef2800dab0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.011 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 shutdown_connections 2023-11-06T03:03:44.011 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef2c15adc0 0x7fef2c0c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.011 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef2c0ca570 0x7fef2c1533f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.012 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 --2- 172.21.15.195:0/882547313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef2c0713c0 0x7fef2c145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.012 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 >> 172.21.15.195:0/882547313 conn(0x7fef2c0cb990 msgr2=0x7fef2c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.012 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 shutdown_connections 2023-11-06T03:03:44.012 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.012+0000 7fef345ca640 1 -- 172.21.15.195:0/882547313 wait complete. 2023-11-06T03:03:44.026 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 4c2c4aaf-be58-419f-ae34-a13d3aafdf16 3 2023-11-06T03:03:44.049 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.048+0000 7f2af11b8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-06T03:03:44.061 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.063+0000 7f1f862f0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T03:03:44.105 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.104+0000 7f2af11b8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-06T03:03:44.113 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.111+0000 7f1f862f0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T03:03:44.144 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 Processor -- start 2023-11-06T03:03:44.144 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 -- start start 2023-11-06T03:03:44.146 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 0x7f8bc4065f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.146 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 0x7f8bc40b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.146 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc406d9f0 con 0x7f8bc415a940 2023-11-06T03:03:44.146 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40cc6f0 con 0x7f8bc40cc240 2023-11-06T03:03:44.147 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bcc010640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc406e200 con 0x7f8bc40664c0 2023-11-06T03:03:44.147 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bc95ab640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 0x7f8bc4065f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.147 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bc9dac640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f8bc415a940 0x7f8bc4070de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:40862/0 (socket says 172.21.15.195:40862) 2023-11-06T03:03:44.147 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bc9dac640 1 -- 172.21.15.195:0/2561539690 learned_addr learned my addr 172.21.15.195:0/2561539690 (peer_addr_for_me v1:172.21.15.195:0/0) 2023-11-06T03:03:44.147 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.144+0000 7f8bca5ad640 1 --2- 172.21.15.195:0/2561539690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 0x7f8bc40b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.148 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc8daa640 1 -- 172.21.15.195:0/2561539690 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003862808 0 0) 0x7f8bc406d9f0 con 0x7f8bc415a940 2023-11-06T03:03:44.148 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc8daa640 1 -- 172.21.15.195:0/2561539690 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb0003530 con 0x7f8bc415a940 2023-11-06T03:03:44.148 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bca5ad640 1 -- 172.21.15.195:0/2561539690 >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 msgr2=0x7f8bc4065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:44.148 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bca5ad640 1 --2- 172.21.15.195:0/2561539690 >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 0x7f8bc4065f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.149 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bca5ad640 1 -- 172.21.15.195:0/2561539690 >> v1:172.21.15.195:6789/0 conn(0x7f8bc415a940 legacy=0x7f8bc4070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.149 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bca5ad640 1 -- 172.21.15.195:0/2561539690 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc406dc40 con 0x7f8bc40664c0 2023-11-06T03:03:44.149 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc95ab640 1 --2- 172.21.15.195:0/2561539690 >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 0x7f8bc4065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.152 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bca5ad640 1 --2- 172.21.15.195:0/2561539690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 0x7f8bc40b1730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00097c0 tx=0x7f8bc001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83205a8d75eee93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.152 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc8daa640 1 -- 172.21.15.195:0/2561539690 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00ab600 con 0x7f8bc40664c0 2023-11-06T03:03:44.153 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc8daa640 1 -- 172.21.15.195:0/2561539690 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc00a7050 con 0x7f8bc40664c0 2023-11-06T03:03:44.153 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bc8daa640 1 -- 172.21.15.195:0/2561539690 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00ab980 con 0x7f8bc40664c0 2023-11-06T03:03:44.153 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 -- 172.21.15.195:0/2561539690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 msgr2=0x7f8bc40b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.153 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 --2- 172.21.15.195:0/2561539690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 0x7f8bc40b1730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00097c0 tx=0x7f8bc001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.154 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 -- 172.21.15.195:0/2561539690 shutdown_connections 2023-11-06T03:03:44.154 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 --2- 172.21.15.195:0/2561539690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40664c0 0x7f8bc40b1730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.154 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 --2- 172.21.15.195:0/2561539690 >> v2:172.21.15.195:3300/0 conn(0x7f8bc40cc240 0x7f8bc4065f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.154 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 -- 172.21.15.195:0/2561539690 >> 172.21.15.195:0/2561539690 conn(0x7f8bc40732f0 msgr2=0x7f8bc4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.154 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 -- 172.21.15.195:0/2561539690 shutdown_connections 2023-11-06T03:03:44.155 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.148+0000 7f8bcc010640 1 -- 172.21.15.195:0/2561539690 wait complete. 2023-11-06T03:03:44.155 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 Processor -- start 2023-11-06T03:03:44.155 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- start start 2023-11-06T03:03:44.155 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.156 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc415a940 0x7f8bc40ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.156 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 0x7f8bc40cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.156 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b2630 con 0x7f8bc415a940 2023-11-06T03:03:44.156 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b2910 con 0x7f8bc40b39b0 2023-11-06T03:03:44.157 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40b2bf0 con 0x7f8bc40cc240 2023-11-06T03:03:44.157 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.157 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54008/0 (socket says 172.21.15.195:54008) 2023-11-06T03:03:44.157 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bca5ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 0x7f8bc40cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.157 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 -- 172.21.15.195:0/1049213874 learned_addr learned my addr 172.21.15.195:0/1049213874 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.158 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc95ab640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc415a940 0x7f8bc40ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.158 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 -- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 msgr2=0x7f8bc40cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.159 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 0x7f8bc40cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.159 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 -- 172.21.15.195:0/1049213874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc415a940 msgr2=0x7f8bc40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.159 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc415a940 0x7f8bc40ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.159 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 -- 172.21.15.195:0/1049213874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4009420 con 0x7f8bc40cc240 2023-11-06T03:03:44.160 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bca5ad640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 0x7f8bc40cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.160 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bc9dac640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00cbc0 tx=0x7f8bbc00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.161 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00ad40 con 0x7f8bc40cc240 2023-11-06T03:03:44.161 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc012400 con 0x7f8bc40cc240 2023-11-06T03:03:44.161 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc0009420 con 0x7f8bc40cc240 2023-11-06T03:03:44.161 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00aea0 con 0x7f8bc40cc240 2023-11-06T03:03:44.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc40b31b0 con 0x7f8bc40cc240 2023-11-06T03:03:44.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8bbc012a90 con 0x7f8bc40cc240 2023-11-06T03:03:44.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f8bbc007500 con 0x7f8bc40cc240 2023-11-06T03:03:44.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.152+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4072750 con 0x7f8bc40cc240 2023-11-06T03:03:44.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.156+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f8bbc017070 con 0x7f8bc40cc240 2023-11-06T03:03:44.164 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.160+0000 7f2af11b8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-06T03:03:44.189 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.191+0000 7f1f862f0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T03:03:44.223 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.224+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4c2c4aaf-be58-419f-ae34-a13d3aafdf16", "id": 3} v 0) v1 -- 0x7f8bc406a6c0 con 0x7f8bc40cc240 2023-11-06T03:03:44.230 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bba7fc640 1 -- 172.21.15.195:0/1049213874 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c2c4aaf-be58-419f-ae34-a13d3aafdf16", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8bbc007140 con 0x7f8bc40cc240 2023-11-06T03:03:44.231 INFO:teuthology.orchestra.run.smithi195.stdout:3 2023-11-06T03:03:44.232 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 msgr2=0x7f8bc4163db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.232 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00cbc0 tx=0x7f8bbc00a430 comp rx=0 tx=0).stop 2023-11-06T03:03:44.232 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 shutdown_connections 2023-11-06T03:03:44.233 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8bc40b39b0 0x7f8bc40cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.233 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bc415a940 0x7f8bc40ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.233 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 --2- 172.21.15.195:0/1049213874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8bc40cc240 0x7f8bc4163db0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.233 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 >> 172.21.15.195:0/1049213874 conn(0x7f8bc40732f0 msgr2=0x7f8bc4165000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.234 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 shutdown_connections 2023-11-06T03:03:44.234 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.232+0000 7f8bcc010640 1 -- 172.21.15.195:0/1049213874 wait complete. 2023-11-06T03:03:44.239 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.240+0000 7f2af11b8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-06T03:03:44.242 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.243+0000 7f1f862f0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T03:03:44.247 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f4d19f9a-a162-4f54-aee2-b326475860e9 4 2023-11-06T03:03:44.292 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.292+0000 7f2af11b8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-06T03:03:44.319 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.319+0000 7f1f862f0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T03:03:44.371 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.372+0000 7f2af11b8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-06T03:03:44.373 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.375+0000 7f1f862f0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T03:03:44.410 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 Processor -- start 2023-11-06T03:03:44.410 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 -- start start 2023-11-06T03:03:44.411 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.412 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.412 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc06d9f0 con 0x7ff0fc070de0 2023-11-06T03:03:44.412 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc06e200 con 0x7ff0fc0b1730 2023-11-06T03:03:44.412 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff10311c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b2090 con 0x7ff0fc15a940 2023-11-06T03:03:44.413 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff100eb8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.413 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.413 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff100eb8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54020/0 (socket says 172.21.15.195:54020) 2023-11-06T03:03:44.413 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57864/0 (socket says 172.21.15.195:57864) 2023-11-06T03:03:44.413 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff100eb8640 1 -- 172.21.15.195:0/1542708434 learned_addr learned my addr 172.21.15.195:0/1542708434 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.414 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 -- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 msgr2=0x7ff0fc0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:44.414 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.414 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 -- 172.21.15.195:0/1542708434 >> v1:172.21.15.195:6789/0 conn(0x7ff0fc070de0 legacy=0x7ff0fc065f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.414 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff1016b9640 1 -- 172.21.15.195:0/1542708434 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc066690 con 0x7ff0fc0b1730 2023-11-06T03:03:44.414 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.412+0000 7ff100eb8640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.416 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff1016b9640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5be570f8fbf410 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.416 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0fb7fe640 1 -- 172.21.15.195:0/1542708434 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40a5950 con 0x7ff0fc0b1730 2023-11-06T03:03:44.416 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0fb7fe640 1 -- 172.21.15.195:0/1542708434 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f40a5ab0 con 0x7ff0fc0b1730 2023-11-06T03:03:44.417 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0fb7fe640 1 -- 172.21.15.195:0/1542708434 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40a5e40 con 0x7ff0fc0b1730 2023-11-06T03:03:44.417 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc0b1730 msgr2=0x7ff0fc0b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.417 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f401c890 comp rx=0 tx=0).stop 2023-11-06T03:03:44.417 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/1542708434 shutdown_connections 2023-11-06T03:03:44.417 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc0b1730 0x7ff0fc0b1b50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.418 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- 172.21.15.195:0/1542708434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc15a940 0x7ff0fc0cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.418 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/1542708434 >> 172.21.15.195:0/1542708434 conn(0x7ff0fc0732f0 msgr2=0x7ff0fc069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.418 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/1542708434 shutdown_connections 2023-11-06T03:03:44.418 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/1542708434 wait complete. 2023-11-06T03:03:44.418 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 Processor -- start 2023-11-06T03:03:44.419 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- start start 2023-11-06T03:03:44.421 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.421 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 0x7ff0fc14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.421 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0fc15a940 0x7ff0fc156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.421 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc14e180 con 0x7ff0fc15a940 2023-11-06T03:03:44.422 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc14d530 con 0x7ff0fc070de0 2023-11-06T03:03:44.422 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc155bf0 con 0x7ff0fc0b1730 2023-11-06T03:03:44.422 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 0x7ff0fc14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.422 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.422 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57880/0 (socket says 172.21.15.195:57880) 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 -- 172.21.15.195:0/29864216 learned_addr learned my addr 172.21.15.195:0/29864216 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 0x7ff0fc14cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54024/0 (socket says 172.21.15.195:54024) 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 -- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 msgr2=0x7ff0fc14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 0x7ff0fc14cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff1016b9640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0fc15a940 0x7ff0fc156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.423 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 -- 172.21.15.195:0/29864216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0fc15a940 msgr2=0x7ff0fc156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.424 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0fc15a940 0x7ff0fc156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.424 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 -- 172.21.15.195:0/29864216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00094d0 con 0x7ff0fc070de0 2023-11-06T03:03:44.424 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff100eb8640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e930 tx=0x7ff0f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.424 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000ca00 con 0x7ff0fc070de0 2023-11-06T03:03:44.424 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f000cb60 con 0x7ff0fc070de0 2023-11-06T03:03:44.425 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f4009a70 con 0x7ff0fc070de0 2023-11-06T03:03:44.425 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0013470 con 0x7ff0fc070de0 2023-11-06T03:03:44.425 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.416+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1561b0 con 0x7ff0fc070de0 2023-11-06T03:03:44.425 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.420+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff0f0005ca0 con 0x7ff0fc070de0 2023-11-06T03:03:44.425 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.420+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff0f0006840 con 0x7ff0fc070de0 2023-11-06T03:03:44.426 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.420+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc072750 con 0x7ff0fc070de0 2023-11-06T03:03:44.426 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.420+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff0f0021060 con 0x7ff0fc070de0 2023-11-06T03:03:44.426 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.428+0000 7f2af11b8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-06T03:03:44.489 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.488+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4d19f9a-a162-4f54-aee2-b326475860e9", "id": 4} v 0) v1 -- 0x7ff0fc06a9a0 con 0x7ff0fc070de0 2023-11-06T03:03:44.498 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff0f97fa640 1 -- 172.21.15.195:0/29864216 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4d19f9a-a162-4f54-aee2-b326475860e9", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff0f002a0b0 con 0x7ff0fc070de0 2023-11-06T03:03:44.498 INFO:teuthology.orchestra.run.smithi195.stdout:4 2023-11-06T03:03:44.500 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 msgr2=0x7ff0fc14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.500 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e930 tx=0x7ff0f000edf0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.500 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 shutdown_connections 2023-11-06T03:03:44.500 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0fc15a940 0x7ff0fc156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.500 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0fc0b1730 0x7ff0fc14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.501 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 --2- 172.21.15.195:0/29864216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0fc070de0 0x7ff0fc14e610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.501 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 >> 172.21.15.195:0/29864216 conn(0x7ff0fc0732f0 msgr2=0x7ff0fc0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.501 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 shutdown_connections 2023-11-06T03:03:44.501 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.500+0000 7ff10311c640 1 -- 172.21.15.195:0/29864216 wait complete. 2023-11-06T03:03:44.515 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.515+0000 7f1f862f0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T03:03:44.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 2822a15c-dc53-4d42-9d13-2b2b0d914eef 5 2023-11-06T03:03:44.572 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.572+0000 7f2af11b8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-06T03:03:44.575 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:44.575+0000 7f1f862f0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T03:03:44.629 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.628+0000 7fcf21d46640 1 Processor -- start 2023-11-06T03:03:44.630 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.628+0000 7fcf21d46640 1 -- start start 2023-11-06T03:03:44.631 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf21d46640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf1c0cca40 0x7fcf1c1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.631 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf21d46640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c0caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.631 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf21d46640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c161020 con 0x7fcf1c15ce60 2023-11-06T03:03:44.631 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf21d46640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c072020 con 0x7fcf1c0cca40 2023-11-06T03:03:44.632 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf21d46640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c072300 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.632 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf1c0cca40 0x7fcf1c1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.632 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf1c0cca40 0x7fcf1c1614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57894/0 (socket says 172.21.15.195:57894) 2023-11-06T03:03:44.632 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c0caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.633 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1b7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fcf1c15ce60 0x7fcf1c0cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:40872/0 (socket says 172.21.15.195:40872) 2023-11-06T03:03:44.633 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 -- 172.21.15.195:0/376545926 learned_addr learned my addr 172.21.15.195:0/376545926 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.633 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 -- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 msgr2=0x7fcf1c0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:44.633 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c0caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.633 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 -- 172.21.15.195:0/376545926 >> v1:172.21.15.195:6789/0 conn(0x7fcf1c15ce60 legacy=0x7fcf1c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.634 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1affd640 1 -- 172.21.15.195:0/376545926 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c1609d0 con 0x7fcf1c0cca40 2023-11-06T03:03:44.634 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1a7fc640 1 -- 172.21.15.195:0/376545926 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390144593 0 0) 0x7fcf1c161020 con 0x7fcf1c15ce60 2023-11-06T03:03:44.634 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.632+0000 7fcf1bfff640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.635 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c1614b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009870 tx=0x7fcf1001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545242fef81137cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.635 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1a7fc640 1 -- 172.21.15.195:0/376545926 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100ab600 con 0x7fcf1c0cca40 2023-11-06T03:03:44.636 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1a7fc640 1 -- 172.21.15.195:0/376545926 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100a7050 con 0x7fcf1c0cca40 2023-11-06T03:03:44.636 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1a7fc640 1 -- 172.21.15.195:0/376545926 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100ab980 con 0x7fcf1c0cca40 2023-11-06T03:03:44.636 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c0cca40 msgr2=0x7fcf1c1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.636 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c1614b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009870 tx=0x7fcf1001c480 comp rx=0 tx=0).stop 2023-11-06T03:03:44.636 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- 172.21.15.195:0/376545926 shutdown_connections 2023-11-06T03:03:44.637 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c0caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.637 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- 172.21.15.195:0/376545926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c1614b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.637 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- 172.21.15.195:0/376545926 >> 172.21.15.195:0/376545926 conn(0x7fcf1c06d3d0 msgr2=0x7fcf1c06d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.637 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- 172.21.15.195:0/376545926 shutdown_connections 2023-11-06T03:03:44.637 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- 172.21.15.195:0/376545926 wait complete. 2023-11-06T03:03:44.638 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:44.636+0000 7f2af11b8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-06T03:03:44.640 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 Processor -- start 2023-11-06T03:03:44.640 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- start start 2023-11-06T03:03:44.640 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.640 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.641 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c15ce60 0x7fcf1c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.641 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0acbf0 con 0x7fcf1c0cca40 2023-11-06T03:03:44.641 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0af0f0 con 0x7fcf1c15ce60 2023-11-06T03:03:44.641 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf21d46640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0af320 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.641 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.642 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c15ce60 0x7fcf1c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.642 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54036/0 (socket says 172.21.15.195:54036) 2023-11-06T03:03:44.642 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 -- 172.21.15.195:0/4157548197 learned_addr learned my addr 172.21.15.195:0/4157548197 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.642 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1b7fe640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.642 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 -- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c15ce60 msgr2=0x7fcf1c0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.643 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c15ce60 0x7fcf1c0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.643 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 -- 172.21.15.195:0/4157548197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 msgr2=0x7fcf1c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.643 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c0aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.643 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1affd640 1 -- 172.21.15.195:0/4157548197 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14009a70 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.643 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.636+0000 7fcf1b7fe640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:44.644 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf1affd640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf100ae040 tx=0x7fcf10019220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.644 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100b4420 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.644 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100b4580 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.644 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf100094d0 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.644 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100b4930 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.645 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c15d5f0 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.645 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcf100b2070 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.645 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fcf100b33c0 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.645 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c0af840 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.646 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.640+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fcf1c0af840 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.708 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.708+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2822a15c-dc53-4d42-9d13-2b2b0d914eef", "id": 5} v 0) v1 -- 0x7fcf1c15d450 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.716 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf03fff640 1 -- 172.21.15.195:0/4157548197 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2822a15c-dc53-4d42-9d13-2b2b0d914eef", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf1c15d450 con 0x7fcf1c0d00c0 2023-11-06T03:03:44.716 INFO:teuthology.orchestra.run.smithi195.stdout:5 2023-11-06T03:03:44.717 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 msgr2=0x7fcf1c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.717 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf100ae040 tx=0x7fcf10019220 comp rx=0 tx=0).stop 2023-11-06T03:03:44.717 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 shutdown_connections 2023-11-06T03:03:44.718 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf1c15ce60 0x7fcf1c0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.718 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf1c0d00c0 0x7fcf1c154170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.718 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 --2- 172.21.15.195:0/4157548197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf1c0cca40 0x7fcf1c0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.718 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 >> 172.21.15.195:0/4157548197 conn(0x7fcf1c06d3d0 msgr2=0x7fcf1c0c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.718 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 shutdown_connections 2023-11-06T03:03:44.719 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.716+0000 7fcf21d46640 1 -- 172.21.15.195:0/4157548197 wait complete. 2023-11-06T03:03:44.733 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 41bae808-491c-4ce9-9f1a-22c974778030 6 2023-11-06T03:03:44.846 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.844+0000 7f08bf0da640 1 Processor -- start 2023-11-06T03:03:44.847 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.844+0000 7f08bf0da640 1 -- start start 2023-11-06T03:03:44.847 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.847 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.847 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08b80ce1a0 0x7f08b80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80d2920 con 0x7f08b80ce1a0 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80d18a0 con 0x7f08b815ce80 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80d1b60 con 0x7f08b815caa0 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.848 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54044/0 (socket says 172.21.15.195:54044) 2023-11-06T03:03:44.849 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57920/0 (socket says 172.21.15.195:57920) 2023-11-06T03:03:44.849 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 -- 172.21.15.195:0/3980815072 learned_addr learned my addr 172.21.15.195:0/3980815072 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.849 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bd677640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.157:3300/0 conn(0x7f08b80ce1a0 0x7f08b80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.849 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 -- 172.21.15.195:0/3980815072 >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 msgr2=0x7f08b80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:44.849 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 -- 172.21.15.195:0/3980815072 >> v2:172.21.15.157:3300/0 conn(0x7f08b80ce1a0 msgr2=0x7f08b80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.157:3300/0 conn(0x7f08b80ce1a0 0x7f08b80d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 -- 172.21.15.195:0/3980815072 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80d1e40 con 0x7f08b815caa0 2023-11-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b7fff640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.854 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bce76640 1 --2- 172.21.15.195:0/3980815072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f08b80d3cb0 tx=0x7f08ac01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b4355f536ac730 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.854 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b77fe640 1 -- 172.21.15.195:0/3980815072 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0aa5f0 con 0x7f08b815caa0 2023-11-06T03:03:44.854 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b77fe640 1 -- 172.21.15.195:0/3980815072 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac0aa750 con 0x7f08b815caa0 2023-11-06T03:03:44.854 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08b77fe640 1 -- 172.21.15.195:0/3980815072 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0aaae0 con 0x7f08b815caa0 2023-11-06T03:03:44.854 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- 172.21.15.195:0/3980815072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 msgr2=0x7f08b80cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.855 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- 172.21.15.195:0/3980815072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f08b80d3cb0 tx=0x7f08ac01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.855 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- 172.21.15.195:0/3980815072 shutdown_connections 2023-11-06T03:03:44.855 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.157:3300/0 conn(0x7f08b80ce1a0 0x7f08b80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.855 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- 172.21.15.195:0/3980815072 >> v2:172.21.15.195:3300/0 conn(0x7f08b815ce80 0x7f08b80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.856 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 --2- 172.21.15.195:0/3980815072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815caa0 0x7f08b80cdc60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.856 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- 172.21.15.195:0/3980815072 >> 172.21.15.195:0/3980815072 conn(0x7f08b8070790 msgr2=0x7f08b8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.856 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- 172.21.15.195:0/3980815072 shutdown_connections 2023-11-06T03:03:44.856 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.848+0000 7f08bf0da640 1 -- 172.21.15.195:0/3980815072 wait complete. 2023-11-06T03:03:44.856 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 Processor -- start 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- start start 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 0x7f08b81501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8157ad0 con 0x7f08b815caa0 2023-11-06T03:03:44.857 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c4f30 con 0x7f08b80ce1a0 2023-11-06T03:03:44.858 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c59f0 con 0x7f08b815ce80 2023-11-06T03:03:44.858 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.858 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bd677640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.858 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57922/0 (socket says 172.21.15.195:57922) 2023-11-06T03:03:44.859 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bd677640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54060/0 (socket says 172.21.15.195:54060) 2023-11-06T03:03:44.859 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 -- 172.21.15.195:0/69925974 learned_addr learned my addr 172.21.15.195:0/69925974 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:44.859 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b7fff640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 0x7f08b81501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:44.859 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 -- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 msgr2=0x7f08b8158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.859 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.860 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 -- 172.21.15.195:0/69925974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 msgr2=0x7f08b81501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.860 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 0x7f08b81501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.860 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 -- 172.21.15.195:0/69925974 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009420 con 0x7f08b80ce1a0 2023-11-06T03:03:44.860 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bd677640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:44.860 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b7fff640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 0x7f08b81501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:03:44.861 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bce76640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08ac01ab80 tx=0x7f08ac01ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:44.861 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0bd5f0 con 0x7f08b80ce1a0 2023-11-06T03:03:44.861 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac0bd750 con 0x7f08b80ce1a0 2023-11-06T03:03:44.861 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0bdb00 con 0x7f08b80ce1a0 2023-11-06T03:03:44.861 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac0094d0 con 0x7f08b80ce1a0 2023-11-06T03:03:44.862 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b8157d20 con 0x7f08b80ce1a0 2023-11-06T03:03:44.862 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f08ac0b36f0 con 0x7f08b80ce1a0 2023-11-06T03:03:44.862 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f08ac0a7070 con 0x7f08b80ce1a0 2023-11-06T03:03:44.862 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.852+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8072750 con 0x7f08b80ce1a0 2023-11-06T03:03:44.863 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.856+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f08ac0c3060 con 0x7f08b80ce1a0 2023-11-06T03:03:44.918 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.920+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "41bae808-491c-4ce9-9f1a-22c974778030", "id": 6} v 0) v1 -- 0x7f08b815c010 con 0x7f08b80ce1a0 2023-11-06T03:03:44.924 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08b57fa640 1 -- 172.21.15.195:0/69925974 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41bae808-491c-4ce9-9f1a-22c974778030", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f08ac0b39a0 con 0x7f08b80ce1a0 2023-11-06T03:03:44.925 INFO:teuthology.orchestra.run.smithi195.stdout:6 2023-11-06T03:03:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 msgr2=0x7f08b80c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08ac01ab80 tx=0x7f08ac01ad10 comp rx=0 tx=0).stop 2023-11-06T03:03:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 shutdown_connections 2023-11-06T03:03:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08b815ce80 0x7f08b8158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b815caa0 0x7f08b81501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 --2- 172.21.15.195:0/69925974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08b80ce1a0 0x7f08b80c49f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.924+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 >> 172.21.15.195:0/69925974 conn(0x7f08b8070790 msgr2=0x7f08b815c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.928+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 shutdown_connections 2023-11-06T03:03:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:44.928+0000 7f08bf0da640 1 -- 172.21.15.195:0/69925974 wait complete. 2023-11-06T03:03:44.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 3ceafe59-a8eb-4593-911c-f9e8338dafb7 7 2023-11-06T03:03:45.079 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 Processor -- start 2023-11-06T03:03:45.080 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 -- start start 2023-11-06T03:03:45.081 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.081 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.081 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c0d2920 con 0x7f7f5c0ce1a0 2023-11-06T03:03:45.082 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c0d18a0 con 0x7f7f5c15caa0 2023-11-06T03:03:45.082 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f60cd0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0d1b60 con 0x7f7f5c15ce80 2023-11-06T03:03:45.082 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.082 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f5a59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.082 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:54070/0 (socket says 172.21.15.195:54070) 2023-11-06T03:03:45.083 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f5a59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57938/0 (socket says 172.21.15.195:57938) 2023-11-06T03:03:45.083 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 -- 172.21.15.195:0/2467633036 learned_addr learned my addr 172.21.15.195:0/2467633036 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:45.083 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 -- 172.21.15.195:0/2467633036 >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 msgr2=0x7f7f5c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:45.083 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 --2- 172.21.15.195:0/2467633036 >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.084 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 -- 172.21.15.195:0/2467633036 >> v1:172.21.15.157:6789/0 conn(0x7f7f5c0ce1a0 legacy=0x7f7f5c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:45.084 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f59d9b640 1 -- 172.21.15.195:0/2467633036 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0d1e40 con 0x7f7f5c15ce80 2023-11-06T03:03:45.084 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.080+0000 7f7f5a59c640 1 --2- 172.21.15.195:0/2467633036 >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:45.087 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f59d9b640 1 --2- 172.21.15.195:0/2467633036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d804c8d8e44e617 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:45.087 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f5959a640 1 -- 172.21.15.195:0/2467633036 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f500ac600 con 0x7f7f5c15ce80 2023-11-06T03:03:45.088 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f5959a640 1 -- 172.21.15.195:0/2467633036 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f500a8050 con 0x7f7f5c15ce80 2023-11-06T03:03:45.088 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f5959a640 1 -- 172.21.15.195:0/2467633036 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f500ac980 con 0x7f7f5c15ce80 2023-11-06T03:03:45.088 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- 172.21.15.195:0/2467633036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 msgr2=0x7f7f5c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.088 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/2467633036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.088 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- 172.21.15.195:0/2467633036 shutdown_connections 2023-11-06T03:03:45.089 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/2467633036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cfb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.089 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/2467633036 >> v2:172.21.15.195:3300/0 conn(0x7f7f5c15caa0 0x7f7f5c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.089 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- 172.21.15.195:0/2467633036 >> 172.21.15.195:0/2467633036 conn(0x7f7f5c070790 msgr2=0x7f7f5c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:45.089 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- 172.21.15.195:0/2467633036 shutdown_connections 2023-11-06T03:03:45.089 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- 172.21.15.195:0/2467633036 wait complete. 2023-11-06T03:03:45.090 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 Processor -- start 2023-11-06T03:03:45.090 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- start start 2023-11-06T03:03:45.090 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 0x7f7f5c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.090 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.090 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c156630 con 0x7f7f5c0ce1a0 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c155a20 con 0x7f7f5c15caa0 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f60cd0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c155d00 con 0x7f7f5c15ce80 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f59d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f5ad9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.091 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f59d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:57944/0 (socket says 172.21.15.195:57944) 2023-11-06T03:03:45.092 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.084+0000 7f7f59d9b640 1 -- 172.21.15.195:0/589668993 learned_addr learned my addr 172.21.15.195:0/589668993 (peer_addr_for_me v2:172.21.15.195:0/0) 2023-11-06T03:03:45.092 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f5a59c640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 0x7f7f5c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.092 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 -- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 msgr2=0x7f7f5c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.092 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.092 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 -- 172.21.15.195:0/589668993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 msgr2=0x7f7f5c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.093 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 0x7f7f5c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.093 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 -- 172.21.15.195:0/589668993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c009420 con 0x7f7f5c15caa0 2023-11-06T03:03:45.093 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f5ad9d640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:03:45.093 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f5a59c640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 0x7f7f5c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:45.093 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f59d9b640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f5001ab80 tx=0x7f7f50019270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:45.094 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b5830 con 0x7f7f5c15caa0 2023-11-06T03:03:45.094 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f500b5990 con 0x7f7f5c15caa0 2023-11-06T03:03:45.094 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b5d40 con 0x7f7f5c15caa0 2023-11-06T03:03:45.094 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f500094d0 con 0x7f7f5c15caa0 2023-11-06T03:03:45.094 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c1562c0 con 0x7f7f5c15caa0 2023-11-06T03:03:45.095 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f500b3070 con 0x7f7f5c15caa0 2023-11-06T03:03:45.095 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7f500b7dd0 con 0x7f7f5c15caa0 2023-11-06T03:03:45.095 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c1567a0 con 0x7f7f5c15caa0 2023-11-06T03:03:45.096 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.088+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7f500b1070 con 0x7f7f5c15caa0 2023-11-06T03:03:45.153 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.152+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ceafe59-a8eb-4593-911c-f9e8338dafb7", "id": 7} v 0) v1 -- 0x7f7f5c072750 con 0x7f7f5c15caa0 2023-11-06T03:03:45.160 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f4affd640 1 -- 172.21.15.195:0/589668993 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ceafe59-a8eb-4593-911c-f9e8338dafb7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f500b7770 con 0x7f7f5c15caa0 2023-11-06T03:03:45.160 INFO:teuthology.orchestra.run.smithi195.stdout:7 2023-11-06T03:03:45.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 msgr2=0x7f7f5c0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f5001ab80 tx=0x7f7f50019270 comp rx=0 tx=0).stop 2023-11-06T03:03:45.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 shutdown_connections 2023-11-06T03:03:45.162 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f5c15ce80 0x7f7f5c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f5c15caa0 0x7f7f5c0b3b10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 --2- 172.21.15.195:0/589668993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f5c0ce1a0 0x7f7f5c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 >> 172.21.15.195:0/589668993 conn(0x7f7f5c070790 msgr2=0x7f7f5c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:45.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 shutdown_connections 2023-11-06T03:03:45.163 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-06T03:03:45.160+0000 7f7f60cd0640 1 -- 172.21.15.195:0/589668993 wait complete. 2023-11-06T03:03:45.176 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-06T03:03:45.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-06T03:03:45.179 INFO:tasks.ceph.osd.0:Started 2023-11-06T03:03:45.180 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-06T03:03:45.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-06T03:03:45.183 INFO:tasks.ceph.osd.1:Started 2023-11-06T03:03:45.183 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-06T03:03:45.184 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-06T03:03:45.186 INFO:tasks.ceph.osd.2:Started 2023-11-06T03:03:45.187 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-06T03:03:45.187 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-06T03:03:45.188 INFO:tasks.ceph.mgr.y.smithi157.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-06T03:03:45.189 INFO:tasks.ceph.mgr.y.smithi157.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-06T03:03:45.189 INFO:tasks.ceph.mgr.y.smithi157.stderr: from numpy import show_config as show_numpy_config 2023-11-06T03:03:45.190 INFO:tasks.ceph.osd.3:Started 2023-11-06T03:03:45.190 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-06T03:03:45.191 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-06T03:03:45.192 INFO:tasks.ceph.osd.4:Started 2023-11-06T03:03:45.193 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-06T03:03:45.193 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-06T03:03:45.195 INFO:tasks.ceph.osd.5:Started 2023-11-06T03:03:45.196 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-06T03:03:45.196 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-06T03:03:45.198 INFO:tasks.ceph.osd.6:Started 2023-11-06T03:03:45.198 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-06T03:03:45.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-06T03:03:45.200 INFO:tasks.ceph.osd.7:Started 2023-11-06T03:03:45.201 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:03:45.257 INFO:tasks.ceph.mgr.x.smithi195.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-06T03:03:45.257 INFO:tasks.ceph.mgr.x.smithi195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-06T03:03:45.257 INFO:tasks.ceph.mgr.x.smithi195.stderr: from numpy import show_config as show_numpy_config 2023-11-06T03:03:45.258 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:45.259+0000 7f1f862f0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T03:03:45.297 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:45.296+0000 7f2af11b8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-06T03:03:45.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 Processor -- start 2023-11-06T03:03:45.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 -- start start 2023-11-06T03:03:45.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f768815d3a0 0x7f768815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768806bfc0 con 0x7f76880cced0 2023-11-06T03:03:45.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768815dcc0 con 0x7f76880cca50 2023-11-06T03:03:45.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f768e022640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76880ce990 con 0x7f768815d3a0 2023-11-06T03:03:45.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f7687fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f76880cced0 0x7f768806c450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34154/0 (socket says 172.21.15.157:34154) 2023-11-06T03:03:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f7687fff640 1 -- 172.21.15.157:0/1618555940 learned_addr learned my addr 172.21.15.157:0/1618555940 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:03:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f76877fe640 1 --2- 172.21.15.157:0/1618555940 >> v2:172.21.15.195:3300/0 conn(0x7f768815d3a0 0x7f768815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.323+0000 7f7686ffd640 1 -- 172.21.15.157:0/1618555940 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798521879 0 0) 0x7f768806bfc0 con 0x7f76880cced0 2023-11-06T03:03:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f7686ffd640 1 -- 172.21.15.157:0/1618555940 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7674003530 con 0x7f76880cced0 2023-11-06T03:03:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f76877fe640 1 -- 172.21.15.157:0/1618555940 >> v1:172.21.15.195:6789/0 conn(0x7f76880cca50 legacy=0x7f76880c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f76877fe640 1 -- 172.21.15.157:0/1618555940 >> v1:172.21.15.157:6789/0 conn(0x7f76880cced0 legacy=0x7f768806c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f76877fe640 1 -- 172.21.15.157:0/1618555940 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768806b970 con 0x7f768815d3a0 2023-11-06T03:03:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f76877fe640 1 --2- 172.21.15.157:0/1618555940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768815d3a0 0x7f768815d780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76780097c0 tx=0x7f767801cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5f2183c1a74f72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f7686ffd640 1 -- 172.21.15.157:0/1618555940 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780a8050 con 0x7f768815d3a0 2023-11-06T03:03:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f7686ffd640 1 -- 172.21.15.157:0/1618555940 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780ac5f0 con 0x7f768815d3a0 2023-11-06T03:03:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f7686ffd640 1 -- 172.21.15.157:0/1618555940 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780ac980 con 0x7f768815d3a0 2023-11-06T03:03:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 -- 172.21.15.157:0/1618555940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768815d3a0 msgr2=0x7f768815d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 --2- 172.21.15.157:0/1618555940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768815d3a0 0x7f768815d780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76780097c0 tx=0x7f767801cee0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 -- 172.21.15.157:0/1618555940 shutdown_connections 2023-11-06T03:03:45.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 --2- 172.21.15.157:0/1618555940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768815d3a0 0x7f768815d780 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 -- 172.21.15.157:0/1618555940 >> 172.21.15.157:0/1618555940 conn(0x7f76880cb990 msgr2=0x7f76880cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:45.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 -- 172.21.15.157:0/1618555940 shutdown_connections 2023-11-06T03:03:45.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.327+0000 7f768e022640 1 -- 172.21.15.157:0/1618555940 wait complete. 2023-11-06T03:03:45.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 Processor -- start 2023-11-06T03:03:45.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- start start 2023-11-06T03:03:45.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 0x7f76880703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 0x7f76880711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76880708f0 con 0x7f76880cca50 2023-11-06T03:03:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768814f1a0 con 0x7f76880cced0 2023-11-06T03:03:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768806f9a0 con 0x7f768815d3a0 2023-11-06T03:03:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56534/0 (socket says 172.21.15.157:56534) 2023-11-06T03:03:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 -- 172.21.15.157:0/1527968293 learned_addr learned my addr 172.21.15.157:0/1527968293 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:03:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f767ffff640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 0x7f76880703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7687fff640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 0x7f76880711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:45.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 -- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 msgr2=0x7f76880711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 0x7f76880711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 -- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 msgr2=0x7f76880703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 0x7f76880703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 -- 172.21.15.157:0/1527968293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678009420 con 0x7f76880cca50 2023-11-06T03:03:45.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f767ffff640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 0x7f76880703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:45.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7687fff640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 0x7f76880711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:45.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f76877fe640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f767801a0b0 tx=0x7f767801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:45.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678006e20 con 0x7f76880cca50 2023-11-06T03:03:45.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76780b7660 con 0x7f76880cca50 2023-11-06T03:03:45.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76880716e0 con 0x7f76880cca50 2023-11-06T03:03:45.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76780b7a30 con 0x7f76880cca50 2023-11-06T03:03:45.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688071bf0 con 0x7f76880cca50 2023-11-06T03:03:45.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f76780b3070 con 0x7f76880cca50 2023-11-06T03:03:45.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f76780c0c80 con 0x7f76880cca50 2023-11-06T03:03:45.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.331+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7688072750 con 0x7f76880cca50 2023-11-06T03:03:45.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.347+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f76780bb020 con 0x7f76880cca50 2023-11-06T03:03:45.398 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:45.399+0000 7f1f862f0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T03:03:45.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.403+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76880cc5f0 con 0x7f76880cca50 2023-11-06T03:03:45.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.403+0000 7f7684ff9640 1 -- 172.21.15.157:0/1527968293 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f76780b1070 con 0x7f76880cca50 2023-11-06T03:03:45.404 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:03:45.405 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":10,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:03:45.158771+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 msgr2=0x7f768814d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f767801a0b0 tx=0x7f767801acc0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 shutdown_connections 2023-11-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f768815d3a0 0x7f76880711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76880cced0 0x7f76880703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 --2- 172.21.15.157:0/1527968293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76880cca50 0x7f768814d110 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 >> 172.21.15.157:0/1527968293 conn(0x7f76880cb990 msgr2=0x7f76880aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 shutdown_connections 2023-11-06T03:03:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:03:45.407+0000 7f768e022640 1 -- 172.21.15.157:0/1527968293 wait complete. 2023-11-06T03:03:45.422 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-06T03:03:45.422 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-06T03:03:45.426 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:45.424+0000 7f2af11b8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-06T03:03:45.574 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:45.575+0000 7f1f862f0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T03:03:45.586 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:45.588+0000 7f2af11b8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-06T03:03:45.644 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:45.643+0000 7f1f862f0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T03:03:45.649 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:45.648+0000 7f2af11b8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-06T03:03:45.768 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:45.767+0000 7f1f862f0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T03:03:45.771 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:45.772+0000 7f2af11b8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-06T03:03:45.799 INFO:tasks.ceph.osd.7.smithi195.stderr:2023-11-06T03:03:45.800+0000 7f9f3bf8d640 -1 Falling back to public interface 2023-11-06T03:03:45.810 INFO:tasks.ceph.osd.4.smithi195.stderr:2023-11-06T03:03:45.812+0000 7fe21743d640 -1 Falling back to public interface 2023-11-06T03:03:45.812 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-11-06T03:03:45.811+0000 7f3e76eb2640 -1 Falling back to public interface 2023-11-06T03:03:45.812 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-11-06T03:03:45.811+0000 7f00f0a1e640 -1 Falling back to public interface 2023-11-06T03:03:45.812 INFO:tasks.ceph.osd.5.smithi195.stderr:2023-11-06T03:03:45.812+0000 7fd183685640 -1 Falling back to public interface 2023-11-06T03:03:45.813 INFO:tasks.ceph.osd.6.smithi195.stderr:2023-11-06T03:03:45.812+0000 7fb3c7622640 -1 Falling back to public interface 2023-11-06T03:03:45.815 INFO:tasks.ceph.osd.5.smithi195.stderr:2023-11-06T03:03:45.816+0000 7fd182646640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6825/0: (98) Address already in use 2023-11-06T03:03:45.826 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-11-06T03:03:45.827+0000 7f4343c63640 -1 Falling back to public interface 2023-11-06T03:03:45.842 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-11-06T03:03:45.843+0000 7f4a64e90640 -1 Falling back to public interface 2023-11-06T03:03:46.228 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T03:03:46.340 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.339+0000 7ffa13ebf640 1 Processor -- start 2023-11-06T03:03:46.341 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.339+0000 7ffa13ebf640 1 -- start start 2023-11-06T03:03:46.342 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa13ebf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa0c159930 0x7ffa0c0afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:46.342 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa13ebf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c0b1910 con 0x7ffa0c070330 2023-11-06T03:03:46.342 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa13ebf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c0af110 con 0x7ffa0c06da00 2023-11-06T03:03:46.343 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa13ebf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c0afa20 con 0x7ffa0c159930 2023-11-06T03:03:46.343 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa1245c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffa0c070330 0x7ffa0c0cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34158/0 (socket says 172.21.15.157:34158) 2023-11-06T03:03:46.343 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa1245c640 1 -- 172.21.15.157:0/1994029967 learned_addr learned my addr 172.21.15.157:0/1994029967 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:03:46.343 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa11c5b640 1 --2- 172.21.15.157:0/1994029967 >> v2:172.21.15.195:3300/0 conn(0x7ffa0c159930 0x7ffa0c0afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:46.343 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877820728 0 0) 0x7ffa040014a0 con 0x7ffa0c070330 2023-11-06T03:03:46.344 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f8003530 con 0x7ffa0c070330 2023-11-06T03:03:46.344 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa11c5b640 1 -- 172.21.15.157:0/1994029967 >> v1:172.21.15.195:6789/0 conn(0x7ffa0c06da00 legacy=0x7ffa0c0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.344 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa11c5b640 1 -- 172.21.15.157:0/1994029967 >> v1:172.21.15.157:6789/0 conn(0x7ffa0c070330 legacy=0x7ffa0c0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.344 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985825835 0 0) 0x7ff9f8003530 con 0x7ffa0c070330 2023-11-06T03:03:46.344 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa11c5b640 1 -- 172.21.15.157:0/1994029967 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c06df50 con 0x7ffa0c159930 2023-11-06T03:03:46.346 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa11c5b640 1 --2- 172.21.15.157:0/1994029967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c159930 0x7ffa0c0afe40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa080097c0 tx=0x7ffa0801c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef2c063ca9f7c5d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:46.346 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04001b90 con 0x7ffa0c159930 2023-11-06T03:03:46.346 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa080a7050 con 0x7ffa0c159930 2023-11-06T03:03:46.346 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.343+0000 7ffa10c59640 1 -- 172.21.15.157:0/1994029967 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080ab620 con 0x7ffa0c159930 2023-11-06T03:03:46.346 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/1994029967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c159930 msgr2=0x7ffa0c0afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.347 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/1994029967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c159930 0x7ffa0c0afe40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa080097c0 tx=0x7ffa0801c310 comp rx=0 tx=0).stop 2023-11-06T03:03:46.347 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/1994029967 shutdown_connections 2023-11-06T03:03:46.347 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/1994029967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c159930 0x7ffa0c0afe40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.347 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/1994029967 >> 172.21.15.157:0/1994029967 conn(0x7ffa0c0732f0 msgr2=0x7ffa0c069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:46.347 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/1994029967 shutdown_connections 2023-11-06T03:03:46.348 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/1994029967 wait complete. 2023-11-06T03:03:46.350 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 Processor -- start 2023-11-06T03:03:46.350 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- start start 2023-11-06T03:03:46.350 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:46.350 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 0x7ffa0c15a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:46.351 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c159930 0x7ffa0c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:46.351 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1547c0 con 0x7ffa0c06da00 2023-11-06T03:03:46.351 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c1530d0 con 0x7ffa0c070330 2023-11-06T03:03:46.351 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c15b3e0 con 0x7ffa0c159930 2023-11-06T03:03:46.351 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:46.352 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56550/0 (socket says 172.21.15.157:56550) 2023-11-06T03:03:46.352 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 -- 172.21.15.157:0/2368914912 learned_addr learned my addr 172.21.15.157:0/2368914912 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:03:46.352 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa1145a640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 0x7ffa0c15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:46.352 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa1245c640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c159930 0x7ffa0c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:46.352 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 -- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c159930 msgr2=0x7ffa0c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c159930 0x7ffa0c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 -- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 msgr2=0x7ffa0c15a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 0x7ffa0c15a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 -- 172.21.15.157:0/2368914912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08009420 con 0x7ffa0c06da00 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa1145a640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 0x7ffa0c15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:03:46.353 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa11c5b640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa080ae040 tx=0x7ffa080190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:46.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08004020 con 0x7ffa0c06da00 2023-11-06T03:03:46.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08004180 con 0x7ffa0c06da00 2023-11-06T03:03:46.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c15b6c0 con 0x7ffa0c06da00 2023-11-06T03:03:46.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080044f0 con 0x7ffa0c06da00 2023-11-06T03:03:46.355 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c1533b0 con 0x7ffa0c06da00 2023-11-06T03:03:46.355 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa080b2040 con 0x7ffa0c06da00 2023-11-06T03:03:46.355 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.347+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ffa080b4c70 con 0x7ffa0c06da00 2023-11-06T03:03:46.355 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.351+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c06ad00 con 0x7ffa0c06da00 2023-11-06T03:03:46.356 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.351+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ffa080c1060 con 0x7ffa0c06da00 2023-11-06T03:03:46.380 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:46.379+0000 7f1f862f0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T03:03:46.385 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:46.384+0000 7f2af11b8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-06T03:03:46.408 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.407+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa0c072750 con 0x7ffa0c06da00 2023-11-06T03:03:46.409 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.407+0000 7ffa027fc640 1 -- 172.21.15.157:0/2368914912 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ffa080b0070 con 0x7ffa0c06da00 2023-11-06T03:03:46.409 INFO:teuthology.misc.health.smithi157.stdout: 2023-11-06T03:03:46.409 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":10,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:03:45.158771+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:03:46.410 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 msgr2=0x7ffa0c15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:46.410 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa080ae040 tx=0x7ffa080190b0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.411 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 shutdown_connections 2023-11-06T03:03:46.411 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c159930 0x7ffa0c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.412 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c070330 0x7ffa0c15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.412 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 --2- 172.21.15.157:0/2368914912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c06da00 0x7ffa0c15a040 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:46.412 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 >> 172.21.15.157:0/2368914912 conn(0x7ffa0c0732f0 msgr2=0x7ffa0c0ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:46.412 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 shutdown_connections 2023-11-06T03:03:46.412 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:46.411+0000 7ffa13ebf640 1 -- 172.21.15.157:0/2368914912 wait complete. 2023-11-06T03:03:46.429 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-06T03:03:46.461 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:46.460+0000 7f2af11b8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T03:03:46.467 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:46.467+0000 7f1f862f0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-06T03:03:46.632 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:46.632+0000 7f2af11b8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T03:03:46.644 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:46.643+0000 7f1f862f0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-06T03:03:46.704 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:46.704+0000 7f2af11b8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T03:03:46.717 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:46.719+0000 7f1f862f0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-06T03:03:46.992 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:46.992+0000 7f2af11b8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T03:03:47.006 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:47.007+0000 7f1f862f0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-06T03:03:47.051 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:47.052+0000 7f2af11b8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T03:03:47.066 INFO:tasks.ceph.mgr.y.smithi157.stderr:2023-11-06T03:03:47.067+0000 7f1f862f0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-06T03:03:48.102 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:48.104+0000 7f2a98a72640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-06T03:03:48.103 INFO:tasks.ceph.mgr.x.smithi195.stderr:2023-11-06T03:03:48.104+0000 7f2a98a72640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-06T03:03:49.646 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-11-06T03:03:49.647+0000 7f4a64e90640 -1 osd.3 0 log_to_monitors true 2023-11-06T03:03:49.668 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-11-06T03:03:49.667+0000 7f4343c63640 -1 osd.2 0 log_to_monitors true 2023-11-06T03:03:49.689 INFO:tasks.ceph.osd.4.smithi195.stderr:2023-11-06T03:03:49.687+0000 7fe21743d640 -1 osd.4 0 log_to_monitors true 2023-11-06T03:03:49.689 INFO:tasks.ceph.osd.6.smithi195.stderr:2023-11-06T03:03:49.687+0000 7fb3c7622640 -1 osd.6 0 log_to_monitors true 2023-11-06T03:03:49.698 INFO:tasks.ceph.osd.7.smithi195.stderr:2023-11-06T03:03:49.699+0000 7f9f3bf8d640 -1 osd.7 0 log_to_monitors true 2023-11-06T03:03:49.704 INFO:tasks.ceph.osd.5.smithi195.stderr:2023-11-06T03:03:49.703+0000 7fd183685640 -1 osd.5 0 log_to_monitors true 2023-11-06T03:03:49.777 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-11-06T03:03:49.779+0000 7f00f0a1e640 -1 osd.1 0 log_to_monitors true 2023-11-06T03:03:49.780 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-11-06T03:03:49.779+0000 7f3e76eb2640 -1 osd.0 0 log_to_monitors true 2023-11-06T03:03:52.707 INFO:tasks.ceph.osd.4.smithi195.stderr:2023-11-06T03:03:52.707+0000 7fe212bbf640 -1 osd.4 0 waiting for initial osdmap 2023-11-06T03:03:52.719 INFO:tasks.ceph.osd.6.smithi195.stderr:2023-11-06T03:03:52.719+0000 7fb3c2da4640 -1 osd.6 0 waiting for initial osdmap 2023-11-06T03:03:52.735 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-11-06T03:03:52.735+0000 7f433f3e5640 -1 osd.2 0 waiting for initial osdmap 2023-11-06T03:03:52.750 INFO:tasks.ceph.osd.4.smithi195.stderr:2023-11-06T03:03:52.751+0000 7fe20b204640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.758 INFO:tasks.ceph.osd.6.smithi195.stderr:2023-11-06T03:03:52.759+0000 7fb3bb3e9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.773 INFO:tasks.ceph.osd.2.smithi157.stderr:2023-11-06T03:03:52.775+0000 7f4337a2a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.788 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-11-06T03:03:52.787+0000 7f3e73e6f640 -1 osd.0 0 waiting for initial osdmap 2023-11-06T03:03:52.789 INFO:tasks.ceph.osd.7.smithi195.stderr:2023-11-06T03:03:52.787+0000 7f9f38f4a640 -1 osd.7 0 waiting for initial osdmap 2023-11-06T03:03:52.796 INFO:tasks.ceph.osd.5.smithi195.stderr:2023-11-06T03:03:52.795+0000 7fd180642640 -1 osd.5 0 waiting for initial osdmap 2023-11-06T03:03:52.797 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-11-06T03:03:52.799+0000 7f4a60612640 -1 osd.3 0 waiting for initial osdmap 2023-11-06T03:03:52.810 INFO:tasks.ceph.osd.0.smithi157.stderr:2023-11-06T03:03:52.811+0000 7f3e6ac79640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.817 INFO:tasks.ceph.osd.3.smithi157.stderr:2023-11-06T03:03:52.819+0000 7f4a58c57640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.827 INFO:tasks.ceph.osd.7.smithi195.stderr:2023-11-06T03:03:52.827+0000 7f9f2fd54640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.830 INFO:tasks.ceph.osd.5.smithi195.stderr:2023-11-06T03:03:52.831+0000 7fd17744c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:52.831 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-11-06T03:03:52.831+0000 7f00ed9db640 -1 osd.1 0 waiting for initial osdmap 2023-11-06T03:03:52.858 INFO:tasks.ceph.osd.1.smithi157.stderr:2023-11-06T03:03:52.859+0000 7f00e47e5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-06T03:03:53.237 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T03:03:53.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 Processor -- start 2023-11-06T03:03:53.354 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 -- start start 2023-11-06T03:03:53.356 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2058156a70 0x7f205806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.357 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f20580b2330 0x7f205806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.357 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205806c490 con 0x7f20580adae0 2023-11-06T03:03:53.357 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205806b950 con 0x7f2058156a70 2023-11-06T03:03:53.357 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205ff06640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205806bb60 con 0x7f20580b2330 2023-11-06T03:03:53.358 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2058156a70 0x7f205806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:53.358 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205d4a1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f20580adae0 0x7f20580b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43732/0 (socket says 172.21.15.157:43732) 2023-11-06T03:03:53.358 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205d4a1640 1 -- 172.21.15.157:0/3266998273 learned_addr learned my addr 172.21.15.157:0/3266998273 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:03:53.359 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2058156a70 0x7f205806f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34280/0 (socket says 172.21.15.157:34280) 2023-11-06T03:03:53.359 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 -- 172.21.15.157:0/3266998273 >> v2:172.21.15.195:3300/0 conn(0x7f20580b2330 msgr2=0x7f205806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:03:53.359 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 --2- 172.21.15.157:0/3266998273 >> v2:172.21.15.195:3300/0 conn(0x7f20580b2330 0x7f205806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.359 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 -- 172.21.15.157:0/3266998273 >> v1:172.21.15.157:6789/0 conn(0x7f20580adae0 legacy=0x7f20580b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.359 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205cca0640 1 -- 172.21.15.157:0/3266998273 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510913229 0 0) 0x7f205806c490 con 0x7f20580adae0 2023-11-06T03:03:53.360 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.355+0000 7f205dca2640 1 -- 172.21.15.157:0/3266998273 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205806be40 con 0x7f2058156a70 2023-11-06T03:03:53.363 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205dca2640 1 --2- 172.21.15.157:0/3266998273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2058156a70 0x7f205806f470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20580ce830 tx=0x7f204801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee81f1176ffb362 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:53.364 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205cca0640 1 -- 172.21.15.157:0/3266998273 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480ab650 con 0x7f2058156a70 2023-11-06T03:03:53.364 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205cca0640 1 -- 172.21.15.157:0/3266998273 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480ab7b0 con 0x7f2058156a70 2023-11-06T03:03:53.364 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205cca0640 1 -- 172.21.15.157:0/3266998273 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480abb40 con 0x7f2058156a70 2023-11-06T03:03:53.365 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- 172.21.15.157:0/3266998273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2058156a70 msgr2=0x7f205806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.365 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- 172.21.15.157:0/3266998273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2058156a70 0x7f205806f470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f20580ce830 tx=0x7f204801c430 comp rx=0 tx=0).stop 2023-11-06T03:03:53.365 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- 172.21.15.157:0/3266998273 shutdown_connections 2023-11-06T03:03:53.365 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- 172.21.15.157:0/3266998273 >> v2:172.21.15.195:3300/0 conn(0x7f20580b2330 0x7f205806c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.366 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- 172.21.15.157:0/3266998273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2058156a70 0x7f205806f470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.366 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- 172.21.15.157:0/3266998273 >> 172.21.15.157:0/3266998273 conn(0x7f2058065930 msgr2=0x7f2058065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:53.366 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- 172.21.15.157:0/3266998273 shutdown_connections 2023-11-06T03:03:53.367 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- 172.21.15.157:0/3266998273 wait complete. 2023-11-06T03:03:53.367 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 Processor -- start 2023-11-06T03:03:53.367 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- start start 2023-11-06T03:03:53.367 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 0x7f20581635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.367 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.368 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 0x7f20580ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.368 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058163b00 con 0x7f20580b2330 2023-11-06T03:03:53.368 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205815a680 con 0x7f2058156a70 2023-11-06T03:03:53.368 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205ff06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205815a960 con 0x7f20580adae0 2023-11-06T03:03:53.369 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:53.369 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34282/0 (socket says 172.21.15.157:34282) 2023-11-06T03:03:53.369 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 -- 172.21.15.157:0/840836205 learned_addr learned my addr 172.21.15.157:0/840836205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:03:53.369 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205e4a3640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 0x7f20580ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:53.369 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205dca2640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 0x7f20581635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:53.370 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 -- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 msgr2=0x7f20581635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.370 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 0x7f20581635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.370 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 -- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 msgr2=0x7f20580ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.370 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 0x7f20580ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.371 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 -- 172.21.15.157:0/840836205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048009420 con 0x7f20580b2330 2023-11-06T03:03:53.371 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205e4a3640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 0x7f20580ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:53.371 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205dca2640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 0x7f20581635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:03:53.371 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.359+0000 7f205d4a1640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f204c0011f0 tx=0x7f204c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:53.372 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0074e0 con 0x7f20580b2330 2023-11-06T03:03:53.372 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c007640 con 0x7f20580b2330 2023-11-06T03:03:53.372 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205815ac40 con 0x7f20580b2330 2023-11-06T03:03:53.372 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c004a10 con 0x7f20580b2330 2023-11-06T03:03:53.372 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205815b200 con 0x7f20580b2330 2023-11-06T03:03:53.373 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058072750 con 0x7f20580b2330 2023-11-06T03:03:53.373 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) v1 ==== 85172+0+0 (secure 0 0 0) 0x7f204c004b70 con 0x7f20580b2330 2023-11-06T03:03:53.373 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 0x7f2044067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:03:53.374 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f204c081810 con 0x7f20580b2330 2023-11-06T03:03:53.374 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f205dca2640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 0x7f2044067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:03:53.374 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.363+0000 7f205dca2640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 0x7f2044067e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20480af000 tx=0x7f20480a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:03:53.375 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.367+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f204c050040 con 0x7f20580b2330 2023-11-06T03:03:53.511 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.511+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205806d740 con 0x7f20580b2330 2023-11-06T03:03:53.512 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.511+0000 7f20567fc640 1 -- 172.21.15.157:0/840836205 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f204c018070 con 0x7f20580b2330 2023-11-06T03:03:53.513 INFO:teuthology.misc.health.smithi157.stdout: 2023-11-06T03:03:53.513 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":12,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:03:51.104896+0000","last_up_change":"0.000000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:03:53.515 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 msgr2=0x7f2044067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.515 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 0x7f2044067e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20480af000 tx=0x7f20480a7000 comp rx=0 tx=0).stop 2023-11-06T03:03:53.516 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 msgr2=0x7f205815b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:03:53.516 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f204c0011f0 tx=0x7f204c00c9b0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.516 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 shutdown_connections 2023-11-06T03:03:53.517 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20440659d0 0x7f2044067e90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.517 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2058156a70 0x7f20580ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.517 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20580b2330 0x7f205815b9f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.517 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 --2- 172.21.15.157:0/840836205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20580adae0 0x7f20581635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:03:53.517 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 >> 172.21.15.157:0/840836205 conn(0x7f2058065930 msgr2=0x7f2058158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:03:53.518 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 shutdown_connections 2023-11-06T03:03:53.518 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:03:53.515+0000 7f205ff06640 1 -- 172.21.15.157:0/840836205 wait complete. 2023-11-06T03:03:53.528 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-06T03:04:00.336 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:00.450 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 Processor -- start 2023-11-06T03:04:00.451 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 -- start start 2023-11-06T03:04:00.453 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3128159850 0x7f31280ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.454 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f312806d1c0 0x7f31280701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.454 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3128159c30 0x7f3128070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.454 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280cbcb0 con 0x7f312806d1c0 2023-11-06T03:04:00.454 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312806da20 con 0x7f3128159c30 2023-11-06T03:04:00.455 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312e32e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3128071070 con 0x7f3128159850 2023-11-06T03:04:00.455 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f312806d1c0 0x7f31280701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.455 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f312806d1c0 0x7f31280701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38584/0 (socket says 172.21.15.157:38584) 2023-11-06T03:04:00.455 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 -- 172.21.15.157:0/2733658058 learned_addr learned my addr 172.21.15.157:0/2733658058 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:00.455 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f3127fff640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3128159850 0x7f31280ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.456 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f312c8cb640 1 --2- 172.21.15.157:0/2733658058 >> v2:172.21.15.195:3300/0 conn(0x7f3128159c30 0x7f3128070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.456 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 -- 172.21.15.157:0/2733658058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3128159850 msgr2=0x7f31280ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.456 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3128159850 0x7f31280ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.456 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 -- 172.21.15.157:0/2733658058 >> v2:172.21.15.195:3300/0 conn(0x7f3128159c30 msgr2=0x7f3128070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.456 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 --2- 172.21.15.157:0/2733658058 >> v2:172.21.15.195:3300/0 conn(0x7f3128159c30 0x7f3128070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.457 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.450+0000 7f31277fe640 1 -- 172.21.15.157:0/2733658058 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312806d740 con 0x7f312806d1c0 2023-11-06T03:04:00.460 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f31277fe640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312806d1c0 0x7f31280701f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31280b2570 tx=0x7f311c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275735637f8804ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.460 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f3126ffd640 1 -- 172.21.15.157:0/2733658058 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0ac600 con 0x7f312806d1c0 2023-11-06T03:04:00.460 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f3126ffd640 1 -- 172.21.15.157:0/2733658058 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c0a8050 con 0x7f312806d1c0 2023-11-06T03:04:00.461 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f3126ffd640 1 -- 172.21.15.157:0/2733658058 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0ac980 con 0x7f312806d1c0 2023-11-06T03:04:00.461 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- 172.21.15.157:0/2733658058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312806d1c0 msgr2=0x7f31280701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.461 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312806d1c0 0x7f31280701f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f31280b2570 tx=0x7f311c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.461 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- 172.21.15.157:0/2733658058 shutdown_connections 2023-11-06T03:04:00.462 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 --2- 172.21.15.157:0/2733658058 >> v2:172.21.15.195:3300/0 conn(0x7f3128159c30 0x7f3128070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.462 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312806d1c0 0x7f31280701f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.462 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 --2- 172.21.15.157:0/2733658058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3128159850 0x7f31280ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.462 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- 172.21.15.157:0/2733658058 >> 172.21.15.157:0/2733658058 conn(0x7f3128069d70 msgr2=0x7f312806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:00.462 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- 172.21.15.157:0/2733658058 shutdown_connections 2023-11-06T03:04:00.463 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- 172.21.15.157:0/2733658058 wait complete. 2023-11-06T03:04:00.463 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 Processor -- start 2023-11-06T03:04:00.463 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.454+0000 7f312e32e640 1 -- start start 2023-11-06T03:04:00.463 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312806d1c0 0x7f31280c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.463 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.464 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 0x7f312814bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.464 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c7dc0 con 0x7f3128159850 2023-11-06T03:04:00.464 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8250 con 0x7f3128159c30 2023-11-06T03:04:00.464 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c71b0 con 0x7f312806d1c0 2023-11-06T03:04:00.465 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.465 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38586/0 (socket says 172.21.15.157:38586) 2023-11-06T03:04:00.465 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 -- 172.21.15.157:0/1466208800 learned_addr learned my addr 172.21.15.157:0/1466208800 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:00.466 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3127fff640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312806d1c0 0x7f31280c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.466 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312c8cb640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 0x7f312814bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.466 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 -- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312806d1c0 msgr2=0x7f31280c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.467 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312806d1c0 0x7f31280c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.467 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 -- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 msgr2=0x7f312814bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.467 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 0x7f312814bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.467 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 -- 172.21.15.157:0/1466208800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0094d0 con 0x7f3128159850 2023-11-06T03:04:00.467 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312c8cb640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 0x7f312814bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:00.468 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f31277fe640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f311c01cd70 tx=0x7f311c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.468 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0bc070 con 0x7f3128159850 2023-11-06T03:04:00.468 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c0ace90 con 0x7f3128159850 2023-11-06T03:04:00.468 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31280c7490 con 0x7f3128159850 2023-11-06T03:04:00.468 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b7670 con 0x7f3128159850 2023-11-06T03:04:00.469 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31280c7a50 con 0x7f3128159850 2023-11-06T03:04:00.469 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128072750 con 0x7f3128159850 2023-11-06T03:04:00.469 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f311c0b3070 con 0x7f3128159850 2023-11-06T03:04:00.470 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 0x7f3114067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.470 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f311c11c8a0 con 0x7f3128159850 2023-11-06T03:04:00.470 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.458+0000 7f3127fff640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 0x7f3114067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.471 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.462+0000 7f3127fff640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 0x7f3114067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3118006fd0 tx=0x7f3118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.471 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.462+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f311c0ea640 con 0x7f3128159850 2023-11-06T03:04:00.620 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.618+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f312806ca90 con 0x7f3128159850 2023-11-06T03:04:00.622 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f3124ff9640 1 -- 172.21.15.157:0/1466208800 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f311c0edc60 con 0x7f3128159850 2023-11-06T03:04:00.622 INFO:teuthology.misc.health.smithi157.stdout: 2023-11-06T03:04:00.623 INFO:teuthology.misc.health.smithi157.stdout:{"epoch":16,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:03:57.192944+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:00.624 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 msgr2=0x7f3114067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.624 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 0x7f3114067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3118006fd0 tx=0x7f3118008040 comp rx=0 tx=0).stop 2023-11-06T03:04:00.624 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 msgr2=0x7f31280cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.625 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f311c01cd70 tx=0x7f311c01ad10 comp rx=0 tx=0).stop 2023-11-06T03:04:00.625 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 shutdown_connections 2023-11-06T03:04:00.625 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3114065a60 0x7f3114067f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.625 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3128159c30 0x7f312814bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.625 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3128159850 0x7f31280cfe00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.626 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 --2- 172.21.15.157:0/1466208800 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312806d1c0 0x7f31280c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.626 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 >> 172.21.15.157:0/1466208800 conn(0x7f3128069d70 msgr2=0x7f31280701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:00.626 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 shutdown_connections 2023-11-06T03:04:00.626 INFO:teuthology.misc.health.smithi157.stderr:2023-11-06T03:04:00.622+0000 7f312e32e640 1 -- 172.21.15.157:0/1466208800 wait complete. 2023-11-06T03:04:00.640 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-06T03:04:00.640 INFO:tasks.ceph:Creating RBD pool 2023-11-06T03:04:00.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-06T03:04:00.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 Processor -- start 2023-11-06T03:04:00.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 -- start start 2023-11-06T03:04:00.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd464146370 0x7fd4640b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd4640cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4640cf1a0 con 0x7fd46406b6a0 2023-11-06T03:04:00.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46406c3d0 con 0x7fd464146370 2023-11-06T03:04:00.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd46ab85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46406f5a0 con 0x7fd4640ce800 2023-11-06T03:04:00.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd469122640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd46406b6a0 0x7fd46406bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32804/0 (socket says 172.21.15.157:32804) 2023-11-06T03:04:00.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd469122640 1 -- 172.21.15.157:0/93026572 learned_addr learned my addr 172.21.15.157:0/93026572 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:00.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd468921640 1 --2- 172.21.15.157:0/93026572 >> v2:172.21.15.195:3300/0 conn(0x7fd464146370 0x7fd4640b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898377724 0 0) 0x7fd4640cf1a0 con 0x7fd46406b6a0 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd463fff640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd4640cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd44c003530 con 0x7fd46406b6a0 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3751438067 0 0) 0x7fd44c003530 con 0x7fd46406b6a0 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 msgr2=0x7fd4640cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd4640cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 >> v2:172.21.15.195:3300/0 conn(0x7fd464146370 msgr2=0x7fd4640b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 --2- 172.21.15.157:0/93026572 >> v2:172.21.15.195:3300/0 conn(0x7fd464146370 0x7fd4640b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46406f880 con 0x7fd46406b6a0 2023-11-06T03:04:00.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd45c001a00 con 0x7fd46406b6a0 2023-11-06T03:04:00.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 >> v1:172.21.15.157:6789/0 conn(0x7fd46406b6a0 legacy=0x7fd46406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c003980 0x7fd44c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c0047d0 0x7fd44c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd44c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46406c3d0 con 0x7fd44c0047d0 2023-11-06T03:04:00.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46406f5a0 con 0x7fd44c003980 2023-11-06T03:04:00.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c00b990 con 0x7fd4640ce800 2023-11-06T03:04:00.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd463fff640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c0047d0 0x7fd44c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.766+0000 7fd468921640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c003980 0x7fd44c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd469122640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd44c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 -- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 msgr2=0x7fd44c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4640ce800 0x7fd44c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 -- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c003980 msgr2=0x7fd44c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c003980 0x7fd44c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 -- 172.21.15.157:0/93026572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c003530 con 0x7fd44c0047d0 2023-11-06T03:04:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd468921640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c003980 0x7fd44c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:00.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd463fff640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c0047d0 0x7fd44c004c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd454009b10 tx=0x7fd45401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3566b47d7e0d1eda server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a6460 con 0x7fd44c0047d0 2023-11-06T03:04:00.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4540a65c0 con 0x7fd44c0047d0 2023-11-06T03:04:00.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd4637fe640 1 -- 172.21.15.157:0/93026572 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a6970 con 0x7fd44c0047d0 2023-11-06T03:04:00.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 -- 172.21.15.157:0/93026572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c0047d0 msgr2=0x7fd44c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 --2- 172.21.15.157:0/93026572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd44c0047d0 0x7fd44c004c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd454009b10 tx=0x7fd45401ca50 comp rx=0 tx=0).stop 2023-11-06T03:04:00.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd469122640 1 -- 172.21.15.157:0/93026572 reap_dead start 2023-11-06T03:04:00.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 -- 172.21.15.157:0/93026572 shutdown_connections 2023-11-06T03:04:00.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 -- 172.21.15.157:0/93026572 >> 172.21.15.157:0/93026572 conn(0x7fd464065930 msgr2=0x7fd464065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:00.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 -- 172.21.15.157:0/93026572 shutdown_connections 2023-11-06T03:04:00.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.770+0000 7fd46ab85640 1 -- 172.21.15.157:0/93026572 wait complete. 2023-11-06T03:04:00.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 Processor -- start 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- start start 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 0x7fd4641f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd46415def0 0x7fd46413c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641f53c0 con 0x7fd46413afe0 2023-11-06T03:04:00.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641f6460 con 0x7fd44c0047d0 2023-11-06T03:04:00.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd464135ce0 con 0x7fd46415def0 2023-11-06T03:04:00.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38598/0 (socket says 172.21.15.157:38598) 2023-11-06T03:04:00.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 -- 172.21.15.157:0/328896005 learned_addr learned my addr 172.21.15.157:0/328896005 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:00.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd469122640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd46415def0 0x7fd46413c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd468921640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 0x7fd4641f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 -- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd46415def0 msgr2=0x7fd46413c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd46415def0 0x7fd46413c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 -- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 msgr2=0x7fd4641f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:00.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 0x7fd4641f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 -- 172.21.15.157:0/328896005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46413b580 con 0x7fd46413afe0 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd468921640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 0x7fd4641f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd463fff640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd454009b10 tx=0x7fd44c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45400f000 con 0x7fd46413afe0 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4540a6db0 con 0x7fd46413afe0 2023-11-06T03:04:00.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a7660 con 0x7fd46413afe0 2023-11-06T03:04:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46413b860 con 0x7fd46413afe0 2023-11-06T03:04:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46413be20 con 0x7fd46413afe0 2023-11-06T03:04:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd464135e50 con 0x7fd46413afe0 2023-11-06T03:04:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.774+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd4540a77c0 con 0x7fd46413afe0 2023-11-06T03:04:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.778+0000 7fd4617fa640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 0x7fd450067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.778+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd45411c950 con 0x7fd46413afe0 2023-11-06T03:04:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.778+0000 7fd468921640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 0x7fd450067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.778+0000 7fd468921640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 0x7fd450067f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd46413c5d0 tx=0x7fd458008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.782+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd464135e50 con 0x7fd46413afe0 2023-11-06T03:04:00.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:00.946+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fd464135e50 con 0x7fd46413afe0 2023-11-06T03:04:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.258+0000 7fd4617fa640 1 -- 172.21.15.157:0/328896005 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fd464135e50 con 0x7fd46413afe0 2023-11-06T03:04:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'rbd' created 2023-11-06T03:04:01.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 msgr2=0x7fd450067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 0x7fd450067f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd46413c5d0 tx=0x7fd458008040 comp rx=0 tx=0).stop 2023-11-06T03:04:01.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 msgr2=0x7fd4641f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd454009b10 tx=0x7fd44c005800 comp rx=0 tx=0).stop 2023-11-06T03:04:01.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 shutdown_connections 2023-11-06T03:04:01.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a90 0x7fd450067f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd46415def0 0x7fd46413c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd46413afe0 0x7fd4641f5f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 --2- 172.21.15.157:0/328896005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd44c0047d0 0x7fd4641f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 >> 172.21.15.157:0/328896005 conn(0x7fd464065930 msgr2=0x7fd4640cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 shutdown_connections 2023-11-06T03:04:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.262+0000 7fd46ab85640 1 -- 172.21.15.157:0/328896005 wait complete. 2023-11-06T03:04:01.284 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-06T03:04:01.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.410+0000 7f8163b8a640 1 Processor -- start 2023-11-06T03:04:01.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.410+0000 7f8163b8a640 1 -- start start 2023-11-06T03:04:01.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8163b8a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f815c0d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8163b8a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f815c0d25c0 0x7f815c0d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8163b8a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c0d12c0 con 0x7f815c15ce70 2023-11-06T03:04:01.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8163b8a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815c0d1520 con 0x7f815c0d25c0 2023-11-06T03:04:01.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8163b8a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0d1800 con 0x7f815c0b8570 2023-11-06T03:04:01.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161926640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f815c15ce70 0x7f815c0d0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32808/0 (socket says 172.21.15.157:32808) 2023-11-06T03:04:01.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161926640 1 -- 172.21.15.157:0/2854303543 learned_addr learned my addr 172.21.15.157:0/2854303543 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:01.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8162127640 1 --2- 172.21.15.157:0/2854303543 >> v2:172.21.15.195:3300/0 conn(0x7f815c0d25c0 0x7f815c0d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f815c0d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069295918 0 0) 0x7f815c0d12c0 con 0x7f815c15ce70 2023-11-06T03:04:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8148003530 con 0x7f815c15ce70 2023-11-06T03:04:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346181669 0 0) 0x7f8148003530 con 0x7f815c15ce70 2023-11-06T03:04:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 msgr2=0x7f815c0d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f815c0d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 >> v2:172.21.15.195:3300/0 conn(0x7f815c0d25c0 msgr2=0x7f815c0d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 --2- 172.21.15.157:0/2854303543 >> v2:172.21.15.195:3300/0 conn(0x7f815c0d25c0 0x7f815c0d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0d1ae0 con 0x7f815c15ce70 2023-11-06T03:04:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8158001680 con 0x7f815c15ce70 2023-11-06T03:04:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 >> v1:172.21.15.157:6789/0 conn(0x7f815c15ce70 legacy=0x7f815c0d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f8148003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8148003f00 0x7f81480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8148004d10 0x7f8148005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0d1520 con 0x7f8148003f00 2023-11-06T03:04:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0d1800 con 0x7f8148004d10 2023-11-06T03:04:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814800b990 con 0x7f815c0b8570 2023-11-06T03:04:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8148003f00 0x7f81480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161926640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f8148003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8162127640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8148004d10 0x7f8148005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 -- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 msgr2=0x7f8148003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:01.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f815c0b8570 0x7f8148003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 -- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8148004d10 msgr2=0x7f8148005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:01.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8148004d10 0x7f8148005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.414+0000 7f8161125640 1 -- 172.21.15.157:0/2854303543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148003530 con 0x7f8148003f00 2023-11-06T03:04:01.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8148003f00 0x7f81480047d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f814c009ae0 tx=0x7f814c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861d264aaf078625 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:01.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0a5530 con 0x7f8148003f00 2023-11-06T03:04:01.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c0a5690 con 0x7f8148003f00 2023-11-06T03:04:01.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8160924640 1 -- 172.21.15.157:0/2854303543 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0a5a40 con 0x7f8148003f00 2023-11-06T03:04:01.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- 172.21.15.157:0/2854303543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8148003f00 msgr2=0x7f81480047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 --2- 172.21.15.157:0/2854303543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8148003f00 0x7f81480047d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f814c009ae0 tx=0x7f814c01c9f0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8162127640 1 -- 172.21.15.157:0/2854303543 reap_dead start 2023-11-06T03:04:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- 172.21.15.157:0/2854303543 shutdown_connections 2023-11-06T03:04:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- 172.21.15.157:0/2854303543 >> 172.21.15.157:0/2854303543 conn(0x7f815c070790 msgr2=0x7f815c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- 172.21.15.157:0/2854303543 shutdown_connections 2023-11-06T03:04:01.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- 172.21.15.157:0/2854303543 wait complete. 2023-11-06T03:04:01.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 Processor -- start 2023-11-06T03:04:01.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- start start 2023-11-06T03:04:01.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8148003f00 0x7f815c0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f815c15eaa0 0x7f815c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c1f4650 con 0x7f815c1f0200 2023-11-06T03:04:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c15f470 con 0x7f815c15eaa0 2023-11-06T03:04:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8163b8a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c8de0 con 0x7f8148003f00 2023-11-06T03:04:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38618/0 (socket says 172.21.15.157:38618) 2023-11-06T03:04:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 -- 172.21.15.157:0/1311051867 learned_addr learned my addr 172.21.15.157:0/1311051867 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161926640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8148003f00 0x7f815c0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8162127640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f815c15eaa0 0x7f815c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 -- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8148003f00 msgr2=0x7f815c0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8148003f00 0x7f815c0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 -- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f815c15eaa0 msgr2=0x7f815c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f815c15eaa0 0x7f815c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 -- 172.21.15.157:0/1311051867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c1f3780 con 0x7f815c1f0200 2023-11-06T03:04:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.418+0000 7f8161125640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f814c01d290 tx=0x7f8148005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0bc070 con 0x7f815c1f0200 2023-11-06T03:04:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c00f0c0 con 0x7f815c1f0200 2023-11-06T03:04:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c004220 con 0x7f815c1f0200 2023-11-06T03:04:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c0a9d90 con 0x7f815c1f0200 2023-11-06T03:04:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c1f3cb0 con 0x7f815c1f0200 2023-11-06T03:04:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c0b5250 con 0x7f815c1f0200 2023-11-06T03:04:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f814c0b4040 con 0x7f815c1f0200 2023-11-06T03:04:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 0x7f8140067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f814c11cb60 con 0x7f815c1f0200 2023-11-06T03:04:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f8161926640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 0x7f8140067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:01.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.422+0000 7f8161926640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 0x7f8140067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8158000c10 tx=0x7f8158009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:01.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.426+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f814c0ea7b0 con 0x7f815c1f0200 2023-11-06T03:04:01.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:01.582+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f815c0d3130 con 0x7f815c1f0200 2023-11-06T03:04:02.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.262+0000 7f81527fc640 1 -- 172.21.15.157:0/1311051867 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f814c0eddd0 con 0x7f815c1f0200 2023-11-06T03:04:02.263 INFO:teuthology.orchestra.run.smithi157.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-06T03:04:02.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 msgr2=0x7f8140067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 0x7f8140067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8158000c10 tx=0x7f8158009040 comp rx=0 tx=0).stop 2023-11-06T03:04:02.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 msgr2=0x7f815c0a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f814c01d290 tx=0x7f8148005800 comp rx=0 tx=0).stop 2023-11-06T03:04:02.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 shutdown_connections 2023-11-06T03:04:02.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8140065a60 0x7f8140067f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f815c15eaa0 0x7f815c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f815c1f0200 0x7f815c0a9850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 --2- 172.21.15.157:0/1311051867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8148003f00 0x7f815c0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 >> 172.21.15.157:0/1311051867 conn(0x7f815c070790 msgr2=0x7f815c0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:02.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 shutdown_connections 2023-11-06T03:04:02.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.266+0000 7f8163b8a640 1 -- 172.21.15.157:0/1311051867 wait complete. 2023-11-06T03:04:02.291 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-06T03:04:02.291 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:04:02.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:04:02.294 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:04:02.294 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:04:02.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:04:02.296 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:04:02.297 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:04:02.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:04:02.300 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:04:02.300 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:04:02.300 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:04:02.303 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:04:02.303 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-06T03:04:02.304 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:02.373 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:04:02.373 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:04:02.383 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:04:02.387 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:04:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 Processor -- start 2023-11-06T03:04:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 -- start start 2023-11-06T03:04:02.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00fc15d3a0 0x7f00fc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc06c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc06bfc0 con 0x7f00fc0cca50 2023-11-06T03:04:02.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc15dcc0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0102d25640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ce990 con 0x7f00fc0cced0 2023-11-06T03:04:02.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00fc15d3a0 0x7f00fc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00fc15d3a0 0x7f00fc15d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38652/0 (socket says 172.21.15.157:38652) 2023-11-06T03:04:02.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 -- 172.21.15.157:0/2079644880 learned_addr learned my addr 172.21.15.157:0/2079644880 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:02.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f00fbfff640 1 --1- 172.21.15.157:0/2079644880 >> v1:172.21.15.195:6789/0 conn(0x7f00fc0cca50 0x7f00fc0c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:51860/0 (socket says 172.21.15.157:51860) 2023-11-06T03:04:02.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f01012c2640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc06c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 -- 172.21.15.157:0/2079644880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 msgr2=0x7f00fc06c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:02.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc06c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 -- 172.21.15.157:0/2079644880 >> v1:172.21.15.195:6789/0 conn(0x7f00fc0cca50 legacy=0x7f00fc0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 -- 172.21.15.157:0/2079644880 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc06b970 con 0x7f00fc15d3a0 2023-11-06T03:04:02.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f00fb7fe640 1 -- 172.21.15.157:0/2079644880 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438345195 0 0) 0x7f00fc06bfc0 con 0x7f00fc0cca50 2023-11-06T03:04:02.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.418+0000 7f0100ac1640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc15d780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009870 tx=0x7f00f401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8568c68f2e10b37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00fb7fe640 1 -- 172.21.15.157:0/2079644880 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40ab600 con 0x7f00fc15d3a0 2023-11-06T03:04:02.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00fb7fe640 1 -- 172.21.15.157:0/2079644880 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f40a7050 con 0x7f00fc15d3a0 2023-11-06T03:04:02.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00fb7fe640 1 -- 172.21.15.157:0/2079644880 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40ab980 con 0x7f00fc15d3a0 2023-11-06T03:04:02.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/2079644880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 msgr2=0x7f00fc15d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc15d780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009870 tx=0x7f00f401c540 comp rx=0 tx=0).stop 2023-11-06T03:04:02.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/2079644880 shutdown_connections 2023-11-06T03:04:02.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc06c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- 172.21.15.157:0/2079644880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc15d780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/2079644880 >> 172.21.15.157:0/2079644880 conn(0x7f00fc0cb990 msgr2=0x7f00fc0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:02.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/2079644880 shutdown_connections 2023-11-06T03:04:02.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/2079644880 wait complete. 2023-11-06T03:04:02.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 Processor -- start 2023-11-06T03:04:02.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- start start 2023-11-06T03:04:02.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0cca50 0x7f00fc0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0712e0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0c2ee0 con 0x7f00fc0cca50 2023-11-06T03:04:02.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0c31c0 con 0x7f00fc0cced0 2023-11-06T03:04:02.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38666/0 (socket says 172.21.15.157:38666) 2023-11-06T03:04:02.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 -- 172.21.15.157:0/578900134 learned_addr learned my addr 172.21.15.157:0/578900134 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:02.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00fbfff640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0100ac1640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0cca50 0x7f00fc0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 -- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 msgr2=0x7f00fc0c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc0c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 -- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0cca50 msgr2=0x7f00fc0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0cca50 0x7f00fc0c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 -- 172.21.15.157:0/578900134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40094d0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f01012c2640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00ef90 tx=0x7f00ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec009ca0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec00f040 con 0x7f00fc15d3a0 2023-11-06T03:04:02.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc0c34a0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0146c0 con 0x7f00fc15d3a0 2023-11-06T03:04:02.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.422+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0c3a60 con 0x7f00fc15d3a0 2023-11-06T03:04:02.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0c3c80 con 0x7f00fc15d3a0 2023-11-06T03:04:02.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f00ec014820 con 0x7f00fc15d3a0 2023-11-06T03:04:02.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f00f97fa640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 0x7f00dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f00ec083d00 con 0x7f00fc15d3a0 2023-11-06T03:04:02.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f0100ac1640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 0x7f00dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.426+0000 7f0100ac1640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 0x7f00dc067f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cb630 tx=0x7f00f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.430+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f00ec051950 con 0x7f00fc15d3a0 2023-11-06T03:04:02.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.582+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00fc072750 con 0x7f00fc15d3a0 2023-11-06T03:04:02.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f00f97fa640 1 -- 172.21.15.157:0/578900134 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14229 (secure 0 0 0) 0x7f00ec018020 con 0x7f00fc15d3a0 2023-11-06T03:04:02.587 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:02.588 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":18,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:02.258107+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:02.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 msgr2=0x7f00dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 0x7f00dc067f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cb630 tx=0x7f00f4008040 comp rx=0 tx=0).stop 2023-11-06T03:04:02.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 msgr2=0x7f00fc152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00ef90 tx=0x7f00ec00c490 comp rx=0 tx=0).stop 2023-11-06T03:04:02.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 shutdown_connections 2023-11-06T03:04:02.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00dc065a60 0x7f00dc067f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00fc15d3a0 0x7f00fc152930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00fc0cced0 0x7f00fc0c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 --2- 172.21.15.157:0/578900134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00fc0cca50 0x7f00fc0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 >> 172.21.15.157:0/578900134 conn(0x7f00fc0cb990 msgr2=0x7f00fc06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:02.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 shutdown_connections 2023-11-06T03:04:02.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.586+0000 7f0102d25640 1 -- 172.21.15.157:0/578900134 wait complete. 2023-11-06T03:04:02.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:02.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:02.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 Processor -- start 2023-11-06T03:04:02.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 -- start start 2023-11-06T03:04:02.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29841599c0 0x7f29840ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 0x7f29840b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f298415dea0 0x7f29840b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298415e440 con 0x7f29841599c0 2023-11-06T03:04:02.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298406c240 con 0x7f298415dea0 2023-11-06T03:04:02.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298bce1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298406c520 con 0x7f298406d000 2023-11-06T03:04:02.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29841599c0 0x7f29840ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29841599c0 0x7f29840ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38682/0 (socket says 172.21.15.157:38682) 2023-11-06T03:04:02.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 -- 172.21.15.157:0/1411426170 learned_addr learned my addr 172.21.15.157:0/1411426170 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:02.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298a27e640 1 --2- 172.21.15.157:0/1411426170 >> v2:172.21.15.195:3300/0 conn(0x7f298415dea0 0x7f29840b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298927c640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 0x7f29840b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 -- 172.21.15.157:0/1411426170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 msgr2=0x7f29840b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:02.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 0x7f29840b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 -- 172.21.15.157:0/1411426170 >> v2:172.21.15.195:3300/0 conn(0x7f298415dea0 msgr2=0x7f29840b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:02.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 --2- 172.21.15.157:0/1411426170 >> v2:172.21.15.195:3300/0 conn(0x7f298415dea0 0x7f29840b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 -- 172.21.15.157:0/1411426170 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298406c800 con 0x7f29841599c0 2023-11-06T03:04:02.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f298927c640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 0x7f29840b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.734+0000 7f2989a7d640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29841599c0 0x7f29840ae000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f298406bbf0 tx=0x7f298001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba55929326c0a576 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2988a7b640 1 -- 172.21.15.157:0/1411426170 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800aa610 con 0x7f29841599c0 2023-11-06T03:04:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2988a7b640 1 -- 172.21.15.157:0/1411426170 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800a6040 con 0x7f29841599c0 2023-11-06T03:04:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2988a7b640 1 -- 172.21.15.157:0/1411426170 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800aa990 con 0x7f29841599c0 2023-11-06T03:04:02.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/1411426170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29841599c0 msgr2=0x7f29840ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29841599c0 0x7f29840ae000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f298406bbf0 tx=0x7f298001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/1411426170 shutdown_connections 2023-11-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- 172.21.15.157:0/1411426170 >> v2:172.21.15.195:3300/0 conn(0x7f298415dea0 0x7f29840b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f298406d000 0x7f29840b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- 172.21.15.157:0/1411426170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29841599c0 0x7f29840ae000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/1411426170 >> 172.21.15.157:0/1411426170 conn(0x7f2984065930 msgr2=0x7f2984065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:02.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/1411426170 shutdown_connections 2023-11-06T03:04:02.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/1411426170 wait complete. 2023-11-06T03:04:02.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 Processor -- start 2023-11-06T03:04:02.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- start start 2023-11-06T03:04:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29841599c0 0x7f29840cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 0x7f29840cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840cf0f0 con 0x7f298406d000 2023-11-06T03:04:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29840ce470 con 0x7f298415dea0 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840ce750 con 0x7f29841599c0 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38696/0 (socket says 172.21.15.157:38696) 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 -- 172.21.15.157:0/3170039406 learned_addr learned my addr 172.21.15.157:0/3170039406 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298a27e640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 0x7f29840cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298927c640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29841599c0 0x7f29840cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 -- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29841599c0 msgr2=0x7f29840cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29841599c0 0x7f29840cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 -- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 msgr2=0x7f29840cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 0x7f29840cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 -- 172.21.15.157:0/3170039406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800094d0 con 0x7f298406d000 2023-11-06T03:04:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298a27e640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 0x7f29840cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f2989a7d640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f29800ac040 tx=0x7f298001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800bd5f0 con 0x7f298406d000 2023-11-06T03:04:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800bd770 con 0x7f298406d000 2023-11-06T03:04:02.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29840cea30 con 0x7f298406d000 2023-11-06T03:04:02.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800bdb20 con 0x7f298406d000 2023-11-06T03:04:02.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.738+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298413df90 con 0x7f298406d000 2023-11-06T03:04:02.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2984072750 con 0x7f298406d000 2023-11-06T03:04:02.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29800b33f0 con 0x7f298406d000 2023-11-06T03:04:02.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f297a7fc640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 0x7f2968067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:02.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f298011d6d0 con 0x7f298406d000 2023-11-06T03:04:02.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f298927c640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 0x7f2968067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:02.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.742+0000 7f298927c640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 0x7f2968067f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2974006fd0 tx=0x7f2974008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:02.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.746+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f29800eb320 con 0x7f298406d000 2023-11-06T03:04:02.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.902+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29840cb110 con 0x7f298406d000 2023-11-06T03:04:02.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.902+0000 7f297a7fc640 1 -- 172.21.15.157:0/3170039406 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29800ee940 con 0x7f298406d000 2023-11-06T03:04:02.904 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:02.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 msgr2=0x7f2968067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 0x7f2968067f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2974006fd0 tx=0x7f2974008040 comp rx=0 tx=0).stop 2023-11-06T03:04:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 msgr2=0x7f298413da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f29800ac040 tx=0x7f298001ad10 comp rx=0 tx=0).stop 2023-11-06T03:04:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 shutdown_connections 2023-11-06T03:04:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2968065a60 0x7f2968067f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f298415dea0 0x7f29840cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29841599c0 0x7f29840cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 --2- 172.21.15.157:0/3170039406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f298406d000 0x7f298413da50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 >> 172.21.15.157:0/3170039406 conn(0x7f2984065930 msgr2=0x7f2984158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 shutdown_connections 2023-11-06T03:04:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:02.906+0000 7f298bce1640 1 -- 172.21.15.157:0/3170039406 wait complete. 2023-11-06T03:04:02.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:03.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 Processor -- start 2023-11-06T03:04:03.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 -- start start 2023-11-06T03:04:03.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f05240d0280 0x7f0524167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240d2920 con 0x7f052415caa0 2023-11-06T03:04:03.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240d18a0 con 0x7f052415ce80 2023-11-06T03:04:03.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0529dc1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05240d1b60 con 0x7f05240d0280 2023-11-06T03:04:03.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f052415caa0 0x7f05240cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32826/0 (socket says 172.21.15.157:32826) 2023-11-06T03:04:03.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 -- 172.21.15.157:0/970596796 learned_addr learned my addr 172.21.15.157:0/970596796 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:03.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0523fff640 1 --2- 172.21.15.157:0/970596796 >> v2:172.21.15.195:3300/0 conn(0x7f05240d0280 0x7f0524167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132503851 0 0) 0x7f05240d2920 con 0x7f052415caa0 2023-11-06T03:04:03.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f050c003530 con 0x7f052415caa0 2023-11-06T03:04:03.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431852407 0 0) 0x7f050c003530 con 0x7f052415caa0 2023-11-06T03:04:03.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 >> v2:172.21.15.195:3300/0 conn(0x7f05240d0280 msgr2=0x7f0524167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 --2- 172.21.15.157:0/970596796 >> v2:172.21.15.195:3300/0 conn(0x7f05240d0280 0x7f0524167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 >> v1:172.21.15.195:6789/0 conn(0x7f052415ce80 legacy=0x7f05240cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240d1e40 con 0x7f052415caa0 2023-11-06T03:04:03.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0514001680 con 0x7f052415caa0 2023-11-06T03:04:03.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 >> v1:172.21.15.157:6789/0 conn(0x7f052415caa0 legacy=0x7f05240cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 0x7f050c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c0047d0 0x7f050c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f050c005190 0x7f050c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240d1b60 con 0x7f050c003980 2023-11-06T03:04:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00c030 con 0x7f050c005190 2023-11-06T03:04:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c00c1f0 con 0x7f050c0047d0 2023-11-06T03:04:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 0x7f050c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0522ffd640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c0047d0 0x7f050c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0523fff640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f050c005190 0x7f050c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 -- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c0047d0 msgr2=0x7f050c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c0047d0 0x7f050c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 -- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f050c005190 msgr2=0x7f050c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f050c005190 0x7f050c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f05237fe640 1 -- 172.21.15.157:0/970596796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c003530 con 0x7f050c003980 2023-11-06T03:04:03.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0523fff640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f050c005190 0x7f050c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:03.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0522ffd640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c0047d0 0x7f050c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:03.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.050+0000 7f0523fff640 1 -- 172.21.15.157:0/970596796 reap_dead start 2023-11-06T03:04:03.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f05237fe640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 0x7f050c003da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f051400c240 tx=0x7f051401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2e9785c903765f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140a5b20 con 0x7f050c003980 2023-11-06T03:04:03.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140a5c80 con 0x7f050c003980 2023-11-06T03:04:03.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f05227fc640 1 -- 172.21.15.157:0/970596796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514005b20 con 0x7f050c003980 2023-11-06T03:04:03.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 msgr2=0x7f050c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 0x7f050c003da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f051400c240 tx=0x7f051401cbd0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- 172.21.15.157:0/970596796 shutdown_connections 2023-11-06T03:04:03.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 --2- 172.21.15.157:0/970596796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f050c003980 0x7f050c003da0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- 172.21.15.157:0/970596796 >> 172.21.15.157:0/970596796 conn(0x7f0524070790 msgr2=0x7f0524070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- 172.21.15.157:0/970596796 shutdown_connections 2023-11-06T03:04:03.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- 172.21.15.157:0/970596796 wait complete. 2023-11-06T03:04:03.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 Processor -- start 2023-11-06T03:04:03.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.054+0000 7f0529dc1640 1 -- start start 2023-11-06T03:04:03.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c003980 0x7f05240c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 0x7f05240cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240ccac0 con 0x7f052415e4b0 2023-11-06T03:04:03.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cba20 con 0x7f05240ca810 2023-11-06T03:04:03.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05240cbd00 con 0x7f050c003980 2023-11-06T03:04:03.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38712/0 (socket says 172.21.15.157:38712) 2023-11-06T03:04:03.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 -- 172.21.15.157:0/4262798701 learned_addr learned my addr 172.21.15.157:0/4262798701 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f05237fe640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c003980 0x7f05240c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0522ffd640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 0x7f05240cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 -- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c003980 msgr2=0x7f05240c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c003980 0x7f05240c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 -- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 msgr2=0x7f05240cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 0x7f05240cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 -- 172.21.15.157:0/4262798701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240cbfe0 con 0x7f052415e4b0 2023-11-06T03:04:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0522ffd640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 0x7f05240cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0523fff640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05240b49b0 tx=0x7f051c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c016420 con 0x7f052415e4b0 2023-11-06T03:04:03.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c016580 con 0x7f052415e4b0 2023-11-06T03:04:03.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05240cc2c0 con 0x7f052415e4b0 2023-11-06T03:04:03.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0168f0 con 0x7f052415e4b0 2023-11-06T03:04:03.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240cb6b0 con 0x7f052415e4b0 2023-11-06T03:04:03.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0524072750 con 0x7f052415e4b0 2023-11-06T03:04:03.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f051c016a50 con 0x7f052415e4b0 2023-11-06T03:04:03.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 0x7f0510067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f051c083bb0 con 0x7f052415e4b0 2023-11-06T03:04:03.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.058+0000 7f05237fe640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 0x7f0510067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.062+0000 7f05237fe640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 0x7f0510067f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05140ab000 tx=0x7f05140a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.062+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f051c0517f0 con 0x7f052415e4b0 2023-11-06T03:04:03.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.218+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05240ccfe0 con 0x7f052415e4b0 2023-11-06T03:04:03.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.218+0000 7f050bfff640 1 -- 172.21.15.157:0/4262798701 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f051c054e10 con 0x7f052415e4b0 2023-11-06T03:04:03.221 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:03.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 msgr2=0x7f0510067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 0x7f0510067f50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05140ab000 tx=0x7f05140a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 msgr2=0x7f052415e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05240b49b0 tx=0x7f051c00f520 comp rx=0 tx=0).stop 2023-11-06T03:04:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 shutdown_connections 2023-11-06T03:04:03.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0510065a90 0x7f0510067f50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f052415e4b0 0x7f052415e960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05240ca810 0x7f05240cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 --2- 172.21.15.157:0/4262798701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f050c003980 0x7f05240c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 >> 172.21.15.157:0/4262798701 conn(0x7f0524070790 msgr2=0x7f05240d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 shutdown_connections 2023-11-06T03:04:03.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.222+0000 7f0529dc1640 1 -- 172.21.15.157:0/4262798701 wait complete. 2023-11-06T03:04:03.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:04:03.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 Processor -- start 2023-11-06T03:04:03.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 -- start start 2023-11-06T03:04:03.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 0x7f45e40b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e415c290 0x7f45e40661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40cef40 con 0x7f45e4065d40 2023-11-06T03:04:03.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e40cd450 con 0x7f45e415c290 2023-11-06T03:04:03.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45ea8ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40cc840 con 0x7f45e4144430 2023-11-06T03:04:03.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e415c290 0x7f45e40661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e415c290 0x7f45e40661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38718/0 (socket says 172.21.15.157:38718) 2023-11-06T03:04:03.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3940843712 learned_addr learned my addr 172.21.15.157:0/3940843712 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e3fff640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 0x7f45e40b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3940843712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 msgr2=0x7f45e40b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 0x7f45e40b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3940843712 >> v1:172.21.15.195:6789/0 conn(0x7f45e4065d40 legacy=0x7f45e40cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3940843712 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4066700 con 0x7f45e415c290 2023-11-06T03:04:03.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.350+0000 7f45e3fff640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 0x7f45e40b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45e8e4b640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e40661c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009870 tx=0x7f45dc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aad99f50bfeee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45e2ffd640 1 -- 172.21.15.157:0/3940843712 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0ab600 con 0x7f45e415c290 2023-11-06T03:04:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45e2ffd640 1 -- 172.21.15.157:0/3940843712 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc0a7050 con 0x7f45e415c290 2023-11-06T03:04:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45e2ffd640 1 -- 172.21.15.157:0/3940843712 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0ab980 con 0x7f45e415c290 2023-11-06T03:04:03.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3940843712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 msgr2=0x7f45e40661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e40661c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009870 tx=0x7f45dc01c500 comp rx=0 tx=0).stop 2023-11-06T03:04:03.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3940843712 shutdown_connections 2023-11-06T03:04:03.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e40661c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3940843712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4144430 0x7f45e40b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3940843712 >> 172.21.15.157:0/3940843712 conn(0x7f45e4065930 msgr2=0x7f45e4071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3940843712 shutdown_connections 2023-11-06T03:04:03.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.354+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3940843712 wait complete. 2023-11-06T03:04:03.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 Processor -- start 2023-11-06T03:04:03.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- start start 2023-11-06T03:04:03.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4065d40 0x7f45e4137c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 0x7f45e4131810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4131d50 con 0x7f45e415c290 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e41571e0 con 0x7f45e4144430 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4132b70 con 0x7f45e4065d40 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38734/0 (socket says 172.21.15.157:38734) 2023-11-06T03:04:03.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3610324082 learned_addr learned my addr 172.21.15.157:0/3610324082 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e3fff640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4065d40 0x7f45e4137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4065d40 msgr2=0x7f45e4137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4065d40 0x7f45e4137c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 msgr2=0x7f45e4131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e37fe640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 0x7f45e4131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 0x7f45e4131810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 -- 172.21.15.157:0/3610324082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0094d0 con 0x7f45e415c290 2023-11-06T03:04:03.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e37fe640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 0x7f45e4131810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:03.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e8e4b640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45dc01abb0 tx=0x7f45dc0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0b4640 con 0x7f45e415c290 2023-11-06T03:04:03.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc0b47a0 con 0x7f45e415c290 2023-11-06T03:04:03.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e4131f60 con 0x7f45e415c290 2023-11-06T03:04:03.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0b4b10 con 0x7f45e415c290 2023-11-06T03:04:03.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e4132520 con 0x7f45e415c290 2023-11-06T03:04:03.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e4072670 con 0x7f45e415c290 2023-11-06T03:04:03.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45dc0b2070 con 0x7f45e415c290 2023-11-06T03:04:03.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 0x7f45d0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.358+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f45dc11cf70 con 0x7f45e415c290 2023-11-06T03:04:03.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.362+0000 7f45e3fff640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 0x7f45d0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.362+0000 7f45e3fff640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 0x7f45d0067f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45e4143a20 tx=0x7f45d40084f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.362+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45dc0eab20 con 0x7f45e415c290 2023-11-06T03:04:03.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e413b4f0 con 0x7f45e415c290 2023-11-06T03:04:03.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45e0ff9640 1 -- 172.21.15.157:0/3610324082 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7f45dc0ee140 con 0x7f45e415c290 2023-11-06T03:04:03.552 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:03.552 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-06T03:04:03.552 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 3 2023-11-06T03:04:03.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 msgr2=0x7f45d0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 0x7f45d0067f50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45e4143a20 tx=0x7f45d40084f0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 msgr2=0x7f45e4156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.550+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45dc01abb0 tx=0x7f45dc0192a0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 shutdown_connections 2023-11-06T03:04:03.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45d0065a90 0x7f45d0067f50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e415c290 0x7f45e4156ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e4144430 0x7f45e4131810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 --2- 172.21.15.157:0/3610324082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e4065d40 0x7f45e4137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 >> 172.21.15.157:0/3610324082 conn(0x7f45e4065930 msgr2=0x7f45e413a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 shutdown_connections 2023-11-06T03:04:03.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.554+0000 7f45ea8ae640 1 -- 172.21.15.157:0/3610324082 wait complete. 2023-11-06T03:04:03.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:03.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.678+0000 7f4c399c7640 1 Processor -- start 2023-11-06T03:04:03.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.678+0000 7f4c399c7640 1 -- start start 2023-11-06T03:04:03.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c399c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c3415ce80 0x7f4c340cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c399c7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c340d00b0 0x7f4c340b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c399c7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340d2920 con 0x7f4c3415caa0 2023-11-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c399c7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340d0520 con 0x7f4c3415ce80 2023-11-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c399c7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340d18a0 con 0x7f4c340d00b0 2023-11-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c3415ce80 0x7f4c340cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c3415ce80 0x7f4c340cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38750/0 (socket says 172.21.15.157:38750) 2023-11-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c32ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4c3415caa0 0x7f4c340cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32838/0 (socket says 172.21.15.157:32838) 2023-11-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 -- 172.21.15.157:0/2446318453 learned_addr learned my addr 172.21.15.157:0/2446318453 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c337fe640 1 --2- 172.21.15.157:0/2446318453 >> v2:172.21.15.195:3300/0 conn(0x7f4c340d00b0 0x7f4c340b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 -- 172.21.15.157:0/2446318453 >> v2:172.21.15.195:3300/0 conn(0x7f4c340d00b0 msgr2=0x7f4c340b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 --2- 172.21.15.157:0/2446318453 >> v2:172.21.15.195:3300/0 conn(0x7f4c340d00b0 0x7f4c340b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 -- 172.21.15.157:0/2446318453 >> v1:172.21.15.157:6789/0 conn(0x7f4c3415caa0 legacy=0x7f4c340cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c31ffb640 1 -- 172.21.15.157:0/2446318453 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754666577 0 0) 0x7f4c340d2920 con 0x7f4c3415caa0 2023-11-06T03:04:03.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.682+0000 7f4c327fc640 1 -- 172.21.15.157:0/2446318453 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340d1b60 con 0x7f4c3415ce80 2023-11-06T03:04:03.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 --2- 172.21.15.157:0/2446318453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415ce80 0x7f4c340cfb70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cf670 tx=0x7f4c1c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8fbaae8e1758ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c31ffb640 1 -- 172.21.15.157:0/2446318453 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0aa430 con 0x7f4c3415ce80 2023-11-06T03:04:03.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c31ffb640 1 -- 172.21.15.157:0/2446318453 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0aa590 con 0x7f4c3415ce80 2023-11-06T03:04:03.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c31ffb640 1 -- 172.21.15.157:0/2446318453 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0aa920 con 0x7f4c3415ce80 2023-11-06T03:04:03.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- 172.21.15.157:0/2446318453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415ce80 msgr2=0x7f4c340cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- 172.21.15.157:0/2446318453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415ce80 0x7f4c340cfb70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cf670 tx=0x7f4c1c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- 172.21.15.157:0/2446318453 shutdown_connections 2023-11-06T03:04:03.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- 172.21.15.157:0/2446318453 >> v2:172.21.15.195:3300/0 conn(0x7f4c340d00b0 0x7f4c340b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- 172.21.15.157:0/2446318453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415ce80 0x7f4c340cfb70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- 172.21.15.157:0/2446318453 >> 172.21.15.157:0/2446318453 conn(0x7f4c34070790 msgr2=0x7f4c34070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- 172.21.15.157:0/2446318453 shutdown_connections 2023-11-06T03:04:03.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- 172.21.15.157:0/2446318453 wait complete. 2023-11-06T03:04:03.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 Processor -- start 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- start start 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c340d00b0 0x7f4c340c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c3415ce80 0x7f4c340cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341671b0 con 0x7f4c3415caa0 2023-11-06T03:04:03.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34157160 con 0x7f4c340d00b0 2023-11-06T03:04:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c399c7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340cc500 con 0x7f4c3415ce80 2023-11-06T03:04:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38752/0 (socket says 172.21.15.157:38752) 2023-11-06T03:04:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 -- 172.21.15.157:0/1200575879 learned_addr learned my addr 172.21.15.157:0/1200575879 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 -- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c3415ce80 msgr2=0x7f4c340cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c337fe640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c3415ce80 0x7f4c340cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c3415ce80 0x7f4c340cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 -- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c340d00b0 msgr2=0x7f4c340c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c32ffd640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c340d00b0 0x7f4c340c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c340d00b0 0x7f4c340c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.686+0000 7f4c327fc640 1 -- 172.21.15.157:0/1200575879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0094d0 con 0x7f4c3415caa0 2023-11-06T03:04:03.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c327fc640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0190e0 tx=0x7f4c1c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0aa430 con 0x7f4c3415caa0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0b3800 con 0x7f4c3415caa0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3415d5a0 con 0x7f4c3415caa0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0b3b90 con 0x7f4c3415caa0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34156830 con 0x7f4c3415caa0 2023-11-06T03:04:03.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34072750 con 0x7f4c3415caa0 2023-11-06T03:04:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c1c0b3cf0 con 0x7f4c3415caa0 2023-11-06T03:04:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4c1c11cc20 con 0x7f4c3415caa0 2023-11-06T03:04:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c32ffd640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.690+0000 7f4c32ffd640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c240012f0 tx=0x7f4c24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.694+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4c1c0ea740 con 0x7f4c3415caa0 2023-11-06T03:04:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.838+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c340c9ab0 con 0x7f4c3415caa0 2023-11-06T03:04:03.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c2bfff640 1 -- 172.21.15.157:0/1200575879 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14216 (secure 0 0 0) 0x7f4c1c0edd60 con 0x7f4c3415caa0 2023-11-06T03:04:03.842 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:03.842 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":19,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:03.269184+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:03.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 msgr2=0x7f4c14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c240012f0 tx=0x7f4c24009040 comp rx=0 tx=0).stop 2023-11-06T03:04:03.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 msgr2=0x7f4c340cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0190e0 tx=0x7f4c1c01ad10 comp rx=0 tx=0).stop 2023-11-06T03:04:03.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 shutdown_connections 2023-11-06T03:04:03.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c3415ce80 0x7f4c340cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c3415caa0 0x7f4c340cba80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 --2- 172.21.15.157:0/1200575879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c340d00b0 0x7f4c340c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 >> 172.21.15.157:0/1200575879 conn(0x7f4c34070790 msgr2=0x7f4c340b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 shutdown_connections 2023-11-06T03:04:03.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.842+0000 7f4c399c7640 1 -- 172.21.15.157:0/1200575879 wait complete. 2023-11-06T03:04:03.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:03.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:03.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 Processor -- start 2023-11-06T03:04:03.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 -- start start 2023-11-06T03:04:03.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 0x7f99d80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99d80cd730 0x7f99d80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d80cc2b0 con 0x7f99d815ac80 2023-11-06T03:04:03.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80b01a0 con 0x7f99d80cd730 2023-11-06T03:04:03.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99dda4b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80cfb10 con 0x7f99d80b2b10 2023-11-06T03:04:03.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99d80cd730 0x7f99d80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99d815ac80 0x7f99d80b2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32846/0 (socket says 172.21.15.157:32846) 2023-11-06T03:04:03.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d6ffd640 1 -- 172.21.15.157:0/3111575668 learned_addr learned my addr 172.21.15.157:0/3111575668 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:03.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d67fc640 1 --2- 172.21.15.157:0/3111575668 >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 0x7f99d80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d77fe640 1 -- 172.21.15.157:0/3111575668 >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 msgr2=0x7f99d80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:03.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d77fe640 1 --2- 172.21.15.157:0/3111575668 >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 0x7f99d80ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d77fe640 1 -- 172.21.15.157:0/3111575668 >> v1:172.21.15.157:6789/0 conn(0x7f99d815ac80 legacy=0x7f99d80b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d5ffb640 1 -- 172.21.15.157:0/3111575668 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237632103 0 0) 0x7f99d80cc2b0 con 0x7f99d815ac80 2023-11-06T03:04:03.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d77fe640 1 -- 172.21.15.157:0/3111575668 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80cfdf0 con 0x7f99d80cd730 2023-11-06T03:04:03.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.974+0000 7f99d67fc640 1 --2- 172.21.15.157:0/3111575668 >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 0x7f99d80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:03.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d77fe640 1 --2- 172.21.15.157:0/3111575668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80cd730 0x7f99d80cea90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b5e40 tx=0x7f99d001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418232b28f9548e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d5ffb640 1 -- 172.21.15.157:0/3111575668 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00a59d0 con 0x7f99d80cd730 2023-11-06T03:04:03.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d5ffb640 1 -- 172.21.15.157:0/3111575668 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d00a5b30 con 0x7f99d80cd730 2023-11-06T03:04:03.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d5ffb640 1 -- 172.21.15.157:0/3111575668 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00ad440 con 0x7f99d80cd730 2023-11-06T03:04:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- 172.21.15.157:0/3111575668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80cd730 msgr2=0x7f99d80cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3111575668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80cd730 0x7f99d80cea90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b5e40 tx=0x7f99d001c910 comp rx=0 tx=0).stop 2023-11-06T03:04:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- 172.21.15.157:0/3111575668 shutdown_connections 2023-11-06T03:04:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3111575668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80cd730 0x7f99d80cea90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3111575668 >> v2:172.21.15.195:3300/0 conn(0x7f99d80b2b10 0x7f99d80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- 172.21.15.157:0/3111575668 >> 172.21.15.157:0/3111575668 conn(0x7f99d806d560 msgr2=0x7f99d806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:03.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- 172.21.15.157:0/3111575668 shutdown_connections 2023-11-06T03:04:03.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- 172.21.15.157:0/3111575668 wait complete. 2023-11-06T03:04:03.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 Processor -- start 2023-11-06T03:04:03.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- start start 2023-11-06T03:04:03.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99d80cd730 0x7f99d80c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99d815ac80 0x7f99d80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81526a0 con 0x7f99d80b2b10 2023-11-06T03:04:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8151430 con 0x7f99d80cd730 2023-11-06T03:04:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99dda4b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d80b1f40 con 0x7f99d815ac80 2023-11-06T03:04:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38760/0 (socket says 172.21.15.157:38760) 2023-11-06T03:04:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 -- 172.21.15.157:0/3526501218 learned_addr learned my addr 172.21.15.157:0/3526501218 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d77fe640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99d815ac80 0x7f99d80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d67fc640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99d80cd730 0x7f99d80c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 -- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99d815ac80 msgr2=0x7f99d80c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99d815ac80 0x7f99d80c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 -- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99d80cd730 msgr2=0x7f99d80c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99d80cd730 0x7f99d80c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 -- 172.21.15.157:0/3526501218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009a70 con 0x7f99d80b2b10 2023-11-06T03:04:03.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99d6ffd640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c480 tx=0x7f99cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc0074e0 con 0x7f99d80b2b10 2023-11-06T03:04:03.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc007660 con 0x7f99d80b2b10 2023-11-06T03:04:03.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.978+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc004a30 con 0x7f99d80b2b10 2023-11-06T03:04:03.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d80b2220 con 0x7f99d80b2b10 2023-11-06T03:04:03.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d8152050 con 0x7f99d80b2b10 2023-11-06T03:04:03.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d8072750 con 0x7f99d80b2b10 2023-11-06T03:04:03.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f99cc004b90 con 0x7f99d80b2b10 2023-11-06T03:04:03.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99bffff640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 0x7f99c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:03.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f99cc082160 con 0x7f99d80b2b10 2023-11-06T03:04:03.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99d67fc640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 0x7f99c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:03.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.982+0000 7f99d67fc640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 0x7f99c4067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99d80c71a0 tx=0x7f99c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:03.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:03.986+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f99cc053790 con 0x7f99d80b2b10 2023-11-06T03:04:04.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.138+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99d806e3e0 con 0x7f99d80b2b10 2023-11-06T03:04:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.138+0000 7f99bffff640 1 -- 172.21.15.157:0/3526501218 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99cc018020 con 0x7f99d80b2b10 2023-11-06T03:04:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 msgr2=0x7f99c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 0x7f99c4067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99d80c71a0 tx=0x7f99c8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 msgr2=0x7f99d8150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00c480 tx=0x7f99cc00c940 comp rx=0 tx=0).stop 2023-11-06T03:04:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 shutdown_connections 2023-11-06T03:04:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99c4065a60 0x7f99c4067f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99d815ac80 0x7f99d80c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99d80cd730 0x7f99d80c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 --2- 172.21.15.157:0/3526501218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99d80b2b10 0x7f99d8150ef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 >> 172.21.15.157:0/3526501218 conn(0x7f99d806d560 msgr2=0x7f99d80b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 shutdown_connections 2023-11-06T03:04:04.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.142+0000 7f99dda4b640 1 -- 172.21.15.157:0/3526501218 wait complete. 2023-11-06T03:04:04.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:04.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 Processor -- start 2023-11-06T03:04:04.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 -- start start 2023-11-06T03:04:04.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f926c0adf40 0x7f926c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c06c490 con 0x7f926c156a70 2023-11-06T03:04:04.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c06b950 con 0x7f926c0adf40 2023-11-06T03:04:04.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.278+0000 7f9271c4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c06fb80 con 0x7f926c0adae0 2023-11-06T03:04:04.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f926c0adf40 0x7f926c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f926c0adf40 0x7f926c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38776/0 (socket says 172.21.15.157:38776) 2023-11-06T03:04:04.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 -- 172.21.15.157:0/3863545995 learned_addr learned my addr 172.21.15.157:0/3863545995 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:04.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f926c156a70 0x7f926c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32860/0 (socket says 172.21.15.157:32860) 2023-11-06T03:04:04.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926affd640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 -- 172.21.15.157:0/3863545995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 msgr2=0x7f926c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 -- 172.21.15.157:0/3863545995 >> v1:172.21.15.157:6789/0 conn(0x7f926c156a70 legacy=0x7f926c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926a7fc640 1 -- 172.21.15.157:0/3863545995 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256627034 0 0) 0x7f926c06c490 con 0x7f926c156a70 2023-11-06T03:04:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 -- 172.21.15.157:0/3863545995 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c06bb60 con 0x7f926c0adf40 2023-11-06T03:04:04.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926bfff640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c0b2160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f328e9e43987551c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926a7fc640 1 -- 172.21.15.157:0/3863545995 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640a5820 con 0x7f926c0adf40 2023-11-06T03:04:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926a7fc640 1 -- 172.21.15.157:0/3863545995 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640a5980 con 0x7f926c0adf40 2023-11-06T03:04:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f926a7fc640 1 -- 172.21.15.157:0/3863545995 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640a5d10 con 0x7f926c0adf40 2023-11-06T03:04:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 -- 172.21.15.157:0/3863545995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 msgr2=0x7f926c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c0b2160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 -- 172.21.15.157:0/3863545995 shutdown_connections 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c0b2160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 --2- 172.21.15.157:0/3863545995 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 -- 172.21.15.157:0/3863545995 >> 172.21.15.157:0/3863545995 conn(0x7f926c065930 msgr2=0x7f926c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 -- 172.21.15.157:0/3863545995 shutdown_connections 2023-11-06T03:04:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.282+0000 7f9271c4a640 1 -- 172.21.15.157:0/3863545995 wait complete. 2023-11-06T03:04:04.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 Processor -- start 2023-11-06T03:04:04.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- start start 2023-11-06T03:04:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c156a70 0x7f926c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0ce6d0 con 0x7f926c0adf40 2023-11-06T03:04:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0ae6e0 con 0x7f926c156a70 2023-11-06T03:04:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0ae9a0 con 0x7f926c0adae0 2023-11-06T03:04:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38778/0 (socket says 172.21.15.157:38778) 2023-11-06T03:04:04.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 -- 172.21.15.157:0/515249204 learned_addr learned my addr 172.21.15.157:0/515249204 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:04.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926b7fe640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926bfff640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c156a70 0x7f926c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 -- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 msgr2=0x7f926c0af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 -- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c156a70 msgr2=0x7f926c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c156a70 0x7f926c0ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 -- 172.21.15.157:0/515249204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9264009b30 con 0x7f926c0adf40 2023-11-06T03:04:04.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f926affd640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f926000d5f0 tx=0x7f926000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600125f0 con 0x7f926c0adf40 2023-11-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260012750 con 0x7f926c0adf40 2023-11-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260012ae0 con 0x7f926c0adf40 2023-11-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0aec80 con 0x7f926c0adf40 2023-11-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c1467c0 con 0x7f926c0adf40 2023-11-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.286+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c072750 con 0x7f926c0adf40 2023-11-06T03:04:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.290+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9260012c40 con 0x7f926c0adf40 2023-11-06T03:04:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.290+0000 7f9253fff640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 0x7f9258067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.290+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f92600834a0 con 0x7f926c0adf40 2023-11-06T03:04:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.290+0000 7f926b7fe640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 0x7f9258067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.290+0000 7f926b7fe640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 0x7f9258067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f925c000f70 tx=0x7f925c007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.294+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9260051050 con 0x7f926c0adf40 2023-11-06T03:04:04.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.446+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f926c15c400 con 0x7f926c0adf40 2023-11-06T03:04:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.446+0000 7f9253fff640 1 -- 172.21.15.157:0/515249204 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9260054670 con 0x7f926c0adf40 2023-11-06T03:04:04.448 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:04.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 msgr2=0x7f9258067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 0x7f9258067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f925c000f70 tx=0x7f925c007b60 comp rx=0 tx=0).stop 2023-11-06T03:04:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 msgr2=0x7f926c148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f926000d5f0 tx=0x7f926000dab0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 shutdown_connections 2023-11-06T03:04:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9258065a60 0x7f9258067f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f926c156a70 0x7f926c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f926c0adf40 0x7f926c148930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 --2- 172.21.15.157:0/515249204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926c0adae0 0x7f926c0af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 >> 172.21.15.157:0/515249204 conn(0x7f926c065930 msgr2=0x7f926c14c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 shutdown_connections 2023-11-06T03:04:04.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.450+0000 7f9271c4a640 1 -- 172.21.15.157:0/515249204 wait complete. 2023-11-06T03:04:04.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:04:04.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.582+0000 7fa093ee1640 1 Processor -- start 2023-11-06T03:04:04.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.582+0000 7fa093ee1640 1 -- start start 2023-11-06T03:04:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa093ee1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa093ee1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa08c0b3b20 0x7fa08c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa093ee1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08c15f580 con 0x7fa08c15e5e0 2023-11-06T03:04:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa093ee1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c15efc0 con 0x7fa08c0b3b20 2023-11-06T03:04:04.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa093ee1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c15f2a0 con 0x7fa08c1551e0 2023-11-06T03:04:04.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:48288/0 (socket says 172.21.15.157:48288) 2023-11-06T03:04:04.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 -- 172.21.15.157:0/148919097 learned_addr learned my addr 172.21.15.157:0/148919097 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:04.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa09247e640 1 --2- 172.21.15.157:0/148919097 >> v2:172.21.15.195:3300/0 conn(0x7fa08c0b3b20 0x7fa08c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 -- 172.21.15.157:0/148919097 >> v2:172.21.15.195:3300/0 conn(0x7fa08c0b3b20 msgr2=0x7fa08c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 --2- 172.21.15.157:0/148919097 >> v2:172.21.15.195:3300/0 conn(0x7fa08c0b3b20 0x7fa08c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 -- 172.21.15.157:0/148919097 >> v1:172.21.15.195:6789/0 conn(0x7fa08c15e5e0 legacy=0x7fa08c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.586+0000 7fa091c7d640 1 -- 172.21.15.157:0/148919097 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c06f5a0 con 0x7fa08c1551e0 2023-11-06T03:04:04.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa091c7d640 1 --2- 172.21.15.157:0/148919097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa08801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6649e38760cb8b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa090c7b640 1 -- 172.21.15.157:0/148919097 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880ad640 con 0x7fa08c1551e0 2023-11-06T03:04:04.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa090c7b640 1 -- 172.21.15.157:0/148919097 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0880ad7a0 con 0x7fa08c1551e0 2023-11-06T03:04:04.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa090c7b640 1 -- 172.21.15.157:0/148919097 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880adb30 con 0x7fa08c1551e0 2023-11-06T03:04:04.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- 172.21.15.157:0/148919097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 msgr2=0x7fa08c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- 172.21.15.157:0/148919097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa08801c480 comp rx=0 tx=0).stop 2023-11-06T03:04:04.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- 172.21.15.157:0/148919097 shutdown_connections 2023-11-06T03:04:04.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- 172.21.15.157:0/148919097 >> v2:172.21.15.195:3300/0 conn(0x7fa08c0b3b20 0x7fa08c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- 172.21.15.157:0/148919097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c1551e0 0x7fa08c06fda0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- 172.21.15.157:0/148919097 >> 172.21.15.157:0/148919097 conn(0x7fa08c065930 msgr2=0x7fa08c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- 172.21.15.157:0/148919097 shutdown_connections 2023-11-06T03:04:04.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- 172.21.15.157:0/148919097 wait complete. 2023-11-06T03:04:04.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 Processor -- start 2023-11-06T03:04:04.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- start start 2023-11-06T03:04:04.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c0b3b20 0x7fa08c14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa08c15e5e0 0x7fa08c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c15a7d0 con 0x7fa08c1551e0 2023-11-06T03:04:04.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c154870 con 0x7fa08c15e5e0 2023-11-06T03:04:04.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa093ee1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c153ea0 con 0x7fa08c0b3b20 2023-11-06T03:04:04.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38786/0 (socket says 172.21.15.157:38786) 2023-11-06T03:04:04.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 -- 172.21.15.157:0/3203297839 learned_addr learned my addr 172.21.15.157:0/3203297839 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:04.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa091c7d640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c0b3b20 0x7fa08c14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09247e640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa08c15e5e0 0x7fa08c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 -- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c0b3b20 msgr2=0x7fa08c14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c0b3b20 0x7fa08c14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 -- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa08c15e5e0 msgr2=0x7fa08c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa08c15e5e0 0x7fa08c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 -- 172.21.15.157:0/3203297839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880094d0 con 0x7fa08c1551e0 2023-11-06T03:04:04.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa09147c640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c3c0 tx=0x7fa07c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.590+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0052d0 con 0x7fa08c1551e0 2023-11-06T03:04:04.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c005430 con 0x7fa08c1551e0 2023-11-06T03:04:04.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c153290 con 0x7fa08c1551e0 2023-11-06T03:04:04.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c003840 con 0x7fa08c1551e0 2023-11-06T03:04:04.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c153850 con 0x7fa08c1551e0 2023-11-06T03:04:04.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c154010 con 0x7fa08c1551e0 2023-11-06T03:04:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa07c0039a0 con 0x7fa08c1551e0 2023-11-06T03:04:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa0827fc640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 0x7fa070067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa07c082350 con 0x7fa08c1551e0 2023-11-06T03:04:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa091c7d640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 0x7fa070067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.594+0000 7fa091c7d640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 0x7fa070067f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0880af000 tx=0x7fa088008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.598+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa07c050040 con 0x7fa08c1551e0 2023-11-06T03:04:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.746+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa08c072750 con 0x7fa08c1551e0 2023-11-06T03:04:04.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa0827fc640 1 -- 172.21.15.157:0/3203297839 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa07c053660 con 0x7fa08c1551e0 2023-11-06T03:04:04.751 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:04.751 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:04:04.751 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:04:04.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:04:04.751 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:04:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:04:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:04:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:04:04.752 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:04:04.753 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:04:04.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 msgr2=0x7fa070067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 0x7fa070067f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0880af000 tx=0x7fa088008040 comp rx=0 tx=0).stop 2023-11-06T03:04:04.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 msgr2=0x7fa08c15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00c3c0 tx=0x7fa07c00c880 comp rx=0 tx=0).stop 2023-11-06T03:04:04.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 shutdown_connections 2023-11-06T03:04:04.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa070065a90 0x7fa070067f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa08c15e5e0 0x7fa08c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa08c1551e0 0x7fa08c15a290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 --2- 172.21.15.157:0/3203297839 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa08c0b3b20 0x7fa08c14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 >> 172.21.15.157:0/3203297839 conn(0x7fa08c065930 msgr2=0x7fa08c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 shutdown_connections 2023-11-06T03:04:04.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.750+0000 7fa093ee1640 1 -- 172.21.15.157:0/3203297839 wait complete. 2023-11-06T03:04:04.768 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:04:04.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:04:04.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 Processor -- start 2023-11-06T03:04:04.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 -- start start 2023-11-06T03:04:04.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7feeac0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feeac0b6590 0x7feeac0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0cfbe0 con 0x7feeac15b120 2023-11-06T03:04:04.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0b3f10 con 0x7feeac0b6590 2023-11-06T03:04:04.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb2d5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0cfea0 con 0x7feeac15ad40 2023-11-06T03:04:04.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeabfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feeac15b120 0x7feeac0b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32870/0 (socket says 172.21.15.157:32870) 2023-11-06T03:04:04.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeabfff640 1 -- 172.21.15.157:0/1119361458 learned_addr learned my addr 172.21.15.157:0/1119361458 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:04.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 --2- 172.21.15.157:0/1119361458 >> v2:172.21.15.195:3300/0 conn(0x7feeac0b6590 0x7feeac0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb0af7640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7feeac0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010282981 0 0) 0x7feeac0cfbe0 con 0x7feeac15b120 2023-11-06T03:04:04.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee94003530 con 0x7feeac15b120 2023-11-06T03:04:04.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619654429 0 0) 0x7fee94003530 con 0x7feeac15b120 2023-11-06T03:04:04.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 msgr2=0x7feeac0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7feeac0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 >> v2:172.21.15.195:3300/0 conn(0x7feeac0b6590 msgr2=0x7feeac0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 --2- 172.21.15.157:0/1119361458 >> v2:172.21.15.195:3300/0 conn(0x7feeac0b6590 0x7feeac0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0d0180 con 0x7feeac15b120 2023-11-06T03:04:04.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7feea0001680 con 0x7feeac15b120 2023-11-06T03:04:04.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 >> v1:172.21.15.157:6789/0 conn(0x7feeac15b120 legacy=0x7feeac0b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7fee94003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee94003f00 0x7fee940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee94004d10 0x7fee94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0b3f10 con 0x7fee94004d10 2023-11-06T03:04:04.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0cfea0 con 0x7fee94003f00 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9400b9e0 con 0x7feeac15ad40 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee94004d10 0x7fee94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeabfff640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee94003f00 0x7fee940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb0af7640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7fee94003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 -- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 msgr2=0x7fee94003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac15ad40 0x7fee94003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 -- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee94003f00 msgr2=0x7fee940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:04.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee94003f00 0x7fee940047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.890+0000 7feeb12f8640 1 -- 172.21.15.157:0/1119361458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94003530 con 0x7fee94004d10 2023-11-06T03:04:04.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb12f8640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee94004d10 0x7fee94005190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feea400e170 tx=0x7feea401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48e4f71e3ccc14a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40a5a90 con 0x7fee94004d10 2023-11-06T03:04:04.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400f040 con 0x7fee94004d10 2023-11-06T03:04:04.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeab7fe640 1 -- 172.21.15.157:0/1119361458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40a5e30 con 0x7fee94004d10 2023-11-06T03:04:04.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- 172.21.15.157:0/1119361458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee94004d10 msgr2=0x7fee94005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/1119361458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee94004d10 0x7fee94005190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feea400e170 tx=0x7feea401cc80 comp rx=0 tx=0).stop 2023-11-06T03:04:04.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb12f8640 1 -- 172.21.15.157:0/1119361458 reap_dead start 2023-11-06T03:04:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- 172.21.15.157:0/1119361458 shutdown_connections 2023-11-06T03:04:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- 172.21.15.157:0/1119361458 >> 172.21.15.157:0/1119361458 conn(0x7feeac06d5f0 msgr2=0x7feeac06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- 172.21.15.157:0/1119361458 shutdown_connections 2023-11-06T03:04:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- 172.21.15.157:0/1119361458 wait complete. 2023-11-06T03:04:04.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 Processor -- start 2023-11-06T03:04:04.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.894+0000 7feeb2d5b640 1 -- start start 2023-11-06T03:04:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac0b42b0 0x7feeac147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 0x7feeac0c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac147fc0 con 0x7feeac0b4690 2023-11-06T03:04:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0b0700 con 0x7feeac0b1c50 2023-11-06T03:04:04.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0b21a0 con 0x7feeac0b42b0 2023-11-06T03:04:04.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38798/0 (socket says 172.21.15.157:38798) 2023-11-06T03:04:04.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 -- 172.21.15.157:0/3989973485 learned_addr learned my addr 172.21.15.157:0/3989973485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:04.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb0af7640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac0b42b0 0x7feeac147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeabfff640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 0x7feeac0c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 -- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac0b42b0 msgr2=0x7feeac147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac0b42b0 0x7feeac147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 -- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 msgr2=0x7feeac0c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:04.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 0x7feeac0c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 -- 172.21.15.157:0/3989973485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0b2480 con 0x7feeac0b4690 2023-11-06T03:04:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeabfff640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 0x7feeac0c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb12f8640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feea400e170 tx=0x7feea4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea40ba070 con 0x7feeac0b4690 2023-11-06T03:04:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea40039a0 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac0b2760 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4003b10 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac0b2d20 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac06bf80 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feea40b4040 con 0x7feeac0b4690 2023-11-06T03:04:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 0x7fee98067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:04.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7feea411cff0 con 0x7feeac0b4690 2023-11-06T03:04:04.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb0af7640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 0x7fee98067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:04.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.898+0000 7feeb0af7640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 0x7fee98067f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009a30 tx=0x7fee9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:04.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:04.902+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feea40eaba0 con 0x7feeac0b4690 2023-11-06T03:04:05.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.058+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feeac04e050 con 0x7feeac0b4690 2023-11-06T03:04:05.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.294+0000 7feea97fa640 1 -- 172.21.15.157:0/3989973485 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7feea40b20b0 con 0x7feeac0b4690 2023-11-06T03:04:05.295 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:04:05.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 msgr2=0x7fee98067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 0x7fee98067f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009a30 tx=0x7fee9c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:05.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 msgr2=0x7feeac0c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feea400e170 tx=0x7feea4009750 comp rx=0 tx=0).stop 2023-11-06T03:04:05.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 shutdown_connections 2023-11-06T03:04:05.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee98065a90 0x7fee98067f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feeac0b4690 0x7feeac0c67c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feeac0b1c50 0x7feeac0c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 --2- 172.21.15.157:0/3989973485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feeac0b42b0 0x7feeac147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 >> 172.21.15.157:0/3989973485 conn(0x7feeac06d5f0 msgr2=0x7feeac158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:05.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 shutdown_connections 2023-11-06T03:04:05.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.298+0000 7feeb2d5b640 1 -- 172.21.15.157:0/3989973485 wait complete. 2023-11-06T03:04:05.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:04:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 Processor -- start 2023-11-06T03:04:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 -- start start 2023-11-06T03:04:05.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 0x7f89700cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89700d2920 con 0x7f89700d00b0 2023-11-06T03:04:05.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89700d18a0 con 0x7f897015caa0 2023-11-06T03:04:05.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f8974a9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89700d0520 con 0x7f897015ce80 2023-11-06T03:04:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f89700d00b0 0x7f89700b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32880/0 (socket says 172.21.15.157:32880) 2023-11-06T03:04:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 -- 172.21.15.157:0/1035578257 learned_addr learned my addr 172.21.15.157:0/1035578257 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896dd9b640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 0x7f89700cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311212308 0 0) 0x7f89700d2920 con 0x7f89700d00b0 2023-11-06T03:04:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8958003530 con 0x7f89700d00b0 2023-11-06T03:04:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1895530545 0 0) 0x7f8958003530 con 0x7f89700d00b0 2023-11-06T03:04:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 msgr2=0x7f89700cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 0x7f89700cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 >> v1:172.21.15.195:6789/0 conn(0x7f897015caa0 legacy=0x7f89700cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89700d1b60 con 0x7f89700d00b0 2023-11-06T03:04:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8968001a00 con 0x7f89700d00b0 2023-11-06T03:04:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 >> v1:172.21.15.157:6789/0 conn(0x7f89700d00b0 legacy=0x7f89700b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89580039b0 0x7f8958003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 0x7f89580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8958004d10 0x7f8958005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89700d0520 con 0x7f8958004d10 2023-11-06T03:04:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895800ba70 con 0x7f89580039b0 2023-11-06T03:04:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895800bc30 con 0x7f897015ce80 2023-11-06T03:04:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8958004d10 0x7f8958005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896e59c640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89580039b0 0x7f8958003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 -- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 msgr2=0x7f89580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:05.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f897015ce80 0x7f89580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 -- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89580039b0 msgr2=0x7f8958003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:05.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89580039b0 0x7f8958003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.450+0000 7f896ed9d640 1 -- 172.21.15.157:0/1035578257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958003530 con 0x7f8958004d10 2023-11-06T03:04:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f896ed9d640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8958004d10 0x7f8958005190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f896800a800 tx=0x7f896801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e165e02ac92c451f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680072d0 con 0x7f8958004d10 2023-11-06T03:04:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89680a8070 con 0x7f8958004d10 2023-11-06T03:04:05.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f896d59a640 1 -- 172.21.15.157:0/1035578257 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968007690 con 0x7f8958004d10 2023-11-06T03:04:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- 172.21.15.157:0/1035578257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8958004d10 msgr2=0x7f8958005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 --2- 172.21.15.157:0/1035578257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8958004d10 0x7f8958005190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f896800a800 tx=0x7f896801d300 comp rx=0 tx=0).stop 2023-11-06T03:04:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f896ed9d640 1 -- 172.21.15.157:0/1035578257 reap_dead start 2023-11-06T03:04:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- 172.21.15.157:0/1035578257 shutdown_connections 2023-11-06T03:04:05.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- 172.21.15.157:0/1035578257 >> 172.21.15.157:0/1035578257 conn(0x7f8970070790 msgr2=0x7f8970070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- 172.21.15.157:0/1035578257 shutdown_connections 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- 172.21.15.157:0/1035578257 wait complete. 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 Processor -- start 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.454+0000 7f8974a9b640 1 -- start start 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8970149ef0 0x7f89700cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89700cc9e0 0x7f8970159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89700cb420 con 0x7f8970159fe0 2023-11-06T03:04:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897015a470 con 0x7f89700cc9e0 2023-11-06T03:04:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89700ca7a0 con 0x7f8970149ef0 2023-11-06T03:04:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38818/0 (socket says 172.21.15.157:38818) 2023-11-06T03:04:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 -- 172.21.15.157:0/3537459776 learned_addr learned my addr 172.21.15.157:0/3537459776 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896dd9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89700cc9e0 0x7f8970159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896e59c640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8970149ef0 0x7f89700cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 -- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8970149ef0 msgr2=0x7f89700cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8970149ef0 0x7f89700cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 -- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89700cc9e0 msgr2=0x7f8970159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:05.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89700cc9e0 0x7f8970159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:05.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 -- 172.21.15.157:0/3537459776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89700caa80 con 0x7f8970159fe0 2023-11-06T03:04:05.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896ed9d640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f896801dba0 tx=0x7f8958005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680ba070 con 0x7f8970159fe0 2023-11-06T03:04:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968001210 con 0x7f8970159fe0 2023-11-06T03:04:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680019c0 con 0x7f8970159fe0 2023-11-06T03:04:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89700cad60 con 0x7f8970159fe0 2023-11-06T03:04:05.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89700cb840 con 0x7f8970159fe0 2023-11-06T03:04:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89700cb560 con 0x7f8970159fe0 2023-11-06T03:04:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f89680aa030 con 0x7f8970159fe0 2023-11-06T03:04:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 0x7f8950067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f896811cda0 con 0x7f8970159fe0 2023-11-06T03:04:05.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.458+0000 7f896e59c640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 0x7f8950067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:05.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.462+0000 7f896e59c640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 0x7f8950067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8964001620 tx=0x7f8964009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:05.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.462+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f89700cb560 con 0x7f8970159fe0 2023-11-06T03:04:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:05.618+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f89700cb560 con 0x7f8970159fe0 2023-11-06T03:04:06.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.306+0000 7f895effd640 1 -- 172.21.15.157:0/3537459776 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f89700cb560 con 0x7f8970159fe0 2023-11-06T03:04:06.309 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:04:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 msgr2=0x7f8950067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 0x7f8950067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8964001620 tx=0x7f8964009040 comp rx=0 tx=0).stop 2023-11-06T03:04:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 msgr2=0x7f8970157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f896801dba0 tx=0x7f8958005800 comp rx=0 tx=0).stop 2023-11-06T03:04:06.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 shutdown_connections 2023-11-06T03:04:06.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8950065a60 0x7f8950067f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8970159fe0 0x7f8970157e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89700cc9e0 0x7f8970159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 --2- 172.21.15.157:0/3537459776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8970149ef0 0x7f89700cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.318+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 >> 172.21.15.157:0/3537459776 conn(0x7f8970070790 msgr2=0x7f89700b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:06.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.322+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 shutdown_connections 2023-11-06T03:04:06.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.326+0000 7f8974a9b640 1 -- 172.21.15.157:0/3537459776 wait complete. 2023-11-06T03:04:06.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:04:06.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 Processor -- start 2023-11-06T03:04:06.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 -- start start 2023-11-06T03:04:06.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9ebc0cc3d0 0x7f9ebc066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc15aff0 con 0x7f9ebc0cf090 2023-11-06T03:04:06.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc06dd80 con 0x7f9ebc0b1e40 2023-11-06T03:04:06.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ec1092640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc06e060 con 0x7f9ebc0cc3d0 2023-11-06T03:04:06.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9ebc0cf090 0x7f9ebc0b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32884/0 (socket says 172.21.15.157:32884) 2023-11-06T03:04:06.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebad9d640 1 -- 172.21.15.157:0/1266969448 learned_addr learned my addr 172.21.15.157:0/1266969448 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 --2- 172.21.15.157:0/1266969448 >> v2:172.21.15.195:3300/0 conn(0x7f9ebc0cc3d0 0x7f9ebc066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363689093 0 0) 0x7f9ebc15aff0 con 0x7f9ebc0cf090 2023-11-06T03:04:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea4003530 con 0x7f9ebc0cf090 2023-11-06T03:04:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155811705 0 0) 0x7f9ea4003530 con 0x7f9ebc0cf090 2023-11-06T03:04:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 >> v2:172.21.15.195:3300/0 conn(0x7f9ebc0cc3d0 msgr2=0x7f9ebc066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 --2- 172.21.15.157:0/1266969448 >> v2:172.21.15.195:3300/0 conn(0x7f9ebc0cc3d0 0x7f9ebc066340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 >> v1:172.21.15.195:6789/0 conn(0x7f9ebc0b1e40 legacy=0x7f9ebc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc15b2b0 con 0x7f9ebc0cf090 2023-11-06T03:04:06.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9eac001680 con 0x7f9ebc0cf090 2023-11-06T03:04:06.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 >> v1:172.21.15.157:6789/0 conn(0x7f9ebc0cf090 legacy=0x7f9ebc0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea4003980 0x7f9ea4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ea4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea4005190 0x7f9ea400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc06e060 con 0x7f9ea40047d0 2023-11-06T03:04:06.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea400c030 con 0x7f9ea4003980 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea400c1f0 con 0x7f9ea4005190 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ea4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebb59e640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea4005190 0x7f9ea400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebad9d640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea4003980 0x7f9ea4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 -- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea4005190 msgr2=0x7f9ea400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea4005190 0x7f9ea400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 -- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea4003980 msgr2=0x7f9ea4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea4003980 0x7f9ea4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9eba59c640 1 -- 172.21.15.157:0/1266969448 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4003530 con 0x7f9ea40047d0 2023-11-06T03:04:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebad9d640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea4003980 0x7f9ea4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:06.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebb59e640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea4005190 0x7f9ea400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.470+0000 7f9ebb59e640 1 -- 172.21.15.157:0/1266969448 reap_dead start 2023-11-06T03:04:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9eba59c640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ea4004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009ae0 tx=0x7f9eb001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da70f1f9707c0f2d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00a5640 con 0x7f9ea40047d0 2023-11-06T03:04:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac001d80 con 0x7f9ea40047d0 2023-11-06T03:04:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9eb9d9b640 1 -- 172.21.15.157:0/1266969448 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00a59e0 con 0x7f9ea40047d0 2023-11-06T03:04:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 msgr2=0x7f9ea4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ea4004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009ae0 tx=0x7f9eb001ca50 comp rx=0 tx=0).stop 2023-11-06T03:04:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- 172.21.15.157:0/1266969448 shutdown_connections 2023-11-06T03:04:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 --2- 172.21.15.157:0/1266969448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ea4004c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- 172.21.15.157:0/1266969448 >> 172.21.15.157:0/1266969448 conn(0x7f9ebc0732f0 msgr2=0x7f9ebc069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- 172.21.15.157:0/1266969448 shutdown_connections 2023-11-06T03:04:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- 172.21.15.157:0/1266969448 wait complete. 2023-11-06T03:04:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 Processor -- start 2023-11-06T03:04:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.474+0000 7f9ec1092640 1 -- start start 2023-11-06T03:04:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc0b4490 0x7f9ebc14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ebc14f680 0x7f9ebc14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc150040 con 0x7f9ea40047d0 2023-11-06T03:04:06.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b4000 con 0x7f9ebc0b4490 2023-11-06T03:04:06.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b33f0 con 0x7f9ebc14f680 2023-11-06T03:04:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38850/0 (socket says 172.21.15.157:38850) 2023-11-06T03:04:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 -- 172.21.15.157:0/2077068361 learned_addr learned my addr 172.21.15.157:0/2077068361 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9eba59c640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc0b4490 0x7f9ebc14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebb59e640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ebc14f680 0x7f9ebc14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 -- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ebc14f680 msgr2=0x7f9ebc14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ebc14f680 0x7f9ebc14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 -- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc0b4490 msgr2=0x7f9ebc14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc0b4490 0x7f9ebc14ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 -- 172.21.15.157:0/2077068361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc0b36d0 con 0x7f9ea40047d0 2023-11-06T03:04:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ebad9d640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9eac001000 tx=0x7f9eac00e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac01a070 con 0x7f9ea40047d0 2023-11-06T03:04:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac00eb40 con 0x7f9ea40047d0 2023-11-06T03:04:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc0b39b0 con 0x7f9ea40047d0 2023-11-06T03:04:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac015400 con 0x7f9ea40047d0 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc0b2f20 con 0x7f9ea40047d0 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc072750 con 0x7f9ea40047d0 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9eac015560 con 0x7f9ea40047d0 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 0x7f9ea8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9eac086530 con 0x7f9ea40047d0 2023-11-06T03:04:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.478+0000 7f9eba59c640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 0x7f9ea8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.482+0000 7f9eba59c640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 0x7f9ea8067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00af000 tx=0x7f9eb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.482+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9eac053d60 con 0x7f9ea40047d0 2023-11-06T03:04:06.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.674+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9ebc06a580 con 0x7f9ea40047d0 2023-11-06T03:04:06.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.686+0000 7f9ea37fe640 1 -- 172.21.15.157:0/2077068361 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9eac057380 con 0x7f9ea40047d0 2023-11-06T03:04:06.689 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 3 and data pool 4 2023-11-06T03:04:06.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 msgr2=0x7f9ea8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 0x7f9ea8067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00af000 tx=0x7f9eb0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:06.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 msgr2=0x7f9ebc1ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9eac001000 tx=0x7f9eac00e100 comp rx=0 tx=0).stop 2023-11-06T03:04:06.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 shutdown_connections 2023-11-06T03:04:06.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ea8065a60 0x7f9ea8067f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ebc14f680 0x7f9ebc14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ebc0b4490 0x7f9ebc14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 --2- 172.21.15.157:0/2077068361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea40047d0 0x7f9ebc1ef8d0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9eac001000 tx=0x7f9eac00e100 comp rx=0 tx=0).stop 2023-11-06T03:04:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 >> 172.21.15.157:0/2077068361 conn(0x7f9ebc0732f0 msgr2=0x7f9ebc070860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:06.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.690+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 shutdown_connections 2023-11-06T03:04:06.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.694+0000 7f9ec1092640 1 -- 172.21.15.157:0/2077068361 wait complete. 2023-11-06T03:04:06.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:06.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.838+0000 7f2a08a12640 1 Processor -- start 2023-11-06T03:04:06.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.838+0000 7f2a08a12640 1 -- start start 2023-11-06T03:04:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a08a12640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2a0415ce80 0x7f2a040cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a08a12640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a040d2920 con 0x7f2a0415caa0 2023-11-06T03:04:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a08a12640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a040d18a0 con 0x7f2a040d00b0 2023-11-06T03:04:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a08a12640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a040d0520 con 0x7f2a0415ce80 2023-11-06T03:04:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2a0415caa0 0x7f2a040cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32900/0 (socket says 172.21.15.157:32900) 2023-11-06T03:04:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0259c640 1 -- 172.21.15.157:0/8715198 learned_addr learned my addr 172.21.15.157:0/8715198 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393802826 0 0) 0x7f2a040d2920 con 0x7f2a0415caa0 2023-11-06T03:04:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ec003530 con 0x7f2a0415caa0 2023-11-06T03:04:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a01d9b640 1 --2- 172.21.15.157:0/8715198 >> v2:172.21.15.195:3300/0 conn(0x7f2a0415ce80 0x7f2a040cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1482300462 0 0) 0x7f29ec003530 con 0x7f2a0415caa0 2023-11-06T03:04:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 >> v2:172.21.15.195:3300/0 conn(0x7f2a0415ce80 msgr2=0x7f2a040cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 --2- 172.21.15.157:0/8715198 >> v2:172.21.15.195:3300/0 conn(0x7f2a0415ce80 0x7f2a040cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 >> v1:172.21.15.195:6789/0 conn(0x7f2a040d00b0 legacy=0x7f2a040b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a040d1b60 con 0x7f2a0415caa0 2023-11-06T03:04:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f29f4001680 con 0x7f2a0415caa0 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 >> v1:172.21.15.157:6789/0 conn(0x7f2a0415caa0 legacy=0x7f2a040cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29ec003980 0x7f29ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29ec0047d0 0x7f29ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f29ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040d0520 con 0x7f29ec005190 2023-11-06T03:04:06.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec00c0e0 con 0x7f29ec003980 2023-11-06T03:04:06.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec00c2a0 con 0x7f29ec0047d0 2023-11-06T03:04:06.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f29ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0259c640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29ec003980 0x7f29ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 -- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29ec0047d0 msgr2=0x7f29ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29ec0047d0 0x7f29ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 -- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29ec003980 msgr2=0x7f29ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:06.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29ec003980 0x7f29ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 -- 172.21.15.157:0/8715198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec003530 con 0x7f29ec005190 2023-11-06T03:04:06.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0259c640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29ec003980 0x7f29ec003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:06.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 -- 172.21.15.157:0/8715198 reap_dead start 2023-11-06T03:04:06.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f29ec00b990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29fc001e70 tx=0x7f29fc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8cc9cca1f6aa95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.842+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc003b80 con 0x7f29ec005190 2023-11-06T03:04:06.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc003d20 con 0x7f29ec005190 2023-11-06T03:04:06.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0159a640 1 -- 172.21.15.157:0/8715198 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005260 con 0x7f29ec005190 2023-11-06T03:04:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 msgr2=0x7f29ec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f29ec00b990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29fc001e70 tx=0x7f29fc01c760 comp rx=0 tx=0).stop 2023-11-06T03:04:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/8715198 shutdown_connections 2023-11-06T03:04:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 --2- 172.21.15.157:0/8715198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f29ec00b990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/8715198 >> 172.21.15.157:0/8715198 conn(0x7f2a04070790 msgr2=0x7f2a04070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/8715198 shutdown_connections 2023-11-06T03:04:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/8715198 wait complete. 2023-11-06T03:04:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 Processor -- start 2023-11-06T03:04:06.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- start start 2023-11-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a040cc510 0x7f2a040ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 0x7f2a040ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040abfe0 con 0x7f29ec005190 2023-11-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040aa050 con 0x7f2a040cc990 2023-11-06T03:04:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04154840 con 0x7f2a040cc510 2023-11-06T03:04:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38874/0 (socket says 172.21.15.157:38874) 2023-11-06T03:04:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 -- 172.21.15.157:0/2512371879 learned_addr learned my addr 172.21.15.157:0/2512371879 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:06.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 0x7f2a040ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a01d9b640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a040cc510 0x7f2a040ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 -- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a040cc510 msgr2=0x7f2a040ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a040cc510 0x7f2a040ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 -- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 msgr2=0x7f2a040ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 0x7f2a040ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 -- 172.21.15.157:0/2512371879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a040a96b0 con 0x7f29ec005190 2023-11-06T03:04:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a02d9d640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 0x7f2a040ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:06.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a0259c640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d810 tx=0x7f29f400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40074e0 con 0x7f29ec005190 2023-11-06T03:04:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4007640 con 0x7f29ec005190 2023-11-06T03:04:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040a9990 con 0x7f29ec005190 2023-11-06T03:04:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004a10 con 0x7f29ec005190 2023-11-06T03:04:06.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.846+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0414de70 con 0x7f29ec005190 2023-11-06T03:04:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a040aa2c0 con 0x7f29ec005190 2023-11-06T03:04:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29f4004b70 con 0x7f29ec005190 2023-11-06T03:04:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f29f2ffd640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 0x7f29e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29f4082c60 con 0x7f29ec005190 2023-11-06T03:04:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f2a01d9b640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 0x7f29e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:06.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.850+0000 7f2a01d9b640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 0x7f29e4067f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29f80047c0 tx=0x7f29f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:06.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:06.854+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f29f40504c0 con 0x7f29ec005190 2023-11-06T03:04:07.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a04072750 con 0x7f29ec005190 2023-11-06T03:04:07.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f29f2ffd640 1 -- 172.21.15.157:0/2512371879 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17872 (secure 0 0 0) 0x7f29f40805e0 con 0x7f29ec005190 2023-11-06T03:04:07.012 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:07.012 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":22,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:06.679593+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 msgr2=0x7f29e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 0x7f29e4067f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29f80047c0 tx=0x7f29f8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 msgr2=0x7f2a04151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d810 tx=0x7f29f400dcd0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 shutdown_connections 2023-11-06T03:04:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29e4065a60 0x7f29e4067f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a040cc990 0x7f2a040ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a040cc510 0x7f2a040ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 --2- 172.21.15.157:0/2512371879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec005190 0x7f2a04151eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 >> 172.21.15.157:0/2512371879 conn(0x7f2a04070790 msgr2=0x7f2a040c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:07.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.010+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 shutdown_connections 2023-11-06T03:04:07.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.014+0000 7f2a08a12640 1 -- 172.21.15.157:0/2512371879 wait complete. 2023-11-06T03:04:07.031 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:07.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 Processor -- start 2023-11-06T03:04:07.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 -- start start 2023-11-06T03:04:07.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cc4156a70 0x7f2cc40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cc406f8d0 0x7f2cc40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc406c490 con 0x7f2cc406f470 2023-11-06T03:04:07.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc406b950 con 0x7f2cc406f8d0 2023-11-06T03:04:07.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc995a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc406bb60 con 0x7f2cc4156a70 2023-11-06T03:04:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc27fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2cc406f470 0x7f2cc40adae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32902/0 (socket says 172.21.15.157:32902) 2023-11-06T03:04:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2cc406f8d0 0x7f2cc40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc27fc640 1 -- 172.21.15.157:0/3554586312 learned_addr learned my addr 172.21.15.157:0/3554586312 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc2ffd640 1 --2- 172.21.15.157:0/3554586312 >> v2:172.21.15.195:3300/0 conn(0x7f2cc4156a70 0x7f2cc40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc37fe640 1 -- 172.21.15.157:0/3554586312 >> v2:172.21.15.195:3300/0 conn(0x7f2cc4156a70 msgr2=0x7f2cc40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:07.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc37fe640 1 --2- 172.21.15.157:0/3554586312 >> v2:172.21.15.195:3300/0 conn(0x7f2cc4156a70 0x7f2cc40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc37fe640 1 -- 172.21.15.157:0/3554586312 >> v1:172.21.15.157:6789/0 conn(0x7f2cc406f470 legacy=0x7f2cc40adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc1ffb640 1 -- 172.21.15.157:0/3554586312 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962419028 0 0) 0x7f2cc406c490 con 0x7f2cc406f470 2023-11-06T03:04:07.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.146+0000 7f2cc37fe640 1 -- 172.21.15.157:0/3554586312 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc406be40 con 0x7f2cc406f8d0 2023-11-06T03:04:07.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc37fe640 1 --2- 172.21.15.157:0/3554586312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f8d0 0x7f2cc40b1c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40ce880 tx=0x7f2cb401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1010070ad83cfc4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc1ffb640 1 -- 172.21.15.157:0/3554586312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40a58b0 con 0x7f2cc406f8d0 2023-11-06T03:04:07.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc1ffb640 1 -- 172.21.15.157:0/3554586312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb40a5a10 con 0x7f2cc406f8d0 2023-11-06T03:04:07.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc1ffb640 1 -- 172.21.15.157:0/3554586312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40a5da0 con 0x7f2cc406f8d0 2023-11-06T03:04:07.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- 172.21.15.157:0/3554586312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f8d0 msgr2=0x7f2cc40b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- 172.21.15.157:0/3554586312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f8d0 0x7f2cc40b1c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40ce880 tx=0x7f2cb401c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- 172.21.15.157:0/3554586312 shutdown_connections 2023-11-06T03:04:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- 172.21.15.157:0/3554586312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f8d0 0x7f2cc40b1c20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- 172.21.15.157:0/3554586312 >> v2:172.21.15.195:3300/0 conn(0x7f2cc4156a70 0x7f2cc40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- 172.21.15.157:0/3554586312 >> 172.21.15.157:0/3554586312 conn(0x7f2cc4065930 msgr2=0x7f2cc4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- 172.21.15.157:0/3554586312 shutdown_connections 2023-11-06T03:04:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- 172.21.15.157:0/3554586312 wait complete. 2023-11-06T03:04:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 Processor -- start 2023-11-06T03:04:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- start start 2023-11-06T03:04:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 0x7f2cc41f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cc4156a70 0x7f2cc41f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc414fba0 con 0x7f2cc406f470 2023-11-06T03:04:07.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40af260 con 0x7f2cc406f8d0 2023-11-06T03:04:07.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc995a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40af540 con 0x7f2cc4156a70 2023-11-06T03:04:07.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38894/0 (socket says 172.21.15.157:38894) 2023-11-06T03:04:07.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.150+0000 7f2cc2ffd640 1 -- 172.21.15.157:0/985973641 learned_addr learned my addr 172.21.15.157:0/985973641 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:07.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc37fe640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cc4156a70 0x7f2cc41f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 -- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cc4156a70 msgr2=0x7f2cc41f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cc4156a70 0x7f2cc41f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc27fc640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 0x7f2cc41f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 -- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 msgr2=0x7f2cc41f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 0x7f2cc41f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 -- 172.21.15.157:0/985973641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009a70 con 0x7f2cc406f470 2023-11-06T03:04:07.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc27fc640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 0x7f2cc41f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:07.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc2ffd640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e870 tx=0x7f2cbc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc009510 con 0x7f2cc406f470 2023-11-06T03:04:07.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc00eec0 con 0x7f2cc406f470 2023-11-06T03:04:07.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00cb40 con 0x7f2cc406f470 2023-11-06T03:04:07.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc40af820 con 0x7f2cc406f470 2023-11-06T03:04:07.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc40b10b0 con 0x7f2cc406f470 2023-11-06T03:04:07.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc4072750 con 0x7f2cc406f470 2023-11-06T03:04:07.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2cbc00ccc0 con 0x7f2cc406f470 2023-11-06T03:04:07.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 0x7f2cb0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2cbc083400 con 0x7f2cc406f470 2023-11-06T03:04:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc27fc640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 0x7f2cb0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.154+0000 7f2cc27fc640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 0x7f2cb0067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2cc414eef0 tx=0x7f2cb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.158+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2cbc050bd0 con 0x7f2cc406f470 2023-11-06T03:04:07.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.318+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cc4157100 con 0x7f2cc406f470 2023-11-06T03:04:07.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.318+0000 7f2cabfff640 1 -- 172.21.15.157:0/985973641 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17872 (secure 0 0 0) 0x7f2cbc0541f0 con 0x7f2cc406f470 2023-11-06T03:04:07.322 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:07.322 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":22,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:06.679593+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:07.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 msgr2=0x7f2cb0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 0x7f2cb0067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2cc414eef0 tx=0x7f2cb8009040 comp rx=0 tx=0).stop 2023-11-06T03:04:07.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 msgr2=0x7f2cc40afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e870 tx=0x7f2cbc00ec70 comp rx=0 tx=0).stop 2023-11-06T03:04:07.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 shutdown_connections 2023-11-06T03:04:07.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cb0065a60 0x7f2cb0067f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cc4156a70 0x7f2cc41f3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cc406f8d0 0x7f2cc41f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 --2- 172.21.15.157:0/985973641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc406f470 0x7f2cc40afd40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 >> 172.21.15.157:0/985973641 conn(0x7f2cc4065930 msgr2=0x7f2cc40b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:07.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 shutdown_connections 2023-11-06T03:04:07.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.322+0000 7f2cc995a640 1 -- 172.21.15.157:0/985973641 wait complete. 2023-11-06T03:04:07.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:04:07.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 Processor -- start 2023-11-06T03:04:07.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 -- start start 2023-11-06T03:04:07.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3b9c15aeb0 0x7f3b9c071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9c15b290 0x7f3b9c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9c0b3c80 con 0x7f3b9c0afc00 2023-11-06T03:04:07.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9c0ce890 con 0x7f3b9c15b290 2023-11-06T03:04:07.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba3df4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9c0af640 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9c15b290 0x7f3b9c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b9c15b290 0x7f3b9c0b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38898/0 (socket says 172.21.15.157:38898) 2023-11-06T03:04:07.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 -- 172.21.15.157:0/2576710375 learned_addr learned my addr 172.21.15.157:0/2576710375 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:07.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 -- 172.21.15.157:0/2576710375 >> v2:172.21.15.195:3300/0 conn(0x7f3b9c15aeb0 msgr2=0x7f3b9c071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:07.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 --2- 172.21.15.157:0/2576710375 >> v2:172.21.15.195:3300/0 conn(0x7f3b9c15aeb0 0x7f3b9c071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 -- 172.21.15.157:0/2576710375 >> v1:172.21.15.157:6789/0 conn(0x7f3b9c0afc00 legacy=0x7f3b9c0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:07.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.466+0000 7f3ba138f640 1 -- 172.21.15.157:0/2576710375 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0af920 con 0x7f3b9c15b290 2023-11-06T03:04:07.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 --2- 172.21.15.157:0/2576710375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15b290 0x7f3b9c0b3740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c066930 tx=0x7f3b8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1ab49414219687 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba0b8e640 1 -- 172.21.15.157:0/2576710375 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0aa430 con 0x7f3b9c15b290 2023-11-06T03:04:07.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba0b8e640 1 -- 172.21.15.157:0/2576710375 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c0aa590 con 0x7f3b9c15b290 2023-11-06T03:04:07.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba0b8e640 1 -- 172.21.15.157:0/2576710375 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0aa920 con 0x7f3b9c15b290 2023-11-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- 172.21.15.157:0/2576710375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15b290 msgr2=0x7f3b9c0b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/2576710375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15b290 0x7f3b9c0b3740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c066930 tx=0x7f3b8c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- 172.21.15.157:0/2576710375 shutdown_connections 2023-11-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/2576710375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15b290 0x7f3b9c0b3740 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/2576710375 >> v2:172.21.15.195:3300/0 conn(0x7f3b9c15aeb0 0x7f3b9c071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- 172.21.15.157:0/2576710375 >> 172.21.15.157:0/2576710375 conn(0x7f3b9c0cd370 msgr2=0x7f3b9c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:07.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- 172.21.15.157:0/2576710375 shutdown_connections 2023-11-06T03:04:07.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- 172.21.15.157:0/2576710375 wait complete. 2023-11-06T03:04:07.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 Processor -- start 2023-11-06T03:04:07.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- start start 2023-11-06T03:04:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b9c0afc00 0x7f3b9c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b9c15b290 0x7f3b9c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c152e10 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c159c60 con 0x7f3b9c0afc00 2023-11-06T03:04:07.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba3df4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c155350 con 0x7f3b9c15b290 2023-11-06T03:04:07.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38904/0 (socket says 172.21.15.157:38904) 2023-11-06T03:04:07.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 -- 172.21.15.157:0/1227346944 learned_addr learned my addr 172.21.15.157:0/1227346944 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:07.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 -- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b9c15b290 msgr2=0x7f3b9c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b9c15b290 0x7f3b9c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 -- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b9c0afc00 msgr2=0x7f3b9c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:07.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b9c0afc00 0x7f3b9c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:07.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.470+0000 7f3ba138f640 1 -- 172.21.15.157:0/1227346944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0094d0 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba138f640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0190e0 tx=0x7f3b8c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0b3800 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c0b3960 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c155630 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0b3d10 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c155bf0 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c069d70 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b8c0c6440 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 0x7f3b80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b8c11de70 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba1b90640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 0x7f3b80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:07.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.474+0000 7f3ba1b90640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 0x7f3b80067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009730 tx=0x7f3b98006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:07.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.478+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3b8c0eb760 con 0x7f3b9c15aeb0 2023-11-06T03:04:07.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:07.670+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3b9c072750 con 0x7f3b9c15aeb0 2023-11-06T03:04:08.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.314+0000 7f3b927fc640 1 -- 172.21.15.157:0/1227346944 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3b8c0eed80 con 0x7f3b9c15aeb0 2023-11-06T03:04:08.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 msgr2=0x7f3b80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:08.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 0x7f3b80067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009730 tx=0x7f3b98006ca0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 msgr2=0x7f3b9c159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:08.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0190e0 tx=0x7f3b8c01ad10 comp rx=0 tx=0).stop 2023-11-06T03:04:08.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 shutdown_connections 2023-11-06T03:04:08.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b80065a60 0x7f3b80067f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b9c15b290 0x7f3b9c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b9c15aeb0 0x7f3b9c159720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 --2- 172.21.15.157:0/1227346944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b9c0afc00 0x7f3b9c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 >> 172.21.15.157:0/1227346944 conn(0x7f3b9c0cd370 msgr2=0x7f3b9c0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 shutdown_connections 2023-11-06T03:04:08.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.318+0000 7f3ba3df4640 1 -- 172.21.15.157:0/1227346944 wait complete. 2023-11-06T03:04:08.339 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:04:08.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:04:08.413 INFO:tasks.ceph.mds.d.smithi195.stderr:2023-11-06T03:04:08.414+0000 7f39abacc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:04:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.462+0000 7f9797a26640 1 Processor -- start 2023-11-06T03:04:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.462+0000 7f9797a26640 1 -- start start 2023-11-06T03:04:08.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f979006d000 0x7f97900b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f979015dea0 0x7f97900b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979015e440 con 0x7f979006d000 2023-11-06T03:04:08.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979006c240 con 0x7f979015dea0 2023-11-06T03:04:08.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979006c520 con 0x7f97901599c0 2023-11-06T03:04:08.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f979006d000 0x7f97900b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f979006d000 0x7f97900b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38906/0 (socket says 172.21.15.157:38906) 2023-11-06T03:04:08.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 -- 172.21.15.157:0/147229299 learned_addr learned my addr 172.21.15.157:0/147229299 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:08.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9795fc3640 1 --2- 172.21.15.157:0/147229299 >> v2:172.21.15.195:3300/0 conn(0x7f979015dea0 0x7f97900b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f97957c2640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 -- 172.21.15.157:0/147229299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 msgr2=0x7f97900ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:08.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97900ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 -- 172.21.15.157:0/147229299 >> v2:172.21.15.195:3300/0 conn(0x7f979015dea0 msgr2=0x7f97900b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:08.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 --2- 172.21.15.157:0/147229299 >> v2:172.21.15.195:3300/0 conn(0x7f979015dea0 0x7f97900b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 -- 172.21.15.157:0/147229299 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979006c800 con 0x7f979006d000 2023-11-06T03:04:08.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f97957c2640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97900ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9794fc1640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900b23e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f979006bbf0 tx=0x7f978001e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605df1ec6df55d4f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9787fff640 1 -- 172.21.15.157:0/147229299 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800ab480 con 0x7f979006d000 2023-11-06T03:04:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9787fff640 1 -- 172.21.15.157:0/147229299 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97800ab5e0 con 0x7f979006d000 2023-11-06T03:04:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9787fff640 1 -- 172.21.15.157:0/147229299 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800ab970 con 0x7f979006d000 2023-11-06T03:04:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 -- 172.21.15.157:0/147229299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 msgr2=0x7f97900b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.466+0000 7f9797a26640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900b23e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f979006bbf0 tx=0x7f978001e6e0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/147229299 shutdown_connections 2023-11-06T03:04:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- 172.21.15.157:0/147229299 >> v2:172.21.15.195:3300/0 conn(0x7f979015dea0 0x7f97900b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900b23e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- 172.21.15.157:0/147229299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97900ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/147229299 >> 172.21.15.157:0/147229299 conn(0x7f9790065930 msgr2=0x7f9790065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/147229299 shutdown_connections 2023-11-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/147229299 wait complete. 2023-11-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 Processor -- start 2023-11-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- start start 2023-11-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97901f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f979015dea0 0x7f9790157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97901f8120 con 0x7f979006d000 2023-11-06T03:04:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900cbb70 con 0x7f979015dea0 2023-11-06T03:04:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900cbe50 con 0x7f97901599c0 2023-11-06T03:04:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38908/0 (socket says 172.21.15.157:38908) 2023-11-06T03:04:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 -- 172.21.15.157:0/2950238701 learned_addr learned my addr 172.21.15.157:0/2950238701 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9795fc3640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f979015dea0 0x7f9790157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9794fc1640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97901f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 -- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 msgr2=0x7f97901f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97901f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 -- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f979015dea0 msgr2=0x7f9790157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f979015dea0 0x7f9790157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 -- 172.21.15.157:0/2950238701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800094d0 con 0x7f979006d000 2023-11-06T03:04:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f97957c2640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c7c0 tx=0x7f978c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c007a30 con 0x7f979006d000 2023-11-06T03:04:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c007b90 con 0x7f979006d000 2023-11-06T03:04:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97900cc130 con 0x7f979006d000 2023-11-06T03:04:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c00ce70 con 0x7f979006d000 2023-11-06T03:04:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9790158000 con 0x7f979006d000 2023-11-06T03:04:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.470+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97900cc650 con 0x7f979006d000 2023-11-06T03:04:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.474+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f978c0113f0 con 0x7f979006d000 2023-11-06T03:04:08.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.474+0000 7f9785ffb640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 0x7f9774067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.474+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f978c083810 con 0x7f979006d000 2023-11-06T03:04:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.474+0000 7f9794fc1640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 0x7f9774067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.474+0000 7f9794fc1640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 0x7f9774067f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97800b0000 tx=0x7f97800a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.478+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f978c051070 con 0x7f979006d000 2023-11-06T03:04:08.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:08.662+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9790072750 con 0x7f979006d000 2023-11-06T03:04:09.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.330+0000 7f9785ffb640 1 -- 172.21.15.157:0/2950238701 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f978c054690 con 0x7f979006d000 2023-11-06T03:04:09.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 msgr2=0x7f9774067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 0x7f9774067f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f97800b0000 tx=0x7f97800a9000 comp rx=0 tx=0).stop 2023-11-06T03:04:09.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 msgr2=0x7f97900ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f978c00c7c0 tx=0x7f978c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:04:09.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 shutdown_connections 2023-11-06T03:04:09.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9774065a60 0x7f9774067f20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f979015dea0 0x7f9790157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97901599c0 0x7f97901f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 --2- 172.21.15.157:0/2950238701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f979006d000 0x7f97900ce550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 >> 172.21.15.157:0/2950238701 conn(0x7f9790065930 msgr2=0x7f97901589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 shutdown_connections 2023-11-06T03:04:09.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.334+0000 7f9797a26640 1 -- 172.21.15.157:0/2950238701 wait complete. 2023-11-06T03:04:09.358 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:04:09.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:04:09.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:04:09.403 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:04:09.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.482+0000 7fce57d87640 1 Processor -- start 2023-11-06T03:04:09.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.482+0000 7fce57d87640 1 -- start start 2023-11-06T03:04:09.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.482+0000 7fce57d87640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce50159930 0x7fce500afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.482+0000 7fce57d87640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce500b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce500af110 con 0x7fce50070330 2023-11-06T03:04:09.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce500afa20 con 0x7fce50159930 2023-11-06T03:04:09.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b0380 con 0x7fce5006da00 2023-11-06T03:04:09.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce50159930 0x7fce500afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce56324640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fce50070330 0x7fce500cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43476/0 (socket says 172.21.15.157:43476) 2023-11-06T03:04:09.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce56324640 1 -- 172.21.15.157:0/4242320560 learned_addr learned my addr 172.21.15.157:0/4242320560 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:09.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55322640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce500b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 -- 172.21.15.157:0/4242320560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 msgr2=0x7fce500b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:09.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce500b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 -- 172.21.15.157:0/4242320560 >> v1:172.21.15.157:6789/0 conn(0x7fce50070330 legacy=0x7fce500cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 -- 172.21.15.157:0/4242320560 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5006df50 con 0x7fce50159930 2023-11-06T03:04:09.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce54b21640 1 -- 172.21.15.157:0/4242320560 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776719664 0 0) 0x7fce500af110 con 0x7fce50070330 2023-11-06T03:04:09.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce55b23640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce500afe40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097f0 tx=0x7fce4c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565cf6be0923ec1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce54b21640 1 -- 172.21.15.157:0/4242320560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0ab600 con 0x7fce50159930 2023-11-06T03:04:09.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce54b21640 1 -- 172.21.15.157:0/4242320560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c0a7050 con 0x7fce50159930 2023-11-06T03:04:09.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce54b21640 1 -- 172.21.15.157:0/4242320560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0ab980 con 0x7fce50159930 2023-11-06T03:04:09.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- 172.21.15.157:0/4242320560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 msgr2=0x7fce500afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce500afe40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097f0 tx=0x7fce4c01c3f0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- 172.21.15.157:0/4242320560 shutdown_connections 2023-11-06T03:04:09.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce500b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 --2- 172.21.15.157:0/4242320560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce500afe40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- 172.21.15.157:0/4242320560 >> 172.21.15.157:0/4242320560 conn(0x7fce500732f0 msgr2=0x7fce50069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:09.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- 172.21.15.157:0/4242320560 shutdown_connections 2023-11-06T03:04:09.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.486+0000 7fce57d87640 1 -- 172.21.15.157:0/4242320560 wait complete. 2023-11-06T03:04:09.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 Processor -- start 2023-11-06T03:04:09.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- start start 2023-11-06T03:04:09.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce50148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce50070330 0x7fce50152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5014bee0 con 0x7fce50159930 2023-11-06T03:04:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce50152cb0 con 0x7fce50070330 2023-11-06T03:04:09.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce501498a0 con 0x7fce5006da00 2023-11-06T03:04:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43542/0 (socket says 172.21.15.157:43542) 2023-11-06T03:04:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 -- 172.21.15.157:0/3277382009 learned_addr learned my addr 172.21.15.157:0/3277382009 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:09.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce55b23640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce50148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce55322640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce50070330 0x7fce50152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 -- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 msgr2=0x7fce50148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce50148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 -- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce50070330 msgr2=0x7fce50152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce50070330 0x7fce50152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 -- 172.21.15.157:0/3277382009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c009420 con 0x7fce50159930 2023-11-06T03:04:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce56324640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce48001570 tx=0x7fce4800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800a640 con 0x7fce50159930 2023-11-06T03:04:09.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800a7c0 con 0x7fce50159930 2023-11-06T03:04:09.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce50070cd0 con 0x7fce50159930 2023-11-06T03:04:09.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800ab70 con 0x7fce50159930 2023-11-06T03:04:09.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce50149000 con 0x7fce50159930 2023-11-06T03:04:09.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.490+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5014c050 con 0x7fce50159930 2023-11-06T03:04:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.494+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fce4800acd0 con 0x7fce50159930 2023-11-06T03:04:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.494+0000 7fce467fc640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 0x7fce34067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.494+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce48082b60 con 0x7fce50159930 2023-11-06T03:04:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.494+0000 7fce55b23640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 0x7fce34067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.494+0000 7fce55b23640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 0x7fce34067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ad000 tx=0x7fce4c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.498+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fce480503c0 con 0x7fce50159930 2023-11-06T03:04:09.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.682+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce5006acd0 con 0x7fce50159930 2023-11-06T03:04:09.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.682+0000 7fce467fc640 1 -- 172.21.15.157:0/3277382009 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fce48018090 con 0x7fce50159930 2023-11-06T03:04:09.685 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:09.685 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:04:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:04:09.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 msgr2=0x7fce34067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 0x7fce34067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ad000 tx=0x7fce4c0a6000 comp rx=0 tx=0).stop 2023-11-06T03:04:09.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 msgr2=0x7fce5014a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce48001570 tx=0x7fce4800cca0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 shutdown_connections 2023-11-06T03:04:09.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce34065a60 0x7fce34067f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50159930 0x7fce5014a420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce50070330 0x7fce50152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 --2- 172.21.15.157:0/3277382009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce5006da00 0x7fce50148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 >> 172.21.15.157:0/3277382009 conn(0x7fce500732f0 msgr2=0x7fce50067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:09.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 shutdown_connections 2023-11-06T03:04:09.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.686+0000 7fce57d87640 1 -- 172.21.15.157:0/3277382009 wait complete. 2023-11-06T03:04:09.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:09.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.866+0000 7fc4cb8da640 1 Processor -- start 2023-11-06T03:04:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.866+0000 7fc4cb8da640 1 -- start start 2023-11-06T03:04:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c41551e0 0x7fc4c406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 0x7fc4c415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c415f580 con 0x7fc4c415e5e0 2023-11-06T03:04:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c415efc0 con 0x7fc4c41551e0 2023-11-06T03:04:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c415f2a0 con 0x7fc4c40b3b20 2023-11-06T03:04:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4c41551e0 0x7fc4c406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c8e75640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc4c415e5e0 0x7fc4c40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43480/0 (socket says 172.21.15.157:43480) 2023-11-06T03:04:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c8e75640 1 -- 172.21.15.157:0/1251731953 learned_addr learned my addr 172.21.15.157:0/1251731953 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9e77640 1 --2- 172.21.15.157:0/1251731953 >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 0x7fc4c415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 -- 172.21.15.157:0/1251731953 >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 msgr2=0x7fc4c415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4bbfff640 1 -- 172.21.15.157:0/1251731953 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711647277 0 0) 0x7fc4c415f580 con 0x7fc4c415e5e0 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 --2- 172.21.15.157:0/1251731953 >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 0x7fc4c415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 -- 172.21.15.157:0/1251731953 >> v1:172.21.15.157:6789/0 conn(0x7fc4c415e5e0 legacy=0x7fc4c40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 -- 172.21.15.157:0/1251731953 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c406f5a0 con 0x7fc4c41551e0 2023-11-06T03:04:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9e77640 1 --2- 172.21.15.157:0/1251731953 >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 0x7fc4c415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4c9676640 1 --2- 172.21.15.157:0/1251731953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c41551e0 0x7fc4c406fda0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cbfe0 tx=0x7fc4c001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee57f787691baa49 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4bbfff640 1 -- 172.21.15.157:0/1251731953 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab600 con 0x7fc4c41551e0 2023-11-06T03:04:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4bbfff640 1 -- 172.21.15.157:0/1251731953 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c00a7050 con 0x7fc4c41551e0 2023-11-06T03:04:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4bbfff640 1 -- 172.21.15.157:0/1251731953 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab980 con 0x7fc4c41551e0 2023-11-06T03:04:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- 172.21.15.157:0/1251731953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c41551e0 msgr2=0x7fc4c406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/1251731953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c41551e0 0x7fc4c406fda0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cbfe0 tx=0x7fc4c001c310 comp rx=0 tx=0).stop 2023-11-06T03:04:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- 172.21.15.157:0/1251731953 shutdown_connections 2023-11-06T03:04:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/1251731953 >> v2:172.21.15.195:3300/0 conn(0x7fc4c40b3b20 0x7fc4c415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/1251731953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c41551e0 0x7fc4c406fda0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- 172.21.15.157:0/1251731953 >> 172.21.15.157:0/1251731953 conn(0x7fc4c4065930 msgr2=0x7fc4c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- 172.21.15.157:0/1251731953 shutdown_connections 2023-11-06T03:04:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.870+0000 7fc4cb8da640 1 -- 172.21.15.157:0/1251731953 wait complete. 2023-11-06T03:04:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 Processor -- start 2023-11-06T03:04:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- start start 2023-11-06T03:04:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c41551e0 0x7fc4c414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 0x7fc4c4150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40ae070 con 0x7fc4c40b3b20 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c406c0a0 con 0x7fc4c415e5e0 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c406c380 con 0x7fc4c41551e0 2023-11-06T03:04:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43558/0 (socket says 172.21.15.157:43558) 2023-11-06T03:04:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 -- 172.21.15.157:0/246426449 learned_addr learned my addr 172.21.15.157:0/246426449 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9e77640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 0x7fc4c4150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c8e75640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c41551e0 0x7fc4c414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 -- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c41551e0 msgr2=0x7fc4c414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c41551e0 0x7fc4c414d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 -- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 msgr2=0x7fc4c4150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 0x7fc4c4150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 -- 172.21.15.157:0/246426449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009420 con 0x7fc4c40b3b20 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9e77640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 0x7fc4c4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4c9676640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b7650 con 0x7fc4c40b3b20 2023-11-06T03:04:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c00b77d0 con 0x7fc4c40b3b20 2023-11-06T03:04:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b7b80 con 0x7fc4c40b3b20 2023-11-06T03:04:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c406c660 con 0x7fc4c40b3b20 2023-11-06T03:04:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c406cc20 con 0x7fc4c40b3b20 2023-11-06T03:04:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.874+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c406ce10 con 0x7fc4c40b3b20 2023-11-06T03:04:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.878+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4c00b7ce0 con 0x7fc4c40b3b20 2023-11-06T03:04:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.878+0000 7fc4b9ffb640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 0x7fc4a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.878+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4c011df70 con 0x7fc4c40b3b20 2023-11-06T03:04:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.878+0000 7fc4c8e75640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 0x7fc4a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.878+0000 7fc4c8e75640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 0x7fc4a8067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cef70 tx=0x7fc4b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:09.882+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4c00eb940 con 0x7fc4c40b3b20 2023-11-06T03:04:10.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.030+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c4072750 con 0x7fc4c40b3b20 2023-11-06T03:04:10.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.034+0000 7fc4b9ffb640 1 -- 172.21.15.157:0/246426449 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fc4c00bb090 con 0x7fc4c40b3b20 2023-11-06T03:04:10.037 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:10.037 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:10.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 msgr2=0x7fc4a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 0x7fc4a8067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cef70 tx=0x7fc4b4009040 comp rx=0 tx=0).stop 2023-11-06T03:04:10.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 msgr2=0x7fc4c406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001ab50 comp rx=0 tx=0).stop 2023-11-06T03:04:10.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 shutdown_connections 2023-11-06T03:04:10.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4a8065a60 0x7fc4a8067f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4c415e5e0 0x7fc4c4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4c41551e0 0x7fc4c414d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 --2- 172.21.15.157:0/246426449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c40b3b20 0x7fc4c406d140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 >> 172.21.15.157:0/246426449 conn(0x7fc4c4065930 msgr2=0x7fc4c40af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 shutdown_connections 2023-11-06T03:04:10.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.038+0000 7fc4cb8da640 1 -- 172.21.15.157:0/246426449 wait complete. 2023-11-06T03:04:10.055 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:04:10.056 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 Processor -- start 2023-11-06T03:04:10.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 -- start start 2023-11-06T03:04:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1208156a70 0x7f120806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120806c490 con 0x7f12080b2330 2023-11-06T03:04:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120806bb60 con 0x7f12080adae0 2023-11-06T03:04:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120ec61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120806b950 con 0x7f1208156a70 2023-11-06T03:04:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.162+0000 7f120c9fd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1208156a70 0x7f120806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120d1fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f12080b2330 0x7f120806c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43486/0 (socket says 172.21.15.157:43486) 2023-11-06T03:04:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120c9fd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1208156a70 0x7f120806f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43572/0 (socket says 172.21.15.157:43572) 2023-11-06T03:04:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120d1fe640 1 -- 172.21.15.157:0/3137944415 learned_addr learned my addr 172.21.15.157:0/3137944415 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120c9fd640 1 -- 172.21.15.157:0/3137944415 >> v1:172.21.15.195:6789/0 conn(0x7f12080adae0 legacy=0x7f12080b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120c9fd640 1 -- 172.21.15.157:0/3137944415 >> v1:172.21.15.157:6789/0 conn(0x7f12080b2330 legacy=0x7f120806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120c9fd640 1 -- 172.21.15.157:0/3137944415 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120806be40 con 0x7f1208156a70 2023-11-06T03:04:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120c9fd640 1 --2- 172.21.15.157:0/3137944415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1208156a70 0x7f120806f470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a960 tx=0x7f11f801d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd8517668398bc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f12077fe640 1 -- 172.21.15.157:0/3137944415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f80a68d0 con 0x7f1208156a70 2023-11-06T03:04:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f12077fe640 1 -- 172.21.15.157:0/3137944415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f80a6a30 con 0x7f1208156a70 2023-11-06T03:04:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f12077fe640 1 -- 172.21.15.157:0/3137944415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f80a6dc0 con 0x7f1208156a70 2023-11-06T03:04:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 -- 172.21.15.157:0/3137944415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1208156a70 msgr2=0x7f120806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 --2- 172.21.15.157:0/3137944415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1208156a70 0x7f120806f470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a960 tx=0x7f11f801d7e0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 -- 172.21.15.157:0/3137944415 shutdown_connections 2023-11-06T03:04:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 --2- 172.21.15.157:0/3137944415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1208156a70 0x7f120806f470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 -- 172.21.15.157:0/3137944415 >> 172.21.15.157:0/3137944415 conn(0x7f1208065930 msgr2=0x7f1208065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 -- 172.21.15.157:0/3137944415 shutdown_connections 2023-11-06T03:04:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.166+0000 7f120ec61640 1 -- 172.21.15.157:0/3137944415 wait complete. 2023-11-06T03:04:10.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 Processor -- start 2023-11-06T03:04:10.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- start start 2023-11-06T03:04:10.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080adae0 0x7f12081f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1208156a70 0x7f12080acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12081f8880 con 0x7f12080b2330 2023-11-06T03:04:10.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120814af20 con 0x7f12080adae0 2023-11-06T03:04:10.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120814b200 con 0x7f1208156a70 2023-11-06T03:04:10.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43574/0 (socket says 172.21.15.157:43574) 2023-11-06T03:04:10.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 -- 172.21.15.157:0/3133960524 learned_addr learned my addr 172.21.15.157:0/3133960524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:10.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 -- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1208156a70 msgr2=0x7f12080acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120d1fe640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1208156a70 0x7f12080acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120c9fd640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080adae0 0x7f12081f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1208156a70 0x7f12080acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 -- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080adae0 msgr2=0x7f12081f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080adae0 0x7f12081f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 -- 172.21.15.157:0/3133960524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f800a5c0 con 0x7f12080b2330 2023-11-06T03:04:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f1207fff640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c450 tx=0x7f11fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc012830 con 0x7f12080b2330 2023-11-06T03:04:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc0129b0 con 0x7f12080b2330 2023-11-06T03:04:10.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc012d60 con 0x7f12080b2330 2023-11-06T03:04:10.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12080af120 con 0x7f12080b2330 2023-11-06T03:04:10.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120814b4e0 con 0x7f12080b2330 2023-11-06T03:04:10.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080af640 con 0x7f12080b2330 2023-11-06T03:04:10.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.170+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f11fc01c460 con 0x7f12080b2330 2023-11-06T03:04:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.174+0000 7f12057fa640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 0x7f11f4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.174+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11fc083040 con 0x7f12080b2330 2023-11-06T03:04:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.174+0000 7f120c9fd640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 0x7f11f4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.174+0000 7f120c9fd640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 0x7f11f4067f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b0000 tx=0x7f11f80a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.178+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f12080af640 con 0x7f12080b2330 2023-11-06T03:04:10.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.314+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f120806f9b0 con 0x7f11f4065a90 2023-11-06T03:04:10.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.338+0000 7f12057fa640 1 -- 172.21.15.157:0/3133960524 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271887 (secure 0 0 0) 0x7f11f80b9420 con 0x7f11f4065a90 2023-11-06T03:04:10.341 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 msgr2=0x7f11f4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 0x7f11f4067f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b0000 tx=0x7f11f80a8070 comp rx=0 tx=0).stop 2023-11-06T03:04:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 msgr2=0x7f12080ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c450 tx=0x7f11fc00c910 comp rx=0 tx=0).stop 2023-11-06T03:04:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 shutdown_connections 2023-11-06T03:04:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f11f4065a90 0x7f11f4067f50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1208156a70 0x7f12080acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12080b2330 0x7f12080ac8b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 --2- 172.21.15.157:0/3133960524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12080adae0 0x7f12081f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 >> 172.21.15.157:0/3133960524 conn(0x7f1208065930 msgr2=0x7f12081584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 shutdown_connections 2023-11-06T03:04:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.342+0000 7f120ec61640 1 -- 172.21.15.157:0/3133960524 wait complete. 2023-11-06T03:04:10.358 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T03:04:09.093887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250976,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723744,"statfs":{"total":773094113280,"available":772837113856,"internally_reserved":0,"allocated":2654208,"data_stored":1653163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001232"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.273409+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:02.273409+0000","last_peered":"2023-11-06T03:04:02.273409+0000","last_clean":"2023-11-06T03:04:02.273409+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:02.273409+0000","last_undegraded":"2023-11-06T03:04:02.273409+0000","last_fullsized":"2023-11-06T03:04:02.273409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281441+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:02.281441+0000","last_peered":"2023-11-06T03:04:02.281441+0000","last_clean":"2023-11-06T03:04:02.281441+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:02.281441+0000","last_undegraded":"2023-11-06T03:04:02.281441+0000","last_fullsized":"2023-11-06T03:04:02.281441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710322+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:03.710322+0000","last_peered":"2023-11-06T03:04:03.710322+0000","last_clean":"2023-11-06T03:04:03.710322+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:03.710322+0000","last_undegraded":"2023-11-06T03:04:03.710322+0000","last_fullsized":"2023-11-06T03:04:03.710322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.651934+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:02.651934+0000","last_peered":"2023-11-06T03:04:02.651934+0000","last_clean":"2023-11-06T03:04:02.651934+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:02.651934+0000","last_undegraded":"2023-11-06T03:04:02.651934+0000","last_fullsized":"2023-11-06T03:04:02.651934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281388+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:02.281388+0000","last_peered":"2023-11-06T03:04:02.281388+0000","last_clean":"2023-11-06T03:04:02.281388+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:02.281388+0000","last_undegraded":"2023-11-06T03:04:02.281388+0000","last_fullsized":"2023-11-06T03:04:02.281388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.622854+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:02.622854+0000","last_peered":"2023-11-06T03:04:02.622854+0000","last_clean":"2023-11-06T03:04:02.622854+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:02.622854+0000","last_undegraded":"2023-11-06T03:04:02.622854+0000","last_fullsized":"2023-11-06T03:04:02.622854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710323+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:03.710323+0000","last_peered":"2023-11-06T03:04:03.710323+0000","last_clean":"2023-11-06T03:04:03.710323+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:03.710323+0000","last_undegraded":"2023-11-06T03:04:03.710323+0000","last_fullsized":"2023-11-06T03:04:03.710323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.277044+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:02.277044+0000","last_peered":"2023-11-06T03:04:02.277044+0000","last_clean":"2023-11-06T03:04:02.277044+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:02.277044+0000","last_undegraded":"2023-11-06T03:04:02.277044+0000","last_fullsized":"2023-11-06T03:04:02.277044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.280926+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:02.280926+0000","last_peered":"2023-11-06T03:04:02.280926+0000","last_clean":"2023-11-06T03:04:02.280926+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:02.280926+0000","last_undegraded":"2023-11-06T03:04:02.280926+0000","last_fullsized":"2023-11-06T03:04:02.280926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:10.368 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:10.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 Processor -- start 2023-11-06T03:04:10.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 -- start start 2023-11-06T03:04:10.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f336c1599c0 0x7f336c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c06d000 0x7f336c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c15dea0 0x7f336c0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c15e440 con 0x7f336c15dea0 2023-11-06T03:04:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c06c240 con 0x7f336c1599c0 2023-11-06T03:04:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3370c04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c06c520 con 0x7f336c06d000 2023-11-06T03:04:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c15dea0 0x7f336c0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f336c15dea0 0x7f336c0b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43584/0 (socket says 172.21.15.157:43584) 2023-11-06T03:04:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 -- 172.21.15.157:0/2475050368 learned_addr learned my addr 172.21.15.157:0/2475050368 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:10.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f3369d9b640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c06d000 0x7f336c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336a59c640 1 --2- 172.21.15.157:0/2475050368 >> v2:172.21.15.195:3300/0 conn(0x7f336c1599c0 0x7f336c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 -- 172.21.15.157:0/2475050368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c06d000 msgr2=0x7f336c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c06d000 0x7f336c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 -- 172.21.15.157:0/2475050368 >> v2:172.21.15.195:3300/0 conn(0x7f336c1599c0 msgr2=0x7f336c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 --2- 172.21.15.157:0/2475050368 >> v2:172.21.15.195:3300/0 conn(0x7f336c1599c0 0x7f336c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.514+0000 7f336ad9d640 1 -- 172.21.15.157:0/2475050368 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c06c800 con 0x7f336c15dea0 2023-11-06T03:04:10.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f336ad9d640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0b2920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f336c066800 tx=0x7f336401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12529f99138dc1ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f336959a640 1 -- 172.21.15.157:0/2475050368 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640a5840 con 0x7f336c15dea0 2023-11-06T03:04:10.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f336959a640 1 -- 172.21.15.157:0/2475050368 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33640a59a0 con 0x7f336c15dea0 2023-11-06T03:04:10.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f336959a640 1 -- 172.21.15.157:0/2475050368 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640a5d30 con 0x7f336c15dea0 2023-11-06T03:04:10.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- 172.21.15.157:0/2475050368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 msgr2=0x7f336c0b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0b2920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f336c066800 tx=0x7f336401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- 172.21.15.157:0/2475050368 shutdown_connections 2023-11-06T03:04:10.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0b2920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 --2- 172.21.15.157:0/2475050368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c06d000 0x7f336c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 --2- 172.21.15.157:0/2475050368 >> v2:172.21.15.195:3300/0 conn(0x7f336c1599c0 0x7f336c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- 172.21.15.157:0/2475050368 >> 172.21.15.157:0/2475050368 conn(0x7f336c065930 msgr2=0x7f336c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- 172.21.15.157:0/2475050368 shutdown_connections 2023-11-06T03:04:10.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- 172.21.15.157:0/2475050368 wait complete. 2023-11-06T03:04:10.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 Processor -- start 2023-11-06T03:04:10.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.518+0000 7f3370c04640 1 -- start start 2023-11-06T03:04:10.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c06d000 0x7f336c15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c1599c0 0x7f336c1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c15a950 con 0x7f336c15dea0 2023-11-06T03:04:10.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c15b750 con 0x7f336c06d000 2023-11-06T03:04:10.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c15ba30 con 0x7f336c1599c0 2023-11-06T03:04:10.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43586/0 (socket says 172.21.15.157:43586) 2023-11-06T03:04:10.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 -- 172.21.15.157:0/1014338779 learned_addr learned my addr 172.21.15.157:0/1014338779 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:10.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3369d9b640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c1599c0 0x7f336c1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 -- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c1599c0 msgr2=0x7f336c1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c1599c0 0x7f336c1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336a59c640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c06d000 0x7f336c15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 -- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c06d000 msgr2=0x7f336c15ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c06d000 0x7f336c15ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 -- 172.21.15.157:0/1014338779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009b30 con 0x7f336c15dea0 2023-11-06T03:04:10.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f336ad9d640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f336401b050 tx=0x7f3364019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640b8670 con 0x7f336c15dea0 2023-11-06T03:04:10.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33640b87d0 con 0x7f336c15dea0 2023-11-06T03:04:10.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c15a300 con 0x7f336c15dea0 2023-11-06T03:04:10.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640b8b80 con 0x7f336c15dea0 2023-11-06T03:04:10.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c15bd10 con 0x7f336c15dea0 2023-11-06T03:04:10.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c15aac0 con 0x7f336c15dea0 2023-11-06T03:04:10.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33640b8ce0 con 0x7f336c15dea0 2023-11-06T03:04:10.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.522+0000 7f3362ffd640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 0x7f334c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.526+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f336411f260 con 0x7f336c15dea0 2023-11-06T03:04:10.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.526+0000 7f336a59c640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 0x7f334c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.526+0000 7f336a59c640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 0x7f334c067f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f335c0099b0 tx=0x7f335c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.526+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33640eccc0 con 0x7f336c15dea0 2023-11-06T03:04:10.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.678+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f336c072750 con 0x7f334c065a60 2023-11-06T03:04:10.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.698+0000 7f3362ffd640 1 -- 172.21.15.157:0/1014338779 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271887 (secure 0 0 0) 0x7f336c072750 con 0x7f334c065a60 2023-11-06T03:04:10.699 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:10.702 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:10.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 msgr2=0x7f334c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 0x7f334c067f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f335c0099b0 tx=0x7f335c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:10.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 msgr2=0x7f336c0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f336401b050 tx=0x7f3364019740 comp rx=0 tx=0).stop 2023-11-06T03:04:10.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 shutdown_connections 2023-11-06T03:04:10.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f334c065a60 0x7f334c067f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f336c15dea0 0x7f336c0ac8b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f336c1599c0 0x7f336c1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 --2- 172.21.15.157:0/1014338779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f336c06d000 0x7f336c15ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 >> 172.21.15.157:0/1014338779 conn(0x7f336c065930 msgr2=0x7f336c150460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 shutdown_connections 2023-11-06T03:04:10.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.702+0000 7f3370c04640 1 -- 172.21.15.157:0/1014338779 wait complete. 2023-11-06T03:04:10.719 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T03:04:09.093887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250976,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723744,"statfs":{"total":773094113280,"available":772837113856,"internally_reserved":0,"allocated":2654208,"data_stored":1653163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001232"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.273409+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:02.273409+0000","last_peered":"2023-11-06T03:04:02.273409+0000","last_clean":"2023-11-06T03:04:02.273409+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:02.273409+0000","last_undegraded":"2023-11-06T03:04:02.273409+0000","last_fullsized":"2023-11-06T03:04:02.273409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281441+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:02.281441+0000","last_peered":"2023-11-06T03:04:02.281441+0000","last_clean":"2023-11-06T03:04:02.281441+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:02.281441+0000","last_undegraded":"2023-11-06T03:04:02.281441+0000","last_fullsized":"2023-11-06T03:04:02.281441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710322+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:03.710322+0000","last_peered":"2023-11-06T03:04:03.710322+0000","last_clean":"2023-11-06T03:04:03.710322+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:03.710322+0000","last_undegraded":"2023-11-06T03:04:03.710322+0000","last_fullsized":"2023-11-06T03:04:03.710322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.651934+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:02.651934+0000","last_peered":"2023-11-06T03:04:02.651934+0000","last_clean":"2023-11-06T03:04:02.651934+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:02.651934+0000","last_undegraded":"2023-11-06T03:04:02.651934+0000","last_fullsized":"2023-11-06T03:04:02.651934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281388+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:02.281388+0000","last_peered":"2023-11-06T03:04:02.281388+0000","last_clean":"2023-11-06T03:04:02.281388+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:02.281388+0000","last_undegraded":"2023-11-06T03:04:02.281388+0000","last_fullsized":"2023-11-06T03:04:02.281388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.622854+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:02.622854+0000","last_peered":"2023-11-06T03:04:02.622854+0000","last_clean":"2023-11-06T03:04:02.622854+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:02.622854+0000","last_undegraded":"2023-11-06T03:04:02.622854+0000","last_fullsized":"2023-11-06T03:04:02.622854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710323+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:03.710323+0000","last_peered":"2023-11-06T03:04:03.710323+0000","last_clean":"2023-11-06T03:04:03.710323+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:03.710323+0000","last_undegraded":"2023-11-06T03:04:03.710323+0000","last_fullsized":"2023-11-06T03:04:03.710323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.277044+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:02.277044+0000","last_peered":"2023-11-06T03:04:02.277044+0000","last_clean":"2023-11-06T03:04:02.277044+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:02.277044+0000","last_undegraded":"2023-11-06T03:04:02.277044+0000","last_fullsized":"2023-11-06T03:04:02.277044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.280926+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:02.280926+0000","last_peered":"2023-11-06T03:04:02.280926+0000","last_clean":"2023-11-06T03:04:02.280926+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:02.280926+0000","last_undegraded":"2023-11-06T03:04:02.280926+0000","last_fullsized":"2023-11-06T03:04:02.280926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:10.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:10.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 Processor -- start 2023-11-06T03:04:10.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 -- start start 2023-11-06T03:04:10.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f370815e5e0 0x7f37080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370815f580 con 0x7f37081551e0 2023-11-06T03:04:10.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370815efc0 con 0x7f370815ea90 2023-11-06T03:04:10.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f370dc42640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080704b0 con 0x7f370815e5e0 2023-11-06T03:04:10.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f37081551e0 0x7f370806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43494/0 (socket says 172.21.15.157:43494) 2023-11-06T03:04:10.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 -- 172.21.15.157:0/4244306708 learned_addr learned my addr 172.21.15.157:0/4244306708 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:10.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f3706ffd640 1 --2- 172.21.15.157:0/4244306708 >> v2:172.21.15.195:3300/0 conn(0x7f370815e5e0 0x7f37080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648001060 0 0) 0x7f370815f580 con 0x7f37081551e0 2023-11-06T03:04:10.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f4003530 con 0x7f37081551e0 2023-11-06T03:04:10.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766081437 0 0) 0x7f36f4003530 con 0x7f37081551e0 2023-11-06T03:04:10.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 >> v2:172.21.15.195:3300/0 conn(0x7f370815e5e0 msgr2=0x7f37080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 --2- 172.21.15.157:0/4244306708 >> v2:172.21.15.195:3300/0 conn(0x7f370815e5e0 0x7f37080b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 >> v1:172.21.15.195:6789/0 conn(0x7f370815ea90 legacy=0x7f370815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370815f2a0 con 0x7f37081551e0 2023-11-06T03:04:10.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f36f0001680 con 0x7f37081551e0 2023-11-06T03:04:10.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 >> v1:172.21.15.157:6789/0 conn(0x7f37081551e0 legacy=0x7f370806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 0x7f36f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36f40047d0 0x7f36f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4005190 0x7f36f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080704b0 con 0x7f36f4003980 2023-11-06T03:04:10.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f400c130 con 0x7f36f4005190 2023-11-06T03:04:10.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f400c2f0 con 0x7f36f40047d0 2023-11-06T03:04:10.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 0x7f36f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f3706ffd640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36f40047d0 0x7f36f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f3707fff640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4005190 0x7f36f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 -- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36f40047d0 msgr2=0x7f36f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36f40047d0 0x7f36f4004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 -- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4005190 msgr2=0x7f36f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:10.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4005190 0x7f36f400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f3707fff640 1 -- 172.21.15.157:0/4244306708 reap_dead start 2023-11-06T03:04:10.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.878+0000 7f37077fe640 1 -- 172.21.15.157:0/4244306708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4003530 con 0x7f36f4003980 2023-11-06T03:04:10.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f37077fe640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 0x7f36f4003da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c240 tx=0x7f36f001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f0a1cb425d67c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00a5af0 con 0x7f36f4003980 2023-11-06T03:04:10.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f00a5c50 con 0x7f36f4003980 2023-11-06T03:04:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f37067fc640 1 -- 172.21.15.157:0/4244306708 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0005b20 con 0x7f36f4003980 2023-11-06T03:04:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 msgr2=0x7f36f4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 0x7f36f4003da0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c240 tx=0x7f36f001cc10 comp rx=0 tx=0).stop 2023-11-06T03:04:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- 172.21.15.157:0/4244306708 shutdown_connections 2023-11-06T03:04:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 --2- 172.21.15.157:0/4244306708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36f4003980 0x7f36f4003da0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- 172.21.15.157:0/4244306708 >> 172.21.15.157:0/4244306708 conn(0x7f3708065930 msgr2=0x7f3708065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- 172.21.15.157:0/4244306708 shutdown_connections 2023-11-06T03:04:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- 172.21.15.157:0/4244306708 wait complete. 2023-11-06T03:04:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 Processor -- start 2023-11-06T03:04:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- start start 2023-11-06T03:04:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4003980 0x7f370814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3708149270 0x7f3708148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080ce470 con 0x7f37080cdab0 2023-11-06T03:04:10.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370814ba00 con 0x7f36f4003980 2023-11-06T03:04:10.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f370dc42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3708149080 con 0x7f3708149270 2023-11-06T03:04:10.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43602/0 (socket says 172.21.15.157:43602) 2023-11-06T03:04:10.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 -- 172.21.15.157:0/2768021564 learned_addr learned my addr 172.21.15.157:0/2768021564 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:10.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3706ffd640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3708149270 0x7f3708148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f37077fe640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4003980 0x7f370814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 -- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3708149270 msgr2=0x7f3708148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3708149270 0x7f3708148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 -- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4003980 msgr2=0x7f370814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:10.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4003980 0x7f370814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:10.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 -- 172.21.15.157:0/2768021564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370814bc30 con 0x7f37080cdab0 2023-11-06T03:04:10.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.882+0000 7f3707fff640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d400 tx=0x7f36f800d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f800b070 con 0x7f37080cdab0 2023-11-06T03:04:10.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8003600 con 0x7f37080cdab0 2023-11-06T03:04:10.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80054e0 con 0x7f37080cdab0 2023-11-06T03:04:10.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370814bf10 con 0x7f37080cdab0 2023-11-06T03:04:10.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370814c4d0 con 0x7f37080cdab0 2023-11-06T03:04:10.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3708072750 con 0x7f37080cdab0 2023-11-06T03:04:10.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f36f80039b0 con 0x7f37080cdab0 2023-11-06T03:04:10.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 0x7f36e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:10.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36f8082880 con 0x7f37080cdab0 2023-11-06T03:04:10.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f37077fe640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 0x7f36e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:10.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.886+0000 7f37077fe640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 0x7f36e8067f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36f00ab000 tx=0x7f36f0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:10.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:10.890+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f36f80500e0 con 0x7f37080cdab0 2023-11-06T03:04:11.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.026+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f370815b1b0 con 0x7f36e8065a60 2023-11-06T03:04:11.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.046+0000 7f36fffff640 1 -- 172.21.15.157:0/2768021564 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271887 (secure 0 0 0) 0x7f36f00b4040 con 0x7f36e8065a60 2023-11-06T03:04:11.049 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:11.053 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:11.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 msgr2=0x7f36e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 0x7f36e8067f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36f00ab000 tx=0x7f36f0006040 comp rx=0 tx=0).stop 2023-11-06T03:04:11.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 msgr2=0x7f37080cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d400 tx=0x7f36f800d8c0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 shutdown_connections 2023-11-06T03:04:11.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36e8065a60 0x7f36e8067f20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37080cdab0 0x7f37080cdf30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3708149270 0x7f3708148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 --2- 172.21.15.157:0/2768021564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36f4003980 0x7f370814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 >> 172.21.15.157:0/2768021564 conn(0x7f3708065930 msgr2=0x7f370815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:11.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 shutdown_connections 2023-11-06T03:04:11.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.050+0000 7f370dc42640 1 -- 172.21.15.157:0/2768021564 wait complete. 2023-11-06T03:04:11.069 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-06T03:04:09.093887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":250976,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723744,"statfs":{"total":773094113280,"available":772837113856,"internally_reserved":0,"allocated":2654208,"data_stored":1653163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001232"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.273409+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:02.273409+0000","last_peered":"2023-11-06T03:04:02.273409+0000","last_clean":"2023-11-06T03:04:02.273409+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:02.273409+0000","last_undegraded":"2023-11-06T03:04:02.273409+0000","last_fullsized":"2023-11-06T03:04:02.273409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281441+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:02.281441+0000","last_peered":"2023-11-06T03:04:02.281441+0000","last_clean":"2023-11-06T03:04:02.281441+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:02.281441+0000","last_undegraded":"2023-11-06T03:04:02.281441+0000","last_fullsized":"2023-11-06T03:04:02.281441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710322+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:03.710322+0000","last_peered":"2023-11-06T03:04:03.710322+0000","last_clean":"2023-11-06T03:04:03.710322+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:03.710322+0000","last_undegraded":"2023-11-06T03:04:03.710322+0000","last_fullsized":"2023-11-06T03:04:03.710322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.651934+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:02.651934+0000","last_peered":"2023-11-06T03:04:02.651934+0000","last_clean":"2023-11-06T03:04:02.651934+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:02.651934+0000","last_undegraded":"2023-11-06T03:04:02.651934+0000","last_fullsized":"2023-11-06T03:04:02.651934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.281388+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:02.281388+0000","last_peered":"2023-11-06T03:04:02.281388+0000","last_clean":"2023-11-06T03:04:02.281388+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:02.281388+0000","last_undegraded":"2023-11-06T03:04:02.281388+0000","last_fullsized":"2023-11-06T03:04:02.281388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.622854+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:02.622854+0000","last_peered":"2023-11-06T03:04:02.622854+0000","last_clean":"2023-11-06T03:04:02.622854+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:02.622854+0000","last_undegraded":"2023-11-06T03:04:02.622854+0000","last_fullsized":"2023-11-06T03:04:02.622854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-06T03:04:03.710323+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:03.710323+0000","last_peered":"2023-11-06T03:04:03.710323+0000","last_clean":"2023-11-06T03:04:03.710323+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:03.710323+0000","last_undegraded":"2023-11-06T03:04:03.710323+0000","last_fullsized":"2023-11-06T03:04:03.710323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.277044+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:02.277044+0000","last_peered":"2023-11-06T03:04:02.277044+0000","last_clean":"2023-11-06T03:04:02.277044+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:02.277044+0000","last_undegraded":"2023-11-06T03:04:02.277044+0000","last_fullsized":"2023-11-06T03:04:02.277044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-06T03:04:02.280926+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:02.280926+0000","last_peered":"2023-11-06T03:04:02.280926+0000","last_clean":"2023-11-06T03:04:02.280926+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:02.280926+0000","last_undegraded":"2023-11-06T03:04:02.280926+0000","last_fullsized":"2023-11-06T03:04:02.280926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:06.293625+0000","last_change":"2023-11-06T03:04:06.293625+0000","last_active":"2023-11-06T03:04:06.293625+0000","last_peered":"2023-11-06T03:04:06.293625+0000","last_clean":"2023-11-06T03:04:06.293625+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:06.293625+0000","last_undegraded":"2023-11-06T03:04:06.293625+0000","last_fullsized":"2023-11-06T03:04:06.293625+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:04:05.282861+0000","last_change":"2023-11-06T03:04:05.282861+0000","last_active":"2023-11-06T03:04:05.282861+0000","last_peered":"2023-11-06T03:04:05.282861+0000","last_clean":"2023-11-06T03:04:05.282861+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:04:05.282861+0000","last_undegraded":"2023-11-06T03:04:05.282861+0000","last_fullsized":"2023-11-06T03:04:05.282861+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31236,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340604,"statfs":{"total":96636764160,"available":96604778496,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31220,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340620,"statfs":{"total":96636764160,"available":96604794880,"internally_reserved":0,"allocated":176128,"data_stored":56261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31224,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340616,"statfs":{"total":96636764160,"available":96604790784,"internally_reserved":0,"allocated":180224,"data_stored":57280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":782336,"data_stored":652377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:11.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-11-06T03:04:11.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-11-06T03:04:11.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2023-11-06T03:04:11.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2023-11-06T03:04:11.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-11-06T03:04:11.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2023-11-06T03:04:11.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2023-11-06T03:04:11.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2023-11-06T03:04:11.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-11-06T03:04:11.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2023-11-06T03:04:11.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2023-11-06T03:04:11.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-11-06T03:04:11.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-11-06T03:04:11.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2023-11-06T03:04:11.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2023-11-06T03:04:11.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2023-11-06T03:04:11.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-11-06T03:04:11.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2023-11-06T03:04:11.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-11-06T03:04:11.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2023-11-06T03:04:11.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2023-11-06T03:04:11.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-11-06T03:04:11.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-11-06T03:04:11.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2023-11-06T03:04:11.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-11-06T03:04:11.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-11-06T03:04:11.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2023-11-06T03:04:11.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-11-06T03:04:11.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-11-06T03:04:11.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2023-11-06T03:04:11.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-11-06T03:04:11.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2023-11-06T03:04:11.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-11-06T03:04:11.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-11-06T03:04:11.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-11-06T03:04:11.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-11-06T03:04:11.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-11-06T03:04:11.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-11-06T03:04:11.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-11-06T03:04:11.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-11-06T03:04:11.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-11-06T03:04:11.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-11-06T03:04:11.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-11-06T03:04:11.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-11-06T03:04:11.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-11-06T03:04:11.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-11-06T03:04:11.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-11-06T03:04:11.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-11-06T03:04:11.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-11-06T03:04:11.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-11-06T03:04:11.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-11-06T03:04:11.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-11-06T03:04:11.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-11-06T03:04:11.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-11-06T03:04:11.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-11-06T03:04:11.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-11-06T03:04:11.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-11-06T03:04:11.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-11-06T03:04:11.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-11-06T03:04:11.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-11-06T03:04:11.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-11-06T03:04:11.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-11-06T03:04:11.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-11-06T03:04:11.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-11-06T03:04:11.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-11-06T03:04:11.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-11-06T03:04:11.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-11-06T03:04:11.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-11-06T03:04:11.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-11-06T03:04:11.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-11-06T03:04:11.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-11-06T03:04:11.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-11-06T03:04:11.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-11-06T03:04:11.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-11-06T03:04:11.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-11-06T03:04:11.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-11-06T03:04:11.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-11-06T03:04:11.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-11-06T03:04:11.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-11-06T03:04:11.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-11-06T03:04:11.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-11-06T03:04:11.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-11-06T03:04:11.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-11-06T03:04:11.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-11-06T03:04:11.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-11-06T03:04:11.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-11-06T03:04:11.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-11-06T03:04:11.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-11-06T03:04:11.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-11-06T03:04:11.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-11-06T03:04:11.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-11-06T03:04:11.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-11-06T03:04:11.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-11-06T03:04:11.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-11-06T03:04:11.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-11-06T03:04:11.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:06.293625+0000', 'last_change': '2023-11-06T03:04:06.293625+0000', 'last_active': '2023-11-06T03:04:06.293625+0000', 'last_peered': '2023-11-06T03:04:06.293625+0000', 'last_clean': '2023-11-06T03:04:06.293625+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:06.293625+0000', 'last_undegraded': '2023-11-06T03:04:06.293625+0000', 'last_fullsized': '2023-11-06T03:04:06.293625+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:06.293625+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-11-06T03:04:11.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:04:05.282861+0000', 'last_change': '2023-11-06T03:04:05.282861+0000', 'last_active': '2023-11-06T03:04:05.282861+0000', 'last_peered': '2023-11-06T03:04:05.282861+0000', 'last_clean': '2023-11-06T03:04:05.282861+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:04:05.282861+0000', 'last_undegraded': '2023-11-06T03:04:05.282861+0000', 'last_fullsized': '2023-11-06T03:04:05.282861+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'last_clean_scrub_stamp': '2023-11-06T03:04:05.282861+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:04:11.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:04:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.270+0000 7f4be5e98640 1 Processor -- start 2023-11-06T03:04:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.270+0000 7f4be5e98640 1 -- start start 2023-11-06T03:04:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4be5e98640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4be5e98640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00b2af0 0x7f4be0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4be5e98640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be015e440 con 0x7f4be006d000 2023-11-06T03:04:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4be5e98640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be006c240 con 0x7f4be00b2af0 2023-11-06T03:04:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4be5e98640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be006c520 con 0x7f4be01599c0 2023-11-06T03:04:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00b2af0 0x7f4be0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00b2af0 0x7f4be0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43612/0 (socket says 172.21.15.157:43612) 2023-11-06T03:04:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 -- 172.21.15.157:0/1880389451 learned_addr learned my addr 172.21.15.157:0/1880389451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdf7fe640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 -- 172.21.15.157:0/1880389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 msgr2=0x7f4be00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 -- 172.21.15.157:0/1880389451 >> v1:172.21.15.195:6789/0 conn(0x7f4be006d000 legacy=0x7f4be00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 -- 172.21.15.157:0/1880389451 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be006c800 con 0x7f4be00b2af0 2023-11-06T03:04:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdffff640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0159da0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d150 tx=0x7f4bd801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674a0d0040107431 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.274+0000 7f4bdeffd640 1 -- 172.21.15.157:0/1880389451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a7cd0 con 0x7f4be00b2af0 2023-11-06T03:04:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdeffd640 1 -- 172.21.15.157:0/1880389451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd80a7e30 con 0x7f4be00b2af0 2023-11-06T03:04:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdeffd640 1 -- 172.21.15.157:0/1880389451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800b2f0 con 0x7f4be00b2af0 2023-11-06T03:04:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1880389451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 msgr2=0x7f4be0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0159da0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d150 tx=0x7f4bd801ebe0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1880389451 shutdown_connections 2023-11-06T03:04:11.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0159da0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1880389451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1880389451 >> 172.21.15.157:0/1880389451 conn(0x7f4be0065930 msgr2=0x7f4be0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:11.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1880389451 shutdown_connections 2023-11-06T03:04:11.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1880389451 wait complete. 2023-11-06T03:04:11.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 Processor -- start 2023-11-06T03:04:11.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- start start 2023-11-06T03:04:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be006d000 0x7f4be014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01f9c50 con 0x7f4be00b2af0 2023-11-06T03:04:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0158e40 con 0x7f4be006d000 2023-11-06T03:04:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be014cf10 con 0x7f4be01599c0 2023-11-06T03:04:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43620/0 (socket says 172.21.15.157:43620) 2023-11-06T03:04:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 -- 172.21.15.157:0/1472657090 learned_addr learned my addr 172.21.15.157:0/1472657090 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdffff640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdf7fe640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be006d000 0x7f4be014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 -- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 msgr2=0x7f4be014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 -- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be006d000 msgr2=0x7f4be014d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be006d000 0x7f4be014d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 -- 172.21.15.157:0/1472657090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd800bd40 con 0x7f4be00b2af0 2023-11-06T03:04:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bd6ffd640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c450 tx=0x7f4bc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80052d0 con 0x7f4be00b2af0 2023-11-06T03:04:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8005430 con 0x7f4be00b2af0 2023-11-06T03:04:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8007260 con 0x7f4be00b2af0 2023-11-06T03:04:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be01f9640 con 0x7f4be00b2af0 2023-11-06T03:04:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.278+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be01f98e0 con 0x7f4be00b2af0 2023-11-06T03:04:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be0072750 con 0x7f4be00b2af0 2023-11-06T03:04:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4bc80073c0 con 0x7f4be00b2af0 2023-11-06T03:04:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4bdcff9640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 0x7f4bc0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4bc80829b0 con 0x7f4be00b2af0 2023-11-06T03:04:11.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4bdf7fe640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 0x7f4bc0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.282+0000 7f4bdf7fe640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 0x7f4bc0067f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4be00cf1f0 tx=0x7f4bd0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.286+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4bc8050350 con 0x7f4be00b2af0 2023-11-06T03:04:11.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.482+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4be0152630 con 0x7f4be00b2af0 2023-11-06T03:04:11.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.482+0000 7f4bdcff9640 1 -- 172.21.15.157:0/1472657090 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7f4bc8053970 con 0x7f4be00b2af0 2023-11-06T03:04:11.486 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:11.486 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592730,"bytes_used":257519616,"bytes_avail":772836593664,"bytes_total":773094113280,"write_bytes_sec":2471,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4306}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-06T03:03:42.475896+0000","services":{}},"progress_events":{}} 2023-11-06T03:04:11.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 msgr2=0x7f4bc0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 0x7f4bc0067f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4be00cf1f0 tx=0x7f4bd0009190 comp rx=0 tx=0).stop 2023-11-06T03:04:11.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 msgr2=0x7f4be0151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c450 tx=0x7f4bc800c910 comp rx=0 tx=0).stop 2023-11-06T03:04:11.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 shutdown_connections 2023-11-06T03:04:11.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a60 0x7f4bc0067f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be01599c0 0x7f4be014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00b2af0 0x7f4be0151a10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 --2- 172.21.15.157:0/1472657090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be006d000 0x7f4be014d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 >> 172.21.15.157:0/1472657090 conn(0x7f4be0065930 msgr2=0x7f4be0065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 shutdown_connections 2023-11-06T03:04:11.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.486+0000 7f4be5e98640 1 -- 172.21.15.157:0/1472657090 wait complete. 2023-11-06T03:04:11.502 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:04:11.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:11.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 Processor -- start 2023-11-06T03:04:11.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 -- start start 2023-11-06T03:04:11.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bd815ad30 0x7f1bd80b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd80cef10 con 0x7f1bd80b23b0 2023-11-06T03:04:11.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd80cfb70 con 0x7f1bd815ad30 2023-11-06T03:04:11.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bdd79a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd80cf160 con 0x7f1bd80b2f20 2023-11-06T03:04:11.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bd815ad30 0x7f1bd80b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd67fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1bd80b23b0 0x7f1bd80b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43504/0 (socket says 172.21.15.157:43504) 2023-11-06T03:04:11.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd67fc640 1 -- 172.21.15.157:0/1641840489 learned_addr learned my addr 172.21.15.157:0/1641840489 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:11.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd77fe640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/1641840489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 msgr2=0x7f1bd80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:11.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/1641840489 >> v1:172.21.15.157:6789/0 conn(0x7f1bd80b23b0 legacy=0x7f1bd80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/1641840489 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd80cf440 con 0x7f1bd815ad30 2023-11-06T03:04:11.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd77fe640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:11.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.614+0000 7f1bd6ffd640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd815ad30 0x7f1bd80b60a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009d90 tx=0x7f1bc801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77e4098e59064c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd5ffb640 1 -- 172.21.15.157:0/1641840489 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80a5970 con 0x7f1bd815ad30 2023-11-06T03:04:11.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd5ffb640 1 -- 172.21.15.157:0/1641840489 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc80a5ad0 con 0x7f1bd815ad30 2023-11-06T03:04:11.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd5ffb640 1 -- 172.21.15.157:0/1641840489 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80a5e60 con 0x7f1bd815ad30 2023-11-06T03:04:11.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/1641840489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd815ad30 msgr2=0x7f1bd80b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd815ad30 0x7f1bd80b60a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009d90 tx=0x7f1bc801c890 comp rx=0 tx=0).stop 2023-11-06T03:04:11.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/1641840489 shutdown_connections 2023-11-06T03:04:11.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/1641840489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd815ad30 0x7f1bd80b60a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/1641840489 >> 172.21.15.157:0/1641840489 conn(0x7f1bd8065d30 msgr2=0x7f1bd8066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:11.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/1641840489 shutdown_connections 2023-11-06T03:04:11.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/1641840489 wait complete. 2023-11-06T03:04:11.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 Processor -- start 2023-11-06T03:04:11.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- start start 2023-11-06T03:04:11.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bd815ad30 0x7f1bd8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd81f4f20 con 0x7f1bd80b23b0 2023-11-06T03:04:11.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8159b50 con 0x7f1bd815ad30 2023-11-06T03:04:11.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd8162980 con 0x7f1bd80b2f20 2023-11-06T03:04:11.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43640/0 (socket says 172.21.15.157:43640) 2023-11-06T03:04:11.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/2373444881 learned_addr learned my addr 172.21.15.157:0/2373444881 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:11.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd67fc640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd77fe640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bd815ad30 0x7f1bd8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 msgr2=0x7f1bd80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bd815ad30 msgr2=0x7f1bd8158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bd815ad30 0x7f1bd8158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 -- 172.21.15.157:0/2373444881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8009a70 con 0x7f1bd80b23b0 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bd6ffd640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80b0040 tx=0x7f1bc8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80a5970 con 0x7f1bd80b23b0 2023-11-06T03:04:11.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc80b9450 con 0x7f1bd80b23b0 2023-11-06T03:04:11.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80b9800 con 0x7f1bd80b23b0 2023-11-06T03:04:11.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd8162c60 con 0x7f1bd80b23b0 2023-11-06T03:04:11.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.618+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd8163220 con 0x7f1bd80b23b0 2023-11-06T03:04:11.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd806bf20 con 0x7f1bd80b23b0 2023-11-06T03:04:11.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1bc80b9960 con 0x7f1bd80b23b0 2023-11-06T03:04:11.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bbf7fe640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 0x7f1bc4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:11.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1bc811ec20 con 0x7f1bd80b23b0 2023-11-06T03:04:11.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bd67fc640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 0x7f1bc4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:11.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.622+0000 7f1bd67fc640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 0x7f1bc4067f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bd81569a0 tx=0x7f1bcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:11.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.626+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1bc80ec540 con 0x7f1bd80b23b0 2023-11-06T03:04:11.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.778+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bd8072750 con 0x7f1bc4065a60 2023-11-06T03:04:11.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.798+0000 7f1bbf7fe640 1 -- 172.21.15.157:0/2373444881 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286674 (secure 0 0 0) 0x7f1bd8072750 con 0x7f1bc4065a60 2023-11-06T03:04:11.802 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:11.804 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:11.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 msgr2=0x7f1bc4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 0x7f1bc4067f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bd81569a0 tx=0x7f1bcc009040 comp rx=0 tx=0).stop 2023-11-06T03:04:11.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 msgr2=0x7f1bd8150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:11.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80b0040 tx=0x7f1bc8019630 comp rx=0 tx=0).stop 2023-11-06T03:04:11.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 shutdown_connections 2023-11-06T03:04:11.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1bc4065a60 0x7f1bc4067f20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bd815ad30 0x7f1bd8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bd80b2f20 0x7f1bd80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 --2- 172.21.15.157:0/2373444881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bd80b23b0 0x7f1bd8150740 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:11.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 >> 172.21.15.157:0/2373444881 conn(0x7f1bd8065d30 msgr2=0x7f1bd8067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:11.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 shutdown_connections 2023-11-06T03:04:11.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:11.802+0000 7f1bdd79a640 1 -- 172.21.15.157:0/2373444881 wait complete. 2023-11-06T03:04:11.821 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-06T03:04:11.094559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251484,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723236,"statfs":{"total":773094113280,"available":772836593664,"internally_reserved":0,"allocated":3174400,"data_stored":1899195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.799925"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314154+0000","last_change":"2023-11-06T03:04:06.723013+0000","last_active":"2023-11-06T03:04:08.314154+0000","last_peered":"2023-11-06T03:04:08.314154+0000","last_clean":"2023-11-06T03:04:08.314154+0000","last_became_active":"2023-11-06T03:04:06.722313+0000","last_became_peered":"2023-11-06T03:04:06.722313+0000","last_unstale":"2023-11-06T03:04:08.314154+0000","last_undegraded":"2023-11-06T03:04:08.314154+0000","last_fullsized":"2023-11-06T03:04:08.314154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:17.965940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313638+0000","last_change":"2023-11-06T03:04:06.709695+0000","last_active":"2023-11-06T03:04:08.313638+0000","last_peered":"2023-11-06T03:04:08.313638+0000","last_clean":"2023-11-06T03:04:08.313638+0000","last_became_active":"2023-11-06T03:04:06.709429+0000","last_became_peered":"2023-11-06T03:04:06.709429+0000","last_unstale":"2023-11-06T03:04:08.313638+0000","last_undegraded":"2023-11-06T03:04:08.313638+0000","last_fullsized":"2023-11-06T03:04:08.313638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697946+0000","last_change":"2023-11-06T03:04:06.700734+0000","last_active":"2023-11-06T03:04:07.697946+0000","last_peered":"2023-11-06T03:04:07.697946+0000","last_clean":"2023-11-06T03:04:07.697946+0000","last_became_active":"2023-11-06T03:04:06.699143+0000","last_became_peered":"2023-11-06T03:04:06.699143+0000","last_unstale":"2023-11-06T03:04:07.697946+0000","last_undegraded":"2023-11-06T03:04:07.697946+0000","last_fullsized":"2023-11-06T03:04:07.697946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315057+0000","last_change":"2023-11-06T03:04:06.725804+0000","last_active":"2023-11-06T03:04:08.315057+0000","last_peered":"2023-11-06T03:04:08.315057+0000","last_clean":"2023-11-06T03:04:08.315057+0000","last_became_active":"2023-11-06T03:04:06.725496+0000","last_became_peered":"2023-11-06T03:04:06.725496+0000","last_unstale":"2023-11-06T03:04:08.315057+0000","last_undegraded":"2023-11-06T03:04:08.315057+0000","last_fullsized":"2023-11-06T03:04:08.315057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:30.176058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315579+0000","last_change":"2023-11-06T03:04:06.725612+0000","last_active":"2023-11-06T03:04:08.315579+0000","last_peered":"2023-11-06T03:04:08.315579+0000","last_clean":"2023-11-06T03:04:08.315579+0000","last_became_active":"2023-11-06T03:04:06.725099+0000","last_became_peered":"2023-11-06T03:04:06.725099+0000","last_unstale":"2023-11-06T03:04:08.315579+0000","last_undegraded":"2023-11-06T03:04:08.315579+0000","last_fullsized":"2023-11-06T03:04:08.315579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697168+0000","last_change":"2023-11-06T03:04:06.710419+0000","last_active":"2023-11-06T03:04:07.697168+0000","last_peered":"2023-11-06T03:04:07.697168+0000","last_clean":"2023-11-06T03:04:07.697168+0000","last_became_active":"2023-11-06T03:04:06.709958+0000","last_became_peered":"2023-11-06T03:04:06.709958+0000","last_unstale":"2023-11-06T03:04:07.697168+0000","last_undegraded":"2023-11-06T03:04:07.697168+0000","last_fullsized":"2023-11-06T03:04:07.697168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:58.848613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811621+0000","last_change":"2023-11-06T03:04:06.720209+0000","last_active":"2023-11-06T03:04:07.811621+0000","last_peered":"2023-11-06T03:04:07.811621+0000","last_clean":"2023-11-06T03:04:07.811621+0000","last_became_active":"2023-11-06T03:04:06.719857+0000","last_became_peered":"2023-11-06T03:04:06.719857+0000","last_unstale":"2023-11-06T03:04:07.811621+0000","last_undegraded":"2023-11-06T03:04:07.811621+0000","last_fullsized":"2023-11-06T03:04:07.811621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313002+0000","last_change":"2023-11-06T03:04:06.706215+0000","last_active":"2023-11-06T03:04:08.313002+0000","last_peered":"2023-11-06T03:04:08.313002+0000","last_clean":"2023-11-06T03:04:08.313002+0000","last_became_active":"2023-11-06T03:04:06.705329+0000","last_became_peered":"2023-11-06T03:04:06.705329+0000","last_unstale":"2023-11-06T03:04:08.313002+0000","last_undegraded":"2023-11-06T03:04:08.313002+0000","last_fullsized":"2023-11-06T03:04:08.313002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315584+0000","last_change":"2023-11-06T03:04:06.725727+0000","last_active":"2023-11-06T03:04:08.315584+0000","last_peered":"2023-11-06T03:04:08.315584+0000","last_clean":"2023-11-06T03:04:08.315584+0000","last_became_active":"2023-11-06T03:04:06.725241+0000","last_became_peered":"2023-11-06T03:04:06.725241+0000","last_unstale":"2023-11-06T03:04:08.315584+0000","last_undegraded":"2023-11-06T03:04:08.315584+0000","last_fullsized":"2023-11-06T03:04:08.315584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313982+0000","last_change":"2023-11-06T03:04:06.701948+0000","last_active":"2023-11-06T03:04:08.313982+0000","last_peered":"2023-11-06T03:04:08.313982+0000","last_clean":"2023-11-06T03:04:08.313982+0000","last_became_active":"2023-11-06T03:04:06.701405+0000","last_became_peered":"2023-11-06T03:04:06.701405+0000","last_unstale":"2023-11-06T03:04:08.313982+0000","last_undegraded":"2023-11-06T03:04:08.313982+0000","last_fullsized":"2023-11-06T03:04:08.313982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698195+0000","last_change":"2023-11-06T03:04:06.711152+0000","last_active":"2023-11-06T03:04:07.698195+0000","last_peered":"2023-11-06T03:04:07.698195+0000","last_clean":"2023-11-06T03:04:07.698195+0000","last_became_active":"2023-11-06T03:04:06.710840+0000","last_became_peered":"2023-11-06T03:04:06.710840+0000","last_unstale":"2023-11-06T03:04:07.698195+0000","last_undegraded":"2023-11-06T03:04:07.698195+0000","last_fullsized":"2023-11-06T03:04:07.698195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314771+0000","last_change":"2023-11-06T03:04:06.703107+0000","last_active":"2023-11-06T03:04:08.314771+0000","last_peered":"2023-11-06T03:04:08.314771+0000","last_clean":"2023-11-06T03:04:08.314771+0000","last_became_active":"2023-11-06T03:04:06.702533+0000","last_became_peered":"2023-11-06T03:04:06.702533+0000","last_unstale":"2023-11-06T03:04:08.314771+0000","last_undegraded":"2023-11-06T03:04:08.314771+0000","last_fullsized":"2023-11-06T03:04:08.314771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314211+0000","last_change":"2023-11-06T03:04:06.710998+0000","last_active":"2023-11-06T03:04:08.314211+0000","last_peered":"2023-11-06T03:04:08.314211+0000","last_clean":"2023-11-06T03:04:08.314211+0000","last_became_active":"2023-11-06T03:04:06.710459+0000","last_became_peered":"2023-11-06T03:04:06.710459+0000","last_unstale":"2023-11-06T03:04:08.314211+0000","last_undegraded":"2023-11-06T03:04:08.314211+0000","last_fullsized":"2023-11-06T03:04:08.314211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:36.030631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313983+0000","last_change":"2023-11-06T03:04:06.708520+0000","last_active":"2023-11-06T03:04:08.313983+0000","last_peered":"2023-11-06T03:04:08.313983+0000","last_clean":"2023-11-06T03:04:08.313983+0000","last_became_active":"2023-11-06T03:04:06.708200+0000","last_became_peered":"2023-11-06T03:04:06.708200+0000","last_unstale":"2023-11-06T03:04:08.313983+0000","last_undegraded":"2023-11-06T03:04:08.313983+0000","last_fullsized":"2023-11-06T03:04:08.313983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811766+0000","last_change":"2023-11-06T03:04:06.723418+0000","last_active":"2023-11-06T03:04:07.811766+0000","last_peered":"2023-11-06T03:04:07.811766+0000","last_clean":"2023-11-06T03:04:07.811766+0000","last_became_active":"2023-11-06T03:04:06.720428+0000","last_became_peered":"2023-11-06T03:04:06.720428+0000","last_unstale":"2023-11-06T03:04:07.811766+0000","last_undegraded":"2023-11-06T03:04:07.811766+0000","last_fullsized":"2023-11-06T03:04:07.811766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697132+0000","last_change":"2023-11-06T03:04:06.699686+0000","last_active":"2023-11-06T03:04:07.697132+0000","last_peered":"2023-11-06T03:04:07.697132+0000","last_clean":"2023-11-06T03:04:07.697132+0000","last_became_active":"2023-11-06T03:04:06.699150+0000","last_became_peered":"2023-11-06T03:04:06.699150+0000","last_unstale":"2023-11-06T03:04:07.697132+0000","last_undegraded":"2023-11-06T03:04:07.697132+0000","last_fullsized":"2023-11-06T03:04:07.697132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315781+0000","last_change":"2023-11-06T03:04:06.718065+0000","last_active":"2023-11-06T03:04:08.315781+0000","last_peered":"2023-11-06T03:04:08.315781+0000","last_clean":"2023-11-06T03:04:08.315781+0000","last_became_active":"2023-11-06T03:04:06.717495+0000","last_became_peered":"2023-11-06T03:04:06.717495+0000","last_unstale":"2023-11-06T03:04:08.315781+0000","last_undegraded":"2023-11-06T03:04:08.315781+0000","last_fullsized":"2023-11-06T03:04:08.315781+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697937+0000","last_change":"2023-11-06T03:04:06.707851+0000","last_active":"2023-11-06T03:04:07.697937+0000","last_peered":"2023-11-06T03:04:07.697937+0000","last_clean":"2023-11-06T03:04:07.697937+0000","last_became_active":"2023-11-06T03:04:06.707587+0000","last_became_peered":"2023-11-06T03:04:06.707587+0000","last_unstale":"2023-11-06T03:04:07.697937+0000","last_undegraded":"2023-11-06T03:04:07.697937+0000","last_fullsized":"2023-11-06T03:04:07.697937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:52:19.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314357+0000","last_change":"2023-11-06T03:04:06.724261+0000","last_active":"2023-11-06T03:04:08.314357+0000","last_peered":"2023-11-06T03:04:08.314357+0000","last_clean":"2023-11-06T03:04:08.314357+0000","last_became_active":"2023-11-06T03:04:06.723974+0000","last_became_peered":"2023-11-06T03:04:06.723974+0000","last_unstale":"2023-11-06T03:04:08.314357+0000","last_undegraded":"2023-11-06T03:04:08.314357+0000","last_fullsized":"2023-11-06T03:04:08.314357+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:16.954584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697610+0000","last_change":"2023-11-06T03:04:06.729633+0000","last_active":"2023-11-06T03:04:07.697610+0000","last_peered":"2023-11-06T03:04:07.697610+0000","last_clean":"2023-11-06T03:04:07.697610+0000","last_became_active":"2023-11-06T03:04:06.729241+0000","last_became_peered":"2023-11-06T03:04:06.729241+0000","last_unstale":"2023-11-06T03:04:07.697610+0000","last_undegraded":"2023-11-06T03:04:07.697610+0000","last_fullsized":"2023-11-06T03:04:07.697610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:05.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811862+0000","last_change":"2023-11-06T03:04:06.728388+0000","last_active":"2023-11-06T03:04:07.811862+0000","last_peered":"2023-11-06T03:04:07.811862+0000","last_clean":"2023-11-06T03:04:07.811862+0000","last_became_active":"2023-11-06T03:04:06.728135+0000","last_became_peered":"2023-11-06T03:04:06.728135+0000","last_unstale":"2023-11-06T03:04:07.811862+0000","last_undegraded":"2023-11-06T03:04:07.811862+0000","last_fullsized":"2023-11-06T03:04:07.811862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315482+0000","last_change":"2023-11-06T03:04:06.713228+0000","last_active":"2023-11-06T03:04:08.315482+0000","last_peered":"2023-11-06T03:04:08.315482+0000","last_clean":"2023-11-06T03:04:08.315482+0000","last_became_active":"2023-11-06T03:04:06.712822+0000","last_became_peered":"2023-11-06T03:04:06.712822+0000","last_unstale":"2023-11-06T03:04:08.315482+0000","last_undegraded":"2023-11-06T03:04:08.315482+0000","last_fullsized":"2023-11-06T03:04:08.315482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696649+0000","last_change":"2023-11-06T03:04:06.711437+0000","last_active":"2023-11-06T03:04:07.696649+0000","last_peered":"2023-11-06T03:04:07.696649+0000","last_clean":"2023-11-06T03:04:07.696649+0000","last_became_active":"2023-11-06T03:04:06.710409+0000","last_became_peered":"2023-11-06T03:04:06.710409+0000","last_unstale":"2023-11-06T03:04:07.696649+0000","last_undegraded":"2023-11-06T03:04:07.696649+0000","last_fullsized":"2023-11-06T03:04:07.696649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811784+0000","last_change":"2023-11-06T03:04:06.728415+0000","last_active":"2023-11-06T03:04:07.811784+0000","last_peered":"2023-11-06T03:04:07.811784+0000","last_clean":"2023-11-06T03:04:07.811784+0000","last_became_active":"2023-11-06T03:04:06.728111+0000","last_became_peered":"2023-11-06T03:04:06.728111+0000","last_unstale":"2023-11-06T03:04:07.811784+0000","last_undegraded":"2023-11-06T03:04:07.811784+0000","last_fullsized":"2023-11-06T03:04:07.811784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317080+0000","last_change":"2023-11-06T03:04:06.727933+0000","last_active":"2023-11-06T03:04:08.317080+0000","last_peered":"2023-11-06T03:04:08.317080+0000","last_clean":"2023-11-06T03:04:08.317080+0000","last_became_active":"2023-11-06T03:04:06.727689+0000","last_became_peered":"2023-11-06T03:04:06.727689+0000","last_unstale":"2023-11-06T03:04:08.317080+0000","last_undegraded":"2023-11-06T03:04:08.317080+0000","last_fullsized":"2023-11-06T03:04:08.317080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317243+0000","last_change":"2023-11-06T03:04:06.709146+0000","last_active":"2023-11-06T03:04:08.317243+0000","last_peered":"2023-11-06T03:04:08.317243+0000","last_clean":"2023-11-06T03:04:08.317243+0000","last_became_active":"2023-11-06T03:04:06.708760+0000","last_became_peered":"2023-11-06T03:04:06.708760+0000","last_unstale":"2023-11-06T03:04:08.317243+0000","last_undegraded":"2023-11-06T03:04:08.317243+0000","last_fullsized":"2023-11-06T03:04:08.317243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313830+0000","last_change":"2023-11-06T03:04:06.715622+0000","last_active":"2023-11-06T03:04:08.313830+0000","last_peered":"2023-11-06T03:04:08.313830+0000","last_clean":"2023-11-06T03:04:08.313830+0000","last_became_active":"2023-11-06T03:04:06.715290+0000","last_became_peered":"2023-11-06T03:04:06.715290+0000","last_unstale":"2023-11-06T03:04:08.313830+0000","last_undegraded":"2023-11-06T03:04:08.313830+0000","last_fullsized":"2023-11-06T03:04:08.313830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316817+0000","last_change":"2023-11-06T03:04:06.709608+0000","last_active":"2023-11-06T03:04:08.316817+0000","last_peered":"2023-11-06T03:04:08.316817+0000","last_clean":"2023-11-06T03:04:08.316817+0000","last_became_active":"2023-11-06T03:04:06.708904+0000","last_became_peered":"2023-11-06T03:04:06.708904+0000","last_unstale":"2023-11-06T03:04:08.316817+0000","last_undegraded":"2023-11-06T03:04:08.316817+0000","last_fullsized":"2023-11-06T03:04:08.316817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697175+0000","last_change":"2023-11-06T03:04:06.730619+0000","last_active":"2023-11-06T03:04:07.697175+0000","last_peered":"2023-11-06T03:04:07.697175+0000","last_clean":"2023-11-06T03:04:07.697175+0000","last_became_active":"2023-11-06T03:04:06.730192+0000","last_became_peered":"2023-11-06T03:04:06.730192+0000","last_unstale":"2023-11-06T03:04:07.697175+0000","last_undegraded":"2023-11-06T03:04:07.697175+0000","last_fullsized":"2023-11-06T03:04:07.697175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697306+0000","last_change":"2023-11-06T03:04:06.727570+0000","last_active":"2023-11-06T03:04:07.697306+0000","last_peered":"2023-11-06T03:04:07.697306+0000","last_clean":"2023-11-06T03:04:07.697306+0000","last_became_active":"2023-11-06T03:04:06.727305+0000","last_became_peered":"2023-11-06T03:04:06.727305+0000","last_unstale":"2023-11-06T03:04:07.697306+0000","last_undegraded":"2023-11-06T03:04:07.697306+0000","last_fullsized":"2023-11-06T03:04:07.697306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696670+0000","last_change":"2023-11-06T03:04:06.727300+0000","last_active":"2023-11-06T03:04:07.696670+0000","last_peered":"2023-11-06T03:04:07.696670+0000","last_clean":"2023-11-06T03:04:07.696670+0000","last_became_active":"2023-11-06T03:04:06.726311+0000","last_became_peered":"2023-11-06T03:04:06.726311+0000","last_unstale":"2023-11-06T03:04:07.696670+0000","last_undegraded":"2023-11-06T03:04:07.696670+0000","last_fullsized":"2023-11-06T03:04:07.696670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313347+0000","last_change":"2023-11-06T03:04:06.722186+0000","last_active":"2023-11-06T03:04:08.313347+0000","last_peered":"2023-11-06T03:04:08.313347+0000","last_clean":"2023-11-06T03:04:08.313347+0000","last_became_active":"2023-11-06T03:04:06.721514+0000","last_became_peered":"2023-11-06T03:04:06.721514+0000","last_unstale":"2023-11-06T03:04:08.313347+0000","last_undegraded":"2023-11-06T03:04:08.313347+0000","last_fullsized":"2023-11-06T03:04:08.313347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697317+0000","last_change":"2023-11-06T03:04:06.725709+0000","last_active":"2023-11-06T03:04:07.697317+0000","last_peered":"2023-11-06T03:04:07.697317+0000","last_clean":"2023-11-06T03:04:07.697317+0000","last_became_active":"2023-11-06T03:04:06.725248+0000","last_became_peered":"2023-11-06T03:04:06.725248+0000","last_unstale":"2023-11-06T03:04:07.697317+0000","last_undegraded":"2023-11-06T03:04:07.697317+0000","last_fullsized":"2023-11-06T03:04:07.697317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812176+0000","last_change":"2023-11-06T03:04:06.363773+0000","last_active":"2023-11-06T03:04:07.812176+0000","last_peered":"2023-11-06T03:04:07.812176+0000","last_clean":"2023-11-06T03:04:07.812176+0000","last_became_active":"2023-11-06T03:04:06.362996+0000","last_became_peered":"2023-11-06T03:04:06.362996+0000","last_unstale":"2023-11-06T03:04:07.812176+0000","last_undegraded":"2023-11-06T03:04:07.812176+0000","last_fullsized":"2023-11-06T03:04:07.812176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314708+0000","last_change":"2023-11-06T03:04:06.709102+0000","last_active":"2023-11-06T03:04:08.314708+0000","last_peered":"2023-11-06T03:04:08.314708+0000","last_clean":"2023-11-06T03:04:08.314708+0000","last_became_active":"2023-11-06T03:04:06.708817+0000","last_became_peered":"2023-11-06T03:04:06.708817+0000","last_unstale":"2023-11-06T03:04:08.314708+0000","last_undegraded":"2023-11-06T03:04:08.314708+0000","last_fullsized":"2023-11-06T03:04:08.314708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315524+0000","last_change":"2023-11-06T03:04:06.351736+0000","last_active":"2023-11-06T03:04:08.315524+0000","last_peered":"2023-11-06T03:04:08.315524+0000","last_clean":"2023-11-06T03:04:08.315524+0000","last_became_active":"2023-11-06T03:04:06.351500+0000","last_became_peered":"2023-11-06T03:04:06.351500+0000","last_unstale":"2023-11-06T03:04:08.315524+0000","last_undegraded":"2023-11-06T03:04:08.315524+0000","last_fullsized":"2023-11-06T03:04:08.315524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316022+0000","last_change":"2023-11-06T03:04:06.719843+0000","last_active":"2023-11-06T03:04:08.316022+0000","last_peered":"2023-11-06T03:04:08.316022+0000","last_clean":"2023-11-06T03:04:08.316022+0000","last_became_active":"2023-11-06T03:04:06.718738+0000","last_became_peered":"2023-11-06T03:04:06.718738+0000","last_unstale":"2023-11-06T03:04:08.316022+0000","last_undegraded":"2023-11-06T03:04:08.316022+0000","last_fullsized":"2023-11-06T03:04:08.316022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812188+0000","last_change":"2023-11-06T03:04:06.364044+0000","last_active":"2023-11-06T03:04:07.812188+0000","last_peered":"2023-11-06T03:04:07.812188+0000","last_clean":"2023-11-06T03:04:07.812188+0000","last_became_active":"2023-11-06T03:04:06.363453+0000","last_became_peered":"2023-11-06T03:04:06.363453+0000","last_unstale":"2023-11-06T03:04:07.812188+0000","last_undegraded":"2023-11-06T03:04:07.812188+0000","last_fullsized":"2023-11-06T03:04:07.812188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697122+0000","last_change":"2023-11-06T03:04:06.732024+0000","last_active":"2023-11-06T03:04:07.697122+0000","last_peered":"2023-11-06T03:04:07.697122+0000","last_clean":"2023-11-06T03:04:07.697122+0000","last_became_active":"2023-11-06T03:04:06.730677+0000","last_became_peered":"2023-11-06T03:04:06.730677+0000","last_unstale":"2023-11-06T03:04:07.697122+0000","last_undegraded":"2023-11-06T03:04:07.697122+0000","last_fullsized":"2023-11-06T03:04:07.697122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313367+0000","last_change":"2023-11-06T03:04:06.333322+0000","last_active":"2023-11-06T03:04:08.313367+0000","last_peered":"2023-11-06T03:04:08.313367+0000","last_clean":"2023-11-06T03:04:08.313367+0000","last_became_active":"2023-11-06T03:04:06.332581+0000","last_became_peered":"2023-11-06T03:04:06.332581+0000","last_unstale":"2023-11-06T03:04:08.313367+0000","last_undegraded":"2023-11-06T03:04:08.313367+0000","last_fullsized":"2023-11-06T03:04:08.313367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812463+0000","last_change":"2023-11-06T03:04:06.363547+0000","last_active":"2023-11-06T03:04:07.812463+0000","last_peered":"2023-11-06T03:04:07.812463+0000","last_clean":"2023-11-06T03:04:07.812463+0000","last_became_active":"2023-11-06T03:04:06.362774+0000","last_became_peered":"2023-11-06T03:04:06.362774+0000","last_unstale":"2023-11-06T03:04:07.812463+0000","last_undegraded":"2023-11-06T03:04:07.812463+0000","last_fullsized":"2023-11-06T03:04:07.812463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313617+0000","last_change":"2023-11-06T03:04:06.715341+0000","last_active":"2023-11-06T03:04:08.313617+0000","last_peered":"2023-11-06T03:04:08.313617+0000","last_clean":"2023-11-06T03:04:08.313617+0000","last_became_active":"2023-11-06T03:04:06.715020+0000","last_became_peered":"2023-11-06T03:04:06.715020+0000","last_unstale":"2023-11-06T03:04:08.313617+0000","last_undegraded":"2023-11-06T03:04:08.313617+0000","last_fullsized":"2023-11-06T03:04:08.313617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812298+0000","last_change":"2023-11-06T03:04:06.364333+0000","last_active":"2023-11-06T03:04:07.812298+0000","last_peered":"2023-11-06T03:04:07.812298+0000","last_clean":"2023-11-06T03:04:07.812298+0000","last_became_active":"2023-11-06T03:04:06.363641+0000","last_became_peered":"2023-11-06T03:04:06.363641+0000","last_unstale":"2023-11-06T03:04:07.812298+0000","last_undegraded":"2023-11-06T03:04:07.812298+0000","last_fullsized":"2023-11-06T03:04:07.812298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:29.165293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697141+0000","last_change":"2023-11-06T03:04:06.712796+0000","last_active":"2023-11-06T03:04:07.697141+0000","last_peered":"2023-11-06T03:04:07.697141+0000","last_clean":"2023-11-06T03:04:07.697141+0000","last_became_active":"2023-11-06T03:04:06.712442+0000","last_became_peered":"2023-11-06T03:04:06.712442+0000","last_unstale":"2023-11-06T03:04:07.697141+0000","last_undegraded":"2023-11-06T03:04:07.697141+0000","last_fullsized":"2023-11-06T03:04:07.697141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812223+0000","last_change":"2023-11-06T03:04:06.352109+0000","last_active":"2023-11-06T03:04:07.812223+0000","last_peered":"2023-11-06T03:04:07.812223+0000","last_clean":"2023-11-06T03:04:07.812223+0000","last_became_active":"2023-11-06T03:04:06.351112+0000","last_became_peered":"2023-11-06T03:04:06.351112+0000","last_unstale":"2023-11-06T03:04:07.812223+0000","last_undegraded":"2023-11-06T03:04:07.812223+0000","last_fullsized":"2023-11-06T03:04:07.812223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:27.571905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313304+0000","last_change":"2023-11-06T03:04:06.721884+0000","last_active":"2023-11-06T03:04:08.313304+0000","last_peered":"2023-11-06T03:04:08.313304+0000","last_clean":"2023-11-06T03:04:08.313304+0000","last_became_active":"2023-11-06T03:04:06.721714+0000","last_became_peered":"2023-11-06T03:04:06.721714+0000","last_unstale":"2023-11-06T03:04:08.313304+0000","last_undegraded":"2023-11-06T03:04:08.313304+0000","last_fullsized":"2023-11-06T03:04:08.313304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:04.132219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313920+0000","last_change":"2023-11-06T03:04:06.336342+0000","last_active":"2023-11-06T03:04:08.313920+0000","last_peered":"2023-11-06T03:04:08.313920+0000","last_clean":"2023-11-06T03:04:08.313920+0000","last_became_active":"2023-11-06T03:04:06.335639+0000","last_became_peered":"2023-11-06T03:04:06.335639+0000","last_unstale":"2023-11-06T03:04:08.313920+0000","last_undegraded":"2023-11-06T03:04:08.313920+0000","last_fullsized":"2023-11-06T03:04:08.313920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:15.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315124+0000","last_change":"2023-11-06T03:04:06.709502+0000","last_active":"2023-11-06T03:04:08.315124+0000","last_peered":"2023-11-06T03:04:08.315124+0000","last_clean":"2023-11-06T03:04:08.315124+0000","last_became_active":"2023-11-06T03:04:06.709157+0000","last_became_peered":"2023-11-06T03:04:06.709157+0000","last_unstale":"2023-11-06T03:04:08.315124+0000","last_undegraded":"2023-11-06T03:04:08.315124+0000","last_fullsized":"2023-11-06T03:04:08.315124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697206+0000","last_change":"2023-11-06T03:04:06.331517+0000","last_active":"2023-11-06T03:04:07.697206+0000","last_peered":"2023-11-06T03:04:07.697206+0000","last_clean":"2023-11-06T03:04:07.697206+0000","last_became_active":"2023-11-06T03:04:06.331107+0000","last_became_peered":"2023-11-06T03:04:06.331107+0000","last_unstale":"2023-11-06T03:04:07.697206+0000","last_undegraded":"2023-11-06T03:04:07.697206+0000","last_fullsized":"2023-11-06T03:04:07.697206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697752+0000","last_change":"2023-11-06T03:04:06.720549+0000","last_active":"2023-11-06T03:04:07.697752+0000","last_peered":"2023-11-06T03:04:07.697752+0000","last_clean":"2023-11-06T03:04:07.697752+0000","last_became_active":"2023-11-06T03:04:06.719939+0000","last_became_peered":"2023-11-06T03:04:06.719939+0000","last_unstale":"2023-11-06T03:04:07.697752+0000","last_undegraded":"2023-11-06T03:04:07.697752+0000","last_fullsized":"2023-11-06T03:04:07.697752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:34.877473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313501+0000","last_change":"2023-11-06T03:04:06.364318+0000","last_active":"2023-11-06T03:04:08.313501+0000","last_peered":"2023-11-06T03:04:08.313501+0000","last_clean":"2023-11-06T03:04:08.313501+0000","last_became_active":"2023-11-06T03:04:06.363095+0000","last_became_peered":"2023-11-06T03:04:06.363095+0000","last_unstale":"2023-11-06T03:04:08.313501+0000","last_undegraded":"2023-11-06T03:04:08.313501+0000","last_fullsized":"2023-11-06T03:04:08.313501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698194+0000","last_change":"2023-11-06T03:04:06.707941+0000","last_active":"2023-11-06T03:04:07.698194+0000","last_peered":"2023-11-06T03:04:07.698194+0000","last_clean":"2023-11-06T03:04:07.698194+0000","last_became_active":"2023-11-06T03:04:06.707628+0000","last_became_peered":"2023-11-06T03:04:06.707628+0000","last_unstale":"2023-11-06T03:04:07.698194+0000","last_undegraded":"2023-11-06T03:04:07.698194+0000","last_fullsized":"2023-11-06T03:04:07.698194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:55.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697420+0000","last_change":"2023-11-06T03:04:06.355002+0000","last_active":"2023-11-06T03:04:07.697420+0000","last_peered":"2023-11-06T03:04:07.697420+0000","last_clean":"2023-11-06T03:04:07.697420+0000","last_became_active":"2023-11-06T03:04:06.352902+0000","last_became_peered":"2023-11-06T03:04:06.352902+0000","last_unstale":"2023-11-06T03:04:07.697420+0000","last_undegraded":"2023-11-06T03:04:07.697420+0000","last_fullsized":"2023-11-06T03:04:07.697420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697853+0000","last_change":"2023-11-06T03:04:06.721894+0000","last_active":"2023-11-06T03:04:07.697853+0000","last_peered":"2023-11-06T03:04:07.697853+0000","last_clean":"2023-11-06T03:04:07.697853+0000","last_became_active":"2023-11-06T03:04:06.721619+0000","last_became_peered":"2023-11-06T03:04:06.721619+0000","last_unstale":"2023-11-06T03:04:07.697853+0000","last_undegraded":"2023-11-06T03:04:07.697853+0000","last_fullsized":"2023-11-06T03:04:07.697853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696727+0000","last_change":"2023-11-06T03:04:06.330685+0000","last_active":"2023-11-06T03:04:07.696727+0000","last_peered":"2023-11-06T03:04:07.696727+0000","last_clean":"2023-11-06T03:04:07.696727+0000","last_became_active":"2023-11-06T03:04:06.330390+0000","last_became_peered":"2023-11-06T03:04:06.330390+0000","last_unstale":"2023-11-06T03:04:07.696727+0000","last_undegraded":"2023-11-06T03:04:07.696727+0000","last_fullsized":"2023-11-06T03:04:07.696727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317295+0000","last_change":"2023-11-06T03:04:06.721960+0000","last_active":"2023-11-06T03:04:08.317295+0000","last_peered":"2023-11-06T03:04:08.317295+0000","last_clean":"2023-11-06T03:04:08.317295+0000","last_became_active":"2023-11-06T03:04:06.721497+0000","last_became_peered":"2023-11-06T03:04:06.721497+0000","last_unstale":"2023-11-06T03:04:08.317295+0000","last_undegraded":"2023-11-06T03:04:08.317295+0000","last_fullsized":"2023-11-06T03:04:08.317295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:28.582669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700445+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:06.700445+0000","last_peered":"2023-11-06T03:04:06.700445+0000","last_clean":"2023-11-06T03:04:06.700445+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:06.700445+0000","last_undegraded":"2023-11-06T03:04:06.700445+0000","last_fullsized":"2023-11-06T03:04:06.700445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700232+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:06.700232+0000","last_peered":"2023-11-06T03:04:06.700232+0000","last_clean":"2023-11-06T03:04:06.700232+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:06.700232+0000","last_undegraded":"2023-11-06T03:04:06.700232+0000","last_fullsized":"2023-11-06T03:04:06.700232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696999+0000","last_change":"2023-11-06T03:04:06.344186+0000","last_active":"2023-11-06T03:04:07.696999+0000","last_peered":"2023-11-06T03:04:07.696999+0000","last_clean":"2023-11-06T03:04:07.696999+0000","last_became_active":"2023-11-06T03:04:06.343854+0000","last_became_peered":"2023-11-06T03:04:06.343854+0000","last_unstale":"2023-11-06T03:04:07.696999+0000","last_undegraded":"2023-11-06T03:04:07.696999+0000","last_fullsized":"2023-11-06T03:04:07.696999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315060+0000","last_change":"2023-11-06T03:04:06.716182+0000","last_active":"2023-11-06T03:04:08.315060+0000","last_peered":"2023-11-06T03:04:08.315060+0000","last_clean":"2023-11-06T03:04:08.315060+0000","last_became_active":"2023-11-06T03:04:06.714829+0000","last_became_peered":"2023-11-06T03:04:06.714829+0000","last_unstale":"2023-11-06T03:04:08.315060+0000","last_undegraded":"2023-11-06T03:04:08.315060+0000","last_fullsized":"2023-11-06T03:04:08.315060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:20.005858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700210+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:06.700210+0000","last_peered":"2023-11-06T03:04:06.700210+0000","last_clean":"2023-11-06T03:04:06.700210+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:06.700210+0000","last_undegraded":"2023-11-06T03:04:06.700210+0000","last_fullsized":"2023-11-06T03:04:06.700210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314962+0000","last_change":"2023-11-06T03:04:06.359178+0000","last_active":"2023-11-06T03:04:08.314962+0000","last_peered":"2023-11-06T03:04:08.314962+0000","last_clean":"2023-11-06T03:04:08.314962+0000","last_became_active":"2023-11-06T03:04:06.358881+0000","last_became_peered":"2023-11-06T03:04:06.358881+0000","last_unstale":"2023-11-06T03:04:08.314962+0000","last_undegraded":"2023-11-06T03:04:08.314962+0000","last_fullsized":"2023-11-06T03:04:08.314962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316077+0000","last_change":"2023-11-06T03:04:06.712984+0000","last_active":"2023-11-06T03:04:08.316077+0000","last_peered":"2023-11-06T03:04:08.316077+0000","last_clean":"2023-11-06T03:04:08.316077+0000","last_became_active":"2023-11-06T03:04:06.712663+0000","last_became_peered":"2023-11-06T03:04:06.712663+0000","last_unstale":"2023-11-06T03:04:08.316077+0000","last_undegraded":"2023-11-06T03:04:08.316077+0000","last_fullsized":"2023-11-06T03:04:08.316077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.688745+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:06.688745+0000","last_peered":"2023-11-06T03:04:06.688745+0000","last_clean":"2023-11-06T03:04:06.688745+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:06.688745+0000","last_undegraded":"2023-11-06T03:04:06.688745+0000","last_fullsized":"2023-11-06T03:04:06.688745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314913+0000","last_change":"2023-11-06T03:04:06.328651+0000","last_active":"2023-11-06T03:04:08.314913+0000","last_peered":"2023-11-06T03:04:08.314913+0000","last_clean":"2023-11-06T03:04:08.314913+0000","last_became_active":"2023-11-06T03:04:06.328314+0000","last_became_peered":"2023-11-06T03:04:06.328314+0000","last_unstale":"2023-11-06T03:04:08.314913+0000","last_undegraded":"2023-11-06T03:04:08.314913+0000","last_fullsized":"2023-11-06T03:04:08.314913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317506+0000","last_change":"2023-11-06T03:04:06.710848+0000","last_active":"2023-11-06T03:04:08.317506+0000","last_peered":"2023-11-06T03:04:08.317506+0000","last_clean":"2023-11-06T03:04:08.317506+0000","last_became_active":"2023-11-06T03:04:06.709767+0000","last_became_peered":"2023-11-06T03:04:06.709767+0000","last_unstale":"2023-11-06T03:04:08.317506+0000","last_undegraded":"2023-11-06T03:04:08.317506+0000","last_fullsized":"2023-11-06T03:04:08.317506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700275+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:06.700275+0000","last_peered":"2023-11-06T03:04:06.700275+0000","last_clean":"2023-11-06T03:04:06.700275+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:06.700275+0000","last_undegraded":"2023-11-06T03:04:06.700275+0000","last_fullsized":"2023-11-06T03:04:06.700275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697593+0000","last_change":"2023-11-06T03:04:06.354390+0000","last_active":"2023-11-06T03:04:07.697593+0000","last_peered":"2023-11-06T03:04:07.697593+0000","last_clean":"2023-11-06T03:04:07.697593+0000","last_became_active":"2023-11-06T03:04:06.353638+0000","last_became_peered":"2023-11-06T03:04:06.353638+0000","last_unstale":"2023-11-06T03:04:07.697593+0000","last_undegraded":"2023-11-06T03:04:07.697593+0000","last_fullsized":"2023-11-06T03:04:07.697593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316563+0000","last_change":"2023-11-06T03:04:06.714255+0000","last_active":"2023-11-06T03:04:08.316563+0000","last_peered":"2023-11-06T03:04:08.316563+0000","last_clean":"2023-11-06T03:04:08.316563+0000","last_became_active":"2023-11-06T03:04:06.709791+0000","last_became_peered":"2023-11-06T03:04:06.709791+0000","last_unstale":"2023-11-06T03:04:08.316563+0000","last_undegraded":"2023-11-06T03:04:08.316563+0000","last_fullsized":"2023-11-06T03:04:08.316563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314303+0000","last_change":"2023-11-06T03:04:06.338556+0000","last_active":"2023-11-06T03:04:08.314303+0000","last_peered":"2023-11-06T03:04:08.314303+0000","last_clean":"2023-11-06T03:04:08.314303+0000","last_became_active":"2023-11-06T03:04:06.337740+0000","last_became_peered":"2023-11-06T03:04:06.337740+0000","last_unstale":"2023-11-06T03:04:08.314303+0000","last_undegraded":"2023-11-06T03:04:08.314303+0000","last_fullsized":"2023-11-06T03:04:08.314303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.701953+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:06.701953+0000","last_peered":"2023-11-06T03:04:06.701953+0000","last_clean":"2023-11-06T03:04:06.701953+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:06.701953+0000","last_undegraded":"2023-11-06T03:04:06.701953+0000","last_fullsized":"2023-11-06T03:04:06.701953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316863+0000","last_change":"2023-11-06T03:04:06.711498+0000","last_active":"2023-11-06T03:04:08.316863+0000","last_peered":"2023-11-06T03:04:08.316863+0000","last_clean":"2023-11-06T03:04:08.316863+0000","last_became_active":"2023-11-06T03:04:06.708625+0000","last_became_peered":"2023-11-06T03:04:06.708625+0000","last_unstale":"2023-11-06T03:04:08.316863+0000","last_undegraded":"2023-11-06T03:04:08.316863+0000","last_fullsized":"2023-11-06T03:04:08.316863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316721+0000","last_change":"2023-11-06T03:04:06.329676+0000","last_active":"2023-11-06T03:04:08.316721+0000","last_peered":"2023-11-06T03:04:08.316721+0000","last_clean":"2023-11-06T03:04:08.316721+0000","last_became_active":"2023-11-06T03:04:06.329333+0000","last_became_peered":"2023-11-06T03:04:06.329333+0000","last_unstale":"2023-11-06T03:04:08.316721+0000","last_undegraded":"2023-11-06T03:04:08.316721+0000","last_fullsized":"2023-11-06T03:04:08.316721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.703855+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:06.703855+0000","last_peered":"2023-11-06T03:04:06.703855+0000","last_clean":"2023-11-06T03:04:06.703855+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:06.703855+0000","last_undegraded":"2023-11-06T03:04:06.703855+0000","last_fullsized":"2023-11-06T03:04:06.703855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315277+0000","last_change":"2023-11-06T03:04:06.701264+0000","last_active":"2023-11-06T03:04:08.315277+0000","last_peered":"2023-11-06T03:04:08.315277+0000","last_clean":"2023-11-06T03:04:08.315277+0000","last_became_active":"2023-11-06T03:04:06.700955+0000","last_became_peered":"2023-11-06T03:04:06.700955+0000","last_unstale":"2023-11-06T03:04:08.315277+0000","last_undegraded":"2023-11-06T03:04:08.315277+0000","last_fullsized":"2023-11-06T03:04:08.315277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697348+0000","last_change":"2023-11-06T03:04:06.346267+0000","last_active":"2023-11-06T03:04:07.697348+0000","last_peered":"2023-11-06T03:04:07.697348+0000","last_clean":"2023-11-06T03:04:07.697348+0000","last_became_active":"2023-11-06T03:04:06.345991+0000","last_became_peered":"2023-11-06T03:04:06.345991+0000","last_unstale":"2023-11-06T03:04:07.697348+0000","last_undegraded":"2023-11-06T03:04:07.697348+0000","last_fullsized":"2023-11-06T03:04:07.697348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.702101+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:06.702101+0000","last_peered":"2023-11-06T03:04:06.702101+0000","last_clean":"2023-11-06T03:04:06.702101+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:06.702101+0000","last_undegraded":"2023-11-06T03:04:06.702101+0000","last_fullsized":"2023-11-06T03:04:06.702101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696622+0000","last_change":"2023-11-06T03:04:06.731112+0000","last_active":"2023-11-06T03:04:07.696622+0000","last_peered":"2023-11-06T03:04:07.696622+0000","last_clean":"2023-11-06T03:04:07.696622+0000","last_became_active":"2023-11-06T03:04:06.730152+0000","last_became_peered":"2023-11-06T03:04:06.730152+0000","last_unstale":"2023-11-06T03:04:07.696622+0000","last_undegraded":"2023-11-06T03:04:07.696622+0000","last_fullsized":"2023-11-06T03:04:07.696622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316243+0000","last_change":"2023-11-06T03:04:06.331522+0000","last_active":"2023-11-06T03:04:08.316243+0000","last_peered":"2023-11-06T03:04:08.316243+0000","last_clean":"2023-11-06T03:04:08.316243+0000","last_became_active":"2023-11-06T03:04:06.331263+0000","last_became_peered":"2023-11-06T03:04:06.331263+0000","last_unstale":"2023-11-06T03:04:08.316243+0000","last_undegraded":"2023-11-06T03:04:08.316243+0000","last_fullsized":"2023-11-06T03:04:08.316243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.725653+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:06.725653+0000","last_peered":"2023-11-06T03:04:06.725653+0000","last_clean":"2023-11-06T03:04:06.725653+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:06.725653+0000","last_undegraded":"2023-11-06T03:04:06.725653+0000","last_fullsized":"2023-11-06T03:04:06.725653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698182+0000","last_change":"2023-11-06T03:04:06.705426+0000","last_active":"2023-11-06T03:04:07.698182+0000","last_peered":"2023-11-06T03:04:07.698182+0000","last_clean":"2023-11-06T03:04:07.698182+0000","last_became_active":"2023-11-06T03:04:06.705116+0000","last_became_peered":"2023-11-06T03:04:06.705116+0000","last_unstale":"2023-11-06T03:04:07.698182+0000","last_undegraded":"2023-11-06T03:04:07.698182+0000","last_fullsized":"2023-11-06T03:04:07.698182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:26.686747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697675+0000","last_change":"2023-11-06T03:04:06.721464+0000","last_active":"2023-11-06T03:04:07.697675+0000","last_peered":"2023-11-06T03:04:07.697675+0000","last_clean":"2023-11-06T03:04:07.697675+0000","last_became_active":"2023-11-06T03:04:06.721193+0000","last_became_peered":"2023-11-06T03:04:06.721193+0000","last_unstale":"2023-11-06T03:04:07.697675+0000","last_undegraded":"2023-11-06T03:04:07.697675+0000","last_fullsized":"2023-11-06T03:04:07.697675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697996+0000","last_change":"2023-11-06T03:04:06.354926+0000","last_active":"2023-11-06T03:04:07.697996+0000","last_peered":"2023-11-06T03:04:07.697996+0000","last_clean":"2023-11-06T03:04:07.697996+0000","last_became_active":"2023-11-06T03:04:06.354287+0000","last_became_peered":"2023-11-06T03:04:06.354287+0000","last_unstale":"2023-11-06T03:04:07.697996+0000","last_undegraded":"2023-11-06T03:04:07.697996+0000","last_fullsized":"2023-11-06T03:04:07.697996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316609+0000","last_change":"2023-11-06T03:04:06.716076+0000","last_active":"2023-11-06T03:04:08.316609+0000","last_peered":"2023-11-06T03:04:08.316609+0000","last_clean":"2023-11-06T03:04:08.316609+0000","last_became_active":"2023-11-06T03:04:06.715726+0000","last_became_peered":"2023-11-06T03:04:06.715726+0000","last_unstale":"2023-11-06T03:04:08.316609+0000","last_undegraded":"2023-11-06T03:04:08.316609+0000","last_fullsized":"2023-11-06T03:04:08.316609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812115+0000","last_change":"2023-11-06T03:04:06.346560+0000","last_active":"2023-11-06T03:04:07.812115+0000","last_peered":"2023-11-06T03:04:07.812115+0000","last_clean":"2023-11-06T03:04:07.812115+0000","last_became_active":"2023-11-06T03:04:06.345767+0000","last_became_peered":"2023-11-06T03:04:06.345767+0000","last_unstale":"2023-11-06T03:04:07.812115+0000","last_undegraded":"2023-11-06T03:04:07.812115+0000","last_fullsized":"2023-11-06T03:04:07.812115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:18.995094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314192+0000","last_change":"2023-11-06T03:04:06.346611+0000","last_active":"2023-11-06T03:04:08.314192+0000","last_peered":"2023-11-06T03:04:08.314192+0000","last_clean":"2023-11-06T03:04:08.314192+0000","last_became_active":"2023-11-06T03:04:06.345852+0000","last_became_peered":"2023-11-06T03:04:06.345852+0000","last_unstale":"2023-11-06T03:04:08.314192+0000","last_undegraded":"2023-11-06T03:04:08.314192+0000","last_fullsized":"2023-11-06T03:04:08.314192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696805+0000","last_change":"2023-11-06T03:04:06.727615+0000","last_active":"2023-11-06T03:04:07.696805+0000","last_peered":"2023-11-06T03:04:07.696805+0000","last_clean":"2023-11-06T03:04:07.696805+0000","last_became_active":"2023-11-06T03:04:06.726871+0000","last_became_peered":"2023-11-06T03:04:06.726871+0000","last_unstale":"2023-11-06T03:04:07.696805+0000","last_undegraded":"2023-11-06T03:04:07.696805+0000","last_fullsized":"2023-11-06T03:04:07.696805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315989+0000","last_change":"2023-11-06T03:04:06.344230+0000","last_active":"2023-11-06T03:04:08.315989+0000","last_peered":"2023-11-06T03:04:08.315989+0000","last_clean":"2023-11-06T03:04:08.315989+0000","last_became_active":"2023-11-06T03:04:06.343852+0000","last_became_peered":"2023-11-06T03:04:06.343852+0000","last_unstale":"2023-11-06T03:04:08.315989+0000","last_undegraded":"2023-11-06T03:04:08.315989+0000","last_fullsized":"2023-11-06T03:04:08.315989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696536+0000","last_change":"2023-11-06T03:04:06.729233+0000","last_active":"2023-11-06T03:04:07.696536+0000","last_peered":"2023-11-06T03:04:07.696536+0000","last_clean":"2023-11-06T03:04:07.696536+0000","last_became_active":"2023-11-06T03:04:06.728828+0000","last_became_peered":"2023-11-06T03:04:06.728828+0000","last_unstale":"2023-11-06T03:04:07.696536+0000","last_undegraded":"2023-11-06T03:04:07.696536+0000","last_fullsized":"2023-11-06T03:04:07.696536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315468+0000","last_change":"2023-11-06T03:04:06.721167+0000","last_active":"2023-11-06T03:04:08.315468+0000","last_peered":"2023-11-06T03:04:08.315468+0000","last_clean":"2023-11-06T03:04:08.315468+0000","last_became_active":"2023-11-06T03:04:06.720914+0000","last_became_peered":"2023-11-06T03:04:06.720914+0000","last_unstale":"2023-11-06T03:04:08.315468+0000","last_undegraded":"2023-11-06T03:04:08.315468+0000","last_fullsized":"2023-11-06T03:04:08.315468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:55.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696763+0000","last_change":"2023-11-06T03:04:06.331738+0000","last_active":"2023-11-06T03:04:07.696763+0000","last_peered":"2023-11-06T03:04:07.696763+0000","last_clean":"2023-11-06T03:04:07.696763+0000","last_became_active":"2023-11-06T03:04:06.330746+0000","last_became_peered":"2023-11-06T03:04:06.330746+0000","last_unstale":"2023-11-06T03:04:07.696763+0000","last_undegraded":"2023-11-06T03:04:07.696763+0000","last_fullsized":"2023-11-06T03:04:07.696763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315727+0000","last_change":"2023-11-06T03:04:06.700157+0000","last_active":"2023-11-06T03:04:08.315727+0000","last_peered":"2023-11-06T03:04:08.315727+0000","last_clean":"2023-11-06T03:04:08.315727+0000","last_became_active":"2023-11-06T03:04:06.699680+0000","last_became_peered":"2023-11-06T03:04:06.699680+0000","last_unstale":"2023-11-06T03:04:08.315727+0000","last_undegraded":"2023-11-06T03:04:08.315727+0000","last_fullsized":"2023-11-06T03:04:08.315727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316079+0000","last_change":"2023-11-06T03:04:06.330647+0000","last_active":"2023-11-06T03:04:08.316079+0000","last_peered":"2023-11-06T03:04:08.316079+0000","last_clean":"2023-11-06T03:04:08.316079+0000","last_became_active":"2023-11-06T03:04:06.330043+0000","last_became_peered":"2023-11-06T03:04:06.330043+0000","last_unstale":"2023-11-06T03:04:08.316079+0000","last_undegraded":"2023-11-06T03:04:08.316079+0000","last_fullsized":"2023-11-06T03:04:08.316079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.724159+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.723798+0000","last_became_peered":"2023-11-06T03:04:06.723798+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.341434+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.340687+0000","last_became_peered":"2023-11-06T03:04:06.340687+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:44.206262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315009+0000","last_change":"2023-11-06T03:04:06.719856+0000","last_active":"2023-11-06T03:04:08.315009+0000","last_peered":"2023-11-06T03:04:08.315009+0000","last_clean":"2023-11-06T03:04:08.315009+0000","last_became_active":"2023-11-06T03:04:06.718775+0000","last_became_peered":"2023-11-06T03:04:06.718775+0000","last_unstale":"2023-11-06T03:04:08.315009+0000","last_undegraded":"2023-11-06T03:04:08.315009+0000","last_fullsized":"2023-11-06T03:04:08.315009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:21.577004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315849+0000","last_change":"2023-11-06T03:04:06.359042+0000","last_active":"2023-11-06T03:04:08.315849+0000","last_peered":"2023-11-06T03:04:08.315849+0000","last_clean":"2023-11-06T03:04:08.315849+0000","last_became_active":"2023-11-06T03:04:06.358768+0000","last_became_peered":"2023-11-06T03:04:06.358768+0000","last_unstale":"2023-11-06T03:04:08.315849+0000","last_undegraded":"2023-11-06T03:04:08.315849+0000","last_fullsized":"2023-11-06T03:04:08.315849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697472+0000","last_change":"2023-11-06T03:04:06.723498+0000","last_active":"2023-11-06T03:04:07.697472+0000","last_peered":"2023-11-06T03:04:07.697472+0000","last_clean":"2023-11-06T03:04:07.697472+0000","last_became_active":"2023-11-06T03:04:06.723271+0000","last_became_peered":"2023-11-06T03:04:06.723271+0000","last_unstale":"2023-11-06T03:04:07.697472+0000","last_undegraded":"2023-11-06T03:04:07.697472+0000","last_fullsized":"2023-11-06T03:04:07.697472+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:26.972193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313662+0000","last_change":"2023-11-06T03:04:06.359575+0000","last_active":"2023-11-06T03:04:08.313662+0000","last_peered":"2023-11-06T03:04:08.313662+0000","last_clean":"2023-11-06T03:04:08.313662+0000","last_became_active":"2023-11-06T03:04:06.358835+0000","last_became_peered":"2023-11-06T03:04:06.358835+0000","last_unstale":"2023-11-06T03:04:08.313662+0000","last_undegraded":"2023-11-06T03:04:08.313662+0000","last_fullsized":"2023-11-06T03:04:08.313662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697217+0000","last_change":"2023-11-06T03:04:06.712260+0000","last_active":"2023-11-06T03:04:07.697217+0000","last_peered":"2023-11-06T03:04:07.697217+0000","last_clean":"2023-11-06T03:04:07.697217+0000","last_became_active":"2023-11-06T03:04:06.711945+0000","last_became_peered":"2023-11-06T03:04:06.711945+0000","last_unstale":"2023-11-06T03:04:07.697217+0000","last_undegraded":"2023-11-06T03:04:07.697217+0000","last_fullsized":"2023-11-06T03:04:07.697217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:06.799825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314081+0000","last_change":"2023-11-06T03:04:06.350220+0000","last_active":"2023-11-06T03:04:08.314081+0000","last_peered":"2023-11-06T03:04:08.314081+0000","last_clean":"2023-11-06T03:04:08.314081+0000","last_became_active":"2023-11-06T03:04:06.349458+0000","last_became_peered":"2023-11-06T03:04:06.349458+0000","last_unstale":"2023-11-06T03:04:08.314081+0000","last_undegraded":"2023-11-06T03:04:08.314081+0000","last_fullsized":"2023-11-06T03:04:08.314081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697048+0000","last_change":"2023-11-06T03:04:06.710979+0000","last_active":"2023-11-06T03:04:07.697048+0000","last_peered":"2023-11-06T03:04:07.697048+0000","last_clean":"2023-11-06T03:04:07.697048+0000","last_became_active":"2023-11-06T03:04:06.709663+0000","last_became_peered":"2023-11-06T03:04:06.709663+0000","last_unstale":"2023-11-06T03:04:07.697048+0000","last_undegraded":"2023-11-06T03:04:07.697048+0000","last_fullsized":"2023-11-06T03:04:07.697048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697612+0000","last_change":"2023-11-06T03:04:06.333374+0000","last_active":"2023-11-06T03:04:07.697612+0000","last_peered":"2023-11-06T03:04:07.697612+0000","last_clean":"2023-11-06T03:04:07.697612+0000","last_became_active":"2023-11-06T03:04:06.332910+0000","last_became_peered":"2023-11-06T03:04:06.332910+0000","last_unstale":"2023-11-06T03:04:07.697612+0000","last_undegraded":"2023-11-06T03:04:07.697612+0000","last_fullsized":"2023-11-06T03:04:07.697612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697342+0000","last_change":"2023-11-06T03:04:06.722605+0000","last_active":"2023-11-06T03:04:07.697342+0000","last_peered":"2023-11-06T03:04:07.697342+0000","last_clean":"2023-11-06T03:04:07.697342+0000","last_became_active":"2023-11-06T03:04:06.722317+0000","last_became_peered":"2023-11-06T03:04:06.722317+0000","last_unstale":"2023-11-06T03:04:07.697342+0000","last_undegraded":"2023-11-06T03:04:07.697342+0000","last_fullsized":"2023-11-06T03:04:07.697342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315853+0000","last_change":"2023-11-06T03:04:06.344293+0000","last_active":"2023-11-06T03:04:08.315853+0000","last_peered":"2023-11-06T03:04:08.315853+0000","last_clean":"2023-11-06T03:04:08.315853+0000","last_became_active":"2023-11-06T03:04:06.344022+0000","last_became_peered":"2023-11-06T03:04:06.344022+0000","last_unstale":"2023-11-06T03:04:08.315853+0000","last_undegraded":"2023-11-06T03:04:08.315853+0000","last_fullsized":"2023-11-06T03:04:08.315853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":237568,"data_stored":88580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":88670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:14.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:14.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.953+0000 7faac794d640 1 Processor -- start 2023-11-06T03:04:14.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.953+0000 7faac794d640 1 -- start start 2023-11-06T03:04:14.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac794d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faac00ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac794d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00cc2b0 con 0x7faac00cd730 2023-11-06T03:04:14.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac794d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00b01a0 con 0x7faac015ac80 2023-11-06T03:04:14.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac794d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00cfb10 con 0x7faac00b2b10 2023-11-06T03:04:14.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac5eea640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faac00cd730 0x7faac00cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43510/0 (socket says 172.21.15.157:43510) 2023-11-06T03:04:14.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac5eea640 1 -- 172.21.15.157:0/1575839996 learned_addr learned my addr 172.21.15.157:0/1575839996 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:14.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac4ee8640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faac00ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640206872 0 0) 0x7faac00cc2b0 con 0x7faac00cd730 2023-11-06T03:04:14.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab0003530 con 0x7faac00cd730 2023-11-06T03:04:14.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806508346 0 0) 0x7faab0003530 con 0x7faac00cd730 2023-11-06T03:04:14.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 msgr2=0x7faac00ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:14.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faac00ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:14.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 >> v1:172.21.15.195:6789/0 conn(0x7faac015ac80 legacy=0x7faac00b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:14.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00cfdf0 con 0x7faac00cd730 2023-11-06T03:04:14.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7faab8001680 con 0x7faac00cd730 2023-11-06T03:04:14.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 >> v1:172.21.15.157:6789/0 conn(0x7faac00cd730 legacy=0x7faac00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00039b0 0x7faab0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faab00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab0004d10 0x7faab0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00cfb10 con 0x7faab00039b0 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab000ba40 con 0x7faab0004d10 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab000bc00 con 0x7faac00b2b10 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00039b0 0x7faab0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac5eea640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab0004d10 0x7faab0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac4ee8640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faab00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 -- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 msgr2=0x7faab00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:14.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac00b2b10 0x7faab00047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:14.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 -- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab0004d10 msgr2=0x7faab0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:14.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab0004d10 0x7faab0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:14.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 -- 172.21.15.157:0/1575839996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0003530 con 0x7faab00039b0 2023-11-06T03:04:14.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faac56e9640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00039b0 0x7faab0003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faab4001e40 tx=0x7faab401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673951ff2b88e2d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:14.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4003b80 con 0x7faab00039b0 2023-11-06T03:04:14.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4003ce0 con 0x7faab00039b0 2023-11-06T03:04:14.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.957+0000 7faabffff640 1 -- 172.21.15.157:0/1575839996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40052c0 con 0x7faab00039b0 2023-11-06T03:04:14.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1575839996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00039b0 msgr2=0x7faab0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:14.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 --2- 172.21.15.157:0/1575839996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab00039b0 0x7faab0003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faab4001e40 tx=0x7faab401c740 comp rx=0 tx=0).stop 2023-11-06T03:04:14.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 -- 172.21.15.157:0/1575839996 reap_dead start 2023-11-06T03:04:14.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1575839996 shutdown_connections 2023-11-06T03:04:14.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1575839996 >> 172.21.15.157:0/1575839996 conn(0x7faac006d560 msgr2=0x7faac006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:14.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1575839996 shutdown_connections 2023-11-06T03:04:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1575839996 wait complete. 2023-11-06T03:04:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 Processor -- start 2023-11-06T03:04:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- start start 2023-11-06T03:04:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 0x7faac0160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faac015f610 0x7faac00ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00adde0 con 0x7faac00ad930 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0160720 con 0x7faac015f610 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b4310 con 0x7faac014f750 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43668/0 (socket says 172.21.15.157:43668) 2023-11-06T03:04:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 -- 172.21.15.157:0/1323199153 learned_addr learned my addr 172.21.15.157:0/1323199153 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac56e9640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 0x7faac0160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac4ee8640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faac015f610 0x7faac00ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 -- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 msgr2=0x7faac0160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:14.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 0x7faac0160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 -- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faac015f610 msgr2=0x7faac00ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faac015f610 0x7faac00ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 -- 172.21.15.157:0/1323199153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac015fb10 con 0x7faac00ad930 2023-11-06T03:04:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac56e9640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 0x7faac0160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac5eea640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faac00ce3e0 tx=0x7faab800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:14.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8012650 con 0x7faac00ad930 2023-11-06T03:04:14.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab80127d0 con 0x7faac00ad930 2023-11-06T03:04:14.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac015fdf0 con 0x7faac00ad930 2023-11-06T03:04:14.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8012b80 con 0x7faac00ad930 2023-11-06T03:04:14.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.961+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01603b0 con 0x7faac00ad930 2023-11-06T03:04:14.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac01608d0 con 0x7faac00ad930 2023-11-06T03:04:14.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faab8012ce0 con 0x7faac00ad930 2023-11-06T03:04:14.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faabdffb640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 0x7faaa4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:14.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faab8082b00 con 0x7faac00ad930 2023-11-06T03:04:14.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faac56e9640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 0x7faaa4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:14.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.965+0000 7faac56e9640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 0x7faaa4067f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faab4005000 tx=0x7faab40a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:14.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:14.969+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faac01608d0 con 0x7faac00ad930 2023-11-06T03:04:15.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.117+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faac01608d0 con 0x7faaa4065a90 2023-11-06T03:04:15.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.137+0000 7faabdffb640 1 -- 172.21.15.157:0/1323199153 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286565 (secure 0 0 0) 0x7faac01608d0 con 0x7faaa4065a90 2023-11-06T03:04:15.144 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:15.146 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:15.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 msgr2=0x7faaa4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 0x7faaa4067f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faab4005000 tx=0x7faab40a9040 comp rx=0 tx=0).stop 2023-11-06T03:04:15.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 msgr2=0x7faac00b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faac00ce3e0 tx=0x7faab800ead0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 shutdown_connections 2023-11-06T03:04:15.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaa4065a90 0x7faaa4067f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faac00ad930 0x7faac00b35a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faac015f610 0x7faac00ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 --2- 172.21.15.157:0/1323199153 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faac014f750 0x7faac0160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 >> 172.21.15.157:0/1323199153 conn(0x7faac006d560 msgr2=0x7faac0165e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:15.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 shutdown_connections 2023-11-06T03:04:15.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.141+0000 7faac794d640 1 -- 172.21.15.157:0/1323199153 wait complete. 2023-11-06T03:04:15.161 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-06T03:04:15.096274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251484,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723236,"statfs":{"total":773094113280,"available":772836593664,"internally_reserved":0,"allocated":3174400,"data_stored":1899195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.410447"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314154+0000","last_change":"2023-11-06T03:04:06.723013+0000","last_active":"2023-11-06T03:04:08.314154+0000","last_peered":"2023-11-06T03:04:08.314154+0000","last_clean":"2023-11-06T03:04:08.314154+0000","last_became_active":"2023-11-06T03:04:06.722313+0000","last_became_peered":"2023-11-06T03:04:06.722313+0000","last_unstale":"2023-11-06T03:04:08.314154+0000","last_undegraded":"2023-11-06T03:04:08.314154+0000","last_fullsized":"2023-11-06T03:04:08.314154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:17.965940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313638+0000","last_change":"2023-11-06T03:04:06.709695+0000","last_active":"2023-11-06T03:04:08.313638+0000","last_peered":"2023-11-06T03:04:08.313638+0000","last_clean":"2023-11-06T03:04:08.313638+0000","last_became_active":"2023-11-06T03:04:06.709429+0000","last_became_peered":"2023-11-06T03:04:06.709429+0000","last_unstale":"2023-11-06T03:04:08.313638+0000","last_undegraded":"2023-11-06T03:04:08.313638+0000","last_fullsized":"2023-11-06T03:04:08.313638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697946+0000","last_change":"2023-11-06T03:04:06.700734+0000","last_active":"2023-11-06T03:04:07.697946+0000","last_peered":"2023-11-06T03:04:07.697946+0000","last_clean":"2023-11-06T03:04:07.697946+0000","last_became_active":"2023-11-06T03:04:06.699143+0000","last_became_peered":"2023-11-06T03:04:06.699143+0000","last_unstale":"2023-11-06T03:04:07.697946+0000","last_undegraded":"2023-11-06T03:04:07.697946+0000","last_fullsized":"2023-11-06T03:04:07.697946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315057+0000","last_change":"2023-11-06T03:04:06.725804+0000","last_active":"2023-11-06T03:04:08.315057+0000","last_peered":"2023-11-06T03:04:08.315057+0000","last_clean":"2023-11-06T03:04:08.315057+0000","last_became_active":"2023-11-06T03:04:06.725496+0000","last_became_peered":"2023-11-06T03:04:06.725496+0000","last_unstale":"2023-11-06T03:04:08.315057+0000","last_undegraded":"2023-11-06T03:04:08.315057+0000","last_fullsized":"2023-11-06T03:04:08.315057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:30.176058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315579+0000","last_change":"2023-11-06T03:04:06.725612+0000","last_active":"2023-11-06T03:04:08.315579+0000","last_peered":"2023-11-06T03:04:08.315579+0000","last_clean":"2023-11-06T03:04:08.315579+0000","last_became_active":"2023-11-06T03:04:06.725099+0000","last_became_peered":"2023-11-06T03:04:06.725099+0000","last_unstale":"2023-11-06T03:04:08.315579+0000","last_undegraded":"2023-11-06T03:04:08.315579+0000","last_fullsized":"2023-11-06T03:04:08.315579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697168+0000","last_change":"2023-11-06T03:04:06.710419+0000","last_active":"2023-11-06T03:04:07.697168+0000","last_peered":"2023-11-06T03:04:07.697168+0000","last_clean":"2023-11-06T03:04:07.697168+0000","last_became_active":"2023-11-06T03:04:06.709958+0000","last_became_peered":"2023-11-06T03:04:06.709958+0000","last_unstale":"2023-11-06T03:04:07.697168+0000","last_undegraded":"2023-11-06T03:04:07.697168+0000","last_fullsized":"2023-11-06T03:04:07.697168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:58.848613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811621+0000","last_change":"2023-11-06T03:04:06.720209+0000","last_active":"2023-11-06T03:04:07.811621+0000","last_peered":"2023-11-06T03:04:07.811621+0000","last_clean":"2023-11-06T03:04:07.811621+0000","last_became_active":"2023-11-06T03:04:06.719857+0000","last_became_peered":"2023-11-06T03:04:06.719857+0000","last_unstale":"2023-11-06T03:04:07.811621+0000","last_undegraded":"2023-11-06T03:04:07.811621+0000","last_fullsized":"2023-11-06T03:04:07.811621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313002+0000","last_change":"2023-11-06T03:04:06.706215+0000","last_active":"2023-11-06T03:04:08.313002+0000","last_peered":"2023-11-06T03:04:08.313002+0000","last_clean":"2023-11-06T03:04:08.313002+0000","last_became_active":"2023-11-06T03:04:06.705329+0000","last_became_peered":"2023-11-06T03:04:06.705329+0000","last_unstale":"2023-11-06T03:04:08.313002+0000","last_undegraded":"2023-11-06T03:04:08.313002+0000","last_fullsized":"2023-11-06T03:04:08.313002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315584+0000","last_change":"2023-11-06T03:04:06.725727+0000","last_active":"2023-11-06T03:04:08.315584+0000","last_peered":"2023-11-06T03:04:08.315584+0000","last_clean":"2023-11-06T03:04:08.315584+0000","last_became_active":"2023-11-06T03:04:06.725241+0000","last_became_peered":"2023-11-06T03:04:06.725241+0000","last_unstale":"2023-11-06T03:04:08.315584+0000","last_undegraded":"2023-11-06T03:04:08.315584+0000","last_fullsized":"2023-11-06T03:04:08.315584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313982+0000","last_change":"2023-11-06T03:04:06.701948+0000","last_active":"2023-11-06T03:04:08.313982+0000","last_peered":"2023-11-06T03:04:08.313982+0000","last_clean":"2023-11-06T03:04:08.313982+0000","last_became_active":"2023-11-06T03:04:06.701405+0000","last_became_peered":"2023-11-06T03:04:06.701405+0000","last_unstale":"2023-11-06T03:04:08.313982+0000","last_undegraded":"2023-11-06T03:04:08.313982+0000","last_fullsized":"2023-11-06T03:04:08.313982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698195+0000","last_change":"2023-11-06T03:04:06.711152+0000","last_active":"2023-11-06T03:04:07.698195+0000","last_peered":"2023-11-06T03:04:07.698195+0000","last_clean":"2023-11-06T03:04:07.698195+0000","last_became_active":"2023-11-06T03:04:06.710840+0000","last_became_peered":"2023-11-06T03:04:06.710840+0000","last_unstale":"2023-11-06T03:04:07.698195+0000","last_undegraded":"2023-11-06T03:04:07.698195+0000","last_fullsized":"2023-11-06T03:04:07.698195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314771+0000","last_change":"2023-11-06T03:04:06.703107+0000","last_active":"2023-11-06T03:04:08.314771+0000","last_peered":"2023-11-06T03:04:08.314771+0000","last_clean":"2023-11-06T03:04:08.314771+0000","last_became_active":"2023-11-06T03:04:06.702533+0000","last_became_peered":"2023-11-06T03:04:06.702533+0000","last_unstale":"2023-11-06T03:04:08.314771+0000","last_undegraded":"2023-11-06T03:04:08.314771+0000","last_fullsized":"2023-11-06T03:04:08.314771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314211+0000","last_change":"2023-11-06T03:04:06.710998+0000","last_active":"2023-11-06T03:04:08.314211+0000","last_peered":"2023-11-06T03:04:08.314211+0000","last_clean":"2023-11-06T03:04:08.314211+0000","last_became_active":"2023-11-06T03:04:06.710459+0000","last_became_peered":"2023-11-06T03:04:06.710459+0000","last_unstale":"2023-11-06T03:04:08.314211+0000","last_undegraded":"2023-11-06T03:04:08.314211+0000","last_fullsized":"2023-11-06T03:04:08.314211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:36.030631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313983+0000","last_change":"2023-11-06T03:04:06.708520+0000","last_active":"2023-11-06T03:04:08.313983+0000","last_peered":"2023-11-06T03:04:08.313983+0000","last_clean":"2023-11-06T03:04:08.313983+0000","last_became_active":"2023-11-06T03:04:06.708200+0000","last_became_peered":"2023-11-06T03:04:06.708200+0000","last_unstale":"2023-11-06T03:04:08.313983+0000","last_undegraded":"2023-11-06T03:04:08.313983+0000","last_fullsized":"2023-11-06T03:04:08.313983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811766+0000","last_change":"2023-11-06T03:04:06.723418+0000","last_active":"2023-11-06T03:04:07.811766+0000","last_peered":"2023-11-06T03:04:07.811766+0000","last_clean":"2023-11-06T03:04:07.811766+0000","last_became_active":"2023-11-06T03:04:06.720428+0000","last_became_peered":"2023-11-06T03:04:06.720428+0000","last_unstale":"2023-11-06T03:04:07.811766+0000","last_undegraded":"2023-11-06T03:04:07.811766+0000","last_fullsized":"2023-11-06T03:04:07.811766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697132+0000","last_change":"2023-11-06T03:04:06.699686+0000","last_active":"2023-11-06T03:04:07.697132+0000","last_peered":"2023-11-06T03:04:07.697132+0000","last_clean":"2023-11-06T03:04:07.697132+0000","last_became_active":"2023-11-06T03:04:06.699150+0000","last_became_peered":"2023-11-06T03:04:06.699150+0000","last_unstale":"2023-11-06T03:04:07.697132+0000","last_undegraded":"2023-11-06T03:04:07.697132+0000","last_fullsized":"2023-11-06T03:04:07.697132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315781+0000","last_change":"2023-11-06T03:04:06.718065+0000","last_active":"2023-11-06T03:04:08.315781+0000","last_peered":"2023-11-06T03:04:08.315781+0000","last_clean":"2023-11-06T03:04:08.315781+0000","last_became_active":"2023-11-06T03:04:06.717495+0000","last_became_peered":"2023-11-06T03:04:06.717495+0000","last_unstale":"2023-11-06T03:04:08.315781+0000","last_undegraded":"2023-11-06T03:04:08.315781+0000","last_fullsized":"2023-11-06T03:04:08.315781+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697937+0000","last_change":"2023-11-06T03:04:06.707851+0000","last_active":"2023-11-06T03:04:07.697937+0000","last_peered":"2023-11-06T03:04:07.697937+0000","last_clean":"2023-11-06T03:04:07.697937+0000","last_became_active":"2023-11-06T03:04:06.707587+0000","last_became_peered":"2023-11-06T03:04:06.707587+0000","last_unstale":"2023-11-06T03:04:07.697937+0000","last_undegraded":"2023-11-06T03:04:07.697937+0000","last_fullsized":"2023-11-06T03:04:07.697937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:52:19.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314357+0000","last_change":"2023-11-06T03:04:06.724261+0000","last_active":"2023-11-06T03:04:08.314357+0000","last_peered":"2023-11-06T03:04:08.314357+0000","last_clean":"2023-11-06T03:04:08.314357+0000","last_became_active":"2023-11-06T03:04:06.723974+0000","last_became_peered":"2023-11-06T03:04:06.723974+0000","last_unstale":"2023-11-06T03:04:08.314357+0000","last_undegraded":"2023-11-06T03:04:08.314357+0000","last_fullsized":"2023-11-06T03:04:08.314357+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:16.954584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697610+0000","last_change":"2023-11-06T03:04:06.729633+0000","last_active":"2023-11-06T03:04:07.697610+0000","last_peered":"2023-11-06T03:04:07.697610+0000","last_clean":"2023-11-06T03:04:07.697610+0000","last_became_active":"2023-11-06T03:04:06.729241+0000","last_became_peered":"2023-11-06T03:04:06.729241+0000","last_unstale":"2023-11-06T03:04:07.697610+0000","last_undegraded":"2023-11-06T03:04:07.697610+0000","last_fullsized":"2023-11-06T03:04:07.697610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:05.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811862+0000","last_change":"2023-11-06T03:04:06.728388+0000","last_active":"2023-11-06T03:04:07.811862+0000","last_peered":"2023-11-06T03:04:07.811862+0000","last_clean":"2023-11-06T03:04:07.811862+0000","last_became_active":"2023-11-06T03:04:06.728135+0000","last_became_peered":"2023-11-06T03:04:06.728135+0000","last_unstale":"2023-11-06T03:04:07.811862+0000","last_undegraded":"2023-11-06T03:04:07.811862+0000","last_fullsized":"2023-11-06T03:04:07.811862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315482+0000","last_change":"2023-11-06T03:04:06.713228+0000","last_active":"2023-11-06T03:04:08.315482+0000","last_peered":"2023-11-06T03:04:08.315482+0000","last_clean":"2023-11-06T03:04:08.315482+0000","last_became_active":"2023-11-06T03:04:06.712822+0000","last_became_peered":"2023-11-06T03:04:06.712822+0000","last_unstale":"2023-11-06T03:04:08.315482+0000","last_undegraded":"2023-11-06T03:04:08.315482+0000","last_fullsized":"2023-11-06T03:04:08.315482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696649+0000","last_change":"2023-11-06T03:04:06.711437+0000","last_active":"2023-11-06T03:04:07.696649+0000","last_peered":"2023-11-06T03:04:07.696649+0000","last_clean":"2023-11-06T03:04:07.696649+0000","last_became_active":"2023-11-06T03:04:06.710409+0000","last_became_peered":"2023-11-06T03:04:06.710409+0000","last_unstale":"2023-11-06T03:04:07.696649+0000","last_undegraded":"2023-11-06T03:04:07.696649+0000","last_fullsized":"2023-11-06T03:04:07.696649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811784+0000","last_change":"2023-11-06T03:04:06.728415+0000","last_active":"2023-11-06T03:04:07.811784+0000","last_peered":"2023-11-06T03:04:07.811784+0000","last_clean":"2023-11-06T03:04:07.811784+0000","last_became_active":"2023-11-06T03:04:06.728111+0000","last_became_peered":"2023-11-06T03:04:06.728111+0000","last_unstale":"2023-11-06T03:04:07.811784+0000","last_undegraded":"2023-11-06T03:04:07.811784+0000","last_fullsized":"2023-11-06T03:04:07.811784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317080+0000","last_change":"2023-11-06T03:04:06.727933+0000","last_active":"2023-11-06T03:04:08.317080+0000","last_peered":"2023-11-06T03:04:08.317080+0000","last_clean":"2023-11-06T03:04:08.317080+0000","last_became_active":"2023-11-06T03:04:06.727689+0000","last_became_peered":"2023-11-06T03:04:06.727689+0000","last_unstale":"2023-11-06T03:04:08.317080+0000","last_undegraded":"2023-11-06T03:04:08.317080+0000","last_fullsized":"2023-11-06T03:04:08.317080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317243+0000","last_change":"2023-11-06T03:04:06.709146+0000","last_active":"2023-11-06T03:04:08.317243+0000","last_peered":"2023-11-06T03:04:08.317243+0000","last_clean":"2023-11-06T03:04:08.317243+0000","last_became_active":"2023-11-06T03:04:06.708760+0000","last_became_peered":"2023-11-06T03:04:06.708760+0000","last_unstale":"2023-11-06T03:04:08.317243+0000","last_undegraded":"2023-11-06T03:04:08.317243+0000","last_fullsized":"2023-11-06T03:04:08.317243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313830+0000","last_change":"2023-11-06T03:04:06.715622+0000","last_active":"2023-11-06T03:04:08.313830+0000","last_peered":"2023-11-06T03:04:08.313830+0000","last_clean":"2023-11-06T03:04:08.313830+0000","last_became_active":"2023-11-06T03:04:06.715290+0000","last_became_peered":"2023-11-06T03:04:06.715290+0000","last_unstale":"2023-11-06T03:04:08.313830+0000","last_undegraded":"2023-11-06T03:04:08.313830+0000","last_fullsized":"2023-11-06T03:04:08.313830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316817+0000","last_change":"2023-11-06T03:04:06.709608+0000","last_active":"2023-11-06T03:04:08.316817+0000","last_peered":"2023-11-06T03:04:08.316817+0000","last_clean":"2023-11-06T03:04:08.316817+0000","last_became_active":"2023-11-06T03:04:06.708904+0000","last_became_peered":"2023-11-06T03:04:06.708904+0000","last_unstale":"2023-11-06T03:04:08.316817+0000","last_undegraded":"2023-11-06T03:04:08.316817+0000","last_fullsized":"2023-11-06T03:04:08.316817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697175+0000","last_change":"2023-11-06T03:04:06.730619+0000","last_active":"2023-11-06T03:04:07.697175+0000","last_peered":"2023-11-06T03:04:07.697175+0000","last_clean":"2023-11-06T03:04:07.697175+0000","last_became_active":"2023-11-06T03:04:06.730192+0000","last_became_peered":"2023-11-06T03:04:06.730192+0000","last_unstale":"2023-11-06T03:04:07.697175+0000","last_undegraded":"2023-11-06T03:04:07.697175+0000","last_fullsized":"2023-11-06T03:04:07.697175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697306+0000","last_change":"2023-11-06T03:04:06.727570+0000","last_active":"2023-11-06T03:04:07.697306+0000","last_peered":"2023-11-06T03:04:07.697306+0000","last_clean":"2023-11-06T03:04:07.697306+0000","last_became_active":"2023-11-06T03:04:06.727305+0000","last_became_peered":"2023-11-06T03:04:06.727305+0000","last_unstale":"2023-11-06T03:04:07.697306+0000","last_undegraded":"2023-11-06T03:04:07.697306+0000","last_fullsized":"2023-11-06T03:04:07.697306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696670+0000","last_change":"2023-11-06T03:04:06.727300+0000","last_active":"2023-11-06T03:04:07.696670+0000","last_peered":"2023-11-06T03:04:07.696670+0000","last_clean":"2023-11-06T03:04:07.696670+0000","last_became_active":"2023-11-06T03:04:06.726311+0000","last_became_peered":"2023-11-06T03:04:06.726311+0000","last_unstale":"2023-11-06T03:04:07.696670+0000","last_undegraded":"2023-11-06T03:04:07.696670+0000","last_fullsized":"2023-11-06T03:04:07.696670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313347+0000","last_change":"2023-11-06T03:04:06.722186+0000","last_active":"2023-11-06T03:04:08.313347+0000","last_peered":"2023-11-06T03:04:08.313347+0000","last_clean":"2023-11-06T03:04:08.313347+0000","last_became_active":"2023-11-06T03:04:06.721514+0000","last_became_peered":"2023-11-06T03:04:06.721514+0000","last_unstale":"2023-11-06T03:04:08.313347+0000","last_undegraded":"2023-11-06T03:04:08.313347+0000","last_fullsized":"2023-11-06T03:04:08.313347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697317+0000","last_change":"2023-11-06T03:04:06.725709+0000","last_active":"2023-11-06T03:04:07.697317+0000","last_peered":"2023-11-06T03:04:07.697317+0000","last_clean":"2023-11-06T03:04:07.697317+0000","last_became_active":"2023-11-06T03:04:06.725248+0000","last_became_peered":"2023-11-06T03:04:06.725248+0000","last_unstale":"2023-11-06T03:04:07.697317+0000","last_undegraded":"2023-11-06T03:04:07.697317+0000","last_fullsized":"2023-11-06T03:04:07.697317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812176+0000","last_change":"2023-11-06T03:04:06.363773+0000","last_active":"2023-11-06T03:04:07.812176+0000","last_peered":"2023-11-06T03:04:07.812176+0000","last_clean":"2023-11-06T03:04:07.812176+0000","last_became_active":"2023-11-06T03:04:06.362996+0000","last_became_peered":"2023-11-06T03:04:06.362996+0000","last_unstale":"2023-11-06T03:04:07.812176+0000","last_undegraded":"2023-11-06T03:04:07.812176+0000","last_fullsized":"2023-11-06T03:04:07.812176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314708+0000","last_change":"2023-11-06T03:04:06.709102+0000","last_active":"2023-11-06T03:04:08.314708+0000","last_peered":"2023-11-06T03:04:08.314708+0000","last_clean":"2023-11-06T03:04:08.314708+0000","last_became_active":"2023-11-06T03:04:06.708817+0000","last_became_peered":"2023-11-06T03:04:06.708817+0000","last_unstale":"2023-11-06T03:04:08.314708+0000","last_undegraded":"2023-11-06T03:04:08.314708+0000","last_fullsized":"2023-11-06T03:04:08.314708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315524+0000","last_change":"2023-11-06T03:04:06.351736+0000","last_active":"2023-11-06T03:04:08.315524+0000","last_peered":"2023-11-06T03:04:08.315524+0000","last_clean":"2023-11-06T03:04:08.315524+0000","last_became_active":"2023-11-06T03:04:06.351500+0000","last_became_peered":"2023-11-06T03:04:06.351500+0000","last_unstale":"2023-11-06T03:04:08.315524+0000","last_undegraded":"2023-11-06T03:04:08.315524+0000","last_fullsized":"2023-11-06T03:04:08.315524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316022+0000","last_change":"2023-11-06T03:04:06.719843+0000","last_active":"2023-11-06T03:04:08.316022+0000","last_peered":"2023-11-06T03:04:08.316022+0000","last_clean":"2023-11-06T03:04:08.316022+0000","last_became_active":"2023-11-06T03:04:06.718738+0000","last_became_peered":"2023-11-06T03:04:06.718738+0000","last_unstale":"2023-11-06T03:04:08.316022+0000","last_undegraded":"2023-11-06T03:04:08.316022+0000","last_fullsized":"2023-11-06T03:04:08.316022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812188+0000","last_change":"2023-11-06T03:04:06.364044+0000","last_active":"2023-11-06T03:04:07.812188+0000","last_peered":"2023-11-06T03:04:07.812188+0000","last_clean":"2023-11-06T03:04:07.812188+0000","last_became_active":"2023-11-06T03:04:06.363453+0000","last_became_peered":"2023-11-06T03:04:06.363453+0000","last_unstale":"2023-11-06T03:04:07.812188+0000","last_undegraded":"2023-11-06T03:04:07.812188+0000","last_fullsized":"2023-11-06T03:04:07.812188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697122+0000","last_change":"2023-11-06T03:04:06.732024+0000","last_active":"2023-11-06T03:04:07.697122+0000","last_peered":"2023-11-06T03:04:07.697122+0000","last_clean":"2023-11-06T03:04:07.697122+0000","last_became_active":"2023-11-06T03:04:06.730677+0000","last_became_peered":"2023-11-06T03:04:06.730677+0000","last_unstale":"2023-11-06T03:04:07.697122+0000","last_undegraded":"2023-11-06T03:04:07.697122+0000","last_fullsized":"2023-11-06T03:04:07.697122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313367+0000","last_change":"2023-11-06T03:04:06.333322+0000","last_active":"2023-11-06T03:04:08.313367+0000","last_peered":"2023-11-06T03:04:08.313367+0000","last_clean":"2023-11-06T03:04:08.313367+0000","last_became_active":"2023-11-06T03:04:06.332581+0000","last_became_peered":"2023-11-06T03:04:06.332581+0000","last_unstale":"2023-11-06T03:04:08.313367+0000","last_undegraded":"2023-11-06T03:04:08.313367+0000","last_fullsized":"2023-11-06T03:04:08.313367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812463+0000","last_change":"2023-11-06T03:04:06.363547+0000","last_active":"2023-11-06T03:04:07.812463+0000","last_peered":"2023-11-06T03:04:07.812463+0000","last_clean":"2023-11-06T03:04:07.812463+0000","last_became_active":"2023-11-06T03:04:06.362774+0000","last_became_peered":"2023-11-06T03:04:06.362774+0000","last_unstale":"2023-11-06T03:04:07.812463+0000","last_undegraded":"2023-11-06T03:04:07.812463+0000","last_fullsized":"2023-11-06T03:04:07.812463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313617+0000","last_change":"2023-11-06T03:04:06.715341+0000","last_active":"2023-11-06T03:04:08.313617+0000","last_peered":"2023-11-06T03:04:08.313617+0000","last_clean":"2023-11-06T03:04:08.313617+0000","last_became_active":"2023-11-06T03:04:06.715020+0000","last_became_peered":"2023-11-06T03:04:06.715020+0000","last_unstale":"2023-11-06T03:04:08.313617+0000","last_undegraded":"2023-11-06T03:04:08.313617+0000","last_fullsized":"2023-11-06T03:04:08.313617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812298+0000","last_change":"2023-11-06T03:04:06.364333+0000","last_active":"2023-11-06T03:04:07.812298+0000","last_peered":"2023-11-06T03:04:07.812298+0000","last_clean":"2023-11-06T03:04:07.812298+0000","last_became_active":"2023-11-06T03:04:06.363641+0000","last_became_peered":"2023-11-06T03:04:06.363641+0000","last_unstale":"2023-11-06T03:04:07.812298+0000","last_undegraded":"2023-11-06T03:04:07.812298+0000","last_fullsized":"2023-11-06T03:04:07.812298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:29.165293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697141+0000","last_change":"2023-11-06T03:04:06.712796+0000","last_active":"2023-11-06T03:04:07.697141+0000","last_peered":"2023-11-06T03:04:07.697141+0000","last_clean":"2023-11-06T03:04:07.697141+0000","last_became_active":"2023-11-06T03:04:06.712442+0000","last_became_peered":"2023-11-06T03:04:06.712442+0000","last_unstale":"2023-11-06T03:04:07.697141+0000","last_undegraded":"2023-11-06T03:04:07.697141+0000","last_fullsized":"2023-11-06T03:04:07.697141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812223+0000","last_change":"2023-11-06T03:04:06.352109+0000","last_active":"2023-11-06T03:04:07.812223+0000","last_peered":"2023-11-06T03:04:07.812223+0000","last_clean":"2023-11-06T03:04:07.812223+0000","last_became_active":"2023-11-06T03:04:06.351112+0000","last_became_peered":"2023-11-06T03:04:06.351112+0000","last_unstale":"2023-11-06T03:04:07.812223+0000","last_undegraded":"2023-11-06T03:04:07.812223+0000","last_fullsized":"2023-11-06T03:04:07.812223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:27.571905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313304+0000","last_change":"2023-11-06T03:04:06.721884+0000","last_active":"2023-11-06T03:04:08.313304+0000","last_peered":"2023-11-06T03:04:08.313304+0000","last_clean":"2023-11-06T03:04:08.313304+0000","last_became_active":"2023-11-06T03:04:06.721714+0000","last_became_peered":"2023-11-06T03:04:06.721714+0000","last_unstale":"2023-11-06T03:04:08.313304+0000","last_undegraded":"2023-11-06T03:04:08.313304+0000","last_fullsized":"2023-11-06T03:04:08.313304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:04.132219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313920+0000","last_change":"2023-11-06T03:04:06.336342+0000","last_active":"2023-11-06T03:04:08.313920+0000","last_peered":"2023-11-06T03:04:08.313920+0000","last_clean":"2023-11-06T03:04:08.313920+0000","last_became_active":"2023-11-06T03:04:06.335639+0000","last_became_peered":"2023-11-06T03:04:06.335639+0000","last_unstale":"2023-11-06T03:04:08.313920+0000","last_undegraded":"2023-11-06T03:04:08.313920+0000","last_fullsized":"2023-11-06T03:04:08.313920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:15.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315124+0000","last_change":"2023-11-06T03:04:06.709502+0000","last_active":"2023-11-06T03:04:08.315124+0000","last_peered":"2023-11-06T03:04:08.315124+0000","last_clean":"2023-11-06T03:04:08.315124+0000","last_became_active":"2023-11-06T03:04:06.709157+0000","last_became_peered":"2023-11-06T03:04:06.709157+0000","last_unstale":"2023-11-06T03:04:08.315124+0000","last_undegraded":"2023-11-06T03:04:08.315124+0000","last_fullsized":"2023-11-06T03:04:08.315124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697206+0000","last_change":"2023-11-06T03:04:06.331517+0000","last_active":"2023-11-06T03:04:07.697206+0000","last_peered":"2023-11-06T03:04:07.697206+0000","last_clean":"2023-11-06T03:04:07.697206+0000","last_became_active":"2023-11-06T03:04:06.331107+0000","last_became_peered":"2023-11-06T03:04:06.331107+0000","last_unstale":"2023-11-06T03:04:07.697206+0000","last_undegraded":"2023-11-06T03:04:07.697206+0000","last_fullsized":"2023-11-06T03:04:07.697206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697752+0000","last_change":"2023-11-06T03:04:06.720549+0000","last_active":"2023-11-06T03:04:07.697752+0000","last_peered":"2023-11-06T03:04:07.697752+0000","last_clean":"2023-11-06T03:04:07.697752+0000","last_became_active":"2023-11-06T03:04:06.719939+0000","last_became_peered":"2023-11-06T03:04:06.719939+0000","last_unstale":"2023-11-06T03:04:07.697752+0000","last_undegraded":"2023-11-06T03:04:07.697752+0000","last_fullsized":"2023-11-06T03:04:07.697752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:34.877473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313501+0000","last_change":"2023-11-06T03:04:06.364318+0000","last_active":"2023-11-06T03:04:08.313501+0000","last_peered":"2023-11-06T03:04:08.313501+0000","last_clean":"2023-11-06T03:04:08.313501+0000","last_became_active":"2023-11-06T03:04:06.363095+0000","last_became_peered":"2023-11-06T03:04:06.363095+0000","last_unstale":"2023-11-06T03:04:08.313501+0000","last_undegraded":"2023-11-06T03:04:08.313501+0000","last_fullsized":"2023-11-06T03:04:08.313501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698194+0000","last_change":"2023-11-06T03:04:06.707941+0000","last_active":"2023-11-06T03:04:07.698194+0000","last_peered":"2023-11-06T03:04:07.698194+0000","last_clean":"2023-11-06T03:04:07.698194+0000","last_became_active":"2023-11-06T03:04:06.707628+0000","last_became_peered":"2023-11-06T03:04:06.707628+0000","last_unstale":"2023-11-06T03:04:07.698194+0000","last_undegraded":"2023-11-06T03:04:07.698194+0000","last_fullsized":"2023-11-06T03:04:07.698194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:55.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697420+0000","last_change":"2023-11-06T03:04:06.355002+0000","last_active":"2023-11-06T03:04:07.697420+0000","last_peered":"2023-11-06T03:04:07.697420+0000","last_clean":"2023-11-06T03:04:07.697420+0000","last_became_active":"2023-11-06T03:04:06.352902+0000","last_became_peered":"2023-11-06T03:04:06.352902+0000","last_unstale":"2023-11-06T03:04:07.697420+0000","last_undegraded":"2023-11-06T03:04:07.697420+0000","last_fullsized":"2023-11-06T03:04:07.697420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697853+0000","last_change":"2023-11-06T03:04:06.721894+0000","last_active":"2023-11-06T03:04:07.697853+0000","last_peered":"2023-11-06T03:04:07.697853+0000","last_clean":"2023-11-06T03:04:07.697853+0000","last_became_active":"2023-11-06T03:04:06.721619+0000","last_became_peered":"2023-11-06T03:04:06.721619+0000","last_unstale":"2023-11-06T03:04:07.697853+0000","last_undegraded":"2023-11-06T03:04:07.697853+0000","last_fullsized":"2023-11-06T03:04:07.697853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696727+0000","last_change":"2023-11-06T03:04:06.330685+0000","last_active":"2023-11-06T03:04:07.696727+0000","last_peered":"2023-11-06T03:04:07.696727+0000","last_clean":"2023-11-06T03:04:07.696727+0000","last_became_active":"2023-11-06T03:04:06.330390+0000","last_became_peered":"2023-11-06T03:04:06.330390+0000","last_unstale":"2023-11-06T03:04:07.696727+0000","last_undegraded":"2023-11-06T03:04:07.696727+0000","last_fullsized":"2023-11-06T03:04:07.696727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317295+0000","last_change":"2023-11-06T03:04:06.721960+0000","last_active":"2023-11-06T03:04:08.317295+0000","last_peered":"2023-11-06T03:04:08.317295+0000","last_clean":"2023-11-06T03:04:08.317295+0000","last_became_active":"2023-11-06T03:04:06.721497+0000","last_became_peered":"2023-11-06T03:04:06.721497+0000","last_unstale":"2023-11-06T03:04:08.317295+0000","last_undegraded":"2023-11-06T03:04:08.317295+0000","last_fullsized":"2023-11-06T03:04:08.317295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:28.582669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700445+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:06.700445+0000","last_peered":"2023-11-06T03:04:06.700445+0000","last_clean":"2023-11-06T03:04:06.700445+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:06.700445+0000","last_undegraded":"2023-11-06T03:04:06.700445+0000","last_fullsized":"2023-11-06T03:04:06.700445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700232+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:06.700232+0000","last_peered":"2023-11-06T03:04:06.700232+0000","last_clean":"2023-11-06T03:04:06.700232+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:06.700232+0000","last_undegraded":"2023-11-06T03:04:06.700232+0000","last_fullsized":"2023-11-06T03:04:06.700232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696999+0000","last_change":"2023-11-06T03:04:06.344186+0000","last_active":"2023-11-06T03:04:07.696999+0000","last_peered":"2023-11-06T03:04:07.696999+0000","last_clean":"2023-11-06T03:04:07.696999+0000","last_became_active":"2023-11-06T03:04:06.343854+0000","last_became_peered":"2023-11-06T03:04:06.343854+0000","last_unstale":"2023-11-06T03:04:07.696999+0000","last_undegraded":"2023-11-06T03:04:07.696999+0000","last_fullsized":"2023-11-06T03:04:07.696999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315060+0000","last_change":"2023-11-06T03:04:06.716182+0000","last_active":"2023-11-06T03:04:08.315060+0000","last_peered":"2023-11-06T03:04:08.315060+0000","last_clean":"2023-11-06T03:04:08.315060+0000","last_became_active":"2023-11-06T03:04:06.714829+0000","last_became_peered":"2023-11-06T03:04:06.714829+0000","last_unstale":"2023-11-06T03:04:08.315060+0000","last_undegraded":"2023-11-06T03:04:08.315060+0000","last_fullsized":"2023-11-06T03:04:08.315060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:20.005858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700210+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:06.700210+0000","last_peered":"2023-11-06T03:04:06.700210+0000","last_clean":"2023-11-06T03:04:06.700210+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:06.700210+0000","last_undegraded":"2023-11-06T03:04:06.700210+0000","last_fullsized":"2023-11-06T03:04:06.700210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314962+0000","last_change":"2023-11-06T03:04:06.359178+0000","last_active":"2023-11-06T03:04:08.314962+0000","last_peered":"2023-11-06T03:04:08.314962+0000","last_clean":"2023-11-06T03:04:08.314962+0000","last_became_active":"2023-11-06T03:04:06.358881+0000","last_became_peered":"2023-11-06T03:04:06.358881+0000","last_unstale":"2023-11-06T03:04:08.314962+0000","last_undegraded":"2023-11-06T03:04:08.314962+0000","last_fullsized":"2023-11-06T03:04:08.314962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316077+0000","last_change":"2023-11-06T03:04:06.712984+0000","last_active":"2023-11-06T03:04:08.316077+0000","last_peered":"2023-11-06T03:04:08.316077+0000","last_clean":"2023-11-06T03:04:08.316077+0000","last_became_active":"2023-11-06T03:04:06.712663+0000","last_became_peered":"2023-11-06T03:04:06.712663+0000","last_unstale":"2023-11-06T03:04:08.316077+0000","last_undegraded":"2023-11-06T03:04:08.316077+0000","last_fullsized":"2023-11-06T03:04:08.316077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.688745+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:06.688745+0000","last_peered":"2023-11-06T03:04:06.688745+0000","last_clean":"2023-11-06T03:04:06.688745+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:06.688745+0000","last_undegraded":"2023-11-06T03:04:06.688745+0000","last_fullsized":"2023-11-06T03:04:06.688745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314913+0000","last_change":"2023-11-06T03:04:06.328651+0000","last_active":"2023-11-06T03:04:08.314913+0000","last_peered":"2023-11-06T03:04:08.314913+0000","last_clean":"2023-11-06T03:04:08.314913+0000","last_became_active":"2023-11-06T03:04:06.328314+0000","last_became_peered":"2023-11-06T03:04:06.328314+0000","last_unstale":"2023-11-06T03:04:08.314913+0000","last_undegraded":"2023-11-06T03:04:08.314913+0000","last_fullsized":"2023-11-06T03:04:08.314913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317506+0000","last_change":"2023-11-06T03:04:06.710848+0000","last_active":"2023-11-06T03:04:08.317506+0000","last_peered":"2023-11-06T03:04:08.317506+0000","last_clean":"2023-11-06T03:04:08.317506+0000","last_became_active":"2023-11-06T03:04:06.709767+0000","last_became_peered":"2023-11-06T03:04:06.709767+0000","last_unstale":"2023-11-06T03:04:08.317506+0000","last_undegraded":"2023-11-06T03:04:08.317506+0000","last_fullsized":"2023-11-06T03:04:08.317506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700275+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:06.700275+0000","last_peered":"2023-11-06T03:04:06.700275+0000","last_clean":"2023-11-06T03:04:06.700275+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:06.700275+0000","last_undegraded":"2023-11-06T03:04:06.700275+0000","last_fullsized":"2023-11-06T03:04:06.700275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697593+0000","last_change":"2023-11-06T03:04:06.354390+0000","last_active":"2023-11-06T03:04:07.697593+0000","last_peered":"2023-11-06T03:04:07.697593+0000","last_clean":"2023-11-06T03:04:07.697593+0000","last_became_active":"2023-11-06T03:04:06.353638+0000","last_became_peered":"2023-11-06T03:04:06.353638+0000","last_unstale":"2023-11-06T03:04:07.697593+0000","last_undegraded":"2023-11-06T03:04:07.697593+0000","last_fullsized":"2023-11-06T03:04:07.697593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316563+0000","last_change":"2023-11-06T03:04:06.714255+0000","last_active":"2023-11-06T03:04:08.316563+0000","last_peered":"2023-11-06T03:04:08.316563+0000","last_clean":"2023-11-06T03:04:08.316563+0000","last_became_active":"2023-11-06T03:04:06.709791+0000","last_became_peered":"2023-11-06T03:04:06.709791+0000","last_unstale":"2023-11-06T03:04:08.316563+0000","last_undegraded":"2023-11-06T03:04:08.316563+0000","last_fullsized":"2023-11-06T03:04:08.316563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314303+0000","last_change":"2023-11-06T03:04:06.338556+0000","last_active":"2023-11-06T03:04:08.314303+0000","last_peered":"2023-11-06T03:04:08.314303+0000","last_clean":"2023-11-06T03:04:08.314303+0000","last_became_active":"2023-11-06T03:04:06.337740+0000","last_became_peered":"2023-11-06T03:04:06.337740+0000","last_unstale":"2023-11-06T03:04:08.314303+0000","last_undegraded":"2023-11-06T03:04:08.314303+0000","last_fullsized":"2023-11-06T03:04:08.314303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.701953+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:06.701953+0000","last_peered":"2023-11-06T03:04:06.701953+0000","last_clean":"2023-11-06T03:04:06.701953+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:06.701953+0000","last_undegraded":"2023-11-06T03:04:06.701953+0000","last_fullsized":"2023-11-06T03:04:06.701953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316863+0000","last_change":"2023-11-06T03:04:06.711498+0000","last_active":"2023-11-06T03:04:08.316863+0000","last_peered":"2023-11-06T03:04:08.316863+0000","last_clean":"2023-11-06T03:04:08.316863+0000","last_became_active":"2023-11-06T03:04:06.708625+0000","last_became_peered":"2023-11-06T03:04:06.708625+0000","last_unstale":"2023-11-06T03:04:08.316863+0000","last_undegraded":"2023-11-06T03:04:08.316863+0000","last_fullsized":"2023-11-06T03:04:08.316863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316721+0000","last_change":"2023-11-06T03:04:06.329676+0000","last_active":"2023-11-06T03:04:08.316721+0000","last_peered":"2023-11-06T03:04:08.316721+0000","last_clean":"2023-11-06T03:04:08.316721+0000","last_became_active":"2023-11-06T03:04:06.329333+0000","last_became_peered":"2023-11-06T03:04:06.329333+0000","last_unstale":"2023-11-06T03:04:08.316721+0000","last_undegraded":"2023-11-06T03:04:08.316721+0000","last_fullsized":"2023-11-06T03:04:08.316721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.703855+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:06.703855+0000","last_peered":"2023-11-06T03:04:06.703855+0000","last_clean":"2023-11-06T03:04:06.703855+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:06.703855+0000","last_undegraded":"2023-11-06T03:04:06.703855+0000","last_fullsized":"2023-11-06T03:04:06.703855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315277+0000","last_change":"2023-11-06T03:04:06.701264+0000","last_active":"2023-11-06T03:04:08.315277+0000","last_peered":"2023-11-06T03:04:08.315277+0000","last_clean":"2023-11-06T03:04:08.315277+0000","last_became_active":"2023-11-06T03:04:06.700955+0000","last_became_peered":"2023-11-06T03:04:06.700955+0000","last_unstale":"2023-11-06T03:04:08.315277+0000","last_undegraded":"2023-11-06T03:04:08.315277+0000","last_fullsized":"2023-11-06T03:04:08.315277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697348+0000","last_change":"2023-11-06T03:04:06.346267+0000","last_active":"2023-11-06T03:04:07.697348+0000","last_peered":"2023-11-06T03:04:07.697348+0000","last_clean":"2023-11-06T03:04:07.697348+0000","last_became_active":"2023-11-06T03:04:06.345991+0000","last_became_peered":"2023-11-06T03:04:06.345991+0000","last_unstale":"2023-11-06T03:04:07.697348+0000","last_undegraded":"2023-11-06T03:04:07.697348+0000","last_fullsized":"2023-11-06T03:04:07.697348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.702101+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:06.702101+0000","last_peered":"2023-11-06T03:04:06.702101+0000","last_clean":"2023-11-06T03:04:06.702101+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:06.702101+0000","last_undegraded":"2023-11-06T03:04:06.702101+0000","last_fullsized":"2023-11-06T03:04:06.702101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696622+0000","last_change":"2023-11-06T03:04:06.731112+0000","last_active":"2023-11-06T03:04:07.696622+0000","last_peered":"2023-11-06T03:04:07.696622+0000","last_clean":"2023-11-06T03:04:07.696622+0000","last_became_active":"2023-11-06T03:04:06.730152+0000","last_became_peered":"2023-11-06T03:04:06.730152+0000","last_unstale":"2023-11-06T03:04:07.696622+0000","last_undegraded":"2023-11-06T03:04:07.696622+0000","last_fullsized":"2023-11-06T03:04:07.696622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316243+0000","last_change":"2023-11-06T03:04:06.331522+0000","last_active":"2023-11-06T03:04:08.316243+0000","last_peered":"2023-11-06T03:04:08.316243+0000","last_clean":"2023-11-06T03:04:08.316243+0000","last_became_active":"2023-11-06T03:04:06.331263+0000","last_became_peered":"2023-11-06T03:04:06.331263+0000","last_unstale":"2023-11-06T03:04:08.316243+0000","last_undegraded":"2023-11-06T03:04:08.316243+0000","last_fullsized":"2023-11-06T03:04:08.316243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.725653+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:06.725653+0000","last_peered":"2023-11-06T03:04:06.725653+0000","last_clean":"2023-11-06T03:04:06.725653+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:06.725653+0000","last_undegraded":"2023-11-06T03:04:06.725653+0000","last_fullsized":"2023-11-06T03:04:06.725653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698182+0000","last_change":"2023-11-06T03:04:06.705426+0000","last_active":"2023-11-06T03:04:07.698182+0000","last_peered":"2023-11-06T03:04:07.698182+0000","last_clean":"2023-11-06T03:04:07.698182+0000","last_became_active":"2023-11-06T03:04:06.705116+0000","last_became_peered":"2023-11-06T03:04:06.705116+0000","last_unstale":"2023-11-06T03:04:07.698182+0000","last_undegraded":"2023-11-06T03:04:07.698182+0000","last_fullsized":"2023-11-06T03:04:07.698182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:26.686747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697675+0000","last_change":"2023-11-06T03:04:06.721464+0000","last_active":"2023-11-06T03:04:07.697675+0000","last_peered":"2023-11-06T03:04:07.697675+0000","last_clean":"2023-11-06T03:04:07.697675+0000","last_became_active":"2023-11-06T03:04:06.721193+0000","last_became_peered":"2023-11-06T03:04:06.721193+0000","last_unstale":"2023-11-06T03:04:07.697675+0000","last_undegraded":"2023-11-06T03:04:07.697675+0000","last_fullsized":"2023-11-06T03:04:07.697675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697996+0000","last_change":"2023-11-06T03:04:06.354926+0000","last_active":"2023-11-06T03:04:07.697996+0000","last_peered":"2023-11-06T03:04:07.697996+0000","last_clean":"2023-11-06T03:04:07.697996+0000","last_became_active":"2023-11-06T03:04:06.354287+0000","last_became_peered":"2023-11-06T03:04:06.354287+0000","last_unstale":"2023-11-06T03:04:07.697996+0000","last_undegraded":"2023-11-06T03:04:07.697996+0000","last_fullsized":"2023-11-06T03:04:07.697996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316609+0000","last_change":"2023-11-06T03:04:06.716076+0000","last_active":"2023-11-06T03:04:08.316609+0000","last_peered":"2023-11-06T03:04:08.316609+0000","last_clean":"2023-11-06T03:04:08.316609+0000","last_became_active":"2023-11-06T03:04:06.715726+0000","last_became_peered":"2023-11-06T03:04:06.715726+0000","last_unstale":"2023-11-06T03:04:08.316609+0000","last_undegraded":"2023-11-06T03:04:08.316609+0000","last_fullsized":"2023-11-06T03:04:08.316609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812115+0000","last_change":"2023-11-06T03:04:06.346560+0000","last_active":"2023-11-06T03:04:07.812115+0000","last_peered":"2023-11-06T03:04:07.812115+0000","last_clean":"2023-11-06T03:04:07.812115+0000","last_became_active":"2023-11-06T03:04:06.345767+0000","last_became_peered":"2023-11-06T03:04:06.345767+0000","last_unstale":"2023-11-06T03:04:07.812115+0000","last_undegraded":"2023-11-06T03:04:07.812115+0000","last_fullsized":"2023-11-06T03:04:07.812115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:18.995094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314192+0000","last_change":"2023-11-06T03:04:06.346611+0000","last_active":"2023-11-06T03:04:08.314192+0000","last_peered":"2023-11-06T03:04:08.314192+0000","last_clean":"2023-11-06T03:04:08.314192+0000","last_became_active":"2023-11-06T03:04:06.345852+0000","last_became_peered":"2023-11-06T03:04:06.345852+0000","last_unstale":"2023-11-06T03:04:08.314192+0000","last_undegraded":"2023-11-06T03:04:08.314192+0000","last_fullsized":"2023-11-06T03:04:08.314192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696805+0000","last_change":"2023-11-06T03:04:06.727615+0000","last_active":"2023-11-06T03:04:07.696805+0000","last_peered":"2023-11-06T03:04:07.696805+0000","last_clean":"2023-11-06T03:04:07.696805+0000","last_became_active":"2023-11-06T03:04:06.726871+0000","last_became_peered":"2023-11-06T03:04:06.726871+0000","last_unstale":"2023-11-06T03:04:07.696805+0000","last_undegraded":"2023-11-06T03:04:07.696805+0000","last_fullsized":"2023-11-06T03:04:07.696805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315989+0000","last_change":"2023-11-06T03:04:06.344230+0000","last_active":"2023-11-06T03:04:08.315989+0000","last_peered":"2023-11-06T03:04:08.315989+0000","last_clean":"2023-11-06T03:04:08.315989+0000","last_became_active":"2023-11-06T03:04:06.343852+0000","last_became_peered":"2023-11-06T03:04:06.343852+0000","last_unstale":"2023-11-06T03:04:08.315989+0000","last_undegraded":"2023-11-06T03:04:08.315989+0000","last_fullsized":"2023-11-06T03:04:08.315989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696536+0000","last_change":"2023-11-06T03:04:06.729233+0000","last_active":"2023-11-06T03:04:07.696536+0000","last_peered":"2023-11-06T03:04:07.696536+0000","last_clean":"2023-11-06T03:04:07.696536+0000","last_became_active":"2023-11-06T03:04:06.728828+0000","last_became_peered":"2023-11-06T03:04:06.728828+0000","last_unstale":"2023-11-06T03:04:07.696536+0000","last_undegraded":"2023-11-06T03:04:07.696536+0000","last_fullsized":"2023-11-06T03:04:07.696536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315468+0000","last_change":"2023-11-06T03:04:06.721167+0000","last_active":"2023-11-06T03:04:08.315468+0000","last_peered":"2023-11-06T03:04:08.315468+0000","last_clean":"2023-11-06T03:04:08.315468+0000","last_became_active":"2023-11-06T03:04:06.720914+0000","last_became_peered":"2023-11-06T03:04:06.720914+0000","last_unstale":"2023-11-06T03:04:08.315468+0000","last_undegraded":"2023-11-06T03:04:08.315468+0000","last_fullsized":"2023-11-06T03:04:08.315468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:55.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696763+0000","last_change":"2023-11-06T03:04:06.331738+0000","last_active":"2023-11-06T03:04:07.696763+0000","last_peered":"2023-11-06T03:04:07.696763+0000","last_clean":"2023-11-06T03:04:07.696763+0000","last_became_active":"2023-11-06T03:04:06.330746+0000","last_became_peered":"2023-11-06T03:04:06.330746+0000","last_unstale":"2023-11-06T03:04:07.696763+0000","last_undegraded":"2023-11-06T03:04:07.696763+0000","last_fullsized":"2023-11-06T03:04:07.696763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315727+0000","last_change":"2023-11-06T03:04:06.700157+0000","last_active":"2023-11-06T03:04:08.315727+0000","last_peered":"2023-11-06T03:04:08.315727+0000","last_clean":"2023-11-06T03:04:08.315727+0000","last_became_active":"2023-11-06T03:04:06.699680+0000","last_became_peered":"2023-11-06T03:04:06.699680+0000","last_unstale":"2023-11-06T03:04:08.315727+0000","last_undegraded":"2023-11-06T03:04:08.315727+0000","last_fullsized":"2023-11-06T03:04:08.315727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316079+0000","last_change":"2023-11-06T03:04:06.330647+0000","last_active":"2023-11-06T03:04:08.316079+0000","last_peered":"2023-11-06T03:04:08.316079+0000","last_clean":"2023-11-06T03:04:08.316079+0000","last_became_active":"2023-11-06T03:04:06.330043+0000","last_became_peered":"2023-11-06T03:04:06.330043+0000","last_unstale":"2023-11-06T03:04:08.316079+0000","last_undegraded":"2023-11-06T03:04:08.316079+0000","last_fullsized":"2023-11-06T03:04:08.316079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.724159+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.723798+0000","last_became_peered":"2023-11-06T03:04:06.723798+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.341434+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.340687+0000","last_became_peered":"2023-11-06T03:04:06.340687+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:44.206262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315009+0000","last_change":"2023-11-06T03:04:06.719856+0000","last_active":"2023-11-06T03:04:08.315009+0000","last_peered":"2023-11-06T03:04:08.315009+0000","last_clean":"2023-11-06T03:04:08.315009+0000","last_became_active":"2023-11-06T03:04:06.718775+0000","last_became_peered":"2023-11-06T03:04:06.718775+0000","last_unstale":"2023-11-06T03:04:08.315009+0000","last_undegraded":"2023-11-06T03:04:08.315009+0000","last_fullsized":"2023-11-06T03:04:08.315009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:21.577004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315849+0000","last_change":"2023-11-06T03:04:06.359042+0000","last_active":"2023-11-06T03:04:08.315849+0000","last_peered":"2023-11-06T03:04:08.315849+0000","last_clean":"2023-11-06T03:04:08.315849+0000","last_became_active":"2023-11-06T03:04:06.358768+0000","last_became_peered":"2023-11-06T03:04:06.358768+0000","last_unstale":"2023-11-06T03:04:08.315849+0000","last_undegraded":"2023-11-06T03:04:08.315849+0000","last_fullsized":"2023-11-06T03:04:08.315849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697472+0000","last_change":"2023-11-06T03:04:06.723498+0000","last_active":"2023-11-06T03:04:07.697472+0000","last_peered":"2023-11-06T03:04:07.697472+0000","last_clean":"2023-11-06T03:04:07.697472+0000","last_became_active":"2023-11-06T03:04:06.723271+0000","last_became_peered":"2023-11-06T03:04:06.723271+0000","last_unstale":"2023-11-06T03:04:07.697472+0000","last_undegraded":"2023-11-06T03:04:07.697472+0000","last_fullsized":"2023-11-06T03:04:07.697472+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:26.972193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313662+0000","last_change":"2023-11-06T03:04:06.359575+0000","last_active":"2023-11-06T03:04:08.313662+0000","last_peered":"2023-11-06T03:04:08.313662+0000","last_clean":"2023-11-06T03:04:08.313662+0000","last_became_active":"2023-11-06T03:04:06.358835+0000","last_became_peered":"2023-11-06T03:04:06.358835+0000","last_unstale":"2023-11-06T03:04:08.313662+0000","last_undegraded":"2023-11-06T03:04:08.313662+0000","last_fullsized":"2023-11-06T03:04:08.313662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697217+0000","last_change":"2023-11-06T03:04:06.712260+0000","last_active":"2023-11-06T03:04:07.697217+0000","last_peered":"2023-11-06T03:04:07.697217+0000","last_clean":"2023-11-06T03:04:07.697217+0000","last_became_active":"2023-11-06T03:04:06.711945+0000","last_became_peered":"2023-11-06T03:04:06.711945+0000","last_unstale":"2023-11-06T03:04:07.697217+0000","last_undegraded":"2023-11-06T03:04:07.697217+0000","last_fullsized":"2023-11-06T03:04:07.697217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:06.799825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314081+0000","last_change":"2023-11-06T03:04:06.350220+0000","last_active":"2023-11-06T03:04:08.314081+0000","last_peered":"2023-11-06T03:04:08.314081+0000","last_clean":"2023-11-06T03:04:08.314081+0000","last_became_active":"2023-11-06T03:04:06.349458+0000","last_became_peered":"2023-11-06T03:04:06.349458+0000","last_unstale":"2023-11-06T03:04:08.314081+0000","last_undegraded":"2023-11-06T03:04:08.314081+0000","last_fullsized":"2023-11-06T03:04:08.314081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697048+0000","last_change":"2023-11-06T03:04:06.710979+0000","last_active":"2023-11-06T03:04:07.697048+0000","last_peered":"2023-11-06T03:04:07.697048+0000","last_clean":"2023-11-06T03:04:07.697048+0000","last_became_active":"2023-11-06T03:04:06.709663+0000","last_became_peered":"2023-11-06T03:04:06.709663+0000","last_unstale":"2023-11-06T03:04:07.697048+0000","last_undegraded":"2023-11-06T03:04:07.697048+0000","last_fullsized":"2023-11-06T03:04:07.697048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697612+0000","last_change":"2023-11-06T03:04:06.333374+0000","last_active":"2023-11-06T03:04:07.697612+0000","last_peered":"2023-11-06T03:04:07.697612+0000","last_clean":"2023-11-06T03:04:07.697612+0000","last_became_active":"2023-11-06T03:04:06.332910+0000","last_became_peered":"2023-11-06T03:04:06.332910+0000","last_unstale":"2023-11-06T03:04:07.697612+0000","last_undegraded":"2023-11-06T03:04:07.697612+0000","last_fullsized":"2023-11-06T03:04:07.697612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697342+0000","last_change":"2023-11-06T03:04:06.722605+0000","last_active":"2023-11-06T03:04:07.697342+0000","last_peered":"2023-11-06T03:04:07.697342+0000","last_clean":"2023-11-06T03:04:07.697342+0000","last_became_active":"2023-11-06T03:04:06.722317+0000","last_became_peered":"2023-11-06T03:04:06.722317+0000","last_unstale":"2023-11-06T03:04:07.697342+0000","last_undegraded":"2023-11-06T03:04:07.697342+0000","last_fullsized":"2023-11-06T03:04:07.697342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315853+0000","last_change":"2023-11-06T03:04:06.344293+0000","last_active":"2023-11-06T03:04:08.315853+0000","last_peered":"2023-11-06T03:04:08.315853+0000","last_clean":"2023-11-06T03:04:08.315853+0000","last_became_active":"2023-11-06T03:04:06.344022+0000","last_became_peered":"2023-11-06T03:04:06.344022+0000","last_unstale":"2023-11-06T03:04:08.315853+0000","last_undegraded":"2023-11-06T03:04:08.315853+0000","last_fullsized":"2023-11-06T03:04:08.315853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":237568,"data_stored":88580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":88670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:15.168 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:04:15.168 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-06T03:04:15.169 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-06T03:04:15.169 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-06T03:04:15.170 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-06T03:04:15.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 Processor -- start 2023-11-06T03:04:15.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 -- start start 2023-11-06T03:04:15.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39d80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f39d80d1720 0x7f39d80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d80d1c60 con 0x7f39d80cfbf0 2023-11-06T03:04:15.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80d0080 con 0x7f39d80d1720 2023-11-06T03:04:15.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39dd58e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80d1f40 con 0x7f39d815cdf0 2023-11-06T03:04:15.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f39d80cfbf0 0x7f39d80b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43514/0 (socket says 172.21.15.157:43514) 2023-11-06T03:04:15.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d77fe640 1 -- 172.21.15.157:0/4182568262 learned_addr learned my addr 172.21.15.157:0/4182568262 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:15.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39d80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 --2- 172.21.15.157:0/4182568262 >> v2:172.21.15.195:3300/0 conn(0x7f39d80d1720 0x7f39d80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779304970 0 0) 0x7f39d80d1c60 con 0x7f39d80cfbf0 2023-11-06T03:04:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c0003530 con 0x7f39d80cfbf0 2023-11-06T03:04:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758822102 0 0) 0x7f39c0003530 con 0x7f39d80cfbf0 2023-11-06T03:04:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 msgr2=0x7f39d80d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39d80d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 >> v2:172.21.15.195:3300/0 conn(0x7f39d80d1720 msgr2=0x7f39d80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 --2- 172.21.15.157:0/4182568262 >> v2:172.21.15.195:3300/0 conn(0x7f39d80d1720 0x7f39d80cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80d2220 con 0x7f39d80cfbf0 2023-11-06T03:04:15.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f39d0001a00 con 0x7f39d80cfbf0 2023-11-06T03:04:15.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39d80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:15.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 >> v1:172.21.15.157:6789/0 conn(0x7f39d80cfbf0 legacy=0x7f39d80b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39c0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0004d10 0x7f39c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80d0080 con 0x7f39c0003ec0 2023-11-06T03:04:15.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80d1f40 con 0x7f39c0004d10 2023-11-06T03:04:15.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c000b990 con 0x7f39d815cdf0 2023-11-06T03:04:15.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39c0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d77fe640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0004d10 0x7f39c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 -- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 msgr2=0x7f39c0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d815cdf0 0x7f39c0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 -- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0004d10 msgr2=0x7f39c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0004d10 0x7f39c0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d67fc640 1 -- 172.21.15.157:0/4182568262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009420 con 0x7f39c0003ec0 2023-11-06T03:04:15.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.325+0000 7f39d77fe640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39c0004d10 0x7f39c0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:15.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39d67fc640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39c00047d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009ae0 tx=0x7f39cc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d580b90915e88317 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0a5680 con 0x7f39c0003ec0 2023-11-06T03:04:15.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc0a57e0 con 0x7f39c0003ec0 2023-11-06T03:04:15.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39d5ffb640 1 -- 172.21.15.157:0/4182568262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0a5b90 con 0x7f39c0003ec0 2023-11-06T03:04:15.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- 172.21.15.157:0/4182568262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 msgr2=0x7f39c00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 --2- 172.21.15.157:0/4182568262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39c00047d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009ae0 tx=0x7f39cc01ca90 comp rx=0 tx=0).stop 2023-11-06T03:04:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39d77fe640 1 -- 172.21.15.157:0/4182568262 reap_dead start 2023-11-06T03:04:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- 172.21.15.157:0/4182568262 shutdown_connections 2023-11-06T03:04:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- 172.21.15.157:0/4182568262 >> 172.21.15.157:0/4182568262 conn(0x7f39d8065930 msgr2=0x7f39d8065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- 172.21.15.157:0/4182568262 shutdown_connections 2023-11-06T03:04:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- 172.21.15.157:0/4182568262 wait complete. 2023-11-06T03:04:15.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 Processor -- start 2023-11-06T03:04:15.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 -- start start 2023-11-06T03:04:15.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.329+0000 7f39dd58e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d80b5290 0x7f39d81e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d80b5ff0 0x7f39d80b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80b69c0 con 0x7f39c0003ec0 2023-11-06T03:04:15.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8155b70 con 0x7f39d80b5ff0 2023-11-06T03:04:15.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80cacc0 con 0x7f39d80b5290 2023-11-06T03:04:15.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43690/0 (socket says 172.21.15.157:43690) 2023-11-06T03:04:15.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 -- 172.21.15.157:0/309165249 learned_addr learned my addr 172.21.15.157:0/309165249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:15.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 -- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d80b5290 msgr2=0x7f39d81e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d67fc640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d80b5290 0x7f39d81e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d80b5290 0x7f39d81e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 -- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d80b5ff0 msgr2=0x7f39d80b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d77fe640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d80b5ff0 0x7f39d80b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d80b5ff0 0x7f39d80b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 -- 172.21.15.157:0/309165249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8154f60 con 0x7f39c0003ec0 2023-11-06T03:04:15.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39c8011cd0 tx=0x7f39c800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800fd20 con 0x7f39c0003ec0 2023-11-06T03:04:15.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800fe80 con 0x7f39c0003ec0 2023-11-06T03:04:15.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8004240 con 0x7f39c0003ec0 2023-11-06T03:04:15.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d8155240 con 0x7f39c0003ec0 2023-11-06T03:04:15.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d8155800 con 0x7f39c0003ec0 2023-11-06T03:04:15.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d8072750 con 0x7f39c0003ec0 2023-11-06T03:04:15.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39c80043a0 con 0x7f39c0003ec0 2023-11-06T03:04:15.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39c80839b0 con 0x7f39c0003ec0 2023-11-06T03:04:15.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.333+0000 7f39d67fc640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.337+0000 7f39d67fc640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af000 tx=0x7f39cc0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.337+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39c8051210 con 0x7f39c0003ec0 2023-11-06T03:04:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.529+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f39d80cd2d0 con 0x7f39c0003ec0 2023-11-06T03:04:15.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39bf7fe640 1 -- 172.21.15.157:0/309165249 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163369 (secure 0 0 0) 0x7f39c8022070 con 0x7f39c0003ec0 2023-11-06T03:04:15.540 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 msgr2=0x7f39c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af000 tx=0x7f39cc0046d0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 msgr2=0x7f39d80ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39c8011cd0 tx=0x7f39c800f610 comp rx=0 tx=0).stop 2023-11-06T03:04:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 shutdown_connections 2023-11-06T03:04:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d80b5ff0 0x7f39d80b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d80b5290 0x7f39d81e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 --2- 172.21.15.157:0/309165249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c0003ec0 0x7f39d80ca780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 >> 172.21.15.157:0/309165249 conn(0x7f39d8065930 msgr2=0x7f39d80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 shutdown_connections 2023-11-06T03:04:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.537+0000 7f39dd58e640 1 -- 172.21.15.157:0/309165249 wait complete. 2023-11-06T03:04:15.557 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":38105},{"type":"v1","addr":"172.21.15.195:6833","nonce":38105}]},"active_addr":"172.21.15.195:6833/38105","active_change":"2023-11-06T03:03:47.078035+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":841092446}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1990931703}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2290939155}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":3021732577}]}]} 2023-11-06T03:04:15.561 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-06T03:04:15.562 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-06T03:04:15.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:15.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.673+0000 7fa5e6c86640 1 Processor -- start 2023-11-06T03:04:15.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.673+0000 7fa5e6c86640 1 -- start start 2023-11-06T03:04:15.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e6c86640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5e006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e6c86640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa5e015e5e0 0x7fa5e00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e6c86640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e015f580 con 0x7fa5e015ea60 2023-11-06T03:04:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e6c86640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e015efc0 con 0x7fa5e015e5e0 2023-11-06T03:04:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e6c86640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015f2a0 con 0x7fa5e01551e0 2023-11-06T03:04:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa5e015ea60 0x7fa5e015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43520/0 (socket says 172.21.15.157:43520) 2023-11-06T03:04:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 -- 172.21.15.157:0/458021853 learned_addr learned my addr 172.21.15.157:0/458021853 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5e006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5dffff640 1 --2- 172.21.15.157:0/458021853 >> v2:172.21.15.195:3300/0 conn(0x7fa5e015e5e0 0x7fa5e00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437412554 0 0) 0x7fa5e015f580 con 0x7fa5e015ea60 2023-11-06T03:04:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c8003530 con 0x7fa5e015ea60 2023-11-06T03:04:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1879722986 0 0) 0x7fa5c8003530 con 0x7fa5e015ea60 2023-11-06T03:04:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 msgr2=0x7fa5e006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5e006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 >> v2:172.21.15.195:3300/0 conn(0x7fa5e015e5e0 msgr2=0x7fa5e00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 --2- 172.21.15.157:0/458021853 >> v2:172.21.15.195:3300/0 conn(0x7fa5e015e5e0 0x7fa5e00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e006f5a0 con 0x7fa5e015ea60 2023-11-06T03:04:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa5d8001a00 con 0x7fa5e015ea60 2023-11-06T03:04:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 >> v1:172.21.15.157:6789/0 conn(0x7fa5e015ea60 legacy=0x7fa5e015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8003980 0x7fa5c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c8004d10 0x7fa5c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015efc0 con 0x7fa5c8004d10 2023-11-06T03:04:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015f2a0 con 0x7fa5c8003980 2023-11-06T03:04:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c800b990 con 0x7fa5e01551e0 2023-11-06T03:04:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c8004d10 0x7fa5c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5dffff640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8003980 0x7fa5c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 -- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 msgr2=0x7fa5c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e01551e0 0x7fa5c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 -- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8003980 msgr2=0x7fa5c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8003980 0x7fa5c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e5223640 1 -- 172.21.15.157:0/458021853 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8003530 con 0x7fa5c8004d10 2023-11-06T03:04:15.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.677+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8003980 0x7fa5c8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:15.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c8004d10 0x7fa5c8005190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800a830 tx=0x7fa5d801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f3f0408a76dc94 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80aa6d0 con 0x7fa5c8004d10 2023-11-06T03:04:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d80aa830 con 0x7fa5c8004d10 2023-11-06T03:04:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5df7fe640 1 -- 172.21.15.157:0/458021853 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80aac20 con 0x7fa5c8004d10 2023-11-06T03:04:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/458021853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c8004d10 msgr2=0x7fa5c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/458021853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5c8004d10 0x7fa5c8005190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800a830 tx=0x7fa5d801faa0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 -- 172.21.15.157:0/458021853 reap_dead start 2023-11-06T03:04:15.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/458021853 shutdown_connections 2023-11-06T03:04:15.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/458021853 >> 172.21.15.157:0/458021853 conn(0x7fa5e0065930 msgr2=0x7fa5e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/458021853 shutdown_connections 2023-11-06T03:04:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/458021853 wait complete. 2023-11-06T03:04:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 Processor -- start 2023-11-06T03:04:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- start start 2023-11-06T03:04:15.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8004d10 0x7fa5e006c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e014d360 0x7fa5e014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00ccd70 con 0x7fa5e00ce160 2023-11-06T03:04:15.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00cd050 con 0x7fa5c8004d10 2023-11-06T03:04:15.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00cd330 con 0x7fa5e014d360 2023-11-06T03:04:15.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43704/0 (socket says 172.21.15.157:43704) 2023-11-06T03:04:15.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 -- 172.21.15.157:0/590126826 learned_addr learned my addr 172.21.15.157:0/590126826 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8004d10 0x7fa5e006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5dffff640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e014d360 0x7fa5e014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 -- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e014d360 msgr2=0x7fa5e014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e014d360 0x7fa5e014c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 -- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8004d10 msgr2=0x7fa5e006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8004d10 0x7fa5e006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 -- 172.21.15.157:0/590126826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e00cd610 con 0x7fa5e00ce160 2023-11-06T03:04:15.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e5223640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5d801f8d0 tx=0x7fa5c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80b8070 con 0x7fa5e00ce160 2023-11-06T03:04:15.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d8001fe0 con 0x7fa5e00ce160 2023-11-06T03:04:15.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e00cd8f0 con 0x7fa5e00ce160 2023-11-06T03:04:15.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80aa6d0 con 0x7fa5e00ce160 2023-11-06T03:04:15.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.681+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e006cb80 con 0x7fa5e00ce160 2023-11-06T03:04:15.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0072750 con 0x7fa5e00ce160 2023-11-06T03:04:15.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa5d80aa9a0 con 0x7fa5e00ce160 2023-11-06T03:04:15.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5dd7fa640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 0x7fa5cc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5d81209a0 con 0x7fa5e00ce160 2023-11-06T03:04:15.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 0x7fa5cc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.685+0000 7fa5e4a22640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 0x7fa5cc067f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009ac0 tx=0x7fa5d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:15.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.689+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa5d80f1d20 con 0x7fa5e00ce160 2023-11-06T03:04:15.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.845+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5e014f990 con 0x7fa5e00ce160 2023-11-06T03:04:15.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5dd7fa640 1 -- 172.21.15.157:0/590126826 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fa5d80f18b0 con 0x7fa5e00ce160 2023-11-06T03:04:15.852 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:15.852 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:15.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 msgr2=0x7fa5cc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 0x7fa5cc067f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009ac0 tx=0x7fa5d0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:15.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 msgr2=0x7fa5e014c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5d801f8d0 tx=0x7fa5c8005800 comp rx=0 tx=0).stop 2023-11-06T03:04:15.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 shutdown_connections 2023-11-06T03:04:15.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.849+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa5cc065a90 0x7fa5cc067f50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e00ce160 0x7fa5e014c770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5e014d360 0x7fa5e014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 --2- 172.21.15.157:0/590126826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5c8004d10 0x7fa5e006c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 >> 172.21.15.157:0/590126826 conn(0x7fa5e0065930 msgr2=0x7fa5e00af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:15.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 shutdown_connections 2023-11-06T03:04:15.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.853+0000 7fa5e6c86640 1 -- 172.21.15.157:0/590126826 wait complete. 2023-11-06T03:04:15.870 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-06T03:04:15.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:15.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 Processor -- start 2023-11-06T03:04:15.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 -- start start 2023-11-06T03:04:15.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f492c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c15f580 con 0x7f492c1551e0 2023-11-06T03:04:15.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c15efc0 con 0x7f492c15ea60 2023-11-06T03:04:15.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4933d5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0704b0 con 0x7f492c15e5e0 2023-11-06T03:04:15.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f492c1551e0 0x7f492c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43534/0 (socket says 172.21.15.157:43534) 2023-11-06T03:04:15.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 -- 172.21.15.157:0/1624659667 learned_addr learned my addr 172.21.15.157:0/1624659667 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f49312f9640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f492c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466645854 0 0) 0x7f492c15f580 con 0x7f492c1551e0 2023-11-06T03:04:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4918003530 con 0x7f492c1551e0 2023-11-06T03:04:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234488773 0 0) 0x7f4918003530 con 0x7f492c1551e0 2023-11-06T03:04:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 msgr2=0x7f492c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f492c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 >> v1:172.21.15.195:6789/0 conn(0x7f492c15ea60 legacy=0x7f492c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c15f2a0 con 0x7f492c1551e0 2023-11-06T03:04:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4928001680 con 0x7f492c1551e0 2023-11-06T03:04:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 >> v1:172.21.15.157:6789/0 conn(0x7f492c1551e0 legacy=0x7f492c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49180039b0 0x7f4918003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49180047d0 0x7f4918004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f49180051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0704b0 con 0x7f49180039b0 2023-11-06T03:04:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491800bb20 con 0x7f49180047d0 2023-11-06T03:04:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491800bcb0 con 0x7f492c15e5e0 2023-11-06T03:04:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49180039b0 0x7f4918003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f49312f9640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49180047d0 0x7f4918004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 -- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 msgr2=0x7f49180051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f49322fb640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f49180051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f49180051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 -- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49180047d0 msgr2=0x7f4918004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49180047d0 0x7f4918004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f4931afa640 1 -- 172.21.15.157:0/1624659667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918003530 con 0x7f49180039b0 2023-11-06T03:04:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f49312f9640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49180047d0 0x7f4918004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.985+0000 7f49322fb640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f492c15e5e0 0x7f49180051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4931afa640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49180039b0 0x7f4918003dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f492800c270 tx=0x7f492801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb523bebda8da60 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928005ae0 con 0x7f49180039b0 2023-11-06T03:04:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928005c40 con 0x7f49180039b0 2023-11-06T03:04:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4930af8640 1 -- 172.21.15.157:0/1624659667 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49280a5800 con 0x7f49180039b0 2023-11-06T03:04:16.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- 172.21.15.157:0/1624659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49180039b0 msgr2=0x7f4918003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1624659667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49180039b0 0x7f4918003dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f492800c270 tx=0x7f492801cc10 comp rx=0 tx=0).stop 2023-11-06T03:04:16.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f49322fb640 1 -- 172.21.15.157:0/1624659667 reap_dead start 2023-11-06T03:04:16.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- 172.21.15.157:0/1624659667 shutdown_connections 2023-11-06T03:04:16.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- 172.21.15.157:0/1624659667 >> 172.21.15.157:0/1624659667 conn(0x7f492c065930 msgr2=0x7f492c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- 172.21.15.157:0/1624659667 shutdown_connections 2023-11-06T03:04:16.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- 172.21.15.157:0/1624659667 wait complete. 2023-11-06T03:04:16.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 Processor -- start 2023-11-06T03:04:16.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.989+0000 7f4933d5e640 1 -- start start 2023-11-06T03:04:16.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49180039b0 0x7f492c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c152e10 0x7f492c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0cca60 con 0x7f492c15a010 2023-11-06T03:04:16.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c156a10 con 0x7f492c152e10 2023-11-06T03:04:16.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c156cf0 con 0x7f49180039b0 2023-11-06T03:04:16.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43720/0 (socket says 172.21.15.157:43720) 2023-11-06T03:04:16.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 -- 172.21.15.157:0/1303905818 learned_addr learned my addr 172.21.15.157:0/1303905818 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49312f9640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c152e10 0x7f492c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4931afa640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49180039b0 0x7f492c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 -- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49180039b0 msgr2=0x7f492c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49180039b0 0x7f492c155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 -- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c152e10 msgr2=0x7f492c1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c152e10 0x7f492c1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 -- 172.21.15.157:0/1303905818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c156fd0 con 0x7f492c15a010 2023-11-06T03:04:16.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49322fb640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f492c0cc5a0 tx=0x7f492400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924006d60 con 0x7f492c15a010 2023-11-06T03:04:16.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4924003980 con 0x7f492c15a010 2023-11-06T03:04:16.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0cccb0 con 0x7f492c15a010 2023-11-06T03:04:16.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924003d70 con 0x7f492c15a010 2023-11-06T03:04:16.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c155bb0 con 0x7f492c15a010 2023-11-06T03:04:16.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c072750 con 0x7f492c15a010 2023-11-06T03:04:16.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4924005070 con 0x7f492c15a010 2023-11-06T03:04:16.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 0x7f4910067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4924082ff0 con 0x7f492c15a010 2023-11-06T03:04:16.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4931afa640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 0x7f4910067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.993+0000 7f4931afa640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 0x7f4910067f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49280ab000 tx=0x7f49280a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:15.997+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4924050850 con 0x7f492c15a010 2023-11-06T03:04:16.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.153+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492c071730 con 0x7f492c15a010 2023-11-06T03:04:16.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.153+0000 7f49227fc640 1 -- 172.21.15.157:0/1303905818 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f4924053e70 con 0x7f492c15a010 2023-11-06T03:04:16.157 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:16.157 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 msgr2=0x7f4910067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 0x7f4910067f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49280ab000 tx=0x7f49280a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 msgr2=0x7f492c15a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f492c0cc5a0 tx=0x7f492400dc00 comp rx=0 tx=0).stop 2023-11-06T03:04:16.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 shutdown_connections 2023-11-06T03:04:16.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4910065a90 0x7f4910067f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f492c15a010 0x7f492c15a4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f492c152e10 0x7f492c1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 --2- 172.21.15.157:0/1303905818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49180039b0 0x7f492c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 >> 172.21.15.157:0/1303905818 conn(0x7f492c065930 msgr2=0x7f492c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 shutdown_connections 2023-11-06T03:04:16.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.157+0000 7f4933d5e640 1 -- 172.21.15.157:0/1303905818 wait complete. 2023-11-06T03:04:16.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-06T03:04:16.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-06T03:04:16.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-06T03:04:16.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-06T03:04:16.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-06T03:04:16.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-06T03:04:16.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-06T03:04:16.180 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-06T03:04:16.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.345+0000 7f1ef56f4640 1 Processor -- start 2023-11-06T03:04:16.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.345+0000 7f1ef56f4640 1 -- start start 2023-11-06T03:04:16.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1ef56f4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ef00c9ce0 0x7f1ef00cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1ef56f4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1ef00ccfa0 0x7f1ef01581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1ef56f4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00cd7d0 con 0x7f1ef00cebf0 2023-11-06T03:04:16.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1ef56f4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00cdab0 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1ef56f4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00b1e50 con 0x7f1ef00ccfa0 2023-11-06T03:04:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ef00c9ce0 0x7f1ef00cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ef00c9ce0 0x7f1ef00cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43734/0 (socket says 172.21.15.157:43734) 2023-11-06T03:04:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 -- 172.21.15.157:0/2892376254 learned_addr learned my addr 172.21.15.157:0/2892376254 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 -- 172.21.15.157:0/2892376254 >> v2:172.21.15.195:3300/0 conn(0x7f1ef00ccfa0 msgr2=0x7f1ef01581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 --2- 172.21.15.157:0/2892376254 >> v2:172.21.15.195:3300/0 conn(0x7f1ef00ccfa0 0x7f1ef01581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 -- 172.21.15.157:0/2892376254 >> v1:172.21.15.157:6789/0 conn(0x7f1ef00cebf0 legacy=0x7f1ef00c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.349+0000 7f1eee7fc640 1 -- 172.21.15.157:0/2892376254 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00b2130 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1eee7fc640 1 --2- 172.21.15.157:0/2892376254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef00cca60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ef006aa80 tx=0x7f1ee801ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7dab3615066c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1eedffb640 1 -- 172.21.15.157:0/2892376254 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80ae600 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1eedffb640 1 -- 172.21.15.157:0/2892376254 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee80aa050 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1eedffb640 1 -- 172.21.15.157:0/2892376254 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80ae980 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 -- 172.21.15.157:0/2892376254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00c9ce0 msgr2=0x7f1ef00cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/2892376254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef00cca60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ef006aa80 tx=0x7f1ee801ef20 comp rx=0 tx=0).stop 2023-11-06T03:04:16.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 -- 172.21.15.157:0/2892376254 shutdown_connections 2023-11-06T03:04:16.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/2892376254 >> v2:172.21.15.195:3300/0 conn(0x7f1ef00ccfa0 0x7f1ef01581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/2892376254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef00cca60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 -- 172.21.15.157:0/2892376254 >> 172.21.15.157:0/2892376254 conn(0x7f1ef0065930 msgr2=0x7f1ef0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 -- 172.21.15.157:0/2892376254 shutdown_connections 2023-11-06T03:04:16.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.353+0000 7f1ef56f4640 1 -- 172.21.15.157:0/2892376254 wait complete. 2023-11-06T03:04:16.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 Processor -- start 2023-11-06T03:04:16.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 Processor -- start 2023-11-06T03:04:16.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- start start 2023-11-06T03:04:16.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 -- start start 2023-11-06T03:04:16.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 Processor -- start 2023-11-06T03:04:16.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 -- start start 2023-11-06T03:04:16.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 Processor -- start 2023-11-06T03:04:16.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef00ccfa0 0x7f1ef015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00cebf0 0x7f1ef00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7aa4005ff0 0x7f7aa4007520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 0x7f7aa4054220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c6ad0 con 0x7f1ef00cebf0 2023-11-06T03:04:16.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef015aac0 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef015ada0 con 0x7f1ef00ccfa0 2023-11-06T03:04:16.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa4006480 con 0x7f7aa40df570 2023-11-06T03:04:16.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa4006760 con 0x7f7aa4005ff0 2023-11-06T03:04:16.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab10d4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4006a40 con 0x7f7aa4050390 2023-11-06T03:04:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00cebf0 0x7f1ef00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00cebf0 0x7f1ef00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43744/0 (socket says 172.21.15.157:43744) 2023-11-06T03:04:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eef7fe640 1 -- 172.21.15.157:0/1440463323 learned_addr learned my addr 172.21.15.157:0/1440463323 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eee7fc640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef00ccfa0 0x7f1ef015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 -- start start 2023-11-06T03:04:16.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7aabfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7aa40df570 0x7f7aa404fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43552/0 (socket says 172.21.15.157:43552) 2023-11-06T03:04:16.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7aabfff640 1 -- 172.21.15.157:0/2914879688 learned_addr learned my addr 172.21.15.157:0/2914879688 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/2914879688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 0x7f7aa4054220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7aab7fe640 1 --2- 172.21.15.157:0/2914879688 >> v2:172.21.15.195:3300/0 conn(0x7f7aa4005ff0 0x7f7aa4007520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 0x7f0b280c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b280cc140 0x7f0b280caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280ce090 con 0x7f0b280cb490 2023-11-06T03:04:16.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280cc620 con 0x7f0b280cc140 2023-11-06T03:04:16.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2f542640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280cb940 con 0x7f0b28161050 2023-11-06T03:04:16.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 -- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef00ccfa0 msgr2=0x7f1ef015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ef00ccfa0 0x7f1ef015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 -- 172.21.15.157:0/1440463323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00cebf0 msgr2=0x7f1ef00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef00cebf0 0x7f1ef00ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee800c050 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1eeeffd640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef0154a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c390 tx=0x7f1ee000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b280cc140 0x7f0b280caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b280cc140 0x7f0b280caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43610/0 (socket says 172.21.15.157:43610) 2023-11-06T03:04:16.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 -- 172.21.15.157:0/52527311 learned_addr learned my addr 172.21.15.157:0/52527311 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 0x7fcb880c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8806d950 con 0x7fcb880c5140 2023-11-06T03:04:16.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8806cd40 con 0x7fcb880cb5b0 2023-11-06T03:04:16.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8ca43640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881569c0 con 0x7fcb88159990 2023-11-06T03:04:16.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2d2de640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 0x7f0b280c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00052d0 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef015b080 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef0154f80 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb86d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcb880c5140 0x7fcb880caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43568/0 (socket says 172.21.15.157:43568) 2023-11-06T03:04:16.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb86d9d640 1 -- 172.21.15.157:0/506320183 learned_addr learned my addr 172.21.15.157:0/506320183 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab08d3640 1 -- 172.21.15.157:0/2914879688 >> v2:172.21.15.195:3300/0 conn(0x7f7aa4005ff0 msgr2=0x7f7aa4007520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/2914879688 >> v2:172.21.15.195:3300/0 conn(0x7f7aa4005ff0 0x7f7aa4007520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab08d3640 1 -- 172.21.15.157:0/2914879688 >> v1:172.21.15.157:6789/0 conn(0x7f7aa40df570 legacy=0x7f7aa404fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f7ab08d3640 1 -- 172.21.15.157:0/2914879688 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4006d20 con 0x7f7aa4050390 2023-11-06T03:04:16.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 -- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 msgr2=0x7f0b280c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 0x7f0b280c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 -- 172.21.15.157:0/52527311 >> v1:172.21.15.195:6789/0 conn(0x7f0b280cb490 legacy=0x7f0b280ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2cadd640 1 -- 172.21.15.157:0/52527311 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28161430 con 0x7f0b280cc140 2023-11-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f0b2d2de640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 0x7f0b280c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7fcb8759e640 1 --2- 172.21.15.157:0/506320183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 0x7fcb880c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fcb8759e640 1 -- 172.21.15.157:0/506320183 >> v1:172.21.15.195:6789/0 conn(0x7fcb880cb5b0 legacy=0x7fcb880c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fcb8759e640 1 -- 172.21.15.157:0/506320183 >> v1:172.21.15.157:6789/0 conn(0x7fcb880c5140 legacy=0x7fcb880caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fcb8759e640 1 -- 172.21.15.157:0/506320183 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8806d020 con 0x7fcb88159990 2023-11-06T03:04:16.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0005430 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.357+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0003820 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ed0000fd0 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 Processor -- start 2023-11-06T03:04:16.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 -- start start 2023-11-06T03:04:16.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4eb80b41b0 0x7f4eb80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80b06b0 0x7f4eb80b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80cdd00 con 0x7f4eb80ccb70 2023-11-06T03:04:16.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb80cd0f0 con 0x7f4eb80b41b0 2023-11-06T03:04:16.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebf249640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80cd3d0 con 0x7f4eb80b06b0 2023-11-06T03:04:16.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/2914879688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 0x7f7aa4054220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097c0 tx=0x7f7a9801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2e2a620f0c1b57 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 Processor -- start 2023-11-06T03:04:16.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4eb80b41b0 0x7f4eb80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4eb80b41b0 0x7f4eb80cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43626/0 (socket says 172.21.15.157:43626) 2023-11-06T03:04:16.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 -- 172.21.15.157:0/335997309 learned_addr learned my addr 172.21.15.157:0/335997309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fef7de4c640 1 Processor -- start 2023-11-06T03:04:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fef7de4c640 1 -- start start 2023-11-06T03:04:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 -- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80b06b0 msgr2=0x7f4eb80b1650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T03:04:16.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 --2- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80b06b0 0x7f4eb80b1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 -- 172.21.15.157:0/335997309 >> v1:172.21.15.195:6789/0 conn(0x7f4eb80ccb70 legacy=0x7f4eb80affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f4ebe247640 1 -- 172.21.15.157:0/335997309 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80cd6b0 con 0x7f4eb80b41b0 2023-11-06T03:04:16.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f0b2cadd640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280cc140 0x7f0b280caf50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009870 tx=0x7f0b2401ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faefe51d4b9a2690 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 -- start start 2023-11-06T03:04:16.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3efc15d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0721f0 con 0x7f3efc0cf490 2023-11-06T03:04:16.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0aed80 con 0x7f3efc0ae9a0 2023-11-06T03:04:16.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f03599640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0cc480 con 0x7f3efc15d4a0 2023-11-06T03:04:16.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f01b36640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3efc0cf490 0x7f3efc06bad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43570/0 (socket says 172.21.15.157:43570) 2023-11-06T03:04:16.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7f3f01b36640 1 -- 172.21.15.157:0/161889301 learned_addr learned my addr 172.21.15.157:0/161889301 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.361+0000 7fcb8759e640 1 --2- 172.21.15.157:0/506320183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 0x7fcb880c9ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097d0 tx=0x7fcb880ca830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1588fa9751d44e09 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592739271 0 0) 0x7f3efc0721f0 con 0x7f3efc0cf490 2023-11-06T03:04:16.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee8005740 con 0x7f3efc0cf490 2023-11-06T03:04:16.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f00b34640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3efc15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7de4c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef7815a7d0 0x7fef780cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903340515 0 0) 0x7f3ee8005740 con 0x7f3efc0cf490 2023-11-06T03:04:16.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 msgr2=0x7f3efc15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3efc15d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 >> v1:172.21.15.195:6789/0 conn(0x7f3efc0ae9a0 legacy=0x7f3efc0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc15e040 con 0x7f3efc0cf490 2023-11-06T03:04:16.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3ef8001680 con 0x7f3efc0cf490 2023-11-06T03:04:16.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef7815a7d0 0x7fef780cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef7815a7d0 0x7fef780cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43758/0 (socket says 172.21.15.157:43758) 2023-11-06T03:04:16.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 -- 172.21.15.157:0/3915538823 learned_addr learned my addr 172.21.15.157:0/3915538823 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 >> v1:172.21.15.157:6789/0 conn(0x7f3efc0cf490 legacy=0x7f3efc06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8005bc0 0x7f3ee8005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ee80069e0 0x7f3ee8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3ee80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0cc480 con 0x7f3ee8005bc0 2023-11-06T03:04:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee800dc50 con 0x7f3ee80069e0 2023-11-06T03:04:16.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee800dde0 con 0x7f3efc15d4a0 2023-11-06T03:04:16.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780cbe40 con 0x7fef7815abb0 2023-11-06T03:04:16.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780cc070 con 0x7fef780b35a0 2023-11-06T03:04:16.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780cc350 con 0x7fef7815a7d0 2023-11-06T03:04:16.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8005bc0 0x7f3ee8005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01b36640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3ee80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 -- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 msgr2=0x7f3ee80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc15d4a0 0x7f3ee80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 -- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ee80069e0 msgr2=0x7f3ee8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ee80069e0 0x7f3ee8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f3f01335640 1 -- 172.21.15.157:0/161889301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc15e040 con 0x7f3ee8005bc0 2023-11-06T03:04:16.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 -- 172.21.15.157:0/3915538823 >> v1:172.21.15.195:6789/0 conn(0x7fef780b35a0 legacy=0x7fef780cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 -- 172.21.15.157:0/3915538823 >> v1:172.21.15.157:6789/0 conn(0x7fef7815abb0 legacy=0x7fef780d0810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:16.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fef7ce4a640 1 -- 172.21.15.157:0/3915538823 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780cc630 con 0x7fef7815a7d0 2023-11-06T03:04:16.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f4ebe247640 1 --2- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb80b41b0 0x7f4eb80cc630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009e50 tx=0x7f4eb001f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d4b494d3e52a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fcb85d9b640 1 -- 172.21.15.157:0/506320183 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0ab670 con 0x7fcb88159990 2023-11-06T03:04:16.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f7aaaffd640 1 -- 172.21.15.157:0/2914879688 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980ad640 con 0x7f7aa4050390 2023-11-06T03:04:16.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f7aaaffd640 1 -- 172.21.15.157:0/2914879688 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a980ad7a0 con 0x7f7aa4050390 2023-11-06T03:04:16.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7f7aaaffd640 1 -- 172.21.15.157:0/2914879688 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980adb30 con 0x7f7aa4050390 2023-11-06T03:04:16.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fcb85d9b640 1 -- 172.21.15.157:0/506320183 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c0ab7d0 con 0x7fcb88159990 2023-11-06T03:04:16.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.365+0000 7fcb85d9b640 1 -- 172.21.15.157:0/506320183 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0abb60 con 0x7fcb88159990 2023-11-06T03:04:16.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b1ffff640 1 -- 172.21.15.157:0/52527311 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b240a8a30 con 0x7f0b280cc140 2023-11-06T03:04:16.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebd245640 1 -- 172.21.15.157:0/335997309 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00af400 con 0x7f4eb80b41b0 2023-11-06T03:04:16.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- 172.21.15.157:0/2914879688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 msgr2=0x7f7aa4054220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/2914879688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 0x7f7aa4054220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097c0 tx=0x7f7a9801c4c0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- 172.21.15.157:0/2914879688 shutdown_connections 2023-11-06T03:04:16.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/2914879688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa4050390 0x7f7aa4054220 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/2914879688 >> v2:172.21.15.195:3300/0 conn(0x7f7aa4005ff0 0x7f7aa4007520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- 172.21.15.157:0/2914879688 >> 172.21.15.157:0/2914879688 conn(0x7f7aa4054c20 msgr2=0x7f7aa4055030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b1ffff640 1 -- 172.21.15.157:0/52527311 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b240a8b90 con 0x7f0b280cc140 2023-11-06T03:04:16.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b1ffff640 1 -- 172.21.15.157:0/52527311 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b240b0440 con 0x7f0b280cc140 2023-11-06T03:04:16.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7ce4a640 1 --2- 172.21.15.157:0/3915538823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815a7d0 0x7fef780cf620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9ecd0252d1f772 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- 172.21.15.157:0/506320183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 msgr2=0x7fcb880c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 --2- 172.21.15.157:0/506320183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 0x7fcb880c9ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097d0 tx=0x7fcb880ca830 comp rx=0 tx=0).stop 2023-11-06T03:04:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebd245640 1 -- 172.21.15.157:0/335997309 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb00ab050 con 0x7f4eb80b41b0 2023-11-06T03:04:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebd245640 1 -- 172.21.15.157:0/335997309 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00af780 con 0x7f4eb80b41b0 2023-11-06T03:04:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- 172.21.15.157:0/506320183 shutdown_connections 2023-11-06T03:04:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 --2- 172.21.15.157:0/506320183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb88159990 0x7fcb880c9ae0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- 172.21.15.157:0/506320183 >> 172.21.15.157:0/506320183 conn(0x7fcb880b0160 msgr2=0x7fcb880b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- 172.21.15.157:0/506320183 shutdown_connections 2023-11-06T03:04:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef777fe640 1 -- 172.21.15.157:0/3915538823 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0aa5f0 con 0x7fef7815a7d0 2023-11-06T03:04:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef777fe640 1 -- 172.21.15.157:0/3915538823 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c0aa750 con 0x7fef7815a7d0 2023-11-06T03:04:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef777fe640 1 -- 172.21.15.157:0/3915538823 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0aaae0 con 0x7fef7815a7d0 2023-11-06T03:04:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- 172.21.15.157:0/2914879688 shutdown_connections 2023-11-06T03:04:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- 172.21.15.157:0/2914879688 wait complete. 2023-11-06T03:04:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 Processor -- start 2023-11-06T03:04:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- start start 2023-11-06T03:04:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa4050390 0x7f7aa40344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa40df570 0x7f7aa417f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 0x7f7aa40371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40359f0 con 0x7f7aa40da140 2023-11-06T03:04:16.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4034d70 con 0x7f7aa4050390 2023-11-06T03:04:16.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab10d4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa4035050 con 0x7f7aa40df570 2023-11-06T03:04:16.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 -- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280cc140 msgr2=0x7f0b280caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280cc140 0x7f0b280caf50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009870 tx=0x7f0b2401ed30 comp rx=0 tx=0).stop 2023-11-06T03:04:16.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 -- 172.21.15.157:0/52527311 shutdown_connections 2023-11-06T03:04:16.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b280cc140 0x7f0b280caf50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 --2- 172.21.15.157:0/52527311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b28161050 0x7f0b280c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 -- 172.21.15.157:0/52527311 >> 172.21.15.157:0/52527311 conn(0x7f0b28065930 msgr2=0x7f0b28065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 -- 172.21.15.157:0/52527311 shutdown_connections 2023-11-06T03:04:16.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815a7d0 msgr2=0x7fef780cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 --2- 172.21.15.157:0/3915538823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815a7d0 0x7fef780cf620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c01c470 comp rx=0 tx=0).stop 2023-11-06T03:04:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 shutdown_connections 2023-11-06T03:04:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 --2- 172.21.15.157:0/3915538823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815a7d0 0x7fef780cf620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 >> 172.21.15.157:0/3915538823 conn(0x7fef7806d350 msgr2=0x7fef7806d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 shutdown_connections 2023-11-06T03:04:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fef7de4c640 1 -- 172.21.15.157:0/3915538823 wait complete. 2023-11-06T03:04:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- 172.21.15.157:0/506320183 wait complete. 2023-11-06T03:04:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 Processor -- start 2023-11-06T03:04:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7fcb8ca43640 1 -- start start 2023-11-06T03:04:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ee0003980 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1edc065a60 0x7f1edc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ee00832d0 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f1edc06e320 0x7f1edc070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1edc070df0 con 0x7f1edc06e320 2023-11-06T03:04:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee0083680 con 0x7f1ef00c9ce0 2023-11-06T03:04:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb80b41b0 msgr2=0x7f4eb80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb80b41b0 0x7f4eb80cc630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009e50 tx=0x7f4eb001f210 comp rx=0 tx=0).stop 2023-11-06T03:04:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- 172.21.15.157:0/335997309 shutdown_connections 2023-11-06T03:04:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80b06b0 0x7f4eb80b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- 172.21.15.157:0/335997309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb80b41b0 0x7f4eb80cc630 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009e50 tx=0x7f4eb001f210 comp rx=0 tx=0).stop 2023-11-06T03:04:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- 172.21.15.157:0/335997309 >> 172.21.15.157:0/335997309 conn(0x7f4eb80cafa0 msgr2=0x7f4eb80cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- 172.21.15.157:0/335997309 shutdown_connections 2023-11-06T03:04:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- 172.21.15.157:0/335997309 wait complete. 2023-11-06T03:04:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 Processor -- start 2023-11-06T03:04:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f3f01335640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8005bc0 0x7f3ee8005fc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009e20 tx=0x7f3ef4025450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed732c44946d0d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1eee7fc640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1edc065a60 0x7f1edc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f1eef7fe640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f1edc06e320 0x7f1edc070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 0x7f7aa40371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 0x7f7aa40371a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43768/0 (socket says 172.21.15.157:43768) 2023-11-06T03:04:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 -- 172.21.15.157:0/3219806817 learned_addr learned my addr 172.21.15.157:0/3219806817 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7aab7fe640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa40df570 0x7f7aa417f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7aabfff640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa4050390 0x7f7aa40344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 -- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa40df570 msgr2=0x7f7aa417f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa40df570 0x7f7aa417f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 -- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa4050390 msgr2=0x7f7aa40344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7aa4050390 0x7f7aa40344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 -- 172.21.15.157:0/3219806817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98009420 con 0x7f7aa40da140 2023-11-06T03:04:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7aab7fe640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7aa40df570 0x7f7aa417f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f7ab08d3640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 0x7f7aa40371a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a9801a0f0 tx=0x7f7a98019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- start start 2023-11-06T03:04:16.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80b06b0 0x7f4eb80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80ccb70 0x7f4eb80c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 0x7f4eb81f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80c87f0 con 0x7f4eb80b06b0 2023-11-06T03:04:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80c7be0 con 0x7f4eb81f59a0 2023-11-06T03:04:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f4ebf249640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80c7ec0 con 0x7f4eb80ccb70 2023-11-06T03:04:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40b6680 con 0x7f3ee8005bc0 2023-11-06T03:04:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40b67e0 con 0x7f3ee8005bc0 2023-11-06T03:04:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f3ef3fff640 1 -- 172.21.15.157:0/161889301 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40b6b90 con 0x7f3ee8005bc0 2023-11-06T03:04:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 0x7fcb88071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebda46640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80ccb70 0x7f4eb80c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb880cb5b0 0x7fcb880c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 0x7f4eb81f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 0x7f4eb81f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43646/0 (socket says 172.21.15.157:43646) 2023-11-06T03:04:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 -- 172.21.15.157:0/789320333 learned_addr learned my addr 172.21.15.157:0/789320333 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebe247640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80b06b0 0x7f4eb80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88159990 0x7fcb880c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 Processor -- start 2023-11-06T03:04:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980b6600 con 0x7f7aa40da140 2023-11-06T03:04:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a980a6040 con 0x7f7aa40da140 2023-11-06T03:04:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980b6960 con 0x7f7aa40da140 2023-11-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa4035330 con 0x7f7aa40da140 2023-11-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40349f0 con 0x7f7aa40da140 2023-11-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 -- start start 2023-11-06T03:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 0x7fcb88071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb880cb5b0 0x7fcb880c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 0x7fcb88071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43650/0 (socket says 172.21.15.157:43650) 2023-11-06T03:04:16.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 -- 172.21.15.157:0/1469292513 learned_addr learned my addr 172.21.15.157:0/1469292513 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.369+0000 7f0b2f542640 1 -- 172.21.15.157:0/52527311 wait complete. 2023-11-06T03:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 Processor -- start 2023-11-06T03:04:16.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 -- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80ccb70 msgr2=0x7f4eb80c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4eb80ccb70 0x7f4eb80c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 -- 172.21.15.157:0/789320333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80b06b0 msgr2=0x7f4eb80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80b06b0 0x7f4eb80c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009b30 con 0x7f4eb81f59a0 2023-11-06T03:04:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88159990 0x7fcb880c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebea48640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 0x7f4eb81f5e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400b3e0 tx=0x7f4eb400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88071250 con 0x7fcb88159990 2023-11-06T03:04:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880add70 con 0x7fcb880c5140 2023-11-06T03:04:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 0x7fef780b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4011830 con 0x7f4eb81f59a0 2023-11-06T03:04:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef7815a7d0 0x7fef7814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881563a0 con 0x7fcb880cb5b0 2023-11-06T03:04:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 -- 172.21.15.157:0/161889301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8005bc0 msgr2=0x7f3ee8005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 --2- 172.21.15.157:0/161889301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ee8005bc0 0x7f3ee8005fc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009e20 tx=0x7f3ef4025450 comp rx=0 tx=0).stop 2023-11-06T03:04:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f01b36640 1 -- 172.21.15.157:0/161889301 reap_dead start 2023-11-06T03:04:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 0x7fef78148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7a980b4040 con 0x7f7aa40da140 2023-11-06T03:04:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a90065b30 0x7f7a90067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 -- start start 2023-11-06T03:04:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280cb490 0x7f0b2814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 0x7f0b281fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b28161050 0x7f0b280b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281f9d60 con 0x7f0b280cc140 2023-11-06T03:04:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281504b0 con 0x7f0b28161050 2023-11-06T03:04:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2f542640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281f9a30 con 0x7f0b280cb490 2023-11-06T03:04:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2d2de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280cb490 0x7f0b2814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 0x7f0b281fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 0x7f0b281fa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43812/0 (socket says 172.21.15.157:43812) 2023-11-06T03:04:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 -- 172.21.15.157:0/2043858584 learned_addr learned my addr 172.21.15.157:0/2043858584 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2dadf640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b28161050 0x7f0b280b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7ce4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 0x7fef780b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7ce4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 0x7fef780b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54048/0 (socket says 172.21.15.157:54048) 2023-11-06T03:04:16.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef77fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef7815a7d0 0x7fef7814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7d64b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 0x7fef78148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7d64b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 0x7fef78148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43804/0 (socket says 172.21.15.157:43804) 2023-11-06T03:04:16.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef77fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef7815a7d0 0x7fef7814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43664/0 (socket says 172.21.15.157:43664) 2023-11-06T03:04:16.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 -- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb880cb5b0 msgr2=0x7fcb880c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb880cb5b0 0x7fcb880c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 -- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88159990 msgr2=0x7fcb880c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88159990 0x7fcb880c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fcb86d9d640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c009400 con 0x7fcb880c5140 2023-11-06T03:04:16.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f1eee7fc640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1edc065a60 0x7f1edc067f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80b0000 tx=0x7f1ee80a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f1eef7fe640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f1edc06e320 0x7f1edc070740 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4011990 con 0x7f4eb81f59a0 2023-11-06T03:04:16.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb80c81a0 con 0x7f4eb81f59a0 2023-11-06T03:04:16.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4011d40 con 0x7f4eb81f59a0 2023-11-06T03:04:16.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb814d7d0 con 0x7f4eb81f59a0 2023-11-06T03:04:16.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4eb80ce190 con 0x7f4eb81f59a0 2023-11-06T03:04:16.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 -- 172.21.15.157:0/161889301 shutdown_connections 2023-11-06T03:04:16.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 -- 172.21.15.157:0/161889301 >> 172.21.15.157:0/161889301 conn(0x7f3efc0713f0 msgr2=0x7f3efc0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 -- 172.21.15.157:0/161889301 shutdown_connections 2023-11-06T03:04:16.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7faee0d9a640 1 Processor -- start 2023-11-06T03:04:16.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7faee0d9a640 1 -- start start 2023-11-06T03:04:16.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a9811d0c0 con 0x7f7aa40da140 2023-11-06T03:04:16.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 -- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280cb490 msgr2=0x7f0b2814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b280cb490 0x7f0b2814ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 -- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b28161050 msgr2=0x7f0b280b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b28161050 0x7f0b280b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f0b2cadd640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240094d0 con 0x7f0b280cc140 2023-11-06T03:04:16.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aabfff640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a90065b30 0x7f7a90067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f7aa4005ff0 0x7f7aa40d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 --> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7aa4008b30 con 0x7f7aa4005ff0 2023-11-06T03:04:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f7aab7fe640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f7aa4005ff0 0x7f7aa40d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7815b2d0 con 0x7fef7815abb0 2023-11-06T03:04:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78149a00 con 0x7fef7815a7d0 2023-11-06T03:04:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7de4c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef78149ce0 con 0x7fef780b35a0 2023-11-06T03:04:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7fef7ce4a640 1 -- 172.21.15.157:0/3930734807 learned_addr learned my addr 172.21.15.157:0/3930734807 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.373+0000 7f3f03599640 1 -- 172.21.15.157:0/161889301 wait complete. 2023-11-06T03:04:16.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faee0d9a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faed404f270 0x7faed4006700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faee0d9a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faee0d9a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed4037be0 con 0x7faed40e7820 2023-11-06T03:04:16.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faee0d9a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed404f6d0 con 0x7faed404f270 2023-11-06T03:04:16.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faee0d9a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40484f0 con 0x7faed40533b0 2023-11-06T03:04:16.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f7aab7fe640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f7aa4005ff0 0x7f7aa40d49d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f7aabfff640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a90065b30 0x7f7a90067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0006120 tx=0x7f7aa0006030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 -- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 msgr2=0x7fef780b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 0x7fef780b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 -- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef7815a7d0 msgr2=0x7fef7814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef7815a7d0 0x7fef7814fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0094d0 con 0x7fef7815abb0 2023-11-06T03:04:16.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7ce4a640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef780b35a0 0x7fef780b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:16.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54062/0 (socket says 172.21.15.157:54062) 2023-11-06T03:04:16.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 -- 172.21.15.157:0/949245548 learned_addr learned my addr 172.21.15.157:0/949245548 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 Processor -- start 2023-11-06T03:04:16.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b2cadd640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 0x7f0b281fa1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b240b3040 tx=0x7f0b2401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb86d9d640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 0x7fcb88071670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000cbd0 tx=0x7fcb8000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== osd.0 v2:172.21.15.157:6801/38220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7aa4008b30 con 0x7f7aa4005ff0 2023-11-06T03:04:16.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b240b0e20 con 0x7f0b280cc140 2023-11-06T03:04:16.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 -- 172.21.15.157:0/949245548 >> v2:172.21.15.195:3300/0 conn(0x7faed404f270 msgr2=0x7faed4006700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 --2- 172.21.15.157:0/949245548 >> v2:172.21.15.195:3300/0 conn(0x7faed404f270 0x7faed4006700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 -- 172.21.15.157:0/949245548 >> v1:172.21.15.195:6789/0 conn(0x7faed40e7820 legacy=0x7faed4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7faedbfff640 1 -- 172.21.15.157:0/949245548 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed4050320 con 0x7faed40533b0 2023-11-06T03:04:16.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b240bb470 con 0x7f0b280cc140 2023-11-06T03:04:16.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b240bb820 con 0x7f0b280cc140 2023-11-06T03:04:16.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7d64b640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 0x7fef78148a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c780 tx=0x7fef7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2806f240 con 0x7f0b280cc140 2023-11-06T03:04:16.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2806f800 con 0x7f0b280cc140 2023-11-06T03:04:16.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 -- start start 2023-11-06T03:04:16.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3efc0af440 0x7f3efc1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 0x7f3efc0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc0af820 0x7f3efc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc155970 con 0x7f3efc0af820 2023-11-06T03:04:16.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc155c50 con 0x7f3efc0af440 2023-11-06T03:04:16.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f03599640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc155f30 con 0x7f3efc155420 2023-11-06T03:04:16.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000ad40 con 0x7fcb880c5140 2023-11-06T03:04:16.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb80012400 con 0x7fcb880c5140 2023-11-06T03:04:16.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000aea0 con 0x7fcb880c5140 2023-11-06T03:04:16.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb880705d0 con 0x7fcb880c5140 2023-11-06T03:04:16.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4eb401a400 con 0x7f4eb81f59a0 2023-11-06T03:04:16.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4e9c065a60 0x7f4e9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4eb4083b00 con 0x7f4eb81f59a0 2023-11-06T03:04:16.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f4e9c06e2a0 0x7f4e9c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e9c070d70 con 0x7f4e9c06e2a0 2023-11-06T03:04:16.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb4083e70 con 0x7f4eb81f59a0 2023-11-06T03:04:16.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb88070b90 con 0x7fcb880c5140 2023-11-06T03:04:16.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 0x7f3efc0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 0x7f3efc0b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54072/0 (socket says 172.21.15.157:54072) 2023-11-06T03:04:16.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 -- 172.21.15.157:0/1998927329 learned_addr learned my addr 172.21.15.157:0/1998927329 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4ebe247640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4e9c065a60 0x7f4e9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4ebda46640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f4e9c06e2a0 0x7f4e9c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4ebe247640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4e9c065a60 0x7f4e9c067f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00b1000 tx=0x7f4eb00aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f4ebda46640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f4e9c06e2a0 0x7f4e9c0706c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedbfff640 1 --2- 172.21.15.157:0/949245548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faecc009870 tx=0x7faecc01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b483bfa519299b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b280b4250 con 0x7f0b280cc140 2023-11-06T03:04:16.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70003d90 con 0x7fef7815abb0 2023-11-06T03:04:16.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7000ce30 con 0x7fef7815abb0 2023-11-06T03:04:16.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70005290 con 0x7fef7815abb0 2023-11-06T03:04:16.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78149fc0 con 0x7fef7815abb0 2023-11-06T03:04:16.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7815cb10 con 0x7fef7815abb0 2023-11-06T03:04:16.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef58000fd0 con 0x7fef7815abb0 2023-11-06T03:04:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faeda7fc640 1 -- 172.21.15.157:0/949245548 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab600 con 0x7faed40533b0 2023-11-06T03:04:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faeda7fc640 1 -- 172.21.15.157:0/949245548 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc0a7050 con 0x7faed40533b0 2023-11-06T03:04:16.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faeda7fc640 1 -- 172.21.15.157:0/949245548 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab980 con 0x7faed40533b0 2023-11-06T03:04:16.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- 172.21.15.157:0/949245548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 msgr2=0x7faed4053840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- 172.21.15.157:0/949245548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faecc009870 tx=0x7faecc01c500 comp rx=0 tx=0).stop 2023-11-06T03:04:16.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- 172.21.15.157:0/949245548 shutdown_connections 2023-11-06T03:04:16.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- 172.21.15.157:0/949245548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed4053840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- 172.21.15.157:0/949245548 >> v2:172.21.15.195:3300/0 conn(0x7faed404f270 0x7faed4006700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- 172.21.15.157:0/949245548 >> 172.21.15.157:0/949245548 conn(0x7faed4054c20 msgr2=0x7faed4055030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- 172.21.15.157:0/949245548 shutdown_connections 2023-11-06T03:04:16.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0b240b7070 con 0x7f0b280cc140 2023-11-06T03:04:16.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== osd.5 v2:172.21.15.195:6816/38428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4e9c070d70 con 0x7f4e9c06e2a0 2023-11-06T03:04:16.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f0b1dffb640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b0c065a60 0x7f0b0c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- 172.21.15.157:0/949245548 wait complete. 2023-11-06T03:04:16.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 Processor -- start 2023-11-06T03:04:16.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- start start 2023-11-06T03:04:16.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f0b2d2de640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b0c065a60 0x7f0b0c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 0x7faed40062a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed40cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40e7820 0x7faed40dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40d1460 con 0x7faed40e7820 2023-11-06T03:04:16.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed40d6c30 con 0x7faed404f270 2023-11-06T03:04:16.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faee0d9a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed40e3ab0 con 0x7faed40533b0 2023-11-06T03:04:16.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedaffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed40cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 0x7faed40062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedaffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed40cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54080/0 (socket says 172.21.15.157:54080) 2023-11-06T03:04:16.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 0x7faed40062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43684/0 (socket says 172.21.15.157:43684) 2023-11-06T03:04:16.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7faedaffd640 1 -- 172.21.15.157:0/678329303 learned_addr learned my addr 172.21.15.157:0/678329303 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb880ccac0 con 0x7fcb880c5140 2023-11-06T03:04:16.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 -- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3efc0af440 msgr2=0x7f3efc1569f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T03:04:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3efc0af440 0x7f3efc1569f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 -- 172.21.15.157:0/1998927329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc0af820 msgr2=0x7f3efc159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3efc0af820 0x7f3efc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc156210 con 0x7f3efc155420 2023-11-06T03:04:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3f00b34640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 0x7f3efc0b4990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e7c0 tx=0x7f3eec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.377+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00eed0 con 0x7f3efc155420 2023-11-06T03:04:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00c940 con 0x7f3efc155420 2023-11-06T03:04:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.381+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00ccf0 con 0x7f3efc155420 2023-11-06T03:04:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== osd.4 v2:172.21.15.195:6808/38425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1edc070df0 con 0x7f1edc06e320 2023-11-06T03:04:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedbfff640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40e7820 0x7faed40dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f3f03599640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc1564f0 con 0x7f3efc155420 2023-11-06T03:04:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 -- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 msgr2=0x7faed40cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faed40533b0 0x7faed40cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 -- 172.21.15.157:0/678329303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40e7820 msgr2=0x7faed40dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faed40e7820 0x7faed40dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0094d0 con 0x7faed404f270 2023-11-06T03:04:16.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f3f03599640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc15a1d0 con 0x7f3efc155420 2023-11-06T03:04:16.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcb80012770 con 0x7fcb880c5140 2023-11-06T03:04:16.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faedb7fe640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 0x7faed40062a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faed000b410 tx=0x7faed000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0011830 con 0x7faed404f270 2023-11-06T03:04:16.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed0011990 con 0x7faed404f270 2023-11-06T03:04:16.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed40cf4e0 con 0x7faed404f270 2023-11-06T03:04:16.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0011d40 con 0x7faed404f270 2023-11-06T03:04:16.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed40e3460 con 0x7faed404f270 2023-11-06T03:04:16.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fef700053f0 con 0x7fef7815abb0 2023-11-06T03:04:16.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef5c065ae0 0x7fef5c067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef70083180 con 0x7fef7815abb0 2023-11-06T03:04:16.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fef5c06e3a0 0x7fef5c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef5c070e70 con 0x7fef5c06e3a0 2023-11-06T03:04:16.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef700834f0 con 0x7fef7815abb0 2023-11-06T03:04:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb74065a60 0x7fcb74067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb80083740 con 0x7fcb880c5140 2023-11-06T03:04:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7fcb7406e2a0 0x7fcb740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb74070d70 con 0x7fcb7406e2a0 2023-11-06T03:04:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb80083ab0 con 0x7fcb880c5140 2023-11-06T03:04:16.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3efc0cfcf0 con 0x7f3efc155420 2023-11-06T03:04:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b24121590 con 0x7f0b280cc140 2023-11-06T03:04:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f0b2d2de640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b0c065a60 0x7f0b0c067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b180098b0 tx=0x7f0b18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f0b1dffb640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f0b0c06e220 0x7f0b0c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b0c070cf0 con 0x7f0b0c06e220 2023-11-06T03:04:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb8659c640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb74065a60 0x7fcb74067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faebc000fd0 con 0x7faed404f270 2023-11-06T03:04:16.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef7ce4a640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef5c065ae0 0x7fef5c067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fef77fff640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fef5c06e3a0 0x7fef5c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7fcb8759e640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7fcb7406e2a0 0x7fcb740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.385+0000 7f0b2dadf640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f0b0c06e220 0x7f0b0c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b240eede0 con 0x7f0b280cc140 2023-11-06T03:04:16.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f0b2dadf640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f0b0c06e220 0x7f0b0c070640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fef7ce4a640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef5c065ae0 0x7fef5c067fa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c10 tx=0x7fef6c0b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fef77fff640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fef5c06e3a0 0x7fef5c0707c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fcb8659c640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb74065a60 0x7fcb74067f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb8816e850 tx=0x7fcb78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fcb8759e640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7fcb7406e2a0 0x7fcb740706c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faed001a400 con 0x7faed404f270 2023-11-06T03:04:16.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faec8065ab0 0x7faec8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faed0083790 con 0x7faed404f270 2023-11-06T03:04:16.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7faec806e370 0x7faec8070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faec8070e40 con 0x7faec806e370 2023-11-06T03:04:16.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed0083c30 con 0x7faed404f270 2023-11-06T03:04:16.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faedaffd640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faec8065ab0 0x7faec8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== osd.2 v2:172.21.15.157:6816/38222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fef5c070e70 con 0x7fef5c06e3a0 2023-11-06T03:04:16.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== osd.3 v2:172.21.15.157:6824/38223 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcb74070d70 con 0x7fcb7406e2a0 2023-11-06T03:04:16.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7faedbfff640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7faec806e370 0x7faec8070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3eec013470 con 0x7f3efc155420 2023-11-06T03:04:16.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ee0065a90 0x7f3ee0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3eec083230 con 0x7f3efc155420 2023-11-06T03:04:16.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3ee006e1e0 0x7f3ee0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ee0070cb0 con 0x7f3ee006e1e0 2023-11-06T03:04:16.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eec085850 con 0x7f3efc155420 2023-11-06T03:04:16.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3f01335640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ee0065a90 0x7f3ee0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3f01b36640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3ee006e1e0 0x7f3ee0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f3f01335640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ee0065a90 0x7f3ee0067f50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40b8000 tx=0x7f3ef4002410 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.389+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== osd.1 v2:172.21.15.157:6800/38221 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b0c070cf0 con 0x7f0b0c06e220 2023-11-06T03:04:16.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.393+0000 7faedaffd640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faec8065ab0 0x7faec8067f70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faedc072b10 tx=0x7faedc06c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.393+0000 7faedbfff640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7faec806e370 0x7faec8070790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.393+0000 7f3f01b36640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3ee006e1e0 0x7f3ee0070600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.397+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== osd.7 v2:172.21.15.195:6800/38427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faec8070e40 con 0x7faec806e370 2023-11-06T03:04:16.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.397+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== osd.6 v2:172.21.15.195:6817/38426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3ee0070cb0 con 0x7f3ee006e1e0 2023-11-06T03:04:16.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.401+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 --> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7aa4052d40 con 0x7f7aa4005ff0 2023-11-06T03:04:16.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7aa8ff9640 1 -- 172.21.15.157:0/3219806817 <== osd.0 v2:172.21.15.157:6801/38220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7aa4052d40 con 0x7f7aa4005ff0 2023-11-06T03:04:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f7aa4005ff0 msgr2=0x7f7aa40d49d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f7aa4005ff0 0x7f7aa40d49d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a90065b30 msgr2=0x7f7a90067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a90065b30 0x7f7a90067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0006120 tx=0x7f7aa0006030 comp rx=0 tx=0).stop 2023-11-06T03:04:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 msgr2=0x7f7aa40371a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 --2- 172.21.15.157:0/3219806817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa40da140 0x7f7aa40371a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a9801a0f0 tx=0x7f7a98019260 comp rx=0 tx=0).stop 2023-11-06T03:04:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab08d3640 1 -- 172.21.15.157:0/3219806817 reap_dead start 2023-11-06T03:04:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 shutdown_connections 2023-11-06T03:04:16.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 >> 172.21.15.157:0/3219806817 conn(0x7f7aa4054c20 msgr2=0x7f7aa40dd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 shutdown_connections 2023-11-06T03:04:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.405+0000 7f7ab10d4640 1 -- 172.21.15.157:0/3219806817 wait complete. 2023-11-06T03:04:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 --> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ed0002cf0 con 0x7f1edc06e320 2023-11-06T03:04:16.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faebc002cc0 con 0x7faec806e370 2023-11-06T03:04:16.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ed77fe640 1 -- 172.21.15.157:0/1440463323 <== osd.4 v2:172.21.15.195:6808/38425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1ed0002cf0 con 0x7f1edc06e320 2023-11-06T03:04:16.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faec3fff640 1 -- 172.21.15.157:0/678329303 <== osd.7 v2:172.21.15.195:6800/38427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faebc002cc0 con 0x7faec806e370 2023-11-06T03:04:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f1edc06e320 msgr2=0x7f1edc070740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f1edc06e320 0x7f1edc070740 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7faec806e370 msgr2=0x7faec8070790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7faec806e370 0x7faec8070790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faec8065ab0 msgr2=0x7faec8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faec8065ab0 0x7faec8067f70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faedc072b10 tx=0x7faedc06c040 comp rx=0 tx=0).stop 2023-11-06T03:04:16.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 msgr2=0x7faed40062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 --2- 172.21.15.157:0/678329303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faed404f270 0x7faed40062a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faed000b410 tx=0x7faed000b8d0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1edc065a60 msgr2=0x7f1edc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1edc065a60 0x7f1edc067f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80b0000 tx=0x7f1ee80a9000 comp rx=0 tx=0).stop 2023-11-06T03:04:16.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef00c9ce0 msgr2=0x7f1ef0154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 --2- 172.21.15.157:0/1440463323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef00c9ce0 0x7f1ef0154a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000c390 tx=0x7f1ee000c850 comp rx=0 tx=0).stop 2023-11-06T03:04:16.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1eef7fe640 1 -- 172.21.15.157:0/1440463323 reap_dead start 2023-11-06T03:04:16.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 shutdown_connections 2023-11-06T03:04:16.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 >> 172.21.15.157:0/1440463323 conn(0x7f1ef0065930 msgr2=0x7f1ef00af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faedbfff640 1 -- 172.21.15.157:0/678329303 reap_dead start 2023-11-06T03:04:16.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 shutdown_connections 2023-11-06T03:04:16.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 >> 172.21.15.157:0/678329303 conn(0x7faed4054c20 msgr2=0x7faed40366f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 shutdown_connections 2023-11-06T03:04:16.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.413+0000 7faee0d9a640 1 -- 172.21.15.157:0/678329303 wait complete. 2023-11-06T03:04:16.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 shutdown_connections 2023-11-06T03:04:16.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 --> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b280b4530 con 0x7f0b0c06e220 2023-11-06T03:04:16.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 --> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4eb80cd990 con 0x7f4e9c06e2a0 2023-11-06T03:04:16.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 --> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb88159de0 con 0x7fcb7406e2a0 2023-11-06T03:04:16.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b1dffb640 1 -- 172.21.15.157:0/2043858584 <== osd.1 v2:172.21.15.157:6800/38221 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b20001aa0 con 0x7f0b0c06e220 2023-11-06T03:04:16.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f0b0c06e220 msgr2=0x7f0b0c070640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f0b0c06e220 0x7f0b0c070640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f4eaeffd640 1 -- 172.21.15.157:0/789320333 <== osd.5 v2:172.21.15.195:6816/38428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ea8001450 con 0x7f4e9c06e2a0 2023-11-06T03:04:16.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b0c065a60 msgr2=0x7f0b0c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b0c065a60 0x7f0b0c067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b180098b0 tx=0x7f0b18008040 comp rx=0 tx=0).stop 2023-11-06T03:04:16.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 msgr2=0x7f0b281fa1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 --2- 172.21.15.157:0/2043858584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b280cc140 0x7f0b281fa1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b240b3040 tx=0x7f0b2401bad0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2dadf640 1 -- 172.21.15.157:0/2043858584 reap_dead start 2023-11-06T03:04:16.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 shutdown_connections 2023-11-06T03:04:16.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 >> 172.21.15.157:0/2043858584 conn(0x7f0b28065930 msgr2=0x7f0b280af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 shutdown_connections 2023-11-06T03:04:16.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb6f7fe640 1 -- 172.21.15.157:0/1469292513 <== osd.3 v2:172.21.15.157:6824/38223 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb7c005a70 con 0x7fcb7406e2a0 2023-11-06T03:04:16.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7f0b2f542640 1 -- 172.21.15.157:0/2043858584 wait complete. 2023-11-06T03:04:16.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7fcb7406e2a0 msgr2=0x7fcb740706c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7fcb7406e2a0 0x7fcb740706c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef58002cc0 con 0x7fef5c06e3a0 2023-11-06T03:04:16.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb74065a60 msgr2=0x7fcb74067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb74065a60 0x7fcb74067f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb8816e850 tx=0x7fcb78009040 comp rx=0 tx=0).stop 2023-11-06T03:04:16.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 msgr2=0x7fcb88071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 --2- 172.21.15.157:0/1469292513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb880c5140 0x7fcb88071670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000cbd0 tx=0x7fcb8000a430 comp rx=0 tx=0).stop 2023-11-06T03:04:16.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8759e640 1 -- 172.21.15.157:0/1469292513 reap_dead start 2023-11-06T03:04:16.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 shutdown_connections 2023-11-06T03:04:16.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 >> 172.21.15.157:0/1469292513 conn(0x7fcb880b0160 msgr2=0x7fcb880af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 shutdown_connections 2023-11-06T03:04:16.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fcb8ca43640 1 -- 172.21.15.157:0/1469292513 wait complete. 2023-11-06T03:04:16.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef757fa640 1 -- 172.21.15.157:0/3930734807 <== osd.2 v2:172.21.15.157:6816/38222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef58002cc0 con 0x7fef5c06e3a0 2023-11-06T03:04:16.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fef5c06e3a0 msgr2=0x7fef5c0707c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fef5c06e3a0 0x7fef5c0707c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef5c065ae0 msgr2=0x7fef5c067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef5c065ae0 0x7fef5c067fa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c10 tx=0x7fef6c0b5040 comp rx=0 tx=0).stop 2023-11-06T03:04:16.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 msgr2=0x7fef78148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.417+0000 7fef7de4c640 1 --2- 172.21.15.157:0/3930734807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7815abb0 0x7fef78148a10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c780 tx=0x7fef7000cc40 comp rx=0 tx=0).stop 2023-11-06T03:04:16.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7fef7d64b640 1 -- 172.21.15.157:0/3930734807 reap_dead start 2023-11-06T03:04:16.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 shutdown_connections 2023-11-06T03:04:16.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 >> 172.21.15.157:0/3930734807 conn(0x7fef7806d350 msgr2=0x7fef78153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 shutdown_connections 2023-11-06T03:04:16.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7fef7de4c640 1 -- 172.21.15.157:0/3930734807 wait complete. 2023-11-06T03:04:16.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f4e9c06e2a0 msgr2=0x7f4e9c0706c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7f4ebf249640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f4e9c06e2a0 0x7f4e9c0706c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.421+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 --> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3efc0cc0a0 con 0x7f3ee006e1e0 2023-11-06T03:04:16.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f1ef56f4640 1 -- 172.21.15.157:0/1440463323 wait complete. 2023-11-06T03:04:16.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4e9c065a60 msgr2=0x7f4e9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f4ebf249640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4e9c065a60 0x7f4e9c067f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00b1000 tx=0x7f4eb00aa000 comp rx=0 tx=0).stop 2023-11-06T03:04:16.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 msgr2=0x7f4eb81f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f4ebf249640 1 --2- 172.21.15.157:0/789320333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4eb81f59a0 0x7f4eb81f5e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400b3e0 tx=0x7f4eb400b8a0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f4ebea48640 1 -- 172.21.15.157:0/789320333 reap_dead start 2023-11-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.425+0000 7f3ef1ffb640 1 -- 172.21.15.157:0/1998927329 <== osd.6 v2:172.21.15.195:6817/38426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ef800d040 con 0x7f3ee006e1e0 2023-11-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 shutdown_connections 2023-11-06T03:04:16.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 >> 172.21.15.157:0/789320333 conn(0x7f4eb80cafa0 msgr2=0x7f4eb80af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 shutdown_connections 2023-11-06T03:04:16.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f4ebf249640 1 -- 172.21.15.157:0/789320333 wait complete. 2023-11-06T03:04:16.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3ee006e1e0 msgr2=0x7f3ee0070600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3ee006e1e0 0x7f3ee0070600 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ee0065a90 msgr2=0x7f3ee0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ee0065a90 0x7f3ee0067f50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40b8000 tx=0x7f3ef4002410 comp rx=0 tx=0).stop 2023-11-06T03:04:16.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 msgr2=0x7f3efc0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 --2- 172.21.15.157:0/1998927329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3efc155420 0x7f3efc0b4990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e7c0 tx=0x7f3eec00ec80 comp rx=0 tx=0).stop 2023-11-06T03:04:16.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3f01b36640 1 -- 172.21.15.157:0/1998927329 reap_dead start 2023-11-06T03:04:16.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 shutdown_connections 2023-11-06T03:04:16.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 >> 172.21.15.157:0/1998927329 conn(0x7f3efc0713f0 msgr2=0x7f3efc0c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 shutdown_connections 2023-11-06T03:04:16.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.429+0000 7f3edf7fe640 1 -- 172.21.15.157:0/1998927329 wait complete. 2023-11-06T03:04:16.553 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:16.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-06T03:04:16.555 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:16.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-06T03:04:16.556 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:16.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-06T03:04:16.558 INFO:teuthology.orchestra.run.smithi157.stdout:55834574857 2023-11-06T03:04:16.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-06T03:04:16.559 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:16.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-06T03:04:16.561 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:16.561 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-06T03:04:16.562 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:16.562 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-06T03:04:16.563 INFO:teuthology.orchestra.run.smithi157.stdout:55834574856 2023-11-06T03:04:16.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-06T03:04:16.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.769+0000 7f903ab29640 1 Processor -- start 2023-11-06T03:04:16.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.769+0000 7f903ab29640 1 -- start start 2023-11-06T03:04:16.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.769+0000 7f4a5ed72640 1 Processor -- start 2023-11-06T03:04:16.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 -- start start 2023-11-06T03:04:16.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a58147450 0x7f4a58070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4a580cf730 0x7f4a580cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903ab29640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90340c9ee0 0x7f90340cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903ab29640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90340cd930 0x7f90340c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903ab29640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903415d650 con 0x7f90340c9b00 2023-11-06T03:04:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903ab29640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90340ccc30 con 0x7f90340cd930 2023-11-06T03:04:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903ab29640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90341599e0 con 0x7f90340c9ee0 2023-11-06T03:04:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90340cd930 0x7f90340c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90340cd930 0x7f90340c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43838/0 (socket says 172.21.15.157:43838) 2023-11-06T03:04:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 -- 172.21.15.157:0/2684957832 learned_addr learned my addr 172.21.15.157:0/2684957832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f9039326640 1 --2- 172.21.15.157:0/2684957832 >> v2:172.21.15.195:3300/0 conn(0x7f90340c9ee0 0x7f90340cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 -- 172.21.15.157:0/2684957832 >> v2:172.21.15.195:3300/0 conn(0x7f90340c9ee0 msgr2=0x7f90340cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 --2- 172.21.15.157:0/2684957832 >> v2:172.21.15.195:3300/0 conn(0x7f90340c9ee0 0x7f90340cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 -- 172.21.15.157:0/2684957832 >> v1:172.21.15.195:6789/0 conn(0x7f90340c9b00 legacy=0x7f9034159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a58065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5814aa10 con 0x7f4a58147450 2023-11-06T03:04:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580cfca0 con 0x7f4a580cf730 2023-11-06T03:04:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5ed72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580cff80 con 0x7f4a580cbee0 2023-11-06T03:04:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f903a328640 1 -- 172.21.15.157:0/2684957832 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034159cc0 con 0x7f90340cd930 2023-11-06T03:04:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 Processor -- start 2023-11-06T03:04:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 -- start start 2023-11-06T03:04:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5d30f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a58065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5d30f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a58065930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54096/0 (socket says 172.21.15.157:54096) 2023-11-06T03:04:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5d30f640 1 -- 172.21.15.157:0/243938442 learned_addr learned my addr 172.21.15.157:0/243938442 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a57fff640 1 --2- 172.21.15.157:0/243938442 >> v2:172.21.15.195:3300/0 conn(0x7f4a580cf730 0x7f4a580cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/243938442 >> v2:172.21.15.157:3300/0 conn(0x7f4a58147450 0x7f4a58070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 Processor -- start 2023-11-06T03:04:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 -- start start 2023-11-06T03:04:16.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/243938442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 msgr2=0x7f4a58065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/243938442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a58065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/243938442 >> v2:172.21.15.195:3300/0 conn(0x7f4a580cf730 msgr2=0x7f4a580cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/243938442 >> v2:172.21.15.195:3300/0 conn(0x7f4a580cf730 0x7f4a580cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/243938442 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580d0260 con 0x7f4a58147450 2023-11-06T03:04:16.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 0x7f74340b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7434161740 0x7f7434161b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340b1840 con 0x7f74340afdd0 2023-11-06T03:04:16.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340b1ab0 con 0x7f7434161740 2023-11-06T03:04:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7438cf2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340b1d90 con 0x7f743406bcb0 2023-11-06T03:04:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f743259c640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f74340afdd0 0x7f743406b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:50110/0 (socket says 172.21.15.157:50110) 2023-11-06T03:04:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f743259c640 1 -- 172.21.15.157:0/1052930272 learned_addr learned my addr 172.21.15.157:0/1052930272 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f743359e640 1 --2- 172.21.15.157:0/1052930272 >> v2:172.21.15.195:3300/0 conn(0x7f7434161740 0x7f7434161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7431d9b640 1 --2- 172.21.15.157:0/1052930272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 0x7f74340b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8c0159a40 0x7fe8c00ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 0x7fe8c00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c00cc5e0 con 0x7fe8c00b3680 2023-11-06T03:04:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00ca6c0 con 0x7fe8c0159a40 2023-11-06T03:04:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7fe8c6923640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00ca9a0 con 0x7fe8c00ceab0 2023-11-06T03:04:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f711a544640 1 Processor -- start 2023-11-06T03:04:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f711a544640 1 -- start start 2023-11-06T03:04:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8c4ec0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 0x7fe8c00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8c4ec0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 0x7fe8c00caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54114/0 (socket says 172.21.15.157:54114) 2023-11-06T03:04:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8c4ec0640 1 -- 172.21.15.157:0/1130543591 learned_addr learned my addr 172.21.15.157:0/1130543591 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 Processor -- start 2023-11-06T03:04:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 -- start start 2023-11-06T03:04:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 --2- 172.21.15.157:0/1130543591 >> v2:172.21.15.195:3300/0 conn(0x7fe8c0159a40 0x7fe8c00ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8beffd640 1 -- 172.21.15.157:0/1130543591 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775871950 0 0) 0x7fe8c00cc5e0 con 0x7fe8c00b3680 2023-11-06T03:04:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8beffd640 1 -- 172.21.15.157:0/1130543591 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a8003530 con 0x7fe8c00b3680 2023-11-06T03:04:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 -- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 msgr2=0x7fe8c00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 --2- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 0x7fe8c00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 -- 172.21.15.157:0/1130543591 >> v1:172.21.15.195:6789/0 conn(0x7fe8c00b3680 legacy=0x7fe8c00ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 -- 172.21.15.157:0/1130543591 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00cc270 con 0x7fe8c0159a40 2023-11-06T03:04:16.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903a328640 1 --2- 172.21.15.157:0/2684957832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f90340c8b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f903406a900 tx=0x7f902c01d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a74cd99eb1444c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f711a544640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7114158130 0x7f7114161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f711a544640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71140ce170 0x7f711406b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f711a544640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711406f470 con 0x7f711406b9d0 2023-11-06T03:04:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f711a544640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711406be60 con 0x7f7114158130 2023-11-06T03:04:16.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f711a544640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7114071730 con 0x7f71140ce170 2023-11-06T03:04:16.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7118ae1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f711406b9d0 0x7f7114164b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43586/0 (socket says 172.21.15.157:43586) 2023-11-06T03:04:16.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7118ae1640 1 -- 172.21.15.157:0/2950602393 learned_addr learned my addr 172.21.15.157:0/2950602393 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 --2- 172.21.15.157:0/2950602393 >> v2:172.21.15.157:3300/0 conn(0x7f7114158130 0x7f7114161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 -- 172.21.15.157:0/2950602393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71140ce170 msgr2=0x7f711406b490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T03:04:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 --2- 172.21.15.157:0/2950602393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71140ce170 0x7f711406b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 -- 172.21.15.157:0/2950602393 >> v1:172.21.15.157:6789/0 conn(0x7f711406b9d0 legacy=0x7f7114164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 -- 172.21.15.157:0/2950602393 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711406d9a0 con 0x7f7114158130 2023-11-06T03:04:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f9038b25640 1 -- 172.21.15.157:0/2684957832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0a67a0 con 0x7f90340cd930 2023-11-06T03:04:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f9038b25640 1 -- 172.21.15.157:0/2684957832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c0a6900 con 0x7f90340cd930 2023-11-06T03:04:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f9038b25640 1 -- 172.21.15.157:0/2684957832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0a6c90 con 0x7f90340cd930 2023-11-06T03:04:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/243938442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58147450 0x7f4a58070d50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a580b10f0 tx=0x7f4a5001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2ee7783a040022 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 -- 172.21.15.157:0/2684957832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 msgr2=0x7f90340c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 --2- 172.21.15.157:0/2684957832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f90340c8b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f903406a900 tx=0x7f902c01d9f0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0100ce8f0 0x7ff01006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff010070480 0x7ff0100adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01006de80 con 0x7ff0100ad700 2023-11-06T03:04:16.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100ae0d0 con 0x7ff010070480 2023-11-06T03:04:16.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff017c62640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0100b2980 con 0x7ff0100ce8f0 2023-11-06T03:04:16.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff010070480 0x7ff0100adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff010070480 0x7ff0100adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43864/0 (socket says 172.21.15.157:43864) 2023-11-06T03:04:16.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 -- 172.21.15.157:0/1298665375 learned_addr learned my addr 172.21.15.157:0/1298665375 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 -- 172.21.15.157:0/1298665375 >> v2:172.21.15.195:3300/0 conn(0x7ff0100ce8f0 msgr2=0x7ff01006d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 --2- 172.21.15.157:0/1298665375 >> v2:172.21.15.195:3300/0 conn(0x7ff0100ce8f0 0x7ff01006d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 -- 172.21.15.157:0/1298665375 >> v1:172.21.15.157:6789/0 conn(0x7ff0100ad700 legacy=0x7ff01015c770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:16.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7ff0161ff640 1 -- 172.21.15.157:0/1298665375 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100b2c60 con 0x7ff010070480 2023-11-06T03:04:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 -- 172.21.15.157:0/2684957832 shutdown_connections 2023-11-06T03:04:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 --2- 172.21.15.157:0/2684957832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f90340c8b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 --2- 172.21.15.157:0/2684957832 >> v2:172.21.15.195:3300/0 conn(0x7f90340c9ee0 0x7f90340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 -- 172.21.15.157:0/2684957832 >> 172.21.15.157:0/2684957832 conn(0x7f90340cafa0 msgr2=0x7f90340cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 -- 172.21.15.157:0/2684957832 shutdown_connections 2023-11-06T03:04:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7431d9b640 1 -- 172.21.15.157:0/1052930272 >> v2:172.21.15.195:3300/0 conn(0x7f7434161740 msgr2=0x7f7434161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7431d9b640 1 --2- 172.21.15.157:0/1052930272 >> v2:172.21.15.195:3300/0 conn(0x7f7434161740 0x7f7434161b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7431d9b640 1 -- 172.21.15.157:0/1052930272 >> v1:172.21.15.195:6789/0 conn(0x7f74340afdd0 legacy=0x7f743406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.773+0000 7f7431d9b640 1 -- 172.21.15.157:0/1052930272 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340b2070 con 0x7f743406bcb0 2023-11-06T03:04:16.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a577fe640 1 -- 172.21.15.157:0/243938442 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500a5790 con 0x7f4a58147450 2023-11-06T03:04:16.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7431d9b640 1 --2- 172.21.15.157:0/1052930272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 0x7f74340b2590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74280097c0 tx=0x7f742801c310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b2873abed112b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a577fe640 1 -- 172.21.15.157:0/243938442 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500a58f0 con 0x7f4a58147450 2023-11-06T03:04:16.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a577fe640 1 -- 172.21.15.157:0/243938442 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500a5c80 con 0x7f4a58147450 2023-11-06T03:04:16.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 -- 172.21.15.157:0/243938442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58147450 msgr2=0x7f4a58070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/243938442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58147450 0x7f4a58070d50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a580b10f0 tx=0x7f4a5001c930 comp rx=0 tx=0).stop 2023-11-06T03:04:16.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f7113fff640 1 --2- 172.21.15.157:0/2950602393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7114158130 0x7f7114161fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71040097c0 tx=0x7f710401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9415ba1feed07b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 -- 172.21.15.157:0/243938442 shutdown_connections 2023-11-06T03:04:16.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/243938442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a58065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/243938442 >> v2:172.21.15.195:3300/0 conn(0x7f4a580cf730 0x7f4a580cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/243938442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58147450 0x7f4a58070d50 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a580b10f0 tx=0x7f4a5001c930 comp rx=0 tx=0).stop 2023-11-06T03:04:16.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f4a5ed72640 1 -- 172.21.15.157:0/243938442 >> 172.21.15.157:0/243938442 conn(0x7f4a58069d70 msgr2=0x7f4a5806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f71137fe640 1 -- 172.21.15.157:0/2950602393 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040ac640 con 0x7f7114158130 2023-11-06T03:04:16.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f71137fe640 1 -- 172.21.15.157:0/2950602393 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71040ac7a0 con 0x7f7114158130 2023-11-06T03:04:16.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f743159a640 1 -- 172.21.15.157:0/1052930272 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280ab600 con 0x7f743406bcb0 2023-11-06T03:04:16.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f71137fe640 1 -- 172.21.15.157:0/2950602393 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040acb30 con 0x7f7114158130 2023-11-06T03:04:16.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7fe8bffff640 1 --2- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0159a40 0x7fe8c00ca030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b801d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53202f657fb73bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f743159a640 1 -- 172.21.15.157:0/1052930272 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280a7050 con 0x7f743406bcb0 2023-11-06T03:04:16.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f743159a640 1 -- 172.21.15.157:0/1052930272 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280ab980 con 0x7f743406bcb0 2023-11-06T03:04:16.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8beffd640 1 -- 172.21.15.157:0/1130543591 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80a58e0 con 0x7fe8c0159a40 2023-11-06T03:04:16.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- 172.21.15.157:0/2950602393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7114158130 msgr2=0x7f7114161fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 --2- 172.21.15.157:0/2950602393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7114158130 0x7f7114161fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71040097c0 tx=0x7f710401c3e0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- 172.21.15.157:0/1052930272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 msgr2=0x7f74340b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- 172.21.15.157:0/1052930272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 0x7f74340b2590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74280097c0 tx=0x7f742801c310 comp rx=0 tx=0).stop 2023-11-06T03:04:16.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- 172.21.15.157:0/1052930272 shutdown_connections 2023-11-06T03:04:16.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- 172.21.15.157:0/1052930272 >> v2:172.21.15.195:3300/0 conn(0x7f7434161740 0x7f7434161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- 172.21.15.157:0/1052930272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f743406bcb0 0x7f74340b2590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- 172.21.15.157:0/1052930272 >> 172.21.15.157:0/1052930272 conn(0x7f7434065b00 msgr2=0x7f7434065f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- 172.21.15.157:0/1052930272 shutdown_connections 2023-11-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- 172.21.15.157:0/2950602393 shutdown_connections 2023-11-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 --2- 172.21.15.157:0/2950602393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71140ce170 0x7f711406b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 --2- 172.21.15.157:0/2950602393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7114158130 0x7f7114161fa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- 172.21.15.157:0/2950602393 >> 172.21.15.157:0/2950602393 conn(0x7f7114065d30 msgr2=0x7f7114066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- 172.21.15.157:0/1052930272 wait complete. 2023-11-06T03:04:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8beffd640 1 -- 172.21.15.157:0/1130543591 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80a5a40 con 0x7fe8c0159a40 2023-11-06T03:04:16.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8beffd640 1 -- 172.21.15.157:0/1130543591 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80a5db0 con 0x7fe8c0159a40 2023-11-06T03:04:16.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.777+0000 7f903ab29640 1 -- 172.21.15.157:0/2684957832 wait complete. 2023-11-06T03:04:16.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f4a5ed72640 1 -- 172.21.15.157:0/243938442 shutdown_connections 2023-11-06T03:04:16.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 Processor -- start 2023-11-06T03:04:16.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f5228e0e640 1 Processor -- start 2023-11-06T03:04:16.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f5228e0e640 1 -- start start 2023-11-06T03:04:16.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 -- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0159a40 msgr2=0x7fe8c00ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 --2- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0159a40 0x7fe8c00ca030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009d90 tx=0x7fe8b801d2e0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 -- 172.21.15.157:0/1130543591 shutdown_connections 2023-11-06T03:04:16.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 --2- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c00ceab0 0x7fe8c00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 --2- 172.21.15.157:0/1130543591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c0159a40 0x7fe8c00ca030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 -- 172.21.15.157:0/1130543591 >> 172.21.15.157:0/1130543591 conn(0x7fe8c0065930 msgr2=0x7fe8c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7ff0161ff640 1 --2- 172.21.15.157:0/1298665375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff010070480 0x7ff0100adb90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e50 tx=0x7ff00801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd36b0442e29be0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 -- 172.21.15.157:0/1130543591 shutdown_connections 2023-11-06T03:04:16.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fe8c6923640 1 -- 172.21.15.157:0/1130543591 wait complete. 2023-11-06T03:04:16.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f4a5ed72640 1 -- 172.21.15.157:0/243938442 wait complete. 2023-11-06T03:04:16.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 Processor -- start 2023-11-06T03:04:16.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f4a5ed72640 1 Processor -- start 2023-11-06T03:04:16.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- 172.21.15.157:0/2950602393 shutdown_connections 2023-11-06T03:04:16.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- 172.21.15.157:0/2950602393 wait complete. 2023-11-06T03:04:16.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f522259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f522415cc70 0x7f52240d2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43602/0 (socket says 172.21.15.157:43602) 2023-11-06T03:04:16.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 -- start start 2023-11-06T03:04:16.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f522259c640 1 -- 172.21.15.157:0/3445360693 learned_addr learned my addr 172.21.15.157:0/3445360693 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 0x7f90342033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 0x7f9034150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f9034150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90340b3c60 con 0x7f90340cd930 2023-11-06T03:04:16.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90342038f0 con 0x7f90340c9b00 2023-11-06T03:04:16.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903ab29640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90340b2fe0 con 0x7f90340c9ee0 2023-11-06T03:04:16.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 Processor -- start 2023-11-06T03:04:16.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 Processor -- start 2023-11-06T03:04:16.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 -- start start 2023-11-06T03:04:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 0x7fc5a40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40cdd00 con 0x7fc5a40ccb70 2023-11-06T03:04:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40cd0f0 con 0x7fc5a40b41b0 2023-11-06T03:04:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aaec7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40cd3d0 con 0x7fc5a40b06b0 2023-11-06T03:04:16.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7ff0149fc640 1 -- 172.21.15.157:0/1298665375 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080a5760 con 0x7ff010070480 2023-11-06T03:04:16.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- start start 2023-11-06T03:04:16.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7ff0149fc640 1 -- 172.21.15.157:0/1298665375 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080a58c0 con 0x7ff010070480 2023-11-06T03:04:16.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7ff0149fc640 1 -- 172.21.15.157:0/1298665375 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080a5c50 con 0x7ff010070480 2023-11-06T03:04:16.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aa6c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f9039b27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 0x7f90342033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f9039326640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 0x7f9034150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f9039b27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 0x7f90342033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43716/0 (socket says 172.21.15.157:43716) 2023-11-06T03:04:16.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f9039b27640 1 -- 172.21.15.157:0/846877503 learned_addr learned my addr 172.21.15.157:0/846877503 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aa6c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54130/0 (socket says 172.21.15.157:54130) 2023-11-06T03:04:16.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5aa6c6640 1 -- 172.21.15.157:0/515385702 learned_addr learned my addr 172.21.15.157:0/515385702 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f743406bcb0 0x7f74340ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7434161740 0x7f743415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340cec10 con 0x7f743406bcb0 2023-11-06T03:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340cf1f0 con 0x7f74340afdd0 2023-11-06T03:04:16.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f7438cf2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340cdbf0 con 0x7f7434161740 2023-11-06T03:04:16.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f5228e0e640 1 --2- 172.21.15.157:0/3445360693 >> v2:172.21.15.157:3300/0 conn(0x7f52240d00a0 0x7f522415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f903a328640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f9034150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5222d9d640 1 --2- 172.21.15.157:0/3445360693 >> v2:172.21.15.157:3300/0 conn(0x7f52240d00a0 0x7f522415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fc5aa6c6640 1 -- 172.21.15.157:0/515385702 >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 msgr2=0x7fc5a40cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 Processor -- start 2023-11-06T03:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43720/0 (socket says 172.21.15.157:43720) 2023-11-06T03:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 -- 172.21.15.157:0/4058383412 learned_addr learned my addr 172.21.15.157:0/4058383412 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5224160830 con 0x7f522415cc70 2023-11-06T03:04:16.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 -- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 msgr2=0x7f90342033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 0x7f90342033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 -- 172.21.15.157:0/846877503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 msgr2=0x7f9034150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f9034150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 -- 172.21.15.157:0/846877503 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009b30 con 0x7f90340c9ee0 2023-11-06T03:04:16.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f9039326640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 0x7f9034150450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f903000c7f0 tx=0x7f903000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- 172.21.15.157:0/1298665375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff010070480 msgr2=0x7ff0100adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 --2- 172.21.15.157:0/1298665375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff010070480 0x7ff0100adb90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e50 tx=0x7ff00801c980 comp rx=0 tx=0).stop 2023-11-06T03:04:16.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- 172.21.15.157:0/1298665375 shutdown_connections 2023-11-06T03:04:16.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 --2- 172.21.15.157:0/1298665375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff010070480 0x7ff0100adb90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 --2- 172.21.15.157:0/1298665375 >> v2:172.21.15.195:3300/0 conn(0x7ff0100ce8f0 0x7ff01006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- 172.21.15.157:0/1298665375 >> 172.21.15.157:0/1298665375 conn(0x7ff010065d30 msgr2=0x7ff010066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- 172.21.15.157:0/1298665375 shutdown_connections 2023-11-06T03:04:16.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- 172.21.15.157:0/1298665375 wait complete. 2023-11-06T03:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f711a544640 1 -- start start 2023-11-06T03:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 0x7f7114156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 0x7f71141543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7114158130 0x7f7114154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7114160560 con 0x7f71140ce170 2023-11-06T03:04:16.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7114160980 con 0x7f7114158130 2023-11-06T03:04:16.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71141f9ca0 con 0x7f711406b9d0 2023-11-06T03:04:16.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7113fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 0x7f7114156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f743359e640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7434161740 0x7f743415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f743259c640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f743406bcb0 0x7f74340ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7f4a5ed72640 1 -- start start 2023-11-06T03:04:16.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a580cf730 0x7f4a581499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58145360 0x7f4a58147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a581464a0 con 0x7f4a58145360 2023-11-06T03:04:16.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58145920 con 0x7f4a580cf730 2023-11-06T03:04:16.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5ed72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58145c00 con 0x7f4a580cbee0 2023-11-06T03:04:16.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7113fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 0x7f7114156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54146/0 (socket says 172.21.15.157:54146) 2023-11-06T03:04:16.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7113fff640 1 -- 172.21.15.157:0/3313489539 learned_addr learned my addr 172.21.15.157:0/3313489539 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5224160b10 con 0x7f52240cde90 2023-11-06T03:04:16.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52241614b0 con 0x7f52240d00a0 2023-11-06T03:04:16.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f522159a640 1 -- 172.21.15.157:0/3445360693 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168770817 0 0) 0x7f5224160830 con 0x7f522415cc70 2023-11-06T03:04:16.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7118ae1640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7114158130 0x7f7114154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 0x7f71141543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f522159a640 1 -- 172.21.15.157:0/3445360693 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f520c003530 con 0x7f522415cc70 2023-11-06T03:04:16.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030004220 con 0x7f90340c9ee0 2023-11-06T03:04:16.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 -- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7434161740 msgr2=0x7f743415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7434161740 0x7f743415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 -- 172.21.15.157:0/4058383412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f743406bcb0 msgr2=0x7f74340ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f743406bcb0 0x7f74340ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 -- 172.21.15.157:0/4058383412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009420 con 0x7f74340afdd0 2023-11-06T03:04:16.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9030004380 con 0x7f90340c9ee0 2023-11-06T03:04:16.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030003840 con 0x7f90340c9ee0 2023-11-06T03:04:16.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 -- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 msgr2=0x7f7114156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 0x7f7114156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 -- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7114158130 msgr2=0x7f7114154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7114158130 0x7f7114154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 -- 172.21.15.157:0/3313489539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009420 con 0x7f71140ce170 2023-11-06T03:04:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54148/0 (socket says 172.21.15.157:54148) 2023-11-06T03:04:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/2759812457 learned_addr learned my addr 172.21.15.157:0/2759812457 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a57fff640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a580cf730 0x7f4a581499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5d30f640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58145360 0x7f4a58147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 -- start start 2023-11-06T03:04:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 0x7fe8c01f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c00ceab0 0x7fe8c0158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c015b560 con 0x7fe8c00b3680 2023-11-06T03:04:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0158eb0 con 0x7fe8c00ceab0 2023-11-06T03:04:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c6923640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01ee1a0 con 0x7fe8c0159a40 2023-11-06T03:04:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7431d9b640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f742801a9c0 tx=0x7f74280190b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90340b32c0 con 0x7f90340c9ee0 2023-11-06T03:04:16.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f710b7fe640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 0x7f71141543e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00b3c0 tx=0x7f70fc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90340b3880 con 0x7f90340c9ee0 2023-11-06T03:04:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5222d9d640 1 -- 172.21.15.157:0/3445360693 >> v1:172.21.15.195:6789/0 conn(0x7f52240cde90 legacy=0x7f52240cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5222d9d640 1 -- 172.21.15.157:0/3445360693 >> v1:172.21.15.157:6789/0 conn(0x7f522415cc70 legacy=0x7f52240d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f5222d9d640 1 -- 172.21.15.157:0/3445360693 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224160df0 con 0x7f52240d00a0 2023-11-06T03:04:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 Processor -- start 2023-11-06T03:04:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011830 con 0x7f71140ce170 2023-11-06T03:04:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc011990 con 0x7f71140ce170 2023-11-06T03:04:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711415fbc0 con 0x7f71140ce170 2023-11-06T03:04:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011d40 con 0x7f71140ce170 2023-11-06T03:04:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7114160180 con 0x7f71140ce170 2023-11-06T03:04:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7ff017c62640 1 -- start start 2023-11-06T03:04:16.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.781+0000 7fc5a9ec5640 1 --2- 172.21.15.157:0/515385702 >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 0x7fc5a40cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fc5aa6c6640 1 --2- 172.21.15.157:0/515385702 >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 0x7fc5a40cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fc5aa6c6640 1 -- 172.21.15.157:0/515385702 >> v1:172.21.15.195:6789/0 conn(0x7fc5a40ccb70 legacy=0x7fc5a40affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fc5a8ec3640 1 -- 172.21.15.157:0/515385702 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939486459 0 0) 0x7fc5a40cdd00 con 0x7fc5a40ccb70 2023-11-06T03:04:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fc5aa6c6640 1 -- 172.21.15.157:0/515385702 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40cd6b0 con 0x7fc5a40b06b0 2023-11-06T03:04:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c4ec0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 0x7fe8c01f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c4ec0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54152/0 (socket says 172.21.15.157:54152) 2023-11-06T03:04:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8c4ec0640 1 -- 172.21.15.157:0/3387611536 learned_addr learned my addr 172.21.15.157:0/3387611536 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8bffff640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 0x7fe8c01f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43936/0 (socket says 172.21.15.157:43936) 2023-11-06T03:04:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7fe8bf7fe640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c00ceab0 0x7fe8c0158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a580cf730 msgr2=0x7f4a581499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a580cf730 0x7f4a581499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/2759812457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58145360 msgr2=0x7f4a58147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58145360 0x7f4a58147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280b4610 con 0x7f74340afdd0 2023-11-06T03:04:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280abe90 con 0x7f74340afdd0 2023-11-06T03:04:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f4a5cb0e640 1 -- 172.21.15.157:0/2759812457 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009a70 con 0x7f4a580cbee0 2023-11-06T03:04:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9034072670 con 0x7f90340c9ee0 2023-11-06T03:04:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.785+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280b49d0 con 0x7f74340afdd0 2023-11-06T03:04:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5222d9d640 1 --2- 172.21.15.157:0/3445360693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52240d00a0 0x7f522415d050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c01f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae2834beeaa97ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 0x7ff010135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0100ce8f0 0x7ff010154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54154/0 (socket says 172.21.15.157:54154) 2023-11-06T03:04:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0151fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 0x7ff010135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0151fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 0x7ff010135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43744/0 (socket says 172.21.15.157:43744) 2023-11-06T03:04:16.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 -- 172.21.15.157:0/3364627379 learned_addr learned my addr 172.21.15.157:0/3364627379 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010154050 con 0x7ff0100ce8f0 2023-11-06T03:04:16.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fc5a9ec5640 1 --2- 172.21.15.157:0/515385702 >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 0x7fc5a40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:16.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f8005260 con 0x7f71140ce170 2023-11-06T03:04:16.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 -- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c00ceab0 msgr2=0x7fe8c0158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c00ceab0 0x7fe8c0158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 -- 172.21.15.157:0/3387611536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 msgr2=0x7fe8c01f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 0x7fe8c01f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 -- 172.21.15.157:0/3387611536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009a70 con 0x7fe8c0159a40 2023-11-06T03:04:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340cded0 con 0x7f74340afdd0 2023-11-06T03:04:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434152760 con 0x7f74340afdd0 2023-11-06T03:04:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a5cb0e640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001d1d0 tx=0x7f4a500196d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f522159a640 1 -- 172.21.15.157:0/3445360693 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0af600 con 0x7f52240d00a0 2023-11-06T03:04:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0161ff640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0100ce8f0 0x7ff010154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010135850 con 0x7ff0100ad700 2023-11-06T03:04:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101533d0 con 0x7ff010070480 2023-11-06T03:04:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f522159a640 1 -- 172.21.15.157:0/3445360693 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c0ab050 con 0x7f52240d00a0 2023-11-06T03:04:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f522159a640 1 -- 172.21.15.157:0/3445360693 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0af960 con 0x7f52240d00a0 2023-11-06T03:04:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500bd070 con 0x7f4a580cbee0 2023-11-06T03:04:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fe8c4ec0640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000e850 tx=0x7fe8b000ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a58145ee0 con 0x7f4a580cbee0 2023-11-06T03:04:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a58154900 con 0x7f4a580cbee0 2023-11-06T03:04:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500ad440 con 0x7f4a580cbee0 2023-11-06T03:04:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500ad7b0 con 0x7f4a580cbee0 2023-11-06T03:04:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74340ce3f0 con 0x7f74340afdd0 2023-11-06T03:04:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7fc5aa6c6640 1 --2- 172.21.15.157:0/515385702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc594009e50 tx=0x7fc59401f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65f70e44e5efbef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 -- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 msgr2=0x7ff010135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 0x7ff010135310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 -- 172.21.15.157:0/3364627379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0100ce8f0 msgr2=0x7ff010154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0100ce8f0 0x7ff010154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009b30 con 0x7ff010070480 2023-11-06T03:04:16.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f70fc01b460 con 0x7f71140ce170 2023-11-06T03:04:16.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52240d00a0 msgr2=0x7f522415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 --2- 172.21.15.157:0/3445360693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52240d00a0 0x7f522415d050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c01f120 comp rx=0 tx=0).stop 2023-11-06T03:04:16.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 shutdown_connections 2023-11-06T03:04:16.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 --2- 172.21.15.157:0/3445360693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52240d00a0 0x7f522415d050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 >> 172.21.15.157:0/3445360693 conn(0x7f5224070560 msgr2=0x7f5224070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 shutdown_connections 2023-11-06T03:04:16.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- 172.21.15.157:0/3445360693 wait complete. 2023-11-06T03:04:16.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f71117fa640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 0x7f70f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9030003bb0 con 0x7f90340c9ee0 2023-11-06T03:04:16.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70fc082f50 con 0x7f71140ce170 2023-11-06T03:04:16.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7ff0159fe640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b4d0 tx=0x7ff00c00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a58072750 con 0x7f4a580cbee0 2023-11-06T03:04:16.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f7113fff640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 0x7f70f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f902a7fc640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 0x7f9018067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9030083750 con 0x7f90340c9ee0 2023-11-06T03:04:16.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f9039b27640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 0x7f9018067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 Processor -- start 2023-11-06T03:04:16.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f5228e0e640 1 -- start start 2023-11-06T03:04:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.789+0000 7f7113fff640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 0x7f70f4067f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f710401c420 tx=0x7f71040a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f9039b27640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 0x7f9018067f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90240050b0 tx=0x7f9024003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52240cde90 0x7f522415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52240d00a0 0x7f5224158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240b3560 con 0x7f522415cc70 2023-11-06T03:04:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240c84d0 con 0x7f52240cde90 2023-11-06T03:04:16.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522415eb50 con 0x7f52240d00a0 2023-11-06T03:04:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43956/0 (socket says 172.21.15.157:43956) 2023-11-06T03:04:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 -- 172.21.15.157:0/2955352569 learned_addr learned my addr 172.21.15.157:0/2955352569 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f522259c640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52240cde90 0x7f522415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5221d9b640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52240d00a0 0x7f5224158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f74280b2040 con 0x7f74340afdd0 2023-11-06T03:04:16.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 -- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52240d00a0 msgr2=0x7f5224158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52240d00a0 0x7f5224158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 -- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52240cde90 msgr2=0x7f522415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52240cde90 0x7f522415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 -- 172.21.15.157:0/2955352569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c009a70 con 0x7f522415cc70 2023-11-06T03:04:16.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5222d9d640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f521c01cd50 tx=0x7f521c01bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f7422ffd640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 0x7f7414067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f742811d9b0 con 0x7f74340afdd0 2023-11-06T03:04:16.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f743259c640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 0x7f7414067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f743259c640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 0x7f7414067f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7424000f70 tx=0x7f7424007e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0ba680 con 0x7f522415cc70 2023-11-06T03:04:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 -- 172.21.15.157:0/2955352569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522415ee30 con 0x7f522415cc70 2023-11-06T03:04:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5228e0e640 1 -- 172.21.15.157:0/2955352569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240c7880 con 0x7f522415cc70 2023-11-06T03:04:16.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70fc0507c0 con 0x7f71140ce170 2023-11-06T03:04:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.793+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c009450 con 0x7f522415cc70 2023-11-06T03:04:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0ba9e0 con 0x7f522415cc70 2023-11-06T03:04:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f5228e0e640 1 -- 172.21.15.157:0/2955352569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5208005260 con 0x7f522415cc70 2023-11-06T03:04:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f521c0b6040 con 0x7f522415cc70 2023-11-06T03:04:16.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f5212ffd640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 0x7f5204067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f521c121ea0 con 0x7f522415cc70 2023-11-06T03:04:16.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f522259c640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 0x7f5204067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.797+0000 7f522259c640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 0x7f5204067f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52140011f0 tx=0x7f5214007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.801+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f521c0ef7b0 con 0x7f522415cc70 2023-11-06T03:04:16.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c010b70 con 0x7ff010070480 2023-11-06T03:04:16.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5a8ec3640 1 -- 172.21.15.157:0/515385702 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940af400 con 0x7fc5a40b06b0 2023-11-06T03:04:16.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0009660 con 0x7fe8c0159a40 2023-11-06T03:04:16.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0101536b0 con 0x7ff010070480 2023-11-06T03:04:16.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c015af10 con 0x7fe8c0159a40 2023-11-06T03:04:16.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9030051040 con 0x7f90340c9ee0 2023-11-06T03:04:16.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010153c70 con 0x7ff010070480 2023-11-06T03:04:16.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c01ee480 con 0x7fe8c0159a40 2023-11-06T03:04:16.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f74280eb430 con 0x7f74340afdd0 2023-11-06T03:04:16.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b000c940 con 0x7fe8c0159a40 2023-11-06T03:04:16.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000ccb0 con 0x7fe8c0159a40 2023-11-06T03:04:16.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5a8ec3640 1 -- 172.21.15.157:0/515385702 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5940ab050 con 0x7fc5a40b06b0 2023-11-06T03:04:16.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5a8ec3640 1 -- 172.21.15.157:0/515385702 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940af780 con 0x7fc5a40b06b0 2023-11-06T03:04:16.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c0092c0 con 0x7ff010070480 2023-11-06T03:04:16.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c022440 con 0x7ff010070480 2023-11-06T03:04:16.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a0005260 con 0x7fe8c0159a40 2023-11-06T03:04:16.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 -- 172.21.15.157:0/515385702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 msgr2=0x7fc5a40b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 --2- 172.21.15.157:0/515385702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc594009e50 tx=0x7fc59401f210 comp rx=0 tx=0).stop 2023-11-06T03:04:16.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 -- 172.21.15.157:0/515385702 shutdown_connections 2023-11-06T03:04:16.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 --2- 172.21.15.157:0/515385702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40b06b0 0x7fc5a40b1650 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 --2- 172.21.15.157:0/515385702 >> v2:172.21.15.157:3300/0 conn(0x7fc5a40b41b0 0x7fc5a40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7fc5aaec7640 1 -- 172.21.15.157:0/515385702 >> 172.21.15.157:0/515385702 conn(0x7fc5a40cafa0 msgr2=0x7fc5a40cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:16.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.805+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8005260 con 0x7ff010070480 2023-11-06T03:04:16.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe8b00059a0 con 0x7fe8c0159a40 2023-11-06T03:04:16.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fe8bcff9640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 0x7fe8ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a500b4070 con 0x7f4a580cbee0 2023-11-06T03:04:16.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff00c0225a0 con 0x7ff010070480 2023-11-06T03:04:16.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7f4a557fa640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 0x7f4a44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a5011e040 con 0x7f4a580cbee0 2023-11-06T03:04:16.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7ff0067fc640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 0x7feff4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fc5aaec7640 1 -- 172.21.15.157:0/515385702 shutdown_connections 2023-11-06T03:04:16.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fc5aaec7640 1 -- 172.21.15.157:0/515385702 wait complete. 2023-11-06T03:04:16.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7f4a57fff640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 0x7f4a44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7ff0151fd640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 0x7feff4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fe8bffff640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 0x7fe8ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7f4a57fff640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 0x7f4a44067f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a58163d00 tx=0x7f4a4c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7ff0151fd640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 0x7feff4067f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0100cdad0 tx=0x7ff000009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fe8bffff640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 0x7fe8ac067f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80af000 tx=0x7fe8b8004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fc5aaec7640 1 Processor -- start 2023-11-06T03:04:16.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.809+0000 7fc5aaec7640 1 -- start start 2023-11-06T03:04:16.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a40b06b0 0x7fc5a40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40ccb70 0x7fc5a4163c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b2b00 con 0x7fc5a40b41b0 2023-11-06T03:04:16.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b1ef0 con 0x7fc5a40b06b0 2023-11-06T03:04:16.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b21d0 con 0x7fc5a40ccb70 2023-11-06T03:04:16.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43958/0 (socket says 172.21.15.157:43958) 2023-11-06T03:04:16.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 -- 172.21.15.157:0/424192827 learned_addr learned my addr 172.21.15.157:0/424192827 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:16.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aa6c6640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40ccb70 0x7fc5a4163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a9ec5640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a40b06b0 0x7fc5a40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 -- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40ccb70 msgr2=0x7fc5a4163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40ccb70 0x7fc5a4163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 -- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a40b06b0 msgr2=0x7fc5a40b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a40b06b0 0x7fc5a40b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 -- 172.21.15.157:0/424192827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594009b30 con 0x7fc5a40b41b0 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a96c4640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc598001ec0 tx=0x7fc59800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980054e0 con 0x7fc5a40b41b0 2023-11-06T03:04:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598003600 con 0x7fc5a40b41b0 2023-11-06T03:04:16.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 -- 172.21.15.157:0/424192827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a40b24b0 con 0x7fc5a40b41b0 2023-11-06T03:04:16.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980039b0 con 0x7fc5a40b41b0 2023-11-06T03:04:16.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5aaec7640 1 -- 172.21.15.157:0/424192827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4147240 con 0x7fc5a40b41b0 2023-11-06T03:04:16.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc598003bd0 con 0x7fc5a40b41b0 2023-11-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.813+0000 7fc5a27fc640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 0x7fc590067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.817+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc580005260 con 0x7fc5a40b41b0 2023-11-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.817+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc598082600 con 0x7fc5a40b41b0 2023-11-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.817+0000 7fc5a9ec5640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 0x7fc590067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.817+0000 7fc5a9ec5640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 0x7fc590067f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00a9a0 tx=0x7fc59c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:16.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.821+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8b00552a0 con 0x7fe8c0159a40 2023-11-06T03:04:16.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.821+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff00c058610 con 0x7ff010070480 2023-11-06T03:04:16.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.821+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a500eb9c0 con 0x7f4a580cbee0 2023-11-06T03:04:16.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.829+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc598017020 con 0x7fc5a40b41b0 2023-11-06T03:04:16.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.833+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe8b00515e0 con 0x7fe8c0159a40 2023-11-06T03:04:16.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:16.841+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff00c054950 con 0x7ff010070480 2023-11-06T03:04:17.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.077+0000 7f5228e0e640 1 -- 172.21.15.157:0/2955352569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5208005510 con 0x7f522415cc70 2023-11-06T03:04:17.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.077+0000 7f5212ffd640 1 -- 172.21.15.157:0/2955352569 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f521c0b4090 con 0x7f522415cc70 2023-11-06T03:04:17.079 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.077+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f70f8005510 con 0x7f71140ce170 2023-11-06T03:04:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.077+0000 7f71117fa640 1 -- 172.21.15.157:0/3313489539 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70fc053de0 con 0x7f71140ce170 2023-11-06T03:04:17.082 INFO:teuthology.orchestra.run.smithi157.stdout:55834574856 2023-11-06T03:04:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 msgr2=0x7f5204067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 0x7f5204067f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52140011f0 tx=0x7f5214007d40 comp rx=0 tx=0).stop 2023-11-06T03:04:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 msgr2=0x7f52240b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f521c01cd50 tx=0x7f521c01bec0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 shutdown_connections 2023-11-06T03:04:17.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5204065a60 0x7f5204067f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f522415cc70 0x7f52240b3020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52240d00a0 0x7f5224158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 --2- 172.21.15.157:0/2955352569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f52240cde90 0x7f522415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 >> 172.21.15.157:0/2955352569 conn(0x7f5224070560 msgr2=0x7f52240b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 shutdown_connections 2023-11-06T03:04:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f5210ff9640 1 -- 172.21.15.157:0/2955352569 wait complete. 2023-11-06T03:04:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 msgr2=0x7f70f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 0x7f70f4067f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f710401c420 tx=0x7f71040a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 msgr2=0x7f71141543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 0x7f71141543e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00b3c0 tx=0x7f70fc00b880 comp rx=0 tx=0).stop 2023-11-06T03:04:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 shutdown_connections 2023-11-06T03:04:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70f4065a60 0x7f70f4067f20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7114158130 0x7f7114154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71140ce170 0x7f71141543e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 --2- 172.21.15.157:0/3313489539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f711406b9d0 0x7f7114156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 >> 172.21.15.157:0/3313489539 conn(0x7f7114065d30 msgr2=0x7f7114162980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 shutdown_connections 2023-11-06T03:04:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.081+0000 7f711a544640 1 -- 172.21.15.157:0/3313489539 wait complete. 2023-11-06T03:04:17.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-11-06T03:04:17.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-11-06T03:04:17.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.109+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90340cc640 con 0x7f90340c9ee0 2023-11-06T03:04:17.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.109+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f74340ce3f0 con 0x7f74340afdd0 2023-11-06T03:04:17.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.109+0000 7f902a7fc640 1 -- 172.21.15.157:0/846877503 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f903001e090 con 0x7f90340c9ee0 2023-11-06T03:04:17.113 INFO:teuthology.orchestra.run.smithi157.stdout:55834574853 2023-11-06T03:04:17.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.109+0000 7f7422ffd640 1 -- 172.21.15.157:0/4058383412 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74340ce3f0 con 0x7f74340afdd0 2023-11-06T03:04:17.113 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:17.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 msgr2=0x7f9018067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 0x7f9018067f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90240050b0 tx=0x7f9024003700 comp rx=0 tx=0).stop 2023-11-06T03:04:17.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 msgr2=0x7f7414067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 msgr2=0x7f9034150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 0x7f9034150450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f903000c7f0 tx=0x7f903000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 0x7f7414067f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7424000f70 tx=0x7f7424007e10 comp rx=0 tx=0).stop 2023-11-06T03:04:17.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 msgr2=0x7f7434152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f742801a9c0 tx=0x7f74280190b0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 shutdown_connections 2023-11-06T03:04:17.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9018065a60 0x7f9018067f20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90340cd930 0x7f9034150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90340c9ee0 0x7f9034150450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 --2- 172.21.15.157:0/846877503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90340c9b00 0x7f90342033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 shutdown_connections 2023-11-06T03:04:17.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7414065ab0 0x7f7414067f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7434161740 0x7f743415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74340afdd0 0x7f7434152220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 --2- 172.21.15.157:0/4058383412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f743406bcb0 0x7f74340ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 >> 172.21.15.157:0/4058383412 conn(0x7f7434065b00 msgr2=0x7f7434163230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 >> 172.21.15.157:0/846877503 conn(0x7f90340cafa0 msgr2=0x7f903415ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 shutdown_connections 2023-11-06T03:04:17.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f7438cf2640 1 -- 172.21.15.157:0/4058383412 wait complete. 2023-11-06T03:04:17.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 shutdown_connections 2023-11-06T03:04:17.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.113+0000 7f903ab29640 1 -- 172.21.15.157:0/846877503 wait complete. 2023-11-06T03:04:17.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.125+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a581445a0 con 0x7f4a580cbee0 2023-11-06T03:04:17.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.125+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe8a00057e0 con 0x7fe8c0159a40 2023-11-06T03:04:17.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.125+0000 7f4a557fa640 1 -- 172.21.15.157:0/2759812457 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a500b2020 con 0x7f4a580cbee0 2023-11-06T03:04:17.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.125+0000 7fe8bcff9640 1 -- 172.21.15.157:0/3387611536 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe8b0021020 con 0x7fe8c0159a40 2023-11-06T03:04:17.130 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:17.130 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:17.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 msgr2=0x7fe8ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 0x7fe8ac067f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80af000 tx=0x7fe8b8004ac0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 msgr2=0x7fe8c00af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000e850 tx=0x7fe8b000ed10 comp rx=0 tx=0).stop 2023-11-06T03:04:17.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 msgr2=0x7f4a44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 0x7f4a44067f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a58163d00 tx=0x7f4a4c009190 comp rx=0 tx=0).stop 2023-11-06T03:04:17.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 msgr2=0x7f4a581543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001d1d0 tx=0x7f4a500196d0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 shutdown_connections 2023-11-06T03:04:17.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8ac065a60 0x7fe8ac067f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0159a40 0x7fe8c00af200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c00ceab0 0x7fe8c0158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 --2- 172.21.15.157:0/3387611536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8c00b3680 0x7fe8c01f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 >> 172.21.15.157:0/3387611536 conn(0x7fe8c0065930 msgr2=0x7fe8c00ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 shutdown_connections 2023-11-06T03:04:17.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a44065a60 0x7f4a44067f20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a58145360 0x7f4a58147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a580cf730 0x7f4a581499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 --2- 172.21.15.157:0/2759812457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a580cbee0 0x7f4a581543c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 >> 172.21.15.157:0/2759812457 conn(0x7f4a58069d70 msgr2=0x7f4a5806ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 shutdown_connections 2023-11-06T03:04:17.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7f4a5ed72640 1 -- 172.21.15.157:0/2759812457 wait complete. 2023-11-06T03:04:17.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 shutdown_connections 2023-11-06T03:04:17.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.129+0000 7fe8c6923640 1 -- 172.21.15.157:0/3387611536 wait complete. 2023-11-06T03:04:17.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.133+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feff80057e0 con 0x7ff010070480 2023-11-06T03:04:17.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.133+0000 7ff0067fc640 1 -- 172.21.15.157:0/3364627379 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff00c01d070 con 0x7ff010070480 2023-11-06T03:04:17.140 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:17.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 msgr2=0x7feff4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 0x7feff4067f20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0100cdad0 tx=0x7ff000009060 comp rx=0 tx=0).stop 2023-11-06T03:04:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 msgr2=0x7ff010152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b4d0 tx=0x7ff00c00b990 comp rx=0 tx=0).stop 2023-11-06T03:04:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 shutdown_connections 2023-11-06T03:04:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feff4065a60 0x7feff4067f20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0100ce8f0 0x7ff010154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0100ad700 0x7ff010135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 --2- 172.21.15.157:0/3364627379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff010070480 0x7ff010152dd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 >> 172.21.15.157:0/3364627379 conn(0x7ff010065d30 msgr2=0x7ff0100ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 shutdown_connections 2023-11-06T03:04:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7ff017c62640 1 -- 172.21.15.157:0/3364627379 wait complete. 2023-11-06T03:04:17.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-11-06T03:04:17.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc5800057e0 con 0x7fc5a40b41b0 2023-11-06T03:04:17.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc5a27fc640 1 -- 172.21.15.157:0/424192827 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc59804fe00 con 0x7fc5a40b41b0 2023-11-06T03:04:17.145 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:17.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-06T03:04:17.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 msgr2=0x7fc590067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 0x7fc590067f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00a9a0 tx=0x7fc59c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:17.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 msgr2=0x7fc5a41636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:17.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc598001ec0 tx=0x7fc59800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 shutdown_connections 2023-11-06T03:04:17.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc590065a60 0x7fc590067f20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5a40ccb70 0x7fc5a4163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5a40b41b0 0x7fc5a41636c0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc598001ec0 tx=0x7fc59800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 --2- 172.21.15.157:0/424192827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5a40b06b0 0x7fc5a40b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:17.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 >> 172.21.15.157:0/424192827 conn(0x7fc5a40cafa0 msgr2=0x7fc5a41501b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:17.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 shutdown_connections 2023-11-06T03:04:17.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:17.141+0000 7fc58bfff640 1 -- 172.21.15.157:0/424192827 wait complete. 2023-11-06T03:04:17.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-11-06T03:04:17.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-06T03:04:17.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-06T03:04:17.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-11-06T03:04:18.111 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-06T03:04:18.112 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-06T03:04:18.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-06T03:04:18.147 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-06T03:04:18.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-06T03:04:18.165 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-06T03:04:18.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-06T03:04:18.168 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-06T03:04:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.257+0000 7efd59ba4640 1 Processor -- start 2023-11-06T03:04:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.257+0000 7efd59ba4640 1 -- start start 2023-11-06T03:04:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.257+0000 7efd59ba4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd540c9ee0 0x7efd540cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd59ba4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efd540cd930 0x7efd540c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd59ba4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5415d650 con 0x7efd540c9b00 2023-11-06T03:04:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd59ba4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd540ccc30 con 0x7efd540c9ee0 2023-11-06T03:04:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd59ba4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd541599e0 con 0x7efd540cd930 2023-11-06T03:04:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd540c9ee0 0x7efd540cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd540c9ee0 0x7efd540cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43964/0 (socket says 172.21.15.157:43964) 2023-11-06T03:04:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 -- 172.21.15.157:0/3053750160 learned_addr learned my addr 172.21.15.157:0/3053750160 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd593a3640 1 --2- 172.21.15.157:0/3053750160 >> v2:172.21.15.195:3300/0 conn(0x7efd540cd930 0x7efd540c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 -- 172.21.15.157:0/3053750160 >> v2:172.21.15.195:3300/0 conn(0x7efd540cd930 msgr2=0x7efd540c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 --2- 172.21.15.157:0/3053750160 >> v2:172.21.15.195:3300/0 conn(0x7efd540cd930 0x7efd540c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 -- 172.21.15.157:0/3053750160 >> v1:172.21.15.157:6789/0 conn(0x7efd540c9b00 legacy=0x7efd54159120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.261+0000 7efd53fff640 1 -- 172.21.15.157:0/3053750160 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54159cc0 con 0x7efd540c9ee0 2023-11-06T03:04:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd53fff640 1 --2- 172.21.15.157:0/3053750160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9ee0 0x7efd540cd3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd540cc5f0 tx=0x7efd4401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898a9ec4b152caed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd537fe640 1 -- 172.21.15.157:0/3053750160 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440ab600 con 0x7efd540c9ee0 2023-11-06T03:04:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd537fe640 1 -- 172.21.15.157:0/3053750160 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd440a7050 con 0x7efd540c9ee0 2023-11-06T03:04:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 -- 172.21.15.157:0/3053750160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9ee0 msgr2=0x7efd540cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 --2- 172.21.15.157:0/3053750160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9ee0 0x7efd540cd3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd540cc5f0 tx=0x7efd4401c430 comp rx=0 tx=0).stop 2023-11-06T03:04:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 -- 172.21.15.157:0/3053750160 shutdown_connections 2023-11-06T03:04:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 --2- 172.21.15.157:0/3053750160 >> v2:172.21.15.195:3300/0 conn(0x7efd540cd930 0x7efd540c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 --2- 172.21.15.157:0/3053750160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9ee0 0x7efd540cd3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 -- 172.21.15.157:0/3053750160 >> 172.21.15.157:0/3053750160 conn(0x7efd540cafa0 msgr2=0x7efd540cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 -- 172.21.15.157:0/3053750160 shutdown_connections 2023-11-06T03:04:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.265+0000 7efd59ba4640 1 -- 172.21.15.157:0/3053750160 wait complete. 2023-11-06T03:04:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 Processor -- start 2023-11-06T03:04:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- start start 2023-11-06T03:04:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 0x7efd540c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 0x7efd5414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c75e0 con 0x7efd540c9b00 2023-11-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5414db90 con 0x7efd540c9ee0 2023-11-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c7810 con 0x7efd540cd930 2023-11-06T03:04:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd58ba2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd58ba2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43970/0 (socket says 172.21.15.157:43970) 2023-11-06T03:04:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd593a3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 0x7efd5414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd58ba2640 1 -- 172.21.15.157:0/3280309148 learned_addr learned my addr 172.21.15.157:0/3280309148 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 0x7efd540c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 -- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 msgr2=0x7efd5414bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 0x7efd5414bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 -- 172.21.15.157:0/3280309148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 msgr2=0x7efd5414cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 -- 172.21.15.157:0/3280309148 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd440094d0 con 0x7efd540c9ee0 2023-11-06T03:04:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7fdef6324640 1 Processor -- start 2023-11-06T03:04:18.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7fdef6324640 1 -- start start 2023-11-06T03:04:18.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd58ba2640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:18.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd593a3640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 0x7efd5414bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:18.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd53fff640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 0x7efd540c70a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd440ab000 tx=0x7efd440191d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44004450 con 0x7efd540c9ee0 2023-11-06T03:04:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd440a7050 con 0x7efd540c9ee0 2023-11-06T03:04:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44003840 con 0x7efd540c9ee0 2023-11-06T03:04:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- 172.21.15.157:0/3280309148 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd540c7af0 con 0x7efd540c9ee0 2023-11-06T03:04:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.269+0000 7efd59ba4640 1 -- 172.21.15.157:0/3280309148 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5414d820 con 0x7efd540c9ee0 2023-11-06T03:04:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef6324640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdef01599c0 0x7fdef00b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef6324640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef006b900 con 0x7fdef01595e0 2023-11-06T03:04:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef6324640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef006bbc0 con 0x7fdef006b490 2023-11-06T03:04:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef6324640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef006bea0 con 0x7fdef01599c0 2023-11-06T03:04:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fdef006b490 0x7fdef00cba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:50150/0 (socket says 172.21.15.157:50150) 2023-11-06T03:04:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 -- 172.21.15.157:0/2698863693 learned_addr learned my addr 172.21.15.157:0/2698863693 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399964906 0 0) 0x7fdef006bbc0 con 0x7fdef006b490 2023-11-06T03:04:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded8003530 con 0x7fdef006b490 2023-11-06T03:04:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301890601 0 0) 0x7fdef006b900 con 0x7fdef01595e0 2023-11-06T03:04:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef006bbc0 con 0x7fdef01595e0 2023-11-06T03:04:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.1 v1:172.21.15.195:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256371914 0 0) 0x7fded8003530 con 0x7fdef006b490 2023-11-06T03:04:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 >> v2:172.21.15.195:3300/0 conn(0x7fdef01599c0 msgr2=0x7fdef00b2550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-06T03:04:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 --2- 172.21.15.157:0/2698863693 >> v2:172.21.15.195:3300/0 conn(0x7fdef01599c0 0x7fdef00b2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 >> v1:172.21.15.157:6789/0 conn(0x7fdef01595e0 legacy=0x7fdef006cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> v1:172.21.15.195:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef006c180 con 0x7fdef006b490 2023-11-06T03:04:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.1 v1:172.21.15.195:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdee8001a00 con 0x7fdef006b490 2023-11-06T03:04:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 >> v1:172.21.15.195:6789/0 conn(0x7fdef006b490 legacy=0x7fdef00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fded8003b10 0x7fded8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80047d0 0x7fded8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 0x7fded800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 0x7fded800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeffff640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fded8003b10 0x7fded8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd440b2070 con 0x7efd540c9ee0 2023-11-06T03:04:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0001240 con 0x7fded80052b0 2023-11-06T03:04:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef006bea0 con 0x7fded80047d0 2023-11-06T03:04:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded800c200 con 0x7fded8003b10 2023-11-06T03:04:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 -- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fded8003b10 msgr2=0x7fded8003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fded8003b10 0x7fded8003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 -- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80047d0 msgr2=0x7fded8004c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80047d0 0x7fded8004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdef48c1640 1 -- 172.21.15.157:0/2698863693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8003530 con 0x7fded80052b0 2023-11-06T03:04:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7fdeeffff640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fded8003b10 0x7fded8003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7fdef48c1640 1 -- 172.21.15.157:0/2698863693 reap_dead start 2023-11-06T03:04:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7fdef48c1640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 0x7fded800b990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdee8002240 tx=0x7fdee801f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52432c9a273f0d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80b0070 con 0x7fded80052b0 2023-11-06T03:04:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee80ac3f0 con 0x7fded80052b0 2023-11-06T03:04:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7fdeeeffd640 1 -- 172.21.15.157:0/2698863693 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80ac760 con 0x7fded80052b0 2023-11-06T03:04:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7efd517fa640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd4411cd00 con 0x7efd540c9ee0 2023-11-06T03:04:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.277+0000 7efd58ba2640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 -- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 msgr2=0x7fded800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 0x7fded800b990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdee8002240 tx=0x7fdee801f940 comp rx=0 tx=0).stop 2023-11-06T03:04:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.273+0000 7efd59ba4640 1 -- 172.21.15.157:0/3280309148 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd540b12d0 con 0x7efd540c9ee0 2023-11-06T03:04:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7efd58ba2640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 0x7efd40067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd48004470 tx=0x7efd48003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 -- 172.21.15.157:0/2698863693 shutdown_connections 2023-11-06T03:04:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 --2- 172.21.15.157:0/2698863693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fded80052b0 0x7fded800b990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 -- 172.21.15.157:0/2698863693 >> 172.21.15.157:0/2698863693 conn(0x7fdef0065d30 msgr2=0x7fdef0066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 -- 172.21.15.157:0/2698863693 shutdown_connections 2023-11-06T03:04:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.281+0000 7fdef6324640 1 -- 172.21.15.157:0/2698863693 wait complete. 2023-11-06T03:04:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 Processor -- start 2023-11-06T03:04:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 -- start start 2023-11-06T03:04:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80052b0 0x7fdef014d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdef01412f0 0x7fdef015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0154ec0 con 0x7fdef0154950 2023-11-06T03:04:18.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef01551a0 con 0x7fded80052b0 2023-11-06T03:04:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdef6324640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0155480 con 0x7fdef01412f0 2023-11-06T03:04:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43990/0 (socket says 172.21.15.157:43990) 2023-11-06T03:04:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 -- 172.21.15.157:0/305466940 learned_addr learned my addr 172.21.15.157:0/305466940 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeeffff640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80052b0 0x7fdef014d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 -- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdef01412f0 msgr2=0x7fdef015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdef01412f0 0x7fdef015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 -- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80052b0 msgr2=0x7fdef014d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80052b0 0x7fdef014d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 -- 172.21.15.157:0/305466940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0155760 con 0x7fdef0154950 2023-11-06T03:04:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeef7fe640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b410 tx=0x7fdee400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.285+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4011830 con 0x7fdef0154950 2023-11-06T03:04:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.289+0000 7fdef6324640 1 -- 172.21.15.157:0/305466940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef0155a40 con 0x7fdef0154950 2023-11-06T03:04:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.289+0000 7fdef6324640 1 -- 172.21.15.157:0/305466940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef014d9d0 con 0x7fdef0154950 2023-11-06T03:04:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.289+0000 7fdef6324640 1 -- 172.21.15.157:0/305466940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef0155c80 con 0x7fdef0154950 2023-11-06T03:04:18.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efd440ea570 con 0x7efd540c9ee0 2023-11-06T03:04:18.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4011990 con 0x7fdef0154950 2023-11-06T03:04:18.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4011d40 con 0x7fdef0154950 2023-11-06T03:04:18.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdee401a400 con 0x7fdef0154950 2023-11-06T03:04:18.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeecff9640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 0x7fded4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdee40839f0 con 0x7fdef0154950 2023-11-06T03:04:18.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeeffff640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 0x7fded4067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.293+0000 7fdeeffff640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 0x7fded4067f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001ec0 tx=0x7fdee0014040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.297+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdee40512e0 con 0x7fdef0154950 2023-11-06T03:04:18.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.317+0000 7f306e673640 1 Processor -- start 2023-11-06T03:04:18.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.317+0000 7f306e673640 1 -- start start 2023-11-06T03:04:18.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f306e673640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 0x7f30680c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f306e673640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680c9f70 con 0x7f30680c9b10 2023-11-06T03:04:18.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f306e673640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680cd7d0 con 0x7f30680cd170 2023-11-06T03:04:18.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f306e673640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680cdab0 con 0x7f30680cebf0 2023-11-06T03:04:18.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f30677fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f30680c9b10 0x7f30680cca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43626/0 (socket says 172.21.15.157:43626) 2023-11-06T03:04:18.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f30677fe640 1 -- 172.21.15.157:0/3893712884 learned_addr learned my addr 172.21.15.157:0/3893712884 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:18.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f3067fff640 1 --2- 172.21.15.157:0/3893712884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 0x7f30680c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f3066ffd640 1 -- 172.21.15.157:0/3893712884 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612025833 0 0) 0x7f30680c9f70 con 0x7f30680c9b10 2023-11-06T03:04:18.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f3067fff640 1 -- 172.21.15.157:0/3893712884 >> v1:172.21.15.195:6789/0 conn(0x7f30680cd170 legacy=0x7f30681581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f3067fff640 1 -- 172.21.15.157:0/3893712884 >> v1:172.21.15.157:6789/0 conn(0x7f30680c9b10 legacy=0x7f30680cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.321+0000 7f3067fff640 1 -- 172.21.15.157:0/3893712884 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680b1e50 con 0x7f30680cebf0 2023-11-06T03:04:18.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 Processor -- start 2023-11-06T03:04:18.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 -- start start 2023-11-06T03:04:18.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f3067fff640 1 --2- 172.21.15.157:0/3893712884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 0x7f30680c95d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f305800ba80 tx=0x7f305801eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73dc8690d69cd801 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f3066ffd640 1 -- 172.21.15.157:0/3893712884 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580a7a50 con 0x7f30680cebf0 2023-11-06T03:04:18.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f3066ffd640 1 -- 172.21.15.157:0/3893712884 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30580ab070 con 0x7f30680cebf0 2023-11-06T03:04:18.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f3066ffd640 1 -- 172.21.15.157:0/3893712884 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580a7dd0 con 0x7f30680cebf0 2023-11-06T03:04:18.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a40c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38a40c9b10 0x7f38a40cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38a40ccfa0 0x7f38a41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40c9f70 con 0x7f38a40ccfa0 2023-11-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40cd7d0 con 0x7f38a40c9b10 2023-11-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38aa8af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40cdab0 con 0x7f38a40cebf0 2023-11-06T03:04:18.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 -- 172.21.15.157:0/3893712884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 msgr2=0x7f30680c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 --2- 172.21.15.157:0/3893712884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 0x7f30680c95d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f305800ba80 tx=0x7f305801eb60 comp rx=0 tx=0).stop 2023-11-06T03:04:18.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 -- 172.21.15.157:0/3893712884 shutdown_connections 2023-11-06T03:04:18.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 --2- 172.21.15.157:0/3893712884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680cebf0 0x7f30680c95d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 -- 172.21.15.157:0/3893712884 >> 172.21.15.157:0/3893712884 conn(0x7f3068065930 msgr2=0x7f3068065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a40c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 -- 172.21.15.157:0/3893712884 shutdown_connections 2023-11-06T03:04:18.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38a40ccfa0 0x7f38a41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38a40ccfa0 0x7f38a41581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44000/0 (socket says 172.21.15.157:44000) 2023-11-06T03:04:18.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 -- 172.21.15.157:0/783116687 learned_addr learned my addr 172.21.15.157:0/783116687 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f306e673640 1 -- 172.21.15.157:0/3893712884 wait complete. 2023-11-06T03:04:18.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 Processor -- start 2023-11-06T03:04:18.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 -- start start 2023-11-06T03:04:18.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30680cd170 0x7f306814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30680cebf0 0x7f306806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c7e40 con 0x7f30680cebf0 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068150da0 con 0x7f30680cd170 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306e673640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30681502e0 con 0x7f30680c9b10 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54234/0 (socket says 172.21.15.157:54234) 2023-11-06T03:04:18.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 -- 172.21.15.157:0/196802416 learned_addr learned my addr 172.21.15.157:0/196802416 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f30677fe640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30680cd170 0x7f306814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 -- 172.21.15.157:0/783116687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 msgr2=0x7f38a40c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 --2- 172.21.15.157:0/783116687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a40c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 -- 172.21.15.157:0/783116687 >> v2:172.21.15.195:3300/0 conn(0x7f38a40c9b10 msgr2=0x7f38a40cca60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-06T03:04:18.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 --2- 172.21.15.157:0/783116687 >> v2:172.21.15.195:3300/0 conn(0x7f38a40c9b10 0x7f38a40cca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.325+0000 7f38a8e4c640 1 -- 172.21.15.157:0/783116687 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40b1e50 con 0x7f38a40ccfa0 2023-11-06T03:04:18.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f306cc10640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30680cebf0 0x7f306806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f38a8e4c640 1 --2- 172.21.15.157:0/783116687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a41581c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38a40cc0c0 tx=0x7f389c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ac87fbf775edd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 -- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30680cd170 msgr2=0x7f306814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30680cd170 0x7f306814fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 -- 172.21.15.157:0/196802416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30680cebf0 msgr2=0x7f306806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30680cebf0 0x7f306806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 -- 172.21.15.157:0/196802416 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305800b6b0 con 0x7f30680c9b10 2023-11-06T03:04:18.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f38a2ffd640 1 -- 172.21.15.157:0/783116687 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0a5790 con 0x7f38a40ccfa0 2023-11-06T03:04:18.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f38a2ffd640 1 -- 172.21.15.157:0/783116687 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0a58f0 con 0x7f38a40ccfa0 2023-11-06T03:04:18.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f3067fff640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f305800ba80 tx=0x7f305801d3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.329+0000 7f38a2ffd640 1 -- 172.21.15.157:0/783116687 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0a5790 con 0x7f38a40ccfa0 2023-11-06T03:04:18.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580b1750 con 0x7f30680c9b10 2023-11-06T03:04:18.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680c7b60 con 0x7f30680c9b10 2023-11-06T03:04:18.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30681594d0 con 0x7f30680c9b10 2023-11-06T03:04:18.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- 172.21.15.157:0/783116687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 msgr2=0x7f38a41581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 --2- 172.21.15.157:0/783116687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a41581c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f38a40cc0c0 tx=0x7f389c01c9e0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- 172.21.15.157:0/783116687 shutdown_connections 2023-11-06T03:04:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 --2- 172.21.15.157:0/783116687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a41581c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 --2- 172.21.15.157:0/783116687 >> v2:172.21.15.195:3300/0 conn(0x7f38a40c9b10 0x7f38a40cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 --2- 172.21.15.157:0/783116687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a40c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- 172.21.15.157:0/783116687 >> 172.21.15.157:0/783116687 conn(0x7f38a4065930 msgr2=0x7f38a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- 172.21.15.157:0/783116687 shutdown_connections 2023-11-06T03:04:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30580b18d0 con 0x7f30680c9b10 2023-11-06T03:04:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580b1c80 con 0x7f30680c9b10 2023-11-06T03:04:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- 172.21.15.157:0/783116687 wait complete. 2023-11-06T03:04:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 Processor -- start 2023-11-06T03:04:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f38aa8af640 1 -- start start 2023-11-06T03:04:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f30580c43b0 con 0x7f30680c9b10 2023-11-06T03:04:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 0x7f3050067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f305811d6f0 con 0x7f30680c9b10 2023-11-06T03:04:18.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.333+0000 7f30677fe640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 0x7f3050067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048005260 con 0x7f30680c9b10 2023-11-06T03:04:18.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 0x7f38a414b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 0x7f38a414b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 0x7f38a414b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:43806/0 (socket says 172.21.15.157:43806) 2023-11-06T03:04:18.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44020/0 (socket says 172.21.15.157:44020) 2023-11-06T03:04:18.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a3fff640 1 -- 172.21.15.157:0/87807963 learned_addr learned my addr 172.21.15.157:0/87807963 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f30677fe640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 0x7f3050067fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30681fcd40 tx=0x7f3060004b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40c3850 con 0x7f38a40ccfa0 2023-11-06T03:04:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4157750 con 0x7f38a40c9b10 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a414fa70 con 0x7f38a40cebf0 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a8e4c640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 -- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 msgr2=0x7f38a414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 -- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 msgr2=0x7f38a414b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 0x7f38a414b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009b30 con 0x7f38a40ccfa0 2023-11-06T03:04:18.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a37fe640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f389400b3c0 tx=0x7f389400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011830 con 0x7f38a40ccfa0 2023-11-06T03:04:18.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894011990 con 0x7f38a40ccfa0 2023-11-06T03:04:18.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011d80 con 0x7f38a40ccfa0 2023-11-06T03:04:18.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a40c3500 con 0x7f38a40ccfa0 2023-11-06T03:04:18.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.337+0000 7f38aa8af640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a414ffa0 con 0x7f38a40ccfa0 2023-11-06T03:04:18.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.341+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f389401b460 con 0x7f38a40ccfa0 2023-11-06T03:04:18.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.341+0000 7f38a0ff9640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 0x7f3890067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.341+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3894082ec0 con 0x7f38a40ccfa0 2023-11-06T03:04:18.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.341+0000 7f38a3fff640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 0x7f3890067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.345+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3884005260 con 0x7f38a40ccfa0 2023-11-06T03:04:18.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.345+0000 7f38a3fff640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 0x7f3890067f70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38a414fa00 tx=0x7f389800c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.349+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f30580ad080 con 0x7f30680c9b10 2023-11-06T03:04:18.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.353+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3894050890 con 0x7f38a40ccfa0 2023-11-06T03:04:18.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 Processor -- start 2023-11-06T03:04:18.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 -- start start 2023-11-06T03:04:18.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f095c0e7710 0x7f095c04f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f095c006880 0x7f095c0eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0483e0 con 0x7f095c006880 2023-11-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c037ad0 con 0x7f095c0e7710 2023-11-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969848640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c050210 con 0x7f095c005ee0 2023-11-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969047640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f095c006880 0x7f095c0eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969047640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f095c006880 0x7f095c0eb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44030/0 (socket says 172.21.15.157:44030) 2023-11-06T03:04:18.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0969047640 1 -- 172.21.15.157:0/267301460 learned_addr learned my addr 172.21.15.157:0/267301460 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.377+0000 7f0968846640 1 --2- 172.21.15.157:0/267301460 >> v2:172.21.15.195:3300/0 conn(0x7f095c0e7710 0x7f095c04f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0963fff640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 -- 172.21.15.157:0/267301460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 msgr2=0x7f095c006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c006340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 -- 172.21.15.157:0/267301460 >> v2:172.21.15.195:3300/0 conn(0x7f095c0e7710 msgr2=0x7f095c04f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 --2- 172.21.15.157:0/267301460 >> v2:172.21.15.195:3300/0 conn(0x7f095c0e7710 0x7f095c04f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 -- 172.21.15.157:0/267301460 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c04fa00 con 0x7f095c006880 2023-11-06T03:04:18.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 Processor -- start 2023-11-06T03:04:18.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 -- start start 2023-11-06T03:04:18.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4159990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ca40c93d0 0x7f1ca40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca406d950 con 0x7f1ca40c5140 2023-11-06T03:04:18.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca41569c0 con 0x7f1ca40c93d0 2023-11-06T03:04:18.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa09d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ca40c93d0 0x7f1ca40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 Processor -- start 2023-11-06T03:04:18.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 -- start start 2023-11-06T03:04:18.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f0969047640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c006880 0x7f095c0eb0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b0b0 tx=0x7f09640965f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca4852af7b4e50e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f09637fe640 1 -- 172.21.15.157:0/267301460 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964125640 con 0x7f095c006880 2023-11-06T03:04:18.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f09637fe640 1 -- 172.21.15.157:0/267301460 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0964121050 con 0x7f095c006880 2023-11-06T03:04:18.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f09637fe640 1 -- 172.21.15.157:0/267301460 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964125a60 con 0x7f095c006880 2023-11-06T03:04:18.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1e80ade30 0x7fd1e815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 0x7fd1e80ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80ae210 con 0x7fd1e80b2420 2023-11-06T03:04:18.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e815e130 con 0x7fd1e80ade30 2023-11-06T03:04:18.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1ec9fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80ad070 con 0x7fd1e8071170 2023-11-06T03:04:18.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1e6d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd1e80b2420 0x7fd1e806fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43658/0 (socket says 172.21.15.157:43658) 2023-11-06T03:04:18.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7fd1e6d9d640 1 -- 172.21.15.157:0/1169860246 learned_addr learned my addr 172.21.15.157:0/1169860246 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:18.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e659c640 1 --2- 172.21.15.157:0/1169860246 >> v2:172.21.15.157:3300/0 conn(0x7fd1e80ade30 0x7fd1e815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa89e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4159ed0 con 0x7f1ca40cb5b0 2023-11-06T03:04:18.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1caa09d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1ca40c93d0 0x7f1ca40c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44042/0 (socket says 172.21.15.157:44042) 2023-11-06T03:04:18.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f1caa09d640 1 -- 172.21.15.157:0/2225517374 learned_addr learned my addr 172.21.15.157:0/2225517374 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1ca989c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1ca40c5140 0x7f1ca40caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43642/0 (socket says 172.21.15.157:43642) 2023-11-06T03:04:18.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.381+0000 7f1ca909b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/1169860246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 0x7fd1e80ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- 172.21.15.157:0/267301460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c006880 msgr2=0x7f095c0eb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c006880 0x7f095c0eb0e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b0b0 tx=0x7f09640965f0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- 172.21.15.157:0/267301460 shutdown_connections 2023-11-06T03:04:18.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c006880 0x7f095c0eb0e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 --2- 172.21.15.157:0/267301460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c006340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 --2- 172.21.15.157:0/267301460 >> v2:172.21.15.195:3300/0 conn(0x7f095c0e7710 0x7f095c04f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- 172.21.15.157:0/267301460 >> 172.21.15.157:0/267301460 conn(0x7f095c054b10 msgr2=0x7f095c054f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- 172.21.15.157:0/267301460 shutdown_connections 2023-11-06T03:04:18.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- 172.21.15.157:0/267301460 wait complete. 2023-11-06T03:04:18.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/1169860246 >> v2:172.21.15.157:3300/0 conn(0x7fd1e80ade30 msgr2=0x7fd1e815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/1169860246 >> v2:172.21.15.157:3300/0 conn(0x7fd1e80ade30 0x7fd1e815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/1169860246 >> v1:172.21.15.157:6789/0 conn(0x7fd1e80b2420 legacy=0x7fd1e806fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/1169860246 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80ad350 con 0x7fd1e8071170 2023-11-06T03:04:18.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 Processor -- start 2023-11-06T03:04:18.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 -- start start 2023-11-06T03:04:18.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f0969848640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c038920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 Processor -- start 2023-11-06T03:04:18.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 0x7f095c0d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f1caa09d640 1 -- 172.21.15.157:0/2225517374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 msgr2=0x7f1ca4159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:18.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f1caa09d640 1 --2- 172.21.15.157:0/2225517374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f1caa09d640 1 -- 172.21.15.157:0/2225517374 >> v1:172.21.15.157:6789/0 conn(0x7f1ca40c5140 legacy=0x7f1ca40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.385+0000 7f1caa09d640 1 -- 172.21.15.157:0/2225517374 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca406cd40 con 0x7f1ca40c93d0 2023-11-06T03:04:18.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c0e7710 0x7f095c0d4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa09d640 1 --2- 172.21.15.157:0/2225517374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40c9860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00ae50 tx=0x7f1c9c01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326d117ce37abc28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0968846640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c038920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0e8b20 con 0x7f095c0e7710 2023-11-06T03:04:18.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0d8430 con 0x7f095c006880 2023-11-06T03:04:18.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 0x7f095c0d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0968846640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c038920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54266/0 (socket says 172.21.15.157:54266) 2023-11-06T03:04:18.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0968846640 1 -- 172.21.15.157:0/634135051 learned_addr learned my addr 172.21.15.157:0/634135051 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c039800 con 0x7f095c005ee0 2023-11-06T03:04:18.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1ca889a640 1 -- 172.21.15.157:0/2225517374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0a79e0 con 0x7f1ca40c93d0 2023-11-06T03:04:18.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1ca889a640 1 -- 172.21.15.157:0/2225517374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0a7b40 con 0x7f1ca40c93d0 2023-11-06T03:04:18.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1ca889a640 1 -- 172.21.15.157:0/2225517374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0af3f0 con 0x7f1ca40c93d0 2023-11-06T03:04:18.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969047640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c0e7710 0x7f095c0d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 -- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 msgr2=0x7f095c038920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c038920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 -- 172.21.15.157:0/634135051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c0e7710 msgr2=0x7f095c0d4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c0e7710 0x7f095c0d4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c039ae0 con 0x7f095c006880 2023-11-06T03:04:18.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 -- start start 2023-11-06T03:04:18.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34e0048310 0x7f34e00eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e00ebeb0 con 0x7f34e0033e20 2023-11-06T03:04:18.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e00ec130 con 0x7f34e0047ce0 2023-11-06T03:04:18.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e7fff640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e004f310 con 0x7f34e0048310 2023-11-06T03:04:18.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0963fff640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 0x7f095c0d66d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f095000d5f0 tx=0x7f095000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 -- 172.21.15.157:0/2225517374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c93d0 msgr2=0x7f1ca40c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 --2- 172.21.15.157:0/2225517374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40c9860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00ae50 tx=0x7f1c9c01eb10 comp rx=0 tx=0).stop 2023-11-06T03:04:18.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f34e0033e20 0x7f34e00eb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43674/0 (socket says 172.21.15.157:43674) 2023-11-06T03:04:18.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e6ffd640 1 -- 172.21.15.157:0/2590609527 learned_addr learned my addr 172.21.15.157:0/2590609527 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:18.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/1169860246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 0x7fd1e80ac4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0097c0 tx=0x7fd1dc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48a1f321de3bfeee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 -- 172.21.15.157:0/2225517374 shutdown_connections 2023-11-06T03:04:18.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 --2- 172.21.15.157:0/2225517374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40c9860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 --2- 172.21.15.157:0/2225517374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 -- 172.21.15.157:0/2225517374 >> 172.21.15.157:0/2225517374 conn(0x7f1ca40b0160 msgr2=0x7f1ca40b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e67fc640 1 --2- 172.21.15.157:0/2590609527 >> v2:172.21.15.195:3300/0 conn(0x7f34e0048310 0x7f34e00eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 -- 172.21.15.157:0/2225517374 shutdown_connections 2023-11-06T03:04:18.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09500125f0 con 0x7f095c006880 2023-11-06T03:04:18.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c0d8710 con 0x7f095c006880 2023-11-06T03:04:18.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950012750 con 0x7f095c006880 2023-11-06T03:04:18.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f1caa89e640 1 -- 172.21.15.157:0/2225517374 wait complete. 2023-11-06T03:04:18.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1e559a640 1 -- 172.21.15.157:0/1169860246 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0ab650 con 0x7fd1e8071170 2023-11-06T03:04:18.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1e559a640 1 -- 172.21.15.157:0/1169860246 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0ab7b0 con 0x7fd1e8071170 2023-11-06T03:04:18.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1e559a640 1 -- 172.21.15.157:0/1169860246 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0abb40 con 0x7fd1e8071170 2023-11-06T03:04:18.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/1169860246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 msgr2=0x7fd1e80ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/1169860246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 0x7fd1e80ac4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0097c0 tx=0x7fd1dc01c470 comp rx=0 tx=0).stop 2023-11-06T03:04:18.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 Processor -- start 2023-11-06T03:04:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- start start 2023-11-06T03:04:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/1169860246 shutdown_connections 2023-11-06T03:04:18.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/1169860246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e8071170 0x7fd1e80ac4c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/1169860246 >> v2:172.21.15.157:3300/0 conn(0x7fd1e80ade30 0x7fd1e815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/1169860246 >> 172.21.15.157:0/1169860246 conn(0x7fd1e8065d30 msgr2=0x7fd1e8066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4205e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b4530 con 0x7f1ca40c5140 2023-11-06T03:04:18.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4071f70 con 0x7f1ca40c93d0 2023-11-06T03:04:18.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4153870 con 0x7f1ca40cb5b0 2023-11-06T03:04:18.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44072/0 (socket says 172.21.15.157:44072) 2023-11-06T03:04:18.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 -- 172.21.15.157:0/1929046624 learned_addr learned my addr 172.21.15.157:0/1929046624 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa09d640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4205e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca909b640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e67fc640 1 -- 172.21.15.157:0/2590609527 >> v1:172.21.15.195:6789/0 conn(0x7f34e0047ce0 legacy=0x7f34e0052ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e67fc640 1 -- 172.21.15.157:0/2590609527 >> v1:172.21.15.157:6789/0 conn(0x7f34e0033e20 legacy=0x7f34e00eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.389+0000 7f34e67fc640 1 -- 172.21.15.157:0/2590609527 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e004f5f0 con 0x7f34e0048310 2023-11-06T03:04:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 -- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 msgr2=0x7f1ca4205e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4205e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 -- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ca40c93d0 msgr2=0x7f1ca40b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 -- 172.21.15.157:0/1929046624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c00ab30 con 0x7f1ca40c5140 2023-11-06T03:04:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1ca989c640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c420 tx=0x7f1ca000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950012ae0 con 0x7f095c006880 2023-11-06T03:04:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c039030 con 0x7f095c006880 2023-11-06T03:04:18.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0950012c40 con 0x7f095c006880 2023-11-06T03:04:18.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00074e0 con 0x7f1ca40c5140 2023-11-06T03:04:18.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0007640 con 0x7f1ca40c5140 2023-11-06T03:04:18.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/1169860246 shutdown_connections 2023-11-06T03:04:18.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f34e67fc640 1 --2- 172.21.15.157:0/2590609527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00eb970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0097c0 tx=0x7f34dc01c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3b8e525f0b8290f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca40b39d0 con 0x7f1ca40c5140 2023-11-06T03:04:18.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004a30 con 0x7f1ca40c5140 2023-11-06T03:04:18.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/1169860246 wait complete. 2023-11-06T03:04:18.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40b3f70 con 0x7f1ca40c5140 2023-11-06T03:04:18.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ca0004b90 con 0x7f1ca40c5140 2023-11-06T03:04:18.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f1c9a7fc640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 0x7f1c88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ca0082ce0 con 0x7f1ca40c5140 2023-11-06T03:04:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 Processor -- start 2023-11-06T03:04:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- start start 2023-11-06T03:04:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1e8071170 0x7fd1e814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e80b2420 0x7fd1e81f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81469f0 con 0x7fd1e80ade30 2023-11-06T03:04:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e81f57c0 con 0x7fd1e8071170 2023-11-06T03:04:18.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8155490 con 0x7fd1e80b2420 2023-11-06T03:04:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44080/0 (socket says 172.21.15.157:44080) 2023-11-06T03:04:18.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/743857218 learned_addr learned my addr 172.21.15.157:0/743857218 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f34e5ffb640 1 -- 172.21.15.157:0/2590609527 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0ab600 con 0x7f34e0048310 2023-11-06T03:04:18.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f34e5ffb640 1 -- 172.21.15.157:0/2590609527 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc0a7050 con 0x7f34e0048310 2023-11-06T03:04:18.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.393+0000 7f34e5ffb640 1 -- 172.21.15.157:0/2590609527 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0ab980 con 0x7f34e0048310 2023-11-06T03:04:18.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- 172.21.15.157:0/2590609527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 msgr2=0x7f34e00eb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 --2- 172.21.15.157:0/2590609527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00eb970 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0097c0 tx=0x7f34dc01c2d0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- 172.21.15.157:0/2590609527 shutdown_connections 2023-11-06T03:04:18.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 --2- 172.21.15.157:0/2590609527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00eb970 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- 172.21.15.157:0/2590609527 >> 172.21.15.157:0/2590609527 conn(0x7f34e0054c20 msgr2=0x7f34e0055030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- 172.21.15.157:0/2590609527 shutdown_connections 2023-11-06T03:04:18.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- 172.21.15.157:0/2590609527 wait complete. 2023-11-06T03:04:18.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 Processor -- start 2023-11-06T03:04:18.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e659c640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1e8071170 0x7fd1e814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e6d9d640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e80b2420 0x7fd1e81f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f09617fa640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 0x7f0948067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f1ca909b640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 0x7f1c88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e80b2420 msgr2=0x7fd1e81f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e80b2420 0x7fd1e81f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1e8071170 msgr2=0x7fd1e814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1e8071170 0x7fd1e814a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 -- 172.21.15.157:0/743857218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009420 con 0x7fd1e80ade30 2023-11-06T03:04:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0950082e20 con 0x7f095c006880 2023-11-06T03:04:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f0968846640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 0x7f0948067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1e5d9b640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ad040 tx=0x7fd1dc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- start start 2023-11-06T03:04:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b5810 con 0x7fd1e80ade30 2023-11-06T03:04:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0b5970 con 0x7fd1e80ade30 2023-11-06T03:04:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b5d20 con 0x7fd1e80ade30 2023-11-06T03:04:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 0x7f34e00deef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e81567e0 con 0x7fd1e80ade30 2023-11-06T03:04:18.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e8155120 con 0x7fd1e80ade30 2023-11-06T03:04:18.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 0x7f34e004b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00d58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00e7d60 con 0x7f34e0047ce0 2023-11-06T03:04:18.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00df5f0 con 0x7f34e0048310 2023-11-06T03:04:18.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.397+0000 7f34e7fff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e004a400 con 0x7f34e0033e20 2023-11-06T03:04:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f1ca909b640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 0x7f1c88067f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40cefc0 tx=0x7f1c94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e6ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 0x7f34e00deef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e6ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 0x7f34e00deef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54288/0 (socket says 172.21.15.157:54288) 2023-11-06T03:04:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e6ffd640 1 -- 172.21.15.157:0/88305339 learned_addr learned my addr 172.21.15.157:0/88305339 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e77fe640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00d58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 0x7f34e004b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f0968846640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 0x7f0948067ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f09580079f0 tx=0x7f0958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 -- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 msgr2=0x7f34e00deef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 0x7f34e00deef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 -- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 msgr2=0x7f34e00d58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00d58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 -- 172.21.15.157:0/88305339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc009420 con 0x7f34e0047ce0 2023-11-06T03:04:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc005260 con 0x7fd1e80ade30 2023-11-06T03:04:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e67fc640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 0x7f34e004b080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0ab000 tx=0x7f34dc019070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c038770 con 0x7f095c006880 2023-11-06T03:04:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0b4600 con 0x7f34e0047ce0 2023-11-06T03:04:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc0b4760 con 0x7f34e0047ce0 2023-11-06T03:04:18.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0b4b50 con 0x7f34e0047ce0 2023-11-06T03:04:18.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca4072670 con 0x7f1ca40c5140 2023-11-06T03:04:18.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e004a6e0 con 0x7f34e0047ce0 2023-11-06T03:04:18.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e00d5df0 con 0x7f34e0047ce0 2023-11-06T03:04:18.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd1dc0bf410 con 0x7fd1e80ade30 2023-11-06T03:04:18.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1d6ffd640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 0x7fd1c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1dc11e1e0 con 0x7fd1e80ade30 2023-11-06T03:04:18.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1e659c640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 0x7fd1c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7fd1e659c640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 0x7fd1c8067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007a60 tx=0x7fd1d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.401+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00e7ed0 con 0x7f34e0047ce0 2023-11-06T03:04:18.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.405+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f34dc0b2040 con 0x7f34e0047ce0 2023-11-06T03:04:18.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.405+0000 7f34cf7fe640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 0x7f34d4067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.405+0000 7f34e6ffd640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 0x7f34d4067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.405+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34dc0ef030 con 0x7f34e0047ce0 2023-11-06T03:04:18.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.405+0000 7f34e6ffd640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 0x7f34d4067f70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000f70 tx=0x7f34d8007ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.409+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1ca0050530 con 0x7f1ca40c5140 2023-11-06T03:04:18.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.413+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd1dc0ebc00 con 0x7fd1e80ade30 2023-11-06T03:04:18.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.417+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0950088060 con 0x7f095c006880 2023-11-06T03:04:18.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.417+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f34dc0eb400 con 0x7f34e0047ce0 2023-11-06T03:04:18.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.601+0000 7efd59ba4640 1 -- 172.21.15.157:0/3280309148 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efd54072670 con 0x7efd540c9ee0 2023-11-06T03:04:18.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.601+0000 7efd517fa640 1 -- 172.21.15.157:0/3280309148 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd440b8070 con 0x7efd540c9ee0 2023-11-06T03:04:18.602 INFO:teuthology.orchestra.run.smithi157.stdout:55834574856 2023-11-06T03:04:18.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 msgr2=0x7efd40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 0x7efd40067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd48004470 tx=0x7efd48003700 comp rx=0 tx=0).stop 2023-11-06T03:04:18.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 msgr2=0x7efd540c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 0x7efd540c70a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd440ab000 tx=0x7efd440191d0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 shutdown_connections 2023-11-06T03:04:18.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd540cd930 0x7efd5414bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd540c9ee0 0x7efd540c70a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 --2- 172.21.15.157:0/3280309148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd540c9b00 0x7efd5414cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 >> 172.21.15.157:0/3280309148 conn(0x7efd540cafa0 msgr2=0x7efd5406d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 shutdown_connections 2023-11-06T03:04:18.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.605+0000 7efd3affd640 1 -- 172.21.15.157:0/3280309148 wait complete. 2023-11-06T03:04:18.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.617+0000 7fdef6324640 1 -- 172.21.15.157:0/305466940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdef0155c80 con 0x7fdef0154950 2023-11-06T03:04:18.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.617+0000 7fdeecff9640 1 -- 172.21.15.157:0/305466940 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdee4015020 con 0x7fdef0154950 2023-11-06T03:04:18.616 INFO:teuthology.orchestra.run.smithi157.stdout:55834574857 2023-11-06T03:04:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 msgr2=0x7fded4067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 0x7fded4067f70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001ec0 tx=0x7fdee0014040 comp rx=0 tx=0).stop 2023-11-06T03:04:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 msgr2=0x7fdef0155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b410 tx=0x7fdee400b8d0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 shutdown_connections 2023-11-06T03:04:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fded4065ab0 0x7fded4067f70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdef01412f0 0x7fdef015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdef0154950 0x7fdef0155f60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 --2- 172.21.15.157:0/305466940 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fded80052b0 0x7fdef014d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 >> 172.21.15.157:0/305466940 conn(0x7fdef0065d30 msgr2=0x7fdef00ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 shutdown_connections 2023-11-06T03:04:18.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.621+0000 7fdede7fc640 1 -- 172.21.15.157:0/305466940 wait complete. 2023-11-06T03:04:18.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-11-06T03:04:18.629 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.637+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f38840057e0 con 0x7f38a40ccfa0 2023-11-06T03:04:18.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f38a0ff9640 1 -- 172.21.15.157:0/87807963 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3894053eb0 con 0x7f38a40ccfa0 2023-11-06T03:04:18.639 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:18.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 msgr2=0x7f3890067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 0x7f3890067f70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38a414fa00 tx=0x7f389800c040 comp rx=0 tx=0).stop 2023-11-06T03:04:18.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 msgr2=0x7f38a40c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f389400b3c0 tx=0x7f389400b880 comp rx=0 tx=0).stop 2023-11-06T03:04:18.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 shutdown_connections 2023-11-06T03:04:18.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065ab0 0x7f3890067f70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a40cebf0 0x7f38a414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a40ccfa0 0x7f38a40c3c70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.641+0000 7f388a7fc640 1 --2- 172.21.15.157:0/87807963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a40c9b10 0x7f38a414b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.645+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 >> 172.21.15.157:0/87807963 conn(0x7f38a4065930 msgr2=0x7f38a40af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.645+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 shutdown_connections 2023-11-06T03:04:18.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.645+0000 7f388a7fc640 1 -- 172.21.15.157:0/87807963 wait complete. 2023-11-06T03:04:18.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-11-06T03:04:18.646 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.653+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3048005510 con 0x7f30680c9b10 2023-11-06T03:04:18.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.653+0000 7f3064ff9640 1 -- 172.21.15.157:0/196802416 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30580c0030 con 0x7f30680c9b10 2023-11-06T03:04:18.655 INFO:teuthology.orchestra.run.smithi157.stdout:55834574854 2023-11-06T03:04:18.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.653+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 msgr2=0x7f3050067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.653+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 0x7f3050067fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30681fcd40 tx=0x7f3060004b70 comp rx=0 tx=0).stop 2023-11-06T03:04:18.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 msgr2=0x7f3068150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f305800ba80 tx=0x7f305801d3a0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 shutdown_connections 2023-11-06T03:04:18.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3050065b20 0x7f3050067fe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30680cebf0 0x7f306806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30680cd170 0x7f306814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 --2- 172.21.15.157:0/196802416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30680c9b10 0x7f3068150520 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 >> 172.21.15.157:0/196802416 conn(0x7f3068065930 msgr2=0x7f30680adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 shutdown_connections 2023-11-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.657+0000 7f306e673640 1 -- 172.21.15.157:0/196802416 wait complete. 2023-11-06T03:04:18.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-06T03:04:18.667 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-11-06T03:04:18.672 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.681+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd1cc005510 con 0x7fd1e80ade30 2023-11-06T03:04:18.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.681+0000 7fd1d6ffd640 1 -- 172.21.15.157:0/743857218 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1dc0c4020 con 0x7fd1e80ade30 2023-11-06T03:04:18.681 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:18.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 msgr2=0x7fd1c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 0x7fd1c8067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8007a60 tx=0x7fd1d8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:18.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 msgr2=0x7fd1e81464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ad040 tx=0x7fd1dc01acb0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 shutdown_connections 2023-11-06T03:04:18.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1c8065a60 0x7fd1c8067f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1e80b2420 0x7fd1e81f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1e80ade30 0x7fd1e81464b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 --2- 172.21.15.157:0/743857218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1e8071170 0x7fd1e814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 >> 172.21.15.157:0/743857218 conn(0x7fd1e8065d30 msgr2=0x7fd1e8156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 shutdown_connections 2023-11-06T03:04:18.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.685+0000 7fd1ec9fb640 1 -- 172.21.15.157:0/743857218 wait complete. 2023-11-06T03:04:18.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.693+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f34e00054a0 con 0x7f34e0047ce0 2023-11-06T03:04:18.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.693+0000 7f34cf7fe640 1 -- 172.21.15.157:0/88305339 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34dc0b0070 con 0x7f34e0047ce0 2023-11-06T03:04:18.694 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:18.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 msgr2=0x7f34d4067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 0x7f34d4067f70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000f70 tx=0x7f34d8007ee0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 msgr2=0x7f34e004b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 0x7f34e004b080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0ab000 tx=0x7f34dc019070 comp rx=0 tx=0).stop 2023-11-06T03:04:18.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 shutdown_connections 2023-11-06T03:04:18.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34d4065ab0 0x7f34d4067f70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34e0048310 0x7f34e00d58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e0047ce0 0x7f34e004b080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 --2- 172.21.15.157:0/88305339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34e0033e20 0x7f34e00deef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 >> 172.21.15.157:0/88305339 conn(0x7f34e0054c20 msgr2=0x7f34e00e0200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 shutdown_connections 2023-11-06T03:04:18.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.697+0000 7f34e7fff640 1 -- 172.21.15.157:0/88305339 wait complete. 2023-11-06T03:04:18.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-06T03:04:18.703 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-06T03:04:18.715 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f095c008a20 con 0x7f095c006880 2023-11-06T03:04:18.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1ca40b4310 con 0x7f1ca40c5140 2023-11-06T03:04:18.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1c9a7fc640 1 -- 172.21.15.157:0/1929046624 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ca0053b50 con 0x7f1ca40c5140 2023-11-06T03:04:18.726 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:18.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f09617fa640 1 -- 172.21.15.157:0/634135051 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0950050830 con 0x7f095c006880 2023-11-06T03:04:18.727 INFO:teuthology.orchestra.run.smithi157.stdout:55834574855 2023-11-06T03:04:18.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 msgr2=0x7f1c88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 0x7f1c88067f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40cefc0 tx=0x7f1c94008040 comp rx=0 tx=0).stop 2023-11-06T03:04:18.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 msgr2=0x7f1ca4070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c420 tx=0x7f1ca000c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 shutdown_connections 2023-11-06T03:04:18.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c88065a60 0x7f1c88067f20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ca40cb5b0 0x7f1ca4205e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ca40c93d0 0x7f1ca40b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 --2- 172.21.15.157:0/1929046624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ca40c5140 0x7f1ca4070c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 >> 172.21.15.157:0/1929046624 conn(0x7f1ca40b0160 msgr2=0x7f1ca40cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 shutdown_connections 2023-11-06T03:04:18.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f1caa89e640 1 -- 172.21.15.157:0/1929046624 wait complete. 2023-11-06T03:04:18.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 msgr2=0x7f0948067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 0x7f0948067ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f09580079f0 tx=0x7f0958008040 comp rx=0 tx=0).stop 2023-11-06T03:04:18.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 msgr2=0x7f095c0d66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 0x7f095c0d66d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f095000d5f0 tx=0x7f095000dab0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 shutdown_connections 2023-11-06T03:04:18.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065b30 0x7f0948067ff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f095c0e7710 0x7f095c0d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f095c006880 0x7f095c0d66d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 --2- 172.21.15.157:0/634135051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f095c005ee0 0x7f095c038920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 >> 172.21.15.157:0/634135051 conn(0x7f095c054b10 msgr2=0x7f095c0479b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 shutdown_connections 2023-11-06T03:04:18.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.725+0000 7f0969848640 1 -- 172.21.15.157:0/634135051 wait complete. 2023-11-06T03:04:18.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-06T03:04:18.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-06T03:04:18.745 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.746 DEBUG:teuthology.parallel:result is None 2023-11-06T03:04:18.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T03:04:18.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:18.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 Processor -- start 2023-11-06T03:04:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 -- start start 2023-11-06T03:04:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8dc0ced70 0x7fc8dc161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc06f470 con 0x7fc8dc0b1910 2023-11-06T03:04:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0b1df0 con 0x7fc8dc160db0 2023-11-06T03:04:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e2a64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc06f6c0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e1001640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc8dc0ced70 0x7fc8dc161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8dbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc8dc0b1910 0x7fc8dc0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52512/0 (socket says 172.21.15.157:52512) 2023-11-06T03:04:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8dbfff640 1 -- 172.21.15.157:0/752383284 learned_addr learned my addr 172.21.15.157:0/752383284 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e1001640 1 -- 172.21.15.157:0/752383284 >> v1:172.21.15.195:6789/0 conn(0x7fc8dc160db0 legacy=0x7fc8dc06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e1001640 1 -- 172.21.15.157:0/752383284 >> v1:172.21.15.157:6789/0 conn(0x7fc8dc0b1910 legacy=0x7fc8dc0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.893+0000 7fc8e1001640 1 -- 172.21.15.157:0/752383284 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc06f9a0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e1001640 1 --2- 172.21.15.157:0/752383284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc161190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009dc0 tx=0x7fc8d401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af04ad33fc13f84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8db7fe640 1 -- 172.21.15.157:0/752383284 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a5980 con 0x7fc8dc0ced70 2023-11-06T03:04:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8db7fe640 1 -- 172.21.15.157:0/752383284 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40a5ae0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8db7fe640 1 -- 172.21.15.157:0/752383284 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a5e70 con 0x7fc8dc0ced70 2023-11-06T03:04:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- 172.21.15.157:0/752383284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 msgr2=0x7fc8dc161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/752383284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc161190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009dc0 tx=0x7fc8d401c8a0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- 172.21.15.157:0/752383284 shutdown_connections 2023-11-06T03:04:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/752383284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc161190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- 172.21.15.157:0/752383284 >> 172.21.15.157:0/752383284 conn(0x7fc8dc065930 msgr2=0x7fc8dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- 172.21.15.157:0/752383284 shutdown_connections 2023-11-06T03:04:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- 172.21.15.157:0/752383284 wait complete. 2023-11-06T03:04:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 Processor -- start 2023-11-06T03:04:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- start start 2023-11-06T03:04:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8dc0b1910 0x7fc8dc157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8dc160db0 0x7fc8dc141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc158290 con 0x7fc8dc0ced70 2023-11-06T03:04:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc1578c0 con 0x7fc8dc0b1910 2023-11-06T03:04:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8e2a64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc144d70 con 0x7fc8dc160db0 2023-11-06T03:04:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35550/0 (socket says 172.21.15.157:35550) 2023-11-06T03:04:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 -- 172.21.15.157:0/450470223 learned_addr learned my addr 172.21.15.157:0/450470223 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 -- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8dc160db0 msgr2=0x7fc8dc141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8dc160db0 0x7fc8dc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 -- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8dc0b1910 msgr2=0x7fc8dc157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:18.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8dc0b1910 0x7fc8dc157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.897+0000 7fc8dbfff640 1 -- 172.21.15.157:0/450470223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4009a70 con 0x7fc8dc0ced70 2023-11-06T03:04:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8dbfff640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c450 tx=0x7fc8d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0005cf0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0005e50 con 0x7fc8dc0ced70 2023-11-06T03:04:18.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc1572b0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0007270 con 0x7fc8dc0ced70 2023-11-06T03:04:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc157550 con 0x7fc8dc0ced70 2023-11-06T03:04:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc072750 con 0x7fc8dc0ced70 2023-11-06T03:04:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc8d00073d0 con 0x7fc8dc0ced70 2023-11-06T03:04:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 0x7fc8c8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8d0082700 con 0x7fc8dc0ced70 2023-11-06T03:04:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8e0800640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 0x7fc8c8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.901+0000 7fc8e0800640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 0x7fc8c8067f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000f70 tx=0x7fc8cc0079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:18.905+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc8d0050010 con 0x7fc8dc0ced70 2023-11-06T03:04:19.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.041+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8dc157a00 con 0x7fc8c8065a90 2023-11-06T03:04:19.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.065+0000 7fc8d97fa640 1 -- 172.21.15.157:0/450470223 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286555 (secure 0 0 0) 0x7fc8cc005080 con 0x7fc8c8065a90 2023-11-06T03:04:19.067 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:19.069 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:19.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 msgr2=0x7fc8c8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 0x7fc8c8067f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000f70 tx=0x7fc8cc0079e0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 msgr2=0x7fc8dc14b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c450 tx=0x7fc8d000c910 comp rx=0 tx=0).stop 2023-11-06T03:04:19.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 shutdown_connections 2023-11-06T03:04:19.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc8c8065a90 0x7fc8c8067f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8dc160db0 0x7fc8dc141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8dc0ced70 0x7fc8dc14b9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 --2- 172.21.15.157:0/450470223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8dc0b1910 0x7fc8dc157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 >> 172.21.15.157:0/450470223 conn(0x7fc8dc065930 msgr2=0x7fc8dc15eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 shutdown_connections 2023-11-06T03:04:19.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.069+0000 7fc8e2a64640 1 -- 172.21.15.157:0/450470223 wait complete. 2023-11-06T03:04:19.083 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-06T03:04:17.097242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251484,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723236,"statfs":{"total":773094113280,"available":772836593664,"internally_reserved":0,"allocated":3174400,"data_stored":1899195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003986"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314154+0000","last_change":"2023-11-06T03:04:06.723013+0000","last_active":"2023-11-06T03:04:08.314154+0000","last_peered":"2023-11-06T03:04:08.314154+0000","last_clean":"2023-11-06T03:04:08.314154+0000","last_became_active":"2023-11-06T03:04:06.722313+0000","last_became_peered":"2023-11-06T03:04:06.722313+0000","last_unstale":"2023-11-06T03:04:08.314154+0000","last_undegraded":"2023-11-06T03:04:08.314154+0000","last_fullsized":"2023-11-06T03:04:08.314154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:17.965940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313638+0000","last_change":"2023-11-06T03:04:06.709695+0000","last_active":"2023-11-06T03:04:08.313638+0000","last_peered":"2023-11-06T03:04:08.313638+0000","last_clean":"2023-11-06T03:04:08.313638+0000","last_became_active":"2023-11-06T03:04:06.709429+0000","last_became_peered":"2023-11-06T03:04:06.709429+0000","last_unstale":"2023-11-06T03:04:08.313638+0000","last_undegraded":"2023-11-06T03:04:08.313638+0000","last_fullsized":"2023-11-06T03:04:08.313638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697946+0000","last_change":"2023-11-06T03:04:06.700734+0000","last_active":"2023-11-06T03:04:07.697946+0000","last_peered":"2023-11-06T03:04:07.697946+0000","last_clean":"2023-11-06T03:04:07.697946+0000","last_became_active":"2023-11-06T03:04:06.699143+0000","last_became_peered":"2023-11-06T03:04:06.699143+0000","last_unstale":"2023-11-06T03:04:07.697946+0000","last_undegraded":"2023-11-06T03:04:07.697946+0000","last_fullsized":"2023-11-06T03:04:07.697946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315057+0000","last_change":"2023-11-06T03:04:06.725804+0000","last_active":"2023-11-06T03:04:08.315057+0000","last_peered":"2023-11-06T03:04:08.315057+0000","last_clean":"2023-11-06T03:04:08.315057+0000","last_became_active":"2023-11-06T03:04:06.725496+0000","last_became_peered":"2023-11-06T03:04:06.725496+0000","last_unstale":"2023-11-06T03:04:08.315057+0000","last_undegraded":"2023-11-06T03:04:08.315057+0000","last_fullsized":"2023-11-06T03:04:08.315057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:30.176058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315579+0000","last_change":"2023-11-06T03:04:06.725612+0000","last_active":"2023-11-06T03:04:08.315579+0000","last_peered":"2023-11-06T03:04:08.315579+0000","last_clean":"2023-11-06T03:04:08.315579+0000","last_became_active":"2023-11-06T03:04:06.725099+0000","last_became_peered":"2023-11-06T03:04:06.725099+0000","last_unstale":"2023-11-06T03:04:08.315579+0000","last_undegraded":"2023-11-06T03:04:08.315579+0000","last_fullsized":"2023-11-06T03:04:08.315579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697168+0000","last_change":"2023-11-06T03:04:06.710419+0000","last_active":"2023-11-06T03:04:07.697168+0000","last_peered":"2023-11-06T03:04:07.697168+0000","last_clean":"2023-11-06T03:04:07.697168+0000","last_became_active":"2023-11-06T03:04:06.709958+0000","last_became_peered":"2023-11-06T03:04:06.709958+0000","last_unstale":"2023-11-06T03:04:07.697168+0000","last_undegraded":"2023-11-06T03:04:07.697168+0000","last_fullsized":"2023-11-06T03:04:07.697168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:58.848613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811621+0000","last_change":"2023-11-06T03:04:06.720209+0000","last_active":"2023-11-06T03:04:07.811621+0000","last_peered":"2023-11-06T03:04:07.811621+0000","last_clean":"2023-11-06T03:04:07.811621+0000","last_became_active":"2023-11-06T03:04:06.719857+0000","last_became_peered":"2023-11-06T03:04:06.719857+0000","last_unstale":"2023-11-06T03:04:07.811621+0000","last_undegraded":"2023-11-06T03:04:07.811621+0000","last_fullsized":"2023-11-06T03:04:07.811621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313002+0000","last_change":"2023-11-06T03:04:06.706215+0000","last_active":"2023-11-06T03:04:08.313002+0000","last_peered":"2023-11-06T03:04:08.313002+0000","last_clean":"2023-11-06T03:04:08.313002+0000","last_became_active":"2023-11-06T03:04:06.705329+0000","last_became_peered":"2023-11-06T03:04:06.705329+0000","last_unstale":"2023-11-06T03:04:08.313002+0000","last_undegraded":"2023-11-06T03:04:08.313002+0000","last_fullsized":"2023-11-06T03:04:08.313002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315584+0000","last_change":"2023-11-06T03:04:06.725727+0000","last_active":"2023-11-06T03:04:08.315584+0000","last_peered":"2023-11-06T03:04:08.315584+0000","last_clean":"2023-11-06T03:04:08.315584+0000","last_became_active":"2023-11-06T03:04:06.725241+0000","last_became_peered":"2023-11-06T03:04:06.725241+0000","last_unstale":"2023-11-06T03:04:08.315584+0000","last_undegraded":"2023-11-06T03:04:08.315584+0000","last_fullsized":"2023-11-06T03:04:08.315584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313982+0000","last_change":"2023-11-06T03:04:06.701948+0000","last_active":"2023-11-06T03:04:08.313982+0000","last_peered":"2023-11-06T03:04:08.313982+0000","last_clean":"2023-11-06T03:04:08.313982+0000","last_became_active":"2023-11-06T03:04:06.701405+0000","last_became_peered":"2023-11-06T03:04:06.701405+0000","last_unstale":"2023-11-06T03:04:08.313982+0000","last_undegraded":"2023-11-06T03:04:08.313982+0000","last_fullsized":"2023-11-06T03:04:08.313982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698195+0000","last_change":"2023-11-06T03:04:06.711152+0000","last_active":"2023-11-06T03:04:07.698195+0000","last_peered":"2023-11-06T03:04:07.698195+0000","last_clean":"2023-11-06T03:04:07.698195+0000","last_became_active":"2023-11-06T03:04:06.710840+0000","last_became_peered":"2023-11-06T03:04:06.710840+0000","last_unstale":"2023-11-06T03:04:07.698195+0000","last_undegraded":"2023-11-06T03:04:07.698195+0000","last_fullsized":"2023-11-06T03:04:07.698195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314771+0000","last_change":"2023-11-06T03:04:06.703107+0000","last_active":"2023-11-06T03:04:08.314771+0000","last_peered":"2023-11-06T03:04:08.314771+0000","last_clean":"2023-11-06T03:04:08.314771+0000","last_became_active":"2023-11-06T03:04:06.702533+0000","last_became_peered":"2023-11-06T03:04:06.702533+0000","last_unstale":"2023-11-06T03:04:08.314771+0000","last_undegraded":"2023-11-06T03:04:08.314771+0000","last_fullsized":"2023-11-06T03:04:08.314771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314211+0000","last_change":"2023-11-06T03:04:06.710998+0000","last_active":"2023-11-06T03:04:08.314211+0000","last_peered":"2023-11-06T03:04:08.314211+0000","last_clean":"2023-11-06T03:04:08.314211+0000","last_became_active":"2023-11-06T03:04:06.710459+0000","last_became_peered":"2023-11-06T03:04:06.710459+0000","last_unstale":"2023-11-06T03:04:08.314211+0000","last_undegraded":"2023-11-06T03:04:08.314211+0000","last_fullsized":"2023-11-06T03:04:08.314211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:36.030631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313983+0000","last_change":"2023-11-06T03:04:06.708520+0000","last_active":"2023-11-06T03:04:08.313983+0000","last_peered":"2023-11-06T03:04:08.313983+0000","last_clean":"2023-11-06T03:04:08.313983+0000","last_became_active":"2023-11-06T03:04:06.708200+0000","last_became_peered":"2023-11-06T03:04:06.708200+0000","last_unstale":"2023-11-06T03:04:08.313983+0000","last_undegraded":"2023-11-06T03:04:08.313983+0000","last_fullsized":"2023-11-06T03:04:08.313983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811766+0000","last_change":"2023-11-06T03:04:06.723418+0000","last_active":"2023-11-06T03:04:07.811766+0000","last_peered":"2023-11-06T03:04:07.811766+0000","last_clean":"2023-11-06T03:04:07.811766+0000","last_became_active":"2023-11-06T03:04:06.720428+0000","last_became_peered":"2023-11-06T03:04:06.720428+0000","last_unstale":"2023-11-06T03:04:07.811766+0000","last_undegraded":"2023-11-06T03:04:07.811766+0000","last_fullsized":"2023-11-06T03:04:07.811766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697132+0000","last_change":"2023-11-06T03:04:06.699686+0000","last_active":"2023-11-06T03:04:07.697132+0000","last_peered":"2023-11-06T03:04:07.697132+0000","last_clean":"2023-11-06T03:04:07.697132+0000","last_became_active":"2023-11-06T03:04:06.699150+0000","last_became_peered":"2023-11-06T03:04:06.699150+0000","last_unstale":"2023-11-06T03:04:07.697132+0000","last_undegraded":"2023-11-06T03:04:07.697132+0000","last_fullsized":"2023-11-06T03:04:07.697132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315781+0000","last_change":"2023-11-06T03:04:06.718065+0000","last_active":"2023-11-06T03:04:08.315781+0000","last_peered":"2023-11-06T03:04:08.315781+0000","last_clean":"2023-11-06T03:04:08.315781+0000","last_became_active":"2023-11-06T03:04:06.717495+0000","last_became_peered":"2023-11-06T03:04:06.717495+0000","last_unstale":"2023-11-06T03:04:08.315781+0000","last_undegraded":"2023-11-06T03:04:08.315781+0000","last_fullsized":"2023-11-06T03:04:08.315781+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697937+0000","last_change":"2023-11-06T03:04:06.707851+0000","last_active":"2023-11-06T03:04:07.697937+0000","last_peered":"2023-11-06T03:04:07.697937+0000","last_clean":"2023-11-06T03:04:07.697937+0000","last_became_active":"2023-11-06T03:04:06.707587+0000","last_became_peered":"2023-11-06T03:04:06.707587+0000","last_unstale":"2023-11-06T03:04:07.697937+0000","last_undegraded":"2023-11-06T03:04:07.697937+0000","last_fullsized":"2023-11-06T03:04:07.697937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:52:19.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314357+0000","last_change":"2023-11-06T03:04:06.724261+0000","last_active":"2023-11-06T03:04:08.314357+0000","last_peered":"2023-11-06T03:04:08.314357+0000","last_clean":"2023-11-06T03:04:08.314357+0000","last_became_active":"2023-11-06T03:04:06.723974+0000","last_became_peered":"2023-11-06T03:04:06.723974+0000","last_unstale":"2023-11-06T03:04:08.314357+0000","last_undegraded":"2023-11-06T03:04:08.314357+0000","last_fullsized":"2023-11-06T03:04:08.314357+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:16.954584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697610+0000","last_change":"2023-11-06T03:04:06.729633+0000","last_active":"2023-11-06T03:04:07.697610+0000","last_peered":"2023-11-06T03:04:07.697610+0000","last_clean":"2023-11-06T03:04:07.697610+0000","last_became_active":"2023-11-06T03:04:06.729241+0000","last_became_peered":"2023-11-06T03:04:06.729241+0000","last_unstale":"2023-11-06T03:04:07.697610+0000","last_undegraded":"2023-11-06T03:04:07.697610+0000","last_fullsized":"2023-11-06T03:04:07.697610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:05.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811862+0000","last_change":"2023-11-06T03:04:06.728388+0000","last_active":"2023-11-06T03:04:07.811862+0000","last_peered":"2023-11-06T03:04:07.811862+0000","last_clean":"2023-11-06T03:04:07.811862+0000","last_became_active":"2023-11-06T03:04:06.728135+0000","last_became_peered":"2023-11-06T03:04:06.728135+0000","last_unstale":"2023-11-06T03:04:07.811862+0000","last_undegraded":"2023-11-06T03:04:07.811862+0000","last_fullsized":"2023-11-06T03:04:07.811862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315482+0000","last_change":"2023-11-06T03:04:06.713228+0000","last_active":"2023-11-06T03:04:08.315482+0000","last_peered":"2023-11-06T03:04:08.315482+0000","last_clean":"2023-11-06T03:04:08.315482+0000","last_became_active":"2023-11-06T03:04:06.712822+0000","last_became_peered":"2023-11-06T03:04:06.712822+0000","last_unstale":"2023-11-06T03:04:08.315482+0000","last_undegraded":"2023-11-06T03:04:08.315482+0000","last_fullsized":"2023-11-06T03:04:08.315482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696649+0000","last_change":"2023-11-06T03:04:06.711437+0000","last_active":"2023-11-06T03:04:07.696649+0000","last_peered":"2023-11-06T03:04:07.696649+0000","last_clean":"2023-11-06T03:04:07.696649+0000","last_became_active":"2023-11-06T03:04:06.710409+0000","last_became_peered":"2023-11-06T03:04:06.710409+0000","last_unstale":"2023-11-06T03:04:07.696649+0000","last_undegraded":"2023-11-06T03:04:07.696649+0000","last_fullsized":"2023-11-06T03:04:07.696649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811784+0000","last_change":"2023-11-06T03:04:06.728415+0000","last_active":"2023-11-06T03:04:07.811784+0000","last_peered":"2023-11-06T03:04:07.811784+0000","last_clean":"2023-11-06T03:04:07.811784+0000","last_became_active":"2023-11-06T03:04:06.728111+0000","last_became_peered":"2023-11-06T03:04:06.728111+0000","last_unstale":"2023-11-06T03:04:07.811784+0000","last_undegraded":"2023-11-06T03:04:07.811784+0000","last_fullsized":"2023-11-06T03:04:07.811784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317080+0000","last_change":"2023-11-06T03:04:06.727933+0000","last_active":"2023-11-06T03:04:08.317080+0000","last_peered":"2023-11-06T03:04:08.317080+0000","last_clean":"2023-11-06T03:04:08.317080+0000","last_became_active":"2023-11-06T03:04:06.727689+0000","last_became_peered":"2023-11-06T03:04:06.727689+0000","last_unstale":"2023-11-06T03:04:08.317080+0000","last_undegraded":"2023-11-06T03:04:08.317080+0000","last_fullsized":"2023-11-06T03:04:08.317080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317243+0000","last_change":"2023-11-06T03:04:06.709146+0000","last_active":"2023-11-06T03:04:08.317243+0000","last_peered":"2023-11-06T03:04:08.317243+0000","last_clean":"2023-11-06T03:04:08.317243+0000","last_became_active":"2023-11-06T03:04:06.708760+0000","last_became_peered":"2023-11-06T03:04:06.708760+0000","last_unstale":"2023-11-06T03:04:08.317243+0000","last_undegraded":"2023-11-06T03:04:08.317243+0000","last_fullsized":"2023-11-06T03:04:08.317243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313830+0000","last_change":"2023-11-06T03:04:06.715622+0000","last_active":"2023-11-06T03:04:08.313830+0000","last_peered":"2023-11-06T03:04:08.313830+0000","last_clean":"2023-11-06T03:04:08.313830+0000","last_became_active":"2023-11-06T03:04:06.715290+0000","last_became_peered":"2023-11-06T03:04:06.715290+0000","last_unstale":"2023-11-06T03:04:08.313830+0000","last_undegraded":"2023-11-06T03:04:08.313830+0000","last_fullsized":"2023-11-06T03:04:08.313830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316817+0000","last_change":"2023-11-06T03:04:06.709608+0000","last_active":"2023-11-06T03:04:08.316817+0000","last_peered":"2023-11-06T03:04:08.316817+0000","last_clean":"2023-11-06T03:04:08.316817+0000","last_became_active":"2023-11-06T03:04:06.708904+0000","last_became_peered":"2023-11-06T03:04:06.708904+0000","last_unstale":"2023-11-06T03:04:08.316817+0000","last_undegraded":"2023-11-06T03:04:08.316817+0000","last_fullsized":"2023-11-06T03:04:08.316817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697175+0000","last_change":"2023-11-06T03:04:06.730619+0000","last_active":"2023-11-06T03:04:07.697175+0000","last_peered":"2023-11-06T03:04:07.697175+0000","last_clean":"2023-11-06T03:04:07.697175+0000","last_became_active":"2023-11-06T03:04:06.730192+0000","last_became_peered":"2023-11-06T03:04:06.730192+0000","last_unstale":"2023-11-06T03:04:07.697175+0000","last_undegraded":"2023-11-06T03:04:07.697175+0000","last_fullsized":"2023-11-06T03:04:07.697175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697306+0000","last_change":"2023-11-06T03:04:06.727570+0000","last_active":"2023-11-06T03:04:07.697306+0000","last_peered":"2023-11-06T03:04:07.697306+0000","last_clean":"2023-11-06T03:04:07.697306+0000","last_became_active":"2023-11-06T03:04:06.727305+0000","last_became_peered":"2023-11-06T03:04:06.727305+0000","last_unstale":"2023-11-06T03:04:07.697306+0000","last_undegraded":"2023-11-06T03:04:07.697306+0000","last_fullsized":"2023-11-06T03:04:07.697306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696670+0000","last_change":"2023-11-06T03:04:06.727300+0000","last_active":"2023-11-06T03:04:07.696670+0000","last_peered":"2023-11-06T03:04:07.696670+0000","last_clean":"2023-11-06T03:04:07.696670+0000","last_became_active":"2023-11-06T03:04:06.726311+0000","last_became_peered":"2023-11-06T03:04:06.726311+0000","last_unstale":"2023-11-06T03:04:07.696670+0000","last_undegraded":"2023-11-06T03:04:07.696670+0000","last_fullsized":"2023-11-06T03:04:07.696670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313347+0000","last_change":"2023-11-06T03:04:06.722186+0000","last_active":"2023-11-06T03:04:08.313347+0000","last_peered":"2023-11-06T03:04:08.313347+0000","last_clean":"2023-11-06T03:04:08.313347+0000","last_became_active":"2023-11-06T03:04:06.721514+0000","last_became_peered":"2023-11-06T03:04:06.721514+0000","last_unstale":"2023-11-06T03:04:08.313347+0000","last_undegraded":"2023-11-06T03:04:08.313347+0000","last_fullsized":"2023-11-06T03:04:08.313347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697317+0000","last_change":"2023-11-06T03:04:06.725709+0000","last_active":"2023-11-06T03:04:07.697317+0000","last_peered":"2023-11-06T03:04:07.697317+0000","last_clean":"2023-11-06T03:04:07.697317+0000","last_became_active":"2023-11-06T03:04:06.725248+0000","last_became_peered":"2023-11-06T03:04:06.725248+0000","last_unstale":"2023-11-06T03:04:07.697317+0000","last_undegraded":"2023-11-06T03:04:07.697317+0000","last_fullsized":"2023-11-06T03:04:07.697317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812176+0000","last_change":"2023-11-06T03:04:06.363773+0000","last_active":"2023-11-06T03:04:07.812176+0000","last_peered":"2023-11-06T03:04:07.812176+0000","last_clean":"2023-11-06T03:04:07.812176+0000","last_became_active":"2023-11-06T03:04:06.362996+0000","last_became_peered":"2023-11-06T03:04:06.362996+0000","last_unstale":"2023-11-06T03:04:07.812176+0000","last_undegraded":"2023-11-06T03:04:07.812176+0000","last_fullsized":"2023-11-06T03:04:07.812176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314708+0000","last_change":"2023-11-06T03:04:06.709102+0000","last_active":"2023-11-06T03:04:08.314708+0000","last_peered":"2023-11-06T03:04:08.314708+0000","last_clean":"2023-11-06T03:04:08.314708+0000","last_became_active":"2023-11-06T03:04:06.708817+0000","last_became_peered":"2023-11-06T03:04:06.708817+0000","last_unstale":"2023-11-06T03:04:08.314708+0000","last_undegraded":"2023-11-06T03:04:08.314708+0000","last_fullsized":"2023-11-06T03:04:08.314708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315524+0000","last_change":"2023-11-06T03:04:06.351736+0000","last_active":"2023-11-06T03:04:08.315524+0000","last_peered":"2023-11-06T03:04:08.315524+0000","last_clean":"2023-11-06T03:04:08.315524+0000","last_became_active":"2023-11-06T03:04:06.351500+0000","last_became_peered":"2023-11-06T03:04:06.351500+0000","last_unstale":"2023-11-06T03:04:08.315524+0000","last_undegraded":"2023-11-06T03:04:08.315524+0000","last_fullsized":"2023-11-06T03:04:08.315524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316022+0000","last_change":"2023-11-06T03:04:06.719843+0000","last_active":"2023-11-06T03:04:08.316022+0000","last_peered":"2023-11-06T03:04:08.316022+0000","last_clean":"2023-11-06T03:04:08.316022+0000","last_became_active":"2023-11-06T03:04:06.718738+0000","last_became_peered":"2023-11-06T03:04:06.718738+0000","last_unstale":"2023-11-06T03:04:08.316022+0000","last_undegraded":"2023-11-06T03:04:08.316022+0000","last_fullsized":"2023-11-06T03:04:08.316022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812188+0000","last_change":"2023-11-06T03:04:06.364044+0000","last_active":"2023-11-06T03:04:07.812188+0000","last_peered":"2023-11-06T03:04:07.812188+0000","last_clean":"2023-11-06T03:04:07.812188+0000","last_became_active":"2023-11-06T03:04:06.363453+0000","last_became_peered":"2023-11-06T03:04:06.363453+0000","last_unstale":"2023-11-06T03:04:07.812188+0000","last_undegraded":"2023-11-06T03:04:07.812188+0000","last_fullsized":"2023-11-06T03:04:07.812188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697122+0000","last_change":"2023-11-06T03:04:06.732024+0000","last_active":"2023-11-06T03:04:07.697122+0000","last_peered":"2023-11-06T03:04:07.697122+0000","last_clean":"2023-11-06T03:04:07.697122+0000","last_became_active":"2023-11-06T03:04:06.730677+0000","last_became_peered":"2023-11-06T03:04:06.730677+0000","last_unstale":"2023-11-06T03:04:07.697122+0000","last_undegraded":"2023-11-06T03:04:07.697122+0000","last_fullsized":"2023-11-06T03:04:07.697122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313367+0000","last_change":"2023-11-06T03:04:06.333322+0000","last_active":"2023-11-06T03:04:08.313367+0000","last_peered":"2023-11-06T03:04:08.313367+0000","last_clean":"2023-11-06T03:04:08.313367+0000","last_became_active":"2023-11-06T03:04:06.332581+0000","last_became_peered":"2023-11-06T03:04:06.332581+0000","last_unstale":"2023-11-06T03:04:08.313367+0000","last_undegraded":"2023-11-06T03:04:08.313367+0000","last_fullsized":"2023-11-06T03:04:08.313367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812463+0000","last_change":"2023-11-06T03:04:06.363547+0000","last_active":"2023-11-06T03:04:07.812463+0000","last_peered":"2023-11-06T03:04:07.812463+0000","last_clean":"2023-11-06T03:04:07.812463+0000","last_became_active":"2023-11-06T03:04:06.362774+0000","last_became_peered":"2023-11-06T03:04:06.362774+0000","last_unstale":"2023-11-06T03:04:07.812463+0000","last_undegraded":"2023-11-06T03:04:07.812463+0000","last_fullsized":"2023-11-06T03:04:07.812463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313617+0000","last_change":"2023-11-06T03:04:06.715341+0000","last_active":"2023-11-06T03:04:08.313617+0000","last_peered":"2023-11-06T03:04:08.313617+0000","last_clean":"2023-11-06T03:04:08.313617+0000","last_became_active":"2023-11-06T03:04:06.715020+0000","last_became_peered":"2023-11-06T03:04:06.715020+0000","last_unstale":"2023-11-06T03:04:08.313617+0000","last_undegraded":"2023-11-06T03:04:08.313617+0000","last_fullsized":"2023-11-06T03:04:08.313617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812298+0000","last_change":"2023-11-06T03:04:06.364333+0000","last_active":"2023-11-06T03:04:07.812298+0000","last_peered":"2023-11-06T03:04:07.812298+0000","last_clean":"2023-11-06T03:04:07.812298+0000","last_became_active":"2023-11-06T03:04:06.363641+0000","last_became_peered":"2023-11-06T03:04:06.363641+0000","last_unstale":"2023-11-06T03:04:07.812298+0000","last_undegraded":"2023-11-06T03:04:07.812298+0000","last_fullsized":"2023-11-06T03:04:07.812298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:29.165293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697141+0000","last_change":"2023-11-06T03:04:06.712796+0000","last_active":"2023-11-06T03:04:07.697141+0000","last_peered":"2023-11-06T03:04:07.697141+0000","last_clean":"2023-11-06T03:04:07.697141+0000","last_became_active":"2023-11-06T03:04:06.712442+0000","last_became_peered":"2023-11-06T03:04:06.712442+0000","last_unstale":"2023-11-06T03:04:07.697141+0000","last_undegraded":"2023-11-06T03:04:07.697141+0000","last_fullsized":"2023-11-06T03:04:07.697141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812223+0000","last_change":"2023-11-06T03:04:06.352109+0000","last_active":"2023-11-06T03:04:07.812223+0000","last_peered":"2023-11-06T03:04:07.812223+0000","last_clean":"2023-11-06T03:04:07.812223+0000","last_became_active":"2023-11-06T03:04:06.351112+0000","last_became_peered":"2023-11-06T03:04:06.351112+0000","last_unstale":"2023-11-06T03:04:07.812223+0000","last_undegraded":"2023-11-06T03:04:07.812223+0000","last_fullsized":"2023-11-06T03:04:07.812223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:27.571905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313304+0000","last_change":"2023-11-06T03:04:06.721884+0000","last_active":"2023-11-06T03:04:08.313304+0000","last_peered":"2023-11-06T03:04:08.313304+0000","last_clean":"2023-11-06T03:04:08.313304+0000","last_became_active":"2023-11-06T03:04:06.721714+0000","last_became_peered":"2023-11-06T03:04:06.721714+0000","last_unstale":"2023-11-06T03:04:08.313304+0000","last_undegraded":"2023-11-06T03:04:08.313304+0000","last_fullsized":"2023-11-06T03:04:08.313304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:04.132219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313920+0000","last_change":"2023-11-06T03:04:06.336342+0000","last_active":"2023-11-06T03:04:08.313920+0000","last_peered":"2023-11-06T03:04:08.313920+0000","last_clean":"2023-11-06T03:04:08.313920+0000","last_became_active":"2023-11-06T03:04:06.335639+0000","last_became_peered":"2023-11-06T03:04:06.335639+0000","last_unstale":"2023-11-06T03:04:08.313920+0000","last_undegraded":"2023-11-06T03:04:08.313920+0000","last_fullsized":"2023-11-06T03:04:08.313920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:15.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315124+0000","last_change":"2023-11-06T03:04:06.709502+0000","last_active":"2023-11-06T03:04:08.315124+0000","last_peered":"2023-11-06T03:04:08.315124+0000","last_clean":"2023-11-06T03:04:08.315124+0000","last_became_active":"2023-11-06T03:04:06.709157+0000","last_became_peered":"2023-11-06T03:04:06.709157+0000","last_unstale":"2023-11-06T03:04:08.315124+0000","last_undegraded":"2023-11-06T03:04:08.315124+0000","last_fullsized":"2023-11-06T03:04:08.315124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697206+0000","last_change":"2023-11-06T03:04:06.331517+0000","last_active":"2023-11-06T03:04:07.697206+0000","last_peered":"2023-11-06T03:04:07.697206+0000","last_clean":"2023-11-06T03:04:07.697206+0000","last_became_active":"2023-11-06T03:04:06.331107+0000","last_became_peered":"2023-11-06T03:04:06.331107+0000","last_unstale":"2023-11-06T03:04:07.697206+0000","last_undegraded":"2023-11-06T03:04:07.697206+0000","last_fullsized":"2023-11-06T03:04:07.697206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697752+0000","last_change":"2023-11-06T03:04:06.720549+0000","last_active":"2023-11-06T03:04:07.697752+0000","last_peered":"2023-11-06T03:04:07.697752+0000","last_clean":"2023-11-06T03:04:07.697752+0000","last_became_active":"2023-11-06T03:04:06.719939+0000","last_became_peered":"2023-11-06T03:04:06.719939+0000","last_unstale":"2023-11-06T03:04:07.697752+0000","last_undegraded":"2023-11-06T03:04:07.697752+0000","last_fullsized":"2023-11-06T03:04:07.697752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:34.877473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313501+0000","last_change":"2023-11-06T03:04:06.364318+0000","last_active":"2023-11-06T03:04:08.313501+0000","last_peered":"2023-11-06T03:04:08.313501+0000","last_clean":"2023-11-06T03:04:08.313501+0000","last_became_active":"2023-11-06T03:04:06.363095+0000","last_became_peered":"2023-11-06T03:04:06.363095+0000","last_unstale":"2023-11-06T03:04:08.313501+0000","last_undegraded":"2023-11-06T03:04:08.313501+0000","last_fullsized":"2023-11-06T03:04:08.313501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698194+0000","last_change":"2023-11-06T03:04:06.707941+0000","last_active":"2023-11-06T03:04:07.698194+0000","last_peered":"2023-11-06T03:04:07.698194+0000","last_clean":"2023-11-06T03:04:07.698194+0000","last_became_active":"2023-11-06T03:04:06.707628+0000","last_became_peered":"2023-11-06T03:04:06.707628+0000","last_unstale":"2023-11-06T03:04:07.698194+0000","last_undegraded":"2023-11-06T03:04:07.698194+0000","last_fullsized":"2023-11-06T03:04:07.698194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:55.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697420+0000","last_change":"2023-11-06T03:04:06.355002+0000","last_active":"2023-11-06T03:04:07.697420+0000","last_peered":"2023-11-06T03:04:07.697420+0000","last_clean":"2023-11-06T03:04:07.697420+0000","last_became_active":"2023-11-06T03:04:06.352902+0000","last_became_peered":"2023-11-06T03:04:06.352902+0000","last_unstale":"2023-11-06T03:04:07.697420+0000","last_undegraded":"2023-11-06T03:04:07.697420+0000","last_fullsized":"2023-11-06T03:04:07.697420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697853+0000","last_change":"2023-11-06T03:04:06.721894+0000","last_active":"2023-11-06T03:04:07.697853+0000","last_peered":"2023-11-06T03:04:07.697853+0000","last_clean":"2023-11-06T03:04:07.697853+0000","last_became_active":"2023-11-06T03:04:06.721619+0000","last_became_peered":"2023-11-06T03:04:06.721619+0000","last_unstale":"2023-11-06T03:04:07.697853+0000","last_undegraded":"2023-11-06T03:04:07.697853+0000","last_fullsized":"2023-11-06T03:04:07.697853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696727+0000","last_change":"2023-11-06T03:04:06.330685+0000","last_active":"2023-11-06T03:04:07.696727+0000","last_peered":"2023-11-06T03:04:07.696727+0000","last_clean":"2023-11-06T03:04:07.696727+0000","last_became_active":"2023-11-06T03:04:06.330390+0000","last_became_peered":"2023-11-06T03:04:06.330390+0000","last_unstale":"2023-11-06T03:04:07.696727+0000","last_undegraded":"2023-11-06T03:04:07.696727+0000","last_fullsized":"2023-11-06T03:04:07.696727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317295+0000","last_change":"2023-11-06T03:04:06.721960+0000","last_active":"2023-11-06T03:04:08.317295+0000","last_peered":"2023-11-06T03:04:08.317295+0000","last_clean":"2023-11-06T03:04:08.317295+0000","last_became_active":"2023-11-06T03:04:06.721497+0000","last_became_peered":"2023-11-06T03:04:06.721497+0000","last_unstale":"2023-11-06T03:04:08.317295+0000","last_undegraded":"2023-11-06T03:04:08.317295+0000","last_fullsized":"2023-11-06T03:04:08.317295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:28.582669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700445+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:06.700445+0000","last_peered":"2023-11-06T03:04:06.700445+0000","last_clean":"2023-11-06T03:04:06.700445+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:06.700445+0000","last_undegraded":"2023-11-06T03:04:06.700445+0000","last_fullsized":"2023-11-06T03:04:06.700445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700232+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:06.700232+0000","last_peered":"2023-11-06T03:04:06.700232+0000","last_clean":"2023-11-06T03:04:06.700232+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:06.700232+0000","last_undegraded":"2023-11-06T03:04:06.700232+0000","last_fullsized":"2023-11-06T03:04:06.700232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696999+0000","last_change":"2023-11-06T03:04:06.344186+0000","last_active":"2023-11-06T03:04:07.696999+0000","last_peered":"2023-11-06T03:04:07.696999+0000","last_clean":"2023-11-06T03:04:07.696999+0000","last_became_active":"2023-11-06T03:04:06.343854+0000","last_became_peered":"2023-11-06T03:04:06.343854+0000","last_unstale":"2023-11-06T03:04:07.696999+0000","last_undegraded":"2023-11-06T03:04:07.696999+0000","last_fullsized":"2023-11-06T03:04:07.696999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315060+0000","last_change":"2023-11-06T03:04:06.716182+0000","last_active":"2023-11-06T03:04:08.315060+0000","last_peered":"2023-11-06T03:04:08.315060+0000","last_clean":"2023-11-06T03:04:08.315060+0000","last_became_active":"2023-11-06T03:04:06.714829+0000","last_became_peered":"2023-11-06T03:04:06.714829+0000","last_unstale":"2023-11-06T03:04:08.315060+0000","last_undegraded":"2023-11-06T03:04:08.315060+0000","last_fullsized":"2023-11-06T03:04:08.315060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:20.005858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700210+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:06.700210+0000","last_peered":"2023-11-06T03:04:06.700210+0000","last_clean":"2023-11-06T03:04:06.700210+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:06.700210+0000","last_undegraded":"2023-11-06T03:04:06.700210+0000","last_fullsized":"2023-11-06T03:04:06.700210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314962+0000","last_change":"2023-11-06T03:04:06.359178+0000","last_active":"2023-11-06T03:04:08.314962+0000","last_peered":"2023-11-06T03:04:08.314962+0000","last_clean":"2023-11-06T03:04:08.314962+0000","last_became_active":"2023-11-06T03:04:06.358881+0000","last_became_peered":"2023-11-06T03:04:06.358881+0000","last_unstale":"2023-11-06T03:04:08.314962+0000","last_undegraded":"2023-11-06T03:04:08.314962+0000","last_fullsized":"2023-11-06T03:04:08.314962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316077+0000","last_change":"2023-11-06T03:04:06.712984+0000","last_active":"2023-11-06T03:04:08.316077+0000","last_peered":"2023-11-06T03:04:08.316077+0000","last_clean":"2023-11-06T03:04:08.316077+0000","last_became_active":"2023-11-06T03:04:06.712663+0000","last_became_peered":"2023-11-06T03:04:06.712663+0000","last_unstale":"2023-11-06T03:04:08.316077+0000","last_undegraded":"2023-11-06T03:04:08.316077+0000","last_fullsized":"2023-11-06T03:04:08.316077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.688745+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:06.688745+0000","last_peered":"2023-11-06T03:04:06.688745+0000","last_clean":"2023-11-06T03:04:06.688745+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:06.688745+0000","last_undegraded":"2023-11-06T03:04:06.688745+0000","last_fullsized":"2023-11-06T03:04:06.688745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314913+0000","last_change":"2023-11-06T03:04:06.328651+0000","last_active":"2023-11-06T03:04:08.314913+0000","last_peered":"2023-11-06T03:04:08.314913+0000","last_clean":"2023-11-06T03:04:08.314913+0000","last_became_active":"2023-11-06T03:04:06.328314+0000","last_became_peered":"2023-11-06T03:04:06.328314+0000","last_unstale":"2023-11-06T03:04:08.314913+0000","last_undegraded":"2023-11-06T03:04:08.314913+0000","last_fullsized":"2023-11-06T03:04:08.314913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317506+0000","last_change":"2023-11-06T03:04:06.710848+0000","last_active":"2023-11-06T03:04:08.317506+0000","last_peered":"2023-11-06T03:04:08.317506+0000","last_clean":"2023-11-06T03:04:08.317506+0000","last_became_active":"2023-11-06T03:04:06.709767+0000","last_became_peered":"2023-11-06T03:04:06.709767+0000","last_unstale":"2023-11-06T03:04:08.317506+0000","last_undegraded":"2023-11-06T03:04:08.317506+0000","last_fullsized":"2023-11-06T03:04:08.317506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700275+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:06.700275+0000","last_peered":"2023-11-06T03:04:06.700275+0000","last_clean":"2023-11-06T03:04:06.700275+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:06.700275+0000","last_undegraded":"2023-11-06T03:04:06.700275+0000","last_fullsized":"2023-11-06T03:04:06.700275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697593+0000","last_change":"2023-11-06T03:04:06.354390+0000","last_active":"2023-11-06T03:04:07.697593+0000","last_peered":"2023-11-06T03:04:07.697593+0000","last_clean":"2023-11-06T03:04:07.697593+0000","last_became_active":"2023-11-06T03:04:06.353638+0000","last_became_peered":"2023-11-06T03:04:06.353638+0000","last_unstale":"2023-11-06T03:04:07.697593+0000","last_undegraded":"2023-11-06T03:04:07.697593+0000","last_fullsized":"2023-11-06T03:04:07.697593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316563+0000","last_change":"2023-11-06T03:04:06.714255+0000","last_active":"2023-11-06T03:04:08.316563+0000","last_peered":"2023-11-06T03:04:08.316563+0000","last_clean":"2023-11-06T03:04:08.316563+0000","last_became_active":"2023-11-06T03:04:06.709791+0000","last_became_peered":"2023-11-06T03:04:06.709791+0000","last_unstale":"2023-11-06T03:04:08.316563+0000","last_undegraded":"2023-11-06T03:04:08.316563+0000","last_fullsized":"2023-11-06T03:04:08.316563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314303+0000","last_change":"2023-11-06T03:04:06.338556+0000","last_active":"2023-11-06T03:04:08.314303+0000","last_peered":"2023-11-06T03:04:08.314303+0000","last_clean":"2023-11-06T03:04:08.314303+0000","last_became_active":"2023-11-06T03:04:06.337740+0000","last_became_peered":"2023-11-06T03:04:06.337740+0000","last_unstale":"2023-11-06T03:04:08.314303+0000","last_undegraded":"2023-11-06T03:04:08.314303+0000","last_fullsized":"2023-11-06T03:04:08.314303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.701953+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:06.701953+0000","last_peered":"2023-11-06T03:04:06.701953+0000","last_clean":"2023-11-06T03:04:06.701953+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:06.701953+0000","last_undegraded":"2023-11-06T03:04:06.701953+0000","last_fullsized":"2023-11-06T03:04:06.701953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316863+0000","last_change":"2023-11-06T03:04:06.711498+0000","last_active":"2023-11-06T03:04:08.316863+0000","last_peered":"2023-11-06T03:04:08.316863+0000","last_clean":"2023-11-06T03:04:08.316863+0000","last_became_active":"2023-11-06T03:04:06.708625+0000","last_became_peered":"2023-11-06T03:04:06.708625+0000","last_unstale":"2023-11-06T03:04:08.316863+0000","last_undegraded":"2023-11-06T03:04:08.316863+0000","last_fullsized":"2023-11-06T03:04:08.316863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316721+0000","last_change":"2023-11-06T03:04:06.329676+0000","last_active":"2023-11-06T03:04:08.316721+0000","last_peered":"2023-11-06T03:04:08.316721+0000","last_clean":"2023-11-06T03:04:08.316721+0000","last_became_active":"2023-11-06T03:04:06.329333+0000","last_became_peered":"2023-11-06T03:04:06.329333+0000","last_unstale":"2023-11-06T03:04:08.316721+0000","last_undegraded":"2023-11-06T03:04:08.316721+0000","last_fullsized":"2023-11-06T03:04:08.316721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.703855+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:06.703855+0000","last_peered":"2023-11-06T03:04:06.703855+0000","last_clean":"2023-11-06T03:04:06.703855+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:06.703855+0000","last_undegraded":"2023-11-06T03:04:06.703855+0000","last_fullsized":"2023-11-06T03:04:06.703855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315277+0000","last_change":"2023-11-06T03:04:06.701264+0000","last_active":"2023-11-06T03:04:08.315277+0000","last_peered":"2023-11-06T03:04:08.315277+0000","last_clean":"2023-11-06T03:04:08.315277+0000","last_became_active":"2023-11-06T03:04:06.700955+0000","last_became_peered":"2023-11-06T03:04:06.700955+0000","last_unstale":"2023-11-06T03:04:08.315277+0000","last_undegraded":"2023-11-06T03:04:08.315277+0000","last_fullsized":"2023-11-06T03:04:08.315277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697348+0000","last_change":"2023-11-06T03:04:06.346267+0000","last_active":"2023-11-06T03:04:07.697348+0000","last_peered":"2023-11-06T03:04:07.697348+0000","last_clean":"2023-11-06T03:04:07.697348+0000","last_became_active":"2023-11-06T03:04:06.345991+0000","last_became_peered":"2023-11-06T03:04:06.345991+0000","last_unstale":"2023-11-06T03:04:07.697348+0000","last_undegraded":"2023-11-06T03:04:07.697348+0000","last_fullsized":"2023-11-06T03:04:07.697348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.702101+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:06.702101+0000","last_peered":"2023-11-06T03:04:06.702101+0000","last_clean":"2023-11-06T03:04:06.702101+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:06.702101+0000","last_undegraded":"2023-11-06T03:04:06.702101+0000","last_fullsized":"2023-11-06T03:04:06.702101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696622+0000","last_change":"2023-11-06T03:04:06.731112+0000","last_active":"2023-11-06T03:04:07.696622+0000","last_peered":"2023-11-06T03:04:07.696622+0000","last_clean":"2023-11-06T03:04:07.696622+0000","last_became_active":"2023-11-06T03:04:06.730152+0000","last_became_peered":"2023-11-06T03:04:06.730152+0000","last_unstale":"2023-11-06T03:04:07.696622+0000","last_undegraded":"2023-11-06T03:04:07.696622+0000","last_fullsized":"2023-11-06T03:04:07.696622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316243+0000","last_change":"2023-11-06T03:04:06.331522+0000","last_active":"2023-11-06T03:04:08.316243+0000","last_peered":"2023-11-06T03:04:08.316243+0000","last_clean":"2023-11-06T03:04:08.316243+0000","last_became_active":"2023-11-06T03:04:06.331263+0000","last_became_peered":"2023-11-06T03:04:06.331263+0000","last_unstale":"2023-11-06T03:04:08.316243+0000","last_undegraded":"2023-11-06T03:04:08.316243+0000","last_fullsized":"2023-11-06T03:04:08.316243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.725653+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:06.725653+0000","last_peered":"2023-11-06T03:04:06.725653+0000","last_clean":"2023-11-06T03:04:06.725653+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:06.725653+0000","last_undegraded":"2023-11-06T03:04:06.725653+0000","last_fullsized":"2023-11-06T03:04:06.725653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698182+0000","last_change":"2023-11-06T03:04:06.705426+0000","last_active":"2023-11-06T03:04:07.698182+0000","last_peered":"2023-11-06T03:04:07.698182+0000","last_clean":"2023-11-06T03:04:07.698182+0000","last_became_active":"2023-11-06T03:04:06.705116+0000","last_became_peered":"2023-11-06T03:04:06.705116+0000","last_unstale":"2023-11-06T03:04:07.698182+0000","last_undegraded":"2023-11-06T03:04:07.698182+0000","last_fullsized":"2023-11-06T03:04:07.698182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:26.686747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697675+0000","last_change":"2023-11-06T03:04:06.721464+0000","last_active":"2023-11-06T03:04:07.697675+0000","last_peered":"2023-11-06T03:04:07.697675+0000","last_clean":"2023-11-06T03:04:07.697675+0000","last_became_active":"2023-11-06T03:04:06.721193+0000","last_became_peered":"2023-11-06T03:04:06.721193+0000","last_unstale":"2023-11-06T03:04:07.697675+0000","last_undegraded":"2023-11-06T03:04:07.697675+0000","last_fullsized":"2023-11-06T03:04:07.697675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697996+0000","last_change":"2023-11-06T03:04:06.354926+0000","last_active":"2023-11-06T03:04:07.697996+0000","last_peered":"2023-11-06T03:04:07.697996+0000","last_clean":"2023-11-06T03:04:07.697996+0000","last_became_active":"2023-11-06T03:04:06.354287+0000","last_became_peered":"2023-11-06T03:04:06.354287+0000","last_unstale":"2023-11-06T03:04:07.697996+0000","last_undegraded":"2023-11-06T03:04:07.697996+0000","last_fullsized":"2023-11-06T03:04:07.697996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316609+0000","last_change":"2023-11-06T03:04:06.716076+0000","last_active":"2023-11-06T03:04:08.316609+0000","last_peered":"2023-11-06T03:04:08.316609+0000","last_clean":"2023-11-06T03:04:08.316609+0000","last_became_active":"2023-11-06T03:04:06.715726+0000","last_became_peered":"2023-11-06T03:04:06.715726+0000","last_unstale":"2023-11-06T03:04:08.316609+0000","last_undegraded":"2023-11-06T03:04:08.316609+0000","last_fullsized":"2023-11-06T03:04:08.316609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812115+0000","last_change":"2023-11-06T03:04:06.346560+0000","last_active":"2023-11-06T03:04:07.812115+0000","last_peered":"2023-11-06T03:04:07.812115+0000","last_clean":"2023-11-06T03:04:07.812115+0000","last_became_active":"2023-11-06T03:04:06.345767+0000","last_became_peered":"2023-11-06T03:04:06.345767+0000","last_unstale":"2023-11-06T03:04:07.812115+0000","last_undegraded":"2023-11-06T03:04:07.812115+0000","last_fullsized":"2023-11-06T03:04:07.812115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:18.995094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314192+0000","last_change":"2023-11-06T03:04:06.346611+0000","last_active":"2023-11-06T03:04:08.314192+0000","last_peered":"2023-11-06T03:04:08.314192+0000","last_clean":"2023-11-06T03:04:08.314192+0000","last_became_active":"2023-11-06T03:04:06.345852+0000","last_became_peered":"2023-11-06T03:04:06.345852+0000","last_unstale":"2023-11-06T03:04:08.314192+0000","last_undegraded":"2023-11-06T03:04:08.314192+0000","last_fullsized":"2023-11-06T03:04:08.314192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696805+0000","last_change":"2023-11-06T03:04:06.727615+0000","last_active":"2023-11-06T03:04:07.696805+0000","last_peered":"2023-11-06T03:04:07.696805+0000","last_clean":"2023-11-06T03:04:07.696805+0000","last_became_active":"2023-11-06T03:04:06.726871+0000","last_became_peered":"2023-11-06T03:04:06.726871+0000","last_unstale":"2023-11-06T03:04:07.696805+0000","last_undegraded":"2023-11-06T03:04:07.696805+0000","last_fullsized":"2023-11-06T03:04:07.696805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315989+0000","last_change":"2023-11-06T03:04:06.344230+0000","last_active":"2023-11-06T03:04:08.315989+0000","last_peered":"2023-11-06T03:04:08.315989+0000","last_clean":"2023-11-06T03:04:08.315989+0000","last_became_active":"2023-11-06T03:04:06.343852+0000","last_became_peered":"2023-11-06T03:04:06.343852+0000","last_unstale":"2023-11-06T03:04:08.315989+0000","last_undegraded":"2023-11-06T03:04:08.315989+0000","last_fullsized":"2023-11-06T03:04:08.315989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696536+0000","last_change":"2023-11-06T03:04:06.729233+0000","last_active":"2023-11-06T03:04:07.696536+0000","last_peered":"2023-11-06T03:04:07.696536+0000","last_clean":"2023-11-06T03:04:07.696536+0000","last_became_active":"2023-11-06T03:04:06.728828+0000","last_became_peered":"2023-11-06T03:04:06.728828+0000","last_unstale":"2023-11-06T03:04:07.696536+0000","last_undegraded":"2023-11-06T03:04:07.696536+0000","last_fullsized":"2023-11-06T03:04:07.696536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315468+0000","last_change":"2023-11-06T03:04:06.721167+0000","last_active":"2023-11-06T03:04:08.315468+0000","last_peered":"2023-11-06T03:04:08.315468+0000","last_clean":"2023-11-06T03:04:08.315468+0000","last_became_active":"2023-11-06T03:04:06.720914+0000","last_became_peered":"2023-11-06T03:04:06.720914+0000","last_unstale":"2023-11-06T03:04:08.315468+0000","last_undegraded":"2023-11-06T03:04:08.315468+0000","last_fullsized":"2023-11-06T03:04:08.315468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:55.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696763+0000","last_change":"2023-11-06T03:04:06.331738+0000","last_active":"2023-11-06T03:04:07.696763+0000","last_peered":"2023-11-06T03:04:07.696763+0000","last_clean":"2023-11-06T03:04:07.696763+0000","last_became_active":"2023-11-06T03:04:06.330746+0000","last_became_peered":"2023-11-06T03:04:06.330746+0000","last_unstale":"2023-11-06T03:04:07.696763+0000","last_undegraded":"2023-11-06T03:04:07.696763+0000","last_fullsized":"2023-11-06T03:04:07.696763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315727+0000","last_change":"2023-11-06T03:04:06.700157+0000","last_active":"2023-11-06T03:04:08.315727+0000","last_peered":"2023-11-06T03:04:08.315727+0000","last_clean":"2023-11-06T03:04:08.315727+0000","last_became_active":"2023-11-06T03:04:06.699680+0000","last_became_peered":"2023-11-06T03:04:06.699680+0000","last_unstale":"2023-11-06T03:04:08.315727+0000","last_undegraded":"2023-11-06T03:04:08.315727+0000","last_fullsized":"2023-11-06T03:04:08.315727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316079+0000","last_change":"2023-11-06T03:04:06.330647+0000","last_active":"2023-11-06T03:04:08.316079+0000","last_peered":"2023-11-06T03:04:08.316079+0000","last_clean":"2023-11-06T03:04:08.316079+0000","last_became_active":"2023-11-06T03:04:06.330043+0000","last_became_peered":"2023-11-06T03:04:06.330043+0000","last_unstale":"2023-11-06T03:04:08.316079+0000","last_undegraded":"2023-11-06T03:04:08.316079+0000","last_fullsized":"2023-11-06T03:04:08.316079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.724159+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.723798+0000","last_became_peered":"2023-11-06T03:04:06.723798+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.341434+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.340687+0000","last_became_peered":"2023-11-06T03:04:06.340687+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:44.206262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315009+0000","last_change":"2023-11-06T03:04:06.719856+0000","last_active":"2023-11-06T03:04:08.315009+0000","last_peered":"2023-11-06T03:04:08.315009+0000","last_clean":"2023-11-06T03:04:08.315009+0000","last_became_active":"2023-11-06T03:04:06.718775+0000","last_became_peered":"2023-11-06T03:04:06.718775+0000","last_unstale":"2023-11-06T03:04:08.315009+0000","last_undegraded":"2023-11-06T03:04:08.315009+0000","last_fullsized":"2023-11-06T03:04:08.315009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:21.577004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315849+0000","last_change":"2023-11-06T03:04:06.359042+0000","last_active":"2023-11-06T03:04:08.315849+0000","last_peered":"2023-11-06T03:04:08.315849+0000","last_clean":"2023-11-06T03:04:08.315849+0000","last_became_active":"2023-11-06T03:04:06.358768+0000","last_became_peered":"2023-11-06T03:04:06.358768+0000","last_unstale":"2023-11-06T03:04:08.315849+0000","last_undegraded":"2023-11-06T03:04:08.315849+0000","last_fullsized":"2023-11-06T03:04:08.315849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697472+0000","last_change":"2023-11-06T03:04:06.723498+0000","last_active":"2023-11-06T03:04:07.697472+0000","last_peered":"2023-11-06T03:04:07.697472+0000","last_clean":"2023-11-06T03:04:07.697472+0000","last_became_active":"2023-11-06T03:04:06.723271+0000","last_became_peered":"2023-11-06T03:04:06.723271+0000","last_unstale":"2023-11-06T03:04:07.697472+0000","last_undegraded":"2023-11-06T03:04:07.697472+0000","last_fullsized":"2023-11-06T03:04:07.697472+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:26.972193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313662+0000","last_change":"2023-11-06T03:04:06.359575+0000","last_active":"2023-11-06T03:04:08.313662+0000","last_peered":"2023-11-06T03:04:08.313662+0000","last_clean":"2023-11-06T03:04:08.313662+0000","last_became_active":"2023-11-06T03:04:06.358835+0000","last_became_peered":"2023-11-06T03:04:06.358835+0000","last_unstale":"2023-11-06T03:04:08.313662+0000","last_undegraded":"2023-11-06T03:04:08.313662+0000","last_fullsized":"2023-11-06T03:04:08.313662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697217+0000","last_change":"2023-11-06T03:04:06.712260+0000","last_active":"2023-11-06T03:04:07.697217+0000","last_peered":"2023-11-06T03:04:07.697217+0000","last_clean":"2023-11-06T03:04:07.697217+0000","last_became_active":"2023-11-06T03:04:06.711945+0000","last_became_peered":"2023-11-06T03:04:06.711945+0000","last_unstale":"2023-11-06T03:04:07.697217+0000","last_undegraded":"2023-11-06T03:04:07.697217+0000","last_fullsized":"2023-11-06T03:04:07.697217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:06.799825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314081+0000","last_change":"2023-11-06T03:04:06.350220+0000","last_active":"2023-11-06T03:04:08.314081+0000","last_peered":"2023-11-06T03:04:08.314081+0000","last_clean":"2023-11-06T03:04:08.314081+0000","last_became_active":"2023-11-06T03:04:06.349458+0000","last_became_peered":"2023-11-06T03:04:06.349458+0000","last_unstale":"2023-11-06T03:04:08.314081+0000","last_undegraded":"2023-11-06T03:04:08.314081+0000","last_fullsized":"2023-11-06T03:04:08.314081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697048+0000","last_change":"2023-11-06T03:04:06.710979+0000","last_active":"2023-11-06T03:04:07.697048+0000","last_peered":"2023-11-06T03:04:07.697048+0000","last_clean":"2023-11-06T03:04:07.697048+0000","last_became_active":"2023-11-06T03:04:06.709663+0000","last_became_peered":"2023-11-06T03:04:06.709663+0000","last_unstale":"2023-11-06T03:04:07.697048+0000","last_undegraded":"2023-11-06T03:04:07.697048+0000","last_fullsized":"2023-11-06T03:04:07.697048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697612+0000","last_change":"2023-11-06T03:04:06.333374+0000","last_active":"2023-11-06T03:04:07.697612+0000","last_peered":"2023-11-06T03:04:07.697612+0000","last_clean":"2023-11-06T03:04:07.697612+0000","last_became_active":"2023-11-06T03:04:06.332910+0000","last_became_peered":"2023-11-06T03:04:06.332910+0000","last_unstale":"2023-11-06T03:04:07.697612+0000","last_undegraded":"2023-11-06T03:04:07.697612+0000","last_fullsized":"2023-11-06T03:04:07.697612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697342+0000","last_change":"2023-11-06T03:04:06.722605+0000","last_active":"2023-11-06T03:04:07.697342+0000","last_peered":"2023-11-06T03:04:07.697342+0000","last_clean":"2023-11-06T03:04:07.697342+0000","last_became_active":"2023-11-06T03:04:06.722317+0000","last_became_peered":"2023-11-06T03:04:06.722317+0000","last_unstale":"2023-11-06T03:04:07.697342+0000","last_undegraded":"2023-11-06T03:04:07.697342+0000","last_fullsized":"2023-11-06T03:04:07.697342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315853+0000","last_change":"2023-11-06T03:04:06.344293+0000","last_active":"2023-11-06T03:04:08.315853+0000","last_peered":"2023-11-06T03:04:08.315853+0000","last_clean":"2023-11-06T03:04:08.315853+0000","last_became_active":"2023-11-06T03:04:06.344022+0000","last_became_peered":"2023-11-06T03:04:06.344022+0000","last_unstale":"2023-11-06T03:04:08.315853+0000","last_undegraded":"2023-11-06T03:04:08.315853+0000","last_fullsized":"2023-11-06T03:04:08.315853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":237568,"data_stored":88580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":88670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:19.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:04:19.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.237+0000 7f0e9762e640 1 Processor -- start 2023-11-06T03:04:19.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.237+0000 7f0e9762e640 1 -- start start 2023-11-06T03:04:19.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e9762e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e900b2330 0x7f0e9006c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e9762e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9006c490 con 0x7f0e90156a70 2023-11-06T03:04:19.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e9762e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9006fb80 con 0x7f0e900adae0 2023-11-06T03:04:19.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e9762e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9006b950 con 0x7f0e900b2330 2023-11-06T03:04:19.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0e90156a70 0x7f0e9006f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52516/0 (socket says 172.21.15.157:52516) 2023-11-06T03:04:19.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 -- 172.21.15.157:0/1491723340 learned_addr learned my addr 172.21.15.157:0/1491723340 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:19.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144610633 0 0) 0x7f0e9006c490 con 0x7f0e90156a70 2023-11-06T03:04:19.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e78004640 con 0x7f0e90156a70 2023-11-06T03:04:19.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e95bcb640 1 --2- 172.21.15.157:0/1491723340 >> v2:172.21.15.195:3300/0 conn(0x7f0e900b2330 0x7f0e9006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542927096 0 0) 0x7f0e78004640 con 0x7f0e90156a70 2023-11-06T03:04:19.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 >> v2:172.21.15.195:3300/0 conn(0x7f0e900b2330 msgr2=0x7f0e9006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 --2- 172.21.15.157:0/1491723340 >> v2:172.21.15.195:3300/0 conn(0x7f0e900b2330 0x7f0e9006c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 >> v1:172.21.15.195:6789/0 conn(0x7f0e900adae0 legacy=0x7f0e900b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9006bb60 con 0x7f0e90156a70 2023-11-06T03:04:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0e88001680 con 0x7f0e90156a70 2023-11-06T03:04:19.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 >> v1:172.21.15.157:6789/0 conn(0x7f0e90156a70 legacy=0x7f0e9006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 0x7f0e78004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e780058e0 0x7f0e78005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e780062a0 0x7f0e7800caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9006b950 con 0x7f0e78004a90 2023-11-06T03:04:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7800d1d0 con 0x7f0e780062a0 2023-11-06T03:04:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7800d390 con 0x7f0e780058e0 2023-11-06T03:04:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 0x7f0e78004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e94bc9640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e780058e0 0x7f0e78005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 -- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e780058e0 msgr2=0x7f0e78005d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e780058e0 0x7f0e78005d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 -- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e780062a0 msgr2=0x7f0e7800caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e780062a0 0x7f0e7800caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e95bcb640 1 -- 172.21.15.157:0/1491723340 reap_dead start 2023-11-06T03:04:19.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 -- 172.21.15.157:0/1491723340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78004640 con 0x7f0e78004a90 2023-11-06T03:04:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.241+0000 7f0e953ca640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 0x7f0e78004e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e88001ec0 tx=0x7f0e8801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb60d4d0b6e8cec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88005ae0 con 0x7f0e78004a90 2023-11-06T03:04:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e88005c60 con 0x7f0e78004a90 2023-11-06T03:04:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8ffff640 1 -- 172.21.15.157:0/1491723340 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880a5ab0 con 0x7f0e78004a90 2023-11-06T03:04:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 msgr2=0x7f0e78004e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 0x7f0e78004e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e88001ec0 tx=0x7f0e8801cbf0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/1491723340 shutdown_connections 2023-11-06T03:04:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 --2- 172.21.15.157:0/1491723340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e78004a90 0x7f0e78004e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/1491723340 >> 172.21.15.157:0/1491723340 conn(0x7f0e90065930 msgr2=0x7f0e90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/1491723340 shutdown_connections 2023-11-06T03:04:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/1491723340 wait complete. 2023-11-06T03:04:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 Processor -- start 2023-11-06T03:04:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- start start 2023-11-06T03:04:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78004a90 0x7f0e9013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e901e6900 0x7f0e901e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9013de10 con 0x7f0e901e6d80 2023-11-06T03:04:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9013ea30 con 0x7f0e901e6900 2023-11-06T03:04:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901f7950 con 0x7f0e78004a90 2023-11-06T03:04:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35568/0 (socket says 172.21.15.157:35568) 2023-11-06T03:04:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 -- 172.21.15.157:0/2986071458 learned_addr learned my addr 172.21.15.157:0/2986071458 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 -- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78004a90 msgr2=0x7f0e9013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78004a90 0x7f0e9013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 -- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e901e6900 msgr2=0x7f0e901e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e901e6900 0x7f0e901e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 -- 172.21.15.157:0/2986071458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9014f460 con 0x7f0e901e6d80 2023-11-06T03:04:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e95bcb640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006c400 tx=0x7f0e8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000ed90 con 0x7f0e901e6d80 2023-11-06T03:04:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80014420 con 0x7f0e901e6d80 2023-11-06T03:04:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9014f740 con 0x7f0e901e6d80 2023-11-06T03:04:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800147d0 con 0x7f0e901e6d80 2023-11-06T03:04:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9014fd00 con 0x7f0e901e6d80 2023-11-06T03:04:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.245+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e90072750 con 0x7f0e901e6d80 2023-11-06T03:04:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.249+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0e80014930 con 0x7f0e901e6d80 2023-11-06T03:04:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.249+0000 7f0e8dffb640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 0x7f0e7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.249+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e80084770 con 0x7f0e901e6d80 2023-11-06T03:04:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.249+0000 7f0e953ca640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 0x7f0e7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.249+0000 7f0e953ca640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 0x7f0e7c067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e880ab000 tx=0x7f0e88001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.253+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0e80051fb0 con 0x7f0e901e6d80 2023-11-06T03:04:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.393+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e90156e50 con 0x7f0e7c065a60 2023-11-06T03:04:19.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.417+0000 7f0e8dffb640 1 -- 172.21.15.157:0/2986071458 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286555 (secure 0 0 0) 0x7f0e880059a0 con 0x7f0e7c065a60 2023-11-06T03:04:19.418 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:19.420 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:04:19.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 msgr2=0x7f0e7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 0x7f0e7c067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e880ab000 tx=0x7f0e88001a10 comp rx=0 tx=0).stop 2023-11-06T03:04:19.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 msgr2=0x7f0e901e4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006c400 tx=0x7f0e8000e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 shutdown_connections 2023-11-06T03:04:19.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e7c065a60 0x7f0e7c067f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e901e6d80 0x7f0e901e4f00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e901e6900 0x7f0e901e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 --2- 172.21.15.157:0/2986071458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78004a90 0x7f0e9013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 >> 172.21.15.157:0/2986071458 conn(0x7f0e90065930 msgr2=0x7f0e900cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 shutdown_connections 2023-11-06T03:04:19.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.421+0000 7f0e9762e640 1 -- 172.21.15.157:0/2986071458 wait complete. 2023-11-06T03:04:19.436 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-11-06T03:04:19.098231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251484,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754723236,"statfs":{"total":773094113280,"available":772836593664,"internally_reserved":0,"allocated":3174400,"data_stored":1899195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.403864"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314154+0000","last_change":"2023-11-06T03:04:06.723013+0000","last_active":"2023-11-06T03:04:08.314154+0000","last_peered":"2023-11-06T03:04:08.314154+0000","last_clean":"2023-11-06T03:04:08.314154+0000","last_became_active":"2023-11-06T03:04:06.722313+0000","last_became_peered":"2023-11-06T03:04:06.722313+0000","last_unstale":"2023-11-06T03:04:08.314154+0000","last_undegraded":"2023-11-06T03:04:08.314154+0000","last_fullsized":"2023-11-06T03:04:08.314154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:17.965940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313638+0000","last_change":"2023-11-06T03:04:06.709695+0000","last_active":"2023-11-06T03:04:08.313638+0000","last_peered":"2023-11-06T03:04:08.313638+0000","last_clean":"2023-11-06T03:04:08.313638+0000","last_became_active":"2023-11-06T03:04:06.709429+0000","last_became_peered":"2023-11-06T03:04:06.709429+0000","last_unstale":"2023-11-06T03:04:08.313638+0000","last_undegraded":"2023-11-06T03:04:08.313638+0000","last_fullsized":"2023-11-06T03:04:08.313638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697946+0000","last_change":"2023-11-06T03:04:06.700734+0000","last_active":"2023-11-06T03:04:07.697946+0000","last_peered":"2023-11-06T03:04:07.697946+0000","last_clean":"2023-11-06T03:04:07.697946+0000","last_became_active":"2023-11-06T03:04:06.699143+0000","last_became_peered":"2023-11-06T03:04:06.699143+0000","last_unstale":"2023-11-06T03:04:07.697946+0000","last_undegraded":"2023-11-06T03:04:07.697946+0000","last_fullsized":"2023-11-06T03:04:07.697946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315057+0000","last_change":"2023-11-06T03:04:06.725804+0000","last_active":"2023-11-06T03:04:08.315057+0000","last_peered":"2023-11-06T03:04:08.315057+0000","last_clean":"2023-11-06T03:04:08.315057+0000","last_became_active":"2023-11-06T03:04:06.725496+0000","last_became_peered":"2023-11-06T03:04:06.725496+0000","last_unstale":"2023-11-06T03:04:08.315057+0000","last_undegraded":"2023-11-06T03:04:08.315057+0000","last_fullsized":"2023-11-06T03:04:08.315057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:30.176058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315579+0000","last_change":"2023-11-06T03:04:06.725612+0000","last_active":"2023-11-06T03:04:08.315579+0000","last_peered":"2023-11-06T03:04:08.315579+0000","last_clean":"2023-11-06T03:04:08.315579+0000","last_became_active":"2023-11-06T03:04:06.725099+0000","last_became_peered":"2023-11-06T03:04:06.725099+0000","last_unstale":"2023-11-06T03:04:08.315579+0000","last_undegraded":"2023-11-06T03:04:08.315579+0000","last_fullsized":"2023-11-06T03:04:08.315579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697168+0000","last_change":"2023-11-06T03:04:06.710419+0000","last_active":"2023-11-06T03:04:07.697168+0000","last_peered":"2023-11-06T03:04:07.697168+0000","last_clean":"2023-11-06T03:04:07.697168+0000","last_became_active":"2023-11-06T03:04:06.709958+0000","last_became_peered":"2023-11-06T03:04:06.709958+0000","last_unstale":"2023-11-06T03:04:07.697168+0000","last_undegraded":"2023-11-06T03:04:07.697168+0000","last_fullsized":"2023-11-06T03:04:07.697168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:58.848613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811621+0000","last_change":"2023-11-06T03:04:06.720209+0000","last_active":"2023-11-06T03:04:07.811621+0000","last_peered":"2023-11-06T03:04:07.811621+0000","last_clean":"2023-11-06T03:04:07.811621+0000","last_became_active":"2023-11-06T03:04:06.719857+0000","last_became_peered":"2023-11-06T03:04:06.719857+0000","last_unstale":"2023-11-06T03:04:07.811621+0000","last_undegraded":"2023-11-06T03:04:07.811621+0000","last_fullsized":"2023-11-06T03:04:07.811621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313002+0000","last_change":"2023-11-06T03:04:06.706215+0000","last_active":"2023-11-06T03:04:08.313002+0000","last_peered":"2023-11-06T03:04:08.313002+0000","last_clean":"2023-11-06T03:04:08.313002+0000","last_became_active":"2023-11-06T03:04:06.705329+0000","last_became_peered":"2023-11-06T03:04:06.705329+0000","last_unstale":"2023-11-06T03:04:08.313002+0000","last_undegraded":"2023-11-06T03:04:08.313002+0000","last_fullsized":"2023-11-06T03:04:08.313002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315584+0000","last_change":"2023-11-06T03:04:06.725727+0000","last_active":"2023-11-06T03:04:08.315584+0000","last_peered":"2023-11-06T03:04:08.315584+0000","last_clean":"2023-11-06T03:04:08.315584+0000","last_became_active":"2023-11-06T03:04:06.725241+0000","last_became_peered":"2023-11-06T03:04:06.725241+0000","last_unstale":"2023-11-06T03:04:08.315584+0000","last_undegraded":"2023-11-06T03:04:08.315584+0000","last_fullsized":"2023-11-06T03:04:08.315584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313982+0000","last_change":"2023-11-06T03:04:06.701948+0000","last_active":"2023-11-06T03:04:08.313982+0000","last_peered":"2023-11-06T03:04:08.313982+0000","last_clean":"2023-11-06T03:04:08.313982+0000","last_became_active":"2023-11-06T03:04:06.701405+0000","last_became_peered":"2023-11-06T03:04:06.701405+0000","last_unstale":"2023-11-06T03:04:08.313982+0000","last_undegraded":"2023-11-06T03:04:08.313982+0000","last_fullsized":"2023-11-06T03:04:08.313982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698195+0000","last_change":"2023-11-06T03:04:06.711152+0000","last_active":"2023-11-06T03:04:07.698195+0000","last_peered":"2023-11-06T03:04:07.698195+0000","last_clean":"2023-11-06T03:04:07.698195+0000","last_became_active":"2023-11-06T03:04:06.710840+0000","last_became_peered":"2023-11-06T03:04:06.710840+0000","last_unstale":"2023-11-06T03:04:07.698195+0000","last_undegraded":"2023-11-06T03:04:07.698195+0000","last_fullsized":"2023-11-06T03:04:07.698195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314771+0000","last_change":"2023-11-06T03:04:06.703107+0000","last_active":"2023-11-06T03:04:08.314771+0000","last_peered":"2023-11-06T03:04:08.314771+0000","last_clean":"2023-11-06T03:04:08.314771+0000","last_became_active":"2023-11-06T03:04:06.702533+0000","last_became_peered":"2023-11-06T03:04:06.702533+0000","last_unstale":"2023-11-06T03:04:08.314771+0000","last_undegraded":"2023-11-06T03:04:08.314771+0000","last_fullsized":"2023-11-06T03:04:08.314771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314211+0000","last_change":"2023-11-06T03:04:06.710998+0000","last_active":"2023-11-06T03:04:08.314211+0000","last_peered":"2023-11-06T03:04:08.314211+0000","last_clean":"2023-11-06T03:04:08.314211+0000","last_became_active":"2023-11-06T03:04:06.710459+0000","last_became_peered":"2023-11-06T03:04:06.710459+0000","last_unstale":"2023-11-06T03:04:08.314211+0000","last_undegraded":"2023-11-06T03:04:08.314211+0000","last_fullsized":"2023-11-06T03:04:08.314211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:36.030631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313983+0000","last_change":"2023-11-06T03:04:06.708520+0000","last_active":"2023-11-06T03:04:08.313983+0000","last_peered":"2023-11-06T03:04:08.313983+0000","last_clean":"2023-11-06T03:04:08.313983+0000","last_became_active":"2023-11-06T03:04:06.708200+0000","last_became_peered":"2023-11-06T03:04:06.708200+0000","last_unstale":"2023-11-06T03:04:08.313983+0000","last_undegraded":"2023-11-06T03:04:08.313983+0000","last_fullsized":"2023-11-06T03:04:08.313983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811766+0000","last_change":"2023-11-06T03:04:06.723418+0000","last_active":"2023-11-06T03:04:07.811766+0000","last_peered":"2023-11-06T03:04:07.811766+0000","last_clean":"2023-11-06T03:04:07.811766+0000","last_became_active":"2023-11-06T03:04:06.720428+0000","last_became_peered":"2023-11-06T03:04:06.720428+0000","last_unstale":"2023-11-06T03:04:07.811766+0000","last_undegraded":"2023-11-06T03:04:07.811766+0000","last_fullsized":"2023-11-06T03:04:07.811766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697132+0000","last_change":"2023-11-06T03:04:06.699686+0000","last_active":"2023-11-06T03:04:07.697132+0000","last_peered":"2023-11-06T03:04:07.697132+0000","last_clean":"2023-11-06T03:04:07.697132+0000","last_became_active":"2023-11-06T03:04:06.699150+0000","last_became_peered":"2023-11-06T03:04:06.699150+0000","last_unstale":"2023-11-06T03:04:07.697132+0000","last_undegraded":"2023-11-06T03:04:07.697132+0000","last_fullsized":"2023-11-06T03:04:07.697132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315781+0000","last_change":"2023-11-06T03:04:06.718065+0000","last_active":"2023-11-06T03:04:08.315781+0000","last_peered":"2023-11-06T03:04:08.315781+0000","last_clean":"2023-11-06T03:04:08.315781+0000","last_became_active":"2023-11-06T03:04:06.717495+0000","last_became_peered":"2023-11-06T03:04:06.717495+0000","last_unstale":"2023-11-06T03:04:08.315781+0000","last_undegraded":"2023-11-06T03:04:08.315781+0000","last_fullsized":"2023-11-06T03:04:08.315781+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697937+0000","last_change":"2023-11-06T03:04:06.707851+0000","last_active":"2023-11-06T03:04:07.697937+0000","last_peered":"2023-11-06T03:04:07.697937+0000","last_clean":"2023-11-06T03:04:07.697937+0000","last_became_active":"2023-11-06T03:04:06.707587+0000","last_became_peered":"2023-11-06T03:04:06.707587+0000","last_unstale":"2023-11-06T03:04:07.697937+0000","last_undegraded":"2023-11-06T03:04:07.697937+0000","last_fullsized":"2023-11-06T03:04:07.697937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:52:19.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314357+0000","last_change":"2023-11-06T03:04:06.724261+0000","last_active":"2023-11-06T03:04:08.314357+0000","last_peered":"2023-11-06T03:04:08.314357+0000","last_clean":"2023-11-06T03:04:08.314357+0000","last_became_active":"2023-11-06T03:04:06.723974+0000","last_became_peered":"2023-11-06T03:04:06.723974+0000","last_unstale":"2023-11-06T03:04:08.314357+0000","last_undegraded":"2023-11-06T03:04:08.314357+0000","last_fullsized":"2023-11-06T03:04:08.314357+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:16.954584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697610+0000","last_change":"2023-11-06T03:04:06.729633+0000","last_active":"2023-11-06T03:04:07.697610+0000","last_peered":"2023-11-06T03:04:07.697610+0000","last_clean":"2023-11-06T03:04:07.697610+0000","last_became_active":"2023-11-06T03:04:06.729241+0000","last_became_peered":"2023-11-06T03:04:06.729241+0000","last_unstale":"2023-11-06T03:04:07.697610+0000","last_undegraded":"2023-11-06T03:04:07.697610+0000","last_fullsized":"2023-11-06T03:04:07.697610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:09:05.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811862+0000","last_change":"2023-11-06T03:04:06.728388+0000","last_active":"2023-11-06T03:04:07.811862+0000","last_peered":"2023-11-06T03:04:07.811862+0000","last_clean":"2023-11-06T03:04:07.811862+0000","last_became_active":"2023-11-06T03:04:06.728135+0000","last_became_peered":"2023-11-06T03:04:06.728135+0000","last_unstale":"2023-11-06T03:04:07.811862+0000","last_undegraded":"2023-11-06T03:04:07.811862+0000","last_fullsized":"2023-11-06T03:04:07.811862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315482+0000","last_change":"2023-11-06T03:04:06.713228+0000","last_active":"2023-11-06T03:04:08.315482+0000","last_peered":"2023-11-06T03:04:08.315482+0000","last_clean":"2023-11-06T03:04:08.315482+0000","last_became_active":"2023-11-06T03:04:06.712822+0000","last_became_peered":"2023-11-06T03:04:06.712822+0000","last_unstale":"2023-11-06T03:04:08.315482+0000","last_undegraded":"2023-11-06T03:04:08.315482+0000","last_fullsized":"2023-11-06T03:04:08.315482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696649+0000","last_change":"2023-11-06T03:04:06.711437+0000","last_active":"2023-11-06T03:04:07.696649+0000","last_peered":"2023-11-06T03:04:07.696649+0000","last_clean":"2023-11-06T03:04:07.696649+0000","last_became_active":"2023-11-06T03:04:06.710409+0000","last_became_peered":"2023-11-06T03:04:06.710409+0000","last_unstale":"2023-11-06T03:04:07.696649+0000","last_undegraded":"2023-11-06T03:04:07.696649+0000","last_fullsized":"2023-11-06T03:04:07.696649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.811784+0000","last_change":"2023-11-06T03:04:06.728415+0000","last_active":"2023-11-06T03:04:07.811784+0000","last_peered":"2023-11-06T03:04:07.811784+0000","last_clean":"2023-11-06T03:04:07.811784+0000","last_became_active":"2023-11-06T03:04:06.728111+0000","last_became_peered":"2023-11-06T03:04:06.728111+0000","last_unstale":"2023-11-06T03:04:07.811784+0000","last_undegraded":"2023-11-06T03:04:07.811784+0000","last_fullsized":"2023-11-06T03:04:07.811784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317080+0000","last_change":"2023-11-06T03:04:06.727933+0000","last_active":"2023-11-06T03:04:08.317080+0000","last_peered":"2023-11-06T03:04:08.317080+0000","last_clean":"2023-11-06T03:04:08.317080+0000","last_became_active":"2023-11-06T03:04:06.727689+0000","last_became_peered":"2023-11-06T03:04:06.727689+0000","last_unstale":"2023-11-06T03:04:08.317080+0000","last_undegraded":"2023-11-06T03:04:08.317080+0000","last_fullsized":"2023-11-06T03:04:08.317080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317243+0000","last_change":"2023-11-06T03:04:06.709146+0000","last_active":"2023-11-06T03:04:08.317243+0000","last_peered":"2023-11-06T03:04:08.317243+0000","last_clean":"2023-11-06T03:04:08.317243+0000","last_became_active":"2023-11-06T03:04:06.708760+0000","last_became_peered":"2023-11-06T03:04:06.708760+0000","last_unstale":"2023-11-06T03:04:08.317243+0000","last_undegraded":"2023-11-06T03:04:08.317243+0000","last_fullsized":"2023-11-06T03:04:08.317243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313830+0000","last_change":"2023-11-06T03:04:06.715622+0000","last_active":"2023-11-06T03:04:08.313830+0000","last_peered":"2023-11-06T03:04:08.313830+0000","last_clean":"2023-11-06T03:04:08.313830+0000","last_became_active":"2023-11-06T03:04:06.715290+0000","last_became_peered":"2023-11-06T03:04:06.715290+0000","last_unstale":"2023-11-06T03:04:08.313830+0000","last_undegraded":"2023-11-06T03:04:08.313830+0000","last_fullsized":"2023-11-06T03:04:08.313830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316817+0000","last_change":"2023-11-06T03:04:06.709608+0000","last_active":"2023-11-06T03:04:08.316817+0000","last_peered":"2023-11-06T03:04:08.316817+0000","last_clean":"2023-11-06T03:04:08.316817+0000","last_became_active":"2023-11-06T03:04:06.708904+0000","last_became_peered":"2023-11-06T03:04:06.708904+0000","last_unstale":"2023-11-06T03:04:08.316817+0000","last_undegraded":"2023-11-06T03:04:08.316817+0000","last_fullsized":"2023-11-06T03:04:08.316817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:20:15.204100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697175+0000","last_change":"2023-11-06T03:04:06.730619+0000","last_active":"2023-11-06T03:04:07.697175+0000","last_peered":"2023-11-06T03:04:07.697175+0000","last_clean":"2023-11-06T03:04:07.697175+0000","last_became_active":"2023-11-06T03:04:06.730192+0000","last_became_peered":"2023-11-06T03:04:06.730192+0000","last_unstale":"2023-11-06T03:04:07.697175+0000","last_undegraded":"2023-11-06T03:04:07.697175+0000","last_fullsized":"2023-11-06T03:04:07.697175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697306+0000","last_change":"2023-11-06T03:04:06.727570+0000","last_active":"2023-11-06T03:04:07.697306+0000","last_peered":"2023-11-06T03:04:07.697306+0000","last_clean":"2023-11-06T03:04:07.697306+0000","last_became_active":"2023-11-06T03:04:06.727305+0000","last_became_peered":"2023-11-06T03:04:06.727305+0000","last_unstale":"2023-11-06T03:04:07.697306+0000","last_undegraded":"2023-11-06T03:04:07.697306+0000","last_fullsized":"2023-11-06T03:04:07.697306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696670+0000","last_change":"2023-11-06T03:04:06.727300+0000","last_active":"2023-11-06T03:04:07.696670+0000","last_peered":"2023-11-06T03:04:07.696670+0000","last_clean":"2023-11-06T03:04:07.696670+0000","last_became_active":"2023-11-06T03:04:06.726311+0000","last_became_peered":"2023-11-06T03:04:06.726311+0000","last_unstale":"2023-11-06T03:04:07.696670+0000","last_undegraded":"2023-11-06T03:04:07.696670+0000","last_fullsized":"2023-11-06T03:04:07.696670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313347+0000","last_change":"2023-11-06T03:04:06.722186+0000","last_active":"2023-11-06T03:04:08.313347+0000","last_peered":"2023-11-06T03:04:08.313347+0000","last_clean":"2023-11-06T03:04:08.313347+0000","last_became_active":"2023-11-06T03:04:06.721514+0000","last_became_peered":"2023-11-06T03:04:06.721514+0000","last_unstale":"2023-11-06T03:04:08.313347+0000","last_undegraded":"2023-11-06T03:04:08.313347+0000","last_fullsized":"2023-11-06T03:04:08.313347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:11.300508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697317+0000","last_change":"2023-11-06T03:04:06.725709+0000","last_active":"2023-11-06T03:04:07.697317+0000","last_peered":"2023-11-06T03:04:07.697317+0000","last_clean":"2023-11-06T03:04:07.697317+0000","last_became_active":"2023-11-06T03:04:06.725248+0000","last_became_peered":"2023-11-06T03:04:06.725248+0000","last_unstale":"2023-11-06T03:04:07.697317+0000","last_undegraded":"2023-11-06T03:04:07.697317+0000","last_fullsized":"2023-11-06T03:04:07.697317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812176+0000","last_change":"2023-11-06T03:04:06.363773+0000","last_active":"2023-11-06T03:04:07.812176+0000","last_peered":"2023-11-06T03:04:07.812176+0000","last_clean":"2023-11-06T03:04:07.812176+0000","last_became_active":"2023-11-06T03:04:06.362996+0000","last_became_peered":"2023-11-06T03:04:06.362996+0000","last_unstale":"2023-11-06T03:04:07.812176+0000","last_undegraded":"2023-11-06T03:04:07.812176+0000","last_fullsized":"2023-11-06T03:04:07.812176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314708+0000","last_change":"2023-11-06T03:04:06.709102+0000","last_active":"2023-11-06T03:04:08.314708+0000","last_peered":"2023-11-06T03:04:08.314708+0000","last_clean":"2023-11-06T03:04:08.314708+0000","last_became_active":"2023-11-06T03:04:06.708817+0000","last_became_peered":"2023-11-06T03:04:06.708817+0000","last_unstale":"2023-11-06T03:04:08.314708+0000","last_undegraded":"2023-11-06T03:04:08.314708+0000","last_fullsized":"2023-11-06T03:04:08.314708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315524+0000","last_change":"2023-11-06T03:04:06.351736+0000","last_active":"2023-11-06T03:04:08.315524+0000","last_peered":"2023-11-06T03:04:08.315524+0000","last_clean":"2023-11-06T03:04:08.315524+0000","last_became_active":"2023-11-06T03:04:06.351500+0000","last_became_peered":"2023-11-06T03:04:06.351500+0000","last_unstale":"2023-11-06T03:04:08.315524+0000","last_undegraded":"2023-11-06T03:04:08.315524+0000","last_fullsized":"2023-11-06T03:04:08.315524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316022+0000","last_change":"2023-11-06T03:04:06.719843+0000","last_active":"2023-11-06T03:04:08.316022+0000","last_peered":"2023-11-06T03:04:08.316022+0000","last_clean":"2023-11-06T03:04:08.316022+0000","last_became_active":"2023-11-06T03:04:06.718738+0000","last_became_peered":"2023-11-06T03:04:06.718738+0000","last_unstale":"2023-11-06T03:04:08.316022+0000","last_undegraded":"2023-11-06T03:04:08.316022+0000","last_fullsized":"2023-11-06T03:04:08.316022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812188+0000","last_change":"2023-11-06T03:04:06.364044+0000","last_active":"2023-11-06T03:04:07.812188+0000","last_peered":"2023-11-06T03:04:07.812188+0000","last_clean":"2023-11-06T03:04:07.812188+0000","last_became_active":"2023-11-06T03:04:06.363453+0000","last_became_peered":"2023-11-06T03:04:06.363453+0000","last_unstale":"2023-11-06T03:04:07.812188+0000","last_undegraded":"2023-11-06T03:04:07.812188+0000","last_fullsized":"2023-11-06T03:04:07.812188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697122+0000","last_change":"2023-11-06T03:04:06.732024+0000","last_active":"2023-11-06T03:04:07.697122+0000","last_peered":"2023-11-06T03:04:07.697122+0000","last_clean":"2023-11-06T03:04:07.697122+0000","last_became_active":"2023-11-06T03:04:06.730677+0000","last_became_peered":"2023-11-06T03:04:06.730677+0000","last_unstale":"2023-11-06T03:04:07.697122+0000","last_undegraded":"2023-11-06T03:04:07.697122+0000","last_fullsized":"2023-11-06T03:04:07.697122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:16:47.763345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313367+0000","last_change":"2023-11-06T03:04:06.333322+0000","last_active":"2023-11-06T03:04:08.313367+0000","last_peered":"2023-11-06T03:04:08.313367+0000","last_clean":"2023-11-06T03:04:08.313367+0000","last_became_active":"2023-11-06T03:04:06.332581+0000","last_became_peered":"2023-11-06T03:04:06.332581+0000","last_unstale":"2023-11-06T03:04:08.313367+0000","last_undegraded":"2023-11-06T03:04:08.313367+0000","last_fullsized":"2023-11-06T03:04:08.313367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812463+0000","last_change":"2023-11-06T03:04:06.363547+0000","last_active":"2023-11-06T03:04:07.812463+0000","last_peered":"2023-11-06T03:04:07.812463+0000","last_clean":"2023-11-06T03:04:07.812463+0000","last_became_active":"2023-11-06T03:04:06.362774+0000","last_became_peered":"2023-11-06T03:04:06.362774+0000","last_unstale":"2023-11-06T03:04:07.812463+0000","last_undegraded":"2023-11-06T03:04:07.812463+0000","last_fullsized":"2023-11-06T03:04:07.812463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313617+0000","last_change":"2023-11-06T03:04:06.715341+0000","last_active":"2023-11-06T03:04:08.313617+0000","last_peered":"2023-11-06T03:04:08.313617+0000","last_clean":"2023-11-06T03:04:08.313617+0000","last_became_active":"2023-11-06T03:04:06.715020+0000","last_became_peered":"2023-11-06T03:04:06.715020+0000","last_unstale":"2023-11-06T03:04:08.313617+0000","last_undegraded":"2023-11-06T03:04:08.313617+0000","last_fullsized":"2023-11-06T03:04:08.313617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812298+0000","last_change":"2023-11-06T03:04:06.364333+0000","last_active":"2023-11-06T03:04:07.812298+0000","last_peered":"2023-11-06T03:04:07.812298+0000","last_clean":"2023-11-06T03:04:07.812298+0000","last_became_active":"2023-11-06T03:04:06.363641+0000","last_became_peered":"2023-11-06T03:04:06.363641+0000","last_unstale":"2023-11-06T03:04:07.812298+0000","last_undegraded":"2023-11-06T03:04:07.812298+0000","last_fullsized":"2023-11-06T03:04:07.812298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:05:29.165293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697141+0000","last_change":"2023-11-06T03:04:06.712796+0000","last_active":"2023-11-06T03:04:07.697141+0000","last_peered":"2023-11-06T03:04:07.697141+0000","last_clean":"2023-11-06T03:04:07.697141+0000","last_became_active":"2023-11-06T03:04:06.712442+0000","last_became_peered":"2023-11-06T03:04:06.712442+0000","last_unstale":"2023-11-06T03:04:07.697141+0000","last_undegraded":"2023-11-06T03:04:07.697141+0000","last_fullsized":"2023-11-06T03:04:07.697141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812223+0000","last_change":"2023-11-06T03:04:06.352109+0000","last_active":"2023-11-06T03:04:07.812223+0000","last_peered":"2023-11-06T03:04:07.812223+0000","last_clean":"2023-11-06T03:04:07.812223+0000","last_became_active":"2023-11-06T03:04:06.351112+0000","last_became_peered":"2023-11-06T03:04:06.351112+0000","last_unstale":"2023-11-06T03:04:07.812223+0000","last_undegraded":"2023-11-06T03:04:07.812223+0000","last_fullsized":"2023-11-06T03:04:07.812223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:27.571905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313304+0000","last_change":"2023-11-06T03:04:06.721884+0000","last_active":"2023-11-06T03:04:08.313304+0000","last_peered":"2023-11-06T03:04:08.313304+0000","last_clean":"2023-11-06T03:04:08.313304+0000","last_became_active":"2023-11-06T03:04:06.721714+0000","last_became_peered":"2023-11-06T03:04:06.721714+0000","last_unstale":"2023-11-06T03:04:08.313304+0000","last_undegraded":"2023-11-06T03:04:08.313304+0000","last_fullsized":"2023-11-06T03:04:08.313304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:04.132219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313920+0000","last_change":"2023-11-06T03:04:06.336342+0000","last_active":"2023-11-06T03:04:08.313920+0000","last_peered":"2023-11-06T03:04:08.313920+0000","last_clean":"2023-11-06T03:04:08.313920+0000","last_became_active":"2023-11-06T03:04:06.335639+0000","last_became_peered":"2023-11-06T03:04:06.335639+0000","last_unstale":"2023-11-06T03:04:08.313920+0000","last_undegraded":"2023-11-06T03:04:08.313920+0000","last_fullsized":"2023-11-06T03:04:08.313920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:15.943820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315124+0000","last_change":"2023-11-06T03:04:06.709502+0000","last_active":"2023-11-06T03:04:08.315124+0000","last_peered":"2023-11-06T03:04:08.315124+0000","last_clean":"2023-11-06T03:04:08.315124+0000","last_became_active":"2023-11-06T03:04:06.709157+0000","last_became_peered":"2023-11-06T03:04:06.709157+0000","last_unstale":"2023-11-06T03:04:08.315124+0000","last_undegraded":"2023-11-06T03:04:08.315124+0000","last_fullsized":"2023-11-06T03:04:08.315124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697206+0000","last_change":"2023-11-06T03:04:06.331517+0000","last_active":"2023-11-06T03:04:07.697206+0000","last_peered":"2023-11-06T03:04:07.697206+0000","last_clean":"2023-11-06T03:04:07.697206+0000","last_became_active":"2023-11-06T03:04:06.331107+0000","last_became_peered":"2023-11-06T03:04:06.331107+0000","last_unstale":"2023-11-06T03:04:07.697206+0000","last_undegraded":"2023-11-06T03:04:07.697206+0000","last_fullsized":"2023-11-06T03:04:07.697206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697752+0000","last_change":"2023-11-06T03:04:06.720549+0000","last_active":"2023-11-06T03:04:07.697752+0000","last_peered":"2023-11-06T03:04:07.697752+0000","last_clean":"2023-11-06T03:04:07.697752+0000","last_became_active":"2023-11-06T03:04:06.719939+0000","last_became_peered":"2023-11-06T03:04:06.719939+0000","last_unstale":"2023-11-06T03:04:07.697752+0000","last_undegraded":"2023-11-06T03:04:07.697752+0000","last_fullsized":"2023-11-06T03:04:07.697752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:34.877473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313501+0000","last_change":"2023-11-06T03:04:06.364318+0000","last_active":"2023-11-06T03:04:08.313501+0000","last_peered":"2023-11-06T03:04:08.313501+0000","last_clean":"2023-11-06T03:04:08.313501+0000","last_became_active":"2023-11-06T03:04:06.363095+0000","last_became_peered":"2023-11-06T03:04:06.363095+0000","last_unstale":"2023-11-06T03:04:08.313501+0000","last_undegraded":"2023-11-06T03:04:08.313501+0000","last_fullsized":"2023-11-06T03:04:08.313501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698194+0000","last_change":"2023-11-06T03:04:06.707941+0000","last_active":"2023-11-06T03:04:07.698194+0000","last_peered":"2023-11-06T03:04:07.698194+0000","last_clean":"2023-11-06T03:04:07.698194+0000","last_became_active":"2023-11-06T03:04:06.707628+0000","last_became_peered":"2023-11-06T03:04:06.707628+0000","last_unstale":"2023-11-06T03:04:07.698194+0000","last_undegraded":"2023-11-06T03:04:07.698194+0000","last_fullsized":"2023-11-06T03:04:07.698194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:55.127510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697420+0000","last_change":"2023-11-06T03:04:06.355002+0000","last_active":"2023-11-06T03:04:07.697420+0000","last_peered":"2023-11-06T03:04:07.697420+0000","last_clean":"2023-11-06T03:04:07.697420+0000","last_became_active":"2023-11-06T03:04:06.352902+0000","last_became_peered":"2023-11-06T03:04:06.352902+0000","last_unstale":"2023-11-06T03:04:07.697420+0000","last_undegraded":"2023-11-06T03:04:07.697420+0000","last_fullsized":"2023-11-06T03:04:07.697420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:46.021589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697853+0000","last_change":"2023-11-06T03:04:06.721894+0000","last_active":"2023-11-06T03:04:07.697853+0000","last_peered":"2023-11-06T03:04:07.697853+0000","last_clean":"2023-11-06T03:04:07.697853+0000","last_became_active":"2023-11-06T03:04:06.721619+0000","last_became_peered":"2023-11-06T03:04:06.721619+0000","last_unstale":"2023-11-06T03:04:07.697853+0000","last_undegraded":"2023-11-06T03:04:07.697853+0000","last_fullsized":"2023-11-06T03:04:07.697853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696727+0000","last_change":"2023-11-06T03:04:06.330685+0000","last_active":"2023-11-06T03:04:07.696727+0000","last_peered":"2023-11-06T03:04:07.696727+0000","last_clean":"2023-11-06T03:04:07.696727+0000","last_became_active":"2023-11-06T03:04:06.330390+0000","last_became_peered":"2023-11-06T03:04:06.330390+0000","last_unstale":"2023-11-06T03:04:07.696727+0000","last_undegraded":"2023-11-06T03:04:07.696727+0000","last_fullsized":"2023-11-06T03:04:07.696727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317295+0000","last_change":"2023-11-06T03:04:06.721960+0000","last_active":"2023-11-06T03:04:08.317295+0000","last_peered":"2023-11-06T03:04:08.317295+0000","last_clean":"2023-11-06T03:04:08.317295+0000","last_became_active":"2023-11-06T03:04:06.721497+0000","last_became_peered":"2023-11-06T03:04:06.721497+0000","last_unstale":"2023-11-06T03:04:08.317295+0000","last_undegraded":"2023-11-06T03:04:08.317295+0000","last_fullsized":"2023-11-06T03:04:08.317295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:28.582669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700445+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:04:06.700445+0000","last_peered":"2023-11-06T03:04:06.700445+0000","last_clean":"2023-11-06T03:04:06.700445+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:04:06.700445+0000","last_undegraded":"2023-11-06T03:04:06.700445+0000","last_fullsized":"2023-11-06T03:04:06.700445+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700232+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:04:06.700232+0000","last_peered":"2023-11-06T03:04:06.700232+0000","last_clean":"2023-11-06T03:04:06.700232+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:04:06.700232+0000","last_undegraded":"2023-11-06T03:04:06.700232+0000","last_fullsized":"2023-11-06T03:04:06.700232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696999+0000","last_change":"2023-11-06T03:04:06.344186+0000","last_active":"2023-11-06T03:04:07.696999+0000","last_peered":"2023-11-06T03:04:07.696999+0000","last_clean":"2023-11-06T03:04:07.696999+0000","last_became_active":"2023-11-06T03:04:06.343854+0000","last_became_peered":"2023-11-06T03:04:06.343854+0000","last_unstale":"2023-11-06T03:04:07.696999+0000","last_undegraded":"2023-11-06T03:04:07.696999+0000","last_fullsized":"2023-11-06T03:04:07.696999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315060+0000","last_change":"2023-11-06T03:04:06.716182+0000","last_active":"2023-11-06T03:04:08.315060+0000","last_peered":"2023-11-06T03:04:08.315060+0000","last_clean":"2023-11-06T03:04:08.315060+0000","last_became_active":"2023-11-06T03:04:06.714829+0000","last_became_peered":"2023-11-06T03:04:06.714829+0000","last_unstale":"2023-11-06T03:04:08.315060+0000","last_undegraded":"2023-11-06T03:04:08.315060+0000","last_fullsized":"2023-11-06T03:04:08.315060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:20.005858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700210+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:04:06.700210+0000","last_peered":"2023-11-06T03:04:06.700210+0000","last_clean":"2023-11-06T03:04:06.700210+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:04:06.700210+0000","last_undegraded":"2023-11-06T03:04:06.700210+0000","last_fullsized":"2023-11-06T03:04:06.700210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314962+0000","last_change":"2023-11-06T03:04:06.359178+0000","last_active":"2023-11-06T03:04:08.314962+0000","last_peered":"2023-11-06T03:04:08.314962+0000","last_clean":"2023-11-06T03:04:08.314962+0000","last_became_active":"2023-11-06T03:04:06.358881+0000","last_became_peered":"2023-11-06T03:04:06.358881+0000","last_unstale":"2023-11-06T03:04:08.314962+0000","last_undegraded":"2023-11-06T03:04:08.314962+0000","last_fullsized":"2023-11-06T03:04:08.314962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316077+0000","last_change":"2023-11-06T03:04:06.712984+0000","last_active":"2023-11-06T03:04:08.316077+0000","last_peered":"2023-11-06T03:04:08.316077+0000","last_clean":"2023-11-06T03:04:08.316077+0000","last_became_active":"2023-11-06T03:04:06.712663+0000","last_became_peered":"2023-11-06T03:04:06.712663+0000","last_unstale":"2023-11-06T03:04:08.316077+0000","last_undegraded":"2023-11-06T03:04:08.316077+0000","last_fullsized":"2023-11-06T03:04:08.316077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:07:53.383272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.688745+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:04:06.688745+0000","last_peered":"2023-11-06T03:04:06.688745+0000","last_clean":"2023-11-06T03:04:06.688745+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:04:06.688745+0000","last_undegraded":"2023-11-06T03:04:06.688745+0000","last_fullsized":"2023-11-06T03:04:06.688745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314913+0000","last_change":"2023-11-06T03:04:06.328651+0000","last_active":"2023-11-06T03:04:08.314913+0000","last_peered":"2023-11-06T03:04:08.314913+0000","last_clean":"2023-11-06T03:04:08.314913+0000","last_became_active":"2023-11-06T03:04:06.328314+0000","last_became_peered":"2023-11-06T03:04:06.328314+0000","last_unstale":"2023-11-06T03:04:08.314913+0000","last_undegraded":"2023-11-06T03:04:08.314913+0000","last_fullsized":"2023-11-06T03:04:08.314913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.317506+0000","last_change":"2023-11-06T03:04:06.710848+0000","last_active":"2023-11-06T03:04:08.317506+0000","last_peered":"2023-11-06T03:04:08.317506+0000","last_clean":"2023-11-06T03:04:08.317506+0000","last_became_active":"2023-11-06T03:04:06.709767+0000","last_became_peered":"2023-11-06T03:04:06.709767+0000","last_unstale":"2023-11-06T03:04:08.317506+0000","last_undegraded":"2023-11-06T03:04:08.317506+0000","last_fullsized":"2023-11-06T03:04:08.317506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:26:58.125361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.700275+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:04:06.700275+0000","last_peered":"2023-11-06T03:04:06.700275+0000","last_clean":"2023-11-06T03:04:06.700275+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:04:06.700275+0000","last_undegraded":"2023-11-06T03:04:06.700275+0000","last_fullsized":"2023-11-06T03:04:06.700275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697593+0000","last_change":"2023-11-06T03:04:06.354390+0000","last_active":"2023-11-06T03:04:07.697593+0000","last_peered":"2023-11-06T03:04:07.697593+0000","last_clean":"2023-11-06T03:04:07.697593+0000","last_became_active":"2023-11-06T03:04:06.353638+0000","last_became_peered":"2023-11-06T03:04:06.353638+0000","last_unstale":"2023-11-06T03:04:07.697593+0000","last_undegraded":"2023-11-06T03:04:07.697593+0000","last_fullsized":"2023-11-06T03:04:07.697593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:33.723645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316563+0000","last_change":"2023-11-06T03:04:06.714255+0000","last_active":"2023-11-06T03:04:08.316563+0000","last_peered":"2023-11-06T03:04:08.316563+0000","last_clean":"2023-11-06T03:04:08.316563+0000","last_became_active":"2023-11-06T03:04:06.709791+0000","last_became_peered":"2023-11-06T03:04:06.709791+0000","last_unstale":"2023-11-06T03:04:08.316563+0000","last_undegraded":"2023-11-06T03:04:08.316563+0000","last_fullsized":"2023-11-06T03:04:08.316563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:30:49.368112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314303+0000","last_change":"2023-11-06T03:04:06.338556+0000","last_active":"2023-11-06T03:04:08.314303+0000","last_peered":"2023-11-06T03:04:08.314303+0000","last_clean":"2023-11-06T03:04:08.314303+0000","last_became_active":"2023-11-06T03:04:06.337740+0000","last_became_peered":"2023-11-06T03:04:06.337740+0000","last_unstale":"2023-11-06T03:04:08.314303+0000","last_undegraded":"2023-11-06T03:04:08.314303+0000","last_fullsized":"2023-11-06T03:04:08.314303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:10.289744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.701953+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:04:06.701953+0000","last_peered":"2023-11-06T03:04:06.701953+0000","last_clean":"2023-11-06T03:04:06.701953+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:04:06.701953+0000","last_undegraded":"2023-11-06T03:04:06.701953+0000","last_fullsized":"2023-11-06T03:04:06.701953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316863+0000","last_change":"2023-11-06T03:04:06.711498+0000","last_active":"2023-11-06T03:04:08.316863+0000","last_peered":"2023-11-06T03:04:08.316863+0000","last_clean":"2023-11-06T03:04:08.316863+0000","last_became_active":"2023-11-06T03:04:06.708625+0000","last_became_peered":"2023-11-06T03:04:06.708625+0000","last_unstale":"2023-11-06T03:04:08.316863+0000","last_undegraded":"2023-11-06T03:04:08.316863+0000","last_fullsized":"2023-11-06T03:04:08.316863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:09.511965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316721+0000","last_change":"2023-11-06T03:04:06.329676+0000","last_active":"2023-11-06T03:04:08.316721+0000","last_peered":"2023-11-06T03:04:08.316721+0000","last_clean":"2023-11-06T03:04:08.316721+0000","last_became_active":"2023-11-06T03:04:06.329333+0000","last_became_peered":"2023-11-06T03:04:06.329333+0000","last_unstale":"2023-11-06T03:04:08.316721+0000","last_undegraded":"2023-11-06T03:04:08.316721+0000","last_fullsized":"2023-11-06T03:04:08.316721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.703855+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:04:06.703855+0000","last_peered":"2023-11-06T03:04:06.703855+0000","last_clean":"2023-11-06T03:04:06.703855+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:04:06.703855+0000","last_undegraded":"2023-11-06T03:04:06.703855+0000","last_fullsized":"2023-11-06T03:04:06.703855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315277+0000","last_change":"2023-11-06T03:04:06.701264+0000","last_active":"2023-11-06T03:04:08.315277+0000","last_peered":"2023-11-06T03:04:08.315277+0000","last_clean":"2023-11-06T03:04:08.315277+0000","last_became_active":"2023-11-06T03:04:06.700955+0000","last_became_peered":"2023-11-06T03:04:06.700955+0000","last_unstale":"2023-11-06T03:04:08.315277+0000","last_undegraded":"2023-11-06T03:04:08.315277+0000","last_fullsized":"2023-11-06T03:04:08.315277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697348+0000","last_change":"2023-11-06T03:04:06.346267+0000","last_active":"2023-11-06T03:04:07.697348+0000","last_peered":"2023-11-06T03:04:07.697348+0000","last_clean":"2023-11-06T03:04:07.697348+0000","last_became_active":"2023-11-06T03:04:06.345991+0000","last_became_peered":"2023-11-06T03:04:06.345991+0000","last_unstale":"2023-11-06T03:04:07.697348+0000","last_undegraded":"2023-11-06T03:04:07.697348+0000","last_fullsized":"2023-11-06T03:04:07.697348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:49:12.789698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.702101+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:04:06.702101+0000","last_peered":"2023-11-06T03:04:06.702101+0000","last_clean":"2023-11-06T03:04:06.702101+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:04:06.702101+0000","last_undegraded":"2023-11-06T03:04:06.702101+0000","last_fullsized":"2023-11-06T03:04:06.702101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696622+0000","last_change":"2023-11-06T03:04:06.731112+0000","last_active":"2023-11-06T03:04:07.696622+0000","last_peered":"2023-11-06T03:04:07.696622+0000","last_clean":"2023-11-06T03:04:07.696622+0000","last_became_active":"2023-11-06T03:04:06.730152+0000","last_became_peered":"2023-11-06T03:04:06.730152+0000","last_unstale":"2023-11-06T03:04:07.696622+0000","last_undegraded":"2023-11-06T03:04:07.696622+0000","last_fullsized":"2023-11-06T03:04:07.696622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:52.837512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316243+0000","last_change":"2023-11-06T03:04:06.331522+0000","last_active":"2023-11-06T03:04:08.316243+0000","last_peered":"2023-11-06T03:04:08.316243+0000","last_clean":"2023-11-06T03:04:08.316243+0000","last_became_active":"2023-11-06T03:04:06.331263+0000","last_became_peered":"2023-11-06T03:04:06.331263+0000","last_unstale":"2023-11-06T03:04:08.316243+0000","last_undegraded":"2023-11-06T03:04:08.316243+0000","last_fullsized":"2023-11-06T03:04:08.316243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-06T03:04:06.725653+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:04:06.725653+0000","last_peered":"2023-11-06T03:04:06.725653+0000","last_clean":"2023-11-06T03:04:06.725653+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:04:06.725653+0000","last_undegraded":"2023-11-06T03:04:06.725653+0000","last_fullsized":"2023-11-06T03:04:06.725653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.698182+0000","last_change":"2023-11-06T03:04:06.705426+0000","last_active":"2023-11-06T03:04:07.698182+0000","last_peered":"2023-11-06T03:04:07.698182+0000","last_clean":"2023-11-06T03:04:07.698182+0000","last_became_active":"2023-11-06T03:04:06.705116+0000","last_became_peered":"2023-11-06T03:04:06.705116+0000","last_unstale":"2023-11-06T03:04:07.698182+0000","last_undegraded":"2023-11-06T03:04:07.698182+0000","last_fullsized":"2023-11-06T03:04:07.698182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:26.686747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697675+0000","last_change":"2023-11-06T03:04:06.721464+0000","last_active":"2023-11-06T03:04:07.697675+0000","last_peered":"2023-11-06T03:04:07.697675+0000","last_clean":"2023-11-06T03:04:07.697675+0000","last_became_active":"2023-11-06T03:04:06.721193+0000","last_became_peered":"2023-11-06T03:04:06.721193+0000","last_unstale":"2023-11-06T03:04:07.697675+0000","last_undegraded":"2023-11-06T03:04:07.697675+0000","last_fullsized":"2023-11-06T03:04:07.697675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697996+0000","last_change":"2023-11-06T03:04:06.354926+0000","last_active":"2023-11-06T03:04:07.697996+0000","last_peered":"2023-11-06T03:04:07.697996+0000","last_clean":"2023-11-06T03:04:07.697996+0000","last_became_active":"2023-11-06T03:04:06.354287+0000","last_became_peered":"2023-11-06T03:04:06.354287+0000","last_unstale":"2023-11-06T03:04:07.697996+0000","last_undegraded":"2023-11-06T03:04:07.697996+0000","last_fullsized":"2023-11-06T03:04:07.697996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:37:11.574735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316609+0000","last_change":"2023-11-06T03:04:06.716076+0000","last_active":"2023-11-06T03:04:08.316609+0000","last_peered":"2023-11-06T03:04:08.316609+0000","last_clean":"2023-11-06T03:04:08.316609+0000","last_became_active":"2023-11-06T03:04:06.715726+0000","last_became_peered":"2023-11-06T03:04:06.715726+0000","last_unstale":"2023-11-06T03:04:08.316609+0000","last_undegraded":"2023-11-06T03:04:08.316609+0000","last_fullsized":"2023-11-06T03:04:08.316609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.812115+0000","last_change":"2023-11-06T03:04:06.346560+0000","last_active":"2023-11-06T03:04:07.812115+0000","last_peered":"2023-11-06T03:04:07.812115+0000","last_clean":"2023-11-06T03:04:07.812115+0000","last_became_active":"2023-11-06T03:04:06.345767+0000","last_became_peered":"2023-11-06T03:04:06.345767+0000","last_unstale":"2023-11-06T03:04:07.812115+0000","last_undegraded":"2023-11-06T03:04:07.812115+0000","last_fullsized":"2023-11-06T03:04:07.812115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:18.995094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314192+0000","last_change":"2023-11-06T03:04:06.346611+0000","last_active":"2023-11-06T03:04:08.314192+0000","last_peered":"2023-11-06T03:04:08.314192+0000","last_clean":"2023-11-06T03:04:08.314192+0000","last_became_active":"2023-11-06T03:04:06.345852+0000","last_became_peered":"2023-11-06T03:04:06.345852+0000","last_unstale":"2023-11-06T03:04:08.314192+0000","last_undegraded":"2023-11-06T03:04:08.314192+0000","last_fullsized":"2023-11-06T03:04:08.314192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696805+0000","last_change":"2023-11-06T03:04:06.727615+0000","last_active":"2023-11-06T03:04:07.696805+0000","last_peered":"2023-11-06T03:04:07.696805+0000","last_clean":"2023-11-06T03:04:07.696805+0000","last_became_active":"2023-11-06T03:04:06.726871+0000","last_became_peered":"2023-11-06T03:04:06.726871+0000","last_unstale":"2023-11-06T03:04:07.696805+0000","last_undegraded":"2023-11-06T03:04:07.696805+0000","last_fullsized":"2023-11-06T03:04:07.696805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:51:38.726668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315989+0000","last_change":"2023-11-06T03:04:06.344230+0000","last_active":"2023-11-06T03:04:08.315989+0000","last_peered":"2023-11-06T03:04:08.315989+0000","last_clean":"2023-11-06T03:04:08.315989+0000","last_became_active":"2023-11-06T03:04:06.343852+0000","last_became_peered":"2023-11-06T03:04:06.343852+0000","last_unstale":"2023-11-06T03:04:08.315989+0000","last_undegraded":"2023-11-06T03:04:08.315989+0000","last_fullsized":"2023-11-06T03:04:08.315989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696536+0000","last_change":"2023-11-06T03:04:06.729233+0000","last_active":"2023-11-06T03:04:07.696536+0000","last_peered":"2023-11-06T03:04:07.696536+0000","last_clean":"2023-11-06T03:04:07.696536+0000","last_became_active":"2023-11-06T03:04:06.728828+0000","last_became_peered":"2023-11-06T03:04:06.728828+0000","last_unstale":"2023-11-06T03:04:07.696536+0000","last_undegraded":"2023-11-06T03:04:07.696536+0000","last_fullsized":"2023-11-06T03:04:07.696536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:44.486838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315468+0000","last_change":"2023-11-06T03:04:06.721167+0000","last_active":"2023-11-06T03:04:08.315468+0000","last_peered":"2023-11-06T03:04:08.315468+0000","last_clean":"2023-11-06T03:04:08.315468+0000","last_became_active":"2023-11-06T03:04:06.720914+0000","last_became_peered":"2023-11-06T03:04:06.720914+0000","last_unstale":"2023-11-06T03:04:08.315468+0000","last_undegraded":"2023-11-06T03:04:08.315468+0000","last_fullsized":"2023-11-06T03:04:08.315468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:55.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.696763+0000","last_change":"2023-11-06T03:04:06.331738+0000","last_active":"2023-11-06T03:04:07.696763+0000","last_peered":"2023-11-06T03:04:07.696763+0000","last_clean":"2023-11-06T03:04:07.696763+0000","last_became_active":"2023-11-06T03:04:06.330746+0000","last_became_peered":"2023-11-06T03:04:06.330746+0000","last_unstale":"2023-11-06T03:04:07.696763+0000","last_undegraded":"2023-11-06T03:04:07.696763+0000","last_fullsized":"2023-11-06T03:04:07.696763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315727+0000","last_change":"2023-11-06T03:04:06.700157+0000","last_active":"2023-11-06T03:04:08.315727+0000","last_peered":"2023-11-06T03:04:08.315727+0000","last_clean":"2023-11-06T03:04:08.315727+0000","last_became_active":"2023-11-06T03:04:06.699680+0000","last_became_peered":"2023-11-06T03:04:06.699680+0000","last_unstale":"2023-11-06T03:04:08.315727+0000","last_undegraded":"2023-11-06T03:04:08.315727+0000","last_fullsized":"2023-11-06T03:04:08.315727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:18.844662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.316079+0000","last_change":"2023-11-06T03:04:06.330647+0000","last_active":"2023-11-06T03:04:08.316079+0000","last_peered":"2023-11-06T03:04:08.316079+0000","last_clean":"2023-11-06T03:04:08.316079+0000","last_became_active":"2023-11-06T03:04:06.330043+0000","last_became_peered":"2023-11-06T03:04:06.330043+0000","last_unstale":"2023-11-06T03:04:08.316079+0000","last_undegraded":"2023-11-06T03:04:08.316079+0000","last_fullsized":"2023-11-06T03:04:08.316079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:14:16.955176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.724159+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.723798+0000","last_became_peered":"2023-11-06T03:04:06.723798+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:15:47.926322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697455+0000","last_change":"2023-11-06T03:04:06.341434+0000","last_active":"2023-11-06T03:04:07.697455+0000","last_peered":"2023-11-06T03:04:07.697455+0000","last_clean":"2023-11-06T03:04:07.697455+0000","last_became_active":"2023-11-06T03:04:06.340687+0000","last_became_peered":"2023-11-06T03:04:06.340687+0000","last_unstale":"2023-11-06T03:04:07.697455+0000","last_undegraded":"2023-11-06T03:04:07.697455+0000","last_fullsized":"2023-11-06T03:04:07.697455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:45:44.206262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315009+0000","last_change":"2023-11-06T03:04:06.719856+0000","last_active":"2023-11-06T03:04:08.315009+0000","last_peered":"2023-11-06T03:04:08.315009+0000","last_clean":"2023-11-06T03:04:08.315009+0000","last_became_active":"2023-11-06T03:04:06.718775+0000","last_became_peered":"2023-11-06T03:04:06.718775+0000","last_unstale":"2023-11-06T03:04:08.315009+0000","last_undegraded":"2023-11-06T03:04:08.315009+0000","last_fullsized":"2023-11-06T03:04:08.315009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:21.577004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315849+0000","last_change":"2023-11-06T03:04:06.359042+0000","last_active":"2023-11-06T03:04:08.315849+0000","last_peered":"2023-11-06T03:04:08.315849+0000","last_clean":"2023-11-06T03:04:08.315849+0000","last_became_active":"2023-11-06T03:04:06.358768+0000","last_became_peered":"2023-11-06T03:04:06.358768+0000","last_unstale":"2023-11-06T03:04:08.315849+0000","last_undegraded":"2023-11-06T03:04:08.315849+0000","last_fullsized":"2023-11-06T03:04:08.315849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:35.019867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697472+0000","last_change":"2023-11-06T03:04:06.723498+0000","last_active":"2023-11-06T03:04:07.697472+0000","last_peered":"2023-11-06T03:04:07.697472+0000","last_clean":"2023-11-06T03:04:07.697472+0000","last_became_active":"2023-11-06T03:04:06.723271+0000","last_became_peered":"2023-11-06T03:04:06.723271+0000","last_unstale":"2023-11-06T03:04:07.697472+0000","last_undegraded":"2023-11-06T03:04:07.697472+0000","last_fullsized":"2023-11-06T03:04:07.697472+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:37:26.972193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.313662+0000","last_change":"2023-11-06T03:04:06.359575+0000","last_active":"2023-11-06T03:04:08.313662+0000","last_peered":"2023-11-06T03:04:08.313662+0000","last_clean":"2023-11-06T03:04:08.313662+0000","last_became_active":"2023-11-06T03:04:06.358835+0000","last_became_peered":"2023-11-06T03:04:06.358835+0000","last_unstale":"2023-11-06T03:04:08.313662+0000","last_undegraded":"2023-11-06T03:04:08.313662+0000","last_fullsized":"2023-11-06T03:04:08.313662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:03.121455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697217+0000","last_change":"2023-11-06T03:04:06.712260+0000","last_active":"2023-11-06T03:04:07.697217+0000","last_peered":"2023-11-06T03:04:07.697217+0000","last_clean":"2023-11-06T03:04:07.697217+0000","last_became_active":"2023-11-06T03:04:06.711945+0000","last_became_peered":"2023-11-06T03:04:06.711945+0000","last_unstale":"2023-11-06T03:04:07.697217+0000","last_undegraded":"2023-11-06T03:04:07.697217+0000","last_fullsized":"2023-11-06T03:04:07.697217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:06.799825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.314081+0000","last_change":"2023-11-06T03:04:06.350220+0000","last_active":"2023-11-06T03:04:08.314081+0000","last_peered":"2023-11-06T03:04:08.314081+0000","last_clean":"2023-11-06T03:04:08.314081+0000","last_became_active":"2023-11-06T03:04:06.349458+0000","last_became_peered":"2023-11-06T03:04:06.349458+0000","last_unstale":"2023-11-06T03:04:08.314081+0000","last_undegraded":"2023-11-06T03:04:08.314081+0000","last_fullsized":"2023-11-06T03:04:08.314081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:54.238556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697048+0000","last_change":"2023-11-06T03:04:06.710979+0000","last_active":"2023-11-06T03:04:07.697048+0000","last_peered":"2023-11-06T03:04:07.697048+0000","last_clean":"2023-11-06T03:04:07.697048+0000","last_became_active":"2023-11-06T03:04:06.709663+0000","last_became_peered":"2023-11-06T03:04:06.709663+0000","last_unstale":"2023-11-06T03:04:07.697048+0000","last_undegraded":"2023-11-06T03:04:07.697048+0000","last_fullsized":"2023-11-06T03:04:07.697048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:37.223544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697612+0000","last_change":"2023-11-06T03:04:06.333374+0000","last_active":"2023-11-06T03:04:07.697612+0000","last_peered":"2023-11-06T03:04:07.697612+0000","last_clean":"2023-11-06T03:04:07.697612+0000","last_became_active":"2023-11-06T03:04:06.332910+0000","last_became_peered":"2023-11-06T03:04:06.332910+0000","last_unstale":"2023-11-06T03:04:07.697612+0000","last_undegraded":"2023-11-06T03:04:07.697612+0000","last_fullsized":"2023-11-06T03:04:07.697612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:07.697342+0000","last_change":"2023-11-06T03:04:06.722605+0000","last_active":"2023-11-06T03:04:07.697342+0000","last_peered":"2023-11-06T03:04:07.697342+0000","last_clean":"2023-11-06T03:04:07.697342+0000","last_became_active":"2023-11-06T03:04:06.722317+0000","last_became_peered":"2023-11-06T03:04:06.722317+0000","last_unstale":"2023-11-06T03:04:07.697342+0000","last_undegraded":"2023-11-06T03:04:07.697342+0000","last_fullsized":"2023-11-06T03:04:07.697342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:06.293625+0000","last_clean_scrub_stamp":"2023-11-06T03:04:06.293625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:19.577382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-06T03:04:08.315853+0000","last_change":"2023-11-06T03:04:06.344293+0000","last_active":"2023-11-06T03:04:08.315853+0000","last_peered":"2023-11-06T03:04:08.315853+0000","last_clean":"2023-11-06T03:04:08.315853+0000","last_became_active":"2023-11-06T03:04:06.344022+0000","last_became_peered":"2023-11-06T03:04:06.344022+0000","last_unstale":"2023-11-06T03:04:08.315853+0000","last_undegraded":"2023-11-06T03:04:08.315853+0000","last_fullsized":"2023-11-06T03:04:08.315853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:05.282861+0000","last_clean_scrub_stamp":"2023-11-06T03:04:05.282861+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:41:20.566240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":237568,"data_stored":88580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":88670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:04:19.443 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T03:04:19.444 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-06T03:04:19.444 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-06T03:04:19.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-06T03:04:19.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 Processor -- start 2023-11-06T03:04:19.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 -- start start 2023-11-06T03:04:19.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66441599c0 0x7f66440ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f664406d000 0x7f66440b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664415e440 con 0x7f664415dea0 2023-11-06T03:04:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664415adb0 con 0x7f66441599c0 2023-11-06T03:04:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6649979640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664406c240 con 0x7f664406d000 2023-11-06T03:04:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.593+0000 7f6642ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66441599c0 0x7f66440ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66441599c0 0x7f66440ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35572/0 (socket says 172.21.15.157:35572) 2023-11-06T03:04:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 -- 172.21.15.157:0/1903551783 learned_addr learned my addr 172.21.15.157:0/1903551783 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f66427fc640 1 --2- 172.21.15.157:0/1903551783 >> v2:172.21.15.195:3300/0 conn(0x7f664406d000 0x7f66440b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 -- 172.21.15.157:0/1903551783 >> v2:172.21.15.195:3300/0 conn(0x7f664406d000 msgr2=0x7f66440b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1903551783 >> v2:172.21.15.195:3300/0 conn(0x7f664406d000 0x7f66440b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 -- 172.21.15.157:0/1903551783 >> v1:172.21.15.195:6789/0 conn(0x7f664415dea0 legacy=0x7f6644070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 -- 172.21.15.157:0/1903551783 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664406c520 con 0x7f66441599c0 2023-11-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1903551783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441599c0 0x7f66440ae000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66440cfa20 tx=0x7f663401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27781f87e1123b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6641ffb640 1 -- 172.21.15.157:0/1903551783 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340ac640 con 0x7f66441599c0 2023-11-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6641ffb640 1 -- 172.21.15.157:0/1903551783 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66340ac7a0 con 0x7f66441599c0 2023-11-06T03:04:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6641ffb640 1 -- 172.21.15.157:0/1903551783 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340acb30 con 0x7f66441599c0 2023-11-06T03:04:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 -- 172.21.15.157:0/1903551783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441599c0 msgr2=0x7f66440ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 --2- 172.21.15.157:0/1903551783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441599c0 0x7f66440ae000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66440cfa20 tx=0x7f663401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 -- 172.21.15.157:0/1903551783 shutdown_connections 2023-11-06T03:04:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 --2- 172.21.15.157:0/1903551783 >> v2:172.21.15.195:3300/0 conn(0x7f664406d000 0x7f66440b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 --2- 172.21.15.157:0/1903551783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66441599c0 0x7f66440ae000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 -- 172.21.15.157:0/1903551783 >> 172.21.15.157:0/1903551783 conn(0x7f6644065930 msgr2=0x7f6644065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 -- 172.21.15.157:0/1903551783 shutdown_connections 2023-11-06T03:04:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.597+0000 7f6649979640 1 -- 172.21.15.157:0/1903551783 wait complete. 2023-11-06T03:04:19.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 Processor -- start 2023-11-06T03:04:19.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- start start 2023-11-06T03:04:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 0x7f664414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 0x7f6644154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644154a90 con 0x7f664415dea0 2023-11-06T03:04:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664413aab0 con 0x7f664406d000 2023-11-06T03:04:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664413bae0 con 0x7f66441599c0 2023-11-06T03:04:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35588/0 (socket says 172.21.15.157:35588) 2023-11-06T03:04:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 -- 172.21.15.157:0/1616178493 learned_addr learned my addr 172.21.15.157:0/1616178493 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66427fc640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 0x7f6644154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 0x7f664414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 -- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 msgr2=0x7f6644154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 0x7f6644154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 -- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 msgr2=0x7f664414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 0x7f664414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 -- 172.21.15.157:0/1616178493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340094d0 con 0x7f664415dea0 2023-11-06T03:04:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66427fc640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 0x7f6644154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 0x7f664414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f66437fe640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c730 tx=0x7f663c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c003b80 con 0x7f664415dea0 2023-11-06T03:04:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c003ce0 con 0x7f664415dea0 2023-11-06T03:04:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664413aed0 con 0x7f664415dea0 2023-11-06T03:04:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c005310 con 0x7f664415dea0 2023-11-06T03:04:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664413b490 con 0x7f664415dea0 2023-11-06T03:04:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664406b950 con 0x7f664415dea0 2023-11-06T03:04:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.601+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f663c005680 con 0x7f664415dea0 2023-11-06T03:04:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.605+0000 7f662bfff640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 0x7f6630067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.605+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f663c083280 con 0x7f664415dea0 2023-11-06T03:04:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.605+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 0x7f6630067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.605+0000 7f6642ffd640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 0x7f6630067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66340ae000 tx=0x7f6634008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.609+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f663c050ae0 con 0x7f664415dea0 2023-11-06T03:04:19.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f66440adcd0 con 0x7f664415dea0 2023-11-06T03:04:19.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f662bfff640 1 -- 172.21.15.157:0/1616178493 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f663c054100 con 0x7f664415dea0 2023-11-06T03:04:19.780 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:19.780 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T03:04:19.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 msgr2=0x7f6630067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 0x7f6630067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f66340ae000 tx=0x7f6634008040 comp rx=0 tx=0).stop 2023-11-06T03:04:19.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 msgr2=0x7f66440cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f663c00c730 tx=0x7f663c00cbf0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 shutdown_connections 2023-11-06T03:04:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6630065a60 0x7f6630067f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f664415dea0 0x7f66440cdff0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66441599c0 0x7f6644154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 --2- 172.21.15.157:0/1616178493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f664406d000 0x7f664414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 >> 172.21.15.157:0/1616178493 conn(0x7f6644065930 msgr2=0x7f664415bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 shutdown_connections 2023-11-06T03:04:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.781+0000 7f6649979640 1 -- 172.21.15.157:0/1616178493 wait complete. 2023-11-06T03:04:19.797 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-06T03:04:19.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:19.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.913+0000 7fe56c095640 1 Processor -- start 2023-11-06T03:04:19.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.913+0000 7fe56c095640 1 -- start start 2023-11-06T03:04:19.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56c095640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56c095640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5640b2af0 0x7fe564159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56c095640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56415e440 con 0x7fe56406d000 2023-11-06T03:04:19.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56c095640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56406c240 con 0x7fe5640b2af0 2023-11-06T03:04:19.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56c095640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56406c520 con 0x7fe5641599c0 2023-11-06T03:04:19.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe569e31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5640b2af0 0x7fe564159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe569e31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:33480/0 (socket says 172.21.15.157:33480) 2023-11-06T03:04:19.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe569e31640 1 -- 172.21.15.157:0/3214794664 learned_addr learned my addr 172.21.15.157:0/3214794664 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:19.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe569630640 1 --1- 172.21.15.157:0/3214794664 >> v1:172.21.15.195:6789/0 conn(0x7fe56406d000 0x7fe5640b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52364/0 (socket says 172.21.15.157:52364) 2023-11-06T03:04:19.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 -- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 msgr2=0x7fe5640ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:19.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 -- 172.21.15.157:0/3214794664 >> v1:172.21.15.195:6789/0 conn(0x7fe56406d000 legacy=0x7fe5640b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 -- 172.21.15.157:0/3214794664 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56406c800 con 0x7fe5640b2af0 2023-11-06T03:04:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe569e31640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe56a632640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b2af0 0x7fe564159da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0097c0 tx=0x7fe55c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0be7f4932367c4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.917+0000 7fe568e2f640 1 -- 172.21.15.157:0/3214794664 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0ab670 con 0x7fe5640b2af0 2023-11-06T03:04:19.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe568e2f640 1 -- 172.21.15.157:0/3214794664 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c0ab7d0 con 0x7fe5640b2af0 2023-11-06T03:04:19.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe568e2f640 1 -- 172.21.15.157:0/3214794664 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0abb60 con 0x7fe5640b2af0 2023-11-06T03:04:19.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b2af0 msgr2=0x7fe564159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b2af0 0x7fe564159da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0097c0 tx=0x7fe55c01c450 comp rx=0 tx=0).stop 2023-11-06T03:04:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3214794664 shutdown_connections 2023-11-06T03:04:19.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b2af0 0x7fe564159da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- 172.21.15.157:0/3214794664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe5640ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3214794664 >> 172.21.15.157:0/3214794664 conn(0x7fe564065930 msgr2=0x7fe564065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:19.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3214794664 shutdown_connections 2023-11-06T03:04:19.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3214794664 wait complete. 2023-11-06T03:04:19.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 Processor -- start 2023-11-06T03:04:19.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- start start 2023-11-06T03:04:19.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe56406d000 0x7fe564152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe56414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56414f660 con 0x7fe5640b2af0 2023-11-06T03:04:19.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56414ed50 con 0x7fe56406d000 2023-11-06T03:04:19.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe564156b80 con 0x7fe5641599c0 2023-11-06T03:04:19.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35600/0 (socket says 172.21.15.157:35600) 2023-11-06T03:04:19.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 -- 172.21.15.157:0/3325407976 learned_addr learned my addr 172.21.15.157:0/3325407976 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:19.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56a632640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe56414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569e31640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe56406d000 0x7fe564152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 -- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 msgr2=0x7fe56414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe56414c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 -- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe56406d000 msgr2=0x7fe564152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:19.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe56406d000 0x7fe564152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:19.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 -- 172.21.15.157:0/3325407976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554009b30 con 0x7fe5640b2af0 2023-11-06T03:04:19.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe569630640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c420 tx=0x7fe55800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580074e0 con 0x7fe5640b2af0 2023-11-06T03:04:19.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c009420 con 0x7fe5640b2af0 2023-11-06T03:04:19.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558007640 con 0x7fe5640b2af0 2023-11-06T03:04:19.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558004a30 con 0x7fe5640b2af0 2023-11-06T03:04:19.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.921+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56414f2f0 con 0x7fe5640b2af0 2023-11-06T03:04:19.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56414f7d0 con 0x7fe5640b2af0 2023-11-06T03:04:19.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe558004b90 con 0x7fe5640b2af0 2023-11-06T03:04:19.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe5627fc640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 0x7fe550067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:19.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe558082e20 con 0x7fe5640b2af0 2023-11-06T03:04:19.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe569e31640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 0x7fe550067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:19.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.925+0000 7fe569e31640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 0x7fe550067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe554009280 tx=0x7fe55400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:19.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:19.929+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe558050600 con 0x7fe5640b2af0 2023-11-06T03:04:20.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.073+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe564072750 con 0x7fe5640b2af0 2023-11-06T03:04:20.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.073+0000 7fe5627fc640 1 -- 172.21.15.157:0/3325407976 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fe558053c20 con 0x7fe5640b2af0 2023-11-06T03:04:20.073 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:20.073 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 msgr2=0x7fe550067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 0x7fe550067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe554009280 tx=0x7fe55400b040 comp rx=0 tx=0).stop 2023-11-06T03:04:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 msgr2=0x7fe564156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c420 tx=0x7fe55800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 shutdown_connections 2023-11-06T03:04:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe550065a60 0x7fe550067f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641599c0 0x7fe56414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5640b2af0 0x7fe564156100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 --2- 172.21.15.157:0/3325407976 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe56406d000 0x7fe564152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 >> 172.21.15.157:0/3325407976 conn(0x7fe564065930 msgr2=0x7fe564153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 shutdown_connections 2023-11-06T03:04:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.077+0000 7fe56c095640 1 -- 172.21.15.157:0/3325407976 wait complete. 2023-11-06T03:04:20.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:20.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:20.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 Processor -- start 2023-11-06T03:04:20.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 -- start start 2023-11-06T03:04:20.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80adf70 0x7f87c815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87c80ccb70 0x7f87c8161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c815e100 con 0x7f87c80cad20 2023-11-06T03:04:20.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c815cff0 con 0x7f87c80ccb70 2023-11-06T03:04:20.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cf907640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80cd130 con 0x7f87c80adf70 2023-11-06T03:04:20.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87c80ccb70 0x7f87c8161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f87c80ccb70 0x7f87c8161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35614/0 (socket says 172.21.15.157:35614) 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 -- 172.21.15.157:0/1641122211 learned_addr learned my addr 172.21.15.157:0/1641122211 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87cd6a3640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80adf70 0x7f87c815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 -- 172.21.15.157:0/1641122211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80adf70 msgr2=0x7f87c815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80adf70 0x7f87c815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 -- 172.21.15.157:0/1641122211 >> v1:172.21.15.195:6789/0 conn(0x7f87c80cad20 legacy=0x7f87c80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:20.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.205+0000 7f87ccea2640 1 -- 172.21.15.157:0/1641122211 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80cd410 con 0x7f87c80ccb70 2023-11-06T03:04:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80ccb70 0x7f87c8161020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7cdcf82d2b7691 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c7fff640 1 -- 172.21.15.157:0/1641122211 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0a7070 con 0x7f87c80ccb70 2023-11-06T03:04:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c7fff640 1 -- 172.21.15.157:0/1641122211 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0ab430 con 0x7f87c80ccb70 2023-11-06T03:04:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c7fff640 1 -- 172.21.15.157:0/1641122211 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0ab7c0 con 0x7f87c80ccb70 2023-11-06T03:04:20.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/1641122211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80ccb70 msgr2=0x7f87c8161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80ccb70 0x7f87c8161020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc01c540 comp rx=0 tx=0).stop 2023-11-06T03:04:20.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/1641122211 shutdown_connections 2023-11-06T03:04:20.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80ccb70 0x7f87c8161020 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- 172.21.15.157:0/1641122211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80adf70 0x7f87c815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/1641122211 >> 172.21.15.157:0/1641122211 conn(0x7f87c80b0b50 msgr2=0x7f87c80b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/1641122211 shutdown_connections 2023-11-06T03:04:20.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/1641122211 wait complete. 2023-11-06T03:04:20.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 Processor -- start 2023-11-06T03:04:20.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- start start 2023-11-06T03:04:20.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87c80adf70 0x7f87c80c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 0x7f87c80c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80c30f0 con 0x7f87c80cad20 2023-11-06T03:04:20.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8156d50 con 0x7f87c80adf70 2023-11-06T03:04:20.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80c51a0 con 0x7f87c80ccb70 2023-11-06T03:04:20.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35618/0 (socket says 172.21.15.157:35618) 2023-11-06T03:04:20.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 -- 172.21.15.157:0/3757627070 learned_addr learned my addr 172.21.15.157:0/3757627070 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cd6a3640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87c80adf70 0x7f87c80c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cdea4640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 0x7f87c80c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 -- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 msgr2=0x7f87c80c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 0x7f87c80c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 -- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87c80adf70 msgr2=0x7f87c80c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87c80adf70 0x7f87c80c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 -- 172.21.15.157:0/3757627070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0094d0 con 0x7f87c80cad20 2023-11-06T03:04:20.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cdea4640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 0x7f87c80c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:20.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87ccea2640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0ad040 tx=0x7f87bc0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0b4830 con 0x7f87c80cad20 2023-11-06T03:04:20.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0b49b0 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c80cb6c0 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.209+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0b4d60 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c8158f60 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c8072750 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87bc0c0460 con 0x7f87c80cad20 2023-11-06T03:04:20.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87c5ffb640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 0x7f87b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87bc11d620 con 0x7f87c80cad20 2023-11-06T03:04:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87cd6a3640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 0x7f87b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.213+0000 7f87cd6a3640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 0x7f87b4067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f87b8003fd0 tx=0x7f87b80039c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.217+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f87bc0eade0 con 0x7f87c80cad20 2023-11-06T03:04:20.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.357+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87c80c5840 con 0x7f87c80cad20 2023-11-06T03:04:20.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.357+0000 7f87c5ffb640 1 -- 172.21.15.157:0/3757627070 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87bc0ee400 con 0x7f87c80cad20 2023-11-06T03:04:20.358 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:20.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 msgr2=0x7f87b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 0x7f87b4067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f87b8003fd0 tx=0x7f87b80039c0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 msgr2=0x7f87c8156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0ad040 tx=0x7f87bc0192e0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 shutdown_connections 2023-11-06T03:04:20.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87b4065a60 0x7f87b4067f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87c80ccb70 0x7f87c80c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87c80cad20 0x7f87c8156810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 --2- 172.21.15.157:0/3757627070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87c80adf70 0x7f87c80c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 >> 172.21.15.157:0/3757627070 conn(0x7f87c80b0b50 msgr2=0x7f87c80aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 shutdown_connections 2023-11-06T03:04:20.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.361+0000 7f87cf907640 1 -- 172.21.15.157:0/3757627070 wait complete. 2023-11-06T03:04:20.376 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:20.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 Processor -- start 2023-11-06T03:04:20.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 -- start start 2023-11-06T03:04:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa840adae0 0x7ffa840b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8406c490 con 0x7ffa840adf40 2023-11-06T03:04:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8406b950 con 0x7ffa840adae0 2023-11-06T03:04:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa8afe9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8406bb60 con 0x7ffa84156a70 2023-11-06T03:04:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa83fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa840adae0 0x7ffa840b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:33500/0 (socket says 172.21.15.157:33500) 2023-11-06T03:04:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 -- 172.21.15.157:0/2872106957 learned_addr learned my addr 172.21.15.157:0/2872106957 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 -- 172.21.15.157:0/2872106957 >> v2:172.21.15.195:3300/0 conn(0x7ffa840adae0 msgr2=0x7ffa840b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 --2- 172.21.15.157:0/2872106957 >> v2:172.21.15.195:3300/0 conn(0x7ffa840adae0 0x7ffa840b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 -- 172.21.15.157:0/2872106957 >> v1:172.21.15.195:6789/0 conn(0x7ffa840adf40 legacy=0x7ffa8406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.493+0000 7ffa88d85640 1 -- 172.21.15.157:0/2872106957 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8406be40 con 0x7ffa84156a70 2023-11-06T03:04:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 --2- 172.21.15.157:0/2872106957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009870 tx=0x7ffa7801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1508a56184a294 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa837fe640 1 -- 172.21.15.157:0/2872106957 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780a95f0 con 0x7ffa84156a70 2023-11-06T03:04:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa837fe640 1 -- 172.21.15.157:0/2872106957 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa780a9750 con 0x7ffa84156a70 2023-11-06T03:04:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa837fe640 1 -- 172.21.15.157:0/2872106957 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780a9ae0 con 0x7ffa84156a70 2023-11-06T03:04:20.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- 172.21.15.157:0/2872106957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 msgr2=0x7ffa8406f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/2872106957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009870 tx=0x7ffa7801c510 comp rx=0 tx=0).stop 2023-11-06T03:04:20.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- 172.21.15.157:0/2872106957 shutdown_connections 2023-11-06T03:04:20.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/2872106957 >> v2:172.21.15.195:3300/0 conn(0x7ffa840adae0 0x7ffa840b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/2872106957 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa8406f470 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- 172.21.15.157:0/2872106957 >> 172.21.15.157:0/2872106957 conn(0x7ffa84065930 msgr2=0x7ffa84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- 172.21.15.157:0/2872106957 shutdown_connections 2023-11-06T03:04:20.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- 172.21.15.157:0/2872106957 wait complete. 2023-11-06T03:04:20.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 Processor -- start 2023-11-06T03:04:20.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- start start 2023-11-06T03:04:20.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa840adf40 0x7ffa841f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa84145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa84148100 con 0x7ffa840adae0 2023-11-06T03:04:20.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8415a780 con 0x7ffa840adf40 2023-11-06T03:04:20.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa8afe9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8414b180 con 0x7ffa84156a70 2023-11-06T03:04:20.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35622/0 (socket says 172.21.15.157:35622) 2023-11-06T03:04:20.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 -- 172.21.15.157:0/715851201 learned_addr learned my addr 172.21.15.157:0/715851201 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa83fff640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa840adf40 0x7ffa841f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa89586640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa84145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 -- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 msgr2=0x7ffa84145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa84145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 -- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa840adf40 msgr2=0x7ffa841f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa840adf40 0x7ffa841f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa88d85640 1 -- 172.21.15.157:0/715851201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780094d0 con 0x7ffa840adae0 2023-11-06T03:04:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.497+0000 7ffa89586640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa84145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa88d85640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ab040 tx=0x7ffa7801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780bc650 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa780bc7d0 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8414b460 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780bcb80 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8414ba20 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa84072750 con 0x7ffa840adae0 2023-11-06T03:04:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffa780c53f0 con 0x7ffa840adae0 2023-11-06T03:04:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 0x7ffa68067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa7811de60 con 0x7ffa840adae0 2023-11-06T03:04:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa83fff640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 0x7ffa68067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.501+0000 7ffa83fff640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 0x7ffa68067f50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa84155f40 tx=0x7ffa70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.505+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffa780eb6d0 con 0x7ffa840adae0 2023-11-06T03:04:20.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa840ce210 con 0x7ffa840adae0 2023-11-06T03:04:20.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa817fa640 1 -- 172.21.15.157:0/715851201 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa780eecf0 con 0x7ffa840adae0 2023-11-06T03:04:20.648 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:20.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 msgr2=0x7ffa68067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 0x7ffa68067f50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa84155f40 tx=0x7ffa70008040 comp rx=0 tx=0).stop 2023-11-06T03:04:20.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 msgr2=0x7ffa8414aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ab040 tx=0x7ffa7801ad50 comp rx=0 tx=0).stop 2023-11-06T03:04:20.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 shutdown_connections 2023-11-06T03:04:20.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa68065a90 0x7ffa68067f50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa84156a70 0x7ffa84145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa840adf40 0x7ffa841f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 --2- 172.21.15.157:0/715851201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa840adae0 0x7ffa8414aba0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 >> 172.21.15.157:0/715851201 conn(0x7ffa84065930 msgr2=0x7ffa84065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 shutdown_connections 2023-11-06T03:04:20.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.649+0000 7ffa8afe9640 1 -- 172.21.15.157:0/715851201 wait complete. 2023-11-06T03:04:20.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:20.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 Processor -- start 2023-11-06T03:04:20.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 -- start start 2023-11-06T03:04:20.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 0x7f25c0065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25c0065e70 0x7f25c006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c00cf820 con 0x7f25c006d100 2023-11-06T03:04:20.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c006d8f0 con 0x7f25c0065e70 2023-11-06T03:04:20.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c7863640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c0161740 con 0x7f25c0157070 2023-11-06T03:04:20.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25c0065e70 0x7f25c006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25c0065e70 0x7f25c006cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35636/0 (socket says 172.21.15.157:35636) 2023-11-06T03:04:20.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 -- 172.21.15.157:0/206523146 learned_addr learned my addr 172.21.15.157:0/206523146 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c5e00640 1 --1- 172.21.15.157:0/206523146 >> v1:172.21.15.157:6789/0 conn(0x7f25c006d100 0x7f25c0070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52530/0 (socket says 172.21.15.157:52530) 2023-11-06T03:04:20.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c55ff640 1 --2- 172.21.15.157:0/206523146 >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 0x7f25c0065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 -- 172.21.15.157:0/206523146 >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 msgr2=0x7f25c0065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:20.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/206523146 >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 0x7f25c0065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 -- 172.21.15.157:0/206523146 >> v1:172.21.15.157:6789/0 conn(0x7f25c006d100 legacy=0x7f25c0070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c4dfe640 1 -- 172.21.15.157:0/206523146 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c015e150 con 0x7f25c0065e70 2023-11-06T03:04:20.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.785+0000 7f25c55ff640 1 --2- 172.21.15.157:0/206523146 >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 0x7f25c0065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:20.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/206523146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c006cbc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25c00669f0 tx=0x7f25b401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee379b235079c940 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25b3fff640 1 -- 172.21.15.157:0/206523146 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40a5610 con 0x7f25c0065e70 2023-11-06T03:04:20.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25b3fff640 1 -- 172.21.15.157:0/206523146 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b40a5770 con 0x7f25c0065e70 2023-11-06T03:04:20.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25b3fff640 1 -- 172.21.15.157:0/206523146 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40a5b00 con 0x7f25c0065e70 2023-11-06T03:04:20.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- 172.21.15.157:0/206523146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 msgr2=0x7f25c006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- 172.21.15.157:0/206523146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c006cbc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25c00669f0 tx=0x7f25b401c940 comp rx=0 tx=0).stop 2023-11-06T03:04:20.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- 172.21.15.157:0/206523146 shutdown_connections 2023-11-06T03:04:20.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- 172.21.15.157:0/206523146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c006cbc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- 172.21.15.157:0/206523146 >> v2:172.21.15.195:3300/0 conn(0x7f25c0157070 0x7f25c0065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- 172.21.15.157:0/206523146 >> 172.21.15.157:0/206523146 conn(0x7f25c0069d70 msgr2=0x7f25c006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- 172.21.15.157:0/206523146 shutdown_connections 2023-11-06T03:04:20.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- 172.21.15.157:0/206523146 wait complete. 2023-11-06T03:04:20.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 Processor -- start 2023-11-06T03:04:20.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- start start 2023-11-06T03:04:20.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 0x7f25c0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 0x7f25c0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0066320 con 0x7f25c0065e70 2023-11-06T03:04:20.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0161170 con 0x7f25c0157070 2023-11-06T03:04:20.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c7863640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01f4fb0 con 0x7f25c006d100 2023-11-06T03:04:20.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35644/0 (socket says 172.21.15.157:35644) 2023-11-06T03:04:20.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 -- 172.21.15.157:0/3212711307 learned_addr learned my addr 172.21.15.157:0/3212711307 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:20.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 0x7f25c0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c5e00640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 0x7f25c0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 -- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 msgr2=0x7f25c0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 0x7f25c0152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 -- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 msgr2=0x7f25c0153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 0x7f25c0153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c55ff640 1 -- 172.21.15.157:0/3212711307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4009b30 con 0x7f25c0065e70 2023-11-06T03:04:20.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 0x7f25c0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:20.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.789+0000 7f25c5e00640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 0x7f25c0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:20.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c55ff640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e810 tx=0x7f25bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00c870 con 0x7f25c0065e70 2023-11-06T03:04:20.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc00c9d0 con 0x7f25c0065e70 2023-11-06T03:04:20.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c015fe60 con 0x7f25c0065e70 2023-11-06T03:04:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00cd60 con 0x7f25c0065e70 2023-11-06T03:04:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c0154120 con 0x7f25c0065e70 2023-11-06T03:04:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c006ad00 con 0x7f25c0065e70 2023-11-06T03:04:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f25bc013410 con 0x7f25c0065e70 2023-11-06T03:04:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 0x7f25a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:20.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25bc082ff0 con 0x7f25c0065e70 2023-11-06T03:04:20.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 0x7f25a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:20.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.793+0000 7f25c4dfe640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 0x7f25a4067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25b40ae000 tx=0x7f25b40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:20.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.797+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f25bc0508e0 con 0x7f25c0065e70 2023-11-06T03:04:20.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25c0072750 con 0x7f25c0065e70 2023-11-06T03:04:20.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25b1ffb640 1 -- 172.21.15.157:0/3212711307 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25bc009480 con 0x7f25c0065e70 2023-11-06T03:04:20.940 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 msgr2=0x7f25a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 0x7f25a4067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25b40ae000 tx=0x7f25b40a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 msgr2=0x7f25c0160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:20.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e810 tx=0x7f25bc00ecd0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 shutdown_connections 2023-11-06T03:04:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25a4065a60 0x7f25a4067f20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25c0157070 0x7f25c0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25c006d100 0x7f25c0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 --2- 172.21.15.157:0/3212711307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c0065e70 0x7f25c0160c30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 >> 172.21.15.157:0/3212711307 conn(0x7f25c0069d70 msgr2=0x7f25c00b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 shutdown_connections 2023-11-06T03:04:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:20.941+0000 7f25c7863640 1 -- 172.21.15.157:0/3212711307 wait complete. 2023-11-06T03:04:20.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:21.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 Processor -- start 2023-11-06T03:04:21.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 -- start start 2023-11-06T03:04:21.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff93006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9300ce480 con 0x7ff9300ae5f0 2023-11-06T03:04:21.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9300cde70 con 0x7ff93006c930 2023-11-06T03:04:21.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff935571640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93006cdc0 con 0x7ff9300ac8b0 2023-11-06T03:04:21.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff9300ae5f0 0x7ff93015ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52546/0 (socket says 172.21.15.157:52546) 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92effd640 1 -- 172.21.15.157:0/3054450751 learned_addr learned my addr 172.21.15.157:0/3054450751 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935626051 0 0) 0x7ff9300ce480 con 0x7ff9300ae5f0 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92e7fc640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff93006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff918003530 con 0x7ff9300ae5f0 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757689944 0 0) 0x7ff918003530 con 0x7ff9300ae5f0 2023-11-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 msgr2=0x7ff93006c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff93006c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 >> v1:172.21.15.195:6789/0 conn(0x7ff93006c930 legacy=0x7ff930160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300cd6f0 con 0x7ff9300ae5f0 2023-11-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff920001800 con 0x7ff9300ae5f0 2023-11-06T03:04:21.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 >> v1:172.21.15.157:6789/0 conn(0x7ff9300ae5f0 legacy=0x7ff93015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9180039b0 0x7ff918003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff9180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff918005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93006cdc0 con 0x7ff918004d10 2023-11-06T03:04:21.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91800ba70 con 0x7ff9180039b0 2023-11-06T03:04:21.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91800bc30 con 0x7ff9300ac8b0 2023-11-06T03:04:21.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff918005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92effd640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9180039b0 0x7ff918003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92e7fc640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff9180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 -- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 msgr2=0x7ff9180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:21.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300ac8b0 0x7ff9180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 -- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9180039b0 msgr2=0x7ff918003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:21.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9180039b0 0x7ff918003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.093+0000 7ff92f7fe640 1 -- 172.21.15.157:0/3054450751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918003530 con 0x7ff918004d10 2023-11-06T03:04:21.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff918005190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff928001e40 tx=0x7ff92801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ebeb250d879b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928003b80 con 0x7ff918004d10 2023-11-06T03:04:21.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928003ce0 con 0x7ff918004d10 2023-11-06T03:04:21.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff92dffb640 1 -- 172.21.15.157:0/3054450751 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280052c0 con 0x7ff918004d10 2023-11-06T03:04:21.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- 172.21.15.157:0/3054450751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 msgr2=0x7ff918005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 --2- 172.21.15.157:0/3054450751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff918005190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff928001e40 tx=0x7ff92801c740 comp rx=0 tx=0).stop 2023-11-06T03:04:21.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff92f7fe640 1 -- 172.21.15.157:0/3054450751 reap_dead start 2023-11-06T03:04:21.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- 172.21.15.157:0/3054450751 shutdown_connections 2023-11-06T03:04:21.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- 172.21.15.157:0/3054450751 >> 172.21.15.157:0/3054450751 conn(0x7ff930069d70 msgr2=0x7ff93006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- 172.21.15.157:0/3054450751 shutdown_connections 2023-11-06T03:04:21.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- 172.21.15.157:0/3054450751 wait complete. 2023-11-06T03:04:21.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 Processor -- start 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.097+0000 7ff935571640 1 -- start start 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff930149780 0x7ff9300c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 0x7ff9300d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300c7790 con 0x7ff918004d10 2023-11-06T03:04:21.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93015ef80 con 0x7ff930149780 2023-11-06T03:04:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300d0540 con 0x7ff9300cfb80 2023-11-06T03:04:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35660/0 (socket says 172.21.15.157:35660) 2023-11-06T03:04:21.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 -- 172.21.15.157:0/2801573411 learned_addr learned my addr 172.21.15.157:0/2801573411 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92e7fc640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff930149780 0x7ff9300c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 0x7ff9300d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 -- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 msgr2=0x7ff9300d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 0x7ff9300d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 -- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff930149780 msgr2=0x7ff9300c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff930149780 0x7ff9300c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 -- 172.21.15.157:0/2801573411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300c68c0 con 0x7ff918004d10 2023-11-06T03:04:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92f7fe640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 0x7ff9300d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92effd640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff92000f8f0 tx=0x7ff92000fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200072e0 con 0x7ff918004d10 2023-11-06T03:04:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920007440 con 0x7ff918004d10 2023-11-06T03:04:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200057b0 con 0x7ff918004d10 2023-11-06T03:04:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9300c8520 con 0x7ff918004d10 2023-11-06T03:04:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9300c6df0 con 0x7ff918004d10 2023-11-06T03:04:21.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff930072750 con 0x7ff918004d10 2023-11-06T03:04:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff920005910 con 0x7ff918004d10 2023-11-06T03:04:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 0x7ff91c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9200834b0 con 0x7ff918004d10 2023-11-06T03:04:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.101+0000 7ff92e7fc640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 0x7ff91c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.105+0000 7ff92e7fc640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 0x7ff91c067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9300c7f10 tx=0x7ff924008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.105+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff920054830 con 0x7ff918004d10 2023-11-06T03:04:21.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff930070d70 con 0x7ff918004d10 2023-11-06T03:04:21.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff9177fe640 1 -- 172.21.15.157:0/2801573411 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9200543c0 con 0x7ff918004d10 2023-11-06T03:04:21.255 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:21.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 msgr2=0x7ff91c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 0x7ff91c067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9300c7f10 tx=0x7ff924008040 comp rx=0 tx=0).stop 2023-11-06T03:04:21.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 msgr2=0x7ff930146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff92000f8f0 tx=0x7ff92000fdb0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 shutdown_connections 2023-11-06T03:04:21.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff91c065a60 0x7ff91c067f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9300cfb80 0x7ff9300d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff930149780 0x7ff9300c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 --2- 172.21.15.157:0/2801573411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff918004d10 0x7ff930146fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 >> 172.21.15.157:0/2801573411 conn(0x7ff930069d70 msgr2=0x7ff93015a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 shutdown_connections 2023-11-06T03:04:21.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.257+0000 7ff935571640 1 -- 172.21.15.157:0/2801573411 wait complete. 2023-11-06T03:04:21.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:04:21.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.385+0000 7f766428d640 1 Processor -- start 2023-11-06T03:04:21.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.385+0000 7f766428d640 1 -- start start 2023-11-06T03:04:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766428d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f765c0afc90 0x7f765c071a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766428d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 0x7f765c1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766428d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0cc8c0 con 0x7f765c0cffa0 2023-11-06T03:04:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766428d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0b0190 con 0x7f765c0afc90 2023-11-06T03:04:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766428d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c160f40 con 0x7f765c0d0380 2023-11-06T03:04:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f765c0afc90 0x7f765c071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7662029640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f765c0cffa0 0x7f765c0cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52556/0 (socket says 172.21.15.157:52556) 2023-11-06T03:04:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7662029640 1 -- 172.21.15.157:0/242774624 learned_addr learned my addr 172.21.15.157:0/242774624 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766282a640 1 --2- 172.21.15.157:0/242774624 >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 0x7f765c1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 -- 172.21.15.157:0/242774624 >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 msgr2=0x7f765c1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 --2- 172.21.15.157:0/242774624 >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 0x7f765c1613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 -- 172.21.15.157:0/242774624 >> v1:172.21.15.157:6789/0 conn(0x7f765c0cffa0 legacy=0x7f765c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 -- 172.21.15.157:0/242774624 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0cd0b0 con 0x7f765c0afc90 2023-11-06T03:04:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f766282a640 1 --2- 172.21.15.157:0/242774624 >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 0x7f765c1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661828640 1 --2- 172.21.15.157:0/242774624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0afc90 0x7f765c071a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4398013652c7aba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661027640 1 -- 172.21.15.157:0/242774624 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ab600 con 0x7f765c0afc90 2023-11-06T03:04:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661027640 1 -- 172.21.15.157:0/242774624 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c0a7050 con 0x7f765c0afc90 2023-11-06T03:04:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.389+0000 7f7661027640 1 -- 172.21.15.157:0/242774624 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ab980 con 0x7f765c0afc90 2023-11-06T03:04:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- 172.21.15.157:0/242774624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0afc90 msgr2=0x7f765c071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- 172.21.15.157:0/242774624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0afc90 0x7f765c071a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c01c410 comp rx=0 tx=0).stop 2023-11-06T03:04:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- 172.21.15.157:0/242774624 shutdown_connections 2023-11-06T03:04:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- 172.21.15.157:0/242774624 >> v2:172.21.15.195:3300/0 conn(0x7f765c0d0380 0x7f765c1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- 172.21.15.157:0/242774624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0afc90 0x7f765c071a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- 172.21.15.157:0/242774624 >> 172.21.15.157:0/242774624 conn(0x7f765c0cd370 msgr2=0x7f765c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- 172.21.15.157:0/242774624 shutdown_connections 2023-11-06T03:04:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- 172.21.15.157:0/242774624 wait complete. 2023-11-06T03:04:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 Processor -- start 2023-11-06T03:04:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- start start 2023-11-06T03:04:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f765c0afc90 0x7f765c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 0x7f765c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c15b810 con 0x7f765c0cffa0 2023-11-06T03:04:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c15d9c0 con 0x7f765c0d0380 2023-11-06T03:04:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766428d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c154fc0 con 0x7f765c0afc90 2023-11-06T03:04:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35668/0 (socket says 172.21.15.157:35668) 2023-11-06T03:04:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 -- 172.21.15.157:0/3648705267 learned_addr learned my addr 172.21.15.157:0/3648705267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7662029640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f765c0afc90 0x7f765c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766282a640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 0x7f765c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 -- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f765c0afc90 msgr2=0x7f765c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f765c0afc90 0x7f765c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 -- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 msgr2=0x7f765c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 0x7f765c06b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 -- 172.21.15.157:0/3648705267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009420 con 0x7f765c0cffa0 2023-11-06T03:04:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f766282a640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 0x7f765c06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.393+0000 7f7661828640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f764c01aac0 tx=0x7f764c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0ba070 con 0x7f765c0cffa0 2023-11-06T03:04:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c0b4410 con 0x7f765c0cffa0 2023-11-06T03:04:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0b47e0 con 0x7f765c0cffa0 2023-11-06T03:04:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c0cf2f0 con 0x7f765c0cffa0 2023-11-06T03:04:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c15ded0 con 0x7f765c0cffa0 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c06bf00 con 0x7f765c0cffa0 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f764c0b2040 con 0x7f765c0cffa0 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 0x7f7640067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f764c11ca50 con 0x7f765c0cffa0 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.397+0000 7f7662029640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 0x7f7640067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.401+0000 7f7662029640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 0x7f7640067f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7658005890 tx=0x7f7658009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.401+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f764c0ea2b0 con 0x7f765c0cffa0 2023-11-06T03:04:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.585+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765c072750 con 0x7f765c0cffa0 2023-11-06T03:04:21.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.585+0000 7f7652ffd640 1 -- 172.21.15.157:0/3648705267 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f764c0b0090 con 0x7f765c0cffa0 2023-11-06T03:04:21.585 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:21.585 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:04:21.585 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:04:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.585+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 msgr2=0x7f7640067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.585+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 0x7f7640067f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7658005890 tx=0x7f7658009040 comp rx=0 tx=0).stop 2023-11-06T03:04:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 msgr2=0x7f765c153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f764c01aac0 tx=0x7f764c0191b0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 shutdown_connections 2023-11-06T03:04:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7640065a60 0x7f7640067f20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f765c0d0380 0x7f765c06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f765c0cffa0 0x7f765c153280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 --2- 172.21.15.157:0/3648705267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f765c0afc90 0x7f765c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 >> 172.21.15.157:0/3648705267 conn(0x7f765c0cd370 msgr2=0x7f765c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 shutdown_connections 2023-11-06T03:04:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.589+0000 7f766428d640 1 -- 172.21.15.157:0/3648705267 wait complete. 2023-11-06T03:04:21.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:21.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.709+0000 7f9a13212640 1 Processor -- start 2023-11-06T03:04:21.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 -- start start 2023-11-06T03:04:21.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a0c15ce30 0x7f9a0c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9a0c0cfc90 0x7f9a0c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c15d210 con 0x7f9a0c0cd520 2023-11-06T03:04:21.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0d17f0 con 0x7f9a0c15ce30 2023-11-06T03:04:21.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a13212640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0ce650 con 0x7f9a0c0cfc90 2023-11-06T03:04:21.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9a0c15ce30 0x7f9a0c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a03fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9a0c0cd520 0x7f9a0c0cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52560/0 (socket says 172.21.15.157:52560) 2023-11-06T03:04:21.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a03fff640 1 -- 172.21.15.157:0/1095181640 learned_addr learned my addr 172.21.15.157:0/1095181640 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:21.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a117af640 1 --2- 172.21.15.157:0/1095181640 >> v2:172.21.15.195:3300/0 conn(0x7f9a0c0cfc90 0x7f9a0c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 -- 172.21.15.157:0/1095181640 >> v2:172.21.15.195:3300/0 conn(0x7f9a0c0cfc90 msgr2=0x7f9a0c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:21.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 --2- 172.21.15.157:0/1095181640 >> v2:172.21.15.195:3300/0 conn(0x7f9a0c0cfc90 0x7f9a0c0cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 -- 172.21.15.157:0/1095181640 >> v1:172.21.15.157:6789/0 conn(0x7f9a0c0cd520 legacy=0x7f9a0c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 -- 172.21.15.157:0/1095181640 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0ce090 con 0x7f9a0c15ce30 2023-11-06T03:04:21.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a10fae640 1 --2- 172.21.15.157:0/1095181640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c0b5f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0d0ce0 tx=0x7f9a0801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50227db2d0842ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a037fe640 1 -- 172.21.15.157:0/1095181640 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080ab600 con 0x7f9a0c15ce30 2023-11-06T03:04:21.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a037fe640 1 -- 172.21.15.157:0/1095181640 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a080a7050 con 0x7f9a0c15ce30 2023-11-06T03:04:21.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.713+0000 7f9a037fe640 1 -- 172.21.15.157:0/1095181640 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080ab980 con 0x7f9a0c15ce30 2023-11-06T03:04:21.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1095181640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 msgr2=0x7f9a0c0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- 172.21.15.157:0/1095181640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c0b5f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0d0ce0 tx=0x7f9a0801c3d0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1095181640 shutdown_connections 2023-11-06T03:04:21.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- 172.21.15.157:0/1095181640 >> v2:172.21.15.195:3300/0 conn(0x7f9a0c0cfc90 0x7f9a0c0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- 172.21.15.157:0/1095181640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c0b5f60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1095181640 >> 172.21.15.157:0/1095181640 conn(0x7f9a0c070990 msgr2=0x7f9a0c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1095181640 shutdown_connections 2023-11-06T03:04:21.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1095181640 wait complete. 2023-11-06T03:04:21.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 Processor -- start 2023-11-06T03:04:21.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- start start 2023-11-06T03:04:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0c0cd520 0x7f9a0c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 0x7f9a0c0b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c159ae0 con 0x7f9a0c15ce30 2023-11-06T03:04:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15bd10 con 0x7f9a0c0cfc90 2023-11-06T03:04:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15bff0 con 0x7f9a0c0cd520 2023-11-06T03:04:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35686/0 (socket says 172.21.15.157:35686) 2023-11-06T03:04:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 -- 172.21.15.157:0/1331880227 learned_addr learned my addr 172.21.15.157:0/1331880227 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a10fae640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0c0cd520 0x7f9a0c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a03fff640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 0x7f9a0c0b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 -- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0c0cd520 msgr2=0x7f9a0c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0c0cd520 0x7f9a0c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 -- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 msgr2=0x7f9a0c0b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 0x7f9a0c0b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 -- 172.21.15.157:0/1331880227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a08009420 con 0x7f9a0c15ce30 2023-11-06T03:04:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a03fff640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 0x7f9a0c0b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a117af640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007e40 tx=0x7f9a0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04014620 con 0x7f9a0c15ce30 2023-11-06T03:04:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0400ee90 con 0x7f9a0c15ce30 2023-11-06T03:04:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c15c2d0 con 0x7f9a0c15ce30 2023-11-06T03:04:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a040149e0 con 0x7f9a0c15ce30 2023-11-06T03:04:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c15ba00 con 0x7f9a0c15ce30 2023-11-06T03:04:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.717+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c159c50 con 0x7f9a0c15ce30 2023-11-06T03:04:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.721+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a04014b40 con 0x7f9a0c15ce30 2023-11-06T03:04:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.721+0000 7f9a017fa640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 0x7f99f8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.721+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a04083c90 con 0x7f9a0c15ce30 2023-11-06T03:04:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.721+0000 7f9a10fae640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 0x7f99f8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.721+0000 7f9a10fae640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 0x7f99f8067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a080ad000 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.725+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9a040514f0 con 0x7f9a0c15ce30 2023-11-06T03:04:21.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.865+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a0c072750 con 0x7f9a0c15ce30 2023-11-06T03:04:21.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.869+0000 7f9a017fa640 1 -- 172.21.15.157:0/1331880227 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f9a04054b10 con 0x7f9a0c15ce30 2023-11-06T03:04:21.869 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:21.870 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:21.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 msgr2=0x7f99f8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 0x7f99f8067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a080ad000 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2023-11-06T03:04:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 msgr2=0x7f9a0c15b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007e40 tx=0x7f9a0400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 shutdown_connections 2023-11-06T03:04:21.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99f8065a90 0x7f99f8067f50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a0c15ce30 0x7f9a0c15b4c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a0c0cfc90 0x7f9a0c0b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 --2- 172.21.15.157:0/1331880227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a0c0cd520 0x7f9a0c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 >> 172.21.15.157:0/1331880227 conn(0x7f9a0c070990 msgr2=0x7f9a0c159190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 shutdown_connections 2023-11-06T03:04:21.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:21.873+0000 7f9a13212640 1 -- 172.21.15.157:0/1331880227 wait complete. 2023-11-06T03:04:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:21.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.009+0000 7f681f73f640 1 Processor -- start 2023-11-06T03:04:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.009+0000 7f681f73f640 1 -- start start 2023-11-06T03:04:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 0x7f68180ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 0x7f68180b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f681815dea0 0x7f68180b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681815e440 con 0x7f681815dea0 2023-11-06T03:04:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681806c240 con 0x7f681806d000 2023-11-06T03:04:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681806c520 con 0x7f68181599c0 2023-11-06T03:04:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f681815dea0 0x7f68180b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f681815dea0 0x7f68180b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35696/0 (socket says 172.21.15.157:35696) 2023-11-06T03:04:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 -- 172.21.15.157:0/1697608131 learned_addr learned my addr 172.21.15.157:0/1697608131 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681d4db640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 0x7f68180ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681ccda640 1 --2- 172.21.15.157:0/1697608131 >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 0x7f68180b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 -- 172.21.15.157:0/1697608131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 msgr2=0x7f68180ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 0x7f68180ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 -- 172.21.15.157:0/1697608131 >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 msgr2=0x7f68180b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 --2- 172.21.15.157:0/1697608131 >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 0x7f68180b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 -- 172.21.15.157:0/1697608131 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681806c800 con 0x7f681815dea0 2023-11-06T03:04:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681d4db640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 0x7f68180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681ccda640 1 --2- 172.21.15.157:0/1697608131 >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 0x7f68180b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681dcdc640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681815dea0 0x7f68180b2920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68180668b0 tx=0x7f681001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ebed7c1cbbcf2d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f680ffff640 1 -- 172.21.15.157:0/1697608131 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100a5840 con 0x7f681815dea0 2023-11-06T03:04:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f680ffff640 1 -- 172.21.15.157:0/1697608131 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68100a59a0 con 0x7f681815dea0 2023-11-06T03:04:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f680ffff640 1 -- 172.21.15.157:0/1697608131 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100a5d30 con 0x7f681815dea0 2023-11-06T03:04:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 -- 172.21.15.157:0/1697608131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681815dea0 msgr2=0x7f68180b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.013+0000 7f681f73f640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681815dea0 0x7f68180b2920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f68180668b0 tx=0x7f681001c9a0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1697608131 shutdown_connections 2023-11-06T03:04:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681815dea0 0x7f68180b2920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- 172.21.15.157:0/1697608131 >> v2:172.21.15.195:3300/0 conn(0x7f681806d000 0x7f68180b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- 172.21.15.157:0/1697608131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68181599c0 0x7f68180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1697608131 >> 172.21.15.157:0/1697608131 conn(0x7f6818065930 msgr2=0x7f6818065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1697608131 shutdown_connections 2023-11-06T03:04:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1697608131 wait complete. 2023-11-06T03:04:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 Processor -- start 2023-11-06T03:04:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- start start 2023-11-06T03:04:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68181599c0 0x7f68180cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f681815dea0 0x7f68181459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681814e450 con 0x7f681806d000 2023-11-06T03:04:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681814db20 con 0x7f68181599c0 2023-11-06T03:04:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681814de00 con 0x7f681815dea0 2023-11-06T03:04:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35698/0 (socket says 172.21.15.157:35698) 2023-11-06T03:04:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 -- 172.21.15.157:0/1064992807 learned_addr learned my addr 172.21.15.157:0/1064992807 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681ccda640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68181599c0 0x7f68180cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681dcdc640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f681815dea0 0x7f68181459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 -- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f681815dea0 msgr2=0x7f68181459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f681815dea0 0x7f68181459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 -- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68181599c0 msgr2=0x7f68180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68181599c0 0x7f68180cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 -- 172.21.15.157:0/1064992807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6810009b30 con 0x7f681806d000 2023-11-06T03:04:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681d4db640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f681400c7c0 tx=0x7f681400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814007a30 con 0x7f681806d000 2023-11-06T03:04:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814007b90 con 0x7f681806d000 2023-11-06T03:04:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681814e0e0 con 0x7f681806d000 2023-11-06T03:04:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681400ce70 con 0x7f681806d000 2023-11-06T03:04:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6818140b60 con 0x7f681806d000 2023-11-06T03:04:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.017+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6818072750 con 0x7f681806d000 2023-11-06T03:04:22.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.021+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f68140113f0 con 0x7f681806d000 2023-11-06T03:04:22.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.021+0000 7f680dffb640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 0x7f67fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.021+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68140836e0 con 0x7f681806d000 2023-11-06T03:04:22.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.021+0000 7f681ccda640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 0x7f67fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.021+0000 7f681ccda640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 0x7f67fc067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f681814fd30 tx=0x7f6808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.025+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6814050f40 con 0x7f681806d000 2023-11-06T03:04:22.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.177+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68180cb110 con 0x7f681806d000 2023-11-06T03:04:22.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.177+0000 7f680dffb640 1 -- 172.21.15.157:0/1064992807 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6814054560 con 0x7f681806d000 2023-11-06T03:04:22.176 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:22.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 msgr2=0x7f67fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 0x7f67fc067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f681814fd30 tx=0x7f6808008040 comp rx=0 tx=0).stop 2023-11-06T03:04:22.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 msgr2=0x7f68181402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f681400c7c0 tx=0x7f681400cc80 comp rx=0 tx=0).stop 2023-11-06T03:04:22.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 shutdown_connections 2023-11-06T03:04:22.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67fc065a60 0x7f67fc067f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f681815dea0 0x7f68181459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68181599c0 0x7f68180cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 --2- 172.21.15.157:0/1064992807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f681806d000 0x7f68181402c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 >> 172.21.15.157:0/1064992807 conn(0x7f6818065930 msgr2=0x7f681815bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 shutdown_connections 2023-11-06T03:04:22.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.181+0000 7f681f73f640 1 -- 172.21.15.157:0/1064992807 wait complete. 2023-11-06T03:04:22.196 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:22.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.305+0000 7ff0466b8640 1 Processor -- start 2023-11-06T03:04:22.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.305+0000 7ff0466b8640 1 -- start start 2023-11-06T03:04:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff0466b8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0401599c0 0x7ff0400ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff0466b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400b2af0 0x7ff040159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff0466b8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015e440 con 0x7ff04006d000 2023-11-06T03:04:22.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff0466b8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04006c240 con 0x7ff0401599c0 2023-11-06T03:04:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff0466b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04006c520 con 0x7ff0400b2af0 2023-11-06T03:04:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff04006d000 0x7ff0400b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52574/0 (socket says 172.21.15.157:52574) 2023-11-06T03:04:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03f7fe640 1 -- 172.21.15.157:0/2223968516 learned_addr learned my addr 172.21.15.157:0/2223968516 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:22.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69618840 0 0) 0x7ff04015e440 con 0x7ff04006d000 2023-11-06T03:04:22.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03ffff640 1 --2- 172.21.15.157:0/2223968516 >> v2:172.21.15.195:3300/0 conn(0x7ff0401599c0 0x7ff0400ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff028003530 con 0x7ff04006d000 2023-11-06T03:04:22.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400b2af0 0x7ff040159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3078416764 0 0) 0x7ff028003530 con 0x7ff04006d000 2023-11-06T03:04:22.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400b2af0 msgr2=0x7ff040159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400b2af0 0x7ff040159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 >> v2:172.21.15.195:3300/0 conn(0x7ff0401599c0 msgr2=0x7ff0400ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 --2- 172.21.15.157:0/2223968516 >> v2:172.21.15.195:3300/0 conn(0x7ff0401599c0 0x7ff0400ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04006c800 con 0x7ff04006d000 2023-11-06T03:04:22.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff0340016a0 con 0x7ff04006d000 2023-11-06T03:04:22.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03ffff640 1 --2- 172.21.15.157:0/2223968516 >> v2:172.21.15.195:3300/0 conn(0x7ff0401599c0 0x7ff0400ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:22.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 >> v1:172.21.15.157:6789/0 conn(0x7ff04006d000 legacy=0x7ff0400b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff028003980 0x7ff028003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0280047d0 0x7ff028004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff028005190 0x7ff02800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04006c240 con 0x7ff028005190 2023-11-06T03:04:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04006c520 con 0x7ff0280047d0 2023-11-06T03:04:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02800bfd0 con 0x7ff028003980 2023-11-06T03:04:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400b2af0 0x7ff040159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:22.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff028005190 0x7ff02800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03f7fe640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0280047d0 0x7ff028004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03ffff640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff028003980 0x7ff028003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 -- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff028003980 msgr2=0x7ff028003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff028003980 0x7ff028003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 -- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0280047d0 msgr2=0x7ff028004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0280047d0 0x7ff028004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff044c55640 1 -- 172.21.15.157:0/2223968516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028003530 con 0x7ff028005190 2023-11-06T03:04:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.309+0000 7ff03f7fe640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0280047d0 0x7ff028004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff028005190 0x7ff02800b990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff04006f250 tx=0x7ff03801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8ff251949fa6c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038003ba0 con 0x7ff028005190 2023-11-06T03:04:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038003d00 con 0x7ff028005190 2023-11-06T03:04:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff03effd640 1 -- 172.21.15.157:0/2223968516 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380a5b80 con 0x7ff028005190 2023-11-06T03:04:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- 172.21.15.157:0/2223968516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff028005190 msgr2=0x7ff02800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 --2- 172.21.15.157:0/2223968516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff028005190 0x7ff02800b990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff04006f250 tx=0x7ff03801cc70 comp rx=0 tx=0).stop 2023-11-06T03:04:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 -- 172.21.15.157:0/2223968516 reap_dead start 2023-11-06T03:04:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- 172.21.15.157:0/2223968516 shutdown_connections 2023-11-06T03:04:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- 172.21.15.157:0/2223968516 >> 172.21.15.157:0/2223968516 conn(0x7ff040065930 msgr2=0x7ff040065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- 172.21.15.157:0/2223968516 shutdown_connections 2023-11-06T03:04:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- 172.21.15.157:0/2223968516 wait complete. 2023-11-06T03:04:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 Processor -- start 2023-11-06T03:04:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- start start 2023-11-06T03:04:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff040158c80 0x7ff0400cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400ce520 0x7ff04014a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400cba10 con 0x7ff04014aed0 2023-11-06T03:04:22.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400ce9a0 con 0x7ff040158c80 2023-11-06T03:04:22.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff0466b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04014b7d0 con 0x7ff0400ce520 2023-11-06T03:04:22.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35714/0 (socket says 172.21.15.157:35714) 2023-11-06T03:04:22.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 -- 172.21.15.157:0/1085156557 learned_addr learned my addr 172.21.15.157:0/1085156557 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff03ffff640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff040158c80 0x7ff0400cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 -- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400ce520 msgr2=0x7ff04014a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400ce520 0x7ff04014a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 -- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff040158c80 msgr2=0x7ff0400cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff040158c80 0x7ff0400cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 -- 172.21.15.157:0/1085156557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04014b350 con 0x7ff04014aed0 2023-11-06T03:04:22.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff044c55640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff038098780 tx=0x7ff028005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.313+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0870 con 0x7ff04014aed0 2023-11-06T03:04:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380b09f0 con 0x7ff04014aed0 2023-11-06T03:04:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04015af20 con 0x7ff04014aed0 2023-11-06T03:04:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0da0 con 0x7ff04014aed0 2023-11-06T03:04:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04015b4e0 con 0x7ff04014aed0 2023-11-06T03:04:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0400cbb80 con 0x7ff04014aed0 2023-11-06T03:04:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff038003e70 con 0x7ff04014aed0 2023-11-06T03:04:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03cff9640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 0x7ff02c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff03811d760 con 0x7ff04014aed0 2023-11-06T03:04:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03ffff640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 0x7ff02c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.317+0000 7ff03ffff640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 0x7ff02c067f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0400ad9e0 tx=0x7ff030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.321+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0400cbb80 con 0x7ff04014aed0 2023-11-06T03:04:22.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.465+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0400cbb80 con 0x7ff04014aed0 2023-11-06T03:04:22.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.465+0000 7ff03cff9640 1 -- 172.21.15.157:0/1085156557 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0400cbb80 con 0x7ff04014aed0 2023-11-06T03:04:22.466 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:22.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 msgr2=0x7ff02c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 0x7ff02c067f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0400ad9e0 tx=0x7ff030008040 comp rx=0 tx=0).stop 2023-11-06T03:04:22.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 msgr2=0x7ff0400a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff038098780 tx=0x7ff028005800 comp rx=0 tx=0).stop 2023-11-06T03:04:22.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 shutdown_connections 2023-11-06T03:04:22.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff02c065a90 0x7ff02c067f50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04014aed0 0x7ff0400a84a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0400ce520 0x7ff04014a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 --2- 172.21.15.157:0/1085156557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff040158c80 0x7ff0400cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 >> 172.21.15.157:0/1085156557 conn(0x7ff040065930 msgr2=0x7ff0400afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 shutdown_connections 2023-11-06T03:04:22.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.469+0000 7ff0466b8640 1 -- 172.21.15.157:0/1085156557 wait complete. 2023-11-06T03:04:22.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:22.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 Processor -- start 2023-11-06T03:04:22.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 -- start start 2023-11-06T03:04:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa51415caa0 0x7fa5140cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa51415ce80 0x7fa5140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5140d2920 con 0x7fa5140ce1a0 2023-11-06T03:04:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5140d18a0 con 0x7fa51415ce80 2023-11-06T03:04:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa519e9f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140d1b60 con 0x7fa51415caa0 2023-11-06T03:04:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa51415ce80 0x7fa5140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa51415ce80 0x7fa5140cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35726/0 (socket says 172.21.15.157:35726) 2023-11-06T03:04:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 -- 172.21.15.157:0/1020958263 learned_addr learned my addr 172.21.15.157:0/1020958263 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 -- 172.21.15.157:0/1020958263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa51415caa0 msgr2=0x7fa5140cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa5137fe640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa51415caa0 0x7fa5140cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa51415caa0 0x7fa5140cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 -- 172.21.15.157:0/1020958263 >> v1:172.21.15.195:6789/0 conn(0x7fa5140ce1a0 legacy=0x7fa5140d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.605+0000 7fa512ffd640 1 -- 172.21.15.157:0/1020958263 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140d1e40 con 0x7fa51415ce80 2023-11-06T03:04:22.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa512ffd640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415ce80 0x7fa5140cfb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa508009870 tx=0x7fa50801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98492fc378d2219 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa5127fc640 1 -- 172.21.15.157:0/1020958263 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080ac600 con 0x7fa51415ce80 2023-11-06T03:04:22.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa5127fc640 1 -- 172.21.15.157:0/1020958263 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5080a8050 con 0x7fa51415ce80 2023-11-06T03:04:22.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa5127fc640 1 -- 172.21.15.157:0/1020958263 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080ac980 con 0x7fa51415ce80 2023-11-06T03:04:22.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- 172.21.15.157:0/1020958263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415ce80 msgr2=0x7fa5140cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415ce80 0x7fa5140cfb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa508009870 tx=0x7fa50801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- 172.21.15.157:0/1020958263 shutdown_connections 2023-11-06T03:04:22.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415ce80 0x7fa5140cfb70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 --2- 172.21.15.157:0/1020958263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa51415caa0 0x7fa5140cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- 172.21.15.157:0/1020958263 >> 172.21.15.157:0/1020958263 conn(0x7fa514070790 msgr2=0x7fa514070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- 172.21.15.157:0/1020958263 shutdown_connections 2023-11-06T03:04:22.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- 172.21.15.157:0/1020958263 wait complete. 2023-11-06T03:04:22.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 Processor -- start 2023-11-06T03:04:22.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.609+0000 7fa519e9f640 1 -- start start 2023-11-06T03:04:22.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5140ce1a0 0x7fa514151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 0x7fa514157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5140cae90 con 0x7fa51415caa0 2023-11-06T03:04:22.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa514158290 con 0x7fa51415ce80 2023-11-06T03:04:22.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa514157cd0 con 0x7fa5140ce1a0 2023-11-06T03:04:22.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35732/0 (socket says 172.21.15.157:35732) 2023-11-06T03:04:22.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 -- 172.21.15.157:0/3354941211 learned_addr learned my addr 172.21.15.157:0/3354941211 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 -- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5140ce1a0 msgr2=0x7fa514151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5140ce1a0 0x7fa514151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 -- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 msgr2=0x7fa514157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa513fff640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 0x7fa514157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 0x7fa514157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 -- 172.21.15.157:0/3354941211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080094d0 con 0x7fa51415caa0 2023-11-06T03:04:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa513fff640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 0x7fa514157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa512ffd640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa50801ab80 tx=0x7fa508019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080b58f0 con 0x7fa51415caa0 2023-11-06T03:04:22.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5080b5a70 con 0x7fa51415caa0 2023-11-06T03:04:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa514157fb0 con 0x7fa51415caa0 2023-11-06T03:04:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080b5e20 con 0x7fa51415caa0 2023-11-06T03:04:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51415b720 con 0x7fa51415caa0 2023-11-06T03:04:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa514072750 con 0x7fa51415caa0 2023-11-06T03:04:22.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa5080b3070 con 0x7fa51415caa0 2023-11-06T03:04:22.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 0x7fa500067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa50811daa0 con 0x7fa51415caa0 2023-11-06T03:04:22.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.613+0000 7fa5137fe640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 0x7fa500067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.617+0000 7fa5137fe640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 0x7fa500067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa50400a8b0 tx=0x7fa504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.617+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa5080eb270 con 0x7fa51415caa0 2023-11-06T03:04:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5140cafd0 con 0x7fa51415caa0 2023-11-06T03:04:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa4fbfff640 1 -- 172.21.15.157:0/3354941211 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5080b10a0 con 0x7fa51415caa0 2023-11-06T03:04:22.772 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 msgr2=0x7fa500067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 0x7fa500067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa50400a8b0 tx=0x7fa504008040 comp rx=0 tx=0).stop 2023-11-06T03:04:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 msgr2=0x7fa514157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa50801ab80 tx=0x7fa508019270 comp rx=0 tx=0).stop 2023-11-06T03:04:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 shutdown_connections 2023-11-06T03:04:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa500065a60 0x7fa500067f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa51415ce80 0x7fa514157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa51415caa0 0x7fa514157110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 --2- 172.21.15.157:0/3354941211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5140ce1a0 0x7fa514151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 >> 172.21.15.157:0/3354941211 conn(0x7fa514070790 msgr2=0x7fa5140d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 shutdown_connections 2023-11-06T03:04:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.773+0000 7fa519e9f640 1 -- 172.21.15.157:0/3354941211 wait complete. 2023-11-06T03:04:22.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 Processor -- start 2023-11-06T03:04:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 -- start start 2023-11-06T03:04:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa9580b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9580ce800 0x7fa9580cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa95806b6a0 0x7fa95806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580cf1a0 con 0x7fa9580ce800 2023-11-06T03:04:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95806c3d0 con 0x7fa95806b6a0 2023-11-06T03:04:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95f2ea640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95806f5a0 con 0x7fa958146370 2023-11-06T03:04:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9580ce800 0x7fa9580cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9580ce800 0x7fa9580cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35740/0 (socket says 172.21.15.157:35740) 2023-11-06T03:04:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 -- 172.21.15.157:0/3223341126 learned_addr learned my addr 172.21.15.157:0/3223341126 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95d086640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa9580b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95d887640 1 --2- 172.21.15.157:0/3223341126 >> v2:172.21.15.195:3300/0 conn(0x7fa95806b6a0 0x7fa95806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 -- 172.21.15.157:0/3223341126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 msgr2=0x7fa9580b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa9580b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 -- 172.21.15.157:0/3223341126 >> v2:172.21.15.195:3300/0 conn(0x7fa95806b6a0 msgr2=0x7fa95806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 --2- 172.21.15.157:0/3223341126 >> v2:172.21.15.195:3300/0 conn(0x7fa95806b6a0 0x7fa95806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95c885640 1 -- 172.21.15.157:0/3223341126 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95806f880 con 0x7fa9580ce800 2023-11-06T03:04:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.901+0000 7fa95d086640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa9580b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:22.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95c885640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9580ce800 0x7fa9580cec60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa95806d360 tx=0x7fa95401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1975eb3ab3758bd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa94ffff640 1 -- 172.21.15.157:0/3223341126 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9540ab600 con 0x7fa9580ce800 2023-11-06T03:04:22.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa94ffff640 1 -- 172.21.15.157:0/3223341126 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9540ab760 con 0x7fa9580ce800 2023-11-06T03:04:22.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa94ffff640 1 -- 172.21.15.157:0/3223341126 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9540abaf0 con 0x7fa9580ce800 2023-11-06T03:04:22.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3223341126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9580ce800 msgr2=0x7fa9580cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9580ce800 0x7fa9580cec60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa95806d360 tx=0x7fa95401c440 comp rx=0 tx=0).stop 2023-11-06T03:04:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3223341126 shutdown_connections 2023-11-06T03:04:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3223341126 >> v2:172.21.15.195:3300/0 conn(0x7fa95806b6a0 0x7fa95806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9580ce800 0x7fa9580cec60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3223341126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa9580b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3223341126 >> 172.21.15.157:0/3223341126 conn(0x7fa958065930 msgr2=0x7fa958065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3223341126 shutdown_connections 2023-11-06T03:04:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3223341126 wait complete. 2023-11-06T03:04:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 Processor -- start 2023-11-06T03:04:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- start start 2023-11-06T03:04:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 0x7fa9580b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa95813be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958158840 con 0x7fa95806b6a0 2023-11-06T03:04:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958158b20 con 0x7fa9580ce800 2023-11-06T03:04:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95f2ea640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa958158e00 con 0x7fa958146370 2023-11-06T03:04:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95d086640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95d086640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35754/0 (socket says 172.21.15.157:35754) 2023-11-06T03:04:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95d086640 1 -- 172.21.15.157:0/3793516861 learned_addr learned my addr 172.21.15.157:0/3793516861 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.905+0000 7fa95d887640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa95813be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 -- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 msgr2=0x7fa95813be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa95813be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 -- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 msgr2=0x7fa9580b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95c885640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 0x7fa9580b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 0x7fa9580b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 -- 172.21.15.157:0/3793516861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540094d0 con 0x7fa95806b6a0 2023-11-06T03:04:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d887640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa95813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95c885640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 0x7fa9580b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95d086640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e810 tx=0x7fa94800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800c8d0 con 0x7fa95806b6a0 2023-11-06T03:04:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94800ca30 con 0x7fa95806b6a0 2023-11-06T03:04:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95815a520 con 0x7fa95806b6a0 2023-11-06T03:04:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94800cde0 con 0x7fa95806b6a0 2023-11-06T03:04:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9580b2a30 con 0x7fa95806b6a0 2023-11-06T03:04:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958072750 con 0x7fa95806b6a0 2023-11-06T03:04:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa948013410 con 0x7fa95806b6a0 2023-11-06T03:04:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 0x7fa93c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa948083090 con 0x7fa95806b6a0 2023-11-06T03:04:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95c885640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 0x7fa93c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.909+0000 7fa95c885640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 0x7fa93c067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9540af000 tx=0x7fa954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:22.913+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa9480508f0 con 0x7fa95806b6a0 2023-11-06T03:04:23.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.053+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa958166d10 con 0x7fa95806b6a0 2023-11-06T03:04:23.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.053+0000 7fa94dffb640 1 -- 172.21.15.157:0/3793516861 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa948053f10 con 0x7fa95806b6a0 2023-11-06T03:04:23.054 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 msgr2=0x7fa93c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 0x7fa93c067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9540af000 tx=0x7fa954008040 comp rx=0 tx=0).stop 2023-11-06T03:04:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 msgr2=0x7fa958158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e810 tx=0x7fa94800ecd0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 shutdown_connections 2023-11-06T03:04:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa93c065a60 0x7fa93c067f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa958146370 0x7fa95813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9580ce800 0x7fa9580b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 --2- 172.21.15.157:0/3793516861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa95806b6a0 0x7fa958158240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 >> 172.21.15.157:0/3793516861 conn(0x7fa958065930 msgr2=0x7fa95815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 shutdown_connections 2023-11-06T03:04:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.057+0000 7fa95f2ea640 1 -- 172.21.15.157:0/3793516861 wait complete. 2023-11-06T03:04:23.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:04:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.185+0000 7f7675f39640 1 Processor -- start 2023-11-06T03:04:23.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.185+0000 7f7675f39640 1 -- start start 2023-11-06T03:04:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f7675f39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f7675f39640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 0x7f76700adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f7675f39640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767006c490 con 0x7f76700ae020 2023-11-06T03:04:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f7675f39640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006b950 con 0x7f767006f470 2023-11-06T03:04:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f7675f39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767006bb60 con 0x7f7670156a70 2023-11-06T03:04:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 0x7f76700adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:33642/0 (socket says 172.21.15.157:33642) 2023-11-06T03:04:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 -- 172.21.15.157:0/1420581049 learned_addr learned my addr 172.21.15.157:0/1420581049 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 -- 172.21.15.157:0/1420581049 >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 msgr2=0x7f76700adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 --2- 172.21.15.157:0/1420581049 >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 0x7f76700adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 -- 172.21.15.157:0/1420581049 >> v1:172.21.15.195:6789/0 conn(0x7f76700ae020 legacy=0x7f76700b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 -- 172.21.15.157:0/1420581049 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767006be40 con 0x7f7670156a70 2023-11-06T03:04:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766effd640 1 --2- 172.21.15.157:0/1420581049 >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 0x7f76700adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.189+0000 7f766f7fe640 1 --2- 172.21.15.157:0/1420581049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f766000bb10 tx=0x7f766001ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473c2565de870f81 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766e7fc640 1 -- 172.21.15.157:0/1420581049 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600a7cd0 con 0x7f7670156a70 2023-11-06T03:04:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766e7fc640 1 -- 172.21.15.157:0/1420581049 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76600a7e30 con 0x7f7670156a70 2023-11-06T03:04:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766e7fc640 1 -- 172.21.15.157:0/1420581049 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600af660 con 0x7f7670156a70 2023-11-06T03:04:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/1420581049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 msgr2=0x7f76700ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- 172.21.15.157:0/1420581049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f766000bb10 tx=0x7f766001ebe0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/1420581049 shutdown_connections 2023-11-06T03:04:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- 172.21.15.157:0/1420581049 >> v2:172.21.15.195:3300/0 conn(0x7f767006f470 0x7f76700adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- 172.21.15.157:0/1420581049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7670156a70 0x7f76700ce920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/1420581049 >> 172.21.15.157:0/1420581049 conn(0x7f7670065930 msgr2=0x7f7670065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/1420581049 shutdown_connections 2023-11-06T03:04:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/1420581049 wait complete. 2023-11-06T03:04:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 Processor -- start 2023-11-06T03:04:23.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- start start 2023-11-06T03:04:23.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 0x7f7670148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670156a70 0x7f7670151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b03d0 con 0x7f76700ae020 2023-11-06T03:04:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b06b0 con 0x7f7670156a70 2023-11-06T03:04:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b0990 con 0x7f767006f470 2023-11-06T03:04:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35766/0 (socket says 172.21.15.157:35766) 2023-11-06T03:04:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 -- 172.21.15.157:0/3126256615 learned_addr learned my addr 172.21.15.157:0/3126256615 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766f7fe640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 0x7f7670148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766ffff640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670156a70 0x7f7670151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 -- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 msgr2=0x7f7670148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 0x7f7670148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 -- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670156a70 msgr2=0x7f7670151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670156a70 0x7f7670151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 -- 172.21.15.157:0/3126256615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009420 con 0x7f76700ae020 2023-11-06T03:04:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766f7fe640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 0x7f7670148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f766effd640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f766400ebe0 tx=0x7f766400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400eee0 con 0x7f76700ae020 2023-11-06T03:04:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400cc30 con 0x7f76700ae020 2023-11-06T03:04:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766000b770 con 0x7f76700ae020 2023-11-06T03:04:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664013470 con 0x7f76700ae020 2023-11-06T03:04:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.193+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7670149080 con 0x7f76700ae020 2023-11-06T03:04:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76700ae990 con 0x7f76700ae020 2023-11-06T03:04:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76640135d0 con 0x7f76700ae020 2023-11-06T03:04:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f7657fff640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 0x7f765c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7664083550 con 0x7f76700ae020 2023-11-06T03:04:23.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f766f7fe640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 0x7f765c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.197+0000 7f766f7fe640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 0x7f765c067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76600b1000 tx=0x7f7660008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.201+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7664050e40 con 0x7f76700ae020 2023-11-06T03:04:23.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76700b1190 con 0x7f76700ae020 2023-11-06T03:04:23.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7657fff640 1 -- 172.21.15.157:0/3126256615 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f7664054460 con 0x7f76700ae020 2023-11-06T03:04:23.371 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:23.372 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:04:23.372 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:04:23.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 msgr2=0x7f765c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 0x7f765c067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76600b1000 tx=0x7f7660008040 comp rx=0 tx=0).stop 2023-11-06T03:04:23.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 msgr2=0x7f767014ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f766400ebe0 tx=0x7f766400c520 comp rx=0 tx=0).stop 2023-11-06T03:04:23.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 shutdown_connections 2023-11-06T03:04:23.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f765c065a60 0x7f765c067f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7670156a70 0x7f7670151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76700ae020 0x7f767014ec00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 --2- 172.21.15.157:0/3126256615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f767006f470 0x7f7670148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 >> 172.21.15.157:0/3126256615 conn(0x7f7670065930 msgr2=0x7f76700b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 shutdown_connections 2023-11-06T03:04:23.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.373+0000 7f7675f39640 1 -- 172.21.15.157:0/3126256615 wait complete. 2023-11-06T03:04:23.389 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:23.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 Processor -- start 2023-11-06T03:04:23.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 -- start start 2023-11-06T03:04:23.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42481551e0 0x7f424806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f424815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424815f580 con 0x7f424815e5e0 2023-11-06T03:04:23.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424815efc0 con 0x7f42481551e0 2023-11-06T03:04:23.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f424d427640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424815f2a0 con 0x7f42480b3b20 2023-11-06T03:04:23.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4246ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42481551e0 0x7f424806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f42467fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f424815e5e0 0x7f42480b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:52578/0 (socket says 172.21.15.157:52578) 2023-11-06T03:04:23.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f42467fc640 1 -- 172.21.15.157:0/3047390933 learned_addr learned my addr 172.21.15.157:0/3047390933 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:23.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4245ffb640 1 -- 172.21.15.157:0/3047390933 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318469952 0 0) 0x7f424815f580 con 0x7f424815e5e0 2023-11-06T03:04:23.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4245ffb640 1 -- 172.21.15.157:0/3047390933 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4230003530 con 0x7f424815e5e0 2023-11-06T03:04:23.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f42477fe640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f424815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4246ffd640 1 -- 172.21.15.157:0/3047390933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 msgr2=0x7f424815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:23.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4246ffd640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f424815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4246ffd640 1 -- 172.21.15.157:0/3047390933 >> v1:172.21.15.157:6789/0 conn(0x7f424815e5e0 legacy=0x7f42480b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f4246ffd640 1 -- 172.21.15.157:0/3047390933 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424806f5a0 con 0x7f42481551e0 2023-11-06T03:04:23.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.505+0000 7f42477fe640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f424815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:23.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f4246ffd640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f424806fda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b1e13381a9d0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f4245ffb640 1 -- 172.21.15.157:0/3047390933 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380ac640 con 0x7f42481551e0 2023-11-06T03:04:23.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f4245ffb640 1 -- 172.21.15.157:0/3047390933 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380ac7a0 con 0x7f42481551e0 2023-11-06T03:04:23.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f4245ffb640 1 -- 172.21.15.157:0/3047390933 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380acb10 con 0x7f42481551e0 2023-11-06T03:04:23.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 -- 172.21.15.157:0/3047390933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 msgr2=0x7f424806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f424806fda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 -- 172.21.15.157:0/3047390933 shutdown_connections 2023-11-06T03:04:23.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f424815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 --2- 172.21.15.157:0/3047390933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f424806fda0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 -- 172.21.15.157:0/3047390933 >> 172.21.15.157:0/3047390933 conn(0x7f4248065930 msgr2=0x7f4248065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 -- 172.21.15.157:0/3047390933 shutdown_connections 2023-11-06T03:04:23.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.509+0000 7f424d427640 1 -- 172.21.15.157:0/3047390933 wait complete. 2023-11-06T03:04:23.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 Processor -- start 2023-11-06T03:04:23.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- start start 2023-11-06T03:04:23.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f42481f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f424815e5e0 0x7f42481540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481faf00 con 0x7f42481551e0 2023-11-06T03:04:23.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481fa470 con 0x7f424815e5e0 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480ce710 con 0x7f42480b3b20 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35794/0 (socket says 172.21.15.157:35794) 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 -- 172.21.15.157:0/1538508942 learned_addr learned my addr 172.21.15.157:0/1538508942 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42477fe640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f424815e5e0 0x7f42481540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f4246ffd640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f42481f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 -- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 msgr2=0x7f42481f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f42481f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 -- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f424815e5e0 msgr2=0x7f42481540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f424815e5e0 0x7f42481540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 -- 172.21.15.157:0/1538508942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238009420 con 0x7f42481551e0 2023-11-06T03:04:23.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f42467fc640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c450 tx=0x7f423c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c005cf0 con 0x7f42481551e0 2023-11-06T03:04:23.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c005e50 con 0x7f42481551e0 2023-11-06T03:04:23.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42481fa660 con 0x7f42481551e0 2023-11-06T03:04:23.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c007290 con 0x7f42481551e0 2023-11-06T03:04:23.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42481fac20 con 0x7f42481551e0 2023-11-06T03:04:23.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.513+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248072750 con 0x7f42481551e0 2023-11-06T03:04:23.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.517+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f423c0073f0 con 0x7f42481551e0 2023-11-06T03:04:23.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.517+0000 7f422f7fe640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 0x7f4234067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.517+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f423c0828e0 con 0x7f42481551e0 2023-11-06T03:04:23.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.517+0000 7f4246ffd640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 0x7f4234067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.517+0000 7f4246ffd640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 0x7f4234067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42380ae000 tx=0x7f42380a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.521+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f423c050140 con 0x7f42481551e0 2023-11-06T03:04:23.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.673+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4248071730 con 0x7f42481551e0 2023-11-06T03:04:23.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f422f7fe640 1 -- 172.21.15.157:0/1538508942 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f423c017070 con 0x7f42481551e0 2023-11-06T03:04:23.676 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:23.676 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:23.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 msgr2=0x7f4234067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 0x7f4234067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42380ae000 tx=0x7f42380a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:23.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 msgr2=0x7f4248153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c450 tx=0x7f423c00c910 comp rx=0 tx=0).stop 2023-11-06T03:04:23.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 shutdown_connections 2023-11-06T03:04:23.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4234065a60 0x7f4234067f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f424815e5e0 0x7f42481540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42481551e0 0x7f4248153b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 --2- 172.21.15.157:0/1538508942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42480b3b20 0x7f42481f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 >> 172.21.15.157:0/1538508942 conn(0x7f4248065930 msgr2=0x7f424814e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 shutdown_connections 2023-11-06T03:04:23.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.677+0000 7f424d427640 1 -- 172.21.15.157:0/1538508942 wait complete. 2023-11-06T03:04:23.695 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:04:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.809+0000 7f321a4b6640 1 Processor -- start 2023-11-06T03:04:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 -- start start 2023-11-06T03:04:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f321415caa0 0x7f32140cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 0x7f32140d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140d2920 con 0x7f321415caa0 2023-11-06T03:04:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140d18a0 con 0x7f32140ce1a0 2023-11-06T03:04:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f321a4b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32140d1b60 con 0x7f321415ce80 2023-11-06T03:04:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f321415caa0 0x7f32140cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f321415caa0 0x7f32140cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35804/0 (socket says 172.21.15.157:35804) 2023-11-06T03:04:23.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 -- 172.21.15.157:0/2578237229 learned_addr learned my addr 172.21.15.157:0/2578237229 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:23.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3218a53640 1 --2- 172.21.15.157:0/2578237229 >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 0x7f32140d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f32137fe640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 -- 172.21.15.157:0/2578237229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 msgr2=0x7f32140cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:23.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 -- 172.21.15.157:0/2578237229 >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 msgr2=0x7f32140d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 --2- 172.21.15.157:0/2578237229 >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 0x7f32140d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 -- 172.21.15.157:0/2578237229 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140d1e40 con 0x7f321415caa0 2023-11-06T03:04:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3218a53640 1 --2- 172.21.15.157:0/2578237229 >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 0x7f32140d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.813+0000 7f3213fff640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32140cdc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32140d0780 tx=0x7f320401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1575263f26ac699 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3212ffd640 1 -- 172.21.15.157:0/2578237229 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32040aa5f0 con 0x7f321415caa0 2023-11-06T03:04:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3212ffd640 1 -- 172.21.15.157:0/2578237229 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32040aa750 con 0x7f321415caa0 2023-11-06T03:04:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3212ffd640 1 -- 172.21.15.157:0/2578237229 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32040aaae0 con 0x7f321415caa0 2023-11-06T03:04:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/2578237229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 msgr2=0x7f32140cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32140cdc60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32140d0780 tx=0x7f320401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/2578237229 shutdown_connections 2023-11-06T03:04:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- 172.21.15.157:0/2578237229 >> v2:172.21.15.195:3300/0 conn(0x7f32140ce1a0 0x7f32140d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- 172.21.15.157:0/2578237229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32140cdc60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/2578237229 >> 172.21.15.157:0/2578237229 conn(0x7f3214070790 msgr2=0x7f3214070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/2578237229 shutdown_connections 2023-11-06T03:04:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/2578237229 wait complete. 2023-11-06T03:04:23.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 Processor -- start 2023-11-06T03:04:23.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- start start 2023-11-06T03:04:23.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32140ce1a0 0x7f32141520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214153050 con 0x7f321415caa0 2023-11-06T03:04:23.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321415a020 con 0x7f32140ce1a0 2023-11-06T03:04:23.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3214152720 con 0x7f321415ce80 2023-11-06T03:04:23.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35816/0 (socket says 172.21.15.157:35816) 2023-11-06T03:04:23.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 -- 172.21.15.157:0/3345181341 learned_addr learned my addr 172.21.15.157:0/3345181341 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:23.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3213fff640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32140ce1a0 0x7f32141520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3218a53640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 -- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 msgr2=0x7f32140b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 -- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32140ce1a0 msgr2=0x7f32141520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:23.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32140ce1a0 0x7f32141520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:23.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 -- 172.21.15.157:0/3345181341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040094d0 con 0x7f321415caa0 2023-11-06T03:04:23.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f32137fe640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f320800d800 tx=0x7f320800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800b6c0 con 0x7f321415caa0 2023-11-06T03:04:23.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800b820 con 0x7f321415caa0 2023-11-06T03:04:23.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3214152a00 con 0x7f321415caa0 2023-11-06T03:04:23.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800bb90 con 0x7f321415caa0 2023-11-06T03:04:23.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.817+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3214153a20 con 0x7f321415caa0 2023-11-06T03:04:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32141531c0 con 0x7f321415caa0 2023-11-06T03:04:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f320800bcf0 con 0x7f321415caa0 2023-11-06T03:04:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f3210ff9640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 0x7f3200067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32080837f0 con 0x7f321415caa0 2023-11-06T03:04:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f3213fff640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 0x7f3200067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.821+0000 7f3213fff640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 0x7f3200067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32040ae000 tx=0x7f32040a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:23.825+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f32080510e0 con 0x7f321415caa0 2023-11-06T03:04:24.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.013+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3214072750 con 0x7f321415caa0 2023-11-06T03:04:24.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f3210ff9640 1 -- 172.21.15.157:0/3345181341 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f3208011340 con 0x7f321415caa0 2023-11-06T03:04:24.015 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:24.015 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:04:24.015 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:04:24.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 msgr2=0x7f3200067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 0x7f3200067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32040ae000 tx=0x7f32040a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:24.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 msgr2=0x7f32141534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f320800d800 tx=0x7f320800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 shutdown_connections 2023-11-06T03:04:24.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3200065a60 0x7f3200067f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f321415ce80 0x7f32140b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f321415caa0 0x7f32141534e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 --2- 172.21.15.157:0/3345181341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32140ce1a0 0x7f32141520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 >> 172.21.15.157:0/3345181341 conn(0x7f3214070790 msgr2=0x7f32140b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 shutdown_connections 2023-11-06T03:04:24.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.017+0000 7f321a4b6640 1 -- 172.21.15.157:0/3345181341 wait complete. 2023-11-06T03:04:24.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:04:06.679919+0000', 'modified': '2023-11-06T03:04:09.322110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6835/1904182839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 1904182839}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 1904182839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:04:24.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:04:24.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T03:04:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 Processor -- start 2023-11-06T03:04:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 -- start start 2023-11-06T03:04:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f299c1599c0 0x7f299c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 0x7f299c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c15e440 con 0x7f299c15dea0 2023-11-06T03:04:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c15adb0 con 0x7f299c1599c0 2023-11-06T03:04:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a2f2d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c06c240 con 0x7f299c06d000 2023-11-06T03:04:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f299c1599c0 0x7f299c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f299c1599c0 0x7f299c0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35828/0 (socket says 172.21.15.157:35828) 2023-11-06T03:04:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 -- 172.21.15.157:0/1441263984 learned_addr learned my addr 172.21.15.157:0/1441263984 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f299bfff640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 0x7f299c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 -- 172.21.15.157:0/1441263984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 msgr2=0x7f299c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 0x7f299c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 -- 172.21.15.157:0/1441263984 >> v1:172.21.15.195:6789/0 conn(0x7f299c15dea0 legacy=0x7f299c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f29a0cc9640 1 -- 172.21.15.157:0/1441263984 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c06c520 con 0x7f299c1599c0 2023-11-06T03:04:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.569+0000 7f299bfff640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 0x7f299c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c0ae000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2994009870 tx=0x7f299401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f48106a0b3bb070 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f299b7fe640 1 -- 172.21.15.157:0/1441263984 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940ab620 con 0x7f299c1599c0 2023-11-06T03:04:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f299b7fe640 1 -- 172.21.15.157:0/1441263984 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29940ab780 con 0x7f299c1599c0 2023-11-06T03:04:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f299b7fe640 1 -- 172.21.15.157:0/1441263984 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940abb10 con 0x7f299c1599c0 2023-11-06T03:04:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 -- 172.21.15.157:0/1441263984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 msgr2=0x7f299c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c0ae000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2994009870 tx=0x7f299401c510 comp rx=0 tx=0).stop 2023-11-06T03:04:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 -- 172.21.15.157:0/1441263984 shutdown_connections 2023-11-06T03:04:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c06d000 0x7f299c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/1441263984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c0ae000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 -- 172.21.15.157:0/1441263984 >> 172.21.15.157:0/1441263984 conn(0x7f299c065930 msgr2=0x7f299c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 -- 172.21.15.157:0/1441263984 shutdown_connections 2023-11-06T03:04:24.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.573+0000 7f29a2f2d640 1 -- 172.21.15.157:0/1441263984 wait complete. 2023-11-06T03:04:24.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 Processor -- start 2023-11-06T03:04:24.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- start start 2023-11-06T03:04:24.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c06d000 0x7f299c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 0x7f299c150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c152050 con 0x7f299c1599c0 2023-11-06T03:04:24.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0ce040 con 0x7f299c06d000 2023-11-06T03:04:24.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0ce320 con 0x7f299c15dea0 2023-11-06T03:04:24.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35832/0 (socket says 172.21.15.157:35832) 2023-11-06T03:04:24.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 -- 172.21.15.157:0/251638016 learned_addr learned my addr 172.21.15.157:0/251638016 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a14ca640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 0x7f299c150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c06d000 0x7f299c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 -- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 msgr2=0x7f299c150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 0x7f299c150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 -- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c06d000 msgr2=0x7f299c0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c06d000 0x7f299c0cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 -- 172.21.15.157:0/251638016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29940094d0 con 0x7f299c1599c0 2023-11-06T03:04:24.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a14ca640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 0x7f299c150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:24.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f299bfff640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f299000c7c0 tx=0x7f299000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29997fa640 1 -- 172.21.15.157:0/251638016 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990007a30 con 0x7f299c1599c0 2023-11-06T03:04:24.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29997fa640 1 -- 172.21.15.157:0/251638016 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990007b90 con 0x7f299c1599c0 2023-11-06T03:04:24.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c0ce600 con 0x7f299c1599c0 2023-11-06T03:04:24.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29997fa640 1 -- 172.21.15.157:0/251638016 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000ce70 con 0x7f299c1599c0 2023-11-06T03:04:24.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c14cc70 con 0x7f299c1599c0 2023-11-06T03:04:24.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.577+0000 7f29997fa640 1 -- 172.21.15.157:0/251638016 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29900113f0 con 0x7f299c1599c0 2023-11-06T03:04:24.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29997fa640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 0x7f2988067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29997fa640 1 -- 172.21.15.157:0/251638016 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2990083770 con 0x7f299c1599c0 2023-11-06T03:04:24.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 0x7f2988067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 Processor -- start 2023-11-06T03:04:24.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- start start 2023-11-06T03:04:24.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 0x7f2988067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29940af000 tx=0x7f2994008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f739960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f73a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f5fdfc0 con 0x564d4f783930 2023-11-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f58c1a0 con 0x564d4f784250 2023-11-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4f589770 con 0x564d4f739ea0 2023-11-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35838/0 (socket says 172.21.15.157:35838) 2023-11-06T03:04:24.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2138937715 learned_addr learned my addr 172.21.15.157:0/2138937715 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f299bfff640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f739960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a14ca640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f73a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 msgr2=0x564d4f73a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f73a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 msgr2=0x564d4f739960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f739960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2138937715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d4f281890 con 0x564d4f783930 2023-11-06T03:04:24.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f299bfff640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f739960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:24.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f29940a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef243d59abe24fce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29827fc640 1 -- 172.21.15.157:0/2138937715 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940b4ac0 con 0x564d4f783930 2023-11-06T03:04:24.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29827fc640 1 -- 172.21.15.157:0/2138937715 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29940bc040 con 0x564d4f783930 2023-11-06T03:04:24.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29827fc640 1 -- 172.21.15.157:0/2138937715 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940b4e20 con 0x564d4f783930 2023-11-06T03:04:24.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- 172.21.15.157:0/2138937715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 msgr2=0x564d4f783d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f29940a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:24.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- 172.21.15.157:0/2138937715 shutdown_connections 2023-11-06T03:04:24.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f73a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f739960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 --2- 172.21.15.157:0/2138937715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f783d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- 172.21.15.157:0/2138937715 >> 172.21.15.157:0/2138937715 conn(0x564d4f73bbf0 msgr2=0x564d4f733e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- 172.21.15.157:0/2138937715 shutdown_connections 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- 172.21.15.157:0/2138937715 wait complete. 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 -- start start 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.581+0000 7f29a6055000 1 Processor -- start 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f770e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f7718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f585060 con 0x564d4f783930 2023-11-06T03:04:24.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f524290 con 0x564d4f784250 2023-11-06T03:04:24.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4f5426f0 con 0x564d4f739ea0 2023-11-06T03:04:24.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 10 client.0 _ll_register_callbacks cb 0x564d4f7211d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:04:24.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35840/0 (socket says 172.21.15.157:35840) 2023-11-06T03:04:24.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 -- 172.21.15.157:0/2117471743 learned_addr learned my addr 172.21.15.157:0/2117471743 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a14ca640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f7718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f770e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 -- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 msgr2=0x564d4f770e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f770e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 -- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 msgr2=0x564d4f7718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f7718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 -- 172.21.15.157:0/2117471743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c06c520 con 0x564d4f783930 2023-11-06T03:04:24.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f770e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:24.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f299bfff640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2990081bf0 tx=0x7f2990058b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f5e75d1a79ce02 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:04:24.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990054210 con 0x564d4f783930 2023-11-06T03:04:24.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2990054370 con 0x564d4f783930 2023-11-06T03:04:24.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d4f5839b0 con 0x564d4f783930 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f299005c870 con 0x564d4f783930 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 10 client.4707 fetch_fsmap learned FSMap version 7 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x564d4f54a1e0 con 0x564d4f783930 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990070080 con 0x564d4f783930 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 7) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2990021070 con 0x564d4f783930 2023-11-06T03:04:24.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 10 client.4707 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T03:04:24.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 10 client.4707 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T03:04:24.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 20 client.4707 populate_metadata read hostname 'smithi157' 2023-11-06T03:04:24.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f772160 0x564d4f7950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 4 client.4707 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:04:24.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 --> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d4f5b44d0 con 0x564d4f772160 2023-11-06T03:04:24.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f772160 0x564d4f7950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f297b7fe640 10 client.4707 ms_handle_connect on v2:172.21.15.195:6834/1904182839 2023-11-06T03:04:24.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.585+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f772160 0x564d4f7950b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12dd369126de4dea server_cookie=ce6f71d7b37b9308 in_seq=0 out_seq=0 2023-11-06T03:04:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f297b7fe640 1 -- 172.21.15.157:0/2117471743 <== mds.0 v2:172.21.15.195:6834/1904182839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564d4f5b44d0 con 0x564d4f772160 2023-11-06T03:04:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f297b7fe640 10 client.4707 handle_command_reply: tid=0 2023-11-06T03:04:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 client.4707 shutdown 2023-11-06T03:04:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 msgr2=0x564d4f771380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2990081bf0 tx=0x7f2990058b50 comp rx=0 tx=0).stop 2023-11-06T03:04:24.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 shutdown_connections 2023-11-06T03:04:24.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f772160 0x564d4f7950b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f784250 0x564d4f7718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f783930 0x564d4f771380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 --2- 172.21.15.157:0/2117471743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f739ea0 0x564d4f770e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f297b7fe640 0 client.4707 ms_handle_reset on v2:172.21.15.195:6834/1904182839 2023-11-06T03:04:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 >> 172.21.15.157:0/2117471743 conn(0x564d4f73bbf0 msgr2=0x564d4f73b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 shutdown_connections 2023-11-06T03:04:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 1 -- 172.21.15.157:0/2117471743 wait complete. 2023-11-06T03:04:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.589+0000 7f29a6055000 20 client.4707 trim_cache size 0 max 16384 2023-11-06T03:04:24.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 Processor -- start 2023-11-06T03:04:24.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 -- start start 2023-11-06T03:04:24.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f733ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f242940 con 0x564d4f739ea0 2023-11-06T03:04:24.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f735400 con 0x564d4f783930 2023-11-06T03:04:24.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4f4bc450 con 0x564d4f784250 2023-11-06T03:04:24.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35850/0 (socket says 172.21.15.157:35850) 2023-11-06T03:04:24.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.601+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2736567978 learned_addr learned my addr 172.21.15.157:0/2736567978 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 msgr2=0x564d4f733ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f733ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 msgr2=0x564d4f7335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2736567978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d4f530330 con 0x564d4f739ea0 2023-11-06T03:04:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f2994004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d4ef35e7f7232c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f2982ffd640 1 -- 172.21.15.157:0/2736567978 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940cf040 con 0x564d4f739ea0 2023-11-06T03:04:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f2982ffd640 1 -- 172.21.15.157:0/2736567978 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29940bc040 con 0x564d4f739ea0 2023-11-06T03:04:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f2982ffd640 1 -- 172.21.15.157:0/2736567978 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940043c0 con 0x564d4f739ea0 2023-11-06T03:04:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2736567978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 msgr2=0x564d4f76ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f2994004740 comp rx=0 tx=0).stop 2023-11-06T03:04:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2736567978 shutdown_connections 2023-11-06T03:04:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f733ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- 172.21.15.157:0/2736567978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f76ffc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2736567978 >> 172.21.15.157:0/2736567978 conn(0x564d4f73bbf0 msgr2=0x564d4f7415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2736567978 shutdown_connections 2023-11-06T03:04:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2736567978 wait complete. 2023-11-06T03:04:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- start start 2023-11-06T03:04:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 Processor -- start 2023-11-06T03:04:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f7e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f553ec0 con 0x564d4f739ea0 2023-11-06T03:04:24.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4f5346a0 con 0x564d4f783930 2023-11-06T03:04:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4f606650 con 0x564d4f784250 2023-11-06T03:04:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 10 client.0 _ll_register_callbacks cb 0x564d4f7019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:04:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35854/0 (socket says 172.21.15.157:35854) 2023-11-06T03:04:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2373807130 learned_addr learned my addr 172.21.15.157:0/2373807130 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a14ca640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f7e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 msgr2=0x564d4f7e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f7e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 msgr2=0x564d4f7e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 -- 172.21.15.157:0/2373807130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d4f281890 con 0x564d4f739ea0 2023-11-06T03:04:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a0cc9640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f299401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310199b3fc1e3f99 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940b47c0 con 0x564d4f739ea0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29940b4920 con 0x564d4f739ea0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d4f554b90 con 0x564d4f739ea0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29940b4ae0 con 0x564d4f739ea0 2023-11-06T03:04:24.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 10 client.4713 fetch_fsmap learned FSMap version 7 2023-11-06T03:04:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x564d4f281cc0 con 0x564d4f739ea0 2023-11-06T03:04:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d4f554b90 con 0x564d4f739ea0 2023-11-06T03:04:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 7) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f29940bf070 con 0x564d4f739ea0 2023-11-06T03:04:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 10 client.4713 fetch_fsmap finished waiting for FSMap version 7 2023-11-06T03:04:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 10 client.4713 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T03:04:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 20 client.4713 populate_metadata read hostname 'smithi157' 2023-11-06T03:04:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f7e5990 0x564d4f7e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 4 client.4713 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-11-06T03:04:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 --> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564d4f57e0e0 con 0x564d4f7e5990 2023-11-06T03:04:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.605+0000 7f299bfff640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f7e5990 0x564d4f7e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f297b7fe640 10 client.4713 ms_handle_connect on v2:172.21.15.195:6834/1904182839 2023-11-06T03:04:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f299bfff640 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f7e5990 0x564d4f7e7d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25cf824b9da8f668 server_cookie=cd92f961221c1c65 in_seq=0 out_seq=0 2023-11-06T03:04:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f297b7fe640 1 -- 172.21.15.157:0/2373807130 <== mds.0 v2:172.21.15.195:6834/1904182839 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564d4f57e0e0 con 0x564d4f7e5990 2023-11-06T03:04:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f297b7fe640 10 client.4713 handle_command_reply: tid=0 2023-11-06T03:04:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 client.4713 shutdown 2023-11-06T03:04:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 msgr2=0x564d4f7e09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2994097ef0 tx=0x7f299401ad50 comp rx=0 tx=0).stop 2023-11-06T03:04:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 shutdown_connections 2023-11-06T03:04:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:6834/1904182839,v1:172.21.15.195:6835/1904182839] conn(0x564d4f7e5990 0x564d4f7e7d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564d4f784250 0x564d4f7e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d4f783930 0x564d4f7e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 --2- 172.21.15.157:0/2373807130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564d4f739ea0 0x564d4f7e09b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f297b7fe640 0 client.4713 ms_handle_reset on v2:172.21.15.195:6834/1904182839 2023-11-06T03:04:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 >> 172.21.15.157:0/2373807130 conn(0x564d4f73bbf0 msgr2=0x564d4f73a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 shutdown_connections 2023-11-06T03:04:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 1 -- 172.21.15.157:0/2373807130 wait complete. 2023-11-06T03:04:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a6055000 20 client.4713 trim_cache size 0 max 16384 2023-11-06T03:04:24.639 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:04:24.640 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:04:24.640 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:04:24.640 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4306, 2023-11-06T03:04:24.640 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:04:24.641 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:04:24.641 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:04:24.641 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 17.924876877999999, 2023-11-06T03:04:24.642 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 7, 2023-11-06T03:04:24.642 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 23, 2023-11-06T03:04:24.642 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 23, 2023-11-06T03:04:24.642 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 22.221305762 2023-11-06T03:04:24.642 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:04:24.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 msgr2=0x7f2988067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 0x7f2988067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29940af000 tx=0x7f2994008040 comp rx=0 tx=0).stop 2023-11-06T03:04:24.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 msgr2=0x7f299c14c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f299000c7c0 tx=0x7f299000cc80 comp rx=0 tx=0).stop 2023-11-06T03:04:24.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 shutdown_connections 2023-11-06T03:04:24.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2988065a60 0x7f2988067f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f299c15dea0 0x7f299c150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f299c1599c0 0x7f299c14c730 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 --2- 172.21.15.157:0/251638016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f299c06d000 0x7f299c0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 >> 172.21.15.157:0/251638016 conn(0x7f299c065930 msgr2=0x7f299c067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 shutdown_connections 2023-11-06T03:04:24.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.609+0000 7f29a2f2d640 1 -- 172.21.15.157:0/251638016 wait complete. 2023-11-06T03:04:24.646 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-06T03:04:24.671 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-06T03:04:24.672 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-06T03:04:24.672 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-06T03:04:24.672 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T03:04:24.673 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:24.673 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-06T03:04:24.674 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T03:04:24.674 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:24.674 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-06T03:04:24.675 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T03:04:24.675 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:24.675 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-06T03:04:24.676 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-06T03:04:24.676 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:24.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:24.677 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:04:24.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:24.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-06T03:04:24.741 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2023-11-06T03:04:24.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:04:24.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:24.743 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2023-11-06T03:04:24.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:24.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2023-11-06T03:04:24.812 INFO:teuthology.orchestra.run.smithi195.stderr:Cannot find device "ceph-brx" 2023-11-06T03:04:24.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:04:24.814 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-06T03:04:24.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:04:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 Processor -- start 2023-11-06T03:04:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 -- start start 2023-11-06T03:04:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa080b0b80 0x7ffa0815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0815a960 con 0x7ffa080cb8e0 2023-11-06T03:04:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0815ac40 con 0x7ffa080b0b80 2023-11-06T03:04:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.941+0000 7ffa0fec9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0815af20 con 0x7ffa080afc40 2023-11-06T03:04:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:33768/0 (socket says 172.21.15.157:33768) 2023-11-06T03:04:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 -- 172.21.15.157:0/2715773763 learned_addr learned my addr 172.21.15.157:0/2715773763 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0e466640 1 --2- 172.21.15.157:0/2715773763 >> v2:172.21.15.195:3300/0 conn(0x7ffa080b0b80 0x7ffa0815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 -- 172.21.15.157:0/2715773763 >> v2:172.21.15.195:3300/0 conn(0x7ffa080b0b80 msgr2=0x7ffa0815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 --2- 172.21.15.157:0/2715773763 >> v2:172.21.15.195:3300/0 conn(0x7ffa080b0b80 0x7ffa0815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 -- 172.21.15.157:0/2715773763 >> v1:172.21.15.195:6789/0 conn(0x7ffa080cb8e0 legacy=0x7ffa080b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 -- 172.21.15.157:0/2715773763 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0815b200 con 0x7ffa080afc40 2023-11-06T03:04:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.945+0000 7ffa0dc65640 1 --2- 172.21.15.157:0/2715773763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009870 tx=0x7ffa0401cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79aeb85d94dae896 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0cc63640 1 -- 172.21.15.157:0/2715773763 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040ab620 con 0x7ffa080afc40 2023-11-06T03:04:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0cc63640 1 -- 172.21.15.157:0/2715773763 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa040ab780 con 0x7ffa080afc40 2023-11-06T03:04:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0cc63640 1 -- 172.21.15.157:0/2715773763 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040abb10 con 0x7ffa080afc40 2023-11-06T03:04:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- 172.21.15.157:0/2715773763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 msgr2=0x7ffa080cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/2715773763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009870 tx=0x7ffa0401cee0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- 172.21.15.157:0/2715773763 shutdown_connections 2023-11-06T03:04:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/2715773763 >> v2:172.21.15.195:3300/0 conn(0x7ffa080b0b80 0x7ffa0815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/2715773763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080afc40 0x7ffa080cb3a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- 172.21.15.157:0/2715773763 >> 172.21.15.157:0/2715773763 conn(0x7ffa08065930 msgr2=0x7ffa08071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- 172.21.15.157:0/2715773763 shutdown_connections 2023-11-06T03:04:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- 172.21.15.157:0/2715773763 wait complete. 2023-11-06T03:04:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 Processor -- start 2023-11-06T03:04:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- start start 2023-11-06T03:04:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa080afc40 0x7ffa08154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080cb8e0 0x7ffa0814c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0814fb00 con 0x7ffa080b0b80 2023-11-06T03:04:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08154c50 con 0x7ffa080afc40 2023-11-06T03:04:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0fec9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08155f80 con 0x7ffa080cb8e0 2023-11-06T03:04:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35860/0 (socket says 172.21.15.157:35860) 2023-11-06T03:04:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 -- 172.21.15.157:0/3531278281 learned_addr learned my addr 172.21.15.157:0/3531278281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0dc65640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa080afc40 0x7ffa08154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0e466640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080cb8e0 0x7ffa0814c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 -- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080cb8e0 msgr2=0x7ffa0814c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080cb8e0 0x7ffa0814c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 -- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa080afc40 msgr2=0x7ffa08154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa080afc40 0x7ffa08154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 -- 172.21.15.157:0/3531278281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040094d0 con 0x7ffa080b0b80 2023-11-06T03:04:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.949+0000 7ffa0d464640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c340 tx=0x7ff9f800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80052d0 con 0x7ffa080b0b80 2023-11-06T03:04:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f8005430 con 0x7ffa080b0b80 2023-11-06T03:04:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa081553d0 con 0x7ffa080b0b80 2023-11-06T03:04:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8007260 con 0x7ffa080b0b80 2023-11-06T03:04:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08155990 con 0x7ffa080b0b80 2023-11-06T03:04:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa08138350 con 0x7ffa080b0b80 2023-11-06T03:04:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9f80073c0 con 0x7ffa080b0b80 2023-11-06T03:04:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 0x7ff9ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9f8082650 con 0x7ffa080b0b80 2023-11-06T03:04:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ffa0dc65640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 0x7ff9ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:24.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.953+0000 7ffa0dc65640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 0x7ff9ec067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa040af000 tx=0x7ffa04004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:24.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:24.957+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9f804fec0 con 0x7ffa080b0b80 2023-11-06T03:04:25.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.129+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa08072670 con 0x7ffa080b0b80 2023-11-06T03:04:25.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ff9fe7fc640 1 -- 172.21.15.157:0/3531278281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9f80534e0 con 0x7ffa080b0b80 2023-11-06T03:04:25.131 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:04:25.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 msgr2=0x7ff9ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:25.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 0x7ff9ec067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa040af000 tx=0x7ffa04004740 comp rx=0 tx=0).stop 2023-11-06T03:04:25.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 msgr2=0x7ffa0814dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:25.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c340 tx=0x7ff9f800c800 comp rx=0 tx=0).stop 2023-11-06T03:04:25.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 shutdown_connections 2023-11-06T03:04:25.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9ec065a60 0x7ff9ec067f20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:25.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa080cb8e0 0x7ffa0814c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:25.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa080b0b80 0x7ffa0814dd60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:25.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 --2- 172.21.15.157:0/3531278281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa080afc40 0x7ffa08154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:25.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 >> 172.21.15.157:0/3531278281 conn(0x7ffa08065930 msgr2=0x7ffa080ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:25.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 shutdown_connections 2023-11-06T03:04:25.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:25.133+0000 7ffa0fec9640 1 -- 172.21.15.157:0/3531278281 wait complete. 2023-11-06T03:04:25.148 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:04:25.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:04:25.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:04:25.149 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:04:25.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:25.149 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:04:25.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:04:25.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:04:25.150 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:04:25.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.151 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:04:25.156 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:25.157 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:25.157 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:25.157 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:25.157 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:25.158 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:25.158 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:25.158 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:25.158 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:04:25.158 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:25.159 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:25.159 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:04:25.159 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:25.159 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:25.159 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42474sec preferred_lft 42474sec 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:25.160 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:04:25.161 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:25.161 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:04:25.161 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:25.161 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T03:04:25.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.162 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:25.162 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:25.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2023-11-06T03:04:25.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2023-11-06T03:04:25.163 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2023-11-06T03:04:25.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T03:04:25.164 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:25.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.233 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:04:25.292 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:04:25.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.293 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:04:25.377 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:04:25.377 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:04:25.378 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:04:25.378 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:04:25.378 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:04:25.378 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:04:25.379 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:25.379 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:25.379 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:25.379 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:04:25.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.380 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:25.381 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:25.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:04:25.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:04:25.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:04:25.382 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:25.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.453 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:04:25.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.500 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-11-06T03:04:25.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.547 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:25.548 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:25.548 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:04:25.548 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-06T03:04:25.548 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:25.614 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-06T03:04:25.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.615 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:25.615 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:25.615 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-06T03:04:25.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T03:04:25.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-06T03:04:25.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-06T03:04:25.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-06T03:04:25.617 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:25.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.721 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:25.721 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:25.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:04:25.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T03:04:25.722 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:25.741 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:04:25.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:25.742 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:25.790 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:04:25.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:25.791 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:25.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:25.893 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:25.894 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:25.944 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:25.946 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:25.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:25.947 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:25.995 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:04:25.995 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:04:26.040 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:26.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:26.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:26.108 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:26.109 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:26.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:26.110 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:26.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:04:26.145+0000 7fc834fb5580 -1 init, newargv = 0x56231139b110 newargc=15 2023-11-06T03:04:26.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[45039]: starting ceph client 2023-11-06T03:04:26.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[45039]: starting fuse 2023-11-06T03:04:27.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:27.265 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:27.266 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:27.318 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:27.319 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:27.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:27.320 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:04:27.371 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:04:27.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:04:27.372 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:04:27.372 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:04:27.373 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:04:27.373 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:04:27.373 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:27.374 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:04:27.374 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:04:27.374 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:04:27.375 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:04:27.375 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:27.375 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:04:27.375 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:04:27.376 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:04:27.376 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:04:27.376 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:27.377 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:04:27.377 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:04:27.377 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:04:27.378 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:04:27.378 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:04:27.378 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:04:27.378 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:04:27.379 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:04:27.379 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:27.379 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:04:27.380 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:04:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.45039.asok 2023-11-06T03:04:27.675 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45039.asok 2023-11-06T03:04:27.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:27.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45039.asok status 2023-11-06T03:04:27.833 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:04:27.833 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:04:27.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:04:27.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:04:27.834 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:04:27.834 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:04:27.834 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:04:27.835 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "45039", 2023-11-06T03:04:27.835 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:04:27.835 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:27.835 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:04:27.835 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:04:27.836 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4725, 2023-11-06T03:04:27.836 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:04:27.836 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:04:27.836 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:04:27.837 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4725 2023-11-06T03:04:27.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:27.837 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:04:27.837 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:04:27.837 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:04:27.838 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 788868910 2023-11-06T03:04:27.838 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:04:27.838 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:27.838 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:04:27.838 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:04:27.839 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:04:27.839 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 788868910 2023-11-06T03:04:27.839 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:27.839 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4725 192.168.0.1:0/788868910", 2023-11-06T03:04:27.839 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/788868910", 2023-11-06T03:04:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:04:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 7, 2023-11-06T03:04:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 23, 2023-11-06T03:04:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:04:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:04:27.841 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:04:27.841 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:04:27.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:04:27.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 Processor -- start 2023-11-06T03:04:27.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 -- start start 2023-11-06T03:04:27.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f341415caa0 0x7f34140cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f341415ce80 0x7f34140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140d2920 con 0x7f34140ce1a0 2023-11-06T03:04:27.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140d18a0 con 0x7f341415ce80 2023-11-06T03:04:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341c437640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140d1b60 con 0x7f341415caa0 2023-11-06T03:04:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f341415ce80 0x7f34140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f341415ce80 0x7f34140cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35864/0 (socket says 172.21.15.157:35864) 2023-11-06T03:04:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 -- 172.21.15.157:0/2947198357 learned_addr learned my addr 172.21.15.157:0/2947198357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f341a1d3640 1 --2- 172.21.15.157:0/2947198357 >> v2:172.21.15.195:3300/0 conn(0x7f341415caa0 0x7f34140cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 -- 172.21.15.157:0/2947198357 >> v2:172.21.15.195:3300/0 conn(0x7f341415caa0 msgr2=0x7f34140cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 --2- 172.21.15.157:0/2947198357 >> v2:172.21.15.195:3300/0 conn(0x7f341415caa0 0x7f34140cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 -- 172.21.15.157:0/2947198357 >> v1:172.21.15.195:6789/0 conn(0x7f34140ce1a0 legacy=0x7f34140d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.973+0000 7f34199d2640 1 -- 172.21.15.157:0/2947198357 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140d1e40 con 0x7f341415ce80 2023-11-06T03:04:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f34199d2640 1 --2- 172.21.15.157:0/2947198357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cfb70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34140b4f80 tx=0x7f340401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4c99c61ed723c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f34191d1640 1 -- 172.21.15.157:0/2947198357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040ac600 con 0x7f341415ce80 2023-11-06T03:04:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f34191d1640 1 -- 172.21.15.157:0/2947198357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34040a8050 con 0x7f341415ce80 2023-11-06T03:04:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f34191d1640 1 -- 172.21.15.157:0/2947198357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34040ac980 con 0x7f341415ce80 2023-11-06T03:04:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- 172.21.15.157:0/2947198357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 msgr2=0x7f34140cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- 172.21.15.157:0/2947198357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cfb70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34140b4f80 tx=0x7f340401c490 comp rx=0 tx=0).stop 2023-11-06T03:04:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- 172.21.15.157:0/2947198357 shutdown_connections 2023-11-06T03:04:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- 172.21.15.157:0/2947198357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cfb70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- 172.21.15.157:0/2947198357 >> v2:172.21.15.195:3300/0 conn(0x7f341415caa0 0x7f34140cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- 172.21.15.157:0/2947198357 >> 172.21.15.157:0/2947198357 conn(0x7f3414070790 msgr2=0x7f3414070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- 172.21.15.157:0/2947198357 shutdown_connections 2023-11-06T03:04:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- 172.21.15.157:0/2947198357 wait complete. 2023-11-06T03:04:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 Processor -- start 2023-11-06T03:04:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- start start 2023-11-06T03:04:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 0x7f34140c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f341415caa0 0x7f34140cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140cc990 con 0x7f341415ce80 2023-11-06T03:04:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414155450 con 0x7f341415caa0 2023-11-06T03:04:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341c437640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3414155730 con 0x7f34140ce1a0 2023-11-06T03:04:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a9d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a9d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35878/0 (socket says 172.21.15.157:35878) 2023-11-06T03:04:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a9d4640 1 -- 172.21.15.157:0/3344929860 learned_addr learned my addr 172.21.15.157:0/3344929860 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f34199d2640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f341415caa0 0x7f34140cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a1d3640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 0x7f34140c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a9d4640 1 -- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 msgr2=0x7f34140c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.977+0000 7f341a9d4640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 0x7f34140c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a9d4640 1 -- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f341415caa0 msgr2=0x7f34140cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a9d4640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f341415caa0 0x7f34140cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a9d4640 1 -- 172.21.15.157:0/3344929860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040094d0 con 0x7f341415ce80 2023-11-06T03:04:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a1d3640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 0x7f34140c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a9d4640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b9e0 tx=0x7f340c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c003b80 con 0x7f341415ce80 2023-11-06T03:04:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c003ce0 con 0x7f341415ce80 2023-11-06T03:04:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3414155a10 con 0x7f341415ce80 2023-11-06T03:04:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c00cb60 con 0x7f341415ce80 2023-11-06T03:04:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34141604b0 con 0x7f341415ce80 2023-11-06T03:04:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414072750 con 0x7f341415ce80 2023-11-06T03:04:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f340c00ccc0 con 0x7f341415ce80 2023-11-06T03:04:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 0x7f33f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f340c083010 con 0x7f341415ce80 2023-11-06T03:04:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a1d3640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 0x7f33f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.981+0000 7f341a1d3640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 0x7f33f8067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3410006fd0 tx=0x7f3410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:27.985+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f340c050870 con 0x7f341415ce80 2023-11-06T03:04:28.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3414155ec0 con 0x7f341415ce80 2023-11-06T03:04:28.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f340affd640 1 -- 172.21.15.157:0/3344929860 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f340c053e90 con 0x7f341415ce80 2023-11-06T03:04:28.168 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:04:28.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 msgr2=0x7f33f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 0x7f33f8067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3410006fd0 tx=0x7f3410008040 comp rx=0 tx=0).stop 2023-11-06T03:04:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 msgr2=0x7f34140cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b9e0 tx=0x7f340c00bea0 comp rx=0 tx=0).stop 2023-11-06T03:04:28.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 shutdown_connections 2023-11-06T03:04:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33f8065a60 0x7f33f8067f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341415ce80 0x7f34140cc450 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f341415caa0 0x7f34140cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 --2- 172.21.15.157:0/3344929860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34140ce1a0 0x7f34140c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:28.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 >> 172.21.15.157:0/3344929860 conn(0x7f3414070790 msgr2=0x7f34140d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 shutdown_connections 2023-11-06T03:04:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:28.169+0000 7f341c437640 1 -- 172.21.15.157:0/3344929860 wait complete. 2023-11-06T03:04:28.185 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:04:28.185 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:04:28.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:04:28.186 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-06T03:04:28.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:28.187 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:04:28.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:04:28.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:04:28.187 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-06T03:04:28.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.188 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:04:28.193 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:28.194 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:28.194 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:28.194 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.194 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:28.195 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.195 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:28.195 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.195 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:04:28.195 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:28.196 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.196 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:04:28.196 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:28.196 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.196 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:04:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:04:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42471sec preferred_lft 42471sec 2023-11-06T03:04:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:04:28.197 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:28.198 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:28.199 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:04:28.199 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.199 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:04:28.199 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.199 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:04:28.200 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:04:28.200 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:04:28.200 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.201 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:04:28.246 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:04:28.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.248 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:04:28.297 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:04:28.298 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:04:28.298 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:04:28.298 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:04:28.298 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:04:28.298 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:04:28.299 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:28.299 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:28.299 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:28.299 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:04:28.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.300 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:28.300 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:28.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:04:28.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:04:28.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:04:28.303 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:28.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.374 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:04:28.418 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:04:28.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.420 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2023-11-06T03:04:28.468 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-06T03:04:28.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.469 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:28.470 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:28.470 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:04:28.470 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-06T03:04:28.471 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:28.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.535 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T03:04:28.595 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:28.596 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:28.596 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:28.596 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.596 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:28.596 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.597 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:28.597 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 92:57:85:b4:62:86 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T03:04:28.597 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T03:04:28.597 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.597 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::9057:85ff:feb4:6286/64 scope link 2023-11-06T03:04:28.598 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.599 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-06T03:04:28.653 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:28.653 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:28.653 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:28.653 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.654 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:28.654 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.654 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:28.654 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 92:57:85:b4:62:86 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T03:04:28.654 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T03:04:28.655 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.655 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::9057:85ff:feb4:6286/64 scope link 2023-11-06T03:04:28.655 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:28.656 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-06T03:04:28.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.657 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:28.657 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:28.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-06T03:04:28.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T03:04:28.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-06T03:04:28.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-06T03:04:28.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-06T03:04:28.659 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:28.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.754 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:04:28.754 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:04:28.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2023-11-06T03:04:28.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T03:04:28.755 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:04:28.775 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-06T03:04:28.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:28.776 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:28.826 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-06T03:04:28.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:28.827 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:28.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:28.928 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:28.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:28.981 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:28.982 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:28.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:28.983 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:29.031 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:04:29.031 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-11-06T03:04:29.032 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-06T03:04:29.076 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:29.091 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:29.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:29.146 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:29.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:29.147 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:29.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi157.stderr:2023-11-06T03:04:29.153+0000 7fc5da192580 -1 init, newargv = 0x55db4b3a6cc0 newargc=15 2023-11-06T03:04:29.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi157.stderr:ceph-fuse[45176]: starting ceph client 2023-11-06T03:04:29.157 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:04:29.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi157.stderr:ceph-fuse[45176]: starting fuse 2023-11-06T03:04:30.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:04:30.275 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:30.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:30.337 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:30.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:30.339 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:04:30.402 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:04:30.403 INFO:teuthology.orchestra.run.smithi157.stdout:46 2023-11-06T03:04:30.403 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-11-06T03:04:30.403 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:04:30.404 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:04:30.404 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:04:30.404 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:04:30.405 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:04:30.405 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:30.405 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:04:30.405 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-06T03:04:30.406 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:04:30.406 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-06T03:04:30.406 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:30.407 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:04:30.407 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:04:30.407 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:04:30.407 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:04:30.408 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:30.408 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:04:30.408 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:04:30.409 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:04:30.409 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:04:30.409 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:04:30.409 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:04:30.410 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:04:30.410 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:04:30.410 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:04:30.411 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.2")) 2023-11-06T03:04:30.411 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:04:30.524 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.2.45176.asok 2023-11-06T03:04:30.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45176.asok 2023-11-06T03:04:30.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:30.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45176.asok status 2023-11-06T03:04:30.863 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:04:30.863 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:04:30.864 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:04:30.864 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:04:30.864 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "2", 2023-11-06T03:04:30.864 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:04:30.864 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-06T03:04:30.865 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "45176", 2023-11-06T03:04:30.865 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:04:30.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:30.865 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:04:30.865 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:04:30.866 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4737, 2023-11-06T03:04:30.866 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:04:30.866 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:04:30.866 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:04:30.866 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4737 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1629194839 2023-11-06T03:04:30.867 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:04:30.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:30.868 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:04:30.868 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:04:30.868 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2023-11-06T03:04:30.868 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1629194839 2023-11-06T03:04:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:04:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4737 192.168.0.2:0/1629194839", 2023-11-06T03:04:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.2:0/1629194839", 2023-11-06T03:04:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:04:30.869 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 7, 2023-11-06T03:04:30.870 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 23, 2023-11-06T03:04:30.870 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:04:30.870 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:04:30.870 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:04:30.870 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:04:30.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:04:30.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 Processor -- start 2023-11-06T03:04:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 -- start start 2023-11-06T03:04:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f24f00b2af0 0x7f24f00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24f006d000 0x7f24f0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f015e440 con 0x7f24f01599c0 2023-11-06T03:04:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f006c240 con 0x7f24f006d000 2023-11-06T03:04:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24f5563640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f006c520 con 0x7f24f00b2af0 2023-11-06T03:04:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24f006d000 0x7f24f0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24f006d000 0x7f24f0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51328/0 (socket says 172.21.15.157:51328) 2023-11-06T03:04:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 -- 172.21.15.157:0/839974128 learned_addr learned my addr 172.21.15.157:0/839974128 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24e67fc640 1 --2- 172.21.15.157:0/839974128 >> v2:172.21.15.195:3300/0 conn(0x7f24f00b2af0 0x7f24f00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 -- 172.21.15.157:0/839974128 >> v2:172.21.15.195:3300/0 conn(0x7f24f00b2af0 msgr2=0x7f24f00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 --2- 172.21.15.157:0/839974128 >> v2:172.21.15.195:3300/0 conn(0x7f24f00b2af0 0x7f24f00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 -- 172.21.15.157:0/839974128 >> v1:172.21.15.195:6789/0 conn(0x7f24f01599c0 legacy=0x7f24f00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.001+0000 7f24ef7fe640 1 -- 172.21.15.157:0/839974128 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f006c800 con 0x7f24f006d000 2023-11-06T03:04:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24ef7fe640 1 --2- 172.21.15.157:0/839974128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f006d000 0x7f24f0159da0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24f00cfa20 tx=0x7f24e801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8368c41c9b52cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24ee7fc640 1 -- 172.21.15.157:0/839974128 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800b2d0 con 0x7f24f006d000 2023-11-06T03:04:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24ee7fc640 1 -- 172.21.15.157:0/839974128 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800b430 con 0x7f24f006d000 2023-11-06T03:04:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24ee7fc640 1 -- 172.21.15.157:0/839974128 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e80a7cb0 con 0x7f24f006d000 2023-11-06T03:04:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- 172.21.15.157:0/839974128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f006d000 msgr2=0x7f24f0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- 172.21.15.157:0/839974128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f006d000 0x7f24f0159da0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24f00cfa20 tx=0x7f24e801eba0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- 172.21.15.157:0/839974128 shutdown_connections 2023-11-06T03:04:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- 172.21.15.157:0/839974128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f006d000 0x7f24f0159da0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- 172.21.15.157:0/839974128 >> v2:172.21.15.195:3300/0 conn(0x7f24f00b2af0 0x7f24f00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- 172.21.15.157:0/839974128 >> 172.21.15.157:0/839974128 conn(0x7f24f0065930 msgr2=0x7f24f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- 172.21.15.157:0/839974128 shutdown_connections 2023-11-06T03:04:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- 172.21.15.157:0/839974128 wait complete. 2023-11-06T03:04:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 Processor -- start 2023-11-06T03:04:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- start start 2023-11-06T03:04:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 0x7f24f014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f01599c0 0x7f24f0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0155860 con 0x7f24f00b2af0 2023-11-06T03:04:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0155b40 con 0x7f24f01599c0 2023-11-06T03:04:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.005+0000 7f24f5563640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f00adfc0 con 0x7f24f006d000 2023-11-06T03:04:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51342/0 (socket says 172.21.15.157:51342) 2023-11-06T03:04:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 -- 172.21.15.157:0/239919963 learned_addr learned my addr 172.21.15.157:0/239919963 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24ef7fe640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f01599c0 0x7f24f0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24eeffd640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 0x7f24f014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 -- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 msgr2=0x7f24f014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 0x7f24f014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 -- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f01599c0 msgr2=0x7f24f0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f01599c0 0x7f24f0158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 -- 172.21.15.157:0/239919963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e800bd40 con 0x7f24f00b2af0 2023-11-06T03:04:31.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24eeffd640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 0x7f24f014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:31.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e67fc640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d800 tx=0x7f24d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:31.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800b6c0 con 0x7f24f00b2af0 2023-11-06T03:04:31.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d800b820 con 0x7f24f00b2af0 2023-11-06T03:04:31.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f00ae2a0 con 0x7f24f00b2af0 2023-11-06T03:04:31.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800bb90 con 0x7f24f00b2af0 2023-11-06T03:04:31.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0153240 con 0x7f24f00b2af0 2023-11-06T03:04:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f0072750 con 0x7f24f00b2af0 2023-11-06T03:04:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f24d800bcf0 con 0x7f24f00b2af0 2023-11-06T03:04:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 0x7f24d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24d80837f0 con 0x7f24f00b2af0 2023-11-06T03:04:31.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24eeffd640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 0x7f24d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:31.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.009+0000 7f24eeffd640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 0x7f24d0067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000f70 tx=0x7f24e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:31.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.013+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f24d80510e0 con 0x7f24f00b2af0 2023-11-06T03:04:31.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.201+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24f01559a0 con 0x7f24f00b2af0 2023-11-06T03:04:31.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24e7fff640 1 -- 172.21.15.157:0/239919963 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f24f01559a0 con 0x7f24f00b2af0 2023-11-06T03:04:31.203 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:04:31.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 msgr2=0x7f24d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 0x7f24d0067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000f70 tx=0x7f24e0009040 comp rx=0 tx=0).stop 2023-11-06T03:04:31.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 msgr2=0x7f24f0152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:31.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d800 tx=0x7f24d800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 shutdown_connections 2023-11-06T03:04:31.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24d0065a60 0x7f24d0067f20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24f01599c0 0x7f24f0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24f00b2af0 0x7f24f0152d00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 --2- 172.21.15.157:0/239919963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24f006d000 0x7f24f014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:31.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 >> 172.21.15.157:0/239919963 conn(0x7f24f0065930 msgr2=0x7f24f00b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:31.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 shutdown_connections 2023-11-06T03:04:31.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:31.205+0000 7f24f5563640 1 -- 172.21.15.157:0/239919963 wait complete. 2023-11-06T03:04:31.221 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:04:31.222 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:04:31.222 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2023-11-06T03:04:31.222 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-06T03:04:31.222 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:31.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:04:31.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:04:31.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:04:31.224 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-06T03:04:31.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.224 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2023-11-06T03:04:31.231 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:31.233 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f0 2023-11-06T03:04:31.233 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f1 2023-11-06T03:04:31.234 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:31.234 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:04:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:04:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42444sec preferred_lft 42444sec 2023-11-06T03:04:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link 2023-11-06T03:04:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:31.235 INFO:teuthology.orchestra.run.smithi195.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:04:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 02:42:f3:d0:44:6b brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:04:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:31.236 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-06T03:04:31.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.237 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:31.237 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:31.238 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add name ceph-brx type bridge 2023-11-06T03:04:31.238 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr flush dev ceph-brx 2023-11-06T03:04:31.238 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set ceph-brx up 2023-11-06T03:04:31.238 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-06T03:04:31.239 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:31.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.316 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:04:31.368 INFO:teuthology.orchestra.run.smithi195.stdout:1 2023-11-06T03:04:31.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.370 DEBUG:teuthology.orchestra.run.smithi195:> route 2023-11-06T03:04:31.651 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2023-11-06T03:04:31.651 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:04:31.651 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:04:31.652 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:04:31.652 INFO:teuthology.orchestra.run.smithi195.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:04:31.652 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:04:31.652 INFO:teuthology.orchestra.run.smithi195.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:31.653 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:31.653 INFO:teuthology.orchestra.run.smithi195.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:31.653 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:04:31.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.654 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:31.654 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:31.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:04:31.655 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:04:31.655 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:04:31.655 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:31.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.691 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2023-11-06T03:04:31.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.741 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2023-11-06T03:04:31.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.789 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:31.789 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:31.789 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T03:04:31.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-06T03:04:31.790 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:31.857 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-06T03:04:31.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:31.858 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:31.858 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:31.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-06T03:04:31.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-06T03:04:31.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-06T03:04:31.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-06T03:04:31.860 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-06T03:04:31.860 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:31.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:32.000 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:32.000 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:32.001 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.0 up 2023-11-06T03:04:32.001 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.0 master ceph-brx 2023-11-06T03:04:32.001 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:32.021 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-06T03:04:32.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:32.022 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:32.067 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-06T03:04:32.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:32.068 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:32.115 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:32.170 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:32.171 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:32.223 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:32.224 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:32.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:32.225 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:32.272 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:04:32.273 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-06T03:04:32.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:32.331 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:32.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:32.386 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:32.387 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:32.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:32.389 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:32.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:2023-11-06T03:04:32.413+0000 7fa9a2fb9580 -1 init, newargv = 0x559080cf5ea0 newargc=15 2023-11-06T03:04:32.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:ceph-fuse[44138]: starting ceph client 2023-11-06T03:04:32.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi195.stderr:ceph-fuse[44138]: starting fuse 2023-11-06T03:04:33.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:33.537 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:33.538 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:33.592 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:33.593 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:33.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:33.594 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:33.640 INFO:teuthology.orchestra.run.smithi195.stdout:45 2023-11-06T03:04:33.640 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:04:33.641 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:04:33.641 DEBUG:teuthology.orchestra.run.smithi195:> import glob 2023-11-06T03:04:33.642 DEBUG:teuthology.orchestra.run.smithi195:> import re 2023-11-06T03:04:33.642 DEBUG:teuthology.orchestra.run.smithi195:> import os 2023-11-06T03:04:33.642 DEBUG:teuthology.orchestra.run.smithi195:> import subprocess 2023-11-06T03:04:33.643 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:33.643 DEBUG:teuthology.orchestra.run.smithi195:> def _find_admin_socket(client_name): 2023-11-06T03:04:33.643 DEBUG:teuthology.orchestra.run.smithi195:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-06T03:04:33.643 DEBUG:teuthology.orchestra.run.smithi195:> files = glob.glob(asok_path) 2023-11-06T03:04:33.644 DEBUG:teuthology.orchestra.run.smithi195:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-06T03:04:33.644 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:33.644 DEBUG:teuthology.orchestra.run.smithi195:> # Given a non-glob path, it better be there 2023-11-06T03:04:33.645 DEBUG:teuthology.orchestra.run.smithi195:> if "*" not in asok_path: 2023-11-06T03:04:33.645 DEBUG:teuthology.orchestra.run.smithi195:> assert(len(files) == 1) 2023-11-06T03:04:33.645 DEBUG:teuthology.orchestra.run.smithi195:> return files[0] 2023-11-06T03:04:33.645 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:33.646 DEBUG:teuthology.orchestra.run.smithi195:> for f in files: 2023-11-06T03:04:33.646 DEBUG:teuthology.orchestra.run.smithi195:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:04:33.646 DEBUG:teuthology.orchestra.run.smithi195:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:04:33.647 DEBUG:teuthology.orchestra.run.smithi195:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:04:33.647 DEBUG:teuthology.orchestra.run.smithi195:> contents = proc_f.read() 2023-11-06T03:04:33.647 DEBUG:teuthology.orchestra.run.smithi195:> if mountpoint in contents: 2023-11-06T03:04:33.647 DEBUG:teuthology.orchestra.run.smithi195:> return f 2023-11-06T03:04:33.648 DEBUG:teuthology.orchestra.run.smithi195:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:04:33.648 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:33.648 DEBUG:teuthology.orchestra.run.smithi195:> print(_find_admin_socket("client.1")) 2023-11-06T03:04:33.649 DEBUG:teuthology.orchestra.run.smithi195:> ' 2023-11-06T03:04:33.763 INFO:teuthology.orchestra.run.smithi195.stdout:/var/run/ceph/ceph-client.1.44138.asok 2023-11-06T03:04:33.946 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44138.asok 2023-11-06T03:04:33.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:33.947 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44138.asok status 2023-11-06T03:04:34.101 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-06T03:04:34.101 INFO:teuthology.orchestra.run.smithi195.stdout: "metadata": { 2023-11-06T03:04:34.102 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:04:34.102 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:04:34.102 INFO:teuthology.orchestra.run.smithi195.stdout: "entity_id": "1", 2023-11-06T03:04:34.102 INFO:teuthology.orchestra.run.smithi195.stdout: "hostname": "smithi195", 2023-11-06T03:04:34.103 INFO:teuthology.orchestra.run.smithi195.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-06T03:04:34.103 INFO:teuthology.orchestra.run.smithi195.stdout: "pid": "44138", 2023-11-06T03:04:34.103 INFO:teuthology.orchestra.run.smithi195.stdout: "root": "/" 2023-11-06T03:04:34.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:34.103 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_count": 0, 2023-11-06T03:04:34.104 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_pinned_count": 0, 2023-11-06T03:04:34.104 INFO:teuthology.orchestra.run.smithi195.stdout: "id": 4499, 2023-11-06T03:04:34.104 INFO:teuthology.orchestra.run.smithi195.stdout: "inst": { 2023-11-06T03:04:34.104 INFO:teuthology.orchestra.run.smithi195.stdout: "name": { 2023-11-06T03:04:34.104 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "client", 2023-11-06T03:04:34.105 INFO:teuthology.orchestra.run.smithi195.stdout: "num": 4499 2023-11-06T03:04:34.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:34.105 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2023-11-06T03:04:34.105 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2023-11-06T03:04:34.105 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:04:34.106 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 2262862663 2023-11-06T03:04:34.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-06T03:04:34.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:34.106 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2023-11-06T03:04:34.106 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2023-11-06T03:04:34.107 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:04:34.107 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 2262862663 2023-11-06T03:04:34.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:34.107 INFO:teuthology.orchestra.run.smithi195.stdout: "inst_str": "client.4499 192.168.0.1:0/2262862663", 2023-11-06T03:04:34.107 INFO:teuthology.orchestra.run.smithi195.stdout: "addr_str": "192.168.0.1:0/2262862663", 2023-11-06T03:04:34.108 INFO:teuthology.orchestra.run.smithi195.stdout: "inode_count": 1, 2023-11-06T03:04:34.108 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_epoch": 7, 2023-11-06T03:04:34.108 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch": 23, 2023-11-06T03:04:34.108 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:04:34.109 INFO:teuthology.orchestra.run.smithi195.stdout: "blocklisted": false, 2023-11-06T03:04:34.109 INFO:teuthology.orchestra.run.smithi195.stdout: "fs_name": "cephfs" 2023-11-06T03:04:34.109 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-06T03:04:34.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:04:34.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.245+0000 7f574ec90640 1 Processor -- start 2023-11-06T03:04:34.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 -- start start 2023-11-06T03:04:34.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1ad0 0x7f57480b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 0x7f57480c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f574815b860 0x7f574815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5748155610 con 0x7f574815b860 2023-11-06T03:04:34.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574815b3d0 con 0x7f57480b1eb0 2023-11-06T03:04:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ec90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574815b0c0 con 0x7f57480b1ad0 2023-11-06T03:04:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f574815b860 0x7f574815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f574815b860 0x7f574815bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51348/0 (socket says 172.21.15.157:51348) 2023-11-06T03:04:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 -- 172.21.15.157:0/925177857 learned_addr learned my addr 172.21.15.157:0/925177857 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f5747fff640 1 --2- 172.21.15.157:0/925177857 >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 0x7f57480c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574ca2c640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1ad0 0x7f57480b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 -- 172.21.15.157:0/925177857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1ad0 msgr2=0x7f57480b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1ad0 0x7f57480b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 -- 172.21.15.157:0/925177857 >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 msgr2=0x7f57480c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 --2- 172.21.15.157:0/925177857 >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 0x7f57480c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 -- 172.21.15.157:0/925177857 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57480b50c0 con 0x7f574815b860 2023-11-06T03:04:34.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f5747fff640 1 --2- 172.21.15.157:0/925177857 >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 0x7f57480c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:34.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.249+0000 7f574d22d640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f574815b860 0x7f574815bcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5748066460 tx=0x7f573801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d12d543ca94339a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:34.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57477fe640 1 -- 172.21.15.157:0/925177857 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380aa5f0 con 0x7f574815b860 2023-11-06T03:04:34.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57477fe640 1 -- 172.21.15.157:0/925177857 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57380aa750 con 0x7f574815b860 2023-11-06T03:04:34.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57477fe640 1 -- 172.21.15.157:0/925177857 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380aaae0 con 0x7f574815b860 2023-11-06T03:04:34.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/925177857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f574815b860 msgr2=0x7f574815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:34.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f574815b860 0x7f574815bcf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5748066460 tx=0x7f573801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/925177857 shutdown_connections 2023-11-06T03:04:34.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f574815b860 0x7f574815bcf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- 172.21.15.157:0/925177857 >> v2:172.21.15.195:3300/0 conn(0x7f57480b1eb0 0x7f57480c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- 172.21.15.157:0/925177857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1ad0 0x7f57480b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/925177857 >> 172.21.15.157:0/925177857 conn(0x7f574806a170 msgr2=0x7f574806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/925177857 shutdown_connections 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/925177857 wait complete. 2023-11-06T03:04:34.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 Processor -- start 2023-11-06T03:04:34.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- start start 2023-11-06T03:04:34.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 0x7f5748159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 0x7f574815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480caad0 con 0x7f57480b1ad0 2023-11-06T03:04:34.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480c9ec0 con 0x7f574815b860 2023-11-06T03:04:34.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480ca1a0 con 0x7f57480b1eb0 2023-11-06T03:04:34.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51352/0 (socket says 172.21.15.157:51352) 2023-11-06T03:04:34.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 -- 172.21.15.157:0/4135511228 learned_addr learned my addr 172.21.15.157:0/4135511228 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:34.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574d22d640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 0x7f574815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 -- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 msgr2=0x7f5748159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:34.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f5747fff640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 0x7f5748159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 0x7f5748159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 -- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 msgr2=0x7f574815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:34.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 0x7f574815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 -- 172.21.15.157:0/4135511228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380094d0 con 0x7f57480b1ad0 2023-11-06T03:04:34.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574d22d640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 0x7f574815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:34.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f5747fff640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 0x7f5748159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ca2c640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5740007e60 tx=0x7f574000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740014620 con 0x7f57480b1ad0 2023-11-06T03:04:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000ee90 con 0x7f57480b1ad0 2023-11-06T03:04:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57480ca480 con 0x7f57480b1ad0 2023-11-06T03:04:34.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57400149c0 con 0x7f57480b1ad0 2023-11-06T03:04:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.253+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574814cdc0 con 0x7f57480b1ad0 2023-11-06T03:04:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5748072750 con 0x7f57480b1ad0 2023-11-06T03:04:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5740014b20 con 0x7f57480b1ad0 2023-11-06T03:04:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f57457fa640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 0x7f5734067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:34.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5740083c80 con 0x7f57480b1ad0 2023-11-06T03:04:34.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f5747fff640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 0x7f5734067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:34.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.257+0000 7f5747fff640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 0x7f5734067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57481554a0 tx=0x7f573c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:34.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.261+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57400514d0 con 0x7f57480b1ad0 2023-11-06T03:04:34.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.449+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5748070f70 con 0x7f57480b1ad0 2023-11-06T03:04:34.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.449+0000 7f57457fa640 1 -- 172.21.15.157:0/4135511228 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5740009a20 con 0x7f57480b1ad0 2023-11-06T03:04:34.450 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:04:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 msgr2=0x7f5734067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 0x7f5734067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57481554a0 tx=0x7f573c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 msgr2=0x7f574814c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5740007e60 tx=0x7f574000e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 shutdown_connections 2023-11-06T03:04:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5734065a60 0x7f5734067f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f574815b860 0x7f574815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57480b1eb0 0x7f5748159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 --2- 172.21.15.157:0/4135511228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57480b1ad0 0x7f574814c880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 >> 172.21.15.157:0/4135511228 conn(0x7f574806a170 msgr2=0x7f57480b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 shutdown_connections 2023-11-06T03:04:34.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:34.453+0000 7f574ec90640 1 -- 172.21.15.157:0/4135511228 wait complete. 2023-11-06T03:04:34.467 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:04:34.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:04:34.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi195.front.sepia.ceph.com 2023-11-06T03:04:34.468 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-06T03:04:34.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:34.469 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:04:34.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:04:34.469 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:04:34.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-06T03:04:34.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.470 DEBUG:teuthology.orchestra.run.smithi195:> ip addr 2023-11-06T03:04:34.477 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:34.477 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:34.478 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:34.478 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.478 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:34.478 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.478 INFO:teuthology.orchestra.run.smithi195.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:34.479 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.479 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f0 2023-11-06T03:04:34.479 INFO:teuthology.orchestra.run.smithi195.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:34.479 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:d9:36:f1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.479 INFO:teuthology.orchestra.run.smithi195.stdout: altname enp7s0f1 2023-11-06T03:04:34.480 INFO:teuthology.orchestra.run.smithi195.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:04:34.480 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:80 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.480 INFO:teuthology.orchestra.run.smithi195.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:04:34.480 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0c:c4:7a:88:80:81 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.480 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.21.15.195/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:04:34.481 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft 42441sec preferred_lft 42441sec 2023-11-06T03:04:34.481 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::ec4:7aff:fe88:8081/64 scope link 2023-11-06T03:04:34.481 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.481 INFO:teuthology.orchestra.run.smithi195.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:04:34.481 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 02:42:f3:d0:44:6b brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:04:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:04:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.483 INFO:teuthology.orchestra.run.smithi195.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:04:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T03:04:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:04:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.484 DEBUG:teuthology.orchestra.run.smithi195:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:04:34.531 INFO:teuthology.orchestra.run.smithi195.stdout:1 2023-11-06T03:04:34.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.533 DEBUG:teuthology.orchestra.run.smithi195:> route 2023-11-06T03:04:34.589 INFO:teuthology.orchestra.run.smithi195.stdout:Kernel IP routing table 2023-11-06T03:04:34.589 INFO:teuthology.orchestra.run.smithi195.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:04:34.590 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:04:34.590 INFO:teuthology.orchestra.run.smithi195.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:04:34.590 INFO:teuthology.orchestra.run.smithi195.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:04:34.590 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:04:34.590 INFO:teuthology.orchestra.run.smithi195.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:34.591 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:34.591 INFO:teuthology.orchestra.run.smithi195.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:04:34.591 INFO:teuthology.orchestra.run.smithi195.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:04:34.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.592 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:34.592 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:34.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:04:34.593 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:04:34.593 DEBUG:teuthology.orchestra.run.smithi195:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:04:34.593 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:34.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.666 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2023-11-06T03:04:34.712 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T03:04:34.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.713 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list-id 2023-11-06T03:04:34.760 INFO:teuthology.orchestra.run.smithi195.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-06T03:04:34.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.761 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:34.761 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:34.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T03:04:34.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-06T03:04:34.762 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:34.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.829 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T03:04:34.882 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:34.883 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:34.883 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:34.883 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.883 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether fa:77:fa:0f:8e:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.884 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::f877:faff:fe0f:8e71/64 scope link 2023-11-06T03:04:34.885 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-06T03:04:34.944 INFO:teuthology.orchestra.run.smithi195.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:04:34.945 INFO:teuthology.orchestra.run.smithi195.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:04:34.945 INFO:teuthology.orchestra.run.smithi195.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:04:34.945 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.945 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 ::1/128 scope host 2023-11-06T03:04:34.945 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.946 INFO:teuthology.orchestra.run.smithi195.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:04:34.946 INFO:teuthology.orchestra.run.smithi195.stdout: link/ether fa:77:fa:0f:8e:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-06T03:04:34.946 INFO:teuthology.orchestra.run.smithi195.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-06T03:04:34.946 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.946 INFO:teuthology.orchestra.run.smithi195.stdout: inet6 fe80::f877:faff:fe0f:8e71/64 scope link 2023-11-06T03:04:34.947 INFO:teuthology.orchestra.run.smithi195.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:04:34.948 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-06T03:04:34.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:34.948 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:34.949 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:34.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-06T03:04:34.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-06T03:04:34.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-06T03:04:34.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-06T03:04:34.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-06T03:04:34.950 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:35.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:35.046 DEBUG:teuthology.orchestra.run.smithi195:> (cd / && exec bash -c ' 2023-11-06T03:04:35.047 DEBUG:teuthology.orchestra.run.smithi195:> set -e 2023-11-06T03:04:35.047 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set brx.1 up 2023-11-06T03:04:35.047 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link set dev brx.1 master ceph-brx 2023-11-06T03:04:35.047 DEBUG:teuthology.orchestra.run.smithi195:> ') 2023-11-06T03:04:35.069 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-06T03:04:35.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:35.070 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:35.116 INFO:teuthology.orchestra.run.smithi195.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-06T03:04:35.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:04:35.116 DEBUG:teuthology.orchestra.run.smithi195:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:35.163 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:35.221 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:35.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:35.275 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:35.278 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:35.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:35.279 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:35.328 INFO:teuthology.orchestra.run.smithi195.stdout:45 2023-11-06T03:04:35.329 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-11-06T03:04:35.329 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-06T03:04:35.373 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:35.388 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:35.388 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:35.446 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:35.447 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:35.448 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:35.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:2023-11-06T03:04:35.457+0000 7f6227dce580 -1 init, newargv = 0x55738934e0d0 newargc=15 2023-11-06T03:04:35.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:ceph-fuse[44249]: starting ceph client 2023-11-06T03:04:35.462 INFO:teuthology.orchestra.run.smithi195.stdout:45 2023-11-06T03:04:35.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi195.stderr:ceph-fuse[44249]: starting fuse 2023-11-06T03:04:36.564 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2023-11-06T03:04:36.581 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:04:36.581 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:04:36.636 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:04:36.637 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:04:36.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:36.637 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2023-11-06T03:04:36.684 INFO:teuthology.orchestra.run.smithi195.stdout:45 2023-11-06T03:04:36.685 INFO:teuthology.orchestra.run.smithi195.stdout:46 2023-11-06T03:04:36.685 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-11-06T03:04:36.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:04:36.686 DEBUG:teuthology.orchestra.run.smithi195:> import glob 2023-11-06T03:04:36.686 DEBUG:teuthology.orchestra.run.smithi195:> import re 2023-11-06T03:04:36.686 DEBUG:teuthology.orchestra.run.smithi195:> import os 2023-11-06T03:04:36.687 DEBUG:teuthology.orchestra.run.smithi195:> import subprocess 2023-11-06T03:04:36.687 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:36.687 DEBUG:teuthology.orchestra.run.smithi195:> def _find_admin_socket(client_name): 2023-11-06T03:04:36.688 DEBUG:teuthology.orchestra.run.smithi195:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-06T03:04:36.688 DEBUG:teuthology.orchestra.run.smithi195:> files = glob.glob(asok_path) 2023-11-06T03:04:36.688 DEBUG:teuthology.orchestra.run.smithi195:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-06T03:04:36.689 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:36.689 DEBUG:teuthology.orchestra.run.smithi195:> # Given a non-glob path, it better be there 2023-11-06T03:04:36.689 DEBUG:teuthology.orchestra.run.smithi195:> if "*" not in asok_path: 2023-11-06T03:04:36.689 DEBUG:teuthology.orchestra.run.smithi195:> assert(len(files) == 1) 2023-11-06T03:04:36.690 DEBUG:teuthology.orchestra.run.smithi195:> return files[0] 2023-11-06T03:04:36.690 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:36.690 DEBUG:teuthology.orchestra.run.smithi195:> for f in files: 2023-11-06T03:04:36.691 DEBUG:teuthology.orchestra.run.smithi195:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:04:36.691 DEBUG:teuthology.orchestra.run.smithi195:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:04:36.691 DEBUG:teuthology.orchestra.run.smithi195:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:04:36.692 DEBUG:teuthology.orchestra.run.smithi195:> contents = proc_f.read() 2023-11-06T03:04:36.692 DEBUG:teuthology.orchestra.run.smithi195:> if mountpoint in contents: 2023-11-06T03:04:36.692 DEBUG:teuthology.orchestra.run.smithi195:> return f 2023-11-06T03:04:36.692 DEBUG:teuthology.orchestra.run.smithi195:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:04:36.693 DEBUG:teuthology.orchestra.run.smithi195:> 2023-11-06T03:04:36.693 DEBUG:teuthology.orchestra.run.smithi195:> print(_find_admin_socket("client.3")) 2023-11-06T03:04:36.694 DEBUG:teuthology.orchestra.run.smithi195:> ' 2023-11-06T03:04:36.808 INFO:teuthology.orchestra.run.smithi195.stdout:/var/run/ceph/ceph-client.3.44249.asok 2023-11-06T03:04:36.991 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44249.asok 2023-11-06T03:04:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:36.992 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44249.asok status 2023-11-06T03:04:37.152 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-06T03:04:37.152 INFO:teuthology.orchestra.run.smithi195.stdout: "metadata": { 2023-11-06T03:04:37.152 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:04:37.153 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:04:37.153 INFO:teuthology.orchestra.run.smithi195.stdout: "entity_id": "3", 2023-11-06T03:04:37.153 INFO:teuthology.orchestra.run.smithi195.stdout: "hostname": "smithi195", 2023-11-06T03:04:37.153 INFO:teuthology.orchestra.run.smithi195.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: "pid": "44249", 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: "root": "/" 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_count": 0, 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_pinned_count": 0, 2023-11-06T03:04:37.154 INFO:teuthology.orchestra.run.smithi195.stdout: "id": 4508, 2023-11-06T03:04:37.155 INFO:teuthology.orchestra.run.smithi195.stdout: "inst": { 2023-11-06T03:04:37.155 INFO:teuthology.orchestra.run.smithi195.stdout: "name": { 2023-11-06T03:04:37.155 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "client", 2023-11-06T03:04:37.155 INFO:teuthology.orchestra.run.smithi195.stdout: "num": 4508 2023-11-06T03:04:37.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:37.156 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2023-11-06T03:04:37.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2023-11-06T03:04:37.156 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.2:0", 2023-11-06T03:04:37.156 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 2635795795 2023-11-06T03:04:37.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-06T03:04:37.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:37.157 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2023-11-06T03:04:37.157 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2023-11-06T03:04:37.157 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "192.168.0.2:0", 2023-11-06T03:04:37.157 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 2635795795 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: "inst_str": "client.4508 192.168.0.2:0/2635795795", 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: "addr_str": "192.168.0.2:0/2635795795", 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: "inode_count": 1, 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_epoch": 7, 2023-11-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch": 23, 2023-11-06T03:04:37.159 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:04:37.159 INFO:teuthology.orchestra.run.smithi195.stdout: "blocklisted": false, 2023-11-06T03:04:37.159 INFO:teuthology.orchestra.run.smithi195.stdout: "fs_name": "cephfs" 2023-11-06T03:04:37.159 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-06T03:04:37.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.166 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:04:37.166 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:37.175 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:04:37.175 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:37.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:37.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.240 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:04:37.240 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:37.287 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:04:37.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:37.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.288 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:37.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.350 DEBUG:teuthology.orchestra.run.smithi195:> stat --file-system '--printf=%T 2023-11-06T03:04:37.350 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:37.358 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2023-11-06T03:04:37.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:37.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.359 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:37.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.413 DEBUG:teuthology.orchestra.run.smithi195:> stat --file-system '--printf=%T 2023-11-06T03:04:37.414 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:37.467 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2023-11-06T03:04:37.467 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:37.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:37.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-06T03:04:37.529 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-06T03:04:37.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:37.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 Processor -- start 2023-11-06T03:04:37.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 -- start start 2023-11-06T03:04:37.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 0x7f763806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763815f580 con 0x7f763815ea60 2023-11-06T03:04:37.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763815efc0 con 0x7f763815e5e0 2023-11-06T03:04:37.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f76401f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763815f2a0 con 0x7f76381551e0 2023-11-06T03:04:37.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763d78e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763df8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 0x7f763806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763e790640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f763815ea60 0x7f763815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:49840/0 (socket says 172.21.15.157:49840) 2023-11-06T03:04:37.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763d78e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:59322/0 (socket says 172.21.15.157:59322) 2023-11-06T03:04:37.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763e790640 1 -- 172.21.15.157:0/3733537816 learned_addr learned my addr 172.21.15.157:0/3733537816 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:37.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763df8f640 1 -- 172.21.15.157:0/3733537816 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 msgr2=0x7f76380b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:37.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763df8f640 1 --2- 172.21.15.157:0/3733537816 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763df8f640 1 -- 172.21.15.157:0/3733537816 >> v1:172.21.15.195:6789/0 conn(0x7f763815ea60 legacy=0x7f763815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763df8f640 1 -- 172.21.15.157:0/3733537816 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763806f5a0 con 0x7f76381551e0 2023-11-06T03:04:37.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.681+0000 7f763d78e640 1 --2- 172.21.15.157:0/3733537816 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:37.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f763df8f640 1 --2- 172.21.15.157:0/3733537816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 0x7f763806fda0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76340097c0 tx=0x7f763401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d792cb788a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f763cf8d640 1 -- 172.21.15.157:0/3733537816 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340ab600 con 0x7f76381551e0 2023-11-06T03:04:37.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f763cf8d640 1 -- 172.21.15.157:0/3733537816 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76340a7050 con 0x7f76381551e0 2023-11-06T03:04:37.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f763cf8d640 1 -- 172.21.15.157:0/3733537816 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340ab980 con 0x7f76381551e0 2023-11-06T03:04:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- 172.21.15.157:0/3733537816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 msgr2=0x7f763806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 --2- 172.21.15.157:0/3733537816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 0x7f763806fda0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76340097c0 tx=0x7f763401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- 172.21.15.157:0/3733537816 shutdown_connections 2023-11-06T03:04:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 --2- 172.21.15.157:0/3733537816 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 --2- 172.21.15.157:0/3733537816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76381551e0 0x7f763806fda0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- 172.21.15.157:0/3733537816 >> 172.21.15.157:0/3733537816 conn(0x7f7638065930 msgr2=0x7f7638065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:37.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- 172.21.15.157:0/3733537816 shutdown_connections 2023-11-06T03:04:37.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- 172.21.15.157:0/3733537816 wait complete. 2023-11-06T03:04:37.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 Processor -- start 2023-11-06T03:04:37.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.685+0000 7f76401f3640 1 -- start start 2023-11-06T03:04:37.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76381551e0 0x7f7638152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 0x7f763806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638153200 con 0x7f763815e5e0 2023-11-06T03:04:37.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638151c60 con 0x7f76381551e0 2023-11-06T03:04:37.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7638151f00 con 0x7f763815ea60 2023-11-06T03:04:37.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51358/0 (socket says 172.21.15.157:51358) 2023-11-06T03:04:37.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 -- 172.21.15.157:0/2403473836 learned_addr learned my addr 172.21.15.157:0/2403473836 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:37.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763df8f640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76381551e0 0x7f7638152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763e790640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 0x7f763806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 -- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 msgr2=0x7f763806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 0x7f763806bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 -- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76381551e0 msgr2=0x7f7638152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76381551e0 0x7f7638152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 -- 172.21.15.157:0/2403473836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7628009420 con 0x7f763815e5e0 2023-11-06T03:04:37.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763e790640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 0x7f763806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:37.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763d78e640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f762800ebe0 tx=0x7f762800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762800eee0 con 0x7f763815e5e0 2023-11-06T03:04:37.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762800cc30 con 0x7f763815e5e0 2023-11-06T03:04:37.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7634009420 con 0x7f763815e5e0 2023-11-06T03:04:37.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628013470 con 0x7f763815e5e0 2023-11-06T03:04:37.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76381524c0 con 0x7f763815e5e0 2023-11-06T03:04:37.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638072750 con 0x7f763815e5e0 2023-11-06T03:04:37.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76280135d0 con 0x7f763815e5e0 2023-11-06T03:04:37.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 0x7f761c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7628083de0 con 0x7f763815e5e0 2023-11-06T03:04:37.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.689+0000 7f763df8f640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 0x7f761c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.693+0000 7f763df8f640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 0x7f761c067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76340ad000 tx=0x7f7634008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.693+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7628051690 con 0x7f763815e5e0 2023-11-06T03:04:37.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.849+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76380cef20 con 0x7f763815e5e0 2023-11-06T03:04:37.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f762e7fc640 1 -- 172.21.15.157:0/2403473836 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f7628017090 con 0x7f763815e5e0 2023-11-06T03:04:37.851 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:37.851 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:37.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 msgr2=0x7f761c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 0x7f761c067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f76340ad000 tx=0x7f7634008040 comp rx=0 tx=0).stop 2023-11-06T03:04:37.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 msgr2=0x7f76381458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f762800ebe0 tx=0x7f762800c520 comp rx=0 tx=0).stop 2023-11-06T03:04:37.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 shutdown_connections 2023-11-06T03:04:37.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f761c065a60 0x7f761c067f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f763815ea60 0x7f763806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763815e5e0 0x7f76381458f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 --2- 172.21.15.157:0/2403473836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76381551e0 0x7f7638152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 >> 172.21.15.157:0/2403473836 conn(0x7f7638065930 msgr2=0x7f7638153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:37.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 shutdown_connections 2023-11-06T03:04:37.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.853+0000 7f76401f3640 1 -- 172.21.15.157:0/2403473836 wait complete. 2023-11-06T03:04:37.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:37.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 Processor -- start 2023-11-06T03:04:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 -- start start 2023-11-06T03:04:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb8815e5e0 0x7ffb880b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8815f580 con 0x7ffb881551e0 2023-11-06T03:04:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8815efc0 con 0x7ffb8815ea90 2023-11-06T03:04:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8fe5e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb880704b0 con 0x7ffb8815e5e0 2023-11-06T03:04:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8d3f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb8815e5e0 0x7ffb880b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8dbfa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffb881551e0 0x7ffb8806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49622/0 (socket says 172.21.15.157:49622) 2023-11-06T03:04:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8dbfa640 1 -- 172.21.15.157:0/2385564240 learned_addr learned my addr 172.21.15.157:0/2385564240 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8d3f9640 1 -- 172.21.15.157:0/2385564240 >> v1:172.21.15.195:6789/0 conn(0x7ffb8815ea90 legacy=0x7ffb8815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8d3f9640 1 -- 172.21.15.157:0/2385564240 >> v1:172.21.15.157:6789/0 conn(0x7ffb881551e0 legacy=0x7ffb8806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.985+0000 7ffb8d3f9640 1 -- 172.21.15.157:0/2385564240 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8815f2a0 con 0x7ffb8815e5e0 2023-11-06T03:04:37.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8d3f9640 1 --2- 172.21.15.157:0/2385564240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815e5e0 0x7ffb880b3410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb780097c0 tx=0x7ffb7801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ada6888a6e3f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8cbf8640 1 -- 172.21.15.157:0/2385564240 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb780ab600 con 0x7ffb8815e5e0 2023-11-06T03:04:37.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8cbf8640 1 -- 172.21.15.157:0/2385564240 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780a7050 con 0x7ffb8815e5e0 2023-11-06T03:04:37.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8cbf8640 1 -- 172.21.15.157:0/2385564240 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb780ab980 con 0x7ffb8815e5e0 2023-11-06T03:04:37.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/2385564240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815e5e0 msgr2=0x7ffb880b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/2385564240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815e5e0 0x7ffb880b3410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb780097c0 tx=0x7ffb7801c380 comp rx=0 tx=0).stop 2023-11-06T03:04:37.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/2385564240 shutdown_connections 2023-11-06T03:04:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/2385564240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815e5e0 0x7ffb880b3410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/2385564240 >> 172.21.15.157:0/2385564240 conn(0x7ffb88065930 msgr2=0x7ffb88065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/2385564240 shutdown_connections 2023-11-06T03:04:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/2385564240 wait complete. 2023-11-06T03:04:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 Processor -- start 2023-11-06T03:04:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- start start 2023-11-06T03:04:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb881551e0 0x7ffb88147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 0x7ffb88152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8815abe0 con 0x7ffb8815ea90 2023-11-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88147870 con 0x7ffb8815e5e0 2023-11-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8815a5b0 con 0x7ffb881551e0 2023-11-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51368/0 (socket says 172.21.15.157:51368) 2023-11-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 -- 172.21.15.157:0/4064998261 learned_addr learned my addr 172.21.15.157:0/4064998261 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8d3f9640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 0x7ffb88152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8dbfa640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb881551e0 0x7ffb88147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 -- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb881551e0 msgr2=0x7ffb88147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb881551e0 0x7ffb88147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 -- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 msgr2=0x7ffb88152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 0x7ffb88152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 -- 172.21.15.157:0/4064998261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009420 con 0x7ffb8815ea90 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8d3f9640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 0x7ffb88152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8e3fb640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b970 tx=0x7ffb8000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000cb40 con 0x7ffb8815ea90 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8000cca0 con 0x7ffb8815ea90 2023-11-06T03:04:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.989+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb880b1d10 con 0x7ffb8815ea90 2023-11-06T03:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80014680 con 0x7ffb8815ea90 2023-11-06T03:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8815a800 con 0x7ffb8815ea90 2023-11-06T03:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb88072750 con 0x7ffb8815ea90 2023-11-06T03:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffb8001d3b0 con 0x7ffb8815ea90 2023-11-06T03:04:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb7e7fc640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 0x7ffb6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb800831b0 con 0x7ffb8815ea90 2023-11-06T03:04:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb8dbfa640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 0x7ffb6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.993+0000 7ffb8dbfa640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 0x7ffb6c067f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005890 tx=0x7ffb84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:37.997+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffb80050aa0 con 0x7ffb8815ea90 2023-11-06T03:04:38.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb88071730 con 0x7ffb8815ea90 2023-11-06T03:04:38.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb7e7fc640 1 -- 172.21.15.157:0/4064998261 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb800540c0 con 0x7ffb8815ea90 2023-11-06T03:04:38.145 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:38.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 msgr2=0x7ffb6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 0x7ffb6c067f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005890 tx=0x7ffb84009040 comp rx=0 tx=0).stop 2023-11-06T03:04:38.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 msgr2=0x7ffb88152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.145+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b970 tx=0x7ffb8000be30 comp rx=0 tx=0).stop 2023-11-06T03:04:38.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 shutdown_connections 2023-11-06T03:04:38.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb6c065a60 0x7ffb6c067f20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb8815ea90 0x7ffb88152a40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb8815e5e0 0x7ffb88152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 --2- 172.21.15.157:0/4064998261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb881551e0 0x7ffb88147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 >> 172.21.15.157:0/4064998261 conn(0x7ffb88065930 msgr2=0x7ffb880af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 shutdown_connections 2023-11-06T03:04:38.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.149+0000 7ffb8fe5e640 1 -- 172.21.15.157:0/4064998261 wait complete. 2023-11-06T03:04:38.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.273+0000 7f0d222ee640 1 Processor -- start 2023-11-06T03:04:38.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.273+0000 7f0d222ee640 1 -- start start 2023-11-06T03:04:38.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d222ee640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0d1c1599c0 0x7f0d1c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d222ee640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c15e440 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d222ee640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c06c240 con 0x7f0d1c06d000 2023-11-06T03:04:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d222ee640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c06c520 con 0x7f0d1c1599c0 2023-11-06T03:04:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d1c0b2af0 0x7f0d1c159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49636/0 (socket says 172.21.15.157:49636) 2023-11-06T03:04:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 -- 172.21.15.157:0/271532534 learned_addr learned my addr 172.21.15.157:0/271532534 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657118122 0 0) 0x7f0d1c15e440 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d04003530 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/271532534 >> v2:172.21.15.195:3300/0 conn(0x7f0d1c1599c0 0x7f0d1c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3141315066 0 0) 0x7f0d04003530 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 >> v2:172.21.15.195:3300/0 conn(0x7f0d1c1599c0 msgr2=0x7f0d1c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 --2- 172.21.15.157:0/271532534 >> v2:172.21.15.195:3300/0 conn(0x7f0d1c1599c0 0x7f0d1c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 >> v1:172.21.15.195:6789/0 conn(0x7f0d1c06d000 legacy=0x7f0d1c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c06c800 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0d14001a00 con 0x7f0d1c0b2af0 2023-11-06T03:04:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 >> v1:172.21.15.157:6789/0 conn(0x7f0d1c0b2af0 legacy=0x7f0d1c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d04003980 0x7f0d04003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d040047d0 0x7f0d04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 0x7f0d0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c06c520 con 0x7f0d04005190 2023-11-06T03:04:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0400c0e0 con 0x7f0d04003980 2023-11-06T03:04:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0400c2a0 con 0x7f0d040047d0 2023-11-06T03:04:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 0x7f0d0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d04003980 0x7f0d04003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 -- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d040047d0 msgr2=0x7f0d04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d040047d0 0x7f0d04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 -- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d04003980 msgr2=0x7f0d04003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d04003980 0x7f0d04003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 -- 172.21.15.157:0/271532534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04003530 con 0x7f0d04005190 2023-11-06T03:04:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d04003980 0x7f0d04003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:38.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 -- 172.21.15.157:0/271532534 reap_dead start 2023-11-06T03:04:38.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d2088b640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 0x7f0d0400b990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400a800 tx=0x7f0d1401d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9270cde58dfa5698 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140072d0 con 0x7f0d04005190 2023-11-06T03:04:38.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d140a8070 con 0x7f0d04005190 2023-11-06T03:04:38.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.277+0000 7f0d1affd640 1 -- 172.21.15.157:0/271532534 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14007670 con 0x7f0d04005190 2023-11-06T03:04:38.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 msgr2=0x7f0d0400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 0x7f0d0400b990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400a800 tx=0x7f0d1401d300 comp rx=0 tx=0).stop 2023-11-06T03:04:38.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/271532534 shutdown_connections 2023-11-06T03:04:38.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 --2- 172.21.15.157:0/271532534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d04005190 0x7f0d0400b990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/271532534 >> 172.21.15.157:0/271532534 conn(0x7f0d1c065930 msgr2=0x7f0d1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/271532534 shutdown_connections 2023-11-06T03:04:38.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/271532534 wait complete. 2023-11-06T03:04:38.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 Processor -- start 2023-11-06T03:04:38.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- start start 2023-11-06T03:04:38.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04005190 0x7f0d1c1efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d1c0cdf20 0x7f0d1c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c145e90 con 0x7f0d1c156cc0 2023-11-06T03:04:38.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0ce3d0 con 0x7f0d1c0cdf20 2023-11-06T03:04:38.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c143000 con 0x7f0d04005190 2023-11-06T03:04:38.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51376/0 (socket says 172.21.15.157:51376) 2023-11-06T03:04:38.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 -- 172.21.15.157:0/2231355596 learned_addr learned my addr 172.21.15.157:0/2231355596 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:38.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d2088b640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d1c0cdf20 0x7f0d1c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04005190 0x7f0d1c1efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 -- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04005190 msgr2=0x7f0d1c1efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04005190 0x7f0d1c1efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 -- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d1c0cdf20 msgr2=0x7f0d1c145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d1c0cdf20 0x7f0d1c145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 -- 172.21.15.157:0/2231355596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c1432e0 con 0x7f0d1c156cc0 2023-11-06T03:04:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d1b7fe640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c3c0 tx=0x7f0d1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100052d0 con 0x7f0d1c156cc0 2023-11-06T03:04:38.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10005450 con 0x7f0d1c156cc0 2023-11-06T03:04:38.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c1435c0 con 0x7f0d1c156cc0 2023-11-06T03:04:38.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10003840 con 0x7f0d1c156cc0 2023-11-06T03:04:38.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c145660 con 0x7f0d1c156cc0 2023-11-06T03:04:38.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.281+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c072750 con 0x7f0d1c156cc0 2023-11-06T03:04:38.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.285+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0d100039a0 con 0x7f0d1c156cc0 2023-11-06T03:04:38.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.285+0000 7f0d18ff9640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 0x7f0d08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.285+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d10082760 con 0x7f0d1c156cc0 2023-11-06T03:04:38.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.285+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 0x7f0d08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.285+0000 7f0d1bfff640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 0x7f0d08067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c890 tx=0x7f0d0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.289+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d1004ffc0 con 0x7f0d1c156cc0 2023-11-06T03:04:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d1c0cb110 con 0x7f0d1c156cc0 2023-11-06T03:04:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d18ff9640 1 -- 172.21.15.157:0/2231355596 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d10001070 con 0x7f0d1c156cc0 2023-11-06T03:04:38.432 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:38.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 msgr2=0x7f0d08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 0x7f0d08067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c890 tx=0x7f0d0c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 msgr2=0x7f0d1c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c3c0 tx=0x7f0d1000c880 comp rx=0 tx=0).stop 2023-11-06T03:04:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 shutdown_connections 2023-11-06T03:04:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d08065a60 0x7f0d08067f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d1c0cdf20 0x7f0d1c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d1c156cc0 0x7f0d1c0cd9e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 --2- 172.21.15.157:0/2231355596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04005190 0x7f0d1c1efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 >> 172.21.15.157:0/2231355596 conn(0x7f0d1c065930 msgr2=0x7f0d1c0ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 shutdown_connections 2023-11-06T03:04:38.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.433+0000 7f0d222ee640 1 -- 172.21.15.157:0/2231355596 wait complete. 2023-11-06T03:04:38.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:38.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.573+0000 7feef4ca0640 1 Processor -- start 2023-11-06T03:04:38.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.573+0000 7feef4ca0640 1 -- start start 2023-11-06T03:04:38.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feef4ca0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feef00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feef4ca0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feef006d000 0x7feef00b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feef4ca0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef015e440 con 0x7feef015dea0 2023-11-06T03:04:38.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feef4ca0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef015adb0 con 0x7feef006d000 2023-11-06T03:04:38.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feef4ca0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef006c240 con 0x7feef01599c0 2023-11-06T03:04:38.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeeed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7feef015dea0 0x7feef0070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49734/0 (socket says 172.21.15.157:49734) 2023-11-06T03:04:38.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeeed9d640 1 -- 172.21.15.157:0/448038787 learned_addr learned my addr 172.21.15.157:0/448038787 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:38.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeee59c640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feef00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 --2- 172.21.15.157:0/448038787 >> v2:172.21.15.195:3300/0 conn(0x7feef006d000 0x7feef00b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578386322 0 0) 0x7feef015e440 con 0x7feef015dea0 2023-11-06T03:04:38.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed8003530 con 0x7feef015dea0 2023-11-06T03:04:38.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162998144 0 0) 0x7feed8003530 con 0x7feef015dea0 2023-11-06T03:04:38.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 msgr2=0x7feef00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feef00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 >> v2:172.21.15.195:3300/0 conn(0x7feef006d000 msgr2=0x7feef00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 --2- 172.21.15.157:0/448038787 >> v2:172.21.15.195:3300/0 conn(0x7feef006d000 0x7feef00b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef006c520 con 0x7feef015dea0 2023-11-06T03:04:38.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7feee8001a00 con 0x7feef015dea0 2023-11-06T03:04:38.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 >> v1:172.21.15.157:6789/0 conn(0x7feef015dea0 legacy=0x7feef0070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feed8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed8003f00 0x7feed80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8004d10 0x7feed8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef015adb0 con 0x7feed8003f00 2023-11-06T03:04:38.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef006c240 con 0x7feed8004d10 2023-11-06T03:04:38.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed800b990 con 0x7feef01599c0 2023-11-06T03:04:38.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed8003f00 0x7feed80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeee59c640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feed8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeeed9d640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8004d10 0x7feed8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 -- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 msgr2=0x7feed8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef01599c0 0x7feed8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 -- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8004d10 msgr2=0x7feed8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed8004d10 0x7feed8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.577+0000 7feeedd9b640 1 -- 172.21.15.157:0/448038787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8003530 con 0x7feed8003f00 2023-11-06T03:04:38.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feeedd9b640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed8003f00 0x7feed80047d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feee4009ae0 tx=0x7feee401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae61df09fd6f56f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40a5570 con 0x7feed8003f00 2023-11-06T03:04:38.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee40a56d0 con 0x7feed8003f00 2023-11-06T03:04:38.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feeed59a640 1 -- 172.21.15.157:0/448038787 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40a5a80 con 0x7feed8003f00 2023-11-06T03:04:38.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- 172.21.15.157:0/448038787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed8003f00 msgr2=0x7feed80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 --2- 172.21.15.157:0/448038787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed8003f00 0x7feed80047d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feee4009ae0 tx=0x7feee401ca30 comp rx=0 tx=0).stop 2023-11-06T03:04:38.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feeeed9d640 1 -- 172.21.15.157:0/448038787 reap_dead start 2023-11-06T03:04:38.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- 172.21.15.157:0/448038787 shutdown_connections 2023-11-06T03:04:38.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- 172.21.15.157:0/448038787 >> 172.21.15.157:0/448038787 conn(0x7feef0065930 msgr2=0x7feef0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- 172.21.15.157:0/448038787 shutdown_connections 2023-11-06T03:04:38.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- 172.21.15.157:0/448038787 wait complete. 2023-11-06T03:04:38.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 Processor -- start 2023-11-06T03:04:38.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 -- start start 2023-11-06T03:04:38.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 0x7feef00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.581+0000 7feef4ca0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef00acdf0 0x7feef014bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00cdc80 con 0x7feef00ce500 2023-11-06T03:04:38.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ce980 con 0x7feef014b590 2023-11-06T03:04:38.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ad270 con 0x7feef00acdf0 2023-11-06T03:04:38.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33216/0 (socket says 172.21.15.157:33216) 2023-11-06T03:04:38.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 -- 172.21.15.157:0/2687364352 learned_addr learned my addr 172.21.15.157:0/2687364352 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:38.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeee59c640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 0x7feef00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeeed9d640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef00acdf0 0x7feef014bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 -- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef00acdf0 msgr2=0x7feef014bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef00acdf0 0x7feef014bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 -- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 msgr2=0x7feef00cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 0x7feef00cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 -- 172.21.15.157:0/2687364352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00cd000 con 0x7feef00ce500 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeee59c640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 0x7feef00cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeedd9b640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feee401b140 tx=0x7feee4019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4004020 con 0x7feef00ce500 2023-11-06T03:04:38.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee40041c0 con 0x7feef00ce500 2023-11-06T03:04:38.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4003620 con 0x7feef00ce500 2023-11-06T03:04:38.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef00cd2e0 con 0x7feef00ce500 2023-11-06T03:04:38.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef00cd8a0 con 0x7feef00ce500 2023-11-06T03:04:38.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef0072750 con 0x7feef00ce500 2023-11-06T03:04:38.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feee40b4070 con 0x7feef00ce500 2023-11-06T03:04:38.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 0x7feed0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feee411d7c0 con 0x7feef00ce500 2023-11-06T03:04:38.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.585+0000 7feeee59c640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 0x7feed0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.589+0000 7feeee59c640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 0x7feed0067f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feee0009a10 tx=0x7feee0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.589+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feee40eb030 con 0x7feef00ce500 2023-11-06T03:04:38.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.745+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feef00cb110 con 0x7feef00ce500 2023-11-06T03:04:38.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.745+0000 7feedeffd640 1 -- 172.21.15.157:0/2687364352 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7feee40ee650 con 0x7feef00ce500 2023-11-06T03:04:38.746 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:38.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 msgr2=0x7feed0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 0x7feed0067f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feee0009a10 tx=0x7feee0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:38.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 msgr2=0x7feef00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feee401b140 tx=0x7feee4019830 comp rx=0 tx=0).stop 2023-11-06T03:04:38.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 shutdown_connections 2023-11-06T03:04:38.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feed0065a90 0x7feed0067f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feef00acdf0 0x7feef014bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feef00ce500 0x7feef00ac8b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 --2- 172.21.15.157:0/2687364352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef014b590 0x7feef00cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 >> 172.21.15.157:0/2687364352 conn(0x7feef0065930 msgr2=0x7feef00b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 shutdown_connections 2023-11-06T03:04:38.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.749+0000 7feef4ca0640 1 -- 172.21.15.157:0/2687364352 wait complete. 2023-11-06T03:04:38.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:38.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 Processor -- start 2023-11-06T03:04:38.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 -- start start 2023-11-06T03:04:38.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2e0073080 0x7fb2e006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb2e006cf20 0x7fb2e00707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e015a910 con 0x7fb2e0159ac0 2023-11-06T03:04:38.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e006d390 con 0x7fb2e0073080 2023-11-06T03:04:38.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2e48b7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e015abf0 con 0x7fb2e006cf20 2023-11-06T03:04:38.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2e0073080 0x7fb2e006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2e0073080 0x7fb2e006c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33226/0 (socket says 172.21.15.157:33226) 2023-11-06T03:04:38.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 -- 172.21.15.157:0/3482040908 learned_addr learned my addr 172.21.15.157:0/3482040908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:38.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2de59c640 1 --1- 172.21.15.157:0/3482040908 >> v1:172.21.15.157:6789/0 conn(0x7fb2e0159ac0 0x7fb2e0159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49742/0 (socket says 172.21.15.157:49742) 2023-11-06T03:04:38.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ded9d640 1 --2- 172.21.15.157:0/3482040908 >> v2:172.21.15.195:3300/0 conn(0x7fb2e006cf20 0x7fb2e00707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2dd59a640 1 -- 172.21.15.157:0/3482040908 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094501158 0 0) 0x7fb2e015a910 con 0x7fb2e0159ac0 2023-11-06T03:04:38.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 -- 172.21.15.157:0/3482040908 >> v2:172.21.15.195:3300/0 conn(0x7fb2e006cf20 msgr2=0x7fb2e00707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:38.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 --2- 172.21.15.157:0/3482040908 >> v2:172.21.15.195:3300/0 conn(0x7fb2e006cf20 0x7fb2e00707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 -- 172.21.15.157:0/3482040908 >> v1:172.21.15.157:6789/0 conn(0x7fb2e0159ac0 legacy=0x7fb2e0159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 -- 172.21.15.157:0/3482040908 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e015aed0 con 0x7fb2e0073080 2023-11-06T03:04:38.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2ddd9b640 1 --2- 172.21.15.157:0/3482040908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e0073080 0x7fb2e006c9e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ca9a0 tx=0x7fb2d401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ba7fc8a577e0d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2dd59a640 1 -- 172.21.15.157:0/3482040908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40ab640 con 0x7fb2e0073080 2023-11-06T03:04:38.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.889+0000 7fb2dd59a640 1 -- 172.21.15.157:0/3482040908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d40a7070 con 0x7fb2e0073080 2023-11-06T03:04:38.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2dd59a640 1 -- 172.21.15.157:0/3482040908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40ab9c0 con 0x7fb2e0073080 2023-11-06T03:04:38.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3482040908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e0073080 msgr2=0x7fb2e006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3482040908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e0073080 0x7fb2e006c9e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ca9a0 tx=0x7fb2d401c440 comp rx=0 tx=0).stop 2023-11-06T03:04:38.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3482040908 shutdown_connections 2023-11-06T03:04:38.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3482040908 >> v2:172.21.15.195:3300/0 conn(0x7fb2e006cf20 0x7fb2e00707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3482040908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e0073080 0x7fb2e006c9e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3482040908 >> 172.21.15.157:0/3482040908 conn(0x7fb2e00cb990 msgr2=0x7fb2e00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:38.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3482040908 shutdown_connections 2023-11-06T03:04:38.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3482040908 wait complete. 2023-11-06T03:04:38.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 Processor -- start 2023-11-06T03:04:38.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- start start 2023-11-06T03:04:38.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2e0073080 0x7fb2e01f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2e0159ac0 0x7fb2e0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e01f6100 con 0x7fb2e006cf20 2023-11-06T03:04:38.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e00ad140 con 0x7fb2e0159ac0 2023-11-06T03:04:38.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e00ad420 con 0x7fb2e0073080 2023-11-06T03:04:38.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33238/0 (socket says 172.21.15.157:33238) 2023-11-06T03:04:38.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 -- 172.21.15.157:0/3499481018 learned_addr learned my addr 172.21.15.157:0/3499481018 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:38.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 -- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2e0073080 msgr2=0x7fb2e01f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2e0073080 0x7fb2e01f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 -- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2e0159ac0 msgr2=0x7fb2e0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:38.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2e0159ac0 0x7fb2e0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:38.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 -- 172.21.15.157:0/3499481018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d40094d0 con 0x7fb2e006cf20 2023-11-06T03:04:38.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2de59c640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c480 tx=0x7fb2d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00074e0 con 0x7fb2e006cf20 2023-11-06T03:04:38.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0007660 con 0x7fb2e006cf20 2023-11-06T03:04:38.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004a30 con 0x7fb2e006cf20 2023-11-06T03:04:38.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e00ad700 con 0x7fb2e006cf20 2023-11-06T03:04:38.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e01f96c0 con 0x7fb2e006cf20 2023-11-06T03:04:38.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.893+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e00adc20 con 0x7fb2e006cf20 2023-11-06T03:04:38.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.897+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2d0004b90 con 0x7fb2e006cf20 2023-11-06T03:04:38.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.897+0000 7fb2ceffd640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 0x7fb2c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:38.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.897+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2d00824b0 con 0x7fb2e006cf20 2023-11-06T03:04:38.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.897+0000 7fb2ddd9b640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 0x7fb2c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:38.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.897+0000 7fb2ddd9b640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 0x7fb2c0067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40af000 tx=0x7fb2d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:38.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:38.901+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb2e00adc20 con 0x7fb2e006cf20 2023-11-06T03:04:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.037+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2e00adc20 con 0x7fb2e006cf20 2023-11-06T03:04:39.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2ceffd640 1 -- 172.21.15.157:0/3499481018 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2e00adc20 con 0x7fb2e006cf20 2023-11-06T03:04:39.040 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:39.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 msgr2=0x7fb2c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 0x7fb2c0067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40af000 tx=0x7fb2d4008040 comp rx=0 tx=0).stop 2023-11-06T03:04:39.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 msgr2=0x7fb2e00adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c480 tx=0x7fb2d000c940 comp rx=0 tx=0).stop 2023-11-06T03:04:39.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 shutdown_connections 2023-11-06T03:04:39.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2c0065a60 0x7fb2c0067f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2e0159ac0 0x7fb2e0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2e0073080 0x7fb2e01f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 --2- 172.21.15.157:0/3499481018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2e006cf20 0x7fb2e00adf00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 >> 172.21.15.157:0/3499481018 conn(0x7fb2e00cb990 msgr2=0x7fb2e00cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 shutdown_connections 2023-11-06T03:04:39.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.041+0000 7fb2e48b7640 1 -- 172.21.15.157:0/3499481018 wait complete. 2023-11-06T03:04:39.056 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:39.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 Processor -- start 2023-11-06T03:04:39.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 -- start start 2023-11-06T03:04:39.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b0415caa0 0x7f8b040cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b0415ce80 0x7f8b040cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b040d2920 con 0x7f8b040ce1a0 2023-11-06T03:04:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040d18a0 con 0x7f8b0415caa0 2023-11-06T03:04:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b09e5a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040d1b60 con 0x7f8b0415ce80 2023-11-06T03:04:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b03fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8b040ce1a0 0x7f8b040d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49744/0 (socket says 172.21.15.157:49744) 2023-11-06T03:04:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b03fff640 1 -- 172.21.15.157:0/117337983 learned_addr learned my addr 172.21.15.157:0/117337983 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b037fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8b0415caa0 0x7f8b040cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b02ffd640 1 --2- 172.21.15.157:0/117337983 >> v2:172.21.15.195:3300/0 conn(0x7f8b0415ce80 0x7f8b040cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b037fe640 1 -- 172.21.15.157:0/117337983 >> v2:172.21.15.195:3300/0 conn(0x7f8b0415ce80 msgr2=0x7f8b040cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b027fc640 1 -- 172.21.15.157:0/117337983 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628613501 0 0) 0x7f8b040d2920 con 0x7f8b040ce1a0 2023-11-06T03:04:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b037fe640 1 --2- 172.21.15.157:0/117337983 >> v2:172.21.15.195:3300/0 conn(0x7f8b0415ce80 0x7f8b040cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b037fe640 1 -- 172.21.15.157:0/117337983 >> v1:172.21.15.157:6789/0 conn(0x7f8b040ce1a0 legacy=0x7f8b040d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.165+0000 7f8b037fe640 1 -- 172.21.15.157:0/117337983 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040d1e40 con 0x7f8b0415caa0 2023-11-06T03:04:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b037fe640 1 --2- 172.21.15.157:0/117337983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b040cdc60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b040c6720 tx=0x7f8af401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935acc320ebeff25 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b027fc640 1 -- 172.21.15.157:0/117337983 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40ab600 con 0x7f8b0415caa0 2023-11-06T03:04:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b027fc640 1 -- 172.21.15.157:0/117337983 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af40a7050 con 0x7f8b0415caa0 2023-11-06T03:04:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b027fc640 1 -- 172.21.15.157:0/117337983 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40ab980 con 0x7f8b0415caa0 2023-11-06T03:04:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 -- 172.21.15.157:0/117337983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 msgr2=0x7f8b040cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/117337983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b040cdc60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b040c6720 tx=0x7f8af401c430 comp rx=0 tx=0).stop 2023-11-06T03:04:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 -- 172.21.15.157:0/117337983 shutdown_connections 2023-11-06T03:04:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/117337983 >> v2:172.21.15.195:3300/0 conn(0x7f8b0415ce80 0x7f8b040cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/117337983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b040cdc60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 -- 172.21.15.157:0/117337983 >> 172.21.15.157:0/117337983 conn(0x7f8b04070790 msgr2=0x7f8b04070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 -- 172.21.15.157:0/117337983 shutdown_connections 2023-11-06T03:04:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.169+0000 7f8b09e5a640 1 -- 172.21.15.157:0/117337983 wait complete. 2023-11-06T03:04:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 Processor -- start 2023-11-06T03:04:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- start start 2023-11-06T03:04:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 0x7f8b0415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b0415ce80 0x7f8b040c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415b2b0 con 0x7f8b0415caa0 2023-11-06T03:04:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040ac570 con 0x7f8b040ce1a0 2023-11-06T03:04:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040acaa0 con 0x7f8b0415ce80 2023-11-06T03:04:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33246/0 (socket says 172.21.15.157:33246) 2023-11-06T03:04:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 -- 172.21.15.157:0/2687620758 learned_addr learned my addr 172.21.15.157:0/2687620758 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b037fe640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 0x7f8b0415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b03fff640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b0415ce80 0x7f8b040c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 -- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b0415ce80 msgr2=0x7f8b040c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b0415ce80 0x7f8b040c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 -- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 msgr2=0x7f8b0415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 0x7f8b0415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 -- 172.21.15.157:0/2687620758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4009420 con 0x7f8b0415caa0 2023-11-06T03:04:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b037fe640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 0x7f8b0415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b02ffd640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d800 tx=0x7f8af800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800b6c0 con 0x7f8b0415caa0 2023-11-06T03:04:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af800b820 con 0x7f8b0415caa0 2023-11-06T03:04:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b040ac7c0 con 0x7f8b0415caa0 2023-11-06T03:04:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af800bbd0 con 0x7f8b0415caa0 2023-11-06T03:04:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b040a9920 con 0x7f8b0415caa0 2023-11-06T03:04:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04072750 con 0x7f8b0415caa0 2023-11-06T03:04:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.173+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8af800bd30 con 0x7f8b0415caa0 2023-11-06T03:04:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.177+0000 7f8aebfff640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 0x7f8af0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.177+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8af8083790 con 0x7f8b0415caa0 2023-11-06T03:04:39.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.177+0000 7f8b037fe640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 0x7f8af0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.177+0000 7f8b037fe640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 0x7f8af0067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cf670 tx=0x7f8af40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.181+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8af8051080 con 0x7f8b0415caa0 2023-11-06T03:04:39.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.321+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b040c9ab0 con 0x7f8b0415caa0 2023-11-06T03:04:39.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.325+0000 7f8aebfff640 1 -- 172.21.15.157:0/2687620758 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f8af8011340 con 0x7f8b0415caa0 2023-11-06T03:04:39.324 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:39.325 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:39.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 msgr2=0x7f8af0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 0x7f8af0067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cf670 tx=0x7f8af40a6000 comp rx=0 tx=0).stop 2023-11-06T03:04:39.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 msgr2=0x7f8b04158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d800 tx=0x7f8af800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 shutdown_connections 2023-11-06T03:04:39.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8af0065a60 0x7f8af0067f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b0415ce80 0x7f8b040c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b0415caa0 0x7f8b04158f50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 --2- 172.21.15.157:0/2687620758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b040ce1a0 0x7f8b0415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 >> 172.21.15.157:0/2687620758 conn(0x7f8b04070790 msgr2=0x7f8b04070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 shutdown_connections 2023-11-06T03:04:39.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.329+0000 7f8b09e5a640 1 -- 172.21.15.157:0/2687620758 wait complete. 2023-11-06T03:04:39.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:39.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:39.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:04:39.456 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:04:39.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 Processor -- start 2023-11-06T03:04:39.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 -- start start 2023-11-06T03:04:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea240cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 0x7fea240cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea240d0140 con 0x7fea2415e090 2023-11-06T03:04:39.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240cc420 con 0x7fea240cce90 2023-11-06T03:04:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2b3e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240d0380 con 0x7fea240719e0 2023-11-06T03:04:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2917f640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fea2415e090 0x7fea240cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56858/0 (socket says 172.21.15.157:56858) 2023-11-06T03:04:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2917f640 1 -- 172.21.15.157:0/1571757139 learned_addr learned my addr 172.21.15.157:0/1571757139 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:39.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2897e640 1 --2- 172.21.15.157:0/1571757139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea240cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea29980640 1 --2- 172.21.15.157:0/1571757139 >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 0x7fea240cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2897e640 1 -- 172.21.15.157:0/1571757139 >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 msgr2=0x7fea240cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:39.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2897e640 1 --2- 172.21.15.157:0/1571757139 >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 0x7fea240cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2897e640 1 -- 172.21.15.157:0/1571757139 >> v1:172.21.15.195:6789/0 conn(0x7fea2415e090 legacy=0x7fea240cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea2897e640 1 -- 172.21.15.157:0/1571757139 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240cc690 con 0x7fea240719e0 2023-11-06T03:04:39.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.465+0000 7fea29980640 1 --2- 172.21.15.157:0/1571757139 >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 0x7fea240cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:39.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.469+0000 7fea2897e640 1 --2- 172.21.15.157:0/1571757139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea240cd320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea140097c0 tx=0x7fea1401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8920311d1c726195 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea1bfff640 1 -- 172.21.15.157:0/1571757139 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140a8970 con 0x7fea240719e0 2023-11-06T03:04:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea1bfff640 1 -- 172.21.15.157:0/1571757139 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea140a8ad0 con 0x7fea240719e0 2023-11-06T03:04:39.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea1bfff640 1 -- 172.21.15.157:0/1571757139 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea140a8e60 con 0x7fea240719e0 2023-11-06T03:04:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- 172.21.15.157:0/1571757139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 msgr2=0x7fea240cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/1571757139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea240cd320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea140097c0 tx=0x7fea1401ec70 comp rx=0 tx=0).stop 2023-11-06T03:04:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- 172.21.15.157:0/1571757139 shutdown_connections 2023-11-06T03:04:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/1571757139 >> v2:172.21.15.195:3300/0 conn(0x7fea240cce90 0x7fea240cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/1571757139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea240cd320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- 172.21.15.157:0/1571757139 >> 172.21.15.157:0/1571757139 conn(0x7fea2406d3d0 msgr2=0x7fea2406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- 172.21.15.157:0/1571757139 shutdown_connections 2023-11-06T03:04:39.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- 172.21.15.157:0/1571757139 wait complete. 2023-11-06T03:04:39.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 Processor -- start 2023-11-06T03:04:39.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- start start 2023-11-06T03:04:39.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea241f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea240cce90 0x7fea240b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea241f3490 con 0x7fea2415e090 2023-11-06T03:04:39.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b28d0 con 0x7fea240cce90 2023-11-06T03:04:39.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2b3e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240c8bb0 con 0x7fea240719e0 2023-11-06T03:04:39.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea29980640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea29980640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33260/0 (socket says 172.21.15.157:33260) 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea29980640 1 -- 172.21.15.157:0/246488353 learned_addr learned my addr 172.21.15.157:0/246488353 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2917f640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea241f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.473+0000 7fea2897e640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea240cce90 0x7fea240b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 -- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 msgr2=0x7fea241f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea241f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 -- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea240cce90 msgr2=0x7fea240b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea240cce90 0x7fea240b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 -- 172.21.15.157:0/246488353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c009a70 con 0x7fea2415e090 2023-11-06T03:04:39.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea29980640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009eb0 tx=0x7fea1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c009d50 con 0x7fea2415e090 2023-11-06T03:04:39.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00f040 con 0x7fea2415e090 2023-11-06T03:04:39.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea14009420 con 0x7fea2415e090 2023-11-06T03:04:39.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0146a0 con 0x7fea2415e090 2023-11-06T03:04:39.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea240b0310 con 0x7fea2415e090 2023-11-06T03:04:39.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea24072750 con 0x7fea2415e090 2023-11-06T03:04:39.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fea1c014800 con 0x7fea2415e090 2023-11-06T03:04:39.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 0x7fea08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea1c084270 con 0x7fea2415e090 2023-11-06T03:04:39.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea2917f640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 0x7fea08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.477+0000 7fea2917f640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 0x7fea08067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea20005890 tx=0x7fea20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.481+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fea1c051ad0 con 0x7fea2415e090 2023-11-06T03:04:39.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.633+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea240cc940 con 0x7fea2415e090 2023-11-06T03:04:39.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.633+0000 7fea19ffb640 1 -- 172.21.15.157:0/246488353 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea1c018020 con 0x7fea2415e090 2023-11-06T03:04:39.634 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:39.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 msgr2=0x7fea08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 0x7fea08067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea20005890 tx=0x7fea20009040 comp rx=0 tx=0).stop 2023-11-06T03:04:39.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 msgr2=0x7fea240c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009eb0 tx=0x7fea1c00c490 comp rx=0 tx=0).stop 2023-11-06T03:04:39.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 shutdown_connections 2023-11-06T03:04:39.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea08065a60 0x7fea08067f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea2415e090 0x7fea240c8670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea240cce90 0x7fea240b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 --2- 172.21.15.157:0/246488353 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea240719e0 0x7fea241f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 >> 172.21.15.157:0/246488353 conn(0x7fea2406d3d0 msgr2=0x7fea240b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 shutdown_connections 2023-11-06T03:04:39.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.637+0000 7fea2b3e3640 1 -- 172.21.15.157:0/246488353 wait complete. 2023-11-06T03:04:39.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:39.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 Processor -- start 2023-11-06T03:04:39.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 -- start start 2023-11-06T03:04:39.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 0x7f4ae415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae415f580 con 0x7f4ae415e5e0 2023-11-06T03:04:39.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae415efc0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4aeb21f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae415f2a0 con 0x7f4ae41551e0 2023-11-06T03:04:39.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae97bc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 0x7f4ae415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39618/0 (socket says 172.21.15.157:39618) 2023-11-06T03:04:39.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae97bc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 0x7f4ae415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:45030/0 (socket says 172.21.15.157:45030) 2023-11-06T03:04:39.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae3fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f4ae415e5e0 0x7f4ae40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56874/0 (socket says 172.21.15.157:56874) 2023-11-06T03:04:39.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/97267868 learned_addr learned my addr 172.21.15.157:0/97267868 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:39.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/97267868 >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 msgr2=0x7f4ae415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:39.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 --2- 172.21.15.157:0/97267868 >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 0x7f4ae415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/97267868 >> v1:172.21.15.195:6789/0 conn(0x7f4ae415e5e0 legacy=0x7f4ae40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.769+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/97267868 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae406f5a0 con 0x7f4ae41551e0 2023-11-06T03:04:39.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4ae8fbb640 1 --2- 172.21.15.157:0/97267868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009870 tx=0x7f4ad801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7551738917ac412f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4ae37fe640 1 -- 172.21.15.157:0/97267868 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80ad640 con 0x7f4ae41551e0 2023-11-06T03:04:39.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4ae37fe640 1 -- 172.21.15.157:0/97267868 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80ad7a0 con 0x7f4ae41551e0 2023-11-06T03:04:39.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4ae37fe640 1 -- 172.21.15.157:0/97267868 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80adb30 con 0x7f4ae41551e0 2023-11-06T03:04:39.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 -- 172.21.15.157:0/97267868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 msgr2=0x7f4ae406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/97267868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009870 tx=0x7f4ad801c480 comp rx=0 tx=0).stop 2023-11-06T03:04:39.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 -- 172.21.15.157:0/97267868 shutdown_connections 2023-11-06T03:04:39.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/97267868 >> v2:172.21.15.195:3300/0 conn(0x7f4ae40b3b20 0x7f4ae415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/97267868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae406fda0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 -- 172.21.15.157:0/97267868 >> 172.21.15.157:0/97267868 conn(0x7f4ae4065930 msgr2=0x7f4ae4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 -- 172.21.15.157:0/97267868 shutdown_connections 2023-11-06T03:04:39.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.773+0000 7f4aeb21f640 1 -- 172.21.15.157:0/97267868 wait complete. 2023-11-06T03:04:39.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 Processor -- start 2023-11-06T03:04:39.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- start start 2023-11-06T03:04:39.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 0x7f4ae414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4152610 con 0x7f4ae40b3b20 2023-11-06T03:04:39.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40cd8a0 con 0x7f4ae415e5e0 2023-11-06T03:04:39.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40cdb80 con 0x7f4ae41551e0 2023-11-06T03:04:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33276/0 (socket says 172.21.15.157:33276) 2023-11-06T03:04:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/3563739142 learned_addr learned my addr 172.21.15.157:0/3563739142 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae3fff640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae97bc640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 0x7f4ae414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 msgr2=0x7f4ae414cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae414cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 msgr2=0x7f4ae414d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 0x7f4ae414d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 -- 172.21.15.157:0/3563739142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80094d0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae97bc640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 0x7f4ae414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:39.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae8fbb640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ad801ab30 tx=0x7f4ad8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80b8620 con 0x7f4ae40b3b20 2023-11-06T03:04:39.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80a6040 con 0x7f4ae40b3b20 2023-11-06T03:04:39.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae40cde60 con 0x7f4ae40b3b20 2023-11-06T03:04:39.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80b89c0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae41356e0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae40ce310 con 0x7f4ae40b3b20 2023-11-06T03:04:39.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4ad80b4040 con 0x7f4ae40b3b20 2023-11-06T03:04:39.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.777+0000 7f4ae17fa640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 0x7f4ac8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:39.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.781+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ad811ded0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.781+0000 7f4ae3fff640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 0x7f4ac8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:39.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.781+0000 7f4ae3fff640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 0x7f4ac8067f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0000f70 tx=0x7f4ad0007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:39.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.785+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4ad80eb8e0 con 0x7f4ae40b3b20 2023-11-06T03:04:39.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ae4073940 con 0x7f4ae40b3b20 2023-11-06T03:04:39.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4ae17fa640 1 -- 172.21.15.157:0/3563739142 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4ad80eef00 con 0x7f4ae40b3b20 2023-11-06T03:04:39.925 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:39.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 msgr2=0x7f4ac8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 0x7f4ac8067f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0000f70 tx=0x7f4ad0007cf0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 msgr2=0x7f4ae41351a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:39.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.925+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ad801ab30 tx=0x7f4ad8019220 comp rx=0 tx=0).stop 2023-11-06T03:04:39.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 shutdown_connections 2023-11-06T03:04:39.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4ac8065a60 0x7f4ac8067f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ae415e5e0 0x7f4ae414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ae41551e0 0x7f4ae414cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 --2- 172.21.15.157:0/3563739142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ae40b3b20 0x7f4ae41351a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:39.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 >> 172.21.15.157:0/3563739142 conn(0x7f4ae4065930 msgr2=0x7f4ae40b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:39.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 shutdown_connections 2023-11-06T03:04:39.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:39.929+0000 7f4aeb21f640 1 -- 172.21.15.157:0/3563739142 wait complete. 2023-11-06T03:04:39.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.065+0000 7f0b3ad5b640 1 Processor -- start 2023-11-06T03:04:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.065+0000 7f0b3ad5b640 1 -- start start 2023-11-06T03:04:40.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b3ad5b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b340b2af0 0x7f0b340ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b3ad5b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b3406d000 0x7f0b34159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b3ad5b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3415e440 con 0x7f0b341599c0 2023-11-06T03:04:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b3ad5b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3406c240 con 0x7f0b3406d000 2023-11-06T03:04:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b3ad5b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3406c520 con 0x7f0b340b2af0 2023-11-06T03:04:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b3406d000 0x7f0b34159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b3406d000 0x7f0b34159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33282/0 (socket says 172.21.15.157:33282) 2023-11-06T03:04:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 -- 172.21.15.157:0/501255565 learned_addr learned my addr 172.21.15.157:0/501255565 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b33fff640 1 --2- 172.21.15.157:0/501255565 >> v2:172.21.15.195:3300/0 conn(0x7f0b340b2af0 0x7f0b340ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 -- 172.21.15.157:0/501255565 >> v2:172.21.15.195:3300/0 conn(0x7f0b340b2af0 msgr2=0x7f0b340ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 --2- 172.21.15.157:0/501255565 >> v2:172.21.15.195:3300/0 conn(0x7f0b340b2af0 0x7f0b340ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 -- 172.21.15.157:0/501255565 >> v1:172.21.15.195:6789/0 conn(0x7f0b341599c0 legacy=0x7f0b340b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 -- 172.21.15.157:0/501255565 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3406c800 con 0x7f0b3406d000 2023-11-06T03:04:40.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b392f8640 1 --2- 172.21.15.157:0/501255565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34159da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b340cfa20 tx=0x7f0b2c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5910a00940b48134 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b337fe640 1 -- 172.21.15.157:0/501255565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0a5840 con 0x7f0b3406d000 2023-11-06T03:04:40.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b337fe640 1 -- 172.21.15.157:0/501255565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0a59a0 con 0x7f0b3406d000 2023-11-06T03:04:40.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.069+0000 7f0b337fe640 1 -- 172.21.15.157:0/501255565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0a5d30 con 0x7f0b3406d000 2023-11-06T03:04:40.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/501255565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 msgr2=0x7f0b34159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/501255565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34159da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b340cfa20 tx=0x7f0b2c01c960 comp rx=0 tx=0).stop 2023-11-06T03:04:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/501255565 shutdown_connections 2023-11-06T03:04:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/501255565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34159da0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/501255565 >> v2:172.21.15.195:3300/0 conn(0x7f0b340b2af0 0x7f0b340ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/501255565 >> 172.21.15.157:0/501255565 conn(0x7f0b34065930 msgr2=0x7f0b34065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/501255565 shutdown_connections 2023-11-06T03:04:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/501255565 wait complete. 2023-11-06T03:04:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 Processor -- start 2023-11-06T03:04:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- start start 2023-11-06T03:04:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b340b2af0 0x7f0b341507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b341599c0 0x7f0b341f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3414b410 con 0x7f0b3406d000 2023-11-06T03:04:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34150e10 con 0x7f0b340b2af0 2023-11-06T03:04:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340cec20 con 0x7f0b341599c0 2023-11-06T03:04:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33288/0 (socket says 172.21.15.157:33288) 2023-11-06T03:04:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 -- 172.21.15.157:0/2700243497 learned_addr learned my addr 172.21.15.157:0/2700243497 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b33fff640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b340b2af0 0x7f0b341507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b392f8640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b341599c0 0x7f0b341f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 -- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b341599c0 msgr2=0x7f0b341f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b341599c0 0x7f0b341f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 -- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b340b2af0 msgr2=0x7f0b341507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b340b2af0 0x7f0b341507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 -- 172.21.15.157:0/2700243497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c009b30 con 0x7f0b3406d000 2023-11-06T03:04:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b38af7640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b410 tx=0x7f0b2400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24004020 con 0x7f0b3406d000 2023-11-06T03:04:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b24004180 con 0x7f0b3406d000 2023-11-06T03:04:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b340cef00 con 0x7f0b3406d000 2023-11-06T03:04:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24011630 con 0x7f0b3406d000 2023-11-06T03:04:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3414cf50 con 0x7f0b3406d000 2023-11-06T03:04:40.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.073+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3414b610 con 0x7f0b3406d000 2023-11-06T03:04:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.077+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0b24011790 con 0x7f0b3406d000 2023-11-06T03:04:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.077+0000 7f0b317fa640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 0x7f0b20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.077+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b24082fa0 con 0x7f0b3406d000 2023-11-06T03:04:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.077+0000 7f0b33fff640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 0x7f0b20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.077+0000 7f0b33fff640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 0x7f0b20067f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b3406fdd0 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.081+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0b240b4070 con 0x7f0b3406d000 2023-11-06T03:04:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.221+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b340cea10 con 0x7f0b3406d000 2023-11-06T03:04:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.221+0000 7f0b317fa640 1 -- 172.21.15.157:0/2700243497 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b240150a0 con 0x7f0b3406d000 2023-11-06T03:04:40.221 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:40.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.221+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 msgr2=0x7f0b20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.221+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 0x7f0b20067f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b3406fdd0 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2023-11-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 msgr2=0x7f0b34150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b410 tx=0x7f0b2400b8d0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 shutdown_connections 2023-11-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b20065a60 0x7f0b20067f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b341599c0 0x7f0b341f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b340b2af0 0x7f0b341507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 --2- 172.21.15.157:0/2700243497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b3406d000 0x7f0b34150260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 >> 172.21.15.157:0/2700243497 conn(0x7f0b34065930 msgr2=0x7f0b340707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 shutdown_connections 2023-11-06T03:04:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.225+0000 7f0b3ad5b640 1 -- 172.21.15.157:0/2700243497 wait complete. 2023-11-06T03:04:40.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:40.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.349+0000 7f218c65b640 1 Processor -- start 2023-11-06T03:04:40.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.349+0000 7f218c65b640 1 -- start start 2023-11-06T03:04:40.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218c65b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2184146370 0x7f21840b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218c65b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218c65b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21840cf1a0 con 0x7f218406b6a0 2023-11-06T03:04:40.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218c65b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218406c3d0 con 0x7f2184146370 2023-11-06T03:04:40.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218c65b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218406f5a0 con 0x7f21840ce800 2023-11-06T03:04:40.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f218a3f7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2184146370 0x7f21840b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39644/0 (socket says 172.21.15.157:39644) 2023-11-06T03:04:40.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 -- 172.21.15.157:0/4125465139 learned_addr learned my addr 172.21.15.157:0/4125465139 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 -- 172.21.15.157:0/4125465139 >> v2:172.21.15.195:3300/0 conn(0x7f2184146370 msgr2=0x7f21840b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 --2- 172.21.15.157:0/4125465139 >> v2:172.21.15.195:3300/0 conn(0x7f2184146370 0x7f21840b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 -- 172.21.15.157:0/4125465139 >> v1:172.21.15.195:6789/0 conn(0x7f218406b6a0 legacy=0x7f218406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 -- 172.21.15.157:0/4125465139 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218406f880 con 0x7f21840ce800 2023-11-06T03:04:40.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.353+0000 7f2189bf6640 1 --2- 172.21.15.157:0/4125465139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2174009870 tx=0x7f217401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c8b0ec4c8c22769 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f21893f5640 1 -- 172.21.15.157:0/4125465139 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740aa610 con 0x7f21840ce800 2023-11-06T03:04:40.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f21893f5640 1 -- 172.21.15.157:0/4125465139 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740a6040 con 0x7f21840ce800 2023-11-06T03:04:40.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f21893f5640 1 -- 172.21.15.157:0/4125465139 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740aa990 con 0x7f21840ce800 2023-11-06T03:04:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/4125465139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 msgr2=0x7f21840cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- 172.21.15.157:0/4125465139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2174009870 tx=0x7f217401c4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/4125465139 shutdown_connections 2023-11-06T03:04:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- 172.21.15.157:0/4125465139 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21840ce800 0x7f21840cec60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- 172.21.15.157:0/4125465139 >> v2:172.21.15.195:3300/0 conn(0x7f2184146370 0x7f21840b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/4125465139 >> 172.21.15.157:0/4125465139 conn(0x7f2184065930 msgr2=0x7f2184065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/4125465139 shutdown_connections 2023-11-06T03:04:40.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/4125465139 wait complete. 2023-11-06T03:04:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 Processor -- start 2023-11-06T03:04:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- start start 2023-11-06T03:04:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f218406b6a0 0x7f21841f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21840ce800 0x7f218415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218415b5f0 con 0x7f2184146370 2023-11-06T03:04:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218415b8d0 con 0x7f21840ce800 2023-11-06T03:04:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21841f8430 con 0x7f218406b6a0 2023-11-06T03:04:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33300/0 (socket says 172.21.15.157:33300) 2023-11-06T03:04:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 -- 172.21.15.157:0/893074107 learned_addr learned my addr 172.21.15.157:0/893074107 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f2189bf6640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21840ce800 0x7f218415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218a3f7640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f218406b6a0 0x7f21841f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 -- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f218406b6a0 msgr2=0x7f21841f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f218406b6a0 0x7f21841f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 -- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21840ce800 msgr2=0x7f218415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21840ce800 0x7f218415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 -- 172.21.15.157:0/893074107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740094d0 con 0x7f2184146370 2023-11-06T03:04:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218abf8640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c730 tx=0x7f217c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c003b80 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c003ce0 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21841f8710 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c00aa30 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.357+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218415b300 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218415b730 con 0x7f2184146370 2023-11-06T03:04:40.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f217c00ab90 con 0x7f2184146370 2023-11-06T03:04:40.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f217affd640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 0x7f2168067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f217c082f60 con 0x7f2184146370 2023-11-06T03:04:40.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f218a3f7640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 0x7f2168067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.361+0000 7f218a3f7640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 0x7f2168067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2184141610 tx=0x7f2180008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.365+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f218415b730 con 0x7f2184146370 2023-11-06T03:04:40.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.517+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f218415b730 con 0x7f2184146370 2023-11-06T03:04:40.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.517+0000 7f217affd640 1 -- 172.21.15.157:0/893074107 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f218415b730 con 0x7f2184146370 2023-11-06T03:04:40.517 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:40.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 msgr2=0x7f2168067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 0x7f2168067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2184141610 tx=0x7f2180008040 comp rx=0 tx=0).stop 2023-11-06T03:04:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 msgr2=0x7f21840af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c730 tx=0x7f217c00cbf0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 shutdown_connections 2023-11-06T03:04:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2168065a60 0x7f2168067f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2184146370 0x7f21840af830 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21840ce800 0x7f218415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 --2- 172.21.15.157:0/893074107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f218406b6a0 0x7f21841f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 >> 172.21.15.157:0/893074107 conn(0x7f2184065930 msgr2=0x7f21840b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 shutdown_connections 2023-11-06T03:04:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.521+0000 7f218c65b640 1 -- 172.21.15.157:0/893074107 wait complete. 2023-11-06T03:04:40.535 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:40.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 Processor -- start 2023-11-06T03:04:40.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 -- start start 2023-11-06T03:04:40.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11501551e0 0x7f115006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115015f580 con 0x7f11500b3b20 2023-11-06T03:04:40.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115015efc0 con 0x7f115015e5e0 2023-11-06T03:04:40.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f1155aa0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115015f2a0 con 0x7f11501551e0 2023-11-06T03:04:40.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11501551e0 0x7f115006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f11500b3b20 0x7f115015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49748/0 (socket says 172.21.15.157:49748) 2023-11-06T03:04:40.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114ffff640 1 -- 172.21.15.157:0/1225368030 learned_addr learned my addr 172.21.15.157:0/1225368030 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:40.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114f7fe640 1 -- 172.21.15.157:0/1225368030 >> v1:172.21.15.195:6789/0 conn(0x7f115015e5e0 legacy=0x7f11500b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114f7fe640 1 -- 172.21.15.157:0/1225368030 >> v1:172.21.15.157:6789/0 conn(0x7f11500b3b20 legacy=0x7f115015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114e7fc640 1 -- 172.21.15.157:0/1225368030 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127794907 0 0) 0x7f115015f580 con 0x7f11500b3b20 2023-11-06T03:04:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.657+0000 7f114f7fe640 1 -- 172.21.15.157:0/1225368030 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115006f5a0 con 0x7f11501551e0 2023-11-06T03:04:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114f7fe640 1 --2- 172.21.15.157:0/1225368030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f115006fda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11400097c0 tx=0x7f114001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683b3b4af07837d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114e7fc640 1 -- 172.21.15.157:0/1225368030 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140004230 con 0x7f11501551e0 2023-11-06T03:04:40.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114e7fc640 1 -- 172.21.15.157:0/1225368030 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11400a7050 con 0x7f11501551e0 2023-11-06T03:04:40.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114e7fc640 1 -- 172.21.15.157:0/1225368030 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140003620 con 0x7f11501551e0 2023-11-06T03:04:40.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/1225368030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 msgr2=0x7f115006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 --2- 172.21.15.157:0/1225368030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f115006fda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11400097c0 tx=0x7f114001c330 comp rx=0 tx=0).stop 2023-11-06T03:04:40.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/1225368030 shutdown_connections 2023-11-06T03:04:40.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 --2- 172.21.15.157:0/1225368030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f115006fda0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/1225368030 >> 172.21.15.157:0/1225368030 conn(0x7f1150065930 msgr2=0x7f1150065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/1225368030 shutdown_connections 2023-11-06T03:04:40.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/1225368030 wait complete. 2023-11-06T03:04:40.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 Processor -- start 2023-11-06T03:04:40.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- start start 2023-11-06T03:04:40.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 0x7f1150155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015e5e0 0x7f11501f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1150155b90 con 0x7f11501551e0 2023-11-06T03:04:40.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115014ac40 con 0x7f11500b3b20 2023-11-06T03:04:40.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115014af20 con 0x7f115015e5e0 2023-11-06T03:04:40.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33304/0 (socket says 172.21.15.157:33304) 2023-11-06T03:04:40.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 -- 172.21.15.157:0/3301044508 learned_addr learned my addr 172.21.15.157:0/3301044508 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114f7fe640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 0x7f1150155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 -- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015e5e0 msgr2=0x7f11501f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114ffff640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015e5e0 0x7f11501f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015e5e0 0x7f11501f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 -- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 msgr2=0x7f1150155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 0x7f1150155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 -- 172.21.15.157:0/3301044508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140009420 con 0x7f11501551e0 2023-11-06T03:04:40.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114f7fe640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 0x7f1150155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:40.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f114effd640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f113800c3c0 tx=0x7f113800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380052d0 con 0x7f11501551e0 2023-11-06T03:04:40.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1138005430 con 0x7f11501551e0 2023-11-06T03:04:40.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115014b200 con 0x7f11501551e0 2023-11-06T03:04:40.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138003840 con 0x7f11501551e0 2023-11-06T03:04:40.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.661+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115014b7c0 con 0x7f11501551e0 2023-11-06T03:04:40.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115014b9b0 con 0x7f11501551e0 2023-11-06T03:04:40.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f11380039a0 con 0x7f11501551e0 2023-11-06T03:04:40.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f1147fff640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 0x7f1130067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11380827a0 con 0x7f11501551e0 2023-11-06T03:04:40.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f114f7fe640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 0x7f1130067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.665+0000 7f114f7fe640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 0x7f1130067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11400ad000 tx=0x7f1140008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.669+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1138050090 con 0x7f11501551e0 2023-11-06T03:04:40.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.809+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1150072750 con 0x7f11501551e0 2023-11-06T03:04:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1147fff640 1 -- 172.21.15.157:0/3301044508 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f11380536b0 con 0x7f11501551e0 2023-11-06T03:04:40.812 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:40.813 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:40.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 msgr2=0x7f1130067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 0x7f1130067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11400ad000 tx=0x7f1140008040 comp rx=0 tx=0).stop 2023-11-06T03:04:40.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 msgr2=0x7f11500cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f113800c3c0 tx=0x7f113800c880 comp rx=0 tx=0).stop 2023-11-06T03:04:40.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 shutdown_connections 2023-11-06T03:04:40.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1130065a60 0x7f1130067f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f115015e5e0 0x7f11501f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11501551e0 0x7f11500cdbd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 --2- 172.21.15.157:0/3301044508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11500b3b20 0x7f1150155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 >> 172.21.15.157:0/3301044508 conn(0x7f1150065930 msgr2=0x7f11500aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 shutdown_connections 2023-11-06T03:04:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.813+0000 7f1155aa0640 1 -- 172.21.15.157:0/3301044508 wait complete. 2023-11-06T03:04:40.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:40.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:40.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.941+0000 7f74866d3640 1 Processor -- start 2023-11-06T03:04:40.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.941+0000 7f74866d3640 1 -- start start 2023-11-06T03:04:40.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f74866d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 0x7f748015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f74866d3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748015f580 con 0x7f74801551e0 2023-11-06T03:04:40.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f74866d3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748015efc0 con 0x7f748015e5e0 2023-11-06T03:04:40.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f74866d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800704b0 con 0x7f74800b3b20 2023-11-06T03:04:40.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f74801551e0 0x7f748006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49754/0 (socket says 172.21.15.157:49754) 2023-11-06T03:04:40.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 -- 172.21.15.157:0/2929560593 learned_addr learned my addr 172.21.15.157:0/2929560593 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:40.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f7484c70640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 0x7f748015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512554140 0 0) 0x7f748015f580 con 0x7f74801551e0 2023-11-06T03:04:40.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003530 con 0x7f74801551e0 2023-11-06T03:04:40.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1016448874 0 0) 0x7f7468003530 con 0x7f74801551e0 2023-11-06T03:04:40.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 msgr2=0x7f748015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 0x7f748015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 >> v1:172.21.15.195:6789/0 conn(0x7f748015e5e0 legacy=0x7f74800b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748015f2a0 con 0x7f74801551e0 2023-11-06T03:04:40.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7470001680 con 0x7f74801551e0 2023-11-06T03:04:40.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 >> v1:172.21.15.157:6789/0 conn(0x7f74801551e0 legacy=0x7f748006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74680039b0 0x7f7468003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680047d0 0x7f7468004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 0x7f74680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800704b0 con 0x7f74680039b0 2023-11-06T03:04:40.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746800baf0 con 0x7f74680047d0 2023-11-06T03:04:40.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746800bcb0 con 0x7f74800b3b20 2023-11-06T03:04:40.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74680039b0 0x7f7468003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747f7fe640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680047d0 0x7f7468004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 -- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 msgr2=0x7f74680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74800b3b20 0x7f74680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 -- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680047d0 msgr2=0x7f7468004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:40.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680047d0 0x7f7468004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 -- 172.21.15.157:0/2929560593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468003530 con 0x7f74680039b0 2023-11-06T03:04:40.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747f7fe640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680047d0 0x7f7468004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:40.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.945+0000 7f747ffff640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74680039b0 0x7f7468003dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f747000c240 tx=0x7f747001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33e377f86660496 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74700a5b10 con 0x7f74680039b0 2023-11-06T03:04:40.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74700a5c90 con 0x7f74680039b0 2023-11-06T03:04:40.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747effd640 1 -- 172.21.15.157:0/2929560593 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74700058e0 con 0x7f74680039b0 2023-11-06T03:04:40.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2929560593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74680039b0 msgr2=0x7f7468003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 --2- 172.21.15.157:0/2929560593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74680039b0 0x7f7468003dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f747000c240 tx=0x7f747001cc10 comp rx=0 tx=0).stop 2023-11-06T03:04:40.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 -- 172.21.15.157:0/2929560593 reap_dead start 2023-11-06T03:04:40.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2929560593 shutdown_connections 2023-11-06T03:04:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2929560593 >> 172.21.15.157:0/2929560593 conn(0x7f7480065930 msgr2=0x7f7480065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2929560593 shutdown_connections 2023-11-06T03:04:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2929560593 wait complete. 2023-11-06T03:04:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 Processor -- start 2023-11-06T03:04:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- start start 2023-11-06T03:04:40.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680039b0 0x7f748014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f748014db30 0x7f74801448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748014d710 con 0x7f7480145060 2023-11-06T03:04:40.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748014ee10 con 0x7f74680039b0 2023-11-06T03:04:40.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7480144e20 con 0x7f748014db30 2023-11-06T03:04:40.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33320/0 (socket says 172.21.15.157:33320) 2023-11-06T03:04:40.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 -- 172.21.15.157:0/2933171760 learned_addr learned my addr 172.21.15.157:0/2933171760 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:40.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747ffff640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680039b0 0x7f748014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747f7fe640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f748014db30 0x7f74801448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 -- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f748014db30 msgr2=0x7f74801448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f748014db30 0x7f74801448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 -- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680039b0 msgr2=0x7f748014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:40.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680039b0 0x7f748014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:40.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 -- 172.21.15.157:0/2933171760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748014cd70 con 0x7f7480145060 2023-11-06T03:04:40.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f7484c70640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7468004750 tx=0x7f747800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478010040 con 0x7f7480145060 2023-11-06T03:04:40.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747800bce0 con 0x7f7480145060 2023-11-06T03:04:40.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478014640 con 0x7f7480145060 2023-11-06T03:04:40.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748014d050 con 0x7f7480145060 2023-11-06T03:04:40.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.949+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74801462f0 con 0x7f7480145060 2023-11-06T03:04:40.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748014d850 con 0x7f7480145060 2023-11-06T03:04:40.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f74780147a0 con 0x7f7480145060 2023-11-06T03:04:40.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f747cff9640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 0x7f746c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:40.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7478083b80 con 0x7f7480145060 2023-11-06T03:04:40.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f747ffff640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 0x7f746c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:40.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.953+0000 7f747ffff640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 0x7f746c067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f74700ab000 tx=0x7f74700a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:40.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:40.957+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7478051550 con 0x7f7480145060 2023-11-06T03:04:41.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f748014d850 con 0x7f7480145060 2023-11-06T03:04:41.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f747cff9640 1 -- 172.21.15.157:0/2933171760 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7478018020 con 0x7f7480145060 2023-11-06T03:04:41.100 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:41.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 msgr2=0x7f746c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 0x7f746c067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f74700ab000 tx=0x7f74700a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:41.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 msgr2=0x7f7480153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7468004750 tx=0x7f747800b480 comp rx=0 tx=0).stop 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 shutdown_connections 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f746c065a90 0x7f746c067f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7480145060 0x7f7480153740 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f748014db30 0x7f74801448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 --2- 172.21.15.157:0/2933171760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74680039b0 0x7f748014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 >> 172.21.15.157:0/2933171760 conn(0x7f7480065930 msgr2=0x7f748014f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 shutdown_connections 2023-11-06T03:04:41.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.101+0000 7f74866d3640 1 -- 172.21.15.157:0/2933171760 wait complete. 2023-11-06T03:04:41.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:41.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.233+0000 7faba646a640 1 Processor -- start 2023-11-06T03:04:41.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 -- start start 2023-11-06T03:04:41.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 0x7faba006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faba00b3b20 0x7faba015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba015f580 con 0x7faba015e5e0 2023-11-06T03:04:41.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba015efc0 con 0x7faba00b3b20 2023-11-06T03:04:41.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba646a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba015f2a0 con 0x7faba01551e0 2023-11-06T03:04:41.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faba00b3b20 0x7faba015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faba00b3b20 0x7faba015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33334/0 (socket says 172.21.15.157:33334) 2023-11-06T03:04:41.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 -- 172.21.15.157:0/3576727982 learned_addr learned my addr 172.21.15.157:0/3576727982 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:41.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7fab9ffff640 1 --2- 172.21.15.157:0/3576727982 >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 0x7faba006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 -- 172.21.15.157:0/3576727982 >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 msgr2=0x7faba006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:41.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 --2- 172.21.15.157:0/3576727982 >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 0x7faba006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 -- 172.21.15.157:0/3576727982 >> v1:172.21.15.195:6789/0 conn(0x7faba015e5e0 legacy=0x7faba00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:41.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 -- 172.21.15.157:0/3576727982 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba006f5a0 con 0x7faba00b3b20 2023-11-06T03:04:41.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7fab9ffff640 1 --2- 172.21.15.157:0/3576727982 >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 0x7faba006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:41.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7faba4a07640 1 --2- 172.21.15.157:0/3576727982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba00b3b20 0x7faba015c1b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faba0066830 tx=0x7fab9801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fee174990bdfbcc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7fab9f7fe640 1 -- 172.21.15.157:0/3576727982 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980a5720 con 0x7faba00b3b20 2023-11-06T03:04:41.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7fab9f7fe640 1 -- 172.21.15.157:0/3576727982 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab980a5880 con 0x7faba00b3b20 2023-11-06T03:04:41.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.237+0000 7fab9f7fe640 1 -- 172.21.15.157:0/3576727982 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980a5c10 con 0x7faba00b3b20 2023-11-06T03:04:41.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/3576727982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba00b3b20 msgr2=0x7faba015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- 172.21.15.157:0/3576727982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba00b3b20 0x7faba015c1b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faba0066830 tx=0x7fab9801c840 comp rx=0 tx=0).stop 2023-11-06T03:04:41.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/3576727982 shutdown_connections 2023-11-06T03:04:41.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- 172.21.15.157:0/3576727982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba00b3b20 0x7faba015c1b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- 172.21.15.157:0/3576727982 >> v2:172.21.15.195:3300/0 conn(0x7faba01551e0 0x7faba006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/3576727982 >> 172.21.15.157:0/3576727982 conn(0x7faba0065930 msgr2=0x7faba0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/3576727982 shutdown_connections 2023-11-06T03:04:41.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/3576727982 wait complete. 2023-11-06T03:04:41.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 Processor -- start 2023-11-06T03:04:41.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- start start 2023-11-06T03:04:41.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba00b3b20 0x7faba0152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba015e5e0 0x7faba01563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0152950 con 0x7faba01551e0 2023-11-06T03:04:41.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba0152c30 con 0x7faba015e5e0 2023-11-06T03:04:41.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba0152f10 con 0x7faba00b3b20 2023-11-06T03:04:41.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33348/0 (socket says 172.21.15.157:33348) 2023-11-06T03:04:41.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 -- 172.21.15.157:0/1804845149 learned_addr learned my addr 172.21.15.157:0/1804845149 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:41.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba4a07640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba015e5e0 0x7faba01563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab9ffff640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba00b3b20 0x7faba0152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 -- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba00b3b20 msgr2=0x7faba0152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba00b3b20 0x7faba0152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 -- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba015e5e0 msgr2=0x7faba01563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba015e5e0 0x7faba01563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 -- 172.21.15.157:0/1804845149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009b30 con 0x7faba01551e0 2023-11-06T03:04:41.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab977fe640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b410 tx=0x7fab8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88004020 con 0x7faba01551e0 2023-11-06T03:04:41.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab88004180 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba01531f0 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88011630 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba01568f0 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.241+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba0153710 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.245+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fab88011790 con 0x7faba01551e0 2023-11-06T03:04:41.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.245+0000 7fab9d7fa640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 0x7fab80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.245+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab88083090 con 0x7faba01551e0 2023-11-06T03:04:41.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.245+0000 7fab9ffff640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 0x7fab80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.245+0000 7fab9ffff640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 0x7fab80067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab90006fd0 tx=0x7fab90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.249+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fab880508f0 con 0x7faba01551e0 2023-11-06T03:04:41.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.389+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faba0072750 con 0x7faba01551e0 2023-11-06T03:04:41.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.389+0000 7fab9d7fa640 1 -- 172.21.15.157:0/1804845149 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fab880150a0 con 0x7faba01551e0 2023-11-06T03:04:41.390 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 msgr2=0x7fab80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 0x7fab80067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fab90006fd0 tx=0x7fab90008040 comp rx=0 tx=0).stop 2023-11-06T03:04:41.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 msgr2=0x7faba01539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b410 tx=0x7fab8800b8d0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 shutdown_connections 2023-11-06T03:04:41.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab80065a60 0x7fab80067f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba015e5e0 0x7faba01563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faba01551e0 0x7faba01539f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 --2- 172.21.15.157:0/1804845149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faba00b3b20 0x7faba0152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 >> 172.21.15.157:0/1804845149 conn(0x7faba0065930 msgr2=0x7faba00ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 shutdown_connections 2023-11-06T03:04:41.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.393+0000 7faba646a640 1 -- 172.21.15.157:0/1804845149 wait complete. 2023-11-06T03:04:41.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 Processor -- start 2023-11-06T03:04:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 -- start start 2023-11-06T03:04:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff8001551e0 0x7ff80006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80015f580 con 0x7ff80015e5e0 2023-11-06T03:04:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80015efc0 con 0x7ff8000b3b20 2023-11-06T03:04:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff805138640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80015f2a0 con 0x7ff8001551e0 2023-11-06T03:04:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fe59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff80015e5e0 0x7ff8000b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49764/0 (socket says 172.21.15.157:49764) 2023-11-06T03:04:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fe59c640 1 -- 172.21.15.157:0/2443480274 learned_addr learned my addr 172.21.15.157:0/2443480274 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/2443480274 >> v2:172.21.15.157:3300/0 conn(0x7ff8001551e0 0x7ff80006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fdd9b640 1 -- 172.21.15.157:0/2443480274 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844695628 0 0) 0x7ff80015f580 con 0x7ff80015e5e0 2023-11-06T03:04:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fdd9b640 1 -- 172.21.15.157:0/2443480274 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e8003530 con 0x7ff80015e5e0 2023-11-06T03:04:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fed9d640 1 -- 172.21.15.157:0/2443480274 >> v1:172.21.15.195:6789/0 conn(0x7ff8000b3b20 legacy=0x7ff80015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fed9d640 1 -- 172.21.15.157:0/2443480274 >> v1:172.21.15.157:6789/0 conn(0x7ff80015e5e0 legacy=0x7ff8000b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.537+0000 7ff7fed9d640 1 -- 172.21.15.157:0/2443480274 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80006f5a0 con 0x7ff8001551e0 2023-11-06T03:04:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/2443480274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80006fda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00097c0 tx=0x7ff7f001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ef7b69b775a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fdd9b640 1 -- 172.21.15.157:0/2443480274 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0004230 con 0x7ff8001551e0 2023-11-06T03:04:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fdd9b640 1 -- 172.21.15.157:0/2443480274 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f00a7050 con 0x7ff8001551e0 2023-11-06T03:04:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fdd9b640 1 -- 172.21.15.157:0/2443480274 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003620 con 0x7ff8001551e0 2023-11-06T03:04:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- 172.21.15.157:0/2443480274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 msgr2=0x7ff80006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 --2- 172.21.15.157:0/2443480274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80006fda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00097c0 tx=0x7ff7f001c330 comp rx=0 tx=0).stop 2023-11-06T03:04:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- 172.21.15.157:0/2443480274 shutdown_connections 2023-11-06T03:04:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 --2- 172.21.15.157:0/2443480274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80006fda0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- 172.21.15.157:0/2443480274 >> 172.21.15.157:0/2443480274 conn(0x7ff800065930 msgr2=0x7ff800065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- 172.21.15.157:0/2443480274 shutdown_connections 2023-11-06T03:04:41.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- 172.21.15.157:0/2443480274 wait complete. 2023-11-06T03:04:41.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 Processor -- start 2023-11-06T03:04:41.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- start start 2023-11-06T03:04:41.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 0x7ff800159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80015e5e0 0x7ff80014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001533e0 con 0x7ff8001551e0 2023-11-06T03:04:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001536c0 con 0x7ff8000b3b20 2023-11-06T03:04:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff805138640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001539a0 con 0x7ff80015e5e0 2023-11-06T03:04:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33352/0 (socket says 172.21.15.157:33352) 2023-11-06T03:04:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 -- 172.21.15.157:0/3105609474 learned_addr learned my addr 172.21.15.157:0/3105609474 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 0x7ff800159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7ff59e640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80015e5e0 0x7ff80014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 -- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80015e5e0 msgr2=0x7ff80014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80015e5e0 0x7ff80014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 -- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 msgr2=0x7ff800159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 0x7ff800159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fe59c640 1 -- 172.21.15.157:0/3105609474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0009420 con 0x7ff8001551e0 2023-11-06T03:04:41.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.541+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 0x7ff800159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:41.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7fe59c640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c420 tx=0x7ff7f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4005cf0 con 0x7ff8001551e0 2023-11-06T03:04:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4005e50 con 0x7ff8001551e0 2023-11-06T03:04:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40072d0 con 0x7ff8001551e0 2023-11-06T03:04:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8001596e0 con 0x7ff8001551e0 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff800154750 con 0x7ff8001551e0 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff800159900 con 0x7ff8001551e0 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7f4007430 con 0x7ff8001551e0 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7f40827c0 con 0x7ff8001551e0 2023-11-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.545+0000 7ff7fed9d640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 0x7ff7ec067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00ad000 tx=0x7ff7f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.549+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff7f4050160 con 0x7ff8001551e0 2023-11-06T03:04:41.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.689+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff800072750 con 0x7ff8001551e0 2023-11-06T03:04:41.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff7e77fe640 1 -- 172.21.15.157:0/3105609474 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7f4053780 con 0x7ff8001551e0 2023-11-06T03:04:41.691 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:41.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 msgr2=0x7ff7ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 0x7ff7ec067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00ad000 tx=0x7ff7f0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:41.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 msgr2=0x7ff80014c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c420 tx=0x7ff7f400c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 shutdown_connections 2023-11-06T03:04:41.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7ec065a60 0x7ff7ec067f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80015e5e0 0x7ff80014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8001551e0 0x7ff80014c730 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 --2- 172.21.15.157:0/3105609474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8000b3b20 0x7ff800159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 >> 172.21.15.157:0/3105609474 conn(0x7ff800065930 msgr2=0x7ff8000ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 shutdown_connections 2023-11-06T03:04:41.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.693+0000 7ff805138640 1 -- 172.21.15.157:0/3105609474 wait complete. 2023-11-06T03:04:41.710 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 Processor -- start 2023-11-06T03:04:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 -- start start 2023-11-06T03:04:41.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33a00b3b20 0x7f33a015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a015f580 con 0x7f33a015e5e0 2023-11-06T03:04:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a015efc0 con 0x7f33a00b3b20 2023-11-06T03:04:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f33a579b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015f2a0 con 0x7f33a01551e0 2023-11-06T03:04:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339e7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f33a015e5e0 0x7f33a00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49778/0 (socket says 172.21.15.157:49778) 2023-11-06T03:04:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339e7fc640 1 -- 172.21.15.157:0/2610327249 learned_addr learned my addr 172.21.15.157:0/2610327249 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339f7fe640 1 --2- 172.21.15.157:0/2610327249 >> v2:172.21.15.157:3300/0 conn(0x7f33a00b3b20 0x7f33a015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339effd640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339f7fe640 1 -- 172.21.15.157:0/2610327249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 msgr2=0x7f33a006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339f7fe640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339f7fe640 1 -- 172.21.15.157:0/2610327249 >> v1:172.21.15.157:6789/0 conn(0x7f33a015e5e0 legacy=0x7f33a00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339f7fe640 1 -- 172.21.15.157:0/2610327249 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a006f5a0 con 0x7f33a00b3b20 2023-11-06T03:04:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.825+0000 7f339dffb640 1 -- 172.21.15.157:0/2610327249 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294687801 0 0) 0x7f33a015f580 con 0x7f33a015e5e0 2023-11-06T03:04:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339f7fe640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00b3b20 0x7f33a015c1b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3398009d90 tx=0x7f339801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f9d876f5ff252b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339dffb640 1 -- 172.21.15.157:0/2610327249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980a5970 con 0x7f33a00b3b20 2023-11-06T03:04:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339dffb640 1 -- 172.21.15.157:0/2610327249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980a5ad0 con 0x7f33a00b3b20 2023-11-06T03:04:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339dffb640 1 -- 172.21.15.157:0/2610327249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980a5e60 con 0x7f33a00b3b20 2023-11-06T03:04:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- 172.21.15.157:0/2610327249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00b3b20 msgr2=0x7f33a015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00b3b20 0x7f33a015c1b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3398009d90 tx=0x7f339801c890 comp rx=0 tx=0).stop 2023-11-06T03:04:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- 172.21.15.157:0/2610327249 shutdown_connections 2023-11-06T03:04:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a00b3b20 0x7f33a015c1b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- 172.21.15.157:0/2610327249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- 172.21.15.157:0/2610327249 >> 172.21.15.157:0/2610327249 conn(0x7f33a0065930 msgr2=0x7f33a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- 172.21.15.157:0/2610327249 shutdown_connections 2023-11-06T03:04:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- 172.21.15.157:0/2610327249 wait complete. 2023-11-06T03:04:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 Processor -- start 2023-11-06T03:04:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- start start 2023-11-06T03:04:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a00b3b20 0x7f33a014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a0152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0150680 con 0x7f33a015e5e0 2023-11-06T03:04:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01500c0 con 0x7f33a00b3b20 2023-11-06T03:04:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f33a579b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01503a0 con 0x7f33a01551e0 2023-11-06T03:04:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33368/0 (socket says 172.21.15.157:33368) 2023-11-06T03:04:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339f7fe640 1 -- 172.21.15.157:0/3133321467 learned_addr learned my addr 172.21.15.157:0/3133321467 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.829+0000 7f339effd640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a00b3b20 0x7f33a014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339e7fc640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a0152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 -- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 msgr2=0x7f33a0152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a0152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 -- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a00b3b20 msgr2=0x7f33a014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a00b3b20 0x7f33a014bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 -- 172.21.15.157:0/3133321467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009a70 con 0x7f33a015e5e0 2023-11-06T03:04:41.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339f7fe640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f339801a4c0 tx=0x7f3398019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980aee30 con 0x7f33a015e5e0 2023-11-06T03:04:41.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398009450 con 0x7f33a015e5e0 2023-11-06T03:04:41.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a014a350 con 0x7f33a015e5e0 2023-11-06T03:04:41.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980b7610 con 0x7f33a015e5e0 2023-11-06T03:04:41.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0148f10 con 0x7f33a015e5e0 2023-11-06T03:04:41.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a014a1b0 con 0x7f33a015e5e0 2023-11-06T03:04:41.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33980b4050 con 0x7f33a015e5e0 2023-11-06T03:04:41.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 0x7f338c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:41.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f339811e5c0 con 0x7f33a015e5e0 2023-11-06T03:04:41.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339effd640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 0x7f338c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:41.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.833+0000 7f339effd640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 0x7f338c067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f33a015a9c0 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:41.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.837+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33a014a1b0 con 0x7f33a015e5e0 2023-11-06T03:04:41.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.985+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33a014a1b0 con 0x7f33a015e5e0 2023-11-06T03:04:41.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.985+0000 7f33877fe640 1 -- 172.21.15.157:0/3133321467 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33a014a1b0 con 0x7f33a015e5e0 2023-11-06T03:04:41.986 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:41.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 msgr2=0x7f338c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 0x7f338c067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f33a015a9c0 tx=0x7f3390008040 comp rx=0 tx=0).stop 2023-11-06T03:04:41.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 msgr2=0x7f33a014d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:41.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f339801a4c0 tx=0x7f3398019630 comp rx=0 tx=0).stop 2023-11-06T03:04:41.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 shutdown_connections 2023-11-06T03:04:41.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f338c065a60 0x7f338c067f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a015e5e0 0x7f33a014d670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33a01551e0 0x7f33a0152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 --2- 172.21.15.157:0/3133321467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33a00b3b20 0x7f33a014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:41.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 >> 172.21.15.157:0/3133321467 conn(0x7f33a0065930 msgr2=0x7f33a015dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:41.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 shutdown_connections 2023-11-06T03:04:41.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:41.989+0000 7f33a579b640 1 -- 172.21.15.157:0/3133321467 wait complete. 2023-11-06T03:04:42.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:42.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.113+0000 7fb3309db640 1 Processor -- start 2023-11-06T03:04:42.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.113+0000 7fb3309db640 1 -- start start 2023-11-06T03:04:42.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb3309db640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb3309db640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 0x7fb32c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb3309db640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c0cc2b0 con 0x7fb32c0afb00 2023-11-06T03:04:42.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb3309db640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0b01a0 con 0x7fb32c0cf560 2023-11-06T03:04:42.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb3309db640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0cfb10 con 0x7fb32c15ac80 2023-11-06T03:04:42.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32ad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 0x7fb32c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39732/0 (socket says 172.21.15.157:39732) 2023-11-06T03:04:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32ad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 0x7fb32c0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:45150/0 (socket says 172.21.15.157:45150) 2023-11-06T03:04:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 -- 172.21.15.157:0/1784250476 learned_addr learned my addr 172.21.15.157:0/1784250476 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb329d9b640 1 --1- 172.21.15.157:0/1784250476 >> v1:172.21.15.195:6789/0 conn(0x7fb32c0afb00 0x7fb32c0cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56932/0 (socket says 172.21.15.157:56932) 2023-11-06T03:04:42.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 -- 172.21.15.157:0/1784250476 >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 msgr2=0x7fb32c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:42.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 --2- 172.21.15.157:0/1784250476 >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 0x7fb32c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 -- 172.21.15.157:0/1784250476 >> v1:172.21.15.195:6789/0 conn(0x7fb32c0afb00 legacy=0x7fb32c0cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 -- 172.21.15.157:0/1784250476 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0cfdf0 con 0x7fb32c15ac80 2023-11-06T03:04:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32a59c640 1 --2- 172.21.15.157:0/1784250476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009870 tx=0x7fb31c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717de0aab85bed71 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32959a640 1 -- 172.21.15.157:0/1784250476 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0ab650 con 0x7fb32c15ac80 2023-11-06T03:04:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32959a640 1 -- 172.21.15.157:0/1784250476 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c0ab7b0 con 0x7fb32c15ac80 2023-11-06T03:04:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.117+0000 7fb32959a640 1 -- 172.21.15.157:0/1784250476 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0abb40 con 0x7fb32c15ac80 2023-11-06T03:04:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- 172.21.15.157:0/1784250476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 msgr2=0x7fb32c0b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- 172.21.15.157:0/1784250476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009870 tx=0x7fb31c01c540 comp rx=0 tx=0).stop 2023-11-06T03:04:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- 172.21.15.157:0/1784250476 shutdown_connections 2023-11-06T03:04:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- 172.21.15.157:0/1784250476 >> v2:172.21.15.195:3300/0 conn(0x7fb32c0cf560 0x7fb32c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- 172.21.15.157:0/1784250476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c0b2400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- 172.21.15.157:0/1784250476 >> 172.21.15.157:0/1784250476 conn(0x7fb32c06d560 msgr2=0x7fb32c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- 172.21.15.157:0/1784250476 shutdown_connections 2023-11-06T03:04:42.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- 172.21.15.157:0/1784250476 wait complete. 2023-11-06T03:04:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 Processor -- start 2023-11-06T03:04:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- start start 2023-11-06T03:04:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb32c0afb00 0x7fb32c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c149e10 con 0x7fb32c0cf560 2023-11-06T03:04:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0c6940 con 0x7fb32c0afb00 2023-11-06T03:04:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb3309db640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0c6c20 con 0x7fb32c15ac80 2023-11-06T03:04:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33380/0 (socket says 172.21.15.157:33380) 2023-11-06T03:04:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 -- 172.21.15.157:0/1085304201 learned_addr learned my addr 172.21.15.157:0/1085304201 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb32a59c640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb32c0afb00 0x7fb32c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb32ad9d640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 -- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 msgr2=0x7fb32c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 -- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb32c0afb00 msgr2=0x7fb32c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb32c0afb00 0x7fb32c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.121+0000 7fb329d9b640 1 -- 172.21.15.157:0/1085304201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0094d0 con 0x7fb32c0cf560 2023-11-06T03:04:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb329d9b640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c370 tx=0x7fb32000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200074e0 con 0x7fb32c0cf560 2023-11-06T03:04:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320007640 con 0x7fb32c0cf560 2023-11-06T03:04:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c0c6f00 con 0x7fb32c0cf560 2023-11-06T03:04:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320004a30 con 0x7fb32c0cf560 2023-11-06T03:04:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c15b100 con 0x7fb32c0cf560 2023-11-06T03:04:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c15b290 con 0x7fb32c0cf560 2023-11-06T03:04:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb320004b90 con 0x7fb32c0cf560 2023-11-06T03:04:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 0x7fb30c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb320082bf0 con 0x7fb32c0cf560 2023-11-06T03:04:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb32a59c640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 0x7fb30c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.125+0000 7fb32a59c640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 0x7fb30c067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0af000 tx=0x7fb31c0b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.129+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb320050450 con 0x7fb32c0cf560 2023-11-06T03:04:42.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.273+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb32c072750 con 0x7fb32c0cf560 2023-11-06T03:04:42.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb31affd640 1 -- 172.21.15.157:0/1085304201 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7fb320053a70 con 0x7fb32c0cf560 2023-11-06T03:04:42.276 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:42.277 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:42.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 msgr2=0x7fb30c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 0x7fb30c067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0af000 tx=0x7fb31c0b6040 comp rx=0 tx=0).stop 2023-11-06T03:04:42.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 msgr2=0x7fb32c0c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c370 tx=0x7fb32000c830 comp rx=0 tx=0).stop 2023-11-06T03:04:42.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 shutdown_connections 2023-11-06T03:04:42.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb30c065a60 0x7fb30c067f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb32c15ac80 0x7fb32c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb32c0cf560 0x7fb32c0c6260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 --2- 172.21.15.157:0/1085304201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb32c0afb00 0x7fb32c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.277+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 >> 172.21.15.157:0/1085304201 conn(0x7fb32c06d560 msgr2=0x7fb32c0b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.281+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 shutdown_connections 2023-11-06T03:04:42.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.281+0000 7fb3309db640 1 -- 172.21.15.157:0/1085304201 wait complete. 2023-11-06T03:04:42.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:42.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:42.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 Processor -- start 2023-11-06T03:04:42.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 -- start start 2023-11-06T03:04:42.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7b81599c0 0x7fd7b80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd7b815dea0 0x7fd7b80b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b815e440 con 0x7fd7b81599c0 2023-11-06T03:04:42.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b806c240 con 0x7fd7b815dea0 2023-11-06T03:04:42.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7bdea7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b806c520 con 0x7fd7b806d000 2023-11-06T03:04:42.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7b81599c0 0x7fd7b80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7b81599c0 0x7fd7b80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33392/0 (socket says 172.21.15.157:33392) 2023-11-06T03:04:42.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 -- 172.21.15.157:0/2729290740 learned_addr learned my addr 172.21.15.157:0/2729290740 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b7fff640 1 --2- 172.21.15.157:0/2729290740 >> v2:172.21.15.195:3300/0 conn(0x7fd7b815dea0 0x7fd7b80b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b6ffd640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 -- 172.21.15.157:0/2729290740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 msgr2=0x7fd7b80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:42.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b80b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 -- 172.21.15.157:0/2729290740 >> v2:172.21.15.195:3300/0 conn(0x7fd7b815dea0 msgr2=0x7fd7b80b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:42.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/2729290740 >> v2:172.21.15.195:3300/0 conn(0x7fd7b815dea0 0x7fd7b80b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.417+0000 7fd7b77fe640 1 -- 172.21.15.157:0/2729290740 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b806c800 con 0x7fd7b81599c0 2023-11-06T03:04:42.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ae000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7b806bbf0 tx=0x7fd7a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9d140b92ad1e8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b67fc640 1 -- 172.21.15.157:0/2729290740 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80aa610 con 0x7fd7b81599c0 2023-11-06T03:04:42.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b67fc640 1 -- 172.21.15.157:0/2729290740 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a80a6040 con 0x7fd7b81599c0 2023-11-06T03:04:42.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b67fc640 1 -- 172.21.15.157:0/2729290740 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80aa990 con 0x7fd7b81599c0 2023-11-06T03:04:42.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- 172.21.15.157:0/2729290740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 msgr2=0x7fd7b80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ae000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7b806bbf0 tx=0x7fd7a801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- 172.21.15.157:0/2729290740 shutdown_connections 2023-11-06T03:04:42.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/2729290740 >> v2:172.21.15.195:3300/0 conn(0x7fd7b815dea0 0x7fd7b80b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b80b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/2729290740 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ae000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- 172.21.15.157:0/2729290740 >> 172.21.15.157:0/2729290740 conn(0x7fd7b8065930 msgr2=0x7fd7b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- 172.21.15.157:0/2729290740 shutdown_connections 2023-11-06T03:04:42.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- 172.21.15.157:0/2729290740 wait complete. 2023-11-06T03:04:42.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 Processor -- start 2023-11-06T03:04:42.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- start start 2023-11-06T03:04:42.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b81465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 0x7fd7b815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80ce7e0 con 0x7fd7b81599c0 2023-11-06T03:04:42.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cde10 con 0x7fd7b815dea0 2023-11-06T03:04:42.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7bdea7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cd200 con 0x7fd7b806d000 2023-11-06T03:04:42.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33408/0 (socket says 172.21.15.157:33408) 2023-11-06T03:04:42.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 -- 172.21.15.157:0/967248146 learned_addr learned my addr 172.21.15.157:0/967248146 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b81465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b7fff640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 0x7fd7b815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 -- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 msgr2=0x7fd7b81465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b81465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 -- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 msgr2=0x7fd7b815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 0x7fd7b815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 -- 172.21.15.157:0/967248146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80094d0 con 0x7fd7b81599c0 2023-11-06T03:04:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b7fff640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 0x7fd7b815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b81465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.421+0000 7fd7b6ffd640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00d830 tx=0x7fd7ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00b6c0 con 0x7fd7b81599c0 2023-11-06T03:04:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac00b820 con 0x7fd7b81599c0 2023-11-06T03:04:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80cd4e0 con 0x7fd7b81599c0 2023-11-06T03:04:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00bb90 con 0x7fd7b81599c0 2023-11-06T03:04:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80cdaa0 con 0x7fd7b81599c0 2023-11-06T03:04:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8072750 con 0x7fd7b81599c0 2023-11-06T03:04:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7ac01e460 con 0x7fd7b81599c0 2023-11-06T03:04:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 0x7fd7a4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7ac0839c0 con 0x7fd7b81599c0 2023-11-06T03:04:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 0x7fd7a4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.425+0000 7fd7b77fe640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 0x7fd7a4067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80ae000 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.429+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7ac0512b0 con 0x7fd7b81599c0 2023-11-06T03:04:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.581+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7b80cb110 con 0x7fd7b81599c0 2023-11-06T03:04:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd79ffff640 1 -- 172.21.15.157:0/967248146 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7ac0548d0 con 0x7fd7b81599c0 2023-11-06T03:04:42.583 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 msgr2=0x7fd7a4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 0x7fd7a4067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80ae000 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 msgr2=0x7fd7b80ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00d830 tx=0x7fd7ac00dcf0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 shutdown_connections 2023-11-06T03:04:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a90 0x7fd7a4067f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7b815dea0 0x7fd7b815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7b81599c0 0x7fd7b80ce2a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 --2- 172.21.15.157:0/967248146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7b806d000 0x7fd7b81465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 >> 172.21.15.157:0/967248146 conn(0x7fd7b8065930 msgr2=0x7fd7b80cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 shutdown_connections 2023-11-06T03:04:42.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.585+0000 7fd7bdea7640 1 -- 172.21.15.157:0/967248146 wait complete. 2023-11-06T03:04:42.602 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 Processor -- start 2023-11-06T03:04:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 -- start start 2023-11-06T03:04:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9b4146370 0x7fe9b40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe9b406bdb0 0x7fe9b40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b40cf1a0 con 0x7fe9b40ce800 2023-11-06T03:04:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b406c3d0 con 0x7fe9b4146370 2023-11-06T03:04:42.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9bbb14640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b406f5a0 con 0x7fe9b406bdb0 2023-11-06T03:04:42.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9b4146370 0x7fe9b40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9b4146370 0x7fe9b40b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33412/0 (socket says 172.21.15.157:33412) 2023-11-06T03:04:42.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 -- 172.21.15.157:0/936902342 learned_addr learned my addr 172.21.15.157:0/936902342 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 -- 172.21.15.157:0/936902342 >> v2:172.21.15.195:3300/0 conn(0x7fe9b406bdb0 msgr2=0x7fe9b40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9ba0b1640 1 --2- 172.21.15.157:0/936902342 >> v2:172.21.15.195:3300/0 conn(0x7fe9b406bdb0 0x7fe9b40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 --2- 172.21.15.157:0/936902342 >> v2:172.21.15.195:3300/0 conn(0x7fe9b406bdb0 0x7fe9b40cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 -- 172.21.15.157:0/936902342 >> v1:172.21.15.195:6789/0 conn(0x7fe9b40ce800 legacy=0x7fe9b406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.717+0000 7fe9b98b0640 1 -- 172.21.15.157:0/936902342 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b406f880 con 0x7fe9b4146370 2023-11-06T03:04:42.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9b98b0640 1 --2- 172.21.15.157:0/936902342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40b1a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009870 tx=0x7fe9b001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8715994d19ef50c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9b88ae640 1 -- 172.21.15.157:0/936902342 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00ac640 con 0x7fe9b4146370 2023-11-06T03:04:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9b88ae640 1 -- 172.21.15.157:0/936902342 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00ac7a0 con 0x7fe9b4146370 2023-11-06T03:04:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9b88ae640 1 -- 172.21.15.157:0/936902342 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00acb30 con 0x7fe9b4146370 2023-11-06T03:04:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- 172.21.15.157:0/936902342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 msgr2=0x7fe9b40b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/936902342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40b1a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009870 tx=0x7fe9b001c400 comp rx=0 tx=0).stop 2023-11-06T03:04:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- 172.21.15.157:0/936902342 shutdown_connections 2023-11-06T03:04:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/936902342 >> v2:172.21.15.195:3300/0 conn(0x7fe9b406bdb0 0x7fe9b40cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/936902342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40b1a10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- 172.21.15.157:0/936902342 >> 172.21.15.157:0/936902342 conn(0x7fe9b4065930 msgr2=0x7fe9b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- 172.21.15.157:0/936902342 shutdown_connections 2023-11-06T03:04:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- 172.21.15.157:0/936902342 wait complete. 2023-11-06T03:04:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 Processor -- start 2023-11-06T03:04:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 -- start start 2023-11-06T03:04:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9b406bdb0 0x7fe9b4156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.721+0000 7fe9bbb14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b40ce800 0x7fe9b4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40b0200 con 0x7fe9b4146370 2023-11-06T03:04:42.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4127a20 con 0x7fe9b40ce800 2023-11-06T03:04:42.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b4127cc0 con 0x7fe9b406bdb0 2023-11-06T03:04:42.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33420/0 (socket says 172.21.15.157:33420) 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 -- 172.21.15.157:0/3035933498 learned_addr learned my addr 172.21.15.157:0/3035933498 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9b90af640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b40ce800 0x7fe9b4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9b98b0640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9b406bdb0 0x7fe9b4156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 -- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9b406bdb0 msgr2=0x7fe9b4156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9b406bdb0 0x7fe9b4156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 -- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b40ce800 msgr2=0x7fe9b4156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b40ce800 0x7fe9b4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 -- 172.21.15.157:0/3035933498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00094d0 con 0x7fe9b4146370 2023-11-06T03:04:42.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9ba0b1640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00cc30 tx=0x7fe9ac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00ce40 con 0x7fe9b4146370 2023-11-06T03:04:42.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0039a0 con 0x7fe9b4146370 2023-11-06T03:04:42.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b4127fa0 con 0x7fe9b4146370 2023-11-06T03:04:42.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac003d30 con 0x7fe9b4146370 2023-11-06T03:04:42.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b413a8f0 con 0x7fe9b4146370 2023-11-06T03:04:42.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b4072750 con 0x7fe9b4146370 2023-11-06T03:04:42.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9ac003e90 con 0x7fe9b4146370 2023-11-06T03:04:42.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 0x7fe998067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:42.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9ac083630 con 0x7fe9b4146370 2023-11-06T03:04:42.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9b98b0640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 0x7fe998067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:42.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.725+0000 7fe9b98b0640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 0x7fe998067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ae000 tx=0x7fe9b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:42.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.729+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9ac050f20 con 0x7fe9b4146370 2023-11-06T03:04:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9b4128450 con 0x7fe9b4146370 2023-11-06T03:04:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9aa7fc640 1 -- 172.21.15.157:0/3035933498 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9ac01d0a0 con 0x7fe9b4146370 2023-11-06T03:04:42.872 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 msgr2=0x7fe998067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 0x7fe998067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ae000 tx=0x7fe9b0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:42.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 msgr2=0x7fe9b40af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00cc30 tx=0x7fe9ac00a5c0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 shutdown_connections 2023-11-06T03:04:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe998065a60 0x7fe998067f20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9b4146370 0x7fe9b40af0f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9b40ce800 0x7fe9b4156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 --2- 172.21.15.157:0/3035933498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9b406bdb0 0x7fe9b4156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 >> 172.21.15.157:0/3035933498 conn(0x7fe9b4065930 msgr2=0x7fe9b40b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 shutdown_connections 2023-11-06T03:04:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:42.873+0000 7fe9bbb14640 1 -- 172.21.15.157:0/3035933498 wait complete. 2023-11-06T03:04:42.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 Processor -- start 2023-11-06T03:04:43.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 -- start start 2023-11-06T03:04:43.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a001551e0 0x7f3a0006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a0015e5e0 0x7f3a000b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015ea90 0x7f3a000b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0015f580 con 0x7f3a001551e0 2023-11-06T03:04:43.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0015efc0 con 0x7f3a0015e5e0 2023-11-06T03:04:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f3a06002640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0015f2a0 con 0x7f3a0015ea90 2023-11-06T03:04:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a001551e0 0x7f3a0006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a001551e0 0x7f3a0006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33426/0 (socket says 172.21.15.157:33426) 2023-11-06T03:04:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3465430932 learned_addr learned my addr 172.21.15.157:0/3465430932 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:43.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39feffd640 1 --2- 172.21.15.157:0/3465430932 >> v2:172.21.15.195:3300/0 conn(0x7f3a0015e5e0 0x7f3a000b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39fffff640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015ea90 0x7f3a000b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3465430932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015ea90 msgr2=0x7f3a000b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015ea90 0x7f3a000b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3465430932 >> v2:172.21.15.195:3300/0 conn(0x7f3a0015e5e0 msgr2=0x7f3a000b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3465430932 >> v2:172.21.15.195:3300/0 conn(0x7f3a0015e5e0 0x7f3a000b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.005+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3465430932 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0006f5a0 con 0x7f3a001551e0 2023-11-06T03:04:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a0006fda0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3a00066880 tx=0x7f39f001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a28ef3b6241ec5e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f39fe7fc640 1 -- 172.21.15.157:0/3465430932 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00ab620 con 0x7f3a001551e0 2023-11-06T03:04:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f39fe7fc640 1 -- 172.21.15.157:0/3465430932 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f00ab780 con 0x7f3a001551e0 2023-11-06T03:04:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f39fe7fc640 1 -- 172.21.15.157:0/3465430932 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00abb10 con 0x7f3a001551e0 2023-11-06T03:04:43.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- 172.21.15.157:0/3465430932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 msgr2=0x7f3a0006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a0006fda0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3a00066880 tx=0x7f39f001c440 comp rx=0 tx=0).stop 2023-11-06T03:04:43.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- 172.21.15.157:0/3465430932 shutdown_connections 2023-11-06T03:04:43.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015ea90 0x7f3a000b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 --2- 172.21.15.157:0/3465430932 >> v2:172.21.15.195:3300/0 conn(0x7f3a0015e5e0 0x7f3a000b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 --2- 172.21.15.157:0/3465430932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a0006fda0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- 172.21.15.157:0/3465430932 >> 172.21.15.157:0/3465430932 conn(0x7f3a00065930 msgr2=0x7f3a00065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- 172.21.15.157:0/3465430932 shutdown_connections 2023-11-06T03:04:43.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- 172.21.15.157:0/3465430932 wait complete. 2023-11-06T03:04:43.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 Processor -- start 2023-11-06T03:04:43.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.009+0000 7f3a06002640 1 -- start start 2023-11-06T03:04:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015e5e0 0x7f3a00144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0015ea90 0x7f3a00147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001543f0 con 0x7f3a001551e0 2023-11-06T03:04:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a001450a0 con 0x7f3a0015ea90 2023-11-06T03:04:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a00153770 con 0x7f3a0015e5e0 2023-11-06T03:04:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33430/0 (socket says 172.21.15.157:33430) 2023-11-06T03:04:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3498802824 learned_addr learned my addr 172.21.15.157:0/3498802824 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015e5e0 msgr2=0x7f3a00144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015e5e0 0x7f3a00144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0015ea90 msgr2=0x7f3a00147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0015ea90 0x7f3a00147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 -- 172.21.15.157:0/3498802824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00094d0 con 0x7f3a001551e0 2023-11-06T03:04:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39ff7fe640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39f001aaf0 tx=0x7f39f001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00be3f0 con 0x7f3a001551e0 2023-11-06T03:04:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f00be550 con 0x7f3a001551e0 2023-11-06T03:04:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a00153a50 con 0x7f3a001551e0 2023-11-06T03:04:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00be8e0 con 0x7f3a001551e0 2023-11-06T03:04:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a00154010 con 0x7f3a001551e0 2023-11-06T03:04:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a00072750 con 0x7f3a001551e0 2023-11-06T03:04:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39f00c7440 con 0x7f3a001551e0 2023-11-06T03:04:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 0x7f39ec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39f011dac0 con 0x7f3a001551e0 2023-11-06T03:04:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39feffd640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 0x7f39ec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.013+0000 7f39feffd640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 0x7f39ec067f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.017+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39f00eb4d0 con 0x7f3a001551e0 2023-11-06T03:04:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a00071730 con 0x7f3a001551e0 2023-11-06T03:04:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f39e7fff640 1 -- 172.21.15.157:0/3498802824 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39f00eeaf0 con 0x7f3a001551e0 2023-11-06T03:04:43.160 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:43.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 msgr2=0x7f39ec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 0x7f39ec067f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f4008040 comp rx=0 tx=0).stop 2023-11-06T03:04:43.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 msgr2=0x7f3a001530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39f001aaf0 tx=0x7f39f001ac80 comp rx=0 tx=0).stop 2023-11-06T03:04:43.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 shutdown_connections 2023-11-06T03:04:43.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39ec065a90 0x7f39ec067f50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0015ea90 0x7f3a00147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0015e5e0 0x7f3a00144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 --2- 172.21.15.157:0/3498802824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a001551e0 0x7f3a001530d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 >> 172.21.15.157:0/3498802824 conn(0x7f3a00065930 msgr2=0x7f3a0015b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:43.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 shutdown_connections 2023-11-06T03:04:43.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.161+0000 7f3a06002640 1 -- 172.21.15.157:0/3498802824 wait complete. 2023-11-06T03:04:43.176 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:43.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 Processor -- start 2023-11-06T03:04:43.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 -- start start 2023-11-06T03:04:43.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc228156a70 0x7fc22806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2280b2330 0x7fc22806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22806c490 con 0x7fc2280adae0 2023-11-06T03:04:43.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22806b950 con 0x7fc228156a70 2023-11-06T03:04:43.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22ef6c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22806bb60 con 0x7fc2280b2330 2023-11-06T03:04:43.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22cd08640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc228156a70 0x7fc22806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc227fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc2280adae0 0x7fc2280b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49782/0 (socket says 172.21.15.157:49782) 2023-11-06T03:04:43.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc227fff640 1 -- 172.21.15.157:0/8932680 learned_addr learned my addr 172.21.15.157:0/8932680 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22d509640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2280b2330 0x7fc22806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22cd08640 1 -- 172.21.15.157:0/8932680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2280b2330 msgr2=0x7fc22806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22cd08640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2280b2330 0x7fc22806c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22cd08640 1 -- 172.21.15.157:0/8932680 >> v1:172.21.15.157:6789/0 conn(0x7fc2280adae0 legacy=0x7fc2280b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc22cd08640 1 -- 172.21.15.157:0/8932680 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22806be40 con 0x7fc228156a70 2023-11-06T03:04:43.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.297+0000 7fc2277fe640 1 -- 172.21.15.157:0/8932680 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236075252 0 0) 0x7fc22806c490 con 0x7fc2280adae0 2023-11-06T03:04:43.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22cd08640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc228156a70 0x7fc22806f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc21800a960 tx=0x7fc21801d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e6fe436ae5d54e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc2277fe640 1 -- 172.21.15.157:0/8932680 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180a6910 con 0x7fc228156a70 2023-11-06T03:04:43.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc2277fe640 1 -- 172.21.15.157:0/8932680 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180a6a70 con 0x7fc228156a70 2023-11-06T03:04:43.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc2277fe640 1 -- 172.21.15.157:0/8932680 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180a6e00 con 0x7fc228156a70 2023-11-06T03:04:43.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- 172.21.15.157:0/8932680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc228156a70 msgr2=0x7fc22806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc228156a70 0x7fc22806f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc21800a960 tx=0x7fc21801d820 comp rx=0 tx=0).stop 2023-11-06T03:04:43.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- 172.21.15.157:0/8932680 shutdown_connections 2023-11-06T03:04:43.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2280b2330 0x7fc22806c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/8932680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc228156a70 0x7fc22806f470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- 172.21.15.157:0/8932680 >> 172.21.15.157:0/8932680 conn(0x7fc228065930 msgr2=0x7fc228065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:43.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- 172.21.15.157:0/8932680 shutdown_connections 2023-11-06T03:04:43.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- 172.21.15.157:0/8932680 wait complete. 2023-11-06T03:04:43.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 Processor -- start 2023-11-06T03:04:43.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.301+0000 7fc22ef6c640 1 -- start start 2023-11-06T03:04:43.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2280b2330 0x7fc228154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc228156a70 0x7fc22813db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228153b20 con 0x7fc2280adae0 2023-11-06T03:04:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228153e00 con 0x7fc2280b2330 2023-11-06T03:04:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281540e0 con 0x7fc228156a70 2023-11-06T03:04:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33438/0 (socket says 172.21.15.157:33438) 2023-11-06T03:04:43.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 -- 172.21.15.157:0/1704800960 learned_addr learned my addr 172.21.15.157:0/1704800960 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22d509640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc228156a70 0x7fc22813db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc227fff640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2280b2330 0x7fc228154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 -- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc228156a70 msgr2=0x7fc22813db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc228156a70 0x7fc22813db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 -- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2280b2330 msgr2=0x7fc228154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2280b2330 0x7fc228154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 -- 172.21.15.157:0/1704800960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21800a5c0 con 0x7fc2280adae0 2023-11-06T03:04:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22cd08640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b1040 tx=0x7fc21801a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180ae990 con 0x7fc2280adae0 2023-11-06T03:04:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180aeb10 con 0x7fc2280adae0 2023-11-06T03:04:43.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2281543c0 con 0x7fc2280adae0 2023-11-06T03:04:43.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180b7430 con 0x7fc2280adae0 2023-11-06T03:04:43.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc228156060 con 0x7fc2280adae0 2023-11-06T03:04:43.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc228072750 con 0x7fc2280adae0 2023-11-06T03:04:43.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc2180b5040 con 0x7fc2280adae0 2023-11-06T03:04:43.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 0x7fc214067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc21811d8b0 con 0x7fc2280adae0 2023-11-06T03:04:43.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.305+0000 7fc227fff640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 0x7fc214067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.309+0000 7fc227fff640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 0x7fc214067f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000f70 tx=0x7fc21c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.309+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc2180eb110 con 0x7fc2280adae0 2023-11-06T03:04:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc228070690 con 0x7fc2280adae0 2023-11-06T03:04:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc2257fa640 1 -- 172.21.15.157:0/1704800960 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2180ee730 con 0x7fc2280adae0 2023-11-06T03:04:43.452 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 msgr2=0x7fc214067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 0x7fc214067f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000f70 tx=0x7fc21c009040 comp rx=0 tx=0).stop 2023-11-06T03:04:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 msgr2=0x7fc228155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b1040 tx=0x7fc21801a5c0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 shutdown_connections 2023-11-06T03:04:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc214065a60 0x7fc214067f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc228156a70 0x7fc22813db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2280b2330 0x7fc228154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 --2- 172.21.15.157:0/1704800960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2280adae0 0x7fc228155b20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 >> 172.21.15.157:0/1704800960 conn(0x7fc228065930 msgr2=0x7fc2281523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 shutdown_connections 2023-11-06T03:04:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.453+0000 7fc22ef6c640 1 -- 172.21.15.157:0/1704800960 wait complete. 2023-11-06T03:04:43.486 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2023-11-06T03:04:43.487 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-06T03:04:43.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-11-06T03:04:43.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 Processor -- start 2023-11-06T03:04:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 -- start start 2023-11-06T03:04:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f895c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c15e440 con 0x7f895c0b2af0 2023-11-06T03:04:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c06c240 con 0x7f895c1599c0 2023-11-06T03:04:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8963393640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c06c520 con 0x7f895c06d000 2023-11-06T03:04:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f896092e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f895c0b2af0 0x7f895c159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49794/0 (socket says 172.21.15.157:49794) 2023-11-06T03:04:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f896092e640 1 -- 172.21.15.157:0/3612712063 learned_addr learned my addr 172.21.15.157:0/3612712063 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f895c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864003251 0 0) 0x7f895c15e440 con 0x7f895c0b2af0 2023-11-06T03:04:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8948003530 con 0x7f895c0b2af0 2023-11-06T03:04:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1943239359 0 0) 0x7f8948003530 con 0x7f895c0b2af0 2023-11-06T03:04:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 msgr2=0x7f895c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f895c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 >> v1:172.21.15.195:6789/0 conn(0x7f895c1599c0 legacy=0x7f895c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c06c800 con 0x7f895c0b2af0 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f894c001680 con 0x7f895c0b2af0 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 >> v1:172.21.15.157:6789/0 conn(0x7f895c0b2af0 legacy=0x7f895c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f89480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8948003f30 0x7f89480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8948004d10 0x7f8948005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c06c520 con 0x7f8948004d10 2023-11-06T03:04:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894800ba70 con 0x7f8948003f30 2023-11-06T03:04:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894800bc30 con 0x7f895c06d000 2023-11-06T03:04:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8948004d10 0x7f8948005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f896112f640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f89480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 -- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 msgr2=0x7f89480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c06d000 0x7f89480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 -- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8948003f30 msgr2=0x7f89480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f896092e640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8948003f30 0x7f89480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8948003f30 0x7f89480047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.609+0000 7f8961930640 1 -- 172.21.15.157:0/3612712063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948003530 con 0x7f8948004d10 2023-11-06T03:04:43.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8961930640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8948004d10 0x7f8948005190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8954009ab0 tx=0x7f895401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152847763a36a31c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540a5620 con 0x7f8948004d10 2023-11-06T03:04:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89540a5780 con 0x7f8948004d10 2023-11-06T03:04:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8953fff640 1 -- 172.21.15.157:0/3612712063 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540a5b30 con 0x7f8948004d10 2023-11-06T03:04:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- 172.21.15.157:0/3612712063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8948004d10 msgr2=0x7f8948005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 --2- 172.21.15.157:0/3612712063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8948004d10 0x7f8948005190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8954009ab0 tx=0x7f895401ca30 comp rx=0 tx=0).stop 2023-11-06T03:04:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8961930640 1 -- 172.21.15.157:0/3612712063 reap_dead start 2023-11-06T03:04:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- 172.21.15.157:0/3612712063 shutdown_connections 2023-11-06T03:04:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- 172.21.15.157:0/3612712063 >> 172.21.15.157:0/3612712063 conn(0x7f895c065930 msgr2=0x7f895c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- 172.21.15.157:0/3612712063 shutdown_connections 2023-11-06T03:04:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- 172.21.15.157:0/3612712063 wait complete. 2023-11-06T03:04:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 Processor -- start 2023-11-06T03:04:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- start start 2023-11-06T03:04:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f895c0aec30 0x7f895c1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c0cc420 0x7f895c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0a9760 con 0x7f895c0af010 2023-11-06T03:04:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0aa0c0 con 0x7f895c0aec30 2023-11-06T03:04:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f8963393640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0ccb90 con 0x7f895c0cc420 2023-11-06T03:04:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f896092e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f896092e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33458/0 (socket says 172.21.15.157:33458) 2023-11-06T03:04:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f896092e640 1 -- 172.21.15.157:0/105956073 learned_addr learned my addr 172.21.15.157:0/105956073 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.613+0000 7f896112f640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f895c0aec30 0x7f895c1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 -- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c0cc420 msgr2=0x7f895c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c0cc420 0x7f895c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 -- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f895c0aec30 msgr2=0x7f895c1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f895c0aec30 0x7f895c1f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 -- 172.21.15.157:0/105956073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0cc8d0 con 0x7f895c0af010 2023-11-06T03:04:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896092e640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f894c00d8b0 tx=0x7f894c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0074e0 con 0x7f895c0af010 2023-11-06T03:04:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c007640 con 0x7f895c0af010 2023-11-06T03:04:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c0a8dc0 con 0x7f895c0af010 2023-11-06T03:04:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0049f0 con 0x7f895c0af010 2023-11-06T03:04:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0a9380 con 0x7f895c0af010 2023-11-06T03:04:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c0a98a0 con 0x7f895c0af010 2023-11-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f894c004b50 con 0x7f895c0af010 2023-11-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 0x7f8940067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f894c082ba0 con 0x7f895c0af010 2023-11-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896112f640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 0x7f8940067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.617+0000 7f896112f640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 0x7f8940067f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8958001ca0 tx=0x7f8958009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:43.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.621+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f895c0a98a0 con 0x7f895c0af010 2023-11-06T03:04:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:43.817+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f895c0a98a0 con 0x7f895c0af010 2023-11-06T03:04:44.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.505+0000 7f8951ffb640 1 -- 172.21.15.157:0/105956073 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v58) v1 ==== 151+0+0 (secure 0 0 0) 0x7f895c0a98a0 con 0x7f895c0af010 2023-11-06T03:04:44.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 msgr2=0x7f8940067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 0x7f8940067f50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8958001ca0 tx=0x7f8958009040 comp rx=0 tx=0).stop 2023-11-06T03:04:44.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 msgr2=0x7f895c0a9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f894c00d8b0 tx=0x7f894c00dd70 comp rx=0 tx=0).stop 2023-11-06T03:04:44.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 shutdown_connections 2023-11-06T03:04:44.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8940065a90 0x7f8940067f50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f895c0cc420 0x7f895c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f895c0af010 0x7f895c0a9b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 --2- 172.21.15.157:0/105956073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f895c0aec30 0x7f895c1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 >> 172.21.15.157:0/105956073 conn(0x7f895c065930 msgr2=0x7f895c0afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:44.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 shutdown_connections 2023-11-06T03:04:44.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.509+0000 7f8963393640 1 -- 172.21.15.157:0/105956073 wait complete. 2023-11-06T03:04:44.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:04:44.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 Processor -- start 2023-11-06T03:04:44.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 -- start start 2023-11-06T03:04:44.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f82f4065d40 0x7f82f40661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40cef40 con 0x7f82f4144430 2023-11-06T03:04:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40cd450 con 0x7f82f4065d40 2023-11-06T03:04:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82fa0df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40cc840 con 0x7f82f415b280 2023-11-06T03:04:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f82f4065d40 0x7f82f40661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f82f4065d40 0x7f82f40661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33474/0 (socket says 172.21.15.157:33474) 2023-11-06T03:04:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 -- 172.21.15.157:0/3948491451 learned_addr learned my addr 172.21.15.157:0/3948491451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f82f4144430 0x7f82f40b0ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49808/0 (socket says 172.21.15.157:49808) 2023-11-06T03:04:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f3fff640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 -- 172.21.15.157:0/3948491451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 msgr2=0x7f82f415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f415b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 -- 172.21.15.157:0/3948491451 >> v1:172.21.15.157:6789/0 conn(0x7f82f4144430 legacy=0x7f82f40b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f2ffd640 1 -- 172.21.15.157:0/3948491451 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4066700 con 0x7f82f4065d40 2023-11-06T03:04:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.665+0000 7f82f27fc640 1 -- 172.21.15.157:0/3948491451 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287910405 0 0) 0x7f82f40cef40 con 0x7f82f4144430 2023-11-06T03:04:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4065d40 0x7f82f40661c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009870 tx=0x7f82e801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2ad4170eeb7f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f27fc640 1 -- 172.21.15.157:0/3948491451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80a7070 con 0x7f82f4065d40 2023-11-06T03:04:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f27fc640 1 -- 172.21.15.157:0/3948491451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e80ab430 con 0x7f82f4065d40 2023-11-06T03:04:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f27fc640 1 -- 172.21.15.157:0/3948491451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e80ab7c0 con 0x7f82f4065d40 2023-11-06T03:04:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- 172.21.15.157:0/3948491451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4065d40 msgr2=0x7f82f40661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4065d40 0x7f82f40661c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009870 tx=0x7f82e801c540 comp rx=0 tx=0).stop 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- 172.21.15.157:0/3948491451 shutdown_connections 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- 172.21.15.157:0/3948491451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4065d40 0x7f82f40661c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- 172.21.15.157:0/3948491451 >> 172.21.15.157:0/3948491451 conn(0x7f82f4065930 msgr2=0x7f82f4071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- 172.21.15.157:0/3948491451 shutdown_connections 2023-11-06T03:04:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- 172.21.15.157:0/3948491451 wait complete. 2023-11-06T03:04:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 Processor -- start 2023-11-06T03:04:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- start start 2023-11-06T03:04:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 0x7f82f415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f4137a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f415aa20 con 0x7f82f4144430 2023-11-06T03:04:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4141550 con 0x7f82f4065d40 2023-11-06T03:04:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82fa0df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4141830 con 0x7f82f415b280 2023-11-06T03:04:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33478/0 (socket says 172.21.15.157:33478) 2023-11-06T03:04:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 -- 172.21.15.157:0/1991799895 learned_addr learned my addr 172.21.15.157:0/1991799895 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f37fe640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 0x7f82f415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f3fff640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f4137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 -- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 msgr2=0x7f82f4137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f4137a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 -- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 msgr2=0x7f82f415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 0x7f82f415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f2ffd640 1 -- 172.21.15.157:0/1991799895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80094d0 con 0x7f82f4144430 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.669+0000 7f82f37fe640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 0x7f82f415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82f2ffd640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82e801a170 tx=0x7f82e801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004220 con 0x7f82f4144430 2023-11-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8004380 con 0x7f82f4144430 2023-11-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f4141b10 con 0x7f82f4144430 2023-11-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8003820 con 0x7f82f4144430 2023-11-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f41420d0 con 0x7f82f4144430 2023-11-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f41422c0 con 0x7f82f4144430 2023-11-06T03:04:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f82e8003980 con 0x7f82f4144430 2023-11-06T03:04:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 0x7f82e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82e811de80 con 0x7f82f4144430 2023-11-06T03:04:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.673+0000 7f82f37fe640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 0x7f82e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.677+0000 7f82f37fe640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 0x7f82e0067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f70 tx=0x7f82e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.677+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f82e80eb7c0 con 0x7f82f4144430 2023-11-06T03:04:44.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.833+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f82f4072670 con 0x7f82f4144430 2023-11-06T03:04:44.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.837+0000 7f82dbfff640 1 -- 172.21.15.157:0/1991799895 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82e80044f0 con 0x7f82f4144430 2023-11-06T03:04:44.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 msgr2=0x7f82e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 0x7f82e0067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f70 tx=0x7f82e4009040 comp rx=0 tx=0).stop 2023-11-06T03:04:44.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 msgr2=0x7f82f4140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:44.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82e801a170 tx=0x7f82e801ad80 comp rx=0 tx=0).stop 2023-11-06T03:04:44.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 shutdown_connections 2023-11-06T03:04:44.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f82e0065a60 0x7f82e0067f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82f415b280 0x7f82f4137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f4144430 0x7f82f4140b00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 --2- 172.21.15.157:0/1991799895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f4065d40 0x7f82f415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:44.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 >> 172.21.15.157:0/1991799895 conn(0x7f82f4065930 msgr2=0x7f82f413eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:44.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 shutdown_connections 2023-11-06T03:04:44.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:44.841+0000 7f82fa0df640 1 -- 172.21.15.157:0/1991799895 wait complete. 2023-11-06T03:04:44.856 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:04:44.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:04:44.920 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:04:44.966 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:04:44.966 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:44.967 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:44.972 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:44.973 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:45.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:45.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 Processor -- start 2023-11-06T03:04:45.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 -- start start 2023-11-06T03:04:45.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79b015caa0 0x7f79b00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79b015ce80 0x7f79b00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00d2920 con 0x7f79b00ce1a0 2023-11-06T03:04:45.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00d18a0 con 0x7f79b015caa0 2023-11-06T03:04:45.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79b587c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00d1b60 con 0x7f79b015ce80 2023-11-06T03:04:45.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79b015caa0 0x7f79b00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79af7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f79b00ce1a0 0x7f79b00d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49810/0 (socket says 172.21.15.157:49810) 2023-11-06T03:04:45.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79b015caa0 0x7f79b00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33492/0 (socket says 172.21.15.157:33492) 2023-11-06T03:04:45.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79af7fe640 1 -- 172.21.15.157:0/705123788 learned_addr learned my addr 172.21.15.157:0/705123788 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:45.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79ae7fc640 1 --2- 172.21.15.157:0/705123788 >> v2:172.21.15.195:3300/0 conn(0x7f79b015ce80 0x7f79b00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 -- 172.21.15.157:0/705123788 >> v2:172.21.15.195:3300/0 conn(0x7f79b015ce80 msgr2=0x7f79b00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:45.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 --2- 172.21.15.157:0/705123788 >> v2:172.21.15.195:3300/0 conn(0x7f79b015ce80 0x7f79b00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 -- 172.21.15.157:0/705123788 >> v1:172.21.15.157:6789/0 conn(0x7f79b00ce1a0 legacy=0x7f79b00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.185+0000 7f79aeffd640 1 -- 172.21.15.157:0/705123788 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00d1e40 con 0x7f79b015caa0 2023-11-06T03:04:45.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79aeffd640 1 --2- 172.21.15.157:0/705123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b00cdc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c6720 tx=0x7f79a001c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8391fe8fbda3e82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79adffb640 1 -- 172.21.15.157:0/705123788 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab600 con 0x7f79b015caa0 2023-11-06T03:04:45.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79adffb640 1 -- 172.21.15.157:0/705123788 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00a7050 con 0x7f79b015caa0 2023-11-06T03:04:45.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79adffb640 1 -- 172.21.15.157:0/705123788 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ab980 con 0x7f79b015caa0 2023-11-06T03:04:45.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- 172.21.15.157:0/705123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 msgr2=0x7f79b00cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- 172.21.15.157:0/705123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b00cdc60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c6720 tx=0x7f79a001c2f0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- 172.21.15.157:0/705123788 shutdown_connections 2023-11-06T03:04:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- 172.21.15.157:0/705123788 >> v2:172.21.15.195:3300/0 conn(0x7f79b015ce80 0x7f79b00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- 172.21.15.157:0/705123788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b00cdc60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- 172.21.15.157:0/705123788 >> 172.21.15.157:0/705123788 conn(0x7f79b0070790 msgr2=0x7f79b0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- 172.21.15.157:0/705123788 shutdown_connections 2023-11-06T03:04:45.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- 172.21.15.157:0/705123788 wait complete. 2023-11-06T03:04:45.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 Processor -- start 2023-11-06T03:04:45.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- start start 2023-11-06T03:04:45.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 0x7f79b01585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b015ce80 0x7f79b0157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00cc790 con 0x7f79b015caa0 2023-11-06T03:04:45.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01e2220 con 0x7f79b00ce1a0 2023-11-06T03:04:45.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79b587c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01e2500 con 0x7f79b015ce80 2023-11-06T03:04:45.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33496/0 (socket says 172.21.15.157:33496) 2023-11-06T03:04:45.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 -- 172.21.15.157:0/565898411 learned_addr learned my addr 172.21.15.157:0/565898411 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:45.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79aeffd640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 0x7f79b01585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79af7fe640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b015ce80 0x7f79b0157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 -- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b015ce80 msgr2=0x7f79b0157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b015ce80 0x7f79b0157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 -- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 msgr2=0x7f79b01585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 0x7f79b01585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79ae7fc640 1 -- 172.21.15.157:0/565898411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009420 con 0x7f79b015caa0 2023-11-06T03:04:45.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.189+0000 7f79aeffd640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 0x7f79b01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:45.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79ae7fc640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d7d0 tx=0x7f79a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400b6c0 con 0x7f79b015caa0 2023-11-06T03:04:45.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400b820 con 0x7f79b015caa0 2023-11-06T03:04:45.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01e27e0 con 0x7f79b015caa0 2023-11-06T03:04:45.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400bbd0 con 0x7f79b015caa0 2023-11-06T03:04:45.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0158b20 con 0x7f79b015caa0 2023-11-06T03:04:45.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b0072750 con 0x7f79b015caa0 2023-11-06T03:04:45.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f79a400bd30 con 0x7f79b015caa0 2023-11-06T03:04:45.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 0x7f799c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79a4083800 con 0x7f79b015caa0 2023-11-06T03:04:45.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79aeffd640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 0x7f799c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.193+0000 7f79aeffd640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 0x7f799c067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.197+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f79a40510f0 con 0x7f79b015caa0 2023-11-06T03:04:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.349+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b00cde00 con 0x7f79b015caa0 2023-11-06T03:04:45.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.353+0000 7f7997fff640 1 -- 172.21.15.157:0/565898411 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f79a4054710 con 0x7f79b015caa0 2023-11-06T03:04:45.353 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:45.353 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:45.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 msgr2=0x7f799c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 0x7f799c067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79a00ad000 tx=0x7f79a0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:45.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 msgr2=0x7f79b01e48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d7d0 tx=0x7f79a400dc90 comp rx=0 tx=0).stop 2023-11-06T03:04:45.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 shutdown_connections 2023-11-06T03:04:45.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f799c065a60 0x7f799c067f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b015ce80 0x7f79b0157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b015caa0 0x7f79b01e48e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 --2- 172.21.15.157:0/565898411 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ce1a0 0x7f79b01585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 >> 172.21.15.157:0/565898411 conn(0x7f79b0070790 msgr2=0x7f79b0065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 shutdown_connections 2023-11-06T03:04:45.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.357+0000 7f79b587c640 1 -- 172.21.15.157:0/565898411 wait complete. 2023-11-06T03:04:45.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:45.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:45.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 Processor -- start 2023-11-06T03:04:45.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 -- start start 2023-11-06T03:04:45.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30e40b2af0 0x7f30e40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e415e440 con 0x7f30e41599c0 2023-11-06T03:04:45.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e406c240 con 0x7f30e40b2af0 2023-11-06T03:04:45.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30eb87c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e406c520 con 0x7f30e406d000 2023-11-06T03:04:45.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e8e17640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30e40b2af0 0x7f30e40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9618640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f30e41599c0 0x7f30e40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56950/0 (socket says 172.21.15.157:56950) 2023-11-06T03:04:45.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39874/0 (socket says 172.21.15.157:39874) 2023-11-06T03:04:45.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9618640 1 -- 172.21.15.157:0/4071677024 learned_addr learned my addr 172.21.15.157:0/4071677024 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:45.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 -- 172.21.15.157:0/4071677024 >> v2:172.21.15.195:3300/0 conn(0x7f30e40b2af0 msgr2=0x7f30e40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:45.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 --2- 172.21.15.157:0/4071677024 >> v2:172.21.15.195:3300/0 conn(0x7f30e40b2af0 0x7f30e40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 -- 172.21.15.157:0/4071677024 >> v1:172.21.15.195:6789/0 conn(0x7f30e41599c0 legacy=0x7f30e40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.501+0000 7f30e9e19640 1 -- 172.21.15.157:0/4071677024 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e406c800 con 0x7f30e406d000 2023-11-06T03:04:45.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.505+0000 7f30e9e19640 1 --2- 172.21.15.157:0/4071677024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009d90 tx=0x7f30dc01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1b981812d04a2f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.505+0000 7f30dbfff640 1 -- 172.21.15.157:0/4071677024 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0a5a00 con 0x7f30e406d000 2023-11-06T03:04:45.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.505+0000 7f30dbfff640 1 -- 172.21.15.157:0/4071677024 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc0a5b60 con 0x7f30e406d000 2023-11-06T03:04:45.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.505+0000 7f30dbfff640 1 -- 172.21.15.157:0/4071677024 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0ad440 con 0x7f30e406d000 2023-11-06T03:04:45.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- 172.21.15.157:0/4071677024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 msgr2=0x7f30e4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- 172.21.15.157:0/4071677024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009d90 tx=0x7f30dc01c920 comp rx=0 tx=0).stop 2023-11-06T03:04:45.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- 172.21.15.157:0/4071677024 shutdown_connections 2023-11-06T03:04:45.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- 172.21.15.157:0/4071677024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4159da0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- 172.21.15.157:0/4071677024 >> v2:172.21.15.195:3300/0 conn(0x7f30e40b2af0 0x7f30e40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- 172.21.15.157:0/4071677024 >> 172.21.15.157:0/4071677024 conn(0x7f30e4065930 msgr2=0x7f30e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- 172.21.15.157:0/4071677024 shutdown_connections 2023-11-06T03:04:45.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- 172.21.15.157:0/4071677024 wait complete. 2023-11-06T03:04:45.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 Processor -- start 2023-11-06T03:04:45.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- start start 2023-11-06T03:04:45.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30e41599c0 0x7f30e4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ac530 con 0x7f30e40b2af0 2023-11-06T03:04:45.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40cd480 con 0x7f30e41599c0 2023-11-06T03:04:45.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30eb87c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ab870 con 0x7f30e406d000 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33500/0 (socket says 172.21.15.157:33500) 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 -- 172.21.15.157:0/605537772 learned_addr learned my addr 172.21.15.157:0/605537772 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e9e19640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30e41599c0 0x7f30e4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e9618640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 -- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 msgr2=0x7f30e4152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 -- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30e41599c0 msgr2=0x7f30e4152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30e41599c0 0x7f30e4152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e8e17640 1 -- 172.21.15.157:0/605537772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc009a70 con 0x7f30e40b2af0 2023-11-06T03:04:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.509+0000 7f30e9618640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:45.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30e8e17640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30d4010e20 tx=0x7f30d400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400b930 con 0x7f30e40b2af0 2023-11-06T03:04:45.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d4011860 con 0x7f30e40b2af0 2023-11-06T03:04:45.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40ab590 con 0x7f30e40b2af0 2023-11-06T03:04:45.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4011bd0 con 0x7f30e40b2af0 2023-11-06T03:04:45.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e40cd0a0 con 0x7f30e40b2af0 2023-11-06T03:04:45.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4072750 con 0x7f30e40b2af0 2023-11-06T03:04:45.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f30d4011d30 con 0x7f30e40b2af0 2023-11-06T03:04:45.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 0x7f30c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30d4086030 con 0x7f30e40b2af0 2023-11-06T03:04:45.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30e9618640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 0x7f30c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.513+0000 7f30e9618640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 0x7f30c8067f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005890 tx=0x7f30e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.517+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f30d40539d0 con 0x7f30e40b2af0 2023-11-06T03:04:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30e40cb110 con 0x7f30e40b2af0 2023-11-06T03:04:45.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30d9ffb640 1 -- 172.21.15.157:0/605537772 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30d401e0a0 con 0x7f30e40b2af0 2023-11-06T03:04:45.672 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:45.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 msgr2=0x7f30c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 0x7f30c8067f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005890 tx=0x7f30e0009040 comp rx=0 tx=0).stop 2023-11-06T03:04:45.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 msgr2=0x7f30e40a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30d4010e20 tx=0x7f30d400e730 comp rx=0 tx=0).stop 2023-11-06T03:04:45.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 shutdown_connections 2023-11-06T03:04:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f30c8065a60 0x7f30c8067f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30e41599c0 0x7f30e4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30e40b2af0 0x7f30e40a84a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 --2- 172.21.15.157:0/605537772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30e406d000 0x7f30e4152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 >> 172.21.15.157:0/605537772 conn(0x7f30e4065930 msgr2=0x7f30e40b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 shutdown_connections 2023-11-06T03:04:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.673+0000 7f30eb87c640 1 -- 172.21.15.157:0/605537772 wait complete. 2023-11-06T03:04:45.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 Processor -- start 2023-11-06T03:04:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 -- start start 2023-11-06T03:04:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0a015caa0 0x7fe0a00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a015ce80 0x7fe0a00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe0a00ce1a0 0x7fe0a00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00d2920 con 0x7fe0a015caa0 2023-11-06T03:04:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00d18a0 con 0x7fe0a00ce1a0 2023-11-06T03:04:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a7dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00d1b60 con 0x7fe0a015ce80 2023-11-06T03:04:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0a015caa0 0x7fe0a00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0a015caa0 0x7fe0a00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33512/0 (socket says 172.21.15.157:33512) 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 -- 172.21.15.157:0/3554070660 learned_addr learned my addr 172.21.15.157:0/3554070660 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a6363640 1 --2- 172.21.15.157:0/3554070660 >> v2:172.21.15.195:3300/0 conn(0x7fe0a00ce1a0 0x7fe0a00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5361640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a015ce80 0x7fe0a00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 -- 172.21.15.157:0/3554070660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a015ce80 msgr2=0x7fe0a00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a015ce80 0x7fe0a00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 -- 172.21.15.157:0/3554070660 >> v2:172.21.15.195:3300/0 conn(0x7fe0a00ce1a0 msgr2=0x7fe0a00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/3554070660 >> v2:172.21.15.195:3300/0 conn(0x7fe0a00ce1a0 0x7fe0a00d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.817+0000 7fe0a5b62640 1 -- 172.21.15.157:0/3554070660 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00d1e40 con 0x7fe0a015caa0 2023-11-06T03:04:45.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00cdc60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00d0780 tx=0x7fe09c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612f0d5eab72e4bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a4b60640 1 -- 172.21.15.157:0/3554070660 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0aa5f0 con 0x7fe0a015caa0 2023-11-06T03:04:45.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a4b60640 1 -- 172.21.15.157:0/3554070660 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c0aa750 con 0x7fe0a015caa0 2023-11-06T03:04:45.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a4b60640 1 -- 172.21.15.157:0/3554070660 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0aaae0 con 0x7fe0a015caa0 2023-11-06T03:04:45.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/3554070660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 msgr2=0x7fe0a00cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00cdc60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00d0780 tx=0x7fe09c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/3554070660 shutdown_connections 2023-11-06T03:04:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/3554070660 >> v2:172.21.15.195:3300/0 conn(0x7fe0a00ce1a0 0x7fe0a00d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a015ce80 0x7fe0a00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/3554070660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00cdc60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/3554070660 >> 172.21.15.157:0/3554070660 conn(0x7fe0a0070790 msgr2=0x7fe0a0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/3554070660 shutdown_connections 2023-11-06T03:04:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/3554070660 wait complete. 2023-11-06T03:04:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 Processor -- start 2023-11-06T03:04:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- start start 2023-11-06T03:04:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a00ce1a0 0x7fe0a00cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 0x7fe0a0156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00cbc90 con 0x7fe0a015caa0 2023-11-06T03:04:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a015f910 con 0x7fe0a015ce80 2023-11-06T03:04:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.821+0000 7fe0a7dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a015ed00 con 0x7fe0a00ce1a0 2023-11-06T03:04:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33526/0 (socket says 172.21.15.157:33526) 2023-11-06T03:04:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 -- 172.21.15.157:0/2380210357 learned_addr learned my addr 172.21.15.157:0/2380210357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a6363640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 0x7fe0a0156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a00ce1a0 0x7fe0a00cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 -- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a00ce1a0 msgr2=0x7fe0a00cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a00ce1a0 0x7fe0a00cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 -- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 msgr2=0x7fe0a0156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 0x7fe0a0156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 -- 172.21.15.157:0/2380210357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c0094d0 con 0x7fe0a015caa0 2023-11-06T03:04:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a6363640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 0x7fe0a0156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5361640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe09000d800 tx=0x7fe09000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000b6c0 con 0x7fe0a015caa0 2023-11-06T03:04:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09000b820 con 0x7fe0a015caa0 2023-11-06T03:04:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a015efe0 con 0x7fe0a015caa0 2023-11-06T03:04:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000bb90 con 0x7fe0a015caa0 2023-11-06T03:04:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a015f5a0 con 0x7fe0a015caa0 2023-11-06T03:04:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a00b8440 con 0x7fe0a015caa0 2023-11-06T03:04:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe09000bcf0 con 0x7fe0a015caa0 2023-11-06T03:04:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 0x7fe084067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe090083840 con 0x7fe0a015caa0 2023-11-06T03:04:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 0x7fe084067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.825+0000 7fe0a5b62640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 0x7fe084067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0ae000 tx=0x7fe09c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.829+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe0900510a0 con 0x7fe0a015caa0 2023-11-06T03:04:45.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.984+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe0a0072750 con 0x7fe0a015caa0 2023-11-06T03:04:45.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.984+0000 7fe0967fc640 1 -- 172.21.15.157:0/2380210357 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe0900546c0 con 0x7fe0a015caa0 2023-11-06T03:04:45.986 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 msgr2=0x7fe084067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 0x7fe084067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0ae000 tx=0x7fe09c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 msgr2=0x7fe0a00b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe09000d800 tx=0x7fe09000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 shutdown_connections 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe084065a90 0x7fe084067f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0a015ce80 0x7fe0a0156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0a015caa0 0x7fe0a00b2e60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 --2- 172.21.15.157:0/2380210357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0a00ce1a0 0x7fe0a00cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 >> 172.21.15.157:0/2380210357 conn(0x7fe0a0070790 msgr2=0x7fe0a00cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 shutdown_connections 2023-11-06T03:04:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:45.988+0000 7fe0a7dc6640 1 -- 172.21.15.157:0/2380210357 wait complete. 2023-11-06T03:04:46.005 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.124+0000 7f4680a79640 1 Processor -- start 2023-11-06T03:04:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.124+0000 7f4680a79640 1 -- start start 2023-11-06T03:04:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4680a79640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f467c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4680a79640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f467c06d000 0x7f467c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4680a79640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c15e440 con 0x7f467c1599c0 2023-11-06T03:04:46.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4680a79640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c06c240 con 0x7f467c06d000 2023-11-06T03:04:46.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4680a79640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c06c520 con 0x7f467c0b2af0 2023-11-06T03:04:46.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f467a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f467c1599c0 0x7f467c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49814/0 (socket says 172.21.15.157:49814) 2023-11-06T03:04:46.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f467a59c640 1 -- 172.21.15.157:0/3933230645 learned_addr learned my addr 172.21.15.157:0/3933230645 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:46.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f467ad9d640 1 --2- 172.21.15.157:0/3933230645 >> v2:172.21.15.195:3300/0 conn(0x7f467c06d000 0x7f467c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f467c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041238041 0 0) 0x7f467c15e440 con 0x7f467c1599c0 2023-11-06T03:04:46.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4668003530 con 0x7f467c1599c0 2023-11-06T03:04:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2986198030 0 0) 0x7f4668003530 con 0x7f467c1599c0 2023-11-06T03:04:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 msgr2=0x7f467c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f467c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 >> v2:172.21.15.195:3300/0 conn(0x7f467c06d000 msgr2=0x7f467c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 --2- 172.21.15.157:0/3933230645 >> v2:172.21.15.195:3300/0 conn(0x7f467c06d000 0x7f467c159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c06c800 con 0x7f467c1599c0 2023-11-06T03:04:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f466c001680 con 0x7f467c1599c0 2023-11-06T03:04:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 >> v1:172.21.15.157:6789/0 conn(0x7f467c1599c0 legacy=0x7f467c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f4668003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668003f00 0x7f46680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4668004d10 0x7f4668005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c06c240 con 0x7f4668003f00 2023-11-06T03:04:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c06c520 con 0x7f4668004d10 2023-11-06T03:04:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466800dba0 con 0x7f467c0b2af0 2023-11-06T03:04:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668003f00 0x7f46680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f467a59c640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f4668003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f467ad9d640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4668004d10 0x7f4668005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 -- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 msgr2=0x7f4668003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f467c0b2af0 0x7f4668003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 -- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4668004d10 msgr2=0x7f4668005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4668004d10 0x7f4668005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 -- 172.21.15.157:0/3933230645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4668003530 con 0x7f4668003f00 2023-11-06T03:04:46.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.128+0000 7f4671d9b640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668003f00 0x7f46680047d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4664009ae0 tx=0x7f466401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c92cc6863dd1e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46640a5530 con 0x7f4668003f00 2023-11-06T03:04:46.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46640a5690 con 0x7f4668003f00 2023-11-06T03:04:46.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4679d9b640 1 -- 172.21.15.157:0/3933230645 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46640a5a40 con 0x7f4668003f00 2023-11-06T03:04:46.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/3933230645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668003f00 msgr2=0x7f46680047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 --2- 172.21.15.157:0/3933230645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4668003f00 0x7f46680047d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4664009ae0 tx=0x7f466401c9f0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 -- 172.21.15.157:0/3933230645 reap_dead start 2023-11-06T03:04:46.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/3933230645 shutdown_connections 2023-11-06T03:04:46.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/3933230645 >> 172.21.15.157:0/3933230645 conn(0x7f467c065930 msgr2=0x7f467c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/3933230645 shutdown_connections 2023-11-06T03:04:46.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/3933230645 wait complete. 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 Processor -- start 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- start start 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4668003f00 0x7f467c152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f467c0cc1b0 0x7f467c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c151900 con 0x7f467c0ccc20 2023-11-06T03:04:46.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c151ba0 con 0x7f467c0cc1b0 2023-11-06T03:04:46.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c151e80 con 0x7f4668003f00 2023-11-06T03:04:46.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33542/0 (socket says 172.21.15.157:33542) 2023-11-06T03:04:46.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 -- 172.21.15.157:0/1632478561 learned_addr learned my addr 172.21.15.157:0/1632478561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:46.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4671d9b640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f467c0cc1b0 0x7f467c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467a59c640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4668003f00 0x7f467c152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 -- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4668003f00 msgr2=0x7f467c152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4668003f00 0x7f467c152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 -- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f467c0cc1b0 msgr2=0x7f467c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f467c0cc1b0 0x7f467c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 -- 172.21.15.157:0/1632478561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c152160 con 0x7f467c0ccc20 2023-11-06T03:04:46.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f467ad9d640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f467c06f250 tx=0x7f46740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400e070 con 0x7f467c0ccc20 2023-11-06T03:04:46.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674011e10 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c0cd720 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740039a0 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.132+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c152d90 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c0adcc0 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4674003b00 con 0x7f467c0ccc20 2023-11-06T03:04:46.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f46737fe640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 0x7f465c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4674083910 con 0x7f467c0ccc20 2023-11-06T03:04:46.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f467a59c640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 0x7f465c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.136+0000 7f467a59c640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 0x7f465c067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f466c001cd0 tx=0x7f466c001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.140+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4674051200 con 0x7f467c0ccc20 2023-11-06T03:04:46.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.292+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f467c072750 con 0x7f467c0ccc20 2023-11-06T03:04:46.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f46737fe640 1 -- 172.21.15.157:0/1632478561 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4674054820 con 0x7f467c0ccc20 2023-11-06T03:04:46.295 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:46.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 msgr2=0x7f465c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 0x7f465c067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f466c001cd0 tx=0x7f466c001440 comp rx=0 tx=0).stop 2023-11-06T03:04:46.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 msgr2=0x7f467c0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f467c06f250 tx=0x7f46740115a0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 shutdown_connections 2023-11-06T03:04:46.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f465c065a60 0x7f465c067f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f467c0ccc20 0x7f467c0cd0d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f467c0cc1b0 0x7f467c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 --2- 172.21.15.157:0/1632478561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4668003f00 0x7f467c152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 >> 172.21.15.157:0/1632478561 conn(0x7f467c065930 msgr2=0x7f467c0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 shutdown_connections 2023-11-06T03:04:46.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.296+0000 7f4680a79640 1 -- 172.21.15.157:0/1632478561 wait complete. 2023-11-06T03:04:46.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:46.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.436+0000 7f022a860640 1 Processor -- start 2023-11-06T03:04:46.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.436+0000 7f022a860640 1 -- start start 2023-11-06T03:04:46.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f022a860640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f022415e5e0 0x7f02240b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f022a860640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022415f580 con 0x7f02241551e0 2023-11-06T03:04:46.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f022a860640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022415efc0 con 0x7f022415ea90 2023-11-06T03:04:46.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f022a860640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02240704b0 con 0x7f022415e5e0 2023-11-06T03:04:46.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f02241551e0 0x7f022406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49824/0 (socket says 172.21.15.157:49824) 2023-11-06T03:04:46.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 -- 172.21.15.157:0/3983990070 learned_addr learned my addr 172.21.15.157:0/3983990070 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:46.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f02237fe640 1 --2- 172.21.15.157:0/3983990070 >> v2:172.21.15.195:3300/0 conn(0x7f022415e5e0 0x7f02240b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848846766 0 0) 0x7f022415f580 con 0x7f02241551e0 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f020c003530 con 0x7f02241551e0 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518463136 0 0) 0x7f020c003530 con 0x7f02241551e0 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 >> v2:172.21.15.195:3300/0 conn(0x7f022415e5e0 msgr2=0x7f02240b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 --2- 172.21.15.157:0/3983990070 >> v2:172.21.15.195:3300/0 conn(0x7f022415e5e0 0x7f02240b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 >> v1:172.21.15.195:6789/0 conn(0x7f022415ea90 legacy=0x7f022415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022415f2a0 con 0x7f02241551e0 2023-11-06T03:04:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0214001680 con 0x7f02241551e0 2023-11-06T03:04:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 >> v1:172.21.15.157:6789/0 conn(0x7f02241551e0 legacy=0x7f022406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 0x7f020c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c0047d0 0x7f020c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f020c005190 0x7f020c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02240704b0 con 0x7f020c003980 2023-11-06T03:04:46.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c00c130 con 0x7f020c005190 2023-11-06T03:04:46.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c00c2f0 con 0x7f020c0047d0 2023-11-06T03:04:46.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 0x7f020c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f02237fe640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c0047d0 0x7f020c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0228dfd640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f020c005190 0x7f020c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 -- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c0047d0 msgr2=0x7f020c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c0047d0 0x7f020c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 -- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f020c005190 msgr2=0x7f020c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f020c005190 0x7f020c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0223fff640 1 -- 172.21.15.157:0/3983990070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c003530 con 0x7f020c003980 2023-11-06T03:04:46.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0228dfd640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f020c005190 0x7f020c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:46.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.440+0000 7f0228dfd640 1 -- 172.21.15.157:0/3983990070 reap_dead start 2023-11-06T03:04:46.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0223fff640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 0x7f020c003da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f021400c240 tx=0x7f021401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d76ff024907f2e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140a5af0 con 0x7f020c003980 2023-11-06T03:04:46.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02140a5c50 con 0x7f020c003980 2023-11-06T03:04:46.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0222ffd640 1 -- 172.21.15.157:0/3983990070 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140058e0 con 0x7f020c003980 2023-11-06T03:04:46.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 msgr2=0x7f020c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 0x7f020c003da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f021400c240 tx=0x7f021401cc10 comp rx=0 tx=0).stop 2023-11-06T03:04:46.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/3983990070 shutdown_connections 2023-11-06T03:04:46.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 --2- 172.21.15.157:0/3983990070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f020c003980 0x7f020c003da0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/3983990070 >> 172.21.15.157:0/3983990070 conn(0x7f0224065930 msgr2=0x7f0224065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/3983990070 shutdown_connections 2023-11-06T03:04:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/3983990070 wait complete. 2023-11-06T03:04:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 Processor -- start 2023-11-06T03:04:46.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- start start 2023-11-06T03:04:46.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c003980 0x7f0224141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022414be60 0x7f02241f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022414f580 con 0x7f02241f3d90 2023-11-06T03:04:46.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022414e9d0 con 0x7f022414be60 2023-11-06T03:04:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02241418a0 con 0x7f020c003980 2023-11-06T03:04:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33556/0 (socket says 172.21.15.157:33556) 2023-11-06T03:04:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 -- 172.21.15.157:0/2571283147 learned_addr learned my addr 172.21.15.157:0/2571283147 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f02237fe640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022414be60 0x7f02241f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0223fff640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c003980 0x7f0224141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 -- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c003980 msgr2=0x7f0224141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c003980 0x7f0224141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 -- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022414be60 msgr2=0x7f02241f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022414be60 0x7f02241f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 -- 172.21.15.157:0/2571283147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022414ebe0 con 0x7f02241f3d90 2023-11-06T03:04:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0228dfd640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02240cc2c0 tx=0x7f021c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c003d90 con 0x7f02241f3d90 2023-11-06T03:04:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c005070 con 0x7f02241f3d90 2023-11-06T03:04:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022414eec0 con 0x7f02241f3d90 2023-11-06T03:04:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.444+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c005420 con 0x7f02241f3d90 2023-11-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0224150fa0 con 0x7f02241f3d90 2023-11-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022414f6c0 con 0x7f02241f3d90 2023-11-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f021c005580 con 0x7f02241f3d90 2023-11-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f0220ff9640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 0x7f0210067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f021c082b40 con 0x7f02241f3d90 2023-11-06T03:04:46.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f0223fff640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 0x7f0210067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.448+0000 7f0223fff640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 0x7f0210067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02140ab000 tx=0x7f0214006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.452+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f022414f6c0 con 0x7f02241f3d90 2023-11-06T03:04:46.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.604+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f022414f6c0 con 0x7f02241f3d90 2023-11-06T03:04:46.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.604+0000 7f0220ff9640 1 -- 172.21.15.157:0/2571283147 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f022414f6c0 con 0x7f02241f3d90 2023-11-06T03:04:46.606 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:46.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 msgr2=0x7f0210067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 0x7f0210067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02140ab000 tx=0x7f0214006040 comp rx=0 tx=0).stop 2023-11-06T03:04:46.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 msgr2=0x7f022414d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02240cc2c0 tx=0x7f021c00d9d0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 shutdown_connections 2023-11-06T03:04:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0210065a60 0x7f0210067f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241f3d90 0x7f022414d1f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022414be60 0x7f02241f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 --2- 172.21.15.157:0/2571283147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f020c003980 0x7f0224141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 >> 172.21.15.157:0/2571283147 conn(0x7f0224065930 msgr2=0x7f02240cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 shutdown_connections 2023-11-06T03:04:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.608+0000 7f022a860640 1 -- 172.21.15.157:0/2571283147 wait complete. 2023-11-06T03:04:46.625 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 Processor -- start 2023-11-06T03:04:46.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 -- start start 2023-11-06T03:04:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb18c1551e0 0x7fb18c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c15e5e0 0x7fb18c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb18c15ea60 0x7fb18c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c15f580 con 0x7fb18c1551e0 2023-11-06T03:04:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c15efc0 con 0x7fb18c15ea60 2023-11-06T03:04:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb1930b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15f2a0 con 0x7fb18c15e5e0 2023-11-06T03:04:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb18c1551e0 0x7fb18c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb18c1551e0 0x7fb18c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33568/0 (socket says 172.21.15.157:33568) 2023-11-06T03:04:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 -- 172.21.15.157:0/1419087242 learned_addr learned my addr 172.21.15.157:0/1419087242 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb191653640 1 --2- 172.21.15.157:0/1419087242 >> v2:172.21.15.195:3300/0 conn(0x7fb18c15ea60 0x7fb18c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb18bfff640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c15e5e0 0x7fb18c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 -- 172.21.15.157:0/1419087242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c15e5e0 msgr2=0x7fb18c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c15e5e0 0x7fb18c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 -- 172.21.15.157:0/1419087242 >> v2:172.21.15.195:3300/0 conn(0x7fb18c15ea60 msgr2=0x7fb18c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 --2- 172.21.15.157:0/1419087242 >> v2:172.21.15.195:3300/0 conn(0x7fb18c15ea60 0x7fb18c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.748+0000 7fb190e52640 1 -- 172.21.15.157:0/1419087242 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c06f5a0 con 0x7fb18c1551e0 2023-11-06T03:04:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb190e52640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c1551e0 0x7fb18c06fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0cc5d0 tx=0x7fb17c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71e11adf1c3139 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb18b7fe640 1 -- 172.21.15.157:0/1419087242 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0ab620 con 0x7fb18c1551e0 2023-11-06T03:04:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb18b7fe640 1 -- 172.21.15.157:0/1419087242 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0ab780 con 0x7fb18c1551e0 2023-11-06T03:04:46.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb18b7fe640 1 -- 172.21.15.157:0/1419087242 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0abb10 con 0x7fb18c1551e0 2023-11-06T03:04:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- 172.21.15.157:0/1419087242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c1551e0 msgr2=0x7fb18c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c1551e0 0x7fb18c06fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0cc5d0 tx=0x7fb17c01c440 comp rx=0 tx=0).stop 2023-11-06T03:04:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- 172.21.15.157:0/1419087242 shutdown_connections 2023-11-06T03:04:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- 172.21.15.157:0/1419087242 >> v2:172.21.15.195:3300/0 conn(0x7fb18c15ea60 0x7fb18c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c15e5e0 0x7fb18c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- 172.21.15.157:0/1419087242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c1551e0 0x7fb18c06fda0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- 172.21.15.157:0/1419087242 >> 172.21.15.157:0/1419087242 conn(0x7fb18c065930 msgr2=0x7fb18c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- 172.21.15.157:0/1419087242 shutdown_connections 2023-11-06T03:04:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- 172.21.15.157:0/1419087242 wait complete. 2023-11-06T03:04:46.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 Processor -- start 2023-11-06T03:04:46.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- start start 2023-11-06T03:04:46.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c1551e0 0x7fb18c1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb18c15e5e0 0x7fb18c142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0ce380 con 0x7fb18c15ea60 2023-11-06T03:04:46.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0cdae0 con 0x7fb18c15e5e0 2023-11-06T03:04:46.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb1930b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0cddc0 con 0x7fb18c1551e0 2023-11-06T03:04:46.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33578/0 (socket says 172.21.15.157:33578) 2023-11-06T03:04:46.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 -- 172.21.15.157:0/864603870 learned_addr learned my addr 172.21.15.157:0/864603870 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:46.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb190e52640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c1551e0 0x7fb18c1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb18bfff640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb18c15e5e0 0x7fb18c142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 -- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c1551e0 msgr2=0x7fb18c1420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c1551e0 0x7fb18c1420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 -- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb18c15e5e0 msgr2=0x7fb18c142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb18c15e5e0 0x7fb18c142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.752+0000 7fb191653640 1 -- 172.21.15.157:0/864603870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0094d0 con 0x7fb18c15ea60 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb191653640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb18400efc0 tx=0x7fb18400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184009b20 con 0x7fb18c15ea60 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18400f040 con 0x7fb18c15ea60 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c0ce0a0 con 0x7fb18c15ea60 2023-11-06T03:04:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840146c0 con 0x7fb18c15ea60 2023-11-06T03:04:46.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c143b40 con 0x7fb18c15ea60 2023-11-06T03:04:46.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c1523e0 con 0x7fb18c15ea60 2023-11-06T03:04:46.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb184014820 con 0x7fb18c15ea60 2023-11-06T03:04:46.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 0x7fb178067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:46.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb184083fe0 con 0x7fb18c15ea60 2023-11-06T03:04:46.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb190e52640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 0x7fb178067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:46.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.756+0000 7fb190e52640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 0x7fb178067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0af000 tx=0x7fb17c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:46.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.760+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb18c1523e0 con 0x7fb18c15ea60 2023-11-06T03:04:46.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.928+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb18c1527c0 con 0x7fb18c15ea60 2023-11-06T03:04:46.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.928+0000 7fb1897fa640 1 -- 172.21.15.157:0/864603870 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb18c1527c0 con 0x7fb18c15ea60 2023-11-06T03:04:46.929 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:46.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 msgr2=0x7fb178067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 0x7fb178067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0af000 tx=0x7fb17c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:46.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 msgr2=0x7fb18c142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:46.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb18400efc0 tx=0x7fb18400c490 comp rx=0 tx=0).stop 2023-11-06T03:04:46.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 shutdown_connections 2023-11-06T03:04:46.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb178065a60 0x7fb178067f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb18c15ea60 0x7fb18c142db0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb18c15e5e0 0x7fb18c142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 --2- 172.21.15.157:0/864603870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb18c1551e0 0x7fb18c1420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:46.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 >> 172.21.15.157:0/864603870 conn(0x7fb18c065930 msgr2=0x7fb18c14e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:46.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 shutdown_connections 2023-11-06T03:04:46.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:46.932+0000 7fb1930b6640 1 -- 172.21.15.157:0/864603870 wait complete. 2023-11-06T03:04:46.948 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:46.948 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:46.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:47.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.112+0000 7f532497e640 1 Processor -- start 2023-11-06T03:04:47.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.112+0000 7f532497e640 1 -- start start 2023-11-06T03:04:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f532497e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53200b2af0 0x7f53200ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f532497e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f532497e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015e440 con 0x7f53201599c0 2023-11-06T03:04:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f532497e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532006c240 con 0x7f53200b2af0 2023-11-06T03:04:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f532497e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532006c520 con 0x7f532006d000 2023-11-06T03:04:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f5317fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53200b2af0 0x7f53200ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531e59c640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f53201599c0 0x7f53200b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56976/0 (socket says 172.21.15.157:56976) 2023-11-06T03:04:47.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531e59c640 1 -- 172.21.15.157:0/2507436504 learned_addr learned my addr 172.21.15.157:0/2507436504 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:47.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39962/0 (socket says 172.21.15.157:39962) 2023-11-06T03:04:47.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 -- 172.21.15.157:0/2507436504 >> v2:172.21.15.195:3300/0 conn(0x7f53200b2af0 msgr2=0x7f53200ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 --2- 172.21.15.157:0/2507436504 >> v2:172.21.15.195:3300/0 conn(0x7f53200b2af0 0x7f53200ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 -- 172.21.15.157:0/2507436504 >> v1:172.21.15.195:6789/0 conn(0x7f53201599c0 legacy=0x7f53200b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.116+0000 7f531ed9d640 1 -- 172.21.15.157:0/2507436504 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532006c800 con 0x7f532006d000 2023-11-06T03:04:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f531ed9d640 1 --2- 172.21.15.157:0/2507436504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5318009d90 tx=0x7f531801c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c81d6150e326d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f531dd9b640 1 -- 172.21.15.157:0/2507436504 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180a5a00 con 0x7f532006d000 2023-11-06T03:04:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f531dd9b640 1 -- 172.21.15.157:0/2507436504 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180a5b60 con 0x7f532006d000 2023-11-06T03:04:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f531dd9b640 1 -- 172.21.15.157:0/2507436504 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180ad440 con 0x7f532006d000 2023-11-06T03:04:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- 172.21.15.157:0/2507436504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 msgr2=0x7f5320159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- 172.21.15.157:0/2507436504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5318009d90 tx=0x7f531801c920 comp rx=0 tx=0).stop 2023-11-06T03:04:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- 172.21.15.157:0/2507436504 shutdown_connections 2023-11-06T03:04:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- 172.21.15.157:0/2507436504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f532006d000 0x7f5320159da0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- 172.21.15.157:0/2507436504 >> v2:172.21.15.195:3300/0 conn(0x7f53200b2af0 0x7f53200ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- 172.21.15.157:0/2507436504 >> 172.21.15.157:0/2507436504 conn(0x7f5320065930 msgr2=0x7f5320065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- 172.21.15.157:0/2507436504 shutdown_connections 2023-11-06T03:04:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- 172.21.15.157:0/2507436504 wait complete. 2023-11-06T03:04:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 Processor -- start 2023-11-06T03:04:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- start start 2023-11-06T03:04:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532006d000 0x7f53200afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53201599c0 0x7f5320152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200af300 con 0x7f53200b2af0 2023-11-06T03:04:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200b0060 con 0x7f532006d000 2023-11-06T03:04:47.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f532497e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5320146d10 con 0x7f53201599c0 2023-11-06T03:04:47.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f5317fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f5317fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33584/0 (socket says 172.21.15.157:33584) 2023-11-06T03:04:47.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.120+0000 7f5317fff640 1 -- 172.21.15.157:0/1954281444 learned_addr learned my addr 172.21.15.157:0/1954281444 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:47.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f531e59c640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532006d000 0x7f53200afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f531ed9d640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53201599c0 0x7f5320152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 -- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53201599c0 msgr2=0x7f5320152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53201599c0 0x7f5320152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 -- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532006d000 msgr2=0x7f53200afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532006d000 0x7f53200afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 -- 172.21.15.157:0/1954281444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009a70 con 0x7f53200b2af0 2023-11-06T03:04:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5317fff640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f530800e960 tx=0x7f530800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308009690 con 0x7f53200b2af0 2023-11-06T03:04:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530800ca00 con 0x7f53200b2af0 2023-11-06T03:04:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532014f060 con 0x7f53200b2af0 2023-11-06T03:04:47.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800cd70 con 0x7f53200b2af0 2023-11-06T03:04:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53200af550 con 0x7f53200b2af0 2023-11-06T03:04:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5320072750 con 0x7f53200b2af0 2023-11-06T03:04:47.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5308013470 con 0x7f53200b2af0 2023-11-06T03:04:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 0x7f5300067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53080834f0 con 0x7f53200b2af0 2023-11-06T03:04:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f531e59c640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 0x7f5300067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.124+0000 7f531e59c640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 0x7f5300067f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f532014fd40 tx=0x7f5310009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.128+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5308050de0 con 0x7f53200b2af0 2023-11-06T03:04:47.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.280+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f532014c0c0 con 0x7f53200b2af0 2023-11-06T03:04:47.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.284+0000 7f5316ffd640 1 -- 172.21.15.157:0/1954281444 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17859 (secure 0 0 0) 0x7f5308054400 con 0x7f53200b2af0 2023-11-06T03:04:47.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:47.285 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":23,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:07.683680+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:47.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 msgr2=0x7f5300067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 0x7f5300067f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f532014fd40 tx=0x7f5310009040 comp rx=0 tx=0).stop 2023-11-06T03:04:47.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 msgr2=0x7f53201467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f530800e960 tx=0x7f530800ee20 comp rx=0 tx=0).stop 2023-11-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 shutdown_connections 2023-11-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a90 0x7f5300067f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53201599c0 0x7f5320152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53200b2af0 0x7f53201467d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 --2- 172.21.15.157:0/1954281444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f532006d000 0x7f53200afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 >> 172.21.15.157:0/1954281444 conn(0x7f5320065930 msgr2=0x7f532006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 shutdown_connections 2023-11-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.288+0000 7f532497e640 1 -- 172.21.15.157:0/1954281444 wait complete. 2023-11-06T03:04:47.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:47.304 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:47.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.428+0000 7f98fc79a640 1 Processor -- start 2023-11-06T03:04:47.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.428+0000 7f98fc79a640 1 -- start start 2023-11-06T03:04:47.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f4070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98f406d900 0x7f98f40659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98f4065f20 0x7f98f40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f406d4e0 con 0x7f98f4065f20 2023-11-06T03:04:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40663b0 con 0x7f98f406d900 2023-11-06T03:04:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fc79a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f406dd40 con 0x7f98f40ca990 2023-11-06T03:04:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98f4065f20 0x7f98f40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98f4065f20 0x7f98f40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33598/0 (socket says 172.21.15.157:33598) 2023-11-06T03:04:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 -- 172.21.15.157:0/2197438709 learned_addr learned my addr 172.21.15.157:0/2197438709 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98f9d35640 1 --2- 172.21.15.157:0/2197438709 >> v2:172.21.15.195:3300/0 conn(0x7f98f406d900 0x7f98f40659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fa536640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f4070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 -- 172.21.15.157:0/2197438709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 msgr2=0x7f98f4070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f4070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 -- 172.21.15.157:0/2197438709 >> v2:172.21.15.195:3300/0 conn(0x7f98f406d900 msgr2=0x7f98f40659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 --2- 172.21.15.157:0/2197438709 >> v2:172.21.15.195:3300/0 conn(0x7f98f406d900 0x7f98f40659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.432+0000 7f98fad37640 1 -- 172.21.15.157:0/2197438709 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f406c860 con 0x7f98f4065f20 2023-11-06T03:04:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fad37640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f40b28f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98f40666f0 tx=0x7f98e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b723699c0f6af73 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98f9534640 1 -- 172.21.15.157:0/2197438709 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40aa5f0 con 0x7f98f4065f20 2023-11-06T03:04:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98f9534640 1 -- 172.21.15.157:0/2197438709 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e40aa750 con 0x7f98f4065f20 2023-11-06T03:04:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98f9534640 1 -- 172.21.15.157:0/2197438709 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40aaae0 con 0x7f98f4065f20 2023-11-06T03:04:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- 172.21.15.157:0/2197438709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 msgr2=0x7f98f40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f40b28f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98f40666f0 tx=0x7f98e401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- 172.21.15.157:0/2197438709 shutdown_connections 2023-11-06T03:04:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f40b28f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- 172.21.15.157:0/2197438709 >> v2:172.21.15.195:3300/0 conn(0x7f98f406d900 0x7f98f40659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- 172.21.15.157:0/2197438709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f4070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- 172.21.15.157:0/2197438709 >> 172.21.15.157:0/2197438709 conn(0x7f98f406a170 msgr2=0x7f98f406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- 172.21.15.157:0/2197438709 shutdown_connections 2023-11-06T03:04:47.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- 172.21.15.157:0/2197438709 wait complete. 2023-11-06T03:04:47.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 Processor -- start 2023-11-06T03:04:47.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- start start 2023-11-06T03:04:47.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 0x7f98f4152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41568f0 con 0x7f98f4065f20 2023-11-06T03:04:47.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4160560 con 0x7f98f406d900 2023-11-06T03:04:47.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fc79a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41544c0 con 0x7f98f40ca990 2023-11-06T03:04:47.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fa536640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fa536640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33600/0 (socket says 172.21.15.157:33600) 2023-11-06T03:04:47.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98fa536640 1 -- 172.21.15.157:0/3115927692 learned_addr learned my addr 172.21.15.157:0/3115927692 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.436+0000 7f98f9d35640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 0x7f98f4152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fad37640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 -- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 msgr2=0x7f98f40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f40b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 -- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 msgr2=0x7f98f4152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 0x7f98f4152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 -- 172.21.15.157:0/3115927692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40094d0 con 0x7f98f4065f20 2023-11-06T03:04:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fad37640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f9d35640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 0x7f98f4152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fa536640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007e40 tx=0x7f98ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec014620 con 0x7f98f4065f20 2023-11-06T03:04:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00ee90 con 0x7f98f4065f20 2023-11-06T03:04:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0149c0 con 0x7f98f4065f20 2023-11-06T03:04:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f40caef0 con 0x7f98f4065f20 2023-11-06T03:04:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f4156580 con 0x7f98f4065f20 2023-11-06T03:04:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4156100 con 0x7f98f4065f20 2023-11-06T03:04:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98ec014b20 con 0x7f98f4065f20 2023-11-06T03:04:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 0x7f98e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98ec083bf0 con 0x7f98f4065f20 2023-11-06T03:04:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f9d35640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 0x7f98e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.440+0000 7f98f9d35640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 0x7f98e0067f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98e8004660 tx=0x7f98e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.444+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f98f4156100 con 0x7f98f4065f20 2023-11-06T03:04:47.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.600+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98f4156100 con 0x7f98f4065f20 2023-11-06T03:04:47.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98f2ffd640 1 -- 172.21.15.157:0/3115927692 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98f4156100 con 0x7f98f4065f20 2023-11-06T03:04:47.603 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 msgr2=0x7f98e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 0x7f98e0067f50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98e8004660 tx=0x7f98e8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 msgr2=0x7f98f4153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007e40 tx=0x7f98ec00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 shutdown_connections 2023-11-06T03:04:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98e0065a90 0x7f98e0067f50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98f40ca990 0x7f98f40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98f406d900 0x7f98f4152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 --2- 172.21.15.157:0/3115927692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98f4065f20 0x7f98f4153dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 >> 172.21.15.157:0/3115927692 conn(0x7f98f406a170 msgr2=0x7f98f40b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 shutdown_connections 2023-11-06T03:04:47.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.604+0000 7f98fc79a640 1 -- 172.21.15.157:0/3115927692 wait complete. 2023-11-06T03:04:47.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:47.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.740+0000 7f0620a74640 1 Processor -- start 2023-11-06T03:04:47.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.740+0000 7f0620a74640 1 -- start start 2023-11-06T03:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0620a74640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f061c0adae0 0x7f061c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0620a74640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f061c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0620a74640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c06c490 con 0x7f061c156a70 2023-11-06T03:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0620a74640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061c06b950 con 0x7f061c0adae0 2023-11-06T03:04:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0620a74640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c06fb80 con 0x7f061c0adf40 2023-11-06T03:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f061c156a70 0x7f061c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49834/0 (socket says 172.21.15.157:49834) 2023-11-06T03:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 -- 172.21.15.157:0/400764802 learned_addr learned my addr 172.21.15.157:0/400764802 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072200690 0 0) 0x7f061c06c490 con 0x7f061c156a70 2023-11-06T03:04:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0608003450 con 0x7f061c156a70 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0613fff640 1 --2- 172.21.15.157:0/400764802 >> v2:172.21.15.195:3300/0 conn(0x7f061c0adae0 0x7f061c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061ad9d640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f061c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000503359 0 0) 0x7f0608003450 con 0x7f061c156a70 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 msgr2=0x7f061c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f061c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 >> v2:172.21.15.195:3300/0 conn(0x7f061c0adae0 msgr2=0x7f061c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 --2- 172.21.15.157:0/400764802 >> v2:172.21.15.195:3300/0 conn(0x7f061c0adae0 0x7f061c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c06bb60 con 0x7f061c156a70 2023-11-06T03:04:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f060c001680 con 0x7f061c156a70 2023-11-06T03:04:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 >> v1:172.21.15.157:6789/0 conn(0x7f061c156a70 legacy=0x7f061c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f0608003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f060800a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060800af50 0x7f060800d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c06b950 con 0x7f0608003a70 2023-11-06T03:04:47.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c06fb80 con 0x7f060800af50 2023-11-06T03:04:47.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060800dc40 con 0x7f061c0adf40 2023-11-06T03:04:47.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f0608003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061ad9d640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060800af50 0x7f060800d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f0613fff640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f060800a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 -- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 msgr2=0x7f060800a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c0adf40 0x7f060800a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 -- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060800af50 msgr2=0x7f060800d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060800af50 0x7f060800d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 -- 172.21.15.157:0/400764802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608003450 con 0x7f0608003a70 2023-11-06T03:04:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.744+0000 7f061a59c640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f0608003e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c240 tx=0x7f060c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7470cca16187db3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0058e0 con 0x7f0608003a70 2023-11-06T03:04:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c005a40 con 0x7f0608003a70 2023-11-06T03:04:47.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0619d9b640 1 -- 172.21.15.157:0/400764802 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c005e30 con 0x7f0608003a70 2023-11-06T03:04:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/400764802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 msgr2=0x7f0608003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 --2- 172.21.15.157:0/400764802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f0608003e90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c240 tx=0x7f060c01cc30 comp rx=0 tx=0).stop 2023-11-06T03:04:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061ad9d640 1 -- 172.21.15.157:0/400764802 reap_dead start 2023-11-06T03:04:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/400764802 shutdown_connections 2023-11-06T03:04:47.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/400764802 >> 172.21.15.157:0/400764802 conn(0x7f061c065930 msgr2=0x7f061c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/400764802 shutdown_connections 2023-11-06T03:04:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/400764802 wait complete. 2023-11-06T03:04:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 Processor -- start 2023-11-06T03:04:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- start start 2023-11-06T03:04:47.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c1437a0 0x7f061c14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f061c14d470 0x7f061c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b1110 con 0x7f0608003a70 2023-11-06T03:04:47.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1491a0 con 0x7f061c14d470 2023-11-06T03:04:47.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c148e70 con 0x7f061c1437a0 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33624/0 (socket says 172.21.15.157:33624) 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 -- 172.21.15.157:0/1254202126 learned_addr learned my addr 172.21.15.157:0/1254202126 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 -- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c1437a0 msgr2=0x7f061c14cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c1437a0 0x7f061c14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 -- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f061c14d470 msgr2=0x7f061c06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f061c14d470 0x7f061c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 -- 172.21.15.157:0/1254202126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0b0870 con 0x7f0608003a70 2023-11-06T03:04:47.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f061a59c640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f060c0ab040 tx=0x7f0608003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c000ee0 con 0x7f0608003a70 2023-11-06T03:04:47.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c001c60 con 0x7f0608003a70 2023-11-06T03:04:47.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c0b0b50 con 0x7f0608003a70 2023-11-06T03:04:47.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c001660 con 0x7f0608003a70 2023-11-06T03:04:47.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.748+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c0b04d0 con 0x7f0608003a70 2023-11-06T03:04:47.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c072750 con 0x7f0608003a70 2023-11-06T03:04:47.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f060c0a57a0 con 0x7f0608003a70 2023-11-06T03:04:47.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0612ffd640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 0x7f05fc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:47.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f060c11cb80 con 0x7f0608003a70 2023-11-06T03:04:47.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0613fff640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 0x7f05fc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:47.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.752+0000 7f0613fff640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 0x7f05fc067f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0604006aa0 tx=0x7f0604006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:47.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.756+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f060c0ea430 con 0x7f0608003a70 2023-11-06T03:04:47.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.896+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f061c06b490 con 0x7f0608003a70 2023-11-06T03:04:47.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0612ffd640 1 -- 172.21.15.157:0/1254202126 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f060c0eda50 con 0x7f0608003a70 2023-11-06T03:04:47.899 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:47.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 msgr2=0x7f05fc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 0x7f05fc067f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0604006aa0 tx=0x7f0604006a30 comp rx=0 tx=0).stop 2023-11-06T03:04:47.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 msgr2=0x7f061c0ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:47.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f060c0ab040 tx=0x7f0608003700 comp rx=0 tx=0).stop 2023-11-06T03:04:47.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 shutdown_connections 2023-11-06T03:04:47.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05fc065a90 0x7f05fc067f50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f061c14d470 0x7f061c06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f061c1437a0 0x7f061c14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 --2- 172.21.15.157:0/1254202126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0608003a70 0x7f061c0ae560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:47.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 >> 172.21.15.157:0/1254202126 conn(0x7f061c065930 msgr2=0x7f061c159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:47.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 shutdown_connections 2023-11-06T03:04:47.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:47.900+0000 7f0620a74640 1 -- 172.21.15.157:0/1254202126 wait complete. 2023-11-06T03:04:47.916 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 Processor -- start 2023-11-06T03:04:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 -- start start 2023-11-06T03:04:48.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca2c1599c0 0x7fca2c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 0x7fca2c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c15e440 con 0x7fca2c15dea0 2023-11-06T03:04:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c15adb0 con 0x7fca2c1599c0 2023-11-06T03:04:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca32b97640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c06c240 con 0x7fca2c06d000 2023-11-06T03:04:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca30933640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca2c1599c0 0x7fca2c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca31134640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fca2c15dea0 0x7fca2c070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49836/0 (socket says 172.21.15.157:49836) 2023-11-06T03:04:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca31134640 1 -- 172.21.15.157:0/695317206 learned_addr learned my addr 172.21.15.157:0/695317206 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca2bfff640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 0x7fca2c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca30933640 1 -- 172.21.15.157:0/695317206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 msgr2=0x7fca2c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca30933640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 0x7fca2c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.044+0000 7fca30933640 1 -- 172.21.15.157:0/695317206 >> v1:172.21.15.157:6789/0 conn(0x7fca2c15dea0 legacy=0x7fca2c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca2b7fe640 1 -- 172.21.15.157:0/695317206 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506628556 0 0) 0x7fca2c15e440 con 0x7fca2c15dea0 2023-11-06T03:04:48.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca30933640 1 -- 172.21.15.157:0/695317206 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c06c520 con 0x7fca2c1599c0 2023-11-06T03:04:48.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca2bfff640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 0x7fca2c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:48.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca30933640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c0ae000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993403ca66fc20ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca2b7fe640 1 -- 172.21.15.157:0/695317206 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ad660 con 0x7fca2c1599c0 2023-11-06T03:04:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca2b7fe640 1 -- 172.21.15.157:0/695317206 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ad7c0 con 0x7fca2c1599c0 2023-11-06T03:04:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca2b7fe640 1 -- 172.21.15.157:0/695317206 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0adb50 con 0x7fca2c1599c0 2023-11-06T03:04:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 -- 172.21.15.157:0/695317206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 msgr2=0x7fca2c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c0ae000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c450 comp rx=0 tx=0).stop 2023-11-06T03:04:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 -- 172.21.15.157:0/695317206 shutdown_connections 2023-11-06T03:04:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c06d000 0x7fca2c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 --2- 172.21.15.157:0/695317206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c0ae000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 -- 172.21.15.157:0/695317206 >> 172.21.15.157:0/695317206 conn(0x7fca2c065930 msgr2=0x7fca2c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 -- 172.21.15.157:0/695317206 shutdown_connections 2023-11-06T03:04:48.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.048+0000 7fca32b97640 1 -- 172.21.15.157:0/695317206 wait complete. 2023-11-06T03:04:48.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 Processor -- start 2023-11-06T03:04:48.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- start start 2023-11-06T03:04:48.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 0x7fca2c14ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c15dea0 0x7fca2c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0cd5f0 con 0x7fca2c1599c0 2023-11-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c156220 con 0x7fca2c06d000 2023-11-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c155c60 con 0x7fca2c15dea0 2023-11-06T03:04:48.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33646/0 (socket says 172.21.15.157:33646) 2023-11-06T03:04:48.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 -- 172.21.15.157:0/1384077063 learned_addr learned my addr 172.21.15.157:0/1384077063 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:48.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 -- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c15dea0 msgr2=0x7fca2c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c15dea0 0x7fca2c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 -- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 msgr2=0x7fca2c14ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca30933640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 0x7fca2c14ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 0x7fca2c14ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 -- 172.21.15.157:0/1384077063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009420 con 0x7fca2c1599c0 2023-11-06T03:04:48.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca30933640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 0x7fca2c14ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:48.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca2bfff640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c7c0 tx=0x7fca2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20007a30 con 0x7fca2c1599c0 2023-11-06T03:04:48.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20007b90 con 0x7fca2c1599c0 2023-11-06T03:04:48.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000ce70 con 0x7fca2c1599c0 2023-11-06T03:04:48.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c155f40 con 0x7fca2c1599c0 2023-11-06T03:04:48.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c0ad480 con 0x7fca2c1599c0 2023-11-06T03:04:48.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.052+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c072750 con 0x7fca2c1599c0 2023-11-06T03:04:48.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.056+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fca200113f0 con 0x7fca2c1599c0 2023-11-06T03:04:48.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.056+0000 7fca297fa640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 0x7fca18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.056+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fca200837b0 con 0x7fca2c1599c0 2023-11-06T03:04:48.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.056+0000 7fca30933640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 0x7fca18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.056+0000 7fca30933640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 0x7fca18067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0af000 tx=0x7fca1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.060+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fca20051060 con 0x7fca2c1599c0 2023-11-06T03:04:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.200+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca2c0adcd0 con 0x7fca2c1599c0 2023-11-06T03:04:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.200+0000 7fca297fa640 1 -- 172.21.15.157:0/1384077063 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca20054680 con 0x7fca2c1599c0 2023-11-06T03:04:48.202 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 msgr2=0x7fca18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 0x7fca18067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0af000 tx=0x7fca1c008040 comp rx=0 tx=0).stop 2023-11-06T03:04:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 msgr2=0x7fca2c156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c7c0 tx=0x7fca2000cc80 comp rx=0 tx=0).stop 2023-11-06T03:04:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 shutdown_connections 2023-11-06T03:04:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fca18065a60 0x7fca18067f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca2c15dea0 0x7fca2c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1599c0 0x7fca2c156550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 --2- 172.21.15.157:0/1384077063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca2c06d000 0x7fca2c14ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 >> 172.21.15.157:0/1384077063 conn(0x7fca2c065930 msgr2=0x7fca2c153c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 shutdown_connections 2023-11-06T03:04:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.204+0000 7fca32b97640 1 -- 172.21.15.157:0/1384077063 wait complete. 2023-11-06T03:04:48.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.336+0000 7f82a893a640 1 Processor -- start 2023-11-06T03:04:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.336+0000 7f82a893a640 1 -- start start 2023-11-06T03:04:48.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a893a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f82a40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a893a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40cef50 con 0x7f82a40d02f0 2023-11-06T03:04:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a893a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40cf230 con 0x7f82a4071940 2023-11-06T03:04:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a893a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cf510 con 0x7f82a415aca0 2023-11-06T03:04:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a2d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f82a40d02f0 0x7f82a40cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49842/0 (socket says 172.21.15.157:49842) 2023-11-06T03:04:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1542405609 learned_addr learned my addr 172.21.15.157:0/1542405609 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a259c640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f82a40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777092531 0 0) 0x7f82a40cef50 con 0x7f82a40d02f0 2023-11-06T03:04:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f828c003530 con 0x7f82a40d02f0 2023-11-06T03:04:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042614213 0 0) 0x7f828c003530 con 0x7f82a40d02f0 2023-11-06T03:04:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 msgr2=0x7f82a40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f82a40ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 >> v1:172.21.15.195:6789/0 conn(0x7f82a4071940 legacy=0x7f82a40cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40cf7f0 con 0x7f82a40d02f0 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f829c001a00 con 0x7f82a40d02f0 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 >> v1:172.21.15.157:6789/0 conn(0x7f82a40d02f0 legacy=0x7f82a40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f828c0047d0 0x7f828c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f828c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cf510 con 0x7f828c0047d0 2023-11-06T03:04:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c00ba40 con 0x7f828c0039b0 2023-11-06T03:04:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c00bc00 con 0x7f82a415aca0 2023-11-06T03:04:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f828c0047d0 0x7f828c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a259c640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a2d9d640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f828c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 -- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 msgr2=0x7f828c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a415aca0 0x7f828c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 -- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0039b0 msgr2=0x7f828c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 -- 172.21.15.157:0/1542405609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c003530 con 0x7f828c0047d0 2023-11-06T03:04:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a1d9b640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f828c0047d0 0x7f828c004c50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82a4071d60 tx=0x7f829801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4271ae0ae0ecf949 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980052d0 con 0x7f828c0047d0 2023-11-06T03:04:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82980a7050 con 0x7f828c0047d0 2023-11-06T03:04:48.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.340+0000 7f82a159a640 1 -- 172.21.15.157:0/1542405609 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298003600 con 0x7f828c0047d0 2023-11-06T03:04:48.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1542405609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f828c0047d0 msgr2=0x7f828c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 --2- 172.21.15.157:0/1542405609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f828c0047d0 0x7f828c004c50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f82a4071d60 tx=0x7f829801c380 comp rx=0 tx=0).stop 2023-11-06T03:04:48.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1542405609 reap_dead start 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1542405609 shutdown_connections 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1542405609 >> 172.21.15.157:0/1542405609 conn(0x7f82a406d560 msgr2=0x7f82a406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1542405609 shutdown_connections 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1542405609 wait complete. 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 Processor -- start 2023-11-06T03:04:48.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- start start 2023-11-06T03:04:48.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0047d0 0x7f82a41f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a40c66e0 0x7f82a415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a41efb90 con 0x7f82a415c8e0 2023-11-06T03:04:48.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a415cd60 con 0x7f828c0047d0 2023-11-06T03:04:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a41f4140 con 0x7f82a40c66e0 2023-11-06T03:04:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:33656/0 (socket says 172.21.15.157:33656) 2023-11-06T03:04:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1739014239 learned_addr learned my addr 172.21.15.157:0/1739014239 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a259c640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0047d0 0x7f82a41f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a1d9b640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a40c66e0 0x7f82a415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a40c66e0 msgr2=0x7f82a415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a40c66e0 0x7f82a415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0047d0 msgr2=0x7f82a41f0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0047d0 0x7f82a41f0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 -- 172.21.15.157:0/1739014239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a41f3e60 con 0x7f82a415c8e0 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a2d9d640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f829c001340 tx=0x7f829c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00eb80 con 0x7f82a415c8e0 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00ece0 con 0x7f82a415c8e0 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a41efd70 con 0x7f82a415c8e0 2023-11-06T03:04:48.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c015640 con 0x7f82a415c8e0 2023-11-06T03:04:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.344+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a41f0330 con 0x7f82a415c8e0 2023-11-06T03:04:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a4072750 con 0x7f82a415c8e0 2023-11-06T03:04:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f829c01e3b0 con 0x7f82a415c8e0 2023-11-06T03:04:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f8292ffd640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 0x7f8284067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f829c085f50 con 0x7f82a415c8e0 2023-11-06T03:04:48.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f82a259c640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 0x7f8284067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.348+0000 7f82a259c640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 0x7f8284067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8294009ae0 tx=0x7f8294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.352+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f829c053820 con 0x7f82a415c8e0 2023-11-06T03:04:48.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82a40a9540 con 0x7f82a415c8e0 2023-11-06T03:04:48.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f8292ffd640 1 -- 172.21.15.157:0/1739014239 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f829c01a020 con 0x7f82a415c8e0 2023-11-06T03:04:48.493 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:48.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 msgr2=0x7f8284067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 0x7f8284067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8294009ae0 tx=0x7f8294008040 comp rx=0 tx=0).stop 2023-11-06T03:04:48.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 msgr2=0x7f82a41f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.492+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f829c001340 tx=0x7f829c005ae0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 shutdown_connections 2023-11-06T03:04:48.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8284065a60 0x7f8284067f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82a415c8e0 0x7f82a41f31e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f82a40c66e0 0x7f82a415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 --2- 172.21.15.157:0/1739014239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f828c0047d0 0x7f82a41f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 >> 172.21.15.157:0/1739014239 conn(0x7f82a406d560 msgr2=0x7f82a40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 shutdown_connections 2023-11-06T03:04:48.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.496+0000 7f82a893a640 1 -- 172.21.15.157:0/1739014239 wait complete. 2023-11-06T03:04:48.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.628+0000 7f6cb00f2640 1 Processor -- start 2023-11-06T03:04:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.628+0000 7f6cb00f2640 1 -- start start 2023-11-06T03:04:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cb00f2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6ca8160db0 0x7f6ca8161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cb00f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca806f470 con 0x7f6ca80b1910 2023-11-06T03:04:48.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cb00f2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca80b1df0 con 0x7f6ca80701a0 2023-11-06T03:04:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cb00f2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca806f6c0 con 0x7f6ca8160db0 2023-11-06T03:04:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6ca80b1910 0x7f6ca806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50628/0 (socket says 172.21.15.157:50628) 2023-11-06T03:04:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 -- 172.21.15.157:0/759858202 learned_addr learned my addr 172.21.15.157:0/759858202 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cade8e640 1 --2- 172.21.15.157:0/759858202 >> v2:172.21.15.195:3300/0 conn(0x7f6ca8160db0 0x7f6ca8161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463367942 0 0) 0x7f6ca806f470 con 0x7f6ca80b1910 2023-11-06T03:04:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c94003530 con 0x7f6ca80b1910 2023-11-06T03:04:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248170359 0 0) 0x7f6c94003530 con 0x7f6ca80b1910 2023-11-06T03:04:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 >> v2:172.21.15.195:3300/0 conn(0x7f6ca8160db0 msgr2=0x7f6ca8161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 --2- 172.21.15.157:0/759858202 >> v2:172.21.15.195:3300/0 conn(0x7f6ca8160db0 0x7f6ca8161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 >> v1:172.21.15.195:6789/0 conn(0x7f6ca80701a0 legacy=0x7f6ca80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca806f9a0 con 0x7f6ca80b1910 2023-11-06T03:04:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6c98001680 con 0x7f6ca80b1910 2023-11-06T03:04:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 >> v1:172.21.15.157:6789/0 conn(0x7f6ca80b1910 legacy=0x7f6ca806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c940039b0 0x7f6c94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 0x7f6c94004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940051e0 0x7f6c9400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806f6c0 con 0x7f6c940047d0 2023-11-06T03:04:48.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9400c110 con 0x7f6c940051e0 2023-11-06T03:04:48.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9400c2d0 con 0x7f6c940039b0 2023-11-06T03:04:48.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 0x7f6c94004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cae68f640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940051e0 0x7f6c9400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cade8e640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c940039b0 0x7f6c94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 -- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c940039b0 msgr2=0x7f6c94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c940039b0 0x7f6c94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 -- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940051e0 msgr2=0x7f6c9400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940051e0 0x7f6c9400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cad68d640 1 -- 172.21.15.157:0/759858202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94003530 con 0x7f6c940047d0 2023-11-06T03:04:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cae68f640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940051e0 0x7f6c9400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.632+0000 7f6cae68f640 1 -- 172.21.15.157:0/759858202 reap_dead start 2023-11-06T03:04:48.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cad68d640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 0x7f6c94004c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80b1d20 tx=0x7f6c9801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fbf8ecc33ab3cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005ae0 con 0x7f6c940047d0 2023-11-06T03:04:48.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98005c40 con 0x7f6c940047d0 2023-11-06T03:04:48.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cace8c640 1 -- 172.21.15.157:0/759858202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980072b0 con 0x7f6c940047d0 2023-11-06T03:04:48.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 msgr2=0x7f6c94004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 0x7f6c94004c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80b1d20 tx=0x7f6c9801cbb0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- 172.21.15.157:0/759858202 shutdown_connections 2023-11-06T03:04:48.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/759858202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c940047d0 0x7f6c94004c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- 172.21.15.157:0/759858202 >> 172.21.15.157:0/759858202 conn(0x7f6ca8065930 msgr2=0x7f6ca8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- 172.21.15.157:0/759858202 shutdown_connections 2023-11-06T03:04:48.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- 172.21.15.157:0/759858202 wait complete. 2023-11-06T03:04:48.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 Processor -- start 2023-11-06T03:04:48.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- start start 2023-11-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 0x7f6ca815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ca81f9460 0x7f6ca8148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8149230 con 0x7f6ca8155d70 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81569d0 con 0x7f6c940047d0 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cb00f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8156cb0 con 0x7f6ca81f9460 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cae68f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cae68f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44018/0 (socket says 172.21.15.157:44018) 2023-11-06T03:04:48.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cae68f640 1 -- 172.21.15.157:0/72022305 learned_addr learned my addr 172.21.15.157:0/72022305 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:48.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cad68d640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ca81f9460 0x7f6ca8148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.636+0000 7f6cade8e640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 0x7f6ca815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 -- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ca81f9460 msgr2=0x7f6ca8148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ca81f9460 0x7f6ca8148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 -- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 msgr2=0x7f6ca815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 0x7f6ca815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 -- 172.21.15.157:0/72022305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8156f90 con 0x7f6ca8155d70 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cade8e640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 0x7f6ca815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cae68f640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8066830 tx=0x7f6ca000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000eed0 con 0x7f6ca8155d70 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0003980 con 0x7f6ca8155d70 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8157270 con 0x7f6ca8155d70 2023-11-06T03:04:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0003cf0 con 0x7f6ca8155d70 2023-11-06T03:04:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81f98e0 con 0x7f6ca8155d70 2023-11-06T03:04:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca8072750 con 0x7f6ca8155d70 2023-11-06T03:04:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ca00050d0 con 0x7f6ca8155d70 2023-11-06T03:04:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 0x7f6c8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6ca0082fe0 con 0x7f6ca8155d70 2023-11-06T03:04:48.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cade8e640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 0x7f6c8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:48.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.640+0000 7f6cade8e640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 0x7f6c8c067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009b70 tx=0x7f6ca4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:48.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.644+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6ca0050800 con 0x7f6ca8155d70 2023-11-06T03:04:48.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.800+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ca8071020 con 0x7f6ca8155d70 2023-11-06T03:04:48.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6c9e7fc640 1 -- 172.21.15.157:0/72022305 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ca0053e20 con 0x7f6ca8155d70 2023-11-06T03:04:48.803 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:48.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 msgr2=0x7f6c8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 0x7f6c8c067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009b70 tx=0x7f6ca4008040 comp rx=0 tx=0).stop 2023-11-06T03:04:48.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 msgr2=0x7f6ca81561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:48.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8066830 tx=0x7f6ca000eb90 comp rx=0 tx=0).stop 2023-11-06T03:04:48.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 shutdown_connections 2023-11-06T03:04:48.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c8c065a60 0x7f6c8c067f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ca8155d70 0x7f6ca81561f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6ca81f9460 0x7f6ca8148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 --2- 172.21.15.157:0/72022305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c940047d0 0x7f6ca815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:48.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 >> 172.21.15.157:0/72022305 conn(0x7f6ca8065930 msgr2=0x7f6ca815ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:48.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 shutdown_connections 2023-11-06T03:04:48.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:48.804+0000 7f6cb00f2640 1 -- 172.21.15.157:0/72022305 wait complete. 2023-11-06T03:04:48.821 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:04:48.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:48.822 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:04:48.842 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:04:48.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:04:48.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:48.843 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:04:48.892 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:48.892 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:48.899 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:48.899 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:48.945 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:49.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.064+0000 7f2e104cc640 1 Processor -- start 2023-11-06T03:04:49.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.064+0000 7f2e104cc640 1 -- start start 2023-11-06T03:04:49.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e104cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e08146370 0x7f2e080b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e104cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e080ce800 0x7f2e080cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e104cc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e080cf1a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e104cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0806c3d0 con 0x7f2e080ce800 2023-11-06T03:04:49.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e104cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0806f5a0 con 0x7f2e08146370 2023-11-06T03:04:49.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e080ce800 0x7f2e080cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0ea69640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2e0806b6a0 0x7f2e0806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50640/0 (socket says 172.21.15.157:50640) 2023-11-06T03:04:49.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0ea69640 1 -- 172.21.15.157:0/2182788650 learned_addr learned my addr 172.21.15.157:0/2182788650 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:49.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0e268640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e08146370 0x7f2e080b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 -- 172.21.15.157:0/2182788650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e08146370 msgr2=0x7f2e080b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0d266640 1 -- 172.21.15.157:0/2182788650 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143806305 0 0) 0x7f2e080cf1a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e08146370 0x7f2e080b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 -- 172.21.15.157:0/2182788650 >> v1:172.21.15.157:6789/0 conn(0x7f2e0806b6a0 legacy=0x7f2e0806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 -- 172.21.15.157:0/2182788650 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0806f880 con 0x7f2e080ce800 2023-11-06T03:04:49.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0da67640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e080ce800 0x7f2e080cec60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2df80097c0 tx=0x7f2df801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f58e49bccaead5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0d266640 1 -- 172.21.15.157:0/2182788650 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80ab5f0 con 0x7f2e080ce800 2023-11-06T03:04:49.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0d266640 1 -- 172.21.15.157:0/2182788650 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80ab750 con 0x7f2e080ce800 2023-11-06T03:04:49.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.068+0000 7f2e0d266640 1 -- 172.21.15.157:0/2182788650 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80abae0 con 0x7f2e080ce800 2023-11-06T03:04:49.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/2182788650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e080ce800 msgr2=0x7f2e080cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e080ce800 0x7f2e080cec60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2df80097c0 tx=0x7f2df801c3f0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/2182788650 shutdown_connections 2023-11-06T03:04:49.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e080ce800 0x7f2e080cec60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- 172.21.15.157:0/2182788650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e08146370 0x7f2e080b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/2182788650 >> 172.21.15.157:0/2182788650 conn(0x7f2e08065930 msgr2=0x7f2e08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/2182788650 shutdown_connections 2023-11-06T03:04:49.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/2182788650 wait complete. 2023-11-06T03:04:49.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 Processor -- start 2023-11-06T03:04:49.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- start start 2023-11-06T03:04:49.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e080ce800 0x7f2e08144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 0x7f2e0813c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08144bd0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08144eb0 con 0x7f2e08146370 2023-11-06T03:04:49.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08145190 con 0x7f2e080ce800 2023-11-06T03:04:49.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44026/0 (socket says 172.21.15.157:44026) 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 -- 172.21.15.157:0/3459292068 learned_addr learned my addr 172.21.15.157:0/3459292068 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0da67640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e080ce800 0x7f2e08144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0ea69640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 0x7f2e0813c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 -- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e080ce800 msgr2=0x7f2e08144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e080ce800 0x7f2e08144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 -- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 msgr2=0x7f2e0813c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 0x7f2e0813c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 -- 172.21.15.157:0/3459292068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009420 con 0x7f2e0806b6a0 2023-11-06T03:04:49.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0ea69640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 0x7f2e0813c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:49.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e0e268640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c7f0 tx=0x7f2e0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004220 con 0x7f2e0806b6a0 2023-11-06T03:04:49.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e040043a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004510 con 0x7f2e0806b6a0 2023-11-06T03:04:49.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0815a740 con 0x7f2e0806b6a0 2023-11-06T03:04:49.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.072+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0813c850 con 0x7f2e0806b6a0 2023-11-06T03:04:49.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0815a5a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e04003a70 con 0x7f2e0806b6a0 2023-11-06T03:04:49.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2dfeffd640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 0x7f2dec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2e04083300 con 0x7f2e0806b6a0 2023-11-06T03:04:49.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2e0da67640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 0x7f2dec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.076+0000 7f2e0da67640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 0x7f2dec067f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2df80af000 tx=0x7f2df80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.080+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e0815a5a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.236+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e0815a5a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.236+0000 7f2dfeffd640 1 -- 172.21.15.157:0/3459292068 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f2e0815a5a0 con 0x7f2e0806b6a0 2023-11-06T03:04:49.237 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:49.238 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:49.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 msgr2=0x7f2dec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 0x7f2dec067f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2df80af000 tx=0x7f2df80a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:49.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 msgr2=0x7f2e0815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c7f0 tx=0x7f2e0400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 shutdown_connections 2023-11-06T03:04:49.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2dec065a60 0x7f2dec067f20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e08146370 0x7f2e0813c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e080ce800 0x7f2e08144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 --2- 172.21.15.157:0/3459292068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e0806b6a0 0x7f2e0815e600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 >> 172.21.15.157:0/3459292068 conn(0x7f2e08065930 msgr2=0x7f2e08065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 shutdown_connections 2023-11-06T03:04:49.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.240+0000 7f2e104cc640 1 -- 172.21.15.157:0/3459292068 wait complete. 2023-11-06T03:04:49.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:49.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:49.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 Processor -- start 2023-11-06T03:04:49.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 -- start start 2023-11-06T03:04:49.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f9415caa0 0x7f7f940cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f9415ce80 0x7f7f940cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f940d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940d2920 con 0x7f7f9415ce80 2023-11-06T03:04:49.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940d18a0 con 0x7f7f9415caa0 2023-11-06T03:04:49.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f9c176640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940d1b60 con 0x7f7f940ce1a0 2023-11-06T03:04:49.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f99711640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f9415ce80 0x7f7f940cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f99711640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7f9415ce80 0x7f7f940cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44032/0 (socket says 172.21.15.157:44032) 2023-11-06T03:04:49.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.380+0000 7f7f99711640 1 -- 172.21.15.157:0/3007206471 learned_addr learned my addr 172.21.15.157:0/3007206471 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9a713640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f940d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99f12640 1 --2- 172.21.15.157:0/3007206471 >> v2:172.21.15.195:3300/0 conn(0x7f7f9415caa0 0x7f7f940cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 -- 172.21.15.157:0/3007206471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 msgr2=0x7f7f940d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f940d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 -- 172.21.15.157:0/3007206471 >> v2:172.21.15.195:3300/0 conn(0x7f7f9415caa0 msgr2=0x7f7f940cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 --2- 172.21.15.157:0/3007206471 >> v2:172.21.15.195:3300/0 conn(0x7f7f9415caa0 0x7f7f940cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 -- 172.21.15.157:0/3007206471 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940d1e40 con 0x7f7f9415ce80 2023-11-06T03:04:49.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f99711640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f940cfb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7f940b4f80 tx=0x7f7f8401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843a7a8ca59fa54a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f98f10640 1 -- 172.21.15.157:0/3007206471 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f840aa5f0 con 0x7f7f9415ce80 2023-11-06T03:04:49.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f98f10640 1 -- 172.21.15.157:0/3007206471 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f840aa750 con 0x7f7f9415ce80 2023-11-06T03:04:49.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f98f10640 1 -- 172.21.15.157:0/3007206471 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f840aaae0 con 0x7f7f9415ce80 2023-11-06T03:04:49.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 -- 172.21.15.157:0/3007206471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 msgr2=0x7f7f940cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f940cfb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7f940b4f80 tx=0x7f7f8401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 -- 172.21.15.157:0/3007206471 shutdown_connections 2023-11-06T03:04:49.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f940d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 --2- 172.21.15.157:0/3007206471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f940cfb70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 --2- 172.21.15.157:0/3007206471 >> v2:172.21.15.195:3300/0 conn(0x7f7f9415caa0 0x7f7f940cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 -- 172.21.15.157:0/3007206471 >> 172.21.15.157:0/3007206471 conn(0x7f7f94070790 msgr2=0x7f7f94070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 -- 172.21.15.157:0/3007206471 shutdown_connections 2023-11-06T03:04:49.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.384+0000 7f7f9c176640 1 -- 172.21.15.157:0/3007206471 wait complete. 2023-11-06T03:04:49.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 Processor -- start 2023-11-06T03:04:49.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- start start 2023-11-06T03:04:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f9414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f9415caa0 0x7f7f940c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94155030 con 0x7f7f9415ce80 2023-11-06T03:04:49.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c95a0 con 0x7f7f9415caa0 2023-11-06T03:04:49.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94155240 con 0x7f7f940ce1a0 2023-11-06T03:04:49.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44034/0 (socket says 172.21.15.157:44034) 2023-11-06T03:04:49.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 -- 172.21.15.157:0/1966522088 learned_addr learned my addr 172.21.15.157:0/1966522088 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f99f12640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f9414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f99711640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f9415caa0 0x7f7f940c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 -- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 msgr2=0x7f7f9414bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f9414bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 -- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f9415caa0 msgr2=0x7f7f940c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f9415caa0 0x7f7f940c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 -- 172.21.15.157:0/1966522088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f840094d0 con 0x7f7f9415ce80 2023-11-06T03:04:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9a713640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00eed0 tx=0x7f7f8c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c010040 con 0x7f7f9415ce80 2023-11-06T03:04:49.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00bd30 con 0x7f7f9415ce80 2023-11-06T03:04:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f94155520 con 0x7f7f9415ce80 2023-11-06T03:04:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00bea0 con 0x7f7f9415ce80 2023-11-06T03:04:49.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f940c9230 con 0x7f7f9415ce80 2023-11-06T03:04:49.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.388+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f940c8db0 con 0x7f7f9415ce80 2023-11-06T03:04:49.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.392+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7f8c014640 con 0x7f7f9415ce80 2023-11-06T03:04:49.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.392+0000 7f7f8a7fc640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.392+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7f8c083fe0 con 0x7f7f9415ce80 2023-11-06T03:04:49.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.392+0000 7f7f99f12640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.392+0000 7f7f99f12640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f94152ef0 tx=0x7f7f90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.396+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7f8c0518e0 con 0x7f7f9415ce80 2023-11-06T03:04:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.548+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f940c8db0 con 0x7f7f9415ce80 2023-11-06T03:04:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.548+0000 7f7f8a7fc640 1 -- 172.21.15.157:0/1966522088 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f940c8db0 con 0x7f7f9415ce80 2023-11-06T03:04:49.550 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:49.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 msgr2=0x7f7f78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f94152ef0 tx=0x7f7f90008040 comp rx=0 tx=0).stop 2023-11-06T03:04:49.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 msgr2=0x7f7f941546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00eed0 tx=0x7f7f8c00b480 comp rx=0 tx=0).stop 2023-11-06T03:04:49.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 shutdown_connections 2023-11-06T03:04:49.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f9415ce80 0x7f7f941546b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f9415caa0 0x7f7f940c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 --2- 172.21.15.157:0/1966522088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f940ce1a0 0x7f7f9414bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 >> 172.21.15.157:0/1966522088 conn(0x7f7f94070790 msgr2=0x7f7f94155850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 shutdown_connections 2023-11-06T03:04:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.552+0000 7f7f9c176640 1 -- 172.21.15.157:0/1966522088 wait complete. 2023-11-06T03:04:49.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:49.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 Processor -- start 2023-11-06T03:04:49.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 -- start start 2023-11-06T03:04:49.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b081551e0 0x7f2b0806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 0x7f2b080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b0815f580 con 0x7f2b0815ea60 2023-11-06T03:04:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0815efc0 con 0x7f2b081551e0 2023-11-06T03:04:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0cfef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0815f2a0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b05d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 0x7f2b080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0659c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2b081551e0 0x7f2b0806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b05d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 0x7f2b080b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37336/0 (socket says 172.21.15.157:37336) 2023-11-06T03:04:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b05d9b640 1 -- 172.21.15.157:0/2921469997 learned_addr learned my addr 172.21.15.157:0/2921469997 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b06d9d640 1 --1- 172.21.15.157:0/2921469997 >> v1:172.21.15.195:6789/0 conn(0x7f2b0815ea60 0x7f2b0815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35728/0 (socket says 172.21.15.157:35728) 2023-11-06T03:04:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0659c640 1 -- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 msgr2=0x7f2b080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0659c640 1 --2- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 0x7f2b080b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0659c640 1 -- 172.21.15.157:0/2921469997 >> v1:172.21.15.195:6789/0 conn(0x7f2b0815ea60 legacy=0x7f2b0815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.688+0000 7f2b0659c640 1 -- 172.21.15.157:0/2921469997 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0806f5a0 con 0x7f2b081551e0 2023-11-06T03:04:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0659c640 1 --2- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b081551e0 0x7f2b0806fda0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b000097c0 tx=0x7f2b0001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeeb0762d4ba8c3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0559a640 1 -- 172.21.15.157:0/2921469997 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000ab600 con 0x7f2b081551e0 2023-11-06T03:04:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0559a640 1 -- 172.21.15.157:0/2921469997 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b000a7050 con 0x7f2b081551e0 2023-11-06T03:04:49.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0559a640 1 -- 172.21.15.157:0/2921469997 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000ab980 con 0x7f2b081551e0 2023-11-06T03:04:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b081551e0 msgr2=0x7f2b0806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b081551e0 0x7f2b0806fda0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b000097c0 tx=0x7f2b0001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- 172.21.15.157:0/2921469997 shutdown_connections 2023-11-06T03:04:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b0815e5e0 0x7f2b080b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/2921469997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b081551e0 0x7f2b0806fda0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- 172.21.15.157:0/2921469997 >> 172.21.15.157:0/2921469997 conn(0x7f2b08065930 msgr2=0x7f2b08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- 172.21.15.157:0/2921469997 shutdown_connections 2023-11-06T03:04:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- 172.21.15.157:0/2921469997 wait complete. 2023-11-06T03:04:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 Processor -- start 2023-11-06T03:04:49.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.692+0000 7f2b0cfef640 1 -- start start 2023-11-06T03:04:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b081551e0 0x7f2b081474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 0x7f2b08154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08149280 con 0x7f2b0815e5e0 2023-11-06T03:04:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08150210 con 0x7f2b0815ea60 2023-11-06T03:04:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b081504f0 con 0x7f2b081551e0 2023-11-06T03:04:49.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44048/0 (socket says 172.21.15.157:44048) 2023-11-06T03:04:49.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 -- 172.21.15.157:0/1740008371 learned_addr learned my addr 172.21.15.157:0/1740008371 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b06d9d640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 0x7f2b08154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0659c640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b081551e0 0x7f2b081474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 -- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b081551e0 msgr2=0x7f2b081474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b081551e0 0x7f2b081474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 -- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 msgr2=0x7f2b08154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 0x7f2b08154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 -- 172.21.15.157:0/1740008371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00009420 con 0x7f2b0815e5e0 2023-11-06T03:04:49.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b06d9d640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 0x7f2b08154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:49.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b05d9b640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e890 tx=0x7f2af000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0009620 con 0x7f2b0815e5e0 2023-11-06T03:04:49.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000c940 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b081494b0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ccf0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b081507d0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b08072750 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2af0013470 con 0x7f2b0815e5e0 2023-11-06T03:04:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 0x7f2ae8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2af0083550 con 0x7f2b0815e5e0 2023-11-06T03:04:49.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.696+0000 7f2b0659c640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 0x7f2ae8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.700+0000 7f2b0659c640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 0x7f2ae8067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b000ad000 tx=0x7f2b00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.700+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2af0050cf0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.844+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b08147fb0 con 0x7f2b0815e5e0 2023-11-06T03:04:49.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.844+0000 7f2afeffd640 1 -- 172.21.15.157:0/1740008371 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2af0054310 con 0x7f2b0815e5e0 2023-11-06T03:04:49.845 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:49.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 msgr2=0x7f2ae8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 0x7f2ae8067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2b000ad000 tx=0x7f2b00008040 comp rx=0 tx=0).stop 2023-11-06T03:04:49.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 msgr2=0x7f2b0814fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e890 tx=0x7f2af000ed50 comp rx=0 tx=0).stop 2023-11-06T03:04:49.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 shutdown_connections 2023-11-06T03:04:49.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ae8065a60 0x7f2ae8067f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2b0815ea60 0x7f2b08154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b0815e5e0 0x7f2b0814fb10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 --2- 172.21.15.157:0/1740008371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2b081551e0 0x7f2b081474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 >> 172.21.15.157:0/1740008371 conn(0x7f2b08065930 msgr2=0x7f2b080ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 shutdown_connections 2023-11-06T03:04:49.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.848+0000 7f2b0cfef640 1 -- 172.21.15.157:0/1740008371 wait complete. 2023-11-06T03:04:49.863 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:49.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 Processor -- start 2023-11-06T03:04:49.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 -- start start 2023-11-06T03:04:49.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f096c146370 0x7f096c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 0x7f096c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0cf1a0 con 0x7f096c06b6a0 2023-11-06T03:04:49.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c06c3d0 con 0x7f096c146370 2023-11-06T03:04:49.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0972a54640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c06f5a0 con 0x7f096c0ce800 2023-11-06T03:04:49.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f096c146370 0x7f096c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f096c146370 0x7f096c0b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44050/0 (socket says 172.21.15.157:44050) 2023-11-06T03:04:49.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 -- 172.21.15.157:0/1513705410 learned_addr learned my addr 172.21.15.157:0/1513705410 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0963fff640 1 --2- 172.21.15.157:0/1513705410 >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 0x7f096c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 -- 172.21.15.157:0/1513705410 >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 msgr2=0x7f096c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 --2- 172.21.15.157:0/1513705410 >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 0x7f096c0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 -- 172.21.15.157:0/1513705410 >> v1:172.21.15.195:6789/0 conn(0x7f096c06b6a0 legacy=0x7f096c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:49.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f096bfff640 1 -- 172.21.15.157:0/1513705410 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c06f880 con 0x7f096c146370 2023-11-06T03:04:49.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.976+0000 7f0963fff640 1 --2- 172.21.15.157:0/1513705410 >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 0x7f096c0cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:49.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f096bfff640 1 --2- 172.21.15.157:0/1513705410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c146370 0x7f096c0b1a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f095c009870 tx=0x7f095c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea412b63139045 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f096b7fe640 1 -- 172.21.15.157:0/1513705410 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0ab600 con 0x7f096c146370 2023-11-06T03:04:49.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f096b7fe640 1 -- 172.21.15.157:0/1513705410 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c0ab760 con 0x7f096c146370 2023-11-06T03:04:49.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f096b7fe640 1 -- 172.21.15.157:0/1513705410 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0abaf0 con 0x7f096c146370 2023-11-06T03:04:49.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- 172.21.15.157:0/1513705410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c146370 msgr2=0x7f096c0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 --2- 172.21.15.157:0/1513705410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c146370 0x7f096c0b1a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f095c009870 tx=0x7f095c01c400 comp rx=0 tx=0).stop 2023-11-06T03:04:49.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- 172.21.15.157:0/1513705410 shutdown_connections 2023-11-06T03:04:49.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 --2- 172.21.15.157:0/1513705410 >> v2:172.21.15.195:3300/0 conn(0x7f096c0ce800 0x7f096c0cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 --2- 172.21.15.157:0/1513705410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c146370 0x7f096c0b1a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- 172.21.15.157:0/1513705410 >> 172.21.15.157:0/1513705410 conn(0x7f096c065930 msgr2=0x7f096c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:49.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- 172.21.15.157:0/1513705410 shutdown_connections 2023-11-06T03:04:49.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- 172.21.15.157:0/1513705410 wait complete. 2023-11-06T03:04:49.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 Processor -- start 2023-11-06T03:04:49.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.980+0000 7f0972a54640 1 -- start start 2023-11-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096c0ce800 0x7f096c159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f096c146370 0x7f096c13ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c144170 con 0x7f096c06b6a0 2023-11-06T03:04:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15a370 con 0x7f096c0ce800 2023-11-06T03:04:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c159db0 con 0x7f096c146370 2023-11-06T03:04:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44064/0 (socket says 172.21.15.157:44064) 2023-11-06T03:04:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 -- 172.21.15.157:0/3741297955 learned_addr learned my addr 172.21.15.157:0/3741297955 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:49.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0963fff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096c0ce800 0x7f096c159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0970ff1640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f096c146370 0x7f096c13ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 -- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f096c146370 msgr2=0x7f096c13ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f096c146370 0x7f096c13ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 -- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096c0ce800 msgr2=0x7f096c159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:49.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096c0ce800 0x7f096c159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:49.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 -- 172.21.15.157:0/3741297955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0094d0 con 0x7f096c06b6a0 2023-11-06T03:04:49.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f096bfff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f095c0ad040 tx=0x7f095c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0b4800 con 0x7f096c06b6a0 2023-11-06T03:04:49.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c0b4960 con 0x7f096c06b6a0 2023-11-06T03:04:49.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c15a090 con 0x7f096c06b6a0 2023-11-06T03:04:49.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0b4cf0 con 0x7f096c06b6a0 2023-11-06T03:04:49.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c13a7a0 con 0x7f096c06b6a0 2023-11-06T03:04:49.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c0b1c10 con 0x7f096c06b6a0 2023-11-06T03:04:49.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f095c0c7440 con 0x7f096c06b6a0 2023-11-06T03:04:49.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 0x7f0950067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:49.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f095c11df50 con 0x7f096c06b6a0 2023-11-06T03:04:49.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.984+0000 7f0963fff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 0x7f0950067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:49.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.988+0000 7f0963fff640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 0x7f0950067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0954006fd0 tx=0x7f0954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:49.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:49.988+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f095c0eb800 con 0x7f096c06b6a0 2023-11-06T03:04:50.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.144+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f096c072750 con 0x7f096c06b6a0 2023-11-06T03:04:50.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.144+0000 7f09697fa640 1 -- 172.21.15.157:0/3741297955 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f095c0eee20 con 0x7f096c06b6a0 2023-11-06T03:04:50.145 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:50.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 msgr2=0x7f0950067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 0x7f0950067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0954006fd0 tx=0x7f0954008040 comp rx=0 tx=0).stop 2023-11-06T03:04:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 msgr2=0x7f096c13a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f095c0ad040 tx=0x7f095c01ac40 comp rx=0 tx=0).stop 2023-11-06T03:04:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 shutdown_connections 2023-11-06T03:04:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0950065a60 0x7f0950067f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f096c146370 0x7f096c13ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096c0ce800 0x7f096c159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 --2- 172.21.15.157:0/3741297955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096c06b6a0 0x7f096c13a260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 >> 172.21.15.157:0/3741297955 conn(0x7f096c065930 msgr2=0x7f096c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 shutdown_connections 2023-11-06T03:04:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.148+0000 7f0972a54640 1 -- 172.21.15.157:0/3741297955 wait complete. 2023-11-06T03:04:50.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.276+0000 7f33970a8640 1 Processor -- start 2023-11-06T03:04:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.276+0000 7f33970a8640 1 -- start start 2023-11-06T03:04:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f33970a8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33901551e0 0x7f339006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f33970a8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f33900b3b20 0x7f339015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f33970a8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339015f580 con 0x7f339015e5e0 2023-11-06T03:04:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f33970a8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339015efc0 con 0x7f33901551e0 2023-11-06T03:04:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f33970a8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339015f2a0 con 0x7f33900b3b20 2023-11-06T03:04:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33901551e0 0x7f339006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f339015e5e0 0x7f33900b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50646/0 (socket says 172.21.15.157:50646) 2023-11-06T03:04:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338ffff640 1 -- 172.21.15.157:0/4159994656 learned_addr learned my addr 172.21.15.157:0/4159994656 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3395645640 1 --2- 172.21.15.157:0/4159994656 >> v2:172.21.15.195:3300/0 conn(0x7f33900b3b20 0x7f339015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 -- 172.21.15.157:0/4159994656 >> v2:172.21.15.195:3300/0 conn(0x7f33900b3b20 msgr2=0x7f339015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:50.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 --2- 172.21.15.157:0/4159994656 >> v2:172.21.15.195:3300/0 conn(0x7f33900b3b20 0x7f339015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 -- 172.21.15.157:0/4159994656 >> v1:172.21.15.157:6789/0 conn(0x7f339015e5e0 legacy=0x7f33900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 -- 172.21.15.157:0/4159994656 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339006f5a0 con 0x7f33901551e0 2023-11-06T03:04:50.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338f7fe640 1 -- 172.21.15.157:0/4159994656 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425585413 0 0) 0x7f339015f580 con 0x7f339015e5e0 2023-11-06T03:04:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f3394e44640 1 --2- 172.21.15.157:0/4159994656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33901551e0 0x7f339006fda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f33900cbfe0 tx=0x7f338001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f1cdb95c2f263a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338f7fe640 1 -- 172.21.15.157:0/4159994656 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800ab600 con 0x7f33901551e0 2023-11-06T03:04:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338f7fe640 1 -- 172.21.15.157:0/4159994656 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800a7050 con 0x7f33901551e0 2023-11-06T03:04:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.280+0000 7f338f7fe640 1 -- 172.21.15.157:0/4159994656 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800ab980 con 0x7f33901551e0 2023-11-06T03:04:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/4159994656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33901551e0 msgr2=0x7f339006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- 172.21.15.157:0/4159994656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33901551e0 0x7f339006fda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f33900cbfe0 tx=0x7f338001c310 comp rx=0 tx=0).stop 2023-11-06T03:04:50.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/4159994656 shutdown_connections 2023-11-06T03:04:50.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- 172.21.15.157:0/4159994656 >> v2:172.21.15.195:3300/0 conn(0x7f33900b3b20 0x7f339015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- 172.21.15.157:0/4159994656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33901551e0 0x7f339006fda0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/4159994656 >> 172.21.15.157:0/4159994656 conn(0x7f3390065930 msgr2=0x7f3390065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/4159994656 shutdown_connections 2023-11-06T03:04:50.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/4159994656 wait complete. 2023-11-06T03:04:50.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 Processor -- start 2023-11-06T03:04:50.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- start start 2023-11-06T03:04:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33900b3b20 0x7f339014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33901551e0 0x7f3390153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901546b0 con 0x7f339015e5e0 2023-11-06T03:04:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3390156610 con 0x7f33900b3b20 2023-11-06T03:04:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33901568f0 con 0x7f33901551e0 2023-11-06T03:04:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44096/0 (socket says 172.21.15.157:44096) 2023-11-06T03:04:50.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 -- 172.21.15.157:0/3859321389 learned_addr learned my addr 172.21.15.157:0/3859321389 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3394e44640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33900b3b20 0x7f339014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f338ffff640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33901551e0 0x7f3390153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 -- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33901551e0 msgr2=0x7f3390153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33901551e0 0x7f3390153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 -- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33900b3b20 msgr2=0x7f339014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33900b3b20 0x7f339014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 -- 172.21.15.157:0/3859321389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009420 con 0x7f339015e5e0 2023-11-06T03:04:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f3395645640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3388007e40 tx=0x7f338800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388014620 con 0x7f339015e5e0 2023-11-06T03:04:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338800ee90 con 0x7f339015e5e0 2023-11-06T03:04:50.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3390156bd0 con 0x7f339015e5e0 2023-11-06T03:04:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33880149e0 con 0x7f339015e5e0 2023-11-06T03:04:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33901584d0 con 0x7f339015e5e0 2023-11-06T03:04:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.284+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339015abf0 con 0x7f339015e5e0 2023-11-06T03:04:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.288+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3388014b40 con 0x7f339015e5e0 2023-11-06T03:04:50.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.288+0000 7f338d7fa640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 0x7f3370067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.288+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3388083c20 con 0x7f339015e5e0 2023-11-06T03:04:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.288+0000 7f3394e44640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 0x7f3370067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.288+0000 7f3394e44640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 0x7f3370067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f33800ad000 tx=0x7f33800a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.292+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33880514d0 con 0x7f339015e5e0 2023-11-06T03:04:50.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.432+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3390072750 con 0x7f339015e5e0 2023-11-06T03:04:50.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.432+0000 7f338d7fa640 1 -- 172.21.15.157:0/3859321389 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3388054af0 con 0x7f339015e5e0 2023-11-06T03:04:50.433 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:50.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.432+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 msgr2=0x7f3370067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.432+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 0x7f3370067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f33800ad000 tx=0x7f33800a6000 comp rx=0 tx=0).stop 2023-11-06T03:04:50.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 msgr2=0x7f3390158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3388007e40 tx=0x7f338800e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 shutdown_connections 2023-11-06T03:04:50.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3370065a60 0x7f3370067f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f339015e5e0 0x7f3390158ad0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33901551e0 0x7f3390153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 --2- 172.21.15.157:0/3859321389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33900b3b20 0x7f339014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 >> 172.21.15.157:0/3859321389 conn(0x7f3390065930 msgr2=0x7f339014e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 shutdown_connections 2023-11-06T03:04:50.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.436+0000 7f33970a8640 1 -- 172.21.15.157:0/3859321389 wait complete. 2023-11-06T03:04:50.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:50.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 Processor -- start 2023-11-06T03:04:50.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 -- start start 2023-11-06T03:04:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f390c0b2b10 0x7f390c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0cc2b0 con 0x7f390c15ac80 2023-11-06T03:04:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c0b01a0 con 0x7f390c0b2b10 2023-11-06T03:04:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3911656640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0cfb10 con 0x7f390c0cd730 2023-11-06T03:04:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f390affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f390c15ac80 0x7f390c0b2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50656/0 (socket says 172.21.15.157:50656) 2023-11-06T03:04:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f390affd640 1 -- 172.21.15.157:0/1978352544 learned_addr learned my addr 172.21.15.157:0/1978352544 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3903fff640 1 --2- 172.21.15.157:0/1978352544 >> v2:172.21.15.157:3300/0 conn(0x7f390c0b2b10 0x7f390c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f390b7fe640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f390a7fc640 1 -- 172.21.15.157:0/1978352544 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660786557 0 0) 0x7f390c0cc2b0 con 0x7f390c15ac80 2023-11-06T03:04:50.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f390a7fc640 1 -- 172.21.15.157:0/1978352544 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f8003530 con 0x7f390c15ac80 2023-11-06T03:04:50.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3903fff640 1 -- 172.21.15.157:0/1978352544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 msgr2=0x7f390c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:50.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3903fff640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3903fff640 1 -- 172.21.15.157:0/1978352544 >> v1:172.21.15.157:6789/0 conn(0x7f390c15ac80 legacy=0x7f390c0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.568+0000 7f3903fff640 1 -- 172.21.15.157:0/1978352544 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0cfdf0 con 0x7f390c0b2b10 2023-11-06T03:04:50.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3903fff640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c0b2b10 0x7f390c0ce550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f38f40097c0 tx=0x7f38f401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25083d44ebbc6c95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f390a7fc640 1 -- 172.21.15.157:0/1978352544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40ab680 con 0x7f390c0b2b10 2023-11-06T03:04:50.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f390a7fc640 1 -- 172.21.15.157:0/1978352544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f40ab7e0 con 0x7f390c0b2b10 2023-11-06T03:04:50.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f390a7fc640 1 -- 172.21.15.157:0/1978352544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40abb70 con 0x7f390c0b2b10 2023-11-06T03:04:50.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- 172.21.15.157:0/1978352544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c0b2b10 msgr2=0x7f390c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c0b2b10 0x7f390c0ce550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f38f40097c0 tx=0x7f38f401c480 comp rx=0 tx=0).stop 2023-11-06T03:04:50.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- 172.21.15.157:0/1978352544 shutdown_connections 2023-11-06T03:04:50.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 --2- 172.21.15.157:0/1978352544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c0b2b10 0x7f390c0ce550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- 172.21.15.157:0/1978352544 >> 172.21.15.157:0/1978352544 conn(0x7f390c06d560 msgr2=0x7f390c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- 172.21.15.157:0/1978352544 shutdown_connections 2023-11-06T03:04:50.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- 172.21.15.157:0/1978352544 wait complete. 2023-11-06T03:04:50.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 Processor -- start 2023-11-06T03:04:50.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.572+0000 7f3911656640 1 -- start start 2023-11-06T03:04:50.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390c0b2b10 0x7f390c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c158400 con 0x7f390c15ac80 2023-11-06T03:04:50.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c157e20 con 0x7f390c0b2b10 2023-11-06T03:04:50.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c159b50 con 0x7f390c0cd730 2023-11-06T03:04:50.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44106/0 (socket says 172.21.15.157:44106) 2023-11-06T03:04:50.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 -- 172.21.15.157:0/242290069 learned_addr learned my addr 172.21.15.157:0/242290069 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390affd640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390c0b2b10 0x7f390c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3903fff640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 -- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 msgr2=0x7f390c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 -- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390c0b2b10 msgr2=0x7f390c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390c0b2b10 0x7f390c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 -- 172.21.15.157:0/242290069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009420 con 0x7f390c15ac80 2023-11-06T03:04:50.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390b7fe640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3904007e30 tx=0x7f390400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904014620 con 0x7f390c15ac80 2023-11-06T03:04:50.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390400ee90 con 0x7f390c15ac80 2023-11-06T03:04:50.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c159de0 con 0x7f390c15ac80 2023-11-06T03:04:50.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39040149c0 con 0x7f390c15ac80 2023-11-06T03:04:50.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c15a0c0 con 0x7f390c15ac80 2023-11-06T03:04:50.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c158570 con 0x7f390c15ac80 2023-11-06T03:04:50.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3904014b20 con 0x7f390c15ac80 2023-11-06T03:04:50.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 0x7f38ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3904083d80 con 0x7f390c15ac80 2023-11-06T03:04:50.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390affd640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 0x7f38ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.576+0000 7f390affd640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 0x7f38ec067f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0011f0 tx=0x7f38fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.580+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3904051510 con 0x7f390c15ac80 2023-11-06T03:04:50.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.732+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f390c072750 con 0x7f390c15ac80 2023-11-06T03:04:50.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f39037fe640 1 -- 172.21.15.157:0/242290069 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3904054b30 con 0x7f390c15ac80 2023-11-06T03:04:50.735 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:50.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 msgr2=0x7f38ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 0x7f38ec067f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0011f0 tx=0x7f38fc009040 comp rx=0 tx=0).stop 2023-11-06T03:04:50.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 msgr2=0x7f390c149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3904007e30 tx=0x7f390400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 shutdown_connections 2023-11-06T03:04:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38ec065a60 0x7f38ec067f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f390c15ac80 0x7f390c149ac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f390c0cd730 0x7f390c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 --2- 172.21.15.157:0/242290069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f390c0b2b10 0x7f390c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 >> 172.21.15.157:0/242290069 conn(0x7f390c06d560 msgr2=0x7f390c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 shutdown_connections 2023-11-06T03:04:50.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.736+0000 7f3911656640 1 -- 172.21.15.157:0/242290069 wait complete. 2023-11-06T03:04:50.752 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:50.753 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:50.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.880+0000 7f6c41236640 1 Processor -- start 2023-11-06T03:04:50.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.880+0000 7f6c41236640 1 -- start start 2023-11-06T03:04:50.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c41236640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c3c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c41236640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c15e440 con 0x7f6c3c1599c0 2023-11-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c41236640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c06c240 con 0x7f6c3c0b2af0 2023-11-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c41236640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06c520 con 0x7f6c3c06d000 2023-11-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6c3c1599c0 0x7f6c3c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50668/0 (socket says 172.21.15.157:50668) 2023-11-06T03:04:50.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3ad9d640 1 -- 172.21.15.157:0/905794352 learned_addr learned my addr 172.21.15.157:0/905794352 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:50.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c3c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071963932 0 0) 0x7f6c3c15e440 con 0x7f6c3c1599c0 2023-11-06T03:04:50.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c24003530 con 0x7f6c3c1599c0 2023-11-06T03:04:50.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333542208 0 0) 0x7f6c24003530 con 0x7f6c3c1599c0 2023-11-06T03:04:50.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 msgr2=0x7f6c3c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:50.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c3c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 >> v1:172.21.15.195:6789/0 conn(0x7f6c3c0b2af0 legacy=0x7f6c3c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c06c800 con 0x7f6c3c1599c0 2023-11-06T03:04:50.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6c2c001680 con 0x7f6c3c1599c0 2023-11-06T03:04:50.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 >> v1:172.21.15.157:6789/0 conn(0x7f6c3c1599c0 legacy=0x7f6c3c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c24003f30 0x7f6c240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c24004d10 0x7f6c24005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c06c520 con 0x7f6c24004d10 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400ba70 con 0x7f6c24003f30 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400bc30 con 0x7f6c3c06d000 2023-11-06T03:04:50.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c24004d10 0x7f6c24005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3ad9d640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3a59c640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c24003f30 0x7f6c240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 -- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 msgr2=0x7f6c240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:50.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c3c06d000 0x7f6c240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 -- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c24003f30 msgr2=0x7f6c240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:50.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c24003f30 0x7f6c240047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 -- 172.21.15.157:0/905794352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24003530 con 0x7f6c24004d10 2023-11-06T03:04:50.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.884+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c24004d10 0x7f6c24005190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009f90 tx=0x7f6c3401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce35f223b6838cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340a5c30 con 0x7f6c24004d10 2023-11-06T03:04:50.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3400f040 con 0x7f6c24004d10 2023-11-06T03:04:50.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c39d9b640 1 -- 172.21.15.157:0/905794352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340ad440 con 0x7f6c24004d10 2023-11-06T03:04:50.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/905794352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c24004d10 msgr2=0x7f6c24005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 --2- 172.21.15.157:0/905794352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c24004d10 0x7f6c24005190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009f90 tx=0x7f6c3401cc20 comp rx=0 tx=0).stop 2023-11-06T03:04:50.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 -- 172.21.15.157:0/905794352 reap_dead start 2023-11-06T03:04:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/905794352 shutdown_connections 2023-11-06T03:04:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/905794352 >> 172.21.15.157:0/905794352 conn(0x7f6c3c065930 msgr2=0x7f6c3c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/905794352 shutdown_connections 2023-11-06T03:04:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/905794352 wait complete. 2023-11-06T03:04:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 Processor -- start 2023-11-06T03:04:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- start start 2023-11-06T03:04:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c24004d10 0x7f6c3c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c3c14d970 0x7f6c3c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15a5e0 con 0x7f6c3c15aa00 2023-11-06T03:04:50.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15c0b0 con 0x7f6c3c14d970 2023-11-06T03:04:50.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15bdd0 con 0x7f6c24004d10 2023-11-06T03:04:50.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44128/0 (socket says 172.21.15.157:44128) 2023-11-06T03:04:50.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 -- 172.21.15.157:0/2245878653 learned_addr learned my addr 172.21.15.157:0/2245878653 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:50.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3a59c640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c3c14d970 0x7f6c3c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3ad9d640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c24004d10 0x7f6c3c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 -- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c24004d10 msgr2=0x7f6c3c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c24004d10 0x7f6c3c14d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 -- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c3c14d970 msgr2=0x7f6c3c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c3c14d970 0x7f6c3c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 -- 172.21.15.157:0/2245878653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c159f20 con 0x7f6c3c15aa00 2023-11-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c3b59e640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b0040 tx=0x7f6c34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34003ba0 con 0x7f6c3c15aa00 2023-11-06T03:04:50.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34003d00 con 0x7f6c3c15aa00 2023-11-06T03:04:50.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c15a200 con 0x7f6c3c15aa00 2023-11-06T03:04:50.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340ada80 con 0x7f6c3c15aa00 2023-11-06T03:04:50.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.888+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c15ba10 con 0x7f6c3c15aa00 2023-11-06T03:04:50.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c15a720 con 0x7f6c3c15aa00 2023-11-06T03:04:50.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c340b4040 con 0x7f6c3c15aa00 2023-11-06T03:04:50.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c237fe640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 0x7f6c28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:50.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6c3411d340 con 0x7f6c3c15aa00 2023-11-06T03:04:50.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c3ad9d640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 0x7f6c28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:50.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.892+0000 7f6c3ad9d640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 0x7f6c28067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001c90 tx=0x7f6c2c001d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:50.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:50.896+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c3c15a720 con 0x7f6c3c15aa00 2023-11-06T03:04:51.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.040+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c3c15a720 con 0x7f6c3c15aa00 2023-11-06T03:04:51.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c237fe640 1 -- 172.21.15.157:0/2245878653 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f6c3c15a720 con 0x7f6c3c15aa00 2023-11-06T03:04:51.044 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:51.044 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 msgr2=0x7f6c28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 0x7f6c28067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001c90 tx=0x7f6c2c001d60 comp rx=0 tx=0).stop 2023-11-06T03:04:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 msgr2=0x7f6c3c15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b0040 tx=0x7f6c34009750 comp rx=0 tx=0).stop 2023-11-06T03:04:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 shutdown_connections 2023-11-06T03:04:51.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c28065a60 0x7f6c28067f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c3c15aa00 0x7f6c3c15ae90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c3c14d970 0x7f6c3c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 --2- 172.21.15.157:0/2245878653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c24004d10 0x7f6c3c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 >> 172.21.15.157:0/2245878653 conn(0x7f6c3c065930 msgr2=0x7f6c3c1546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 shutdown_connections 2023-11-06T03:04:51.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.044+0000 7f6c41236640 1 -- 172.21.15.157:0/2245878653 wait complete. 2023-11-06T03:04:51.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:51.063 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:51.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.184+0000 7f6a964b1640 1 Processor -- start 2023-11-06T03:04:51.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.184+0000 7f6a964b1640 1 -- start start 2023-11-06T03:04:51.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a964b1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a901599c0 0x7f6a900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a964b1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6a900b2af0 0x7f6a90159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a964b1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9015e440 con 0x7f6a9006d000 2023-11-06T03:04:51.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a964b1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9006c240 con 0x7f6a901599c0 2023-11-06T03:04:51.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a964b1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9006c520 con 0x7f6a900b2af0 2023-11-06T03:04:51.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a901599c0 0x7f6a900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6a901599c0 0x7f6a900ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44142/0 (socket says 172.21.15.157:44142) 2023-11-06T03:04:51.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3727262138 learned_addr learned my addr 172.21.15.157:0/3727262138 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:51.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3727262138 >> v2:172.21.15.195:3300/0 conn(0x7f6a900b2af0 msgr2=0x7f6a90159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 --2- 172.21.15.157:0/3727262138 >> v2:172.21.15.195:3300/0 conn(0x7f6a900b2af0 0x7f6a90159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3727262138 >> v1:172.21.15.195:6789/0 conn(0x7f6a9006d000 legacy=0x7f6a900b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3727262138 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9006c800 con 0x7f6a901599c0 2023-11-06T03:04:51.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8ffff640 1 --2- 172.21.15.157:0/3727262138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a901599c0 0x7f6a900ae000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a900b19a0 tx=0x7f6a8001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb899f5a42567f48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8f7fe640 1 -- 172.21.15.157:0/3727262138 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800aa610 con 0x7f6a901599c0 2023-11-06T03:04:51.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8f7fe640 1 -- 172.21.15.157:0/3727262138 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a800a6040 con 0x7f6a901599c0 2023-11-06T03:04:51.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.188+0000 7f6a8f7fe640 1 -- 172.21.15.157:0/3727262138 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800aa990 con 0x7f6a901599c0 2023-11-06T03:04:51.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3727262138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a901599c0 msgr2=0x7f6a900ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3727262138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a901599c0 0x7f6a900ae000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a900b19a0 tx=0x7f6a8001c490 comp rx=0 tx=0).stop 2023-11-06T03:04:51.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3727262138 shutdown_connections 2023-11-06T03:04:51.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3727262138 >> v2:172.21.15.195:3300/0 conn(0x7f6a900b2af0 0x7f6a90159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3727262138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a901599c0 0x7f6a900ae000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3727262138 >> 172.21.15.157:0/3727262138 conn(0x7f6a90065930 msgr2=0x7f6a90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3727262138 shutdown_connections 2023-11-06T03:04:51.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3727262138 wait complete. 2023-11-06T03:04:51.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 Processor -- start 2023-11-06T03:04:51.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- start start 2023-11-06T03:04:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a900b2af0 0x7f6a900cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a901599c0 0x7f6a900ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a900ae1f0 con 0x7f6a9006d000 2023-11-06T03:04:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9015f560 con 0x7f6a900b2af0 2023-11-06T03:04:51.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901f6350 con 0x7f6a901599c0 2023-11-06T03:04:51.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44144/0 (socket says 172.21.15.157:44144) 2023-11-06T03:04:51.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3482304807 learned_addr learned my addr 172.21.15.157:0/3482304807 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:51.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a877fe640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a900b2af0 0x7f6a900cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a94a4e640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a901599c0 0x7f6a900ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a901599c0 msgr2=0x7f6a900ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a901599c0 0x7f6a900ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a900b2af0 msgr2=0x7f6a900cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a900b2af0 0x7f6a900cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 -- 172.21.15.157:0/3482304807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800094d0 con 0x7f6a9006d000 2023-11-06T03:04:51.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8ffff640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a8001a0c0 tx=0x7f6a8001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800bd5f0 con 0x7f6a9006d000 2023-11-06T03:04:51.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a800bd770 con 0x7f6a9006d000 2023-11-06T03:04:51.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a901f5ab0 con 0x7f6a9006d000 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800bdb20 con 0x7f6a9006d000 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.192+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a901f6070 con 0x7f6a9006d000 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a900ae330 con 0x7f6a9006d000 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a800b33f0 con 0x7f6a9006d000 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a8d7fa640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 0x7f6a70067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6a8011e000 con 0x7f6a9006d000 2023-11-06T03:04:51.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a877fe640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 0x7f6a70067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.196+0000 7f6a877fe640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 0x7f6a70067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a901f8900 tx=0x7f6a78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.200+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6a900ae330 con 0x7f6a9006d000 2023-11-06T03:04:51.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a90072750 con 0x7f6a9006d000 2023-11-06T03:04:51.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a8d7fa640 1 -- 172.21.15.157:0/3482304807 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a800eb960 con 0x7f6a9006d000 2023-11-06T03:04:51.343 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 msgr2=0x7f6a70067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 0x7f6a70067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6a901f8900 tx=0x7f6a78009040 comp rx=0 tx=0).stop 2023-11-06T03:04:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 msgr2=0x7f6a9015f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a8001a0c0 tx=0x7f6a8001acd0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 shutdown_connections 2023-11-06T03:04:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6a70065a60 0x7f6a70067f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6a901599c0 0x7f6a900ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6a900b2af0 0x7f6a900cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 --2- 172.21.15.157:0/3482304807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a9006d000 0x7f6a9015f020 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 >> 172.21.15.157:0/3482304807 conn(0x7f6a90065930 msgr2=0x7f6a9006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 shutdown_connections 2023-11-06T03:04:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.344+0000 7f6a964b1640 1 -- 172.21.15.157:0/3482304807 wait complete. 2023-11-06T03:04:51.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:51.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 Processor -- start 2023-11-06T03:04:51.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 -- start start 2023-11-06T03:04:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57fc15aab0 0x7f57fc0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 0x7f57fc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc15e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc0ce260 con 0x7f57fc15aab0 2023-11-06T03:04:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc15ae90 con 0x7f57fc0cb490 2023-11-06T03:04:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f58020e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0cc7f0 con 0x7f57fc0cc2c0 2023-11-06T03:04:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57fc15aab0 0x7f57fc0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57fc15aab0 0x7f57fc0cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44154/0 (socket says 172.21.15.157:44154) 2023-11-06T03:04:51.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 -- 172.21.15.157:0/1096438947 learned_addr learned my addr 172.21.15.157:0/1096438947 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:51.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fbfff640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57faffd640 1 --2- 172.21.15.157:0/1096438947 >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 0x7f57fc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 -- 172.21.15.157:0/1096438947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 msgr2=0x7f57fc15e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc15e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 -- 172.21.15.157:0/1096438947 >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 msgr2=0x7f57fc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/1096438947 >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 0x7f57fc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57fb7fe640 1 -- 172.21.15.157:0/1096438947 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0cd930 con 0x7f57fc15aab0 2023-11-06T03:04:51.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.484+0000 7f57faffd640 1 --2- 172.21.15.157:0/1096438947 >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 0x7f57fc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:51.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc0cef90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57fc072f50 tx=0x7f57ec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e405cdc6e2176fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fa7fc640 1 -- 172.21.15.157:0/1096438947 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0aa5f0 con 0x7f57fc15aab0 2023-11-06T03:04:51.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fa7fc640 1 -- 172.21.15.157:0/1096438947 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec0aa750 con 0x7f57fc15aab0 2023-11-06T03:04:51.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fa7fc640 1 -- 172.21.15.157:0/1096438947 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0aaae0 con 0x7f57fc15aab0 2023-11-06T03:04:51.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- 172.21.15.157:0/1096438947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 msgr2=0x7f57fc0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc0cef90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f57fc072f50 tx=0x7f57ec01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- 172.21.15.157:0/1096438947 shutdown_connections 2023-11-06T03:04:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc15e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- 172.21.15.157:0/1096438947 >> v2:172.21.15.195:3300/0 conn(0x7f57fc0cb490 0x7f57fc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- 172.21.15.157:0/1096438947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc0cef90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- 172.21.15.157:0/1096438947 >> 172.21.15.157:0/1096438947 conn(0x7f57fc06a4d0 msgr2=0x7f57fc06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- 172.21.15.157:0/1096438947 shutdown_connections 2023-11-06T03:04:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- 172.21.15.157:0/1096438947 wait complete. 2023-11-06T03:04:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 Processor -- start 2023-11-06T03:04:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- start start 2023-11-06T03:04:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 0x7f57fc150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0710a0 con 0x7f57fc15aab0 2023-11-06T03:04:51.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc070490 con 0x7f57fc0cb490 2023-11-06T03:04:51.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f58020e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc070770 con 0x7f57fc0cc2c0 2023-11-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44164/0 (socket says 172.21.15.157:44164) 2023-11-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57fbfff640 1 -- 172.21.15.157:0/3180505161 learned_addr learned my addr 172.21.15.157:0/3180505161 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.488+0000 7f57faffd640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 0x7f57fc150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 -- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 msgr2=0x7f57fc150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 -- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 msgr2=0x7f57fc150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 0x7f57fc150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 -- 172.21.15.157:0/3180505161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009480 con 0x7f57fc15aab0 2023-11-06T03:04:51.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 0x7f57fc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:51.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fbfff640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57f4007e30 tx=0x7f57f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4014620 con 0x7f57fc15aab0 2023-11-06T03:04:51.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f400ee90 con 0x7f57fc15aab0 2023-11-06T03:04:51.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc070a50 con 0x7f57fc15aab0 2023-11-06T03:04:51.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40149c0 con 0x7f57fc15aab0 2023-11-06T03:04:51.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc071a70 con 0x7f57fc15aab0 2023-11-06T03:04:51.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc071210 con 0x7f57fc15aab0 2023-11-06T03:04:51.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57f4014b20 con 0x7f57fc15aab0 2023-11-06T03:04:51.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 0x7f57e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f57f4083c90 con 0x7f57fc15aab0 2023-11-06T03:04:51.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 0x7f57e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.492+0000 7f57fb7fe640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 0x7f57e8067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae000 tx=0x7f57ec0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.496+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57f40514b0 con 0x7f57fc15aab0 2023-11-06T03:04:51.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f57fc072750 con 0x7f57fc15aab0 2023-11-06T03:04:51.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f57e3fff640 1 -- 172.21.15.157:0/3180505161 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f57f4054ad0 con 0x7f57fc15aab0 2023-11-06T03:04:51.644 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 msgr2=0x7f57e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 0x7f57e8067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ae000 tx=0x7f57ec0a7000 comp rx=0 tx=0).stop 2023-11-06T03:04:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 msgr2=0x7f57fc071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57f4007e30 tx=0x7f57f400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 shutdown_connections 2023-11-06T03:04:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57e8065a60 0x7f57e8067f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc15aab0 0x7f57fc071530 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57fc0cc2c0 0x7f57fc150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 --2- 172.21.15.157:0/3180505161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57fc0cb490 0x7f57fc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 >> 172.21.15.157:0/3180505161 conn(0x7f57fc06a4d0 msgr2=0x7f57fc0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 shutdown_connections 2023-11-06T03:04:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.644+0000 7f58020e3640 1 -- 172.21.15.157:0/3180505161 wait complete. 2023-11-06T03:04:51.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 Processor -- start 2023-11-06T03:04:51.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 -- start start 2023-11-06T03:04:51.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f494c161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494c06f470 con 0x7f494c0b1910 2023-11-06T03:04:51.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494c0b1df0 con 0x7f494c160db0 2023-11-06T03:04:51.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f4951e0f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c06f6c0 con 0x7f494c0ced70 2023-11-06T03:04:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.772+0000 7f494affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f494c0b1910 0x7f494c0ce660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50674/0 (socket says 172.21.15.157:50674) 2023-11-06T03:04:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494affd640 1 -- 172.21.15.157:0/730641947 learned_addr learned my addr 172.21.15.157:0/730641947 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275791942 0 0) 0x7f494c06f470 con 0x7f494c0b1910 2023-11-06T03:04:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494bfff640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f494c161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4934003530 con 0x7f494c0b1910 2023-11-06T03:04:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577304414 0 0) 0x7f4934003530 con 0x7f494c0b1910 2023-11-06T03:04:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 msgr2=0x7f494c161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f494c161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 >> v1:172.21.15.195:6789/0 conn(0x7f494c160db0 legacy=0x7f494c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c06f9a0 con 0x7f494c0b1910 2023-11-06T03:04:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4940001680 con 0x7f494c0b1910 2023-11-06T03:04:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 >> v1:172.21.15.157:6789/0 conn(0x7f494c0b1910 legacy=0x7f494c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49340039b0 0x7f4934003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f49340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4934004d10 0x7f4934005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c06f6c0 con 0x7f49340039b0 2023-11-06T03:04:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493400bb20 con 0x7f4934004d10 2023-11-06T03:04:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493400bce0 con 0x7f494c0ced70 2023-11-06T03:04:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49340039b0 0x7f4934003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494bfff640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4934004d10 0x7f4934005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494affd640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f49340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 -- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 msgr2=0x7f49340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f49340047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 -- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4934004d10 msgr2=0x7f4934005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4934004d10 0x7f4934005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 -- 172.21.15.157:0/730641947 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934003530 con 0x7f49340039b0 2023-11-06T03:04:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494affd640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c0ced70 0x7f49340047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:51.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494b7fe640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49340039b0 0x7f4934003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f493c001780 tx=0x7f493c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5700e3809d1932 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0052d0 con 0x7f49340039b0 2023-11-06T03:04:51.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c0a7050 con 0x7f49340039b0 2023-11-06T03:04:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494a7fc640 1 -- 172.21.15.157:0/730641947 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c003600 con 0x7f49340039b0 2023-11-06T03:04:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 -- 172.21.15.157:0/730641947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49340039b0 msgr2=0x7f4934003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 --2- 172.21.15.157:0/730641947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49340039b0 0x7f4934003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f493c001780 tx=0x7f493c01c400 comp rx=0 tx=0).stop 2023-11-06T03:04:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f494bfff640 1 -- 172.21.15.157:0/730641947 reap_dead start 2023-11-06T03:04:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 -- 172.21.15.157:0/730641947 shutdown_connections 2023-11-06T03:04:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 -- 172.21.15.157:0/730641947 >> 172.21.15.157:0/730641947 conn(0x7f494c065930 msgr2=0x7f494c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 -- 172.21.15.157:0/730641947 shutdown_connections 2023-11-06T03:04:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.776+0000 7f4951e0f640 1 -- 172.21.15.157:0/730641947 wait complete. 2023-11-06T03:04:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 Processor -- start 2023-11-06T03:04:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- start start 2023-11-06T03:04:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49340039b0 0x7f494c14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c1544c0 0x7f494c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c151cf0 con 0x7f494c14cbd0 2023-11-06T03:04:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c15e910 con 0x7f49340039b0 2023-11-06T03:04:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c154a30 con 0x7f494c1544c0 2023-11-06T03:04:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44168/0 (socket says 172.21.15.157:44168) 2023-11-06T03:04:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 -- 172.21.15.157:0/1816198708 learned_addr learned my addr 172.21.15.157:0/1816198708 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494b7fe640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49340039b0 0x7f494c14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494affd640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c1544c0 0x7f494c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 -- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c1544c0 msgr2=0x7f494c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c1544c0 0x7f494c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 -- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49340039b0 msgr2=0x7f494c14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49340039b0 0x7f494c14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 -- 172.21.15.157:0/1816198708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c154d10 con 0x7f494c14cbd0 2023-11-06T03:04:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f494bfff640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b3050 tx=0x7f494400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944017400 con 0x7f494c14cbd0 2023-11-06T03:04:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4944011040 con 0x7f494c14cbd0 2023-11-06T03:04:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c15ebb0 con 0x7f494c14cbd0 2023-11-06T03:04:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440177c0 con 0x7f494c14cbd0 2023-11-06T03:04:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c14c8d0 con 0x7f494c14cbd0 2023-11-06T03:04:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c072750 con 0x7f494c14cbd0 2023-11-06T03:04:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.780+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4944017920 con 0x7f494c14cbd0 2023-11-06T03:04:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.784+0000 7f4933fff640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 0x7f4938067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.784+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4944083600 con 0x7f494c14cbd0 2023-11-06T03:04:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.784+0000 7f494b7fe640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 0x7f4938067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.784+0000 7f494b7fe640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 0x7f4938067f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f493c001750 tx=0x7f493c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.788+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4944050eb0 con 0x7f494c14cbd0 2023-11-06T03:04:51.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.940+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f494c15a9b0 con 0x7f494c14cbd0 2023-11-06T03:04:51.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.940+0000 7f4933fff640 1 -- 172.21.15.157:0/1816198708 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49440544d0 con 0x7f494c14cbd0 2023-11-06T03:04:51.941 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 msgr2=0x7f4938067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 0x7f4938067f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f493c001750 tx=0x7f493c0011a0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 msgr2=0x7f494c15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b3050 tx=0x7f494400f4a0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 shutdown_connections 2023-11-06T03:04:51.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4938065a90 0x7f4938067f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f494c14cbd0 0x7f494c15d250 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f494c1544c0 0x7f494c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 --2- 172.21.15.157:0/1816198708 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49340039b0 0x7f494c14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:51.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 >> 172.21.15.157:0/1816198708 conn(0x7f494c065930 msgr2=0x7f494c0ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:51.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 shutdown_connections 2023-11-06T03:04:51.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:51.944+0000 7f4951e0f640 1 -- 172.21.15.157:0/1816198708 wait complete. 2023-11-06T03:04:51.959 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:52.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.080+0000 7f0d08fb6640 1 Processor -- start 2023-11-06T03:04:52.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.080+0000 7f0d08fb6640 1 -- start start 2023-11-06T03:04:52.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d08fb6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d040b2330 0x7f0d0406c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d08fb6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0406c490 con 0x7f0d04156a70 2023-11-06T03:04:52.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d08fb6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0406fb80 con 0x7f0d040adae0 2023-11-06T03:04:52.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d08fb6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0406b950 con 0x7f0d040b2330 2023-11-06T03:04:52.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d04156a70 0x7f0d0406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50676/0 (socket says 172.21.15.157:50676) 2023-11-06T03:04:52.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0259c640 1 -- 172.21.15.157:0/2293608464 learned_addr learned my addr 172.21.15.157:0/2293608464 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:52.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d02d9d640 1 --2- 172.21.15.157:0/2293608464 >> v2:172.21.15.157:3300/0 conn(0x7f0d040b2330 0x7f0d0406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d02d9d640 1 -- 172.21.15.157:0/2293608464 >> v1:172.21.15.195:6789/0 conn(0x7f0d040adae0 legacy=0x7f0d040b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:52.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d02d9d640 1 -- 172.21.15.157:0/2293608464 >> v1:172.21.15.157:6789/0 conn(0x7f0d04156a70 legacy=0x7f0d0406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d02d9d640 1 -- 172.21.15.157:0/2293608464 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0406bb60 con 0x7f0d040b2330 2023-11-06T03:04:52.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0159a640 1 -- 172.21.15.157:0/2293608464 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125540649 0 0) 0x7f0d0406c490 con 0x7f0d04156a70 2023-11-06T03:04:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d02d9d640 1 --2- 172.21.15.157:0/2293608464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040b2330 0x7f0d0406c920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009d90 tx=0x7f0cfc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c31d7d4e818af4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0159a640 1 -- 172.21.15.157:0/2293608464 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0a59a0 con 0x7f0d040b2330 2023-11-06T03:04:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0159a640 1 -- 172.21.15.157:0/2293608464 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0a5b00 con 0x7f0d040b2330 2023-11-06T03:04:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.084+0000 7f0d0159a640 1 -- 172.21.15.157:0/2293608464 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0a5e90 con 0x7f0d040b2330 2023-11-06T03:04:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/2293608464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040b2330 msgr2=0x7f0d0406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/2293608464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040b2330 0x7f0d0406c920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009d90 tx=0x7f0cfc01c8e0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/2293608464 shutdown_connections 2023-11-06T03:04:52.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/2293608464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040b2330 0x7f0d0406c920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/2293608464 >> 172.21.15.157:0/2293608464 conn(0x7f0d04065930 msgr2=0x7f0d04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:52.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/2293608464 shutdown_connections 2023-11-06T03:04:52.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/2293608464 wait complete. 2023-11-06T03:04:52.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 Processor -- start 2023-11-06T03:04:52.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- start start 2023-11-06T03:04:52.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 0x7f0d0414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04156a70 0x7f0d0414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04155100 con 0x7f0d040adae0 2023-11-06T03:04:52.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041553e0 con 0x7f0d040b2330 2023-11-06T03:04:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041556c0 con 0x7f0d04156a70 2023-11-06T03:04:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44184/0 (socket says 172.21.15.157:44184) 2023-11-06T03:04:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 -- 172.21.15.157:0/4024052275 learned_addr learned my addr 172.21.15.157:0/4024052275 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d01d9b640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 0x7f0d0414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d02d9d640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04156a70 0x7f0d0414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 -- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04156a70 msgr2=0x7f0d0414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04156a70 0x7f0d0414aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 -- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 msgr2=0x7f0d0414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 0x7f0d0414a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 -- 172.21.15.157:0/4024052275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009a70 con 0x7f0d040adae0 2023-11-06T03:04:52.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d01d9b640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 0x7f0d0414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:52.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d0259c640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400c480 tx=0x7f0cf400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40074e0 con 0x7f0d040adae0 2023-11-06T03:04:52.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4007660 con 0x7f0d040adae0 2023-11-06T03:04:52.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d041559a0 con 0x7f0d040adae0 2023-11-06T03:04:52.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004a30 con 0x7f0d040adae0 2023-11-06T03:04:52.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.088+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d04151b90 con 0x7f0d040adae0 2023-11-06T03:04:52.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d040af330 con 0x7f0d040adae0 2023-11-06T03:04:52.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0cf4004b90 con 0x7f0d040adae0 2023-11-06T03:04:52.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0cf2ffd640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 0x7f0ce4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0cf40827d0 con 0x7f0d040adae0 2023-11-06T03:04:52.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0d01d9b640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 0x7f0ce4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.092+0000 7f0d01d9b640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 0x7f0ce4067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d0414f240 tx=0x7f0cf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.096+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d040af330 con 0x7f0d040adae0 2023-11-06T03:04:52.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.244+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d040af330 con 0x7f0d040adae0 2023-11-06T03:04:52.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.244+0000 7f0cf2ffd640 1 -- 172.21.15.157:0/4024052275 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0cf40180a0 con 0x7f0d040adae0 2023-11-06T03:04:52.245 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:52.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 msgr2=0x7f0ce4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 0x7f0ce4067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d0414f240 tx=0x7f0cf8009040 comp rx=0 tx=0).stop 2023-11-06T03:04:52.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 msgr2=0x7f0d04151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400c480 tx=0x7f0cf400c940 comp rx=0 tx=0).stop 2023-11-06T03:04:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 shutdown_connections 2023-11-06T03:04:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ce4065a60 0x7f0ce4067f20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d04156a70 0x7f0d0414aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d040b2330 0x7f0d0414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 --2- 172.21.15.157:0/4024052275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d040adae0 0x7f0d04151650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 >> 172.21.15.157:0/4024052275 conn(0x7f0d04065930 msgr2=0x7f0d0415b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:52.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 shutdown_connections 2023-11-06T03:04:52.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.248+0000 7f0d08fb6640 1 -- 172.21.15.157:0/4024052275 wait complete. 2023-11-06T03:04:52.262 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:52.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 Processor -- start 2023-11-06T03:04:52.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 -- start start 2023-11-06T03:04:52.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa41815e5e0 0x7fa4180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa41815ea90 0x7fa4180b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41815f580 con 0x7fa4181551e0 2023-11-06T03:04:52.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4180704b0 con 0x7fa41815ea90 2023-11-06T03:04:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa41de5e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41815efc0 con 0x7fa41815e5e0 2023-11-06T03:04:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa41815ea90 0x7fa4180b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa41815ea90 0x7fa4180b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44194/0 (socket says 172.21.15.157:44194) 2023-11-06T03:04:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 -- 172.21.15.157:0/2008269479 learned_addr learned my addr 172.21.15.157:0/2008269479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa416ffd640 1 --2- 172.21.15.157:0/2008269479 >> v2:172.21.15.195:3300/0 conn(0x7fa41815e5e0 0x7fa4180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 -- 172.21.15.157:0/2008269479 >> v2:172.21.15.195:3300/0 conn(0x7fa41815e5e0 msgr2=0x7fa4180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:52.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 --2- 172.21.15.157:0/2008269479 >> v2:172.21.15.195:3300/0 conn(0x7fa41815e5e0 0x7fa4180b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 -- 172.21.15.157:0/2008269479 >> v1:172.21.15.195:6789/0 conn(0x7fa4181551e0 legacy=0x7fa41806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:52.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.372+0000 7fa417fff640 1 -- 172.21.15.157:0/2008269479 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41815f2a0 con 0x7fa41815ea90 2023-11-06T03:04:52.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- 172.21.15.157:0/2008269479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa4180b3950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa418066880 tx=0x7fa40801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa19b96f756f966 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa4167fc640 1 -- 172.21.15.157:0/2008269479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080ab600 con 0x7fa41815ea90 2023-11-06T03:04:52.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa4167fc640 1 -- 172.21.15.157:0/2008269479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080ab760 con 0x7fa41815ea90 2023-11-06T03:04:52.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa4167fc640 1 -- 172.21.15.157:0/2008269479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080abaf0 con 0x7fa41815ea90 2023-11-06T03:04:52.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- 172.21.15.157:0/2008269479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 msgr2=0x7fa4180b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2008269479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa4180b3950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa418066880 tx=0x7fa40801c400 comp rx=0 tx=0).stop 2023-11-06T03:04:52.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- 172.21.15.157:0/2008269479 shutdown_connections 2023-11-06T03:04:52.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2008269479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa4180b3950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2008269479 >> v2:172.21.15.195:3300/0 conn(0x7fa41815e5e0 0x7fa4180b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- 172.21.15.157:0/2008269479 >> 172.21.15.157:0/2008269479 conn(0x7fa418065930 msgr2=0x7fa418065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:52.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- 172.21.15.157:0/2008269479 shutdown_connections 2023-11-06T03:04:52.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- 172.21.15.157:0/2008269479 wait complete. 2023-11-06T03:04:52.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 Processor -- start 2023-11-06T03:04:52.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- start start 2023-11-06T03:04:52.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4181551e0 0x7fa418158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41815e5e0 0x7fa41814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181589f0 con 0x7fa41815ea90 2023-11-06T03:04:52.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418158cd0 con 0x7fa4181551e0 2023-11-06T03:04:52.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa41de5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa418158fb0 con 0x7fa41815e5e0 2023-11-06T03:04:52.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44198/0 (socket says 172.21.15.157:44198) 2023-11-06T03:04:52.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 -- 172.21.15.157:0/2578300188 learned_addr learned my addr 172.21.15.157:0/2578300188 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:52.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa416ffd640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41815e5e0 0x7fa41814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa4177fe640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4181551e0 0x7fa418158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 -- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41815e5e0 msgr2=0x7fa41814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41815e5e0 0x7fa41814d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 -- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4181551e0 msgr2=0x7fa418158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4181551e0 0x7fa418158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 -- 172.21.15.157:0/2578300188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080094d0 con 0x7fa41815ea90 2023-11-06T03:04:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.376+0000 7fa417fff640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4080ad040 tx=0x7fa40801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b4800 con 0x7fa41815ea90 2023-11-06T03:04:52.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080b4960 con 0x7fa41815ea90 2023-11-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418159290 con 0x7fa41815ea90 2023-11-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080b4d10 con 0x7fa41815ea90 2023-11-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41814d960 con 0x7fa41815ea90 2023-11-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa418072750 con 0x7fa41815ea90 2023-11-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4080c7440 con 0x7fa41815ea90 2023-11-06T03:04:52.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 0x7fa404067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa40811dd40 con 0x7fa41815ea90 2023-11-06T03:04:52.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa4177fe640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 0x7fa404067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.380+0000 7fa4177fe640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 0x7fa404067f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa410004450 tx=0x7fa410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.384+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa4080eb5f0 con 0x7fa41815ea90 2023-11-06T03:04:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.552+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa418159740 con 0x7fa41815ea90 2023-11-06T03:04:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa3fffff640 1 -- 172.21.15.157:0/2578300188 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa4080eec10 con 0x7fa41815ea90 2023-11-06T03:04:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 msgr2=0x7fa404067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 0x7fa404067f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa410004450 tx=0x7fa410009040 comp rx=0 tx=0).stop 2023-11-06T03:04:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 msgr2=0x7fa418155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4080ad040 tx=0x7fa40801ac40 comp rx=0 tx=0).stop 2023-11-06T03:04:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 shutdown_connections 2023-11-06T03:04:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa404065a60 0x7fa404067f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41815ea90 0x7fa418155af0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41815e5e0 0x7fa41814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 --2- 172.21.15.157:0/2578300188 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4181551e0 0x7fa418158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 >> 172.21.15.157:0/2578300188 conn(0x7fa418065930 msgr2=0x7fa4180aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 shutdown_connections 2023-11-06T03:04:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.556+0000 7fa41de5e640 1 -- 172.21.15.157:0/2578300188 wait complete. 2023-11-06T03:04:52.573 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi195.front.sepia.ceph.com... 2023-11-06T03:04:52.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:52.574 DEBUG:teuthology.orchestra.run.smithi195:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-06T03:04:52.675 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:04:52.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:04:52.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:52.677 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:04:52.688 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:52.688 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:52.694 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:52.695 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:52.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:52.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 Processor -- start 2023-11-06T03:04:52.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 -- start start 2023-11-06T03:04:52.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63a815caa0 0x7f63a80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f63a80d0280 0x7f63a80b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a80d2920 con 0x7f63a815ce80 2023-11-06T03:04:52.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a80d18a0 con 0x7f63a815caa0 2023-11-06T03:04:52.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63af446640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a80d1b60 con 0x7f63a80d0280 2023-11-06T03:04:52.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63a815caa0 0x7f63a80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63a815caa0 0x7f63a80cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44212/0 (socket says 172.21.15.157:44212) 2023-11-06T03:04:52.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 -- 172.21.15.157:0/1823038911 learned_addr learned my addr 172.21.15.157:0/1823038911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:52.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad9e3640 1 --2- 172.21.15.157:0/1823038911 >> v2:172.21.15.195:3300/0 conn(0x7f63a80d0280 0x7f63a80b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 -- 172.21.15.157:0/1823038911 >> v2:172.21.15.195:3300/0 conn(0x7f63a80d0280 msgr2=0x7f63a80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:52.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 --2- 172.21.15.157:0/1823038911 >> v2:172.21.15.195:3300/0 conn(0x7f63a80d0280 0x7f63a80b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 -- 172.21.15.157:0/1823038911 >> v1:172.21.15.195:6789/0 conn(0x7f63a815ce80 legacy=0x7f63a80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.924+0000 7f63ad1e2640 1 -- 172.21.15.157:0/1823038911 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80d1e40 con 0x7f63a815caa0 2023-11-06T03:04:52.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 --2- 172.21.15.157:0/1823038911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a815caa0 0x7f63a80cdc60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cf670 tx=0x7f63a401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20176025bcf59f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f639ffff640 1 -- 172.21.15.157:0/1823038911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40aa430 con 0x7f63a815caa0 2023-11-06T03:04:52.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f639ffff640 1 -- 172.21.15.157:0/1823038911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a40aa590 con 0x7f63a815caa0 2023-11-06T03:04:52.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f639ffff640 1 -- 172.21.15.157:0/1823038911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40aa920 con 0x7f63a815caa0 2023-11-06T03:04:52.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- 172.21.15.157:0/1823038911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a815caa0 msgr2=0x7f63a80cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- 172.21.15.157:0/1823038911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a815caa0 0x7f63a80cdc60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cf670 tx=0x7f63a401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- 172.21.15.157:0/1823038911 shutdown_connections 2023-11-06T03:04:52.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- 172.21.15.157:0/1823038911 >> v2:172.21.15.195:3300/0 conn(0x7f63a80d0280 0x7f63a80b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- 172.21.15.157:0/1823038911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a815caa0 0x7f63a80cdc60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- 172.21.15.157:0/1823038911 >> 172.21.15.157:0/1823038911 conn(0x7f63a8070790 msgr2=0x7f63a8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:52.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- 172.21.15.157:0/1823038911 shutdown_connections 2023-11-06T03:04:52.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- 172.21.15.157:0/1823038911 wait complete. 2023-11-06T03:04:52.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 Processor -- start 2023-11-06T03:04:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- start start 2023-11-06T03:04:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63a815caa0 0x7f63a80c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63a815ce80 0x7f63a815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8152bc0 con 0x7f63a80d0280 2023-11-06T03:04:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cb150 con 0x7f63a815caa0 2023-11-06T03:04:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63af446640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cb430 con 0x7f63a815ce80 2023-11-06T03:04:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44218/0 (socket says 172.21.15.157:44218) 2023-11-06T03:04:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 -- 172.21.15.157:0/2668946291 learned_addr learned my addr 172.21.15.157:0/2668946291 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ac9e1640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63a815caa0 0x7f63a80c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad9e3640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63a815ce80 0x7f63a815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 -- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63a815ce80 msgr2=0x7f63a815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63a815ce80 0x7f63a815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 -- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63a815caa0 msgr2=0x7f63a80c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63a815caa0 0x7f63a80c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.928+0000 7f63ad1e2640 1 -- 172.21.15.157:0/2668946291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40094d0 con 0x7f63a80d0280 2023-11-06T03:04:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63ad1e2640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f63a40190e0 tx=0x7f63a401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40b3800 con 0x7f63a80d0280 2023-11-06T03:04:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a40b3960 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a80cb710 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40b3cf0 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a80cbcd0 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a80cb570 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f63a40b2380 con 0x7f63a80d0280 2023-11-06T03:04:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 0x7f638c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f63a411d800 con 0x7f63a80d0280 2023-11-06T03:04:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63ac9e1640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 0x7f638c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:52.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.932+0000 7f63ac9e1640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 0x7f638c067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63a8155f30 tx=0x7f6398009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:52.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:52.936+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f63a80cb570 con 0x7f63a80d0280 2023-11-06T03:04:53.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.080+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63a80cb570 con 0x7f63a80d0280 2023-11-06T03:04:53.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.084+0000 7f639dffb640 1 -- 172.21.15.157:0/2668946291 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f63a80cb570 con 0x7f63a80d0280 2023-11-06T03:04:53.085 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:53.086 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:53.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 msgr2=0x7f638c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 0x7f638c067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63a8155f30 tx=0x7f6398009040 comp rx=0 tx=0).stop 2023-11-06T03:04:53.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 msgr2=0x7f63a8152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f63a40190e0 tx=0x7f63a401ad10 comp rx=0 tx=0).stop 2023-11-06T03:04:53.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 shutdown_connections 2023-11-06T03:04:53.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f638c065a60 0x7f638c067f20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f63a815ce80 0x7f63a815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63a815caa0 0x7f63a80c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 --2- 172.21.15.157:0/2668946291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63a80d0280 0x7f63a8152680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 >> 172.21.15.157:0/2668946291 conn(0x7f63a8070790 msgr2=0x7f63a8156480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 shutdown_connections 2023-11-06T03:04:53.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.088+0000 7f63af446640 1 -- 172.21.15.157:0/2668946291 wait complete. 2023-11-06T03:04:53.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:53.104 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:53.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 Processor -- start 2023-11-06T03:04:53.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 -- start start 2023-11-06T03:04:53.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06c81599c0 0x7f06c80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c815e440 con 0x7f06c815dea0 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c815adb0 con 0x7f06c81599c0 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06ce08f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c806c240 con 0x7f06c806d000 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06c81599c0 0x7f06c80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f06c815dea0 0x7f06c8070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50682/0 (socket says 172.21.15.157:50682) 2023-11-06T03:04:53.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c7fff640 1 -- 172.21.15.157:0/2289544648 learned_addr learned my addr 172.21.15.157:0/2289544648 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:53.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06bffff640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c77fe640 1 -- 172.21.15.157:0/2289544648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 msgr2=0x7f06c80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:53.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c77fe640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c80b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c77fe640 1 -- 172.21.15.157:0/2289544648 >> v1:172.21.15.157:6789/0 conn(0x7f06c815dea0 legacy=0x7f06c8070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.224+0000 7f06c77fe640 1 -- 172.21.15.157:0/2289544648 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c806c520 con 0x7f06c81599c0 2023-11-06T03:04:53.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06c77fe640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c80ae000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06b80097c0 tx=0x7f06b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845e40847e7b24a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06c6ffd640 1 -- 172.21.15.157:0/2289544648 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80ad660 con 0x7f06c81599c0 2023-11-06T03:04:53.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06c6ffd640 1 -- 172.21.15.157:0/2289544648 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b80ad7c0 con 0x7f06c81599c0 2023-11-06T03:04:53.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06c6ffd640 1 -- 172.21.15.157:0/2289544648 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80adb50 con 0x7f06c81599c0 2023-11-06T03:04:53.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- 172.21.15.157:0/2289544648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 msgr2=0x7f06c80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c80ae000 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06b80097c0 tx=0x7f06b801c450 comp rx=0 tx=0).stop 2023-11-06T03:04:53.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- 172.21.15.157:0/2289544648 shutdown_connections 2023-11-06T03:04:53.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c80b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 --2- 172.21.15.157:0/2289544648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c80ae000 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- 172.21.15.157:0/2289544648 >> 172.21.15.157:0/2289544648 conn(0x7f06c8065930 msgr2=0x7f06c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- 172.21.15.157:0/2289544648 shutdown_connections 2023-11-06T03:04:53.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- 172.21.15.157:0/2289544648 wait complete. 2023-11-06T03:04:53.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 Processor -- start 2023-11-06T03:04:53.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.228+0000 7f06ce08f640 1 -- start start 2023-11-06T03:04:53.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 0x7f06c80a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80cc300 con 0x7f06c81599c0 2023-11-06T03:04:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8152740 con 0x7f06c815dea0 2023-11-06T03:04:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80cb9d0 con 0x7f06c806d000 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44238/0 (socket says 172.21.15.157:44238) 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 -- 172.21.15.157:0/4124764819 learned_addr learned my addr 172.21.15.157:0/4124764819 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c77fe640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c7fff640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 0x7f06c80a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 -- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 msgr2=0x7f06c814ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c814ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 -- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 msgr2=0x7f06c80a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 0x7f06c80a8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 -- 172.21.15.157:0/4124764819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b8009420 con 0x7f06c81599c0 2023-11-06T03:04:53.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c7fff640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 0x7f06c80a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:53.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06bffff640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f06b000d800 tx=0x7f06b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000b6c0 con 0x7f06c81599c0 2023-11-06T03:04:53.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b000b820 con 0x7f06c81599c0 2023-11-06T03:04:53.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000bb90 con 0x7f06c81599c0 2023-11-06T03:04:53.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c80cbcb0 con 0x7f06c81599c0 2023-11-06T03:04:53.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c814eae0 con 0x7f06c81599c0 2023-11-06T03:04:53.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c8072750 con 0x7f06c81599c0 2023-11-06T03:04:53.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f06b000bcf0 con 0x7f06c81599c0 2023-11-06T03:04:53.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.232+0000 7f06c4ff9640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 0x7f06a8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.236+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f06b0083790 con 0x7f06c81599c0 2023-11-06T03:04:53.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.236+0000 7f06c77fe640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 0x7f06a8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.236+0000 7f06c77fe640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 0x7f06a8067f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06b80af000 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.240+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f06b0051040 con 0x7f06c81599c0 2023-11-06T03:04:53.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.380+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06c80adcd0 con 0x7f06c81599c0 2023-11-06T03:04:53.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.380+0000 7f06c4ff9640 1 -- 172.21.15.157:0/4124764819 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06b0054660 con 0x7f06c81599c0 2023-11-06T03:04:53.381 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:53.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 msgr2=0x7f06a8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 0x7f06a8067f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06b80af000 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:53.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 msgr2=0x7f06c814e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f06b000d800 tx=0x7f06b000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 shutdown_connections 2023-11-06T03:04:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06a8065a90 0x7f06a8067f50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06c815dea0 0x7f06c80a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06c81599c0 0x7f06c814e5a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 --2- 172.21.15.157:0/4124764819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06c806d000 0x7f06c814ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 >> 172.21.15.157:0/4124764819 conn(0x7f06c8065930 msgr2=0x7f06c8070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 shutdown_connections 2023-11-06T03:04:53.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.384+0000 7f06ce08f640 1 -- 172.21.15.157:0/4124764819 wait complete. 2023-11-06T03:04:53.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:53.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 Processor -- start 2023-11-06T03:04:53.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 -- start start 2023-11-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 0x7fbbd40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbd406d000 0x7fbbd4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd415e440 con 0x7fbbd41599c0 2023-11-06T03:04:53.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd406c240 con 0x7fbbd406d000 2023-11-06T03:04:53.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbdb28f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd406c520 con 0x7fbbd40b2af0 2023-11-06T03:04:53.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbd406d000 0x7fbbd4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbd406d000 0x7fbbd4159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44252/0 (socket says 172.21.15.157:44252) 2023-11-06T03:04:53.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 -- 172.21.15.157:0/2357935683 learned_addr learned my addr 172.21.15.157:0/2357935683 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:53.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd882a640 1 --2- 172.21.15.157:0/2357935683 >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 0x7fbbd40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 -- 172.21.15.157:0/2357935683 >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 msgr2=0x7fbbd40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:53.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 --2- 172.21.15.157:0/2357935683 >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 0x7fbbd40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 -- 172.21.15.157:0/2357935683 >> v1:172.21.15.195:6789/0 conn(0x7fbbd41599c0 legacy=0x7fbbd40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 -- 172.21.15.157:0/2357935683 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd406c800 con 0x7fbbd406d000 2023-11-06T03:04:53.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd882a640 1 --2- 172.21.15.157:0/2357935683 >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 0x7fbbd40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:53.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.512+0000 7fbbd982c640 1 --2- 172.21.15.157:0/2357935683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd406d000 0x7fbbd4159da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40cfa20 tx=0x7fbbcc01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c40d6aa5897ab6b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbcbfff640 1 -- 172.21.15.157:0/2357935683 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0a7aa0 con 0x7fbbd406d000 2023-11-06T03:04:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbcbfff640 1 -- 172.21.15.157:0/2357935683 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc0a7c00 con 0x7fbbd406d000 2023-11-06T03:04:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbcbfff640 1 -- 172.21.15.157:0/2357935683 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0af3f0 con 0x7fbbd406d000 2023-11-06T03:04:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2357935683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd406d000 msgr2=0x7fbbd4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2357935683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd406d000 0x7fbbd4159da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40cfa20 tx=0x7fbbcc01ebb0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2357935683 shutdown_connections 2023-11-06T03:04:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2357935683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd406d000 0x7fbbd4159da0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2357935683 >> v2:172.21.15.195:3300/0 conn(0x7fbbd40b2af0 0x7fbbd40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2357935683 >> 172.21.15.157:0/2357935683 conn(0x7fbbd4065930 msgr2=0x7fbbd4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2357935683 shutdown_connections 2023-11-06T03:04:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2357935683 wait complete. 2023-11-06T03:04:53.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 Processor -- start 2023-11-06T03:04:53.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- start start 2023-11-06T03:04:53.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbd406d000 0x7fbbd414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbd41599c0 0x7fbbd41482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd41487e0 con 0x7fbbd40b2af0 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd414bd80 con 0x7fbbd406d000 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd414c020 con 0x7fbbd41599c0 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44254/0 (socket says 172.21.15.157:44254) 2023-11-06T03:04:53.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 -- 172.21.15.157:0/2484041989 learned_addr learned my addr 172.21.15.157:0/2484041989 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:53.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd982c640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbd41599c0 0x7fbbd41482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd902b640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbd406d000 0x7fbbd414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 -- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbd41599c0 msgr2=0x7fbbd41482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbd41599c0 0x7fbbd41482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 -- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbd406d000 msgr2=0x7fbbd414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbd406d000 0x7fbbd414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 -- 172.21.15.157:0/2484041989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc009b30 con 0x7fbbd40b2af0 2023-11-06T03:04:53.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbd882a640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e810 tx=0x7fbbc400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400c8d0 con 0x7fbbd40b2af0 2023-11-06T03:04:53.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc400ca30 con 0x7fbbd40b2af0 2023-11-06T03:04:53.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd414c300 con 0x7fbbd40b2af0 2023-11-06T03:04:53.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400cde0 con 0x7fbbd40b2af0 2023-11-06T03:04:53.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.516+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd414c8c0 con 0x7fbbd40b2af0 2023-11-06T03:04:53.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd4072750 con 0x7fbbd40b2af0 2023-11-06T03:04:53.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbbc4013410 con 0x7fbbd40b2af0 2023-11-06T03:04:53.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbc9ffb640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 0x7fbbb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbbc40831d0 con 0x7fbbd40b2af0 2023-11-06T03:04:53.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbd902b640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 0x7fbbb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.520+0000 7fbbd902b640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 0x7fbbb8067f50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000f70 tx=0x7fbbd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.524+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbbc4050950 con 0x7fbbd40b2af0 2023-11-06T03:04:53.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.664+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbbd4159180 con 0x7fbbd40b2af0 2023-11-06T03:04:53.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbc9ffb640 1 -- 172.21.15.157:0/2484041989 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbc4053f70 con 0x7fbbd40b2af0 2023-11-06T03:04:53.667 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:53.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 msgr2=0x7fbbb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 0x7fbbb8067f50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000f70 tx=0x7fbbd0009040 comp rx=0 tx=0).stop 2023-11-06T03:04:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 msgr2=0x7fbbd41489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400e810 tx=0x7fbbc400ecd0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 shutdown_connections 2023-11-06T03:04:53.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbb8065a90 0x7fbbb8067f50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbd41599c0 0x7fbbd41482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd40b2af0 0x7fbbd41489c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 --2- 172.21.15.157:0/2484041989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbd406d000 0x7fbbd414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 >> 172.21.15.157:0/2484041989 conn(0x7fbbd4065930 msgr2=0x7fbbd406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 shutdown_connections 2023-11-06T03:04:53.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.668+0000 7fbbdb28f640 1 -- 172.21.15.157:0/2484041989 wait complete. 2023-11-06T03:04:53.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.800+0000 7f4a7c9a2640 1 Processor -- start 2023-11-06T03:04:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7c9a2640 1 -- start start 2023-11-06T03:04:53.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7c9a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a78167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7c9a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a780d2920 con 0x7f4a7815ce80 2023-11-06T03:04:53.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7c9a2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a780d18a0 con 0x7f4a7815caa0 2023-11-06T03:04:53.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7c9a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780d1b60 con 0x7f4a780d0280 2023-11-06T03:04:53.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a75d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4a7815ce80 0x7f4a780cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50690/0 (socket says 172.21.15.157:50690) 2023-11-06T03:04:53.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a75d9b640 1 -- 172.21.15.157:0/3521564021 learned_addr learned my addr 172.21.15.157:0/3521564021 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:53.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a76d9d640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a78167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242266293 0 0) 0x7f4a780d2920 con 0x7f4a7815ce80 2023-11-06T03:04:53.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a64003530 con 0x7f4a7815ce80 2023-11-06T03:04:53.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3503743192 0 0) 0x7f4a64003530 con 0x7f4a7815ce80 2023-11-06T03:04:53.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 msgr2=0x7f4a78167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:53.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a78167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 >> v1:172.21.15.195:6789/0 conn(0x7f4a7815caa0 legacy=0x7f4a780cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780d1e40 con 0x7f4a7815ce80 2023-11-06T03:04:53.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4a60001680 con 0x7f4a7815ce80 2023-11-06T03:04:53.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a76d9d640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a78167860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:53.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 >> v1:172.21.15.157:6789/0 conn(0x7f4a7815ce80 legacy=0x7f4a780cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640047d0 0x7f4a64004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a640051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780d1b60 con 0x7f4a640039b0 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6400bb50 con 0x7f4a640047d0 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6400bd10 con 0x7f4a780d0280 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a76d9d640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a640051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a75d9b640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640047d0 0x7f4a64004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 -- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 msgr2=0x7f4a640051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a780d0280 0x7f4a640051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 -- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640047d0 msgr2=0x7f4a64004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640047d0 0x7f4a64004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.804+0000 7f4a7659c640 1 -- 172.21.15.157:0/3521564021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a64003530 con 0x7f4a640039b0 2023-11-06T03:04:53.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7659c640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4a68001e70 tx=0x7f4a6801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2919247fc1c24c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68003b80 con 0x7f4a640039b0 2023-11-06T03:04:53.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68003ce0 con 0x7f4a640039b0 2023-11-06T03:04:53.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7559a640 1 -- 172.21.15.157:0/3521564021 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680052c0 con 0x7f4a640039b0 2023-11-06T03:04:53.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/3521564021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a640039b0 msgr2=0x7f4a64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/3521564021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4a68001e70 tx=0x7f4a6801c730 comp rx=0 tx=0).stop 2023-11-06T03:04:53.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a76d9d640 1 -- 172.21.15.157:0/3521564021 reap_dead start 2023-11-06T03:04:53.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/3521564021 shutdown_connections 2023-11-06T03:04:53.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/3521564021 >> 172.21.15.157:0/3521564021 conn(0x7f4a78070790 msgr2=0x7f4a78070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/3521564021 shutdown_connections 2023-11-06T03:04:53.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/3521564021 wait complete. 2023-11-06T03:04:53.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 Processor -- start 2023-11-06T03:04:53.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- start start 2023-11-06T03:04:53.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640039b0 0x7f4a78159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 0x7f4a7815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780a9510 con 0x7f4a780acc30 2023-11-06T03:04:53.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7815a8b0 con 0x7f4a640039b0 2023-11-06T03:04:53.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781599c0 con 0x7f4a7815a120 2023-11-06T03:04:53.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44268/0 (socket says 172.21.15.157:44268) 2023-11-06T03:04:53.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 -- 172.21.15.157:0/969214769 learned_addr learned my addr 172.21.15.157:0/969214769 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:53.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7659c640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640039b0 0x7f4a78159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a76d9d640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 0x7f4a7815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 -- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 msgr2=0x7f4a7815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 0x7f4a7815bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 -- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640039b0 msgr2=0x7f4a78159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640039b0 0x7f4a78159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 -- 172.21.15.157:0/969214769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780a9760 con 0x7f4a780acc30 2023-11-06T03:04:53.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a76d9d640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 0x7f4a7815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a75d9b640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d7a0 tx=0x7f4a6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a600074e0 con 0x7f4a780acc30 2023-11-06T03:04:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60007640 con 0x7f4a780acc30 2023-11-06T03:04:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a780a9a40 con 0x7f4a780acc30 2023-11-06T03:04:53.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a600049f0 con 0x7f4a780acc30 2023-11-06T03:04:53.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.808+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a780aa000 con 0x7f4a780acc30 2023-11-06T03:04:53.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78072750 con 0x7f4a780acc30 2023-11-06T03:04:53.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a60004b50 con 0x7f4a780acc30 2023-11-06T03:04:53.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a6effd640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 0x7f4a58067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:53.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4a60082fb0 con 0x7f4a780acc30 2023-11-06T03:04:53.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a7659c640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 0x7f4a58067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:53.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.812+0000 7f4a7659c640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 0x7f4a58067f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4a68005000 tx=0x7f4a68001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:53.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.816+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a600b4070 con 0x7f4a780acc30 2023-11-06T03:04:53.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a780c9ab0 con 0x7f4a780acc30 2023-11-06T03:04:53.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a6effd640 1 -- 172.21.15.157:0/969214769 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a60019020 con 0x7f4a780acc30 2023-11-06T03:04:53.960 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:53.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 msgr2=0x7f4a58067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 0x7f4a58067f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4a68005000 tx=0x7f4a68001520 comp rx=0 tx=0).stop 2023-11-06T03:04:53.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 msgr2=0x7f4a7815b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:53.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d7a0 tx=0x7f4a6000dc60 comp rx=0 tx=0).stop 2023-11-06T03:04:53.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 shutdown_connections 2023-11-06T03:04:53.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a58065a90 0x7f4a58067f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a7815a120 0x7f4a7815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a780acc30 0x7f4a7815b980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 --2- 172.21.15.157:0/969214769 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a640039b0 0x7f4a78159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 >> 172.21.15.157:0/969214769 conn(0x7f4a78070790 msgr2=0x7f4a780b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:53.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 shutdown_connections 2023-11-06T03:04:53.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:53.960+0000 7f4a7c9a2640 1 -- 172.21.15.157:0/969214769 wait complete. 2023-11-06T03:04:53.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:54.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.100+0000 7f4b96651640 1 Processor -- start 2023-11-06T03:04:54.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.100+0000 7f4b96651640 1 -- start start 2023-11-06T03:04:54.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b96651640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b900b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b96651640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9015f580 con 0x7f4b901551e0 2023-11-06T03:04:54.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b96651640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9015efc0 con 0x7f4b9015ea60 2023-11-06T03:04:54.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b96651640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900704b0 con 0x7f4b9015e5e0 2023-11-06T03:04:54.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4b901551e0 0x7f4b9006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50698/0 (socket says 172.21.15.157:50698) 2023-11-06T03:04:54.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8ffff640 1 -- 172.21.15.157:0/2798875726 learned_addr learned my addr 172.21.15.157:0/2798875726 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:54.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086896553 0 0) 0x7f4b9015f580 con 0x7f4b901551e0 2023-11-06T03:04:54.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003530 con 0x7f4b901551e0 2023-11-06T03:04:54.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b900b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809460191 0 0) 0x7f4b78003530 con 0x7f4b901551e0 2023-11-06T03:04:54.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 msgr2=0x7f4b900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b900b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 >> v1:172.21.15.195:6789/0 conn(0x7f4b9015ea60 legacy=0x7f4b9015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9015f2a0 con 0x7f4b901551e0 2023-11-06T03:04:54.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4b80001680 con 0x7f4b901551e0 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 >> v1:172.21.15.157:6789/0 conn(0x7f4b901551e0 legacy=0x7f4b9006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78003f30 0x7f4b780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78004d10 0x7f4b78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900704b0 con 0x7f4b78003f30 2023-11-06T03:04:54.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7800ff30 con 0x7f4b78004d10 2023-11-06T03:04:54.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780100f0 con 0x7f4b9015e5e0 2023-11-06T03:04:54.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78003f30 0x7f4b780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b94bee640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78004d10 0x7f4b78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 -- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 msgr2=0x7f4b780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8ffff640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9015e5e0 0x7f4b780039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 -- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78004d10 msgr2=0x7f4b78005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78004d10 0x7f4b78005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 -- 172.21.15.157:0/2798875726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78003530 con 0x7f4b78003f30 2023-11-06T03:04:54.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78003f30 0x7f4b780047d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009ae0 tx=0x7f4b8401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751e222d4634ea08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a6510 con 0x7f4b78003f30 2023-11-06T03:04:54.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b840a6670 con 0x7f4b78003f30 2023-11-06T03:04:54.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.104+0000 7f4b8effd640 1 -- 172.21.15.157:0/2798875726 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840a6a20 con 0x7f4b78003f30 2023-11-06T03:04:54.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/2798875726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78003f30 msgr2=0x7f4b780047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 --2- 172.21.15.157:0/2798875726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b78003f30 0x7f4b780047d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009ae0 tx=0x7f4b8401ca20 comp rx=0 tx=0).stop 2023-11-06T03:04:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 -- 172.21.15.157:0/2798875726 reap_dead start 2023-11-06T03:04:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/2798875726 shutdown_connections 2023-11-06T03:04:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/2798875726 >> 172.21.15.157:0/2798875726 conn(0x7f4b90065930 msgr2=0x7f4b90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/2798875726 shutdown_connections 2023-11-06T03:04:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/2798875726 wait complete. 2023-11-06T03:04:54.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 Processor -- start 2023-11-06T03:04:54.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- start start 2023-11-06T03:04:54.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78003f30 0x7f4b90138030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9013a820 0x7f4b9006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9013b7c0 con 0x7f4b9013aca0 2023-11-06T03:04:54.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90138d10 con 0x7f4b78003f30 2023-11-06T03:04:54.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9013b200 con 0x7f4b9013a820 2023-11-06T03:04:54.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44286/0 (socket says 172.21.15.157:44286) 2023-11-06T03:04:54.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 -- 172.21.15.157:0/3089907466 learned_addr learned my addr 172.21.15.157:0/3089907466 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:54.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b8ffff640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78003f30 0x7f4b90138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b8f7fe640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9013a820 0x7f4b9006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 -- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9013a820 msgr2=0x7f4b9006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9013a820 0x7f4b9006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 -- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78003f30 msgr2=0x7f4b90138030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78003f30 0x7f4b90138030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 -- 172.21.15.157:0/3089907466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9013b4e0 con 0x7f4b9013aca0 2023-11-06T03:04:54.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b94bee640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b900cf420 tx=0x7f4b8800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003980 con 0x7f4b9013aca0 2023-11-06T03:04:54.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b88003ae0 con 0x7f4b9013aca0 2023-11-06T03:04:54.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003e90 con 0x7f4b9013aca0 2023-11-06T03:04:54.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b90138930 con 0x7f4b9013aca0 2023-11-06T03:04:54.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.108+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9014bb80 con 0x7f4b9013aca0 2023-11-06T03:04:54.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b90138790 con 0x7f4b9013aca0 2023-11-06T03:04:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4b880050d0 con 0x7f4b9013aca0 2023-11-06T03:04:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b8cff9640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 0x7f4b7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4b88083090 con 0x7f4b9013aca0 2023-11-06T03:04:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b8ffff640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 0x7f4b7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.112+0000 7f4b8ffff640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 0x7f4b7c067f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b80001210 tx=0x7f4b80001540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.116+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4b90138790 con 0x7f4b9013aca0 2023-11-06T03:04:54.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.256+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b90138790 con 0x7f4b9013aca0 2023-11-06T03:04:54.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.256+0000 7f4b8cff9640 1 -- 172.21.15.157:0/3089907466 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b90138790 con 0x7f4b9013aca0 2023-11-06T03:04:54.258 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:54.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 msgr2=0x7f4b7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 0x7f4b7c067f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b80001210 tx=0x7f4b80001540 comp rx=0 tx=0).stop 2023-11-06T03:04:54.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 msgr2=0x7f4b9006bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b900cf420 tx=0x7f4b8800ec10 comp rx=0 tx=0).stop 2023-11-06T03:04:54.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 shutdown_connections 2023-11-06T03:04:54.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b7c065a60 0x7f4b7c067f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b9013aca0 0x7f4b9006bd00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4b9013a820 0x7f4b9006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 --2- 172.21.15.157:0/3089907466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b78003f30 0x7f4b90138030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 >> 172.21.15.157:0/3089907466 conn(0x7f4b90065930 msgr2=0x7f4b9015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 shutdown_connections 2023-11-06T03:04:54.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.260+0000 7f4b96651640 1 -- 172.21.15.157:0/3089907466 wait complete. 2023-11-06T03:04:54.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:54.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.396+0000 7fa0c75bb640 1 Processor -- start 2023-11-06T03:04:54.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.396+0000 7fa0c75bb640 1 -- start start 2023-11-06T03:04:54.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c75bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0c00b2af0 0x7fa0c00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c75bb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa0c006d000 0x7fa0c0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c75bb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c015e440 con 0x7fa0c01599c0 2023-11-06T03:04:54.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c75bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c006c240 con 0x7fa0c00b2af0 2023-11-06T03:04:54.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c75bb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c006c520 con 0x7fa0c006d000 2023-11-06T03:04:54.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0c00b2af0 0x7fa0c00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0c00b2af0 0x7fa0c00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44290/0 (socket says 172.21.15.157:44290) 2023-11-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 -- 172.21.15.157:0/2511788255 learned_addr learned my addr 172.21.15.157:0/2511788255 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 -- 172.21.15.157:0/2511788255 >> v2:172.21.15.195:3300/0 conn(0x7fa0c006d000 msgr2=0x7fa0c0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 --2- 172.21.15.157:0/2511788255 >> v2:172.21.15.195:3300/0 conn(0x7fa0c006d000 0x7fa0c0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 -- 172.21.15.157:0/2511788255 >> v1:172.21.15.195:6789/0 conn(0x7fa0c01599c0 legacy=0x7fa0c00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 -- 172.21.15.157:0/2511788255 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c006c800 con 0x7fa0c00b2af0 2023-11-06T03:04:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0c4b56640 1 --2- 172.21.15.157:0/2511788255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00ae000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa0c006d930 tx=0x7fa0b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96beb100a7b51376 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0b7fff640 1 -- 172.21.15.157:0/2511788255 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b00aa610 con 0x7fa0c00b2af0 2023-11-06T03:04:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0b7fff640 1 -- 172.21.15.157:0/2511788255 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b00a6040 con 0x7fa0c00b2af0 2023-11-06T03:04:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.400+0000 7fa0b7fff640 1 -- 172.21.15.157:0/2511788255 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b00aa990 con 0x7fa0c00b2af0 2023-11-06T03:04:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/2511788255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c00b2af0 msgr2=0x7fa0c00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/2511788255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00ae000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa0c006d930 tx=0x7fa0b001c490 comp rx=0 tx=0).stop 2023-11-06T03:04:54.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/2511788255 shutdown_connections 2023-11-06T03:04:54.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/2511788255 >> v2:172.21.15.195:3300/0 conn(0x7fa0c006d000 0x7fa0c0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/2511788255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00ae000 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/2511788255 >> 172.21.15.157:0/2511788255 conn(0x7fa0c0065930 msgr2=0x7fa0c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/2511788255 shutdown_connections 2023-11-06T03:04:54.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/2511788255 wait complete. 2023-11-06T03:04:54.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 Processor -- start 2023-11-06T03:04:54.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- start start 2023-11-06T03:04:54.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0c006d000 0x7fa0c014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0150980 con 0x7fa0c01599c0 2023-11-06T03:04:54.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0150c60 con 0x7fa0c00b2af0 2023-11-06T03:04:54.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0150f40 con 0x7fa0c006d000 2023-11-06T03:04:54.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44300/0 (socket says 172.21.15.157:44300) 2023-11-06T03:04:54.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 -- 172.21.15.157:0/3040274266 learned_addr learned my addr 172.21.15.157:0/3040274266 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c4b56640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5357640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0c006d000 0x7fa0c014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 -- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0c006d000 msgr2=0x7fa0c014b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0c006d000 0x7fa0c014b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 -- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 msgr2=0x7fa0c00cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 -- 172.21.15.157:0/3040274266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00094d0 con 0x7fa0c01599c0 2023-11-06T03:04:54.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c4b56640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:54.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c5b58640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b9e0 tx=0x7fa0b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800cb20 con 0x7fa0c01599c0 2023-11-06T03:04:54.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800cc80 con 0x7fa0c01599c0 2023-11-06T03:04:54.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8014680 con 0x7fa0c01599c0 2023-11-06T03:04:54.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0151220 con 0x7fa0c01599c0 2023-11-06T03:04:54.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c014b9d0 con 0x7fa0c01599c0 2023-11-06T03:04:54.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.404+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c0072750 con 0x7fa0c01599c0 2023-11-06T03:04:54.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.408+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa0b801d3b0 con 0x7fa0c01599c0 2023-11-06T03:04:54.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.408+0000 7fa0b5ffb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 0x7fa0a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.408+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa0b80831f0 con 0x7fa0c01599c0 2023-11-06T03:04:54.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.408+0000 7fa0c5357640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 0x7fa0a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.408+0000 7fa0c5357640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 0x7fa0a4067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b810 tx=0x7fa0bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.412+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa0b8050aa0 con 0x7fa0c01599c0 2023-11-06T03:04:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.576+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0c01516d0 con 0x7fa0c01599c0 2023-11-06T03:04:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.576+0000 7fa0b5ffb640 1 -- 172.21.15.157:0/3040274266 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa0b80540c0 con 0x7fa0c01599c0 2023-11-06T03:04:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 msgr2=0x7fa0a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 0x7fa0a4067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b810 tx=0x7fa0bc008040 comp rx=0 tx=0).stop 2023-11-06T03:04:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 msgr2=0x7fa0c014c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b9e0 tx=0x7fa0b800bea0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 shutdown_connections 2023-11-06T03:04:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa0a4065a60 0x7fa0a4067f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0c01599c0 0x7fa0c014c520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0c00b2af0 0x7fa0c00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 --2- 172.21.15.157:0/3040274266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0c006d000 0x7fa0c014b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 >> 172.21.15.157:0/3040274266 conn(0x7fa0c0065930 msgr2=0x7fa0c0158440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 shutdown_connections 2023-11-06T03:04:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.580+0000 7fa0c75bb640 1 -- 172.21.15.157:0/3040274266 wait complete. 2023-11-06T03:04:54.597 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:04:54.597 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:54.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:54.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.768+0000 7f591ccdc640 1 Processor -- start 2023-11-06T03:04:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.768+0000 7f591ccdc640 1 -- start start 2023-11-06T03:04:54.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591ccdc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591ccdc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f59180b3b20 0x7f591815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591ccdc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591815f580 con 0x7f591815e5e0 2023-11-06T03:04:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591ccdc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591815efc0 con 0x7f59180b3b20 2023-11-06T03:04:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591ccdc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591815f2a0 con 0x7f59181551e0 2023-11-06T03:04:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37582/0 (socket says 172.21.15.157:37582) 2023-11-06T03:04:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591659c640 1 -- 172.21.15.157:0/2661543576 learned_addr learned my addr 172.21.15.157:0/2661543576 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5916d9d640 1 --2- 172.21.15.157:0/2661543576 >> v2:172.21.15.195:3300/0 conn(0x7f59180b3b20 0x7f591815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5916d9d640 1 -- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 msgr2=0x7f591806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:54.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5916d9d640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5916d9d640 1 -- 172.21.15.157:0/2661543576 >> v1:172.21.15.195:6789/0 conn(0x7f591815e5e0 legacy=0x7f59180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5916d9d640 1 -- 172.21.15.157:0/2661543576 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591806f5a0 con 0x7f59180b3b20 2023-11-06T03:04:54.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f5915d9b640 1 -- 172.21.15.157:0/2661543576 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101562677 0 0) 0x7f591815f580 con 0x7f591815e5e0 2023-11-06T03:04:54.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.772+0000 7f591659c640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:54.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f5916d9d640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180b3b20 0x7f591815c1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5910009d90 tx=0x7f591001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c4cc6ee782b0d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f5915d9b640 1 -- 172.21.15.157:0/2661543576 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100a5970 con 0x7f59180b3b20 2023-11-06T03:04:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f5915d9b640 1 -- 172.21.15.157:0/2661543576 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59100a5ad0 con 0x7f59180b3b20 2023-11-06T03:04:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f5915d9b640 1 -- 172.21.15.157:0/2661543576 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100a5e60 con 0x7f59180b3b20 2023-11-06T03:04:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180b3b20 msgr2=0x7f591815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180b3b20 0x7f591815c1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5910009d90 tx=0x7f591001c890 comp rx=0 tx=0).stop 2023-11-06T03:04:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/2661543576 shutdown_connections 2023-11-06T03:04:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59180b3b20 0x7f591815c1b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- 172.21.15.157:0/2661543576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59181551e0 0x7f591806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/2661543576 >> 172.21.15.157:0/2661543576 conn(0x7f5918065930 msgr2=0x7f5918065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/2661543576 shutdown_connections 2023-11-06T03:04:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/2661543576 wait complete. 2023-11-06T03:04:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 Processor -- start 2023-11-06T03:04:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- start start 2023-11-06T03:04:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59180b3b20 0x7f591814b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f591815e5e0 0x7f591814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591814b980 con 0x7f59181551e0 2023-11-06T03:04:54.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918159fb0 con 0x7f591815e5e0 2023-11-06T03:04:54.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591814bbf0 con 0x7f59180b3b20 2023-11-06T03:04:54.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44312/0 (socket says 172.21.15.157:44312) 2023-11-06T03:04:54.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 -- 172.21.15.157:0/88095005 learned_addr learned my addr 172.21.15.157:0/88095005 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:54.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591659c640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59180b3b20 0x7f591814b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f5916d9d640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f591815e5e0 0x7f591814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 -- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59180b3b20 msgr2=0x7f591814b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59180b3b20 0x7f591814b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 -- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f591815e5e0 msgr2=0x7f591814e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f591815e5e0 0x7f591814e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 -- 172.21.15.157:0/88095005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080094d0 con 0x7f59181551e0 2023-11-06T03:04:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590ffff640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f590000c410 tx=0x7f590000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000054e0 con 0x7f59181551e0 2023-11-06T03:04:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5900003600 con 0x7f59181551e0 2023-11-06T03:04:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5910009a70 con 0x7f59181551e0 2023-11-06T03:04:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.776+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000039b0 con 0x7f59181551e0 2023-11-06T03:04:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591815a290 con 0x7f59181551e0 2023-11-06T03:04:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5918072750 con 0x7f59181551e0 2023-11-06T03:04:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5900003b10 con 0x7f59181551e0 2023-11-06T03:04:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f590effd640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 0x7f58f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f59000828e0 con 0x7f59181551e0 2023-11-06T03:04:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f591659c640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 0x7f58f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.780+0000 7f591659c640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 0x7f58f8067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5908006cb0 tx=0x7f5908009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.784+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5900053be0 con 0x7f59181551e0 2023-11-06T03:04:54.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f591814c0a0 con 0x7f59181551e0 2023-11-06T03:04:54.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f590effd640 1 -- 172.21.15.157:0/88095005 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f5900017020 con 0x7f59181551e0 2023-11-06T03:04:54.936 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:54.937 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:54.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 msgr2=0x7f58f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 0x7f58f8067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5908006cb0 tx=0x7f5908009240 comp rx=0 tx=0).stop 2023-11-06T03:04:54.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 msgr2=0x7f591814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:54.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f590000c410 tx=0x7f590000c8d0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 shutdown_connections 2023-11-06T03:04:54.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58f8065a60 0x7f58f8067f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f591815e5e0 0x7f591814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59181551e0 0x7f591814e160 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 --2- 172.21.15.157:0/88095005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59180b3b20 0x7f591814b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:54.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.936+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 >> 172.21.15.157:0/88095005 conn(0x7f5918065930 msgr2=0x7f59180ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:54.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.940+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 shutdown_connections 2023-11-06T03:04:54.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:54.940+0000 7f591ccdc640 1 -- 172.21.15.157:0/88095005 wait complete. 2023-11-06T03:04:54.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:54.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:55.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 Processor -- start 2023-11-06T03:04:55.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 -- start start 2023-11-06T03:04:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f234c1551e0 0x7f234c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c15f580 con 0x7f234c15ea60 2023-11-06T03:04:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c15efc0 con 0x7f234c1551e0 2023-11-06T03:04:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f2351b64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c15f2a0 con 0x7f234c15e5e0 2023-11-06T03:04:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f234c1551e0 0x7f234c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f234c15ea60 0x7f234c15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50710/0 (socket says 172.21.15.157:50710) 2023-11-06T03:04:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234bfff640 1 -- 172.21.15.157:0/1112940458 learned_addr learned my addr 172.21.15.157:0/1112940458 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234affd640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 -- 172.21.15.157:0/1112940458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 msgr2=0x7f234c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 -- 172.21.15.157:0/1112940458 >> v1:172.21.15.157:6789/0 conn(0x7f234c15ea60 legacy=0x7f234c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 -- 172.21.15.157:0/1112940458 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c06f5a0 con 0x7f234c1551e0 2023-11-06T03:04:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.072+0000 7f234b7fe640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c06fda0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097c0 tx=0x7f233c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e030042060014958 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234a7fc640 1 -- 172.21.15.157:0/1112940458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0ab600 con 0x7f234c1551e0 2023-11-06T03:04:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234a7fc640 1 -- 172.21.15.157:0/1112940458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c0a7050 con 0x7f234c1551e0 2023-11-06T03:04:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234a7fc640 1 -- 172.21.15.157:0/1112940458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0ab980 con 0x7f234c1551e0 2023-11-06T03:04:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/1112940458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 msgr2=0x7f234c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c06fda0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097c0 tx=0x7f233c01c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/1112940458 shutdown_connections 2023-11-06T03:04:55.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- 172.21.15.157:0/1112940458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c06fda0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/1112940458 >> 172.21.15.157:0/1112940458 conn(0x7f234c065930 msgr2=0x7f234c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/1112940458 shutdown_connections 2023-11-06T03:04:55.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/1112940458 wait complete. 2023-11-06T03:04:55.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 Processor -- start 2023-11-06T03:04:55.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- start start 2023-11-06T03:04:55.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234c15ea60 0x7f234c1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c1527b0 con 0x7f234c1551e0 2023-11-06T03:04:55.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c06bbc0 con 0x7f234c15ea60 2023-11-06T03:04:55.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c06b8e0 con 0x7f234c15e5e0 2023-11-06T03:04:55.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44328/0 (socket says 172.21.15.157:44328) 2023-11-06T03:04:55.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 -- 172.21.15.157:0/2280070569 learned_addr learned my addr 172.21.15.157:0/2280070569 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:55.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234affd640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 -- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 msgr2=0x7f234c14cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234bfff640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234c15ea60 0x7f234c1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c14cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 -- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234c15ea60 msgr2=0x7f234c1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234c15ea60 0x7f234c1f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 -- 172.21.15.157:0/2280070569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c009420 con 0x7f234c1551e0 2023-11-06T03:04:55.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234affd640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c14cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:55.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f234b7fe640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f233c01aa70 tx=0x7f233c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0b4600 con 0x7f234c1551e0 2023-11-06T03:04:55.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c0b4760 con 0x7f234c1551e0 2023-11-06T03:04:55.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c06bed0 con 0x7f234c1551e0 2023-11-06T03:04:55.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0b4b10 con 0x7f234c1551e0 2023-11-06T03:04:55.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.076+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c06c490 con 0x7f234c1551e0 2023-11-06T03:04:55.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c072750 con 0x7f234c1551e0 2023-11-06T03:04:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f233c0b2040 con 0x7f234c1551e0 2023-11-06T03:04:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f2333fff640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 0x7f2338067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f233c11d340 con 0x7f234c1551e0 2023-11-06T03:04:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f234affd640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 0x7f2338067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.080+0000 7f234affd640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 0x7f2338067f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2340009840 tx=0x7f2340008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.084+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f233c0eacd0 con 0x7f234c1551e0 2023-11-06T03:04:55.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.224+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f234c071730 con 0x7f234c1551e0 2023-11-06T03:04:55.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2333fff640 1 -- 172.21.15.157:0/2280070569 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f233c0ee2f0 con 0x7f234c1551e0 2023-11-06T03:04:55.227 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:55.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 msgr2=0x7f2338067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 0x7f2338067f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2340009840 tx=0x7f2340008040 comp rx=0 tx=0).stop 2023-11-06T03:04:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 msgr2=0x7f234c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f233c01aa70 tx=0x7f233c019160 comp rx=0 tx=0).stop 2023-11-06T03:04:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 shutdown_connections 2023-11-06T03:04:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2338065ab0 0x7f2338067f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f234c15ea60 0x7f234c1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f234c15e5e0 0x7f234c14cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 --2- 172.21.15.157:0/2280070569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f234c1551e0 0x7f234c152270 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 >> 172.21.15.157:0/2280070569 conn(0x7f234c065930 msgr2=0x7f234c0af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 shutdown_connections 2023-11-06T03:04:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.228+0000 7f2351b64640 1 -- 172.21.15.157:0/2280070569 wait complete. 2023-11-06T03:04:55.244 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:55.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.364+0000 7f2da8e0a640 1 Processor -- start 2023-11-06T03:04:55.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.364+0000 7f2da8e0a640 1 -- start start 2023-11-06T03:04:55.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da8e0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da8e0a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2da415e5e0 0x7f2da40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da8e0a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da415f580 con 0x7f2da415ea60 2023-11-06T03:04:55.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da8e0a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da415efc0 con 0x7f2da415e5e0 2023-11-06T03:04:55.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da8e0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da415f2a0 con 0x7f2da41551e0 2023-11-06T03:04:55.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2da415e5e0 0x7f2da40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da2d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2da415ea60 0x7f2da415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50716/0 (socket says 172.21.15.157:50716) 2023-11-06T03:04:55.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da2d9d640 1 -- 172.21.15.157:0/3617787237 learned_addr learned my addr 172.21.15.157:0/3617787237 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:55.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da259c640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 -- 172.21.15.157:0/3617787237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 msgr2=0x7f2da406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:55.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 -- 172.21.15.157:0/3617787237 >> v1:172.21.15.157:6789/0 conn(0x7f2da415ea60 legacy=0x7f2da415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 -- 172.21.15.157:0/3617787237 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da406f5a0 con 0x7f2da415e5e0 2023-11-06T03:04:55.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da159a640 1 -- 172.21.15.157:0/3617787237 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249650365 0 0) 0x7f2da415f580 con 0x7f2da415ea60 2023-11-06T03:04:55.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da259c640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:04:55.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da1d9b640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da40b3410 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097c0 tx=0x7f2d9801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb590d2eb9b47d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da159a640 1 -- 172.21.15.157:0/3617787237 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980ab600 con 0x7f2da415e5e0 2023-11-06T03:04:55.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da159a640 1 -- 172.21.15.157:0/3617787237 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d980a7050 con 0x7f2da415e5e0 2023-11-06T03:04:55.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.368+0000 7f2da159a640 1 -- 172.21.15.157:0/3617787237 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980ab980 con 0x7f2da415e5e0 2023-11-06T03:04:55.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/3617787237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 msgr2=0x7f2da40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da40b3410 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097c0 tx=0x7f2d9801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/3617787237 shutdown_connections 2023-11-06T03:04:55.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da40b3410 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/3617787237 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/3617787237 >> 172.21.15.157:0/3617787237 conn(0x7f2da4065930 msgr2=0x7f2da4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/3617787237 shutdown_connections 2023-11-06T03:04:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/3617787237 wait complete. 2023-11-06T03:04:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 Processor -- start 2023-11-06T03:04:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- start start 2023-11-06T03:04:55.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da413fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 0x7f2da414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41468c0 con 0x7f2da415e5e0 2023-11-06T03:04:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da40b2850 con 0x7f2da415ea60 2023-11-06T03:04:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41f43e0 con 0x7f2da41551e0 2023-11-06T03:04:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44344/0 (socket says 172.21.15.157:44344) 2023-11-06T03:04:55.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 -- 172.21.15.157:0/1196635397 learned_addr learned my addr 172.21.15.157:0/1196635397 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:55.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da259c640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da413fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da2d9d640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 0x7f2da414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 -- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 msgr2=0x7f2da413fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da413fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 -- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 msgr2=0x7f2da414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 0x7f2da414be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 -- 172.21.15.157:0/1196635397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009420 con 0x7f2da415e5e0 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da2d9d640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 0x7f2da414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da1d9b640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d98019ff0 tx=0x7f2d98019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980b4840 con 0x7f2da415e5e0 2023-11-06T03:04:55.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d980b49c0 con 0x7f2da415e5e0 2023-11-06T03:04:55.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980b4d70 con 0x7f2da415e5e0 2023-11-06T03:04:55.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da41f46c0 con 0x7f2da415e5e0 2023-11-06T03:04:55.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.372+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da4146550 con 0x7f2da415e5e0 2023-11-06T03:04:55.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da4146a30 con 0x7f2da415e5e0 2023-11-06T03:04:55.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d980b2040 con 0x7f2da415e5e0 2023-11-06T03:04:55.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2d92ffd640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 0x7f2d84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2d9811d620 con 0x7f2da415e5e0 2023-11-06T03:04:55.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2da259c640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 0x7f2d84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.376+0000 7f2da259c640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 0x7f2d84067f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2da414b730 tx=0x7f2d94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.380+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2d980eaed0 con 0x7f2da415e5e0 2023-11-06T03:04:55.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.532+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2da4072750 con 0x7f2da415e5e0 2023-11-06T03:04:55.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2d92ffd640 1 -- 172.21.15.157:0/1196635397 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d980ee4f0 con 0x7f2da415e5e0 2023-11-06T03:04:55.535 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:55.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 msgr2=0x7f2d84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 0x7f2d84067f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2da414b730 tx=0x7f2d94008040 comp rx=0 tx=0).stop 2023-11-06T03:04:55.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 msgr2=0x7f2da4146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d98019ff0 tx=0x7f2d98019160 comp rx=0 tx=0).stop 2023-11-06T03:04:55.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 shutdown_connections 2023-11-06T03:04:55.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d84065a60 0x7f2d84067f20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2da415ea60 0x7f2da414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da415e5e0 0x7f2da4146d50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 --2- 172.21.15.157:0/1196635397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da41551e0 0x7f2da413fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 >> 172.21.15.157:0/1196635397 conn(0x7f2da4065930 msgr2=0x7f2da40ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 shutdown_connections 2023-11-06T03:04:55.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.536+0000 7f2da8e0a640 1 -- 172.21.15.157:0/1196635397 wait complete. 2023-11-06T03:04:55.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:55.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 Processor -- start 2023-11-06T03:04:55.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 -- start start 2023-11-06T03:04:55.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3c015ce00 0x7ff3c00cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c015d510 con 0x7ff3c0071dd0 2023-11-06T03:04:55.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c015d7f0 con 0x7ff3c00cd950 2023-11-06T03:04:55.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3c4c13640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c00cdde0 con 0x7ff3c015ce00 2023-11-06T03:04:55.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3be59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3c015ce00 0x7ff3c00cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3bed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff3c0071dd0 0x7ff3c0160880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50726/0 (socket says 172.21.15.157:50726) 2023-11-06T03:04:55.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3bed9d640 1 -- 172.21.15.157:0/211941869 learned_addr learned my addr 172.21.15.157:0/211941869 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:55.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3be59c640 1 -- 172.21.15.157:0/211941869 >> v1:172.21.15.195:6789/0 conn(0x7ff3c00cd950 legacy=0x7ff3c00716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:55.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3be59c640 1 -- 172.21.15.157:0/211941869 >> v1:172.21.15.157:6789/0 conn(0x7ff3c0071dd0 legacy=0x7ff3c0160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3bd59a640 1 -- 172.21.15.157:0/211941869 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996237965 0 0) 0x7ff3c015d510 con 0x7ff3c0071dd0 2023-11-06T03:04:55.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.676+0000 7ff3be59c640 1 -- 172.21.15.157:0/211941869 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c015dad0 con 0x7ff3c015ce00 2023-11-06T03:04:55.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3be59c640 1 --2- 172.21.15.157:0/211941869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c015ce00 0x7ff3c00cd410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00097c0 tx=0x7ff3b001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e31b678aabf0a9a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bd59a640 1 -- 172.21.15.157:0/211941869 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ab600 con 0x7ff3c015ce00 2023-11-06T03:04:55.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bd59a640 1 -- 172.21.15.157:0/211941869 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b00a7050 con 0x7ff3c015ce00 2023-11-06T03:04:55.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bd59a640 1 -- 172.21.15.157:0/211941869 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ab980 con 0x7ff3c015ce00 2023-11-06T03:04:55.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- 172.21.15.157:0/211941869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c015ce00 msgr2=0x7ff3c00cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/211941869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c015ce00 0x7ff3c00cd410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00097c0 tx=0x7ff3b001c3b0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- 172.21.15.157:0/211941869 shutdown_connections 2023-11-06T03:04:55.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/211941869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c015ce00 0x7ff3c00cd410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- 172.21.15.157:0/211941869 >> 172.21.15.157:0/211941869 conn(0x7ff3c006d560 msgr2=0x7ff3c006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- 172.21.15.157:0/211941869 shutdown_connections 2023-11-06T03:04:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- 172.21.15.157:0/211941869 wait complete. 2023-11-06T03:04:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 Processor -- start 2023-11-06T03:04:55.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- start start 2023-11-06T03:04:55.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 0x7ff3c00c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c015ce00 0x7ff3c00b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00b0bb0 con 0x7ff3c00cd950 2023-11-06T03:04:55.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00b08d0 con 0x7ff3c0071dd0 2023-11-06T03:04:55.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3c4c13640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00b2e40 con 0x7ff3c015ce00 2023-11-06T03:04:55.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44360/0 (socket says 172.21.15.157:44360) 2023-11-06T03:04:55.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 -- 172.21.15.157:0/836430571 learned_addr learned my addr 172.21.15.157:0/836430571 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:55.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bed9d640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c015ce00 0x7ff3c00b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3be59c640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 0x7ff3c00c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 -- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c015ce00 msgr2=0x7ff3c00b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c015ce00 0x7ff3c00b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 -- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 msgr2=0x7ff3c00c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 0x7ff3c00c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.680+0000 7ff3bdd9b640 1 -- 172.21.15.157:0/836430571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009420 con 0x7ff3c00cd950 2023-11-06T03:04:55.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3be59c640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 0x7ff3c00c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:55.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3bdd9b640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c430 tx=0x7ff3b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4011790 con 0x7ff3c00cd950 2023-11-06T03:04:55.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b40118f0 con 0x7ff3c00cd950 2023-11-06T03:04:55.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c00b3120 con 0x7ff3c00cd950 2023-11-06T03:04:55.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4011ce0 con 0x7ff3c00cd950 2023-11-06T03:04:55.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c00b2c10 con 0x7ff3c00cd950 2023-11-06T03:04:55.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c00c4380 con 0x7ff3c00cd950 2023-11-06T03:04:55.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff3b401c460 con 0x7ff3c00cd950 2023-11-06T03:04:55.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 0x7ff3a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff3b40832b0 con 0x7ff3c00cd950 2023-11-06T03:04:55.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3be59c640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 0x7ff3a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.684+0000 7ff3be59c640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 0x7ff3a0067f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ad000 tx=0x7ff3b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.688+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff3b4050ad0 con 0x7ff3c00cd950 2023-11-06T03:04:55.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.836+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3c006bf00 con 0x7ff3c00cd950 2023-11-06T03:04:55.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3aeffd640 1 -- 172.21.15.157:0/836430571 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3b40540f0 con 0x7ff3c00cd950 2023-11-06T03:04:55.839 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 msgr2=0x7ff3a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 0x7ff3a0067f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00ad000 tx=0x7ff3b0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 msgr2=0x7ff3c00b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c430 tx=0x7ff3b400c8f0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 shutdown_connections 2023-11-06T03:04:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3a0065a60 0x7ff3a0067f20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3c015ce00 0x7ff3c00b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3c00cd950 0x7ff3c00b1fc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 --2- 172.21.15.157:0/836430571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3c0071dd0 0x7ff3c00c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 >> 172.21.15.157:0/836430571 conn(0x7ff3c006d560 msgr2=0x7ff3c006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 shutdown_connections 2023-11-06T03:04:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.840+0000 7ff3c4c13640 1 -- 172.21.15.157:0/836430571 wait complete. 2023-11-06T03:04:55.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:55.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 Processor -- start 2023-11-06T03:04:55.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 -- start start 2023-11-06T03:04:55.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49fc15ce30 0x7f49fc0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc06ecf0 0x7f49fc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc06da10 con 0x7f49fc06ac30 2023-11-06T03:04:55.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49fc06dcf0 con 0x7f49fc15ce30 2023-11-06T03:04:55.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f4a00f4f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc06dfd0 con 0x7f49fc06ecf0 2023-11-06T03:04:55.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49fc15ce30 0x7f49fc0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f49fc15ce30 0x7f49fc0d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44364/0 (socket says 172.21.15.157:44364) 2023-11-06T03:04:55.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 -- 172.21.15.157:0/1135723382 learned_addr learned my addr 172.21.15.157:0/1135723382 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:55.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fad9d640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc06ecf0 0x7f49fc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 -- 172.21.15.157:0/1135723382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc06ecf0 msgr2=0x7f49fc0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:55.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc06ecf0 0x7f49fc0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 -- 172.21.15.157:0/1135723382 >> v1:172.21.15.195:6789/0 conn(0x7f49fc06ac30 legacy=0x7f49fc0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.972+0000 7f49fa59c640 1 -- 172.21.15.157:0/1135723382 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc06e2b0 con 0x7f49fc15ce30 2023-11-06T03:04:55.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f49fa59c640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc15ce30 0x7f49fc0d07d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009870 tx=0x7f49ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b339324614bfa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f49f959a640 1 -- 172.21.15.157:0/1135723382 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0ab600 con 0x7f49fc15ce30 2023-11-06T03:04:55.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f49f959a640 1 -- 172.21.15.157:0/1135723382 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0ab760 con 0x7f49fc15ce30 2023-11-06T03:04:55.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f49f959a640 1 -- 172.21.15.157:0/1135723382 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0abaf0 con 0x7f49fc15ce30 2023-11-06T03:04:55.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1135723382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc15ce30 msgr2=0x7f49fc0d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc15ce30 0x7f49fc0d07d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009870 tx=0x7f49ec01c520 comp rx=0 tx=0).stop 2023-11-06T03:04:55.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1135723382 shutdown_connections 2023-11-06T03:04:55.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc06ecf0 0x7f49fc0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1135723382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc15ce30 0x7f49fc0d07d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1135723382 >> 172.21.15.157:0/1135723382 conn(0x7f49fc071b90 msgr2=0x7f49fc099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:55.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1135723382 shutdown_connections 2023-11-06T03:04:55.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.976+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1135723382 wait complete. 2023-11-06T03:04:55.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 Processor -- start 2023-11-06T03:04:55.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- start start 2023-11-06T03:04:55.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49fc06ecf0 0x7f49fc152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc15ce30 0x7f49fc14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc15d710 con 0x7f49fc06ac30 2023-11-06T03:04:55.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc15d9f0 con 0x7f49fc06ecf0 2023-11-06T03:04:55.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc15dcd0 con 0x7f49fc15ce30 2023-11-06T03:04:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44378/0 (socket says 172.21.15.157:44378) 2023-11-06T03:04:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 -- 172.21.15.157:0/1436418016 learned_addr learned my addr 172.21.15.157:0/1436418016 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fad9d640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc15ce30 0x7f49fc14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49f9d9b640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49fc06ecf0 0x7f49fc152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 -- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc15ce30 msgr2=0x7f49fc14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc15ce30 0x7f49fc14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 -- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49fc06ecf0 msgr2=0x7f49fc152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49fc06ecf0 0x7f49fc152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 -- 172.21.15.157:0/1436418016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec009480 con 0x7f49fc06ac30 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49fa59c640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01a150 tx=0x7f49ec0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0abce0 con 0x7f49fc06ac30 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0abe60 con 0x7f49fc06ac30 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec003860 con 0x7f49fc06ac30 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc15dfb0 con 0x7f49fc06ac30 2023-11-06T03:04:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc15d330 con 0x7f49fc06ac30 2023-11-06T03:04:55.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.980+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc072750 con 0x7f49fc06ac30 2023-11-06T03:04:55.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.984+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f49ec0039c0 con 0x7f49fc06ac30 2023-11-06T03:04:55.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.984+0000 7f49eaffd640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 0x7f49dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:55.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.984+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f49ec11d8d0 con 0x7f49fc06ac30 2023-11-06T03:04:55.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.984+0000 7f49f9d9b640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 0x7f49dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:55.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.984+0000 7f49f9d9b640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 0x7f49dc067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ca640 tx=0x7f49f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:55.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:55.988+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f49ec0eb180 con 0x7f49fc06ac30 2023-11-06T03:04:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.140+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49fc15d850 con 0x7f49fc06ac30 2023-11-06T03:04:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.140+0000 7f49eaffd640 1 -- 172.21.15.157:0/1436418016 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49fc15d850 con 0x7f49fc06ac30 2023-11-06T03:04:56.141 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 msgr2=0x7f49dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 0x7f49dc067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ca640 tx=0x7f49f0009040 comp rx=0 tx=0).stop 2023-11-06T03:04:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 msgr2=0x7f49fc15f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49ec01a150 tx=0x7f49ec0192c0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 shutdown_connections 2023-11-06T03:04:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49dc065a60 0x7f49dc067f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49fc15ce30 0x7f49fc14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49fc06ecf0 0x7f49fc152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 --2- 172.21.15.157:0/1436418016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49fc06ac30 0x7f49fc15f920 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 >> 172.21.15.157:0/1436418016 conn(0x7f49fc071b90 msgr2=0x7f49fc157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 shutdown_connections 2023-11-06T03:04:56.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.144+0000 7f4a00f4f640 1 -- 172.21.15.157:0/1436418016 wait complete. 2023-11-06T03:04:56.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 Processor -- start 2023-11-06T03:04:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 -- start start 2023-11-06T03:04:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd6f806bdb0 0x7fd6f80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80cf1a0 con 0x7fd6f80ce800 2023-11-06T03:04:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f806c3d0 con 0x7fd6f806bdb0 2023-11-06T03:04:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6fe0b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f806f5a0 con 0x7fd6f8146370 2023-11-06T03:04:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd6f806bdb0 0x7fd6f80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd6f806bdb0 0x7fd6f80cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:32794/0 (socket says 172.21.15.157:32794) 2023-11-06T03:04:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f6ffd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd6f80ce800 0x7fd6f806b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35884/0 (socket says 172.21.15.157:35884) 2023-11-06T03:04:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 -- 172.21.15.157:0/3644641277 learned_addr learned my addr 172.21.15.157:0/3644641277 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37656/0 (socket says 172.21.15.157:37656) 2023-11-06T03:04:56.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 -- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 msgr2=0x7fd6f80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 -- 172.21.15.157:0/3644641277 >> v1:172.21.15.195:6789/0 conn(0x7fd6f80ce800 legacy=0x7fd6f806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f7fff640 1 -- 172.21.15.157:0/3644641277 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f806f880 con 0x7fd6f806bdb0 2023-11-06T03:04:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.272+0000 7fd6f77fe640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:04:56.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6f7fff640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f80cec60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009e50 tx=0x7fd6f001c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e50f74431f0eea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6f67fc640 1 -- 172.21.15.157:0/3644641277 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a5840 con 0x7fd6f806bdb0 2023-11-06T03:04:56.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6f67fc640 1 -- 172.21.15.157:0/3644641277 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00a59a0 con 0x7fd6f806bdb0 2023-11-06T03:04:56.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6f67fc640 1 -- 172.21.15.157:0/3644641277 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00a5d30 con 0x7fd6f806bdb0 2023-11-06T03:04:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 msgr2=0x7fd6f80cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f80cec60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009e50 tx=0x7fd6f001c980 comp rx=0 tx=0).stop 2023-11-06T03:04:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/3644641277 shutdown_connections 2023-11-06T03:04:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f80cec60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/3644641277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f8146370 0x7fd6f80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/3644641277 >> 172.21.15.157:0/3644641277 conn(0x7fd6f8065930 msgr2=0x7fd6f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/3644641277 shutdown_connections 2023-11-06T03:04:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/3644641277 wait complete. 2023-11-06T03:04:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 Processor -- start 2023-11-06T03:04:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- start start 2023-11-06T03:04:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f8160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f80ce800 0x7fd6f80abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81f9ec0 con 0x7fd6f8146370 2023-11-06T03:04:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8160c60 con 0x7fd6f806bdb0 2023-11-06T03:04:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.276+0000 7fd6fe0b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81f9030 con 0x7fd6f80ce800 2023-11-06T03:04:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44392/0 (socket says 172.21.15.157:44392) 2023-11-06T03:04:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 -- 172.21.15.157:0/4096000556 learned_addr learned my addr 172.21.15.157:0/4096000556 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f6ffd640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f80ce800 0x7fd6f80abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f77fe640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f8160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 -- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f80ce800 msgr2=0x7fd6f80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f80ce800 0x7fd6f80abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 -- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 msgr2=0x7fd6f8160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f8160720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 -- 172.21.15.157:0/4096000556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009420 con 0x7fd6f8146370 2023-11-06T03:04:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f7fff640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6f001a5b0 tx=0x7fd6f0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b8670 con 0x7fd6f8146370 2023-11-06T03:04:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00b87f0 con 0x7fd6f8146370 2023-11-06T03:04:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f0009b30 con 0x7fd6f8146370 2023-11-06T03:04:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b8ba0 con 0x7fd6f8146370 2023-11-06T03:04:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f81f9520 con 0x7fd6f8146370 2023-11-06T03:04:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f81fa000 con 0x7fd6f8146370 2023-11-06T03:04:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd6f00b8d00 con 0x7fd6f8146370 2023-11-06T03:04:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 0x7fd6e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd6f011e730 con 0x7fd6f8146370 2023-11-06T03:04:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f77fe640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 0x7fd6e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.280+0000 7fd6f77fe640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 0x7fd6e4067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b0100 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.284+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd6f81fa000 con 0x7fd6f8146370 2023-11-06T03:04:56.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6f81fa000 con 0x7fd6f8146370 2023-11-06T03:04:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6dffff640 1 -- 172.21.15.157:0/4096000556 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd6f81fa000 con 0x7fd6f8146370 2023-11-06T03:04:56.439 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:56.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 msgr2=0x7fd6e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 0x7fd6e4067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80b0100 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 msgr2=0x7fd6f80cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6f001a5b0 tx=0x7fd6f0019720 comp rx=0 tx=0).stop 2023-11-06T03:04:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 shutdown_connections 2023-11-06T03:04:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e4065a60 0x7fd6e4067f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f8146370 0x7fd6f80cca30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6f80ce800 0x7fd6f80abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 --2- 172.21.15.157:0/4096000556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f806bdb0 0x7fd6f8160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 >> 172.21.15.157:0/4096000556 conn(0x7fd6f8065930 msgr2=0x7fd6f80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 shutdown_connections 2023-11-06T03:04:56.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.440+0000 7fd6fe0b6640 1 -- 172.21.15.157:0/4096000556 wait complete. 2023-11-06T03:04:56.457 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:04:56.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:56.457 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-06T03:04:56.469 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:04:56.470 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:04:56.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:04:56.471 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:04:56.515 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:56.516 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:56.522 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:56.523 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:56.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:56.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.684+0000 7fefd0401640 1 Processor -- start 2023-11-06T03:04:56.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.684+0000 7fefd0401640 1 -- start start 2023-11-06T03:04:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefd0401640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefc80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefd0401640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fefc806d000 0x7fefc80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefd0401640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc815e440 con 0x7fefc815dea0 2023-11-06T03:04:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefd0401640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc815adb0 con 0x7fefc806d000 2023-11-06T03:04:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefd0401640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc806c240 con 0x7fefc81599c0 2023-11-06T03:04:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fefc815dea0 0x7fefc8070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50742/0 (socket says 172.21.15.157:50742) 2023-11-06T03:04:56.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 -- 172.21.15.157:0/1426036733 learned_addr learned my addr 172.21.15.157:0/1426036733 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027240719 0 0) 0x7fefc815e440 con 0x7fefc815dea0 2023-11-06T03:04:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd99c640 1 --2- 172.21.15.157:0/1426036733 >> v2:172.21.15.195:3300/0 conn(0x7fefc806d000 0x7fefc80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefb4003530 con 0x7fefc815dea0 2023-11-06T03:04:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce19d640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefc80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445732318 0 0) 0x7fefb4003530 con 0x7fefc815dea0 2023-11-06T03:04:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 msgr2=0x7fefc80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefc80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 >> v2:172.21.15.195:3300/0 conn(0x7fefc806d000 msgr2=0x7fefc80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 --2- 172.21.15.157:0/1426036733 >> v2:172.21.15.195:3300/0 conn(0x7fefc806d000 0x7fefc80b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc806c520 con 0x7fefc815dea0 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fefc0001a00 con 0x7fefc815dea0 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 >> v1:172.21.15.157:6789/0 conn(0x7fefc815dea0 legacy=0x7fefc8070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefb4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefb4003f00 0x7fefb40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb4004d10 0x7fefb4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc815adb0 con 0x7fefb4004d10 2023-11-06T03:04:56.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc806c240 con 0x7fefb4003f00 2023-11-06T03:04:56.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb400b990 con 0x7fefc81599c0 2023-11-06T03:04:56.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb4004d10 0x7fefb4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd99c640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefb4003f00 0x7fefb40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce19d640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefb4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 -- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 msgr2=0x7fefb4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefc81599c0 0x7fefb4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 -- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefb4003f00 msgr2=0x7fefb40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefb4003f00 0x7fefb40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 -- 172.21.15.157:0/1426036733 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4003530 con 0x7fefb4004d10 2023-11-06T03:04:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefce99e640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb4004d10 0x7fefb4005190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fefc000a830 tx=0x7fefc001d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7075d4224d9eb513 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00ad630 con 0x7fefb4004d10 2023-11-06T03:04:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc00a8070 con 0x7fefb4004d10 2023-11-06T03:04:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.688+0000 7fefcd19b640 1 -- 172.21.15.157:0/1426036733 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00ad9d0 con 0x7fefb4004d10 2023-11-06T03:04:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/1426036733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb4004d10 msgr2=0x7fefb4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 --2- 172.21.15.157:0/1426036733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefb4004d10 0x7fefb4005190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fefc000a830 tx=0x7fefc001d2e0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 -- 172.21.15.157:0/1426036733 reap_dead start 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/1426036733 shutdown_connections 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/1426036733 >> 172.21.15.157:0/1426036733 conn(0x7fefc8065930 msgr2=0x7fefc8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/1426036733 shutdown_connections 2023-11-06T03:04:56.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/1426036733 wait complete. 2023-11-06T03:04:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 Processor -- start 2023-11-06T03:04:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- start start 2023-11-06T03:04:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefb4004d10 0x7fefc80cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefc815c4f0 0x7fefc8143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80cc0b0 con 0x7fefc80cd150 2023-11-06T03:04:56.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80cc390 con 0x7fefc815c4f0 2023-11-06T03:04:56.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80cc670 con 0x7fefb4004d10 2023-11-06T03:04:56.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44416/0 (socket says 172.21.15.157:44416) 2023-11-06T03:04:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 -- 172.21.15.157:0/3316497193 learned_addr learned my addr 172.21.15.157:0/3316497193 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce19d640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefb4004d10 0x7fefc80cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefcd99c640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefc815c4f0 0x7fefc8143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 -- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefb4004d10 msgr2=0x7fefc80cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefb4004d10 0x7fefc80cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 -- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefc815c4f0 msgr2=0x7fefc8143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefc815c4f0 0x7fefc8143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 -- 172.21.15.157:0/3316497193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80cc950 con 0x7fefc80cd150 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefce99e640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fefc001b9f0 tx=0x7fefb4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00017c0 con 0x7fefc80cd150 2023-11-06T03:04:56.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0001920 con 0x7fefc80cd150 2023-11-06T03:04:56.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0004d70 con 0x7fefc80cd150 2023-11-06T03:04:56.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc80ccc30 con 0x7fefc80cd150 2023-11-06T03:04:56.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.692+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc8143850 con 0x7fefc80cd150 2023-11-06T03:04:56.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc80cce20 con 0x7fefc80cd150 2023-11-06T03:04:56.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefc00aa030 con 0x7fefc80cd150 2023-11-06T03:04:56.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefbeffd640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 0x7fefac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fefc011d050 con 0x7fefc80cd150 2023-11-06T03:04:56.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefce19d640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 0x7fefac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.696+0000 7fefce19d640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 0x7fefac067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefc400bcd0 tx=0x7fefc400b3a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.700+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fefc00ea870 con 0x7fefc80cd150 2023-11-06T03:04:56.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.844+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefc8072750 con 0x7fefc80cd150 2023-11-06T03:04:56.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefbeffd640 1 -- 172.21.15.157:0/3316497193 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7fefc00ede90 con 0x7fefc80cd150 2023-11-06T03:04:56.847 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:56.848 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:56.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 msgr2=0x7fefac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 0x7fefac067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fefc400bcd0 tx=0x7fefc400b3a0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 msgr2=0x7fefc81532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fefc001b9f0 tx=0x7fefb4005800 comp rx=0 tx=0).stop 2023-11-06T03:04:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 shutdown_connections 2023-11-06T03:04:56.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefac065a60 0x7fefac067f20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefc80cd150 0x7fefc81532b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefc815c4f0 0x7fefc8143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 --2- 172.21.15.157:0/3316497193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefb4004d10 0x7fefc80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 >> 172.21.15.157:0/3316497193 conn(0x7fefc8065930 msgr2=0x7fefc8153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 shutdown_connections 2023-11-06T03:04:56.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.848+0000 7fefd0401640 1 -- 172.21.15.157:0/3316497193 wait complete. 2023-11-06T03:04:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:56.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 Processor -- start 2023-11-06T03:04:56.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 -- start start 2023-11-06T03:04:56.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2181551e0 0x7fd21806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 0x7fd2180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21815f580 con 0x7fd21815ea60 2023-11-06T03:04:56.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21815efc0 con 0x7fd2181551e0 2023-11-06T03:04:56.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21e26f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21815f2a0 con 0x7fd21815e5e0 2023-11-06T03:04:56.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd20ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 0x7fd2180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21c80c640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd21815ea60 0x7fd21815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35886/0 (socket says 172.21.15.157:35886) 2023-11-06T03:04:56.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd21c80c640 1 -- 172.21.15.157:0/3189835678 learned_addr learned my addr 172.21.15.157:0/3189835678 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:56.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd217fff640 1 --2- 172.21.15.157:0/3189835678 >> v2:172.21.15.195:3300/0 conn(0x7fd2181551e0 0x7fd21806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:56.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd20ffff640 1 -- 172.21.15.157:0/3189835678 >> v2:172.21.15.195:3300/0 conn(0x7fd2181551e0 msgr2=0x7fd21806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:56.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd20ffff640 1 --2- 172.21.15.157:0/3189835678 >> v2:172.21.15.195:3300/0 conn(0x7fd2181551e0 0x7fd21806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd20ffff640 1 -- 172.21.15.157:0/3189835678 >> v1:172.21.15.195:6789/0 conn(0x7fd21815ea60 legacy=0x7fd21815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd20ffff640 1 -- 172.21.15.157:0/3189835678 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21806f5a0 con 0x7fd21815e5e0 2023-11-06T03:04:56.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.988+0000 7fd2177fe640 1 -- 172.21.15.157:0/3189835678 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941408065 0 0) 0x7fd21815f580 con 0x7fd21815ea60 2023-11-06T03:04:56.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- 172.21.15.157:0/3189835678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 0x7fd2180b3410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2000097c0 tx=0x7fd20001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cdd12af7879a1ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:56.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd2177fe640 1 -- 172.21.15.157:0/3189835678 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000ab600 con 0x7fd21815e5e0 2023-11-06T03:04:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd2177fe640 1 -- 172.21.15.157:0/3189835678 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2000a7050 con 0x7fd21815e5e0 2023-11-06T03:04:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd2177fe640 1 -- 172.21.15.157:0/3189835678 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000ab980 con 0x7fd21815e5e0 2023-11-06T03:04:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- 172.21.15.157:0/3189835678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 msgr2=0x7fd2180b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- 172.21.15.157:0/3189835678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 0x7fd2180b3410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2000097c0 tx=0x7fd20001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- 172.21.15.157:0/3189835678 shutdown_connections 2023-11-06T03:04:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- 172.21.15.157:0/3189835678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd21815e5e0 0x7fd2180b3410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- 172.21.15.157:0/3189835678 >> v2:172.21.15.195:3300/0 conn(0x7fd2181551e0 0x7fd21806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- 172.21.15.157:0/3189835678 >> 172.21.15.157:0/3189835678 conn(0x7fd218065930 msgr2=0x7fd218065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- 172.21.15.157:0/3189835678 shutdown_connections 2023-11-06T03:04:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- 172.21.15.157:0/3189835678 wait complete. 2023-11-06T03:04:56.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 Processor -- start 2023-11-06T03:04:56.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- start start 2023-11-06T03:04:56.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2181551e0 0x7fd2180adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21815ea60 0x7fd21815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:56.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21814caa0 con 0x7fd21815e5e0 2023-11-06T03:04:56.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218159770 con 0x7fd21815ea60 2023-11-06T03:04:56.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21e26f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd218159a50 con 0x7fd2181551e0 2023-11-06T03:04:56.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44426/0 (socket says 172.21.15.157:44426) 2023-11-06T03:04:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 -- 172.21.15.157:0/2008251563 learned_addr learned my addr 172.21.15.157:0/2008251563 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd21c80c640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21815ea60 0x7fd21815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd217fff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2181551e0 0x7fd2180adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 -- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2181551e0 msgr2=0x7fd2180adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2181551e0 0x7fd2180adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 -- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21815ea60 msgr2=0x7fd21815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21815ea60 0x7fd21815d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 -- 172.21.15.157:0/2008251563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200009420 con 0x7fd21815e5e0 2023-11-06T03:04:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.992+0000 7fd20ffff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd20001aa70 tx=0x7fd200019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200006e20 con 0x7fd21815e5e0 2023-11-06T03:04:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd200004020 con 0x7fd21815e5e0 2023-11-06T03:04:57.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd218159d30 con 0x7fd21815e5e0 2023-11-06T03:04:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200004390 con 0x7fd21815e5e0 2023-11-06T03:04:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21815a2f0 con 0x7fd21815e5e0 2023-11-06T03:04:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21815a4e0 con 0x7fd21815e5e0 2023-11-06T03:04:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd2000b2040 con 0x7fd21815e5e0 2023-11-06T03:04:57.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 0x7fd1fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd20011d190 con 0x7fd21815e5e0 2023-11-06T03:04:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd217fff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 0x7fd1fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:56.996+0000 7fd217fff640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 0x7fd1fc067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd208009af0 tx=0x7fd208008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.000+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd2000eaa40 con 0x7fd21815e5e0 2023-11-06T03:04:57.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.152+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd218072750 con 0x7fd21815e5e0 2023-11-06T03:04:57.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.152+0000 7fd2157fa640 1 -- 172.21.15.157:0/2008251563 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2000044f0 con 0x7fd21815e5e0 2023-11-06T03:04:57.153 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 msgr2=0x7fd1fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 0x7fd1fc067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd208009af0 tx=0x7fd208008040 comp rx=0 tx=0).stop 2023-11-06T03:04:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 msgr2=0x7fd21814c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd20001aa70 tx=0x7fd200019160 comp rx=0 tx=0).stop 2023-11-06T03:04:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 shutdown_connections 2023-11-06T03:04:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd1fc065a60 0x7fd1fc067f20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd21815ea60 0x7fd21815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21815e5e0 0x7fd21814c560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 --2- 172.21.15.157:0/2008251563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2181551e0 0x7fd2180adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 >> 172.21.15.157:0/2008251563 conn(0x7fd218065930 msgr2=0x7fd2180ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 shutdown_connections 2023-11-06T03:04:57.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.156+0000 7fd21e26f640 1 -- 172.21.15.157:0/2008251563 wait complete. 2023-11-06T03:04:57.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:57.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 Processor -- start 2023-11-06T03:04:57.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 -- start start 2023-11-06T03:04:57.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73f806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f815f580 con 0x7f73f80b3b20 2023-11-06T03:04:57.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f815efc0 con 0x7f73f815e5e0 2023-11-06T03:04:57.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73fdee0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f815f2a0 con 0x7f73f81551e0 2023-11-06T03:04:57.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f73f80b3b20 0x7f73f815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50756/0 (socket says 172.21.15.157:50756) 2023-11-06T03:04:57.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f7fff640 1 -- 172.21.15.157:0/940115953 learned_addr learned my addr 172.21.15.157:0/940115953 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:57.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73f806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433822457 0 0) 0x7f73f815f580 con 0x7f73f80b3b20 2023-11-06T03:04:57.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e0003530 con 0x7f73f80b3b20 2023-11-06T03:04:57.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372669301 0 0) 0x7f73e0003530 con 0x7f73f80b3b20 2023-11-06T03:04:57.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 msgr2=0x7f73f806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:57.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73f806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 >> v1:172.21.15.195:6789/0 conn(0x7f73f815e5e0 legacy=0x7f73f80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f806f5a0 con 0x7f73f80b3b20 2023-11-06T03:04:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f73f0001a00 con 0x7f73f80b3b20 2023-11-06T03:04:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 >> v1:172.21.15.157:6789/0 conn(0x7f73f80b3b20 legacy=0x7f73f815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e00039b0 0x7f73e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e0004360 0x7f73e0004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73e0005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f815f2a0 con 0x7f73e00039b0 2023-11-06T03:04:57.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e000ba40 con 0x7f73e0004360 2023-11-06T03:04:57.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e000bc00 con 0x7f73f81551e0 2023-11-06T03:04:57.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e00039b0 0x7f73e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f6ffd640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e0004360 0x7f73e0004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f7fff640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73e0005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 -- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 msgr2=0x7f73e0005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:57.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f81551e0 0x7f73e0005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 -- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e0004360 msgr2=0x7f73e0004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:57.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e0004360 0x7f73e0004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.292+0000 7f73f77fe640 1 -- 172.21.15.157:0/940115953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0003530 con 0x7f73e00039b0 2023-11-06T03:04:57.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.296+0000 7f73f77fe640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e00039b0 0x7f73e0003d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ae0 tx=0x7f73e801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffa08fc70e2a099 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e80a6510 con 0x7f73e00039b0 2023-11-06T03:04:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e80a6670 con 0x7f73e00039b0 2023-11-06T03:04:57.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f67fc640 1 -- 172.21.15.157:0/940115953 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e80a6a20 con 0x7f73e00039b0 2023-11-06T03:04:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/940115953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e00039b0 msgr2=0x7f73e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 --2- 172.21.15.157:0/940115953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e00039b0 0x7f73e0003d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ae0 tx=0x7f73e801ca20 comp rx=0 tx=0).stop 2023-11-06T03:04:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f7fff640 1 -- 172.21.15.157:0/940115953 reap_dead start 2023-11-06T03:04:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/940115953 shutdown_connections 2023-11-06T03:04:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/940115953 >> 172.21.15.157:0/940115953 conn(0x7f73f8065930 msgr2=0x7f73f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/940115953 shutdown_connections 2023-11-06T03:04:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/940115953 wait complete. 2023-11-06T03:04:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 Processor -- start 2023-11-06T03:04:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- start start 2023-11-06T03:04:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 0x7f73f8153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 0x7f73f814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f814e580 con 0x7f73f814a940 2023-11-06T03:04:57.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f814e860 con 0x7f73f814af70 2023-11-06T03:04:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f814eb40 con 0x7f73f814a560 2023-11-06T03:04:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44440/0 (socket says 172.21.15.157:44440) 2023-11-06T03:04:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 -- 172.21.15.157:0/4270875123 learned_addr learned my addr 172.21.15.157:0/4270875123 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f77fe640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 0x7f73f8153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f7fff640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 0x7f73f814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 -- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 msgr2=0x7f73f8153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 0x7f73f8153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 -- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 msgr2=0x7f73f814b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 0x7f73f814b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 -- 172.21.15.157:0/4270875123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f814ee20 con 0x7f73f814a940 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f77fe640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 0x7f73f8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f7fff640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 0x7f73f814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73f6ffd640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00f7c0 tx=0x7f73ec00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00fe70 con 0x7f73f814a940 2023-11-06T03:04:57.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0052e0 con 0x7f73f814a940 2023-11-06T03:04:57.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f814bce0 con 0x7f73f814a940 2023-11-06T03:04:57.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00d730 con 0x7f73f814a940 2023-11-06T03:04:57.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.300+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f8152a30 con 0x7f73f814a940 2023-11-06T03:04:57.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f814e6c0 con 0x7f73f814a940 2023-11-06T03:04:57.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73ec00d8b0 con 0x7f73f814a940 2023-11-06T03:04:57.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73dffff640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 0x7f73e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f73ec0833e0 con 0x7f73f814a940 2023-11-06T03:04:57.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73f77fe640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 0x7f73e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.304+0000 7f73f77fe640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 0x7f73e4067f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.308+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f73f814e6c0 con 0x7f73f814a940 2023-11-06T03:04:57.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.448+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f73f814e6c0 con 0x7f73f814a940 2023-11-06T03:04:57.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.448+0000 7f73dffff640 1 -- 172.21.15.157:0/4270875123 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f73f814e6c0 con 0x7f73f814a940 2023-11-06T03:04:57.449 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:57.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 msgr2=0x7f73e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 0x7f73e4067f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73e80af000 tx=0x7f73e8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:57.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 msgr2=0x7f73f8154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00f7c0 tx=0x7f73ec00fc80 comp rx=0 tx=0).stop 2023-11-06T03:04:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 shutdown_connections 2023-11-06T03:04:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73e4065a60 0x7f73e4067f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73f814af70 0x7f73f814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73f814a940 0x7f73f8154360 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 --2- 172.21.15.157:0/4270875123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73f814a560 0x7f73f8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 >> 172.21.15.157:0/4270875123 conn(0x7f73f8065930 msgr2=0x7f73f80ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 shutdown_connections 2023-11-06T03:04:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.452+0000 7f73fdee0640 1 -- 172.21.15.157:0/4270875123 wait complete. 2023-11-06T03:04:57.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:57.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 Processor -- start 2023-11-06T03:04:57.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 -- start start 2023-11-06T03:04:57.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26c00b0ed0 0x7f26c0167820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c01606c0 con 0x7f26c0066040 2023-11-06T03:04:57.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c0160910 con 0x7f26c015ccc0 2023-11-06T03:04:57.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26c53e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c0070d30 con 0x7f26c00b0ed0 2023-11-06T03:04:57.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26bf7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26c00b0ed0 0x7f26c0167820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26b67fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f26c0066040 0x7f26c00b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50764/0 (socket says 172.21.15.157:50764) 2023-11-06T03:04:57.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26b67fc640 1 -- 172.21.15.157:0/3072709506 learned_addr learned my addr 172.21.15.157:0/3072709506 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:57.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26bf7fe640 1 -- 172.21.15.157:0/3072709506 >> v1:172.21.15.195:6789/0 conn(0x7f26c015ccc0 legacy=0x7f26c0065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:57.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26bf7fe640 1 -- 172.21.15.157:0/3072709506 >> v1:172.21.15.157:6789/0 conn(0x7f26c0066040 legacy=0x7f26c00b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.584+0000 7f26bf7fe640 1 -- 172.21.15.157:0/3072709506 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0160bf0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26bf7fe640 1 --2- 172.21.15.157:0/3072709506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c0167820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009d90 tx=0x7f26b801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e208421f907e19 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26be7fc640 1 -- 172.21.15.157:0/3072709506 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b80a59d0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26be7fc640 1 -- 172.21.15.157:0/3072709506 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b80a5b30 con 0x7f26c00b0ed0 2023-11-06T03:04:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26be7fc640 1 -- 172.21.15.157:0/3072709506 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b80ad440 con 0x7f26c00b0ed0 2023-11-06T03:04:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- 172.21.15.157:0/3072709506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 msgr2=0x7f26c0167820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 --2- 172.21.15.157:0/3072709506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c0167820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009d90 tx=0x7f26b801c910 comp rx=0 tx=0).stop 2023-11-06T03:04:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- 172.21.15.157:0/3072709506 shutdown_connections 2023-11-06T03:04:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 --2- 172.21.15.157:0/3072709506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c0167820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- 172.21.15.157:0/3072709506 >> 172.21.15.157:0/3072709506 conn(0x7f26c00732f0 msgr2=0x7f26c0069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- 172.21.15.157:0/3072709506 shutdown_connections 2023-11-06T03:04:57.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- 172.21.15.157:0/3072709506 wait complete. 2023-11-06T03:04:57.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 Processor -- start 2023-11-06T03:04:57.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.588+0000 7f26c53e3640 1 -- start start 2023-11-06T03:04:57.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 0x7f26c015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 0x7f26c01f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c015ea80 con 0x7f26c00b0ed0 2023-11-06T03:04:57.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c015ecb0 con 0x7f26c0066040 2023-11-06T03:04:57.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c015ef90 con 0x7f26c015ccc0 2023-11-06T03:04:57.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44450/0 (socket says 172.21.15.157:44450) 2023-11-06T03:04:57.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 -- 172.21.15.157:0/484488468 learned_addr learned my addr 172.21.15.157:0/484488468 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26bf7fe640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 0x7f26c01f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 -- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 msgr2=0x7f26c01f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 0x7f26c01f8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 -- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 msgr2=0x7f26c015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26beffd640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 0x7f26c015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 0x7f26c015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 -- 172.21.15.157:0/484488468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b8009a70 con 0x7f26c00b0ed0 2023-11-06T03:04:57.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26bf7fe640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 0x7f26c01f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:57.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26beffd640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 0x7f26c015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:57.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b67fc640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c500 tx=0x7f26a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8005cf0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8005e50 con 0x7f26c00b0ed0 2023-11-06T03:04:57.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c015f270 con 0x7f26c00b0ed0 2023-11-06T03:04:57.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80072f0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c0153c20 con 0x7f26c00b0ed0 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c015f4b0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26a8007450 con 0x7f26c00b0ed0 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 0x7f26a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f26a80827f0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.592+0000 7f26beffd640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 0x7f26a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.596+0000 7f26beffd640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 0x7f26a0067f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26c015d8b0 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.596+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f26c015f4b0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.736+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26c015f4b0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.736+0000 7f26b7fff640 1 -- 172.21.15.157:0/484488468 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26c015f4b0 con 0x7f26c00b0ed0 2023-11-06T03:04:57.739 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:57.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 msgr2=0x7f26a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 0x7f26a0067f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26c015d8b0 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2023-11-06T03:04:57.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 msgr2=0x7f26c01536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c500 tx=0x7f26a800c9c0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 shutdown_connections 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f26a0065a60 0x7f26a0067f20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26c015ccc0 0x7f26c01f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26c00b0ed0 0x7f26c01536e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 --2- 172.21.15.157:0/484488468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26c0066040 0x7f26c015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 >> 172.21.15.157:0/484488468 conn(0x7f26c00732f0 msgr2=0x7f26c015a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 shutdown_connections 2023-11-06T03:04:57.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.740+0000 7f26c53e3640 1 -- 172.21.15.157:0/484488468 wait complete. 2023-11-06T03:04:57.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 Processor -- start 2023-11-06T03:04:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 -- start start 2023-11-06T03:04:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f422c0cd730 0x7f422c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0cc2b0 con 0x7f422c15ac80 2023-11-06T03:04:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c0b01a0 con 0x7f422c0cd730 2023-11-06T03:04:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f4232b33640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0cfb10 con 0x7f422c0b2b10 2023-11-06T03:04:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f42310d0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f422c0cd730 0x7f422c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37762/0 (socket says 172.21.15.157:37762) 2023-11-06T03:04:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 -- 172.21.15.157:0/160520668 learned_addr learned my addr 172.21.15.157:0/160520668 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 -- 172.21.15.157:0/160520668 >> v2:172.21.15.195:3300/0 conn(0x7f422c0cd730 msgr2=0x7f422c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 --2- 172.21.15.157:0/160520668 >> v2:172.21.15.195:3300/0 conn(0x7f422c0cd730 0x7f422c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 -- 172.21.15.157:0/160520668 >> v1:172.21.15.195:6789/0 conn(0x7f422c15ac80 legacy=0x7f422c0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.880+0000 7f422bfff640 1 -- 172.21.15.157:0/160520668 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0cfdf0 con 0x7f422c0b2b10 2023-11-06T03:04:57.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f422bfff640 1 --2- 172.21.15.157:0/160520668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f421c009870 tx=0x7f421c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9510841a8a62239b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f422b7fe640 1 -- 172.21.15.157:0/160520668 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0a9400 con 0x7f422c0b2b10 2023-11-06T03:04:57.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f422b7fe640 1 -- 172.21.15.157:0/160520668 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c0a9560 con 0x7f422c0b2b10 2023-11-06T03:04:57.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f422b7fe640 1 -- 172.21.15.157:0/160520668 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0a98f0 con 0x7f422c0b2b10 2023-11-06T03:04:57.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 -- 172.21.15.157:0/160520668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 msgr2=0x7f422c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 --2- 172.21.15.157:0/160520668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f421c009870 tx=0x7f421c01c540 comp rx=0 tx=0).stop 2023-11-06T03:04:57.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 -- 172.21.15.157:0/160520668 shutdown_connections 2023-11-06T03:04:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 --2- 172.21.15.157:0/160520668 >> v2:172.21.15.195:3300/0 conn(0x7f422c0cd730 0x7f422c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 --2- 172.21.15.157:0/160520668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c0ce550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 -- 172.21.15.157:0/160520668 >> 172.21.15.157:0/160520668 conn(0x7f422c06d560 msgr2=0x7f422c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 -- 172.21.15.157:0/160520668 shutdown_connections 2023-11-06T03:04:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.884+0000 7f4232b33640 1 -- 172.21.15.157:0/160520668 wait complete. 2023-11-06T03:04:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 Processor -- start 2023-11-06T03:04:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- start start 2023-11-06T03:04:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f422c15ac80 0x7f422c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c150ee0 con 0x7f422c0cd730 2023-11-06T03:04:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c14f8f0 con 0x7f422c15ac80 2023-11-06T03:04:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c150480 con 0x7f422c0b2b10 2023-11-06T03:04:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44456/0 (socket says 172.21.15.157:44456) 2023-11-06T03:04:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 -- 172.21.15.157:0/1185177582 learned_addr learned my addr 172.21.15.157:0/1185177582 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42308cf640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42310d0640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f422c15ac80 0x7f422c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 -- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 msgr2=0x7f422c14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c14f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 -- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f422c15ac80 msgr2=0x7f422c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f422c15ac80 0x7f422c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:57.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 -- 172.21.15.157:0/1185177582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0094d0 con 0x7f422c0cd730 2023-11-06T03:04:57.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f422bfff640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c01cde0 tx=0x7f421c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0a9400 con 0x7f422c0cd730 2023-11-06T03:04:57.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c0b1800 con 0x7f422c0cd730 2023-11-06T03:04:57.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c15b060 con 0x7f422c0cd730 2023-11-06T03:04:57.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0b1bb0 con 0x7f422c0cd730 2023-11-06T03:04:57.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c15b620 con 0x7f422c0cd730 2023-11-06T03:04:57.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c072750 con 0x7f422c0cd730 2023-11-06T03:04:57.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.888+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f421c0b1d10 con 0x7f422c0cd730 2023-11-06T03:04:57.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.892+0000 7f42297fa640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 0x7f4218067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:57.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.892+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f421c11da50 con 0x7f422c0cd730 2023-11-06T03:04:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.892+0000 7f42308cf640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 0x7f4218067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.892+0000 7f42308cf640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 0x7f4218067f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b2340 tx=0x7f4220009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:57.896+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f421c0eb380 con 0x7f422c0cd730 2023-11-06T03:04:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.036+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f422c06e3e0 con 0x7f422c0cd730 2023-11-06T03:04:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.036+0000 7f42297fa640 1 -- 172.21.15.157:0/1185177582 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f421c0a6020 con 0x7f422c0cd730 2023-11-06T03:04:58.038 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:58.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 msgr2=0x7f4218067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 0x7f4218067f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b2340 tx=0x7f4220009040 comp rx=0 tx=0).stop 2023-11-06T03:04:58.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 msgr2=0x7f422c15daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f421c01cde0 tx=0x7f421c0192e0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 shutdown_connections 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4218065a90 0x7f4218067f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f422c15ac80 0x7f422c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f422c0cd730 0x7f422c15daa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 --2- 172.21.15.157:0/1185177582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f422c0b2b10 0x7f422c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 >> 172.21.15.157:0/1185177582 conn(0x7f422c06d560 msgr2=0x7f422c159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 shutdown_connections 2023-11-06T03:04:58.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.040+0000 7f4232b33640 1 -- 172.21.15.157:0/1185177582 wait complete. 2023-11-06T03:04:58.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:58.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 Processor -- start 2023-11-06T03:04:58.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 -- start start 2023-11-06T03:04:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1e00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00d2920 con 0x7fa1e015caa0 2023-11-06T03:04:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00d18a0 con 0x7fa1e00d00b0 2023-11-06T03:04:58.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1e537a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00d0520 con 0x7fa1e015ce80 2023-11-06T03:04:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa1e015caa0 0x7fa1e00cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50772/0 (socket says 172.21.15.157:50772) 2023-11-06T03:04:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 -- 172.21.15.157:0/2336793439 learned_addr learned my addr 172.21.15.157:0/2336793439 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1de7fc640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1e00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710854382 0 0) 0x7fa1e00d2920 con 0x7fa1e015caa0 2023-11-06T03:04:58.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c8003530 con 0x7fa1e015caa0 2023-11-06T03:04:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746339405 0 0) 0x7fa1c8003530 con 0x7fa1e015caa0 2023-11-06T03:04:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 msgr2=0x7fa1e00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1e00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 >> v1:172.21.15.195:6789/0 conn(0x7fa1e00d00b0 legacy=0x7fa1e00b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00d1b60 con 0x7fa1e015caa0 2023-11-06T03:04:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa1d0001680 con 0x7fa1e015caa0 2023-11-06T03:04:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 >> v1:172.21.15.157:6789/0 conn(0x7fa1e015caa0 legacy=0x7fa1e00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c80039b0 0x7fa1c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1c80047d0 0x7fa1c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00d0520 con 0x7fa1c80039b0 2023-11-06T03:04:58.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800ba70 con 0x7fa1c80047d0 2023-11-06T03:04:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800bc30 con 0x7fa1e015ce80 2023-11-06T03:04:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c80039b0 0x7fa1c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1de7fc640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1c80047d0 0x7fa1c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1df7fe640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 -- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 msgr2=0x7fa1c8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:58.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1c8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 -- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1c80047d0 msgr2=0x7fa1c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:58.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1c80047d0 0x7fa1c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1deffd640 1 -- 172.21.15.157:0/2336793439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8003530 con 0x7fa1c80039b0 2023-11-06T03:04:58.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.164+0000 7fa1df7fe640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1e015ce80 0x7fa1c8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:58.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1deffd640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c80039b0 0x7fa1c8003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c210 tx=0x7fa1d001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c96fad0c62949d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0005ae0 con 0x7fa1c80039b0 2023-11-06T03:04:58.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0005c60 con 0x7fa1c80039b0 2023-11-06T03:04:58.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1ddffb640 1 -- 172.21.15.157:0/2336793439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00a57c0 con 0x7fa1c80039b0 2023-11-06T03:04:58.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- 172.21.15.157:0/2336793439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c80039b0 msgr2=0x7fa1c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 --2- 172.21.15.157:0/2336793439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c80039b0 0x7fa1c8003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c210 tx=0x7fa1d001cbd0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1df7fe640 1 -- 172.21.15.157:0/2336793439 reap_dead start 2023-11-06T03:04:58.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- 172.21.15.157:0/2336793439 shutdown_connections 2023-11-06T03:04:58.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- 172.21.15.157:0/2336793439 >> 172.21.15.157:0/2336793439 conn(0x7fa1e0070790 msgr2=0x7fa1e0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- 172.21.15.157:0/2336793439 shutdown_connections 2023-11-06T03:04:58.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- 172.21.15.157:0/2336793439 wait complete. 2023-11-06T03:04:58.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 Processor -- start 2023-11-06T03:04:58.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.168+0000 7fa1e537a640 1 -- start start 2023-11-06T03:04:58.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1c80039b0 0x7fa1e00c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e00cad00 0x7fa1e00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c9cf0 con 0x7fa1e0157080 2023-11-06T03:04:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00cb180 con 0x7fa1e00cad00 2023-11-06T03:04:58.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00c9f40 con 0x7fa1c80039b0 2023-11-06T03:04:58.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44482/0 (socket says 172.21.15.157:44482) 2023-11-06T03:04:58.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 -- 172.21.15.157:0/620943984 learned_addr learned my addr 172.21.15.157:0/620943984 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1deffd640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1c80039b0 0x7fa1e00c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1de7fc640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e00cad00 0x7fa1e00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 -- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1c80039b0 msgr2=0x7fa1e00c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1c80039b0 0x7fa1e00c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 -- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e00cad00 msgr2=0x7fa1e00c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e00cad00 0x7fa1e00c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 -- 172.21.15.157:0/620943984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00ca220 con 0x7fa1e0157080 2023-11-06T03:04:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1df7fe640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00b5610 tx=0x7fa1d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8006d60 con 0x7fa1e0157080 2023-11-06T03:04:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8003980 con 0x7fa1e0157080 2023-11-06T03:04:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e00ca500 con 0x7fa1e0157080 2023-11-06T03:04:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003d30 con 0x7fa1e0157080 2023-11-06T03:04:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e00c8680 con 0x7fa1e0157080 2023-11-06T03:04:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0072750 con 0x7fa1e0157080 2023-11-06T03:04:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa1d8005070 con 0x7fa1e0157080 2023-11-06T03:04:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 0x7fa1cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa1d80830f0 con 0x7fa1e0157080 2023-11-06T03:04:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.172+0000 7fa1deffd640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 0x7fa1cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.176+0000 7fa1deffd640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 0x7fa1cc067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00ab000 tx=0x7fa1d00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.176+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa1d80509a0 con 0x7fa1e0157080 2023-11-06T03:04:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.332+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1e00c9ab0 con 0x7fa1e0157080 2023-11-06T03:04:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.332+0000 7fa1c77fe640 1 -- 172.21.15.157:0/620943984 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1d8053fc0 con 0x7fa1e0157080 2023-11-06T03:04:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:04:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 msgr2=0x7fa1cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 0x7fa1cc067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00ab000 tx=0x7fa1d00a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 msgr2=0x7fa1e00b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00b5610 tx=0x7fa1d800dc00 comp rx=0 tx=0).stop 2023-11-06T03:04:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 shutdown_connections 2023-11-06T03:04:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa1cc065a60 0x7fa1cc067f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1e0157080 0x7fa1e00b7430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1e00cad00 0x7fa1e00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 --2- 172.21.15.157:0/620943984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1c80039b0 0x7fa1e00c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 >> 172.21.15.157:0/620943984 conn(0x7fa1e0070790 msgr2=0x7fa1e015b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 shutdown_connections 2023-11-06T03:04:58.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.336+0000 7fa1e537a640 1 -- 172.21.15.157:0/620943984 wait complete. 2023-11-06T03:04:58.350 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:04:58.350 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:04:58.357 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:04:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 Processor -- start 2023-11-06T03:04:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 -- start start 2023-11-06T03:04:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a741551e0 0x7f8a7406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a740b3b20 0x7f8a7415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7415f580 con 0x7f8a7415e5e0 2023-11-06T03:04:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7415efc0 con 0x7f8a741551e0 2023-11-06T03:04:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a7b477640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7415f2a0 con 0x7f8a740b3b20 2023-11-06T03:04:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79213640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a741551e0 0x7f8a7406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a78a12640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8a7415e5e0 0x7f8a740b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:50788/0 (socket says 172.21.15.157:50788) 2023-11-06T03:04:58.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a78a12640 1 -- 172.21.15.157:0/1666672054 learned_addr learned my addr 172.21.15.157:0/1666672054 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79a14640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a740b3b20 0x7f8a7415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a6bfff640 1 -- 172.21.15.157:0/1666672054 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642625094 0 0) 0x7f8a7415f580 con 0x7f8a7415e5e0 2023-11-06T03:04:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a6bfff640 1 -- 172.21.15.157:0/1666672054 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a60003530 con 0x7f8a7415e5e0 2023-11-06T03:04:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79213640 1 -- 172.21.15.157:0/1666672054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a740b3b20 msgr2=0x7f8a7415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:58.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79213640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a740b3b20 0x7f8a7415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79213640 1 -- 172.21.15.157:0/1666672054 >> v1:172.21.15.157:6789/0 conn(0x7f8a7415e5e0 legacy=0x7f8a740b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.464+0000 7f8a79213640 1 -- 172.21.15.157:0/1666672054 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7406f5a0 con 0x7f8a741551e0 2023-11-06T03:04:58.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a79213640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a741551e0 0x7f8a7406fda0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a700097c0 tx=0x7f8a7001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28231b9c9147bd35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a6bfff640 1 -- 172.21.15.157:0/1666672054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700ac640 con 0x7f8a741551e0 2023-11-06T03:04:58.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a6bfff640 1 -- 172.21.15.157:0/1666672054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a700ac7a0 con 0x7f8a741551e0 2023-11-06T03:04:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a6bfff640 1 -- 172.21.15.157:0/1666672054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700acb30 con 0x7f8a741551e0 2023-11-06T03:04:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 -- 172.21.15.157:0/1666672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a741551e0 msgr2=0x7f8a7406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a741551e0 0x7f8a7406fda0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a700097c0 tx=0x7f8a7001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 -- 172.21.15.157:0/1666672054 shutdown_connections 2023-11-06T03:04:58.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a740b3b20 0x7f8a7415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 --2- 172.21.15.157:0/1666672054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a741551e0 0x7f8a7406fda0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 -- 172.21.15.157:0/1666672054 >> 172.21.15.157:0/1666672054 conn(0x7f8a74065930 msgr2=0x7f8a74065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 -- 172.21.15.157:0/1666672054 shutdown_connections 2023-11-06T03:04:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.468+0000 7f8a7b477640 1 -- 172.21.15.157:0/1666672054 wait complete. 2023-11-06T03:04:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 Processor -- start 2023-11-06T03:04:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- start start 2023-11-06T03:04:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7415e5e0 0x7f8a74153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7414c9d0 0x7f8a74158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74153fb0 con 0x7f8a740b3b20 2023-11-06T03:04:58.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74154a60 con 0x7f8a7414c9d0 2023-11-06T03:04:58.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a741541c0 con 0x7f8a7415e5e0 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44514/0 (socket says 172.21.15.157:44514) 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 -- 172.21.15.157:0/2333757640 learned_addr learned my addr 172.21.15.157:0/2333757640 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a78a12640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7415e5e0 0x7f8a74153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79a14640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7414c9d0 0x7f8a74158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 -- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7415e5e0 msgr2=0x7f8a74153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7415e5e0 0x7f8a74153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 -- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7414c9d0 msgr2=0x7f8a74158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7414c9d0 0x7f8a74158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 -- 172.21.15.157:0/2333757640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70009420 con 0x7f8a740b3b20 2023-11-06T03:04:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a79213640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8a70018fd0 tx=0x7f8a7001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700b8650 con 0x7f8a740b3b20 2023-11-06T03:04:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a700b87b0 con 0x7f8a740b3b20 2023-11-06T03:04:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a741544a0 con 0x7f8a740b3b20 2023-11-06T03:04:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700b8b60 con 0x7f8a740b3b20 2023-11-06T03:04:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a741556d0 con 0x7f8a740b3b20 2023-11-06T03:04:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7415bb10 con 0x7f8a740b3b20 2023-11-06T03:04:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8a700b8cc0 con 0x7f8a740b3b20 2023-11-06T03:04:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 0x7f8a58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8a7011dd40 con 0x7f8a740b3b20 2023-11-06T03:04:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.472+0000 7f8a78a12640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 0x7f8a58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.476+0000 7f8a78a12640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 0x7f8a58067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a741f7300 tx=0x7f8a64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.476+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8a700eb5f0 con 0x7f8a740b3b20 2023-11-06T03:04:58.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.620+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a740b2a70 con 0x7f8a740b3b20 2023-11-06T03:04:58.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.624+0000 7f8a69ffb640 1 -- 172.21.15.157:0/2333757640 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f8a700bc090 con 0x7f8a740b3b20 2023-11-06T03:04:58.624 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:04:58.625 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:04:58.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 msgr2=0x7f8a58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 0x7f8a58067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8a741f7300 tx=0x7f8a64009040 comp rx=0 tx=0).stop 2023-11-06T03:04:58.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 msgr2=0x7f8a7415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8a70018fd0 tx=0x7f8a7001ac00 comp rx=0 tx=0).stop 2023-11-06T03:04:58.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 shutdown_connections 2023-11-06T03:04:58.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8a58065a60 0x7f8a58067f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a7414c9d0 0x7f8a74158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8a7415e5e0 0x7f8a74153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 --2- 172.21.15.157:0/2333757640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a740b3b20 0x7f8a7415a7f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 >> 172.21.15.157:0/2333757640 conn(0x7f8a74065930 msgr2=0x7f8a740b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 shutdown_connections 2023-11-06T03:04:58.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.628+0000 7f8a7b477640 1 -- 172.21.15.157:0/2333757640 wait complete. 2023-11-06T03:04:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:04:58.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:04:58.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 Processor -- start 2023-11-06T03:04:58.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 -- start start 2023-11-06T03:04:58.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d81599c0 0x7ff9d80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9d806d000 0x7ff9d80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d815e440 con 0x7ff9d815dea0 2023-11-06T03:04:58.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d815adb0 con 0x7ff9d806d000 2023-11-06T03:04:58.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9df87f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d806c240 con 0x7ff9d81599c0 2023-11-06T03:04:58.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dce1a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9d806d000 0x7ff9d80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dde1c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff9d815dea0 0x7ff9d8070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49788/0 (socket says 172.21.15.157:49788) 2023-11-06T03:04:58.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dde1c640 1 -- 172.21.15.157:0/477822970 learned_addr learned my addr 172.21.15.157:0/477822970 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:58.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dd61b640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d81599c0 0x7ff9d80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dce1a640 1 -- 172.21.15.157:0/477822970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d81599c0 msgr2=0x7ff9d80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:58.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dce1a640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d81599c0 0x7ff9d80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dce1a640 1 -- 172.21.15.157:0/477822970 >> v1:172.21.15.157:6789/0 conn(0x7ff9d815dea0 legacy=0x7ff9d8070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.764+0000 7ff9dce1a640 1 -- 172.21.15.157:0/477822970 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d806c520 con 0x7ff9d806d000 2023-11-06T03:04:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dce1a640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d806d000 0x7ff9d80b23e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80097c0 tx=0x7ff9c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89310cac41ac05fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cffff640 1 -- 172.21.15.157:0/477822970 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80ad660 con 0x7ff9d806d000 2023-11-06T03:04:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cffff640 1 -- 172.21.15.157:0/477822970 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c80ad7c0 con 0x7ff9d806d000 2023-11-06T03:04:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cffff640 1 -- 172.21.15.157:0/477822970 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80adb50 con 0x7ff9d806d000 2023-11-06T03:04:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/477822970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d806d000 msgr2=0x7ff9d80b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d806d000 0x7ff9d80b23e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80097c0 tx=0x7ff9c801c450 comp rx=0 tx=0).stop 2023-11-06T03:04:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/477822970 shutdown_connections 2023-11-06T03:04:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d806d000 0x7ff9d80b23e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- 172.21.15.157:0/477822970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d81599c0 0x7ff9d80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/477822970 >> 172.21.15.157:0/477822970 conn(0x7ff9d8065930 msgr2=0x7ff9d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/477822970 shutdown_connections 2023-11-06T03:04:58.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/477822970 wait complete. 2023-11-06T03:04:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 Processor -- start 2023-11-06T03:04:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- start start 2023-11-06T03:04:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 0x7ff9d8155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81599c0 0x7ff9d81f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cb9d0 con 0x7ff9d815dea0 2023-11-06T03:04:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cc510 con 0x7ff9d81599c0 2023-11-06T03:04:58.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80cbbe0 con 0x7ff9d806d000 2023-11-06T03:04:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47178/0 (socket says 172.21.15.157:47178) 2023-11-06T03:04:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 -- 172.21.15.157:0/3624581003 learned_addr learned my addr 172.21.15.157:0/3624581003 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dd61b640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 0x7ff9d8155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dce1a640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81599c0 0x7ff9d81f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 -- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 msgr2=0x7ff9d8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 0x7ff9d8155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 -- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81599c0 msgr2=0x7ff9d81f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81599c0 0x7ff9d81f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 -- 172.21.15.157:0/3624581003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009420 con 0x7ff9d815dea0 2023-11-06T03:04:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dd61b640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 0x7ff9d8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9dde1c640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000ca80 tx=0x7ff9d000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000ac30 con 0x7ff9d815dea0 2023-11-06T03:04:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d000ad90 con 0x7ff9d815dea0 2023-11-06T03:04:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d80cbec0 con 0x7ff9d815dea0 2023-11-06T03:04:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0012600 con 0x7ff9d815dea0 2023-11-06T03:04:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.768+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d8156140 con 0x7ff9d815dea0 2023-11-06T03:04:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d80cc680 con 0x7ff9d815dea0 2023-11-06T03:04:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9d0007070 con 0x7ff9d815dea0 2023-11-06T03:04:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9cdffb640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 0x7ff9bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff9d0083130 con 0x7ff9d815dea0 2023-11-06T03:04:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9dd61b640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 0x7ff9bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.772+0000 7ff9dd61b640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 0x7ff9bc067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009840 tx=0x7ff9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:58.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.776+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9d0050a00 con 0x7ff9d815dea0 2023-11-06T03:04:58.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.916+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9d8072750 con 0x7ff9d815dea0 2023-11-06T03:04:58.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9cdffb640 1 -- 172.21.15.157:0/3624581003 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9d0054020 con 0x7ff9d815dea0 2023-11-06T03:04:58.919 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:04:58.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 msgr2=0x7ff9bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 0x7ff9bc067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009840 tx=0x7ff9d4008040 comp rx=0 tx=0).stop 2023-11-06T03:04:58.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 msgr2=0x7ff9d81f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:58.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000ca80 tx=0x7ff9d000cf40 comp rx=0 tx=0).stop 2023-11-06T03:04:58.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 shutdown_connections 2023-11-06T03:04:58.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9bc065a60 0x7ff9bc067f20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9d815dea0 0x7ff9d81f6fb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9d81599c0 0x7ff9d81f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 --2- 172.21.15.157:0/3624581003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9d806d000 0x7ff9d8155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:58.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 >> 172.21.15.157:0/3624581003 conn(0x7ff9d8065930 msgr2=0x7ff9d80b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:58.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 shutdown_connections 2023-11-06T03:04:58.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:58.920+0000 7ff9df87f640 1 -- 172.21.15.157:0/3624581003 wait complete. 2023-11-06T03:04:58.936 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:04:59.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.064+0000 7f8d0d3e6640 1 Processor -- start 2023-11-06T03:04:59.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.064+0000 7f8d0d3e6640 1 -- start start 2023-11-06T03:04:59.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d081551e0 0x7f8d0806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 0x7f8d0815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0815f580 con 0x7f8d0815e5e0 2023-11-06T03:04:59.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0815efc0 con 0x7f8d081551e0 2023-11-06T03:04:59.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0815f2a0 con 0x7f8d080b3b20 2023-11-06T03:04:59.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d081551e0 0x7f8d0806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d067fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8d0815e5e0 0x7f8d080b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49790/0 (socket says 172.21.15.157:49790) 2023-11-06T03:04:59.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d067fc640 1 -- 172.21.15.157:0/3970295049 learned_addr learned my addr 172.21.15.157:0/3970295049 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d081551e0 0x7f8d0806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47186/0 (socket says 172.21.15.157:47186) 2023-11-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d077fe640 1 --2- 172.21.15.157:0/3970295049 >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 0x7f8d0815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 -- 172.21.15.157:0/3970295049 >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 msgr2=0x7f8d0815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 --2- 172.21.15.157:0/3970295049 >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 0x7f8d0815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 -- 172.21.15.157:0/3970295049 >> v1:172.21.15.157:6789/0 conn(0x7f8d0815e5e0 legacy=0x7f8d080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 -- 172.21.15.157:0/3970295049 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0806f5a0 con 0x7f8d081551e0 2023-11-06T03:04:59.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d05ffb640 1 -- 172.21.15.157:0/3970295049 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482512840 0 0) 0x7f8d0815f580 con 0x7f8d0815e5e0 2023-11-06T03:04:59.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d077fe640 1 --2- 172.21.15.157:0/3970295049 >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 0x7f8d0815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:59.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d06ffd640 1 --2- 172.21.15.157:0/3970295049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d081551e0 0x7f8d0806fda0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d080cbfe0 tx=0x7f8cf801c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7f095268814ca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d05ffb640 1 -- 172.21.15.157:0/3970295049 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a58a0 con 0x7f8d081551e0 2023-11-06T03:04:59.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d05ffb640 1 -- 172.21.15.157:0/3970295049 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf80a5a00 con 0x7f8d081551e0 2023-11-06T03:04:59.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d05ffb640 1 -- 172.21.15.157:0/3970295049 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a5d90 con 0x7f8d081551e0 2023-11-06T03:04:59.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3970295049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d081551e0 msgr2=0x7f8d0806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3970295049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d081551e0 0x7f8d0806fda0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d080cbfe0 tx=0x7f8cf801c7c0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3970295049 shutdown_connections 2023-11-06T03:04:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3970295049 >> v2:172.21.15.195:3300/0 conn(0x7f8d080b3b20 0x7f8d0815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3970295049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d081551e0 0x7f8d0806fda0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.068+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3970295049 >> 172.21.15.157:0/3970295049 conn(0x7f8d08065930 msgr2=0x7f8d08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3970295049 shutdown_connections 2023-11-06T03:04:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3970295049 wait complete. 2023-11-06T03:04:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 Processor -- start 2023-11-06T03:04:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- start start 2023-11-06T03:04:59.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080b3b20 0x7f8d0806d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d081551e0 0x7f8d0814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d080ae070 con 0x7f8d0815e5e0 2023-11-06T03:04:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0806c0a0 con 0x7f8d080b3b20 2023-11-06T03:04:59.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0806c380 con 0x7f8d081551e0 2023-11-06T03:04:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47192/0 (socket says 172.21.15.157:47192) 2023-11-06T03:04:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 -- 172.21.15.157:0/3097045667 learned_addr learned my addr 172.21.15.157:0/3097045667 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d067fc640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d081551e0 0x7f8d0814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d06ffd640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080b3b20 0x7f8d0806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 -- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d081551e0 msgr2=0x7f8d0814d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d081551e0 0x7f8d0814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 -- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080b3b20 msgr2=0x7f8d0806d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080b3b20 0x7f8d0806d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 -- 172.21.15.157:0/3097045667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf8009a70 con 0x7f8d0815e5e0 2023-11-06T03:04:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d077fe640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000c7f0 tx=0x7f8d0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00007820 con 0x7f8d0815e5e0 2023-11-06T03:04:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d000079c0 con 0x7f8d0815e5e0 2023-11-06T03:04:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0806c660 con 0x7f8d0815e5e0 2023-11-06T03:04:59.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00007d70 con 0x7f8d0815e5e0 2023-11-06T03:04:59.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0806cc20 con 0x7f8d0815e5e0 2023-11-06T03:04:59.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.072+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0806ce40 con 0x7f8d0815e5e0 2023-11-06T03:04:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.076+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d000113f0 con 0x7f8d0815e5e0 2023-11-06T03:04:59.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.076+0000 7f8cff7fe640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 0x7f8ce8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.076+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8d00083440 con 0x7f8d0815e5e0 2023-11-06T03:04:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.076+0000 7f8d06ffd640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 0x7f8ce8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.076+0000 7f8d06ffd640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 0x7f8ce8067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80af000 tx=0x7f8cf80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.080+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d00050cf0 con 0x7f8d0815e5e0 2023-11-06T03:04:59.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.220+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d08072750 con 0x7f8d0815e5e0 2023-11-06T03:04:59.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8cff7fe640 1 -- 172.21.15.157:0/3097045667 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d00054310 con 0x7f8d0815e5e0 2023-11-06T03:04:59.223 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:04:59.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 msgr2=0x7f8ce8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 0x7f8ce8067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80af000 tx=0x7f8cf80a7070 comp rx=0 tx=0).stop 2023-11-06T03:04:59.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 msgr2=0x7f8d08150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000c7f0 tx=0x7f8d0000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 shutdown_connections 2023-11-06T03:04:59.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ce8065a60 0x7f8ce8067f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d0815e5e0 0x7f8d08150470 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d081551e0 0x7f8d0814d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 --2- 172.21.15.157:0/3097045667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080b3b20 0x7f8d0806d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 >> 172.21.15.157:0/3097045667 conn(0x7f8d08065930 msgr2=0x7f8d080af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 shutdown_connections 2023-11-06T03:04:59.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.224+0000 7f8d0d3e6640 1 -- 172.21.15.157:0/3097045667 wait complete. 2023-11-06T03:04:59.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:04:59.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 Processor -- start 2023-11-06T03:04:59.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 -- start start 2023-11-06T03:04:59.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba081551e0 0x7fba0806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fba0815e5e0 0x7fba080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0815f580 con 0x7fba0815ea90 2023-11-06T03:04:59.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0815efc0 con 0x7fba081551e0 2023-11-06T03:04:59.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba0e3e9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0815f2a0 con 0x7fba0815e5e0 2023-11-06T03:04:59.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba081551e0 0x7fba0806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba081551e0 0x7fba0806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47194/0 (socket says 172.21.15.157:47194) 2023-11-06T03:04:59.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 -- 172.21.15.157:0/140722301 learned_addr learned my addr 172.21.15.157:0/140722301 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba077fe640 1 --2- 172.21.15.157:0/140722301 >> v2:172.21.15.195:3300/0 conn(0x7fba0815e5e0 0x7fba080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 -- 172.21.15.157:0/140722301 >> v2:172.21.15.195:3300/0 conn(0x7fba0815e5e0 msgr2=0x7fba080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:59.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 --2- 172.21.15.157:0/140722301 >> v2:172.21.15.195:3300/0 conn(0x7fba0815e5e0 0x7fba080b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 -- 172.21.15.157:0/140722301 >> v1:172.21.15.195:6789/0 conn(0x7fba0815ea90 legacy=0x7fba0815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.364+0000 7fba07fff640 1 -- 172.21.15.157:0/140722301 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0806f5a0 con 0x7fba081551e0 2023-11-06T03:04:59.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba07fff640 1 --2- 172.21.15.157:0/140722301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba081551e0 0x7fba0806fda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fba08066880 tx=0x7fb9f801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6682bca98856f6e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba06ffd640 1 -- 172.21.15.157:0/140722301 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004230 con 0x7fba081551e0 2023-11-06T03:04:59.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba06ffd640 1 -- 172.21.15.157:0/140722301 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80a7050 con 0x7fba081551e0 2023-11-06T03:04:59.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba06ffd640 1 -- 172.21.15.157:0/140722301 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8003620 con 0x7fba081551e0 2023-11-06T03:04:59.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- 172.21.15.157:0/140722301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba081551e0 msgr2=0x7fba0806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/140722301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba081551e0 0x7fba0806fda0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fba08066880 tx=0x7fb9f801c440 comp rx=0 tx=0).stop 2023-11-06T03:04:59.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- 172.21.15.157:0/140722301 shutdown_connections 2023-11-06T03:04:59.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/140722301 >> v2:172.21.15.195:3300/0 conn(0x7fba0815e5e0 0x7fba080b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/140722301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba081551e0 0x7fba0806fda0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- 172.21.15.157:0/140722301 >> 172.21.15.157:0/140722301 conn(0x7fba08065930 msgr2=0x7fba08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- 172.21.15.157:0/140722301 shutdown_connections 2023-11-06T03:04:59.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- 172.21.15.157:0/140722301 wait complete. 2023-11-06T03:04:59.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 Processor -- start 2023-11-06T03:04:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- start start 2023-11-06T03:04:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 0x7fba080aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba0815ea90 0x7fba08151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080af1d0 con 0x7fba0815e5e0 2023-11-06T03:04:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814f750 con 0x7fba081551e0 2023-11-06T03:04:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba0e3e9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0815a210 con 0x7fba0815ea90 2023-11-06T03:04:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47210/0 (socket says 172.21.15.157:47210) 2023-11-06T03:04:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.368+0000 7fba077fe640 1 -- 172.21.15.157:0/3788724352 learned_addr learned my addr 172.21.15.157:0/3788724352 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba07fff640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 0x7fba080aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba0c986640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba0815ea90 0x7fba08151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 -- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba0815ea90 msgr2=0x7fba08151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba0815ea90 0x7fba08151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 -- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 msgr2=0x7fba080aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 0x7fba080aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 -- 172.21.15.157:0/3788724352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80094d0 con 0x7fba0815e5e0 2023-11-06T03:04:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba07fff640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 0x7fba080aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba077fe640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00d800 tx=0x7fb9fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00b6c0 con 0x7fba0815e5e0 2023-11-06T03:04:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00b820 con 0x7fba0815e5e0 2023-11-06T03:04:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0815a4f0 con 0x7fba0815e5e0 2023-11-06T03:04:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00bb90 con 0x7fba0815e5e0 2023-11-06T03:04:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0815a7d0 con 0x7fba0815e5e0 2023-11-06T03:04:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba08072750 con 0x7fba0815e5e0 2023-11-06T03:04:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb9fc00bcf0 con 0x7fba0815e5e0 2023-11-06T03:04:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 0x7fb9f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb9fc0838d0 con 0x7fba0815e5e0 2023-11-06T03:04:59.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba07fff640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 0x7fb9f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.372+0000 7fba07fff640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 0x7fb9f4067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80ad000 tx=0x7fb9f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.376+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb9fc0510f0 con 0x7fba0815e5e0 2023-11-06T03:04:59.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.528+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba08071730 con 0x7fba0815e5e0 2023-11-06T03:04:59.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.528+0000 7fba04ff9640 1 -- 172.21.15.157:0/3788724352 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9fc054710 con 0x7fba0815e5e0 2023-11-06T03:04:59.530 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:04:59.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 msgr2=0x7fb9f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 0x7fb9f4067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80ad000 tx=0x7fb9f8008040 comp rx=0 tx=0).stop 2023-11-06T03:04:59.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 msgr2=0x7fba0814fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00d800 tx=0x7fb9fc00dcc0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 shutdown_connections 2023-11-06T03:04:59.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9f4065a60 0x7fb9f4067f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba0815ea90 0x7fba08151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba0815e5e0 0x7fba0814fba0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 --2- 172.21.15.157:0/3788724352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba081551e0 0x7fba080aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 >> 172.21.15.157:0/3788724352 conn(0x7fba08065930 msgr2=0x7fba08154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 shutdown_connections 2023-11-06T03:04:59.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.532+0000 7fba0e3e9640 1 -- 172.21.15.157:0/3788724352 wait complete. 2023-11-06T03:04:59.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:04:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 Processor -- start 2023-11-06T03:04:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 -- start start 2023-11-06T03:04:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815e5e0 0x7f0d380b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3815ea60 0x7f0d380b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3815f580 con 0x7f0d381551e0 2023-11-06T03:04:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380704b0 con 0x7f0d3815ea60 2023-11-06T03:04:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d3cf06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3815efc0 con 0x7f0d3815e5e0 2023-11-06T03:04:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d36d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3815ea60 0x7f0d380b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d36d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d3815ea60 0x7f0d380b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47222/0 (socket says 172.21.15.157:47222) 2023-11-06T03:04:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d36d9d640 1 -- 172.21.15.157:0/2304948190 learned_addr learned my addr 172.21.15.157:0/2304948190 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.676+0000 7f0d35d9b640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815e5e0 0x7f0d380b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d36d9d640 1 -- 172.21.15.157:0/2304948190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815e5e0 msgr2=0x7f0d380b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d36d9d640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815e5e0 0x7f0d380b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d36d9d640 1 -- 172.21.15.157:0/2304948190 >> v1:172.21.15.195:6789/0 conn(0x7f0d381551e0 legacy=0x7f0d3806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d36d9d640 1 -- 172.21.15.157:0/2304948190 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3815f2a0 con 0x7f0d3815ea60 2023-11-06T03:04:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d36d9d640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3815ea60 0x7f0d380b3950 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009e50 tx=0x7f0d3001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb31f4d0aced4b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3559a640 1 -- 172.21.15.157:0/2304948190 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300a5810 con 0x7f0d3815ea60 2023-11-06T03:04:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3559a640 1 -- 172.21.15.157:0/2304948190 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d300a5970 con 0x7f0d3815ea60 2023-11-06T03:04:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3559a640 1 -- 172.21.15.157:0/2304948190 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300a5d00 con 0x7f0d3815ea60 2023-11-06T03:04:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2304948190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3815ea60 msgr2=0x7f0d380b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3815ea60 0x7f0d380b3950 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009e50 tx=0x7f0d3001c950 comp rx=0 tx=0).stop 2023-11-06T03:04:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2304948190 shutdown_connections 2023-11-06T03:04:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3815ea60 0x7f0d380b3950 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2304948190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815e5e0 0x7f0d380b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2304948190 >> 172.21.15.157:0/2304948190 conn(0x7f0d38065930 msgr2=0x7f0d38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2304948190 shutdown_connections 2023-11-06T03:04:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.680+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2304948190 wait complete. 2023-11-06T03:04:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 Processor -- start 2023-11-06T03:04:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- start start 2023-11-06T03:04:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 0x7f0d3814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815ea60 0x7f0d380af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d381543e0 con 0x7f0d381551e0 2023-11-06T03:04:59.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38156c00 con 0x7f0d3815e5e0 2023-11-06T03:04:59.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38153760 con 0x7f0d3815ea60 2023-11-06T03:04:59.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47226/0 (socket says 172.21.15.157:47226) 2023-11-06T03:04:59.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 -- 172.21.15.157:0/2656023583 learned_addr learned my addr 172.21.15.157:0/2656023583 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d35d9b640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 0x7f0d3814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d36d9d640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815ea60 0x7f0d380af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 -- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815ea60 msgr2=0x7f0d380af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815ea60 0x7f0d380af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 -- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 msgr2=0x7f0d3814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 0x7f0d3814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 -- 172.21.15.157:0/2656023583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30009b30 con 0x7f0d381551e0 2023-11-06T03:04:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d35d9b640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 0x7f0d3814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:04:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3659c640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c500 tx=0x7f0d2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c005220 con 0x7f0d381551e0 2023-11-06T03:04:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c005380 con 0x7f0d381551e0 2023-11-06T03:04:59.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d38153a40 con 0x7f0d381551e0 2023-11-06T03:04:59.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c007270 con 0x7f0d381551e0 2023-11-06T03:04:59.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d38154000 con 0x7f0d381551e0 2023-11-06T03:04:59.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d38072750 con 0x7f0d381551e0 2023-11-06T03:04:59.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.684+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0d2c0073d0 con 0x7f0d381551e0 2023-11-06T03:04:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.688+0000 7f0d26ffd640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 0x7f0d18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.688+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0d2c082c50 con 0x7f0d381551e0 2023-11-06T03:04:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.688+0000 7f0d35d9b640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 0x7f0d18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.688+0000 7f0d35d9b640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 0x7f0d18067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d380b02b0 tx=0x7f0d28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.692+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d2c088020 con 0x7f0d381551e0 2023-11-06T03:04:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d38071730 con 0x7f0d381551e0 2023-11-06T03:04:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d26ffd640 1 -- 172.21.15.157:0/2656023583 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d2c050480 con 0x7f0d381551e0 2023-11-06T03:04:59.847 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:04:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 msgr2=0x7f0d18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 0x7f0d18067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d380b02b0 tx=0x7f0d28008040 comp rx=0 tx=0).stop 2023-11-06T03:04:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 msgr2=0x7f0d381555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00c500 tx=0x7f0d2c00c9c0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 shutdown_connections 2023-11-06T03:04:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d18065a60 0x7f0d18067f20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3815ea60 0x7f0d380af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3815e5e0 0x7f0d3814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 --2- 172.21.15.157:0/2656023583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d381551e0 0x7f0d381555c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 >> 172.21.15.157:0/2656023583 conn(0x7f0d38065930 msgr2=0x7f0d3815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 shutdown_connections 2023-11-06T03:04:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.848+0000 7f0d3cf06640 1 -- 172.21.15.157:0/2656023583 wait complete. 2023-11-06T03:04:59.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:04:59.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 Processor -- start 2023-11-06T03:04:59.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 -- start start 2023-11-06T03:04:59.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328070fa0 0x7ff328071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff328159350 0x7ff3281597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:04:59.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff328071cd0 con 0x7ff3280cef80 2023-11-06T03:04:59.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32815cc80 con 0x7ff328159350 2023-11-06T03:04:59.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32e994640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32815ce90 con 0x7ff328070fa0 2023-11-06T03:04:59.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff328159350 0x7ff3281597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff328159350 0x7ff3281597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47232/0 (socket says 172.21.15.157:47232) 2023-11-06T03:04:59.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 -- 172.21.15.157:0/2781244181 learned_addr learned my addr 172.21.15.157:0/2781244181 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:04:59.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff327fff640 1 --1- 172.21.15.157:0/2781244181 >> v1:172.21.15.157:6789/0 conn(0x7ff3280cef80 0x7ff3280cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49802/0 (socket says 172.21.15.157:49802) 2023-11-06T03:04:59.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff3277fe640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328070fa0 0x7ff328071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:04:59.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 -- 172.21.15.157:0/2781244181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328070fa0 msgr2=0x7ff328071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:04:59.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328070fa0 0x7ff328071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff326ffd640 1 -- 172.21.15.157:0/2781244181 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177735851 0 0) 0x7ff328071cd0 con 0x7ff3280cef80 2023-11-06T03:04:59.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 -- 172.21.15.157:0/2781244181 >> v1:172.21.15.157:6789/0 conn(0x7ff3280cef80 legacy=0x7ff3280cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.992+0000 7ff32cf31640 1 -- 172.21.15.157:0/2781244181 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32815d170 con 0x7ff328159350 2023-11-06T03:04:59.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32cf31640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328159350 0x7ff3281597e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff31800bb10 tx=0x7ff31801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb9b4d32b36ee24 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:04:59.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff326ffd640 1 -- 172.21.15.157:0/2781244181 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180a7b90 con 0x7ff328159350 2023-11-06T03:04:59.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff326ffd640 1 -- 172.21.15.157:0/2781244181 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3180a7cf0 con 0x7ff328159350 2023-11-06T03:04:59.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff326ffd640 1 -- 172.21.15.157:0/2781244181 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180af660 con 0x7ff328159350 2023-11-06T03:04:59.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/2781244181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328159350 msgr2=0x7ff3281597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:04:59.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328159350 0x7ff3281597e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff31800bb10 tx=0x7ff31801ebe0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/2781244181 shutdown_connections 2023-11-06T03:04:59.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328159350 0x7ff3281597e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- 172.21.15.157:0/2781244181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff328070fa0 0x7ff328071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/2781244181 >> 172.21.15.157:0/2781244181 conn(0x7ff32806a4d0 msgr2=0x7ff32806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/2781244181 shutdown_connections 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/2781244181 wait complete. 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 Processor -- start 2023-11-06T03:04:59.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- start start 2023-11-06T03:05:00.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3280cef80 0x7ff328152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff328159350 0x7ff3280b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328146980 con 0x7ff328070fa0 2023-11-06T03:05:00.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32815ca70 con 0x7ff328159350 2023-11-06T03:05:00.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32814c4f0 con 0x7ff3280cef80 2023-11-06T03:05:00.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47248/0 (socket says 172.21.15.157:47248) 2023-11-06T03:05:00.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 -- 172.21.15.157:0/474057767 learned_addr learned my addr 172.21.15.157:0/474057767 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:00.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff3277fe640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3280cef80 0x7ff328152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 -- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3280cef80 msgr2=0x7ff328152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3280cef80 0x7ff328152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 -- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff328159350 msgr2=0x7ff3280b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff328159350 0x7ff3280b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 -- 172.21.15.157:0/474057767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31800b770 con 0x7ff328070fa0 2023-11-06T03:05:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff327fff640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c580 tx=0x7ff31c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0074e0 con 0x7ff328070fa0 2023-11-06T03:05:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c007640 con 0x7ff328070fa0 2023-11-06T03:05:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c004a30 con 0x7ff328070fa0 2023-11-06T03:05:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:04:59.996+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32815c4b0 con 0x7ff328070fa0 2023-11-06T03:05:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32814c180 con 0x7ff328070fa0 2023-11-06T03:05:00.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff328072750 con 0x7ff328070fa0 2023-11-06T03:05:00.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff31c004b90 con 0x7ff328070fa0 2023-11-06T03:05:00.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff324ff9640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 0x7ff314067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff31c082a10 con 0x7ff328070fa0 2023-11-06T03:05:00.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff3277fe640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 0x7ff314067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.000+0000 7ff3277fe640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 0x7ff314067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff32814a8d0 tx=0x7ff32000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.004+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff31c050230 con 0x7ff328070fa0 2023-11-06T03:05:00.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff32806d330 con 0x7ff328070fa0 2023-11-06T03:05:00.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff324ff9640 1 -- 172.21.15.157:0/474057767 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff31c018070 con 0x7ff328070fa0 2023-11-06T03:05:00.164 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:05:00.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 msgr2=0x7ff314067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 0x7ff314067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff32814a8d0 tx=0x7ff32000b040 comp rx=0 tx=0).stop 2023-11-06T03:05:00.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 msgr2=0x7ff328146440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00c580 tx=0x7ff31c00ca40 comp rx=0 tx=0).stop 2023-11-06T03:05:00.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 shutdown_connections 2023-11-06T03:05:00.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff314065a90 0x7ff314067f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff328159350 0x7ff3280b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3280cef80 0x7ff328152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 --2- 172.21.15.157:0/474057767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff328070fa0 0x7ff328146440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 >> 172.21.15.157:0/474057767 conn(0x7ff32806a4d0 msgr2=0x7ff3281533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:00.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 shutdown_connections 2023-11-06T03:05:00.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.164+0000 7ff32e994640 1 -- 172.21.15.157:0/474057767 wait complete. 2023-11-06T03:05:00.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi195.front.sepia.ceph.com... 2023-11-06T03:05:00.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:00.182 DEBUG:teuthology.orchestra.run.smithi195:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-06T03:05:00.196 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:05:00.196 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:05:00.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:00.197 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:05:00.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:05:00.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:00.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 Processor -- start 2023-11-06T03:05:00.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 -- start start 2023-11-06T03:05:00.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa4c0cf8e0 0x7faa4c0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa4c0d0280 con 0x7faa4c1597d0 2023-11-06T03:05:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4c0cd1c0 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa54115640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c15a2c0 con 0x7faa4c0cc7f0 2023-11-06T03:05:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa516b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa4c0cf8e0 0x7faa4c0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa51eb1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faa4c1597d0 0x7faa4c159bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49804/0 (socket says 172.21.15.157:49804) 2023-11-06T03:05:00.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa51eb1640 1 -- 172.21.15.157:0/2452579005 learned_addr learned my addr 172.21.15.157:0/2452579005 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:00.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa526b2640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa516b0640 1 -- 172.21.15.157:0/2452579005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 msgr2=0x7faa4c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:00.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa516b0640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa516b0640 1 -- 172.21.15.157:0/2452579005 >> v1:172.21.15.157:6789/0 conn(0x7faa4c1597d0 legacy=0x7faa4c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa50eaf640 1 -- 172.21.15.157:0/2452579005 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938747578 0 0) 0x7faa4c0d0280 con 0x7faa4c1597d0 2023-11-06T03:05:00.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.372+0000 7faa516b0640 1 -- 172.21.15.157:0/2452579005 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c06c4b0 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa516b0640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c0cfd40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0097c0 tx=0x7faa3c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104987ae55afd64d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa50eaf640 1 -- 172.21.15.157:0/2452579005 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0ab680 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa50eaf640 1 -- 172.21.15.157:0/2452579005 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c0ab7e0 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa50eaf640 1 -- 172.21.15.157:0/2452579005 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0abb70 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 -- 172.21.15.157:0/2452579005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c0cf8e0 msgr2=0x7faa4c0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c0cfd40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0097c0 tx=0x7faa3c01c480 comp rx=0 tx=0).stop 2023-11-06T03:05:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 -- 172.21.15.157:0/2452579005 shutdown_connections 2023-11-06T03:05:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 --2- 172.21.15.157:0/2452579005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c0cfd40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 -- 172.21.15.157:0/2452579005 >> 172.21.15.157:0/2452579005 conn(0x7faa4c0732f0 msgr2=0x7faa4c069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 -- 172.21.15.157:0/2452579005 shutdown_connections 2023-11-06T03:05:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.376+0000 7faa54115640 1 -- 172.21.15.157:0/2452579005 wait complete. 2023-11-06T03:05:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 Processor -- start 2023-11-06T03:05:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- start start 2023-11-06T03:05:00.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c14f640 con 0x7faa4c1597d0 2023-11-06T03:05:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c14f360 con 0x7faa4c0cf8e0 2023-11-06T03:05:00.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c154050 con 0x7faa4c0cc7f0 2023-11-06T03:05:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47274/0 (socket says 172.21.15.157:47274) 2023-11-06T03:05:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 -- 172.21.15.157:0/4230844440 learned_addr learned my addr 172.21.15.157:0/4230844440 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa516b0640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa51eb1640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 -- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 msgr2=0x7faa4c150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 -- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa4c0cf8e0 msgr2=0x7faa4c14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 -- 172.21.15.157:0/4230844440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009420 con 0x7faa4c1597d0 2023-11-06T03:05:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa526b2640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faa44007e40 tx=0x7faa4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44014620 con 0x7faa4c1597d0 2023-11-06T03:05:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4400ee90 con 0x7faa4c1597d0 2023-11-06T03:05:00.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c154330 con 0x7faa4c1597d0 2023-11-06T03:05:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440149e0 con 0x7faa4c1597d0 2023-11-06T03:05:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c149380 con 0x7faa4c1597d0 2023-11-06T03:05:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c072750 con 0x7faa4c1597d0 2023-11-06T03:05:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faa44014b40 con 0x7faa4c1597d0 2023-11-06T03:05:00.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 0x7faa30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.380+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faa44083bc0 con 0x7faa4c1597d0 2023-11-06T03:05:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.384+0000 7faa51eb1640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 0x7faa30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.384+0000 7faa51eb1640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 0x7faa30067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa48000f70 tx=0x7faa48007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.384+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faa44051470 con 0x7faa4c1597d0 2023-11-06T03:05:00.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.572+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa4c06a0f0 con 0x7faa4c1597d0 2023-11-06T03:05:00.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.572+0000 7faa427fc640 1 -- 172.21.15.157:0/4230844440 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7faa44054a90 con 0x7faa4c1597d0 2023-11-06T03:05:00.575 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:00.575 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:05:00.575 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:05:00.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 msgr2=0x7faa30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 0x7faa30067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faa48000f70 tx=0x7faa48007c10 comp rx=0 tx=0).stop 2023-11-06T03:05:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 msgr2=0x7faa4c148000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faa44007e40 tx=0x7faa4400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 shutdown_connections 2023-11-06T03:05:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa30065a60 0x7faa30067f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4c1597d0 0x7faa4c148000 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa4c0cf8e0 0x7faa4c14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 --2- 172.21.15.157:0/4230844440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa4c0cc7f0 0x7faa4c150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 >> 172.21.15.157:0/4230844440 conn(0x7faa4c0732f0 msgr2=0x7faa4c0ceeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:00.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 shutdown_connections 2023-11-06T03:05:00.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.576+0000 7faa54115640 1 -- 172.21.15.157:0/4230844440 wait complete. 2023-11-06T03:05:00.593 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.708+0000 7f919c862640 1 Processor -- start 2023-11-06T03:05:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.708+0000 7f919c862640 1 -- start start 2023-11-06T03:05:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919c862640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91981551e0 0x7f919806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919c862640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f91980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919c862640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919815f580 con 0x7f919815ea60 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919c862640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919815efc0 con 0x7f91981551e0 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919c862640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919815f2a0 con 0x7f919815e5e0 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91981551e0 0x7f919806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91981551e0 0x7f919806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47276/0 (socket says 172.21.15.157:47276) 2023-11-06T03:05:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 -- 172.21.15.157:0/3285767663 learned_addr learned my addr 172.21.15.157:0/3285767663 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f918ffff640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f91980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 -- 172.21.15.157:0/3285767663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 msgr2=0x7f91980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f91980b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f9195d9b640 1 -- 172.21.15.157:0/3285767663 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556027496 0 0) 0x7f919815f580 con 0x7f919815ea60 2023-11-06T03:05:00.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 -- 172.21.15.157:0/3285767663 >> v1:172.21.15.157:6789/0 conn(0x7f919815ea60 legacy=0x7f919815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 -- 172.21.15.157:0/3285767663 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919806f5a0 con 0x7f91981551e0 2023-11-06T03:05:00.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f919659c640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f919806fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9188009870 tx=0x7f918801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3db008ece9ac023 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f9195d9b640 1 -- 172.21.15.157:0/3285767663 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91880ad640 con 0x7f91981551e0 2023-11-06T03:05:00.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f9195d9b640 1 -- 172.21.15.157:0/3285767663 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880ad7a0 con 0x7f91981551e0 2023-11-06T03:05:00.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.712+0000 7f9195d9b640 1 -- 172.21.15.157:0/3285767663 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91880adb30 con 0x7f91981551e0 2023-11-06T03:05:00.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/3285767663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 msgr2=0x7f919806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f919806fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9188009870 tx=0x7f918801c480 comp rx=0 tx=0).stop 2023-11-06T03:05:00.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/3285767663 shutdown_connections 2023-11-06T03:05:00.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f91980b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- 172.21.15.157:0/3285767663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f919806fda0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/3285767663 >> 172.21.15.157:0/3285767663 conn(0x7f9198065930 msgr2=0x7f9198065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:00.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/3285767663 shutdown_connections 2023-11-06T03:05:00.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/3285767663 wait complete. 2023-11-06T03:05:00.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 Processor -- start 2023-11-06T03:05:00.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- start start 2023-11-06T03:05:00.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f919806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f919815ea60 0x7f919814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919814e2d0 con 0x7f91981551e0 2023-11-06T03:05:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919814d560 con 0x7f919815ea60 2023-11-06T03:05:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919814d9c0 con 0x7f919815e5e0 2023-11-06T03:05:00.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47286/0 (socket says 172.21.15.157:47286) 2023-11-06T03:05:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 -- 172.21.15.157:0/2989985281 learned_addr learned my addr 172.21.15.157:0/2989985281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f918ffff640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f919806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 -- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 msgr2=0x7f919806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f919806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 -- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f919815ea60 msgr2=0x7f919814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f919815ea60 0x7f919814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 -- 172.21.15.157:0/2989985281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880094d0 con 0x7f91981551e0 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919659c640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91880b0040 tx=0x7f9188019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91880b6830 con 0x7f91981551e0 2023-11-06T03:05:00.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880a6040 con 0x7f91981551e0 2023-11-06T03:05:00.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919806d250 con 0x7f91981551e0 2023-11-06T03:05:00.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91880b6bf0 con 0x7f91981551e0 2023-11-06T03:05:00.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.716+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919814def0 con 0x7f91981551e0 2023-11-06T03:05:00.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919806b5c0 con 0x7f91981551e0 2023-11-06T03:05:00.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f91880b4070 con 0x7f91981551e0 2023-11-06T03:05:00.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f918effd640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 0x7f9178067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:00.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f918811dc10 con 0x7f91981551e0 2023-11-06T03:05:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f918ffff640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 0x7f9178067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.720+0000 7f918ffff640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 0x7f9178067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91980cad30 tx=0x7f9180008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.724+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f919806b5c0 con 0x7f91981551e0 2023-11-06T03:05:00.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.864+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f919814e410 con 0x7f91981551e0 2023-11-06T03:05:00.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.868+0000 7f918effd640 1 -- 172.21.15.157:0/2989985281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f918811b640 con 0x7f91981551e0 2023-11-06T03:05:00.870 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:00.870 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:00.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 msgr2=0x7f9178067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 0x7f9178067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91980cad30 tx=0x7f9180008040 comp rx=0 tx=0).stop 2023-11-06T03:05:00.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 msgr2=0x7f91981465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:00.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91880b0040 tx=0x7f9188019220 comp rx=0 tx=0).stop 2023-11-06T03:05:00.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 shutdown_connections 2023-11-06T03:05:00.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9178065a60 0x7f9178067f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f919815ea60 0x7f919814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f919815e5e0 0x7f919806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 --2- 172.21.15.157:0/2989985281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91981551e0 0x7f91981465d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:00.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 >> 172.21.15.157:0/2989985281 conn(0x7f9198065930 msgr2=0x7f91981544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:00.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 shutdown_connections 2023-11-06T03:05:00.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:00.872+0000 7f919c862640 1 -- 172.21.15.157:0/2989985281 wait complete. 2023-11-06T03:05:00.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:04:05.099545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:04:05.662817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:05:00.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:05:01.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 Processor -- start 2023-11-06T03:05:01.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 -- start start 2023-11-06T03:05:01.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39e015e5e0 0x7f39e00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e015f580 con 0x7f39e015ea90 2023-11-06T03:05:01.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e015efc0 con 0x7f39e01551e0 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e8635640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00704b0 con 0x7f39e015e5e0 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e5bd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39e015e5e0 0x7f39e00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e6bd2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f39e015ea90 0x7f39e015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49820/0 (socket says 172.21.15.157:49820) 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e6bd2640 1 -- 172.21.15.157:0/3142187234 learned_addr learned my addr 172.21.15.157:0/3142187234 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e5bd0640 1 -- 172.21.15.157:0/3142187234 >> v1:172.21.15.195:6789/0 conn(0x7f39e01551e0 legacy=0x7f39e006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e5bd0640 1 -- 172.21.15.157:0/3142187234 >> v1:172.21.15.157:6789/0 conn(0x7f39e015ea90 legacy=0x7f39e015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e5bd0640 1 -- 172.21.15.157:0/3142187234 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e015f2a0 con 0x7f39e015e5e0 2023-11-06T03:05:01.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.012+0000 7f39e5bd0640 1 --2- 172.21.15.157:0/3142187234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e00b3410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f39d00097c0 tx=0x7f39d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6fa653c66680d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e53cf640 1 -- 172.21.15.157:0/3142187234 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00ab600 con 0x7f39e015e5e0 2023-11-06T03:05:01.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e53cf640 1 -- 172.21.15.157:0/3142187234 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d00a7050 con 0x7f39e015e5e0 2023-11-06T03:05:01.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e53cf640 1 -- 172.21.15.157:0/3142187234 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00ab980 con 0x7f39e015e5e0 2023-11-06T03:05:01.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- 172.21.15.157:0/3142187234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 msgr2=0x7f39e00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 --2- 172.21.15.157:0/3142187234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e00b3410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f39d00097c0 tx=0x7f39d001c380 comp rx=0 tx=0).stop 2023-11-06T03:05:01.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- 172.21.15.157:0/3142187234 shutdown_connections 2023-11-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 --2- 172.21.15.157:0/3142187234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e00b3410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- 172.21.15.157:0/3142187234 >> 172.21.15.157:0/3142187234 conn(0x7f39e0065930 msgr2=0x7f39e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- 172.21.15.157:0/3142187234 shutdown_connections 2023-11-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- 172.21.15.157:0/3142187234 wait complete. 2023-11-06T03:05:01.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 Processor -- start 2023-11-06T03:05:01.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- start start 2023-11-06T03:05:01.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e01551e0 0x7f39e0132cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e015ea90 0x7f39e0150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0151620 con 0x7f39e015e5e0 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0151800 con 0x7f39e01551e0 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e8635640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0151ae0 con 0x7f39e015ea90 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47306/0 (socket says 172.21.15.157:47306) 2023-11-06T03:05:01.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 -- 172.21.15.157:0/2168121898 learned_addr learned my addr 172.21.15.157:0/2168121898 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e6bd2640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e015ea90 0x7f39e0150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e63d1640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e01551e0 0x7f39e0132cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 -- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e015ea90 msgr2=0x7f39e0150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e015ea90 0x7f39e0150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 -- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e01551e0 msgr2=0x7f39e0132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.016+0000 7f39e5bd0640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e01551e0 0x7f39e0132cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e5bd0640 1 -- 172.21.15.157:0/2168121898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d0009420 con 0x7f39e015e5e0 2023-11-06T03:05:01.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e5bd0640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009710 tx=0x7f39d0019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00b4610 con 0x7f39e015e5e0 2023-11-06T03:05:01.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d00b4770 con 0x7f39e015e5e0 2023-11-06T03:05:01.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0151dc0 con 0x7f39e015e5e0 2023-11-06T03:05:01.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00b4b20 con 0x7f39e015e5e0 2023-11-06T03:05:01.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0149070 con 0x7f39e015e5e0 2023-11-06T03:05:01.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0072750 con 0x7f39e015e5e0 2023-11-06T03:05:01.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39d00b2040 con 0x7f39e015e5e0 2023-11-06T03:05:01.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f39d011d480 con 0x7f39e015e5e0 2023-11-06T03:05:01.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e63d1640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.020+0000 7f39e63d1640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39dc004e30 tx=0x7f39dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.024+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39d00eaca0 con 0x7f39e015e5e0 2023-11-06T03:05:01.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.172+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39e0152270 con 0x7f39e015e5e0 2023-11-06T03:05:01.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39d6ffd640 1 -- 172.21.15.157:0/2168121898 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39d00ee2c0 con 0x7f39e015e5e0 2023-11-06T03:05:01.175 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:05:01.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 msgr2=0x7f39c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39dc004e30 tx=0x7f39dc009040 comp rx=0 tx=0).stop 2023-11-06T03:05:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 msgr2=0x7f39e0148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009710 tx=0x7f39d0019120 comp rx=0 tx=0).stop 2023-11-06T03:05:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 shutdown_connections 2023-11-06T03:05:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39c4065a60 0x7f39c4067f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e015ea90 0x7f39e0150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e015e5e0 0x7f39e0148b30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 --2- 172.21.15.157:0/2168121898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e01551e0 0x7f39e0132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 >> 172.21.15.157:0/2168121898 conn(0x7f39e0065930 msgr2=0x7f39e00afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 shutdown_connections 2023-11-06T03:05:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.176+0000 7f39e8635640 1 -- 172.21.15.157:0/2168121898 wait complete. 2023-11-06T03:05:01.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:05:01.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 Processor -- start 2023-11-06T03:05:01.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 -- start start 2023-11-06T03:05:01.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f79b01551e0 0x7f79b006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 0x7f79b015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b015f580 con 0x7f79b015e5e0 2023-11-06T03:05:01.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b015efc0 con 0x7f79b01551e0 2023-11-06T03:05:01.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b7264640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015f2a0 con 0x7f79b00b3b20 2023-11-06T03:05:01.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79affff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f79b015e5e0 0x7f79b00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49824/0 (socket says 172.21.15.157:49824) 2023-11-06T03:05:01.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79affff640 1 -- 172.21.15.157:0/713203680 learned_addr learned my addr 172.21.15.157:0/713203680 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:01.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 --2- 172.21.15.157:0/713203680 >> v2:172.21.15.195:3300/0 conn(0x7f79b01551e0 0x7f79b006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5801640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 0x7f79b015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873972607 0 0) 0x7f79b015f580 con 0x7f79b015e5e0 2023-11-06T03:05:01.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003530 con 0x7f79b015e5e0 2023-11-06T03:05:01.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3732149300 0 0) 0x7f799c003530 con 0x7f79b015e5e0 2023-11-06T03:05:01.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 msgr2=0x7f79b015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 0x7f79b015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 >> v2:172.21.15.195:3300/0 conn(0x7f79b01551e0 msgr2=0x7f79b006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 --2- 172.21.15.157:0/713203680 >> v2:172.21.15.195:3300/0 conn(0x7f79b01551e0 0x7f79b006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b006f5a0 con 0x7f79b015e5e0 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7998001680 con 0x7f79b015e5e0 2023-11-06T03:05:01.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 >> v1:172.21.15.157:6789/0 conn(0x7f79b015e5e0 legacy=0x7f79b00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f799c003980 0x7f799c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 0x7f799c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f799c004d10 0x7f799c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015efc0 con 0x7f799c003980 2023-11-06T03:05:01.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015f2a0 con 0x7f799c004d10 2023-11-06T03:05:01.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00b9c0 con 0x7f79b00b3b20 2023-11-06T03:05:01.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f799c003980 0x7f799c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 -- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 msgr2=0x7f799c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b00b3b20 0x7f799c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 -- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f799c004d10 msgr2=0x7f799c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f799c004d10 0x7f799c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.304+0000 7f79b5000640 1 -- 172.21.15.157:0/713203680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003530 con 0x7f799c003980 2023-11-06T03:05:01.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b5000640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f799c003980 0x7f799c003da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009ae0 tx=0x7f79a801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad8ce3d14362801 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80a5650 con 0x7f799c003980 2023-11-06T03:05:01.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a80a57b0 con 0x7f799c003980 2023-11-06T03:05:01.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79af7fe640 1 -- 172.21.15.157:0/713203680 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80a5960 con 0x7f799c003980 2023-11-06T03:05:01.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- 172.21.15.157:0/713203680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f799c003980 msgr2=0x7f799c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 --2- 172.21.15.157:0/713203680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f799c003980 0x7f799c003da0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009ae0 tx=0x7f79a801ca60 comp rx=0 tx=0).stop 2023-11-06T03:05:01.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b5801640 1 -- 172.21.15.157:0/713203680 reap_dead start 2023-11-06T03:05:01.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- 172.21.15.157:0/713203680 shutdown_connections 2023-11-06T03:05:01.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- 172.21.15.157:0/713203680 >> 172.21.15.157:0/713203680 conn(0x7f79b0065930 msgr2=0x7f79b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- 172.21.15.157:0/713203680 shutdown_connections 2023-11-06T03:05:01.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- 172.21.15.157:0/713203680 wait complete. 2023-11-06T03:05:01.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 Processor -- start 2023-11-06T03:05:01.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- start start 2023-11-06T03:05:01.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f799c003980 0x7f79b01f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ccf60 0x7f79b01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01433a0 con 0x7f79b0155c70 2023-11-06T03:05:01.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00cd3e0 con 0x7f79b00ccf60 2023-11-06T03:05:01.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79b7264640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f5a40 con 0x7f799c003980 2023-11-06T03:05:01.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47330/0 (socket says 172.21.15.157:47330) 2023-11-06T03:05:01.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.308+0000 7f79affff640 1 -- 172.21.15.157:0/3948382477 learned_addr learned my addr 172.21.15.157:0/3948382477 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b5000640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f799c003980 0x7f79b01f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b5801640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ccf60 0x7f79b01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 -- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f799c003980 msgr2=0x7f79b01f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f799c003980 0x7f79b01f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 -- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ccf60 msgr2=0x7f79b01f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ccf60 0x7f79b01f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 -- 172.21.15.157:0/3948382477 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b01f5760 con 0x7f79b0155c70 2023-11-06T03:05:01.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79affff640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7a0 tx=0x7f799800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980074e0 con 0x7f79b0155c70 2023-11-06T03:05:01.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7998007660 con 0x7f79b0155c70 2023-11-06T03:05:01.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998004a30 con 0x7f79b0155c70 2023-11-06T03:05:01.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01f1670 con 0x7f79b0155c70 2023-11-06T03:05:01.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01f1c30 con 0x7f79b0155c70 2023-11-06T03:05:01.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b0072750 con 0x7f79b0155c70 2023-11-06T03:05:01.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7998004b90 con 0x7f79b0155c70 2023-11-06T03:05:01.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 0x7f7990067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f79980826b0 con 0x7f79b0155c70 2023-11-06T03:05:01.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b5000640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 0x7f7990067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.312+0000 7f79b5000640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 0x7f7990067f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f79a80af000 tx=0x7f79a80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.316+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f799804fed0 con 0x7f79b0155c70 2023-11-06T03:05:01.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.472+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79b0071730 con 0x7f79b0155c70 2023-11-06T03:05:01.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.472+0000 7f79ad7fa640 1 -- 172.21.15.157:0/3948382477 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7998018020 con 0x7f79b0155c70 2023-11-06T03:05:01.473 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:05:01.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 msgr2=0x7f7990067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 0x7f7990067f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f79a80af000 tx=0x7f79a80046d0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 msgr2=0x7f79b00cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7a0 tx=0x7f799800cc60 comp rx=0 tx=0).stop 2023-11-06T03:05:01.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 shutdown_connections 2023-11-06T03:05:01.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7990065a90 0x7f7990067f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b00ccf60 0x7f79b01f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b0155c70 0x7f79b00cca20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 --2- 172.21.15.157:0/3948382477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f799c003980 0x7f79b01f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 >> 172.21.15.157:0/3948382477 conn(0x7f79b0065930 msgr2=0x7f79b00b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 shutdown_connections 2023-11-06T03:05:01.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.476+0000 7f79b7264640 1 -- 172.21.15.157:0/3948382477 wait complete. 2023-11-06T03:05:01.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:05:01.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 Processor -- start 2023-11-06T03:05:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 -- start start 2023-11-06T03:05:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f65cc15e5e0 0x7f65cc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65cc15ea90 0x7f65cc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc15f580 con 0x7f65cc1551e0 2023-11-06T03:05:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0704b0 con 0x7f65cc15ea90 2023-11-06T03:05:01.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d2ce6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc15efc0 con 0x7f65cc15e5e0 2023-11-06T03:05:01.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65cc15ea90 0x7f65cc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65cc15ea90 0x7f65cc0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47346/0 (socket says 172.21.15.157:47346) 2023-11-06T03:05:01.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 -- 172.21.15.157:0/1735290118 learned_addr learned my addr 172.21.15.157:0/1735290118 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65cbfff640 1 --2- 172.21.15.157:0/1735290118 >> v2:172.21.15.195:3300/0 conn(0x7f65cc15e5e0 0x7f65cc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 -- 172.21.15.157:0/1735290118 >> v2:172.21.15.195:3300/0 conn(0x7f65cc15e5e0 msgr2=0x7f65cc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 --2- 172.21.15.157:0/1735290118 >> v2:172.21.15.195:3300/0 conn(0x7f65cc15e5e0 0x7f65cc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 -- 172.21.15.157:0/1735290118 >> v1:172.21.15.195:6789/0 conn(0x7f65cc1551e0 legacy=0x7f65cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.616+0000 7f65d1283640 1 -- 172.21.15.157:0/1735290118 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc15f2a0 con 0x7f65cc15ea90 2023-11-06T03:05:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d1283640 1 --2- 172.21.15.157:0/1735290118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc15ea90 0x7f65cc0b3950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65cc066880 tx=0x7f65c401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a319a4b7473dbb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65cb7fe640 1 -- 172.21.15.157:0/1735290118 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c40a57d0 con 0x7f65cc15ea90 2023-11-06T03:05:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65cb7fe640 1 -- 172.21.15.157:0/1735290118 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c40a5930 con 0x7f65cc15ea90 2023-11-06T03:05:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65cb7fe640 1 -- 172.21.15.157:0/1735290118 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c40a5cc0 con 0x7f65cc15ea90 2023-11-06T03:05:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- 172.21.15.157:0/1735290118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc15ea90 msgr2=0x7f65cc0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/1735290118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc15ea90 0x7f65cc0b3950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65cc066880 tx=0x7f65c401c910 comp rx=0 tx=0).stop 2023-11-06T03:05:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- 172.21.15.157:0/1735290118 shutdown_connections 2023-11-06T03:05:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/1735290118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc15ea90 0x7f65cc0b3950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/1735290118 >> v2:172.21.15.195:3300/0 conn(0x7f65cc15e5e0 0x7f65cc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- 172.21.15.157:0/1735290118 >> 172.21.15.157:0/1735290118 conn(0x7f65cc065930 msgr2=0x7f65cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- 172.21.15.157:0/1735290118 shutdown_connections 2023-11-06T03:05:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- 172.21.15.157:0/1735290118 wait complete. 2023-11-06T03:05:01.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 Processor -- start 2023-11-06T03:05:01.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- start start 2023-11-06T03:05:01.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65cc15e5e0 0x7f65cc1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 0x7f65cc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1589d0 con 0x7f65cc1551e0 2023-11-06T03:05:01.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc15a460 con 0x7f65cc15e5e0 2023-11-06T03:05:01.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d2ce6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1556f0 con 0x7f65cc15ea90 2023-11-06T03:05:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47348/0 (socket says 172.21.15.157:47348) 2023-11-06T03:05:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 -- 172.21.15.157:0/169422968 learned_addr learned my addr 172.21.15.157:0/169422968 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d1283640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 0x7f65cc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65cbfff640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65cc15e5e0 0x7f65cc1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 -- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 msgr2=0x7f65cc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 0x7f65cc1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 -- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65cc15e5e0 msgr2=0x7f65cc1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65cc15e5e0 0x7f65cc1564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d0a82640 1 -- 172.21.15.157:0/169422968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4009b30 con 0x7f65cc1551e0 2023-11-06T03:05:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.620+0000 7f65d1283640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 0x7f65cc1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65d0a82640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c530 tx=0x7f65bc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc005220 con 0x7f65cc1551e0 2023-11-06T03:05:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc005380 con 0x7f65cc1551e0 2023-11-06T03:05:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc1559d0 con 0x7f65cc1551e0 2023-11-06T03:05:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc007270 con 0x7f65cc1551e0 2023-11-06T03:05:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc155f90 con 0x7f65cc1551e0 2023-11-06T03:05:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc072750 con 0x7f65cc1551e0 2023-11-06T03:05:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f65bc0073d0 con 0x7f65cc1551e0 2023-11-06T03:05:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 0x7f65b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f65bc082bb0 con 0x7f65cc1551e0 2023-11-06T03:05:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65cbfff640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 0x7f65b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.624+0000 7f65cbfff640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 0x7f65b8067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0cec50 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.628+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f65bc050460 con 0x7f65cc1551e0 2023-11-06T03:05:01.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.776+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65cc15a840 con 0x7f65cc1551e0 2023-11-06T03:05:01.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.776+0000 7f65c97fa640 1 -- 172.21.15.157:0/169422968 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65bc053a80 con 0x7f65cc1551e0 2023-11-06T03:05:01.778 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:05:01.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 msgr2=0x7f65b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 0x7f65b8067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0cec50 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2023-11-06T03:05:01.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 msgr2=0x7f65cc157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c530 tx=0x7f65bc00c9f0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 shutdown_connections 2023-11-06T03:05:01.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b8065a60 0x7f65b8067f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65cc15ea90 0x7f65cc1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65cc15e5e0 0x7f65cc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 --2- 172.21.15.157:0/169422968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65cc1551e0 0x7f65cc157140 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 >> 172.21.15.157:0/169422968 conn(0x7f65cc065930 msgr2=0x7f65cc0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 shutdown_connections 2023-11-06T03:05:01.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.780+0000 7f65d2ce6640 1 -- 172.21.15.157:0/169422968 wait complete. 2023-11-06T03:05:01.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:05:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 Processor -- start 2023-11-06T03:05:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 -- start start 2023-11-06T03:05:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fed1015ccc0 0x7fed10065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed10065e70 0x7fed100b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed100662d0 con 0x7fed100b0d00 2023-11-06T03:05:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed10070d30 con 0x7fed10065e70 2023-11-06T03:05:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed17a5c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed100b1190 con 0x7fed1015ccc0 2023-11-06T03:05:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed10065e70 0x7fed100b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fed10065e70 0x7fed100b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47354/0 (socket says 172.21.15.157:47354) 2023-11-06T03:05:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 -- 172.21.15.157:0/522583205 learned_addr learned my addr 172.21.15.157:0/522583205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 -- 172.21.15.157:0/522583205 >> v2:172.21.15.195:3300/0 conn(0x7fed1015ccc0 msgr2=0x7fed10065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 --2- 172.21.15.157:0/522583205 >> v2:172.21.15.195:3300/0 conn(0x7fed1015ccc0 0x7fed10065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 -- 172.21.15.157:0/522583205 >> v1:172.21.15.195:6789/0 conn(0x7fed100b0d00 legacy=0x7fed1006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 -- 172.21.15.157:0/522583205 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed101606c0 con 0x7fed10065e70 2023-11-06T03:05:01.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed14ff7640 1 --2- 172.21.15.157:0/522583205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed100b07c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed1006abd0 tx=0x7fed0001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e8d0f99a3d7043 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed07fff640 1 -- 172.21.15.157:0/522583205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000aa620 con 0x7fed10065e70 2023-11-06T03:05:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed07fff640 1 -- 172.21.15.157:0/522583205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed000aa780 con 0x7fed10065e70 2023-11-06T03:05:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.916+0000 7fed07fff640 1 -- 172.21.15.157:0/522583205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000aab10 con 0x7fed10065e70 2023-11-06T03:05:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/522583205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 msgr2=0x7fed100b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- 172.21.15.157:0/522583205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed100b07c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed1006abd0 tx=0x7fed0001c4c0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/522583205 shutdown_connections 2023-11-06T03:05:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- 172.21.15.157:0/522583205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed100b07c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- 172.21.15.157:0/522583205 >> v2:172.21.15.195:3300/0 conn(0x7fed1015ccc0 0x7fed10065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/522583205 >> 172.21.15.157:0/522583205 conn(0x7fed100732f0 msgr2=0x7fed10069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/522583205 shutdown_connections 2023-11-06T03:05:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/522583205 wait complete. 2023-11-06T03:05:01.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 Processor -- start 2023-11-06T03:05:01.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- start start 2023-11-06T03:05:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed100b0d00 0x7fed10154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed1015ccc0 0x7fed1015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10155500 con 0x7fed10065e70 2023-11-06T03:05:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10154f40 con 0x7fed100b0d00 2023-11-06T03:05:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed10155220 con 0x7fed1015ccc0 2023-11-06T03:05:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47360/0 (socket says 172.21.15.157:47360) 2023-11-06T03:05:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 -- 172.21.15.157:0/878886872 learned_addr learned my addr 172.21.15.157:0/878886872 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 -- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed1015ccc0 msgr2=0x7fed1015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed1015ccc0 0x7fed1015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 -- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed100b0d00 msgr2=0x7fed10154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed100b0d00 0x7fed10154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 -- 172.21.15.157:0/878886872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000094d0 con 0x7fed10065e70 2023-11-06T03:05:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed157f8640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00b410 tx=0x7fed0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c011830 con 0x7fed10065e70 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c011990 con 0x7fed10065e70 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1015e940 con 0x7fed10065e70 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c011d40 con 0x7fed10065e70 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1014cc40 con 0x7fed10065e70 2023-11-06T03:05:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.920+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1015ee60 con 0x7fed10065e70 2023-11-06T03:05:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.924+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fed0c01a400 con 0x7fed10065e70 2023-11-06T03:05:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.924+0000 7fed05ffb640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 0x7fecf4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.924+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fed0c0834a0 con 0x7fed10065e70 2023-11-06T03:05:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.924+0000 7fed14ff7640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 0x7fecf4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.924+0000 7fed14ff7640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 0x7fecf4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed000ae000 tx=0x7fed00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:01.928+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fed0c050d70 con 0x7fed10065e70 2023-11-06T03:05:02.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.080+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed10072750 con 0x7fed10065e70 2023-11-06T03:05:02.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed05ffb640 1 -- 172.21.15.157:0/878886872 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed0c054390 con 0x7fed10065e70 2023-11-06T03:05:02.083 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:05:02.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 msgr2=0x7fecf4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 0x7fecf4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed000ae000 tx=0x7fed00008040 comp rx=0 tx=0).stop 2023-11-06T03:05:02.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 msgr2=0x7fed1014c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00b410 tx=0x7fed0c00b8d0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 shutdown_connections 2023-11-06T03:05:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecf4065a60 0x7fecf4067f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fed1015ccc0 0x7fed1015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fed100b0d00 0x7fed10154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 --2- 172.21.15.157:0/878886872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed10065e70 0x7fed1014c700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 >> 172.21.15.157:0/878886872 conn(0x7fed100732f0 msgr2=0x7fed100ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 shutdown_connections 2023-11-06T03:05:02.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.084+0000 7fed17a5c640 1 -- 172.21.15.157:0/878886872 wait complete. 2023-11-06T03:05:02.102 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:02.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.216+0000 7fd2bd37e640 1 Processor -- start 2023-11-06T03:05:02.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.216+0000 7fd2bd37e640 1 -- start start 2023-11-06T03:05:02.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2bd37e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2b80c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2bd37e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2b80c9d70 0x7fd2b80aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2bd37e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b80ac9f0 con 0x7fd2b815cea0 2023-11-06T03:05:02.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2bd37e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80ce2a0 con 0x7fd2b80c9d70 2023-11-06T03:05:02.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2bd37e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80ca200 con 0x7fd2b81609a0 2023-11-06T03:05:02.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd2b815cea0 0x7fd2b80cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49826/0 (socket says 172.21.15.157:49826) 2023-11-06T03:05:02.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b6ffd640 1 -- 172.21.15.157:0/3788501644 learned_addr learned my addr 172.21.15.157:0/3788501644 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:02.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585030313 0 0) 0x7fd2b80ac9f0 con 0x7fd2b815cea0 2023-11-06T03:05:02.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b67fc640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2b80c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/3788501644 >> v2:172.21.15.195:3300/0 conn(0x7fd2b80c9d70 0x7fd2b80aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a4003530 con 0x7fd2b815cea0 2023-11-06T03:05:02.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3616718339 0 0) 0x7fd2a4003530 con 0x7fd2b815cea0 2023-11-06T03:05:02.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 msgr2=0x7fd2b80c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2b80c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 >> v2:172.21.15.195:3300/0 conn(0x7fd2b80c9d70 msgr2=0x7fd2b80aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 --2- 172.21.15.157:0/3788501644 >> v2:172.21.15.195:3300/0 conn(0x7fd2b80c9d70 0x7fd2b80aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80ced30 con 0x7fd2b815cea0 2023-11-06T03:05:02.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd2b0001a00 con 0x7fd2b815cea0 2023-11-06T03:05:02.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 >> v1:172.21.15.157:6789/0 conn(0x7fd2b815cea0 legacy=0x7fd2b80cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2a4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4003f00 0x7fd2a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a4004d10 0x7fd2a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80ca200 con 0x7fd2a4004d10 2023-11-06T03:05:02.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a400b990 con 0x7fd2a4003f00 2023-11-06T03:05:02.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a400bb20 con 0x7fd2b81609a0 2023-11-06T03:05:02.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a4004d10 0x7fd2a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b6ffd640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2a4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b67fc640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4003f00 0x7fd2a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 -- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 msgr2=0x7fd2a4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b81609a0 0x7fd2a4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 -- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4003f00 msgr2=0x7fd2a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4003f00 0x7fd2a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.220+0000 7fd2b77fe640 1 -- 172.21.15.157:0/3788501644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4003530 con 0x7fd2a4004d10 2023-11-06T03:05:02.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a4004d10 0x7fd2a4005190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b10 tx=0x7fd2a001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b447bb54519b2a0a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00a5570 con 0x7fd2a4004d10 2023-11-06T03:05:02.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a00a56d0 con 0x7fd2a4004d10 2023-11-06T03:05:02.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b5ffb640 1 -- 172.21.15.157:0/3788501644 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00a5a80 con 0x7fd2a4004d10 2023-11-06T03:05:02.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/3788501644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a4004d10 msgr2=0x7fd2a4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/3788501644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a4004d10 0x7fd2a4005190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b10 tx=0x7fd2a001ca90 comp rx=0 tx=0).stop 2023-11-06T03:05:02.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 -- 172.21.15.157:0/3788501644 reap_dead start 2023-11-06T03:05:02.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/3788501644 shutdown_connections 2023-11-06T03:05:02.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/3788501644 >> 172.21.15.157:0/3788501644 conn(0x7fd2b80cba30 msgr2=0x7fd2b80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/3788501644 shutdown_connections 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/3788501644 wait complete. 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 Processor -- start 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- start start 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 0x7fd2b815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b8147550 0x7fd2b80b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80b3cc0 con 0x7fd2b80b3840 2023-11-06T03:05:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80b4230 con 0x7fd2a4004d10 2023-11-06T03:05:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80c6080 con 0x7fd2b8147550 2023-11-06T03:05:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47376/0 (socket says 172.21.15.157:47376) 2023-11-06T03:05:02.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 -- 172.21.15.157:0/2029161137 learned_addr learned my addr 172.21.15.157:0/2029161137 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:02.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b6ffd640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 0x7fd2b815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b67fc640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b8147550 0x7fd2b80b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 -- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b8147550 msgr2=0x7fd2b80b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b8147550 0x7fd2b80b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 -- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 msgr2=0x7fd2b815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 0x7fd2b815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 -- 172.21.15.157:0/2029161137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80c6640 con 0x7fd2b80b3840 2023-11-06T03:05:02.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b6ffd640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 0x7fd2b815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:02.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2b77fe640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b10 tx=0x7fd2a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00ba070 con 0x7fd2b80b3840 2023-11-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0004020 con 0x7fd2b80b3840 2023-11-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00a5570 con 0x7fd2b80b3840 2023-11-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.224+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b815d620 con 0x7fd2b80b3840 2023-11-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b80c62d0 con 0x7fd2b80b3840 2023-11-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b815db40 con 0x7fd2b80b3840 2023-11-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd2a00b4070 con 0x7fd2b80b3840 2023-11-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2af7fe640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd2a011cdb0 con 0x7fd2b80b3840 2023-11-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2b6ffd640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.228+0000 7fd2b6ffd640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 0x7fd298067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0004c00 tx=0x7fd2b0004b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.232+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd2a00ea780 con 0x7fd2b80b3840 2023-11-06T03:05:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.412+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b81617a0 con 0x7fd2b80b3840 2023-11-06T03:05:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.412+0000 7fd2af7fe640 1 -- 172.21.15.157:0/2029161137 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fd2a00edda0 con 0x7fd2b80b3840 2023-11-06T03:05:02.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:02.415 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:05:02.415 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:05:02.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 msgr2=0x7fd298067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 0x7fd298067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0004c00 tx=0x7fd2b0004b70 comp rx=0 tx=0).stop 2023-11-06T03:05:02.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 msgr2=0x7fd2b81f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009b10 tx=0x7fd2a4005800 comp rx=0 tx=0).stop 2023-11-06T03:05:02.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 shutdown_connections 2023-11-06T03:05:02.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd298065a60 0x7fd298067f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80b3840 0x7fd2b81f5a80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2b8147550 0x7fd2b80b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 --2- 172.21.15.157:0/2029161137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a4004d10 0x7fd2b815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 >> 172.21.15.157:0/2029161137 conn(0x7fd2b80cba30 msgr2=0x7fd2b815c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 shutdown_connections 2023-11-06T03:05:02.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.416+0000 7fd2bd37e640 1 -- 172.21.15.157:0/2029161137 wait complete. 2023-11-06T03:05:02.432 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.540+0000 7f1666781640 1 Processor -- start 2023-11-06T03:05:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.540+0000 7f1666781640 1 -- start start 2023-11-06T03:05:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f1666781640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16601551e0 0x7f166006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f1666781640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f166015e5e0 0x7f16600b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f1666781640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166015f580 con 0x7f166015ea90 2023-11-06T03:05:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f1666781640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166015efc0 con 0x7f16601551e0 2023-11-06T03:05:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f1666781640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166015f2a0 con 0x7f166015e5e0 2023-11-06T03:05:02.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16601551e0 0x7f166006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16601551e0 0x7f166006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47378/0 (socket says 172.21.15.157:47378) 2023-11-06T03:05:02.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 -- 172.21.15.157:0/1963478252 learned_addr learned my addr 172.21.15.157:0/1963478252 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:02.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165f7fe640 1 --2- 172.21.15.157:0/1963478252 >> v2:172.21.15.195:3300/0 conn(0x7f166015e5e0 0x7f16600b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 -- 172.21.15.157:0/1963478252 >> v2:172.21.15.195:3300/0 conn(0x7f166015e5e0 msgr2=0x7f16600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 --2- 172.21.15.157:0/1963478252 >> v2:172.21.15.195:3300/0 conn(0x7f166015e5e0 0x7f16600b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 -- 172.21.15.157:0/1963478252 >> v1:172.21.15.195:6789/0 conn(0x7f166015ea90 legacy=0x7f166015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 -- 172.21.15.157:0/1963478252 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166006f5a0 con 0x7f16601551e0 2023-11-06T03:05:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165ffff640 1 --2- 172.21.15.157:0/1963478252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16601551e0 0x7f166006fda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1660066880 tx=0x7f165001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef02d7b78b71f6d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165effd640 1 -- 172.21.15.157:0/1963478252 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500ac640 con 0x7f16601551e0 2023-11-06T03:05:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165effd640 1 -- 172.21.15.157:0/1963478252 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16500ac7a0 con 0x7f16601551e0 2023-11-06T03:05:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.544+0000 7f165effd640 1 -- 172.21.15.157:0/1963478252 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500acb30 con 0x7f16601551e0 2023-11-06T03:05:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/1963478252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16601551e0 msgr2=0x7f166006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- 172.21.15.157:0/1963478252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16601551e0 0x7f166006fda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1660066880 tx=0x7f165001c400 comp rx=0 tx=0).stop 2023-11-06T03:05:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/1963478252 shutdown_connections 2023-11-06T03:05:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- 172.21.15.157:0/1963478252 >> v2:172.21.15.195:3300/0 conn(0x7f166015e5e0 0x7f16600b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- 172.21.15.157:0/1963478252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16601551e0 0x7f166006fda0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/1963478252 >> 172.21.15.157:0/1963478252 conn(0x7f1660065930 msgr2=0x7f1660065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/1963478252 shutdown_connections 2023-11-06T03:05:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/1963478252 wait complete. 2023-11-06T03:05:02.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 Processor -- start 2023-11-06T03:05:02.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- start start 2023-11-06T03:05:02.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 0x7f1660147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 0x7f1660154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166014d700 con 0x7f166015ea90 2023-11-06T03:05:02.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660153ed0 con 0x7f166015e5e0 2023-11-06T03:05:02.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16601484c0 con 0x7f16601551e0 2023-11-06T03:05:02.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47384/0 (socket says 172.21.15.157:47384) 2023-11-06T03:05:02.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 -- 172.21.15.157:0/3108212081 learned_addr learned my addr 172.21.15.157:0/3108212081 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:02.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165f7fe640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 0x7f1660154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165ffff640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 0x7f1660147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 -- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 msgr2=0x7f1660147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 0x7f1660147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 -- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 msgr2=0x7f1660154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 0x7f1660154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 -- 172.21.15.157:0/3108212081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500094d0 con 0x7f166015ea90 2023-11-06T03:05:02.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165ffff640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 0x7f1660147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:02.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165f7fe640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 0x7f1660154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:02.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1664d1e640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f165800b9a0 tx=0x7f165800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165800cba0 con 0x7f166015ea90 2023-11-06T03:05:02.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165800cd00 con 0x7f166015ea90 2023-11-06T03:05:02.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16601f5ae0 con 0x7f166015ea90 2023-11-06T03:05:02.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658014680 con 0x7f166015ea90 2023-11-06T03:05:02.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1660153880 con 0x7f166015ea90 2023-11-06T03:05:02.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.548+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1660154080 con 0x7f166015ea90 2023-11-06T03:05:02.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.552+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f165801d3b0 con 0x7f166015ea90 2023-11-06T03:05:02.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.552+0000 7f165cff9640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 0x7f164c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.552+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1658083130 con 0x7f166015ea90 2023-11-06T03:05:02.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.552+0000 7f165ffff640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 0x7f164c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.552+0000 7f165ffff640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 0x7f164c067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16500ae000 tx=0x7f1650008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.556+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1660154080 con 0x7f166015ea90 2023-11-06T03:05:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.700+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1660154080 con 0x7f166015ea90 2023-11-06T03:05:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.700+0000 7f165cff9640 1 -- 172.21.15.157:0/3108212081 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f1660154080 con 0x7f166015ea90 2023-11-06T03:05:02.703 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:02.703 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 msgr2=0x7f164c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 0x7f164c067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16500ae000 tx=0x7f1650008040 comp rx=0 tx=0).stop 2023-11-06T03:05:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 msgr2=0x7f1660150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f165800b9a0 tx=0x7f165800be60 comp rx=0 tx=0).stop 2023-11-06T03:05:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 shutdown_connections 2023-11-06T03:05:02.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f164c065a60 0x7f164c067f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f166015ea90 0x7f1660150120 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f166015e5e0 0x7f1660154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 --2- 172.21.15.157:0/3108212081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16601551e0 0x7f1660147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 >> 172.21.15.157:0/3108212081 conn(0x7f1660065930 msgr2=0x7f16600afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 shutdown_connections 2023-11-06T03:05:02.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.704+0000 7f1666781640 1 -- 172.21.15.157:0/3108212081 wait complete. 2023-11-06T03:05:02.722 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:05:02.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:02.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.828+0000 7f6f9ed72640 1 Processor -- start 2023-11-06T03:05:02.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.828+0000 7f6f9ed72640 1 -- start start 2023-11-06T03:05:02.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9ed72640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6f980b3b20 0x7f6f9815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9ed72640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9815f580 con 0x7f6f981551e0 2023-11-06T03:05:02.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9ed72640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9815efc0 con 0x7f6f9815e5e0 2023-11-06T03:05:02.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9ed72640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f980704b0 con 0x7f6f980b3b20 2023-11-06T03:05:02.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9cb0e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6f981551e0 0x7f6f9806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49832/0 (socket says 172.21.15.157:49832) 2023-11-06T03:05:02.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9cb0e640 1 -- 172.21.15.157:0/287316799 learned_addr learned my addr 172.21.15.157:0/287316799 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:02.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239978858 0 0) 0x7f6f9815f580 con 0x7f6f981551e0 2023-11-06T03:05:02.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/287316799 >> v2:172.21.15.195:3300/0 conn(0x7f6f980b3b20 0x7f6f9815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f80003530 con 0x7f6f981551e0 2023-11-06T03:05:02.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207804649 0 0) 0x7f6f80003530 con 0x7f6f981551e0 2023-11-06T03:05:02.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 >> v2:172.21.15.195:3300/0 conn(0x7f6f980b3b20 msgr2=0x7f6f9815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 --2- 172.21.15.157:0/287316799 >> v2:172.21.15.195:3300/0 conn(0x7f6f980b3b20 0x7f6f9815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 >> v1:172.21.15.195:6789/0 conn(0x7f6f9815e5e0 legacy=0x7f6f980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9815f2a0 con 0x7f6f981551e0 2023-11-06T03:05:02.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6f88001680 con 0x7f6f981551e0 2023-11-06T03:05:02.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 >> v1:172.21.15.157:6789/0 conn(0x7f6f981551e0 legacy=0x7f6f9806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800039d0 0x7f6f80003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 0x7f6f80004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f800052b0 0x7f6f8000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980704b0 con 0x7f6f800047d0 2023-11-06T03:05:02.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8000c1b0 con 0x7f6f800052b0 2023-11-06T03:05:02.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8000c370 con 0x7f6f800039d0 2023-11-06T03:05:02.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 0x7f6f80004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9cb0e640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800039d0 0x7f6f80003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f800052b0 0x7f6f8000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 -- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800039d0 msgr2=0x7f6f80003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800039d0 0x7f6f80003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 -- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f800052b0 msgr2=0x7f6f8000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f800052b0 0x7f6f8000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 -- 172.21.15.157:0/287316799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80003530 con 0x7f6f800047d0 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f800052b0 0x7f6f8000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f9d30f640 1 -- 172.21.15.157:0/287316799 reap_dead start 2023-11-06T03:05:02.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.832+0000 7f6f97fff640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 0x7f6f80004c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c001780 tx=0x7f6f8c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0381a6ce5eadef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0052d0 con 0x7f6f800047d0 2023-11-06T03:05:02.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c0a7050 con 0x7f6f800047d0 2023-11-06T03:05:02.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f977fe640 1 -- 172.21.15.157:0/287316799 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c003600 con 0x7f6f800047d0 2023-11-06T03:05:02.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 msgr2=0x7f6f80004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 0x7f6f80004c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c001780 tx=0x7f6f8c01c3b0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- 172.21.15.157:0/287316799 shutdown_connections 2023-11-06T03:05:02.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/287316799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f800047d0 0x7f6f80004c50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- 172.21.15.157:0/287316799 >> 172.21.15.157:0/287316799 conn(0x7f6f98065930 msgr2=0x7f6f98065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:02.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- 172.21.15.157:0/287316799 shutdown_connections 2023-11-06T03:05:02.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- 172.21.15.157:0/287316799 wait complete. 2023-11-06T03:05:02.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 Processor -- start 2023-11-06T03:05:02.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- start start 2023-11-06T03:05:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800047d0 0x7f6f9813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f981e8230 0x7f6f981e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981e75c0 con 0x7f6f980cddc0 2023-11-06T03:05:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981e8bf0 con 0x7f6f981e8230 2023-11-06T03:05:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9ed72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980ce740 con 0x7f6f800047d0 2023-11-06T03:05:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47402/0 (socket says 172.21.15.157:47402) 2023-11-06T03:05:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 -- 172.21.15.157:0/4132590438 learned_addr learned my addr 172.21.15.157:0/4132590438 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f97fff640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f981e8230 0x7f6f981e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9cb0e640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800047d0 0x7f6f9813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 -- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800047d0 msgr2=0x7f6f9813e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800047d0 0x7f6f9813e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 -- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f981e8230 msgr2=0x7f6f981e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f981e8230 0x7f6f981e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 -- 172.21.15.157:0/4132590438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f981e6660 con 0x7f6f980cddc0 2023-11-06T03:05:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.836+0000 7f6f9d30f640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6f980cbfe0 tx=0x7f6f90010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001b070 con 0x7f6f980cddc0 2023-11-06T03:05:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000f040 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f981e6940 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000e050 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981e6f00 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f98072750 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6f90015440 con 0x7f6f980cddc0 2023-11-06T03:05:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 0x7f6f84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6f90083660 con 0x7f6f980cddc0 2023-11-06T03:05:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f9cb0e640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 0x7f6f84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.840+0000 7f6f9cb0e640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 0x7f6f84067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f88001ec0 tx=0x7f6f88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:02.844+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6f90050e60 con 0x7f6f980cddc0 2023-11-06T03:05:03.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f98071730 con 0x7f6f980cddc0 2023-11-06T03:05:03.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f957fa640 1 -- 172.21.15.157:0/4132590438 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f6f90054480 con 0x7f6f980cddc0 2023-11-06T03:05:03.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:03.033 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:05:03.034 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:05:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 msgr2=0x7f6f84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 0x7f6f84067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f88001ec0 tx=0x7f6f88009040 comp rx=0 tx=0).stop 2023-11-06T03:05:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 msgr2=0x7f6f980ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6f980cbfe0 tx=0x7f6f90010590 comp rx=0 tx=0).stop 2023-11-06T03:05:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 shutdown_connections 2023-11-06T03:05:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f84065a60 0x7f6f84067f20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f980cddc0 0x7f6f980ce200 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f981e8230 0x7f6f981e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 --2- 172.21.15.157:0/4132590438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f800047d0 0x7f6f9813e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 >> 172.21.15.157:0/4132590438 conn(0x7f6f98065930 msgr2=0x7f6f980b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 shutdown_connections 2023-11-06T03:05:03.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.032+0000 7f6f9ed72640 1 -- 172.21.15.157:0/4132590438 wait complete. 2023-11-06T03:05:03.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:03.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.156+0000 7f9b03a06640 1 Processor -- start 2023-11-06T03:05:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.156+0000 7f9b03a06640 1 -- start start 2023-11-06T03:05:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9afc15cf60 0x7f9afc0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0cb6e0 con 0x7f9afc15cb80 2023-11-06T03:05:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc071830 con 0x7f9afc0cc910 2023-11-06T03:05:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc071b10 con 0x7f9afc15cf60 2023-11-06T03:05:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b017a2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9afc15cb80 0x7f9afc0cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49838/0 (socket says 172.21.15.157:49838) 2023-11-06T03:05:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b017a2640 1 -- 172.21.15.157:0/4279428142 learned_addr learned my addr 172.21.15.157:0/4279428142 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b00fa1640 1 --2- 172.21.15.157:0/4279428142 >> v2:172.21.15.157:3300/0 conn(0x7f9afc15cf60 0x7f9afc0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9af3fff640 1 -- 172.21.15.157:0/4279428142 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992397227 0 0) 0x7f9afc0cb6e0 con 0x7f9afc15cb80 2023-11-06T03:05:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9af3fff640 1 -- 172.21.15.157:0/4279428142 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003530 con 0x7f9afc15cb80 2023-11-06T03:05:03.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b00fa1640 1 -- 172.21.15.157:0/4279428142 >> v1:172.21.15.195:6789/0 conn(0x7f9afc0cc910 legacy=0x7f9afc0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b00fa1640 1 -- 172.21.15.157:0/4279428142 >> v1:172.21.15.157:6789/0 conn(0x7f9afc15cb80 legacy=0x7f9afc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b00fa1640 1 -- 172.21.15.157:0/4279428142 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc071df0 con 0x7f9afc15cf60 2023-11-06T03:05:03.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b00fa1640 1 --2- 172.21.15.157:0/4279428142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cf60 0x7f9afc0cd7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097c0 tx=0x7f9af801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc9ed6210db8a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9af3fff640 1 -- 172.21.15.157:0/4279428142 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80a8970 con 0x7f9afc15cf60 2023-11-06T03:05:03.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9af3fff640 1 -- 172.21.15.157:0/4279428142 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80a8ad0 con 0x7f9afc15cf60 2023-11-06T03:05:03.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9af3fff640 1 -- 172.21.15.157:0/4279428142 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80a8e60 con 0x7f9afc15cf60 2023-11-06T03:05:03.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- 172.21.15.157:0/4279428142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cf60 msgr2=0x7f9afc0cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 --2- 172.21.15.157:0/4279428142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cf60 0x7f9afc0cd7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097c0 tx=0x7f9af801ec70 comp rx=0 tx=0).stop 2023-11-06T03:05:03.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- 172.21.15.157:0/4279428142 shutdown_connections 2023-11-06T03:05:03.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 --2- 172.21.15.157:0/4279428142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cf60 0x7f9afc0cd7e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.160+0000 7f9b03a06640 1 -- 172.21.15.157:0/4279428142 >> 172.21.15.157:0/4279428142 conn(0x7f9afc06d560 msgr2=0x7f9afc06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- 172.21.15.157:0/4279428142 shutdown_connections 2023-11-06T03:05:03.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- 172.21.15.157:0/4279428142 wait complete. 2023-11-06T03:05:03.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 Processor -- start 2023-11-06T03:05:03.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- start start 2023-11-06T03:05:03.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 0x7f9afc14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9afc15cf60 0x7f9afc1e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1e5220 con 0x7f9afc15cb80 2023-11-06T03:05:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc157540 con 0x7f9afc0cc910 2023-11-06T03:05:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc157ac0 con 0x7f9afc15cf60 2023-11-06T03:05:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47422/0 (socket says 172.21.15.157:47422) 2023-11-06T03:05:03.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 -- 172.21.15.157:0/2198982705 learned_addr learned my addr 172.21.15.157:0/2198982705 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:03.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b017a2640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 0x7f9afc14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b01fa3640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9afc15cf60 0x7f9afc1e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 -- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9afc15cf60 msgr2=0x7f9afc1e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9afc15cf60 0x7f9afc1e42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 -- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 msgr2=0x7f9afc14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 0x7f9afc14dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 -- 172.21.15.157:0/2198982705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009420 con 0x7f9afc15cb80 2023-11-06T03:05:03.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b017a2640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 0x7f9afc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:03.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b00fa1640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3040 tx=0x7f9af801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80b9630 con 0x7f9afc15cb80 2023-11-06T03:05:03.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80b97b0 con 0x7f9afc15cb80 2023-11-06T03:05:03.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80b9b60 con 0x7f9afc15cb80 2023-11-06T03:05:03.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc1e4c60 con 0x7f9afc15cb80 2023-11-06T03:05:03.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc157750 con 0x7f9afc15cb80 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.164+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc06bf00 con 0x7f9afc15cb80 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.168+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9af80b7040 con 0x7f9afc15cb80 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.168+0000 7f9af1ffb640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 0x7f9ae0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.168+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9af8121290 con 0x7f9afc15cb80 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.168+0000 7f9b017a2640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 0x7f9ae0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.168+0000 7f9b017a2640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 0x7f9ae0067f50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0011f0 tx=0x7f9aec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.172+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9af80eeaa0 con 0x7f9afc15cb80 2023-11-06T03:05:03.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.344+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc072750 con 0x7f9afc15cb80 2023-11-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.344+0000 7f9af1ffb640 1 -- 172.21.15.157:0/2198982705 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f9af80f20c0 con 0x7f9afc15cb80 2023-11-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/256769313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":256769313},{"type":"v1","addr":"172.21.15.157:6834","nonce":256769313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2079398080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2079398080},{"type":"v1","addr":"172.21.15.157:6835","nonce":2079398080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3391898231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3391898231},{"type":"v1","addr":"172.21.15.195:6837","nonce":3391898231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:04:06.679919+0000","modified":"2023-11-06T03:04:09.322110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1904182839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1904182839},{"type":"v1","addr":"172.21.15.195:6835","nonce":1904182839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-06T03:05:03.346 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 7 2023-11-06T03:05:03.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.344+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 msgr2=0x7f9ae0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 0x7f9ae0067f50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0011f0 tx=0x7f9aec009040 comp rx=0 tx=0).stop 2023-11-06T03:05:03.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 msgr2=0x7f9afc0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3040 tx=0x7f9af801ba10 comp rx=0 tx=0).stop 2023-11-06T03:05:03.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 shutdown_connections 2023-11-06T03:05:03.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ae0065a90 0x7f9ae0067f50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9afc15cf60 0x7f9afc1e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9afc15cb80 0x7f9afc0c9f00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 --2- 172.21.15.157:0/2198982705 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9afc0cc910 0x7f9afc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 >> 172.21.15.157:0/2198982705 conn(0x7f9afc06d560 msgr2=0x7f9afc152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 shutdown_connections 2023-11-06T03:05:03.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.348+0000 7f9b03a06640 1 -- 172.21.15.157:0/2198982705 wait complete. 2023-11-06T03:05:03.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:03.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 Processor -- start 2023-11-06T03:05:03.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 -- start start 2023-11-06T03:05:03.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3e7c0d0280 0x7f3e7c167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0d2920 con 0x7f3e7c15caa0 2023-11-06T03:05:03.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0d18a0 con 0x7f3e7c15ce80 2023-11-06T03:05:03.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e8455c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d1b60 con 0x7f3e7c0d0280 2023-11-06T03:05:03.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e822f8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3e7c15caa0 0x7f3e7c0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49848/0 (socket says 172.21.15.157:49848) 2023-11-06T03:05:03.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e822f8640 1 -- 172.21.15.157:0/1420665886 learned_addr learned my addr 172.21.15.157:0/1420665886 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:03.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995507834 0 0) 0x7f3e7c0d2920 con 0x7f3e7c15caa0 2023-11-06T03:05:03.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e82af9640 1 --2- 172.21.15.157:0/1420665886 >> v2:172.21.15.195:3300/0 conn(0x7f3e7c0d0280 0x7f3e7c167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e68003530 con 0x7f3e7c15caa0 2023-11-06T03:05:03.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908185239 0 0) 0x7f3e68003530 con 0x7f3e7c15caa0 2023-11-06T03:05:03.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 >> v2:172.21.15.195:3300/0 conn(0x7f3e7c0d0280 msgr2=0x7f3e7c167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 --2- 172.21.15.157:0/1420665886 >> v2:172.21.15.195:3300/0 conn(0x7f3e7c0d0280 0x7f3e7c167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 >> v1:172.21.15.195:6789/0 conn(0x7f3e7c15ce80 legacy=0x7f3e7c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0d1e40 con 0x7f3e7c15caa0 2023-11-06T03:05:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3e6c001680 con 0x7f3e7c15caa0 2023-11-06T03:05:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 >> v1:172.21.15.157:6789/0 conn(0x7f3e7c15caa0 legacy=0x7f3e7c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e68003980 0x7f3e68003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 0x7f3e68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e68005190 0x7f3e6800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0d1b60 con 0x7f3e680047d0 2023-11-06T03:05:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c030 con 0x7f3e68003980 2023-11-06T03:05:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c1f0 con 0x7f3e68005190 2023-11-06T03:05:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 0x7f3e68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 -- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e68005190 msgr2=0x7f3e6800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e68005190 0x7f3e6800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 -- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e68003980 msgr2=0x7f3e68003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e68003980 0x7f3e68003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e81af7640 1 -- 172.21.15.157:0/1420665886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68003530 con 0x7f3e680047d0 2023-11-06T03:05:03.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.472+0000 7f3e82af9640 1 -- 172.21.15.157:0/1420665886 reap_dead start 2023-11-06T03:05:03.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e81af7640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 0x7f3e68004c50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3e78001750 tx=0x7f3e7801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e903bb1043d24c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780052d0 con 0x7f3e680047d0 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780a7050 con 0x7f3e680047d0 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e812f6640 1 -- 172.21.15.157:0/1420665886 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003600 con 0x7f3e680047d0 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 msgr2=0x7f3e68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 0x7f3e68004c50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3e78001750 tx=0x7f3e7801c3a0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- 172.21.15.157:0/1420665886 shutdown_connections 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1420665886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e680047d0 0x7f3e68004c50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- 172.21.15.157:0/1420665886 >> 172.21.15.157:0/1420665886 conn(0x7f3e7c070790 msgr2=0x7f3e7c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- 172.21.15.157:0/1420665886 shutdown_connections 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- 172.21.15.157:0/1420665886 wait complete. 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 Processor -- start 2023-11-06T03:05:03.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- start start 2023-11-06T03:05:03.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e680047d0 0x7f3e7c14bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e7c14c5c0 0x7f3e7c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14cb30 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14ce10 con 0x7f3e7c14c5c0 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e8455c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14d0f0 con 0x7f3e680047d0 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e82af9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e82af9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47448/0 (socket says 172.21.15.157:47448) 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e82af9640 1 -- 172.21.15.157:0/1502493717 learned_addr learned my addr 172.21.15.157:0/1502493717 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e822f8640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e680047d0 0x7f3e7c14bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.476+0000 7f3e81af7640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e7c14c5c0 0x7f3e7c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 -- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e680047d0 msgr2=0x7f3e7c14bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e680047d0 0x7f3e7c14bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 -- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e7c14c5c0 msgr2=0x7f3e7c15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e7c14c5c0 0x7f3e7c15f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 -- 172.21.15.157:0/1502493717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c14d3d0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e82af9640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0b49b0 tx=0x7f3e7400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74010040 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7400bb70 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c14d6b0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74014400 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c15fbc0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c072750 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3e74014560 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 0x7f3e60067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3e740836b0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e822f8640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 0x7f3e60067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.480+0000 7f3e822f8640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 0x7f3e60067f50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c004880 tx=0x7f3e6c001580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.484+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3e74050ed0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.644+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e7c0c9ab0 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.644+0000 7f3e72ffd640 1 -- 172.21.15.157:0/1502493717 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f3e74081e60 con 0x7f3e7c14dbd0 2023-11-06T03:05:03.646 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:03.646 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":25,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:04:48.517362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:04:05.099545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-06T03:04:05.662817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:03.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 msgr2=0x7f3e60067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 0x7f3e60067f50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c004880 tx=0x7f3e6c001580 comp rx=0 tx=0).stop 2023-11-06T03:05:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 msgr2=0x7f3e7c14e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0b49b0 tx=0x7f3e7400ef30 comp rx=0 tx=0).stop 2023-11-06T03:05:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 shutdown_connections 2023-11-06T03:05:03.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e60065a90 0x7f3e60067f50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e7c14dbd0 0x7f3e7c14e050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e7c14c5c0 0x7f3e7c15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 --2- 172.21.15.157:0/1502493717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e680047d0 0x7f3e7c14bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 >> 172.21.15.157:0/1502493717 conn(0x7f3e7c070790 msgr2=0x7f3e7c0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 shutdown_connections 2023-11-06T03:05:03.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.648+0000 7f3e8455c640 1 -- 172.21.15.157:0/1502493717 wait complete. 2023-11-06T03:05:03.665 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:05:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 Processor -- start 2023-11-06T03:05:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 -- start start 2023-11-06T03:05:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce7c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c15e440 con 0x7fce7c0b2af0 2023-11-06T03:05:03.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c06c240 con 0x7fce7c06d000 2023-11-06T03:05:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce83542640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c06c520 con 0x7fce7c1599c0 2023-11-06T03:05:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce81adf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fce7c0b2af0 0x7fce7c159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49862/0 (socket says 172.21.15.157:49862) 2023-11-06T03:05:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce81adf640 1 -- 172.21.15.157:0/1493562138 learned_addr learned my addr 172.21.15.157:0/1493562138 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce812de640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce7c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292077981 0 0) 0x7fce7c15e440 con 0x7fce7c0b2af0 2023-11-06T03:05:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce68003530 con 0x7fce7c0b2af0 2023-11-06T03:05:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252252978 0 0) 0x7fce68003530 con 0x7fce7c0b2af0 2023-11-06T03:05:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 msgr2=0x7fce7c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce7c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 >> v1:172.21.15.195:6789/0 conn(0x7fce7c06d000 legacy=0x7fce7c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c06c800 con 0x7fce7c0b2af0 2023-11-06T03:05:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fce74001a00 con 0x7fce7c0b2af0 2023-11-06T03:05:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 >> v1:172.21.15.157:6789/0 conn(0x7fce7c0b2af0 legacy=0x7fce7c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce680039b0 0x7fce68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce68004360 0x7fce68004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce68005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c06c520 con 0x7fce68004360 2023-11-06T03:05:03.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6800b990 con 0x7fce680039b0 2023-11-06T03:05:03.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6800bb50 con 0x7fce7c1599c0 2023-11-06T03:05:03.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce80add640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce68004360 0x7fce68004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce812de640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce680039b0 0x7fce68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.784+0000 7fce81adf640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce68005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 -- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 msgr2=0x7fce68005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce68005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 -- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce680039b0 msgr2=0x7fce68003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:03.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce680039b0 0x7fce68003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 -- 172.21.15.157:0/1493562138 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68003530 con 0x7fce68004360 2023-11-06T03:05:03.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce81adf640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1599c0 0x7fce68005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:03.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce80add640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce68004360 0x7fce68004830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce6c001860 tx=0x7fce6c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fcb91e9bc96793 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0ab5f0 con 0x7fce68004360 2023-11-06T03:05:03.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c0ab750 con 0x7fce68004360 2023-11-06T03:05:03.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce73fff640 1 -- 172.21.15.157:0/1493562138 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0abb00 con 0x7fce68004360 2023-11-06T03:05:03.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 -- 172.21.15.157:0/1493562138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce68004360 msgr2=0x7fce68004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 --2- 172.21.15.157:0/1493562138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce68004360 0x7fce68004830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce6c001860 tx=0x7fce6c01c4b0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce81adf640 1 -- 172.21.15.157:0/1493562138 reap_dead start 2023-11-06T03:05:03.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 -- 172.21.15.157:0/1493562138 shutdown_connections 2023-11-06T03:05:03.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 -- 172.21.15.157:0/1493562138 >> 172.21.15.157:0/1493562138 conn(0x7fce7c065930 msgr2=0x7fce7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:03.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 -- 172.21.15.157:0/1493562138 shutdown_connections 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.788+0000 7fce83542640 1 -- 172.21.15.157:0/1493562138 wait complete. 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 Processor -- start 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- start start 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 0x7fce7c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c0a89e0 0x7fce7c0a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0ce8d0 con 0x7fce7c154c70 2023-11-06T03:05:03.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c158bb0 con 0x7fce7c0a89e0 2023-11-06T03:05:03.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c157b10 con 0x7fce68004360 2023-11-06T03:05:03.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47468/0 (socket says 172.21.15.157:47468) 2023-11-06T03:05:03.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 -- 172.21.15.157:0/4274742384 learned_addr learned my addr 172.21.15.157:0/4274742384 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:03.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce81adf640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c0a89e0 0x7fce7c0a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce812de640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 0x7fce7c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 -- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 msgr2=0x7fce7c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 0x7fce7c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 -- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c0a89e0 msgr2=0x7fce7c0a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:03.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c0a89e0 0x7fce7c0a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:03.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 -- 172.21.15.157:0/4274742384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c157df0 con 0x7fce7c154c70 2023-11-06T03:05:03.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce812de640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 0x7fce7c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce80add640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0ad040 tx=0x7fce68005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0ab5f0 con 0x7fce7c154c70 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c005070 con 0x7fce7c154c70 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1580d0 con 0x7fce7c154c70 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c005400 con 0x7fce7c154c70 2023-11-06T03:05:03.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c158690 con 0x7fce7c154c70 2023-11-06T03:05:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c158850 con 0x7fce7c154c70 2023-11-06T03:05:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fce6c0bb070 con 0x7fce7c154c70 2023-11-06T03:05:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fce6c0a8050 con 0x7fce7c154c70 2023-11-06T03:05:03.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.792+0000 7fce812de640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:03.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.796+0000 7fce812de640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf0b0 tx=0x7fce78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:03.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.796+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fce6c0eb140 con 0x7fce7c154c70 2023-11-06T03:05:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:03.988+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fce7c072750 con 0x7fce7c154c70 2023-11-06T03:05:03.988 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:05:03.988+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:05:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.000+0000 7fce71ffb640 1 -- 172.21.15.157:0/4274742384 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fce6c0ee760 con 0x7fce7c154c70 2023-11-06T03:05:04.001 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:05:04.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 msgr2=0x7fce60067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:04.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0cf0b0 tx=0x7fce78008040 comp rx=0 tx=0).stop 2023-11-06T03:05:04.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 msgr2=0x7fce7c0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:04.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0ad040 tx=0x7fce68005800 comp rx=0 tx=0).stop 2023-11-06T03:05:04.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 shutdown_connections 2023-11-06T03:05:04.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c0a89e0 0x7fce7c0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c154c70 0x7fce7c0a84a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 --2- 172.21.15.157:0/4274742384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce68004360 0x7fce7c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 >> 172.21.15.157:0/4274742384 conn(0x7fce7c065930 msgr2=0x7fce7c15b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:04.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 shutdown_connections 2023-11-06T03:05:04.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.004+0000 7fce83542640 1 -- 172.21.15.157:0/4274742384 wait complete. 2023-11-06T03:05:04.012 INFO:tasks.ceph.mds.d.smithi195.stderr: -3231> 2023-11-06T03:04:08.414+0000 7f39abacc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:05:04.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:05:04.074 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:05:04.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 Processor -- start 2023-11-06T03:05:04.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 -- start start 2023-11-06T03:05:04.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933c1596a0 0x7f933c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c0ca1d0 0x7f933c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f933c06ca80 0x7f933c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c15d2c0 con 0x7f933c1596a0 2023-11-06T03:05:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c15e0e0 con 0x7f933c06ca80 2023-11-06T03:05:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933b59e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c15d4d0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933c1596a0 0x7f933c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f933c1596a0 0x7f933c0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47470/0 (socket says 172.21.15.157:47470) 2023-11-06T03:05:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 -- 172.21.15.157:0/1158039524 learned_addr learned my addr 172.21.15.157:0/1158039524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:04.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933ad9d640 1 --2- 172.21.15.157:0/1158039524 >> v2:172.21.15.195:3300/0 conn(0x7f933c06ca80 0x7f933c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:04.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 -- 172.21.15.157:0/1158039524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c0ca1d0 msgr2=0x7f933c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:04.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 --2- 172.21.15.157:0/1158039524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c0ca1d0 0x7f933c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 -- 172.21.15.157:0/1158039524 >> v2:172.21.15.195:3300/0 conn(0x7f933c06ca80 msgr2=0x7f933c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:04.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 --2- 172.21.15.157:0/1158039524 >> v2:172.21.15.195:3300/0 conn(0x7f933c06ca80 0x7f933c0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.136+0000 7f933a59c640 1 -- 172.21.15.157:0/1158039524 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c15d7b0 con 0x7f933c1596a0 2023-11-06T03:05:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933a59c640 1 --2- 172.21.15.157:0/1158039524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c1596a0 0x7f933c0c9c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f933c071310 tx=0x7f932c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a24c3a2bed59ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933959a640 1 -- 172.21.15.157:0/1158039524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0aa5f0 con 0x7f933c1596a0 2023-11-06T03:05:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933959a640 1 -- 172.21.15.157:0/1158039524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c0aa750 con 0x7f933c1596a0 2023-11-06T03:05:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933959a640 1 -- 172.21.15.157:0/1158039524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0aaae0 con 0x7f933c1596a0 2023-11-06T03:05:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- 172.21.15.157:0/1158039524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c1596a0 msgr2=0x7f933c0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- 172.21.15.157:0/1158039524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c1596a0 0x7f933c0c9c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f933c071310 tx=0x7f932c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- 172.21.15.157:0/1158039524 shutdown_connections 2023-11-06T03:05:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- 172.21.15.157:0/1158039524 >> v2:172.21.15.195:3300/0 conn(0x7f933c06ca80 0x7f933c0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- 172.21.15.157:0/1158039524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c0ca1d0 0x7f933c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- 172.21.15.157:0/1158039524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c1596a0 0x7f933c0c9c90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- 172.21.15.157:0/1158039524 >> 172.21.15.157:0/1158039524 conn(0x7f933c0cc150 msgr2=0x7f933c0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:04.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- 172.21.15.157:0/1158039524 shutdown_connections 2023-11-06T03:05:04.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- 172.21.15.157:0/1158039524 wait complete. 2023-11-06T03:05:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 Processor -- start 2023-11-06T03:05:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- start start 2023-11-06T03:05:04.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c06ca80 0x7f933c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f933c1596a0 0x7f933c13c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c143720 con 0x7f933c0ca1d0 2023-11-06T03:05:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0af8f0 con 0x7f933c1596a0 2023-11-06T03:05:04.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f933b59e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0afbd0 con 0x7f933c06ca80 2023-11-06T03:05:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f9339d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f9339d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47480/0 (socket says 172.21.15.157:47480) 2023-11-06T03:05:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.140+0000 7f9339d9b640 1 -- 172.21.15.157:0/2866771210 learned_addr learned my addr 172.21.15.157:0/2866771210 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 -- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c06ca80 msgr2=0x7f933c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:04.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c06ca80 0x7f933c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 -- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f933c1596a0 msgr2=0x7f933c13c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f933c1596a0 0x7f933c13c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 -- 172.21.15.157:0/2866771210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0094d0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f9339d9b640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f933000b480 tx=0x7f933000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330011830 con 0x7f933c0ca1d0 2023-11-06T03:05:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330011990 con 0x7f933c0ca1d0 2023-11-06T03:05:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0afeb0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330011d40 con 0x7f933c0ca1d0 2023-11-06T03:05:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c0b37c0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c0b3620 con 0x7f933c0ca1d0 2023-11-06T03:05:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f93300193b0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 0x7f931c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f93300831c0 con 0x7f933c0ca1d0 2023-11-06T03:05:04.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f933a59c640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 0x7f931c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:04.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.144+0000 7f933a59c640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 0x7f931c067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ae000 tx=0x7f932c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:04.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.148+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f933c0b3620 con 0x7f933c0ca1d0 2023-11-06T03:05:04.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:04.324+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f933c0b3620 con 0x7f933c0ca1d0 2023-11-06T03:05:05.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.004+0000 7f932affd640 1 -- 172.21.15.157:0/2866771210 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f933c0b3620 con 0x7f933c0ca1d0 2023-11-06T03:05:05.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 msgr2=0x7f931c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 0x7f931c067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ae000 tx=0x7f932c008040 comp rx=0 tx=0).stop 2023-11-06T03:05:05.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 msgr2=0x7f933c0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f933000b480 tx=0x7f933000b940 comp rx=0 tx=0).stop 2023-11-06T03:05:05.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 shutdown_connections 2023-11-06T03:05:05.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f931c065a60 0x7f931c067f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f933c1596a0 0x7f933c13c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f933c0ca1d0 0x7f933c0b0af0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 --2- 172.21.15.157:0/2866771210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f933c06ca80 0x7f933c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 >> 172.21.15.157:0/2866771210 conn(0x7f933c0cc150 msgr2=0x7f933c152340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:05.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 shutdown_connections 2023-11-06T03:05:05.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.008+0000 7f933b59e640 1 -- 172.21.15.157:0/2866771210 wait complete. 2023-11-06T03:05:05.029 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:05:05.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 Processor -- start 2023-11-06T03:05:05.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 -- start start 2023-11-06T03:05:05.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87f40b15b0 0x7f87f40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40d0410 con 0x7f87f415cc70 2023-11-06T03:05:05.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f40d27c0 con 0x7f87f40b15b0 2023-11-06T03:05:05.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87fa79c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40d0f90 con 0x7f87f40cfa90 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f8d39640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87f40b15b0 0x7f87f40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f8d39640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87f40b15b0 0x7f87f40b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:54892/0 (socket says 172.21.15.157:54892) 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:39286/0 (socket says 172.21.15.157:39286) 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f3fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f87f415cc70 0x7f87f40d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56498/0 (socket says 172.21.15.157:56498) 2023-11-06T03:05:05.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.156+0000 7f87f8d39640 1 -- 172.21.15.157:0/2082821743 learned_addr learned my addr 172.21.15.157:0/2082821743 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:05.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f8d39640 1 -- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 msgr2=0x7f87f40cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f8d39640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f8d39640 1 -- 172.21.15.157:0/2082821743 >> v1:172.21.15.195:6789/0 conn(0x7f87f415cc70 legacy=0x7f87f40d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f8d39640 1 -- 172.21.15.157:0/2082821743 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40d1270 con 0x7f87f40b15b0 2023-11-06T03:05:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f37fe640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:05:05.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f8d39640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b15b0 0x7f87f40b1a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009e50 tx=0x7f87ec01c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0430b836248b291 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:05.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f2ffd640 1 -- 172.21.15.157:0/2082821743 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0a57c0 con 0x7f87f40b15b0 2023-11-06T03:05:05.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f2ffd640 1 -- 172.21.15.157:0/2082821743 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec0a5920 con 0x7f87f40b15b0 2023-11-06T03:05:05.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87f2ffd640 1 -- 172.21.15.157:0/2082821743 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0a5cb0 con 0x7f87f40b15b0 2023-11-06T03:05:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 -- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b15b0 msgr2=0x7f87f40b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b15b0 0x7f87f40b1a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009e50 tx=0x7f87ec01c9e0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 -- 172.21.15.157:0/2082821743 shutdown_connections 2023-11-06T03:05:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40b15b0 0x7f87f40b1a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 --2- 172.21.15.157:0/2082821743 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f40cfa90 0x7f87f40cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 -- 172.21.15.157:0/2082821743 >> 172.21.15.157:0/2082821743 conn(0x7f87f4070790 msgr2=0x7f87f4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 -- 172.21.15.157:0/2082821743 shutdown_connections 2023-11-06T03:05:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.160+0000 7f87fa79c640 1 -- 172.21.15.157:0/2082821743 wait complete. 2023-11-06T03:05:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 Processor -- start 2023-11-06T03:05:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- start start 2023-11-06T03:05:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40cfa90 0x7f87f414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f415cc70 0x7f87f4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f41f5900 con 0x7f87f40b15b0 2023-11-06T03:05:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f414f7b0 con 0x7f87f40cfa90 2023-11-06T03:05:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f41f5610 con 0x7f87f415cc70 2023-11-06T03:05:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47496/0 (socket says 172.21.15.157:47496) 2023-11-06T03:05:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 -- 172.21.15.157:0/1895534747 learned_addr learned my addr 172.21.15.157:0/1895534747 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:05.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f8d39640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f415cc70 0x7f87f4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f37fe640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40cfa90 0x7f87f414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 -- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f415cc70 msgr2=0x7f87f4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f415cc70 0x7f87f4155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 -- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40cfa90 msgr2=0x7f87f414f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40cfa90 0x7f87f414f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 -- 172.21.15.157:0/1895534747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8009420 con 0x7f87f40b15b0 2023-11-06T03:05:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f3fff640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c370 tx=0x7f87e400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e40074e0 con 0x7f87f40b15b0 2023-11-06T03:05:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4007640 con 0x7f87f40b15b0 2023-11-06T03:05:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec009b30 con 0x7f87f40b15b0 2023-11-06T03:05:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004a30 con 0x7f87f40b15b0 2023-11-06T03:05:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f40c6ef0 con 0x7f87f40b15b0 2023-11-06T03:05:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.164+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f404e050 con 0x7f87f40b15b0 2023-11-06T03:05:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.168+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87e4004b90 con 0x7f87f40b15b0 2023-11-06T03:05:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.168+0000 7f87f0ff9640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 0x7f87e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.168+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f87e4082d90 con 0x7f87f40b15b0 2023-11-06T03:05:05.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.168+0000 7f87f37fe640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 0x7f87e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:05.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.168+0000 7f87f37fe640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 0x7f87e0067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87f40b2d30 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:05.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.172+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f87e4050560 con 0x7f87f40b15b0 2023-11-06T03:05:05.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:05.324+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87f4155a80 con 0x7f87f40b15b0 2023-11-06T03:05:06.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.016+0000 7f87f0ff9640 1 -- 172.21.15.157:0/1895534747 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f87e4053b80 con 0x7f87f40b15b0 2023-11-06T03:05:06.018 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:05:06.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.020+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 msgr2=0x7f87e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.020+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 0x7f87e0067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87f40b2d30 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2023-11-06T03:05:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.020+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 msgr2=0x7f87f40c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.020+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c370 tx=0x7f87e400c830 comp rx=0 tx=0).stop 2023-11-06T03:05:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 shutdown_connections 2023-11-06T03:05:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f87e0065a60 0x7f87e0067f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87f415cc70 0x7f87f4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87f40cfa90 0x7f87f414f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 --2- 172.21.15.157:0/1895534747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87f40b15b0 0x7f87f40c6630 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 >> 172.21.15.157:0/1895534747 conn(0x7f87f4070790 msgr2=0x7f87f40d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 shutdown_connections 2023-11-06T03:05:06.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.024+0000 7f87fa79c640 1 -- 172.21.15.157:0/1895534747 wait complete. 2023-11-06T03:05:06.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:05:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 Processor -- start 2023-11-06T03:05:06.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 -- start start 2023-11-06T03:05:06.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51d806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d8161930 con 0x7f51d80ce040 2023-11-06T03:05:06.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d806db70 con 0x7f51d80cdbe0 2023-11-06T03:05:06.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51e0262640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d806b010 con 0x7f51d806a6f0 2023-11-06T03:05:06.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51de7ff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f51d80ce040 0x7f51d8161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49874/0 (socket says 172.21.15.157:49874) 2023-11-06T03:05:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51de7ff640 1 -- 172.21.15.157:0/3873030904 learned_addr learned my addr 172.21.15.157:0/3873030904 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51d806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856403859 0 0) 0x7f51d8161930 con 0x7f51d80ce040 2023-11-06T03:05:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.168+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c4003530 con 0x7f51d80ce040 2023-11-06T03:05:06.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435949841 0 0) 0x7f51c4003530 con 0x7f51d80ce040 2023-11-06T03:05:06.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 msgr2=0x7f51d806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:06.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51d806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 >> v1:172.21.15.195:6789/0 conn(0x7f51d80cdbe0 legacy=0x7f51d8160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d806eeb0 con 0x7f51d80ce040 2023-11-06T03:05:06.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f51d0001a00 con 0x7f51d80ce040 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 >> v1:172.21.15.157:6789/0 conn(0x7f51d80ce040 legacy=0x7f51d8161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51c40039b0 0x7f51c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c40047d0 0x7f51c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d806b010 con 0x7f51c40047d0 2023-11-06T03:05:06.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400ba40 con 0x7f51c40039b0 2023-11-06T03:05:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400bc00 con 0x7f51d806a6f0 2023-11-06T03:05:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c40047d0 0x7f51c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51de7ff640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51c40039b0 0x7f51c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 -- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 msgr2=0x7f51c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51d806a6f0 0x7f51c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 -- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51c40039b0 msgr2=0x7f51c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51c40039b0 0x7f51c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 -- 172.21.15.157:0/3873030904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4003530 con 0x7f51c40047d0 2023-11-06T03:05:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51c40039b0 0x7f51c4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:06.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c40047d0 0x7f51c4004c50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001750 tx=0x7f51c801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f85c78711c76c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80052d0 con 0x7f51c40047d0 2023-11-06T03:05:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80a7050 con 0x7f51c40047d0 2023-11-06T03:05:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51dcffc640 1 -- 172.21.15.157:0/3873030904 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8003600 con 0x7f51c40047d0 2023-11-06T03:05:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 -- 172.21.15.157:0/3873030904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c40047d0 msgr2=0x7f51c4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 --2- 172.21.15.157:0/3873030904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51c40047d0 0x7f51c4004c50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001750 tx=0x7f51c801c3a0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51de7ff640 1 -- 172.21.15.157:0/3873030904 reap_dead start 2023-11-06T03:05:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 -- 172.21.15.157:0/3873030904 shutdown_connections 2023-11-06T03:05:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 -- 172.21.15.157:0/3873030904 >> 172.21.15.157:0/3873030904 conn(0x7f51d80d0950 msgr2=0x7f51d80d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 -- 172.21.15.157:0/3873030904 shutdown_connections 2023-11-06T03:05:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.172+0000 7f51e0262640 1 -- 172.21.15.157:0/3873030904 wait complete. 2023-11-06T03:05:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 Processor -- start 2023-11-06T03:05:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- start start 2023-11-06T03:05:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51c40047d0 0x7f51d81f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 0x7f51d80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80b0600 con 0x7f51d8160220 2023-11-06T03:05:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80b02b0 con 0x7f51d815fda0 2023-11-06T03:05:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81f2850 con 0x7f51c40047d0 2023-11-06T03:05:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47508/0 (socket says 172.21.15.157:47508) 2023-11-06T03:05:06.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 -- 172.21.15.157:0/3815181359 learned_addr learned my addr 172.21.15.157:0/3815181359 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:06.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51c40047d0 0x7f51d81f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 0x7f51d80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 -- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51c40047d0 msgr2=0x7f51d81f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51c40047d0 0x7f51d81f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 -- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 msgr2=0x7f51d80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 0x7f51d80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 -- 172.21.15.157:0/3815181359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80c95a0 con 0x7f51d8160220 2023-11-06T03:05:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51dd7fd640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 0x7f51d80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51de7ff640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51d8161bb0 tx=0x7f51d0011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000f8f0 con 0x7f51d8160220 2023-11-06T03:05:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d000fa50 con 0x7f51d8160220 2023-11-06T03:05:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d80c9880 con 0x7f51d8160220 2023-11-06T03:05:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d000fe00 con 0x7f51d8160220 2023-11-06T03:05:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d80ca290 con 0x7f51d8160220 2023-11-06T03:05:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.176+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d80c9d30 con 0x7f51d8160220 2023-11-06T03:05:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.180+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51d001f460 con 0x7f51d8160220 2023-11-06T03:05:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.180+0000 7f51ce7fc640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 0x7f51bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.180+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f51d0086060 con 0x7f51d8160220 2023-11-06T03:05:06.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.180+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 0x7f51bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.180+0000 7f51ddffe640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 0x7f51bc067f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009ac0 tx=0x7f51d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:06.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.184+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f51d0053a70 con 0x7f51d8160220 2023-11-06T03:05:06.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:06.340+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f51d80b0740 con 0x7f51d8160220 2023-11-06T03:05:07.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.020+0000 7f51ce7fc640 1 -- 172.21.15.157:0/3815181359 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f51d80b0740 con 0x7f51d8160220 2023-11-06T03:05:07.022 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 msgr2=0x7f51bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 0x7f51bc067f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009ac0 tx=0x7f51d4008040 comp rx=0 tx=0).stop 2023-11-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 msgr2=0x7f51d80b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f51d8161bb0 tx=0x7f51d0011c20 comp rx=0 tx=0).stop 2023-11-06T03:05:07.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 shutdown_connections 2023-11-06T03:05:07.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f51bc065a60 0x7f51bc067f20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51d8160220 0x7f51d80b2640 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51d815fda0 0x7f51d80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 --2- 172.21.15.157:0/3815181359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51c40047d0 0x7f51d81f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 >> 172.21.15.157:0/3815181359 conn(0x7f51d80d0950 msgr2=0x7f51d80cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 shutdown_connections 2023-11-06T03:05:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.036+0000 7f51e0262640 1 -- 172.21.15.157:0/3815181359 wait complete. 2023-11-06T03:05:07.060 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:05:07.060 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:05:07.061 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:05:07.061 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:05:07.062 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:05:07.062 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:05:07.063 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:05:07.063 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:05:07.063 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:05:07.064 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:05:07.064 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:05:07.064 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:05:07.065 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:05:07.065 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:05:07.066 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:05:07.066 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:05:07.072 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:05:07.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:05:07.078 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:05:07.078 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:05:07.080 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:05:07.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:05:07.081 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:05:07.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:05:07.123 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:05:07.124 DEBUG:teuthology.parallel:result is None 2023-11-06T03:05:07.125 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:05:07.125 DEBUG:teuthology.parallel:result is None 2023-11-06T03:05:07.128 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:05:07.129 DEBUG:teuthology.parallel:result is None 2023-11-06T03:05:07.130 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:05:07.130 DEBUG:teuthology.parallel:result is None 2023-11-06T03:05:07.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:05:07.191 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:05:07.192 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:05:07.195 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:05:07.195 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:05:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 Processor -- start 2023-11-06T03:05:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 -- start start 2023-11-06T03:05:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84400d09b0 0x7f84400d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f84400af930 0x7f844006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84400ad880 con 0x7f84400cf080 2023-11-06T03:05:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400adb60 con 0x7f84400d09b0 2023-11-06T03:05:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.264+0000 7f8444b07640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400ade40 con 0x7f84400af930 2023-11-06T03:05:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84400d09b0 0x7f84400d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84400cf080 0x7f844006d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49886/0 (socket says 172.21.15.157:49886) 2023-11-06T03:05:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843dd9b640 1 -- 172.21.15.157:0/3313371507 learned_addr learned my addr 172.21.15.157:0/3313371507 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843ed9d640 1 --2- 172.21.15.157:0/3313371507 >> v2:172.21.15.195:3300/0 conn(0x7f84400af930 0x7f844006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 -- 172.21.15.157:0/3313371507 >> v2:172.21.15.195:3300/0 conn(0x7f84400af930 msgr2=0x7f844006ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 --2- 172.21.15.157:0/3313371507 >> v2:172.21.15.195:3300/0 conn(0x7f84400af930 0x7f844006ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 -- 172.21.15.157:0/3313371507 >> v1:172.21.15.157:6789/0 conn(0x7f84400cf080 legacy=0x7f844006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 -- 172.21.15.157:0/3313371507 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400ae120 con 0x7f84400d09b0 2023-11-06T03:05:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843d59a640 1 -- 172.21.15.157:0/3313371507 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954369946 0 0) 0x7f84400ad880 con 0x7f84400cf080 2023-11-06T03:05:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843e59c640 1 --2- 172.21.15.157:0/3313371507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400d09b0 0x7f84400d0d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84400ce590 tx=0x7f843401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f264d869abedfa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843d59a640 1 -- 172.21.15.157:0/3313371507 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340ab600 con 0x7f84400d09b0 2023-11-06T03:05:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843d59a640 1 -- 172.21.15.157:0/3313371507 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340a7050 con 0x7f84400d09b0 2023-11-06T03:05:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f843d59a640 1 -- 172.21.15.157:0/3313371507 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340ab980 con 0x7f84400d09b0 2023-11-06T03:05:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 -- 172.21.15.157:0/3313371507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400d09b0 msgr2=0x7f84400d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 --2- 172.21.15.157:0/3313371507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400d09b0 0x7f84400d0d90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f84400ce590 tx=0x7f843401c380 comp rx=0 tx=0).stop 2023-11-06T03:05:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 -- 172.21.15.157:0/3313371507 shutdown_connections 2023-11-06T03:05:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 --2- 172.21.15.157:0/3313371507 >> v2:172.21.15.195:3300/0 conn(0x7f84400af930 0x7f844006ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 --2- 172.21.15.157:0/3313371507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400d09b0 0x7f84400d0d90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 -- 172.21.15.157:0/3313371507 >> 172.21.15.157:0/3313371507 conn(0x7f8440071b90 msgr2=0x7f8440099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 -- 172.21.15.157:0/3313371507 shutdown_connections 2023-11-06T03:05:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.268+0000 7f8444b07640 1 -- 172.21.15.157:0/3313371507 wait complete. 2023-11-06T03:05:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 Processor -- start 2023-11-06T03:05:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- start start 2023-11-06T03:05:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84400cf080 0x7f84400c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84400d09b0 0x7f8440152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400c7380 con 0x7f84400af930 2023-11-06T03:05:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440157f50 con 0x7f84400d09b0 2023-11-06T03:05:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8440157c20 con 0x7f84400cf080 2023-11-06T03:05:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47566/0 (socket says 172.21.15.157:47566) 2023-11-06T03:05:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 -- 172.21.15.157:0/1641746875 learned_addr learned my addr 172.21.15.157:0/1641746875 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843dd9b640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84400cf080 0x7f84400c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843ed9d640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84400d09b0 0x7f8440152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 -- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84400cf080 msgr2=0x7f84400c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84400cf080 0x7f84400c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 -- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84400d09b0 msgr2=0x7f8440152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84400d09b0 0x7f8440152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 -- 172.21.15.157:0/1641746875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340093b0 con 0x7f84400af930 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f843e59c640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f84340ae040 tx=0x7f8434019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434005ba0 con 0x7f84400af930 2023-11-06T03:05:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434005d00 con 0x7f84400af930 2023-11-06T03:05:07.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434006260 con 0x7f84400af930 2023-11-06T03:05:07.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400b0370 con 0x7f84400af930 2023-11-06T03:05:07.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440153d90 con 0x7f84400af930 2023-11-06T03:05:07.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.272+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440072750 con 0x7f84400af930 2023-11-06T03:05:07.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.276+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84340b2040 con 0x7f84400af930 2023-11-06T03:05:07.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.276+0000 7f842effd640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 0x7f8420067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:07.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.276+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f843411cd70 con 0x7f84400af930 2023-11-06T03:05:07.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.276+0000 7f843dd9b640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 0x7f8420067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:07.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.276+0000 7f843dd9b640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 0x7f8420067f20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f844015a010 tx=0x7f8430005ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:07.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.280+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84340ea960 con 0x7f84400af930 2023-11-06T03:05:07.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:07.444+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f84400cdbb0 con 0x7f84400af930 2023-11-06T03:05:08.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.240+0000 7f842effd640 1 -- 172.21.15.157:0/1641746875 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f84340c1080 con 0x7f84400af930 2023-11-06T03:05:08.241 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:05:08.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 msgr2=0x7f8420067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 0x7f8420067f20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f844015a010 tx=0x7f8430005ec0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 msgr2=0x7f8440153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f84340ae040 tx=0x7f8434019120 comp rx=0 tx=0).stop 2023-11-06T03:05:08.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 shutdown_connections 2023-11-06T03:05:08.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8420065a60 0x7f8420067f20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84400d09b0 0x7f8440152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84400cf080 0x7f84400c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 --2- 172.21.15.157:0/1641746875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84400af930 0x7f8440153440 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 >> 172.21.15.157:0/1641746875 conn(0x7f8440071b90 msgr2=0x7f84400cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:08.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 shutdown_connections 2023-11-06T03:05:08.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.244+0000 7f8444b07640 1 -- 172.21.15.157:0/1641746875 wait complete. 2023-11-06T03:05:08.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:05:08.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.384+0000 7f2eebc5e640 1 Processor -- start 2023-11-06T03:05:08.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.384+0000 7f2eebc5e640 1 -- start start 2023-11-06T03:05:08.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eebc5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 0x7f2ee4167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eebc5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40d2920 con 0x7f2ee415ce80 2023-11-06T03:05:08.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eebc5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40d18a0 con 0x7f2ee415caa0 2023-11-06T03:05:08.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eebc5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40d1b60 con 0x7f2ee40d0280 2023-11-06T03:05:08.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee91f9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2ee415ce80 0x7f2ee40cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49894/0 (socket says 172.21.15.157:49894) 2023-11-06T03:05:08.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee91f9640 1 -- 172.21.15.157:0/1674702633 learned_addr learned my addr 172.21.15.157:0/1674702633 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:08.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 0x7f2ee4167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000004214 0 0) 0x7f2ee40d2920 con 0x7f2ee415ce80 2023-11-06T03:05:08.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed0003530 con 0x7f2ee415ce80 2023-11-06T03:05:08.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3614781134 0 0) 0x7f2ed0003530 con 0x7f2ee415ce80 2023-11-06T03:05:08.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 msgr2=0x7f2ee4167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:08.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 0x7f2ee4167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 >> v1:172.21.15.195:6789/0 conn(0x7f2ee415caa0 legacy=0x7f2ee40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40d1e40 con 0x7f2ee415ce80 2023-11-06T03:05:08.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2ee0001680 con 0x7f2ee415ce80 2023-11-06T03:05:08.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 >> v1:172.21.15.157:6789/0 conn(0x7f2ee415ce80 legacy=0x7f2ee40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 0x7f2ed00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed0003f30 0x7f2ed00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed0004d10 0x7f2ed0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40d1b60 con 0x7f2ed0004d10 2023-11-06T03:05:08.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed000ba70 con 0x7f2ed0003f30 2023-11-06T03:05:08.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed000bc30 con 0x7f2ee40d0280 2023-11-06T03:05:08.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed0004d10 0x7f2ed0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2ee91f9640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed0003f30 0x7f2ed00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 -- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 msgr2=0x7f2ed00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee40d0280 0x7f2ed00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 -- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed0003f30 msgr2=0x7f2ed00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ed0003f30 0x7f2ed00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.388+0000 7f2eea1fb640 1 -- 172.21.15.157:0/1674702633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0003530 con 0x7f2ed0004d10 2023-11-06T03:05:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed0004d10 0x7f2ed0005190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fc0 tx=0x7f2edc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7e00696af97c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0a5cb0 con 0x7f2ed0004d10 2023-11-06T03:05:08.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc00f040 con 0x7f2ed0004d10 2023-11-06T03:05:08.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee89f8640 1 -- 172.21.15.157:0/1674702633 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0ad680 con 0x7f2ed0004d10 2023-11-06T03:05:08.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- 172.21.15.157:0/1674702633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed0004d10 msgr2=0x7f2ed0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/1674702633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ed0004d10 0x7f2ed0005190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fc0 tx=0x7f2edc01cca0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eea1fb640 1 -- 172.21.15.157:0/1674702633 reap_dead start 2023-11-06T03:05:08.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- 172.21.15.157:0/1674702633 shutdown_connections 2023-11-06T03:05:08.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- 172.21.15.157:0/1674702633 >> 172.21.15.157:0/1674702633 conn(0x7f2ee4070790 msgr2=0x7f2ee4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:08.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- 172.21.15.157:0/1674702633 shutdown_connections 2023-11-06T03:05:08.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- 172.21.15.157:0/1674702633 wait complete. 2023-11-06T03:05:08.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 Processor -- start 2023-11-06T03:05:08.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- start start 2023-11-06T03:05:08.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee415b3e0 0x7f2ee40cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 0x7f2ee40b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ccbe0 con 0x7f2ee415b7c0 2023-11-06T03:05:08.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cb440 con 0x7f2ee41f21a0 2023-11-06T03:05:08.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eebc5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40cb720 con 0x7f2ee415b3e0 2023-11-06T03:05:08.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47590/0 (socket says 172.21.15.157:47590) 2023-11-06T03:05:08.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 -- 172.21.15.157:0/595232950 learned_addr learned my addr 172.21.15.157:0/595232950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:08.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee99fa640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee415b3e0 0x7f2ee40cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 0x7f2ee40b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 -- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee415b3e0 msgr2=0x7f2ee40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee415b3e0 0x7f2ee40cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 -- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 msgr2=0x7f2ee40b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 0x7f2ee40b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2ee91f9640 1 -- 172.21.15.157:0/595232950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40cba00 con 0x7f2ee415b7c0 2023-11-06T03:05:08.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.392+0000 7f2eea1fb640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 0x7f2ee40b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:08.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2ee91f9640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000eb20 tx=0x7f2ee00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000c850 con 0x7f2ee415b7c0 2023-11-06T03:05:08.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee000c9d0 con 0x7f2ee415b7c0 2023-11-06T03:05:08.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40a95b0 con 0x7f2ee415b7c0 2023-11-06T03:05:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000cd80 con 0x7f2ee415b7c0 2023-11-06T03:05:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee41f2620 con 0x7f2ee415b7c0 2023-11-06T03:05:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee40a97d0 con 0x7f2ee415b7c0 2023-11-06T03:05:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ee001d460 con 0x7f2ee415b7c0 2023-11-06T03:05:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 0x7f2ec8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2ee0083020 con 0x7f2ee415b7c0 2023-11-06T03:05:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2ee99fa640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 0x7f2ec8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.396+0000 7f2ee99fa640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 0x7f2ec8067f20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40016b0 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.400+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ee0050c60 con 0x7f2ee415b7c0 2023-11-06T03:05:08.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.608+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2ee4072750 con 0x7f2ee415b7c0 2023-11-06T03:05:08.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.608+0000 7f2eda7fc640 1 -- 172.21.15.157:0/595232950 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2ee0054280 con 0x7f2ee415b7c0 2023-11-06T03:05:08.611 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:08.611 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:05:08.611 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:05:08.612 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.612 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:05:08.612 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:05:08.612 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.612 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.613 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:05:08.613 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:05:08.613 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.613 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.614 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.614 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.614 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:05:08.614 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:05:08.614 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.615 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.616 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.616 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:05:08.616 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:05:08.616 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.616 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.617 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:05:08.617 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:05:08.617 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.617 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.617 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.618 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.618 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:05:08.618 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:05:08.618 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.618 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.619 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:05:08.619 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:05:08.619 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.620 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.620 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.620 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.621 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:05:08.621 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:05:08.621 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.622 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.622 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.623 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.623 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.623 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.624 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.624 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:05:08.624 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:05:08.625 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.625 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.625 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.626 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.626 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.627 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.627 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.627 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:05:08.628 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:05:08.628 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.628 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.629 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.629 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.629 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.630 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.630 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.631 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:05:08.631 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:05:08.631 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.631 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.632 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.632 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.632 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.633 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.633 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.634 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:05:08.634 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:05:08.634 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.635 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.635 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.635 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.636 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.636 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.636 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.637 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:05:08.637 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:05:08.637 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.638 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.638 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.639 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.639 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.639 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.640 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.640 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:05:08.640 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:05:08.641 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.641 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.641 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.642 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.642 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.643 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.643 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.643 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:05:08.644 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:05:08.644 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.644 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:05:08.645 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:05:08.645 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.645 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.646 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.646 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:05:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:05:08.647 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.648 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.648 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:05:08.648 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:05:08.649 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:05:08.649 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.650 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.650 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:05:08.651 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:05:08.651 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.651 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.652 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:05:08.652 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:05:08.652 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:05:08.653 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.653 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.653 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.653 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:05:08.654 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:05:08.654 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.654 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.655 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:05:08.655 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:05:08.655 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:05:08.656 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.656 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.656 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:05:08.657 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:05:08.657 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.657 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:05:08.657 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:05:08.657 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:05:08.658 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:05:08.658 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.658 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.658 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:05:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:05:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:05:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:05:08.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-mds", 2023-11-06T03:05:08.660 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hlIT9BHBAAiOEqQE50lpbg6JRTM9PkYw==", 2023-11-06T03:05:08.661 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.661 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-mds" 2023-11-06T03:05:08.661 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.661 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.662 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.662 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-mgr", 2023-11-06T03:05:08.662 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hla1lBHBAAJ/CoX0t7Ey6M/gcrcCVKow==", 2023-11-06T03:05:08.662 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.662 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-06T03:05:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-osd", 2023-11-06T03:05:08.663 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hlUW9BHBAA2ASupMVm1BRbQCyztDlsIQ==", 2023-11-06T03:05:08.664 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.664 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-osd" 2023-11-06T03:05:08.664 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.664 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.664 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-rbd", 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hlNIRBHBAAPkstv8NS0AVp/C7FXyZo3Q==", 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.665 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.666 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.666 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-06T03:05:08.666 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hl2JlBHBAAg8Kl6H7h7BToogQHReS+Cw==", 2023-11-06T03:05:08.666 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.666 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-06T03:05:08.667 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.667 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.667 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.667 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.bootstrap-rgw", 2023-11-06T03:05:08.667 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCOV0hlILFBHBAAvHUu7p1wNnLGq9nJkV2Wfg==", 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.668 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:05:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:05:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:05:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:05:08.669 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:05:08.670 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:05:08.671 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:05:08.671 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:05:08.671 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:05:08.671 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:05:08.672 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.672 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:08.672 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:05:08.672 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:05:08.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 msgr2=0x7f2ec8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 0x7f2ec8067f20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40016b0 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 msgr2=0x7f2ee40cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000eb20 tx=0x7f2ee00016f0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 shutdown_connections 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ec8065a60 0x7f2ec8067f20 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40016b0 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ee41f21a0 0x7f2ee40b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ee415b7c0 0x7f2ee40cc6a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 --2- 172.21.15.157:0/595232950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ee415b3e0 0x7f2ee40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 >> 172.21.15.157:0/595232950 conn(0x7f2ee4070790 msgr2=0x7f2ee4066080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:08.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 shutdown_connections 2023-11-06T03:05:08.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.612+0000 7f2eebc5e640 1 -- 172.21.15.157:0/595232950 wait complete. 2023-11-06T03:05:08.677 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-06T03:05:08.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 Processor -- start 2023-11-06T03:05:08.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 -- start start 2023-11-06T03:05:08.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f24a41551e0 0x7f24a406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24a40b3b20 0x7f24a415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a415f580 con 0x7f24a415e5e0 2023-11-06T03:05:08.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a415efc0 con 0x7f24a40b3b20 2023-11-06T03:05:08.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a9032640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a415f2a0 con 0x7f24a41551e0 2023-11-06T03:05:08.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24a40b3b20 0x7f24a415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24a40b3b20 0x7f24a415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55852/0 (socket says 172.21.15.157:55852) 2023-11-06T03:05:08.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 -- 172.21.15.157:0/2514845859 learned_addr learned my addr 172.21.15.157:0/2514845859 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:08.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a2d9d640 1 --2- 172.21.15.157:0/2514845859 >> v2:172.21.15.195:3300/0 conn(0x7f24a41551e0 0x7f24a406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 -- 172.21.15.157:0/2514845859 >> v2:172.21.15.195:3300/0 conn(0x7f24a41551e0 msgr2=0x7f24a406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:08.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 --2- 172.21.15.157:0/2514845859 >> v2:172.21.15.195:3300/0 conn(0x7f24a41551e0 0x7f24a406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 -- 172.21.15.157:0/2514845859 >> v1:172.21.15.195:6789/0 conn(0x7f24a415e5e0 legacy=0x7f24a40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:08.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.800+0000 7f24a359e640 1 -- 172.21.15.157:0/2514845859 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a406f5a0 con 0x7f24a40b3b20 2023-11-06T03:05:08.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a359e640 1 --2- 172.21.15.157:0/2514845859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a415c1b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24a4066830 tx=0x7f249c01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c67a8ae417eca4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a1d9b640 1 -- 172.21.15.157:0/2514845859 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5720 con 0x7f24a40b3b20 2023-11-06T03:05:08.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a1d9b640 1 -- 172.21.15.157:0/2514845859 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c0a5880 con 0x7f24a40b3b20 2023-11-06T03:05:08.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a1d9b640 1 -- 172.21.15.157:0/2514845859 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5c10 con 0x7f24a40b3b20 2023-11-06T03:05:08.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- 172.21.15.157:0/2514845859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 msgr2=0x7f24a415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- 172.21.15.157:0/2514845859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a415c1b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24a4066830 tx=0x7f249c01c840 comp rx=0 tx=0).stop 2023-11-06T03:05:08.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- 172.21.15.157:0/2514845859 shutdown_connections 2023-11-06T03:05:08.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- 172.21.15.157:0/2514845859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a415c1b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- 172.21.15.157:0/2514845859 >> v2:172.21.15.195:3300/0 conn(0x7f24a41551e0 0x7f24a406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- 172.21.15.157:0/2514845859 >> 172.21.15.157:0/2514845859 conn(0x7f24a4065930 msgr2=0x7f24a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:08.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- 172.21.15.157:0/2514845859 shutdown_connections 2023-11-06T03:05:08.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- 172.21.15.157:0/2514845859 wait complete. 2023-11-06T03:05:08.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 Processor -- start 2023-11-06T03:05:08.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 -- start start 2023-11-06T03:05:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 0x7f24a41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.804+0000 7f24a9032640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24a415e5e0 0x7f24a414e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a406b630 con 0x7f24a40b3b20 2023-11-06T03:05:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415a800 con 0x7f24a41551e0 2023-11-06T03:05:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415a240 con 0x7f24a415e5e0 2023-11-06T03:05:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55862/0 (socket says 172.21.15.157:55862) 2023-11-06T03:05:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 -- 172.21.15.157:0/2730102083 learned_addr learned my addr 172.21.15.157:0/2730102083 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a259c640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 0x7f24a41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a359e640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24a415e5e0 0x7f24a414e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 -- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24a415e5e0 msgr2=0x7f24a414e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24a415e5e0 0x7f24a414e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 -- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 msgr2=0x7f24a41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:08.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 0x7f24a41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:08.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 -- 172.21.15.157:0/2730102083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c009b30 con 0x7f24a40b3b20 2023-11-06T03:05:08.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a259c640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 0x7f24a41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:08.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a2d9d640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f249400c820 tx=0x7f249400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494004220 con 0x7f24a40b3b20 2023-11-06T03:05:08.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24940043a0 con 0x7f24a40b3b20 2023-11-06T03:05:08.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a415a520 con 0x7f24a40b3b20 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494004510 con 0x7f24a40b3b20 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a414c8e0 con 0x7f24a40b3b20 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a4072750 con 0x7f24a40b3b20 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2494007a70 con 0x7f24a40b3b20 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 0x7f2490067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:08.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2494082d10 con 0x7f24a40b3b20 2023-11-06T03:05:08.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a259c640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 0x7f2490067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:08.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.808+0000 7f24a259c640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 0x7f2490067f20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24a414bf60 tx=0x7f2498008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:08.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:08.812+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2494050950 con 0x7f24a40b3b20 2023-11-06T03:05:09.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.024+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f24a4071730 con 0x7f24a40b3b20 2023-11-06T03:05:09.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.032+0000 7f248b7fe640 1 -- 172.21.15.157:0/2730102083 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f249401e020 con 0x7f24a40b3b20 2023-11-06T03:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 msgr2=0x7f2490067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 0x7f2490067f20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24a414bf60 tx=0x7f2498008040 comp rx=0 tx=0).stop 2023-11-06T03:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 msgr2=0x7f24a414c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f249400c820 tx=0x7f249400cce0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 shutdown_connections 2023-11-06T03:05:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2490065a60 0x7f2490067f20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24a415e5e0 0x7f24a414e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24a41551e0 0x7f24a41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 --2- 172.21.15.157:0/2730102083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24a40b3b20 0x7f24a414c300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 >> 172.21.15.157:0/2730102083 conn(0x7f24a4065930 msgr2=0x7f24a40aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 shutdown_connections 2023-11-06T03:05:09.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.036+0000 7f24a9032640 1 -- 172.21.15.157:0/2730102083 wait complete. 2023-11-06T03:05:09.054 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-06T03:05:09.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.176+0000 7f456e6e2640 1 Processor -- start 2023-11-06T03:05:09.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.176+0000 7f456e6e2640 1 -- start start 2023-11-06T03:05:09.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456e6e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456e6e2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 0x7f45680b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456e6e2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45680cc2b0 con 0x7f45680afb00 2023-11-06T03:05:09.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456e6e2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45680b01a0 con 0x7f45680cf560 2023-11-06T03:05:09.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456e6e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45680cfb10 con 0x7f456815ac80 2023-11-06T03:05:09.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456cc7f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 0x7f45680b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43608/0 (socket says 172.21.15.157:43608) 2023-11-06T03:05:09.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f455f7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f45680afb00 0x7f45680cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:47466/0 (socket says 172.21.15.157:47466) 2023-11-06T03:05:09.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 -- 172.21.15.157:0/4148290671 learned_addr learned my addr 172.21.15.157:0/4148290671 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:09.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 -- 172.21.15.157:0/4148290671 >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 msgr2=0x7f45680b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:09.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 --2- 172.21.15.157:0/4148290671 >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 0x7f45680b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 -- 172.21.15.157:0/4148290671 >> v1:172.21.15.195:6789/0 conn(0x7f45680afb00 legacy=0x7f45680cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f4567fff640 1 -- 172.21.15.157:0/4148290671 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680cfdf0 con 0x7f456815ac80 2023-11-06T03:05:09.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.180+0000 7f456cc7f640 1 --2- 172.21.15.157:0/4148290671 >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 0x7f45680b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:05:09.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 --2- 172.21.15.157:0/4148290671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4558009870 tx=0x7f455801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87a283eb5ea5792 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f45677fe640 1 -- 172.21.15.157:0/4148290671 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580ab650 con 0x7f456815ac80 2023-11-06T03:05:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f45677fe640 1 -- 172.21.15.157:0/4148290671 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45580ab7b0 con 0x7f456815ac80 2023-11-06T03:05:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f45677fe640 1 -- 172.21.15.157:0/4148290671 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580abb40 con 0x7f456815ac80 2023-11-06T03:05:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- 172.21.15.157:0/4148290671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 msgr2=0x7f45680b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- 172.21.15.157:0/4148290671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4558009870 tx=0x7f455801c540 comp rx=0 tx=0).stop 2023-11-06T03:05:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- 172.21.15.157:0/4148290671 shutdown_connections 2023-11-06T03:05:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- 172.21.15.157:0/4148290671 >> v2:172.21.15.195:3300/0 conn(0x7f45680cf560 0x7f45680b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- 172.21.15.157:0/4148290671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f456815ac80 0x7f45680b2400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- 172.21.15.157:0/4148290671 >> 172.21.15.157:0/4148290671 conn(0x7f456806d560 msgr2=0x7f456806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- 172.21.15.157:0/4148290671 shutdown_connections 2023-11-06T03:05:09.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- 172.21.15.157:0/4148290671 wait complete. 2023-11-06T03:05:09.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 Processor -- start 2023-11-06T03:05:09.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- start start 2023-11-06T03:05:09.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45680cf560 0x7f45681484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f456815ac80 0x7f45680b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680c5110 con 0x7f45680afb00 2023-11-06T03:05:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4568157eb0 con 0x7f456815ac80 2023-11-06T03:05:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456e6e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4568158190 con 0x7f45680cf560 2023-11-06T03:05:09.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f456cc7f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f456815ac80 0x7f45680b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55868/0 (socket says 172.21.15.157:55868) 2023-11-06T03:05:09.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 -- 172.21.15.157:0/1616603389 learned_addr learned my addr 172.21.15.157:0/1616603389 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f455f7fe640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45680cf560 0x7f45681484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 -- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45680cf560 msgr2=0x7f45681484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45680cf560 0x7f45681484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.184+0000 7f4567fff640 1 -- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f456815ac80 msgr2=0x7f45680b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f4567fff640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f456815ac80 0x7f45680b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f4567fff640 1 -- 172.21.15.157:0/1616603389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4560009a70 con 0x7f45680afb00 2023-11-06T03:05:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f4567fff640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f45580ad040 tx=0x7f455801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580b4800 con 0x7f45680afb00 2023-11-06T03:05:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45580b4980 con 0x7f45680afb00 2023-11-06T03:05:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45580094d0 con 0x7f45680afb00 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580b4d30 con 0x7f45680afb00 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4568158750 con 0x7f45680afb00 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4568072750 con 0x7f45680afb00 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45580c7390 con 0x7f45680afb00 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 0x7f4548067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f455811d830 con 0x7f45680afb00 2023-11-06T03:05:09.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f455f7fe640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 0x7f4548067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.188+0000 7f455f7fe640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 0x7f4548067f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f456815a1f0 tx=0x7f4550007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.192+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45580eb5b0 con 0x7f45680afb00 2023-11-06T03:05:09.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.388+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f456806e3e0 con 0x7f45680afb00 2023-11-06T03:05:09.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f45657fa640 1 -- 172.21.15.157:0/1616603389 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f45580eebd0 con 0x7f45680afb00 2023-11-06T03:05:09.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 msgr2=0x7f4548067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 0x7f4548067f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f456815a1f0 tx=0x7f4550007e50 comp rx=0 tx=0).stop 2023-11-06T03:05:09.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 msgr2=0x7f45680b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f45580ad040 tx=0x7f455801ad80 comp rx=0 tx=0).stop 2023-11-06T03:05:09.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 shutdown_connections 2023-11-06T03:05:09.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4548065a60 0x7f4548067f20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f456815ac80 0x7f45680b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45680cf560 0x7f45681484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 --2- 172.21.15.157:0/1616603389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45680afb00 0x7f45680b0640 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 >> 172.21.15.157:0/1616603389 conn(0x7f456806d560 msgr2=0x7f4568153ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 shutdown_connections 2023-11-06T03:05:09.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.396+0000 7f456e6e2640 1 -- 172.21.15.157:0/1616603389 wait complete. 2023-11-06T03:05:09.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-06T03:05:09.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 Processor -- start 2023-11-06T03:05:09.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 -- start start 2023-11-06T03:05:09.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25dc1599c0 0x7f25dc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc06d000 0x7f25dc0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc15e440 con 0x7f25dc15dea0 2023-11-06T03:05:09.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25dc15adb0 con 0x7f25dc1599c0 2023-11-06T03:05:09.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25e27d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc06c240 con 0x7f25dc06d000 2023-11-06T03:05:09.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25db7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc06d000 0x7f25dc0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25dbfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25dc1599c0 0x7f25dc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25dbfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25dc1599c0 0x7f25dc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:56690/0 (socket says 172.21.15.157:56690) 2023-11-06T03:05:09.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.532+0000 7f25dbfff640 1 -- 172.21.15.157:0/3547216053 learned_addr learned my addr 172.21.15.157:0/3547216053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:09.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25dbfff640 1 -- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc06d000 msgr2=0x7f25dc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:09.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25dbfff640 1 --2- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc06d000 0x7f25dc0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25dbfff640 1 -- 172.21.15.157:0/3547216053 >> v1:172.21.15.195:6789/0 conn(0x7f25dc15dea0 legacy=0x7f25dc070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25dbfff640 1 -- 172.21.15.157:0/3547216053 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc06c520 con 0x7f25dc1599c0 2023-11-06T03:05:09.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25dbfff640 1 --2- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc1599c0 0x7f25dc0ae000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bb10 tx=0x7f25cc01ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24c0c3f34432f69 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25daffd640 1 -- 172.21.15.157:0/3547216053 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0a7bc0 con 0x7f25dc1599c0 2023-11-06T03:05:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25daffd640 1 -- 172.21.15.157:0/3547216053 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0a7d20 con 0x7f25dc1599c0 2023-11-06T03:05:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25daffd640 1 -- 172.21.15.157:0/3547216053 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0af660 con 0x7f25dc1599c0 2023-11-06T03:05:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 -- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc1599c0 msgr2=0x7f25dc0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc1599c0 0x7f25dc0ae000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bb10 tx=0x7f25cc01ebe0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 -- 172.21.15.157:0/3547216053 shutdown_connections 2023-11-06T03:05:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc06d000 0x7f25dc0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3547216053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc1599c0 0x7f25dc0ae000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 -- 172.21.15.157:0/3547216053 >> 172.21.15.157:0/3547216053 conn(0x7f25dc065930 msgr2=0x7f25dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 -- 172.21.15.157:0/3547216053 shutdown_connections 2023-11-06T03:05:09.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.536+0000 7f25e27d3640 1 -- 172.21.15.157:0/3547216053 wait complete. 2023-11-06T03:05:09.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 Processor -- start 2023-11-06T03:05:09.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- start start 2023-11-06T03:05:09.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc1599c0 0x7f25dc1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 0x7f25dc1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1f6270 con 0x7f25dc06d000 2023-11-06T03:05:09.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc155d30 con 0x7f25dc15dea0 2023-11-06T03:05:09.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc151190 con 0x7f25dc1599c0 2023-11-06T03:05:09.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55870/0 (socket says 172.21.15.157:55870) 2023-11-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 -- 172.21.15.157:0/3291606900 learned_addr learned my addr 172.21.15.157:0/3291606900 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e0d70640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 0x7f25dc1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25db7fe640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc1599c0 0x7f25dc1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 -- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc1599c0 msgr2=0x7f25dc1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc1599c0 0x7f25dc1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 -- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 msgr2=0x7f25dc1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 0x7f25dc1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 -- 172.21.15.157:0/3291606900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc00b770 con 0x7f25dc06d000 2023-11-06T03:05:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e0d70640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 0x7f25dc1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25dbfff640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00b2c0 tx=0x7f25cc01b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0be070 con 0x7f25dc06d000 2023-11-06T03:05:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0a7e90 con 0x7f25dc06d000 2023-11-06T03:05:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc155f70 con 0x7f25dc06d000 2023-11-06T03:05:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0afdb0 con 0x7f25dc06d000 2023-11-06T03:05:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc150b40 con 0x7f25dc06d000 2023-11-06T03:05:09.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.540+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc072750 con 0x7f25dc06d000 2023-11-06T03:05:09.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.544+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f25cc0b6070 con 0x7f25dc06d000 2023-11-06T03:05:09.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.544+0000 7f25d8ff9640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 0x7f25c8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.544+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f25cc11d740 con 0x7f25dc06d000 2023-11-06T03:05:09.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.544+0000 7f25db7fe640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 0x7f25c8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.544+0000 7f25db7fe640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 0x7f25c8067f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25dc14d400 tx=0x7f25d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.548+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f25cc0eb380 con 0x7f25dc06d000 2023-11-06T03:05:09.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:05:09.571 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:05:09.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.760+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f25dc0cb110 con 0x7f25dc06d000 2023-11-06T03:05:09.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.764+0000 7f25d8ff9640 1 -- 172.21.15.157:0/3291606900 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25cc0ee9a0 con 0x7f25dc06d000 2023-11-06T03:05:09.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 msgr2=0x7f25c8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 0x7f25c8067f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25dc14d400 tx=0x7f25d0008040 comp rx=0 tx=0).stop 2023-11-06T03:05:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 msgr2=0x7f25dc1f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00b2c0 tx=0x7f25cc01b980 comp rx=0 tx=0).stop 2023-11-06T03:05:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 shutdown_connections 2023-11-06T03:05:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f25c8065a90 0x7f25c8067f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f25dc15dea0 0x7f25dc1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25dc1599c0 0x7f25dc1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 --2- 172.21.15.157:0/3291606900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25dc06d000 0x7f25dc1f5d30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 >> 172.21.15.157:0/3291606900 conn(0x7f25dc065930 msgr2=0x7f25dc065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 shutdown_connections 2023-11-06T03:05:09.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.768+0000 7f25e27d3640 1 -- 172.21.15.157:0/3291606900 wait complete. 2023-11-06T03:05:09.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-06T03:05:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 Processor -- start 2023-11-06T03:05:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 -- start start 2023-11-06T03:05:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcff80aff30 0x7fcff80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff806bda0 con 0x7fcff80b2490 2023-11-06T03:05:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff806b7e0 con 0x7fcff8071db0 2023-11-06T03:05:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffe863640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff80b0310 con 0x7fcff80aff30 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffce00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcff80b2490 0x7fcff8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58516/0 (socket says 172.21.15.157:58516) 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffce00640 1 -- 172.21.15.157:0/593541413 learned_addr learned my addr 172.21.15.157:0/593541413 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff7fff640 1 --2- 172.21.15.157:0/593541413 >> v2:172.21.15.195:3300/0 conn(0x7fcff80aff30 0x7fcff80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682883986 0 0) 0x7fcff806bda0 con 0x7fcff80b2490 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe4003530 con 0x7fcff80b2490 2023-11-06T03:05:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4086172131 0 0) 0x7fcfe4003530 con 0x7fcff80b2490 2023-11-06T03:05:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 >> v2:172.21.15.195:3300/0 conn(0x7fcff80aff30 msgr2=0x7fcff80ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 --2- 172.21.15.157:0/593541413 >> v2:172.21.15.195:3300/0 conn(0x7fcff80aff30 0x7fcff80ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 >> v1:172.21.15.195:6789/0 conn(0x7fcff8071db0 legacy=0x7fcff806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff806bac0 con 0x7fcff80b2490 2023-11-06T03:05:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fcff0001a00 con 0x7fcff80b2490 2023-11-06T03:05:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 >> v1:172.21.15.157:6789/0 conn(0x7fcff80b2490 legacy=0x7fcff8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe4003980 0x7fcfe4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 0x7fcfe4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe4005190 0x7fcfe400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80b0310 con 0x7fcfe40047d0 2023-11-06T03:05:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe400c130 con 0x7fcfe4003980 2023-11-06T03:05:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe400c2f0 con 0x7fcfe4005190 2023-11-06T03:05:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 0x7fcfe4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcff7fff640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe4003980 0x7fcfe4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 -- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe4005190 msgr2=0x7fcfe400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe4005190 0x7fcfe400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 -- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe4003980 msgr2=0x7fcfe4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcfe4003980 0x7fcfe4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcfeffff640 1 -- 172.21.15.157:0/593541413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4003530 con 0x7fcfe40047d0 2023-11-06T03:05:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.904+0000 7fcffce00640 1 -- 172.21.15.157:0/593541413 reap_dead start 2023-11-06T03:05:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcfeffff640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 0x7fcfe4004c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0001750 tx=0x7fcfe001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d650494949a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00052d0 con 0x7fcfe40047d0 2023-11-06T03:05:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe00a7050 con 0x7fcfe40047d0 2023-11-06T03:05:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcff77fe640 1 -- 172.21.15.157:0/593541413 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0003600 con 0x7fcfe40047d0 2023-11-06T03:05:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 msgr2=0x7fcfe4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 0x7fcfe4004c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0001750 tx=0x7fcfe001c370 comp rx=0 tx=0).stop 2023-11-06T03:05:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- 172.21.15.157:0/593541413 shutdown_connections 2023-11-06T03:05:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 --2- 172.21.15.157:0/593541413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfe40047d0 0x7fcfe4004c50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- 172.21.15.157:0/593541413 >> 172.21.15.157:0/593541413 conn(0x7fcff80cd490 msgr2=0x7fcff80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- 172.21.15.157:0/593541413 shutdown_connections 2023-11-06T03:05:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- 172.21.15.157:0/593541413 wait complete. 2023-11-06T03:05:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 Processor -- start 2023-11-06T03:05:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- start start 2023-11-06T03:05:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe40047d0 0x7fcff81f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff815f940 0x7fcff815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff815f320 con 0x7fcff80b47b0 2023-11-06T03:05:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff815f4b0 con 0x7fcff815f940 2023-11-06T03:05:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffe863640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81f62e0 con 0x7fcfe40047d0 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcfeffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcfeffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55884/0 (socket says 172.21.15.157:55884) 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcfeffff640 1 -- 172.21.15.157:0/59025253 learned_addr learned my addr 172.21.15.157:0/59025253 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.908+0000 7fcffce00640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff815f940 0x7fcff815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff7fff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe40047d0 0x7fcff81f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 -- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe40047d0 msgr2=0x7fcff81f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe40047d0 0x7fcff81f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 -- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff815f940 msgr2=0x7fcff815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff815f940 0x7fcff815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 -- 172.21.15.157:0/59025253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80b2e30 con 0x7fcff80b47b0 2023-11-06T03:05:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcfeffff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001c1a0 tx=0x7fcfe4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00072f0 con 0x7fcff80b47b0 2023-11-06T03:05:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0007450 con 0x7fcff80b47b0 2023-11-06T03:05:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff81f6540 con 0x7fcff80b47b0 2023-11-06T03:05:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00067e0 con 0x7fcff80b47b0 2023-11-06T03:05:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff81f6b00 con 0x7fcff80b47b0 2023-11-06T03:05:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff815f640 con 0x7fcff80b47b0 2023-11-06T03:05:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcfe0006940 con 0x7fcff80b47b0 2023-11-06T03:05:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 0x7fcfd8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcfe011cf80 con 0x7fcff80b47b0 2023-11-06T03:05:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff7fff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 0x7fcfd8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:09.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.912+0000 7fcff7fff640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 0x7fcfd8067f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ab0 tx=0x7fcfe8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:09.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:09.916+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcfe00ee650 con 0x7fcff80b47b0 2023-11-06T03:05:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.132+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcff8072750 con 0x7fcff80b47b0 2023-11-06T03:05:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcff57fa640 1 -- 172.21.15.157:0/59025253 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcfe00ee1e0 con 0x7fcff80b47b0 2023-11-06T03:05:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 msgr2=0x7fcfd8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 0x7fcfd8067f50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009ab0 tx=0x7fcfe8008040 comp rx=0 tx=0).stop 2023-11-06T03:05:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 msgr2=0x7fcff815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001c1a0 tx=0x7fcfe4005800 comp rx=0 tx=0).stop 2023-11-06T03:05:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 shutdown_connections 2023-11-06T03:05:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcfd8065a90 0x7fcfd8067f50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcff815f940 0x7fcff815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff80b47b0 0x7fcff815d2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 --2- 172.21.15.157:0/59025253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcfe40047d0 0x7fcff81f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 >> 172.21.15.157:0/59025253 conn(0x7fcff80cd490 msgr2=0x7fcff8161740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 shutdown_connections 2023-11-06T03:05:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.140+0000 7fcffe863640 1 -- 172.21.15.157:0/59025253 wait complete. 2023-11-06T03:05:10.158 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-06T03:05:10.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 Processor -- start 2023-11-06T03:05:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 -- start start 2023-11-06T03:05:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9a80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80d2920 con 0x7fc9a815caa0 2023-11-06T03:05:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a80d18a0 con 0x7fc9a80d00b0 2023-11-06T03:05:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9af882640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80d0520 con 0x7fc9a815ce80 2023-11-06T03:05:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc9a815caa0 0x7fc9a80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58522/0 (socket says 172.21.15.157:58522) 2023-11-06T03:05:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 -- 172.21.15.157:0/3645677005 learned_addr learned my addr 172.21.15.157:0/3645677005 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ace1d640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9a80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657553171 0 0) 0x7fc9a80d2920 con 0x7fc9a815caa0 2023-11-06T03:05:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc994003530 con 0x7fc9a815caa0 2023-11-06T03:05:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676584176 0 0) 0x7fc994003530 con 0x7fc9a815caa0 2023-11-06T03:05:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 msgr2=0x7fc9a80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9a80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 >> v1:172.21.15.195:6789/0 conn(0x7fc9a80d00b0 legacy=0x7fc9a80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80d1b60 con 0x7fc9a815caa0 2023-11-06T03:05:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc9a4001680 con 0x7fc9a815caa0 2023-11-06T03:05:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 >> v1:172.21.15.157:6789/0 conn(0x7fc9a815caa0 legacy=0x7fc9a80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc994003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc994004d10 0x7fc994005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80d0520 con 0x7fc9940039b0 2023-11-06T03:05:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400bb20 con 0x7fc994004d10 2023-11-06T03:05:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400bce0 con 0x7fc9a815ce80 2023-11-06T03:05:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc994003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ace1d640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 -- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 msgr2=0x7fc9940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a815ce80 0x7fc9940047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 -- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc994004d10 msgr2=0x7fc994005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc994004d10 0x7fc994005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.276+0000 7fc9ad61e640 1 -- 172.21.15.157:0/3645677005 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994003530 con 0x7fc9940039b0 2023-11-06T03:05:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc994003dd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c240 tx=0x7fc9a401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e868d4bd532df629 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4005ae0 con 0x7fc9940039b0 2023-11-06T03:05:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4005c40 con 0x7fc9940039b0 2023-11-06T03:05:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc99ffff640 1 -- 172.21.15.157:0/3645677005 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40a5800 con 0x7fc9940039b0 2023-11-06T03:05:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- 172.21.15.157:0/3645677005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 msgr2=0x7fc994003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 --2- 172.21.15.157:0/3645677005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc994003dd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c240 tx=0x7fc9a401cc10 comp rx=0 tx=0).stop 2023-11-06T03:05:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9ade1f640 1 -- 172.21.15.157:0/3645677005 reap_dead start 2023-11-06T03:05:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- 172.21.15.157:0/3645677005 shutdown_connections 2023-11-06T03:05:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- 172.21.15.157:0/3645677005 >> 172.21.15.157:0/3645677005 conn(0x7fc9a8070790 msgr2=0x7fc9a8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- 172.21.15.157:0/3645677005 shutdown_connections 2023-11-06T03:05:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- 172.21.15.157:0/3645677005 wait complete. 2023-11-06T03:05:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 Processor -- start 2023-11-06T03:05:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- start start 2023-11-06T03:05:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9a81ec0b0 0x7fc9a8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a81ec530 0x7fc9a81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81ebc90 con 0x7fc9940039b0 2023-11-06T03:05:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8158af0 con 0x7fc9a81ec0b0 2023-11-06T03:05:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9af882640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81eb010 con 0x7fc9a81ec530 2023-11-06T03:05:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9ad61e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9ad61e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55912/0 (socket says 172.21.15.157:55912) 2023-11-06T03:05:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.280+0000 7fc9ad61e640 1 -- 172.21.15.157:0/2060784311 learned_addr learned my addr 172.21.15.157:0/2060784311 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ace1d640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9a81ec0b0 0x7fc9a8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ade1f640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a81ec530 0x7fc9a81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 -- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a81ec530 msgr2=0x7fc9a81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a81ec530 0x7fc9a81572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 -- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9a81ec0b0 msgr2=0x7fc9a8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9a81ec0b0 0x7fc9a8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 -- 172.21.15.157:0/2060784311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a81eb2f0 con 0x7fc9940039b0 2023-11-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ad61e640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c10 tx=0x7fc994005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40b4070 con 0x7fc9940039b0 2023-11-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4005e70 con 0x7fc9940039b0 2023-11-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a81eb5d0 con 0x7fc9940039b0 2023-11-06T03:05:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4001660 con 0x7fc9940039b0 2023-11-06T03:05:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a80cb700 con 0x7fc9940039b0 2023-11-06T03:05:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a8072750 con 0x7fc9940039b0 2023-11-06T03:05:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc9a40a5800 con 0x7fc9940039b0 2023-11-06T03:05:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 0x7fc98c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9a411c960 con 0x7fc9940039b0 2023-11-06T03:05:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ace1d640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 0x7fc98c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.284+0000 7fc9ace1d640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 0x7fc98c067f50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8160ee0 tx=0x7fc998008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.288+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc9a40ea5a0 con 0x7fc9940039b0 2023-11-06T03:05:10.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.488+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc9a80b7ea0 con 0x7fc9940039b0 2023-11-06T03:05:10.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc99dffb640 1 -- 172.21.15.157:0/2060784311 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9a40edbc0 con 0x7fc9940039b0 2023-11-06T03:05:10.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 msgr2=0x7fc98c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 0x7fc98c067f50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8160ee0 tx=0x7fc998008040 comp rx=0 tx=0).stop 2023-11-06T03:05:10.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 msgr2=0x7fc9a80cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c10 tx=0x7fc994005800 comp rx=0 tx=0).stop 2023-11-06T03:05:10.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 shutdown_connections 2023-11-06T03:05:10.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc98c065a90 0x7fc98c067f50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9a81ec530 0x7fc9a81572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9a81ec0b0 0x7fc9a8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 --2- 172.21.15.157:0/2060784311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9940039b0 0x7fc9a80cb1c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 >> 172.21.15.157:0/2060784311 conn(0x7fc9a8070790 msgr2=0x7fc9a80b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 shutdown_connections 2023-11-06T03:05:10.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.492+0000 7fc9af882640 1 -- 172.21.15.157:0/2060784311 wait complete. 2023-11-06T03:05:10.510 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-06T03:05:10.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 Processor -- start 2023-11-06T03:05:10.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 -- start start 2023-11-06T03:05:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6f1c15ce80 0x7f6f1c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0d2920 con 0x7f6f1c0ce1a0 2023-11-06T03:05:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c0d18a0 con 0x7f6f1c15caa0 2023-11-06T03:05:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f23d30640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c0d1b60 con 0x7f6f1c15ce80 2023-11-06T03:05:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55924/0 (socket says 172.21.15.157:55924) 2023-11-06T03:05:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 -- 172.21.15.157:0/3005999150 learned_addr learned my addr 172.21.15.157:0/3005999150 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 -- 172.21.15.157:0/3005999150 >> v2:172.21.15.195:3300/0 conn(0x7f6f1c15ce80 msgr2=0x7f6f1c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3005999150 >> v2:172.21.15.195:3300/0 conn(0x7f6f1c15ce80 0x7f6f1c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 -- 172.21.15.157:0/3005999150 >> v1:172.21.15.195:6789/0 conn(0x7f6f1c0ce1a0 legacy=0x7f6f1c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.624+0000 7f6f21acc640 1 -- 172.21.15.157:0/3005999150 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0d1e40 con 0x7f6f1c15caa0 2023-11-06T03:05:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3005999150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0c6720 tx=0x7f6f1801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbbe1f990f51221 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f20aca640 1 -- 172.21.15.157:0/3005999150 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180ac600 con 0x7f6f1c15caa0 2023-11-06T03:05:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f20aca640 1 -- 172.21.15.157:0/3005999150 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180a8050 con 0x7f6f1c15caa0 2023-11-06T03:05:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f20aca640 1 -- 172.21.15.157:0/3005999150 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f180ac980 con 0x7f6f1c15caa0 2023-11-06T03:05:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- 172.21.15.157:0/3005999150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c15caa0 msgr2=0x7f6f1c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3005999150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0c6720 tx=0x7f6f1801c490 comp rx=0 tx=0).stop 2023-11-06T03:05:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- 172.21.15.157:0/3005999150 shutdown_connections 2023-11-06T03:05:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3005999150 >> v2:172.21.15.195:3300/0 conn(0x7f6f1c15ce80 0x7f6f1c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3005999150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c15caa0 0x7f6f1c0cdc60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- 172.21.15.157:0/3005999150 >> 172.21.15.157:0/3005999150 conn(0x7f6f1c070790 msgr2=0x7f6f1c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- 172.21.15.157:0/3005999150 shutdown_connections 2023-11-06T03:05:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- 172.21.15.157:0/3005999150 wait complete. 2023-11-06T03:05:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 Processor -- start 2023-11-06T03:05:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- start start 2023-11-06T03:05:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 0x7f6f1c166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f1c15ce80 0x7f6f1c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1670c0 con 0x7f6f1c160df0 2023-11-06T03:05:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c148cf0 con 0x7f6f1c0ce1a0 2023-11-06T03:05:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f23d30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c148fd0 con 0x7f6f1c15ce80 2023-11-06T03:05:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f222cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f222cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55932/0 (socket says 172.21.15.157:55932) 2023-11-06T03:05:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.628+0000 7f6f222cd640 1 -- 172.21.15.157:0/3462227796 learned_addr learned my addr 172.21.15.157:0/3462227796 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 0x7f6f1c166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f212cb640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f1c15ce80 0x7f6f1c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 -- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f1c15ce80 msgr2=0x7f6f1c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f1c15ce80 0x7f6f1c1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 -- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 msgr2=0x7f6f1c166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 0x7f6f1c166b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 -- 172.21.15.157:0/3462227796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180094d0 con 0x7f6f1c160df0 2023-11-06T03:05:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 0x7f6f1c166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f222cd640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c780 tx=0x7f6f1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14003d90 con 0x7f6f1c160df0 2023-11-06T03:05:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400ce30 con 0x7f6f1c160df0 2023-11-06T03:05:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c1492b0 con 0x7f6f1c160df0 2023-11-06T03:05:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14005290 con 0x7f6f1c160df0 2023-11-06T03:05:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c149870 con 0x7f6f1c160df0 2023-11-06T03:05:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c149a90 con 0x7f6f1c160df0 2023-11-06T03:05:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6f140053f0 con 0x7f6f1c160df0 2023-11-06T03:05:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 0x7f6f00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6f140827e0 con 0x7f6f1c160df0 2023-11-06T03:05:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 0x7f6f00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.632+0000 7f6f21acc640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 0x7f6f00067f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f180ae000 tx=0x7f6f18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.636+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6f14050390 con 0x7f6f1c160df0 2023-11-06T03:05:10.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.832+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f6f1c072750 con 0x7f6f1c160df0 2023-11-06T03:05:10.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.840+0000 7f6f127fc640 1 -- 172.21.15.157:0/3462227796 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f14017020 con 0x7f6f1c160df0 2023-11-06T03:05:10.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 msgr2=0x7f6f00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 0x7f6f00067f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f180ae000 tx=0x7f6f18008040 comp rx=0 tx=0).stop 2023-11-06T03:05:10.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 msgr2=0x7f6f1c0b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400c780 tx=0x7f6f1400cc40 comp rx=0 tx=0).stop 2023-11-06T03:05:10.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 shutdown_connections 2023-11-06T03:05:10.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f00065a60 0x7f6f00067f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c160df0 0x7f6f1c0b70b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f1c15ce80 0x7f6f1c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 --2- 172.21.15.157:0/3462227796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f1c0ce1a0 0x7f6f1c166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 >> 172.21.15.157:0/3462227796 conn(0x7f6f1c070790 msgr2=0x7f6f1c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 shutdown_connections 2023-11-06T03:05:10.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.844+0000 7f6f23d30640 1 -- 172.21.15.157:0/3462227796 wait complete. 2023-11-06T03:05:10.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 Processor -- start 2023-11-06T03:05:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 -- start start 2023-11-06T03:05:10.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faabc1599c0 0x7faabc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc15e440 con 0x7faabc15dea0 2023-11-06T03:05:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faabc15adb0 con 0x7faabc1599c0 2023-11-06T03:05:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faac105e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc06c240 con 0x7faabc06d000 2023-11-06T03:05:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faab3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faabad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faabc1599c0 0x7faabc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faabad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faabc1599c0 0x7faabc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:56754/0 (socket says 172.21.15.157:56754) 2023-11-06T03:05:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faabb59e640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7faabc15dea0 0x7faabc070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:47522/0 (socket says 172.21.15.157:47522) 2023-11-06T03:05:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faabad9d640 1 -- 172.21.15.157:0/2680941149 learned_addr learned my addr 172.21.15.157:0/2680941149 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.968+0000 7faab3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43710/0 (socket says 172.21.15.157:43710) 2023-11-06T03:05:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faabad9d640 1 -- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 msgr2=0x7faabc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faabad9d640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faabad9d640 1 -- 172.21.15.157:0/2680941149 >> v1:172.21.15.195:6789/0 conn(0x7faabc15dea0 legacy=0x7faabc070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faabad9d640 1 -- 172.21.15.157:0/2680941149 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc06c520 con 0x7faabc1599c0 2023-11-06T03:05:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faab3fff640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:05:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faabad9d640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc1599c0 0x7faabc0ae000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faaac009870 tx=0x7faaac01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa91f819204bed6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faaba59c640 1 -- 172.21.15.157:0/2680941149 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0ab620 con 0x7faabc1599c0 2023-11-06T03:05:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faaba59c640 1 -- 172.21.15.157:0/2680941149 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac0ab780 con 0x7faabc1599c0 2023-11-06T03:05:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faaba59c640 1 -- 172.21.15.157:0/2680941149 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0abb10 con 0x7faabc1599c0 2023-11-06T03:05:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 -- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc1599c0 msgr2=0x7faabc0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc1599c0 0x7faabc0ae000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faaac009870 tx=0x7faaac01c510 comp rx=0 tx=0).stop 2023-11-06T03:05:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 -- 172.21.15.157:0/2680941149 shutdown_connections 2023-11-06T03:05:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc06d000 0x7faabc0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 --2- 172.21.15.157:0/2680941149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc1599c0 0x7faabc0ae000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 -- 172.21.15.157:0/2680941149 >> 172.21.15.157:0/2680941149 conn(0x7faabc065930 msgr2=0x7faabc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 -- 172.21.15.157:0/2680941149 shutdown_connections 2023-11-06T03:05:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.972+0000 7faac105e640 1 -- 172.21.15.157:0/2680941149 wait complete. 2023-11-06T03:05:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 Processor -- start 2023-11-06T03:05:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- start start 2023-11-06T03:05:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc1599c0 0x7faabc165860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 0x7faabc0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc166c80 con 0x7faabc06d000 2023-11-06T03:05:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc165da0 con 0x7faabc15dea0 2023-11-06T03:05:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc166070 con 0x7faabc1599c0 2023-11-06T03:05:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55944/0 (socket says 172.21.15.157:55944) 2023-11-06T03:05:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 -- 172.21.15.157:0/305878228 learned_addr learned my addr 172.21.15.157:0/305878228 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faab3fff640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc1599c0 0x7faabc165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabb59e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 0x7faabc0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 -- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc1599c0 msgr2=0x7faabc165860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc1599c0 0x7faabc165860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 -- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 msgr2=0x7faabc0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 0x7faabc0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 -- 172.21.15.157:0/305878228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa4009420 con 0x7faabc06d000 2023-11-06T03:05:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabb59e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 0x7faabc0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faabad9d640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faaac0ad040 tx=0x7faaac01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0abe80 con 0x7faabc06d000 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac0be3f0 con 0x7faabc06d000 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac0be7a0 con 0x7faabc06d000 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac0094d0 con 0x7faabc06d000 2023-11-06T03:05:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.976+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc166630 con 0x7faabc06d000 2023-11-06T03:05:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc072750 con 0x7faabc06d000 2023-11-06T03:05:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faaac0b43f0 con 0x7faabc06d000 2023-11-06T03:05:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faab37fe640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 0x7faa9c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faaac11d280 con 0x7faabc06d000 2023-11-06T03:05:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faab3fff640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 0x7faa9c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.980+0000 7faab3fff640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 0x7faa9c067f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009c80 tx=0x7faaa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:10.984+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faaac0eae60 con 0x7faabc06d000 2023-11-06T03:05:11.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.128+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faabc0cb110 con 0x7faabc06d000 2023-11-06T03:05:11.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.128+0000 7faab37fe640 1 -- 172.21.15.157:0/305878228 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14219 (secure 0 0 0) 0x7faaac0ee480 con 0x7faabc06d000 2023-11-06T03:05:11.130 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:11.131 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":33,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:05:08.234392+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 msgr2=0x7faa9c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 0x7faa9c067f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009c80 tx=0x7faaa4008040 comp rx=0 tx=0).stop 2023-11-06T03:05:11.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 msgr2=0x7faabc157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faaac0ad040 tx=0x7faaac01ad50 comp rx=0 tx=0).stop 2023-11-06T03:05:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 shutdown_connections 2023-11-06T03:05:11.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a90 0x7faa9c067f50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faabc15dea0 0x7faabc0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faabc1599c0 0x7faabc165860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 --2- 172.21.15.157:0/305878228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faabc06d000 0x7faabc157eb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 >> 172.21.15.157:0/305878228 conn(0x7faabc065930 msgr2=0x7faabc065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 shutdown_connections 2023-11-06T03:05:11.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.132+0000 7faac105e640 1 -- 172.21.15.157:0/305878228 wait complete. 2023-11-06T03:05:11.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:05:11.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:05:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 Processor -- start 2023-11-06T03:05:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 -- start start 2023-11-06T03:05:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4841551e0 0x7fc48406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc48415e5e0 0x7fc4840b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc48415ea60 0x7fc4840b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48415f580 con 0x7fc4841551e0 2023-11-06T03:05:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48415efc0 con 0x7fc48415ea60 2023-11-06T03:05:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48c58a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48415f2a0 con 0x7fc48415e5e0 2023-11-06T03:05:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4841551e0 0x7fc48406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4841551e0 0x7fc48406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55948/0 (socket says 172.21.15.157:55948) 2023-11-06T03:05:11.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 -- 172.21.15.157:0/565728890 learned_addr learned my addr 172.21.15.157:0/565728890 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:11.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48ab27640 1 --2- 172.21.15.157:0/565728890 >> v2:172.21.15.195:3300/0 conn(0x7fc48415ea60 0x7fc4840b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc489b25640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc48415e5e0 0x7fc4840b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 -- 172.21.15.157:0/565728890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc48415e5e0 msgr2=0x7fc4840b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc48415e5e0 0x7fc4840b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 -- 172.21.15.157:0/565728890 >> v2:172.21.15.195:3300/0 conn(0x7fc48415ea60 msgr2=0x7fc4840b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 --2- 172.21.15.157:0/565728890 >> v2:172.21.15.195:3300/0 conn(0x7fc48415ea60 0x7fc4840b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.271+0000 7fc48a326640 1 -- 172.21.15.157:0/565728890 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48406f5a0 con 0x7fc4841551e0 2023-11-06T03:05:11.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48a326640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4841551e0 0x7fc48406fda0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc4840cc5d0 tx=0x7fc48001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f75a1eb6a3643 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc489324640 1 -- 172.21.15.157:0/565728890 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800ab620 con 0x7fc4841551e0 2023-11-06T03:05:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc489324640 1 -- 172.21.15.157:0/565728890 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800ab780 con 0x7fc4841551e0 2023-11-06T03:05:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc489324640 1 -- 172.21.15.157:0/565728890 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800abb10 con 0x7fc4841551e0 2023-11-06T03:05:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- 172.21.15.157:0/565728890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4841551e0 msgr2=0x7fc48406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4841551e0 0x7fc48406fda0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc4840cc5d0 tx=0x7fc48001c440 comp rx=0 tx=0).stop 2023-11-06T03:05:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- 172.21.15.157:0/565728890 shutdown_connections 2023-11-06T03:05:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- 172.21.15.157:0/565728890 >> v2:172.21.15.195:3300/0 conn(0x7fc48415ea60 0x7fc4840b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc48415e5e0 0x7fc4840b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- 172.21.15.157:0/565728890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4841551e0 0x7fc48406fda0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- 172.21.15.157:0/565728890 >> 172.21.15.157:0/565728890 conn(0x7fc484065930 msgr2=0x7fc484065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- 172.21.15.157:0/565728890 shutdown_connections 2023-11-06T03:05:11.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- 172.21.15.157:0/565728890 wait complete. 2023-11-06T03:05:11.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 Processor -- start 2023-11-06T03:05:11.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- start start 2023-11-06T03:05:11.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4841551e0 0x7fc4841f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48415ea60 0x7fc4840ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841508c0 con 0x7fc48415e5e0 2023-11-06T03:05:11.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484150b10 con 0x7fc48415ea60 2023-11-06T03:05:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.275+0000 7fc48c58a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc484150df0 con 0x7fc4841551e0 2023-11-06T03:05:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55958/0 (socket says 172.21.15.157:55958) 2023-11-06T03:05:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 -- 172.21.15.157:0/1390356699 learned_addr learned my addr 172.21.15.157:0/1390356699 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48a326640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4841551e0 0x7fc4841f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48ab27640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48415ea60 0x7fc4840ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 -- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4841551e0 msgr2=0x7fc4841f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4841551e0 0x7fc4841f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 -- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48415ea60 msgr2=0x7fc4840ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48415ea60 0x7fc4840ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 -- 172.21.15.157:0/1390356699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800094d0 con 0x7fc48415e5e0 2023-11-06T03:05:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc489b25640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc47400d800 tx=0x7fc47400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740096a0 con 0x7fc48415e5e0 2023-11-06T03:05:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc474009800 con 0x7fc48415e5e0 2023-11-06T03:05:11.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841510d0 con 0x7fc48415e5e0 2023-11-06T03:05:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474009b70 con 0x7fc48415e5e0 2023-11-06T03:05:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc484151690 con 0x7fc48415e5e0 2023-11-06T03:05:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484072750 con 0x7fc48415e5e0 2023-11-06T03:05:11.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc474009cd0 con 0x7fc48415e5e0 2023-11-06T03:05:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 0x7fc468067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc474083b10 con 0x7fc48415e5e0 2023-11-06T03:05:11.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.279+0000 7fc48a326640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 0x7fc468067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.283+0000 7fc48a326640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 0x7fc468067f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4800af000 tx=0x7fc4800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.283+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4740517a0 con 0x7fc48415e5e0 2023-11-06T03:05:11.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.427+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc484153100 con 0x7fc48415e5e0 2023-11-06T03:05:11.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.427+0000 7fc47affd640 1 -- 172.21.15.157:0/1390356699 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc474054dc0 con 0x7fc48415e5e0 2023-11-06T03:05:11.431 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:05:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 msgr2=0x7fc468067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 0x7fc468067f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4800af000 tx=0x7fc4800a7000 comp rx=0 tx=0).stop 2023-11-06T03:05:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 msgr2=0x7fc4840cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc47400d800 tx=0x7fc47400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 shutdown_connections 2023-11-06T03:05:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc468065ab0 0x7fc468067f70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48415ea60 0x7fc4840ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc48415e5e0 0x7fc4840cdad0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 --2- 172.21.15.157:0/1390356699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4841551e0 0x7fc4841f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 >> 172.21.15.157:0/1390356699 conn(0x7fc484065930 msgr2=0x7fc4841544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 shutdown_connections 2023-11-06T03:05:11.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.431+0000 7fc48c58a640 1 -- 172.21.15.157:0/1390356699 wait complete. 2023-11-06T03:05:11.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:05:11.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.567+0000 7fa219075640 1 Processor -- start 2023-11-06T03:05:11.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.567+0000 7fa219075640 1 -- start start 2023-11-06T03:05:11.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa219075640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2140adae0 0x7fa2140b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa219075640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 0x7fa2140b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa219075640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21406c490 con 0x7fa214156a70 2023-11-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa219075640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21406b950 con 0x7fa2140adae0 2023-11-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa219075640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21406fb80 con 0x7fa2140adf40 2023-11-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa2140adae0 0x7fa2140b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa212d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa214156a70 0x7fa21406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58530/0 (socket says 172.21.15.157:58530) 2023-11-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa212d9d640 1 -- 172.21.15.157:0/973293778 learned_addr learned my addr 172.21.15.157:0/973293778 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21359e640 1 --2- 172.21.15.157:0/973293778 >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 0x7fa2140b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 -- 172.21.15.157:0/973293778 >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 msgr2=0x7fa2140b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 --2- 172.21.15.157:0/973293778 >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 0x7fa2140b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 -- 172.21.15.157:0/973293778 >> v1:172.21.15.157:6789/0 conn(0x7fa214156a70 legacy=0x7fa21406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 -- 172.21.15.157:0/973293778 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21406bb60 con 0x7fa2140adae0 2023-11-06T03:05:11.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21359e640 1 --2- 172.21.15.157:0/973293778 >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 0x7fa2140b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:11.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.571+0000 7fa21259c640 1 --2- 172.21.15.157:0/973293778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adae0 0x7fa2140b1c20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2140ce830 tx=0x7fa20801d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3902a6e1d62aa11b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa211d9b640 1 -- 172.21.15.157:0/973293778 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080a6830 con 0x7fa2140adae0 2023-11-06T03:05:11.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa211d9b640 1 -- 172.21.15.157:0/973293778 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080a6990 con 0x7fa2140adae0 2023-11-06T03:05:11.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa211d9b640 1 -- 172.21.15.157:0/973293778 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080a6d20 con 0x7fa2140adae0 2023-11-06T03:05:11.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/973293778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adae0 msgr2=0x7fa2140b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- 172.21.15.157:0/973293778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adae0 0x7fa2140b1c20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2140ce830 tx=0x7fa20801d800 comp rx=0 tx=0).stop 2023-11-06T03:05:11.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/973293778 shutdown_connections 2023-11-06T03:05:11.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- 172.21.15.157:0/973293778 >> v2:172.21.15.195:3300/0 conn(0x7fa2140adf40 0x7fa2140b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- 172.21.15.157:0/973293778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adae0 0x7fa2140b1c20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/973293778 >> 172.21.15.157:0/973293778 conn(0x7fa214065930 msgr2=0x7fa214065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/973293778 shutdown_connections 2023-11-06T03:05:11.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/973293778 wait complete. 2023-11-06T03:05:11.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 Processor -- start 2023-11-06T03:05:11.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- start start 2023-11-06T03:05:11.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2140adae0 0x7fa21413cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 0x7fa214158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21413d510 con 0x7fa2140adf40 2023-11-06T03:05:11.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214152340 con 0x7fa2140adae0 2023-11-06T03:05:11.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa214152620 con 0x7fa214156a70 2023-11-06T03:05:11.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55970/0 (socket says 172.21.15.157:55970) 2023-11-06T03:05:11.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 -- 172.21.15.157:0/435751597 learned_addr learned my addr 172.21.15.157:0/435751597 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:11.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa212d9d640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2140adae0 0x7fa21413cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21359e640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 0x7fa214158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 -- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 msgr2=0x7fa214158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 0x7fa214158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 -- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2140adae0 msgr2=0x7fa21413cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2140adae0 0x7fa21413cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 -- 172.21.15.157:0/435751597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20800a5c0 con 0x7fa2140adf40 2023-11-06T03:05:11.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21359e640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 0x7fa214158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:11.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa21259c640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2080b1040 tx=0x7fa20801a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080ae8d0 con 0x7fa2140adf40 2023-11-06T03:05:11.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080aea30 con 0x7fa2140adf40 2023-11-06T03:05:11.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa214152900 con 0x7fa2140adf40 2023-11-06T03:05:11.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080aede0 con 0x7fa2140adf40 2023-11-06T03:05:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.575+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa214151f80 con 0x7fa2140adf40 2023-11-06T03:05:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21413d680 con 0x7fa2140adf40 2023-11-06T03:05:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa2080b5040 con 0x7fa2140adf40 2023-11-06T03:05:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa1fb7fe640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 0x7fa200067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa20811d730 con 0x7fa2140adf40 2023-11-06T03:05:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa212d9d640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 0x7fa200067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.579+0000 7fa212d9d640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 0x7fa200067f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa204005890 tx=0x7fa204009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.583+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa21413d680 con 0x7fa2140adf40 2023-11-06T03:05:11.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.735+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa21413d650 con 0x7fa2140adf40 2023-11-06T03:05:11.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.735+0000 7fa1fb7fe640 1 -- 172.21.15.157:0/435751597 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2080eb250 con 0x7fa2140adf40 2023-11-06T03:05:11.739 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:05:11.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 msgr2=0x7fa200067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 0x7fa200067f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa204005890 tx=0x7fa204009040 comp rx=0 tx=0).stop 2023-11-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 msgr2=0x7fa214151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2080b1040 tx=0x7fa20801a5a0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 shutdown_connections 2023-11-06T03:05:11.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa200065a90 0x7fa200067f50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa214156a70 0x7fa214158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2140adf40 0x7fa214151a40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 --2- 172.21.15.157:0/435751597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2140adae0 0x7fa21413cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 >> 172.21.15.157:0/435751597 conn(0x7fa214065930 msgr2=0x7fa21415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 shutdown_connections 2023-11-06T03:05:11.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.739+0000 7fa219075640 1 -- 172.21.15.157:0/435751597 wait complete. 2023-11-06T03:05:11.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:05:11.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.871+0000 7f22fe546640 1 Processor -- start 2023-11-06T03:05:11.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.871+0000 7f22fe546640 1 -- start start 2023-11-06T03:05:11.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fe546640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 0x7f22f80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fe546640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f80ce800 0x7f22f80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fe546640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f80cf1a0 con 0x7f22f806b6a0 2023-11-06T03:05:11.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fe546640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f806c3d0 con 0x7f22f80ce800 2023-11-06T03:05:11.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fe546640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f806f5a0 con 0x7f22f8146370 2023-11-06T03:05:11.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 0x7f22f80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f80ce800 0x7f22f80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fcae3640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f22f806b6a0 0x7f22f806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:47526/0 (socket says 172.21.15.157:47526) 2023-11-06T03:05:11.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f7fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 0x7f22f80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:56796/0 (socket says 172.21.15.157:56796) 2023-11-06T03:05:11.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22f80ce800 0x7f22f80cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55984/0 (socket says 172.21.15.157:55984) 2023-11-06T03:05:11.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22fcae3640 1 -- 172.21.15.157:0/287531913 learned_addr learned my addr 172.21.15.157:0/287531913 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:11.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 -- 172.21.15.157:0/287531913 >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 msgr2=0x7f22f80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:11.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 --2- 172.21.15.157:0/287531913 >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 0x7f22f80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 -- 172.21.15.157:0/287531913 >> v1:172.21.15.195:6789/0 conn(0x7f22f806b6a0 legacy=0x7f22f806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 -- 172.21.15.157:0/287531913 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f806f880 con 0x7f22f80ce800 2023-11-06T03:05:11.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f77fe640 1 --2- 172.21.15.157:0/287531913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f80ce800 0x7f22f80cec60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0097c0 tx=0x7f22ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9450ede0495451d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f6ffd640 1 -- 172.21.15.157:0/287531913 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0ad680 con 0x7f22f80ce800 2023-11-06T03:05:11.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f6ffd640 1 -- 172.21.15.157:0/287531913 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec0ad7e0 con 0x7f22f80ce800 2023-11-06T03:05:11.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.875+0000 7f22f6ffd640 1 -- 172.21.15.157:0/287531913 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0adb70 con 0x7f22f80ce800 2023-11-06T03:05:11.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/287531913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f80ce800 msgr2=0x7f22f80cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- 172.21.15.157:0/287531913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f80ce800 0x7f22f80cec60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0097c0 tx=0x7f22ec01c380 comp rx=0 tx=0).stop 2023-11-06T03:05:11.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/287531913 shutdown_connections 2023-11-06T03:05:11.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- 172.21.15.157:0/287531913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f80ce800 0x7f22f80cec60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- 172.21.15.157:0/287531913 >> v2:172.21.15.195:3300/0 conn(0x7f22f8146370 0x7f22f80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/287531913 >> 172.21.15.157:0/287531913 conn(0x7f22f8065930 msgr2=0x7f22f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:11.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/287531913 shutdown_connections 2023-11-06T03:05:11.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/287531913 wait complete. 2023-11-06T03:05:11.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 Processor -- start 2023-11-06T03:05:11.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- start start 2023-11-06T03:05:11.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80ce800 0x7f22f8145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f8146370 0x7f22f815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8145940 con 0x7f22f806b6a0 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80aeb60 con 0x7f22f80ce800 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80aee40 con 0x7f22f8146370 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55988/0 (socket says 172.21.15.157:55988) 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 -- 172.21.15.157:0/2164908204 learned_addr learned my addr 172.21.15.157:0/2164908204 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:11.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 -- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f8146370 msgr2=0x7f22f815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f8146370 0x7f22f815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 -- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80ce800 msgr2=0x7f22f8145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:11.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80ce800 0x7f22f8145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:11.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 -- 172.21.15.157:0/2164908204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009420 con 0x7f22f806b6a0 2023-11-06T03:05:11.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f7fff640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f22e800e930 tx=0x7f22e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8009690 con 0x7f22f806b6a0 2023-11-06T03:05:11.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e800ca00 con 0x7f22f806b6a0 2023-11-06T03:05:11.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f80af120 con 0x7f22f806b6a0 2023-11-06T03:05:11.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e800cd70 con 0x7f22f806b6a0 2023-11-06T03:05:11.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.879+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f80af6e0 con 0x7f22f806b6a0 2023-11-06T03:05:11.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f8072750 con 0x7f22f806b6a0 2023-11-06T03:05:11.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22e8013470 con 0x7f22f806b6a0 2023-11-06T03:05:11.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22f4ff9640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 0x7f22e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:11.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f22e8083240 con 0x7f22f806b6a0 2023-11-06T03:05:11.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22f77fe640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 0x7f22e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:11.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.883+0000 7f22f77fe640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 0x7f22e4067f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0af000 tx=0x7f22ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:11.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:11.887+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f22e8050e90 con 0x7f22f806b6a0 2023-11-06T03:05:12.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.031+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22f8160de0 con 0x7f22f806b6a0 2023-11-06T03:05:12.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22f4ff9640 1 -- 172.21.15.157:0/2164908204 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7f22e80544b0 con 0x7f22f806b6a0 2023-11-06T03:05:12.036 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:12.036 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:05:12.037 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:12.037 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:05:12.037 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:05:12.038 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:12.038 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:05:12.038 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:05:12.039 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:05:12.039 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:12.039 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:05:12.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 msgr2=0x7f22e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 0x7f22e4067f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0af000 tx=0x7f22ec008040 comp rx=0 tx=0).stop 2023-11-06T03:05:12.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 msgr2=0x7f22f80ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f22e800e930 tx=0x7f22e800edf0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 shutdown_connections 2023-11-06T03:05:12.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22e4065a60 0x7f22e4067f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22f8146370 0x7f22f815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22f80ce800 0x7f22f8145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 --2- 172.21.15.157:0/2164908204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22f806b6a0 0x7f22f80ae580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 >> 172.21.15.157:0/2164908204 conn(0x7f22f8065930 msgr2=0x7f22f80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:12.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 shutdown_connections 2023-11-06T03:05:12.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.035+0000 7f22fe546640 1 -- 172.21.15.157:0/2164908204 wait complete. 2023-11-06T03:05:12.054 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:05:12.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:05:12.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.171+0000 7f8007589640 1 Processor -- start 2023-11-06T03:05:12.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.171+0000 7f8007589640 1 -- start start 2023-11-06T03:05:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8007589640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f800015ce70 0x7f80000d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8007589640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80000b8570 0x7f80000d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8007589640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80000d1520 con 0x7f80000d25c0 2023-11-06T03:05:12.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8007589640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000d1800 con 0x7f80000b8570 2023-11-06T03:05:12.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8007589640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000d1ae0 con 0x7f800015ce70 2023-11-06T03:05:12.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80000b8570 0x7f80000d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8005b26640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f80000d25c0 0x7f80000b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58542/0 (socket says 172.21.15.157:58542) 2023-11-06T03:05:12.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8005b26640 1 -- 172.21.15.157:0/1039828355 learned_addr learned my addr 172.21.15.157:0/1039828355 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:12.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8005325640 1 --2- 172.21.15.157:0/1039828355 >> v2:172.21.15.195:3300/0 conn(0x7f800015ce70 0x7f80000d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 -- 172.21.15.157:0/1039828355 >> v2:172.21.15.195:3300/0 conn(0x7f800015ce70 msgr2=0x7f80000d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:12.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 --2- 172.21.15.157:0/1039828355 >> v2:172.21.15.195:3300/0 conn(0x7f800015ce70 0x7f80000d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 -- 172.21.15.157:0/1039828355 >> v1:172.21.15.157:6789/0 conn(0x7f80000d25c0 legacy=0x7f80000b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 -- 172.21.15.157:0/1039828355 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000d1dc0 con 0x7f80000b8570 2023-11-06T03:05:12.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f8004b24640 1 --2- 172.21.15.157:0/1039828355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80000b8570 0x7f80000d2a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80000d2a60 tx=0x7f7ff001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413e1ea4a5a45292 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:12.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f7ff7fff640 1 -- 172.21.15.157:0/1039828355 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00ab600 con 0x7f80000b8570 2023-11-06T03:05:12.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f7ff7fff640 1 -- 172.21.15.157:0/1039828355 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff00a7050 con 0x7f80000b8570 2023-11-06T03:05:12.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.175+0000 7f7ff7fff640 1 -- 172.21.15.157:0/1039828355 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00ab980 con 0x7f80000b8570 2023-11-06T03:05:12.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/1039828355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80000b8570 msgr2=0x7f80000d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- 172.21.15.157:0/1039828355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80000b8570 0x7f80000d2a90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80000d2a60 tx=0x7f7ff001c460 comp rx=0 tx=0).stop 2023-11-06T03:05:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/1039828355 shutdown_connections 2023-11-06T03:05:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- 172.21.15.157:0/1039828355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80000b8570 0x7f80000d2a90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- 172.21.15.157:0/1039828355 >> v2:172.21.15.195:3300/0 conn(0x7f800015ce70 0x7f80000d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/1039828355 >> 172.21.15.157:0/1039828355 conn(0x7f8000070790 msgr2=0x7f8000070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:12.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/1039828355 shutdown_connections 2023-11-06T03:05:12.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/1039828355 wait complete. 2023-11-06T03:05:12.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 Processor -- start 2023-11-06T03:05:12.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- start start 2023-11-06T03:05:12.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80000b8570 0x7f800014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80000d25c0 0x7f800015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800015a120 con 0x7f800015ce70 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800014b320 con 0x7f80000b8570 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800015a600 con 0x7f80000d25c0 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56014/0 (socket says 172.21.15.157:56014) 2023-11-06T03:05:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 -- 172.21.15.157:0/4036382261 learned_addr learned my addr 172.21.15.157:0/4036382261 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005325640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80000b8570 0x7f800014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8004b24640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80000d25c0 0x7f800015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 -- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80000d25c0 msgr2=0x7f800015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80000d25c0 0x7f800015fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 -- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80000b8570 msgr2=0x7f800014ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:12.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80000b8570 0x7f800014ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:12.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 -- 172.21.15.157:0/4036382261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0009420 con 0x7f800015ce70 2023-11-06T03:05:12.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8005b26640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800cbc0 tx=0x7f7ff800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:12.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ad40 con 0x7f800015ce70 2023-11-06T03:05:12.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff8012400 con 0x7f800015ce70 2023-11-06T03:05:12.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8000160280 con 0x7f800015ce70 2023-11-06T03:05:12.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800aea0 con 0x7f800015ce70 2023-11-06T03:05:12.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.179+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8000159e40 con 0x7f800015ce70 2023-11-06T03:05:12.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80000c9940 con 0x7f800015ce70 2023-11-06T03:05:12.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7ff8007070 con 0x7f800015ce70 2023-11-06T03:05:12.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f7ff5ffb640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:12.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7ff8082dd0 con 0x7f800015ce70 2023-11-06T03:05:12.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f8005325640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:12.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.183+0000 7f8005325640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b9d0 tx=0x7f7ffc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:12.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.187+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7ff8050a30 con 0x7f800015ce70 2023-11-06T03:05:12.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:12.335+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f80000b30c0 con 0x7f800015ce70 2023-11-06T03:05:13.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.219+0000 7f7ff5ffb640 1 -- 172.21.15.157:0/4036382261 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7ff80840b0 con 0x7f800015ce70 2023-11-06T03:05:13.221 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:05:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 msgr2=0x7f7fe4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b9d0 tx=0x7f7ffc008040 comp rx=0 tx=0).stop 2023-11-06T03:05:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 msgr2=0x7f80000b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800cbc0 tx=0x7f7ff800a430 comp rx=0 tx=0).stop 2023-11-06T03:05:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 shutdown_connections 2023-11-06T03:05:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f800015ce70 0x7f80000b3970 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80000d25c0 0x7f800015fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 --2- 172.21.15.157:0/4036382261 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80000b8570 0x7f800014ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 >> 172.21.15.157:0/4036382261 conn(0x7f8000070790 msgr2=0x7f80000b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 shutdown_connections 2023-11-06T03:05:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.223+0000 7f8007589640 1 -- 172.21.15.157:0/4036382261 wait complete. 2023-11-06T03:05:13.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:05:13.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 Processor -- start 2023-11-06T03:05:13.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 -- start start 2023-11-06T03:05:13.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73781599c0 0x7f73780ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73780b2af0 0x7f7378159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737815e440 con 0x7f737806d000 2023-11-06T03:05:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737806c240 con 0x7f73781599c0 2023-11-06T03:05:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737cf4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737806c520 con 0x7f73780b2af0 2023-11-06T03:05:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73781599c0 0x7f73780ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73781599c0 0x7f73780ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56030/0 (socket says 172.21.15.157:56030) 2023-11-06T03:05:13.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 -- 172.21.15.157:0/967059417 learned_addr learned my addr 172.21.15.157:0/967059417 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f7376d9d640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73780b2af0 0x7f7378159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 -- 172.21.15.157:0/967059417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73780b2af0 msgr2=0x7f7378159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73780b2af0 0x7f7378159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 -- 172.21.15.157:0/967059417 >> v1:172.21.15.195:6789/0 conn(0x7f737806d000 legacy=0x7f73780b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.375+0000 7f737659c640 1 -- 172.21.15.157:0/967059417 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737806c800 con 0x7f73781599c0 2023-11-06T03:05:13.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737659c640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73781599c0 0x7f73780ae000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7368009870 tx=0x7f736801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b168fca381a374ee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:13.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737559a640 1 -- 172.21.15.157:0/967059417 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a95f0 con 0x7f73781599c0 2023-11-06T03:05:13.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737559a640 1 -- 172.21.15.157:0/967059417 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680a9750 con 0x7f73781599c0 2023-11-06T03:05:13.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737559a640 1 -- 172.21.15.157:0/967059417 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a9ae0 con 0x7f73781599c0 2023-11-06T03:05:13.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- 172.21.15.157:0/967059417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73781599c0 msgr2=0x7f73780ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73781599c0 0x7f73780ae000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7368009870 tx=0x7f736801c510 comp rx=0 tx=0).stop 2023-11-06T03:05:13.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- 172.21.15.157:0/967059417 shutdown_connections 2023-11-06T03:05:13.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73780b2af0 0x7f7378159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 --2- 172.21.15.157:0/967059417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73781599c0 0x7f73780ae000 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- 172.21.15.157:0/967059417 >> 172.21.15.157:0/967059417 conn(0x7f7378065930 msgr2=0x7f7378065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:13.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- 172.21.15.157:0/967059417 shutdown_connections 2023-11-06T03:05:13.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- 172.21.15.157:0/967059417 wait complete. 2023-11-06T03:05:13.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 Processor -- start 2023-11-06T03:05:13.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.379+0000 7f737cf4d640 1 -- start start 2023-11-06T03:05:13.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 0x7f73780ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 0x7f73780ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780cc190 con 0x7f73780b2af0 2023-11-06T03:05:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73780ad280 con 0x7f737806d000 2023-11-06T03:05:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73780acfa0 con 0x7f73781599c0 2023-11-06T03:05:13.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56042/0 (socket says 172.21.15.157:56042) 2023-11-06T03:05:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 -- 172.21.15.157:0/2356240597 learned_addr learned my addr 172.21.15.157:0/2356240597 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737659c640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 0x7f73780ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7376d9d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 0x7f73780ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 -- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 msgr2=0x7f73780ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 0x7f73780ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 -- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 msgr2=0x7f73780ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 0x7f73780ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 -- 172.21.15.157:0/2356240597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680094d0 con 0x7f73780b2af0 2023-11-06T03:05:13.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737659c640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 0x7f73780ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7376d9d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 0x7f73780ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7375d9b640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c450 tx=0x7f736c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c0052d0 con 0x7f73780b2af0 2023-11-06T03:05:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c005450 con 0x7f73780b2af0 2023-11-06T03:05:13.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73780cbb40 con 0x7f73780b2af0 2023-11-06T03:05:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c012870 con 0x7f73780b2af0 2023-11-06T03:05:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737814d650 con 0x7f73780b2af0 2023-11-06T03:05:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378072750 con 0x7f73780b2af0 2023-11-06T03:05:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f736c0129d0 con 0x7f73780b2af0 2023-11-06T03:05:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 0x7f7358067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:13.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f736c083640 con 0x7f73780b2af0 2023-11-06T03:05:13.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.383+0000 7f737659c640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 0x7f7358067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:13.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.387+0000 7f737659c640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 0x7f7358067f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f73680ae000 tx=0x7f7368009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:13.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.387+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f736c051160 con 0x7f73780b2af0 2023-11-06T03:05:13.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:13.543+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f73780cb110 con 0x7f73780b2af0 2023-11-06T03:05:14.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.227+0000 7f7366ffd640 1 -- 172.21.15.157:0/2356240597 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7f736c019300 con 0x7f73780b2af0 2023-11-06T03:05:14.228 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:05:14.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 msgr2=0x7f7358067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 0x7f7358067f20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f73680ae000 tx=0x7f7368009040 comp rx=0 tx=0).stop 2023-11-06T03:05:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 msgr2=0x7f73780cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c450 tx=0x7f736c00c910 comp rx=0 tx=0).stop 2023-11-06T03:05:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 shutdown_connections 2023-11-06T03:05:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7358065a60 0x7f7358067f20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73781599c0 0x7f73780ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73780b2af0 0x7f73780cde50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 --2- 172.21.15.157:0/2356240597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737806d000 0x7f73780ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.235+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 >> 172.21.15.157:0/2356240597 conn(0x7f7378065930 msgr2=0x7f737806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:14.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.247+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 shutdown_connections 2023-11-06T03:05:14.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.247+0000 7f737cf4d640 1 -- 172.21.15.157:0/2356240597 wait complete. 2023-11-06T03:05:14.273 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:05:14.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.387+0000 7fc7cd884640 1 Processor -- start 2023-11-06T03:05:14.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.387+0000 7fc7cd884640 1 -- start start 2023-11-06T03:05:14.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7cd884640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815ce80 0x7fc7c80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7cd884640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c80d00b0 0x7fc7c80b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7cd884640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80d2920 con 0x7fc7c815caa0 2023-11-06T03:05:14.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7cd884640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80d0520 con 0x7fc7c80d00b0 2023-11-06T03:05:14.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7cd884640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80d18a0 con 0x7fc7c815ce80 2023-11-06T03:05:14.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c80d00b0 0x7fc7c80b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc7c815caa0 0x7fc7c80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58546/0 (socket says 172.21.15.157:58546) 2023-11-06T03:05:14.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c6ffd640 1 -- 172.21.15.157:0/2673632174 learned_addr learned my addr 172.21.15.157:0/2673632174 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:14.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c67fc640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815ce80 0x7fc7c80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 -- 172.21.15.157:0/2673632174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815ce80 msgr2=0x7fc7c80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:14.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815ce80 0x7fc7c80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 -- 172.21.15.157:0/2673632174 >> v1:172.21.15.157:6789/0 conn(0x7fc7c815caa0 legacy=0x7fc7c80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c5ffb640 1 -- 172.21.15.157:0/2673632174 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562149150 0 0) 0x7fc7c80d2920 con 0x7fc7c815caa0 2023-11-06T03:05:14.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 -- 172.21.15.157:0/2673632174 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80d1b60 con 0x7fc7c80d00b0 2023-11-06T03:05:14.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c77fe640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c80b5d50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097c0 tx=0x7fc7b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb4c2f24a042c6d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c5ffb640 1 -- 172.21.15.157:0/2673632174 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80ab650 con 0x7fc7c80d00b0 2023-11-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c5ffb640 1 -- 172.21.15.157:0/2673632174 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b80ab7b0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.391+0000 7fc7c5ffb640 1 -- 172.21.15.157:0/2673632174 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80abb40 con 0x7fc7c80d00b0 2023-11-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/2673632174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 msgr2=0x7fc7c80b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c80b5d50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097c0 tx=0x7fc7b801c450 comp rx=0 tx=0).stop 2023-11-06T03:05:14.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/2673632174 shutdown_connections 2023-11-06T03:05:14.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c80b5d50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- 172.21.15.157:0/2673632174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815ce80 0x7fc7c80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/2673632174 >> 172.21.15.157:0/2673632174 conn(0x7fc7c8070790 msgr2=0x7fc7c8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:14.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/2673632174 shutdown_connections 2023-11-06T03:05:14.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/2673632174 wait complete. 2023-11-06T03:05:14.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 Processor -- start 2023-11-06T03:05:14.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- start start 2023-11-06T03:05:14.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815caa0 0x7fc7c8148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 0x7fc7c80b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c814e6e0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c814fb30 con 0x7fc7c815ce80 2023-11-06T03:05:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c814fe10 con 0x7fc7c815caa0 2023-11-06T03:05:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56060/0 (socket says 172.21.15.157:56060) 2023-11-06T03:05:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 -- 172.21.15.157:0/3720827258 learned_addr learned my addr 172.21.15.157:0/3720827258 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c67fc640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815caa0 0x7fc7c8148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c77fe640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 0x7fc7c80b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 -- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815caa0 msgr2=0x7fc7c8148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815caa0 0x7fc7c8148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 -- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 msgr2=0x7fc7c80b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 0x7fc7c80b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 -- 172.21.15.157:0/3720827258 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8009420 con 0x7fc7c80d00b0 2023-11-06T03:05:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c77fe640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 0x7fc7c80b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7c6ffd640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0001570 tx=0x7fc7c000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000a640 con 0x7fc7c80d00b0 2023-11-06T03:05:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c81500f0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000a7c0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000ab70 con 0x7fc7c80d00b0 2023-11-06T03:05:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.395+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81506b0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c81508a0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7c000acd0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7affff640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 0x7fc7b4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc7c0082b10 con 0x7fc7c80d00b0 2023-11-06T03:05:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7c67fc640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 0x7fc7b4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.399+0000 7fc7c67fc640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 0x7fc7b4067f50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7c81541e0 tx=0x7fc7bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.403+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7c00503d0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.595+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc7c8072750 con 0x7fc7c80d00b0 2023-11-06T03:05:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.611+0000 7fc7affff640 1 -- 172.21.15.157:0/3720827258 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc7c00180c0 con 0x7fc7c80d00b0 2023-11-06T03:05:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 5 and data pool 6 2023-11-06T03:05:14.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 msgr2=0x7fc7b4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 0x7fc7b4067f50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7c81541e0 tx=0x7fc7bc008040 comp rx=0 tx=0).stop 2023-11-06T03:05:14.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 msgr2=0x7fc7c814f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0001570 tx=0x7fc7c000cd20 comp rx=0 tx=0).stop 2023-11-06T03:05:14.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 shutdown_connections 2023-11-06T03:05:14.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7b4065a90 0x7fc7b4067f50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c815ce80 0x7fc7c80b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c815caa0 0x7fc7c8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 --2- 172.21.15.157:0/3720827258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c80d00b0 0x7fc7c814f470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.615+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 >> 172.21.15.157:0/3720827258 conn(0x7fc7c8070790 msgr2=0x7fc7c80cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.623+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 shutdown_connections 2023-11-06T03:05:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.623+0000 7fc7cd884640 1 -- 172.21.15.157:0/3720827258 wait complete. 2023-11-06T03:05:14.649 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:14.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 Processor -- start 2023-11-06T03:05:14.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 -- start start 2023-11-06T03:05:14.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2a6415ac80 0x7f2a640b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a640cc2b0 con 0x7f2a640afb00 2023-11-06T03:05:14.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a640b01a0 con 0x7f2a6415ac80 2023-11-06T03:05:14.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6b1c1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640cfb10 con 0x7f2a640cf560 2023-11-06T03:05:14.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43812/0 (socket says 172.21.15.157:43812) 2023-11-06T03:05:14.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 -- 172.21.15.157:0/4119183135 learned_addr learned my addr 172.21.15.157:0/4119183135 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:14.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a68f5d640 1 --2- 172.21.15.157:0/4119183135 >> v2:172.21.15.195:3300/0 conn(0x7f2a6415ac80 0x7f2a640b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a637fe640 1 -- 172.21.15.157:0/4119183135 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273931432 0 0) 0x7f2a640cc2b0 con 0x7f2a640afb00 2023-11-06T03:05:14.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a637fe640 1 -- 172.21.15.157:0/4119183135 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a50003530 con 0x7f2a640afb00 2023-11-06T03:05:14.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 -- 172.21.15.157:0/4119183135 >> v2:172.21.15.195:3300/0 conn(0x7f2a6415ac80 msgr2=0x7f2a640b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:14.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 --2- 172.21.15.157:0/4119183135 >> v2:172.21.15.195:3300/0 conn(0x7f2a6415ac80 0x7f2a640b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 -- 172.21.15.157:0/4119183135 >> v1:172.21.15.195:6789/0 conn(0x7f2a640afb00 legacy=0x7f2a640cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.771+0000 7f2a6975e640 1 -- 172.21.15.157:0/4119183135 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a640cfdf0 con 0x7f2a640cf560 2023-11-06T03:05:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6975e640 1 --2- 172.21.15.157:0/4119183135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009870 tx=0x7f2a5401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a64528c575001e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a637fe640 1 -- 172.21.15.157:0/4119183135 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540ab650 con 0x7f2a640cf560 2023-11-06T03:05:14.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a637fe640 1 -- 172.21.15.157:0/4119183135 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540ab7b0 con 0x7f2a640cf560 2023-11-06T03:05:14.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a637fe640 1 -- 172.21.15.157:0/4119183135 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540abb40 con 0x7f2a640cf560 2023-11-06T03:05:14.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/4119183135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 msgr2=0x7f2a640b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/4119183135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009870 tx=0x7f2a5401c540 comp rx=0 tx=0).stop 2023-11-06T03:05:14.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/4119183135 shutdown_connections 2023-11-06T03:05:14.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/4119183135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a640cf560 0x7f2a640b2940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/4119183135 >> v2:172.21.15.195:3300/0 conn(0x7f2a6415ac80 0x7f2a640b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/4119183135 >> 172.21.15.157:0/4119183135 conn(0x7f2a6406d560 msgr2=0x7f2a6406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:14.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/4119183135 shutdown_connections 2023-11-06T03:05:14.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/4119183135 wait complete. 2023-11-06T03:05:14.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 Processor -- start 2023-11-06T03:05:14.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.775+0000 7f2a6b1c1640 1 -- start start 2023-11-06T03:05:14.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 0x7f2a640b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 0x7f2a64156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6414a160 con 0x7f2a640cf560 2023-11-06T03:05:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6415b6b0 con 0x7f2a640afb00 2023-11-06T03:05:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641548f0 con 0x7f2a6415ac80 2023-11-06T03:05:14.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56074/0 (socket says 172.21.15.157:56074) 2023-11-06T03:05:14.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 -- 172.21.15.157:0/3988262551 learned_addr learned my addr 172.21.15.157:0/3988262551 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:14.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a68f5d640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 0x7f2a640b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6975e640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 0x7f2a64156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 -- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 msgr2=0x7f2a64156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 0x7f2a64156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 -- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 msgr2=0x7f2a640b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 0x7f2a640b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 -- 172.21.15.157:0/3988262551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540094d0 con 0x7f2a640cf560 2023-11-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6975e640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 0x7f2a64156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a68f5d640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 0x7f2a640b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a63fff640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c770 tx=0x7f2a4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c003b80 con 0x7f2a640cf560 2023-11-06T03:05:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c003ce0 con 0x7f2a640cf560 2023-11-06T03:05:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a64154bd0 con 0x7f2a640cf560 2023-11-06T03:05:14.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00aa90 con 0x7f2a640cf560 2023-11-06T03:05:14.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a64155190 con 0x7f2a640cf560 2023-11-06T03:05:14.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a64072750 con 0x7f2a640cf560 2023-11-06T03:05:14.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2a4c00abf0 con 0x7f2a640cf560 2023-11-06T03:05:14.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 0x7f2a44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:14.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a4c083180 con 0x7f2a640cf560 2023-11-06T03:05:14.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.779+0000 7f2a68f5d640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 0x7f2a44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:14.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.783+0000 7f2a68f5d640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 0x7f2a44067f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009bb0 tx=0x7f2a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:14.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.783+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2a4c0509e0 con 0x7f2a640cf560 2023-11-06T03:05:14.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.943+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a6406e3e0 con 0x7f2a640cf560 2023-11-06T03:05:14.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a617fa640 1 -- 172.21.15.157:0/3988262551 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17944 (secure 0 0 0) 0x7f2a4c054000 con 0x7f2a640cf560 2023-11-06T03:05:14.947 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:14.948 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:05:14.601600+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:14.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 msgr2=0x7f2a44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 0x7f2a44067f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009bb0 tx=0x7f2a5c008040 comp rx=0 tx=0).stop 2023-11-06T03:05:14.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 msgr2=0x7f2a6415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:14.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c770 tx=0x7f2a4c00cc30 comp rx=0 tx=0).stop 2023-11-06T03:05:14.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 shutdown_connections 2023-11-06T03:05:14.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a44065a60 0x7f2a44067f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a6415ac80 0x7f2a64156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640cf560 0x7f2a6415b060 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 --2- 172.21.15.157:0/3988262551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a640afb00 0x7f2a640b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:14.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 >> 172.21.15.157:0/3988262551 conn(0x7f2a6406d560 msgr2=0x7f2a640cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:14.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 shutdown_connections 2023-11-06T03:05:14.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:14.947+0000 7f2a6b1c1640 1 -- 172.21.15.157:0/3988262551 wait complete. 2023-11-06T03:05:14.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.087+0000 7f8eb3c5e640 1 Processor -- start 2023-11-06T03:05:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.087+0000 7f8eb3c5e640 1 -- start start 2023-11-06T03:05:15.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb3c5e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eac15e5e0 0x7f8eac0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb3c5e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8eac15ea90 0x7f8eac0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb3c5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac15f580 con 0x7f8eac1551e0 2023-11-06T03:05:15.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb3c5e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac0704b0 con 0x7f8eac15e5e0 2023-11-06T03:05:15.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb3c5e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac15efc0 con 0x7f8eac15ea90 2023-11-06T03:05:15.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eac15e5e0 0x7f8eac0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eac15e5e0 0x7f8eac0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56078/0 (socket says 172.21.15.157:56078) 2023-11-06T03:05:15.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 -- 172.21.15.157:0/4124937238 learned_addr learned my addr 172.21.15.157:0/4124937238 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:15.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 -- 172.21.15.157:0/4124937238 >> v2:172.21.15.195:3300/0 conn(0x7f8eac15ea90 msgr2=0x7f8eac0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:15.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb21fb640 1 --2- 172.21.15.157:0/4124937238 >> v2:172.21.15.195:3300/0 conn(0x7f8eac15ea90 0x7f8eac0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/4124937238 >> v2:172.21.15.195:3300/0 conn(0x7f8eac15ea90 0x7f8eac0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 -- 172.21.15.157:0/4124937238 >> v1:172.21.15.195:6789/0 conn(0x7f8eac1551e0 legacy=0x7f8eac06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:15.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 -- 172.21.15.157:0/4124937238 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac15f2a0 con 0x7f8eac15e5e0 2023-11-06T03:05:15.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/4124937238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac15e5e0 0x7f8eac0b3410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0cc2f0 tx=0x7f8e9c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3937ca461a838 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb09f8640 1 -- 172.21.15.157:0/4124937238 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0ab600 con 0x7f8eac15e5e0 2023-11-06T03:05:15.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb09f8640 1 -- 172.21.15.157:0/4124937238 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c0ab760 con 0x7f8eac15e5e0 2023-11-06T03:05:15.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.091+0000 7f8eb09f8640 1 -- 172.21.15.157:0/4124937238 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0abaf0 con 0x7f8eac15e5e0 2023-11-06T03:05:15.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/4124937238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac15e5e0 msgr2=0x7f8eac0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/4124937238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac15e5e0 0x7f8eac0b3410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0cc2f0 tx=0x7f8e9c01c400 comp rx=0 tx=0).stop 2023-11-06T03:05:15.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/4124937238 shutdown_connections 2023-11-06T03:05:15.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/4124937238 >> v2:172.21.15.195:3300/0 conn(0x7f8eac15ea90 0x7f8eac0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/4124937238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac15e5e0 0x7f8eac0b3410 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/4124937238 >> 172.21.15.157:0/4124937238 conn(0x7f8eac065930 msgr2=0x7f8eac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:15.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/4124937238 shutdown_connections 2023-11-06T03:05:15.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/4124937238 wait complete. 2023-11-06T03:05:15.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 Processor -- start 2023-11-06T03:05:15.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- start start 2023-11-06T03:05:15.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 0x7f8eac14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eac15ea90 0x7f8eac152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac138e50 con 0x7f8eac1551e0 2023-11-06T03:05:15.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1390e0 con 0x7f8eac15ea90 2023-11-06T03:05:15.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac1549d0 con 0x7f8eac15e5e0 2023-11-06T03:05:15.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56094/0 (socket says 172.21.15.157:56094) 2023-11-06T03:05:15.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 -- 172.21.15.157:0/1425060075 learned_addr learned my addr 172.21.15.157:0/1425060075 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 0x7f8eac14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb21fb640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eac15ea90 0x7f8eac152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 -- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 msgr2=0x7f8eac14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 0x7f8eac14e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 -- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eac15ea90 msgr2=0x7f8eac152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eac15ea90 0x7f8eac152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 -- 172.21.15.157:0/1425060075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0094d0 con 0x7f8eac1551e0 2023-11-06T03:05:15.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 0x7f8eac14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:15.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb19fa640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c490 tx=0x7f8ea8001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003600 con 0x7f8eac1551e0 2023-11-06T03:05:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8003780 con 0x7f8eac1551e0 2023-11-06T03:05:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac153120 con 0x7f8eac1551e0 2023-11-06T03:05:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003b70 con 0x7f8eac1551e0 2023-11-06T03:05:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.095+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac0b1f90 con 0x7f8eac1551e0 2023-11-06T03:05:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac072750 con 0x7f8eac1551e0 2023-11-06T03:05:15.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ea8003cd0 con 0x7f8eac1551e0 2023-11-06T03:05:15.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8ea27fc640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 0x7f8e90067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ea80828e0 con 0x7f8eac1551e0 2023-11-06T03:05:15.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 0x7f8e90067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.099+0000 7f8eb11f9640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 0x7f8e90067f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0af000 tx=0x7f8e9c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.103+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8ea80501d0 con 0x7f8eac1551e0 2023-11-06T03:05:15.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.259+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eac0701f0 con 0x7f8eac1551e0 2023-11-06T03:05:15.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8ea27fc640 1 -- 172.21.15.157:0/1425060075 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+17944 (secure 0 0 0) 0x7f8ea8017070 con 0x7f8eac1551e0 2023-11-06T03:05:15.266 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:15.266 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:05:14.601600+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:15.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 msgr2=0x7f8e90067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 0x7f8e90067f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0af000 tx=0x7f8e9c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:05:15.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 msgr2=0x7f8eac0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c490 tx=0x7f8ea8001e80 comp rx=0 tx=0).stop 2023-11-06T03:05:15.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 shutdown_connections 2023-11-06T03:05:15.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e90065a60 0x7f8e90067f20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eac15ea90 0x7f8eac152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eac15e5e0 0x7f8eac14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 --2- 172.21.15.157:0/1425060075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eac1551e0 0x7f8eac0b16d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 >> 172.21.15.157:0/1425060075 conn(0x7f8eac065930 msgr2=0x7f8eac0aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:15.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 shutdown_connections 2023-11-06T03:05:15.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.263+0000 7f8eb3c5e640 1 -- 172.21.15.157:0/1425060075 wait complete. 2023-11-06T03:05:15.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:05:15.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.391+0000 7f56eedb1640 1 Processor -- start 2023-11-06T03:05:15.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.391+0000 7f56eedb1640 1 -- start start 2023-11-06T03:05:15.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f56e80cefc0 0x7f56e80cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 0x7f56e815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e8070880 0x7f56e815d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e815d9c0 con 0x7f56e8070880 2023-11-06T03:05:15.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80cf8e0 con 0x7f56e80cefc0 2023-11-06T03:05:15.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e815e220 con 0x7f56e8070420 2023-11-06T03:05:15.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e8070880 0x7f56e815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f56e8070880 0x7f56e815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56110/0 (socket says 172.21.15.157:56110) 2023-11-06T03:05:15.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 -- 172.21.15.157:0/677046692 learned_addr learned my addr 172.21.15.157:0/677046692 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:15.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56e7fff640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 0x7f56e815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ecb4d640 1 --2- 172.21.15.157:0/677046692 >> v2:172.21.15.195:3300/0 conn(0x7f56e80cefc0 0x7f56e80cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 -- 172.21.15.157:0/677046692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 msgr2=0x7f56e815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:15.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 0x7f56e815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 -- 172.21.15.157:0/677046692 >> v2:172.21.15.195:3300/0 conn(0x7f56e80cefc0 msgr2=0x7f56e80cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:15.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 --2- 172.21.15.157:0/677046692 >> v2:172.21.15.195:3300/0 conn(0x7f56e80cefc0 0x7f56e80cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 -- 172.21.15.157:0/677046692 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e815dc60 con 0x7f56e8070880 2023-11-06T03:05:15.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56e7fff640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 0x7f56e815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:15.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56ed34e640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070880 0x7f56e815d480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56e8071f60 tx=0x7f56e001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed132f8d58e77d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56e77fe640 1 -- 172.21.15.157:0/677046692 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00a5790 con 0x7f56e8070880 2023-11-06T03:05:15.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56e77fe640 1 -- 172.21.15.157:0/677046692 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e00a58f0 con 0x7f56e8070880 2023-11-06T03:05:15.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56e77fe640 1 -- 172.21.15.157:0/677046692 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00a5c80 con 0x7f56e8070880 2023-11-06T03:05:15.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- 172.21.15.157:0/677046692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070880 msgr2=0x7f56e815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070880 0x7f56e815d480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56e8071f60 tx=0x7f56e001c910 comp rx=0 tx=0).stop 2023-11-06T03:05:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- 172.21.15.157:0/677046692 shutdown_connections 2023-11-06T03:05:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070880 0x7f56e815d480 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- 172.21.15.157:0/677046692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070420 0x7f56e815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 --2- 172.21.15.157:0/677046692 >> v2:172.21.15.195:3300/0 conn(0x7f56e80cefc0 0x7f56e80cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- 172.21.15.157:0/677046692 >> 172.21.15.157:0/677046692 conn(0x7f56e806a4d0 msgr2=0x7f56e806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- 172.21.15.157:0/677046692 shutdown_connections 2023-11-06T03:05:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.395+0000 7f56eedb1640 1 -- 172.21.15.157:0/677046692 wait complete. 2023-11-06T03:05:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 Processor -- start 2023-11-06T03:05:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- start start 2023-11-06T03:05:15.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070880 0x7f56e80aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e80cefc0 0x7f56e80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80b0190 con 0x7f56e8070420 2023-11-06T03:05:15.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80af580 con 0x7f56e80cefc0 2023-11-06T03:05:15.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80af860 con 0x7f56e8070880 2023-11-06T03:05:15.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56116/0 (socket says 172.21.15.157:56116) 2023-11-06T03:05:15.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 -- 172.21.15.157:0/1482489928 learned_addr learned my addr 172.21.15.157:0/1482489928 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:15.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56e7fff640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070880 0x7f56e80aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ed34e640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e80cefc0 0x7f56e80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 -- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070880 msgr2=0x7f56e80aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070880 0x7f56e80aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 -- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e80cefc0 msgr2=0x7f56e80b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e80cefc0 0x7f56e80b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 -- 172.21.15.157:0/1482489928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e0009b30 con 0x7f56e8070420 2023-11-06T03:05:15.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56ecb4d640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c7f0 tx=0x7f56d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004220 con 0x7f56e8070420 2023-11-06T03:05:15.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80043a0 con 0x7f56e8070420 2023-11-06T03:05:15.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e80afb40 con 0x7f56e8070420 2023-11-06T03:05:15.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004510 con 0x7f56e8070420 2023-11-06T03:05:15.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e8161070 con 0x7f56e8070420 2023-11-06T03:05:15.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.399+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8072750 con 0x7f56e8070420 2023-11-06T03:05:15.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.403+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f56d8003a70 con 0x7f56e8070420 2023-11-06T03:05:15.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.403+0000 7f56e57fa640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 0x7f56d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.403+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56d8083270 con 0x7f56e8070420 2023-11-06T03:05:15.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.403+0000 7f56e7fff640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 0x7f56d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.403+0000 7f56e7fff640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 0x7f56d4067f20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.407+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f56d8050ac0 con 0x7f56e8070420 2023-11-06T03:05:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.579+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56e806c5c0 con 0x7f56e8070420 2023-11-06T03:05:15.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56e57fa640 1 -- 172.21.15.157:0/1482489928 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56d80540e0 con 0x7f56e8070420 2023-11-06T03:05:15.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 msgr2=0x7f56d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 0x7f56d4067f20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc008040 comp rx=0 tx=0).stop 2023-11-06T03:05:15.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 msgr2=0x7f56e8160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c7f0 tx=0x7f56d800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 shutdown_connections 2023-11-06T03:05:15.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f56d4065a60 0x7f56d4067f20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e80cefc0 0x7f56e80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e8070880 0x7f56e80aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 --2- 172.21.15.157:0/1482489928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56e8070420 0x7f56e8160b30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 >> 172.21.15.157:0/1482489928 conn(0x7f56e806a4d0 msgr2=0x7f56e8158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:15.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.615+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 shutdown_connections 2023-11-06T03:05:15.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.619+0000 7f56eedb1640 1 -- 172.21.15.157:0/1482489928 wait complete. 2023-11-06T03:05:15.634 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:05:15.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:05:15.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 Processor -- start 2023-11-06T03:05:15.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 -- start start 2023-11-06T03:05:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02281551e0 0x7f022806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f02280b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022815f580 con 0x7f022815ea60 2023-11-06T03:05:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022815efc0 con 0x7f02281551e0 2023-11-06T03:05:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f022c87c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022815f2a0 con 0x7f022815e5e0 2023-11-06T03:05:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f0226d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f022815ea60 0x7f022815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58558/0 (socket says 172.21.15.157:58558) 2023-11-06T03:05:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.751+0000 7f0226d9d640 1 -- 172.21.15.157:0/1868064247 learned_addr learned my addr 172.21.15.157:0/1868064247 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 --2- 172.21.15.157:0/1868064247 >> v2:172.21.15.157:3300/0 conn(0x7f02281551e0 0x7f022806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f0225d9b640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f02280b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022559a640 1 -- 172.21.15.157:0/1868064247 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175876925 0 0) 0x7f022815f580 con 0x7f022815ea60 2023-11-06T03:05:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022559a640 1 -- 172.21.15.157:0/1868064247 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0214003530 con 0x7f022815ea60 2023-11-06T03:05:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 -- 172.21.15.157:0/1868064247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 msgr2=0x7f02280b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f02280b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 -- 172.21.15.157:0/1868064247 >> v1:172.21.15.157:6789/0 conn(0x7f022815ea60 legacy=0x7f022815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 -- 172.21.15.157:0/1868064247 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022806f5a0 con 0x7f02281551e0 2023-11-06T03:05:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022659c640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f022806fda0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f02180097c0 tx=0x7f021801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e47c72cbbc3f4e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022559a640 1 -- 172.21.15.157:0/1868064247 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180ab600 con 0x7f02281551e0 2023-11-06T03:05:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022559a640 1 -- 172.21.15.157:0/1868064247 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180a7050 con 0x7f02281551e0 2023-11-06T03:05:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022559a640 1 -- 172.21.15.157:0/1868064247 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180ab980 con 0x7f02281551e0 2023-11-06T03:05:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 -- 172.21.15.157:0/1868064247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 msgr2=0x7f022806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f022806fda0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f02180097c0 tx=0x7f021801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 -- 172.21.15.157:0/1868064247 shutdown_connections 2023-11-06T03:05:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f02280b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 --2- 172.21.15.157:0/1868064247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f022806fda0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 -- 172.21.15.157:0/1868064247 >> 172.21.15.157:0/1868064247 conn(0x7f0228065930 msgr2=0x7f0228065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 -- 172.21.15.157:0/1868064247 shutdown_connections 2023-11-06T03:05:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.755+0000 7f022c87c640 1 -- 172.21.15.157:0/1868064247 wait complete. 2023-11-06T03:05:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 Processor -- start 2023-11-06T03:05:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- start start 2023-11-06T03:05:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f022814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022815ea60 0x7f0228159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0228158240 con 0x7f02281551e0 2023-11-06T03:05:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022814cb20 con 0x7f022815ea60 2023-11-06T03:05:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0228156150 con 0x7f022815e5e0 2023-11-06T03:05:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56144/0 (socket says 172.21.15.157:56144) 2023-11-06T03:05:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 -- 172.21.15.157:0/3558343099 learned_addr learned my addr 172.21.15.157:0/3558343099 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 -- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 msgr2=0x7f022814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f022814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 -- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022815ea60 msgr2=0x7f0228159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022815ea60 0x7f0228159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 -- 172.21.15.157:0/3558343099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009420 con 0x7f02281551e0 2023-11-06T03:05:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022659c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f02180ae040 tx=0x7f021801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180b7450 con 0x7f02281551e0 2023-11-06T03:05:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180b75b0 con 0x7f02281551e0 2023-11-06T03:05:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0228156430 con 0x7f02281551e0 2023-11-06T03:05:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180b7960 con 0x7f02281551e0 2023-11-06T03:05:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022814c8c0 con 0x7f02281551e0 2023-11-06T03:05:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228072750 con 0x7f02281551e0 2023-11-06T03:05:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f02180b7ac0 con 0x7f02281551e0 2023-11-06T03:05:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.759+0000 7f021effd640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 0x7f0208067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:15.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.763+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f021811dab0 con 0x7f02281551e0 2023-11-06T03:05:15.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.763+0000 7f0225d9b640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 0x7f0208067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:15.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.763+0000 7f0225d9b640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 0x7f0208067f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0210006aa0 tx=0x7f0210006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:15.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.767+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f02180eb410 con 0x7f02281551e0 2023-11-06T03:05:15.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:15.959+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0228071730 con 0x7f02281551e0 2023-11-06T03:05:16.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.619+0000 7f021effd640 1 -- 172.21.15.157:0/3558343099 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02180b00c0 con 0x7f02281551e0 2023-11-06T03:05:16.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 msgr2=0x7f0208067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 0x7f0208067f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0210006aa0 tx=0x7f0210006a30 comp rx=0 tx=0).stop 2023-11-06T03:05:16.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 msgr2=0x7f02280ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f02180ae040 tx=0x7f021801ac00 comp rx=0 tx=0).stop 2023-11-06T03:05:16.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 shutdown_connections 2023-11-06T03:05:16.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0208065a60 0x7f0208067f20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f022815ea60 0x7f0228159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f022815e5e0 0x7f022814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 --2- 172.21.15.157:0/3558343099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02281551e0 0x7f02280ce8a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 >> 172.21.15.157:0/3558343099 conn(0x7f0228065930 msgr2=0x7f02280b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:16.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 shutdown_connections 2023-11-06T03:05:16.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.623+0000 7f022c87c640 1 -- 172.21.15.157:0/3558343099 wait complete. 2023-11-06T03:05:16.646 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:05:16.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:16.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.771+0000 7fe3b821c640 1 Processor -- start 2023-11-06T03:05:16.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.771+0000 7fe3b821c640 1 -- start start 2023-11-06T03:05:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b821c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe3b00b3b20 0x7fe3b015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b821c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b015f580 con 0x7fe3b01551e0 2023-11-06T03:05:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b821c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b015efc0 con 0x7fe3b015e5e0 2023-11-06T03:05:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b821c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00704b0 con 0x7fe3b00b3b20 2023-11-06T03:05:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b5fb8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe3b01551e0 0x7fe3b006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58572/0 (socket says 172.21.15.157:58572) 2023-11-06T03:05:16.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b5fb8640 1 -- 172.21.15.157:0/491101288 learned_addr learned my addr 172.21.15.157:0/491101288 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4884905 0 0) 0x7fe3b015f580 con 0x7fe3b01551e0 2023-11-06T03:05:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c003530 con 0x7fe3b01551e0 2023-11-06T03:05:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> v2:172.21.15.195:3300/0 conn(0x7fe3b00b3b20 0x7fe3b015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 260330480 0 0) 0x7fe39c003530 con 0x7fe3b01551e0 2023-11-06T03:05:16.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 >> v2:172.21.15.195:3300/0 conn(0x7fe3b00b3b20 msgr2=0x7fe3b015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 --2- 172.21.15.157:0/491101288 >> v2:172.21.15.195:3300/0 conn(0x7fe3b00b3b20 0x7fe3b015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 >> v1:172.21.15.195:6789/0 conn(0x7fe3b015e5e0 legacy=0x7fe3b00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b015f2a0 con 0x7fe3b01551e0 2023-11-06T03:05:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe3ac001680 con 0x7fe3b01551e0 2023-11-06T03:05:16.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 >> v1:172.21.15.157:6789/0 conn(0x7fe3b01551e0 legacy=0x7fe3b006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c003980 0x7fe39c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe39c0047d0 0x7fe39c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 0x7fe39c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00704b0 con 0x7fe39c005390 2023-11-06T03:05:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c220 con 0x7fe39c0047d0 2023-11-06T03:05:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c3e0 con 0x7fe39c003980 2023-11-06T03:05:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> v2:172.21.15.195:3300/0 conn(0x7fe3b00b3b20 0x7fe3b015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 0x7fe39c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b57b7640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe39c0047d0 0x7fe39c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b5fb8640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c003980 0x7fe39c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 -- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c003980 msgr2=0x7fe39c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c003980 0x7fe39c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 -- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe39c0047d0 msgr2=0x7fe39c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe39c0047d0 0x7fe39c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 -- 172.21.15.157:0/491101288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c003530 con 0x7fe39c005390 2023-11-06T03:05:16.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b57b7640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe39c0047d0 0x7fe39c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 -- 172.21.15.157:0/491101288 reap_dead start 2023-11-06T03:05:16.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.775+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 0x7fe39c00b990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80073e0 tx=0x7fe3a801cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c442aeaadaab1696 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80a5df0 con 0x7fe39c005390 2023-11-06T03:05:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a80ad440 con 0x7fe39c005390 2023-11-06T03:05:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b4fb6640 1 -- 172.21.15.157:0/491101288 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80ad7f0 con 0x7fe39c005390 2023-11-06T03:05:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 msgr2=0x7fe39c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 0x7fe39c00b990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80073e0 tx=0x7fe3a801cef0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/491101288 shutdown_connections 2023-11-06T03:05:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 --2- 172.21.15.157:0/491101288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c005390 0x7fe39c00b990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/491101288 >> 172.21.15.157:0/491101288 conn(0x7fe3b0065930 msgr2=0x7fe3b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/491101288 shutdown_connections 2023-11-06T03:05:16.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/491101288 wait complete. 2023-11-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 Processor -- start 2023-11-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- start start 2023-11-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c005390 0x7fe3b0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3b014b970 0x7fe3b0152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0152a40 con 0x7fe3b0153410 2023-11-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b014c5c0 con 0x7fe3b014b970 2023-11-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0151e30 con 0x7fe39c005390 2023-11-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56162/0 (socket says 172.21.15.157:56162) 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 -- 172.21.15.157:0/2230326639 learned_addr learned my addr 172.21.15.157:0/2230326639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b5fb8640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c005390 0x7fe3b0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b57b7640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3b014b970 0x7fe3b0152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 -- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c005390 msgr2=0x7fe3b0151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c005390 0x7fe3b0151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 -- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3b014b970 msgr2=0x7fe3b0152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3b014b970 0x7fe3b0152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 -- 172.21.15.157:0/2230326639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0152110 con 0x7fe3b0153410 2023-11-06T03:05:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b67b9640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe3a801cd20 tx=0x7fe3a8019c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:16.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8009640 con 0x7fe3b0153410 2023-11-06T03:05:16.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a80adb90 con 0x7fe3b0153410 2023-11-06T03:05:16.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b01523f0 con 0x7fe3b0153410 2023-11-06T03:05:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80b7430 con 0x7fe3b0153410 2023-11-06T03:05:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.779+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b0154580 con 0x7fe3b0153410 2023-11-06T03:05:16.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0072750 con 0x7fe3b0153410 2023-11-06T03:05:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe3a80c03b0 con 0x7fe3b0153410 2023-11-06T03:05:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3a67fc640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 0x7fe394067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3a811d550 con 0x7fe3b0153410 2023-11-06T03:05:16.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3b5fb8640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 0x7fe394067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.783+0000 7fe3b5fb8640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 0x7fe394067f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000c10 tx=0x7fe3ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:16.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.787+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe3a80eaef0 con 0x7fe3b0153410 2023-11-06T03:05:16.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3b0152b80 con 0x7fe3b0153410 2023-11-06T03:05:16.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3a67fc640 1 -- 172.21.15.157:0/2230326639 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fe3a80ee510 con 0x7fe3b0153410 2023-11-06T03:05:16.961 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:16.961 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:05:16.614800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:05:16.961 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 13 2023-11-06T03:05:16.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 msgr2=0x7fe394067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 0x7fe394067f20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000c10 tx=0x7fe3ac009040 comp rx=0 tx=0).stop 2023-11-06T03:05:16.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 msgr2=0x7fe3b0154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:16.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe3a801cd20 tx=0x7fe3a8019c90 comp rx=0 tx=0).stop 2023-11-06T03:05:16.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 shutdown_connections 2023-11-06T03:05:16.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe394065a60 0x7fe394067f20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0153410 0x7fe3b0154040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3b014b970 0x7fe3b0152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 --2- 172.21.15.157:0/2230326639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe39c005390 0x7fe3b0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:16.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 >> 172.21.15.157:0/2230326639 conn(0x7fe3b0065930 msgr2=0x7fe3b0150830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:16.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 shutdown_connections 2023-11-06T03:05:16.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:16.959+0000 7fe3b821c640 1 -- 172.21.15.157:0/2230326639 wait complete. 2023-11-06T03:05:16.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:05:17.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 Processor -- start 2023-11-06T03:05:17.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 -- start start 2023-11-06T03:05:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6c80cca50 0x7fa6c80c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa6c80cced0 0x7fa6c80c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c806bfc0 con 0x7fa6c815d3a0 2023-11-06T03:05:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c80ce990 con 0x7fa6c80cca50 2023-11-06T03:05:17.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6d0669640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c806b970 con 0x7fa6c80cced0 2023-11-06T03:05:17.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6c80cca50 0x7fa6c80c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6c80cca50 0x7fa6c80c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56170/0 (socket says 172.21.15.157:56170) 2023-11-06T03:05:17.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 -- 172.21.15.157:0/1669087267 learned_addr learned my addr 172.21.15.157:0/1669087267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cec06640 1 --2- 172.21.15.157:0/1669087267 >> v2:172.21.15.195:3300/0 conn(0x7fa6c80cced0 0x7fa6c80c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 -- 172.21.15.157:0/1669087267 >> v2:172.21.15.195:3300/0 conn(0x7fa6c80cced0 msgr2=0x7fa6c80c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 --2- 172.21.15.157:0/1669087267 >> v2:172.21.15.195:3300/0 conn(0x7fa6c80cced0 0x7fa6c80c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 -- 172.21.15.157:0/1669087267 >> v1:172.21.15.195:6789/0 conn(0x7fa6c815d3a0 legacy=0x7fa6c815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.083+0000 7fa6cdc04640 1 -- 172.21.15.157:0/1669087267 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c806bc50 con 0x7fa6c80cca50 2023-11-06T03:05:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- 172.21.15.157:0/1669087267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cca50 0x7fa6c80c94d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80c9460 tx=0x7fa6c401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300face5ea026115 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cd403640 1 -- 172.21.15.157:0/1669087267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ab600 con 0x7fa6c80cca50 2023-11-06T03:05:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cd403640 1 -- 172.21.15.157:0/1669087267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40a7050 con 0x7fa6c80cca50 2023-11-06T03:05:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cd403640 1 -- 172.21.15.157:0/1669087267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ab980 con 0x7fa6c80cca50 2023-11-06T03:05:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- 172.21.15.157:0/1669087267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cca50 msgr2=0x7fa6c80c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- 172.21.15.157:0/1669087267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cca50 0x7fa6c80c94d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80c9460 tx=0x7fa6c401c4c0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- 172.21.15.157:0/1669087267 shutdown_connections 2023-11-06T03:05:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- 172.21.15.157:0/1669087267 >> v2:172.21.15.195:3300/0 conn(0x7fa6c80cced0 0x7fa6c80c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- 172.21.15.157:0/1669087267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cca50 0x7fa6c80c94d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- 172.21.15.157:0/1669087267 >> 172.21.15.157:0/1669087267 conn(0x7fa6c80cb990 msgr2=0x7fa6c80cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- 172.21.15.157:0/1669087267 shutdown_connections 2023-11-06T03:05:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- 172.21.15.157:0/1669087267 wait complete. 2023-11-06T03:05:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 Processor -- start 2023-11-06T03:05:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- start start 2023-11-06T03:05:17.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6c80cca50 0x7fa6c80acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6c815d3a0 0x7fa6c8153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80b1b10 con 0x7fa6c80cced0 2023-11-06T03:05:17.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80c3540 con 0x7fa6c815d3a0 2023-11-06T03:05:17.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6d0669640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80c37e0 con 0x7fa6c80cca50 2023-11-06T03:05:17.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56174/0 (socket says 172.21.15.157:56174) 2023-11-06T03:05:17.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 -- 172.21.15.157:0/3477915317 learned_addr learned my addr 172.21.15.157:0/3477915317 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 -- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6c80cca50 msgr2=0x7fa6c80acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6c80cca50 0x7fa6c80acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 -- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6c815d3a0 msgr2=0x7fa6c8153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6c815d3a0 0x7fa6c8153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 -- 172.21.15.157:0/3477915317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40094d0 con 0x7fa6c80cced0 2023-11-06T03:05:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.087+0000 7fa6cdc04640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa6c401ab70 tx=0x7fa6c4019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ac220 con 0x7fa6c80cced0 2023-11-06T03:05:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40aa5b0 con 0x7fa6c80cced0 2023-11-06T03:05:17.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c80c3ac0 con 0x7fa6c80cced0 2023-11-06T03:05:17.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40aa960 con 0x7fa6c80cced0 2023-11-06T03:05:17.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c80c4080 con 0x7fa6c80cced0 2023-11-06T03:05:17.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c80c4270 con 0x7fa6c80cced0 2023-11-06T03:05:17.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6c40b2070 con 0x7fa6c80cced0 2023-11-06T03:05:17.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 0x7fa6ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6c411cad0 con 0x7fa6c80cced0 2023-11-06T03:05:17.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6ce405640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 0x7fa6ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.091+0000 7fa6ce405640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 0x7fa6ac067f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009670 tx=0x7fa6b8006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.095+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa6c40ea330 con 0x7fa6c80cced0 2023-11-06T03:05:17.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.255+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c8072750 con 0x7fa6c80cced0 2023-11-06T03:05:17.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.255+0000 7fa6beffd640 1 -- 172.21.15.157:0/3477915317 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+17931 (secure 0 0 0) 0x7fa6c40b0020 con 0x7fa6c80cced0 2023-11-06T03:05:17.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:17.259 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":37,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:05:15.606362+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:05:17.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 msgr2=0x7fa6ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 0x7fa6ac067f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009670 tx=0x7fa6b8006bd0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 msgr2=0x7fa6c80c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa6c401ab70 tx=0x7fa6c4019260 comp rx=0 tx=0).stop 2023-11-06T03:05:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 shutdown_connections 2023-11-06T03:05:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa6ac065a60 0x7fa6ac067f20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6c815d3a0 0x7fa6c8153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c80cced0 0x7fa6c80c8740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 --2- 172.21.15.157:0/3477915317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6c80cca50 0x7fa6c80acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 >> 172.21.15.157:0/3477915317 conn(0x7fa6c80cb990 msgr2=0x7fa6c806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 shutdown_connections 2023-11-06T03:05:17.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.259+0000 7fa6d0669640 1 -- 172.21.15.157:0/3477915317 wait complete. 2023-11-06T03:05:17.277 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:05:17.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:05:17.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 Processor -- start 2023-11-06T03:05:17.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 -- start start 2023-11-06T03:05:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 0x7fe80c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe80c0adf40 0x7fe80c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80c06b950 con 0x7fe80c0adf40 2023-11-06T03:05:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80c06c490 con 0x7fe80c156a70 2023-11-06T03:05:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe813eaf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c06bb60 con 0x7fe80c0adae0 2023-11-06T03:05:17.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe80c0adf40 0x7fe80c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe80c0adf40 0x7fe80c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56180/0 (socket says 172.21.15.157:56180) 2023-11-06T03:05:17.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 -- 172.21.15.157:0/2751887025 learned_addr learned my addr 172.21.15.157:0/2751887025 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe811c4b640 1 --2- 172.21.15.157:0/2751887025 >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 0x7fe80c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81144a640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 -- 172.21.15.157:0/2751887025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 msgr2=0x7fe80c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 -- 172.21.15.157:0/2751887025 >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 msgr2=0x7fe80c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 --2- 172.21.15.157:0/2751887025 >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 0x7fe80c06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe81244c640 1 -- 172.21.15.157:0/2751887025 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c06be40 con 0x7fe80c0adf40 2023-11-06T03:05:17.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.399+0000 7fe811c4b640 1 --2- 172.21.15.157:0/2751887025 >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 0x7fe80c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:17.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81244c640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b2160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0ce880 tx=0x7fe80401daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f0c5ac3ac0ccce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe810c49640 1 -- 172.21.15.157:0/2751887025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400a2d0 con 0x7fe80c0adf40 2023-11-06T03:05:17.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe810c49640 1 -- 172.21.15.157:0/2751887025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8040a68e0 con 0x7fe80c0adf40 2023-11-06T03:05:17.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe810c49640 1 -- 172.21.15.157:0/2751887025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8040a6c70 con 0x7fe80c0adf40 2023-11-06T03:05:17.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/2751887025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 msgr2=0x7fe80c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b2160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0ce880 tx=0x7fe80401daa0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/2751887025 shutdown_connections 2023-11-06T03:05:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b2160 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- 172.21.15.157:0/2751887025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- 172.21.15.157:0/2751887025 >> v2:172.21.15.195:3300/0 conn(0x7fe80c156a70 0x7fe80c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/2751887025 >> 172.21.15.157:0/2751887025 conn(0x7fe80c065930 msgr2=0x7fe80c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/2751887025 shutdown_connections 2023-11-06T03:05:17.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/2751887025 wait complete. 2023-11-06T03:05:17.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 Processor -- start 2023-11-06T03:05:17.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- start start 2023-11-06T03:05:17.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c156a70 0x7fe80c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c155750 con 0x7fe80c0adf40 2023-11-06T03:05:17.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0709c0 con 0x7fe80c156a70 2023-11-06T03:05:17.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c1546b0 con 0x7fe80c0adae0 2023-11-06T03:05:17.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56190/0 (socket says 172.21.15.157:56190) 2023-11-06T03:05:17.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 -- 172.21.15.157:0/3464757079 learned_addr learned my addr 172.21.15.157:0/3464757079 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81244c640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c156a70 0x7fe80c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe811c4b640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 -- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 msgr2=0x7fe80c0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 -- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c156a70 msgr2=0x7fe80c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c156a70 0x7fe80c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 -- 172.21.15.157:0/3464757079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80400ac40 con 0x7fe80c0adf40 2023-11-06T03:05:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe81144a640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d800 tx=0x7fe7fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00b6c0 con 0x7fe80c0adf40 2023-11-06T03:05:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc00b820 con 0x7fe80c0adf40 2023-11-06T03:05:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c154990 con 0x7fe80c0adf40 2023-11-06T03:05:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00bb90 con 0x7fe80c0adf40 2023-11-06T03:05:17.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.403+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c154f50 con 0x7fe80c0adf40 2023-11-06T03:05:17.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c072750 con 0x7fe80c0adf40 2023-11-06T03:05:17.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe7fc00bcf0 con 0x7fe80c0adf40 2023-11-06T03:05:17.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe8027fc640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 0x7fe7f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7fc083910 con 0x7fe80c0adf40 2023-11-06T03:05:17.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe811c4b640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 0x7fe7f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.407+0000 7fe811c4b640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 0x7fe7f0067f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe808006fd0 tx=0x7fe808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.411+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe7fc0510e0 con 0x7fe80c0adf40 2023-11-06T03:05:17.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.555+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe80c06f9b0 con 0x7fe7f0065a60 2023-11-06T03:05:17.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.579+0000 7fe8027fc640 1 -- 172.21.15.157:0/3464757079 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299183 (secure 0 0 0) 0x7fe80800c050 con 0x7fe7f0065a60 2023-11-06T03:05:17.583 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:17.585 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:05:17.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 msgr2=0x7fe7f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 0x7fe7f0067f20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe808006fd0 tx=0x7fe808008040 comp rx=0 tx=0).stop 2023-11-06T03:05:17.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 msgr2=0x7fe80c0b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d800 tx=0x7fe7fc00dcc0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 shutdown_connections 2023-11-06T03:05:17.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f0065a60 0x7fe7f0067f20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe80c156a70 0x7fe80c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe80c0adf40 0x7fe80c0b0c20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 --2- 172.21.15.157:0/3464757079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe80c0adae0 0x7fe80c0b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 >> 172.21.15.157:0/3464757079 conn(0x7fe80c065930 msgr2=0x7fe80c157480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 shutdown_connections 2023-11-06T03:05:17.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.583+0000 7fe813eaf640 1 -- 172.21.15.157:0/3464757079 wait complete. 2023-11-06T03:05:17.602 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-06T03:05:17.118153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252540,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754722180,"statfs":{"total":773094113280,"available":772835512320,"internally_reserved":0,"allocated":4255744,"data_stored":2416119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231100+0000","last_change":"2023-11-06T03:05:13.231100+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231100+0000","last_undegraded":"2023-11-06T03:05:13.231100+0000","last_fullsized":"2023-11-06T03:05:13.231100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229773+0000","last_change":"2023-11-06T03:05:13.229773+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229773+0000","last_undegraded":"2023-11-06T03:05:13.229773+0000","last_fullsized":"2023-11-06T03:05:13.229773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230577+0000","last_change":"2023-11-06T03:05:13.230577+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230577+0000","last_undegraded":"2023-11-06T03:05:13.230577+0000","last_fullsized":"2023-11-06T03:05:13.230577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228983+0000","last_change":"2023-11-06T03:05:13.228983+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228983+0000","last_undegraded":"2023-11-06T03:05:13.228983+0000","last_fullsized":"2023-11-06T03:05:13.228983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235683+0000","last_change":"2023-11-06T03:05:13.235683+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235683+0000","last_undegraded":"2023-11-06T03:05:13.235683+0000","last_fullsized":"2023-11-06T03:05:13.235683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224107+0000","last_change":"2023-11-06T03:05:13.224107+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224107+0000","last_undegraded":"2023-11-06T03:05:13.224107+0000","last_fullsized":"2023-11-06T03:05:13.224107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230651+0000","last_change":"2023-11-06T03:05:13.230651+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230651+0000","last_undegraded":"2023-11-06T03:05:13.230651+0000","last_fullsized":"2023-11-06T03:05:13.230651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228932+0000","last_change":"2023-11-06T03:05:13.228932+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228932+0000","last_undegraded":"2023-11-06T03:05:13.228932+0000","last_fullsized":"2023-11-06T03:05:13.228932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225336+0000","last_change":"2023-11-06T03:05:13.225336+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225336+0000","last_undegraded":"2023-11-06T03:05:13.225336+0000","last_fullsized":"2023-11-06T03:05:13.225336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235798+0000","last_change":"2023-11-06T03:05:13.235798+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235798+0000","last_undegraded":"2023-11-06T03:05:13.235798+0000","last_fullsized":"2023-11-06T03:05:13.235798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238487+0000","last_change":"2023-11-06T03:05:13.238487+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238487+0000","last_undegraded":"2023-11-06T03:05:13.238487+0000","last_fullsized":"2023-11-06T03:05:13.238487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225520+0000","last_change":"2023-11-06T03:05:13.225520+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225520+0000","last_undegraded":"2023-11-06T03:05:13.225520+0000","last_fullsized":"2023-11-06T03:05:13.225520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.226090+0000","last_change":"2023-11-06T03:05:13.226090+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.226090+0000","last_undegraded":"2023-11-06T03:05:13.226090+0000","last_fullsized":"2023-11-06T03:05:13.226090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229794+0000","last_change":"2023-11-06T03:05:13.229794+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229794+0000","last_undegraded":"2023-11-06T03:05:13.229794+0000","last_fullsized":"2023-11-06T03:05:13.229794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230501+0000","last_change":"2023-11-06T03:05:13.230501+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230501+0000","last_undegraded":"2023-11-06T03:05:13.230501+0000","last_fullsized":"2023-11-06T03:05:13.230501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.223902+0000","last_change":"2023-11-06T03:05:13.223902+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.223902+0000","last_undegraded":"2023-11-06T03:05:13.223902+0000","last_fullsized":"2023-11-06T03:05:13.223902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238099+0000","last_change":"2023-11-06T03:05:13.238099+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238099+0000","last_undegraded":"2023-11-06T03:05:13.238099+0000","last_fullsized":"2023-11-06T03:05:13.238099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229038+0000","last_change":"2023-11-06T03:05:13.229038+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229038+0000","last_undegraded":"2023-11-06T03:05:13.229038+0000","last_fullsized":"2023-11-06T03:05:13.229038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229509+0000","last_change":"2023-11-06T03:05:13.229509+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229509+0000","last_undegraded":"2023-11-06T03:05:13.229509+0000","last_fullsized":"2023-11-06T03:05:13.229509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231213+0000","last_change":"2023-11-06T03:05:13.231213+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231213+0000","last_undegraded":"2023-11-06T03:05:13.231213+0000","last_fullsized":"2023-11-06T03:05:13.231213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230423+0000","last_change":"2023-11-06T03:05:13.230423+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230423+0000","last_undegraded":"2023-11-06T03:05:13.230423+0000","last_fullsized":"2023-11-06T03:05:13.230423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229513+0000","last_change":"2023-11-06T03:05:13.229513+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229513+0000","last_undegraded":"2023-11-06T03:05:13.229513+0000","last_fullsized":"2023-11-06T03:05:13.229513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224151+0000","last_change":"2023-11-06T03:05:13.224151+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224151+0000","last_undegraded":"2023-11-06T03:05:13.224151+0000","last_fullsized":"2023-11-06T03:05:13.224151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229678+0000","last_change":"2023-11-06T03:05:13.229678+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229678+0000","last_undegraded":"2023-11-06T03:05:13.229678+0000","last_fullsized":"2023-11-06T03:05:13.229678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224119+0000","last_change":"2023-11-06T03:05:13.224119+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224119+0000","last_undegraded":"2023-11-06T03:05:13.224119+0000","last_fullsized":"2023-11-06T03:05:13.224119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007919+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:05:09.007919+0000","last_peered":"2023-11-06T03:05:09.007919+0000","last_clean":"2023-11-06T03:05:09.007919+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:05:09.007919+0000","last_undegraded":"2023-11-06T03:05:09.007919+0000","last_fullsized":"2023-11-06T03:05:09.007919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829794+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:05:09.829794+0000","last_peered":"2023-11-06T03:05:09.829794+0000","last_clean":"2023-11-06T03:05:09.829794+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:05:09.829794+0000","last_undegraded":"2023-11-06T03:05:09.829794+0000","last_fullsized":"2023-11-06T03:05:09.829794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.237064+0000","last_change":"2023-11-06T03:05:13.237064+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.237064+0000","last_undegraded":"2023-11-06T03:05:13.237064+0000","last_fullsized":"2023-11-06T03:05:13.237064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.008070+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:05:09.008070+0000","last_peered":"2023-11-06T03:05:09.008070+0000","last_clean":"2023-11-06T03:05:09.008070+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:05:09.008070+0000","last_undegraded":"2023-11-06T03:05:09.008070+0000","last_fullsized":"2023-11-06T03:05:09.008070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230034+0000","last_change":"2023-11-06T03:05:13.230034+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230034+0000","last_undegraded":"2023-11-06T03:05:13.230034+0000","last_fullsized":"2023-11-06T03:05:13.230034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330499+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:05:10.330499+0000","last_peered":"2023-11-06T03:05:10.330499+0000","last_clean":"2023-11-06T03:05:10.330499+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:05:10.330499+0000","last_undegraded":"2023-11-06T03:05:10.330499+0000","last_fullsized":"2023-11-06T03:05:10.330499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230260+0000","last_change":"2023-11-06T03:05:13.230260+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230260+0000","last_undegraded":"2023-11-06T03:05:13.230260+0000","last_fullsized":"2023-11-06T03:05:13.230260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829786+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:05:09.829786+0000","last_peered":"2023-11-06T03:05:09.829786+0000","last_clean":"2023-11-06T03:05:09.829786+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:05:09.829786+0000","last_undegraded":"2023-11-06T03:05:09.829786+0000","last_fullsized":"2023-11-06T03:05:09.829786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229604+0000","last_change":"2023-11-06T03:05:13.229604+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229604+0000","last_undegraded":"2023-11-06T03:05:13.229604+0000","last_fullsized":"2023-11-06T03:05:13.229604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499031+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:05:09.499031+0000","last_peered":"2023-11-06T03:05:09.499031+0000","last_clean":"2023-11-06T03:05:09.499031+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:05:09.499031+0000","last_undegraded":"2023-11-06T03:05:09.499031+0000","last_fullsized":"2023-11-06T03:05:09.499031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230514+0000","last_change":"2023-11-06T03:05:13.230514+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230514+0000","last_undegraded":"2023-11-06T03:05:13.230514+0000","last_fullsized":"2023-11-06T03:05:13.230514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007975+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:05:09.007975+0000","last_peered":"2023-11-06T03:05:09.007975+0000","last_clean":"2023-11-06T03:05:09.007975+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:05:09.007975+0000","last_undegraded":"2023-11-06T03:05:09.007975+0000","last_fullsized":"2023-11-06T03:05:09.007975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228945+0000","last_change":"2023-11-06T03:05:13.228945+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228945+0000","last_undegraded":"2023-11-06T03:05:13.228945+0000","last_fullsized":"2023-11-06T03:05:13.228945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499057+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:05:09.499057+0000","last_peered":"2023-11-06T03:05:09.499057+0000","last_clean":"2023-11-06T03:05:09.499057+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:05:09.499057+0000","last_undegraded":"2023-11-06T03:05:09.499057+0000","last_fullsized":"2023-11-06T03:05:09.499057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225413+0000","last_change":"2023-11-06T03:05:13.225413+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225413+0000","last_undegraded":"2023-11-06T03:05:13.225413+0000","last_fullsized":"2023-11-06T03:05:13.225413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330252+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:05:10.330252+0000","last_peered":"2023-11-06T03:05:10.330252+0000","last_clean":"2023-11-06T03:05:10.330252+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:05:10.330252+0000","last_undegraded":"2023-11-06T03:05:10.330252+0000","last_fullsized":"2023-11-06T03:05:10.330252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":745173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0229999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.93899999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.052}]},{"osd":4,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.90500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.45600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.026},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.98899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.876},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.133},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.1559999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82299999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.008},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.032}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.016}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.083},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.95299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.88200000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.90200000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.873},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.083}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.749},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.505}]},{"osd":3,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.0329999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":974848,"data_stored":744154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.26200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.755}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:05:17.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:05:17.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.767+0000 7fe81dadc640 1 Processor -- start 2023-11-06T03:05:17.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.767+0000 7fe81dadc640 1 -- start start 2023-11-06T03:05:17.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe81815caa0 0x7fe8180cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8180ce1a0 0x7fe8180d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8180d2920 con 0x7fe81815caa0 2023-11-06T03:05:17.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8180d18a0 con 0x7fe8180ce1a0 2023-11-06T03:05:17.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe81dadc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180d1b60 con 0x7fe81815ce80 2023-11-06T03:05:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe81815caa0 0x7fe8180cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe81815caa0 0x7fe8180cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56206/0 (socket says 172.21.15.157:56206) 2023-11-06T03:05:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 -- 172.21.15.157:0/548935451 learned_addr learned my addr 172.21.15.157:0/548935451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe817fff640 1 --2- 172.21.15.157:0/548935451 >> v2:172.21.15.195:3300/0 conn(0x7fe8180ce1a0 0x7fe8180d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe816ffd640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 -- 172.21.15.157:0/548935451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 msgr2=0x7fe8180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 -- 172.21.15.157:0/548935451 >> v2:172.21.15.195:3300/0 conn(0x7fe8180ce1a0 msgr2=0x7fe8180d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 --2- 172.21.15.157:0/548935451 >> v2:172.21.15.195:3300/0 conn(0x7fe8180ce1a0 0x7fe8180d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.771+0000 7fe8177fe640 1 -- 172.21.15.157:0/548935451 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180d1e40 con 0x7fe81815caa0 2023-11-06T03:05:17.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe8177fe640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8180cdc60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8180d0780 tx=0x7fe81001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a80537cb41d90 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe8167fc640 1 -- 172.21.15.157:0/548935451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a5820 con 0x7fe81815caa0 2023-11-06T03:05:17.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe8167fc640 1 -- 172.21.15.157:0/548935451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8100a5980 con 0x7fe81815caa0 2023-11-06T03:05:17.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe8167fc640 1 -- 172.21.15.157:0/548935451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a5d10 con 0x7fe81815caa0 2023-11-06T03:05:17.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- 172.21.15.157:0/548935451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 msgr2=0x7fe8180cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8180cdc60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8180d0780 tx=0x7fe81001c9a0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- 172.21.15.157:0/548935451 shutdown_connections 2023-11-06T03:05:17.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- 172.21.15.157:0/548935451 >> v2:172.21.15.195:3300/0 conn(0x7fe8180ce1a0 0x7fe8180d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- 172.21.15.157:0/548935451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8180cdc60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- 172.21.15.157:0/548935451 >> 172.21.15.157:0/548935451 conn(0x7fe818070790 msgr2=0x7fe818070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- 172.21.15.157:0/548935451 shutdown_connections 2023-11-06T03:05:17.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- 172.21.15.157:0/548935451 wait complete. 2023-11-06T03:05:17.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 Processor -- start 2023-11-06T03:05:17.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- start start 2023-11-06T03:05:17.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8180ce1a0 0x7fe81815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81815d280 con 0x7fe81815caa0 2023-11-06T03:05:17.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81815d490 con 0x7fe8180ce1a0 2023-11-06T03:05:17.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe81dadc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81815d770 con 0x7fe81815ce80 2023-11-06T03:05:17.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe816ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe816ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56218/0 (socket says 172.21.15.157:56218) 2023-11-06T03:05:17.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.775+0000 7fe816ffd640 1 -- 172.21.15.157:0/2255534103 learned_addr learned my addr 172.21.15.157:0/2255534103 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:17.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe817fff640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe8177fe640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8180ce1a0 0x7fe81815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 -- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 msgr2=0x7fe8180cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 -- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8180ce1a0 msgr2=0x7fe81815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8180ce1a0 0x7fe81815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 -- 172.21.15.157:0/2255534103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009b30 con 0x7fe81815caa0 2023-11-06T03:05:17.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe817fff640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:17.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe816ffd640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe80000d800 tx=0x7fe80000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000b6c0 con 0x7fe81815caa0 2023-11-06T03:05:17.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000b820 con 0x7fe81815caa0 2023-11-06T03:05:17.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000bb90 con 0x7fe81815caa0 2023-11-06T03:05:17.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81815da50 con 0x7fe81815caa0 2023-11-06T03:05:17.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8180cc750 con 0x7fe81815caa0 2023-11-06T03:05:17.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe818072750 con 0x7fe81815caa0 2023-11-06T03:05:17.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe80000bcf0 con 0x7fe81815caa0 2023-11-06T03:05:17.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 0x7fe7f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:17.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8000837b0 con 0x7fe81815caa0 2023-11-06T03:05:17.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe8177fe640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 0x7fe7f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:17.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.779+0000 7fe8177fe640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 0x7fe7f8067f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8100ae000 tx=0x7fe8100a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:17.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.787+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe8000510a0 con 0x7fe81815caa0 2023-11-06T03:05:17.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.935+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8180b1f00 con 0x7fe7f8065a60 2023-11-06T03:05:17.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.959+0000 7fe80ffff640 1 -- 172.21.15.157:0/2255534103 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299183 (secure 0 0 0) 0x7fe8100091d0 con 0x7fe7f8065a60 2023-11-06T03:05:17.963 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:17.964 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:05:17.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 msgr2=0x7fe7f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 0x7fe7f8067f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8100ae000 tx=0x7fe8100a7070 comp rx=0 tx=0).stop 2023-11-06T03:05:17.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 msgr2=0x7fe8181f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:17.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe80000d800 tx=0x7fe80000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 shutdown_connections 2023-11-06T03:05:17.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe7f8065a60 0x7fe7f8067f20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe81815ce80 0x7fe8180cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe81815caa0 0x7fe8181f53e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 --2- 172.21.15.157:0/2255534103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8180ce1a0 0x7fe81815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:17.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 >> 172.21.15.157:0/2255534103 conn(0x7fe818070790 msgr2=0x7fe8180b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:17.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 shutdown_connections 2023-11-06T03:05:17.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:17.963+0000 7fe81dadc640 1 -- 172.21.15.157:0/2255534103 wait complete. 2023-11-06T03:05:17.980 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-06T03:05:17.118153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252540,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754722180,"statfs":{"total":773094113280,"available":772835512320,"internally_reserved":0,"allocated":4255744,"data_stored":2416119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231100+0000","last_change":"2023-11-06T03:05:13.231100+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231100+0000","last_undegraded":"2023-11-06T03:05:13.231100+0000","last_fullsized":"2023-11-06T03:05:13.231100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229773+0000","last_change":"2023-11-06T03:05:13.229773+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229773+0000","last_undegraded":"2023-11-06T03:05:13.229773+0000","last_fullsized":"2023-11-06T03:05:13.229773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230577+0000","last_change":"2023-11-06T03:05:13.230577+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230577+0000","last_undegraded":"2023-11-06T03:05:13.230577+0000","last_fullsized":"2023-11-06T03:05:13.230577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228983+0000","last_change":"2023-11-06T03:05:13.228983+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228983+0000","last_undegraded":"2023-11-06T03:05:13.228983+0000","last_fullsized":"2023-11-06T03:05:13.228983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235683+0000","last_change":"2023-11-06T03:05:13.235683+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235683+0000","last_undegraded":"2023-11-06T03:05:13.235683+0000","last_fullsized":"2023-11-06T03:05:13.235683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224107+0000","last_change":"2023-11-06T03:05:13.224107+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224107+0000","last_undegraded":"2023-11-06T03:05:13.224107+0000","last_fullsized":"2023-11-06T03:05:13.224107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230651+0000","last_change":"2023-11-06T03:05:13.230651+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230651+0000","last_undegraded":"2023-11-06T03:05:13.230651+0000","last_fullsized":"2023-11-06T03:05:13.230651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228932+0000","last_change":"2023-11-06T03:05:13.228932+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228932+0000","last_undegraded":"2023-11-06T03:05:13.228932+0000","last_fullsized":"2023-11-06T03:05:13.228932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225336+0000","last_change":"2023-11-06T03:05:13.225336+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225336+0000","last_undegraded":"2023-11-06T03:05:13.225336+0000","last_fullsized":"2023-11-06T03:05:13.225336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235798+0000","last_change":"2023-11-06T03:05:13.235798+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235798+0000","last_undegraded":"2023-11-06T03:05:13.235798+0000","last_fullsized":"2023-11-06T03:05:13.235798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238487+0000","last_change":"2023-11-06T03:05:13.238487+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238487+0000","last_undegraded":"2023-11-06T03:05:13.238487+0000","last_fullsized":"2023-11-06T03:05:13.238487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225520+0000","last_change":"2023-11-06T03:05:13.225520+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225520+0000","last_undegraded":"2023-11-06T03:05:13.225520+0000","last_fullsized":"2023-11-06T03:05:13.225520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.226090+0000","last_change":"2023-11-06T03:05:13.226090+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.226090+0000","last_undegraded":"2023-11-06T03:05:13.226090+0000","last_fullsized":"2023-11-06T03:05:13.226090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229794+0000","last_change":"2023-11-06T03:05:13.229794+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229794+0000","last_undegraded":"2023-11-06T03:05:13.229794+0000","last_fullsized":"2023-11-06T03:05:13.229794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230501+0000","last_change":"2023-11-06T03:05:13.230501+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230501+0000","last_undegraded":"2023-11-06T03:05:13.230501+0000","last_fullsized":"2023-11-06T03:05:13.230501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.223902+0000","last_change":"2023-11-06T03:05:13.223902+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.223902+0000","last_undegraded":"2023-11-06T03:05:13.223902+0000","last_fullsized":"2023-11-06T03:05:13.223902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238099+0000","last_change":"2023-11-06T03:05:13.238099+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238099+0000","last_undegraded":"2023-11-06T03:05:13.238099+0000","last_fullsized":"2023-11-06T03:05:13.238099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229038+0000","last_change":"2023-11-06T03:05:13.229038+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229038+0000","last_undegraded":"2023-11-06T03:05:13.229038+0000","last_fullsized":"2023-11-06T03:05:13.229038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229509+0000","last_change":"2023-11-06T03:05:13.229509+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229509+0000","last_undegraded":"2023-11-06T03:05:13.229509+0000","last_fullsized":"2023-11-06T03:05:13.229509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231213+0000","last_change":"2023-11-06T03:05:13.231213+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231213+0000","last_undegraded":"2023-11-06T03:05:13.231213+0000","last_fullsized":"2023-11-06T03:05:13.231213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230423+0000","last_change":"2023-11-06T03:05:13.230423+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230423+0000","last_undegraded":"2023-11-06T03:05:13.230423+0000","last_fullsized":"2023-11-06T03:05:13.230423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229513+0000","last_change":"2023-11-06T03:05:13.229513+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229513+0000","last_undegraded":"2023-11-06T03:05:13.229513+0000","last_fullsized":"2023-11-06T03:05:13.229513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224151+0000","last_change":"2023-11-06T03:05:13.224151+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224151+0000","last_undegraded":"2023-11-06T03:05:13.224151+0000","last_fullsized":"2023-11-06T03:05:13.224151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229678+0000","last_change":"2023-11-06T03:05:13.229678+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229678+0000","last_undegraded":"2023-11-06T03:05:13.229678+0000","last_fullsized":"2023-11-06T03:05:13.229678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224119+0000","last_change":"2023-11-06T03:05:13.224119+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224119+0000","last_undegraded":"2023-11-06T03:05:13.224119+0000","last_fullsized":"2023-11-06T03:05:13.224119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007919+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:05:09.007919+0000","last_peered":"2023-11-06T03:05:09.007919+0000","last_clean":"2023-11-06T03:05:09.007919+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:05:09.007919+0000","last_undegraded":"2023-11-06T03:05:09.007919+0000","last_fullsized":"2023-11-06T03:05:09.007919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829794+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:05:09.829794+0000","last_peered":"2023-11-06T03:05:09.829794+0000","last_clean":"2023-11-06T03:05:09.829794+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:05:09.829794+0000","last_undegraded":"2023-11-06T03:05:09.829794+0000","last_fullsized":"2023-11-06T03:05:09.829794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.237064+0000","last_change":"2023-11-06T03:05:13.237064+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.237064+0000","last_undegraded":"2023-11-06T03:05:13.237064+0000","last_fullsized":"2023-11-06T03:05:13.237064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.008070+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:05:09.008070+0000","last_peered":"2023-11-06T03:05:09.008070+0000","last_clean":"2023-11-06T03:05:09.008070+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:05:09.008070+0000","last_undegraded":"2023-11-06T03:05:09.008070+0000","last_fullsized":"2023-11-06T03:05:09.008070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230034+0000","last_change":"2023-11-06T03:05:13.230034+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230034+0000","last_undegraded":"2023-11-06T03:05:13.230034+0000","last_fullsized":"2023-11-06T03:05:13.230034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330499+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:05:10.330499+0000","last_peered":"2023-11-06T03:05:10.330499+0000","last_clean":"2023-11-06T03:05:10.330499+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:05:10.330499+0000","last_undegraded":"2023-11-06T03:05:10.330499+0000","last_fullsized":"2023-11-06T03:05:10.330499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230260+0000","last_change":"2023-11-06T03:05:13.230260+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230260+0000","last_undegraded":"2023-11-06T03:05:13.230260+0000","last_fullsized":"2023-11-06T03:05:13.230260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829786+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:05:09.829786+0000","last_peered":"2023-11-06T03:05:09.829786+0000","last_clean":"2023-11-06T03:05:09.829786+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:05:09.829786+0000","last_undegraded":"2023-11-06T03:05:09.829786+0000","last_fullsized":"2023-11-06T03:05:09.829786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229604+0000","last_change":"2023-11-06T03:05:13.229604+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229604+0000","last_undegraded":"2023-11-06T03:05:13.229604+0000","last_fullsized":"2023-11-06T03:05:13.229604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499031+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:05:09.499031+0000","last_peered":"2023-11-06T03:05:09.499031+0000","last_clean":"2023-11-06T03:05:09.499031+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:05:09.499031+0000","last_undegraded":"2023-11-06T03:05:09.499031+0000","last_fullsized":"2023-11-06T03:05:09.499031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230514+0000","last_change":"2023-11-06T03:05:13.230514+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230514+0000","last_undegraded":"2023-11-06T03:05:13.230514+0000","last_fullsized":"2023-11-06T03:05:13.230514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007975+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:05:09.007975+0000","last_peered":"2023-11-06T03:05:09.007975+0000","last_clean":"2023-11-06T03:05:09.007975+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:05:09.007975+0000","last_undegraded":"2023-11-06T03:05:09.007975+0000","last_fullsized":"2023-11-06T03:05:09.007975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228945+0000","last_change":"2023-11-06T03:05:13.228945+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228945+0000","last_undegraded":"2023-11-06T03:05:13.228945+0000","last_fullsized":"2023-11-06T03:05:13.228945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499057+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:05:09.499057+0000","last_peered":"2023-11-06T03:05:09.499057+0000","last_clean":"2023-11-06T03:05:09.499057+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:05:09.499057+0000","last_undegraded":"2023-11-06T03:05:09.499057+0000","last_fullsized":"2023-11-06T03:05:09.499057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225413+0000","last_change":"2023-11-06T03:05:13.225413+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225413+0000","last_undegraded":"2023-11-06T03:05:13.225413+0000","last_fullsized":"2023-11-06T03:05:13.225413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330252+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:05:10.330252+0000","last_peered":"2023-11-06T03:05:10.330252+0000","last_clean":"2023-11-06T03:05:10.330252+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:05:10.330252+0000","last_undegraded":"2023-11-06T03:05:10.330252+0000","last_fullsized":"2023-11-06T03:05:10.330252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":745173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0229999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.93899999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.052}]},{"osd":4,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.90500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.45600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.026},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.98899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.876},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.133},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.1559999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82299999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.008},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.032}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.016}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.083},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.95299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.88200000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.90200000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.873},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.083}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.749},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.505}]},{"osd":3,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.0329999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":974848,"data_stored":744154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.26200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.755}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:05:17.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:05:18.050 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:05:18.047+0000 7f5d681d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:05:18.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.147+0000 7fa7fcf63640 1 Processor -- start 2023-11-06T03:05:18.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.147+0000 7fa7fcf63640 1 -- start start 2023-11-06T03:05:18.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7f8156a70 0x7fa7f806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa7f80b2330 0x7fa7f806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f806c490 con 0x7fa7f80adae0 2023-11-06T03:05:18.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f806b950 con 0x7fa7f8156a70 2023-11-06T03:05:18.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f806bb60 con 0x7fa7f80b2330 2023-11-06T03:05:18.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7f8156a70 0x7fa7f806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f5d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa7f80adae0 0x7fa7f80b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58574/0 (socket says 172.21.15.157:58574) 2023-11-06T03:05:18.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f5d9b640 1 -- 172.21.15.157:0/73446240 learned_addr learned my addr 172.21.15.157:0/73446240 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:18.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f6d9d640 1 --2- 172.21.15.157:0/73446240 >> v2:172.21.15.195:3300/0 conn(0x7fa7f80b2330 0x7fa7f806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 -- 172.21.15.157:0/73446240 >> v2:172.21.15.195:3300/0 conn(0x7fa7f80b2330 msgr2=0x7fa7f806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:18.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 --2- 172.21.15.157:0/73446240 >> v2:172.21.15.195:3300/0 conn(0x7fa7f80b2330 0x7fa7f806c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 -- 172.21.15.157:0/73446240 >> v1:172.21.15.157:6789/0 conn(0x7fa7f80adae0 legacy=0x7fa7f80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 -- 172.21.15.157:0/73446240 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f806be40 con 0x7fa7f8156a70 2023-11-06T03:05:18.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f659c640 1 --2- 172.21.15.157:0/73446240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f806f470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80ce830 tx=0x7fa7e801d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3da7e27ebba074d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f559a640 1 -- 172.21.15.157:0/73446240 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80a69e0 con 0x7fa7f8156a70 2023-11-06T03:05:18.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f559a640 1 -- 172.21.15.157:0/73446240 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e80a6b40 con 0x7fa7f8156a70 2023-11-06T03:05:18.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7f559a640 1 -- 172.21.15.157:0/73446240 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80ae440 con 0x7fa7f8156a70 2023-11-06T03:05:18.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- 172.21.15.157:0/73446240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 msgr2=0x7fa7f806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/73446240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f806f470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80ce830 tx=0x7fa7e801d9f0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- 172.21.15.157:0/73446240 shutdown_connections 2023-11-06T03:05:18.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/73446240 >> v2:172.21.15.195:3300/0 conn(0x7fa7f80b2330 0x7fa7f806c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/73446240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f806f470 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- 172.21.15.157:0/73446240 >> 172.21.15.157:0/73446240 conn(0x7fa7f8065930 msgr2=0x7fa7f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:18.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- 172.21.15.157:0/73446240 shutdown_connections 2023-11-06T03:05:18.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.151+0000 7fa7fcf63640 1 -- 172.21.15.157:0/73446240 wait complete. 2023-11-06T03:05:18.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 Processor -- start 2023-11-06T03:05:18.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- start start 2023-11-06T03:05:18.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7f80adae0 0x7fa7f8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7f80b2330 0x7fa7f8157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81f6ab0 con 0x7fa7f8156a70 2023-11-06T03:05:18.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81f7830 con 0x7fa7f80b2330 2023-11-06T03:05:18.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81f6d50 con 0x7fa7f80adae0 2023-11-06T03:05:18.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56236/0 (socket says 172.21.15.157:56236) 2023-11-06T03:05:18.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 -- 172.21.15.157:0/2707416884 learned_addr learned my addr 172.21.15.157:0/2707416884 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:18.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f5d9b640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7f80b2330 0x7fa7f8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f659c640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7f80adae0 0x7fa7f8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 -- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7f80adae0 msgr2=0x7fa7f8154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7f80adae0 0x7fa7f8154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 -- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7f80b2330 msgr2=0x7fa7f8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7f80b2330 0x7fa7f8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 -- 172.21.15.157:0/2707416884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8009a70 con 0x7fa7f8156a70 2023-11-06T03:05:18.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7f6d9d640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d800 tx=0x7fa7f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000b6c0 con 0x7fa7f8156a70 2023-11-06T03:05:18.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000b820 con 0x7fa7f8156a70 2023-11-06T03:05:18.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f81f7030 con 0x7fa7f8156a70 2023-11-06T03:05:18.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000bbd0 con 0x7fa7f8156a70 2023-11-06T03:05:18.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f8154cc0 con 0x7fa7f8156a70 2023-11-06T03:05:18.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.155+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f8072750 con 0x7fa7f8156a70 2023-11-06T03:05:18.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.159+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7f000bd30 con 0x7fa7f8156a70 2023-11-06T03:05:18.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.159+0000 7fa7eeffd640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 0x7fa7d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.159+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7f0083830 con 0x7fa7f8156a70 2023-11-06T03:05:18.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.159+0000 7fa7f659c640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 0x7fa7d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.159+0000 7fa7f659c640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 0x7fa7d8067f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b0000 tx=0x7fa7e80a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.163+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa7f0051090 con 0x7fa7f8156a70 2023-11-06T03:05:18.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.311+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7f806d270 con 0x7fa7d8065a60 2023-11-06T03:05:18.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.335+0000 7fa7eeffd640 1 -- 172.21.15.157:0/2707416884 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299183 (secure 0 0 0) 0x7fa7e80ae440 con 0x7fa7d8065a60 2023-11-06T03:05:18.337 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:18.340 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:05:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 msgr2=0x7fa7d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 0x7fa7d8067f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b0000 tx=0x7fa7e80a8070 comp rx=0 tx=0).stop 2023-11-06T03:05:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 msgr2=0x7fa7f8157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d800 tx=0x7fa7f000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 shutdown_connections 2023-11-06T03:05:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa7d8065a60 0x7fa7d8067f20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7f8156a70 0x7fa7f8157cb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7f80b2330 0x7fa7f8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 --2- 172.21.15.157:0/2707416884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7f80adae0 0x7fa7f8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 >> 172.21.15.157:0/2707416884 conn(0x7fa7f8065930 msgr2=0x7fa7f80ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 shutdown_connections 2023-11-06T03:05:18.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.339+0000 7fa7fcf63640 1 -- 172.21.15.157:0/2707416884 wait complete. 2023-11-06T03:05:18.356 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-06T03:05:17.118153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252540,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754722180,"statfs":{"total":773094113280,"available":772835512320,"internally_reserved":0,"allocated":4255744,"data_stored":2416119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231100+0000","last_change":"2023-11-06T03:05:13.231100+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231100+0000","last_undegraded":"2023-11-06T03:05:13.231100+0000","last_fullsized":"2023-11-06T03:05:13.231100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229773+0000","last_change":"2023-11-06T03:05:13.229773+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229773+0000","last_undegraded":"2023-11-06T03:05:13.229773+0000","last_fullsized":"2023-11-06T03:05:13.229773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230577+0000","last_change":"2023-11-06T03:05:13.230577+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230577+0000","last_undegraded":"2023-11-06T03:05:13.230577+0000","last_fullsized":"2023-11-06T03:05:13.230577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228983+0000","last_change":"2023-11-06T03:05:13.228983+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228983+0000","last_undegraded":"2023-11-06T03:05:13.228983+0000","last_fullsized":"2023-11-06T03:05:13.228983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235683+0000","last_change":"2023-11-06T03:05:13.235683+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235683+0000","last_undegraded":"2023-11-06T03:05:13.235683+0000","last_fullsized":"2023-11-06T03:05:13.235683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224107+0000","last_change":"2023-11-06T03:05:13.224107+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224107+0000","last_undegraded":"2023-11-06T03:05:13.224107+0000","last_fullsized":"2023-11-06T03:05:13.224107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230651+0000","last_change":"2023-11-06T03:05:13.230651+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230651+0000","last_undegraded":"2023-11-06T03:05:13.230651+0000","last_fullsized":"2023-11-06T03:05:13.230651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228932+0000","last_change":"2023-11-06T03:05:13.228932+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228932+0000","last_undegraded":"2023-11-06T03:05:13.228932+0000","last_fullsized":"2023-11-06T03:05:13.228932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225336+0000","last_change":"2023-11-06T03:05:13.225336+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225336+0000","last_undegraded":"2023-11-06T03:05:13.225336+0000","last_fullsized":"2023-11-06T03:05:13.225336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235798+0000","last_change":"2023-11-06T03:05:13.235798+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235798+0000","last_undegraded":"2023-11-06T03:05:13.235798+0000","last_fullsized":"2023-11-06T03:05:13.235798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238487+0000","last_change":"2023-11-06T03:05:13.238487+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238487+0000","last_undegraded":"2023-11-06T03:05:13.238487+0000","last_fullsized":"2023-11-06T03:05:13.238487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225520+0000","last_change":"2023-11-06T03:05:13.225520+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225520+0000","last_undegraded":"2023-11-06T03:05:13.225520+0000","last_fullsized":"2023-11-06T03:05:13.225520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.226090+0000","last_change":"2023-11-06T03:05:13.226090+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.226090+0000","last_undegraded":"2023-11-06T03:05:13.226090+0000","last_fullsized":"2023-11-06T03:05:13.226090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229794+0000","last_change":"2023-11-06T03:05:13.229794+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229794+0000","last_undegraded":"2023-11-06T03:05:13.229794+0000","last_fullsized":"2023-11-06T03:05:13.229794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230501+0000","last_change":"2023-11-06T03:05:13.230501+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230501+0000","last_undegraded":"2023-11-06T03:05:13.230501+0000","last_fullsized":"2023-11-06T03:05:13.230501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.223902+0000","last_change":"2023-11-06T03:05:13.223902+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.223902+0000","last_undegraded":"2023-11-06T03:05:13.223902+0000","last_fullsized":"2023-11-06T03:05:13.223902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238099+0000","last_change":"2023-11-06T03:05:13.238099+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238099+0000","last_undegraded":"2023-11-06T03:05:13.238099+0000","last_fullsized":"2023-11-06T03:05:13.238099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229038+0000","last_change":"2023-11-06T03:05:13.229038+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229038+0000","last_undegraded":"2023-11-06T03:05:13.229038+0000","last_fullsized":"2023-11-06T03:05:13.229038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229509+0000","last_change":"2023-11-06T03:05:13.229509+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229509+0000","last_undegraded":"2023-11-06T03:05:13.229509+0000","last_fullsized":"2023-11-06T03:05:13.229509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231213+0000","last_change":"2023-11-06T03:05:13.231213+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231213+0000","last_undegraded":"2023-11-06T03:05:13.231213+0000","last_fullsized":"2023-11-06T03:05:13.231213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230423+0000","last_change":"2023-11-06T03:05:13.230423+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230423+0000","last_undegraded":"2023-11-06T03:05:13.230423+0000","last_fullsized":"2023-11-06T03:05:13.230423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229513+0000","last_change":"2023-11-06T03:05:13.229513+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229513+0000","last_undegraded":"2023-11-06T03:05:13.229513+0000","last_fullsized":"2023-11-06T03:05:13.229513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224151+0000","last_change":"2023-11-06T03:05:13.224151+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224151+0000","last_undegraded":"2023-11-06T03:05:13.224151+0000","last_fullsized":"2023-11-06T03:05:13.224151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229678+0000","last_change":"2023-11-06T03:05:13.229678+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229678+0000","last_undegraded":"2023-11-06T03:05:13.229678+0000","last_fullsized":"2023-11-06T03:05:13.229678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224119+0000","last_change":"2023-11-06T03:05:13.224119+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224119+0000","last_undegraded":"2023-11-06T03:05:13.224119+0000","last_fullsized":"2023-11-06T03:05:13.224119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007919+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:05:09.007919+0000","last_peered":"2023-11-06T03:05:09.007919+0000","last_clean":"2023-11-06T03:05:09.007919+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:05:09.007919+0000","last_undegraded":"2023-11-06T03:05:09.007919+0000","last_fullsized":"2023-11-06T03:05:09.007919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829794+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:05:09.829794+0000","last_peered":"2023-11-06T03:05:09.829794+0000","last_clean":"2023-11-06T03:05:09.829794+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:05:09.829794+0000","last_undegraded":"2023-11-06T03:05:09.829794+0000","last_fullsized":"2023-11-06T03:05:09.829794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.237064+0000","last_change":"2023-11-06T03:05:13.237064+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.237064+0000","last_undegraded":"2023-11-06T03:05:13.237064+0000","last_fullsized":"2023-11-06T03:05:13.237064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.008070+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:05:09.008070+0000","last_peered":"2023-11-06T03:05:09.008070+0000","last_clean":"2023-11-06T03:05:09.008070+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:05:09.008070+0000","last_undegraded":"2023-11-06T03:05:09.008070+0000","last_fullsized":"2023-11-06T03:05:09.008070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230034+0000","last_change":"2023-11-06T03:05:13.230034+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230034+0000","last_undegraded":"2023-11-06T03:05:13.230034+0000","last_fullsized":"2023-11-06T03:05:13.230034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330499+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:05:10.330499+0000","last_peered":"2023-11-06T03:05:10.330499+0000","last_clean":"2023-11-06T03:05:10.330499+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:05:10.330499+0000","last_undegraded":"2023-11-06T03:05:10.330499+0000","last_fullsized":"2023-11-06T03:05:10.330499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230260+0000","last_change":"2023-11-06T03:05:13.230260+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230260+0000","last_undegraded":"2023-11-06T03:05:13.230260+0000","last_fullsized":"2023-11-06T03:05:13.230260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829786+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:05:09.829786+0000","last_peered":"2023-11-06T03:05:09.829786+0000","last_clean":"2023-11-06T03:05:09.829786+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:05:09.829786+0000","last_undegraded":"2023-11-06T03:05:09.829786+0000","last_fullsized":"2023-11-06T03:05:09.829786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229604+0000","last_change":"2023-11-06T03:05:13.229604+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229604+0000","last_undegraded":"2023-11-06T03:05:13.229604+0000","last_fullsized":"2023-11-06T03:05:13.229604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499031+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:05:09.499031+0000","last_peered":"2023-11-06T03:05:09.499031+0000","last_clean":"2023-11-06T03:05:09.499031+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:05:09.499031+0000","last_undegraded":"2023-11-06T03:05:09.499031+0000","last_fullsized":"2023-11-06T03:05:09.499031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230514+0000","last_change":"2023-11-06T03:05:13.230514+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230514+0000","last_undegraded":"2023-11-06T03:05:13.230514+0000","last_fullsized":"2023-11-06T03:05:13.230514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007975+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:05:09.007975+0000","last_peered":"2023-11-06T03:05:09.007975+0000","last_clean":"2023-11-06T03:05:09.007975+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:05:09.007975+0000","last_undegraded":"2023-11-06T03:05:09.007975+0000","last_fullsized":"2023-11-06T03:05:09.007975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228945+0000","last_change":"2023-11-06T03:05:13.228945+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228945+0000","last_undegraded":"2023-11-06T03:05:13.228945+0000","last_fullsized":"2023-11-06T03:05:13.228945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499057+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:05:09.499057+0000","last_peered":"2023-11-06T03:05:09.499057+0000","last_clean":"2023-11-06T03:05:09.499057+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:05:09.499057+0000","last_undegraded":"2023-11-06T03:05:09.499057+0000","last_fullsized":"2023-11-06T03:05:09.499057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225413+0000","last_change":"2023-11-06T03:05:13.225413+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225413+0000","last_undegraded":"2023-11-06T03:05:13.225413+0000","last_fullsized":"2023-11-06T03:05:13.225413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330252+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:05:10.330252+0000","last_peered":"2023-11-06T03:05:10.330252+0000","last_clean":"2023-11-06T03:05:10.330252+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:05:10.330252+0000","last_undegraded":"2023-11-06T03:05:10.330252+0000","last_fullsized":"2023-11-06T03:05:10.330252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":745173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0229999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.93899999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.052}]},{"osd":4,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.90500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.45600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.026},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.98899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.876},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.133},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.1559999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82299999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.008},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.032}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.016}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.083},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.95299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.88200000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.90200000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.873},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.083}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.749},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.505}]},{"osd":3,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.0329999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":974848,"data_stored":744154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.26200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.755}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:05:18.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-11-06T03:05:18.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-11-06T03:05:18.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-11-06T03:05:18.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-11-06T03:05:18.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-11-06T03:05:18.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-11-06T03:05:18.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-11-06T03:05:18.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-11-06T03:05:18.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-11-06T03:05:18.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-11-06T03:05:18.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-11-06T03:05:18.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-11-06T03:05:18.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-11-06T03:05:18.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-11-06T03:05:18.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-11-06T03:05:18.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-11-06T03:05:18.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-11-06T03:05:18.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-11-06T03:05:18.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-11-06T03:05:18.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-11-06T03:05:18.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-11-06T03:05:18.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-11-06T03:05:18.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-11-06T03:05:18.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-11-06T03:05:18.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-11-06T03:05:18.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-11-06T03:05:18.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-11-06T03:05:18.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-11-06T03:05:18.383 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-11-06T03:05:18.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-11-06T03:05:18.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-11-06T03:05:18.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-11-06T03:05:18.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-11-06T03:05:18.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-06T03:05:18.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.231100+0000', 'last_change': '2023-11-06T03:05:13.231100+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.231100+0000', 'last_undegraded': '2023-11-06T03:05:13.231100+0000', 'last_fullsized': '2023-11-06T03:05:13.231100+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:05:18.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-06T03:05:18.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-06T03:05:18.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229773+0000', 'last_change': '2023-11-06T03:05:13.229773+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229773+0000', 'last_undegraded': '2023-11-06T03:05:13.229773+0000', 'last_fullsized': '2023-11-06T03:05:13.229773+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:05:18.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-11-06T03:05:18.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-06T03:05:18.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230577+0000', 'last_change': '2023-11-06T03:05:13.230577+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230577+0000', 'last_undegraded': '2023-11-06T03:05:13.230577+0000', 'last_fullsized': '2023-11-06T03:05:13.230577+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:05:18.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-11-06T03:05:18.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-11-06T03:05:18.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.228983+0000', 'last_change': '2023-11-06T03:05:13.228983+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.228983+0000', 'last_undegraded': '2023-11-06T03:05:13.228983+0000', 'last_fullsized': '2023-11-06T03:05:13.228983+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:05:18.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-06T03:05:18.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-11-06T03:05:18.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.235683+0000', 'last_change': '2023-11-06T03:05:13.235683+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.235683+0000', 'last_undegraded': '2023-11-06T03:05:13.235683+0000', 'last_fullsized': '2023-11-06T03:05:13.235683+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:05:18.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-11-06T03:05:18.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-06T03:05:18.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.224107+0000', 'last_change': '2023-11-06T03:05:13.224107+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.224107+0000', 'last_undegraded': '2023-11-06T03:05:13.224107+0000', 'last_fullsized': '2023-11-06T03:05:13.224107+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:05:18.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-11-06T03:05:18.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-06T03:05:18.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230651+0000', 'last_change': '2023-11-06T03:05:13.230651+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230651+0000', 'last_undegraded': '2023-11-06T03:05:13.230651+0000', 'last_fullsized': '2023-11-06T03:05:13.230651+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:05:18.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-11-06T03:05:18.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-11-06T03:05:18.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.228932+0000', 'last_change': '2023-11-06T03:05:13.228932+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.228932+0000', 'last_undegraded': '2023-11-06T03:05:13.228932+0000', 'last_fullsized': '2023-11-06T03:05:13.228932+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:05:18.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-11-06T03:05:18.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-06T03:05:18.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.225336+0000', 'last_change': '2023-11-06T03:05:13.225336+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.225336+0000', 'last_undegraded': '2023-11-06T03:05:13.225336+0000', 'last_fullsized': '2023-11-06T03:05:13.225336+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:05:18.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-11-06T03:05:18.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-06T03:05:18.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.235798+0000', 'last_change': '2023-11-06T03:05:13.235798+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.235798+0000', 'last_undegraded': '2023-11-06T03:05:13.235798+0000', 'last_fullsized': '2023-11-06T03:05:13.235798+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:05:18.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-06T03:05:18.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-06T03:05:18.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.238487+0000', 'last_change': '2023-11-06T03:05:13.238487+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.238487+0000', 'last_undegraded': '2023-11-06T03:05:13.238487+0000', 'last_fullsized': '2023-11-06T03:05:13.238487+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:05:18.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-11-06T03:05:18.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-06T03:05:18.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.225520+0000', 'last_change': '2023-11-06T03:05:13.225520+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.225520+0000', 'last_undegraded': '2023-11-06T03:05:13.225520+0000', 'last_fullsized': '2023-11-06T03:05:13.225520+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:05:18.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-06T03:05:18.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-06T03:05:18.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.226090+0000', 'last_change': '2023-11-06T03:05:13.226090+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.226090+0000', 'last_undegraded': '2023-11-06T03:05:13.226090+0000', 'last_fullsized': '2023-11-06T03:05:13.226090+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:05:18.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-11-06T03:05:18.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-06T03:05:18.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229794+0000', 'last_change': '2023-11-06T03:05:13.229794+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229794+0000', 'last_undegraded': '2023-11-06T03:05:13.229794+0000', 'last_fullsized': '2023-11-06T03:05:13.229794+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:05:18.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-11-06T03:05:18.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-06T03:05:18.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230501+0000', 'last_change': '2023-11-06T03:05:13.230501+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230501+0000', 'last_undegraded': '2023-11-06T03:05:13.230501+0000', 'last_fullsized': '2023-11-06T03:05:13.230501+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:05:18.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-11-06T03:05:18.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-06T03:05:18.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.223902+0000', 'last_change': '2023-11-06T03:05:13.223902+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.223902+0000', 'last_undegraded': '2023-11-06T03:05:13.223902+0000', 'last_fullsized': '2023-11-06T03:05:13.223902+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:05:18.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-11-06T03:05:18.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-11-06T03:05:18.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.238099+0000', 'last_change': '2023-11-06T03:05:13.238099+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.238099+0000', 'last_undegraded': '2023-11-06T03:05:13.238099+0000', 'last_fullsized': '2023-11-06T03:05:13.238099+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:05:18.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-11-06T03:05:18.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-06T03:05:18.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229038+0000', 'last_change': '2023-11-06T03:05:13.229038+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229038+0000', 'last_undegraded': '2023-11-06T03:05:13.229038+0000', 'last_fullsized': '2023-11-06T03:05:13.229038+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:05:18.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-11-06T03:05:18.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-06T03:05:18.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229509+0000', 'last_change': '2023-11-06T03:05:13.229509+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229509+0000', 'last_undegraded': '2023-11-06T03:05:13.229509+0000', 'last_fullsized': '2023-11-06T03:05:13.229509+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:05:18.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-11-06T03:05:18.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-06T03:05:18.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.231213+0000', 'last_change': '2023-11-06T03:05:13.231213+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.231213+0000', 'last_undegraded': '2023-11-06T03:05:13.231213+0000', 'last_fullsized': '2023-11-06T03:05:13.231213+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:05:18.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-11-06T03:05:18.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-06T03:05:18.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230423+0000', 'last_change': '2023-11-06T03:05:13.230423+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230423+0000', 'last_undegraded': '2023-11-06T03:05:13.230423+0000', 'last_fullsized': '2023-11-06T03:05:13.230423+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:05:18.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-06T03:05:18.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-06T03:05:18.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229513+0000', 'last_change': '2023-11-06T03:05:13.229513+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229513+0000', 'last_undegraded': '2023-11-06T03:05:13.229513+0000', 'last_fullsized': '2023-11-06T03:05:13.229513+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:05:18.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-11-06T03:05:18.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-06T03:05:18.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.224151+0000', 'last_change': '2023-11-06T03:05:13.224151+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.224151+0000', 'last_undegraded': '2023-11-06T03:05:13.224151+0000', 'last_fullsized': '2023-11-06T03:05:13.224151+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:05:18.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-11-06T03:05:18.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-06T03:05:18.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229678+0000', 'last_change': '2023-11-06T03:05:13.229678+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229678+0000', 'last_undegraded': '2023-11-06T03:05:13.229678+0000', 'last_fullsized': '2023-11-06T03:05:13.229678+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:05:18.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-11-06T03:05:18.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-06T03:05:18.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.224119+0000', 'last_change': '2023-11-06T03:05:13.224119+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.224119+0000', 'last_undegraded': '2023-11-06T03:05:13.224119+0000', 'last_fullsized': '2023-11-06T03:05:13.224119+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:05:18.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-06T03:05:18.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-06T03:05:18.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.237064+0000', 'last_change': '2023-11-06T03:05:13.237064+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.237064+0000', 'last_undegraded': '2023-11-06T03:05:13.237064+0000', 'last_fullsized': '2023-11-06T03:05:13.237064+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:05:18.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-11-06T03:05:18.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-06T03:05:18.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230034+0000', 'last_change': '2023-11-06T03:05:13.230034+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230034+0000', 'last_undegraded': '2023-11-06T03:05:13.230034+0000', 'last_fullsized': '2023-11-06T03:05:13.230034+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:05:18.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-11-06T03:05:18.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-06T03:05:18.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230260+0000', 'last_change': '2023-11-06T03:05:13.230260+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230260+0000', 'last_undegraded': '2023-11-06T03:05:13.230260+0000', 'last_fullsized': '2023-11-06T03:05:13.230260+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:05:18.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-06T03:05:18.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-06T03:05:18.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.229604+0000', 'last_change': '2023-11-06T03:05:13.229604+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.229604+0000', 'last_undegraded': '2023-11-06T03:05:13.229604+0000', 'last_fullsized': '2023-11-06T03:05:13.229604+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:05:18.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-06T03:05:18.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-06T03:05:18.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.230514+0000', 'last_change': '2023-11-06T03:05:13.230514+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.230514+0000', 'last_undegraded': '2023-11-06T03:05:13.230514+0000', 'last_fullsized': '2023-11-06T03:05:13.230514+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:05:18.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-11-06T03:05:18.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-06T03:05:18.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.228945+0000', 'last_change': '2023-11-06T03:05:13.228945+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.228945+0000', 'last_undegraded': '2023-11-06T03:05:13.228945+0000', 'last_fullsized': '2023-11-06T03:05:13.228945+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:05:18.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-11-06T03:05:18.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:05:14.217547+0000', 'last_change': '2023-11-06T03:05:14.217547+0000', 'last_active': '2023-11-06T03:05:14.217547+0000', 'last_peered': '2023-11-06T03:05:14.217547+0000', 'last_clean': '2023-11-06T03:05:14.217547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:14.217547+0000', 'last_undegraded': '2023-11-06T03:05:14.217547+0000', 'last_fullsized': '2023-11-06T03:05:14.217547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:14.217547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:05:18.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-06T03:05:18.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:05:13.225413+0000', 'last_change': '2023-11-06T03:05:13.225413+0000', 'last_active': '2023-11-06T03:05:13.206875+0000', 'last_peered': '2023-11-06T03:05:13.206875+0000', 'last_clean': '2023-11-06T03:05:13.206875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:05:13.225413+0000', 'last_undegraded': '2023-11-06T03:05:13.225413+0000', 'last_fullsized': '2023-11-06T03:05:13.225413+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'last_clean_scrub_stamp': '2023-11-06T03:05:13.206875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:05:18.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:05:18.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 Processor -- start 2023-11-06T03:05:18.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 -- start start 2023-11-06T03:05:18.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5af4146370 0x7f5af40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5af40ce800 0x7f5af40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40cf1a0 con 0x7f5af406b6a0 2023-11-06T03:05:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af406c3d0 con 0x7f5af40ce800 2023-11-06T03:05:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af9a94640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af406f5a0 con 0x7f5af4146370 2023-11-06T03:05:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5af40ce800 0x7f5af40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5af406b6a0 0x7f5af406bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40978/0 (socket says 172.21.15.157:40978) 2023-11-06T03:05:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.555+0000 7f5af3fff640 1 -- 172.21.15.157:0/2327705069 learned_addr learned my addr 172.21.15.157:0/2327705069 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af37fe640 1 --2- 172.21.15.157:0/2327705069 >> v2:172.21.15.195:3300/0 conn(0x7f5af4146370 0x7f5af40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af2ffd640 1 -- 172.21.15.157:0/2327705069 >> v2:172.21.15.195:3300/0 conn(0x7f5af4146370 msgr2=0x7f5af40b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af2ffd640 1 --2- 172.21.15.157:0/2327705069 >> v2:172.21.15.195:3300/0 conn(0x7f5af4146370 0x7f5af40b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af2ffd640 1 -- 172.21.15.157:0/2327705069 >> v1:172.21.15.157:6789/0 conn(0x7f5af406b6a0 legacy=0x7f5af406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af2ffd640 1 -- 172.21.15.157:0/2327705069 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af406f880 con 0x7f5af40ce800 2023-11-06T03:05:18.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af27fc640 1 -- 172.21.15.157:0/2327705069 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432172168 0 0) 0x7f5af40cf1a0 con 0x7f5af406b6a0 2023-11-06T03:05:18.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af2ffd640 1 --2- 172.21.15.157:0/2327705069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af40ce800 0x7f5af40cec60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80097c0 tx=0x7f5ae801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b50d0fc2066486 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af27fc640 1 -- 172.21.15.157:0/2327705069 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80ad680 con 0x7f5af40ce800 2023-11-06T03:05:18.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af27fc640 1 -- 172.21.15.157:0/2327705069 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae80ad7e0 con 0x7f5af40ce800 2023-11-06T03:05:18.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af27fc640 1 -- 172.21.15.157:0/2327705069 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80adb70 con 0x7f5af40ce800 2023-11-06T03:05:18.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 -- 172.21.15.157:0/2327705069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af40ce800 msgr2=0x7f5af40cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 --2- 172.21.15.157:0/2327705069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af40ce800 0x7f5af40cec60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80097c0 tx=0x7f5ae801c380 comp rx=0 tx=0).stop 2023-11-06T03:05:18.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 -- 172.21.15.157:0/2327705069 shutdown_connections 2023-11-06T03:05:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 --2- 172.21.15.157:0/2327705069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af40ce800 0x7f5af40cec60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 --2- 172.21.15.157:0/2327705069 >> v2:172.21.15.195:3300/0 conn(0x7f5af4146370 0x7f5af40b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 -- 172.21.15.157:0/2327705069 >> 172.21.15.157:0/2327705069 conn(0x7f5af4065930 msgr2=0x7f5af4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 -- 172.21.15.157:0/2327705069 shutdown_connections 2023-11-06T03:05:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.559+0000 7f5af9a94640 1 -- 172.21.15.157:0/2327705069 wait complete. 2023-11-06T03:05:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 Processor -- start 2023-11-06T03:05:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- start start 2023-11-06T03:05:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5af40ce800 0x7f5af40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5af4146370 0x7f5af40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4160090 con 0x7f5af406b6a0 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4160370 con 0x7f5af40ce800 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af413d870 con 0x7f5af4146370 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39438/0 (socket says 172.21.15.157:39438) 2023-11-06T03:05:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 -- 172.21.15.157:0/221459510 learned_addr learned my addr 172.21.15.157:0/221459510 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af2ffd640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5af40ce800 0x7f5af40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af3fff640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5af4146370 0x7f5af40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 -- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5af4146370 msgr2=0x7f5af40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5af4146370 0x7f5af40acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 -- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5af40ce800 msgr2=0x7f5af40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5af40ce800 0x7f5af40ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 -- 172.21.15.157:0/221459510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009420 con 0x7f5af406b6a0 2023-11-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af37fe640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400d800 tx=0x7f5ae400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400b6c0 con 0x7f5af406b6a0 2023-11-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400b820 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af413db50 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400bb90 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af415cd40 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af413d6d0 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ae400bcf0 con 0x7f5af406b6a0 2023-11-06T03:05:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 0x7f5ae0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.563+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ae40837f0 con 0x7f5af406b6a0 2023-11-06T03:05:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.567+0000 7f5af2ffd640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 0x7f5ae0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.567+0000 7f5af2ffd640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 0x7f5ae0067f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80af000 tx=0x7f5ae8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.567+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5af413d6d0 con 0x7f5af406b6a0 2023-11-06T03:05:18.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.775+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5af413d6d0 con 0x7f5af406b6a0 2023-11-06T03:05:18.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5adbfff640 1 -- 172.21.15.157:0/221459510 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1269 (secure 0 0 0) 0x7f5af413d6d0 con 0x7f5af406b6a0 2023-11-06T03:05:18.780 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:18.780 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":96,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":258600960,"bytes_avail":772835512320,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5331}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-06T03:05:05.115131+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:05:18.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 msgr2=0x7f5ae0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 0x7f5ae0067f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80af000 tx=0x7f5ae8008040 comp rx=0 tx=0).stop 2023-11-06T03:05:18.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 msgr2=0x7f5af415c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400d800 tx=0x7f5ae400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 shutdown_connections 2023-11-06T03:05:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ae0065a60 0x7f5ae0067f20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5af4146370 0x7f5af40acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5af40ce800 0x7f5af40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 --2- 172.21.15.157:0/221459510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af406b6a0 0x7f5af415c800 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 >> 172.21.15.157:0/221459510 conn(0x7f5af4065930 msgr2=0x7f5af40b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:18.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 shutdown_connections 2023-11-06T03:05:18.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.779+0000 7f5af9a94640 1 -- 172.21.15.157:0/221459510 wait complete. 2023-11-06T03:05:18.797 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:05:18.797 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:05:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.899+0000 7ff9d42f2640 1 Processor -- start 2023-11-06T03:05:18.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.899+0000 7ff9d42f2640 1 -- start start 2023-11-06T03:05:18.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d42f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d42f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9cc0b3b20 0x7ff9cc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d42f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc15f580 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d42f2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc15efc0 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d42f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15f2a0 con 0x7ff9cc1551e0 2023-11-06T03:05:18.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9cc0b3b20 0x7ff9cc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d188d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff9cc15e5e0 0x7ff9cc0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40992/0 (socket says 172.21.15.157:40992) 2023-11-06T03:05:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d188d640 1 -- 172.21.15.157:0/3680522714 learned_addr learned my addr 172.21.15.157:0/3680522714 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 -- 172.21.15.157:0/3680522714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 msgr2=0x7ff9cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:18.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d208e640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 -- 172.21.15.157:0/3680522714 >> v1:172.21.15.157:6789/0 conn(0x7ff9cc15e5e0 legacy=0x7ff9cc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d108c640 1 -- 172.21.15.157:0/3680522714 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409878473 0 0) 0x7ff9cc15f580 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 -- 172.21.15.157:0/3680522714 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc06f5a0 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d208e640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d288f640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc15c1b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009d90 tx=0x7ff9c401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783ab91f4db6652d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d108c640 1 -- 172.21.15.157:0/3680522714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a5970 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d108c640 1 -- 172.21.15.157:0/3680522714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a5ad0 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.903+0000 7ff9d108c640 1 -- 172.21.15.157:0/3680522714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40a5e60 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/3680522714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc0b3b20 msgr2=0x7ff9cc15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc15c1b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009d90 tx=0x7ff9c401c890 comp rx=0 tx=0).stop 2023-11-06T03:05:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/3680522714 shutdown_connections 2023-11-06T03:05:18.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc15c1b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/3680522714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/3680522714 >> 172.21.15.157:0/3680522714 conn(0x7ff9cc065930 msgr2=0x7ff9cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/3680522714 shutdown_connections 2023-11-06T03:05:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/3680522714 wait complete. 2023-11-06T03:05:18.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 Processor -- start 2023-11-06T03:05:18.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- start start 2023-11-06T03:05:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc154950 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1540b0 con 0x7ff9cc0b3b20 2023-11-06T03:05:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc154390 con 0x7ff9cc1551e0 2023-11-06T03:05:18.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39462/0 (socket says 172.21.15.157:39462) 2023-11-06T03:05:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 -- 172.21.15.157:0/1865886003 learned_addr learned my addr 172.21.15.157:0/1865886003 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d208e640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d188d640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 -- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 msgr2=0x7ff9cc1488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc1488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 -- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc0b3b20 msgr2=0x7ff9cc146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 -- 172.21.15.157:0/1865886003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4009a70 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d188d640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc1488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d288f640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401af40 tx=0x7ff9c4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40ade40 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40b9450 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc154670 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b97e0 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.907+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc147070 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc072750 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9c40b9940 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9c2ffd640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 0x7ff9b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:18.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9c411eb30 con 0x7ff9cc15e5e0 2023-11-06T03:05:18.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9d208e640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 0x7ff9b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:18.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.911+0000 7ff9d208e640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 0x7ff9b0067f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800a9a0 tx=0x7ff9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:18.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:18.915+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9c40ec4e0 con 0x7ff9cc15e5e0 2023-11-06T03:05:19.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.063+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9cc15b1b0 con 0x7ff9b0065a60 2023-11-06T03:05:19.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.087+0000 7ff9c2ffd640 1 -- 172.21.15.157:0/1865886003 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299183 (secure 0 0 0) 0x7ff9c800a720 con 0x7ff9b0065a60 2023-11-06T03:05:19.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:05:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 msgr2=0x7ff9b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 0x7ff9b0067f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800a9a0 tx=0x7ff9c8008040 comp rx=0 tx=0).stop 2023-11-06T03:05:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 msgr2=0x7ff9cc142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401af40 tx=0x7ff9c4019630 comp rx=0 tx=0).stop 2023-11-06T03:05:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 shutdown_connections 2023-11-06T03:05:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9b0065a60 0x7ff9b0067f20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9cc15e5e0 0x7ff9cc142cd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9cc1551e0 0x7ff9cc1488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 --2- 172.21.15.157:0/1865886003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9cc0b3b20 0x7ff9cc146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 >> 172.21.15.157:0/1865886003 conn(0x7ff9cc065930 msgr2=0x7ff9cc0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 shutdown_connections 2023-11-06T03:05:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:19.091+0000 7ff9d42f2640 1 -- 172.21.15.157:0/1865886003 wait complete. 2023-11-06T03:05:19.109 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-11-06T03:05:17.118153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252540,"kb_used_data":4156,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754722180,"statfs":{"total":773094113280,"available":772835512320,"internally_reserved":0,"allocated":4255744,"data_stored":2416119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231100+0000","last_change":"2023-11-06T03:05:13.231100+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231100+0000","last_undegraded":"2023-11-06T03:05:13.231100+0000","last_fullsized":"2023-11-06T03:05:13.231100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229773+0000","last_change":"2023-11-06T03:05:13.229773+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229773+0000","last_undegraded":"2023-11-06T03:05:13.229773+0000","last_fullsized":"2023-11-06T03:05:13.229773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230577+0000","last_change":"2023-11-06T03:05:13.230577+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230577+0000","last_undegraded":"2023-11-06T03:05:13.230577+0000","last_fullsized":"2023-11-06T03:05:13.230577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228983+0000","last_change":"2023-11-06T03:05:13.228983+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228983+0000","last_undegraded":"2023-11-06T03:05:13.228983+0000","last_fullsized":"2023-11-06T03:05:13.228983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235683+0000","last_change":"2023-11-06T03:05:13.235683+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235683+0000","last_undegraded":"2023-11-06T03:05:13.235683+0000","last_fullsized":"2023-11-06T03:05:13.235683+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224107+0000","last_change":"2023-11-06T03:05:13.224107+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224107+0000","last_undegraded":"2023-11-06T03:05:13.224107+0000","last_fullsized":"2023-11-06T03:05:13.224107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230651+0000","last_change":"2023-11-06T03:05:13.230651+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230651+0000","last_undegraded":"2023-11-06T03:05:13.230651+0000","last_fullsized":"2023-11-06T03:05:13.230651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228932+0000","last_change":"2023-11-06T03:05:13.228932+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228932+0000","last_undegraded":"2023-11-06T03:05:13.228932+0000","last_fullsized":"2023-11-06T03:05:13.228932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225336+0000","last_change":"2023-11-06T03:05:13.225336+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225336+0000","last_undegraded":"2023-11-06T03:05:13.225336+0000","last_fullsized":"2023-11-06T03:05:13.225336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.235798+0000","last_change":"2023-11-06T03:05:13.235798+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.235798+0000","last_undegraded":"2023-11-06T03:05:13.235798+0000","last_fullsized":"2023-11-06T03:05:13.235798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238487+0000","last_change":"2023-11-06T03:05:13.238487+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238487+0000","last_undegraded":"2023-11-06T03:05:13.238487+0000","last_fullsized":"2023-11-06T03:05:13.238487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225520+0000","last_change":"2023-11-06T03:05:13.225520+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225520+0000","last_undegraded":"2023-11-06T03:05:13.225520+0000","last_fullsized":"2023-11-06T03:05:13.225520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.226090+0000","last_change":"2023-11-06T03:05:13.226090+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.226090+0000","last_undegraded":"2023-11-06T03:05:13.226090+0000","last_fullsized":"2023-11-06T03:05:13.226090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229794+0000","last_change":"2023-11-06T03:05:13.229794+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229794+0000","last_undegraded":"2023-11-06T03:05:13.229794+0000","last_fullsized":"2023-11-06T03:05:13.229794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230501+0000","last_change":"2023-11-06T03:05:13.230501+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230501+0000","last_undegraded":"2023-11-06T03:05:13.230501+0000","last_fullsized":"2023-11-06T03:05:13.230501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.223902+0000","last_change":"2023-11-06T03:05:13.223902+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.223902+0000","last_undegraded":"2023-11-06T03:05:13.223902+0000","last_fullsized":"2023-11-06T03:05:13.223902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.238099+0000","last_change":"2023-11-06T03:05:13.238099+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.238099+0000","last_undegraded":"2023-11-06T03:05:13.238099+0000","last_fullsized":"2023-11-06T03:05:13.238099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229038+0000","last_change":"2023-11-06T03:05:13.229038+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229038+0000","last_undegraded":"2023-11-06T03:05:13.229038+0000","last_fullsized":"2023-11-06T03:05:13.229038+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229509+0000","last_change":"2023-11-06T03:05:13.229509+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229509+0000","last_undegraded":"2023-11-06T03:05:13.229509+0000","last_fullsized":"2023-11-06T03:05:13.229509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.231213+0000","last_change":"2023-11-06T03:05:13.231213+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.231213+0000","last_undegraded":"2023-11-06T03:05:13.231213+0000","last_fullsized":"2023-11-06T03:05:13.231213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230423+0000","last_change":"2023-11-06T03:05:13.230423+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230423+0000","last_undegraded":"2023-11-06T03:05:13.230423+0000","last_fullsized":"2023-11-06T03:05:13.230423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229513+0000","last_change":"2023-11-06T03:05:13.229513+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229513+0000","last_undegraded":"2023-11-06T03:05:13.229513+0000","last_fullsized":"2023-11-06T03:05:13.229513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224151+0000","last_change":"2023-11-06T03:05:13.224151+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224151+0000","last_undegraded":"2023-11-06T03:05:13.224151+0000","last_fullsized":"2023-11-06T03:05:13.224151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229678+0000","last_change":"2023-11-06T03:05:13.229678+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229678+0000","last_undegraded":"2023-11-06T03:05:13.229678+0000","last_fullsized":"2023-11-06T03:05:13.229678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.224119+0000","last_change":"2023-11-06T03:05:13.224119+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.224119+0000","last_undegraded":"2023-11-06T03:05:13.224119+0000","last_fullsized":"2023-11-06T03:05:13.224119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007919+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:05:09.007919+0000","last_peered":"2023-11-06T03:05:09.007919+0000","last_clean":"2023-11-06T03:05:09.007919+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:05:09.007919+0000","last_undegraded":"2023-11-06T03:05:09.007919+0000","last_fullsized":"2023-11-06T03:05:09.007919+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829794+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:05:09.829794+0000","last_peered":"2023-11-06T03:05:09.829794+0000","last_clean":"2023-11-06T03:05:09.829794+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:05:09.829794+0000","last_undegraded":"2023-11-06T03:05:09.829794+0000","last_fullsized":"2023-11-06T03:05:09.829794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.237064+0000","last_change":"2023-11-06T03:05:13.237064+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.237064+0000","last_undegraded":"2023-11-06T03:05:13.237064+0000","last_fullsized":"2023-11-06T03:05:13.237064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.008070+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:05:09.008070+0000","last_peered":"2023-11-06T03:05:09.008070+0000","last_clean":"2023-11-06T03:05:09.008070+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:05:09.008070+0000","last_undegraded":"2023-11-06T03:05:09.008070+0000","last_fullsized":"2023-11-06T03:05:09.008070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230034+0000","last_change":"2023-11-06T03:05:13.230034+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230034+0000","last_undegraded":"2023-11-06T03:05:13.230034+0000","last_fullsized":"2023-11-06T03:05:13.230034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330499+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:05:10.330499+0000","last_peered":"2023-11-06T03:05:10.330499+0000","last_clean":"2023-11-06T03:05:10.330499+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:05:10.330499+0000","last_undegraded":"2023-11-06T03:05:10.330499+0000","last_fullsized":"2023-11-06T03:05:10.330499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230260+0000","last_change":"2023-11-06T03:05:13.230260+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230260+0000","last_undegraded":"2023-11-06T03:05:13.230260+0000","last_fullsized":"2023-11-06T03:05:13.230260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.829786+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:05:09.829786+0000","last_peered":"2023-11-06T03:05:09.829786+0000","last_clean":"2023-11-06T03:05:09.829786+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:05:09.829786+0000","last_undegraded":"2023-11-06T03:05:09.829786+0000","last_fullsized":"2023-11-06T03:05:09.829786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.229604+0000","last_change":"2023-11-06T03:05:13.229604+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.229604+0000","last_undegraded":"2023-11-06T03:05:13.229604+0000","last_fullsized":"2023-11-06T03:05:13.229604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499031+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:05:09.499031+0000","last_peered":"2023-11-06T03:05:09.499031+0000","last_clean":"2023-11-06T03:05:09.499031+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:05:09.499031+0000","last_undegraded":"2023-11-06T03:05:09.499031+0000","last_fullsized":"2023-11-06T03:05:09.499031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.230514+0000","last_change":"2023-11-06T03:05:13.230514+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.230514+0000","last_undegraded":"2023-11-06T03:05:13.230514+0000","last_fullsized":"2023-11-06T03:05:13.230514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.007975+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:05:09.007975+0000","last_peered":"2023-11-06T03:05:09.007975+0000","last_clean":"2023-11-06T03:05:09.007975+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:05:09.007975+0000","last_undegraded":"2023-11-06T03:05:09.007975+0000","last_fullsized":"2023-11-06T03:05:09.007975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.228945+0000","last_change":"2023-11-06T03:05:13.228945+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.228945+0000","last_undegraded":"2023-11-06T03:05:13.228945+0000","last_fullsized":"2023-11-06T03:05:13.228945+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:09.499057+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:05:09.499057+0000","last_peered":"2023-11-06T03:05:09.499057+0000","last_clean":"2023-11-06T03:05:09.499057+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:05:09.499057+0000","last_undegraded":"2023-11-06T03:05:09.499057+0000","last_fullsized":"2023-11-06T03:05:09.499057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:05:14.217547+0000","last_change":"2023-11-06T03:05:14.217547+0000","last_active":"2023-11-06T03:05:14.217547+0000","last_peered":"2023-11-06T03:05:14.217547+0000","last_clean":"2023-11-06T03:05:14.217547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:14.217547+0000","last_undegraded":"2023-11-06T03:05:14.217547+0000","last_fullsized":"2023-11-06T03:05:14.217547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-11-06T03:05:13.225413+0000","last_change":"2023-11-06T03:05:13.225413+0000","last_active":"2023-11-06T03:05:13.206875+0000","last_peered":"2023-11-06T03:05:13.206875+0000","last_clean":"2023-11-06T03:05:13.206875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:05:13.225413+0000","last_undegraded":"2023-11-06T03:05:13.225413+0000","last_fullsized":"2023-11-06T03:05:13.225413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-06T03:05:10.330252+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:05:10.330252+0000","last_peered":"2023-11-06T03:05:10.330252+0000","last_clean":"2023-11-06T03:05:10.330252+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:05:10.330252+0000","last_undegraded":"2023-11-06T03:05:10.330252+0000","last_fullsized":"2023-11-06T03:05:10.330252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":745173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.0229999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.93899999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.052}]},{"osd":4,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.68899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.90500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.45600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.026},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.98899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0760000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.876},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.133},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.96699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.1559999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82299999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0569999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.008},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.032}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.016}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.083},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.95299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":380928,"data_stored":153786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.88200000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.90200000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.873},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.083}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":385024,"data_stored":154805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.749},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.505}]},{"osd":3,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.0329999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":974848,"data_stored":744154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.26200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.755}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.79000000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:05:22.116 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:05:22.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 Processor -- start 2023-11-06T03:05:22.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 -- start start 2023-11-06T03:05:22.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a28073080 0x7f3a2806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2815a910 con 0x7f3a28159ac0 2023-11-06T03:05:22.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2815abf0 con 0x7f3a2806cf20 2023-11-06T03:05:22.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a2e02a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2806d390 con 0x7f3a28073080 2023-11-06T03:05:22.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a277fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3a28159ac0 0x7f3a28159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41004/0 (socket says 172.21.15.157:41004) 2023-11-06T03:05:22.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a277fe640 1 -- 172.21.15.157:0/3341170641 learned_addr learned my addr 172.21.15.157:0/3341170641 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/3341170641 >> v2:172.21.15.195:3300/0 conn(0x7f3a28073080 0x7f3a2806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548176728 0 0) 0x7f3a2815a910 con 0x7f3a28159ac0 2023-11-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a10003530 con 0x7f3a28159ac0 2023-11-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 955287696 0 0) 0x7f3a10003530 con 0x7f3a28159ac0 2023-11-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 >> v2:172.21.15.195:3300/0 conn(0x7f3a28073080 msgr2=0x7f3a2806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:22.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 --2- 172.21.15.157:0/3341170641 >> v2:172.21.15.195:3300/0 conn(0x7f3a28073080 0x7f3a2806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 >> v1:172.21.15.195:6789/0 conn(0x7f3a2806cf20 legacy=0x7f3a280707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2815aed0 con 0x7f3a28159ac0 2023-11-06T03:05:22.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3a18001680 con 0x7f3a28159ac0 2023-11-06T03:05:22.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 >> v1:172.21.15.157:6789/0 conn(0x7f3a28159ac0 legacy=0x7f3a28159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a10003980 0x7f3a10003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a100047d0 0x7f3a10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 0x7f3a1000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2806d390 con 0x7f3a10005190 2023-11-06T03:05:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1000c130 con 0x7f3a10003980 2023-11-06T03:05:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1000e500 con 0x7f3a100047d0 2023-11-06T03:05:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 0x7f3a1000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a277fe640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a10003980 0x7f3a10003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a100047d0 0x7f3a10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 -- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a100047d0 msgr2=0x7f3a10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a100047d0 0x7f3a10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 -- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a10003980 msgr2=0x7f3a10003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a10003980 0x7f3a10003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a27fff640 1 -- 172.21.15.157:0/3341170641 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10003530 con 0x7f3a10005190 2023-11-06T03:05:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.251+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a100047d0 0x7f3a10004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a27fff640 1 -- 172.21.15.157:0/3341170641 reap_dead start 2023-11-06T03:05:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a27fff640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 0x7f3a1000b990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001e70 tx=0x7f3a2001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e5ff1d860a3c54 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20003b80 con 0x7f3a10005190 2023-11-06T03:05:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a20003d20 con 0x7f3a10005190 2023-11-06T03:05:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a267fc640 1 -- 172.21.15.157:0/3341170641 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20005310 con 0x7f3a10005190 2023-11-06T03:05:22.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 msgr2=0x7f3a1000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 0x7f3a1000b990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001e70 tx=0x7f3a2001c760 comp rx=0 tx=0).stop 2023-11-06T03:05:22.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- 172.21.15.157:0/3341170641 shutdown_connections 2023-11-06T03:05:22.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/3341170641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a10005190 0x7f3a1000b990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- 172.21.15.157:0/3341170641 >> 172.21.15.157:0/3341170641 conn(0x7f3a280cb990 msgr2=0x7f3a280cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- 172.21.15.157:0/3341170641 shutdown_connections 2023-11-06T03:05:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- 172.21.15.157:0/3341170641 wait complete. 2023-11-06T03:05:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 Processor -- start 2023-11-06T03:05:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 -- start start 2023-11-06T03:05:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.255+0000 7f3a2e02a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a10005190 0x7f3a280c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a280b1b10 0x7f3a280b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28154e10 con 0x7f3a28071050 2023-11-06T03:05:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815c3b0 con 0x7f3a280b1b10 2023-11-06T03:05:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815c690 con 0x7f3a10005190 2023-11-06T03:05:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39472/0 (socket says 172.21.15.157:39472) 2023-11-06T03:05:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 -- 172.21.15.157:0/2000624109 learned_addr learned my addr 172.21.15.157:0/2000624109 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a27fff640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a280b1b10 0x7f3a280b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a277fe640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a10005190 0x7f3a280c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 -- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a10005190 msgr2=0x7f3a280c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a10005190 0x7f3a280c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 -- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a280b1b10 msgr2=0x7f3a280b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a280b1b10 0x7f3a280b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 -- 172.21.15.157:0/2000624109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2815c970 con 0x7f3a28071050 2023-11-06T03:05:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a26ffd640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c010870 tx=0x7f3a1c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00eb60 con 0x7f3a28071050 2023-11-06T03:05:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c00ecc0 con 0x7f3a28071050 2023-11-06T03:05:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2815cc50 con 0x7f3a28071050 2023-11-06T03:05:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c01c400 con 0x7f3a28071050 2023-11-06T03:05:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2815d450 con 0x7f3a28071050 2023-11-06T03:05:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28073500 con 0x7f3a28071050 2023-11-06T03:05:22.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3a1c01c560 con 0x7f3a28071050 2023-11-06T03:05:22.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 0x7f3a14067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a1c083900 con 0x7f3a28071050 2023-11-06T03:05:22.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a277fe640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 0x7f3a14067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.259+0000 7f3a277fe640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 0x7f3a14067f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a180052a0 tx=0x7f3a18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.263+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3a1c051380 con 0x7f3a28071050 2023-11-06T03:05:22.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.415+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a28070290 con 0x7f3a14065a90 2023-11-06T03:05:22.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.439+0000 7f3a0ffff640 1 -- 172.21.15.157:0/2000624109 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309886 (secure 0 0 0) 0x7f3a180138f0 con 0x7f3a14065a90 2023-11-06T03:05:22.446 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:22.448 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:05:22.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 msgr2=0x7f3a14067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 0x7f3a14067f50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a180052a0 tx=0x7f3a18009040 comp rx=0 tx=0).stop 2023-11-06T03:05:22.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 msgr2=0x7f3a281548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c010870 tx=0x7f3a1c010d30 comp rx=0 tx=0).stop 2023-11-06T03:05:22.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 shutdown_connections 2023-11-06T03:05:22.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a14065a90 0x7f3a14067f50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a280b1b10 0x7f3a280b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a28071050 0x7f3a281548d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 --2- 172.21.15.157:0/2000624109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a10005190 0x7f3a280c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 >> 172.21.15.157:0/2000624109 conn(0x7f3a280cb990 msgr2=0x7f3a280aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:22.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 shutdown_connections 2023-11-06T03:05:22.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.443+0000 7f3a2e02a640 1 -- 172.21.15.157:0/2000624109 wait complete. 2023-11-06T03:05:22.462 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-11-06T03:05:21.119297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252892,"kb_used_data":4508,"kb_used_omap":0,"kb_used_meta":248320,"kb_avail":754721828,"statfs":{"total":773094113280,"available":772835151872,"internally_reserved":0,"allocated":4616192,"data_stored":2584115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":254279680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":65,"apply_latency_ms":65,"commit_latency_ns":65000000,"apply_latency_ns":65000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.894643"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.173115+0000","last_change":"2023-11-06T03:05:14.627939+0000","last_active":"2023-11-06T03:05:16.173115+0000","last_peered":"2023-11-06T03:05:16.173115+0000","last_clean":"2023-11-06T03:05:16.173115+0000","last_became_active":"2023-11-06T03:05:14.627631+0000","last_became_peered":"2023-11-06T03:05:14.627631+0000","last_unstale":"2023-11-06T03:05:16.173115+0000","last_undegraded":"2023-11-06T03:05:16.173115+0000","last_fullsized":"2023-11-06T03:05:16.173115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:11:06.443996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172730+0000","last_change":"2023-11-06T03:05:14.623739+0000","last_active":"2023-11-06T03:05:16.172730+0000","last_peered":"2023-11-06T03:05:16.172730+0000","last_clean":"2023-11-06T03:05:16.172730+0000","last_became_active":"2023-11-06T03:05:14.623464+0000","last_became_peered":"2023-11-06T03:05:14.623464+0000","last_unstale":"2023-11-06T03:05:16.172730+0000","last_undegraded":"2023-11-06T03:05:16.172730+0000","last_fullsized":"2023-11-06T03:05:16.172730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:37:10.380953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492150+0000","last_change":"2023-11-06T03:05:14.663871+0000","last_active":"2023-11-06T03:05:16.492150+0000","last_peered":"2023-11-06T03:05:16.492150+0000","last_clean":"2023-11-06T03:05:16.492150+0000","last_became_active":"2023-11-06T03:05:14.663549+0000","last_became_peered":"2023-11-06T03:05:14.663549+0000","last_unstale":"2023-11-06T03:05:16.492150+0000","last_undegraded":"2023-11-06T03:05:16.492150+0000","last_fullsized":"2023-11-06T03:05:16.492150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:38.269085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.619005+0000","last_change":"2023-11-06T03:05:14.651084+0000","last_active":"2023-11-06T03:05:15.619005+0000","last_peered":"2023-11-06T03:05:15.619005+0000","last_clean":"2023-11-06T03:05:15.619005+0000","last_became_active":"2023-11-06T03:05:14.650806+0000","last_became_peered":"2023-11-06T03:05:14.650806+0000","last_unstale":"2023-11-06T03:05:15.619005+0000","last_undegraded":"2023-11-06T03:05:15.619005+0000","last_fullsized":"2023-11-06T03:05:15.619005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:43.913338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.618337+0000","last_change":"2023-11-06T03:05:14.675657+0000","last_active":"2023-11-06T03:05:15.618337+0000","last_peered":"2023-11-06T03:05:15.618337+0000","last_clean":"2023-11-06T03:05:15.618337+0000","last_became_active":"2023-11-06T03:05:14.675369+0000","last_became_peered":"2023-11-06T03:05:14.675369+0000","last_unstale":"2023-11-06T03:05:15.618337+0000","last_undegraded":"2023-11-06T03:05:15.618337+0000","last_fullsized":"2023-11-06T03:05:15.618337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:37.328355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.802081+0000","last_change":"2023-11-06T03:05:14.628104+0000","last_active":"2023-11-06T03:05:15.802081+0000","last_peered":"2023-11-06T03:05:15.802081+0000","last_clean":"2023-11-06T03:05:15.802081+0000","last_became_active":"2023-11-06T03:05:14.627851+0000","last_became_peered":"2023-11-06T03:05:14.627851+0000","last_unstale":"2023-11-06T03:05:15.802081+0000","last_undegraded":"2023-11-06T03:05:15.802081+0000","last_fullsized":"2023-11-06T03:05:15.802081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:06:00.045958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494841+0000","last_change":"2023-11-06T03:05:14.660712+0000","last_active":"2023-11-06T03:05:16.494841+0000","last_peered":"2023-11-06T03:05:16.494841+0000","last_clean":"2023-11-06T03:05:16.494841+0000","last_became_active":"2023-11-06T03:05:14.658854+0000","last_became_peered":"2023-11-06T03:05:14.658854+0000","last_unstale":"2023-11-06T03:05:16.494841+0000","last_undegraded":"2023-11-06T03:05:16.494841+0000","last_fullsized":"2023-11-06T03:05:16.494841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:50.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172165+0000","last_change":"2023-11-06T03:05:14.660454+0000","last_active":"2023-11-06T03:05:16.172165+0000","last_peered":"2023-11-06T03:05:16.172165+0000","last_clean":"2023-11-06T03:05:16.172165+0000","last_became_active":"2023-11-06T03:05:14.660136+0000","last_became_peered":"2023-11-06T03:05:14.660136+0000","last_unstale":"2023-11-06T03:05:16.172165+0000","last_undegraded":"2023-11-06T03:05:16.172165+0000","last_fullsized":"2023-11-06T03:05:16.172165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:38.269085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492197+0000","last_change":"2023-11-06T03:05:14.661258+0000","last_active":"2023-11-06T03:05:16.492197+0000","last_peered":"2023-11-06T03:05:16.492197+0000","last_clean":"2023-11-06T03:05:16.492197+0000","last_became_active":"2023-11-06T03:05:14.660993+0000","last_became_peered":"2023-11-06T03:05:14.660993+0000","last_unstale":"2023-11-06T03:05:16.492197+0000","last_undegraded":"2023-11-06T03:05:16.492197+0000","last_fullsized":"2023-11-06T03:05:16.492197+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:23.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.495176+0000","last_change":"2023-11-06T03:05:14.670676+0000","last_active":"2023-11-06T03:05:16.495176+0000","last_peered":"2023-11-06T03:05:16.495176+0000","last_clean":"2023-11-06T03:05:16.495176+0000","last_became_active":"2023-11-06T03:05:14.670006+0000","last_became_peered":"2023-11-06T03:05:14.670006+0000","last_unstale":"2023-11-06T03:05:16.495176+0000","last_undegraded":"2023-11-06T03:05:16.495176+0000","last_fullsized":"2023-11-06T03:05:16.495176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:09.761691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494045+0000","last_change":"2023-11-06T03:05:14.661254+0000","last_active":"2023-11-06T03:05:16.494045+0000","last_peered":"2023-11-06T03:05:16.494045+0000","last_clean":"2023-11-06T03:05:16.494045+0000","last_became_active":"2023-11-06T03:05:14.660003+0000","last_became_peered":"2023-11-06T03:05:14.660003+0000","last_unstale":"2023-11-06T03:05:16.494045+0000","last_undegraded":"2023-11-06T03:05:16.494045+0000","last_fullsized":"2023-11-06T03:05:16.494045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:38.269085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.491856+0000","last_change":"2023-11-06T03:05:14.636418+0000","last_active":"2023-11-06T03:05:16.491856+0000","last_peered":"2023-11-06T03:05:16.491856+0000","last_clean":"2023-11-06T03:05:16.491856+0000","last_became_active":"2023-11-06T03:05:14.636184+0000","last_became_peered":"2023-11-06T03:05:14.636184+0000","last_unstale":"2023-11-06T03:05:16.491856+0000","last_undegraded":"2023-11-06T03:05:16.491856+0000","last_fullsized":"2023-11-06T03:05:16.491856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:11:06.443996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.495617+0000","last_change":"2023-11-06T03:05:14.670137+0000","last_active":"2023-11-06T03:05:16.495617+0000","last_peered":"2023-11-06T03:05:16.495617+0000","last_clean":"2023-11-06T03:05:16.495617+0000","last_became_active":"2023-11-06T03:05:14.669631+0000","last_became_peered":"2023-11-06T03:05:14.669631+0000","last_unstale":"2023-11-06T03:05:16.495617+0000","last_undegraded":"2023-11-06T03:05:16.495617+0000","last_fullsized":"2023-11-06T03:05:16.495617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:08.151353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620154+0000","last_change":"2023-11-06T03:05:14.651310+0000","last_active":"2023-11-06T03:05:15.620154+0000","last_peered":"2023-11-06T03:05:15.620154+0000","last_clean":"2023-11-06T03:05:15.620154+0000","last_became_active":"2023-11-06T03:05:14.650533+0000","last_became_peered":"2023-11-06T03:05:14.650533+0000","last_unstale":"2023-11-06T03:05:15.620154+0000","last_undegraded":"2023-11-06T03:05:15.620154+0000","last_fullsized":"2023-11-06T03:05:15.620154+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:50.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.495145+0000","last_change":"2023-11-06T03:05:14.660149+0000","last_active":"2023-11-06T03:05:16.495145+0000","last_peered":"2023-11-06T03:05:16.495145+0000","last_clean":"2023-11-06T03:05:16.495145+0000","last_became_active":"2023-11-06T03:05:14.657086+0000","last_became_peered":"2023-11-06T03:05:14.657086+0000","last_unstale":"2023-11-06T03:05:16.495145+0000","last_undegraded":"2023-11-06T03:05:16.495145+0000","last_fullsized":"2023-11-06T03:05:16.495145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:41:51.574212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494259+0000","last_change":"2023-11-06T03:05:14.677854+0000","last_active":"2023-11-06T03:05:16.494259+0000","last_peered":"2023-11-06T03:05:16.494259+0000","last_clean":"2023-11-06T03:05:16.494259+0000","last_became_active":"2023-11-06T03:05:14.677001+0000","last_became_peered":"2023-11-06T03:05:14.677001+0000","last_unstale":"2023-11-06T03:05:16.494259+0000","last_undegraded":"2023-11-06T03:05:16.494259+0000","last_fullsized":"2023-11-06T03:05:16.494259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:23.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493462+0000","last_change":"2023-11-06T03:05:14.679568+0000","last_active":"2023-11-06T03:05:16.493462+0000","last_peered":"2023-11-06T03:05:16.493462+0000","last_clean":"2023-11-06T03:05:16.493462+0000","last_became_active":"2023-11-06T03:05:14.679012+0000","last_became_peered":"2023-11-06T03:05:14.679012+0000","last_unstale":"2023-11-06T03:05:16.493462+0000","last_undegraded":"2023-11-06T03:05:16.493462+0000","last_fullsized":"2023-11-06T03:05:16.493462+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:07.590728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493536+0000","last_change":"2023-11-06T03:05:14.642871+0000","last_active":"2023-11-06T03:05:16.493536+0000","last_peered":"2023-11-06T03:05:16.493536+0000","last_clean":"2023-11-06T03:05:16.493536+0000","last_became_active":"2023-11-06T03:05:14.642159+0000","last_became_peered":"2023-11-06T03:05:14.642159+0000","last_unstale":"2023-11-06T03:05:16.493536+0000","last_undegraded":"2023-11-06T03:05:16.493536+0000","last_fullsized":"2023-11-06T03:05:16.493536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:54.243548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.619086+0000","last_change":"2023-11-06T03:05:14.617944+0000","last_active":"2023-11-06T03:05:15.619086+0000","last_peered":"2023-11-06T03:05:15.619086+0000","last_clean":"2023-11-06T03:05:15.619086+0000","last_became_active":"2023-11-06T03:05:14.617660+0000","last_became_peered":"2023-11-06T03:05:14.617660+0000","last_unstale":"2023-11-06T03:05:15.619086+0000","last_undegraded":"2023-11-06T03:05:15.619086+0000","last_fullsized":"2023-11-06T03:05:15.619086+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:30.794707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493021+0000","last_change":"2023-11-06T03:05:14.655799+0000","last_active":"2023-11-06T03:05:16.493021+0000","last_peered":"2023-11-06T03:05:16.493021+0000","last_clean":"2023-11-06T03:05:16.493021+0000","last_became_active":"2023-11-06T03:05:14.655148+0000","last_became_peered":"2023-11-06T03:05:14.655148+0000","last_unstale":"2023-11-06T03:05:16.493021+0000","last_undegraded":"2023-11-06T03:05:16.493021+0000","last_fullsized":"2023-11-06T03:05:16.493021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:37.328355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.495132+0000","last_change":"2023-11-06T03:05:14.656795+0000","last_active":"2023-11-06T03:05:16.495132+0000","last_peered":"2023-11-06T03:05:16.495132+0000","last_clean":"2023-11-06T03:05:16.495132+0000","last_became_active":"2023-11-06T03:05:14.656005+0000","last_became_peered":"2023-11-06T03:05:14.656005+0000","last_unstale":"2023-11-06T03:05:16.495132+0000","last_undegraded":"2023-11-06T03:05:16.495132+0000","last_fullsized":"2023-11-06T03:05:16.495132+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:13:44.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.621268+0000","last_change":"2023-11-06T03:05:14.675031+0000","last_active":"2023-11-06T03:05:15.621268+0000","last_peered":"2023-11-06T03:05:15.621268+0000","last_clean":"2023-11-06T03:05:15.621268+0000","last_became_active":"2023-11-06T03:05:14.674347+0000","last_became_peered":"2023-11-06T03:05:14.674347+0000","last_unstale":"2023-11-06T03:05:15.621268+0000","last_undegraded":"2023-11-06T03:05:15.621268+0000","last_fullsized":"2023-11-06T03:05:15.621268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:58.658929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492821+0000","last_change":"2023-11-06T03:05:14.669142+0000","last_active":"2023-11-06T03:05:16.492821+0000","last_peered":"2023-11-06T03:05:16.492821+0000","last_clean":"2023-11-06T03:05:16.492821+0000","last_became_active":"2023-11-06T03:05:14.668368+0000","last_became_peered":"2023-11-06T03:05:14.668368+0000","last_unstale":"2023-11-06T03:05:16.492821+0000","last_undegraded":"2023-11-06T03:05:16.492821+0000","last_fullsized":"2023-11-06T03:05:16.492821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:11:06.443996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.617975+0000","last_change":"2023-11-06T03:05:14.670059+0000","last_active":"2023-11-06T03:05:15.617975+0000","last_peered":"2023-11-06T03:05:15.617975+0000","last_clean":"2023-11-06T03:05:15.617975+0000","last_became_active":"2023-11-06T03:05:14.669693+0000","last_became_peered":"2023-11-06T03:05:14.669693+0000","last_unstale":"2023-11-06T03:05:15.617975+0000","last_undegraded":"2023-11-06T03:05:15.617975+0000","last_fullsized":"2023-11-06T03:05:15.617975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:01.668865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494538+0000","last_change":"2023-11-06T03:05:14.633051+0000","last_active":"2023-11-06T03:05:16.494538+0000","last_peered":"2023-11-06T03:05:16.494538+0000","last_clean":"2023-11-06T03:05:16.494538+0000","last_became_active":"2023-11-06T03:05:14.631893+0000","last_became_peered":"2023-11-06T03:05:14.631893+0000","last_unstale":"2023-11-06T03:05:16.494538+0000","last_undegraded":"2023-11-06T03:05:16.494538+0000","last_fullsized":"2023-11-06T03:05:16.494538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:07:06.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.802222+0000","last_change":"2023-11-06T03:05:14.659772+0000","last_active":"2023-11-06T03:05:15.802222+0000","last_peered":"2023-11-06T03:05:15.802222+0000","last_clean":"2023-11-06T03:05:15.802222+0000","last_became_active":"2023-11-06T03:05:14.659493+0000","last_became_peered":"2023-11-06T03:05:14.659493+0000","last_unstale":"2023-11-06T03:05:15.802222+0000","last_undegraded":"2023-11-06T03:05:15.802222+0000","last_fullsized":"2023-11-06T03:05:15.802222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:10.576125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493635+0000","last_change":"2023-11-06T03:05:14.619651+0000","last_active":"2023-11-06T03:05:16.493635+0000","last_peered":"2023-11-06T03:05:16.493635+0000","last_clean":"2023-11-06T03:05:16.493635+0000","last_became_active":"2023-11-06T03:05:14.618937+0000","last_became_peered":"2023-11-06T03:05:14.618937+0000","last_unstale":"2023-11-06T03:05:16.493635+0000","last_undegraded":"2023-11-06T03:05:16.493635+0000","last_fullsized":"2023-11-06T03:05:16.493635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:33.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494319+0000","last_change":"2023-11-06T03:05:14.658760+0000","last_active":"2023-11-06T03:05:16.494319+0000","last_peered":"2023-11-06T03:05:16.494319+0000","last_clean":"2023-11-06T03:05:16.494319+0000","last_became_active":"2023-11-06T03:05:14.656586+0000","last_became_peered":"2023-11-06T03:05:14.656586+0000","last_unstale":"2023-11-06T03:05:16.494319+0000","last_undegraded":"2023-11-06T03:05:16.494319+0000","last_fullsized":"2023-11-06T03:05:16.494319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:58.658929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492749+0000","last_change":"2023-11-06T03:05:14.618064+0000","last_active":"2023-11-06T03:05:16.492749+0000","last_peered":"2023-11-06T03:05:16.492749+0000","last_clean":"2023-11-06T03:05:16.492749+0000","last_became_active":"2023-11-06T03:05:14.617281+0000","last_became_peered":"2023-11-06T03:05:14.617281+0000","last_unstale":"2023-11-06T03:05:16.492749+0000","last_undegraded":"2023-11-06T03:05:16.492749+0000","last_fullsized":"2023-11-06T03:05:16.492749+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:30.794707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492713+0000","last_change":"2023-11-06T03:05:14.630691+0000","last_active":"2023-11-06T03:05:16.492713+0000","last_peered":"2023-11-06T03:05:16.492713+0000","last_clean":"2023-11-06T03:05:16.492713+0000","last_became_active":"2023-11-06T03:05:14.628709+0000","last_became_peered":"2023-11-06T03:05:14.628709+0000","last_unstale":"2023-11-06T03:05:16.492713+0000","last_undegraded":"2023-11-06T03:05:16.492713+0000","last_fullsized":"2023-11-06T03:05:16.492713+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:16.462369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492985+0000","last_change":"2023-11-06T03:05:14.669044+0000","last_active":"2023-11-06T03:05:16.492985+0000","last_peered":"2023-11-06T03:05:16.492985+0000","last_clean":"2023-11-06T03:05:16.492985+0000","last_became_active":"2023-11-06T03:05:14.668220+0000","last_became_peered":"2023-11-06T03:05:14.668220+0000","last_unstale":"2023-11-06T03:05:16.492985+0000","last_undegraded":"2023-11-06T03:05:16.492985+0000","last_fullsized":"2023-11-06T03:05:16.492985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:33.717536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.801612+0000","last_change":"2023-11-06T03:05:14.619295+0000","last_active":"2023-11-06T03:05:15.801612+0000","last_peered":"2023-11-06T03:05:15.801612+0000","last_clean":"2023-11-06T03:05:15.801612+0000","last_became_active":"2023-11-06T03:05:14.618962+0000","last_became_peered":"2023-11-06T03:05:14.618962+0000","last_unstale":"2023-11-06T03:05:15.801612+0000","last_undegraded":"2023-11-06T03:05:15.801612+0000","last_fullsized":"2023-11-06T03:05:15.801612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:29.172412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493849+0000","last_change":"2023-11-06T03:05:14.677497+0000","last_active":"2023-11-06T03:05:16.493849+0000","last_peered":"2023-11-06T03:05:16.493849+0000","last_clean":"2023-11-06T03:05:16.493849+0000","last_became_active":"2023-11-06T03:05:14.676910+0000","last_became_peered":"2023-11-06T03:05:14.676910+0000","last_unstale":"2023-11-06T03:05:16.493849+0000","last_undegraded":"2023-11-06T03:05:16.493849+0000","last_fullsized":"2023-11-06T03:05:16.493849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:13:44.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.676610+0000","last_change":"2023-11-06T03:05:14.271534+0000","last_active":"2023-11-06T03:05:14.676610+0000","last_peered":"2023-11-06T03:05:14.676610+0000","last_clean":"2023-11-06T03:05:14.676610+0000","last_became_active":"2023-11-06T03:05:14.270778+0000","last_became_peered":"2023-11-06T03:05:14.270778+0000","last_unstale":"2023-11-06T03:05:14.676610+0000","last_undegraded":"2023-11-06T03:05:14.676610+0000","last_fullsized":"2023-11-06T03:05:14.676610+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.801989+0000","last_change":"2023-11-06T03:05:14.662475+0000","last_active":"2023-11-06T03:05:15.801989+0000","last_peered":"2023-11-06T03:05:15.801989+0000","last_clean":"2023-11-06T03:05:15.801989+0000","last_became_active":"2023-11-06T03:05:14.662088+0000","last_became_peered":"2023-11-06T03:05:14.662088+0000","last_unstale":"2023-11-06T03:05:15.801989+0000","last_undegraded":"2023-11-06T03:05:15.801989+0000","last_fullsized":"2023-11-06T03:05:15.801989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:45.865989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.668234+0000","last_change":"2023-11-06T03:05:14.277653+0000","last_active":"2023-11-06T03:05:14.668234+0000","last_peered":"2023-11-06T03:05:14.668234+0000","last_clean":"2023-11-06T03:05:14.668234+0000","last_became_active":"2023-11-06T03:05:14.277255+0000","last_became_peered":"2023-11-06T03:05:14.277255+0000","last_unstale":"2023-11-06T03:05:14.668234+0000","last_undegraded":"2023-11-06T03:05:14.668234+0000","last_fullsized":"2023-11-06T03:05:14.668234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:15:12.401530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620806+0000","last_change":"2023-11-06T03:05:14.651294+0000","last_active":"2023-11-06T03:05:15.620806+0000","last_peered":"2023-11-06T03:05:15.620806+0000","last_clean":"2023-11-06T03:05:15.620806+0000","last_became_active":"2023-11-06T03:05:14.650576+0000","last_became_peered":"2023-11-06T03:05:14.650576+0000","last_unstale":"2023-11-06T03:05:15.620806+0000","last_undegraded":"2023-11-06T03:05:15.620806+0000","last_fullsized":"2023-11-06T03:05:15.620806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:41:51.574212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.674547+0000","last_change":"2023-11-06T03:05:14.248905+0000","last_active":"2023-11-06T03:05:14.674547+0000","last_peered":"2023-11-06T03:05:14.674547+0000","last_clean":"2023-11-06T03:05:14.674547+0000","last_became_active":"2023-11-06T03:05:14.248562+0000","last_became_peered":"2023-11-06T03:05:14.248562+0000","last_unstale":"2023-11-06T03:05:14.674547+0000","last_undegraded":"2023-11-06T03:05:14.674547+0000","last_fullsized":"2023-11-06T03:05:14.674547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:15.451696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.173155+0000","last_change":"2023-11-06T03:05:14.632462+0000","last_active":"2023-11-06T03:05:16.173155+0000","last_peered":"2023-11-06T03:05:16.173155+0000","last_clean":"2023-11-06T03:05:16.173155+0000","last_became_active":"2023-11-06T03:05:14.632123+0000","last_became_peered":"2023-11-06T03:05:14.632123+0000","last_unstale":"2023-11-06T03:05:16.173155+0000","last_undegraded":"2023-11-06T03:05:16.173155+0000","last_fullsized":"2023-11-06T03:05:16.173155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:41:51.574212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.674136+0000","last_change":"2023-11-06T03:05:14.261635+0000","last_active":"2023-11-06T03:05:14.674136+0000","last_peered":"2023-11-06T03:05:14.674136+0000","last_clean":"2023-11-06T03:05:14.674136+0000","last_became_active":"2023-11-06T03:05:14.260323+0000","last_became_peered":"2023-11-06T03:05:14.260323+0000","last_unstale":"2023-11-06T03:05:14.674136+0000","last_undegraded":"2023-11-06T03:05:14.674136+0000","last_fullsized":"2023-11-06T03:05:14.674136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:15:44.855316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.491450+0000","last_change":"2023-11-06T03:05:14.627661+0000","last_active":"2023-11-06T03:05:16.491450+0000","last_peered":"2023-11-06T03:05:16.491450+0000","last_clean":"2023-11-06T03:05:16.491450+0000","last_became_active":"2023-11-06T03:05:14.627378+0000","last_became_peered":"2023-11-06T03:05:14.627378+0000","last_unstale":"2023-11-06T03:05:16.491450+0000","last_undegraded":"2023-11-06T03:05:16.491450+0000","last_fullsized":"2023-11-06T03:05:16.491450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:37:10.380953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.676111+0000","last_change":"2023-11-06T03:05:14.261875+0000","last_active":"2023-11-06T03:05:14.676111+0000","last_peered":"2023-11-06T03:05:14.676111+0000","last_clean":"2023-11-06T03:05:14.676111+0000","last_became_active":"2023-11-06T03:05:14.261064+0000","last_became_peered":"2023-11-06T03:05:14.261064+0000","last_unstale":"2023-11-06T03:05:14.676111+0000","last_undegraded":"2023-11-06T03:05:14.676111+0000","last_fullsized":"2023-11-06T03:05:14.676111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.495438+0000","last_change":"2023-11-06T03:05:14.656452+0000","last_active":"2023-11-06T03:05:16.495438+0000","last_peered":"2023-11-06T03:05:16.495438+0000","last_clean":"2023-11-06T03:05:16.495438+0000","last_became_active":"2023-11-06T03:05:14.655663+0000","last_became_peered":"2023-11-06T03:05:14.655663+0000","last_unstale":"2023-11-06T03:05:16.495438+0000","last_undegraded":"2023-11-06T03:05:16.495438+0000","last_fullsized":"2023-11-06T03:05:16.495438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:37:10.380953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.675734+0000","last_change":"2023-11-06T03:05:14.279887+0000","last_active":"2023-11-06T03:05:14.675734+0000","last_peered":"2023-11-06T03:05:14.675734+0000","last_clean":"2023-11-06T03:05:14.675734+0000","last_became_active":"2023-11-06T03:05:14.279205+0000","last_became_peered":"2023-11-06T03:05:14.279205+0000","last_unstale":"2023-11-06T03:05:14.675734+0000","last_undegraded":"2023-11-06T03:05:14.675734+0000","last_fullsized":"2023-11-06T03:05:14.675734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620890+0000","last_change":"2023-11-06T03:05:14.663423+0000","last_active":"2023-11-06T03:05:15.620890+0000","last_peered":"2023-11-06T03:05:15.620890+0000","last_clean":"2023-11-06T03:05:15.620890+0000","last_became_active":"2023-11-06T03:05:14.662534+0000","last_became_peered":"2023-11-06T03:05:14.662534+0000","last_unstale":"2023-11-06T03:05:15.620890+0000","last_undegraded":"2023-11-06T03:05:15.620890+0000","last_fullsized":"2023-11-06T03:05:15.620890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:11.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.614505+0000","last_change":"2023-11-06T03:05:14.267687+0000","last_active":"2023-11-06T03:05:14.614505+0000","last_peered":"2023-11-06T03:05:14.614505+0000","last_clean":"2023-11-06T03:05:14.614505+0000","last_became_active":"2023-11-06T03:05:14.267158+0000","last_became_peered":"2023-11-06T03:05:14.267158+0000","last_unstale":"2023-11-06T03:05:14.614505+0000","last_undegraded":"2023-11-06T03:05:14.614505+0000","last_fullsized":"2023-11-06T03:05:14.614505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:32.523436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172536+0000","last_change":"2023-11-06T03:05:14.670912+0000","last_active":"2023-11-06T03:05:16.172536+0000","last_peered":"2023-11-06T03:05:16.172536+0000","last_clean":"2023-11-06T03:05:16.172536+0000","last_became_active":"2023-11-06T03:05:14.670630+0000","last_became_peered":"2023-11-06T03:05:14.670630+0000","last_unstale":"2023-11-06T03:05:16.172536+0000","last_undegraded":"2023-11-06T03:05:16.172536+0000","last_fullsized":"2023-11-06T03:05:16.172536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:11.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.620647+0000","last_change":"2023-11-06T03:05:14.274750+0000","last_active":"2023-11-06T03:05:14.620647+0000","last_peered":"2023-11-06T03:05:14.620647+0000","last_clean":"2023-11-06T03:05:14.620647+0000","last_became_active":"2023-11-06T03:05:14.273962+0000","last_became_peered":"2023-11-06T03:05:14.273962+0000","last_unstale":"2023-11-06T03:05:14.620647+0000","last_undegraded":"2023-11-06T03:05:14.620647+0000","last_fullsized":"2023-11-06T03:05:14.620647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:15:09.565452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.619001+0000","last_change":"2023-11-06T03:05:14.651080+0000","last_active":"2023-11-06T03:05:15.619001+0000","last_peered":"2023-11-06T03:05:15.619001+0000","last_clean":"2023-11-06T03:05:15.619001+0000","last_became_active":"2023-11-06T03:05:14.650812+0000","last_became_peered":"2023-11-06T03:05:14.650812+0000","last_unstale":"2023-11-06T03:05:15.619001+0000","last_undegraded":"2023-11-06T03:05:15.619001+0000","last_fullsized":"2023-11-06T03:05:15.619001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:54.243548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.619847+0000","last_change":"2023-11-06T03:05:14.248519+0000","last_active":"2023-11-06T03:05:14.619847+0000","last_peered":"2023-11-06T03:05:14.619847+0000","last_clean":"2023-11-06T03:05:14.619847+0000","last_became_active":"2023-11-06T03:05:14.248167+0000","last_became_peered":"2023-11-06T03:05:14.248167+0000","last_unstale":"2023-11-06T03:05:14.619847+0000","last_undegraded":"2023-11-06T03:05:14.619847+0000","last_fullsized":"2023-11-06T03:05:14.619847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494162+0000","last_change":"2023-11-06T03:05:14.668470+0000","last_active":"2023-11-06T03:05:16.494162+0000","last_peered":"2023-11-06T03:05:16.494162+0000","last_clean":"2023-11-06T03:05:16.494162+0000","last_became_active":"2023-11-06T03:05:14.667919+0000","last_became_peered":"2023-11-06T03:05:14.667919+0000","last_unstale":"2023-11-06T03:05:16.494162+0000","last_undegraded":"2023-11-06T03:05:16.494162+0000","last_fullsized":"2023-11-06T03:05:16.494162+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:58.658929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.627846+0000","last_change":"2023-11-06T03:05:14.253918+0000","last_active":"2023-11-06T03:05:14.627846+0000","last_peered":"2023-11-06T03:05:14.627846+0000","last_clean":"2023-11-06T03:05:14.627846+0000","last_became_active":"2023-11-06T03:05:14.253133+0000","last_became_peered":"2023-11-06T03:05:14.253133+0000","last_unstale":"2023-11-06T03:05:14.627846+0000","last_undegraded":"2023-11-06T03:05:14.627846+0000","last_fullsized":"2023-11-06T03:05:14.627846+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.492276+0000","last_change":"2023-11-06T03:05:14.668934+0000","last_active":"2023-11-06T03:05:16.492276+0000","last_peered":"2023-11-06T03:05:16.492276+0000","last_clean":"2023-11-06T03:05:16.492276+0000","last_became_active":"2023-11-06T03:05:14.668626+0000","last_became_peered":"2023-11-06T03:05:14.668626+0000","last_unstale":"2023-11-06T03:05:16.492276+0000","last_undegraded":"2023-11-06T03:05:16.492276+0000","last_fullsized":"2023-11-06T03:05:16.492276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:11.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.678205+0000","last_change":"2023-11-06T03:05:14.246374+0000","last_active":"2023-11-06T03:05:14.678205+0000","last_peered":"2023-11-06T03:05:14.678205+0000","last_clean":"2023-11-06T03:05:14.678205+0000","last_became_active":"2023-11-06T03:05:14.244991+0000","last_became_peered":"2023-11-06T03:05:14.244991+0000","last_unstale":"2023-11-06T03:05:14.678205+0000","last_undegraded":"2023-11-06T03:05:14.678205+0000","last_fullsized":"2023-11-06T03:05:14.678205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620775+0000","last_change":"2023-11-06T03:05:14.641030+0000","last_active":"2023-11-06T03:05:15.620775+0000","last_peered":"2023-11-06T03:05:15.620775+0000","last_clean":"2023-11-06T03:05:15.620775+0000","last_became_active":"2023-11-06T03:05:14.640310+0000","last_became_peered":"2023-11-06T03:05:14.640310+0000","last_unstale":"2023-11-06T03:05:15.620775+0000","last_undegraded":"2023-11-06T03:05:15.620775+0000","last_fullsized":"2023-11-06T03:05:15.620775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:23.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.683534+0000","last_change":"2023-11-06T03:05:14.277249+0000","last_active":"2023-11-06T03:05:14.683534+0000","last_peered":"2023-11-06T03:05:14.683534+0000","last_clean":"2023-11-06T03:05:14.683534+0000","last_became_active":"2023-11-06T03:05:14.276985+0000","last_became_peered":"2023-11-06T03:05:14.276985+0000","last_unstale":"2023-11-06T03:05:14.683534+0000","last_undegraded":"2023-11-06T03:05:14.683534+0000","last_fullsized":"2023-11-06T03:05:14.683534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:06.506146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.491610+0000","last_change":"2023-11-06T03:05:14.660338+0000","last_active":"2023-11-06T03:05:16.491610+0000","last_peered":"2023-11-06T03:05:16.491610+0000","last_clean":"2023-11-06T03:05:16.491610+0000","last_became_active":"2023-11-06T03:05:14.660027+0000","last_became_peered":"2023-11-06T03:05:14.660027+0000","last_unstale":"2023-11-06T03:05:16.491610+0000","last_undegraded":"2023-11-06T03:05:16.491610+0000","last_fullsized":"2023-11-06T03:05:16.491610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:58.658929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.633892+0000","last_change":"2023-11-06T03:05:14.249357+0000","last_active":"2023-11-06T03:05:14.633892+0000","last_peered":"2023-11-06T03:05:14.633892+0000","last_clean":"2023-11-06T03:05:14.633892+0000","last_became_active":"2023-11-06T03:05:14.248482+0000","last_became_peered":"2023-11-06T03:05:14.248482+0000","last_unstale":"2023-11-06T03:05:14.633892+0000","last_undegraded":"2023-11-06T03:05:14.633892+0000","last_fullsized":"2023-11-06T03:05:14.633892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172433+0000","last_change":"2023-11-06T03:05:14.633696+0000","last_active":"2023-11-06T03:05:16.172433+0000","last_peered":"2023-11-06T03:05:16.172433+0000","last_clean":"2023-11-06T03:05:16.172433+0000","last_became_active":"2023-11-06T03:05:14.633413+0000","last_became_peered":"2023-11-06T03:05:14.633413+0000","last_unstale":"2023-11-06T03:05:16.172433+0000","last_undegraded":"2023-11-06T03:05:16.172433+0000","last_fullsized":"2023-11-06T03:05:16.172433+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:06:00.045958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.614724+0000","last_change":"2023-11-06T03:05:14.252569+0000","last_active":"2023-11-06T03:05:14.614724+0000","last_peered":"2023-11-06T03:05:14.614724+0000","last_clean":"2023-11-06T03:05:14.614724+0000","last_became_active":"2023-11-06T03:05:14.252042+0000","last_became_peered":"2023-11-06T03:05:14.252042+0000","last_unstale":"2023-11-06T03:05:14.614724+0000","last_undegraded":"2023-11-06T03:05:14.614724+0000","last_fullsized":"2023-11-06T03:05:14.614724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.232876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493278+0000","last_change":"2023-11-06T03:05:14.670154+0000","last_active":"2023-11-06T03:05:16.493278+0000","last_peered":"2023-11-06T03:05:16.493278+0000","last_clean":"2023-11-06T03:05:16.493278+0000","last_became_active":"2023-11-06T03:05:14.669671+0000","last_became_peered":"2023-11-06T03:05:14.669671+0000","last_unstale":"2023-11-06T03:05:16.493278+0000","last_undegraded":"2023-11-06T03:05:16.493278+0000","last_fullsized":"2023-11-06T03:05:16.493278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:42.557952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.612364+0000","last_change":"2023-11-06T03:05:14.272613+0000","last_active":"2023-11-06T03:05:14.612364+0000","last_peered":"2023-11-06T03:05:14.612364+0000","last_clean":"2023-11-06T03:05:14.612364+0000","last_became_active":"2023-11-06T03:05:14.271919+0000","last_became_peered":"2023-11-06T03:05:14.271919+0000","last_unstale":"2023-11-06T03:05:14.612364+0000","last_undegraded":"2023-11-06T03:05:14.612364+0000","last_fullsized":"2023-11-06T03:05:14.612364+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172791+0000","last_change":"2023-11-06T03:05:14.634039+0000","last_active":"2023-11-06T03:05:16.172791+0000","last_peered":"2023-11-06T03:05:16.172791+0000","last_clean":"2023-11-06T03:05:16.172791+0000","last_became_active":"2023-11-06T03:05:14.633614+0000","last_became_peered":"2023-11-06T03:05:14.633614+0000","last_unstale":"2023-11-06T03:05:16.172791+0000","last_undegraded":"2023-11-06T03:05:16.172791+0000","last_fullsized":"2023-11-06T03:05:16.172791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:23.987183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.681891+0000","last_change":"2023-11-06T03:05:14.280222+0000","last_active":"2023-11-06T03:05:14.681891+0000","last_peered":"2023-11-06T03:05:14.681891+0000","last_clean":"2023-11-06T03:05:14.681891+0000","last_became_active":"2023-11-06T03:05:14.279061+0000","last_became_peered":"2023-11-06T03:05:14.279061+0000","last_unstale":"2023-11-06T03:05:14.681891+0000","last_undegraded":"2023-11-06T03:05:14.681891+0000","last_fullsized":"2023-11-06T03:05:14.681891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:36.317683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172864+0000","last_change":"2023-11-06T03:05:14.632220+0000","last_active":"2023-11-06T03:05:16.172864+0000","last_peered":"2023-11-06T03:05:16.172864+0000","last_clean":"2023-11-06T03:05:16.172864+0000","last_became_active":"2023-11-06T03:05:14.631894+0000","last_became_peered":"2023-11-06T03:05:14.631894+0000","last_unstale":"2023-11-06T03:05:16.172864+0000","last_undegraded":"2023-11-06T03:05:16.172864+0000","last_fullsized":"2023-11-06T03:05:16.172864+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:23:50.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.676999+0000","last_change":"2023-11-06T03:05:14.256855+0000","last_active":"2023-11-06T03:05:14.676999+0000","last_peered":"2023-11-06T03:05:14.676999+0000","last_clean":"2023-11-06T03:05:14.676999+0000","last_became_active":"2023-11-06T03:05:14.255368+0000","last_became_peered":"2023-11-06T03:05:14.255368+0000","last_unstale":"2023-11-06T03:05:14.676999+0000","last_undegraded":"2023-11-06T03:05:14.676999+0000","last_fullsized":"2023-11-06T03:05:14.676999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494525+0000","last_change":"2023-11-06T03:05:14.660739+0000","last_active":"2023-11-06T03:05:16.494525+0000","last_peered":"2023-11-06T03:05:16.494525+0000","last_clean":"2023-11-06T03:05:16.494525+0000","last_became_active":"2023-11-06T03:05:14.659270+0000","last_became_peered":"2023-11-06T03:05:14.659270+0000","last_unstale":"2023-11-06T03:05:16.494525+0000","last_undegraded":"2023-11-06T03:05:16.494525+0000","last_fullsized":"2023-11-06T03:05:16.494525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:42.557952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.679043+0000","last_change":"2023-11-06T03:05:14.268318+0000","last_active":"2023-11-06T03:05:14.679043+0000","last_peered":"2023-11-06T03:05:14.679043+0000","last_clean":"2023-11-06T03:05:14.679043+0000","last_became_active":"2023-11-06T03:05:14.267618+0000","last_became_peered":"2023-11-06T03:05:14.267618+0000","last_unstale":"2023-11-06T03:05:14.679043+0000","last_undegraded":"2023-11-06T03:05:14.679043+0000","last_fullsized":"2023-11-06T03:05:14.679043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:18.876110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620733+0000","last_change":"2023-11-06T03:05:14.639189+0000","last_active":"2023-11-06T03:05:15.620733+0000","last_peered":"2023-11-06T03:05:15.620733+0000","last_clean":"2023-11-06T03:05:15.620733+0000","last_became_active":"2023-11-06T03:05:14.638411+0000","last_became_peered":"2023-11-06T03:05:14.638411+0000","last_unstale":"2023-11-06T03:05:15.620733+0000","last_undegraded":"2023-11-06T03:05:15.620733+0000","last_fullsized":"2023-11-06T03:05:15.620733+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:37:10.380953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.668102+0000","last_change":"2023-11-06T03:05:14.273701+0000","last_active":"2023-11-06T03:05:14.668102+0000","last_peered":"2023-11-06T03:05:14.668102+0000","last_clean":"2023-11-06T03:05:14.668102+0000","last_became_active":"2023-11-06T03:05:14.273094+0000","last_became_peered":"2023-11-06T03:05:14.273094+0000","last_unstale":"2023-11-06T03:05:14.668102+0000","last_undegraded":"2023-11-06T03:05:14.668102+0000","last_fullsized":"2023-11-06T03:05:14.668102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.619284+0000","last_change":"2023-11-06T03:05:14.621628+0000","last_active":"2023-11-06T03:05:15.619284+0000","last_peered":"2023-11-06T03:05:15.619284+0000","last_clean":"2023-11-06T03:05:15.619284+0000","last_became_active":"2023-11-06T03:05:14.621329+0000","last_became_peered":"2023-11-06T03:05:14.621329+0000","last_unstale":"2023-11-06T03:05:15.619284+0000","last_undegraded":"2023-11-06T03:05:15.619284+0000","last_fullsized":"2023-11-06T03:05:15.619284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:33.534108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.616010+0000","last_change":"2023-11-06T03:05:14.251819+0000","last_active":"2023-11-06T03:05:14.616010+0000","last_peered":"2023-11-06T03:05:14.616010+0000","last_clean":"2023-11-06T03:05:14.616010+0000","last_became_active":"2023-11-06T03:05:14.251042+0000","last_became_peered":"2023-11-06T03:05:14.251042+0000","last_unstale":"2023-11-06T03:05:14.616010+0000","last_undegraded":"2023-11-06T03:05:14.616010+0000","last_fullsized":"2023-11-06T03:05:14.616010+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:29.784035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494710+0000","last_change":"2023-11-06T03:05:14.671908+0000","last_active":"2023-11-06T03:05:16.494710+0000","last_peered":"2023-11-06T03:05:16.494710+0000","last_clean":"2023-11-06T03:05:16.494710+0000","last_became_active":"2023-11-06T03:05:14.671444+0000","last_became_peered":"2023-11-06T03:05:14.671444+0000","last_unstale":"2023-11-06T03:05:16.494710+0000","last_undegraded":"2023-11-06T03:05:16.494710+0000","last_fullsized":"2023-11-06T03:05:16.494710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:37:02.129386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.636751+0000","last_change":"2023-11-06T03:05:14.249760+0000","last_active":"2023-11-06T03:05:14.636751+0000","last_peered":"2023-11-06T03:05:14.636751+0000","last_clean":"2023-11-06T03:05:14.636751+0000","last_became_active":"2023-11-06T03:05:14.249248+0000","last_became_peered":"2023-11-06T03:05:14.249248+0000","last_unstale":"2023-11-06T03:05:14.636751+0000","last_undegraded":"2023-11-06T03:05:14.636751+0000","last_fullsized":"2023-11-06T03:05:14.636751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:09.403307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.491878+0000","last_change":"2023-11-06T03:05:14.671001+0000","last_active":"2023-11-06T03:05:16.491878+0000","last_peered":"2023-11-06T03:05:16.491878+0000","last_clean":"2023-11-06T03:05:16.491878+0000","last_became_active":"2023-11-06T03:05:14.670685+0000","last_became_peered":"2023-11-06T03:05:14.670685+0000","last_unstale":"2023-11-06T03:05:16.491878+0000","last_undegraded":"2023-11-06T03:05:16.491878+0000","last_fullsized":"2023-11-06T03:05:16.491878+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:41:51.574212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.608961+0000","last_change":"2023-11-06T03:05:14.271912+0000","last_active":"2023-11-06T03:05:14.608961+0000","last_peered":"2023-11-06T03:05:14.608961+0000","last_clean":"2023-11-06T03:05:14.608961+0000","last_became_active":"2023-11-06T03:05:14.271512+0000","last_became_peered":"2023-11-06T03:05:14.271512+0000","last_unstale":"2023-11-06T03:05:14.608961+0000","last_undegraded":"2023-11-06T03:05:14.608961+0000","last_fullsized":"2023-11-06T03:05:14.608961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:41:56.059747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493411+0000","last_change":"2023-11-06T03:05:14.657077+0000","last_active":"2023-11-06T03:05:16.493411+0000","last_peered":"2023-11-06T03:05:16.493411+0000","last_clean":"2023-11-06T03:05:16.493411+0000","last_became_active":"2023-11-06T03:05:14.656060+0000","last_became_peered":"2023-11-06T03:05:14.656060+0000","last_unstale":"2023-11-06T03:05:16.493411+0000","last_undegraded":"2023-11-06T03:05:16.493411+0000","last_fullsized":"2023-11-06T03:05:16.493411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:58:14.178485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.641681+0000","last_change":"2023-11-06T03:05:14.277501+0000","last_active":"2023-11-06T03:05:14.641681+0000","last_peered":"2023-11-06T03:05:14.641681+0000","last_clean":"2023-11-06T03:05:14.641681+0000","last_became_active":"2023-11-06T03:05:14.277021+0000","last_became_peered":"2023-11-06T03:05:14.277021+0000","last_unstale":"2023-11-06T03:05:14.641681+0000","last_undegraded":"2023-11-06T03:05:14.641681+0000","last_fullsized":"2023-11-06T03:05:14.641681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:00.658193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172950+0000","last_change":"2023-11-06T03:05:14.668935+0000","last_active":"2023-11-06T03:05:16.172950+0000","last_peered":"2023-11-06T03:05:16.172950+0000","last_clean":"2023-11-06T03:05:16.172950+0000","last_became_active":"2023-11-06T03:05:14.668655+0000","last_became_peered":"2023-11-06T03:05:14.668655+0000","last_unstale":"2023-11-06T03:05:16.172950+0000","last_undegraded":"2023-11-06T03:05:16.172950+0000","last_fullsized":"2023-11-06T03:05:16.172950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:58.658929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.609282+0000","last_change":"2023-11-06T03:05:14.250928+0000","last_active":"2023-11-06T03:05:14.609282+0000","last_peered":"2023-11-06T03:05:14.609282+0000","last_clean":"2023-11-06T03:05:14.609282+0000","last_became_active":"2023-11-06T03:05:14.249962+0000","last_became_peered":"2023-11-06T03:05:14.249962+0000","last_unstale":"2023-11-06T03:05:14.609282+0000","last_undegraded":"2023-11-06T03:05:14.609282+0000","last_fullsized":"2023-11-06T03:05:14.609282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:22.047291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494535+0000","last_change":"2023-11-06T03:05:14.657447+0000","last_active":"2023-11-06T03:05:16.494535+0000","last_peered":"2023-11-06T03:05:16.494535+0000","last_clean":"2023-11-06T03:05:16.494535+0000","last_became_active":"2023-11-06T03:05:14.656611+0000","last_became_peered":"2023-11-06T03:05:14.656611+0000","last_unstale":"2023-11-06T03:05:16.494535+0000","last_undegraded":"2023-11-06T03:05:16.494535+0000","last_fullsized":"2023-11-06T03:05:16.494535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:11.933664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.651917+0000","last_change":"2023-11-06T03:05:14.279865+0000","last_active":"2023-11-06T03:05:14.651917+0000","last_peered":"2023-11-06T03:05:14.651917+0000","last_clean":"2023-11-06T03:05:14.651917+0000","last_became_active":"2023-11-06T03:05:14.278652+0000","last_became_peered":"2023-11-06T03:05:14.278652+0000","last_unstale":"2023-11-06T03:05:14.651917+0000","last_undegraded":"2023-11-06T03:05:14.651917+0000","last_fullsized":"2023-11-06T03:05:14.651917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:22:42.902665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.650902+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:05:14.650902+0000","last_peered":"2023-11-06T03:05:14.650902+0000","last_clean":"2023-11-06T03:05:14.650902+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:05:14.650902+0000","last_undegraded":"2023-11-06T03:05:14.650902+0000","last_fullsized":"2023-11-06T03:05:14.650902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.627209+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:05:14.627209+0000","last_peered":"2023-11-06T03:05:14.627209+0000","last_clean":"2023-11-06T03:05:14.627209+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:05:14.627209+0000","last_undegraded":"2023-11-06T03:05:14.627209+0000","last_fullsized":"2023-11-06T03:05:14.627209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.801775+0000","last_change":"2023-11-06T03:05:14.659065+0000","last_active":"2023-11-06T03:05:15.801775+0000","last_peered":"2023-11-06T03:05:15.801775+0000","last_clean":"2023-11-06T03:05:15.801775+0000","last_became_active":"2023-11-06T03:05:14.658738+0000","last_became_peered":"2023-11-06T03:05:14.658738+0000","last_unstale":"2023-11-06T03:05:15.801775+0000","last_undegraded":"2023-11-06T03:05:15.801775+0000","last_fullsized":"2023-11-06T03:05:15.801775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:19.886782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.676697+0000","last_change":"2023-11-06T03:05:14.245684+0000","last_active":"2023-11-06T03:05:14.676697+0000","last_peered":"2023-11-06T03:05:14.676697+0000","last_clean":"2023-11-06T03:05:14.676697+0000","last_became_active":"2023-11-06T03:05:14.244958+0000","last_became_peered":"2023-11-06T03:05:14.244958+0000","last_unstale":"2023-11-06T03:05:14.676697+0000","last_undegraded":"2023-11-06T03:05:14.676697+0000","last_fullsized":"2023-11-06T03:05:14.676697+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.650455+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:05:14.650455+0000","last_peered":"2023-11-06T03:05:14.650455+0000","last_clean":"2023-11-06T03:05:14.650455+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:05:14.650455+0000","last_undegraded":"2023-11-06T03:05:14.650455+0000","last_fullsized":"2023-11-06T03:05:14.650455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.494417+0000","last_change":"2023-11-06T03:05:14.670617+0000","last_active":"2023-11-06T03:05:16.494417+0000","last_peered":"2023-11-06T03:05:16.494417+0000","last_clean":"2023-11-06T03:05:16.494417+0000","last_became_active":"2023-11-06T03:05:14.668601+0000","last_became_peered":"2023-11-06T03:05:14.668601+0000","last_unstale":"2023-11-06T03:05:16.494417+0000","last_undegraded":"2023-11-06T03:05:16.494417+0000","last_fullsized":"2023-11-06T03:05:16.494417+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:38.269085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.619344+0000","last_change":"2023-11-06T03:05:14.270823+0000","last_active":"2023-11-06T03:05:15.619344+0000","last_peered":"2023-11-06T03:05:15.619344+0000","last_clean":"2023-11-06T03:05:15.619344+0000","last_became_active":"2023-11-06T03:05:14.270512+0000","last_became_peered":"2023-11-06T03:05:14.270512+0000","last_unstale":"2023-11-06T03:05:15.619344+0000","last_undegraded":"2023-11-06T03:05:15.619344+0000","last_fullsized":"2023-11-06T03:05:15.619344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:12.060123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.608777+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:05:14.608777+0000","last_peered":"2023-11-06T03:05:14.608777+0000","last_clean":"2023-11-06T03:05:14.608777+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:05:14.608777+0000","last_undegraded":"2023-11-06T03:05:14.608777+0000","last_fullsized":"2023-11-06T03:05:14.608777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.172770+0000","last_change":"2023-11-06T03:05:14.675118+0000","last_active":"2023-11-06T03:05:16.172770+0000","last_peered":"2023-11-06T03:05:16.172770+0000","last_clean":"2023-11-06T03:05:16.172770+0000","last_became_active":"2023-11-06T03:05:14.674860+0000","last_became_peered":"2023-11-06T03:05:14.674860+0000","last_unstale":"2023-11-06T03:05:16.172770+0000","last_undegraded":"2023-11-06T03:05:16.172770+0000","last_fullsized":"2023-11-06T03:05:16.172770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:58:14.178485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.626847+0000","last_change":"2023-11-06T03:05:14.251780+0000","last_active":"2023-11-06T03:05:14.626847+0000","last_peered":"2023-11-06T03:05:14.626847+0000","last_clean":"2023-11-06T03:05:14.626847+0000","last_became_active":"2023-11-06T03:05:14.251488+0000","last_became_peered":"2023-11-06T03:05:14.251488+0000","last_unstale":"2023-11-06T03:05:14.626847+0000","last_undegraded":"2023-11-06T03:05:14.626847+0000","last_fullsized":"2023-11-06T03:05:14.626847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:49.183291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.626874+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:05:14.626874+0000","last_peered":"2023-11-06T03:05:14.626874+0000","last_clean":"2023-11-06T03:05:14.626874+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:05:14.626874+0000","last_undegraded":"2023-11-06T03:05:14.626874+0000","last_fullsized":"2023-11-06T03:05:14.626874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.491775+0000","last_change":"2023-11-06T03:05:14.625562+0000","last_active":"2023-11-06T03:05:16.491775+0000","last_peered":"2023-11-06T03:05:16.491775+0000","last_clean":"2023-11-06T03:05:16.491775+0000","last_became_active":"2023-11-06T03:05:14.625280+0000","last_became_peered":"2023-11-06T03:05:14.625280+0000","last_unstale":"2023-11-06T03:05:16.491775+0000","last_undegraded":"2023-11-06T03:05:16.491775+0000","last_fullsized":"2023-11-06T03:05:16.491775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:13:44.982238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.623128+0000","last_change":"2023-11-06T03:05:14.274118+0000","last_active":"2023-11-06T03:05:14.623128+0000","last_peered":"2023-11-06T03:05:14.623128+0000","last_clean":"2023-11-06T03:05:14.623128+0000","last_became_active":"2023-11-06T03:05:14.273783+0000","last_became_peered":"2023-11-06T03:05:14.273783+0000","last_unstale":"2023-11-06T03:05:14.623128+0000","last_undegraded":"2023-11-06T03:05:14.623128+0000","last_fullsized":"2023-11-06T03:05:14.623128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:05:36.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.616026+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:05:14.616026+0000","last_peered":"2023-11-06T03:05:14.616026+0000","last_clean":"2023-11-06T03:05:14.616026+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:05:14.616026+0000","last_undegraded":"2023-11-06T03:05:14.616026+0000","last_fullsized":"2023-11-06T03:05:14.616026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.801787+0000","last_change":"2023-11-06T03:05:14.632097+0000","last_active":"2023-11-06T03:05:15.801787+0000","last_peered":"2023-11-06T03:05:15.801787+0000","last_clean":"2023-11-06T03:05:15.801787+0000","last_became_active":"2023-11-06T03:05:14.631774+0000","last_became_peered":"2023-11-06T03:05:14.631774+0000","last_unstale":"2023-11-06T03:05:15.801787+0000","last_undegraded":"2023-11-06T03:05:15.801787+0000","last_fullsized":"2023-11-06T03:05:15.801787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:11.394296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.614015+0000","last_change":"2023-11-06T03:05:14.260423+0000","last_active":"2023-11-06T03:05:14.614015+0000","last_peered":"2023-11-06T03:05:14.614015+0000","last_clean":"2023-11-06T03:05:14.614015+0000","last_became_active":"2023-11-06T03:05:14.259637+0000","last_became_peered":"2023-11-06T03:05:14.259637+0000","last_unstale":"2023-11-06T03:05:14.614015+0000","last_undegraded":"2023-11-06T03:05:14.614015+0000","last_fullsized":"2023-11-06T03:05:14.614015+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:15.290303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.639817+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:05:14.639817+0000","last_peered":"2023-11-06T03:05:14.639817+0000","last_clean":"2023-11-06T03:05:14.639817+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:05:14.639817+0000","last_undegraded":"2023-11-06T03:05:14.639817+0000","last_fullsized":"2023-11-06T03:05:14.639817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:15.620436+0000","last_change":"2023-11-06T03:05:14.638182+0000","last_active":"2023-11-06T03:05:15.620436+0000","last_peered":"2023-11-06T03:05:15.620436+0000","last_clean":"2023-11-06T03:05:15.620436+0000","last_became_active":"2023-11-06T03:05:14.637391+0000","last_became_peered":"2023-11-06T03:05:14.637391+0000","last_unstale":"2023-11-06T03:05:15.620436+0000","last_undegraded":"2023-11-06T03:05:15.620436+0000","last_fullsized":"2023-11-06T03:05:15.620436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:38.269085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.620489+0000","last_change":"2023-11-06T03:05:14.252956+0000","last_active":"2023-11-06T03:05:14.620489+0000","last_peered":"2023-11-06T03:05:14.620489+0000","last_clean":"2023-11-06T03:05:14.620489+0000","last_became_active":"2023-11-06T03:05:14.252194+0000","last_became_peered":"2023-11-06T03:05:14.252194+0000","last_unstale":"2023-11-06T03:05:14.620489+0000","last_undegraded":"2023-11-06T03:05:14.620489+0000","last_fullsized":"2023-11-06T03:05:14.620489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:46:28.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.615098+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:05:14.615098+0000","last_peered":"2023-11-06T03:05:14.615098+0000","last_clean":"2023-11-06T03:05:14.615098+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:05:14.615098+0000","last_undegraded":"2023-11-06T03:05:14.615098+0000","last_fullsized":"2023-11-06T03:05:14.615098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-06T03:05:16.493196+0000","last_change":"2023-11-06T03:05:14.669020+0000","last_active":"2023-11-06T03:05:16.493196+0000","last_peered":"2023-11-06T03:05:16.493196+0000","last_clean":"2023-11-06T03:05:16.493196+0000","last_became_active":"2023-11-06T03:05:14.668235+0000","last_became_peered":"2023-11-06T03:05:14.668235+0000","last_unstale":"2023-11-06T03:05:16.493196+0000","last_undegraded":"2023-11-06T03:05:16.493196+0000","last_fullsized":"2023-11-06T03:05:16.493196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:14.217547+0000","last_clean_scrub_stamp":"2023-11-06T03:05:14.217547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:11.394296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.616264+0000","last_change":"2023-11-06T03:05:14.248031+0000","last_active":"2023-11-06T03:05:14.616264+0000","last_peered":"2023-11-06T03:05:14.616264+0000","last_clean":"2023-11-06T03:05:14.616264+0000","last_became_active":"2023-11-06T03:05:14.247687+0000","last_became_peered":"2023-11-06T03:05:14.247687+0000","last_unstale":"2023-11-06T03:05:14.616264+0000","last_undegraded":"2023-11-06T03:05:14.616264+0000","last_fullsized":"2023-11-06T03:05:14.616264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:05:13.206875+0000","last_clean_scrub_stamp":"2023-11-06T03:05:13.206875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:10:49.302494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-06T03:05:14.613214+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:05:14.613214+0000","last_peered":"2023-11-06T03:05:14.613214+0000","last_clean":"2023-11-06T03:05:14.613214+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:05:14.613214+0000","last_undegraded":"2023-11-06T03:05:14.613214+0000","last_fullsized":"2023-11-06T03:05:14.613214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":1032192,"data_stored":767216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.873},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.05},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.98999999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.024}]},{"osd":6,"last update":"Mon Nov 6 03:04:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.94899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":421888,"data_stored":174777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.879},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.746}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.82699999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":1.0009999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":430080,"data_stored":175326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.98599999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.1200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.024}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.077}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":430080,"data_stored":175818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.93500000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.016}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.2589999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.91000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":434176,"data_stored":175793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":1.353},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.3109999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.1040000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.22}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.018},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.2}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.749}]},{"osd":5,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.82499999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.84999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":425984,"data_stored":175304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.39700000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.751},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.86499999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:04:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.81699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":764597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.745}]},{"osd":2,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85799999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.90700000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":1.113},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1.0549999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.151},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.2809999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":1.0860000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.355},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:05:22.471 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:05:22.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:05:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.635+0000 7f905745e640 1 Processor -- start 2023-11-06T03:05:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.635+0000 7f905745e640 1 -- start start 2023-11-06T03:05:22.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f905745e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 0x7f9050159f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f905745e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90500c94d0 0x7f90500c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f905745e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90500c9e70 con 0x7f905015cc70 2023-11-06T03:05:22.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f905745e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500ccf90 con 0x7f90500c94d0 2023-11-06T03:05:22.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f905745e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500cd270 con 0x7f9050156da0 2023-11-06T03:05:22.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90551fa640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 0x7f9050159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90500c94d0 0x7f90500c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90551fa640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 0x7f9050159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:34470/0 (socket says 172.21.15.157:34470) 2023-11-06T03:05:22.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90551fa640 1 -- 172.21.15.157:0/1553904700 learned_addr learned my addr 172.21.15.157:0/1553904700 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:22.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90500c94d0 0x7f90500c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:36818/0 (socket says 172.21.15.157:36818) 2023-11-06T03:05:22.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90559fb640 1 --1- 172.21.15.157:0/1553904700 >> v1:172.21.15.195:6789/0 conn(0x7f905015cc70 0x7f905015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:33112/0 (socket says 172.21.15.157:33112) 2023-11-06T03:05:22.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 -- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 msgr2=0x7f9050159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:22.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 --2- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 0x7f9050159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 -- 172.21.15.157:0/1553904700 >> v1:172.21.15.195:6789/0 conn(0x7f905015cc70 legacy=0x7f905015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.639+0000 7f90549f9640 1 -- 172.21.15.157:0/1553904700 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500cd550 con 0x7f90500c94d0 2023-11-06T03:05:22.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f90500c9930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe1b08e5497afb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f9047fff640 1 -- 172.21.15.157:0/1553904700 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400ab5f0 con 0x7f90500c94d0 2023-11-06T03:05:22.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f9047fff640 1 -- 172.21.15.157:0/1553904700 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400ab750 con 0x7f90500c94d0 2023-11-06T03:05:22.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f9047fff640 1 -- 172.21.15.157:0/1553904700 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400abae0 con 0x7f90500c94d0 2023-11-06T03:05:22.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 msgr2=0x7f90500c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f90500c9930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f90400097c0 tx=0x7f904001c3f0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- 172.21.15.157:0/1553904700 shutdown_connections 2023-11-06T03:05:22.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f90500c9930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- 172.21.15.157:0/1553904700 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9050156da0 0x7f9050159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- 172.21.15.157:0/1553904700 >> 172.21.15.157:0/1553904700 conn(0x7f90500cb990 msgr2=0x7f90500cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:22.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- 172.21.15.157:0/1553904700 shutdown_connections 2023-11-06T03:05:22.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- 172.21.15.157:0/1553904700 wait complete. 2023-11-06T03:05:22.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 Processor -- start 2023-11-06T03:05:22.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- start start 2023-11-06T03:05:22.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f905014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f905015cc70 0x7f90501589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905015ba90 con 0x7f9050156da0 2023-11-06T03:05:22.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050157460 con 0x7f90500c94d0 2023-11-06T03:05:22.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f905745e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905015af80 con 0x7f905015cc70 2023-11-06T03:05:22.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39486/0 (socket says 172.21.15.157:39486) 2023-11-06T03:05:22.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 -- 172.21.15.157:0/4134769187 learned_addr learned my addr 172.21.15.157:0/4134769187 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:22.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90551fa640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f905014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90559fb640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f905015cc70 0x7f90501589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 -- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f905015cc70 msgr2=0x7f90501589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f905015cc70 0x7f90501589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 -- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 msgr2=0x7f905014a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f905014a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 -- 172.21.15.157:0/4134769187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009420 con 0x7f9050156da0 2023-11-06T03:05:22.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90551fa640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f905014a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:22.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.643+0000 7f90549f9640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f904001cc90 tx=0x7f904001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400bf610 con 0x7f9050156da0 2023-11-06T03:05:22.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400bf770 con 0x7f9050156da0 2023-11-06T03:05:22.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90501fa070 con 0x7f9050156da0 2023-11-06T03:05:22.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400bfae0 con 0x7f9050156da0 2023-11-06T03:05:22.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905015b4d0 con 0x7f9050156da0 2023-11-06T03:05:22.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9050072750 con 0x7f9050156da0 2023-11-06T03:05:22.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f90400b5400 con 0x7f9050156da0 2023-11-06T03:05:22.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 0x7f9034067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:22.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f90400a8050 con 0x7f9050156da0 2023-11-06T03:05:22.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f90551fa640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 0x7f9034067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:22.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.647+0000 7f90551fa640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 0x7f9034067f20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ba0 tx=0x7f904c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:22.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.651+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f90400eba90 con 0x7f9050156da0 2023-11-06T03:05:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.863+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f90500b4e90 con 0x7f9050156da0 2023-11-06T03:05:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f9045ffb640 1 -- 172.21.15.157:0/4134769187 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f90400ef0b0 con 0x7f9050156da0 2023-11-06T03:05:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:05:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 msgr2=0x7f9034067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 0x7f9034067f20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f904c009ba0 tx=0x7f904c008040 comp rx=0 tx=0).stop 2023-11-06T03:05:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 msgr2=0x7f90501f9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f904001cc90 tx=0x7f904001ac30 comp rx=0 tx=0).stop 2023-11-06T03:05:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 shutdown_connections 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9034065a60 0x7f9034067f20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f905015cc70 0x7f90501589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9050156da0 0x7f90501f9b30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 --2- 172.21.15.157:0/4134769187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90500c94d0 0x7f905014a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 >> 172.21.15.157:0/4134769187 conn(0x7f90500cb990 msgr2=0x7f905006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 shutdown_connections 2023-11-06T03:05:22.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.871+0000 7f905745e640 1 -- 172.21.15.157:0/4134769187 wait complete. 2023-11-06T03:05:22.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:05:22.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 Processor -- start 2023-11-06T03:05:22.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 -- start start 2023-11-06T03:05:23.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f181815e5e0 0x7f18180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f181815ea90 0x7f18180b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181815f580 con 0x7f18181551e0 2023-11-06T03:05:23.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18180704b0 con 0x7f181815ea90 2023-11-06T03:05:23.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181f1c7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181815efc0 con 0x7f181815e5e0 2023-11-06T03:05:23.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181cf63640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f18181551e0 0x7f181806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41018/0 (socket says 172.21.15.157:41018) 2023-11-06T03:05:23.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181cf63640 1 -- 172.21.15.157:0/1767025734 learned_addr learned my addr 172.21.15.157:0/1767025734 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:23.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f181815ea90 0x7f18180b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f1817fff640 1 --2- 172.21.15.157:0/1767025734 >> v2:172.21.15.195:3300/0 conn(0x7f181815e5e0 0x7f18180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 -- 172.21.15.157:0/1767025734 >> v2:172.21.15.195:3300/0 conn(0x7f181815e5e0 msgr2=0x7f18180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 --2- 172.21.15.157:0/1767025734 >> v2:172.21.15.195:3300/0 conn(0x7f181815e5e0 0x7f18180b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f18177fe640 1 -- 172.21.15.157:0/1767025734 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145338554 0 0) 0x7f181815f580 con 0x7f18181551e0 2023-11-06T03:05:23.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 -- 172.21.15.157:0/1767025734 >> v1:172.21.15.157:6789/0 conn(0x7f18181551e0 legacy=0x7f181806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 -- 172.21.15.157:0/1767025734 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181815f2a0 con 0x7f181815ea90 2023-11-06T03:05:23.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:22.999+0000 7f181d764640 1 --2- 172.21.15.157:0/1767025734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f181815ea90 0x7f18180b3950 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1818066880 tx=0x7f181001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da068d75604b7498 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18177fe640 1 -- 172.21.15.157:0/1767025734 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100ac600 con 0x7f181815ea90 2023-11-06T03:05:23.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18177fe640 1 -- 172.21.15.157:0/1767025734 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100a8050 con 0x7f181815ea90 2023-11-06T03:05:23.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18177fe640 1 -- 172.21.15.157:0/1767025734 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100ac980 con 0x7f181815ea90 2023-11-06T03:05:23.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1767025734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f181815ea90 msgr2=0x7f18180b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1767025734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f181815ea90 0x7f18180b3950 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1818066880 tx=0x7f181001c3a0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1767025734 shutdown_connections 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1767025734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f181815ea90 0x7f18180b3950 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1767025734 >> v2:172.21.15.195:3300/0 conn(0x7f181815e5e0 0x7f18180b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1767025734 >> 172.21.15.157:0/1767025734 conn(0x7f1818065930 msgr2=0x7f1818065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1767025734 shutdown_connections 2023-11-06T03:05:23.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1767025734 wait complete. 2023-11-06T03:05:23.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 Processor -- start 2023-11-06T03:05:23.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- start start 2023-11-06T03:05:23.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f181815e5e0 0x7f181814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181815ea90 0x7f181815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181814e4f0 con 0x7f18181551e0 2023-11-06T03:05:23.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1818158430 con 0x7f181815ea90 2023-11-06T03:05:23.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18181578b0 con 0x7f181815e5e0 2023-11-06T03:05:23.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39490/0 (socket says 172.21.15.157:39490) 2023-11-06T03:05:23.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 -- 172.21.15.157:0/1095070335 learned_addr learned my addr 172.21.15.157:0/1095070335 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f1817fff640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f181815e5e0 0x7f181814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 -- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f181815e5e0 msgr2=0x7f181814db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181d764640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181815ea90 0x7f181815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f181815e5e0 0x7f181814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 -- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181815ea90 msgr2=0x7f181815dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181815ea90 0x7f181815dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 -- 172.21.15.157:0/1095070335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810009420 con 0x7f18181551e0 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181cf63640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18080011f0 tx=0x7f180800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080074e0 con 0x7f18181551e0 2023-11-06T03:05:23.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1808007640 con 0x7f18181551e0 2023-11-06T03:05:23.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1818157b90 con 0x7f18181551e0 2023-11-06T03:05:23.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808004a10 con 0x7f18181551e0 2023-11-06T03:05:23.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.003+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1818158150 con 0x7f18181551e0 2023-11-06T03:05:23.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818072750 con 0x7f18181551e0 2023-11-06T03:05:23.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1808004b70 con 0x7f18181551e0 2023-11-06T03:05:23.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f18157fa640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 0x7f1800067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1808082800 con 0x7f18181551e0 2023-11-06T03:05:23.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f1817fff640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 0x7f1800067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.007+0000 7f1817fff640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 0x7f1800067f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18181f6af0 tx=0x7f180400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.011+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f18080500f0 con 0x7f18181551e0 2023-11-06T03:05:23.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.195+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1818071730 con 0x7f18181551e0 2023-11-06T03:05:23.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.195+0000 7f18157fa640 1 -- 172.21.15.157:0/1095070335 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1808018020 con 0x7f18181551e0 2023-11-06T03:05:23.197 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:05:23.198 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:05:16.614800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:05:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 13 2023-11-06T03:05:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 msgr2=0x7f1800067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 0x7f1800067f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18181f6af0 tx=0x7f180400b040 comp rx=0 tx=0).stop 2023-11-06T03:05:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 msgr2=0x7f18181572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f18080011f0 tx=0x7f180800c9b0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 shutdown_connections 2023-11-06T03:05:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1800065a60 0x7f1800067f20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f181815ea90 0x7f181815dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f181815e5e0 0x7f181814db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 --2- 172.21.15.157:0/1095070335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18181551e0 0x7f18181572d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 >> 172.21.15.157:0/1095070335 conn(0x7f1818065930 msgr2=0x7f181815ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 shutdown_connections 2023-11-06T03:05:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.199+0000 7f181f1c7640 1 -- 172.21.15.157:0/1095070335 wait complete. 2023-11-06T03:05:23.618 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:05:14.602320+0000', 'modified': '2023-11-06T03:05:16.614800+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5331': {'gid': 5331, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/4193594648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6833', 'nonce': 4193594648}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 4193594648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:05:23.619 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:05:23.619 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:05:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 Processor -- start 2023-11-06T03:05:23.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 -- start start 2023-11-06T03:05:23.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ea415e5e0 0x7f9ea40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9ea415ea60 0x7f9ea40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea415f580 con 0x7f9ea415e5e0 2023-11-06T03:05:23.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea415efc0 con 0x7f9ea415ea60 2023-11-06T03:05:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9eab32b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415f2a0 con 0x7f9ea41551e0 2023-11-06T03:05:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ea415e5e0 0x7f9ea40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ea415e5e0 0x7f9ea40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39492/0 (socket says 172.21.15.157:39492) 2023-11-06T03:05:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3192444356 learned_addr learned my addr 172.21.15.157:0/3192444356 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/3192444356 >> v2:172.21.15.195:3300/0 conn(0x7f9ea415ea60 0x7f9ea40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3192444356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 msgr2=0x7f9ea406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3192444356 >> v2:172.21.15.195:3300/0 conn(0x7f9ea415ea60 msgr2=0x7f9ea40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3192444356 >> v2:172.21.15.195:3300/0 conn(0x7f9ea415ea60 0x7f9ea40b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.751+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3192444356 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea406f5a0 con 0x7f9ea415e5e0 2023-11-06T03:05:23.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea40b3410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40cc5d0 tx=0x7f9e9401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfed71ff1bfbd67b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9e9bfff640 1 -- 172.21.15.157:0/3192444356 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940ab620 con 0x7f9ea415e5e0 2023-11-06T03:05:23.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9e9bfff640 1 -- 172.21.15.157:0/3192444356 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e940ab780 con 0x7f9ea415e5e0 2023-11-06T03:05:23.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9e9bfff640 1 -- 172.21.15.157:0/3192444356 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940abb10 con 0x7f9ea415e5e0 2023-11-06T03:05:23.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- 172.21.15.157:0/3192444356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 msgr2=0x7f9ea40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea40b3410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40cc5d0 tx=0x7f9e9401c440 comp rx=0 tx=0).stop 2023-11-06T03:05:23.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- 172.21.15.157:0/3192444356 shutdown_connections 2023-11-06T03:05:23.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 --2- 172.21.15.157:0/3192444356 >> v2:172.21.15.195:3300/0 conn(0x7f9ea415ea60 0x7f9ea40b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea40b3410 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 --2- 172.21.15.157:0/3192444356 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- 172.21.15.157:0/3192444356 >> 172.21.15.157:0/3192444356 conn(0x7f9ea4065930 msgr2=0x7f9ea4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- 172.21.15.157:0/3192444356 shutdown_connections 2023-11-06T03:05:23.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- 172.21.15.157:0/3192444356 wait complete. 2023-11-06T03:05:23.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 Processor -- start 2023-11-06T03:05:23.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.755+0000 7f9eab32b640 1 -- start start 2023-11-06T03:05:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea415ea60 0x7f9ea41447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4154470 con 0x7f9ea415e5e0 2023-11-06T03:05:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4153930 con 0x7f9ea415ea60 2023-11-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4153bd0 con 0x7f9ea41551e0 2023-11-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39494/0 (socket says 172.21.15.157:39494) 2023-11-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 -- 172.21.15.157:0/4017975250 learned_addr learned my addr 172.21.15.157:0/4017975250 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea415ea60 0x7f9ea41447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 -- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 msgr2=0x7f9ea414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 -- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea415ea60 msgr2=0x7f9ea41447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea415ea60 0x7f9ea41447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 -- 172.21.15.157:0/4017975250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940094d0 con 0x7f9ea415e5e0 2023-11-06T03:05:23.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9e9401aaf0 tx=0x7f9e9401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 -- 172.21.15.157:0/4017975250 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940be3f0 con 0x7f9ea415e5e0 2023-11-06T03:05:23.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 -- 172.21.15.157:0/4017975250 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e940be550 con 0x7f9ea415e5e0 2023-11-06T03:05:23.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea4153eb0 con 0x7f9ea415e5e0 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 -- 172.21.15.157:0/4017975250 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940be900 con 0x7f9ea415e5e0 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea4146130 con 0x7f9ea415e5e0 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 -- 172.21.15.157:0/4017975250 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9e940c7440 con 0x7f9ea415e5e0 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 0x7f9e88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9e99ffb640 1 -- 172.21.15.157:0/4017975250 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e9411da60 con 0x7f9ea415e5e0 2023-11-06T03:05:23.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.759+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 0x7f9e88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 Processor -- start 2023-11-06T03:05:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- start start 2023-11-06T03:05:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 0x7f9e88067f20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40cc240 tx=0x7f9e9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f82ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f38a90 0x55f143f38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143e00fc0 con 0x55f143f83000 2023-11-06T03:05:23.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143d8f1a0 con 0x55f143f826e0 2023-11-06T03:05:23.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f143d8c770 con 0x55f143f38a90 2023-11-06T03:05:23.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39500/0 (socket says 172.21.15.157:39500) 2023-11-06T03:05:23.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3581410989 learned_addr learned my addr 172.21.15.157:0/3581410989 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f38a90 0x55f143f38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f82ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f38a90 msgr2=0x55f143f38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f38a90 0x55f143f38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 msgr2=0x55f143f82ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f82ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 -- 172.21.15.157:0/3581410989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f143a84890 con 0x55f143f83000 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9e940f6be0 tx=0x7f9e940f5050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3acaff9f16181 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9e8effd640 1 -- 172.21.15.157:0/3581410989 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940f2f60 con 0x55f143f83000 2023-11-06T03:05:23.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9e8effd640 1 -- 172.21.15.157:0/3581410989 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e940f1030 con 0x55f143f83000 2023-11-06T03:05:23.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9e8effd640 1 -- 172.21.15.157:0/3581410989 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940f7110 con 0x55f143f83000 2023-11-06T03:05:23.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- 172.21.15.157:0/3581410989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 msgr2=0x55f143f38550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9e940f6be0 tx=0x7f9e940f5050 comp rx=0 tx=0).stop 2023-11-06T03:05:23.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- 172.21.15.157:0/3581410989 shutdown_connections 2023-11-06T03:05:23.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f38a90 0x55f143f38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f83000 0x55f143f38550 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- 172.21.15.157:0/3581410989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f82ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- 172.21.15.157:0/3581410989 >> 172.21.15.157:0/3581410989 conn(0x55f143f3ebf0 msgr2=0x55f143f36e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- 172.21.15.157:0/3581410989 shutdown_connections 2023-11-06T03:05:23.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- 172.21.15.157:0/3581410989 wait complete. 2023-11-06T03:05:23.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- start start 2023-11-06T03:05:23.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 Processor -- start 2023-11-06T03:05:23.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 0x55f143f83920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143d88060 con 0x55f143f38a90 2023-11-06T03:05:23.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143d27290 con 0x55f143f826e0 2023-11-06T03:05:23.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f143d456f0 con 0x55f143f83000 2023-11-06T03:05:23.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9eae453000 10 client.0 _ll_register_callbacks cb 0x55f143f241d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:05:23.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea90c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea90c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39504/0 (socket says 172.21.15.157:39504) 2023-11-06T03:05:23.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.763+0000 7f9ea90c7640 1 -- 172.21.15.157:0/263794694 learned_addr learned my addr 172.21.15.157:0/263794694 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 0x55f143f83920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 -- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 msgr2=0x55f143f83920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 0x55f143f83920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 -- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 msgr2=0x55f143f833e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f833e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 -- 172.21.15.157:0/263794694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea4146130 con 0x55f143f38a90 2023-11-06T03:05:23.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 0x55f143f83920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:05:23.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c014710 tx=0x7f9e9c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b5db0037a65abe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:05:23.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c015960 con 0x55f143f38a90 2023-11-06T03:05:23.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e9c015ac0 con 0x55f143f38a90 2023-11-06T03:05:23.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f143d869b0 con 0x55f143f38a90 2023-11-06T03:05:23.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e9c015cd0 con 0x55f143f38a90 2023-11-06T03:05:23.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 10 client.5532 fetch_fsmap learned FSMap version 13 2023-11-06T03:05:23.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55f143d4d1e0 con 0x55f143f38a90 2023-11-06T03:05:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f143d869b0 con 0x55f143f38a90 2023-11-06T03:05:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 13) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9e9c021420 con 0x55f143f38a90 2023-11-06T03:05:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 10 client.5532 fetch_fsmap finished waiting for FSMap version 13 2023-11-06T03:05:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 10 client.5532 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:05:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 20 client.5532 populate_metadata read hostname 'smithi157' 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f89230 0x55f143f89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 4 client.5532 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f143db74d0 con 0x55f143f89230 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f89230 0x55f143f89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f89230 0x55f143f89610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1085498cfb6368b server_cookie=f36f5f42dbf49e35 in_seq=0 out_seq=0 2023-11-06T03:05:23.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 10 client.5532 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:05:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 1 -- 172.21.15.157:0/263794694 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f143db74d0 con 0x55f143f89230 2023-11-06T03:05:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 10 client.5532 handle_command_reply: tid=0 2023-11-06T03:05:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 client.5532 shutdown 2023-11-06T03:05:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 msgr2=0x55f143f3d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c014710 tx=0x7f9e9c004740 comp rx=0 tx=0).stop 2023-11-06T03:05:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 shutdown_connections 2023-11-06T03:05:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f89230 0x55f143f89610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f83000 0x55f143f83920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9e83fff640 0 client.5532 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:05:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 --2- 172.21.15.157:0/263794694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143f3d8c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 >> 172.21.15.157:0/263794694 conn(0x55f143f3ebf0 msgr2=0x55f143f37530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 shutdown_connections 2023-11-06T03:05:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 1 -- 172.21.15.157:0/263794694 wait complete. 2023-11-06T03:05:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.767+0000 7f9eae453000 20 client.5532 trim_cache size 0 max 16384 2023-11-06T03:05:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 Processor -- start 2023-11-06T03:05:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- start start 2023-11-06T03:05:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 0x55f143f3cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143a45940 con 0x55f143f826e0 2023-11-06T03:05:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143f82390 con 0x55f143f38a90 2023-11-06T03:05:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f143d2d4f0 con 0x55f143f3ebf0 2023-11-06T03:05:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39520/0 (socket says 172.21.15.157:39520) 2023-11-06T03:05:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 -- 172.21.15.157:0/375242034 learned_addr learned my addr 172.21.15.157:0/375242034 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 0x55f143f3cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 -- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 msgr2=0x55f143f3c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f3c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 -- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 msgr2=0x55f143f3cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 0x55f143f3cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 -- 172.21.15.157:0/375242034 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f143d33330 con 0x55f143f826e0 2023-11-06T03:05:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 0x55f143f3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009f20 tx=0x7f9ea00077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1cc62977bb55b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9e8f7fe640 1 -- 172.21.15.157:0/375242034 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0017640 con 0x55f143f826e0 2023-11-06T03:05:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9e8f7fe640 1 -- 172.21.15.157:0/375242034 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000f040 con 0x55f143f826e0 2023-11-06T03:05:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9e8f7fe640 1 -- 172.21.15.157:0/375242034 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00179e0 con 0x55f143f826e0 2023-11-06T03:05:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- 172.21.15.157:0/375242034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 msgr2=0x55f143f92320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009f20 tx=0x7f9ea00077c0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- 172.21.15.157:0/375242034 shutdown_connections 2023-11-06T03:05:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f826e0 0x55f143f92320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f38a90 0x55f143f3cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 --2- 172.21.15.157:0/375242034 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f3c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- 172.21.15.157:0/375242034 >> 172.21.15.157:0/375242034 conn(0x55f143f3ded0 msgr2=0x55f143f3c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- 172.21.15.157:0/375242034 shutdown_connections 2023-11-06T03:05:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- 172.21.15.157:0/375242034 wait complete. 2023-11-06T03:05:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 -- start start 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.783+0000 7f9eae453000 1 Processor -- start 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f94f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143d8c060 con 0x55f143f38a90 2023-11-06T03:05:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f143d85460 con 0x55f143f826e0 2023-11-06T03:05:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f143f83840 con 0x55f143f3ebf0 2023-11-06T03:05:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 10 client.0 _ll_register_callbacks cb 0x55f143f049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:05:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39530/0 (socket says 172.21.15.157:39530) 2023-11-06T03:05:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 -- 172.21.15.157:0/3200779218 learned_addr learned my addr 172.21.15.157:0/3200779218 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f94f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea98c8640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 -- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 msgr2=0x55f143f94f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f94f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 -- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 msgr2=0x55f143f954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 -- 172.21.15.157:0/3200779218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea0009b50 con 0x55f143f38a90 2023-11-06T03:05:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f94f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea90c7640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c010 tx=0x7f9e9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39defff7577262b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c015760 con 0x55f143f38a90 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e9c0158c0 con 0x55f143f38a90 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f143d57b90 con 0x55f143f38a90 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e9c015a80 con 0x55f143f38a90 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 10 client.5538 fetch_fsmap learned FSMap version 13 2023-11-06T03:05:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55f143a84cc0 con 0x55f143f38a90 2023-11-06T03:05:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f143d57b90 con 0x55f143f38a90 2023-11-06T03:05:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 13) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9e9c039030 con 0x55f143f38a90 2023-11-06T03:05:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 10 client.5538 fetch_fsmap finished waiting for FSMap version 13 2023-11-06T03:05:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 10 client.5538 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:05:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 20 client.5538 populate_metadata read hostname 'smithi157' 2023-11-06T03:05:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f83000 0x55f143f959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 4 client.5538 mds_command: new command op to 5331 tid=0[{"prefix": "status"}] 2023-11-06T03:05:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f143d810e0 con 0x55f143f83000 2023-11-06T03:05:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f83000 0x55f143f959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 10 client.5538 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:05:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9ea88c6640 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f83000 0x55f143f959e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd9befcc3ae11f9 server_cookie=bc3d6e8dd9576d53 in_seq=0 out_seq=0 2023-11-06T03:05:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 1 -- 172.21.15.157:0/3200779218 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f143d810e0 con 0x55f143f83000 2023-11-06T03:05:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 10 client.5538 handle_command_reply: tid=0 2023-11-06T03:05:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 client.5538 shutdown 2023-11-06T03:05:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 msgr2=0x55f143fdb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c010 tx=0x7f9e9c014000 comp rx=0 tx=0).stop 2023-11-06T03:05:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 shutdown_connections 2023-11-06T03:05:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55f143f83000 0x55f143f959e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f143f826e0 0x55f143f954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f143f3ebf0 0x55f143f94f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9e83fff640 0 client.5538 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 --2- 172.21.15.157:0/3200779218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f143f38a90 0x55f143fdb890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 >> 172.21.15.157:0/3200779218 conn(0x55f143f3ded0 msgr2=0x55f143f87e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 shutdown_connections 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 1 -- 172.21.15.157:0/3200779218 wait complete. 2023-11-06T03:05:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eae453000 20 client.5538 trim_cache size 0 max 16384 2023-11-06T03:05:23.813 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:05:23.813 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:05:23.813 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:05:23.813 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5331, 2023-11-06T03:05:23.814 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:05:23.814 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:05:23.814 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:05:23.814 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 9.1857575669999996, 2023-11-06T03:05:23.814 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 13, 2023-11-06T03:05:23.815 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 37, 2023-11-06T03:05:23.815 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 37, 2023-11-06T03:05:23.815 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 16.595890937 2023-11-06T03:05:23.815 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:05:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 msgr2=0x7f9e88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 0x7f9e88067f20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40cc240 tx=0x7f9e9c008040 comp rx=0 tx=0).stop 2023-11-06T03:05:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 msgr2=0x7f9ea41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9e9401aaf0 tx=0x7f9e9401ac80 comp rx=0 tx=0).stop 2023-11-06T03:05:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 shutdown_connections 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e88065a60 0x7f9e88067f20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ea415ea60 0x7f9ea41447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ea415e5e0 0x7f9ea41533f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 --2- 172.21.15.157:0/4017975250 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ea41551e0 0x7f9ea414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 >> 172.21.15.157:0/4017975250 conn(0x7f9ea4065930 msgr2=0x7f9ea40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 shutdown_connections 2023-11-06T03:05:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.787+0000 7f9eab32b640 1 -- 172.21.15.157:0/4017975250 wait complete. 2023-11-06T03:05:23.819 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:05:23.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 Processor -- start 2023-11-06T03:05:23.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 -- start start 2023-11-06T03:05:23.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c15ce80 0x7fb20c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb20c0d00b0 0x7fb20c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c0d2920 con 0x7fb20c15caa0 2023-11-06T03:05:23.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c0d0520 con 0x7fb20c0d00b0 2023-11-06T03:05:23.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.923+0000 7fb21151f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0d18a0 con 0x7fb20c15ce80 2023-11-06T03:05:23.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb20c0d00b0 0x7fb20c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb20c15caa0 0x7fb20c0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41024/0 (socket says 172.21.15.157:41024) 2023-11-06T03:05:23.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20affd640 1 -- 172.21.15.157:0/4136461042 learned_addr learned my addr 172.21.15.157:0/4136461042 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:05:23.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20a7fc640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c15ce80 0x7fb20c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 -- 172.21.15.157:0/4136461042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c15ce80 msgr2=0x7fb20c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:23.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c15ce80 0x7fb20c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 -- 172.21.15.157:0/4136461042 >> v1:172.21.15.157:6789/0 conn(0x7fb20c15caa0 legacy=0x7fb20c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb209ffb640 1 -- 172.21.15.157:0/4136461042 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271277691 0 0) 0x7fb20c0d2920 con 0x7fb20c15caa0 2023-11-06T03:05:23.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 -- 172.21.15.157:0/4136461042 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0d1b60 con 0x7fb20c0d00b0 2023-11-06T03:05:23.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb20b7fe640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c0d00b0 0x7fb20c0b5d50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb20401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b83e7a2f3b2741 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb209ffb640 1 -- 172.21.15.157:0/4136461042 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040a59e0 con 0x7fb20c0d00b0 2023-11-06T03:05:23.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb209ffb640 1 -- 172.21.15.157:0/4136461042 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2040a5b40 con 0x7fb20c0d00b0 2023-11-06T03:05:23.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb209ffb640 1 -- 172.21.15.157:0/4136461042 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040ad440 con 0x7fb20c0d00b0 2023-11-06T03:05:23.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb21151f640 1 -- 172.21.15.157:0/4136461042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c0d00b0 msgr2=0x7fb20c0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.927+0000 7fb21151f640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c0d00b0 0x7fb20c0b5d50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb20401c920 comp rx=0 tx=0).stop 2023-11-06T03:05:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/4136461042 shutdown_connections 2023-11-06T03:05:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c0d00b0 0x7fb20c0b5d50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 --2- 172.21.15.157:0/4136461042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c15ce80 0x7fb20c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/4136461042 >> 172.21.15.157:0/4136461042 conn(0x7fb20c070790 msgr2=0x7fb20c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:23.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/4136461042 shutdown_connections 2023-11-06T03:05:23.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/4136461042 wait complete. 2023-11-06T03:05:23.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 Processor -- start 2023-11-06T03:05:23.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- start start 2023-11-06T03:05:23.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 0x7fb20c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb20c15caa0 0x7fb20c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c153fc0 con 0x7fb20c15ce80 2023-11-06T03:05:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c154490 con 0x7fb20c15caa0 2023-11-06T03:05:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c1566c0 con 0x7fb20c0d00b0 2023-11-06T03:05:23.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39552/0 (socket says 172.21.15.157:39552) 2023-11-06T03:05:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 -- 172.21.15.157:0/2883577255 learned_addr learned my addr 172.21.15.157:0/2883577255 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20affd640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 0x7fb20c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20a7fc640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb20c15caa0 0x7fb20c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 -- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 msgr2=0x7fb20c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 0x7fb20c15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 -- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb20c15caa0 msgr2=0x7fb20c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb20c15caa0 0x7fb20c150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 -- 172.21.15.157:0/2883577255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009a70 con 0x7fb20c15ce80 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20affd640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 0x7fb20c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:23.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb20b7fe640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb2040196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040ade40 con 0x7fb20c15ce80 2023-11-06T03:05:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204009450 con 0x7fb20c15ce80 2023-11-06T03:05:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c15b880 con 0x7fb20c15ce80 2023-11-06T03:05:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040b9690 con 0x7fb20c15ce80 2023-11-06T03:05:23.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c0b1a40 con 0x7fb20c15ce80 2023-11-06T03:05:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.931+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c155f10 con 0x7fb20c15ce80 2023-11-06T03:05:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.935+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2040b97f0 con 0x7fb20c15ce80 2023-11-06T03:05:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.935+0000 7fb1f37fe640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 0x7fb1f8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.935+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb20411ea70 con 0x7fb20c15ce80 2023-11-06T03:05:23.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.935+0000 7fb20affd640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 0x7fb1f8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:23.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.935+0000 7fb20affd640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 0x7fb1f8067f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc000f70 tx=0x7fb1fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:23.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:23.939+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb20c155f10 con 0x7fb20c15ce80 2023-11-06T03:05:24.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.115+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb20c155f10 con 0x7fb20c15ce80 2023-11-06T03:05:24.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.115+0000 7fb1f37fe640 1 -- 172.21.15.157:0/2883577255 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb20c155f10 con 0x7fb20c15ce80 2023-11-06T03:05:24.119 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:05:24.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 msgr2=0x7fb1f8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 0x7fb1f8067f50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc000f70 tx=0x7fb1fc009040 comp rx=0 tx=0).stop 2023-11-06T03:05:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 msgr2=0x7fb20c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb204009d90 tx=0x7fb2040196c0 comp rx=0 tx=0).stop 2023-11-06T03:05:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 shutdown_connections 2023-11-06T03:05:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb1f8065a90 0x7fb1f8067f50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb20c15ce80 0x7fb20c150ab0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb20c15caa0 0x7fb20c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 --2- 172.21.15.157:0/2883577255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb20c0d00b0 0x7fb20c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 >> 172.21.15.157:0/2883577255 conn(0x7fb20c070790 msgr2=0x7fb20c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 shutdown_connections 2023-11-06T03:05:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:24.119+0000 7fb21151f640 1 -- 172.21.15.157:0/2883577255 wait complete. 2023-11-06T03:05:24.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:05:24.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:05:24.136 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:05:24.136 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:05:24.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:24.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:05:24.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:05:24.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:05:24.138 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:05:24.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.139 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:05:24.145 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:05:24.145 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:05:24.145 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:05:24.145 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.145 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:05:24.146 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.146 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:05:24.146 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.146 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:05:24.146 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:05:24.147 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.147 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:05:24.147 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:05:24.147 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.147 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42415sec preferred_lft 42415sec 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.148 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:05:24.149 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.149 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:05:24.149 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.149 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:05:24.149 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:05:24.150 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:05:24.150 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.150 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:05:24.150 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.150 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:05:24.151 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:05:24.152 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:05:24.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.152 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:05:24.195 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:05:24.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.197 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:05:24.246 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:05:24.246 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:05:24.246 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:05:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:05:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:05:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:05:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:05:24.247 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:05:24.248 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:05:24.248 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:05:24.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.249 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:05:24.249 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:05:24.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:05:24.250 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:05:24.250 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:05:24.250 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:05:24.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.319 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:05:24.364 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:05:24.364 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:05:24.364 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:05:24.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.365 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:05:24.419 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:05:24.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:05:24.420 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:24.467 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:05:24.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:05:24.468 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:24.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:05:24.573 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:05:24.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:05:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:05:24.632 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:05:24.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.634 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:05:24.680 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:05:24.680 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:05:24.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:05:24.739 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:05:24.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:05:24.793 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:05:24.794 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:05:24.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:24.795 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:05:24.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:05:24.799+0000 7f89b8f99580 -1 init, newargv = 0x55895e0a4ec0 newargc=15 2023-11-06T03:05:24.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[48440]: starting ceph client 2023-11-06T03:05:24.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[48440]: starting fuse 2023-11-06T03:05:25.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:05:25.926 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:05:25.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:05:25.979 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:05:25.980 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:05:25.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:25.981 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:05:26.032 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:05:26.032 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:05:26.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:05:26.033 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:05:26.033 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:05:26.034 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:05:26.034 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:05:26.034 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:26.035 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:05:26.035 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:05:26.036 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:05:26.036 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:05:26.036 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:26.037 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:05:26.037 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:05:26.037 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:05:26.038 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:05:26.038 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:26.038 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:05:26.039 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:05:26.039 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:05:26.040 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:05:26.040 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:05:26.040 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:05:26.041 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:05:26.041 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:05:26.041 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:26.042 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:05:26.042 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:05:26.155 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.48440.asok 2023-11-06T03:05:26.338 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48440.asok 2023-11-06T03:05:26.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:26.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48440.asok status 2023-11-06T03:05:26.496 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:05:26.496 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:05:26.496 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:05:26.496 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:05:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:05:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:05:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:05:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "48440", 2023-11-06T03:05:26.497 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:05:26.498 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:26.498 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:05:26.498 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:05:26.498 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5553, 2023-11-06T03:05:26.498 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:05:26.499 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:05:26.499 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:05:26.499 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 5553 2023-11-06T03:05:26.499 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:26.499 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:05:26.500 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:05:26.500 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:05:26.500 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3354323506 2023-11-06T03:05:26.500 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:05:26.500 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:26.501 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:05:26.501 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:05:26.501 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:05:26.501 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3354323506 2023-11-06T03:05:26.501 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:05:26.502 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.5553 192.168.0.1:0/3354323506", 2023-11-06T03:05:26.502 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3354323506", 2023-11-06T03:05:26.502 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:05:26.502 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 13, 2023-11-06T03:05:26.502 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 37, 2023-11-06T03:05:26.503 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:05:26.503 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:05:26.503 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:05:26.503 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:05:26.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:26.511 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:05:26.512 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:26.518 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:05:26.518 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:26.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:26.519 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:05:26.573 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-06T03:05:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.743+0000 7fc0c8a78640 1 Processor -- start 2023-11-06T03:05:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.743+0000 7fc0c8a78640 1 -- start start 2023-11-06T03:05:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0c41593d0 0x7fc0c40ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 0x7fc0c40cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 0x7fc0c40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c40cbad0 con 0x7fc0c41593d0 2023-11-06T03:05:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c41597b0 con 0x7fc0c40cb650 2023-11-06T03:05:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b2b20 con 0x7fc0c406f290 2023-11-06T03:05:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0c41593d0 0x7fc0c40ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0c41593d0 0x7fc0c40ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39562/0 (socket says 172.21.15.157:39562) 2023-11-06T03:05:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 -- 172.21.15.157:0/1248720701 learned_addr learned my addr 172.21.15.157:0/1248720701 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0b9d9b640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 0x7fc0c40cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c2d9d640 1 --2- 172.21.15.157:0/1248720701 >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 0x7fc0c40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 -- 172.21.15.157:0/1248720701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 msgr2=0x7fc0c40cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 0x7fc0c40cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 -- 172.21.15.157:0/1248720701 >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 msgr2=0x7fc0c40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:05:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 --2- 172.21.15.157:0/1248720701 >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 0x7fc0c40b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 -- 172.21.15.157:0/1248720701 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40b2e00 con 0x7fc0c41593d0 2023-11-06T03:05:26.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0b9d9b640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 0x7fc0c40cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:26.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c2d9d640 1 --2- 172.21.15.157:0/1248720701 >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 0x7fc0c40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:26.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c259c640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40ca830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc0c415ce50 tx=0x7fc0b401d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d102a99ac792beb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:26.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c1d9b640 1 -- 172.21.15.157:0/1248720701 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40a68c0 con 0x7fc0c41593d0 2023-11-06T03:05:26.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c1d9b640 1 -- 172.21.15.157:0/1248720701 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b40a6a20 con 0x7fc0c41593d0 2023-11-06T03:05:26.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c1d9b640 1 -- 172.21.15.157:0/1248720701 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40a6db0 con 0x7fc0c41593d0 2023-11-06T03:05:26.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- 172.21.15.157:0/1248720701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 msgr2=0x7fc0c40ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:26.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40ca830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc0c415ce50 tx=0x7fc0b401d8d0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- 172.21.15.157:0/1248720701 shutdown_connections 2023-11-06T03:05:26.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/1248720701 >> v2:172.21.15.195:3300/0 conn(0x7fc0c40cb650 0x7fc0c40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c406f290 0x7fc0c40cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/1248720701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40ca830 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- 172.21.15.157:0/1248720701 >> 172.21.15.157:0/1248720701 conn(0x7fc0c40b43b0 msgr2=0x7fc0c4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- 172.21.15.157:0/1248720701 shutdown_connections 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.747+0000 7fc0c8a78640 1 -- 172.21.15.157:0/1248720701 wait complete. 2023-11-06T03:05:26.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 Processor -- start 2023-11-06T03:05:26.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- start start 2023-11-06T03:05:26.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 0x7fc0c4141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c40cb650 0x7fc0c40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b1de0 con 0x7fc0c41593d0 2023-11-06T03:05:26.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b11d0 con 0x7fc0c406f290 2023-11-06T03:05:26.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40b14b0 con 0x7fc0c40cb650 2023-11-06T03:05:26.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39574/0 (socket says 172.21.15.157:39574) 2023-11-06T03:05:26.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 -- 172.21.15.157:0/2092838682 learned_addr learned my addr 172.21.15.157:0/2092838682 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0b9d9b640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c40cb650 0x7fc0c40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 -- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c40cb650 msgr2=0x7fc0c40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c259c640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 0x7fc0c4141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c40cb650 0x7fc0c40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 -- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 msgr2=0x7fc0c4141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:26.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 0x7fc0c4141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:26.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 -- 172.21.15.157:0/2092838682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b400a670 con 0x7fc0c41593d0 2023-11-06T03:05:26.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c259c640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 0x7fc0c4141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:05:26.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c2d9d640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00cef0 tx=0x7fc0bc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:26.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00d840 con 0x7fc0c41593d0 2023-11-06T03:05:26.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00d9c0 con 0x7fc0c41593d0 2023-11-06T03:05:26.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c40b1790 con 0x7fc0c41593d0 2023-11-06T03:05:26.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00dd70 con 0x7fc0c41593d0 2023-11-06T03:05:26.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c40b0ec0 con 0x7fc0c41593d0 2023-11-06T03:05:26.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.751+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c40b1f50 con 0x7fc0c41593d0 2023-11-06T03:05:26.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.755+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc0bc01e3b0 con 0x7fc0c41593d0 2023-11-06T03:05:26.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.755+0000 7fc0bb7fe640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 0x7fc0a4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:05:26.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.755+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc0bc083d70 con 0x7fc0c41593d0 2023-11-06T03:05:26.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.755+0000 7fc0c259c640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 0x7fc0a4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:05:26.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.755+0000 7fc0c259c640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 0x7fc0a4067f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b0000 tx=0x7fc0b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:05:26.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.759+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc0bc051690 con 0x7fc0c41593d0 2023-11-06T03:05:26.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:26.947+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0c404e050 con 0x7fc0c41593d0 2023-11-06T03:05:27.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.227+0000 7fc0bb7fe640 1 -- 172.21.15.157:0/2092838682 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v14) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc0bc01a0d0 con 0x7fc0c41593d0 2023-11-06T03:05:27.232 INFO:teuthology.orchestra.run.smithi157.stderr:enabled new snapshots 2023-11-06T03:05:27.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 msgr2=0x7fc0a4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:27.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 0x7fc0a4067f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b0000 tx=0x7fc0b4008040 comp rx=0 tx=0).stop 2023-11-06T03:05:27.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 msgr2=0x7fc0c40b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:05:27.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00cef0 tx=0x7fc0bc00a710 comp rx=0 tx=0).stop 2023-11-06T03:05:27.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 shutdown_connections 2023-11-06T03:05:27.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0a4065a90 0x7fc0a4067f50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:27.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c41593d0 0x7fc0c40b0980 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0c40cb650 0x7fc0c40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 --2- 172.21.15.157:0/2092838682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c406f290 0x7fc0c4141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:05:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 >> 172.21.15.157:0/2092838682 conn(0x7fc0c40b43b0 msgr2=0x7fc0c40ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:05:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 shutdown_connections 2023-11-06T03:05:27.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:05:27.235+0000 7fc0c8a78640 1 -- 172.21.15.157:0/2092838682 wait complete. 2023-11-06T03:05:27.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:27.257 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2023-11-06T03:05:27.270 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:05:27.270 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:05:27.316 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:05:27.317 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:05:27.317 DEBUG:teuthology.orchestra.run.smithi157:> import uuid 2023-11-06T03:05:27.318 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:27.318 DEBUG:teuthology.orchestra.run.smithi157:> n = 2000 2023-11-06T03:05:27.318 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = False 2023-11-06T03:05:27.319 DEBUG:teuthology.orchestra.run.smithi157:> if 3 > 0: 2023-11-06T03:05:27.319 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = True 2023-11-06T03:05:27.319 DEBUG:teuthology.orchestra.run.smithi157:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2023-11-06T03:05:27.320 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:27.320 DEBUG:teuthology.orchestra.run.smithi157:> dpath = os.path.dirname(path) 2023-11-06T03:05:27.320 DEBUG:teuthology.orchestra.run.smithi157:> fnameprefix = os.path.basename(path) 2023-11-06T03:05:27.321 DEBUG:teuthology.orchestra.run.smithi157:> os.makedirs(dpath, exist_ok=True) 2023-11-06T03:05:27.321 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:27.322 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:05:27.322 DEBUG:teuthology.orchestra.run.smithi157:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-11-06T03:05:27.322 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:05:27.323 DEBUG:teuthology.orchestra.run.smithi157:> for i in range(n): 2023-11-06T03:05:27.323 DEBUG:teuthology.orchestra.run.smithi157:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-11-06T03:05:27.324 DEBUG:teuthology.orchestra.run.smithi157:> with open(fpath, '"'"'w'"'"') as f: 2023-11-06T03:05:27.324 DEBUG:teuthology.orchestra.run.smithi157:> f.write(f"{i}") 2023-11-06T03:05:27.325 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:05:27.325 DEBUG:teuthology.orchestra.run.smithi157:> f.flush() 2023-11-06T03:05:27.325 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(f.fileno()) 2023-11-06T03:05:27.326 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:05:27.326 DEBUG:teuthology.orchestra.run.smithi157:> os.unlink(fpath) 2023-11-06T03:05:27.326 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:05:27.327 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:05:27.327 DEBUG:teuthology.orchestra.run.smithi157:> if create_hard_links: 2023-11-06T03:05:27.327 DEBUG:teuthology.orchestra.run.smithi157:> for j in range(3): 2023-11-06T03:05:27.328 DEBUG:teuthology.orchestra.run.smithi157:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-11-06T03:05:27.328 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:05:27.328 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:05:27.329 DEBUG:teuthology.orchestra.run.smithi157:> finally: 2023-11-06T03:05:27.329 DEBUG:teuthology.orchestra.run.smithi157:> os.close(dirfd) 2023-11-06T03:05:27.329 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:05:39.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:05:39.694 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:05:51.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:51.043 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-06T03:05:51.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:05:51.098 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2023-11-06T03:06:09.780 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:06:09.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:06:40.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:06:40.142 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:06:50.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:06:50.564 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-06T03:06:50.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2023-11-06T03:06:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 Processor -- start 2023-11-06T03:06:50.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 -- start start 2023-11-06T03:06:50.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5b1c146370 0x7f5b1c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0ce800 0x7f5b1c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0cf1a0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c06c3d0 con 0x7f5b1c146370 2023-11-06T03:06:50.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b246e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c06f5a0 con 0x7f5b1c06b6a0 2023-11-06T03:06:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b21c7f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0ce800 0x7f5b1c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b21c7f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b1c0ce800 0x7f5b1c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60780/0 (socket says 172.21.15.157:60780) 2023-11-06T03:06:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.733+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2773166508 learned_addr learned my addr 172.21.15.157:0/2773166508 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b22480640 1 --2- 172.21.15.157:0/2773166508 >> v2:172.21.15.195:3300/0 conn(0x7f5b1c146370 0x7f5b1c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b22c81640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2773166508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 msgr2=0x7f5b1c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2773166508 >> v2:172.21.15.195:3300/0 conn(0x7f5b1c146370 msgr2=0x7f5b1c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2773166508 >> v2:172.21.15.195:3300/0 conn(0x7f5b1c146370 0x7f5b1c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2773166508 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c06f880 con 0x7f5b1c0ce800 2023-11-06T03:06:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c0cec60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0ae190 tx=0x7f5b0c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2045a0ccf59c49d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b2147e640 1 -- 172.21.15.157:0/2773166508 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ab600 con 0x7f5b1c0ce800 2023-11-06T03:06:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b2147e640 1 -- 172.21.15.157:0/2773166508 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0ab760 con 0x7f5b1c0ce800 2023-11-06T03:06:50.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b2147e640 1 -- 172.21.15.157:0/2773166508 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0abaf0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 -- 172.21.15.157:0/2773166508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 msgr2=0x7f5b1c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c0cec60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0ae190 tx=0x7f5b0c01c440 comp rx=0 tx=0).stop 2023-11-06T03:06:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 -- 172.21.15.157:0/2773166508 shutdown_connections 2023-11-06T03:06:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 --2- 172.21.15.157:0/2773166508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c0cec60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 --2- 172.21.15.157:0/2773166508 >> v2:172.21.15.195:3300/0 conn(0x7f5b1c146370 0x7f5b1c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 -- 172.21.15.157:0/2773166508 >> 172.21.15.157:0/2773166508 conn(0x7f5b1c065930 msgr2=0x7f5b1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 -- 172.21.15.157:0/2773166508 shutdown_connections 2023-11-06T03:06:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.737+0000 7f5b246e4640 1 -- 172.21.15.157:0/2773166508 wait complete. 2023-11-06T03:06:50.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 Processor -- start 2023-11-06T03:06:50.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- start start 2023-11-06T03:06:50.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c13bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1c146370 0x7f5b1c13ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c13d3b0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c13b640 con 0x7f5b1c146370 2023-11-06T03:06:50.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c13e820 con 0x7f5b1c06b6a0 2023-11-06T03:06:50.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60786/0 (socket says 172.21.15.157:60786) 2023-11-06T03:06:50.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 -- 172.21.15.157:0/476230213 learned_addr learned my addr 172.21.15.157:0/476230213 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b22c81640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1c146370 0x7f5b1c13ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b22480640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c13bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 -- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 msgr2=0x7f5b1c13bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c13bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 -- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1c146370 msgr2=0x7f5b1c13ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1c146370 0x7f5b1c13ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 -- 172.21.15.157:0/476230213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0094d0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01aaf0 tx=0x7f5b0c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b12ffd640 1 -- 172.21.15.157:0/476230213 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b4800 con 0x7f5b1c0ce800 2023-11-06T03:06:50.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b12ffd640 1 -- 172.21.15.157:0/476230213 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0b4960 con 0x7f5b1c0ce800 2023-11-06T03:06:50.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c13ad10 con 0x7f5b1c0ce800 2023-11-06T03:06:50.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b12ffd640 1 -- 172.21.15.157:0/476230213 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b4cf0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.741+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c13b2d0 con 0x7f5b1c0ce800 2023-11-06T03:06:50.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b12ffd640 1 -- 172.21.15.157:0/476230213 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b0c0b3380 con 0x7f5b1c0ce800 2023-11-06T03:06:50.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b12ffd640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 0x7f5b00067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b12ffd640 1 -- 172.21.15.157:0/476230213 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5b0c11d940 con 0x7f5b1c0ce800 2023-11-06T03:06:50.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 0x7f5b00067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 Processor -- start 2023-11-06T03:06:50.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 -- start start 2023-11-06T03:06:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 0x7f5b00067f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b950 tx=0x7f5b18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a581ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a581f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a56e6bb0 con 0x5574a58690d0 2023-11-06T03:06:50.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a5675380 con 0x5574a58699f0 2023-11-06T03:06:50.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574a5672950 con 0x5574a581f200 2023-11-06T03:06:50.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60800/0 (socket says 172.21.15.157:60800) 2023-11-06T03:06:50.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 -- 172.21.15.157:0/1629039300 learned_addr learned my addr 172.21.15.157:0/1629039300 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22c81640 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a581f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a581ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 -- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 msgr2=0x5574a581f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a581f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 -- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 msgr2=0x5574a581ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a581ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 -- 172.21.15.157:0/1629039300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a536adc0 con 0x5574a58690d0 2023-11-06T03:06:50.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b22480640 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b2f0 tx=0x7f5b18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1a072b7b34243c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5afffff640 1 -- 172.21.15.157:0/1629039300 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18014be0 con 0x5574a58690d0 2023-11-06T03:06:50.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5afffff640 1 -- 172.21.15.157:0/1629039300 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18019070 con 0x5574a58690d0 2023-11-06T03:06:50.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5afffff640 1 -- 172.21.15.157:0/1629039300 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18020440 con 0x5574a58690d0 2023-11-06T03:06:50.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 -- 172.21.15.157:0/1629039300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 msgr2=0x5574a58694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.745+0000 7f5b2780c000 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b2f0 tx=0x7f5b18012000 comp rx=0 tx=0).stop 2023-11-06T03:06:50.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/1629039300 shutdown_connections 2023-11-06T03:06:50.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a581f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a581ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- 172.21.15.157:0/1629039300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58694b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/1629039300 >> 172.21.15.157:0/1629039300 conn(0x5574a5825990 msgr2=0x5574a581d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/1629039300 shutdown_connections 2023-11-06T03:06:50.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/1629039300 wait complete. 2023-11-06T03:06:50.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- start start 2023-11-06T03:06:50.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 Processor -- start 2023-11-06T03:06:50.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a58704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a5869dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a566e240 con 0x5574a58690d0 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a560d410 con 0x5574a58699f0 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574a562b600 con 0x5574a581f200 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 10 client.0 _ll_register_callbacks cb 0x5574a580abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60806/0 (socket says 172.21.15.157:60806) 2023-11-06T03:06:50.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 -- 172.21.15.157:0/3936794614 learned_addr learned my addr 172.21.15.157:0/3936794614 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 -- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 msgr2=0x5574a58704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b22480640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a58704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b22c81640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a5869dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a58704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 -- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 msgr2=0x5574a5869dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a5869dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 -- 172.21.15.157:0/3936794614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1800e040 con 0x5574a58690d0 2023-11-06T03:06:50.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01c9b0 tx=0x7f5b0c0ee8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b714d17a1338c440 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c106360 con 0x5574a58690d0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5b0c0f68f0 con 0x5574a58690d0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574a562b040 con 0x5574a58690d0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b0c0f6ab0 con 0x5574a58690d0 2023-11-06T03:06:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 10 client.5571 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:50.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5574a5633170 con 0x5574a58690d0 2023-11-06T03:06:50.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574a562b040 con 0x5574a58690d0 2023-11-06T03:06:50.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5b0c0eac40 con 0x5574a58690d0 2023-11-06T03:06:50.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 10 client.5571 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:50.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 10 client.5571 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:50.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 20 client.5571 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:50.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a586ab70 0x5574a586af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 4 client.5571 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:06:50.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574a566cb90 con 0x5574a586ab70 2023-11-06T03:06:50.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b22480640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a586ab70 0x5574a586af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 10 client.5571 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:50.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b22480640 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a586ab70 0x5574a586af50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c531617cc52c3e3f server_cookie=b64202507646a4e9 in_seq=0 out_seq=0 2023-11-06T03:06:50.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 1 -- 172.21.15.157:0/3936794614 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5574a566cb90 con 0x5574a586ab70 2023-11-06T03:06:50.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5afd7fa640 10 client.5571 handle_command_reply: tid=0 2023-11-06T03:06:50.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.749+0000 7f5b2780c000 1 client.5571 shutdown 2023-11-06T03:06:50.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 msgr2=0x5574a5870a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01c9b0 tx=0x7f5b0c0ee8a0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 shutdown_connections 2023-11-06T03:06:50.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a586ab70 0x5574a586af50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a58699f0 0x5574a5869dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a5870a20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 --2- 172.21.15.157:0/3936794614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a581f200 0x5574a58704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5afd7fa640 0 client.5571 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:50.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 >> 172.21.15.157:0/3936794614 conn(0x5574a5825990 msgr2=0x5574a581dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 shutdown_connections 2023-11-06T03:06:50.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 1 -- 172.21.15.157:0/3936794614 wait complete. 2023-11-06T03:06:50.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.753+0000 7f5b2780c000 20 client.5571 trim_cache size 0 max 16384 2023-11-06T03:06:50.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 Processor -- start 2023-11-06T03:06:50.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 -- start start 2023-11-06T03:06:50.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a532ba40 con 0x5574a58690d0 2023-11-06T03:06:50.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a56f6c10 con 0x5574a581f200 2023-11-06T03:06:50.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574a5876560 con 0x5574a58699f0 2023-11-06T03:06:50.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60816/0 (socket says 172.21.15.157:60816) 2023-11-06T03:06:50.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2611191572 learned_addr learned my addr 172.21.15.157:0/2611191572 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b22c81640 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b22480640 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 msgr2=0x5574a58c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58c1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 msgr2=0x5574a58c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2611191572 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a56194c0 con 0x5574a58690d0 2023-11-06T03:06:50.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0eb350 tx=0x7f5b0c0c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb3522844e1ddde server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b10ff9640 1 -- 172.21.15.157:0/2611191572 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ee990 con 0x5574a58690d0 2023-11-06T03:06:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b10ff9640 1 -- 172.21.15.157:0/2611191572 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0eb190 con 0x5574a58690d0 2023-11-06T03:06:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.765+0000 7f5b10ff9640 1 -- 172.21.15.157:0/2611191572 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c106380 con 0x5574a58690d0 2023-11-06T03:06:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2611191572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 msgr2=0x5574a58c0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0eb350 tx=0x7f5b0c0c0000 comp rx=0 tx=0).stop 2023-11-06T03:06:50.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2611191572 shutdown_connections 2023-11-06T03:06:50.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c0b40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2611191572 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2611191572 >> 172.21.15.157:0/2611191572 conn(0x5574a5825990 msgr2=0x5574a58249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2611191572 shutdown_connections 2023-11-06T03:06:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2611191572 wait complete. 2023-11-06T03:06:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- start start 2023-11-06T03:06:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 Processor -- start 2023-11-06T03:06:50.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a58224d0 con 0x5574a58690d0 2023-11-06T03:06:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a566b640 con 0x5574a581f200 2023-11-06T03:06:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574a56685e0 con 0x5574a58699f0 2023-11-06T03:06:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 10 client.0 _ll_register_callbacks cb 0x5574a5632c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60822/0 (socket says 172.21.15.157:60822) 2023-11-06T03:06:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2113832902 learned_addr learned my addr 172.21.15.157:0/2113832902 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:50.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b22480640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b22c81640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 msgr2=0x5574a58a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 msgr2=0x5574a58c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 -- 172.21.15.157:0/2113832902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574a5633170 con 0x5574a58690d0 2023-11-06T03:06:50.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b21c7f640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01c920 tx=0x7f5b0c11bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca897fb8e8555ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:50.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:50.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0f2bd0 con 0x5574a58690d0 2023-11-06T03:06:50.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5b0c0eddb0 con 0x5574a58690d0 2023-11-06T03:06:50.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574a56375c0 con 0x5574a58690d0 2023-11-06T03:06:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b0c0edfb0 con 0x5574a58690d0 2023-11-06T03:06:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 10 client.5577 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5574a560d030 con 0x5574a58690d0 2023-11-06T03:06:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574a56375c0 con 0x5574a58690d0 2023-11-06T03:06:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5b0c0059a0 con 0x5574a58690d0 2023-11-06T03:06:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 10 client.5577 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 10 client.5577 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 20 client.5577 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a58a9be0 0x5574a58a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 4 client.5577 mds_command: new command op to 5331 tid=0[{"prefix": "perf dump"}] 2023-11-06T03:06:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5574a56672c0 con 0x5574a58a9be0 2023-11-06T03:06:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b22480640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a58a9be0 0x5574a58a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5afd7fa640 10 client.5577 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.769+0000 7f5b22480640 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a58a9be0 0x5574a58a9fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d2d0d617d1b240 server_cookie=bba349d84bffefbe in_seq=0 out_seq=0 2023-11-06T03:06:50.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5afd7fa640 1 -- 172.21.15.157:0/2113832902 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17910 (crc 0 0 0) 0x5574a56672c0 con 0x5574a58a9be0 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5afd7fa640 10 client.5577 handle_command_reply: tid=0 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 client.5577 shutdown 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 msgr2=0x5574a58c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01c920 tx=0x7f5b0c11bb20 comp rx=0 tx=0).stop 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 shutdown_connections 2023-11-06T03:06:50.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x5574a58a9be0 0x5574a58a9fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574a58699f0 0x5574a58a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574a58690d0 0x5574a58c6200 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 --2- 172.21.15.157:0/2113832902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574a581f200 0x5574a58c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5afd7fa640 0 client.5577 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:50.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 >> 172.21.15.157:0/2113832902 conn(0x5574a5825990 msgr2=0x5574a586cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 shutdown_connections 2023-11-06T03:06:50.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 1 -- 172.21.15.157:0/2113832902 wait complete. 2023-11-06T03:06:50.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b2780c000 20 client.5577 trim_cache size 0 max 16384 2023-11-06T03:06:50.798 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:06:50.799 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T03:06:50.799 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 6133, 2023-11-06T03:06:50.799 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 6132, 2023-11-06T03:06:50.799 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1707700, 2023-11-06T03:06:50.799 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 49515132, 2023-11-06T03:06:50.800 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2023-11-06T03:06:50.800 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2023-11-06T03:06:50.800 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.666570063, 2023-11-06T03:06:50.800 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.330785424, 2023-11-06T03:06:50.800 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.120053124, 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.182124644, 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6131, 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.073259212, 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001479898 2023-11-06T03:06:50.801 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.802 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:06:50.802 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2023-11-06T03:06:50.802 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000246, 2023-11-06T03:06:50.802 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000123 2023-11-06T03:06:50.802 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.803 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1568, 2023-11-06T03:06:50.803 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2023-11-06T03:06:50.803 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.803 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T03:06:50.803 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4890, 2023-11-06T03:06:50.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4872, 2023-11-06T03:06:50.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1505931, 2023-11-06T03:06:50.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 19119137, 2023-11-06T03:06:50.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2023-11-06T03:06:50.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:06:50.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.387416770, 2023-11-06T03:06:50.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.147017374, 2023-11-06T03:06:50.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.091525775, 2023-11-06T03:06:50.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.130832096, 2023-11-06T03:06:50.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4872, 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.021060329, 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000620086 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:06:50.806 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.807 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.807 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.807 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.807 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 130256, 2023-11-06T03:06:50.808 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 25408 2023-11-06T03:06:50.808 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.808 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T03:06:50.808 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 35359, 2023-11-06T03:06:50.808 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 61498, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 12760935, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 58671847, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 4.716205031, 2023-11-06T03:06:50.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.337419429, 2023-11-06T03:06:50.810 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 2.717079486, 2023-11-06T03:06:50.810 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.180573849, 2023-11-06T03:06:50.810 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:06:50.810 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 61496, 2023-11-06T03:06:50.810 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.941669668, 2023-11-06T03:06:50.811 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000064096 2023-11-06T03:06:50.811 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:06:50.811 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 87194, 2023-11-06T03:06:50.811 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.019689702, 2023-11-06T03:06:50.812 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000225 2023-11-06T03:06:50.812 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T03:06:50.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 118928 2023-11-06T03:06:50.812 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-11-06T03:06:50.813 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:06:50.814 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:06:50.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8402, 2023-11-06T03:06:50.814 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.398542765, 2023-11-06T03:06:50.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003022916 2023-11-06T03:06:50.814 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.815 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.815 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-11-06T03:06:50.815 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:06:50.815 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:06:50.815 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2040, 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.354745806, 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000664091 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-11-06T03:06:50.816 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 28522, 2023-11-06T03:06:50.817 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 28522, 2023-11-06T03:06:50.817 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-11-06T03:06:50.817 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28522, 2023-11-06T03:06:50.817 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 786.287741089, 2023-11-06T03:06:50.817 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.027567763 2023-11-06T03:06:50.818 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.818 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-11-06T03:06:50.818 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-11-06T03:06:50.818 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-11-06T03:06:50.818 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-11-06T03:06:50.819 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 51, 2023-11-06T03:06:50.819 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-11-06T03:06:50.819 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-11-06T03:06:50.819 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 18030, 2023-11-06T03:06:50.819 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 11536, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 6494, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 1550, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 336, 2023-11-06T03:06:50.820 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 336, 2023-11-06T03:06:50.821 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2023-11-06T03:06:50.821 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 56058, 2023-11-06T03:06:50.821 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 54606, 2023-11-06T03:06:50.821 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-11-06T03:06:50.821 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-11-06T03:06:50.822 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-11-06T03:06:50.822 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-11-06T03:06:50.822 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 1130, 2023-11-06T03:06:50.822 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 327473, 2023-11-06T03:06:50.822 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-11-06T03:06:50.823 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2023-11-06T03:06:50.824 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-11-06T03:06:50.824 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2023-11-06T03:06:50.824 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2023-11-06T03:06:50.824 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2023-11-06T03:06:50.824 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T03:06:50.825 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T03:06:50.826 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2023-11-06T03:06:50.826 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-11-06T03:06:50.826 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 7867, 2023-11-06T03:06:50.826 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 6240, 2023-11-06T03:06:50.826 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T03:06:50.827 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T03:06:50.827 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T03:06:50.827 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 7867, 2023-11-06T03:06:50.827 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 2000, 2023-11-06T03:06:50.827 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 44, 2023-11-06T03:06:50.828 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 17980 2023-11-06T03:06:50.828 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.828 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-11-06T03:06:50.828 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-11-06T03:06:50.828 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 2000, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-11-06T03:06:50.829 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-11-06T03:06:50.830 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 4157, 2023-11-06T03:06:50.830 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 1666, 2023-11-06T03:06:50.830 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 4384, 2023-11-06T03:06:50.830 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-11-06T03:06:50.830 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-11-06T03:06:50.831 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-11-06T03:06:50.831 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-11-06T03:06:50.831 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-11-06T03:06:50.831 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-11-06T03:06:50.831 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T03:06:50.832 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-11-06T03:06:50.832 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-11-06T03:06:50.832 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-11-06T03:06:50.832 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-11-06T03:06:50.832 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 23776, 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 22514, 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 21355, 2023-11-06T03:06:50.833 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 2421, 2023-11-06T03:06:50.834 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-11-06T03:06:50.834 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 1159, 2023-11-06T03:06:50.834 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 30, 2023-11-06T03:06:50.834 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 27, 2023-11-06T03:06:50.834 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 25, 2023-11-06T03:06:50.835 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 5, 2023-11-06T03:06:50.835 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-11-06T03:06:50.835 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 2, 2023-11-06T03:06:50.835 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 75506685, 2023-11-06T03:06:50.835 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 84070794, 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6931, 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 144.050620371, 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.020783526 2023-11-06T03:06:50.836 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.837 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2023-11-06T03:06:50.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.837 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-11-06T03:06:50.837 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 2014, 2023-11-06T03:06:50.837 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2014, 2023-11-06T03:06:50.838 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2023-11-06T03:06:50.838 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 13, 2023-11-06T03:06:50.838 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 13, 2023-11-06T03:06:50.838 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2023-11-06T03:06:50.838 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 18030, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 19995, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 1965, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 336, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2002, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 1666, 2023-11-06T03:06:50.839 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 112636, 2023-11-06T03:06:50.840 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207144 2023-11-06T03:06:50.840 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.840 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-11-06T03:06:50.840 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 42489, 2023-11-06T03:06:50.840 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 28522, 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 7, 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2000, 2023-11-06T03:06:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.454271185, 2023-11-06T03:06:50.842 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000727135 2023-11-06T03:06:50.842 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.842 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-11-06T03:06:50.842 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 187, 2023-11-06T03:06:50.842 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.060657438, 2023-11-06T03:06:50.843 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000324371 2023-11-06T03:06:50.843 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.843 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-11-06T03:06:50.843 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.843 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6000, 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.698859712, 2023-11-06T03:06:50.844 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001616476 2023-11-06T03:06:50.845 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.845 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-11-06T03:06:50.845 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7938, 2023-11-06T03:06:50.845 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 14.077108772, 2023-11-06T03:06:50.845 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001773382 2023-11-06T03:06:50.846 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.846 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-11-06T03:06:50.846 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.846 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.846 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.847 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.848 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-11-06T03:06:50.848 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.848 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.848 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.848 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.849 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-11-06T03:06:50.850 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2023-11-06T03:06:50.850 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.021098310, 2023-11-06T03:06:50.850 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007032770 2023-11-06T03:06:50.850 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.850 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-11-06T03:06:50.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.851 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.851 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.851 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004238638, 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004238638 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-11-06T03:06:50.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.853 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.853 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.853 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-11-06T03:06:50.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:06:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.010023467, 2023-11-06T03:06:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.010023467 2023-11-06T03:06:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-11-06T03:06:50.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-11-06T03:06:50.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6, 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.219465638, 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.036577606 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4384, 2023-11-06T03:06:50.856 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 730.913860502, 2023-11-06T03:06:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.166723052 2023-11-06T03:06:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-11-06T03:06:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.857 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-11-06T03:06:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.858 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-11-06T03:06:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.859 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.007054149, 2023-11-06T03:06:50.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007054149 2023-11-06T03:06:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-11-06T03:06:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.861 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-11-06T03:06:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:06:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000563313, 2023-11-06T03:06:50.862 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000563313 2023-11-06T03:06:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-11-06T03:06:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-11-06T03:06:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-11-06T03:06:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.867 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-11-06T03:06:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8000, 2023-11-06T03:06:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.820539965, 2023-11-06T03:06:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003727567 2023-11-06T03:06:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.868 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.869 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-11-06T03:06:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2023-11-06T03:06:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2023-11-06T03:06:50.870 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2023-11-06T03:06:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2023-11-06T03:06:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-11-06T03:06:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 15199, 2023-11-06T03:06:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 15199, 2023-11-06T03:06:50.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 82, 2023-11-06T03:06:50.872 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2023-11-06T03:06:50.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.872 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-11-06T03:06:50.872 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-11-06T03:06:50.872 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T03:06:50.873 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T03:06:50.874 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T03:06:50.874 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T03:06:50.874 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T03:06:50.874 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-11-06T03:06:50.874 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T03:06:50.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T03:06:50.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-11-06T03:06:50.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-11-06T03:06:50.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-11-06T03:06:50.875 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T03:06:50.876 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-11-06T03:06:50.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-11-06T03:06:50.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-11-06T03:06:50.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T03:06:50.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T03:06:50.877 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-11-06T03:06:50.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-11-06T03:06:50.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-11-06T03:06:50.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-11-06T03:06:50.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T03:06:50.878 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 18077014, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 4545, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 176, 2023-11-06T03:06:50.879 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 2, 2023-11-06T03:06:50.880 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-11-06T03:06:50.880 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-11-06T03:06:50.880 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-11-06T03:06:50.880 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-11-06T03:06:50.880 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 6272, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 80, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-11-06T03:06:50.881 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-11-06T03:06:50.882 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-11-06T03:06:50.882 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 19320076, 2023-11-06T03:06:50.882 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 800701, 2023-11-06T03:06:50.882 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-11-06T03:06:50.882 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-11-06T03:06:50.883 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-11-06T03:06:50.883 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-11-06T03:06:50.883 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.883 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-11-06T03:06:50.883 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-11-06T03:06:50.884 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-11-06T03:06:50.884 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 14138, 2023-11-06T03:06:50.884 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 91095429, 2023-11-06T03:06:50.884 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 37, 2023-11-06T03:06:50.884 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 14138, 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 14138, 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 373.069643297, 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.026387724 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.885 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-11-06T03:06:50.886 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-11-06T03:06:50.886 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 14138, 2023-11-06T03:06:50.886 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 24539 2023-11-06T03:06:50.886 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.886 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 14138, 2023-11-06T03:06:50.887 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 100, 2023-11-06T03:06:50.887 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 14038, 2023-11-06T03:06:50.887 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-11-06T03:06:50.887 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-11-06T03:06:50.887 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 38, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 5134, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 100, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 7031, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 76, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-11-06T03:06:50.888 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-11-06T03:06:50.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 1666, 2023-11-06T03:06:50.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-11-06T03:06:50.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 48, 2023-11-06T03:06:50.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-11-06T03:06:50.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-11-06T03:06:50.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-11-06T03:06:50.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2023-11-06T03:06:50.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 10268, 2023-11-06T03:06:50.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-11-06T03:06:50.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T03:06:50.891 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-11-06T03:06:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-11-06T03:06:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-11-06T03:06:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-11-06T03:06:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-11-06T03:06:50.892 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-11-06T03:06:50.893 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-11-06T03:06:50.893 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-11-06T03:06:50.893 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-11-06T03:06:50.893 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-11-06T03:06:50.893 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-11-06T03:06:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-11-06T03:06:50.895 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-11-06T03:06:50.895 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-11-06T03:06:50.895 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 39, 2023-11-06T03:06:50.895 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-11-06T03:06:50.895 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 3, 2023-11-06T03:06:50.896 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 8, 2023-11-06T03:06:50.896 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 8, 2023-11-06T03:06:50.896 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-11-06T03:06:50.896 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-11-06T03:06:50.896 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 142, 2023-11-06T03:06:50.897 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2023-11-06T03:06:50.897 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 36 2023-11-06T03:06:50.897 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.897 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-11-06T03:06:50.897 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2023-11-06T03:06:50.898 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-11-06T03:06:50.898 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-11-06T03:06:50.898 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-11-06T03:06:50.898 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.898 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 28522 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 32, 2023-11-06T03:06:50.899 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-11-06T03:06:50.900 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 32, 2023-11-06T03:06:50.900 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 1666, 2023-11-06T03:06:50.900 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-11-06T03:06:50.900 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.900 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T03:06:50.901 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:06:50.901 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:06:50.901 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:06:50.901 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 46377, 2023-11-06T03:06:50.901 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 11794469, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 46377, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 46377, 2023-11-06T03:06:50.902 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 11794469, 2023-11-06T03:06:50.903 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:06:50.903 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.903 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.903 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.903 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.904 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.904 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-11-06T03:06:50.904 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:06:50.904 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:06:50.904 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:06:50.905 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:06:50.905 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:06:50.905 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:06:50.905 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:06:50.905 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 14138, 2023-11-06T03:06:50.906 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 91277023, 2023-11-06T03:06:50.906 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 14138, 2023-11-06T03:06:50.906 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 91277023, 2023-11-06T03:06:50.906 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:06:50.906 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-11-06T03:06:50.907 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:06:50.908 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-11-06T03:06:50.908 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:06:50.908 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:06:50.908 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:06:50.908 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:06:50.909 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:06:50.909 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 14138, 2023-11-06T03:06:50.909 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 14138, 2023-11-06T03:06:50.909 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 14138, 2023-11-06T03:06:50.909 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 14138, 2023-11-06T03:06:50.910 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:06:50.910 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.910 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.910 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.910 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:06:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1666, 2023-11-06T03:06:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 181594, 2023-11-06T03:06:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:06:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1666, 2023-11-06T03:06:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:06:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:06:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 100, 2023-11-06T03:06:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 181594, 2023-11-06T03:06:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:06:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:06:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x556dcf170d20": { 2023-11-06T03:06:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 84, 2023-11-06T03:06:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:06:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:06:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 23776, 2023-11-06T03:06:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 79876490, 2023-11-06T03:06:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 23776, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6931, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 79876406, 2023-11-06T03:06:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:06:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:06:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:06:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:06:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:50.918 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:06:50.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 msgr2=0x7f5b00067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 0x7f5b00067f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b950 tx=0x7f5b18008040 comp rx=0 tx=0).stop 2023-11-06T03:06:50.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 msgr2=0x7f5b1c13e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:50.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01aaf0 tx=0x7f5b0c01ac80 comp rx=0 tx=0).stop 2023-11-06T03:06:50.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 shutdown_connections 2023-11-06T03:06:50.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b00065a90 0x7f5b00067f50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b1c146370 0x7f5b1c13ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c0ce800 0x7f5b1c13e120 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 --2- 172.21.15.157:0/476230213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b1c06b6a0 0x7f5b1c13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:50.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 >> 172.21.15.157:0/476230213 conn(0x7f5b1c065930 msgr2=0x7f5b1c0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:50.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 shutdown_connections 2023-11-06T03:06:50.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:50.773+0000 7f5b246e4640 1 -- 172.21.15.157:0/476230213 wait complete. 2023-11-06T03:06:50.922 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2023-11-06T03:06:50.922 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2023-11-06T03:06:51.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.053+0000 7f5d33531640 1 Processor -- start 2023-11-06T03:06:51.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.053+0000 7f5d33531640 1 -- start start 2023-11-06T03:06:51.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d33531640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c155310 0x7f5d2c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d33531640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d2c06c550 0x7f5d2c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d33531640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0cb100 con 0x7f5d2c06c170 2023-11-06T03:06:51.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d33531640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c0ce0a0 con 0x7f5d2c06c550 2023-11-06T03:06:51.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d33531640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c15bef0 con 0x7f5d2c155310 2023-11-06T03:06:51.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d2c06c550 0x7f5d2c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d2c06c550 0x7f5d2c15b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60834/0 (socket says 172.21.15.157:60834) 2023-11-06T03:06:51.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 -- 172.21.15.157:0/2953195413 learned_addr learned my addr 172.21.15.157:0/2953195413 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d30acc640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c155310 0x7f5d2c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 -- 172.21.15.157:0/2953195413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c155310 msgr2=0x7f5d2c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c155310 0x7f5d2c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 -- 172.21.15.157:0/2953195413 >> v1:172.21.15.195:6789/0 conn(0x7f5d2c06c170 legacy=0x7f5d2c154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 -- 172.21.15.157:0/2953195413 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0cb350 con 0x7f5d2c06c550 2023-11-06T03:06:51.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d31ace640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c550 0x7f5d2c15b690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009e50 tx=0x7f5d2401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c02fb88dfc9e86 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d23fff640 1 -- 172.21.15.157:0/2953195413 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d240a58a0 con 0x7f5d2c06c550 2023-11-06T03:06:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d23fff640 1 -- 172.21.15.157:0/2953195413 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d240a5a00 con 0x7f5d2c06c550 2023-11-06T03:06:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.057+0000 7f5d23fff640 1 -- 172.21.15.157:0/2953195413 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d240a5d90 con 0x7f5d2c06c550 2023-11-06T03:06:51.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/2953195413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c550 msgr2=0x7f5d2c15b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c550 0x7f5d2c15b690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009e50 tx=0x7f5d2401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/2953195413 shutdown_connections 2023-11-06T03:06:51.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c550 0x7f5d2c15b690 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- 172.21.15.157:0/2953195413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c155310 0x7f5d2c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/2953195413 >> 172.21.15.157:0/2953195413 conn(0x7f5d2c0b43b0 msgr2=0x7f5d2c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/2953195413 shutdown_connections 2023-11-06T03:06:51.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/2953195413 wait complete. 2023-11-06T03:06:51.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 Processor -- start 2023-11-06T03:06:51.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- start start 2023-11-06T03:06:51.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c06c550 0x7f5d2c139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d2c155310 0x7f5d2c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c151910 con 0x7f5d2c06c170 2023-11-06T03:06:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c14ccf0 con 0x7f5d2c155310 2023-11-06T03:06:51.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c1545f0 con 0x7f5d2c06c550 2023-11-06T03:06:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60838/0 (socket says 172.21.15.157:60838) 2023-11-06T03:06:51.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 -- 172.21.15.157:0/4222564269 learned_addr learned my addr 172.21.15.157:0/4222564269 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c06c550 0x7f5d2c139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d31ace640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d2c155310 0x7f5d2c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 -- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c06c550 msgr2=0x7f5d2c139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c06c550 0x7f5d2c139180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 -- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d2c155310 msgr2=0x7f5d2c14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d2c155310 0x7f5d2c14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 -- 172.21.15.157:0/4222564269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24009b30 con 0x7f5d2c06c170 2023-11-06T03:06:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c500 tx=0x7f5d1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d21ffb640 1 -- 172.21.15.157:0/4222564269 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c005220 con 0x7f5d2c06c170 2023-11-06T03:06:51.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d21ffb640 1 -- 172.21.15.157:0/4222564269 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c005380 con 0x7f5d2c06c170 2023-11-06T03:06:51.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0cbfb0 con 0x7f5d2c06c170 2023-11-06T03:06:51.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d21ffb640 1 -- 172.21.15.157:0/4222564269 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c007270 con 0x7f5d2c06c170 2023-11-06T03:06:51.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.061+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c14e270 con 0x7f5d2c06c170 2023-11-06T03:06:51.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d21ffb640 1 -- 172.21.15.157:0/4222564269 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5d1c0073d0 con 0x7f5d2c06c170 2023-11-06T03:06:51.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d21ffb640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 0x7f5d10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d21ffb640 1 -- 172.21.15.157:0/4222564269 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5d1c082ba0 con 0x7f5d2c06c170 2023-11-06T03:06:51.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 0x7f5d10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 Processor -- start 2023-11-06T03:06:51.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 0x7f5d10067f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0cf650 tx=0x7f5d28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 -- start start 2023-11-06T03:06:51.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f4604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f3282d0 con 0x55db7f4aa620 2023-11-06T03:06:51.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f2b5200 con 0x55db7f4a9d00 2023-11-06T03:06:51.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7f2b27d0 con 0x55db7f4600e0 2023-11-06T03:06:51.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60844/0 (socket says 172.21.15.157:60844) 2023-11-06T03:06:51.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 -- 172.21.15.157:0/3059243137 learned_addr learned my addr 172.21.15.157:0/3059243137 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d312cd640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d31ace640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f4604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 -- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 msgr2=0x55db7f4604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f4604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 -- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 msgr2=0x55db7f4aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 -- 172.21.15.157:0/3059243137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7efaad50 con 0x55db7f4aa620 2023-11-06T03:06:51.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d31ace640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f4604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:06:51.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.065+0000 7f5d30acc640 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b2d0 tx=0x7f5d28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3ef701ac99cc3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d16ffd640 1 -- 172.21.15.157:0/3059243137 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28014be0 con 0x55db7f4aa620 2023-11-06T03:06:51.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d16ffd640 1 -- 172.21.15.157:0/3059243137 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28019070 con 0x55db7f4aa620 2023-11-06T03:06:51.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d16ffd640 1 -- 172.21.15.157:0/3059243137 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800fe80 con 0x55db7f4aa620 2023-11-06T03:06:51.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/3059243137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 msgr2=0x55db7f45fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b2d0 tx=0x7f5d28012000 comp rx=0 tx=0).stop 2023-11-06T03:06:51.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/3059243137 shutdown_connections 2023-11-06T03:06:51.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f4604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4aa620 0x55db7f45fba0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- 172.21.15.157:0/3059243137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/3059243137 >> 172.21.15.157:0/3059243137 conn(0x55db7f466450 msgr2=0x55db7f45e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/3059243137 shutdown_connections 2023-11-06T03:06:51.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/3059243137 wait complete. 2023-11-06T03:06:51.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- start start 2023-11-06T03:06:51.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 Processor -- start 2023-11-06T03:06:51.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f464d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f46b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f2ae0c0 con 0x55db7f4a9d00 2023-11-06T03:06:51.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f24d890 con 0x55db7f4aa620 2023-11-06T03:06:51.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7f24d530 con 0x55db7f4600e0 2023-11-06T03:06:51.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 10 client.0 _ll_register_callbacks cb 0x55db7f44b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:51.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60856/0 (socket says 172.21.15.157:60856) 2023-11-06T03:06:51.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 -- 172.21.15.157:0/1968366174 learned_addr learned my addr 172.21.15.157:0/1968366174 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d312cd640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f464d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d31ace640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f46b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 -- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 msgr2=0x55db7f464d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f464d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 -- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 msgr2=0x55db7f46b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f46b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 -- 172.21.15.157:0/1968366174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2800e040 con 0x55db7f4a9d00 2023-11-06T03:06:51.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d30acc640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d28022040 tx=0x7f5d28004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3c590ad7036f47 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:51.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28020c80 con 0x55db7f4a9d00 2023-11-06T03:06:51.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5d2800fe80 con 0x55db7f4a9d00 2023-11-06T03:06:51.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db7f2aca10 con 0x55db7f4a9d00 2023-11-06T03:06:51.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d28014e90 con 0x55db7f4a9d00 2023-11-06T03:06:51.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 10 client.5589 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:51.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55db7f2732d0 con 0x55db7f4a9d00 2023-11-06T03:06:51.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db7f2aca10 con 0x55db7f4a9d00 2023-11-06T03:06:51.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5d28015410 con 0x55db7f4a9d00 2023-11-06T03:06:51.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 10 client.5589 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:51.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 10 client.5589 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:51.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 20 client.5589 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:51.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4b8250 0x55db7f4ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 4 client.5589 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:06:51.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db7f2a7140 con 0x55db7f4b8250 2023-11-06T03:06:51.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d312cd640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4b8250 0x55db7f4ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d312cd640 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4b8250 0x55db7f4ba640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bfe834154c715c server_cookie=d70fa23b2a8c833 in_seq=0 out_seq=0 2023-11-06T03:06:51.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.069+0000 7f5d0bfff640 10 client.5589 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d0bfff640 1 -- 172.21.15.157:0/1968366174 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55db7f2a7140 con 0x55db7f4b8250 2023-11-06T03:06:51.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d0bfff640 10 client.5589 handle_command_reply: tid=0 2023-11-06T03:06:51.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 client.5589 shutdown 2023-11-06T03:06:51.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 msgr2=0x55db7f46acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d28022040 tx=0x7f5d28004740 comp rx=0 tx=0).stop 2023-11-06T03:06:51.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 shutdown_connections 2023-11-06T03:06:51.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4b8250 0x55db7f4ba640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f46b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4a9d00 0x55db7f46acc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d0bfff640 0 client.5589 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 --2- 172.21.15.157:0/1968366174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4600e0 0x55db7f464d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 >> 172.21.15.157:0/1968366174 conn(0x55db7f466450 msgr2=0x55db7f45eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 shutdown_connections 2023-11-06T03:06:51.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 1 -- 172.21.15.157:0/1968366174 wait complete. 2023-11-06T03:06:51.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.073+0000 7f5d36659000 20 client.5589 trim_cache size 0 max 16384 2023-11-06T03:06:51.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 Processor -- start 2023-11-06T03:06:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- start start 2023-11-06T03:06:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4aa620 0x55db7f4a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7ef6b9d0 con 0x55db7f4600e0 2023-11-06T03:06:51.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f24b5d0 con 0x55db7f4a9d00 2023-11-06T03:06:51.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7f338950 con 0x55db7f4aa620 2023-11-06T03:06:51.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60864/0 (socket says 172.21.15.157:60864) 2023-11-06T03:06:51.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 -- 172.21.15.157:0/598718154 learned_addr learned my addr 172.21.15.157:0/598718154 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d31ace640 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4aa620 0x55db7f4a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d30acc640 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 -- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4aa620 msgr2=0x55db7f4a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4aa620 0x55db7f4a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 -- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 msgr2=0x55db7f4bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4bc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 -- 172.21.15.157:0/598718154 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db7f259420 con 0x55db7f4600e0 2023-11-06T03:06:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d312cd640 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0505b0 tx=0x7f5d1c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cd3038a4fe3456 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d177fe640 1 -- 172.21.15.157:0/598718154 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c057ee0 con 0x55db7f4600e0 2023-11-06T03:06:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d177fe640 1 -- 172.21.15.157:0/598718154 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c05c090 con 0x55db7f4600e0 2023-11-06T03:06:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d177fe640 1 -- 172.21.15.157:0/598718154 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c052450 con 0x55db7f4600e0 2023-11-06T03:06:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- 172.21.15.157:0/598718154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 msgr2=0x55db7f4bbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0505b0 tx=0x7f5d1c001e50 comp rx=0 tx=0).stop 2023-11-06T03:06:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- 172.21.15.157:0/598718154 shutdown_connections 2023-11-06T03:06:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4aa620 0x55db7f4a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4a9d00 0x55db7f4bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 --2- 172.21.15.157:0/598718154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f4bbd60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- 172.21.15.157:0/598718154 >> 172.21.15.157:0/598718154 conn(0x55db7f466450 msgr2=0x55db7f4ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- 172.21.15.157:0/598718154 shutdown_connections 2023-11-06T03:06:51.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- 172.21.15.157:0/598718154 wait complete. 2023-11-06T03:06:51.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 -- start start 2023-11-06T03:06:51.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.089+0000 7f5d36659000 1 Processor -- start 2023-11-06T03:06:51.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4a9d00 0x55db7f50a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f4c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f45fc00 con 0x55db7f4600e0 2023-11-06T03:06:51.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db7f2cead0 con 0x55db7f4aa620 2023-11-06T03:06:51.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db7f24d670 con 0x55db7f4a9d00 2023-11-06T03:06:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 10 client.0 _ll_register_callbacks cb 0x55db7f328220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60866/0 (socket says 172.21.15.157:60866) 2023-11-06T03:06:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 -- 172.21.15.157:0/4083612393 learned_addr learned my addr 172.21.15.157:0/4083612393 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d31ace640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f4c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4a9d00 0x55db7f50a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 -- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4a9d00 msgr2=0x55db7f50a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4a9d00 0x55db7f50a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 -- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 msgr2=0x55db7f4c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f4c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 -- 172.21.15.157:0/4083612393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c14e270 con 0x55db7f4600e0 2023-11-06T03:06:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d312cd640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c017040 tx=0x7f5d1c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca7c2a8e11e9526 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c057ee0 con 0x55db7f4600e0 2023-11-06T03:06:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5d1c052450 con 0x55db7f4600e0 2023-11-06T03:06:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d28019070 con 0x55db7f4600e0 2023-11-06T03:06:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0507c0 con 0x55db7f4600e0 2023-11-06T03:06:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 10 client.5595 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55db7f24cf40 con 0x55db7f4600e0 2023-11-06T03:06:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28019070 con 0x55db7f4600e0 2023-11-06T03:06:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5d1c019030 con 0x55db7f4600e0 2023-11-06T03:06:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 10 client.5595 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 10 client.5595 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:51.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 20 client.5595 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:51.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4c55f0 0x55db7f4c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 4 client.5595 mds_command: new command op to 5331 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2023-11-06T03:06:51.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55db7f2a6e60 con 0x55db7f4c55f0 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4c55f0 0x55db7f4c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d0bfff640 10 client.5595 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.093+0000 7f5d30acc640 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4c55f0 0x55db7f4c59d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49032ae639df69f9 server_cookie=a6b83dc5f8e0e570 in_seq=0 out_seq=0 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.097+0000 7f5d0bfff640 1 -- 172.21.15.157:0/4083612393 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55db7f2a6e60 con 0x55db7f4c55f0 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.097+0000 7f5d0bfff640 10 client.5595 handle_command_reply: tid=0 2023-11-06T03:06:51.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.097+0000 7f5d36659000 1 client.5595 shutdown 2023-11-06T03:06:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 msgr2=0x55db7f509c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c017040 tx=0x7f5d1c001130 comp rx=0 tx=0).stop 2023-11-06T03:06:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 shutdown_connections 2023-11-06T03:06:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55db7f4c55f0 0x55db7f4c59d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55db7f4aa620 0x55db7f4c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55db7f4a9d00 0x55db7f50a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d0bfff640 0 client.5595 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 --2- 172.21.15.157:0/4083612393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55db7f4600e0 0x55db7f509c50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 >> 172.21.15.157:0/4083612393 conn(0x55db7f466450 msgr2=0x55db7f4ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 shutdown_connections 2023-11-06T03:06:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 1 -- 172.21.15.157:0/4083612393 wait complete. 2023-11-06T03:06:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d36659000 20 client.5595 trim_cache size 0 max 16384 2023-11-06T03:06:51.136 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:06:51.136 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:06:51.137 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "f9e2d576-5680-452f-b3ad-c8182a29b69c", 2023-11-06T03:06:51.137 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:06:51.137 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:06:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 msgr2=0x7f5d10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 0x7f5d10067f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0cf650 tx=0x7f5d28008040 comp rx=0 tx=0).stop 2023-11-06T03:06:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 msgr2=0x7f5d2c0b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c500 tx=0x7f5d1c00c9c0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 shutdown_connections 2023-11-06T03:06:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d10065a60 0x7f5d10067f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d2c155310 0x7f5d2c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d2c06c550 0x7f5d2c139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 --2- 172.21.15.157:0/4222564269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d2c06c170 0x7f5d2c0b0f10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 >> 172.21.15.157:0/4222564269 conn(0x7f5d2c0b43b0 msgr2=0x7f5d2c14cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 shutdown_connections 2023-11-06T03:06:51.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.101+0000 7f5d33531640 1 -- 172.21.15.157:0/4222564269 wait complete. 2023-11-06T03:06:51.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-06T03:06:51.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.277+0000 7fd83646e640 1 Processor -- start 2023-11-06T03:06:51.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.277+0000 7fd83646e640 1 -- start start 2023-11-06T03:06:51.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd83646e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd8300b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd83646e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300b2150 con 0x7fd83006b9a0 2023-11-06T03:06:51.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd83646e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd830071270 con 0x7fd8300b2b00 2023-11-06T03:06:51.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd83646e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830071550 con 0x7fd830146490 2023-11-06T03:06:51.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd827fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd83006b9a0 0x7fd8300b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53456/0 (socket says 172.21.15.157:53456) 2023-11-06T03:06:51.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd827fff640 1 -- 172.21.15.157:0/550349991 learned_addr learned my addr 172.21.15.157:0/550349991 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:06:51.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686810548 0 0) 0x7fd8300b2150 con 0x7fd83006b9a0 2023-11-06T03:06:51.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003530 con 0x7fd83006b9a0 2023-11-06T03:06:51.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82ffff640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd8300b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3415496935 0 0) 0x7fd81c003530 con 0x7fd83006b9a0 2023-11-06T03:06:51.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 msgr2=0x7fd8300b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd8300b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 >> v1:172.21.15.195:6789/0 conn(0x7fd8300b2b00 legacy=0x7fd8300ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300aca70 con 0x7fd83006b9a0 2023-11-06T03:06:51.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd818001680 con 0x7fd83006b9a0 2023-11-06T03:06:51.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 >> v1:172.21.15.157:6789/0 conn(0x7fd83006b9a0 legacy=0x7fd8300b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd81c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c004e30 0x7fd81c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830071550 con 0x7fd81c004e30 2023-11-06T03:06:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0054b0 con 0x7fd81c0039b0 2023-11-06T03:06:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c005670 con 0x7fd830146490 2023-11-06T03:06:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c004e30 0x7fd81c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd82ffff640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd827fff640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd81c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 -- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 msgr2=0x7fd81c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd830146490 0x7fd81c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 -- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd81c0039b0 msgr2=0x7fd81c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.281+0000 7fd834a0b640 1 -- 172.21.15.157:0/550349991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c003530 con 0x7fd81c004e30 2023-11-06T03:06:51.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd834a0b640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c004e30 0x7fd81c00b990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd828001e40 tx=0x7fd82801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385824d864562d2d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003b80 con 0x7fd81c004e30 2023-11-06T03:06:51.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828003ce0 con 0x7fd81c004e30 2023-11-06T03:06:51.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd82f7fe640 1 -- 172.21.15.157:0/550349991 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280052c0 con 0x7fd81c004e30 2023-11-06T03:06:51.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 -- 172.21.15.157:0/550349991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c004e30 msgr2=0x7fd81c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 --2- 172.21.15.157:0/550349991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd81c004e30 0x7fd81c00b990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd828001e40 tx=0x7fd82801c740 comp rx=0 tx=0).stop 2023-11-06T03:06:51.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd834a0b640 1 -- 172.21.15.157:0/550349991 reap_dead start 2023-11-06T03:06:51.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 -- 172.21.15.157:0/550349991 shutdown_connections 2023-11-06T03:06:51.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 -- 172.21.15.157:0/550349991 >> 172.21.15.157:0/550349991 conn(0x7fd830065d30 msgr2=0x7fd830066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 -- 172.21.15.157:0/550349991 shutdown_connections 2023-11-06T03:06:51.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.285+0000 7fd83646e640 1 -- 172.21.15.157:0/550349991 wait complete. 2023-11-06T03:06:51.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 Processor -- start 2023-11-06T03:06:51.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 -- start start 2023-11-06T03:06:51.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd81c004e30 0x7fd8301f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 0x7fd8300aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 0x7fd8300ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301f6900 con 0x7fd8300cafc0 2023-11-06T03:06:51.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830141560 con 0x7fd8300cc930 2023-11-06T03:06:51.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd83646e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301f6060 con 0x7fd81c004e30 2023-11-06T03:06:51.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd827fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 0x7fd8300aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd827fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 0x7fd8300aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60892/0 (socket says 172.21.15.157:60892) 2023-11-06T03:06:51.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd827fff640 1 -- 172.21.15.157:0/3251254418 learned_addr learned my addr 172.21.15.157:0/3251254418 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd81c004e30 0x7fd8301f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 0x7fd8300ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 -- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd81c004e30 msgr2=0x7fd8301f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd81c004e30 0x7fd8301f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 -- 172.21.15.157:0/3251254418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 msgr2=0x7fd8300aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 0x7fd8300aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.289+0000 7fd834a0b640 1 -- 172.21.15.157:0/3251254418 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8301f6340 con 0x7fd8300cc930 2023-11-06T03:06:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 0x7fd8300ccde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd828009000 tx=0x7fd81c005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 -- 172.21.15.157:0/3251254418 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005280 con 0x7fd8300cc930 2023-11-06T03:06:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8301f6620 con 0x7fd8300cc930 2023-11-06T03:06:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300af2a0 con 0x7fd8300cc930 2023-11-06T03:06:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 -- 172.21.15.157:0/3251254418 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8280053e0 con 0x7fd8300cc930 2023-11-06T03:06:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 -- 172.21.15.157:0/3251254418 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005750 con 0x7fd8300cc930 2023-11-06T03:06:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 -- 172.21.15.157:0/3251254418 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8280a6440 con 0x7fd8300cc930 2023-11-06T03:06:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 0x7fd810067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82d7fa640 1 -- 172.21.15.157:0/3251254418 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd82811ccf0 con 0x7fd8300cc930 2023-11-06T03:06:51.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 0x7fd810067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd839596000 1 Processor -- start 2023-11-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.293+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 0x7fd810067f50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd830141280 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- start start 2023-11-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091ee96d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ed5ec10 con 0x56091ee97260 2023-11-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091eced3e0 con 0x56091eee1a50 2023-11-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091ecea9b0 con 0x56091eee1130 2023-11-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60900/0 (socket says 172.21.15.157:60900) 2023-11-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 -- 172.21.15.157:0/1977462793 learned_addr learned my addr 172.21.15.157:0/1977462793 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd827fff640 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091ee96d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd82ffff640 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 -- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 msgr2=0x56091eee1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 -- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 msgr2=0x56091ee96d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091ee96d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 -- 172.21.15.157:0/1977462793 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56091e9e2e20 con 0x56091ee97260 2023-11-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd834a0b640 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd8280edf10 tx=0x7fd8280edb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df90584bfcebdfc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd825ffb640 1 -- 172.21.15.157:0/1977462793 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280f61d0 con 0x56091ee97260 2023-11-06T03:06:51.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd825ffb640 1 -- 172.21.15.157:0/1977462793 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8280ec5b0 con 0x56091ee97260 2023-11-06T03:06:51.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd825ffb640 1 -- 172.21.15.157:0/1977462793 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280ea920 con 0x56091ee97260 2023-11-06T03:06:51.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- 172.21.15.157:0/1977462793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 msgr2=0x56091ee97640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd8280edf10 tx=0x7fd8280edb70 comp rx=0 tx=0).stop 2023-11-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- 172.21.15.157:0/1977462793 shutdown_connections 2023-11-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee97640 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091ee96d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 --2- 172.21.15.157:0/1977462793 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- 172.21.15.157:0/1977462793 >> 172.21.15.157:0/1977462793 conn(0x56091ee9d9f0 msgr2=0x56091ee95600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- 172.21.15.157:0/1977462793 shutdown_connections 2023-11-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.297+0000 7fd839596000 1 -- 172.21.15.157:0/1977462793 wait complete. 2023-11-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- start start 2023-11-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 Processor -- start 2023-11-06T03:06:51.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091eea20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ece62a0 con 0x56091ee97260 2023-11-06T03:06:51.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ec85470 con 0x56091eee1a50 2023-11-06T03:06:51.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091eca3660 con 0x56091eee1130 2023-11-06T03:06:51.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 10 client.0 _ll_register_callbacks cb 0x56091ee82c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:51.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60904/0 (socket says 172.21.15.157:60904) 2023-11-06T03:06:51.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 -- 172.21.15.157:0/3398589538 learned_addr learned my addr 172.21.15.157:0/3398589538 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd827fff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091eea20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 -- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 msgr2=0x56091eee87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 -- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 msgr2=0x56091eea20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091eea20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 -- 172.21.15.157:0/3398589538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300af2a0 con 0x56091ee97260 2023-11-06T03:06:51.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd8200146f0 tx=0x7fd820004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b206c0726a70fc66 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820014d60 con 0x56091ee97260 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd82002a400 con 0x56091ee97260 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56091eca30a0 con 0x56091ee97260 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82002a5c0 con 0x56091ee97260 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 10 client.5607 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:51.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56091ecab1d0 con 0x56091ee97260 2023-11-06T03:06:51.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56091eca30a0 con 0x56091ee97260 2023-11-06T03:06:51.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd820028020 con 0x56091ee97260 2023-11-06T03:06:51.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 10 client.5607 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:51.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 10 client.5607 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:51.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 20 client.5607 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:51.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091eea2e90 0x56091eea3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 4 client.5607 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:06:51.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56091ece4bf0 con 0x56091eea2e90 2023-11-06T03:06:51.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.301+0000 7fd827fff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091eea2e90 0x56091eea3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd827fff640 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091eea2e90 0x56091eea3270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe50ba65d2bbea7c server_cookie=450f831ae693c298 in_seq=0 out_seq=0 2023-11-06T03:06:51.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd80effd640 10 client.5607 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd80effd640 1 -- 172.21.15.157:0/3398589538 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56091ece4bf0 con 0x56091eea2e90 2023-11-06T03:06:51.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd80effd640 10 client.5607 handle_command_reply: tid=0 2023-11-06T03:06:51.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 client.5607 shutdown 2023-11-06T03:06:51.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 msgr2=0x56091eee8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd8200146f0 tx=0x7fd820004740 comp rx=0 tx=0).stop 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 shutdown_connections 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091eea2e90 0x56091eea3270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1a50 0x56091eea20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091eee87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 --2- 172.21.15.157:0/3398589538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091eee8280 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 >> 172.21.15.157:0/3398589538 conn(0x56091ee9d9f0 msgr2=0x56091ee95d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd80effd640 0 client.5607 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 shutdown_connections 2023-11-06T03:06:51.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 1 -- 172.21.15.157:0/3398589538 wait complete. 2023-11-06T03:06:51.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.305+0000 7fd839596000 20 client.5607 trim_cache size 0 max 16384 2023-11-06T03:06:51.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.329+0000 7fd839596000 1 Processor -- start 2023-11-06T03:06:51.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.329+0000 7fd839596000 1 -- start start 2023-11-06T03:06:51.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1130 0x56091ee9ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1a50 0x56091eee3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091e9a3aa0 con 0x56091ee97260 2023-11-06T03:06:51.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ece36a0 con 0x56091eee1130 2023-11-06T03:06:51.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091ece0640 con 0x56091eee1a50 2023-11-06T03:06:51.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60920/0 (socket says 172.21.15.157:60920) 2023-11-06T03:06:51.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 -- 172.21.15.157:0/2868022809 learned_addr learned my addr 172.21.15.157:0/2868022809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd834a0b640 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1a50 0x56091eee3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd827fff640 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1130 0x56091ee9ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 -- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1a50 msgr2=0x56091eee3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1a50 0x56091eee3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 -- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1130 msgr2=0x56091ee9ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1130 0x56091ee9ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 -- 172.21.15.157:0/2868022809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56091ec91520 con 0x56091ee97260 2023-11-06T03:06:51.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd82ffff640 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd8200043c0 tx=0x7fd820024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ed4841389434ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd8267fc640 1 -- 172.21.15.157:0/2868022809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000cfa0 con 0x56091ee97260 2023-11-06T03:06:51.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd8267fc640 1 -- 172.21.15.157:0/2868022809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820014bb0 con 0x56091ee97260 2023-11-06T03:06:51.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd8267fc640 1 -- 172.21.15.157:0/2868022809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82002ab30 con 0x56091ee97260 2023-11-06T03:06:51.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- 172.21.15.157:0/2868022809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 msgr2=0x56091ee9c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd8200043c0 tx=0x7fd820024000 comp rx=0 tx=0).stop 2023-11-06T03:06:51.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- 172.21.15.157:0/2868022809 shutdown_connections 2023-11-06T03:06:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1a50 0x56091eee3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091eee1130 0x56091ee9ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 --2- 172.21.15.157:0/2868022809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091ee97260 0x56091ee9c8c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- 172.21.15.157:0/2868022809 >> 172.21.15.157:0/2868022809 conn(0x56091ee9d9f0 msgr2=0x56091eee7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- 172.21.15.157:0/2868022809 shutdown_connections 2023-11-06T03:06:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- 172.21.15.157:0/2868022809 wait complete. 2023-11-06T03:06:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 -- start start 2023-11-06T03:06:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.333+0000 7fd839596000 1 Processor -- start 2023-11-06T03:06:51.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 0x56091ef215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091ef21b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 0x56091ef22060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ed8abc0 con 0x56091eee1a50 2023-11-06T03:06:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091ece5130 con 0x56091ee97260 2023-11-06T03:06:51.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091ece1c10 con 0x56091eee1130 2023-11-06T03:06:51.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 10 client.0 _ll_register_callbacks cb 0x56091ecaacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:06:51.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd827fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091ef21b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 0x56091ef215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd827fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091ef21b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:35200/0 (socket says 172.21.15.157:35200) 2023-11-06T03:06:51.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 0x56091ef215e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:37064/0 (socket says 172.21.15.157:37064) 2023-11-06T03:06:51.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd827fff640 1 -- 172.21.15.157:0/3414252857 learned_addr learned my addr 172.21.15.157:0/3414252857 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:06:51.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 0x56091ef22060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 -- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 msgr2=0x56091ef21b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091ef21b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 -- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 msgr2=0x56091ef215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:06:51.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 0x56091ef215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 -- 172.21.15.157:0/3414252857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56091ecab1d0 con 0x56091eee1a50 2023-11-06T03:06:51.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd834a0b640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 0x56091ef22060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd82811b0a0 tx=0x7fd8280c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5beb61700522572 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:06:51.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:06:51.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828001260 con 0x56091eee1a50 2023-11-06T03:06:51.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56091ecaf620 con 0x56091eee1a50 2023-11-06T03:06:51.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd8280f61d0 con 0x56091eee1a50 2023-11-06T03:06:51.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82810e020 con 0x56091eee1a50 2023-11-06T03:06:51.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 10 client.5613 fetch_fsmap learned FSMap version 27 2023-11-06T03:06:51.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56091ec85090 con 0x56091eee1a50 2023-11-06T03:06:51.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56091ecaf620 con 0x56091eee1a50 2023-11-06T03:06:51.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 27) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd8280af040 con 0x56091eee1a50 2023-11-06T03:06:51.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 10 client.5613 fetch_fsmap finished waiting for FSMap version 27 2023-11-06T03:06:51.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 10 client.5613 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:06:51.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 20 client.5613 populate_metadata read hostname 'smithi157' 2023-11-06T03:06:51.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091ef225a0 0x56091ef22980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:06:51.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 4 client.5613 mds_command: new command op to 5331 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:06:51.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56091ecdf320 con 0x56091ef225a0 2023-11-06T03:06:51.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091ef225a0 0x56091ef22980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:06:51.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 10 client.5613 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd82ffff640 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091ef225a0 0x56091ef22980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c0e5a08145d38f server_cookie=d3cb84e6ddda0983 in_seq=0 out_seq=0 2023-11-06T03:06:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 1 -- 172.21.15.157:0/3414252857 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+275 (crc 0 0 0) 0x56091ecdf320 con 0x56091ef225a0 2023-11-06T03:06:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd80effd640 10 client.5613 handle_command_reply: tid=0 2023-11-06T03:06:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.337+0000 7fd839596000 1 client.5613 shutdown 2023-11-06T03:06:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 msgr2=0x56091ef22060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 0x56091ef22060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd82811b0a0 tx=0x7fd8280c0000 comp rx=0 tx=0).stop 2023-11-06T03:06:51.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 shutdown_connections 2023-11-06T03:06:51.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x56091ef225a0 0x56091ef22980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56091eee1a50 0x56091ef22060 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56091eee1130 0x56091ef21b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 --2- 172.21.15.157:0/3414252857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56091ee97260 0x56091ef215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 >> 172.21.15.157:0/3414252857 conn(0x56091ee9d9f0 msgr2=0x56091eee7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd80effd640 0 client.5613 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:06:51.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 shutdown_connections 2023-11-06T03:06:51.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 1 -- 172.21.15.157:0/3414252857 wait complete. 2023-11-06T03:06:51.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd839596000 20 client.5613 trim_cache size 0 max 16384 2023-11-06T03:06:51.359 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:06:51.359 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "scrub active (1166 inodes in the stack)", 2023-11-06T03:06:51.359 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:06:51.360 INFO:teuthology.orchestra.run.smithi157.stdout: "f9e2d576-5680-452f-b3ad-c8182a29b69c": { 2023-11-06T03:06:51.360 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/", 2023-11-06T03:06:51.360 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "f9e2d576-5680-452f-b3ad-c8182a29b69c", 2023-11-06T03:06:51.360 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive,scrub_mdsdir" 2023-11-06T03:06:51.360 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:51.361 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:06:51.361 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:06:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 msgr2=0x7fd810067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 0x7fd810067f50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd830141280 tx=0x7fd820008040 comp rx=0 tx=0).stop 2023-11-06T03:06:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 msgr2=0x7fd8300ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:06:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 0x7fd8300ccde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd828009000 tx=0x7fd81c005800 comp rx=0 tx=0).stop 2023-11-06T03:06:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 shutdown_connections 2023-11-06T03:06:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd810065a90 0x7fd810067f50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8300cc930 0x7fd8300ccde0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8300cafc0 0x7fd8300aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 --2- 172.21.15.157:0/3251254418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd81c004e30 0x7fd8301f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:06:51.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 >> 172.21.15.157:0/3251254418 conn(0x7fd830065d30 msgr2=0x7fd83006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:06:51.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 shutdown_connections 2023-11-06T03:06:51.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:06:51.341+0000 7fd83646e640 1 -- 172.21.15.157:0/3251254418 wait complete. 2023-11-06T03:06:51.367 INFO:tasks.cephfs.filesystem:scrub status for tag:f9e2d576-5680-452f-b3ad-c8182a29b69c - {'path': '/', 'tag': 'f9e2d576-5680-452f-b3ad-c8182a29b69c', 'options': 'recursive,scrub_mdsdir'} 2023-11-06T03:07:10.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:07:10.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:07:21.368 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-06T03:07:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 Processor -- start 2023-11-06T03:07:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 -- start start 2023-11-06T03:07:21.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 0x7fb15c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb15c0713c0 0x7fb15c0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c06b8d0 con 0x7fb15c0713c0 2023-11-06T03:07:21.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c06be90 con 0x7fb15c15adc0 2023-11-06T03:07:21.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb1642be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c071880 con 0x7fb15c0ca570 2023-11-06T03:07:21.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb15c0713c0 0x7fb15c0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb15c0713c0 0x7fb15c0b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55318/0 (socket says 172.21.15.157:55318) 2023-11-06T03:07:21.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 -- 172.21.15.157:0/3101110797 learned_addr learned my addr 172.21.15.157:0/3101110797 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb16205a640 1 --2- 172.21.15.157:0/3101110797 >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 0x7fb15c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb16285b640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 -- 172.21.15.157:0/3101110797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 msgr2=0x7fb15c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 -- 172.21.15.157:0/3101110797 >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 msgr2=0x7fb15c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 --2- 172.21.15.157:0/3101110797 >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 0x7fb15c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb161859640 1 -- 172.21.15.157:0/3101110797 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c071b60 con 0x7fb15c0713c0 2023-11-06T03:07:21.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb16205a640 1 --2- 172.21.15.157:0/3101110797 >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 0x7fb15c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.505+0000 7fb16285b640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb161859640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c0713c0 0x7fb15c0b4570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0ced80 tx=0x7fb15401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44925b76b74419a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb161058640 1 -- 172.21.15.157:0/3101110797 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540a5720 con 0x7fb15c0713c0 2023-11-06T03:07:21.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb161058640 1 -- 172.21.15.157:0/3101110797 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1540a5880 con 0x7fb15c0713c0 2023-11-06T03:07:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb161058640 1 -- 172.21.15.157:0/3101110797 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540a5c10 con 0x7fb15c0713c0 2023-11-06T03:07:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- 172.21.15.157:0/3101110797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c0713c0 msgr2=0x7fb15c0b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c0713c0 0x7fb15c0b4570 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0ced80 tx=0x7fb15401c880 comp rx=0 tx=0).stop 2023-11-06T03:07:21.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- 172.21.15.157:0/3101110797 shutdown_connections 2023-11-06T03:07:21.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- 172.21.15.157:0/3101110797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c0713c0 0x7fb15c0b4570 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- 172.21.15.157:0/3101110797 >> v2:172.21.15.195:3300/0 conn(0x7fb15c15adc0 0x7fb15c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- 172.21.15.157:0/3101110797 >> 172.21.15.157:0/3101110797 conn(0x7fb15c0cb990 msgr2=0x7fb15c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- 172.21.15.157:0/3101110797 shutdown_connections 2023-11-06T03:07:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- 172.21.15.157:0/3101110797 wait complete. 2023-11-06T03:07:21.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 Processor -- start 2023-11-06T03:07:21.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- start start 2023-11-06T03:07:21.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0713c0 0x7fb15c0c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b10e0 con 0x7fb15c15adc0 2023-11-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c1f7290 con 0x7fb15c0713c0 2023-11-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb1642be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0c8b20 con 0x7fb15c0ca570 2023-11-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55328/0 (socket says 172.21.15.157:55328) 2023-11-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.509+0000 7fb16285b640 1 -- 172.21.15.157:0/850509511 learned_addr learned my addr 172.21.15.157:0/850509511 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16205a640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0713c0 0x7fb15c0c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb161859640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 -- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 msgr2=0x7fb15c0afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 -- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0713c0 msgr2=0x7fb15c0c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0713c0 0x7fb15c0c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 -- 172.21.15.157:0/850509511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb154009b30 con 0x7fb15c15adc0 2023-11-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16285b640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c7f0 tx=0x7fb15000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 -- 172.21.15.157:0/850509511 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150007820 con 0x7fb15c15adc0 2023-11-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 -- 172.21.15.157:0/850509511 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150007980 con 0x7fb15c15adc0 2023-11-06T03:07:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c0b0a20 con 0x7fb15c15adc0 2023-11-06T03:07:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 -- 172.21.15.157:0/850509511 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150007d30 con 0x7fb15c15adc0 2023-11-06T03:07:21.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c0c8de0 con 0x7fb15c15adc0 2023-11-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 -- 172.21.15.157:0/850509511 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb1500113f0 con 0x7fb15c15adc0 2023-11-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 0x7fb140067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb14effd640 1 -- 172.21.15.157:0/850509511 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fb1500835d0 con 0x7fb15c15adc0 2023-11-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16205a640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 0x7fb140067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb1673e6000 1 Processor -- start 2023-11-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb1673e6000 1 -- start start 2023-11-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.513+0000 7fb16205a640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 0x7fb140067f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb158006fd0 tx=0x7fb158008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 0x556279041510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556278ff7260 0x556278ff7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278ebec10 con 0x556279041a50 2023-11-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278e4d3e0 con 0x556278ff7260 2023-11-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556278e4a9b0 con 0x556279041130 2023-11-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55344/0 (socket says 172.21.15.157:55344) 2023-11-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/2681289838 learned_addr learned my addr 172.21.15.157:0/2681289838 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16205a640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 0x556279041510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16285b640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556278ff7260 0x556278ff7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 msgr2=0x556279041510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 0x556279041510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556278ff7260 msgr2=0x556278ff7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556278ff7260 0x556278ff7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/2681289838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556278b42e20 con 0x556279041a50 2023-11-06T03:07:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16205a640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 0x556279041510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb15401d120 tx=0x7fb154004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b51f92ab847555 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb13ffff640 1 -- 172.21.15.157:0/2681289838 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540a5bf0 con 0x556279041a50 2023-11-06T03:07:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb13ffff640 1 -- 172.21.15.157:0/2681289838 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1540a5d50 con 0x556279041a50 2023-11-06T03:07:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb13ffff640 1 -- 172.21.15.157:0/2681289838 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540b6660 con 0x556279041a50 2023-11-06T03:07:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- 172.21.15.157:0/2681289838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 msgr2=0x556278ff6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb15401d120 tx=0x7fb154004ac0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- 172.21.15.157:0/2681289838 shutdown_connections 2023-11-06T03:07:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556278ff7260 0x556278ff7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556278ff6d20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- 172.21.15.157:0/2681289838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556279041130 0x556279041510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- 172.21.15.157:0/2681289838 >> 172.21.15.157:0/2681289838 conn(0x556278ffd9f0 msgr2=0x556278ff5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- 172.21.15.157:0/2681289838 shutdown_connections 2023-11-06T03:07:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- 172.21.15.157:0/2681289838 wait complete. 2023-11-06T03:07:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- start start 2023-11-06T03:07:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 Processor -- start 2023-11-06T03:07:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 0x556279042370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278e462a0 con 0x556279041130 2023-11-06T03:07:21.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278de5470 con 0x556279041a50 2023-11-06T03:07:21.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556278e03660 con 0x556278ff7260 2023-11-06T03:07:21.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb1673e6000 10 client.0 _ll_register_callbacks cb 0x556278fe2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:21.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55352/0 (socket says 172.21.15.157:55352) 2023-11-06T03:07:21.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/1446206722 learned_addr learned my addr 172.21.15.157:0/1446206722 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16285b640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 0x556279042370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16205a640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 msgr2=0x5562790405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 msgr2=0x556279042370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 0x556279042370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb161859640 1 -- 172.21.15.157:0/1446206722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c071b60 con 0x556279041130 2023-11-06T03:07:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16205a640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.517+0000 7fb16285b640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 0x556279042370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb161859640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb1540b9040 tx=0x7fb15401b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86c492b38c23a08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540c7070 con 0x556279041130 2023-11-06T03:07:21.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fb1540b6420 con 0x556279041130 2023-11-06T03:07:21.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556278e030a0 con 0x556279041130 2023-11-06T03:07:21.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb154007000 con 0x556279041130 2023-11-06T03:07:21.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 10 client.5625 fetch_fsmap learned FSMap version 28 2023-11-06T03:07:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556278e0b1d0 con 0x556279041130 2023-11-06T03:07:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556278e030a0 con 0x556279041130 2023-11-06T03:07:21.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb1540c0070 con 0x556279041130 2023-11-06T03:07:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 10 client.5625 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T03:07:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 10 client.5625 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:07:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 20 client.5625 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279042d80 0x556279043160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 4 client.5625 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:07:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556278e44bf0 con 0x556279042d80 2023-11-06T03:07:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb16205a640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279042d80 0x556279043160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 10 client.5625 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb16205a640 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279042d80 0x556279043160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b567623657270e85 server_cookie=9b6095fe56af01fd in_seq=0 out_seq=0 2023-11-06T03:07:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 1 -- 172.21.15.157:0/1446206722 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556278e44bf0 con 0x556279042d80 2023-11-06T03:07:21.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 10 client.5625 handle_command_reply: tid=0 2023-11-06T03:07:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 client.5625 shutdown 2023-11-06T03:07:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 msgr2=0x556279041e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb1540b9040 tx=0x7fb15401b0c0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 shutdown_connections 2023-11-06T03:07:21.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279042d80 0x556279043160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041a50 0x556279042370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb13d7fa640 0 client.5625 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041130 0x556279041e30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1446206722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 >> 172.21.15.157:0/1446206722 conn(0x556278ffd9f0 msgr2=0x556278ff5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 shutdown_connections 2023-11-06T03:07:21.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 1 -- 172.21.15.157:0/1446206722 wait complete. 2023-11-06T03:07:21.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.521+0000 7fb1673e6000 20 client.5625 trim_cache size 0 max 16384 2023-11-06T03:07:21.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 Processor -- start 2023-11-06T03:07:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- start start 2023-11-06T03:07:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278b03aa0 con 0x556279041a50 2023-11-06T03:07:21.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278e43ab0 con 0x556279041130 2023-11-06T03:07:21.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556278ffb700 con 0x556278ff7260 2023-11-06T03:07:21.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55368/0 (socket says 172.21.15.157:55368) 2023-11-06T03:07:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 -- 172.21.15.157:0/1458520736 learned_addr learned my addr 172.21.15.157:0/1458520736 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb161859640 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16205a640 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 -- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 msgr2=0x5562790a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 -- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 msgr2=0x5562790a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 -- 172.21.15.157:0/1458520736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556278df1520 con 0x556279041a50 2023-11-06T03:07:21.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb16285b640 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb15000a520 tx=0x7fb150054600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbbad5a6df7c689 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb14cff9640 1 -- 172.21.15.157:0/1458520736 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15005cb50 con 0x556279041a50 2023-11-06T03:07:21.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb14cff9640 1 -- 172.21.15.157:0/1458520736 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150052f10 con 0x556279041a50 2023-11-06T03:07:21.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb14cff9640 1 -- 172.21.15.157:0/1458520736 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15005c6c0 con 0x556279041a50 2023-11-06T03:07:21.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- 172.21.15.157:0/1458520736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 msgr2=0x5562790a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb15000a520 tx=0x7fb150054600 comp rx=0 tx=0).stop 2023-11-06T03:07:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- 172.21.15.157:0/1458520736 shutdown_connections 2023-11-06T03:07:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x5562790a1b50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 --2- 172.21.15.157:0/1458520736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- 172.21.15.157:0/1458520736 >> 172.21.15.157:0/1458520736 conn(0x556278ffd9f0 msgr2=0x55627904d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- 172.21.15.157:0/1458520736 shutdown_connections 2023-11-06T03:07:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- 172.21.15.157:0/1458520736 wait complete. 2023-11-06T03:07:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 -- start start 2023-11-06T03:07:21.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.537+0000 7fb1673e6000 1 Processor -- start 2023-11-06T03:07:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556279042060 con 0x556279041a50 2023-11-06T03:07:21.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556278ffbcf0 con 0x556279041130 2023-11-06T03:07:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556279042600 con 0x556278ff7260 2023-11-06T03:07:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 10 client.0 _ll_register_callbacks cb 0x556278e0acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55382/0 (socket says 172.21.15.157:55382) 2023-11-06T03:07:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 -- 172.21.15.157:0/160937776 learned_addr learned my addr 172.21.15.157:0/160937776 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16205a640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb161859640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 -- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 msgr2=0x5562790a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 -- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 msgr2=0x5562790a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 -- 172.21.15.157:0/160937776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c0c8de0 con 0x556279041a50 2023-11-06T03:07:21.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16205a640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb161859640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16285b640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb150051070 tx=0x7fb15001c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730e06fb6fa27576 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150055b80 con 0x556279041a50 2023-11-06T03:07:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fb150055ce0 con 0x556279041a50 2023-11-06T03:07:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556279042740 con 0x556279041a50 2023-11-06T03:07:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb150055ea0 con 0x556279041a50 2023-11-06T03:07:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 10 client.5631 fetch_fsmap learned FSMap version 28 2023-11-06T03:07:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556278de5090 con 0x556279041a50 2023-11-06T03:07:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556279042740 con 0x556279041a50 2023-11-06T03:07:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb15006d070 con 0x556279041a50 2023-11-06T03:07:21.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 10 client.5631 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T03:07:21.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 10 client.5631 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:07:21.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 20 client.5631 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:21.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279053b50 0x556279053f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 4 client.5631 mds_command: new command op to 5331 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:07:21.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556278e0f620 con 0x556279053b50 2023-11-06T03:07:21.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16205a640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279053b50 0x556279053f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb16205a640 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279053b50 0x556279053f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b84ea05182deafa server_cookie=a7de61b7524e2dd in_seq=0 out_seq=0 2023-11-06T03:07:21.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 10 client.5631 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 1 -- 172.21.15.157:0/160937776 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556278e0f620 con 0x556279053b50 2023-11-06T03:07:21.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 10 client.5631 handle_command_reply: tid=0 2023-11-06T03:07:21.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 client.5631 shutdown 2023-11-06T03:07:21.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 msgr2=0x556279053610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb150051070 tx=0x7fb15001c070 comp rx=0 tx=0).stop 2023-11-06T03:07:21.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 shutdown_connections 2023-11-06T03:07:21.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x556279053b50 0x556279053f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556279041a50 0x556279053610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556279041130 0x5562790a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb13d7fa640 0 client.5631 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 --2- 172.21.15.157:0/160937776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556278ff7260 0x5562790a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 >> 172.21.15.157:0/160937776 conn(0x556278ffd9f0 msgr2=0x55627904da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 shutdown_connections 2023-11-06T03:07:21.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 1 -- 172.21.15.157:0/160937776 wait complete. 2023-11-06T03:07:21.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1673e6000 20 client.5631 trim_cache size 0 max 16384 2023-11-06T03:07:21.603 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:07:21.604 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:07:21.604 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:07:21.604 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:07:21.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 msgr2=0x7fb140067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 0x7fb140067f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb158006fd0 tx=0x7fb158008040 comp rx=0 tx=0).stop 2023-11-06T03:07:21.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 msgr2=0x7fb15c0b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb15000c7f0 tx=0x7fb15000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 shutdown_connections 2023-11-06T03:07:21.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb140065a60 0x7fb140067f20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb15c15adc0 0x7fb15c0b03b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0ca570 0x7fb15c0afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 --2- 172.21.15.157:0/850509511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0713c0 0x7fb15c0c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 >> 172.21.15.157:0/850509511 conn(0x7fb15c0cb990 msgr2=0x7fb15c0c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 shutdown_connections 2023-11-06T03:07:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.541+0000 7fb1642be640 1 -- 172.21.15.157:0/850509511 wait complete. 2023-11-06T03:07:21.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:07:21.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2023-11-06T03:07:21.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.733+0000 7f9d48ba0640 1 Processor -- start 2023-11-06T03:07:21.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.733+0000 7f9d48ba0640 1 -- start start 2023-11-06T03:07:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d440b2330 0x7f9d4406c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4406c490 con 0x7f9d44156a70 2023-11-06T03:07:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4406fb80 con 0x7f9d440adae0 2023-11-06T03:07:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4406b950 con 0x7f9d440b2330 2023-11-06T03:07:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d42d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d440b2330 0x7f9d4406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9d44156a70 0x7f9d4406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57794/0 (socket says 172.21.15.157:57794) 2023-11-06T03:07:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4259c640 1 -- 172.21.15.157:0/3562843362 learned_addr learned my addr 172.21.15.157:0/3562843362 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3562843362 >> v1:172.21.15.195:6789/0 conn(0x7f9d440adae0 legacy=0x7f9d440b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3562843362 >> v1:172.21.15.157:6789/0 conn(0x7f9d44156a70 legacy=0x7f9d4406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4159a640 1 -- 172.21.15.157:0/3562843362 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238369364 0 0) 0x7f9d4406c490 con 0x7f9d44156a70 2023-11-06T03:07:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3562843362 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4406bb60 con 0x7f9d440b2330 2023-11-06T03:07:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/3562843362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d440b2330 0x7f9d4406c920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009d90 tx=0x7f9d3c01da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10d33562f1bd05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4159a640 1 -- 172.21.15.157:0/3562843362 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0a6a20 con 0x7f9d440b2330 2023-11-06T03:07:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4159a640 1 -- 172.21.15.157:0/3562843362 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0a6b80 con 0x7f9d440b2330 2023-11-06T03:07:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d4159a640 1 -- 172.21.15.157:0/3562843362 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0ae440 con 0x7f9d440b2330 2023-11-06T03:07:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- 172.21.15.157:0/3562843362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d440b2330 msgr2=0x7f9d4406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/3562843362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d440b2330 0x7f9d4406c920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009d90 tx=0x7f9d3c01da30 comp rx=0 tx=0).stop 2023-11-06T03:07:21.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- 172.21.15.157:0/3562843362 shutdown_connections 2023-11-06T03:07:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/3562843362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d440b2330 0x7f9d4406c920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- 172.21.15.157:0/3562843362 >> 172.21.15.157:0/3562843362 conn(0x7f9d44065930 msgr2=0x7f9d44065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- 172.21.15.157:0/3562843362 shutdown_connections 2023-11-06T03:07:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.737+0000 7f9d48ba0640 1 -- 172.21.15.157:0/3562843362 wait complete. 2023-11-06T03:07:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 Processor -- start 2023-11-06T03:07:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- start start 2023-11-06T03:07:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d440adae0 0x7f9d440b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d440b2330 0x7f9d44149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4414c370 con 0x7f9d44156a70 2023-11-06T03:07:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4414c100 con 0x7f9d440b2330 2023-11-06T03:07:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440b0550 con 0x7f9d440adae0 2023-11-06T03:07:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55406/0 (socket says 172.21.15.157:55406) 2023-11-06T03:07:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 -- 172.21.15.157:0/397731482 learned_addr learned my addr 172.21.15.157:0/397731482 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d440b2330 0x7f9d44149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d4259c640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d440adae0 0x7f9d440b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 -- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d440adae0 msgr2=0x7f9d440b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d440adae0 0x7f9d440b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 -- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d440b2330 msgr2=0x7f9d44149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d440b2330 0x7f9d44149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 -- 172.21.15.157:0/397731482 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009a70 con 0x7f9d44156a70 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0b1040 tx=0x7f9d3c01a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d3affd640 1 -- 172.21.15.157:0/397731482 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0aee40 con 0x7f9d44156a70 2023-11-06T03:07:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d3affd640 1 -- 172.21.15.157:0/397731482 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0ba450 con 0x7f9d44156a70 2023-11-06T03:07:21.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d3affd640 1 -- 172.21.15.157:0/397731482 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0ba800 con 0x7f9d44156a70 2023-11-06T03:07:21.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d440b0830 con 0x7f9d44156a70 2023-11-06T03:07:21.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.741+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441497d0 con 0x7f9d44156a70 2023-11-06T03:07:21.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d3affd640 1 -- 172.21.15.157:0/397731482 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d3c0ba960 con 0x7f9d44156a70 2023-11-06T03:07:21.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d3affd640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d3affd640 1 -- 172.21.15.157:0/397731482 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f9d3c11fca0 con 0x7f9d44156a70 2023-11-06T03:07:21.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4259c640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 Processor -- start 2023-11-06T03:07:21.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 -- start start 2023-11-06T03:07:21.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4259c640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 0x7f9d2c067f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d30000f70 tx=0x7f9d30007bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6236356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e6235eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6234b0bb0 con 0x55e6235eb410 2023-11-06T03:07:21.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62343f380 con 0x55e623635300 2023-11-06T03:07:21.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e62343c950 con 0x55e623635c20 2023-11-06T03:07:21.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55418/0 (socket says 172.21.15.157:55418) 2023-11-06T03:07:21.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 -- 172.21.15.157:0/2260064301 learned_addr learned my addr 172.21.15.157:0/2260064301 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4259c640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6236356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 -- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 msgr2=0x55e6235eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e6235eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e6235eaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 -- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 msgr2=0x55e6236356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6236356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 -- 172.21.15.157:0/2260064301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e623134dc0 con 0x55e6235eb410 2023-11-06T03:07:21.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d4259c640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6236356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.745+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c01d4a0 tx=0x7f9d3c0f0fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810c375e0ddf6b16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d27fff640 1 -- 172.21.15.157:0/2260064301 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0f5180 con 0x55e6235eb410 2023-11-06T03:07:21.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d27fff640 1 -- 172.21.15.157:0/2260064301 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0f9330 con 0x55e6235eb410 2023-11-06T03:07:21.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d27fff640 1 -- 172.21.15.157:0/2260064301 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0eda60 con 0x55e6235eb410 2023-11-06T03:07:21.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/2260064301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 msgr2=0x55e6235eb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c01d4a0 tx=0x7f9d3c0f0fc0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/2260064301 shutdown_connections 2023-11-06T03:07:21.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6235eb410 0x55e6235eb7f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e6235eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/2260064301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6236356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/2260064301 >> 172.21.15.157:0/2260064301 conn(0x55e6235ef990 msgr2=0x55e6235e75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/2260064301 shutdown_connections 2023-11-06T03:07:21.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/2260064301 wait complete. 2023-11-06T03:07:21.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- start start 2023-11-06T03:07:21.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 Processor -- start 2023-11-06T03:07:21.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e62363c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 0x55e62363ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e623438240 con 0x55e623635c20 2023-11-06T03:07:21.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6233d7410 con 0x55e6235eb410 2023-11-06T03:07:21.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6233f5600 con 0x55e623635300 2023-11-06T03:07:21.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 10 client.0 _ll_register_callbacks cb 0x55e6235d4bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:21.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55424/0 (socket says 172.21.15.157:55424) 2023-11-06T03:07:21.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3779835465 learned_addr learned my addr 172.21.15.157:0/3779835465 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4259c640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e62363c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 0x55e62363ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 msgr2=0x55e62363ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 0x55e62363ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 msgr2=0x55e62363c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e62363c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 -- 172.21.15.157:0/3779835465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441497d0 con 0x55e623635c20 2023-11-06T03:07:21.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 0x55e62363ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0f1510 tx=0x7f9d3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71ee5c7de86dee7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:21.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0f9330 con 0x55e623635c20 2023-11-06T03:07:21.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f9d3c0f7050 con 0x55e623635c20 2023-11-06T03:07:21.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6233f5040 con 0x55e623635c20 2023-11-06T03:07:21.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d3c0f7210 con 0x55e623635c20 2023-11-06T03:07:21.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 10 client.5646 fetch_fsmap learned FSMap version 28 2023-11-06T03:07:21.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55e6233fd170 con 0x55e623635c20 2023-11-06T03:07:21.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6233f5040 con 0x55e623635c20 2023-11-06T03:07:21.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9d3c005d20 con 0x55e623635c20 2023-11-06T03:07:21.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 10 client.5646 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T03:07:21.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 10 client.5646 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:07:21.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 20 client.5646 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:21.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62361fcd0 0x55e6236200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 4 client.5646 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:07:21.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e623436b90 con 0x55e62361fcd0 2023-11-06T03:07:21.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.749+0000 7f9d4259c640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62361fcd0 0x55e6236200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d257fa640 10 client.5646 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4259c640 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62361fcd0 0x55e6236200b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272743a768f8f061 server_cookie=144587ad70885dc in_seq=0 out_seq=0 2023-11-06T03:07:21.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d257fa640 1 -- 172.21.15.157:0/3779835465 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e623436b90 con 0x55e62361fcd0 2023-11-06T03:07:21.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d257fa640 10 client.5646 handle_command_reply: tid=0 2023-11-06T03:07:21.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 client.5646 shutdown 2023-11-06T03:07:21.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 msgr2=0x55e62361f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0f1510 tx=0x7f9d3c009750 comp rx=0 tx=0).stop 2023-11-06T03:07:21.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 shutdown_connections 2023-11-06T03:07:21.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62361fcd0 0x55e6236200b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e62361f2c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635300 0x55e62363ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d257fa640 0 client.5646 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/3779835465 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e62363c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 >> 172.21.15.157:0/3779835465 conn(0x55e6235ef990 msgr2=0x55e6235ee790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 shutdown_connections 2023-11-06T03:07:21.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/3779835465 wait complete. 2023-11-06T03:07:21.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.753+0000 7f9d4bcc8000 20 client.5646 trim_cache size 0 max 16384 2023-11-06T03:07:21.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 Processor -- start 2023-11-06T03:07:21.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 -- start start 2023-11-06T03:07:21.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6235eb410 0x55e6235ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6235edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6235e7710 con 0x55e623635c20 2023-11-06T03:07:21.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6230f5a40 con 0x55e623635300 2023-11-06T03:07:21.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e62343ada0 con 0x55e6235eb410 2023-11-06T03:07:21.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55440/0 (socket says 172.21.15.157:55440) 2023-11-06T03:07:21.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 -- 172.21.15.157:0/710163527 learned_addr learned my addr 172.21.15.157:0/710163527 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6235edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 -- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6235eb410 msgr2=0x55e6235ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6235eb410 0x55e6235ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 -- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 msgr2=0x55e6235edd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6235edd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 -- 172.21.15.157:0/710163527 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6233e34c0 con 0x55e623635c20 2023-11-06T03:07:21.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6235edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.765+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0ef6a0 tx=0x7f9d3c0fc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3818cd3ceb7439 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d38ff9640 1 -- 172.21.15.157:0/710163527 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0f0e90 con 0x55e623635c20 2023-11-06T03:07:21.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d38ff9640 1 -- 172.21.15.157:0/710163527 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c0f5180 con 0x55e623635c20 2023-11-06T03:07:21.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d38ff9640 1 -- 172.21.15.157:0/710163527 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0f9330 con 0x55e623635c20 2023-11-06T03:07:21.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/710163527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 msgr2=0x55e623638010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0ef6a0 tx=0x7f9d3c0fc000 comp rx=0 tx=0).stop 2023-11-06T03:07:21.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/710163527 shutdown_connections 2023-11-06T03:07:21.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635c20 0x55e623638010 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e623635300 0x55e6235edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/710163527 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6235eb410 0x55e6235ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/710163527 >> 172.21.15.157:0/710163527 conn(0x55e6235ef990 msgr2=0x55e623642ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/710163527 shutdown_connections 2023-11-06T03:07:21.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/710163527 wait complete. 2023-11-06T03:07:21.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- start start 2023-11-06T03:07:21.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 Processor -- start 2023-11-06T03:07:21.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e6236398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e623678090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62343c240 con 0x55e623635300 2023-11-06T03:07:21.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6235e75d0 con 0x55e6235eb410 2023-11-06T03:07:21.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e623433bb0 con 0x55e623635c20 2023-11-06T03:07:21.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 10 client.0 _ll_register_callbacks cb 0x55e6233fcc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:21.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55442/0 (socket says 172.21.15.157:55442) 2023-11-06T03:07:21.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 -- 172.21.15.157:0/698228479 learned_addr learned my addr 172.21.15.157:0/698228479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:21.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4259c640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e6236398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d42d9d640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e623678090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 -- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 msgr2=0x55e623678090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e623678090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 -- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 msgr2=0x55e6236398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e6236398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 -- 172.21.15.157:0/698228479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d441497d0 con 0x55e623635300 2023-11-06T03:07:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4259c640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e6236398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d41d9b640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d34007bb0 tx=0x7f9d34001500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40813934c5d3b0f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:21.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:21.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d34019030 con 0x55e623635300 2023-11-06T03:07:21.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f9d3401a070 con 0x55e623635300 2023-11-06T03:07:21.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6234015c0 con 0x55e623635300 2023-11-06T03:07:21.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d34001d60 con 0x55e623635300 2023-11-06T03:07:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 10 client.5652 fetch_fsmap learned FSMap version 28 2023-11-06T03:07:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55e6233d7030 con 0x55e623635300 2023-11-06T03:07:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6234015c0 con 0x55e623635300 2023-11-06T03:07:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 28) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9d34012040 con 0x55e623635300 2023-11-06T03:07:21.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 10 client.5652 fetch_fsmap finished waiting for FSMap version 28 2023-11-06T03:07:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 10 client.5652 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-06T03:07:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 20 client.5652 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62368dd80 0x55e62368e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 4 client.5652 mds_command: new command op to 5331 tid=0[{"prefix": "perf dump"}] 2023-11-06T03:07:21.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 --> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55e6234312c0 con 0x55e62368dd80 2023-11-06T03:07:21.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.769+0000 7f9d4259c640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62368dd80 0x55e62368e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:21.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d257fa640 10 client.5652 ms_handle_connect on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4259c640 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62368dd80 0x55e62368e160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc6b18f3e4e90cf server_cookie=5f43b808b0c135eb in_seq=0 out_seq=0 2023-11-06T03:07:21.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d257fa640 1 -- 172.21.15.157:0/698228479 <== mds.0 v2:172.21.15.157:6833/4193594648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17911 (crc 0 0 0) 0x55e6234312c0 con 0x55e62368dd80 2023-11-06T03:07:21.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d257fa640 10 client.5652 handle_command_reply: tid=0 2023-11-06T03:07:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 client.5652 shutdown 2023-11-06T03:07:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 msgr2=0x55e623677b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d34007bb0 tx=0x7f9d34001500 comp rx=0 tx=0).stop 2023-11-06T03:07:21.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 shutdown_connections 2023-11-06T03:07:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:6833/4193594648,v1:172.21.15.157:6835/4193594648] conn(0x55e62368dd80 0x55e62368e160 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e623635c20 0x55e623678090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e623635300 0x55e623677b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 --2- 172.21.15.157:0/698228479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6235eb410 0x55e6236398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 >> 172.21.15.157:0/698228479 conn(0x55e6235ef990 msgr2=0x55e6236349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d257fa640 0 client.5652 ms_handle_reset on v2:172.21.15.157:6833/4193594648 2023-11-06T03:07:21.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 shutdown_connections 2023-11-06T03:07:21.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 1 -- 172.21.15.157:0/698228479 wait complete. 2023-11-06T03:07:21.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d4bcc8000 20 client.5652 trim_cache size 0 max 16384 2023-11-06T03:07:21.803 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:07:21.804 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T03:07:21.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 7764, 2023-11-06T03:07:21.804 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 7763, 2023-11-06T03:07:21.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2209662, 2023-11-06T03:07:21.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 50225352, 2023-11-06T03:07:21.805 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 17, 2023-11-06T03:07:21.806 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2023-11-06T03:07:21.806 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.773528149, 2023-11-06T03:07:21.806 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.371337326, 2023-11-06T03:07:21.807 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.144516626, 2023-11-06T03:07:21.807 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.210472304, 2023-11-06T03:07:21.807 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:07:21.808 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7762, 2023-11-06T03:07:21.808 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.163165924, 2023-11-06T03:07:21.808 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001180516 2023-11-06T03:07:21.808 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.809 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:07:21.809 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10, 2023-11-06T03:07:21.809 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000001333, 2023-11-06T03:07:21.809 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000133 2023-11-06T03:07:21.809 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.810 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1568, 2023-11-06T03:07:21.810 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2023-11-06T03:07:21.810 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.810 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 6513, 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 6494, 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2007805, 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 19737104, 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2023-11-06T03:07:21.811 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:07:21.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.495136573, 2023-11-06T03:07:21.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.187863838, 2023-11-06T03:07:21.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.114429708, 2023-11-06T03:07:21.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.158629887, 2023-11-06T03:07:21.812 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:07:21.813 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6494, 2023-11-06T03:07:21.813 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.115840606, 2023-11-06T03:07:21.813 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000479802 2023-11-06T03:07:21.813 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.813 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:07:21.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.814 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.814 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.814 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.815 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 135696, 2023-11-06T03:07:21.815 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 32064 2023-11-06T03:07:21.815 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.815 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T03:07:21.815 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 36228, 2023-11-06T03:07:21.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 62340, 2023-11-06T03:07:21.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 13241238, 2023-11-06T03:07:21.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 60726523, 2023-11-06T03:07:21.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-11-06T03:07:21.816 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:07:21.817 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 4.799021980, 2023-11-06T03:07:21.817 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.374075518, 2023-11-06T03:07:21.817 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 2.732416815, 2023-11-06T03:07:21.817 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.204327908, 2023-11-06T03:07:21.817 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:07:21.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 62338, 2023-11-06T03:07:21.818 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.985746876, 2023-11-06T03:07:21.818 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000063937 2023-11-06T03:07:21.818 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.818 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:07:21.819 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 87229, 2023-11-06T03:07:21.819 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.019697871, 2023-11-06T03:07:21.819 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000225 2023-11-06T03:07:21.819 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.820 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T03:07:21.820 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 129488 2023-11-06T03:07:21.820 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.820 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-11-06T03:07:21.820 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-11-06T03:07:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-11-06T03:07:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-11-06T03:07:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:07:21.821 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:07:21.822 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 9976, 2023-11-06T03:07:21.822 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.959093385, 2023-11-06T03:07:21.822 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002602154 2023-11-06T03:07:21.822 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.823 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-11-06T03:07:21.823 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:07:21.823 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:07:21.823 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3619, 2023-11-06T03:07:21.823 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.857926505, 2023-11-06T03:07:21.824 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000513381 2023-11-06T03:07:21.824 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.824 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.824 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-11-06T03:07:21.824 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 28522, 2023-11-06T03:07:21.825 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 28522, 2023-11-06T03:07:21.825 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-11-06T03:07:21.825 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28522, 2023-11-06T03:07:21.825 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 786.287741089, 2023-11-06T03:07:21.825 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.027567763 2023-11-06T03:07:21.826 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.826 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-11-06T03:07:21.826 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-11-06T03:07:21.826 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-11-06T03:07:21.826 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 68, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 15839, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 11060, 2023-11-06T03:07:21.827 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 4779, 2023-11-06T03:07:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2023-11-06T03:07:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 17, 2023-11-06T03:07:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2023-11-06T03:07:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 2, 2023-11-06T03:07:21.828 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 2, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 56059, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 54607, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-11-06T03:07:21.829 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 1130, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 16027, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-11-06T03:07:21.830 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 2013, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-11-06T03:07:21.831 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2023-11-06T03:07:21.832 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2023-11-06T03:07:21.832 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2023-11-06T03:07:21.832 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 2013, 2023-11-06T03:07:21.832 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-11-06T03:07:21.832 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 13, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 2, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 11, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 2000, 2023-11-06T03:07:21.833 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-11-06T03:07:21.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 7867, 2023-11-06T03:07:21.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 6240, 2023-11-06T03:07:21.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T03:07:21.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T03:07:21.834 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 7867, 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 2000, 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 46, 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 17980 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.835 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-11-06T03:07:21.836 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2023-11-06T03:07:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-11-06T03:07:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-11-06T03:07:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 4157, 2023-11-06T03:07:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 3666, 2023-11-06T03:07:21.837 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 4384, 2023-11-06T03:07:21.838 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-11-06T03:07:21.838 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-11-06T03:07:21.838 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-11-06T03:07:21.838 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-11-06T03:07:21.838 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-11-06T03:07:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-11-06T03:07:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 1, 2023-11-06T03:07:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-11-06T03:07:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-11-06T03:07:21.839 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-11-06T03:07:21.840 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-11-06T03:07:21.840 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-11-06T03:07:21.840 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.840 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-11-06T03:07:21.840 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 25784, 2023-11-06T03:07:21.841 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 25781, 2023-11-06T03:07:21.841 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 25781, 2023-11-06T03:07:21.841 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 2051, 2023-11-06T03:07:21.841 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-11-06T03:07:21.841 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 2048, 2023-11-06T03:07:21.842 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 32, 2023-11-06T03:07:21.842 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 31, 2023-11-06T03:07:21.842 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 31, 2023-11-06T03:07:21.842 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 3, 2023-11-06T03:07:21.842 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 2, 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 85282427, 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 85283543, 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-11-06T03:07:21.843 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6935, 2023-11-06T03:07:21.844 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 144.104199311, 2023-11-06T03:07:21.844 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.020779264 2023-11-06T03:07:21.844 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.844 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2023-11-06T03:07:21.844 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.845 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-11-06T03:07:21.845 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 14, 2023-11-06T03:07:21.845 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2016, 2023-11-06T03:07:21.845 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2002, 2023-11-06T03:07:21.845 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 13, 2023-11-06T03:07:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 13, 2023-11-06T03:07:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2023-11-06T03:07:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 15839, 2023-11-06T03:07:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 19995, 2023-11-06T03:07:21.846 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 4156, 2023-11-06T03:07:21.847 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 2, 2023-11-06T03:07:21.847 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2002, 2023-11-06T03:07:21.847 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 2000, 2023-11-06T03:07:21.847 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 94396, 2023-11-06T03:07:21.847 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207144 2023-11-06T03:07:21.848 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.848 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-11-06T03:07:21.848 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 42489, 2023-11-06T03:07:21.848 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-11-06T03:07:21.848 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 28522, 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 8, 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2000, 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.454271185, 2023-11-06T03:07:21.849 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000727135 2023-11-06T03:07:21.850 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.850 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-11-06T03:07:21.850 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 187, 2023-11-06T03:07:21.850 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.060657438, 2023-11-06T03:07:21.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000324371 2023-11-06T03:07:21.851 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.851 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-11-06T03:07:21.851 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.851 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.852 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.852 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-11-06T03:07:21.852 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6000, 2023-11-06T03:07:21.852 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.698859712, 2023-11-06T03:07:21.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001616476 2023-11-06T03:07:21.853 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.853 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-11-06T03:07:21.853 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7938, 2023-11-06T03:07:21.854 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 14.077108772, 2023-11-06T03:07:21.854 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001773382 2023-11-06T03:07:21.854 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.854 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-11-06T03:07:21.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.855 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.855 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.855 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.855 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-11-06T03:07:21.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.856 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.856 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.856 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.857 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-11-06T03:07:21.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.858 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.858 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.858 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.859 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-11-06T03:07:21.859 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2023-11-06T03:07:21.859 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.021098310, 2023-11-06T03:07:21.859 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007032770 2023-11-06T03:07:21.860 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.860 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-11-06T03:07:21.860 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.860 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-11-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:07:21.862 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.004238638, 2023-11-06T03:07:21.862 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.004238638 2023-11-06T03:07:21.862 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.862 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-11-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-11-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.010023467, 2023-11-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.010023467 2023-11-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-11-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.866 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.866 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-11-06T03:07:21.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6, 2023-11-06T03:07:21.867 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.219465638, 2023-11-06T03:07:21.867 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.036577606 2023-11-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-11-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4384, 2023-11-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 730.913860502, 2023-11-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.166723052 2023-11-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-11-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.871 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-11-06T03:07:21.871 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.872 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.872 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.872 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.872 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-11-06T03:07:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.873 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-11-06T03:07:21.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:07:21.874 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.007054149, 2023-11-06T03:07:21.874 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.007054149 2023-11-06T03:07:21.874 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.874 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-11-06T03:07:21.875 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.875 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.875 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.875 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.875 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-11-06T03:07:21.876 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:07:21.876 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000563313, 2023-11-06T03:07:21.876 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000563313 2023-11-06T03:07:21.876 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.876 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-11-06T03:07:21.877 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.877 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.877 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.877 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.877 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-11-06T03:07:21.878 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.878 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.878 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.878 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.879 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-11-06T03:07:21.879 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.879 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.879 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.879 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.880 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-11-06T03:07:21.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.880 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.880 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.880 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.881 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-11-06T03:07:21.881 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.882 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.882 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.882 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.883 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-11-06T03:07:21.883 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8000, 2023-11-06T03:07:21.883 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.820539965, 2023-11-06T03:07:21.884 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003727567 2023-11-06T03:07:21.884 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.884 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-11-06T03:07:21.885 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-11-06T03:07:21.885 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-11-06T03:07:21.885 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.886 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.886 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.886 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.886 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.887 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-11-06T03:07:21.887 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2023-11-06T03:07:21.887 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2023-11-06T03:07:21.887 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2023-11-06T03:07:21.887 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2023-11-06T03:07:21.888 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-11-06T03:07:21.888 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 15199, 2023-11-06T03:07:21.888 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 15199, 2023-11-06T03:07:21.888 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 112, 2023-11-06T03:07:21.888 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2023-11-06T03:07:21.889 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.889 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-11-06T03:07:21.889 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-11-06T03:07:21.889 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-11-06T03:07:21.889 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T03:07:21.890 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-11-06T03:07:21.890 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T03:07:21.890 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-11-06T03:07:21.890 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T03:07:21.891 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T03:07:21.891 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T03:07:21.891 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T03:07:21.892 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T03:07:21.892 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-11-06T03:07:21.892 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T03:07:21.893 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T03:07:21.893 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-11-06T03:07:21.893 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-11-06T03:07:21.894 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-11-06T03:07:21.894 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-11-06T03:07:21.895 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T03:07:21.895 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T03:07:21.895 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T03:07:21.895 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T03:07:21.896 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T03:07:21.896 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-11-06T03:07:21.896 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-11-06T03:07:21.896 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-11-06T03:07:21.896 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T03:07:21.897 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T03:07:21.897 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-11-06T03:07:21.897 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-11-06T03:07:21.897 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-11-06T03:07:21.897 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-11-06T03:07:21.898 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T03:07:21.898 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-11-06T03:07:21.898 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T03:07:21.898 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-11-06T03:07:21.898 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 251481, 2023-11-06T03:07:21.899 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 86, 2023-11-06T03:07:21.899 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-11-06T03:07:21.899 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-11-06T03:07:21.899 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-11-06T03:07:21.899 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-11-06T03:07:21.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-11-06T03:07:21.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-11-06T03:07:21.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-11-06T03:07:21.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-11-06T03:07:21.900 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 6376, 2023-11-06T03:07:21.901 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 82, 2023-11-06T03:07:21.901 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T03:07:21.901 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-11-06T03:07:21.901 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-11-06T03:07:21.901 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-11-06T03:07:21.902 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 11826414, 2023-11-06T03:07:21.902 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 643354, 2023-11-06T03:07:21.902 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-11-06T03:07:21.902 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-11-06T03:07:21.902 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-11-06T03:07:21.903 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-11-06T03:07:21.903 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.903 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-11-06T03:07:21.903 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 18203, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 93072796, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 37, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 18203, 2023-11-06T03:07:21.904 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-11-06T03:07:21.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 18203, 2023-11-06T03:07:21.905 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 386.837479028, 2023-11-06T03:07:21.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.021251303 2023-11-06T03:07:21.905 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.905 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-11-06T03:07:21.906 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-11-06T03:07:21.906 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 18203, 2023-11-06T03:07:21.906 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 28643 2023-11-06T03:07:21.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.906 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 18203, 2023-11-06T03:07:21.907 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2123, 2023-11-06T03:07:21.907 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 16080, 2023-11-06T03:07:21.907 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-11-06T03:07:21.907 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-11-06T03:07:21.907 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 55, 2023-11-06T03:07:21.908 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 5134, 2023-11-06T03:07:21.908 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 108, 2023-11-06T03:07:21.908 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 7041, 2023-11-06T03:07:21.909 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 85, 2023-11-06T03:07:21.909 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-11-06T03:07:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-11-06T03:07:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 1666, 2023-11-06T03:07:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-11-06T03:07:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 2050, 2023-11-06T03:07:21.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-11-06T03:07:21.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-11-06T03:07:21.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-11-06T03:07:21.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2015, 2023-11-06T03:07:21.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 10268, 2023-11-06T03:07:21.911 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-11-06T03:07:21.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T03:07:21.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-11-06T03:07:21.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-11-06T03:07:21.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-11-06T03:07:21.913 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-11-06T03:07:21.913 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-11-06T03:07:21.914 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-11-06T03:07:21.914 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-11-06T03:07:21.914 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-11-06T03:07:21.914 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-11-06T03:07:21.914 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-11-06T03:07:21.915 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-11-06T03:07:21.915 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-11-06T03:07:21.915 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-11-06T03:07:21.915 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-11-06T03:07:21.915 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-11-06T03:07:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-11-06T03:07:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-11-06T03:07:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-11-06T03:07:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-11-06T03:07:21.917 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 41, 2023-11-06T03:07:21.917 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-11-06T03:07:21.917 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 5, 2023-11-06T03:07:21.917 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 8, 2023-11-06T03:07:21.918 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 8, 2023-11-06T03:07:21.918 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-11-06T03:07:21.918 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-11-06T03:07:21.918 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 170, 2023-11-06T03:07:21.919 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2023-11-06T03:07:21.919 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 51 2023-11-06T03:07:21.919 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.919 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-11-06T03:07:21.920 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2023-11-06T03:07:21.920 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-11-06T03:07:21.920 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-11-06T03:07:21.920 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-11-06T03:07:21.920 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-11-06T03:07:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 28525 2023-11-06T03:07:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-11-06T03:07:21.921 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-11-06T03:07:21.922 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 32, 2023-11-06T03:07:21.922 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-11-06T03:07:21.922 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 32, 2023-11-06T03:07:21.922 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 3666, 2023-11-06T03:07:21.922 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-11-06T03:07:21.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.923 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T03:07:21.923 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:07:21.923 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:07:21.923 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:07:21.924 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 50500, 2023-11-06T03:07:21.924 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 12906785, 2023-11-06T03:07:21.924 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:07:21.924 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 50500, 2023-11-06T03:07:21.924 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:07:21.925 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:07:21.925 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 50500, 2023-11-06T03:07:21.925 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 12906785, 2023-11-06T03:07:21.925 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:07:21.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.926 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:07:21.927 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:07:21.928 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:07:21.928 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 18203, 2023-11-06T03:07:21.928 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 93456390, 2023-11-06T03:07:21.928 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 18203, 2023-11-06T03:07:21.928 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 93456390, 2023-11-06T03:07:21.929 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:07:21.929 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.929 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.930 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.930 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.930 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.931 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-11-06T03:07:21.931 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:07:21.931 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-11-06T03:07:21.932 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:07:21.932 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:07:21.932 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:07:21.933 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:07:21.933 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:07:21.933 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 18203, 2023-11-06T03:07:21.934 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 18203, 2023-11-06T03:07:21.934 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 18203, 2023-11-06T03:07:21.935 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 18203, 2023-11-06T03:07:21.935 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:07:21.935 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.936 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.936 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.936 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.937 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-11-06T03:07:21.937 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:07:21.938 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:07:21.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:07:21.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 3666, 2023-11-06T03:07:21.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 383594, 2023-11-06T03:07:21.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:07:21.940 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 3666, 2023-11-06T03:07:21.940 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:07:21.940 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:07:21.940 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 106, 2023-11-06T03:07:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 383594, 2023-11-06T03:07:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:07:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x556dcf170d20": { 2023-11-06T03:07:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:07:21.942 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:07:21.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:07:21.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 25784, 2023-11-06T03:07:21.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 81089239, 2023-11-06T03:07:21.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:07:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 25784, 2023-11-06T03:07:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:07:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:07:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6935, 2023-11-06T03:07:21.944 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 81089239, 2023-11-06T03:07:21.945 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:07:21.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:07:21.945 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:07:21.945 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:07:21.945 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.946 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:21.946 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:07:21.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 msgr2=0x7f9d2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 0x7f9d2c067f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d30000f70 tx=0x7f9d30007bc0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 msgr2=0x7f9d4414cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:21.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0b1040 tx=0x7f9d3c01a7d0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 shutdown_connections 2023-11-06T03:07:21.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d2c065a60 0x7f9d2c067f20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d44156a70 0x7f9d4414cf20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d440b2330 0x7f9d44149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 --2- 172.21.15.157:0/397731482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d440adae0 0x7f9d440b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:21.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 >> 172.21.15.157:0/397731482 conn(0x7f9d44065930 msgr2=0x7f9d4415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:21.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 shutdown_connections 2023-11-06T03:07:21.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:21.773+0000 7f9d48ba0640 1 -- 172.21.15.157:0/397731482 wait complete. 2023-11-06T03:07:21.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:07:21.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:07:21.997 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:07:22.045 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:07:22.046 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:07:22.046 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:22.052 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:22.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.221+0000 7fb13c9d5640 1 Processor -- start 2023-11-06T03:07:22.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.221+0000 7fb13c9d5640 1 -- start start 2023-11-06T03:07:22.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13c9d5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1381599c0 0x7fb1380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13c9d5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb1380b2af0 0x7fb138159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13c9d5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13815e440 con 0x7fb13806d000 2023-11-06T03:07:22.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13c9d5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13806c240 con 0x7fb1381599c0 2023-11-06T03:07:22.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13c9d5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13806c520 con 0x7fb1380b2af0 2023-11-06T03:07:22.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1381599c0 0x7fb1380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1381599c0 0x7fb1380ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55452/0 (socket says 172.21.15.157:55452) 2023-11-06T03:07:22.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 -- 172.21.15.157:0/849323577 learned_addr learned my addr 172.21.15.157:0/849323577 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb136d9d640 1 --2- 172.21.15.157:0/849323577 >> v2:172.21.15.195:3300/0 conn(0x7fb1380b2af0 0x7fb138159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 -- 172.21.15.157:0/849323577 >> v2:172.21.15.195:3300/0 conn(0x7fb1380b2af0 msgr2=0x7fb138159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 --2- 172.21.15.157:0/849323577 >> v2:172.21.15.195:3300/0 conn(0x7fb1380b2af0 0x7fb138159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 -- 172.21.15.157:0/849323577 >> v1:172.21.15.195:6789/0 conn(0x7fb13806d000 legacy=0x7fb1380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:22.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.225+0000 7fb13659c640 1 -- 172.21.15.157:0/849323577 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13806c800 con 0x7fb1381599c0 2023-11-06T03:07:22.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13659c640 1 --2- 172.21.15.157:0/849323577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1381599c0 0x7fb1380ae000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1380b19a0 tx=0x7fb12801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672e925ba85cd308 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13559a640 1 -- 172.21.15.157:0/849323577 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1280aa610 con 0x7fb1381599c0 2023-11-06T03:07:22.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13559a640 1 -- 172.21.15.157:0/849323577 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1280a6040 con 0x7fb1381599c0 2023-11-06T03:07:22.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13559a640 1 -- 172.21.15.157:0/849323577 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1280aa990 con 0x7fb1381599c0 2023-11-06T03:07:22.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/849323577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1381599c0 msgr2=0x7fb1380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/849323577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1381599c0 0x7fb1380ae000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1380b19a0 tx=0x7fb12801c490 comp rx=0 tx=0).stop 2023-11-06T03:07:22.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/849323577 shutdown_connections 2023-11-06T03:07:22.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/849323577 >> v2:172.21.15.195:3300/0 conn(0x7fb1380b2af0 0x7fb138159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/849323577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1381599c0 0x7fb1380ae000 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/849323577 >> 172.21.15.157:0/849323577 conn(0x7fb138065930 msgr2=0x7fb138065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:22.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/849323577 shutdown_connections 2023-11-06T03:07:22.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/849323577 wait complete. 2023-11-06T03:07:22.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 Processor -- start 2023-11-06T03:07:22.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- start start 2023-11-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 0x7fb138144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1381599c0 0x7fb138148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb138145450 con 0x7fb1380b2af0 2023-11-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380cc430 con 0x7fb13806d000 2023-11-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380cc710 con 0x7fb1381599c0 2023-11-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55468/0 (socket says 172.21.15.157:55468) 2023-11-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 -- 172.21.15.157:0/3249768070 learned_addr learned my addr 172.21.15.157:0/3249768070 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13659c640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 0x7fb138144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb136d9d640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1381599c0 0x7fb138148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 -- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1381599c0 msgr2=0x7fb138148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1381599c0 0x7fb138148dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 -- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 msgr2=0x7fb138144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 0x7fb138144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 -- 172.21.15.157:0/3249768070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280094d0 con 0x7fb1380b2af0 2023-11-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13659c640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 0x7fb138144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb135d9b640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c450 tx=0x7fb12c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c012830 con 0x7fb1380b2af0 2023-11-06T03:07:22.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c0129b0 con 0x7fb1380b2af0 2023-11-06T03:07:22.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.229+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380cc9f0 con 0x7fb1380b2af0 2023-11-06T03:07:22.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c012d60 con 0x7fb1380b2af0 2023-11-06T03:07:22.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb138149450 con 0x7fb1380b2af0 2023-11-06T03:07:22.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb138072750 con 0x7fb1380b2af0 2023-11-06T03:07:22.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb12c01c460 con 0x7fb1380b2af0 2023-11-06T03:07:22.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb126ffd640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 0x7fb118067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fb12c0831a0 con 0x7fb1380b2af0 2023-11-06T03:07:22.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb13659c640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 0x7fb118067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.233+0000 7fb13659c640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 0x7fb118067f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ae000 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.237+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb12c0509c0 con 0x7fb1380b2af0 2023-11-06T03:07:22.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.397+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1380cb110 con 0x7fb1380b2af0 2023-11-06T03:07:22.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.401+0000 7fb126ffd640 1 -- 172.21.15.157:0/3249768070 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18045 (secure 0 0 0) 0x7fb12c053fe0 con 0x7fb1380b2af0 2023-11-06T03:07:22.401 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:22.402 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":41,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:06:57.240474+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":40,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:22.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 msgr2=0x7fb118067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 0x7fb118067f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1280ae000 tx=0x7fb128008040 comp rx=0 tx=0).stop 2023-11-06T03:07:22.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 msgr2=0x7fb138158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c450 tx=0x7fb12c00c910 comp rx=0 tx=0).stop 2023-11-06T03:07:22.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 shutdown_connections 2023-11-06T03:07:22.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb118065a60 0x7fb118067f20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1381599c0 0x7fb138148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1380b2af0 0x7fb138158e50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 --2- 172.21.15.157:0/3249768070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb13806d000 0x7fb138144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 >> 172.21.15.157:0/3249768070 conn(0x7fb138065930 msgr2=0x7fb138070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:22.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 shutdown_connections 2023-11-06T03:07:22.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.405+0000 7fb13c9d5640 1 -- 172.21.15.157:0/3249768070 wait complete. 2023-11-06T03:07:22.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:05:12.378890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:05:13.585131+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 40, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:07:22.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:07:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 Processor -- start 2023-11-06T03:07:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 -- start start 2023-11-06T03:07:22.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22d015cf60 0x7f22d00cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 0x7f22d00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00cb6e0 con 0x7f22d015cb80 2023-11-06T03:07:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00ce260 con 0x7f22d015cf60 2023-11-06T03:07:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d7a5f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d0071830 con 0x7f22d00cc910 2023-11-06T03:07:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d57fb640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f22d015cb80 0x7f22d00cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57808/0 (socket says 172.21.15.157:57808) 2023-11-06T03:07:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d4ffa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22d015cf60 0x7f22d00cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d57fb640 1 -- 172.21.15.157:0/1827093736 learned_addr learned my addr 172.21.15.157:0/1827093736 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d5ffc640 1 --2- 172.21.15.157:0/1827093736 >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 0x7f22d00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d4ffa640 1 -- 172.21.15.157:0/1827093736 >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 msgr2=0x7f22d00cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d4ffa640 1 --2- 172.21.15.157:0/1827093736 >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 0x7f22d00cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d4ffa640 1 -- 172.21.15.157:0/1827093736 >> v1:172.21.15.157:6789/0 conn(0x7f22d015cb80 legacy=0x7f22d00cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22c7fff640 1 -- 172.21.15.157:0/1827093736 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633040827 0 0) 0x7f22d00cb6e0 con 0x7f22d015cb80 2023-11-06T03:07:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d4ffa640 1 -- 172.21.15.157:0/1827093736 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0071b10 con 0x7f22d015cf60 2023-11-06T03:07:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.549+0000 7f22d5ffc640 1 --2- 172.21.15.157:0/1827093736 >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 0x7f22d00cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:22.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 --2- 172.21.15.157:0/1827093736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cf60 0x7f22d00cd7e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f22d00c4450 tx=0x7f22c001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e31d9b01d0ad22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22c7fff640 1 -- 172.21.15.157:0/1827093736 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00a8950 con 0x7f22d015cf60 2023-11-06T03:07:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22c7fff640 1 -- 172.21.15.157:0/1827093736 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00a8ab0 con 0x7f22d015cf60 2023-11-06T03:07:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22c7fff640 1 -- 172.21.15.157:0/1827093736 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00a8e40 con 0x7f22d015cf60 2023-11-06T03:07:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- 172.21.15.157:0/1827093736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cf60 msgr2=0x7f22d00cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/1827093736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cf60 0x7f22d00cd7e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f22d00c4450 tx=0x7f22c001ec50 comp rx=0 tx=0).stop 2023-11-06T03:07:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- 172.21.15.157:0/1827093736 shutdown_connections 2023-11-06T03:07:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/1827093736 >> v2:172.21.15.195:3300/0 conn(0x7f22d00cc910 0x7f22d00cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/1827093736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cf60 0x7f22d00cd7e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- 172.21.15.157:0/1827093736 >> 172.21.15.157:0/1827093736 conn(0x7f22d006d560 msgr2=0x7f22d006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- 172.21.15.157:0/1827093736 shutdown_connections 2023-11-06T03:07:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- 172.21.15.157:0/1827093736 wait complete. 2023-11-06T03:07:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 Processor -- start 2023-11-06T03:07:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- start start 2023-11-06T03:07:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22d00cc910 0x7f22d01549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22d015cf60 0x7f22d00c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00c7cf0 con 0x7f22d015cb80 2023-11-06T03:07:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d0155b60 con 0x7f22d015cf60 2023-11-06T03:07:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d7a5f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d01552c0 con 0x7f22d00cc910 2023-11-06T03:07:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55486/0 (socket says 172.21.15.157:55486) 2023-11-06T03:07:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 -- 172.21.15.157:0/3204759576 learned_addr learned my addr 172.21.15.157:0/3204759576 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d5ffc640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22d015cf60 0x7f22d00c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 -- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22d00cc910 msgr2=0x7f22d01549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22d00cc910 0x7f22d01549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 -- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22d015cf60 msgr2=0x7f22d00c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22d015cf60 0x7f22d00c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.553+0000 7f22d4ffa640 1 -- 172.21.15.157:0/3204759576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009420 con 0x7f22d015cb80 2023-11-06T03:07:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d4ffa640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f22c00b3040 tx=0x7f22c001b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00b9630 con 0x7f22d015cb80 2023-11-06T03:07:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00b97b0 con 0x7f22d015cb80 2023-11-06T03:07:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d01555a0 con 0x7f22d015cb80 2023-11-06T03:07:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00b9b60 con 0x7f22d015cb80 2023-11-06T03:07:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d0154f00 con 0x7f22d015cb80 2023-11-06T03:07:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d00c7e30 con 0x7f22d015cb80 2023-11-06T03:07:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22c00b7040 con 0x7f22d015cb80 2023-11-06T03:07:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 0x7f22b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f22c01212d0 con 0x7f22d015cb80 2023-11-06T03:07:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d57fb640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 0x7f22b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.557+0000 7f22d57fb640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 0x7f22b4067f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0011f0 tx=0x7f22cc007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.561+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f22d00c7e30 con 0x7f22d015cb80 2023-11-06T03:07:22.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22d00c7e30 con 0x7f22d015cb80 2023-11-06T03:07:22.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22c5ffb640 1 -- 172.21.15.157:0/3204759576 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22d00c7e30 con 0x7f22d015cb80 2023-11-06T03:07:22.720 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:07:22.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 msgr2=0x7f22b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 0x7f22b4067f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0011f0 tx=0x7f22cc007930 comp rx=0 tx=0).stop 2023-11-06T03:07:22.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 msgr2=0x7f22d01f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f22c00b3040 tx=0x7f22c001b9f0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 shutdown_connections 2023-11-06T03:07:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22b4065a60 0x7f22b4067f20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22d015cf60 0x7f22d00c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d015cb80 0x7f22d01f1ac0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 --2- 172.21.15.157:0/3204759576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22d00cc910 0x7f22d01549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 >> 172.21.15.157:0/3204759576 conn(0x7f22d006d560 msgr2=0x7f22d006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 shutdown_connections 2023-11-06T03:07:22.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.721+0000 7f22d7a5f640 1 -- 172.21.15.157:0/3204759576 wait complete. 2023-11-06T03:07:22.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:07:22.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 Processor -- start 2023-11-06T03:07:22.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 -- start start 2023-11-06T03:07:22.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4f41551e0 0x7fc4f406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc4f415e5e0 0x7fc4f40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f415f580 con 0x7fc4f415ea90 2023-11-06T03:07:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f415efc0 con 0x7fc4f41551e0 2023-11-06T03:07:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4fb842640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f415f2a0 con 0x7fc4f415e5e0 2023-11-06T03:07:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4f41551e0 0x7fc4f406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4f41551e0 0x7fc4f406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55496/0 (socket says 172.21.15.157:55496) 2023-11-06T03:07:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 -- 172.21.15.157:0/4184712128 learned_addr learned my addr 172.21.15.157:0/4184712128 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f8ddd640 1 --2- 172.21.15.157:0/4184712128 >> v2:172.21.15.195:3300/0 conn(0x7fc4f415e5e0 0x7fc4f40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 -- 172.21.15.157:0/4184712128 >> v2:172.21.15.195:3300/0 conn(0x7fc4f415e5e0 msgr2=0x7fc4f40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 --2- 172.21.15.157:0/4184712128 >> v2:172.21.15.195:3300/0 conn(0x7fc4f415e5e0 0x7fc4f40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 -- 172.21.15.157:0/4184712128 >> v1:172.21.15.195:6789/0 conn(0x7fc4f415ea90 legacy=0x7fc4f415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.861+0000 7fc4f95de640 1 -- 172.21.15.157:0/4184712128 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f406f5a0 con 0x7fc4f41551e0 2023-11-06T03:07:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4f95de640 1 --2- 172.21.15.157:0/4184712128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f406fda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4066880 tx=0x7fc4e401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae973ceadbe69d35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4ebfff640 1 -- 172.21.15.157:0/4184712128 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4004230 con 0x7fc4f41551e0 2023-11-06T03:07:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4ebfff640 1 -- 172.21.15.157:0/4184712128 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40a7050 con 0x7fc4f41551e0 2023-11-06T03:07:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4ebfff640 1 -- 172.21.15.157:0/4184712128 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003620 con 0x7fc4f41551e0 2023-11-06T03:07:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- 172.21.15.157:0/4184712128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 msgr2=0x7fc4f406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- 172.21.15.157:0/4184712128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f406fda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4066880 tx=0x7fc4e401c440 comp rx=0 tx=0).stop 2023-11-06T03:07:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- 172.21.15.157:0/4184712128 shutdown_connections 2023-11-06T03:07:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- 172.21.15.157:0/4184712128 >> v2:172.21.15.195:3300/0 conn(0x7fc4f415e5e0 0x7fc4f40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- 172.21.15.157:0/4184712128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f406fda0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- 172.21.15.157:0/4184712128 >> 172.21.15.157:0/4184712128 conn(0x7fc4f4065930 msgr2=0x7fc4f4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- 172.21.15.157:0/4184712128 shutdown_connections 2023-11-06T03:07:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- 172.21.15.157:0/4184712128 wait complete. 2023-11-06T03:07:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 Processor -- start 2023-11-06T03:07:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- start start 2023-11-06T03:07:22.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 0x7fc4f4140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 0x7fc4f41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f406d550 con 0x7fc4f41551e0 2023-11-06T03:07:22.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f41f6390 con 0x7fc4f415e5e0 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4fb842640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f406c9d0 con 0x7fc4f415ea90 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.865+0000 7fc4f95de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55512/0 (socket says 172.21.15.157:55512) 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 -- 172.21.15.157:0/3181853233 learned_addr learned my addr 172.21.15.157:0/3181853233 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f8ddd640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 0x7fc4f4140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f9ddf640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 0x7fc4f41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 -- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 msgr2=0x7fc4f41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 0x7fc4f41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 -- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 msgr2=0x7fc4f4140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 0x7fc4f4140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 -- 172.21.15.157:0/3181853233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40094d0 con 0x7fc4f41551e0 2023-11-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f9ddf640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 0x7fc4f41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f8ddd640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 0x7fc4f4140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f95de640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4019050 tx=0x7fc4e40191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003b30 con 0x7fc4f41551e0 2023-11-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e4003c90 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f406ccb0 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40ab440 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f406d270 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f41f6500 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4e40ab5a0 con 0x7fc4f41551e0 2023-11-06T03:07:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 0x7fc4d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fc4e411dc80 con 0x7fc4f41551e0 2023-11-06T03:07:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.869+0000 7fc4f8ddd640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 0x7fc4d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.873+0000 7fc4f8ddd640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 0x7fc4d8067f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415e470 tx=0x7fc4f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:22.873+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4e40eb580 con 0x7fc4f41551e0 2023-11-06T03:07:23.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.029+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4f4072750 con 0x7fc4f41551e0 2023-11-06T03:07:23.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.029+0000 7fc4e9ffb640 1 -- 172.21.15.157:0/3181853233 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4e40eeba0 con 0x7fc4f41551e0 2023-11-06T03:07:23.029 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:07:23.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 msgr2=0x7fc4d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 0x7fc4d8067f20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4f415e470 tx=0x7fc4f0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:23.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 msgr2=0x7fc4f41536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4019050 tx=0x7fc4e40191e0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 shutdown_connections 2023-11-06T03:07:23.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d8065a60 0x7fc4d8067f20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4f415ea90 0x7fc4f41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4f415e5e0 0x7fc4f4140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 --2- 172.21.15.157:0/3181853233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4f41551e0 0x7fc4f41536a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 >> 172.21.15.157:0/3181853233 conn(0x7fc4f4065930 msgr2=0x7fc4f40b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 shutdown_connections 2023-11-06T03:07:23.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.033+0000 7fc4fb842640 1 -- 172.21.15.157:0/3181853233 wait complete. 2023-11-06T03:07:23.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:07:23.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 Processor -- start 2023-11-06T03:07:23.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 -- start start 2023-11-06T03:07:23.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd51015caa0 0x7fd5100cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100d2920 con 0x7fd5100d0280 2023-11-06T03:07:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100d18a0 con 0x7fd51015ce80 2023-11-06T03:07:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd5150c5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100d1b60 con 0x7fd51015caa0 2023-11-06T03:07:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd51015caa0 0x7fd5100cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd51015caa0 0x7fd5100cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55514/0 (socket says 172.21.15.157:55514) 2023-11-06T03:07:23.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 -- 172.21.15.157:0/4178853156 learned_addr learned my addr 172.21.15.157:0/4178853156 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:23.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 -- 172.21.15.157:0/4178853156 >> v1:172.21.15.195:6789/0 conn(0x7fd51015ce80 legacy=0x7fd5100cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 -- 172.21.15.157:0/4178853156 >> v1:172.21.15.157:6789/0 conn(0x7fd5100d0280 legacy=0x7fd5100b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50e59c640 1 -- 172.21.15.157:0/4178853156 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124941960 0 0) 0x7fd5100d2920 con 0x7fd5100d0280 2023-11-06T03:07:23.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 -- 172.21.15.157:0/4178853156 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100d1e40 con 0x7fd51015caa0 2023-11-06T03:07:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.177+0000 7fd50ed9d640 1 --2- 172.21.15.157:0/4178853156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51015caa0 0x7fd5100cdc60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd500009870 tx=0x7fd50001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc9c46a8fd5fa9a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50e59c640 1 -- 172.21.15.157:0/4178853156 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000aa670 con 0x7fd51015caa0 2023-11-06T03:07:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50e59c640 1 -- 172.21.15.157:0/4178853156 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5000aa7d0 con 0x7fd51015caa0 2023-11-06T03:07:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50e59c640 1 -- 172.21.15.157:0/4178853156 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000aab60 con 0x7fd51015caa0 2023-11-06T03:07:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/4178853156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51015caa0 msgr2=0x7fd5100cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 --2- 172.21.15.157:0/4178853156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51015caa0 0x7fd5100cdc60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd500009870 tx=0x7fd50001c510 comp rx=0 tx=0).stop 2023-11-06T03:07:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/4178853156 shutdown_connections 2023-11-06T03:07:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 --2- 172.21.15.157:0/4178853156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd51015caa0 0x7fd5100cdc60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/4178853156 >> 172.21.15.157:0/4178853156 conn(0x7fd510070790 msgr2=0x7fd510070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/4178853156 shutdown_connections 2023-11-06T03:07:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/4178853156 wait complete. 2023-11-06T03:07:23.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 Processor -- start 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- start start 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd51015caa0 0x7fd5101526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd51015ce80 0x7fd51015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100cbb90 con 0x7fd5100d0280 2023-11-06T03:07:23.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015dc60 con 0x7fd51015ce80 2023-11-06T03:07:23.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51015fa90 con 0x7fd51015caa0 2023-11-06T03:07:23.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55528/0 (socket says 172.21.15.157:55528) 2023-11-06T03:07:23.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 -- 172.21.15.157:0/168138281 learned_addr learned my addr 172.21.15.157:0/168138281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 -- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd51015caa0 msgr2=0x7fd5101526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd51015caa0 0x7fd5101526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 -- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd51015ce80 msgr2=0x7fd51015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50f59e640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd51015ce80 0x7fd51015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd51015ce80 0x7fd51015efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 -- 172.21.15.157:0/168138281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000094d0 con 0x7fd5100d0280 2023-11-06T03:07:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd50ed9d640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ac040 tx=0x7fd50001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000b38c0 con 0x7fd5100d0280 2023-11-06T03:07:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5000b3a40 con 0x7fd5100d0280 2023-11-06T03:07:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51015d2c0 con 0x7fd5100d0280 2023-11-06T03:07:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000b3df0 con 0x7fd5100d0280 2023-11-06T03:07:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.181+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51015d880 con 0x7fd5100d0280 2023-11-06T03:07:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51015ded0 con 0x7fd5100d0280 2023-11-06T03:07:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd5000c6390 con 0x7fd5100d0280 2023-11-06T03:07:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd5077fe640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 0x7fd4f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fd50011e170 con 0x7fd5100d0280 2023-11-06T03:07:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd507fff640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 0x7fd4f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.185+0000 7fd507fff640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 0x7fd4f0067f20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8000f70 tx=0x7fd4f80079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.189+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd5000eba20 con 0x7fd5100d0280 2023-11-06T03:07:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.341+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5100b1ed0 con 0x7fd5100d0280 2023-11-06T03:07:23.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.341+0000 7fd5077fe640 1 -- 172.21.15.157:0/168138281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5000ef040 con 0x7fd5100d0280 2023-11-06T03:07:23.342 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:07:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 msgr2=0x7fd4f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 0x7fd4f0067f20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8000f70 tx=0x7fd4f80079c0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 msgr2=0x7fd51015e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ac040 tx=0x7fd50001ad50 comp rx=0 tx=0).stop 2023-11-06T03:07:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 shutdown_connections 2023-11-06T03:07:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4f0065a60 0x7fd4f0067f20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd51015ce80 0x7fd51015efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd51015caa0 0x7fd5101526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 --2- 172.21.15.157:0/168138281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5100d0280 0x7fd51015e080 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 >> 172.21.15.157:0/168138281 conn(0x7fd510070790 msgr2=0x7fd5100b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 shutdown_connections 2023-11-06T03:07:23.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.345+0000 7fd5150c5640 1 -- 172.21.15.157:0/168138281 wait complete. 2023-11-06T03:07:23.360 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:07:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.481+0000 7fd71fbe0640 1 Processor -- start 2023-11-06T03:07:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.481+0000 7fd71fbe0640 1 -- start start 2023-11-06T03:07:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71fbe0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71fbe0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7180b3b20 0x7fd71815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71fbe0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71815f580 con 0x7fd71815e5e0 2023-11-06T03:07:23.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71fbe0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71815efc0 con 0x7fd7180b3b20 2023-11-06T03:07:23.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71fbe0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815f2a0 con 0x7fd7181551e0 2023-11-06T03:07:23.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7180b3b20 0x7fd71815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7180b3b20 0x7fd71815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55536/0 (socket says 172.21.15.157:55536) 2023-11-06T03:07:23.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 -- 172.21.15.157:0/748472 learned_addr learned my addr 172.21.15.157:0/748472 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:23.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71d97c640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 -- 172.21.15.157:0/748472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 msgr2=0x7fd71806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:23.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 -- 172.21.15.157:0/748472 >> v1:172.21.15.157:6789/0 conn(0x7fd71815e5e0 legacy=0x7fd7180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 -- 172.21.15.157:0/748472 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71806f5a0 con 0x7fd7180b3b20 2023-11-06T03:07:23.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71c97a640 1 -- 172.21.15.157:0/748472 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300413385 0 0) 0x7fd71815f580 con 0x7fd71815e5e0 2023-11-06T03:07:23.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71e17d640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd71815c1b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd710009e50 tx=0x7fd71001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a37a5a9ee011ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71c97a640 1 -- 172.21.15.157:0/748472 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100a5830 con 0x7fd7180b3b20 2023-11-06T03:07:23.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71c97a640 1 -- 172.21.15.157:0/748472 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7100a5990 con 0x7fd7180b3b20 2023-11-06T03:07:23.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.485+0000 7fd71c97a640 1 -- 172.21.15.157:0/748472 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100a5d20 con 0x7fd7180b3b20 2023-11-06T03:07:23.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/748472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 msgr2=0x7fd71815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd71815c1b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd710009e50 tx=0x7fd71001c950 comp rx=0 tx=0).stop 2023-11-06T03:07:23.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/748472 shutdown_connections 2023-11-06T03:07:23.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd71815c1b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/748472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/748472 >> 172.21.15.157:0/748472 conn(0x7fd718065930 msgr2=0x7fd718065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/748472 shutdown_connections 2023-11-06T03:07:23.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/748472 wait complete. 2023-11-06T03:07:23.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 Processor -- start 2023-11-06T03:07:23.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- start start 2023-11-06T03:07:23.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 0x7fd7180cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815a4a0 con 0x7fd7180b3b20 2023-11-06T03:07:23.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180ae350 con 0x7fd71815e5e0 2023-11-06T03:07:23.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71806c0c0 con 0x7fd7181551e0 2023-11-06T03:07:23.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55552/0 (socket says 172.21.15.157:55552) 2023-11-06T03:07:23.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 -- 172.21.15.157:0/3540529127 learned_addr learned my addr 172.21.15.157:0/3540529127 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:23.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d17b640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71e17d640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 0x7fd7180cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 -- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 msgr2=0x7fd71814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 -- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 msgr2=0x7fd7180cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 0x7fd7180cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 -- 172.21.15.157:0/3540529127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009b30 con 0x7fd7180b3b20 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71e17d640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 0x7fd7180cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71d97c640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d800 tx=0x7fd71400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400b6c0 con 0x7fd7180b3b20 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400b820 con 0x7fd7180b3b20 2023-11-06T03:07:23.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd718159b70 con 0x7fd7180b3b20 2023-11-06T03:07:23.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400bb90 con 0x7fd7180b3b20 2023-11-06T03:07:23.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71815a130 con 0x7fd7180b3b20 2023-11-06T03:07:23.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.489+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718072750 con 0x7fd7180b3b20 2023-11-06T03:07:23.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.493+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd71400bcf0 con 0x7fd7180b3b20 2023-11-06T03:07:23.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.493+0000 7fd70e7fc640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 0x7fd6fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.493+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fd714084310 con 0x7fd7180b3b20 2023-11-06T03:07:23.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.493+0000 7fd71d17b640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 0x7fd6fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.493+0000 7fd71d17b640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 0x7fd6fc067f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd71815d450 tx=0x7fd708009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.497+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd714051c10 con 0x7fd7180b3b20 2023-11-06T03:07:23.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.653+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7180cb110 con 0x7fd7180b3b20 2023-11-06T03:07:23.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.653+0000 7fd70e7fc640 1 -- 172.21.15.157:0/3540529127 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd714055230 con 0x7fd7180b3b20 2023-11-06T03:07:23.654 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:07:23.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 msgr2=0x7fd6fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 0x7fd6fc067f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd71815d450 tx=0x7fd708009040 comp rx=0 tx=0).stop 2023-11-06T03:07:23.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 msgr2=0x7fd7180adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d800 tx=0x7fd71400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 shutdown_connections 2023-11-06T03:07:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6fc065a60 0x7fd6fc067f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd71815e5e0 0x7fd7180cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7181551e0 0x7fd71814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 --2- 172.21.15.157:0/3540529127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7180b3b20 0x7fd7180adcc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 >> 172.21.15.157:0/3540529127 conn(0x7fd718065930 msgr2=0x7fd71814e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 shutdown_connections 2023-11-06T03:07:23.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.657+0000 7fd71fbe0640 1 -- 172.21.15.157:0/3540529127 wait complete. 2023-11-06T03:07:23.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:07:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 Processor -- start 2023-11-06T03:07:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 -- start start 2023-11-06T03:07:23.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc2815cf60 0x7fcc280cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc280cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280cb6e0 con 0x7fcc2815cb80 2023-11-06T03:07:23.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280ce260 con 0x7fcc2815cf60 2023-11-06T03:07:23.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2cff4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28071830 con 0x7fcc280cc910 2023-11-06T03:07:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcc2815cb80 0x7fcc280cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57828/0 (socket says 172.21.15.157:57828) 2023-11-06T03:07:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2659c640 1 -- 172.21.15.157:0/764770433 learned_addr learned my addr 172.21.15.157:0/764770433 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc25d9b640 1 --2- 172.21.15.157:0/764770433 >> v2:172.21.15.195:3300/0 conn(0x7fcc2815cf60 0x7fcc280cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc280cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139299985 0 0) 0x7fcc280cb6e0 con 0x7fcc2815cb80 2023-11-06T03:07:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc10003530 con 0x7fcc2815cb80 2023-11-06T03:07:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236498003 0 0) 0x7fcc10003530 con 0x7fcc2815cb80 2023-11-06T03:07:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 msgr2=0x7fcc280cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc280cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 >> v2:172.21.15.195:3300/0 conn(0x7fcc2815cf60 msgr2=0x7fcc280cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 --2- 172.21.15.157:0/764770433 >> v2:172.21.15.195:3300/0 conn(0x7fcc2815cf60 0x7fcc280cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28071b10 con 0x7fcc2815cb80 2023-11-06T03:07:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fcc1c001680 con 0x7fcc2815cb80 2023-11-06T03:07:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 >> v1:172.21.15.157:6789/0 conn(0x7fcc2815cb80 legacy=0x7fcc280cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc10003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc10003f00 0x7fcc100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc10005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28071830 con 0x7fcc10004d10 2023-11-06T03:07:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1000b990 con 0x7fcc10003f00 2023-11-06T03:07:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1000bb00 con 0x7fcc280cc910 2023-11-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc10005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc25d9b640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc10003f00 0x7fcc100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc2659c640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc10003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 -- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 msgr2=0x7fcc10003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc280cc910 0x7fcc10003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 -- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc10003f00 msgr2=0x7fcc100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc10003f00 0x7fcc100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.793+0000 7fcc26d9d640 1 -- 172.21.15.157:0/764770433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10003530 con 0x7fcc10004d10 2023-11-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc10005190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009fc0 tx=0x7fcc2001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9284f983e6210dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200a5c70 con 0x7fcc10004d10 2023-11-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000f040 con 0x7fcc10004d10 2023-11-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2559a640 1 -- 172.21.15.157:0/764770433 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc200ad440 con 0x7fcc10004d10 2023-11-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- 172.21.15.157:0/764770433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 msgr2=0x7fcc10005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/764770433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc10005190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009fc0 tx=0x7fcc2001cc60 comp rx=0 tx=0).stop 2023-11-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc26d9d640 1 -- 172.21.15.157:0/764770433 reap_dead start 2023-11-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- 172.21.15.157:0/764770433 shutdown_connections 2023-11-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- 172.21.15.157:0/764770433 >> 172.21.15.157:0/764770433 conn(0x7fcc2806d560 msgr2=0x7fcc2806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- 172.21.15.157:0/764770433 shutdown_connections 2023-11-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- 172.21.15.157:0/764770433 wait complete. 2023-11-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 Processor -- start 2023-11-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.797+0000 7fcc2cff4640 1 -- start start 2023-11-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc28154880 0x7fcc280c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc28157eb0 0x7fcc28151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28152180 con 0x7fcc10004d10 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c6670 con 0x7fcc28154880 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c6950 con 0x7fcc28157eb0 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55562/0 (socket says 172.21.15.157:55562) 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 -- 172.21.15.157:0/1881511046 learned_addr learned my addr 172.21.15.157:0/1881511046 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc26d9d640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc28157eb0 0x7fcc28151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc25d9b640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc28154880 0x7fcc280c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 -- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc28157eb0 msgr2=0x7fcc28151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc28157eb0 0x7fcc28151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 -- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc28154880 msgr2=0x7fcc280c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc28154880 0x7fcc280c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 -- 172.21.15.157:0/1881511046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280c6c30 con 0x7fcc10004d10 2023-11-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2659c640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00d900 tx=0x7fcc1c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c007560 con 0x7fcc10004d10 2023-11-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c0076c0 con 0x7fcc10004d10 2023-11-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004a30 con 0x7fcc10004d10 2023-11-06T03:07:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc280c6f10 con 0x7fcc10004d10 2023-11-06T03:07:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28158330 con 0x7fcc10004d10 2023-11-06T03:07:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc281585b0 con 0x7fcc10004d10 2023-11-06T03:07:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcc1c004b90 con 0x7fcc10004d10 2023-11-06T03:07:23.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 0x7fcc08067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fcc1c082d70 con 0x7fcc10004d10 2023-11-06T03:07:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.801+0000 7fcc25d9b640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 0x7fcc08067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.805+0000 7fcc25d9b640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 0x7fcc08067f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcc280afaa0 tx=0x7fcc18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.805+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcc1c050620 con 0x7fcc10004d10 2023-11-06T03:07:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcc28152340 con 0x7fcc10004d10 2023-11-06T03:07:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc16ffd640 1 -- 172.21.15.157:0/1881511046 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcc1c019080 con 0x7fcc10004d10 2023-11-06T03:07:23.960 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 msgr2=0x7fcc08067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 0x7fcc08067f50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcc280afaa0 tx=0x7fcc18008040 comp rx=0 tx=0).stop 2023-11-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 msgr2=0x7fcc280c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00d900 tx=0x7fcc1c00ddc0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 shutdown_connections 2023-11-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc08065a90 0x7fcc08067f50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc28157eb0 0x7fcc28151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc28154880 0x7fcc280c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 --2- 172.21.15.157:0/1881511046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc10004d10 0x7fcc280c4470 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 >> 172.21.15.157:0/1881511046 conn(0x7fcc2806d560 msgr2=0x7fcc280cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 shutdown_connections 2023-11-06T03:07:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:23.961+0000 7fcc2cff4640 1 -- 172.21.15.157:0/1881511046 wait complete. 2023-11-06T03:07:23.978 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:07:23.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:23.978 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:23.987 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[48440]: fuse finished with error 0 and tester_r 0 2023-11-06T03:07:23.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:07:23.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:23.992 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:07:24.171 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:07:24.172 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:24.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:07:24.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:07:24.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:24.188 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:07:24.235 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:07:24.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:07:24.237 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:07:24.237 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:24.243 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:07:24.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:07:24.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:24.244 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:07:24.291 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:07:24.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:07:24.293 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:07:24.293 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:24.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:07:24.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:07:24.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:24.300 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:07:24.348 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:07:24.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:07:24.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:24.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 Processor -- start 2023-11-06T03:07:24.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 -- start start 2023-11-06T03:07:24.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d440b3b20 0x7f7d4415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4415f580 con 0x7f7d4415e5e0 2023-11-06T03:07:24.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4415efc0 con 0x7f7d441551e0 2023-11-06T03:07:24.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d4898e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d440704b0 con 0x7f7d440b3b20 2023-11-06T03:07:24.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d3bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7d4415e5e0 0x7f7d440b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57832/0 (socket says 172.21.15.157:57832) 2023-11-06T03:07:24.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d42d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d440b3b20 0x7f7d4415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d3bfff640 1 -- 172.21.15.157:0/890254292 learned_addr learned my addr 172.21.15.157:0/890254292 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:24.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d42d9d640 1 -- 172.21.15.157:0/890254292 >> v1:172.21.15.195:6789/0 conn(0x7f7d441551e0 legacy=0x7f7d4406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:24.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d42d9d640 1 -- 172.21.15.157:0/890254292 >> v1:172.21.15.157:6789/0 conn(0x7f7d4415e5e0 legacy=0x7f7d440b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d41d9b640 1 -- 172.21.15.157:0/890254292 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830925676 0 0) 0x7f7d4415f580 con 0x7f7d4415e5e0 2023-11-06T03:07:24.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.477+0000 7f7d42d9d640 1 -- 172.21.15.157:0/890254292 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4415f2a0 con 0x7f7d440b3b20 2023-11-06T03:07:24.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d42d9d640 1 --2- 172.21.15.157:0/890254292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4415c1b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009d90 tx=0x7f7d3c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4048257600077ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d41d9b640 1 -- 172.21.15.157:0/890254292 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0a58c0 con 0x7f7d440b3b20 2023-11-06T03:07:24.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d41d9b640 1 -- 172.21.15.157:0/890254292 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c0a5a20 con 0x7f7d440b3b20 2023-11-06T03:07:24.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d41d9b640 1 -- 172.21.15.157:0/890254292 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0a5db0 con 0x7f7d440b3b20 2023-11-06T03:07:24.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- 172.21.15.157:0/890254292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 msgr2=0x7f7d4415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 --2- 172.21.15.157:0/890254292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4415c1b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c009d90 tx=0x7f7d3c01c800 comp rx=0 tx=0).stop 2023-11-06T03:07:24.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- 172.21.15.157:0/890254292 shutdown_connections 2023-11-06T03:07:24.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 --2- 172.21.15.157:0/890254292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4415c1b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- 172.21.15.157:0/890254292 >> 172.21.15.157:0/890254292 conn(0x7f7d44065930 msgr2=0x7f7d44065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- 172.21.15.157:0/890254292 shutdown_connections 2023-11-06T03:07:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- 172.21.15.157:0/890254292 wait complete. 2023-11-06T03:07:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 Processor -- start 2023-11-06T03:07:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- start start 2023-11-06T03:07:24.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 0x7f7d4414cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d4415e5e0 0x7f7d441590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4414bb40 con 0x7f7d440b3b20 2023-11-06T03:07:24.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4415a310 con 0x7f7d4415e5e0 2023-11-06T03:07:24.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4898e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4414c060 con 0x7f7d441551e0 2023-11-06T03:07:24.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55576/0 (socket says 172.21.15.157:55576) 2023-11-06T03:07:24.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 -- 172.21.15.157:0/2977627540 learned_addr learned my addr 172.21.15.157:0/2977627540 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d42d9d640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d4415e5e0 0x7f7d441590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d3bfff640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 0x7f7d4414cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 -- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 msgr2=0x7f7d4414cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 0x7f7d4414cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 -- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d4415e5e0 msgr2=0x7f7d441590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d4415e5e0 0x7f7d441590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 -- 172.21.15.157:0/2977627540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c009a70 con 0x7f7d440b3b20 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d3bfff640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 0x7f7d4414cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.481+0000 7f7d4259c640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c500 tx=0x7f7d3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34005220 con 0x7f7d440b3b20 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d34005380 con 0x7f7d440b3b20 2023-11-06T03:07:24.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4414c340 con 0x7f7d440b3b20 2023-11-06T03:07:24.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34007270 con 0x7f7d440b3b20 2023-11-06T03:07:24.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d44159fa0 con 0x7f7d440b3b20 2023-11-06T03:07:24.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44072750 con 0x7f7d440b3b20 2023-11-06T03:07:24.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d340073d0 con 0x7f7d440b3b20 2023-11-06T03:07:24.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 0x7f7d24067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f7d34082be0 con 0x7f7d440b3b20 2023-11-06T03:07:24.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3bfff640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 0x7f7d24067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.485+0000 7f7d3bfff640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 0x7f7d24067f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d440cef40 tx=0x7f7d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.489+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d34050410 con 0x7f7d440b3b20 2023-11-06T03:07:24.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.677+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d44071730 con 0x7f7d440b3b20 2023-11-06T03:07:24.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d3affd640 1 -- 172.21.15.157:0/2977627540 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f7d34053a30 con 0x7f7d440b3b20 2023-11-06T03:07:24.679 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:24.680 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:07:11.278907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":32,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:07:24.680 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-11-06T03:07:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 msgr2=0x7f7d24067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 0x7f7d24067f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d440cef40 tx=0x7f7d2c009040 comp rx=0 tx=0).stop 2023-11-06T03:07:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 msgr2=0x7f7d4414b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c500 tx=0x7f7d3400c9c0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 shutdown_connections 2023-11-06T03:07:24.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d24065a90 0x7f7d24067f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d4415e5e0 0x7f7d441590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d441551e0 0x7f7d4414cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 --2- 172.21.15.157:0/2977627540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d440b3b20 0x7f7d4414b600 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 >> 172.21.15.157:0/2977627540 conn(0x7f7d44065930 msgr2=0x7f7d44065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 shutdown_connections 2023-11-06T03:07:24.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.681+0000 7f7d4898e640 1 -- 172.21.15.157:0/2977627540 wait complete. 2023-11-06T03:07:24.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:24.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.821+0000 7fefadb06640 1 Processor -- start 2023-11-06T03:07:24.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.821+0000 7fefadb06640 1 -- start start 2023-11-06T03:07:24.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefa815ea60 0x7fefa80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa815f580 con 0x7fefa81551e0 2023-11-06T03:07:24.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa80704b0 con 0x7fefa815ea60 2023-11-06T03:07:24.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa815efc0 con 0x7fefa815e5e0 2023-11-06T03:07:24.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefa815ea60 0x7fefa80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefa815ea60 0x7fefa80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55578/0 (socket says 172.21.15.157:55578) 2023-11-06T03:07:24.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 -- 172.21.15.157:0/3191699018 learned_addr learned my addr 172.21.15.157:0/3191699018 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:24.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fef9effd640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 -- 172.21.15.157:0/3191699018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 msgr2=0x7fefa80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:24.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 -- 172.21.15.157:0/3191699018 >> v1:172.21.15.195:6789/0 conn(0x7fefa81551e0 legacy=0x7fefa806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 -- 172.21.15.157:0/3191699018 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa815f2a0 con 0x7fefa815ea60 2023-11-06T03:07:24.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa7fff640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80b3950 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009e50 tx=0x7fefa001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa7510372d3eb28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa6ffd640 1 -- 172.21.15.157:0/3191699018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00a5810 con 0x7fefa815ea60 2023-11-06T03:07:24.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa6ffd640 1 -- 172.21.15.157:0/3191699018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa00a5970 con 0x7fefa815ea60 2023-11-06T03:07:24.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefa6ffd640 1 -- 172.21.15.157:0/3191699018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00a5d00 con 0x7fefa815ea60 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- 172.21.15.157:0/3191699018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 msgr2=0x7fefa80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80b3950 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009e50 tx=0x7fefa001c950 comp rx=0 tx=0).stop 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- 172.21.15.157:0/3191699018 shutdown_connections 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80b3950 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 --2- 172.21.15.157:0/3191699018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- 172.21.15.157:0/3191699018 >> 172.21.15.157:0/3191699018 conn(0x7fefa8065930 msgr2=0x7fefa8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:24.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- 172.21.15.157:0/3191699018 shutdown_connections 2023-11-06T03:07:24.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.825+0000 7fefadb06640 1 -- 172.21.15.157:0/3191699018 wait complete. 2023-11-06T03:07:24.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 Processor -- start 2023-11-06T03:07:24.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- start start 2023-11-06T03:07:24.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefa81551e0 0x7fefa81555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa81543e0 con 0x7fefa815ea60 2023-11-06T03:07:24.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa8156c00 con 0x7fefa81551e0 2023-11-06T03:07:24.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa8153760 con 0x7fefa815e5e0 2023-11-06T03:07:24.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55592/0 (socket says 172.21.15.157:55592) 2023-11-06T03:07:24.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 -- 172.21.15.157:0/3245666683 learned_addr learned my addr 172.21.15.157:0/3245666683 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:24.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fef9effd640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa77fe640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefa81551e0 0x7fefa81555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 -- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 msgr2=0x7fefa814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa814d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 -- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefa81551e0 msgr2=0x7fefa81555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefa81551e0 0x7fefa81555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 -- 172.21.15.157:0/3245666683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009b30 con 0x7fefa815ea60 2023-11-06T03:07:24.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa7fff640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefa001a580 tx=0x7fefa00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00b8670 con 0x7fefa815ea60 2023-11-06T03:07:24.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa00b87d0 con 0x7fefa815ea60 2023-11-06T03:07:24.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00b8b40 con 0x7fefa815ea60 2023-11-06T03:07:24.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa8153a40 con 0x7fefa815ea60 2023-11-06T03:07:24.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa8154000 con 0x7fefa815ea60 2023-11-06T03:07:24.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.829+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa8072750 con 0x7fefa815ea60 2023-11-06T03:07:24.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.833+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefa00b8ca0 con 0x7fefa815ea60 2023-11-06T03:07:24.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.833+0000 7fefa4ff9640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 0x7fef88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:24.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.833+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fefa011e8a0 con 0x7fefa815ea60 2023-11-06T03:07:24.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.833+0000 7fefa77fe640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 0x7fef88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:24.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.833+0000 7fefa77fe640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 0x7fef88067f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef98000f70 tx=0x7fef98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:24.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.837+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fefa00ec1e0 con 0x7fefa815ea60 2023-11-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.989+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa8071730 con 0x7fefa815ea60 2023-11-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefa4ff9640 1 -- 172.21.15.157:0/3245666683 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18045 (secure 0 0 0) 0x7fefa00ef800 con 0x7fefa815ea60 2023-11-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":41,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:06:57.240474+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":40,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 msgr2=0x7fef88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 0x7fef88067f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef98000f70 tx=0x7fef98009040 comp rx=0 tx=0).stop 2023-11-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 msgr2=0x7fefa80af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fefa001a580 tx=0x7fefa00196f0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 shutdown_connections 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef88065a60 0x7fef88067f20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefa815ea60 0x7fefa80af6c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefa815e5e0 0x7fefa814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 --2- 172.21.15.157:0/3245666683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefa81551e0 0x7fefa81555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 >> 172.21.15.157:0/3245666683 conn(0x7fefa8065930 msgr2=0x7fefa815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 shutdown_connections 2023-11-06T03:07:24.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:24.993+0000 7fefadb06640 1 -- 172.21.15.157:0/3245666683 wait complete. 2023-11-06T03:07:25.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:05:12.378890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:05:13.585131+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 40, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:07:25.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:07:25.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.137+0000 7f7e38146640 1 Processor -- start 2023-11-06T03:07:25.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.137+0000 7f7e38146640 1 -- start start 2023-11-06T03:07:25.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e301551e0 0x7f7e3006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 0x7f7e300b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 0x7f7e300b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3015f580 con 0x7f7e301551e0 2023-11-06T03:07:25.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3015efc0 con 0x7f7e3015e5e0 2023-11-06T03:07:25.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e38146640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015f2a0 con 0x7f7e3015ea90 2023-11-06T03:07:25.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e301551e0 0x7f7e3006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e301551e0 0x7f7e3006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55608/0 (socket says 172.21.15.157:55608) 2023-11-06T03:07:25.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 -- 172.21.15.157:0/4276378025 learned_addr learned my addr 172.21.15.157:0/4276378025 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:25.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e356e1640 1 --2- 172.21.15.157:0/4276378025 >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 0x7f7e300b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e366e3640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 0x7f7e300b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 -- 172.21.15.157:0/4276378025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 msgr2=0x7f7e300b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 0x7f7e300b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 -- 172.21.15.157:0/4276378025 >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 msgr2=0x7f7e300b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/4276378025 >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 0x7f7e300b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 -- 172.21.15.157:0/4276378025 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3006f5a0 con 0x7f7e301551e0 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e356e1640 1 --2- 172.21.15.157:0/4276378025 >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 0x7f7e300b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:25.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e366e3640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 0x7f7e300b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:25.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e301551e0 0x7f7e3006fda0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7e30066880 tx=0x7f7e2c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33961540b67d97e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e34ee0640 1 -- 172.21.15.157:0/4276378025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0ab620 con 0x7f7e301551e0 2023-11-06T03:07:25.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e34ee0640 1 -- 172.21.15.157:0/4276378025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0ab780 con 0x7f7e301551e0 2023-11-06T03:07:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.141+0000 7f7e34ee0640 1 -- 172.21.15.157:0/4276378025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0abb10 con 0x7f7e301551e0 2023-11-06T03:07:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/4276378025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e301551e0 msgr2=0x7f7e3006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e301551e0 0x7f7e3006fda0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7e30066880 tx=0x7f7e2c01c440 comp rx=0 tx=0).stop 2023-11-06T03:07:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/4276378025 shutdown_connections 2023-11-06T03:07:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e3015ea90 0x7f7e300b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- 172.21.15.157:0/4276378025 >> v2:172.21.15.195:3300/0 conn(0x7f7e3015e5e0 0x7f7e300b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- 172.21.15.157:0/4276378025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e301551e0 0x7f7e3006fda0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/4276378025 >> 172.21.15.157:0/4276378025 conn(0x7f7e30065930 msgr2=0x7f7e30065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/4276378025 shutdown_connections 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/4276378025 wait complete. 2023-11-06T03:07:25.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 Processor -- start 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- start start 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e301551e0 0x7f7e3014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e3015e5e0 0x7f7e301f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300b16d0 con 0x7f7e3015ea90 2023-11-06T03:07:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300b19b0 con 0x7f7e3015e5e0 2023-11-06T03:07:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300b1c90 con 0x7f7e301551e0 2023-11-06T03:07:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55624/0 (socket says 172.21.15.157:55624) 2023-11-06T03:07:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 -- 172.21.15.157:0/2354128083 learned_addr learned my addr 172.21.15.157:0/2354128083 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e301551e0 0x7f7e3014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e356e1640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e3015e5e0 0x7f7e301f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 -- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e301551e0 msgr2=0x7f7e3014d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e301551e0 0x7f7e3014d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 -- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e3015e5e0 msgr2=0x7f7e301f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e3015e5e0 0x7f7e301f8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 -- 172.21.15.157:0/2354128083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0094d0 con 0x7f7e3015ea90 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e366e3640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800cd50 tx=0x7f7e2800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800f680 con 0x7f7e3015ea90 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800f800 con 0x7f7e3015ea90 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e300b1f70 con 0x7f7e3015ea90 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800fbb0 con 0x7f7e3015ea90 2023-11-06T03:07:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e301f8fe0 con 0x7f7e3015ea90 2023-11-06T03:07:25.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.145+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e30072750 con 0x7f7e3015ea90 2023-11-06T03:07:25.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.149+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7e2800b040 con 0x7f7e3015ea90 2023-11-06T03:07:25.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.149+0000 7f7e267fc640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 0x7f7e14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.149+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f7e28083d50 con 0x7f7e3015ea90 2023-11-06T03:07:25.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.149+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 0x7f7e14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.149+0000 7f7e35ee2640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 0x7f7e14067f20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0af000 tx=0x7f7e2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.153+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7e28087030 con 0x7f7e3015ea90 2023-11-06T03:07:25.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e30071730 con 0x7f7e3015ea90 2023-11-06T03:07:25.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e267fc640 1 -- 172.21.15.157:0/2354128083 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e28051570 con 0x7f7e3015ea90 2023-11-06T03:07:25.308 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 msgr2=0x7f7e14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 0x7f7e14067f20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0af000 tx=0x7f7e2c008040 comp rx=0 tx=0).stop 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 msgr2=0x7f7e3015a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800cd50 tx=0x7f7e2800a650 comp rx=0 tx=0).stop 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 shutdown_connections 2023-11-06T03:07:25.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e14065a60 0x7f7e14067f20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e3015ea90 0x7f7e3015a340 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e3015e5e0 0x7f7e301f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 --2- 172.21.15.157:0/2354128083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e301551e0 0x7f7e3014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 >> 172.21.15.157:0/2354128083 conn(0x7f7e30065930 msgr2=0x7f7e300aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 shutdown_connections 2023-11-06T03:07:25.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.309+0000 7f7e38146640 1 -- 172.21.15.157:0/2354128083 wait complete. 2023-11-06T03:07:25.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:07:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 Processor -- start 2023-11-06T03:07:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 -- start start 2023-11-06T03:07:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc401551e0 0x7fbc4006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc400b3b20 0x7fbc4015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4015f580 con 0x7fbc4015e5e0 2023-11-06T03:07:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4015efc0 con 0x7fbc401551e0 2023-11-06T03:07:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc47399640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4015f2a0 con 0x7fbc400b3b20 2023-11-06T03:07:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc401551e0 0x7fbc4006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc44934640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbc4015e5e0 0x7fbc400b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57838/0 (socket says 172.21.15.157:57838) 2023-11-06T03:07:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc44934640 1 -- 172.21.15.157:0/3534144218 learned_addr learned my addr 172.21.15.157:0/3534144218 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbc401551e0 0x7fbc4006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55626/0 (socket says 172.21.15.157:55626) 2023-11-06T03:07:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45936640 1 --2- 172.21.15.157:0/3534144218 >> v2:172.21.15.195:3300/0 conn(0x7fbc400b3b20 0x7fbc4015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 -- 172.21.15.157:0/3534144218 >> v2:172.21.15.195:3300/0 conn(0x7fbc400b3b20 msgr2=0x7fbc4015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 --2- 172.21.15.157:0/3534144218 >> v2:172.21.15.195:3300/0 conn(0x7fbc400b3b20 0x7fbc4015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 -- 172.21.15.157:0/3534144218 >> v1:172.21.15.157:6789/0 conn(0x7fbc4015e5e0 legacy=0x7fbc400b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc37fff640 1 -- 172.21.15.157:0/3534144218 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163601162 0 0) 0x7fbc4015f580 con 0x7fbc4015e5e0 2023-11-06T03:07:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.453+0000 7fbc45135640 1 -- 172.21.15.157:0/3534144218 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4006f5a0 con 0x7fbc401551e0 2023-11-06T03:07:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc45135640 1 --2- 172.21.15.157:0/3534144218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc401551e0 0x7fbc4006fda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc400cbfe0 tx=0x7fbc3c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11386c31d9ad2fc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc37fff640 1 -- 172.21.15.157:0/3534144218 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0ab600 con 0x7fbc401551e0 2023-11-06T03:07:25.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc37fff640 1 -- 172.21.15.157:0/3534144218 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0a7050 con 0x7fbc401551e0 2023-11-06T03:07:25.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc37fff640 1 -- 172.21.15.157:0/3534144218 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0ab980 con 0x7fbc401551e0 2023-11-06T03:07:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- 172.21.15.157:0/3534144218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc401551e0 msgr2=0x7fbc4006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 --2- 172.21.15.157:0/3534144218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc401551e0 0x7fbc4006fda0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc400cbfe0 tx=0x7fbc3c01c310 comp rx=0 tx=0).stop 2023-11-06T03:07:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- 172.21.15.157:0/3534144218 shutdown_connections 2023-11-06T03:07:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 --2- 172.21.15.157:0/3534144218 >> v2:172.21.15.195:3300/0 conn(0x7fbc400b3b20 0x7fbc4015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 --2- 172.21.15.157:0/3534144218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc401551e0 0x7fbc4006fda0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- 172.21.15.157:0/3534144218 >> 172.21.15.157:0/3534144218 conn(0x7fbc40065930 msgr2=0x7fbc40065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- 172.21.15.157:0/3534144218 shutdown_connections 2023-11-06T03:07:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- 172.21.15.157:0/3534144218 wait complete. 2023-11-06T03:07:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 Processor -- start 2023-11-06T03:07:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.457+0000 7fbc47399640 1 -- start start 2023-11-06T03:07:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 0x7fbc4014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc401551e0 0x7fbc40153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401546b0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40156610 con 0x7fbc400b3b20 2023-11-06T03:07:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401568f0 con 0x7fbc401551e0 2023-11-06T03:07:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55638/0 (socket says 172.21.15.157:55638) 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 -- 172.21.15.157:0/2703226191 learned_addr learned my addr 172.21.15.157:0/2703226191 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45135640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 0x7fbc4014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc44934640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc401551e0 0x7fbc40153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 -- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc401551e0 msgr2=0x7fbc40153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc401551e0 0x7fbc40153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 -- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 msgr2=0x7fbc4014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 0x7fbc4014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 -- 172.21.15.157:0/2703226191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c009420 con 0x7fbc4015e5e0 2023-11-06T03:07:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45135640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 0x7fbc4014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45936640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007e40 tx=0x7fbc3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38014620 con 0x7fbc4015e5e0 2023-11-06T03:07:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800ee90 con 0x7fbc4015e5e0 2023-11-06T03:07:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 -- 172.21.15.157:0/2703226191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc40156bd0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380149e0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc47399640 1 -- 172.21.15.157:0/2703226191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc401584d0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc18005260 con 0x7fbc4015e5e0 2023-11-06T03:07:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc38014b40 con 0x7fbc4015e5e0 2023-11-06T03:07:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 0x7fbc24067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fbc38083c20 con 0x7fbc4015e5e0 2023-11-06T03:07:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.461+0000 7fbc45135640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 0x7fbc24067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.465+0000 7fbc45135640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 0x7fbc24067f20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ad000 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.465+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbc380514d0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.621+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc18005a90 con 0x7fbc4015e5e0 2023-11-06T03:07:25.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.621+0000 7fbc35ffb640 1 -- 172.21.15.157:0/2703226191 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc38054af0 con 0x7fbc4015e5e0 2023-11-06T03:07:25.621 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:07:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.621+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 msgr2=0x7fbc24067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.621+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 0x7fbc24067f20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ad000 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2023-11-06T03:07:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.621+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 msgr2=0x7fbc40158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007e40 tx=0x7fbc3800e4b0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 shutdown_connections 2023-11-06T03:07:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc24065a60 0x7fbc24067f20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc4015e5e0 0x7fbc40158ad0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc401551e0 0x7fbc40153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 --2- 172.21.15.157:0/2703226191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc400b3b20 0x7fbc4014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 >> 172.21.15.157:0/2703226191 conn(0x7fbc40065930 msgr2=0x7fbc4014e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 shutdown_connections 2023-11-06T03:07:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.625+0000 7fbc2b7fe640 1 -- 172.21.15.157:0/2703226191 wait complete. 2023-11-06T03:07:25.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:07:25.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 Processor -- start 2023-11-06T03:07:25.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 -- start start 2023-11-06T03:07:25.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa138159950 0x7fa1380b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa138159d30 con 0x7fa1380ccb90 2023-11-06T03:07:25.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380b3f00 con 0x7fa13806d7f0 2023-11-06T03:07:25.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13c99c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1380702b0 con 0x7fa138159950 2023-11-06T03:07:25.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa138159950 0x7fa1380b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa136d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa1380ccb90 0x7fa13815d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57854/0 (socket says 172.21.15.157:57854) 2023-11-06T03:07:25.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa136d9d640 1 -- 172.21.15.157:0/2178263508 learned_addr learned my addr 172.21.15.157:0/2178263508 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:25.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13659c640 1 -- 172.21.15.157:0/2178263508 >> v1:172.21.15.195:6789/0 conn(0x7fa13806d7f0 legacy=0x7fa1380cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:25.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13659c640 1 -- 172.21.15.157:0/2178263508 >> v1:172.21.15.157:6789/0 conn(0x7fa1380ccb90 legacy=0x7fa13815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.761+0000 7fa13659c640 1 -- 172.21.15.157:0/2178263508 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380704c0 con 0x7fa138159950 2023-11-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13659c640 1 --2- 172.21.15.157:0/2178263508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa1380b39c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c36e946e175d1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13559a640 1 -- 172.21.15.157:0/2178263508 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280ab600 con 0x7fa138159950 2023-11-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13559a640 1 -- 172.21.15.157:0/2178263508 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1280a7050 con 0x7fa138159950 2023-11-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13559a640 1 -- 172.21.15.157:0/2178263508 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280ab980 con 0x7fa138159950 2023-11-06T03:07:25.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- 172.21.15.157:0/2178263508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 msgr2=0x7fa1380b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 --2- 172.21.15.157:0/2178263508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa1380b39c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c2d0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- 172.21.15.157:0/2178263508 shutdown_connections 2023-11-06T03:07:25.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 --2- 172.21.15.157:0/2178263508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa1380b39c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- 172.21.15.157:0/2178263508 >> 172.21.15.157:0/2178263508 conn(0x7fa13806a170 msgr2=0x7fa13806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- 172.21.15.157:0/2178263508 shutdown_connections 2023-11-06T03:07:25.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- 172.21.15.157:0/2178263508 wait complete. 2023-11-06T03:07:25.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 Processor -- start 2023-11-06T03:07:25.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- start start 2023-11-06T03:07:25.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 0x7fa1381f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1380ccb90 0x7fa13814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381520f0 con 0x7fa138159950 2023-11-06T03:07:25.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa138151520 con 0x7fa13806d7f0 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.765+0000 7fa13c99c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa138151750 con 0x7fa1380ccb90 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55646/0 (socket says 172.21.15.157:55646) 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 -- 172.21.15.157:0/2597486328 learned_addr learned my addr 172.21.15.157:0/2597486328 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13659c640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 0x7fa1381f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 -- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1380ccb90 msgr2=0x7fa13814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa135d9b640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1380ccb90 0x7fa13814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1380ccb90 0x7fa13814f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 -- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 msgr2=0x7fa1381f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 0x7fa1381f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 -- 172.21.15.157:0/2597486328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009420 con 0x7fa138159950 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13659c640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 0x7fa1381f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa136d9d640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa13000e7b0 tx=0x7fa13000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000f930 con 0x7fa138159950 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13000fa90 con 0x7fa138159950 2023-11-06T03:07:25.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13c99c640 1 -- 172.21.15.157:0/2597486328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa138151a30 con 0x7fa138159950 2023-11-06T03:07:25.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000fe00 con 0x7fa138159950 2023-11-06T03:07:25.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13c99c640 1 -- 172.21.15.157:0/2597486328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa138150050 con 0x7fa138159950 2023-11-06T03:07:25.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa110005260 con 0x7fa138159950 2023-11-06T03:07:25.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa130015070 con 0x7fa138159950 2023-11-06T03:07:25.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 0x7fa118067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:25.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fa130085c60 con 0x7fa138159950 2023-11-06T03:07:25.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13659c640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 0x7fa118067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:25.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.769+0000 7fa13659c640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 0x7fa118067f20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1280ad000 tx=0x7fa128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:25.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.773+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa130053510 con 0x7fa138159950 2023-11-06T03:07:25.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa110005a90 con 0x7fa138159950 2023-11-06T03:07:25.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa126ffd640 1 -- 172.21.15.157:0/2597486328 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa130056b30 con 0x7fa138159950 2023-11-06T03:07:25.924 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:07:25.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 msgr2=0x7fa118067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 0x7fa118067f20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1280ad000 tx=0x7fa128008040 comp rx=0 tx=0).stop 2023-11-06T03:07:25.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 msgr2=0x7fa138065b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:25.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa13000e7b0 tx=0x7fa13000ec70 comp rx=0 tx=0).stop 2023-11-06T03:07:25.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 shutdown_connections 2023-11-06T03:07:25.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa118065a60 0x7fa118067f20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa138159950 0x7fa138065b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa1380ccb90 0x7fa13814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 --2- 172.21.15.157:0/2597486328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa13806d7f0 0x7fa1381f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:25.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 >> 172.21.15.157:0/2597486328 conn(0x7fa13806a170 msgr2=0x7fa1380cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:25.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 shutdown_connections 2023-11-06T03:07:25.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:25.925+0000 7fa124ff9640 1 -- 172.21.15.157:0/2597486328 wait complete. 2023-11-06T03:07:25.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:07:26.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.053+0000 7f5cf780f640 1 Processor -- start 2023-11-06T03:07:26.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.053+0000 7f5cf780f640 1 -- start start 2023-11-06T03:07:26.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf0156a70 0x7f5cf006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 0x7f5cf00b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5cf00adf40 0x7f5cf00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf006b950 con 0x7f5cf00adf40 2023-11-06T03:07:26.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf006c490 con 0x7f5cf00adae0 2023-11-06T03:07:26.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf780f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf006bb60 con 0x7f5cf0156a70 2023-11-06T03:07:26.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5cf00adf40 0x7f5cf00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5cf00adf40 0x7f5cf00b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55656/0 (socket says 172.21.15.157:55656) 2023-11-06T03:07:26.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 -- 172.21.15.157:0/2447740950 learned_addr learned my addr 172.21.15.157:0/2447740950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:26.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf4daa640 1 --2- 172.21.15.157:0/2447740950 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 0x7f5cf00b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf55ab640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf0156a70 0x7f5cf006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 -- 172.21.15.157:0/2447740950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf0156a70 msgr2=0x7f5cf006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf0156a70 0x7f5cf006f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 -- 172.21.15.157:0/2447740950 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 msgr2=0x7f5cf00b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 --2- 172.21.15.157:0/2447740950 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 0x7f5cf00b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 -- 172.21.15.157:0/2447740950 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf006be40 con 0x7f5cf00adf40 2023-11-06T03:07:26.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf4daa640 1 --2- 172.21.15.157:0/2447740950 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 0x7f5cf00b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:26.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5cf5dac640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adf40 0x7f5cf00b2160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00668b0 tx=0x7f5ce801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ccc8ce7ff682f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5ce7fff640 1 -- 172.21.15.157:0/2447740950 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80a57c0 con 0x7f5cf00adf40 2023-11-06T03:07:26.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5ce7fff640 1 -- 172.21.15.157:0/2447740950 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce80a5920 con 0x7f5cf00adf40 2023-11-06T03:07:26.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.057+0000 7f5ce7fff640 1 -- 172.21.15.157:0/2447740950 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80a5cb0 con 0x7f5cf00adf40 2023-11-06T03:07:26.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/2447740950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adf40 msgr2=0x7f5cf00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adf40 0x7f5cf00b2160 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00668b0 tx=0x7f5ce801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/2447740950 shutdown_connections 2023-11-06T03:07:26.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adf40 0x7f5cf00b2160 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- 172.21.15.157:0/2447740950 >> v2:172.21.15.195:3300/0 conn(0x7f5cf00adae0 0x7f5cf00b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- 172.21.15.157:0/2447740950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf0156a70 0x7f5cf006f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/2447740950 >> 172.21.15.157:0/2447740950 conn(0x7f5cf0065930 msgr2=0x7f5cf0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/2447740950 shutdown_connections 2023-11-06T03:07:26.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/2447740950 wait complete. 2023-11-06T03:07:26.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 Processor -- start 2023-11-06T03:07:26.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- start start 2023-11-06T03:07:26.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00adf40 0x7f5cf015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0156a70 0x7f5cf0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015e060 con 0x7f5cf00adae0 2023-11-06T03:07:26.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015e340 con 0x7f5cf0156a70 2023-11-06T03:07:26.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015e620 con 0x7f5cf00adf40 2023-11-06T03:07:26.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55664/0 (socket says 172.21.15.157:55664) 2023-11-06T03:07:26.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 -- 172.21.15.157:0/549582648 learned_addr learned my addr 172.21.15.157:0/549582648 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:26.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf5dac640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0156a70 0x7f5cf0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf4daa640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00adf40 0x7f5cf015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 -- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00adf40 msgr2=0x7f5cf015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00adf40 0x7f5cf015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 -- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0156a70 msgr2=0x7f5cf0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0156a70 0x7f5cf0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 -- 172.21.15.157:0/549582648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce8009b30 con 0x7f5cf00adae0 2023-11-06T03:07:26.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf55ab640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00d6d0 tx=0x7f5cec00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec007900 con 0x7f5cf00adae0 2023-11-06T03:07:26.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec007a60 con 0x7f5cf00adae0 2023-11-06T03:07:26.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf015e900 con 0x7f5cf00adae0 2023-11-06T03:07:26.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec007e10 con 0x7f5cf00adae0 2023-11-06T03:07:26.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf015dc80 con 0x7f5cf00adae0 2023-11-06T03:07:26.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.061+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf0071640 con 0x7f5cf00adae0 2023-11-06T03:07:26.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.065+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5cec011460 con 0x7f5cf00adae0 2023-11-06T03:07:26.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.065+0000 7f5ce5ffb640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 0x7f5cd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.065+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f5cec0834b0 con 0x7f5cf00adae0 2023-11-06T03:07:26.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.065+0000 7f5cf4daa640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 0x7f5cd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.065+0000 7f5cf4daa640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 0x7f5cd4067f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00b0b20 tx=0x7f5ce0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.069+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5cec050cd0 con 0x7f5cf00adae0 2023-11-06T03:07:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.217+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5cf0072750 con 0x7f5cf00adae0 2023-11-06T03:07:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.217+0000 7f5ce5ffb640 1 -- 172.21.15.157:0/549582648 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5cec00b040 con 0x7f5cf00adae0 2023-11-06T03:07:26.218 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:07:26.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 msgr2=0x7f5cd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 0x7f5cd4067f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00b0b20 tx=0x7f5ce0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:26.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 msgr2=0x7f5cf015a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00d6d0 tx=0x7f5cec00db90 comp rx=0 tx=0).stop 2023-11-06T03:07:26.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 shutdown_connections 2023-11-06T03:07:26.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd4065a60 0x7f5cd4067f20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0156a70 0x7f5cf0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf00adf40 0x7f5cf015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 --2- 172.21.15.157:0/549582648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cf00adae0 0x7f5cf015a6b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 >> 172.21.15.157:0/549582648 conn(0x7f5cf0065930 msgr2=0x7f5cf00b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 shutdown_connections 2023-11-06T03:07:26.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.221+0000 7f5cf780f640 1 -- 172.21.15.157:0/549582648 wait complete. 2023-11-06T03:07:26.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.345+0000 7f6ae2c41640 1 Processor -- start 2023-11-06T03:07:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.345+0000 7f6ae2c41640 1 -- start start 2023-11-06T03:07:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae2c41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 0x7f6adc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae2c41640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc15f580 con 0x7f6adc15e5e0 2023-11-06T03:07:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae2c41640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc15efc0 con 0x7f6adc0b3b20 2023-11-06T03:07:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae2c41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc15f2a0 con 0x7f6adc1551e0 2023-11-06T03:07:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6adc15e5e0 0x7f6adc0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57858/0 (socket says 172.21.15.157:57858) 2023-11-06T03:07:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adbfff640 1 -- 172.21.15.157:0/564371872 learned_addr learned my addr 172.21.15.157:0/564371872 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 0x7f6adc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290336963 0 0) 0x7f6adc15f580 con 0x7f6adc15e5e0 2023-11-06T03:07:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc003530 con 0x7f6adc15e5e0 2023-11-06T03:07:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1275035779 0 0) 0x7f6acc003530 con 0x7f6adc15e5e0 2023-11-06T03:07:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 msgr2=0x7f6adc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 0x7f6adc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 >> v1:172.21.15.195:6789/0 conn(0x7f6adc0b3b20 legacy=0x7f6adc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc06f5a0 con 0x7f6adc15e5e0 2023-11-06T03:07:26.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6ac8001680 con 0x7f6adc15e5e0 2023-11-06T03:07:26.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 >> v1:172.21.15.157:6789/0 conn(0x7f6adc15e5e0 legacy=0x7f6adc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6acc0039b0 0x7f6acc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 0x7f6acc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6acc004d10 0x7f6acc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc15f2a0 con 0x7f6acc004d10 2023-11-06T03:07:26.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc00bb20 con 0x7f6acc0039b0 2023-11-06T03:07:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc00bce0 con 0x7f6adc1551e0 2023-11-06T03:07:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6acc004d10 0x7f6acc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6acc0039b0 0x7f6acc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 -- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 msgr2=0x7f6acc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6adc1551e0 0x7f6acc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 -- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6acc0039b0 msgr2=0x7f6acc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6acc0039b0 0x7f6acc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 -- 172.21.15.157:0/564371872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc003530 con 0x7f6acc004d10 2023-11-06T03:07:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6acc0039b0 0x7f6acc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:26.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6ae11de640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6acc004d10 0x7f6acc005190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4001e40 tx=0x7f6ad401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538af0e7bf71b75b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4003b80 con 0x7f6acc004d10 2023-11-06T03:07:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4003ce0 con 0x7f6acc004d10 2023-11-06T03:07:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.349+0000 7f6adb7fe640 1 -- 172.21.15.157:0/564371872 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40052c0 con 0x7f6acc004d10 2023-11-06T03:07:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/564371872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6acc004d10 msgr2=0x7f6acc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/564371872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6acc004d10 0x7f6acc005190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4001e40 tx=0x7f6ad401c740 comp rx=0 tx=0).stop 2023-11-06T03:07:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 -- 172.21.15.157:0/564371872 reap_dead start 2023-11-06T03:07:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/564371872 shutdown_connections 2023-11-06T03:07:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/564371872 >> 172.21.15.157:0/564371872 conn(0x7f6adc065930 msgr2=0x7f6adc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/564371872 shutdown_connections 2023-11-06T03:07:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/564371872 wait complete. 2023-11-06T03:07:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 Processor -- start 2023-11-06T03:07:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- start start 2023-11-06T03:07:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6acc004d10 0x7f6adc0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6adc152320 0x7f6adc143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc143690 con 0x7f6adc1f5c70 2023-11-06T03:07:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1f6ba0 con 0x7f6adc152320 2023-11-06T03:07:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1f65e0 con 0x7f6acc004d10 2023-11-06T03:07:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55684/0 (socket says 172.21.15.157:55684) 2023-11-06T03:07:26.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 -- 172.21.15.157:0/4013035999 learned_addr learned my addr 172.21.15.157:0/4013035999 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:26.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6adbfff640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6adc152320 0x7f6adc143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6acc004d10 0x7f6adc0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 -- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6acc004d10 msgr2=0x7f6adc0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6acc004d10 0x7f6adc0add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 -- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6adc152320 msgr2=0x7f6adc143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6adc152320 0x7f6adc143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 -- 172.21.15.157:0/4013035999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc1f68c0 con 0x7f6adc1f5c70 2023-11-06T03:07:26.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae11de640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401cc20 tx=0x7f6acc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4006d60 con 0x7f6adc1f5c70 2023-11-06T03:07:26.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad40012e0 con 0x7f6adc1f5c70 2023-11-06T03:07:26.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc143320 con 0x7f6adc1f5c70 2023-11-06T03:07:26.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4005230 con 0x7f6adc1f5c70 2023-11-06T03:07:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc144060 con 0x7f6adc1f5c70 2023-11-06T03:07:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.353+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc143840 con 0x7f6adc1f5c70 2023-11-06T03:07:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.357+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ad4005390 con 0x7f6adc1f5c70 2023-11-06T03:07:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.357+0000 7f6ad97fa640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 0x7f6ac0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.357+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f6ad411c3f0 con 0x7f6adc1f5c70 2023-11-06T03:07:26.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.357+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 0x7f6ac0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.357+0000 7f6ae09dd640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 0x7f6ac0067f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ac0 tx=0x7f6ad0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.361+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6adc143840 con 0x7f6adc1f5c70 2023-11-06T03:07:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.545+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6adc072750 con 0x7f6adc1f5c70 2023-11-06T03:07:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.545+0000 7f6ad97fa640 1 -- 172.21.15.157:0/4013035999 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f6ad40e9c80 con 0x7f6adc1f5c70 2023-11-06T03:07:26.545 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:26.545 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:07:11.278907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":32,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:07:26.546 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-11-06T03:07:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 msgr2=0x7f6ac0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 0x7f6ac0067f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ac0 tx=0x7f6ad0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 msgr2=0x7f6adc15c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401cc20 tx=0x7f6acc005800 comp rx=0 tx=0).stop 2023-11-06T03:07:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 shutdown_connections 2023-11-06T03:07:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ac0065a90 0x7f6ac0067f50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc1f5c70 0x7f6adc15c120 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6adc152320 0x7f6adc143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 --2- 172.21.15.157:0/4013035999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6acc004d10 0x7f6adc0add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 >> 172.21.15.157:0/4013035999 conn(0x7f6adc065930 msgr2=0x7f6adc0b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 shutdown_connections 2023-11-06T03:07:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.549+0000 7f6ae2c41640 1 -- 172.21.15.157:0/4013035999 wait complete. 2023-11-06T03:07:26.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:26.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 Processor -- start 2023-11-06T03:07:26.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 -- start start 2023-11-06T03:07:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f37f4146370 0x7f37f40b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f37f406bdb0 0x7f37f40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40cf1a0 con 0x7f37f40ce800 2023-11-06T03:07:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f406c3d0 con 0x7f37f4146370 2023-11-06T03:07:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f9913640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f406f5a0 con 0x7f37f406bdb0 2023-11-06T03:07:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f37f4146370 0x7f37f40b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f27fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f37f40ce800 0x7f37f406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57866/0 (socket says 172.21.15.157:57866) 2023-11-06T03:07:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f27fc640 1 -- 172.21.15.157:0/1367690326 learned_addr learned my addr 172.21.15.157:0/1367690326 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f37fe640 1 --2- 172.21.15.157:0/1367690326 >> v2:172.21.15.195:3300/0 conn(0x7f37f406bdb0 0x7f37f40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f2ffd640 1 -- 172.21.15.157:0/1367690326 >> v2:172.21.15.195:3300/0 conn(0x7f37f406bdb0 msgr2=0x7f37f40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f2ffd640 1 --2- 172.21.15.157:0/1367690326 >> v2:172.21.15.195:3300/0 conn(0x7f37f406bdb0 0x7f37f40cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f2ffd640 1 -- 172.21.15.157:0/1367690326 >> v1:172.21.15.157:6789/0 conn(0x7f37f40ce800 legacy=0x7f37f406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.681+0000 7f37f2ffd640 1 -- 172.21.15.157:0/1367690326 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f406f880 con 0x7f37f4146370 2023-11-06T03:07:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f2ffd640 1 --2- 172.21.15.157:0/1367690326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f4146370 0x7f37f40b1a10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0097c0 tx=0x7f37ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7337e198e6ced1a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f1ffb640 1 -- 172.21.15.157:0/1367690326 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0ab600 con 0x7f37f4146370 2023-11-06T03:07:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f1ffb640 1 -- 172.21.15.157:0/1367690326 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec0a7050 con 0x7f37f4146370 2023-11-06T03:07:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f1ffb640 1 -- 172.21.15.157:0/1367690326 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0ab980 con 0x7f37f4146370 2023-11-06T03:07:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- 172.21.15.157:0/1367690326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f4146370 msgr2=0x7f37f40b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 --2- 172.21.15.157:0/1367690326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f4146370 0x7f37f40b1a10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0097c0 tx=0x7f37ec01c380 comp rx=0 tx=0).stop 2023-11-06T03:07:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- 172.21.15.157:0/1367690326 shutdown_connections 2023-11-06T03:07:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 --2- 172.21.15.157:0/1367690326 >> v2:172.21.15.195:3300/0 conn(0x7f37f406bdb0 0x7f37f40cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 --2- 172.21.15.157:0/1367690326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f4146370 0x7f37f40b1a10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- 172.21.15.157:0/1367690326 >> 172.21.15.157:0/1367690326 conn(0x7f37f4065930 msgr2=0x7f37f4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- 172.21.15.157:0/1367690326 shutdown_connections 2023-11-06T03:07:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- 172.21.15.157:0/1367690326 wait complete. 2023-11-06T03:07:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 Processor -- start 2023-11-06T03:07:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.685+0000 7f37f9913640 1 -- start start 2023-11-06T03:07:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f406bdb0 0x7f37f413a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4146370 0x7f37f40b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f4161a90 con 0x7f37f40ce800 2023-11-06T03:07:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f4165af0 con 0x7f37f406bdb0 2023-11-06T03:07:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f4165dd0 con 0x7f37f4146370 2023-11-06T03:07:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55700/0 (socket says 172.21.15.157:55700) 2023-11-06T03:07:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 -- 172.21.15.157:0/1850670321 learned_addr learned my addr 172.21.15.157:0/1850670321 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f2ffd640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f406bdb0 0x7f37f413a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f37fe640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4146370 0x7f37f40b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 -- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4146370 msgr2=0x7f37f40b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4146370 0x7f37f40b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 -- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f406bdb0 msgr2=0x7f37f413a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f406bdb0 0x7f37f413a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 -- 172.21.15.157:0/1850670321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009420 con 0x7f37f40ce800 2023-11-06T03:07:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f27fc640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c400 tx=0x7f37dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc005cf0 con 0x7f37f40ce800 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc005e50 con 0x7f37f40ce800 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f41660b0 con 0x7f37f40ce800 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0072d0 con 0x7f37f40ce800 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f413a930 con 0x7f37f40ce800 2023-11-06T03:07:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f41662f0 con 0x7f37f40ce800 2023-11-06T03:07:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f37dc007430 con 0x7f37f40ce800 2023-11-06T03:07:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 0x7f37d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f37dc0828e0 con 0x7f37f40ce800 2023-11-06T03:07:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f2ffd640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 0x7f37d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.689+0000 7f37f2ffd640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 0x7f37d4067f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0ad000 tx=0x7f37ec0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.693+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f37dc050120 con 0x7f37f40ce800 2023-11-06T03:07:26.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.837+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37f4072750 con 0x7f37f40ce800 2023-11-06T03:07:26.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.841+0000 7f37ebfff640 1 -- 172.21.15.157:0/1850670321 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18045 (secure 0 0 0) 0x7f37dc017020 con 0x7f37f40ce800 2023-11-06T03:07:26.841 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:26.842 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":41,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:06:57.240474+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":40,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:26.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 msgr2=0x7f37d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 0x7f37d4067f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0ad000 tx=0x7f37ec0a6000 comp rx=0 tx=0).stop 2023-11-06T03:07:26.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 msgr2=0x7f37f40b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c400 tx=0x7f37dc00c8c0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 shutdown_connections 2023-11-06T03:07:26.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f37d4065a60 0x7f37d4067f20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f37f4146370 0x7f37f40b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37f40ce800 0x7f37f40b2250 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 --2- 172.21.15.157:0/1850670321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37f406bdb0 0x7f37f413a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 >> 172.21.15.157:0/1850670321 conn(0x7f37f4065930 msgr2=0x7f37f40b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 shutdown_connections 2023-11-06T03:07:26.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.845+0000 7f37f9913640 1 -- 172.21.15.157:0/1850670321 wait complete. 2023-11-06T03:07:26.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:07:26.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:26.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.965+0000 7f3be8143640 1 Processor -- start 2023-11-06T03:07:26.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.965+0000 7f3be8143640 1 -- start start 2023-11-06T03:07:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be8143640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3be00ce800 0x7f3be00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be8143640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be00cf1a0 con 0x7f3be0146370 2023-11-06T03:07:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be8143640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be006c3d0 con 0x7f3be006b6a0 2023-11-06T03:07:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be8143640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be00b2120 con 0x7f3be00ce800 2023-11-06T03:07:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3be00ce800 0x7f3be00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be5edf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3be0146370 0x7f3be00b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57868/0 (socket says 172.21.15.157:57868) 2023-11-06T03:07:26.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be5edf640 1 -- 172.21.15.157:0/3300970841 learned_addr learned my addr 172.21.15.157:0/3300970841 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:26.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3be00ce800 0x7f3be00cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55716/0 (socket says 172.21.15.157:55716) 2023-11-06T03:07:26.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 -- 172.21.15.157:0/3300970841 >> v1:172.21.15.195:6789/0 conn(0x7f3be006b6a0 legacy=0x7f3be006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:26.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 -- 172.21.15.157:0/3300970841 >> v1:172.21.15.157:6789/0 conn(0x7f3be0146370 legacy=0x7f3be00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be4edd640 1 -- 172.21.15.157:0/3300970841 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160380283 0 0) 0x7f3be00cf1a0 con 0x7f3be0146370 2023-11-06T03:07:26.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 -- 172.21.15.157:0/3300970841 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be006f5a0 con 0x7f3be00ce800 2023-11-06T03:07:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be56de640 1 --2- 172.21.15.157:0/3300970841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be00cec60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00097f0 tx=0x7f3bd001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d2c56961f522a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be4edd640 1 -- 172.21.15.157:0/3300970841 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00ad680 con 0x7f3be00ce800 2023-11-06T03:07:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be4edd640 1 -- 172.21.15.157:0/3300970841 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd00ad7e0 con 0x7f3be00ce800 2023-11-06T03:07:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.969+0000 7f3be4edd640 1 -- 172.21.15.157:0/3300970841 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00adb70 con 0x7f3be00ce800 2023-11-06T03:07:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3300970841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 msgr2=0x7f3be00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 --2- 172.21.15.157:0/3300970841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be00cec60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00097f0 tx=0x7f3bd001c380 comp rx=0 tx=0).stop 2023-11-06T03:07:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3300970841 shutdown_connections 2023-11-06T03:07:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 --2- 172.21.15.157:0/3300970841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be00cec60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3300970841 >> 172.21.15.157:0/3300970841 conn(0x7f3be0065930 msgr2=0x7f3be0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3300970841 shutdown_connections 2023-11-06T03:07:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3300970841 wait complete. 2023-11-06T03:07:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 Processor -- start 2023-11-06T03:07:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- start start 2023-11-06T03:07:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 0x7f3be015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 0x7f3be0141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be0158190 con 0x7f3be00ce800 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be015b6f0 con 0x7f3be006b6a0 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be015e7f0 con 0x7f3be0146370 2023-11-06T03:07:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55730/0 (socket says 172.21.15.157:55730) 2023-11-06T03:07:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 -- 172.21.15.157:0/3443272784 learned_addr learned my addr 172.21.15.157:0/3443272784 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be5edf640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 0x7f3be015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be66e0640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 0x7f3be0141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 -- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 msgr2=0x7f3be0141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 0x7f3be0141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 -- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 msgr2=0x7f3be015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 0x7f3be015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 -- 172.21.15.157:0/3443272784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0009420 con 0x7f3be00ce800 2023-11-06T03:07:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be5edf640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 0x7f3be015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be66e0640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 0x7f3be0141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be56de640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0097e50 tx=0x7f3bd0019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00ade20 con 0x7f3be00ce800 2023-11-06T03:07:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd0004030 con 0x7f3be00ce800 2023-11-06T03:07:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be01583e0 con 0x7f3be00ce800 2023-11-06T03:07:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00043a0 con 0x7f3be00ce800 2023-11-06T03:07:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.973+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be01589a0 con 0x7f3be00ce800 2023-11-06T03:07:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be0072750 con 0x7f3be00ce800 2023-11-06T03:07:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3bd00b4040 con 0x7f3be00ce800 2023-11-06T03:07:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3bd67fc640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 0x7f3bc4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:26.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f3bd011d400 con 0x7f3be00ce800 2023-11-06T03:07:26.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3be5edf640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 0x7f3bc4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:26.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.977+0000 7f3be5edf640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 0x7f3bc4067f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc000f70 tx=0x7f3bdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:26.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:26.981+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3bd00ead40 con 0x7f3be00ce800 2023-11-06T03:07:27.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.153+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3be00ad850 con 0x7f3be00ce800 2023-11-06T03:07:27.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3bd67fc640 1 -- 172.21.15.157:0/3443272784 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f3bd00ee360 con 0x7f3be00ce800 2023-11-06T03:07:27.156 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:27.156 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:07:11.278907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":32,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:07:27.156 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-11-06T03:07:27.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 msgr2=0x7f3bc4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 0x7f3bc4067f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc000f70 tx=0x7f3bdc009040 comp rx=0 tx=0).stop 2023-11-06T03:07:27.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 msgr2=0x7f3be013c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0097e50 tx=0x7f3bd0019120 comp rx=0 tx=0).stop 2023-11-06T03:07:27.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 shutdown_connections 2023-11-06T03:07:27.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bc4065a60 0x7f3bc4067f20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3be0146370 0x7f3be0141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3be00ce800 0x7f3be013c310 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 --2- 172.21.15.157:0/3443272784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3be006b6a0 0x7f3be015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 >> 172.21.15.157:0/3443272784 conn(0x7f3be0065930 msgr2=0x7f3be0070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 shutdown_connections 2023-11-06T03:07:27.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.157+0000 7f3be8143640 1 -- 172.21.15.157:0/3443272784 wait complete. 2023-11-06T03:07:27.173 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.281+0000 7f627f6fd640 1 Processor -- start 2023-11-06T03:07:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.281+0000 7f627f6fd640 1 -- start start 2023-11-06T03:07:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 0x7f62780d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 0x7f62780cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62780cfbf0 0x7f62780b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780d0080 con 0x7f62780cfbf0 2023-11-06T03:07:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780d1c60 con 0x7f62780d1720 2023-11-06T03:07:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62780d1f40 con 0x7f627815cdf0 2023-11-06T03:07:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62780cfbf0 0x7f62780b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f62780cfbf0 0x7f62780b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55746/0 (socket says 172.21.15.157:55746) 2023-11-06T03:07:27.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 -- 172.21.15.157:0/3891251998 learned_addr learned my addr 172.21.15.157:0/3891251998 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627d499640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 0x7f62780d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627cc98640 1 --2- 172.21.15.157:0/3891251998 >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 0x7f62780cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 -- 172.21.15.157:0/3891251998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 msgr2=0x7f62780d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 0x7f62780d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 -- 172.21.15.157:0/3891251998 >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 msgr2=0x7f62780cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 --2- 172.21.15.157:0/3891251998 >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 0x7f62780cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 -- 172.21.15.157:0/3891251998 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780d2220 con 0x7f62780cfbf0 2023-11-06T03:07:27.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627d499640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 0x7f62780d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:27.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627cc98640 1 --2- 172.21.15.157:0/3891251998 >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 0x7f62780cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:27.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627dc9a640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780cfbf0 0x7f62780b4720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62780b5800 tx=0x7f627001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d098e0c7873a76a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f626ffff640 1 -- 172.21.15.157:0/3891251998 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62700a5560 con 0x7f62780cfbf0 2023-11-06T03:07:27.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f626ffff640 1 -- 172.21.15.157:0/3891251998 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62700a56c0 con 0x7f62780cfbf0 2023-11-06T03:07:27.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f626ffff640 1 -- 172.21.15.157:0/3891251998 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62700a5a50 con 0x7f62780cfbf0 2023-11-06T03:07:27.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- 172.21.15.157:0/3891251998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780cfbf0 msgr2=0x7f62780b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780cfbf0 0x7f62780b4720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62780b5800 tx=0x7f627001c6e0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- 172.21.15.157:0/3891251998 shutdown_connections 2023-11-06T03:07:27.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780cfbf0 0x7f62780b4720 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- 172.21.15.157:0/3891251998 >> v2:172.21.15.195:3300/0 conn(0x7f62780d1720 0x7f62780cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 --2- 172.21.15.157:0/3891251998 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f627815cdf0 0x7f62780d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- 172.21.15.157:0/3891251998 >> 172.21.15.157:0/3891251998 conn(0x7f6278065930 msgr2=0x7f6278065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- 172.21.15.157:0/3891251998 shutdown_connections 2023-11-06T03:07:27.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.285+0000 7f627f6fd640 1 -- 172.21.15.157:0/3891251998 wait complete. 2023-11-06T03:07:27.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 Processor -- start 2023-11-06T03:07:27.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- start start 2023-11-06T03:07:27.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62780cfbf0 0x7f627814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f627815cdf0 0x7f627814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62780c74b0 con 0x7f62780d1720 2023-11-06T03:07:27.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278150730 con 0x7f627815cdf0 2023-11-06T03:07:27.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627814fe90 con 0x7f62780cfbf0 2023-11-06T03:07:27.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55756/0 (socket says 172.21.15.157:55756) 2023-11-06T03:07:27.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 -- 172.21.15.157:0/385787167 learned_addr learned my addr 172.21.15.157:0/385787167 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:27.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627d499640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62780cfbf0 0x7f627814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 -- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62780cfbf0 msgr2=0x7f627814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62780cfbf0 0x7f627814f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627dc9a640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f627815cdf0 0x7f627814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 -- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f627815cdf0 msgr2=0x7f627814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f627815cdf0 0x7f627814ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 -- 172.21.15.157:0/385787167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270009b30 con 0x7f62780d1720 2023-11-06T03:07:27.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627cc98640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f626800c7c0 tx=0x7f626800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268007a30 con 0x7f62780d1720 2023-11-06T03:07:27.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268007b90 con 0x7f62780d1720 2023-11-06T03:07:27.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6278150170 con 0x7f62780d1720 2023-11-06T03:07:27.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800ce70 con 0x7f62780d1720 2023-11-06T03:07:27.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62780cb0c0 con 0x7f62780d1720 2023-11-06T03:07:27.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.289+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62780cb2e0 con 0x7f62780d1720 2023-11-06T03:07:27.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.293+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f62680113f0 con 0x7f62780d1720 2023-11-06T03:07:27.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.293+0000 7f626dffb640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 0x7f625c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.293+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f62680833c0 con 0x7f62780d1720 2023-11-06T03:07:27.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.293+0000 7f627d499640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 0x7f625c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.293+0000 7f627d499640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 0x7f625c067f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6274006fd0 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.297+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6268050c70 con 0x7f62780d1720 2023-11-06T03:07:27.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.481+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6278072750 con 0x7f62780d1720 2023-11-06T03:07:27.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.481+0000 7f626dffb640 1 -- 172.21.15.157:0/385787167 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f6268054290 con 0x7f62780d1720 2023-11-06T03:07:27.482 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":28,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:05:14.602320+0000","modified":"2023-11-06T03:07:11.278907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":32,"addr":"172.21.15.157:6835/4193594648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":4193594648},{"type":"v1","addr":"172.21.15.157:6835","nonce":4193594648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-06T03:07:27.483 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 28 2023-11-06T03:07:27.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 msgr2=0x7f625c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 0x7f625c067f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6274006fd0 tx=0x7f6274008040 comp rx=0 tx=0).stop 2023-11-06T03:07:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 msgr2=0x7f62780c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f626800c7c0 tx=0x7f626800cc80 comp rx=0 tx=0).stop 2023-11-06T03:07:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 shutdown_connections 2023-11-06T03:07:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f625c065a60 0x7f625c067f20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f627815cdf0 0x7f627814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62780d1720 0x7f62780c6f70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 --2- 172.21.15.157:0/385787167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62780cfbf0 0x7f627814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 >> 172.21.15.157:0/385787167 conn(0x7f6278065930 msgr2=0x7f62780b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 shutdown_connections 2023-11-06T03:07:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.485+0000 7f627f6fd640 1 -- 172.21.15.157:0/385787167 wait complete. 2023-11-06T03:07:27.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.617+0000 7fe45a793640 1 Processor -- start 2023-11-06T03:07:27.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.617+0000 7fe45a793640 1 -- start start 2023-11-06T03:07:27.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe45a793640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 0x7fe4540b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe45a793640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45415f580 con 0x7fe45415ea60 2023-11-06T03:07:27.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe45a793640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45415efc0 con 0x7fe4541551e0 2023-11-06T03:07:27.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe45a793640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540704b0 con 0x7fe45415e5e0 2023-11-06T03:07:27.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe458d30640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe45415ea60 0x7fe45415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57880/0 (socket says 172.21.15.157:57880) 2023-11-06T03:07:27.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe458d30640 1 -- 172.21.15.157:0/2922669996 learned_addr learned my addr 172.21.15.157:0/2922669996 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:27.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 0x7fe4540b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739619554 0 0) 0x7fe45415f580 con 0x7fe45415ea60 2023-11-06T03:07:27.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe43c003530 con 0x7fe45415ea60 2023-11-06T03:07:27.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699057336 0 0) 0x7fe43c003530 con 0x7fe45415ea60 2023-11-06T03:07:27.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 msgr2=0x7fe4540b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 0x7fe4540b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 >> v1:172.21.15.195:6789/0 conn(0x7fe4541551e0 legacy=0x7fe45406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45415f2a0 con 0x7fe45415ea60 2023-11-06T03:07:27.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe44c001a00 con 0x7fe45415ea60 2023-11-06T03:07:27.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 >> v1:172.21.15.157:6789/0 conn(0x7fe45415ea60 legacy=0x7fe45415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 0x7fe43c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c003f30 0x7fe43c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe43c004d10 0x7fe43c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540704b0 con 0x7fe43c003f30 2023-11-06T03:07:27.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00bb20 con 0x7fe43c004d10 2023-11-06T03:07:27.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00bce0 con 0x7fe45415e5e0 2023-11-06T03:07:27.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c003f30 0x7fe43c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 -- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 msgr2=0x7fe43c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45415e5e0 0x7fe43c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 -- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe43c004d10 msgr2=0x7fe43c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe43c004d10 0x7fe43c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 -- 172.21.15.157:0/2922669996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c003530 con 0x7fe43c003f30 2023-11-06T03:07:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe4537fe640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c003f30 0x7fe43c0047d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe448009ae0 tx=0x7fe44801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178de1c9e98dfc34 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a5540 con 0x7fe43c003f30 2023-11-06T03:07:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480a56a0 con 0x7fe43c003f30 2023-11-06T03:07:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.621+0000 7fe452ffd640 1 -- 172.21.15.157:0/2922669996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a5a50 con 0x7fe43c003f30 2023-11-06T03:07:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/2922669996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c003f30 msgr2=0x7fe43c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 --2- 172.21.15.157:0/2922669996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe43c003f30 0x7fe43c0047d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe448009ae0 tx=0x7fe44801ca60 comp rx=0 tx=0).stop 2023-11-06T03:07:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 -- 172.21.15.157:0/2922669996 reap_dead start 2023-11-06T03:07:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/2922669996 shutdown_connections 2023-11-06T03:07:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/2922669996 >> 172.21.15.157:0/2922669996 conn(0x7fe454065930 msgr2=0x7fe454065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/2922669996 shutdown_connections 2023-11-06T03:07:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/2922669996 wait complete. 2023-11-06T03:07:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 Processor -- start 2023-11-06T03:07:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- start start 2023-11-06T03:07:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe43c003f30 0x7fe454158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4540b1f10 0x7fe45414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454156ac0 con 0x7fe454154190 2023-11-06T03:07:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4541f3dd0 con 0x7fe4540b1f10 2023-11-06T03:07:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe454154b50 con 0x7fe43c003f30 2023-11-06T03:07:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55762/0 (socket says 172.21.15.157:55762) 2023-11-06T03:07:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 -- 172.21.15.157:0/381423907 learned_addr learned my addr 172.21.15.157:0/381423907 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe4537fe640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4540b1f10 0x7fe45414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe453fff640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe43c003f30 0x7fe454158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 -- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe43c003f30 msgr2=0x7fe454158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe43c003f30 0x7fe454158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 -- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4540b1f10 msgr2=0x7fe45414fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4540b1f10 0x7fe45414fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 -- 172.21.15.157:0/381423907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454158e30 con 0x7fe454154190 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe458d30640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0012f0 tx=0x7fe44c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00a5f0 con 0x7fe454154190 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00a770 con 0x7fe454154190 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe454159110 con 0x7fe454154190 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00ab20 con 0x7fe454154190 2023-11-06T03:07:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540b2390 con 0x7fe454154190 2023-11-06T03:07:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.625+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe454156c00 con 0x7fe454154190 2023-11-06T03:07:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.629+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe44c00ac80 con 0x7fe454154190 2023-11-06T03:07:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.629+0000 7fe450ff9640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.629+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fe44c0829d0 con 0x7fe454154190 2023-11-06T03:07:27.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.629+0000 7fe453fff640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.629+0000 7fe453fff640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 0x7fe440067f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4440016a0 tx=0x7fe444009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.633+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe454156c00 con 0x7fe454154190 2023-11-06T03:07:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.785+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe454156c00 con 0x7fe454154190 2023-11-06T03:07:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.789+0000 7fe450ff9640 1 -- 172.21.15.157:0/381423907 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18045 (secure 0 0 0) 0x7fe454156c00 con 0x7fe454154190 2023-11-06T03:07:27.789 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:27.789 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":41,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:06:57.240474+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:05:12.378890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-06T03:05:13.585131+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":40,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:27.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 msgr2=0x7fe440067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 0x7fe440067f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4440016a0 tx=0x7fe444009040 comp rx=0 tx=0).stop 2023-11-06T03:07:27.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 msgr2=0x7fe454154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0012f0 tx=0x7fe44c00ce50 comp rx=0 tx=0).stop 2023-11-06T03:07:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 shutdown_connections 2023-11-06T03:07:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe454154190 0x7fe454154610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4540b1f10 0x7fe45414fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 --2- 172.21.15.157:0/381423907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe43c003f30 0x7fe454158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 >> 172.21.15.157:0/381423907 conn(0x7fe454065930 msgr2=0x7fe45415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 shutdown_connections 2023-11-06T03:07:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.793+0000 7fe45a793640 1 -- 172.21.15.157:0/381423907 wait complete. 2023-11-06T03:07:27.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:07:27.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 Processor -- start 2023-11-06T03:07:27.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 -- start start 2023-11-06T03:07:27.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3f180d0280 0x7f3f18167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f180d2920 con 0x7f3f1815ce80 2023-11-06T03:07:27.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f180d18a0 con 0x7f3f1815caa0 2023-11-06T03:07:27.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f1c972640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f180d1b60 con 0x7f3f180d0280 2023-11-06T03:07:27.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f0dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3f1815ce80 0x7f3f180cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57896/0 (socket says 172.21.15.157:57896) 2023-11-06T03:07:27.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.925+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/3144320543 learned_addr learned my addr 172.21.15.157:0/3144320543 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:27.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f16d9d640 1 --2- 172.21.15.157:0/3144320543 >> v2:172.21.15.195:3300/0 conn(0x7f3f180d0280 0x7f3f18167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251833336 0 0) 0x7f3f180d2920 con 0x7f3f1815ce80 2023-11-06T03:07:27.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f04003530 con 0x7f3f1815ce80 2023-11-06T03:07:27.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3792255039 0 0) 0x7f3f04003530 con 0x7f3f1815ce80 2023-11-06T03:07:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 >> v2:172.21.15.195:3300/0 conn(0x7f3f180d0280 msgr2=0x7f3f18167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 --2- 172.21.15.157:0/3144320543 >> v2:172.21.15.195:3300/0 conn(0x7f3f180d0280 0x7f3f18167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 >> v1:172.21.15.195:6789/0 conn(0x7f3f1815caa0 legacy=0x7f3f180cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180d1e40 con 0x7f3f1815ce80 2023-11-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3f00001680 con 0x7f3f1815ce80 2023-11-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 >> v1:172.21.15.157:6789/0 conn(0x7f3f1815ce80 legacy=0x7f3f180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f04003980 0x7f3f04003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180d1b60 con 0x7f3f040047d0 2023-11-06T03:07:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0400c030 con 0x7f3f04005190 2023-11-06T03:07:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0400c1f0 con 0x7f3f04003980 2023-11-06T03:07:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1659c640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f04003980 0x7f3f04003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f04003980 msgr2=0x7f3f04003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f04003980 0x7f3f04003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f04005190 msgr2=0x7f3f0400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:27.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/3144320543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04003530 con 0x7f3f040047d0 2023-11-06T03:07:27.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f16d9d640 1 -- 172.21.15.157:0/3144320543 reap_dead start 2023-11-06T03:07:27.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 0x7f3f04004c50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c210 tx=0x7f3f0001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15899dcb2586d56c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00005ae0 con 0x7f3f040047d0 2023-11-06T03:07:27.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f000ab070 con 0x7f3f040047d0 2023-11-06T03:07:27.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f15d9b640 1 -- 172.21.15.157:0/3144320543 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00007070 con 0x7f3f040047d0 2023-11-06T03:07:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 -- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 msgr2=0x7f3f04004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 0x7f3f04004c50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c210 tx=0x7f3f0001f370 comp rx=0 tx=0).stop 2023-11-06T03:07:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 -- 172.21.15.157:0/3144320543 shutdown_connections 2023-11-06T03:07:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 --2- 172.21.15.157:0/3144320543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 -- 172.21.15.157:0/3144320543 >> 172.21.15.157:0/3144320543 conn(0x7f3f18070790 msgr2=0x7f3f18070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.929+0000 7f3f1c972640 1 -- 172.21.15.157:0/3144320543 shutdown_connections 2023-11-06T03:07:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- 172.21.15.157:0/3144320543 wait complete. 2023-11-06T03:07:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 Processor -- start 2023-11-06T03:07:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- start start 2023-11-06T03:07:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f040047d0 0x7f3f180ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 0x7f3f1815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181600b0 con 0x7f3f180cbb10 2023-11-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815ab30 con 0x7f3f040047d0 2023-11-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815d3f0 con 0x7f3f1815a170 2023-11-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55780/0 (socket says 172.21.15.157:55780) 2023-11-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/1002372757 learned_addr learned my addr 172.21.15.157:0/1002372757 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f16d9d640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 0x7f3f1815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 msgr2=0x7f3f1815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 0x7f3f1815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1659c640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f040047d0 0x7f3f180ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f040047d0 msgr2=0x7f3f180ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f040047d0 0x7f3f180ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 -- 172.21.15.157:0/1002372757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180cad90 con 0x7f3f180cbb10 2023-11-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f16d9d640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 0x7f3f1815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0dd9b640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3f00001940 tx=0x7f3f04005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00001640 con 0x7f3f180cbb10 2023-11-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f00007070 con 0x7f3f180cbb10 2023-11-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00007420 con 0x7f3f180cbb10 2023-11-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f180cb070 con 0x7f3f180cbb10 2023-11-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f180cbf90 con 0x7f3f180cbb10 2023-11-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.933+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18072750 con 0x7f3f180cbb10 2023-11-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.937+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3f000b9030 con 0x7f3f180cbb10 2023-11-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.937+0000 7f3f0f7fe640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 0x7f3ef8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.937+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f3f000b7070 con 0x7f3f180cbb10 2023-11-06T03:07:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.937+0000 7f3f1659c640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 0x7f3ef8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.937+0000 7f3f1659c640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 0x7f3ef8067f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f08007fd0 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:27.941+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3f000edae0 con 0x7f3f180cbb10 2023-11-06T03:07:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.113+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f1815d530 con 0x7f3f180cbb10 2023-11-06T03:07:28.113 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:07:28.113+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:07:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.129+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/1002372757 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f000c1020 con 0x7f3f180cbb10 2023-11-06T03:07:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:07:28.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 msgr2=0x7f3ef8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:28.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 0x7f3ef8067f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f08007fd0 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2023-11-06T03:07:28.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 msgr2=0x7f3f181f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:28.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3f00001940 tx=0x7f3f04005800 comp rx=0 tx=0).stop 2023-11-06T03:07:28.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 shutdown_connections 2023-11-06T03:07:28.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef8065a60 0x7f3ef8067f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f1815a170 0x7f3f1815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f180cbb10 0x7f3f181f5bd0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 --2- 172.21.15.157:0/1002372757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f040047d0 0x7f3f180ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 >> 172.21.15.157:0/1002372757 conn(0x7f3f18070790 msgr2=0x7f3f180c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:28.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 shutdown_connections 2023-11-06T03:07:28.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.133+0000 7f3f1c972640 1 -- 172.21.15.157:0/1002372757 wait complete. 2023-11-06T03:07:28.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:07:28.187 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:07:28.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 Processor -- start 2023-11-06T03:07:28.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 -- start start 2023-11-06T03:07:28.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d0160db0 0x7f00d0161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00d00701a0 0x7f00d00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d006f470 con 0x7f00d00b1910 2023-11-06T03:07:28.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d00b1df0 con 0x7f00d0160db0 2023-11-06T03:07:28.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00d4bc0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d006f6c0 con 0x7f00d00701a0 2023-11-06T03:07:28.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d0160db0 0x7f00d0161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d0160db0 0x7f00d0161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55816/0 (socket says 172.21.15.157:55816) 2023-11-06T03:07:28.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 -- 172.21.15.157:0/1175350959 learned_addr learned my addr 172.21.15.157:0/1175350959 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:28.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ced9d640 1 --2- 172.21.15.157:0/1175350959 >> v2:172.21.15.195:3300/0 conn(0x7f00d00701a0 0x7f00d00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 -- 172.21.15.157:0/1175350959 >> v2:172.21.15.195:3300/0 conn(0x7f00d00701a0 msgr2=0x7f00d00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:28.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 --2- 172.21.15.157:0/1175350959 >> v2:172.21.15.195:3300/0 conn(0x7f00d00701a0 0x7f00d00ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 -- 172.21.15.157:0/1175350959 >> v1:172.21.15.195:6789/0 conn(0x7f00d00b1910 legacy=0x7f00d006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:28.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.269+0000 7f00ce59c640 1 -- 172.21.15.157:0/1175350959 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d006f9a0 con 0x7f00d0160db0 2023-11-06T03:07:28.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00ce59c640 1 --2- 172.21.15.157:0/1175350959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d0160db0 0x7f00d0161190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f00c00098a0 tx=0x7f00c001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc4b0f361687322 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:28.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00cdd9b640 1 -- 172.21.15.157:0/1175350959 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00ab600 con 0x7f00d0160db0 2023-11-06T03:07:28.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00cdd9b640 1 -- 172.21.15.157:0/1175350959 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c00a7050 con 0x7f00d0160db0 2023-11-06T03:07:28.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00cdd9b640 1 -- 172.21.15.157:0/1175350959 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00ab980 con 0x7f00d0160db0 2023-11-06T03:07:28.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- 172.21.15.157:0/1175350959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d0160db0 msgr2=0x7f00d0161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:28.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/1175350959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d0160db0 0x7f00d0161190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f00c00098a0 tx=0x7f00c001c410 comp rx=0 tx=0).stop 2023-11-06T03:07:28.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- 172.21.15.157:0/1175350959 shutdown_connections 2023-11-06T03:07:28.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/1175350959 >> v2:172.21.15.195:3300/0 conn(0x7f00d00701a0 0x7f00d00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/1175350959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d0160db0 0x7f00d0161190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- 172.21.15.157:0/1175350959 >> 172.21.15.157:0/1175350959 conn(0x7f00d0065930 msgr2=0x7f00d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:28.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- 172.21.15.157:0/1175350959 shutdown_connections 2023-11-06T03:07:28.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- 172.21.15.157:0/1175350959 wait complete. 2023-11-06T03:07:28.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 Processor -- start 2023-11-06T03:07:28.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- start start 2023-11-06T03:07:28.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d00701a0 0x7f00d014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0160db0 0x7f00d0158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d013a1d0 con 0x7f00d00b1910 2023-11-06T03:07:28.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d014b990 con 0x7f00d0160db0 2023-11-06T03:07:28.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00d4bc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d014bc70 con 0x7f00d00701a0 2023-11-06T03:07:28.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55822/0 (socket says 172.21.15.157:55822) 2023-11-06T03:07:28.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00c7fff640 1 -- 172.21.15.157:0/2768986208 learned_addr learned my addr 172.21.15.157:0/2768986208 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:28.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00ce59c640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d00701a0 0x7f00d014b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.273+0000 7f00ced9d640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0160db0 0x7f00d0158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 -- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d00701a0 msgr2=0x7f00d014b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:28.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d00701a0 0x7f00d014b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 -- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0160db0 msgr2=0x7f00d0158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:28.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0160db0 0x7f00d0158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:28.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 -- 172.21.15.157:0/2768986208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00094d0 con 0x7f00d00b1910 2023-11-06T03:07:28.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c7fff640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c400 tx=0x7f00b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:28.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8016070 con 0x7f00d00b1910 2023-11-06T03:07:28.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b8007020 con 0x7f00d00b1910 2023-11-06T03:07:28.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d014bf50 con 0x7f00d00b1910 2023-11-06T03:07:28.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8007410 con 0x7f00d00b1910 2023-11-06T03:07:28.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d014c510 con 0x7f00d00b1910 2023-11-06T03:07:28.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d0072750 con 0x7f00d00b1910 2023-11-06T03:07:28.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f00b8007570 con 0x7f00d00b1910 2023-11-06T03:07:28.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 0x7f00b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:28.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f00b8082ba0 con 0x7f00d00b1910 2023-11-06T03:07:28.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00ce59c640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 0x7f00b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:28.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.277+0000 7f00ce59c640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 0x7f00b0067f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ad000 tx=0x7f00c00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:28.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.281+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f00b8050410 con 0x7f00d00b1910 2023-11-06T03:07:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:28.457+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f00d0071020 con 0x7f00d00b1910 2023-11-06T03:07:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.133+0000 7f00c6ffd640 1 -- 172.21.15.157:0/2768986208 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f00b8053a30 con 0x7f00d00b1910 2023-11-06T03:07:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 msgr2=0x7f00b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 0x7f00b0067f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f00c00ad000 tx=0x7f00c00a6000 comp rx=0 tx=0).stop 2023-11-06T03:07:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 msgr2=0x7f00d0139c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c400 tx=0x7f00b800c8c0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 shutdown_connections 2023-11-06T03:07:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00b0065a60 0x7f00b0067f20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d0160db0 0x7f00d0158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d00b1910 0x7f00d0139c90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 --2- 172.21.15.157:0/2768986208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d00701a0 0x7f00d014b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 >> 172.21.15.157:0/2768986208 conn(0x7f00d0065930 msgr2=0x7f00d01639b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 shutdown_connections 2023-11-06T03:07:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.137+0000 7f00d4bc0640 1 -- 172.21.15.157:0/2768986208 wait complete. 2023-11-06T03:07:29.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:07:29.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 Processor -- start 2023-11-06T03:07:29.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 -- start start 2023-11-06T03:07:29.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f294415cc70 0x7f29440d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 0x7f29440cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29440b15b0 0x7f29440b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29440d0410 con 0x7f29440b15b0 2023-11-06T03:07:29.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29440d0f90 con 0x7f294415cc70 2023-11-06T03:07:29.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2948ad8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29440d1270 con 0x7f29440cfa90 2023-11-06T03:07:29.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29440b15b0 0x7f29440b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29440b15b0 0x7f29440b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45196/0 (socket says 172.21.15.157:45196) 2023-11-06T03:07:29.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 -- 172.21.15.157:0/1684854964 learned_addr learned my addr 172.21.15.157:0/1684854964 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:29.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f293bfff640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 0x7f29440cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f294259c640 1 --2- 172.21.15.157:0/1684854964 >> v2:172.21.15.195:3300/0 conn(0x7f294415cc70 0x7f29440d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 -- 172.21.15.157:0/1684854964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 msgr2=0x7f29440cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:29.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 0x7f29440cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 -- 172.21.15.157:0/1684854964 >> v2:172.21.15.195:3300/0 conn(0x7f294415cc70 msgr2=0x7f29440d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:29.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 --2- 172.21.15.157:0/1684854964 >> v2:172.21.15.195:3300/0 conn(0x7f294415cc70 0x7f29440d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f2942d9d640 1 -- 172.21.15.157:0/1684854964 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29440d1550 con 0x7f29440b15b0 2023-11-06T03:07:29.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.341+0000 7f293bfff640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 0x7f29440cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:29.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2942d9d640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440b15b0 0x7f29440b1a10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f29440cf4e0 tx=0x7f293c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c8b16b32cdb609 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:29.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2941d9b640 1 -- 172.21.15.157:0/1684854964 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0a5730 con 0x7f29440b15b0 2023-11-06T03:07:29.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2941d9b640 1 -- 172.21.15.157:0/1684854964 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c0a5890 con 0x7f29440b15b0 2023-11-06T03:07:29.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2941d9b640 1 -- 172.21.15.157:0/1684854964 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0a5c20 con 0x7f29440b15b0 2023-11-06T03:07:29.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- 172.21.15.157:0/1684854964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440b15b0 msgr2=0x7f29440b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:29.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440b15b0 0x7f29440b1a10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f29440cf4e0 tx=0x7f293c01c9a0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- 172.21.15.157:0/1684854964 shutdown_connections 2023-11-06T03:07:29.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440b15b0 0x7f29440b1a10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- 172.21.15.157:0/1684854964 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440cfa90 0x7f29440cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- 172.21.15.157:0/1684854964 >> v2:172.21.15.195:3300/0 conn(0x7f294415cc70 0x7f29440d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- 172.21.15.157:0/1684854964 >> 172.21.15.157:0/1684854964 conn(0x7f2944070790 msgr2=0x7f2944070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:29.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- 172.21.15.157:0/1684854964 shutdown_connections 2023-11-06T03:07:29.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- 172.21.15.157:0/1684854964 wait complete. 2023-11-06T03:07:29.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 Processor -- start 2023-11-06T03:07:29.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- start start 2023-11-06T03:07:29.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440b15b0 0x7f2944153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294415cc70 0x7f29440cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29440cc170 con 0x7f29440cfa90 2023-11-06T03:07:29.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294415a960 con 0x7f294415cc70 2023-11-06T03:07:29.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2948ad8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294415a030 con 0x7f29440b15b0 2023-11-06T03:07:29.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45204/0 (socket says 172.21.15.157:45204) 2023-11-06T03:07:29.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 -- 172.21.15.157:0/744417911 learned_addr learned my addr 172.21.15.157:0/744417911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:29.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f294259c640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440b15b0 0x7f2944153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f2942d9d640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294415cc70 0x7f29440cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 -- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440b15b0 msgr2=0x7f2944153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:29.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440b15b0 0x7f2944153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 -- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294415cc70 msgr2=0x7f29440cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:29.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294415cc70 0x7f29440cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:29.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 -- 172.21.15.157:0/744417911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c009b30 con 0x7f29440cfa90 2023-11-06T03:07:29.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293bfff640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c7c0 tx=0x7f292c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:29.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.345+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c007a30 con 0x7f29440cfa90 2023-11-06T03:07:29.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c007b90 con 0x7f29440cfa90 2023-11-06T03:07:29.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c00ce70 con 0x7f29440cfa90 2023-11-06T03:07:29.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294415a310 con 0x7f29440cfa90 2023-11-06T03:07:29.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29441517d0 con 0x7f29440cfa90 2023-11-06T03:07:29.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294404e050 con 0x7f29440cfa90 2023-11-06T03:07:29.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f292c0113f0 con 0x7f29440cfa90 2023-11-06T03:07:29.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f293affd640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 0x7f2924067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:29.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f292c083890 con 0x7f29440cfa90 2023-11-06T03:07:29.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f294259c640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 0x7f2924067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:29.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.349+0000 7f294259c640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 0x7f2924067f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f29441551b0 tx=0x7f2934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:29.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.353+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f292c051070 con 0x7f29440cfa90 2023-11-06T03:07:29.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:29.509+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2944155280 con 0x7f29440cfa90 2023-11-06T03:07:30.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.361+0000 7f293affd640 1 -- 172.21.15.157:0/744417911 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f292c054690 con 0x7f29440cfa90 2023-11-06T03:07:30.361 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:07:30.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 msgr2=0x7f2924067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 0x7f2924067f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f29441551b0 tx=0x7f2934008040 comp rx=0 tx=0).stop 2023-11-06T03:07:30.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 msgr2=0x7f2944151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c7c0 tx=0x7f292c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:07:30.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 shutdown_connections 2023-11-06T03:07:30.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2924065a60 0x7f2924067f20 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f29441551b0 tx=0x7f2934008040 comp rx=0 tx=0).stop 2023-11-06T03:07:30.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f294415cc70 0x7f29440cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29440cfa90 0x7f2944151290 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 --2- 172.21.15.157:0/744417911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29440b15b0 0x7f2944153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.365+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 >> 172.21.15.157:0/744417911 conn(0x7f2944070790 msgr2=0x7f29440cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:30.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.369+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 shutdown_connections 2023-11-06T03:07:30.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.369+0000 7f2948ad8640 1 -- 172.21.15.157:0/744417911 wait complete. 2023-11-06T03:07:30.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:07:30.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.513+0000 7f01b0208640 1 Processor -- start 2023-11-06T03:07:30.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.513+0000 7f01b0208640 1 -- start start 2023-11-06T03:07:30.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01b0208640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a8159c80 0x7f01a80b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01b0208640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01a815dda0 0x7f01a815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01b0208640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a8070ee0 con 0x7f01a806b2f0 2023-11-06T03:07:30.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01b0208640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a815b060 con 0x7f01a815dda0 2023-11-06T03:07:30.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01b0208640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a806b780 con 0x7f01a8159c80 2023-11-06T03:07:30.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01a815dda0 0x7f01a815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ad7a3640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f01a806b2f0 0x7f01a815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51064/0 (socket says 172.21.15.157:51064) 2023-11-06T03:07:30.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ad7a3640 1 -- 172.21.15.157:0/850991966 learned_addr learned my addr 172.21.15.157:0/850991966 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:30.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01adfa4640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a8159c80 0x7f01a80b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 -- 172.21.15.157:0/850991966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a8159c80 msgr2=0x7f01a80b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:30.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a8159c80 0x7f01a80b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 -- 172.21.15.157:0/850991966 >> v1:172.21.15.157:6789/0 conn(0x7f01a806b2f0 legacy=0x7f01a815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 -- 172.21.15.157:0/850991966 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80b2a90 con 0x7f01a815dda0 2023-11-06T03:07:30.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01acfa2640 1 -- 172.21.15.157:0/850991966 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058539403 0 0) 0x7f01a8070ee0 con 0x7f01a806b2f0 2023-11-06T03:07:30.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01ae7a5640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a815dda0 0x7f01a815ab20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009d90 tx=0x7f01a001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce4ce7aefd013a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:30.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01acfa2640 1 -- 172.21.15.157:0/850991966 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a00a5950 con 0x7f01a815dda0 2023-11-06T03:07:30.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01acfa2640 1 -- 172.21.15.157:0/850991966 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a00a5ab0 con 0x7f01a815dda0 2023-11-06T03:07:30.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.517+0000 7f01acfa2640 1 -- 172.21.15.157:0/850991966 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a00a5e40 con 0x7f01a815dda0 2023-11-06T03:07:30.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/850991966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a815dda0 msgr2=0x7f01a815ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a815dda0 0x7f01a815ab20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009d90 tx=0x7f01a001c890 comp rx=0 tx=0).stop 2023-11-06T03:07:30.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/850991966 shutdown_connections 2023-11-06T03:07:30.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a815dda0 0x7f01a815ab20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- 172.21.15.157:0/850991966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a8159c80 0x7f01a80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/850991966 >> 172.21.15.157:0/850991966 conn(0x7f01a80b43b0 msgr2=0x7f01a8065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/850991966 shutdown_connections 2023-11-06T03:07:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/850991966 wait complete. 2023-11-06T03:07:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 Processor -- start 2023-11-06T03:07:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- start start 2023-11-06T03:07:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a815dda0 0x7f01a814b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 0x7f01a80ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80cd020 con 0x7f01a8159c80 2023-11-06T03:07:30.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8156a60 con 0x7f01a80cc630 2023-11-06T03:07:30.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a813c540 con 0x7f01a815dda0 2023-11-06T03:07:30.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45230/0 (socket says 172.21.15.157:45230) 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 -- 172.21.15.157:0/2311423094 learned_addr learned my addr 172.21.15.157:0/2311423094 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01ad7a3640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a815dda0 0x7f01a814b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01ae7a5640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 0x7f01a80ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 -- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a815dda0 msgr2=0x7f01a814b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a815dda0 0x7f01a814b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 -- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 msgr2=0x7f01a80ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:30.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 0x7f01a80ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:30.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 -- 172.21.15.157:0/2311423094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0009a70 con 0x7f01a8159c80 2023-11-06T03:07:30.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01ae7a5640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 0x7f01a80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:30.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01adfa4640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f01a400d800 tx=0x7f01a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:30.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400b660 con 0x7f01a8159c80 2023-11-06T03:07:30.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400b7c0 con 0x7f01a8159c80 2023-11-06T03:07:30.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400bb30 con 0x7f01a8159c80 2023-11-06T03:07:30.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80ae340 con 0x7f01a8159c80 2023-11-06T03:07:30.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.521+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a8155170 con 0x7f01a8159c80 2023-11-06T03:07:30.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a80cf3b0 con 0x7f01a8159c80 2023-11-06T03:07:30.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01a400bc90 con 0x7f01a8159c80 2023-11-06T03:07:30.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f019e7fc640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 0x7f018c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:30.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f01a4083650 con 0x7f01a8159c80 2023-11-06T03:07:30.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f01ad7a3640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 0x7f018c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:30.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.525+0000 7f01ad7a3640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 0x7f018c067f50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f01a80afde0 tx=0x7f0198007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:30.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.529+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f01a4050ff0 con 0x7f01a8159c80 2023-11-06T03:07:30.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:30.693+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01a8155300 con 0x7f01a8159c80 2023-11-06T03:07:31.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.369+0000 7f019e7fc640 1 -- 172.21.15.157:0/2311423094 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01a4054610 con 0x7f01a8159c80 2023-11-06T03:07:31.368 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:07:31.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 msgr2=0x7f018c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 0x7f018c067f50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f01a80afde0 tx=0x7f0198007b60 comp rx=0 tx=0).stop 2023-11-06T03:07:31.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 msgr2=0x7f01a813c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f01a400d800 tx=0x7f01a400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 shutdown_connections 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f018c065a90 0x7f018c067f50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80cc630 0x7f01a80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a815dda0 0x7f01a814b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 --2- 172.21.15.157:0/2311423094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8159c80 0x7f01a813c000 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 >> 172.21.15.157:0/2311423094 conn(0x7f01a80b43b0 msgr2=0x7f01a814c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:31.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 shutdown_connections 2023-11-06T03:07:31.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.373+0000 7f01b0208640 1 -- 172.21.15.157:0/2311423094 wait complete. 2023-11-06T03:07:31.404 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:07:31.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 Processor -- start 2023-11-06T03:07:31.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 -- start start 2023-11-06T03:07:31.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1f100ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1015e440 con 0x7f1f1006d000 2023-11-06T03:07:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1006c240 con 0x7f1f100b2af0 2023-11-06T03:07:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f16d73640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006c520 con 0x7f1f101599c0 2023-11-06T03:07:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f0ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1f1006d000 0x7f1f100b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51078/0 (socket says 172.21.15.157:51078) 2023-11-06T03:07:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.529+0000 7f1f0ffff640 1 -- 172.21.15.157:0/1237480682 learned_addr learned my addr 172.21.15.157:0/1237480682 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1f100ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078590666 0 0) 0x7f1f1015e440 con 0x7f1f1006d000 2023-11-06T03:07:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef8003530 con 0x7f1f1006d000 2023-11-06T03:07:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132421351 0 0) 0x7f1ef8003530 con 0x7f1f1006d000 2023-11-06T03:07:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 msgr2=0x7f1f100ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1f100ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 >> v1:172.21.15.195:6789/0 conn(0x7f1f100b2af0 legacy=0x7f1f10159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1006c800 con 0x7f1f1006d000 2023-11-06T03:07:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1f04001680 con 0x7f1f1006d000 2023-11-06T03:07:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 >> v1:172.21.15.157:6789/0 conn(0x7f1f1006d000 legacy=0x7f1f100b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1ef80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8004d10 0x7f1ef8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006c520 con 0x7f1ef8004d10 2023-11-06T03:07:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800ba40 con 0x7f1ef80039b0 2023-11-06T03:07:31.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800bc00 con 0x7f1f101599c0 2023-11-06T03:07:31.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8004d10 0x7f1ef8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0ffff640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1ef80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 -- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 msgr2=0x7f1ef80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:31.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1ef80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 -- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef80039b0 msgr2=0x7f1ef8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:31.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 -- 172.21.15.157:0/1237480682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8003530 con 0x7f1ef8004d10 2023-11-06T03:07:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0ffff640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101599c0 0x7f1ef80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8004d10 0x7f1ef8005190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f1006f250 tx=0x7f1f0801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2422d05e68c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08003b80 con 0x7f1ef8004d10 2023-11-06T03:07:31.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08003ce0 con 0x7f1ef8004d10 2023-11-06T03:07:31.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f0f7fe640 1 -- 172.21.15.157:0/1237480682 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08005310 con 0x7f1ef8004d10 2023-11-06T03:07:31.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f16d73640 1 -- 172.21.15.157:0/1237480682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8004d10 msgr2=0x7f1ef8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f16d73640 1 --2- 172.21.15.157:0/1237480682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ef8004d10 0x7f1ef8005190 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f1006f250 tx=0x7f1f0801c700 comp rx=0 tx=0).stop 2023-11-06T03:07:31.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f15310640 1 -- 172.21.15.157:0/1237480682 reap_dead start 2023-11-06T03:07:31.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f16d73640 1 -- 172.21.15.157:0/1237480682 shutdown_connections 2023-11-06T03:07:31.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f16d73640 1 -- 172.21.15.157:0/1237480682 >> 172.21.15.157:0/1237480682 conn(0x7f1f10065930 msgr2=0x7f1f10065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:31.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.533+0000 7f1f16d73640 1 -- 172.21.15.157:0/1237480682 shutdown_connections 2023-11-06T03:07:31.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- 172.21.15.157:0/1237480682 wait complete. 2023-11-06T03:07:31.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 Processor -- start 2023-11-06T03:07:31.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- start start 2023-11-06T03:07:31.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f10157dc0 0x7f1f100a8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101581a0 0x7f1f100a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100ccad0 con 0x7f1f101f2580 2023-11-06T03:07:31.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101f32f0 con 0x7f1f10157dc0 2023-11-06T03:07:31.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100ccd20 con 0x7f1f101581a0 2023-11-06T03:07:31.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45246/0 (socket says 172.21.15.157:45246) 2023-11-06T03:07:31.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 -- 172.21.15.157:0/503654953 learned_addr learned my addr 172.21.15.157:0/503654953 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:31.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f0ffff640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101581a0 0x7f1f100a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f10157dc0 0x7f1f100a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 -- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101581a0 msgr2=0x7f1f100a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101581a0 0x7f1f100a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 -- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f10157dc0 msgr2=0x7f1f100a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f10157dc0 0x7f1f100a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 -- 172.21.15.157:0/503654953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100cd000 con 0x7f1f101f2580 2023-11-06T03:07:31.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f15310640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1f08098110 tx=0x7f1ef8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080a8660 con 0x7f1f101f2580 2023-11-06T03:07:31.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080a87c0 con 0x7f1f101f2580 2023-11-06T03:07:31.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080a8b70 con 0x7f1f101f2580 2023-11-06T03:07:31.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f100cd2e0 con 0x7f1f101f2580 2023-11-06T03:07:31.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f100ac640 con 0x7f1f101f2580 2023-11-06T03:07:31.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.537+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f10072750 con 0x7f1f101f2580 2023-11-06T03:07:31.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.541+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f080a8cd0 con 0x7f1f101f2580 2023-11-06T03:07:31.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.541+0000 7f1f0d7fa640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 0x7f1efc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.541+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1f0811cfb0 con 0x7f1f101f2580 2023-11-06T03:07:31.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.541+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 0x7f1efc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.541+0000 7f1f14b0f640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 0x7f1efc067f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a30 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.545+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f080eaba0 con 0x7f1f101f2580 2023-11-06T03:07:31.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.701+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1f100adcd0 con 0x7f1f101f2580 2023-11-06T03:07:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.705+0000 7f1f0d7fa640 1 -- 172.21.15.157:0/503654953 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1f08007dc0 con 0x7f1f101f2580 2023-11-06T03:07:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 msgr2=0x7f1efc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 0x7f1efc067f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a30 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2023-11-06T03:07:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 msgr2=0x7f1f101f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1f08098110 tx=0x7f1ef8005800 comp rx=0 tx=0).stop 2023-11-06T03:07:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 shutdown_connections 2023-11-06T03:07:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1efc065a90 0x7f1efc067f50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f101f2580 0x7f1f101f2a00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f101581a0 0x7f1f100a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 --2- 172.21.15.157:0/503654953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f10157dc0 0x7f1f100a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 >> 172.21.15.157:0/503654953 conn(0x7f1f10065930 msgr2=0x7f1f10158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 shutdown_connections 2023-11-06T03:07:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.709+0000 7f1f16d73640 1 -- 172.21.15.157:0/503654953 wait complete. 2023-11-06T03:07:31.726 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-11-06T03:07:31.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.845+0000 7f0172d6a640 1 Processor -- start 2023-11-06T03:07:31.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.845+0000 7f0172d6a640 1 -- start start 2023-11-06T03:07:31.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0172d6a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c15ce30 0x7f016c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0172d6a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f016c06ac30 0x7f016c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0172d6a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c06da10 con 0x7f016c06ecf0 2023-11-06T03:07:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0172d6a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c06dcf0 con 0x7f016c06ac30 2023-11-06T03:07:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0172d6a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c06dfd0 con 0x7f016c15ce30 2023-11-06T03:07:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f016c06ac30 0x7f016c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0171307640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f016c06ecf0 0x7f016c0cc170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51082/0 (socket says 172.21.15.157:51082) 2023-11-06T03:07:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0171307640 1 -- 172.21.15.157:0/2303975047 learned_addr learned my addr 172.21.15.157:0/2303975047 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f0170b06640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c15ce30 0x7f016c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016bfff640 1 -- 172.21.15.157:0/2303975047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c15ce30 msgr2=0x7f016c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016bfff640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c15ce30 0x7f016c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016bfff640 1 -- 172.21.15.157:0/2303975047 >> v1:172.21.15.157:6789/0 conn(0x7f016c06ecf0 legacy=0x7f016c0cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016b7fe640 1 -- 172.21.15.157:0/2303975047 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894942673 0 0) 0x7f016c06da10 con 0x7f016c06ecf0 2023-11-06T03:07:31.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.849+0000 7f016bfff640 1 -- 172.21.15.157:0/2303975047 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c06e2b0 con 0x7f016c06ac30 2023-11-06T03:07:31.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f016bfff640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c06ac30 0x7f016c0d0d10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f01600097c0 tx=0x7f016001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec80966e2be4ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f016b7fe640 1 -- 172.21.15.157:0/2303975047 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01600ad660 con 0x7f016c06ac30 2023-11-06T03:07:31.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f016b7fe640 1 -- 172.21.15.157:0/2303975047 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01600ad7c0 con 0x7f016c06ac30 2023-11-06T03:07:31.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f016b7fe640 1 -- 172.21.15.157:0/2303975047 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01600adb50 con 0x7f016c06ac30 2023-11-06T03:07:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- 172.21.15.157:0/2303975047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c06ac30 msgr2=0x7f016c0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c06ac30 0x7f016c0d0d10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f01600097c0 tx=0x7f016001c480 comp rx=0 tx=0).stop 2023-11-06T03:07:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- 172.21.15.157:0/2303975047 shutdown_connections 2023-11-06T03:07:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c06ac30 0x7f016c0d0d10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- 172.21.15.157:0/2303975047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c15ce30 0x7f016c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- 172.21.15.157:0/2303975047 >> 172.21.15.157:0/2303975047 conn(0x7f016c071b90 msgr2=0x7f016c099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- 172.21.15.157:0/2303975047 shutdown_connections 2023-11-06T03:07:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- 172.21.15.157:0/2303975047 wait complete. 2023-11-06T03:07:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 Processor -- start 2023-11-06T03:07:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- start start 2023-11-06T03:07:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016c06ac30 0x7f016c14b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c06ecf0 0x7f016c0ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0ae2b0 con 0x7f016c15ce30 2023-11-06T03:07:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c14e250 con 0x7f016c06ac30 2023-11-06T03:07:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0172d6a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0b4630 con 0x7f016c06ecf0 2023-11-06T03:07:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0171307640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0171307640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45266/0 (socket says 172.21.15.157:45266) 2023-11-06T03:07:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.853+0000 7f0171307640 1 -- 172.21.15.157:0/920228031 learned_addr learned my addr 172.21.15.157:0/920228031 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0170b06640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016c06ac30 0x7f016c14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f016bfff640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c06ecf0 0x7f016c0ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 -- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c06ecf0 msgr2=0x7f016c0ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c06ecf0 0x7f016c0ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 -- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016c06ac30 msgr2=0x7f016c14b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016c06ac30 0x7f016c14b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 -- 172.21.15.157:0/920228031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160009420 con 0x7f016c15ce30 2023-11-06T03:07:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0171307640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0164001570 tx=0x7f016400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400a640 con 0x7f016c15ce30 2023-11-06T03:07:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016400a7c0 con 0x7f016c15ce30 2023-11-06T03:07:31.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c0b4910 con 0x7f016c15ce30 2023-11-06T03:07:31.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400ab70 con 0x7f016c15ce30 2023-11-06T03:07:31.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c14b9c0 con 0x7f016c15ce30 2023-11-06T03:07:31.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c0b4b50 con 0x7f016c15ce30 2023-11-06T03:07:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f016400acd0 con 0x7f016c15ce30 2023-11-06T03:07:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 0x7f0150067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0164082500 con 0x7f016c15ce30 2023-11-06T03:07:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0170b06640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 0x7f0150067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.857+0000 7f0170b06640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 0x7f0150067f50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0158006fd0 tx=0x7f0158008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:31.861+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0164050060 con 0x7f016c15ce30 2023-11-06T03:07:32.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.073+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f016c072750 con 0x7f016c15ce30 2023-11-06T03:07:32.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.381+0000 7f01697fa640 1 -- 172.21.15.157:0/920228031 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v176) v1 ==== 148+0+0 (secure 0 0 0) 0x7f01640180d0 con 0x7f016c15ce30 2023-11-06T03:07:32.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 msgr2=0x7f0150067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:32.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 0x7f0150067f50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0158006fd0 tx=0x7f0158008040 comp rx=0 tx=0).stop 2023-11-06T03:07:32.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 msgr2=0x7f016c0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:32.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0164001570 tx=0x7f016400cca0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 shutdown_connections 2023-11-06T03:07:32.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0150065a90 0x7f0150067f50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016c15ce30 0x7f016c0b5fa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f016c06ecf0 0x7f016c0ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 --2- 172.21.15.157:0/920228031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016c06ac30 0x7f016c14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 >> 172.21.15.157:0/920228031 conn(0x7f016c071b90 msgr2=0x7f016c0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:32.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 shutdown_connections 2023-11-06T03:07:32.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.385+0000 7f0172d6a640 1 -- 172.21.15.157:0/920228031 wait complete. 2023-11-06T03:07:32.411 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2023-11-06T03:07:32.412 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-06T03:07:32.413 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2023-11-06T03:07:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 Processor -- start 2023-11-06T03:07:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 -- start start 2023-11-06T03:07:32.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e041551e0 0x7f1e0406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e040b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e0415ea60 0x7f1e040b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0415f580 con 0x7f1e0415ea60 2023-11-06T03:07:32.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0415efc0 con 0x7f1e041551e0 2023-11-06T03:07:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e08880640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0415f2a0 con 0x7f1e0415e5e0 2023-11-06T03:07:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e0415ea60 0x7f1e040b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e0415ea60 0x7f1e040b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45278/0 (socket says 172.21.15.157:45278) 2023-11-06T03:07:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 -- 172.21.15.157:0/1416711000 learned_addr learned my addr 172.21.15.157:0/1416711000 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e01d9b640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e040b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e0259c640 1 --2- 172.21.15.157:0/1416711000 >> v2:172.21.15.195:3300/0 conn(0x7f1e041551e0 0x7f1e0406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 -- 172.21.15.157:0/1416711000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 msgr2=0x7f1e040b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e040b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 -- 172.21.15.157:0/1416711000 >> v2:172.21.15.195:3300/0 conn(0x7f1e041551e0 msgr2=0x7f1e0406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.533+0000 7f1e02d9d640 1 --2- 172.21.15.157:0/1416711000 >> v2:172.21.15.195:3300/0 conn(0x7f1e041551e0 0x7f1e0406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e02d9d640 1 -- 172.21.15.157:0/1416711000 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0406f5a0 con 0x7f1e0415ea60 2023-11-06T03:07:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e02d9d640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e0415ea60 0x7f1e040b3950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1e04066830 tx=0x7f1dfc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce36afdf97d1991e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e0159a640 1 -- 172.21.15.157:0/1416711000 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a57b0 con 0x7f1e0415ea60 2023-11-06T03:07:32.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e0159a640 1 -- 172.21.15.157:0/1416711000 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc0a5910 con 0x7f1e0415ea60 2023-11-06T03:07:32.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e0159a640 1 -- 172.21.15.157:0/1416711000 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0a5ca0 con 0x7f1e0415ea60 2023-11-06T03:07:32.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 -- 172.21.15.157:0/1416711000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e0415ea60 msgr2=0x7f1e040b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:32.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e0415ea60 0x7f1e040b3950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1e04066830 tx=0x7f1dfc01c910 comp rx=0 tx=0).stop 2023-11-06T03:07:32.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 -- 172.21.15.157:0/1416711000 shutdown_connections 2023-11-06T03:07:32.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e0415ea60 0x7f1e040b3950 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 --2- 172.21.15.157:0/1416711000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e040b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 --2- 172.21.15.157:0/1416711000 >> v2:172.21.15.195:3300/0 conn(0x7f1e041551e0 0x7f1e0406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 -- 172.21.15.157:0/1416711000 >> 172.21.15.157:0/1416711000 conn(0x7f1e04065930 msgr2=0x7f1e04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:32.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 -- 172.21.15.157:0/1416711000 shutdown_connections 2023-11-06T03:07:32.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.537+0000 7f1e08880640 1 -- 172.21.15.157:0/1416711000 wait complete. 2023-11-06T03:07:32.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 Processor -- start 2023-11-06T03:07:32.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- start start 2023-11-06T03:07:32.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e041562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0415ea60 0x7f1e0413a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e04156fb0 con 0x7f1e041551e0 2023-11-06T03:07:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0413a8c0 con 0x7f1e0415ea60 2023-11-06T03:07:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040b1960 con 0x7f1e0415e5e0 2023-11-06T03:07:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45280/0 (socket says 172.21.15.157:45280) 2023-11-06T03:07:32.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 -- 172.21.15.157:0/2364459122 learned_addr learned my addr 172.21.15.157:0/2364459122 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:32.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e01d9b640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e041562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e02d9d640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0415ea60 0x7f1e0413a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 -- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 msgr2=0x7f1e041562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:32.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e041562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 -- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0415ea60 msgr2=0x7f1e0413a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:32.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0415ea60 0x7f1e0413a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:32.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 -- 172.21.15.157:0/2364459122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc009b30 con 0x7f1e041551e0 2023-11-06T03:07:32.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e0259c640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d800 tx=0x7f1df400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:32.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40096a0 con 0x7f1e041551e0 2023-11-06T03:07:32.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4009800 con 0x7f1e041551e0 2023-11-06T03:07:32.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4009b70 con 0x7f1e041551e0 2023-11-06T03:07:32.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e041569f0 con 0x7f1e041551e0 2023-11-06T03:07:32.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e041525a0 con 0x7f1e041551e0 2023-11-06T03:07:32.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04152400 con 0x7f1e041551e0 2023-11-06T03:07:32.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.541+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1df4009cd0 con 0x7f1e041551e0 2023-11-06T03:07:32.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.545+0000 7f1df2ffd640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 0x7f1de4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:32.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.545+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1df40836f0 con 0x7f1e041551e0 2023-11-06T03:07:32.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.545+0000 7f1e01d9b640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 0x7f1de4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:32.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.545+0000 7f1e01d9b640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 0x7f1de4067f50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b9d0 tx=0x7f1df8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:32.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.549+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1e04152400 con 0x7f1e041551e0 2023-11-06T03:07:32.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:32.757+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f1e04152400 con 0x7f1e041551e0 2023-11-06T03:07:33.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1df2ffd640 1 -- 172.21.15.157:0/2364459122 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v177) v1 ==== 146+0+0 (secure 0 0 0) 0x7f1e04152400 con 0x7f1e041551e0 2023-11-06T03:07:33.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 msgr2=0x7f1de4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 0x7f1de4067f50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b9d0 tx=0x7f1df8008040 comp rx=0 tx=0).stop 2023-11-06T03:07:33.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 msgr2=0x7f1e0414ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1df400d800 tx=0x7f1df400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 shutdown_connections 2023-11-06T03:07:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1de4065a90 0x7f1de4067f50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e0415ea60 0x7f1e0413a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e0415e5e0 0x7f1e041562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 --2- 172.21.15.157:0/2364459122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e041551e0 0x7f1e0414ce60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 >> 172.21.15.157:0/2364459122 conn(0x7f1e04065930 msgr2=0x7f1e0415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 shutdown_connections 2023-11-06T03:07:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.389+0000 7f1e08880640 1 -- 172.21.15.157:0/2364459122 wait complete. 2023-11-06T03:07:33.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:07:33.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 Processor -- start 2023-11-06T03:07:33.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 -- start start 2023-11-06T03:07:33.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30300ce800 0x7f30300cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f303006b6a0 0x7f303006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30300cf1a0 con 0x7f3030146370 2023-11-06T03:07:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30300b2120 con 0x7f303006b6a0 2023-11-06T03:07:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.537+0000 7f30355a3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303006c3d0 con 0x7f30300ce800 2023-11-06T03:07:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f303006b6a0 0x7f303006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f303006b6a0 0x7f303006bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45286/0 (socket says 172.21.15.157:45286) 2023-11-06T03:07:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 -- 172.21.15.157:0/291723342 learned_addr learned my addr 172.21.15.157:0/291723342 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302e7fc640 1 --2- 172.21.15.157:0/291723342 >> v2:172.21.15.195:3300/0 conn(0x7f30300ce800 0x7f30300cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 -- 172.21.15.157:0/291723342 >> v2:172.21.15.195:3300/0 conn(0x7f30300ce800 msgr2=0x7f30300cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 --2- 172.21.15.157:0/291723342 >> v2:172.21.15.195:3300/0 conn(0x7f30300ce800 0x7f30300cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 -- 172.21.15.157:0/291723342 >> v1:172.21.15.195:6789/0 conn(0x7f3030146370 legacy=0x7f30300b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 -- 172.21.15.157:0/291723342 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303006f5a0 con 0x7f303006b6a0 2023-11-06T03:07:33.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302f7fe640 1 --2- 172.21.15.157:0/291723342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f303006b6a0 0x7f303006bb30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3028009e50 tx=0x7f302801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b6b9e1bdf2214b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:33.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302dffb640 1 -- 172.21.15.157:0/291723342 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280a5790 con 0x7f303006b6a0 2023-11-06T03:07:33.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302dffb640 1 -- 172.21.15.157:0/291723342 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30280a58f0 con 0x7f303006b6a0 2023-11-06T03:07:33.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f302dffb640 1 -- 172.21.15.157:0/291723342 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280a5c80 con 0x7f303006b6a0 2023-11-06T03:07:33.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 -- 172.21.15.157:0/291723342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f303006b6a0 msgr2=0x7f303006bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 --2- 172.21.15.157:0/291723342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f303006b6a0 0x7f303006bb30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3028009e50 tx=0x7f302801c8d0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 -- 172.21.15.157:0/291723342 shutdown_connections 2023-11-06T03:07:33.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 --2- 172.21.15.157:0/291723342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f303006b6a0 0x7f303006bb30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 --2- 172.21.15.157:0/291723342 >> v2:172.21.15.195:3300/0 conn(0x7f30300ce800 0x7f30300cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 -- 172.21.15.157:0/291723342 >> 172.21.15.157:0/291723342 conn(0x7f3030065930 msgr2=0x7f3030065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:33.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 -- 172.21.15.157:0/291723342 shutdown_connections 2023-11-06T03:07:33.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.541+0000 7f30355a3640 1 -- 172.21.15.157:0/291723342 wait complete. 2023-11-06T03:07:33.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 Processor -- start 2023-11-06T03:07:33.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- start start 2023-11-06T03:07:33.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f303006b6a0 0x7f3030159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3030146370 0x7f303013b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303015c7a0 con 0x7f30300ce800 2023-11-06T03:07:33.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303015ca80 con 0x7f3030146370 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303015cd60 con 0x7f303006b6a0 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45292/0 (socket says 172.21.15.157:45292) 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 -- 172.21.15.157:0/3604149867 learned_addr learned my addr 172.21.15.157:0/3604149867 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302f7fe640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3030146370 0x7f303013b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302effd640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f303006b6a0 0x7f3030159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 -- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f303006b6a0 msgr2=0x7f3030159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f303006b6a0 0x7f3030159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 -- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3030146370 msgr2=0x7f303013b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3030146370 0x7f303013b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 -- 172.21.15.157:0/3604149867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028009b30 con 0x7f30300ce800 2023-11-06T03:07:33.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f302e7fc640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f302000d5f0 tx=0x7f302000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:33.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30200125f0 con 0x7f30300ce800 2023-11-06T03:07:33.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020012750 con 0x7f30300ce800 2023-11-06T03:07:33.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020012ae0 con 0x7f30300ce800 2023-11-06T03:07:33.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303013ab00 con 0x7f30300ce800 2023-11-06T03:07:33.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303013b0c0 con 0x7f30300ce800 2023-11-06T03:07:33.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.545+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303013b280 con 0x7f30300ce800 2023-11-06T03:07:33.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.549+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3020012c40 con 0x7f30300ce800 2023-11-06T03:07:33.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.549+0000 7f30177fe640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 0x7f301c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:33.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.549+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3020083460 con 0x7f30300ce800 2023-11-06T03:07:33.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.549+0000 7f302effd640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 0x7f301c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:33.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.549+0000 7f302effd640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 0x7f301c067f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3024004e30 tx=0x7f3024009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:33.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.553+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3020051050 con 0x7f30300ce800 2023-11-06T03:07:33.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.709+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3030072750 con 0x7f30300ce800 2023-11-06T03:07:33.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.717+0000 7f30177fe640 1 -- 172.21.15.157:0/3604149867 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3020054670 con 0x7f30300ce800 2023-11-06T03:07:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 msgr2=0x7f301c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 0x7f301c067f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3024004e30 tx=0x7f3024009040 comp rx=0 tx=0).stop 2023-11-06T03:07:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 msgr2=0x7f303013a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f302000d5f0 tx=0x7f302000dab0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 shutdown_connections 2023-11-06T03:07:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f301c065a60 0x7f301c067f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3030146370 0x7f303013b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30300ce800 0x7f303013a280 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 --2- 172.21.15.157:0/3604149867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f303006b6a0 0x7f3030159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 >> 172.21.15.157:0/3604149867 conn(0x7f3030065930 msgr2=0x7f30300b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:33.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 shutdown_connections 2023-11-06T03:07:33.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:33.721+0000 7f30355a3640 1 -- 172.21.15.157:0/3604149867 wait complete. 2023-11-06T03:07:33.738 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:07:33.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:07:33.776 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:07:33.816 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:07:33.817 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:07:33.817 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:33.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:07:33.823 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:07:33.823 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:33.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:07:33.864 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:07:33.865 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:33.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:07:33.872 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:07:33.873 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:07:33.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:07:33.912 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:07:33.912 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:34.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 Processor -- start 2023-11-06T03:07:34.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 -- start start 2023-11-06T03:07:34.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0d2920 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0d18a0 con 0x7f0b6c15ce80 2023-11-06T03:07:34.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b73331640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0d1b60 con 0x7f0b6c15caa0 2023-11-06T03:07:34.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b708cc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b710cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b718ce640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f0b6c0ce1a0 0x7f0b6c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:59314/0 (socket says 172.21.15.157:59314) 2023-11-06T03:07:34.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b708cc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:44368/0 (socket says 172.21.15.157:44368) 2023-11-06T03:07:34.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.045+0000 7f0b718ce640 1 -- 172.21.15.157:0/166878792 learned_addr learned my addr 172.21.15.157:0/166878792 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:34.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b708cc640 1 -- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 msgr2=0x7f0b6c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:34.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b708cc640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b708cc640 1 -- 172.21.15.157:0/166878792 >> v1:172.21.15.195:6789/0 conn(0x7f0b6c0ce1a0 legacy=0x7f0b6c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b708cc640 1 -- 172.21.15.157:0/166878792 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0d1e40 con 0x7f0b6c15ce80 2023-11-06T03:07:34.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b710cd640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:07:34.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b708cc640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0097c0 tx=0x7f0b5c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67493ac1543d70fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b63fff640 1 -- 172.21.15.157:0/166878792 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0ad660 con 0x7f0b6c15ce80 2023-11-06T03:07:34.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b63fff640 1 -- 172.21.15.157:0/166878792 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c0ad7c0 con 0x7f0b6c15ce80 2023-11-06T03:07:34.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b63fff640 1 -- 172.21.15.157:0/166878792 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0adb50 con 0x7f0b6c15ce80 2023-11-06T03:07:34.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 -- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 msgr2=0x7f0b6c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0097c0 tx=0x7f0b5c01c450 comp rx=0 tx=0).stop 2023-11-06T03:07:34.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 -- 172.21.15.157:0/166878792 shutdown_connections 2023-11-06T03:07:34.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c0cfb70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 --2- 172.21.15.157:0/166878792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 -- 172.21.15.157:0/166878792 >> 172.21.15.157:0/166878792 conn(0x7f0b6c070790 msgr2=0x7f0b6c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:34.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 -- 172.21.15.157:0/166878792 shutdown_connections 2023-11-06T03:07:34.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.049+0000 7f0b73331640 1 -- 172.21.15.157:0/166878792 wait complete. 2023-11-06T03:07:34.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 Processor -- start 2023-11-06T03:07:34.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- start start 2023-11-06T03:07:34.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c13b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c13ad40 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c150f70 con 0x7f0b6c15ce80 2023-11-06T03:07:34.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c1514a0 con 0x7f0b6c15caa0 2023-11-06T03:07:34.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45296/0 (socket says 172.21.15.157:45296) 2023-11-06T03:07:34.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 -- 172.21.15.157:0/3904562133 learned_addr learned my addr 172.21.15.157:0/3904562133 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:34.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b708cc640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c13b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b718ce640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 -- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 msgr2=0x7f0b6c13b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c13b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 -- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 msgr2=0x7f0b6c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 -- 172.21.15.157:0/3904562133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009420 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b710cd640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800ec10 tx=0x7f0b6800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800cc30 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c009420 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6800cd90 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68013690 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c13e770 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c072750 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.053+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0b680137f0 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.057+0000 7f0b61ffb640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 0x7f0b50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.057+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0b68083740 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.057+0000 7f0b708cc640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 0x7f0b50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.057+0000 7f0b708cc640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 0x7f0b50067f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0af000 tx=0x7f0b5c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.061+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0b68051220 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b6c0d0db0 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b61ffb640 1 -- 172.21.15.157:0/3904562133 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f0b68017df0 con 0x7f0b6c0ce1a0 2023-11-06T03:07:34.249 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:34.249 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/531815515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":531815515},{"type":"v1","addr":"172.21.15.195:6836","nonce":531815515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4008141435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4008141435},{"type":"v1","addr":"172.21.15.195:6837","nonce":4008141435}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3557201699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3557201699},{"type":"v1","addr":"172.21.15.157:6834","nonce":3557201699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1740749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1740749143},{"type":"v1","addr":"172.21.15.157:6835","nonce":1740749143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[]} 2023-11-06T03:07:34.249 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 30 2023-11-06T03:07:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 msgr2=0x7f0b50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 0x7f0b50067f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0af000 tx=0x7f0b5c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:07:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 msgr2=0x7f0b6c150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.249+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800ec10 tx=0x7f0b6800c520 comp rx=0 tx=0).stop 2023-11-06T03:07:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 shutdown_connections 2023-11-06T03:07:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0b50065a60 0x7f0b50067f20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b6c15ce80 0x7f0b6c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b6c15caa0 0x7f0b6c13b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 --2- 172.21.15.157:0/3904562133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b6c0ce1a0 0x7f0b6c150a30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 >> 172.21.15.157:0/3904562133 conn(0x7f0b6c070790 msgr2=0x7f0b6c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:34.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 shutdown_connections 2023-11-06T03:07:34.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.253+0000 7f0b73331640 1 -- 172.21.15.157:0/3904562133 wait complete. 2023-11-06T03:07:34.268 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:07:34.268 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:07:34.269 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:07:34.269 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:07:34.269 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:07:34.269 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:07:34.270 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:07:34.270 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:07:34.270 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:07:34.270 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:07:34.271 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:07:34.271 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:07:34.271 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:07:34.272 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:07:34.272 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:07:34.272 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:07:34.280 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:07:34.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:07:34.282 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:07:34.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:07:34.284 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:07:34.285 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:07:34.286 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:07:34.286 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:07:34.324 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:07:34.324 DEBUG:teuthology.parallel:result is None 2023-11-06T03:07:34.325 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:07:34.325 DEBUG:teuthology.parallel:result is None 2023-11-06T03:07:34.328 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:07:34.328 DEBUG:teuthology.parallel:result is None 2023-11-06T03:07:34.329 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:07:34.329 DEBUG:teuthology.parallel:result is None 2023-11-06T03:07:34.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:07:34.388 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:07:34.392 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:07:34.394 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:07:34.398 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:07:34.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 Processor -- start 2023-11-06T03:07:34.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 -- start start 2023-11-06T03:07:34.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efdf41585d0 0x7efdf4066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efdf40cc380 0x7efdf415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf415bff0 con 0x7efdf40b1730 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf415b620 con 0x7efdf40cc380 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf9d8e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf415aff0 con 0x7efdf41585d0 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.449+0000 7efdf2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efdf40cc380 0x7efdf415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efdf40b1730 0x7efdf40cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51106/0 (socket says 172.21.15.157:51106) 2023-11-06T03:07:34.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf3fff640 1 -- 172.21.15.157:0/209107245 learned_addr learned my addr 172.21.15.157:0/209107245 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/209107245 >> v2:172.21.15.157:3300/0 conn(0x7efdf40cc380 0x7efdf415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45348/0 (socket says 172.21.15.157:45348) 2023-11-06T03:07:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf37fe640 1 --2- 172.21.15.157:0/209107245 >> v2:172.21.15.195:3300/0 conn(0x7efdf41585d0 0x7efdf4066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 -- 172.21.15.157:0/209107245 >> v2:172.21.15.195:3300/0 conn(0x7efdf41585d0 msgr2=0x7efdf4066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/209107245 >> v2:172.21.15.195:3300/0 conn(0x7efdf41585d0 0x7efdf4066340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 -- 172.21.15.157:0/209107245 >> v1:172.21.15.157:6789/0 conn(0x7efdf40b1730 legacy=0x7efdf40cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf27fc640 1 -- 172.21.15.157:0/209107245 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473992523 0 0) 0x7efdec0014a0 con 0x7efdf40b1730 2023-11-06T03:07:34.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 -- 172.21.15.157:0/209107245 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf406dd60 con 0x7efdf40cc380 2023-11-06T03:07:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/209107245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40cc380 0x7efdf415bab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efdf40669f0 tx=0x7efde801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a5316e2028f55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf27fc640 1 -- 172.21.15.157:0/209107245 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80a7070 con 0x7efdf40cc380 2023-11-06T03:07:34.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf27fc640 1 -- 172.21.15.157:0/209107245 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80ab430 con 0x7efdf40cc380 2023-11-06T03:07:34.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf27fc640 1 -- 172.21.15.157:0/209107245 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80ab7c0 con 0x7efdf40cc380 2023-11-06T03:07:34.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/209107245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40cc380 msgr2=0x7efdf415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/209107245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40cc380 0x7efdf415bab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efdf40669f0 tx=0x7efde801c430 comp rx=0 tx=0).stop 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/209107245 shutdown_connections 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/209107245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40cc380 0x7efdf415bab0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/209107245 >> v2:172.21.15.195:3300/0 conn(0x7efdf41585d0 0x7efdf4066340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/209107245 >> 172.21.15.157:0/209107245 conn(0x7efdf40732f0 msgr2=0x7efdf4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/209107245 shutdown_connections 2023-11-06T03:07:34.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/209107245 wait complete. 2023-11-06T03:07:34.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 Processor -- start 2023-11-06T03:07:34.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- start start 2023-11-06T03:07:34.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 0x7efdf40b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdf41585d0 0x7efdf40b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4156f30 con 0x7efdf40b1730 2023-11-06T03:07:34.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4155e90 con 0x7efdf41585d0 2023-11-06T03:07:34.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4156170 con 0x7efdf40cc380 2023-11-06T03:07:34.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45364/0 (socket says 172.21.15.157:45364) 2023-11-06T03:07:34.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 -- 172.21.15.157:0/677628920 learned_addr learned my addr 172.21.15.157:0/677628920 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:34.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf3fff640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdf41585d0 0x7efdf40b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 0x7efdf40b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 -- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 msgr2=0x7efdf40b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 0x7efdf40b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 -- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdf41585d0 msgr2=0x7efdf40b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:34.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdf41585d0 0x7efdf40b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:34.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 -- 172.21.15.157:0/677628920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8009420 con 0x7efdf40b1730 2023-11-06T03:07:34.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 0x7efdf40b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:34.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf37fe640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efde400e870 tx=0x7efde400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde4009510 con 0x7efdf40b1730 2023-11-06T03:07:34.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde400eec0 con 0x7efdf40b1730 2023-11-06T03:07:34.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4156450 con 0x7efdf40b1730 2023-11-06T03:07:34.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde400cb40 con 0x7efdf40b1730 2023-11-06T03:07:34.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf4156a10 con 0x7efdf40b1730 2023-11-06T03:07:34.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4072750 con 0x7efdf40b1730 2023-11-06T03:07:34.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efde400cca0 con 0x7efdf40b1730 2023-11-06T03:07:34.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.457+0000 7efddbfff640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 0x7efde0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:34.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.461+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efde4083070 con 0x7efdf40b1730 2023-11-06T03:07:34.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.461+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 0x7efde0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:34.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.461+0000 7efdf2ffd640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 0x7efde0067f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efde80af000 tx=0x7efde80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:34.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.465+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efde4050bf0 con 0x7efdf40b1730 2023-11-06T03:07:34.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:34.629+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efdf406b3f0 con 0x7efdf40b1730 2023-11-06T03:07:35.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.449+0000 7efddbfff640 1 -- 172.21.15.157:0/677628920 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7efde4017080 con 0x7efdf40b1730 2023-11-06T03:07:35.449 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:07:35.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 msgr2=0x7efde0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 0x7efde0067f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efde80af000 tx=0x7efde80a7000 comp rx=0 tx=0).stop 2023-11-06T03:07:35.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 msgr2=0x7efdf41372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efde400e870 tx=0x7efde400ec70 comp rx=0 tx=0).stop 2023-11-06T03:07:35.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 shutdown_connections 2023-11-06T03:07:35.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efde0065a90 0x7efde0067f50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efdf41585d0 0x7efdf40b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efdf40cc380 0x7efdf40b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 --2- 172.21.15.157:0/677628920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdf40b1730 0x7efdf41372c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 >> 172.21.15.157:0/677628920 conn(0x7efdf40732f0 msgr2=0x7efdf4158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:35.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 shutdown_connections 2023-11-06T03:07:35.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.453+0000 7efdf9d8e640 1 -- 172.21.15.157:0/677628920 wait complete. 2023-11-06T03:07:35.473 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:07:35.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.597+0000 7fcb59b01640 1 Processor -- start 2023-11-06T03:07:35.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.597+0000 7fcb59b01640 1 -- start start 2023-11-06T03:07:35.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb59b01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb540ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb59b01640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb5406d000 0x7fcb54159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb59b01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5415e440 con 0x7fcb541599c0 2023-11-06T03:07:35.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb59b01640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5406c240 con 0x7fcb5406d000 2023-11-06T03:07:35.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb59b01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5406c520 con 0x7fcb540b2af0 2023-11-06T03:07:35.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb537fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcb541599c0 0x7fcb540b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51116/0 (socket says 172.21.15.157:51116) 2023-11-06T03:07:35.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb537fe640 1 -- 172.21.15.157:0/1997823754 learned_addr learned my addr 172.21.15.157:0/1997823754 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:35.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb4affd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb540ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 --2- 172.21.15.157:0/1997823754 >> v2:172.21.15.195:3300/0 conn(0x7fcb5406d000 0x7fcb54159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439200608 0 0) 0x7fcb5415e440 con 0x7fcb541599c0 2023-11-06T03:07:35.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb40005740 con 0x7fcb541599c0 2023-11-06T03:07:35.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260504317 0 0) 0x7fcb40005740 con 0x7fcb541599c0 2023-11-06T03:07:35.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 msgr2=0x7fcb540ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:35.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb540ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 >> v2:172.21.15.195:3300/0 conn(0x7fcb5406d000 msgr2=0x7fcb54159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:35.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 --2- 172.21.15.157:0/1997823754 >> v2:172.21.15.195:3300/0 conn(0x7fcb5406d000 0x7fcb54159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5406c800 con 0x7fcb541599c0 2023-11-06T03:07:35.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fcb44001680 con 0x7fcb541599c0 2023-11-06T03:07:35.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 >> v1:172.21.15.157:6789/0 conn(0x7fcb541599c0 legacy=0x7fcb540b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb40005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb40006110 0x7fcb400069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb40006f20 0x7fcb400073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5406c240 con 0x7fcb40006f20 2023-11-06T03:07:35.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5406c520 con 0x7fcb40006110 2023-11-06T03:07:35.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4000dba0 con 0x7fcb540b2af0 2023-11-06T03:07:35.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb40006f20 0x7fcb400073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb537fe640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb40005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb4affd640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb40006110 0x7fcb400069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 -- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 msgr2=0x7fcb40005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb40005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 -- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb40006110 msgr2=0x7fcb400069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb40006110 0x7fcb400069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb53fff640 1 -- 172.21.15.157:0/1997823754 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40005740 con 0x7fcb40006f20 2023-11-06T03:07:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.601+0000 7fcb537fe640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb540b2af0 0x7fcb40005b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb53fff640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb40006f20 0x7fcb400073a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406d430 tx=0x7fcb4c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f44c23b39480d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0a5c50 con 0x7fcb40006f20 2023-11-06T03:07:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c00f040 con 0x7fcb40006f20 2023-11-06T03:07:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb52ffd640 1 -- 172.21.15.157:0/1997823754 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0ad440 con 0x7fcb40006f20 2023-11-06T03:07:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- 172.21.15.157:0/1997823754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb40006f20 msgr2=0x7fcb400073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 --2- 172.21.15.157:0/1997823754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb40006f20 0x7fcb400073a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406d430 tx=0x7fcb4c01cbe0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb53fff640 1 -- 172.21.15.157:0/1997823754 reap_dead start 2023-11-06T03:07:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- 172.21.15.157:0/1997823754 shutdown_connections 2023-11-06T03:07:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- 172.21.15.157:0/1997823754 >> 172.21.15.157:0/1997823754 conn(0x7fcb54065930 msgr2=0x7fcb54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- 172.21.15.157:0/1997823754 shutdown_connections 2023-11-06T03:07:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- 172.21.15.157:0/1997823754 wait complete. 2023-11-06T03:07:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 Processor -- start 2023-11-06T03:07:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- start start 2023-11-06T03:07:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415c3e0 0x7fcb541f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb541f3030 0x7fcb541f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540cc3a0 con 0x7fcb540cba10 2023-11-06T03:07:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541f23c0 con 0x7fcb5415c3e0 2023-11-06T03:07:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb59b01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541f2660 con 0x7fcb541f3030 2023-11-06T03:07:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45390/0 (socket says 172.21.15.157:45390) 2023-11-06T03:07:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 -- 172.21.15.157:0/3857412861 learned_addr learned my addr 172.21.15.157:0/3857412861 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb537fe640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415c3e0 0x7fcb541f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb53fff640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb541f3030 0x7fcb541f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 -- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb541f3030 msgr2=0x7fcb541f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb541f3030 0x7fcb541f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 -- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415c3e0 msgr2=0x7fcb541f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415c3e0 0x7fcb541f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 -- 172.21.15.157:0/3857412861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540ccb70 con 0x7fcb540cba10 2023-11-06T03:07:35.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.605+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c011560 tx=0x7fcb3c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c015930 con 0x7fcb540cba10 2023-11-06T03:07:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c015a90 con 0x7fcb540cba10 2023-11-06T03:07:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5415cdb0 con 0x7fcb540cba10 2023-11-06T03:07:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c015e60 con 0x7fcb540cba10 2023-11-06T03:07:35.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb540cc890 con 0x7fcb540cba10 2023-11-06T03:07:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb541f27d0 con 0x7fcb540cba10 2023-11-06T03:07:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcb3c00f460 con 0x7fcb540cba10 2023-11-06T03:07:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 0x7fcb34067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcb3c0837d0 con 0x7fcb540cba10 2023-11-06T03:07:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb537fe640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 0x7fcb34067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.609+0000 7fcb537fe640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 0x7fcb34067f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001060 tx=0x7fcb44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.613+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcb3c051370 con 0x7fcb540cba10 2023-11-06T03:07:35.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcb54072750 con 0x7fcb540cba10 2023-11-06T03:07:35.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb50ff9640 1 -- 172.21.15.157:0/3857412861 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcb3c054990 con 0x7fcb540cba10 2023-11-06T03:07:35.808 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:35.808 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:07:35.809 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:07:35.809 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.809 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:07:35.809 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:07:35.810 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.810 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.810 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:07:35.810 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:07:35.810 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.811 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.811 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.811 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.811 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:07:35.811 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:07:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:07:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:07:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.813 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.813 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.813 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.813 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:07:35.813 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:07:35.814 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.814 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.814 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:07:35.814 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:07:35.814 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:07:35.815 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:07:35.816 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.816 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.816 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:07:35.816 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:07:35.816 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.819 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.819 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.819 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:07:35.819 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:07:35.819 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.820 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.820 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.820 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.820 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.820 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.826 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.826 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.826 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.826 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:07:35.826 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:07:35.831 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:07:35.831 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.831 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.831 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.831 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.832 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:07:35.832 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:07:35.832 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.832 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.833 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:07:35.833 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:07:35.833 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.833 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.833 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.834 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:07:35.834 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:07:35.834 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.834 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.834 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:07:35.835 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:07:35.835 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:07:35.835 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.835 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.835 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.836 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:07:35.836 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:07:35.836 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.836 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.836 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:07:35.837 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:07:35.837 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:07:35.837 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.837 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.837 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.838 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:07:35.838 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:07:35.838 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.838 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:07:35.838 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:07:35.839 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:07:35.839 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:07:35.839 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.839 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.839 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:07:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:07:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:07:35.840 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.841 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:07:35.842 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:07:35.842 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.842 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:07:35.842 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:07:35.842 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:07:35.843 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:07:35.844 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:07:35.844 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:07:35.844 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:07:35.844 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:07:35.844 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.845 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:35.845 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:07:35.845 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:07:35.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 msgr2=0x7fcb34067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 0x7fcb34067f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001060 tx=0x7fcb44009040 comp rx=0 tx=0).stop 2023-11-06T03:07:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 msgr2=0x7fcb541f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c011560 tx=0x7fcb3c011a20 comp rx=0 tx=0).stop 2023-11-06T03:07:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 shutdown_connections 2023-11-06T03:07:35.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb34065a60 0x7fcb34067f20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb541f3030 0x7fcb541f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb540cba10 0x7fcb541f2af0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 --2- 172.21.15.157:0/3857412861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb5415c3e0 0x7fcb541f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 >> 172.21.15.157:0/3857412861 conn(0x7fcb54065930 msgr2=0x7fcb54157450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:35.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 shutdown_connections 2023-11-06T03:07:35.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.809+0000 7fcb59b01640 1 -- 172.21.15.157:0/3857412861 wait complete. 2023-11-06T03:07:35.849 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:35.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 Processor -- start 2023-11-06T03:07:35.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 -- start start 2023-11-06T03:07:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f27dc15ce30 0x7f27dc0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 0x7f27dc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc15d210 con 0x7f27dc0cf580 2023-11-06T03:07:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc0d17f0 con 0x7f27dc15ce30 2023-11-06T03:07:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e298c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0ce650 con 0x7f27dc0cd520 2023-11-06T03:07:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27d3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 0x7f27dc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27dbfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f27dc15ce30 0x7f27dc0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e0f29640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f27dc0cf580 0x7f27dc0cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:59334/0 (socket says 172.21.15.157:59334) 2023-11-06T03:07:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27e0f29640 1 -- 172.21.15.157:0/371228924 learned_addr learned my addr 172.21.15.157:0/371228924 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:35.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27dbfff640 1 -- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 msgr2=0x7f27dc0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:35.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27dbfff640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 0x7f27dc0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27dbfff640 1 -- 172.21.15.157:0/371228924 >> v1:172.21.15.195:6789/0 conn(0x7f27dc0cf580 legacy=0x7f27dc0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27dbfff640 1 -- 172.21.15.157:0/371228924 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0ce090 con 0x7f27dc15ce30 2023-11-06T03:07:35.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.969+0000 7f27d3fff640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 0x7f27dc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:07:35.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27dbfff640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc15ce30 0x7f27dc0b5f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0097c0 tx=0x7f27cc01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc92e6c94e933514 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27db7fe640 1 -- 172.21.15.157:0/371228924 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0ab600 con 0x7f27dc15ce30 2023-11-06T03:07:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27db7fe640 1 -- 172.21.15.157:0/371228924 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc0a7050 con 0x7f27dc15ce30 2023-11-06T03:07:35.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27db7fe640 1 -- 172.21.15.157:0/371228924 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0ab980 con 0x7f27dc15ce30 2023-11-06T03:07:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc15ce30 msgr2=0x7f27dc0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc15ce30 0x7f27dc0b5f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0097c0 tx=0x7f27cc01c360 comp rx=0 tx=0).stop 2023-11-06T03:07:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- 172.21.15.157:0/371228924 shutdown_connections 2023-11-06T03:07:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cd520 0x7f27dc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- 172.21.15.157:0/371228924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc15ce30 0x7f27dc0b5f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- 172.21.15.157:0/371228924 >> 172.21.15.157:0/371228924 conn(0x7f27dc070990 msgr2=0x7f27dc065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- 172.21.15.157:0/371228924 shutdown_connections 2023-11-06T03:07:35.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- 172.21.15.157:0/371228924 wait complete. 2023-11-06T03:07:35.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 Processor -- start 2023-11-06T03:07:35.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- start start 2023-11-06T03:07:35.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc0cd520 0x7f27dc15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cf580 0x7f27dc1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc15a900 con 0x7f27dc15ce30 2023-11-06T03:07:35.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1533f0 con 0x7f27dc0cd520 2023-11-06T03:07:35.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e298c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc152020 con 0x7f27dc0cf580 2023-11-06T03:07:35.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e0f29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e0f29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45392/0 (socket says 172.21.15.157:45392) 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.973+0000 7f27e0f29640 1 -- 172.21.15.157:0/475533744 learned_addr learned my addr 172.21.15.157:0/475533744 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27dbfff640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc0cd520 0x7f27dc15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d3fff640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cf580 0x7f27dc1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 -- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cf580 msgr2=0x7f27dc1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cf580 0x7f27dc1538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 -- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc0cd520 msgr2=0x7f27dc15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc0cd520 0x7f27dc15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 -- 172.21.15.157:0/475533744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c4009420 con 0x7f27dc15ce30 2023-11-06T03:07:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e0f29640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f27d400cbc0 tx=0x7f27d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400ad40 con 0x7f27dc15ce30 2023-11-06T03:07:35.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4012400 con 0x7f27dc15ce30 2023-11-06T03:07:35.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc009420 con 0x7f27dc15ce30 2023-11-06T03:07:35.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400aea0 con 0x7f27dc15ce30 2023-11-06T03:07:35.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc1525e0 con 0x7f27dc15ce30 2023-11-06T03:07:35.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc072750 con 0x7f27dc15ce30 2023-11-06T03:07:35.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f27d4007070 con 0x7f27dc15ce30 2023-11-06T03:07:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 0x7f27bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f27d4082e20 con 0x7f27dc15ce30 2023-11-06T03:07:35.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27dbfff640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 0x7f27bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.977+0000 7f27dbfff640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 0x7f27bc067f20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ad000 tx=0x7f27cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:35.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:35.981+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f27d4050a80 con 0x7f27dc15ce30 2023-11-06T03:07:36.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.141+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27dc1609d0 con 0x7f27dc15ce30 2023-11-06T03:07:36.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.141+0000 7f27d97fa640 1 -- 172.21.15.157:0/475533744 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+14219 (secure 0 0 0) 0x7f27d4017020 con 0x7f27dc15ce30 2023-11-06T03:07:36.142 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:36.142 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":49,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:35.437971+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:36.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 msgr2=0x7f27bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 0x7f27bc067f20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ad000 tx=0x7f27cc008040 comp rx=0 tx=0).stop 2023-11-06T03:07:36.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 msgr2=0x7f27dc151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f27d400cbc0 tx=0x7f27d400a430 comp rx=0 tx=0).stop 2023-11-06T03:07:36.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 shutdown_connections 2023-11-06T03:07:36.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27bc065a60 0x7f27bc067f20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15ce30 0x7f27dc151060 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27dc0cf580 0x7f27dc1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 --2- 172.21.15.157:0/475533744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27dc0cd520 0x7f27dc15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 >> 172.21.15.157:0/475533744 conn(0x7f27dc070990 msgr2=0x7f27dc15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 shutdown_connections 2023-11-06T03:07:36.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.145+0000 7f27e298c640 1 -- 172.21.15.157:0/475533744 wait complete. 2023-11-06T03:07:36.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:07:36.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:07:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 Processor -- start 2023-11-06T03:07:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 -- start start 2023-11-06T03:07:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b50160490 0x7f6b500cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160950 0x7f6b50160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b501612f0 con 0x7f6b5015d360 2023-11-06T03:07:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b50161780 con 0x7f6b50160490 2023-11-06T03:07:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b552f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500d0450 con 0x7f6b50160950 2023-11-06T03:07:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b50160490 0x7f6b500cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b47fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160950 0x7f6b50160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6b50160490 0x7f6b500cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:44454/0 (socket says 172.21.15.157:44454) 2023-11-06T03:07:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 -- 172.21.15.157:0/993491167 learned_addr learned my addr 172.21.15.157:0/993491167 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 -- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160950 msgr2=0x7f6b50160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 --2- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160950 0x7f6b50160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 -- 172.21.15.157:0/993491167 >> v1:172.21.15.195:6789/0 conn(0x7f6b5015d360 legacy=0x7f6b5015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.285+0000 7f6b4effd640 1 -- 172.21.15.157:0/993491167 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500cd860 con 0x7f6b50160490 2023-11-06T03:07:36.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4effd640 1 --2- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b50160490 0x7f6b500cff10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009870 tx=0x7f6b4001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73444d7a4565d364 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4e7fc640 1 -- 172.21.15.157:0/993491167 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400aa670 con 0x7f6b50160490 2023-11-06T03:07:36.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4e7fc640 1 -- 172.21.15.157:0/993491167 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b400aa7d0 con 0x7f6b50160490 2023-11-06T03:07:36.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4e7fc640 1 -- 172.21.15.157:0/993491167 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400aab60 con 0x7f6b50160490 2023-11-06T03:07:36.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b50160490 msgr2=0x7f6b500cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b50160490 0x7f6b500cff10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009870 tx=0x7f6b4001c510 comp rx=0 tx=0).stop 2023-11-06T03:07:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- 172.21.15.157:0/993491167 shutdown_connections 2023-11-06T03:07:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160950 0x7f6b50160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- 172.21.15.157:0/993491167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b50160490 0x7f6b500cff10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- 172.21.15.157:0/993491167 >> 172.21.15.157:0/993491167 conn(0x7f6b50070790 msgr2=0x7f6b50070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- 172.21.15.157:0/993491167 shutdown_connections 2023-11-06T03:07:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- 172.21.15.157:0/993491167 wait complete. 2023-11-06T03:07:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 Processor -- start 2023-11-06T03:07:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- start start 2023-11-06T03:07:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 0x7f6b501592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160490 0x7f6b500c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50154490 con 0x7f6b50160950 2023-11-06T03:07:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50159940 con 0x7f6b5015d360 2023-11-06T03:07:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b552f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50159c20 con 0x7f6b50160490 2023-11-06T03:07:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45406/0 (socket says 172.21.15.157:45406) 2023-11-06T03:07:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4f7fe640 1 -- 172.21.15.157:0/3352067262 learned_addr learned my addr 172.21.15.157:0/3352067262 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b4effd640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 0x7f6b501592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.289+0000 7f6b47fff640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160490 0x7f6b500c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 -- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160490 msgr2=0x7f6b500c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160490 0x7f6b500c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 -- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 msgr2=0x7f6b501592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 0x7f6b501592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 -- 172.21.15.157:0/3352067262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400094d0 con 0x7f6b50160950 2023-11-06T03:07:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4effd640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 0x7f6b501592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4f7fe640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c770 tx=0x7f6b4800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003b80 con 0x7f6b50160950 2023-11-06T03:07:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b48003ce0 con 0x7f6b50160950 2023-11-06T03:07:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b50159f00 con 0x7f6b50160950 2023-11-06T03:07:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800aa90 con 0x7f6b50160950 2023-11-06T03:07:36.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b50157ce0 con 0x7f6b50160950 2023-11-06T03:07:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b50072750 con 0x7f6b50160950 2023-11-06T03:07:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6b4800abf0 con 0x7f6b50160950 2023-11-06T03:07:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 0x7f6b30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6b48082e20 con 0x7f6b50160950 2023-11-06T03:07:36.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4effd640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 0x7f6b30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.293+0000 7f6b4effd640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 0x7f6b30067f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ae000 tx=0x7f6b400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.297+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6b480509d0 con 0x7f6b50160950 2023-11-06T03:07:36.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.437+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b500d09d0 con 0x7f6b50160950 2023-11-06T03:07:36.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b477fe640 1 -- 172.21.15.157:0/3352067262 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b48053ff0 con 0x7f6b50160950 2023-11-06T03:07:36.439 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:07:36.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 msgr2=0x7f6b30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 0x7f6b30067f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6b400ae000 tx=0x7f6b400a7000 comp rx=0 tx=0).stop 2023-11-06T03:07:36.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 msgr2=0x7f6b50153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c770 tx=0x7f6b4800cc30 comp rx=0 tx=0).stop 2023-11-06T03:07:36.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 shutdown_connections 2023-11-06T03:07:36.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b30065a60 0x7f6b30067f20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b50160950 0x7f6b50153f50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b50160490 0x7f6b500c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 --2- 172.21.15.157:0/3352067262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b5015d360 0x7f6b501592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 >> 172.21.15.157:0/3352067262 conn(0x7f6b50070790 msgr2=0x7f6b5015b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 shutdown_connections 2023-11-06T03:07:36.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.441+0000 7f6b552f9640 1 -- 172.21.15.157:0/3352067262 wait complete. 2023-11-06T03:07:36.457 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:07:36.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 Processor -- start 2023-11-06T03:07:36.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 -- start start 2023-11-06T03:07:36.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f140719e0 0x7f1f140cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140cce90 0x7f1f140cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140d0140 con 0x7f1f1415e090 2023-11-06T03:07:36.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140cc420 con 0x7f1f140719e0 2023-11-06T03:07:36.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f1aab7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140d0380 con 0x7f1f140cce90 2023-11-06T03:07:36.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f140719e0 0x7f1f140cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f140719e0 0x7f1f140cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45410/0 (socket says 172.21.15.157:45410) 2023-11-06T03:07:36.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 -- 172.21.15.157:0/1192374215 learned_addr learned my addr 172.21.15.157:0/1192374215 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f19054640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140cce90 0x7f1f140cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 -- 172.21.15.157:0/1192374215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140cce90 msgr2=0x7f1f140cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:36.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140cce90 0x7f1f140cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 -- 172.21.15.157:0/1192374215 >> v1:172.21.15.195:6789/0 conn(0x7f1f1415e090 legacy=0x7f1f140cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.573+0000 7f1f13fff640 1 -- 172.21.15.157:0/1192374215 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140cc690 con 0x7f1f140719e0 2023-11-06T03:07:36.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140719e0 0x7f1f140cd320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009870 tx=0x7f1f0001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d000f0d4d71c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f137fe640 1 -- 172.21.15.157:0/1192374215 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000a8a30 con 0x7f1f140719e0 2023-11-06T03:07:36.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f137fe640 1 -- 172.21.15.157:0/1192374215 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f000a8b90 con 0x7f1f140719e0 2023-11-06T03:07:36.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f137fe640 1 -- 172.21.15.157:0/1192374215 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000b0440 con 0x7f1f140719e0 2023-11-06T03:07:36.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- 172.21.15.157:0/1192374215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140719e0 msgr2=0x7f1f140cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140719e0 0x7f1f140cd320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009870 tx=0x7f1f0001ed30 comp rx=0 tx=0).stop 2023-11-06T03:07:36.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- 172.21.15.157:0/1192374215 shutdown_connections 2023-11-06T03:07:36.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140cce90 0x7f1f140cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/1192374215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140719e0 0x7f1f140cd320 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- 172.21.15.157:0/1192374215 >> 172.21.15.157:0/1192374215 conn(0x7f1f1406d3d0 msgr2=0x7f1f1406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- 172.21.15.157:0/1192374215 shutdown_connections 2023-11-06T03:07:36.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- 172.21.15.157:0/1192374215 wait complete. 2023-11-06T03:07:36.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 Processor -- start 2023-11-06T03:07:36.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- start start 2023-11-06T03:07:36.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 0x7f1f14159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 0x7f1f1415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140a92d0 con 0x7f1f140cce90 2023-11-06T03:07:36.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141f10b0 con 0x7f1f1415e090 2023-11-06T03:07:36.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f1aab7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140a8650 con 0x7f1f140719e0 2023-11-06T03:07:36.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45426/0 (socket says 172.21.15.157:45426) 2023-11-06T03:07:36.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 -- 172.21.15.157:0/2462131889 learned_addr learned my addr 172.21.15.157:0/2462131889 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f18853640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 0x7f1f14159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f19054640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 0x7f1f1415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 -- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 msgr2=0x7f1f14159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 0x7f1f14159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 -- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 msgr2=0x7f1f1415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 0x7f1f1415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f13fff640 1 -- 172.21.15.157:0/2462131889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000094d0 con 0x7f1f140cce90 2023-11-06T03:07:36.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f18853640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 0x7f1f14159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:36.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.577+0000 7f1f19054640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 0x7f1f1415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:36.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f13fff640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1f000b3040 tx=0x7f1f0001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000be070 con 0x7f1f140cce90 2023-11-06T03:07:36.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f000ba400 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140a8930 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000ba7b0 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f140a8ef0 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14072750 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f000ba910 con 0x7f1f140cce90 2023-11-06T03:07:36.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1f00120d10 con 0x7f1f140cce90 2023-11-06T03:07:36.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f18853640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.581+0000 7f1f18853640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 0x7f1ef8067f20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f140c6aa0 tx=0x7f1f08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.585+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f000ee950 con 0x7f1f140cce90 2023-11-06T03:07:36.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.737+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f140b6260 con 0x7f1f140cce90 2023-11-06T03:07:36.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f117fa640 1 -- 172.21.15.157:0/2462131889 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f000f1f70 con 0x7f1f140cce90 2023-11-06T03:07:36.739 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:07:36.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 msgr2=0x7f1ef8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 0x7f1ef8067f20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f140c6aa0 tx=0x7f1f08009040 comp rx=0 tx=0).stop 2023-11-06T03:07:36.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 msgr2=0x7f1f140c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1f000b3040 tx=0x7f1f0001bad0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 shutdown_connections 2023-11-06T03:07:36.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f1415e090 0x7f1f1415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f140cce90 0x7f1f140c97c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 --2- 172.21.15.157:0/2462131889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f140719e0 0x7f1f14159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 >> 172.21.15.157:0/2462131889 conn(0x7f1f1406d3d0 msgr2=0x7f1f1415be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 shutdown_connections 2023-11-06T03:07:36.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.741+0000 7f1f1aab7640 1 -- 172.21.15.157:0/2462131889 wait complete. 2023-11-06T03:07:36.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:07:36.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 Processor -- start 2023-11-06T03:07:36.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 -- start start 2023-11-06T03:07:36.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5c9c1551e0 0x7f5c9c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c15f580 con 0x7f5c9c15ea90 2023-11-06T03:07:36.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c15efc0 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5ca27c5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c15f2a0 con 0x7f5c9c1551e0 2023-11-06T03:07:36.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45434/0 (socket says 172.21.15.157:45434) 2023-11-06T03:07:36.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 -- 172.21.15.157:0/3019311249 learned_addr learned my addr 172.21.15.157:0/3019311249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9bfff640 1 --2- 172.21.15.157:0/3019311249 >> v2:172.21.15.195:3300/0 conn(0x7f5c9c1551e0 0x7f5c9c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 -- 172.21.15.157:0/3019311249 >> v2:172.21.15.195:3300/0 conn(0x7f5c9c1551e0 msgr2=0x7f5c9c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:36.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 --2- 172.21.15.157:0/3019311249 >> v2:172.21.15.195:3300/0 conn(0x7f5c9c1551e0 0x7f5c9c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 -- 172.21.15.157:0/3019311249 >> v1:172.21.15.195:6789/0 conn(0x7f5c9c15ea90 legacy=0x7f5c9c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.869+0000 7f5c9b7fe640 1 -- 172.21.15.157:0/3019311249 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c06f5a0 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5c9b7fe640 1 --2- 172.21.15.157:0/3019311249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c066830 tx=0x7f5c8401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b8cb77358b5f3e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5c9affd640 1 -- 172.21.15.157:0/3019311249 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84004230 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5c9affd640 1 -- 172.21.15.157:0/3019311249 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c840a7050 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5c9affd640 1 -- 172.21.15.157:0/3019311249 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84003620 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3019311249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15e5e0 msgr2=0x7f5c9c0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3019311249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c066830 tx=0x7f5c8401c440 comp rx=0 tx=0).stop 2023-11-06T03:07:36.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3019311249 shutdown_connections 2023-11-06T03:07:36.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3019311249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15e5e0 0x7f5c9c0b3410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3019311249 >> v2:172.21.15.195:3300/0 conn(0x7f5c9c1551e0 0x7f5c9c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3019311249 >> 172.21.15.157:0/3019311249 conn(0x7f5c9c065930 msgr2=0x7f5c9c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:36.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3019311249 shutdown_connections 2023-11-06T03:07:36.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3019311249 wait complete. 2023-11-06T03:07:36.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 Processor -- start 2023-11-06T03:07:36.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.873+0000 7f5ca27c5640 1 -- start start 2023-11-06T03:07:36.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 0x7f5c9c1f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c9c15e5e0 0x7f5c9c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c14d680 con 0x7f5c9c15ea90 2023-11-06T03:07:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1f4520 con 0x7f5c9c1551e0 2023-11-06T03:07:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1f4800 con 0x7f5c9c15e5e0 2023-11-06T03:07:36.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45440/0 (socket says 172.21.15.157:45440) 2023-11-06T03:07:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 -- 172.21.15.157:0/3332207370 learned_addr learned my addr 172.21.15.157:0/3332207370 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c9bfff640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 0x7f5c9c1f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 -- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c9c15e5e0 msgr2=0x7f5c9c14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c9c15e5e0 0x7f5c9c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 -- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 msgr2=0x7f5c9c1f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 0x7f5c9c1f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 -- 172.21.15.157:0/3332207370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840094d0 con 0x7f5c9c15ea90 2023-11-06T03:07:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c9bfff640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 0x7f5c9c1f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:36.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca0d62640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c340 tx=0x7f5c9400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94005220 con 0x7f5c9c15ea90 2023-11-06T03:07:36.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94005380 con 0x7f5c9c15ea90 2023-11-06T03:07:36.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94007270 con 0x7f5c9c15ea90 2023-11-06T03:07:36.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c1f4ae0 con 0x7f5c9c15ea90 2023-11-06T03:07:36.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c1f50a0 con 0x7f5c9c15ea90 2023-11-06T03:07:36.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c072750 con 0x7f5c9c15ea90 2023-11-06T03:07:36.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5c940073d0 con 0x7f5c9c15ea90 2023-11-06T03:07:36.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 0x7f5c7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:36.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5c94082470 con 0x7f5c9c15ea90 2023-11-06T03:07:36.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.877+0000 7f5c9bfff640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 0x7f5c7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:36.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.881+0000 7f5c9bfff640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 0x7f5c7c067f20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007330 tx=0x7f5c8c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:36.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:36.881+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5c940500b0 con 0x7f5c9c15ea90 2023-11-06T03:07:37.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.033+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5c9c1f5290 con 0x7f5c9c15ea90 2023-11-06T03:07:37.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5c98ff9640 1 -- 172.21.15.157:0/3332207370 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5c940536d0 con 0x7f5c9c15ea90 2023-11-06T03:07:37.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:37.035 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:07:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:07:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:07:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:37.036 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:07:37.037 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:07:37.037 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:07:37.037 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:37.037 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:07:37.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 msgr2=0x7f5c7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 0x7f5c7c067f20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007330 tx=0x7f5c8c00b040 comp rx=0 tx=0).stop 2023-11-06T03:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 msgr2=0x7f5c9c153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c340 tx=0x7f5c9400c800 comp rx=0 tx=0).stop 2023-11-06T03:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 shutdown_connections 2023-11-06T03:07:37.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c7c065a60 0x7f5c7c067f20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c9c15ea90 0x7f5c9c153cc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c9c15e5e0 0x7f5c9c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 --2- 172.21.15.157:0/3332207370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c9c1551e0 0x7f5c9c1f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 >> 172.21.15.157:0/3332207370 conn(0x7f5c9c065930 msgr2=0x7f5c9c0aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 shutdown_connections 2023-11-06T03:07:37.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.037+0000 7f5ca27c5640 1 -- 172.21.15.157:0/3332207370 wait complete. 2023-11-06T03:07:37.053 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:07:37.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:07:37.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 Processor -- start 2023-11-06T03:07:37.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 -- start start 2023-11-06T03:07:37.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b406c170 0x7f75b4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40cb100 con 0x7f75b415b860 2023-11-06T03:07:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40ce0a0 con 0x7f75b406c550 2023-11-06T03:07:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bc7bc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b415bcf0 con 0x7f75b406c170 2023-11-06T03:07:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b406c170 0x7f75b4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75b406c170 0x7f75b4154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45454/0 (socket says 172.21.15.157:45454) 2023-11-06T03:07:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 -- 172.21.15.157:0/29769754 learned_addr learned my addr 172.21.15.157:0/29769754 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75bad59640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75b415b860 0x7f75b40ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51120/0 (socket says 172.21.15.157:51120) 2023-11-06T03:07:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 -- 172.21.15.157:0/29769754 >> v1:172.21.15.195:6789/0 conn(0x7f75b406c550 legacy=0x7f75b415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 -- 172.21.15.157:0/29769754 >> v1:172.21.15.157:6789/0 conn(0x7f75b415b860 legacy=0x7f75b40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.161+0000 7f75ba558640 1 -- 172.21.15.157:0/29769754 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b415bef0 con 0x7f75b406c170 2023-11-06T03:07:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75ba558640 1 --2- 172.21.15.157:0/29769754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b406c170 0x7f75b4154c00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009870 tx=0x7f75b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1942487c3873cb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75b9556640 1 -- 172.21.15.157:0/29769754 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00aa430 con 0x7f75b406c170 2023-11-06T03:07:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75b9556640 1 -- 172.21.15.157:0/29769754 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00aa590 con 0x7f75b406c170 2023-11-06T03:07:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75b9556640 1 -- 172.21.15.157:0/29769754 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00aa920 con 0x7f75b406c170 2023-11-06T03:07:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- 172.21.15.157:0/29769754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b406c170 msgr2=0x7f75b4154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/29769754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b406c170 0x7f75b4154c00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009870 tx=0x7f75b001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- 172.21.15.157:0/29769754 shutdown_connections 2023-11-06T03:07:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/29769754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b406c170 0x7f75b4154c00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- 172.21.15.157:0/29769754 >> 172.21.15.157:0/29769754 conn(0x7f75b40b43b0 msgr2=0x7f75b4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:37.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- 172.21.15.157:0/29769754 shutdown_connections 2023-11-06T03:07:37.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- 172.21.15.157:0/29769754 wait complete. 2023-11-06T03:07:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 Processor -- start 2023-11-06T03:07:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.165+0000 7f75bc7bc640 1 -- start start 2023-11-06T03:07:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 0x7f75b41458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75b406c550 0x7f75b4145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4153bc0 con 0x7f75b415b860 2023-11-06T03:07:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4153ea0 con 0x7f75b406c550 2023-11-06T03:07:37.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b414cf30 con 0x7f75b406c170 2023-11-06T03:07:37.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:37.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45468/0 (socket says 172.21.15.157:45468) 2023-11-06T03:07:37.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 -- 172.21.15.157:0/490330899 learned_addr learned my addr 172.21.15.157:0/490330899 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:37.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75b9d57640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75b406c550 0x7f75b4145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:37.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75ba558640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 0x7f75b41458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:37.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 -- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 msgr2=0x7f75b41458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 0x7f75b41458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 -- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75b406c550 msgr2=0x7f75b4145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:37.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75b406c550 0x7f75b4145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:37.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 -- 172.21.15.157:0/490330899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00094d0 con 0x7f75b415b860 2023-11-06T03:07:37.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75ba558640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 0x7f75b41458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:37.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bad59640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c700 tx=0x7f75ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:37.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00ce10 con 0x7f75b415b860 2023-11-06T03:07:37.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac00a640 con 0x7f75b415b860 2023-11-06T03:07:37.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00a9b0 con 0x7f75b415b860 2023-11-06T03:07:37.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b414d210 con 0x7f75b415b860 2023-11-06T03:07:37.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b41455c0 con 0x7f75b415b860 2023-11-06T03:07:37.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b414d430 con 0x7f75b415b860 2023-11-06T03:07:37.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75ac00ab10 con 0x7f75b415b860 2023-11-06T03:07:37.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 0x7f7598067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:37.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75ac082660 con 0x7f75b415b860 2023-11-06T03:07:37.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.169+0000 7f75ba558640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 0x7f7598067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:37.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.173+0000 7f75ba558640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 0x7f7598067f20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ae000 tx=0x7f75b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:37.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.173+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f75ac050350 con 0x7f75b415b860 2023-11-06T03:07:37.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:37.333+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f75b404e050 con 0x7f75b415b860 2023-11-06T03:07:38.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.265+0000 7f75aaffd640 1 -- 172.21.15.157:0/490330899 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) v1 ==== 138+0+0 (secure 0 0 0) 0x7f75ac0170b0 con 0x7f75b415b860 2023-11-06T03:07:38.264 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:07:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 msgr2=0x7f7598067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 0x7f7598067f20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ae000 tx=0x7f75b0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 msgr2=0x7f75b414c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c700 tx=0x7f75ac00cbc0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 shutdown_connections 2023-11-06T03:07:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7598065a60 0x7f7598067f20 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ae000 tx=0x7f75b0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75b415b860 0x7f75b414c5d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75b406c550 0x7f75b4145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 --2- 172.21.15.157:0/490330899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75b406c170 0x7f75b41458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.269+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 >> 172.21.15.157:0/490330899 conn(0x7f75b40b43b0 msgr2=0x7f75b415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.273+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 shutdown_connections 2023-11-06T03:07:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.273+0000 7f75bc7bc640 1 -- 172.21.15.157:0/490330899 wait complete. 2023-11-06T03:07:38.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:07:38.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.413+0000 7fe3704e7640 1 Processor -- start 2023-11-06T03:07:38.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 -- start start 2023-11-06T03:07:38.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3681551e0 0x7fe36806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3680b3b20 0x7fe36815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36815f580 con 0x7fe36815e5e0 2023-11-06T03:07:38.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36815efc0 con 0x7fe3681551e0 2023-11-06T03:07:38.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe3704e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815f2a0 con 0x7fe3680b3b20 2023-11-06T03:07:38.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3681551e0 0x7fe36806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36da82640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe36815e5e0 0x7fe3680b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51134/0 (socket says 172.21.15.157:51134) 2023-11-06T03:07:38.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36da82640 1 -- 172.21.15.157:0/4181343256 learned_addr learned my addr 172.21.15.157:0/4181343256 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:38.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36ea84640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3680b3b20 0x7fe36815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 -- 172.21.15.157:0/4181343256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3680b3b20 msgr2=0x7fe36815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:38.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3680b3b20 0x7fe36815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 -- 172.21.15.157:0/4181343256 >> v1:172.21.15.157:6789/0 conn(0x7fe36815e5e0 legacy=0x7fe3680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36d281640 1 -- 172.21.15.157:0/4181343256 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30193775 0 0) 0x7fe36815f580 con 0x7fe36815e5e0 2023-11-06T03:07:38.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 -- 172.21.15.157:0/4181343256 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36806f5a0 con 0x7fe3681551e0 2023-11-06T03:07:38.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36e283640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe36806fda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe3640097c0 tx=0x7fe36401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9fdb04a74b4cb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:38.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36d281640 1 -- 172.21.15.157:0/4181343256 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640ac640 con 0x7fe3681551e0 2023-11-06T03:07:38.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36d281640 1 -- 172.21.15.157:0/4181343256 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3640ac7a0 con 0x7fe3681551e0 2023-11-06T03:07:38.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.417+0000 7fe36d281640 1 -- 172.21.15.157:0/4181343256 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640acb30 con 0x7fe3681551e0 2023-11-06T03:07:38.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/4181343256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 msgr2=0x7fe36806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe36806fda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe3640097c0 tx=0x7fe36401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/4181343256 shutdown_connections 2023-11-06T03:07:38.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3680b3b20 0x7fe36815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- 172.21.15.157:0/4181343256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe36806fda0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/4181343256 >> 172.21.15.157:0/4181343256 conn(0x7fe368065930 msgr2=0x7fe368065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:38.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/4181343256 shutdown_connections 2023-11-06T03:07:38.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/4181343256 wait complete. 2023-11-06T03:07:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 Processor -- start 2023-11-06T03:07:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- start start 2023-11-06T03:07:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 0x7fe36815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe36815e5e0 0x7fe36814b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815abf0 con 0x7fe3681551e0 2023-11-06T03:07:38.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368152390 con 0x7fe3680b3b20 2023-11-06T03:07:38.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36814d350 con 0x7fe36815e5e0 2023-11-06T03:07:38.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45488/0 (socket says 172.21.15.157:45488) 2023-11-06T03:07:38.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 -- 172.21.15.157:0/1817309663 learned_addr learned my addr 172.21.15.157:0/1817309663 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:38.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36e283640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 0x7fe36815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36ea84640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe36815e5e0 0x7fe36814b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 -- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe36815e5e0 msgr2=0x7fe36814b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe36815e5e0 0x7fe36814b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 -- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 msgr2=0x7fe36815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 0x7fe36815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 -- 172.21.15.157:0/1817309663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364009420 con 0x7fe3681551e0 2023-11-06T03:07:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36e283640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 0x7fe36815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:38.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe36da82640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c480 tx=0x7fe35800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580074e0 con 0x7fe3681551e0 2023-11-06T03:07:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe358007660 con 0x7fe3681551e0 2023-11-06T03:07:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3681f76c0 con 0x7fe3681551e0 2023-11-06T03:07:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004a30 con 0x7fe3681551e0 2023-11-06T03:07:38.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.421+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3681f7c80 con 0x7fe3681551e0 2023-11-06T03:07:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368072750 con 0x7fe3681551e0 2023-11-06T03:07:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe358004b90 con 0x7fe3681551e0 2023-11-06T03:07:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe35effd640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 0x7fe34c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fe3580825b0 con 0x7fe3681551e0 2023-11-06T03:07:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe36e283640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 0x7fe34c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.425+0000 7fe36e283640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 0x7fe34c067f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ae000 tx=0x7fe364008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.429+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe358053b20 con 0x7fe3681551e0 2023-11-06T03:07:38.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:38.585+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe368071730 con 0x7fe3681551e0 2023-11-06T03:07:39.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.273+0000 7fe35effd640 1 -- 172.21.15.157:0/1817309663 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe3580536b0 con 0x7fe3681551e0 2023-11-06T03:07:39.272 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:07:39.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 msgr2=0x7fe34c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 0x7fe34c067f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe3640ae000 tx=0x7fe364008040 comp rx=0 tx=0).stop 2023-11-06T03:07:39.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 msgr2=0x7fe3681555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe35800c480 tx=0x7fe35800c940 comp rx=0 tx=0).stop 2023-11-06T03:07:39.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 shutdown_connections 2023-11-06T03:07:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe34c065a90 0x7fe34c067f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe36815e5e0 0x7fe36814b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3681551e0 0x7fe3681555c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.277+0000 7fe3704e7640 1 --2- 172.21.15.157:0/1817309663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3680b3b20 0x7fe36815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.281+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 >> 172.21.15.157:0/1817309663 conn(0x7fe368065930 msgr2=0x7fe36814dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.281+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 shutdown_connections 2023-11-06T03:07:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.281+0000 7fe3704e7640 1 -- 172.21.15.157:0/1817309663 wait complete. 2023-11-06T03:07:39.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:07:39.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.437+0000 7f0d40796640 1 Processor -- start 2023-11-06T03:07:39.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.437+0000 7f0d40796640 1 -- start start 2023-11-06T03:07:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d40796640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d380b2af0 0x7f0d380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d40796640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 0x7f0d38159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d40796640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3815e440 con 0x7f0d381599c0 2023-11-06T03:07:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d40796640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3806c240 con 0x7f0d380b2af0 2023-11-06T03:07:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d40796640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3806c520 con 0x7f0d3806d000 2023-11-06T03:07:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d380b2af0 0x7f0d380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3e532640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d381599c0 0x7f0d380b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39718/0 (socket says 172.21.15.157:39718) 2023-11-06T03:07:39.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d380b2af0 0x7f0d380ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50406/0 (socket says 172.21.15.157:50406) 2023-11-06T03:07:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3e532640 1 -- 172.21.15.157:0/302911232 learned_addr learned my addr 172.21.15.157:0/302911232 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3ed33640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 0x7f0d38159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 -- 172.21.15.157:0/302911232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 msgr2=0x7f0d38159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 0x7f0d38159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 -- 172.21.15.157:0/302911232 >> v1:172.21.15.157:6789/0 conn(0x7f0d381599c0 legacy=0x7f0d380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 -- 172.21.15.157:0/302911232 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3806c800 con 0x7f0d380b2af0 2023-11-06T03:07:39.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3ed33640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 0x7f0d38159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:39.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3dd31640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d380b2af0 0x7f0d380ae000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097c0 tx=0x7f0d2801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc9c5e2e8970cd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3d530640 1 -- 172.21.15.157:0/302911232 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280ab670 con 0x7f0d380b2af0 2023-11-06T03:07:39.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3d530640 1 -- 172.21.15.157:0/302911232 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d280ab7d0 con 0x7f0d380b2af0 2023-11-06T03:07:39.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.441+0000 7f0d3d530640 1 -- 172.21.15.157:0/302911232 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280abb60 con 0x7f0d380b2af0 2023-11-06T03:07:39.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/302911232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d380b2af0 msgr2=0x7f0d380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d380b2af0 0x7f0d380ae000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097c0 tx=0x7f0d2801c450 comp rx=0 tx=0).stop 2023-11-06T03:07:39.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/302911232 shutdown_connections 2023-11-06T03:07:39.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d3806d000 0x7f0d38159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- 172.21.15.157:0/302911232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d380b2af0 0x7f0d380ae000 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/302911232 >> 172.21.15.157:0/302911232 conn(0x7f0d38065930 msgr2=0x7f0d38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:39.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/302911232 shutdown_connections 2023-11-06T03:07:39.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/302911232 wait complete. 2023-11-06T03:07:39.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 Processor -- start 2023-11-06T03:07:39.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- start start 2023-11-06T03:07:39.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d380b2af0 0x7f0d38142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d381599c0 0x7f0d38152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380aa990 con 0x7f0d3806d000 2023-11-06T03:07:39.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38152f70 con 0x7f0d381599c0 2023-11-06T03:07:39.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38153250 con 0x7f0d380b2af0 2023-11-06T03:07:39.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50408/0 (socket says 172.21.15.157:50408) 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 -- 172.21.15.157:0/3260358058 learned_addr learned my addr 172.21.15.157:0/3260358058 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 -- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d380b2af0 msgr2=0x7f0d38142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3dd31640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d380b2af0 0x7f0d38142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d380b2af0 0x7f0d38142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3ed33640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d381599c0 0x7f0d38152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 -- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d381599c0 msgr2=0x7f0d38152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d381599c0 0x7f0d38152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 -- 172.21.15.157:0/3260358058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009420 con 0x7f0d3806d000 2023-11-06T03:07:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d3e532640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c500 tx=0x7f0d3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34005cf0 con 0x7f0d3806d000 2023-11-06T03:07:39.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34005e50 con 0x7f0d3806d000 2023-11-06T03:07:39.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d38153530 con 0x7f0d3806d000 2023-11-06T03:07:39.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340072f0 con 0x7f0d3806d000 2023-11-06T03:07:39.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.445+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d38153af0 con 0x7f0d3806d000 2023-11-06T03:07:39.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d38072750 con 0x7f0d3806d000 2023-11-06T03:07:39.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0d34007450 con 0x7f0d3806d000 2023-11-06T03:07:39.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d2effd640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 0x7f0d1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0d340826e0 con 0x7f0d3806d000 2023-11-06T03:07:39.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d3dd31640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 0x7f0d1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.449+0000 7f0d3dd31640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 0x7f0d1c067f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d280af000 tx=0x7f0d28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.453+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d340500e0 con 0x7f0d3806d000 2023-11-06T03:07:39.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.641+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0d380cb110 con 0x7f0d3806d000 2023-11-06T03:07:39.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.653+0000 7f0d2effd640 1 -- 172.21.15.157:0/3260358058 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v32) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0d34053700 con 0x7f0d3806d000 2023-11-06T03:07:39.658 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 7 and data pool 8 2023-11-06T03:07:39.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 msgr2=0x7f0d1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 0x7f0d1c067f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d280af000 tx=0x7f0d28008040 comp rx=0 tx=0).stop 2023-11-06T03:07:39.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 msgr2=0x7f0d380aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c500 tx=0x7f0d3400c9c0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 shutdown_connections 2023-11-06T03:07:39.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d1c065a60 0x7f0d1c067f20 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d280af000 tx=0x7f0d28008040 comp rx=0 tx=0).stop 2023-11-06T03:07:39.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d381599c0 0x7f0d38152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d380b2af0 0x7f0d38142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 --2- 172.21.15.157:0/3260358058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d3806d000 0x7f0d380aa450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.661+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 >> 172.21.15.157:0/3260358058 conn(0x7f0d38065930 msgr2=0x7f0d38157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:39.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.673+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 shutdown_connections 2023-11-06T03:07:39.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.673+0000 7f0d40796640 1 -- 172.21.15.157:0/3260358058 wait complete. 2023-11-06T03:07:39.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.813+0000 7f61e11fb640 1 Processor -- start 2023-11-06T03:07:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.813+0000 7f61e11fb640 1 -- start start 2023-11-06T03:07:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc146370 0x7f61dc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61dc0ce800 0x7f61dc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0cf1a0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc06c3d0 con 0x7f61dc0ce800 2023-11-06T03:07:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc06f5a0 con 0x7f61dc146370 2023-11-06T03:07:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61dc0ce800 0x7f61dc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61dc0ce800 0x7f61dc0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50424/0 (socket says 172.21.15.157:50424) 2023-11-06T03:07:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 -- 172.21.15.157:0/3652080507 learned_addr learned my addr 172.21.15.157:0/3652080507 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61db59e640 1 --1- 172.21.15.157:0/3652080507 >> v1:172.21.15.157:6789/0 conn(0x7f61dc06b6a0 0x7f61dc06bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39720/0 (socket says 172.21.15.157:39720) 2023-11-06T03:07:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 -- 172.21.15.157:0/3652080507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc146370 msgr2=0x7f61dc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 --2- 172.21.15.157:0/3652080507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc146370 0x7f61dc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 -- 172.21.15.157:0/3652080507 >> v1:172.21.15.157:6789/0 conn(0x7f61dc06b6a0 legacy=0x7f61dc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 -- 172.21.15.157:0/3652080507 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc06f880 con 0x7f61dc0ce800 2023-11-06T03:07:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61d259c640 1 --2- 172.21.15.157:0/3652080507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc0ce800 0x7f61dc0cec60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009870 tx=0x7f61c401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88bd2cce54ca0bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61da59c640 1 -- 172.21.15.157:0/3652080507 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40aa610 con 0x7f61dc0ce800 2023-11-06T03:07:39.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61da59c640 1 -- 172.21.15.157:0/3652080507 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40a6040 con 0x7f61dc0ce800 2023-11-06T03:07:39.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61da59c640 1 -- 172.21.15.157:0/3652080507 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40aa990 con 0x7f61dc0ce800 2023-11-06T03:07:39.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- 172.21.15.157:0/3652080507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc0ce800 msgr2=0x7f61dc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 --2- 172.21.15.157:0/3652080507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc0ce800 0x7f61dc0cec60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009870 tx=0x7f61c401c4b0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- 172.21.15.157:0/3652080507 shutdown_connections 2023-11-06T03:07:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 --2- 172.21.15.157:0/3652080507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc0ce800 0x7f61dc0cec60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 --2- 172.21.15.157:0/3652080507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc146370 0x7f61dc0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- 172.21.15.157:0/3652080507 >> 172.21.15.157:0/3652080507 conn(0x7f61dc065930 msgr2=0x7f61dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- 172.21.15.157:0/3652080507 shutdown_connections 2023-11-06T03:07:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.817+0000 7f61e11fb640 1 -- 172.21.15.157:0/3652080507 wait complete. 2023-11-06T03:07:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 Processor -- start 2023-11-06T03:07:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- start start 2023-11-06T03:07:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 0x7f61dc160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61dc146370 0x7f61dc141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1415c0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc13a810 con 0x7f61dc146370 2023-11-06T03:07:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1409b0 con 0x7f61dc0ce800 2023-11-06T03:07:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50430/0 (socket says 172.21.15.157:50430) 2023-11-06T03:07:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 -- 172.21.15.157:0/1423190364 learned_addr learned my addr 172.21.15.157:0/1423190364 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61d259c640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 0x7f61dc160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61db59e640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61dc146370 0x7f61dc141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 -- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 msgr2=0x7f61dc160570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 0x7f61dc160570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 -- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61dc146370 msgr2=0x7f61dc141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61dc146370 0x7f61dc141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 -- 172.21.15.157:0/1423190364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40094d0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61d259c640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 0x7f61dc160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:39.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61dad9d640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b550 tx=0x7f61cc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc010610 con 0x7f61dc06b6a0 2023-11-06T03:07:39.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc010790 con 0x7f61dc06b6a0 2023-11-06T03:07:39.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc010b40 con 0x7f61dc06b6a0 2023-11-06T03:07:39.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc140c90 con 0x7f61dc06b6a0 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.821+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc141250 con 0x7f61dc06b6a0 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc072750 con 0x7f61dc06b6a0 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61cc010ca0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61d3fff640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 0x7f61bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61cc082f60 con 0x7f61dc06b6a0 2023-11-06T03:07:39.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61d259c640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 0x7f61bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:39.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.825+0000 7f61d259c640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 0x7f61bc067f20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ae000 tx=0x7f61c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:39.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.829+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f61cc0507c0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.985+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61dc0ae8b0 con 0x7f61dc06b6a0 2023-11-06T03:07:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.985+0000 7f61d3fff640 1 -- 172.21.15.157:0/1423190364 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17944 (secure 0 0 0) 0x7f61cc014090 con 0x7f61dc06b6a0 2023-11-06T03:07:39.987 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:39.987 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":52,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:39.644007+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:39.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 msgr2=0x7f61bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 0x7f61bc067f20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ae000 tx=0x7f61c4008040 comp rx=0 tx=0).stop 2023-11-06T03:07:39.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 msgr2=0x7f61dc13a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00b550 tx=0x7f61cc00ba10 comp rx=0 tx=0).stop 2023-11-06T03:07:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 shutdown_connections 2023-11-06T03:07:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f61bc065a60 0x7f61bc067f20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61dc146370 0x7f61dc141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61dc0ce800 0x7f61dc160570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 --2- 172.21.15.157:0/1423190364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61dc06b6a0 0x7f61dc13a2d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 >> 172.21.15.157:0/1423190364 conn(0x7f61dc065930 msgr2=0x7f61dc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 shutdown_connections 2023-11-06T03:07:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:39.989+0000 7f61e11fb640 1 -- 172.21.15.157:0/1423190364 wait complete. 2023-11-06T03:07:40.007 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:40.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.129+0000 7f70cff52640 1 Processor -- start 2023-11-06T03:07:40.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.129+0000 7f70cff52640 1 -- start start 2023-11-06T03:07:40.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cff52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cff52640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70c80d02f0 0x7f70c80cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cff52640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c80cef50 con 0x7f70c8071940 2023-11-06T03:07:40.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cff52640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c80cf230 con 0x7f70c80d02f0 2023-11-06T03:07:40.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cff52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80cf510 con 0x7f70c815aca0 2023-11-06T03:07:40.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70ce4ef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70c80d02f0 0x7f70c80cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:59714/0 (socket says 172.21.15.157:59714) 2023-11-06T03:07:40.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 -- 172.21.15.157:0/281947327 learned_addr learned my addr 172.21.15.157:0/281947327 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:40.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cd4ed640 1 --1- 172.21.15.157:0/281947327 >> v1:172.21.15.195:6789/0 conn(0x7f70c8071940 0x7f70c80cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35648/0 (socket says 172.21.15.157:35648) 2023-11-06T03:07:40.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 -- 172.21.15.157:0/281947327 >> v2:172.21.15.195:3300/0 conn(0x7f70c80d02f0 msgr2=0x7f70c80cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:40.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 --2- 172.21.15.157:0/281947327 >> v2:172.21.15.195:3300/0 conn(0x7f70c80d02f0 0x7f70c80cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 -- 172.21.15.157:0/281947327 >> v1:172.21.15.195:6789/0 conn(0x7f70c8071940 legacy=0x7f70c80cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cdcee640 1 -- 172.21.15.157:0/281947327 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80cf7f0 con 0x7f70c815aca0 2023-11-06T03:07:40.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.133+0000 7f70cccec640 1 -- 172.21.15.157:0/281947327 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610788275 0 0) 0x7f70c80cef50 con 0x7f70c8071940 2023-11-06T03:07:40.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cdcee640 1 --2- 172.21.15.157:0/281947327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009870 tx=0x7f70c401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f598536470b8569b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cccec640 1 -- 172.21.15.157:0/281947327 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40a9620 con 0x7f70c815aca0 2023-11-06T03:07:40.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cccec640 1 -- 172.21.15.157:0/281947327 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c40a9780 con 0x7f70c815aca0 2023-11-06T03:07:40.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cccec640 1 -- 172.21.15.157:0/281947327 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40a9b10 con 0x7f70c815aca0 2023-11-06T03:07:40.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- 172.21.15.157:0/281947327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 msgr2=0x7f70c80ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- 172.21.15.157:0/281947327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009870 tx=0x7f70c401c540 comp rx=0 tx=0).stop 2023-11-06T03:07:40.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- 172.21.15.157:0/281947327 shutdown_connections 2023-11-06T03:07:40.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- 172.21.15.157:0/281947327 >> v2:172.21.15.195:3300/0 conn(0x7f70c80d02f0 0x7f70c80cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- 172.21.15.157:0/281947327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c815aca0 0x7f70c80ce5e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- 172.21.15.157:0/281947327 >> 172.21.15.157:0/281947327 conn(0x7f70c806d560 msgr2=0x7f70c806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:40.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- 172.21.15.157:0/281947327 shutdown_connections 2023-11-06T03:07:40.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- 172.21.15.157:0/281947327 wait complete. 2023-11-06T03:07:40.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 Processor -- start 2023-11-06T03:07:40.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- start start 2023-11-06T03:07:40.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c8071940 0x7f70c81f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c80d02f0 0x7f70c80c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80c9a70 con 0x7f70c815aca0 2023-11-06T03:07:40.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80c9c60 con 0x7f70c80d02f0 2023-11-06T03:07:40.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70cff52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80c9f40 con 0x7f70c8071940 2023-11-06T03:07:40.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70ce4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70ce4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50432/0 (socket says 172.21.15.157:50432) 2023-11-06T03:07:40.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.137+0000 7f70ce4ef640 1 -- 172.21.15.157:0/175743161 learned_addr learned my addr 172.21.15.157:0/175743161 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:40.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cd4ed640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c80d02f0 0x7f70c80c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cdcee640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c8071940 0x7f70c81f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 -- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c8071940 msgr2=0x7f70c81f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c8071940 0x7f70c81f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 -- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c80d02f0 msgr2=0x7f70c80c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c80d02f0 0x7f70c80c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 -- 172.21.15.157:0/175743161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40094d0 con 0x7f70c815aca0 2023-11-06T03:07:40.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70ce4ef640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f70c000efc0 tx=0x7f70c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0009ca0 con 0x7f70c815aca0 2023-11-06T03:07:40.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c000f040 con 0x7f70c815aca0 2023-11-06T03:07:40.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c80ca220 con 0x7f70c815aca0 2023-11-06T03:07:40.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00146c0 con 0x7f70c815aca0 2023-11-06T03:07:40.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c80c8020 con 0x7f70c815aca0 2023-11-06T03:07:40.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c8072750 con 0x7f70c815aca0 2023-11-06T03:07:40.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f70c0014820 con 0x7f70c815aca0 2023-11-06T03:07:40.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 0x7f70ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70c0084260 con 0x7f70c815aca0 2023-11-06T03:07:40.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cdcee640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 0x7f70ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.141+0000 7f70cdcee640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 0x7f70ac067f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70c40ae000 tx=0x7f70c40b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.145+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70c0051aa0 con 0x7f70c815aca0 2023-11-06T03:07:40.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.301+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c81546c0 con 0x7f70c815aca0 2023-11-06T03:07:40.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.301+0000 7f70be7fc640 1 -- 172.21.15.157:0/175743161 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17944 (secure 0 0 0) 0x7f70c00550c0 con 0x7f70c815aca0 2023-11-06T03:07:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:40.303 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":52,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:39.644007+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:40.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 msgr2=0x7f70ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 0x7f70ac067f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70c40ae000 tx=0x7f70c40b3040 comp rx=0 tx=0).stop 2023-11-06T03:07:40.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 msgr2=0x7f70c80c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f70c000efc0 tx=0x7f70c000c490 comp rx=0 tx=0).stop 2023-11-06T03:07:40.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 shutdown_connections 2023-11-06T03:07:40.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70ac065a60 0x7f70ac067f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c815aca0 0x7f70c80c7ae0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c80d02f0 0x7f70c80c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 --2- 172.21.15.157:0/175743161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c8071940 0x7f70c81f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 >> 172.21.15.157:0/175743161 conn(0x7f70c806d560 msgr2=0x7f70c815cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:40.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 shutdown_connections 2023-11-06T03:07:40.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.305+0000 7f70cff52640 1 -- 172.21.15.157:0/175743161 wait complete. 2023-11-06T03:07:40.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:07:40.321 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:07:40.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:07:40.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 Processor -- start 2023-11-06T03:07:40.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 -- start start 2023-11-06T03:07:40.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2480b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24815f580 con 0x7fc24815ea60 2023-11-06T03:07:40.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24815efc0 con 0x7fc2481551e0 2023-11-06T03:07:40.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24f28c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480704b0 con 0x7fc24815e5e0 2023-11-06T03:07:40.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d829640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc24815ea60 0x7fc24815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39732/0 (socket says 172.21.15.157:39732) 2023-11-06T03:07:40.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d829640 1 -- 172.21.15.157:0/1709315525 learned_addr learned my addr 172.21.15.157:0/1709315525 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:40.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24c827640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2480b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076353448 0 0) 0x7fc24815f580 con 0x7fc24815ea60 2023-11-06T03:07:40.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc234003530 con 0x7fc24815ea60 2023-11-06T03:07:40.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 291145263 0 0) 0x7fc234003530 con 0x7fc24815ea60 2023-11-06T03:07:40.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 msgr2=0x7fc2480b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:40.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2480b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 >> v1:172.21.15.195:6789/0 conn(0x7fc2481551e0 legacy=0x7fc24806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24815f2a0 con 0x7fc24815ea60 2023-11-06T03:07:40.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc240001a00 con 0x7fc24815ea60 2023-11-06T03:07:40.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 >> v1:172.21.15.157:6789/0 conn(0x7fc24815ea60 legacy=0x7fc24815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340039b0 0x7fc234003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc234004d10 0x7fc234005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480704b0 con 0x7fc2340039b0 2023-11-06T03:07:40.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400baf0 con 0x7fc234004d10 2023-11-06T03:07:40.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400bcb0 con 0x7fc24815e5e0 2023-11-06T03:07:40.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340039b0 0x7fc234003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24c827640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 -- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 msgr2=0x7fc2340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:40.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc24815e5e0 0x7fc2340047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 -- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc234004d10 msgr2=0x7fc234005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:40.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc234004d10 0x7fc234005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.485+0000 7fc24d028640 1 -- 172.21.15.157:0/1709315525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234003530 con 0x7fc2340039b0 2023-11-06T03:07:40.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24d028640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340039b0 0x7fc234003dd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc244001780 tx=0x7fc24401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107cc31d9be8d197 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440052d0 con 0x7fc2340039b0 2023-11-06T03:07:40.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2440a7050 con 0x7fc2340039b0 2023-11-06T03:07:40.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc23ffff640 1 -- 172.21.15.157:0/1709315525 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244003600 con 0x7fc2340039b0 2023-11-06T03:07:40.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- 172.21.15.157:0/1709315525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340039b0 msgr2=0x7fc234003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 --2- 172.21.15.157:0/1709315525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2340039b0 0x7fc234003dd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc244001780 tx=0x7fc24401c310 comp rx=0 tx=0).stop 2023-11-06T03:07:40.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24d829640 1 -- 172.21.15.157:0/1709315525 reap_dead start 2023-11-06T03:07:40.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- 172.21.15.157:0/1709315525 shutdown_connections 2023-11-06T03:07:40.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- 172.21.15.157:0/1709315525 >> 172.21.15.157:0/1709315525 conn(0x7fc248065930 msgr2=0x7fc248065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:40.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- 172.21.15.157:0/1709315525 shutdown_connections 2023-11-06T03:07:40.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- 172.21.15.157:0/1709315525 wait complete. 2023-11-06T03:07:40.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 Processor -- start 2023-11-06T03:07:40.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.489+0000 7fc24f28c640 1 -- start start 2023-11-06T03:07:40.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2340039b0 0x7fc248160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 0x7fc2480cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480ce470 con 0x7fc24814a3a0 2023-11-06T03:07:40.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248151f60 con 0x7fc2480cdab0 2023-11-06T03:07:40.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc248160ee0 con 0x7fc2340039b0 2023-11-06T03:07:40.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50450/0 (socket says 172.21.15.157:50450) 2023-11-06T03:07:40.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 -- 172.21.15.157:0/954651306 learned_addr learned my addr 172.21.15.157:0/954651306 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:40.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24d829640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 0x7fc2480cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24d028640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2340039b0 0x7fc248160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 -- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2340039b0 msgr2=0x7fc248160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2340039b0 0x7fc248160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 -- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 msgr2=0x7fc2480cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:40.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 0x7fc2480cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:40.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 -- 172.21.15.157:0/954651306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc248160650 con 0x7fc24814a3a0 2023-11-06T03:07:40.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24d829640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 0x7fc2480cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:40.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24c827640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc238010890 tx=0x7fc238010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238009630 con 0x7fc24814a3a0 2023-11-06T03:07:40.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800eb60 con 0x7fc24814a3a0 2023-11-06T03:07:40.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc248160920 con 0x7fc24814a3a0 2023-11-06T03:07:40.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23801c400 con 0x7fc24814a3a0 2023-11-06T03:07:40.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc248151bf0 con 0x7fc24814a3a0 2023-11-06T03:07:40.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc248072750 con 0x7fc24814a3a0 2023-11-06T03:07:40.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc23801c560 con 0x7fc24814a3a0 2023-11-06T03:07:40.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:40.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2380837c0 con 0x7fc24814a3a0 2023-11-06T03:07:40.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24d028640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:40.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.493+0000 7fc24d028640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 0x7fc22c067f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc244001750 tx=0x7fc2440011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:40.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.497+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc238051160 con 0x7fc24814a3a0 2023-11-06T03:07:40.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:40.689+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc248071730 con 0x7fc24814a3a0 2023-11-06T03:07:41.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.669+0000 7fc23dffb640 1 -- 172.21.15.157:0/954651306 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc238054780 con 0x7fc24814a3a0 2023-11-06T03:07:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 msgr2=0x7fc22c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 0x7fc22c067f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc244001750 tx=0x7fc2440011a0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 msgr2=0x7fc24813c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc238010890 tx=0x7fc238010d50 comp rx=0 tx=0).stop 2023-11-06T03:07:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 shutdown_connections 2023-11-06T03:07:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc2480cdab0 0x7fc2480cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24814a3a0 0x7fc24813c2a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 --2- 172.21.15.157:0/954651306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2340039b0 0x7fc248160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 >> 172.21.15.157:0/954651306 conn(0x7fc248065930 msgr2=0x7fc2480afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 shutdown_connections 2023-11-06T03:07:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.673+0000 7fc24f28c640 1 -- 172.21.15.157:0/954651306 wait complete. 2023-11-06T03:07:41.693 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:07:41.693 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:07:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 Processor -- start 2023-11-06T03:07:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 -- start start 2023-11-06T03:07:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4006c490 con 0x7f7d400adf40 2023-11-06T03:07:41.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4006b950 con 0x7f7d40156a70 2023-11-06T03:07:41.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d46543640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4006bb60 con 0x7f7d400adae0 2023-11-06T03:07:41.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d37fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d37fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:37790/0 (socket says 172.21.15.157:37790) 2023-11-06T03:07:41.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:59760/0 (socket says 172.21.15.157:59760) 2023-11-06T03:07:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d44ae0640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f7d400adf40 0x7f7d4006c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35668/0 (socket says 172.21.15.157:35668) 2023-11-06T03:07:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d37fff640 1 -- 172.21.15.157:0/2074044581 learned_addr learned my addr 172.21.15.157:0/2074044581 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 -- 172.21.15.157:0/2074044581 >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 msgr2=0x7f7d4006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 --2- 172.21.15.157:0/2074044581 >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 -- 172.21.15.157:0/2074044581 >> v1:172.21.15.195:6789/0 conn(0x7f7d400adf40 legacy=0x7f7d4006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d3ffff640 1 -- 172.21.15.157:0/2074044581 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4006be40 con 0x7f7d400adae0 2023-11-06T03:07:41.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.821+0000 7f7d37fff640 1 --2- 172.21.15.157:0/2074044581 >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:07:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d3ffff640 1 --2- 172.21.15.157:0/2074044581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097c0 tx=0x7f7d2801e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe265bce98491ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d3f7fe640 1 -- 172.21.15.157:0/2074044581 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280a7970 con 0x7f7d400adae0 2023-11-06T03:07:41.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d3f7fe640 1 -- 172.21.15.157:0/2074044581 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d280a7ad0 con 0x7f7d400adae0 2023-11-06T03:07:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d3f7fe640 1 -- 172.21.15.157:0/2074044581 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280a7e60 con 0x7f7d400adae0 2023-11-06T03:07:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- 172.21.15.157:0/2074044581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 msgr2=0x7f7d400b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 --2- 172.21.15.157:0/2074044581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097c0 tx=0x7f7d2801e880 comp rx=0 tx=0).stop 2023-11-06T03:07:41.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- 172.21.15.157:0/2074044581 shutdown_connections 2023-11-06T03:07:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 --2- 172.21.15.157:0/2074044581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adae0 0x7f7d400b1c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 --2- 172.21.15.157:0/2074044581 >> v2:172.21.15.195:3300/0 conn(0x7f7d40156a70 0x7f7d4006f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- 172.21.15.157:0/2074044581 >> 172.21.15.157:0/2074044581 conn(0x7f7d40065930 msgr2=0x7f7d40065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- 172.21.15.157:0/2074044581 shutdown_connections 2023-11-06T03:07:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- 172.21.15.157:0/2074044581 wait complete. 2023-11-06T03:07:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 Processor -- start 2023-11-06T03:07:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.825+0000 7f7d46543640 1 -- start start 2023-11-06T03:07:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 0x7f7d4006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adf40 0x7f7d40162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4006fbd0 con 0x7f7d40156a70 2023-11-06T03:07:41.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4014a880 con 0x7f7d400adae0 2023-11-06T03:07:41.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40162a70 con 0x7f7d400adf40 2023-11-06T03:07:41.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50462/0 (socket says 172.21.15.157:50462) 2023-11-06T03:07:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 -- 172.21.15.157:0/681427637 learned_addr learned my addr 172.21.15.157:0/681427637 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d37fff640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 0x7f7d4006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d3ffff640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adf40 0x7f7d40162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 -- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adf40 msgr2=0x7f7d40162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adf40 0x7f7d40162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 -- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 msgr2=0x7f7d4006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 0x7f7d4006fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 -- 172.21.15.157:0/681427637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3000a670 con 0x7f7d40156a70 2023-11-06T03:07:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d37fff640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 0x7f7d4006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d44ae0640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c730 tx=0x7f7d3800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38004d70 con 0x7f7d40156a70 2023-11-06T03:07:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800a640 con 0x7f7d40156a70 2023-11-06T03:07:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800a9f0 con 0x7f7d40156a70 2023-11-06T03:07:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d28009420 con 0x7f7d40156a70 2023-11-06T03:07:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4006f510 con 0x7f7d40156a70 2023-11-06T03:07:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4006fd10 con 0x7f7d40156a70 2023-11-06T03:07:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.829+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d3800ab50 con 0x7f7d40156a70 2023-11-06T03:07:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.833+0000 7f7d3d7fa640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 0x7f7d20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.833+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d38082560 con 0x7f7d40156a70 2023-11-06T03:07:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.833+0000 7f7d37fff640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 0x7f7d20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.833+0000 7f7d37fff640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 0x7f7d20067f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000ae30 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:41.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:41.837+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d4006fd10 con 0x7f7d40156a70 2023-11-06T03:07:42.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.017+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7d4006fd10 con 0x7f7d40156a70 2023-11-06T03:07:42.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.673+0000 7f7d3d7fa640 1 -- 172.21.15.157:0/681427637 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d4006fd10 con 0x7f7d40156a70 2023-11-06T03:07:42.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 msgr2=0x7f7d20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 0x7f7d20067f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000ae30 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 msgr2=0x7f7d40149790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c730 tx=0x7f7d3800cb30 comp rx=0 tx=0).stop 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 shutdown_connections 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d20065a60 0x7f7d20067f20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d40156a70 0x7f7d40149790 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d400adf40 0x7f7d40162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 --2- 172.21.15.157:0/681427637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d400adae0 0x7f7d4006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 >> 172.21.15.157:0/681427637 conn(0x7f7d40065930 msgr2=0x7f7d40158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 shutdown_connections 2023-11-06T03:07:42.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.677+0000 7f7d46543640 1 -- 172.21.15.157:0/681427637 wait complete. 2023-11-06T03:07:42.697 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:07:42.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:42.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 Processor -- start 2023-11-06T03:07:42.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 -- start start 2023-11-06T03:07:42.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4cc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15f580 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15efc0 con 0x7fb4cc0b3b20 2023-11-06T03:07:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d3d41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15f2a0 con 0x7fb4cc1551e0 2023-11-06T03:07:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb4cc15e5e0 0x7fb4cc0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39738/0 (socket says 172.21.15.157:39738) 2023-11-06T03:07:42.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 -- 172.21.15.157:0/1497434095 learned_addr learned my addr 172.21.15.157:0/1497434095 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d1add640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4cc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210669567 0 0) 0x7fb4cc15f580 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b8003530 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3219265461 0 0) 0x7fb4b8003530 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 msgr2=0x7fb4cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:42.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4cc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 >> v1:172.21.15.195:6789/0 conn(0x7fb4cc0b3b20 legacy=0x7fb4cc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc06f5a0 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb4c0001680 con 0x7fb4cc15e5e0 2023-11-06T03:07:42.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 >> v1:172.21.15.157:6789/0 conn(0x7fb4cc15e5e0 legacy=0x7fb4cc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b80039b0 0x7fb4b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15f2a0 con 0x7fb4b80047d0 2023-11-06T03:07:42.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b800bac0 con 0x7fb4b80039b0 2023-11-06T03:07:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b800bc80 con 0x7fb4cc1551e0 2023-11-06T03:07:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d1add640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b80039b0 0x7fb4b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d22de640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 -- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 msgr2=0x7fb4b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc1551e0 0x7fb4b80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 -- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b80039b0 msgr2=0x7fb4b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4b80039b0 0x7fb4b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.821+0000 7fb4d12dc640 1 -- 172.21.15.157:0/1497434095 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8003530 con 0x7fb4b80047d0 2023-11-06T03:07:42.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4b8004c50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c240 tx=0x7fb4c001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7298ffcb6b5e5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00a5b10 con 0x7fb4b80047d0 2023-11-06T03:07:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c00a5c70 con 0x7fb4b80047d0 2023-11-06T03:07:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d0adb640 1 -- 172.21.15.157:0/1497434095 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0005b20 con 0x7fb4b80047d0 2023-11-06T03:07:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- 172.21.15.157:0/1497434095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 msgr2=0x7fb4b8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/1497434095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4b8004c50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c240 tx=0x7fb4c001cc10 comp rx=0 tx=0).stop 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d22de640 1 -- 172.21.15.157:0/1497434095 reap_dead start 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- 172.21.15.157:0/1497434095 shutdown_connections 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- 172.21.15.157:0/1497434095 >> 172.21.15.157:0/1497434095 conn(0x7fb4cc065930 msgr2=0x7fb4cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- 172.21.15.157:0/1497434095 shutdown_connections 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- 172.21.15.157:0/1497434095 wait complete. 2023-11-06T03:07:42.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 Processor -- start 2023-11-06T03:07:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- start start 2023-11-06T03:07:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc0cd430 0x7fb4cc159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4cc150060 0x7fb4cc13bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cf030 con 0x7fb4b80047d0 2023-11-06T03:07:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cdb10 con 0x7fb4cc150060 2023-11-06T03:07:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d3d41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cd8b0 con 0x7fb4cc0cd430 2023-11-06T03:07:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50482/0 (socket says 172.21.15.157:50482) 2023-11-06T03:07:42.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 -- 172.21.15.157:0/3235494334 learned_addr learned my addr 172.21.15.157:0/3235494334 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:42.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc0cd430 0x7fb4cc159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d22de640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4cc150060 0x7fb4cc13bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 -- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc0cd430 msgr2=0x7fb4cc159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc0cd430 0x7fb4cc159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 -- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4cc150060 msgr2=0x7fb4cc13bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:42.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4cc150060 0x7fb4cc13bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:42.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.825+0000 7fb4d1add640 1 -- 172.21.15.157:0/3235494334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc14f2a0 con 0x7fb4b80047d0 2023-11-06T03:07:42.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d1add640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80115e0 tx=0x7fb4c8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:42.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800f8f0 con 0x7fb4b80047d0 2023-11-06T03:07:42.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800fa50 con 0x7fb4b80047d0 2023-11-06T03:07:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc14f580 con 0x7fb4b80047d0 2023-11-06T03:07:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800fdc0 con 0x7fb4b80047d0 2023-11-06T03:07:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc14fb40 con 0x7fb4b80047d0 2023-11-06T03:07:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc072750 con 0x7fb4b80047d0 2023-11-06T03:07:42.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4c8015460 con 0x7fb4b80047d0 2023-11-06T03:07:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4c8083740 con 0x7fb4b80047d0 2023-11-06T03:07:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:42.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.829+0000 7fb4d12dc640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 0x7fb4b0067f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00ab000 tx=0x7fb4c00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:42.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:42.833+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb4c8051030 con 0x7fb4b80047d0 2023-11-06T03:07:43.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.017+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc071730 con 0x7fb4b80047d0 2023-11-06T03:07:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4be7fc640 1 -- 172.21.15.157:0/3235494334 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fb4c8054650 con 0x7fb4b80047d0 2023-11-06T03:07:43.019 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:43.019 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:07:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:07:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 msgr2=0x7fb4b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 0x7fb4b0067f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00ab000 tx=0x7fb4c00a7070 comp rx=0 tx=0).stop 2023-11-06T03:07:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 msgr2=0x7fb4cc0cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80115e0 tx=0x7fb4c8011aa0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 shutdown_connections 2023-11-06T03:07:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4b0065a60 0x7fb4b0067f20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4cc150060 0x7fb4cc13bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4cc0cd430 0x7fb4cc159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 --2- 172.21.15.157:0/3235494334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4b80047d0 0x7fb4cc0cca20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 >> 172.21.15.157:0/3235494334 conn(0x7fb4cc065930 msgr2=0x7fb4cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 shutdown_connections 2023-11-06T03:07:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.021+0000 7fb4d3d41640 1 -- 172.21.15.157:0/3235494334 wait complete. 2023-11-06T03:07:43.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:07:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 Processor -- start 2023-11-06T03:07:43.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 -- start start 2023-11-06T03:07:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95c00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f95c015ea60 0x7f95c00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c015f580 con 0x7f95c01551e0 2023-11-06T03:07:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c00704b0 con 0x7f95c015ea60 2023-11-06T03:07:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c7b05640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c015efc0 con 0x7f95c015e5e0 2023-11-06T03:07:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c58a1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f95c01551e0 0x7f95c006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39744/0 (socket says 172.21.15.157:39744) 2023-11-06T03:07:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c58a1640 1 -- 172.21.15.157:0/1987645782 learned_addr learned my addr 172.21.15.157:0/1987645782 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c60a2640 1 --2- 172.21.15.157:0/1987645782 >> v2:172.21.15.195:3300/0 conn(0x7f95c015ea60 0x7f95c00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95c00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928008095 0 0) 0x7f95c015f580 con 0x7f95c01551e0 2023-11-06T03:07:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ac003530 con 0x7f95c01551e0 2023-11-06T03:07:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752487750 0 0) 0x7f95ac003530 con 0x7f95c01551e0 2023-11-06T03:07:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 msgr2=0x7f95c00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95c00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 >> v2:172.21.15.195:3300/0 conn(0x7f95c015ea60 msgr2=0x7f95c00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 --2- 172.21.15.157:0/1987645782 >> v2:172.21.15.195:3300/0 conn(0x7f95c015ea60 0x7f95c00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c015f2a0 con 0x7f95c01551e0 2023-11-06T03:07:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f95bc001680 con 0x7f95c01551e0 2023-11-06T03:07:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 >> v1:172.21.15.157:6789/0 conn(0x7f95c01551e0 legacy=0x7f95c006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95ac003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95ac004d10 0x7f95ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00704b0 con 0x7f95ac003f00 2023-11-06T03:07:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c015efc0 con 0x7f95ac004d10 2023-11-06T03:07:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac00b9e0 con 0x7f95c015e5e0 2023-11-06T03:07:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c60a2640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95ac004d10 0x7f95ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c58a1640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95ac003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 -- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 msgr2=0x7f95ac003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c015e5e0 0x7f95ac003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 -- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95ac004d10 msgr2=0x7f95ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95ac004d10 0x7f95ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c50a0640 1 -- 172.21.15.157:0/1987645782 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac003530 con 0x7f95ac003f00 2023-11-06T03:07:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.153+0000 7f95c60a2640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95ac004d10 0x7f95ac005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:43.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c50a0640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95ac0047d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b10 tx=0x7f95b001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9f1e331e737259 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00a6510 con 0x7f95ac003f00 2023-11-06T03:07:43.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b00a6670 con 0x7f95ac003f00 2023-11-06T03:07:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c489f640 1 -- 172.21.15.157:0/1987645782 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00a6a20 con 0x7f95ac003f00 2023-11-06T03:07:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- 172.21.15.157:0/1987645782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 msgr2=0x7f95ac0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 --2- 172.21.15.157:0/1987645782 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95ac0047d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009b10 tx=0x7f95b001ca20 comp rx=0 tx=0).stop 2023-11-06T03:07:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c60a2640 1 -- 172.21.15.157:0/1987645782 reap_dead start 2023-11-06T03:07:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- 172.21.15.157:0/1987645782 shutdown_connections 2023-11-06T03:07:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- 172.21.15.157:0/1987645782 >> 172.21.15.157:0/1987645782 conn(0x7f95c0065930 msgr2=0x7f95c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- 172.21.15.157:0/1987645782 shutdown_connections 2023-11-06T03:07:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- 172.21.15.157:0/1987645782 wait complete. 2023-11-06T03:07:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 Processor -- start 2023-11-06T03:07:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.157+0000 7f95c7b05640 1 -- start start 2023-11-06T03:07:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 0x7f95c0159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 0x7f95c01605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c013c7e0 con 0x7f95ac003f00 2023-11-06T03:07:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c015a370 con 0x7f95c0160140 2023-11-06T03:07:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0147d20 con 0x7f95c01599d0 2023-11-06T03:07:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50514/0 (socket says 172.21.15.157:50514) 2023-11-06T03:07:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 -- 172.21.15.157:0/394340162 learned_addr learned my addr 172.21.15.157:0/394340162 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:43.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c50a0640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 0x7f95c0159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c60a2640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 0x7f95c01605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 -- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 msgr2=0x7f95c0159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 0x7f95c0159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 -- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 msgr2=0x7f95c01605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 0x7f95c01605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 -- 172.21.15.157:0/394340162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0160e30 con 0x7f95ac003f00 2023-11-06T03:07:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c50a0640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 0x7f95c0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c60a2640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 0x7f95c01605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c58a1640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d720 tx=0x7f95bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0074e0 con 0x7f95ac003f00 2023-11-06T03:07:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc007640 con 0x7f95ac003f00 2023-11-06T03:07:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc004a30 con 0x7f95ac003f00 2023-11-06T03:07:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c0161110 con 0x7f95ac003f00 2023-11-06T03:07:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c01616d0 con 0x7f95ac003f00 2023-11-06T03:07:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c0147ee0 con 0x7f95ac003f00 2023-11-06T03:07:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f95bc004b90 con 0x7f95ac003f00 2023-11-06T03:07:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 0x7f95a4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95bc082f80 con 0x7f95ac003f00 2023-11-06T03:07:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.161+0000 7f95c50a0640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 0x7f95a4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.165+0000 7f95c50a0640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 0x7f95a4067f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95b00af000 tx=0x7f95b00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.165+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f95bc0b4070 con 0x7f95ac003f00 2023-11-06T03:07:43.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.309+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c0072750 con 0x7f95ac003f00 2023-11-06T03:07:43.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95b67fc640 1 -- 172.21.15.157:0/394340162 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17931 (secure 0 0 0) 0x7f95bc019090 con 0x7f95ac003f00 2023-11-06T03:07:43.312 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:43.313 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":53,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:40.648864+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:07:43.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 msgr2=0x7f95a4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 0x7f95a4067f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95b00af000 tx=0x7f95b00046d0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 msgr2=0x7f95c013c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d720 tx=0x7f95bc00dbe0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 shutdown_connections 2023-11-06T03:07:43.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f95a4065a90 0x7f95a4067f50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95c0160140 0x7f95c01605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95c01599d0 0x7f95c0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 --2- 172.21.15.157:0/394340162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95ac003f00 0x7f95c013c2a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.313+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 >> 172.21.15.157:0/394340162 conn(0x7f95c0065930 msgr2=0x7f95c00aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.317+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 shutdown_connections 2023-11-06T03:07:43.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.317+0000 7f95c7b05640 1 -- 172.21.15.157:0/394340162 wait complete. 2023-11-06T03:07:43.331 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:07:43.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:07:43.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 Processor -- start 2023-11-06T03:07:43.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 -- start start 2023-11-06T03:07:43.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa91815e5e0 0x7fa9180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91815f580 con 0x7fa9181551e0 2023-11-06T03:07:43.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91815efc0 con 0x7fa91815ea90 2023-11-06T03:07:43.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa91e63c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180704b0 con 0x7fa91815e5e0 2023-11-06T03:07:43.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa917fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa9181551e0 0x7fa91806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39750/0 (socket says 172.21.15.157:39750) 2023-11-06T03:07:43.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa917fff640 1 -- 172.21.15.157:0/1437343326 learned_addr learned my addr 172.21.15.157:0/1437343326 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:43.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.441+0000 7fa9177fe640 1 --2- 172.21.15.157:0/1437343326 >> v2:172.21.15.157:3300/0 conn(0x7fa91815e5e0 0x7fa9180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa916ffd640 1 -- 172.21.15.157:0/1437343326 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197781678 0 0) 0x7fa91815f580 con 0x7fa9181551e0 2023-11-06T03:07:43.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa916ffd640 1 -- 172.21.15.157:0/1437343326 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa900003530 con 0x7fa9181551e0 2023-11-06T03:07:43.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa9177fe640 1 -- 172.21.15.157:0/1437343326 >> v1:172.21.15.195:6789/0 conn(0x7fa91815ea90 legacy=0x7fa91815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa9177fe640 1 -- 172.21.15.157:0/1437343326 >> v1:172.21.15.157:6789/0 conn(0x7fa9181551e0 legacy=0x7fa91806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa9177fe640 1 -- 172.21.15.157:0/1437343326 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91815f2a0 con 0x7fa91815e5e0 2023-11-06T03:07:43.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa9177fe640 1 --2- 172.21.15.157:0/1437343326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815e5e0 0x7fa9180b3410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097c0 tx=0x7fa90c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743e5c970369c821 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa916ffd640 1 -- 172.21.15.157:0/1437343326 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0ab600 con 0x7fa91815e5e0 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa916ffd640 1 -- 172.21.15.157:0/1437343326 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c0a7050 con 0x7fa91815e5e0 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa916ffd640 1 -- 172.21.15.157:0/1437343326 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0ab980 con 0x7fa91815e5e0 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 -- 172.21.15.157:0/1437343326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815e5e0 msgr2=0x7fa9180b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 --2- 172.21.15.157:0/1437343326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815e5e0 0x7fa9180b3410 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097c0 tx=0x7fa90c01c380 comp rx=0 tx=0).stop 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 -- 172.21.15.157:0/1437343326 shutdown_connections 2023-11-06T03:07:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 --2- 172.21.15.157:0/1437343326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815e5e0 0x7fa9180b3410 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 -- 172.21.15.157:0/1437343326 >> 172.21.15.157:0/1437343326 conn(0x7fa918065930 msgr2=0x7fa918065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 -- 172.21.15.157:0/1437343326 shutdown_connections 2023-11-06T03:07:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 -- 172.21.15.157:0/1437343326 wait complete. 2023-11-06T03:07:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.445+0000 7fa91e63c640 1 Processor -- start 2023-11-06T03:07:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- start start 2023-11-06T03:07:43.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9181551e0 0x7fa918149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91815e5e0 0x7fa91806be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91806c3c0 con 0x7fa91815ea90 2023-11-06T03:07:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91814ea50 con 0x7fa91815e5e0 2023-11-06T03:07:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91814f7f0 con 0x7fa9181551e0 2023-11-06T03:07:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50536/0 (socket says 172.21.15.157:50536) 2023-11-06T03:07:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 -- 172.21.15.157:0/3035422500 learned_addr learned my addr 172.21.15.157:0/3035422500 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 -- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9181551e0 msgr2=0x7fa918149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9181551e0 0x7fa918149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa9177fe640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91815e5e0 0x7fa91806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 -- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91815e5e0 msgr2=0x7fa91806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91815e5e0 0x7fa91806be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 -- 172.21.15.157:0/3035422500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009420 con 0x7fa91815ea90 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91cbd9640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b970 tx=0x7fa91000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000cb40 con 0x7fa91815ea90 2023-11-06T03:07:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91000cca0 con 0x7fa91815ea90 2023-11-06T03:07:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91814ed10 con 0x7fa91815ea90 2023-11-06T03:07:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910014680 con 0x7fa91815ea90 2023-11-06T03:07:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91814f2d0 con 0x7fa91815ea90 2023-11-06T03:07:43.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91814f4c0 con 0x7fa91815ea90 2023-11-06T03:07:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa91001d3b0 con 0x7fa91815ea90 2023-11-06T03:07:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 0x7fa904067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa910083140 con 0x7fa91815ea90 2023-11-06T03:07:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.449+0000 7fa917fff640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 0x7fa904067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.453+0000 7fa917fff640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 0x7fa904067f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa908007fd0 tx=0x7fa908007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.453+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa910050980 con 0x7fa91815ea90 2023-11-06T03:07:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.601+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9180b3d60 con 0x7fa904065a60 2023-11-06T03:07:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.625+0000 7fa914ff9640 1 -- 172.21.15.157:0/3035422500 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299435 (secure 0 0 0) 0x7fa9080058c0 con 0x7fa904065a60 2023-11-06T03:07:43.628 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:07:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 msgr2=0x7fa904067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 0x7fa904067f20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa908007fd0 tx=0x7fa908007940 comp rx=0 tx=0).stop 2023-11-06T03:07:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 msgr2=0x7fa9180aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b970 tx=0x7fa91000be30 comp rx=0 tx=0).stop 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 shutdown_connections 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa904065a60 0x7fa904067f20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa91815ea90 0x7fa9180aebb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa91815e5e0 0x7fa91806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 --2- 172.21.15.157:0/3035422500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9181551e0 0x7fa918149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 >> 172.21.15.157:0/3035422500 conn(0x7fa918065930 msgr2=0x7fa9180afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 shutdown_connections 2023-11-06T03:07:43.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.629+0000 7fa91e63c640 1 -- 172.21.15.157:0/3035422500 wait complete. 2023-11-06T03:07:43.646 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-06T03:07:43.173699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346812,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":340992,"kb_avail":754627908,"statfs":{"total":773094113280,"available":772738977792,"internally_reserved":0,"allocated":5894144,"data_stored":3204583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":349175808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909522"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274775+0000","last_change":"2023-11-06T03:07:38.274775+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274775+0000","last_undegraded":"2023-11-06T03:07:38.274775+0000","last_fullsized":"2023-11-06T03:07:38.274775+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273492+0000","last_change":"2023-11-06T03:07:38.273492+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273492+0000","last_undegraded":"2023-11-06T03:07:38.273492+0000","last_fullsized":"2023-11-06T03:07:38.273492+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272757+0000","last_change":"2023-11-06T03:07:38.272757+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272757+0000","last_undegraded":"2023-11-06T03:07:38.272757+0000","last_fullsized":"2023-11-06T03:07:38.272757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.280184+0000","last_change":"2023-11-06T03:07:38.280184+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.280184+0000","last_undegraded":"2023-11-06T03:07:38.280184+0000","last_fullsized":"2023-11-06T03:07:38.280184+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273187+0000","last_change":"2023-11-06T03:07:38.273187+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273187+0000","last_undegraded":"2023-11-06T03:07:38.273187+0000","last_fullsized":"2023-11-06T03:07:38.273187+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268342+0000","last_change":"2023-11-06T03:07:38.268342+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268342+0000","last_undegraded":"2023-11-06T03:07:38.268342+0000","last_fullsized":"2023-11-06T03:07:38.268342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274161+0000","last_change":"2023-11-06T03:07:38.274161+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274161+0000","last_undegraded":"2023-11-06T03:07:38.274161+0000","last_fullsized":"2023-11-06T03:07:38.274161+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271731+0000","last_change":"2023-11-06T03:07:38.271731+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271731+0000","last_undegraded":"2023-11-06T03:07:38.271731+0000","last_fullsized":"2023-11-06T03:07:38.271731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277966+0000","last_change":"2023-11-06T03:07:38.277966+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277966+0000","last_undegraded":"2023-11-06T03:07:38.277966+0000","last_fullsized":"2023-11-06T03:07:38.277966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268292+0000","last_change":"2023-11-06T03:07:38.268292+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268292+0000","last_undegraded":"2023-11-06T03:07:38.268292+0000","last_fullsized":"2023-11-06T03:07:38.268292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.279319+0000","last_change":"2023-11-06T03:07:38.279319+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.279319+0000","last_undegraded":"2023-11-06T03:07:38.279319+0000","last_fullsized":"2023-11-06T03:07:38.279319+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270185+0000","last_change":"2023-11-06T03:07:38.270185+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270185+0000","last_undegraded":"2023-11-06T03:07:38.270185+0000","last_fullsized":"2023-11-06T03:07:38.270185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272337+0000","last_change":"2023-11-06T03:07:38.272337+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272337+0000","last_undegraded":"2023-11-06T03:07:38.272337+0000","last_fullsized":"2023-11-06T03:07:38.272337+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273107+0000","last_change":"2023-11-06T03:07:38.273107+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273107+0000","last_undegraded":"2023-11-06T03:07:38.273107+0000","last_fullsized":"2023-11-06T03:07:38.273107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277799+0000","last_change":"2023-11-06T03:07:38.277799+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277799+0000","last_undegraded":"2023-11-06T03:07:38.277799+0000","last_fullsized":"2023-11-06T03:07:38.277799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270291+0000","last_change":"2023-11-06T03:07:38.270291+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270291+0000","last_undegraded":"2023-11-06T03:07:38.270291+0000","last_fullsized":"2023-11-06T03:07:38.270291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271213+0000","last_change":"2023-11-06T03:07:38.271213+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271213+0000","last_undegraded":"2023-11-06T03:07:38.271213+0000","last_fullsized":"2023-11-06T03:07:38.271213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274204+0000","last_change":"2023-11-06T03:07:38.274204+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274204+0000","last_undegraded":"2023-11-06T03:07:38.274204+0000","last_fullsized":"2023-11-06T03:07:38.274204+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273197+0000","last_change":"2023-11-06T03:07:38.273197+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273197+0000","last_undegraded":"2023-11-06T03:07:38.273197+0000","last_fullsized":"2023-11-06T03:07:38.273197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273072+0000","last_change":"2023-11-06T03:07:38.273072+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273072+0000","last_undegraded":"2023-11-06T03:07:38.273072+0000","last_fullsized":"2023-11-06T03:07:38.273072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273478+0000","last_change":"2023-11-06T03:07:38.273478+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273478+0000","last_undegraded":"2023-11-06T03:07:38.273478+0000","last_fullsized":"2023-11-06T03:07:38.273478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.269961+0000","last_change":"2023-11-06T03:07:38.269961+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.269961+0000","last_undegraded":"2023-11-06T03:07:38.269961+0000","last_fullsized":"2023-11-06T03:07:38.269961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270873+0000","last_change":"2023-11-06T03:07:38.270873+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270873+0000","last_undegraded":"2023-11-06T03:07:38.270873+0000","last_fullsized":"2023-11-06T03:07:38.270873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274121+0000","last_change":"2023-11-06T03:07:38.274121+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274121+0000","last_undegraded":"2023-11-06T03:07:38.274121+0000","last_fullsized":"2023-11-06T03:07:38.274121+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270776+0000","last_change":"2023-11-06T03:07:38.270776+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270776+0000","last_undegraded":"2023-11-06T03:07:38.270776+0000","last_fullsized":"2023-11-06T03:07:38.270776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454639+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:07:35.454639+0000","last_peered":"2023-11-06T03:07:35.454639+0000","last_clean":"2023-11-06T03:07:35.454639+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:07:35.454639+0000","last_undegraded":"2023-11-06T03:07:35.454639+0000","last_fullsized":"2023-11-06T03:07:35.454639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159097+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:07:36.159097+0000","last_peered":"2023-11-06T03:07:36.159097+0000","last_clean":"2023-11-06T03:07:36.159097+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:07:36.159097+0000","last_undegraded":"2023-11-06T03:07:36.159097+0000","last_fullsized":"2023-11-06T03:07:36.159097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277505+0000","last_change":"2023-11-06T03:07:38.277505+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277505+0000","last_undegraded":"2023-11-06T03:07:38.277505+0000","last_fullsized":"2023-11-06T03:07:38.277505+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454636+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:07:35.454636+0000","last_peered":"2023-11-06T03:07:35.454636+0000","last_clean":"2023-11-06T03:07:35.454636+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:07:35.454636+0000","last_undegraded":"2023-11-06T03:07:35.454636+0000","last_fullsized":"2023-11-06T03:07:35.454636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272819+0000","last_change":"2023-11-06T03:07:38.272819+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272819+0000","last_undegraded":"2023-11-06T03:07:38.272819+0000","last_fullsized":"2023-11-06T03:07:38.272819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.556066+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:07:36.556066+0000","last_peered":"2023-11-06T03:07:36.556066+0000","last_clean":"2023-11-06T03:07:36.556066+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:07:36.556066+0000","last_undegraded":"2023-11-06T03:07:36.556066+0000","last_fullsized":"2023-11-06T03:07:36.556066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273572+0000","last_change":"2023-11-06T03:07:38.273572+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273572+0000","last_undegraded":"2023-11-06T03:07:38.273572+0000","last_fullsized":"2023-11-06T03:07:38.273572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159137+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:07:36.159137+0000","last_peered":"2023-11-06T03:07:36.159137+0000","last_clean":"2023-11-06T03:07:36.159137+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:07:36.159137+0000","last_undegraded":"2023-11-06T03:07:36.159137+0000","last_fullsized":"2023-11-06T03:07:36.159137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268417+0000","last_change":"2023-11-06T03:07:38.268417+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268417+0000","last_undegraded":"2023-11-06T03:07:38.268417+0000","last_fullsized":"2023-11-06T03:07:38.268417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226174+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:07:36.226174+0000","last_peered":"2023-11-06T03:07:36.226174+0000","last_clean":"2023-11-06T03:07:36.226174+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:07:36.226174+0000","last_undegraded":"2023-11-06T03:07:36.226174+0000","last_fullsized":"2023-11-06T03:07:36.226174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.281950+0000","last_change":"2023-11-06T03:07:38.281950+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.281950+0000","last_undegraded":"2023-11-06T03:07:38.281950+0000","last_fullsized":"2023-11-06T03:07:38.281950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454684+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:07:35.454684+0000","last_peered":"2023-11-06T03:07:35.454684+0000","last_clean":"2023-11-06T03:07:35.454684+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:07:35.454684+0000","last_undegraded":"2023-11-06T03:07:35.454684+0000","last_fullsized":"2023-11-06T03:07:35.454684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268511+0000","last_change":"2023-11-06T03:07:38.268511+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268511+0000","last_undegraded":"2023-11-06T03:07:38.268511+0000","last_fullsized":"2023-11-06T03:07:38.268511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226199+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:07:36.226199+0000","last_peered":"2023-11-06T03:07:36.226199+0000","last_clean":"2023-11-06T03:07:36.226199+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:07:36.226199+0000","last_undegraded":"2023-11-06T03:07:36.226199+0000","last_fullsized":"2023-11-06T03:07:36.226199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268606+0000","last_change":"2023-11-06T03:07:38.268606+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268606+0000","last_undegraded":"2023-11-06T03:07:38.268606+0000","last_fullsized":"2023-11-06T03:07:38.268606+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.544981+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:07:35.544981+0000","last_peered":"2023-11-06T03:07:35.544981+0000","last_clean":"2023-11-06T03:07:35.544981+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:07:35.544981+0000","last_undegraded":"2023-11-06T03:07:35.544981+0000","last_fullsized":"2023-11-06T03:07:35.544981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50252,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321588,"statfs":{"total":96636764160,"available":96585306112,"internally_reserved":0,"allocated":1183744,"data_stored":843731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.397,"15min":1.397},"last":0.73799999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.872},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1739999999999999,"5min":1.444,"15min":1.444},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.627}]},{"osd":2,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93300000000000005,"15min":0.97099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3180000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.151}]},{"osd":3,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.111,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.016},{"interface":"front","average":{"1min":0.995,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.387,"5min":1.857,"15min":1.857},"last":1.103}]},{"osd":4,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.65000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.298,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.022},"min":{"1min":0.63400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1200000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3140000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.284},{"interface":"front","average":{"1min":0.999,"5min":1.069,"15min":1.099},"min":{"1min":0.66800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3700000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3120000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.80300000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6850000000000001,"5min":1.774,"15min":1.774},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.77300000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.335,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.367}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43908,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327932,"statfs":{"total":96636764160,"available":96591802368,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.226,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2210000000000001,"5min":2.073,"15min":2.073},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.411,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.036},{"interface":"front","average":{"1min":0.872,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.63800000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3600000000000001,"5min":1.778,"15min":1.778},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.167},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.329,"5min":1.806,"15min":1.806},"last":0.69599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3819999999999999,"5min":1.857,"15min":1.857},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.89700000000000002,"15min":0.88900000000000001},"min":{"1min":0.67800000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.117},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.294,"5min":2.931,"15min":2.931},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.137,"15min":1.163},"min":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.3100000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.036},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.429,"5min":1.506,"15min":1.506},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4750000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.093,"15min":1.105},"min":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4279999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.083,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3440000000000001,"5min":1.415,"15min":1.415},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43912,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327928,"statfs":{"total":96636764160,"available":96591798272,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2290000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.748,"15min":0.71499999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.018,"5min":1.2,"15min":1.2},"last":0.62},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2649999999999999,"5min":1.357,"15min":1.357},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.264,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.98499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":1.403,"15min":1.403},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93400000000000005,"15min":0.93100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2170000000000001,"5min":1.429,"15min":1.429},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":1.079},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1479999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.004,"15min":0.99099999999999999},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":1.248,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.999,"15min":1.0069999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.296,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2649999999999999,"5min":1.353,"15min":1.353},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.282,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2310000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49672,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322168,"statfs":{"total":96636764160,"available":96585900032,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.46999999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.0740000000000001},"min":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5820000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.60499999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0600000000000001,"5min":1.329,"15min":1.329},"last":1.042}]},{"osd":3,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1659999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.169},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.105}]},{"osd":5,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0129999999999999,"15min":1.0189999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.256},{"interface":"front","average":{"1min":1.006,"5min":1.087,"15min":1.111},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1990000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.012,"15min":1.0289999999999999},"min":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.1299999999999999,"15min":1.1659999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8120000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.357}]},{"osd":7,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.2549999999999999,"15min":1.1910000000000001},"min":{"1min":0.91000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":13.582000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2090000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49668,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322172,"statfs":{"total":96636764160,"available":96585904128,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.151,"5min":1.524,"15min":1.524},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94099999999999995,"5min":1.353,"15min":1.353},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1970000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1850000000000001,"5min":1.349,"15min":1.349},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":1.327,"15min":1.327},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93500000000000005,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0960000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97099999999999997,"15min":0.99199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1819999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":1.698,"15min":1.698},"last":0.88900000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97699999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4610000000000001,"5min":1.544,"15min":1.544},"last":1.0029999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5249999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.044,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.22,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0309999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.215,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.64600000000000002},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0940000000000001,"5min":1.121,"15min":1.121},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.67200000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.0249999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.75600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.92600000000000005,"15min":0.96399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2150000000000001,"5min":1.341,"15min":1.341},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86499999999999999,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.111,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93899999999999995,"15min":0.96999999999999997},"min":{"1min":0.73899999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5509999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.155}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.111},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.95299999999999996},"min":{"1min":0.70099999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1850000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6379999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95399999999999996,"15min":0.96999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.96099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.19,"15min":0.19},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.1749999999999998,"5min":0.91000000000000003,"15min":0.72099999999999997},"min":{"1min":0.29999999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":30.861000000000001,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.312,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.0449999999999999,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":14.753,"5min":14.753,"15min":14.753},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":3.306,"5min":1.4339999999999999,"15min":1.1539999999999999},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.103999999999999,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":3.0459999999999998,"5min":1.409,"15min":1.181},"min":{"1min":0.53100000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":44.753999999999998,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.2230000000000001,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":23.971,"5min":23.971,"15min":23.971},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.1469999999999998,"5min":1.2410000000000001,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.503,"15min":0.503},"max":{"1min":14.813000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.089},"min":{"1min":0.65400000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0589999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":1179648,"data_stored":842712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.27000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.70599999999999996,"15min":0.65400000000000003},"min":{"1min":0.377,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":7.1230000000000002,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.71999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.40500000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.96199999999999997,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.94799999999999995},"min":{"1min":0.62,"5min":0.432,"15min":0.432},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.238,"5min":0.98599999999999999,"15min":0.94799999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.7430000000000003,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.3}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.92000000000000004,"15min":0.90400000000000003},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.143},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0269999999999999,"15min":1.0169999999999999},"min":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.177}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0469999999999999,"15min":1.0149999999999999},"min":{"1min":0.65500000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0009999999999999,"15min":0.997},"min":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:07:43.654 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:07:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 Processor -- start 2023-11-06T03:07:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 -- start start 2023-11-06T03:07:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff41551e0 0x7fbff406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbff415e5e0 0x7fbff40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff415f580 con 0x7fbff415ea60 2023-11-06T03:07:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff415efc0 con 0x7fbff415e5e0 2023-11-06T03:07:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbffb658640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff415f2a0 con 0x7fbff41551e0 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbff8bf3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbff415e5e0 0x7fbff40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbff8bf3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbff415e5e0 0x7fbff40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50546/0 (socket says 172.21.15.157:50546) 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1439989770 learned_addr learned my addr 172.21.15.157:0/1439989770 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbff93f4640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff41551e0 0x7fbff406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.801+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1439989770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff41551e0 msgr2=0x7fbff406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbff8bf3640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff41551e0 0x7fbff406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1439989770 >> v1:172.21.15.195:6789/0 conn(0x7fbff415ea60 legacy=0x7fbff415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:43.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1439989770 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff406f5a0 con 0x7fbff415e5e0 2023-11-06T03:07:43.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbff8bf3640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff40b3410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fee3ac202ef9aaa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbfebfff640 1 -- 172.21.15.157:0/1439989770 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40ac640 con 0x7fbff415e5e0 2023-11-06T03:07:43.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbfebfff640 1 -- 172.21.15.157:0/1439989770 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe40ac7a0 con 0x7fbff415e5e0 2023-11-06T03:07:43.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbfebfff640 1 -- 172.21.15.157:0/1439989770 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40acb30 con 0x7fbff415e5e0 2023-11-06T03:07:43.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 -- 172.21.15.157:0/1439989770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 msgr2=0x7fbff40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff40b3410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe401c440 comp rx=0 tx=0).stop 2023-11-06T03:07:43.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 -- 172.21.15.157:0/1439989770 shutdown_connections 2023-11-06T03:07:43.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff40b3410 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 --2- 172.21.15.157:0/1439989770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff41551e0 0x7fbff406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 -- 172.21.15.157:0/1439989770 >> 172.21.15.157:0/1439989770 conn(0x7fbff4065930 msgr2=0x7fbff4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 -- 172.21.15.157:0/1439989770 shutdown_connections 2023-11-06T03:07:43.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.805+0000 7fbffb658640 1 -- 172.21.15.157:0/1439989770 wait complete. 2023-11-06T03:07:43.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 Processor -- start 2023-11-06T03:07:43.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- start start 2023-11-06T03:07:43.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 0x7fbff414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff415ea60 0x7fbff41493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414cbf0 con 0x7fbff415e5e0 2023-11-06T03:07:43.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414c350 con 0x7fbff41551e0 2023-11-06T03:07:43.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff414c630 con 0x7fbff415ea60 2023-11-06T03:07:43.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50562/0 (socket says 172.21.15.157:50562) 2023-11-06T03:07:43.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1255783714 learned_addr learned my addr 172.21.15.157:0/1255783714 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:43.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff9bf5640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff415ea60 0x7fbff41493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff93f4640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 0x7fbff414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff415ea60 msgr2=0x7fbff41493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff415ea60 0x7fbff41493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 msgr2=0x7fbff414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 0x7fbff414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 -- 172.21.15.157:0/1255783714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40094d0 con 0x7fbff415e5e0 2023-11-06T03:07:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff93f4640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 0x7fbff414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:43.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff8bf3640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40af040 tx=0x7fbfe40191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40ac640 con 0x7fbff415e5e0 2023-11-06T03:07:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe40b5870 con 0x7fbff415e5e0 2023-11-06T03:07:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff414c910 con 0x7fbff415e5e0 2023-11-06T03:07:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40b5c20 con 0x7fbff415e5e0 2023-11-06T03:07:43.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff4149920 con 0x7fbff415e5e0 2023-11-06T03:07:43.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff4149ab0 con 0x7fbff415e5e0 2023-11-06T03:07:43.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbfe40b3070 con 0x7fbff415e5e0 2023-11-06T03:07:43.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 0x7fbfd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:43.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfe411d560 con 0x7fbff415e5e0 2023-11-06T03:07:43.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.809+0000 7fbff93f4640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 0x7fbfd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:43.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.813+0000 7fbff93f4640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 0x7fbfd8067f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007eb0 tx=0x7fbff0007400 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:43.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.813+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbfe40eae50 con 0x7fbff415e5e0 2023-11-06T03:07:43.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.953+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbff40b3d60 con 0x7fbfd8065a60 2023-11-06T03:07:43.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.977+0000 7fbfe9ffb640 1 -- 172.21.15.157:0/1255783714 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299435 (secure 0 0 0) 0x7fbff000ddf0 con 0x7fbfd8065a60 2023-11-06T03:07:43.977 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:07:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 msgr2=0x7fbfd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 0x7fbfd8067f20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007eb0 tx=0x7fbff0007400 comp rx=0 tx=0).stop 2023-11-06T03:07:43.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 msgr2=0x7fbff41545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:43.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40af040 tx=0x7fbfe40191e0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 shutdown_connections 2023-11-06T03:07:43.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbfd8065a60 0x7fbfd8067f20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbff415ea60 0x7fbff41493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbff415e5e0 0x7fbff41545a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 --2- 172.21.15.157:0/1255783714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff41551e0 0x7fbff414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:43.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 >> 172.21.15.157:0/1255783714 conn(0x7fbff4065930 msgr2=0x7fbff40b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:43.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 shutdown_connections 2023-11-06T03:07:43.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:43.981+0000 7fbffb658640 1 -- 172.21.15.157:0/1255783714 wait complete. 2023-11-06T03:07:43.996 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-06T03:07:43.173699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346812,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":340992,"kb_avail":754627908,"statfs":{"total":773094113280,"available":772738977792,"internally_reserved":0,"allocated":5894144,"data_stored":3204583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":349175808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909522"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274775+0000","last_change":"2023-11-06T03:07:38.274775+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274775+0000","last_undegraded":"2023-11-06T03:07:38.274775+0000","last_fullsized":"2023-11-06T03:07:38.274775+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273492+0000","last_change":"2023-11-06T03:07:38.273492+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273492+0000","last_undegraded":"2023-11-06T03:07:38.273492+0000","last_fullsized":"2023-11-06T03:07:38.273492+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272757+0000","last_change":"2023-11-06T03:07:38.272757+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272757+0000","last_undegraded":"2023-11-06T03:07:38.272757+0000","last_fullsized":"2023-11-06T03:07:38.272757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.280184+0000","last_change":"2023-11-06T03:07:38.280184+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.280184+0000","last_undegraded":"2023-11-06T03:07:38.280184+0000","last_fullsized":"2023-11-06T03:07:38.280184+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273187+0000","last_change":"2023-11-06T03:07:38.273187+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273187+0000","last_undegraded":"2023-11-06T03:07:38.273187+0000","last_fullsized":"2023-11-06T03:07:38.273187+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268342+0000","last_change":"2023-11-06T03:07:38.268342+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268342+0000","last_undegraded":"2023-11-06T03:07:38.268342+0000","last_fullsized":"2023-11-06T03:07:38.268342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274161+0000","last_change":"2023-11-06T03:07:38.274161+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274161+0000","last_undegraded":"2023-11-06T03:07:38.274161+0000","last_fullsized":"2023-11-06T03:07:38.274161+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271731+0000","last_change":"2023-11-06T03:07:38.271731+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271731+0000","last_undegraded":"2023-11-06T03:07:38.271731+0000","last_fullsized":"2023-11-06T03:07:38.271731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277966+0000","last_change":"2023-11-06T03:07:38.277966+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277966+0000","last_undegraded":"2023-11-06T03:07:38.277966+0000","last_fullsized":"2023-11-06T03:07:38.277966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268292+0000","last_change":"2023-11-06T03:07:38.268292+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268292+0000","last_undegraded":"2023-11-06T03:07:38.268292+0000","last_fullsized":"2023-11-06T03:07:38.268292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.279319+0000","last_change":"2023-11-06T03:07:38.279319+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.279319+0000","last_undegraded":"2023-11-06T03:07:38.279319+0000","last_fullsized":"2023-11-06T03:07:38.279319+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270185+0000","last_change":"2023-11-06T03:07:38.270185+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270185+0000","last_undegraded":"2023-11-06T03:07:38.270185+0000","last_fullsized":"2023-11-06T03:07:38.270185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272337+0000","last_change":"2023-11-06T03:07:38.272337+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272337+0000","last_undegraded":"2023-11-06T03:07:38.272337+0000","last_fullsized":"2023-11-06T03:07:38.272337+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273107+0000","last_change":"2023-11-06T03:07:38.273107+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273107+0000","last_undegraded":"2023-11-06T03:07:38.273107+0000","last_fullsized":"2023-11-06T03:07:38.273107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277799+0000","last_change":"2023-11-06T03:07:38.277799+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277799+0000","last_undegraded":"2023-11-06T03:07:38.277799+0000","last_fullsized":"2023-11-06T03:07:38.277799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270291+0000","last_change":"2023-11-06T03:07:38.270291+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270291+0000","last_undegraded":"2023-11-06T03:07:38.270291+0000","last_fullsized":"2023-11-06T03:07:38.270291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271213+0000","last_change":"2023-11-06T03:07:38.271213+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271213+0000","last_undegraded":"2023-11-06T03:07:38.271213+0000","last_fullsized":"2023-11-06T03:07:38.271213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274204+0000","last_change":"2023-11-06T03:07:38.274204+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274204+0000","last_undegraded":"2023-11-06T03:07:38.274204+0000","last_fullsized":"2023-11-06T03:07:38.274204+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273197+0000","last_change":"2023-11-06T03:07:38.273197+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273197+0000","last_undegraded":"2023-11-06T03:07:38.273197+0000","last_fullsized":"2023-11-06T03:07:38.273197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273072+0000","last_change":"2023-11-06T03:07:38.273072+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273072+0000","last_undegraded":"2023-11-06T03:07:38.273072+0000","last_fullsized":"2023-11-06T03:07:38.273072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273478+0000","last_change":"2023-11-06T03:07:38.273478+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273478+0000","last_undegraded":"2023-11-06T03:07:38.273478+0000","last_fullsized":"2023-11-06T03:07:38.273478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.269961+0000","last_change":"2023-11-06T03:07:38.269961+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.269961+0000","last_undegraded":"2023-11-06T03:07:38.269961+0000","last_fullsized":"2023-11-06T03:07:38.269961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270873+0000","last_change":"2023-11-06T03:07:38.270873+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270873+0000","last_undegraded":"2023-11-06T03:07:38.270873+0000","last_fullsized":"2023-11-06T03:07:38.270873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274121+0000","last_change":"2023-11-06T03:07:38.274121+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274121+0000","last_undegraded":"2023-11-06T03:07:38.274121+0000","last_fullsized":"2023-11-06T03:07:38.274121+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270776+0000","last_change":"2023-11-06T03:07:38.270776+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270776+0000","last_undegraded":"2023-11-06T03:07:38.270776+0000","last_fullsized":"2023-11-06T03:07:38.270776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454639+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:07:35.454639+0000","last_peered":"2023-11-06T03:07:35.454639+0000","last_clean":"2023-11-06T03:07:35.454639+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:07:35.454639+0000","last_undegraded":"2023-11-06T03:07:35.454639+0000","last_fullsized":"2023-11-06T03:07:35.454639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159097+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:07:36.159097+0000","last_peered":"2023-11-06T03:07:36.159097+0000","last_clean":"2023-11-06T03:07:36.159097+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:07:36.159097+0000","last_undegraded":"2023-11-06T03:07:36.159097+0000","last_fullsized":"2023-11-06T03:07:36.159097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277505+0000","last_change":"2023-11-06T03:07:38.277505+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277505+0000","last_undegraded":"2023-11-06T03:07:38.277505+0000","last_fullsized":"2023-11-06T03:07:38.277505+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454636+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:07:35.454636+0000","last_peered":"2023-11-06T03:07:35.454636+0000","last_clean":"2023-11-06T03:07:35.454636+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:07:35.454636+0000","last_undegraded":"2023-11-06T03:07:35.454636+0000","last_fullsized":"2023-11-06T03:07:35.454636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272819+0000","last_change":"2023-11-06T03:07:38.272819+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272819+0000","last_undegraded":"2023-11-06T03:07:38.272819+0000","last_fullsized":"2023-11-06T03:07:38.272819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.556066+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:07:36.556066+0000","last_peered":"2023-11-06T03:07:36.556066+0000","last_clean":"2023-11-06T03:07:36.556066+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:07:36.556066+0000","last_undegraded":"2023-11-06T03:07:36.556066+0000","last_fullsized":"2023-11-06T03:07:36.556066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273572+0000","last_change":"2023-11-06T03:07:38.273572+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273572+0000","last_undegraded":"2023-11-06T03:07:38.273572+0000","last_fullsized":"2023-11-06T03:07:38.273572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159137+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:07:36.159137+0000","last_peered":"2023-11-06T03:07:36.159137+0000","last_clean":"2023-11-06T03:07:36.159137+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:07:36.159137+0000","last_undegraded":"2023-11-06T03:07:36.159137+0000","last_fullsized":"2023-11-06T03:07:36.159137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268417+0000","last_change":"2023-11-06T03:07:38.268417+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268417+0000","last_undegraded":"2023-11-06T03:07:38.268417+0000","last_fullsized":"2023-11-06T03:07:38.268417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226174+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:07:36.226174+0000","last_peered":"2023-11-06T03:07:36.226174+0000","last_clean":"2023-11-06T03:07:36.226174+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:07:36.226174+0000","last_undegraded":"2023-11-06T03:07:36.226174+0000","last_fullsized":"2023-11-06T03:07:36.226174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.281950+0000","last_change":"2023-11-06T03:07:38.281950+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.281950+0000","last_undegraded":"2023-11-06T03:07:38.281950+0000","last_fullsized":"2023-11-06T03:07:38.281950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454684+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:07:35.454684+0000","last_peered":"2023-11-06T03:07:35.454684+0000","last_clean":"2023-11-06T03:07:35.454684+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:07:35.454684+0000","last_undegraded":"2023-11-06T03:07:35.454684+0000","last_fullsized":"2023-11-06T03:07:35.454684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268511+0000","last_change":"2023-11-06T03:07:38.268511+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268511+0000","last_undegraded":"2023-11-06T03:07:38.268511+0000","last_fullsized":"2023-11-06T03:07:38.268511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226199+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:07:36.226199+0000","last_peered":"2023-11-06T03:07:36.226199+0000","last_clean":"2023-11-06T03:07:36.226199+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:07:36.226199+0000","last_undegraded":"2023-11-06T03:07:36.226199+0000","last_fullsized":"2023-11-06T03:07:36.226199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268606+0000","last_change":"2023-11-06T03:07:38.268606+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268606+0000","last_undegraded":"2023-11-06T03:07:38.268606+0000","last_fullsized":"2023-11-06T03:07:38.268606+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.544981+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:07:35.544981+0000","last_peered":"2023-11-06T03:07:35.544981+0000","last_clean":"2023-11-06T03:07:35.544981+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:07:35.544981+0000","last_undegraded":"2023-11-06T03:07:35.544981+0000","last_fullsized":"2023-11-06T03:07:35.544981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50252,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321588,"statfs":{"total":96636764160,"available":96585306112,"internally_reserved":0,"allocated":1183744,"data_stored":843731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.397,"15min":1.397},"last":0.73799999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.872},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1739999999999999,"5min":1.444,"15min":1.444},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.627}]},{"osd":2,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93300000000000005,"15min":0.97099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3180000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.151}]},{"osd":3,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.111,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.016},{"interface":"front","average":{"1min":0.995,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.387,"5min":1.857,"15min":1.857},"last":1.103}]},{"osd":4,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.65000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.298,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.022},"min":{"1min":0.63400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1200000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3140000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.284},{"interface":"front","average":{"1min":0.999,"5min":1.069,"15min":1.099},"min":{"1min":0.66800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3700000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3120000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.80300000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6850000000000001,"5min":1.774,"15min":1.774},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.77300000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.335,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.367}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43908,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327932,"statfs":{"total":96636764160,"available":96591802368,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.226,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2210000000000001,"5min":2.073,"15min":2.073},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.411,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.036},{"interface":"front","average":{"1min":0.872,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.63800000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3600000000000001,"5min":1.778,"15min":1.778},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.167},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.329,"5min":1.806,"15min":1.806},"last":0.69599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3819999999999999,"5min":1.857,"15min":1.857},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.89700000000000002,"15min":0.88900000000000001},"min":{"1min":0.67800000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.117},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.294,"5min":2.931,"15min":2.931},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.137,"15min":1.163},"min":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.3100000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.036},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.429,"5min":1.506,"15min":1.506},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4750000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.093,"15min":1.105},"min":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4279999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.083,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3440000000000001,"5min":1.415,"15min":1.415},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43912,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327928,"statfs":{"total":96636764160,"available":96591798272,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2290000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.748,"15min":0.71499999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.018,"5min":1.2,"15min":1.2},"last":0.62},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2649999999999999,"5min":1.357,"15min":1.357},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.264,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.98499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":1.403,"15min":1.403},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93400000000000005,"15min":0.93100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2170000000000001,"5min":1.429,"15min":1.429},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":1.079},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1479999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.004,"15min":0.99099999999999999},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":1.248,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.999,"15min":1.0069999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.296,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2649999999999999,"5min":1.353,"15min":1.353},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.282,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2310000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49672,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322168,"statfs":{"total":96636764160,"available":96585900032,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.46999999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.0740000000000001},"min":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5820000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.60499999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0600000000000001,"5min":1.329,"15min":1.329},"last":1.042}]},{"osd":3,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1659999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.169},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.105}]},{"osd":5,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0129999999999999,"15min":1.0189999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.256},{"interface":"front","average":{"1min":1.006,"5min":1.087,"15min":1.111},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1990000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.012,"15min":1.0289999999999999},"min":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.1299999999999999,"15min":1.1659999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8120000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.357}]},{"osd":7,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.2549999999999999,"15min":1.1910000000000001},"min":{"1min":0.91000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":13.582000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2090000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49668,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322172,"statfs":{"total":96636764160,"available":96585904128,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.151,"5min":1.524,"15min":1.524},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94099999999999995,"5min":1.353,"15min":1.353},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1970000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1850000000000001,"5min":1.349,"15min":1.349},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":1.327,"15min":1.327},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93500000000000005,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0960000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97099999999999997,"15min":0.99199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1819999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":1.698,"15min":1.698},"last":0.88900000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97699999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4610000000000001,"5min":1.544,"15min":1.544},"last":1.0029999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5249999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.044,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.22,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0309999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.215,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.64600000000000002},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0940000000000001,"5min":1.121,"15min":1.121},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.67200000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.0249999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.75600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.92600000000000005,"15min":0.96399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2150000000000001,"5min":1.341,"15min":1.341},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86499999999999999,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.111,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93899999999999995,"15min":0.96999999999999997},"min":{"1min":0.73899999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5509999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.155}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.111},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.95299999999999996},"min":{"1min":0.70099999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1850000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6379999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95399999999999996,"15min":0.96999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.96099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.19,"15min":0.19},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.1749999999999998,"5min":0.91000000000000003,"15min":0.72099999999999997},"min":{"1min":0.29999999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":30.861000000000001,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.312,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.0449999999999999,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":14.753,"5min":14.753,"15min":14.753},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":3.306,"5min":1.4339999999999999,"15min":1.1539999999999999},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.103999999999999,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":3.0459999999999998,"5min":1.409,"15min":1.181},"min":{"1min":0.53100000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":44.753999999999998,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.2230000000000001,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":23.971,"5min":23.971,"15min":23.971},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.1469999999999998,"5min":1.2410000000000001,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.503,"15min":0.503},"max":{"1min":14.813000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.089},"min":{"1min":0.65400000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0589999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":1179648,"data_stored":842712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.27000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.70599999999999996,"15min":0.65400000000000003},"min":{"1min":0.377,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":7.1230000000000002,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.71999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.40500000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.96199999999999997,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.94799999999999995},"min":{"1min":0.62,"5min":0.432,"15min":0.432},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.238,"5min":0.98599999999999999,"15min":0.94799999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.7430000000000003,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.3}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.92000000000000004,"15min":0.90400000000000003},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.143},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0269999999999999,"15min":1.0169999999999999},"min":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.177}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0469999999999999,"15min":1.0149999999999999},"min":{"1min":0.65500000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0009999999999999,"15min":0.997},"min":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:07:44.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:07:44.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 Processor -- start 2023-11-06T03:07:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 -- start start 2023-11-06T03:07:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db00ca570 0x7f1db00b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1db00b4ab0 0x7f1db006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00b4f20 con 0x7f1db015adc0 2023-11-06T03:07:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db006b8d0 con 0x7f1db00ca570 2023-11-06T03:07:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db72f9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db006be90 con 0x7f1db00b4ab0 2023-11-06T03:07:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db00ca570 0x7f1db00b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1db00ca570 0x7f1db00b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50578/0 (socket says 172.21.15.157:50578) 2023-11-06T03:07:44.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 -- 172.21.15.157:0/124929555 learned_addr learned my addr 172.21.15.157:0/124929555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:44.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db5896640 1 --2- 172.21.15.157:0/124929555 >> v2:172.21.15.195:3300/0 conn(0x7f1db00b4ab0 0x7f1db006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 -- 172.21.15.157:0/124929555 >> v2:172.21.15.195:3300/0 conn(0x7f1db00b4ab0 msgr2=0x7f1db006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 --2- 172.21.15.157:0/124929555 >> v2:172.21.15.195:3300/0 conn(0x7f1db00b4ab0 0x7f1db006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 -- 172.21.15.157:0/124929555 >> v1:172.21.15.195:6789/0 conn(0x7f1db015adc0 legacy=0x7f1db0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.157+0000 7f1db4894640 1 -- 172.21.15.157:0/124929555 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db0071880 con 0x7f1db00ca570 2023-11-06T03:07:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db4894640 1 --2- 172.21.15.157:0/124929555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db00ca570 0x7f1db00b4570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009870 tx=0x7f1da001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d9504886e9a6d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1da7fff640 1 -- 172.21.15.157:0/124929555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00ab600 con 0x7f1db00ca570 2023-11-06T03:07:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1da7fff640 1 -- 172.21.15.157:0/124929555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da00a7050 con 0x7f1db00ca570 2023-11-06T03:07:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1da7fff640 1 -- 172.21.15.157:0/124929555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00ab980 con 0x7f1db00ca570 2023-11-06T03:07:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- 172.21.15.157:0/124929555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db00ca570 msgr2=0x7f1db00b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- 172.21.15.157:0/124929555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db00ca570 0x7f1db00b4570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009870 tx=0x7f1da001c3b0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- 172.21.15.157:0/124929555 shutdown_connections 2023-11-06T03:07:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- 172.21.15.157:0/124929555 >> v2:172.21.15.195:3300/0 conn(0x7f1db00b4ab0 0x7f1db006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- 172.21.15.157:0/124929555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db00ca570 0x7f1db00b4570 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- 172.21.15.157:0/124929555 >> 172.21.15.157:0/124929555 conn(0x7f1db00cb990 msgr2=0x7f1db00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- 172.21.15.157:0/124929555 shutdown_connections 2023-11-06T03:07:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- 172.21.15.157:0/124929555 wait complete. 2023-11-06T03:07:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 Processor -- start 2023-11-06T03:07:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- start start 2023-11-06T03:07:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 0x7f1db0145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1db00ca570 0x7f1db0146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0149e80 con 0x7f1db015adc0 2023-11-06T03:07:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0146b50 con 0x7f1db00ca570 2023-11-06T03:07:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db72f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00af860 con 0x7f1db00b4ab0 2023-11-06T03:07:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db5896640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db5896640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50592/0 (socket says 172.21.15.157:50592) 2023-11-06T03:07:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.161+0000 7f1db5896640 1 -- 172.21.15.157:0/169613880 learned_addr learned my addr 172.21.15.157:0/169613880 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db4894640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1db00ca570 0x7f1db0146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5095640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 0x7f1db0145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 -- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 msgr2=0x7f1db0145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 0x7f1db0145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 -- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1db00ca570 msgr2=0x7f1db0146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1db00ca570 0x7f1db0146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 -- 172.21.15.157:0/169613880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00094d0 con 0x7f1db015adc0 2023-11-06T03:07:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5095640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 0x7f1db0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5896640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009d30 tx=0x7f1da800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8009ba0 con 0x7f1db015adc0 2023-11-06T03:07:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800f040 con 0x7f1db015adc0 2023-11-06T03:07:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db01498c0 con 0x7f1db015adc0 2023-11-06T03:07:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da80146c0 con 0x7f1db015adc0 2023-11-06T03:07:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db00af4f0 con 0x7f1db015adc0 2023-11-06T03:07:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db00af350 con 0x7f1db015adc0 2023-11-06T03:07:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1da8014820 con 0x7f1db015adc0 2023-11-06T03:07:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 0x7f1d94067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1da80840d0 con 0x7f1db015adc0 2023-11-06T03:07:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5095640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 0x7f1d94067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.165+0000 7f1db5095640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 0x7f1d94067f20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1db01462f0 tx=0x7f1dac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.169+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1db00af350 con 0x7f1db015adc0 2023-11-06T03:07:44.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.317+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1db0071590 con 0x7f1d94065a60 2023-11-06T03:07:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.341+0000 7f1da5ffb640 1 -- 172.21.15.157:0/169613880 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299435 (secure 0 0 0) 0x7f1dac0013f0 con 0x7f1d94065a60 2023-11-06T03:07:44.343 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:44.345 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:07:44.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 msgr2=0x7f1d94067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 0x7f1d94067f20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1db01462f0 tx=0x7f1dac009040 comp rx=0 tx=0).stop 2023-11-06T03:07:44.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 msgr2=0x7f1db00afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009d30 tx=0x7f1da800c490 comp rx=0 tx=0).stop 2023-11-06T03:07:44.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 shutdown_connections 2023-11-06T03:07:44.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1d94065a60 0x7f1d94067f20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1db015adc0 0x7f1db00afcf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1db00ca570 0x7f1db0146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 --2- 172.21.15.157:0/169613880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1db00b4ab0 0x7f1db0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 >> 172.21.15.157:0/169613880 conn(0x7f1db00cb990 msgr2=0x7f1db015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:44.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 shutdown_connections 2023-11-06T03:07:44.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.345+0000 7f1db72f9640 1 -- 172.21.15.157:0/169613880 wait complete. 2023-11-06T03:07:44.362 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-06T03:07:43.173699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346812,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":340992,"kb_avail":754627908,"statfs":{"total":773094113280,"available":772738977792,"internally_reserved":0,"allocated":5894144,"data_stored":3204583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":349175808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909522"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274775+0000","last_change":"2023-11-06T03:07:38.274775+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274775+0000","last_undegraded":"2023-11-06T03:07:38.274775+0000","last_fullsized":"2023-11-06T03:07:38.274775+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273492+0000","last_change":"2023-11-06T03:07:38.273492+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273492+0000","last_undegraded":"2023-11-06T03:07:38.273492+0000","last_fullsized":"2023-11-06T03:07:38.273492+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272757+0000","last_change":"2023-11-06T03:07:38.272757+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272757+0000","last_undegraded":"2023-11-06T03:07:38.272757+0000","last_fullsized":"2023-11-06T03:07:38.272757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.280184+0000","last_change":"2023-11-06T03:07:38.280184+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.280184+0000","last_undegraded":"2023-11-06T03:07:38.280184+0000","last_fullsized":"2023-11-06T03:07:38.280184+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273187+0000","last_change":"2023-11-06T03:07:38.273187+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273187+0000","last_undegraded":"2023-11-06T03:07:38.273187+0000","last_fullsized":"2023-11-06T03:07:38.273187+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268342+0000","last_change":"2023-11-06T03:07:38.268342+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268342+0000","last_undegraded":"2023-11-06T03:07:38.268342+0000","last_fullsized":"2023-11-06T03:07:38.268342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274161+0000","last_change":"2023-11-06T03:07:38.274161+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274161+0000","last_undegraded":"2023-11-06T03:07:38.274161+0000","last_fullsized":"2023-11-06T03:07:38.274161+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271731+0000","last_change":"2023-11-06T03:07:38.271731+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271731+0000","last_undegraded":"2023-11-06T03:07:38.271731+0000","last_fullsized":"2023-11-06T03:07:38.271731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277966+0000","last_change":"2023-11-06T03:07:38.277966+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277966+0000","last_undegraded":"2023-11-06T03:07:38.277966+0000","last_fullsized":"2023-11-06T03:07:38.277966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268292+0000","last_change":"2023-11-06T03:07:38.268292+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268292+0000","last_undegraded":"2023-11-06T03:07:38.268292+0000","last_fullsized":"2023-11-06T03:07:38.268292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.279319+0000","last_change":"2023-11-06T03:07:38.279319+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.279319+0000","last_undegraded":"2023-11-06T03:07:38.279319+0000","last_fullsized":"2023-11-06T03:07:38.279319+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270185+0000","last_change":"2023-11-06T03:07:38.270185+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270185+0000","last_undegraded":"2023-11-06T03:07:38.270185+0000","last_fullsized":"2023-11-06T03:07:38.270185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272337+0000","last_change":"2023-11-06T03:07:38.272337+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272337+0000","last_undegraded":"2023-11-06T03:07:38.272337+0000","last_fullsized":"2023-11-06T03:07:38.272337+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273107+0000","last_change":"2023-11-06T03:07:38.273107+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273107+0000","last_undegraded":"2023-11-06T03:07:38.273107+0000","last_fullsized":"2023-11-06T03:07:38.273107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277799+0000","last_change":"2023-11-06T03:07:38.277799+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277799+0000","last_undegraded":"2023-11-06T03:07:38.277799+0000","last_fullsized":"2023-11-06T03:07:38.277799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270291+0000","last_change":"2023-11-06T03:07:38.270291+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270291+0000","last_undegraded":"2023-11-06T03:07:38.270291+0000","last_fullsized":"2023-11-06T03:07:38.270291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271213+0000","last_change":"2023-11-06T03:07:38.271213+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271213+0000","last_undegraded":"2023-11-06T03:07:38.271213+0000","last_fullsized":"2023-11-06T03:07:38.271213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274204+0000","last_change":"2023-11-06T03:07:38.274204+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274204+0000","last_undegraded":"2023-11-06T03:07:38.274204+0000","last_fullsized":"2023-11-06T03:07:38.274204+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273197+0000","last_change":"2023-11-06T03:07:38.273197+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273197+0000","last_undegraded":"2023-11-06T03:07:38.273197+0000","last_fullsized":"2023-11-06T03:07:38.273197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273072+0000","last_change":"2023-11-06T03:07:38.273072+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273072+0000","last_undegraded":"2023-11-06T03:07:38.273072+0000","last_fullsized":"2023-11-06T03:07:38.273072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273478+0000","last_change":"2023-11-06T03:07:38.273478+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273478+0000","last_undegraded":"2023-11-06T03:07:38.273478+0000","last_fullsized":"2023-11-06T03:07:38.273478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.269961+0000","last_change":"2023-11-06T03:07:38.269961+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.269961+0000","last_undegraded":"2023-11-06T03:07:38.269961+0000","last_fullsized":"2023-11-06T03:07:38.269961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270873+0000","last_change":"2023-11-06T03:07:38.270873+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270873+0000","last_undegraded":"2023-11-06T03:07:38.270873+0000","last_fullsized":"2023-11-06T03:07:38.270873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274121+0000","last_change":"2023-11-06T03:07:38.274121+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274121+0000","last_undegraded":"2023-11-06T03:07:38.274121+0000","last_fullsized":"2023-11-06T03:07:38.274121+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270776+0000","last_change":"2023-11-06T03:07:38.270776+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270776+0000","last_undegraded":"2023-11-06T03:07:38.270776+0000","last_fullsized":"2023-11-06T03:07:38.270776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454639+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:07:35.454639+0000","last_peered":"2023-11-06T03:07:35.454639+0000","last_clean":"2023-11-06T03:07:35.454639+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:07:35.454639+0000","last_undegraded":"2023-11-06T03:07:35.454639+0000","last_fullsized":"2023-11-06T03:07:35.454639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159097+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:07:36.159097+0000","last_peered":"2023-11-06T03:07:36.159097+0000","last_clean":"2023-11-06T03:07:36.159097+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:07:36.159097+0000","last_undegraded":"2023-11-06T03:07:36.159097+0000","last_fullsized":"2023-11-06T03:07:36.159097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277505+0000","last_change":"2023-11-06T03:07:38.277505+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277505+0000","last_undegraded":"2023-11-06T03:07:38.277505+0000","last_fullsized":"2023-11-06T03:07:38.277505+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454636+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:07:35.454636+0000","last_peered":"2023-11-06T03:07:35.454636+0000","last_clean":"2023-11-06T03:07:35.454636+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:07:35.454636+0000","last_undegraded":"2023-11-06T03:07:35.454636+0000","last_fullsized":"2023-11-06T03:07:35.454636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272819+0000","last_change":"2023-11-06T03:07:38.272819+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272819+0000","last_undegraded":"2023-11-06T03:07:38.272819+0000","last_fullsized":"2023-11-06T03:07:38.272819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.556066+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:07:36.556066+0000","last_peered":"2023-11-06T03:07:36.556066+0000","last_clean":"2023-11-06T03:07:36.556066+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:07:36.556066+0000","last_undegraded":"2023-11-06T03:07:36.556066+0000","last_fullsized":"2023-11-06T03:07:36.556066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273572+0000","last_change":"2023-11-06T03:07:38.273572+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273572+0000","last_undegraded":"2023-11-06T03:07:38.273572+0000","last_fullsized":"2023-11-06T03:07:38.273572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159137+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:07:36.159137+0000","last_peered":"2023-11-06T03:07:36.159137+0000","last_clean":"2023-11-06T03:07:36.159137+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:07:36.159137+0000","last_undegraded":"2023-11-06T03:07:36.159137+0000","last_fullsized":"2023-11-06T03:07:36.159137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268417+0000","last_change":"2023-11-06T03:07:38.268417+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268417+0000","last_undegraded":"2023-11-06T03:07:38.268417+0000","last_fullsized":"2023-11-06T03:07:38.268417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226174+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:07:36.226174+0000","last_peered":"2023-11-06T03:07:36.226174+0000","last_clean":"2023-11-06T03:07:36.226174+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:07:36.226174+0000","last_undegraded":"2023-11-06T03:07:36.226174+0000","last_fullsized":"2023-11-06T03:07:36.226174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.281950+0000","last_change":"2023-11-06T03:07:38.281950+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.281950+0000","last_undegraded":"2023-11-06T03:07:38.281950+0000","last_fullsized":"2023-11-06T03:07:38.281950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454684+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:07:35.454684+0000","last_peered":"2023-11-06T03:07:35.454684+0000","last_clean":"2023-11-06T03:07:35.454684+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:07:35.454684+0000","last_undegraded":"2023-11-06T03:07:35.454684+0000","last_fullsized":"2023-11-06T03:07:35.454684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268511+0000","last_change":"2023-11-06T03:07:38.268511+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268511+0000","last_undegraded":"2023-11-06T03:07:38.268511+0000","last_fullsized":"2023-11-06T03:07:38.268511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226199+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:07:36.226199+0000","last_peered":"2023-11-06T03:07:36.226199+0000","last_clean":"2023-11-06T03:07:36.226199+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:07:36.226199+0000","last_undegraded":"2023-11-06T03:07:36.226199+0000","last_fullsized":"2023-11-06T03:07:36.226199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268606+0000","last_change":"2023-11-06T03:07:38.268606+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268606+0000","last_undegraded":"2023-11-06T03:07:38.268606+0000","last_fullsized":"2023-11-06T03:07:38.268606+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.544981+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:07:35.544981+0000","last_peered":"2023-11-06T03:07:35.544981+0000","last_clean":"2023-11-06T03:07:35.544981+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:07:35.544981+0000","last_undegraded":"2023-11-06T03:07:35.544981+0000","last_fullsized":"2023-11-06T03:07:35.544981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50252,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321588,"statfs":{"total":96636764160,"available":96585306112,"internally_reserved":0,"allocated":1183744,"data_stored":843731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.397,"15min":1.397},"last":0.73799999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.872},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1739999999999999,"5min":1.444,"15min":1.444},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.627}]},{"osd":2,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93300000000000005,"15min":0.97099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3180000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.151}]},{"osd":3,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.111,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.016},{"interface":"front","average":{"1min":0.995,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.387,"5min":1.857,"15min":1.857},"last":1.103}]},{"osd":4,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.65000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.298,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.022},"min":{"1min":0.63400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1200000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3140000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.284},{"interface":"front","average":{"1min":0.999,"5min":1.069,"15min":1.099},"min":{"1min":0.66800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3700000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3120000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.80300000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6850000000000001,"5min":1.774,"15min":1.774},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.77300000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.335,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.367}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43908,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327932,"statfs":{"total":96636764160,"available":96591802368,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.226,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2210000000000001,"5min":2.073,"15min":2.073},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.411,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.036},{"interface":"front","average":{"1min":0.872,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.63800000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3600000000000001,"5min":1.778,"15min":1.778},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.167},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.329,"5min":1.806,"15min":1.806},"last":0.69599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3819999999999999,"5min":1.857,"15min":1.857},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.89700000000000002,"15min":0.88900000000000001},"min":{"1min":0.67800000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.117},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.294,"5min":2.931,"15min":2.931},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.137,"15min":1.163},"min":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.3100000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.036},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.429,"5min":1.506,"15min":1.506},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4750000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.093,"15min":1.105},"min":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4279999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.083,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3440000000000001,"5min":1.415,"15min":1.415},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43912,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327928,"statfs":{"total":96636764160,"available":96591798272,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2290000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.748,"15min":0.71499999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.018,"5min":1.2,"15min":1.2},"last":0.62},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2649999999999999,"5min":1.357,"15min":1.357},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.264,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.98499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":1.403,"15min":1.403},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93400000000000005,"15min":0.93100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2170000000000001,"5min":1.429,"15min":1.429},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":1.079},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1479999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.004,"15min":0.99099999999999999},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":1.248,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.999,"15min":1.0069999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.296,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2649999999999999,"5min":1.353,"15min":1.353},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.282,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2310000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49672,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322168,"statfs":{"total":96636764160,"available":96585900032,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.46999999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.0740000000000001},"min":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5820000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.60499999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0600000000000001,"5min":1.329,"15min":1.329},"last":1.042}]},{"osd":3,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1659999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.169},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.105}]},{"osd":5,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0129999999999999,"15min":1.0189999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.256},{"interface":"front","average":{"1min":1.006,"5min":1.087,"15min":1.111},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1990000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.012,"15min":1.0289999999999999},"min":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.1299999999999999,"15min":1.1659999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8120000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.357}]},{"osd":7,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.2549999999999999,"15min":1.1910000000000001},"min":{"1min":0.91000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":13.582000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2090000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49668,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322172,"statfs":{"total":96636764160,"available":96585904128,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.151,"5min":1.524,"15min":1.524},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94099999999999995,"5min":1.353,"15min":1.353},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1970000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1850000000000001,"5min":1.349,"15min":1.349},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":1.327,"15min":1.327},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93500000000000005,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0960000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97099999999999997,"15min":0.99199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1819999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":1.698,"15min":1.698},"last":0.88900000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97699999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4610000000000001,"5min":1.544,"15min":1.544},"last":1.0029999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5249999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.044,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.22,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0309999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.215,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.64600000000000002},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0940000000000001,"5min":1.121,"15min":1.121},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.67200000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.0249999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.75600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.92600000000000005,"15min":0.96399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2150000000000001,"5min":1.341,"15min":1.341},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86499999999999999,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.111,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93899999999999995,"15min":0.96999999999999997},"min":{"1min":0.73899999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5509999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.155}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.111},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.95299999999999996},"min":{"1min":0.70099999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1850000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6379999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95399999999999996,"15min":0.96999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.96099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.19,"15min":0.19},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.1749999999999998,"5min":0.91000000000000003,"15min":0.72099999999999997},"min":{"1min":0.29999999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":30.861000000000001,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.312,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.0449999999999999,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":14.753,"5min":14.753,"15min":14.753},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":3.306,"5min":1.4339999999999999,"15min":1.1539999999999999},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.103999999999999,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":3.0459999999999998,"5min":1.409,"15min":1.181},"min":{"1min":0.53100000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":44.753999999999998,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.2230000000000001,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":23.971,"5min":23.971,"15min":23.971},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.1469999999999998,"5min":1.2410000000000001,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.503,"15min":0.503},"max":{"1min":14.813000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.089},"min":{"1min":0.65400000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0589999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":1179648,"data_stored":842712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.27000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.70599999999999996,"15min":0.65400000000000003},"min":{"1min":0.377,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":7.1230000000000002,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.71999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.40500000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.96199999999999997,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.94799999999999995},"min":{"1min":0.62,"5min":0.432,"15min":0.432},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.238,"5min":0.98599999999999999,"15min":0.94799999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.7430000000000003,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.3}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.92000000000000004,"15min":0.90400000000000003},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.143},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0269999999999999,"15min":1.0169999999999999},"min":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.177}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0469999999999999,"15min":1.0149999999999999},"min":{"1min":0.65500000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0009999999999999,"15min":0.997},"min":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:07:44.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-11-06T03:07:44.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-11-06T03:07:44.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-11-06T03:07:44.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-11-06T03:07:44.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-11-06T03:07:44.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-11-06T03:07:44.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-11-06T03:07:44.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-11-06T03:07:44.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-11-06T03:07:44.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-11-06T03:07:44.376 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.376 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-11-06T03:07:44.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.377 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-11-06T03:07:44.377 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.378 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-11-06T03:07:44.378 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.379 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-11-06T03:07:44.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-11-06T03:07:44.380 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.380 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-11-06T03:07:44.381 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.381 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-11-06T03:07:44.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.382 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-11-06T03:07:44.382 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-11-06T03:07:44.383 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-11-06T03:07:44.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-11-06T03:07:44.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-11-06T03:07:44.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-11-06T03:07:44.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-11-06T03:07:44.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-11-06T03:07:44.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-11-06T03:07:44.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-11-06T03:07:44.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-11-06T03:07:44.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-11-06T03:07:44.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-11-06T03:07:44.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-11-06T03:07:44.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-11-06T03:07:44.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-11-06T03:07:44.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-06T03:07:44.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.274775+0000', 'last_change': '2023-11-06T03:07:38.274775+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.274775+0000', 'last_undegraded': '2023-11-06T03:07:38.274775+0000', 'last_fullsized': '2023-11-06T03:07:38.274775+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-11-06T03:07:44.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-06T03:07:44.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273492+0000', 'last_change': '2023-11-06T03:07:38.273492+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273492+0000', 'last_undegraded': '2023-11-06T03:07:38.273492+0000', 'last_fullsized': '2023-11-06T03:07:38.273492+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:07:44.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-11-06T03:07:44.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-06T03:07:44.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.272757+0000', 'last_change': '2023-11-06T03:07:38.272757+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.272757+0000', 'last_undegraded': '2023-11-06T03:07:38.272757+0000', 'last_fullsized': '2023-11-06T03:07:38.272757+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:07:44.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-11-06T03:07:44.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-06T03:07:44.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.280184+0000', 'last_change': '2023-11-06T03:07:38.280184+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.280184+0000', 'last_undegraded': '2023-11-06T03:07:38.280184+0000', 'last_fullsized': '2023-11-06T03:07:38.280184+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:07:44.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-11-06T03:07:44.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-06T03:07:44.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273187+0000', 'last_change': '2023-11-06T03:07:38.273187+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273187+0000', 'last_undegraded': '2023-11-06T03:07:38.273187+0000', 'last_fullsized': '2023-11-06T03:07:38.273187+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-11-06T03:07:44.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-06T03:07:44.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.268342+0000', 'last_change': '2023-11-06T03:07:38.268342+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.268342+0000', 'last_undegraded': '2023-11-06T03:07:38.268342+0000', 'last_fullsized': '2023-11-06T03:07:38.268342+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:07:44.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-11-06T03:07:44.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-06T03:07:44.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.274161+0000', 'last_change': '2023-11-06T03:07:38.274161+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.274161+0000', 'last_undegraded': '2023-11-06T03:07:38.274161+0000', 'last_fullsized': '2023-11-06T03:07:38.274161+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-11-06T03:07:44.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-06T03:07:44.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.271731+0000', 'last_change': '2023-11-06T03:07:38.271731+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.271731+0000', 'last_undegraded': '2023-11-06T03:07:38.271731+0000', 'last_fullsized': '2023-11-06T03:07:38.271731+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-11-06T03:07:44.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-06T03:07:44.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.277966+0000', 'last_change': '2023-11-06T03:07:38.277966+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.277966+0000', 'last_undegraded': '2023-11-06T03:07:38.277966+0000', 'last_fullsized': '2023-11-06T03:07:38.277966+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:07:44.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-11-06T03:07:44.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-06T03:07:44.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.268292+0000', 'last_change': '2023-11-06T03:07:38.268292+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.268292+0000', 'last_undegraded': '2023-11-06T03:07:38.268292+0000', 'last_fullsized': '2023-11-06T03:07:38.268292+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:07:44.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-11-06T03:07:44.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-06T03:07:44.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.279319+0000', 'last_change': '2023-11-06T03:07:38.279319+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.279319+0000', 'last_undegraded': '2023-11-06T03:07:38.279319+0000', 'last_fullsized': '2023-11-06T03:07:38.279319+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:07:44.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-11-06T03:07:44.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-06T03:07:44.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.270185+0000', 'last_change': '2023-11-06T03:07:38.270185+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.270185+0000', 'last_undegraded': '2023-11-06T03:07:38.270185+0000', 'last_fullsized': '2023-11-06T03:07:38.270185+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:07:44.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-11-06T03:07:44.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-06T03:07:44.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.272337+0000', 'last_change': '2023-11-06T03:07:38.272337+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.272337+0000', 'last_undegraded': '2023-11-06T03:07:38.272337+0000', 'last_fullsized': '2023-11-06T03:07:38.272337+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-11-06T03:07:44.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-06T03:07:44.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273107+0000', 'last_change': '2023-11-06T03:07:38.273107+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273107+0000', 'last_undegraded': '2023-11-06T03:07:38.273107+0000', 'last_fullsized': '2023-11-06T03:07:38.273107+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-11-06T03:07:44.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-06T03:07:44.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.277799+0000', 'last_change': '2023-11-06T03:07:38.277799+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.277799+0000', 'last_undegraded': '2023-11-06T03:07:38.277799+0000', 'last_fullsized': '2023-11-06T03:07:38.277799+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:07:44.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-11-06T03:07:44.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-06T03:07:44.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.270291+0000', 'last_change': '2023-11-06T03:07:38.270291+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.270291+0000', 'last_undegraded': '2023-11-06T03:07:38.270291+0000', 'last_fullsized': '2023-11-06T03:07:38.270291+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:07:44.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-11-06T03:07:44.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-06T03:07:44.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.271213+0000', 'last_change': '2023-11-06T03:07:38.271213+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.271213+0000', 'last_undegraded': '2023-11-06T03:07:38.271213+0000', 'last_fullsized': '2023-11-06T03:07:38.271213+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-11-06T03:07:44.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-06T03:07:44.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.274204+0000', 'last_change': '2023-11-06T03:07:38.274204+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.274204+0000', 'last_undegraded': '2023-11-06T03:07:38.274204+0000', 'last_fullsized': '2023-11-06T03:07:38.274204+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:07:44.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-11-06T03:07:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-06T03:07:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273197+0000', 'last_change': '2023-11-06T03:07:38.273197+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273197+0000', 'last_undegraded': '2023-11-06T03:07:38.273197+0000', 'last_fullsized': '2023-11-06T03:07:38.273197+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-11-06T03:07:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-06T03:07:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273072+0000', 'last_change': '2023-11-06T03:07:38.273072+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273072+0000', 'last_undegraded': '2023-11-06T03:07:38.273072+0000', 'last_fullsized': '2023-11-06T03:07:38.273072+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-11-06T03:07:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-06T03:07:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273478+0000', 'last_change': '2023-11-06T03:07:38.273478+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273478+0000', 'last_undegraded': '2023-11-06T03:07:38.273478+0000', 'last_fullsized': '2023-11-06T03:07:38.273478+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:07:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-11-06T03:07:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-06T03:07:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.269961+0000', 'last_change': '2023-11-06T03:07:38.269961+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.269961+0000', 'last_undegraded': '2023-11-06T03:07:38.269961+0000', 'last_fullsized': '2023-11-06T03:07:38.269961+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:07:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-11-06T03:07:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-06T03:07:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.270873+0000', 'last_change': '2023-11-06T03:07:38.270873+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.270873+0000', 'last_undegraded': '2023-11-06T03:07:38.270873+0000', 'last_fullsized': '2023-11-06T03:07:38.270873+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-11-06T03:07:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-06T03:07:44.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.274121+0000', 'last_change': '2023-11-06T03:07:38.274121+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.274121+0000', 'last_undegraded': '2023-11-06T03:07:38.274121+0000', 'last_fullsized': '2023-11-06T03:07:38.274121+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:07:44.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-11-06T03:07:44.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-06T03:07:44.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.270776+0000', 'last_change': '2023-11-06T03:07:38.270776+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.270776+0000', 'last_undegraded': '2023-11-06T03:07:38.270776+0000', 'last_fullsized': '2023-11-06T03:07:38.270776+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-11-06T03:07:44.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-06T03:07:44.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.277505+0000', 'last_change': '2023-11-06T03:07:38.277505+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.277505+0000', 'last_undegraded': '2023-11-06T03:07:38.277505+0000', 'last_fullsized': '2023-11-06T03:07:38.277505+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:07:44.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-11-06T03:07:44.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-06T03:07:44.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.272819+0000', 'last_change': '2023-11-06T03:07:38.272819+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.272819+0000', 'last_undegraded': '2023-11-06T03:07:38.272819+0000', 'last_fullsized': '2023-11-06T03:07:38.272819+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:07:44.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-11-06T03:07:44.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-06T03:07:44.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.273572+0000', 'last_change': '2023-11-06T03:07:38.273572+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.273572+0000', 'last_undegraded': '2023-11-06T03:07:38.273572+0000', 'last_fullsized': '2023-11-06T03:07:38.273572+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:07:44.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-11-06T03:07:44.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-06T03:07:44.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.268417+0000', 'last_change': '2023-11-06T03:07:38.268417+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.268417+0000', 'last_undegraded': '2023-11-06T03:07:38.268417+0000', 'last_fullsized': '2023-11-06T03:07:38.268417+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:07:44.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-11-06T03:07:44.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-06T03:07:44.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.281950+0000', 'last_change': '2023-11-06T03:07:38.281950+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.281950+0000', 'last_undegraded': '2023-11-06T03:07:38.281950+0000', 'last_fullsized': '2023-11-06T03:07:38.281950+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:07:44.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-11-06T03:07:44.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-06T03:07:44.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.268511+0000', 'last_change': '2023-11-06T03:07:38.268511+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.268511+0000', 'last_undegraded': '2023-11-06T03:07:38.268511+0000', 'last_fullsized': '2023-11-06T03:07:38.268511+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:07:44.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-11-06T03:07:44.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:07:39.261260+0000', 'last_change': '2023-11-06T03:07:39.261260+0000', 'last_active': '2023-11-06T03:07:39.261260+0000', 'last_peered': '2023-11-06T03:07:39.261260+0000', 'last_clean': '2023-11-06T03:07:39.261260+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:39.261260+0000', 'last_undegraded': '2023-11-06T03:07:39.261260+0000', 'last_fullsized': '2023-11-06T03:07:39.261260+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:39.261260+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:07:44.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-06T03:07:44.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:07:38.268606+0000', 'last_change': '2023-11-06T03:07:38.268606+0000', 'last_active': '2023-11-06T03:07:38.257552+0000', 'last_peered': '2023-11-06T03:07:38.257552+0000', 'last_clean': '2023-11-06T03:07:38.257552+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:07:38.268606+0000', 'last_undegraded': '2023-11-06T03:07:38.268606+0000', 'last_fullsized': '2023-11-06T03:07:38.268606+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'last_clean_scrub_stamp': '2023-11-06T03:07:38.257552+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:07:44.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:07:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 Processor -- start 2023-11-06T03:07:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 -- start start 2023-11-06T03:07:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23f815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f815f580 con 0x7f23f81551e0 2023-11-06T03:07:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f815efc0 con 0x7f23f815e5e0 2023-11-06T03:07:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23ff8b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f80704b0 con 0x7f23f80b3b20 2023-11-06T03:07:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23fd64c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f23f81551e0 0x7f23f806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39758/0 (socket says 172.21.15.157:39758) 2023-11-06T03:07:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23fd64c640 1 -- 172.21.15.157:0/2356221812 learned_addr learned my addr 172.21.15.157:0/2356221812 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23fde4d640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23f815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904819926 0 0) 0x7f23f815f580 con 0x7f23f81551e0 2023-11-06T03:07:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.569+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e4003530 con 0x7f23f81551e0 2023-11-06T03:07:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1218308527 0 0) 0x7f23e4003530 con 0x7f23f81551e0 2023-11-06T03:07:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 msgr2=0x7f23f815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23f815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 >> v1:172.21.15.195:6789/0 conn(0x7f23f815e5e0 legacy=0x7f23f80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f815f2a0 con 0x7f23f81551e0 2023-11-06T03:07:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f23f4001a00 con 0x7f23f81551e0 2023-11-06T03:07:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 >> v1:172.21.15.157:6789/0 conn(0x7f23f81551e0 legacy=0x7f23f806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23e4004d10 0x7f23e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f80704b0 con 0x7f23e4003f80 2023-11-06T03:07:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e400bb80 con 0x7f23e4004d10 2023-11-06T03:07:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e400bd40 con 0x7f23f80b3b20 2023-11-06T03:07:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fde4d640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23e4004d10 0x7f23e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fd64c640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 -- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 msgr2=0x7f23e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f80b3b20 0x7f23e40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 -- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23e4004d10 msgr2=0x7f23e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23e4004d10 0x7f23e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 -- 172.21.15.157:0/2356221812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4003530 con 0x7f23e4003f80 2023-11-06T03:07:44.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fde4d640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23e4004d10 0x7f23e4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23e40047d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f23e8001780 tx=0x7f23e801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913cc3fb13dc7813 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e80052d0 con 0x7f23e4003f80 2023-11-06T03:07:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e80a7050 con 0x7f23e4003f80 2023-11-06T03:07:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23effff640 1 -- 172.21.15.157:0/2356221812 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8003600 con 0x7f23e4003f80 2023-11-06T03:07:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2356221812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 msgr2=0x7f23e40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2356221812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23e40047d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f23e8001780 tx=0x7f23e801c3d0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23fde4d640 1 -- 172.21.15.157:0/2356221812 reap_dead start 2023-11-06T03:07:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2356221812 shutdown_connections 2023-11-06T03:07:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2356221812 >> 172.21.15.157:0/2356221812 conn(0x7f23f8065930 msgr2=0x7f23f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2356221812 shutdown_connections 2023-11-06T03:07:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.573+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2356221812 wait complete. 2023-11-06T03:07:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 Processor -- start 2023-11-06T03:07:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- start start 2023-11-06T03:07:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f814e990 0x7f23f81538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f814cdf0 0x7f23f8153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f814dfc0 con 0x7f23e4003f80 2023-11-06T03:07:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f814d3b0 con 0x7f23f814e990 2023-11-06T03:07:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f814d690 con 0x7f23f814cdf0 2023-11-06T03:07:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50608/0 (socket says 172.21.15.157:50608) 2023-11-06T03:07:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 -- 172.21.15.157:0/2426630223 learned_addr learned my addr 172.21.15.157:0/2426630223 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f814e990 0x7f23f81538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 -- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f814cdf0 msgr2=0x7f23f8153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f814cdf0 0x7f23f8153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 -- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f814e990 msgr2=0x7f23f81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f814e990 0x7f23f81538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 -- 172.21.15.157:0/2426630223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f814d970 con 0x7f23e4003f80 2023-11-06T03:07:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23fd64c640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f23f400de30 tx=0x7f23f400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004b70 con 0x7f23e4003f80 2023-11-06T03:07:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f4004cd0 con 0x7f23e4003f80 2023-11-06T03:07:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f814dc50 con 0x7f23e4003f80 2023-11-06T03:07:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4007310 con 0x7f23e4003f80 2023-11-06T03:07:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f814bae0 con 0x7f23e4003f80 2023-11-06T03:07:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.577+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f8072750 con 0x7f23e4003f80 2023-11-06T03:07:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.581+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23f4007470 con 0x7f23e4003f80 2023-11-06T03:07:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.581+0000 7f23edffb640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 0x7f23dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.581+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23f40835d0 con 0x7f23e4003f80 2023-11-06T03:07:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.581+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 0x7f23dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.581+0000 7f23fce4b640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 0x7f23dc067f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23e8001750 tx=0x7f23e80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.585+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f23f4050ec0 con 0x7f23e4003f80 2023-11-06T03:07:44.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f23f8071730 con 0x7f23e4003f80 2023-11-06T03:07:44.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23edffb640 1 -- 172.21.15.157:0/2426630223 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7f23f40544e0 con 0x7f23e4003f80 2023-11-06T03:07:44.780 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:44.780 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":242,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":53,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":355135488,"bytes_avail":772738977792,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":36,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5868}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-11-06T03:06:19.141124+0000","services":{}},"progress_events":{}} 2023-11-06T03:07:44.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 msgr2=0x7f23dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 0x7f23dc067f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23e8001750 tx=0x7f23e80011a0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 msgr2=0x7f23f814e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f23f400de30 tx=0x7f23f400b4a0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 shutdown_connections 2023-11-06T03:07:44.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23dc065a60 0x7f23dc067f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f814cdf0 0x7f23f8153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f814e990 0x7f23f81538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 --2- 172.21.15.157:0/2426630223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e4003f80 0x7f23f814e450 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 >> 172.21.15.157:0/2426630223 conn(0x7f23f8065930 msgr2=0x7f23f815da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:44.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 shutdown_connections 2023-11-06T03:07:44.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.781+0000 7f23ff8b0640 1 -- 172.21.15.157:0/2426630223 wait complete. 2023-11-06T03:07:44.798 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:07:44.798 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:07:44.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 Processor -- start 2023-11-06T03:07:44.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 -- start start 2023-11-06T03:07:44.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f393015ce80 0x7f39300cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39300d00b0 0x7f39300b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300d2920 con 0x7f393015caa0 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300d0520 con 0x7f39300d00b0 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f39371ef640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300d18a0 con 0x7f393015ce80 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39300d00b0 0x7f39300b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39300d00b0 0x7f39300b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50614/0 (socket says 172.21.15.157:50614) 2023-11-06T03:07:44.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 -- 172.21.15.157:0/3560174025 learned_addr learned my addr 172.21.15.157:0/3560174025 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:44.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f392ffff640 1 --2- 172.21.15.157:0/3560174025 >> v2:172.21.15.195:3300/0 conn(0x7f393015ce80 0x7f39300cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 -- 172.21.15.157:0/3560174025 >> v2:172.21.15.195:3300/0 conn(0x7f393015ce80 msgr2=0x7f39300cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 --2- 172.21.15.157:0/3560174025 >> v2:172.21.15.195:3300/0 conn(0x7f393015ce80 0x7f39300cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 -- 172.21.15.157:0/3560174025 >> v1:172.21.15.195:6789/0 conn(0x7f393015caa0 legacy=0x7f39300cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:44.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.905+0000 7f393578c640 1 -- 172.21.15.157:0/3560174025 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300d1b60 con 0x7f39300d00b0 2023-11-06T03:07:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f393578c640 1 --2- 172.21.15.157:0/3560174025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f39300b5d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f39300c6720 tx=0x7f392801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a7fc7406c094b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f392f7fe640 1 -- 172.21.15.157:0/3560174025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280a5820 con 0x7f39300d00b0 2023-11-06T03:07:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f392f7fe640 1 -- 172.21.15.157:0/3560174025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39280a5980 con 0x7f39300d00b0 2023-11-06T03:07:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f392f7fe640 1 -- 172.21.15.157:0/3560174025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280a5d10 con 0x7f39300d00b0 2023-11-06T03:07:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 -- 172.21.15.157:0/3560174025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 msgr2=0x7f39300b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 --2- 172.21.15.157:0/3560174025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f39300b5d50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f39300c6720 tx=0x7f392801c960 comp rx=0 tx=0).stop 2023-11-06T03:07:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 -- 172.21.15.157:0/3560174025 shutdown_connections 2023-11-06T03:07:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 --2- 172.21.15.157:0/3560174025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f39300b5d50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 --2- 172.21.15.157:0/3560174025 >> v2:172.21.15.195:3300/0 conn(0x7f393015ce80 0x7f39300cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 -- 172.21.15.157:0/3560174025 >> 172.21.15.157:0/3560174025 conn(0x7f3930070790 msgr2=0x7f3930070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:44.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 -- 172.21.15.157:0/3560174025 shutdown_connections 2023-11-06T03:07:44.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.909+0000 7f39371ef640 1 -- 172.21.15.157:0/3560174025 wait complete. 2023-11-06T03:07:44.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 Processor -- start 2023-11-06T03:07:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- start start 2023-11-06T03:07:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 0x7f3930153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 0x7f3930153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39301540b0 con 0x7f39300d00b0 2023-11-06T03:07:44.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cd190 con 0x7f393015caa0 2023-11-06T03:07:44.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cc250 con 0x7f393015ce80 2023-11-06T03:07:44.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50626/0 (socket says 172.21.15.157:50626) 2023-11-06T03:07:44.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 -- 172.21.15.157:0/593655569 learned_addr learned my addr 172.21.15.157:0/593655569 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:44.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392ffff640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 0x7f3930153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f393578c640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 0x7f3930153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 -- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 msgr2=0x7f3930153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 0x7f3930153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 -- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 msgr2=0x7f3930153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 0x7f3930153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 -- 172.21.15.157:0/593655569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928009b30 con 0x7f39300d00b0 2023-11-06T03:07:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392ffff640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 0x7f3930153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f393578c640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 0x7f3930153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f3934f8b640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f392000c490 tx=0x7f3920001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920003600 con 0x7f39300d00b0 2023-11-06T03:07:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920003780 con 0x7f39300d00b0 2023-11-06T03:07:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39300cc530 con 0x7f39300d00b0 2023-11-06T03:07:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920003b70 con 0x7f39300d00b0 2023-11-06T03:07:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3930152550 con 0x7f39300d00b0 2023-11-06T03:07:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39300cc770 con 0x7f39300d00b0 2023-11-06T03:07:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3920003cd0 con 0x7f39300d00b0 2023-11-06T03:07:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 0x7f391c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39200824b0 con 0x7f39300d00b0 2023-11-06T03:07:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.913+0000 7f392ffff640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 0x7f391c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.917+0000 7f392ffff640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 0x7f391c067f20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3924006fd0 tx=0x7f3924008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:44.917+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39300cc770 con 0x7f39300d00b0 2023-11-06T03:07:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.057+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39300b1f00 con 0x7f391c065a60 2023-11-06T03:07:45.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.081+0000 7f392d7fa640 1 -- 172.21.15.157:0/593655569 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299435 (secure 0 0 0) 0x7f392400c040 con 0x7f391c065a60 2023-11-06T03:07:45.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:07:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 msgr2=0x7f391c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 0x7f391c067f20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3924006fd0 tx=0x7f3924008040 comp rx=0 tx=0).stop 2023-11-06T03:07:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 msgr2=0x7f3930152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f392000c490 tx=0x7f3920001e80 comp rx=0 tx=0).stop 2023-11-06T03:07:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 shutdown_connections 2023-11-06T03:07:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f391c065a60 0x7f391c067f20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f393015ce80 0x7f3930153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f393015caa0 0x7f3930153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 --2- 172.21.15.157:0/593655569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39300d00b0 0x7f3930152010 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 >> 172.21.15.157:0/593655569 conn(0x7f3930070790 msgr2=0x7f39300cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 shutdown_connections 2023-11-06T03:07:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:45.085+0000 7f39371ef640 1 -- 172.21.15.157:0/593655569 wait complete. 2023-11-06T03:07:45.101 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2023-11-06T03:07:43.173699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346812,"kb_used_data":5756,"kb_used_omap":0,"kb_used_meta":340992,"kb_avail":754627908,"statfs":{"total":773094113280,"available":772738977792,"internally_reserved":0,"allocated":5894144,"data_stored":3204583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":349175808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.909522"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274775+0000","last_change":"2023-11-06T03:07:38.274775+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274775+0000","last_undegraded":"2023-11-06T03:07:38.274775+0000","last_fullsized":"2023-11-06T03:07:38.274775+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273492+0000","last_change":"2023-11-06T03:07:38.273492+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273492+0000","last_undegraded":"2023-11-06T03:07:38.273492+0000","last_fullsized":"2023-11-06T03:07:38.273492+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272757+0000","last_change":"2023-11-06T03:07:38.272757+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272757+0000","last_undegraded":"2023-11-06T03:07:38.272757+0000","last_fullsized":"2023-11-06T03:07:38.272757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.280184+0000","last_change":"2023-11-06T03:07:38.280184+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.280184+0000","last_undegraded":"2023-11-06T03:07:38.280184+0000","last_fullsized":"2023-11-06T03:07:38.280184+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273187+0000","last_change":"2023-11-06T03:07:38.273187+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273187+0000","last_undegraded":"2023-11-06T03:07:38.273187+0000","last_fullsized":"2023-11-06T03:07:38.273187+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268342+0000","last_change":"2023-11-06T03:07:38.268342+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268342+0000","last_undegraded":"2023-11-06T03:07:38.268342+0000","last_fullsized":"2023-11-06T03:07:38.268342+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274161+0000","last_change":"2023-11-06T03:07:38.274161+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274161+0000","last_undegraded":"2023-11-06T03:07:38.274161+0000","last_fullsized":"2023-11-06T03:07:38.274161+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271731+0000","last_change":"2023-11-06T03:07:38.271731+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271731+0000","last_undegraded":"2023-11-06T03:07:38.271731+0000","last_fullsized":"2023-11-06T03:07:38.271731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277966+0000","last_change":"2023-11-06T03:07:38.277966+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277966+0000","last_undegraded":"2023-11-06T03:07:38.277966+0000","last_fullsized":"2023-11-06T03:07:38.277966+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268292+0000","last_change":"2023-11-06T03:07:38.268292+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268292+0000","last_undegraded":"2023-11-06T03:07:38.268292+0000","last_fullsized":"2023-11-06T03:07:38.268292+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.279319+0000","last_change":"2023-11-06T03:07:38.279319+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.279319+0000","last_undegraded":"2023-11-06T03:07:38.279319+0000","last_fullsized":"2023-11-06T03:07:38.279319+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270185+0000","last_change":"2023-11-06T03:07:38.270185+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270185+0000","last_undegraded":"2023-11-06T03:07:38.270185+0000","last_fullsized":"2023-11-06T03:07:38.270185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272337+0000","last_change":"2023-11-06T03:07:38.272337+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272337+0000","last_undegraded":"2023-11-06T03:07:38.272337+0000","last_fullsized":"2023-11-06T03:07:38.272337+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273107+0000","last_change":"2023-11-06T03:07:38.273107+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273107+0000","last_undegraded":"2023-11-06T03:07:38.273107+0000","last_fullsized":"2023-11-06T03:07:38.273107+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277799+0000","last_change":"2023-11-06T03:07:38.277799+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277799+0000","last_undegraded":"2023-11-06T03:07:38.277799+0000","last_fullsized":"2023-11-06T03:07:38.277799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270291+0000","last_change":"2023-11-06T03:07:38.270291+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270291+0000","last_undegraded":"2023-11-06T03:07:38.270291+0000","last_fullsized":"2023-11-06T03:07:38.270291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.271213+0000","last_change":"2023-11-06T03:07:38.271213+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.271213+0000","last_undegraded":"2023-11-06T03:07:38.271213+0000","last_fullsized":"2023-11-06T03:07:38.271213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274204+0000","last_change":"2023-11-06T03:07:38.274204+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274204+0000","last_undegraded":"2023-11-06T03:07:38.274204+0000","last_fullsized":"2023-11-06T03:07:38.274204+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273197+0000","last_change":"2023-11-06T03:07:38.273197+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273197+0000","last_undegraded":"2023-11-06T03:07:38.273197+0000","last_fullsized":"2023-11-06T03:07:38.273197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273072+0000","last_change":"2023-11-06T03:07:38.273072+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273072+0000","last_undegraded":"2023-11-06T03:07:38.273072+0000","last_fullsized":"2023-11-06T03:07:38.273072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273478+0000","last_change":"2023-11-06T03:07:38.273478+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273478+0000","last_undegraded":"2023-11-06T03:07:38.273478+0000","last_fullsized":"2023-11-06T03:07:38.273478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.269961+0000","last_change":"2023-11-06T03:07:38.269961+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.269961+0000","last_undegraded":"2023-11-06T03:07:38.269961+0000","last_fullsized":"2023-11-06T03:07:38.269961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270873+0000","last_change":"2023-11-06T03:07:38.270873+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270873+0000","last_undegraded":"2023-11-06T03:07:38.270873+0000","last_fullsized":"2023-11-06T03:07:38.270873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.274121+0000","last_change":"2023-11-06T03:07:38.274121+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.274121+0000","last_undegraded":"2023-11-06T03:07:38.274121+0000","last_fullsized":"2023-11-06T03:07:38.274121+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.270776+0000","last_change":"2023-11-06T03:07:38.270776+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.270776+0000","last_undegraded":"2023-11-06T03:07:38.270776+0000","last_fullsized":"2023-11-06T03:07:38.270776+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454639+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:07:35.454639+0000","last_peered":"2023-11-06T03:07:35.454639+0000","last_clean":"2023-11-06T03:07:35.454639+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:07:35.454639+0000","last_undegraded":"2023-11-06T03:07:35.454639+0000","last_fullsized":"2023-11-06T03:07:35.454639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159097+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:07:36.159097+0000","last_peered":"2023-11-06T03:07:36.159097+0000","last_clean":"2023-11-06T03:07:36.159097+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:07:36.159097+0000","last_undegraded":"2023-11-06T03:07:36.159097+0000","last_fullsized":"2023-11-06T03:07:36.159097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.277505+0000","last_change":"2023-11-06T03:07:38.277505+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.277505+0000","last_undegraded":"2023-11-06T03:07:38.277505+0000","last_fullsized":"2023-11-06T03:07:38.277505+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454636+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:07:35.454636+0000","last_peered":"2023-11-06T03:07:35.454636+0000","last_clean":"2023-11-06T03:07:35.454636+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:07:35.454636+0000","last_undegraded":"2023-11-06T03:07:35.454636+0000","last_fullsized":"2023-11-06T03:07:35.454636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.272819+0000","last_change":"2023-11-06T03:07:38.272819+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.272819+0000","last_undegraded":"2023-11-06T03:07:38.272819+0000","last_fullsized":"2023-11-06T03:07:38.272819+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.556066+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:07:36.556066+0000","last_peered":"2023-11-06T03:07:36.556066+0000","last_clean":"2023-11-06T03:07:36.556066+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:07:36.556066+0000","last_undegraded":"2023-11-06T03:07:36.556066+0000","last_fullsized":"2023-11-06T03:07:36.556066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.273572+0000","last_change":"2023-11-06T03:07:38.273572+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.273572+0000","last_undegraded":"2023-11-06T03:07:38.273572+0000","last_fullsized":"2023-11-06T03:07:38.273572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.159137+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:07:36.159137+0000","last_peered":"2023-11-06T03:07:36.159137+0000","last_clean":"2023-11-06T03:07:36.159137+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:07:36.159137+0000","last_undegraded":"2023-11-06T03:07:36.159137+0000","last_fullsized":"2023-11-06T03:07:36.159137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268417+0000","last_change":"2023-11-06T03:07:38.268417+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268417+0000","last_undegraded":"2023-11-06T03:07:38.268417+0000","last_fullsized":"2023-11-06T03:07:38.268417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226174+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:07:36.226174+0000","last_peered":"2023-11-06T03:07:36.226174+0000","last_clean":"2023-11-06T03:07:36.226174+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:07:36.226174+0000","last_undegraded":"2023-11-06T03:07:36.226174+0000","last_fullsized":"2023-11-06T03:07:36.226174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.281950+0000","last_change":"2023-11-06T03:07:38.281950+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.281950+0000","last_undegraded":"2023-11-06T03:07:38.281950+0000","last_fullsized":"2023-11-06T03:07:38.281950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.454684+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:07:35.454684+0000","last_peered":"2023-11-06T03:07:35.454684+0000","last_clean":"2023-11-06T03:07:35.454684+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:07:35.454684+0000","last_undegraded":"2023-11-06T03:07:35.454684+0000","last_fullsized":"2023-11-06T03:07:35.454684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268511+0000","last_change":"2023-11-06T03:07:38.268511+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268511+0000","last_undegraded":"2023-11-06T03:07:38.268511+0000","last_fullsized":"2023-11-06T03:07:38.268511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:36.226199+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:07:36.226199+0000","last_peered":"2023-11-06T03:07:36.226199+0000","last_clean":"2023-11-06T03:07:36.226199+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:07:36.226199+0000","last_undegraded":"2023-11-06T03:07:36.226199+0000","last_fullsized":"2023-11-06T03:07:36.226199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:07:39.261260+0000","last_change":"2023-11-06T03:07:39.261260+0000","last_active":"2023-11-06T03:07:39.261260+0000","last_peered":"2023-11-06T03:07:39.261260+0000","last_clean":"2023-11-06T03:07:39.261260+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:39.261260+0000","last_undegraded":"2023-11-06T03:07:39.261260+0000","last_fullsized":"2023-11-06T03:07:39.261260+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2023-11-06T03:07:38.268606+0000","last_change":"2023-11-06T03:07:38.268606+0000","last_active":"2023-11-06T03:07:38.257552+0000","last_peered":"2023-11-06T03:07:38.257552+0000","last_clean":"2023-11-06T03:07:38.257552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:07:38.268606+0000","last_undegraded":"2023-11-06T03:07:38.268606+0000","last_fullsized":"2023-11-06T03:07:38.268606+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-06T03:07:35.544981+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:07:35.544981+0000","last_peered":"2023-11-06T03:07:35.544981+0000","last_clean":"2023-11-06T03:07:35.544981+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:07:35.544981+0000","last_undegraded":"2023-11-06T03:07:35.544981+0000","last_fullsized":"2023-11-06T03:07:35.544981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50252,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321588,"statfs":{"total":96636764160,"available":96585306112,"internally_reserved":0,"allocated":1183744,"data_stored":843731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.397,"15min":1.397},"last":0.73799999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.872},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1739999999999999,"5min":1.444,"15min":1.444},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.627}]},{"osd":2,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93300000000000005,"15min":0.97099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3180000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.151}]},{"osd":3,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.111,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.016},{"interface":"front","average":{"1min":0.995,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.387,"5min":1.857,"15min":1.857},"last":1.103}]},{"osd":4,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.65000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.298,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.022},"min":{"1min":0.63400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1200000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3140000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.284},{"interface":"front","average":{"1min":0.999,"5min":1.069,"15min":1.099},"min":{"1min":0.66800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3700000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3120000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.80300000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6850000000000001,"5min":1.774,"15min":1.774},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.77300000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.335,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.367}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43908,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327932,"statfs":{"total":96636764160,"available":96591802368,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.226,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2210000000000001,"5min":2.073,"15min":2.073},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.411,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.036},{"interface":"front","average":{"1min":0.872,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.63800000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3600000000000001,"5min":1.778,"15min":1.778},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.167},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.329,"5min":1.806,"15min":1.806},"last":0.69599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3819999999999999,"5min":1.857,"15min":1.857},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.89700000000000002,"15min":0.88900000000000001},"min":{"1min":0.67800000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.117},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.294,"5min":2.931,"15min":2.931},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.137,"15min":1.163},"min":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.3100000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.036},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.429,"5min":1.506,"15min":1.506},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4750000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.093,"15min":1.105},"min":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4279999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.083,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3440000000000001,"5min":1.415,"15min":1.415},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43912,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327928,"statfs":{"total":96636764160,"available":96591798272,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2290000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.748,"15min":0.71499999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.018,"5min":1.2,"15min":1.2},"last":0.62},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2649999999999999,"5min":1.357,"15min":1.357},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.264,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.98499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":1.403,"15min":1.403},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93400000000000005,"15min":0.93100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2170000000000001,"5min":1.429,"15min":1.429},"last":1.1180000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":1.079},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1479999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.004,"15min":0.99099999999999999},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":1.248,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.999,"15min":1.0069999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.296,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2649999999999999,"5min":1.353,"15min":1.353},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.282,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2310000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49672,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322168,"statfs":{"total":96636764160,"available":96585900032,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.46999999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.0740000000000001},"min":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5820000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.60499999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0600000000000001,"5min":1.329,"15min":1.329},"last":1.042}]},{"osd":3,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1659999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.169},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.105}]},{"osd":5,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0129999999999999,"15min":1.0189999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.256},{"interface":"front","average":{"1min":1.006,"5min":1.087,"15min":1.111},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1990000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.012,"15min":1.0289999999999999},"min":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.1299999999999999,"15min":1.1659999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8120000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.357}]},{"osd":7,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.2549999999999999,"15min":1.1910000000000001},"min":{"1min":0.91000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":13.582000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2090000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49668,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322172,"statfs":{"total":96636764160,"available":96585904128,"internally_reserved":0,"allocated":585728,"data_stored":252344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.151,"5min":1.524,"15min":1.524},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94099999999999995,"5min":1.353,"15min":1.353},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1970000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1850000000000001,"5min":1.349,"15min":1.349},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":1.327,"15min":1.327},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93500000000000005,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0960000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97099999999999997,"15min":0.99199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1819999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":1.698,"15min":1.698},"last":0.88900000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97699999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4610000000000001,"5min":1.544,"15min":1.544},"last":1.0029999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5249999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.044,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.22,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0309999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.215,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.64600000000000002},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0940000000000001,"5min":1.121,"15min":1.121},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.67200000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.0249999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.75600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.64800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.92600000000000005,"15min":0.96399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2150000000000001,"5min":1.341,"15min":1.341},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86499999999999999,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.111,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93899999999999995,"15min":0.96999999999999997},"min":{"1min":0.73899999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5509999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.155}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.111},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.95299999999999996},"min":{"1min":0.70099999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1850000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0289999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6379999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95399999999999996,"15min":0.96999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.96099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":589824,"data_stored":253363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.19,"15min":0.19},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.1749999999999998,"5min":0.91000000000000003,"15min":0.72099999999999997},"min":{"1min":0.29999999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":30.861000000000001,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.312,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.0449999999999999,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":14.753,"5min":14.753,"15min":14.753},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":3.306,"5min":1.4339999999999999,"15min":1.1539999999999999},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.103999999999999,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":3.0459999999999998,"5min":1.409,"15min":1.181},"min":{"1min":0.53100000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":44.753999999999998,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.2230000000000001,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":23.971,"5min":23.971,"15min":23.971},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.1469999999999998,"5min":1.2410000000000001,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.503,"15min":0.503},"max":{"1min":14.813000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.089},"min":{"1min":0.65400000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0589999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54344,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317496,"statfs":{"total":96636764160,"available":96581115904,"internally_reserved":0,"allocated":1179648,"data_stored":842712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.27000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.70599999999999996,"15min":0.65400000000000003},"min":{"1min":0.377,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":7.1230000000000002,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.71999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.40500000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.80600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.96199999999999997,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.94799999999999995},"min":{"1min":0.62,"5min":0.432,"15min":0.432},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.2230000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.238,"5min":0.98599999999999999,"15min":0.94799999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.7430000000000003,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.3}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.92000000000000004,"15min":0.90400000000000003},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":1.143},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0269999999999999,"15min":1.0169999999999999},"min":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.177}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0469999999999999,"15min":1.0149999999999999},"min":{"1min":0.65500000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0009999999999999,"15min":0.997},"min":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:07:45.407 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:07:45.409+0000 7f0797ba6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:07:48.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:07:48.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 Processor -- start 2023-11-06T03:07:48.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 -- start start 2023-11-06T03:07:48.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdfc01551e0 0x7fdfc006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfc015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc015f580 con 0x7fdfc015e5e0 2023-11-06T03:07:48.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc015efc0 con 0x7fdfc01551e0 2023-11-06T03:07:48.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfc5a3d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc015f2a0 con 0x7fdfc00b3b20 2023-11-06T03:07:48.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbe7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdfc015e5e0 0x7fdfc00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:39770/0 (socket says 172.21.15.157:39770) 2023-11-06T03:07:48.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbe7fc640 1 -- 172.21.15.157:0/1088509179 learned_addr learned my addr 172.21.15.157:0/1088509179 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:48.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbeffd640 1 --2- 172.21.15.157:0/1088509179 >> v2:172.21.15.195:3300/0 conn(0x7fdfc01551e0 0x7fdfc006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844599468 0 0) 0x7fdfc015f580 con 0x7fdfc015e5e0 2023-11-06T03:07:48.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfc015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa8003530 con 0x7fdfc015e5e0 2023-11-06T03:07:48.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1761995329 0 0) 0x7fdfa8003530 con 0x7fdfc015e5e0 2023-11-06T03:07:48.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 msgr2=0x7fdfc015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfc015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 >> v2:172.21.15.195:3300/0 conn(0x7fdfc01551e0 msgr2=0x7fdfc006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 --2- 172.21.15.157:0/1088509179 >> v2:172.21.15.195:3300/0 conn(0x7fdfc01551e0 0x7fdfc006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc006f5a0 con 0x7fdfc015e5e0 2023-11-06T03:07:48.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdfb4001680 con 0x7fdfc015e5e0 2023-11-06T03:07:48.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 >> v1:172.21.15.157:6789/0 conn(0x7fdfc015e5e0 legacy=0x7fdfc00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfa8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8003f00 0x7fdfa80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa8004d10 0x7fdfa8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc015efc0 con 0x7fdfa8004d10 2023-11-06T03:07:48.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc015f2a0 con 0x7fdfa8003f00 2023-11-06T03:07:48.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa800b9e0 con 0x7fdfc00b3b20 2023-11-06T03:07:48.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa8004d10 0x7fdfa8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbeffd640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfa8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbe7fc640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8003f00 0x7fdfa80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 msgr2=0x7fdfa8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc00b3b20 0x7fdfa8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8003f00 msgr2=0x7fdfa80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8003f00 0x7fdfa80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.241+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/1088509179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8003530 con 0x7fdfa8004d10 2023-11-06T03:07:48.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa8004d10 0x7fdfa8005190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009fc0 tx=0x7fdfb801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d7f62ba9671ac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80a5b20 con 0x7fdfa8004d10 2023-11-06T03:07:48.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb800f040 con 0x7fdfa8004d10 2023-11-06T03:07:48.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfbdffb640 1 -- 172.21.15.157:0/1088509179 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80ad440 con 0x7fdfa8004d10 2023-11-06T03:07:48.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/1088509179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa8004d10 msgr2=0x7fdfa8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/1088509179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa8004d10 0x7fdfa8005190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009fc0 tx=0x7fdfb801cc20 comp rx=0 tx=0).stop 2023-11-06T03:07:48.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/1088509179 reap_dead start 2023-11-06T03:07:48.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/1088509179 shutdown_connections 2023-11-06T03:07:48.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/1088509179 >> 172.21.15.157:0/1088509179 conn(0x7fdfc0065930 msgr2=0x7fdfc0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:48.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/1088509179 shutdown_connections 2023-11-06T03:07:48.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.245+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/1088509179 wait complete. 2023-11-06T03:07:48.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 Processor -- start 2023-11-06T03:07:48.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- start start 2023-11-06T03:07:48.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8004d10 0x7fdfc0151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 0x7fdfc01e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01559a0 con 0x7fdfc0157440 2023-11-06T03:07:48.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0157e10 con 0x7fdfa8004d10 2023-11-06T03:07:48.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01e57c0 con 0x7fdfc0152450 2023-11-06T03:07:48.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50646/0 (socket says 172.21.15.157:50646) 2023-11-06T03:07:48.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/2480216428 learned_addr learned my addr 172.21.15.157:0/2480216428 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:48.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbeffd640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8004d10 0x7fdfc0151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbe7fc640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 0x7fdfc01e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 msgr2=0x7fdfc01e4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 0x7fdfc01e4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8004d10 msgr2=0x7fdfc0151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8004d10 0x7fdfc0151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 -- 172.21.15.157:0/2480216428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc01e5aa0 con 0x7fdfc0157440 2023-11-06T03:07:48.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbe7fc640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 0x7fdfc01e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:48.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfbf7fe640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8002240 tx=0x7fdfb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80039a0 con 0x7fdfc0157440 2023-11-06T03:07:48.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb8003b00 con 0x7fdfc0157440 2023-11-06T03:07:48.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc01e5d80 con 0x7fdfc0157440 2023-11-06T03:07:48.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8003c70 con 0x7fdfc0157440 2023-11-06T03:07:48.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc0152110 con 0x7fdfc0157440 2023-11-06T03:07:48.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc00ae3f0 con 0x7fdfc0157440 2023-11-06T03:07:48.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdfb80b4040 con 0x7fdfc0157440 2023-11-06T03:07:48.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 0x7fdfac067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.249+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfb811d1e0 con 0x7fdfc0157440 2023-11-06T03:07:48.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.253+0000 7fdfbeffd640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 0x7fdfac067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.253+0000 7fdfbeffd640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 0x7fdfac067f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ab0 tx=0x7fdfb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.257+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdfb80eaae0 con 0x7fdfc0157440 2023-11-06T03:07:48.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.405+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfc006f880 con 0x7fdfac065a90 2023-11-06T03:07:48.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.429+0000 7fdfa7fff640 1 -- 172.21.15.157:0/2480216428 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310114 (secure 0 0 0) 0x7fdfb0013a20 con 0x7fdfac065a90 2023-11-06T03:07:48.433 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:48.435 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:07:48.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 msgr2=0x7fdfac067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 0x7fdfac067f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ab0 tx=0x7fdfb0008040 comp rx=0 tx=0).stop 2023-11-06T03:07:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 msgr2=0x7fdfc01578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8002240 tx=0x7fdfb8009750 comp rx=0 tx=0).stop 2023-11-06T03:07:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 shutdown_connections 2023-11-06T03:07:48.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdfac065a90 0x7fdfac067f50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc0157440 0x7fdfc01578d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdfc0152450 0x7fdfc01e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 --2- 172.21.15.157:0/2480216428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdfa8004d10 0x7fdfc0151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 >> 172.21.15.157:0/2480216428 conn(0x7fdfc0065930 msgr2=0x7fdfc00cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:48.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 shutdown_connections 2023-11-06T03:07:48.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.433+0000 7fdfc5a3d640 1 -- 172.21.15.157:0/2480216428 wait complete. 2023-11-06T03:07:48.450 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-06T03:07:47.175217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347164,"kb_used_data":6108,"kb_used_omap":0,"kb_used_meta":340992,"kb_avail":754627556,"statfs":{"total":773094113280,"available":772738617344,"internally_reserved":0,"allocated":6254592,"data_stored":3372579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":349175808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.905846"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.662182+0000","last_change":"2023-11-06T03:07:39.666950+0000","last_active":"2023-11-06T03:07:40.662182+0000","last_peered":"2023-11-06T03:07:40.662182+0000","last_clean":"2023-11-06T03:07:40.662182+0000","last_became_active":"2023-11-06T03:07:39.666702+0000","last_became_peered":"2023-11-06T03:07:39.666702+0000","last_unstale":"2023-11-06T03:07:40.662182+0000","last_undegraded":"2023-11-06T03:07:40.662182+0000","last_fullsized":"2023-11-06T03:07:40.662182+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:45.590048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061442+0000","last_change":"2023-11-06T03:07:39.708756+0000","last_active":"2023-11-06T03:07:42.061442+0000","last_peered":"2023-11-06T03:07:42.061442+0000","last_clean":"2023-11-06T03:07:42.061442+0000","last_became_active":"2023-11-06T03:07:39.708022+0000","last_became_peered":"2023-11-06T03:07:39.708022+0000","last_unstale":"2023-11-06T03:07:42.061442+0000","last_undegraded":"2023-11-06T03:07:42.061442+0000","last_fullsized":"2023-11-06T03:07:42.061442+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:47.703903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060672+0000","last_change":"2023-11-06T03:07:39.670365+0000","last_active":"2023-11-06T03:07:42.060672+0000","last_peered":"2023-11-06T03:07:42.060672+0000","last_clean":"2023-11-06T03:07:42.060672+0000","last_became_active":"2023-11-06T03:07:39.669721+0000","last_became_peered":"2023-11-06T03:07:39.669721+0000","last_unstale":"2023-11-06T03:07:42.060672+0000","last_undegraded":"2023-11-06T03:07:42.060672+0000","last_fullsized":"2023-11-06T03:07:42.060672+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:30.386230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061139+0000","last_change":"2023-11-06T03:07:39.707773+0000","last_active":"2023-11-06T03:07:42.061139+0000","last_peered":"2023-11-06T03:07:42.061139+0000","last_clean":"2023-11-06T03:07:42.061139+0000","last_became_active":"2023-11-06T03:07:39.707147+0000","last_became_peered":"2023-11-06T03:07:39.707147+0000","last_unstale":"2023-11-06T03:07:42.061139+0000","last_undegraded":"2023-11-06T03:07:42.061139+0000","last_fullsized":"2023-11-06T03:07:42.061139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:45.590048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060052+0000","last_change":"2023-11-06T03:07:39.710911+0000","last_active":"2023-11-06T03:07:42.060052+0000","last_peered":"2023-11-06T03:07:42.060052+0000","last_clean":"2023-11-06T03:07:42.060052+0000","last_became_active":"2023-11-06T03:07:39.710336+0000","last_became_peered":"2023-11-06T03:07:39.710336+0000","last_unstale":"2023-11-06T03:07:42.060052+0000","last_undegraded":"2023-11-06T03:07:42.060052+0000","last_fullsized":"2023-11-06T03:07:42.060052+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:47.703903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.661669+0000","last_change":"2023-11-06T03:07:39.697127+0000","last_active":"2023-11-06T03:07:40.661669+0000","last_peered":"2023-11-06T03:07:40.661669+0000","last_clean":"2023-11-06T03:07:40.661669+0000","last_became_active":"2023-11-06T03:07:39.696869+0000","last_became_peered":"2023-11-06T03:07:39.696869+0000","last_unstale":"2023-11-06T03:07:40.661669+0000","last_undegraded":"2023-11-06T03:07:40.661669+0000","last_fullsized":"2023-11-06T03:07:40.661669+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:48:16.133682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061121+0000","last_change":"2023-11-06T03:07:39.672057+0000","last_active":"2023-11-06T03:07:42.061121+0000","last_peered":"2023-11-06T03:07:42.061121+0000","last_clean":"2023-11-06T03:07:42.061121+0000","last_became_active":"2023-11-06T03:07:39.670044+0000","last_became_peered":"2023-11-06T03:07:39.670044+0000","last_unstale":"2023-11-06T03:07:42.061121+0000","last_undegraded":"2023-11-06T03:07:42.061121+0000","last_fullsized":"2023-11-06T03:07:42.061121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:30.266620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.058995+0000","last_change":"2023-11-06T03:07:39.676095+0000","last_active":"2023-11-06T03:07:42.058995+0000","last_peered":"2023-11-06T03:07:42.058995+0000","last_clean":"2023-11-06T03:07:42.058995+0000","last_became_active":"2023-11-06T03:07:39.675796+0000","last_became_peered":"2023-11-06T03:07:39.675796+0000","last_unstale":"2023-11-06T03:07:42.058995+0000","last_undegraded":"2023-11-06T03:07:42.058995+0000","last_fullsized":"2023-11-06T03:07:42.058995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:02:02.926039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060960+0000","last_change":"2023-11-06T03:07:39.684722+0000","last_active":"2023-11-06T03:07:42.060960+0000","last_peered":"2023-11-06T03:07:42.060960+0000","last_clean":"2023-11-06T03:07:42.060960+0000","last_became_active":"2023-11-06T03:07:39.681999+0000","last_became_peered":"2023-11-06T03:07:39.681999+0000","last_unstale":"2023-11-06T03:07:42.060960+0000","last_undegraded":"2023-11-06T03:07:42.060960+0000","last_fullsized":"2023-11-06T03:07:42.060960+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:41.255106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.661866+0000","last_change":"2023-11-06T03:07:39.699558+0000","last_active":"2023-11-06T03:07:40.661866+0000","last_peered":"2023-11-06T03:07:40.661866+0000","last_clean":"2023-11-06T03:07:40.661866+0000","last_became_active":"2023-11-06T03:07:39.699232+0000","last_became_peered":"2023-11-06T03:07:39.699232+0000","last_unstale":"2023-11-06T03:07:40.661866+0000","last_undegraded":"2023-11-06T03:07:40.661866+0000","last_fullsized":"2023-11-06T03:07:40.661866+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:19.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.617451+0000","last_change":"2023-11-06T03:07:39.697802+0000","last_active":"2023-11-06T03:07:41.617451+0000","last_peered":"2023-11-06T03:07:41.617451+0000","last_clean":"2023-11-06T03:07:41.617451+0000","last_became_active":"2023-11-06T03:07:39.697541+0000","last_became_peered":"2023-11-06T03:07:39.697541+0000","last_unstale":"2023-11-06T03:07:41.617451+0000","last_undegraded":"2023-11-06T03:07:41.617451+0000","last_fullsized":"2023-11-06T03:07:41.617451+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:01:19.220568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059633+0000","last_change":"2023-11-06T03:07:39.686060+0000","last_active":"2023-11-06T03:07:42.059633+0000","last_peered":"2023-11-06T03:07:42.059633+0000","last_clean":"2023-11-06T03:07:42.059633+0000","last_became_active":"2023-11-06T03:07:39.685716+0000","last_became_peered":"2023-11-06T03:07:39.685716+0000","last_unstale":"2023-11-06T03:07:42.059633+0000","last_undegraded":"2023-11-06T03:07:42.059633+0000","last_fullsized":"2023-11-06T03:07:42.059633+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:46.662034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.661724+0000","last_change":"2023-11-06T03:07:39.700011+0000","last_active":"2023-11-06T03:07:40.661724+0000","last_peered":"2023-11-06T03:07:40.661724+0000","last_clean":"2023-11-06T03:07:40.661724+0000","last_became_active":"2023-11-06T03:07:39.699725+0000","last_became_peered":"2023-11-06T03:07:39.699725+0000","last_unstale":"2023-11-06T03:07:40.661724+0000","last_undegraded":"2023-11-06T03:07:40.661724+0000","last_fullsized":"2023-11-06T03:07:40.661724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:29:31.845970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060713+0000","last_change":"2023-11-06T03:07:39.671046+0000","last_active":"2023-11-06T03:07:42.060713+0000","last_peered":"2023-11-06T03:07:42.060713+0000","last_clean":"2023-11-06T03:07:42.060713+0000","last_became_active":"2023-11-06T03:07:39.670786+0000","last_became_peered":"2023-11-06T03:07:39.670786+0000","last_unstale":"2023-11-06T03:07:42.060713+0000","last_undegraded":"2023-11-06T03:07:42.060713+0000","last_fullsized":"2023-11-06T03:07:42.060713+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:54:30.601959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060993+0000","last_change":"2023-11-06T03:07:39.710998+0000","last_active":"2023-11-06T03:07:42.060993+0000","last_peered":"2023-11-06T03:07:42.060993+0000","last_clean":"2023-11-06T03:07:42.060993+0000","last_became_active":"2023-11-06T03:07:39.706834+0000","last_became_peered":"2023-11-06T03:07:39.706834+0000","last_unstale":"2023-11-06T03:07:42.060993+0000","last_undegraded":"2023-11-06T03:07:42.060993+0000","last_fullsized":"2023-11-06T03:07:42.060993+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:29:31.845970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.661699+0000","last_change":"2023-11-06T03:07:39.693799+0000","last_active":"2023-11-06T03:07:40.661699+0000","last_peered":"2023-11-06T03:07:40.661699+0000","last_clean":"2023-11-06T03:07:40.661699+0000","last_became_active":"2023-11-06T03:07:39.693378+0000","last_became_peered":"2023-11-06T03:07:39.693378+0000","last_unstale":"2023-11-06T03:07:40.661699+0000","last_undegraded":"2023-11-06T03:07:40.661699+0000","last_fullsized":"2023-11-06T03:07:40.661699+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:39.409325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060553+0000","last_change":"2023-11-06T03:07:39.686792+0000","last_active":"2023-11-06T03:07:42.060553+0000","last_peered":"2023-11-06T03:07:42.060553+0000","last_clean":"2023-11-06T03:07:42.060553+0000","last_became_active":"2023-11-06T03:07:39.686515+0000","last_became_peered":"2023-11-06T03:07:39.686515+0000","last_unstale":"2023-11-06T03:07:42.060553+0000","last_undegraded":"2023-11-06T03:07:42.060553+0000","last_fullsized":"2023-11-06T03:07:42.060553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:27:12.631692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061708+0000","last_change":"2023-11-06T03:07:39.711225+0000","last_active":"2023-11-06T03:07:42.061708+0000","last_peered":"2023-11-06T03:07:42.061708+0000","last_clean":"2023-11-06T03:07:42.061708+0000","last_became_active":"2023-11-06T03:07:39.708470+0000","last_became_peered":"2023-11-06T03:07:39.708470+0000","last_unstale":"2023-11-06T03:07:42.061708+0000","last_undegraded":"2023-11-06T03:07:42.061708+0000","last_fullsized":"2023-11-06T03:07:42.061708+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:33.550315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060667+0000","last_change":"2023-11-06T03:07:39.705329+0000","last_active":"2023-11-06T03:07:42.060667+0000","last_peered":"2023-11-06T03:07:42.060667+0000","last_clean":"2023-11-06T03:07:42.060667+0000","last_became_active":"2023-11-06T03:07:39.704892+0000","last_became_peered":"2023-11-06T03:07:39.704892+0000","last_unstale":"2023-11-06T03:07:42.060667+0000","last_undegraded":"2023-11-06T03:07:42.060667+0000","last_fullsized":"2023-11-06T03:07:42.060667+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:37.862975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.947134+0000","last_change":"2023-11-06T03:07:39.706130+0000","last_active":"2023-11-06T03:07:41.947134+0000","last_peered":"2023-11-06T03:07:41.947134+0000","last_clean":"2023-11-06T03:07:41.947134+0000","last_became_active":"2023-11-06T03:07:39.705386+0000","last_became_peered":"2023-11-06T03:07:39.705386+0000","last_unstale":"2023-11-06T03:07:41.947134+0000","last_undegraded":"2023-11-06T03:07:41.947134+0000","last_fullsized":"2023-11-06T03:07:41.947134+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:36.699737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.947092+0000","last_change":"2023-11-06T03:07:39.710112+0000","last_active":"2023-11-06T03:07:41.947092+0000","last_peered":"2023-11-06T03:07:41.947092+0000","last_clean":"2023-11-06T03:07:41.947092+0000","last_became_active":"2023-11-06T03:07:39.709472+0000","last_became_peered":"2023-11-06T03:07:39.709472+0000","last_unstale":"2023-11-06T03:07:41.947092+0000","last_undegraded":"2023-11-06T03:07:41.947092+0000","last_fullsized":"2023-11-06T03:07:41.947092+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:09:33.854352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.058888+0000","last_change":"2023-11-06T03:07:39.675579+0000","last_active":"2023-11-06T03:07:42.058888+0000","last_peered":"2023-11-06T03:07:42.058888+0000","last_clean":"2023-11-06T03:07:42.058888+0000","last_became_active":"2023-11-06T03:07:39.673359+0000","last_became_peered":"2023-11-06T03:07:39.673359+0000","last_unstale":"2023-11-06T03:07:42.058888+0000","last_undegraded":"2023-11-06T03:07:42.058888+0000","last_fullsized":"2023-11-06T03:07:42.058888+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:23.384269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.616847+0000","last_change":"2023-11-06T03:07:39.697303+0000","last_active":"2023-11-06T03:07:41.616847+0000","last_peered":"2023-11-06T03:07:41.616847+0000","last_clean":"2023-11-06T03:07:41.616847+0000","last_became_active":"2023-11-06T03:07:39.697011+0000","last_became_peered":"2023-11-06T03:07:39.697011+0000","last_unstale":"2023-11-06T03:07:41.616847+0000","last_undegraded":"2023-11-06T03:07:41.616847+0000","last_fullsized":"2023-11-06T03:07:41.616847+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:21:59.326576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059908+0000","last_change":"2023-11-06T03:07:39.671329+0000","last_active":"2023-11-06T03:07:42.059908+0000","last_peered":"2023-11-06T03:07:42.059908+0000","last_clean":"2023-11-06T03:07:42.059908+0000","last_became_active":"2023-11-06T03:07:39.671167+0000","last_became_peered":"2023-11-06T03:07:39.671167+0000","last_unstale":"2023-11-06T03:07:42.059908+0000","last_undegraded":"2023-11-06T03:07:42.059908+0000","last_fullsized":"2023-11-06T03:07:42.059908+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:26.789970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060690+0000","last_change":"2023-11-06T03:07:39.678756+0000","last_active":"2023-11-06T03:07:42.060690+0000","last_peered":"2023-11-06T03:07:42.060690+0000","last_clean":"2023-11-06T03:07:42.060690+0000","last_became_active":"2023-11-06T03:07:39.678188+0000","last_became_peered":"2023-11-06T03:07:39.678188+0000","last_unstale":"2023-11-06T03:07:42.060690+0000","last_undegraded":"2023-11-06T03:07:42.060690+0000","last_fullsized":"2023-11-06T03:07:42.060690+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:45.199135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059372+0000","last_change":"2023-11-06T03:07:39.674738+0000","last_active":"2023-11-06T03:07:42.059372+0000","last_peered":"2023-11-06T03:07:42.059372+0000","last_clean":"2023-11-06T03:07:42.059372+0000","last_became_active":"2023-11-06T03:07:39.674470+0000","last_became_peered":"2023-11-06T03:07:39.674470+0000","last_unstale":"2023-11-06T03:07:42.059372+0000","last_undegraded":"2023-11-06T03:07:42.059372+0000","last_fullsized":"2023-11-06T03:07:42.059372+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:18.025337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.662135+0000","last_change":"2023-11-06T03:07:39.667940+0000","last_active":"2023-11-06T03:07:40.662135+0000","last_peered":"2023-11-06T03:07:40.662135+0000","last_clean":"2023-11-06T03:07:40.662135+0000","last_became_active":"2023-11-06T03:07:39.667702+0000","last_became_peered":"2023-11-06T03:07:39.667702+0000","last_unstale":"2023-11-06T03:07:40.662135+0000","last_undegraded":"2023-11-06T03:07:40.662135+0000","last_fullsized":"2023-11-06T03:07:40.662135+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:33.550315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.947431+0000","last_change":"2023-11-06T03:07:39.679213+0000","last_active":"2023-11-06T03:07:41.947431+0000","last_peered":"2023-11-06T03:07:41.947431+0000","last_clean":"2023-11-06T03:07:41.947431+0000","last_became_active":"2023-11-06T03:07:39.678429+0000","last_became_peered":"2023-11-06T03:07:39.678429+0000","last_unstale":"2023-11-06T03:07:41.947431+0000","last_undegraded":"2023-11-06T03:07:41.947431+0000","last_fullsized":"2023-11-06T03:07:41.947431+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:40.333246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060259+0000","last_change":"2023-11-06T03:07:39.683029+0000","last_active":"2023-11-06T03:07:42.060259+0000","last_peered":"2023-11-06T03:07:42.060259+0000","last_clean":"2023-11-06T03:07:42.060259+0000","last_became_active":"2023-11-06T03:07:39.682765+0000","last_became_peered":"2023-11-06T03:07:39.682765+0000","last_unstale":"2023-11-06T03:07:42.060259+0000","last_undegraded":"2023-11-06T03:07:42.060259+0000","last_fullsized":"2023-11-06T03:07:42.060259+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:08.274187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061243+0000","last_change":"2023-11-06T03:07:39.708164+0000","last_active":"2023-11-06T03:07:42.061243+0000","last_peered":"2023-11-06T03:07:42.061243+0000","last_clean":"2023-11-06T03:07:42.061243+0000","last_became_active":"2023-11-06T03:07:39.707311+0000","last_became_peered":"2023-11-06T03:07:39.707311+0000","last_unstale":"2023-11-06T03:07:42.061243+0000","last_undegraded":"2023-11-06T03:07:42.061243+0000","last_fullsized":"2023-11-06T03:07:42.061243+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:11:05.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060231+0000","last_change":"2023-11-06T03:07:39.672330+0000","last_active":"2023-11-06T03:07:42.060231+0000","last_peered":"2023-11-06T03:07:42.060231+0000","last_clean":"2023-11-06T03:07:42.060231+0000","last_became_active":"2023-11-06T03:07:39.671798+0000","last_became_peered":"2023-11-06T03:07:39.671798+0000","last_unstale":"2023-11-06T03:07:42.060231+0000","last_undegraded":"2023-11-06T03:07:42.060231+0000","last_fullsized":"2023-11-06T03:07:42.060231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:44.434772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.665892+0000","last_change":"2023-11-06T03:07:39.678538+0000","last_active":"2023-11-06T03:07:40.665892+0000","last_peered":"2023-11-06T03:07:40.665892+0000","last_clean":"2023-11-06T03:07:40.665892+0000","last_became_active":"2023-11-06T03:07:39.677353+0000","last_became_peered":"2023-11-06T03:07:39.677353+0000","last_unstale":"2023-11-06T03:07:40.665892+0000","last_undegraded":"2023-11-06T03:07:40.665892+0000","last_fullsized":"2023-11-06T03:07:40.665892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:48:16.133682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059844+0000","last_change":"2023-11-06T03:07:39.698375+0000","last_active":"2023-11-06T03:07:42.059844+0000","last_peered":"2023-11-06T03:07:42.059844+0000","last_clean":"2023-11-06T03:07:42.059844+0000","last_became_active":"2023-11-06T03:07:39.697422+0000","last_became_peered":"2023-11-06T03:07:39.697422+0000","last_unstale":"2023-11-06T03:07:42.059844+0000","last_undegraded":"2023-11-06T03:07:42.059844+0000","last_fullsized":"2023-11-06T03:07:42.059844+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:23:09.054772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.705800+0000","last_change":"2023-11-06T03:07:39.299875+0000","last_active":"2023-11-06T03:07:39.705800+0000","last_peered":"2023-11-06T03:07:39.705800+0000","last_clean":"2023-11-06T03:07:39.705800+0000","last_became_active":"2023-11-06T03:07:39.299198+0000","last_became_peered":"2023-11-06T03:07:39.299198+0000","last_unstale":"2023-11-06T03:07:39.705800+0000","last_undegraded":"2023-11-06T03:07:39.705800+0000","last_fullsized":"2023-11-06T03:07:39.705800+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:02.148491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.666082+0000","last_change":"2023-11-06T03:07:39.699020+0000","last_active":"2023-11-06T03:07:40.666082+0000","last_peered":"2023-11-06T03:07:40.666082+0000","last_clean":"2023-11-06T03:07:40.666082+0000","last_became_active":"2023-11-06T03:07:39.698228+0000","last_became_peered":"2023-11-06T03:07:39.698228+0000","last_unstale":"2023-11-06T03:07:40.666082+0000","last_undegraded":"2023-11-06T03:07:40.666082+0000","last_fullsized":"2023-11-06T03:07:40.666082+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:33.550315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.676504+0000","last_change":"2023-11-06T03:07:39.306885+0000","last_active":"2023-11-06T03:07:39.676504+0000","last_peered":"2023-11-06T03:07:39.676504+0000","last_clean":"2023-11-06T03:07:39.676504+0000","last_became_active":"2023-11-06T03:07:39.306105+0000","last_became_peered":"2023-11-06T03:07:39.306105+0000","last_unstale":"2023-11-06T03:07:39.676504+0000","last_undegraded":"2023-11-06T03:07:39.676504+0000","last_fullsized":"2023-11-06T03:07:39.676504+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:52:02.121483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.947521+0000","last_change":"2023-11-06T03:07:39.709918+0000","last_active":"2023-11-06T03:07:41.947521+0000","last_peered":"2023-11-06T03:07:41.947521+0000","last_clean":"2023-11-06T03:07:41.947521+0000","last_became_active":"2023-11-06T03:07:39.709319+0000","last_became_peered":"2023-11-06T03:07:39.709319+0000","last_unstale":"2023-11-06T03:07:41.947521+0000","last_undegraded":"2023-11-06T03:07:41.947521+0000","last_fullsized":"2023-11-06T03:07:41.947521+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:33.902329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.677462+0000","last_change":"2023-11-06T03:07:39.313804+0000","last_active":"2023-11-06T03:07:39.677462+0000","last_peered":"2023-11-06T03:07:39.677462+0000","last_clean":"2023-11-06T03:07:39.677462+0000","last_became_active":"2023-11-06T03:07:39.313520+0000","last_became_peered":"2023-11-06T03:07:39.313520+0000","last_unstale":"2023-11-06T03:07:39.677462+0000","last_undegraded":"2023-11-06T03:07:39.677462+0000","last_fullsized":"2023-11-06T03:07:39.677462+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:41.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.058771+0000","last_change":"2023-11-06T03:07:39.677729+0000","last_active":"2023-11-06T03:07:42.058771+0000","last_peered":"2023-11-06T03:07:42.058771+0000","last_clean":"2023-11-06T03:07:42.058771+0000","last_became_active":"2023-11-06T03:07:39.677463+0000","last_became_peered":"2023-11-06T03:07:39.677463+0000","last_unstale":"2023-11-06T03:07:42.058771+0000","last_undegraded":"2023-11-06T03:07:42.058771+0000","last_fullsized":"2023-11-06T03:07:42.058771+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:30.266620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.688255+0000","last_change":"2023-11-06T03:07:39.305334+0000","last_active":"2023-11-06T03:07:39.688255+0000","last_peered":"2023-11-06T03:07:39.688255+0000","last_clean":"2023-11-06T03:07:39.688255+0000","last_became_active":"2023-11-06T03:07:39.304536+0000","last_became_peered":"2023-11-06T03:07:39.304536+0000","last_unstale":"2023-11-06T03:07:39.688255+0000","last_undegraded":"2023-11-06T03:07:39.688255+0000","last_fullsized":"2023-11-06T03:07:39.688255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060328+0000","last_change":"2023-11-06T03:07:39.700017+0000","last_active":"2023-11-06T03:07:42.060328+0000","last_peered":"2023-11-06T03:07:42.060328+0000","last_clean":"2023-11-06T03:07:42.060328+0000","last_became_active":"2023-11-06T03:07:39.699355+0000","last_became_peered":"2023-11-06T03:07:39.699355+0000","last_unstale":"2023-11-06T03:07:42.060328+0000","last_undegraded":"2023-11-06T03:07:42.060328+0000","last_fullsized":"2023-11-06T03:07:42.060328+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:19.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.673765+0000","last_change":"2023-11-06T03:07:39.293582+0000","last_active":"2023-11-06T03:07:39.673765+0000","last_peered":"2023-11-06T03:07:39.673765+0000","last_clean":"2023-11-06T03:07:39.673765+0000","last_became_active":"2023-11-06T03:07:39.292812+0000","last_became_peered":"2023-11-06T03:07:39.292812+0000","last_unstale":"2023-11-06T03:07:39.673765+0000","last_undegraded":"2023-11-06T03:07:39.673765+0000","last_fullsized":"2023-11-06T03:07:39.673765+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:05.690109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061493+0000","last_change":"2023-11-06T03:07:39.712148+0000","last_active":"2023-11-06T03:07:42.061493+0000","last_peered":"2023-11-06T03:07:42.061493+0000","last_clean":"2023-11-06T03:07:42.061493+0000","last_became_active":"2023-11-06T03:07:39.708279+0000","last_became_peered":"2023-11-06T03:07:39.708279+0000","last_unstale":"2023-11-06T03:07:42.061493+0000","last_undegraded":"2023-11-06T03:07:42.061493+0000","last_fullsized":"2023-11-06T03:07:42.061493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:46.662034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.704571+0000","last_change":"2023-11-06T03:07:39.299823+0000","last_active":"2023-11-06T03:07:39.704571+0000","last_peered":"2023-11-06T03:07:39.704571+0000","last_clean":"2023-11-06T03:07:39.704571+0000","last_became_active":"2023-11-06T03:07:39.299171+0000","last_became_peered":"2023-11-06T03:07:39.299171+0000","last_unstale":"2023-11-06T03:07:39.704571+0000","last_undegraded":"2023-11-06T03:07:39.704571+0000","last_fullsized":"2023-11-06T03:07:39.704571+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059890+0000","last_change":"2023-11-06T03:07:39.681681+0000","last_active":"2023-11-06T03:07:42.059890+0000","last_peered":"2023-11-06T03:07:42.059890+0000","last_clean":"2023-11-06T03:07:42.059890+0000","last_became_active":"2023-11-06T03:07:39.681404+0000","last_became_peered":"2023-11-06T03:07:39.681404+0000","last_unstale":"2023-11-06T03:07:42.059890+0000","last_undegraded":"2023-11-06T03:07:42.059890+0000","last_fullsized":"2023-11-06T03:07:42.059890+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:11:05.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.664212+0000","last_change":"2023-11-06T03:07:39.310473+0000","last_active":"2023-11-06T03:07:39.664212+0000","last_peered":"2023-11-06T03:07:39.664212+0000","last_clean":"2023-11-06T03:07:39.664212+0000","last_became_active":"2023-11-06T03:07:39.309765+0000","last_became_peered":"2023-11-06T03:07:39.309765+0000","last_unstale":"2023-11-06T03:07:39.664212+0000","last_undegraded":"2023-11-06T03:07:39.664212+0000","last_fullsized":"2023-11-06T03:07:39.664212+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:57.352267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.062072+0000","last_change":"2023-11-06T03:07:39.708184+0000","last_active":"2023-11-06T03:07:42.062072+0000","last_peered":"2023-11-06T03:07:42.062072+0000","last_clean":"2023-11-06T03:07:42.062072+0000","last_became_active":"2023-11-06T03:07:39.707566+0000","last_became_peered":"2023-11-06T03:07:39.707566+0000","last_unstale":"2023-11-06T03:07:42.062072+0000","last_undegraded":"2023-11-06T03:07:42.062072+0000","last_fullsized":"2023-11-06T03:07:42.062072+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:48:16.133682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.660252+0000","last_change":"2023-11-06T03:07:39.299523+0000","last_active":"2023-11-06T03:07:39.660252+0000","last_peered":"2023-11-06T03:07:39.660252+0000","last_clean":"2023-11-06T03:07:39.660252+0000","last_became_active":"2023-11-06T03:07:39.297099+0000","last_became_peered":"2023-11-06T03:07:39.297099+0000","last_unstale":"2023-11-06T03:07:39.660252+0000","last_undegraded":"2023-11-06T03:07:39.660252+0000","last_fullsized":"2023-11-06T03:07:39.660252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059660+0000","last_change":"2023-11-06T03:07:39.693370+0000","last_active":"2023-11-06T03:07:42.059660+0000","last_peered":"2023-11-06T03:07:42.059660+0000","last_clean":"2023-11-06T03:07:42.059660+0000","last_became_active":"2023-11-06T03:07:39.692559+0000","last_became_peered":"2023-11-06T03:07:39.692559+0000","last_unstale":"2023-11-06T03:07:42.059660+0000","last_undegraded":"2023-11-06T03:07:42.059660+0000","last_fullsized":"2023-11-06T03:07:42.059660+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:33.550315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.673410+0000","last_change":"2023-11-06T03:07:39.309883+0000","last_active":"2023-11-06T03:07:39.673410+0000","last_peered":"2023-11-06T03:07:39.673410+0000","last_clean":"2023-11-06T03:07:39.673410+0000","last_became_active":"2023-11-06T03:07:39.307703+0000","last_became_peered":"2023-11-06T03:07:39.307703+0000","last_unstale":"2023-11-06T03:07:39.673410+0000","last_undegraded":"2023-11-06T03:07:39.673410+0000","last_fullsized":"2023-11-06T03:07:39.673410+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.665862+0000","last_change":"2023-11-06T03:07:39.693700+0000","last_active":"2023-11-06T03:07:40.665862+0000","last_peered":"2023-11-06T03:07:40.665862+0000","last_clean":"2023-11-06T03:07:40.665862+0000","last_became_active":"2023-11-06T03:07:39.692960+0000","last_became_peered":"2023-11-06T03:07:39.692960+0000","last_unstale":"2023-11-06T03:07:40.665862+0000","last_undegraded":"2023-11-06T03:07:40.665862+0000","last_fullsized":"2023-11-06T03:07:40.665862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:29:31.845970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.684235+0000","last_change":"2023-11-06T03:07:39.293164+0000","last_active":"2023-11-06T03:07:39.684235+0000","last_peered":"2023-11-06T03:07:39.684235+0000","last_clean":"2023-11-06T03:07:39.684235+0000","last_became_active":"2023-11-06T03:07:39.292470+0000","last_became_peered":"2023-11-06T03:07:39.292470+0000","last_unstale":"2023-11-06T03:07:39.684235+0000","last_undegraded":"2023-11-06T03:07:39.684235+0000","last_fullsized":"2023-11-06T03:07:39.684235+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060950+0000","last_change":"2023-11-06T03:07:39.680258+0000","last_active":"2023-11-06T03:07:42.060950+0000","last_peered":"2023-11-06T03:07:42.060950+0000","last_clean":"2023-11-06T03:07:42.060950+0000","last_became_active":"2023-11-06T03:07:39.678921+0000","last_became_peered":"2023-11-06T03:07:39.678921+0000","last_unstale":"2023-11-06T03:07:42.060950+0000","last_undegraded":"2023-11-06T03:07:42.060950+0000","last_fullsized":"2023-11-06T03:07:42.060950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:58.026552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.707757+0000","last_change":"2023-11-06T03:07:39.308315+0000","last_active":"2023-11-06T03:07:39.707757+0000","last_peered":"2023-11-06T03:07:39.707757+0000","last_clean":"2023-11-06T03:07:39.707757+0000","last_became_active":"2023-11-06T03:07:39.307648+0000","last_became_peered":"2023-11-06T03:07:39.307648+0000","last_unstale":"2023-11-06T03:07:39.707757+0000","last_undegraded":"2023-11-06T03:07:39.707757+0000","last_fullsized":"2023-11-06T03:07:39.707757+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060182+0000","last_change":"2023-11-06T03:07:39.693902+0000","last_active":"2023-11-06T03:07:42.060182+0000","last_peered":"2023-11-06T03:07:42.060182+0000","last_clean":"2023-11-06T03:07:42.060182+0000","last_became_active":"2023-11-06T03:07:39.692996+0000","last_became_peered":"2023-11-06T03:07:39.692996+0000","last_unstale":"2023-11-06T03:07:42.060182+0000","last_undegraded":"2023-11-06T03:07:42.060182+0000","last_fullsized":"2023-11-06T03:07:42.060182+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:50:53.591027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.689452+0000","last_change":"2023-11-06T03:07:39.302732+0000","last_active":"2023-11-06T03:07:39.689452+0000","last_peered":"2023-11-06T03:07:39.689452+0000","last_clean":"2023-11-06T03:07:39.689452+0000","last_became_active":"2023-11-06T03:07:39.302042+0000","last_became_peered":"2023-11-06T03:07:39.302042+0000","last_unstale":"2023-11-06T03:07:39.689452+0000","last_undegraded":"2023-11-06T03:07:39.689452+0000","last_fullsized":"2023-11-06T03:07:39.689452+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.665932+0000","last_change":"2023-11-06T03:07:39.698551+0000","last_active":"2023-11-06T03:07:40.665932+0000","last_peered":"2023-11-06T03:07:40.665932+0000","last_clean":"2023-11-06T03:07:40.665932+0000","last_became_active":"2023-11-06T03:07:39.697893+0000","last_became_peered":"2023-11-06T03:07:39.697893+0000","last_unstale":"2023-11-06T03:07:40.665932+0000","last_undegraded":"2023-11-06T03:07:40.665932+0000","last_fullsized":"2023-11-06T03:07:40.665932+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:23:09.054772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.696964+0000","last_change":"2023-11-06T03:07:39.310650+0000","last_active":"2023-11-06T03:07:39.696964+0000","last_peered":"2023-11-06T03:07:39.696964+0000","last_clean":"2023-11-06T03:07:39.696964+0000","last_became_active":"2023-11-06T03:07:39.310312+0000","last_became_peered":"2023-11-06T03:07:39.310312+0000","last_unstale":"2023-11-06T03:07:39.696964+0000","last_undegraded":"2023-11-06T03:07:39.696964+0000","last_fullsized":"2023-11-06T03:07:39.696964+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:31:33.492605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061551+0000","last_change":"2023-11-06T03:07:39.671901+0000","last_active":"2023-11-06T03:07:42.061551+0000","last_peered":"2023-11-06T03:07:42.061551+0000","last_clean":"2023-11-06T03:07:42.061551+0000","last_became_active":"2023-11-06T03:07:39.671638+0000","last_became_peered":"2023-11-06T03:07:39.671638+0000","last_unstale":"2023-11-06T03:07:42.061551+0000","last_undegraded":"2023-11-06T03:07:42.061551+0000","last_fullsized":"2023-11-06T03:07:42.061551+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:24:34.081691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.663692+0000","last_change":"2023-11-06T03:07:39.296745+0000","last_active":"2023-11-06T03:07:39.663692+0000","last_peered":"2023-11-06T03:07:39.663692+0000","last_clean":"2023-11-06T03:07:39.663692+0000","last_became_active":"2023-11-06T03:07:39.296009+0000","last_became_peered":"2023-11-06T03:07:39.296009+0000","last_unstale":"2023-11-06T03:07:39.663692+0000","last_undegraded":"2023-11-06T03:07:39.663692+0000","last_fullsized":"2023-11-06T03:07:39.663692+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059670+0000","last_change":"2023-11-06T03:07:39.708643+0000","last_active":"2023-11-06T03:07:42.059670+0000","last_peered":"2023-11-06T03:07:42.059670+0000","last_clean":"2023-11-06T03:07:42.059670+0000","last_became_active":"2023-11-06T03:07:39.708337+0000","last_became_peered":"2023-11-06T03:07:39.708337+0000","last_unstale":"2023-11-06T03:07:42.059670+0000","last_undegraded":"2023-11-06T03:07:42.059670+0000","last_fullsized":"2023-11-06T03:07:42.059670+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:44.434772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.672635+0000","last_change":"2023-11-06T03:07:39.315222+0000","last_active":"2023-11-06T03:07:39.672635+0000","last_peered":"2023-11-06T03:07:39.672635+0000","last_clean":"2023-11-06T03:07:39.672635+0000","last_became_active":"2023-11-06T03:07:39.314480+0000","last_became_peered":"2023-11-06T03:07:39.314480+0000","last_unstale":"2023-11-06T03:07:39.672635+0000","last_undegraded":"2023-11-06T03:07:39.672635+0000","last_fullsized":"2023-11-06T03:07:39.672635+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:37.522149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060624+0000","last_change":"2023-11-06T03:07:39.686750+0000","last_active":"2023-11-06T03:07:42.060624+0000","last_peered":"2023-11-06T03:07:42.060624+0000","last_clean":"2023-11-06T03:07:42.060624+0000","last_became_active":"2023-11-06T03:07:39.686498+0000","last_became_peered":"2023-11-06T03:07:39.686498+0000","last_unstale":"2023-11-06T03:07:42.060624+0000","last_undegraded":"2023-11-06T03:07:42.060624+0000","last_fullsized":"2023-11-06T03:07:42.060624+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:02.791499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.684936+0000","last_change":"2023-11-06T03:07:39.303057+0000","last_active":"2023-11-06T03:07:39.684936+0000","last_peered":"2023-11-06T03:07:39.684936+0000","last_clean":"2023-11-06T03:07:39.684936+0000","last_became_active":"2023-11-06T03:07:39.301922+0000","last_became_peered":"2023-11-06T03:07:39.301922+0000","last_unstale":"2023-11-06T03:07:39.684936+0000","last_undegraded":"2023-11-06T03:07:39.684936+0000","last_fullsized":"2023-11-06T03:07:39.684936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.058879+0000","last_change":"2023-11-06T03:07:39.688961+0000","last_active":"2023-11-06T03:07:42.058879+0000","last_peered":"2023-11-06T03:07:42.058879+0000","last_clean":"2023-11-06T03:07:42.058879+0000","last_became_active":"2023-11-06T03:07:39.688559+0000","last_became_peered":"2023-11-06T03:07:39.688559+0000","last_unstale":"2023-11-06T03:07:42.058879+0000","last_undegraded":"2023-11-06T03:07:42.058879+0000","last_fullsized":"2023-11-06T03:07:42.058879+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:26.789970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.677968+0000","last_change":"2023-11-06T03:07:39.303919+0000","last_active":"2023-11-06T03:07:39.677968+0000","last_peered":"2023-11-06T03:07:39.677968+0000","last_clean":"2023-11-06T03:07:39.677968+0000","last_became_active":"2023-11-06T03:07:39.303101+0000","last_became_peered":"2023-11-06T03:07:39.303101+0000","last_unstale":"2023-11-06T03:07:39.677968+0000","last_undegraded":"2023-11-06T03:07:39.677968+0000","last_fullsized":"2023-11-06T03:07:39.677968+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:15:38.405617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.662427+0000","last_change":"2023-11-06T03:07:39.667880+0000","last_active":"2023-11-06T03:07:40.662427+0000","last_peered":"2023-11-06T03:07:40.662427+0000","last_clean":"2023-11-06T03:07:40.662427+0000","last_became_active":"2023-11-06T03:07:39.667617+0000","last_became_peered":"2023-11-06T03:07:39.667617+0000","last_unstale":"2023-11-06T03:07:40.662427+0000","last_undegraded":"2023-11-06T03:07:40.662427+0000","last_fullsized":"2023-11-06T03:07:40.662427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:56.532484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.682417+0000","last_change":"2023-11-06T03:07:39.293261+0000","last_active":"2023-11-06T03:07:39.682417+0000","last_peered":"2023-11-06T03:07:39.682417+0000","last_clean":"2023-11-06T03:07:39.682417+0000","last_became_active":"2023-11-06T03:07:39.292983+0000","last_became_peered":"2023-11-06T03:07:39.292983+0000","last_unstale":"2023-11-06T03:07:39.682417+0000","last_undegraded":"2023-11-06T03:07:39.682417+0000","last_fullsized":"2023-11-06T03:07:39.682417+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059656+0000","last_change":"2023-11-06T03:07:39.693487+0000","last_active":"2023-11-06T03:07:42.059656+0000","last_peered":"2023-11-06T03:07:42.059656+0000","last_clean":"2023-11-06T03:07:42.059656+0000","last_became_active":"2023-11-06T03:07:39.692870+0000","last_became_peered":"2023-11-06T03:07:39.692870+0000","last_unstale":"2023-11-06T03:07:42.059656+0000","last_undegraded":"2023-11-06T03:07:42.059656+0000","last_fullsized":"2023-11-06T03:07:42.059656+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:56.532484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.683196+0000","last_change":"2023-11-06T03:07:39.303243+0000","last_active":"2023-11-06T03:07:39.683196+0000","last_peered":"2023-11-06T03:07:39.683196+0000","last_clean":"2023-11-06T03:07:39.683196+0000","last_became_active":"2023-11-06T03:07:39.302369+0000","last_became_peered":"2023-11-06T03:07:39.302369+0000","last_unstale":"2023-11-06T03:07:39.683196+0000","last_undegraded":"2023-11-06T03:07:39.683196+0000","last_fullsized":"2023-11-06T03:07:39.683196+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.661795+0000","last_change":"2023-11-06T03:07:39.666228+0000","last_active":"2023-11-06T03:07:40.661795+0000","last_peered":"2023-11-06T03:07:40.661795+0000","last_clean":"2023-11-06T03:07:40.661795+0000","last_became_active":"2023-11-06T03:07:39.665955+0000","last_became_peered":"2023-11-06T03:07:39.665955+0000","last_unstale":"2023-11-06T03:07:40.661795+0000","last_undegraded":"2023-11-06T03:07:40.661795+0000","last_fullsized":"2023-11-06T03:07:40.661795+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:47.703903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.664646+0000","last_change":"2023-11-06T03:07:39.306740+0000","last_active":"2023-11-06T03:07:39.664646+0000","last_peered":"2023-11-06T03:07:39.664646+0000","last_clean":"2023-11-06T03:07:39.664646+0000","last_became_active":"2023-11-06T03:07:39.305975+0000","last_became_peered":"2023-11-06T03:07:39.305975+0000","last_unstale":"2023-11-06T03:07:39.664646+0000","last_undegraded":"2023-11-06T03:07:39.664646+0000","last_fullsized":"2023-11-06T03:07:39.664646+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.344540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.666193+0000","last_change":"2023-11-06T03:07:39.701979+0000","last_active":"2023-11-06T03:07:40.666193+0000","last_peered":"2023-11-06T03:07:40.666193+0000","last_clean":"2023-11-06T03:07:40.666193+0000","last_became_active":"2023-11-06T03:07:39.701260+0000","last_became_peered":"2023-11-06T03:07:39.701260+0000","last_unstale":"2023-11-06T03:07:40.666193+0000","last_undegraded":"2023-11-06T03:07:40.666193+0000","last_fullsized":"2023-11-06T03:07:40.666193+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:45.590048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.676059+0000","last_change":"2023-11-06T03:07:39.303923+0000","last_active":"2023-11-06T03:07:39.676059+0000","last_peered":"2023-11-06T03:07:39.676059+0000","last_clean":"2023-11-06T03:07:39.676059+0000","last_became_active":"2023-11-06T03:07:39.303575+0000","last_became_peered":"2023-11-06T03:07:39.303575+0000","last_unstale":"2023-11-06T03:07:39.676059+0000","last_undegraded":"2023-11-06T03:07:39.676059+0000","last_fullsized":"2023-11-06T03:07:39.676059+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:19:50.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.947257+0000","last_change":"2023-11-06T03:07:39.701621+0000","last_active":"2023-11-06T03:07:41.947257+0000","last_peered":"2023-11-06T03:07:41.947257+0000","last_clean":"2023-11-06T03:07:41.947257+0000","last_became_active":"2023-11-06T03:07:39.700910+0000","last_became_peered":"2023-11-06T03:07:39.700910+0000","last_unstale":"2023-11-06T03:07:41.947257+0000","last_undegraded":"2023-11-06T03:07:41.947257+0000","last_fullsized":"2023-11-06T03:07:41.947257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:55:35.694050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.657660+0000","last_change":"2023-11-06T03:07:39.302148+0000","last_active":"2023-11-06T03:07:39.657660+0000","last_peered":"2023-11-06T03:07:39.657660+0000","last_clean":"2023-11-06T03:07:39.657660+0000","last_became_active":"2023-11-06T03:07:39.301670+0000","last_became_peered":"2023-11-06T03:07:39.301670+0000","last_unstale":"2023-11-06T03:07:39.657660+0000","last_undegraded":"2023-11-06T03:07:39.657660+0000","last_fullsized":"2023-11-06T03:07:39.657660+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.061350+0000","last_change":"2023-11-06T03:07:39.679359+0000","last_active":"2023-11-06T03:07:42.061350+0000","last_peered":"2023-11-06T03:07:42.061350+0000","last_clean":"2023-11-06T03:07:42.061350+0000","last_became_active":"2023-11-06T03:07:39.678962+0000","last_became_peered":"2023-11-06T03:07:39.678962+0000","last_unstale":"2023-11-06T03:07:42.061350+0000","last_undegraded":"2023-11-06T03:07:42.061350+0000","last_fullsized":"2023-11-06T03:07:42.061350+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:02.643455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.660668+0000","last_change":"2023-11-06T03:07:39.300812+0000","last_active":"2023-11-06T03:07:39.660668+0000","last_peered":"2023-11-06T03:07:39.660668+0000","last_clean":"2023-11-06T03:07:39.660668+0000","last_became_active":"2023-11-06T03:07:39.300522+0000","last_became_peered":"2023-11-06T03:07:39.300522+0000","last_unstale":"2023-11-06T03:07:39.660668+0000","last_undegraded":"2023-11-06T03:07:39.660668+0000","last_fullsized":"2023-11-06T03:07:39.660668+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.662093+0000","last_change":"2023-11-06T03:07:39.677341+0000","last_active":"2023-11-06T03:07:40.662093+0000","last_peered":"2023-11-06T03:07:40.662093+0000","last_clean":"2023-11-06T03:07:40.662093+0000","last_became_active":"2023-11-06T03:07:39.676898+0000","last_became_peered":"2023-11-06T03:07:39.676898+0000","last_unstale":"2023-11-06T03:07:40.662093+0000","last_undegraded":"2023-11-06T03:07:40.662093+0000","last_fullsized":"2023-11-06T03:07:40.662093+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:23:09.054772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.658285+0000","last_change":"2023-11-06T03:07:39.307439+0000","last_active":"2023-11-06T03:07:39.658285+0000","last_peered":"2023-11-06T03:07:39.658285+0000","last_clean":"2023-11-06T03:07:39.658285+0000","last_became_active":"2023-11-06T03:07:39.307129+0000","last_became_peered":"2023-11-06T03:07:39.307129+0000","last_unstale":"2023-11-06T03:07:39.658285+0000","last_undegraded":"2023-11-06T03:07:39.658285+0000","last_fullsized":"2023-11-06T03:07:39.658285+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:54:47.299131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:41.946850+0000","last_change":"2023-11-06T03:07:39.686755+0000","last_active":"2023-11-06T03:07:41.946850+0000","last_peered":"2023-11-06T03:07:41.946850+0000","last_clean":"2023-11-06T03:07:41.946850+0000","last_became_active":"2023-11-06T03:07:39.684544+0000","last_became_peered":"2023-11-06T03:07:39.684544+0000","last_unstale":"2023-11-06T03:07:41.946850+0000","last_undegraded":"2023-11-06T03:07:41.946850+0000","last_fullsized":"2023-11-06T03:07:41.946850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:19:46.511847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.665196+0000","last_change":"2023-11-06T03:07:39.310209+0000","last_active":"2023-11-06T03:07:39.665196+0000","last_peered":"2023-11-06T03:07:39.665196+0000","last_clean":"2023-11-06T03:07:39.665196+0000","last_became_active":"2023-11-06T03:07:39.309642+0000","last_became_peered":"2023-11-06T03:07:39.309642+0000","last_unstale":"2023-11-06T03:07:39.665196+0000","last_undegraded":"2023-11-06T03:07:39.665196+0000","last_fullsized":"2023-11-06T03:07:39.665196+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:34.476269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.678237+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:07:39.678237+0000","last_peered":"2023-11-06T03:07:39.678237+0000","last_clean":"2023-11-06T03:07:39.678237+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:07:39.678237+0000","last_undegraded":"2023-11-06T03:07:39.678237+0000","last_fullsized":"2023-11-06T03:07:39.678237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.658206+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:07:39.658206+0000","last_peered":"2023-11-06T03:07:39.658206+0000","last_clean":"2023-11-06T03:07:39.658206+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:07:39.658206+0000","last_undegraded":"2023-11-06T03:07:39.658206+0000","last_fullsized":"2023-11-06T03:07:39.658206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.662507+0000","last_change":"2023-11-06T03:07:39.668481+0000","last_active":"2023-11-06T03:07:40.662507+0000","last_peered":"2023-11-06T03:07:40.662507+0000","last_clean":"2023-11-06T03:07:40.662507+0000","last_became_active":"2023-11-06T03:07:39.668222+0000","last_became_peered":"2023-11-06T03:07:39.668222+0000","last_unstale":"2023-11-06T03:07:40.662507+0000","last_undegraded":"2023-11-06T03:07:40.662507+0000","last_fullsized":"2023-11-06T03:07:40.662507+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:15:06.693816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.673704+0000","last_change":"2023-11-06T03:07:39.310540+0000","last_active":"2023-11-06T03:07:39.673704+0000","last_peered":"2023-11-06T03:07:39.673704+0000","last_clean":"2023-11-06T03:07:39.673704+0000","last_became_active":"2023-11-06T03:07:39.309089+0000","last_became_peered":"2023-11-06T03:07:39.309089+0000","last_unstale":"2023-11-06T03:07:39.673704+0000","last_undegraded":"2023-11-06T03:07:39.673704+0000","last_fullsized":"2023-11-06T03:07:39.673704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:21:53.925471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.671721+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:07:39.671721+0000","last_peered":"2023-11-06T03:07:39.671721+0000","last_clean":"2023-11-06T03:07:39.671721+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:07:39.671721+0000","last_undegraded":"2023-11-06T03:07:39.671721+0000","last_fullsized":"2023-11-06T03:07:39.671721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059292+0000","last_change":"2023-11-06T03:07:39.705414+0000","last_active":"2023-11-06T03:07:42.059292+0000","last_peered":"2023-11-06T03:07:42.059292+0000","last_clean":"2023-11-06T03:07:42.059292+0000","last_became_active":"2023-11-06T03:07:39.705089+0000","last_became_peered":"2023-11-06T03:07:39.705089+0000","last_unstale":"2023-11-06T03:07:42.059292+0000","last_undegraded":"2023-11-06T03:07:42.059292+0000","last_fullsized":"2023-11-06T03:07:42.059292+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:45.590048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.661504+0000","last_change":"2023-11-06T03:07:39.296084+0000","last_active":"2023-11-06T03:07:39.661504+0000","last_peered":"2023-11-06T03:07:39.661504+0000","last_clean":"2023-11-06T03:07:39.661504+0000","last_became_active":"2023-11-06T03:07:39.295725+0000","last_became_peered":"2023-11-06T03:07:39.295725+0000","last_unstale":"2023-11-06T03:07:39.661504+0000","last_undegraded":"2023-11-06T03:07:39.661504+0000","last_fullsized":"2023-11-06T03:07:39.661504+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:30:33.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.658730+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:07:39.658730+0000","last_peered":"2023-11-06T03:07:39.658730+0000","last_clean":"2023-11-06T03:07:39.658730+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:07:39.658730+0000","last_undegraded":"2023-11-06T03:07:39.658730+0000","last_fullsized":"2023-11-06T03:07:39.658730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.666156+0000","last_change":"2023-11-06T03:07:39.688166+0000","last_active":"2023-11-06T03:07:40.666156+0000","last_peered":"2023-11-06T03:07:40.666156+0000","last_clean":"2023-11-06T03:07:40.666156+0000","last_became_active":"2023-11-06T03:07:39.687185+0000","last_became_peered":"2023-11-06T03:07:39.687185+0000","last_unstale":"2023-11-06T03:07:40.666156+0000","last_undegraded":"2023-11-06T03:07:40.666156+0000","last_fullsized":"2023-11-06T03:07:40.666156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:47.703903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.682469+0000","last_change":"2023-11-06T03:07:39.298434+0000","last_active":"2023-11-06T03:07:39.682469+0000","last_peered":"2023-11-06T03:07:39.682469+0000","last_clean":"2023-11-06T03:07:39.682469+0000","last_became_active":"2023-11-06T03:07:39.296646+0000","last_became_peered":"2023-11-06T03:07:39.296646+0000","last_unstale":"2023-11-06T03:07:39.682469+0000","last_undegraded":"2023-11-06T03:07:39.682469+0000","last_fullsized":"2023-11-06T03:07:39.682469+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:06.467492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.657650+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:07:39.657650+0000","last_peered":"2023-11-06T03:07:39.657650+0000","last_clean":"2023-11-06T03:07:39.657650+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:07:39.657650+0000","last_undegraded":"2023-11-06T03:07:39.657650+0000","last_fullsized":"2023-11-06T03:07:39.657650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060995+0000","last_change":"2023-11-06T03:07:39.670280+0000","last_active":"2023-11-06T03:07:42.060995+0000","last_peered":"2023-11-06T03:07:42.060995+0000","last_clean":"2023-11-06T03:07:42.060995+0000","last_became_active":"2023-11-06T03:07:39.669987+0000","last_became_peered":"2023-11-06T03:07:39.669987+0000","last_unstale":"2023-11-06T03:07:42.060995+0000","last_undegraded":"2023-11-06T03:07:42.060995+0000","last_fullsized":"2023-11-06T03:07:42.060995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:18.025337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.683927+0000","last_change":"2023-11-06T03:07:39.304975+0000","last_active":"2023-11-06T03:07:39.683927+0000","last_peered":"2023-11-06T03:07:39.683927+0000","last_clean":"2023-11-06T03:07:39.683927+0000","last_became_active":"2023-11-06T03:07:39.304184+0000","last_became_peered":"2023-11-06T03:07:39.304184+0000","last_unstale":"2023-11-06T03:07:39.683927+0000","last_undegraded":"2023-11-06T03:07:39.683927+0000","last_fullsized":"2023-11-06T03:07:39.683927+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:00:10.299048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.652748+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:07:39.652748+0000","last_peered":"2023-11-06T03:07:39.652748+0000","last_clean":"2023-11-06T03:07:39.652748+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:07:39.652748+0000","last_undegraded":"2023-11-06T03:07:39.652748+0000","last_fullsized":"2023-11-06T03:07:39.652748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.059977+0000","last_change":"2023-11-06T03:07:39.681689+0000","last_active":"2023-11-06T03:07:42.059977+0000","last_peered":"2023-11-06T03:07:42.059977+0000","last_clean":"2023-11-06T03:07:42.059977+0000","last_became_active":"2023-11-06T03:07:39.681408+0000","last_became_peered":"2023-11-06T03:07:39.681408+0000","last_unstale":"2023-11-06T03:07:42.059977+0000","last_undegraded":"2023-11-06T03:07:42.059977+0000","last_fullsized":"2023-11-06T03:07:42.059977+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:48:16.133682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.657962+0000","last_change":"2023-11-06T03:07:39.302064+0000","last_active":"2023-11-06T03:07:39.657962+0000","last_peered":"2023-11-06T03:07:39.657962+0000","last_clean":"2023-11-06T03:07:39.657962+0000","last_became_active":"2023-11-06T03:07:39.301389+0000","last_became_peered":"2023-11-06T03:07:39.301389+0000","last_unstale":"2023-11-06T03:07:39.657962+0000","last_undegraded":"2023-11-06T03:07:39.657962+0000","last_fullsized":"2023-11-06T03:07:39.657962+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.678452+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:07:39.678452+0000","last_peered":"2023-11-06T03:07:39.678452+0000","last_clean":"2023-11-06T03:07:39.678452+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:07:39.678452+0000","last_undegraded":"2023-11-06T03:07:39.678452+0000","last_fullsized":"2023-11-06T03:07:39.678452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:42.060234+0000","last_change":"2023-11-06T03:07:39.682060+0000","last_active":"2023-11-06T03:07:42.060234+0000","last_peered":"2023-11-06T03:07:42.060234+0000","last_clean":"2023-11-06T03:07:42.060234+0000","last_became_active":"2023-11-06T03:07:39.681781+0000","last_became_peered":"2023-11-06T03:07:39.681781+0000","last_unstale":"2023-11-06T03:07:42.060234+0000","last_undegraded":"2023-11-06T03:07:42.060234+0000","last_fullsized":"2023-11-06T03:07:42.060234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:50.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.659423+0000","last_change":"2023-11-06T03:07:39.302379+0000","last_active":"2023-11-06T03:07:39.659423+0000","last_peered":"2023-11-06T03:07:39.659423+0000","last_clean":"2023-11-06T03:07:39.659423+0000","last_became_active":"2023-11-06T03:07:39.301774+0000","last_became_peered":"2023-11-06T03:07:39.301774+0000","last_unstale":"2023-11-06T03:07:39.659423+0000","last_undegraded":"2023-11-06T03:07:39.659423+0000","last_fullsized":"2023-11-06T03:07:39.659423+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:25.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.653134+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:07:39.653134+0000","last_peered":"2023-11-06T03:07:39.653134+0000","last_clean":"2023-11-06T03:07:39.653134+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:07:39.653134+0000","last_undegraded":"2023-11-06T03:07:39.653134+0000","last_fullsized":"2023-11-06T03:07:39.653134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-06T03:07:40.666768+0000","last_change":"2023-11-06T03:07:39.711345+0000","last_active":"2023-11-06T03:07:40.666768+0000","last_peered":"2023-11-06T03:07:40.666768+0000","last_clean":"2023-11-06T03:07:40.666768+0000","last_became_active":"2023-11-06T03:07:39.710771+0000","last_became_peered":"2023-11-06T03:07:39.710771+0000","last_unstale":"2023-11-06T03:07:40.666768+0000","last_undegraded":"2023-11-06T03:07:40.666768+0000","last_fullsized":"2023-11-06T03:07:40.666768+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:39.261260+0000","last_clean_scrub_stamp":"2023-11-06T03:07:39.261260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:43:19.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.657759+0000","last_change":"2023-11-06T03:07:39.297389+0000","last_active":"2023-11-06T03:07:39.657759+0000","last_peered":"2023-11-06T03:07:39.657759+0000","last_clean":"2023-11-06T03:07:39.657759+0000","last_became_active":"2023-11-06T03:07:39.297114+0000","last_became_peered":"2023-11-06T03:07:39.297114+0000","last_unstale":"2023-11-06T03:07:39.657759+0000","last_undegraded":"2023-11-06T03:07:39.657759+0000","last_fullsized":"2023-11-06T03:07:39.657759+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:07:38.257552+0000","last_clean_scrub_stamp":"2023-11-06T03:07:38.257552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:58.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-06T03:07:39.664617+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:07:39.664617+0000","last_peered":"2023-11-06T03:07:39.664617+0000","last_clean":"2023-11-06T03:07:39.664617+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:07:39.664617+0000","last_undegraded":"2023-11-06T03:07:39.664617+0000","last_fullsized":"2023-11-06T03:07:39.664617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50296,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321544,"statfs":{"total":96636764160,"available":96585261056,"internally_reserved":0,"allocated":1228800,"data_stored":864264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.81899999999999995},"min":{"1min":0.58599999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.397,"15min":1.397},"last":0.70399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.872},"min":{"1min":0.49399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1739999999999999,"5min":1.444,"15min":1.444},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.87,"15min":0.89600000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1319999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86599999999999999,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1499999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93300000000000005,"15min":0.97099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3180000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.111,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.995,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.628,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.387,"5min":1.857,"15min":1.857},"last":1.119}]},{"osd":4,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.65000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.298,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.98799999999999999,"15min":1.022},"min":{"1min":0.63400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1200000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.66800000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3140000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.069,"15min":1.099},"min":{"1min":0.66800000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3700000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.272}]},{"osd":6,"last update":"Mon Nov 6 03:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.80300000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6850000000000001,"5min":1.774,"15min":1.774},"last":1.159},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.095,"15min":1.115},"min":{"1min":0.77300000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.335,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.0509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43952,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327888,"statfs":{"total":96636764160,"available":96591757312,"internally_reserved":0,"allocated":630784,"data_stored":273405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77800000000000002,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.226,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.2210000000000001,"5min":2.073,"15min":2.073},"last":1.23}]},{"osd":1,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.411,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.86599999999999999,"15min":0.87},"min":{"1min":0.63800000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3600000000000001,"5min":1.778,"15min":1.778},"last":0.91700000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79900000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0389999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.121},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.329,"5min":1.806,"15min":1.806},"last":1.1659999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3819999999999999,"5min":1.857,"15min":1.857},"last":1.075},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.89700000000000002,"15min":0.88900000000000001},"min":{"1min":0.67800000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.89100000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.117},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.294,"5min":2.931,"15min":2.931},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.137,"15min":1.163},"min":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.3100000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.3100000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.036},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.429,"5min":1.506,"15min":1.506},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.097},"min":{"1min":0.90800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.4750000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.2609999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:07:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.093,"15min":1.105},"min":{"1min":0.71399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4279999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3440000000000001,"5min":1.415,"15min":1.415},"last":1.2010000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43948,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94327892,"statfs":{"total":96636764160,"available":96591761408,"internally_reserved":0,"allocated":626688,"data_stored":273772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2290000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":1.097}]},{"osd":1,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.748,"15min":0.71499999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.018,"5min":1.2,"15min":1.2},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.85099999999999998},"min":{"1min":0.502,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2649999999999999,"5min":1.357,"15min":1.357},"last":0.75600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.264,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.24},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3089999999999999,"5min":1.403,"15min":1.403},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93400000000000005,"15min":0.93100000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2170000000000001,"5min":1.429,"15min":1.429},"last":1.0309999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.91900000000000004,"15min":0.92500000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1479999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.83099999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.004,"15min":0.99099999999999999},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":1.248,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.357},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.999,"15min":1.0069999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.296,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.5629999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0009999999999999,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2649999999999999,"5min":1.353,"15min":1.353},"last":1.476},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.077,"15min":1.0780000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.282,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.141}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49712,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322128,"statfs":{"total":96636764160,"available":96585859072,"internally_reserved":0,"allocated":630784,"data_stored":274354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81200000000000006,"15min":0.82899999999999996},"min":{"1min":0.46999999999999997,"5min":0.372,"15min":0.372},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.0740000000000001},"min":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5820000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.60499999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0600000000000001,"5min":1.329,"15min":1.329},"last":0.86899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.93000000000000005,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1659999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.026},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90400000000000003,"15min":0.89800000000000002},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.80500000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0129999999999999,"15min":1.0189999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.087,"15min":1.111},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.1990000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.208}]},{"osd":6,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.012,"15min":1.0289999999999999},"min":{"1min":0.64100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.069},{"interface":"front","average":{"1min":1.077,"5min":1.1299999999999999,"15min":1.1659999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8120000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.101}]},{"osd":7,"last update":"Mon Nov 6 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0669999999999999,"15min":1.073},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.137},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.2549999999999999,"15min":1.1910000000000001},"min":{"1min":0.91000000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":13.582000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.1739999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49720,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322120,"statfs":{"total":96636764160,"available":96585850880,"internally_reserved":0,"allocated":638976,"data_stored":274339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.433,"5min":0.193,"15min":0.193},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.151,"5min":1.524,"15min":1.524},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94099999999999995,"5min":1.353,"15min":1.353},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1970000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1850000000000001,"5min":1.349,"15min":1.349},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.443}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.51700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":1.327,"15min":1.327},"last":0.752},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93500000000000005,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0960000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.1220000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97099999999999997,"15min":0.99199999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1819999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":1.698,"15min":1.698},"last":0.99399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.075},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.97699999999999998,"15min":1.0089999999999999},"min":{"1min":0.65400000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4610000000000001,"5min":1.544,"15min":1.544},"last":1.0229999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.0349999999999999,"15min":1.0649999999999999},"min":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5249999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.044,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.22,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":275336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.55000000000000004},"min":{"1min":0.215,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60999999999999999,"15min":0.64600000000000002},"min":{"1min":0.30199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.0940000000000001,"5min":1.121,"15min":1.121},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.67200000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.68100000000000005},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.0249999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.72799999999999998,"15min":0.75600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":0.89200000000000002},"min":{"1min":0.52700000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.92600000000000005,"15min":0.96399999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2150000000000001,"5min":1.341,"15min":1.341},"last":0.98099999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86499999999999999,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.111,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.006},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93899999999999995,"15min":0.96999999999999997},"min":{"1min":0.73899999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5509999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.92300000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.93999999999999995,"15min":0.95299999999999996},"min":{"1min":0.70099999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1850000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.998},"min":{"1min":0.59199999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6379999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95399999999999996,"15min":0.96999999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.054}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":634880,"data_stored":273932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.32400000000000001,"5min":0.19,"15min":0.19},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.621},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.1749999999999998,"5min":0.91000000000000003,"15min":0.72099999999999997},"min":{"1min":0.29999999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":30.861000000000001,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66800000000000004,"15min":0.69299999999999995},"min":{"1min":0.312,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.33700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.33900000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.878},"min":{"1min":0.51400000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.0449999999999999,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":14.753,"5min":14.753,"15min":14.753},"last":0.80900000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":3.306,"5min":1.4339999999999999,"15min":1.1539999999999999},"min":{"1min":0.64000000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.103999999999999,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":3.0459999999999998,"5min":1.409,"15min":1.181},"min":{"1min":0.53100000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":44.753999999999998,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.878}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":2.2029999999999998,"5min":1.2230000000000001,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":23.971,"5min":23.971,"15min":23.971},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.1469999999999998,"5min":1.2410000000000001,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.503,"15min":0.503},"max":{"1min":14.813000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.86299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0509999999999999,"15min":1.089},"min":{"1min":0.65400000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0589999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54388,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317452,"statfs":{"total":96636764160,"available":96581070848,"internally_reserved":0,"allocated":1224704,"data_stored":863177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.27000000000000002,"5min":0.218,"15min":0.218},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":0.70599999999999996,"15min":0.65400000000000003},"min":{"1min":0.377,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":7.1230000000000002,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.309}]},{"osd":2,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.36899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.502},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.33400000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.6920000000000002,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.39200000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.40500000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.96199999999999997,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.94799999999999995},"min":{"1min":0.62,"5min":0.432,"15min":0.432},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.238,"5min":0.98599999999999999,"15min":0.94799999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.7430000000000003,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.73999999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.92000000000000004,"15min":0.90400000000000003},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.931,"5min":2.931,"15min":2.931},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.0269999999999999,"15min":1.0169999999999999},"min":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:06:58 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0469999999999999,"15min":1.0149999999999999},"min":{"1min":0.65500000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.097,"5min":1.0009999999999999,"15min":0.997},"min":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.871}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:07:48.457 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:07:48.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:07:48.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 Processor -- start 2023-11-06T03:07:48.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 -- start start 2023-11-06T03:07:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91841551e0 0x7f918406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918415f580 con 0x7f91840b3b20 2023-11-06T03:07:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918415efc0 con 0x7f918415e5e0 2023-11-06T03:07:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9188df8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918415f2a0 con 0x7f91841551e0 2023-11-06T03:07:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f918259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91841551e0 0x7f918406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9182d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f91840b3b20 0x7f918415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47426/0 (socket says 172.21.15.157:47426) 2023-11-06T03:07:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.621+0000 7f9182d9d640 1 -- 172.21.15.157:0/532175988 learned_addr learned my addr 172.21.15.157:0/532175988 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918159a640 1 -- 172.21.15.157:0/532175988 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697748821 0 0) 0x7f918415f580 con 0x7f91840b3b20 2023-11-06T03:07:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918159a640 1 -- 172.21.15.157:0/532175988 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916c003530 con 0x7f91840b3b20 2023-11-06T03:07:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918259c640 1 -- 172.21.15.157:0/532175988 >> v1:172.21.15.195:6789/0 conn(0x7f918415e5e0 legacy=0x7f91840b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918259c640 1 -- 172.21.15.157:0/532175988 >> v1:172.21.15.157:6789/0 conn(0x7f91840b3b20 legacy=0x7f918415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918259c640 1 -- 172.21.15.157:0/532175988 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918406f5a0 con 0x7f91841551e0 2023-11-06T03:07:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918259c640 1 --2- 172.21.15.157:0/532175988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f918406fda0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f91740097c0 tx=0x7f917401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35c3eeaccdfda39 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918159a640 1 -- 172.21.15.157:0/532175988 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740ac640 con 0x7f91841551e0 2023-11-06T03:07:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918159a640 1 -- 172.21.15.157:0/532175988 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91740ac7a0 con 0x7f91841551e0 2023-11-06T03:07:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f918159a640 1 -- 172.21.15.157:0/532175988 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740acb30 con 0x7f91841551e0 2023-11-06T03:07:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 -- 172.21.15.157:0/532175988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 msgr2=0x7f918406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 --2- 172.21.15.157:0/532175988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f918406fda0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f91740097c0 tx=0x7f917401c2f0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 -- 172.21.15.157:0/532175988 shutdown_connections 2023-11-06T03:07:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 --2- 172.21.15.157:0/532175988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f918406fda0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 -- 172.21.15.157:0/532175988 >> 172.21.15.157:0/532175988 conn(0x7f9184065930 msgr2=0x7f9184065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 -- 172.21.15.157:0/532175988 shutdown_connections 2023-11-06T03:07:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.625+0000 7f9188df8640 1 -- 172.21.15.157:0/532175988 wait complete. 2023-11-06T03:07:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 Processor -- start 2023-11-06T03:07:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- start start 2023-11-06T03:07:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91840b3b20 0x7f918415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f918415e5e0 0x7f91841f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184155ee0 con 0x7f91841551e0 2023-11-06T03:07:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841561c0 con 0x7f91840b3b20 2023-11-06T03:07:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918414a430 con 0x7f918415e5e0 2023-11-06T03:07:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47464/0 (socket says 172.21.15.157:47464) 2023-11-06T03:07:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 -- 172.21.15.157:0/641508401 learned_addr learned my addr 172.21.15.157:0/641508401 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9182d9d640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f918415e5e0 0x7f91841f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 -- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f918415e5e0 msgr2=0x7f91841f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f918415e5e0 0x7f91841f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 -- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91840b3b20 msgr2=0x7f918415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f918259c640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91840b3b20 0x7f918415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91840b3b20 0x7f918415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 -- 172.21.15.157:0/641508401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174009420 con 0x7f91841551e0 2023-11-06T03:07:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9181d9b640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f917800b410 tx=0x7f917800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178004020 con 0x7f91841551e0 2023-11-06T03:07:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9178004180 con 0x7f91841551e0 2023-11-06T03:07:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178011630 con 0x7f91841551e0 2023-11-06T03:07:48.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918414a710 con 0x7f91841551e0 2023-11-06T03:07:48.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841564a0 con 0x7f91841551e0 2023-11-06T03:07:48.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918414a930 con 0x7f91841551e0 2023-11-06T03:07:48.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.629+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9178011790 con 0x7f91841551e0 2023-11-06T03:07:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.633+0000 7f9172ffd640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 0x7f9164067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.633+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9178083030 con 0x7f91841551e0 2023-11-06T03:07:48.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.633+0000 7f918259c640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 0x7f9164067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.633+0000 7f918259c640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 0x7f9164067f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91740ae000 tx=0x7f9174008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.637+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9178050920 con 0x7f91841551e0 2023-11-06T03:07:48.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.829+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9184072750 con 0x7f91841551e0 2023-11-06T03:07:48.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9172ffd640 1 -- 172.21.15.157:0/641508401 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9178053f40 con 0x7f91841551e0 2023-11-06T03:07:48.840 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:07:48.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 msgr2=0x7f9164067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 0x7f9164067f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f91740ae000 tx=0x7f9174008040 comp rx=0 tx=0).stop 2023-11-06T03:07:48.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 msgr2=0x7f91841f7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f917800b410 tx=0x7f917800b8d0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 shutdown_connections 2023-11-06T03:07:48.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9164065a60 0x7f9164067f20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f918415e5e0 0x7f91841f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91841551e0 0x7f91841f7400 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 --2- 172.21.15.157:0/641508401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91840b3b20 0x7f918415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 >> 172.21.15.157:0/641508401 conn(0x7f9184065930 msgr2=0x7f91840afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:48.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 shutdown_connections 2023-11-06T03:07:48.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.841+0000 7f9188df8640 1 -- 172.21.15.157:0/641508401 wait complete. 2023-11-06T03:07:48.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:07:48.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.973+0000 7f23557ad640 1 Processor -- start 2023-11-06T03:07:48.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 -- start start 2023-11-06T03:07:48.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f235015ccc0 0x7f2350065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f23500b0ed0 0x7f23500cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23500662d0 con 0x7f2350065e70 2023-11-06T03:07:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2350070d30 con 0x7f235015ccc0 2023-11-06T03:07:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f23557ad640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23501606c0 con 0x7f23500b0ed0 2023-11-06T03:07:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f235015ccc0 0x7f2350065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234f7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f23500b0ed0 0x7f23500cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f235015ccc0 0x7f2350065930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47480/0 (socket says 172.21.15.157:47480) 2023-11-06T03:07:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 -- 172.21.15.157:0/2743348751 learned_addr learned my addr 172.21.15.157:0/2743348751 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 -- 172.21.15.157:0/2743348751 >> v2:172.21.15.195:3300/0 conn(0x7f23500b0ed0 msgr2=0x7f23500cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 --2- 172.21.15.157:0/2743348751 >> v2:172.21.15.195:3300/0 conn(0x7f23500b0ed0 0x7f23500cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 -- 172.21.15.157:0/2743348751 >> v1:172.21.15.195:6789/0 conn(0x7f2350065e70 legacy=0x7f23500b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.977+0000 7f234effd640 1 -- 172.21.15.157:0/2743348751 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350160910 con 0x7f235015ccc0 2023-11-06T03:07:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- 172.21.15.157:0/2743348751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015ccc0 0x7f2350065930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2350066850 tx=0x7f234401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9b9de644ca00cc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234dffb640 1 -- 172.21.15.157:0/2743348751 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440aa660 con 0x7f235015ccc0 2023-11-06T03:07:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234dffb640 1 -- 172.21.15.157:0/2743348751 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23440aa7c0 con 0x7f235015ccc0 2023-11-06T03:07:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234dffb640 1 -- 172.21.15.157:0/2743348751 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440aab50 con 0x7f235015ccc0 2023-11-06T03:07:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- 172.21.15.157:0/2743348751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015ccc0 msgr2=0x7f2350065930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- 172.21.15.157:0/2743348751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015ccc0 0x7f2350065930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2350066850 tx=0x7f234401c500 comp rx=0 tx=0).stop 2023-11-06T03:07:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- 172.21.15.157:0/2743348751 shutdown_connections 2023-11-06T03:07:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- 172.21.15.157:0/2743348751 >> v2:172.21.15.195:3300/0 conn(0x7f23500b0ed0 0x7f23500cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- 172.21.15.157:0/2743348751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015ccc0 0x7f2350065930 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- 172.21.15.157:0/2743348751 >> 172.21.15.157:0/2743348751 conn(0x7f23500732f0 msgr2=0x7f2350069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:48.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- 172.21.15.157:0/2743348751 shutdown_connections 2023-11-06T03:07:48.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- 172.21.15.157:0/2743348751 wait complete. 2023-11-06T03:07:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 Processor -- start 2023-11-06T03:07:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- start start 2023-11-06T03:07:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23500b0ed0 0x7f2350158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f235015ccc0 0x7f23500c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500c9f90 con 0x7f2350065e70 2023-11-06T03:07:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501581c0 con 0x7f23500b0ed0 2023-11-06T03:07:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f23557ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23501584a0 con 0x7f235015ccc0 2023-11-06T03:07:48.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47486/0 (socket says 172.21.15.157:47486) 2023-11-06T03:07:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 -- 172.21.15.157:0/2918569978 learned_addr learned my addr 172.21.15.157:0/2918569978 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234f7fe640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f235015ccc0 0x7f23500c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 -- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f235015ccc0 msgr2=0x7f23500c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234e7fc640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23500b0ed0 0x7f2350158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f235015ccc0 0x7f23500c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 -- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23500b0ed0 msgr2=0x7f2350158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23500b0ed0 0x7f2350158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 -- 172.21.15.157:0/2918569978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440094d0 con 0x7f2350065e70 2023-11-06T03:07:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.981+0000 7f234effd640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2344019110 tx=0x7f234401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440b3800 con 0x7f2350065e70 2023-11-06T03:07:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23440b3980 con 0x7f2350065e70 2023-11-06T03:07:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2350158780 con 0x7f2350065e70 2023-11-06T03:07:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440b3d30 con 0x7f2350065e70 2023-11-06T03:07:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2350146600 con 0x7f2350065e70 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2350072750 con 0x7f2350065e70 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23440c6390 con 0x7f2350065e70 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 0x7f233c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23440a6040 con 0x7f2350065e70 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f234e7fc640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 0x7f233c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.985+0000 7f234e7fc640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 0x7f233c067f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f235015fb10 tx=0x7f2340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:48.989+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f23440eb5d0 con 0x7f2350065e70 2023-11-06T03:07:49.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.165+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23500b44f0 con 0x7f2350065e70 2023-11-06T03:07:49.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.165+0000 7f23377fe640 1 -- 172.21.15.157:0/2918569978 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f23440aa620 con 0x7f2350065e70 2023-11-06T03:07:49.165 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:07:49.166 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:07:49.166 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:07:49.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 msgr2=0x7f233c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 0x7f233c067f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f235015fb10 tx=0x7f2340009040 comp rx=0 tx=0).stop 2023-11-06T03:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 msgr2=0x7f23501460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2344019110 tx=0x7f234401ad40 comp rx=0 tx=0).stop 2023-11-06T03:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 shutdown_connections 2023-11-06T03:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f233c065a60 0x7f233c067f20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f235015ccc0 0x7f23500c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23500b0ed0 0x7f2350158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 --2- 172.21.15.157:0/2918569978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2350065e70 0x7f23501460c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 >> 172.21.15.157:0/2918569978 conn(0x7f23500732f0 msgr2=0x7f23500b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 shutdown_connections 2023-11-06T03:07:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.169+0000 7f23557ad640 1 -- 172.21.15.157:0/2918569978 wait complete. 2023-11-06T03:07:49.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:07:39.644623+0000', 'modified': '2023-11-06T03:07:42.668622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5868}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5868': {'gid': 5868, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/3341473298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3341473298}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3341473298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:07:49.586 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:07:49.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:07:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 Processor -- start 2023-11-06T03:07:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 -- start start 2023-11-06T03:07:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7281599c0 0x7fc7280ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72815e440 con 0x7fc72806d000 2023-11-06T03:07:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72806c240 con 0x7fc7280b2af0 2023-11-06T03:07:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc72e99e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72806c520 con 0x7fc7281599c0 2023-11-06T03:07:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc727fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7281599c0 0x7fc7280ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc71f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc72806d000 0x7fc7280b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47436/0 (socket says 172.21.15.157:47436) 2023-11-06T03:07:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc71f7fe640 1 -- 172.21.15.157:0/3580867054 learned_addr learned my addr 172.21.15.157:0/3580867054 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:07:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc727fff640 1 -- 172.21.15.157:0/3580867054 >> v1:172.21.15.195:6789/0 conn(0x7fc7280b2af0 legacy=0x7fc728159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc727fff640 1 -- 172.21.15.157:0/3580867054 >> v1:172.21.15.157:6789/0 conn(0x7fc72806d000 legacy=0x7fc7280b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc7277fe640 1 -- 172.21.15.157:0/3580867054 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979567514 0 0) 0x7fc72815e440 con 0x7fc72806d000 2023-11-06T03:07:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.721+0000 7fc727fff640 1 -- 172.21.15.157:0/3580867054 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72806c800 con 0x7fc7281599c0 2023-11-06T03:07:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc727fff640 1 --2- 172.21.15.157:0/3580867054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7281599c0 0x7fc7280ae000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc7180097c0 tx=0x7fc71801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c560871173085a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc7277fe640 1 -- 172.21.15.157:0/3580867054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180ab670 con 0x7fc7281599c0 2023-11-06T03:07:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc7277fe640 1 -- 172.21.15.157:0/3580867054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180ab7d0 con 0x7fc7281599c0 2023-11-06T03:07:49.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc7277fe640 1 -- 172.21.15.157:0/3580867054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180abb60 con 0x7fc7281599c0 2023-11-06T03:07:49.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- 172.21.15.157:0/3580867054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7281599c0 msgr2=0x7fc7280ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 --2- 172.21.15.157:0/3580867054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7281599c0 0x7fc7280ae000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc7180097c0 tx=0x7fc71801c450 comp rx=0 tx=0).stop 2023-11-06T03:07:49.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- 172.21.15.157:0/3580867054 shutdown_connections 2023-11-06T03:07:49.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 --2- 172.21.15.157:0/3580867054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7281599c0 0x7fc7280ae000 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- 172.21.15.157:0/3580867054 >> 172.21.15.157:0/3580867054 conn(0x7fc728065930 msgr2=0x7fc728065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- 172.21.15.157:0/3580867054 shutdown_connections 2023-11-06T03:07:49.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- 172.21.15.157:0/3580867054 wait complete. 2023-11-06T03:07:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 Processor -- start 2023-11-06T03:07:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.725+0000 7fc72e99e640 1 -- start start 2023-11-06T03:07:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7280b2af0 0x7fc72814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 0x7fc72813b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72814e370 con 0x7fc72806d000 2023-11-06T03:07:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728138b90 con 0x7fc7281599c0 2023-11-06T03:07:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72813bde0 con 0x7fc7280b2af0 2023-11-06T03:07:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47496/0 (socket says 172.21.15.157:47496) 2023-11-06T03:07:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 -- 172.21.15.157:0/2953986557 learned_addr learned my addr 172.21.15.157:0/2953986557 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 -- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7280b2af0 msgr2=0x7fc72814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7280b2af0 0x7fc72814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 0x7fc72813b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7280b2af0 0x7fc72814de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 -- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 msgr2=0x7fc72813b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 0x7fc72813b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 -- 172.21.15.157:0/2953986557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718009420 con 0x7fc72806d000 2023-11-06T03:07:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 0x7fc72813b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc727fff640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ad040 tx=0x7fc71801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc7257fa640 1 -- 172.21.15.157:0/2953986557 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180b5810 con 0x7fc72806d000 2023-11-06T03:07:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc7257fa640 1 -- 172.21.15.157:0/2953986557 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180b5990 con 0x7fc72806d000 2023-11-06T03:07:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728139430 con 0x7fc72806d000 2023-11-06T03:07:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc7257fa640 1 -- 172.21.15.157:0/2953986557 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180b5d40 con 0x7fc72806d000 2023-11-06T03:07:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7281399f0 con 0x7fc72806d000 2023-11-06T03:07:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc731ac6000 1 Processor -- start 2023-11-06T03:07:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.729+0000 7fc731ac6000 1 -- start start 2023-11-06T03:07:49.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b496d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49686550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4954efc0 con 0x561b49686a90 2023-11-06T03:07:49.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b494dd1a0 con 0x561b496d06e0 2023-11-06T03:07:49.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b494da770 con 0x561b496d1000 2023-11-06T03:07:49.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47506/0 (socket says 172.21.15.157:47506) 2023-11-06T03:07:49.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 -- 172.21.15.157:0/2657286880 learned_addr learned my addr 172.21.15.157:0/2657286880 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc7257fa640 1 -- 172.21.15.157:0/2953986557 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7180c83f0 con 0x7fc72806d000 2023-11-06T03:07:49.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b496d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 -- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 msgr2=0x561b49686550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49686550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 -- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 msgr2=0x561b496d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b496d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 -- 172.21.15.157:0/2657286880 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b491d2890 con 0x561b49686a90 2023-11-06T03:07:49.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc7100016d0 tx=0x7fc710001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf373d858a762c11 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc7257fa640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 0x7fc708067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71dffb640 1 -- 172.21.15.157:0/2657286880 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100074e0 con 0x561b49686a90 2023-11-06T03:07:49.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc7257fa640 1 -- 172.21.15.157:0/2953986557 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc71811e0f0 con 0x7fc72806d000 2023-11-06T03:07:49.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71dffb640 1 -- 172.21.15.157:0/2657286880 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710007640 con 0x561b49686a90 2023-11-06T03:07:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc71dffb640 1 -- 172.21.15.157:0/2657286880 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100049f0 con 0x561b49686a90 2023-11-06T03:07:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 0x7fc708067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- 172.21.15.157:0/2657286880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 msgr2=0x561b49686e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc7100016d0 tx=0x7fc710001b90 comp rx=0 tx=0).stop 2023-11-06T03:07:49.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- 172.21.15.157:0/2657286880 shutdown_connections 2023-11-06T03:07:49.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b49686e70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49686550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- 172.21.15.157:0/2657286880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b496d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- 172.21.15.157:0/2657286880 >> 172.21.15.157:0/2657286880 conn(0x561b4968cbf0 msgr2=0x561b49684e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- 172.21.15.157:0/2657286880 shutdown_connections 2023-11-06T03:07:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- 172.21.15.157:0/2657286880 wait complete. 2023-11-06T03:07:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 0x7fc708067f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc720007fd0 tx=0x7fc720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- start start 2023-11-06T03:07:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 Processor -- start 2023-11-06T03:07:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b49686a90 0x561b49691880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49691dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b494d6060 con 0x561b496d1000 2023-11-06T03:07:49.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b49475290 con 0x561b496d06e0 2023-11-06T03:07:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b494936f0 con 0x561b49686a90 2023-11-06T03:07:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc731ac6000 10 client.0 _ll_register_callbacks cb 0x561b496721d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47518/0 (socket says 172.21.15.157:47518) 2023-11-06T03:07:49.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.733+0000 7fc72cf3b640 1 -- 172.21.15.157:0/3850341045 learned_addr learned my addr 172.21.15.157:0/3850341045 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc727fff640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b49686a90 0x561b49691880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49691dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 -- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b49686a90 msgr2=0x561b49691880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b49686a90 0x561b49691880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 -- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 msgr2=0x561b49691dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49691dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 -- 172.21.15.157:0/3850341045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc710001300 con 0x561b496d1000 2023-11-06T03:07:49.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49691dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:07:49.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc72cf3b640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7200147e0 tx=0x7fc720001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c2fbce9f38cea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:49.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720015980 con 0x561b496d1000 2023-11-06T03:07:49.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc720015ae0 con 0x561b496d1000 2023-11-06T03:07:49.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b494d49b0 con 0x561b496d1000 2023-11-06T03:07:49.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc720015ca0 con 0x561b496d1000 2023-11-06T03:07:49.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 10 client.6036 fetch_fsmap learned FSMap version 36 2023-11-06T03:07:49.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x561b4949b1e0 con 0x561b496d1000 2023-11-06T03:07:49.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b494d49b0 con 0x561b496d1000 2023-11-06T03:07:49.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc720012070 con 0x561b496d1000 2023-11-06T03:07:49.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 10 client.6036 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:07:49.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 10 client.6036 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:07:49.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 20 client.6036 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:49.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b496df770 0x561b496e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 4 client.6036 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:07:49.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b495054d0 con 0x561b496df770 2023-11-06T03:07:49.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc727fff640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b496df770 0x561b496e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 10 client.6036 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:07:49.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc727fff640 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b496df770 0x561b496e1b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a45bb0466e3bcf server_cookie=1a25297092f2e87d in_seq=0 out_seq=0 2023-11-06T03:07:49.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 1 -- 172.21.15.157:0/3850341045 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561b495054d0 con 0x561b496df770 2023-11-06T03:07:49.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 10 client.6036 handle_command_reply: tid=0 2023-11-06T03:07:49.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 client.6036 shutdown 2023-11-06T03:07:49.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 msgr2=0x561b49692300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7200147e0 tx=0x7fc720001260 comp rx=0 tx=0).stop 2023-11-06T03:07:49.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 shutdown_connections 2023-11-06T03:07:49.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b496df770 0x561b496e1b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d1000 0x561b49692300 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49691dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc706ffd640 0 client.6036 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:07:49.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3850341045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b49686a90 0x561b49691880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 >> 172.21.15.157:0/3850341045 conn(0x561b4968cbf0 msgr2=0x561b49685530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 shutdown_connections 2023-11-06T03:07:49.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 1 -- 172.21.15.157:0/3850341045 wait complete. 2023-11-06T03:07:49.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.737+0000 7fc731ac6000 20 client.6036 trim_cache size 0 max 16384 2023-11-06T03:07:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 Processor -- start 2023-11-06T03:07:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- start start 2023-11-06T03:07:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b49686a90 0x561b496e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b496cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b49193940 con 0x561b496d06e0 2023-11-06T03:07:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b494d0400 con 0x561b49686a90 2023-11-06T03:07:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b49689ee0 con 0x561b496d1000 2023-11-06T03:07:49.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47530/0 (socket says 172.21.15.157:47530) 2023-11-06T03:07:49.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 -- 172.21.15.157:0/3867021192 learned_addr learned my addr 172.21.15.157:0/3867021192 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 -- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 msgr2=0x561b496cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b496cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 -- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b49686a90 msgr2=0x561b496e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b49686a90 0x561b496e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 -- 172.21.15.157:0/3867021192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b49481330 con 0x561b496d06e0 2023-11-06T03:07:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc7100070a0 tx=0x7fc710006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a9eaab64370f03 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71e7fc640 1 -- 172.21.15.157:0/3867021192 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100072e0 con 0x561b496d06e0 2023-11-06T03:07:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71e7fc640 1 -- 172.21.15.157:0/3867021192 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7100074a0 con 0x561b496d06e0 2023-11-06T03:07:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc71e7fc640 1 -- 172.21.15.157:0/3867021192 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100047f0 con 0x561b496d06e0 2023-11-06T03:07:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- 172.21.15.157:0/3867021192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 msgr2=0x561b496e30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc7100070a0 tx=0x7fc710006000 comp rx=0 tx=0).stop 2023-11-06T03:07:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- 172.21.15.157:0/3867021192 shutdown_connections 2023-11-06T03:07:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b496cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b496d06e0 0x561b496e30b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3867021192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b49686a90 0x561b496e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- 172.21.15.157:0/3867021192 >> 172.21.15.157:0/3867021192 conn(0x561b4968cbf0 msgr2=0x561b4968bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- 172.21.15.157:0/3867021192 shutdown_connections 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- 172.21.15.157:0/3867021192 wait complete. 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.753+0000 7fc731ac6000 1 -- start start 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 Processor -- start 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49731af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49715220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b4967e2a0 con 0x561b49686a90 2023-11-06T03:07:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b494de430 con 0x561b496d06e0 2023-11-06T03:07:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b494cad30 con 0x561b496d1000 2023-11-06T03:07:49.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 10 client.0 _ll_register_callbacks cb 0x561b496529e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:07:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47544/0 (socket says 172.21.15.157:47544) 2023-11-06T03:07:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 -- 172.21.15.157:0/3589172906 learned_addr learned my addr 172.21.15.157:0/3589172906 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 -- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 msgr2=0x561b49715220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49715220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 -- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 msgr2=0x561b49731af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49731af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 -- 172.21.15.157:0/3589172906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b491d2890 con 0x561b49686a90 2023-11-06T03:07:49.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc727fff640 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ef3d0 tx=0x7fc7180eff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311862162b3e5204 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:07:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180b5ea0 con 0x561b49686a90 2023-11-06T03:07:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc718117df0 con 0x561b49686a90 2023-11-06T03:07:49.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b494a5b90 con 0x561b49686a90 2023-11-06T03:07:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7180ebde0 con 0x561b49686a90 2023-11-06T03:07:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 10 client.6042 fetch_fsmap learned FSMap version 36 2023-11-06T03:07:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x561b491d2cc0 con 0x561b49686a90 2023-11-06T03:07:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b494a5b90 con 0x561b49686a90 2023-11-06T03:07:49.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc7180eebc0 con 0x561b49686a90 2023-11-06T03:07:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 10 client.6042 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:07:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 10 client.6042 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:07:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 20 client.6042 populate_metadata read hostname 'smithi157' 2023-11-06T03:07:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b49715760 0x561b49715b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 4 client.6042 mds_command: new command op to 5868 tid=0[{"prefix": "status"}] 2023-11-06T03:07:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561b494cf0e0 con 0x561b49715760 2023-11-06T03:07:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b49715760 0x561b49715b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 10 client.6042 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:07:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc71f7fe640 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b49715760 0x561b49715b40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a3d7ed6cdcb61e server_cookie=34e0704c22581381 in_seq=0 out_seq=0 2023-11-06T03:07:49.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 1 -- 172.21.15.157:0/3589172906 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561b494cf0e0 con 0x561b49715760 2023-11-06T03:07:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 10 client.6042 handle_command_reply: tid=0 2023-11-06T03:07:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 client.6042 shutdown 2023-11-06T03:07:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 msgr2=0x561b497315b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ef3d0 tx=0x7fc7180eff10 comp rx=0 tx=0).stop 2023-11-06T03:07:49.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 shutdown_connections 2023-11-06T03:07:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x561b49715760 0x561b49715b40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x561b496d1000 0x561b49715220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561b496d06e0 0x561b49731af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc706ffd640 0 client.6042 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:07:49.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 --2- 172.21.15.157:0/3589172906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561b49686a90 0x561b497315b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 >> 172.21.15.157:0/3589172906 conn(0x561b4968cbf0 msgr2=0x561b496dfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 shutdown_connections 2023-11-06T03:07:49.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 1 -- 172.21.15.157:0/3589172906 wait complete. 2023-11-06T03:07:49.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc731ac6000 20 client.6042 trim_cache size 0 max 16384 2023-11-06T03:07:49.781 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:07:49.781 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:07:49.781 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:07:49.781 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 5868, 2023-11-06T03:07:49.781 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:07:49.782 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:07:49.782 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:07:49.782 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 10.111234318999999, 2023-11-06T03:07:49.782 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 36, 2023-11-06T03:07:49.782 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 54, 2023-11-06T03:07:49.783 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 53, 2023-11-06T03:07:49.783 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 15.363410206999999 2023-11-06T03:07:49.783 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:07:49.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 msgr2=0x7fc708067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 0x7fc708067f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc720007fd0 tx=0x7fc720009040 comp rx=0 tx=0).stop 2023-11-06T03:07:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 msgr2=0x7fc728138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ad040 tx=0x7fc71801ac90 comp rx=0 tx=0).stop 2023-11-06T03:07:49.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 shutdown_connections 2023-11-06T03:07:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc708065a90 0x7fc708067f50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7281599c0 0x7fc72813b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7280b2af0 0x7fc72814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 --2- 172.21.15.157:0/2953986557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc72806d000 0x7fc728138e50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 >> 172.21.15.157:0/2953986557 conn(0x7fc728065930 msgr2=0x7fc72815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 shutdown_connections 2023-11-06T03:07:49.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.757+0000 7fc72e99e640 1 -- 172.21.15.157:0/2953986557 wait complete. 2023-11-06T03:07:49.787 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:07:49.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.901+0000 7f3453f75640 1 Processor -- start 2023-11-06T03:07:49.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.901+0000 7f3453f75640 1 -- start start 2023-11-06T03:07:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3453f75640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c15e5e0 0x7f344c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3453f75640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15ea90 0x7f344c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3453f75640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c15f580 con 0x7f344c1551e0 2023-11-06T03:07:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3453f75640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c0704b0 con 0x7f344c15e5e0 2023-11-06T03:07:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3453f75640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c15efc0 con 0x7f344c15ea90 2023-11-06T03:07:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c15e5e0 0x7f344c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c15e5e0 0x7f344c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47560/0 (socket says 172.21.15.157:47560) 2023-11-06T03:07:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 -- 172.21.15.157:0/1304329714 learned_addr learned my addr 172.21.15.157:0/1304329714 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3452512640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15ea90 0x7f344c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 -- 172.21.15.157:0/1304329714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15ea90 msgr2=0x7f344c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15ea90 0x7f344c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 -- 172.21.15.157:0/1304329714 >> v1:172.21.15.195:6789/0 conn(0x7f344c1551e0 legacy=0x7f344c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:07:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 -- 172.21.15.157:0/1304329714 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c15f2a0 con 0x7f344c15e5e0 2023-11-06T03:07:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3451510640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15e5e0 0x7f344c0b3410 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b967905f96e7247 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.905+0000 7f3450d0f640 1 -- 172.21.15.157:0/1304329714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0ab600 con 0x7f344c15e5e0 2023-11-06T03:07:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3450d0f640 1 -- 172.21.15.157:0/1304329714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0ab760 con 0x7f344c15e5e0 2023-11-06T03:07:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3450d0f640 1 -- 172.21.15.157:0/1304329714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0abaf0 con 0x7f344c15e5e0 2023-11-06T03:07:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/1304329714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15e5e0 msgr2=0x7f344c0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15e5e0 0x7f344c0b3410 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c01c440 comp rx=0 tx=0).stop 2023-11-06T03:07:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/1304329714 shutdown_connections 2023-11-06T03:07:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15ea90 0x7f344c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- 172.21.15.157:0/1304329714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15e5e0 0x7f344c0b3410 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/1304329714 >> 172.21.15.157:0/1304329714 conn(0x7f344c065930 msgr2=0x7f344c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/1304329714 shutdown_connections 2023-11-06T03:07:49.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/1304329714 wait complete. 2023-11-06T03:07:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 Processor -- start 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- start start 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 0x7f344c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 0x7f344c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0ce8b0 con 0x7f344c15ea90 2023-11-06T03:07:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0cdd30 con 0x7f344c1551e0 2023-11-06T03:07:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0ce010 con 0x7f344c15e5e0 2023-11-06T03:07:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47576/0 (socket says 172.21.15.157:47576) 2023-11-06T03:07:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 -- 172.21.15.157:0/466113549 learned_addr learned my addr 172.21.15.157:0/466113549 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:07:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3451510640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 0x7f344c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3451d11640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 0x7f344c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 -- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 msgr2=0x7f344c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 0x7f344c0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 -- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 msgr2=0x7f344c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:49.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 0x7f344c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:49.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 -- 172.21.15.157:0/466113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0094d0 con 0x7f344c15ea90 2023-11-06T03:07:49.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3451510640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 0x7f344c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:49.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3451d11640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 0x7f344c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:07:49.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3452512640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f344400cc30 tx=0x7f344400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344400ce40 con 0x7f344c15ea90 2023-11-06T03:07:49.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34440039a0 con 0x7f344c15ea90 2023-11-06T03:07:49.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444003d30 con 0x7f344c15ea90 2023-11-06T03:07:49.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c0ce2f0 con 0x7f344c15ea90 2023-11-06T03:07:49.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.909+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c0cd970 con 0x7f344c15ea90 2023-11-06T03:07:49.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c072750 con 0x7f344c15ea90 2023-11-06T03:07:49.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3444003e90 con 0x7f344c15ea90 2023-11-06T03:07:49.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f34427fc640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:07:49.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f34440837a0 con 0x7f344c15ea90 2023-11-06T03:07:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f3451d11640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:07:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.913+0000 7f3451d11640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3448003e40 tx=0x7f3448009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:07:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:49.917+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3444050f20 con 0x7f344c15ea90 2023-11-06T03:07:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.101+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f344c071730 con 0x7f344c15ea90 2023-11-06T03:07:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.101+0000 7f34427fc640 1 -- 172.21.15.157:0/466113549 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7f344401d070 con 0x7f344c15ea90 2023-11-06T03:07:50.102 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:07:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 msgr2=0x7f3430067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3448003e40 tx=0x7f3448009040 comp rx=0 tx=0).stop 2023-11-06T03:07:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 msgr2=0x7f344c1f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:07:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f344400cc30 tx=0x7f344400a5c0 comp rx=0 tx=0).stop 2023-11-06T03:07:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 shutdown_connections 2023-11-06T03:07:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c15ea90 0x7f344c1f5800 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c15e5e0 0x7f344c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 --2- 172.21.15.157:0/466113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c1551e0 0x7f344c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:07:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 >> 172.21.15.157:0/466113549 conn(0x7f344c065930 msgr2=0x7f344c14e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:07:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 shutdown_connections 2023-11-06T03:07:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:07:50.105+0000 7f3453f75640 1 -- 172.21.15.157:0/466113549 wait complete. 2023-11-06T03:07:50.119 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:07:50.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:07:50.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:07:50.121 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:07:50.121 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:50.121 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:07:50.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:07:50.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:07:50.122 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:07:50.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.123 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:07:50.129 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:07:50.129 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:07:50.129 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:07:50.130 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.130 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:07:50.130 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.130 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:07:50.130 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.131 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:07:50.131 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:07:50.131 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.131 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:07:50.131 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:07:50.132 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.132 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:07:50.132 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.132 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:07:50.132 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42269sec preferred_lft 42269sec 2023-11-06T03:07:50.133 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:07:50.133 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.133 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:07:50.133 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.133 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:07:50.134 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.134 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:07:50.134 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:07:50.134 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:07:50.134 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.135 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:07:50.135 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.135 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:07:50.135 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:07:50.135 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:07:50.136 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:07:50.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.137 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:07:50.179 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:07:50.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.180 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:07:50.233 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:07:50.233 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:07:50.234 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:07:50.234 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:07:50.234 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:07:50.235 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:07:50.235 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:07:50.235 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:07:50.236 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:07:50.236 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:07:50.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.237 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:07:50.238 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:07:50.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:07:50.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:07:50.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:07:50.241 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:07:50.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.307 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:07:50.355 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:07:50.355 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:07:50.356 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:07:50.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:07:50.409 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:07:50.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:07:50.410 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:50.464 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:07:50.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:07:50.465 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:50.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:07:50.568 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:07:50.569 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:07:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:07:50.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:07:50.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.628 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:07:50.675 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:07:50.676 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:07:50.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:07:50.736 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:07:50.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:07:50.789 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:07:50.790 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:07:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:50.791 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:07:50.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:07:50.805+0000 7fc673f76580 -1 init, newargv = 0x55c3ae609da0 newargc=15 2023-11-06T03:07:50.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[62277]: starting ceph client 2023-11-06T03:07:50.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[62277]: starting fuse 2023-11-06T03:07:51.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:07:51.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:07:51.926 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:07:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:07:51.980 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:07:51.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:51.981 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:07:52.027 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:07:52.028 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:07:52.028 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:07:52.028 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:07:52.029 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:07:52.029 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:07:52.030 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:07:52.030 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.030 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:07:52.031 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:07:52.031 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:07:52.032 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:07:52.032 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.032 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:07:52.033 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:07:52.033 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:07:52.033 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:07:52.034 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.034 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:07:52.035 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:07:52.035 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:07:52.035 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:07:52.036 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:07:52.036 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:07:52.037 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:07:52.037 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:07:52.037 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.038 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:07:52.038 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:07:52.148 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.62277.asok 2023-11-06T03:07:52.331 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62277.asok 2023-11-06T03:07:52.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:52.332 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62277.asok status 2023-11-06T03:07:52.490 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:07:52.490 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:07:52.491 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:07:52.491 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:07:52.491 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:07:52.491 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:07:52.491 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:07:52.492 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "62277", 2023-11-06T03:07:52.492 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:07:52.492 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:52.492 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:07:52.492 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:07:52.493 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6057, 2023-11-06T03:07:52.493 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:07:52.493 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:07:52.493 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:07:52.493 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6057 2023-11-06T03:07:52.494 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:52.494 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:07:52.494 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:07:52.494 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:07:52.494 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3021514631 2023-11-06T03:07:52.495 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:07:52.495 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:52.495 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:07:52.495 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:07:52.495 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:07:52.496 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3021514631 2023-11-06T03:07:52.496 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:07:52.496 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6057 192.168.0.1:0/3021514631", 2023-11-06T03:07:52.496 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3021514631", 2023-11-06T03:07:52.496 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 36, 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 55, 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:07:52.497 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:07:52.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:52.504 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:07:52.504 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:52.510 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:07:52.510 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:52.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:52.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:52.566 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2023-11-06T03:07:52.566 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2023-11-06T03:07:52.566 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2023-11-06T03:07:52.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:52.567 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2023-11-06T03:07:52.621 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:07:52.622 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:07:52.622 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:07:52.622 DEBUG:teuthology.orchestra.run.smithi157:> import json 2023-11-06T03:07:52.623 DEBUG:teuthology.orchestra.run.smithi157:> import sys 2023-11-06T03:07:52.623 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.623 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:07:52.624 DEBUG:teuthology.orchestra.run.smithi157:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2023-11-06T03:07:52.624 DEBUG:teuthology.orchestra.run.smithi157:> except OSError as e: 2023-11-06T03:07:52.624 DEBUG:teuthology.orchestra.run.smithi157:> sys.exit(e.errno) 2023-11-06T03:07:52.625 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:07:52.625 DEBUG:teuthology.orchestra.run.smithi157:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-06T03:07:52.625 DEBUG:teuthology.orchestra.run.smithi157:> print(json.dumps( 2023-11-06T03:07:52.626 DEBUG:teuthology.orchestra.run.smithi157:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-06T03:07:52.626 DEBUG:teuthology.orchestra.run.smithi157:> indent=2)) 2023-11-06T03:07:52.627 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:07:52.905 INFO:teuthology.orchestra.run.smithi157.stderr:daemon-helper: command failed with exit status 2 2023-11-06T03:07:52.912 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-06T03:07:52.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:07:52.914 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2023-11-06T03:07:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2023-11-06T03:07:53.340 INFO:teuthology.orchestra.run.smithi157.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2023-11-06T03:08:04.950 INFO:teuthology.orchestra.run.smithi157.stderr:Updating files: 13% (188/1398) Updating files: 14% (196/1398) Updating files: 15% (210/1398) Updating files: 16% (224/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 34% (477/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 51% (720/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 70% (985/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 85% (1201/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2023-11-06T03:08:04.958 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2023-11-06T03:08:04.958 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2023-11-06T03:08:04.959 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.113+0000 7efc409fd640 1 Processor -- start 2023-11-06T03:08:05.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.113+0000 7efc409fd640 1 -- start start 2023-11-06T03:08:05.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc409fd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efc3c06d000 0x7efc3c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc409fd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c15e440 con 0x7efc3c1599c0 2023-11-06T03:08:05.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc409fd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c06c240 con 0x7efc3c0b2af0 2023-11-06T03:08:05.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc409fd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c06c520 con 0x7efc3c06d000 2023-11-06T03:08:05.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efc3c1599c0 0x7efc3c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34976/0 (socket says 172.21.15.157:34976) 2023-11-06T03:08:05.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3a59c640 1 -- 172.21.15.157:0/450516091 learned_addr learned my addr 172.21.15.157:0/450516091 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:05.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118028362 0 0) 0x7efc3c15e440 con 0x7efc3c1599c0 2023-11-06T03:08:05.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 --2- 172.21.15.157:0/450516091 >> v2:172.21.15.195:3300/0 conn(0x7efc3c06d000 0x7efc3c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc28003530 con 0x7efc3c1599c0 2023-11-06T03:08:05.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049933971 0 0) 0x7efc28003530 con 0x7efc3c1599c0 2023-11-06T03:08:05.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 >> v2:172.21.15.195:3300/0 conn(0x7efc3c06d000 msgr2=0x7efc3c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 --2- 172.21.15.157:0/450516091 >> v2:172.21.15.195:3300/0 conn(0x7efc3c06d000 0x7efc3c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 >> v1:172.21.15.195:6789/0 conn(0x7efc3c0b2af0 legacy=0x7efc3c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c06c800 con 0x7efc3c1599c0 2023-11-06T03:08:05.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7efc2c001680 con 0x7efc3c1599c0 2023-11-06T03:08:05.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 >> v1:172.21.15.157:6789/0 conn(0x7efc3c1599c0 legacy=0x7efc3c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc28003980 0x7efc28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc280047d0 0x7efc28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c06c520 con 0x7efc28005190 2023-11-06T03:08:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2800c030 con 0x7efc28003980 2023-11-06T03:08:05.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2800c1f0 con 0x7efc280047d0 2023-11-06T03:08:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3a59c640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc28003980 0x7efc28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc31d9b640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc280047d0 0x7efc28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 -- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc280047d0 msgr2=0x7efc28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc280047d0 0x7efc28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 -- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc28003980 msgr2=0x7efc28003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc28003980 0x7efc28003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc3ad9d640 1 -- 172.21.15.157:0/450516091 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28003530 con 0x7efc28005190 2023-11-06T03:08:05.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.117+0000 7efc31d9b640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc280047d0 0x7efc28004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:05.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3ad9d640 1 -- 172.21.15.157:0/450516091 reap_dead start 2023-11-06T03:08:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3ad9d640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc2800b990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efc34009fc0 tx=0x7efc3401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e414ba0409318d3f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340a5cb0 con 0x7efc28005190 2023-11-06T03:08:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400f040 con 0x7efc28005190 2023-11-06T03:08:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc39d9b640 1 -- 172.21.15.157:0/450516091 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34003be0 con 0x7efc28005190 2023-11-06T03:08:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 msgr2=0x7efc2800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc2800b990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7efc34009fc0 tx=0x7efc3401cca0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- 172.21.15.157:0/450516091 shutdown_connections 2023-11-06T03:08:05.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 --2- 172.21.15.157:0/450516091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc2800b990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- 172.21.15.157:0/450516091 >> 172.21.15.157:0/450516091 conn(0x7efc3c065930 msgr2=0x7efc3c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- 172.21.15.157:0/450516091 shutdown_connections 2023-11-06T03:08:05.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- 172.21.15.157:0/450516091 wait complete. 2023-11-06T03:08:05.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 Processor -- start 2023-11-06T03:08:05.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- start start 2023-11-06T03:08:05.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c0cbcd0 0x7efc3c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3c14bd90 0x7efc3c14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c153c30 con 0x7efc28005190 2023-11-06T03:08:05.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15adf0 con 0x7efc3c0cbcd0 2023-11-06T03:08:05.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc409fd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15b0d0 con 0x7efc3c14bd90 2023-11-06T03:08:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44656/0 (socket says 172.21.15.157:44656) 2023-11-06T03:08:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 -- 172.21.15.157:0/3324225772 learned_addr learned my addr 172.21.15.157:0/3324225772 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 -- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3c14bd90 msgr2=0x7efc3c14c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3c14bd90 0x7efc3c14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc31d9b640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c0cbcd0 0x7efc3c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 -- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c0cbcd0 msgr2=0x7efc3c0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c0cbcd0 0x7efc3c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.121+0000 7efc3a59c640 1 -- 172.21.15.157:0/3324225772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c15b3b0 con 0x7efc28005190 2023-11-06T03:08:05.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc3a59c640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e780 tx=0x7efc2c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00ee30 con 0x7efc28005190 2023-11-06T03:08:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c00c8d0 con 0x7efc28005190 2023-11-06T03:08:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c15b690 con 0x7efc28005190 2023-11-06T03:08:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00cc60 con 0x7efc28005190 2023-11-06T03:08:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c0ce180 con 0x7efc28005190 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c15d630 con 0x7efc28005190 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efc2c01d460 con 0x7efc28005190 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 0x7efc1c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efc2c0831d0 con 0x7efc28005190 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc31d9b640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 0x7efc1c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.125+0000 7efc31d9b640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 0x7efc1c067f50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc24000f70 tx=0x7efc24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.129+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efc2c0509f0 con 0x7efc28005190 2023-11-06T03:08:05.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.317+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc3c0adcd0 con 0x7efc28005190 2023-11-06T03:08:05.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.317+0000 7efc337fe640 1 -- 172.21.15.157:0/3324225772 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7efc2c054010 con 0x7efc28005190 2023-11-06T03:08:05.318 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:05.318 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:05.319 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:08:05.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 msgr2=0x7efc1c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 0x7efc1c067f50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc24000f70 tx=0x7efc24009040 comp rx=0 tx=0).stop 2023-11-06T03:08:05.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 msgr2=0x7efc3c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e780 tx=0x7efc2c00ec40 comp rx=0 tx=0).stop 2023-11-06T03:08:05.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 shutdown_connections 2023-11-06T03:08:05.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efc1c065a90 0x7efc1c067f50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3c14bd90 0x7efc3c14c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc3c0cbcd0 0x7efc3c0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 --2- 172.21.15.157:0/3324225772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc28005190 0x7efc3c0ccbc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 >> 172.21.15.157:0/3324225772 conn(0x7efc3c065930 msgr2=0x7efc3c0af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 shutdown_connections 2023-11-06T03:08:05.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.321+0000 7efc409fd640 1 -- 172.21.15.157:0/3324225772 wait complete. 2023-11-06T03:08:05.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2023-11-06T03:08:05.453 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:05.453 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": -2 2023-11-06T03:08:05.453 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:05.468 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2023-11-06T03:08:05.469 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2023-11-06T03:08:05.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2023-11-06T03:08:05.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 Processor -- start 2023-11-06T03:08:05.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 -- start start 2023-11-06T03:08:05.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8cc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc15f580 con 0x7fe8cc1551e0 2023-11-06T03:08:05.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc15efc0 con 0x7fe8cc15e5e0 2023-11-06T03:08:05.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8d17d7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0704b0 con 0x7fe8cc0b3b20 2023-11-06T03:08:05.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8caffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe8cc1551e0 0x7fe8cc06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34980/0 (socket says 172.21.15.157:34980) 2023-11-06T03:08:05.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8caffd640 1 -- 172.21.15.157:0/1602901466 learned_addr learned my addr 172.21.15.157:0/1602901466 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:05.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983377129 0 0) 0x7fe8cc15f580 con 0x7fe8cc1551e0 2023-11-06T03:08:05.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b4003530 con 0x7fe8cc1551e0 2023-11-06T03:08:05.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8cc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457752090 0 0) 0x7fe8b4003530 con 0x7fe8cc1551e0 2023-11-06T03:08:05.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 msgr2=0x7fe8cc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8cc15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 >> v1:172.21.15.195:6789/0 conn(0x7fe8cc15e5e0 legacy=0x7fe8cc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc15f2a0 con 0x7fe8cc1551e0 2023-11-06T03:08:05.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe8bc001680 con 0x7fe8cc1551e0 2023-11-06T03:08:05.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 >> v1:172.21.15.157:6789/0 conn(0x7fe8cc1551e0 legacy=0x7fe8cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8b4003fe0 0x7fe8b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b4004d10 0x7fe8b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0704b0 con 0x7fe8b4004d10 2023-11-06T03:08:05.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400bbd0 con 0x7fe8b4003fe0 2023-11-06T03:08:05.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b400bd90 con 0x7fe8cc0b3b20 2023-11-06T03:08:05.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b4004d10 0x7fe8b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8b4003fe0 0x7fe8b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 msgr2=0x7fe8b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8b4003fe0 msgr2=0x7fe8b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8b4003fe0 0x7fe8b40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1602901466 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4003530 con 0x7fe8b4004d10 2023-11-06T03:08:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.593+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc0b3b20 0x7fe8b40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b4004d10 0x7fe8b4005190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007330 tx=0x7fe8c401ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96a35a9b22cf0a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40a5d00 con 0x7fe8b4004d10 2023-11-06T03:08:05.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c40a5e60 con 0x7fe8b4004d10 2023-11-06T03:08:05.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8c9ffb640 1 -- 172.21.15.157:0/1602901466 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40ad660 con 0x7fe8b4004d10 2023-11-06T03:08:05.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1602901466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b4004d10 msgr2=0x7fe8b4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1602901466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8b4004d10 0x7fe8b4005190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007330 tx=0x7fe8c401ce00 comp rx=0 tx=0).stop 2023-11-06T03:08:05.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1602901466 reap_dead start 2023-11-06T03:08:05.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1602901466 shutdown_connections 2023-11-06T03:08:05.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1602901466 >> 172.21.15.157:0/1602901466 conn(0x7fe8cc065930 msgr2=0x7fe8cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1602901466 shutdown_connections 2023-11-06T03:08:05.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1602901466 wait complete. 2023-11-06T03:08:05.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 Processor -- start 2023-11-06T03:08:05.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- start start 2023-11-06T03:08:05.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 0x7fe8cc13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc157740 0x7fe8cc13c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc13ffc0 con 0x7fe8cc0b2230 2023-11-06T03:08:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc140b40 con 0x7fe8cc0b1e50 2023-11-06T03:08:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8d17d7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc140210 con 0x7fe8cc157740 2023-11-06T03:08:05.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44666/0 (socket says 172.21.15.157:44666) 2023-11-06T03:08:05.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.597+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1854079834 learned_addr learned my addr 172.21.15.157:0/1854079834 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc157740 0x7fe8cc13c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 0x7fe8cc13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc157740 msgr2=0x7fe8cc13c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc157740 0x7fe8cc13c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 msgr2=0x7fe8cc13bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 0x7fe8cc13bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1854079834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc1404f0 con 0x7fe8cc0b2230 2023-11-06T03:08:05.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 0x7fe8cc13bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:05.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4002240 tx=0x7fe8c4019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 -- 172.21.15.157:0/1854079834 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40aeb60 con 0x7fe8cc0b2230 2023-11-06T03:08:05.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 -- 172.21.15.157:0/1854079834 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c40aecc0 con 0x7fe8cc0b2230 2023-11-06T03:08:05.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc1407d0 con 0x7fe8cc0b2230 2023-11-06T03:08:05.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 -- 172.21.15.157:0/1854079834 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4003be0 con 0x7fe8cc0b2230 2023-11-06T03:08:05.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc1f4410 con 0x7fe8cc0b2230 2023-11-06T03:08:05.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 -- 172.21.15.157:0/1854079834 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe8c40b4070 con 0x7fe8cc0b2230 2023-11-06T03:08:05.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 0x7fe8b8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8b37fe640 1 -- 172.21.15.157:0/1854079834 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe8c411d320 con 0x7fe8cc0b2230 2023-11-06T03:08:05.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 0x7fe8b8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8d48ff000 1 Processor -- start 2023-11-06T03:08:05.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8d48ff000 1 -- start start 2023-11-06T03:08:05.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.601+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 0x7fe8b8067f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc154460 tx=0x7fe8bc0015f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e466cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e46680c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e46456050 con 0x555e466811d0 2023-11-06T03:08:05.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e464d6140 con 0x555e466cb350 2023-11-06T03:08:05.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e464d3710 con 0x555e466caa30 2023-11-06T03:08:05.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44668/0 (socket says 172.21.15.157:44668) 2023-11-06T03:08:05.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1628093952 learned_addr learned my addr 172.21.15.157:0/1628093952 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e466cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e46680c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 msgr2=0x555e466cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e466cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 msgr2=0x555e46680c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e46680c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 -- 172.21.15.157:0/1628093952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e461cbcb0 con 0x555e466811d0 2023-11-06T03:08:05.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ead90 tx=0x7fe8c40ee1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797499c4c66606c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8b0ff9640 1 -- 172.21.15.157:0/1628093952 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40ee890 con 0x555e466811d0 2023-11-06T03:08:05.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8b0ff9640 1 -- 172.21.15.157:0/1628093952 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c40f26c0 con 0x555e466811d0 2023-11-06T03:08:05.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8b0ff9640 1 -- 172.21.15.157:0/1628093952 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40f6870 con 0x555e466811d0 2023-11-06T03:08:05.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1628093952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 msgr2=0x555e466815b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ead90 tx=0x7fe8c40ee1f0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1628093952 shutdown_connections 2023-11-06T03:08:05.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466815b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e46680c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1628093952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e466cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1628093952 >> 172.21.15.157:0/1628093952 conn(0x555e4667d180 msgr2=0x555e4667f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1628093952 shutdown_connections 2023-11-06T03:08:05.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1628093952 wait complete. 2023-11-06T03:08:05.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- start start 2023-11-06T03:08:05.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 Processor -- start 2023-11-06T03:08:05.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e4668c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e4668cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e464cf000 con 0x555e466811d0 2023-11-06T03:08:05.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4646e9d0 con 0x555e466cb350 2023-11-06T03:08:05.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e4646e670 con 0x555e466caa30 2023-11-06T03:08:05.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 10 client.0 _ll_register_callbacks cb 0x555e4666c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:05.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44676/0 (socket says 172.21.15.157:44676) 2023-11-06T03:08:05.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 -- 172.21.15.157:0/1307155559 learned_addr learned my addr 172.21.15.157:0/1307155559 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e4668c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e4668cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 -- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 msgr2=0x555e4668c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e4668c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 -- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 msgr2=0x555e4668cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e4668cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 -- 172.21.15.157:0/1307155559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc1f4410 con 0x555e466811d0 2023-11-06T03:08:05.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8caffd640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0167d0 tx=0x7fe8bc0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3ef1010b86f2ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:05.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc015830 con 0x555e466811d0 2023-11-06T03:08:05.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe8bc01e070 con 0x555e466811d0 2023-11-06T03:08:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.605+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4649c7a0 con 0x555e466811d0 2023-11-06T03:08:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8bc015990 con 0x555e466811d0 2023-11-06T03:08:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 10 client.6075 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x555e464942f0 con 0x555e466811d0 2023-11-06T03:08:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e4649c7a0 con 0x555e466811d0 2023-11-06T03:08:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe8bc015af0 con 0x555e466811d0 2023-11-06T03:08:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 10 client.6075 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 20 client.6075 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e466d2080 0x555e466d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 4 client.6075 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e464cd950 con 0x555e466d2080 2023-11-06T03:08:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e466d2080 0x555e466d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 10 client.6075 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e466d2080 0x555e466d2490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73dcfd11938ce90 server_cookie=483b13179bd9dbfb in_seq=0 out_seq=0 2023-11-06T03:08:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/1307155559 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555e464cd950 con 0x555e466d2080 2023-11-06T03:08:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 10 client.6075 handle_command_reply: tid=0 2023-11-06T03:08:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 client.6075 shutdown 2023-11-06T03:08:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 msgr2=0x555e466863a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0167d0 tx=0x7fe8bc0047f0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 shutdown_connections 2023-11-06T03:08:05.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e466d2080 0x555e466d2490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e4668cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8a9ffb640 0 client.6075 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:05.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e4668c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/1307155559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466811d0 0x555e466863a0 secure :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0167d0 tx=0x7fe8bc0047f0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 >> 172.21.15.157:0/1307155559 conn(0x555e4667d180 msgr2=0x555e4667fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 shutdown_connections 2023-11-06T03:08:05.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 1 -- 172.21.15.157:0/1307155559 wait complete. 2023-11-06T03:08:05.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.609+0000 7fe8d48ff000 20 client.6075 trim_cache size 0 max 16384 2023-11-06T03:08:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 Processor -- start 2023-11-06T03:08:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 -- start start 2023-11-06T03:08:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466cb350 0x555e46680ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466c77e0 0x555e466c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4618cd60 con 0x555e466caa30 2023-11-06T03:08:05.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e464c8450 con 0x555e466c77e0 2023-11-06T03:08:05.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e46487e20 con 0x555e466cb350 2023-11-06T03:08:05.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44688/0 (socket says 172.21.15.157:44688) 2023-11-06T03:08:05.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.621+0000 7fe8caffd640 1 -- 172.21.15.157:0/3878383658 learned_addr learned my addr 172.21.15.157:0/3878383658 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466cb350 msgr2=0x555e46680ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466cb350 0x555e46680ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466c77e0 msgr2=0x555e466c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466c77e0 0x555e466c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/3878383658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4647a440 con 0x555e466caa30 2023-11-06T03:08:05.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0126f0 tx=0x7fe8bc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd429654c82fc2e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8b17fa640 1 -- 172.21.15.157:0/3878383658 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00e040 con 0x555e466caa30 2023-11-06T03:08:05.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8b17fa640 1 -- 172.21.15.157:0/3878383658 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc0153e0 con 0x555e466caa30 2023-11-06T03:08:05.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8b17fa640 1 -- 172.21.15.157:0/3878383658 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc015830 con 0x555e466caa30 2023-11-06T03:08:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/3878383658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 msgr2=0x555e46680580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0126f0 tx=0x7fe8bc01e000 comp rx=0 tx=0).stop 2023-11-06T03:08:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/3878383658 shutdown_connections 2023-11-06T03:08:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466c77e0 0x555e466c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466cb350 0x555e46680ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/3878383658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466caa30 0x555e46680580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/3878383658 >> 172.21.15.157:0/3878383658 conn(0x555e4667d180 msgr2=0x555e4668c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/3878383658 shutdown_connections 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/3878383658 wait complete. 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- start start 2023-11-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 Processor -- start 2023-11-06T03:08:05.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e467131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e467136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e46490480 con 0x555e466c77e0 2023-11-06T03:08:05.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e464cde90 con 0x555e466cb350 2023-11-06T03:08:05.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e464d3710 con 0x555e466caa30 2023-11-06T03:08:05.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 10 client.0 _ll_register_callbacks cb 0x555e4647d6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:05.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44692/0 (socket says 172.21.15.157:44692) 2023-11-06T03:08:05.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/2501736263 learned_addr learned my addr 172.21.15.157:0/2501736263 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e467131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8cb7fe640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e467136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 msgr2=0x555e467131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e467131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 msgr2=0x555e467136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e467136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 -- 172.21.15.157:0/2501736263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e464942f0 con 0x555e466c77e0 2023-11-06T03:08:05.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8caffd640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0126f0 tx=0x7fe8bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b93f3066e2153ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:05.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0153e0 con 0x555e466c77e0 2023-11-06T03:08:05.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe8bc01e040 con 0x555e466c77e0 2023-11-06T03:08:05.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e464c8080 con 0x555e466c77e0 2023-11-06T03:08:05.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8bc00e040 con 0x555e466c77e0 2023-11-06T03:08:05.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 10 client.6081 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:05.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x555e461cc0e0 con 0x555e466c77e0 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e464c8080 con 0x555e466c77e0 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe8bc037070 con 0x555e466c77e0 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 10 client.6081 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 10 client.6081 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 20 client.6081 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e46713c30 0x555e4672c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 4 client.6081 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2023-11-06T03:08:05.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x555e464c7da0 con 0x555e46713c30 2023-11-06T03:08:05.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.625+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e46713c30 0x555e4672c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8a9ffb640 10 client.6081 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:05.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8ca7fc640 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e46713c30 0x555e4672c510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c09538200a36f5 server_cookie=3d03ddc21d99639c in_seq=0 out_seq=0 2023-11-06T03:08:05.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8a9ffb640 1 -- 172.21.15.157:0/2501736263 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x555e464c7da0 con 0x555e46713c30 2023-11-06T03:08:05.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8a9ffb640 10 client.6081 handle_command_reply: tid=0 2023-11-06T03:08:05.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 client.6081 shutdown 2023-11-06T03:08:05.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 msgr2=0x555e466c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0126f0 tx=0x7fe8bc017000 comp rx=0 tx=0).stop 2023-11-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 shutdown_connections 2023-11-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555e46713c30 0x555e4672c510 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555e466cb350 0x555e467136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555e466caa30 0x555e467131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8a9ffb640 0 client.6081 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 --2- 172.21.15.157:0/2501736263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555e466c77e0 0x555e466c9e40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 >> 172.21.15.157:0/2501736263 conn(0x555e4667d180 msgr2=0x555e4667f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 shutdown_connections 2023-11-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 1 -- 172.21.15.157:0/2501736263 wait complete. 2023-11-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d48ff000 20 client.6081 trim_cache size 0 max 16384 2023-11-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": -2 2023-11-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 msgr2=0x7fe8b8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 0x7fe8b8067f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc154460 tx=0x7fe8bc0015f0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 msgr2=0x7fe8cc1f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4002240 tx=0x7fe8c4019ba0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 shutdown_connections 2023-11-06T03:08:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe8b8065a90 0x7fe8b8067f50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe8cc0b2230 0x7fe8cc1f3d00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8cc157740 0x7fe8cc13c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 --2- 172.21.15.157:0/1854079834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8cc0b1e50 0x7fe8cc13bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 >> 172.21.15.157:0/1854079834 conn(0x7fe8cc065930 msgr2=0x7fe8cc0afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 shutdown_connections 2023-11-06T03:08:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.629+0000 7fe8d17d7640 1 -- 172.21.15.157:0/1854079834 wait complete. 2023-11-06T03:08:05.667 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2023-11-06T03:08:05.668 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2023-11-06T03:08:05.668 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2023-11-06T03:08:05.668 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.781+0000 7f3850d6a640 1 Processor -- start 2023-11-06T03:08:05.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.781+0000 7f3850d6a640 1 -- start start 2023-11-06T03:08:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3850d6a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 0x7f384c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3850d6a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f384c065f70 0x7f384c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3850d6a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c15d0f0 con 0x7f384c15cc60 2023-11-06T03:08:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3850d6a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c0b2840 con 0x7f384c065f70 2023-11-06T03:08:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3850d6a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c160a00 con 0x7f384c065b90 2023-11-06T03:08:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f384c065f70 0x7f384c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f384c15cc60 0x7f384c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34986/0 (socket says 172.21.15.157:34986) 2023-11-06T03:08:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384ad9d640 1 -- 172.21.15.157:0/2114469143 learned_addr learned my addr 172.21.15.157:0/2114469143 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384a59c640 1 --2- 172.21.15.157:0/2114469143 >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 0x7f384c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 -- 172.21.15.157:0/2114469143 >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 msgr2=0x7f384c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 --2- 172.21.15.157:0/2114469143 >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 0x7f384c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 -- 172.21.15.157:0/2114469143 >> v1:172.21.15.157:6789/0 conn(0x7f384c15cc60 legacy=0x7f384c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384959a640 1 -- 172.21.15.157:0/2114469143 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383122154 0 0) 0x7f384c15d0f0 con 0x7f384c15cc60 2023-11-06T03:08:05.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 -- 172.21.15.157:0/2114469143 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c160ce0 con 0x7f384c065f70 2023-11-06T03:08:05.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384a59c640 1 --2- 172.21.15.157:0/2114469143 >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 0x7f384c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:05.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f3849d9b640 1 --2- 172.21.15.157:0/2114469143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c15c720 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f384c066780 tx=0x7f384001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd73f137b82a927 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384959a640 1 -- 172.21.15.157:0/2114469143 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400a7070 con 0x7f384c065f70 2023-11-06T03:08:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.785+0000 7f384959a640 1 -- 172.21.15.157:0/2114469143 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38400ab430 con 0x7f384c065f70 2023-11-06T03:08:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f384959a640 1 -- 172.21.15.157:0/2114469143 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400ab7c0 con 0x7f384c065f70 2023-11-06T03:08:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/2114469143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 msgr2=0x7f384c15c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- 172.21.15.157:0/2114469143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c15c720 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f384c066780 tx=0x7f384001c430 comp rx=0 tx=0).stop 2023-11-06T03:08:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/2114469143 shutdown_connections 2023-11-06T03:08:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- 172.21.15.157:0/2114469143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c15c720 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- 172.21.15.157:0/2114469143 >> v2:172.21.15.195:3300/0 conn(0x7f384c065b90 0x7f384c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/2114469143 >> 172.21.15.157:0/2114469143 conn(0x7f384c06a170 msgr2=0x7f384c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/2114469143 shutdown_connections 2023-11-06T03:08:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/2114469143 wait complete. 2023-11-06T03:08:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 Processor -- start 2023-11-06T03:08:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- start start 2023-11-06T03:08:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f384c065b90 0x7f384c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f384c15cc60 0x7f384c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c157e30 con 0x7f384c065f70 2023-11-06T03:08:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c158100 con 0x7f384c065b90 2023-11-06T03:08:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c1583e0 con 0x7f384c15cc60 2023-11-06T03:08:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44710/0 (socket says 172.21.15.157:44710) 2023-11-06T03:08:05.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 -- 172.21.15.157:0/1624168174 learned_addr learned my addr 172.21.15.157:0/1624168174 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:05.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 -- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f384c15cc60 msgr2=0x7f384c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f384c15cc60 0x7f384c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 -- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f384c065b90 msgr2=0x7f384c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:05.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f384c065b90 0x7f384c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:05.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 -- 172.21.15.157:0/1624168174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840009420 con 0x7f384c065f70 2023-11-06T03:08:05.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3849d9b640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad040 tx=0x7f384001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400b5810 con 0x7f384c065f70 2023-11-06T03:08:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38400b5990 con 0x7f384c065f70 2023-11-06T03:08:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c1586c0 con 0x7f384c065f70 2023-11-06T03:08:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400b5d40 con 0x7f384c065f70 2023-11-06T03:08:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.789+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c15a160 con 0x7f384c065f70 2023-11-06T03:08:05.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c072750 con 0x7f384c065f70 2023-11-06T03:08:05.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38400b4390 con 0x7f384c065f70 2023-11-06T03:08:05.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f383affd640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 0x7f382c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:05.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f384011d790 con 0x7f384c065f70 2023-11-06T03:08:05.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f384a59c640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 0x7f382c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:05.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.793+0000 7f384a59c640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 0x7f382c067f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f383c006f10 tx=0x7f383c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:05.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:05.797+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38400eb040 con 0x7f384c065f70 2023-11-06T03:08:06.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.053+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384c0c3180 con 0x7f384c065f70 2023-11-06T03:08:06.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f383affd640 1 -- 172.21.15.157:0/1624168174 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f3840006e20 con 0x7f384c065f70 2023-11-06T03:08:06.056 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:06.056 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:06.056 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:08:06.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 msgr2=0x7f382c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 0x7f382c067f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f383c006f10 tx=0x7f383c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:06.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 msgr2=0x7f384c159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad040 tx=0x7f384001ac70 comp rx=0 tx=0).stop 2023-11-06T03:08:06.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 shutdown_connections 2023-11-06T03:08:06.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f382c065a60 0x7f382c067f20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f384c15cc60 0x7f384c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f384c065f70 0x7f384c159510 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 --2- 172.21.15.157:0/1624168174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f384c065b90 0x7f384c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 >> 172.21.15.157:0/1624168174 conn(0x7f384c06a170 msgr2=0x7f384c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 shutdown_connections 2023-11-06T03:08:06.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.057+0000 7f3850d6a640 1 -- 172.21.15.157:0/1624168174 wait complete. 2023-11-06T03:08:06.074 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2023-11-06T03:08:06.270 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:06.270 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:06.270 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:06.283 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:06.283 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2023-11-06T03:08:06.284 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2023-11-06T03:08:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 Processor -- start 2023-11-06T03:08:06.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 -- start start 2023-11-06T03:08:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84300ca570 0x7f84300b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f843006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843006b8d0 con 0x7f843015adc0 2023-11-06T03:08:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300b4f60 con 0x7f84300ca570 2023-11-06T03:08:06.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f843650e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843006be90 con 0x7f84300b4ab0 2023-11-06T03:08:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84300ca570 0x7f84300b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84300ca570 0x7f84300b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44714/0 (socket says 172.21.15.157:44714) 2023-11-06T03:08:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 -- 172.21.15.157:0/2306846101 learned_addr learned my addr 172.21.15.157:0/2306846101 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842ffff640 1 --1- 172.21.15.157:0/2306846101 >> v1:172.21.15.157:6789/0 conn(0x7f843015adc0 0x7f8430070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34998/0 (socket says 172.21.15.157:34998) 2023-11-06T03:08:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 -- 172.21.15.157:0/2306846101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 msgr2=0x7f843006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f8434aab640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f843006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f843006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 -- 172.21.15.157:0/2306846101 >> v1:172.21.15.157:6789/0 conn(0x7f843015adc0 legacy=0x7f8430070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.397+0000 7f842f7fe640 1 -- 172.21.15.157:0/2306846101 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8430071880 con 0x7f84300ca570 2023-11-06T03:08:06.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842f7fe640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84300ca570 0x7f84300b4570 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8424009e80 tx=0x7f842401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a580681090273f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842effd640 1 -- 172.21.15.157:0/2306846101 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240a5760 con 0x7f84300ca570 2023-11-06T03:08:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842effd640 1 -- 172.21.15.157:0/2306846101 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84240a58c0 con 0x7f84300ca570 2023-11-06T03:08:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842effd640 1 -- 172.21.15.157:0/2306846101 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240a5c50 con 0x7f84300ca570 2023-11-06T03:08:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2306846101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84300ca570 msgr2=0x7f84300b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84300ca570 0x7f84300b4570 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8424009e80 tx=0x7f842401c8a0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2306846101 shutdown_connections 2023-11-06T03:08:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f843006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- 172.21.15.157:0/2306846101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84300ca570 0x7f84300b4570 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2306846101 >> 172.21.15.157:0/2306846101 conn(0x7f84300cb990 msgr2=0x7f84300cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2306846101 shutdown_connections 2023-11-06T03:08:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2306846101 wait complete. 2023-11-06T03:08:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 Processor -- start 2023-11-06T03:08:06.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- start start 2023-11-06T03:08:06.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f84300b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84300ca570 0x7f8430149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300b0320 con 0x7f843015adc0 2023-11-06T03:08:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300b0600 con 0x7f84300ca570 2023-11-06T03:08:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84300b08e0 con 0x7f84300b4ab0 2023-11-06T03:08:06.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44722/0 (socket says 172.21.15.157:44722) 2023-11-06T03:08:06.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 -- 172.21.15.157:0/2386335178 learned_addr learned my addr 172.21.15.157:0/2386335178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842f7fe640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84300ca570 0x7f8430149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 -- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 msgr2=0x7f84300b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f84300b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 -- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84300ca570 msgr2=0x7f8430149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84300ca570 0x7f8430149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 -- 172.21.15.157:0/2386335178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8424009b30 con 0x7f843015adc0 2023-11-06T03:08:06.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f8434aab640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f842800afd0 tx=0x7f842800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842cff9640 1 -- 172.21.15.157:0/2386335178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428004220 con 0x7f843015adc0 2023-11-06T03:08:06.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842cff9640 1 -- 172.21.15.157:0/2386335178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428004380 con 0x7f843015adc0 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84300b0bc0 con 0x7f843015adc0 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.401+0000 7f842cff9640 1 -- 172.21.15.157:0/2386335178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428003840 con 0x7f843015adc0 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843014a3b0 con 0x7f843015adc0 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842cff9640 1 -- 172.21.15.157:0/2386335178 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84280039a0 con 0x7f843015adc0 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842cff9640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 0x7f841c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842cff9640 1 -- 172.21.15.157:0/2386335178 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8428082fe0 con 0x7f843015adc0 2023-11-06T03:08:06.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842ffff640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 0x7f841c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 Processor -- start 2023-11-06T03:08:06.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 -- start start 2023-11-06T03:08:06.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842ffff640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 0x7f841c067f20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84200015f0 tx=0x7f84200079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x561789251490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892519d0 0x561789206d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178905c270 con 0x5617892072b0 2023-11-06T03:08:06.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561789059840 con 0x5617892510b0 2023-11-06T03:08:06.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561789055130 con 0x5617892519d0 2023-11-06T03:08:06.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44732/0 (socket says 172.21.15.157:44732) 2023-11-06T03:08:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 -- 172.21.15.157:0/3390381064 learned_addr learned my addr 172.21.15.157:0/3390381064 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842ffff640 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x561789251490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 -- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892519d0 msgr2=0x561789206d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892519d0 0x561789206d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 -- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 msgr2=0x561789251490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x561789251490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f8434aab640 1 -- 172.21.15.157:0/3390381064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561788d51d70 con 0x5617892072b0 2023-11-06T03:08:06.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.405+0000 7f842ffff640 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x561789251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:06.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8434aab640 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f842800a480 tx=0x7f8428054c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f639588e884804 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8415ffb640 1 -- 172.21.15.157:0/3390381064 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842805c490 con 0x5617892072b0 2023-11-06T03:08:06.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8415ffb640 1 -- 172.21.15.157:0/3390381064 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428052850 con 0x5617892072b0 2023-11-06T03:08:06.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8415ffb640 1 -- 172.21.15.157:0/3390381064 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84280539a0 con 0x5617892072b0 2023-11-06T03:08:06.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/3390381064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 msgr2=0x561789207690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f842800a480 tx=0x7f8428054c70 comp rx=0 tx=0).stop 2023-11-06T03:08:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/3390381064 shutdown_connections 2023-11-06T03:08:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892072b0 0x561789207690 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892519d0 0x561789206d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- 172.21.15.157:0/3390381064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x561789251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/3390381064 >> 172.21.15.157:0/3390381064 conn(0x56178920d620 msgr2=0x561789205650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/3390381064 shutdown_connections 2023-11-06T03:08:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/3390381064 wait complete. 2023-11-06T03:08:06.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- start start 2023-11-06T03:08:06.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 Processor -- start 2023-11-06T03:08:06.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178920c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x56178923f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561788ff49b0 con 0x5617892510b0 2023-11-06T03:08:06.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561788ff4650 con 0x5617892519d0 2023-11-06T03:08:06.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561788d12e20 con 0x5617892072b0 2023-11-06T03:08:06.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 10 client.0 _ll_register_callbacks cb 0x5617890034d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:06.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44748/0 (socket says 172.21.15.157:44748) 2023-11-06T03:08:06.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 -- 172.21.15.157:0/4162730229 learned_addr learned my addr 172.21.15.157:0/4162730229 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842ffff640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178920c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8434aab640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x56178923f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 -- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 msgr2=0x56178920c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178920c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 -- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 msgr2=0x56178923f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x56178923f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 -- 172.21.15.157:0/4162730229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843014a3b0 con 0x5617892510b0 2023-11-06T03:08:06.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842f7fe640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f842401af50 tx=0x7f8424019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ac1963e0bde926 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:06.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240a5cf0 con 0x5617892510b0 2023-11-06T03:08:06.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f84240b8440 con 0x5617892510b0 2023-11-06T03:08:06.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561789020290 con 0x5617892510b0 2023-11-06T03:08:06.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84240a5e50 con 0x5617892510b0 2023-11-06T03:08:06.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 10 client.6105 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:06.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x56178901a360 con 0x5617892510b0 2023-11-06T03:08:06.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561789020290 con 0x5617892510b0 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f84240b8620 con 0x5617892510b0 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 10 client.6105 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 10 client.6105 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 20 client.6105 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923fd90 0x561789263370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 4 client.6105 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561789053a80 con 0x56178923fd90 2023-11-06T03:08:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.409+0000 7f842ffff640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923fd90 0x561789263370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f840effd640 10 client.6105 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f842ffff640 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923fd90 0x561789263370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83029f3436632f3 server_cookie=d62148ef6df50fdd in_seq=0 out_seq=0 2023-11-06T03:08:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f840effd640 1 -- 172.21.15.157:0/4162730229 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561789053a80 con 0x56178923fd90 2023-11-06T03:08:06.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f840effd640 10 client.6105 handle_command_reply: tid=0 2023-11-06T03:08:06.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 client.6105 shutdown 2023-11-06T03:08:06.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 msgr2=0x56178920c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f842401af50 tx=0x7f8424019640 comp rx=0 tx=0).stop 2023-11-06T03:08:06.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 shutdown_connections 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923fd90 0x561789263370 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x56178923f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x56178920c8b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 --2- 172.21.15.157:0/4162730229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178920c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 >> 172.21.15.157:0/4162730229 conn(0x56178920d620 msgr2=0x561789205d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f840effd640 0 client.6105 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:06.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 shutdown_connections 2023-11-06T03:08:06.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 1 -- 172.21.15.157:0/4162730229 wait complete. 2023-11-06T03:08:06.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.413+0000 7f8439636000 20 client.6105 trim_cache size 0 max 16384 2023-11-06T03:08:06.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 Processor -- start 2023-11-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 -- start start 2023-11-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x5617892652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x5617892628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178909a480 con 0x5617892510b0 2023-11-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561789050aa0 con 0x5617892519d0 2023-11-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561789052530 con 0x5617892072b0 2023-11-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f842f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44756/0 (socket says 172.21.15.157:44756) 2023-11-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.425+0000 7f842f7fe640 1 -- 172.21.15.157:0/1251748688 learned_addr learned my addr 172.21.15.157:0/1251748688 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 -- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 msgr2=0x5617892652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x5617892652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 -- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 msgr2=0x5617892628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x5617892628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 -- 172.21.15.157:0/1251748688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300c17f0 con 0x5617892510b0 2023-11-06T03:08:06.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842f7fe640 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8424098480 tx=0x7f84240b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dfa1bace6aa9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f84167fc640 1 -- 172.21.15.157:0/1251748688 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240b8ae0 con 0x5617892510b0 2023-11-06T03:08:06.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f84167fc640 1 -- 172.21.15.157:0/1251748688 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424009510 con 0x5617892510b0 2023-11-06T03:08:06.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f84167fc640 1 -- 172.21.15.157:0/1251748688 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240b8440 con 0x5617892510b0 2023-11-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/1251748688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 msgr2=0x561789262370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8424098480 tx=0x7f84240b9000 comp rx=0 tx=0).stop 2023-11-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/1251748688 shutdown_connections 2023-11-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892519d0 0x5617892628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892510b0 0x561789262370 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- 172.21.15.157:0/1251748688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x5617892652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/1251748688 >> 172.21.15.157:0/1251748688 conn(0x56178920d620 msgr2=0x5617892506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/1251748688 shutdown_connections 2023-11-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/1251748688 wait complete. 2023-11-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- start start 2023-11-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 Processor -- start 2023-11-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178923e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x56178923aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178905d500 con 0x5617892519d0 2023-11-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x561789017b70 con 0x5617892510b0 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x561789052940 con 0x5617892072b0 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 10 client.0 _ll_register_callbacks cb 0x5617891d3530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44762/0 (socket says 172.21.15.157:44762) 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 -- 172.21.15.157:0/937664711 learned_addr learned my addr 172.21.15.157:0/937664711 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 -- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 msgr2=0x56178923e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178923e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 -- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 msgr2=0x56178923aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x56178923aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 -- 172.21.15.157:0/937664711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56178901a360 con 0x5617892519d0 2023-11-06T03:08:06.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8434aab640 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f84280509b0 tx=0x7f8428081230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce08b41d02b24b36 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:06.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428053ba0 con 0x5617892519d0 2023-11-06T03:08:06.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f842805c490 con 0x5617892519d0 2023-11-06T03:08:06.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617890162e0 con 0x5617892519d0 2023-11-06T03:08:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842805c000 con 0x5617892519d0 2023-11-06T03:08:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 10 client.6111 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5617890004b0 con 0x5617892519d0 2023-11-06T03:08:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617890162e0 con 0x5617892519d0 2023-11-06T03:08:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f842805a090 con 0x5617892519d0 2023-11-06T03:08:06.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 10 client.6111 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:06.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 10 client.6111 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:06.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 20 client.6111 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:06.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923be00 0x56178923c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 4 client.6111 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x56178904e1b0 con 0x56178923be00 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842ffff640 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923be00 0x56178923c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f840effd640 10 client.6111 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.429+0000 7f842ffff640 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923be00 0x56178923c1e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4babfdd0629a63fd server_cookie=17ced8b63ab06254 in_seq=0 out_seq=0 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f840effd640 1 -- 172.21.15.157:0/937664711 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56178904e1b0 con 0x56178923be00 2023-11-06T03:08:06.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f840effd640 10 client.6111 handle_command_reply: tid=0 2023-11-06T03:08:06.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 client.6111 shutdown 2023-11-06T03:08:06.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 msgr2=0x56178923b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f84280509b0 tx=0x7f8428081230 comp rx=0 tx=0).stop 2023-11-06T03:08:06.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 shutdown_connections 2023-11-06T03:08:06.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56178923be00 0x56178923c1e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5617892519d0 0x56178923b530 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5617892510b0 0x56178923aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f840effd640 0 client.6111 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:06.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 --2- 172.21.15.157:0/937664711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5617892072b0 0x56178923e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 >> 172.21.15.157:0/937664711 conn(0x56178920d620 msgr2=0x561789261360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 shutdown_connections 2023-11-06T03:08:06.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 1 -- 172.21.15.157:0/937664711 wait complete. 2023-11-06T03:08:06.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f8439636000 20 client.6111 trim_cache size 0 max 16384 2023-11-06T03:08:06.457 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:06.458 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:06.458 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "d0fd1dc4-54b4-4c61-a72f-5a1d00784466", 2023-11-06T03:08:06.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:06.458 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:06.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 msgr2=0x7f841c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 0x7f841c067f20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84200015f0 tx=0x7f84200079d0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 msgr2=0x7f843014a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f842800afd0 tx=0x7f842800ca20 comp rx=0 tx=0).stop 2023-11-06T03:08:06.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 shutdown_connections 2023-11-06T03:08:06.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f841c065a60 0x7f841c067f20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f843015adc0 0x7f843014a690 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84300ca570 0x7f8430149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 --2- 172.21.15.157:0/2386335178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84300b4ab0 0x7f84300b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 >> 172.21.15.157:0/2386335178 conn(0x7f84300cb990 msgr2=0x7f84300b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 shutdown_connections 2023-11-06T03:08:06.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.433+0000 7f843650e640 1 -- 172.21.15.157:0/2386335178 wait complete. 2023-11-06T03:08:06.462 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'd0fd1dc4-54b4-4c61-a72f-5a1d00784466', 'mode': 'asynchronous'}' 2023-11-06T03:08:06.463 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-06T03:08:06.463 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-06T03:08:06.463 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:06.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.565+0000 7f4f664f7640 1 Processor -- start 2023-11-06T03:08:06.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.565+0000 7f4f664f7640 1 -- start start 2023-11-06T03:08:06.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f664f7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f600b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f664f7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cef40 con 0x7f4f6015c290 2023-11-06T03:08:06.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f664f7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cd450 con 0x7f4f60065d40 2023-11-06T03:08:06.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f664f7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600cbf80 con 0x7f4f60144430 2023-11-06T03:08:06.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f64a94640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4f6015c290 0x7f4f600cb870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35000/0 (socket says 172.21.15.157:35000) 2023-11-06T03:08:06.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f64a94640 1 -- 172.21.15.157:0/5722963 learned_addr learned my addr 172.21.15.157:0/5722963 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:06.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516000765 0 0) 0x7f4f600cef40 con 0x7f4f6015c290 2023-11-06T03:08:06.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f48003530 con 0x7f4f6015c290 2023-11-06T03:08:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f600b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669598210 0 0) 0x7f4f48003530 con 0x7f4f6015c290 2023-11-06T03:08:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 msgr2=0x7f4f600b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f600b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 >> v1:172.21.15.195:6789/0 conn(0x7f4f60065d40 legacy=0x7f4f600cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600cc840 con 0x7f4f6015c290 2023-11-06T03:08:06.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4f58001a00 con 0x7f4f6015c290 2023-11-06T03:08:06.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 >> v1:172.21.15.157:6789/0 conn(0x7f4f6015c290 legacy=0x7f4f600cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480039b0 0x7f4f48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f480047d0 0x7f4f48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f600cbf80 con 0x7f4f480047d0 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4800bac0 con 0x7f4f480039b0 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4800bc80 con 0x7f4f60144430 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f480047d0 0x7f4f48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480039b0 0x7f4f48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f64a94640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 msgr2=0x7f4f480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60144430 0x7f4f480051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480039b0 msgr2=0x7f4f48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:06.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480039b0 0x7f4f48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/5722963 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48003530 con 0x7f4f480047d0 2023-11-06T03:08:06.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480039b0 0x7f4f48003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:06.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f480047d0 0x7f4f48004c50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4f54001780 tx=0x7f4f5401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd70e43ac79591b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540052d0 con 0x7f4f480047d0 2023-11-06T03:08:06.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f540a7050 con 0x7f4f480047d0 2023-11-06T03:08:06.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.569+0000 7f4f5effd640 1 -- 172.21.15.157:0/5722963 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54003600 con 0x7f4f480047d0 2023-11-06T03:08:06.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/5722963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f480047d0 msgr2=0x7f4f48004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 --2- 172.21.15.157:0/5722963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f480047d0 0x7f4f48004c50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4f54001780 tx=0x7f4f5401c400 comp rx=0 tx=0).stop 2023-11-06T03:08:06.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f64a94640 1 -- 172.21.15.157:0/5722963 reap_dead start 2023-11-06T03:08:06.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/5722963 shutdown_connections 2023-11-06T03:08:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/5722963 >> 172.21.15.157:0/5722963 conn(0x7f4f60065930 msgr2=0x7f4f60071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/5722963 shutdown_connections 2023-11-06T03:08:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/5722963 wait complete. 2023-11-06T03:08:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 Processor -- start 2023-11-06T03:08:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- start start 2023-11-06T03:08:06.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480047d0 0x7f4f60158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60143110 0x7f4f6013ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6013c630 con 0x7f4f6013dc10 2023-11-06T03:08:06.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f601435a0 con 0x7f4f480047d0 2023-11-06T03:08:06.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6013f650 con 0x7f4f60143110 2023-11-06T03:08:06.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44778/0 (socket says 172.21.15.157:44778) 2023-11-06T03:08:06.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/1600889001 learned_addr learned my addr 172.21.15.157:0/1600889001 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:06.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480047d0 0x7f4f60158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f64a94640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60143110 0x7f4f6013ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60143110 msgr2=0x7f4f6013ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60143110 0x7f4f6013ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480047d0 msgr2=0x7f4f60158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480047d0 0x7f4f60158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 -- 172.21.15.157:0/1600889001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6013ce50 con 0x7f4f6013dc10 2023-11-06T03:08:06.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5f7fe640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4f5401c8e0 tx=0x7f4f48005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54007310 con 0x7f4f6013dc10 2023-11-06T03:08:06.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54007470 con 0x7f4f6013dc10 2023-11-06T03:08:06.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540067e0 con 0x7f4f6013dc10 2023-11-06T03:08:06.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6013d130 con 0x7f4f6013dc10 2023-11-06T03:08:06.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6013d6f0 con 0x7f4f6013dc10 2023-11-06T03:08:06.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.573+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f60072670 con 0x7f4f6013dc10 2023-11-06T03:08:06.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.577+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4f54006940 con 0x7f4f6013dc10 2023-11-06T03:08:06.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.577+0000 7f4f5cff9640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 0x7f4f4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:06.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.577+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4f5411d980 con 0x7f4f6013dc10 2023-11-06T03:08:06.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.577+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 0x7f4f4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:06.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.577+0000 7f4f5ffff640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 0x7f4f4c067f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f601437e0 tx=0x7f4f50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:06.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.581+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4f540eed50 con 0x7f4f6013dc10 2023-11-06T03:08:06.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.753+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f601412c0 con 0x7f4f6013dc10 2023-11-06T03:08:06.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.753+0000 7f4f5cff9640 1 -- 172.21.15.157:0/1600889001 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f4f540ee8e0 con 0x7f4f6013dc10 2023-11-06T03:08:06.753 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:06.753 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:06.753 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:08:06.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 msgr2=0x7f4f4c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 0x7f4f4c067f20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f601437e0 tx=0x7f4f50008040 comp rx=0 tx=0).stop 2023-11-06T03:08:06.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 msgr2=0x7f4f60142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:06.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4f5401c8e0 tx=0x7f4f48005800 comp rx=0 tx=0).stop 2023-11-06T03:08:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 shutdown_connections 2023-11-06T03:08:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f4c065a60 0x7f4f4c067f20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f60143110 0x7f4f6013ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f6013dc10 0x7f4f60142bd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 --2- 172.21.15.157:0/1600889001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f480047d0 0x7f4f60158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:06.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 >> 172.21.15.157:0/1600889001 conn(0x7f4f60065930 msgr2=0x7f4f60071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:06.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 shutdown_connections 2023-11-06T03:08:06.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:06.757+0000 7f4f664f7640 1 -- 172.21.15.157:0/1600889001 wait complete. 2023-11-06T03:08:06.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-06T03:08:07.056 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:07.056 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:07.057 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:07.073 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:07.073 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-06T03:08:07.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-06T03:08:07.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 Processor -- start 2023-11-06T03:08:07.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 -- start start 2023-11-06T03:08:07.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe40806d000 0x7fe4080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40815e440 con 0x7fe4080b2af0 2023-11-06T03:08:07.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40806c240 con 0x7fe4081599c0 2023-11-06T03:08:07.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe40e157640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40806c520 con 0x7fe40806d000 2023-11-06T03:08:07.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe4080b2af0 0x7fe408159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35008/0 (socket says 172.21.15.157:35008) 2023-11-06T03:08:07.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 -- 172.21.15.157:0/23511485 learned_addr learned my addr 172.21.15.157:0/23511485 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575014570 0 0) 0x7fe40815e440 con 0x7fe4080b2af0 2023-11-06T03:08:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe407fff640 1 --2- 172.21.15.157:0/23511485 >> v2:172.21.15.195:3300/0 conn(0x7fe40806d000 0x7fe4080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f0003530 con 0x7fe4080b2af0 2023-11-06T03:08:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000842338 0 0) 0x7fe3f0003530 con 0x7fe4080b2af0 2023-11-06T03:08:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 >> v2:172.21.15.195:3300/0 conn(0x7fe40806d000 msgr2=0x7fe4080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 --2- 172.21.15.157:0/23511485 >> v2:172.21.15.195:3300/0 conn(0x7fe40806d000 0x7fe4080ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 >> v1:172.21.15.195:6789/0 conn(0x7fe4081599c0 legacy=0x7fe4080b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40806c800 con 0x7fe4080b2af0 2023-11-06T03:08:07.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe3fc001680 con 0x7fe4080b2af0 2023-11-06T03:08:07.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 >> v1:172.21.15.157:6789/0 conn(0x7fe4080b2af0 legacy=0x7fe408159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3f0003980 0x7fe3f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 0x7fe3f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f0005190 0x7fe3f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40806c520 con 0x7fe3f00047d0 2023-11-06T03:08:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f000c0e0 con 0x7fe3f0003980 2023-11-06T03:08:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f000c2a0 con 0x7fe3f0005190 2023-11-06T03:08:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 0x7fe3f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe4077fe640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3f0003980 0x7fe3f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 -- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f0005190 msgr2=0x7fe3f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f0005190 0x7fe3f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 -- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3f0003980 msgr2=0x7fe3f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3f0003980 0x7fe3f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe406ffd640 1 -- 172.21.15.157:0/23511485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0003530 con 0x7fe3f00047d0 2023-11-06T03:08:07.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.193+0000 7fe407fff640 1 -- 172.21.15.157:0/23511485 reap_dead start 2023-11-06T03:08:07.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe406ffd640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 0x7fe3f0004c50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c240 tx=0x7fe3fc01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6390f087f34561 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc005ae0 con 0x7fe3f00047d0 2023-11-06T03:08:07.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0ab070 con 0x7fe3f00047d0 2023-11-06T03:08:07.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe4067fc640 1 -- 172.21.15.157:0/23511485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc007070 con 0x7fe3f00047d0 2023-11-06T03:08:07.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 msgr2=0x7fe3f0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 0x7fe3f0004c50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c240 tx=0x7fe3fc01f3b0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- 172.21.15.157:0/23511485 shutdown_connections 2023-11-06T03:08:07.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 --2- 172.21.15.157:0/23511485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3f00047d0 0x7fe3f0004c50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- 172.21.15.157:0/23511485 >> 172.21.15.157:0/23511485 conn(0x7fe408065930 msgr2=0x7fe408065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- 172.21.15.157:0/23511485 shutdown_connections 2023-11-06T03:08:07.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- 172.21.15.157:0/23511485 wait complete. 2023-11-06T03:08:07.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 Processor -- start 2023-11-06T03:08:07.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.197+0000 7fe40e157640 1 -- start start 2023-11-06T03:08:07.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 0x7fe40814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4080ce470 0x7fe4080acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe408148740 con 0x7fe408146710 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4081482b0 con 0x7fe4080ce470 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4081476a0 con 0x7fe3f00047d0 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44796/0 (socket says 172.21.15.157:44796) 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 -- 172.21.15.157:0/807109196 learned_addr learned my addr 172.21.15.157:0/807109196 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe4077fe640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 0x7fe40814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe406ffd640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4080ce470 0x7fe4080acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 -- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 msgr2=0x7fe40814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 0x7fe40814cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 -- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4080ce470 msgr2=0x7fe4080acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4080ce470 0x7fe4080acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 -- 172.21.15.157:0/807109196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408147980 con 0x7fe408146710 2023-11-06T03:08:07.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe4077fe640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 0x7fe40814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:07.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe407fff640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe4080b3b00 tx=0x7fe40000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 -- 172.21.15.157:0/807109196 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400007e40 con 0x7fe408146710 2023-11-06T03:08:07.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 -- 172.21.15.157:0/807109196 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400014420 con 0x7fe408146710 2023-11-06T03:08:07.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe408147c60 con 0x7fe408146710 2023-11-06T03:08:07.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 -- 172.21.15.157:0/807109196 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4000147d0 con 0x7fe408146710 2023-11-06T03:08:07.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080ad050 con 0x7fe408146710 2023-11-06T03:08:07.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 -- 172.21.15.157:0/807109196 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe400014930 con 0x7fe408146710 2023-11-06T03:08:07.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 0x7fe3f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe3effff640 1 -- 172.21.15.157:0/807109196 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe4000838a0 con 0x7fe408146710 2023-11-06T03:08:07.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe4077fe640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 0x7fe3f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe41127f000 1 Processor -- start 2023-11-06T03:08:07.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe41127f000 1 -- start start 2023-11-06T03:08:07.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.201+0000 7fe4077fe640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 0x7fe3f4067f20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8000c10 tx=0x7fe3f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb02a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb00cba10 con 0x563cb025ba00 2023-11-06T03:08:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb00b15e0 con 0x563cb02a5d90 2023-11-06T03:08:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb00aebb0 con 0x563cb02a5470 2023-11-06T03:08:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44800/0 (socket says 172.21.15.157:44800) 2023-11-06T03:08:07.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 -- 172.21.15.157:0/245118453 learned_addr learned my addr 172.21.15.157:0/245118453 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb02a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe406ffd640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 -- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 msgr2=0x563cb02a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb02a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 -- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 msgr2=0x563cb025b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 -- 172.21.15.157:0/245118453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cafda70d0 con 0x563cb025ba00 2023-11-06T03:08:07.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe406ffd640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:07.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe400002240 tx=0x7fe4000555c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d75466a2713b4b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3ed7fa640 1 -- 172.21.15.157:0/245118453 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40006c070 con 0x563cb025ba00 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3ed7fa640 1 -- 172.21.15.157:0/245118453 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40005cd60 con 0x563cb025ba00 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3ed7fa640 1 -- 172.21.15.157:0/245118453 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400051490 con 0x563cb025ba00 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/245118453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 msgr2=0x563cb025bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe400002240 tx=0x7fe4000555c0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/245118453 shutdown_connections 2023-11-06T03:08:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb025bde0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- 172.21.15.157:0/245118453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb02a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/245118453 >> 172.21.15.157:0/245118453 conn(0x563cb02579b0 msgr2=0x563cb0259da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/245118453 shutdown_connections 2023-11-06T03:08:07.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/245118453 wait complete. 2023-11-06T03:08:07.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- start start 2023-11-06T03:08:07.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 Processor -- start 2023-11-06T03:08:07.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5470 0x563cb02acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5d90 0x563cb0289450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb00aa4a0 con 0x563cb025ba00 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb0049a70 con 0x563cb02a5470 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb0067af0 con 0x563cb02a5d90 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 10 client.0 _ll_register_callbacks cb 0x563cb0246d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:07.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44804/0 (socket says 172.21.15.157:44804) 2023-11-06T03:08:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 -- 172.21.15.157:0/308813451 learned_addr learned my addr 172.21.15.157:0/308813451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe406ffd640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5470 0x563cb02acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe407fff640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5d90 0x563cb0289450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 -- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5d90 msgr2=0x563cb0289450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5d90 0x563cb0289450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 -- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5470 msgr2=0x563cb02acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5470 0x563cb02acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 -- 172.21.15.157:0/308813451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4080ad050 con 0x563cb025ba00 2023-11-06T03:08:07.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe4077fe640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800ff50 tx=0x7fe3f8001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a463b2a90940d5aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:07.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8026070 con 0x563cb025ba00 2023-11-06T03:08:07.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe3f801c040 con 0x563cb025ba00 2023-11-06T03:08:07.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.205+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb0072550 con 0x563cb025ba00 2023-11-06T03:08:07.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f8014e60 con 0x563cb025ba00 2023-11-06T03:08:07.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 10 client.6129 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:07.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x563cb006f5a0 con 0x563cb025ba00 2023-11-06T03:08:07.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8030020 con 0x563cb025ba00 2023-11-06T03:08:07.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe3f8029020 con 0x563cb025ba00 2023-11-06T03:08:07.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 10 client.6129 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 10 client.6129 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 20 client.6129 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb028a1c0 0x563cb028a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 4 client.6129 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb00af390 con 0x563cb028a1c0 2023-11-06T03:08:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe406ffd640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb028a1c0 0x563cb028a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 10 client.6129 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe406ffd640 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb028a1c0 0x563cb028a5d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe6963ef78b76a6 server_cookie=84bef38415d143dd in_seq=0 out_seq=0 2023-11-06T03:08:07.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 1 -- 172.21.15.157:0/308813451 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563cb00af390 con 0x563cb028a1c0 2023-11-06T03:08:07.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 10 client.6129 handle_command_reply: tid=0 2023-11-06T03:08:07.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 client.6129 shutdown 2023-11-06T03:08:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 msgr2=0x563cb02ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800ff50 tx=0x7fe3f8001130 comp rx=0 tx=0).stop 2023-11-06T03:08:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 shutdown_connections 2023-11-06T03:08:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb028a1c0 0x563cb028a5d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5d90 0x563cb0289450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe3e67fc640 0 client.6129 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5470 0x563cb02acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 --2- 172.21.15.157:0/308813451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02ac7d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 >> 172.21.15.157:0/308813451 conn(0x563cb02579b0 msgr2=0x563cb0259da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 shutdown_connections 2023-11-06T03:08:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 1 -- 172.21.15.157:0/308813451 wait complete. 2023-11-06T03:08:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.209+0000 7fe41127f000 20 client.6129 trim_cache size 0 max 16384 2023-11-06T03:08:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 Processor -- start 2023-11-06T03:08:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 -- start start 2023-11-06T03:08:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb025f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cafd68180 con 0x563cb025ba00 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb00a7cb0 con 0x563cb02a5d90 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.221+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb00ae4a0 con 0x563cb02a5470 2023-11-06T03:08:07.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44820/0 (socket says 172.21.15.157:44820) 2023-11-06T03:08:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 -- 172.21.15.157:0/1256196082 learned_addr learned my addr 172.21.15.157:0/1256196082 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 -- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 msgr2=0x563cb025f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb025f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 -- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 msgr2=0x563cb025fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 -- 172.21.15.157:0/1256196082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb00556f0 con 0x563cb025ba00 2023-11-06T03:08:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800f270 tx=0x7fe3f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8d7fe27d59a907 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3edffb640 1 -- 172.21.15.157:0/1256196082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8026070 con 0x563cb025ba00 2023-11-06T03:08:07.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3edffb640 1 -- 172.21.15.157:0/1256196082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8030040 con 0x563cb025ba00 2023-11-06T03:08:07.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3edffb640 1 -- 172.21.15.157:0/1256196082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8014bf0 con 0x563cb025ba00 2023-11-06T03:08:07.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/1256196082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 msgr2=0x563cb02606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800f270 tx=0x7fe3f8019000 comp rx=0 tx=0).stop 2023-11-06T03:08:07.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/1256196082 shutdown_connections 2023-11-06T03:08:07.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb02a5d90 0x563cb025fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb025f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- 172.21.15.157:0/1256196082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb025ba00 0x563cb02606b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/1256196082 >> 172.21.15.157:0/1256196082 conn(0x563cb02579b0 msgr2=0x563cb02889b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/1256196082 shutdown_connections 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/1256196082 wait complete. 2023-11-06T03:08:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- start start 2023-11-06T03:08:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 Processor -- start 2023-11-06T03:08:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb025ba00 0x563cb030b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb030bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb0102900 con 0x563cb02a5d90 2023-11-06T03:08:07.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb02a5030 con 0x563cb025ba00 2023-11-06T03:08:07.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb0149680 con 0x563cb02a5470 2023-11-06T03:08:07.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 10 client.0 _ll_register_callbacks cb 0x563cb006f080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:07.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44828/0 (socket says 172.21.15.157:44828) 2023-11-06T03:08:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 -- 172.21.15.157:0/2554638644 learned_addr learned my addr 172.21.15.157:0/2554638644 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 -- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 msgr2=0x563cb030bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb030bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 -- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb025ba00 msgr2=0x563cb030b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb025ba00 0x563cb030b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 -- 172.21.15.157:0/2554638644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cb006f5a0 con 0x563cb02a5d90 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe407fff640 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe400081c80 tx=0x7fe400081b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bf2e575bfc41f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40006c040 con 0x563cb02a5d90 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe40006c1a0 con 0x563cb02a5d90 2023-11-06T03:08:07.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb025eca0 con 0x563cb02a5d90 2023-11-06T03:08:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe40006c3a0 con 0x563cb02a5d90 2023-11-06T03:08:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 10 client.6135 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x563cafda7500 con 0x563cb02a5d90 2023-11-06T03:08:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563cb025eca0 con 0x563cb02a5d90 2023-11-06T03:08:07.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe400005d20 con 0x563cb02a5d90 2023-11-06T03:08:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 10 client.6135 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 10 client.6135 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 20 client.6135 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb02a9b60 0x563cb02a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 4 client.6135 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x563cb00a8df0 con 0x563cb02a9b60 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.225+0000 7fe4077fe640 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb02a9b60 0x563cb02a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe3e67fc640 10 client.6135 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe4077fe640 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb02a9b60 0x563cb02a9f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc88f8ce16ed6c5e server_cookie=5ba5aac0c4458365 in_seq=0 out_seq=0 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe3e67fc640 1 -- 172.21.15.157:0/2554638644 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563cb00a8df0 con 0x563cb02a9b60 2023-11-06T03:08:07.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe3e67fc640 10 client.6135 handle_command_reply: tid=0 2023-11-06T03:08:07.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 client.6135 shutdown 2023-11-06T03:08:07.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 msgr2=0x563cb02a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe400081c80 tx=0x7fe400081b00 comp rx=0 tx=0).stop 2023-11-06T03:08:07.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 shutdown_connections 2023-11-06T03:08:07.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x563cb02a9b60 0x563cb02a9f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563cb02a5d90 0x563cb02a91d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe3e67fc640 0 client.6135 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x563cb02a5470 0x563cb030bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 --2- 172.21.15.157:0/2554638644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cb025ba00 0x563cb030b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 >> 172.21.15.157:0/2554638644 conn(0x563cb02579b0 msgr2=0x563cb0259da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 shutdown_connections 2023-11-06T03:08:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 1 -- 172.21.15.157:0/2554638644 wait complete. 2023-11-06T03:08:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe41127f000 20 client.6135 trim_cache size 0 max 16384 2023-11-06T03:08:07.273 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:07.274 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:07.274 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "ea921ad1-9beb-4157-b3b5-76aabad8e859", 2023-11-06T03:08:07.274 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:07.274 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 msgr2=0x7fe3f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 0x7fe3f4067f20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8000c10 tx=0x7fe3f8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 msgr2=0x7fe408146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe4080b3b00 tx=0x7fe40000e4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 shutdown_connections 2023-11-06T03:08:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe3f4065a60 0x7fe3f4067f20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe408146710 0x7fe408146b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4080ce470 0x7fe4080acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 --2- 172.21.15.157:0/807109196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3f00047d0 0x7fe40814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 >> 172.21.15.157:0/807109196 conn(0x7fe408065930 msgr2=0x7fe408065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 shutdown_connections 2023-11-06T03:08:07.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.229+0000 7fe40e157640 1 -- 172.21.15.157:0/807109196 wait complete. 2023-11-06T03:08:07.278 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'ea921ad1-9beb-4157-b3b5-76aabad8e859', 'mode': 'asynchronous'}' 2023-11-06T03:08:07.278 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2023-11-06T03:08:07.279 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-06T03:08:07.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:07.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 Processor -- start 2023-11-06T03:08:07.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 -- start start 2023-11-06T03:08:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8f340ce9c0 0x7f8f34070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f3406c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f3406caa0 0x7f8f3415d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f340715f0 con 0x7f8f3406caa0 2023-11-06T03:08:07.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f340ceda0 con 0x7f8f340ce9c0 2023-11-06T03:08:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f3b395640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3406d290 con 0x7f8f34071140 2023-11-06T03:08:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f3406caa0 0x7f8f3415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f3406caa0 0x7f8f3415d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44834/0 (socket says 172.21.15.157:44834) 2023-11-06T03:08:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 -- 172.21.15.157:0/1552499978 learned_addr learned my addr 172.21.15.157:0/1552499978 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f38930640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f3406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39131640 1 --2- 172.21.15.157:0/1552499978 >> v2:172.21.15.195:3300/0 conn(0x7f8f340ce9c0 0x7f8f34070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 -- 172.21.15.157:0/1552499978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 msgr2=0x7f8f3406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f3406c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 -- 172.21.15.157:0/1552499978 >> v2:172.21.15.195:3300/0 conn(0x7f8f340ce9c0 msgr2=0x7f8f34070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 --2- 172.21.15.157:0/1552499978 >> v2:172.21.15.195:3300/0 conn(0x7f8f340ce9c0 0x7f8f34070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f39932640 1 -- 172.21.15.157:0/1552499978 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3406d570 con 0x7f8f3406caa0 2023-11-06T03:08:07.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.401+0000 7f8f38930640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f3406c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:07.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39932640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f3415d430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8f340b1b90 tx=0x7f8f2c01c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40fe2708d7ac217 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f27fff640 1 -- 172.21.15.157:0/1552499978 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0a6450 con 0x7f8f3406caa0 2023-11-06T03:08:07.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f27fff640 1 -- 172.21.15.157:0/1552499978 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0a65b0 con 0x7f8f3406caa0 2023-11-06T03:08:07.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f27fff640 1 -- 172.21.15.157:0/1552499978 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0a6940 con 0x7f8f3406caa0 2023-11-06T03:08:07.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- 172.21.15.157:0/1552499978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 msgr2=0x7f8f3415d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f3415d430 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8f340b1b90 tx=0x7f8f2c01c5c0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- 172.21.15.157:0/1552499978 shutdown_connections 2023-11-06T03:08:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f3415d430 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- 172.21.15.157:0/1552499978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f3406c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- 172.21.15.157:0/1552499978 >> v2:172.21.15.195:3300/0 conn(0x7f8f340ce9c0 0x7f8f34070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- 172.21.15.157:0/1552499978 >> 172.21.15.157:0/1552499978 conn(0x7f8f340b0ca0 msgr2=0x7f8f340b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- 172.21.15.157:0/1552499978 shutdown_connections 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- 172.21.15.157:0/1552499978 wait complete. 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 Processor -- start 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- start start 2023-11-06T03:08:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f340af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f340ce9c0 0x7f8f340af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340b33d0 con 0x7f8f3406caa0 2023-11-06T03:08:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340b36b0 con 0x7f8f340ce9c0 2023-11-06T03:08:07.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f3b395640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340b3990 con 0x7f8f34071140 2023-11-06T03:08:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44844/0 (socket says 172.21.15.157:44844) 2023-11-06T03:08:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 -- 172.21.15.157:0/3282254340 learned_addr learned my addr 172.21.15.157:0/3282254340 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39932640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f340ce9c0 0x7f8f340af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f38930640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f340af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 -- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 msgr2=0x7f8f340af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f340af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 -- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f340ce9c0 msgr2=0x7f8f340af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f340ce9c0 0x7f8f340af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.405+0000 7f8f39131640 1 -- 172.21.15.157:0/3282254340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c009b30 con 0x7f8f3406caa0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f39131640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c7f0 tx=0x7f8f3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30007820 con 0x7f8f3406caa0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30007980 con 0x7f8f3406caa0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340b3c70 con 0x7f8f3406caa0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30007d30 con 0x7f8f3406caa0 2023-11-06T03:08:07.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f34150200 con 0x7f8f3406caa0 2023-11-06T03:08:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f34066670 con 0x7f8f3406caa0 2023-11-06T03:08:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8f300113f0 con 0x7f8f3406caa0 2023-11-06T03:08:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 0x7f8f18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8f30083580 con 0x7f8f3406caa0 2023-11-06T03:08:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f38930640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 0x7f8f18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.409+0000 7f8f38930640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 0x7f8f18067f20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f3406c6d0 tx=0x7f8f28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.413+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8f30050e30 con 0x7f8f3406caa0 2023-11-06T03:08:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f340b3e20 con 0x7f8f3406caa0 2023-11-06T03:08:07.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f25ffb640 1 -- 172.21.15.157:0/3282254340 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f8f340b3e20 con 0x7f8f3406caa0 2023-11-06T03:08:07.604 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:07.604 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:07.604 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:08:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 msgr2=0x7f8f18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 0x7f8f18067f20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f3406c6d0 tx=0x7f8f28008040 comp rx=0 tx=0).stop 2023-11-06T03:08:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 msgr2=0x7f8f341475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c7f0 tx=0x7f8f3000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 shutdown_connections 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f18065a60 0x7f8f18067f20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f340ce9c0 0x7f8f340af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f34071140 0x7f8f340af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 --2- 172.21.15.157:0/3282254340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f3406caa0 0x7f8f341475d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 >> 172.21.15.157:0/3282254340 conn(0x7f8f340b0ca0 msgr2=0x7f8f340b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 shutdown_connections 2023-11-06T03:08:07.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.605+0000 7f8f3b395640 1 -- 172.21.15.157:0/3282254340 wait complete. 2023-11-06T03:08:07.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2023-11-06T03:08:07.741 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:07.742 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:07.742 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:07.756 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:07.757 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2023-11-06T03:08:07.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2023-11-06T03:08:07.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.881+0000 7f088c8e6640 1 Processor -- start 2023-11-06T03:08:07.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.881+0000 7f088c8e6640 1 -- start start 2023-11-06T03:08:07.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088c8e6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0888159950 0x7f08880b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088c8e6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08880ccb90 0x7f088806dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088c8e6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088806e190 con 0x7f088806d7f0 2023-11-06T03:08:07.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088c8e6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880702b0 con 0x7f08880ccb90 2023-11-06T03:08:07.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088c8e6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0888159d30 con 0x7f0888159950 2023-11-06T03:08:07.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08880ccb90 0x7f088806dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08880ccb90 0x7f088806dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44856/0 (socket says 172.21.15.157:44856) 2023-11-06T03:08:07.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 -- 172.21.15.157:0/428615619 learned_addr learned my addr 172.21.15.157:0/428615619 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f088659c640 1 --2- 172.21.15.157:0/428615619 >> v2:172.21.15.195:3300/0 conn(0x7f0888159950 0x7f08880b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 -- 172.21.15.157:0/428615619 >> v2:172.21.15.195:3300/0 conn(0x7f0888159950 msgr2=0x7f08880b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 --2- 172.21.15.157:0/428615619 >> v2:172.21.15.195:3300/0 conn(0x7f0888159950 0x7f08880b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 -- 172.21.15.157:0/428615619 >> v1:172.21.15.195:6789/0 conn(0x7f088806d7f0 legacy=0x7f08880cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.885+0000 7f0886d9d640 1 -- 172.21.15.157:0/428615619 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880b3f00 con 0x7f08880ccb90 2023-11-06T03:08:07.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f0886d9d640 1 --2- 172.21.15.157:0/428615619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08880ccb90 0x7f088806dc50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f088806c740 tx=0x7f088001ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea824293f1bcbd77 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088559a640 1 -- 172.21.15.157:0/428615619 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800a7ab0 con 0x7f08880ccb90 2023-11-06T03:08:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088559a640 1 -- 172.21.15.157:0/428615619 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800a7c10 con 0x7f08880ccb90 2023-11-06T03:08:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088559a640 1 -- 172.21.15.157:0/428615619 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800af3f0 con 0x7f08880ccb90 2023-11-06T03:08:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- 172.21.15.157:0/428615619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08880ccb90 msgr2=0x7f088806dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 --2- 172.21.15.157:0/428615619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08880ccb90 0x7f088806dc50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f088806c740 tx=0x7f088001ebe0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- 172.21.15.157:0/428615619 shutdown_connections 2023-11-06T03:08:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 --2- 172.21.15.157:0/428615619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08880ccb90 0x7f088806dc50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 --2- 172.21.15.157:0/428615619 >> v2:172.21.15.195:3300/0 conn(0x7f0888159950 0x7f08880b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- 172.21.15.157:0/428615619 >> 172.21.15.157:0/428615619 conn(0x7f088806a170 msgr2=0x7f088806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- 172.21.15.157:0/428615619 shutdown_connections 2023-11-06T03:08:07.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- 172.21.15.157:0/428615619 wait complete. 2023-11-06T03:08:07.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 Processor -- start 2023-11-06T03:08:07.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.889+0000 7f088c8e6640 1 -- start start 2023-11-06T03:08:07.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08880ccb90 0x7f0888154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0888159950 0x7f08880b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088815a920 con 0x7f088806d7f0 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c3b80 con 0x7f0888159950 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0888153240 con 0x7f08880ccb90 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44870/0 (socket says 172.21.15.157:44870) 2023-11-06T03:08:07.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 -- 172.21.15.157:0/2441073189 learned_addr learned my addr 172.21.15.157:0/2441073189 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0888159950 0x7f08880b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f0885d9b640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08880ccb90 0x7f0888154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 -- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08880ccb90 msgr2=0x7f0888154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08880ccb90 0x7f0888154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 -- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0888159950 msgr2=0x7f08880b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0888159950 0x7f08880b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 -- 172.21.15.157:0/2441073189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880009b30 con 0x7f088806d7f0 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088659c640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f087800e960 tx=0x7f087800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f0876ffd640 1 -- 172.21.15.157:0/2441073189 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878009690 con 0x7f088806d7f0 2023-11-06T03:08:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f0876ffd640 1 -- 172.21.15.157:0/2441073189 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800ca00 con 0x7f088806d7f0 2023-11-06T03:08:07.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f0876ffd640 1 -- 172.21.15.157:0/2441073189 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800cd70 con 0x7f088806d7f0 2023-11-06T03:08:07.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0888153520 con 0x7f088806d7f0 2023-11-06T03:08:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.893+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088815ab70 con 0x7f088806d7f0 2023-11-06T03:08:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 Processor -- start 2023-11-06T03:08:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 -- start start 2023-11-06T03:08:07.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0876ffd640 1 -- 172.21.15.157:0/2441073189 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0878013470 con 0x7f088806d7f0 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace43ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace305d00 con 0x55eace490de0 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace294270 con 0x55eace43e690 2023-11-06T03:08:07.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eace291840 con 0x55eace4904c0 2023-11-06T03:08:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0876ffd640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 0x7f0868067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0876ffd640 1 -- 172.21.15.157:0/2441073189 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f08780835c0 con 0x7f088806d7f0 2023-11-06T03:08:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44882/0 (socket says 172.21.15.157:44882) 2023-11-06T03:08:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 -- 172.21.15.157:0/1675833031 learned_addr learned my addr 172.21.15.157:0/1675833031 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0885d9b640 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace43ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 0x7f0868067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 -- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 msgr2=0x55eace4908a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4908a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 -- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 msgr2=0x55eace43ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace43ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 -- 172.21.15.157:0/1675833031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eacdf89eb0 con 0x55eace490de0 2023-11-06T03:08:07.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f088659c640 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f087800c520 tx=0x7f0878054710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120a787e12afa294 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0867fff640 1 -- 172.21.15.157:0/1675833031 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780588d0 con 0x55eace490de0 2023-11-06T03:08:07.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0867fff640 1 -- 172.21.15.157:0/1675833031 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087805ca80 con 0x55eace490de0 2023-11-06T03:08:07.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.897+0000 7f0867fff640 1 -- 172.21.15.157:0/1675833031 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780511b0 con 0x55eace490de0 2023-11-06T03:08:07.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- 172.21.15.157:0/1675833031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 msgr2=0x55eace43e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 0x7f0868067f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f08800b0000 tx=0x7f08800a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f087800c520 tx=0x7f0878054710 comp rx=0 tx=0).stop 2023-11-06T03:08:07.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- 172.21.15.157:0/1675833031 shutdown_connections 2023-11-06T03:08:07.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace43ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace43e150 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1675833031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4908a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- 172.21.15.157:0/1675833031 >> 172.21.15.157:0/1675833031 conn(0x55eace4447f0 msgr2=0x55eace43ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- 172.21.15.157:0/1675833031 shutdown_connections 2023-11-06T03:08:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- 172.21.15.157:0/1675833031 wait complete. 2023-11-06T03:08:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- start start 2023-11-06T03:08:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 Processor -- start 2023-11-06T03:08:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace451ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace28d130 con 0x55eace490de0 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace22c5c0 con 0x55eace43e690 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eace24a7e0 con 0x55eace4904c0 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088fa0e000 10 client.0 _ll_register_callbacks cb 0x55eace429dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44894/0 (socket says 172.21.15.157:44894) 2023-11-06T03:08:07.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 -- 172.21.15.157:0/1622122389 learned_addr learned my addr 172.21.15.157:0/1622122389 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0885d9b640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace451ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f088659c640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 -- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 msgr2=0x55eace451ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace451ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 -- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 msgr2=0x55eace4435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.901+0000 7f0886d9d640 1 -- 172.21.15.157:0/1622122389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088815ab70 con 0x55eace490de0 2023-11-06T03:08:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08800b1040 tx=0x7f088001d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45181de4d045df05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800c1070 con 0x55eace490de0 2023-11-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0880009250 con 0x55eace490de0 2023-11-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eace28ba80 con 0x55eace490de0 2023-11-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08800bc900 con 0x55eace490de0 2023-11-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 10 client.6153 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55eace2521c0 con 0x55eace490de0 2023-11-06T03:08:07.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eace28ba80 con 0x55eace490de0 2023-11-06T03:08:07.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f08800bca60 con 0x55eace490de0 2023-11-06T03:08:07.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 10 client.6153 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:07.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 10 client.6153 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:07.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 20 client.6153 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace4976f0 0x55eace497ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 4 client.6153 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eace24ba30 con 0x55eace4976f0 2023-11-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088659c640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace4976f0 0x55eace497ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 10 client.6153 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088659c640 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace4976f0 0x55eace497ad0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5946af9cfa4d7c8e server_cookie=34cd85f86ef79ccb in_seq=0 out_seq=0 2023-11-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 1 -- 172.21.15.157:0/1622122389 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eace24ba30 con 0x55eace4976f0 2023-11-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 10 client.6153 handle_command_reply: tid=0 2023-11-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 client.6153 shutdown 2023-11-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 msgr2=0x55eace451ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08800b1040 tx=0x7f088001d420 comp rx=0 tx=0).stop 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 shutdown_connections 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace4976f0 0x55eace497ad0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace451ff0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace451ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1622122389 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 >> 172.21.15.157:0/1622122389 conn(0x55eace4447f0 msgr2=0x55eace43d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f08657fa640 0 client.6153 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 shutdown_connections 2023-11-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 1 -- 172.21.15.157:0/1622122389 wait complete. 2023-11-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.905+0000 7f088fa0e000 20 client.6153 trim_cache size 0 max 16384 2023-11-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 Processor -- start 2023-11-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 -- start start 2023-11-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacdf4ab30 con 0x55eace490de0 2023-11-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace2e5360 con 0x55eace43e690 2023-11-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eace435c30 con 0x55eace4904c0 2023-11-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44896/0 (socket says 172.21.15.157:44896) 2023-11-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 -- 172.21.15.157:0/1297693751 learned_addr learned my addr 172.21.15.157:0/1297693751 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f088659c640 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0885d9b640 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 -- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 msgr2=0x55eace4e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 -- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 msgr2=0x55eace4e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 -- 172.21.15.157:0/1297693751 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eace238510 con 0x55eace490de0 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0886d9d640 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f088000e250 tx=0x7f08800be050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd59ed16608f28a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0874ff9640 1 -- 172.21.15.157:0/1297693751 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880009250 con 0x55eace490de0 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0874ff9640 1 -- 172.21.15.157:0/1297693751 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800c7420 con 0x55eace490de0 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.921+0000 7f0874ff9640 1 -- 172.21.15.157:0/1297693751 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800bca40 con 0x55eace490de0 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/1297693751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 msgr2=0x55eace4e8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f088000e250 tx=0x7f08800be050 comp rx=0 tx=0).stop 2023-11-06T03:08:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/1297693751 shutdown_connections 2023-11-06T03:08:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4e8980 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace4904c0 0x55eace4e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- 172.21.15.157:0/1297693751 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace43e690 0x55eace4e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/1297693751 >> 172.21.15.157:0/1297693751 conn(0x55eace4447f0 msgr2=0x55eace442670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/1297693751 shutdown_connections 2023-11-06T03:08:07.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/1297693751 wait complete. 2023-11-06T03:08:07.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- start start 2023-11-06T03:08:07.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 Processor -- start 2023-11-06T03:08:07.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace43e690 0x55eace48f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 0x55eace493490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace2340e0 con 0x55eace490de0 2023-11-06T03:08:07.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eace1c48e0 con 0x55eace4904c0 2023-11-06T03:08:07.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eace23c880 con 0x55eace43e690 2023-11-06T03:08:07.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 10 client.0 _ll_register_callbacks cb 0x55eace40a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:07.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44912/0 (socket says 172.21.15.157:44912) 2023-11-06T03:08:07.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 -- 172.21.15.157:0/2143048564 learned_addr learned my addr 172.21.15.157:0/2143048564 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:07.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088659c640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace43e690 0x55eace48f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0885d9b640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 0x55eace493490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 -- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace43e690 msgr2=0x55eace48f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace43e690 0x55eace48f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 -- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 msgr2=0x55eace493490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:07.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 0x55eace493490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 -- 172.21.15.157:0/2143048564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eace2521c0 con 0x55eace490de0 2023-11-06T03:08:07.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0885d9b640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 0x55eace493490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:07.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f0886d9d640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f088009a7d0 tx=0x7f08800ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48d72dc72acbf38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:07.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f08657fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800c7420 con 0x55eace490de0 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f08800dfc50 con 0x55eace490de0 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.925+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eace441500 con 0x55eace490de0 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08800dfe10 con 0x55eace490de0 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 10 client.6159 fetch_fsmap learned FSMap version 36 2023-11-06T03:08:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55eace22c080 con 0x55eace490de0 2023-11-06T03:08:07.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eace441500 con 0x55eace490de0 2023-11-06T03:08:07.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 36) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f08800dc020 con 0x55eace490de0 2023-11-06T03:08:07.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 10 client.6159 fetch_fsmap finished waiting for FSMap version 36 2023-11-06T03:08:07.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 10 client.6159 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:07.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 20 client.6159 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:07.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace493f10 0x55eace4ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:07.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 4 client.6159 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/"}] 2023-11-06T03:08:07.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55eace25b1f0 con 0x55eace493f10 2023-11-06T03:08:07.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088659c640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace493f10 0x55eace4ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:07.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 10 client.6159 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088659c640 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace493f10 0x55eace4ed2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9907d1173c43dfcf server_cookie=957a9c1773538a2a in_seq=0 out_seq=0 2023-11-06T03:08:07.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 1 -- 172.21.15.157:0/2143048564 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55eace25b1f0 con 0x55eace493f10 2023-11-06T03:08:07.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 10 client.6159 handle_command_reply: tid=0 2023-11-06T03:08:07.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 client.6159 shutdown 2023-11-06T03:08:07.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 msgr2=0x55eace4939d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f088009a7d0 tx=0x7f08800ba000 comp rx=0 tx=0).stop 2023-11-06T03:08:07.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 shutdown_connections 2023-11-06T03:08:07.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55eace493f10 0x55eace4ed2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eace490de0 0x55eace4939d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eace4904c0 0x55eace493490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f08657fa640 0 client.6159 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:07.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 --2- 172.21.15.157:0/2143048564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eace43e690 0x55eace48f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 >> 172.21.15.157:0/2143048564 conn(0x55eace4447f0 msgr2=0x55eace48eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 shutdown_connections 2023-11-06T03:08:07.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 1 -- 172.21.15.157:0/2143048564 wait complete. 2023-11-06T03:08:07.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088fa0e000 20 client.6159 trim_cache size 0 max 16384 2023-11-06T03:08:07.943 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:07.943 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:07.943 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "83dcf8a8-a13e-48d2-b42d-0f06626ceb44", 2023-11-06T03:08:07.944 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:07.944 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 msgr2=0x7f0868067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 0x7f0868067f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f08800b0000 tx=0x7f08800a9070 comp rx=0 tx=0).stop 2023-11-06T03:08:07.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 msgr2=0x7f08880c3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:07.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f087800e960 tx=0x7f087800ee20 comp rx=0 tx=0).stop 2023-11-06T03:08:07.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 shutdown_connections 2023-11-06T03:08:07.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0868065a90 0x7f0868067f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0888159950 0x7f08880b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08880ccb90 0x7f0888154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 --2- 172.21.15.157:0/2441073189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f088806d7f0 0x7f08880c3640 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:07.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 >> 172.21.15.157:0/2441073189 conn(0x7f088806a170 msgr2=0x7f08880cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:07.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 shutdown_connections 2023-11-06T03:08:07.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:07.929+0000 7f088c8e6640 1 -- 172.21.15.157:0/2441073189 wait complete. 2023-11-06T03:08:07.951 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '83dcf8a8-a13e-48d2-b42d-0f06626ceb44', 'mode': 'asynchronous'}' 2023-11-06T03:08:07.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:07.952 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2023-11-06T03:08:08.016 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2023-11-06T03:08:08.017 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:08.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.129+0000 7fa22ce22640 1 Processor -- start 2023-11-06T03:08:08.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.129+0000 7fa22ce22640 1 -- start start 2023-11-06T03:08:08.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22ce22640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa22815e5e0 0x7fa2280b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22ce22640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22815f580 con 0x7fa2281551e0 2023-11-06T03:08:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22ce22640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22815efc0 con 0x7fa22815ea90 2023-11-06T03:08:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22ce22640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2280704b0 con 0x7fa22815e5e0 2023-11-06T03:08:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa22815e5e0 0x7fa2280b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa22815e5e0 0x7fa2280b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44928/0 (socket says 172.21.15.157:44928) 2023-11-06T03:08:08.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa2281551e0 0x7fa22806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35022/0 (socket says 172.21.15.157:35022) 2023-11-06T03:08:08.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 -- 172.21.15.157:0/3346394987 learned_addr learned my addr 172.21.15.157:0/3346394987 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:08.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 -- 172.21.15.157:0/3346394987 >> v1:172.21.15.195:6789/0 conn(0x7fa22815ea90 legacy=0x7fa22815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:08.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 -- 172.21.15.157:0/3346394987 >> v1:172.21.15.157:6789/0 conn(0x7fa2281551e0 legacy=0x7fa22806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 -- 172.21.15.157:0/3346394987 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22815f2a0 con 0x7fa22815e5e0 2023-11-06T03:08:08.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa22559a640 1 -- 172.21.15.157:0/3346394987 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491000314 0 0) 0x7fa22815f580 con 0x7fa2281551e0 2023-11-06T03:08:08.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.133+0000 7fa225d9b640 1 --2- 172.21.15.157:0/3346394987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22815e5e0 0x7fa2280b3410 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa210009870 tx=0x7fa21001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e3aad4be3aca1d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22559a640 1 -- 172.21.15.157:0/3346394987 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004230 con 0x7fa22815e5e0 2023-11-06T03:08:08.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22559a640 1 -- 172.21.15.157:0/3346394987 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2100a7050 con 0x7fa22815e5e0 2023-11-06T03:08:08.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22559a640 1 -- 172.21.15.157:0/3346394987 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210003620 con 0x7fa22815e5e0 2023-11-06T03:08:08.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/3346394987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22815e5e0 msgr2=0x7fa2280b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 --2- 172.21.15.157:0/3346394987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22815e5e0 0x7fa2280b3410 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa210009870 tx=0x7fa21001c440 comp rx=0 tx=0).stop 2023-11-06T03:08:08.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/3346394987 shutdown_connections 2023-11-06T03:08:08.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 --2- 172.21.15.157:0/3346394987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22815e5e0 0x7fa2280b3410 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/3346394987 >> 172.21.15.157:0/3346394987 conn(0x7fa228065930 msgr2=0x7fa228065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/3346394987 shutdown_connections 2023-11-06T03:08:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/3346394987 wait complete. 2023-11-06T03:08:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 Processor -- start 2023-11-06T03:08:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- start start 2023-11-06T03:08:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 0x7fa2281517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa22815ea90 0x7fa22814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228151cf0 con 0x7fa2281551e0 2023-11-06T03:08:08.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228152aa0 con 0x7fa22815e5e0 2023-11-06T03:08:08.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa228151f20 con 0x7fa22815ea90 2023-11-06T03:08:08.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44942/0 (socket says 172.21.15.157:44942) 2023-11-06T03:08:08.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 -- 172.21.15.157:0/2741724939 learned_addr learned my addr 172.21.15.157:0/2741724939 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:08.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa225d9b640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 0x7fa2281517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 -- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa22815ea90 msgr2=0x7fa22814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa22815ea90 0x7fa22814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 -- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 msgr2=0x7fa2281517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 0x7fa2281517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 -- 172.21.15.157:0/2741724939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100094d0 con 0x7fa2281551e0 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa225d9b640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 0x7fa2281517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22659c640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa2200011f0 tx=0x7fa22000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200074e0 con 0x7fa2281551e0 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220007640 con 0x7fa2281551e0 2023-11-06T03:08:08.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa228152200 con 0x7fa2281551e0 2023-11-06T03:08:08.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220004a10 con 0x7fa2281551e0 2023-11-06T03:08:08.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.137+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2281527c0 con 0x7fa2281551e0 2023-11-06T03:08:08.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa228156400 con 0x7fa2281551e0 2023-11-06T03:08:08.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa220004b70 con 0x7fa2281551e0 2023-11-06T03:08:08.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa21effd640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 0x7fa208067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa220082830 con 0x7fa2281551e0 2023-11-06T03:08:08.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa225d9b640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 0x7fa208067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.141+0000 7fa225d9b640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 0x7fa208067f50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ad000 tx=0x7fa2100a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.145+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa220050050 con 0x7fa2281551e0 2023-11-06T03:08:08.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa228072750 con 0x7fa2281551e0 2023-11-06T03:08:08.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa21effd640 1 -- 172.21.15.157:0/2741724939 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fa220018090 con 0x7fa2281551e0 2023-11-06T03:08:08.336 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:08.336 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:07:42.668622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 36 2023-11-06T03:08:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 msgr2=0x7fa208067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 0x7fa208067f50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ad000 tx=0x7fa2100a6000 comp rx=0 tx=0).stop 2023-11-06T03:08:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 msgr2=0x7fa228154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa2200011f0 tx=0x7fa22000c9b0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 shutdown_connections 2023-11-06T03:08:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa208065a90 0x7fa208067f50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa22815ea90 0x7fa22814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa22815e5e0 0x7fa2281517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 --2- 172.21.15.157:0/2741724939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2281551e0 0x7fa228154010 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 >> 172.21.15.157:0/2741724939 conn(0x7fa228065930 msgr2=0x7fa2280af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 shutdown_connections 2023-11-06T03:08:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.337+0000 7fa22ce22640 1 -- 172.21.15.157:0/2741724939 wait complete. 2023-11-06T03:08:08.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2023-11-06T03:08:08.515 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:08.516 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:08.516 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:08.529 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:08.530 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:08.530 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:08.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:08.537 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2023-11-06T03:08:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:08:08.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:08:08.606 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014022 s, 74.8 MB/s 2023-11-06T03:08:08.607 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2023-11-06T03:08:08.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:08.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.773+0000 7fd356bc5640 1 Processor -- start 2023-11-06T03:08:08.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.773+0000 7fd356bc5640 1 -- start start 2023-11-06T03:08:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd356bc5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd35006d000 0x7fd3500ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd356bc5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35015e440 con 0x7fd3501599c0 2023-11-06T03:08:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd356bc5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35006c240 con 0x7fd3500b2af0 2023-11-06T03:08:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd356bc5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35006c520 con 0x7fd35006d000 2023-11-06T03:08:08.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd354961640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd3501599c0 0x7fd3500b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59356/0 (socket says 172.21.15.157:59356) 2023-11-06T03:08:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd354961640 1 -- 172.21.15.157:0/715224230 learned_addr learned my addr 172.21.15.157:0/715224230 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd355162640 1 --2- 172.21.15.157:0/715224230 >> v2:172.21.15.157:3300/0 conn(0x7fd35006d000 0x7fd3500ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd34f7fe640 1 -- 172.21.15.157:0/715224230 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630716463 0 0) 0x7fd35015e440 con 0x7fd3501599c0 2023-11-06T03:08:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd34f7fe640 1 -- 172.21.15.157:0/715224230 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd338005740 con 0x7fd3501599c0 2023-11-06T03:08:08.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd355162640 1 -- 172.21.15.157:0/715224230 >> v1:172.21.15.195:6789/0 conn(0x7fd3500b2af0 legacy=0x7fd350159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:08.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd355162640 1 -- 172.21.15.157:0/715224230 >> v1:172.21.15.157:6789/0 conn(0x7fd3501599c0 legacy=0x7fd3500b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd355162640 1 -- 172.21.15.157:0/715224230 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35006c800 con 0x7fd35006d000 2023-11-06T03:08:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd355162640 1 --2- 172.21.15.157:0/715224230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35006d000 0x7fd3500ae000 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10b58f35bcec19f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd34f7fe640 1 -- 172.21.15.157:0/715224230 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480a59c0 con 0x7fd35006d000 2023-11-06T03:08:08.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd34f7fe640 1 -- 172.21.15.157:0/715224230 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3480a5b20 con 0x7fd35006d000 2023-11-06T03:08:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.777+0000 7fd34f7fe640 1 -- 172.21.15.157:0/715224230 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480a5e90 con 0x7fd35006d000 2023-11-06T03:08:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/715224230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35006d000 msgr2=0x7fd3500ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 --2- 172.21.15.157:0/715224230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35006d000 0x7fd3500ae000 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34801c8e0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/715224230 shutdown_connections 2023-11-06T03:08:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 --2- 172.21.15.157:0/715224230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35006d000 0x7fd3500ae000 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/715224230 >> 172.21.15.157:0/715224230 conn(0x7fd350065930 msgr2=0x7fd350065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/715224230 shutdown_connections 2023-11-06T03:08:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/715224230 wait complete. 2023-11-06T03:08:08.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 Processor -- start 2023-11-06T03:08:08.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- start start 2023-11-06T03:08:08.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd35006d000 0x7fd35014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3501599c0 0x7fd3500ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd350159110 con 0x7fd3500b2af0 2023-11-06T03:08:08.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35014f220 con 0x7fd35006d000 2023-11-06T03:08:08.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35014e980 con 0x7fd3501599c0 2023-11-06T03:08:08.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35252/0 (socket says 172.21.15.157:35252) 2023-11-06T03:08:08.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 -- 172.21.15.157:0/2613944197 learned_addr learned my addr 172.21.15.157:0/2613944197 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:08.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd354961640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd35006d000 0x7fd35014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd355162640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3501599c0 0x7fd3500ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 -- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3501599c0 msgr2=0x7fd3500ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3501599c0 0x7fd3500ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 -- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd35006d000 msgr2=0x7fd35014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd35006d000 0x7fd35014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 -- 172.21.15.157:0/2613944197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009a70 con 0x7fd3500b2af0 2023-11-06T03:08:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34ffff640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c480 tx=0x7fd34400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344012830 con 0x7fd3500b2af0 2023-11-06T03:08:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3440129b0 con 0x7fd3500b2af0 2023-11-06T03:08:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344012d60 con 0x7fd3500b2af0 2023-11-06T03:08:08.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35014ec60 con 0x7fd3500b2af0 2023-11-06T03:08:08.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.781+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3501f5c80 con 0x7fd3500b2af0 2023-11-06T03:08:08.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3501f5ea0 con 0x7fd3500b2af0 2023-11-06T03:08:08.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd34401c460 con 0x7fd3500b2af0 2023-11-06T03:08:08.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd34d7fa640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 0x7fd33c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:08.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd344083150 con 0x7fd3500b2af0 2023-11-06T03:08:08.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd354961640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 0x7fd33c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:08.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.785+0000 7fd354961640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 0x7fd33c067f20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd340005890 tx=0x7fd340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:08.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.789+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd344050a00 con 0x7fd3500b2af0 2023-11-06T03:08:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.981+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3501f5ab0 con 0x7fd3500b2af0 2023-11-06T03:08:08.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.981+0000 7fd34d7fa640 1 -- 172.21.15.157:0/2613944197 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fd344054020 con 0x7fd3500b2af0 2023-11-06T03:08:08.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:08.980 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:08.345625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:08.981 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 37 2023-11-06T03:08:08.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 msgr2=0x7fd33c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 0x7fd33c067f20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd340005890 tx=0x7fd340009040 comp rx=0 tx=0).stop 2023-11-06T03:08:08.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 msgr2=0x7fd350156fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:08.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c480 tx=0x7fd34400c940 comp rx=0 tx=0).stop 2023-11-06T03:08:08.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 shutdown_connections 2023-11-06T03:08:08.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd33c065a60 0x7fd33c067f20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd3501599c0 0x7fd3500ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3500b2af0 0x7fd350156fc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 --2- 172.21.15.157:0/2613944197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd35006d000 0x7fd35014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:08.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 >> 172.21.15.157:0/2613944197 conn(0x7fd350065930 msgr2=0x7fd350065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:08.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 shutdown_connections 2023-11-06T03:08:08.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:08.985+0000 7fd356bc5640 1 -- 172.21.15.157:0/2613944197 wait complete. 2023-11-06T03:08:08.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2023-11-06T03:08:09.144 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:09.145 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:09.145 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:09.158 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:09.159 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:08:09.159 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:08:09.159 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:08:09.160 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:09.160 DEBUG:teuthology.orchestra.run.smithi157:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2023-11-06T03:08:09.160 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:08:09.219 INFO:teuthology.orchestra.run.smithi157.stdout:1099511630409 2023-11-06T03:08:09.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-06T03:08:09.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 Processor -- start 2023-11-06T03:08:09.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 -- start start 2023-11-06T03:08:09.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f977c169740 0x7f977c169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f977c0cedd0 0x7f977c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c06dec0 con 0x7f977c0ce220 2023-11-06T03:08:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0cf7c0 con 0x7f977c0cedd0 2023-11-06T03:08:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f97838f6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c06c1d0 con 0x7f977c169740 2023-11-06T03:08:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f9780e91640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f977c0ce220 0x7f977c06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59372/0 (socket says 172.21.15.157:59372) 2023-11-06T03:08:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f9780e91640 1 -- 172.21.15.157:0/551640432 learned_addr learned my addr 172.21.15.157:0/551640432 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f9781e93640 1 --2- 172.21.15.157:0/551640432 >> v2:172.21.15.157:3300/0 conn(0x7f977c0cedd0 0x7f977c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609017113 0 0) 0x7f977c06dec0 con 0x7f977c0ce220 2023-11-06T03:08:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9768003510 con 0x7f977c0ce220 2023-11-06T03:08:09.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.733+0000 7f9781692640 1 --2- 172.21.15.157:0/551640432 >> v2:172.21.15.195:3300/0 conn(0x7f977c169740 0x7f977c169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326338055 0 0) 0x7f9768003510 con 0x7f977c0ce220 2023-11-06T03:08:09.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 >> v2:172.21.15.195:3300/0 conn(0x7f977c169740 msgr2=0x7f977c169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 --2- 172.21.15.157:0/551640432 >> v2:172.21.15.195:3300/0 conn(0x7f977c169740 0x7f977c169b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 >> v2:172.21.15.157:3300/0 conn(0x7f977c0cedd0 msgr2=0x7f977c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 --2- 172.21.15.157:0/551640432 >> v2:172.21.15.157:3300/0 conn(0x7f977c0cedd0 0x7f977c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0b2100 con 0x7f977c0ce220 2023-11-06T03:08:09.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9774001a00 con 0x7f977c0ce220 2023-11-06T03:08:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 >> v1:172.21.15.157:6789/0 conn(0x7f977c0ce220 legacy=0x7f977c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768003ab0 0x7f9768005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 0x7f97680069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9768007060 0x7f9768007510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0cf7c0 con 0x7f9768003ab0 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c06c1d0 con 0x7f9768007060 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.761+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976801cf60 con 0x7f9768006570 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9781e93640 1 --2- 172.21.15.157:0/551640432 >> v2:172.21.15.157:3300/0 conn(0x7f977c0cedd0 0x7f977c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 0x7f97680069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9781692640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768003ab0 0x7f9768005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 -- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9768007060 msgr2=0x7f9768007510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9781e93640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9768007060 0x7f9768007510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9768007060 0x7f9768007510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 -- 172.21.15.157:0/551640432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768003ab0 msgr2=0x7f9768005f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768003ab0 0x7f9768005f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.765+0000 7f9780e91640 1 -- 172.21.15.157:0/551640432 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0b2100 con 0x7f9768006570 2023-11-06T03:08:09.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9781e93640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9768007060 0x7f9768007510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:09.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9781e93640 1 -- 172.21.15.157:0/551640432 reap_dead start 2023-11-06T03:08:09.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9780e91640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 0x7f97680069f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f977400a8a0 tx=0x7f977401d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55e1f1e57219bb88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740ad630 con 0x7f9768006570 2023-11-06T03:08:09.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97740a8070 con 0x7f9768006570 2023-11-06T03:08:09.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 -- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 msgr2=0x7f97680069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f9773fff640 1 -- 172.21.15.157:0/551640432 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740ad630 con 0x7f9768006570 2023-11-06T03:08:09.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 0x7f97680069f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f977400a8a0 tx=0x7f977401d360 comp rx=0 tx=0).stop 2023-11-06T03:08:09.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 -- 172.21.15.157:0/551640432 shutdown_connections 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 --2- 172.21.15.157:0/551640432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9768006570 0x7f97680069f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 -- 172.21.15.157:0/551640432 >> 172.21.15.157:0/551640432 conn(0x7f977c065d30 msgr2=0x7f977c066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 -- 172.21.15.157:0/551640432 shutdown_connections 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.769+0000 7f97838f6640 1 -- 172.21.15.157:0/551640432 wait complete. 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 Processor -- start 2023-11-06T03:08:09.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 -- start start 2023-11-06T03:08:09.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f977c089b20 0x7f977c164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f977c15c5b0 0x7f977c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c165ae0 con 0x7f9768006570 2023-11-06T03:08:09.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c164bc0 con 0x7f977c15c5b0 2023-11-06T03:08:09.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f97838f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c089fa0 con 0x7f977c089b20 2023-11-06T03:08:09.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f9781692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f9781692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35284/0 (socket says 172.21.15.157:35284) 2023-11-06T03:08:09.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.773+0000 7f9781692640 1 -- 172.21.15.157:0/1672230071 learned_addr learned my addr 172.21.15.157:0/1672230071 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 -- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f977c089b20 msgr2=0x7f977c164680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-06T03:08:09.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f977c089b20 0x7f977c164680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 -- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f977c15c5b0 msgr2=0x7f977c15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f977c15c5b0 0x7f977c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 -- 172.21.15.157:0/1672230071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c164e60 con 0x7f9768006570 2023-11-06T03:08:09.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9781692640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f976c033910 tx=0x7f976c033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9771ffb640 1 -- 172.21.15.157:0/1672230071 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0319e0 con 0x7f9768006570 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c165140 con 0x7f9768006570 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c165700 con 0x7f9768006570 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9771ffb640 1 -- 172.21.15.157:0/1672230071 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c031b40 con 0x7f9768006570 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9771ffb640 1 -- 172.21.15.157:0/1672230071 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c031eb0 con 0x7f9768006570 2023-11-06T03:08:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9771ffb640 1 -- 172.21.15.157:0/1672230071 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f976c03f400 con 0x7f9768006570 2023-11-06T03:08:09.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.777+0000 7f9771ffb640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 0x7f9768069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.781+0000 7f9771ffb640 1 -- 172.21.15.157:0/1672230071 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f976c0a57e0 con 0x7f9768006570 2023-11-06T03:08:09.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.781+0000 7f9780e91640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 0x7f9768069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.781+0000 7f9780e91640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 0x7f9768069a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f97740ad000 tx=0x7f97740a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.781+0000 7f9786a1e000 1 Processor -- start 2023-11-06T03:08:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.781+0000 7f9786a1e000 1 -- start start 2023-11-06T03:08:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df06568d60 con 0x55df066eac10 2023-11-06T03:08:09.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df064f73b0 con 0x55df066eb530 2023-11-06T03:08:09.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df064f4980 con 0x55df066a0f90 2023-11-06T03:08:09.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066eaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35288/0 (socket says 172.21.15.157:35288) 2023-11-06T03:08:09.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781692640 1 -- 172.21.15.157:0/3718662320 learned_addr learned my addr 172.21.15.157:0/3718662320 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9780e91640 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 -- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 msgr2=0x55df066a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 -- 172.21.15.157:0/3718662320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 msgr2=0x55df066eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 -- 172.21.15.157:0/3718662320 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df06491110 con 0x55df066a0f90 2023-11-06T03:08:09.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9781e93640 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a1370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f977c0704d0 tx=0x7f9778009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4133543c1d8506 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.785+0000 7f9766ffd640 1 -- 172.21.15.157:0/3718662320 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780040f0 con 0x55df066a0f90 2023-11-06T03:08:09.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9766ffd640 1 -- 172.21.15.157:0/3718662320 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9778003cf0 con 0x55df066a0f90 2023-11-06T03:08:09.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 msgr2=0x55df066a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a1370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f977c0704d0 tx=0x7f9778009530 comp rx=0 tx=0).stop 2023-11-06T03:08:09.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- 172.21.15.157:0/3718662320 shutdown_connections 2023-11-06T03:08:09.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a1370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 --2- 172.21.15.157:0/3718662320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- 172.21.15.157:0/3718662320 >> 172.21.15.157:0/3718662320 conn(0x55df0669cf40 msgr2=0x55df0669f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- 172.21.15.157:0/3718662320 shutdown_connections 2023-11-06T03:08:09.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- 172.21.15.157:0/3718662320 wait complete. 2023-11-06T03:08:09.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 -- start start 2023-11-06T03:08:09.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.789+0000 7f9786a1e000 1 Processor -- start 2023-11-06T03:08:09.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df064f0270 con 0x55df066eac10 2023-11-06T03:08:09.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0648f5a0 con 0x55df066eb530 2023-11-06T03:08:09.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df064ad7c0 con 0x55df066a0f90 2023-11-06T03:08:09.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 10 client.0 _ll_register_callbacks cb 0x55df064e4130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:09.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:56784/0 (socket says 172.21.15.157:56784) 2023-11-06T03:08:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781692640 1 -- 172.21.15.157:0/4138027290 learned_addr learned my addr 172.21.15.157:0/4138027290 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9780e91640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 -- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 msgr2=0x55df066a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 -- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 msgr2=0x55df066aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 -- 172.21.15.157:0/4138027290 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9778009e40 con 0x55df066eb530 2023-11-06T03:08:09.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781692640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:08:09.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781e93640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066acf50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9778009fd0 tx=0x7f9778009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c539bd1990259a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2023-11-06T03:08:09.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778009870 con 0x55df066eb530 2023-11-06T03:08:09.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df064b4350 con 0x55df066eb530 2023-11-06T03:08:09.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9778004060 con 0x55df066eb530 2023-11-06T03:08:09.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9778013740 con 0x55df066eb530 2023-11-06T03:08:09.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 10 client.5530 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:09.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55df061ece20 con 0x55df066eb530 2023-11-06T03:08:09.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df064b4350 con 0x55df066eb530 2023-11-06T03:08:09.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f97780138a0 con 0x55df066eb530 2023-11-06T03:08:09.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 10 client.5530 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:09.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 10 client.5530 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:09.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 20 client.5530 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:09.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df066f2470 0x55df066f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 4 client.5530 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df064eebc0 con 0x55df066f2470 2023-11-06T03:08:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.793+0000 7f9781692640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df066f2470 0x55df066f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.797+0000 7f9781692640 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df066f2470 0x55df066f2880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642a5c262167c3f9 server_cookie=3e0feb1b95ea0d0e in_seq=0 out_seq=0 2023-11-06T03:08:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.797+0000 7f975bfff640 10 client.5530 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:09.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f975bfff640 1 -- 172.21.15.157:0/4138027290 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df064eebc0 con 0x55df066f2470 2023-11-06T03:08:09.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f975bfff640 10 client.5530 handle_command_reply: tid=0 2023-11-06T03:08:09.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f9786a1e000 1 client.5530 shutdown 2023-11-06T03:08:09.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 msgr2=0x55df066acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066acf50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9778009fd0 tx=0x7f9778009b20 comp rx=0 tx=0).stop 2023-11-06T03:08:09.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 shutdown_connections 2023-11-06T03:08:09.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.801+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df066f2470 0x55df066f2880 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066eb530 0x55df066acf50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df066aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 --2- 172.21.15.157:0/4138027290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066a0f90 0x55df066a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 >> 172.21.15.157:0/4138027290 conn(0x55df0669cf40 msgr2=0x55df0669fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f975bfff640 0 client.5530 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:09.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 shutdown_connections 2023-11-06T03:08:09.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 1 -- 172.21.15.157:0/4138027290 wait complete. 2023-11-06T03:08:09.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.805+0000 7f9786a1e000 20 client.5530 trim_cache size 0 max 16384 2023-11-06T03:08:09.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 Processor -- start 2023-11-06T03:08:09.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 -- start start 2023-11-06T03:08:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df0669f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eb530 0x55df0669fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df061adaa0 con 0x55df066eac10 2023-11-06T03:08:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df064ebbe0 con 0x55df066a0f90 2023-11-06T03:08:09.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df064eda80 con 0x55df066eb530 2023-11-06T03:08:09.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35318/0 (socket says 172.21.15.157:35318) 2023-11-06T03:08:09.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 -- 172.21.15.157:0/2959684220 learned_addr learned my addr 172.21.15.157:0/2959684220 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9781692640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df0669f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9781e93640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eb530 0x55df0669fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 -- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eb530 msgr2=0x55df0669fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eb530 0x55df0669fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 -- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 msgr2=0x55df0669f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df0669f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 -- 172.21.15.157:0/2959684220 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df064b51a0 con 0x55df066eac10 2023-11-06T03:08:09.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9781692640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df0669f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:08:09.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f9780e91640 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f977401d3a0 tx=0x7f977400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80756ff23a3d8143 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f97677fe640 1 -- 172.21.15.157:0/2959684220 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740b8ce0 con 0x55df066eac10 2023-11-06T03:08:09.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f97677fe640 1 -- 172.21.15.157:0/2959684220 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97740be040 con 0x55df066eac10 2023-11-06T03:08:09.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.833+0000 7f97677fe640 1 -- 172.21.15.157:0/2959684220 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740b6650 con 0x55df066eac10 2023-11-06T03:08:09.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- 172.21.15.157:0/2959684220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 msgr2=0x55df0669f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f977401d3a0 tx=0x7f977400a060 comp rx=0 tx=0).stop 2023-11-06T03:08:09.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- 172.21.15.157:0/2959684220 shutdown_connections 2023-11-06T03:08:09.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eb530 0x55df0669fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eac10 0x55df0669f870 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- 172.21.15.157:0/2959684220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df0669f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- 172.21.15.157:0/2959684220 >> 172.21.15.157:0/2959684220 conn(0x55df0669cf40 msgr2=0x55df066e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- 172.21.15.157:0/2959684220 shutdown_connections 2023-11-06T03:08:09.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- 172.21.15.157:0/2959684220 wait complete. 2023-11-06T03:08:09.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- start start 2023-11-06T03:08:09.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 Processor -- start 2023-11-06T03:08:09.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df066e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eac10 0x55df0672a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df06533170 con 0x55df066eb530 2023-11-06T03:08:09.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df064f4270 con 0x55df066a0f90 2023-11-06T03:08:09.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df066aa180 con 0x55df066eac10 2023-11-06T03:08:09.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9786a1e000 10 client.0 _ll_register_callbacks cb 0x55df0649e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:09.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35328/0 (socket says 172.21.15.157:35328) 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 -- 172.21.15.157:0/936853493 learned_addr learned my addr 172.21.15.157:0/936853493 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781692640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df066e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9780e91640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eac10 0x55df0672a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 -- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eac10 msgr2=0x55df0672a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eac10 0x55df0672a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 -- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 msgr2=0x55df066e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df066e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 -- 172.21.15.157:0/936853493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c165700 con 0x55df066eb530 2023-11-06T03:08:09.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.837+0000 7f9781e93640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9778013b50 tx=0x7f9778012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e07aa1b3a76f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.841+0000 7f975bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:09.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778005bb0 con 0x55df066eb530 2023-11-06T03:08:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9778004000 con 0x55df066eb530 2023-11-06T03:08:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df0649fc20 con 0x55df066eb530 2023-11-06T03:08:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9778005f60 con 0x55df066eb530 2023-11-06T03:08:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 10 client.6198 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:09.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x7f97740b1040 con 0x55df066eb530 2023-11-06T03:08:09.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df0649fc20 con 0x55df066eb530 2023-11-06T03:08:09.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f97780062d0 con 0x55df066eb530 2023-11-06T03:08:09.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 10 client.6198 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:09.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 10 client.6198 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:09.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 20 client.6198 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:09.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df0672b4e0 0x55df0672b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 4 client.6198 mds_command: new command op to 5868 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:08:09.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.845+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df064e92f0 con 0x55df0672b4e0 2023-11-06T03:08:09.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9781692640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df0672b4e0 0x55df0672b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9781692640 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df0672b4e0 0x55df0672b8c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77dd1652c88e55f server_cookie=ee15c67476dcc187 in_seq=0 out_seq=0 2023-11-06T03:08:09.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f975bfff640 10 client.6198 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:09.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f975bfff640 1 -- 172.21.15.157:0/936853493 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df064e92f0 con 0x55df0672b4e0 2023-11-06T03:08:09.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f975bfff640 10 client.6198 handle_command_reply: tid=0 2023-11-06T03:08:09.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 client.6198 shutdown 2023-11-06T03:08:09.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 msgr2=0x55df0672abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9778013b50 tx=0x7f9778012000 comp rx=0 tx=0).stop 2023-11-06T03:08:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 shutdown_connections 2023-11-06T03:08:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55df0672b4e0 0x55df0672b8c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df066eb530 0x55df0672abb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55df066eac10 0x55df0672a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 --2- 172.21.15.157:0/936853493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55df066a0f90 0x55df066e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 >> 172.21.15.157:0/936853493 conn(0x55df0669cf40 msgr2=0x55df066a02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f975bfff640 0 client.6198 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 shutdown_connections 2023-11-06T03:08:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 1 -- 172.21.15.157:0/936853493 wait complete. 2023-11-06T03:08:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f9786a1e000 20 client.6198 trim_cache size 0 max 16384 2023-11-06T03:08:09.854 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2023-11-06T03:08:09.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 msgr2=0x7f9768069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 0x7f9768069a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f97740ad000 tx=0x7f97740a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:09.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 msgr2=0x7f977c202640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f976c033910 tx=0x7f976c033dd0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 shutdown_connections 2023-11-06T03:08:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9768067550 0x7f9768069a10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f977c15c5b0 0x7f977c15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f977c089b20 0x7f977c164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 --2- 172.21.15.157:0/1672230071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9768006570 0x7f977c202640 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 >> 172.21.15.157:0/1672230071 conn(0x7f977c065d30 msgr2=0x7f977c066110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 shutdown_connections 2023-11-06T03:08:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.849+0000 7f97838f6640 1 -- 172.21.15.157:0/1672230071 wait complete. 2023-11-06T03:08:09.875 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a49.00000000 parent 2023-11-06T03:08:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.913+0000 7f0712d51080 1 Processor -- start 2023-11-06T03:08:09.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- start start 2023-11-06T03:08:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c4ef2d0 0x55f98c4f3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c57ba30 0x55f98c4c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f98c40a450 con 0x55f98c57df30 2023-11-06T03:08:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f98c40a150 con 0x55f98c57ba30 2023-11-06T03:08:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f98c400dc0 con 0x55f98c4ef2d0 2023-11-06T03:08:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35356/0 (socket says 172.21.15.157:35356) 2023-11-06T03:08:09.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 -- 172.21.15.157:0/1423082514 learned_addr learned my addr 172.21.15.157:0/1423082514 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071086b640 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c4ef2d0 0x55f98c4f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f070bfff640 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c57ba30 0x55f98c4c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 -- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c4ef2d0 msgr2=0x55f98c4f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c4ef2d0 0x55f98c4f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 -- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c57ba30 msgr2=0x55f98c4c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c57ba30 0x55f98c4c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 -- 172.21.15.157:0/1423082514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c00a100 con 0x55f98c57df30 2023-11-06T03:08:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f071106c640 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f070c0ae040 tx=0x7f070c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcf30b9d7d70be3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f070affd640 1 -- 172.21.15.157:0/1423082514 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0ab5b0 con 0x55f98c57df30 2023-11-06T03:08:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f070affd640 1 -- 172.21.15.157:0/1423082514 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0ab710 con 0x55f98c57df30 2023-11-06T03:08:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f070affd640 1 -- 172.21.15.157:0/1423082514 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0aba80 con 0x55f98c57df30 2023-11-06T03:08:09.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- 172.21.15.157:0/1423082514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 msgr2=0x55f98c578040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f070c0ae040 tx=0x7f070c0033b0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- 172.21.15.157:0/1423082514 shutdown_connections 2023-11-06T03:08:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c578040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c57ba30 0x55f98c4c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 --2- 172.21.15.157:0/1423082514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c4ef2d0 0x55f98c4f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.917+0000 7f0712d51080 1 -- 172.21.15.157:0/1423082514 >> 172.21.15.157:0/1423082514 conn(0x55f98c4ed7a0 msgr2=0x55f98c578720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- 172.21.15.157:0/1423082514 shutdown_connections 2023-11-06T03:08:09.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- 172.21.15.157:0/1423082514 wait complete. 2023-11-06T03:08:09.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 Processor -- start 2023-11-06T03:08:09.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- start start 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c4ef2d0 0x55f98c578640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c57ba30 0x55f98c57c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c598f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f98c59afa0 con 0x55f98c57df30 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f98c59a520 con 0x55f98c4ef2d0 2023-11-06T03:08:09.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f98c59a7c0 con 0x55f98c57ba30 2023-11-06T03:08:09.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c598f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c598f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35358/0 (socket says 172.21.15.157:35358) 2023-11-06T03:08:09.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 -- 172.21.15.157:0/1600911620 learned_addr learned my addr 172.21.15.157:0/1600911620 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:09.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f070bfff640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c57ba30 0x55f98c57c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071086b640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c4ef2d0 0x55f98c578640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 -- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c57ba30 msgr2=0x55f98c57c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f98c57ba30 0x55f98c57c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 -- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c4ef2d0 msgr2=0x55f98c578640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c4ef2d0 0x55f98c578640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 -- 172.21.15.157:0/1600911620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f98c433190 con 0x55f98c57df30 2023-11-06T03:08:09.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071086b640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f98c4ef2d0 0x55f98c578640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:09.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f071106c640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c598f80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f070c018c20 tx=0x7f070c017d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0709ffb640 1 -- 172.21.15.157:0/1600911620 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0b4840 con 0x55f98c57df30 2023-11-06T03:08:09.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0709ffb640 1 -- 172.21.15.157:0/1600911620 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0b49a0 con 0x55f98c57df30 2023-11-06T03:08:09.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f98c418ff0 con 0x55f98c57df30 2023-11-06T03:08:09.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0709ffb640 1 -- 172.21.15.157:0/1600911620 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0b4d10 con 0x55f98c57df30 2023-11-06T03:08:09.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.921+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f98c40c430 con 0x55f98c57df30 2023-11-06T03:08:09.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f0709ffb640 1 -- 172.21.15.157:0/1600911620 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f070c0b2040 con 0x55f98c57df30 2023-11-06T03:08:09.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f0709ffb640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06f4064440 0x7f06f4066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f0709ffb640 1 -- 172.21.15.157:0/1600911620 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f070c11cbd0 con 0x55f98c57df30 2023-11-06T03:08:09.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f0712d51080 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x55f98c578b80 0x55f98c574210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:09.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f071086b640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06f4064440 0x7f06f4066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 --> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] -- osd_op(unknown.0.0:1 8.1f 8:f95eeb57:::10000000a49.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v8 -- 0x55f98c57c960 con 0x55f98c578b80 2023-11-06T03:08:09.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f070bfff640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x55f98c578b80 0x55f98c574210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:09.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f070bfff640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x55f98c578b80 0x55f98c574210 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.925+0000 7f071086b640 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06f4064440 0x7f06f4066900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0704000f70 tx=0x7f0704001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:09.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f070bfff640 1 -- 172.21.15.157:0/1600911620 <== osd.0 v2:172.21.15.157:6801/38220 1 ==== osd_op_reply(1 10000000a49.00000000 [rmxattr] v55'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f06fc0098c0 con 0x55f98c578b80 2023-11-06T03:08:09.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x55f98c578b80 msgr2=0x55f98c574210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x55f98c578b80 0x55f98c574210 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:09.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06f4064440 msgr2=0x7f06f4066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06f4064440 0x7f06f4066900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0704000f70 tx=0x7f0704001630 comp rx=0 tx=0).stop 2023-11-06T03:08:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 msgr2=0x55f98c598f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 --2- 172.21.15.157:0/1600911620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f98c57df30 0x55f98c598f80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f070c018c20 tx=0x7f070c017d90 comp rx=0 tx=0).stop 2023-11-06T03:08:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f071106c640 1 -- 172.21.15.157:0/1600911620 reap_dead start 2023-11-06T03:08:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 shutdown_connections 2023-11-06T03:08:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 >> 172.21.15.157:0/1600911620 conn(0x55f98c4ed7a0 msgr2=0x55f98c4b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:09.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 shutdown_connections 2023-11-06T03:08:09.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:09.929+0000 7f0712d51080 1 -- 172.21.15.157:0/1600911620 wait complete. 2023-11-06T03:08:09.937 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2023-11-06T03:08:09.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2023-11-06T03:08:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.109+0000 7f989a01a640 1 Processor -- start 2023-11-06T03:08:10.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.109+0000 7f989a01a640 1 -- start start 2023-11-06T03:08:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f989415aca0 0x7f98940ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940cef50 con 0x7f98940d02f0 2023-11-06T03:08:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940cf230 con 0x7f9894071940 2023-11-06T03:08:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98940cf510 con 0x7f989415aca0 2023-11-06T03:08:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f98937fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f989415aca0 0x7f98940ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9893fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f98940d02f0 0x7f98940cc960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59390/0 (socket says 172.21.15.157:59390) 2023-11-06T03:08:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9893fff640 1 -- 172.21.15.157:0/3433039613 learned_addr learned my addr 172.21.15.157:0/3433039613 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f98937fe640 1 -- 172.21.15.157:0/3433039613 >> v1:172.21.15.195:6789/0 conn(0x7f9894071940 legacy=0x7f98940cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f98937fe640 1 -- 172.21.15.157:0/3433039613 >> v1:172.21.15.157:6789/0 conn(0x7f98940d02f0 legacy=0x7f98940cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9892ffd640 1 -- 172.21.15.157:0/3433039613 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197681096 0 0) 0x7f98940cef50 con 0x7f98940d02f0 2023-11-06T03:08:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f98937fe640 1 -- 172.21.15.157:0/3433039613 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940cf7f0 con 0x7f989415aca0 2023-11-06T03:08:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f98937fe640 1 --2- 172.21.15.157:0/3433039613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989415aca0 0x7f98940ce5e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3392dae111411b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9892ffd640 1 -- 172.21.15.157:0/3433039613 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840ab660 con 0x7f989415aca0 2023-11-06T03:08:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9892ffd640 1 -- 172.21.15.157:0/3433039613 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840ab7c0 con 0x7f989415aca0 2023-11-06T03:08:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f9892ffd640 1 -- 172.21.15.157:0/3433039613 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840abb50 con 0x7f989415aca0 2023-11-06T03:08:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- 172.21.15.157:0/3433039613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989415aca0 msgr2=0x7f98940ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 --2- 172.21.15.157:0/3433039613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989415aca0 0x7f98940ce5e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c440 comp rx=0 tx=0).stop 2023-11-06T03:08:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- 172.21.15.157:0/3433039613 shutdown_connections 2023-11-06T03:08:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 --2- 172.21.15.157:0/3433039613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f989415aca0 0x7f98940ce5e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- 172.21.15.157:0/3433039613 >> 172.21.15.157:0/3433039613 conn(0x7f989406d560 msgr2=0x7f989406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- 172.21.15.157:0/3433039613 shutdown_connections 2023-11-06T03:08:10.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.113+0000 7f989a01a640 1 -- 172.21.15.157:0/3433039613 wait complete. 2023-11-06T03:08:10.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 Processor -- start 2023-11-06T03:08:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- start start 2023-11-06T03:08:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 0x7f98941580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f989415aca0 0x7f98940b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940c7ae0 con 0x7f98940d02f0 2023-11-06T03:08:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941585f0 con 0x7f9894071940 2023-11-06T03:08:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940ca7f0 con 0x7f989415aca0 2023-11-06T03:08:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35362/0 (socket says 172.21.15.157:35362) 2023-11-06T03:08:10.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 -- 172.21.15.157:0/3483994184 learned_addr learned my addr 172.21.15.157:0/3483994184 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f98937fe640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 0x7f98941580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f9893fff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f989415aca0 0x7f98940b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 -- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f989415aca0 msgr2=0x7f98940b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f989415aca0 0x7f98940b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 -- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 msgr2=0x7f98941580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 0x7f98941580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 -- 172.21.15.157:0/3483994184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9884009420 con 0x7f98940d02f0 2023-11-06T03:08:10.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f98937fe640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 0x7f98941580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f988bfff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c450 tx=0x7f987c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f9890ff9640 1 -- 172.21.15.157:0/3483994184 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c012830 con 0x7f98940d02f0 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f9890ff9640 1 -- 172.21.15.157:0/3483994184 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0129b0 con 0x7f98940d02f0 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98940caad0 con 0x7f98940d02f0 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f9890ff9640 1 -- 172.21.15.157:0/3483994184 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c012d60 con 0x7f98940d02f0 2023-11-06T03:08:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.117+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940c8010 con 0x7f98940d02f0 2023-11-06T03:08:10.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 Processor -- start 2023-11-06T03:08:10.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- start start 2023-11-06T03:08:10.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9890ff9640 1 -- 172.21.15.157:0/3483994184 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f987c01c460 con 0x7f98940d02f0 2023-11-06T03:08:10.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9890ff9640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 0x7f9874067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9890ff9640 1 -- 172.21.15.157:0/3483994184 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f987c083160 con 0x7f98940d02f0 2023-11-06T03:08:10.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a882250 con 0x55747aa03f00 2023-11-06T03:08:10.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a811220 con 0x55747a9ba300 2023-11-06T03:08:10.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747a80e7f0 con 0x55747aa04820 2023-11-06T03:08:10.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35374/0 (socket says 172.21.15.157:35374) 2023-11-06T03:08:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 -- 172.21.15.157:0/1457598555 learned_addr learned my addr 172.21.15.157:0/1457598555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9893fff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 0x7f9874067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9893fff640 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 -- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 msgr2=0x55747a9b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 -- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 msgr2=0x55747a9ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 -- 172.21.15.157:0/1457598555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55747a8c0ff0 con 0x55747aa03f00 2023-11-06T03:08:10.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98937fe640 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98840ad040 tx=0x7f98840a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26131762467c92ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98897fa640 1 -- 172.21.15.157:0/1457598555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840abb30 con 0x55747aa03f00 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98897fa640 1 -- 172.21.15.157:0/1457598555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840b9070 con 0x55747aa03f00 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f98897fa640 1 -- 172.21.15.157:0/1457598555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840c0480 con 0x55747aa03f00 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- 172.21.15.157:0/1457598555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 msgr2=0x55747aa042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f9893fff640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 0x7f9874067f20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f988c001570 tx=0x7f988c007d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98840ad040 tx=0x7f98840a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:10.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- 172.21.15.157:0/1457598555 shutdown_connections 2023-11-06T03:08:10.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 --2- 172.21.15.157:0/1457598555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa042e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- 172.21.15.157:0/1457598555 >> 172.21.15.157:0/1457598555 conn(0x55747a9b1670 msgr2=0x55747a9b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- 172.21.15.157:0/1457598555 shutdown_connections 2023-11-06T03:08:10.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.121+0000 7f989d142000 1 -- 172.21.15.157:0/1457598555 wait complete. 2023-11-06T03:08:10.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- start start 2023-11-06T03:08:10.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 Processor -- start 2023-11-06T03:08:10.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a80a0e0 con 0x55747aa03f00 2023-11-06T03:08:10.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a7a93f0 con 0x55747a9ba300 2023-11-06T03:08:10.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747a7c7830 con 0x55747aa04820 2023-11-06T03:08:10.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 10 client.0 _ll_register_callbacks cb 0x55747a9a57e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:10.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35382/0 (socket says 172.21.15.157:35382) 2023-11-06T03:08:10.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 -- 172.21.15.157:0/1009599328 learned_addr learned my addr 172.21.15.157:0/1009599328 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f9893fff640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98937fe640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 -- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 msgr2=0x55747a9f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 -- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 msgr2=0x55747a9bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9bf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 -- 172.21.15.157:0/1009599328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940cf7f0 con 0x55747aa03f00 2023-11-06T03:08:10.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f988bfff640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f987c05c0f0 tx=0x7f987c054e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81f6354a8eab429 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:10.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c05c620 con 0x55747aa03f00 2023-11-06T03:08:10.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f987c07ce80 con 0x55747aa03f00 2023-11-06T03:08:10.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55747a7d1df0 con 0x55747aa03f00 2023-11-06T03:08:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987c081260 con 0x55747aa03f00 2023-11-06T03:08:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 10 client.6225 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55747a506dd0 con 0x55747aa03f00 2023-11-06T03:08:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c06f030 con 0x55747aa03f00 2023-11-06T03:08:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f987c0580d0 con 0x55747aa03f00 2023-11-06T03:08:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 10 client.6225 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 10 client.6225 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 20 client.6225 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747aa141b0 0x55747aa165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 4 client.6225 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55747a7d5e90 con 0x55747aa141b0 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98937fe640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747aa141b0 0x55747aa165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 10 client.6225 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98937fe640 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747aa141b0 0x55747aa165a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6f4b249fa2ca4a server_cookie=92ac4e8411fea026 in_seq=0 out_seq=0 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 1 -- 172.21.15.157:0/1009599328 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55747a7d5e90 con 0x55747aa141b0 2023-11-06T03:08:10.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 10 client.6225 handle_command_reply: tid=0 2023-11-06T03:08:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 client.6225 shutdown 2023-11-06T03:08:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 msgr2=0x55747a9bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f987c05c0f0 tx=0x7f987c054e80 comp rx=0 tx=0).stop 2023-11-06T03:08:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 shutdown_connections 2023-11-06T03:08:10.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747aa141b0 0x55747aa165a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9bf8f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 --2- 172.21.15.157:0/1009599328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747a9bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f98727fc640 0 client.6225 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 >> 172.21.15.157:0/1009599328 conn(0x55747a9b1670 msgr2=0x55747a9b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 shutdown_connections 2023-11-06T03:08:10.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 1 -- 172.21.15.157:0/1009599328 wait complete. 2023-11-06T03:08:10.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.125+0000 7f989d142000 20 client.6225 trim_cache size 0 max 16384 2023-11-06T03:08:10.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 Processor -- start 2023-11-06T03:08:10.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- start start 2023-11-06T03:08:10.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa17cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a4c7e80 con 0x55747aa03f00 2023-11-06T03:08:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a8078f0 con 0x55747a9ba300 2023-11-06T03:08:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747a7aa0a0 con 0x55747aa04820 2023-11-06T03:08:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35398/0 (socket says 172.21.15.157:35398) 2023-11-06T03:08:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 -- 172.21.15.157:0/2549598254 learned_addr learned my addr 172.21.15.157:0/2549598254 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 -- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 msgr2=0x55747a9eade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 -- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 msgr2=0x55747aa17cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa17cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 -- 172.21.15.157:0/2549598254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55747a7cf1e0 con 0x55747aa03f00 2023-11-06T03:08:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f988bfff640 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f987c05a450 tx=0x7f987c057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b92e4f4c3c83f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f9889ffb640 1 -- 172.21.15.157:0/2549598254 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c05c620 con 0x55747aa03f00 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f9889ffb640 1 -- 172.21.15.157:0/2549598254 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c06f070 con 0x55747aa03f00 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f9889ffb640 1 -- 172.21.15.157:0/2549598254 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00e040 con 0x55747aa03f00 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- 172.21.15.157:0/2549598254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 msgr2=0x55747aa18200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f987c05a450 tx=0x7f987c057000 comp rx=0 tx=0).stop 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- 172.21.15.157:0/2549598254 shutdown_connections 2023-11-06T03:08:10.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9eade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747aa18200 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 --2- 172.21.15.157:0/2549598254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa17cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- 172.21.15.157:0/2549598254 >> 172.21.15.157:0/2549598254 conn(0x55747a9b1670 msgr2=0x55747aa165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- 172.21.15.157:0/2549598254 shutdown_connections 2023-11-06T03:08:10.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- 172.21.15.157:0/2549598254 wait complete. 2023-11-06T03:08:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 -- start start 2023-11-06T03:08:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.141+0000 7f989d142000 1 Processor -- start 2023-11-06T03:08:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa5e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9ee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747a804480 con 0x55747aa03f00 2023-11-06T03:08:10.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747aa1adb0 con 0x55747a9ba300 2023-11-06T03:08:10.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747a7d90e0 con 0x55747aa04820 2023-11-06T03:08:10.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 10 client.0 _ll_register_callbacks cb 0x7f9884009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:10.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35412/0 (socket says 172.21.15.157:35412) 2023-11-06T03:08:10.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 -- 172.21.15.157:0/197308673 learned_addr learned my addr 172.21.15.157:0/197308673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 -- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 msgr2=0x55747a9ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 -- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 msgr2=0x55747aa5e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa5e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 -- 172.21.15.157:0/197308673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55747a506dd0 con 0x55747aa03f00 2023-11-06T03:08:10.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f988bfff640 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f987c054580 tx=0x7f987c003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb217c94ba75ee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:10.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00e280 con 0x55747aa03f00 2023-11-06T03:08:10.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f987c06be40 con 0x55747aa03f00 2023-11-06T03:08:10.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55747a7cb340 con 0x55747aa03f00 2023-11-06T03:08:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987c00e040 con 0x55747aa03f00 2023-11-06T03:08:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 10 client.6231 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55747a7b5330 con 0x55747aa03f00 2023-11-06T03:08:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55747a7cb340 con 0x55747aa03f00 2023-11-06T03:08:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f987c05c190 con 0x55747aa03f00 2023-11-06T03:08:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 10 client.6231 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 20 client.6231 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:10.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747a9eeb80 0x55747a9eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 4 client.6231 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2023-11-06T03:08:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55747a803160 con 0x55747a9eeb80 2023-11-06T03:08:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98937fe640 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747a9eeb80 0x55747a9eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 10 client.6231 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98937fe640 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747a9eeb80 0x55747a9eef60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697cca6bd9a93a05 server_cookie=7a81ce26d605522e in_seq=0 out_seq=0 2023-11-06T03:08:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 1 -- 172.21.15.157:0/197308673 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55747a803160 con 0x55747a9eeb80 2023-11-06T03:08:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 10 client.6231 handle_command_reply: tid=0 2023-11-06T03:08:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 client.6231 shutdown 2023-11-06T03:08:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 msgr2=0x55747a9ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f987c054580 tx=0x7f987c003770 comp rx=0 tx=0).stop 2023-11-06T03:08:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 shutdown_connections 2023-11-06T03:08:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55747a9eeb80 0x55747a9eef60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55747aa04820 0x55747a9ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55747aa03f00 0x55747a9ee100 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 --2- 172.21.15.157:0/197308673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55747a9ba300 0x55747aa5e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 >> 172.21.15.157:0/197308673 conn(0x55747a9b1670 msgr2=0x55747aa165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f98727fc640 0 client.6231 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 shutdown_connections 2023-11-06T03:08:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 1 -- 172.21.15.157:0/197308673 wait complete. 2023-11-06T03:08:10.172 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f079439f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a49 [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v578 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6057=0-4194304@1} caps={6057=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6057 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5582b6cf1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a49:[<0x10000000001/new_file_0 v578>,<0x10000000000/ceph-qa-suite v80>,<0x1/scrub_test_path v38>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-06T03:08:10.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989d142000 20 client.6231 trim_cache size 0 max 16384 2023-11-06T03:08:10.173 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:10.173 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:10.173 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "23040d9e-0682-4301-a12a-5423214eb804", 2023-11-06T03:08:10.173 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:10.174 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:10.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 msgr2=0x7f9874067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 0x7f9874067f20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f988c001570 tx=0x7f988c007d90 comp rx=0 tx=0).stop 2023-11-06T03:08:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 msgr2=0x7f989414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c450 tx=0x7f987c00c910 comp rx=0 tx=0).stop 2023-11-06T03:08:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 shutdown_connections 2023-11-06T03:08:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9874065a60 0x7f9874067f20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f989415aca0 0x7f98940b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98940d02f0 0x7f989414e5a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 --2- 172.21.15.157:0/3483994184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9894071940 0x7f98941580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 >> 172.21.15.157:0/3483994184 conn(0x7f989406d560 msgr2=0x7f98940c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 shutdown_connections 2023-11-06T03:08:10.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.145+0000 7f989a01a640 1 -- 172.21.15.157:0/3483994184 wait complete. 2023-11-06T03:08:10.178 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '23040d9e-0682-4301-a12a-5423214eb804', 'mode': 'asynchronous'}' 2023-11-06T03:08:10.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T03:08:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 Processor -- start 2023-11-06T03:08:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 -- start start 2023-11-06T03:08:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c0cd9b0 0x7f344c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c071ea0 con 0x7f344c0cc5d0 2023-11-06T03:08:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c15d6e0 con 0x7f344c0cd9b0 2023-11-06T03:08:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f3453b70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0cfdc0 con 0x7f344c06b8b0 2023-11-06T03:08:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c0cd9b0 0x7f344c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f344c0cd9b0 0x7f344c0afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35422/0 (socket says 172.21.15.157:35422) 2023-11-06T03:08:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 -- 172.21.15.157:0/823773730 learned_addr learned my addr 172.21.15.157:0/823773730 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345210d640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 -- 172.21.15.157:0/823773730 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 msgr2=0x7f344c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c069d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 -- 172.21.15.157:0/823773730 >> v1:172.21.15.195:6789/0 conn(0x7f344c0cc5d0 legacy=0x7f344c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.285+0000 7f345110b640 1 -- 172.21.15.157:0/823773730 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c072110 con 0x7f344c0cd9b0 2023-11-06T03:08:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f345110b640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c0afd00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd1ba74470bfffd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f345090a640 1 -- 172.21.15.157:0/823773730 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a89b0 con 0x7f344c0cd9b0 2023-11-06T03:08:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f345090a640 1 -- 172.21.15.157:0/823773730 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0a8b10 con 0x7f344c0cd9b0 2023-11-06T03:08:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f345090a640 1 -- 172.21.15.157:0/823773730 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a8ea0 con 0x7f344c0cd9b0 2023-11-06T03:08:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 -- 172.21.15.157:0/823773730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 msgr2=0x7f344c0afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c0afd00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c01ecf0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 -- 172.21.15.157:0/823773730 shutdown_connections 2023-11-06T03:08:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 --2- 172.21.15.157:0/823773730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c0afd00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 -- 172.21.15.157:0/823773730 >> 172.21.15.157:0/823773730 conn(0x7f344c0b23e0 msgr2=0x7f344c0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 -- 172.21.15.157:0/823773730 shutdown_connections 2023-11-06T03:08:10.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.289+0000 7f3453b70640 1 -- 172.21.15.157:0/823773730 wait complete. 2023-11-06T03:08:10.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 Processor -- start 2023-11-06T03:08:10.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- start start 2023-11-06T03:08:10.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c0cc5d0 0x7f344c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0c5430 con 0x7f344c0cd9b0 2023-11-06T03:08:10.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0ca870 con 0x7f344c0cc5d0 2023-11-06T03:08:10.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c0c5680 con 0x7f344c06b8b0 2023-11-06T03:08:10.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35424/0 (socket says 172.21.15.157:35424) 2023-11-06T03:08:10.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 -- 172.21.15.157:0/1968918216 learned_addr learned my addr 172.21.15.157:0/1968918216 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345110b640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c0cc5d0 0x7f344c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345190c640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 -- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 msgr2=0x7f344c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 -- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c0cc5d0 msgr2=0x7f344c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c0cc5d0 0x7f344c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 -- 172.21.15.157:0/1968918216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0094d0 con 0x7f344c0cd9b0 2023-11-06T03:08:10.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f345210d640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f344400cc30 tx=0x7f344400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f34427fc640 1 -- 172.21.15.157:0/1968918216 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344400ce40 con 0x7f344c0cd9b0 2023-11-06T03:08:10.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f34427fc640 1 -- 172.21.15.157:0/1968918216 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34440039a0 con 0x7f344c0cd9b0 2023-11-06T03:08:10.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c0c5960 con 0x7f344c0cd9b0 2023-11-06T03:08:10.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f34427fc640 1 -- 172.21.15.157:0/1968918216 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444003d30 con 0x7f344c0cd9b0 2023-11-06T03:08:10.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c0ca430 con 0x7f344c0cd9b0 2023-11-06T03:08:10.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f34427fc640 1 -- 172.21.15.157:0/1968918216 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3444003e90 con 0x7f344c0cd9b0 2023-11-06T03:08:10.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.293+0000 7f34427fc640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f34427fc640 1 -- 172.21.15.157:0/1968918216 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3444083710 con 0x7f344c0cd9b0 2023-11-06T03:08:10.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 Processor -- start 2023-11-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- start start 2023-11-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a5500d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a5501640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a53c8c10 con 0x5600a554b130 2023-11-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a53573e0 con 0x5600a554ba50 2023-11-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600a53549b0 con 0x5600a5501260 2023-11-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35438/0 (socket says 172.21.15.157:35438) 2023-11-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 -- 172.21.15.157:0/3590878316 learned_addr learned my addr 172.21.15.157:0/3590878316 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 -- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 msgr2=0x5600a5501640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a5501640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 -- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 msgr2=0x5600a5500d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345110b640 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a5500d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a5500d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 -- 172.21.15.157:0/3590878316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a504ce20 con 0x5600a554b130 2023-11-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f345190c640 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f344800d880 tx=0x7f3448010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30bd468cf1fe633 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f342f7fe640 1 -- 172.21.15.157:0/3590878316 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448014ba0 con 0x5600a554b130 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f342f7fe640 1 -- 172.21.15.157:0/3590878316 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448014d00 con 0x5600a554b130 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f342f7fe640 1 -- 172.21.15.157:0/3590878316 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344801f600 con 0x5600a554b130 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- 172.21.15.157:0/3590878316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 msgr2=0x5600a554b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f344800d880 tx=0x7f3448010000 comp rx=0 tx=0).stop 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- 172.21.15.157:0/3590878316 shutdown_connections 2023-11-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a5501640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a5500d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 --2- 172.21.15.157:0/3590878316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554b130 0x5600a554b510 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- 172.21.15.157:0/3590878316 >> 172.21.15.157:0/3590878316 conn(0x5600a55079f0 msgr2=0x5600a54ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- 172.21.15.157:0/3590878316 shutdown_connections 2023-11-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- 172.21.15.157:0/3590878316 wait complete. 2023-11-06T03:08:10.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 -- start start 2023-11-06T03:08:10.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.297+0000 7f3456c98000 1 Processor -- start 2023-11-06T03:08:10.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a554c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a554c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a53502a0 con 0x5600a554ba50 2023-11-06T03:08:10.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a52ef470 con 0x5600a554b130 2023-11-06T03:08:10.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600a530d660 con 0x5600a5501260 2023-11-06T03:08:10.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 10 client.0 _ll_register_callbacks cb 0x5600a54ecc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:10.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35440/0 (socket says 172.21.15.157:35440) 2023-11-06T03:08:10.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 -- 172.21.15.157:0/4223826617 learned_addr learned my addr 172.21.15.157:0/4223826617 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 -- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 msgr2=0x5600a554c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a554c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 -- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 msgr2=0x5600a554c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345110b640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a554c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a554c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 -- 172.21.15.157:0/4223826617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344800d5c0 con 0x5600a554ba50 2023-11-06T03:08:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345210d640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3444054b70 tx=0x7f3444054be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd1086195b8ffc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344406c1c0 con 0x5600a554ba50 2023-11-06T03:08:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f344405cbc0 con 0x5600a554ba50 2023-11-06T03:08:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a530d0a0 con 0x5600a554ba50 2023-11-06T03:08:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344407bbd0 con 0x5600a554ba50 2023-11-06T03:08:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 10 client.6243 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x5600a53151d0 con 0x5600a554ba50 2023-11-06T03:08:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5600a530d0a0 con 0x5600a554ba50 2023-11-06T03:08:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3444016030 con 0x5600a554ba50 2023-11-06T03:08:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 10 client.6243 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:10.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 10 client.6243 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:10.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 20 client.6243 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:10.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a554d4d0 0x5600a555cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 4 client.6243 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:10.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a534ebf0 con 0x5600a554d4d0 2023-11-06T03:08:10.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345190c640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a554d4d0 0x5600a555cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 10 client.6243 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f345190c640 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a554d4d0 0x5600a555cf10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1c9e678f576ec4 server_cookie=7d806668fb871e54 in_seq=0 out_seq=0 2023-11-06T03:08:10.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 1 -- 172.21.15.157:0/4223826617 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5600a534ebf0 con 0x5600a554d4d0 2023-11-06T03:08:10.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 10 client.6243 handle_command_reply: tid=0 2023-11-06T03:08:10.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 client.6243 shutdown 2023-11-06T03:08:10.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 msgr2=0x5600a554cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3444054b70 tx=0x7f3444054be0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 shutdown_connections 2023-11-06T03:08:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a554d4d0 0x5600a555cf10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a554ba50 0x5600a554cac0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a554c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f342cff9640 0 client.6243 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 --2- 172.21.15.157:0/4223826617 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a5501260 0x5600a554c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 >> 172.21.15.157:0/4223826617 conn(0x5600a55079f0 msgr2=0x5600a54ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 shutdown_connections 2023-11-06T03:08:10.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 1 -- 172.21.15.157:0/4223826617 wait complete. 2023-11-06T03:08:10.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.301+0000 7f3456c98000 20 client.6243 trim_cache size 0 max 16384 2023-11-06T03:08:10.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 Processor -- start 2023-11-06T03:08:10.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- start start 2023-11-06T03:08:10.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554b130 0x5600a555df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a555e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a500daa0 con 0x5600a5501260 2023-11-06T03:08:10.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a53d8c70 con 0x5600a554ba50 2023-11-06T03:08:10.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600a5504550 con 0x5600a554b130 2023-11-06T03:08:10.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35444/0 (socket says 172.21.15.157:35444) 2023-11-06T03:08:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 -- 172.21.15.157:0/2829756161 learned_addr learned my addr 172.21.15.157:0/2829756161 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 -- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554b130 msgr2=0x5600a555df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345110b640 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554b130 0x5600a555df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554b130 0x5600a555df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 -- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 msgr2=0x5600a555e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a555e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 -- 172.21.15.157:0/2829756161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a52fb520 con 0x5600a5501260 2023-11-06T03:08:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3448009010 tx=0x7f3448012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82efb7dc849fd22 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f342ffff640 1 -- 172.21.15.157:0/2829756161 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344802f3e0 con 0x5600a5501260 2023-11-06T03:08:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f342ffff640 1 -- 172.21.15.157:0/2829756161 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448018070 con 0x5600a5501260 2023-11-06T03:08:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f342ffff640 1 -- 172.21.15.157:0/2829756161 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448014e00 con 0x5600a5501260 2023-11-06T03:08:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- 172.21.15.157:0/2829756161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 msgr2=0x5600a555ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3448009010 tx=0x7f3448012000 comp rx=0 tx=0).stop 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- 172.21.15.157:0/2829756161 shutdown_connections 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554ba50 0x5600a555e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554b130 0x5600a555df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- 172.21.15.157:0/2829756161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a555ee70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- 172.21.15.157:0/2829756161 >> 172.21.15.157:0/2829756161 conn(0x5600a55079f0 msgr2=0x5600a5506820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- 172.21.15.157:0/2829756161 shutdown_connections 2023-11-06T03:08:10.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- 172.21.15.157:0/2829756161 wait complete. 2023-11-06T03:08:10.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- start start 2023-11-06T03:08:10.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 Processor -- start 2023-11-06T03:08:10.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a558c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554ba50 0x5600a558cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a554cb40 con 0x5600a5501260 2023-11-06T03:08:10.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a530e400 con 0x5600a554b130 2023-11-06T03:08:10.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600a52f57a0 con 0x5600a554ba50 2023-11-06T03:08:10.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f3456c98000 10 client.0 _ll_register_callbacks cb 0x5600a5314cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35460/0 (socket says 172.21.15.157:35460) 2023-11-06T03:08:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.317+0000 7f345190c640 1 -- 172.21.15.157:0/1500063401 learned_addr learned my addr 172.21.15.157:0/1500063401 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345110b640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a558c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 -- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554ba50 msgr2=0x5600a558cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554ba50 0x5600a558cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 -- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 msgr2=0x5600a558c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:10.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a558c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 -- 172.21.15.157:0/1500063401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a504ce20 con 0x5600a5501260 2023-11-06T03:08:10.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345190c640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3448009010 tx=0x7f3448003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37925462f17bc9eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:10.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:10.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448014de0 con 0x5600a5501260 2023-11-06T03:08:10.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f344802f960 con 0x5600a5501260 2023-11-06T03:08:10.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a5319620 con 0x5600a5501260 2023-11-06T03:08:10.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344802fb20 con 0x5600a5501260 2023-11-06T03:08:10.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 10 client.6249 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:10.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x5600a52ef090 con 0x5600a5501260 2023-11-06T03:08:10.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5600a5319620 con 0x5600a5501260 2023-11-06T03:08:10.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f344802fc80 con 0x5600a5501260 2023-11-06T03:08:10.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 10 client.6249 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 20 client.6249 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a555ea60 0x5600a558d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 4 client.6249 mds_command: new command op to 5868 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:08:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600a5349320 con 0x5600a555ea60 2023-11-06T03:08:10.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345110b640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a555ea60 0x5600a558d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:10.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 10 client.6249 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f345110b640 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a555ea60 0x5600a558d0d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56cf8c2b4cf7f40 server_cookie=8c53eef8038e572c in_seq=0 out_seq=0 2023-11-06T03:08:10.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 1 -- 172.21.15.157:0/1500063401 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x5600a5349320 con 0x5600a555ea60 2023-11-06T03:08:10.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 10 client.6249 handle_command_reply: tid=0 2023-11-06T03:08:10.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 client.6249 shutdown 2023-11-06T03:08:10.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 msgr2=0x5600a558bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3448009010 tx=0x7f3448003700 comp rx=0 tx=0).stop 2023-11-06T03:08:10.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 shutdown_connections 2023-11-06T03:08:10.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5600a555ea60 0x5600a558d0d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5600a554ba50 0x5600a558cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600a554b130 0x5600a558c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f342cff9640 0 client.6249 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:10.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 --2- 172.21.15.157:0/1500063401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5600a5501260 0x5600a558bcf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 >> 172.21.15.157:0/1500063401 conn(0x5600a55079f0 msgr2=0x5600a5506f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 shutdown_connections 2023-11-06T03:08:10.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 1 -- 172.21.15.157:0/1500063401 wait complete. 2023-11-06T03:08:10.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3456c98000 20 client.6249 trim_cache size 0 max 16384 2023-11-06T03:08:10.362 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:08:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:08:10.363 INFO:teuthology.orchestra.run.smithi157.stdout: "23040d9e-0682-4301-a12a-5423214eb804": { 2023-11-06T03:08:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2023-11-06T03:08:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "23040d9e-0682-4301-a12a-5423214eb804", 2023-11-06T03:08:10.364 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "" 2023-11-06T03:08:10.365 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:10.365 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:10.365 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:10.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 msgr2=0x7f3430067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).stop 2023-11-06T03:08:10.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 msgr2=0x7f344c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:10.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f344400cc30 tx=0x7f344400a5c0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 shutdown_connections 2023-11-06T03:08:10.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3430065a60 0x7f3430067f20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f344c0cd9b0 0x7f344c15ed60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f344c0cc5d0 0x7f344c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 --2- 172.21.15.157:0/1968918216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f344c06b8b0 0x7f344c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:10.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 >> 172.21.15.157:0/1968918216 conn(0x7f344c0b23e0 msgr2=0x7f344c0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:10.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 shutdown_connections 2023-11-06T03:08:10.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:10.321+0000 7f3453b70640 1 -- 172.21.15.157:0/1968918216 wait complete. 2023-11-06T03:08:10.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:08:10.428 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:08:15.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T03:08:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.513+0000 7f4bc95af640 1 Processor -- start 2023-11-06T03:08:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.513+0000 7f4bc95af640 1 -- start start 2023-11-06T03:08:15.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc95af640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4bc40aff30 0x7f4bc40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc95af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bc40b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc95af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc406bda0 con 0x7f4bc4071db0 2023-11-06T03:08:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc95af640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40b2e80 con 0x7f4bc40aff30 2023-11-06T03:08:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc95af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b0310 con 0x7f4bc40b2490 2023-11-06T03:08:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc27fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4bc4071db0 0x7f4bc406af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59396/0 (socket says 172.21.15.157:59396) 2023-11-06T03:08:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc27fc640 1 -- 172.21.15.157:0/2006904928 learned_addr learned my addr 172.21.15.157:0/2006904928 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/2006904928 >> v2:172.21.15.195:3300/0 conn(0x7f4bc40aff30 0x7f4bc40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bc40b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489104225 0 0) 0x7f4bc406bda0 con 0x7f4bc4071db0 2023-11-06T03:08:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac003530 con 0x7f4bc4071db0 2023-11-06T03:08:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528584281 0 0) 0x7f4bac003530 con 0x7f4bc4071db0 2023-11-06T03:08:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 msgr2=0x7f4bc40b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bc40b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 >> v2:172.21.15.195:3300/0 conn(0x7f4bc40aff30 msgr2=0x7f4bc40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 --2- 172.21.15.157:0/2006904928 >> v2:172.21.15.195:3300/0 conn(0x7f4bc40aff30 0x7f4bc40ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc406b7e0 con 0x7f4bc4071db0 2023-11-06T03:08:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4bb8001680 con 0x7f4bc4071db0 2023-11-06T03:08:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 >> v1:172.21.15.157:6789/0 conn(0x7f4bc4071db0 legacy=0x7f4bc406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bac003980 0x7f4bac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b2e80 con 0x7f4bac004d10 2023-11-06T03:08:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40b0310 con 0x7f4bac003980 2023-11-06T03:08:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00b990 con 0x7f4bc40b2490 2023-11-06T03:08:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bac003980 0x7f4bac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 msgr2=0x7f4bac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bc40b2490 0x7f4bac0047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bac003980 msgr2=0x7f4bac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bac003980 0x7f4bac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2006904928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac003530 con 0x7f4bac004d10 2023-11-06T03:08:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.517+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fc0 tx=0x7f4bbc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f51e715b6eed934 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0a5a70 con 0x7f4bac004d10 2023-11-06T03:08:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc00f040 con 0x7f4bac004d10 2023-11-06T03:08:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc1ffb640 1 -- 172.21.15.157:0/2006904928 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0a5e10 con 0x7f4bac004d10 2023-11-06T03:08:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/2006904928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac004d10 msgr2=0x7f4bac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 --2- 172.21.15.157:0/2006904928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bac004d10 0x7f4bac005190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fc0 tx=0x7f4bbc01cc60 comp rx=0 tx=0).stop 2023-11-06T03:08:15.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2006904928 reap_dead start 2023-11-06T03:08:15.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/2006904928 shutdown_connections 2023-11-06T03:08:15.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/2006904928 >> 172.21.15.157:0/2006904928 conn(0x7f4bc40cd490 msgr2=0x7f4bc40cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/2006904928 shutdown_connections 2023-11-06T03:08:15.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/2006904928 wait complete. 2023-11-06T03:08:15.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 Processor -- start 2023-11-06T03:08:15.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- start start 2023-11-06T03:08:15.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bac004d10 0x7f4bc4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bc40b3ed0 0x7f4bc40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c9670 con 0x7f4bc40c91e0 2023-11-06T03:08:15.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c9950 con 0x7f4bc40b3ed0 2023-11-06T03:08:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc40c9c30 con 0x7f4bac004d10 2023-11-06T03:08:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35484/0 (socket says 172.21.15.157:35484) 2023-11-06T03:08:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 -- 172.21.15.157:0/959931010 learned_addr learned my addr 172.21.15.157:0/959931010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bac004d10 0x7f4bc4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bc40b3ed0 0x7f4bc40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 -- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bac004d10 msgr2=0x7f4bc4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bac004d10 0x7f4bc4155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 -- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bc40b3ed0 msgr2=0x7f4bc40cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bc40b3ed0 0x7f4bc40cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 -- 172.21.15.157:0/959931010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40c7c10 con 0x7f4bc40c91e0 2023-11-06T03:08:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010b10 tx=0x7f4bb8005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bab7fe640 1 -- 172.21.15.157:0/959931010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800eb80 con 0x7f4bc40c91e0 2023-11-06T03:08:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bab7fe640 1 -- 172.21.15.157:0/959931010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800ece0 con 0x7f4bc40c91e0 2023-11-06T03:08:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc40c9f10 con 0x7f4bc40c91e0 2023-11-06T03:08:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bab7fe640 1 -- 172.21.15.157:0/959931010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80145c0 con 0x7f4bc40c91e0 2023-11-06T03:08:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.521+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40b4380 con 0x7f4bc40c91e0 2023-11-06T03:08:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bab7fe640 1 -- 172.21.15.157:0/959931010 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4bb8014720 con 0x7f4bc40c91e0 2023-11-06T03:08:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bab7fe640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 0x7f4bb0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bab7fe640 1 -- 172.21.15.157:0/959931010 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4bb80860d0 con 0x7f4bc40c91e0 2023-11-06T03:08:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 0x7f4bb0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 Processor -- start 2023-11-06T03:08:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 -- start start 2023-11-06T03:08:15.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 0x7f4bb0067f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009a30 tx=0x7f4bb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651eb7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651d34c10 con 0x556651eb7a50 2023-11-06T03:08:15.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651cc33e0 con 0x556651eb7130 2023-11-06T03:08:15.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556651cc09b0 con 0x556651e6d260 2023-11-06T03:08:15.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35490/0 (socket says 172.21.15.157:35490) 2023-11-06T03:08:15.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 -- 172.21.15.157:0/1097716926 learned_addr learned my addr 172.21.15.157:0/1097716926 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651eb7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 -- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 msgr2=0x556651e6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e6d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 -- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 msgr2=0x556651eb7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651eb7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 -- 172.21.15.157:0/1097716926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566519b8e20 con 0x556651eb7a50 2023-11-06T03:08:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4bb805f0f0 tx=0x7f4bb8057d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a1170fa75f455e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4ba8ff9640 1 -- 172.21.15.157:0/1097716926 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb805f620 con 0x556651eb7a50 2023-11-06T03:08:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4ba8ff9640 1 -- 172.21.15.157:0/1097716926 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb805b1b0 con 0x556651eb7a50 2023-11-06T03:08:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.525+0000 7f4ba8ff9640 1 -- 172.21.15.157:0/1097716926 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8053d50 con 0x556651eb7a50 2023-11-06T03:08:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/1097716926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 msgr2=0x556651e6cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4bb805f0f0 tx=0x7f4bb8057d40 comp rx=0 tx=0).stop 2023-11-06T03:08:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/1097716926 shutdown_connections 2023-11-06T03:08:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651e6cd20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/1097716926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651eb7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/1097716926 >> 172.21.15.157:0/1097716926 conn(0x556651e739f0 msgr2=0x556651e6b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/1097716926 shutdown_connections 2023-11-06T03:08:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/1097716926 wait complete. 2023-11-06T03:08:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- start start 2023-11-06T03:08:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 Processor -- start 2023-11-06T03:08:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651eb7130 0x556651ebeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651eb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651cbc2a0 con 0x556651e6d260 2023-11-06T03:08:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651c5b470 con 0x556651eb7a50 2023-11-06T03:08:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556651c79660 con 0x556651eb7130 2023-11-06T03:08:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 10 client.0 _ll_register_callbacks cb 0x556651e58c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35500/0 (socket says 172.21.15.157:35500) 2023-11-06T03:08:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 -- 172.21.15.157:0/3707205881 learned_addr learned my addr 172.21.15.157:0/3707205881 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651eb8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651eb7130 0x556651ebeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 -- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651eb7130 msgr2=0x556651ebeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651eb7130 0x556651ebeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 -- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 msgr2=0x556651eb8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651eb8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 -- 172.21.15.157:0/3707205881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc40b4380 con 0x556651e6d260 2023-11-06T03:08:15.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4013ed0 tx=0x7f4bb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f7e3957d1ea985 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:15.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004220 con 0x556651e6d260 2023-11-06T03:08:15.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556651c790a0 con 0x556651e6d260 2023-11-06T03:08:15.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4bb4004380 con 0x556651e6d260 2023-11-06T03:08:15.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb4016e70 con 0x556651e6d260 2023-11-06T03:08:15.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 10 client.6261 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:15.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x556651c811d0 con 0x556651e6d260 2023-11-06T03:08:15.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556651c790a0 con 0x556651e6d260 2023-11-06T03:08:15.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4bb4003620 con 0x556651e6d260 2023-11-06T03:08:15.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 10 client.6261 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:15.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:15.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 20 client.6261 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:15.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb8de0 0x556651eb91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 4 client.6261 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:15.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556651cbabf0 con 0x556651eb8de0 2023-11-06T03:08:15.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb8de0 0x556651eb91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4ba1ffb640 10 client.6261 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.529+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb8de0 0x556651eb91f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84af8fb0b1492e2a server_cookie=e7875de432308f88 in_seq=0 out_seq=0 2023-11-06T03:08:15.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3707205881 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556651cbabf0 con 0x556651eb8de0 2023-11-06T03:08:15.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4ba1ffb640 10 client.6261 handle_command_reply: tid=0 2023-11-06T03:08:15.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 client.6261 shutdown 2023-11-06T03:08:15.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 msgr2=0x556651ebe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4013ed0 tx=0x7f4bb4004740 comp rx=0 tx=0).stop 2023-11-06T03:08:15.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 shutdown_connections 2023-11-06T03:08:15.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb8de0 0x556651eb91f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651eb8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651eb7130 0x556651ebeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4ba1ffb640 0 client.6261 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3707205881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651e6d260 0x556651ebe600 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 >> 172.21.15.157:0/3707205881 conn(0x556651e739f0 msgr2=0x556651e6bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 shutdown_connections 2023-11-06T03:08:15.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3707205881 wait complete. 2023-11-06T03:08:15.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.533+0000 7f4bcc6d7000 20 client.6261 trim_cache size 0 max 16384 2023-11-06T03:08:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 Processor -- start 2023-11-06T03:08:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 -- start start 2023-11-06T03:08:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e72850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651e72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651979aa0 con 0x556651eb7a50 2023-11-06T03:08:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651cb9ab0 con 0x556651eb7130 2023-11-06T03:08:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556651c601d0 con 0x556651e6d260 2023-11-06T03:08:15.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.545+0000 7f4bc37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35506/0 (socket says 172.21.15.157:35506) 2023-11-06T03:08:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2338745292 learned_addr learned my addr 172.21.15.157:0/2338745292 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 msgr2=0x556651e72850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e72850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 msgr2=0x556651e72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651e72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 -- 172.21.15.157:0/2338745292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556651c67520 con 0x556651eb7a50 2023-11-06T03:08:15.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc002240 tx=0x7f4bbc01b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39722e6e08c47405 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba97fa640 1 -- 172.21.15.157:0/2338745292 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0a5a70 con 0x556651eb7a50 2023-11-06T03:08:15.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba97fa640 1 -- 172.21.15.157:0/2338745292 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc0a5bd0 con 0x556651eb7a50 2023-11-06T03:08:15.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba97fa640 1 -- 172.21.15.157:0/2338745292 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0039a0 con 0x556651eb7a50 2023-11-06T03:08:15.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/2338745292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 msgr2=0x556651ebc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc002240 tx=0x7f4bbc01b4a0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/2338745292 shutdown_connections 2023-11-06T03:08:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7a50 0x556651ebc1e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7130 0x556651e72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/2338745292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651e72850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/2338745292 >> 172.21.15.157:0/2338745292 conn(0x556651e739f0 msgr2=0x556651e72140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/2338745292 shutdown_connections 2023-11-06T03:08:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/2338745292 wait complete. 2023-11-06T03:08:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- start start 2023-11-06T03:08:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 Processor -- start 2023-11-06T03:08:15.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651f1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651f21b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651c617a0 con 0x556651eb7130 2023-11-06T03:08:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556651cfecd0 con 0x556651eb7a50 2023-11-06T03:08:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556651cb96a0 con 0x556651e6d260 2023-11-06T03:08:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 10 client.0 _ll_register_callbacks cb 0x556651c80cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35520/0 (socket says 172.21.15.157:35520) 2023-11-06T03:08:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 -- 172.21.15.157:0/3493872358 learned_addr learned my addr 172.21.15.157:0/3493872358 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651f1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 -- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 msgr2=0x556651f1cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651f1cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 -- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 msgr2=0x556651f21b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc37fe640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651f21b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651f21b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 -- 172.21.15.157:0/3493872358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac003530 con 0x556651eb7130 2023-11-06T03:08:15.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bc27fc640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010fd0 tx=0x7f4bb8001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05549931d503f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:15.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb805f6a0 con 0x556651eb7130 2023-11-06T03:08:15.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556651c85620 con 0x556651eb7130 2023-11-06T03:08:15.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4bb806ee40 con 0x556651eb7130 2023-11-06T03:08:15.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb8057410 con 0x556651eb7130 2023-11-06T03:08:15.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 10 client.6267 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:15.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x556651c5b090 con 0x556651eb7130 2023-11-06T03:08:15.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8079080 con 0x556651eb7130 2023-11-06T03:08:15.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4bb8053d50 con 0x556651eb7130 2023-11-06T03:08:15.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 10 client.6267 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:15.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 10 client.6267 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:15.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 20 client.6267 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:15.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb3dc0 0x556651f24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 4 client.6267 mds_command: new command op to 5868 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:08:15.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.549+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556651cb5320 con 0x556651eb3dc0 2023-11-06T03:08:15.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb3dc0 0x556651f24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4ba1ffb640 10 client.6267 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc2ffd640 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb3dc0 0x556651f24070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480cebf2f07addbf server_cookie=55b3bf89c0fa5a66 in_seq=0 out_seq=0 2023-11-06T03:08:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4ba1ffb640 1 -- 172.21.15.157:0/3493872358 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556651cb5320 con 0x556651eb3dc0 2023-11-06T03:08:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4ba1ffb640 10 client.6267 handle_command_reply: tid=0 2023-11-06T03:08:15.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 client.6267 shutdown 2023-11-06T03:08:15.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 msgr2=0x556651f1d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010fd0 tx=0x7f4bb8001a10 comp rx=0 tx=0).stop 2023-11-06T03:08:15.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 shutdown_connections 2023-11-06T03:08:15.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x556651eb3dc0 0x556651f24070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556651eb7a50 0x556651f21b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4ba1ffb640 0 client.6267 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556651eb7130 0x556651f1d1a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 --2- 172.21.15.157:0/3493872358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556651e6d260 0x556651f1cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 >> 172.21.15.157:0/3493872358 conn(0x556651e739f0 msgr2=0x556651e72140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 shutdown_connections 2023-11-06T03:08:15.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 1 -- 172.21.15.157:0/3493872358 wait complete. 2023-11-06T03:08:15.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bcc6d7000 20 client.6267 trim_cache size 0 max 16384 2023-11-06T03:08:15.586 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:15.586 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:08:15.586 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:08:15.586 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:15.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 msgr2=0x7f4bb0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 0x7f4bb0067f20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009a30 tx=0x7f4bb4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:15.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 msgr2=0x7f4bc40cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8010b10 tx=0x7f4bb8005700 comp rx=0 tx=0).stop 2023-11-06T03:08:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 shutdown_connections 2023-11-06T03:08:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bb0065a60 0x7f4bb0067f20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bc40b3ed0 0x7f4bc40cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40c91e0 0x7f4bc40cbf50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 --2- 172.21.15.157:0/959931010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bac004d10 0x7f4bc4155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 >> 172.21.15.157:0/959931010 conn(0x7f4bc40cd490 msgr2=0x7f4bc41e73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 shutdown_connections 2023-11-06T03:08:15.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.553+0000 7f4bc95af640 1 -- 172.21.15.157:0/959931010 wait complete. 2023-11-06T03:08:15.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:08:15.591 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-06T03:08:15.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.709+0000 7fb5ceb46640 1 Processor -- start 2023-11-06T03:08:15.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.709+0000 7fb5ceb46640 1 -- start start 2023-11-06T03:08:15.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5c8071cc0 0x7fb5c806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb5c80ce570 0x7fb5c806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80cc960 con 0x7fb5c80b4910 2023-11-06T03:08:15.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c806e170 con 0x7fb5c8071cc0 2023-11-06T03:08:15.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80cc680 con 0x7fb5c80ce570 2023-11-06T03:08:15.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5c8071cc0 0x7fb5c806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb5c8071cc0 0x7fb5c806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35534/0 (socket says 172.21.15.157:35534) 2023-11-06T03:08:15.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/944025208 learned_addr learned my addr 172.21.15.157:0/944025208 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/944025208 >> v2:172.21.15.195:3300/0 conn(0x7fb5c80ce570 msgr2=0x7fb5c806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/944025208 >> v2:172.21.15.195:3300/0 conn(0x7fb5c80ce570 0x7fb5c806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/944025208 >> v2:172.21.15.195:3300/0 conn(0x7fb5c80ce570 0x7fb5c806dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/944025208 >> v1:172.21.15.157:6789/0 conn(0x7fb5c80b4910 legacy=0x7fb5c806fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5c77fe640 1 -- 172.21.15.157:0/944025208 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564613025 0 0) 0x7fb5c80cc960 con 0x7fb5c80b4910 2023-11-06T03:08:15.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/944025208 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c806bde0 con 0x7fb5c8071cc0 2023-11-06T03:08:15.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/944025208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8071cc0 0x7fb5c806d6f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80b5120 tx=0x7fb5b801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ecb8f9052a50b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5c77fe640 1 -- 172.21.15.157:0/944025208 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80ac410 con 0x7fb5c8071cc0 2023-11-06T03:08:15.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5c77fe640 1 -- 172.21.15.157:0/944025208 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b80ac570 con 0x7fb5c8071cc0 2023-11-06T03:08:15.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5c77fe640 1 -- 172.21.15.157:0/944025208 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80ac900 con 0x7fb5c8071cc0 2023-11-06T03:08:15.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- 172.21.15.157:0/944025208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8071cc0 msgr2=0x7fb5c806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/944025208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8071cc0 0x7fb5c806d6f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80b5120 tx=0x7fb5b801c3b0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- 172.21.15.157:0/944025208 shutdown_connections 2023-11-06T03:08:15.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/944025208 >> v2:172.21.15.195:3300/0 conn(0x7fb5c80ce570 0x7fb5c806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/944025208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c8071cc0 0x7fb5c806d6f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- 172.21.15.157:0/944025208 >> 172.21.15.157:0/944025208 conn(0x7fb5c80cba30 msgr2=0x7fb5c80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- 172.21.15.157:0/944025208 shutdown_connections 2023-11-06T03:08:15.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.713+0000 7fb5ceb46640 1 -- 172.21.15.157:0/944025208 wait complete. 2023-11-06T03:08:15.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 Processor -- start 2023-11-06T03:08:15.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- start start 2023-11-06T03:08:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5c8071cc0 0x7fb5c80c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5c80b4910 0x7fb5c8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8159570 con 0x7fb5c80ce570 2023-11-06T03:08:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81588f0 con 0x7fb5c80b4910 2023-11-06T03:08:15.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8158bd0 con 0x7fb5c8071cc0 2023-11-06T03:08:15.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35540/0 (socket says 172.21.15.157:35540) 2023-11-06T03:08:15.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 -- 172.21.15.157:0/1970190251 learned_addr learned my addr 172.21.15.157:0/1970190251 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5c8071cc0 0x7fb5c80c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5c80b4910 0x7fb5c8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 -- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5c8071cc0 msgr2=0x7fb5c80c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5c8071cc0 0x7fb5c80c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 -- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5c80b4910 msgr2=0x7fb5c8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5c80b4910 0x7fb5c8157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 -- 172.21.15.157:0/1970190251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80094d0 con 0x7fb5c80ce570 2023-11-06T03:08:15.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5cd0e3640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000c450 tx=0x7fb5b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5c57fa640 1 -- 172.21.15.157:0/1970190251 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0005480 con 0x7fb5c80ce570 2023-11-06T03:08:15.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5c57fa640 1 -- 172.21.15.157:0/1970190251 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0003600 con 0x7fb5c80ce570 2023-11-06T03:08:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c8158eb0 con 0x7fb5c80ce570 2023-11-06T03:08:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5c57fa640 1 -- 172.21.15.157:0/1970190251 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00039b0 con 0x7fb5c80ce570 2023-11-06T03:08:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.717+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c80c3280 con 0x7fb5c80ce570 2023-11-06T03:08:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5c57fa640 1 -- 172.21.15.157:0/1970190251 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb5b0003b10 con 0x7fb5c80ce570 2023-11-06T03:08:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5c57fa640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 0x7fb5a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5c57fa640 1 -- 172.21.15.157:0/1970190251 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb5b00828e0 con 0x7fb5c80ce570 2023-11-06T03:08:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 0x7fb5a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 Processor -- start 2023-11-06T03:08:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 -- start start 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 0x7fb5a8067f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80ae000 tx=0x7fb5b80a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560537fdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560537fe03a0 0x560537fe0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537ea3d60 con 0x560538029e60 2023-11-06T03:08:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537e323b0 con 0x56053802a780 2023-11-06T03:08:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560537e2f980 con 0x560537fe03a0 2023-11-06T03:08:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35542/0 (socket says 172.21.15.157:35542) 2023-11-06T03:08:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/4053448122 learned_addr learned my addr 172.21.15.157:0/4053448122 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560537fdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560537fe03a0 msgr2=0x560537fe0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560537fe03a0 0x560537fe0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 msgr2=0x560537fdfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560537fdfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/4053448122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560537dcc110 con 0x560538029e60 2023-11-06T03:08:15.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560537fdfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:15.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8019fe0 tx=0x7fb5b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373b85ec6cbd77d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5be7fc640 1 -- 172.21.15.157:0/4053448122 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80acdb0 con 0x560538029e60 2023-11-06T03:08:15.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5be7fc640 1 -- 172.21.15.157:0/4053448122 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b80be040 con 0x560538029e60 2023-11-06T03:08:15.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.721+0000 7fb5be7fc640 1 -- 172.21.15.157:0/4053448122 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8003820 con 0x560538029e60 2023-11-06T03:08:15.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/4053448122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 msgr2=0x56053802a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8019fe0 tx=0x7fb5b8004740 comp rx=0 tx=0).stop 2023-11-06T03:08:15.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/4053448122 shutdown_connections 2023-11-06T03:08:15.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560537fe03a0 0x560537fe0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560537fdfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/4053448122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x56053802a240 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/4053448122 >> 172.21.15.157:0/4053448122 conn(0x560537fd7f40 msgr2=0x560537fda330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/4053448122 shutdown_connections 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/4053448122 wait complete. 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- start start 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 Processor -- start 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538031700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538017d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537e2b270 con 0x560537fe03a0 2023-11-06T03:08:15.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537dca5a0 con 0x56053802a780 2023-11-06T03:08:15.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560537de87c0 con 0x560538029e60 2023-11-06T03:08:15.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 10 client.0 _ll_register_callbacks cb 0x560537e1f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:15.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35554/0 (socket says 172.21.15.157:35554) 2023-11-06T03:08:15.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3709410272 learned_addr learned my addr 172.21.15.157:0/3709410272 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538031700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cd0e3640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538017d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 msgr2=0x560538031700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538031700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 msgr2=0x560538017d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538017d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3709410272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c806bde0 con 0x560537fe03a0 2023-11-06T03:08:15.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538031700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:15.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8004780 tx=0x7fb5b801abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c035db039ea76f29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:15.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8003c90 con 0x560537fe03a0 2023-11-06T03:08:15.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb5b8003df0 con 0x560537fe03a0 2023-11-06T03:08:15.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560537def350 con 0x560537fe03a0 2023-11-06T03:08:15.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5b80c4480 con 0x560537fe03a0 2023-11-06T03:08:15.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 10 client.6282 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:15.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x560537b27e20 con 0x560537fe03a0 2023-11-06T03:08:15.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560537def350 con 0x560537fe03a0 2023-11-06T03:08:15.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb5b80c45e0 con 0x560537fe03a0 2023-11-06T03:08:15.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 10 client.6282 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:15.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 10 client.6282 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 20 client.6282 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560538018ad0 0x560538018eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 4 client.6282 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560537e29bc0 con 0x560538018ad0 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560538018ad0 0x560538018eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5a77fe640 10 client.6282 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.725+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560538018ad0 0x560538018eb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8248391d9e1cec35 server_cookie=57e5380933f07001 in_seq=0 out_seq=0 2023-11-06T03:08:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3709410272 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560537e29bc0 con 0x560538018ad0 2023-11-06T03:08:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5a77fe640 10 client.6282 handle_command_reply: tid=0 2023-11-06T03:08:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 client.6282 shutdown 2023-11-06T03:08:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 msgr2=0x5605380311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8004780 tx=0x7fb5b801abf0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 shutdown_connections 2023-11-06T03:08:15.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560538018ad0 0x560538018eb0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538017d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5a77fe640 0 client.6282 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538031700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3709410272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380311c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 >> 172.21.15.157:0/3709410272 conn(0x560537fd7f40 msgr2=0x560537fe1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 shutdown_connections 2023-11-06T03:08:15.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3709410272 wait complete. 2023-11-06T03:08:15.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.729+0000 7fb5d1c6e000 20 client.6282 trim_cache size 0 max 16384 2023-11-06T03:08:15.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 Processor -- start 2023-11-06T03:08:15.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 -- start start 2023-11-06T03:08:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560537fe03a0 0x560537fe69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56053802a780 0x56053802ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537ae8aa0 con 0x560538029e60 2023-11-06T03:08:15.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537e33640 con 0x560537fe03a0 2023-11-06T03:08:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.737+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560537e28a80 con 0x56053802a780 2023-11-06T03:08:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35562/0 (socket says 172.21.15.157:35562) 2023-11-06T03:08:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 -- 172.21.15.157:0/894356721 learned_addr learned my addr 172.21.15.157:0/894356721 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 -- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56053802a780 msgr2=0x56053802ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56053802a780 0x56053802ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 -- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560537fe03a0 msgr2=0x560537fe69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560537fe03a0 0x560537fe69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 -- 172.21.15.157:0/894356721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560537df01a0 con 0x560538029e60 2023-11-06T03:08:15.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000f90 tx=0x7fb5c00075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185f8257eb4f1017 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5beffd640 1 -- 172.21.15.157:0/894356721 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0018270 con 0x560538029e60 2023-11-06T03:08:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5beffd640 1 -- 172.21.15.157:0/894356721 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0018040 con 0x560538029e60 2023-11-06T03:08:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5beffd640 1 -- 172.21.15.157:0/894356721 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004730 con 0x560538029e60 2023-11-06T03:08:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/894356721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 msgr2=0x560537fe6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000f90 tx=0x7fb5c00075d0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/894356721 shutdown_connections 2023-11-06T03:08:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56053802a780 0x56053802ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560538029e60 0x560537fe6f00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/894356721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560537fe03a0 0x560537fe69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/894356721 >> 172.21.15.157:0/894356721 conn(0x560537fd7f40 msgr2=0x5605380378a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/894356721 shutdown_connections 2023-11-06T03:08:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/894356721 wait complete. 2023-11-06T03:08:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- start start 2023-11-06T03:08:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 Processor -- start 2023-11-06T03:08:15.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538082cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538083200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560537fd33c0 con 0x560537fe03a0 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560538040580 con 0x56053802a780 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560537e28670 con 0x560538029e60 2023-11-06T03:08:15.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 10 client.0 _ll_register_callbacks cb 0x560537dd9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35578/0 (socket says 172.21.15.157:35578) 2023-11-06T03:08:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3233314205 learned_addr learned my addr 172.21.15.157:0/3233314205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:15.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 msgr2=0x560538082cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538082cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 msgr2=0x560538083200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538083200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 -- 172.21.15.157:0/3233314205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c0008040 con 0x560537fe03a0 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5cc8e2640 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb5b801ac30 tx=0x7fb5b8019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e16e961cc40fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80cd760 con 0x560537fe03a0 2023-11-06T03:08:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb5b80d39f0 con 0x560537fe03a0 2023-11-06T03:08:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560537ddac20 con 0x560537fe03a0 2023-11-06T03:08:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5b80acb70 con 0x560537fe03a0 2023-11-06T03:08:15.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 10 client.6288 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x560537dd64f0 con 0x560537fe03a0 2023-11-06T03:08:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560537ddac20 con 0x560537fe03a0 2023-11-06T03:08:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb5b80d3bd0 con 0x560537fe03a0 2023-11-06T03:08:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 10 client.6288 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:15.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 10 client.6288 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 20 client.6288 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56053802d290 0x56053802d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.741+0000 7fb5d1c6e000 4 client.6288 mds_command: new command op to 5868 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:08:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560537e242f0 con 0x56053802d290 2023-11-06T03:08:15.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56053802d290 0x56053802d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5a77fe640 10 client.6288 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5c7fff640 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56053802d290 0x56053802d6a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44370e3ae43358fa server_cookie=2596c62e84ffbecc in_seq=0 out_seq=0 2023-11-06T03:08:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5a77fe640 1 -- 172.21.15.157:0/3233314205 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x560537e242f0 con 0x56053802d290 2023-11-06T03:08:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5a77fe640 10 client.6288 handle_command_reply: tid=0 2023-11-06T03:08:15.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 client.6288 shutdown 2023-11-06T03:08:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 msgr2=0x5605380305f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb5b801ac30 tx=0x7fb5b8019150 comp rx=0 tx=0).stop 2023-11-06T03:08:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 shutdown_connections 2023-11-06T03:08:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x56053802d290 0x56053802d6a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56053802a780 0x560538083200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5a77fe640 0 client.6288 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560538029e60 0x560538082cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 --2- 172.21.15.157:0/3233314205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560537fe03a0 0x5605380305f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 >> 172.21.15.157:0/3233314205 conn(0x560537fd7f40 msgr2=0x560537fe69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 shutdown_connections 2023-11-06T03:08:15.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 1 -- 172.21.15.157:0/3233314205 wait complete. 2023-11-06T03:08:15.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5d1c6e000 20 client.6288 trim_cache size 0 max 16384 2023-11-06T03:08:15.770 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:08:15.770 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:15.771 INFO:teuthology.orchestra.run.smithi157.stdout: "damage_type": "backtrace", 2023-11-06T03:08:15.771 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 3230749136, 2023-11-06T03:08:15.771 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1099511630409, 2023-11-06T03:08:15.771 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2023-11-06T03:08:15.771 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:15.772 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:08:15.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 msgr2=0x7fb5a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 0x7fb5a8067f20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80ae000 tx=0x7fb5b80a6040 comp rx=0 tx=0).stop 2023-11-06T03:08:15.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 msgr2=0x7fb5c81582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000c450 tx=0x7fb5b000c910 comp rx=0 tx=0).stop 2023-11-06T03:08:15.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 shutdown_connections 2023-11-06T03:08:15.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb5a8065a60 0x7fb5a8067f20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80ce570 0x7fb5c81582c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb5c80b4910 0x7fb5c8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 --2- 172.21.15.157:0/1970190251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb5c8071cc0 0x7fb5c80c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 >> 172.21.15.157:0/1970190251 conn(0x7fb5c80cba30 msgr2=0x7fb5c815a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 shutdown_connections 2023-11-06T03:08:15.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.745+0000 7fb5ceb46640 1 -- 172.21.15.157:0/1970190251 wait complete. 2023-11-06T03:08:15.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:15.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.889+0000 7f54d17bb640 1 Processor -- start 2023-11-06T03:08:15.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.889+0000 7f54d17bb640 1 -- start start 2023-11-06T03:08:15.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54d17bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54cc15e5e0 0x7f54cc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54d17bb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54cc15ea90 0x7f54cc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:15.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54d17bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc15f580 con 0x7f54cc1551e0 2023-11-06T03:08:15.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54d17bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0704b0 con 0x7f54cc15e5e0 2023-11-06T03:08:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54d17bb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc15efc0 con 0x7f54cc15ea90 2023-11-06T03:08:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54cc15e5e0 0x7f54cc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54caffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f54cc1551e0 0x7f54cc06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59410/0 (socket says 172.21.15.157:59410) 2023-11-06T03:08:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54caffd640 1 -- 172.21.15.157:0/1738623112 learned_addr learned my addr 172.21.15.157:0/1738623112 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54cb7fe640 1 --2- 172.21.15.157:0/1738623112 >> v2:172.21.15.195:3300/0 conn(0x7f54cc15ea90 0x7f54cc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 -- 172.21.15.157:0/1738623112 >> v2:172.21.15.195:3300/0 conn(0x7f54cc15ea90 msgr2=0x7f54cc0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 --2- 172.21.15.157:0/1738623112 >> v2:172.21.15.195:3300/0 conn(0x7f54cc15ea90 0x7f54cc0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 -- 172.21.15.157:0/1738623112 >> v1:172.21.15.157:6789/0 conn(0x7f54cc1551e0 legacy=0x7f54cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 -- 172.21.15.157:0/1738623112 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc15f2a0 con 0x7f54cc15e5e0 2023-11-06T03:08:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54ca7fc640 1 --2- 172.21.15.157:0/1738623112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc0b3410 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0cc2f0 tx=0x7f54c001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9fdd66d28d4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54c9ffb640 1 -- 172.21.15.157:0/1738623112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00ac600 con 0x7f54cc15e5e0 2023-11-06T03:08:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54c9ffb640 1 -- 172.21.15.157:0/1738623112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00a8050 con 0x7f54cc15e5e0 2023-11-06T03:08:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.893+0000 7f54c9ffb640 1 -- 172.21.15.157:0/1738623112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00ac980 con 0x7f54cc15e5e0 2023-11-06T03:08:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/1738623112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 msgr2=0x7f54cc0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:15.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- 172.21.15.157:0/1738623112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc0b3410 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0cc2f0 tx=0x7f54c001c3a0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/1738623112 shutdown_connections 2023-11-06T03:08:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- 172.21.15.157:0/1738623112 >> v2:172.21.15.195:3300/0 conn(0x7f54cc15ea90 0x7f54cc0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- 172.21.15.157:0/1738623112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc0b3410 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/1738623112 >> 172.21.15.157:0/1738623112 conn(0x7f54cc065930 msgr2=0x7f54cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/1738623112 shutdown_connections 2023-11-06T03:08:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/1738623112 wait complete. 2023-11-06T03:08:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 Processor -- start 2023-11-06T03:08:15.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- start start 2023-11-06T03:08:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc1551e0 0x7f54cc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15ea90 0x7f54cc14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc154a60 con 0x7f54cc15e5e0 2023-11-06T03:08:16.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc1592b0 con 0x7f54cc1551e0 2023-11-06T03:08:16.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc153860 con 0x7f54cc15ea90 2023-11-06T03:08:16.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35596/0 (socket says 172.21.15.157:35596) 2023-11-06T03:08:16.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 -- 172.21.15.157:0/969965486 learned_addr learned my addr 172.21.15.157:0/969965486 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54cb7fe640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15ea90 0x7f54cc14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54caffd640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc1551e0 0x7f54cc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 -- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15ea90 msgr2=0x7f54cc14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15ea90 0x7f54cc14c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 -- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc1551e0 msgr2=0x7f54cc159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc1551e0 0x7f54cc159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 -- 172.21.15.157:0/969965486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c0009420 con 0x7f54cc15e5e0 2023-11-06T03:08:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54ca7fc640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f54c0018fb0 tx=0x7f54c001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00b7650 con 0x7f54cc15e5e0 2023-11-06T03:08:16.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c00b77b0 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc159550 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c00b7b20 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.897+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc153b40 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc072750 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f54c00b3070 con 0x7f54cc15e5e0 2023-11-06T03:08:16.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54b37fe640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f54c011db60 con 0x7f54cc15e5e0 2023-11-06T03:08:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54caffd640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.901+0000 7f54caffd640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 0x7f54b8067f20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005890 tx=0x7f54bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:15.905+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f54c00eb2e0 con 0x7f54cc15e5e0 2023-11-06T03:08:16.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54cc159a00 con 0x7f54cc15e5e0 2023-11-06T03:08:16.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54b37fe640 1 -- 172.21.15.157:0/969965486 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f54c00ee900 con 0x7f54cc15e5e0 2023-11-06T03:08:16.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:16.081 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":37,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:08.345625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":10,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:16.081 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 37 2023-11-06T03:08:16.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 msgr2=0x7f54b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 0x7f54b8067f20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005890 tx=0x7f54bc009040 comp rx=0 tx=0).stop 2023-11-06T03:08:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 msgr2=0x7f54cc154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f54c0018fb0 tx=0x7f54c001abe0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 shutdown_connections 2023-11-06T03:08:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f54b8065a60 0x7f54b8067f20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54cc15ea90 0x7f54cc14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54cc15e5e0 0x7f54cc154520 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 --2- 172.21.15.157:0/969965486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54cc1551e0 0x7f54cc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 >> 172.21.15.157:0/969965486 conn(0x7f54cc065930 msgr2=0x7f54cc0aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 shutdown_connections 2023-11-06T03:08:16.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.081+0000 7f54d17bb640 1 -- 172.21.15.157:0/969965486 wait complete. 2023-11-06T03:08:16.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3230749136 2023-11-06T03:08:16.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 Processor -- start 2023-11-06T03:08:16.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 -- start start 2023-11-06T03:08:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 0x7f97d80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f97d815ea90 0x7f97d80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d815f580 con 0x7f97d81551e0 2023-11-06T03:08:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d80704b0 con 0x7f97d815ea90 2023-11-06T03:08:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97df43f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d815efc0 con 0x7f97d815e5e0 2023-11-06T03:08:16.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f97d815ea90 0x7f97d80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f97d815ea90 0x7f97d80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35608/0 (socket says 172.21.15.157:35608) 2023-11-06T03:08:16.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 -- 172.21.15.157:0/957934 learned_addr learned my addr 172.21.15.157:0/957934 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 -- 172.21.15.157:0/957934 >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 msgr2=0x7f97d80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dc9da640 1 --2- 172.21.15.157:0/957934 >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 0x7f97d80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/957934 >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 0x7f97d80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 -- 172.21.15.157:0/957934 >> v1:172.21.15.157:6789/0 conn(0x7f97d81551e0 legacy=0x7f97d806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97d7fff640 1 -- 172.21.15.157:0/957934 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303277142 0 0) 0x7f97d815f580 con 0x7f97d81551e0 2023-11-06T03:08:16.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dd9dc640 1 -- 172.21.15.157:0/957934 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d815f2a0 con 0x7f97d815ea90 2023-11-06T03:08:16.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.225+0000 7f97dc9da640 1 --2- 172.21.15.157:0/957934 >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 0x7f97d80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:16.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/957934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815ea90 0x7f97d80b3950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f97d8066880 tx=0x7f97c801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4328d3774e2fe92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97d7fff640 1 -- 172.21.15.157:0/957934 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c80ab640 con 0x7f97d815ea90 2023-11-06T03:08:16.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97d7fff640 1 -- 172.21.15.157:0/957934 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c80a7070 con 0x7f97d815ea90 2023-11-06T03:08:16.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97d7fff640 1 -- 172.21.15.157:0/957934 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c80ab9c0 con 0x7f97d815ea90 2023-11-06T03:08:16.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- 172.21.15.157:0/957934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815ea90 msgr2=0x7f97d80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- 172.21.15.157:0/957934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815ea90 0x7f97d80b3950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f97d8066880 tx=0x7f97c801c440 comp rx=0 tx=0).stop 2023-11-06T03:08:16.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- 172.21.15.157:0/957934 shutdown_connections 2023-11-06T03:08:16.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- 172.21.15.157:0/957934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815ea90 0x7f97d80b3950 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- 172.21.15.157:0/957934 >> v2:172.21.15.195:3300/0 conn(0x7f97d815e5e0 0x7f97d80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- 172.21.15.157:0/957934 >> 172.21.15.157:0/957934 conn(0x7f97d8065930 msgr2=0x7f97d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- 172.21.15.157:0/957934 shutdown_connections 2023-11-06T03:08:16.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- 172.21.15.157:0/957934 wait complete. 2023-11-06T03:08:16.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 Processor -- start 2023-11-06T03:08:16.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- start start 2023-11-06T03:08:16.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97d81551e0 0x7f97d814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97d815ea90 0x7f97d80ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d81f4030 con 0x7f97d815e5e0 2023-11-06T03:08:16.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8155c00 con 0x7f97d81551e0 2023-11-06T03:08:16.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97df43f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8155ee0 con 0x7f97d815ea90 2023-11-06T03:08:16.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35624/0 (socket says 172.21.15.157:35624) 2023-11-06T03:08:16.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 -- 172.21.15.157:0/966876739 learned_addr learned my addr 172.21.15.157:0/966876739 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97d815ea90 0x7f97d80ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dd1db640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97d81551e0 0x7f97d814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 -- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97d815ea90 msgr2=0x7f97d80ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97d815ea90 0x7f97d80ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 -- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97d81551e0 msgr2=0x7f97d814b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97d81551e0 0x7f97d814b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 -- 172.21.15.157:0/966876739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80094d0 con 0x7f97d815e5e0 2023-11-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97dc9da640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c7c0 tx=0x7f97cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.229+0000 7f97d5ffb640 1 -- 172.21.15.157:0/966876739 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc007a30 con 0x7f97d815e5e0 2023-11-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97d5ffb640 1 -- 172.21.15.157:0/966876739 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc007b90 con 0x7f97d815e5e0 2023-11-06T03:08:16.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d81561c0 con 0x7f97d815e5e0 2023-11-06T03:08:16.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97d5ffb640 1 -- 172.21.15.157:0/966876739 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00ce70 con 0x7f97d815e5e0 2023-11-06T03:08:16.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d8156f00 con 0x7f97d815e5e0 2023-11-06T03:08:16.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97d5ffb640 1 -- 172.21.15.157:0/966876739 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f97cc0113f0 con 0x7f97d815e5e0 2023-11-06T03:08:16.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97d5ffb640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 0x7f97c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97d5ffb640 1 -- 172.21.15.157:0/966876739 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f97cc083860 con 0x7f97d815e5e0 2023-11-06T03:08:16.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97dd1db640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 0x7f97c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 Processor -- start 2023-11-06T03:08:16.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 -- start start 2023-11-06T03:08:16.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97dd1db640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 0x7f97c4067f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f97d00012f0 tx=0x7f97d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f7384d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f7384d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73839aec0 con 0x55f738521910 2023-11-06T03:08:16.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7383283b0 con 0x55f738522230 2023-11-06T03:08:16.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f738325980 con 0x55f7384d7be0 2023-11-06T03:08:16.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97dd1db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97dd1db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35638/0 (socket says 172.21.15.157:35638) 2023-11-06T03:08:16.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.233+0000 7f97dd1db640 1 -- 172.21.15.157:0/1566433932 learned_addr learned my addr 172.21.15.157:0/1566433932 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f7384d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f7384d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 -- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 msgr2=0x55f7384d7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f7384d7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 -- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 msgr2=0x55f7384d76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f7384d76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 -- 172.21.15.157:0/1566433932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7382c2040 con 0x55f738521910 2023-11-06T03:08:16.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f97d0003d50 tx=0x7f97d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d044edb7799503aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97beffd640 1 -- 172.21.15.157:0/1566433932 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0005850 con 0x55f738521910 2023-11-06T03:08:16.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97beffd640 1 -- 172.21.15.157:0/1566433932 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0003e40 con 0x55f738521910 2023-11-06T03:08:16.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97beffd640 1 -- 172.21.15.157:0/1566433932 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0020670 con 0x55f738521910 2023-11-06T03:08:16.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/1566433932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 msgr2=0x55f738521cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f97d0003d50 tx=0x7f97d0010000 comp rx=0 tx=0).stop 2023-11-06T03:08:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/1566433932 shutdown_connections 2023-11-06T03:08:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f7384d7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f7384d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- 172.21.15.157:0/1566433932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738521cf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/1566433932 >> 172.21.15.157:0/1566433932 conn(0x55f7384cf780 msgr2=0x55f7384d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/1566433932 shutdown_connections 2023-11-06T03:08:16.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/1566433932 wait complete. 2023-11-06T03:08:16.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- start start 2023-11-06T03:08:16.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 Processor -- start 2023-11-06T03:08:16.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7384d7be0 0x55f73850ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f738522230 0x55f73850f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f738321270 con 0x55f738521910 2023-11-06T03:08:16.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7382c06d0 con 0x55f7384d7be0 2023-11-06T03:08:16.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7382c0370 con 0x55f738522230 2023-11-06T03:08:16.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 10 client.0 _ll_register_callbacks cb 0x55f7384bed90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:16.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35646/0 (socket says 172.21.15.157:35646) 2023-11-06T03:08:16.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 -- 172.21.15.157:0/3332922415 learned_addr learned my addr 172.21.15.157:0/3332922415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7384d7be0 0x55f73850ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f738522230 0x55f73850f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 -- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f738522230 msgr2=0x55f73850f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f738522230 0x55f73850f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 -- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7384d7be0 msgr2=0x55f73850ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7384d7be0 0x55f73850ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 -- 172.21.15.157:0/3332922415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d0003ac0 con 0x55f738521910 2023-11-06T03:08:16.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f97cc05c750 tx=0x7f97cc054d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba17b4956c457fca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:16.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc054390 con 0x55f738521910 2023-11-06T03:08:16.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f97cc05c7f0 con 0x55f738521910 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7382c6b90 con 0x55f738521910 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97cc05c9b0 con 0x55f738521910 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 10 client.6312 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55f73801db30 con 0x55f738521910 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc07f070 con 0x55f738521910 2023-11-06T03:08:16.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f97cc021070 con 0x55f738521910 2023-11-06T03:08:16.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 10 client.6312 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:16.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 10 client.6312 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:16.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 20 client.6312 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:16.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f7385309a0 0x55f738532d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 4 client.6312 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:16.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f73831fbc0 con 0x55f7385309a0 2023-11-06T03:08:16.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.237+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f7385309a0 0x55f738532d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97b7fff640 10 client.6312 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:16.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f7385309a0 0x55f738532d90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594638ddbd676c58 server_cookie=ff5acb7c46635f8e in_seq=0 out_seq=0 2023-11-06T03:08:16.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97b7fff640 1 -- 172.21.15.157:0/3332922415 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f73831fbc0 con 0x55f7385309a0 2023-11-06T03:08:16.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97b7fff640 10 client.6312 handle_command_reply: tid=0 2023-11-06T03:08:16.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 client.6312 shutdown 2023-11-06T03:08:16.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 msgr2=0x55f73850f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f97cc05c750 tx=0x7f97cc054d60 comp rx=0 tx=0).stop 2023-11-06T03:08:16.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 shutdown_connections 2023-11-06T03:08:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f7385309a0 0x55f738532d90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f738522230 0x55f73850f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f73850f4b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97b7fff640 0 client.6312 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 --2- 172.21.15.157:0/3332922415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f7384d7be0 0x55f73850ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 >> 172.21.15.157:0/3332922415 conn(0x55f7384cf780 msgr2=0x55f7384d8850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 shutdown_connections 2023-11-06T03:08:16.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 1 -- 172.21.15.157:0/3332922415 wait complete. 2023-11-06T03:08:16.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.241+0000 7f97e2567000 20 client.6312 trim_cache size 0 max 16384 2023-11-06T03:08:16.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.249+0000 7f97e2567000 1 Processor -- start 2023-11-06T03:08:16.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.249+0000 7f97e2567000 1 -- start start 2023-11-06T03:08:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f73850d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 0x55f73850d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f737fdebe0 con 0x55f738522230 2023-11-06T03:08:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7384cf640 con 0x55f738521910 2023-11-06T03:08:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7385375a0 con 0x55f7384d7be0 2023-11-06T03:08:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35650/0 (socket says 172.21.15.157:35650) 2023-11-06T03:08:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 -- 172.21.15.157:0/1939270974 learned_addr learned my addr 172.21.15.157:0/1939270974 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 0x55f73850d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd1db640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f73850d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 -- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 msgr2=0x55f73850d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f73850d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 -- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 msgr2=0x55f73850d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 0x55f73850d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 -- 172.21.15.157:0/1939270974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d81f3e00 con 0x55f738522230 2023-11-06T03:08:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 0x55f73850d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f97c8019050 tx=0x7f97c80a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ead1feef562e934 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97bf7fe640 1 -- 172.21.15.157:0/1939270974 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c80ab680 con 0x55f738522230 2023-11-06T03:08:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97bf7fe640 1 -- 172.21.15.157:0/1939270974 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c80ab7e0 con 0x55f738522230 2023-11-06T03:08:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97bf7fe640 1 -- 172.21.15.157:0/1939270974 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c80abb90 con 0x55f738522230 2023-11-06T03:08:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- 172.21.15.157:0/1939270974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 msgr2=0x55f7384de260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f97c8019050 tx=0x7f97c80a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- 172.21.15.157:0/1939270974 shutdown_connections 2023-11-06T03:08:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738522230 0x55f7384de260 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738521910 0x55f73850d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- 172.21.15.157:0/1939270974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f73850d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- 172.21.15.157:0/1939270974 >> 172.21.15.157:0/1939270974 conn(0x55f7384cf780 msgr2=0x55f73850cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- 172.21.15.157:0/1939270974 shutdown_connections 2023-11-06T03:08:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- 172.21.15.157:0/1939270974 wait complete. 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- start start 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 Processor -- start 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f738531dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f738532850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73831e670 con 0x55f738521910 2023-11-06T03:08:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7385209e0 con 0x55f738522230 2023-11-06T03:08:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f73831ea80 con 0x55f7384d7be0 2023-11-06T03:08:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97e2567000 10 client.0 _ll_register_callbacks cb 0x55f7382e9250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35654/0 (socket says 172.21.15.157:35654) 2023-11-06T03:08:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 -- 172.21.15.157:0/3862605978 learned_addr learned my addr 172.21.15.157:0/3862605978 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd9dc640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f738532850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 -- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 msgr2=0x55f738531dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f738531dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f738531dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 -- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 msgr2=0x55f738532850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f738532850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.253+0000 7f97dc9da640 1 -- 172.21.15.157:0/3862605978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d815f2a0 con 0x55f738521910 2023-11-06T03:08:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97dc9da640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f97cc05cbd0 tx=0x7f97cc081a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5583001240e8db29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc06c1c0 con 0x55f738521910 2023-11-06T03:08:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f97cc054190 con 0x55f738521910 2023-11-06T03:08:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7382e84c0 con 0x55f738521910 2023-11-06T03:08:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97cc054350 con 0x55f738521910 2023-11-06T03:08:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 10 client.6318 fetch_fsmap learned FSMap version 37 2023-11-06T03:08:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55f7382e61a0 con 0x55f738521910 2023-11-06T03:08:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7382e84c0 con 0x55f738521910 2023-11-06T03:08:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 37) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f97cc021070 con 0x55f738521910 2023-11-06T03:08:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 10 client.6318 fetch_fsmap finished waiting for FSMap version 37 2023-11-06T03:08:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 10 client.6318 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:16.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 20 client.6318 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:16.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f738532d90 0x55f738533170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 4 client.6318 mds_command: new command op to 5868 tid=0[{"prefix": "damage rm", "damage_id": 3230749136}] 2023-11-06T03:08:16.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3230749136}) v1 -- 0x55f73831a2f0 con 0x55f738532d90 2023-11-06T03:08:16.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f738532d90 0x55f738533170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 10 client.6318 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97dd1db640 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f738532d90 0x55f738533170 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6517b7b05bdeab0 server_cookie=6f925e73a6a62293 in_seq=0 out_seq=0 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 1 -- 172.21.15.157:0/3862605978 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55f73831a2f0 con 0x55f738532d90 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 10 client.6318 handle_command_reply: tid=0 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 client.6318 shutdown 2023-11-06T03:08:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 msgr2=0x55f738532310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f97cc05cbd0 tx=0x7f97cc081a20 comp rx=0 tx=0).stop 2023-11-06T03:08:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 shutdown_connections 2023-11-06T03:08:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55f738532d90 0x55f738533170 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f738522230 0x55f738532850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f738521910 0x55f738532310 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 --2- 172.21.15.157:0/3862605978 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f7384d7be0 0x55f738531dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97b7fff640 0 client.6318 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 >> 172.21.15.157:0/3862605978 conn(0x55f7384cf780 msgr2=0x55f73850cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 shutdown_connections 2023-11-06T03:08:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 1 -- 172.21.15.157:0/3862605978 wait complete. 2023-11-06T03:08:16.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97e2567000 20 client.6318 trim_cache size 0 max 16384 2023-11-06T03:08:16.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 msgr2=0x7f97c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 0x7f97c4067f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f97d00012f0 tx=0x7f97d0009040 comp rx=0 tx=0).stop 2023-11-06T03:08:16.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 msgr2=0x7f97d81569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c7c0 tx=0x7f97cc00cc80 comp rx=0 tx=0).stop 2023-11-06T03:08:16.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 shutdown_connections 2023-11-06T03:08:16.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f97c4065a60 0x7f97c4067f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97d815ea90 0x7f97d80ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97d815e5e0 0x7f97d81569c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 --2- 172.21.15.157:0/966876739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97d81551e0 0x7f97d814b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 >> 172.21.15.157:0/966876739 conn(0x7f97d8065930 msgr2=0x7f97d814e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 shutdown_connections 2023-11-06T03:08:16.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.257+0000 7f97df43f640 1 -- 172.21.15.157:0/966876739 wait complete. 2023-11-06T03:08:16.293 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-06T03:08:16.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:16.385 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:08:16.385+0000 7f09efb48640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-06T03:08:16.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.409+0000 7f2e71c8b640 1 Processor -- start 2023-11-06T03:08:16.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.409+0000 7f2e71c8b640 1 -- start start 2023-11-06T03:08:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 0x7f2e6c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c0adf40 0x7f2e6c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c06b950 con 0x7f2e6c0adf40 2023-11-06T03:08:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c06c490 con 0x7f2e6c156a70 2023-11-06T03:08:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e71c8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c06bb60 con 0x7f2e6c0adae0 2023-11-06T03:08:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c0adf40 0x7f2e6c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c0adf40 0x7f2e6c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35660/0 (socket says 172.21.15.157:35660) 2023-11-06T03:08:16.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 -- 172.21.15.157:0/2396727862 learned_addr learned my addr 172.21.15.157:0/2396727862 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6b7fe640 1 --2- 172.21.15.157:0/2396727862 >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 0x7f2e6c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6affd640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 -- 172.21.15.157:0/2396727862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 msgr2=0x7f2e6c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 -- 172.21.15.157:0/2396727862 >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 msgr2=0x7f2e6c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/2396727862 >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 0x7f2e6c06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 -- 172.21.15.157:0/2396727862 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c06be40 con 0x7f2e6c0adf40 2023-11-06T03:08:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6b7fe640 1 --2- 172.21.15.157:0/2396727862 >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 0x7f2e6c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:16.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6affd640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c0b2160 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0ce880 tx=0x7f2e6401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f371f6e8304de2e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6a7fc640 1 -- 172.21.15.157:0/2396727862 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640a7bd0 con 0x7f2e6c0adf40 2023-11-06T03:08:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6a7fc640 1 -- 172.21.15.157:0/2396727862 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e640ab070 con 0x7f2e6c0adf40 2023-11-06T03:08:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.413+0000 7f2e6a7fc640 1 -- 172.21.15.157:0/2396727862 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640b1400 con 0x7f2e6c0adf40 2023-11-06T03:08:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/2396727862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0adf40 msgr2=0x7f2e6c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c0b2160 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0ce880 tx=0x7f2e6401eba0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/2396727862 shutdown_connections 2023-11-06T03:08:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c0b2160 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/2396727862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/2396727862 >> v2:172.21.15.195:3300/0 conn(0x7f2e6c156a70 0x7f2e6c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/2396727862 >> 172.21.15.157:0/2396727862 conn(0x7f2e6c065930 msgr2=0x7f2e6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/2396727862 shutdown_connections 2023-11-06T03:08:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/2396727862 wait complete. 2023-11-06T03:08:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 Processor -- start 2023-11-06T03:08:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- start start 2023-11-06T03:08:16.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c14c950 con 0x7f2e6c156a70 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c163600 con 0x7f2e6c0adf40 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c1500a0 con 0x7f2e6c0adae0 2023-11-06T03:08:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35670/0 (socket says 172.21.15.157:35670) 2023-11-06T03:08:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 -- 172.21.15.157:0/1866404868 learned_addr learned my addr 172.21.15.157:0/1866404868 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6b7fe640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6affd640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 -- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 msgr2=0x7f2e6c14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 -- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 msgr2=0x7f2e6c14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 -- 172.21.15.157:0/1866404868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6400bd40 con 0x7f2e6c156a70 2023-11-06T03:08:16.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6affd640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e6bfff640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2e6401d250 tx=0x7f2e6401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640c2070 con 0x7f2e6c156a70 2023-11-06T03:08:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e640b1690 con 0x7f2e6c156a70 2023-11-06T03:08:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640b1a20 con 0x7f2e6c156a70 2023-11-06T03:08:16.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c150380 con 0x7f2e6c156a70 2023-11-06T03:08:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.417+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c150940 con 0x7f2e6c156a70 2023-11-06T03:08:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c072750 con 0x7f2e6c156a70 2023-11-06T03:08:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e640b1b80 con 0x7f2e6c156a70 2023-11-06T03:08:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e53fff640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 0x7f2e58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:16.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2e6411fd10 con 0x7f2e6c156a70 2023-11-06T03:08:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e6b7fe640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 0x7f2e58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.421+0000 7f2e6b7fe640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 0x7f2e58067f20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006fd0 tx=0x7f2e5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.425+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e640ed530 con 0x7f2e6c156a70 2023-11-06T03:08:16.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e6c15a750 con 0x7f2e6c156a70 2023-11-06T03:08:16.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e53fff640 1 -- 172.21.15.157:0/1866404868 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f2e640f0b50 con 0x7f2e6c156a70 2023-11-06T03:08:16.599 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:16.600 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:16.389043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:16.600 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-11-06T03:08:16.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 msgr2=0x7f2e58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 0x7f2e58067f20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006fd0 tx=0x7f2e5c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:16.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 msgr2=0x7f2e6c1630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:16.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2e6401d250 tx=0x7f2e6401b940 comp rx=0 tx=0).stop 2023-11-06T03:08:16.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 shutdown_connections 2023-11-06T03:08:16.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e58065a60 0x7f2e58067f20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c156a70 0x7f2e6c1630c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c0adf40 0x7f2e6c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 --2- 172.21.15.157:0/1866404868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0adae0 0x7f2e6c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:16.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 >> 172.21.15.157:0/1866404868 conn(0x7f2e6c065930 msgr2=0x7f2e6c15c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:16.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 shutdown_connections 2023-11-06T03:08:16.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:16.601+0000 7f2e71c8b640 1 -- 172.21.15.157:0/1866404868 wait complete. 2023-11-06T03:08:16.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2023-11-06T03:08:16.735 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:16.736 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:16.736 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:16.750 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:16.750 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:16.751 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2023-11-06T03:08:16.751 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2023-11-06T03:08:16.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:16.752 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2023-11-06T03:08:16.759 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:08:16.759 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:08:16.760 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:08:16.760 DEBUG:teuthology.orchestra.run.smithi157:> import json 2023-11-06T03:08:16.760 DEBUG:teuthology.orchestra.run.smithi157:> import sys 2023-11-06T03:08:16.761 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:16.761 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:08:16.761 DEBUG:teuthology.orchestra.run.smithi157:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2023-11-06T03:08:16.762 DEBUG:teuthology.orchestra.run.smithi157:> except OSError as e: 2023-11-06T03:08:16.762 DEBUG:teuthology.orchestra.run.smithi157:> sys.exit(e.errno) 2023-11-06T03:08:16.762 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:16.763 DEBUG:teuthology.orchestra.run.smithi157:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-06T03:08:16.763 DEBUG:teuthology.orchestra.run.smithi157:> print(json.dumps( 2023-11-06T03:08:16.763 DEBUG:teuthology.orchestra.run.smithi157:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-06T03:08:16.764 DEBUG:teuthology.orchestra.run.smithi157:> indent=2)) 2023-11-06T03:08:16.764 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:08:16.865 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:16.865 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mode": 16893, 2023-11-06T03:08:16.865 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ino": 1099511627777, 2023-11-06T03:08:16.866 INFO:teuthology.orchestra.run.smithi157.stdout: "st_dev": 45, 2023-11-06T03:08:16.866 INFO:teuthology.orchestra.run.smithi157.stdout: "st_nlink": 16, 2023-11-06T03:08:16.866 INFO:teuthology.orchestra.run.smithi157.stdout: "st_uid": 1000, 2023-11-06T03:08:16.866 INFO:teuthology.orchestra.run.smithi157.stdout: "st_gid": 1271, 2023-11-06T03:08:16.866 INFO:teuthology.orchestra.run.smithi157.stdout: "st_size": 9167303, 2023-11-06T03:08:16.867 INFO:teuthology.orchestra.run.smithi157.stdout: "st_atime": 1699240072.9703205, 2023-11-06T03:08:16.867 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mtime": 1699240088.593571, 2023-11-06T03:08:16.867 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ctime": 1699240088.593571 2023-11-06T03:08:16.867 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:17.050 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2023-11-06T03:08:17.051 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2023-11-06T03:08:17.052 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:17.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 Processor -- start 2023-11-06T03:08:17.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 -- start start 2023-11-06T03:08:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b815f580 con 0x7f94b815ea60 2023-11-06T03:08:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b815efc0 con 0x7f94b81551e0 2023-11-06T03:08:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94c05f0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b815f2a0 con 0x7f94b815e5e0 2023-11-06T03:08:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94be38c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94bdb8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94beb8d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f94b815ea60 0x7f94b815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:50358/0 (socket says 172.21.15.157:50358) 2023-11-06T03:08:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94be38c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:60822/0 (socket says 172.21.15.157:60822) 2023-11-06T03:08:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94beb8d640 1 -- 172.21.15.157:0/1969775954 learned_addr learned my addr 172.21.15.157:0/1969775954 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94bdb8b640 1 -- 172.21.15.157:0/1969775954 >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 msgr2=0x7f94b806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94bdb8b640 1 --2- 172.21.15.157:0/1969775954 >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94bdb8b640 1 -- 172.21.15.157:0/1969775954 >> v1:172.21.15.195:6789/0 conn(0x7f94b815ea60 legacy=0x7f94b815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94bdb8b640 1 -- 172.21.15.157:0/1969775954 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b806f5a0 con 0x7f94b815e5e0 2023-11-06T03:08:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.225+0000 7f94be38c640 1 --2- 172.21.15.157:0/1969775954 >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:08:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94bdb8b640 1 --2- 172.21.15.157:0/1969775954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b80b3410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f94a80097c0 tx=0x7f94a801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7d8ee6e86ef461 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94bd38a640 1 -- 172.21.15.157:0/1969775954 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ab600 con 0x7f94b815e5e0 2023-11-06T03:08:17.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94bd38a640 1 -- 172.21.15.157:0/1969775954 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a80a7050 con 0x7f94b815e5e0 2023-11-06T03:08:17.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94bd38a640 1 -- 172.21.15.157:0/1969775954 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ab980 con 0x7f94b815e5e0 2023-11-06T03:08:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- 172.21.15.157:0/1969775954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 msgr2=0x7f94b80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- 172.21.15.157:0/1969775954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b80b3410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f94a80097c0 tx=0x7f94a801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- 172.21.15.157:0/1969775954 shutdown_connections 2023-11-06T03:08:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- 172.21.15.157:0/1969775954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b80b3410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- 172.21.15.157:0/1969775954 >> v2:172.21.15.195:3300/0 conn(0x7f94b81551e0 0x7f94b806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- 172.21.15.157:0/1969775954 >> 172.21.15.157:0/1969775954 conn(0x7f94b8065930 msgr2=0x7f94b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- 172.21.15.157:0/1969775954 shutdown_connections 2023-11-06T03:08:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- 172.21.15.157:0/1969775954 wait complete. 2023-11-06T03:08:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 Processor -- start 2023-11-06T03:08:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 -- start start 2023-11-06T03:08:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b81551e0 0x7f94b806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b8141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.229+0000 7f94c05f0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b81426a0 con 0x7f94b815ea60 2023-11-06T03:08:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8152ed0 con 0x7f94b81551e0 2023-11-06T03:08:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b806d510 con 0x7f94b815e5e0 2023-11-06T03:08:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35678/0 (socket says 172.21.15.157:35678) 2023-11-06T03:08:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 -- 172.21.15.157:0/3856288874 learned_addr learned my addr 172.21.15.157:0/3856288874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94be38c640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b81551e0 0x7f94b806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94bdb8b640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b8141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 -- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 msgr2=0x7f94b8141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b8141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 -- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b81551e0 msgr2=0x7f94b806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b81551e0 0x7f94b806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 -- 172.21.15.157:0/3856288874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4009420 con 0x7f94b815ea60 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94bdb8b640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b8141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94beb8d640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f94b000caa0 tx=0x7f94b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b000ac00 con 0x7f94b815ea60 2023-11-06T03:08:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b000ad60 con 0x7f94b815ea60 2023-11-06T03:08:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a8009420 con 0x7f94b815ea60 2023-11-06T03:08:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0012600 con 0x7f94b815ea60 2023-11-06T03:08:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b8153100 con 0x7f94b815ea60 2023-11-06T03:08:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b8072750 con 0x7f94b815ea60 2023-11-06T03:08:17.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f94b0007070 con 0x7f94b815ea60 2023-11-06T03:08:17.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 0x7f949c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f94b0083090 con 0x7f94b815ea60 2023-11-06T03:08:17.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.233+0000 7f94be38c640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 0x7f949c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.237+0000 7f94be38c640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 0x7f949c067f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f94b8151460 tx=0x7f94b4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.237+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f94b0050960 con 0x7f94b815ea60 2023-11-06T03:08:17.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b8071730 con 0x7f94b815ea60 2023-11-06T03:08:17.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94aeffd640 1 -- 172.21.15.157:0/3856288874 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f94b0017090 con 0x7f94b815ea60 2023-11-06T03:08:17.424 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:17.424 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:16.389043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:17.425 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-11-06T03:08:17.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 msgr2=0x7f949c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 0x7f949c067f50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f94b8151460 tx=0x7f94b4009190 comp rx=0 tx=0).stop 2023-11-06T03:08:17.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 msgr2=0x7f94b8153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f94b000caa0 tx=0x7f94b000cf60 comp rx=0 tx=0).stop 2023-11-06T03:08:17.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 shutdown_connections 2023-11-06T03:08:17.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f949c065a90 0x7f949c067f50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b815ea60 0x7f94b8153be0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94b815e5e0 0x7f94b8141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 --2- 172.21.15.157:0/3856288874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94b81551e0 0x7f94b806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.425+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 >> 172.21.15.157:0/3856288874 conn(0x7f94b8065930 msgr2=0x7f94b815d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.429+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 shutdown_connections 2023-11-06T03:08:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.429+0000 7f94c05f0640 1 -- 172.21.15.157:0/3856288874 wait complete. 2023-11-06T03:08:17.443 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2023-11-06T03:08:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:17.551 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": -2 2023-11-06T03:08:17.551 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:17.565 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2023-11-06T03:08:17.565 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2023-11-06T03:08:17.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2023-11-06T03:08:17.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 Processor -- start 2023-11-06T03:08:17.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 -- start start 2023-11-06T03:08:17.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff24006b750 0x7ff24006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2400cb850 con 0x7ff24006bbe0 2023-11-06T03:08:17.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24006c110 con 0x7ff24006d5f0 2023-11-06T03:08:17.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff245734640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff240151f00 con 0x7ff24006b750 2023-11-06T03:08:17.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff24006bbe0 0x7ff2400701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59438/0 (socket says 172.21.15.157:59438) 2023-11-06T03:08:17.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23f7fe640 1 -- 172.21.15.157:0/1487017483 learned_addr learned my addr 172.21.15.157:0/1487017483 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:17.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/1487017483 >> v2:172.21.15.195:3300/0 conn(0x7ff24006b750 0x7ff24006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570794861 0 0) 0x7ff2400cb850 con 0x7ff24006bbe0 2023-11-06T03:08:17.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff228003530 con 0x7ff24006bbe0 2023-11-06T03:08:17.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325458382 0 0) 0x7ff228003530 con 0x7ff24006bbe0 2023-11-06T03:08:17.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 >> v2:172.21.15.195:3300/0 conn(0x7ff24006b750 msgr2=0x7ff24006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 --2- 172.21.15.157:0/1487017483 >> v2:172.21.15.195:3300/0 conn(0x7ff24006b750 0x7ff24006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 >> v1:172.21.15.195:6789/0 conn(0x7ff24006d5f0 legacy=0x7ff24006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24006c3f0 con 0x7ff24006bbe0 2023-11-06T03:08:17.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff238001a00 con 0x7ff24006bbe0 2023-11-06T03:08:17.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 >> v1:172.21.15.157:6789/0 conn(0x7ff24006bbe0 legacy=0x7ff2400701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2280047d0 0x7ff228004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff228005190 0x7ff22800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240151f00 con 0x7ff2280039b0 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800c060 con 0x7ff228005190 2023-11-06T03:08:17.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800c220 con 0x7ff2280047d0 2023-11-06T03:08:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff228005190 0x7ff22800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 -- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2280047d0 msgr2=0x7ff228004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2280047d0 0x7ff228004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 -- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff228005190 msgr2=0x7ff22800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff228005190 0x7ff22800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23effd640 1 -- 172.21.15.157:0/1487017483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228003530 con 0x7ff2280039b0 2023-11-06T03:08:17.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff228005190 0x7ff22800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:17.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.693+0000 7ff23f7fe640 1 -- 172.21.15.157:0/1487017483 reap_dead start 2023-11-06T03:08:17.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff23effd640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff230001780 tx=0x7ff23001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2824c5e26dac79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300052d0 con 0x7ff2280039b0 2023-11-06T03:08:17.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2300a7050 con 0x7ff2280039b0 2023-11-06T03:08:17.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff23dffb640 1 -- 172.21.15.157:0/1487017483 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230003600 con 0x7ff2280039b0 2023-11-06T03:08:17.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 msgr2=0x7ff228003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff230001780 tx=0x7ff23001c3e0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- 172.21.15.157:0/1487017483 shutdown_connections 2023-11-06T03:08:17.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 --2- 172.21.15.157:0/1487017483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- 172.21.15.157:0/1487017483 >> 172.21.15.157:0/1487017483 conn(0x7ff240066090 msgr2=0x7ff2400b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- 172.21.15.157:0/1487017483 shutdown_connections 2023-11-06T03:08:17.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- 172.21.15.157:0/1487017483 wait complete. 2023-11-06T03:08:17.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 Processor -- start 2023-11-06T03:08:17.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.697+0000 7ff245734640 1 -- start start 2023-11-06T03:08:17.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2401f8940 0x7ff2401f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2401f9300 0x7ff2400af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240159390 con 0x7ff2280039b0 2023-11-06T03:08:17.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240158b80 con 0x7ff2401f8940 2023-11-06T03:08:17.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff240158dd0 con 0x7ff2401f9300 2023-11-06T03:08:17.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35692/0 (socket says 172.21.15.157:35692) 2023-11-06T03:08:17.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 -- 172.21.15.157:0/3872354532 learned_addr learned my addr 172.21.15.157:0/3872354532 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2401f8940 0x7ff2401f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2401f9300 0x7ff2400af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 -- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2401f9300 msgr2=0x7ff2400af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2401f9300 0x7ff2400af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 -- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2401f8940 msgr2=0x7ff2401f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2401f8940 0x7ff2401f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 -- 172.21.15.157:0/3872354532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2401590b0 con 0x7ff2280039b0 2023-11-06T03:08:17.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23effd640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff23001c210 tx=0x7ff228005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 -- 172.21.15.157:0/3872354532 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300072f0 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 -- 172.21.15.157:0/3872354532 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230007450 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2400ae900 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 -- 172.21.15.157:0/3872354532 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300067e0 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2401f9780 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 -- 172.21.15.157:0/3872354532 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff230006940 con 0x7ff2280039b0 2023-11-06T03:08:17.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 0x7ff22c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff2277fe640 1 -- 172.21.15.157:0/3872354532 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff23011d2e0 con 0x7ff2280039b0 2023-11-06T03:08:17.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 0x7ff22c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff24885c000 1 Processor -- start 2023-11-06T03:08:17.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff24885c000 1 -- start start 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.701+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 0x7ff22c067f50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff24015afe0 tx=0x7ff234008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d130350 0x55a94d0e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d0e65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94cebb050 con 0x55a94d12fa30 2023-11-06T03:08:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94cf3b140 con 0x55a94d0e61d0 2023-11-06T03:08:17.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a94cf38710 con 0x55a94d130350 2023-11-06T03:08:17.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35702/0 (socket says 172.21.15.157:35702) 2023-11-06T03:08:17.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/2841397885 learned_addr learned my addr 172.21.15.157:0/2841397885 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d130350 msgr2=0x55a94d0e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d130350 0x55a94d0e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 msgr2=0x55a94d0e65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d0e65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d0e65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/2841397885 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a94cc30cb0 con 0x55a94d12fa30 2023-11-06T03:08:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff2300050a0 tx=0x7ff2300ee430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3683192cd7c90fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff224ff9640 1 -- 172.21.15.157:0/2841397885 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300f25f0 con 0x55a94d12fa30 2023-11-06T03:08:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff224ff9640 1 -- 172.21.15.157:0/2841397885 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2300f67a0 con 0x55a94d12fa30 2023-11-06T03:08:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff224ff9640 1 -- 172.21.15.157:0/2841397885 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300ecb60 con 0x55a94d12fa30 2023-11-06T03:08:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- 172.21.15.157:0/2841397885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 msgr2=0x55a94d12fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff2300050a0 tx=0x7ff2300ee430 comp rx=0 tx=0).stop 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- 172.21.15.157:0/2841397885 shutdown_connections 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d0e65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d130350 0x55a94d0e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- 172.21.15.157:0/2841397885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d12fe10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- 172.21.15.157:0/2841397885 >> 172.21.15.157:0/2841397885 conn(0x55a94d0e2180 msgr2=0x55a94d0e4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- 172.21.15.157:0/2841397885 shutdown_connections 2023-11-06T03:08:17.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- 172.21.15.157:0/2841397885 wait complete. 2023-11-06T03:08:17.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- start start 2023-11-06T03:08:17.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 Processor -- start 2023-11-06T03:08:17.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d0f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d130350 0x55a94d0f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94cf34000 con 0x55a94d0e61d0 2023-11-06T03:08:17.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94ced39d0 con 0x55a94d130350 2023-11-06T03:08:17.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a94ced3670 con 0x55a94d12fa30 2023-11-06T03:08:17.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff24885c000 10 client.0 _ll_register_callbacks cb 0x55a94d0d1530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:17.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35718/0 (socket says 172.21.15.157:35718) 2023-11-06T03:08:17.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/1768791544 learned_addr learned my addr 172.21.15.157:0/1768791544 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d130350 0x55a94d0f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d0f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 msgr2=0x55a94d0f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d0f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d130350 msgr2=0x55a94d0f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d130350 0x55a94d0f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.705+0000 7ff23effd640 1 -- 172.21.15.157:0/1768791544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2401f9780 con 0x55a94d0e61d0 2023-11-06T03:08:17.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff23effd640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff23001c950 tx=0x7ff2300077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cee26423a742e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300f25f0 con 0x55a94d0e61d0 2023-11-06T03:08:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff2300013a0 con 0x55a94d0e61d0 2023-11-06T03:08:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a94cf017a0 con 0x55a94d0e61d0 2023-11-06T03:08:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2300ecb60 con 0x55a94d0e61d0 2023-11-06T03:08:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 10 client.6339 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55a94cef92f0 con 0x55a94d0e61d0 2023-11-06T03:08:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a94cf017a0 con 0x55a94d0e61d0 2023-11-06T03:08:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff23010c1a0 con 0x55a94d0e61d0 2023-11-06T03:08:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 10 client.6339 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:17.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 10 client.6339 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:17.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 20 client.6339 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:17.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d137080 0x55a94d137460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 4 client.6339 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:17.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a94cf32950 con 0x55a94d137080 2023-11-06T03:08:17.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d137080 0x55a94d137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d137080 0x55a94d137460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5aa20fef213c46d server_cookie=45313ac9b1516851 in_seq=0 out_seq=0 2023-11-06T03:08:17.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 10 client.6339 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 1 -- 172.21.15.157:0/1768791544 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a94cf32950 con 0x55a94d137080 2023-11-06T03:08:17.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 10 client.6339 handle_command_reply: tid=0 2023-11-06T03:08:17.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 client.6339 shutdown 2023-11-06T03:08:17.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 msgr2=0x55a94d0eb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff23001c950 tx=0x7ff2300077e0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 shutdown_connections 2023-11-06T03:08:17.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d137080 0x55a94d137460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d130350 0x55a94d0f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d0f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff21dffb640 0 client.6339 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 --2- 172.21.15.157:0/1768791544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0eb370 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 >> 172.21.15.157:0/1768791544 conn(0x55a94d0e2180 msgr2=0x55a94d0e4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 shutdown_connections 2023-11-06T03:08:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 1 -- 172.21.15.157:0/1768791544 wait complete. 2023-11-06T03:08:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.709+0000 7ff24885c000 20 client.6339 trim_cache size 0 max 16384 2023-11-06T03:08:17.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 Processor -- start 2023-11-06T03:08:17.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- start start 2023-11-06T03:08:17.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e2180 0x55a94d0e4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d130730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94cbf1d60 con 0x55a94d0e61d0 2023-11-06T03:08:17.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94cf34000 con 0x55a94d0e2180 2023-11-06T03:08:17.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a94cefbf00 con 0x55a94d12fa30 2023-11-06T03:08:17.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35724/0 (socket says 172.21.15.157:35724) 2023-11-06T03:08:17.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 -- 172.21.15.157:0/4127717240 learned_addr learned my addr 172.21.15.157:0/4127717240 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 -- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 msgr2=0x55a94d130730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d130730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 -- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e2180 msgr2=0x55a94d0e4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e2180 0x55a94d0e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 -- 172.21.15.157:0/4127717240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a94cedf440 con 0x55a94d0e61d0 2023-11-06T03:08:17.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff234009720 tx=0x7ff234013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e5ad6d94268188 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff2257fa640 1 -- 172.21.15.157:0/4127717240 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340250e0 con 0x55a94d0e61d0 2023-11-06T03:08:17.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff2257fa640 1 -- 172.21.15.157:0/4127717240 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23401c070 con 0x55a94d0e61d0 2023-11-06T03:08:17.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff2257fa640 1 -- 172.21.15.157:0/4127717240 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234025440 con 0x55a94d0e61d0 2023-11-06T03:08:17.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- 172.21.15.157:0/4127717240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 msgr2=0x55a94d0e4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff234009720 tx=0x7ff234013000 comp rx=0 tx=0).stop 2023-11-06T03:08:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- 172.21.15.157:0/4127717240 shutdown_connections 2023-11-06T03:08:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d12fa30 0x55a94d130730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d0e61d0 0x55a94d0e4ab0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 --2- 172.21.15.157:0/4127717240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e2180 0x55a94d0e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- 172.21.15.157:0/4127717240 >> 172.21.15.157:0/4127717240 conn(0x55a94d0eb750 msgr2=0x55a94d0f16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- 172.21.15.157:0/4127717240 shutdown_connections 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- 172.21.15.157:0/4127717240 wait complete. 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 -- start start 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.725+0000 7ff24885c000 1 Processor -- start 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d0e2180 0x55a94d18bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d18c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94d0e9900 con 0x55a94d12fa30 2023-11-06T03:08:17.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94d0dec10 con 0x55a94d0e61d0 2023-11-06T03:08:17.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a94d0e1e50 con 0x55a94d0e2180 2023-11-06T03:08:17.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 10 client.0 _ll_register_callbacks cb 0x55a94cee26d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:17.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35738/0 (socket says 172.21.15.157:35738) 2023-11-06T03:08:17.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 -- 172.21.15.157:0/2228598421 learned_addr learned my addr 172.21.15.157:0/2228598421 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23effd640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d0e2180 0x55a94d18bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23e7fc640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d18c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 -- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d0e2180 msgr2=0x55a94d18bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d0e2180 0x55a94d18bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 -- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 msgr2=0x55a94d18c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d18c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 -- 172.21.15.157:0/2228598421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23400e050 con 0x55a94d12fa30 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23f7fe640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff238000f90 tx=0x7ff238010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494f14552844b15d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238018030 con 0x55a94d12fa30 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff238019070 con 0x55a94d12fa30 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a94d0e1f90 con 0x55a94d12fa30 2023-11-06T03:08:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff238004d70 con 0x55a94d12fa30 2023-11-06T03:08:17.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 10 client.6345 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:17.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55a94cc310e0 con 0x55a94d12fa30 2023-11-06T03:08:17.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a94d0e1f90 con 0x55a94d12fa30 2023-11-06T03:08:17.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff23800e750 con 0x55a94d12fa30 2023-11-06T03:08:17.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 10 client.6345 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:17.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 10 client.6345 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:17.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 20 client.6345 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:17.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d130350 0x55a94d133980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 4 client.6345 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2023-11-06T03:08:17.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55a94cef47b0 con 0x55a94d130350 2023-11-06T03:08:17.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23effd640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d130350 0x55a94d133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff23effd640 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d130350 0x55a94d133980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382ff1e846a6a3a4 server_cookie=c5fc4663d2f49ed5 in_seq=0 out_seq=0 2023-11-06T03:08:17.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 10 client.6345 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 1 -- 172.21.15.157:0/2228598421 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55a94cef47b0 con 0x55a94d130350 2023-11-06T03:08:17.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 10 client.6345 handle_command_reply: tid=0 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 client.6345 shutdown 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 msgr2=0x55a94d132a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff238000f90 tx=0x7ff238010cd0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 shutdown_connections 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55a94d130350 0x55a94d133980 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a94d12fa30 0x55a94d132a00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a94d0e61d0 0x55a94d18c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff21dffb640 0 client.6345 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 --2- 172.21.15.157:0/2228598421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a94d0e2180 0x55a94d18bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 >> 172.21.15.157:0/2228598421 conn(0x55a94d0eb750 msgr2=0x55a94d12e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 shutdown_connections 2023-11-06T03:08:17.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 1 -- 172.21.15.157:0/2228598421 wait complete. 2023-11-06T03:08:17.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff24885c000 20 client.6345 trim_cache size 0 max 16384 2023-11-06T03:08:17.753 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:17.753 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": -2 2023-11-06T03:08:17.754 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:17.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 msgr2=0x7ff22c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 0x7ff22c067f50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff24015afe0 tx=0x7ff234008040 comp rx=0 tx=0).stop 2023-11-06T03:08:17.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 msgr2=0x7ff24014d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff23001c210 tx=0x7ff228005800 comp rx=0 tx=0).stop 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 shutdown_connections 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff22c065a90 0x7ff22c067f50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2401f9300 0x7ff2400af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2401f8940 0x7ff2401f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 --2- 172.21.15.157:0/3872354532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2280039b0 0x7ff24014d670 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 >> 172.21.15.157:0/3872354532 conn(0x7ff240066090 msgr2=0x7ff24006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 shutdown_connections 2023-11-06T03:08:17.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.729+0000 7ff245734640 1 -- 172.21.15.157:0/3872354532 wait complete. 2023-11-06T03:08:17.757 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2023-11-06T03:08:17.758 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2023-11-06T03:08:17.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2023-11-06T03:08:17.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 Processor -- start 2023-11-06T03:08:17.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 -- start start 2023-11-06T03:08:17.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415aca0 0x7f24e40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e4071940 0x7f24e40cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e40cef50 con 0x7f24e40d0120 2023-11-06T03:08:17.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40d05d0 con 0x7f24e4071940 2023-11-06T03:08:17.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ec49b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40cf230 con 0x7f24e415aca0 2023-11-06T03:08:17.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e4071940 0x7f24e40cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e4071940 0x7f24e40cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35744/0 (socket says 172.21.15.157:35744) 2023-11-06T03:08:17.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 -- 172.21.15.157:0/835517361 learned_addr learned my addr 172.21.15.157:0/835517361 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24ea237640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415aca0 0x7f24e40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 -- 172.21.15.157:0/835517361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415aca0 msgr2=0x7f24e40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415aca0 0x7f24e40ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 -- 172.21.15.157:0/835517361 >> v1:172.21.15.195:6789/0 conn(0x7f24e40d0120 legacy=0x7f24e40b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 -- 172.21.15.157:0/835517361 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40cf510 con 0x7f24e4071940 2023-11-06T03:08:17.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.877+0000 7f24e9a36640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e40cfbe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009870 tx=0x7f24d401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e769d5638ccd58c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24e9235640 1 -- 172.21.15.157:0/835517361 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40a95f0 con 0x7f24e4071940 2023-11-06T03:08:17.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24e9235640 1 -- 172.21.15.157:0/835517361 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40a9750 con 0x7f24e4071940 2023-11-06T03:08:17.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24e9235640 1 -- 172.21.15.157:0/835517361 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40a9ae0 con 0x7f24e4071940 2023-11-06T03:08:17.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/835517361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 msgr2=0x7f24e40cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e40cfbe0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009870 tx=0x7f24d401c510 comp rx=0 tx=0).stop 2023-11-06T03:08:17.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/835517361 shutdown_connections 2023-11-06T03:08:17.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e40cfbe0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- 172.21.15.157:0/835517361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415aca0 0x7f24e40ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/835517361 >> 172.21.15.157:0/835517361 conn(0x7f24e406d560 msgr2=0x7f24e406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/835517361 shutdown_connections 2023-11-06T03:08:17.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/835517361 wait complete. 2023-11-06T03:08:17.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 Processor -- start 2023-11-06T03:08:17.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- start start 2023-11-06T03:08:17.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e40d0120 0x7f24e4155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 0x7f24e40c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41562c0 con 0x7f24e4071940 2023-11-06T03:08:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e415da50 con 0x7f24e415aca0 2023-11-06T03:08:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40b2860 con 0x7f24e40d0120 2023-11-06T03:08:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35748/0 (socket says 172.21.15.157:35748) 2023-11-06T03:08:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 -- 172.21.15.157:0/3209835279 learned_addr learned my addr 172.21.15.157:0/3209835279 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24e9a36640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e40d0120 0x7f24e4155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24eaa38640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 0x7f24e40c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 -- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e40d0120 msgr2=0x7f24e4155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e40d0120 0x7f24e4155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 -- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 msgr2=0x7f24e40c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 0x7f24e40c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 -- 172.21.15.157:0/3209835279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40094d0 con 0x7f24e4071940 2023-11-06T03:08:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24eaa38640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 0x7f24e40c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ea237640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d800 tx=0x7f24e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24daffd640 1 -- 172.21.15.157:0/3209835279 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000b6c0 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24daffd640 1 -- 172.21.15.157:0/3209835279 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000b820 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24daffd640 1 -- 172.21.15.157:0/3209835279 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000bb90 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e4153650 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.881+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4153c10 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24daffd640 1 -- 172.21.15.157:0/3209835279 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f24e000bcf0 con 0x7f24e4071940 2023-11-06T03:08:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24daffd640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 0x7f24c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24daffd640 1 -- 172.21.15.157:0/3209835279 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24e0083830 con 0x7f24e4071940 2023-11-06T03:08:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24e9a36640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 0x7f24c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 Processor -- start 2023-11-06T03:08:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 -- start start 2023-11-06T03:08:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24e9a36640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 0x7f24c8067f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ae000 tx=0x7f24d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78ed5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed59d0 0x557b78e8ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78ce0270 con 0x557b78e8b2b0 2023-11-06T03:08:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78cdd840 con 0x557b78ed59d0 2023-11-06T03:08:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b78cd9130 con 0x557b78ed50b0 2023-11-06T03:08:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35754/0 (socket says 172.21.15.157:35754) 2023-11-06T03:08:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.885+0000 7f24eaa38640 1 -- 172.21.15.157:0/836339581 learned_addr learned my addr 172.21.15.157:0/836339581 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ea237640 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78ed5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24e9a36640 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed59d0 0x557b78e8ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 msgr2=0x557b78ed5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78ed5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed59d0 msgr2=0x557b78e8ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed59d0 0x557b78e8ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/836339581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b789d5d70 con 0x557b78e8b2b0 2023-11-06T03:08:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f24dc001cd0 tx=0x7f24dc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c274071595365e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c7fff640 1 -- 172.21.15.157:0/836339581 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00b070 con 0x557b78e8b2b0 2023-11-06T03:08:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c7fff640 1 -- 172.21.15.157:0/836339581 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc003db0 con 0x557b78e8b2b0 2023-11-06T03:08:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c7fff640 1 -- 172.21.15.157:0/836339581 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00dab0 con 0x557b78e8b2b0 2023-11-06T03:08:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/836339581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 msgr2=0x557b78e8b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f24dc001cd0 tx=0x7f24dc002190 comp rx=0 tx=0).stop 2023-11-06T03:08:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/836339581 shutdown_connections 2023-11-06T03:08:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78e8b2b0 0x557b78e8b690 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed59d0 0x557b78e8ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/836339581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78ed5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/836339581 >> 172.21.15.157:0/836339581 conn(0x557b78e91620 msgr2=0x557b78e89650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/836339581 shutdown_connections 2023-11-06T03:08:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/836339581 wait complete. 2023-11-06T03:08:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- start start 2023-11-06T03:08:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 Processor -- start 2023-11-06T03:08:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78e90330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78e90870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78c789b0 con 0x557b78ed59d0 2023-11-06T03:08:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78c78650 con 0x557b78e8b2b0 2023-11-06T03:08:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b78996e20 con 0x557b78ed50b0 2023-11-06T03:08:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 10 client.0 _ll_register_callbacks cb 0x557b78c874d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35758/0 (socket says 172.21.15.157:35758) 2023-11-06T03:08:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/2216916438 learned_addr learned my addr 172.21.15.157:0/2216916438 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ea237640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78e90330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24e9a36640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78e90870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 msgr2=0x557b78e90870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78e90870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 msgr2=0x557b78e90330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78e90330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 -- 172.21.15.157:0/2216916438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40cef50 con 0x557b78ed59d0 2023-11-06T03:08:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24eaa38640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x557b78e7b350 tx=0x7f24dc001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de5cdd87cc51377 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc007dd0 con 0x557b78ed59d0 2023-11-06T03:08:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24dc0015b0 con 0x557b78ed59d0 2023-11-06T03:08:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b78ca4290 con 0x557b78ed59d0 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24dc00dac0 con 0x557b78ed59d0 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 10 client.6357 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x557b78c9e360 con 0x557b78ed59d0 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b78ca4290 con 0x557b78ed59d0 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f24dc00dc20 con 0x557b78ed59d0 2023-11-06T03:08:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 10 client.6357 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 10 client.6357 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 20 client.6357 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78ec2d90 0x557b78ec3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 4 client.6357 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.889+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b78cd7a80 con 0x557b78ec2d90 2023-11-06T03:08:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ea237640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78ec2d90 0x557b78ec3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24c57fa640 10 client.6357 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ea237640 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78ec2d90 0x557b78ec3d90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc4d6394086bdda server_cookie=2c72b36a0af7cb88 in_seq=0 out_seq=0 2023-11-06T03:08:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24c57fa640 1 -- 172.21.15.157:0/2216916438 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557b78cd7a80 con 0x557b78ec2d90 2023-11-06T03:08:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24c57fa640 10 client.6357 handle_command_reply: tid=0 2023-11-06T03:08:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 client.6357 shutdown 2023-11-06T03:08:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 msgr2=0x557b78ec3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x557b78e7b350 tx=0x7f24dc001260 comp rx=0 tx=0).stop 2023-11-06T03:08:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 shutdown_connections 2023-11-06T03:08:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78ec2d90 0x557b78ec3d90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ec3380 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed50b0 0x557b78e90870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/2216916438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78e90330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24c57fa640 0 client.6357 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 >> 172.21.15.157:0/2216916438 conn(0x557b78e91620 msgr2=0x557b78e89d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 shutdown_connections 2023-11-06T03:08:17.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 1 -- 172.21.15.157:0/2216916438 wait complete. 2023-11-06T03:08:17.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.893+0000 7f24ef5c3000 20 client.6357 trim_cache size 0 max 16384 2023-11-06T03:08:17.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.905+0000 7f24ef5c3000 1 Processor -- start 2023-11-06T03:08:17.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.905+0000 7f24ef5c3000 1 -- start start 2023-11-06T03:08:17.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78ec16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed59d0 0x557b78ee5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78e82ac0 con 0x557b78ed50b0 2023-11-06T03:08:17.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78cd7fc0 con 0x557b78e8b2b0 2023-11-06T03:08:17.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b78c9bb70 con 0x557b78ed59d0 2023-11-06T03:08:17.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35760/0 (socket says 172.21.15.157:35760) 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 -- 172.21.15.157:0/1795255307 learned_addr learned my addr 172.21.15.157:0/1795255307 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ea237640 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78ec16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 -- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed59d0 msgr2=0x557b78ee5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed59d0 0x557b78ee5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 -- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 msgr2=0x557b78ec16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78ec16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 -- 172.21.15.157:0/1795255307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b78c844b0 con 0x557b78ed50b0 2023-11-06T03:08:17.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009010 tx=0x7f24d4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998baf325d09e524 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24d8ff9640 1 -- 172.21.15.157:0/1795255307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40b6070 con 0x557b78ed50b0 2023-11-06T03:08:17.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24d8ff9640 1 -- 172.21.15.157:0/1795255307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40bc4d0 con 0x557b78ed50b0 2023-11-06T03:08:17.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24d8ff9640 1 -- 172.21.15.157:0/1795255307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40bc840 con 0x557b78ed50b0 2023-11-06T03:08:17.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/1795255307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 msgr2=0x557b78ec1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009010 tx=0x7f24d4004740 comp rx=0 tx=0).stop 2023-11-06T03:08:17.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/1795255307 shutdown_connections 2023-11-06T03:08:17.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78ed59d0 0x557b78ee5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed50b0 0x557b78ec1c10 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/1795255307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78e8b2b0 0x557b78ec16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/1795255307 >> 172.21.15.157:0/1795255307 conn(0x557b78e91620 msgr2=0x557b78ebffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/1795255307 shutdown_connections 2023-11-06T03:08:17.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/1795255307 wait complete. 2023-11-06T03:08:17.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- start start 2023-11-06T03:08:17.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 Processor -- start 2023-11-06T03:08:17.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78e8b2b0 0x557b78ee8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 0x557b78ee8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78e8e890 con 0x557b78ed59d0 2023-11-06T03:08:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b78ed4900 con 0x557b78ed50b0 2023-11-06T03:08:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b78cd6940 con 0x557b78e8b2b0 2023-11-06T03:08:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 10 client.0 _ll_register_callbacks cb 0x557b78e57530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35764/0 (socket says 172.21.15.157:35764) 2023-11-06T03:08:17.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 -- 172.21.15.157:0/3220121954 learned_addr learned my addr 172.21.15.157:0/3220121954 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:17.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 0x557b78ee8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ea237640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78e8b2b0 0x557b78ee8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 -- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78e8b2b0 msgr2=0x557b78ee8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78e8b2b0 0x557b78ee8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 -- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 msgr2=0x557b78ee8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:17.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 0x557b78ee8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 -- 172.21.15.157:0/3220121954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40cf510 con 0x557b78ed59d0 2023-11-06T03:08:17.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24e9a36640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 0x557b78ee8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:17.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24eaa38640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f24dc001c20 tx=0x7f24dc0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f885414806f7f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:17.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:17.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0015b0 con 0x557b78ed59d0 2023-11-06T03:08:17.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f24dc0017b0 con 0x557b78ed59d0 2023-11-06T03:08:17.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b78c9a2e0 con 0x557b78ed59d0 2023-11-06T03:08:17.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24dc00dcd0 con 0x557b78ed59d0 2023-11-06T03:08:17.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 10 client.6363 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:17.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.909+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x557b789d61a0 con 0x557b78ed59d0 2023-11-06T03:08:17.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b78c9a2e0 con 0x557b78ed59d0 2023-11-06T03:08:17.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f24dc005440 con 0x557b78ed59d0 2023-11-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 10 client.6363 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 10 client.6363 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 20 client.6363 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78f37330 0x557b78f39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 4 client.6363 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2023-11-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x557b78cd21b0 con 0x557b78f37330 2023-11-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ea237640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78f37330 0x557b78f39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ea237640 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78f37330 0x557b78f39720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a96270bc575205 server_cookie=d8c8452d257577 in_seq=0 out_seq=0 2023-11-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 10 client.6363 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 1 -- 172.21.15.157:0/3220121954 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557b78cd21b0 con 0x557b78f37330 2023-11-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 10 client.6363 handle_command_reply: tid=0 2023-11-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 client.6363 shutdown 2023-11-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 msgr2=0x557b78ee93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f24dc001c20 tx=0x7f24dc0014b0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 shutdown_connections 2023-11-06T03:08:17.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x557b78f37330 0x557b78f39720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b78ed59d0 0x557b78ee93b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24c57fa640 0 client.6363 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:17.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557b78ed50b0 0x557b78ee8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 --2- 172.21.15.157:0/3220121954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557b78e8b2b0 0x557b78ee8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 >> 172.21.15.157:0/3220121954 conn(0x557b78e91620 msgr2=0x557b78ec2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 shutdown_connections 2023-11-06T03:08:17.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 1 -- 172.21.15.157:0/3220121954 wait complete. 2023-11-06T03:08:17.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ef5c3000 20 client.6363 trim_cache size 0 max 16384 2023-11-06T03:08:17.936 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:17.936 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:17.936 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "421cc32c-5739-4247-b2a8-2a00b770b494", 2023-11-06T03:08:17.936 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:17.936 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:17.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 msgr2=0x7f24c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 0x7f24c8067f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ae000 tx=0x7f24d4009040 comp rx=0 tx=0).stop 2023-11-06T03:08:17.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 msgr2=0x7f24e41f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:17.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d800 tx=0x7f24e000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 shutdown_connections 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c8065a60 0x7f24c8067f20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e415aca0 0x7f24e40c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e40d0120 0x7f24e4155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 --2- 172.21.15.157:0/3209835279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e4071940 0x7f24e41f4c90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:17.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 >> 172.21.15.157:0/3209835279 conn(0x7f24e406d560 msgr2=0x7f24e4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:17.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 shutdown_connections 2023-11-06T03:08:17.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:17.913+0000 7f24ec49b640 1 -- 172.21.15.157:0/3209835279 wait complete. 2023-11-06T03:08:17.940 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '421cc32c-5739-4247-b2a8-2a00b770b494', 'mode': 'asynchronous'}' 2023-11-06T03:08:17.941 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2023-11-06T03:08:17.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2023-11-06T03:08:18.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 Processor -- start 2023-11-06T03:08:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 -- start start 2023-11-06T03:08:18.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc84146370 0x7fcc840b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc840cf1a0 con 0x7fcc8406b6a0 2023-11-06T03:08:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8406c3d0 con 0x7fcc84146370 2023-11-06T03:08:18.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc88b58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8406f5a0 con 0x7fcc840ce800 2023-11-06T03:08:18.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc8259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcc84146370 0x7fcc840b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:57264/0 (socket says 172.21.15.157:57264) 2023-11-06T03:08:18.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 -- 172.21.15.157:0/4281897553 learned_addr learned my addr 172.21.15.157:0/4281897553 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc82d9d640 1 --1- 172.21.15.157:0/4281897553 >> v1:172.21.15.195:6789/0 conn(0x7fcc8406b6a0 0x7fcc8406bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:50376/0 (socket says 172.21.15.157:50376) 2023-11-06T03:08:18.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 -- 172.21.15.157:0/4281897553 >> v2:172.21.15.195:3300/0 conn(0x7fcc84146370 msgr2=0x7fcc840b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/4281897553 >> v2:172.21.15.195:3300/0 conn(0x7fcc84146370 0x7fcc840b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 -- 172.21.15.157:0/4281897553 >> v1:172.21.15.195:6789/0 conn(0x7fcc8406b6a0 legacy=0x7fcc8406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.061+0000 7fcc79d9b640 1 -- 172.21.15.157:0/4281897553 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8406f880 con 0x7fcc840ce800 2023-11-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/4281897553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009870 tx=0x7fcc6c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32364f564e79468 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc81d9b640 1 -- 172.21.15.157:0/4281897553 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0aa610 con 0x7fcc840ce800 2023-11-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc81d9b640 1 -- 172.21.15.157:0/4281897553 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0a6040 con 0x7fcc840ce800 2023-11-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc81d9b640 1 -- 172.21.15.157:0/4281897553 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0aa990 con 0x7fcc840ce800 2023-11-06T03:08:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- 172.21.15.157:0/4281897553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 msgr2=0x7fcc840cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- 172.21.15.157:0/4281897553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009870 tx=0x7fcc6c01c4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- 172.21.15.157:0/4281897553 shutdown_connections 2023-11-06T03:08:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- 172.21.15.157:0/4281897553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc840cec60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- 172.21.15.157:0/4281897553 >> v2:172.21.15.195:3300/0 conn(0x7fcc84146370 0x7fcc840b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- 172.21.15.157:0/4281897553 >> 172.21.15.157:0/4281897553 conn(0x7fcc84065930 msgr2=0x7fcc84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- 172.21.15.157:0/4281897553 shutdown_connections 2023-11-06T03:08:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- 172.21.15.157:0/4281897553 wait complete. 2023-11-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 Processor -- start 2023-11-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- start start 2023-11-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 0x7fcc841f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc8415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415b5f0 con 0x7fcc84146370 2023-11-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415b8d0 con 0x7fcc8406b6a0 2023-11-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.065+0000 7fcc88b58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc841f8430 con 0x7fcc840ce800 2023-11-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35770/0 (socket says 172.21.15.157:35770) 2023-11-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 -- 172.21.15.157:0/1732180785 learned_addr learned my addr 172.21.15.157:0/1732180785 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 0x7fcc841f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc8415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 -- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 msgr2=0x7fcc8415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc8415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 -- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 msgr2=0x7fcc841f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 0x7fcc841f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 -- 172.21.15.157:0/1732180785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0094d0 con 0x7fcc84146370 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 0x7fcc841f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c700 tx=0x7fcc7c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 -- 172.21.15.157:0/1732180785 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00ce10 con 0x7fcc84146370 2023-11-06T03:08:18.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 -- 172.21.15.157:0/1732180785 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c00a640 con 0x7fcc84146370 2023-11-06T03:08:18.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc841f8710 con 0x7fcc84146370 2023-11-06T03:08:18.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 -- 172.21.15.157:0/1732180785 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00a9b0 con 0x7fcc84146370 2023-11-06T03:08:18.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8415b300 con 0x7fcc84146370 2023-11-06T03:08:18.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 -- 172.21.15.157:0/1732180785 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcc7c00ab10 con 0x7fcc84146370 2023-11-06T03:08:18.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 0x7fcc64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc7b7fe640 1 -- 172.21.15.157:0/1732180785 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcc7c082960 con 0x7fcc84146370 2023-11-06T03:08:18.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 0x7fcc64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc8bc80000 1 Processor -- start 2023-11-06T03:08:18.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.069+0000 7fcc8bc80000 1 -- start start 2023-11-06T03:08:18.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 0x7fcc64067f20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcc84141610 tx=0x7fcc74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d97d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d97dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7eda10 con 0x560c4d9c7470 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7d35e0 con 0x560c4d97da00 2023-11-06T03:08:18.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7d0bb0 con 0x560c4d9c7d90 2023-11-06T03:08:18.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35784/0 (socket says 172.21.15.157:35784) 2023-11-06T03:08:18.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 -- 172.21.15.157:0/2792917698 learned_addr learned my addr 172.21.15.157:0/2792917698 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d97d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d97dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 -- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 msgr2=0x560c4d97d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d97d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 -- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 msgr2=0x560c4d97dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d97dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 -- 172.21.15.157:0/2792917698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4d4c90d0 con 0x560c4d9c7470 2023-11-06T03:08:18.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d97d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:18.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011e50 tx=0x7fcc74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afea6794ccc5f1b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc63fff640 1 -- 172.21.15.157:0/2792917698 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011e80 con 0x560c4d9c7470 2023-11-06T03:08:18.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc63fff640 1 -- 172.21.15.157:0/2792917698 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc74004030 con 0x560c4d9c7470 2023-11-06T03:08:18.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc63fff640 1 -- 172.21.15.157:0/2792917698 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc740043a0 con 0x560c4d9c7470 2023-11-06T03:08:18.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2792917698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 msgr2=0x560c4d9c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011e50 tx=0x7fcc74014000 comp rx=0 tx=0).stop 2023-11-06T03:08:18.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2792917698 shutdown_connections 2023-11-06T03:08:18.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d97dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d97d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2792917698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c7850 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2792917698 >> 172.21.15.157:0/2792917698 conn(0x560c4d9799b0 msgr2=0x560c4d97bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2792917698 shutdown_connections 2023-11-06T03:08:18.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2792917698 wait complete. 2023-11-06T03:08:18.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- start start 2023-11-06T03:08:18.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 Processor -- start 2023-11-06T03:08:18.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d9acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7cc4a0 con 0x560c4d9c7470 2023-11-06T03:08:18.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d76ba70 con 0x560c4d97da00 2023-11-06T03:08:18.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4d789af0 con 0x560c4d9c7d90 2023-11-06T03:08:18.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8bc80000 10 client.0 _ll_register_callbacks cb 0x560c4d968d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:18.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35790/0 (socket says 172.21.15.157:35790) 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 -- 172.21.15.157:0/1562124011 learned_addr learned my addr 172.21.15.157:0/1562124011 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d9acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 -- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 msgr2=0x560c4d9acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d9acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 -- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 msgr2=0x560c4d9ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9ac4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc79d9b640 1 -- 172.21.15.157:0/1562124011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74011b00 con 0x560c4d9c7470 2023-11-06T03:08:18.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.073+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d9acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:18.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c01a0e0 tx=0x7fcc6c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149c446f6c312d86 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:18.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0aadd0 con 0x560c4d9c7470 2023-11-06T03:08:18.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4d794550 con 0x560c4d9c7470 2023-11-06T03:08:18.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcc6c0b8070 con 0x560c4d9c7470 2023-11-06T03:08:18.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c0b33f0 con 0x560c4d9c7470 2023-11-06T03:08:18.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 10 client.6372 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x560c4d7915a0 con 0x560c4d9c7470 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c4d794550 con 0x560c4d9c7470 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcc6c0b3550 con 0x560c4d9c7470 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 10 client.6372 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 10 client.6372 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:18.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 20 client.6372 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:18.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9ad930 0x560c4d9d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 4 client.6372 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:18.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c4d7d1390 con 0x560c4d9ad930 2023-11-06T03:08:18.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9ad930 0x560c4d9d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 10 client.6372 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8259c640 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9ad930 0x560c4d9d9520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b18d5a7ca848113 server_cookie=d8a2c229e86b0cd8 in_seq=0 out_seq=0 2023-11-06T03:08:18.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 1 -- 172.21.15.157:0/1562124011 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560c4d7d1390 con 0x560c4d9ad930 2023-11-06T03:08:18.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 10 client.6372 handle_command_reply: tid=0 2023-11-06T03:08:18.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 client.6372 shutdown 2023-11-06T03:08:18.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 msgr2=0x560c4d9ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c01a0e0 tx=0x7fcc6c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 shutdown_connections 2023-11-06T03:08:18.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9ad930 0x560c4d9d9520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9c7d90 0x560c4d9acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9ac9e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/1562124011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc617fa640 0 client.6372 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 >> 172.21.15.157:0/1562124011 conn(0x560c4d9799b0 msgr2=0x560c4d97bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 shutdown_connections 2023-11-06T03:08:18.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 1 -- 172.21.15.157:0/1562124011 wait complete. 2023-11-06T03:08:18.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.077+0000 7fcc8bc80000 20 client.6372 trim_cache size 0 max 16384 2023-11-06T03:08:18.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 Processor -- start 2023-11-06T03:08:18.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- start start 2023-11-06T03:08:18.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9799b0 0x560c4d97f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d983df0 con 0x560c4d9c7470 2023-11-06T03:08:18.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d48a180 con 0x560c4d97da00 2023-11-06T03:08:18.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4d979350 con 0x560c4d9799b0 2023-11-06T03:08:18.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35798/0 (socket says 172.21.15.157:35798) 2023-11-06T03:08:18.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 -- 172.21.15.157:0/2375796348 learned_addr learned my addr 172.21.15.157:0/2375796348 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 -- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9799b0 msgr2=0x560c4d97f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9799b0 0x560c4d97f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 -- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 msgr2=0x560c4d9c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 -- 172.21.15.157:0/2375796348 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4d7776f0 con 0x560c4d9c7470 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c053c80 tx=0x7fcc7c0539f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6bb28dbd81bde2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc78d99640 1 -- 172.21.15.157:0/2375796348 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c06b070 con 0x560c4d9c7470 2023-11-06T03:08:18.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc78d99640 1 -- 172.21.15.157:0/2375796348 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c05beb0 con 0x560c4d9c7470 2023-11-06T03:08:18.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc78d99640 1 -- 172.21.15.157:0/2375796348 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c052320 con 0x560c4d9c7470 2023-11-06T03:08:18.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2375796348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 msgr2=0x560c4d9c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c053c80 tx=0x7fcc7c0539f0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2375796348 shutdown_connections 2023-11-06T03:08:18.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9c7470 0x560c4d9c6a20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d97da00 0x560c4d9c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/2375796348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d9799b0 0x560c4d97f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2375796348 >> 172.21.15.157:0/2375796348 conn(0x560c4d9835c0 msgr2=0x560c4d9c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2375796348 shutdown_connections 2023-11-06T03:08:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- 172.21.15.157:0/2375796348 wait complete. 2023-11-06T03:08:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- start start 2023-11-06T03:08:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 Processor -- start 2023-11-06T03:08:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d97da00 0x560c4da2d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d9c7470 0x560c4da29420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7d4870 con 0x560c4d9799b0 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4d983480 con 0x560c4d9c7470 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c4d7d54c0 con 0x560c4d97da00 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8bc80000 10 client.0 _ll_register_callbacks cb 0x560c4d791080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:18.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35810/0 (socket says 172.21.15.157:35810) 2023-11-06T03:08:18.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.093+0000 7fcc8259c640 1 -- 172.21.15.157:0/3626379924 learned_addr learned my addr 172.21.15.157:0/3626379924 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc82d9d640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d9c7470 0x560c4da29420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 -- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d97da00 msgr2=0x560c4da2d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d97da00 0x560c4da2d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 -- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d9c7470 msgr2=0x560c4da29420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d9c7470 0x560c4da29420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 -- 172.21.15.157:0/3626379924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8415b300 con 0x560c4d9799b0 2023-11-06T03:08:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8259c640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009a70 tx=0x7fcc740046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3c7140eb6ec985 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74004130 con 0x560c4d9799b0 2023-11-06T03:08:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fcc74035c50 con 0x560c4d9799b0 2023-11-06T03:08:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4d7cadf0 con 0x560c4d9799b0 2023-11-06T03:08:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc74035e10 con 0x560c4d9799b0 2023-11-06T03:08:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 10 client.6378 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x560c4d4c9500 con 0x560c4d9799b0 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c4d7cadf0 con 0x560c4d9799b0 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcc74004290 con 0x560c4d9799b0 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 10 client.6378 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 10 client.6378 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 20 client.6378 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9c7d90 0x560c4da2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 4 client.6378 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2023-11-06T03:08:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x560c4d7c5520 con 0x560c4d9c7d90 2023-11-06T03:08:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9c7d90 0x560c4da2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 10 client.6378 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc79d9b640 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9c7d90 0x560c4da2a2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321409813aa8ba9a server_cookie=cfb81cb041ea0214 in_seq=0 out_seq=0 2023-11-06T03:08:18.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 1 -- 172.21.15.157:0/3626379924 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560c4d7c5520 con 0x560c4d9c7d90 2023-11-06T03:08:18.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 10 client.6378 handle_command_reply: tid=0 2023-11-06T03:08:18.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 client.6378 shutdown 2023-11-06T03:08:18.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 msgr2=0x560c4da2cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009a70 tx=0x7fcc740046d0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 shutdown_connections 2023-11-06T03:08:18.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x560c4d9c7d90 0x560c4da2a2d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560c4d9c7470 0x560c4da29420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560c4d97da00 0x560c4da2d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 --2- 172.21.15.157:0/3626379924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560c4d9799b0 0x560c4da2cdb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 >> 172.21.15.157:0/3626379924 conn(0x560c4d9835c0 msgr2=0x560c4d9c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc617fa640 0 client.6378 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 shutdown_connections 2023-11-06T03:08:18.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 1 -- 172.21.15.157:0/3626379924 wait complete. 2023-11-06T03:08:18.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc8bc80000 20 client.6378 trim_cache size 0 max 16384 2023-11-06T03:08:18.116 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:18.116 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:18.116 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "0303af1b-1935-4181-8088-fc632259c945", 2023-11-06T03:08:18.117 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:18.117 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:18.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 msgr2=0x7fcc64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 0x7fcc64067f20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcc84141610 tx=0x7fcc74008040 comp rx=0 tx=0).stop 2023-11-06T03:08:18.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 msgr2=0x7fcc840af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c700 tx=0x7fcc7c00cbc0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 shutdown_connections 2023-11-06T03:08:18.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc64065a60 0x7fcc64067f20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc84146370 0x7fcc840af830 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc840ce800 0x7fcc8415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 --2- 172.21.15.157:0/1732180785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc8406b6a0 0x7fcc841f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 >> 172.21.15.157:0/1732180785 conn(0x7fcc84065930 msgr2=0x7fcc840b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 shutdown_connections 2023-11-06T03:08:18.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.097+0000 7fcc88b58640 1 -- 172.21.15.157:0/1732180785 wait complete. 2023-11-06T03:08:18.120 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '0303af1b-1935-4181-8088-fc632259c945', 'mode': 'asynchronous'}' 2023-11-06T03:08:18.121 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2023-11-06T03:08:18.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2023-11-06T03:08:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 Processor -- start 2023-11-06T03:08:18.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 -- start start 2023-11-06T03:08:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac04070880 0x7fac0415d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0415d9c0 con 0x7fac040cefc0 2023-11-06T03:08:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0415e220 con 0x7fac04070880 2023-11-06T03:08:18.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac0b365640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0415dc60 con 0x7fac04070420 2023-11-06T03:08:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac09902640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac04070880 0x7fac0415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac09101640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fac040cefc0 0x7fac040cf3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:50388/0 (socket says 172.21.15.157:50388) 2023-11-06T03:08:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:57322/0 (socket says 172.21.15.157:57322) 2023-11-06T03:08:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac09101640 1 -- 172.21.15.157:0/2763663343 learned_addr learned my addr 172.21.15.157:0/2763663343 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 -- 172.21.15.157:0/2763663343 >> v2:172.21.15.195:3300/0 conn(0x7fac04070880 msgr2=0x7fac0415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 --2- 172.21.15.157:0/2763663343 >> v2:172.21.15.195:3300/0 conn(0x7fac04070880 0x7fac0415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 -- 172.21.15.157:0/2763663343 >> v1:172.21.15.195:6789/0 conn(0x7fac040cefc0 legacy=0x7fac040cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.237+0000 7fac08900640 1 -- 172.21.15.157:0/2763663343 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0415df40 con 0x7fac04070420 2023-11-06T03:08:18.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac08900640 1 --2- 172.21.15.157:0/2763663343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabf40097c0 tx=0x7fabf401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7fe21b39ab8587d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fabfbfff640 1 -- 172.21.15.157:0/2763663343 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40ac620 con 0x7fac04070420 2023-11-06T03:08:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fabfbfff640 1 -- 172.21.15.157:0/2763663343 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf40ac780 con 0x7fac04070420 2023-11-06T03:08:18.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fabfbfff640 1 -- 172.21.15.157:0/2763663343 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf40acb10 con 0x7fac04070420 2023-11-06T03:08:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- 172.21.15.157:0/2763663343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 msgr2=0x7fac0415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 --2- 172.21.15.157:0/2763663343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabf40097c0 tx=0x7fabf401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- 172.21.15.157:0/2763663343 shutdown_connections 2023-11-06T03:08:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 --2- 172.21.15.157:0/2763663343 >> v2:172.21.15.195:3300/0 conn(0x7fac04070880 0x7fac0415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 --2- 172.21.15.157:0/2763663343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac04070420 0x7fac0415cf40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- 172.21.15.157:0/2763663343 >> 172.21.15.157:0/2763663343 conn(0x7fac0406a4d0 msgr2=0x7fac0406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- 172.21.15.157:0/2763663343 shutdown_connections 2023-11-06T03:08:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- 172.21.15.157:0/2763663343 wait complete. 2023-11-06T03:08:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 Processor -- start 2023-11-06T03:08:18.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.241+0000 7fac0b365640 1 -- start start 2023-11-06T03:08:18.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 0x7fac0414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac040cefc0 0x7fac04152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0415e590 con 0x7fac04070420 2023-11-06T03:08:18.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04152dd0 con 0x7fac04070880 2023-11-06T03:08:18.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac041f95e0 con 0x7fac040cefc0 2023-11-06T03:08:18.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35812/0 (socket says 172.21.15.157:35812) 2023-11-06T03:08:18.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 -- 172.21.15.157:0/2989378404 learned_addr learned my addr 172.21.15.157:0/2989378404 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09902640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac040cefc0 0x7fac04152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac08900640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 0x7fac0414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 -- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac040cefc0 msgr2=0x7fac04152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac040cefc0 0x7fac04152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 -- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 msgr2=0x7fac0414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 0x7fac0414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 -- 172.21.15.157:0/2989378404 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009420 con 0x7fac04070420 2023-11-06T03:08:18.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac08900640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 0x7fac0414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:18.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac09101640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fac0000c480 tx=0x7fac0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 -- 172.21.15.157:0/2989378404 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00005d40 con 0x7fac04070420 2023-11-06T03:08:18.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 -- 172.21.15.157:0/2989378404 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac000070d0 con 0x7fac04070420 2023-11-06T03:08:18.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac041f8d40 con 0x7fac04070420 2023-11-06T03:08:18.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 -- 172.21.15.157:0/2989378404 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00005ea0 con 0x7fac04070420 2023-11-06T03:08:18.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac041f9300 con 0x7fac04070420 2023-11-06T03:08:18.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 -- 172.21.15.157:0/2989378404 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fac00007480 con 0x7fac04070420 2023-11-06T03:08:18.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 0x7fabe8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fabf9ffb640 1 -- 172.21.15.157:0/2989378404 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fac00082780 con 0x7fac04070420 2023-11-06T03:08:18.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac08900640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 0x7fabe8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.245+0000 7fac0e48d000 1 Processor -- start 2023-11-06T03:08:18.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- start start 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac08900640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 0x7fabe8067f20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fabf40ae000 tx=0x7fabf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c811fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c811fa690 0x555c811faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c810c1d00 con 0x555c81243cb0 2023-11-06T03:08:18.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c81050270 con 0x555c812445d0 2023-11-06T03:08:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c8104d840 con 0x555c811fa690 2023-11-06T03:08:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35826/0 (socket says 172.21.15.157:35826) 2023-11-06T03:08:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/4136993108 learned_addr learned my addr 172.21.15.157:0/4136993108 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac08900640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c811fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c811fa690 msgr2=0x555c811faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09902640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c811fa690 0x555c811faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c811fa690 0x555c811faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 msgr2=0x555c811fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c811fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/4136993108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c80d45eb0 con 0x555c81243cb0 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac08900640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c811fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fac0005b720 tx=0x7fac00053de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a8660638e32a46 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fabeeffd640 1 -- 172.21.15.157:0/4136993108 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00051e60 con 0x555c81243cb0 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fabeeffd640 1 -- 172.21.15.157:0/4136993108 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac000520c0 con 0x555c81243cb0 2023-11-06T03:08:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fabeeffd640 1 -- 172.21.15.157:0/4136993108 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0005b7c0 con 0x555c81243cb0 2023-11-06T03:08:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- 172.21.15.157:0/4136993108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 msgr2=0x555c81244090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fac0005b720 tx=0x7fac00053de0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- 172.21.15.157:0/4136993108 shutdown_connections 2023-11-06T03:08:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c811fa690 0x555c811faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c811fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- 172.21.15.157:0/4136993108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c81243cb0 0x555c81244090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- 172.21.15.157:0/4136993108 >> 172.21.15.157:0/4136993108 conn(0x555c812007f0 msgr2=0x555c811f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- 172.21.15.157:0/4136993108 shutdown_connections 2023-11-06T03:08:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- 172.21.15.157:0/4136993108 wait complete. 2023-11-06T03:08:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- start start 2023-11-06T03:08:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 Processor -- start 2023-11-06T03:08:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c812052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c812057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c81049130 con 0x555c811fa690 2023-11-06T03:08:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c80fe85c0 con 0x555c812445d0 2023-11-06T03:08:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c810067e0 con 0x555c81243cb0 2023-11-06T03:08:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35842/0 (socket says 172.21.15.157:35842) 2023-11-06T03:08:18.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac0e48d000 10 client.0 _ll_register_callbacks cb 0x555c811e5dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/572413065 learned_addr learned my addr 172.21.15.157:0/572413065 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac08900640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c812052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 msgr2=0x555c812052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c812052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 -- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 msgr2=0x555c812057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.249+0000 7fac09101640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c812057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac09101640 1 -- 172.21.15.157:0/572413065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac041f9300 con 0x555c811fa690 2023-11-06T03:08:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac08900640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c812052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac09101640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fac000058e0 tx=0x7fac00000f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4502983646db4c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0006b150 con 0x555c811fa690 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fac0006ccc0 con 0x555c811fa690 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c81047a80 con 0x555c811fa690 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0006b2b0 con 0x555c811fa690 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 10 client.6387 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x555c8100e1c0 con 0x555c811fa690 2023-11-06T03:08:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c81047a80 con 0x555c811fa690 2023-11-06T03:08:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fac0005b7c0 con 0x555c811fa690 2023-11-06T03:08:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 10 client.6387 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 10 client.6387 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 20 client.6387 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c8124b0f0 0x555c8124b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 4 client.6387 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c81007a30 con 0x555c8124b0f0 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac08900640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c8124b0f0 0x555c8124b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac08900640 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c8124b0f0 0x555c8124b4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9d61b58c7f773a server_cookie=3b6c50319a5984b5 in_seq=0 out_seq=0 2023-11-06T03:08:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 10 client.6387 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 1 -- 172.21.15.157:0/572413065 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555c81007a30 con 0x555c8124b0f0 2023-11-06T03:08:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 10 client.6387 handle_command_reply: tid=0 2023-11-06T03:08:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 client.6387 shutdown 2023-11-06T03:08:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 msgr2=0x555c811ff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fac000058e0 tx=0x7fac00000f30 comp rx=0 tx=0).stop 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 shutdown_connections 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c8124b0f0 0x555c8124b4d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c812057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c812052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fabe3fff640 0 client.6387 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 --2- 172.21.15.157:0/572413065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c811ff4f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 >> 172.21.15.157:0/572413065 conn(0x555c812007f0 msgr2=0x555c811f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 shutdown_connections 2023-11-06T03:08:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 1 -- 172.21.15.157:0/572413065 wait complete. 2023-11-06T03:08:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.253+0000 7fac0e48d000 20 client.6387 trim_cache size 0 max 16384 2023-11-06T03:08:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.265+0000 7fac0e48d000 1 Processor -- start 2023-11-06T03:08:18.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.265+0000 7fac0e48d000 1 -- start start 2023-11-06T03:08:18.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c8129c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c8129c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c80d06b30 con 0x555c811fa690 2023-11-06T03:08:18.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c80fe85c0 con 0x555c812445d0 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c8104bc90 con 0x555c81243cb0 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35854/0 (socket says 172.21.15.157:35854) 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1925779281 learned_addr learned my addr 172.21.15.157:0/1925779281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 msgr2=0x555c8129c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c8129c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 msgr2=0x555c8129c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c8129c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1925779281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c80ff4510 con 0x555c811fa690 2023-11-06T03:08:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ce40 tx=0x7fac0006a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ae975eefee1ae0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabef7fe640 1 -- 172.21.15.157:0/1925779281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00050060 con 0x555c811fa690 2023-11-06T03:08:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabef7fe640 1 -- 172.21.15.157:0/1925779281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac00053680 con 0x555c811fa690 2023-11-06T03:08:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabef7fe640 1 -- 172.21.15.157:0/1925779281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00057aa0 con 0x555c811fa690 2023-11-06T03:08:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1925779281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 msgr2=0x555c8129bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ce40 tx=0x7fac0006a040 comp rx=0 tx=0).stop 2023-11-06T03:08:18.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1925779281 shutdown_connections 2023-11-06T03:08:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c812445d0 0x555c8129c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c81243cb0 0x555c8129c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1925779281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c8129bb10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1925779281 >> 172.21.15.157:0/1925779281 conn(0x555c812007f0 msgr2=0x555c81242260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1925779281 shutdown_connections 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1925779281 wait complete. 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- start start 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 Processor -- start 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c81243cb0 0x555c812475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c812445d0 0x555c81247af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c81044aa0 con 0x555c811fa690 2023-11-06T03:08:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c81047fc0 con 0x555c81243cb0 2023-11-06T03:08:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c81047c20 con 0x555c812445d0 2023-11-06T03:08:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 10 client.0 _ll_register_callbacks cb 0x555c811c6610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35868/0 (socket says 172.21.15.157:35868) 2023-11-06T03:08:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1112503739 learned_addr learned my addr 172.21.15.157:0/1112503739 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac08900640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c81243cb0 0x555c812475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c812445d0 msgr2=0x555c81247af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c812445d0 0x555c81247af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c81243cb0 msgr2=0x555c812475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c81243cb0 0x555c812475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 -- 172.21.15.157:0/1112503739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c8100e1c0 con 0x555c811fa690 2023-11-06T03:08:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac09101640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fac00050220 tx=0x7fac0000c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bf2f71fe8e6848 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00071da0 con 0x555c811fa690 2023-11-06T03:08:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fac0004fb10 con 0x555c811fa690 2023-11-06T03:08:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c810171f0 con 0x555c811fa690 2023-11-06T03:08:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0004fcd0 con 0x555c811fa690 2023-11-06T03:08:18.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 10 client.6393 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.269+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x555c80fe8080 con 0x555c811fa690 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c810171f0 con 0x555c811fa690 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fac00052c80 con 0x555c811fa690 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 10 client.6393 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 10 client.6393 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:18.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 20 client.6393 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c81248030 0x555c812a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 4 client.6393 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/"}] 2023-11-06T03:08:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x555c810421b0 con 0x555c81248030 2023-11-06T03:08:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac08900640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c81248030 0x555c812a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 10 client.6393 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac08900640 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c81248030 0x555c812a8db0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6063e656fbc0839e server_cookie=dfc0ce22573ab37a in_seq=0 out_seq=0 2023-11-06T03:08:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 1 -- 172.21.15.157:0/1112503739 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c810421b0 con 0x555c81248030 2023-11-06T03:08:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 10 client.6393 handle_command_reply: tid=0 2023-11-06T03:08:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 client.6393 shutdown 2023-11-06T03:08:18.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 msgr2=0x555c81247070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fac00050220 tx=0x7fac0000c050 comp rx=0 tx=0).stop 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 shutdown_connections 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x555c81248030 0x555c812a8db0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555c812445d0 0x555c81247af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555c81243cb0 0x555c812475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fabe3fff640 0 client.6393 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 --2- 172.21.15.157:0/1112503739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c811fa690 0x555c81247070 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 >> 172.21.15.157:0/1112503739 conn(0x555c812007f0 msgr2=0x555c811fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 shutdown_connections 2023-11-06T03:08:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 1 -- 172.21.15.157:0/1112503739 wait complete. 2023-11-06T03:08:18.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0e48d000 20 client.6393 trim_cache size 0 max 16384 2023-11-06T03:08:18.293 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:18.293 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:18.293 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "d0be7f40-8c01-4957-a75a-f347300e6c26", 2023-11-06T03:08:18.293 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:18.294 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 msgr2=0x7fabe8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 0x7fabe8067f20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fabf40ae000 tx=0x7fabf4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:18.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 msgr2=0x7fac041f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fac0000c480 tx=0x7fac0000c940 comp rx=0 tx=0).stop 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 shutdown_connections 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe8065a60 0x7fabe8067f20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac040cefc0 0x7fac04152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac04070880 0x7fac0414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 --2- 172.21.15.157:0/2989378404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac04070420 0x7fac041f86d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 >> 172.21.15.157:0/2989378404 conn(0x7fac0406a4d0 msgr2=0x7fac041558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 shutdown_connections 2023-11-06T03:08:18.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.273+0000 7fac0b365640 1 -- 172.21.15.157:0/2989378404 wait complete. 2023-11-06T03:08:18.297 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'd0be7f40-8c01-4957-a75a-f347300e6c26', 'mode': 'asynchronous'}' 2023-11-06T03:08:18.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:18.298 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2023-11-06T03:08:18.305 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2023-11-06T03:08:18.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:18.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.449+0000 7f4be4beb640 1 Processor -- start 2023-11-06T03:08:18.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.449+0000 7f4be4beb640 1 -- start start 2023-11-06T03:08:18.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4be4beb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00d0280 0x7f4be0167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4be4beb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00d2920 con 0x7f4be015ce80 2023-11-06T03:08:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4be4beb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00d18a0 con 0x7f4be015caa0 2023-11-06T03:08:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4be4beb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be00d1b60 con 0x7f4be00d0280 2023-11-06T03:08:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00d0280 0x7f4be0167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4be00d0280 0x7f4be0167860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35880/0 (socket says 172.21.15.157:35880) 2023-11-06T03:08:18.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 -- 172.21.15.157:0/1488187780 learned_addr learned my addr 172.21.15.157:0/1488187780 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 -- 172.21.15.157:0/1488187780 >> v1:172.21.15.195:6789/0 conn(0x7f4be015caa0 legacy=0x7f4be00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 -- 172.21.15.157:0/1488187780 >> v1:172.21.15.157:6789/0 conn(0x7f4be015ce80 legacy=0x7f4be00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 -- 172.21.15.157:0/1488187780 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00d1e40 con 0x7f4be00d0280 2023-11-06T03:08:18.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bdd59a640 1 -- 172.21.15.157:0/1488187780 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317006282 0 0) 0x7f4be00d2920 con 0x7f4be015ce80 2023-11-06T03:08:18.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bded9d640 1 --2- 172.21.15.157:0/1488187780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be0167860 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009e50 tx=0x7f4bd801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c88e7cf6e3e2a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bdd59a640 1 -- 172.21.15.157:0/1488187780 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5860 con 0x7f4be00d0280 2023-11-06T03:08:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bdd59a640 1 -- 172.21.15.157:0/1488187780 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd80a59c0 con 0x7f4be00d0280 2023-11-06T03:08:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.453+0000 7f4bdd59a640 1 -- 172.21.15.157:0/1488187780 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5d50 con 0x7f4be00d0280 2023-11-06T03:08:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/1488187780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 msgr2=0x7f4be0167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 --2- 172.21.15.157:0/1488187780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be0167860 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009e50 tx=0x7f4bd801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/1488187780 shutdown_connections 2023-11-06T03:08:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 --2- 172.21.15.157:0/1488187780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be0167860 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/1488187780 >> 172.21.15.157:0/1488187780 conn(0x7f4be0070790 msgr2=0x7f4be0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/1488187780 shutdown_connections 2023-11-06T03:08:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/1488187780 wait complete. 2023-11-06T03:08:18.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 Processor -- start 2023-11-06T03:08:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- start start 2023-11-06T03:08:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be015caa0 0x7f4be01554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be015ce80 0x7f4be0155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0151c90 con 0x7f4be00d0280 2023-11-06T03:08:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00cbc90 con 0x7f4be015ce80 2023-11-06T03:08:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0151f30 con 0x7f4be015caa0 2023-11-06T03:08:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35892/0 (socket says 172.21.15.157:35892) 2023-11-06T03:08:18.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 -- 172.21.15.157:0/3038982960 learned_addr learned my addr 172.21.15.157:0/3038982960 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 -- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be015caa0 msgr2=0x7f4be01554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be015caa0 0x7f4be01554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 -- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be015ce80 msgr2=0x7f4be0155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be015ce80 0x7f4be0155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 -- 172.21.15.157:0/3038982960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8009b30 con 0x7f4be00d0280 2023-11-06T03:08:18.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bde59c640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c450 tx=0x7f4bd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0012830 con 0x7f4be00d0280 2023-11-06T03:08:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd0012990 con 0x7f4be00d0280 2023-11-06T03:08:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00cbf30 con 0x7f4be00d0280 2023-11-06T03:08:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0012d40 con 0x7f4be00d0280 2023-11-06T03:08:18.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be0152210 con 0x7f4be00d0280 2023-11-06T03:08:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.457+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be00cbaf0 con 0x7f4be00d0280 2023-11-06T03:08:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.461+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4bd001c460 con 0x7f4be00d0280 2023-11-06T03:08:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.461+0000 7f4bceffd640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 0x7f4bc0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:18.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.461+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4bd0083110 con 0x7f4be00d0280 2023-11-06T03:08:18.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.461+0000 7f4bddd9b640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 0x7f4bc0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:18.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.461+0000 7f4bddd9b640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 0x7f4bc0067f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b7530 tx=0x7f4bd4007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:18.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.465+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4be00cbaf0 con 0x7f4be00d0280 2023-11-06T03:08:18.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.637+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be00cbaf0 con 0x7f4be00d0280 2023-11-06T03:08:18.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.637+0000 7f4bceffd640 1 -- 172.21.15.157:0/3038982960 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f4be00cbaf0 con 0x7f4be00d0280 2023-11-06T03:08:18.638 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:18.639 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:16.389043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:18.639 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-11-06T03:08:18.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 msgr2=0x7f4bc0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 0x7f4bc0067f50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b7530 tx=0x7f4bd4007940 comp rx=0 tx=0).stop 2023-11-06T03:08:18.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 msgr2=0x7f4be01513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c450 tx=0x7f4bd000c910 comp rx=0 tx=0).stop 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 shutdown_connections 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4bc0065a90 0x7f4bc0067f50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be015ce80 0x7f4be0155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4be015caa0 0x7f4be01554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 --2- 172.21.15.157:0/3038982960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4be00d0280 0x7f4be01513b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:18.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 >> 172.21.15.157:0/3038982960 conn(0x7f4be0070790 msgr2=0x7f4be0070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:18.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 shutdown_connections 2023-11-06T03:08:18.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:18.641+0000 7f4be4beb640 1 -- 172.21.15.157:0/3038982960 wait complete. 2023-11-06T03:08:18.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2023-11-06T03:08:18.769 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:18.769 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:18.769 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:18.782 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:18.782 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:18.783 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:18.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:18.789 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2023-11-06T03:08:18.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:08:18.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:08:18.849 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012436 s, 84.3 MB/s 2023-11-06T03:08:18.850 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2023-11-06T03:08:18.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:19.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.013+0000 7f2becbf8640 1 Processor -- start 2023-11-06T03:08:19.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.013+0000 7f2becbf8640 1 -- start start 2023-11-06T03:08:19.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2becbf8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2be80b3b20 0x7f2be815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2becbf8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be815f580 con 0x7f2be81551e0 2023-11-06T03:08:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2becbf8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be815efc0 con 0x7f2be815e5e0 2023-11-06T03:08:19.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2becbf8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80704b0 con 0x7f2be80b3b20 2023-11-06T03:08:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2be81551e0 0x7f2be806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32852/0 (socket says 172.21.15.157:32852) 2023-11-06T03:08:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be659c640 1 -- 172.21.15.157:0/2395789371 learned_addr learned my addr 172.21.15.157:0/2395789371 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:19.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be6d9d640 1 --2- 172.21.15.157:0/2395789371 >> v2:172.21.15.157:3300/0 conn(0x7f2be80b3b20 0x7f2be815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881619029 0 0) 0x7f2be815f580 con 0x7f2be81551e0 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd0003530 con 0x7f2be81551e0 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be6d9d640 1 -- 172.21.15.157:0/2395789371 >> v1:172.21.15.195:6789/0 conn(0x7f2be815e5e0 legacy=0x7f2be80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be6d9d640 1 -- 172.21.15.157:0/2395789371 >> v1:172.21.15.157:6789/0 conn(0x7f2be81551e0 legacy=0x7f2be806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2590813357 0 0) 0x7f2bd0003530 con 0x7f2be81551e0 2023-11-06T03:08:19.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be6d9d640 1 -- 172.21.15.157:0/2395789371 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be815f2a0 con 0x7f2be80b3b20 2023-11-06T03:08:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be6d9d640 1 --2- 172.21.15.157:0/2395789371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be80b3b20 0x7f2be815c1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2be00097c0 tx=0x7f2be001c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16abc99c75fc3a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8001a00 con 0x7f2be80b3b20 2023-11-06T03:08:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00a7050 con 0x7f2be80b3b20 2023-11-06T03:08:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.017+0000 7f2be559a640 1 -- 172.21.15.157:0/2395789371 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00ab620 con 0x7f2be80b3b20 2023-11-06T03:08:19.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/2395789371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be80b3b20 msgr2=0x7f2be815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 --2- 172.21.15.157:0/2395789371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be80b3b20 0x7f2be815c1b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2be00097c0 tx=0x7f2be001c250 comp rx=0 tx=0).stop 2023-11-06T03:08:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/2395789371 shutdown_connections 2023-11-06T03:08:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 --2- 172.21.15.157:0/2395789371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be80b3b20 0x7f2be815c1b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/2395789371 >> 172.21.15.157:0/2395789371 conn(0x7f2be8065930 msgr2=0x7f2be8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/2395789371 shutdown_connections 2023-11-06T03:08:19.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/2395789371 wait complete. 2023-11-06T03:08:19.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 Processor -- start 2023-11-06T03:08:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- start start 2023-11-06T03:08:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80b3b20 0x7f2be814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815e5e0 0x7f2be814c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be814b160 con 0x7f2be81551e0 2023-11-06T03:08:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be814b390 con 0x7f2be815e5e0 2023-11-06T03:08:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be814b670 con 0x7f2be80b3b20 2023-11-06T03:08:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58426/0 (socket says 172.21.15.157:58426) 2023-11-06T03:08:19.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 -- 172.21.15.157:0/15926787 learned_addr learned my addr 172.21.15.157:0/15926787 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be6d9d640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815e5e0 0x7f2be814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be659c640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80b3b20 0x7f2be814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 -- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80b3b20 msgr2=0x7f2be814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80b3b20 0x7f2be814ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 -- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815e5e0 msgr2=0x7f2be814c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815e5e0 0x7f2be814c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 -- 172.21.15.157:0/15926787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009420 con 0x7f2be81551e0 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2be5d9b640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c500 tx=0x7f2bdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc005220 con 0x7f2be81551e0 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc005380 con 0x7f2be81551e0 2023-11-06T03:08:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be814b950 con 0x7f2be81551e0 2023-11-06T03:08:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc007270 con 0x7f2be81551e0 2023-11-06T03:08:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be814bf10 con 0x7f2be81551e0 2023-11-06T03:08:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.021+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be814c130 con 0x7f2be81551e0 2023-11-06T03:08:19.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.025+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2bdc0073d0 con 0x7f2be81551e0 2023-11-06T03:08:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.025+0000 7f2bd6ffd640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 0x7f2bc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.025+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2bdc082bf0 con 0x7f2be81551e0 2023-11-06T03:08:19.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.025+0000 7f2be659c640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 0x7f2bc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.025+0000 7f2be659c640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 0x7f2bc8067f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8002240 tx=0x7f2bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.029+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2bdc050420 con 0x7f2be81551e0 2023-11-06T03:08:19.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.213+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2be8072750 con 0x7f2be81551e0 2023-11-06T03:08:19.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.213+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/15926787 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f2bdc053a40 con 0x7f2be81551e0 2023-11-06T03:08:19.213 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:19.213 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:16.389043+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":12,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:19.214 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 38 2023-11-06T03:08:19.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 msgr2=0x7f2bc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 0x7f2bc8067f20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8002240 tx=0x7f2bd8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 msgr2=0x7f2be80cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c500 tx=0x7f2bdc00c9c0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 shutdown_connections 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bc8065a60 0x7f2bc8067f20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2be815e5e0 0x7f2be814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2be81551e0 0x7f2be80cdde0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 --2- 172.21.15.157:0/15926787 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2be80b3b20 0x7f2be814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 >> 172.21.15.157:0/15926787 conn(0x7f2be8065930 msgr2=0x7f2be8065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 shutdown_connections 2023-11-06T03:08:19.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.217+0000 7f2becbf8640 1 -- 172.21.15.157:0/15926787 wait complete. 2023-11-06T03:08:19.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2023-11-06T03:08:19.346 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:19.347 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:19.347 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:19.360 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:19.360 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:08:19.361 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:08:19.361 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:08:19.361 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:19.362 DEBUG:teuthology.orchestra.run.smithi157:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2023-11-06T03:08:19.362 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:08:19.422 INFO:teuthology.orchestra.run.smithi157.stdout:1099511630411 2023-11-06T03:08:19.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-06T03:08:19.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.773+0000 7fe069ad7640 1 Processor -- start 2023-11-06T03:08:19.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 -- start start 2023-11-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe06415caa0 0x7fe0640cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415ce80 0x7fe0640cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0640d2920 con 0x7fe0640ce1a0 2023-11-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0640d18a0 con 0x7fe06415caa0 2023-11-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe069ad7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640d1b60 con 0x7fe06415ce80 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe06415caa0 0x7fe0640cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe06415caa0 0x7fe0640cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58440/0 (socket says 172.21.15.157:58440) 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 -- 172.21.15.157:0/2979568227 learned_addr learned my addr 172.21.15.157:0/2979568227 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 -- 172.21.15.157:0/2979568227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415ce80 msgr2=0x7fe0640cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe05affd640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415ce80 0x7fe0640cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415ce80 0x7fe0640cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 -- 172.21.15.157:0/2979568227 >> v1:172.21.15.195:6789/0 conn(0x7fe0640ce1a0 legacy=0x7fe0640d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 -- 172.21.15.157:0/2979568227 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640d1e40 con 0x7fe06415caa0 2023-11-06T03:08:19.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe0637fe640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06415caa0 0x7fe0640cdc60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe054009870 tx=0x7fe05401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfef905992209a16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.777+0000 7fe062ffd640 1 -- 172.21.15.157:0/2979568227 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540ac600 con 0x7fe06415caa0 2023-11-06T03:08:19.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe062ffd640 1 -- 172.21.15.157:0/2979568227 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0540a8050 con 0x7fe06415caa0 2023-11-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe062ffd640 1 -- 172.21.15.157:0/2979568227 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540ac980 con 0x7fe06415caa0 2023-11-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2979568227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06415caa0 msgr2=0x7fe0640cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06415caa0 0x7fe0640cdc60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe054009870 tx=0x7fe05401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2979568227 shutdown_connections 2023-11-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415ce80 0x7fe0640cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2979568227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe06415caa0 0x7fe0640cdc60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2979568227 >> 172.21.15.157:0/2979568227 conn(0x7fe064070790 msgr2=0x7fe064070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2979568227 shutdown_connections 2023-11-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2979568227 wait complete. 2023-11-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 Processor -- start 2023-11-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- start start 2023-11-06T03:08:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415caa0 0x7fe0640c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe06415ce80 0x7fe06414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640c9290 con 0x7fe0640ce1a0 2023-11-06T03:08:19.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe064155040 con 0x7fe06415ce80 2023-11-06T03:08:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe064155320 con 0x7fe06415caa0 2023-11-06T03:08:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58450/0 (socket says 172.21.15.157:58450) 2023-11-06T03:08:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 -- 172.21.15.157:0/2404271858 learned_addr learned my addr 172.21.15.157:0/2404271858 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 -- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415caa0 msgr2=0x7fe0640c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe05affd640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415caa0 0x7fe0640c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe063fff640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe06415ce80 0x7fe06414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415caa0 0x7fe0640c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 -- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe06415ce80 msgr2=0x7fe06414c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe06415ce80 0x7fe06414c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 -- 172.21.15.157:0/2404271858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540094d0 con 0x7fe0640ce1a0 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe0637fe640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe05401cd70 tx=0x7fe05401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe060ff9640 1 -- 172.21.15.157:0/2404271858 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540b7620 con 0x7fe0640ce1a0 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe060ff9640 1 -- 172.21.15.157:0/2404271858 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0540b77a0 con 0x7fe0640ce1a0 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe064155600 con 0x7fe0640ce1a0 2023-11-06T03:08:19.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe060ff9640 1 -- 172.21.15.157:0/2404271858 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0540b7b50 con 0x7fe0640ce1a0 2023-11-06T03:08:19.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.781+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe064155bc0 con 0x7fe0640ce1a0 2023-11-06T03:08:19.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe060ff9640 1 -- 172.21.15.157:0/2404271858 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe0540b3040 con 0x7fe0640ce1a0 2023-11-06T03:08:19.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe060ff9640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 0x7fe044067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe060ff9640 1 -- 172.21.15.157:0/2404271858 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe05411d850 con 0x7fe0640ce1a0 2023-11-06T03:08:19.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe05affd640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 0x7fe044067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 Processor -- start 2023-11-06T03:08:19.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 -- start start 2023-11-06T03:08:19.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe05affd640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 0x7fe044067f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe06415c7d0 tx=0x7fe04c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3ce2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce3530 0x559ce3c98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3b60d60 con 0x559ce3c98f90 2023-11-06T03:08:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3aef3b0 con 0x559ce3ce3530 2023-11-06T03:08:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce3aec980 con 0x559ce3ce2c10 2023-11-06T03:08:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58464/0 (socket says 172.21.15.157:58464) 2023-11-06T03:08:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 -- 172.21.15.157:0/1277926294 learned_addr learned my addr 172.21.15.157:0/1277926294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe05affd640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce3530 0x559ce3c98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3ce2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 -- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 msgr2=0x559ce3ce2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3ce2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 -- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce3530 msgr2=0x559ce3c98a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce3530 0x559ce3c98a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 -- 172.21.15.157:0/1277926294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce3a89110 con 0x559ce3c98f90 2023-11-06T03:08:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3ce2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:19.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe063fff640 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007d60 tx=0x7fe05c00c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32aad47010526ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe0597fa640 1 -- 172.21.15.157:0/1277926294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00a070 con 0x559ce3c98f90 2023-11-06T03:08:19.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe0597fa640 1 -- 172.21.15.157:0/1277926294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00ce70 con 0x559ce3c98f90 2023-11-06T03:08:19.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.785+0000 7fe0597fa640 1 -- 172.21.15.157:0/1277926294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c011630 con 0x559ce3c98f90 2023-11-06T03:08:19.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/1277926294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 msgr2=0x559ce3c99370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007d60 tx=0x7fe05c00c540 comp rx=0 tx=0).stop 2023-11-06T03:08:19.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/1277926294 shutdown_connections 2023-11-06T03:08:19.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3c98f90 0x559ce3c99370 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce3530 0x559ce3c98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1277926294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3ce2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/1277926294 >> 172.21.15.157:0/1277926294 conn(0x559ce3c94f40 msgr2=0x559ce3c97330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/1277926294 shutdown_connections 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/1277926294 wait complete. 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- start start 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 Processor -- start 2023-11-06T03:08:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3c9dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3c9e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3ae8270 con 0x559ce3ce3530 2023-11-06T03:08:19.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3a875a0 con 0x559ce3c98f90 2023-11-06T03:08:19.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce3aa57c0 con 0x559ce3ce2c10 2023-11-06T03:08:19.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 10 client.0 _ll_register_callbacks cb 0x559ce3adc130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:19.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58480/0 (socket says 172.21.15.157:58480) 2023-11-06T03:08:19.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 -- 172.21.15.157:0/3309058092 learned_addr learned my addr 172.21.15.157:0/3309058092 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0637fe640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3c9dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe05affd640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3c9e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 -- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 msgr2=0x559ce3c9e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3c9e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 -- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 msgr2=0x559ce3c9dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3c9dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 -- 172.21.15.157:0/3309058092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c007990 con 0x559ce3ce3530 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe063fff640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c580 tx=0x7fe05c004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707308f15ed5e5e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c01f070 con 0x559ce3ce3530 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe05c01a750 con 0x559ce3ce3530 2023-11-06T03:08:19.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce3aac350 con 0x559ce3ce3530 2023-11-06T03:08:19.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe05c01a910 con 0x559ce3ce3530 2023-11-06T03:08:19.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 10 client.6423 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:19.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x559ce37e4e20 con 0x559ce3ce3530 2023-11-06T03:08:19.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ce3aac350 con 0x559ce3ce3530 2023-11-06T03:08:19.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe05c01aa70 con 0x559ce3ce3530 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 10 client.6423 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 10 client.6423 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 20 client.6423 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3cd18f0 0x559ce3cf4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 4 client.6423 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:19.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ce3ae6bc0 con 0x559ce3cd18f0 2023-11-06T03:08:19.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0637fe640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3cd18f0 0x559ce3cf4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0637fe640 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3cd18f0 0x559ce3cf4cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9626f1d355038733 server_cookie=9e0bc2a7b404aa39 in_seq=0 out_seq=0 2023-11-06T03:08:19.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.789+0000 7fe0427fc640 10 client.6423 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:19.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe0427fc640 1 -- 172.21.15.157:0/3309058092 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559ce3ae6bc0 con 0x559ce3cd18f0 2023-11-06T03:08:19.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe0427fc640 10 client.6423 handle_command_reply: tid=0 2023-11-06T03:08:19.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 client.6423 shutdown 2023-11-06T03:08:19.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 msgr2=0x559ce3cd0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c580 tx=0x7fe05c004a10 comp rx=0 tx=0).stop 2023-11-06T03:08:19.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 shutdown_connections 2023-11-06T03:08:19.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3cd18f0 0x559ce3cf4cc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3cd0ee0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3c9e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 --2- 172.21.15.157:0/3309058092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3c9dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe0427fc640 0 client.6423 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 >> 172.21.15.157:0/3309058092 conn(0x559ce3c94f40 msgr2=0x559ce3c97bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 shutdown_connections 2023-11-06T03:08:19.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 1 -- 172.21.15.157:0/3309058092 wait complete. 2023-11-06T03:08:19.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.793+0000 7fe06cbff000 20 client.6423 trim_cache size 0 max 16384 2023-11-06T03:08:19.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 Processor -- start 2023-11-06T03:08:19.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 -- start start 2023-11-06T03:08:19.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3c98f90 0x559ce3cf0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce2c10 0x559ce3ccfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce37a5aa0 con 0x559ce3ce3530 2023-11-06T03:08:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3ae5670 con 0x559ce3ce2c10 2023-11-06T03:08:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce3c9f580 con 0x559ce3c98f90 2023-11-06T03:08:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58482/0 (socket says 172.21.15.157:58482) 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.801+0000 7fe063fff640 1 -- 172.21.15.157:0/979431567 learned_addr learned my addr 172.21.15.157:0/979431567 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3c98f90 msgr2=0x559ce3cf0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3c98f90 0x559ce3cf0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce2c10 msgr2=0x559ce3ccfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe05affd640 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce2c10 0x559ce3ccfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce2c10 0x559ce3ccfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/979431567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce3aad1a0 con 0x559ce3ce3530 2023-11-06T03:08:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007c30 tx=0x7fe05c01c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40df6ac914648d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe059ffb640 1 -- 172.21.15.157:0/979431567 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00ca40 con 0x559ce3ce3530 2023-11-06T03:08:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe059ffb640 1 -- 172.21.15.157:0/979431567 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c025420 con 0x559ce3ce3530 2023-11-06T03:08:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe059ffb640 1 -- 172.21.15.157:0/979431567 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c003980 con 0x559ce3ce3530 2023-11-06T03:08:19.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/979431567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 msgr2=0x559ce3c98340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007c30 tx=0x7fe05c01c050 comp rx=0 tx=0).stop 2023-11-06T03:08:19.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/979431567 shutdown_connections 2023-11-06T03:08:19.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3c98340 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3ce2c10 0x559ce3ccfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- 172.21.15.157:0/979431567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3c98f90 0x559ce3cf0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/979431567 >> 172.21.15.157:0/979431567 conn(0x559ce3c94f40 msgr2=0x559ce3cef970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/979431567 shutdown_connections 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/979431567 wait complete. 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- start start 2023-11-06T03:08:19.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 Processor -- start 2023-11-06T03:08:19.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3d3ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3d3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3aeadd0 con 0x559ce3ce3530 2023-11-06T03:08:19.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce3ae3be0 con 0x559ce3c98f90 2023-11-06T03:08:19.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce3ae5a80 con 0x559ce3ce2c10 2023-11-06T03:08:19.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 10 client.0 _ll_register_callbacks cb 0x559ce3a96780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:19.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58490/0 (socket says 172.21.15.157:58490) 2023-11-06T03:08:19.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/1993191439 learned_addr learned my addr 172.21.15.157:0/1993191439 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe05affd640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3d3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3d3ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 msgr2=0x559ce3d3f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3d3f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 msgr2=0x559ce3d3ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3d3ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 -- 172.21.15.157:0/1993191439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ce37e4e20 con 0x559ce3ce3530 2023-11-06T03:08:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3d3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe063fff640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007f90 tx=0x7fe05c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c8672120b7747a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c01a630 con 0x559ce3ce3530 2023-11-06T03:08:19.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe05c01a790 con 0x559ce3ce3530 2023-11-06T03:08:19.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce3a97c20 con 0x559ce3ce3530 2023-11-06T03:08:19.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe05c01a950 con 0x559ce3ce3530 2023-11-06T03:08:19.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 10 client.6429 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:19.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x559ce3a934f0 con 0x559ce3ce3530 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ce3a97c20 con 0x559ce3ce3530 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe05c02d070 con 0x559ce3ce3530 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 10 client.6429 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 10 client.6429 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 20 client.6429 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:19.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3d3bbf0 0x559ce3d3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 4 client.6429 mds_command: new command op to 5868 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:08:19.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ce3ae12f0 con 0x559ce3d3bbf0 2023-11-06T03:08:19.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.805+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3d3bbf0 0x559ce3d3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe0427fc640 10 client.6429 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:19.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe0637fe640 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3d3bbf0 0x559ce3d3bfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a51f0ccca11429 server_cookie=f2896182689fc23 in_seq=0 out_seq=0 2023-11-06T03:08:19.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe0427fc640 1 -- 172.21.15.157:0/1993191439 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ce3ae12f0 con 0x559ce3d3bbf0 2023-11-06T03:08:19.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe0427fc640 10 client.6429 handle_command_reply: tid=0 2023-11-06T03:08:19.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 client.6429 shutdown 2023-11-06T03:08:19.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 msgr2=0x559ce3d3f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe05c007f90 tx=0x7fe05c007650 comp rx=0 tx=0).stop 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 shutdown_connections 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559ce3d3bbf0 0x559ce3d3bfd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559ce3ce3530 0x559ce3d3f7b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559ce3ce2c10 0x559ce3d3f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 --2- 172.21.15.157:0/1993191439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559ce3c98f90 0x559ce3d3ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe0427fc640 0 client.6429 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:19.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 >> 172.21.15.157:0/1993191439 conn(0x559ce3c94f40 msgr2=0x559ce3c97330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 shutdown_connections 2023-11-06T03:08:19.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 1 -- 172.21.15.157:0/1993191439 wait complete. 2023-11-06T03:08:19.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe06cbff000 20 client.6429 trim_cache size 0 max 16384 2023-11-06T03:08:19.831 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2023-11-06T03:08:19.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 msgr2=0x7fe044067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 0x7fe044067f20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe06415c7d0 tx=0x7fe04c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:19.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 msgr2=0x7fe0640ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe05401cd70 tx=0x7fe05401ad10 comp rx=0 tx=0).stop 2023-11-06T03:08:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 shutdown_connections 2023-11-06T03:08:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe044065a60 0x7fe044067f20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe06415ce80 0x7fe06414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe06415caa0 0x7fe0640c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 --2- 172.21.15.157:0/2404271858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0640ce1a0 0x7fe0640ca890 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 >> 172.21.15.157:0/2404271858 conn(0x7fe064070790 msgr2=0x7fe064070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 shutdown_connections 2023-11-06T03:08:19.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.809+0000 7fe069ad7640 1 -- 172.21.15.157:0/2404271858 wait complete. 2023-11-06T03:08:19.834 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4b.00000000 parent 2023-11-06T03:08:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 Processor -- start 2023-11-06T03:08:19.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 -- start start 2023-11-06T03:08:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9340003ba0 0x5642af5e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5642af5574b0 0x5642af678400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af4b7940 con 0x5642af53bae0 2023-11-06T03:08:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af453450 con 0x7f9340003ba0 2023-11-06T03:08:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f935aea8080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642af453150 con 0x5642af5574b0 2023-11-06T03:08:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58512/0 (socket says 172.21.15.157:58512) 2023-11-06T03:08:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 -- 172.21.15.157:0/4091326392 learned_addr learned my addr 172.21.15.157:0/4091326392 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93581c1640 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5642af5574b0 0x5642af678400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93589c2640 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9340003ba0 0x5642af5e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 -- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5642af5574b0 msgr2=0x5642af678400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5642af5574b0 0x5642af678400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 -- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9340003ba0 msgr2=0x5642af5e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9340003ba0 0x5642af5e5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 -- 172.21.15.157:0/4091326392 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642af455430 con 0x5642af53bae0 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93591c3640 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f935001c120 tx=0x7f9350001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a669aa2a2d851e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93571bf640 1 -- 172.21.15.157:0/4091326392 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350001800 con 0x5642af53bae0 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93571bf640 1 -- 172.21.15.157:0/4091326392 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9350001960 con 0x5642af53bae0 2023-11-06T03:08:19.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.865+0000 7f93571bf640 1 -- 172.21.15.157:0/4091326392 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500a8480 con 0x5642af53bae0 2023-11-06T03:08:19.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/4091326392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 msgr2=0x5642af678940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f935001c120 tx=0x7f9350001d90 comp rx=0 tx=0).stop 2023-11-06T03:08:19.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/4091326392 shutdown_connections 2023-11-06T03:08:19.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af678940 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5642af5574b0 0x5642af678400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- 172.21.15.157:0/4091326392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9340003ba0 0x5642af5e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/4091326392 >> 172.21.15.157:0/4091326392 conn(0x5642af539060 msgr2=0x5642af5e28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/4091326392 shutdown_connections 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/4091326392 wait complete. 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 Processor -- start 2023-11-06T03:08:19.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- start start 2023-11-06T03:08:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af5dce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642af5574b0 0x5642af5c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9340003ba0 0x5642af5c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af449dc0 con 0x5642af53bae0 2023-11-06T03:08:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642af50c4a0 con 0x5642af5574b0 2023-11-06T03:08:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642af5c7440 con 0x7f9340003ba0 2023-11-06T03:08:19.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af5dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af5dce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58520/0 (socket says 172.21.15.157:58520) 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 -- 172.21.15.157:0/1213197156 learned_addr learned my addr 172.21.15.157:0/1213197156 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 -- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9340003ba0 msgr2=0x5642af5c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9340003ba0 0x5642af5c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 -- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642af5574b0 msgr2=0x5642af5c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642af5574b0 0x5642af5c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 -- 172.21.15.157:0/1213197156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340001e20 con 0x5642af53bae0 2023-11-06T03:08:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93589c2640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af5dce70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9348007760 tx=0x7f9348007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93561bd640 1 -- 172.21.15.157:0/1213197156 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348006d80 con 0x5642af53bae0 2023-11-06T03:08:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93561bd640 1 -- 172.21.15.157:0/1213197156 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348015400 con 0x5642af53bae0 2023-11-06T03:08:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5642af454e90 con 0x5642af53bae0 2023-11-06T03:08:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93561bd640 1 -- 172.21.15.157:0/1213197156 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348006ee0 con 0x5642af53bae0 2023-11-06T03:08:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642af5c4110 con 0x5642af53bae0 2023-11-06T03:08:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93561bd640 1 -- 172.21.15.157:0/1213197156 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9348007d50 con 0x5642af53bae0 2023-11-06T03:08:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.869+0000 7f93561bd640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9344064410 0x7f93440668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93561bd640 1 -- 172.21.15.157:0/1213197156 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9348081c90 con 0x5642af53bae0 2023-11-06T03:08:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x5642af5dd3b0 0x5642af67cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 --> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] -- osd_op(unknown.0.0:1 8.24 8:256cebcc:::10000000a4b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v8 -- 0x5642af67d010 con 0x5642af5dd3b0 2023-11-06T03:08:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93581c1640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9344064410 0x7f93440668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93591c3640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x5642af5dd3b0 0x5642af67cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93581c1640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9344064410 0x7f93440668d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f934c002e40 tx=0x7f934c0023a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93591c3640 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x5642af5dd3b0 0x5642af67cad0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93591c3640 1 -- 172.21.15.157:0/1213197156 <== osd.6 v2:172.21.15.195:6817/38426 1 ==== osd_op_reply(1 10000000a4b.00000000 [rmxattr] v55'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9350006130 con 0x5642af5dd3b0 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x5642af5dd3b0 msgr2=0x5642af67cad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x5642af5dd3b0 0x5642af67cad0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9344064410 msgr2=0x7f93440668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9344064410 0x7f93440668d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f934c002e40 tx=0x7f934c0023a0 comp rx=0 tx=0).stop 2023-11-06T03:08:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 msgr2=0x5642af5dce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 --2- 172.21.15.157:0/1213197156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5642af53bae0 0x5642af5dce70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9348007760 tx=0x7f9348007c20 comp rx=0 tx=0).stop 2023-11-06T03:08:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f93591c3640 1 -- 172.21.15.157:0/1213197156 reap_dead start 2023-11-06T03:08:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 shutdown_connections 2023-11-06T03:08:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 >> 172.21.15.157:0/1213197156 conn(0x5642af539060 msgr2=0x5642af4fb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 shutdown_connections 2023-11-06T03:08:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:19.873+0000 7f935aea8080 1 -- 172.21.15.157:0/1213197156 wait complete. 2023-11-06T03:08:19.884 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2023-11-06T03:08:19.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2023-11-06T03:08:20.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.053+0000 7f118aeb7640 1 Processor -- start 2023-11-06T03:08:20.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.053+0000 7f118aeb7640 1 -- start start 2023-11-06T03:08:20.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f118415caa0 0x7f11840cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 0x7f11840cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11840d2920 con 0x7f11840ce1a0 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840d18a0 con 0x7f118415caa0 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11840d1b60 con 0x7f118415ce80 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f118415caa0 0x7f11840cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f118415caa0 0x7f11840cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58532/0 (socket says 172.21.15.157:58532) 2023-11-06T03:08:20.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 -- 172.21.15.157:0/1350817381 learned_addr learned my addr 172.21.15.157:0/1350817381 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1183fff640 1 --2- 172.21.15.157:0/1350817381 >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 0x7f11840cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 -- 172.21.15.157:0/1350817381 >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 msgr2=0x7f11840cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 --2- 172.21.15.157:0/1350817381 >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 0x7f11840cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 -- 172.21.15.157:0/1350817381 >> v1:172.21.15.195:6789/0 conn(0x7f11840ce1a0 legacy=0x7f11840d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 -- 172.21.15.157:0/1350817381 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840d1e40 con 0x7f118415caa0 2023-11-06T03:08:20.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1183fff640 1 --2- 172.21.15.157:0/1350817381 >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 0x7f11840cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:20.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f1188c53640 1 --2- 172.21.15.157:0/1350817381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f11840cdc60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f11840c6720 tx=0x7f117401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53108e9efbc5f2c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f11837fe640 1 -- 172.21.15.157:0/1350817381 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740ac600 con 0x7f118415caa0 2023-11-06T03:08:20.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f11837fe640 1 -- 172.21.15.157:0/1350817381 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11740a8050 con 0x7f118415caa0 2023-11-06T03:08:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f11837fe640 1 -- 172.21.15.157:0/1350817381 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740ac980 con 0x7f118415caa0 2023-11-06T03:08:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 -- 172.21.15.157:0/1350817381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 msgr2=0x7f11840cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.057+0000 7f118aeb7640 1 --2- 172.21.15.157:0/1350817381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f11840cdc60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f11840c6720 tx=0x7f117401c490 comp rx=0 tx=0).stop 2023-11-06T03:08:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/1350817381 shutdown_connections 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 --2- 172.21.15.157:0/1350817381 >> v2:172.21.15.195:3300/0 conn(0x7f118415ce80 0x7f11840cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 --2- 172.21.15.157:0/1350817381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f11840cdc60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/1350817381 >> 172.21.15.157:0/1350817381 conn(0x7f1184070790 msgr2=0x7f1184070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/1350817381 shutdown_connections 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/1350817381 wait complete. 2023-11-06T03:08:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 Processor -- start 2023-11-06T03:08:20.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- start start 2023-11-06T03:08:20.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11840ce1a0 0x7f1184150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f118415ce80 0x7f11840b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840b4280 con 0x7f118415caa0 2023-11-06T03:08:20.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840c72e0 con 0x7f118415ce80 2023-11-06T03:08:20.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1184156c60 con 0x7f11840ce1a0 2023-11-06T03:08:20.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58546/0 (socket says 172.21.15.157:58546) 2023-11-06T03:08:20.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 -- 172.21.15.157:0/3846023842 learned_addr learned my addr 172.21.15.157:0/3846023842 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1188c53640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11840ce1a0 0x7f1184150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1189454640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f118415ce80 0x7f11840b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 -- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11840ce1a0 msgr2=0x7f1184150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11840ce1a0 0x7f1184150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 -- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f118415ce80 msgr2=0x7f11840b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f118415ce80 0x7f11840b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 -- 172.21.15.157:0/3846023842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740094d0 con 0x7f118415caa0 2023-11-06T03:08:20.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f1183fff640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f117800c7f0 tx=0x7f117800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f11817fa640 1 -- 172.21.15.157:0/3846023842 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800cea0 con 0x7f118415caa0 2023-11-06T03:08:20.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f11817fa640 1 -- 172.21.15.157:0/3846023842 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178007a30 con 0x7f118415caa0 2023-11-06T03:08:20.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f11817fa640 1 -- 172.21.15.157:0/3846023842 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178007dc0 con 0x7f118415caa0 2023-11-06T03:08:20.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11840c9e20 con 0x7f118415caa0 2023-11-06T03:08:20.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.061+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840c7530 con 0x7f118415caa0 2023-11-06T03:08:20.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f11817fa640 1 -- 172.21.15.157:0/3846023842 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f11780113f0 con 0x7f118415caa0 2023-11-06T03:08:20.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f11817fa640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 0x7f1170067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f11817fa640 1 -- 172.21.15.157:0/3846023842 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1178083360 con 0x7f118415caa0 2023-11-06T03:08:20.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1188c53640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 0x7f1170067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1188c53640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 0x7f1170067f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11740ae000 tx=0x7f11740a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 Processor -- start 2023-11-06T03:08:20.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 -- start start 2023-11-06T03:08:20.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732c9d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732b65250 con 0x55b732ce7820 2023-11-06T03:08:20.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732af4220 con 0x55b732c9d300 2023-11-06T03:08:20.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b732af17f0 con 0x55b732ce6f00 2023-11-06T03:08:20.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58548/0 (socket says 172.21.15.157:58548) 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 -- 172.21.15.157:0/853751658 learned_addr learned my addr 172.21.15.157:0/853751658 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1188c53640 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1189454640 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732c9d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 -- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 msgr2=0x55b732ce72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 -- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 msgr2=0x55b732c9d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732c9d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 -- 172.21.15.157:0/853751658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b732ba3ff0 con 0x55b732ce7820 2023-11-06T03:08:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f1183fff640 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f117805c380 tx=0x7f1178054410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22abef9d815182e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f116a7fc640 1 -- 172.21.15.157:0/853751658 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178052d20 con 0x55b732ce7820 2023-11-06T03:08:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f116a7fc640 1 -- 172.21.15.157:0/853751658 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178083070 con 0x55b732ce7820 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.065+0000 7f116a7fc640 1 -- 172.21.15.157:0/853751658 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780831e0 con 0x55b732ce7820 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/853751658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 msgr2=0x55b732c9cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f117805c380 tx=0x7f1178054410 comp rx=0 tx=0).stop 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/853751658 shutdown_connections 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732c9d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732c9cdc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- 172.21.15.157:0/853751658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/853751658 >> 172.21.15.157:0/853751658 conn(0x55b732c94670 msgr2=0x55b732c9b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/853751658 shutdown_connections 2023-11-06T03:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/853751658 wait complete. 2023-11-06T03:08:20.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- start start 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 Processor -- start 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce7820 0x55b732ce8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732aed0e0 con 0x55b732c9d300 2023-11-06T03:08:20.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732a8c3f0 con 0x55b732ce7820 2023-11-06T03:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b732aaa830 con 0x55b732ce6f00 2023-11-06T03:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 10 client.0 _ll_register_callbacks cb 0x55b732c887e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58562/0 (socket says 172.21.15.157:58562) 2023-11-06T03:08:20.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 -- 172.21.15.157:0/180392678 learned_addr learned my addr 172.21.15.157:0/180392678 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1189454640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce7820 0x55b732ce8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1183fff640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 -- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 msgr2=0x55b732ce8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 -- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce7820 msgr2=0x55b732ce8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce7820 0x55b732ce8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 -- 172.21.15.157:0/180392678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840c7530 con 0x55b732c9d300 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1188c53640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f117401ab40 tx=0x7f117401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c4d30c20e731d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740b6780 con 0x55b732c9d300 2023-11-06T03:08:20.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f11740c7070 con 0x55b732c9d300 2023-11-06T03:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b732ab4df0 con 0x55b732c9d300 2023-11-06T03:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11740b68e0 con 0x55b732c9d300 2023-11-06T03:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 10 client.6450 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55b7327e9dd0 con 0x55b732c9d300 2023-11-06T03:08:20.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b732ab4df0 con 0x55b732c9d300 2023-11-06T03:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f11740be040 con 0x55b732c9d300 2023-11-06T03:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 10 client.6450 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 10 client.6450 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 20 client.6450 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:20.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce9090 0x55b732cf9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 4 client.6450 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b732ab8e90 con 0x55b732ce9090 2023-11-06T03:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1183fff640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce9090 0x55b732cf9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f11637fe640 10 client.6450 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.069+0000 7f1183fff640 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce9090 0x55b732cf9190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3ccfb051dd233a server_cookie=7aabaeef5026e5ce in_seq=0 out_seq=0 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f11637fe640 1 -- 172.21.15.157:0/180392678 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b732ab8e90 con 0x55b732ce9090 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f11637fe640 10 client.6450 handle_command_reply: tid=0 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 client.6450 shutdown 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 msgr2=0x55b732ce7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f117401ab40 tx=0x7f117401acd0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 shutdown_connections 2023-11-06T03:08:20.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce9090 0x55b732cf9190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce7820 0x55b732ce8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732ce8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f11637fe640 0 client.6450 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 --2- 172.21.15.157:0/180392678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732ce7c00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 >> 172.21.15.157:0/180392678 conn(0x55b732c94670 msgr2=0x55b732c9bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 shutdown_connections 2023-11-06T03:08:20.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 1 -- 172.21.15.157:0/180392678 wait complete. 2023-11-06T03:08:20.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.073+0000 7f118dfdf000 20 client.6450 trim_cache size 0 max 16384 2023-11-06T03:08:20.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 Processor -- start 2023-11-06T03:08:20.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 -- start start 2023-11-06T03:08:20.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732cfa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732cfa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7327aae80 con 0x55b732ce7820 2023-11-06T03:08:20.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732aea4e0 con 0x55b732c9d300 2023-11-06T03:08:20.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b732ae8a50 con 0x55b732ce6f00 2023-11-06T03:08:20.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f1189454640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f1189454640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58578/0 (socket says 172.21.15.157:58578) 2023-11-06T03:08:20.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.085+0000 7f1189454640 1 -- 172.21.15.157:0/2250955752 learned_addr learned my addr 172.21.15.157:0/2250955752 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 -- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 msgr2=0x55b732cfa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732cfa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 -- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 msgr2=0x55b732cfa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732cfa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 -- 172.21.15.157:0/2250955752 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b732ab21e0 con 0x55b732ce7820 2023-11-06T03:08:20.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f117c00be70 tx=0x7f117c00a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab435edafb1e55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f116affd640 1 -- 172.21.15.157:0/2250955752 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0097a0 con 0x55b732ce7820 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f116affd640 1 -- 172.21.15.157:0/2250955752 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c011040 con 0x55b732ce7820 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f116affd640 1 -- 172.21.15.157:0/2250955752 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c009b60 con 0x55b732ce7820 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/2250955752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 msgr2=0x55b732cdfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f117c00be70 tx=0x7f117c00a7b0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/2250955752 shutdown_connections 2023-11-06T03:08:20.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732ce7820 0x55b732cdfe70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce6f00 0x55b732cfa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- 172.21.15.157:0/2250955752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732c9d300 0x55b732cfa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/2250955752 >> 172.21.15.157:0/2250955752 conn(0x55b732c94670 msgr2=0x55b732cea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/2250955752 shutdown_connections 2023-11-06T03:08:20.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/2250955752 wait complete. 2023-11-06T03:08:20.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- start start 2023-11-06T03:08:20.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 Processor -- start 2023-11-06T03:08:20.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce6f00 0x55b732d45da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce7820 0x55b732ce3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732a92730 con 0x55b732c9d300 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b732c9f920 con 0x55b732ce6f00 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b732ab2bd0 con 0x55b732ce7820 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 10 client.0 _ll_register_callbacks cb 0x7f1178050d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:20.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58592/0 (socket says 172.21.15.157:58592) 2023-11-06T03:08:20.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 -- 172.21.15.157:0/3609211543 learned_addr learned my addr 172.21.15.157:0/3609211543 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 -- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce7820 msgr2=0x55b732ce3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1189454640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce7820 0x55b732ce3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce7820 0x55b732ce3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1183fff640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce6f00 0x55b732d45da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 -- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce6f00 msgr2=0x55b732d45da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce6f00 0x55b732d45da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 -- 172.21.15.157:0/3609211543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c00baa0 con 0x55b732c9d300 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1188c53640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f117401c2c0 tx=0x7f1174019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de58e512c2ee650e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:20.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11740b6ae0 con 0x55b732c9d300 2023-11-06T03:08:20.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f11740c7040 con 0x55b732c9d300 2023-11-06T03:08:20.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b732aae340 con 0x55b732c9d300 2023-11-06T03:08:20.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11740cc5d0 con 0x55b732c9d300 2023-11-06T03:08:20.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 10 client.6456 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:20.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55b732a98330 con 0x55b732c9d300 2023-11-06T03:08:20.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b732aae340 con 0x55b732c9d300 2023-11-06T03:08:20.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f11740be070 con 0x55b732c9d300 2023-11-06T03:08:20.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 10 client.6456 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:20.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 10 client.6456 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:20.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 20 client.6456 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:20.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce3590 0x55b732ce39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 4 client.6456 mds_command: new command op to 5868 tid=0[{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2023-11-06T03:08:20.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55b732ae6160 con 0x55b732ce3590 2023-11-06T03:08:20.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.089+0000 7f1183fff640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce3590 0x55b732ce39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f1183fff640 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce3590 0x55b732ce39a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7baea48d1eb1c084 server_cookie=40c0ad70d073fd09 in_seq=0 out_seq=0 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f11637fe640 10 client.6456 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f11637fe640 1 -- 172.21.15.157:0/3609211543 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b732ae6160 con 0x55b732ce3590 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f11637fe640 10 client.6456 handle_command_reply: tid=0 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 client.6456 shutdown 2023-11-06T03:08:20.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 msgr2=0x55b732d45860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f117401c2c0 tx=0x7f1174019230 comp rx=0 tx=0).stop 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 shutdown_connections 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55b732ce3590 0x55b732ce39a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b732ce7820 0x55b732ce3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b732ce6f00 0x55b732d45da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f11637fe640 0 client.6456 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 --2- 172.21.15.157:0/3609211543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b732c9d300 0x55b732d45860 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 >> 172.21.15.157:0/3609211543 conn(0x55b732c94670 msgr2=0x55b732cf9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 shutdown_connections 2023-11-06T03:08:20.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 1 -- 172.21.15.157:0/3609211543 wait complete. 2023-11-06T03:08:20.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118dfdf000 20 client.6456 trim_cache size 0 max 16384 2023-11-06T03:08:20.110 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:20.110 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:08:20.110 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "df07f050-dcaf-4840-8b35-b79aecdba1dc", 2023-11-06T03:08:20.110 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:08:20.111 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:20.111 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f079439f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4b [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v808 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6057=0-4194304@1} caps={6057=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=6057 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5582b64a0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4b:[<0x10000000001/new_file_1 v808>,<0x10000000000/ceph-qa-suite v101>,<0x1/scrub_test_path v51>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-06T03:08:20.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 msgr2=0x7f1170067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 0x7f1170067f20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11740ae000 tx=0x7f11740a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:20.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 msgr2=0x7f1184156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f117800c7f0 tx=0x7f117800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 shutdown_connections 2023-11-06T03:08:20.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1170065a60 0x7f1170067f20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f118415ce80 0x7f11840b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f118415caa0 0x7f1184156e80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 --2- 172.21.15.157:0/3846023842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11840ce1a0 0x7f1184150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 >> 172.21.15.157:0/3846023842 conn(0x7f1184070790 msgr2=0x7f11840d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 shutdown_connections 2023-11-06T03:08:20.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.093+0000 7f118aeb7640 1 -- 172.21.15.157:0/3846023842 wait complete. 2023-11-06T03:08:20.115 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': 'df07f050-dcaf-4840-8b35-b79aecdba1dc', 'mode': 'asynchronous'}' 2023-11-06T03:08:20.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T03:08:20.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 Processor -- start 2023-11-06T03:08:20.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 -- start start 2023-11-06T03:08:20.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03640ce800 0x7f03640cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640cf1a0 con 0x7f0364146370 2023-11-06T03:08:20.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036406c3d0 con 0x7f036406b6a0 2023-11-06T03:08:20.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036c5f3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640b2120 con 0x7f03640ce800 2023-11-06T03:08:20.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036a38f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0364146370 0x7f03640b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32860/0 (socket says 172.21.15.157:32860) 2023-11-06T03:08:20.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f0369b8e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03640ce800 0x7f03640cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036a38f640 1 -- 172.21.15.157:0/3146138824 learned_addr learned my addr 172.21.15.157:0/3146138824 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:20.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f0369b8e640 1 -- 172.21.15.157:0/3146138824 >> v1:172.21.15.195:6789/0 conn(0x7f036406b6a0 legacy=0x7f036406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f0369b8e640 1 -- 172.21.15.157:0/3146138824 >> v1:172.21.15.157:6789/0 conn(0x7f0364146370 legacy=0x7f03640b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f0369b8e640 1 -- 172.21.15.157:0/3146138824 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036406f5a0 con 0x7f03640ce800 2023-11-06T03:08:20.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.233+0000 7f036938d640 1 -- 172.21.15.157:0/3146138824 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200741750 0 0) 0x7f03640cf1a0 con 0x7f0364146370 2023-11-06T03:08:20.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f0369b8e640 1 --2- 172.21.15.157:0/3146138824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03640ce800 0x7f03640cec60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f03540097f0 tx=0x7f035401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bc9d7d211e81d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036938d640 1 -- 172.21.15.157:0/3146138824 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540ad680 con 0x7f03640ce800 2023-11-06T03:08:20.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036938d640 1 -- 172.21.15.157:0/3146138824 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03540ad7e0 con 0x7f03640ce800 2023-11-06T03:08:20.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036938d640 1 -- 172.21.15.157:0/3146138824 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540adb70 con 0x7f03640ce800 2023-11-06T03:08:20.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 -- 172.21.15.157:0/3146138824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03640ce800 msgr2=0x7f03640cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 --2- 172.21.15.157:0/3146138824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03640ce800 0x7f03640cec60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f03540097f0 tx=0x7f035401c380 comp rx=0 tx=0).stop 2023-11-06T03:08:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 -- 172.21.15.157:0/3146138824 shutdown_connections 2023-11-06T03:08:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 --2- 172.21.15.157:0/3146138824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03640ce800 0x7f03640cec60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 -- 172.21.15.157:0/3146138824 >> 172.21.15.157:0/3146138824 conn(0x7f0364065930 msgr2=0x7f0364065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 -- 172.21.15.157:0/3146138824 shutdown_connections 2023-11-06T03:08:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.237+0000 7f036c5f3640 1 -- 172.21.15.157:0/3146138824 wait complete. 2023-11-06T03:08:20.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 Processor -- start 2023-11-06T03:08:20.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- start start 2023-11-06T03:08:20.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 0x7f036413c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 0x7f0364141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364158190 con 0x7f036406b6a0 2023-11-06T03:08:20.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036415b6f0 con 0x7f03640ce800 2023-11-06T03:08:20.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036415e7f0 con 0x7f0364146370 2023-11-06T03:08:20.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58610/0 (socket says 172.21.15.157:58610) 2023-11-06T03:08:20.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 -- 172.21.15.157:0/2973070916 learned_addr learned my addr 172.21.15.157:0/2973070916 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036ab90640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 0x7f0364141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f0369b8e640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 0x7f036413c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 -- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 msgr2=0x7f0364141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 0x7f0364141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 -- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 msgr2=0x7f036413c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 0x7f036413c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 -- 172.21.15.157:0/2973070916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354009420 con 0x7f036406b6a0 2023-11-06T03:08:20.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f0369b8e640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 0x7f036413c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:20.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036ab90640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 0x7f0364141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:20.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036a38f640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f036000c520 tx=0x7f036000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f035affd640 1 -- 172.21.15.157:0/2973070916 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600074e0 con 0x7f036406b6a0 2023-11-06T03:08:20.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f035affd640 1 -- 172.21.15.157:0/2973070916 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360007640 con 0x7f036406b6a0 2023-11-06T03:08:20.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f035affd640 1 -- 172.21.15.157:0/2973070916 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600049f0 con 0x7f036406b6a0 2023-11-06T03:08:20.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03641583e0 con 0x7f036406b6a0 2023-11-06T03:08:20.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03641589a0 con 0x7f036406b6a0 2023-11-06T03:08:20.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f035affd640 1 -- 172.21.15.157:0/2973070916 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0360004b50 con 0x7f036406b6a0 2023-11-06T03:08:20.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.241+0000 7f035affd640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 0x7f0348067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f035affd640 1 -- 172.21.15.157:0/2973070916 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f03600825e0 con 0x7f036406b6a0 2023-11-06T03:08:20.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0369b8e640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 0x7f0348067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 Processor -- start 2023-11-06T03:08:20.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- start start 2023-11-06T03:08:20.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0369b8e640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 0x7f0348067f20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f03540af000 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b245a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b240fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b22d7c10 con 0x55e7b2410260 2023-11-06T03:08:20.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b22663e0 con 0x55e7b245aa50 2023-11-06T03:08:20.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b22639b0 con 0x55e7b245a130 2023-11-06T03:08:20.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58622/0 (socket says 172.21.15.157:58622) 2023-11-06T03:08:20.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 -- 172.21.15.157:0/2740729093 learned_addr learned my addr 172.21.15.157:0/2740729093 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0369b8e640 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b240fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036a38f640 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b245a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 -- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 msgr2=0x55e7b245a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b245a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 -- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 msgr2=0x55e7b240fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b240fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 -- 172.21.15.157:0/2740729093 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7b1f5be20 con 0x55e7b2410260 2023-11-06T03:08:20.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036ab90640 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f035c007f10 tx=0x7f035c00c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5d5430c913c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0347fff640 1 -- 172.21.15.157:0/2740729093 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00f830 con 0x55e7b2410260 2023-11-06T03:08:20.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0347fff640 1 -- 172.21.15.157:0/2740729093 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00f990 con 0x55e7b2410260 2023-11-06T03:08:20.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f0347fff640 1 -- 172.21.15.157:0/2740729093 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00fd20 con 0x55e7b2410260 2023-11-06T03:08:20.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- 172.21.15.157:0/2740729093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 msgr2=0x55e7b2410640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f035c007f10 tx=0x7f035c00c670 comp rx=0 tx=0).stop 2023-11-06T03:08:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- 172.21.15.157:0/2740729093 shutdown_connections 2023-11-06T03:08:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b2410260 0x55e7b2410640 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b240fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 --2- 172.21.15.157:0/2740729093 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b245a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- 172.21.15.157:0/2740729093 >> 172.21.15.157:0/2740729093 conn(0x55e7b24169f0 msgr2=0x55e7b240e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- 172.21.15.157:0/2740729093 shutdown_connections 2023-11-06T03:08:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- 172.21.15.157:0/2740729093 wait complete. 2023-11-06T03:08:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 -- start start 2023-11-06T03:08:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.245+0000 7f036f71b000 1 Processor -- start 2023-11-06T03:08:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b2459a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b245b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b225f2a0 con 0x55e7b245a130 2023-11-06T03:08:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b21fe470 con 0x55e7b245aa50 2023-11-06T03:08:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b221c660 con 0x55e7b2410260 2023-11-06T03:08:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 10 client.0 _ll_register_callbacks cb 0x55e7b23fbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58628/0 (socket says 172.21.15.157:58628) 2023-11-06T03:08:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 -- 172.21.15.157:0/1786565217 learned_addr learned my addr 172.21.15.157:0/1786565217 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036a38f640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b2459a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036ab90640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b245b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 -- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 msgr2=0x55e7b2459a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b2459a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 -- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 msgr2=0x55e7b245b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b245b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 -- 172.21.15.157:0/1786565217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c007b40 con 0x55e7b245a130 2023-11-06T03:08:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f0369b8e640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0354006cf0 tx=0x7f035401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b1b7f2ac80fcf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:20.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540b8af0 con 0x55e7b245a130 2023-11-06T03:08:20.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f03540be040 con 0x55e7b245a130 2023-11-06T03:08:20.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7b221c0a0 con 0x55e7b245a130 2023-11-06T03:08:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03540b8c50 con 0x55e7b245a130 2023-11-06T03:08:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 10 client.6471 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55e7b22241d0 con 0x55e7b245a130 2023-11-06T03:08:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7b221c0a0 con 0x55e7b245a130 2023-11-06T03:08:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f03540cc410 con 0x55e7b245a130 2023-11-06T03:08:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 10 client.6471 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 10 client.6471 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 20 client.6471 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b245c110 0x55e7b246c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 4 client.6471 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7b225dbf0 con 0x55e7b245c110 2023-11-06T03:08:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036a38f640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b245c110 0x55e7b246c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 10 client.6471 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036a38f640 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b245c110 0x55e7b246c3c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c311bdb0774e5f0 server_cookie=35a2e10bda3eebc9 in_seq=0 out_seq=0 2023-11-06T03:08:20.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 1 -- 172.21.15.157:0/1786565217 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e7b225dbf0 con 0x55e7b245c110 2023-11-06T03:08:20.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 10 client.6471 handle_command_reply: tid=0 2023-11-06T03:08:20.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 client.6471 shutdown 2023-11-06T03:08:20.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 msgr2=0x55e7b245ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0354006cf0 tx=0x7f035401abc0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 shutdown_connections 2023-11-06T03:08:20.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b245c110 0x55e7b246c3c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245aa50 0x55e7b245b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245a130 0x55e7b245ae30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f03457fa640 0 client.6471 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 --2- 172.21.15.157:0/1786565217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b2459a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 >> 172.21.15.157:0/1786565217 conn(0x55e7b24169f0 msgr2=0x55e7b240ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 shutdown_connections 2023-11-06T03:08:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 1 -- 172.21.15.157:0/1786565217 wait complete. 2023-11-06T03:08:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.249+0000 7f036f71b000 20 client.6471 trim_cache size 0 max 16384 2023-11-06T03:08:20.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 Processor -- start 2023-11-06T03:08:20.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- start start 2023-11-06T03:08:20.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b24ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245a130 0x55e7b24ba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b1f1caa0 con 0x55e7b245aa50 2023-11-06T03:08:20.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b22632a0 con 0x55e7b245a130 2023-11-06T03:08:20.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b225e130 con 0x55e7b2410260 2023-11-06T03:08:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58640/0 (socket says 172.21.15.157:58640) 2023-11-06T03:08:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 -- 172.21.15.157:0/3941169256 learned_addr learned my addr 172.21.15.157:0/3941169256 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036a38f640 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b24ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f0369b8e640 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245a130 0x55e7b24ba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 -- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 msgr2=0x55e7b24ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b24ba470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 -- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245a130 msgr2=0x55e7b24ba9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245a130 0x55e7b24ba9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 -- 172.21.15.157:0/3941169256 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7b220a520 con 0x55e7b245aa50 2023-11-06T03:08:20.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036ab90640 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55e7b2261f10 tx=0x7f035c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75a01796ec85279 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f0358ff9640 1 -- 172.21.15.157:0/3941169256 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c003980 con 0x55e7b245aa50 2023-11-06T03:08:20.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f0358ff9640 1 -- 172.21.15.157:0/3941169256 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c01f050 con 0x55e7b245aa50 2023-11-06T03:08:20.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f0358ff9640 1 -- 172.21.15.157:0/3941169256 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c003d00 con 0x55e7b245aa50 2023-11-06T03:08:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- 172.21.15.157:0/3941169256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 msgr2=0x55e7b24baef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55e7b2261f10 tx=0x7f035c011000 comp rx=0 tx=0).stop 2023-11-06T03:08:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- 172.21.15.157:0/3941169256 shutdown_connections 2023-11-06T03:08:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24baef0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b245a130 0x55e7b24ba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 --2- 172.21.15.157:0/3941169256 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b2410260 0x55e7b24ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- 172.21.15.157:0/3941169256 >> 172.21.15.157:0/3941169256 conn(0x55e7b24169f0 msgr2=0x55e7b2414b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- 172.21.15.157:0/3941169256 shutdown_connections 2023-11-06T03:08:20.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.265+0000 7f036f71b000 1 -- 172.21.15.157:0/3941169256 wait complete. 2023-11-06T03:08:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- start start 2023-11-06T03:08:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 Processor -- start 2023-11-06T03:08:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b2410260 0x55e7b24c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b24c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b221d400 con 0x55e7b245aa50 2023-11-06T03:08:20.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7b245b4c0 con 0x55e7b2410260 2023-11-06T03:08:20.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7b225cab0 con 0x55e7b245a130 2023-11-06T03:08:20.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 10 client.0 _ll_register_callbacks cb 0x55e7b2223cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58644/0 (socket says 172.21.15.157:58644) 2023-11-06T03:08:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 -- 172.21.15.157:0/367132861 learned_addr learned my addr 172.21.15.157:0/367132861 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036a38f640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b2410260 0x55e7b24c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f0369b8e640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b24c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 -- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 msgr2=0x55e7b24c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b24c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 -- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b2410260 msgr2=0x55e7b24c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:20.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b2410260 0x55e7b24c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 -- 172.21.15.157:0/367132861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7b1f5be20 con 0x55e7b245aa50 2023-11-06T03:08:20.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036ab90640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f035c026040 tx=0x7f035c004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bafa21580f7f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:20.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:20.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00fa40 con 0x55e7b245aa50 2023-11-06T03:08:20.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f035c008040 con 0x55e7b245aa50 2023-11-06T03:08:20.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7b2460e40 con 0x55e7b245aa50 2023-11-06T03:08:20.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f035c00fba0 con 0x55e7b245aa50 2023-11-06T03:08:20.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 10 client.6477 fetch_fsmap learned FSMap version 38 2023-11-06T03:08:20.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55e7b21fe090 con 0x55e7b245aa50 2023-11-06T03:08:20.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7b2460e40 con 0x55e7b245aa50 2023-11-06T03:08:20.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 38) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f035c00fd00 con 0x55e7b245aa50 2023-11-06T03:08:20.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 10 client.6477 fetch_fsmap finished waiting for FSMap version 38 2023-11-06T03:08:20.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 10 client.6477 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:20.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 20 client.6477 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:20.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b24c56c0 0x55e7b24c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:20.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 4 client.6477 mds_command: new command op to 5868 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:08:20.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7b2228620 con 0x55e7b24c56c0 2023-11-06T03:08:20.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036a38f640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b24c56c0 0x55e7b24c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:20.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 10 client.6477 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036a38f640 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b24c56c0 0x55e7b24c7ab0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=196b50a143484ee9 server_cookie=8f17a015f3c1e6ae in_seq=0 out_seq=0 2023-11-06T03:08:20.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 1 -- 172.21.15.157:0/367132861 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x55e7b2228620 con 0x55e7b24c56c0 2023-11-06T03:08:20.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 10 client.6477 handle_command_reply: tid=0 2023-11-06T03:08:20.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 client.6477 shutdown 2023-11-06T03:08:20.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 msgr2=0x55e7b24c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f035c026040 tx=0x7f035c004a10 comp rx=0 tx=0).stop 2023-11-06T03:08:20.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 shutdown_connections 2023-11-06T03:08:20.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55e7b24c56c0 0x55e7b24c7ab0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e7b245aa50 0x55e7b24c5180 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e7b245a130 0x55e7b24c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 --2- 172.21.15.157:0/367132861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e7b2410260 0x55e7b24c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f03457fa640 0 client.6477 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 >> 172.21.15.157:0/367132861 conn(0x55e7b24169f0 msgr2=0x55e7b24153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 shutdown_connections 2023-11-06T03:08:20.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 1 -- 172.21.15.157:0/367132861 wait complete. 2023-11-06T03:08:20.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.269+0000 7f036f71b000 20 client.6477 trim_cache size 0 max 16384 2023-11-06T03:08:20.296 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:20.297 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:08:20.297 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:08:20.297 INFO:teuthology.orchestra.run.smithi157.stdout: "df07f050-dcaf-4840-8b35-b79aecdba1dc": { 2023-11-06T03:08:20.297 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2023-11-06T03:08:20.297 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "df07f050-dcaf-4840-8b35-b79aecdba1dc", 2023-11-06T03:08:20.298 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "" 2023-11-06T03:08:20.298 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:20.298 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:20.298 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:20.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 msgr2=0x7f0348067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 0x7f0348067f20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f03540af000 tx=0x7f0354008040 comp rx=0 tx=0).stop 2023-11-06T03:08:20.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 msgr2=0x7f036415e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:20.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f036000c520 tx=0x7f036000c9e0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 shutdown_connections 2023-11-06T03:08:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0348065a60 0x7f0348067f20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0364146370 0x7f0364141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03640ce800 0x7f036413c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 --2- 172.21.15.157:0/2973070916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f036406b6a0 0x7f036415e2b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:20.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 >> 172.21.15.157:0/2973070916 conn(0x7f0364065930 msgr2=0x7f0364070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:20.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 shutdown_connections 2023-11-06T03:08:20.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:20.273+0000 7f036c5f3640 1 -- 172.21.15.157:0/2973070916 wait complete. 2023-11-06T03:08:25.303 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-06T03:08:25.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.437+0000 7f5580965640 1 Processor -- start 2023-11-06T03:08:25.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.437+0000 7f5580965640 1 -- start start 2023-11-06T03:08:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5580965640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f557c0b1ad0 0x7f557c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5580965640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f557c0b1eb0 0x7f557c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5580965640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c15b3d0 con 0x7f557c15b860 2023-11-06T03:08:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5580965640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c155610 con 0x7f557c0b1ad0 2023-11-06T03:08:25.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5580965640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c15b0c0 con 0x7f557c0b1eb0 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f557c0b1ad0 0x7f557c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f557c0b1ad0 0x7f557c0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58654/0 (socket says 172.21.15.157:58654) 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 -- 172.21.15.157:0/378531084 learned_addr learned my addr 172.21.15.157:0/378531084 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f5573fff640 1 --2- 172.21.15.157:0/378531084 >> v2:172.21.15.195:3300/0 conn(0x7f557c0b1eb0 0x7f557c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 -- 172.21.15.157:0/378531084 >> v2:172.21.15.195:3300/0 conn(0x7f557c0b1eb0 msgr2=0x7f557c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 --2- 172.21.15.157:0/378531084 >> v2:172.21.15.195:3300/0 conn(0x7f557c0b1eb0 0x7f557c0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 -- 172.21.15.157:0/378531084 >> v1:172.21.15.195:6789/0 conn(0x7f557c15b860 legacy=0x7f557c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 -- 172.21.15.157:0/378531084 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0b50c0 con 0x7f557c0b1ad0 2023-11-06T03:08:25.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.441+0000 7f557a59c640 1 --2- 172.21.15.157:0/378531084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0b1ad0 0x7f557c0b4ac0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f557c06e350 tx=0x7f556401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0c8c62a57a0ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5579d9b640 1 -- 172.21.15.157:0/378531084 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640ac600 con 0x7f557c0b1ad0 2023-11-06T03:08:25.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5579d9b640 1 -- 172.21.15.157:0/378531084 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55640a8050 con 0x7f557c0b1ad0 2023-11-06T03:08:25.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5579d9b640 1 -- 172.21.15.157:0/378531084 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640ac980 con 0x7f557c0b1ad0 2023-11-06T03:08:25.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/378531084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0b1ad0 msgr2=0x7f557c0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- 172.21.15.157:0/378531084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0b1ad0 0x7f557c0b4ac0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f557c06e350 tx=0x7f556401c490 comp rx=0 tx=0).stop 2023-11-06T03:08:25.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/378531084 shutdown_connections 2023-11-06T03:08:25.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- 172.21.15.157:0/378531084 >> v2:172.21.15.195:3300/0 conn(0x7f557c0b1eb0 0x7f557c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- 172.21.15.157:0/378531084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c0b1ad0 0x7f557c0b4ac0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/378531084 >> 172.21.15.157:0/378531084 conn(0x7f557c06a170 msgr2=0x7f557c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/378531084 shutdown_connections 2023-11-06T03:08:25.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/378531084 wait complete. 2023-11-06T03:08:25.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 Processor -- start 2023-11-06T03:08:25.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- start start 2023-11-06T03:08:25.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557c0b1ad0 0x7f557c0c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 0x7f557c160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0cae90 con 0x7f557c15b860 2023-11-06T03:08:25.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0ca280 con 0x7f557c0b1ad0 2023-11-06T03:08:25.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0ca560 con 0x7f557c0b1eb0 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58664/0 (socket says 172.21.15.157:58664) 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 -- 172.21.15.157:0/2085936092 learned_addr learned my addr 172.21.15.157:0/2085936092 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557a59c640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557c0b1ad0 0x7f557c0c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5573fff640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 0x7f557c160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 -- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 msgr2=0x7f557c160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 0x7f557c160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 -- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557c0b1ad0 msgr2=0x7f557c0c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557c0b1ad0 0x7f557c0c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 -- 172.21.15.157:0/2085936092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640094d0 con 0x7f557c15b860 2023-11-06T03:08:25.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5573fff640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 0x7f557c160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f557400b970 tx=0x7f557400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5572ffd640 1 -- 172.21.15.157:0/2085936092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557400cb40 con 0x7f557c15b860 2023-11-06T03:08:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5572ffd640 1 -- 172.21.15.157:0/2085936092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557400cca0 con 0x7f557c15b860 2023-11-06T03:08:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5572ffd640 1 -- 172.21.15.157:0/2085936092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574014680 con 0x7f557c15b860 2023-11-06T03:08:25.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c0ca840 con 0x7f557c15b860 2023-11-06T03:08:25.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.445+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c1542d0 con 0x7f557c15b860 2023-11-06T03:08:25.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5572ffd640 1 -- 172.21.15.157:0/2085936092 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f557401d3b0 con 0x7f557c15b860 2023-11-06T03:08:25.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5572ffd640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 0x7f555c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5572ffd640 1 -- 172.21.15.157:0/2085936092 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5574083190 con 0x7f557c15b860 2023-11-06T03:08:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557a59c640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 0x7f555c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 Processor -- start 2023-11-06T03:08:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 -- start start 2023-11-06T03:08:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cca50 0x5599e4781d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4649c10 con 0x5599e4782260 2023-11-06T03:08:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e45d83e0 con 0x5599e47cca50 2023-11-06T03:08:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e45d59b0 con 0x5599e47cc130 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557a59c640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 0x7f555c067f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f55640ae000 tx=0x7f55640a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58680/0 (socket says 172.21.15.157:58680) 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557ad9d640 1 -- 172.21.15.157:0/2618226615 learned_addr learned my addr 172.21.15.157:0/2618226615 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f5573fff640 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cca50 0x5599e4781d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.449+0000 7f557a59c640 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 msgr2=0x5599e47cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cca50 msgr2=0x5599e4781d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cca50 0x5599e4781d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/2618226615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e42cde20 con 0x5599e4782260 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5574081480 tx=0x7f5574054e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39452232ee7cb9c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f555bfff640 1 -- 172.21.15.157:0/2618226615 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557406a020 con 0x5599e4782260 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f555bfff640 1 -- 172.21.15.157:0/2618226615 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557405c640 con 0x5599e4782260 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f555bfff640 1 -- 172.21.15.157:0/2618226615 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574052a00 con 0x5599e4782260 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- 172.21.15.157:0/2618226615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 msgr2=0x5599e4782640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5574081480 tx=0x7f5574054e20 comp rx=0 tx=0).stop 2023-11-06T03:08:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- 172.21.15.157:0/2618226615 shutdown_connections 2023-11-06T03:08:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e4782260 0x5599e4782640 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cca50 0x5599e4781d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- 172.21.15.157:0/2618226615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- 172.21.15.157:0/2618226615 >> 172.21.15.157:0/2618226615 conn(0x5599e47889f0 msgr2=0x5599e4780600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- 172.21.15.157:0/2618226615 shutdown_connections 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- 172.21.15.157:0/2618226615 wait complete. 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- start start 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 Processor -- start 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e4782260 0x5599e47cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cc130 0x5599e47cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e45d12a0 con 0x5599e47cca50 2023-11-06T03:08:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e4570470 con 0x5599e47cc130 2023-11-06T03:08:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e458e660 con 0x5599e4782260 2023-11-06T03:08:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5583a8d000 10 client.0 _ll_register_callbacks cb 0x5599e476dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58694/0 (socket says 172.21.15.157:58694) 2023-11-06T03:08:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/4001179758 learned_addr learned my addr 172.21.15.157:0/4001179758 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557a59c640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e4782260 0x5599e47cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f5573fff640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cc130 0x5599e47cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e4782260 msgr2=0x5599e47cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e4782260 0x5599e47cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cc130 msgr2=0x5599e47cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cc130 0x5599e47cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.453+0000 7f557ad9d640 1 -- 172.21.15.157:0/4001179758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c1542d0 con 0x5599e47cca50 2023-11-06T03:08:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5574054310 tx=0x7f557400e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7830b0b72882bf13 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557406a070 con 0x5599e47cca50 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5574081200 con 0x5599e47cca50 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e458e0a0 con 0x5599e47cca50 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5574053b50 con 0x5599e47cca50 2023-11-06T03:08:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 10 client.6489 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5599e45961d0 con 0x5599e47cca50 2023-11-06T03:08:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599e458e0a0 con 0x5599e47cca50 2023-11-06T03:08:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f557405c1b0 con 0x5599e47cca50 2023-11-06T03:08:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 10 client.6489 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 10 client.6489 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 20 client.6489 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e47dbcf0 0x5599e47de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 4 client.6489 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e45cfbf0 con 0x5599e47dbcf0 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f557a59c640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e47dbcf0 0x5599e47de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 10 client.6489 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f557a59c640 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e47dbcf0 0x5599e47de0e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf4c6858a5a72b7 server_cookie=58f164afbc4e8cb5 in_seq=0 out_seq=0 2023-11-06T03:08:25.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 1 -- 172.21.15.157:0/4001179758 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5599e45cfbf0 con 0x5599e47dbcf0 2023-11-06T03:08:25.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 10 client.6489 handle_command_reply: tid=0 2023-11-06T03:08:25.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 client.6489 shutdown 2023-11-06T03:08:25.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 msgr2=0x5599e47cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5574054310 tx=0x7f557400e070 comp rx=0 tx=0).stop 2023-11-06T03:08:25.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 shutdown_connections 2023-11-06T03:08:25.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e47dbcf0 0x5599e47de0e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47cd8b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e47cc130 0x5599e47cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f55597fa640 0 client.6489 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4001179758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e4782260 0x5599e47cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 >> 172.21.15.157:0/4001179758 conn(0x5599e47889f0 msgr2=0x5599e4780d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 shutdown_connections 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 1 -- 172.21.15.157:0/4001179758 wait complete. 2023-11-06T03:08:25.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.457+0000 7f5583a8d000 20 client.6489 trim_cache size 0 max 16384 2023-11-06T03:08:25.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 Processor -- start 2023-11-06T03:08:25.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- start start 2023-11-06T03:08:25.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e47de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47de620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e47cda60 con 0x5599e47cca50 2023-11-06T03:08:25.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e428eaa0 con 0x5599e4782260 2023-11-06T03:08:25.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e45d3e00 con 0x5599e47cc130 2023-11-06T03:08:25.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58698/0 (socket says 172.21.15.157:58698) 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 -- 172.21.15.157:0/4159716878 learned_addr learned my addr 172.21.15.157:0/4159716878 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 -- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 msgr2=0x5599e47de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 -- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 msgr2=0x5599e47de0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e47de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 -- 172.21.15.157:0/4159716878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e457c520 con 0x5599e47cca50 2023-11-06T03:08:25.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f557ad9d640 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5574054820 tx=0x7f5574020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73358bac33f563c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5570ff9640 1 -- 172.21.15.157:0/4159716878 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557406a070 con 0x5599e47cca50 2023-11-06T03:08:25.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5570ff9640 1 -- 172.21.15.157:0/4159716878 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5574081200 con 0x5599e47cca50 2023-11-06T03:08:25.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5570ff9640 1 -- 172.21.15.157:0/4159716878 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557407b400 con 0x5599e47cca50 2023-11-06T03:08:25.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- 172.21.15.157:0/4159716878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 msgr2=0x5599e47deb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5574054820 tx=0x7f5574020040 comp rx=0 tx=0).stop 2023-11-06T03:08:25.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- 172.21.15.157:0/4159716878 shutdown_connections 2023-11-06T03:08:25.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cca50 0x5599e47deb60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cc130 0x5599e47de620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 --2- 172.21.15.157:0/4159716878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e47de0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- 172.21.15.157:0/4159716878 >> 172.21.15.157:0/4159716878 conn(0x5599e47889f0 msgr2=0x5599e4786b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- 172.21.15.157:0/4159716878 shutdown_connections 2023-11-06T03:08:25.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- 172.21.15.157:0/4159716878 wait complete. 2023-11-06T03:08:25.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 -- start start 2023-11-06T03:08:25.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.473+0000 7f5583a8d000 1 Processor -- start 2023-11-06T03:08:25.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e4825100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 0x5599e4825b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e462a2b0 con 0x5599e47cc130 2023-11-06T03:08:25.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e45d52a0 con 0x5599e4782260 2023-11-06T03:08:25.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e45ceab0 con 0x5599e47cca50 2023-11-06T03:08:25.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 10 client.0 _ll_register_callbacks cb 0x5599e4595cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:25.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58702/0 (socket says 172.21.15.157:58702) 2023-11-06T03:08:25.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 -- 172.21.15.157:0/3258430172 learned_addr learned my addr 172.21.15.157:0/3258430172 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557a59c640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e4825100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557ad9d640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 0x5599e4825b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 -- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 msgr2=0x5599e4825b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 0x5599e4825b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 -- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 msgr2=0x5599e4825100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e4825100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 -- 172.21.15.157:0/3258430172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e45961d0 con 0x5599e47cc130 2023-11-06T03:08:25.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557ad9d640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 0x5599e4825b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557a59c640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e4825100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5573fff640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f556c0122c0 tx=0x7f556c012780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0d6edfe59f5e43 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:25.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c004440 con 0x5599e47cc130 2023-11-06T03:08:25.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f556c003620 con 0x5599e47cc130 2023-11-06T03:08:25.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e459a620 con 0x5599e47cc130 2023-11-06T03:08:25.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556c0037e0 con 0x5599e47cc130 2023-11-06T03:08:25.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 10 client.6495 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:25.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x5599e4570090 con 0x5599e47cc130 2023-11-06T03:08:25.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599e459a620 con 0x5599e47cc130 2023-11-06T03:08:25.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f556c003940 con 0x5599e47cc130 2023-11-06T03:08:25.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 10 client.6495 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:25.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 10 client.6495 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:25.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 20 client.6495 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:25.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e48260c0 0x5599e48308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 4 client.6495 mds_command: new command op to 5868 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:08:25.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599e45ca320 con 0x5599e48260c0 2023-11-06T03:08:25.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557a59c640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e48260c0 0x5599e48308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 10 client.6495 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f557a59c640 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e48260c0 0x5599e48308e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99c604c0c7d7523 server_cookie=12b89fa0630d9ac0 in_seq=0 out_seq=0 2023-11-06T03:08:25.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 1 -- 172.21.15.157:0/3258430172 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599e45ca320 con 0x5599e48260c0 2023-11-06T03:08:25.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 10 client.6495 handle_command_reply: tid=0 2023-11-06T03:08:25.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 client.6495 shutdown 2023-11-06T03:08:25.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 msgr2=0x5599e4825640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f556c0122c0 tx=0x7f556c012780 comp rx=0 tx=0).stop 2023-11-06T03:08:25.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 shutdown_connections 2023-11-06T03:08:25.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x5599e48260c0 0x5599e48308e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5599e47cca50 0x5599e4825b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f55597fa640 0 client.6495 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e47cc130 0x5599e4825640 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 --2- 172.21.15.157:0/3258430172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5599e4782260 0x5599e4825100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 >> 172.21.15.157:0/3258430172 conn(0x5599e47889f0 msgr2=0x5599e47873b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 shutdown_connections 2023-11-06T03:08:25.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 1 -- 172.21.15.157:0/3258430172 wait complete. 2023-11-06T03:08:25.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5583a8d000 20 client.6495 trim_cache size 0 max 16384 2023-11-06T03:08:25.505 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:25.506 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:08:25.506 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:08:25.506 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:25.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 msgr2=0x7f555c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 0x7f555c067f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f55640ae000 tx=0x7f55640a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:25.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 msgr2=0x7f557c153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f557400b970 tx=0x7f557400be30 comp rx=0 tx=0).stop 2023-11-06T03:08:25.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 shutdown_connections 2023-11-06T03:08:25.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f555c065a60 0x7f555c067f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f557c15b860 0x7f557c153d90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f557c0b1eb0 0x7f557c160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 --2- 172.21.15.157:0/2085936092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557c0b1ad0 0x7f557c0c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 >> 172.21.15.157:0/2085936092 conn(0x7f557c06a170 msgr2=0x7f557c15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 shutdown_connections 2023-11-06T03:08:25.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.477+0000 7f5580965640 1 -- 172.21.15.157:0/2085936092 wait complete. 2023-11-06T03:08:25.514 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:08:25.514 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2023-11-06T03:08:25.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 Processor -- start 2023-11-06T03:08:25.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 -- start start 2023-11-06T03:08:25.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb66c15b220 0x7fb66c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c0afb00 0x7fb66c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66c0cf920 con 0x7fb66c15ae40 2023-11-06T03:08:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c0cff50 con 0x7fb66c15b220 2023-11-06T03:08:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb670d58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0cdf80 con 0x7fb66c0afb00 2023-11-06T03:08:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb66a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb66c15ae40 0x7fb66c0cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32864/0 (socket says 172.21.15.157:32864) 2023-11-06T03:08:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb66a59c640 1 -- 172.21.15.157:0/3286404664 learned_addr learned my addr 172.21.15.157:0/3286404664 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3286404664 >> v2:172.21.15.157:3300/0 conn(0x7fb66c15b220 0x7fb66c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c0afb00 0x7fb66c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb66959a640 1 -- 172.21.15.157:0/3286404664 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519407218 0 0) 0x7fb66c0cf920 con 0x7fb66c15ae40 2023-11-06T03:08:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb66959a640 1 -- 172.21.15.157:0/3286404664 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb654003530 con 0x7fb66c15ae40 2023-11-06T03:08:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb669d9b640 1 -- 172.21.15.157:0/3286404664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c0afb00 msgr2=0x7fb66c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c0afb00 0x7fb66c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb669d9b640 1 -- 172.21.15.157:0/3286404664 >> v1:172.21.15.157:6789/0 conn(0x7fb66c15ae40 legacy=0x7fb66c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.637+0000 7fb669d9b640 1 -- 172.21.15.157:0/3286404664 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c0ce260 con 0x7fb66c15b220 2023-11-06T03:08:25.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0d0370 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097c0 tx=0x7fb66001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b186205df3583875 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb66959a640 1 -- 172.21.15.157:0/3286404664 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600ac640 con 0x7fb66c15b220 2023-11-06T03:08:25.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb66959a640 1 -- 172.21.15.157:0/3286404664 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6600ac7a0 con 0x7fb66c15b220 2023-11-06T03:08:25.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb66959a640 1 -- 172.21.15.157:0/3286404664 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600acb30 con 0x7fb66c15b220 2023-11-06T03:08:25.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- 172.21.15.157:0/3286404664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 msgr2=0x7fb66c0d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0d0370 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097c0 tx=0x7fb66001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- 172.21.15.157:0/3286404664 shutdown_connections 2023-11-06T03:08:25.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c0afb00 0x7fb66c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 --2- 172.21.15.157:0/3286404664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0d0370 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- 172.21.15.157:0/3286404664 >> 172.21.15.157:0/3286404664 conn(0x7fb66c0cd490 msgr2=0x7fb66c0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- 172.21.15.157:0/3286404664 shutdown_connections 2023-11-06T03:08:25.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- 172.21.15.157:0/3286404664 wait complete. 2023-11-06T03:08:25.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 Processor -- start 2023-11-06T03:08:25.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.641+0000 7fb670d58640 1 -- start start 2023-11-06T03:08:25.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66c0afb00 0x7fb66c06b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c15ae40 0x7fb66c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c06be10 con 0x7fb66c15b220 2023-11-06T03:08:25.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c069fb0 con 0x7fb66c0afb00 2023-11-06T03:08:25.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0b4240 con 0x7fb66c15ae40 2023-11-06T03:08:25.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58710/0 (socket says 172.21.15.157:58710) 2023-11-06T03:08:25.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 -- 172.21.15.157:0/1440746959 learned_addr learned my addr 172.21.15.157:0/1440746959 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb669d9b640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c15ae40 0x7fb66c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66c0afb00 0x7fb66c06b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 -- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c15ae40 msgr2=0x7fb66c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c15ae40 0x7fb66c153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 -- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66c0afb00 msgr2=0x7fb66c06b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66c0afb00 0x7fb66c06b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 -- 172.21.15.157:0/1440746959 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009420 con 0x7fb66c15b220 2023-11-06T03:08:25.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb664007e30 tx=0x7fb66400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 -- 172.21.15.157:0/1440746959 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664014620 con 0x7fb66c15b220 2023-11-06T03:08:25.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 -- 172.21.15.157:0/1440746959 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66400ee90 con 0x7fb66c15b220 2023-11-06T03:08:25.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c0b4520 con 0x7fb66c15b220 2023-11-06T03:08:25.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 -- 172.21.15.157:0/1440746959 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640149c0 con 0x7fb66c15b220 2023-11-06T03:08:25.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c06a4e0 con 0x7fb66c15b220 2023-11-06T03:08:25.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 -- 172.21.15.157:0/1440746959 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb664014b20 con 0x7fb66c15b220 2023-11-06T03:08:25.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 0x7fb64c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.645+0000 7fb65affd640 1 -- 172.21.15.157:0/1440746959 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb664083cc0 con 0x7fb66c15b220 2023-11-06T03:08:25.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 0x7fb64c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 Processor -- start 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- start start 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 0x7fb64c067f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0011f0 tx=0x7fb65c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac06511ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0638fd60 con 0x55ac064c7f90 2023-11-06T03:08:25.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0631e3b0 con 0x55ac06511c10 2023-11-06T03:08:25.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac0631b980 con 0x55ac06512530 2023-11-06T03:08:25.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58724/0 (socket says 172.21.15.157:58724) 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 -- 172.21.15.157:0/744246357 learned_addr learned my addr 172.21.15.157:0/744246357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb669d9b640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66a59c640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac06511ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 -- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 msgr2=0x55ac064c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 -- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 msgr2=0x55ac06511ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac06511ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 -- 172.21.15.157:0/744246357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac062b8110 con 0x55ac064c7f90 2023-11-06T03:08:25.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66a59c640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac06511ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb66405cc50 tx=0x7fb6640558a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c774f47c829cd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb64bfff640 1 -- 172.21.15.157:0/744246357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66406c070 con 0x55ac064c7f90 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb64bfff640 1 -- 172.21.15.157:0/744246357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66405b0c0 con 0x55ac064c7f90 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb64bfff640 1 -- 172.21.15.157:0/744246357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664058fd0 con 0x55ac064c7f90 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- 172.21.15.157:0/744246357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 msgr2=0x55ac064c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb66405cc50 tx=0x7fb6640558a0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- 172.21.15.157:0/744246357 shutdown_connections 2023-11-06T03:08:25.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c8370 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 --2- 172.21.15.157:0/744246357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac06511ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- 172.21.15.157:0/744246357 >> 172.21.15.157:0/744246357 conn(0x55ac064c3f40 msgr2=0x55ac064c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- 172.21.15.157:0/744246357 shutdown_connections 2023-11-06T03:08:25.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- 172.21.15.157:0/744246357 wait complete. 2023-11-06T03:08:25.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 -- start start 2023-11-06T03:08:25.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.649+0000 7fb673e80000 1 Processor -- start 2023-11-06T03:08:25.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 0x55ac064ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac06317270 con 0x55ac06511c10 2023-11-06T03:08:25.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac062b65a0 con 0x55ac064c7f90 2023-11-06T03:08:25.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac062d47c0 con 0x55ac06512530 2023-11-06T03:08:25.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 10 client.0 _ll_register_callbacks cb 0x55ac0630b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:25.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58738/0 (socket says 172.21.15.157:58738) 2023-11-06T03:08:25.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 -- 172.21.15.157:0/1182394912 learned_addr learned my addr 172.21.15.157:0/1182394912 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 0x55ac064ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 -- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 msgr2=0x55ac064ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 -- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 msgr2=0x55ac064ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 0x55ac064ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 -- 172.21.15.157:0/1182394912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c06a4e0 con 0x55ac06511c10 2023-11-06T03:08:25.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 0x55ac064ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66ad9d640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb669d9b640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb660019ff0 tx=0x7fb66001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89879958a4acf5b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:25.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600acc90 con 0x55ac06511c10 2023-11-06T03:08:25.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb6600acdf0 con 0x55ac06511c10 2023-11-06T03:08:25.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac062db350 con 0x55ac06511c10 2023-11-06T03:08:25.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6600b6440 con 0x55ac06511c10 2023-11-06T03:08:25.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 10 client.6510 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:25.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55ac06013e20 con 0x55ac06511c10 2023-11-06T03:08:25.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac062db350 con 0x55ac06511c10 2023-11-06T03:08:25.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb6600b65a0 con 0x55ac06511c10 2023-11-06T03:08:25.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 10 client.6510 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:25.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 10 client.6510 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:25.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 20 client.6510 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:25.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac065008f0 0x55ac06523cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 4 client.6510 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:25.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac06315bc0 con 0x55ac065008f0 2023-11-06T03:08:25.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac065008f0 0x55ac06523cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 10 client.6510 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac065008f0 0x55ac06523cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b70c347e290c9c server_cookie=57a781110d78626c in_seq=0 out_seq=0 2023-11-06T03:08:25.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 1 -- 172.21.15.157:0/1182394912 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ac06315bc0 con 0x55ac065008f0 2023-11-06T03:08:25.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 10 client.6510 handle_command_reply: tid=0 2023-11-06T03:08:25.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 client.6510 shutdown 2023-11-06T03:08:25.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 msgr2=0x55ac064cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb660019ff0 tx=0x7fb66001ac00 comp rx=0 tx=0).stop 2023-11-06T03:08:25.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 shutdown_connections 2023-11-06T03:08:25.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac065008f0 0x55ac06523cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac064ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac064cd2b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 --2- 172.21.15.157:0/1182394912 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac064c7f90 0x55ac064ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb6497fa640 0 client.6510 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 >> 172.21.15.157:0/1182394912 conn(0x55ac064c3f40 msgr2=0x55ac064c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 shutdown_connections 2023-11-06T03:08:25.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 1 -- 172.21.15.157:0/1182394912 wait complete. 2023-11-06T03:08:25.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.653+0000 7fb673e80000 20 client.6510 trim_cache size 0 max 16384 2023-11-06T03:08:25.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 Processor -- start 2023-11-06T03:08:25.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- start start 2023-11-06T03:08:25.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac064c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac06523cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac05fd4aa0 con 0x55ac064c7f90 2023-11-06T03:08:25.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac0631b270 con 0x55ac06511c10 2023-11-06T03:08:25.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac064ce580 con 0x55ac06512530 2023-11-06T03:08:25.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58754/0 (socket says 172.21.15.157:58754) 2023-11-06T03:08:25.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 -- 172.21.15.157:0/1953986848 learned_addr learned my addr 172.21.15.157:0/1953986848 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 -- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 msgr2=0x55ac06523cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac06523cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 -- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 msgr2=0x55ac064c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac064c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 -- 172.21.15.157:0/1953986848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac062dc1a0 con 0x55ac064c7f90 2023-11-06T03:08:25.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb66a59c640 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x55ac0636f530 tx=0x7fb65c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27a0d128bfb031c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb658ff9640 1 -- 172.21.15.157:0/1953986848 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c01a040 con 0x55ac064c7f90 2023-11-06T03:08:25.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb658ff9640 1 -- 172.21.15.157:0/1953986848 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c01b040 con 0x55ac064c7f90 2023-11-06T03:08:25.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb658ff9640 1 -- 172.21.15.157:0/1953986848 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c005ce0 con 0x55ac064c7f90 2023-11-06T03:08:25.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- 172.21.15.157:0/1953986848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 msgr2=0x55ac064c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x55ac0636f530 tx=0x7fb65c012000 comp rx=0 tx=0).stop 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- 172.21.15.157:0/1953986848 shutdown_connections 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac06512530 0x55ac06523cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06511c10 0x55ac064c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 --2- 172.21.15.157:0/1953986848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac064c7f90 0x55ac064c7340 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- 172.21.15.157:0/1953986848 >> 172.21.15.157:0/1953986848 conn(0x55ac064c3f40 msgr2=0x55ac0651f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- 172.21.15.157:0/1953986848 shutdown_connections 2023-11-06T03:08:25.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.665+0000 7fb673e80000 1 -- 172.21.15.157:0/1953986848 wait complete. 2023-11-06T03:08:25.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- start start 2023-11-06T03:08:25.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 Processor -- start 2023-11-06T03:08:25.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac064c7f90 0x55ac064feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06512530 0x55ac06573950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac06312be0 con 0x55ac06511c10 2023-11-06T03:08:25.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac064ca5b0 con 0x55ac06512530 2023-11-06T03:08:25.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac06314a80 con 0x55ac064c7f90 2023-11-06T03:08:25.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 10 client.0 _ll_register_callbacks cb 0x55ac062c5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:25.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58768/0 (socket says 172.21.15.157:58768) 2023-11-06T03:08:25.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 -- 172.21.15.157:0/3299047863 learned_addr learned my addr 172.21.15.157:0/3299047863 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 -- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac064c7f90 msgr2=0x55ac064feca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac064c7f90 0x55ac064feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 -- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06512530 msgr2=0x55ac06573950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06512530 0x55ac06573950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 -- 172.21.15.157:0/3299047863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c00f040 con 0x55ac06511c10 2023-11-06T03:08:25.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb669d9b640 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb66001aa70 tx=0x7fb660019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9797a0406c7a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600b6970 con 0x55ac06511c10 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb660004250 con 0x55ac06511c10 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac062c6c20 con 0x55ac06511c10 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb660004410 con 0x55ac06511c10 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 10 client.6516 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:25.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55ac062c24f0 con 0x55ac06511c10 2023-11-06T03:08:25.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac062c6c20 con 0x55ac06511c10 2023-11-06T03:08:25.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb6600ac910 con 0x55ac06511c10 2023-11-06T03:08:25.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 10 client.6516 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:25.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 10 client.6516 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-06T03:08:25.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 20 client.6516 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:25.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac064fbb50 0x55ac064fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 4 client.6516 mds_command: new command op to 5868 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:08:25.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac063102f0 con 0x55ac064fbb50 2023-11-06T03:08:25.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb66a59c640 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac064fbb50 0x55ac064fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb66a59c640 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac064fbb50 0x55ac064fbf60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c315b60e80978cc server_cookie=7e8595be6e3f8e97 in_seq=0 out_seq=0 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 10 client.6516 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 1 -- 172.21.15.157:0/3299047863 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55ac063102f0 con 0x55ac064fbb50 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 10 client.6516 handle_command_reply: tid=0 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 client.6516 shutdown 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 msgr2=0x55ac06573410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb66001aa70 tx=0x7fb660019160 comp rx=0 tx=0).stop 2023-11-06T03:08:25.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 shutdown_connections 2023-11-06T03:08:25.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x55ac064fbb50 0x55ac064fbf60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ac06512530 0x55ac06573950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ac06511c10 0x55ac06573410 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb6497fa640 0 client.6516 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:25.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 --2- 172.21.15.157:0/3299047863 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ac064c7f90 0x55ac064feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 >> 172.21.15.157:0/3299047863 conn(0x55ac064c3f40 msgr2=0x55ac064c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 shutdown_connections 2023-11-06T03:08:25.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 1 -- 172.21.15.157:0/3299047863 wait complete. 2023-11-06T03:08:25.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb673e80000 20 client.6516 trim_cache size 0 max 16384 2023-11-06T03:08:25.710 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:08:25.710 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:25.710 INFO:teuthology.orchestra.run.smithi157.stdout: "damage_type": "backtrace", 2023-11-06T03:08:25.710 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 1412960396, 2023-11-06T03:08:25.711 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1099511630411, 2023-11-06T03:08:25.711 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2023-11-06T03:08:25.711 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:25.711 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:08:25.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 msgr2=0x7fb64c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.669+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 0x7fb64c067f20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0011f0 tx=0x7fb65c009040 comp rx=0 tx=0).stop 2023-11-06T03:08:25.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 msgr2=0x7fb66c0acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb664007e30 tx=0x7fb66400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 shutdown_connections 2023-11-06T03:08:25.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb64c065a60 0x7fb64c067f20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb66c15b220 0x7fb66c0acc50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb66c15ae40 0x7fb66c153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 --2- 172.21.15.157:0/1440746959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb66c0afb00 0x7fb66c06b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 >> 172.21.15.157:0/1440746959 conn(0x7fb66c0cd490 msgr2=0x7fb66c15a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 shutdown_connections 2023-11-06T03:08:25.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.673+0000 7fb670d58640 1 -- 172.21.15.157:0/1440746959 wait complete. 2023-11-06T03:08:25.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:25.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 Processor -- start 2023-11-06T03:08:25.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 -- start start 2023-11-06T03:08:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7eff8c15ce80 0x7eff8c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c0d2920 con 0x7eff8c0d00b0 2023-11-06T03:08:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c0d18a0 con 0x7eff8c15caa0 2023-11-06T03:08:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff924d8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0d0520 con 0x7eff8c15ce80 2023-11-06T03:08:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7eff8c0d00b0 0x7eff8c0b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32874/0 (socket says 172.21.15.157:32874) 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 -- 172.21.15.157:0/2736394138 learned_addr learned my addr 172.21.15.157:0/2736394138 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/2736394138 >> v2:172.21.15.195:3300/0 conn(0x7eff8c15ce80 0x7eff8c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756156098 0 0) 0x7eff8c0d2920 con 0x7eff8c0d00b0 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff78003530 con 0x7eff8c0d00b0 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46788605 0 0) 0x7eff78003530 con 0x7eff8c0d00b0 2023-11-06T03:08:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 >> v2:172.21.15.195:3300/0 conn(0x7eff8c15ce80 msgr2=0x7eff8c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 --2- 172.21.15.157:0/2736394138 >> v2:172.21.15.195:3300/0 conn(0x7eff8c15ce80 0x7eff8c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 >> v1:172.21.15.195:6789/0 conn(0x7eff8c15caa0 legacy=0x7eff8c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0d1b60 con 0x7eff8c0d00b0 2023-11-06T03:08:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7eff7c001a00 con 0x7eff8c0d00b0 2023-11-06T03:08:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 >> v1:172.21.15.157:6789/0 conn(0x7eff8c0d00b0 legacy=0x7eff8c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78003980 0x7eff78003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff780047d0 0x7eff78004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 0x7eff7800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0d0520 con 0x7eff78005190 2023-11-06T03:08:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7800c030 con 0x7eff780047d0 2023-11-06T03:08:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7800c1f0 con 0x7eff78003980 2023-11-06T03:08:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 0x7eff7800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff780047d0 0x7eff78004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 -- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78003980 msgr2=0x7eff78003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8bfff640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78003980 0x7eff78003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff78003980 0x7eff78003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 -- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff780047d0 msgr2=0x7eff78004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff780047d0 0x7eff78004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff90a75640 1 -- 172.21.15.157:0/2736394138 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78003530 con 0x7eff78005190 2023-11-06T03:08:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.829+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff780047d0 0x7eff78004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:25.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff90a75640 1 -- 172.21.15.157:0/2736394138 reap_dead start 2023-11-06T03:08:25.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff90a75640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 0x7eff7800b990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00a830 tx=0x7eff7c01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cca3fbd91156894 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0072d0 con 0x7eff78005190 2023-11-06T03:08:25.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c0a8070 con 0x7eff78005190 2023-11-06T03:08:25.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8affd640 1 -- 172.21.15.157:0/2736394138 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0076b0 con 0x7eff78005190 2023-11-06T03:08:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 msgr2=0x7eff7800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 0x7eff7800b990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00a830 tx=0x7eff7c01d340 comp rx=0 tx=0).stop 2023-11-06T03:08:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- 172.21.15.157:0/2736394138 shutdown_connections 2023-11-06T03:08:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 --2- 172.21.15.157:0/2736394138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff78005190 0x7eff7800b990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- 172.21.15.157:0/2736394138 >> 172.21.15.157:0/2736394138 conn(0x7eff8c070790 msgr2=0x7eff8c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- 172.21.15.157:0/2736394138 shutdown_connections 2023-11-06T03:08:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- 172.21.15.157:0/2736394138 wait complete. 2023-11-06T03:08:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 Processor -- start 2023-11-06T03:08:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- start start 2023-11-06T03:08:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78005190 0x7eff8c1f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8c0cc4d0 0x7eff8c0ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0a9f20 con 0x7eff8c0cc070 2023-11-06T03:08:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0aad80 con 0x7eff78005190 2023-11-06T03:08:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff924d8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0aa100 con 0x7eff8c0cc4d0 2023-11-06T03:08:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58786/0 (socket says 172.21.15.157:58786) 2023-11-06T03:08:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.833+0000 7eff8b7fe640 1 -- 172.21.15.157:0/3688551642 learned_addr learned my addr 172.21.15.157:0/3688551642 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 -- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8c0cc4d0 msgr2=0x7eff8c0ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8c0cc4d0 0x7eff8c0ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8bfff640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78005190 0x7eff8c1f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 -- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78005190 msgr2=0x7eff8c1f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78005190 0x7eff8c1f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 -- 172.21.15.157:0/3688551642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0aa3e0 con 0x7eff8c0cc070 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8b7fe640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7eff74009fd0 tx=0x7eff74010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74004220 con 0x7eff8c0cc070 2023-11-06T03:08:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74004380 con 0x7eff8c0cc070 2023-11-06T03:08:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c0aa6c0 con 0x7eff8c0cc070 2023-11-06T03:08:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400eb60 con 0x7eff8c0cc070 2023-11-06T03:08:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c154af0 con 0x7eff8c0cc070 2023-11-06T03:08:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c072750 con 0x7eff8c0cc070 2023-11-06T03:08:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7eff7400ecc0 con 0x7eff8c0cc070 2023-11-06T03:08:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 0x7eff6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7eff74083270 con 0x7eff8c0cc070 2023-11-06T03:08:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8bfff640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 0x7eff6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.837+0000 7eff8bfff640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 0x7eff6c067f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7eff84000f70 tx=0x7eff84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:25.841+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7eff74050b20 con 0x7eff8c0cc070 2023-11-06T03:08:26.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.025+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff8c0d1e10 con 0x7eff8c0cc070 2023-11-06T03:08:26.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.025+0000 7eff88ff9640 1 -- 172.21.15.157:0/3688551642 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4661 (secure 0 0 0) 0x7eff740044f0 con 0x7eff8c0cc070 2023-11-06T03:08:26.026 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:26.027 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:20.421381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:26.029 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-11-06T03:08:26.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 msgr2=0x7eff6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 0x7eff6c067f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7eff84000f70 tx=0x7eff84009040 comp rx=0 tx=0).stop 2023-11-06T03:08:26.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 msgr2=0x7eff8c0ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7eff74009fd0 tx=0x7eff74010d60 comp rx=0 tx=0).stop 2023-11-06T03:08:26.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 shutdown_connections 2023-11-06T03:08:26.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff6c065a60 0x7eff6c067f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff8c0cc4d0 0x7eff8c0ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c0cc070 0x7eff8c0ab150 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 --2- 172.21.15.157:0/3688551642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff78005190 0x7eff8c1f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 >> 172.21.15.157:0/3688551642 conn(0x7eff8c070790 msgr2=0x7eff8c070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 shutdown_connections 2023-11-06T03:08:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.029+0000 7eff924d8640 1 -- 172.21.15.157:0/3688551642 wait complete. 2023-11-06T03:08:26.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 1412960396 2023-11-06T03:08:26.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 Processor -- start 2023-11-06T03:08:26.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 -- start start 2023-11-06T03:08:26.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa0800adf70 0x7fa08015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0800cad20 0x7fa0800cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08015e100 con 0x7fa0800ccb70 2023-11-06T03:08:26.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0800cb690 con 0x7fa0800cad20 2023-11-06T03:08:26.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa086616640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08015cff0 con 0x7fa0800adf70 2023-11-06T03:08:26.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0800cad20 0x7fa0800cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0800cad20 0x7fa0800cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58796/0 (socket says 172.21.15.157:58796) 2023-11-06T03:08:26.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 -- 172.21.15.157:0/3128946830 learned_addr learned my addr 172.21.15.157:0/3128946830 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa07ffff640 1 --2- 172.21.15.157:0/3128946830 >> v2:172.21.15.195:3300/0 conn(0x7fa0800adf70 0x7fa08015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 -- 172.21.15.157:0/3128946830 >> v2:172.21.15.195:3300/0 conn(0x7fa0800adf70 msgr2=0x7fa08015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 --2- 172.21.15.157:0/3128946830 >> v2:172.21.15.195:3300/0 conn(0x7fa0800adf70 0x7fa08015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 -- 172.21.15.157:0/3128946830 >> v1:172.21.15.195:6789/0 conn(0x7fa0800ccb70 legacy=0x7fa080161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.161+0000 7fa084bb3640 1 -- 172.21.15.157:0/3128946830 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800cd130 con 0x7fa0800cad20 2023-11-06T03:08:26.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa084bb3640 1 --2- 172.21.15.157:0/3128946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800cad20 0x7fa0800cb150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c9580 tx=0x7fa07801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d9d094a6b396ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07effd640 1 -- 172.21.15.157:0/3128946830 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0780a55a0 con 0x7fa0800cad20 2023-11-06T03:08:26.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07effd640 1 -- 172.21.15.157:0/3128946830 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0780a5700 con 0x7fa0800cad20 2023-11-06T03:08:26.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07effd640 1 -- 172.21.15.157:0/3128946830 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0780a5a90 con 0x7fa0800cad20 2023-11-06T03:08:26.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- 172.21.15.157:0/3128946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800cad20 msgr2=0x7fa0800cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- 172.21.15.157:0/3128946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800cad20 0x7fa0800cb150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c9580 tx=0x7fa07801c8e0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- 172.21.15.157:0/3128946830 shutdown_connections 2023-11-06T03:08:26.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- 172.21.15.157:0/3128946830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800cad20 0x7fa0800cb150 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- 172.21.15.157:0/3128946830 >> v2:172.21.15.195:3300/0 conn(0x7fa0800adf70 0x7fa08015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- 172.21.15.157:0/3128946830 >> 172.21.15.157:0/3128946830 conn(0x7fa0800b0b50 msgr2=0x7fa0800b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- 172.21.15.157:0/3128946830 shutdown_connections 2023-11-06T03:08:26.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- 172.21.15.157:0/3128946830 wait complete. 2023-11-06T03:08:26.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 Processor -- start 2023-11-06T03:08:26.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- start start 2023-11-06T03:08:26.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0800cad20 0x7fa0801524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0800ccb70 0x7fa0801578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006bdc0 con 0x7fa0800adf70 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800709b0 con 0x7fa0800ccb70 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa086616640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006c060 con 0x7fa0800cad20 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58808/0 (socket says 172.21.15.157:58808) 2023-11-06T03:08:26.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.165+0000 7fa07ffff640 1 -- 172.21.15.157:0/1368281503 learned_addr learned my addr 172.21.15.157:0/1368281503 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 -- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0800cad20 msgr2=0x7fa0801524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0800cad20 0x7fa0801524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 -- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0800ccb70 msgr2=0x7fa0801578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0800ccb70 0x7fa0801578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 -- 172.21.15.157:0/1368281503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa078009b30 con 0x7fa0800adf70 2023-11-06T03:08:26.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07ffff640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa07400d800 tx=0x7fa07400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 -- 172.21.15.157:0/1368281503 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400b6c0 con 0x7fa0800adf70 2023-11-06T03:08:26.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 -- 172.21.15.157:0/1368281503 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07400b820 con 0x7fa0800adf70 2023-11-06T03:08:26.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08006c340 con 0x7fa0800adf70 2023-11-06T03:08:26.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 -- 172.21.15.157:0/1368281503 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400bb90 con 0x7fa0800adf70 2023-11-06T03:08:26.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa080070c90 con 0x7fa0800adf70 2023-11-06T03:08:26.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 -- 172.21.15.157:0/1368281503 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa07400bcf0 con 0x7fa0800adf70 2023-11-06T03:08:26.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 0x7fa06c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07cff9640 1 -- 172.21.15.157:0/1368281503 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa0740838c0 con 0x7fa0800adf70 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 0x7fa06c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa08973e000 1 Processor -- start 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa08973e000 1 -- start start 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.169+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 0x7fa06c067f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa08015caf0 tx=0x7fa070006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff35bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfdfcec0 con 0x559bbff7f4b0 2023-11-06T03:08:26.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd8a3b0 con 0x559bbff357d0 2023-11-06T03:08:26.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd87980 con 0x559bbff7fdd0 2023-11-06T03:08:26.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58814/0 (socket says 172.21.15.157:58814) 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/3399729241 learned_addr learned my addr 172.21.15.157:0/3399729241 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa084bb3640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff35bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 msgr2=0x559bbff35290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff35290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 msgr2=0x559bbff35bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff35bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/3399729241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbfd24040 con 0x559bbff7f4b0 2023-11-06T03:08:26.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa084bb3640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff35bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:26.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa07405c850 tx=0x7fa074054830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad92fa2ce284173e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa065ffb640 1 -- 172.21.15.157:0/3399729241 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07405cd80 con 0x559bbff7f4b0 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa065ffb640 1 -- 172.21.15.157:0/3399729241 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074053140 con 0x559bbff7f4b0 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa065ffb640 1 -- 172.21.15.157:0/3399729241 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07406c1c0 con 0x559bbff7f4b0 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/3399729241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 msgr2=0x559bbff7f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa07405c850 tx=0x7fa074054830 comp rx=0 tx=0).stop 2023-11-06T03:08:26.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/3399729241 shutdown_connections 2023-11-06T03:08:26.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff35bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff35290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- 172.21.15.157:0/3399729241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff7f890 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/3399729241 >> 172.21.15.157:0/3399729241 conn(0x559bbff31780 msgr2=0x559bbff33b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/3399729241 shutdown_connections 2023-11-06T03:08:26.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/3399729241 wait complete. 2023-11-06T03:08:26.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- start start 2023-11-06T03:08:26.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 Processor -- start 2023-11-06T03:08:26.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbff86d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbff6d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd83270 con 0x559bbff357d0 2023-11-06T03:08:26.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd226d0 con 0x559bbff7fdd0 2023-11-06T03:08:26.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd22370 con 0x559bbff7f4b0 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 10 client.0 _ll_register_callbacks cb 0x559bbff20d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58816/0 (socket says 172.21.15.157:58816) 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/468579956 learned_addr learned my addr 172.21.15.157:0/468579956 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa084bb3640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbff6d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbff86d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 msgr2=0x559bbff86d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbff86d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 msgr2=0x559bbff6d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbff6d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 -- 172.21.15.157:0/468579956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa080070c90 con 0x559bbff357d0 2023-11-06T03:08:26.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa07ffff640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa0740512a0 tx=0x7fa074081b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4a4c37327acd37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa05effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:26.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07405cd80 con 0x559bbff357d0 2023-11-06T03:08:26.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa07406c1c0 con 0x559bbff357d0 2023-11-06T03:08:26.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bbfd28b90 con 0x559bbff357d0 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07406c380 con 0x559bbff357d0 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.173+0000 7fa08973e000 10 client.6534 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x559bbfa7fb30 con 0x559bbff357d0 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bbfd28b90 con 0x559bbff357d0 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa07405c8f0 con 0x559bbff357d0 2023-11-06T03:08:26.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 10 client.6534 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:26.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 10 client.6534 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:26.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 20 client.6534 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:26.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbff6e000 0x559bbff6e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 4 client.6534 mds_command: new command op to 5868 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:26.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bbfd81bc0 con 0x559bbff6e000 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbff6e000 0x559bbff6e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 10 client.6534 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbff6e000 0x559bbff6e3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32adaff2073e693c server_cookie=cc6a5aba032ec9ef in_seq=0 out_seq=0 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 1 -- 172.21.15.157:0/468579956 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559bbfd81bc0 con 0x559bbff6e000 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 10 client.6534 handle_command_reply: tid=0 2023-11-06T03:08:26.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 client.6534 shutdown 2023-11-06T03:08:26.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 msgr2=0x559bbff86810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa0740512a0 tx=0x7fa074081b20 comp rx=0 tx=0).stop 2023-11-06T03:08:26.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 shutdown_connections 2023-11-06T03:08:26.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbff6e000 0x559bbff6e3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbff6d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa05effd640 0 client.6534 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:26.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbff86d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 --2- 172.21.15.157:0/468579956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbff86810 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 >> 172.21.15.157:0/468579956 conn(0x559bbff31780 msgr2=0x559bbff33b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 shutdown_connections 2023-11-06T03:08:26.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 1 -- 172.21.15.157:0/468579956 wait complete. 2023-11-06T03:08:26.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.177+0000 7fa08973e000 20 client.6534 trim_cache size 0 max 16384 2023-11-06T03:08:26.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 Processor -- start 2023-11-06T03:08:26.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- start start 2023-11-06T03:08:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbffd8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbffd8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfa40be0 con 0x559bbff357d0 2023-11-06T03:08:26.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbff2cc00 con 0x559bbff7fdd0 2023-11-06T03:08:26.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbff392f0 con 0x559bbff7f4b0 2023-11-06T03:08:26.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58822/0 (socket says 172.21.15.157:58822) 2023-11-06T03:08:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 -- 172.21.15.157:0/2397244274 learned_addr learned my addr 172.21.15.157:0/2397244274 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 -- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 msgr2=0x559bbffd8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbffd8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 -- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 msgr2=0x559bbffd8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbffd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 -- 172.21.15.157:0/2397244274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbfd481a0 con 0x559bbff357d0 2023-11-06T03:08:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa07ffff640 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa074083ba0 tx=0x7fa074015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7500420cf4298f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa0667fc640 1 -- 172.21.15.157:0/2397244274 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740548e0 con 0x559bbff357d0 2023-11-06T03:08:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa0667fc640 1 -- 172.21.15.157:0/2397244274 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07405cd80 con 0x559bbff357d0 2023-11-06T03:08:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa0667fc640 1 -- 172.21.15.157:0/2397244274 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074054da0 con 0x559bbff357d0 2023-11-06T03:08:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- 172.21.15.157:0/2397244274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 msgr2=0x559bbffd7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa074083ba0 tx=0x7fa074015000 comp rx=0 tx=0).stop 2023-11-06T03:08:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- 172.21.15.157:0/2397244274 shutdown_connections 2023-11-06T03:08:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff7fdd0 0x559bbffd8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7f4b0 0x559bbffd8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- 172.21.15.157:0/2397244274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff357d0 0x559bbffd7ae0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- 172.21.15.157:0/2397244274 >> 172.21.15.157:0/2397244274 conn(0x559bbff31780 msgr2=0x559bbff6ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- 172.21.15.157:0/2397244274 shutdown_connections 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- 172.21.15.157:0/2397244274 wait complete. 2023-11-06T03:08:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- start start 2023-11-06T03:08:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 Processor -- start 2023-11-06T03:08:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff37220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd861d0 con 0x559bbff7f4b0 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd80a80 con 0x559bbff357d0 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bbfd4d5e0 con 0x559bbff7fdd0 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.189+0000 7fa08973e000 10 client.0 _ll_register_callbacks cb 0x559bbfd4b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58832/0 (socket says 172.21.15.157:58832) 2023-11-06T03:08:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 -- 172.21.15.157:0/2542792006 learned_addr learned my addr 172.21.15.157:0/2542792006 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 -- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 msgr2=0x559bbff83750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 -- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 msgr2=0x559bbff37220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff37220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 -- 172.21.15.157:0/2542792006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074078070 con 0x559bbff7f4b0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07f7fe640 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa070022bb0 tx=0x7fa070004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10821e215b3d83b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070018040 con 0x559bbff7f4b0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa0700222b0 con 0x559bbff7f4b0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bbff38c80 con 0x559bbff7f4b0 2023-11-06T03:08:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa070004020 con 0x559bbff7f4b0 2023-11-06T03:08:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 10 client.6540 fetch_fsmap learned FSMap version 39 2023-11-06T03:08:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x559bbfd2e2f0 con 0x559bbff7f4b0 2023-11-06T03:08:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bbff38c80 con 0x559bbff7f4b0 2023-11-06T03:08:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 39) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa070004180 con 0x559bbff7f4b0 2023-11-06T03:08:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 10 client.6540 fetch_fsmap finished waiting for FSMap version 39 2023-11-06T03:08:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 10 client.6540 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 20 client.6540 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbffdbda0 0x559bbffdc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 4 client.6540 mds_command: new command op to 5868 tid=0[{"prefix": "damage rm", "damage_id": 1412960396}] 2023-11-06T03:08:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 --> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1412960396}) v1 -- 0x559bbfd4a4c0 con 0x559bbffdbda0 2023-11-06T03:08:26.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07ffff640 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbffdbda0 0x559bbffdc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa07ffff640 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbffdbda0 0x559bbffdc180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b907f0e70c94a2c2 server_cookie=d41d5807d48af461 in_seq=0 out_seq=0 2023-11-06T03:08:26.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 10 client.6540 ms_handle_connect on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:26.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 1 -- 172.21.15.157:0/2542792006 <== mds.0 v2:172.21.15.157:6834/3341473298 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x559bbfd4a4c0 con 0x559bbffdbda0 2023-11-06T03:08:26.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 10 client.6540 handle_command_reply: tid=0 2023-11-06T03:08:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 client.6540 shutdown 2023-11-06T03:08:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 msgr2=0x559bbff83210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa070022bb0 tx=0x7fa070004700 comp rx=0 tx=0).stop 2023-11-06T03:08:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 shutdown_connections 2023-11-06T03:08:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:6834/3341473298,v1:172.21.15.157:6835/3341473298] conn(0x559bbffdbda0 0x559bbffdc180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559bbff7fdd0 0x559bbff83750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa05effd640 0 client.6540 ms_handle_reset on v2:172.21.15.157:6834/3341473298 2023-11-06T03:08:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559bbff7f4b0 0x559bbff83210 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 --2- 172.21.15.157:0/2542792006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559bbff357d0 0x559bbff37220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 >> 172.21.15.157:0/2542792006 conn(0x559bbff31780 msgr2=0x559bbff39710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 shutdown_connections 2023-11-06T03:08:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 1 -- 172.21.15.157:0/2542792006 wait complete. 2023-11-06T03:08:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa08973e000 20 client.6540 trim_cache size 0 max 16384 2023-11-06T03:08:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 msgr2=0x7fa06c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 0x7fa06c067f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa08015caf0 tx=0x7fa070006bd0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 msgr2=0x7fa080156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa07400d800 tx=0x7fa07400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 shutdown_connections 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa06c065a60 0x7fa06c067f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0800ccb70 0x7fa0801578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0800cad20 0x7fa0801524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 --2- 172.21.15.157:0/1368281503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0800adf70 0x7fa080156d10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 >> 172.21.15.157:0/1368281503 conn(0x7fa0800b0b50 msgr2=0x7fa0800b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 shutdown_connections 2023-11-06T03:08:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.193+0000 7fa086616640 1 -- 172.21.15.157:0/1368281503 wait complete. 2023-11-06T03:08:26.219 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2023-11-06T03:08:26.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:26.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 Processor -- start 2023-11-06T03:08:26.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 -- start start 2023-11-06T03:08:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1b90159950 0x7f1b900b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b9006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9006e190 con 0x7f1b900cc480 2023-11-06T03:08:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900702b0 con 0x7f1b90159950 2023-11-06T03:08:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b954f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90159d30 con 0x7f1b9006d7f0 2023-11-06T03:08:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1b900cc480 0x7f1b9015d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32882/0 (socket says 172.21.15.157:32882) 2023-11-06T03:08:26.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/2582019593 learned_addr learned my addr 172.21.15.157:0/2582019593 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8e7fc640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b9006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 --2- 172.21.15.157:0/2582019593 >> v2:172.21.15.195:3300/0 conn(0x7f1b90159950 0x7f1b900b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486871779 0 0) 0x7f1b9006e190 con 0x7f1b900cc480 2023-11-06T03:08:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b78003530 con 0x7f1b900cc480 2023-11-06T03:08:26.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549194175 0 0) 0x7f1b78003530 con 0x7f1b900cc480 2023-11-06T03:08:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 msgr2=0x7f1b9006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b9006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 >> v2:172.21.15.195:3300/0 conn(0x7f1b90159950 msgr2=0x7f1b900b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 --2- 172.21.15.157:0/2582019593 >> v2:172.21.15.195:3300/0 conn(0x7f1b90159950 0x7f1b900b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900b3f00 con 0x7f1b900cc480 2023-11-06T03:08:26.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1b88001a00 con 0x7f1b900cc480 2023-11-06T03:08:26.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 >> v1:172.21.15.157:6789/0 conn(0x7f1b900cc480 legacy=0x7f1b9015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b78003980 0x7f1b78003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b780047d0 0x7f1b78004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900702b0 con 0x7f1b78003980 2023-11-06T03:08:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90159d30 con 0x7f1b780047d0 2023-11-06T03:08:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7800b990 con 0x7f1b9006d7f0 2023-11-06T03:08:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b78003980 0x7f1b78003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8e7fc640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b780047d0 0x7f1b78004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8f7fe640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 -- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 msgr2=0x7f1b78005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b9006d7f0 0x7f1b78005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 -- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b780047d0 msgr2=0x7f1b78004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b780047d0 0x7f1b78004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8effd640 1 -- 172.21.15.157:0/2582019593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78003530 con 0x7f1b78003980 2023-11-06T03:08:26.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.325+0000 7f1b8e7fc640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b780047d0 0x7f1b78004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:26.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8effd640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b78003980 0x7f1b78003da0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009ae0 tx=0x7f1b8001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83f17df2b77615a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800b0400 con 0x7f1b78003980 2023-11-06T03:08:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800b0560 con 0x7f1b78003980 2023-11-06T03:08:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8dffb640 1 -- 172.21.15.157:0/2582019593 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800b0910 con 0x7f1b78003980 2023-11-06T03:08:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- 172.21.15.157:0/2582019593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b78003980 msgr2=0x7f1b78003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 --2- 172.21.15.157:0/2582019593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b78003980 0x7f1b78003da0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009ae0 tx=0x7f1b8001ca90 comp rx=0 tx=0).stop 2023-11-06T03:08:26.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/2582019593 reap_dead start 2023-11-06T03:08:26.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- 172.21.15.157:0/2582019593 shutdown_connections 2023-11-06T03:08:26.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- 172.21.15.157:0/2582019593 >> 172.21.15.157:0/2582019593 conn(0x7f1b9006a170 msgr2=0x7f1b9006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- 172.21.15.157:0/2582019593 shutdown_connections 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- 172.21.15.157:0/2582019593 wait complete. 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 Processor -- start 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- start start 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 0x7f1b9015fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b90159ed0 0x7f1b900c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c6140 con 0x7f1b9015a350 2023-11-06T03:08:26.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c33a0 con 0x7f1b78003980 2023-11-06T03:08:26.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b954f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9015c910 con 0x7f1b90159ed0 2023-11-06T03:08:26.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58838/0 (socket says 172.21.15.157:58838) 2023-11-06T03:08:26.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.329+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/1941709732 learned_addr learned my addr 172.21.15.157:0/1941709732 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8effd640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 0x7f1b9015fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8e7fc640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b90159ed0 0x7f1b900c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b90159ed0 msgr2=0x7f1b900c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b90159ed0 0x7f1b900c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 msgr2=0x7f1b9015fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 0x7f1b9015fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 -- 172.21.15.157:0/1941709732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9015cbf0 con 0x7f1b9015a350 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8effd640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 0x7f1b9015fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8f7fe640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cd050 tx=0x7f1b8800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88004d70 con 0x7f1b9015a350 2023-11-06T03:08:26.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800bb80 con 0x7f1b9015a350 2023-11-06T03:08:26.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900c3640 con 0x7f1b9015a350 2023-11-06T03:08:26.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88007110 con 0x7f1b9015a350 2023-11-06T03:08:26.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b90160380 con 0x7f1b9015a350 2023-11-06T03:08:26.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b90072750 con 0x7f1b9015a350 2023-11-06T03:08:26.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b88007270 con 0x7f1b9015a350 2023-11-06T03:08:26.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 0x7f1b7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1b880836e0 con 0x7f1b9015a350 2023-11-06T03:08:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8effd640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 0x7f1b7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.333+0000 7f1b8effd640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 0x7f1b7c067f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b800af000 tx=0x7f1b800b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.337+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b88050f90 con 0x7f1b9015a350 2023-11-06T03:08:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.512+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b9015cd80 con 0x7f1b9015a350 2023-11-06T03:08:26.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.512+0000 7f1b777fe640 1 -- 172.21.15.157:0/1941709732 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f1b880545b0 con 0x7f1b9015a350 2023-11-06T03:08:26.513 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:26.513 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:20.421381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":13,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:26.513 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 39 2023-11-06T03:08:26.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 msgr2=0x7f1b7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 0x7f1b7c067f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b800af000 tx=0x7f1b800b2040 comp rx=0 tx=0).stop 2023-11-06T03:08:26.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 msgr2=0x7f1b900ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cd050 tx=0x7f1b8800b4a0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 shutdown_connections 2023-11-06T03:08:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b7c065a60 0x7f1b7c067f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b9015a350 0x7f1b900ca3e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b90159ed0 0x7f1b900c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 --2- 172.21.15.157:0/1941709732 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b78003980 0x7f1b9015fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 >> 172.21.15.157:0/1941709732 conn(0x7f1b9006a170 msgr2=0x7f1b90153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 shutdown_connections 2023-11-06T03:08:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.516+0000 7f1b954f3640 1 -- 172.21.15.157:0/1941709732 wait complete. 2023-11-06T03:08:26.530 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2023-11-06T03:08:26.641 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:26.642 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:08:26.642 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:26.654 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2023-11-06T03:08:26.655 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:08:26.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:08:26.687 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:08:26.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:08:26.741 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:26.742 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:26.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:26.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.916+0000 7f9789ffd640 1 Processor -- start 2023-11-06T03:08:26.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.916+0000 7f9789ffd640 1 -- start start 2023-11-06T03:08:26.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9789ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f97841551e0 0x7f978406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9789ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 0x7f97840b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9789ffd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978415f580 con 0x7f978415ea60 2023-11-06T03:08:26.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9789ffd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978415efc0 con 0x7f97841551e0 2023-11-06T03:08:26.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9789ffd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978415f2a0 con 0x7f978415e5e0 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97837fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f97841551e0 0x7f978406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9782ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 0x7f97840b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9783fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f978415ea60 0x7f978415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35746/0 (socket says 172.21.15.157:35746) 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9783fff640 1 -- 172.21.15.157:0/1238571095 learned_addr learned my addr 172.21.15.157:0/1238571095 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97827fc640 1 -- 172.21.15.157:0/1238571095 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761845851 0 0) 0x7f978415f580 con 0x7f978415ea60 2023-11-06T03:08:26.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97827fc640 1 -- 172.21.15.157:0/1238571095 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9770003530 con 0x7f978415ea60 2023-11-06T03:08:26.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97837fe640 1 -- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 msgr2=0x7f97840b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:26.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97837fe640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 0x7f97840b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97837fe640 1 -- 172.21.15.157:0/1238571095 >> v1:172.21.15.195:6789/0 conn(0x7f978415ea60 legacy=0x7f978415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f97837fe640 1 -- 172.21.15.157:0/1238571095 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978406f5a0 con 0x7f97841551e0 2023-11-06T03:08:26.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.920+0000 7f9782ffd640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 0x7f97840b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:08:26.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97837fe640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97841551e0 0x7f978406fda0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f976c0097c0 tx=0x7f976c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fafc651fb4ca20a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97827fc640 1 -- 172.21.15.157:0/1238571095 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0ab600 con 0x7f97841551e0 2023-11-06T03:08:26.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97827fc640 1 -- 172.21.15.157:0/1238571095 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c0a7050 con 0x7f97841551e0 2023-11-06T03:08:26.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97827fc640 1 -- 172.21.15.157:0/1238571095 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0ab980 con 0x7f97841551e0 2023-11-06T03:08:26.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97841551e0 msgr2=0x7f978406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97841551e0 0x7f978406fda0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f976c0097c0 tx=0x7f976c01c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- 172.21.15.157:0/1238571095 shutdown_connections 2023-11-06T03:08:26.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415e5e0 0x7f97840b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1238571095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97841551e0 0x7f978406fda0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- 172.21.15.157:0/1238571095 >> 172.21.15.157:0/1238571095 conn(0x7f9784065930 msgr2=0x7f9784065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:26.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- 172.21.15.157:0/1238571095 shutdown_connections 2023-11-06T03:08:26.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- 172.21.15.157:0/1238571095 wait complete. 2023-11-06T03:08:26.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 Processor -- start 2023-11-06T03:08:26.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- start start 2023-11-06T03:08:26.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978415e5e0 0x7f978414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 0x7f9784159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9784158240 con 0x7f97841551e0 2023-11-06T03:08:26.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978414cb20 con 0x7f978415e5e0 2023-11-06T03:08:26.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9789ffd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9784156150 con 0x7f978415ea60 2023-11-06T03:08:26.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97837fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58850/0 (socket says 172.21.15.157:58850) 2023-11-06T03:08:26.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f97837fe640 1 -- 172.21.15.157:0/1595315970 learned_addr learned my addr 172.21.15.157:0/1595315970 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:26.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.924+0000 7f9782ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978415e5e0 0x7f978414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9783fff640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 0x7f9784159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 -- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 msgr2=0x7f9784159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 0x7f9784159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 -- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978415e5e0 msgr2=0x7f978414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:26.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978415e5e0 0x7f978414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:26.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 -- 172.21.15.157:0/1595315970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774009420 con 0x7f97841551e0 2023-11-06T03:08:26.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9783fff640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 0x7f9784159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:26.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f97837fe640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f976c0ae040 tx=0x7f976c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0b7450 con 0x7f97841551e0 2023-11-06T03:08:26.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c0b75d0 con 0x7f97841551e0 2023-11-06T03:08:26.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c009420 con 0x7f97841551e0 2023-11-06T03:08:26.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0b7980 con 0x7f97841551e0 2023-11-06T03:08:26.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9784156710 con 0x7f97841551e0 2023-11-06T03:08:26.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784072750 con 0x7f97841551e0 2023-11-06T03:08:26.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f976c0b7ae0 con 0x7f97841551e0 2023-11-06T03:08:26.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 0x7f9764067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:26.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f976c11dac0 con 0x7f97841551e0 2023-11-06T03:08:26.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9782ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 0x7f9764067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:26.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.928+0000 7f9782ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 0x7f9764067f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f978415bcb0 tx=0x7f9774008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:26.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:26.932+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f976c0eb2e0 con 0x7f97841551e0 2023-11-06T03:08:27.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.076+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9784071730 con 0x7f97841551e0 2023-11-06T03:08:27.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f977bfff640 1 -- 172.21.15.157:0/1595315970 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f976c0b0090 con 0x7f97841551e0 2023-11-06T03:08:27.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:27.079 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:50.274092+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:27.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 msgr2=0x7f9764067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 0x7f9764067f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f978415bcb0 tx=0x7f9774008040 comp rx=0 tx=0).stop 2023-11-06T03:08:27.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 msgr2=0x7f97840ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f976c0ae040 tx=0x7f976c01ac00 comp rx=0 tx=0).stop 2023-11-06T03:08:27.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 shutdown_connections 2023-11-06T03:08:27.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9764065a60 0x7f9764067f20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978415ea60 0x7f9784159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978415e5e0 0x7f978414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 --2- 172.21.15.157:0/1595315970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97841551e0 0x7f97840ce8a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 >> 172.21.15.157:0/1595315970 conn(0x7f9784065930 msgr2=0x7f97840b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 shutdown_connections 2023-11-06T03:08:27.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.080+0000 7f9789ffd640 1 -- 172.21.15.157:0/1595315970 wait complete. 2023-11-06T03:08:27.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:07:37.373400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:07:38.627196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:08:27.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:08:27.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 Processor -- start 2023-11-06T03:08:27.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 -- start start 2023-11-06T03:08:27.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feb081599c0 0x7feb080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb0806d000 0x7feb080b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0815dea0 0x7feb080b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0815e440 con 0x7feb0806d000 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0806c240 con 0x7feb081599c0 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0c91f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0806c520 con 0x7feb0815dea0 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb0806d000 0x7feb080b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb0806d000 0x7feb080b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58866/0 (socket says 172.21.15.157:58866) 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 -- 172.21.15.157:0/422902661 learned_addr learned my addr 172.21.15.157:0/422902661 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb0659c640 1 --2- 172.21.15.157:0/422902661 >> v2:172.21.15.195:3300/0 conn(0x7feb081599c0 0x7feb080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 -- 172.21.15.157:0/422902661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0815dea0 msgr2=0x7feb080b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 --2- 172.21.15.157:0/422902661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0815dea0 0x7feb080b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 -- 172.21.15.157:0/422902661 >> v2:172.21.15.195:3300/0 conn(0x7feb081599c0 msgr2=0x7feb080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 --2- 172.21.15.157:0/422902661 >> v2:172.21.15.195:3300/0 conn(0x7feb081599c0 0x7feb080ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 -- 172.21.15.157:0/422902661 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0806c800 con 0x7feb0806d000 2023-11-06T03:08:27.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feafdd9b640 1 --2- 172.21.15.157:0/422902661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0806d000 0x7feb080b23e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb08066800 tx=0x7feaf001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c02e51bf414e2a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb05d9b640 1 -- 172.21.15.157:0/422902661 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00aa610 con 0x7feb0806d000 2023-11-06T03:08:27.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb05d9b640 1 -- 172.21.15.157:0/422902661 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf00a6040 con 0x7feb0806d000 2023-11-06T03:08:27.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.212+0000 7feb05d9b640 1 -- 172.21.15.157:0/422902661 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00aa990 con 0x7feb0806d000 2023-11-06T03:08:27.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/422902661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0806d000 msgr2=0x7feb080b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- 172.21.15.157:0/422902661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0806d000 0x7feb080b23e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb08066800 tx=0x7feaf001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/422902661 shutdown_connections 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- 172.21.15.157:0/422902661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0815dea0 0x7feb080b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- 172.21.15.157:0/422902661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0806d000 0x7feb080b23e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- 172.21.15.157:0/422902661 >> v2:172.21.15.195:3300/0 conn(0x7feb081599c0 0x7feb080ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/422902661 >> 172.21.15.157:0/422902661 conn(0x7feb08065930 msgr2=0x7feb08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/422902661 shutdown_connections 2023-11-06T03:08:27.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/422902661 wait complete. 2023-11-06T03:08:27.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 Processor -- start 2023-11-06T03:08:27.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- start start 2023-11-06T03:08:27.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0806d000 0x7feb0814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 0x7feb0815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081591a0 con 0x7feb0815dea0 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0814df20 con 0x7feb081599c0 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0814d310 con 0x7feb0806d000 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58870/0 (socket says 172.21.15.157:58870) 2023-11-06T03:08:27.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 -- 172.21.15.157:0/3578645335 learned_addr learned my addr 172.21.15.157:0/3578645335 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feafdd9b640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 0x7feb0815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0659c640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0806d000 0x7feb0814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 -- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0806d000 msgr2=0x7feb0814f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0806d000 0x7feb0814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 -- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 msgr2=0x7feb0815b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 0x7feb0815b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 -- 172.21.15.157:0/3578645335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00094d0 con 0x7feb0815dea0 2023-11-06T03:08:27.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feafdd9b640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 0x7feb0815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:27.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb06d9d640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b970 tx=0x7feb0000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000cb40 con 0x7feb0815dea0 2023-11-06T03:08:27.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000ccc0 con 0x7feb0815dea0 2023-11-06T03:08:27.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0814d5f0 con 0x7feb0815dea0 2023-11-06T03:08:27.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00014680 con 0x7feb0815dea0 2023-11-06T03:08:27.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0814dbb0 con 0x7feb0815dea0 2023-11-06T03:08:27.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.216+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0814e090 con 0x7feb0815dea0 2023-11-06T03:08:27.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.220+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb0001d3b0 con 0x7feb0815dea0 2023-11-06T03:08:27.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.220+0000 7feaff7fe640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 0x7feae8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.220+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7feb00083210 con 0x7feb0815dea0 2023-11-06T03:08:27.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.220+0000 7feb0659c640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 0x7feae8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.220+0000 7feb0659c640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 0x7feae8067f20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b800 tx=0x7feaf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.224+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feb00050ac0 con 0x7feb0815dea0 2023-11-06T03:08:27.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.384+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb08072750 con 0x7feb0815dea0 2023-11-06T03:08:27.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.384+0000 7feaff7fe640 1 -- 172.21.15.157:0/3578645335 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb00019020 con 0x7feb0815dea0 2023-11-06T03:08:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:08:27.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 msgr2=0x7feae8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 0x7feae8067f20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b800 tx=0x7feaf8008040 comp rx=0 tx=0).stop 2023-11-06T03:08:27.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 msgr2=0x7feb08159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b970 tx=0x7feb0000be30 comp rx=0 tx=0).stop 2023-11-06T03:08:27.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 shutdown_connections 2023-11-06T03:08:27.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feae8065a60 0x7feae8067f20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb0815dea0 0x7feb08159da0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb081599c0 0x7feb0815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 --2- 172.21.15.157:0/3578645335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb0806d000 0x7feb0814f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 >> 172.21.15.157:0/3578645335 conn(0x7feb08065930 msgr2=0x7feb0806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 shutdown_connections 2023-11-06T03:08:27.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.388+0000 7feb0c91f640 1 -- 172.21.15.157:0/3578645335 wait complete. 2023-11-06T03:08:27.404 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:08:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 Processor -- start 2023-11-06T03:08:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 -- start start 2023-11-06T03:08:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 0x7fdae40c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdae4072cf0 0x7fdae4164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40ccf90 con 0x7fdae4156da0 2023-11-06T03:08:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae40c9ea0 con 0x7fdae4072cf0 2023-11-06T03:08:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaebafc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae415a6a0 con 0x7fdae40c94d0 2023-11-06T03:08:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdae4072cf0 0x7fdae4164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdae4072cf0 0x7fdae4164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58880/0 (socket says 172.21.15.157:58880) 2023-11-06T03:08:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 -- 172.21.15.157:0/2857270622 learned_addr learned my addr 172.21.15.157:0/2857270622 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdae9097640 1 --2- 172.21.15.157:0/2857270622 >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 0x7fdae40c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 -- 172.21.15.157:0/2857270622 >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 msgr2=0x7fdae40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 --2- 172.21.15.157:0/2857270622 >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 0x7fdae40c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 -- 172.21.15.157:0/2857270622 >> v1:172.21.15.195:6789/0 conn(0x7fdae4156da0 legacy=0x7fdae4159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 -- 172.21.15.157:0/2857270622 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40cd270 con 0x7fdae4072cf0 2023-11-06T03:08:27.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdae9097640 1 --2- 172.21.15.157:0/2857270622 >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 0x7fdae40c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.528+0000 7fdaea099640 1 --2- 172.21.15.157:0/2857270622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4072cf0 0x7fdae4164000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdae40cee80 tx=0x7fdadc01eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5850c8f9de336774 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae8896640 1 -- 172.21.15.157:0/2857270622 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00b2d0 con 0x7fdae4072cf0 2023-11-06T03:08:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae8896640 1 -- 172.21.15.157:0/2857270622 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc00b430 con 0x7fdae4072cf0 2023-11-06T03:08:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae8896640 1 -- 172.21.15.157:0/2857270622 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0a7bf0 con 0x7fdae4072cf0 2023-11-06T03:08:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- 172.21.15.157:0/2857270622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4072cf0 msgr2=0x7fdae4164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2857270622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4072cf0 0x7fdae4164000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fdae40cee80 tx=0x7fdadc01eb00 comp rx=0 tx=0).stop 2023-11-06T03:08:27.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- 172.21.15.157:0/2857270622 shutdown_connections 2023-11-06T03:08:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2857270622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae4072cf0 0x7fdae4164000 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2857270622 >> v2:172.21.15.195:3300/0 conn(0x7fdae40c94d0 0x7fdae40c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- 172.21.15.157:0/2857270622 >> 172.21.15.157:0/2857270622 conn(0x7fdae40cb990 msgr2=0x7fdae40cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- 172.21.15.157:0/2857270622 shutdown_connections 2023-11-06T03:08:27.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- 172.21.15.157:0/2857270622 wait complete. 2023-11-06T03:08:27.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 Processor -- start 2023-11-06T03:08:27.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- start start 2023-11-06T03:08:27.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdae4072cf0 0x7fdae4149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdae4156da0 0x7fdae4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae414e160 con 0x7fdae40c94d0 2023-11-06T03:08:27.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae414ece0 con 0x7fdae4156da0 2023-11-06T03:08:27.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaebafc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae40afb40 con 0x7fdae4072cf0 2023-11-06T03:08:27.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58892/0 (socket says 172.21.15.157:58892) 2023-11-06T03:08:27.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 -- 172.21.15.157:0/2960649327 learned_addr learned my addr 172.21.15.157:0/2960649327 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9898640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdae4072cf0 0x7fdae4149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdaea099640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdae4156da0 0x7fdae4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 -- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdae4072cf0 msgr2=0x7fdae4149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdae4072cf0 0x7fdae4149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 -- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdae4156da0 msgr2=0x7fdae4158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdae4156da0 0x7fdae4158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 -- 172.21.15.157:0/2960649327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc00bd40 con 0x7fdae40c94d0 2023-11-06T03:08:27.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdae9097640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c7f0 tx=0x7fdad400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad40077c0 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.532+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4007920 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae414e3b0 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007cd0 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae414e970 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae4073540 con 0x7fdae40c94d0 2023-11-06T03:08:27.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdad40113f0 con 0x7fdae40c94d0 2023-11-06T03:08:27.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdada7fc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 0x7fdac8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdad40835c0 con 0x7fdae40c94d0 2023-11-06T03:08:27.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdae9898640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 0x7fdac8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.536+0000 7fdae9898640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 0x7fdac8067f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdae0004e30 tx=0x7fdae0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.540+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdad4050e70 con 0x7fdae40c94d0 2023-11-06T03:08:27.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.696+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdae4155a80 con 0x7fdae40c94d0 2023-11-06T03:08:27.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.696+0000 7fdada7fc640 1 -- 172.21.15.157:0/2960649327 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdad4054490 con 0x7fdae40c94d0 2023-11-06T03:08:27.697 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:08:27.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 msgr2=0x7fdac8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 0x7fdac8067f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdae0004e30 tx=0x7fdae0009040 comp rx=0 tx=0).stop 2023-11-06T03:08:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 msgr2=0x7fdae4158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c7f0 tx=0x7fdad400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 shutdown_connections 2023-11-06T03:08:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdac8065a60 0x7fdac8067f20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdae4156da0 0x7fdae4158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdae40c94d0 0x7fdae4158630 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 --2- 172.21.15.157:0/2960649327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdae4072cf0 0x7fdae4149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 >> 172.21.15.157:0/2960649327 conn(0x7fdae40cb990 msgr2=0x7fdae406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 shutdown_connections 2023-11-06T03:08:27.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.700+0000 7fdaebafc640 1 -- 172.21.15.157:0/2960649327 wait complete. 2023-11-06T03:08:27.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:08:27.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 Processor -- start 2023-11-06T03:08:27.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 -- start start 2023-11-06T03:08:27.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31c815ce30 0x7f31c80b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c80cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c815d210 con 0x7f31c80cd520 2023-11-06T03:08:27.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c80d17f0 con 0x7f31c815ce30 2023-11-06T03:08:27.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31cda54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80ce650 con 0x7f31c80cfc90 2023-11-06T03:08:27.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31c815ce30 0x7f31c80b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31c815ce30 0x7f31c80b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58898/0 (socket says 172.21.15.157:58898) 2023-11-06T03:08:27.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 -- 172.21.15.157:0/1320761535 learned_addr learned my addr 172.21.15.157:0/1320761535 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 -- 172.21.15.157:0/1320761535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 msgr2=0x7f31c80cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c77fe640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c80cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c80cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 -- 172.21.15.157:0/1320761535 >> v1:172.21.15.195:6789/0 conn(0x7f31c80cd520 legacy=0x7f31c80cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:27.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.840+0000 7f31c6ffd640 1 -- 172.21.15.157:0/1320761535 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c80ce090 con 0x7f31c815ce30 2023-11-06T03:08:27.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c80b5f60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009870 tx=0x7f31b001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2838916c958e64 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31c5ffb640 1 -- 172.21.15.157:0/1320761535 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ab600 con 0x7f31c815ce30 2023-11-06T03:08:27.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31c5ffb640 1 -- 172.21.15.157:0/1320761535 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b00a7050 con 0x7f31c815ce30 2023-11-06T03:08:27.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31c5ffb640 1 -- 172.21.15.157:0/1320761535 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ab980 con 0x7f31c815ce30 2023-11-06T03:08:27.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- 172.21.15.157:0/1320761535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 msgr2=0x7f31c80b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c80b5f60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009870 tx=0x7f31b001c470 comp rx=0 tx=0).stop 2023-11-06T03:08:27.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- 172.21.15.157:0/1320761535 shutdown_connections 2023-11-06T03:08:27.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c80cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- 172.21.15.157:0/1320761535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c80b5f60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- 172.21.15.157:0/1320761535 >> 172.21.15.157:0/1320761535 conn(0x7f31c8070990 msgr2=0x7f31c8065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:27.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- 172.21.15.157:0/1320761535 shutdown_connections 2023-11-06T03:08:27.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- 172.21.15.157:0/1320761535 wait complete. 2023-11-06T03:08:27.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 Processor -- start 2023-11-06T03:08:27.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- start start 2023-11-06T03:08:27.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 0x7f31c80b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c81501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8152e40 con 0x7f31c815ce30 2023-11-06T03:08:27.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80b5f20 con 0x7f31c80cd520 2023-11-06T03:08:27.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.844+0000 7f31cda54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80b6170 con 0x7f31c80cfc90 2023-11-06T03:08:27.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58908/0 (socket says 172.21.15.157:58908) 2023-11-06T03:08:27.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 -- 172.21.15.157:0/850476018 learned_addr learned my addr 172.21.15.157:0/850476018 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:27.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c67fc640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c81501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 0x7f31c80b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 -- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 msgr2=0x7f31c81501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c81501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 -- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 msgr2=0x7f31c80b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:27.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 0x7f31c80b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:27.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 -- 172.21.15.157:0/850476018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00094d0 con 0x7f31c815ce30 2023-11-06T03:08:27.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 0x7f31c80b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:27.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c77fe640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f31c000cd70 tx=0x7f31c000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0003ba0 con 0x7f31c815ce30 2023-11-06T03:08:27.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0003d00 con 0x7f31c815ce30 2023-11-06T03:08:27.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c80b6450 con 0x7f31c815ce30 2023-11-06T03:08:27.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0004240 con 0x7f31c815ce30 2023-11-06T03:08:27.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c80b6a10 con 0x7f31c815ce30 2023-11-06T03:08:27.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c80b5b50 con 0x7f31c815ce30 2023-11-06T03:08:27.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f31c00043a0 con 0x7f31c815ce30 2023-11-06T03:08:27.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 0x7f31a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:27.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f31c0012070 con 0x7f31c815ce30 2023-11-06T03:08:27.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.848+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 0x7f31a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:27.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.852+0000 7f31c6ffd640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 0x7f31a8067f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31b00ad000 tx=0x7f31b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:27.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:27.852+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f31c00514c0 con 0x7f31c815ce30 2023-11-06T03:08:28.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31c8072750 con 0x7f31c815ce30 2023-11-06T03:08:28.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31bffff640 1 -- 172.21.15.157:0/850476018 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31c0054ae0 con 0x7f31c815ce30 2023-11-06T03:08:28.004 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:08:28.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 msgr2=0x7f31a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 0x7f31a8067f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31b00ad000 tx=0x7f31b0008040 comp rx=0 tx=0).stop 2023-11-06T03:08:28.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 msgr2=0x7f31c815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f31c000cd70 tx=0x7f31c000a660 comp rx=0 tx=0).stop 2023-11-06T03:08:28.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 shutdown_connections 2023-11-06T03:08:28.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31a8065a60 0x7f31a8067f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31c815ce30 0x7f31c815a5b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31c80cfc90 0x7f31c81501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 --2- 172.21.15.157:0/850476018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31c80cd520 0x7f31c80b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 >> 172.21.15.157:0/850476018 conn(0x7f31c8070990 msgr2=0x7f31c8155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:28.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 shutdown_connections 2023-11-06T03:08:28.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.004+0000 7f31cda54640 1 -- 172.21.15.157:0/850476018 wait complete. 2023-11-06T03:08:28.022 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:08:28.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 Processor -- start 2023-11-06T03:08:28.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 -- start start 2023-11-06T03:08:28.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f13280b1d40 0x7f13280cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13280b17f0 0x7f13280cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280c94f0 con 0x7f132815aa80 2023-11-06T03:08:28.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132815ae60 con 0x7f13280b17f0 2023-11-06T03:08:28.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132fe16640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280c97b0 con 0x7f13280b1d40 2023-11-06T03:08:28.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132dbb2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f132815aa80 0x7f13280b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32898/0 (socket says 172.21.15.157:32898) 2023-11-06T03:08:28.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132e3b3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f13280b17f0 0x7f13280cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132dbb2640 1 -- 172.21.15.157:0/3915999893 learned_addr learned my addr 172.21.15.157:0/3915999893 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:28.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132d3b1640 1 --2- 172.21.15.157:0/3915999893 >> v2:172.21.15.195:3300/0 conn(0x7f13280b1d40 0x7f13280cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132e3b3640 1 -- 172.21.15.157:0/3915999893 >> v2:172.21.15.195:3300/0 conn(0x7f13280b1d40 msgr2=0x7f13280cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:28.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132e3b3640 1 --2- 172.21.15.157:0/3915999893 >> v2:172.21.15.195:3300/0 conn(0x7f13280b1d40 0x7f13280cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132e3b3640 1 -- 172.21.15.157:0/3915999893 >> v1:172.21.15.157:6789/0 conn(0x7f132815aa80 legacy=0x7f13280b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132cbb0640 1 -- 172.21.15.157:0/3915999893 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344352575 0 0) 0x7f13280c94f0 con 0x7f132815aa80 2023-11-06T03:08:28.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.144+0000 7f132e3b3640 1 -- 172.21.15.157:0/3915999893 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280c9a90 con 0x7f13280b17f0 2023-11-06T03:08:28.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132e3b3640 1 --2- 172.21.15.157:0/3915999893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f13280cc890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1328070480 tx=0x7f132001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319973925c801606 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132cbb0640 1 -- 172.21.15.157:0/3915999893 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200a59a0 con 0x7f13280b17f0 2023-11-06T03:08:28.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132cbb0640 1 -- 172.21.15.157:0/3915999893 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13200a5b00 con 0x7f13280b17f0 2023-11-06T03:08:28.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132cbb0640 1 -- 172.21.15.157:0/3915999893 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200a5e90 con 0x7f13280b17f0 2023-11-06T03:08:28.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- 172.21.15.157:0/3915999893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 msgr2=0x7f13280cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- 172.21.15.157:0/3915999893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f13280cc890 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1328070480 tx=0x7f132001c8e0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- 172.21.15.157:0/3915999893 shutdown_connections 2023-11-06T03:08:28.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- 172.21.15.157:0/3915999893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f13280cc890 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- 172.21.15.157:0/3915999893 >> v2:172.21.15.195:3300/0 conn(0x7f13280b1d40 0x7f13280cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- 172.21.15.157:0/3915999893 >> 172.21.15.157:0/3915999893 conn(0x7f132806a170 msgr2=0x7f132806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:28.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- 172.21.15.157:0/3915999893 shutdown_connections 2023-11-06T03:08:28.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- 172.21.15.157:0/3915999893 wait complete. 2023-11-06T03:08:28.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 Processor -- start 2023-11-06T03:08:28.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- start start 2023-11-06T03:08:28.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 0x7f1328153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 0x7f1328154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c7600 con 0x7f13280b17f0 2023-11-06T03:08:28.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c78e0 con 0x7f132815aa80 2023-11-06T03:08:28.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132fe16640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c7bc0 con 0x7f13280b1d40 2023-11-06T03:08:28.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132dbb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132dbb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58928/0 (socket says 172.21.15.157:58928) 2023-11-06T03:08:28.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132dbb2640 1 -- 172.21.15.157:0/1238547000 learned_addr learned my addr 172.21.15.157:0/1238547000 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:28.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.148+0000 7f132e3b3640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 0x7f1328154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132d3b1640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 0x7f1328153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 -- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 msgr2=0x7f1328153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 0x7f1328153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 -- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 msgr2=0x7f1328154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 0x7f1328154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 -- 172.21.15.157:0/1238547000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009a70 con 0x7f13280b17f0 2023-11-06T03:08:28.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132e3b3640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 0x7f1328154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132d3b1640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 0x7f1328153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132dbb2640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f13240011f0 tx=0x7f132400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240074e0 con 0x7f13280b17f0 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324007640 con 0x7f13280b17f0 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13280c7ea0 con 0x7f13280b17f0 2023-11-06T03:08:28.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324004a10 con 0x7f13280b17f0 2023-11-06T03:08:28.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328151330 con 0x7f13280b17f0 2023-11-06T03:08:28.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1328072750 con 0x7f13280b17f0 2023-11-06T03:08:28.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1324004b70 con 0x7f13280b17f0 2023-11-06T03:08:28.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 0x7f130c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1324082830 con 0x7f13280b17f0 2023-11-06T03:08:28.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132d3b1640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 0x7f130c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.152+0000 7f132d3b1640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 0x7f130c067f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.156+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1324050050 con 0x7f13280b17f0 2023-11-06T03:08:28.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.308+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1328070f50 con 0x7f13280b17f0 2023-11-06T03:08:28.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f131e7fc640 1 -- 172.21.15.157:0/1238547000 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13240180a0 con 0x7f13280b17f0 2023-11-06T03:08:28.311 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:08:28.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 msgr2=0x7f130c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 0x7f130c067f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318008040 comp rx=0 tx=0).stop 2023-11-06T03:08:28.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 msgr2=0x7f1328150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f13240011f0 tx=0x7f132400c9b0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 shutdown_connections 2023-11-06T03:08:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f130c065a90 0x7f130c067f50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f132815aa80 0x7f1328154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13280b1d40 0x7f1328153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 --2- 172.21.15.157:0/1238547000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13280b17f0 0x7f1328150df0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 >> 172.21.15.157:0/1238547000 conn(0x7f132806a170 msgr2=0x7f1328156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 shutdown_connections 2023-11-06T03:08:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.312+0000 7f132fe16640 1 -- 172.21.15.157:0/1238547000 wait complete. 2023-11-06T03:08:28.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:08:28.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 Processor -- start 2023-11-06T03:08:28.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 -- start start 2023-11-06T03:08:28.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c146370 0x7fe29c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29c0ce800 0x7fe29c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0cf1a0 con 0x7fe29c06b6a0 2023-11-06T03:08:28.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c06c3d0 con 0x7fe29c0ce800 2023-11-06T03:08:28.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a3fb8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c06f5a0 con 0x7fe29c146370 2023-11-06T03:08:28.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29c0ce800 0x7fe29c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29c0ce800 0x7fe29c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58940/0 (socket says 172.21.15.157:58940) 2023-11-06T03:08:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 -- 172.21.15.157:0/842677311 learned_addr learned my addr 172.21.15.157:0/842677311 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1d54640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c146370 0x7fe29c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 -- 172.21.15.157:0/842677311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c146370 msgr2=0x7fe29c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c146370 0x7fe29c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 -- 172.21.15.157:0/842677311 >> v1:172.21.15.195:6789/0 conn(0x7fe29c06b6a0 legacy=0x7fe29c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.448+0000 7fe2a1553640 1 -- 172.21.15.157:0/842677311 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c06f880 con 0x7fe29c0ce800 2023-11-06T03:08:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a1553640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0ce800 0x7fe29c0cec60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009870 tx=0x7fe28c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d963403ea0943eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a0d52640 1 -- 172.21.15.157:0/842677311 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0aa610 con 0x7fe29c0ce800 2023-11-06T03:08:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a0d52640 1 -- 172.21.15.157:0/842677311 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c0a6040 con 0x7fe29c0ce800 2023-11-06T03:08:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a0d52640 1 -- 172.21.15.157:0/842677311 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0aa990 con 0x7fe29c0ce800 2023-11-06T03:08:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/842677311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0ce800 msgr2=0x7fe29c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0ce800 0x7fe29c0cec60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009870 tx=0x7fe28c01c4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/842677311 shutdown_connections 2023-11-06T03:08:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0ce800 0x7fe29c0cec60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/842677311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c146370 0x7fe29c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/842677311 >> 172.21.15.157:0/842677311 conn(0x7fe29c065930 msgr2=0x7fe29c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/842677311 shutdown_connections 2023-11-06T03:08:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/842677311 wait complete. 2023-11-06T03:08:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 Processor -- start 2023-11-06T03:08:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- start start 2023-11-06T03:08:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c06b6a0 0x7fe29c141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c0ce800 0x7fe29c144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c144fb0 con 0x7fe29c146370 2023-11-06T03:08:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c145290 con 0x7fe29c0ce800 2023-11-06T03:08:28.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a3fb8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c145570 con 0x7fe29c06b6a0 2023-11-06T03:08:28.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a2555640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a2555640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58950/0 (socket says 172.21.15.157:58950) 2023-11-06T03:08:28.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.452+0000 7fe2a2555640 1 -- 172.21.15.157:0/2805843310 learned_addr learned my addr 172.21.15.157:0/2805843310 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:28.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a1d54640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c06b6a0 0x7fe29c141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a1553640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c0ce800 0x7fe29c144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 -- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c06b6a0 msgr2=0x7fe29c141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c06b6a0 0x7fe29c141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 -- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c0ce800 msgr2=0x7fe29c144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c0ce800 0x7fe29c144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 -- 172.21.15.157:0/2805843310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0094d0 con 0x7fe29c146370 2023-11-06T03:08:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a2555640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c730 tx=0x7fe29400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294003b80 con 0x7fe29c146370 2023-11-06T03:08:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294003ce0 con 0x7fe29c146370 2023-11-06T03:08:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c145850 con 0x7fe29c146370 2023-11-06T03:08:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400aa30 con 0x7fe29c146370 2023-11-06T03:08:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c144ca0 con 0x7fe29c146370 2023-11-06T03:08:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c1595e0 con 0x7fe29c146370 2023-11-06T03:08:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe29400ab90 con 0x7fe29c146370 2023-11-06T03:08:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 0x7fe280067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe2940831f0 con 0x7fe29c146370 2023-11-06T03:08:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a1d54640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 0x7fe280067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.456+0000 7fe2a1d54640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 0x7fe280067f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe298009790 tx=0x7fe298008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.460+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe29c1595e0 con 0x7fe29c146370 2023-11-06T03:08:28.463 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:08:28.460+0000 7f09efb48640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-06T03:08:28.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.620+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe29c1595e0 con 0x7fe29c146370 2023-11-06T03:08:28.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.620+0000 7fe2927fc640 1 -- 172.21.15.157:0/2805843310 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe29c1595e0 con 0x7fe29c146370 2023-11-06T03:08:28.622 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:08:28.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 msgr2=0x7fe280067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 0x7fe280067f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe298009790 tx=0x7fe298008040 comp rx=0 tx=0).stop 2023-11-06T03:08:28.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 msgr2=0x7fe29c13c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:28.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c730 tx=0x7fe29400cbf0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 shutdown_connections 2023-11-06T03:08:28.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe280065a60 0x7fe280067f20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c146370 0x7fe29c13c280 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29c0ce800 0x7fe29c144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 --2- 172.21.15.157:0/2805843310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe29c06b6a0 0x7fe29c141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 >> 172.21.15.157:0/2805843310 conn(0x7fe29c065930 msgr2=0x7fe29c0ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:28.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 shutdown_connections 2023-11-06T03:08:28.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:28.624+0000 7fe2a3fb8640 1 -- 172.21.15.157:0/2805843310 wait complete. 2023-11-06T03:08:28.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:08:28.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:28.642 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:28.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[62277]: fuse finished with error 0 and tester_r 0 2023-11-06T03:08:28.657 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:08:28.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:28.658 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:08:28.827 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:08:28.828 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:28.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:08:28.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:08:28.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:28.841 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:08:28.887 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:08:28.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:08:28.888 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:28.888 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:28.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:08:28.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:08:28.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:28.894 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:08:28.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:08:28.943 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:08:28.944 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:08:28.944 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:28.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:08:28.950 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:08:28.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:28.951 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:08:28.999 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:08:29.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:08:29.001 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:29.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.116+0000 7f149d815640 1 Processor -- start 2023-11-06T03:08:29.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.116+0000 7f149d815640 1 -- start start 2023-11-06T03:08:29.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f149d815640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14980ce800 0x7f14980cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f149d815640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980cf1a0 con 0x7f1498146370 2023-11-06T03:08:29.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f149d815640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149806c3d0 con 0x7f149806b6a0 2023-11-06T03:08:29.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f149d815640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980b2120 con 0x7f14980ce800 2023-11-06T03:08:29.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f1496ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1498146370 0x7f14980b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44596/0 (socket says 172.21.15.157:44596) 2023-11-06T03:08:29.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f1496ffd640 1 -- 172.21.15.157:0/741883552 learned_addr learned my addr 172.21.15.157:0/741883552 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:29.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831709929 0 0) 0x7f14980cf1a0 con 0x7f1498146370 2023-11-06T03:08:29.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1484003530 con 0x7f1498146370 2023-11-06T03:08:29.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f148e7fc640 1 --2- 172.21.15.157:0/741883552 >> v2:172.21.15.195:3300/0 conn(0x7f14980ce800 0x7f14980cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452648246 0 0) 0x7f1484003530 con 0x7f1498146370 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 >> v2:172.21.15.195:3300/0 conn(0x7f14980ce800 msgr2=0x7f14980cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 --2- 172.21.15.157:0/741883552 >> v2:172.21.15.195:3300/0 conn(0x7f14980ce800 0x7f14980cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 >> v1:172.21.15.195:6789/0 conn(0x7f149806b6a0 legacy=0x7f149806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149806f5a0 con 0x7f1498146370 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1488001680 con 0x7f1498146370 2023-11-06T03:08:29.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 >> v1:172.21.15.157:6789/0 conn(0x7f1498146370 legacy=0x7f14980b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1484003980 0x7f1484003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840047d0 0x7f1484004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 0x7f148400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b2120 con 0x7f14840052b0 2023-11-06T03:08:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148400c180 con 0x7f1484003980 2023-11-06T03:08:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148400c340 con 0x7f14840047d0 2023-11-06T03:08:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 0x7f148400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f1496ffd640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1484003980 0x7f1484003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f148e7fc640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840047d0 0x7f1484004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 -- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840047d0 msgr2=0x7f1484004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840047d0 0x7f1484004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 -- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1484003980 msgr2=0x7f1484003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1484003980 0x7f1484003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.120+0000 7f14977fe640 1 -- 172.21.15.157:0/741883552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484003530 con 0x7f14840052b0 2023-11-06T03:08:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14977fe640 1 -- 172.21.15.157:0/741883552 reap_dead start 2023-11-06T03:08:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14977fe640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 0x7f148400b990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1490001e40 tx=0x7f149001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec6482c9c41ae4b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490003b80 con 0x7f14840052b0 2023-11-06T03:08:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490003ce0 con 0x7f14840052b0 2023-11-06T03:08:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14967fc640 1 -- 172.21.15.157:0/741883552 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490005310 con 0x7f14840052b0 2023-11-06T03:08:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 msgr2=0x7f148400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 0x7f148400b990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1490001e40 tx=0x7f149001c740 comp rx=0 tx=0).stop 2023-11-06T03:08:29.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- 172.21.15.157:0/741883552 shutdown_connections 2023-11-06T03:08:29.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 --2- 172.21.15.157:0/741883552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14840052b0 0x7f148400b990 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- 172.21.15.157:0/741883552 >> 172.21.15.157:0/741883552 conn(0x7f1498065930 msgr2=0x7f1498065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- 172.21.15.157:0/741883552 shutdown_connections 2023-11-06T03:08:29.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- 172.21.15.157:0/741883552 wait complete. 2023-11-06T03:08:29.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 Processor -- start 2023-11-06T03:08:29.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- start start 2023-11-06T03:08:29.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840052b0 0x7f14981606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 0x7f149815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149815be30 con 0x7f14981f2110 2023-11-06T03:08:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149813bdf0 con 0x7f149815b9b0 2023-11-06T03:08:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f149d815640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149815d2c0 con 0x7f14840052b0 2023-11-06T03:08:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34030/0 (socket says 172.21.15.157:34030) 2023-11-06T03:08:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 -- 172.21.15.157:0/1683847231 learned_addr learned my addr 172.21.15.157:0/1683847231 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14977fe640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 0x7f149815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f1496ffd640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840052b0 0x7f14981606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 -- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840052b0 msgr2=0x7f14981606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840052b0 0x7f14981606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 -- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 msgr2=0x7f149815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 0x7f149815da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 -- 172.21.15.157:0/1683847231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498161340 con 0x7f14981f2110 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f14977fe640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 0x7f149815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:29.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.124+0000 7f148e7fc640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f148000f580 tx=0x7f148000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480013810 con 0x7f14981f2110 2023-11-06T03:08:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480013970 con 0x7f14981f2110 2023-11-06T03:08:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1498160d80 con 0x7f14981f2110 2023-11-06T03:08:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480013d20 con 0x7f14981f2110 2023-11-06T03:08:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149815d560 con 0x7f14981f2110 2023-11-06T03:08:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149815d720 con 0x7f14981f2110 2023-11-06T03:08:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f148001f460 con 0x7f14981f2110 2023-11-06T03:08:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 0x7f1478067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f14800836e0 con 0x7f14981f2110 2023-11-06T03:08:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1496ffd640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 0x7f1478067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.128+0000 7f1496ffd640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 0x7f1478067f20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1488004880 tx=0x7f14880047f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.132+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1480050f10 con 0x7f14981f2110 2023-11-06T03:08:29.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.324+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f149806f860 con 0x7f14981f2110 2023-11-06T03:08:29.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.324+0000 7f1494ff9640 1 -- 172.21.15.157:0/1683847231 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f149806f860 con 0x7f14981f2110 2023-11-06T03:08:29.326 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:29.326 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:28.465085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":15,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:29.327 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 40 2023-11-06T03:08:29.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 msgr2=0x7f1478067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 0x7f1478067f20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1488004880 tx=0x7f14880047f0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 msgr2=0x7f149815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f148000f580 tx=0x7f148000fa40 comp rx=0 tx=0).stop 2023-11-06T03:08:29.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 shutdown_connections 2023-11-06T03:08:29.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1478065a60 0x7f1478067f20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815b9b0 0x7f149815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14981f2110 0x7f149815b470 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 --2- 172.21.15.157:0/1683847231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14840052b0 0x7f14981606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 >> 172.21.15.157:0/1683847231 conn(0x7f1498065930 msgr2=0x7f1498070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 shutdown_connections 2023-11-06T03:08:29.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.328+0000 7f149d815640 1 -- 172.21.15.157:0/1683847231 wait complete. 2023-11-06T03:08:29.347 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:29.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.468+0000 7f777b967640 1 Processor -- start 2023-11-06T03:08:29.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.468+0000 7f777b967640 1 -- start start 2023-11-06T03:08:29.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f777b967640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77741551e0 0x7f777406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f777b967640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777415f580 con 0x7f777415e5e0 2023-11-06T03:08:29.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f777b967640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777415efc0 con 0x7f77740b3b20 2023-11-06T03:08:29.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f777b967640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777415f2a0 con 0x7f77741551e0 2023-11-06T03:08:29.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77741551e0 0x7f777406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7778f02640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f777415e5e0 0x7f77740b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44608/0 (socket says 172.21.15.157:44608) 2023-11-06T03:08:29.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7778f02640 1 -- 172.21.15.157:0/2052369055 learned_addr learned my addr 172.21.15.157:0/2052369055 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:29.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77741551e0 0x7f777406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34032/0 (socket says 172.21.15.157:34032) 2023-11-06T03:08:29.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 -- 172.21.15.157:0/2052369055 >> v1:172.21.15.195:6789/0 conn(0x7f77740b3b20 legacy=0x7f777415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 -- 172.21.15.157:0/2052369055 >> v1:172.21.15.157:6789/0 conn(0x7f777415e5e0 legacy=0x7f77740b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f776bfff640 1 -- 172.21.15.157:0/2052369055 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583598697 0 0) 0x7f777415f580 con 0x7f777415e5e0 2023-11-06T03:08:29.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 -- 172.21.15.157:0/2052369055 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777406f5a0 con 0x7f77741551e0 2023-11-06T03:08:29.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.472+0000 7f7779703640 1 --2- 172.21.15.157:0/2052369055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f777406fda0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f77640097c0 tx=0x7f776401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470503ef1877bf88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f776bfff640 1 -- 172.21.15.157:0/2052369055 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640ac640 con 0x7f77741551e0 2023-11-06T03:08:29.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f776bfff640 1 -- 172.21.15.157:0/2052369055 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640ac7a0 con 0x7f77741551e0 2023-11-06T03:08:29.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f776bfff640 1 -- 172.21.15.157:0/2052369055 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640acb30 con 0x7f77741551e0 2023-11-06T03:08:29.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/2052369055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 msgr2=0x7f777406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 --2- 172.21.15.157:0/2052369055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f777406fda0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f77640097c0 tx=0x7f776401c2f0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/2052369055 shutdown_connections 2023-11-06T03:08:29.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 --2- 172.21.15.157:0/2052369055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f777406fda0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/2052369055 >> 172.21.15.157:0/2052369055 conn(0x7f7774065930 msgr2=0x7f7774065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/2052369055 shutdown_connections 2023-11-06T03:08:29.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/2052369055 wait complete. 2023-11-06T03:08:29.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 Processor -- start 2023-11-06T03:08:29.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- start start 2023-11-06T03:08:29.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740b3b20 0x7f77740cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f777415e5e0 0x7f7774145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777414dca0 con 0x7f77741551e0 2023-11-06T03:08:29.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774151ff0 con 0x7f77740b3b20 2023-11-06T03:08:29.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777414d9c0 con 0x7f777415e5e0 2023-11-06T03:08:29.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34042/0 (socket says 172.21.15.157:34042) 2023-11-06T03:08:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 -- 172.21.15.157:0/1047792417 learned_addr learned my addr 172.21.15.157:0/1047792417 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7779703640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740b3b20 0x7f77740cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7779f04640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f777415e5e0 0x7f7774145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 -- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f777415e5e0 msgr2=0x7f7774145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f777415e5e0 0x7f7774145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 -- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740b3b20 msgr2=0x7f77740cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740b3b20 0x7f77740cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 -- 172.21.15.157:0/1047792417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7764009420 con 0x7f77741551e0 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7778f02640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f777000c480 tx=0x7f777000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700074e0 con 0x7f77741551e0 2023-11-06T03:08:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770007660 con 0x7f77741551e0 2023-11-06T03:08:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741519a0 con 0x7f77741551e0 2023-11-06T03:08:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770004a30 con 0x7f77741551e0 2023-11-06T03:08:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.476+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77740ce210 con 0x7f77741551e0 2023-11-06T03:08:29.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774151800 con 0x7f77741551e0 2023-11-06T03:08:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7770004b90 con 0x7f77741551e0 2023-11-06T03:08:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f7769ffb640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 0x7f7758067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7770082780 con 0x7f77741551e0 2023-11-06T03:08:29.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f7779703640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 0x7f7758067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.480+0000 7f7779703640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 0x7f7758067f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77640ae000 tx=0x7f7764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.484+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7774151800 con 0x7f77741551e0 2023-11-06T03:08:29.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.640+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7774151800 con 0x7f77741551e0 2023-11-06T03:08:29.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.640+0000 7f7769ffb640 1 -- 172.21.15.157:0/1047792417 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f7774151800 con 0x7f77741551e0 2023-11-06T03:08:29.642 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:29.642 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:50.274092+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:29.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 msgr2=0x7f7758067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 0x7f7758067f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77640ae000 tx=0x7f7764008040 comp rx=0 tx=0).stop 2023-11-06T03:08:29.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 msgr2=0x7f7774152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f777000c480 tx=0x7f777000c940 comp rx=0 tx=0).stop 2023-11-06T03:08:29.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 shutdown_connections 2023-11-06T03:08:29.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7758065a90 0x7f7758067f50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f777415e5e0 0x7f7774145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77741551e0 0x7f7774152480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 --2- 172.21.15.157:0/1047792417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740b3b20 0x7f77740cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 >> 172.21.15.157:0/1047792417 conn(0x7f7774065930 msgr2=0x7f77740af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 shutdown_connections 2023-11-06T03:08:29.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.644+0000 7f777b967640 1 -- 172.21.15.157:0/1047792417 wait complete. 2023-11-06T03:08:29.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:07:37.373400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:07:38.627196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:08:29.662 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.784+0000 7fadc3d04640 1 Processor -- start 2023-11-06T03:08:29.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 -- start start 2023-11-06T03:08:29.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 0x7fadbc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 0x7fadbc0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fadbc0b2d50 0x7fadbc0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cef10 con 0x7fadbc0b2d50 2023-11-06T03:08:29.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cfb70 con 0x7fadbc0b23b0 2023-11-06T03:08:29.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc3d04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b31e0 con 0x7fadbc15ad30 2023-11-06T03:08:29.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fadbc0b2d50 0x7fadbc0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fadbc0b2d50 0x7fadbc0cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34046/0 (socket says 172.21.15.157:34046) 2023-11-06T03:08:29.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 -- 172.21.15.157:0/1234737006 learned_addr learned my addr 172.21.15.157:0/1234737006 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:29.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 0x7fadbc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc129f640 1 --2- 172.21.15.157:0/1234737006 >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 0x7fadbc0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 -- 172.21.15.157:0/1234737006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 msgr2=0x7fadbc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 0x7fadbc0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 -- 172.21.15.157:0/1234737006 >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 msgr2=0x7fadbc0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:29.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1234737006 >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 0x7fadbc0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 -- 172.21.15.157:0/1234737006 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0cf160 con 0x7fadbc0b2d50 2023-11-06T03:08:29.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 0x7fadbc0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:29.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc129f640 1 --2- 172.21.15.157:0/1234737006 >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 0x7fadbc0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:29.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc0b2d50 0x7fadbc0cc360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0b3480 tx=0x7fadb401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37669a4784c7c53 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc0a9e640 1 -- 172.21.15.157:0/1234737006 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40a5790 con 0x7fadbc0b2d50 2023-11-06T03:08:29.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc0a9e640 1 -- 172.21.15.157:0/1234737006 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb40a58f0 con 0x7fadbc0b2d50 2023-11-06T03:08:29.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.788+0000 7fadc0a9e640 1 -- 172.21.15.157:0/1234737006 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40a5c80 con 0x7fadbc0b2d50 2023-11-06T03:08:29.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1234737006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc0b2d50 msgr2=0x7fadbc0cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc0b2d50 0x7fadbc0cc360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0b3480 tx=0x7fadb401c910 comp rx=0 tx=0).stop 2023-11-06T03:08:29.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1234737006 shutdown_connections 2023-11-06T03:08:29.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc0b2d50 0x7fadbc0cc360 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1234737006 >> v2:172.21.15.195:3300/0 conn(0x7fadbc0b23b0 0x7fadbc0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1234737006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc15ad30 0x7fadbc0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1234737006 >> 172.21.15.157:0/1234737006 conn(0x7fadbc065d30 msgr2=0x7fadbc066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1234737006 shutdown_connections 2023-11-06T03:08:29.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1234737006 wait complete. 2023-11-06T03:08:29.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 Processor -- start 2023-11-06T03:08:29.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- start start 2023-11-06T03:08:29.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 0x7fadbc149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 0x7fadbc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc156930 con 0x7fadbc15ad30 2023-11-06T03:08:29.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0c5c20 con 0x7fadbc0b2d50 2023-11-06T03:08:29.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc156650 con 0x7fadbc0b23b0 2023-11-06T03:08:29.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34062/0 (socket says 172.21.15.157:34062) 2023-11-06T03:08:29.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 -- 172.21.15.157:0/1430838656 learned_addr learned my addr 172.21.15.157:0/1430838656 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:29.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc129f640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 0x7fadbc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 0x7fadbc149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 -- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 msgr2=0x7fadbc149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 0x7fadbc149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 -- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 msgr2=0x7fadbc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 0x7fadbc155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 -- 172.21.15.157:0/1430838656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009b30 con 0x7fadbc15ad30 2023-11-06T03:08:29.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 0x7fadbc149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:29.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc129f640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 0x7fadbc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:29.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc22a1640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fadb401afc0 tx=0x7fadb40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b8670 con 0x7fadbc15ad30 2023-11-06T03:08:29.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb40b87f0 con 0x7fadbc15ad30 2023-11-06T03:08:29.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0c5ea0 con 0x7fadbc15ad30 2023-11-06T03:08:29.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40b8ba0 con 0x7fadbc15ad30 2023-11-06T03:08:29.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.792+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0c6460 con 0x7fadbc15ad30 2023-11-06T03:08:29.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc072750 con 0x7fadbc15ad30 2023-11-06T03:08:29.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fadb40b8d00 con 0x7fadbc15ad30 2023-11-06T03:08:29.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadb27fc640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 0x7fada0067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:29.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fadb411eae0 con 0x7fadbc15ad30 2023-11-06T03:08:29.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 0x7fada0067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:29.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.796+0000 7fadc1aa0640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 0x7fada0067ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fadb8008890 tx=0x7fadb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:29.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.800+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fadb40ec310 con 0x7fadbc15ad30 2023-11-06T03:08:29.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.952+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadbc153e60 con 0x7fadbc15ad30 2023-11-06T03:08:29.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadb27fc640 1 -- 172.21.15.157:0/1430838656 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadb40bc020 con 0x7fadbc15ad30 2023-11-06T03:08:29.955 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:08:29.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 msgr2=0x7fada0067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 0x7fada0067ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fadb8008890 tx=0x7fadb8008040 comp rx=0 tx=0).stop 2023-11-06T03:08:29.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 msgr2=0x7fadbc151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:29.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fadb401afc0 tx=0x7fadb40196b0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 shutdown_connections 2023-11-06T03:08:29.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fada0065a30 0x7fada0067ef0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fadbc15ad30 0x7fadbc151110 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadbc0b2d50 0x7fadbc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 --2- 172.21.15.157:0/1430838656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadbc0b23b0 0x7fadbc149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:29.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 >> 172.21.15.157:0/1430838656 conn(0x7fadbc065d30 msgr2=0x7fadbc066110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:29.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 shutdown_connections 2023-11-06T03:08:29.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:29.956+0000 7fadc3d04640 1 -- 172.21.15.157:0/1430838656 wait complete. 2023-11-06T03:08:29.974 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.088+0000 7f205ff94640 1 Processor -- start 2023-11-06T03:08:30.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.088+0000 7f205ff94640 1 -- start start 2023-11-06T03:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205ff94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581551e0 0x7f205806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205ff94640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205815f580 con 0x7f205815e5e0 2023-11-06T03:08:30.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205ff94640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205815efc0 con 0x7f20580b3b20 2023-11-06T03:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205ff94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205815f2a0 con 0x7f20581551e0 2023-11-06T03:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f205815e5e0 0x7f20580b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44612/0 (socket says 172.21.15.157:44612) 2023-11-06T03:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 -- 172.21.15.157:0/96106713 learned_addr learned my addr 172.21.15.157:0/96106713 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205dd30640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581551e0 0x7f205806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043891983 0 0) 0x7f205815f580 con 0x7f205815e5e0 2023-11-06T03:08:30.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2044003530 con 0x7f205815e5e0 2023-11-06T03:08:30.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491995778 0 0) 0x7f2044003530 con 0x7f205815e5e0 2023-11-06T03:08:30.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581551e0 msgr2=0x7f205806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581551e0 0x7f205806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 >> v1:172.21.15.195:6789/0 conn(0x7f20580b3b20 legacy=0x7f205815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205806f5a0 con 0x7f205815e5e0 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2048001680 con 0x7f205815e5e0 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 >> v1:172.21.15.157:6789/0 conn(0x7f205815e5e0 legacy=0x7f20580b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440039b0 0x7f2044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20440047d0 0x7f2044004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20440052b0 0x7f204400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205815f2a0 con 0x7f20440047d0 2023-11-06T03:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204400c0e0 con 0x7f20440039b0 2023-11-06T03:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204400c2a0 con 0x7f20440052b0 2023-11-06T03:08:30.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205dd30640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581551e0 0x7f205806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20440047d0 0x7f2044004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205e531640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20440052b0 0x7f204400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 -- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20440052b0 msgr2=0x7f204400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205dd30640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440039b0 0x7f2044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20440052b0 0x7f204400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 -- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440039b0 msgr2=0x7f2044003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440039b0 0x7f2044003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 -- 172.21.15.157:0/96106713 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044003530 con 0x7f20440047d0 2023-11-06T03:08:30.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205e531640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20440052b0 0x7f204400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:30.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.092+0000 7f205d52f640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20440047d0 0x7f2044004c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f204800c240 tx=0x7f204801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed95149576a4398d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20480a5b10 con 0x7f20440047d0 2023-11-06T03:08:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20480a5c70 con 0x7f20440047d0 2023-11-06T03:08:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205cd2e640 1 -- 172.21.15.157:0/96106713 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048005b20 con 0x7f20440047d0 2023-11-06T03:08:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/96106713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20440047d0 msgr2=0x7f2044004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 --2- 172.21.15.157:0/96106713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20440047d0 0x7f2044004c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f204800c240 tx=0x7f204801cc10 comp rx=0 tx=0).stop 2023-11-06T03:08:30.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 -- 172.21.15.157:0/96106713 reap_dead start 2023-11-06T03:08:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/96106713 shutdown_connections 2023-11-06T03:08:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/96106713 >> 172.21.15.157:0/96106713 conn(0x7f2058065930 msgr2=0x7f2058065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/96106713 shutdown_connections 2023-11-06T03:08:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/96106713 wait complete. 2023-11-06T03:08:30.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 Processor -- start 2023-11-06T03:08:30.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- start start 2023-11-06T03:08:30.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 0x7f205813e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581637d0 0x7f2058163c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205813f7e0 con 0x7f205813ee20 2023-11-06T03:08:30.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205813e970 con 0x7f20440047d0 2023-11-06T03:08:30.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2058163340 con 0x7f20581637d0 2023-11-06T03:08:30.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34088/0 (socket says 172.21.15.157:34088) 2023-11-06T03:08:30.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 -- 172.21.15.157:0/1967609854 learned_addr learned my addr 172.21.15.157:0/1967609854 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:30.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205d52f640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581637d0 0x7f2058163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205dd30640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 0x7f205813e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 -- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581637d0 msgr2=0x7f2058163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581637d0 0x7f2058163c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 -- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 msgr2=0x7f205813e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 0x7f205813e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 -- 172.21.15.157:0/1967609854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058162730 con 0x7f205813ee20 2023-11-06T03:08:30.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205dd30640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 0x7f205813e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:30.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205e531640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f205815b030 tx=0x7f205000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050003b80 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050003ce0 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2058162a10 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500052b0 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.096+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2058162fd0 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20581634b0 con 0x7f205813ee20 2023-11-06T03:08:30.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2050005410 con 0x7f205813ee20 2023-11-06T03:08:30.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f204e7fc640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 0x7f203c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2050082c20 con 0x7f205813ee20 2023-11-06T03:08:30.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f205dd30640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 0x7f203c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.100+0000 7f205dd30640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 0x7f203c067f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2058066830 tx=0x7f2054008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.104+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2050050440 con 0x7f205813ee20 2023-11-06T03:08:30.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2058072750 con 0x7f205813ee20 2023-11-06T03:08:30.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f204e7fc640 1 -- 172.21.15.157:0/1967609854 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2050018020 con 0x7f205813ee20 2023-11-06T03:08:30.259 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:08:30.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 msgr2=0x7f203c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 0x7f203c067f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2058066830 tx=0x7f2054008040 comp rx=0 tx=0).stop 2023-11-06T03:08:30.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 msgr2=0x7f205813f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f205815b030 tx=0x7f205000dc00 comp rx=0 tx=0).stop 2023-11-06T03:08:30.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 shutdown_connections 2023-11-06T03:08:30.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f203c065a90 0x7f203c067f50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f205813ee20 0x7f205813f2a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20581637d0 0x7f2058163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 --2- 172.21.15.157:0/1967609854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20440047d0 0x7f205813e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 >> 172.21.15.157:0/1967609854 conn(0x7f2058065930 msgr2=0x7f20580af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 shutdown_connections 2023-11-06T03:08:30.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.260+0000 7f205ff94640 1 -- 172.21.15.157:0/1967609854 wait complete. 2023-11-06T03:08:30.278 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:08:30.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 Processor -- start 2023-11-06T03:08:30.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 -- start start 2023-11-06T03:08:30.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbdc815caa0 0x7fbdc80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80d2920 con 0x7fbdc815ce80 2023-11-06T03:08:30.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80d18a0 con 0x7fbdc80d0280 2023-11-06T03:08:30.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdcd4b6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80d1b60 con 0x7fbdc815caa0 2023-11-06T03:08:30.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc67fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbdc815ce80 0x7fbdc80cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44616/0 (socket says 172.21.15.157:44616) 2023-11-06T03:08:30.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc67fc640 1 -- 172.21.15.157:0/2994109691 learned_addr learned my addr 172.21.15.157:0/2994109691 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:30.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc6ffd640 1 --2- 172.21.15.157:0/2994109691 >> v2:172.21.15.195:3300/0 conn(0x7fbdc815caa0 0x7fbdc80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744271961 0 0) 0x7fbdc80d2920 con 0x7fbdc815ce80 2023-11-06T03:08:30.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003530 con 0x7fbdc815ce80 2023-11-06T03:08:30.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2032442236 0 0) 0x7fbdb4003530 con 0x7fbdc815ce80 2023-11-06T03:08:30.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 >> v2:172.21.15.195:3300/0 conn(0x7fbdc815caa0 msgr2=0x7fbdc80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 --2- 172.21.15.157:0/2994109691 >> v2:172.21.15.195:3300/0 conn(0x7fbdc815caa0 0x7fbdc80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 >> v1:172.21.15.195:6789/0 conn(0x7fbdc80d0280 legacy=0x7fbdc80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80d1e40 con 0x7fbdc815ce80 2023-11-06T03:08:30.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fbdb0001680 con 0x7fbdc815ce80 2023-11-06T03:08:30.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 >> v1:172.21.15.157:6789/0 conn(0x7fbdc815ce80 legacy=0x7fbdc80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4003980 0x7fbdb4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdb40047d0 0x7fbdb4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 0x7fbdb400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80d1b60 con 0x7fbdb4005190 2023-11-06T03:08:30.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb400c030 con 0x7fbdb40047d0 2023-11-06T03:08:30.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb400c1f0 con 0x7fbdb4003980 2023-11-06T03:08:30.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 0x7fbdb400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc67fc640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdb40047d0 0x7fbdb4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc6ffd640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4003980 0x7fbdb4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 -- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4003980 msgr2=0x7fbdb4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4003980 0x7fbdb4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 -- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdb40047d0 msgr2=0x7fbdb4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdb40047d0 0x7fbdb4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.400+0000 7fbdc77fe640 1 -- 172.21.15.157:0/2994109691 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4003530 con 0x7fbdb4005190 2023-11-06T03:08:30.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdc77fe640 1 -- 172.21.15.157:0/2994109691 reap_dead start 2023-11-06T03:08:30.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdc77fe640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 0x7fbdb400b990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0001780 tx=0x7fbdc001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef09e06326d805e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00052d0 con 0x7fbdb4005190 2023-11-06T03:08:30.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc00a7050 con 0x7fbdb4005190 2023-11-06T03:08:30.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdc5ffb640 1 -- 172.21.15.157:0/2994109691 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003840 con 0x7fbdb4005190 2023-11-06T03:08:30.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 msgr2=0x7fbdb400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 0x7fbdb400b990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0001780 tx=0x7fbdc001c400 comp rx=0 tx=0).stop 2023-11-06T03:08:30.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/2994109691 shutdown_connections 2023-11-06T03:08:30.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/2994109691 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdb4005190 0x7fbdb400b990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/2994109691 >> 172.21.15.157:0/2994109691 conn(0x7fbdc8070790 msgr2=0x7fbdc8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/2994109691 shutdown_connections 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/2994109691 wait complete. 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 Processor -- start 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.404+0000 7fbdcd4b6640 1 -- start start 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4005190 0x7fbdc8154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdc81e7180 0x7fbdc81e2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81579c0 con 0x7fbdc81e6d00 2023-11-06T03:08:30.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc8155b10 con 0x7fbdc81e7180 2023-11-06T03:08:30.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81e57e0 con 0x7fbdb4005190 2023-11-06T03:08:30.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34110/0 (socket says 172.21.15.157:34110) 2023-11-06T03:08:30.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 -- 172.21.15.157:0/3721385347 learned_addr learned my addr 172.21.15.157:0/3721385347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:30.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc6ffd640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4005190 0x7fbdc8154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc77fe640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdc81e7180 0x7fbdc81e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 -- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4005190 msgr2=0x7fbdc8154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4005190 0x7fbdc8154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 -- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdc81e7180 msgr2=0x7fbdc81e2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdc81e7180 0x7fbdc81e2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 -- 172.21.15.157:0/3721385347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc81e5500 con 0x7fbdc81e6d00 2023-11-06T03:08:30.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc67fc640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0001210 tx=0x7fbdb000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000ee60 con 0x7fbdc81e6d00 2023-11-06T03:08:30.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000c8d0 con 0x7fbdc81e6d00 2023-11-06T03:08:30.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc8157300 con 0x7fbdc81e6d00 2023-11-06T03:08:30.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000cc60 con 0x7fbdc81e6d00 2023-11-06T03:08:30.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc81554c0 con 0x7fbdc81e6d00 2023-11-06T03:08:30.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8157b00 con 0x7fbdc81e6d00 2023-11-06T03:08:30.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbdb001d460 con 0x7fbdc81e6d00 2023-11-06T03:08:30.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbdb0083230 con 0x7fbdc81e6d00 2023-11-06T03:08:30.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.408+0000 7fbdc6ffd640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.412+0000 7fbdc6ffd640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 0x7fbda8067f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009fd0 tx=0x7fbdb800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.412+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbdc8157b00 con 0x7fbdc81e6d00 2023-11-06T03:08:30.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.568+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbdc8157b00 con 0x7fbdc81e6d00 2023-11-06T03:08:30.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdbf7fe640 1 -- 172.21.15.157:0/3721385347 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbdc8157b00 con 0x7fbdc81e6d00 2023-11-06T03:08:30.571 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:08:30.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 msgr2=0x7fbda8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 0x7fbda8067f20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009fd0 tx=0x7fbdb800b040 comp rx=0 tx=0).stop 2023-11-06T03:08:30.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 msgr2=0x7fbdc81e1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0001210 tx=0x7fbdb000ec10 comp rx=0 tx=0).stop 2023-11-06T03:08:30.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 shutdown_connections 2023-11-06T03:08:30.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbda8065a60 0x7fbda8067f20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbdc81e7180 0x7fbdc81e2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdc81e6d00 0x7fbdc81e1fd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 --2- 172.21.15.157:0/3721385347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbdb4005190 0x7fbdc8154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 >> 172.21.15.157:0/3721385347 conn(0x7fbdc8070790 msgr2=0x7fbdc80b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 shutdown_connections 2023-11-06T03:08:30.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.572+0000 7fbdcd4b6640 1 -- 172.21.15.157:0/3721385347 wait complete. 2023-11-06T03:08:30.590 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:08:30.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 Processor -- start 2023-11-06T03:08:30.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 -- start start 2023-11-06T03:08:30.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd06415cb80 0x7fd0640cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 0x7fd0640cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd064071830 con 0x7fd0640cc910 2023-11-06T03:08:30.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064071b10 con 0x7fd06415cb80 2023-11-06T03:08:30.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd06a04b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064071df0 con 0x7fd06415cf60 2023-11-06T03:08:30.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd063fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd0640cc910 0x7fd0640cfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44622/0 (socket says 172.21.15.157:44622) 2023-11-06T03:08:30.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd063fff640 1 -- 172.21.15.157:0/4070271876 learned_addr learned my addr 172.21.15.157:0/4070271876 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 --2- 172.21.15.157:0/4070271876 >> v2:172.21.15.157:3300/0 conn(0x7fd06415cb80 0x7fd0640cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd05bfff640 1 --2- 172.21.15.157:0/4070271876 >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 0x7fd0640cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 -- 172.21.15.157:0/4070271876 >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 msgr2=0x7fd0640cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 --2- 172.21.15.157:0/4070271876 >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 0x7fd0640cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 -- 172.21.15.157:0/4070271876 >> v1:172.21.15.157:6789/0 conn(0x7fd0640cc910 legacy=0x7fd0640cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 -- 172.21.15.157:0/4070271876 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640ccfe0 con 0x7fd06415cb80 2023-11-06T03:08:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd062ffd640 1 -- 172.21.15.157:0/4070271876 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35731556 0 0) 0x7fd064071830 con 0x7fd0640cc910 2023-11-06T03:08:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd05bfff640 1 --2- 172.21.15.157:0/4070271876 >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 0x7fd0640cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.716+0000 7fd0637fe640 1 --2- 172.21.15.157:0/4070271876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd0640cafd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd06406e2b0 tx=0x7fd05401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01f47000a62db8f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd062ffd640 1 -- 172.21.15.157:0/4070271876 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540a8950 con 0x7fd06415cb80 2023-11-06T03:08:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd062ffd640 1 -- 172.21.15.157:0/4070271876 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0540a8ab0 con 0x7fd06415cb80 2023-11-06T03:08:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd062ffd640 1 -- 172.21.15.157:0/4070271876 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540a8e40 con 0x7fd06415cb80 2023-11-06T03:08:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/4070271876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 msgr2=0x7fd0640cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- 172.21.15.157:0/4070271876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd0640cafd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd06406e2b0 tx=0x7fd05401ec50 comp rx=0 tx=0).stop 2023-11-06T03:08:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/4070271876 shutdown_connections 2023-11-06T03:08:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- 172.21.15.157:0/4070271876 >> v2:172.21.15.195:3300/0 conn(0x7fd06415cf60 0x7fd0640cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- 172.21.15.157:0/4070271876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd0640cafd0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/4070271876 >> 172.21.15.157:0/4070271876 conn(0x7fd06406d560 msgr2=0x7fd06406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/4070271876 shutdown_connections 2023-11-06T03:08:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/4070271876 wait complete. 2023-11-06T03:08:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 Processor -- start 2023-11-06T03:08:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- start start 2023-11-06T03:08:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 0x7fd0641f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd06415cf60 0x7fd06406b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06415b840 con 0x7fd06415cb80 2023-11-06T03:08:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640c7a50 con 0x7fd0640cc910 2023-11-06T03:08:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640c7d30 con 0x7fd06415cf60 2023-11-06T03:08:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34134/0 (socket says 172.21.15.157:34134) 2023-11-06T03:08:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 -- 172.21.15.157:0/422361030 learned_addr learned my addr 172.21.15.157:0/422361030 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd063fff640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd06415cf60 0x7fd06406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd0637fe640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 0x7fd0641f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 -- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd06415cf60 msgr2=0x7fd06406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd06415cf60 0x7fd06406b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 -- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 msgr2=0x7fd0641f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 0x7fd0641f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 -- 172.21.15.157:0/422361030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009420 con 0x7fd06415cb80 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd0637fe640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 0x7fd0641f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd05bfff640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c7f0 tx=0x7fd04c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c007820 con 0x7fd06415cb80 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c007980 con 0x7fd06415cb80 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640c8010 con 0x7fd06415cb80 2023-11-06T03:08:30.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c007d30 con 0x7fd06415cb80 2023-11-06T03:08:30.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.720+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd064158c40 con 0x7fd06415cb80 2023-11-06T03:08:30.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06406bf00 con 0x7fd06415cb80 2023-11-06T03:08:30.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd04c0113f0 con 0x7fd06415cb80 2023-11-06T03:08:30.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd060ff9640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 0x7fd044067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd04c083430 con 0x7fd06415cb80 2023-11-06T03:08:30.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd0637fe640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 0x7fd044067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.724+0000 7fd0637fe640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 0x7fd044067f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b2000 tx=0x7fd0540aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.728+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd04c050c50 con 0x7fd06415cb80 2023-11-06T03:08:30.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0640b2e90 con 0x7fd06415cb80 2023-11-06T03:08:30.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd060ff9640 1 -- 172.21.15.157:0/422361030 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd04c054270 con 0x7fd06415cb80 2023-11-06T03:08:30.872 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:08:30.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 msgr2=0x7fd044067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 0x7fd044067f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b2000 tx=0x7fd0540aa070 comp rx=0 tx=0).stop 2023-11-06T03:08:30.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 msgr2=0x7fd06415adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c7f0 tx=0x7fd04c00ccb0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 shutdown_connections 2023-11-06T03:08:30.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd044065a60 0x7fd044067f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd06415cf60 0x7fd06406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06415cb80 0x7fd06415adf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 --2- 172.21.15.157:0/422361030 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0640cc910 0x7fd0641f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 >> 172.21.15.157:0/422361030 conn(0x7fd06406d560 msgr2=0x7fd0640b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:30.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 shutdown_connections 2023-11-06T03:08:30.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.872+0000 7fd06a04b640 1 -- 172.21.15.157:0/422361030 wait complete. 2023-11-06T03:08:30.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:30.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.992+0000 7f5a548e3640 1 Processor -- start 2023-11-06T03:08:30.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.992+0000 7f5a548e3640 1 -- start start 2023-11-06T03:08:30.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a548e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a5015caa0 0x7f5a500cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a548e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 0x7f5a500cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:30.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a548e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a500d2920 con 0x7f5a500ce1a0 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a548e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a500d18a0 con 0x7f5a5015caa0 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a548e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500d1b60 con 0x7f5a5015ce80 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a5015caa0 0x7f5a500cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5a500ce1a0 0x7f5a500d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44626/0 (socket says 172.21.15.157:44626) 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4ed9d640 1 -- 172.21.15.157:0/1715588370 learned_addr learned my addr 172.21.15.157:0/1715588370 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4dd9b640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 0x7f5a500cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:30.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 -- 172.21.15.157:0/1715588370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 msgr2=0x7f5a500cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:30.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 0x7f5a500cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:30.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 -- 172.21.15.157:0/1715588370 >> v1:172.21.15.157:6789/0 conn(0x7f5a500ce1a0 legacy=0x7f5a500d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:30.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4d59a640 1 -- 172.21.15.157:0/1715588370 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549490561 0 0) 0x7f5a500d2920 con 0x7f5a500ce1a0 2023-11-06T03:08:30.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 -- 172.21.15.157:0/1715588370 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500d1e40 con 0x7f5a5015caa0 2023-11-06T03:08:30.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4dd9b640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 0x7f5a500cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4e59c640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a500cdc60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5a400097c0 tx=0x7f5a4001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcba57e7e8dbefd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4d59a640 1 -- 172.21.15.157:0/1715588370 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400ad660 con 0x7f5a5015caa0 2023-11-06T03:08:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4d59a640 1 -- 172.21.15.157:0/1715588370 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a400ad7c0 con 0x7f5a5015caa0 2023-11-06T03:08:30.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:30.996+0000 7f5a4d59a640 1 -- 172.21.15.157:0/1715588370 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400adb50 con 0x7f5a5015caa0 2023-11-06T03:08:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1715588370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 msgr2=0x7f5a500cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a500cdc60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5a400097c0 tx=0x7f5a4001c450 comp rx=0 tx=0).stop 2023-11-06T03:08:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1715588370 shutdown_connections 2023-11-06T03:08:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a5015ce80 0x7f5a500cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1715588370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a500cdc60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1715588370 >> 172.21.15.157:0/1715588370 conn(0x7f5a50070790 msgr2=0x7f5a50070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1715588370 shutdown_connections 2023-11-06T03:08:31.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1715588370 wait complete. 2023-11-06T03:08:31.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 Processor -- start 2023-11-06T03:08:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- start start 2023-11-06T03:08:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a500ce1a0 0x7f5a50156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ce80 0x7f5a5015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015d7f0 con 0x7f5a5015caa0 2023-11-06T03:08:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501555a0 con 0x7f5a5015ce80 2023-11-06T03:08:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015fb70 con 0x7f5a500ce1a0 2023-11-06T03:08:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34142/0 (socket says 172.21.15.157:34142) 2023-11-06T03:08:31.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 -- 172.21.15.157:0/1051037270 learned_addr learned my addr 172.21.15.157:0/1051037270 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 -- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a500ce1a0 msgr2=0x7f5a50156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a500ce1a0 0x7f5a50156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 -- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ce80 msgr2=0x7f5a5015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ce80 0x7f5a5015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 -- 172.21.15.157:0/1051037270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009420 con 0x7f5a5015caa0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a4dd9b640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400c7c0 tx=0x7f5a4400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44007a30 con 0x7f5a5015caa0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a44007b90 con 0x7f5a5015caa0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400ce70 con 0x7f5a5015caa0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5015fe50 con 0x7f5a5015caa0 2023-11-06T03:08:31.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.000+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50160410 con 0x7f5a5015caa0 2023-11-06T03:08:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5015d9b0 con 0x7f5a5015caa0 2023-11-06T03:08:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a440113f0 con 0x7f5a5015caa0 2023-11-06T03:08:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a3effd640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 0x7f5a30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5a44083770 con 0x7f5a5015caa0 2023-11-06T03:08:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a4e59c640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 0x7f5a30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.004+0000 7f5a4e59c640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 0x7f5a30067f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a400af000 tx=0x7f5a40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.008+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5a5015d9b0 con 0x7f5a5015caa0 2023-11-06T03:08:31.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a5015d9b0 con 0x7f5a5015caa0 2023-11-06T03:08:31.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a3effd640 1 -- 172.21.15.157:0/1051037270 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f5a5015d9b0 con 0x7f5a5015caa0 2023-11-06T03:08:31.192 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:31.192 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:28.465085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":15,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:31.193 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 40 2023-11-06T03:08:31.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 msgr2=0x7f5a30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 0x7f5a30067f20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a400af000 tx=0x7f5a40008040 comp rx=0 tx=0).stop 2023-11-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 msgr2=0x7f5a50155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400c7c0 tx=0x7f5a4400cc80 comp rx=0 tx=0).stop 2023-11-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 shutdown_connections 2023-11-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a30065a60 0x7f5a30067f20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a5015ce80 0x7f5a5015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a5015caa0 0x7f5a50155060 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 --2- 172.21.15.157:0/1051037270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a500ce1a0 0x7f5a50156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 >> 172.21.15.157:0/1051037270 conn(0x7f5a50070790 msgr2=0x7f5a500b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.192+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 shutdown_connections 2023-11-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.196+0000 7f5a548e3640 1 -- 172.21.15.157:0/1051037270 wait complete. 2023-11-06T03:08:31.211 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:31.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 Processor -- start 2023-11-06T03:08:31.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 -- start start 2023-11-06T03:08:31.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f26b41551e0 0x7f26b406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26b415e5e0 0x7f26b40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b415ea90 0x7f26b40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b415f580 con 0x7f26b415e5e0 2023-11-06T03:08:31.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b415efc0 con 0x7f26b41551e0 2023-11-06T03:08:31.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.332+0000 7f26bb708640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b415f2a0 con 0x7f26b415ea90 2023-11-06T03:08:31.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26b415e5e0 0x7f26b40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f26b415e5e0 0x7f26b40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34158/0 (socket says 172.21.15.157:34158) 2023-11-06T03:08:31.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 -- 172.21.15.157:0/4132612164 learned_addr learned my addr 172.21.15.157:0/4132612164 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b94a4640 1 --2- 172.21.15.157:0/4132612164 >> v2:172.21.15.195:3300/0 conn(0x7f26b41551e0 0x7f26b406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b9ca5640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b415ea90 0x7f26b40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 -- 172.21.15.157:0/4132612164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b415ea90 msgr2=0x7f26b40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b415ea90 0x7f26b40b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 -- 172.21.15.157:0/4132612164 >> v2:172.21.15.195:3300/0 conn(0x7f26b41551e0 msgr2=0x7f26b406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 --2- 172.21.15.157:0/4132612164 >> v2:172.21.15.195:3300/0 conn(0x7f26b41551e0 0x7f26b406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 -- 172.21.15.157:0/4132612164 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b406f5a0 con 0x7f26b415e5e0 2023-11-06T03:08:31.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26b8ca3640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415e5e0 0x7f26b40b3410 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f26b4066830 tx=0x7f26a401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6149e455ef356d40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26abfff640 1 -- 172.21.15.157:0/4132612164 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40ab620 con 0x7f26b415e5e0 2023-11-06T03:08:31.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26abfff640 1 -- 172.21.15.157:0/4132612164 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a40ab780 con 0x7f26b415e5e0 2023-11-06T03:08:31.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.336+0000 7f26abfff640 1 -- 172.21.15.157:0/4132612164 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40abb10 con 0x7f26b415e5e0 2023-11-06T03:08:31.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/4132612164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415e5e0 msgr2=0x7f26b40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415e5e0 0x7f26b40b3410 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f26b4066830 tx=0x7f26a401c440 comp rx=0 tx=0).stop 2023-11-06T03:08:31.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/4132612164 shutdown_connections 2023-11-06T03:08:31.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b415ea90 0x7f26b40b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- 172.21.15.157:0/4132612164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415e5e0 0x7f26b40b3410 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- 172.21.15.157:0/4132612164 >> v2:172.21.15.195:3300/0 conn(0x7f26b41551e0 0x7f26b406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/4132612164 >> 172.21.15.157:0/4132612164 conn(0x7f26b4065930 msgr2=0x7f26b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/4132612164 shutdown_connections 2023-11-06T03:08:31.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/4132612164 wait complete. 2023-11-06T03:08:31.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 Processor -- start 2023-11-06T03:08:31.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- start start 2023-11-06T03:08:31.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 0x7f26b40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26b415e5e0 0x7f26b40cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40cd4e0 con 0x7f26b415ea90 2023-11-06T03:08:31.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40cd7c0 con 0x7f26b415e5e0 2023-11-06T03:08:31.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40cdaa0 con 0x7f26b41551e0 2023-11-06T03:08:31.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34172/0 (socket says 172.21.15.157:34172) 2023-11-06T03:08:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 -- 172.21.15.157:0/3573224264 learned_addr learned my addr 172.21.15.157:0/3573224264 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b8ca3640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26b415e5e0 0x7f26b40cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b94a4640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 0x7f26b40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 -- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 msgr2=0x7f26b40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 0x7f26b40adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 -- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26b415e5e0 msgr2=0x7f26b40cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26b415e5e0 0x7f26b40cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 -- 172.21.15.157:0/3573224264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40094d0 con 0x7f26b415ea90 2023-11-06T03:08:31.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b94a4640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 0x7f26b40adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:31.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26b9ca5640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f26ac007e60 tx=0x7f26ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac014420 con 0x7f26b415ea90 2023-11-06T03:08:31.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac0145a0 con 0x7f26b415ea90 2023-11-06T03:08:31.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b406b490 con 0x7f26b415ea90 2023-11-06T03:08:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac014950 con 0x7f26b415ea90 2023-11-06T03:08:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b4154850 con 0x7f26b415ea90 2023-11-06T03:08:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.340+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b40cd620 con 0x7f26b415ea90 2023-11-06T03:08:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.344+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26ac014ab0 con 0x7f26b415ea90 2023-11-06T03:08:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.344+0000 7f26a9ffb640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 0x7f2698067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.344+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f26ac083bf0 con 0x7f26b415ea90 2023-11-06T03:08:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.344+0000 7f26b94a4640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 0x7f2698067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.344+0000 7f26b94a4640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 0x7f2698067f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26b0004660 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.348+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f26b40cd620 con 0x7f26b415ea90 2023-11-06T03:08:31.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.492+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b40cd620 con 0x7f26b415ea90 2023-11-06T03:08:31.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26a9ffb640 1 -- 172.21.15.157:0/3573224264 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f26b40cd620 con 0x7f26b415ea90 2023-11-06T03:08:31.495 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:31.495 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:50.274092+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 msgr2=0x7f2698067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 0x7f2698067f20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26b0004660 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2023-11-06T03:08:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 msgr2=0x7f26b41599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f26ac007e60 tx=0x7f26ac00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 shutdown_connections 2023-11-06T03:08:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2698065a60 0x7f2698067f20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b415ea90 0x7f26b41599d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26b415e5e0 0x7f26b40cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 --2- 172.21.15.157:0/3573224264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26b41551e0 0x7f26b40adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 >> 172.21.15.157:0/3573224264 conn(0x7f26b4065930 msgr2=0x7f26b40b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 shutdown_connections 2023-11-06T03:08:31.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.496+0000 7f26bb708640 1 -- 172.21.15.157:0/3573224264 wait complete. 2023-11-06T03:08:31.513 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:08:31.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:31.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 Processor -- start 2023-11-06T03:08:31.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 -- start start 2023-11-06T03:08:31.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7181551e0 0x7fc71806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc71815e5e0 0x7fc7180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc71815ea60 0x7fc7180b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71815f580 con 0x7fc7181551e0 2023-11-06T03:08:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71815efc0 con 0x7fc71815ea60 2023-11-06T03:08:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc71e1cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71815f2a0 con 0x7fc71815e5e0 2023-11-06T03:08:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7181551e0 0x7fc71806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7181551e0 0x7fc71806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34178/0 (socket says 172.21.15.157:34178) 2023-11-06T03:08:31.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 -- 172.21.15.157:0/2579201091 learned_addr learned my addr 172.21.15.157:0/2579201091 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc716ffd640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc71815e5e0 0x7fc7180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 -- 172.21.15.157:0/2579201091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc71815e5e0 msgr2=0x7fc7180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc71815e5e0 0x7fc7180b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc717fff640 1 --2- 172.21.15.157:0/2579201091 >> v2:172.21.15.195:3300/0 conn(0x7fc71815ea60 0x7fc7180b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 -- 172.21.15.157:0/2579201091 >> v2:172.21.15.195:3300/0 conn(0x7fc71815ea60 msgr2=0x7fc7180b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 --2- 172.21.15.157:0/2579201091 >> v2:172.21.15.195:3300/0 conn(0x7fc71815ea60 0x7fc7180b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.632+0000 7fc7177fe640 1 -- 172.21.15.157:0/2579201091 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71806f5a0 con 0x7fc7181551e0 2023-11-06T03:08:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7177fe640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7181551e0 0x7fc71806fda0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc7180cc5d0 tx=0x7fc70c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a456d9b172837792 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7167fc640 1 -- 172.21.15.157:0/2579201091 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0ab620 con 0x7fc7181551e0 2023-11-06T03:08:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7167fc640 1 -- 172.21.15.157:0/2579201091 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c0ab780 con 0x7fc7181551e0 2023-11-06T03:08:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7167fc640 1 -- 172.21.15.157:0/2579201091 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0abb10 con 0x7fc7181551e0 2023-11-06T03:08:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/2579201091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7181551e0 msgr2=0x7fc71806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7181551e0 0x7fc71806fda0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc7180cc5d0 tx=0x7fc70c01c440 comp rx=0 tx=0).stop 2023-11-06T03:08:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/2579201091 shutdown_connections 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/2579201091 >> v2:172.21.15.195:3300/0 conn(0x7fc71815ea60 0x7fc7180b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc71815e5e0 0x7fc7180b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/2579201091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7181551e0 0x7fc71806fda0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/2579201091 >> 172.21.15.157:0/2579201091 conn(0x7fc718065930 msgr2=0x7fc718065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/2579201091 shutdown_connections 2023-11-06T03:08:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/2579201091 wait complete. 2023-11-06T03:08:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 Processor -- start 2023-11-06T03:08:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- start start 2023-11-06T03:08:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 0x7fc7181420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 0x7fc718142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180ce380 con 0x7fc71815ea60 2023-11-06T03:08:31.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180cdae0 con 0x7fc71815e5e0 2023-11-06T03:08:31.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180cddc0 con 0x7fc7181551e0 2023-11-06T03:08:31.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34194/0 (socket says 172.21.15.157:34194) 2023-11-06T03:08:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 -- 172.21.15.157:0/175232440 learned_addr learned my addr 172.21.15.157:0/175232440 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc716ffd640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 0x7fc718142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7177fe640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 0x7fc7181420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 -- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 msgr2=0x7fc7181420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 0x7fc7181420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 -- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 msgr2=0x7fc718142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 0x7fc718142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 -- 172.21.15.157:0/175232440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0094d0 con 0x7fc71815ea60 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc7177fe640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 0x7fc7181420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc716ffd640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 0x7fc718142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:31.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc717fff640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc71000cc80 tx=0x7fc71000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000ce90 con 0x7fc71815ea60 2023-11-06T03:08:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000ae80 con 0x7fc71815ea60 2023-11-06T03:08:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7180ce0a0 con 0x7fc71815ea60 2023-11-06T03:08:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710003be0 con 0x7fc71815ea60 2023-11-06T03:08:31.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.636+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718143b40 con 0x7fc71815ea60 2023-11-06T03:08:31.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7181523e0 con 0x7fc71815ea60 2023-11-06T03:08:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc71000f3f0 con 0x7fc71815ea60 2023-11-06T03:08:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc6fffff640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 0x7fc704067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc7100835f0 con 0x7fc71815ea60 2023-11-06T03:08:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc7177fe640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 0x7fc704067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.640+0000 7fc7177fe640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 0x7fc704067f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0af000 tx=0x7fc70c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.644+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7181523e0 con 0x7fc71815ea60 2023-11-06T03:08:31.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.816+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7181527c0 con 0x7fc71815ea60 2023-11-06T03:08:31.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc6fffff640 1 -- 172.21.15.157:0/175232440 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fc7181527c0 con 0x7fc71815ea60 2023-11-06T03:08:31.819 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:31.819 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:28.465085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":15,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:31.820 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 40 2023-11-06T03:08:31.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 msgr2=0x7fc704067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 0x7fc704067f20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0af000 tx=0x7fc70c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:31.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 msgr2=0x7fc718142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc71000cc80 tx=0x7fc71000a5c0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 shutdown_connections 2023-11-06T03:08:31.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc704065a60 0x7fc704067f20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc71815ea60 0x7fc718142db0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc71815e5e0 0x7fc718142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 --2- 172.21.15.157:0/175232440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7181551e0 0x7fc7181420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 >> 172.21.15.157:0/175232440 conn(0x7fc718065930 msgr2=0x7fc71814e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 shutdown_connections 2023-11-06T03:08:31.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.820+0000 7fc71e1cc640 1 -- 172.21.15.157:0/175232440 wait complete. 2023-11-06T03:08:31.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.948+0000 7f5c0376f640 1 Processor -- start 2023-11-06T03:08:31.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.948+0000 7f5c0376f640 1 -- start start 2023-11-06T03:08:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0376f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5bfc15d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0376f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc06c530 con 0x7f5bfc0b2730 2023-11-06T03:08:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0376f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc15dec0 con 0x7f5bfc06c9c0 2023-11-06T03:08:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0376f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc06d740 con 0x7f5bfc0b11f0 2023-11-06T03:08:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5bfc0b2730 0x7f5bfc0b0ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44638/0 (socket says 172.21.15.157:44638) 2023-11-06T03:08:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 -- 172.21.15.157:0/157604952 learned_addr learned my addr 172.21.15.157:0/157604952 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848051696 0 0) 0x7f5bfc06c530 con 0x7f5bfc0b2730 2023-11-06T03:08:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be8003530 con 0x7f5bfc0b2730 2023-11-06T03:08:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5bfc15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256159848 0 0) 0x7f5be8003530 con 0x7f5bfc0b2730 2023-11-06T03:08:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 msgr2=0x7f5bfc15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5bfc15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 >> v1:172.21.15.195:6789/0 conn(0x7f5bfc06c9c0 legacy=0x7f5bfc15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc06bee0 con 0x7f5bfc0b2730 2023-11-06T03:08:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5bf8001680 con 0x7f5bfc0b2730 2023-11-06T03:08:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 >> v1:172.21.15.157:6789/0 conn(0x7f5bfc0b2730 legacy=0x7f5bfc0b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5be8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5be80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be8004d10 0x7f5be8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be800bb20 con 0x7f5be80039b0 2023-11-06T03:08:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be800bc60 con 0x7f5be8004d10 2023-11-06T03:08:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be800bdf0 con 0x7f5bfc0b11f0 2023-11-06T03:08:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5be8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5be80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c01d0c640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be8004d10 0x7f5be8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 -- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 msgr2=0x7f5be80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5be80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 -- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be8004d10 msgr2=0x7f5be8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be8004d10 0x7f5be8005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 -- 172.21.15.157:0/157604952 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8003530 con 0x7f5be80039b0 2023-11-06T03:08:31.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc0b11f0 0x7f5be80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:31.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.952+0000 7f5c0150b640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5be8003dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c270 tx=0x7f5bf801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e637e207a4d4eaf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8005ae0 con 0x7f5be80039b0 2023-11-06T03:08:31.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8005c60 con 0x7f5be80039b0 2023-11-06T03:08:31.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf3fff640 1 -- 172.21.15.157:0/157604952 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80a5830 con 0x7f5be80039b0 2023-11-06T03:08:31.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/157604952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 msgr2=0x7f5be8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 --2- 172.21.15.157:0/157604952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5be8003dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c270 tx=0x7f5bf801cc40 comp rx=0 tx=0).stop 2023-11-06T03:08:31.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c01d0c640 1 -- 172.21.15.157:0/157604952 reap_dead start 2023-11-06T03:08:31.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/157604952 shutdown_connections 2023-11-06T03:08:31.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/157604952 >> 172.21.15.157:0/157604952 conn(0x7f5bfc069d70 msgr2=0x7f5bfc06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:31.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/157604952 shutdown_connections 2023-11-06T03:08:31.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/157604952 wait complete. 2023-11-06T03:08:31.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 Processor -- start 2023-11-06T03:08:31.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- start start 2023-11-06T03:08:31.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bfc15f6e0 0x7f5bfc0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc15fb60 0x7f5bfc0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc13ef40 con 0x7f5be80039b0 2023-11-06T03:08:31.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc162e80 con 0x7f5bfc15f6e0 2023-11-06T03:08:31.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc163160 con 0x7f5bfc15fb60 2023-11-06T03:08:31.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34212/0 (socket says 172.21.15.157:34212) 2023-11-06T03:08:31.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 -- 172.21.15.157:0/2610655819 learned_addr learned my addr 172.21.15.157:0/2610655819 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:31.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c01d0c640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc15fb60 0x7f5bfc0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bfc15f6e0 0x7f5bfc0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 -- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc15fb60 msgr2=0x7f5bfc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc15fb60 0x7f5bfc0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 -- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bfc15f6e0 msgr2=0x7f5bfc0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:31.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bfc15f6e0 0x7f5bfc0cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:31.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 -- 172.21.15.157:0/2610655819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc13eb60 con 0x7f5be80039b0 2023-11-06T03:08:31.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0150b640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c210 tx=0x7f5be8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80b4070 con 0x7f5be80039b0 2023-11-06T03:08:31.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8005e90 con 0x7f5be80039b0 2023-11-06T03:08:31.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80a5830 con 0x7f5be80039b0 2023-11-06T03:08:31.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc13e610 con 0x7f5be80039b0 2023-11-06T03:08:31.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.956+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc13f530 con 0x7f5be80039b0 2023-11-06T03:08:31.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc13f080 con 0x7f5be80039b0 2023-11-06T03:08:31.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5bf80a5990 con 0x7f5be80039b0 2023-11-06T03:08:31.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5bf1ffb640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 0x7f5be0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:31.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5bf811ce60 con 0x7f5be80039b0 2023-11-06T03:08:31.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 0x7f5be0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:31.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.960+0000 7f5c00d0a640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 0x7f5be0067f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc06d930 tx=0x7f5bec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:31.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:31.964+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5bfc13f080 con 0x7f5be80039b0 2023-11-06T03:08:32.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bfc13f080 con 0x7f5be80039b0 2023-11-06T03:08:32.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5bf1ffb640 1 -- 172.21.15.157:0/2610655819 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f5bfc13f080 con 0x7f5be80039b0 2023-11-06T03:08:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:32.136 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:07:39.644623+0000","modified":"2023-11-06T03:08:28.465085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5868},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5868":{"gid":5868,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":15,"addr":"172.21.15.157:6835/3341473298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3341473298},{"type":"v1","addr":"172.21.15.157:6835","nonce":3341473298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-06T03:08:32.137 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 40 2023-11-06T03:08:32.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 msgr2=0x7f5be0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 0x7f5be0067f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc06d930 tx=0x7f5bec008040 comp rx=0 tx=0).stop 2023-11-06T03:08:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 msgr2=0x7f5bfc13e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.136+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c210 tx=0x7f5be8005800 comp rx=0 tx=0).stop 2023-11-06T03:08:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 shutdown_connections 2023-11-06T03:08:32.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be0065a90 0x7f5be0067f50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bfc15fb60 0x7f5bfc0ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bfc15f6e0 0x7f5bfc0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 --2- 172.21.15.157:0/2610655819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be80039b0 0x7f5bfc13e0d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 >> 172.21.15.157:0/2610655819 conn(0x7f5bfc069d70 msgr2=0x7f5bfc15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 shutdown_connections 2023-11-06T03:08:32.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.140+0000 7f5c0376f640 1 -- 172.21.15.157:0/2610655819 wait complete. 2023-11-06T03:08:32.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.268+0000 7f540edb7640 1 Processor -- start 2023-11-06T03:08:32.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.268+0000 7f540edb7640 1 -- start start 2023-11-06T03:08:32.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540edb7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54080b1910 0x7f5408161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540edb7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540806f470 con 0x7f5408160db0 2023-11-06T03:08:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540edb7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54080b1df0 con 0x7f54080701a0 2023-11-06T03:08:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540edb7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540806f6c0 con 0x7f54080b1910 2023-11-06T03:08:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5408160db0 0x7f540806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44652/0 (socket says 172.21.15.157:44652) 2023-11-06T03:08:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 -- 172.21.15.157:0/3566025281 learned_addr learned my addr 172.21.15.157:0/3566025281 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f5407fff640 1 --2- 172.21.15.157:0/3566025281 >> v2:172.21.15.195:3300/0 conn(0x7f54080b1910 0x7f5408161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488302010 0 0) 0x7f540806f470 con 0x7f5408160db0 2023-11-06T03:08:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f0003530 con 0x7f5408160db0 2023-11-06T03:08:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608527519 0 0) 0x7f53f0003530 con 0x7f5408160db0 2023-11-06T03:08:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 >> v2:172.21.15.195:3300/0 conn(0x7f54080b1910 msgr2=0x7f5408161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 --2- 172.21.15.157:0/3566025281 >> v2:172.21.15.195:3300/0 conn(0x7f54080b1910 0x7f5408161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 >> v1:172.21.15.195:6789/0 conn(0x7f54080701a0 legacy=0x7f54080ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540806f9a0 con 0x7f5408160db0 2023-11-06T03:08:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f53fc001680 con 0x7f5408160db0 2023-11-06T03:08:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 >> v1:172.21.15.157:6789/0 conn(0x7f5408160db0 legacy=0x7f540806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 0x7f53f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00047d0 0x7f53f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f00051e0 0x7f53f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540806f6c0 con 0x7f53f00039b0 2023-11-06T03:08:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f000c110 con 0x7f53f00051e0 2023-11-06T03:08:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f000c2d0 con 0x7f53f00047d0 2023-11-06T03:08:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 0x7f53f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f5407fff640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00047d0 0x7f53f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540d354640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f00051e0 0x7f53f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 -- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00047d0 msgr2=0x7f53f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00047d0 0x7f53f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 -- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f00051e0 msgr2=0x7f53f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53f00051e0 0x7f53f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540cb53640 1 -- 172.21.15.157:0/3566025281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0003530 con 0x7f53f00039b0 2023-11-06T03:08:32.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.272+0000 7f540d354640 1 -- 172.21.15.157:0/3566025281 reap_dead start 2023-11-06T03:08:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540cb53640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 0x7f53f0003dd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c240 tx=0x7f53fc01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d2039f5e947b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc005ae0 con 0x7f53f00039b0 2023-11-06T03:08:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc005c60 con 0x7f53f00039b0 2023-11-06T03:08:32.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f54077fe640 1 -- 172.21.15.157:0/3566025281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a57b0 con 0x7f53f00039b0 2023-11-06T03:08:32.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 msgr2=0x7f53f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 0x7f53f0003dd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c240 tx=0x7f53fc01cc10 comp rx=0 tx=0).stop 2023-11-06T03:08:32.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/3566025281 shutdown_connections 2023-11-06T03:08:32.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 --2- 172.21.15.157:0/3566025281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f00039b0 0x7f53f0003dd0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/3566025281 >> 172.21.15.157:0/3566025281 conn(0x7f5408065930 msgr2=0x7f5408065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/3566025281 shutdown_connections 2023-11-06T03:08:32.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/3566025281 wait complete. 2023-11-06T03:08:32.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 Processor -- start 2023-11-06T03:08:32.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- start start 2023-11-06T03:08:32.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00039b0 0x7f5408142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408153c00 0x7f540815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408143000 con 0x7f5408153780 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5408157bc0 con 0x7f5408153c00 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5408158880 con 0x7f53f00039b0 2023-11-06T03:08:32.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34224/0 (socket says 172.21.15.157:34224) 2023-11-06T03:08:32.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 -- 172.21.15.157:0/2397454287 learned_addr learned my addr 172.21.15.157:0/2397454287 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:32.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540cb53640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00039b0 0x7f5408142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540d354640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408153c00 0x7f540815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 -- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00039b0 msgr2=0x7f5408142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00039b0 0x7f5408142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 -- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408153c00 msgr2=0x7f540815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408153c00 0x7f540815f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 -- 172.21.15.157:0/2397454287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408158b60 con 0x7f5408153780 2023-11-06T03:08:32.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f5407fff640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011490 tx=0x7f53f8011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004500 con 0x7f5408153780 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f8003620 con 0x7f5408153780 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.276+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5408158e40 con 0x7f5408153780 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f80039b0 con 0x7f5408153780 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5408159640 con 0x7f5408153780 2023-11-06T03:08:32.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5408072750 con 0x7f5408153780 2023-11-06T03:08:32.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53f8003b10 con 0x7f5408153780 2023-11-06T03:08:32.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f54057fa640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 0x7f53f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f53f8083ca0 con 0x7f5408153780 2023-11-06T03:08:32.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f540cb53640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 0x7f53f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.280+0000 7f540cb53640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 0x7f53f4067f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0ab000 tx=0x7f53fc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.284+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f53f80514a0 con 0x7f5408153780 2023-11-06T03:08:32.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.444+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5408071020 con 0x7f5408153780 2023-11-06T03:08:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.444+0000 7f54057fa640 1 -- 172.21.15.157:0/2397454287 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f53f8054ac0 con 0x7f5408153780 2023-11-06T03:08:32.446 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:32.446 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":55,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:07:50.274092+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:07:37.373400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-06T03:07:38.627196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:32.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 msgr2=0x7f53f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 0x7f53f4067f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0ab000 tx=0x7f53fc0a7070 comp rx=0 tx=0).stop 2023-11-06T03:08:32.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 msgr2=0x7f540815ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011490 tx=0x7f53f8011950 comp rx=0 tx=0).stop 2023-11-06T03:08:32.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 shutdown_connections 2023-11-06T03:08:32.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f53f4065a60 0x7f53f4067f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5408153c00 0x7f540815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5408153780 0x7f540815ef30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 --2- 172.21.15.157:0/2397454287 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53f00039b0 0x7f5408142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 >> 172.21.15.157:0/2397454287 conn(0x7f5408065930 msgr2=0x7f54081615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 shutdown_connections 2023-11-06T03:08:32.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.448+0000 7f540edb7640 1 -- 172.21.15.157:0/2397454287 wait complete. 2023-11-06T03:08:32.465 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:08:32.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 Processor -- start 2023-11-06T03:08:32.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 -- start start 2023-11-06T03:08:32.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb710065d40 0x7fb7100661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7100cef40 con 0x7fb710144430 2023-11-06T03:08:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb710066700 con 0x7fb71015b280 2023-11-06T03:08:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb715d8d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100cd450 con 0x7fb710065d40 2023-11-06T03:08:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb710144430 0x7fb7100b0ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44668/0 (socket says 172.21.15.157:44668) 2023-11-06T03:08:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2013816552 learned_addr learned my addr 172.21.15.157:0/2013816552 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb706ffd640 1 --2- 172.21.15.157:0/2013816552 >> v2:172.21.15.195:3300/0 conn(0x7fb710065d40 0x7fb7100661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918166789 0 0) 0x7fb7100cef40 con 0x7fb710144430 2023-11-06T03:08:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6fc003530 con 0x7fb710144430 2023-11-06T03:08:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191624105 0 0) 0x7fb710066700 con 0x7fb71015b280 2023-11-06T03:08:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7100cef40 con 0x7fb71015b280 2023-11-06T03:08:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387104774 0 0) 0x7fb6fc003530 con 0x7fb710144430 2023-11-06T03:08:32.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 >> v2:172.21.15.195:3300/0 conn(0x7fb710065d40 msgr2=0x7fb7100661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 --2- 172.21.15.157:0/2013816552 >> v2:172.21.15.195:3300/0 conn(0x7fb710065d40 0x7fb7100661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 >> v1:172.21.15.195:6789/0 conn(0x7fb71015b280 legacy=0x7fb71015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100cc840 con 0x7fb710144430 2023-11-06T03:08:32.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb700001600 con 0x7fb710144430 2023-11-06T03:08:32.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 >> v1:172.21.15.157:6789/0 conn(0x7fb710144430 legacy=0x7fb7100b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb6fc003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6fc0047d0 0x7fb6fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6fc005190 0x7fb6fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100cd450 con 0x7fb6fc003b60 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc00c060 con 0x7fb6fc005190 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc00c220 con 0x7fb6fc0047d0 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb6fc003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb706ffd640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6fc0047d0 0x7fb6fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70ffff640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6fc005190 0x7fb6fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6fc0047d0 msgr2=0x7fb6fc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb6fc0047d0 0x7fb6fc004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6fc005190 msgr2=0x7fb6fc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6fc005190 0x7fb6fc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2013816552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc003530 con 0x7fb6fc003b60 2023-11-06T03:08:32.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70ffff640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6fc005190 0x7fb6fc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:32.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.584+0000 7fb70ffff640 1 -- 172.21.15.157:0/2013816552 reap_dead start 2023-11-06T03:08:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb6fc003f60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb700007f50 tx=0x7fb70001caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888c2881b140518a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700005ba0 con 0x7fb6fc003b60 2023-11-06T03:08:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700005d00 con 0x7fb6fc003b60 2023-11-06T03:08:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb70effd640 1 -- 172.21.15.157:0/2013816552 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700007270 con 0x7fb6fc003b60 2023-11-06T03:08:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 msgr2=0x7fb6fc003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb6fc003f60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb700007f50 tx=0x7fb70001caa0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- 172.21.15.157:0/2013816552 shutdown_connections 2023-11-06T03:08:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2013816552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb6fc003f60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- 172.21.15.157:0/2013816552 >> 172.21.15.157:0/2013816552 conn(0x7fb710065930 msgr2=0x7fb710071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- 172.21.15.157:0/2013816552 shutdown_connections 2023-11-06T03:08:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- 172.21.15.157:0/2013816552 wait complete. 2023-11-06T03:08:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 Processor -- start 2023-11-06T03:08:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.588+0000 7fb715d8d640 1 -- start start 2023-11-06T03:08:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb71013d8c0 0x7fb710139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 0x7fb7101396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71013b0d0 con 0x7fb6fc003b60 2023-11-06T03:08:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71015cb80 con 0x7fb71013d8c0 2023-11-06T03:08:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71013b300 con 0x7fb71013dd00 2023-11-06T03:08:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34254/0 (socket says 172.21.15.157:34254) 2023-11-06T03:08:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2848451589 learned_addr learned my addr 172.21.15.157:0/2848451589 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb706ffd640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb71013d8c0 0x7fb710139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70ffff640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 0x7fb7101396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 msgr2=0x7fb7101396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 0x7fb7101396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb71013d8c0 msgr2=0x7fb710139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb71013d8c0 0x7fb710139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 -- 172.21.15.157:0/2848451589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71013b5e0 con 0x7fb6fc003b60 2023-11-06T03:08:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70ffff640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 0x7fb7101396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70f7fe640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb700007f50 tx=0x7fb6fc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000b3070 con 0x7fb6fc003b60 2023-11-06T03:08:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700007650 con 0x7fb6fc003b60 2023-11-06T03:08:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71015cdd0 con 0x7fb6fc003b60 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700005ba0 con 0x7fb6fc003b60 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100a84a0 con 0x7fb6fc003b60 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710072670 con 0x7fb6fc003b60 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb700001400 con 0x7fb6fc003b60 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb70011c770 con 0x7fb6fc003b60 2023-11-06T03:08:32.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.592+0000 7fb706ffd640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.596+0000 7fb706ffd640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 0x7fb6f0067f50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7100cee10 tx=0x7fb6f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.596+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7000ea020 con 0x7fb6fc003b60 2023-11-06T03:08:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.772+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb7100afad0 con 0x7fb6fc003b60 2023-11-06T03:08:32.773 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:08:32.772+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:08:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.788+0000 7fb70cff9640 1 -- 172.21.15.157:0/2848451589 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb7000bc090 con 0x7fb6fc003b60 2023-11-06T03:08:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:08:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 msgr2=0x7fb6f0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 0x7fb6f0067f50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb7100cee10 tx=0x7fb6f8008040 comp rx=0 tx=0).stop 2023-11-06T03:08:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 msgr2=0x7fb71013ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb700007f50 tx=0x7fb6fc005800 comp rx=0 tx=0).stop 2023-11-06T03:08:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 shutdown_connections 2023-11-06T03:08:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb71013dd00 0x7fb7101396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb71013d8c0 0x7fb710139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 --2- 172.21.15.157:0/2848451589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc003b60 0x7fb71013ba60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 >> 172.21.15.157:0/2848451589 conn(0x7fb710065930 msgr2=0x7fb710143120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 shutdown_connections 2023-11-06T03:08:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.792+0000 7fb715d8d640 1 -- 172.21.15.157:0/2848451589 wait complete. 2023-11-06T03:08:32.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:08:32.852 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:08:32.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 Processor -- start 2023-11-06T03:08:32.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 -- start start 2023-11-06T03:08:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d4c15caa0 0x7f5d4c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5d4c15ce80 0x7f5d4c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0d2920 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0d18a0 con 0x7f5d4c15caa0 2023-11-06T03:08:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d541d6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0d1b60 con 0x7f5d4c15ce80 2023-11-06T03:08:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51f72640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5d4c15caa0 0x7f5d4c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d52773640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5d4c0ce1a0 0x7f5d4c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44686/0 (socket says 172.21.15.157:44686) 2023-11-06T03:08:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d52773640 1 -- 172.21.15.157:0/2567509313 learned_addr learned my addr 172.21.15.157:0/2567509313 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51771640 1 --2- 172.21.15.157:0/2567509313 >> v2:172.21.15.195:3300/0 conn(0x7f5d4c15ce80 0x7f5d4c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51f72640 1 -- 172.21.15.157:0/2567509313 >> v2:172.21.15.195:3300/0 conn(0x7f5d4c15ce80 msgr2=0x7f5d4c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:32.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51f72640 1 --2- 172.21.15.157:0/2567509313 >> v2:172.21.15.195:3300/0 conn(0x7f5d4c15ce80 0x7f5d4c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51f72640 1 -- 172.21.15.157:0/2567509313 >> v1:172.21.15.157:6789/0 conn(0x7f5d4c0ce1a0 legacy=0x7f5d4c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d51f72640 1 -- 172.21.15.157:0/2567509313 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0d1e40 con 0x7f5d4c15caa0 2023-11-06T03:08:32.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.928+0000 7f5d50f70640 1 -- 172.21.15.157:0/2567509313 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726858332 0 0) 0x7f5d4c0d2920 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d51f72640 1 --2- 172.21.15.157:0/2567509313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c0cdc60 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0c6720 tx=0x7f5d4801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb10d5bab03f4621 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d50f70640 1 -- 172.21.15.157:0/2567509313 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480ab600 con 0x7f5d4c15caa0 2023-11-06T03:08:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d50f70640 1 -- 172.21.15.157:0/2567509313 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d480a7050 con 0x7f5d4c15caa0 2023-11-06T03:08:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d50f70640 1 -- 172.21.15.157:0/2567509313 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480ab980 con 0x7f5d4c15caa0 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- 172.21.15.157:0/2567509313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c15caa0 msgr2=0x7f5d4c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 --2- 172.21.15.157:0/2567509313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c0cdc60 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0c6720 tx=0x7f5d4801c430 comp rx=0 tx=0).stop 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- 172.21.15.157:0/2567509313 shutdown_connections 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 --2- 172.21.15.157:0/2567509313 >> v2:172.21.15.195:3300/0 conn(0x7f5d4c15ce80 0x7f5d4c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 --2- 172.21.15.157:0/2567509313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c0cdc60 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- 172.21.15.157:0/2567509313 >> 172.21.15.157:0/2567509313 conn(0x7f5d4c070790 msgr2=0x7f5d4c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- 172.21.15.157:0/2567509313 shutdown_connections 2023-11-06T03:08:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- 172.21.15.157:0/2567509313 wait complete. 2023-11-06T03:08:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 Processor -- start 2023-11-06T03:08:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.932+0000 7f5d541d6640 1 -- start start 2023-11-06T03:08:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d4c15ce80 0x7f5d4c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c151fb0 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c152230 con 0x7f5d4c15caa0 2023-11-06T03:08:32.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c152510 con 0x7f5d4c15ce80 2023-11-06T03:08:32.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34286/0 (socket says 172.21.15.157:34286) 2023-11-06T03:08:32.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 -- 172.21.15.157:0/3577046609 learned_addr learned my addr 172.21.15.157:0/3577046609 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:32.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 -- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d4c15ce80 msgr2=0x7f5d4c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d4c15ce80 0x7f5d4c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51771640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 -- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d4c15caa0 msgr2=0x7f5d4c154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:32.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:32.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 -- 172.21.15.157:0/3577046609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48009420 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51f72640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5d48019040 tx=0x7f5d4801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48004230 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d48004390 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c1527f0 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003820 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c152db0 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c152fa0 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5d480b2050 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 0x7f5d30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:32.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f5d4811d5a0 con 0x7f5d4c0ce1a0 2023-11-06T03:08:32.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.936+0000 7f5d51771640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 0x7f5d30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:32.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.940+0000 7f5d51771640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 0x7f5d30067f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ccd80 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:32.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:32.940+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5d480eae00 con 0x7f5d4c0ce1a0 2023-11-06T03:08:33.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.116+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5d4c072750 con 0x7f5d4c0ce1a0 2023-11-06T03:08:33.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.800+0000 7f5d3e7fc640 1 -- 172.21.15.157:0/3577046609 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5d48006e20 con 0x7f5d4c0ce1a0 2023-11-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 msgr2=0x7f5d30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 0x7f5d30067f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ccd80 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2023-11-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 msgr2=0x7f5d4c0cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5d48019040 tx=0x7f5d4801ac70 comp rx=0 tx=0).stop 2023-11-06T03:08:33.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 shutdown_connections 2023-11-06T03:08:33.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5d30065a60 0x7f5d30067f20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:33.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d4c15ce80 0x7f5d4c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:33.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d4c15caa0 0x7f5d4c154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:33.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 --2- 172.21.15.157:0/3577046609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d4c0ce1a0 0x7f5d4c0cbd20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 >> 172.21.15.157:0/3577046609 conn(0x7f5d4c070790 msgr2=0x7f5d4c0c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 shutdown_connections 2023-11-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.804+0000 7f5d541d6640 1 -- 172.21.15.157:0/3577046609 wait complete. 2023-11-06T03:08:33.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:08:33.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 Processor -- start 2023-11-06T03:08:33.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 -- start start 2023-11-06T03:08:33.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1081593d0 0x7fe1080ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:33.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe1080cb820 0x7fe1080cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:33.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1081597b0 con 0x7fe10806f290 2023-11-06T03:08:33.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080b2b20 con 0x7fe1081593d0 2023-11-06T03:08:33.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe10d95a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080b2e00 con 0x7fe1080cb820 2023-11-06T03:08:33.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1081593d0 0x7fe1080ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:33.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1081593d0 0x7fe1080ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34288/0 (socket says 172.21.15.157:34288) 2023-11-06T03:08:33.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 -- 172.21.15.157:0/3367637808 learned_addr learned my addr 172.21.15.157:0/3367637808 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:33.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe1077fe640 1 --2- 172.21.15.157:0/3367637808 >> v2:172.21.15.195:3300/0 conn(0x7fe1080cb820 0x7fe1080cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:33.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 -- 172.21.15.157:0/3367637808 >> v2:172.21.15.195:3300/0 conn(0x7fe1080cb820 msgr2=0x7fe1080cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:33.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 --2- 172.21.15.157:0/3367637808 >> v2:172.21.15.195:3300/0 conn(0x7fe1080cb820 0x7fe1080cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 -- 172.21.15.157:0/3367637808 >> v1:172.21.15.195:6789/0 conn(0x7fe10806f290 legacy=0x7fe1080cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:34.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:33.996+0000 7fe106ffd640 1 -- 172.21.15.157:0/3367637808 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080b30e0 con 0x7fe1081593d0 2023-11-06T03:08:34.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe106ffd640 1 --2- 172.21.15.157:0/3367637808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1081593d0 0x7fe1080ca830 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe10806bd00 tx=0x7fe0f801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c84ff9dad40d11d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe105ffb640 1 -- 172.21.15.157:0/3367637808 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80aa620 con 0x7fe1081593d0 2023-11-06T03:08:34.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe105ffb640 1 -- 172.21.15.157:0/3367637808 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80aa780 con 0x7fe1081593d0 2023-11-06T03:08:34.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe105ffb640 1 -- 172.21.15.157:0/3367637808 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80aab10 con 0x7fe1081593d0 2023-11-06T03:08:34.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- 172.21.15.157:0/3367637808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1081593d0 msgr2=0x7fe1080ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 --2- 172.21.15.157:0/3367637808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1081593d0 0x7fe1080ca830 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe10806bd00 tx=0x7fe0f801c4c0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- 172.21.15.157:0/3367637808 shutdown_connections 2023-11-06T03:08:34.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 --2- 172.21.15.157:0/3367637808 >> v2:172.21.15.195:3300/0 conn(0x7fe1080cb820 0x7fe1080cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 --2- 172.21.15.157:0/3367637808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1081593d0 0x7fe1080ca830 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- 172.21.15.157:0/3367637808 >> 172.21.15.157:0/3367637808 conn(0x7fe1080b43b0 msgr2=0x7fe108065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:34.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- 172.21.15.157:0/3367637808 shutdown_connections 2023-11-06T03:08:34.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- 172.21.15.157:0/3367637808 wait complete. 2023-11-06T03:08:34.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 Processor -- start 2023-11-06T03:08:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.000+0000 7fe10d95a640 1 -- start start 2023-11-06T03:08:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1080cb820 0x7fe1081599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1081593d0 0x7fe108157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10814c9a0 con 0x7fe10806f290 2023-11-06T03:08:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10814c300 con 0x7fe1080cb820 2023-11-06T03:08:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10815c7f0 con 0x7fe1081593d0 2023-11-06T03:08:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34298/0 (socket says 172.21.15.157:34298) 2023-11-06T03:08:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 -- 172.21.15.157:0/2814911365 learned_addr learned my addr 172.21.15.157:0/2814911365 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe1067fc640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1080cb820 0x7fe1081599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 -- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1081593d0 msgr2=0x7fe108157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1081593d0 0x7fe108157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 -- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1080cb820 msgr2=0x7fe1081599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1080cb820 0x7fe1081599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 -- 172.21.15.157:0/2814911365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80094d0 con 0x7fe10806f290 2023-11-06T03:08:34.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe106ffd640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ac040 tx=0x7fe0f801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80bd3f0 con 0x7fe10806f290 2023-11-06T03:08:34.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80bd570 con 0x7fe10806f290 2023-11-06T03:08:34.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80bd920 con 0x7fe10806f290 2023-11-06T03:08:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108144df0 con 0x7fe10806f290 2023-11-06T03:08:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10814c5c0 con 0x7fe10806f290 2023-11-06T03:08:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10804e050 con 0x7fe10806f290 2023-11-06T03:08:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe0f80b33f0 con 0x7fe10806f290 2023-11-06T03:08:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 0x7fe0f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fe0f811d660 con 0x7fe10806f290 2023-11-06T03:08:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.004+0000 7fe1067fc640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 0x7fe0f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.008+0000 7fe1067fc640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 0x7fe0f4067f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cad00 tx=0x7fe0fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.008+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe0f80eb090 con 0x7fe10806f290 2023-11-06T03:08:34.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.152+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe10814dd70 con 0x7fe10806f290 2023-11-06T03:08:34.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.804+0000 7fe0effff640 1 -- 172.21.15.157:0/2814911365 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0f80ee6b0 con 0x7fe10806f290 2023-11-06T03:08:34.806 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:08:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 msgr2=0x7fe0f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 0x7fe0f4067f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cad00 tx=0x7fe0fc009040 comp rx=0 tx=0).stop 2023-11-06T03:08:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 msgr2=0x7fe10815c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ac040 tx=0x7fe0f801ad00 comp rx=0 tx=0).stop 2023-11-06T03:08:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 shutdown_connections 2023-11-06T03:08:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe0f4065a60 0x7fe0f4067f20 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cad00 tx=0x7fe0fc009040 comp rx=0 tx=0).stop 2023-11-06T03:08:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1081593d0 0x7fe108157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1080cb820 0x7fe1081599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 --2- 172.21.15.157:0/2814911365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe10806f290 0x7fe10815c2b0 secure :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ac040 tx=0x7fe0f801ad00 comp rx=0 tx=0).stop 2023-11-06T03:08:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 >> 172.21.15.157:0/2814911365 conn(0x7fe1080b43b0 msgr2=0x7fe1080ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 shutdown_connections 2023-11-06T03:08:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.812+0000 7fe10d95a640 1 -- 172.21.15.157:0/2814911365 wait complete. 2023-11-06T03:08:34.833 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:08:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 Processor -- start 2023-11-06T03:08:34.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 -- start start 2023-11-06T03:08:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0dc0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0d0410 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0d27c0 con 0x7fc0dc15cc70 2023-11-06T03:08:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e4618640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0d0f90 con 0x7fc0dc0cfa90 2023-11-06T03:08:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc0dc0b15b0 0x7fc0dc0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44692/0 (socket says 172.21.15.157:44692) 2023-11-06T03:08:34.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 -- 172.21.15.157:0/2827328215 learned_addr learned my addr 172.21.15.157:0/2827328215 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:34.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0dc0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564615743 0 0) 0x7fc0dc0d0410 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c8003530 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585860051 0 0) 0x7fc0c8003530 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 msgr2=0x7fc0dc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:34.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0dc0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 >> v1:172.21.15.195:6789/0 conn(0x7fc0dc15cc70 legacy=0x7fc0dc0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0d1270 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc0d4001a00 con 0x7fc0dc0b15b0 2023-11-06T03:08:34.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 >> v1:172.21.15.157:6789/0 conn(0x7fc0dc0b15b0 legacy=0x7fc0dc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c80039b0 0x7fc0c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c8004d10 0x7fc0c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0d0f90 con 0x7fc0c8004d10 2023-11-06T03:08:34.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c800bb20 con 0x7fc0c80039b0 2023-11-06T03:08:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c800bcb0 con 0x7fc0dc0cfa90 2023-11-06T03:08:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c8004d10 0x7fc0c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e23b4640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c80039b0 0x7fc0c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 -- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 msgr2=0x7fc0c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0c80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 -- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c80039b0 msgr2=0x7fc0c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0c80039b0 0x7fc0c8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e2bb5640 1 -- 172.21.15.157:0/2827328215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c8003530 con 0x7fc0c8004d10 2023-11-06T03:08:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.956+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc0cfa90 0x7fc0c80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:34.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e2bb5640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c8004d10 0x7fc0c8005190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400a860 tx=0x7fc0d401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8622d356989c30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40072d0 con 0x7fc0c8004d10 2023-11-06T03:08:34.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d40a8070 con 0x7fc0c8004d10 2023-11-06T03:08:34.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e13b2640 1 -- 172.21.15.157:0/2827328215 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40076b0 con 0x7fc0c8004d10 2023-11-06T03:08:34.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- 172.21.15.157:0/2827328215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c8004d10 msgr2=0x7fc0c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 --2- 172.21.15.157:0/2827328215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0c8004d10 0x7fc0c8005190 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400a860 tx=0x7fc0d401d320 comp rx=0 tx=0).stop 2023-11-06T03:08:34.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e2bb5640 1 -- 172.21.15.157:0/2827328215 reap_dead start 2023-11-06T03:08:34.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- 172.21.15.157:0/2827328215 shutdown_connections 2023-11-06T03:08:34.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- 172.21.15.157:0/2827328215 >> 172.21.15.157:0/2827328215 conn(0x7fc0dc070790 msgr2=0x7fc0dc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:34.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- 172.21.15.157:0/2827328215 shutdown_connections 2023-11-06T03:08:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- 172.21.15.157:0/2827328215 wait complete. 2023-11-06T03:08:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 Processor -- start 2023-11-06T03:08:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- start start 2023-11-06T03:08:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0dc065b90 0x7fc0dc1f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc065f70 0x7fc0dc1f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1f33d0 con 0x7fc0dc1f3530 2023-11-06T03:08:34.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc154fb0 con 0x7fc0dc065b90 2023-11-06T03:08:34.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e4618640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1f68b0 con 0x7fc0dc065f70 2023-11-06T03:08:34.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34318/0 (socket says 172.21.15.157:34318) 2023-11-06T03:08:34.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 -- 172.21.15.157:0/3343515911 learned_addr learned my addr 172.21.15.157:0/3343515911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:34.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 -- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc065f70 msgr2=0x7fc0dc1f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc065f70 0x7fc0dc1f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 -- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0dc065b90 msgr2=0x7fc0dc1f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:34.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0dc065b90 0x7fc0dc1f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:34.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 -- 172.21.15.157:0/3343515911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc1f27c0 con 0x7fc0dc1f3530 2023-11-06T03:08:34.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.960+0000 7fc0e1bb3640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e810 tx=0x7fc0cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00c650 con 0x7fc0dc1f3530 2023-11-06T03:08:34.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00c7b0 con 0x7fc0dc1f3530 2023-11-06T03:08:34.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00cb20 con 0x7fc0dc1f3530 2023-11-06T03:08:34.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc1f2aa0 con 0x7fc0dc1f3530 2023-11-06T03:08:34.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc1f3060 con 0x7fc0dc1f3530 2023-11-06T03:08:34.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc04e050 con 0x7fc0dc1f3530 2023-11-06T03:08:34.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc0cc00cc80 con 0x7fc0dc1f3530 2023-11-06T03:08:34.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 0x7fc0c0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:34.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fc0cc082ec0 con 0x7fc0dc1f3530 2023-11-06T03:08:34.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0e23b4640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 0x7fc0c0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:34.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.964+0000 7fc0e23b4640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 0x7fc0c0067f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8007fd0 tx=0x7fc0d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:34.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:34.968+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc0cc050840 con 0x7fc0dc1f3530 2023-11-06T03:08:35.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.120+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0dc155120 con 0x7fc0dc1f3530 2023-11-06T03:08:35.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.820+0000 7fc0d2ffd640 1 -- 172.21.15.157:0/3343515911 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc0dc155120 con 0x7fc0dc1f3530 2023-11-06T03:08:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:08:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 msgr2=0x7fc0c0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 0x7fc0c0067f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8007fd0 tx=0x7fc0d8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 msgr2=0x7fc0dc1f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e810 tx=0x7fc0cc00ecd0 comp rx=0 tx=0).stop 2023-11-06T03:08:35.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 shutdown_connections 2023-11-06T03:08:35.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0c0065a90 0x7fc0c0067f50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:35.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0dc065f70 0x7fc0dc1f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:35.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0dc1f3530 0x7fc0dc1f4d70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:35.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 --2- 172.21.15.157:0/3343515911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0dc065b90 0x7fc0dc1f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:35.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 >> 172.21.15.157:0/3343515911 conn(0x7fc0dc070790 msgr2=0x7fc0dc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:35.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 shutdown_connections 2023-11-06T03:08:35.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.824+0000 7fc0e4618640 1 -- 172.21.15.157:0/3343515911 wait complete. 2023-11-06T03:08:35.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:08:35.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 Processor -- start 2023-11-06T03:08:35.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 -- start start 2023-11-06T03:08:35.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 0x7fd1740afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:35.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd17406b8b0 0x7fd1740cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd174071ea0 con 0x7fd1740cd7e0 2023-11-06T03:08:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17415d6e0 con 0x7fd17406b8b0 2023-11-06T03:08:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17bf71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740cdc50 con 0x7fd1740cc5d0 2023-11-06T03:08:35.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17950c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd17406b8b0 0x7fd1740cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17a50e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd1740cd7e0 0x7fd174160d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44698/0 (socket says 172.21.15.157:44698) 2023-11-06T03:08:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17a50e640 1 -- 172.21.15.157:0/3083607602 learned_addr learned my addr 172.21.15.157:0/3083607602 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd179d0d640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 0x7fd1740afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17950c640 1 -- 172.21.15.157:0/3083607602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 msgr2=0x7fd1740afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:36.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17950c640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 0x7fd1740afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17950c640 1 -- 172.21.15.157:0/3083607602 >> v1:172.21.15.157:6789/0 conn(0x7fd1740cd7e0 legacy=0x7fd174160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd17950c640 1 -- 172.21.15.157:0/3083607602 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740cfdc0 con 0x7fd17406b8b0 2023-11-06T03:08:36.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:35.996+0000 7fd179d0d640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 0x7fd1740afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:36.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17950c640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd17406b8b0 0x7fd1740cd2a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097f0 tx=0x7fd16401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360630b648b832f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd178d0b640 1 -- 172.21.15.157:0/3083607602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640a8970 con 0x7fd17406b8b0 2023-11-06T03:08:36.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd178d0b640 1 -- 172.21.15.157:0/3083607602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1640a8ad0 con 0x7fd17406b8b0 2023-11-06T03:08:36.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd178d0b640 1 -- 172.21.15.157:0/3083607602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640a8e60 con 0x7fd17406b8b0 2023-11-06T03:08:36.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 -- 172.21.15.157:0/3083607602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd17406b8b0 msgr2=0x7fd1740cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd17406b8b0 0x7fd1740cd2a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097f0 tx=0x7fd16401ec70 comp rx=0 tx=0).stop 2023-11-06T03:08:36.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 -- 172.21.15.157:0/3083607602 shutdown_connections 2023-11-06T03:08:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd17406b8b0 0x7fd1740cd2a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 --2- 172.21.15.157:0/3083607602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1740cc5d0 0x7fd1740afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 -- 172.21.15.157:0/3083607602 >> 172.21.15.157:0/3083607602 conn(0x7fd1740b23e0 msgr2=0x7fd1740b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 -- 172.21.15.157:0/3083607602 shutdown_connections 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.000+0000 7fd17bf71640 1 -- 172.21.15.157:0/3083607602 wait complete. 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 Processor -- start 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- start start 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd17406b8b0 0x7fd1740b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 0x7fd17414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740ca8b0 con 0x7fd1740cd7e0 2023-11-06T03:08:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740ca000 con 0x7fd1740cc5d0 2023-11-06T03:08:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740c8190 con 0x7fd17406b8b0 2023-11-06T03:08:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34326/0 (socket says 172.21.15.157:34326) 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 -- 172.21.15.157:0/2407443299 learned_addr learned my addr 172.21.15.157:0/2407443299 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd179d0d640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd17406b8b0 0x7fd1740b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17950c640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 0x7fd17414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 -- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd17406b8b0 msgr2=0x7fd1740b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd17406b8b0 0x7fd1740b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 -- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 msgr2=0x7fd17414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 0x7fd17414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 -- 172.21.15.157:0/2407443299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164009420 con 0x7fd1740cd7e0 2023-11-06T03:08:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17950c640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 0x7fd17414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17a50e640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00ca80 tx=0x7fd16c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00ac30 con 0x7fd1740cd7e0 2023-11-06T03:08:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c00ad90 con 0x7fd1740cd7e0 2023-11-06T03:08:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1740c7eb0 con 0x7fd1740cd7e0 2023-11-06T03:08:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c012600 con 0x7fd1740cd7e0 2023-11-06T03:08:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1740ca540 con 0x7fd1740cd7e0 2023-11-06T03:08:36.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd174070ca0 con 0x7fd1740cd7e0 2023-11-06T03:08:36.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd16c007070 con 0x7fd1740cd7e0 2023-11-06T03:08:36.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 0x7fd158067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd16c082da0 con 0x7fd1740cd7e0 2023-11-06T03:08:36.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.004+0000 7fd179d0d640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 0x7fd158067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.008+0000 7fd179d0d640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 0x7fd158067f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.008+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd16c0509b0 con 0x7fd1740cd7e0 2023-11-06T03:08:36.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.188+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd1740d0320 con 0x7fd1740cd7e0 2023-11-06T03:08:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.196+0000 7fd16a7fc640 1 -- 172.21.15.157:0/2407443299 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd1740d0320 con 0x7fd1740cd7e0 2023-11-06T03:08:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 msgr2=0x7fd158067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 0x7fd158067f50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170008040 comp rx=0 tx=0).stop 2023-11-06T03:08:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 msgr2=0x7fd1740cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00ca80 tx=0x7fd16c00cf40 comp rx=0 tx=0).stop 2023-11-06T03:08:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 shutdown_connections 2023-11-06T03:08:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd158065a90 0x7fd158067f50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1740cd7e0 0x7fd1740cad40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1740cc5d0 0x7fd17414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 --2- 172.21.15.157:0/2407443299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd17406b8b0 0x7fd1740b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 >> 172.21.15.157:0/2407443299 conn(0x7fd1740b23e0 msgr2=0x7fd1740b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 shutdown_connections 2023-11-06T03:08:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.200+0000 7fd17bf71640 1 -- 172.21.15.157:0/2407443299 wait complete. 2023-11-06T03:08:36.217 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2023-11-06T03:08:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.332+0000 7f7fc1caf640 1 Processor -- start 2023-11-06T03:08:36.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.332+0000 7f7fc1caf640 1 -- start start 2023-11-06T03:08:36.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fc1caf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fc1caf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fbc0722f0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fc1caf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fbc06bda0 con 0x7f7fbc0aff30 2023-11-06T03:08:36.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fc1caf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b0310 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbaffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7fbc0ce010 0x7f7fbc0b2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44700/0 (socket says 172.21.15.157:44700) 2023-11-06T03:08:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34342/0 (socket says 172.21.15.157:34342) 2023-11-06T03:08:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbaffd640 1 -- 172.21.15.157:0/3839483018 learned_addr learned my addr 172.21.15.157:0/3839483018 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3839483018 >> v1:172.21.15.195:6789/0 conn(0x7f7fbc0aff30 legacy=0x7f7fbc06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:36.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3839483018 >> v1:172.21.15.157:6789/0 conn(0x7f7fbc0ce010 legacy=0x7f7fbc0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3839483018 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc06b7e0 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fba7fc640 1 -- 172.21.15.157:0/3839483018 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357043846 0 0) 0x7f7fa4001120 con 0x7f7fbc0ce010 2023-11-06T03:08:36.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fbbfff640 1 --2- 172.21.15.157:0/3839483018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009d90 tx=0x7f7fb401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cbeb0a643baefe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fba7fc640 1 -- 172.21.15.157:0/3839483018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40a5820 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fba7fc640 1 -- 172.21.15.157:0/3839483018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb40a5980 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.336+0000 7f7fba7fc640 1 -- 172.21.15.157:0/3839483018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40a5d10 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3839483018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0b2ba0 msgr2=0x7f7fbc071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3839483018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009d90 tx=0x7f7fb401c850 comp rx=0 tx=0).stop 2023-11-06T03:08:36.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3839483018 shutdown_connections 2023-11-06T03:08:36.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3839483018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc071db0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3839483018 >> 172.21.15.157:0/3839483018 conn(0x7f7fbc0cd490 msgr2=0x7f7fbc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:36.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3839483018 shutdown_connections 2023-11-06T03:08:36.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3839483018 wait complete. 2023-11-06T03:08:36.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 Processor -- start 2023-11-06T03:08:36.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- start start 2023-11-06T03:08:36.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fbc0aff30 0x7f7fbc0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0af1b0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc156e30 con 0x7f7fbc0b2ba0 2023-11-06T03:08:36.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc157110 con 0x7f7fbc0aff30 2023-11-06T03:08:36.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34348/0 (socket says 172.21.15.157:34348) 2023-11-06T03:08:36.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3460525157 learned_addr learned my addr 172.21.15.157:0/3460525157 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:36.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbb7fe640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fbc0aff30 0x7f7fbc0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbaffd640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fbc0aff30 msgr2=0x7f7fbc0aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fbc0aff30 0x7f7fbc0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fbc0b2ba0 msgr2=0x7f7fbc069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc069fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 -- 172.21.15.157:0/3460525157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009a70 con 0x7f7fbc0ce010 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fbbfff640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7fb401a480 tx=0x7f7fb40195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40ad850 con 0x7f7fbc0ce010 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb40ad9b0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc1573f0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40add40 con 0x7f7fbc0ce010 2023-11-06T03:08:36.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.340+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc1579b0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc072750 con 0x7f7fbc0ce010 2023-11-06T03:08:36.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7fb40b4050 con 0x7f7fbc0ce010 2023-11-06T03:08:36.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fb3fff640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 0x7f7f9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7fb411e3a0 con 0x7f7fbc0ce010 2023-11-06T03:08:36.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fbb7fe640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 0x7f7f9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.344+0000 7f7fbb7fe640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 0x7f7f9c067f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7fac003d60 tx=0x7f7fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.348+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7fb40ebf40 con 0x7f7fbc0ce010 2023-11-06T03:08:36.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.560+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f7fbc0b6990 con 0x7f7fbc0ce010 2023-11-06T03:08:36.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.816+0000 7f7fb3fff640 1 -- 172.21.15.157:0/3460525157 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v229) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7fb40ef560 con 0x7f7fbc0ce010 2023-11-06T03:08:36.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 msgr2=0x7f7f9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 0x7f7f9c067f20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7fac003d60 tx=0x7f7fac009040 comp rx=0 tx=0).stop 2023-11-06T03:08:36.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 msgr2=0x7f7fbc0c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7fb401a480 tx=0x7f7fb40195f0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 shutdown_connections 2023-11-06T03:08:36.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f9c065a60 0x7f7f9c067f20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fbc0ce010 0x7f7fbc0c61f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fbc0b2ba0 0x7f7fbc069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 --2- 172.21.15.157:0/3460525157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fbc0aff30 0x7f7fbc0aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 >> 172.21.15.157:0/3460525157 conn(0x7f7fbc0cd490 msgr2=0x7f7fbc0b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 shutdown_connections 2023-11-06T03:08:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.820+0000 7f7fc1caf640 1 -- 172.21.15.157:0/3460525157 wait complete. 2023-11-06T03:08:36.842 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2023-11-06T03:08:36.842 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-06T03:08:36.843 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2023-11-06T03:08:36.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 Processor -- start 2023-11-06T03:08:36.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 -- start start 2023-11-06T03:08:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc1f00b1910 0x7fc1f0161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f006f470 con 0x7fc1f0160db0 2023-11-06T03:08:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00b1df0 con 0x7fc1f00701a0 2023-11-06T03:08:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1f4b24640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f006f6c0 con 0x7fc1f00b1910 2023-11-06T03:08:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1ee59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc1f0160db0 0x7fc1f006d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44712/0 (socket says 172.21.15.157:44712) 2023-11-06T03:08:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1ee59c640 1 -- 172.21.15.157:0/1296711360 learned_addr learned my addr 172.21.15.157:0/1296711360 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/1296711360 >> v2:172.21.15.157:3300/0 conn(0x7fc1f00b1910 0x7fc1f0161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1ed59a640 1 -- 172.21.15.157:0/1296711360 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577212948 0 0) 0x7fc1f006f470 con 0x7fc1f0160db0 2023-11-06T03:08:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1ed59a640 1 -- 172.21.15.157:0/1296711360 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1dc003530 con 0x7fc1f0160db0 2023-11-06T03:08:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1edd9b640 1 -- 172.21.15.157:0/1296711360 >> v1:172.21.15.195:6789/0 conn(0x7fc1f00701a0 legacy=0x7fc1f00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1edd9b640 1 -- 172.21.15.157:0/1296711360 >> v1:172.21.15.157:6789/0 conn(0x7fc1f0160db0 legacy=0x7fc1f006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.968+0000 7fc1edd9b640 1 -- 172.21.15.157:0/1296711360 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f006f9a0 con 0x7fc1f00b1910 2023-11-06T03:08:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/1296711360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00b1910 0x7fc1f0161190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009df0 tx=0x7fc1d801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407bdbc2054314d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ed59a640 1 -- 172.21.15.157:0/1296711360 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80a5940 con 0x7fc1f00b1910 2023-11-06T03:08:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ed59a640 1 -- 172.21.15.157:0/1296711360 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d80a5aa0 con 0x7fc1f00b1910 2023-11-06T03:08:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ed59a640 1 -- 172.21.15.157:0/1296711360 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80a5e10 con 0x7fc1f00b1910 2023-11-06T03:08:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- 172.21.15.157:0/1296711360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00b1910 msgr2=0x7fc1f0161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/1296711360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00b1910 0x7fc1f0161190 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009df0 tx=0x7fc1d801c860 comp rx=0 tx=0).stop 2023-11-06T03:08:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- 172.21.15.157:0/1296711360 shutdown_connections 2023-11-06T03:08:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/1296711360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00b1910 0x7fc1f0161190 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- 172.21.15.157:0/1296711360 >> 172.21.15.157:0/1296711360 conn(0x7fc1f0065930 msgr2=0x7fc1f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- 172.21.15.157:0/1296711360 shutdown_connections 2023-11-06T03:08:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- 172.21.15.157:0/1296711360 wait complete. 2023-11-06T03:08:36.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 Processor -- start 2023-11-06T03:08:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- start start 2023-11-06T03:08:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 0x7fc1f014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1f0160db0 0x7fc1f0151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0161e40 con 0x7fc1f00701a0 2023-11-06T03:08:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0161230 con 0x7fc1f00b1910 2023-11-06T03:08:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1f4b24640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0161510 con 0x7fc1f0160db0 2023-11-06T03:08:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ee59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ee59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34370/0 (socket says 172.21.15.157:34370) 2023-11-06T03:08:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ee59c640 1 -- 172.21.15.157:0/2552784929 learned_addr learned my addr 172.21.15.157:0/2552784929 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 0x7fc1f014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ee59c640 1 -- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1f0160db0 msgr2=0x7fc1f0151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.972+0000 7fc1ee59c640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1f0160db0 0x7fc1f0151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1ee59c640 1 -- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 msgr2=0x7fc1f014c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1ee59c640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 0x7fc1f014c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1ee59c640 1 -- 172.21.15.157:0/2552784929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009a70 con 0x7fc1f00701a0 2023-11-06T03:08:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 0x7fc1f014c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1ee59c640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c3c0 tx=0x7fc1e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8005cf0 con 0x7fc1f00701a0 2023-11-06T03:08:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8005e50 con 0x7fc1f00701a0 2023-11-06T03:08:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f01617f0 con 0x7fc1f00701a0 2023-11-06T03:08:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80072f0 con 0x7fc1f00701a0 2023-11-06T03:08:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f014b980 con 0x7fc1f00701a0 2023-11-06T03:08:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0072750 con 0x7fc1f00701a0 2023-11-06T03:08:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc1e8007450 con 0x7fc1f00701a0 2023-11-06T03:08:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 0x7fc1d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc1e80825e0 con 0x7fc1f00701a0 2023-11-06T03:08:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 0x7fc1d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.976+0000 7fc1edd9b640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 0x7fc1d0067f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80af000 tx=0x7fc1d80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:36.980+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc1e8050140 con 0x7fc1f00701a0 2023-11-06T03:08:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.192+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fc1f0164b20 con 0x7fc1f00701a0 2023-11-06T03:08:37.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.824+0000 7fc1e6ffd640 1 -- 172.21.15.157:0/2552784929 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v230) v1 ==== 146+0+0 (secure 0 0 0) 0x7fc1e8017020 con 0x7fc1f00701a0 2023-11-06T03:08:37.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 msgr2=0x7fc1d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 0x7fc1d0067f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80af000 tx=0x7fc1d80a7070 comp rx=0 tx=0).stop 2023-11-06T03:08:37.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 msgr2=0x7fc1f014af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c3c0 tx=0x7fc1e800c880 comp rx=0 tx=0).stop 2023-11-06T03:08:37.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 shutdown_connections 2023-11-06T03:08:37.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc1d0065a60 0x7fc1d0067f20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1f0160db0 0x7fc1f0151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1f00b1910 0x7fc1f014c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 --2- 172.21.15.157:0/2552784929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f00701a0 0x7fc1f014af10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 >> 172.21.15.157:0/2552784929 conn(0x7fc1f0065930 msgr2=0x7fc1f0165430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:37.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 shutdown_connections 2023-11-06T03:08:37.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.828+0000 7fc1f4b24640 1 -- 172.21.15.157:0/2552784929 wait complete. 2023-11-06T03:08:37.848 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:08:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.976+0000 7f0ef14ca640 1 Processor -- start 2023-11-06T03:08:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.976+0000 7f0ef14ca640 1 -- start start 2023-11-06T03:08:37.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ef14ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0eec167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ef14ca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec0d2920 con 0x7f0eec15ce80 2023-11-06T03:08:37.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ef14ca640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec0d18a0 con 0x7f0eec15caa0 2023-11-06T03:08:37.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ef14ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0d1b60 con 0x7f0eec0d0280 2023-11-06T03:08:37.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eea7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0eec15ce80 0x7f0eec0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44724/0 (socket says 172.21.15.157:44724) 2023-11-06T03:08:37.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eea7fc640 1 -- 172.21.15.157:0/1059395785 learned_addr learned my addr 172.21.15.157:0/1059395785 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:37.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeb7fe640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0eec167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431990661 0 0) 0x7f0eec0d2920 con 0x7f0eec15ce80 2023-11-06T03:08:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed4003530 con 0x7f0eec15ce80 2023-11-06T03:08:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859599520 0 0) 0x7f0ed4003530 con 0x7f0eec15ce80 2023-11-06T03:08:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 msgr2=0x7f0eec167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0eec167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 >> v1:172.21.15.195:6789/0 conn(0x7f0eec15caa0 legacy=0x7f0eec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0d1e40 con 0x7f0eec15ce80 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0ee0001680 con 0x7f0eec15ce80 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 >> v1:172.21.15.157:6789/0 conn(0x7f0eec15ce80 legacy=0x7f0eec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40039b0 0x7f0ed4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40047d0 0x7f0ed4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0ed4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0d1b60 con 0x7f0ed40039b0 2023-11-06T03:08:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed400ba70 con 0x7f0ed40047d0 2023-11-06T03:08:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed400bc30 con 0x7f0eec0d0280 2023-11-06T03:08:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40039b0 0x7f0ed4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eea7fc640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40047d0 0x7f0ed4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeb7fe640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0ed4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 -- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 msgr2=0x7f0ed4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0eec0d0280 0x7f0ed4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 -- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40047d0 msgr2=0x7f0ed4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:37.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed40047d0 0x7f0ed4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.980+0000 7f0eeaffd640 1 -- 172.21.15.157:0/1059395785 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4003530 con 0x7f0ed40039b0 2023-11-06T03:08:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40039b0 0x7f0ed4003dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0edc001780 tx=0x7f0edc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f468865223c920 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:37.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0052d0 con 0x7f0ed40039b0 2023-11-06T03:08:37.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc0a7050 con 0x7f0ed40039b0 2023-11-06T03:08:37.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ee9ffb640 1 -- 172.21.15.157:0/1059395785 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc003600 con 0x7f0ed40039b0 2023-11-06T03:08:37.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1059395785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40039b0 msgr2=0x7f0ed4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1059395785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed40039b0 0x7f0ed4003dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0edc001780 tx=0x7f0edc01c3d0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0eeb7fe640 1 -- 172.21.15.157:0/1059395785 reap_dead start 2023-11-06T03:08:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1059395785 shutdown_connections 2023-11-06T03:08:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1059395785 >> 172.21.15.157:0/1059395785 conn(0x7f0eec070790 msgr2=0x7f0eec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1059395785 shutdown_connections 2023-11-06T03:08:37.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1059395785 wait complete. 2023-11-06T03:08:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 Processor -- start 2023-11-06T03:08:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.984+0000 7f0ef14ca640 1 -- start start 2023-11-06T03:08:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 0x7f0eec0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eec152580 0x7f0eec15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec152a00 con 0x7f0eec0cc600 2023-11-06T03:08:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0c6f40 con 0x7f0eec152580 2023-11-06T03:08:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0c7220 con 0x7f0ed40039b0 2023-11-06T03:08:37.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34392/0 (socket says 172.21.15.157:34392) 2023-11-06T03:08:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 -- 172.21.15.157:0/1982885936 learned_addr learned my addr 172.21.15.157:0/1982885936 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 0x7f0eec0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 -- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 msgr2=0x7f0eec0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 0x7f0eec0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 -- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eec152580 msgr2=0x7f0eec15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eec152580 0x7f0eec15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 -- 172.21.15.157:0/1982885936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0c7500 con 0x7f0eec0cc600 2023-11-06T03:08:37.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 0x7f0eec0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eea7fc640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0007290 tx=0x7f0ee0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000f940 con 0x7f0eec0cc600 2023-11-06T03:08:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee000faa0 con 0x7f0eec0cc600 2023-11-06T03:08:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000fe50 con 0x7f0eec0cc600 2023-11-06T03:08:37.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec0c77e0 con 0x7f0eec0cc600 2023-11-06T03:08:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec0c6c10 con 0x7f0eec0cc600 2023-11-06T03:08:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec072750 con 0x7f0eec0cc600 2023-11-06T03:08:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0ee001d460 con 0x7f0eec0cc600 2023-11-06T03:08:37.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 0x7f0ed8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0ee0083140 con 0x7f0eec0cc600 2023-11-06T03:08:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.988+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 0x7f0ed8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:37.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.992+0000 7f0eeaffd640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 0x7f0ed8067f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0edc001750 tx=0x7f0edc0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:37.992+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0ee0050ca0 con 0x7f0eec0cc600 2023-11-06T03:08:38.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.148+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0eec0c7c90 con 0x7f0eec0cc600 2023-11-06T03:08:38.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.156+0000 7f0ed37fe640 1 -- 172.21.15.157:0/1982885936 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ee00542c0 con 0x7f0eec0cc600 2023-11-06T03:08:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 msgr2=0x7f0ed8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 0x7f0ed8067f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0edc001750 tx=0x7f0edc0011a0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 msgr2=0x7f0eec0c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0007290 tx=0x7f0ee0007750 comp rx=0 tx=0).stop 2023-11-06T03:08:38.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 shutdown_connections 2023-11-06T03:08:38.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ed8065a60 0x7f0ed8067f20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0eec152580 0x7f0eec15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eec0cc600 0x7f0eec0c7fe0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 --2- 172.21.15.157:0/1982885936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed40039b0 0x7f0eec0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 >> 172.21.15.157:0/1982885936 conn(0x7f0eec070790 msgr2=0x7f0eec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 shutdown_connections 2023-11-06T03:08:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.160+0000 7f0ef14ca640 1 -- 172.21.15.157:0/1982885936 wait complete. 2023-11-06T03:08:38.178 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:08:38.178 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:08:38.232 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:08:38.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:08:38.276 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:38.277 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:38.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:08:38.285 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:08:38.285 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:38.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:08:38.324 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:38.324 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:38.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:08:38.331 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:08:38.331 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:38.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:08:38.372 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:08:38.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:38.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 Processor -- start 2023-11-06T03:08:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 -- start start 2023-11-06T03:08:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd74415e5e0 0x7fd7440b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd74415ea90 0x7fd7440b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74415f580 con 0x7fd7441551e0 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7440704b0 con 0x7fd74415ea90 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd748d61640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74415efc0 con 0x7fd74415e5e0 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd742d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd74415ea90 0x7fd7440b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd74259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd7441551e0 0x7fd74406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44738/0 (socket says 172.21.15.157:44738) 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd74259c640 1 -- 172.21.15.157:0/1711034180 learned_addr learned my addr 172.21.15.157:0/1711034180 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd742d9d640 1 -- 172.21.15.157:0/1711034180 >> v2:172.21.15.195:3300/0 conn(0x7fd74415e5e0 msgr2=0x7fd7440b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:38.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd742d9d640 1 --2- 172.21.15.157:0/1711034180 >> v2:172.21.15.195:3300/0 conn(0x7fd74415e5e0 0x7fd7440b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd742d9d640 1 -- 172.21.15.157:0/1711034180 >> v1:172.21.15.157:6789/0 conn(0x7fd7441551e0 legacy=0x7fd74406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd742d9d640 1 -- 172.21.15.157:0/1711034180 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74415f2a0 con 0x7fd74415ea90 2023-11-06T03:08:38.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.504+0000 7fd74159a640 1 -- 172.21.15.157:0/1711034180 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655233561 0 0) 0x7fd74415f580 con 0x7fd7441551e0 2023-11-06T03:08:38.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd742d9d640 1 --2- 172.21.15.157:0/1711034180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415ea90 0x7fd7440b3950 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd744066880 tx=0x7fd73c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd04fb92d04c065 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd74159a640 1 -- 172.21.15.157:0/1711034180 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0a5910 con 0x7fd74415ea90 2023-11-06T03:08:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd74159a640 1 -- 172.21.15.157:0/1711034180 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c0a5a70 con 0x7fd74415ea90 2023-11-06T03:08:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd74159a640 1 -- 172.21.15.157:0/1711034180 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0a5e00 con 0x7fd74415ea90 2023-11-06T03:08:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- 172.21.15.157:0/1711034180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415ea90 msgr2=0x7fd7440b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- 172.21.15.157:0/1711034180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415ea90 0x7fd7440b3950 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd744066880 tx=0x7fd73c01c850 comp rx=0 tx=0).stop 2023-11-06T03:08:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- 172.21.15.157:0/1711034180 shutdown_connections 2023-11-06T03:08:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- 172.21.15.157:0/1711034180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415ea90 0x7fd7440b3950 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- 172.21.15.157:0/1711034180 >> v2:172.21.15.195:3300/0 conn(0x7fd74415e5e0 0x7fd7440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- 172.21.15.157:0/1711034180 >> 172.21.15.157:0/1711034180 conn(0x7fd744065930 msgr2=0x7fd744065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- 172.21.15.157:0/1711034180 shutdown_connections 2023-11-06T03:08:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- 172.21.15.157:0/1711034180 wait complete. 2023-11-06T03:08:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 Processor -- start 2023-11-06T03:08:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- start start 2023-11-06T03:08:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 0x7fd7440ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 0x7fd7441580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd744152ee0 con 0x7fd74415e5e0 2023-11-06T03:08:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441531c0 con 0x7fd74415ea90 2023-11-06T03:08:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd748d61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441534a0 con 0x7fd7441551e0 2023-11-06T03:08:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34406/0 (socket says 172.21.15.157:34406) 2023-11-06T03:08:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 -- 172.21.15.157:0/712927087 learned_addr learned my addr 172.21.15.157:0/712927087 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd74259c640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 0x7fd7440ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd742d9d640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 0x7fd7441580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 -- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 msgr2=0x7fd7440ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 0x7fd7440ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 -- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 msgr2=0x7fd7441580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 0x7fd7441580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 -- 172.21.15.157:0/712927087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c009a70 con 0x7fd74415e5e0 2023-11-06T03:08:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd742d9d640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 0x7fd7441580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd74259c640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 0x7fd7440ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.508+0000 7fd741d9b640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd73800b550 tx=0x7fd73800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738010870 con 0x7fd74415e5e0 2023-11-06T03:08:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7380109f0 con 0x7fd74415e5e0 2023-11-06T03:08:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd744153780 con 0x7fd74415e5e0 2023-11-06T03:08:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738010da0 con 0x7fd74415e5e0 2023-11-06T03:08:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7441557f0 con 0x7fd74415e5e0 2023-11-06T03:08:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd744072750 con 0x7fd74415e5e0 2023-11-06T03:08:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd73800f3c0 con 0x7fd74415e5e0 2023-11-06T03:08:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 0x7fd724067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd738082fd0 con 0x7fd74415e5e0 2023-11-06T03:08:38.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd74259c640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 0x7fd724067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.512+0000 7fd74259c640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 0x7fd724067f20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd734000f70 tx=0x7fd734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.516+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd738050b30 con 0x7fd74415e5e0 2023-11-06T03:08:38.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd744153c30 con 0x7fd74415e5e0 2023-11-06T03:08:38.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd732ffd640 1 -- 172.21.15.157:0/712927087 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fd738014df0 con 0x7fd74415e5e0 2023-11-06T03:08:38.704 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:38.704 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1872532098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1872532098},{"type":"v1","addr":"172.21.15.195:6835","nonce":1872532098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1088388212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1088388212},{"type":"v1","addr":"172.21.15.195:6837","nonce":1088388212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3982455406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3982455406},{"type":"v1","addr":"172.21.15.157:6833","nonce":3982455406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":6669,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1262490954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1262490954},{"type":"v1","addr":"172.21.15.157:6835","nonce":1262490954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2023-11-06T03:08:38.705 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 43 2023-11-06T03:08:38.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 msgr2=0x7fd724067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 0x7fd724067f20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd734000f70 tx=0x7fd734009040 comp rx=0 tx=0).stop 2023-11-06T03:08:38.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 msgr2=0x7fd74414ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd73800b550 tx=0x7fd73800ba10 comp rx=0 tx=0).stop 2023-11-06T03:08:38.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 shutdown_connections 2023-11-06T03:08:38.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd724065a60 0x7fd724067f20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd74415ea90 0x7fd7441580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd74415e5e0 0x7fd74414ce40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 --2- 172.21.15.157:0/712927087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7441551e0 0x7fd7440ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.704+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 >> 172.21.15.157:0/712927087 conn(0x7fd744065930 msgr2=0x7fd74415de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.708+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 shutdown_connections 2023-11-06T03:08:38.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.708+0000 7fd748d61640 1 -- 172.21.15.157:0/712927087 wait complete. 2023-11-06T03:08:38.724 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:08:38.724 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:08:38.724 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:08:38.725 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:08:38.725 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:08:38.725 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:08:38.726 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:08:38.726 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:08:38.726 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:08:38.726 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:08:38.727 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:08:38.727 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:08:38.727 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:08:38.727 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:08:38.728 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:08:38.728 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:08:38.735 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:08:38.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:08:38.737 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:08:38.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:08:38.745 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:08:38.746 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:08:38.746 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:08:38.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:08:38.779 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:08:38.780 DEBUG:teuthology.parallel:result is None 2023-11-06T03:08:38.781 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:08:38.781 DEBUG:teuthology.parallel:result is None 2023-11-06T03:08:38.792 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:08:38.792 DEBUG:teuthology.parallel:result is None 2023-11-06T03:08:38.793 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:08:38.794 DEBUG:teuthology.parallel:result is None 2023-11-06T03:08:38.797 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:08:38.843 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:08:38.851 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:08:38.862 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:08:38.867 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:08:38.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 Processor -- start 2023-11-06T03:08:38.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 -- start start 2023-11-06T03:08:38.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b8406d7f0 0x7f1b8406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1b840cc480 0x7f1b840cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8406e190 con 0x7f1b84159950 2023-11-06T03:08:38.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840702b0 con 0x7f1b8406d7f0 2023-11-06T03:08:38.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8bc39640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b84159d30 con 0x7f1b840cc480 2023-11-06T03:08:38.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b891d4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b8406d7f0 0x7f1b8406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b899d5640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1b84159950 0x7f1b840b39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53144/0 (socket says 172.21.15.157:53144) 2023-11-06T03:08:38.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b899d5640 1 -- 172.21.15.157:0/4082950636 learned_addr learned my addr 172.21.15.157:0/4082950636 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:38.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b8a1d6640 1 --2- 172.21.15.157:0/4082950636 >> v2:172.21.15.195:3300/0 conn(0x7f1b840cc480 0x7f1b840cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b891d4640 1 -- 172.21.15.157:0/4082950636 >> v2:172.21.15.195:3300/0 conn(0x7f1b840cc480 msgr2=0x7f1b840cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:38.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b891d4640 1 --2- 172.21.15.157:0/4082950636 >> v2:172.21.15.195:3300/0 conn(0x7f1b840cc480 0x7f1b840cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b891d4640 1 -- 172.21.15.157:0/4082950636 >> v1:172.21.15.157:6789/0 conn(0x7f1b84159950 legacy=0x7f1b840b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b891d4640 1 -- 172.21.15.157:0/4082950636 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840ccdd0 con 0x7f1b8406d7f0 2023-11-06T03:08:38.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.920+0000 7f1b889d3640 1 -- 172.21.15.157:0/4082950636 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822616608 0 0) 0x7f1b8406e190 con 0x7f1b84159950 2023-11-06T03:08:38.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b891d4640 1 --2- 172.21.15.157:0/4082950636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b8406d7f0 0x7f1b8406dc50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1b840b40d0 tx=0x7f1b7401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a823419a24c57e2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b889d3640 1 -- 172.21.15.157:0/4082950636 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740ab660 con 0x7f1b8406d7f0 2023-11-06T03:08:38.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b889d3640 1 -- 172.21.15.157:0/4082950636 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b740ab7c0 con 0x7f1b8406d7f0 2023-11-06T03:08:38.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b889d3640 1 -- 172.21.15.157:0/4082950636 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740abb50 con 0x7f1b8406d7f0 2023-11-06T03:08:38.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- 172.21.15.157:0/4082950636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b8406d7f0 msgr2=0x7f1b8406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/4082950636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b8406d7f0 0x7f1b8406dc50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1b840b40d0 tx=0x7f1b7401c460 comp rx=0 tx=0).stop 2023-11-06T03:08:38.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- 172.21.15.157:0/4082950636 shutdown_connections 2023-11-06T03:08:38.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/4082950636 >> v2:172.21.15.195:3300/0 conn(0x7f1b840cc480 0x7f1b840cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/4082950636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b8406d7f0 0x7f1b8406dc50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- 172.21.15.157:0/4082950636 >> 172.21.15.157:0/4082950636 conn(0x7f1b8406a170 msgr2=0x7f1b8406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:38.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- 172.21.15.157:0/4082950636 shutdown_connections 2023-11-06T03:08:38.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- 172.21.15.157:0/4082950636 wait complete. 2023-11-06T03:08:38.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 Processor -- start 2023-11-06T03:08:38.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.924+0000 7f1b8bc39640 1 -- start start 2023-11-06T03:08:38.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b8406d7f0 0x7f1b84158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b840cc480 0x7f1b8415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8415a2d0 con 0x7f1b84159950 2023-11-06T03:08:38.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8415a5b0 con 0x7f1b840cc480 2023-11-06T03:08:38.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8415a890 con 0x7f1b8406d7f0 2023-11-06T03:08:38.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57646/0 (socket says 172.21.15.157:57646) 2023-11-06T03:08:38.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 -- 172.21.15.157:0/3193557301 learned_addr learned my addr 172.21.15.157:0/3193557301 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:38.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b891d4640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b840cc480 0x7f1b8415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b899d5640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b8406d7f0 0x7f1b84158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 -- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b8406d7f0 msgr2=0x7f1b84158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b8406d7f0 0x7f1b84158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 -- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b840cc480 msgr2=0x7f1b8415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:38.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b840cc480 0x7f1b8415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:38.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 -- 172.21.15.157:0/3193557301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b74009420 con 0x7f1b84159950 2023-11-06T03:08:38.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8a1d6640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000efc0 tx=0x7f1b8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80014480 con 0x7f1b84159950 2023-11-06T03:08:38.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000f040 con 0x7f1b84159950 2023-11-06T03:08:38.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80014820 con 0x7f1b84159950 2023-11-06T03:08:38.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8415ab70 con 0x7f1b84159950 2023-11-06T03:08:38.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84159400 con 0x7f1b84159950 2023-11-06T03:08:38.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84072750 con 0x7f1b84159950 2023-11-06T03:08:38.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b80014980 con 0x7f1b84159950 2023-11-06T03:08:38.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 0x7f1b68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:38.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1b800839f0 con 0x7f1b84159950 2023-11-06T03:08:38.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.928+0000 7f1b899d5640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 0x7f1b68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:38.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.932+0000 7f1b899d5640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 0x7f1b68067f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000f70 tx=0x7f1b7c007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:38.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:38.936+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b80051600 con 0x7f1b84159950 2023-11-06T03:08:39.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.096+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1b841667d0 con 0x7f1b84159950 2023-11-06T03:08:39.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.912+0000 7f1b7a7fc640 1 -- 172.21.15.157:0/3193557301 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1b80054c20 con 0x7f1b84159950 2023-11-06T03:08:39.911 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:08:39.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 msgr2=0x7f1b68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:39.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 0x7f1b68067f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c000f70 tx=0x7f1b7c007df0 comp rx=0 tx=0).stop 2023-11-06T03:08:39.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 msgr2=0x7f1b840ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:39.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000efc0 tx=0x7f1b8000c490 comp rx=0 tx=0).stop 2023-11-06T03:08:39.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 shutdown_connections 2023-11-06T03:08:39.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b68065a60 0x7f1b68067f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:39.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b84159950 0x7f1b840ca7a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:39.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b840cc480 0x7f1b8415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:39.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 --2- 172.21.15.157:0/3193557301 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b8406d7f0 0x7f1b84158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:39.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 >> 172.21.15.157:0/3193557301 conn(0x7f1b8406a170 msgr2=0x7f1b8406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:39.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 shutdown_connections 2023-11-06T03:08:39.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:39.916+0000 7f1b8bc39640 1 -- 172.21.15.157:0/3193557301 wait complete. 2023-11-06T03:08:39.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:08:40.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.060+0000 7fb8a7428640 1 Processor -- start 2023-11-06T03:08:40.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.060+0000 7fb8a7428640 1 -- start start 2023-11-06T03:08:40.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a7428640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb8a01599c0 0x7fb8a00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a7428640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8a00b2af0 0x7fb8a0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a7428640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a015e440 con 0x7fb8a006d000 2023-11-06T03:08:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a7428640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a006c240 con 0x7fb8a00b2af0 2023-11-06T03:08:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a7428640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a006c520 con 0x7fb8a01599c0 2023-11-06T03:08:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8a00b2af0 0x7fb8a0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8a00b2af0 0x7fb8a0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57662/0 (socket says 172.21.15.157:57662) 2023-11-06T03:08:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 -- 172.21.15.157:0/1961380128 learned_addr learned my addr 172.21.15.157:0/1961380128 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a51c4640 1 --2- 172.21.15.157:0/1961380128 >> v2:172.21.15.195:3300/0 conn(0x7fb8a01599c0 0x7fb8a00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 -- 172.21.15.157:0/1961380128 >> v2:172.21.15.195:3300/0 conn(0x7fb8a01599c0 msgr2=0x7fb8a00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 --2- 172.21.15.157:0/1961380128 >> v2:172.21.15.195:3300/0 conn(0x7fb8a01599c0 0x7fb8a00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 -- 172.21.15.157:0/1961380128 >> v1:172.21.15.195:6789/0 conn(0x7fb8a006d000 legacy=0x7fb8a00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 -- 172.21.15.157:0/1961380128 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a006c800 con 0x7fb8a00b2af0 2023-11-06T03:08:40.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb8a59c5640 1 --2- 172.21.15.157:0/1961380128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a00b2af0 0x7fb8a0159da0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0066800 tx=0x7fb89801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6294861fff92d29d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb897fff640 1 -- 172.21.15.157:0/1961380128 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89800b2d0 con 0x7fb8a00b2af0 2023-11-06T03:08:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb897fff640 1 -- 172.21.15.157:0/1961380128 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89800b430 con 0x7fb8a00b2af0 2023-11-06T03:08:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.064+0000 7fb897fff640 1 -- 172.21.15.157:0/1961380128 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980a7c70 con 0x7fb8a00b2af0 2023-11-06T03:08:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/1961380128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a00b2af0 msgr2=0x7fb8a0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- 172.21.15.157:0/1961380128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a00b2af0 0x7fb8a0159da0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0066800 tx=0x7fb89801eb60 comp rx=0 tx=0).stop 2023-11-06T03:08:40.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/1961380128 shutdown_connections 2023-11-06T03:08:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- 172.21.15.157:0/1961380128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a00b2af0 0x7fb8a0159da0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- 172.21.15.157:0/1961380128 >> v2:172.21.15.195:3300/0 conn(0x7fb8a01599c0 0x7fb8a00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/1961380128 >> 172.21.15.157:0/1961380128 conn(0x7fb8a0065930 msgr2=0x7fb8a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/1961380128 shutdown_connections 2023-11-06T03:08:40.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/1961380128 wait complete. 2023-11-06T03:08:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 Processor -- start 2023-11-06T03:08:40.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- start start 2023-11-06T03:08:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 0x7fb8a014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8a00b2af0 0x7fb8a00af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a014da70 con 0x7fb8a01599c0 2023-11-06T03:08:40.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a01fcff0 con 0x7fb8a006d000 2023-11-06T03:08:40.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00cc970 con 0x7fb8a00b2af0 2023-11-06T03:08:40.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57672/0 (socket says 172.21.15.157:57672) 2023-11-06T03:08:40.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 -- 172.21.15.157:0/590854920 learned_addr learned my addr 172.21.15.157:0/590854920 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a51c4640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 0x7fb8a014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 -- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8a00b2af0 msgr2=0x7fb8a00af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a49c3640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8a00b2af0 0x7fb8a00af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8a00b2af0 0x7fb8a00af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 -- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 msgr2=0x7fb8a014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 0x7fb8a014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 -- 172.21.15.157:0/590854920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89800bd40 con 0x7fb8a01599c0 2023-11-06T03:08:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a51c4640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 0x7fb8a014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a59c5640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb89801c790 tx=0x7fb89801b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980bd070 con 0x7fb8a01599c0 2023-11-06T03:08:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898009600 con 0x7fb8a01599c0 2023-11-06T03:08:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980099b0 con 0x7fb8a01599c0 2023-11-06T03:08:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a00ccc50 con 0x7fb8a01599c0 2023-11-06T03:08:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.068+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a00cd210 con 0x7fb8a01599c0 2023-11-06T03:08:40.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a0072750 con 0x7fb8a01599c0 2023-11-06T03:08:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb898009b10 con 0x7fb8a01599c0 2023-11-06T03:08:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb895ffb640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 0x7fb884067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb89811e110 con 0x7fb8a01599c0 2023-11-06T03:08:40.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb8a51c4640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 0x7fb884067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.072+0000 7fb8a51c4640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 0x7fb884067f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00b0570 tx=0x7fb89c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.076+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb8980ebe10 con 0x7fb8a01599c0 2023-11-06T03:08:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb8a00cb110 con 0x7fb8a01599c0 2023-11-06T03:08:40.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb895ffb640 1 -- 172.21.15.157:0/590854920 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb8980ef430 con 0x7fb8a01599c0 2023-11-06T03:08:40.284 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:40.284 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:40.284 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:08:40.285 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.285 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:08:40.285 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:08:40.285 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.285 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.286 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:08:40.286 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:08:40.286 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.286 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.286 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.287 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.287 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:08:40.287 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:08:40.287 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.287 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.288 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:08:40.288 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:08:40.288 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.288 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.288 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.289 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:08:40.290 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:08:40.290 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.290 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.290 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.290 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.291 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:08:40.291 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:08:40.291 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.291 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.291 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:08:40.292 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:08:40.292 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.292 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.292 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.292 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.293 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:08:40.293 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:08:40.293 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.293 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.293 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:08:40.294 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:08:40.295 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.295 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.295 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.295 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.295 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.296 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.296 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.296 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:08:40.296 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:08:40.296 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.297 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.297 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.297 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.297 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.297 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.298 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.298 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:08:40.298 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:08:40.298 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.298 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.299 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.299 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.299 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.299 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.299 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.300 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:08:40.300 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:08:40.300 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.300 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.300 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:08:40.302 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:08:40.302 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.302 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.302 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.303 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:08:40.304 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:08:40.304 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.304 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.304 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.304 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.305 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.305 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.305 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.305 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:08:40.305 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:08:40.306 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.306 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:08:40.306 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:08:40.306 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.306 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.307 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.307 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.307 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:08:40.307 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:08:40.307 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.308 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.308 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:08:40.308 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:08:40.308 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.308 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.309 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.309 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:08:40.309 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:08:40.309 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.309 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.310 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:08:40.310 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:08:40.310 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:08:40.310 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.310 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.311 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.311 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:08:40.311 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:08:40.311 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.311 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.312 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:08:40.312 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:08:40.312 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:08:40.312 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.312 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.313 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.313 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:08:40.313 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:08:40.313 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.313 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:08:40.314 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:08:40.314 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:08:40.314 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:08:40.314 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.314 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:08:40.315 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:08:40.316 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:08:40.316 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.316 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.317 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.317 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.317 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:08:40.318 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:08:40.318 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.318 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:08:40.319 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:08:40.319 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.319 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.320 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:40.320 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:40.320 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:08:40.321 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:08:40.321 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:08:40.321 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:08:40.322 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:08:40.322 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:08:40.322 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.323 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:40.323 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:08:40.323 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:40.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 msgr2=0x7fb884067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 0x7fb884067f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00b0570 tx=0x7fb89c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:40.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 msgr2=0x7fb8a00b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb89801c790 tx=0x7fb89801b900 comp rx=0 tx=0).stop 2023-11-06T03:08:40.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 shutdown_connections 2023-11-06T03:08:40.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb884065a60 0x7fb884067f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a01599c0 0x7fb8a00b1e20 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8a00b2af0 0x7fb8a00af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 --2- 172.21.15.157:0/590854920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a006d000 0x7fb8a014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 >> 172.21.15.157:0/590854920 conn(0x7fb8a0065930 msgr2=0x7fb8a00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.284+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 shutdown_connections 2023-11-06T03:08:40.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.288+0000 7fb8a7428640 1 -- 172.21.15.157:0/590854920 wait complete. 2023-11-06T03:08:40.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:40.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 Processor -- start 2023-11-06T03:08:40.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 -- start start 2023-11-06T03:08:40.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faf5815ce30 0x7faf580b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf580cfc90 0x7faf580cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815d210 con 0x7faf580cd520 2023-11-06T03:08:40.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf580d17f0 con 0x7faf580cfc90 2023-11-06T03:08:40.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5e66e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf580ce650 con 0x7faf5815ce30 2023-11-06T03:08:40.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf580cfc90 0x7faf580cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf580cfc90 0x7faf580cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57678/0 (socket says 172.21.15.157:57678) 2023-11-06T03:08:40.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 -- 172.21.15.157:0/2659218451 learned_addr learned my addr 172.21.15.157:0/2659218451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:40.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf57fff640 1 --2- 172.21.15.157:0/2659218451 >> v2:172.21.15.195:3300/0 conn(0x7faf5815ce30 0x7faf580b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 -- 172.21.15.157:0/2659218451 >> v2:172.21.15.195:3300/0 conn(0x7faf5815ce30 msgr2=0x7faf580b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 --2- 172.21.15.157:0/2659218451 >> v2:172.21.15.195:3300/0 conn(0x7faf5815ce30 0x7faf580b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 -- 172.21.15.157:0/2659218451 >> v1:172.21.15.195:6789/0 conn(0x7faf580cd520 legacy=0x7faf580cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.452+0000 7faf5cc0b640 1 -- 172.21.15.157:0/2659218451 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580ce090 con 0x7faf580cfc90 2023-11-06T03:08:40.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5cc0b640 1 --2- 172.21.15.157:0/2659218451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf580cd9d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faf58066240 tx=0x7faf5001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd413d53b771d29 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf56ffd640 1 -- 172.21.15.157:0/2659218451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500a56b0 con 0x7faf580cfc90 2023-11-06T03:08:40.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf56ffd640 1 -- 172.21.15.157:0/2659218451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf500a5810 con 0x7faf580cfc90 2023-11-06T03:08:40.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf56ffd640 1 -- 172.21.15.157:0/2659218451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500a5ba0 con 0x7faf580cfc90 2023-11-06T03:08:40.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- 172.21.15.157:0/2659218451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 msgr2=0x7faf580cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- 172.21.15.157:0/2659218451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf580cd9d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faf58066240 tx=0x7faf5001c900 comp rx=0 tx=0).stop 2023-11-06T03:08:40.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- 172.21.15.157:0/2659218451 shutdown_connections 2023-11-06T03:08:40.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- 172.21.15.157:0/2659218451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf580cd9d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- 172.21.15.157:0/2659218451 >> v2:172.21.15.195:3300/0 conn(0x7faf5815ce30 0x7faf580b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- 172.21.15.157:0/2659218451 >> 172.21.15.157:0/2659218451 conn(0x7faf58070990 msgr2=0x7faf58065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- 172.21.15.157:0/2659218451 shutdown_connections 2023-11-06T03:08:40.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- 172.21.15.157:0/2659218451 wait complete. 2023-11-06T03:08:40.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 Processor -- start 2023-11-06T03:08:40.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- start start 2023-11-06T03:08:40.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf580cd520 0x7faf58155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf5815ce30 0x7faf58065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58157ab0 con 0x7faf580cfc90 2023-11-06T03:08:40.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58156330 con 0x7faf580cd520 2023-11-06T03:08:40.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5e66e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5813ffc0 con 0x7faf5815ce30 2023-11-06T03:08:40.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57684/0 (socket says 172.21.15.157:57684) 2023-11-06T03:08:40.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 -- 172.21.15.157:0/307463051 learned_addr learned my addr 172.21.15.157:0/307463051 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:40.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf57fff640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf580cd520 0x7faf58155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf5cc0b640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf5815ce30 0x7faf58065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 -- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf5815ce30 msgr2=0x7faf58065e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf5815ce30 0x7faf58065e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 -- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf580cd520 msgr2=0x7faf58155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf580cd520 0x7faf58155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 -- 172.21.15.157:0/307463051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50009b30 con 0x7faf580cfc90 2023-11-06T03:08:40.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf577fe640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c450 tx=0x7faf4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.456+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c012830 con 0x7faf580cfc90 2023-11-06T03:08:40.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c0129b0 con 0x7faf580cfc90 2023-11-06T03:08:40.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf581402a0 con 0x7faf580cfc90 2023-11-06T03:08:40.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c012d60 con 0x7faf580cfc90 2023-11-06T03:08:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58140860 con 0x7faf580cfc90 2023-11-06T03:08:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58072750 con 0x7faf580cfc90 2023-11-06T03:08:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faf4c01c460 con 0x7faf580cfc90 2023-11-06T03:08:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf54ff9640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 0x7faf44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faf4c082e90 con 0x7faf580cfc90 2023-11-06T03:08:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf57fff640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 0x7faf44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.460+0000 7faf57fff640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 0x7faf44067f20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faf4800b930 tx=0x7faf48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.464+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faf4c050a40 con 0x7faf580cfc90 2023-11-06T03:08:40.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:08:40.479 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:08:40.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.612+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf58140a80 con 0x7faf580cfc90 2023-11-06T03:08:40.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf54ff9640 1 -- 172.21.15.157:0/307463051 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+14219 (secure 0 0 0) 0x7faf4c054060 con 0x7faf580cfc90 2023-11-06T03:08:40.615 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:40.615 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":63,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:39.899742+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:40.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 msgr2=0x7faf44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 0x7faf44067f20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faf4800b930 tx=0x7faf48008040 comp rx=0 tx=0).stop 2023-11-06T03:08:40.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 msgr2=0x7faf58065930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c450 tx=0x7faf4c00c910 comp rx=0 tx=0).stop 2023-11-06T03:08:40.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 shutdown_connections 2023-11-06T03:08:40.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf44065a60 0x7faf44067f20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf5815ce30 0x7faf58065e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf580cfc90 0x7faf58065930 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 --2- 172.21.15.157:0/307463051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf580cd520 0x7faf58155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 >> 172.21.15.157:0/307463051 conn(0x7faf58070990 msgr2=0x7faf580cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 shutdown_connections 2023-11-06T03:08:40.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.616+0000 7faf5e66e640 1 -- 172.21.15.157:0/307463051 wait complete. 2023-11-06T03:08:40.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:08:40.633 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:08:40.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.744+0000 7f7cf834e640 1 Processor -- start 2023-11-06T03:08:40.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.744+0000 7f7cf834e640 1 -- start start 2023-11-06T03:08:40.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf834e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cf00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf834e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7cf00b2af0 0x7f7cf0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf834e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf015e440 con 0x7f7cf006d000 2023-11-06T03:08:40.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf834e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf006c240 con 0x7f7cf00b2af0 2023-11-06T03:08:40.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf834e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf006c520 con 0x7f7cf01599c0 2023-11-06T03:08:40.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf58e9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7cf006d000 0x7f7cf00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53158/0 (socket says 172.21.15.157:53158) 2023-11-06T03:08:40.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf58e9640 1 -- 172.21.15.157:0/3142693045 learned_addr learned my addr 172.21.15.157:0/3142693045 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cf00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190300737 0 0) 0x7f7cf015e440 con 0x7f7cf006d000 2023-11-06T03:08:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf68eb640 1 --2- 172.21.15.157:0/3142693045 >> v2:172.21.15.195:3300/0 conn(0x7f7cf00b2af0 0x7f7cf0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cdc003530 con 0x7f7cf006d000 2023-11-06T03:08:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 179673079 0 0) 0x7f7cdc003530 con 0x7f7cf006d000 2023-11-06T03:08:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 msgr2=0x7f7cf00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cf00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 >> v2:172.21.15.195:3300/0 conn(0x7f7cf00b2af0 msgr2=0x7f7cf0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 --2- 172.21.15.157:0/3142693045 >> v2:172.21.15.195:3300/0 conn(0x7f7cf00b2af0 0x7f7cf0159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf006c800 con 0x7f7cf006d000 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7ce0001680 con 0x7f7cf006d000 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf68eb640 1 --2- 172.21.15.157:0/3142693045 >> v2:172.21.15.195:3300/0 conn(0x7f7cf00b2af0 0x7f7cf0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 >> v1:172.21.15.157:6789/0 conn(0x7f7cf006d000 legacy=0x7f7cf00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cdc003980 0x7f7cdc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cdc0047d0 0x7f7cdc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cdc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf006c240 con 0x7f7cdc003980 2023-11-06T03:08:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf006c520 con 0x7f7cdc0047d0 2023-11-06T03:08:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc00b990 con 0x7f7cf01599c0 2023-11-06T03:08:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cdc003980 0x7f7cdc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf68eb640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cdc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf58e9640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cdc0047d0 0x7f7cdc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 -- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 msgr2=0x7f7cdc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf01599c0 0x7f7cdc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 -- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cdc0047d0 msgr2=0x7f7cdc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cdc0047d0 0x7f7cdc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.748+0000 7f7cf60ea640 1 -- 172.21.15.157:0/3142693045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc003530 con 0x7f7cdc003980 2023-11-06T03:08:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cdc003980 0x7f7cdc003da0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009ae0 tx=0x7f7cec01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfe993a20050f08 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0a5530 con 0x7f7cdc003980 2023-11-06T03:08:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0a5690 con 0x7f7cdc003980 2023-11-06T03:08:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf50e8640 1 -- 172.21.15.157:0/3142693045 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0a5a40 con 0x7f7cdc003980 2023-11-06T03:08:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/3142693045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cdc003980 msgr2=0x7f7cdc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 --2- 172.21.15.157:0/3142693045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cdc003980 0x7f7cdc003da0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009ae0 tx=0x7f7cec01ca20 comp rx=0 tx=0).stop 2023-11-06T03:08:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf68eb640 1 -- 172.21.15.157:0/3142693045 reap_dead start 2023-11-06T03:08:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/3142693045 shutdown_connections 2023-11-06T03:08:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/3142693045 >> 172.21.15.157:0/3142693045 conn(0x7f7cf0065930 msgr2=0x7f7cf0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/3142693045 shutdown_connections 2023-11-06T03:08:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/3142693045 wait complete. 2023-11-06T03:08:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 Processor -- start 2023-11-06T03:08:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- start start 2023-11-06T03:08:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf014e3a0 0x7f7cf00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cf0150560 0x7f7cf00ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0154d20 con 0x7f7cf0150180 2023-11-06T03:08:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf01513e0 con 0x7f7cf0150560 2023-11-06T03:08:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0150af0 con 0x7f7cf014e3a0 2023-11-06T03:08:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57710/0 (socket says 172.21.15.157:57710) 2023-11-06T03:08:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 -- 172.21.15.157:0/630679714 learned_addr learned my addr 172.21.15.157:0/630679714 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf68eb640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cf0150560 0x7f7cf00ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf58e9640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf014e3a0 0x7f7cf00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 -- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf014e3a0 msgr2=0x7f7cf00cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf014e3a0 0x7f7cf00cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 -- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cf0150560 msgr2=0x7f7cf00ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cf0150560 0x7f7cf00ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 -- 172.21.15.157:0/630679714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf014de20 con 0x7f7cf0150180 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf60ea640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0b0040 tx=0x7f7cec0197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0a5ea0 con 0x7f7cf0150180 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec004020 con 0x7f7cf0150180 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf0150d90 con 0x7f7cf0150180 2023-11-06T03:08:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0043d0 con 0x7f7cf0150180 2023-11-06T03:08:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.752+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf014e8a0 con 0x7f7cf0150180 2023-11-06T03:08:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf0072750 con 0x7f7cf0150180 2023-11-06T03:08:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7cec0b4070 con 0x7f7cf0150180 2023-11-06T03:08:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7ce6ffd640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 0x7f7cd4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7cec11caf0 con 0x7f7cf0150180 2023-11-06T03:08:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7cf58e9640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 0x7f7cd4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.756+0000 7f7cf58e9640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 0x7f7cd4067f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0001ec0 tx=0x7f7ce00047f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.760+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7cec0ea6c0 con 0x7f7cf0150180 2023-11-06T03:08:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.912+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cf00cb110 con 0x7f7cf0150180 2023-11-06T03:08:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.912+0000 7f7ce6ffd640 1 -- 172.21.15.157:0/630679714 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cec0b2070 con 0x7f7cf0150180 2023-11-06T03:08:40.914 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:08:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 msgr2=0x7f7cd4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 0x7f7cd4067f50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0001ec0 tx=0x7f7ce00047f0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 msgr2=0x7f7cf01547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0b0040 tx=0x7f7cec0197c0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 shutdown_connections 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cd4065a90 0x7f7cd4067f50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cf0150560 0x7f7cf00ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cf014e3a0 0x7f7cf00cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 --2- 172.21.15.157:0/630679714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf0150180 0x7f7cf01547e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 >> 172.21.15.157:0/630679714 conn(0x7f7cf0065930 msgr2=0x7f7cf015c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:40.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 shutdown_connections 2023-11-06T03:08:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:40.916+0000 7f7cf834e640 1 -- 172.21.15.157:0/630679714 wait complete. 2023-11-06T03:08:40.932 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:08:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 Processor -- start 2023-11-06T03:08:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 -- start start 2023-11-06T03:08:41.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0cde90 0x7f08bc0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc160830 con 0x7f08bc15cc70 2023-11-06T03:08:41.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc1614b0 con 0x7f08bc0cfed0 2023-11-06T03:08:41.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08c1fa1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0d0360 con 0x7f08bc0cde90 2023-11-06T03:08:41.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57714/0 (socket says 172.21.15.157:57714) 2023-11-06T03:08:41.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 -- 172.21.15.157:0/1305548797 learned_addr learned my addr 172.21.15.157:0/1305548797 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:41.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08baffd640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0cde90 0x7f08bc0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 -- 172.21.15.157:0/1305548797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0cde90 msgr2=0x7f08bc0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:41.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0cde90 0x7f08bc0cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 -- 172.21.15.157:0/1305548797 >> v1:172.21.15.195:6789/0 conn(0x7f08bc15cc70 legacy=0x7f08bc0d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:41.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.060+0000 7f08bbfff640 1 -- 172.21.15.157:0/1305548797 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc160b10 con 0x7f08bc0cfed0 2023-11-06T03:08:41.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08bbfff640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46824f0e102052f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08ba7fc640 1 -- 172.21.15.157:0/1305548797 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40af400 con 0x7f08bc0cfed0 2023-11-06T03:08:41.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08ba7fc640 1 -- 172.21.15.157:0/1305548797 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b40ab050 con 0x7f08bc0cfed0 2023-11-06T03:08:41.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08ba7fc640 1 -- 172.21.15.157:0/1305548797 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40af780 con 0x7f08bc0cfed0 2023-11-06T03:08:41.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- 172.21.15.157:0/1305548797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cfed0 msgr2=0x7f08bc0d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009e50 tx=0x7f08b401f1c0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- 172.21.15.157:0/1305548797 shutdown_connections 2023-11-06T03:08:41.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cfed0 0x7f08bc0d0bb0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/1305548797 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc0cde90 0x7f08bc0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- 172.21.15.157:0/1305548797 >> 172.21.15.157:0/1305548797 conn(0x7f08bc070560 msgr2=0x7f08bc070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- 172.21.15.157:0/1305548797 shutdown_connections 2023-11-06T03:08:41.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- 172.21.15.157:0/1305548797 wait complete. 2023-11-06T03:08:41.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 Processor -- start 2023-11-06T03:08:41.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.064+0000 7f08c1fa1640 1 -- start start 2023-11-06T03:08:41.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0cfed0 0x7f08bc15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc15cc70 0x7f08bc0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15e2d0 con 0x7f08bc0cde90 2023-11-06T03:08:41.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15e6d0 con 0x7f08bc0cfed0 2023-11-06T03:08:41.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1506c0 con 0x7f08bc15cc70 2023-11-06T03:08:41.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57728/0 (socket says 172.21.15.157:57728) 2023-11-06T03:08:41.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 -- 172.21.15.157:0/234689421 learned_addr learned my addr 172.21.15.157:0/234689421 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:41.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bbfff640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc15cc70 0x7f08bc0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08baffd640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0cfed0 0x7f08bc15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 -- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc15cc70 msgr2=0x7f08bc0d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc15cc70 0x7f08bc0d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 -- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0cfed0 msgr2=0x7f08bc15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0cfed0 0x7f08bc15d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 -- 172.21.15.157:0/234689421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009b30 con 0x7f08bc0cde90 2023-11-06T03:08:41.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08bb7fe640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b410 tx=0x7f08ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac004020 con 0x7f08bc0cde90 2023-11-06T03:08:41.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac004180 con 0x7f08bc0cde90 2023-11-06T03:08:41.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc15da30 con 0x7f08bc0cde90 2023-11-06T03:08:41.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac011630 con 0x7f08bc0cde90 2023-11-06T03:08:41.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15dff0 con 0x7f08bc0cde90 2023-11-06T03:08:41.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc1610b0 con 0x7f08bc0cde90 2023-11-06T03:08:41.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f08ac011790 con 0x7f08bc0cde90 2023-11-06T03:08:41.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 0x7f08a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f08ac082c10 con 0x7f08bc0cde90 2023-11-06T03:08:41.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.068+0000 7f08baffd640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 0x7f08a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.072+0000 7f08baffd640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 0x7f08a8067f20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0c99d0 tx=0x7f08b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.072+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f08bc1610b0 con 0x7f08bc0cde90 2023-11-06T03:08:41.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08bc1610b0 con 0x7f08bc0cde90 2023-11-06T03:08:41.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08a3fff640 1 -- 172.21.15.157:0/234689421 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v63) v1 ==== 92+0+10 (secure 0 0 0) 0x7f08bc1610b0 con 0x7f08bc0cde90 2023-11-06T03:08:41.227 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:08:41.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 msgr2=0x7f08a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 0x7f08a8067f20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0c99d0 tx=0x7f08b0008040 comp rx=0 tx=0).stop 2023-11-06T03:08:41.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 msgr2=0x7f08bc150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b410 tx=0x7f08ac00b8d0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 shutdown_connections 2023-11-06T03:08:41.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08a8065a60 0x7f08a8067f20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08bc15cc70 0x7f08bc0d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08bc0cfed0 0x7f08bc15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 --2- 172.21.15.157:0/234689421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08bc0cde90 0x7f08bc150180 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 >> 172.21.15.157:0/234689421 conn(0x7f08bc070560 msgr2=0x7f08bc0b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 shutdown_connections 2023-11-06T03:08:41.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.228+0000 7f08c1fa1640 1 -- 172.21.15.157:0/234689421 wait complete. 2023-11-06T03:08:41.246 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:08:41.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 Processor -- start 2023-11-06T03:08:41.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 -- start start 2023-11-06T03:08:41.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0601551e0 0x7fa06006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 0x7fa06015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06015f580 con 0x7fa06015e5e0 2023-11-06T03:08:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06015efc0 con 0x7fa0601551e0 2023-11-06T03:08:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.368+0000 7fa0665ad640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06015f2a0 con 0x7fa0600b3b20 2023-11-06T03:08:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0601551e0 0x7fa06006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0601551e0 0x7fa06006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57740/0 (socket says 172.21.15.157:57740) 2023-11-06T03:08:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 -- 172.21.15.157:0/1116500344 learned_addr learned my addr 172.21.15.157:0/1116500344 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa064b4a640 1 --2- 172.21.15.157:0/1116500344 >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 0x7fa06015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 -- 172.21.15.157:0/1116500344 >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 msgr2=0x7fa06015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 --2- 172.21.15.157:0/1116500344 >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 0x7fa06015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 -- 172.21.15.157:0/1116500344 >> v1:172.21.15.195:6789/0 conn(0x7fa06015e5e0 legacy=0x7fa0600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 -- 172.21.15.157:0/1116500344 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06006f5a0 con 0x7fa0601551e0 2023-11-06T03:08:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa064b4a640 1 --2- 172.21.15.157:0/1116500344 >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 0x7fa06015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:41.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05ffff640 1 --2- 172.21.15.157:0/1116500344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0601551e0 0x7fa06006fda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa0600cbfe0 tx=0x7fa05001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeb0955b5b91cd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05effd640 1 -- 172.21.15.157:0/1116500344 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500ac630 con 0x7fa0601551e0 2023-11-06T03:08:41.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05effd640 1 -- 172.21.15.157:0/1116500344 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0500ac790 con 0x7fa0601551e0 2023-11-06T03:08:41.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa05effd640 1 -- 172.21.15.157:0/1116500344 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500acb20 con 0x7fa0601551e0 2023-11-06T03:08:41.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 -- 172.21.15.157:0/1116500344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0601551e0 msgr2=0x7fa06006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 --2- 172.21.15.157:0/1116500344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0601551e0 0x7fa06006fda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa0600cbfe0 tx=0x7fa05001c3b0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 -- 172.21.15.157:0/1116500344 shutdown_connections 2023-11-06T03:08:41.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 --2- 172.21.15.157:0/1116500344 >> v2:172.21.15.195:3300/0 conn(0x7fa0600b3b20 0x7fa06015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 --2- 172.21.15.157:0/1116500344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0601551e0 0x7fa06006fda0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 -- 172.21.15.157:0/1116500344 >> 172.21.15.157:0/1116500344 conn(0x7fa060065930 msgr2=0x7fa060065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 -- 172.21.15.157:0/1116500344 shutdown_connections 2023-11-06T03:08:41.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.372+0000 7fa0665ad640 1 -- 172.21.15.157:0/1116500344 wait complete. 2023-11-06T03:08:41.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 Processor -- start 2023-11-06T03:08:41.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- start start 2023-11-06T03:08:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 0x7fa06014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 0x7fa0600b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600ccb50 con 0x7fa0600b3b20 2023-11-06T03:08:41.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06014c5a0 con 0x7fa06015e5e0 2023-11-06T03:08:41.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06014c300 con 0x7fa0601551e0 2023-11-06T03:08:41.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57746/0 (socket says 172.21.15.157:57746) 2023-11-06T03:08:41.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 -- 172.21.15.157:0/3013735850 learned_addr learned my addr 172.21.15.157:0/3013735850 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:41.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05f7fe640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 0x7fa06014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 -- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 msgr2=0x7fa06014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa064b4a640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 0x7fa0600b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 0x7fa06014bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 -- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 msgr2=0x7fa0600b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 0x7fa0600b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 -- 172.21.15.157:0/3013735850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500094d0 con 0x7fa0600b3b20 2023-11-06T03:08:41.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05f7fe640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 0x7fa06014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:41.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa064b4a640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 0x7fa0600b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:41.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05ffff640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa0500af040 tx=0x7fa05001abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500b7620 con 0x7fa0600b3b20 2023-11-06T03:08:41.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0500b77a0 con 0x7fa0600b3b20 2023-11-06T03:08:41.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa060155740 con 0x7fa0600b3b20 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500b7b50 con 0x7fa0600b3b20 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060155d00 con 0x7fa0600b3b20 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa060072750 con 0x7fa0600b3b20 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.376+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa0500b3040 con 0x7fa0600b3b20 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.380+0000 7fa05cff9640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 0x7fa04c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.380+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa05011d480 con 0x7fa0600b3b20 2023-11-06T03:08:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.380+0000 7fa05f7fe640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 0x7fa04c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.380+0000 7fa05f7fe640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 0x7fa04c067f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0600b2be0 tx=0x7fa054009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.384+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa0500eb1e0 con 0x7fa0600b3b20 2023-11-06T03:08:41.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.532+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa060071730 con 0x7fa0600b3b20 2023-11-06T03:08:41.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.532+0000 7fa05cff9640 1 -- 172.21.15.157:0/3013735850 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa0500b7cb0 con 0x7fa0600b3b20 2023-11-06T03:08:41.533 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:41.534 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:08:41.534 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:41.534 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:08:41.534 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:08:41.535 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:41.535 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:08:41.535 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:08:41.535 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:08:41.535 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:41.536 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:08:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 msgr2=0x7fa04c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 0x7fa04c067f50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0600b2be0 tx=0x7fa054009040 comp rx=0 tx=0).stop 2023-11-06T03:08:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 msgr2=0x7fa0600af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa0500af040 tx=0x7fa05001abf0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 shutdown_connections 2023-11-06T03:08:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa04c065a90 0x7fa04c067f50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa06015e5e0 0x7fa0600b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa0601551e0 0x7fa06014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 --2- 172.21.15.157:0/3013735850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0600b3b20 0x7fa0600af450 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 >> 172.21.15.157:0/3013735850 conn(0x7fa060065930 msgr2=0x7fa0600ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 shutdown_connections 2023-11-06T03:08:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.536+0000 7fa0665ad640 1 -- 172.21.15.157:0/3013735850 wait complete. 2023-11-06T03:08:41.551 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:08:41.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:08:41.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.660+0000 7fc65bde1640 1 Processor -- start 2023-11-06T03:08:41.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.660+0000 7fc65bde1640 1 -- start start 2023-11-06T03:08:41.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65bde1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 0x7fc6540b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65bde1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc65415dda0 0x7fc65415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65bde1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654070ee0 con 0x7fc65406b2f0 2023-11-06T03:08:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65bde1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65415b060 con 0x7fc65415dda0 2023-11-06T03:08:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65bde1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65406b780 con 0x7fc654159c80 2023-11-06T03:08:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc659b7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 0x7fc6540b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65a37e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc65415dda0 0x7fc65415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65937c640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fc65406b2f0 0x7fc65415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52776/0 (socket says 172.21.15.157:52776) 2023-11-06T03:08:41.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc65937c640 1 -- 172.21.15.157:0/2928787960 learned_addr learned my addr 172.21.15.157:0/2928787960 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:41.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc659b7d640 1 -- 172.21.15.157:0/2928787960 >> v2:172.21.15.195:3300/0 conn(0x7fc65415dda0 msgr2=0x7fc65415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:41.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc659b7d640 1 --2- 172.21.15.157:0/2928787960 >> v2:172.21.15.195:3300/0 conn(0x7fc65415dda0 0x7fc65415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc659b7d640 1 -- 172.21.15.157:0/2928787960 >> v1:172.21.15.195:6789/0 conn(0x7fc65406b2f0 legacy=0x7fc65415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.664+0000 7fc659b7d640 1 -- 172.21.15.157:0/2928787960 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540b2a90 con 0x7fc654159c80 2023-11-06T03:08:41.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc659b7d640 1 --2- 172.21.15.157:0/2928787960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 0x7fc6540b3570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6500097c0 tx=0x7fc65001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b154e377b30c90a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc658b7b640 1 -- 172.21.15.157:0/2928787960 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500ac620 con 0x7fc654159c80 2023-11-06T03:08:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc658b7b640 1 -- 172.21.15.157:0/2928787960 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6500ac780 con 0x7fc654159c80 2023-11-06T03:08:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc658b7b640 1 -- 172.21.15.157:0/2928787960 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500acb10 con 0x7fc654159c80 2023-11-06T03:08:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- 172.21.15.157:0/2928787960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 msgr2=0x7fc6540b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- 172.21.15.157:0/2928787960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 0x7fc6540b3570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6500097c0 tx=0x7fc65001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- 172.21.15.157:0/2928787960 shutdown_connections 2023-11-06T03:08:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- 172.21.15.157:0/2928787960 >> v2:172.21.15.195:3300/0 conn(0x7fc65415dda0 0x7fc65415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- 172.21.15.157:0/2928787960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc654159c80 0x7fc6540b3570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- 172.21.15.157:0/2928787960 >> 172.21.15.157:0/2928787960 conn(0x7fc6540b43b0 msgr2=0x7fc654065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- 172.21.15.157:0/2928787960 shutdown_connections 2023-11-06T03:08:41.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- 172.21.15.157:0/2928787960 wait complete. 2023-11-06T03:08:41.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 Processor -- start 2023-11-06T03:08:41.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- start start 2023-11-06T03:08:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 0x7fc654146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6541643b0 0x7fc6540ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654148470 con 0x7fc65415dda0 2023-11-06T03:08:41.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541629b0 con 0x7fc654159c80 2023-11-06T03:08:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65bde1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6541477f0 con 0x7fc6541643b0 2023-11-06T03:08:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57752/0 (socket says 172.21.15.157:57752) 2023-11-06T03:08:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 -- 172.21.15.157:0/4010418923 learned_addr learned my addr 172.21.15.157:0/4010418923 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:41.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc659b7d640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 0x7fc654146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65a37e640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6541643b0 0x7fc6540ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 -- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6541643b0 msgr2=0x7fc6540ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6541643b0 0x7fc6540ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 -- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 msgr2=0x7fc654146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 0x7fc654146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 -- 172.21.15.157:0/4010418923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc650009420 con 0x7fc65415dda0 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc659b7d640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 0x7fc654146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.668+0000 7fc65937c640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c500 tx=0x7fc64400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644005cf0 con 0x7fc65415dda0 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644005e50 con 0x7fc65415dda0 2023-11-06T03:08:41.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc654147ad0 con 0x7fc65415dda0 2023-11-06T03:08:41.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440072f0 con 0x7fc65415dda0 2023-11-06T03:08:41.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc654148090 con 0x7fc65415dda0 2023-11-06T03:08:41.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6540cf3b0 con 0x7fc65415dda0 2023-11-06T03:08:41.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc644007450 con 0x7fc65415dda0 2023-11-06T03:08:41.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 0x7fc638067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:41.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6440824f0 con 0x7fc65415dda0 2023-11-06T03:08:41.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc659b7d640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 0x7fc638067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:41.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.672+0000 7fc659b7d640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 0x7fc638067f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc6500ae000 tx=0x7fc6500a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:41.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.676+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc644050160 con 0x7fc65415dda0 2023-11-06T03:08:41.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.828+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc6541485b0 con 0x7fc65415dda0 2023-11-06T03:08:41.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.924+0000 7fc64a7fc640 1 -- 172.21.15.157:0/4010418923 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc6541485b0 con 0x7fc65415dda0 2023-11-06T03:08:41.926 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:08:41.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.928+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 msgr2=0x7fc638067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.928+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 0x7fc638067f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc6500ae000 tx=0x7fc6500a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:41.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.928+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 msgr2=0x7fc654163e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:41.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.928+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c500 tx=0x7fc64400c9c0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 shutdown_connections 2023-11-06T03:08:41.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc638065a60 0x7fc638067f20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6541643b0 0x7fc6540ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65415dda0 0x7fc654163e70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 --2- 172.21.15.157:0/4010418923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc654159c80 0x7fc654146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:41.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 >> 172.21.15.157:0/4010418923 conn(0x7fc6540b43b0 msgr2=0x7fc6540ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:41.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 shutdown_connections 2023-11-06T03:08:41.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:41.932+0000 7fc65bde1640 1 -- 172.21.15.157:0/4010418923 wait complete. 2023-11-06T03:08:41.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:08:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 Processor -- start 2023-11-06T03:08:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 -- start start 2023-11-06T03:08:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1f00b2af0 0x7ff1f00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f015e440 con 0x7ff1f006d000 2023-11-06T03:08:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006c240 con 0x7ff1f01599c0 2023-11-06T03:08:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f73c6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f006c520 con 0x7ff1f00b2af0 2023-11-06T03:08:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f4961640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1f00b2af0 0x7ff1f00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f5963640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff1f006d000 0x7ff1f0159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53168/0 (socket says 172.21.15.157:53168) 2023-11-06T03:08:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f4961640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1f00b2af0 0x7ff1f00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57760/0 (socket says 172.21.15.157:57760) 2023-11-06T03:08:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f5963640 1 -- 172.21.15.157:0/3966868825 learned_addr learned my addr 172.21.15.157:0/3966868825 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f4961640 1 -- 172.21.15.157:0/3966868825 >> v1:172.21.15.195:6789/0 conn(0x7ff1f01599c0 legacy=0x7ff1f00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f4961640 1 -- 172.21.15.157:0/3966868825 >> v1:172.21.15.157:6789/0 conn(0x7ff1f006d000 legacy=0x7ff1f0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1f4961640 1 -- 172.21.15.157:0/3966868825 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f006c800 con 0x7ff1f00b2af0 2023-11-06T03:08:42.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.076+0000 7ff1e7fff640 1 -- 172.21.15.157:0/3966868825 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655603057 0 0) 0x7ff1f015e440 con 0x7ff1f006d000 2023-11-06T03:08:42.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f4961640 1 --2- 172.21.15.157:0/3966868825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f00b2af0 0x7ff1f00ae000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000ba60 tx=0x7ff1e001eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d627c0da3f48d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:42.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1e7fff640 1 -- 172.21.15.157:0/3966868825 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00a7bd0 con 0x7ff1f00b2af0 2023-11-06T03:08:42.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1e7fff640 1 -- 172.21.15.157:0/3966868825 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e00a7d30 con 0x7ff1f00b2af0 2023-11-06T03:08:42.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1e7fff640 1 -- 172.21.15.157:0/3966868825 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00a7ea0 con 0x7ff1f00b2af0 2023-11-06T03:08:42.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- 172.21.15.157:0/3966868825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f00b2af0 msgr2=0x7ff1f00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/3966868825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f00b2af0 0x7ff1f00ae000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000ba60 tx=0x7ff1e001eae0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- 172.21.15.157:0/3966868825 shutdown_connections 2023-11-06T03:08:42.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/3966868825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f00b2af0 0x7ff1f00ae000 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- 172.21.15.157:0/3966868825 >> 172.21.15.157:0/3966868825 conn(0x7ff1f0065930 msgr2=0x7ff1f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:42.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- 172.21.15.157:0/3966868825 shutdown_connections 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- 172.21.15.157:0/3966868825 wait complete. 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 Processor -- start 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.080+0000 7ff1f73c6640 1 -- start start 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 0x7ff1f014ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:42.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1f01599c0 0x7ff1f015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00cd160 con 0x7ff1f006d000 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00cc030 con 0x7ff1f00b2af0 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f014e2f0 con 0x7ff1f01599c0 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57776/0 (socket says 172.21.15.157:57776) 2023-11-06T03:08:42.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 -- 172.21.15.157:0/2904991146 learned_addr learned my addr 172.21.15.157:0/2904991146 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:42.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f4961640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 0x7ff1f014ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:42.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5963640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1f01599c0 0x7ff1f015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:42.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 -- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1f01599c0 msgr2=0x7ff1f015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1f01599c0 0x7ff1f015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 -- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 msgr2=0x7ff1f014ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 0x7ff1f014ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 -- 172.21.15.157:0/2904991146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e000b6c0 con 0x7ff1f006d000 2023-11-06T03:08:42.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f4961640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 0x7ff1f014ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:42.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f5162640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00b410 tx=0x7ff1ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:42.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec004020 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ec004180 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f00cda10 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec011630 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00cdfd0 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f0072750 con 0x7ff1f006d000 2023-11-06T03:08:42.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff1ec011790 con 0x7ff1f006d000 2023-11-06T03:08:42.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:42.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5580+0+0 (secure 0 0 0) 0x7ff1ec082eb0 con 0x7ff1f006d000 2023-11-06T03:08:42.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.084+0000 7ff1f4961640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:42.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.088+0000 7ff1f4961640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 0x7ff1d4067f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b1000 tx=0x7ff1e00a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:42.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.088+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff1ec0508f0 con 0x7ff1f006d000 2023-11-06T03:08:42.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.244+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff1f00cb110 con 0x7ff1f006d000 2023-11-06T03:08:42.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.932+0000 7ff1e5ffb640 1 -- 172.21.15.157:0/2904991146 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff1ec015020 con 0x7ff1f006d000 2023-11-06T03:08:42.942 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:08:42.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 msgr2=0x7ff1d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 0x7ff1d4067f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b1000 tx=0x7ff1e00a9070 comp rx=0 tx=0).stop 2023-11-06T03:08:42.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 msgr2=0x7ff1f00ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:42.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00b410 tx=0x7ff1ec00b8d0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 shutdown_connections 2023-11-06T03:08:42.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1d4065a60 0x7ff1d4067f20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1f01599c0 0x7ff1f015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1f00b2af0 0x7ff1f014ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 --2- 172.21.15.157:0/2904991146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1f006d000 0x7ff1f00ccc20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:42.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 >> 172.21.15.157:0/2904991146 conn(0x7ff1f0065930 msgr2=0x7ff1f00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:42.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 shutdown_connections 2023-11-06T03:08:42.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:42.948+0000 7ff1f73c6640 1 -- 172.21.15.157:0/2904991146 wait complete. 2023-11-06T03:08:42.976 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:08:43.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 Processor -- start 2023-11-06T03:08:43.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 -- start start 2023-11-06T03:08:43.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27f01551e0 0x7f27f006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 0x7f27f00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 0x7f27f00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f015f580 con 0x7f27f01551e0 2023-11-06T03:08:43.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f015efc0 con 0x7f27f015ea60 2023-11-06T03:08:43.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27f4bd4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015f2a0 con 0x7f27f015e5e0 2023-11-06T03:08:43.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27f01551e0 0x7f27f006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27f01551e0 0x7f27f006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57778/0 (socket says 172.21.15.157:57778) 2023-11-06T03:08:43.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 -- 172.21.15.157:0/904752154 learned_addr learned my addr 172.21.15.157:0/904752154 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27edd9b640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 0x7f27f00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 -- 172.21.15.157:0/904752154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 msgr2=0x7f27f00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 0x7f27f00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 -- 172.21.15.157:0/904752154 >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 msgr2=0x7f27f00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27eed9d640 1 --2- 172.21.15.157:0/904752154 >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 0x7f27f00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 --2- 172.21.15.157:0/904752154 >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 0x7f27f00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27ee59c640 1 -- 172.21.15.157:0/904752154 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f006f5a0 con 0x7f27f01551e0 2023-11-06T03:08:43.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27edd9b640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 0x7f27f00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:43.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.100+0000 7f27eed9d640 1 --2- 172.21.15.157:0/904752154 >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 0x7f27f00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:43.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f006fda0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f27f00cc5d0 tx=0x7f27e001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db3afd488ebf7a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ed59a640 1 -- 172.21.15.157:0/904752154 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab620 con 0x7f27f01551e0 2023-11-06T03:08:43.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ed59a640 1 -- 172.21.15.157:0/904752154 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00ab780 con 0x7f27f01551e0 2023-11-06T03:08:43.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ed59a640 1 -- 172.21.15.157:0/904752154 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00abb10 con 0x7f27f01551e0 2023-11-06T03:08:43.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- 172.21.15.157:0/904752154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 msgr2=0x7f27f006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f006fda0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f27f00cc5d0 tx=0x7f27e001c440 comp rx=0 tx=0).stop 2023-11-06T03:08:43.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- 172.21.15.157:0/904752154 shutdown_connections 2023-11-06T03:08:43.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/904752154 >> v2:172.21.15.195:3300/0 conn(0x7f27f015ea60 0x7f27f00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015e5e0 0x7f27f00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/904752154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f006fda0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- 172.21.15.157:0/904752154 >> 172.21.15.157:0/904752154 conn(0x7f27f0065930 msgr2=0x7f27f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- 172.21.15.157:0/904752154 shutdown_connections 2023-11-06T03:08:43.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- 172.21.15.157:0/904752154 wait complete. 2023-11-06T03:08:43.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 Processor -- start 2023-11-06T03:08:43.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- start start 2023-11-06T03:08:43.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f015e5e0 0x7f27f00ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015ea60 0x7f27f00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00cd4f0 con 0x7f27f01551e0 2023-11-06T03:08:43.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00cee00 con 0x7f27f015e5e0 2023-11-06T03:08:43.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27f4bd4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0142600 con 0x7f27f015ea60 2023-11-06T03:08:43.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57790/0 (socket says 172.21.15.157:57790) 2023-11-06T03:08:43.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 -- 172.21.15.157:0/2210560450 learned_addr learned my addr 172.21.15.157:0/2210560450 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27eed9d640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015ea60 0x7f27f00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27edd9b640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f015e5e0 0x7f27f00ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 -- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015ea60 msgr2=0x7f27f00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015ea60 0x7f27f00ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 -- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f015e5e0 msgr2=0x7f27f00ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f015e5e0 0x7f27f00ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.104+0000 7f27ee59c640 1 -- 172.21.15.157:0/2210560450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00094d0 con 0x7f27f01551e0 2023-11-06T03:08:43.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27ee59c640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f27e00ad040 tx=0x7f27e001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00abe80 con 0x7f27f01551e0 2023-11-06T03:08:43.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00be3f0 con 0x7f27f01551e0 2023-11-06T03:08:43.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f00ccbc0 con 0x7f27f01551e0 2023-11-06T03:08:43.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00be7a0 con 0x7f27f01551e0 2023-11-06T03:08:43.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f00cd180 con 0x7f27f01551e0 2023-11-06T03:08:43.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f0072750 con 0x7f27f01551e0 2023-11-06T03:08:43.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f27e00c8420 con 0x7f27f01551e0 2023-11-06T03:08:43.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 0x7f27d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f27e011d980 con 0x7f27f01551e0 2023-11-06T03:08:43.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27edd9b640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 0x7f27d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.108+0000 7f27edd9b640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 0x7f27d0067f20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27f01f6cd0 tx=0x7f27e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.112+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f27e00eb440 con 0x7f27f01551e0 2023-11-06T03:08:43.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.300+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f27f00b3e10 con 0x7f27f01551e0 2023-11-06T03:08:43.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.308+0000 7f27deffd640 1 -- 172.21.15.157:0/2210560450 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v45) v1 ==== 173+0+0 (secure 0 0 0) 0x7f27e00eea60 con 0x7f27f01551e0 2023-11-06T03:08:43.315 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 9 and data pool 10 2023-11-06T03:08:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 msgr2=0x7f27d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 0x7f27d0067f20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f27f01f6cd0 tx=0x7f27e4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 msgr2=0x7f27f014ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f27e00ad040 tx=0x7f27e001ac80 comp rx=0 tx=0).stop 2023-11-06T03:08:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 shutdown_connections 2023-11-06T03:08:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f27d0065a60 0x7f27d0067f20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27f015ea60 0x7f27f00ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f27f015e5e0 0x7f27f00ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 --2- 172.21.15.157:0/2210560450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f01551e0 0x7f27f014ce10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 >> 172.21.15.157:0/2210560450 conn(0x7f27f0065930 msgr2=0x7f27f00b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 shutdown_connections 2023-11-06T03:08:43.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.320+0000 7f27f4bd4640 1 -- 172.21.15.157:0/2210560450 wait complete. 2023-11-06T03:08:43.343 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:43.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 Processor -- start 2023-11-06T03:08:43.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 -- start start 2023-11-06T03:08:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b015ce80 0x7fa9b00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa9b00ce1a0 0x7fa9b00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00d2920 con 0x7fa9b015ce80 2023-11-06T03:08:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b00d18a0 con 0x7fa9b00ce1a0 2023-11-06T03:08:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b82f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00d1b60 con 0x7fa9b015caa0 2023-11-06T03:08:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b015ce80 0x7fa9b00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b015ce80 0x7fa9b00cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57804/0 (socket says 172.21.15.157:57804) 2023-11-06T03:08:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 -- 172.21.15.157:0/4123955524 learned_addr learned my addr 172.21.15.157:0/4123955524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b608f640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b6890640 1 --2- 172.21.15.157:0/4123955524 >> v2:172.21.15.195:3300/0 conn(0x7fa9b00ce1a0 0x7fa9b00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 -- 172.21.15.157:0/4123955524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 msgr2=0x7fa9b00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 -- 172.21.15.157:0/4123955524 >> v2:172.21.15.195:3300/0 conn(0x7fa9b00ce1a0 msgr2=0x7fa9b00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 --2- 172.21.15.157:0/4123955524 >> v2:172.21.15.195:3300/0 conn(0x7fa9b00ce1a0 0x7fa9b00d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.464+0000 7fa9b588e640 1 -- 172.21.15.157:0/4123955524 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00d1e40 con 0x7fa9b015ce80 2023-11-06T03:08:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b588e640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b00cfb70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00d0780 tx=0x7fa9a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ab7887384091b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b508d640 1 -- 172.21.15.157:0/4123955524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00aa5f0 con 0x7fa9b015ce80 2023-11-06T03:08:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b508d640 1 -- 172.21.15.157:0/4123955524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a00aa750 con 0x7fa9b015ce80 2023-11-06T03:08:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b508d640 1 -- 172.21.15.157:0/4123955524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00aaae0 con 0x7fa9b015ce80 2023-11-06T03:08:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- 172.21.15.157:0/4123955524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 msgr2=0x7fa9b00cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b00cfb70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00d0780 tx=0x7fa9a001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- 172.21.15.157:0/4123955524 shutdown_connections 2023-11-06T03:08:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/4123955524 >> v2:172.21.15.195:3300/0 conn(0x7fa9b00ce1a0 0x7fa9b00d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b00cfb70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/4123955524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- 172.21.15.157:0/4123955524 >> 172.21.15.157:0/4123955524 conn(0x7fa9b0070790 msgr2=0x7fa9b0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- 172.21.15.157:0/4123955524 shutdown_connections 2023-11-06T03:08:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- 172.21.15.157:0/4123955524 wait complete. 2023-11-06T03:08:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 Processor -- start 2023-11-06T03:08:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- start start 2023-11-06T03:08:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 0x7fa9b00c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00c8910 con 0x7fa9b015ce80 2023-11-06T03:08:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00c9610 con 0x7fa9b00ce1a0 2023-11-06T03:08:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b82f3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b014da90 con 0x7fa9b015caa0 2023-11-06T03:08:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57806/0 (socket says 172.21.15.157:57806) 2023-11-06T03:08:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 -- 172.21.15.157:0/778985127 learned_addr learned my addr 172.21.15.157:0/778985127 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b588e640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b608f640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 0x7fa9b00c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 -- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 msgr2=0x7fa9b014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 -- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 msgr2=0x7fa9b00c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 0x7fa9b00c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 -- 172.21.15.157:0/778985127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00094d0 con 0x7fa9b015ce80 2023-11-06T03:08:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b608f640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 0x7fa9b00c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9b6890640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.468+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004220 con 0x7fa9b015ce80 2023-11-06T03:08:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac004380 con 0x7fa9b015ce80 2023-11-06T03:08:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b014c020 con 0x7fa9b015ce80 2023-11-06T03:08:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac003820 con 0x7fa9b015ce80 2023-11-06T03:08:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b014d440 con 0x7fa9b015ce80 2023-11-06T03:08:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b0072750 con 0x7fa9b015ce80 2023-11-06T03:08:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa9ac003980 con 0x7fa9b015ce80 2023-11-06T03:08:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9a6ffd640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 0x7fa994067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9ac0832b0 con 0x7fa9b015ce80 2023-11-06T03:08:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9b608f640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 0x7fa994067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.472+0000 7fa9b608f640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 0x7fa994067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00cd1d0 tx=0x7fa9a80074c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.476+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa9ac050b70 con 0x7fa9b015ce80 2023-11-06T03:08:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.632+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9b00b1a30 con 0x7fa9b015ce80 2023-11-06T03:08:43.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.632+0000 7fa9a6ffd640 1 -- 172.21.15.157:0/778985127 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fa9ac054190 con 0x7fa9b015ce80 2023-11-06T03:08:43.633 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:43.634 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":66,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:43.305224+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:43.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 msgr2=0x7fa994067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 0x7fa994067f20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00cd1d0 tx=0x7fa9a80074c0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 msgr2=0x7fa9b014df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac00c950 comp rx=0 tx=0).stop 2023-11-06T03:08:43.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 shutdown_connections 2023-11-06T03:08:43.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa994065a60 0x7fa994067f20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b015ce80 0x7fa9b014df20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b015caa0 0x7fa9b014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 --2- 172.21.15.157:0/778985127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b00ce1a0 0x7fa9b00c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 >> 172.21.15.157:0/778985127 conn(0x7fa9b0070790 msgr2=0x7fa9b0070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 shutdown_connections 2023-11-06T03:08:43.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.636+0000 7fa9b82f3640 1 -- 172.21.15.157:0/778985127 wait complete. 2023-11-06T03:08:43.652 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:43.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 Processor -- start 2023-11-06T03:08:43.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 -- start start 2023-11-06T03:08:43.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd25015caa0 0x7fd2500cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 0x7fd2500b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2500d2920 con 0x7fd25015ce80 2023-11-06T03:08:43.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2500d18a0 con 0x7fd25015caa0 2023-11-06T03:08:43.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd257e24640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500d1b60 con 0x7fd2500d0280 2023-11-06T03:08:43.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd25015caa0 0x7fd2500cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd25015caa0 0x7fd2500cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57810/0 (socket says 172.21.15.157:57810) 2023-11-06T03:08:43.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 -- 172.21.15.157:0/2340110643 learned_addr learned my addr 172.21.15.157:0/2340110643 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd2563c1640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 0x7fd2500b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 -- 172.21.15.157:0/2340110643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 msgr2=0x7fd2500b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 0x7fd2500b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 -- 172.21.15.157:0/2340110643 >> v1:172.21.15.195:6789/0 conn(0x7fd25015ce80 legacy=0x7fd2500cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:43.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd255bc0640 1 -- 172.21.15.157:0/2340110643 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2500d1e40 con 0x7fd25015caa0 2023-11-06T03:08:43.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.776+0000 7fd2563c1640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 0x7fd2500b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd255bc0640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cdc60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009870 tx=0x7fd24c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a7eb877815342 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd254bbe640 1 -- 172.21.15.157:0/2340110643 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0aa5f0 con 0x7fd25015caa0 2023-11-06T03:08:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd254bbe640 1 -- 172.21.15.157:0/2340110643 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c0aa750 con 0x7fd25015caa0 2023-11-06T03:08:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd254bbe640 1 -- 172.21.15.157:0/2340110643 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0aaae0 con 0x7fd25015caa0 2023-11-06T03:08:43.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 -- 172.21.15.157:0/2340110643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 msgr2=0x7fd2500cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cdc60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009870 tx=0x7fd24c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 -- 172.21.15.157:0/2340110643 shutdown_connections 2023-11-06T03:08:43.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2500d0280 0x7fd2500b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 --2- 172.21.15.157:0/2340110643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cdc60 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 -- 172.21.15.157:0/2340110643 >> 172.21.15.157:0/2340110643 conn(0x7fd250070790 msgr2=0x7fd250070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.780+0000 7fd257e24640 1 -- 172.21.15.157:0/2340110643 shutdown_connections 2023-11-06T03:08:43.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- 172.21.15.157:0/2340110643 wait complete. 2023-11-06T03:08:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 Processor -- start 2023-11-06T03:08:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- start start 2023-11-06T03:08:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 0x7fd2500c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25015ce80 0x7fd2500cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500cbed0 con 0x7fd25015caa0 2023-11-06T03:08:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500c6880 con 0x7fd2500d0280 2023-11-06T03:08:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500c6b60 con 0x7fd25015ce80 2023-11-06T03:08:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57822/0 (socket says 172.21.15.157:57822) 2023-11-06T03:08:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 -- 172.21.15.157:0/3860245452 learned_addr learned my addr 172.21.15.157:0/3860245452 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd255bc0640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 0x7fd2500c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2563c1640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25015ce80 0x7fd2500cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 -- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25015ce80 msgr2=0x7fd2500cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25015ce80 0x7fd2500cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 -- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 msgr2=0x7fd2500c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 0x7fd2500c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 -- 172.21.15.157:0/3860245452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0094d0 con 0x7fd25015caa0 2023-11-06T03:08:43.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd255bc0640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 0x7fd2500c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:43.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2553bf640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c410 tx=0x7fd24000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240011790 con 0x7fd25015caa0 2023-11-06T03:08:43.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2400118f0 con 0x7fd25015caa0 2023-11-06T03:08:43.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2500c6e40 con 0x7fd25015caa0 2023-11-06T03:08:43.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240011ce0 con 0x7fd25015caa0 2023-11-06T03:08:43.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2500cbb30 con 0x7fd25015caa0 2023-11-06T03:08:43.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.784+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250072750 con 0x7fd25015caa0 2023-11-06T03:08:43.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.788+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd24001c460 con 0x7fd25015caa0 2023-11-06T03:08:43.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.788+0000 7fd2467fc640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 0x7fd234067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.788+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd240082f90 con 0x7fd25015caa0 2023-11-06T03:08:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.788+0000 7fd255bc0640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 0x7fd234067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.788+0000 7fd255bc0640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 0x7fd234067f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0ae000 tx=0x7fd24c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.792+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd2400b4070 con 0x7fd25015caa0 2023-11-06T03:08:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.948+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2500c72f0 con 0x7fd25015caa0 2023-11-06T03:08:43.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.948+0000 7fd2467fc640 1 -- 172.21.15.157:0/3860245452 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fd240050880 con 0x7fd25015caa0 2023-11-06T03:08:43.949 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:43.950 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":66,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:43.305224+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:43.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 msgr2=0x7fd234067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 0x7fd234067f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0ae000 tx=0x7fd24c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:43.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 msgr2=0x7fd2500cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:43.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c410 tx=0x7fd24000c8d0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 shutdown_connections 2023-11-06T03:08:43.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd234065a60 0x7fd234067f20 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0ae000 tx=0x7fd24c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:43.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd25015ce80 0x7fd2500cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25015caa0 0x7fd2500cb5f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 --2- 172.21.15.157:0/3860245452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2500d0280 0x7fd2500c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:43.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 >> 172.21.15.157:0/3860245452 conn(0x7fd250070790 msgr2=0x7fd2500b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:43.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 shutdown_connections 2023-11-06T03:08:43.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:43.952+0000 7fd257e24640 1 -- 172.21.15.157:0/3860245452 wait complete. 2023-11-06T03:08:43.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:08:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 Processor -- start 2023-11-06T03:08:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 -- start start 2023-11-06T03:08:44.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f5784167c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57840ac9f0 con 0x7f578415cea0 2023-11-06T03:08:44.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57840ced30 con 0x7f57841609a0 2023-11-06T03:08:44.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578c45a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57840ce2a0 con 0x7f57840c9f40 2023-11-06T03:08:44.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f578415cea0 0x7f57840cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53184/0 (socket says 172.21.15.157:53184) 2023-11-06T03:08:44.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 -- 172.21.15.157:0/1215120895 learned_addr learned my addr 172.21.15.157:0/1215120895 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:44.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a9f7640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f5784167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931666791 0 0) 0x7f57840ac9f0 con 0x7f578415cea0 2023-11-06T03:08:44.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5770003530 con 0x7f578415cea0 2023-11-06T03:08:44.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 345277800 0 0) 0x7f5770003530 con 0x7f578415cea0 2023-11-06T03:08:44.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 msgr2=0x7f5784167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:44.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f5784167c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 >> v1:172.21.15.195:6789/0 conn(0x7f57841609a0 legacy=0x7f57840c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57840ce490 con 0x7f578415cea0 2023-11-06T03:08:44.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5780001680 con 0x7f578415cea0 2023-11-06T03:08:44.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 >> v1:172.21.15.157:6789/0 conn(0x7f578415cea0 legacy=0x7f57840cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57700039b0 0x7f5770003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f57700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5770004d10 0x7f5770005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577000ba90 con 0x7f57700039b0 2023-11-06T03:08:44.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577000bc00 con 0x7f5770004d10 2023-11-06T03:08:44.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577000bdc0 con 0x7f57840c9f40 2023-11-06T03:08:44.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57700039b0 0x7f5770003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f57899f5640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f57700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 -- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 msgr2=0x7f57700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:44.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c9f40 0x7f57700047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a9f7640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5770004d10 0x7f5770005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 -- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5770004d10 msgr2=0x7f5770005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:44.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5770004d10 0x7f5770005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.096+0000 7f578a1f6640 1 -- 172.21.15.157:0/1215120895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770003530 con 0x7f57700039b0 2023-11-06T03:08:44.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578a1f6640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57700039b0 0x7f5770003dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f578000c240 tx=0x7f578001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547ec70d4c82cfac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780005ae0 con 0x7f57700039b0 2023-11-06T03:08:44.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780005c40 con 0x7f57700039b0 2023-11-06T03:08:44.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f57891f4640 1 -- 172.21.15.157:0/1215120895 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780007070 con 0x7f57700039b0 2023-11-06T03:08:44.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- 172.21.15.157:0/1215120895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57700039b0 msgr2=0x7f5770003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 --2- 172.21.15.157:0/1215120895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57700039b0 0x7f5770003dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f578000c240 tx=0x7f578001cb80 comp rx=0 tx=0).stop 2023-11-06T03:08:44.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578a9f7640 1 -- 172.21.15.157:0/1215120895 reap_dead start 2023-11-06T03:08:44.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- 172.21.15.157:0/1215120895 shutdown_connections 2023-11-06T03:08:44.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- 172.21.15.157:0/1215120895 >> 172.21.15.157:0/1215120895 conn(0x7f57840cba30 msgr2=0x7f57840cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:44.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- 172.21.15.157:0/1215120895 shutdown_connections 2023-11-06T03:08:44.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- 172.21.15.157:0/1215120895 wait complete. 2023-11-06T03:08:44.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 Processor -- start 2023-11-06T03:08:44.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.100+0000 7f578c45a640 1 -- start start 2023-11-06T03:08:44.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57700039b0 0x7f57840b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c69d0 0x7f57840c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578415e240 con 0x7f57840c8ca0 2023-11-06T03:08:44.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578415e520 con 0x7f57700039b0 2023-11-06T03:08:44.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578415e800 con 0x7f57840c69d0 2023-11-06T03:08:44.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57848/0 (socket says 172.21.15.157:57848) 2023-11-06T03:08:44.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 -- 172.21.15.157:0/4063492666 learned_addr learned my addr 172.21.15.157:0/4063492666 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:44.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f57899f5640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c69d0 0x7f57840c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a1f6640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57700039b0 0x7f57840b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 -- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c69d0 msgr2=0x7f57840c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c69d0 0x7f57840c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 -- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57700039b0 msgr2=0x7f57840b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57700039b0 0x7f57840b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 -- 172.21.15.157:0/4063492666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578415eae0 con 0x7f57840c8ca0 2023-11-06T03:08:44.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a9f7640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f57840cb590 tx=0x7f577c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c017400 con 0x7f57840c8ca0 2023-11-06T03:08:44.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c011040 con 0x7f57840c8ca0 2023-11-06T03:08:44.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578415edc0 con 0x7f57840c8ca0 2023-11-06T03:08:44.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c0177e0 con 0x7f57840c8ca0 2023-11-06T03:08:44.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57840c6e30 con 0x7f57840c8ca0 2023-11-06T03:08:44.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57840c70b0 con 0x7f57840c8ca0 2023-11-06T03:08:44.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f577c017940 con 0x7f57840c8ca0 2023-11-06T03:08:44.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 0x7f5768067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f577c083750 con 0x7f57840c8ca0 2023-11-06T03:08:44.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.104+0000 7f578a1f6640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 0x7f5768067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.108+0000 7f578a1f6640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 0x7f5768067f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57800ab000 tx=0x7f57800a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.108+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f577c050fb0 con 0x7f57840c8ca0 2023-11-06T03:08:44.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.296+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f578415efb0 con 0x7f57840c8ca0 2023-11-06T03:08:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f577affd640 1 -- 172.21.15.157:0/4063492666 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f577c0545d0 con 0x7f57840c8ca0 2023-11-06T03:08:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 msgr2=0x7f5768067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 0x7f5768067f20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57800ab000 tx=0x7f57800a7070 comp rx=0 tx=0).stop 2023-11-06T03:08:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 msgr2=0x7f578415f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f57840cb590 tx=0x7f577c00f4a0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 shutdown_connections 2023-11-06T03:08:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5768065a60 0x7f5768067f20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57840c8ca0 0x7f578415f2e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57840c69d0 0x7f57840c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 --2- 172.21.15.157:0/4063492666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57700039b0 0x7f57840b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 >> 172.21.15.157:0/4063492666 conn(0x7f57840cba30 msgr2=0x7f578406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 shutdown_connections 2023-11-06T03:08:44.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.312+0000 7f578c45a640 1 -- 172.21.15.157:0/4063492666 wait complete. 2023-11-06T03:08:44.330 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:08:44.331 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:08:44.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 Processor -- start 2023-11-06T03:08:44.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 -- start start 2023-11-06T03:08:44.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9df41551e0 0x7f9df406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 0x7f9df415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df415f580 con 0x7f9df415e5e0 2023-11-06T03:08:44.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df415efc0 con 0x7f9df41551e0 2023-11-06T03:08:44.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9dfa9a4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df415f2a0 con 0x7f9df40b3b20 2023-11-06T03:08:44.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df3fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9df41551e0 0x7f9df406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df8f41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 0x7f9df415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df37fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f9df415e5e0 0x7f9df40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52804/0 (socket says 172.21.15.157:52804) 2023-11-06T03:08:44.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df8f41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 0x7f9df415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:52796/0 (socket says 172.21.15.157:52796) 2023-11-06T03:08:44.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df37fe640 1 -- 172.21.15.157:0/3439681861 learned_addr learned my addr 172.21.15.157:0/3439681861 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:44.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df3fff640 1 -- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 msgr2=0x7f9df415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:44.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df3fff640 1 --2- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 0x7f9df415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df3fff640 1 -- 172.21.15.157:0/3439681861 >> v1:172.21.15.195:6789/0 conn(0x7f9df415e5e0 legacy=0x7f9df40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.452+0000 7f9df3fff640 1 -- 172.21.15.157:0/3439681861 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df406f5a0 con 0x7f9df41551e0 2023-11-06T03:08:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9df3fff640 1 --2- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df41551e0 0x7f9df406fda0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9de40097c0 tx=0x7f9de401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af4e379db3146d09 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9df2ffd640 1 -- 172.21.15.157:0/3439681861 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40ac640 con 0x7f9df41551e0 2023-11-06T03:08:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9df2ffd640 1 -- 172.21.15.157:0/3439681861 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de40ac7a0 con 0x7f9df41551e0 2023-11-06T03:08:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9df2ffd640 1 -- 172.21.15.157:0/3439681861 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40acb30 con 0x7f9df41551e0 2023-11-06T03:08:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df41551e0 msgr2=0x7f9df406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df41551e0 0x7f9df406fda0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9de40097c0 tx=0x7f9de401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3439681861 shutdown_connections 2023-11-06T03:08:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df40b3b20 0x7f9df415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3439681861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df41551e0 0x7f9df406fda0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3439681861 >> 172.21.15.157:0/3439681861 conn(0x7f9df4065930 msgr2=0x7f9df4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3439681861 shutdown_connections 2023-11-06T03:08:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.456+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3439681861 wait complete. 2023-11-06T03:08:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 Processor -- start 2023-11-06T03:08:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- start start 2023-11-06T03:08:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 0x7f9df4147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 0x7f9df4159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df415dd50 con 0x7f9df40b3b20 2023-11-06T03:08:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df41467c0 con 0x7f9df415e5e0 2023-11-06T03:08:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4146aa0 con 0x7f9df41551e0 2023-11-06T03:08:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57862/0 (socket says 172.21.15.157:57862) 2023-11-06T03:08:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 -- 172.21.15.157:0/3307294991 learned_addr learned my addr 172.21.15.157:0/3307294991 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df8f41640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 0x7f9df4159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df37fe640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 0x7f9df4147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 -- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 msgr2=0x7f9df4147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 0x7f9df4147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 -- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 msgr2=0x7f9df4159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:44.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 0x7f9df4159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:44.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 -- 172.21.15.157:0/3307294991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009420 con 0x7f9df40b3b20 2023-11-06T03:08:44.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df37fe640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 0x7f9df4147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:44.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df8f41640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 0x7f9df4159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:44.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df3fff640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9de4019ff0 tx=0x7f9de401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40b8650 con 0x7f9df40b3b20 2023-11-06T03:08:44.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de40b87b0 con 0x7f9df40b3b20 2023-11-06T03:08:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df4146d80 con 0x7f9df40b3b20 2023-11-06T03:08:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40b8b60 con 0x7f9df40b3b20 2023-11-06T03:08:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df4146470 con 0x7f9df40b3b20 2023-11-06T03:08:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df4072750 con 0x7f9df40b3b20 2023-11-06T03:08:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.460+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9de40b8cc0 con 0x7f9df40b3b20 2023-11-06T03:08:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.464+0000 7f9df0ff9640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 0x7f9de0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.464+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9de411e160 con 0x7f9df40b3b20 2023-11-06T03:08:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.464+0000 7f9df37fe640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 0x7f9de0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.464+0000 7f9df37fe640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 0x7f9de0067f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df406d670 tx=0x7f9de8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:44.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.464+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9de40ebb30 con 0x7f9df40b3b20 2023-11-06T03:08:44.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:44.656+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9df4147230 con 0x7f9df40b3b20 2023-11-06T03:08:44.846 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:08:44.848+0000 7fe26894b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:08:45.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.324+0000 7f9df0ff9640 1 -- 172.21.15.157:0/3307294991 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9de40ef150 con 0x7f9df40b3b20 2023-11-06T03:08:45.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 msgr2=0x7f9de0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 0x7f9de0067f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df406d670 tx=0x7f9de8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 msgr2=0x7f9df415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9de4019ff0 tx=0x7f9de401ac00 comp rx=0 tx=0).stop 2023-11-06T03:08:45.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 shutdown_connections 2023-11-06T03:08:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9de0065a60 0x7f9de0067f20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9df415e5e0 0x7f9df4159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9df41551e0 0x7f9df4147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 --2- 172.21.15.157:0/3307294991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df40b3b20 0x7f9df415d810 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 >> 172.21.15.157:0/3307294991 conn(0x7f9df4065930 msgr2=0x7f9df4151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 shutdown_connections 2023-11-06T03:08:45.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.328+0000 7f9dfa9a4640 1 -- 172.21.15.157:0/3307294991 wait complete. 2023-11-06T03:08:45.348 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:08:45.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:45.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.472+0000 7fc3b520c640 1 Processor -- start 2023-11-06T03:08:45.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.472+0000 7fc3b520c640 1 -- start start 2023-11-06T03:08:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3b00ce800 0x7fc3b00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00cf1a0 con 0x7fc3b0146370 2023-11-06T03:08:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b00b2120 con 0x7fc3b00ce800 2023-11-06T03:08:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b006c3d0 con 0x7fc3b006b6a0 2023-11-06T03:08:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3aed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc3b0146370 0x7fc3b00b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53190/0 (socket says 172.21.15.157:53190) 2023-11-06T03:08:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3aed9d640 1 -- 172.21.15.157:0/991089052 learned_addr learned my addr 172.21.15.157:0/991089052 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:45.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/991089052 >> v2:172.21.15.157:3300/0 conn(0x7fc3b00ce800 0x7fc3b00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3af59e640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 -- 172.21.15.157:0/991089052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 msgr2=0x7fc3b006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b006bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 -- 172.21.15.157:0/991089052 >> v1:172.21.15.157:6789/0 conn(0x7fc3b0146370 legacy=0x7fc3b00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 -- 172.21.15.157:0/991089052 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b006f5a0 con 0x7fc3b00ce800 2023-11-06T03:08:45.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3af59e640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:45.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b00cec60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097c0 tx=0x7fc3a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4280daa72a6588 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3add9b640 1 -- 172.21.15.157:0/991089052 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40ac620 con 0x7fc3b00ce800 2023-11-06T03:08:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3add9b640 1 -- 172.21.15.157:0/991089052 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40ac780 con 0x7fc3b00ce800 2023-11-06T03:08:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3add9b640 1 -- 172.21.15.157:0/991089052 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40acb10 con 0x7fc3b00ce800 2023-11-06T03:08:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 -- 172.21.15.157:0/991089052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 msgr2=0x7fc3b00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.476+0000 7fc3b520c640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b00cec60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097c0 tx=0x7fc3a401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/991089052 shutdown_connections 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 --2- 172.21.15.157:0/991089052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b00cec60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/991089052 >> 172.21.15.157:0/991089052 conn(0x7fc3b0065930 msgr2=0x7fc3b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/991089052 shutdown_connections 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/991089052 wait complete. 2023-11-06T03:08:45.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 Processor -- start 2023-11-06T03:08:45.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- start start 2023-11-06T03:08:45.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b013a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b0146370 0x7fc3b0144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b013a700 con 0x7fc3b00ce800 2023-11-06T03:08:45.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01564d0 con 0x7fc3b0146370 2023-11-06T03:08:45.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01567b0 con 0x7fc3b006b6a0 2023-11-06T03:08:45.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57888/0 (socket says 172.21.15.157:57888) 2023-11-06T03:08:45.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 -- 172.21.15.157:0/3661083214 learned_addr learned my addr 172.21.15.157:0/3661083214 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:45.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3aed9d640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b013a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 -- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 msgr2=0x7fc3b013a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3af59e640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b0146370 0x7fc3b0144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b013a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 -- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b0146370 msgr2=0x7fc3b0144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b0146370 0x7fc3b0144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 -- 172.21.15.157:0/3661083214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009420 con 0x7fc3b00ce800 2023-11-06T03:08:45.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3ae59c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40af040 tx=0x7fc3a4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40b5610 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40b5790 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40b5b40 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b0156a90 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b0144e20 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.480+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b0072750 con 0x7fc3b00ce800 2023-11-06T03:08:45.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.484+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc3a40b3040 con 0x7fc3b00ce800 2023-11-06T03:08:45.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.484+0000 7fc3977fe640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 0x7fc39c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.484+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3a411d4d0 con 0x7fc3b00ce800 2023-11-06T03:08:45.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.484+0000 7fc3aed9d640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 0x7fc39c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.484+0000 7fc3aed9d640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 0x7fc39c067f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8005890 tx=0x7fc3a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.488+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc3a40ead30 con 0x7fc3b00ce800 2023-11-06T03:08:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.676+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3b0065fb0 con 0x7fc3b00ce800 2023-11-06T03:08:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.676+0000 7fc3977fe640 1 -- 172.21.15.157:0/3661083214 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fc3a40ac5e0 con 0x7fc3b00ce800 2023-11-06T03:08:45.678 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:45.678 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2083483222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2083483222},{"type":"v1","addr":"172.21.15.195:6835","nonce":2083483222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5752,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3999445109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3999445109},{"type":"v1","addr":"172.21.15.195:6837","nonce":3999445109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1572709083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1572709083},{"type":"v1","addr":"172.21.15.157:6833","nonce":1572709083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:08:45.318890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1218323165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1218323165},{"type":"v1","addr":"172.21.15.157:6835","nonce":1218323165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:08:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 47 2023-11-06T03:08:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 msgr2=0x7fc39c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 0x7fc39c067f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8005890 tx=0x7fc3a8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 msgr2=0x7fc3b01583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40af040 tx=0x7fc3a4019160 comp rx=0 tx=0).stop 2023-11-06T03:08:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 shutdown_connections 2023-11-06T03:08:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc39c065a60 0x7fc39c067f20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3b0146370 0x7fc3b0144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3b00ce800 0x7fc3b01583b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 --2- 172.21.15.157:0/3661083214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3b006b6a0 0x7fc3b013a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 >> 172.21.15.157:0/3661083214 conn(0x7fc3b0065930 msgr2=0x7fc3b00b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 shutdown_connections 2023-11-06T03:08:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.680+0000 7fc3b520c640 1 -- 172.21.15.157:0/3661083214 wait complete. 2023-11-06T03:08:45.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.812+0000 7ffb4cc44640 1 Processor -- start 2023-11-06T03:08:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.812+0000 7ffb4cc44640 1 -- start start 2023-11-06T03:08:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4cc44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb481551e0 0x7ffb4806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4cc44640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4815f580 con 0x7ffb480b3b20 2023-11-06T03:08:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4cc44640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4815efc0 con 0x7ffb4815e5e0 2023-11-06T03:08:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4cc44640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4815f2a0 con 0x7ffb481551e0 2023-11-06T03:08:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb46d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffb480b3b20 0x7ffb4815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53200/0 (socket says 172.21.15.157:53200) 2023-11-06T03:08:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb46d9d640 1 -- 172.21.15.157:0/838619732 learned_addr learned my addr 172.21.15.157:0/838619732 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:45.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4659c640 1 --2- 172.21.15.157:0/838619732 >> v2:172.21.15.157:3300/0 conn(0x7ffb481551e0 0x7ffb4806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4559a640 1 -- 172.21.15.157:0/838619732 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729094512 0 0) 0x7ffb4815f580 con 0x7ffb480b3b20 2023-11-06T03:08:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4559a640 1 -- 172.21.15.157:0/838619732 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003530 con 0x7ffb480b3b20 2023-11-06T03:08:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4659c640 1 -- 172.21.15.157:0/838619732 >> v1:172.21.15.195:6789/0 conn(0x7ffb4815e5e0 legacy=0x7ffb480b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:45.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4659c640 1 -- 172.21.15.157:0/838619732 >> v1:172.21.15.157:6789/0 conn(0x7ffb480b3b20 legacy=0x7ffb4815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4659c640 1 -- 172.21.15.157:0/838619732 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4806f5a0 con 0x7ffb481551e0 2023-11-06T03:08:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4659c640 1 --2- 172.21.15.157:0/838619732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb481551e0 0x7ffb4806fda0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3801c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de090e44d2cce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4559a640 1 -- 172.21.15.157:0/838619732 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380a58a0 con 0x7ffb481551e0 2023-11-06T03:08:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4559a640 1 -- 172.21.15.157:0/838619732 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380a5a00 con 0x7ffb481551e0 2023-11-06T03:08:45.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.816+0000 7ffb4559a640 1 -- 172.21.15.157:0/838619732 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380a5d70 con 0x7ffb481551e0 2023-11-06T03:08:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/838619732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb481551e0 msgr2=0x7ffb4806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/838619732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb481551e0 0x7ffb4806fda0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3801c7c0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/838619732 shutdown_connections 2023-11-06T03:08:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/838619732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb481551e0 0x7ffb4806fda0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/838619732 >> 172.21.15.157:0/838619732 conn(0x7ffb48065930 msgr2=0x7ffb48065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:45.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/838619732 shutdown_connections 2023-11-06T03:08:45.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/838619732 wait complete. 2023-11-06T03:08:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 Processor -- start 2023-11-06T03:08:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- start start 2023-11-06T03:08:45.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb481551e0 0x7ffb48159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb4815e5e0 0x7ffb4814bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4806b490 con 0x7ffb480b3b20 2023-11-06T03:08:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480cdc00 con 0x7ffb481551e0 2023-11-06T03:08:45.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48151a00 con 0x7ffb4815e5e0 2023-11-06T03:08:45.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57916/0 (socket says 172.21.15.157:57916) 2023-11-06T03:08:45.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 -- 172.21.15.157:0/4287503758 learned_addr learned my addr 172.21.15.157:0/4287503758 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:45.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 -- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb4815e5e0 msgr2=0x7ffb4814bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb4815e5e0 0x7ffb4814bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 -- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb481551e0 msgr2=0x7ffb48159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb481551e0 0x7ffb48159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 -- 172.21.15.157:0/4287503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009a70 con 0x7ffb480b3b20 2023-11-06T03:08:45.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4659c640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb38019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380a58a0 con 0x7ffb480b3b20 2023-11-06T03:08:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380ad850 con 0x7ffb480b3b20 2023-11-06T03:08:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48151cc0 con 0x7ffb480b3b20 2023-11-06T03:08:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380adbe0 con 0x7ffb480b3b20 2023-11-06T03:08:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb48152280 con 0x7ffb480b3b20 2023-11-06T03:08:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.820+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb48070a80 con 0x7ffb480b3b20 2023-11-06T03:08:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.824+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffb380b4070 con 0x7ffb480b3b20 2023-11-06T03:08:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.824+0000 7ffb3effd640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 0x7ffb28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.824+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb3811e430 con 0x7ffb480b3b20 2023-11-06T03:08:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.824+0000 7ffb45d9b640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 0x7ffb28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.824+0000 7ffb45d9b640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 0x7ffb28067f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffb48150120 tx=0x7ffb30006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:45.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.828+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffb380ebc90 con 0x7ffb480b3b20 2023-11-06T03:08:45.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.984+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb48152470 con 0x7ffb480b3b20 2023-11-06T03:08:45.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.984+0000 7ffb3effd640 1 -- 172.21.15.157:0/4287503758 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17873 (secure 0 0 0) 0x7ffb380ef2b0 con 0x7ffb480b3b20 2023-11-06T03:08:45.985 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:45.986 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":67,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:44.309374+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 msgr2=0x7ffb28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 0x7ffb28067f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffb48150120 tx=0x7ffb30006bd0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 msgr2=0x7ffb480cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:45.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb38019560 comp rx=0 tx=0).stop 2023-11-06T03:08:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 shutdown_connections 2023-11-06T03:08:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb28065a60 0x7ffb28067f20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb4815e5e0 0x7ffb4814bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb481551e0 0x7ffb48159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 --2- 172.21.15.157:0/4287503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb480b3b20 0x7ffb480cd6c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 >> 172.21.15.157:0/4287503758 conn(0x7ffb48065930 msgr2=0x7ffb480af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 shutdown_connections 2023-11-06T03:08:45.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:45.988+0000 7ffb4cc44640 1 -- 172.21.15.157:0/4287503758 wait complete. 2023-11-06T03:08:46.005 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:08:46.005 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:08:46.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.120+0000 7f6c85edd640 1 Processor -- start 2023-11-06T03:08:46.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.120+0000 7f6c85edd640 1 -- start start 2023-11-06T03:08:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c85edd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c8015e5e0 0x7f6c800b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c85edd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8015f580 con 0x7f6c8015ea90 2023-11-06T03:08:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c85edd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8015efc0 con 0x7f6c801551e0 2023-11-06T03:08:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c85edd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c800704b0 con 0x7f6c8015e5e0 2023-11-06T03:08:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c76ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c8015e5e0 0x7f6c800b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6c8015ea90 0x7f6c8015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53202/0 (socket says 172.21.15.157:53202) 2023-11-06T03:08:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7ffff640 1 -- 172.21.15.157:0/2918000145 learned_addr learned my addr 172.21.15.157:0/2918000145 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c76ffd640 1 -- 172.21.15.157:0/2918000145 >> v1:172.21.15.195:6789/0 conn(0x7f6c801551e0 legacy=0x7f6c8006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7effd640 1 -- 172.21.15.157:0/2918000145 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648580940 0 0) 0x7f6c8015f580 con 0x7f6c8015ea90 2023-11-06T03:08:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c76ffd640 1 -- 172.21.15.157:0/2918000145 >> v1:172.21.15.157:6789/0 conn(0x7f6c8015ea90 legacy=0x7f6c8015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c76ffd640 1 -- 172.21.15.157:0/2918000145 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8015f2a0 con 0x7f6c8015e5e0 2023-11-06T03:08:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c76ffd640 1 --2- 172.21.15.157:0/2918000145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8015e5e0 0x7f6c800b3410 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6c680097c0 tx=0x7f6c6801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed21354c89f6d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7effd640 1 -- 172.21.15.157:0/2918000145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680ab600 con 0x7f6c8015e5e0 2023-11-06T03:08:46.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7effd640 1 -- 172.21.15.157:0/2918000145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c680a7050 con 0x7f6c8015e5e0 2023-11-06T03:08:46.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.124+0000 7f6c7effd640 1 -- 172.21.15.157:0/2918000145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680ab980 con 0x7f6c8015e5e0 2023-11-06T03:08:46.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2918000145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8015e5e0 msgr2=0x7f6c800b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2918000145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8015e5e0 0x7f6c800b3410 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6c680097c0 tx=0x7f6c6801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2918000145 shutdown_connections 2023-11-06T03:08:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2918000145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c8015e5e0 0x7f6c800b3410 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2918000145 >> 172.21.15.157:0/2918000145 conn(0x7f6c80065930 msgr2=0x7f6c80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2918000145 shutdown_connections 2023-11-06T03:08:46.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2918000145 wait complete. 2023-11-06T03:08:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 Processor -- start 2023-11-06T03:08:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- start start 2023-11-06T03:08:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8015e5e0 0x7f6c800cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c8015ea90 0x7f6c8014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801517f0 con 0x7f6c801551e0 2023-11-06T03:08:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8014d230 con 0x7f6c8015e5e0 2023-11-06T03:08:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80151c80 con 0x7f6c8015ea90 2023-11-06T03:08:46.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57938/0 (socket says 172.21.15.157:57938) 2023-11-06T03:08:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 -- 172.21.15.157:0/2468326649 learned_addr learned my addr 172.21.15.157:0/2468326649 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 -- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c8015ea90 msgr2=0x7f6c8014e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c8015ea90 0x7f6c8014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 -- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8015e5e0 msgr2=0x7f6c800cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8015e5e0 0x7f6c800cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 -- 172.21.15.157:0/2468326649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68009420 con 0x7f6c801551e0 2023-11-06T03:08:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7f7fe640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c530 tx=0x7f6c7000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70005220 con 0x7f6c801551e0 2023-11-06T03:08:46.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70005380 con 0x7f6c801551e0 2023-11-06T03:08:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c80150be0 con 0x7f6c801551e0 2023-11-06T03:08:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70007270 con 0x7f6c801551e0 2023-11-06T03:08:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c801511a0 con 0x7f6c801551e0 2023-11-06T03:08:46.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.128+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c80072750 con 0x7f6c801551e0 2023-11-06T03:08:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.132+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c700073d0 con 0x7f6c801551e0 2023-11-06T03:08:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.132+0000 7f6c7cff9640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 0x7f6c60067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.132+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c70082b70 con 0x7f6c801551e0 2023-11-06T03:08:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.132+0000 7f6c76ffd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 0x7f6c60067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.132+0000 7f6c76ffd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 0x7f6c60067f50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ad000 tx=0x7f6c68004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.136+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c70050460 con 0x7f6c801551e0 2023-11-06T03:08:46.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.280+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c800b3d60 con 0x7f6c60065a90 2023-11-06T03:08:46.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.308+0000 7f6c7cff9640 1 -- 172.21.15.157:0/2468326649 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310300 (secure 0 0 0) 0x7f6c680aa3a0 con 0x7f6c60065a90 2023-11-06T03:08:46.309 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:46.311 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:08:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 msgr2=0x7f6c60067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 0x7f6c60067f50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ad000 tx=0x7f6c68004740 comp rx=0 tx=0).stop 2023-11-06T03:08:46.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 msgr2=0x7f6c8014ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c530 tx=0x7f6c7000c9f0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 shutdown_connections 2023-11-06T03:08:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c60065a90 0x7f6c60067f50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c8015ea90 0x7f6c8014e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8015e5e0 0x7f6c800cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 --2- 172.21.15.157:0/2468326649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c801551e0 0x7f6c8014ccf0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 >> 172.21.15.157:0/2468326649 conn(0x7f6c80065930 msgr2=0x7f6c8015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 shutdown_connections 2023-11-06T03:08:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.312+0000 7f6c85edd640 1 -- 172.21.15.157:0/2468326649 wait complete. 2023-11-06T03:08:46.328 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2023-11-06T03:08:45.195802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":359956,"kb_used_data":7380,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754614764,"statfs":{"total":773094113280,"available":772725518336,"internally_reserved":0,"allocated":7557120,"data_stored":3982793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001128"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371764+0000","last_change":"2023-11-06T03:08:43.371377+0000","last_active":"2023-11-06T03:08:43.371764+0000","last_peered":"2023-11-06T03:08:43.371764+0000","last_clean":"2023-11-06T03:08:43.371764+0000","last_became_active":"2023-11-06T03:08:43.371095+0000","last_became_peered":"2023-11-06T03:08:43.371095+0000","last_unstale":"2023-11-06T03:08:43.371764+0000","last_undegraded":"2023-11-06T03:08:43.371764+0000","last_fullsized":"2023-11-06T03:08:43.371764+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.320147+0000","last_change":"2023-11-06T03:08:43.318920+0000","last_active":"2023-11-06T03:08:43.320147+0000","last_peered":"2023-11-06T03:08:43.320147+0000","last_clean":"2023-11-06T03:08:43.320147+0000","last_became_active":"2023-11-06T03:08:43.318555+0000","last_became_peered":"2023-11-06T03:08:43.318555+0000","last_unstale":"2023-11-06T03:08:43.320147+0000","last_undegraded":"2023-11-06T03:08:43.320147+0000","last_fullsized":"2023-11-06T03:08:43.320147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376771+0000","last_change":"2023-11-06T03:08:43.372054+0000","last_active":"2023-11-06T03:08:43.376771+0000","last_peered":"2023-11-06T03:08:43.376771+0000","last_clean":"2023-11-06T03:08:43.376771+0000","last_became_active":"2023-11-06T03:08:43.371474+0000","last_became_peered":"2023-11-06T03:08:43.371474+0000","last_unstale":"2023-11-06T03:08:43.376771+0000","last_undegraded":"2023-11-06T03:08:43.376771+0000","last_fullsized":"2023-11-06T03:08:43.376771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:54.953106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379121+0000","last_change":"2023-11-06T03:08:43.356227+0000","last_active":"2023-11-06T03:08:43.379121+0000","last_peered":"2023-11-06T03:08:43.379121+0000","last_clean":"2023-11-06T03:08:43.379121+0000","last_became_active":"2023-11-06T03:08:43.350719+0000","last_became_peered":"2023-11-06T03:08:43.350719+0000","last_unstale":"2023-11-06T03:08:43.379121+0000","last_undegraded":"2023-11-06T03:08:43.379121+0000","last_fullsized":"2023-11-06T03:08:43.379121+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:52.709530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380794+0000","last_change":"2023-11-06T03:08:43.373777+0000","last_active":"2023-11-06T03:08:43.380794+0000","last_peered":"2023-11-06T03:08:43.380794+0000","last_clean":"2023-11-06T03:08:43.380794+0000","last_became_active":"2023-11-06T03:08:43.373068+0000","last_became_peered":"2023-11-06T03:08:43.373068+0000","last_unstale":"2023-11-06T03:08:43.380794+0000","last_undegraded":"2023-11-06T03:08:43.380794+0000","last_fullsized":"2023-11-06T03:08:43.380794+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:48:43.530094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365755+0000","last_change":"2023-11-06T03:08:43.365209+0000","last_active":"2023-11-06T03:08:43.365755+0000","last_peered":"2023-11-06T03:08:43.365755+0000","last_clean":"2023-11-06T03:08:43.365755+0000","last_became_active":"2023-11-06T03:08:43.364872+0000","last_became_peered":"2023-11-06T03:08:43.364872+0000","last_unstale":"2023-11-06T03:08:43.365755+0000","last_undegraded":"2023-11-06T03:08:43.365755+0000","last_fullsized":"2023-11-06T03:08:43.365755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:02:20.030442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364413+0000","last_change":"2023-11-06T03:08:43.363807+0000","last_active":"2023-11-06T03:08:43.364413+0000","last_peered":"2023-11-06T03:08:43.364413+0000","last_clean":"2023-11-06T03:08:43.364413+0000","last_became_active":"2023-11-06T03:08:43.363455+0000","last_became_peered":"2023-11-06T03:08:43.363455+0000","last_unstale":"2023-11-06T03:08:43.364413+0000","last_undegraded":"2023-11-06T03:08:43.364413+0000","last_fullsized":"2023-11-06T03:08:43.364413+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:35.779457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384563+0000","last_change":"2023-11-06T03:08:43.376870+0000","last_active":"2023-11-06T03:08:43.384563+0000","last_peered":"2023-11-06T03:08:43.384563+0000","last_clean":"2023-11-06T03:08:43.384563+0000","last_became_active":"2023-11-06T03:08:43.376282+0000","last_became_peered":"2023-11-06T03:08:43.376282+0000","last_unstale":"2023-11-06T03:08:43.384563+0000","last_undegraded":"2023-11-06T03:08:43.384563+0000","last_fullsized":"2023-11-06T03:08:43.384563+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:32:17.267201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.381164+0000","last_change":"2023-11-06T03:08:43.378404+0000","last_active":"2023-11-06T03:08:43.381164+0000","last_peered":"2023-11-06T03:08:43.381164+0000","last_clean":"2023-11-06T03:08:43.381164+0000","last_became_active":"2023-11-06T03:08:43.377698+0000","last_became_peered":"2023-11-06T03:08:43.377698+0000","last_unstale":"2023-11-06T03:08:43.381164+0000","last_undegraded":"2023-11-06T03:08:43.381164+0000","last_fullsized":"2023-11-06T03:08:43.381164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:58.576582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379028+0000","last_change":"2023-11-06T03:08:43.374077+0000","last_active":"2023-11-06T03:08:43.379028+0000","last_peered":"2023-11-06T03:08:43.379028+0000","last_clean":"2023-11-06T03:08:43.379028+0000","last_became_active":"2023-11-06T03:08:43.372782+0000","last_became_peered":"2023-11-06T03:08:43.372782+0000","last_unstale":"2023-11-06T03:08:43.379028+0000","last_undegraded":"2023-11-06T03:08:43.379028+0000","last_fullsized":"2023-11-06T03:08:43.379028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374741+0000","last_change":"2023-11-06T03:08:43.350306+0000","last_active":"2023-11-06T03:08:43.374741+0000","last_peered":"2023-11-06T03:08:43.374741+0000","last_clean":"2023-11-06T03:08:43.374741+0000","last_became_active":"2023-11-06T03:08:43.343842+0000","last_became_peered":"2023-11-06T03:08:43.343842+0000","last_unstale":"2023-11-06T03:08:43.374741+0000","last_undegraded":"2023-11-06T03:08:43.374741+0000","last_fullsized":"2023-11-06T03:08:43.374741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384491+0000","last_change":"2023-11-06T03:08:43.373110+0000","last_active":"2023-11-06T03:08:43.384491+0000","last_peered":"2023-11-06T03:08:43.384491+0000","last_clean":"2023-11-06T03:08:43.384491+0000","last_became_active":"2023-11-06T03:08:43.372255+0000","last_became_peered":"2023-11-06T03:08:43.372255+0000","last_unstale":"2023-11-06T03:08:43.384491+0000","last_undegraded":"2023-11-06T03:08:43.384491+0000","last_fullsized":"2023-11-06T03:08:43.384491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:54.953106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382957+0000","last_change":"2023-11-06T03:08:43.368957+0000","last_active":"2023-11-06T03:08:43.382957+0000","last_peered":"2023-11-06T03:08:43.382957+0000","last_clean":"2023-11-06T03:08:43.382957+0000","last_became_active":"2023-11-06T03:08:43.368357+0000","last_became_peered":"2023-11-06T03:08:43.368357+0000","last_unstale":"2023-11-06T03:08:43.382957+0000","last_undegraded":"2023-11-06T03:08:43.382957+0000","last_fullsized":"2023-11-06T03:08:43.382957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376823+0000","last_change":"2023-11-06T03:08:43.371997+0000","last_active":"2023-11-06T03:08:43.376823+0000","last_peered":"2023-11-06T03:08:43.376823+0000","last_clean":"2023-11-06T03:08:43.376823+0000","last_became_active":"2023-11-06T03:08:43.370023+0000","last_became_peered":"2023-11-06T03:08:43.370023+0000","last_unstale":"2023-11-06T03:08:43.376823+0000","last_undegraded":"2023-11-06T03:08:43.376823+0000","last_fullsized":"2023-11-06T03:08:43.376823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376630+0000","last_change":"2023-11-06T03:08:43.371385+0000","last_active":"2023-11-06T03:08:43.376630+0000","last_peered":"2023-11-06T03:08:43.376630+0000","last_clean":"2023-11-06T03:08:43.376630+0000","last_became_active":"2023-11-06T03:08:43.370680+0000","last_became_peered":"2023-11-06T03:08:43.370680+0000","last_unstale":"2023-11-06T03:08:43.376630+0000","last_undegraded":"2023-11-06T03:08:43.376630+0000","last_fullsized":"2023-11-06T03:08:43.376630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.369874+0000","last_change":"2023-11-06T03:08:43.369401+0000","last_active":"2023-11-06T03:08:43.369874+0000","last_peered":"2023-11-06T03:08:43.369874+0000","last_clean":"2023-11-06T03:08:43.369874+0000","last_became_active":"2023-11-06T03:08:43.369051+0000","last_became_peered":"2023-11-06T03:08:43.369051+0000","last_unstale":"2023-11-06T03:08:43.369874+0000","last_undegraded":"2023-11-06T03:08:43.369874+0000","last_fullsized":"2023-11-06T03:08:43.369874+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:47:02.035595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.344356+0000","last_change":"2023-11-06T03:08:43.343583+0000","last_active":"2023-11-06T03:08:43.344356+0000","last_peered":"2023-11-06T03:08:43.344356+0000","last_clean":"2023-11-06T03:08:43.344356+0000","last_became_active":"2023-11-06T03:08:43.343309+0000","last_became_peered":"2023-11-06T03:08:43.343309+0000","last_unstale":"2023-11-06T03:08:43.344356+0000","last_undegraded":"2023-11-06T03:08:43.344356+0000","last_fullsized":"2023-11-06T03:08:43.344356+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374469+0000","last_change":"2023-11-06T03:08:43.344995+0000","last_active":"2023-11-06T03:08:43.374469+0000","last_peered":"2023-11-06T03:08:43.374469+0000","last_clean":"2023-11-06T03:08:43.374469+0000","last_became_active":"2023-11-06T03:08:43.344192+0000","last_became_peered":"2023-11-06T03:08:43.344192+0000","last_unstale":"2023-11-06T03:08:43.374469+0000","last_undegraded":"2023-11-06T03:08:43.374469+0000","last_fullsized":"2023-11-06T03:08:43.374469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:05:04.920453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364584+0000","last_change":"2023-11-06T03:08:43.352964+0000","last_active":"2023-11-06T03:08:43.364584+0000","last_peered":"2023-11-06T03:08:43.364584+0000","last_clean":"2023-11-06T03:08:43.364584+0000","last_became_active":"2023-11-06T03:08:43.352197+0000","last_became_peered":"2023-11-06T03:08:43.352197+0000","last_unstale":"2023-11-06T03:08:43.364584+0000","last_undegraded":"2023-11-06T03:08:43.364584+0000","last_fullsized":"2023-11-06T03:08:43.364584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:23:47.479034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357713+0000","last_change":"2023-11-06T03:08:43.356614+0000","last_active":"2023-11-06T03:08:43.357713+0000","last_peered":"2023-11-06T03:08:43.357713+0000","last_clean":"2023-11-06T03:08:43.357713+0000","last_became_active":"2023-11-06T03:08:43.356324+0000","last_became_peered":"2023-11-06T03:08:43.356324+0000","last_unstale":"2023-11-06T03:08:43.357713+0000","last_undegraded":"2023-11-06T03:08:43.357713+0000","last_fullsized":"2023-11-06T03:08:43.357713+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:55.634682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382916+0000","last_change":"2023-11-06T03:08:43.354883+0000","last_active":"2023-11-06T03:08:43.382916+0000","last_peered":"2023-11-06T03:08:43.382916+0000","last_clean":"2023-11-06T03:08:43.382916+0000","last_became_active":"2023-11-06T03:08:43.349971+0000","last_became_peered":"2023-11-06T03:08:43.349971+0000","last_unstale":"2023-11-06T03:08:43.382916+0000","last_undegraded":"2023-11-06T03:08:43.382916+0000","last_fullsized":"2023-11-06T03:08:43.382916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377689+0000","last_change":"2023-11-06T03:08:43.377192+0000","last_active":"2023-11-06T03:08:43.377689+0000","last_peered":"2023-11-06T03:08:43.377689+0000","last_clean":"2023-11-06T03:08:43.377689+0000","last_became_active":"2023-11-06T03:08:43.376809+0000","last_became_peered":"2023-11-06T03:08:43.376809+0000","last_unstale":"2023-11-06T03:08:43.377689+0000","last_undegraded":"2023-11-06T03:08:43.377689+0000","last_fullsized":"2023-11-06T03:08:43.377689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:22:31.965269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.363908+0000","last_change":"2023-11-06T03:08:43.363409+0000","last_active":"2023-11-06T03:08:43.363908+0000","last_peered":"2023-11-06T03:08:43.363908+0000","last_clean":"2023-11-06T03:08:43.363908+0000","last_became_active":"2023-11-06T03:08:43.362850+0000","last_became_peered":"2023-11-06T03:08:43.362850+0000","last_unstale":"2023-11-06T03:08:43.363908+0000","last_undegraded":"2023-11-06T03:08:43.363908+0000","last_fullsized":"2023-11-06T03:08:43.363908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:51.025831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.372914+0000","last_change":"2023-11-06T03:08:43.363939+0000","last_active":"2023-11-06T03:08:43.372914+0000","last_peered":"2023-11-06T03:08:43.372914+0000","last_clean":"2023-11-06T03:08:43.372914+0000","last_became_active":"2023-11-06T03:08:43.363104+0000","last_became_peered":"2023-11-06T03:08:43.363104+0000","last_unstale":"2023-11-06T03:08:43.372914+0000","last_undegraded":"2023-11-06T03:08:43.372914+0000","last_fullsized":"2023-11-06T03:08:43.372914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:12.711490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383077+0000","last_change":"2023-11-06T03:08:43.370350+0000","last_active":"2023-11-06T03:08:43.383077+0000","last_peered":"2023-11-06T03:08:43.383077+0000","last_clean":"2023-11-06T03:08:43.383077+0000","last_became_active":"2023-11-06T03:08:43.369766+0000","last_became_peered":"2023-11-06T03:08:43.369766+0000","last_unstale":"2023-11-06T03:08:43.383077+0000","last_undegraded":"2023-11-06T03:08:43.383077+0000","last_fullsized":"2023-11-06T03:08:43.383077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383070+0000","last_change":"2023-11-06T03:08:43.370361+0000","last_active":"2023-11-06T03:08:43.383070+0000","last_peered":"2023-11-06T03:08:43.383070+0000","last_clean":"2023-11-06T03:08:43.383070+0000","last_became_active":"2023-11-06T03:08:43.369251+0000","last_became_peered":"2023-11-06T03:08:43.369251+0000","last_unstale":"2023-11-06T03:08:43.383070+0000","last_undegraded":"2023-11-06T03:08:43.383070+0000","last_fullsized":"2023-11-06T03:08:43.383070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:32.425326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374709+0000","last_change":"2023-11-06T03:08:43.345100+0000","last_active":"2023-11-06T03:08:43.374709+0000","last_peered":"2023-11-06T03:08:43.374709+0000","last_clean":"2023-11-06T03:08:43.374709+0000","last_became_active":"2023-11-06T03:08:43.343854+0000","last_became_peered":"2023-11-06T03:08:43.343854+0000","last_unstale":"2023-11-06T03:08:43.374709+0000","last_undegraded":"2023-11-06T03:08:43.374709+0000","last_fullsized":"2023-11-06T03:08:43.374709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.360015+0000","last_change":"2023-11-06T03:08:43.359573+0000","last_active":"2023-11-06T03:08:43.360015+0000","last_peered":"2023-11-06T03:08:43.360015+0000","last_clean":"2023-11-06T03:08:43.360015+0000","last_became_active":"2023-11-06T03:08:43.359253+0000","last_became_peered":"2023-11-06T03:08:43.359253+0000","last_unstale":"2023-11-06T03:08:43.360015+0000","last_undegraded":"2023-11-06T03:08:43.360015+0000","last_fullsized":"2023-11-06T03:08:43.360015+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379136+0000","last_change":"2023-11-06T03:08:43.375535+0000","last_active":"2023-11-06T03:08:43.379136+0000","last_peered":"2023-11-06T03:08:43.379136+0000","last_clean":"2023-11-06T03:08:43.379136+0000","last_became_active":"2023-11-06T03:08:43.374859+0000","last_became_peered":"2023-11-06T03:08:43.374859+0000","last_unstale":"2023-11-06T03:08:43.379136+0000","last_undegraded":"2023-11-06T03:08:43.379136+0000","last_fullsized":"2023-11-06T03:08:43.379136+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:32.425326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384527+0000","last_change":"2023-11-06T03:08:43.375594+0000","last_active":"2023-11-06T03:08:43.384527+0000","last_peered":"2023-11-06T03:08:43.384527+0000","last_clean":"2023-11-06T03:08:43.384527+0000","last_became_active":"2023-11-06T03:08:43.375010+0000","last_became_peered":"2023-11-06T03:08:43.375010+0000","last_unstale":"2023-11-06T03:08:43.384527+0000","last_undegraded":"2023-11-06T03:08:43.384527+0000","last_fullsized":"2023-11-06T03:08:43.384527+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:58.576582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.375114+0000","last_change":"2023-11-06T03:08:43.343938+0000","last_active":"2023-11-06T03:08:43.375114+0000","last_peered":"2023-11-06T03:08:43.375114+0000","last_clean":"2023-11-06T03:08:43.375114+0000","last_became_active":"2023-11-06T03:08:43.343176+0000","last_became_peered":"2023-11-06T03:08:43.343176+0000","last_unstale":"2023-11-06T03:08:43.375114+0000","last_undegraded":"2023-11-06T03:08:43.375114+0000","last_fullsized":"2023-11-06T03:08:43.375114+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:40.486810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330556+0000","last_change":"2023-11-06T03:08:43.329666+0000","last_active":"2023-11-06T03:08:43.330556+0000","last_peered":"2023-11-06T03:08:43.330556+0000","last_clean":"2023-11-06T03:08:43.330556+0000","last_became_active":"2023-11-06T03:08:43.329355+0000","last_became_peered":"2023-11-06T03:08:43.329355+0000","last_unstale":"2023-11-06T03:08:43.330556+0000","last_undegraded":"2023-11-06T03:08:43.330556+0000","last_fullsized":"2023-11-06T03:08:43.330556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:22.093674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364661+0000","last_change":"2023-11-06T03:08:43.354096+0000","last_active":"2023-11-06T03:08:43.364661+0000","last_peered":"2023-11-06T03:08:43.364661+0000","last_clean":"2023-11-06T03:08:43.364661+0000","last_became_active":"2023-11-06T03:08:43.353327+0000","last_became_peered":"2023-11-06T03:08:43.353327+0000","last_unstale":"2023-11-06T03:08:43.364661+0000","last_undegraded":"2023-11-06T03:08:43.364661+0000","last_fullsized":"2023-11-06T03:08:43.364661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:42:58.834197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.372109+0000","last_change":"2023-11-06T03:08:42.964638+0000","last_active":"2023-11-06T03:08:43.372109+0000","last_peered":"2023-11-06T03:08:43.372109+0000","last_clean":"2023-11-06T03:08:43.372109+0000","last_became_active":"2023-11-06T03:08:42.964310+0000","last_became_peered":"2023-11-06T03:08:42.964310+0000","last_unstale":"2023-11-06T03:08:43.372109+0000","last_undegraded":"2023-11-06T03:08:43.372109+0000","last_fullsized":"2023-11-06T03:08:43.372109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:28.549155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376095+0000","last_change":"2023-11-06T03:08:43.364045+0000","last_active":"2023-11-06T03:08:43.376095+0000","last_peered":"2023-11-06T03:08:43.376095+0000","last_clean":"2023-11-06T03:08:43.376095+0000","last_became_active":"2023-11-06T03:08:43.357715+0000","last_became_peered":"2023-11-06T03:08:43.357715+0000","last_unstale":"2023-11-06T03:08:43.376095+0000","last_undegraded":"2023-11-06T03:08:43.376095+0000","last_fullsized":"2023-11-06T03:08:43.376095+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357296+0000","last_change":"2023-11-06T03:08:42.970165+0000","last_active":"2023-11-06T03:08:43.357296+0000","last_peered":"2023-11-06T03:08:43.357296+0000","last_clean":"2023-11-06T03:08:43.357296+0000","last_became_active":"2023-11-06T03:08:42.969830+0000","last_became_peered":"2023-11-06T03:08:42.969830+0000","last_unstale":"2023-11-06T03:08:43.357296+0000","last_undegraded":"2023-11-06T03:08:43.357296+0000","last_fullsized":"2023-11-06T03:08:43.357296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:31.421714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330210+0000","last_change":"2023-11-06T03:08:43.329444+0000","last_active":"2023-11-06T03:08:43.330210+0000","last_peered":"2023-11-06T03:08:43.330210+0000","last_clean":"2023-11-06T03:08:43.330210+0000","last_became_active":"2023-11-06T03:08:43.329150+0000","last_became_peered":"2023-11-06T03:08:43.329150+0000","last_unstale":"2023-11-06T03:08:43.330210+0000","last_undegraded":"2023-11-06T03:08:43.330210+0000","last_fullsized":"2023-11-06T03:08:43.330210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:45:38.926881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.337041+0000","last_change":"2023-11-06T03:08:42.975492+0000","last_active":"2023-11-06T03:08:43.337041+0000","last_peered":"2023-11-06T03:08:43.337041+0000","last_clean":"2023-11-06T03:08:43.337041+0000","last_became_active":"2023-11-06T03:08:42.975091+0000","last_became_peered":"2023-11-06T03:08:42.975091+0000","last_unstale":"2023-11-06T03:08:43.337041+0000","last_undegraded":"2023-11-06T03:08:43.337041+0000","last_fullsized":"2023-11-06T03:08:43.337041+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:17:06.318438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.378124+0000","last_change":"2023-11-06T03:08:43.371211+0000","last_active":"2023-11-06T03:08:43.378124+0000","last_peered":"2023-11-06T03:08:43.378124+0000","last_clean":"2023-11-06T03:08:43.378124+0000","last_became_active":"2023-11-06T03:08:43.370480+0000","last_became_peered":"2023-11-06T03:08:43.370480+0000","last_unstale":"2023-11-06T03:08:43.378124+0000","last_undegraded":"2023-11-06T03:08:43.378124+0000","last_fullsized":"2023-11-06T03:08:43.378124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325476+0000","last_change":"2023-11-06T03:08:42.949379+0000","last_active":"2023-11-06T03:08:43.325476+0000","last_peered":"2023-11-06T03:08:43.325476+0000","last_clean":"2023-11-06T03:08:43.325476+0000","last_became_active":"2023-11-06T03:08:42.948738+0000","last_became_peered":"2023-11-06T03:08:42.948738+0000","last_unstale":"2023-11-06T03:08:43.325476+0000","last_undegraded":"2023-11-06T03:08:43.325476+0000","last_fullsized":"2023-11-06T03:08:43.325476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:57.572971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330623+0000","last_change":"2023-11-06T03:08:43.328472+0000","last_active":"2023-11-06T03:08:43.330623+0000","last_peered":"2023-11-06T03:08:43.330623+0000","last_clean":"2023-11-06T03:08:43.330623+0000","last_became_active":"2023-11-06T03:08:43.328155+0000","last_became_peered":"2023-11-06T03:08:43.328155+0000","last_unstale":"2023-11-06T03:08:43.330623+0000","last_undegraded":"2023-11-06T03:08:43.330623+0000","last_fullsized":"2023-11-06T03:08:43.330623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:00.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325878+0000","last_change":"2023-11-06T03:08:42.950530+0000","last_active":"2023-11-06T03:08:43.325878+0000","last_peered":"2023-11-06T03:08:43.325878+0000","last_clean":"2023-11-06T03:08:43.325878+0000","last_became_active":"2023-11-06T03:08:42.950221+0000","last_became_peered":"2023-11-06T03:08:42.950221+0000","last_unstale":"2023-11-06T03:08:43.325878+0000","last_undegraded":"2023-11-06T03:08:43.325878+0000","last_fullsized":"2023-11-06T03:08:43.325878+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:11.707879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.354145+0000","last_change":"2023-11-06T03:08:43.353004+0000","last_active":"2023-11-06T03:08:43.354145+0000","last_peered":"2023-11-06T03:08:43.354145+0000","last_clean":"2023-11-06T03:08:43.354145+0000","last_became_active":"2023-11-06T03:08:43.352767+0000","last_became_peered":"2023-11-06T03:08:43.352767+0000","last_unstale":"2023-11-06T03:08:43.354145+0000","last_undegraded":"2023-11-06T03:08:43.354145+0000","last_fullsized":"2023-11-06T03:08:43.354145+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:48.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.328548+0000","last_change":"2023-11-06T03:08:42.950604+0000","last_active":"2023-11-06T03:08:43.328548+0000","last_peered":"2023-11-06T03:08:43.328548+0000","last_clean":"2023-11-06T03:08:43.328548+0000","last_became_active":"2023-11-06T03:08:42.950071+0000","last_became_peered":"2023-11-06T03:08:42.950071+0000","last_unstale":"2023-11-06T03:08:43.328548+0000","last_undegraded":"2023-11-06T03:08:43.328548+0000","last_fullsized":"2023-11-06T03:08:43.328548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:29.119351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365873+0000","last_change":"2023-11-06T03:08:43.357389+0000","last_active":"2023-11-06T03:08:43.365873+0000","last_peered":"2023-11-06T03:08:43.365873+0000","last_clean":"2023-11-06T03:08:43.365873+0000","last_became_active":"2023-11-06T03:08:43.354008+0000","last_became_peered":"2023-11-06T03:08:43.354008+0000","last_unstale":"2023-11-06T03:08:43.365873+0000","last_undegraded":"2023-11-06T03:08:43.365873+0000","last_fullsized":"2023-11-06T03:08:43.365873+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:16.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324338+0000","last_change":"2023-11-06T03:08:42.960598+0000","last_active":"2023-11-06T03:08:43.324338+0000","last_peered":"2023-11-06T03:08:43.324338+0000","last_clean":"2023-11-06T03:08:43.324338+0000","last_became_active":"2023-11-06T03:08:42.960234+0000","last_became_peered":"2023-11-06T03:08:42.960234+0000","last_unstale":"2023-11-06T03:08:43.324338+0000","last_undegraded":"2023-11-06T03:08:43.324338+0000","last_fullsized":"2023-11-06T03:08:43.324338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:37.721239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.367718+0000","last_change":"2023-11-06T03:08:43.367260+0000","last_active":"2023-11-06T03:08:43.367718+0000","last_peered":"2023-11-06T03:08:43.367718+0000","last_clean":"2023-11-06T03:08:43.367718+0000","last_became_active":"2023-11-06T03:08:43.366987+0000","last_became_peered":"2023-11-06T03:08:43.366987+0000","last_unstale":"2023-11-06T03:08:43.367718+0000","last_undegraded":"2023-11-06T03:08:43.367718+0000","last_fullsized":"2023-11-06T03:08:43.367718+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:53.728288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.336302+0000","last_change":"2023-11-06T03:08:42.965806+0000","last_active":"2023-11-06T03:08:43.336302+0000","last_peered":"2023-11-06T03:08:43.336302+0000","last_clean":"2023-11-06T03:08:43.336302+0000","last_became_active":"2023-11-06T03:08:42.965188+0000","last_became_peered":"2023-11-06T03:08:42.965188+0000","last_unstale":"2023-11-06T03:08:43.336302+0000","last_undegraded":"2023-11-06T03:08:43.336302+0000","last_fullsized":"2023-11-06T03:08:43.336302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:10.213675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330535+0000","last_change":"2023-11-06T03:08:43.329198+0000","last_active":"2023-11-06T03:08:43.330535+0000","last_peered":"2023-11-06T03:08:43.330535+0000","last_clean":"2023-11-06T03:08:43.330535+0000","last_became_active":"2023-11-06T03:08:43.328961+0000","last_became_peered":"2023-11-06T03:08:43.328961+0000","last_unstale":"2023-11-06T03:08:43.330535+0000","last_undegraded":"2023-11-06T03:08:43.330535+0000","last_fullsized":"2023-11-06T03:08:43.330535+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:40.486810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.348716+0000","last_change":"2023-11-06T03:08:42.973496+0000","last_active":"2023-11-06T03:08:43.348716+0000","last_peered":"2023-11-06T03:08:43.348716+0000","last_clean":"2023-11-06T03:08:43.348716+0000","last_became_active":"2023-11-06T03:08:42.973138+0000","last_became_peered":"2023-11-06T03:08:42.973138+0000","last_unstale":"2023-11-06T03:08:43.348716+0000","last_undegraded":"2023-11-06T03:08:43.348716+0000","last_fullsized":"2023-11-06T03:08:43.348716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:14:58.329507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.369168+0000","last_change":"2023-11-06T03:08:43.368351+0000","last_active":"2023-11-06T03:08:43.369168+0000","last_peered":"2023-11-06T03:08:43.369168+0000","last_clean":"2023-11-06T03:08:43.369168+0000","last_became_active":"2023-11-06T03:08:43.367999+0000","last_became_peered":"2023-11-06T03:08:43.367999+0000","last_unstale":"2023-11-06T03:08:43.369168+0000","last_undegraded":"2023-11-06T03:08:43.369168+0000","last_fullsized":"2023-11-06T03:08:43.369168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:37.981092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.321532+0000","last_change":"2023-11-06T03:08:42.951624+0000","last_active":"2023-11-06T03:08:43.321532+0000","last_peered":"2023-11-06T03:08:43.321532+0000","last_clean":"2023-11-06T03:08:43.321532+0000","last_became_active":"2023-11-06T03:08:42.950975+0000","last_became_peered":"2023-11-06T03:08:42.950975+0000","last_unstale":"2023-11-06T03:08:43.321532+0000","last_undegraded":"2023-11-06T03:08:43.321532+0000","last_fullsized":"2023-11-06T03:08:43.321532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:14:05.129068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380682+0000","last_change":"2023-11-06T03:08:43.372484+0000","last_active":"2023-11-06T03:08:43.380682+0000","last_peered":"2023-11-06T03:08:43.380682+0000","last_clean":"2023-11-06T03:08:43.380682+0000","last_became_active":"2023-11-06T03:08:43.372009+0000","last_became_peered":"2023-11-06T03:08:43.372009+0000","last_unstale":"2023-11-06T03:08:43.380682+0000","last_undegraded":"2023-11-06T03:08:43.380682+0000","last_fullsized":"2023-11-06T03:08:43.380682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:07.753466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371101+0000","last_change":"2023-11-06T03:08:42.973612+0000","last_active":"2023-11-06T03:08:43.371101+0000","last_peered":"2023-11-06T03:08:43.371101+0000","last_clean":"2023-11-06T03:08:43.371101+0000","last_became_active":"2023-11-06T03:08:42.972995+0000","last_became_peered":"2023-11-06T03:08:42.972995+0000","last_unstale":"2023-11-06T03:08:43.371101+0000","last_undegraded":"2023-11-06T03:08:43.371101+0000","last_fullsized":"2023-11-06T03:08:43.371101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:27:02.251880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374646+0000","last_change":"2023-11-06T03:08:43.345062+0000","last_active":"2023-11-06T03:08:43.374646+0000","last_peered":"2023-11-06T03:08:43.374646+0000","last_clean":"2023-11-06T03:08:43.374646+0000","last_became_active":"2023-11-06T03:08:43.343640+0000","last_became_peered":"2023-11-06T03:08:43.343640+0000","last_unstale":"2023-11-06T03:08:43.374646+0000","last_undegraded":"2023-11-06T03:08:43.374646+0000","last_fullsized":"2023-11-06T03:08:43.374646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:58.562854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379012+0000","last_change":"2023-11-06T03:08:42.964343+0000","last_active":"2023-11-06T03:08:43.379012+0000","last_peered":"2023-11-06T03:08:43.379012+0000","last_clean":"2023-11-06T03:08:43.379012+0000","last_became_active":"2023-11-06T03:08:42.963792+0000","last_became_peered":"2023-11-06T03:08:42.963792+0000","last_unstale":"2023-11-06T03:08:43.379012+0000","last_undegraded":"2023-11-06T03:08:43.379012+0000","last_fullsized":"2023-11-06T03:08:43.379012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:39.217255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.378937+0000","last_change":"2023-11-06T03:08:43.368354+0000","last_active":"2023-11-06T03:08:43.378937+0000","last_peered":"2023-11-06T03:08:43.378937+0000","last_clean":"2023-11-06T03:08:43.378937+0000","last_became_active":"2023-11-06T03:08:43.367395+0000","last_became_peered":"2023-11-06T03:08:43.367395+0000","last_unstale":"2023-11-06T03:08:43.378937+0000","last_undegraded":"2023-11-06T03:08:43.378937+0000","last_fullsized":"2023-11-06T03:08:43.378937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.373308+0000","last_change":"2023-11-06T03:08:42.971096+0000","last_active":"2023-11-06T03:08:43.373308+0000","last_peered":"2023-11-06T03:08:43.373308+0000","last_clean":"2023-11-06T03:08:43.373308+0000","last_became_active":"2023-11-06T03:08:42.970536+0000","last_became_peered":"2023-11-06T03:08:42.970536+0000","last_unstale":"2023-11-06T03:08:43.373308+0000","last_undegraded":"2023-11-06T03:08:43.373308+0000","last_fullsized":"2023-11-06T03:08:43.373308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370780+0000","last_change":"2023-11-06T03:08:43.349037+0000","last_active":"2023-11-06T03:08:43.370780+0000","last_peered":"2023-11-06T03:08:43.370780+0000","last_clean":"2023-11-06T03:08:43.370780+0000","last_became_active":"2023-11-06T03:08:43.348314+0000","last_became_peered":"2023-11-06T03:08:43.348314+0000","last_unstale":"2023-11-06T03:08:43.370780+0000","last_undegraded":"2023-11-06T03:08:43.370780+0000","last_fullsized":"2023-11-06T03:08:43.370780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.338488+0000","last_change":"2023-11-06T03:08:42.974747+0000","last_active":"2023-11-06T03:08:43.338488+0000","last_peered":"2023-11-06T03:08:43.338488+0000","last_clean":"2023-11-06T03:08:43.338488+0000","last_became_active":"2023-11-06T03:08:42.974070+0000","last_became_peered":"2023-11-06T03:08:42.974070+0000","last_unstale":"2023-11-06T03:08:43.338488+0000","last_undegraded":"2023-11-06T03:08:43.338488+0000","last_fullsized":"2023-11-06T03:08:43.338488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365830+0000","last_change":"2023-11-06T03:08:43.354754+0000","last_active":"2023-11-06T03:08:43.365830+0000","last_peered":"2023-11-06T03:08:43.365830+0000","last_clean":"2023-11-06T03:08:43.365830+0000","last_became_active":"2023-11-06T03:08:43.353210+0000","last_became_peered":"2023-11-06T03:08:43.353210+0000","last_unstale":"2023-11-06T03:08:43.365830+0000","last_undegraded":"2023-11-06T03:08:43.365830+0000","last_fullsized":"2023-11-06T03:08:43.365830+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.788410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.333916+0000","last_change":"2023-11-06T03:08:42.971960+0000","last_active":"2023-11-06T03:08:43.333916+0000","last_peered":"2023-11-06T03:08:43.333916+0000","last_clean":"2023-11-06T03:08:43.333916+0000","last_became_active":"2023-11-06T03:08:42.970728+0000","last_became_peered":"2023-11-06T03:08:42.970728+0000","last_unstale":"2023-11-06T03:08:43.333916+0000","last_undegraded":"2023-11-06T03:08:43.333916+0000","last_fullsized":"2023-11-06T03:08:43.333916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:12.578755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364454+0000","last_change":"2023-11-06T03:08:43.363588+0000","last_active":"2023-11-06T03:08:43.364454+0000","last_peered":"2023-11-06T03:08:43.364454+0000","last_clean":"2023-11-06T03:08:43.364454+0000","last_became_active":"2023-11-06T03:08:43.363170+0000","last_became_peered":"2023-11-06T03:08:43.363170+0000","last_unstale":"2023-11-06T03:08:43.364454+0000","last_undegraded":"2023-11-06T03:08:43.364454+0000","last_fullsized":"2023-11-06T03:08:43.364454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380316+0000","last_change":"2023-11-06T03:08:42.966933+0000","last_active":"2023-11-06T03:08:43.380316+0000","last_peered":"2023-11-06T03:08:43.380316+0000","last_clean":"2023-11-06T03:08:43.380316+0000","last_became_active":"2023-11-06T03:08:42.966530+0000","last_became_peered":"2023-11-06T03:08:42.966530+0000","last_unstale":"2023-11-06T03:08:43.380316+0000","last_undegraded":"2023-11-06T03:08:43.380316+0000","last_fullsized":"2023-11-06T03:08:43.380316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:43:21.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357403+0000","last_change":"2023-11-06T03:08:43.356580+0000","last_active":"2023-11-06T03:08:43.357403+0000","last_peered":"2023-11-06T03:08:43.357403+0000","last_clean":"2023-11-06T03:08:43.357403+0000","last_became_active":"2023-11-06T03:08:43.356217+0000","last_became_peered":"2023-11-06T03:08:43.356217+0000","last_unstale":"2023-11-06T03:08:43.357403+0000","last_undegraded":"2023-11-06T03:08:43.357403+0000","last_fullsized":"2023-11-06T03:08:43.357403+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:28.743220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371235+0000","last_change":"2023-11-06T03:08:42.974894+0000","last_active":"2023-11-06T03:08:43.371235+0000","last_peered":"2023-11-06T03:08:43.371235+0000","last_clean":"2023-11-06T03:08:43.371235+0000","last_became_active":"2023-11-06T03:08:42.973983+0000","last_became_peered":"2023-11-06T03:08:42.973983+0000","last_unstale":"2023-11-06T03:08:43.371235+0000","last_undegraded":"2023-11-06T03:08:43.371235+0000","last_fullsized":"2023-11-06T03:08:43.371235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.329926+0000","last_change":"2023-11-06T03:08:43.328241+0000","last_active":"2023-11-06T03:08:43.329926+0000","last_peered":"2023-11-06T03:08:43.329926+0000","last_clean":"2023-11-06T03:08:43.329926+0000","last_became_active":"2023-11-06T03:08:43.327864+0000","last_became_peered":"2023-11-06T03:08:43.327864+0000","last_unstale":"2023-11-06T03:08:43.329926+0000","last_undegraded":"2023-11-06T03:08:43.329926+0000","last_fullsized":"2023-11-06T03:08:43.329926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:11.217287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.348028+0000","last_change":"2023-11-06T03:08:42.976683+0000","last_active":"2023-11-06T03:08:43.348028+0000","last_peered":"2023-11-06T03:08:43.348028+0000","last_clean":"2023-11-06T03:08:43.348028+0000","last_became_active":"2023-11-06T03:08:42.974113+0000","last_became_peered":"2023-11-06T03:08:42.974113+0000","last_unstale":"2023-11-06T03:08:43.348028+0000","last_undegraded":"2023-11-06T03:08:43.348028+0000","last_fullsized":"2023-11-06T03:08:43.348028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:43:21.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383037+0000","last_change":"2023-11-06T03:08:43.369889+0000","last_active":"2023-11-06T03:08:43.383037+0000","last_peered":"2023-11-06T03:08:43.383037+0000","last_clean":"2023-11-06T03:08:43.383037+0000","last_became_active":"2023-11-06T03:08:43.368617+0000","last_became_peered":"2023-11-06T03:08:43.368617+0000","last_unstale":"2023-11-06T03:08:43.383037+0000","last_undegraded":"2023-11-06T03:08:43.383037+0000","last_fullsized":"2023-11-06T03:08:43.383037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.311592+0000","last_change":"2023-11-06T03:08:42.975494+0000","last_active":"2023-11-06T03:08:43.311592+0000","last_peered":"2023-11-06T03:08:43.311592+0000","last_clean":"2023-11-06T03:08:43.311592+0000","last_became_active":"2023-11-06T03:08:42.975110+0000","last_became_peered":"2023-11-06T03:08:42.975110+0000","last_unstale":"2023-11-06T03:08:43.311592+0000","last_undegraded":"2023-11-06T03:08:43.311592+0000","last_fullsized":"2023-11-06T03:08:43.311592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:54.631071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364804+0000","last_change":"2023-11-06T03:08:43.356412+0000","last_active":"2023-11-06T03:08:43.364804+0000","last_peered":"2023-11-06T03:08:43.364804+0000","last_clean":"2023-11-06T03:08:43.364804+0000","last_became_active":"2023-11-06T03:08:43.355675+0000","last_became_peered":"2023-11-06T03:08:43.355675+0000","last_unstale":"2023-11-06T03:08:43.364804+0000","last_undegraded":"2023-11-06T03:08:43.364804+0000","last_fullsized":"2023-11-06T03:08:43.364804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:13.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.318163+0000","last_change":"2023-11-06T03:08:42.964126+0000","last_active":"2023-11-06T03:08:43.318163+0000","last_peered":"2023-11-06T03:08:43.318163+0000","last_clean":"2023-11-06T03:08:43.318163+0000","last_became_active":"2023-11-06T03:08:42.963418+0000","last_became_peered":"2023-11-06T03:08:42.963418+0000","last_unstale":"2023-11-06T03:08:43.318163+0000","last_undegraded":"2023-11-06T03:08:43.318163+0000","last_fullsized":"2023-11-06T03:08:43.318163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364671+0000","last_change":"2023-11-06T03:08:43.353662+0000","last_active":"2023-11-06T03:08:43.364671+0000","last_peered":"2023-11-06T03:08:43.364671+0000","last_clean":"2023-11-06T03:08:43.364671+0000","last_became_active":"2023-11-06T03:08:43.352879+0000","last_became_peered":"2023-11-06T03:08:43.352879+0000","last_unstale":"2023-11-06T03:08:43.364671+0000","last_undegraded":"2023-11-06T03:08:43.364671+0000","last_fullsized":"2023-11-06T03:08:43.364671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:09:20.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.332466+0000","last_change":"2023-11-06T03:08:42.972131+0000","last_active":"2023-11-06T03:08:43.332466+0000","last_peered":"2023-11-06T03:08:43.332466+0000","last_clean":"2023-11-06T03:08:43.332466+0000","last_became_active":"2023-11-06T03:08:42.971296+0000","last_became_peered":"2023-11-06T03:08:42.971296+0000","last_unstale":"2023-11-06T03:08:43.332466+0000","last_undegraded":"2023-11-06T03:08:43.332466+0000","last_fullsized":"2023-11-06T03:08:43.332466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377326+0000","last_change":"2023-11-06T03:08:43.366715+0000","last_active":"2023-11-06T03:08:43.377326+0000","last_peered":"2023-11-06T03:08:43.377326+0000","last_clean":"2023-11-06T03:08:43.377326+0000","last_became_active":"2023-11-06T03:08:43.366031+0000","last_became_peered":"2023-11-06T03:08:43.366031+0000","last_unstale":"2023-11-06T03:08:43.377326+0000","last_undegraded":"2023-11-06T03:08:43.377326+0000","last_fullsized":"2023-11-06T03:08:43.377326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324619+0000","last_change":"2023-11-06T03:08:42.946965+0000","last_active":"2023-11-06T03:08:43.324619+0000","last_peered":"2023-11-06T03:08:43.324619+0000","last_clean":"2023-11-06T03:08:43.324619+0000","last_became_active":"2023-11-06T03:08:42.946626+0000","last_became_peered":"2023-11-06T03:08:42.946626+0000","last_unstale":"2023-11-06T03:08:43.324619+0000","last_undegraded":"2023-11-06T03:08:43.324619+0000","last_fullsized":"2023-11-06T03:08:43.324619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:58.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.344362+0000","last_change":"2023-11-06T03:08:43.343575+0000","last_active":"2023-11-06T03:08:43.344362+0000","last_peered":"2023-11-06T03:08:43.344362+0000","last_clean":"2023-11-06T03:08:43.344362+0000","last_became_active":"2023-11-06T03:08:43.343262+0000","last_became_peered":"2023-11-06T03:08:43.343262+0000","last_unstale":"2023-11-06T03:08:43.344362+0000","last_undegraded":"2023-11-06T03:08:43.344362+0000","last_fullsized":"2023-11-06T03:08:43.344362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:24.026904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324288+0000","last_change":"2023-11-06T03:08:42.960615+0000","last_active":"2023-11-06T03:08:43.324288+0000","last_peered":"2023-11-06T03:08:43.324288+0000","last_clean":"2023-11-06T03:08:43.324288+0000","last_became_active":"2023-11-06T03:08:42.960283+0000","last_became_peered":"2023-11-06T03:08:42.960283+0000","last_unstale":"2023-11-06T03:08:43.324288+0000","last_undegraded":"2023-11-06T03:08:43.324288+0000","last_fullsized":"2023-11-06T03:08:43.324288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:47.875677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.350331+0000","last_change":"2023-11-06T03:08:43.349637+0000","last_active":"2023-11-06T03:08:43.350331+0000","last_peered":"2023-11-06T03:08:43.350331+0000","last_clean":"2023-11-06T03:08:43.350331+0000","last_became_active":"2023-11-06T03:08:43.349418+0000","last_became_peered":"2023-11-06T03:08:43.349418+0000","last_unstale":"2023-11-06T03:08:43.350331+0000","last_undegraded":"2023-11-06T03:08:43.350331+0000","last_fullsized":"2023-11-06T03:08:43.350331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:31:53.222159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.316123+0000","last_change":"2023-11-06T03:08:42.962077+0000","last_active":"2023-11-06T03:08:43.316123+0000","last_peered":"2023-11-06T03:08:43.316123+0000","last_clean":"2023-11-06T03:08:43.316123+0000","last_became_active":"2023-11-06T03:08:42.961752+0000","last_became_peered":"2023-11-06T03:08:42.961752+0000","last_unstale":"2023-11-06T03:08:43.316123+0000","last_undegraded":"2023-11-06T03:08:43.316123+0000","last_fullsized":"2023-11-06T03:08:43.316123+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364744+0000","last_change":"2023-11-06T03:08:43.355434+0000","last_active":"2023-11-06T03:08:43.364744+0000","last_peered":"2023-11-06T03:08:43.364744+0000","last_clean":"2023-11-06T03:08:43.364744+0000","last_became_active":"2023-11-06T03:08:43.354433+0000","last_became_peered":"2023-11-06T03:08:43.354433+0000","last_unstale":"2023-11-06T03:08:43.364744+0000","last_undegraded":"2023-11-06T03:08:43.364744+0000","last_fullsized":"2023-11-06T03:08:43.364744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:32.978191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.323901+0000","last_change":"2023-11-06T03:08:42.950671+0000","last_active":"2023-11-06T03:08:43.323901+0000","last_peered":"2023-11-06T03:08:43.323901+0000","last_clean":"2023-11-06T03:08:43.323901+0000","last_became_active":"2023-11-06T03:08:42.950065+0000","last_became_peered":"2023-11-06T03:08:42.950065+0000","last_unstale":"2023-11-06T03:08:43.323901+0000","last_undegraded":"2023-11-06T03:08:43.323901+0000","last_fullsized":"2023-11-06T03:08:43.323901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:18.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.960727+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:08:42.960727+0000","last_peered":"2023-11-06T03:08:42.960727+0000","last_clean":"2023-11-06T03:08:42.960727+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:08:42.960727+0000","last_undegraded":"2023-11-06T03:08:42.960727+0000","last_fullsized":"2023-11-06T03:08:42.960727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.958608+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:08:42.958608+0000","last_peered":"2023-11-06T03:08:42.958608+0000","last_clean":"2023-11-06T03:08:42.958608+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:08:42.958608+0000","last_undegraded":"2023-11-06T03:08:42.958608+0000","last_fullsized":"2023-11-06T03:08:42.958608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330573+0000","last_change":"2023-11-06T03:08:43.329679+0000","last_active":"2023-11-06T03:08:43.330573+0000","last_peered":"2023-11-06T03:08:43.330573+0000","last_clean":"2023-11-06T03:08:43.330573+0000","last_became_active":"2023-11-06T03:08:43.329299+0000","last_became_peered":"2023-11-06T03:08:43.329299+0000","last_unstale":"2023-11-06T03:08:43.330573+0000","last_undegraded":"2023-11-06T03:08:43.330573+0000","last_fullsized":"2023-11-06T03:08:43.330573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:51.414010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.334799+0000","last_change":"2023-11-06T03:08:42.960669+0000","last_active":"2023-11-06T03:08:43.334799+0000","last_peered":"2023-11-06T03:08:43.334799+0000","last_clean":"2023-11-06T03:08:43.334799+0000","last_became_active":"2023-11-06T03:08:42.960299+0000","last_became_peered":"2023-11-06T03:08:42.960299+0000","last_unstale":"2023-11-06T03:08:43.334799+0000","last_undegraded":"2023-11-06T03:08:43.334799+0000","last_fullsized":"2023-11-06T03:08:43.334799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:53.949494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.960500+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:08:42.960500+0000","last_peered":"2023-11-06T03:08:42.960500+0000","last_clean":"2023-11-06T03:08:42.960500+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:08:42.960500+0000","last_undegraded":"2023-11-06T03:08:42.960500+0000","last_fullsized":"2023-11-06T03:08:42.960500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370739+0000","last_change":"2023-11-06T03:08:43.349008+0000","last_active":"2023-11-06T03:08:43.370739+0000","last_peered":"2023-11-06T03:08:43.370739+0000","last_clean":"2023-11-06T03:08:43.370739+0000","last_became_active":"2023-11-06T03:08:43.348275+0000","last_became_peered":"2023-11-06T03:08:43.348275+0000","last_unstale":"2023-11-06T03:08:43.370739+0000","last_undegraded":"2023-11-06T03:08:43.370739+0000","last_fullsized":"2023-11-06T03:08:43.370739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:12.711490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.338758+0000","last_change":"2023-11-06T03:08:42.974455+0000","last_active":"2023-11-06T03:08:43.338758+0000","last_peered":"2023-11-06T03:08:43.338758+0000","last_clean":"2023-11-06T03:08:43.338758+0000","last_became_active":"2023-11-06T03:08:42.974003+0000","last_became_peered":"2023-11-06T03:08:42.974003+0000","last_unstale":"2023-11-06T03:08:43.338758+0000","last_undegraded":"2023-11-06T03:08:43.338758+0000","last_fullsized":"2023-11-06T03:08:43.338758+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:12.578755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.948760+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:08:42.948760+0000","last_peered":"2023-11-06T03:08:42.948760+0000","last_clean":"2023-11-06T03:08:42.948760+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:08:42.948760+0000","last_undegraded":"2023-11-06T03:08:42.948760+0000","last_fullsized":"2023-11-06T03:08:42.948760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365776+0000","last_change":"2023-11-06T03:08:43.352606+0000","last_active":"2023-11-06T03:08:43.365776+0000","last_peered":"2023-11-06T03:08:43.365776+0000","last_clean":"2023-11-06T03:08:43.365776+0000","last_became_active":"2023-11-06T03:08:43.351854+0000","last_became_peered":"2023-11-06T03:08:43.351854+0000","last_unstale":"2023-11-06T03:08:43.365776+0000","last_undegraded":"2023-11-06T03:08:43.365776+0000","last_fullsized":"2023-11-06T03:08:43.365776+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:13.755530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.311469+0000","last_change":"2023-11-06T03:08:42.965447+0000","last_active":"2023-11-06T03:08:43.311469+0000","last_peered":"2023-11-06T03:08:43.311469+0000","last_clean":"2023-11-06T03:08:43.311469+0000","last_became_active":"2023-11-06T03:08:42.965177+0000","last_became_peered":"2023-11-06T03:08:42.965177+0000","last_unstale":"2023-11-06T03:08:43.311469+0000","last_undegraded":"2023-11-06T03:08:43.311469+0000","last_fullsized":"2023-11-06T03:08:43.311469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:13.483865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.958975+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:08:42.958975+0000","last_peered":"2023-11-06T03:08:42.958975+0000","last_clean":"2023-11-06T03:08:42.958975+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:08:42.958975+0000","last_undegraded":"2023-11-06T03:08:42.958975+0000","last_fullsized":"2023-11-06T03:08:42.958975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.381779+0000","last_change":"2023-11-06T03:08:43.380692+0000","last_active":"2023-11-06T03:08:43.381779+0000","last_peered":"2023-11-06T03:08:43.381779+0000","last_clean":"2023-11-06T03:08:43.381779+0000","last_became_active":"2023-11-06T03:08:43.380191+0000","last_became_peered":"2023-11-06T03:08:43.380191+0000","last_unstale":"2023-11-06T03:08:43.381779+0000","last_undegraded":"2023-11-06T03:08:43.381779+0000","last_fullsized":"2023-11-06T03:08:43.381779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:32:17.267201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382887+0000","last_change":"2023-11-06T03:08:42.975393+0000","last_active":"2023-11-06T03:08:43.382887+0000","last_peered":"2023-11-06T03:08:43.382887+0000","last_clean":"2023-11-06T03:08:43.382887+0000","last_became_active":"2023-11-06T03:08:42.974827+0000","last_became_peered":"2023-11-06T03:08:42.974827+0000","last_unstale":"2023-11-06T03:08:43.382887+0000","last_undegraded":"2023-11-06T03:08:43.382887+0000","last_fullsized":"2023-11-06T03:08:43.382887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:28.549155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.946309+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:08:42.946309+0000","last_peered":"2023-11-06T03:08:42.946309+0000","last_clean":"2023-11-06T03:08:42.946309+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:08:42.946309+0000","last_undegraded":"2023-11-06T03:08:42.946309+0000","last_fullsized":"2023-11-06T03:08:42.946309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380729+0000","last_change":"2023-11-06T03:08:43.373787+0000","last_active":"2023-11-06T03:08:43.380729+0000","last_peered":"2023-11-06T03:08:43.380729+0000","last_clean":"2023-11-06T03:08:43.380729+0000","last_became_active":"2023-11-06T03:08:43.373177+0000","last_became_peered":"2023-11-06T03:08:43.373177+0000","last_unstale":"2023-11-06T03:08:43.380729+0000","last_undegraded":"2023-11-06T03:08:43.380729+0000","last_fullsized":"2023-11-06T03:08:43.380729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:39.093081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377035+0000","last_change":"2023-11-06T03:08:42.973579+0000","last_active":"2023-11-06T03:08:43.377035+0000","last_peered":"2023-11-06T03:08:43.377035+0000","last_clean":"2023-11-06T03:08:43.377035+0000","last_became_active":"2023-11-06T03:08:42.972693+0000","last_became_peered":"2023-11-06T03:08:42.972693+0000","last_unstale":"2023-11-06T03:08:43.377035+0000","last_undegraded":"2023-11-06T03:08:43.377035+0000","last_fullsized":"2023-11-06T03:08:43.377035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:39.217255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.968026+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:08:42.968026+0000","last_peered":"2023-11-06T03:08:42.968026+0000","last_clean":"2023-11-06T03:08:42.968026+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:08:42.968026+0000","last_undegraded":"2023-11-06T03:08:42.968026+0000","last_fullsized":"2023-11-06T03:08:42.968026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.375902+0000","last_change":"2023-11-06T03:08:43.358006+0000","last_active":"2023-11-06T03:08:43.375902+0000","last_peered":"2023-11-06T03:08:43.375902+0000","last_clean":"2023-11-06T03:08:43.375902+0000","last_became_active":"2023-11-06T03:08:43.356769+0000","last_became_peered":"2023-11-06T03:08:43.356769+0000","last_unstale":"2023-11-06T03:08:43.375902+0000","last_undegraded":"2023-11-06T03:08:43.375902+0000","last_fullsized":"2023-11-06T03:08:43.375902+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:48.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370930+0000","last_change":"2023-11-06T03:08:42.974968+0000","last_active":"2023-11-06T03:08:43.370930+0000","last_peered":"2023-11-06T03:08:43.370930+0000","last_clean":"2023-11-06T03:08:43.370930+0000","last_became_active":"2023-11-06T03:08:42.974291+0000","last_became_peered":"2023-11-06T03:08:42.974291+0000","last_unstale":"2023-11-06T03:08:43.370930+0000","last_undegraded":"2023-11-06T03:08:43.370930+0000","last_fullsized":"2023-11-06T03:08:43.370930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:33.728832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.946415+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:08:42.946415+0000","last_peered":"2023-11-06T03:08:42.946415+0000","last_clean":"2023-11-06T03:08:42.946415+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:08:42.946415+0000","last_undegraded":"2023-11-06T03:08:42.946415+0000","last_fullsized":"2023-11-06T03:08:42.946415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365805+0000","last_change":"2023-11-06T03:08:43.357446+0000","last_active":"2023-11-06T03:08:43.365805+0000","last_peered":"2023-11-06T03:08:43.365805+0000","last_clean":"2023-11-06T03:08:43.365805+0000","last_became_active":"2023-11-06T03:08:43.355144+0000","last_became_peered":"2023-11-06T03:08:43.355144+0000","last_unstale":"2023-11-06T03:08:43.365805+0000","last_undegraded":"2023-11-06T03:08:43.365805+0000","last_fullsized":"2023-11-06T03:08:43.365805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:41.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325989+0000","last_change":"2023-11-06T03:08:42.963378+0000","last_active":"2023-11-06T03:08:43.325989+0000","last_peered":"2023-11-06T03:08:43.325989+0000","last_clean":"2023-11-06T03:08:43.325989+0000","last_became_active":"2023-11-06T03:08:42.963090+0000","last_became_peered":"2023-11-06T03:08:42.963090+0000","last_unstale":"2023-11-06T03:08:43.325989+0000","last_undegraded":"2023-11-06T03:08:43.325989+0000","last_fullsized":"2023-11-06T03:08:43.325989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:39.483199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.947046+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:08:42.947046+0000","last_peered":"2023-11-06T03:08:42.947046+0000","last_clean":"2023-11-06T03:08:42.947046+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:08:42.947046+0000","last_undegraded":"2023-11-06T03:08:42.947046+0000","last_fullsized":"2023-11-06T03:08:42.947046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1404928,"data_stored":943870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85099999999999998,"15min":0.89200000000000002},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.367,"5min":1.397,"15min":1.397},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.86899999999999999},"min":{"1min":0.53200000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.444,"15min":1.444},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85599999999999998,"15min":0.89200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.212,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.877,"15min":0.877},"min":{"1min":0.65600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4330000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.96499999999999997},"min":{"1min":0.56399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.222,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.91200000000000003},"min":{"1min":0.76600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.1970000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.76900000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.411,"5min":1.857,"15min":1.857},"last":0.94899999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.97999999999999998},"min":{"1min":0.622,"5min":0.499,"15min":0.499},"max":{"1min":1.123,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.161},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.97199999999999998,"15min":1.0169999999999999},"min":{"1min":0.70899999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.03,"15min":1.046},"min":{"1min":0.66100000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.754,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0660000000000001,"15min":1.0980000000000001},"min":{"1min":0.751,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.347,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1140000000000001,"15min":1.1559999999999999},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4379999999999999,"5min":1.774,"15min":1.774},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.476,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":778240,"data_stored":344579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.56599999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.036,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80800000000000005,"15min":0.82599999999999996},"min":{"1min":0.53200000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.23,"5min":2.073,"15min":2.073},"last":0.42599999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.85299999999999998,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.248,"5min":1.778,"15min":1.778},"last":0.61799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.997,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.65700000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2929999999999999,"5min":1.806,"15min":1.806},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93700000000000006,"15min":0.96399999999999997},"min":{"1min":0.63100000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3460000000000001,"5min":1.857,"15min":1.857},"last":0.748},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.91700000000000004,"15min":0.89500000000000002},"min":{"1min":0.81000000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0580000000000001,"15min":1.109},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4350000000000001,"5min":2.931,"15min":2.931},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.121,"15min":1.157},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.446,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.036},"min":{"1min":0.71499999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4079999999999999,"5min":1.506,"15min":1.506},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.113,"15min":1.1000000000000001},"min":{"1min":0.80500000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5229999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.1060000000000001},"min":{"1min":0.80800000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4019999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1100000000000001,"15min":1.121},"min":{"1min":0.84099999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":2.0059999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":352748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1439999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":1.175}]},{"osd":1,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78200000000000003,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.55700000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.238,"5min":1.357,"15min":1.357},"last":1.014}]},{"osd":2,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.68400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.24,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84699999999999998,"15min":0.83899999999999997},"min":{"1min":0.54500000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.83999999999999997},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2729999999999999,"5min":1.403,"15min":1.403},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.58499999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.429,"15min":1.429},"last":0.92100000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.58999999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.95799999999999996},"min":{"1min":0.64600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2030000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0109999999999999,"15min":0.99299999999999999},"min":{"1min":0.66400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":1.357,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.206},{"interface":"front","average":{"1min":1.022,"5min":1.0009999999999999,"15min":1.008},"min":{"1min":0.69799999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.01,"15min":0.98399999999999999},"min":{"1min":0.66600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.72299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3340000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.135}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":806912,"data_stored":354290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.82499999999999996},"min":{"1min":0.53500000000000003,"5min":0.372,"15min":0.372},"max":{"1min":1.109,"5min":1.484,"15min":1.484},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.137,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.503,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1279999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83399999999999996,"15min":0.80500000000000005},"min":{"1min":0.61799999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.131}]},{"osd":2,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.98099999999999998,"15min":1.0609999999999999},"min":{"1min":0.69899999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3069999999999999,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.85399999999999998},"min":{"1min":0.75,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.194,"5min":1.329,"15min":1.329},"last":1.018}]},{"osd":3,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91900000000000004,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.482,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.278,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.103}]},{"osd":5,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1,"15min":1.014},"min":{"1min":0.59499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2589999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.113,"5min":1.085,"15min":1.1100000000000001},"min":{"1min":0.877,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3600000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.232}]},{"osd":6,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1160000000000001,"15min":1.1619999999999999},"min":{"1min":0.89500000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.357,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.80300000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2969999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2509999999999999,"15min":1.1899999999999999},"min":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.4410000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49868,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321972,"statfs":{"total":96636764160,"available":96585699328,"internally_reserved":0,"allocated":790528,"data_stored":351707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88300000000000001,"5min":1.236,"15min":1.236},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.149,"5min":1.524,"15min":1.524},"last":0.35699999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.95199999999999996,"5min":1.353,"15min":1.353},"last":0.501},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98999999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.1419999999999999,"5min":1.349,"15min":1.349},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":1.0029999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88300000000000001,"15min":0.91100000000000003},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.319,"5min":1.327,"15min":1.327},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92200000000000004,"15min":0.95599999999999996},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1220000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95499999999999996,"15min":0.98699999999999999},"min":{"1min":0.496,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.236,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.52000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3160000000000001,"5min":1.698,"15min":1.698},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98199999999999998,"15min":1.014},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3700000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.96699999999999997,"15min":1.006},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.67100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.47,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.86299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":782336,"data_stored":345598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.23799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.628,"15min":0.65200000000000002},"min":{"1min":0.29299999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.109,"5min":1.121,"15min":1.121},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.65300000000000002,"15min":0.67700000000000005},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.38300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.752},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.36499999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.86799999999999999,"15min":0.89500000000000002},"min":{"1min":0.52400000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.238,"5min":1.385,"15min":1.385},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.96499999999999997},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.169}]},{"osd":5,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.87,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.252,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95099999999999996,"15min":0.97399999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.411,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.998,"15min":0.998},"min":{"1min":0.73799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.012,"5min":0.95099999999999996,"15min":0.95699999999999996},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0269999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0209999999999999,"15min":1.0049999999999999},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6859999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.96099999999999997,"15min":0.97199999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":355054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.32300000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.997,"5min":1.157,"15min":1.157},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0609999999999999,"5min":1.544,"15min":1.544},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.91800000000000004,"15min":0.72399999999999998},"min":{"1min":0.34599999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1659999999999999,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.69199999999999995},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1819999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71399999999999997,"15min":0.73299999999999998},"min":{"1min":0.40200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0089999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.874,"15min":0.88200000000000001},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.4319999999999999,"5min":2.258,"15min":2.258},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":0.95599999999999996},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.736,"5min":14.753,"15min":14.753},"last":0.81399999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.425,"15min":1.151},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4730000000000001,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.405,"15min":1.179},"min":{"1min":0.67900000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4279999999999999,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.2170000000000001,"15min":1.089},"min":{"1min":0.47999999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":23.971,"15min":23.971},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.226,"15min":1.1160000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.617,"5min":14.813000000000001,"15min":14.813000000000001},"last":1.036}]},{"osd":7,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0429999999999999,"15min":1.0860000000000001},"min":{"1min":0.72599999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4159999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.583,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313212,"statfs":{"total":96636764160,"available":96576729088,"internally_reserved":0,"allocated":1372160,"data_stored":934947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42299999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.89600000000000002,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.438},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72099999999999997,"15min":0.65900000000000003},"min":{"1min":0.309,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.006,"5min":1.155,"15min":1.155},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70399999999999996,"15min":0.66000000000000003},"min":{"1min":0.47399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1779999999999999,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97199999999999998,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.751,"15min":0.72499999999999998},"min":{"1min":0.44,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.083,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.98199999999999998,"15min":0.93400000000000005},"min":{"1min":0.68600000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.446,"5min":4.641,"15min":4.641},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.56599999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.286,"5min":2.028,"15min":2.028},"last":0.42899999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98399999999999999,"15min":0.995},"min":{"1min":0.79100000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4530000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.95299999999999996},"min":{"1min":0.66900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.405,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.90700000000000003},"min":{"1min":0.69499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.232,"5min":2.931,"15min":2.931},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.034,"5min":1.0309999999999999,"15min":1.0189999999999999},"min":{"1min":0.68899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.056,"15min":1.018},"min":{"1min":0.65600000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3540000000000001,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.403,"5min":1.97,"15min":1.97},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:08:46.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:08:46.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.484+0000 7f1bcaff9640 1 Processor -- start 2023-11-06T03:08:46.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.484+0000 7f1bcaff9640 1 -- start start 2023-11-06T03:08:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bcaff9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bc415caa0 0x7f1bc40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bcaff9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1bc40d0280 0x7f1bc40b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bcaff9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc40d2920 con 0x7f1bc415ce80 2023-11-06T03:08:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bcaff9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc40d18a0 con 0x7f1bc415caa0 2023-11-06T03:08:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bcaff9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc40d1b60 con 0x7f1bc40d0280 2023-11-06T03:08:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bc415caa0 0x7f1bc40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1bc415caa0 0x7f1bc40cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57948/0 (socket says 172.21.15.157:57948) 2023-11-06T03:08:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 -- 172.21.15.157:0/2006628846 learned_addr learned my addr 172.21.15.157:0/2006628846 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 -- 172.21.15.157:0/2006628846 >> v2:172.21.15.195:3300/0 conn(0x7f1bc40d0280 msgr2=0x7f1bc40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc9596640 1 --2- 172.21.15.157:0/2006628846 >> v2:172.21.15.195:3300/0 conn(0x7f1bc40d0280 0x7f1bc40b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 --2- 172.21.15.157:0/2006628846 >> v2:172.21.15.195:3300/0 conn(0x7f1bc40d0280 0x7f1bc40b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 -- 172.21.15.157:0/2006628846 >> v1:172.21.15.195:6789/0 conn(0x7f1bc415ce80 legacy=0x7f1bc40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 -- 172.21.15.157:0/2006628846 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc40d1e40 con 0x7f1bc415caa0 2023-11-06T03:08:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc8d95640 1 --2- 172.21.15.157:0/2006628846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415caa0 0x7f1bc40cdc60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40cf670 tx=0x7f1bb801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fde779ded85c632 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc37fe640 1 -- 172.21.15.157:0/2006628846 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80aa5f0 con 0x7f1bc415caa0 2023-11-06T03:08:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc37fe640 1 -- 172.21.15.157:0/2006628846 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb80aa750 con 0x7f1bc415caa0 2023-11-06T03:08:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.488+0000 7f1bc37fe640 1 -- 172.21.15.157:0/2006628846 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80aaae0 con 0x7f1bc415caa0 2023-11-06T03:08:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/2006628846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415caa0 msgr2=0x7f1bc40cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/2006628846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415caa0 0x7f1bc40cdc60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40cf670 tx=0x7f1bb801c490 comp rx=0 tx=0).stop 2023-11-06T03:08:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/2006628846 shutdown_connections 2023-11-06T03:08:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/2006628846 >> v2:172.21.15.195:3300/0 conn(0x7f1bc40d0280 0x7f1bc40b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/2006628846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415caa0 0x7f1bc40cdc60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/2006628846 >> 172.21.15.157:0/2006628846 conn(0x7f1bc4070790 msgr2=0x7f1bc4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/2006628846 shutdown_connections 2023-11-06T03:08:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/2006628846 wait complete. 2023-11-06T03:08:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 Processor -- start 2023-11-06T03:08:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- start start 2023-11-06T03:08:46.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bc40d0280 0x7f1bc40b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bc415caa0 0x7f1bc40b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc415bfd0 con 0x7f1bc415ce80 2023-11-06T03:08:46.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c67e0 con 0x7f1bc415caa0 2023-11-06T03:08:46.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c6a30 con 0x7f1bc40d0280 2023-11-06T03:08:46.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57960/0 (socket says 172.21.15.157:57960) 2023-11-06T03:08:46.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 -- 172.21.15.157:0/4192043544 learned_addr learned my addr 172.21.15.157:0/4192043544 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:46.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc8d95640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bc40d0280 0x7f1bc40b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc3fff640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bc415caa0 0x7f1bc40b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 -- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bc40d0280 msgr2=0x7f1bc40b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bc40d0280 0x7f1bc40b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 -- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bc415caa0 msgr2=0x7f1bc40b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bc415caa0 0x7f1bc40b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 -- 172.21.15.157:0/4192043544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80094d0 con 0x7f1bc415ce80 2023-11-06T03:08:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc9596640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00cc30 tx=0x7f1bbc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00ce40 con 0x7f1bc415ce80 2023-11-06T03:08:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc0039a0 con 0x7f1bc415ce80 2023-11-06T03:08:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc40c6d10 con 0x7f1bc415ce80 2023-11-06T03:08:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003d30 con 0x7f1bc415ce80 2023-11-06T03:08:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc40c72d0 con 0x7f1bc415ce80 2023-11-06T03:08:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.492+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc40c74f0 con 0x7f1bc415ce80 2023-11-06T03:08:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.496+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1bbc003e90 con 0x7f1bc415ce80 2023-11-06T03:08:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.496+0000 7f1bc17fa640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 0x7f1ba8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.496+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1bbc083760 con 0x7f1bc415ce80 2023-11-06T03:08:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.496+0000 7f1bc8d95640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 0x7f1ba8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.496+0000 7f1bc8d95640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 0x7f1ba8067f50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80ae000 tx=0x7f1bb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.500+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1bbc050f20 con 0x7f1bc415ce80 2023-11-06T03:08:46.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.644+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bc40b1f00 con 0x7f1ba8065a90 2023-11-06T03:08:46.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.668+0000 7f1bc17fa640 1 -- 172.21.15.157:0/4192043544 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310300 (secure 0 0 0) 0x7f1bb80ba090 con 0x7f1ba8065a90 2023-11-06T03:08:46.671 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:46.674 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:08:46.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 msgr2=0x7f1ba8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 0x7f1ba8067f50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80ae000 tx=0x7f1bb8008040 comp rx=0 tx=0).stop 2023-11-06T03:08:46.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 msgr2=0x7f1bc40c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00cc30 tx=0x7f1bbc00a5c0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 shutdown_connections 2023-11-06T03:08:46.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ba8065a90 0x7f1ba8067f50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bc415ce80 0x7f1bc40c77f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1bc415caa0 0x7f1bc40b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 --2- 172.21.15.157:0/4192043544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1bc40d0280 0x7f1bc40b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.672+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 >> 172.21.15.157:0/4192043544 conn(0x7f1bc4070790 msgr2=0x7f1bc41594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:46.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.676+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 shutdown_connections 2023-11-06T03:08:46.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.676+0000 7f1bcaff9640 1 -- 172.21.15.157:0/4192043544 wait complete. 2023-11-06T03:08:46.689 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2023-11-06T03:08:45.195802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":359956,"kb_used_data":7380,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754614764,"statfs":{"total":773094113280,"available":772725518336,"internally_reserved":0,"allocated":7557120,"data_stored":3982793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001128"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371764+0000","last_change":"2023-11-06T03:08:43.371377+0000","last_active":"2023-11-06T03:08:43.371764+0000","last_peered":"2023-11-06T03:08:43.371764+0000","last_clean":"2023-11-06T03:08:43.371764+0000","last_became_active":"2023-11-06T03:08:43.371095+0000","last_became_peered":"2023-11-06T03:08:43.371095+0000","last_unstale":"2023-11-06T03:08:43.371764+0000","last_undegraded":"2023-11-06T03:08:43.371764+0000","last_fullsized":"2023-11-06T03:08:43.371764+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.320147+0000","last_change":"2023-11-06T03:08:43.318920+0000","last_active":"2023-11-06T03:08:43.320147+0000","last_peered":"2023-11-06T03:08:43.320147+0000","last_clean":"2023-11-06T03:08:43.320147+0000","last_became_active":"2023-11-06T03:08:43.318555+0000","last_became_peered":"2023-11-06T03:08:43.318555+0000","last_unstale":"2023-11-06T03:08:43.320147+0000","last_undegraded":"2023-11-06T03:08:43.320147+0000","last_fullsized":"2023-11-06T03:08:43.320147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376771+0000","last_change":"2023-11-06T03:08:43.372054+0000","last_active":"2023-11-06T03:08:43.376771+0000","last_peered":"2023-11-06T03:08:43.376771+0000","last_clean":"2023-11-06T03:08:43.376771+0000","last_became_active":"2023-11-06T03:08:43.371474+0000","last_became_peered":"2023-11-06T03:08:43.371474+0000","last_unstale":"2023-11-06T03:08:43.376771+0000","last_undegraded":"2023-11-06T03:08:43.376771+0000","last_fullsized":"2023-11-06T03:08:43.376771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:54.953106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379121+0000","last_change":"2023-11-06T03:08:43.356227+0000","last_active":"2023-11-06T03:08:43.379121+0000","last_peered":"2023-11-06T03:08:43.379121+0000","last_clean":"2023-11-06T03:08:43.379121+0000","last_became_active":"2023-11-06T03:08:43.350719+0000","last_became_peered":"2023-11-06T03:08:43.350719+0000","last_unstale":"2023-11-06T03:08:43.379121+0000","last_undegraded":"2023-11-06T03:08:43.379121+0000","last_fullsized":"2023-11-06T03:08:43.379121+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:52.709530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380794+0000","last_change":"2023-11-06T03:08:43.373777+0000","last_active":"2023-11-06T03:08:43.380794+0000","last_peered":"2023-11-06T03:08:43.380794+0000","last_clean":"2023-11-06T03:08:43.380794+0000","last_became_active":"2023-11-06T03:08:43.373068+0000","last_became_peered":"2023-11-06T03:08:43.373068+0000","last_unstale":"2023-11-06T03:08:43.380794+0000","last_undegraded":"2023-11-06T03:08:43.380794+0000","last_fullsized":"2023-11-06T03:08:43.380794+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:48:43.530094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365755+0000","last_change":"2023-11-06T03:08:43.365209+0000","last_active":"2023-11-06T03:08:43.365755+0000","last_peered":"2023-11-06T03:08:43.365755+0000","last_clean":"2023-11-06T03:08:43.365755+0000","last_became_active":"2023-11-06T03:08:43.364872+0000","last_became_peered":"2023-11-06T03:08:43.364872+0000","last_unstale":"2023-11-06T03:08:43.365755+0000","last_undegraded":"2023-11-06T03:08:43.365755+0000","last_fullsized":"2023-11-06T03:08:43.365755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:02:20.030442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364413+0000","last_change":"2023-11-06T03:08:43.363807+0000","last_active":"2023-11-06T03:08:43.364413+0000","last_peered":"2023-11-06T03:08:43.364413+0000","last_clean":"2023-11-06T03:08:43.364413+0000","last_became_active":"2023-11-06T03:08:43.363455+0000","last_became_peered":"2023-11-06T03:08:43.363455+0000","last_unstale":"2023-11-06T03:08:43.364413+0000","last_undegraded":"2023-11-06T03:08:43.364413+0000","last_fullsized":"2023-11-06T03:08:43.364413+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:35.779457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384563+0000","last_change":"2023-11-06T03:08:43.376870+0000","last_active":"2023-11-06T03:08:43.384563+0000","last_peered":"2023-11-06T03:08:43.384563+0000","last_clean":"2023-11-06T03:08:43.384563+0000","last_became_active":"2023-11-06T03:08:43.376282+0000","last_became_peered":"2023-11-06T03:08:43.376282+0000","last_unstale":"2023-11-06T03:08:43.384563+0000","last_undegraded":"2023-11-06T03:08:43.384563+0000","last_fullsized":"2023-11-06T03:08:43.384563+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:32:17.267201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.381164+0000","last_change":"2023-11-06T03:08:43.378404+0000","last_active":"2023-11-06T03:08:43.381164+0000","last_peered":"2023-11-06T03:08:43.381164+0000","last_clean":"2023-11-06T03:08:43.381164+0000","last_became_active":"2023-11-06T03:08:43.377698+0000","last_became_peered":"2023-11-06T03:08:43.377698+0000","last_unstale":"2023-11-06T03:08:43.381164+0000","last_undegraded":"2023-11-06T03:08:43.381164+0000","last_fullsized":"2023-11-06T03:08:43.381164+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:58.576582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379028+0000","last_change":"2023-11-06T03:08:43.374077+0000","last_active":"2023-11-06T03:08:43.379028+0000","last_peered":"2023-11-06T03:08:43.379028+0000","last_clean":"2023-11-06T03:08:43.379028+0000","last_became_active":"2023-11-06T03:08:43.372782+0000","last_became_peered":"2023-11-06T03:08:43.372782+0000","last_unstale":"2023-11-06T03:08:43.379028+0000","last_undegraded":"2023-11-06T03:08:43.379028+0000","last_fullsized":"2023-11-06T03:08:43.379028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374741+0000","last_change":"2023-11-06T03:08:43.350306+0000","last_active":"2023-11-06T03:08:43.374741+0000","last_peered":"2023-11-06T03:08:43.374741+0000","last_clean":"2023-11-06T03:08:43.374741+0000","last_became_active":"2023-11-06T03:08:43.343842+0000","last_became_peered":"2023-11-06T03:08:43.343842+0000","last_unstale":"2023-11-06T03:08:43.374741+0000","last_undegraded":"2023-11-06T03:08:43.374741+0000","last_fullsized":"2023-11-06T03:08:43.374741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384491+0000","last_change":"2023-11-06T03:08:43.373110+0000","last_active":"2023-11-06T03:08:43.384491+0000","last_peered":"2023-11-06T03:08:43.384491+0000","last_clean":"2023-11-06T03:08:43.384491+0000","last_became_active":"2023-11-06T03:08:43.372255+0000","last_became_peered":"2023-11-06T03:08:43.372255+0000","last_unstale":"2023-11-06T03:08:43.384491+0000","last_undegraded":"2023-11-06T03:08:43.384491+0000","last_fullsized":"2023-11-06T03:08:43.384491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:54.953106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382957+0000","last_change":"2023-11-06T03:08:43.368957+0000","last_active":"2023-11-06T03:08:43.382957+0000","last_peered":"2023-11-06T03:08:43.382957+0000","last_clean":"2023-11-06T03:08:43.382957+0000","last_became_active":"2023-11-06T03:08:43.368357+0000","last_became_peered":"2023-11-06T03:08:43.368357+0000","last_unstale":"2023-11-06T03:08:43.382957+0000","last_undegraded":"2023-11-06T03:08:43.382957+0000","last_fullsized":"2023-11-06T03:08:43.382957+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376823+0000","last_change":"2023-11-06T03:08:43.371997+0000","last_active":"2023-11-06T03:08:43.376823+0000","last_peered":"2023-11-06T03:08:43.376823+0000","last_clean":"2023-11-06T03:08:43.376823+0000","last_became_active":"2023-11-06T03:08:43.370023+0000","last_became_peered":"2023-11-06T03:08:43.370023+0000","last_unstale":"2023-11-06T03:08:43.376823+0000","last_undegraded":"2023-11-06T03:08:43.376823+0000","last_fullsized":"2023-11-06T03:08:43.376823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376630+0000","last_change":"2023-11-06T03:08:43.371385+0000","last_active":"2023-11-06T03:08:43.376630+0000","last_peered":"2023-11-06T03:08:43.376630+0000","last_clean":"2023-11-06T03:08:43.376630+0000","last_became_active":"2023-11-06T03:08:43.370680+0000","last_became_peered":"2023-11-06T03:08:43.370680+0000","last_unstale":"2023-11-06T03:08:43.376630+0000","last_undegraded":"2023-11-06T03:08:43.376630+0000","last_fullsized":"2023-11-06T03:08:43.376630+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.369874+0000","last_change":"2023-11-06T03:08:43.369401+0000","last_active":"2023-11-06T03:08:43.369874+0000","last_peered":"2023-11-06T03:08:43.369874+0000","last_clean":"2023-11-06T03:08:43.369874+0000","last_became_active":"2023-11-06T03:08:43.369051+0000","last_became_peered":"2023-11-06T03:08:43.369051+0000","last_unstale":"2023-11-06T03:08:43.369874+0000","last_undegraded":"2023-11-06T03:08:43.369874+0000","last_fullsized":"2023-11-06T03:08:43.369874+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:47:02.035595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.344356+0000","last_change":"2023-11-06T03:08:43.343583+0000","last_active":"2023-11-06T03:08:43.344356+0000","last_peered":"2023-11-06T03:08:43.344356+0000","last_clean":"2023-11-06T03:08:43.344356+0000","last_became_active":"2023-11-06T03:08:43.343309+0000","last_became_peered":"2023-11-06T03:08:43.343309+0000","last_unstale":"2023-11-06T03:08:43.344356+0000","last_undegraded":"2023-11-06T03:08:43.344356+0000","last_fullsized":"2023-11-06T03:08:43.344356+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374469+0000","last_change":"2023-11-06T03:08:43.344995+0000","last_active":"2023-11-06T03:08:43.374469+0000","last_peered":"2023-11-06T03:08:43.374469+0000","last_clean":"2023-11-06T03:08:43.374469+0000","last_became_active":"2023-11-06T03:08:43.344192+0000","last_became_peered":"2023-11-06T03:08:43.344192+0000","last_unstale":"2023-11-06T03:08:43.374469+0000","last_undegraded":"2023-11-06T03:08:43.374469+0000","last_fullsized":"2023-11-06T03:08:43.374469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:05:04.920453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364584+0000","last_change":"2023-11-06T03:08:43.352964+0000","last_active":"2023-11-06T03:08:43.364584+0000","last_peered":"2023-11-06T03:08:43.364584+0000","last_clean":"2023-11-06T03:08:43.364584+0000","last_became_active":"2023-11-06T03:08:43.352197+0000","last_became_peered":"2023-11-06T03:08:43.352197+0000","last_unstale":"2023-11-06T03:08:43.364584+0000","last_undegraded":"2023-11-06T03:08:43.364584+0000","last_fullsized":"2023-11-06T03:08:43.364584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:23:47.479034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357713+0000","last_change":"2023-11-06T03:08:43.356614+0000","last_active":"2023-11-06T03:08:43.357713+0000","last_peered":"2023-11-06T03:08:43.357713+0000","last_clean":"2023-11-06T03:08:43.357713+0000","last_became_active":"2023-11-06T03:08:43.356324+0000","last_became_peered":"2023-11-06T03:08:43.356324+0000","last_unstale":"2023-11-06T03:08:43.357713+0000","last_undegraded":"2023-11-06T03:08:43.357713+0000","last_fullsized":"2023-11-06T03:08:43.357713+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:55.634682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382916+0000","last_change":"2023-11-06T03:08:43.354883+0000","last_active":"2023-11-06T03:08:43.382916+0000","last_peered":"2023-11-06T03:08:43.382916+0000","last_clean":"2023-11-06T03:08:43.382916+0000","last_became_active":"2023-11-06T03:08:43.349971+0000","last_became_peered":"2023-11-06T03:08:43.349971+0000","last_unstale":"2023-11-06T03:08:43.382916+0000","last_undegraded":"2023-11-06T03:08:43.382916+0000","last_fullsized":"2023-11-06T03:08:43.382916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377689+0000","last_change":"2023-11-06T03:08:43.377192+0000","last_active":"2023-11-06T03:08:43.377689+0000","last_peered":"2023-11-06T03:08:43.377689+0000","last_clean":"2023-11-06T03:08:43.377689+0000","last_became_active":"2023-11-06T03:08:43.376809+0000","last_became_peered":"2023-11-06T03:08:43.376809+0000","last_unstale":"2023-11-06T03:08:43.377689+0000","last_undegraded":"2023-11-06T03:08:43.377689+0000","last_fullsized":"2023-11-06T03:08:43.377689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:22:31.965269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.363908+0000","last_change":"2023-11-06T03:08:43.363409+0000","last_active":"2023-11-06T03:08:43.363908+0000","last_peered":"2023-11-06T03:08:43.363908+0000","last_clean":"2023-11-06T03:08:43.363908+0000","last_became_active":"2023-11-06T03:08:43.362850+0000","last_became_peered":"2023-11-06T03:08:43.362850+0000","last_unstale":"2023-11-06T03:08:43.363908+0000","last_undegraded":"2023-11-06T03:08:43.363908+0000","last_fullsized":"2023-11-06T03:08:43.363908+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:51.025831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.372914+0000","last_change":"2023-11-06T03:08:43.363939+0000","last_active":"2023-11-06T03:08:43.372914+0000","last_peered":"2023-11-06T03:08:43.372914+0000","last_clean":"2023-11-06T03:08:43.372914+0000","last_became_active":"2023-11-06T03:08:43.363104+0000","last_became_peered":"2023-11-06T03:08:43.363104+0000","last_unstale":"2023-11-06T03:08:43.372914+0000","last_undegraded":"2023-11-06T03:08:43.372914+0000","last_fullsized":"2023-11-06T03:08:43.372914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:12.711490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383077+0000","last_change":"2023-11-06T03:08:43.370350+0000","last_active":"2023-11-06T03:08:43.383077+0000","last_peered":"2023-11-06T03:08:43.383077+0000","last_clean":"2023-11-06T03:08:43.383077+0000","last_became_active":"2023-11-06T03:08:43.369766+0000","last_became_peered":"2023-11-06T03:08:43.369766+0000","last_unstale":"2023-11-06T03:08:43.383077+0000","last_undegraded":"2023-11-06T03:08:43.383077+0000","last_fullsized":"2023-11-06T03:08:43.383077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383070+0000","last_change":"2023-11-06T03:08:43.370361+0000","last_active":"2023-11-06T03:08:43.383070+0000","last_peered":"2023-11-06T03:08:43.383070+0000","last_clean":"2023-11-06T03:08:43.383070+0000","last_became_active":"2023-11-06T03:08:43.369251+0000","last_became_peered":"2023-11-06T03:08:43.369251+0000","last_unstale":"2023-11-06T03:08:43.383070+0000","last_undegraded":"2023-11-06T03:08:43.383070+0000","last_fullsized":"2023-11-06T03:08:43.383070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:32.425326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374709+0000","last_change":"2023-11-06T03:08:43.345100+0000","last_active":"2023-11-06T03:08:43.374709+0000","last_peered":"2023-11-06T03:08:43.374709+0000","last_clean":"2023-11-06T03:08:43.374709+0000","last_became_active":"2023-11-06T03:08:43.343854+0000","last_became_peered":"2023-11-06T03:08:43.343854+0000","last_unstale":"2023-11-06T03:08:43.374709+0000","last_undegraded":"2023-11-06T03:08:43.374709+0000","last_fullsized":"2023-11-06T03:08:43.374709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.360015+0000","last_change":"2023-11-06T03:08:43.359573+0000","last_active":"2023-11-06T03:08:43.360015+0000","last_peered":"2023-11-06T03:08:43.360015+0000","last_clean":"2023-11-06T03:08:43.360015+0000","last_became_active":"2023-11-06T03:08:43.359253+0000","last_became_peered":"2023-11-06T03:08:43.359253+0000","last_unstale":"2023-11-06T03:08:43.360015+0000","last_undegraded":"2023-11-06T03:08:43.360015+0000","last_fullsized":"2023-11-06T03:08:43.360015+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:59.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379136+0000","last_change":"2023-11-06T03:08:43.375535+0000","last_active":"2023-11-06T03:08:43.379136+0000","last_peered":"2023-11-06T03:08:43.379136+0000","last_clean":"2023-11-06T03:08:43.379136+0000","last_became_active":"2023-11-06T03:08:43.374859+0000","last_became_peered":"2023-11-06T03:08:43.374859+0000","last_unstale":"2023-11-06T03:08:43.379136+0000","last_undegraded":"2023-11-06T03:08:43.379136+0000","last_fullsized":"2023-11-06T03:08:43.379136+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:32.425326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.384527+0000","last_change":"2023-11-06T03:08:43.375594+0000","last_active":"2023-11-06T03:08:43.384527+0000","last_peered":"2023-11-06T03:08:43.384527+0000","last_clean":"2023-11-06T03:08:43.384527+0000","last_became_active":"2023-11-06T03:08:43.375010+0000","last_became_peered":"2023-11-06T03:08:43.375010+0000","last_unstale":"2023-11-06T03:08:43.384527+0000","last_undegraded":"2023-11-06T03:08:43.384527+0000","last_fullsized":"2023-11-06T03:08:43.384527+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:58.576582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.375114+0000","last_change":"2023-11-06T03:08:43.343938+0000","last_active":"2023-11-06T03:08:43.375114+0000","last_peered":"2023-11-06T03:08:43.375114+0000","last_clean":"2023-11-06T03:08:43.375114+0000","last_became_active":"2023-11-06T03:08:43.343176+0000","last_became_peered":"2023-11-06T03:08:43.343176+0000","last_unstale":"2023-11-06T03:08:43.375114+0000","last_undegraded":"2023-11-06T03:08:43.375114+0000","last_fullsized":"2023-11-06T03:08:43.375114+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:40.486810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330556+0000","last_change":"2023-11-06T03:08:43.329666+0000","last_active":"2023-11-06T03:08:43.330556+0000","last_peered":"2023-11-06T03:08:43.330556+0000","last_clean":"2023-11-06T03:08:43.330556+0000","last_became_active":"2023-11-06T03:08:43.329355+0000","last_became_peered":"2023-11-06T03:08:43.329355+0000","last_unstale":"2023-11-06T03:08:43.330556+0000","last_undegraded":"2023-11-06T03:08:43.330556+0000","last_fullsized":"2023-11-06T03:08:43.330556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:06:22.093674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364661+0000","last_change":"2023-11-06T03:08:43.354096+0000","last_active":"2023-11-06T03:08:43.364661+0000","last_peered":"2023-11-06T03:08:43.364661+0000","last_clean":"2023-11-06T03:08:43.364661+0000","last_became_active":"2023-11-06T03:08:43.353327+0000","last_became_peered":"2023-11-06T03:08:43.353327+0000","last_unstale":"2023-11-06T03:08:43.364661+0000","last_undegraded":"2023-11-06T03:08:43.364661+0000","last_fullsized":"2023-11-06T03:08:43.364661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:42:58.834197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.372109+0000","last_change":"2023-11-06T03:08:42.964638+0000","last_active":"2023-11-06T03:08:43.372109+0000","last_peered":"2023-11-06T03:08:43.372109+0000","last_clean":"2023-11-06T03:08:43.372109+0000","last_became_active":"2023-11-06T03:08:42.964310+0000","last_became_peered":"2023-11-06T03:08:42.964310+0000","last_unstale":"2023-11-06T03:08:43.372109+0000","last_undegraded":"2023-11-06T03:08:43.372109+0000","last_fullsized":"2023-11-06T03:08:43.372109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:28.549155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.376095+0000","last_change":"2023-11-06T03:08:43.364045+0000","last_active":"2023-11-06T03:08:43.376095+0000","last_peered":"2023-11-06T03:08:43.376095+0000","last_clean":"2023-11-06T03:08:43.376095+0000","last_became_active":"2023-11-06T03:08:43.357715+0000","last_became_peered":"2023-11-06T03:08:43.357715+0000","last_unstale":"2023-11-06T03:08:43.376095+0000","last_undegraded":"2023-11-06T03:08:43.376095+0000","last_fullsized":"2023-11-06T03:08:43.376095+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357296+0000","last_change":"2023-11-06T03:08:42.970165+0000","last_active":"2023-11-06T03:08:43.357296+0000","last_peered":"2023-11-06T03:08:43.357296+0000","last_clean":"2023-11-06T03:08:43.357296+0000","last_became_active":"2023-11-06T03:08:42.969830+0000","last_became_peered":"2023-11-06T03:08:42.969830+0000","last_unstale":"2023-11-06T03:08:43.357296+0000","last_undegraded":"2023-11-06T03:08:43.357296+0000","last_fullsized":"2023-11-06T03:08:43.357296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:31.421714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330210+0000","last_change":"2023-11-06T03:08:43.329444+0000","last_active":"2023-11-06T03:08:43.330210+0000","last_peered":"2023-11-06T03:08:43.330210+0000","last_clean":"2023-11-06T03:08:43.330210+0000","last_became_active":"2023-11-06T03:08:43.329150+0000","last_became_peered":"2023-11-06T03:08:43.329150+0000","last_unstale":"2023-11-06T03:08:43.330210+0000","last_undegraded":"2023-11-06T03:08:43.330210+0000","last_fullsized":"2023-11-06T03:08:43.330210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:45:38.926881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.337041+0000","last_change":"2023-11-06T03:08:42.975492+0000","last_active":"2023-11-06T03:08:43.337041+0000","last_peered":"2023-11-06T03:08:43.337041+0000","last_clean":"2023-11-06T03:08:43.337041+0000","last_became_active":"2023-11-06T03:08:42.975091+0000","last_became_peered":"2023-11-06T03:08:42.975091+0000","last_unstale":"2023-11-06T03:08:43.337041+0000","last_undegraded":"2023-11-06T03:08:43.337041+0000","last_fullsized":"2023-11-06T03:08:43.337041+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:17:06.318438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.378124+0000","last_change":"2023-11-06T03:08:43.371211+0000","last_active":"2023-11-06T03:08:43.378124+0000","last_peered":"2023-11-06T03:08:43.378124+0000","last_clean":"2023-11-06T03:08:43.378124+0000","last_became_active":"2023-11-06T03:08:43.370480+0000","last_became_peered":"2023-11-06T03:08:43.370480+0000","last_unstale":"2023-11-06T03:08:43.378124+0000","last_undegraded":"2023-11-06T03:08:43.378124+0000","last_fullsized":"2023-11-06T03:08:43.378124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325476+0000","last_change":"2023-11-06T03:08:42.949379+0000","last_active":"2023-11-06T03:08:43.325476+0000","last_peered":"2023-11-06T03:08:43.325476+0000","last_clean":"2023-11-06T03:08:43.325476+0000","last_became_active":"2023-11-06T03:08:42.948738+0000","last_became_peered":"2023-11-06T03:08:42.948738+0000","last_unstale":"2023-11-06T03:08:43.325476+0000","last_undegraded":"2023-11-06T03:08:43.325476+0000","last_fullsized":"2023-11-06T03:08:43.325476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:57.572971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330623+0000","last_change":"2023-11-06T03:08:43.328472+0000","last_active":"2023-11-06T03:08:43.330623+0000","last_peered":"2023-11-06T03:08:43.330623+0000","last_clean":"2023-11-06T03:08:43.330623+0000","last_became_active":"2023-11-06T03:08:43.328155+0000","last_became_peered":"2023-11-06T03:08:43.328155+0000","last_unstale":"2023-11-06T03:08:43.330623+0000","last_undegraded":"2023-11-06T03:08:43.330623+0000","last_fullsized":"2023-11-06T03:08:43.330623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:02:00.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325878+0000","last_change":"2023-11-06T03:08:42.950530+0000","last_active":"2023-11-06T03:08:43.325878+0000","last_peered":"2023-11-06T03:08:43.325878+0000","last_clean":"2023-11-06T03:08:43.325878+0000","last_became_active":"2023-11-06T03:08:42.950221+0000","last_became_peered":"2023-11-06T03:08:42.950221+0000","last_unstale":"2023-11-06T03:08:43.325878+0000","last_undegraded":"2023-11-06T03:08:43.325878+0000","last_fullsized":"2023-11-06T03:08:43.325878+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:11.707879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.354145+0000","last_change":"2023-11-06T03:08:43.353004+0000","last_active":"2023-11-06T03:08:43.354145+0000","last_peered":"2023-11-06T03:08:43.354145+0000","last_clean":"2023-11-06T03:08:43.354145+0000","last_became_active":"2023-11-06T03:08:43.352767+0000","last_became_peered":"2023-11-06T03:08:43.352767+0000","last_unstale":"2023-11-06T03:08:43.354145+0000","last_undegraded":"2023-11-06T03:08:43.354145+0000","last_fullsized":"2023-11-06T03:08:43.354145+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:48.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.328548+0000","last_change":"2023-11-06T03:08:42.950604+0000","last_active":"2023-11-06T03:08:43.328548+0000","last_peered":"2023-11-06T03:08:43.328548+0000","last_clean":"2023-11-06T03:08:43.328548+0000","last_became_active":"2023-11-06T03:08:42.950071+0000","last_became_peered":"2023-11-06T03:08:42.950071+0000","last_unstale":"2023-11-06T03:08:43.328548+0000","last_undegraded":"2023-11-06T03:08:43.328548+0000","last_fullsized":"2023-11-06T03:08:43.328548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:29.119351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365873+0000","last_change":"2023-11-06T03:08:43.357389+0000","last_active":"2023-11-06T03:08:43.365873+0000","last_peered":"2023-11-06T03:08:43.365873+0000","last_clean":"2023-11-06T03:08:43.365873+0000","last_became_active":"2023-11-06T03:08:43.354008+0000","last_became_peered":"2023-11-06T03:08:43.354008+0000","last_unstale":"2023-11-06T03:08:43.365873+0000","last_undegraded":"2023-11-06T03:08:43.365873+0000","last_fullsized":"2023-11-06T03:08:43.365873+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:16.909983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324338+0000","last_change":"2023-11-06T03:08:42.960598+0000","last_active":"2023-11-06T03:08:43.324338+0000","last_peered":"2023-11-06T03:08:43.324338+0000","last_clean":"2023-11-06T03:08:43.324338+0000","last_became_active":"2023-11-06T03:08:42.960234+0000","last_became_peered":"2023-11-06T03:08:42.960234+0000","last_unstale":"2023-11-06T03:08:43.324338+0000","last_undegraded":"2023-11-06T03:08:43.324338+0000","last_fullsized":"2023-11-06T03:08:43.324338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:37.721239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.367718+0000","last_change":"2023-11-06T03:08:43.367260+0000","last_active":"2023-11-06T03:08:43.367718+0000","last_peered":"2023-11-06T03:08:43.367718+0000","last_clean":"2023-11-06T03:08:43.367718+0000","last_became_active":"2023-11-06T03:08:43.366987+0000","last_became_peered":"2023-11-06T03:08:43.366987+0000","last_unstale":"2023-11-06T03:08:43.367718+0000","last_undegraded":"2023-11-06T03:08:43.367718+0000","last_fullsized":"2023-11-06T03:08:43.367718+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:44:53.728288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.336302+0000","last_change":"2023-11-06T03:08:42.965806+0000","last_active":"2023-11-06T03:08:43.336302+0000","last_peered":"2023-11-06T03:08:43.336302+0000","last_clean":"2023-11-06T03:08:43.336302+0000","last_became_active":"2023-11-06T03:08:42.965188+0000","last_became_peered":"2023-11-06T03:08:42.965188+0000","last_unstale":"2023-11-06T03:08:43.336302+0000","last_undegraded":"2023-11-06T03:08:43.336302+0000","last_fullsized":"2023-11-06T03:08:43.336302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:10.213675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330535+0000","last_change":"2023-11-06T03:08:43.329198+0000","last_active":"2023-11-06T03:08:43.330535+0000","last_peered":"2023-11-06T03:08:43.330535+0000","last_clean":"2023-11-06T03:08:43.330535+0000","last_became_active":"2023-11-06T03:08:43.328961+0000","last_became_peered":"2023-11-06T03:08:43.328961+0000","last_unstale":"2023-11-06T03:08:43.330535+0000","last_undegraded":"2023-11-06T03:08:43.330535+0000","last_fullsized":"2023-11-06T03:08:43.330535+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:40.486810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.348716+0000","last_change":"2023-11-06T03:08:42.973496+0000","last_active":"2023-11-06T03:08:43.348716+0000","last_peered":"2023-11-06T03:08:43.348716+0000","last_clean":"2023-11-06T03:08:43.348716+0000","last_became_active":"2023-11-06T03:08:42.973138+0000","last_became_peered":"2023-11-06T03:08:42.973138+0000","last_unstale":"2023-11-06T03:08:43.348716+0000","last_undegraded":"2023-11-06T03:08:43.348716+0000","last_fullsized":"2023-11-06T03:08:43.348716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:14:58.329507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.369168+0000","last_change":"2023-11-06T03:08:43.368351+0000","last_active":"2023-11-06T03:08:43.369168+0000","last_peered":"2023-11-06T03:08:43.369168+0000","last_clean":"2023-11-06T03:08:43.369168+0000","last_became_active":"2023-11-06T03:08:43.367999+0000","last_became_peered":"2023-11-06T03:08:43.367999+0000","last_unstale":"2023-11-06T03:08:43.369168+0000","last_undegraded":"2023-11-06T03:08:43.369168+0000","last_fullsized":"2023-11-06T03:08:43.369168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:37.981092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.321532+0000","last_change":"2023-11-06T03:08:42.951624+0000","last_active":"2023-11-06T03:08:43.321532+0000","last_peered":"2023-11-06T03:08:43.321532+0000","last_clean":"2023-11-06T03:08:43.321532+0000","last_became_active":"2023-11-06T03:08:42.950975+0000","last_became_peered":"2023-11-06T03:08:42.950975+0000","last_unstale":"2023-11-06T03:08:43.321532+0000","last_undegraded":"2023-11-06T03:08:43.321532+0000","last_fullsized":"2023-11-06T03:08:43.321532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:14:05.129068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380682+0000","last_change":"2023-11-06T03:08:43.372484+0000","last_active":"2023-11-06T03:08:43.380682+0000","last_peered":"2023-11-06T03:08:43.380682+0000","last_clean":"2023-11-06T03:08:43.380682+0000","last_became_active":"2023-11-06T03:08:43.372009+0000","last_became_peered":"2023-11-06T03:08:43.372009+0000","last_unstale":"2023-11-06T03:08:43.380682+0000","last_undegraded":"2023-11-06T03:08:43.380682+0000","last_fullsized":"2023-11-06T03:08:43.380682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:59:07.753466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371101+0000","last_change":"2023-11-06T03:08:42.973612+0000","last_active":"2023-11-06T03:08:43.371101+0000","last_peered":"2023-11-06T03:08:43.371101+0000","last_clean":"2023-11-06T03:08:43.371101+0000","last_became_active":"2023-11-06T03:08:42.972995+0000","last_became_peered":"2023-11-06T03:08:42.972995+0000","last_unstale":"2023-11-06T03:08:43.371101+0000","last_undegraded":"2023-11-06T03:08:43.371101+0000","last_fullsized":"2023-11-06T03:08:43.371101+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:27:02.251880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.374646+0000","last_change":"2023-11-06T03:08:43.345062+0000","last_active":"2023-11-06T03:08:43.374646+0000","last_peered":"2023-11-06T03:08:43.374646+0000","last_clean":"2023-11-06T03:08:43.374646+0000","last_became_active":"2023-11-06T03:08:43.343640+0000","last_became_peered":"2023-11-06T03:08:43.343640+0000","last_unstale":"2023-11-06T03:08:43.374646+0000","last_undegraded":"2023-11-06T03:08:43.374646+0000","last_fullsized":"2023-11-06T03:08:43.374646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:58.562854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.379012+0000","last_change":"2023-11-06T03:08:42.964343+0000","last_active":"2023-11-06T03:08:43.379012+0000","last_peered":"2023-11-06T03:08:43.379012+0000","last_clean":"2023-11-06T03:08:43.379012+0000","last_became_active":"2023-11-06T03:08:42.963792+0000","last_became_peered":"2023-11-06T03:08:42.963792+0000","last_unstale":"2023-11-06T03:08:43.379012+0000","last_undegraded":"2023-11-06T03:08:43.379012+0000","last_fullsized":"2023-11-06T03:08:43.379012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:39.217255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.378937+0000","last_change":"2023-11-06T03:08:43.368354+0000","last_active":"2023-11-06T03:08:43.378937+0000","last_peered":"2023-11-06T03:08:43.378937+0000","last_clean":"2023-11-06T03:08:43.378937+0000","last_became_active":"2023-11-06T03:08:43.367395+0000","last_became_peered":"2023-11-06T03:08:43.367395+0000","last_unstale":"2023-11-06T03:08:43.378937+0000","last_undegraded":"2023-11-06T03:08:43.378937+0000","last_fullsized":"2023-11-06T03:08:43.378937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.373308+0000","last_change":"2023-11-06T03:08:42.971096+0000","last_active":"2023-11-06T03:08:43.373308+0000","last_peered":"2023-11-06T03:08:43.373308+0000","last_clean":"2023-11-06T03:08:43.373308+0000","last_became_active":"2023-11-06T03:08:42.970536+0000","last_became_peered":"2023-11-06T03:08:42.970536+0000","last_unstale":"2023-11-06T03:08:43.373308+0000","last_undegraded":"2023-11-06T03:08:43.373308+0000","last_fullsized":"2023-11-06T03:08:43.373308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370780+0000","last_change":"2023-11-06T03:08:43.349037+0000","last_active":"2023-11-06T03:08:43.370780+0000","last_peered":"2023-11-06T03:08:43.370780+0000","last_clean":"2023-11-06T03:08:43.370780+0000","last_became_active":"2023-11-06T03:08:43.348314+0000","last_became_peered":"2023-11-06T03:08:43.348314+0000","last_unstale":"2023-11-06T03:08:43.370780+0000","last_undegraded":"2023-11-06T03:08:43.370780+0000","last_fullsized":"2023-11-06T03:08:43.370780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:47:19.395673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.338488+0000","last_change":"2023-11-06T03:08:42.974747+0000","last_active":"2023-11-06T03:08:43.338488+0000","last_peered":"2023-11-06T03:08:43.338488+0000","last_clean":"2023-11-06T03:08:43.338488+0000","last_became_active":"2023-11-06T03:08:42.974070+0000","last_became_peered":"2023-11-06T03:08:42.974070+0000","last_unstale":"2023-11-06T03:08:43.338488+0000","last_undegraded":"2023-11-06T03:08:43.338488+0000","last_fullsized":"2023-11-06T03:08:43.338488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365830+0000","last_change":"2023-11-06T03:08:43.354754+0000","last_active":"2023-11-06T03:08:43.365830+0000","last_peered":"2023-11-06T03:08:43.365830+0000","last_clean":"2023-11-06T03:08:43.365830+0000","last_became_active":"2023-11-06T03:08:43.353210+0000","last_became_peered":"2023-11-06T03:08:43.353210+0000","last_unstale":"2023-11-06T03:08:43.365830+0000","last_undegraded":"2023-11-06T03:08:43.365830+0000","last_fullsized":"2023-11-06T03:08:43.365830+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:14:21.788410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.333916+0000","last_change":"2023-11-06T03:08:42.971960+0000","last_active":"2023-11-06T03:08:43.333916+0000","last_peered":"2023-11-06T03:08:43.333916+0000","last_clean":"2023-11-06T03:08:43.333916+0000","last_became_active":"2023-11-06T03:08:42.970728+0000","last_became_peered":"2023-11-06T03:08:42.970728+0000","last_unstale":"2023-11-06T03:08:43.333916+0000","last_undegraded":"2023-11-06T03:08:43.333916+0000","last_fullsized":"2023-11-06T03:08:43.333916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:12.578755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364454+0000","last_change":"2023-11-06T03:08:43.363588+0000","last_active":"2023-11-06T03:08:43.364454+0000","last_peered":"2023-11-06T03:08:43.364454+0000","last_clean":"2023-11-06T03:08:43.364454+0000","last_became_active":"2023-11-06T03:08:43.363170+0000","last_became_peered":"2023-11-06T03:08:43.363170+0000","last_unstale":"2023-11-06T03:08:43.364454+0000","last_undegraded":"2023-11-06T03:08:43.364454+0000","last_fullsized":"2023-11-06T03:08:43.364454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:19.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380316+0000","last_change":"2023-11-06T03:08:42.966933+0000","last_active":"2023-11-06T03:08:43.380316+0000","last_peered":"2023-11-06T03:08:43.380316+0000","last_clean":"2023-11-06T03:08:43.380316+0000","last_became_active":"2023-11-06T03:08:42.966530+0000","last_became_peered":"2023-11-06T03:08:42.966530+0000","last_unstale":"2023-11-06T03:08:43.380316+0000","last_undegraded":"2023-11-06T03:08:43.380316+0000","last_fullsized":"2023-11-06T03:08:43.380316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:43:21.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.357403+0000","last_change":"2023-11-06T03:08:43.356580+0000","last_active":"2023-11-06T03:08:43.357403+0000","last_peered":"2023-11-06T03:08:43.357403+0000","last_clean":"2023-11-06T03:08:43.357403+0000","last_became_active":"2023-11-06T03:08:43.356217+0000","last_became_peered":"2023-11-06T03:08:43.356217+0000","last_unstale":"2023-11-06T03:08:43.357403+0000","last_undegraded":"2023-11-06T03:08:43.357403+0000","last_fullsized":"2023-11-06T03:08:43.357403+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:28.743220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.371235+0000","last_change":"2023-11-06T03:08:42.974894+0000","last_active":"2023-11-06T03:08:43.371235+0000","last_peered":"2023-11-06T03:08:43.371235+0000","last_clean":"2023-11-06T03:08:43.371235+0000","last_became_active":"2023-11-06T03:08:42.973983+0000","last_became_peered":"2023-11-06T03:08:42.973983+0000","last_unstale":"2023-11-06T03:08:43.371235+0000","last_undegraded":"2023-11-06T03:08:43.371235+0000","last_fullsized":"2023-11-06T03:08:43.371235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.329926+0000","last_change":"2023-11-06T03:08:43.328241+0000","last_active":"2023-11-06T03:08:43.329926+0000","last_peered":"2023-11-06T03:08:43.329926+0000","last_clean":"2023-11-06T03:08:43.329926+0000","last_became_active":"2023-11-06T03:08:43.327864+0000","last_became_peered":"2023-11-06T03:08:43.327864+0000","last_unstale":"2023-11-06T03:08:43.329926+0000","last_undegraded":"2023-11-06T03:08:43.329926+0000","last_fullsized":"2023-11-06T03:08:43.329926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:11.217287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.348028+0000","last_change":"2023-11-06T03:08:42.976683+0000","last_active":"2023-11-06T03:08:43.348028+0000","last_peered":"2023-11-06T03:08:43.348028+0000","last_clean":"2023-11-06T03:08:43.348028+0000","last_became_active":"2023-11-06T03:08:42.974113+0000","last_became_peered":"2023-11-06T03:08:42.974113+0000","last_unstale":"2023-11-06T03:08:43.348028+0000","last_undegraded":"2023-11-06T03:08:43.348028+0000","last_fullsized":"2023-11-06T03:08:43.348028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:43:21.160543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.383037+0000","last_change":"2023-11-06T03:08:43.369889+0000","last_active":"2023-11-06T03:08:43.383037+0000","last_peered":"2023-11-06T03:08:43.383037+0000","last_clean":"2023-11-06T03:08:43.383037+0000","last_became_active":"2023-11-06T03:08:43.368617+0000","last_became_peered":"2023-11-06T03:08:43.368617+0000","last_unstale":"2023-11-06T03:08:43.383037+0000","last_undegraded":"2023-11-06T03:08:43.383037+0000","last_fullsized":"2023-11-06T03:08:43.383037+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:38.724851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.311592+0000","last_change":"2023-11-06T03:08:42.975494+0000","last_active":"2023-11-06T03:08:43.311592+0000","last_peered":"2023-11-06T03:08:43.311592+0000","last_clean":"2023-11-06T03:08:43.311592+0000","last_became_active":"2023-11-06T03:08:42.975110+0000","last_became_peered":"2023-11-06T03:08:42.975110+0000","last_unstale":"2023-11-06T03:08:43.311592+0000","last_undegraded":"2023-11-06T03:08:43.311592+0000","last_fullsized":"2023-11-06T03:08:43.311592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:40:54.631071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364804+0000","last_change":"2023-11-06T03:08:43.356412+0000","last_active":"2023-11-06T03:08:43.364804+0000","last_peered":"2023-11-06T03:08:43.364804+0000","last_clean":"2023-11-06T03:08:43.364804+0000","last_became_active":"2023-11-06T03:08:43.355675+0000","last_became_peered":"2023-11-06T03:08:43.355675+0000","last_unstale":"2023-11-06T03:08:43.364804+0000","last_undegraded":"2023-11-06T03:08:43.364804+0000","last_fullsized":"2023-11-06T03:08:43.364804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:00:13.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.318163+0000","last_change":"2023-11-06T03:08:42.964126+0000","last_active":"2023-11-06T03:08:43.318163+0000","last_peered":"2023-11-06T03:08:43.318163+0000","last_clean":"2023-11-06T03:08:43.318163+0000","last_became_active":"2023-11-06T03:08:42.963418+0000","last_became_peered":"2023-11-06T03:08:42.963418+0000","last_unstale":"2023-11-06T03:08:43.318163+0000","last_undegraded":"2023-11-06T03:08:43.318163+0000","last_fullsized":"2023-11-06T03:08:43.318163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364671+0000","last_change":"2023-11-06T03:08:43.353662+0000","last_active":"2023-11-06T03:08:43.364671+0000","last_peered":"2023-11-06T03:08:43.364671+0000","last_clean":"2023-11-06T03:08:43.364671+0000","last_became_active":"2023-11-06T03:08:43.352879+0000","last_became_peered":"2023-11-06T03:08:43.352879+0000","last_unstale":"2023-11-06T03:08:43.364671+0000","last_undegraded":"2023-11-06T03:08:43.364671+0000","last_fullsized":"2023-11-06T03:08:43.364671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:09:20.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.332466+0000","last_change":"2023-11-06T03:08:42.972131+0000","last_active":"2023-11-06T03:08:43.332466+0000","last_peered":"2023-11-06T03:08:43.332466+0000","last_clean":"2023-11-06T03:08:43.332466+0000","last_became_active":"2023-11-06T03:08:42.971296+0000","last_became_peered":"2023-11-06T03:08:42.971296+0000","last_unstale":"2023-11-06T03:08:43.332466+0000","last_undegraded":"2023-11-06T03:08:43.332466+0000","last_fullsized":"2023-11-06T03:08:43.332466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:18.250318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377326+0000","last_change":"2023-11-06T03:08:43.366715+0000","last_active":"2023-11-06T03:08:43.377326+0000","last_peered":"2023-11-06T03:08:43.377326+0000","last_clean":"2023-11-06T03:08:43.377326+0000","last_became_active":"2023-11-06T03:08:43.366031+0000","last_became_peered":"2023-11-06T03:08:43.366031+0000","last_unstale":"2023-11-06T03:08:43.377326+0000","last_undegraded":"2023-11-06T03:08:43.377326+0000","last_fullsized":"2023-11-06T03:08:43.377326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:30.122963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324619+0000","last_change":"2023-11-06T03:08:42.946965+0000","last_active":"2023-11-06T03:08:43.324619+0000","last_peered":"2023-11-06T03:08:43.324619+0000","last_clean":"2023-11-06T03:08:43.324619+0000","last_became_active":"2023-11-06T03:08:42.946626+0000","last_became_peered":"2023-11-06T03:08:42.946626+0000","last_unstale":"2023-11-06T03:08:43.324619+0000","last_undegraded":"2023-11-06T03:08:43.324619+0000","last_fullsized":"2023-11-06T03:08:43.324619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:52:58.166014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.344362+0000","last_change":"2023-11-06T03:08:43.343575+0000","last_active":"2023-11-06T03:08:43.344362+0000","last_peered":"2023-11-06T03:08:43.344362+0000","last_clean":"2023-11-06T03:08:43.344362+0000","last_became_active":"2023-11-06T03:08:43.343262+0000","last_became_peered":"2023-11-06T03:08:43.343262+0000","last_unstale":"2023-11-06T03:08:43.344362+0000","last_undegraded":"2023-11-06T03:08:43.344362+0000","last_fullsized":"2023-11-06T03:08:43.344362+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:25:24.026904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.324288+0000","last_change":"2023-11-06T03:08:42.960615+0000","last_active":"2023-11-06T03:08:43.324288+0000","last_peered":"2023-11-06T03:08:43.324288+0000","last_clean":"2023-11-06T03:08:43.324288+0000","last_became_active":"2023-11-06T03:08:42.960283+0000","last_became_peered":"2023-11-06T03:08:42.960283+0000","last_unstale":"2023-11-06T03:08:43.324288+0000","last_undegraded":"2023-11-06T03:08:43.324288+0000","last_fullsized":"2023-11-06T03:08:43.324288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:47.875677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.350331+0000","last_change":"2023-11-06T03:08:43.349637+0000","last_active":"2023-11-06T03:08:43.350331+0000","last_peered":"2023-11-06T03:08:43.350331+0000","last_clean":"2023-11-06T03:08:43.350331+0000","last_became_active":"2023-11-06T03:08:43.349418+0000","last_became_peered":"2023-11-06T03:08:43.349418+0000","last_unstale":"2023-11-06T03:08:43.350331+0000","last_undegraded":"2023-11-06T03:08:43.350331+0000","last_fullsized":"2023-11-06T03:08:43.350331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:31:53.222159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.316123+0000","last_change":"2023-11-06T03:08:42.962077+0000","last_active":"2023-11-06T03:08:43.316123+0000","last_peered":"2023-11-06T03:08:43.316123+0000","last_clean":"2023-11-06T03:08:43.316123+0000","last_became_active":"2023-11-06T03:08:42.961752+0000","last_became_peered":"2023-11-06T03:08:42.961752+0000","last_unstale":"2023-11-06T03:08:43.316123+0000","last_undegraded":"2023-11-06T03:08:43.316123+0000","last_fullsized":"2023-11-06T03:08:43.316123+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:56:40.872718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.364744+0000","last_change":"2023-11-06T03:08:43.355434+0000","last_active":"2023-11-06T03:08:43.364744+0000","last_peered":"2023-11-06T03:08:43.364744+0000","last_clean":"2023-11-06T03:08:43.364744+0000","last_became_active":"2023-11-06T03:08:43.354433+0000","last_became_peered":"2023-11-06T03:08:43.354433+0000","last_unstale":"2023-11-06T03:08:43.364744+0000","last_undegraded":"2023-11-06T03:08:43.364744+0000","last_fullsized":"2023-11-06T03:08:43.364744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:53:32.978191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.323901+0000","last_change":"2023-11-06T03:08:42.950671+0000","last_active":"2023-11-06T03:08:43.323901+0000","last_peered":"2023-11-06T03:08:43.323901+0000","last_clean":"2023-11-06T03:08:43.323901+0000","last_became_active":"2023-11-06T03:08:42.950065+0000","last_became_peered":"2023-11-06T03:08:42.950065+0000","last_unstale":"2023-11-06T03:08:43.323901+0000","last_undegraded":"2023-11-06T03:08:43.323901+0000","last_fullsized":"2023-11-06T03:08:43.323901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:30:18.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":205,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.960727+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:08:42.960727+0000","last_peered":"2023-11-06T03:08:42.960727+0000","last_clean":"2023-11-06T03:08:42.960727+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:08:42.960727+0000","last_undegraded":"2023-11-06T03:08:42.960727+0000","last_fullsized":"2023-11-06T03:08:42.960727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.958608+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:08:42.958608+0000","last_peered":"2023-11-06T03:08:42.958608+0000","last_clean":"2023-11-06T03:08:42.958608+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:08:42.958608+0000","last_undegraded":"2023-11-06T03:08:42.958608+0000","last_fullsized":"2023-11-06T03:08:42.958608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.330573+0000","last_change":"2023-11-06T03:08:43.329679+0000","last_active":"2023-11-06T03:08:43.330573+0000","last_peered":"2023-11-06T03:08:43.330573+0000","last_clean":"2023-11-06T03:08:43.330573+0000","last_became_active":"2023-11-06T03:08:43.329299+0000","last_became_peered":"2023-11-06T03:08:43.329299+0000","last_unstale":"2023-11-06T03:08:43.330573+0000","last_undegraded":"2023-11-06T03:08:43.330573+0000","last_fullsized":"2023-11-06T03:08:43.330573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:52:51.414010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.334799+0000","last_change":"2023-11-06T03:08:42.960669+0000","last_active":"2023-11-06T03:08:43.334799+0000","last_peered":"2023-11-06T03:08:43.334799+0000","last_clean":"2023-11-06T03:08:43.334799+0000","last_became_active":"2023-11-06T03:08:42.960299+0000","last_became_peered":"2023-11-06T03:08:42.960299+0000","last_unstale":"2023-11-06T03:08:43.334799+0000","last_undegraded":"2023-11-06T03:08:43.334799+0000","last_fullsized":"2023-11-06T03:08:43.334799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:53.949494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.960500+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:08:42.960500+0000","last_peered":"2023-11-06T03:08:42.960500+0000","last_clean":"2023-11-06T03:08:42.960500+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:08:42.960500+0000","last_undegraded":"2023-11-06T03:08:42.960500+0000","last_fullsized":"2023-11-06T03:08:42.960500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370739+0000","last_change":"2023-11-06T03:08:43.349008+0000","last_active":"2023-11-06T03:08:43.370739+0000","last_peered":"2023-11-06T03:08:43.370739+0000","last_clean":"2023-11-06T03:08:43.370739+0000","last_became_active":"2023-11-06T03:08:43.348275+0000","last_became_peered":"2023-11-06T03:08:43.348275+0000","last_unstale":"2023-11-06T03:08:43.370739+0000","last_undegraded":"2023-11-06T03:08:43.370739+0000","last_fullsized":"2023-11-06T03:08:43.370739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:17:12.711490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.338758+0000","last_change":"2023-11-06T03:08:42.974455+0000","last_active":"2023-11-06T03:08:43.338758+0000","last_peered":"2023-11-06T03:08:43.338758+0000","last_clean":"2023-11-06T03:08:43.338758+0000","last_became_active":"2023-11-06T03:08:42.974003+0000","last_became_peered":"2023-11-06T03:08:42.974003+0000","last_unstale":"2023-11-06T03:08:43.338758+0000","last_undegraded":"2023-11-06T03:08:43.338758+0000","last_fullsized":"2023-11-06T03:08:43.338758+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:12.578755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.948760+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:08:42.948760+0000","last_peered":"2023-11-06T03:08:42.948760+0000","last_clean":"2023-11-06T03:08:42.948760+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:08:42.948760+0000","last_undegraded":"2023-11-06T03:08:42.948760+0000","last_fullsized":"2023-11-06T03:08:42.948760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365776+0000","last_change":"2023-11-06T03:08:43.352606+0000","last_active":"2023-11-06T03:08:43.365776+0000","last_peered":"2023-11-06T03:08:43.365776+0000","last_clean":"2023-11-06T03:08:43.365776+0000","last_became_active":"2023-11-06T03:08:43.351854+0000","last_became_peered":"2023-11-06T03:08:43.351854+0000","last_unstale":"2023-11-06T03:08:43.365776+0000","last_undegraded":"2023-11-06T03:08:43.365776+0000","last_fullsized":"2023-11-06T03:08:43.365776+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:28:13.755530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.311469+0000","last_change":"2023-11-06T03:08:42.965447+0000","last_active":"2023-11-06T03:08:43.311469+0000","last_peered":"2023-11-06T03:08:43.311469+0000","last_clean":"2023-11-06T03:08:43.311469+0000","last_became_active":"2023-11-06T03:08:42.965177+0000","last_became_peered":"2023-11-06T03:08:42.965177+0000","last_unstale":"2023-11-06T03:08:43.311469+0000","last_undegraded":"2023-11-06T03:08:43.311469+0000","last_fullsized":"2023-11-06T03:08:43.311469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:13.483865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.958975+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:08:42.958975+0000","last_peered":"2023-11-06T03:08:42.958975+0000","last_clean":"2023-11-06T03:08:42.958975+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:08:42.958975+0000","last_undegraded":"2023-11-06T03:08:42.958975+0000","last_fullsized":"2023-11-06T03:08:42.958975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.381779+0000","last_change":"2023-11-06T03:08:43.380692+0000","last_active":"2023-11-06T03:08:43.381779+0000","last_peered":"2023-11-06T03:08:43.381779+0000","last_clean":"2023-11-06T03:08:43.381779+0000","last_became_active":"2023-11-06T03:08:43.380191+0000","last_became_peered":"2023-11-06T03:08:43.380191+0000","last_unstale":"2023-11-06T03:08:43.381779+0000","last_undegraded":"2023-11-06T03:08:43.381779+0000","last_fullsized":"2023-11-06T03:08:43.381779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:32:17.267201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.382887+0000","last_change":"2023-11-06T03:08:42.975393+0000","last_active":"2023-11-06T03:08:43.382887+0000","last_peered":"2023-11-06T03:08:43.382887+0000","last_clean":"2023-11-06T03:08:43.382887+0000","last_became_active":"2023-11-06T03:08:42.974827+0000","last_became_peered":"2023-11-06T03:08:42.974827+0000","last_unstale":"2023-11-06T03:08:43.382887+0000","last_undegraded":"2023-11-06T03:08:43.382887+0000","last_fullsized":"2023-11-06T03:08:43.382887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:13:28.549155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.946309+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:08:42.946309+0000","last_peered":"2023-11-06T03:08:42.946309+0000","last_clean":"2023-11-06T03:08:42.946309+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:08:42.946309+0000","last_undegraded":"2023-11-06T03:08:42.946309+0000","last_fullsized":"2023-11-06T03:08:42.946309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.380729+0000","last_change":"2023-11-06T03:08:43.373787+0000","last_active":"2023-11-06T03:08:43.380729+0000","last_peered":"2023-11-06T03:08:43.380729+0000","last_clean":"2023-11-06T03:08:43.380729+0000","last_became_active":"2023-11-06T03:08:43.373177+0000","last_became_peered":"2023-11-06T03:08:43.373177+0000","last_unstale":"2023-11-06T03:08:43.380729+0000","last_undegraded":"2023-11-06T03:08:43.380729+0000","last_fullsized":"2023-11-06T03:08:43.380729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:39.093081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.377035+0000","last_change":"2023-11-06T03:08:42.973579+0000","last_active":"2023-11-06T03:08:43.377035+0000","last_peered":"2023-11-06T03:08:43.377035+0000","last_clean":"2023-11-06T03:08:43.377035+0000","last_became_active":"2023-11-06T03:08:42.972693+0000","last_became_peered":"2023-11-06T03:08:42.972693+0000","last_unstale":"2023-11-06T03:08:43.377035+0000","last_undegraded":"2023-11-06T03:08:43.377035+0000","last_fullsized":"2023-11-06T03:08:43.377035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:36:39.217255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.968026+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:08:42.968026+0000","last_peered":"2023-11-06T03:08:42.968026+0000","last_clean":"2023-11-06T03:08:42.968026+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:08:42.968026+0000","last_undegraded":"2023-11-06T03:08:42.968026+0000","last_fullsized":"2023-11-06T03:08:42.968026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.375902+0000","last_change":"2023-11-06T03:08:43.358006+0000","last_active":"2023-11-06T03:08:43.375902+0000","last_peered":"2023-11-06T03:08:43.375902+0000","last_clean":"2023-11-06T03:08:43.375902+0000","last_became_active":"2023-11-06T03:08:43.356769+0000","last_became_peered":"2023-11-06T03:08:43.356769+0000","last_unstale":"2023-11-06T03:08:43.375902+0000","last_undegraded":"2023-11-06T03:08:43.375902+0000","last_fullsized":"2023-11-06T03:08:43.375902+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:10:48.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.370930+0000","last_change":"2023-11-06T03:08:42.974968+0000","last_active":"2023-11-06T03:08:43.370930+0000","last_peered":"2023-11-06T03:08:43.370930+0000","last_clean":"2023-11-06T03:08:43.370930+0000","last_became_active":"2023-11-06T03:08:42.974291+0000","last_became_peered":"2023-11-06T03:08:42.974291+0000","last_unstale":"2023-11-06T03:08:43.370930+0000","last_undegraded":"2023-11-06T03:08:43.370930+0000","last_fullsized":"2023-11-06T03:08:43.370930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:32:33.728832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.946415+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:08:42.946415+0000","last_peered":"2023-11-06T03:08:42.946415+0000","last_clean":"2023-11-06T03:08:42.946415+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:08:42.946415+0000","last_undegraded":"2023-11-06T03:08:42.946415+0000","last_fullsized":"2023-11-06T03:08:42.946415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.365805+0000","last_change":"2023-11-06T03:08:43.357446+0000","last_active":"2023-11-06T03:08:43.365805+0000","last_peered":"2023-11-06T03:08:43.365805+0000","last_clean":"2023-11-06T03:08:43.365805+0000","last_became_active":"2023-11-06T03:08:43.355144+0000","last_became_peered":"2023-11-06T03:08:43.355144+0000","last_unstale":"2023-11-06T03:08:43.365805+0000","last_undegraded":"2023-11-06T03:08:43.365805+0000","last_fullsized":"2023-11-06T03:08:43.365805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:42.923323+0000","last_clean_scrub_stamp":"2023-11-06T03:08:42.923323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:41.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-06T03:08:43.325989+0000","last_change":"2023-11-06T03:08:42.963378+0000","last_active":"2023-11-06T03:08:43.325989+0000","last_peered":"2023-11-06T03:08:43.325989+0000","last_clean":"2023-11-06T03:08:43.325989+0000","last_became_active":"2023-11-06T03:08:42.963090+0000","last_became_peered":"2023-11-06T03:08:42.963090+0000","last_unstale":"2023-11-06T03:08:43.325989+0000","last_undegraded":"2023-11-06T03:08:43.325989+0000","last_fullsized":"2023-11-06T03:08:43.325989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:08:41.919711+0000","last_clean_scrub_stamp":"2023-11-06T03:08:41.919711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:04:39.483199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-06T03:08:42.947046+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:08:42.947046+0000","last_peered":"2023-11-06T03:08:42.947046+0000","last_clean":"2023-11-06T03:08:42.947046+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:08:42.947046+0000","last_undegraded":"2023-11-06T03:08:42.947046+0000","last_fullsized":"2023-11-06T03:08:42.947046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50468,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321372,"statfs":{"total":96636764160,"available":96585084928,"internally_reserved":0,"allocated":1404928,"data_stored":943870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.48599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85099999999999998,"15min":0.89200000000000002},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.367,"5min":1.397,"15min":1.397},"last":1.0309999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.86899999999999999},"min":{"1min":0.53200000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.444,"15min":1.444},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85599999999999998,"15min":0.89200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.212,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.877,"15min":0.877},"min":{"1min":0.65600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4330000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.96499999999999997},"min":{"1min":0.56399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.222,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.91200000000000003},"min":{"1min":0.76600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.1970000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0069999999999999,"15min":1.0269999999999999},"min":{"1min":0.76900000000000002,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.411,"5min":1.857,"15min":1.857},"last":0.94899999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.97999999999999998},"min":{"1min":0.622,"5min":0.499,"15min":0.499},"max":{"1min":1.123,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.161},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.97199999999999998,"15min":1.0169999999999999},"min":{"1min":0.70899999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.03,"15min":1.046},"min":{"1min":0.66100000000000003,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.754,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0660000000000001,"15min":1.0980000000000001},"min":{"1min":0.751,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.347,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1140000000000001,"15min":1.1559999999999999},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4379999999999999,"5min":1.774,"15min":1.774},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.0860000000000001,"15min":1.1120000000000001},"min":{"1min":0.755,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.476,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45760,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94326080,"statfs":{"total":96636764160,"available":96589905920,"internally_reserved":0,"allocated":778240,"data_stored":344579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78300000000000003,"15min":0.78900000000000003},"min":{"1min":0.56599999999999995,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.036,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.80800000000000005,"15min":0.82599999999999996},"min":{"1min":0.53200000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.23,"5min":2.073,"15min":2.073},"last":0.42599999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.85299999999999998,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.248,"5min":1.778,"15min":1.778},"last":0.61799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.997,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.65700000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2929999999999999,"5min":1.806,"15min":1.806},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93700000000000006,"15min":0.96399999999999997},"min":{"1min":0.63100000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3460000000000001,"5min":1.857,"15min":1.857},"last":0.748},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.91700000000000004,"15min":0.89500000000000002},"min":{"1min":0.81000000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0580000000000001,"15min":1.109},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.4350000000000001,"5min":2.931,"15min":2.931},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.121,"15min":1.157},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.446,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.01,"15min":1.036},"min":{"1min":0.71499999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4079999999999999,"5min":1.506,"15min":1.506},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.113,"15min":1.1000000000000001},"min":{"1min":0.80500000000000005,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5229999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:08:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.1060000000000001},"min":{"1min":0.80800000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4019999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1100000000000001,"15min":1.121},"min":{"1min":0.84099999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":2.0059999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":352748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79000000000000004,"15min":0.81499999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1439999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.49299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":1.175}]},{"osd":1,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.78200000000000003,"15min":0.72599999999999998},"min":{"1min":0.60299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.85599999999999998,"15min":0.85699999999999998},"min":{"1min":0.55700000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.238,"5min":1.357,"15min":1.357},"last":1.014}]},{"osd":2,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.68400000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.24,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84699999999999998,"15min":0.83899999999999997},"min":{"1min":0.54500000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.86899999999999999,"15min":0.83999999999999997},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2729999999999999,"5min":1.403,"15min":1.403},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.94699999999999995,"15min":0.93500000000000005},"min":{"1min":0.58499999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3380000000000001,"5min":1.429,"15min":1.429},"last":0.92100000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.58999999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.96199999999999997,"15min":0.95799999999999996},"min":{"1min":0.64600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2030000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.97299999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0109999999999999,"15min":0.99299999999999999},"min":{"1min":0.66400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":1.357,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.206},{"interface":"front","average":{"1min":1.022,"5min":1.0009999999999999,"15min":1.008},"min":{"1min":0.69799999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.01,"15min":0.98399999999999999},"min":{"1min":0.66600000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.0880000000000001,"15min":1.0820000000000001},"min":{"1min":0.72299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3340000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.135}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49884,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321956,"statfs":{"total":96636764160,"available":96585682944,"internally_reserved":0,"allocated":806912,"data_stored":354290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80000000000000004,"15min":0.82499999999999996},"min":{"1min":0.53500000000000003,"5min":0.372,"15min":0.372},"max":{"1min":1.109,"5min":1.484,"15min":1.484},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.58499999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.137,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.503,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1279999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83399999999999996,"15min":0.80500000000000005},"min":{"1min":0.61799999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.161,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.131}]},{"osd":2,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.98099999999999998,"15min":1.0609999999999999},"min":{"1min":0.69899999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3069999999999999,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.85899999999999999,"15min":0.85399999999999998},"min":{"1min":0.75,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.194,"5min":1.329,"15min":1.329},"last":1.018}]},{"osd":3,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91900000000000004,"15min":0.94299999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.482,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.278,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.103}]},{"osd":5,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1,"15min":1.014},"min":{"1min":0.59499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2589999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.113,"5min":1.085,"15min":1.1100000000000001},"min":{"1min":0.877,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3600000000000001,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.232}]},{"osd":6,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.264,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1160000000000001,"15min":1.1619999999999999},"min":{"1min":0.89500000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.357,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Nov 6 03:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.0720000000000001,"15min":1.0740000000000001},"min":{"1min":0.80300000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2969999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2509999999999999,"15min":1.1899999999999999},"min":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.4410000000000001,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49868,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321972,"statfs":{"total":96636764160,"available":96585699328,"internally_reserved":0,"allocated":790528,"data_stored":351707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.88300000000000001,"5min":1.236,"15min":1.236},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.149,"5min":1.524,"15min":1.524},"last":0.35699999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.95199999999999996,"5min":1.353,"15min":1.353},"last":0.501},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.25700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98999999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.69099999999999995,"15min":0.66900000000000004},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.1419999999999999,"5min":1.349,"15min":1.349},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.29699999999999999,"5min":0.255,"15min":0.255},"max":{"1min":1.0029999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88300000000000001,"15min":0.91100000000000003},"min":{"1min":0.61199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.319,"5min":1.327,"15min":1.327},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92200000000000004,"15min":0.95599999999999996},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1220000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95499999999999996,"15min":0.98699999999999999},"min":{"1min":0.496,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.236,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.52000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3160000000000001,"5min":1.698,"15min":1.698},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98199999999999998,"15min":1.014},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3700000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.96699999999999997,"15min":1.006},"min":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.67100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.47,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.86299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":782336,"data_stored":345598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.23799999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.628,"15min":0.65200000000000002},"min":{"1min":0.29299999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.109,"5min":1.121,"15min":1.121},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.65300000000000002,"15min":0.67700000000000005},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.38300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.752},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.36499999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.236,"5min":1.26,"15min":1.26},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.86799999999999999,"15min":0.89500000000000002},"min":{"1min":0.52400000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.238,"5min":1.385,"15min":1.385},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.92700000000000005,"15min":0.96499999999999997},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.169}]},{"osd":5,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.87,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.252,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95099999999999996,"15min":0.97399999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.411,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.998,"15min":0.998},"min":{"1min":0.73799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.012,"5min":0.95099999999999996,"15min":0.95699999999999996},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0269999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0209999999999999,"15min":1.0049999999999999},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.6859999999999999,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.96099999999999997,"15min":0.97199999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2649999999999999,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":355054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.59799999999999998},"min":{"1min":0.32300000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.997,"5min":1.157,"15min":1.157},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0609999999999999,"5min":1.544,"15min":1.544},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.91800000000000004,"15min":0.72399999999999998},"min":{"1min":0.34599999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.1659999999999999,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66600000000000004,"15min":0.69199999999999995},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1819999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.53800000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71399999999999997,"15min":0.73299999999999998},"min":{"1min":0.40200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0089999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.048,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.874,"15min":0.88200000000000001},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.4319999999999999,"5min":2.258,"15min":2.258},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0549999999999999,"15min":0.95599999999999996},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.736,"5min":14.753,"15min":14.753},"last":0.81399999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.425,"15min":1.151},"min":{"1min":0.54600000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4730000000000001,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.749},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.405,"15min":1.179},"min":{"1min":0.67900000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4279999999999999,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.2170000000000001,"15min":1.089},"min":{"1min":0.47999999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6930000000000001,"5min":23.971,"15min":23.971},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.226,"15min":1.1160000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.617,"5min":14.813000000000001,"15min":14.813000000000001},"last":1.036}]},{"osd":7,"last update":"Mon Nov 6 03:08:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0429999999999999,"15min":1.0860000000000001},"min":{"1min":0.72599999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4159999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.583,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.99299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313212,"statfs":{"total":96636764160,"available":96576729088,"internally_reserved":0,"allocated":1372160,"data_stored":934947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42299999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.89600000000000002,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.438},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72099999999999997,"15min":0.65900000000000003},"min":{"1min":0.309,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.093,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.61699999999999999},"min":{"1min":0.36799999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.006,"5min":1.155,"15min":1.155},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70399999999999996,"15min":0.66000000000000003},"min":{"1min":0.47399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1779999999999999,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97199999999999998,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.751,"15min":0.72499999999999998},"min":{"1min":0.44,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.083,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.98199999999999998,"15min":0.93400000000000005},"min":{"1min":0.68600000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.446,"5min":4.641,"15min":4.641},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.56599999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.286,"5min":2.028,"15min":2.028},"last":0.42899999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.98399999999999999,"15min":0.995},"min":{"1min":0.79100000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4530000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.002,"15min":0.95299999999999996},"min":{"1min":0.66900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.405,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.90700000000000003},"min":{"1min":0.69499999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.232,"5min":2.931,"15min":2.931},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.034,"5min":1.0309999999999999,"15min":1.0189999999999999},"min":{"1min":0.68899999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3899999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.056,"15min":1.018},"min":{"1min":0.65600000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3540000000000001,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.79500000000000004,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.403,"5min":1.97,"15min":1.97},"last":0.64800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:08:46.696 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:08:46.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:08:46.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 Processor -- start 2023-11-06T03:08:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 -- start start 2023-11-06T03:08:46.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d415caa0 0x7f40d40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40d40d0280 0x7f40d40b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40d2920 con 0x7f40d415ce80 2023-11-06T03:08:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40d18a0 con 0x7f40d40d0280 2023-11-06T03:08:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d972a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40d1b60 con 0x7f40d415caa0 2023-11-06T03:08:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40d40d0280 0x7f40d40b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40d40d0280 0x7f40d40b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57962/0 (socket says 172.21.15.157:57962) 2023-11-06T03:08:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 -- 172.21.15.157:0/3947975713 learned_addr learned my addr 172.21.15.157:0/3947975713 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d2ffd640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d415caa0 0x7f40d40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 -- 172.21.15.157:0/3947975713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d415caa0 msgr2=0x7f40d40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d415caa0 0x7f40d40cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 -- 172.21.15.157:0/3947975713 >> v1:172.21.15.195:6789/0 conn(0x7f40d415ce80 legacy=0x7f40d40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.852+0000 7f40d37fe640 1 -- 172.21.15.157:0/3947975713 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40d1e40 con 0x7f40d40d0280 2023-11-06T03:08:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d37fe640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d40d0280 0x7f40d40b5d50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009e50 tx=0x7f40cc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2a7bf22bb85f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d1ffb640 1 -- 172.21.15.157:0/3947975713 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0a5820 con 0x7f40d40d0280 2023-11-06T03:08:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d1ffb640 1 -- 172.21.15.157:0/3947975713 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0a5980 con 0x7f40d40d0280 2023-11-06T03:08:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d1ffb640 1 -- 172.21.15.157:0/3947975713 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0a5d10 con 0x7f40d40d0280 2023-11-06T03:08:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 -- 172.21.15.157:0/3947975713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d40d0280 msgr2=0x7f40d40b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d40d0280 0x7f40d40b5d50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009e50 tx=0x7f40cc01c9a0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 -- 172.21.15.157:0/3947975713 shutdown_connections 2023-11-06T03:08:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d40d0280 0x7f40d40b5d50 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 --2- 172.21.15.157:0/3947975713 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d415caa0 0x7f40d40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 -- 172.21.15.157:0/3947975713 >> 172.21.15.157:0/3947975713 conn(0x7f40d4070790 msgr2=0x7f40d4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 -- 172.21.15.157:0/3947975713 shutdown_connections 2023-11-06T03:08:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.856+0000 7f40d972a640 1 -- 172.21.15.157:0/3947975713 wait complete. 2023-11-06T03:08:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 Processor -- start 2023-11-06T03:08:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- start start 2023-11-06T03:08:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d40d0280 0x7f40d41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 0x7f40d4150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4150f60 con 0x7f40d415ce80 2023-11-06T03:08:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40cace0 con 0x7f40d415caa0 2023-11-06T03:08:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40ca160 con 0x7f40d40d0280 2023-11-06T03:08:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57978/0 (socket says 172.21.15.157:57978) 2023-11-06T03:08:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 -- 172.21.15.157:0/3472716741 learned_addr learned my addr 172.21.15.157:0/3472716741 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d2ffd640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d40d0280 0x7f40d41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d27fc640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 0x7f40d4150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 -- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d40d0280 msgr2=0x7f40d41532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d40d0280 0x7f40d41532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 -- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 msgr2=0x7f40d4150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:46.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 0x7f40d4150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:46.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 -- 172.21.15.157:0/3472716741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc009b30 con 0x7f40d415ce80 2023-11-06T03:08:46.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d27fc640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 0x7f40d4150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:46.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d37fe640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0af040 tx=0x7f40cc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b8670 con 0x7f40d415ce80 2023-11-06T03:08:46.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc0b8810 con 0x7f40d415ce80 2023-11-06T03:08:46.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d40ca440 con 0x7f40d415ce80 2023-11-06T03:08:46.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0b8bc0 con 0x7f40d415ce80 2023-11-06T03:08:46.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d40caa00 con 0x7f40d415ce80 2023-11-06T03:08:46.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d415b650 con 0x7f40d415ce80 2023-11-06T03:08:46.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f40cc0b8d20 con 0x7f40d415ce80 2023-11-06T03:08:46.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 0x7f40c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:46.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40cc11e9b0 con 0x7f40d415ce80 2023-11-06T03:08:46.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.860+0000 7f40d2ffd640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 0x7f40c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:46.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.864+0000 7f40d2ffd640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 0x7f40c0067f20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40d415b5a0 tx=0x7f40c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:46.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:46.864+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f40d415b650 con 0x7f40d415ce80 2023-11-06T03:08:47.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.060+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f40d415b650 con 0x7f40d415ce80 2023-11-06T03:08:47.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.064+0000 7f40bb7fe640 1 -- 172.21.15.157:0/3472716741 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f40d415b650 con 0x7f40d415ce80 2023-11-06T03:08:47.065 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:08:47.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 msgr2=0x7f40c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 0x7f40c0067f20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40d415b5a0 tx=0x7f40c4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:47.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 msgr2=0x7f40d414bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0af040 tx=0x7f40cc019740 comp rx=0 tx=0).stop 2023-11-06T03:08:47.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 shutdown_connections 2023-11-06T03:08:47.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f40c0065a60 0x7f40c0067f20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40d415ce80 0x7f40d414bbf0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f40d415caa0 0x7f40d4150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 --2- 172.21.15.157:0/3472716741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40d40d0280 0x7f40d41532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 >> 172.21.15.157:0/3472716741 conn(0x7f40d4070790 msgr2=0x7f40d40b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 shutdown_connections 2023-11-06T03:08:47.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.068+0000 7f40d972a640 1 -- 172.21.15.157:0/3472716741 wait complete. 2023-11-06T03:08:47.083 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:08:47.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 Processor -- start 2023-11-06T03:08:47.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 -- start start 2023-11-06T03:08:47.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe95415caa0 0x7fe9540cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe95415ce80 0x7fe9540cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9540d2920 con 0x7fe9540ce1a0 2023-11-06T03:08:47.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9540d18a0 con 0x7fe95415ce80 2023-11-06T03:08:47.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe95b8d8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9540d1b60 con 0x7fe95415caa0 2023-11-06T03:08:47.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe95415ce80 0x7fe9540cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe95415ce80 0x7fe9540cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57994/0 (socket says 172.21.15.157:57994) 2023-11-06T03:08:47.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 -- 172.21.15.157:0/721521594 learned_addr learned my addr 172.21.15.157:0/721521594 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe959674640 1 --2- 172.21.15.157:0/721521594 >> v2:172.21.15.195:3300/0 conn(0x7fe95415caa0 0x7fe9540cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 -- 172.21.15.157:0/721521594 >> v2:172.21.15.195:3300/0 conn(0x7fe95415caa0 msgr2=0x7fe9540cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 --2- 172.21.15.157:0/721521594 >> v2:172.21.15.195:3300/0 conn(0x7fe95415caa0 0x7fe9540cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 -- 172.21.15.157:0/721521594 >> v1:172.21.15.195:6789/0 conn(0x7fe9540ce1a0 legacy=0x7fe9540d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.204+0000 7fe958e73640 1 -- 172.21.15.157:0/721521594 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540d1e40 con 0x7fe95415ce80 2023-11-06T03:08:47.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe958e73640 1 --2- 172.21.15.157:0/721521594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe95415ce80 0x7fe9540cfb70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe9540b4f80 tx=0x7fe94401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52aaa881b99096f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe94bfff640 1 -- 172.21.15.157:0/721521594 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440ac600 con 0x7fe95415ce80 2023-11-06T03:08:47.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe94bfff640 1 -- 172.21.15.157:0/721521594 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9440a8050 con 0x7fe95415ce80 2023-11-06T03:08:47.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe94bfff640 1 -- 172.21.15.157:0/721521594 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440ac980 con 0x7fe95415ce80 2023-11-06T03:08:47.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- 172.21.15.157:0/721521594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe95415ce80 msgr2=0x7fe9540cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/721521594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe95415ce80 0x7fe9540cfb70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe9540b4f80 tx=0x7fe94401c490 comp rx=0 tx=0).stop 2023-11-06T03:08:47.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- 172.21.15.157:0/721521594 shutdown_connections 2023-11-06T03:08:47.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/721521594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe95415ce80 0x7fe9540cfb70 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/721521594 >> v2:172.21.15.195:3300/0 conn(0x7fe95415caa0 0x7fe9540cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- 172.21.15.157:0/721521594 >> 172.21.15.157:0/721521594 conn(0x7fe954070790 msgr2=0x7fe954070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- 172.21.15.157:0/721521594 shutdown_connections 2023-11-06T03:08:47.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- 172.21.15.157:0/721521594 wait complete. 2023-11-06T03:08:47.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 Processor -- start 2023-11-06T03:08:47.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- start start 2023-11-06T03:08:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe95415caa0 0x7fe9540cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe95415ce80 0x7fe9540cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9540cc990 con 0x7fe9540ce1a0 2023-11-06T03:08:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe954155450 con 0x7fe95415caa0 2023-11-06T03:08:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.208+0000 7fe95b8d8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe954155730 con 0x7fe95415ce80 2023-11-06T03:08:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57998/0 (socket says 172.21.15.157:57998) 2023-11-06T03:08:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 -- 172.21.15.157:0/3612442813 learned_addr learned my addr 172.21.15.157:0/3612442813 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe958e73640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe95415caa0 0x7fe9540cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959e75640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe95415ce80 0x7fe9540cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 -- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe95415ce80 msgr2=0x7fe9540cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe95415ce80 0x7fe9540cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 -- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe95415caa0 msgr2=0x7fe9540cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe95415caa0 0x7fe9540cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 -- 172.21.15.157:0/3612442813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440094d0 con 0x7fe9540ce1a0 2023-11-06T03:08:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe959674640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d800 tx=0x7fe95000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000b6c0 con 0x7fe9540ce1a0 2023-11-06T03:08:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000b820 con 0x7fe9540ce1a0 2023-11-06T03:08:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe954155a10 con 0x7fe9540ce1a0 2023-11-06T03:08:47.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000bb90 con 0x7fe9540ce1a0 2023-11-06T03:08:47.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9541604b0 con 0x7fe9540ce1a0 2023-11-06T03:08:47.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe954072750 con 0x7fe9540ce1a0 2023-11-06T03:08:47.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe95000bcf0 con 0x7fe9540ce1a0 2023-11-06T03:08:47.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 0x7fe938067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe950083880 con 0x7fe9540ce1a0 2023-11-06T03:08:47.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe958e73640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 0x7fe938067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.212+0000 7fe958e73640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 0x7fe938067f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9440ae000 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.216+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9500510e0 con 0x7fe9540ce1a0 2023-11-06T03:08:47.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.392+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9540c9340 con 0x7fe9540ce1a0 2023-11-06T03:08:47.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.392+0000 7fe949ffb640 1 -- 172.21.15.157:0/3612442813 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fe950011340 con 0x7fe9540ce1a0 2023-11-06T03:08:47.394 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:47.394 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2083483222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2083483222},{"type":"v1","addr":"172.21.15.195:6835","nonce":2083483222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5752,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3999445109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3999445109},{"type":"v1","addr":"172.21.15.195:6837","nonce":3999445109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1572709083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1572709083},{"type":"v1","addr":"172.21.15.157:6833","nonce":1572709083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:08:45.318890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1218323165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1218323165},{"type":"v1","addr":"172.21.15.157:6835","nonce":1218323165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:08:47.394 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 47 2023-11-06T03:08:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 msgr2=0x7fe938067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 0x7fe938067f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe9440ae000 tx=0x7fe944008040 comp rx=0 tx=0).stop 2023-11-06T03:08:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 msgr2=0x7fe9540c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d800 tx=0x7fe95000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 shutdown_connections 2023-11-06T03:08:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe938065a60 0x7fe938067f20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe95415ce80 0x7fe9540cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe95415caa0 0x7fe9540cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 --2- 172.21.15.157:0/3612442813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9540ce1a0 0x7fe9540c6ab0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 >> 172.21.15.157:0/3612442813 conn(0x7fe954070790 msgr2=0x7fe9540d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 shutdown_connections 2023-11-06T03:08:47.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.396+0000 7fe95b8d8640 1 -- 172.21.15.157:0/3612442813 wait complete. 2023-11-06T03:08:47.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:08:43.305986+0000', 'modified': '2023-11-06T03:08:45.318890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6741}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6741': {'gid': 6741, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1218323165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1218323165}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1218323165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:08:47.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:08:47.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:08:47.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 Processor -- start 2023-11-06T03:08:47.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 -- start start 2023-11-06T03:08:47.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03e81599c0 0x7f03e80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e815e440 con 0x7f03e806d000 2023-11-06T03:08:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e806c240 con 0x7f03e81599c0 2023-11-06T03:08:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03f04ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e806c520 con 0x7f03e80b2af0 2023-11-06T03:08:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03ee24a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03e81599c0 0x7f03e80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:52870/0 (socket says 172.21.15.157:52870) 2023-11-06T03:08:47.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 -- 172.21.15.157:0/997339938 learned_addr learned my addr 172.21.15.157:0/997339938 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 -- 172.21.15.157:0/997339938 >> v2:172.21.15.195:3300/0 conn(0x7f03e81599c0 msgr2=0x7f03e80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 --2- 172.21.15.157:0/997339938 >> v2:172.21.15.195:3300/0 conn(0x7f03e81599c0 0x7f03e80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 -- 172.21.15.157:0/997339938 >> v1:172.21.15.195:6789/0 conn(0x7f03e806d000 legacy=0x7f03e80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.944+0000 7f03eea4b640 1 -- 172.21.15.157:0/997339938 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e806c800 con 0x7f03e80b2af0 2023-11-06T03:08:47.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03eea4b640 1 --2- 172.21.15.157:0/997339938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009e50 tx=0x7f03e001c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc11cce91903d82a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03ed248640 1 -- 172.21.15.157:0/997339938 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00a58d0 con 0x7f03e80b2af0 2023-11-06T03:08:47.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03ed248640 1 -- 172.21.15.157:0/997339938 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00a5a30 con 0x7f03e80b2af0 2023-11-06T03:08:47.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03ed248640 1 -- 172.21.15.157:0/997339938 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00a5dc0 con 0x7f03e80b2af0 2023-11-06T03:08:47.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 -- 172.21.15.157:0/997339938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 msgr2=0x7f03e8159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 --2- 172.21.15.157:0/997339938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009e50 tx=0x7f03e001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 -- 172.21.15.157:0/997339938 shutdown_connections 2023-11-06T03:08:47.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 --2- 172.21.15.157:0/997339938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e80b2af0 0x7f03e8159da0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 --2- 172.21.15.157:0/997339938 >> v2:172.21.15.195:3300/0 conn(0x7f03e81599c0 0x7f03e80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 -- 172.21.15.157:0/997339938 >> 172.21.15.157:0/997339938 conn(0x7f03e8065930 msgr2=0x7f03e8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 -- 172.21.15.157:0/997339938 shutdown_connections 2023-11-06T03:08:47.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.948+0000 7f03f04ae640 1 -- 172.21.15.157:0/997339938 wait complete. 2023-11-06T03:08:47.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 Processor -- start 2023-11-06T03:08:47.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- start start 2023-11-06T03:08:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 0x7f03e8150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 0x7f03e814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80cc4b0 con 0x7f03e81599c0 2023-11-06T03:08:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8155af0 con 0x7f03e80b2af0 2023-11-06T03:08:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80cc700 con 0x7f03e806d000 2023-11-06T03:08:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58000/0 (socket says 172.21.15.157:58000) 2023-11-06T03:08:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 -- 172.21.15.157:0/933852902 learned_addr learned my addr 172.21.15.157:0/933852902 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03ee24a640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 0x7f03e8150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eda49640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 0x7f03e814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 -- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 msgr2=0x7f03e8150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 0x7f03e8150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 -- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 msgr2=0x7f03e814d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 0x7f03e814d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 -- 172.21.15.157:0/933852902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e0009b30 con 0x7f03e81599c0 2023-11-06T03:08:47.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03ee24a640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 0x7f03e8150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:47.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eda49640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 0x7f03e814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:47.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03eea4b640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f03e001a610 tx=0x7f03e0019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03deffd640 1 -- 172.21.15.157:0/933852902 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00b8670 con 0x7f03e81599c0 2023-11-06T03:08:47.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03deffd640 1 -- 172.21.15.157:0/933852902 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00b87f0 con 0x7f03e81599c0 2023-11-06T03:08:47.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80cc9e0 con 0x7f03e81599c0 2023-11-06T03:08:47.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03deffd640 1 -- 172.21.15.157:0/933852902 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00b8ba0 con 0x7f03e81599c0 2023-11-06T03:08:47.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e8154bf0 con 0x7f03e81599c0 2023-11-06T03:08:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.952+0000 7f03deffd640 1 -- 172.21.15.157:0/933852902 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f03e00b8d00 con 0x7f03e81599c0 2023-11-06T03:08:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03deffd640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 0x7f03cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03deffd640 1 -- 172.21.15.157:0/933852902 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03e011e900 con 0x7f03e81599c0 2023-11-06T03:08:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03ee24a640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 0x7f03cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 Processor -- start 2023-11-06T03:08:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03ee24a640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 0x7f03cc067f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b10 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- start start 2023-11-06T03:08:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de506e0 0x56316de50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dccefc0 con 0x56316de06a90 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dc5d1a0 con 0x56316de506e0 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56316dc5a770 con 0x56316de51000 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58004/0 (socket says 172.21.15.157:58004) 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 -- 172.21.15.157:0/1699180347 learned_addr learned my addr 172.21.15.157:0/1699180347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eda49640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03ee24a640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de506e0 0x56316de50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 -- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 msgr2=0x56316de06550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de06550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 -- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de506e0 msgr2=0x56316de50ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de506e0 0x56316de50ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 -- 172.21.15.157:0/1699180347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56316d952890 con 0x56316de06a90 2023-11-06T03:08:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eda49640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de06550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f03e00ec480 tx=0x7f03e001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f5d4171a98d3dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03cbfff640 1 -- 172.21.15.157:0/1699180347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00f7f60 con 0x56316de06a90 2023-11-06T03:08:47.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03cbfff640 1 -- 172.21.15.157:0/1699180347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00ee320 con 0x56316de06a90 2023-11-06T03:08:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03cbfff640 1 -- 172.21.15.157:0/1699180347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00f01e0 con 0x56316de06a90 2023-11-06T03:08:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- 172.21.15.157:0/1699180347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 msgr2=0x56316de06e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f03e00ec480 tx=0x7f03e001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- 172.21.15.157:0/1699180347 shutdown_connections 2023-11-06T03:08:47.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de06a90 0x56316de06e70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de06550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1699180347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de506e0 0x56316de50ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- 172.21.15.157:0/1699180347 >> 172.21.15.157:0/1699180347 conn(0x56316de0cbf0 msgr2=0x56316de04e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- 172.21.15.157:0/1699180347 shutdown_connections 2023-11-06T03:08:47.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.956+0000 7f03f35d6000 1 -- 172.21.15.157:0/1699180347 wait complete. 2023-11-06T03:08:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- start start 2023-11-06T03:08:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 Processor -- start 2023-11-06T03:08:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de11880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de506e0 0x56316de11dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dc56060 con 0x56316de51000 2023-11-06T03:08:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dbf5290 con 0x56316de06a90 2023-11-06T03:08:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56316dc136f0 con 0x56316de506e0 2023-11-06T03:08:47.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 10 client.0 _ll_register_callbacks cb 0x56316ddf21d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58008/0 (socket says 172.21.15.157:58008) 2023-11-06T03:08:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 -- 172.21.15.157:0/1636286902 learned_addr learned my addr 172.21.15.157:0/1636286902 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 -- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de506e0 msgr2=0x56316de11dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de506e0 0x56316de11dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 -- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 msgr2=0x56316de11880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de11880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 -- 172.21.15.157:0/1636286902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e8154bf0 con 0x56316de51000 2023-11-06T03:08:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03eea4b640 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f03e00efc30 tx=0x7f03e00a59c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b934a3b3790fa2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:47.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e010c240 con 0x56316de51000 2023-11-06T03:08:47.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03e010de80 con 0x56316de51000 2023-11-06T03:08:47.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56316dc549b0 con 0x56316de51000 2023-11-06T03:08:47.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03e010c3a0 con 0x56316de51000 2023-11-06T03:08:47.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 10 client.6885 fetch_fsmap learned FSMap version 47 2023-11-06T03:08:47.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x56316dc1b1e0 con 0x56316de51000 2023-11-06T03:08:47.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56316dc549b0 con 0x56316de51000 2023-11-06T03:08:47.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f03e0113020 con 0x56316de51000 2023-11-06T03:08:47.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 10 client.6885 fetch_fsmap finished waiting for FSMap version 47 2023-11-06T03:08:47.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 10 client.6885 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:47.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 20 client.6885 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:47.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316de5f770 0x56316de61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 4 client.6885 mds_command: new command op to 6741 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:08:47.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 --> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56316dc854d0 con 0x56316de5f770 2023-11-06T03:08:47.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03ee24a640 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316de5f770 0x56316de61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03ee24a640 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316de5f770 0x56316de61b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f59fa583b1e194e server_cookie=4a59789bc695c24f in_seq=0 out_seq=0 2023-11-06T03:08:47.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 10 client.6885 ms_handle_connect on v2:172.21.15.157:6834/1218323165 2023-11-06T03:08:47.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 1 -- 172.21.15.157:0/1636286902 <== mds.0 v2:172.21.15.157:6834/1218323165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56316dc854d0 con 0x56316de5f770 2023-11-06T03:08:47.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 10 client.6885 handle_command_reply: tid=0 2023-11-06T03:08:47.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 client.6885 shutdown 2023-11-06T03:08:47.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 msgr2=0x56316de12300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f03e00efc30 tx=0x7f03e00a59c0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 shutdown_connections 2023-11-06T03:08:47.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316de5f770 0x56316de61b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de51000 0x56316de12300 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de506e0 0x56316de11dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03c97fa640 0 client.6885 ms_handle_reset on v2:172.21.15.157:6834/1218323165 2023-11-06T03:08:47.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 --2- 172.21.15.157:0/1636286902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de11880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 >> 172.21.15.157:0/1636286902 conn(0x56316de0cbf0 msgr2=0x56316de05530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 shutdown_connections 2023-11-06T03:08:47.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 1 -- 172.21.15.157:0/1636286902 wait complete. 2023-11-06T03:08:47.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.960+0000 7f03f35d6000 20 client.6885 trim_cache size 0 max 16384 2023-11-06T03:08:47.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 Processor -- start 2023-11-06T03:08:47.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 -- start start 2023-11-06T03:08:47.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de63180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316d913940 con 0x56316de506e0 2023-11-06T03:08:47.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dc23bb0 con 0x56316de06a90 2023-11-06T03:08:47.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56316dc187f0 con 0x56316de51000 2023-11-06T03:08:47.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03eda49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03eda49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58014/0 (socket says 172.21.15.157:58014) 2023-11-06T03:08:47.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03eda49640 1 -- 172.21.15.157:0/500727773 learned_addr learned my addr 172.21.15.157:0/500727773 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03ee24a640 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de63180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03eea4b640 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.976+0000 7f03eda49640 1 -- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 msgr2=0x56316de48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de48220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 msgr2=0x56316de63180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de63180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/500727773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56316dc01330 con 0x56316de506e0 2023-11-06T03:08:47.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f03e806bc20 tx=0x7f03d8001880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8220bfdc5fa75a9d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03dcff9640 1 -- 172.21.15.157:0/500727773 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80052e0 con 0x56316de506e0 2023-11-06T03:08:47.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03dcff9640 1 -- 172.21.15.157:0/500727773 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800d050 con 0x56316de506e0 2023-11-06T03:08:47.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03dcff9640 1 -- 172.21.15.157:0/500727773 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8003600 con 0x56316de506e0 2023-11-06T03:08:47.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/500727773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 msgr2=0x56316de636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:47.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f03e806bc20 tx=0x7f03d8001880 comp rx=0 tx=0).stop 2023-11-06T03:08:47.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/500727773 shutdown_connections 2023-11-06T03:08:47.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de51000 0x56316de48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316de636c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- 172.21.15.157:0/500727773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de06a90 0x56316de63180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/500727773 >> 172.21.15.157:0/500727773 conn(0x56316de0cbf0 msgr2=0x56316de62a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:47.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/500727773 shutdown_connections 2023-11-06T03:08:47.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/500727773 wait complete. 2023-11-06T03:08:47.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- start start 2023-11-06T03:08:47.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 Processor -- start 2023-11-06T03:08:47.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 0x56316deacba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de51000 0x56316dea91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:47.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dbfcd90 con 0x56316de506e0 2023-11-06T03:08:47.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56316dc136d0 con 0x56316de51000 2023-11-06T03:08:47.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56316dc1d7d0 con 0x56316de06a90 2023-11-06T03:08:47.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 10 client.0 _ll_register_callbacks cb 0x56316ddd29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:08:47.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58030/0 (socket says 172.21.15.157:58030) 2023-11-06T03:08:47.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/3160322519 learned_addr learned my addr 172.21.15.157:0/3160322519 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:47.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03ee24a640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 0x56316deacba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eea4b640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de51000 0x56316dea91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 msgr2=0x56316deacba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 0x56316deacba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de51000 msgr2=0x56316dea91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de51000 0x56316dea91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:47.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 -- 172.21.15.157:0/3160322519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e815e440 con 0x56316de506e0 2023-11-06T03:08:47.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03ee24a640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 0x56316deacba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:47.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03eda49640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f03d80018c0 tx=0x7f03d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab55b848643ac09 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:47.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:08:47.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8026470 con 0x56316de506e0 2023-11-06T03:08:47.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03d80265d0 con 0x56316de506e0 2023-11-06T03:08:47.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56316dc25b90 con 0x56316de506e0 2023-11-06T03:08:47.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03d80267d0 con 0x56316de506e0 2023-11-06T03:08:47.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 10 client.6891 fetch_fsmap learned FSMap version 47 2023-11-06T03:08:47.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x56316d952cc0 con 0x56316de506e0 2023-11-06T03:08:47.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56316dc25b90 con 0x56316de506e0 2023-11-06T03:08:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 47) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f03d8012070 con 0x56316de506e0 2023-11-06T03:08:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 10 client.6891 fetch_fsmap finished waiting for FSMap version 47 2023-11-06T03:08:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 10 client.6891 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:08:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 20 client.6891 populate_metadata read hostname 'smithi157' 2023-11-06T03:08:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316dea9720 0x56316dea9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 4 client.6891 mds_command: new command op to 6741 tid=0[{"prefix": "status"}] 2023-11-06T03:08:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 --> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56316dc4f0e0 con 0x56316dea9720 2023-11-06T03:08:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.980+0000 7f03ee24a640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316dea9720 0x56316dea9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03c97fa640 10 client.6891 ms_handle_connect on v2:172.21.15.157:6834/1218323165 2023-11-06T03:08:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03ee24a640 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316dea9720 0x56316dea9b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50df6fc6b6ca3c6e server_cookie=abf80dfda4df815d in_seq=0 out_seq=0 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03c97fa640 1 -- 172.21.15.157:0/3160322519 <== mds.0 v2:172.21.15.157:6834/1218323165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56316dc4f0e0 con 0x56316dea9720 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03c97fa640 10 client.6891 handle_command_reply: tid=0 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 client.6891 shutdown 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 msgr2=0x56316dead0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f03d80018c0 tx=0x7f03d8009040 comp rx=0 tx=0).stop 2023-11-06T03:08:48.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 shutdown_connections 2023-11-06T03:08:48.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:6834/1218323165,v1:172.21.15.157:6835/1218323165] conn(0x56316dea9720 0x56316dea9b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56316de51000 0x56316dea91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56316de506e0 0x56316dead0e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03c97fa640 0 client.6891 ms_handle_reset on v2:172.21.15.157:6834/1218323165 2023-11-06T03:08:48.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 --2- 172.21.15.157:0/3160322519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56316de06a90 0x56316deacba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 >> 172.21.15.157:0/3160322519 conn(0x56316de0cbf0 msgr2=0x56316de62a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:48.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 shutdown_connections 2023-11-06T03:08:48.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 1 -- 172.21.15.157:0/3160322519 wait complete. 2023-11-06T03:08:48.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f35d6000 20 client.6891 trim_cache size 0 max 16384 2023-11-06T03:08:48.005 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:48.005 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:08:48.005 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:08:48.005 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6741, 2023-11-06T03:08:48.005 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:08:48.006 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:08:48.006 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:08:48.006 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.6765755349999996, 2023-11-06T03:08:48.006 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 47, 2023-11-06T03:08:48.006 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 67, 2023-11-06T03:08:48.007 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 67, 2023-11-06T03:08:48.007 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 9.1297234370000009 2023-11-06T03:08:48.007 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 msgr2=0x7f03cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 0x7f03cc067f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b10 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 msgr2=0x7f03e814d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f03e001a610 tx=0x7f03e0019780 comp rx=0 tx=0).stop 2023-11-06T03:08:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 shutdown_connections 2023-11-06T03:08:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03cc065a60 0x7f03cc067f20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e81599c0 0x7f03e814d900 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03e80b2af0 0x7f03e814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 --2- 172.21.15.157:0/933852902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03e806d000 0x7f03e8150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 >> 172.21.15.157:0/933852902 conn(0x7f03e8065930 msgr2=0x7f03e815be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 shutdown_connections 2023-11-06T03:08:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:47.984+0000 7f03f04ae640 1 -- 172.21.15.157:0/933852902 wait complete. 2023-11-06T03:08:48.011 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:08:48.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 Processor -- start 2023-11-06T03:08:48.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 -- start start 2023-11-06T03:08:48.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5e80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff5e815ce80 0x7ff5e80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80d2920 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80d18a0 con 0x7ff5e815ce80 2023-11-06T03:08:48.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ef5b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d1b60 con 0x7ff5e815caa0 2023-11-06T03:08:48.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5edb52640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff5e80ce1a0 0x7ff5e80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53218/0 (socket says 172.21.15.157:53218) 2023-11-06T03:08:48.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5edb52640 1 -- 172.21.15.157:0/813452127 learned_addr learned my addr 172.21.15.157:0/813452127 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:48.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/813452127 >> v2:172.21.15.195:3300/0 conn(0x7ff5e815ce80 0x7ff5e80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5e80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261839751 0 0) 0x7ff5e80d2920 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d4003530 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323115002 0 0) 0x7ff5d4003530 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 msgr2=0x7ff5e80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5e80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 >> v2:172.21.15.195:3300/0 conn(0x7ff5e815ce80 msgr2=0x7ff5e80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 --2- 172.21.15.157:0/813452127 >> v2:172.21.15.195:3300/0 conn(0x7ff5e815ce80 0x7ff5e80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80d1e40 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff5e0001a00 con 0x7ff5e80ce1a0 2023-11-06T03:08:48.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5e80cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:48.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/813452127 >> v2:172.21.15.195:3300/0 conn(0x7ff5e815ce80 0x7ff5e80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:48.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 >> v1:172.21.15.157:6789/0 conn(0x7ff5e80ce1a0 legacy=0x7ff5e80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4003980 0x7ff5d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5d40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d4004d10 0x7ff5d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d18a0 con 0x7ff5d4003980 2023-11-06T03:08:48.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d1b60 con 0x7ff5d4004d10 2023-11-06T03:08:48.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d400ba10 con 0x7ff5e815caa0 2023-11-06T03:08:48.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4003980 0x7ff5d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5d40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 -- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 msgr2=0x7ff5d40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:48.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5edb52640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d4004d10 0x7ff5d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5d40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 -- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d4004d10 msgr2=0x7ff5d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:48.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5d4004d10 0x7ff5d4005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ed351640 1 -- 172.21.15.157:0/813452127 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4003530 con 0x7ff5d4003980 2023-11-06T03:08:48.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.128+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e815caa0 0x7ff5d40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:48.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ed351640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4003980 0x7ff5d4003da0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098d0 tx=0x7ff5e401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af26e38b87b12b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:48.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40aa620 con 0x7ff5d4003980 2023-11-06T03:08:48.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40aa780 con 0x7ff5d4003980 2023-11-06T03:08:48.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5dffff640 1 -- 172.21.15.157:0/813452127 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40aaaf0 con 0x7ff5d4003980 2023-11-06T03:08:48.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/813452127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4003980 msgr2=0x7ff5d4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/813452127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d4003980 0x7ff5d4003da0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098d0 tx=0x7ff5e401c580 comp rx=0 tx=0).stop 2023-11-06T03:08:48.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 -- 172.21.15.157:0/813452127 reap_dead start 2023-11-06T03:08:48.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/813452127 shutdown_connections 2023-11-06T03:08:48.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/813452127 >> 172.21.15.157:0/813452127 conn(0x7ff5e8070790 msgr2=0x7ff5e8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:48.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/813452127 shutdown_connections 2023-11-06T03:08:48.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/813452127 wait complete. 2023-11-06T03:08:48.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 Processor -- start 2023-11-06T03:08:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- start start 2023-11-06T03:08:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 0x7ff5e8156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 0x7ff5e80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80c9120 con 0x7ff5e80c8760 2023-11-06T03:08:48.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815eb20 con 0x7ff5e815fea0 2023-11-06T03:08:48.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ef5b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815ee00 con 0x7ff5d4003980 2023-11-06T03:08:48.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58044/0 (socket says 172.21.15.157:58044) 2023-11-06T03:08:48.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 -- 172.21.15.157:0/4248663598 learned_addr learned my addr 172.21.15.157:0/4248663598 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:48.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 0x7ff5e80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ed351640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 0x7ff5e8156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 -- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 msgr2=0x7ff5e8156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 0x7ff5e8156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 -- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 msgr2=0x7ff5e80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 0x7ff5e80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 -- 172.21.15.157:0/4248663598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e815f0e0 con 0x7ff5e80c8760 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ed351640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 0x7ff5e8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5ecb50640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 0x7ff5e80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.132+0000 7ff5edb52640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b49b0 tx=0x7ff5e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:48.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0004d70 con 0x7ff5e80c8760 2023-11-06T03:08:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000bb80 con 0x7ff5e80c8760 2023-11-06T03:08:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e815f3c0 con 0x7ff5e80c8760 2023-11-06T03:08:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0007070 con 0x7ff5e80c8760 2023-11-06T03:08:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e815f980 con 0x7ff5e80c8760 2023-11-06T03:08:48.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8072750 con 0x7ff5e80c8760 2023-11-06T03:08:48.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff5e00071d0 con 0x7ff5e80c8760 2023-11-06T03:08:48.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 0x7ff5cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:48.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5e00835a0 con 0x7ff5e80c8760 2023-11-06T03:08:48.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ed351640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 0x7ff5cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:48.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.136+0000 7ff5ed351640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 0x7ff5cc067f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40ae000 tx=0x7ff5e40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:48.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.140+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff5e0050e00 con 0x7ff5e80c8760 2023-11-06T03:08:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.332+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5e80b17c0 con 0x7ff5e80c8760 2023-11-06T03:08:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.332+0000 7ff5ddffb640 1 -- 172.21.15.157:0/4248663598 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5e0054420 con 0x7ff5e80c8760 2023-11-06T03:08:48.333 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:08:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 msgr2=0x7ff5cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 0x7ff5cc067f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40ae000 tx=0x7ff5e40046d0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 msgr2=0x7ff5e80c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80b49b0 tx=0x7ff5e000b4a0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 shutdown_connections 2023-11-06T03:08:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff5cc065a60 0x7ff5cc067f20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80c8760 0x7ff5e80c8be0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e815fea0 0x7ff5e80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 --2- 172.21.15.157:0/4248663598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5d4003980 0x7ff5e8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 >> 172.21.15.157:0/4248663598 conn(0x7ff5e8070790 msgr2=0x7ff5e81607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 shutdown_connections 2023-11-06T03:08:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:48.336+0000 7ff5ef5b5640 1 -- 172.21.15.157:0/4248663598 wait complete. 2023-11-06T03:08:48.350 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:08:48.350 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:08:48.351 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:08:48.351 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:08:48.351 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:48.352 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:08:48.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:08:48.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:08:48.353 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:08:48.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.354 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:08:48.360 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:08:48.360 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:08:48.360 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:08:48.360 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.360 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:08:48.361 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.361 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:08:48.361 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.361 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:08:48.361 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:08:48.362 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.362 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:08:48.362 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:08:48.362 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.362 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:08:48.363 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.363 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:08:48.363 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42211sec preferred_lft 42211sec 2023-11-06T03:08:48.363 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:08:48.363 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.364 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:08:48.364 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.364 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:08:48.364 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.364 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.365 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:08:48.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:08:48.366 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:08:48.366 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.366 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:08:48.366 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:08:48.367 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:08:48.367 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:08:48.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.368 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:08:48.412 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:08:48.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.414 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:08:48.465 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:08:48.465 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:08:48.465 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:08:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:08:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:08:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:08:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:08:48.466 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:08:48.467 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:08:48.467 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:08:48.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:08:48.468 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:08:48.468 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:08:48.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:08:48.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:08:48.470 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:08:48.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.537 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:08:48.583 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:08:48.583 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:08:48.584 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:08:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:08:48.639 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:08:48.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:08:48.640 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:48.687 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:08:48.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:08:48.688 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:48.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:08:48.796 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:08:48.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:08:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:08:48.851 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:08:48.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:48.852 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:08:48.899 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:08:48.899 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:08:48.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:08:48.958 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:08:48.958 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:08:49.013 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:08:49.014 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:08:49.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:49.015 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:08:49.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:08:49.024+0000 7fb1c129c580 -1 init, newargv = 0x55d9492790a0 newargc=15 2023-11-06T03:08:49.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65176]: starting ceph client 2023-11-06T03:08:49.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65176]: starting fuse 2023-11-06T03:08:50.130 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:08:50.147 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:08:50.147 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:08:50.202 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:08:50.203 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:08:50.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.204 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:08:50.251 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:08:50.251 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:08:50.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:08:50.252 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:08:50.253 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:08:50.253 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:08:50.254 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:08:50.254 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:50.254 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:08:50.255 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:08:50.255 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:08:50.256 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:08:50.256 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:50.256 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:08:50.257 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:08:50.257 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:08:50.257 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:08:50.258 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:50.258 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:08:50.258 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:08:50.259 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:08:50.260 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:08:50.260 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:08:50.261 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:08:50.261 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:08:50.262 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:08:50.262 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:50.263 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:08:50.263 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:08:50.375 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.65176.asok 2023-11-06T03:08:50.555 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65176.asok 2023-11-06T03:08:50.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65176.asok status 2023-11-06T03:08:50.714 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:08:50.715 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:08:50.715 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:08:50.715 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:08:50.715 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:08:50.716 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:08:50.716 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:08:50.716 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "65176", 2023-11-06T03:08:50.716 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:08:50.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:50.717 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:08:50.717 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:08:50.717 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 6903, 2023-11-06T03:08:50.717 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:08:50.717 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:08:50.718 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:08:50.718 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 6903 2023-11-06T03:08:50.718 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:50.718 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:08:50.718 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:08:50.719 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:08:50.719 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1909499590 2023-11-06T03:08:50.719 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:08:50.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:50.719 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:08:50.720 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:08:50.720 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:08:50.720 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1909499590 2023-11-06T03:08:50.720 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:08:50.720 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.6903 192.168.0.1:0/1909499590", 2023-11-06T03:08:50.721 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1909499590", 2023-11-06T03:08:50.721 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:08:50.721 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 47, 2023-11-06T03:08:50.721 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 67, 2023-11-06T03:08:50.721 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:08:50.722 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:08:50.722 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:08:50.722 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:08:50.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.729 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:08:50.729 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:50.736 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:08:50.736 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:50.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.791 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2023-11-06T03:08:50.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:50.849 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2023-11-06T03:08:50.900 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:08:50.901 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:08:50.901 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:08:50.902 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:08:50.902 DEBUG:teuthology.orchestra.run.smithi157:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2023-11-06T03:08:50.902 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:08:51.005 INFO:teuthology.orchestra.run.smithi157.stdout:1099511627776 2023-11-06T03:08:51.193 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:51.194 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:51.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:51.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 Processor -- start 2023-11-06T03:08:51.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 -- start start 2023-11-06T03:08:51.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f984c1599c0 0x7f984c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984c15e440 con 0x7f984c15dea0 2023-11-06T03:08:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c15adb0 con 0x7f984c1599c0 2023-11-06T03:08:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f985258b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06c240 con 0x7f984c06d000 2023-11-06T03:08:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f984c1599c0 0x7f984c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f9850b28640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f984c15dea0 0x7f984c070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37098/0 (socket says 172.21.15.157:37098) 2023-11-06T03:08:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f9850b28640 1 -- 172.21.15.157:0/2196482278 learned_addr learned my addr 172.21.15.157:0/2196482278 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984b7fe640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984bfff640 1 -- 172.21.15.157:0/2196482278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 msgr2=0x7f984c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984bfff640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984bfff640 1 -- 172.21.15.157:0/2196482278 >> v1:172.21.15.157:6789/0 conn(0x7f984c15dea0 legacy=0x7f984c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.416+0000 7f984bfff640 1 -- 172.21.15.157:0/2196482278 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c06c520 con 0x7f984c1599c0 2023-11-06T03:08:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984bfff640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c0ae000 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f983c0097c0 tx=0x7f983c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95c134f32820ece server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984affd640 1 -- 172.21.15.157:0/2196482278 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0ad660 con 0x7f984c1599c0 2023-11-06T03:08:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984affd640 1 -- 172.21.15.157:0/2196482278 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c0ad7c0 con 0x7f984c1599c0 2023-11-06T03:08:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984affd640 1 -- 172.21.15.157:0/2196482278 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0adb50 con 0x7f984c1599c0 2023-11-06T03:08:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- 172.21.15.157:0/2196482278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 msgr2=0x7f984c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c0ae000 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f983c0097c0 tx=0x7f983c01c450 comp rx=0 tx=0).stop 2023-11-06T03:08:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- 172.21.15.157:0/2196482278 shutdown_connections 2023-11-06T03:08:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- 172.21.15.157:0/2196482278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c0ae000 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- 172.21.15.157:0/2196482278 >> 172.21.15.157:0/2196482278 conn(0x7f984c065930 msgr2=0x7f984c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- 172.21.15.157:0/2196482278 shutdown_connections 2023-11-06T03:08:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- 172.21.15.157:0/2196482278 wait complete. 2023-11-06T03:08:51.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 Processor -- start 2023-11-06T03:08:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- start start 2023-11-06T03:08:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f984c15dea0 0x7f984c14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c157b00 con 0x7f984c1599c0 2023-11-06T03:08:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c14f4c0 con 0x7f984c15dea0 2023-11-06T03:08:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f985258b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c157160 con 0x7f984c06d000 2023-11-06T03:08:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53986/0 (socket says 172.21.15.157:53986) 2023-11-06T03:08:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984b7fe640 1 -- 172.21.15.157:0/63975076 learned_addr learned my addr 172.21.15.157:0/63975076 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:51.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f984bfff640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.420+0000 7f9850b28640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f984c15dea0 0x7f984c14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 -- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 msgr2=0x7f984c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 -- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f984c15dea0 msgr2=0x7f984c14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f984c15dea0 0x7f984c14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 -- 172.21.15.157:0/63975076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c009420 con 0x7f984c1599c0 2023-11-06T03:08:51.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984b7fe640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f984000d830 tx=0x7f984000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000b6c0 con 0x7f984c1599c0 2023-11-06T03:08:51.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984000b820 con 0x7f984c1599c0 2023-11-06T03:08:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000bb90 con 0x7f984c1599c0 2023-11-06T03:08:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c157440 con 0x7f984c1599c0 2023-11-06T03:08:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c150510 con 0x7f984c1599c0 2023-11-06T03:08:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c072750 con 0x7f984c1599c0 2023-11-06T03:08:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f984001e460 con 0x7f984c1599c0 2023-11-06T03:08:51.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 0x7f9838067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9840083ab0 con 0x7f984c1599c0 2023-11-06T03:08:51.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984bfff640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 0x7f9838067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.424+0000 7f984bfff640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 0x7f9838067f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f983c0af000 tx=0x7f983c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.428+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9840051310 con 0x7f984c1599c0 2023-11-06T03:08:51.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.584+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f984c0adcd0 con 0x7f984c1599c0 2023-11-06T03:08:51.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.584+0000 7f9848ff9640 1 -- 172.21.15.157:0/63975076 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18112 (secure 0 0 0) 0x7f9840054930 con 0x7f984c1599c0 2023-11-06T03:08:51.586 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:51.586 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":69,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:50.340995+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:51.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 msgr2=0x7f9838067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 0x7f9838067f20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f983c0af000 tx=0x7f983c008040 comp rx=0 tx=0).stop 2023-11-06T03:08:51.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 msgr2=0x7f984c14ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f984000d830 tx=0x7f984000dcf0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 shutdown_connections 2023-11-06T03:08:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9838065a60 0x7f9838067f20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f984c15dea0 0x7f984c14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f984c1599c0 0x7f984c14ea40 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 --2- 172.21.15.157:0/63975076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f984c06d000 0x7f984c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 >> 172.21.15.157:0/63975076 conn(0x7f984c065930 msgr2=0x7f984c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:51.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 shutdown_connections 2023-11-06T03:08:51.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.588+0000 7f985258b640 1 -- 172.21.15.157:0/63975076 wait complete. 2023-11-06T03:08:51.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:08:41.869490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:08:42.287789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:08:51.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:08:51.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.732+0000 7facd6718640 1 Processor -- start 2023-11-06T03:08:51.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.732+0000 7facd6718640 1 -- start start 2023-11-06T03:08:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facd6718640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7facd00b2af0 0x7facd00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facd6718640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd015e440 con 0x7facd01599c0 2023-11-06T03:08:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facd6718640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd006c240 con 0x7facd006d000 2023-11-06T03:08:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facd6718640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd006c520 con 0x7facd00b2af0 2023-11-06T03:08:51.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7facd01599c0 0x7facd00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37106/0 (socket says 172.21.15.157:37106) 2023-11-06T03:08:51.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccffff640 1 -- 172.21.15.157:0/1175868408 learned_addr learned my addr 172.21.15.157:0/1175868408 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:51.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1175868408 >> v2:172.21.15.195:3300/0 conn(0x7facd00b2af0 0x7facd00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745220171 0 0) 0x7facd015e440 con 0x7facd01599c0 2023-11-06T03:08:51.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb8003530 con 0x7facd01599c0 2023-11-06T03:08:51.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3451809705 0 0) 0x7facb8003530 con 0x7facd01599c0 2023-11-06T03:08:51.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 >> v2:172.21.15.195:3300/0 conn(0x7facd00b2af0 msgr2=0x7facd00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 --2- 172.21.15.157:0/1175868408 >> v2:172.21.15.195:3300/0 conn(0x7facd00b2af0 0x7facd00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 >> v1:172.21.15.195:6789/0 conn(0x7facd006d000 legacy=0x7facd0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd006c800 con 0x7facd01599c0 2023-11-06T03:08:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7facc0001680 con 0x7facd01599c0 2023-11-06T03:08:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 >> v1:172.21.15.157:6789/0 conn(0x7facd01599c0 legacy=0x7facd00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb8003980 0x7facb8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 0x7facb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facb8005190 0x7facb800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd006c520 con 0x7facb80047d0 2023-11-06T03:08:51.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb800c030 con 0x7facb8003980 2023-11-06T03:08:51.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb800c1f0 con 0x7facb8005190 2023-11-06T03:08:51.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 0x7facb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccffff640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb8003980 0x7facb8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 -- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facb8005190 msgr2=0x7facb800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:51.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facb8005190 0x7facb800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 -- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb8003980 msgr2=0x7facb8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:51.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb8003980 0x7facb8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7facd4cb5640 1 -- 172.21.15.157:0/1175868408 reap_dead start 2023-11-06T03:08:51.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccf7fe640 1 -- 172.21.15.157:0/1175868408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8003530 con 0x7facb80047d0 2023-11-06T03:08:51.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.736+0000 7faccffff640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb8003980 0x7facb8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 0x7facb8004c50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7facc4009ae0 tx=0x7facc401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11278d50b2bf5e04 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40a5660 con 0x7facb80047d0 2023-11-06T03:08:51.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0001d80 con 0x7facb80047d0 2023-11-06T03:08:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facceffd640 1 -- 172.21.15.157:0/1175868408 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40a5a00 con 0x7facb80047d0 2023-11-06T03:08:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 msgr2=0x7facb8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 0x7facb8004c50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7facc4009ae0 tx=0x7facc401ca70 comp rx=0 tx=0).stop 2023-11-06T03:08:51.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- 172.21.15.157:0/1175868408 shutdown_connections 2023-11-06T03:08:51.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 --2- 172.21.15.157:0/1175868408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80047d0 0x7facb8004c50 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- 172.21.15.157:0/1175868408 >> 172.21.15.157:0/1175868408 conn(0x7facd0065930 msgr2=0x7facd0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:51.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- 172.21.15.157:0/1175868408 shutdown_connections 2023-11-06T03:08:51.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- 172.21.15.157:0/1175868408 wait complete. 2023-11-06T03:08:51.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 Processor -- start 2023-11-06T03:08:51.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- start start 2023-11-06T03:08:51.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 0x7facd00cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facd014d870 0x7facd014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01f06b0 con 0x7facd014dcf0 2023-11-06T03:08:51.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd014e250 con 0x7facb80047d0 2023-11-06T03:08:51.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd6718640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd014e530 con 0x7facd014d870 2023-11-06T03:08:51.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54004/0 (socket says 172.21.15.157:54004) 2023-11-06T03:08:51.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 -- 172.21.15.157:0/1518971616 learned_addr learned my addr 172.21.15.157:0/1518971616 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:51.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7faccffff640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 0x7facd00cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7faccf7fe640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facd014d870 0x7facd014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 -- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facd014d870 msgr2=0x7facd014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facd014d870 0x7facd014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 -- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 msgr2=0x7facd00cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 0x7facd00cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 -- 172.21.15.157:0/1518971616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd014e810 con 0x7facd014dcf0 2023-11-06T03:08:51.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7faccffff640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 0x7facd00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:51.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.740+0000 7facd4cb5640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7facd006c130 tx=0x7facc800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8007070 con 0x7facd014dcf0 2023-11-06T03:08:51.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc80071f0 con 0x7facd014dcf0 2023-11-06T03:08:51.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd014eaf0 con 0x7facd014dcf0 2023-11-06T03:08:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80075a0 con 0x7facd014dcf0 2023-11-06T03:08:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00ccd40 con 0x7facd014dcf0 2023-11-06T03:08:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd015de20 con 0x7facd014dcf0 2023-11-06T03:08:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7facc8007700 con 0x7facd014dcf0 2023-11-06T03:08:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 0x7facbc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7facc8082290 con 0x7facd014dcf0 2023-11-06T03:08:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7faccffff640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 0x7facbc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.744+0000 7faccffff640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 0x7facbc067f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7facc0004880 tx=0x7facc0001460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.748+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7facc804fba0 con 0x7facd014dcf0 2023-11-06T03:08:51.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.896+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facd00adcd0 con 0x7facd014dcf0 2023-11-06T03:08:51.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facccff9640 1 -- 172.21.15.157:0/1518971616 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7facc8017020 con 0x7facd014dcf0 2023-11-06T03:08:51.898 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:08:51.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 msgr2=0x7facbc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 0x7facbc067f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7facc0004880 tx=0x7facc0001460 comp rx=0 tx=0).stop 2023-11-06T03:08:51.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 msgr2=0x7facd014c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:51.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7facd006c130 tx=0x7facc800c960 comp rx=0 tx=0).stop 2023-11-06T03:08:51.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 shutdown_connections 2023-11-06T03:08:51.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7facbc065a60 0x7facbc067f20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd014dcf0 0x7facd014c810 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7facd014d870 0x7facd014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 --2- 172.21.15.157:0/1518971616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7facb80047d0 0x7facd00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 >> 172.21.15.157:0/1518971616 conn(0x7facd0065930 msgr2=0x7facd00ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 shutdown_connections 2023-11-06T03:08:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:51.900+0000 7facd6718640 1 -- 172.21.15.157:0/1518971616 wait complete. 2023-11-06T03:08:51.917 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:08:52.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 Processor -- start 2023-11-06T03:08:52.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 -- start start 2023-11-06T03:08:52.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f891c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c15a860 con 0x7f891c0b5320 2023-11-06T03:08:52.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0711b0 con 0x7f891c0b3020 2023-11-06T03:08:52.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f89233f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c159d10 con 0x7f891c159930 2023-11-06T03:08:52.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f892198f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f891c0b5320 0x7f891c06c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37108/0 (socket says 172.21.15.157:37108) 2023-11-06T03:08:52.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f892198f640 1 -- 172.21.15.157:0/1193556907 learned_addr learned my addr 172.21.15.157:0/1193556907 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:52.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f892118e640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f891c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356633170 0 0) 0x7f891c15a860 con 0x7f891c0b5320 2023-11-06T03:08:52.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.040+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8908003530 con 0x7f891c0b5320 2023-11-06T03:08:52.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878307647 0 0) 0x7f8908003530 con 0x7f891c0b5320 2023-11-06T03:08:52.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 msgr2=0x7f891c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f891c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 >> v1:172.21.15.195:6789/0 conn(0x7f891c0b3020 legacy=0x7f891c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c159f30 con 0x7f891c0b5320 2023-11-06T03:08:52.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8914001a00 con 0x7f891c0b5320 2023-11-06T03:08:52.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 >> v1:172.21.15.157:6789/0 conn(0x7f891c0b5320 legacy=0x7f891c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89080039b0 0x7f8908003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f89080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8908004d10 0x7f8908005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c159d10 con 0x7f89080039b0 2023-11-06T03:08:52.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890800bac0 con 0x7f8908004d10 2023-11-06T03:08:52.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890800bc80 con 0x7f891c159930 2023-11-06T03:08:52.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89080039b0 0x7f8908003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892098d640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f89080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892198f640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8908004d10 0x7f8908005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 -- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 msgr2=0x7f89080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f891c159930 0x7f89080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 -- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8908004d10 msgr2=0x7f8908005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8908004d10 0x7f8908005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 -- 172.21.15.157:0/1193556907 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908003530 con 0x7f89080039b0 2023-11-06T03:08:52.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892198f640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8908004d10 0x7f8908005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:52.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892118e640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89080039b0 0x7f8908003dd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8918009ae0 tx=0x7f891801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0b64cd59e0fd68 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180a9470 con 0x7f89080039b0 2023-11-06T03:08:52.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89180a95d0 con 0x7f89080039b0 2023-11-06T03:08:52.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f8913fff640 1 -- 172.21.15.157:0/1193556907 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180a9980 con 0x7f89080039b0 2023-11-06T03:08:52.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 -- 172.21.15.157:0/1193556907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89080039b0 msgr2=0x7f8908003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 --2- 172.21.15.157:0/1193556907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89080039b0 0x7f8908003dd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8918009ae0 tx=0x7f891801c990 comp rx=0 tx=0).stop 2023-11-06T03:08:52.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f892198f640 1 -- 172.21.15.157:0/1193556907 reap_dead start 2023-11-06T03:08:52.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 -- 172.21.15.157:0/1193556907 shutdown_connections 2023-11-06T03:08:52.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 -- 172.21.15.157:0/1193556907 >> 172.21.15.157:0/1193556907 conn(0x7f891c069d70 msgr2=0x7f891c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 -- 172.21.15.157:0/1193556907 shutdown_connections 2023-11-06T03:08:52.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.044+0000 7f89233f2640 1 -- 172.21.15.157:0/1193556907 wait complete. 2023-11-06T03:08:52.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 Processor -- start 2023-11-06T03:08:52.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- start start 2023-11-06T03:08:52.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89080039b0 0x7f891c14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 0x7f891c14b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c14f380 con 0x7f891c14c4d0 2023-11-06T03:08:52.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c14e700 con 0x7f891c1e9b00 2023-11-06T03:08:52.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c14e9e0 con 0x7f89080039b0 2023-11-06T03:08:52.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54028/0 (socket says 172.21.15.157:54028) 2023-11-06T03:08:52.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 -- 172.21.15.157:0/3780927717 learned_addr learned my addr 172.21.15.157:0/3780927717 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892118e640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89080039b0 0x7f891c14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892198f640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 0x7f891c14b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 -- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89080039b0 msgr2=0x7f891c14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89080039b0 0x7f891c14f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 -- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 msgr2=0x7f891c14b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 0x7f891c14b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 -- 172.21.15.157:0/3780927717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c14ecc0 con 0x7f891c14c4d0 2023-11-06T03:08:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892198f640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 0x7f891c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f892098d640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f890c00e550 tx=0x7f890c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c0052e0 con 0x7f891c14c4d0 2023-11-06T03:08:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c005440 con 0x7f891c14c4d0 2023-11-06T03:08:52.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c003820 con 0x7f891c14c4d0 2023-11-06T03:08:52.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c14efa0 con 0x7f891c14c4d0 2023-11-06T03:08:52.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c1e9f90 con 0x7f891c14c4d0 2023-11-06T03:08:52.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.048+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c14f5f0 con 0x7f891c14c4d0 2023-11-06T03:08:52.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.052+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f890c003980 con 0x7f891c14c4d0 2023-11-06T03:08:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.052+0000 7f8911ffb640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.052+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f890c0830b0 con 0x7f891c14c4d0 2023-11-06T03:08:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.052+0000 7f892118e640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.052+0000 7f892118e640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 0x7f8900067f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f89180ae000 tx=0x7f89180046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.056+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f890c050910 con 0x7f891c14c4d0 2023-11-06T03:08:52.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.208+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f891c072750 con 0x7f891c14c4d0 2023-11-06T03:08:52.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f8911ffb640 1 -- 172.21.15.157:0/3780927717 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7f890c053f30 con 0x7f891c14c4d0 2023-11-06T03:08:52.211 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:08:52.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 msgr2=0x7f8900067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 0x7f8900067f20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f89180ae000 tx=0x7f89180046d0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 msgr2=0x7f891c14de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f890c00e550 tx=0x7f890c00ea10 comp rx=0 tx=0).stop 2023-11-06T03:08:52.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 shutdown_connections 2023-11-06T03:08:52.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8900065a60 0x7f8900067f20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f891c1e9b00 0x7f891c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f891c14c4d0 0x7f891c14de40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 --2- 172.21.15.157:0/3780927717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89080039b0 0x7f891c14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 >> 172.21.15.157:0/3780927717 conn(0x7f891c069d70 msgr2=0x7f891c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 shutdown_connections 2023-11-06T03:08:52.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.212+0000 7f89233f2640 1 -- 172.21.15.157:0/3780927717 wait complete. 2023-11-06T03:08:52.229 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:08:52.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.352+0000 7f925f804640 1 Processor -- start 2023-11-06T03:08:52.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.352+0000 7f925f804640 1 -- start start 2023-11-06T03:08:52.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92581551e0 0x7f925806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f925815e5e0 0x7f92580b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925815f580 con 0x7f925815ea60 2023-11-06T03:08:52.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925815efc0 con 0x7f92581551e0 2023-11-06T03:08:52.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925815f2a0 con 0x7f925815e5e0 2023-11-06T03:08:52.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92581551e0 0x7f925806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92581551e0 0x7f925806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54032/0 (socket says 172.21.15.157:54032) 2023-11-06T03:08:52.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 -- 172.21.15.157:0/563754521 learned_addr learned my addr 172.21.15.157:0/563754521 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925cd9f640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f925815e5e0 0x7f92580b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 -- 172.21.15.157:0/563754521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f925815e5e0 msgr2=0x7f92580b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f925815e5e0 0x7f92580b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 -- 172.21.15.157:0/563754521 >> v1:172.21.15.157:6789/0 conn(0x7f925815ea60 legacy=0x7f925815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f924ffff640 1 -- 172.21.15.157:0/563754521 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473945813 0 0) 0x7f925815f580 con 0x7f925815ea60 2023-11-06T03:08:52.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 -- 172.21.15.157:0/563754521 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925806f5a0 con 0x7f92581551e0 2023-11-06T03:08:52.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925d5a0640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92581551e0 0x7f925806fda0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9254009870 tx=0x7f925401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d0b0236c75d150 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f924ffff640 1 -- 172.21.15.157:0/563754521 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540ad640 con 0x7f92581551e0 2023-11-06T03:08:52.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f924ffff640 1 -- 172.21.15.157:0/563754521 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92540ad7a0 con 0x7f92581551e0 2023-11-06T03:08:52.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f924ffff640 1 -- 172.21.15.157:0/563754521 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540adb30 con 0x7f92581551e0 2023-11-06T03:08:52.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- 172.21.15.157:0/563754521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92581551e0 msgr2=0x7f925806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92581551e0 0x7f925806fda0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9254009870 tx=0x7f925401c480 comp rx=0 tx=0).stop 2023-11-06T03:08:52.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- 172.21.15.157:0/563754521 shutdown_connections 2023-11-06T03:08:52.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f925815e5e0 0x7f92580b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 --2- 172.21.15.157:0/563754521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92581551e0 0x7f925806fda0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- 172.21.15.157:0/563754521 >> 172.21.15.157:0/563754521 conn(0x7f9258065930 msgr2=0x7f9258065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- 172.21.15.157:0/563754521 shutdown_connections 2023-11-06T03:08:52.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.356+0000 7f925f804640 1 -- 172.21.15.157:0/563754521 wait complete. 2023-11-06T03:08:52.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 Processor -- start 2023-11-06T03:08:52.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- start start 2023-11-06T03:08:52.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92581551e0 0x7f925813e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f925815ea60 0x7f9258154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92581567e0 con 0x7f925815e5e0 2023-11-06T03:08:52.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258155b60 con 0x7f925815ea60 2023-11-06T03:08:52.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9258155e40 con 0x7f92581551e0 2023-11-06T03:08:52.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54042/0 (socket says 172.21.15.157:54042) 2023-11-06T03:08:52.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 -- 172.21.15.157:0/4081587125 learned_addr learned my addr 172.21.15.157:0/4081587125 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925d5a0640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92581551e0 0x7f925813e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 -- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92581551e0 msgr2=0x7f925813e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92581551e0 0x7f925813e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 -- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f925815ea60 msgr2=0x7f9258154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925dda1640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f925815ea60 0x7f9258154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f925815ea60 0x7f9258154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 -- 172.21.15.157:0/4081587125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92540094d0 con 0x7f925815e5e0 2023-11-06T03:08:52.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925cd9f640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f924800d800 tx=0x7f924800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800b6c0 con 0x7f925815e5e0 2023-11-06T03:08:52.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800b820 con 0x7f925815e5e0 2023-11-06T03:08:52.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258156120 con 0x7f925815e5e0 2023-11-06T03:08:52.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800bb90 con 0x7f925815e5e0 2023-11-06T03:08:52.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92581590d0 con 0x7f925815e5e0 2023-11-06T03:08:52.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.360+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925814c5e0 con 0x7f925815e5e0 2023-11-06T03:08:52.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.364+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f924800bcf0 con 0x7f925815e5e0 2023-11-06T03:08:52.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.364+0000 7f924dffb640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 0x7f923c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.364+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f92480838f0 con 0x7f925815e5e0 2023-11-06T03:08:52.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.364+0000 7f925d5a0640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 0x7f923c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.364+0000 7f925d5a0640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 0x7f923c067f20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92540af000 tx=0x7f9254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.368+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f92480510c0 con 0x7f925815e5e0 2023-11-06T03:08:52.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.516+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9258072750 con 0x7f925815e5e0 2023-11-06T03:08:52.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.516+0000 7f924dffb640 1 -- 172.21.15.157:0/4081587125 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92480546e0 con 0x7f925815e5e0 2023-11-06T03:08:52.518 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:08:52.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 msgr2=0x7f923c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 0x7f923c067f20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92540af000 tx=0x7f9254008040 comp rx=0 tx=0).stop 2023-11-06T03:08:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 msgr2=0x7f9258153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f924800d800 tx=0x7f924800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 shutdown_connections 2023-11-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f923c065a60 0x7f923c067f20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f925815ea60 0x7f9258154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f925815e5e0 0x7f9258153b20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 --2- 172.21.15.157:0/4081587125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92581551e0 0x7f925813e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 >> 172.21.15.157:0/4081587125 conn(0x7f9258065930 msgr2=0x7f92580b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 shutdown_connections 2023-11-06T03:08:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.520+0000 7f925f804640 1 -- 172.21.15.157:0/4081587125 wait complete. 2023-11-06T03:08:52.535 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:08:52.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.648+0000 7f053650c640 1 Processor -- start 2023-11-06T03:08:52.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.648+0000 7f053650c640 1 -- start start 2023-11-06T03:08:52.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f053650c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05301551e0 0x7f053006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f053650c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053015f580 con 0x7f053015e5e0 2023-11-06T03:08:52.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f053650c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053015efc0 con 0x7f05300b3b20 2023-11-06T03:08:52.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f053650c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053015f2a0 con 0x7f05301551e0 2023-11-06T03:08:52.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05301551e0 0x7f053006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05301551e0 0x7f053006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54050/0 (socket says 172.21.15.157:54050) 2023-11-06T03:08:52.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f053015e5e0 0x7f05300b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37118/0 (socket says 172.21.15.157:37118) 2023-11-06T03:08:52.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 -- 172.21.15.157:0/1491868425 learned_addr learned my addr 172.21.15.157:0/1491868425 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 -- 172.21.15.157:0/1491868425 >> v1:172.21.15.195:6789/0 conn(0x7f05300b3b20 legacy=0x7f053015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052effd640 1 -- 172.21.15.157:0/1491868425 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441027368 0 0) 0x7f053015f580 con 0x7f053015e5e0 2023-11-06T03:08:52.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 -- 172.21.15.157:0/1491868425 >> v1:172.21.15.157:6789/0 conn(0x7f053015e5e0 legacy=0x7f05300b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.652+0000 7f052ffff640 1 -- 172.21.15.157:0/1491868425 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053006f5a0 con 0x7f05301551e0 2023-11-06T03:08:52.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f052ffff640 1 --2- 172.21.15.157:0/1491868425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05301551e0 0x7f053006fda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0520009870 tx=0x7f052001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb701c3b0cde2e1a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f052effd640 1 -- 172.21.15.157:0/1491868425 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200ac630 con 0x7f05301551e0 2023-11-06T03:08:52.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f052effd640 1 -- 172.21.15.157:0/1491868425 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05200ac790 con 0x7f05301551e0 2023-11-06T03:08:52.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f052effd640 1 -- 172.21.15.157:0/1491868425 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200acb20 con 0x7f05301551e0 2023-11-06T03:08:52.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 -- 172.21.15.157:0/1491868425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05301551e0 msgr2=0x7f053006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 --2- 172.21.15.157:0/1491868425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05301551e0 0x7f053006fda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0520009870 tx=0x7f052001c3b0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 -- 172.21.15.157:0/1491868425 shutdown_connections 2023-11-06T03:08:52.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 --2- 172.21.15.157:0/1491868425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05301551e0 0x7f053006fda0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 -- 172.21.15.157:0/1491868425 >> 172.21.15.157:0/1491868425 conn(0x7f0530065930 msgr2=0x7f0530065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 -- 172.21.15.157:0/1491868425 shutdown_connections 2023-11-06T03:08:52.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.656+0000 7f053650c640 1 -- 172.21.15.157:0/1491868425 wait complete. 2023-11-06T03:08:52.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 Processor -- start 2023-11-06T03:08:52.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- start start 2023-11-06T03:08:52.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05301551e0 0x7f05301f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053015e5e0 0x7f05301f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530157640 con 0x7f05300b3b20 2023-11-06T03:08:52.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053014b1b0 con 0x7f053015e5e0 2023-11-06T03:08:52.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053014b490 con 0x7f05301551e0 2023-11-06T03:08:52.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54066/0 (socket says 172.21.15.157:54066) 2023-11-06T03:08:52.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 -- 172.21.15.157:0/3603275264 learned_addr learned my addr 172.21.15.157:0/3603275264 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052f7fe640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05301551e0 0x7f05301f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f0534aa9640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053015e5e0 0x7f05301f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 -- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05301551e0 msgr2=0x7f05301f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05301551e0 0x7f05301f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 -- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053015e5e0 msgr2=0x7f05301f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053015e5e0 0x7f05301f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 -- 172.21.15.157:0/3603275264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200094d0 con 0x7f05300b3b20 2023-11-06T03:08:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052ffff640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f05200097c0 tx=0x7f052001abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004220 con 0x7f05300b3b20 2023-11-06T03:08:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520004380 con 0x7f05300b3b20 2023-11-06T03:08:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053014b770 con 0x7f05300b3b20 2023-11-06T03:08:52.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520003820 con 0x7f05300b3b20 2023-11-06T03:08:52.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300ce210 con 0x7f05300b3b20 2023-11-06T03:08:52.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.660+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530072750 con 0x7f05300b3b20 2023-11-06T03:08:52.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.664+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f05200b3050 con 0x7f05300b3b20 2023-11-06T03:08:52.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.664+0000 7f052cff9640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 0x7f051c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.664+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f052011dd80 con 0x7f05300b3b20 2023-11-06T03:08:52.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.664+0000 7f052f7fe640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 0x7f051c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.664+0000 7f052f7fe640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 0x7f051c067f20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f053015a9a0 tx=0x7f0524007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.668+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f05200eb6c0 con 0x7f05300b3b20 2023-11-06T03:08:52.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0530071730 con 0x7f05300b3b20 2023-11-06T03:08:52.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f052cff9640 1 -- 172.21.15.157:0/3603275264 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05200b10a0 con 0x7f05300b3b20 2023-11-06T03:08:52.816 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:08:52.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 msgr2=0x7f051c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 0x7f051c067f20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f053015a9a0 tx=0x7f0524007d10 comp rx=0 tx=0).stop 2023-11-06T03:08:52.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 msgr2=0x7f05300cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.816+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f05200097c0 tx=0x7f052001abf0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 shutdown_connections 2023-11-06T03:08:52.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f051c065a60 0x7f051c067f20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053015e5e0 0x7f05301f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05301551e0 0x7f05301f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 --2- 172.21.15.157:0/3603275264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05300b3b20 0x7f05300cdcd0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 >> 172.21.15.157:0/3603275264 conn(0x7f0530065930 msgr2=0x7f05300afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 shutdown_connections 2023-11-06T03:08:52.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.820+0000 7f053650c640 1 -- 172.21.15.157:0/3603275264 wait complete. 2023-11-06T03:08:52.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:08:52.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 Processor -- start 2023-11-06T03:08:52.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 -- start start 2023-11-06T03:08:52.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67dc1551e0 0x7f67dc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15e5e0 0x7f67dc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67dc15ea60 0x7f67dc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc15f580 con 0x7f67dc15ea60 2023-11-06T03:08:52.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc15efc0 con 0x7f67dc1551e0 2023-11-06T03:08:52.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e39d9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc15f2a0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67dc15ea60 0x7f67dc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67dc15ea60 0x7f67dc0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54068/0 (socket says 172.21.15.157:54068) 2023-11-06T03:08:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 -- 172.21.15.157:0/3002592517 learned_addr learned my addr 172.21.15.157:0/3002592517 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1775640 1 --2- 172.21.15.157:0/3002592517 >> v2:172.21.15.195:3300/0 conn(0x7f67dc1551e0 0x7f67dc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e0f74640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15e5e0 0x7f67dc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 -- 172.21.15.157:0/3002592517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15e5e0 msgr2=0x7f67dc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15e5e0 0x7f67dc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 -- 172.21.15.157:0/3002592517 >> v2:172.21.15.195:3300/0 conn(0x7f67dc1551e0 msgr2=0x7f67dc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 --2- 172.21.15.157:0/3002592517 >> v2:172.21.15.195:3300/0 conn(0x7f67dc1551e0 0x7f67dc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.952+0000 7f67e1f76640 1 -- 172.21.15.157:0/3002592517 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc06f5a0 con 0x7f67dc15ea60 2023-11-06T03:08:52.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e1f76640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15ea60 0x7f67dc0b3950 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f67dc066830 tx=0x7f67cc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9594c7615f0aeba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67dbfff640 1 -- 172.21.15.157:0/3002592517 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0a57b0 con 0x7f67dc15ea60 2023-11-06T03:08:52.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67dbfff640 1 -- 172.21.15.157:0/3002592517 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc0a5910 con 0x7f67dc15ea60 2023-11-06T03:08:52.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67dbfff640 1 -- 172.21.15.157:0/3002592517 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0a5ca0 con 0x7f67dc15ea60 2023-11-06T03:08:52.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- 172.21.15.157:0/3002592517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15ea60 msgr2=0x7f67dc0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15ea60 0x7f67dc0b3950 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f67dc066830 tx=0x7f67cc01c910 comp rx=0 tx=0).stop 2023-11-06T03:08:52.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- 172.21.15.157:0/3002592517 shutdown_connections 2023-11-06T03:08:52.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15ea60 0x7f67dc0b3950 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 --2- 172.21.15.157:0/3002592517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15e5e0 0x7f67dc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 --2- 172.21.15.157:0/3002592517 >> v2:172.21.15.195:3300/0 conn(0x7f67dc1551e0 0x7f67dc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- 172.21.15.157:0/3002592517 >> 172.21.15.157:0/3002592517 conn(0x7f67dc065930 msgr2=0x7f67dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:52.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- 172.21.15.157:0/3002592517 shutdown_connections 2023-11-06T03:08:52.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- 172.21.15.157:0/3002592517 wait complete. 2023-11-06T03:08:52.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 Processor -- start 2023-11-06T03:08:52.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.956+0000 7f67e39d9640 1 -- start start 2023-11-06T03:08:52.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 0x7f67dc14ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 0x7f67dc13a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc156fb0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc13a8c0 con 0x7f67dc1551e0 2023-11-06T03:08:52.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0b1960 con 0x7f67dc15ea60 2023-11-06T03:08:52.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54078/0 (socket says 172.21.15.157:54078) 2023-11-06T03:08:52.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 -- 172.21.15.157:0/2589348809 learned_addr learned my addr 172.21.15.157:0/2589348809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:52.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e1775640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 0x7f67dc14ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e1f76640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 0x7f67dc13a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 -- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 msgr2=0x7f67dc13a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 0x7f67dc13a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 -- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 msgr2=0x7f67dc14ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:52.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 0x7f67dc14ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:52.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 -- 172.21.15.157:0/2589348809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009b30 con 0x7f67dc15e5e0 2023-11-06T03:08:52.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e1f76640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 0x7f67dc13a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e1775640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 0x7f67dc14ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:08:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e0f74640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d800 tx=0x7f67d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000b6c0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d000b820 con 0x7f67dc15e5e0 2023-11-06T03:08:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000bb90 con 0x7f67dc15e5e0 2023-11-06T03:08:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc1569f0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc1525a0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc152400 con 0x7f67dc15e5e0 2023-11-06T03:08:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f67d000bcf0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 0x7f67c8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:52.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f67d00838d0 con 0x7f67dc15e5e0 2023-11-06T03:08:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.960+0000 7f67e1775640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 0x7f67c8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.964+0000 7f67e1775640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 0x7f67c8067f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f67d4004660 tx=0x7f67d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:52.964+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f67dc152400 con 0x7f67dc15e5e0 2023-11-06T03:08:53.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67dc152400 con 0x7f67dc15e5e0 2023-11-06T03:08:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67d9ffb640 1 -- 172.21.15.157:0/2589348809 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67dc152400 con 0x7f67dc15e5e0 2023-11-06T03:08:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:08:53.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 msgr2=0x7f67c8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 0x7f67c8067f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f67d4004660 tx=0x7f67d4008040 comp rx=0 tx=0).stop 2023-11-06T03:08:53.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 msgr2=0x7f67dc1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d800 tx=0x7f67d000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 shutdown_connections 2023-11-06T03:08:53.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f67c8065a90 0x7f67c8067f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67dc15ea60 0x7f67dc13a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67dc15e5e0 0x7f67dc1562f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 --2- 172.21.15.157:0/2589348809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67dc1551e0 0x7f67dc14ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 >> 172.21.15.157:0/2589348809 conn(0x7f67dc065930 msgr2=0x7f67dc15daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 shutdown_connections 2023-11-06T03:08:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.128+0000 7f67e39d9640 1 -- 172.21.15.157:0/2589348809 wait complete. 2023-11-06T03:08:53.144 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:08:53.145 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:08:53.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:08:53.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.308+0000 7f6b4f27e640 1 Processor -- start 2023-11-06T03:08:53.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.308+0000 7f6b4f27e640 1 -- start start 2023-11-06T03:08:53.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4f27e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b48156a70 0x7f6b4806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4f27e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b480adae0 0x7f6b480b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4f27e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4806c490 con 0x7f6b480adf40 2023-11-06T03:08:53.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4f27e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4806b950 con 0x7f6b480adae0 2023-11-06T03:08:53.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4f27e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4806bb60 con 0x7f6b48156a70 2023-11-06T03:08:53.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b480adae0 0x7f6b480b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6b480adae0 0x7f6b480b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54094/0 (socket says 172.21.15.157:54094) 2023-11-06T03:08:53.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 -- 172.21.15.157:0/761402113 learned_addr learned my addr 172.21.15.157:0/761402113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:53.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4d01a640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b48156a70 0x7f6b4806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 -- 172.21.15.157:0/761402113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b48156a70 msgr2=0x7f6b4806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b48156a70 0x7f6b4806f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 -- 172.21.15.157:0/761402113 >> v1:172.21.15.195:6789/0 conn(0x7f6b480adf40 legacy=0x7f6b4806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 -- 172.21.15.157:0/761402113 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4806be40 con 0x7f6b480adae0 2023-11-06T03:08:53.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b4c819640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b480b1c20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009870 tx=0x7f6b3c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467dbc0a3792c7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b47fff640 1 -- 172.21.15.157:0/761402113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c0aa610 con 0x7f6b480adae0 2023-11-06T03:08:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b47fff640 1 -- 172.21.15.157:0/761402113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0a6040 con 0x7f6b480adae0 2023-11-06T03:08:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.312+0000 7f6b47fff640 1 -- 172.21.15.157:0/761402113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c0aa990 con 0x7f6b480adae0 2023-11-06T03:08:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/761402113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 msgr2=0x7f6b480b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b480b1c20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009870 tx=0x7f6b3c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/761402113 shutdown_connections 2023-11-06T03:08:53.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b480b1c20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/761402113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b48156a70 0x7f6b4806f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/761402113 >> 172.21.15.157:0/761402113 conn(0x7f6b48065930 msgr2=0x7f6b48065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/761402113 shutdown_connections 2023-11-06T03:08:53.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/761402113 wait complete. 2023-11-06T03:08:53.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 Processor -- start 2023-11-06T03:08:53.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- start start 2023-11-06T03:08:53.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b480adf40 0x7f6b48158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b48156a70 0x7f6b481587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4815b2c0 con 0x7f6b480adae0 2023-11-06T03:08:53.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4815afe0 con 0x7f6b48156a70 2023-11-06T03:08:53.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48155960 con 0x7f6b480adf40 2023-11-06T03:08:53.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54102/0 (socket says 172.21.15.157:54102) 2023-11-06T03:08:53.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 -- 172.21.15.157:0/2438075746 learned_addr learned my addr 172.21.15.157:0/2438075746 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:53.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4c819640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b480adf40 0x7f6b48158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d81b640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b48156a70 0x7f6b481587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 -- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b480adf40 msgr2=0x7f6b48158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b480adf40 0x7f6b48158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 -- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b48156a70 msgr2=0x7f6b481587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b48156a70 0x7f6b481587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 -- 172.21.15.157:0/2438075746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0094d0 con 0x7f6b480adae0 2023-11-06T03:08:53.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4d01a640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6b40007e40 tx=0x7f6b4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40013670 con 0x7f6b480adae0 2023-11-06T03:08:53.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4000ee90 con 0x7f6b480adae0 2023-11-06T03:08:53.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40013a10 con 0x7f6b480adae0 2023-11-06T03:08:53.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b48155c40 con 0x7f6b480adae0 2023-11-06T03:08:53.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.316+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4814ed70 con 0x7f6b480adae0 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b48072750 con 0x7f6b480adae0 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6b40013b70 con 0x7f6b480adae0 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b45ffb640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 0x7f6b34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6b40085a50 con 0x7f6b480adae0 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b4c819640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 0x7f6b34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.320+0000 7f6b4c819640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 0x7f6b34067f50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0ae000 tx=0x7f6b3c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.324+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6b400532b0 con 0x7f6b480adae0 2023-11-06T03:08:53.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.468+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b48163810 con 0x7f6b480adae0 2023-11-06T03:08:53.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.468+0000 7f6b45ffb640 1 -- 172.21.15.157:0/2438075746 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18112 (secure 0 0 0) 0x7f6b40018090 con 0x7f6b480adae0 2023-11-06T03:08:53.469 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:08:53.469 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":69,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:08:50.340995+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:08:53.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 msgr2=0x7f6b34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 0x7f6b34067f50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0ae000 tx=0x7f6b3c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:08:53.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 msgr2=0x7f6b4814e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6b40007e40 tx=0x7f6b4000e4b0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 shutdown_connections 2023-11-06T03:08:53.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6b34065a90 0x7f6b34067f50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b48156a70 0x7f6b481587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6b480adf40 0x7f6b48158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 --2- 172.21.15.157:0/2438075746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b480adae0 0x7f6b4814e830 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 >> 172.21.15.157:0/2438075746 conn(0x7f6b48065930 msgr2=0x7f6b48159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 shutdown_connections 2023-11-06T03:08:53.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.472+0000 7f6b4f27e640 1 -- 172.21.15.157:0/2438075746 wait complete. 2023-11-06T03:08:53.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:08:41.869490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:08:42.287789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:08:53.487 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:08:53.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 Processor -- start 2023-11-06T03:08:53.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 -- start start 2023-11-06T03:08:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 0x7fd4dc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc06c490 con 0x7fd4dc156a70 2023-11-06T03:08:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc06b950 con 0x7fd4dc0adae0 2023-11-06T03:08:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e4110640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc06fb80 con 0x7fd4dc0adf40 2023-11-06T03:08:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54104/0 (socket says 172.21.15.157:54104) 2023-11-06T03:08:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e1eac640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd4dc156a70 0x7fd4dc06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37128/0 (socket says 172.21.15.157:37128) 2023-11-06T03:08:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 -- 172.21.15.157:0/3611692839 learned_addr learned my addr 172.21.15.157:0/3611692839 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e26ad640 1 --2- 172.21.15.157:0/3611692839 >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 0x7fd4dc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 -- 172.21.15.157:0/3611692839 >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 msgr2=0x7fd4dc0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 --2- 172.21.15.157:0/3611692839 >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 0x7fd4dc0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 -- 172.21.15.157:0/3611692839 >> v1:172.21.15.157:6789/0 conn(0x7fd4dc156a70 legacy=0x7fd4dc06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e16ab640 1 -- 172.21.15.157:0/3611692839 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc06bb60 con 0x7fd4dc0adae0 2023-11-06T03:08:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.600+0000 7fd4e26ad640 1 --2- 172.21.15.157:0/3611692839 >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 0x7fd4dc0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:53.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e16ab640 1 --2- 172.21.15.157:0/3611692839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0ce830 tx=0x7fd4d001e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb51beccda72a51 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e0eaa640 1 -- 172.21.15.157:0/3611692839 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0004230 con 0x7fd4dc0adae0 2023-11-06T03:08:53.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e0eaa640 1 -- 172.21.15.157:0/3611692839 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00ab070 con 0x7fd4dc0adae0 2023-11-06T03:08:53.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e0eaa640 1 -- 172.21.15.157:0/3611692839 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00a77d0 con 0x7fd4dc0adae0 2023-11-06T03:08:53.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- 172.21.15.157:0/3611692839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc0adae0 msgr2=0x7fd4dc0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- 172.21.15.157:0/3611692839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0ce830 tx=0x7fd4d001e980 comp rx=0 tx=0).stop 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- 172.21.15.157:0/3611692839 shutdown_connections 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- 172.21.15.157:0/3611692839 >> v2:172.21.15.195:3300/0 conn(0x7fd4dc0adf40 0x7fd4dc0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- 172.21.15.157:0/3611692839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc0b1c20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- 172.21.15.157:0/3611692839 >> 172.21.15.157:0/3611692839 conn(0x7fd4dc065930 msgr2=0x7fd4dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- 172.21.15.157:0/3611692839 shutdown_connections 2023-11-06T03:08:53.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- 172.21.15.157:0/3611692839 wait complete. 2023-11-06T03:08:53.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 Processor -- start 2023-11-06T03:08:53.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- start start 2023-11-06T03:08:53.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4dc0adf40 0x7fd4dc15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15fb40 con 0x7fd4dc156a70 2023-11-06T03:08:53.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15fe20 con 0x7fd4dc0adae0 2023-11-06T03:08:53.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e4110640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc160100 con 0x7fd4dc0adf40 2023-11-06T03:08:53.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e26ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e26ad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54110/0 (socket says 172.21.15.157:54110) 2023-11-06T03:08:53.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.604+0000 7fd4e26ad640 1 -- 172.21.15.157:0/1607299890 learned_addr learned my addr 172.21.15.157:0/1607299890 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:53.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e1eac640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e16ab640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4dc0adf40 0x7fd4dc15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 -- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4dc0adf40 msgr2=0x7fd4dc15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4dc0adf40 0x7fd4dc15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 -- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4dc0adae0 msgr2=0x7fd4dc15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 -- 172.21.15.157:0/1607299890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d000b780 con 0x7fd4dc156a70 2023-11-06T03:08:53.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e26ad640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400cef0 tx=0x7fd4d400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400d840 con 0x7fd4dc156a70 2023-11-06T03:08:53.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400d9c0 con 0x7fd4dc156a70 2023-11-06T03:08:53.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc1603e0 con 0x7fd4dc156a70 2023-11-06T03:08:53.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400dd70 con 0x7fd4dc156a70 2023-11-06T03:08:53.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc15f810 con 0x7fd4dc156a70 2023-11-06T03:08:53.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc072750 con 0x7fd4dc156a70 2023-11-06T03:08:53.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd4d401e3b0 con 0x7fd4dc156a70 2023-11-06T03:08:53.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 0x7fd4c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd4d4083ed0 con 0x7fd4dc156a70 2023-11-06T03:08:53.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e1eac640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 0x7fd4c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.608+0000 7fd4e1eac640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 0x7fd4c0067f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8000f70 tx=0x7fd4d8007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.612+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd4d40516d0 con 0x7fd4dc156a70 2023-11-06T03:08:53.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4dc160890 con 0x7fd4dc156a70 2023-11-06T03:08:53.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4ce7fc640 1 -- 172.21.15.157:0/1607299890 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4d401a070 con 0x7fd4dc156a70 2023-11-06T03:08:53.755 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:08:53.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 msgr2=0x7fd4c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 0x7fd4c0067f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8000f70 tx=0x7fd4d8007c90 comp rx=0 tx=0).stop 2023-11-06T03:08:53.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 msgr2=0x7fd4dc160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400cef0 tx=0x7fd4d400a710 comp rx=0 tx=0).stop 2023-11-06T03:08:53.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 shutdown_connections 2023-11-06T03:08:53.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd4c0065a60 0x7fd4c0067f20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4dc156a70 0x7fd4dc160be0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4dc0adf40 0x7fd4dc15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 --2- 172.21.15.157:0/1607299890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4dc0adae0 0x7fd4dc15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 >> 172.21.15.157:0/1607299890 conn(0x7fd4dc065930 msgr2=0x7fd4dc15af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 shutdown_connections 2023-11-06T03:08:53.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.756+0000 7fd4e4110640 1 -- 172.21.15.157:0/1607299890 wait complete. 2023-11-06T03:08:53.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:08:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 Processor -- start 2023-11-06T03:08:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 -- start start 2023-11-06T03:08:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a5015d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a50160830 con 0x7f2a500cde90 2023-11-06T03:08:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a50160b10 con 0x7f2a5015cc70 2023-11-06T03:08:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a56104640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a501614b0 con 0x7f2a500d00a0 2023-11-06T03:08:53.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2a500cde90 0x7f2a500cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37132/0 (socket says 172.21.15.157:37132) 2023-11-06T03:08:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 -- 172.21.15.157:0/2904381560 learned_addr learned my addr 172.21.15.157:0/2904381560 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4ffff640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a5015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193198427 0 0) 0x7f2a50160830 con 0x7f2a500cde90 2023-11-06T03:08:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a30003530 con 0x7f2a500cde90 2023-11-06T03:08:53.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 59703504 0 0) 0x7f2a30003530 con 0x7f2a500cde90 2023-11-06T03:08:53.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 msgr2=0x7f2a5015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a5015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 >> v1:172.21.15.195:6789/0 conn(0x7f2a5015cc70 legacy=0x7f2a500d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a50160df0 con 0x7f2a500cde90 2023-11-06T03:08:53.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2a40001680 con 0x7f2a500cde90 2023-11-06T03:08:53.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 >> v1:172.21.15.157:6789/0 conn(0x7f2a500cde90 legacy=0x7f2a500cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a30003f30 0x7f2a300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a30004d10 0x7f2a30005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a501614b0 con 0x7f2a30003f30 2023-11-06T03:08:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3000bb00 con 0x7f2a30004d10 2023-11-06T03:08:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3000bcc0 con 0x7f2a500d00a0 2023-11-06T03:08:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a30003f30 0x7f2a300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a4f7fe640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 -- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 msgr2=0x7f2a300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a500d00a0 0x7f2a300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 -- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a30004d10 msgr2=0x7f2a30005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a30004d10 0x7f2a30005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.888+0000 7f2a46ffd640 1 -- 172.21.15.157:0/2904381560 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30003530 con 0x7f2a30003f30 2023-11-06T03:08:53.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a30003f30 0x7f2a300047d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2a40001060 tx=0x7f2a4001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a948783e847d7866 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400a5900 con 0x7f2a30003f30 2023-11-06T03:08:53.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a400a5a60 con 0x7f2a30003f30 2023-11-06T03:08:53.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4effd640 1 -- 172.21.15.157:0/2904381560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400a5e90 con 0x7f2a30003f30 2023-11-06T03:08:53.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- 172.21.15.157:0/2904381560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a30003f30 msgr2=0x7f2a300047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 --2- 172.21.15.157:0/2904381560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a30003f30 0x7f2a300047d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2a40001060 tx=0x7f2a4001cc00 comp rx=0 tx=0).stop 2023-11-06T03:08:53.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4ffff640 1 -- 172.21.15.157:0/2904381560 reap_dead start 2023-11-06T03:08:53.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- 172.21.15.157:0/2904381560 shutdown_connections 2023-11-06T03:08:53.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- 172.21.15.157:0/2904381560 >> 172.21.15.157:0/2904381560 conn(0x7f2a50070560 msgr2=0x7f2a50070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:53.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- 172.21.15.157:0/2904381560 shutdown_connections 2023-11-06T03:08:53.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- 172.21.15.157:0/2904381560 wait complete. 2023-11-06T03:08:53.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 Processor -- start 2023-11-06T03:08:53.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- start start 2023-11-06T03:08:53.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a30003f30 0x7f2a5015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a501f2640 0x7f2a500c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015fea0 con 0x7f2a500cbd10 2023-11-06T03:08:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015f0b0 con 0x7f2a501f2640 2023-11-06T03:08:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a56104640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500ca0b0 con 0x7f2a30003f30 2023-11-06T03:08:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54122/0 (socket says 172.21.15.157:54122) 2023-11-06T03:08:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 -- 172.21.15.157:0/1402961986 learned_addr learned my addr 172.21.15.157:0/1402961986 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4ffff640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a501f2640 0x7f2a500c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a4f7fe640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a30003f30 0x7f2a5015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 -- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a30003f30 msgr2=0x7f2a5015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a30003f30 0x7f2a5015c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 -- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a501f2640 msgr2=0x7f2a500c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a501f2640 0x7f2a500c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 -- 172.21.15.157:0/1402961986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5015e5e0 con 0x7f2a500cbd10 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.892+0000 7f2a46ffd640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2a40007040 tx=0x7f2a30005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40004c00 con 0x7f2a500cbd10 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a400b3070 con 0x7f2a500cbd10 2023-11-06T03:08:53.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5015e810 con 0x7f2a500cbd10 2023-11-06T03:08:53.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40001c50 con 0x7f2a500cbd10 2023-11-06T03:08:53.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5015edd0 con 0x7f2a500cbd10 2023-11-06T03:08:53.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a50072750 con 0x7f2a500cbd10 2023-11-06T03:08:53.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2a400070d0 con 0x7f2a500cbd10 2023-11-06T03:08:53.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 0x7f2a34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:53.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2a400b5070 con 0x7f2a500cbd10 2023-11-06T03:08:53.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4f7fe640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 0x7f2a34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:53.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.896+0000 7f2a4f7fe640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 0x7f2a34067f50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2a380015a0 tx=0x7f2a38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:53.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:53.900+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2a400eaeb0 con 0x7f2a500cbd10 2023-11-06T03:08:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.040+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a501610a0 con 0x7f2a500cbd10 2023-11-06T03:08:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.040+0000 7f2a4cff9640 1 -- 172.21.15.157:0/1402961986 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a400bf0a0 con 0x7f2a500cbd10 2023-11-06T03:08:54.041 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:08:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 msgr2=0x7f2a34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 0x7f2a34067f50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2a380015a0 tx=0x7f2a38009040 comp rx=0 tx=0).stop 2023-11-06T03:08:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 msgr2=0x7f2a5015e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2a40007040 tx=0x7f2a30005800 comp rx=0 tx=0).stop 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 shutdown_connections 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2a34065a90 0x7f2a34067f50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a501f2640 0x7f2a500c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a500cbd10 0x7f2a5015e0a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 --2- 172.21.15.157:0/1402961986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2a30003f30 0x7f2a5015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 >> 172.21.15.157:0/1402961986 conn(0x7f2a50070560 msgr2=0x7f2a500b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 shutdown_connections 2023-11-06T03:08:54.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.044+0000 7f2a56104640 1 -- 172.21.15.157:0/1402961986 wait complete. 2023-11-06T03:08:54.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:08:54.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 Processor -- start 2023-11-06T03:08:54.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 -- start start 2023-11-06T03:08:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40881599c0 0x7f40880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f40880b2af0 0x7f4088159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408815e440 con 0x7f408806d000 2023-11-06T03:08:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408806c240 con 0x7f40881599c0 2023-11-06T03:08:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408ec41640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408806c520 con 0x7f40880b2af0 2023-11-06T03:08:54.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40881599c0 0x7f40880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f40881599c0 0x7f40880ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54136/0 (socket says 172.21.15.157:54136) 2023-11-06T03:08:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 -- 172.21.15.157:0/1739218792 learned_addr learned my addr 172.21.15.157:0/1739218792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408d1de640 1 --2- 172.21.15.157:0/1739218792 >> v2:172.21.15.195:3300/0 conn(0x7f40880b2af0 0x7f4088159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 -- 172.21.15.157:0/1739218792 >> v2:172.21.15.195:3300/0 conn(0x7f40880b2af0 msgr2=0x7f4088159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:54.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 --2- 172.21.15.157:0/1739218792 >> v2:172.21.15.195:3300/0 conn(0x7f40880b2af0 0x7f4088159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 -- 172.21.15.157:0/1739218792 >> v1:172.21.15.195:6789/0 conn(0x7f408806d000 legacy=0x7f40880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.184+0000 7f408c9dd640 1 -- 172.21.15.157:0/1739218792 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408806c800 con 0x7f40881599c0 2023-11-06T03:08:54.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408c9dd640 1 --2- 172.21.15.157:0/1739218792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40881599c0 0x7f40880ae000 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f40880b19a0 tx=0x7f407801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675949db109c1c8f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f40877fe640 1 -- 172.21.15.157:0/1739218792 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780aa650 con 0x7f40881599c0 2023-11-06T03:08:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f40877fe640 1 -- 172.21.15.157:0/1739218792 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40780aa7b0 con 0x7f40881599c0 2023-11-06T03:08:54.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f40877fe640 1 -- 172.21.15.157:0/1739218792 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780aab40 con 0x7f40881599c0 2023-11-06T03:08:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- 172.21.15.157:0/1739218792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40881599c0 msgr2=0x7f40880ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 --2- 172.21.15.157:0/1739218792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40881599c0 0x7f40880ae000 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f40880b19a0 tx=0x7f407801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- 172.21.15.157:0/1739218792 shutdown_connections 2023-11-06T03:08:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 --2- 172.21.15.157:0/1739218792 >> v2:172.21.15.195:3300/0 conn(0x7f40880b2af0 0x7f4088159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 --2- 172.21.15.157:0/1739218792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40881599c0 0x7f40880ae000 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- 172.21.15.157:0/1739218792 >> 172.21.15.157:0/1739218792 conn(0x7f4088065930 msgr2=0x7f4088065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- 172.21.15.157:0/1739218792 shutdown_connections 2023-11-06T03:08:54.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- 172.21.15.157:0/1739218792 wait complete. 2023-11-06T03:08:54.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 Processor -- start 2023-11-06T03:08:54.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.188+0000 7f408ec41640 1 -- start start 2023-11-06T03:08:54.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f408806d000 0x7f408814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40881599c0 0x7f408814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408814c930 con 0x7f40880b2af0 2023-11-06T03:08:54.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408814c0a0 con 0x7f408806d000 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408814c370 con 0x7f40881599c0 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54150/0 (socket says 172.21.15.157:54150) 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 -- 172.21.15.157:0/778971015 learned_addr learned my addr 172.21.15.157:0/778971015 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408c9dd640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f408806d000 0x7f408814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408d1de640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40881599c0 0x7f408814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 -- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40881599c0 msgr2=0x7f408814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40881599c0 0x7f408814e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 -- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f408806d000 msgr2=0x7f408814bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f408806d000 0x7f408814bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 -- 172.21.15.157:0/778971015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780094d0 con 0x7f40880b2af0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f4087fff640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c500 tx=0x7f407c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c005220 con 0x7f40880b2af0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c005380 con 0x7f40880b2af0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408814c650 con 0x7f40880b2af0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c007270 con 0x7f40880b2af0 2023-11-06T03:08:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4088146430 con 0x7f40880b2af0 2023-11-06T03:08:54.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088146650 con 0x7f40880b2af0 2023-11-06T03:08:54.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f407c0073d0 con 0x7f40880b2af0 2023-11-06T03:08:54.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 0x7f4074067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f407c082c50 con 0x7f40880b2af0 2023-11-06T03:08:54.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.192+0000 7f408c9dd640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 0x7f4074067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.196+0000 7f408c9dd640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 0x7f4074067f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f40780ae000 tx=0x7f40780b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.196+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f407c088020 con 0x7f40880b2af0 2023-11-06T03:08:54.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4088072750 con 0x7f40880b2af0 2023-11-06T03:08:54.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f40857fa640 1 -- 172.21.15.157:0/778971015 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f407c050430 con 0x7f40880b2af0 2023-11-06T03:08:54.352 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:08:54.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 msgr2=0x7f4074067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 0x7f4074067f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f40780ae000 tx=0x7f40780b5040 comp rx=0 tx=0).stop 2023-11-06T03:08:54.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 msgr2=0x7f408814dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c500 tx=0x7f407c00c9c0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 shutdown_connections 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4074065a60 0x7f4074067f20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f40881599c0 0x7f408814e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40880b2af0 0x7f408814dd30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 --2- 172.21.15.157:0/778971015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f408806d000 0x7f408814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 >> 172.21.15.157:0/778971015 conn(0x7f4088065930 msgr2=0x7f40880b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 shutdown_connections 2023-11-06T03:08:54.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.352+0000 7f408ec41640 1 -- 172.21.15.157:0/778971015 wait complete. 2023-11-06T03:08:54.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:08:54.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.492+0000 7f757e964640 1 Processor -- start 2023-11-06T03:08:54.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.492+0000 7f757e964640 1 -- start start 2023-11-06T03:08:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757e964640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f757806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757e964640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75780b3b20 0x7f757815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757e964640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815f580 con 0x7f757815e5e0 2023-11-06T03:08:54.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757e964640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757815efc0 con 0x7f75780b3b20 2023-11-06T03:08:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757e964640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757815f2a0 con 0x7f75781551e0 2023-11-06T03:08:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75780b3b20 0x7f757815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f7577fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f757806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f7577fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f757806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:52584/0 (socket says 172.21.15.157:52584) 2023-11-06T03:08:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75780b3b20 0x7f757815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:60834/0 (socket says 172.21.15.157:60834) 2023-11-06T03:08:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f7577fff640 1 -- 172.21.15.157:0/111911350 learned_addr learned my addr 172.21.15.157:0/111911350 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:54.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f75777fe640 1 --1- 172.21.15.157:0/111911350 >> v1:172.21.15.195:6789/0 conn(0x7f757815e5e0 0x7f75780b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:38210/0 (socket says 172.21.15.157:38210) 2023-11-06T03:08:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 -- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 msgr2=0x7f757806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 --2- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f757806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 -- 172.21.15.157:0/111911350 >> v1:172.21.15.195:6789/0 conn(0x7f757815e5e0 legacy=0x7f75780b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.496+0000 7f757cf01640 1 -- 172.21.15.157:0/111911350 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757806f5a0 con 0x7f75780b3b20 2023-11-06T03:08:54.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757cf01640 1 --2- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f757815c1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7570009d90 tx=0x7f757001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca017c2f26fd5248 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f7576ffd640 1 -- 172.21.15.157:0/111911350 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700a5860 con 0x7f75780b3b20 2023-11-06T03:08:54.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f7576ffd640 1 -- 172.21.15.157:0/111911350 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700a59c0 con 0x7f75780b3b20 2023-11-06T03:08:54.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f7576ffd640 1 -- 172.21.15.157:0/111911350 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700a5d50 con 0x7f75780b3b20 2023-11-06T03:08:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 msgr2=0x7f757815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f757815c1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7570009d90 tx=0x7f757001c890 comp rx=0 tx=0).stop 2023-11-06T03:08:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- 172.21.15.157:0/111911350 shutdown_connections 2023-11-06T03:08:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f757815c1b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- 172.21.15.157:0/111911350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f757806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- 172.21.15.157:0/111911350 >> 172.21.15.157:0/111911350 conn(0x7f7578065930 msgr2=0x7f7578065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- 172.21.15.157:0/111911350 shutdown_connections 2023-11-06T03:08:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- 172.21.15.157:0/111911350 wait complete. 2023-11-06T03:08:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 Processor -- start 2023-11-06T03:08:54.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- start start 2023-11-06T03:08:54.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f7578145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f7578146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781464a0 con 0x7f757815e5e0 2023-11-06T03:08:54.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757814e0a0 con 0x7f75780b3b20 2023-11-06T03:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757e964640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757814e2f0 con 0x7f75781551e0 2023-11-06T03:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757cf01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757cf01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54160/0 (socket says 172.21.15.157:54160) 2023-11-06T03:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.500+0000 7f757cf01640 1 -- 172.21.15.157:0/253355634 learned_addr learned my addr 172.21.15.157:0/253355634 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:54.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7577fff640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f7578145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f75777fe640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f7578146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 -- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 msgr2=0x7f7578146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f7578146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 -- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 msgr2=0x7f7578145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f7578145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 -- 172.21.15.157:0/253355634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570009a70 con 0x7f757815e5e0 2023-11-06T03:08:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7577fff640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f7578145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f75777fe640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f7578146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757cf01640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f757001af40 tx=0x7f7570019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700b9860 con 0x7f757815e5e0 2023-11-06T03:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700b99c0 con 0x7f757815e5e0 2023-11-06T03:08:54.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757814e5d0 con 0x7f757815e5e0 2023-11-06T03:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700b9d30 con 0x7f757815e5e0 2023-11-06T03:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757814eb90 con 0x7f757815e5e0 2023-11-06T03:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578072750 con 0x7f757815e5e0 2023-11-06T03:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75700ad440 con 0x7f757815e5e0 2023-11-06T03:08:54.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 0x7f7564067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f757011e6e0 con 0x7f757815e5e0 2023-11-06T03:08:54.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7577fff640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 0x7f7564067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.504+0000 7f7577fff640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 0x7f7564067f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ad0 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.508+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f75700ebeb0 con 0x7f757815e5e0 2023-11-06T03:08:54.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.648+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7578071730 con 0x7f757815e5e0 2023-11-06T03:08:54.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.648+0000 7f7574ff9640 1 -- 172.21.15.157:0/253355634 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75700ef4d0 con 0x7f757815e5e0 2023-11-06T03:08:54.650 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:08:54.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 msgr2=0x7f7564067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 0x7f7564067f20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ad0 tx=0x7f7568008040 comp rx=0 tx=0).stop 2023-11-06T03:08:54.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 msgr2=0x7f7578146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f757001af40 tx=0x7f7570019630 comp rx=0 tx=0).stop 2023-11-06T03:08:54.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 shutdown_connections 2023-11-06T03:08:54.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7564065a60 0x7f7564067f20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f757815e5e0 0x7f7578146b40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75781551e0 0x7f7578146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 --2- 172.21.15.157:0/253355634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75780b3b20 0x7f7578145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 >> 172.21.15.157:0/253355634 conn(0x7f7578065930 msgr2=0x7f757815ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 shutdown_connections 2023-11-06T03:08:54.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.652+0000 7f757e964640 1 -- 172.21.15.157:0/253355634 wait complete. 2023-11-06T03:08:54.667 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:08:54.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 Processor -- start 2023-11-06T03:08:54.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 -- start start 2023-11-06T03:08:54.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf2815caa0 0x7faf280cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 0x7faf280cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf280d2920 con 0x7faf280ce1a0 2023-11-06T03:08:54.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf280d18a0 con 0x7faf2815caa0 2023-11-06T03:08:54.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf2d4f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf280d1b60 con 0x7faf2815ce80 2023-11-06T03:08:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf277fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faf280ce1a0 0x7faf280d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37144/0 (socket says 172.21.15.157:37144) 2023-11-06T03:08:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf277fe640 1 -- 172.21.15.157:0/2591016588 learned_addr learned my addr 172.21.15.157:0/2591016588 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:08:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf26ffd640 1 --2- 172.21.15.157:0/2591016588 >> v2:172.21.15.157:3300/0 conn(0x7faf2815caa0 0x7faf280cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf25ffb640 1 -- 172.21.15.157:0/2591016588 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719385808 0 0) 0x7faf280d2920 con 0x7faf280ce1a0 2023-11-06T03:08:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf267fc640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 0x7faf280cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf25ffb640 1 -- 172.21.15.157:0/2591016588 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf10003530 con 0x7faf280ce1a0 2023-11-06T03:08:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf26ffd640 1 -- 172.21.15.157:0/2591016588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 msgr2=0x7faf280cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:08:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf26ffd640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 0x7faf280cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf26ffd640 1 -- 172.21.15.157:0/2591016588 >> v1:172.21.15.157:6789/0 conn(0x7faf280ce1a0 legacy=0x7faf280d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf26ffd640 1 -- 172.21.15.157:0/2591016588 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf280d1e40 con 0x7faf2815caa0 2023-11-06T03:08:54.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.776+0000 7faf267fc640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 0x7faf280cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:08:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf26ffd640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815caa0 0x7faf280cdc60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7faf180097c0 tx=0x7faf1801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753b14609e8305e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf25ffb640 1 -- 172.21.15.157:0/2591016588 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180ad660 con 0x7faf2815caa0 2023-11-06T03:08:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf25ffb640 1 -- 172.21.15.157:0/2591016588 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf180ad7c0 con 0x7faf2815caa0 2023-11-06T03:08:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf25ffb640 1 -- 172.21.15.157:0/2591016588 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180adb50 con 0x7faf2815caa0 2023-11-06T03:08:54.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- 172.21.15.157:0/2591016588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815caa0 msgr2=0x7faf280cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815caa0 0x7faf280cdc60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7faf180097c0 tx=0x7faf1801c450 comp rx=0 tx=0).stop 2023-11-06T03:08:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- 172.21.15.157:0/2591016588 shutdown_connections 2023-11-06T03:08:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf2815ce80 0x7faf280cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/2591016588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815caa0 0x7faf280cdc60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- 172.21.15.157:0/2591016588 >> 172.21.15.157:0/2591016588 conn(0x7faf28070790 msgr2=0x7faf28070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- 172.21.15.157:0/2591016588 shutdown_connections 2023-11-06T03:08:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- 172.21.15.157:0/2591016588 wait complete. 2023-11-06T03:08:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 Processor -- start 2023-11-06T03:08:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.780+0000 7faf2d4f9640 1 -- start start 2023-11-06T03:08:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf280ce1a0 0x7faf280c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf2815caa0 0x7faf28147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf28159b90 con 0x7faf2815ce80 2023-11-06T03:08:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf28147d70 con 0x7faf2815caa0 2023-11-06T03:08:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf28158f10 con 0x7faf280ce1a0 2023-11-06T03:08:54.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54168/0 (socket says 172.21.15.157:54168) 2023-11-06T03:08:54.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 -- 172.21.15.157:0/450845192 learned_addr learned my addr 172.21.15.157:0/450845192 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:08:54.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 -- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf280ce1a0 msgr2=0x7faf280c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf280ce1a0 0x7faf280c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 -- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf2815caa0 msgr2=0x7faf28147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf267fc640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf2815caa0 0x7faf28147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf2815caa0 0x7faf28147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 -- 172.21.15.157:0/450845192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009420 con 0x7faf2815ce80 2023-11-06T03:08:54.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf277fe640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ca70 tx=0x7faf2000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000abd0 con 0x7faf2815ce80 2023-11-06T03:08:54.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000ad30 con 0x7faf2815ce80 2023-11-06T03:08:54.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20012600 con 0x7faf2815ce80 2023-11-06T03:08:54.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf281591f0 con 0x7faf2815ce80 2023-11-06T03:08:54.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf281597b0 con 0x7faf2815ce80 2023-11-06T03:08:54.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf28159cd0 con 0x7faf2815ce80 2023-11-06T03:08:54.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faf20007070 con 0x7faf2815ce80 2023-11-06T03:08:54.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 0x7faf14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:08:54.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7faf20083150 con 0x7faf2815ce80 2023-11-06T03:08:54.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.784+0000 7faf26ffd640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 0x7faf14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:08:54.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.788+0000 7faf26ffd640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 0x7faf14067f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf180af000 tx=0x7faf18004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:08:54.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.788+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faf28159cd0 con 0x7faf2815ce80 2023-11-06T03:08:54.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.952+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faf28159cd0 con 0x7faf2815ce80 2023-11-06T03:08:54.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.952+0000 7faf0f7fe640 1 -- 172.21.15.157:0/450845192 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7faf28159cd0 con 0x7faf2815ce80 2023-11-06T03:08:54.954 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:08:54.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 msgr2=0x7faf14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 0x7faf14067f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faf180af000 tx=0x7faf18004740 comp rx=0 tx=0).stop 2023-11-06T03:08:54.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 msgr2=0x7faf280ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:08:54.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ca70 tx=0x7faf2000cf30 comp rx=0 tx=0).stop 2023-11-06T03:08:54.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 shutdown_connections 2023-11-06T03:08:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf14065a60 0x7faf14067f20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf2815ce80 0x7faf280ccd10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf2815caa0 0x7faf28147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 --2- 172.21.15.157:0/450845192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf280ce1a0 0x7faf280c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:08:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 >> 172.21.15.157:0/450845192 conn(0x7faf28070790 msgr2=0x7faf28065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:08:54.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 shutdown_connections 2023-11-06T03:08:54.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:08:54.956+0000 7faf2d4f9640 1 -- 172.21.15.157:0/450845192 wait complete. 2023-11-06T03:08:54.973 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:08:54.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:08:54.974 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:08:54.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65176]: fuse finished with error 0 and tester_r 0 2023-11-06T03:08:54.986 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:01.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:09:01.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:01.089 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:09:01.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.264+0000 7f525afb2640 1 Processor -- start 2023-11-06T03:09:01.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525afb2640 1 -- start start 2023-11-06T03:09:01.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525afb2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f52540cfc90 0x7f52540d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525afb2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525415d210 con 0x7f52540b5f60 2023-11-06T03:09:01.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525afb2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52540ce090 con 0x7f525415ce30 2023-11-06T03:09:01.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525afb2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52540d17f0 con 0x7f52540cfc90 2023-11-06T03:09:01.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f52540b5f60 0x7f52540cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54760/0 (socket says 172.21.15.157:54760) 2023-11-06T03:09:01.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 -- 172.21.15.157:0/1184876405 learned_addr learned my addr 172.21.15.157:0/1184876405 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:01.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525954f640 1 --2- 172.21.15.157:0/1184876405 >> v2:172.21.15.195:3300/0 conn(0x7f52540cfc90 0x7f52540d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901484640 0 0) 0x7f525415d210 con 0x7f52540b5f60 2023-11-06T03:09:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5240003530 con 0x7f52540b5f60 2023-11-06T03:09:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3711289636 0 0) 0x7f5240003530 con 0x7f52540b5f60 2023-11-06T03:09:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 >> v2:172.21.15.195:3300/0 conn(0x7f52540cfc90 msgr2=0x7f52540d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 --2- 172.21.15.157:0/1184876405 >> v2:172.21.15.195:3300/0 conn(0x7f52540cfc90 0x7f52540d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 >> v1:172.21.15.195:6789/0 conn(0x7f525415ce30 legacy=0x7f52540cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52540ce370 con 0x7f52540b5f60 2023-11-06T03:09:01.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f523c001680 con 0x7f52540b5f60 2023-11-06T03:09:01.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 >> v1:172.21.15.157:6789/0 conn(0x7f52540b5f60 legacy=0x7f52540cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5240003980 0x7f5240003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 0x7f5240004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52400050f0 0x7f524000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540d17f0 con 0x7f5240004380 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524000c030 con 0x7f5240003980 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524000c1f0 con 0x7f52400050f0 2023-11-06T03:09:01.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 0x7f5240004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5258d4e640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5240003980 0x7f5240003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 -- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52400050f0 msgr2=0x7f524000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525954f640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52400050f0 0x7f524000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f52400050f0 0x7f524000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 -- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5240003980 msgr2=0x7f5240003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5240003980 0x7f5240003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5253fff640 1 -- 172.21.15.157:0/1184876405 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240003530 con 0x7f5240004380 2023-11-06T03:09:01.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f5258d4e640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5240003980 0x7f5240003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:01.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.268+0000 7f525954f640 1 -- 172.21.15.157:0/1184876405 reap_dead start 2023-11-06T03:09:01.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f5253fff640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 0x7f5240004850 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c190 tx=0x7f523c01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7838aabdbfee326 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c005ae0 con 0x7f5240004380 2023-11-06T03:09:01.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c005c60 con 0x7f5240004380 2023-11-06T03:09:01.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f52537fe640 1 -- 172.21.15.157:0/1184876405 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0072b0 con 0x7f5240004380 2023-11-06T03:09:01.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 msgr2=0x7f5240004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 0x7f5240004850 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c190 tx=0x7f523c01cb40 comp rx=0 tx=0).stop 2023-11-06T03:09:01.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- 172.21.15.157:0/1184876405 shutdown_connections 2023-11-06T03:09:01.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 --2- 172.21.15.157:0/1184876405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5240004380 0x7f5240004850 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- 172.21.15.157:0/1184876405 >> 172.21.15.157:0/1184876405 conn(0x7f5254070990 msgr2=0x7f5254065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- 172.21.15.157:0/1184876405 shutdown_connections 2023-11-06T03:09:01.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- 172.21.15.157:0/1184876405 wait complete. 2023-11-06T03:09:01.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 Processor -- start 2023-11-06T03:09:01.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- start start 2023-11-06T03:09:01.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5240004380 0x7f5254156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 0x7f52541525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254151110 con 0x7f5254152ae0 2023-11-06T03:09:01.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5254150e30 con 0x7f525414f5d0 2023-11-06T03:09:01.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525afb2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5254153370 con 0x7f5240004380 2023-11-06T03:09:01.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525954f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525954f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38856/0 (socket says 172.21.15.157:38856) 2023-11-06T03:09:01.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.272+0000 7f525954f640 1 -- 172.21.15.157:0/1090859950 learned_addr learned my addr 172.21.15.157:0/1090859950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f5253fff640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 0x7f52541525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f5258d4e640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5240004380 0x7f5254156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 -- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5240004380 msgr2=0x7f5254156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5240004380 0x7f5254156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 -- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 msgr2=0x7f52541525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 0x7f52541525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 -- 172.21.15.157:0/1090859950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254153650 con 0x7f5254152ae0 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f5253fff640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 0x7f52541525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525954f640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5254161a50 tx=0x7f5244010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400e8a0 con 0x7f5254152ae0 2023-11-06T03:09:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524400ea00 con 0x7f5254152ae0 2023-11-06T03:09:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5254153930 con 0x7f5254152ae0 2023-11-06T03:09:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524400edb0 con 0x7f5254152ae0 2023-11-06T03:09:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52541504c0 con 0x7f5254152ae0 2023-11-06T03:09:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5254153b20 con 0x7f5254152ae0 2023-11-06T03:09:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5244021460 con 0x7f5254152ae0 2023-11-06T03:09:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 0x7f5234067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5244083150 con 0x7f5254152ae0 2023-11-06T03:09:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f5258d4e640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 0x7f5234067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.276+0000 7f5258d4e640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 0x7f5234067f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f524c007fd0 tx=0x7f524c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.280+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5244050920 con 0x7f5254152ae0 2023-11-06T03:09:01.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5254072750 con 0x7f5254152ae0 2023-11-06T03:09:01.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f52517fa640 1 -- 172.21.15.157:0/1090859950 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f5244053f40 con 0x7f5254152ae0 2023-11-06T03:09:01.467 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:01.467 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2083483222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2083483222},{"type":"v1","addr":"172.21.15.195:6835","nonce":2083483222}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5752,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3999445109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3999445109},{"type":"v1","addr":"172.21.15.195:6837","nonce":3999445109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1572709083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1572709083},{"type":"v1","addr":"172.21.15.157:6833","nonce":1572709083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:08:45.318890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6741},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6741":{"gid":6741,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1218323165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1218323165},{"type":"v1","addr":"172.21.15.157:6835","nonce":1218323165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:01.467 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 47 2023-11-06T03:09:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 msgr2=0x7f5234067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 0x7f5234067f20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f524c007fd0 tx=0x7f524c009040 comp rx=0 tx=0).stop 2023-11-06T03:09:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 msgr2=0x7f52541574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5254161a50 tx=0x7f5244010b70 comp rx=0 tx=0).stop 2023-11-06T03:09:01.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 shutdown_connections 2023-11-06T03:09:01.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5234065a60 0x7f5234067f20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5254152ae0 0x7f52541574e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f525414f5d0 0x7f52541525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 --2- 172.21.15.157:0/1090859950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5240004380 0x7f5254156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 >> 172.21.15.157:0/1090859950 conn(0x7f5254070990 msgr2=0x7f525415f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 shutdown_connections 2023-11-06T03:09:01.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.468+0000 7f525afb2640 1 -- 172.21.15.157:0/1090859950 wait complete. 2023-11-06T03:09:01.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-06T03:09:01.630 INFO:teuthology.orchestra.run.smithi157.stdout:{"message":"","return_code":0} 2023-11-06T03:09:01.631 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T03:09:01.632 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:09:01.632 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:01.632 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:09:01.633 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:01.633 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:09:01.633 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:01.634 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:09:01.634 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:01.642 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:09:01.643 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:01.644 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:09:01.644 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:01.647 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:09:01.647 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:01.648 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:09:01.648 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:01.649 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2023-11-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 Processor -- start 2023-11-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- start start 2023-11-06T03:09:01.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21f80022a0 0x55b22cb59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b22cb5bcd0 0x55b22cbe99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22cabb940 con 0x55b22cb5ad80 2023-11-06T03:09:01.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22ca57450 con 0x55b22cb5bcd0 2023-11-06T03:09:01.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b22ca57150 con 0x7f21f80022a0 2023-11-06T03:09:01.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38884/0 (socket says 172.21.15.157:38884) 2023-11-06T03:09:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 -- 172.21.15.157:0/1565320544 learned_addr learned my addr 172.21.15.157:0/1565320544 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2210e8f640 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21f80022a0 0x55b22cb59d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2211690640 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b22cb5bcd0 0x55b22cbe99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 -- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21f80022a0 msgr2=0x55b22cb59d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21f80022a0 0x55b22cb59d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 -- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b22cb5bcd0 msgr2=0x55b22cbe99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b22cb5bcd0 0x55b22cbe99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 -- 172.21.15.157:0/1565320544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b22ca59430 con 0x55b22cb5ad80 2023-11-06T03:09:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f221068e640 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f220400c450 tx=0x7f220400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a595d5d389be026 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f220f68c640 1 -- 172.21.15.157:0/1565320544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400e920 con 0x55b22cb5ad80 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f220f68c640 1 -- 172.21.15.157:0/1565320544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220400ea80 con 0x55b22cb5ad80 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f220f68c640 1 -- 172.21.15.157:0/1565320544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400ee10 con 0x55b22cb5ad80 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- 172.21.15.157:0/1565320544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 msgr2=0x55b22cbe9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f220400c450 tx=0x7f220400c910 comp rx=0 tx=0).stop 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- 172.21.15.157:0/1565320544 shutdown_connections 2023-11-06T03:09:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b22cb5bcd0 0x55b22cbe99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cbe9460 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 --2- 172.21.15.157:0/1565320544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21f80022a0 0x55b22cb59d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- 172.21.15.157:0/1565320544 >> 172.21.15.157:0/1565320544 conn(0x55b22cb3e2f0 msgr2=0x55b22cb13ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- 172.21.15.157:0/1565320544 shutdown_connections 2023-11-06T03:09:01.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- 172.21.15.157:0/1565320544 wait complete. 2023-11-06T03:09:01.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 Processor -- start 2023-11-06T03:09:01.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.724+0000 7f2213375080 1 -- start start 2023-11-06T03:09:01.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cc813f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b22cb5bcd0 0x55b22cbe7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21f80022a0 0x55b22cb10220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22ca4ddc0 con 0x55b22cb5ad80 2023-11-06T03:09:01.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22cbeecd0 con 0x7f21f80022a0 2023-11-06T03:09:01.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b22cbeef70 con 0x55b22cb5bcd0 2023-11-06T03:09:01.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cc813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cc813f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38896/0 (socket says 172.21.15.157:38896) 2023-11-06T03:09:01.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 -- 172.21.15.157:0/3358066371 learned_addr learned my addr 172.21.15.157:0/3358066371 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2211690640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21f80022a0 0x55b22cb10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f221068e640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b22cb5bcd0 0x55b22cbe7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 -- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b22cb5bcd0 msgr2=0x55b22cbe7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b22cb5bcd0 0x55b22cbe7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 -- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21f80022a0 msgr2=0x55b22cb10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21f80022a0 0x55b22cb10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 -- 172.21.15.157:0/3358066371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b22ca65ff0 con 0x55b22cb5ad80 2023-11-06T03:09:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2210e8f640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cc813f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f220800a6e0 tx=0x7f2208005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 -- 172.21.15.157:0/3358066371 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208004810 con 0x55b22cb5ad80 2023-11-06T03:09:01.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 -- 172.21.15.157:0/3358066371 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2208004970 con 0x55b22cb5ad80 2023-11-06T03:09:01.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b22ca58e90 con 0x55b22cb5ad80 2023-11-06T03:09:01.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 -- 172.21.15.157:0/3358066371 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208004ce0 con 0x55b22cb5ad80 2023-11-06T03:09:01.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b22cbef4b0 con 0x55b22cb5ad80 2023-11-06T03:09:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 -- 172.21.15.157:0/3358066371 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22080b9470 con 0x55b22cb5ad80 2023-11-06T03:09:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21fc064410 0x7f21fc0668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f220e68a640 1 -- 172.21.15.157:0/3358066371 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f220811cea0 con 0x55b22cb5ad80 2023-11-06T03:09:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x55b22cbe7b80 0x55b22cbefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 --> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) v8 -- 0x55b22cb10760 con 0x55b22cbe7b80 2023-11-06T03:09:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f221068e640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21fc064410 0x7f21fc0668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.728+0000 7f2211690640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x55b22cbe7b80 0x55b22cbefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2211690640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x55b22cbe7b80 0x55b22cbefbf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f221068e640 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21fc064410 0x7f21fc0668d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f220400c950 tx=0x7f220401b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2211690640 1 -- 172.21.15.157:0/3358066371 <== osd.4 v2:172.21.15.195:6808/38425 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v69'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2200013450 con 0x55b22cbe7b80 2023-11-06T03:09:01.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x55b22cbe7b80 msgr2=0x55b22cbefbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x55b22cbe7b80 0x55b22cbefbf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21fc064410 msgr2=0x7f21fc0668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21fc064410 0x7f21fc0668d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f220400c950 tx=0x7f220401b040 comp rx=0 tx=0).stop 2023-11-06T03:09:01.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 msgr2=0x55b22cc813f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 --2- 172.21.15.157:0/3358066371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b22cb5ad80 0x55b22cc813f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f220800a6e0 tx=0x7f2208005d90 comp rx=0 tx=0).stop 2023-11-06T03:09:01.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2211690640 1 -- 172.21.15.157:0/3358066371 reap_dead start 2023-11-06T03:09:01.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 shutdown_connections 2023-11-06T03:09:01.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 >> 172.21.15.157:0/3358066371 conn(0x55b22cb3e2f0 msgr2=0x55b22cbeb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 shutdown_connections 2023-11-06T03:09:01.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.732+0000 7f2213375080 1 -- 172.21.15.157:0/3358066371 wait complete. 2023-11-06T03:09:01.745 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2023-11-06T03:09:01.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2023-11-06T03:09:01.746 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2023-11-06T03:09:01.746 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2023-11-06T03:09:01.746 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2023-11-06T03:09:01.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2023-11-06T03:09:01.747 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2023-11-06T03:09:01.748 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2023-11-06T03:09:01.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.896+0000 7fb8232f5640 1 Processor -- start 2023-11-06T03:09:01.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.896+0000 7fb8232f5640 1 -- start start 2023-11-06T03:09:01.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.896+0000 7fb8232f5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb81c15a700 0x7fb81c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.896+0000 7fb8232f5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 0x7fb81c065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb8232f5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0b2020 con 0x7fb81c065e70 2023-11-06T03:09:01.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb8232f5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0b2840 con 0x7fb81c15a700 2023-11-06T03:09:01.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb8232f5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0cc220 con 0x7fb81c15aae0 2023-11-06T03:09:01.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 Processor -- start 2023-11-06T03:09:01.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821892640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb81c065e70 0x7fb81c15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54784/0 (socket says 172.21.15.157:54784) 2023-11-06T03:09:01.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821892640 1 -- 172.21.15.157:0/203590891 learned_addr learned my addr 172.21.15.157:0/203590891 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:01.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 -- start start 2023-11-06T03:09:01.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 Processor -- start 2023-11-06T03:09:01.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 --2- 172.21.15.157:0/203590891 >> v2:172.21.15.157:3300/0 conn(0x7fb81c15a700 0x7fb81c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb820890640 1 --2- 172.21.15.157:0/203590891 >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 0x7fb81c065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 -- start start 2023-11-06T03:09:01.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb813fff640 1 -- 172.21.15.157:0/203590891 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167944342 0 0) 0x7fb81c0b2020 con 0x7fb81c065e70 2023-11-06T03:09:01.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb813fff640 1 -- 172.21.15.157:0/203590891 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb808003530 con 0x7fb81c065e70 2023-11-06T03:09:01.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 -- 172.21.15.157:0/203590891 >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 msgr2=0x7fb81c065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 --2- 172.21.15.157:0/203590891 >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 0x7fb81c065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 -- 172.21.15.157:0/203590891 >> v1:172.21.15.157:6789/0 conn(0x7fb81c065e70 legacy=0x7fb81c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 -- 172.21.15.157:0/203590891 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c066290 con 0x7fb81c15a700 2023-11-06T03:09:01.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb820890640 1 --2- 172.21.15.157:0/203590891 >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 0x7fb81c065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:01.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47280ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280710a0 con 0x7f472815d1f0 2023-11-06T03:09:01.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4728070000 con 0x7f4728160ef0 2023-11-06T03:09:01.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472cf16640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280702e0 con 0x7f47280adf80 2023-11-06T03:09:01.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 0x7fafa40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4725d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f472815d1f0 0x7f47281607e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54788/0 (socket says 172.21.15.157:54788) 2023-11-06T03:09:01.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4725d9b640 1 -- 172.21.15.157:0/1608155325 learned_addr learned my addr 172.21.15.157:0/1608155325 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:01.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa40d0050 con 0x7fafa40b6110 2023-11-06T03:09:01.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa40d1c60 con 0x7fafa415cdf0 2023-11-06T03:09:01.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa9eb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40d1f40 con 0x7fafa40cf6b0 2023-11-06T03:09:01.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fafa40b6110 0x7fafa40b65a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54802/0 (socket says 172.21.15.157:54802) 2023-11-06T03:09:01.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa3fff640 1 -- 172.21.15.157:0/1359087102 learned_addr learned my addr 172.21.15.157:0/1359087102 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:01.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472659c640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47280ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016785246 0 0) 0x7f4718001120 con 0x7f472815d1f0 2023-11-06T03:09:01.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4714003530 con 0x7f472815d1f0 2023-11-06T03:09:01.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa2ffd640 1 --2- 172.21.15.157:0/1359087102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 0x7fafa40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328062937 0 0) 0x7f4714003530 con 0x7f472815d1f0 2023-11-06T03:09:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 msgr2=0x7f47280ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47280ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 >> v1:172.21.15.195:6789/0 conn(0x7f4728160ef0 legacy=0x7f47280ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280705c0 con 0x7f472815d1f0 2023-11-06T03:09:01.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4718001810 con 0x7f472815d1f0 2023-11-06T03:09:01.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 >> v1:172.21.15.157:6789/0 conn(0x7f472815d1f0 legacy=0x7f47281607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714003f30 0x7f47140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f4714005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280702e0 con 0x7f4714004d10 2023-11-06T03:09:01.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471400ba70 con 0x7f4714003f30 2023-11-06T03:09:01.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471400bc30 con 0x7f47280adf80 2023-11-06T03:09:01.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262988695 0 0) 0x7fafa40d1c60 con 0x7fafa415cdf0 2023-11-06T03:09:01.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf8c003530 con 0x7fafa415cdf0 2023-11-06T03:09:01.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167872153 0 0) 0x7fafa40d0050 con 0x7fafa40b6110 2023-11-06T03:09:01.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa40d1c60 con 0x7fafa40b6110 2023-11-06T03:09:01.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f4714005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa2ffd640 1 -- 172.21.15.157:0/1359087102 >> v1:172.21.15.195:6789/0 conn(0x7fafa415cdf0 legacy=0x7fafa40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa2ffd640 1 -- 172.21.15.157:0/1359087102 >> v1:172.21.15.157:6789/0 conn(0x7fafa40b6110 legacy=0x7fafa40b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fafa2ffd640 1 -- 172.21.15.157:0/1359087102 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa40d2220 con 0x7fafa40cf6b0 2023-11-06T03:09:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4725d9b640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714003f30 0x7f47140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472659c640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 -- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 msgr2=0x7f47140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 -- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714003f30 msgr2=0x7f47140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714003f30 0x7f47140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4726d9d640 1 -- 172.21.15.157:0/1608155325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714003530 con 0x7f4714004d10 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f472659c640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47280adf80 0x7f47140039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:01.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f4725d9b640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4714003f30 0x7f47140047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f3964b28640 1 Processor -- start 2023-11-06T03:09:01.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7f3964b28640 1 -- start start 2023-11-06T03:09:01.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb821091640 1 --2- 172.21.15.157:0/203590891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c15a700 0x7fb81c0ccc90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0c9f50 tx=0x7fb81801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ea5e4251fead82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb813fff640 1 -- 172.21.15.157:0/203590891 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180a7070 con 0x7fb81c15a700 2023-11-06T03:09:01.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb813fff640 1 -- 172.21.15.157:0/203590891 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180ab430 con 0x7fb81c15a700 2023-11-06T03:09:01.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.900+0000 7fb813fff640 1 -- 172.21.15.157:0/203590891 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180ab7c0 con 0x7fb81c15a700 2023-11-06T03:09:01.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f3964b28640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f39600afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f3964b28640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600b1910 con 0x7f3960070330 2023-11-06T03:09:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f3964b28640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600af110 con 0x7f3960159930 2023-11-06T03:09:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f3964b28640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600afa20 con 0x7f396006da00 2023-11-06T03:09:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- 172.21.15.157:0/203590891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c15a700 msgr2=0x7fb81c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- 172.21.15.157:0/203590891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c15a700 0x7fb81c0ccc90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0c9f50 tx=0x7fb81801c430 comp rx=0 tx=0).stop 2023-11-06T03:09:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3960070330 0x7f39600cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54818/0 (socket says 172.21.15.157:54818) 2023-11-06T03:09:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 -- 172.21.15.157:0/2754126726 learned_addr learned my addr 172.21.15.157:0/2754126726 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:01.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- 172.21.15.157:0/203590891 shutdown_connections 2023-11-06T03:09:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- 172.21.15.157:0/203590891 >> v2:172.21.15.195:3300/0 conn(0x7fb81c15aae0 0x7fb81c065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- 172.21.15.157:0/203590891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c15a700 0x7fb81c0ccc90 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- 172.21.15.157:0/203590891 >> 172.21.15.157:0/203590891 conn(0x7fb81c06a170 msgr2=0x7fb81c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395dd9b640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f39600afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- 172.21.15.157:0/203590891 shutdown_connections 2023-11-06T03:09:01.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- 172.21.15.157:0/203590891 wait complete. 2023-11-06T03:09:01.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330332417 0 0) 0x7f39580014a0 con 0x7f3960070330 2023-11-06T03:09:01.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f394c003530 con 0x7f3960070330 2023-11-06T03:09:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242324622 0 0) 0x7f394c003530 con 0x7f3960070330 2023-11-06T03:09:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 msgr2=0x7f39600afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f39600afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 >> v1:172.21.15.195:6789/0 conn(0x7f3960159930 legacy=0x7f39600b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396006df50 con 0x7f3960070330 2023-11-06T03:09:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3958001b90 con 0x7f3960070330 2023-11-06T03:09:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 >> v1:172.21.15.157:6789/0 conn(0x7f3960070330 legacy=0x7f39600cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f394c0039b0 0x7f394c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f394c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f394c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600afa20 con 0x7f394c004d10 2023-11-06T03:09:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c004640 con 0x7f394c0039b0 2023-11-06T03:09:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c009c00 con 0x7f396006da00 2023-11-06T03:09:01.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f394c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa2ffd640 1 --2- 172.21.15.157:0/1359087102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 0x7fafa40cfb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf940097f0 tx=0x7faf9401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d5128f3019f20f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395e59c640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f394c0039b0 0x7f394c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395dd9b640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f394c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f4726d9d640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f4714005190 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4720001eb0 tx=0x7f472001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57e76378d105015 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720003bd0 con 0x7f4714004d10 2023-11-06T03:09:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720003d30 con 0x7f4714004d10 2023-11-06T03:09:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472559a640 1 -- 172.21.15.157:0/1608155325 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720003ea0 con 0x7f4714004d10 2023-11-06T03:09:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 -- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 msgr2=0x7f394c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f396006da00 0x7f394c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 -- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f394c0039b0 msgr2=0x7f394c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:01.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f394c0039b0 0x7f394c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f395ed9d640 1 -- 172.21.15.157:0/2754126726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c003530 con 0x7f394c004d10 2023-11-06T03:09:01.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940aa5f0 con 0x7fafa40cf6b0 2023-11-06T03:09:01.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf940aa750 con 0x7fafa40cf6b0 2023-11-06T03:09:01.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa27fc640 1 -- 172.21.15.157:0/1359087102 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940aaac0 con 0x7fafa40cf6b0 2023-11-06T03:09:01.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 -- 172.21.15.157:0/1608155325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 msgr2=0x7f4714005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 --2- 172.21.15.157:0/1608155325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f4714005190 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4720001eb0 tx=0x7f472001c760 comp rx=0 tx=0).stop 2023-11-06T03:09:01.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f4726d9d640 1 -- 172.21.15.157:0/1608155325 reap_dead start 2023-11-06T03:09:01.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 Processor -- start 2023-11-06T03:09:01.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- start start 2023-11-06T03:09:01.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 -- 172.21.15.157:0/1608155325 shutdown_connections 2023-11-06T03:09:01.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 -- 172.21.15.157:0/1608155325 >> 172.21.15.157:0/1608155325 conn(0x7f47280b0ca0 msgr2=0x7f47280b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 -- 172.21.15.157:0/1608155325 shutdown_connections 2023-11-06T03:09:01.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7f472cf16640 1 -- 172.21.15.157:0/1608155325 wait complete. 2023-11-06T03:09:01.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 -- 172.21.15.157:0/1359087102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 msgr2=0x7fafa40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 --2- 172.21.15.157:0/1359087102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 0x7fafa40cfb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf940097f0 tx=0x7faf9401c490 comp rx=0 tx=0).stop 2023-11-06T03:09:01.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 -- 172.21.15.157:0/1359087102 shutdown_connections 2023-11-06T03:09:01.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 --2- 172.21.15.157:0/1359087102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa40cf6b0 0x7fafa40cfb10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 -- 172.21.15.157:0/1359087102 >> 172.21.15.157:0/1359087102 conn(0x7fafa4065930 msgr2=0x7fafa4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 -- 172.21.15.157:0/1359087102 shutdown_connections 2023-11-06T03:09:01.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fafa9eb6640 1 -- 172.21.15.157:0/1359087102 wait complete. 2023-11-06T03:09:01.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb81c15a700 0x7fb81c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb81c15aae0 0x7fb81c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0c8450 con 0x7fb81c065e70 2023-11-06T03:09:01.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1f69c0 con 0x7fb81c15a700 2023-11-06T03:09:01.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb8232f5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0c8170 con 0x7fb81c15aae0 2023-11-06T03:09:01.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38928/0 (socket says 172.21.15.157:38928) 2023-11-06T03:09:01.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 -- 172.21.15.157:0/4224443306 learned_addr learned my addr 172.21.15.157:0/4224443306 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb820890640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb81c15a700 0x7fb81c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821892640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb81c15aae0 0x7fb81c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 -- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb81c15aae0 msgr2=0x7fb81c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb81c15aae0 0x7fb81c15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 -- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb81c15a700 msgr2=0x7fb81c156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb81c15a700 0x7fb81c156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 -- 172.21.15.157:0/4224443306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009420 con 0x7fb81c065e70 2023-11-06T03:09:01.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.904+0000 7fb821091640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb818019040 tx=0x7fb81801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180b5610 con 0x7fb81c065e70 2023-11-06T03:09:01.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f395ed9d640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f394c005190 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f395800a840 tx=0x7f395801d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f2a028221925d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 Processor -- start 2023-11-06T03:09:01.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c158cc0 con 0x7fb81c065e70 2023-11-06T03:09:01.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c159280 con 0x7fb81c065e70 2023-11-06T03:09:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580ad630 con 0x7f394c004d10 2023-11-06T03:09:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39580a8070 con 0x7f394c004d10 2023-11-06T03:09:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f395d59a640 1 -- 172.21.15.157:0/2754126726 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39580ad9f0 con 0x7f394c004d10 2023-11-06T03:09:01.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 Processor -- start 2023-11-06T03:09:01.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180b5790 con 0x7fb81c065e70 2023-11-06T03:09:01.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180b5b40 con 0x7fb81c065e70 2023-11-06T03:09:01.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 -- start start 2023-11-06T03:09:01.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa40b6110 0x7fafa40b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 0x7fafa4151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa414ace0 con 0x7fafa40b6110 2023-11-06T03:09:01.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa414a0d0 con 0x7fafa40cf6b0 2023-11-06T03:09:01.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa414a3b0 con 0x7fafa415cdf0 2023-11-06T03:09:01.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 -- start start 2023-11-06T03:09:01.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f47281538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47281578b0 0x7f47280c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 0x7f47281558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728153e10 con 0x7f4714004d10 2023-11-06T03:09:01.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280c4b60 con 0x7f47281578b0 2023-11-06T03:09:01.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472cf16640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728157d90 con 0x7f472815fa20 2023-11-06T03:09:01.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 -- 172.21.15.157:0/2754126726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 msgr2=0x7f394c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 --2- 172.21.15.157:0/2754126726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f394c005190 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f395800a840 tx=0x7f395801d320 comp rx=0 tx=0).stop 2023-11-06T03:09:01.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f395ed9d640 1 -- 172.21.15.157:0/2754126726 reap_dead start 2023-11-06T03:09:01.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 -- 172.21.15.157:0/2754126726 shutdown_connections 2023-11-06T03:09:01.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 -- 172.21.15.157:0/2754126726 >> 172.21.15.157:0/2754126726 conn(0x7f39600732f0 msgr2=0x7f3960069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:01.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 -- 172.21.15.157:0/2754126726 shutdown_connections 2023-11-06T03:09:01.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 -- 172.21.15.157:0/2754126726 wait complete. 2023-11-06T03:09:01.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 0x7fafa4151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:58590/0 (socket says 172.21.15.157:58590) 2023-11-06T03:09:01.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 0x7fafa4151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:58974/0 (socket says 172.21.15.157:58974) 2023-11-06T03:09:01.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 -- 172.21.15.157:0/2333484392 learned_addr learned my addr 172.21.15.157:0/2333484392 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f4725d9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47281578b0 0x7f47280c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f4726d9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 0x7f47281558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f47281538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f4726d9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 0x7f47281558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:58982/0 (socket says 172.21.15.157:58982) 2023-11-06T03:09:01.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f4726d9d640 1 -- 172.21.15.157:0/4145137692 learned_addr learned my addr 172.21.15.157:0/4145137692 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa37fe640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa40b6110 0x7fafa40b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c158b20 con 0x7fb81c065e70 2023-11-06T03:09:01.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb8180b5ca0 con 0x7fb81c065e70 2023-11-06T03:09:01.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 -- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 msgr2=0x7fafa4151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 0x7fafa4151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 -- 172.21.15.157:0/2333484392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa40b6110 msgr2=0x7fafa40b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa40b6110 0x7fafa40b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 -- 172.21.15.157:0/2333484392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94009420 con 0x7fafa40cf6b0 2023-11-06T03:09:01.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 -- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 msgr2=0x7f47281558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 0x7f47281558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 -- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47281578b0 msgr2=0x7f47280c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47281578b0 0x7f47280c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 -- 172.21.15.157:0/4145137692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728158070 con 0x7f4714004d10 2023-11-06T03:09:01.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa2ffd640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf940190a0 tx=0x7faf94019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f472659c640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f47281538d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4710010810 tx=0x7f4710010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 0x7fb800067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb81811de90 con 0x7fb81c065e70 2023-11-06T03:09:01.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94004450 con 0x7fafa40cf6b0 2023-11-06T03:09:01.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf940aa8c0 con 0x7fafa40cf6b0 2023-11-06T03:09:01.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940aac30 con 0x7fafa40cf6b0 2023-11-06T03:09:01.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fb820890640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 0x7fb800067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7fafa9eb6640 1 -- 172.21.15.157:0/2333484392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa414a690 con 0x7fafa40cf6b0 2023-11-06T03:09:01.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7fafa9eb6640 1 -- 172.21.15.157:0/2333484392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa40b4a70 con 0x7fafa40cf6b0 2023-11-06T03:09:01.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7fb820890640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 0x7fb800067f20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c1f8810 tx=0x7fb80c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c005260 con 0x7fafa40cf6b0 2023-11-06T03:09:01.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faf94003620 con 0x7fafa40cf6b0 2023-11-06T03:09:01.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7faf8bfff640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 0x7faf90067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7faf940a7070 con 0x7fafa40cf6b0 2023-11-06T03:09:01.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.912+0000 7fafa37fe640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 0x7faf90067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.908+0000 7f3964b28640 1 Processor -- start 2023-11-06T03:09:01.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7fafa37fe640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 0x7faf90067f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faf9c001570 tx=0x7faf9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 -- start start 2023-11-06T03:09:01.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb81c158b20 con 0x7fb81c065e70 2023-11-06T03:09:01.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3960140150 0x7f39600b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710004220 con 0x7f4714004d10 2023-11-06T03:09:01.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39600b2f50 0x7f396015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396015ca20 con 0x7f394c004d10 2023-11-06T03:09:01.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710004380 con 0x7f4714004d10 2023-11-06T03:09:01.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728158350 con 0x7f4714004d10 2023-11-06T03:09:01.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471000eb80 con 0x7f4714004d10 2023-11-06T03:09:01.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4728158910 con 0x7f4714004d10 2023-11-06T03:09:01.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38966/0 (socket says 172.21.15.157:38966) 2023-11-06T03:09:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 -- 172.21.15.157:0/4076710014 learned_addr learned my addr 172.21.15.157:0/4076710014 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b33d0 con 0x7f39600b2f50 2023-11-06T03:09:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396015a120 con 0x7f3960140150 2023-11-06T03:09:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395ed9d640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39600b2f50 0x7f396015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395dd9b640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3960140150 0x7f39600b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 -- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3960140150 msgr2=0x7f39600b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3960140150 0x7f39600b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 -- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39600b2f50 msgr2=0x7f396015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:01.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39600b2f50 0x7f396015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:01.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960070cd0 con 0x7f394c004d10 2023-11-06T03:09:01.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f395e59c640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f394800f740 tx=0x7f394800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4728072750 con 0x7f4714004d10 2023-11-06T03:09:01.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.916+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f471000ece0 con 0x7f4714004d10 2023-11-06T03:09:01.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f471effd640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 0x7f4708067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800fe50 con 0x7f394c004d10 2023-11-06T03:09:01.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4710083880 con 0x7f4714004d10 2023-11-06T03:09:01.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396015a400 con 0x7f394c004d10 2023-11-06T03:09:01.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396015a9c0 con 0x7f394c004d10 2023-11-06T03:09:01.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.920+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396006ad00 con 0x7f394c004d10 2023-11-06T03:09:01.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.924+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faf940eabc0 con 0x7fafa40cf6b0 2023-11-06T03:09:01.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.924+0000 7f4725d9b640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 0x7f4708067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4710051270 con 0x7f4714004d10 2023-11-06T03:09:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39480052e0 con 0x7f394c004d10 2023-11-06T03:09:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f4725d9b640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 0x7f4708067f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4718001940 tx=0x7f4718001610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800d930 con 0x7f394c004d10 2023-11-06T03:09:01.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f394800db10 con 0x7f394c004d10 2023-11-06T03:09:01.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f3956ffd640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 0x7f3940067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:01.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3948083c80 con 0x7f394c004d10 2023-11-06T03:09:01.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f395dd9b640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 0x7f3940067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:01.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.928+0000 7f395dd9b640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 0x7f3940067f50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f396015b950 tx=0x7f3950008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:01.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:01.936+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3948051590 con 0x7f394c004d10 2023-11-06T03:09:02.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.132+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7fb81c158b20 con 0x7fb81c065e70 2023-11-06T03:09:02.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.140+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7faf7c0057e0 con 0x7fafa40cf6b0 2023-11-06T03:09:02.143 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:09:02.144+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:09:02.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.152+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f472806e010 con 0x7f4714004d10 2023-11-06T03:09:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb811ffb640 1 -- 172.21.15.157:0/4224443306 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5749 v48) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb81c158b20 con 0x7fb81c065e70 2023-11-06T03:09:02.158 INFO:teuthology.orchestra.run.smithi157.stderr:failed mds gid 5749 2023-11-06T03:09:02.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7faf8bfff640 1 -- 172.21.15.157:0/2333484392 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6741 v48) v1 ==== 95+0+0 (secure 0 0 0) 0x7faf940ee1e0 con 0x7fafa40cf6b0 2023-11-06T03:09:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:failed mds gid 6741 2023-11-06T03:09:02.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 msgr2=0x7fb800067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 0x7fb800067f20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c1f8810 tx=0x7fb80c008040 comp rx=0 tx=0).stop 2023-11-06T03:09:02.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 msgr2=0x7fb81c1f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb818019040 tx=0x7fb81801ac70 comp rx=0 tx=0).stop 2023-11-06T03:09:02.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 shutdown_connections 2023-11-06T03:09:02.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb800065a60 0x7fb800067f20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb81c15aae0 0x7fb81c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb81c15a700 0x7fb81c156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 --2- 172.21.15.157:0/4224443306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb81c065e70 0x7fb81c1f6480 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 >> 172.21.15.157:0/4224443306 conn(0x7fb81c06a170 msgr2=0x7fb81c15fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 shutdown_connections 2023-11-06T03:09:02.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.160+0000 7fb8232f5640 1 -- 172.21.15.157:0/4224443306 wait complete. 2023-11-06T03:09:02.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 msgr2=0x7faf90067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 0x7faf90067f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faf9c001570 tx=0x7faf9c009040 comp rx=0 tx=0).stop 2023-11-06T03:09:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 msgr2=0x7fafa40b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf940190a0 tx=0x7faf94019230 comp rx=0 tx=0).stop 2023-11-06T03:09:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 shutdown_connections 2023-11-06T03:09:02.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faf90065a90 0x7faf90067f50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafa415cdf0 0x7fafa4151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafa40cf6b0 0x7fafa40b1d20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 --2- 172.21.15.157:0/2333484392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafa40b6110 0x7fafa40b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 >> 172.21.15.157:0/2333484392 conn(0x7fafa4065930 msgr2=0x7fafa40b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 shutdown_connections 2023-11-06T03:09:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7faf89ffb640 1 -- 172.21.15.157:0/2333484392 wait complete. 2023-11-06T03:09:02.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.164+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f3960072750 con 0x7f394c004d10 2023-11-06T03:09:02.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.172+0000 7f471effd640 1 -- 172.21.15.157:0/4145137692 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6738 v49) v1 ==== 95+0+0 (secure 0 0 0) 0x7f4710054890 con 0x7f4714004d10 2023-11-06T03:09:02.171 INFO:teuthology.orchestra.run.smithi157.stderr:failed mds gid 6738 2023-11-06T03:09:02.172 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:09:02.172+0000 7f09eab3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:09:02.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 msgr2=0x7f4708067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 0x7f4708067f20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4718001940 tx=0x7f4718001610 comp rx=0 tx=0).stop 2023-11-06T03:09:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 msgr2=0x7f47281538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f47281538d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4710010810 tx=0x7f4710010cd0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 shutdown_connections 2023-11-06T03:09:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4708065a60 0x7f4708067f20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f472815fa20 0x7f47281558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47281578b0 0x7f47280c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 --2- 172.21.15.157:0/4145137692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4714004d10 0x7f47281538d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 >> 172.21.15.157:0/4145137692 conn(0x7f47280b0ca0 msgr2=0x7f47280ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 shutdown_connections 2023-11-06T03:09:02.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.176+0000 7f472cf16640 1 -- 172.21.15.157:0/4145137692 wait complete. 2023-11-06T03:09:02.181 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:09:02.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:09:02.184 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:09:02.185 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:02.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3956ffd640 1 -- 172.21.15.157:0/4076710014 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5752 v50) v1 ==== 95+0+0 (secure 0 0 0) 0x7f3948054bb0 con 0x7f394c004d10 2023-11-06T03:09:02.187 INFO:teuthology.orchestra.run.smithi157.stderr:failed mds gid 5752 2023-11-06T03:09:02.188 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:09:02.189 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:09:02.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 msgr2=0x7f3940067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 0x7f3940067f50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f396015b950 tx=0x7f3950008040 comp rx=0 tx=0).stop 2023-11-06T03:09:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 msgr2=0x7f396015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f394800f740 tx=0x7f394800fc00 comp rx=0 tx=0).stop 2023-11-06T03:09:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 shutdown_connections 2023-11-06T03:09:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3940065a90 0x7f3940067f50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39600b2f50 0x7f396015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3960140150 0x7f39600b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 --2- 172.21.15.157:0/4076710014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f394c004d10 0x7f396015c4e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 >> 172.21.15.157:0/4076710014 conn(0x7f39600732f0 msgr2=0x7f39600cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 shutdown_connections 2023-11-06T03:09:02.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.188+0000 7f3964b28640 1 -- 172.21.15.157:0/4076710014 wait complete. 2023-11-06T03:09:02.194 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:09:02.195 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:02.197 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:09:02.198 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:09:02.207 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:09:02.208 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:09:02.211 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:09:02.212 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:02.247 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:09:02.248 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:09:02.249 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:02.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:02.263 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:09:02.278 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:09:02.305 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:09:02.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 Processor -- start 2023-11-06T03:09:02.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 -- start start 2023-11-06T03:09:02.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f267815ce80 0x7f26780cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f26780d00b0 0x7f26780b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780d2920 con 0x7f267815caa0 2023-11-06T03:09:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780d0520 con 0x7f267815ce80 2023-11-06T03:09:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267fe94640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780d18a0 con 0x7f26780d00b0 2023-11-06T03:09:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267d42f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f267815ce80 0x7f26780cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267dc30640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f267815caa0 0x7f26780cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54836/0 (socket says 172.21.15.157:54836) 2023-11-06T03:09:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.356+0000 7f267dc30640 1 -- 172.21.15.157:0/2320351500 learned_addr learned my addr 172.21.15.157:0/2320351500 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267d42f640 1 -- 172.21.15.157:0/2320351500 >> v2:172.21.15.195:3300/0 conn(0x7f26780d00b0 msgr2=0x7f26780b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267e431640 1 --2- 172.21.15.157:0/2320351500 >> v2:172.21.15.195:3300/0 conn(0x7f26780d00b0 0x7f26780b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267cc2e640 1 -- 172.21.15.157:0/2320351500 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994446405 0 0) 0x7f26780d2920 con 0x7f267815caa0 2023-11-06T03:09:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267d42f640 1 --2- 172.21.15.157:0/2320351500 >> v2:172.21.15.195:3300/0 conn(0x7f26780d00b0 0x7f26780b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267d42f640 1 -- 172.21.15.157:0/2320351500 >> v1:172.21.15.157:6789/0 conn(0x7f267815caa0 legacy=0x7f26780cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267d42f640 1 -- 172.21.15.157:0/2320351500 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780d1b60 con 0x7f267815ce80 2023-11-06T03:09:02.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267d42f640 1 --2- 172.21.15.157:0/2320351500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cfb70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f26780cf670 tx=0x7f266801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea3df11d6bf9a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:02.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267cc2e640 1 -- 172.21.15.157:0/2320351500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680a7070 con 0x7f267815ce80 2023-11-06T03:09:02.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267cc2e640 1 -- 172.21.15.157:0/2320351500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26680ab430 con 0x7f267815ce80 2023-11-06T03:09:02.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267cc2e640 1 -- 172.21.15.157:0/2320351500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680ab7c0 con 0x7f267815ce80 2023-11-06T03:09:02.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- 172.21.15.157:0/2320351500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 msgr2=0x7f26780cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 --2- 172.21.15.157:0/2320351500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cfb70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f26780cf670 tx=0x7f266801c430 comp rx=0 tx=0).stop 2023-11-06T03:09:02.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- 172.21.15.157:0/2320351500 shutdown_connections 2023-11-06T03:09:02.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 --2- 172.21.15.157:0/2320351500 >> v2:172.21.15.195:3300/0 conn(0x7f26780d00b0 0x7f26780b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 --2- 172.21.15.157:0/2320351500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cfb70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- 172.21.15.157:0/2320351500 >> 172.21.15.157:0/2320351500 conn(0x7f2678070790 msgr2=0x7f2678070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- 172.21.15.157:0/2320351500 shutdown_connections 2023-11-06T03:09:02.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- 172.21.15.157:0/2320351500 wait complete. 2023-11-06T03:09:02.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 Processor -- start 2023-11-06T03:09:02.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.360+0000 7f267fe94640 1 -- start start 2023-11-06T03:09:02.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 0x7f267815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 0x7f267815f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267815ee90 con 0x7f267815ce80 2023-11-06T03:09:02.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780ccc90 con 0x7f267815caa0 2023-11-06T03:09:02.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26781557b0 con 0x7f26780d00b0 2023-11-06T03:09:02.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39010/0 (socket says 172.21.15.157:39010) 2023-11-06T03:09:02.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 -- 172.21.15.157:0/2544240123 learned_addr learned my addr 172.21.15.157:0/2544240123 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:02.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267d42f640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 0x7f267815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267dc30640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 0x7f267815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 -- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 msgr2=0x7f267815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 0x7f267815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 -- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 msgr2=0x7f267815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 0x7f267815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 -- 172.21.15.157:0/2544240123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2668009420 con 0x7f267815ce80 2023-11-06T03:09:02.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267dc30640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 0x7f267815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:02.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267d42f640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 0x7f267815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:02.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267e431640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f267400cc90 tx=0x7f267400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:02.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400cea0 con 0x7f267815ce80 2023-11-06T03:09:02.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26740039a0 con 0x7f267815ce80 2023-11-06T03:09:02.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267815e280 con 0x7f267815ce80 2023-11-06T03:09:02.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674003d30 con 0x7f267815ce80 2023-11-06T03:09:02.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267815e840 con 0x7f267815ce80 2023-11-06T03:09:02.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267815f040 con 0x7f267815ce80 2023-11-06T03:09:02.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2674003e90 con 0x7f267815ce80 2023-11-06T03:09:02.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 0x7f265c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:02.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f2674083840 con 0x7f267815ce80 2023-11-06T03:09:02.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.364+0000 7f267dc30640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 0x7f265c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:02.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.368+0000 7f267dc30640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 0x7f265c067f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2670000f70 tx=0x7f2670009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:02.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.368+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f267815f040 con 0x7f267815ce80 2023-11-06T03:09:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.544+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f267815f040 con 0x7f267815ce80 2023-11-06T03:09:02.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f266e7fc640 1 -- 172.21.15.157:0/2544240123 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3902 (secure 0 0 0) 0x7f267815f040 con 0x7f267815ce80 2023-11-06T03:09:02.547 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:02.547 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:02.295663+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 51 2023-11-06T03:09:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 msgr2=0x7f265c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 0x7f265c067f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2670000f70 tx=0x7f2670009040 comp rx=0 tx=0).stop 2023-11-06T03:09:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 msgr2=0x7f26780cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f267400cc90 tx=0x7f267400a5c0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 shutdown_connections 2023-11-06T03:09:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f265c065a60 0x7f265c067f20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f267815ce80 0x7f26780cc750 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f267815caa0 0x7f267815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 --2- 172.21.15.157:0/2544240123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f26780d00b0 0x7f267815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:02.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 >> 172.21.15.157:0/2544240123 conn(0x7f2678070790 msgr2=0x7f26780d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:02.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 shutdown_connections 2023-11-06T03:09:02.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:02.548+0000 7f267fe94640 1 -- 172.21.15.157:0/2544240123 wait complete. 2023-11-06T03:09:02.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:08:43.305986+0000', 'modified': '2023-11-06T03:09:02.295663+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7038}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7038': {'gid': 7038, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/1223634033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1223634033}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1223634033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:09:02.969 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7038:up:replay 2023-11-06T03:09:03.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:04.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.096+0000 7f033e9bf640 1 Processor -- start 2023-11-06T03:09:04.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033e9bf640 1 -- start start 2023-11-06T03:09:04.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033e9bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03381599c0 0x7f03380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:04.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033e9bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033815e440 con 0x7f03380b2af0 2023-11-06T03:09:04.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033e9bf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033806c240 con 0x7f033806d000 2023-11-06T03:09:04.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033e9bf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033806c520 con 0x7f03381599c0 2023-11-06T03:09:04.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f0337fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03381599c0 0x7f03380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:04.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033cf5c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f03380b2af0 0x7f0338159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54842/0 (socket says 172.21.15.157:54842) 2023-11-06T03:09:04.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f033cf5c640 1 -- 172.21.15.157:0/4115300850 learned_addr learned my addr 172.21.15.157:0/4115300850 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:04.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f0337fff640 1 -- 172.21.15.157:0/4115300850 >> v1:172.21.15.195:6789/0 conn(0x7f033806d000 legacy=0x7f03380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:04.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f0337fff640 1 -- 172.21.15.157:0/4115300850 >> v1:172.21.15.157:6789/0 conn(0x7f03380b2af0 legacy=0x7f0338159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f03377fe640 1 -- 172.21.15.157:0/4115300850 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247909772 0 0) 0x7f033815e440 con 0x7f03380b2af0 2023-11-06T03:09:04.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.100+0000 7f0337fff640 1 -- 172.21.15.157:0/4115300850 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033806c800 con 0x7f03381599c0 2023-11-06T03:09:04.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f0337fff640 1 --2- 172.21.15.157:0/4115300850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03381599c0 0x7f03380ae000 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f03280097c0 tx=0x7f032801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be171b9fc5d3ac6d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:04.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f03377fe640 1 -- 172.21.15.157:0/4115300850 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280ab670 con 0x7f03381599c0 2023-11-06T03:09:04.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f03377fe640 1 -- 172.21.15.157:0/4115300850 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03280ab7d0 con 0x7f03381599c0 2023-11-06T03:09:04.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f03377fe640 1 -- 172.21.15.157:0/4115300850 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280abb60 con 0x7f03381599c0 2023-11-06T03:09:04.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- 172.21.15.157:0/4115300850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03381599c0 msgr2=0x7f03380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 --2- 172.21.15.157:0/4115300850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03381599c0 0x7f03380ae000 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f03280097c0 tx=0x7f032801c450 comp rx=0 tx=0).stop 2023-11-06T03:09:04.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- 172.21.15.157:0/4115300850 shutdown_connections 2023-11-06T03:09:04.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 --2- 172.21.15.157:0/4115300850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03381599c0 0x7f03380ae000 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- 172.21.15.157:0/4115300850 >> 172.21.15.157:0/4115300850 conn(0x7f0338065930 msgr2=0x7f0338065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:04.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- 172.21.15.157:0/4115300850 shutdown_connections 2023-11-06T03:09:04.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- 172.21.15.157:0/4115300850 wait complete. 2023-11-06T03:09:04.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 Processor -- start 2023-11-06T03:09:04.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- start start 2023-11-06T03:09:04.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f033806d000 0x7f0338137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:04.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:04.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03381599c0 0x7f0338139b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:04.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033813aa90 con 0x7f03380b2af0 2023-11-06T03:09:04.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033813a4d0 con 0x7f03381599c0 2023-11-06T03:09:04.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033e9bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033813a7b0 con 0x7f033806d000 2023-11-06T03:09:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39034/0 (socket says 172.21.15.157:39034) 2023-11-06T03:09:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 -- 172.21.15.157:0/868671856 learned_addr learned my addr 172.21.15.157:0/868671856 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f033cf5c640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03381599c0 0x7f0338139b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:04.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f0337fff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f033806d000 0x7f0338137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:04.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 -- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f033806d000 msgr2=0x7f0338137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f033806d000 0x7f0338137af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 -- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03381599c0 msgr2=0x7f0338139b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03381599c0 0x7f0338139b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.104+0000 7f032ffff640 1 -- 172.21.15.157:0/868671856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009420 con 0x7f03380b2af0 2023-11-06T03:09:04.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f032ffff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f032000c500 tx=0x7f032000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:04.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320005220 con 0x7f03380b2af0 2023-11-06T03:09:04.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320005380 con 0x7f03380b2af0 2023-11-06T03:09:04.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033814bca0 con 0x7f03380b2af0 2023-11-06T03:09:04.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320007270 con 0x7f03380b2af0 2023-11-06T03:09:04.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0338138030 con 0x7f03380b2af0 2023-11-06T03:09:04.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338072750 con 0x7f03380b2af0 2023-11-06T03:09:04.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f03200073d0 con 0x7f03380b2af0 2023-11-06T03:09:04.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 0x7f031c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:04.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f0320082d50 con 0x7f03380b2af0 2023-11-06T03:09:04.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f0337fff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 0x7f031c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:04.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.108+0000 7f0337fff640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 0x7f031c067f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03280af000 tx=0x7f0328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:04.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.112+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0320086030 con 0x7f03380b2af0 2023-11-06T03:09:04.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.288+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f033814c150 con 0x7f03380b2af0 2023-11-06T03:09:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.288+0000 7f03357fa640 1 -- 172.21.15.157:0/868671856 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4665 (secure 0 0 0) 0x7f0320050480 con 0x7f03380b2af0 2023-11-06T03:09:04.288 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":52,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:03.301489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:04.289 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 52 2023-11-06T03:09:04.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 msgr2=0x7f031c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 0x7f031c067f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03280af000 tx=0x7f0328008040 comp rx=0 tx=0).stop 2023-11-06T03:09:04.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 msgr2=0x7f033814dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f032000c500 tx=0x7f032000c9c0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 shutdown_connections 2023-11-06T03:09:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f031c065a60 0x7f031c067f20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03381599c0 0x7f0338139b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03380b2af0 0x7f033814dfb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 --2- 172.21.15.157:0/868671856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f033806d000 0x7f0338137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 >> 172.21.15.157:0/868671856 conn(0x7f0338065930 msgr2=0x7f033815c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 shutdown_connections 2023-11-06T03:09:04.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:04.292+0000 7f033e9bf640 1 -- 172.21.15.157:0/868671856 wait complete. 2023-11-06T03:09:04.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:08:43.305986+0000', 'modified': '2023-11-06T03:09:03.301489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7038}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7038': {'gid': 7038, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.157:6833/1223634033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1223634033}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1223634033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:09:04.710 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7038:up:reconnect 2023-11-06T03:09:05.711 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:05.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.844+0000 7f3c53602640 1 Processor -- start 2023-11-06T03:09:05.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.844+0000 7f3c53602640 1 -- start start 2023-11-06T03:09:05.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c53602640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c53602640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c4c0b2af0 0x7f3c4c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c53602640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4c15e440 con 0x7f3c4c06d000 2023-11-06T03:09:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c53602640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4c06c240 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c53602640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06c520 con 0x7f3c4c1599c0 2023-11-06T03:09:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c4c0b2af0 0x7f3c4c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c4c0b2af0 0x7f3c4c159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39040/0 (socket says 172.21.15.157:39040) 2023-11-06T03:09:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 -- 172.21.15.157:0/1798732135 learned_addr learned my addr 172.21.15.157:0/1798732135 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:05.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c5139e640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 -- 172.21.15.157:0/1798732135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 msgr2=0x7f3c4c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:05.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 -- 172.21.15.157:0/1798732135 >> v1:172.21.15.157:6789/0 conn(0x7f3c4c06d000 legacy=0x7f3c4c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:05.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 -- 172.21.15.157:0/1798732135 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c06c800 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c51b9f640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c159da0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009e50 tx=0x7f3c4401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc349ff29513079c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:05.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c43fff640 1 -- 172.21.15.157:0/1798732135 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440a58d0 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c43fff640 1 -- 172.21.15.157:0/1798732135 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440a5a30 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.848+0000 7f3c43fff640 1 -- 172.21.15.157:0/1798732135 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440a5dc0 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/1798732135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c0b2af0 msgr2=0x7f3c4c159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:05.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c159da0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009e50 tx=0x7f3c4401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/1798732135 shutdown_connections 2023-11-06T03:09:05.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c159da0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- 172.21.15.157:0/1798732135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/1798732135 >> 172.21.15.157:0/1798732135 conn(0x7f3c4c065930 msgr2=0x7f3c4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:05.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/1798732135 shutdown_connections 2023-11-06T03:09:05.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/1798732135 wait complete. 2023-11-06T03:09:05.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 Processor -- start 2023-11-06T03:09:05.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- start start 2023-11-06T03:09:05.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c14eba0 con 0x7f3c4c06d000 2023-11-06T03:09:05.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c14ee80 con 0x7f3c4c0b2af0 2023-11-06T03:09:05.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c14f160 con 0x7f3c4c1599c0 2023-11-06T03:09:05.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39044/0 (socket says 172.21.15.157:39044) 2023-11-06T03:09:05.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 -- 172.21.15.157:0/4075976288 learned_addr learned my addr 172.21.15.157:0/4075976288 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:05.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c51b9f640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c50b9d640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 -- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 msgr2=0x7f3c4c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:05.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 -- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c4c0b2af0 msgr2=0x7f3c4c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:05.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:05.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 -- 172.21.15.157:0/4075976288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44009b30 con 0x7f3c4c06d000 2023-11-06T03:09:05.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c5139e640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800d5f0 tx=0x7f3c4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:05.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480125f0 con 0x7f3c4c06d000 2023-11-06T03:09:05.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c48012750 con 0x7f3c4c06d000 2023-11-06T03:09:05.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c14f440 con 0x7f3c4c06d000 2023-11-06T03:09:05.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48012ae0 con 0x7f3c4c06d000 2023-11-06T03:09:05.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.852+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c158030 con 0x7f3c4c06d000 2023-11-06T03:09:05.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c1582b0 con 0x7f3c4c06d000 2023-11-06T03:09:05.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3c48012c40 con 0x7f3c4c06d000 2023-11-06T03:09:05.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c41ffb640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 0x7f3c30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:05.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f3c48083930 con 0x7f3c4c06d000 2023-11-06T03:09:05.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c50b9d640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 0x7f3c30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:05.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.856+0000 7f3c50b9d640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 0x7f3c30067f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c1591a0 tx=0x7f3c3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:05.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:05.860+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3c48051050 con 0x7f3c4c06d000 2023-11-06T03:09:06.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.048+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c4c15b550 con 0x7f3c4c06d000 2023-11-06T03:09:06.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.048+0000 7f3c41ffb640 1 -- 172.21.15.157:0/4075976288 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f3c48054670 con 0x7f3c4c06d000 2023-11-06T03:09:06.050 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:06.050 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:05.306735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:06.050 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-11-06T03:09:06.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 msgr2=0x7f3c30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 0x7f3c30067f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c1591a0 tx=0x7f3c3c009040 comp rx=0 tx=0).stop 2023-11-06T03:09:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 msgr2=0x7f3c4c157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800d5f0 tx=0x7f3c4800dab0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 shutdown_connections 2023-11-06T03:09:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c30065a60 0x7f3c30067f20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c4c1599c0 0x7f3c4c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c4c0b2af0 0x7f3c4c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 --2- 172.21.15.157:0/4075976288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c4c06d000 0x7f3c4c157af0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 >> 172.21.15.157:0/4075976288 conn(0x7f3c4c065930 msgr2=0x7f3c4c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 shutdown_connections 2023-11-06T03:09:06.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.052+0000 7f3c53602640 1 -- 172.21.15.157:0/4075976288 wait complete. 2023-11-06T03:09:06.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:08:43.305986+0000', 'modified': '2023-11-06T03:09:05.306735+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 73, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7038}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7038': {'gid': 7038, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6833/1223634033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 1223634033}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 1223634033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:09:06.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:09:06.472 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:09:06.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 Processor -- start 2023-11-06T03:09:06.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 -- start start 2023-11-06T03:09:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f23b806d000 0x7f23b80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b815e440 con 0x7f23b81599c0 2023-11-06T03:09:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b806c240 con 0x7f23b80b2af0 2023-11-06T03:09:06.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bfb9b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b806c520 con 0x7f23b806d000 2023-11-06T03:09:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f23b81599c0 0x7f23b80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54868/0 (socket says 172.21.15.157:54868) 2023-11-06T03:09:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 -- 172.21.15.157:0/2773321898 learned_addr learned my addr 172.21.15.157:0/2773321898 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23be138640 1 --2- 172.21.15.157:0/2773321898 >> v2:172.21.15.195:3300/0 conn(0x7f23b806d000 0x7f23b80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275760197 0 0) 0x7f23b815e440 con 0x7f23b81599c0 2023-11-06T03:09:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a4003530 con 0x7f23b81599c0 2023-11-06T03:09:06.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283752043 0 0) 0x7f23a4003530 con 0x7f23b81599c0 2023-11-06T03:09:06.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 >> v2:172.21.15.195:3300/0 conn(0x7f23b806d000 msgr2=0x7f23b80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 --2- 172.21.15.157:0/2773321898 >> v2:172.21.15.195:3300/0 conn(0x7f23b806d000 0x7f23b80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 >> v1:172.21.15.195:6789/0 conn(0x7f23b80b2af0 legacy=0x7f23b8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b806c800 con 0x7f23b81599c0 2023-11-06T03:09:06.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f23b4001680 con 0x7f23b81599c0 2023-11-06T03:09:06.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 >> v1:172.21.15.157:6789/0 conn(0x7f23b81599c0 legacy=0x7f23b80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 0x7f23a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23a40047d0 0x7f23a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4005190 0x7f23a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806c520 con 0x7f23a4003980 2023-11-06T03:09:06.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400c030 con 0x7f23a4005190 2023-11-06T03:09:06.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400e400 con 0x7f23a40047d0 2023-11-06T03:09:06.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 0x7f23a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23be138640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4005190 0x7f23a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd136640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23a40047d0 0x7f23a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 -- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23a40047d0 msgr2=0x7f23a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23a40047d0 0x7f23a4004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 -- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4005190 msgr2=0x7f23a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4005190 0x7f23a400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23bd937640 1 -- 172.21.15.157:0/2773321898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4003530 con 0x7f23a4003980 2023-11-06T03:09:06.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23be138640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4005190 0x7f23a400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:06.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.604+0000 7f23be138640 1 -- 172.21.15.157:0/2773321898 reap_dead start 2023-11-06T03:09:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bd937640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 0x7f23a4003da0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c240 tx=0x7f23b401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a17a3dd63b0756a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40058e0 con 0x7f23a4003980 2023-11-06T03:09:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4005a60 con 0x7f23a4003980 2023-11-06T03:09:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bc935640 1 -- 172.21.15.157:0/2773321898 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4005e10 con 0x7f23a4003980 2023-11-06T03:09:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 -- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 msgr2=0x7f23a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 0x7f23a4003da0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c240 tx=0x7f23b401cbd0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 -- 172.21.15.157:0/2773321898 shutdown_connections 2023-11-06T03:09:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/2773321898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23a4003980 0x7f23a4003da0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 -- 172.21.15.157:0/2773321898 >> 172.21.15.157:0/2773321898 conn(0x7f23b8065930 msgr2=0x7f23b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 -- 172.21.15.157:0/2773321898 shutdown_connections 2023-11-06T03:09:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.608+0000 7f23bfb9b640 1 -- 172.21.15.157:0/2773321898 wait complete. 2023-11-06T03:09:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 Processor -- start 2023-11-06T03:09:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- start start 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4003980 0x7f23b8141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b81426b0 0x7f23b80cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8141e80 con 0x7f23b8143c60 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8142bc0 con 0x7f23a4003980 2023-11-06T03:09:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8142ea0 con 0x7f23b81426b0 2023-11-06T03:09:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39066/0 (socket says 172.21.15.157:39066) 2023-11-06T03:09:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 -- 172.21.15.157:0/1525479122 learned_addr learned my addr 172.21.15.157:0/1525479122 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd937640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4003980 0x7f23b8141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23be138640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b81426b0 0x7f23b80cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 -- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b81426b0 msgr2=0x7f23b80cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b81426b0 0x7f23b80cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 -- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4003980 msgr2=0x7f23b8141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4003980 0x7f23b8141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 -- 172.21.15.157:0/1525479122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8143180 con 0x7f23b8143c60 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd136640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d4c0 tx=0x7f23a800d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 -- 172.21.15.157:0/1525479122 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8005480 con 0x7f23b8143c60 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 -- 172.21.15.157:0/1525479122 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8003620 con 0x7f23b8143c60 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b8143460 con 0x7f23b8143c60 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 -- 172.21.15.157:0/1525479122 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80039f0 con 0x7f23b8143c60 2023-11-06T03:09:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8155420 con 0x7f23b8143c60 2023-11-06T03:09:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 -- 172.21.15.157:0/1525479122 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23a8003b50 con 0x7f23b8143c60 2023-11-06T03:09:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 0x7f239c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23ae7fc640 1 -- 172.21.15.157:0/1525479122 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f23a8083390 con 0x7f23b8143c60 2023-11-06T03:09:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.612+0000 7f23bd937640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 0x7f239c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 Processor -- start 2023-11-06T03:09:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- start start 2023-11-06T03:09:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23bd937640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 0x7f239c067f20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f23b40ab000 tx=0x7f23b40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca064550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9f2cfc0 con 0x5586ca064a90 2023-11-06T03:09:06.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9ebb1a0 con 0x5586ca0ae6e0 2023-11-06T03:09:06.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eb8770 con 0x5586ca0af000 2023-11-06T03:09:06.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39082/0 (socket says 172.21.15.157:39082) 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 -- 172.21.15.157:0/887096786 learned_addr learned my addr 172.21.15.157:0/887096786 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23bd136640 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca064550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 -- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 msgr2=0x5586ca064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca064550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 -- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 msgr2=0x5586ca0aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 -- 172.21.15.157:0/887096786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c9bb0890 con 0x5586ca064a90 2023-11-06T03:09:06.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23be138640 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f23b80b19a0 tx=0x7f23b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b452faca96767d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f239b7fe640 1 -- 172.21.15.157:0/887096786 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000e070 con 0x5586ca064a90 2023-11-06T03:09:06.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f239b7fe640 1 -- 172.21.15.157:0/887096786 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b000f040 con 0x5586ca064a90 2023-11-06T03:09:06.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f239b7fe640 1 -- 172.21.15.157:0/887096786 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0003dc0 con 0x5586ca064a90 2023-11-06T03:09:06.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- 172.21.15.157:0/887096786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 msgr2=0x5586ca064e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f23b80b19a0 tx=0x7f23b0009750 comp rx=0 tx=0).stop 2023-11-06T03:09:06.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- 172.21.15.157:0/887096786 shutdown_connections 2023-11-06T03:09:06.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca064a90 0x5586ca064e70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca064550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/887096786 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- 172.21.15.157:0/887096786 >> 172.21.15.157:0/887096786 conn(0x5586ca06abf0 msgr2=0x5586ca062e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- 172.21.15.157:0/887096786 shutdown_connections 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- 172.21.15.157:0/887096786 wait complete. 2023-11-06T03:09:06.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 -- start start 2023-11-06T03:09:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.616+0000 7f23c2cc3000 1 Processor -- start 2023-11-06T03:09:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0ae6e0 0x5586ca06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eb4060 con 0x5586ca0af000 2023-11-06T03:09:06.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9e53290 con 0x5586ca064a90 2023-11-06T03:09:06.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c9e716f0 con 0x5586ca0ae6e0 2023-11-06T03:09:06.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 10 client.0 _ll_register_callbacks cb 0x5586ca0501d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:06.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39098/0 (socket says 172.21.15.157:39098) 2023-11-06T03:09:06.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 -- 172.21.15.157:0/483686196 learned_addr learned my addr 172.21.15.157:0/483686196 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23bd136640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0ae6e0 0x5586ca06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23bd937640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 -- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0ae6e0 msgr2=0x5586ca06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0ae6e0 0x5586ca06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 -- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 msgr2=0x5586ca06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca06fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 -- 172.21.15.157:0/483686196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b0009d10 con 0x5586ca0af000 2023-11-06T03:09:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23be138640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f23b806c130 tx=0x7f23b0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83547b8f31a83697 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0004610 con 0x5586ca0af000 2023-11-06T03:09:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f23b0029d80 con 0x5586ca0af000 2023-11-06T03:09:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586c9eb29b0 con 0x5586ca0af000 2023-11-06T03:09:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b0003e50 con 0x5586ca0af000 2023-11-06T03:09:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 10 client.7098 fetch_fsmap learned FSMap version 54 2023-11-06T03:09:06.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5586c9e791e0 con 0x5586ca0af000 2023-11-06T03:09:06.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586c9eb29b0 con 0x5586ca0af000 2023-11-06T03:09:06.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 54) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f23b001c070 con 0x5586ca0af000 2023-11-06T03:09:06.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 10 client.7098 fetch_fsmap finished waiting for FSMap version 54 2023-11-06T03:09:06.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 10 client.7098 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:09:06.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 20 client.7098 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:06.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca070d80 0x5586ca0bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 4 client.7098 mds_command: new command op to 7038 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:09:06.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 --> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586c9ee34d0 con 0x5586ca070d80 2023-11-06T03:09:06.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23bd937640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca070d80 0x5586ca0bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 10 client.7098 ms_handle_connect on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:06.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23bd937640 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca070d80 0x5586ca0bfc50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3caee3df8929fea6 server_cookie=dcd2fc73cc2fc888 in_seq=0 out_seq=0 2023-11-06T03:09:06.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 1 -- 172.21.15.157:0/483686196 <== mds.0 v2:172.21.15.157:6832/1223634033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5586c9ee34d0 con 0x5586ca070d80 2023-11-06T03:09:06.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 10 client.7098 handle_command_reply: tid=0 2023-11-06T03:09:06.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 client.7098 shutdown 2023-11-06T03:09:06.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 msgr2=0x5586ca070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f23b806c130 tx=0x7f23b0004a50 comp rx=0 tx=0).stop 2023-11-06T03:09:06.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 shutdown_connections 2023-11-06T03:09:06.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca070d80 0x5586ca0bfc50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca070510 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0ae6e0 0x5586ca06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/483686196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f2398ff9640 0 client.7098 ms_handle_reset on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 >> 172.21.15.157:0/483686196 conn(0x5586ca06abf0 msgr2=0x5586ca063530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 shutdown_connections 2023-11-06T03:09:06.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 1 -- 172.21.15.157:0/483686196 wait complete. 2023-11-06T03:09:06.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.620+0000 7f23c2cc3000 20 client.7098 trim_cache size 0 max 16384 2023-11-06T03:09:06.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 Processor -- start 2023-11-06T03:09:06.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 -- start start 2023-11-06T03:09:06.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca0c1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca0ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9b71940 con 0x5586ca0ae6e0 2023-11-06T03:09:06.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9ef7560 con 0x5586ca064a90 2023-11-06T03:09:06.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eb1460 con 0x5586ca0af000 2023-11-06T03:09:06.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39108/0 (socket says 172.21.15.157:39108) 2023-11-06T03:09:06.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 -- 172.21.15.157:0/840515400 learned_addr learned my addr 172.21.15.157:0/840515400 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 -- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 msgr2=0x5586ca0ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca0ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 -- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 msgr2=0x5586ca0c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca0c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 -- 172.21.15.157:0/840515400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c9e5f330 con 0x5586ca0ae6e0 2023-11-06T03:09:06.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f23bd136640 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f23a8050be0 tx=0x7f23a8054100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd87735db95e682 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f239bfff640 1 -- 172.21.15.157:0/840515400 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a807f070 con 0x5586ca0ae6e0 2023-11-06T03:09:06.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f239bfff640 1 -- 172.21.15.157:0/840515400 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8058510 con 0x5586ca0ae6e0 2023-11-06T03:09:06.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.636+0000 7f239bfff640 1 -- 172.21.15.157:0/840515400 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a805c750 con 0x5586ca0ae6e0 2023-11-06T03:09:06.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/840515400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 msgr2=0x5586ca0c18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f23a8050be0 tx=0x7f23a8054100 comp rx=0 tx=0).stop 2023-11-06T03:09:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/840515400 shutdown_connections 2023-11-06T03:09:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca0af000 0x5586ca0ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0ae6e0 0x5586ca0c18b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/840515400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca064a90 0x5586ca0c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/840515400 >> 172.21.15.157:0/840515400 conn(0x5586ca06abf0 msgr2=0x5586ca0bfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/840515400 shutdown_connections 2023-11-06T03:09:06.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/840515400 wait complete. 2023-11-06T03:09:06.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- start start 2023-11-06T03:09:06.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 Processor -- start 2023-11-06T03:09:06.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca064a90 0x5586ca0c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0f3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eaf9d0 con 0x5586ca0af000 2023-11-06T03:09:06.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eb8060 con 0x5586ca0ae6e0 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586c9eae400 con 0x5586ca064a90 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 10 client.0 _ll_register_callbacks cb 0x5586ca0309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39124/0 (socket says 172.21.15.157:39124) 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 -- 172.21.15.157:0/3970397950 learned_addr learned my addr 172.21.15.157:0/3970397950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23bd937640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca064a90 0x5586ca0c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 -- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca064a90 msgr2=0x5586ca0c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23bd136640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca064a90 0x5586ca0c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 -- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 msgr2=0x5586ca0f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 -- 172.21.15.157:0/3970397950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8155420 con 0x5586ca0af000 2023-11-06T03:09:06.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23be138640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009820 tx=0x7f23b0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36bcee4480c24e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:06.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000e070 con 0x5586ca0af000 2023-11-06T03:09:06.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f23b0037070 con 0x5586ca0af000 2023-11-06T03:09:06.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586c9e83b90 con 0x5586ca0af000 2023-11-06T03:09:06.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b0003eb0 con 0x5586ca0af000 2023-11-06T03:09:06.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 10 client.7104 fetch_fsmap learned FSMap version 54 2023-11-06T03:09:06.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5586c9bb0cc0 con 0x5586ca0af000 2023-11-06T03:09:06.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586c9e83b90 con 0x5586ca0af000 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 54) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f23b001c040 con 0x5586ca0af000 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 10 client.7104 fetch_fsmap finished waiting for FSMap version 54 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 10 client.7104 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 20 client.7104 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca0c1e20 0x5586ca0f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 4 client.7104 mds_command: new command op to 7038 tid=0[{"prefix": "status"}] 2023-11-06T03:09:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 --> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5586c9ead0e0 con 0x5586ca0c1e20 2023-11-06T03:09:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23bd937640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca0c1e20 0x5586ca0f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23bd937640 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca0c1e20 0x5586ca0f3e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74afbe2da9200196 server_cookie=f1d36bbc9742642d in_seq=0 out_seq=0 2023-11-06T03:09:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 10 client.7104 ms_handle_connect on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 1 -- 172.21.15.157:0/3970397950 <== mds.0 v2:172.21.15.157:6832/1223634033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5586c9ead0e0 con 0x5586ca0c1e20 2023-11-06T03:09:06.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 10 client.7104 handle_command_reply: tid=0 2023-11-06T03:09:06.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 client.7104 shutdown 2023-11-06T03:09:06.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 msgr2=0x5586ca0f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009820 tx=0x7f23b0004ac0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 shutdown_connections 2023-11-06T03:09:06.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x5586ca0c1e20 0x5586ca0f3e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5586ca0af000 0x5586ca0f3950 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5586ca0ae6e0 0x5586ca0f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 --2- 172.21.15.157:0/3970397950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5586ca064a90 0x5586ca0c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 >> 172.21.15.157:0/3970397950 conn(0x5586ca06abf0 msgr2=0x5586ca0bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.640+0000 7f2398ff9640 0 client.7104 ms_handle_reset on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 shutdown_connections 2023-11-06T03:09:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23c2cc3000 1 -- 172.21.15.157:0/3970397950 wait complete. 2023-11-06T03:09:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23c2cc3000 20 client.7104 trim_cache size 0 max 16384 2023-11-06T03:09:06.672 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:06.672 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:09:06.672 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:09:06.672 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7038, 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.3476179899999998, 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 54, 2023-11-06T03:09:06.673 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 73, 2023-11-06T03:09:06.674 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 73, 2023-11-06T03:09:06.674 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 4.3926722429999998 2023-11-06T03:09:06.674 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:06.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 msgr2=0x7f239c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 0x7f239c067f20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f23b40ab000 tx=0x7f23b40a7070 comp rx=0 tx=0).stop 2023-11-06T03:09:06.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 msgr2=0x7f23b8142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d4c0 tx=0x7f23a800d980 comp rx=0 tx=0).stop 2023-11-06T03:09:06.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 shutdown_connections 2023-11-06T03:09:06.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f239c065a60 0x7f239c067f20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b81426b0 0x7f23b80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23b8143c60 0x7f23b8142170 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 --2- 172.21.15.157:0/1525479122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23a4003980 0x7f23b8141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 >> 172.21.15.157:0/1525479122 conn(0x7f23b8065930 msgr2=0x7f23b8157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 shutdown_connections 2023-11-06T03:09:06.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.644+0000 7f23bfb9b640 1 -- 172.21.15.157:0/1525479122 wait complete. 2023-11-06T03:09:06.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:09:06.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.792+0000 7fcc36ef5640 1 Processor -- start 2023-11-06T03:09:06.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.792+0000 7fcc36ef5640 1 -- start start 2023-11-06T03:09:06.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc36ef5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc301599c0 0x7fcc300ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc36ef5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 0x7fcc300b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc36ef5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3015e440 con 0x7fcc3015dea0 2023-11-06T03:09:06.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc36ef5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3015adb0 con 0x7fcc301599c0 2023-11-06T03:09:06.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc36ef5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3006c240 con 0x7fcc3006d000 2023-11-06T03:09:06.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc301599c0 0x7fcc300ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc301599c0 0x7fcc300ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39130/0 (socket says 172.21.15.157:39130) 2023-11-06T03:09:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 -- 172.21.15.157:0/1556840642 learned_addr learned my addr 172.21.15.157:0/1556840642 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc35492640 1 --1- 172.21.15.157:0/1556840642 >> v1:172.21.15.157:6789/0 conn(0x7fcc3015dea0 0x7fcc30070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54880/0 (socket says 172.21.15.157:54880) 2023-11-06T03:09:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc2ffff640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 0x7fcc300b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 -- 172.21.15.157:0/1556840642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 msgr2=0x7fcc300b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 0x7fcc300b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 -- 172.21.15.157:0/1556840642 >> v1:172.21.15.157:6789/0 conn(0x7fcc3015dea0 legacy=0x7fcc30070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 -- 172.21.15.157:0/1556840642 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3006c520 con 0x7fcc301599c0 2023-11-06T03:09:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc2ffff640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 0x7fcc300b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc34c91640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300ae000 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45f57112d20483e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc2f7fe640 1 -- 172.21.15.157:0/1556840642 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240ab620 con 0x7fcc301599c0 2023-11-06T03:09:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc2f7fe640 1 -- 172.21.15.157:0/1556840642 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc240ab780 con 0x7fcc301599c0 2023-11-06T03:09:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.796+0000 7fcc2f7fe640 1 -- 172.21.15.157:0/1556840642 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240abb10 con 0x7fcc301599c0 2023-11-06T03:09:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- 172.21.15.157:0/1556840642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 msgr2=0x7fcc300ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300ae000 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2401c510 comp rx=0 tx=0).stop 2023-11-06T03:09:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- 172.21.15.157:0/1556840642 shutdown_connections 2023-11-06T03:09:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3006d000 0x7fcc300b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/1556840642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300ae000 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- 172.21.15.157:0/1556840642 >> 172.21.15.157:0/1556840642 conn(0x7fcc30065930 msgr2=0x7fcc30065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- 172.21.15.157:0/1556840642 shutdown_connections 2023-11-06T03:09:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- 172.21.15.157:0/1556840642 wait complete. 2023-11-06T03:09:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 Processor -- start 2023-11-06T03:09:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- start start 2023-11-06T03:09:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3006d000 0x7fcc3014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3015dea0 0x7fcc300ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30165cd0 con 0x7fcc301599c0 2023-11-06T03:09:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30165fb0 con 0x7fcc3006d000 2023-11-06T03:09:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc36ef5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30166290 con 0x7fcc3015dea0 2023-11-06T03:09:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39132/0 (socket says 172.21.15.157:39132) 2023-11-06T03:09:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc35492640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3015dea0 0x7fcc300ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc34c91640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3006d000 0x7fcc3014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 -- 172.21.15.157:0/700633451 learned_addr learned my addr 172.21.15.157:0/700633451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 -- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3015dea0 msgr2=0x7fcc300ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3015dea0 0x7fcc300ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 -- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3006d000 msgr2=0x7fcc3014e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3006d000 0x7fcc3014e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 -- 172.21.15.157:0/700633451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240094d0 con 0x7fcc301599c0 2023-11-06T03:09:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2ffff640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000e810 tx=0x7fcc2000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000c8d0 con 0x7fcc301599c0 2023-11-06T03:09:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.800+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000ca30 con 0x7fcc301599c0 2023-11-06T03:09:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc30166570 con 0x7fcc301599c0 2023-11-06T03:09:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000cde0 con 0x7fcc301599c0 2023-11-06T03:09:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3014ebe0 con 0x7fcc301599c0 2023-11-06T03:09:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc30072750 con 0x7fcc301599c0 2023-11-06T03:09:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcc20013410 con 0x7fcc301599c0 2023-11-06T03:09:06.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc2d7fa640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 0x7fcc1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:06.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fcc20083160 con 0x7fcc301599c0 2023-11-06T03:09:06.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc34c91640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 0x7fcc1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:06.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.804+0000 7fcc34c91640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 0x7fcc1c067f20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af000 tx=0x7fcc240b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:06.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:06.808+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcc20050930 con 0x7fcc301599c0 2023-11-06T03:09:07.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.000+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcc300cb110 con 0x7fcc301599c0 2023-11-06T03:09:07.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.000+0000 7fcc2d7fa640 1 -- 172.21.15.157:0/700633451 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcc20053f50 con 0x7fcc301599c0 2023-11-06T03:09:07.001 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:09:07.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 msgr2=0x7fcc1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:07.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 0x7fcc1c067f20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcc240af000 tx=0x7fcc240b6040 comp rx=0 tx=0).stop 2023-11-06T03:09:07.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 msgr2=0x7fcc300afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:07.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000e810 tx=0x7fcc2000ecd0 comp rx=0 tx=0).stop 2023-11-06T03:09:07.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 shutdown_connections 2023-11-06T03:09:07.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcc1c065a60 0x7fcc1c067f20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:07.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcc3015dea0 0x7fcc300ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:07.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc301599c0 0x7fcc300afc80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:07.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 --2- 172.21.15.157:0/700633451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3006d000 0x7fcc3014e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:07.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 >> 172.21.15.157:0/700633451 conn(0x7fcc30065930 msgr2=0x7fcc30065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:07.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 shutdown_connections 2023-11-06T03:09:07.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:07.004+0000 7fcc36ef5640 1 -- 172.21.15.157:0/700633451 wait complete. 2023-11-06T03:09:07.020 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:09:07.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:09:07.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:09:07.021 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:09:07.022 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:07.022 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:09:07.022 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:09:07.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:09:07.023 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:07.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.024 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:09:07.030 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:09:07.030 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:09:07.030 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.031 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:09:07.032 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:07.032 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.032 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:09:07.032 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:07.032 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.033 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:09:07.033 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.033 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:09:07.033 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42192sec preferred_lft 42192sec 2023-11-06T03:09:07.033 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:09:07.034 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.034 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:09:07.034 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.034 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:09:07.034 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.035 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:09:07.035 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:07.035 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:09:07.035 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.036 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:09:07.036 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.036 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:09:07.036 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:09:07.036 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:09:07.037 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.037 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:09:07.037 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:09:07.037 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:09:07.037 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:07.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.038 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:09:07.082 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:09:07.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.086 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:09:07.134 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:09:07.134 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:09:07.134 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:09:07.135 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:09:07.135 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:09:07.135 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:09:07.135 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:07.135 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:07.136 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:07.136 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:09:07.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.137 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:09:07.137 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:09:07.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:09:07.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:09:07.139 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:09:07.139 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:09:07.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.209 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:09:07.255 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:09:07.255 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:09:07.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:07.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.258 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:09:07.314 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:09:07.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:09:07.315 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:07.363 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:07.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:09:07.364 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:07.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:09:07.468 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:09:07.469 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:09:07.522 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:09:07.523 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:09:07.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.524 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:09:07.570 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:09:07.571 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:09:07.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:09:07.632 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:09:07.633 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:09:07.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:09:07.692+0000 7fe5d6b4e580 -1 init, newargv = 0x560d78600bc0 newargc=15 2023-11-06T03:09:07.693 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:09:07.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65952]: starting ceph client 2023-11-06T03:09:07.694 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:09:07.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.695 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:09:07.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65952]: starting fuse 2023-11-06T03:09:07.702 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:09:07.703 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:09:07.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:09:07.704 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:09:07.704 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:09:07.704 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:09:07.705 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:09:07.705 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:07.706 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:09:07.706 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:09:07.706 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:09:07.707 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:09:07.707 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:07.707 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:09:07.708 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:09:07.708 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:09:07.708 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:09:07.709 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:07.709 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:09:07.709 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:09:07.710 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:09:07.710 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:09:07.711 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:09:07.711 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:09:07.712 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:09:07.712 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:09:07.712 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:07.713 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:09:07.713 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:09:07.805 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.65952.asok 2023-11-06T03:09:07.993 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65952.asok 2023-11-06T03:09:07.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:07.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65952.asok status 2023-11-06T03:09:08.153 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:08.153 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:09:08.154 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:09:08.154 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:09:08.154 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:09:08.154 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:09:08.154 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:09:08.155 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "65952", 2023-11-06T03:09:08.155 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:09:08.155 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:08.155 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:09:08.155 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:09:08.156 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7122, 2023-11-06T03:09:08.156 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:09:08.156 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:09:08.156 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:09:08.156 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7122 2023-11-06T03:09:08.157 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:08.157 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:09:08.157 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:09:08.157 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:09:08.157 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1075907869 2023-11-06T03:09:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:09:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:09:08.158 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1075907869 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7122 192.168.0.1:0/1075907869", 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1075907869", 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:09:08.159 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 54, 2023-11-06T03:09:08.160 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 73, 2023-11-06T03:09:08.160 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:09:08.160 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:09:08.160 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:09:08.160 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:08.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:08.169 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:09:08.169 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:08.176 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:09:08.176 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:08.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:08.177 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:08.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:08.228 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2023-11-06T03:09:08.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:09:08.278 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2023-11-06T03:09:08.281 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2023-11-06T03:09:08.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2023-11-06T03:09:08.302 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:09:08.300+0000 7ff34d987640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:09:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 Processor -- start 2023-11-06T03:09:08.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 -- start start 2023-11-06T03:09:08.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f806006c170 0x7f8060154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f806015b860 0x7f8060155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80600cb100 con 0x7f806006c550 2023-11-06T03:09:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806015bcf0 con 0x7f806015b860 2023-11-06T03:09:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8067066640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80600ce0a0 con 0x7f806006c170 2023-11-06T03:09:08.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f806015b860 0x7f8060155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f806015b860 0x7f8060155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39140/0 (socket says 172.21.15.157:39140) 2023-11-06T03:09:08.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 -- 172.21.15.157:0/1958981631 learned_addr learned my addr 172.21.15.157:0/1958981631 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8064e02640 1 --2- 172.21.15.157:0/1958981631 >> v2:172.21.15.195:3300/0 conn(0x7f806006c170 0x7f8060154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 -- 172.21.15.157:0/1958981631 >> v2:172.21.15.195:3300/0 conn(0x7f806006c170 msgr2=0x7f8060154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 --2- 172.21.15.157:0/1958981631 >> v2:172.21.15.195:3300/0 conn(0x7f806006c170 0x7f8060154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 -- 172.21.15.157:0/1958981631 >> v1:172.21.15.195:6789/0 conn(0x7f806006c550 legacy=0x7f806015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.432+0000 7f8065603640 1 -- 172.21.15.157:0/1958981631 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806015bef0 con 0x7f806015b860 2023-11-06T03:09:08.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8065603640 1 --2- 172.21.15.157:0/1958981631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060155140 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8060066460 tx=0x7f805801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174cf57c1dddc365 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f805f7fe640 1 -- 172.21.15.157:0/1958981631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580a5820 con 0x7f806015b860 2023-11-06T03:09:08.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f805f7fe640 1 -- 172.21.15.157:0/1958981631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580a5980 con 0x7f806015b860 2023-11-06T03:09:08.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f805f7fe640 1 -- 172.21.15.157:0/1958981631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580a5d10 con 0x7f806015b860 2023-11-06T03:09:08.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- 172.21.15.157:0/1958981631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 msgr2=0x7f8060155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 --2- 172.21.15.157:0/1958981631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060155140 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8060066460 tx=0x7f805801c960 comp rx=0 tx=0).stop 2023-11-06T03:09:08.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- 172.21.15.157:0/1958981631 shutdown_connections 2023-11-06T03:09:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 --2- 172.21.15.157:0/1958981631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060155140 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 --2- 172.21.15.157:0/1958981631 >> v2:172.21.15.195:3300/0 conn(0x7f806006c170 0x7f8060154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- 172.21.15.157:0/1958981631 >> 172.21.15.157:0/1958981631 conn(0x7f80600b43b0 msgr2=0x7f8060065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- 172.21.15.157:0/1958981631 shutdown_connections 2023-11-06T03:09:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- 172.21.15.157:0/1958981631 wait complete. 2023-11-06T03:09:08.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 Processor -- start 2023-11-06T03:09:08.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.436+0000 7f8067066640 1 -- start start 2023-11-06T03:09:08.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 0x7f806013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f806006c550 0x7f8060155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8060156e50 con 0x7f806015b860 2023-11-06T03:09:08.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8060153e00 con 0x7f806006c550 2023-11-06T03:09:08.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806013f790 con 0x7f806006c170 2023-11-06T03:09:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39146/0 (socket says 172.21.15.157:39146) 2023-11-06T03:09:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 -- 172.21.15.157:0/427544078 learned_addr learned my addr 172.21.15.157:0/427544078 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f805ffff640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f806006c550 0x7f8060155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8064e02640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 0x7f806013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 -- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 msgr2=0x7f806013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 0x7f806013f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 -- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f806006c550 msgr2=0x7f8060155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f806006c550 0x7f8060155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 -- 172.21.15.157:0/427544078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8058009b30 con 0x7f806015b860 2023-11-06T03:09:08.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8064e02640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 0x7f806013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:08.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8065603640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f80580af040 tx=0x7f8058019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f805d7fa640 1 -- 172.21.15.157:0/427544078 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580b8670 con 0x7f806015b860 2023-11-06T03:09:08.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f805d7fa640 1 -- 172.21.15.157:0/427544078 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580b87d0 con 0x7f806015b860 2023-11-06T03:09:08.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80600cbfb0 con 0x7f806015b860 2023-11-06T03:09:08.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f805d7fa640 1 -- 172.21.15.157:0/427544078 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580b8b80 con 0x7f806015b860 2023-11-06T03:09:08.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.440+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8060156890 con 0x7f806015b860 2023-11-06T03:09:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f805d7fa640 1 -- 172.21.15.157:0/427544078 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80580b8ce0 con 0x7f806015b860 2023-11-06T03:09:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f805d7fa640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 0x7f804c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f805d7fa640 1 -- 172.21.15.157:0/427544078 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f805811e990 con 0x7f806015b860 2023-11-06T03:09:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8064e02640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 0x7f804c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 Processor -- start 2023-11-06T03:09:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 -- start start 2023-11-06T03:09:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8064e02640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 0x7f804c067f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8050007a70 tx=0x7f8050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec75a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec7106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec556f20 con 0x557cec710c00 2023-11-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec5665b0 con 0x557cec759c40 2023-11-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cec563b80 con 0x557cec75a560 2023-11-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39156/0 (socket says 172.21.15.157:39156) 2023-11-06T03:09:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 -- 172.21.15.157:0/1066428080 learned_addr learned my addr 172.21.15.157:0/1066428080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8064e02640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec75a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f805ffff640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec7106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 -- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 msgr2=0x557cec7106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec7106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 -- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 msgr2=0x557cec75a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec75a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f8065603640 1 -- 172.21.15.157:0/1066428080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cec252610 con 0x557cec710c00 2023-11-06T03:09:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.444+0000 7f805ffff640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec7106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:08.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f8065603640 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f80580efe90 tx=0x7f80580efb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9640df30a4f55c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f80467fc640 1 -- 172.21.15.157:0/1066428080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580f3d30 con 0x557cec710c00 2023-11-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f80467fc640 1 -- 172.21.15.157:0/1066428080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580f7ee0 con 0x557cec710c00 2023-11-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f80467fc640 1 -- 172.21.15.157:0/1066428080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580ec610 con 0x557cec710c00 2023-11-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/1066428080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 msgr2=0x557cec710fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f80580efe90 tx=0x7f80580efb40 comp rx=0 tx=0).stop 2023-11-06T03:09:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/1066428080 shutdown_connections 2023-11-06T03:09:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec710c00 0x557cec710fe0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec7106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- 172.21.15.157:0/1066428080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec75a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/1066428080 >> 172.21.15.157:0/1066428080 conn(0x557cec70cbb0 msgr2=0x557cec70efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/1066428080 shutdown_connections 2023-11-06T03:09:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/1066428080 wait complete. 2023-11-06T03:09:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- start start 2023-11-06T03:09:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 Processor -- start 2023-11-06T03:09:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec710c00 0x557cec716840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec71bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec55f470 con 0x557cec759c40 2023-11-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec51c890 con 0x557cec710c00 2023-11-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cec51c530 con 0x557cec75a560 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 10 client.0 _ll_register_callbacks cb 0x557cec6fc320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39158/0 (socket says 172.21.15.157:39158) 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 -- 172.21.15.157:0/688375373 learned_addr learned my addr 172.21.15.157:0/688375373 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f8064e02640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec710c00 0x557cec716840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f8065603640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec71bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 -- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 msgr2=0x557cec71bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec71bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 -- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec710c00 msgr2=0x557cec716840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec710c00 0x557cec716840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 -- 172.21.15.157:0/688375373 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8060156890 con 0x557cec759c40 2023-11-06T03:09:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f805ffff640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8054005510 tx=0x7f805400f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80855fd6669a853b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805400d070 con 0x557cec759c40 2023-11-06T03:09:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f80540067e0 con 0x557cec759c40 2023-11-06T03:09:08.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cec55ddc0 con 0x557cec759c40 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80540069a0 con 0x557cec759c40 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 10 client.7134 fetch_fsmap learned FSMap version 54 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x557cec524390 con 0x557cec759c40 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cec55ddc0 con 0x557cec759c40 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 54) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f805401b060 con 0x557cec759c40 2023-11-06T03:09:08.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 10 client.7134 fetch_fsmap finished waiting for FSMap version 54 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 10 client.7134 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 20 client.7134 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec760e70 0x557cec761280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 4 client.7134 mds_command: new command op to 7038 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 --> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cec526f60 con 0x557cec760e70 2023-11-06T03:09:08.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.448+0000 7f8064e02640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec760e70 0x557cec761280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f803f7fe640 10 client.7134 ms_handle_connect on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f8064e02640 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec760e70 0x557cec761280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cc2697a4c3fc7c server_cookie=dba48f6332f4cf0d in_seq=0 out_seq=0 2023-11-06T03:09:08.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f803f7fe640 1 -- 172.21.15.157:0/688375373 <== mds.0 v2:172.21.15.157:6832/1223634033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557cec526f60 con 0x557cec760e70 2023-11-06T03:09:08.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f803f7fe640 10 client.7134 handle_command_reply: tid=0 2023-11-06T03:09:08.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 client.7134 shutdown 2023-11-06T03:09:08.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 msgr2=0x557cec71b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8054005510 tx=0x7f805400f540 comp rx=0 tx=0).stop 2023-11-06T03:09:08.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 shutdown_connections 2023-11-06T03:09:08.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec760e70 0x557cec761280 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec75a560 0x557cec71bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec759c40 0x557cec71b7f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 --2- 172.21.15.157:0/688375373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec710c00 0x557cec716840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f803f7fe640 0 client.7134 ms_handle_reset on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 >> 172.21.15.157:0/688375373 conn(0x557cec70cbb0 msgr2=0x557cec70f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 shutdown_connections 2023-11-06T03:09:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 1 -- 172.21.15.157:0/688375373 wait complete. 2023-11-06T03:09:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.452+0000 7f806a18e000 20 client.7134 trim_cache size 0 max 16384 2023-11-06T03:09:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.464+0000 7f806a18e000 1 Processor -- start 2023-11-06T03:09:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.464+0000 7f806a18e000 1 -- start start 2023-11-06T03:09:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec70efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec70f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec21cda0 con 0x557cec75a560 2023-11-06T03:09:08.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec567840 con 0x557cec759c40 2023-11-06T03:09:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cec5b88d0 con 0x557cec710c00 2023-11-06T03:09:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39162/0 (socket says 172.21.15.157:39162) 2023-11-06T03:09:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/4285476566 learned_addr learned my addr 172.21.15.157:0/4285476566 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8064e02640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec70efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f805ffff640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec70f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 msgr2=0x557cec70efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec70efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 msgr2=0x557cec70f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec70f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/4285476566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cec50a6e0 con 0x557cec75a560 2023-11-06T03:09:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8064e02640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec70efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f805801a590 tx=0x7f80580f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1efba395b9aa8f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8046ffd640 1 -- 172.21.15.157:0/4285476566 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580f9030 con 0x557cec75a560 2023-11-06T03:09:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8046ffd640 1 -- 172.21.15.157:0/4285476566 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580efa40 con 0x557cec75a560 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8046ffd640 1 -- 172.21.15.157:0/4285476566 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580f7ee0 con 0x557cec75a560 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/4285476566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 msgr2=0x557cec75a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f805801a590 tx=0x7f80580f2000 comp rx=0 tx=0).stop 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/4285476566 shutdown_connections 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec75a940 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec70f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- 172.21.15.157:0/4285476566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec70efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/4285476566 >> 172.21.15.157:0/4285476566 conn(0x557cec70cbb0 msgr2=0x557cec7158a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/4285476566 shutdown_connections 2023-11-06T03:09:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/4285476566 wait complete. 2023-11-06T03:09:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- start start 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 Processor -- start 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec7bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec7bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec713bf0 con 0x557cec75a560 2023-11-06T03:09:08.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cec4fc230 con 0x557cec759c40 2023-11-06T03:09:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557cec52a5e0 con 0x557cec710c00 2023-11-06T03:09:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 10 client.0 _ll_register_callbacks cb 0x557cec5202b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39178/0 (socket says 172.21.15.157:39178) 2023-11-06T03:09:08.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/262975097 learned_addr learned my addr 172.21.15.157:0/262975097 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f805ffff640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec7bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8064e02640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec7bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 msgr2=0x557cec7bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec7bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 msgr2=0x557cec7bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:08.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec7bfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 -- 172.21.15.157:0/262975097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cec252610 con 0x557cec75a560 2023-11-06T03:09:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8064e02640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec7bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f8065603640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f80580efbb0 tx=0x7f8058009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c951689cda13b1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f803f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:08.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058103070 con 0x557cec75a560 2023-11-06T03:09:08.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f80580fa070 con 0x557cec75a560 2023-11-06T03:09:08.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.468+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cec521b40 con 0x557cec75a560 2023-11-06T03:09:08.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80580f9040 con 0x557cec75a560 2023-11-06T03:09:08.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 10 client.7140 fetch_fsmap learned FSMap version 54 2023-11-06T03:09:08.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x557cec25c120 con 0x557cec75a560 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557cec521b40 con 0x557cec75a560 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 54) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8058005d20 con 0x557cec75a560 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 10 client.7140 fetch_fsmap finished waiting for FSMap version 54 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 10 client.7140 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 20 client.7140 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:08.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec7a3c40 0x557cec7a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 4 client.7140 mds_command: new command op to 7038 tid=0[{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2023-11-06T03:09:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 --> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x557cec5584f0 con 0x557cec7a3c40 2023-11-06T03:09:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8064e02640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec7a3c40 0x557cec7a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 10 client.7140 ms_handle_connect on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8064e02640 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec7a3c40 0x557cec7a4020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0739f2babe20742 server_cookie=5881df000aa8d762 in_seq=0 out_seq=0 2023-11-06T03:09:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 1 -- 172.21.15.157:0/262975097 <== mds.0 v2:172.21.15.157:6832/1223634033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557cec5584f0 con 0x557cec7a3c40 2023-11-06T03:09:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 10 client.7140 handle_command_reply: tid=0 2023-11-06T03:09:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 client.7140 shutdown 2023-11-06T03:09:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 msgr2=0x557cec7a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f80580efbb0 tx=0x7f8058009810 comp rx=0 tx=0).stop 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 shutdown_connections 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:6832/1223634033,v1:172.21.15.157:6833/1223634033] conn(0x557cec7a3c40 0x557cec7a4020 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cec75a560 0x557cec7a3700 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557cec759c40 0x557cec7bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 --2- 172.21.15.157:0/262975097 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557cec710c00 0x557cec7bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f803f7fe640 0 client.7140 ms_handle_reset on v2:172.21.15.157:6832/1223634033 2023-11-06T03:09:08.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 >> 172.21.15.157:0/262975097 conn(0x557cec70cbb0 msgr2=0x557cec7587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 shutdown_connections 2023-11-06T03:09:08.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 1 -- 172.21.15.157:0/262975097 wait complete. 2023-11-06T03:09:08.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f806a18e000 20 client.7140 trim_cache size 0 max 16384 2023-11-06T03:09:08.492 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:08.492 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:09:08.492 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "ed2f676d-558f-4934-80b7-b0f5fa1518c3", 2023-11-06T03:09:08.492 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:09:08.492 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:08.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 msgr2=0x7f804c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 0x7f804c067f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8050007a70 tx=0x7f8050008040 comp rx=0 tx=0).stop 2023-11-06T03:09:08.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 msgr2=0x7f8060158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:08.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f80580af040 tx=0x7f8058019700 comp rx=0 tx=0).stop 2023-11-06T03:09:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 shutdown_connections 2023-11-06T03:09:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a60 0x7f804c067f20 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8050007a70 tx=0x7f8050008040 comp rx=0 tx=0).stop 2023-11-06T03:09:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f806015b860 0x7f8060158260 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f806006c550 0x7f8060155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 --2- 172.21.15.157:0/427544078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f806006c170 0x7f806013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:08.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 >> 172.21.15.157:0/427544078 conn(0x7f80600b43b0 msgr2=0x7f806015b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:08.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 shutdown_connections 2023-11-06T03:09:08.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:08.472+0000 7f8067066640 1 -- 172.21.15.157:0/427544078 wait complete. 2023-11-06T03:09:08.496 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': 'ed2f676d-558f-4934-80b7-b0f5fa1518c3', 'mode': 'asynchronous'}' 2023-11-06T03:09:10.533 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:09:10.536 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:09:18.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:18.498 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2023-11-06T03:09:18.512 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:09:18.512 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:09:18.591 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:09:18.643 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:09:18.644 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:09:18.644 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:18.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:18.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 Processor -- start 2023-11-06T03:09:18.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 -- start start 2023-11-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc9e015cbc0 0x7fc9e00cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e0160980 con 0x7fc9e01613d0 2023-11-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e00cf0e0 con 0x7fc9e015cfa0 2023-11-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e6f90640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e00cf3c0 con 0x7fc9e015cbc0 2023-11-06T03:09:18.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc9e01613d0 0x7fc9e0065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60178/0 (socket says 172.21.15.157:60178) 2023-11-06T03:09:18.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 -- 172.21.15.157:0/3575545668 learned_addr learned my addr 172.21.15.157:0/3575545668 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:18.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e4d2c640 1 --2- 172.21.15.157:0/3575545668 >> v2:172.21.15.195:3300/0 conn(0x7fc9e015cbc0 0x7fc9e00cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455750540 0 0) 0x7fc9e0160980 con 0x7fc9e01613d0 2023-11-06T03:09:18.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d0003530 con 0x7fc9e01613d0 2023-11-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859059294 0 0) 0x7fc9d0003530 con 0x7fc9e01613d0 2023-11-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 >> v2:172.21.15.195:3300/0 conn(0x7fc9e015cbc0 msgr2=0x7fc9e00cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 --2- 172.21.15.157:0/3575545668 >> v2:172.21.15.195:3300/0 conn(0x7fc9e015cbc0 0x7fc9e00cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 >> v1:172.21.15.195:6789/0 conn(0x7fc9e015cfa0 legacy=0x7fc9e0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0160bd0 con 0x7fc9e01613d0 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc9d8001a00 con 0x7fc9e01613d0 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 >> v1:172.21.15.157:6789/0 conn(0x7fc9e01613d0 legacy=0x7fc9e0065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0003980 0x7fc9d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d00047d0 0x7fc9d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 0x7fc9d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00cf3c0 con 0x7fc9d0005190 2023-11-06T03:09:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d000c130 con 0x7fc9d0003980 2023-11-06T03:09:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d000c2f0 con 0x7fc9d00047d0 2023-11-06T03:09:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 0x7fc9d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e4d2c640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0003980 0x7fc9d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9dffff640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d00047d0 0x7fc9d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 -- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d00047d0 msgr2=0x7fc9d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:18.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d00047d0 0x7fc9d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 -- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0003980 msgr2=0x7fc9d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:18.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0003980 0x7fc9d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9e552d640 1 -- 172.21.15.157:0/3575545668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0003530 con 0x7fc9d0005190 2023-11-06T03:09:18.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.828+0000 7fc9dffff640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d00047d0 0x7fc9d0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:18.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e552d640 1 -- 172.21.15.157:0/3575545668 reap_dead start 2023-11-06T03:09:18.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e552d640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 0x7fc9d000b990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800a830 tx=0x7fc9d801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721505b53c01443b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:18.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80aa6d0 con 0x7fc9d0005190 2023-11-06T03:09:18.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80aa870 con 0x7fc9d0005190 2023-11-06T03:09:18.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9df7fe640 1 -- 172.21.15.157:0/3575545668 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80aac20 con 0x7fc9d0005190 2023-11-06T03:09:18.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 msgr2=0x7fc9d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:18.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 0x7fc9d000b990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800a830 tx=0x7fc9d801faa0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- 172.21.15.157:0/3575545668 shutdown_connections 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/3575545668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9d0005190 0x7fc9d000b990 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- 172.21.15.157:0/3575545668 >> 172.21.15.157:0/3575545668 conn(0x7fc9e00cc2a0 msgr2=0x7fc9e00cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- 172.21.15.157:0/3575545668 shutdown_connections 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- 172.21.15.157:0/3575545668 wait complete. 2023-11-06T03:09:18.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 Processor -- start 2023-11-06T03:09:18.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.832+0000 7fc9e6f90640 1 -- start start 2023-11-06T03:09:18.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0005190 0x7fc9e00b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 0x7fc9e015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e015b4e0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0147040 con 0x7fc9d0005190 2023-11-06T03:09:18.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01f68b0 con 0x7fc9e015ab20 2023-11-06T03:09:18.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44428/0 (socket says 172.21.15.157:44428) 2023-11-06T03:09:18.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 -- 172.21.15.157:0/339550408 learned_addr learned my addr 172.21.15.157:0/339550408 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:18.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e4d2c640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0005190 0x7fc9e00b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dffff640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 0x7fc9e015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 -- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 msgr2=0x7fc9e015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:18.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 0x7fc9e015afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 -- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0005190 msgr2=0x7fc9e00b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0005190 0x7fc9e00b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 -- 172.21.15.157:0/339550408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0146710 con 0x7fc9e01f5b80 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dffff640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 0x7fc9e015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e552d640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8007040 tx=0x7fc9d0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80aa6d0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8001800 con 0x7fc9e01f5b80 2023-11-06T03:09:18.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e01469f0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8004dd0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e01f6540 con 0x7fc9e01f5b80 2023-11-06T03:09:18.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e006ad00 con 0x7fc9e01f5b80 2023-11-06T03:09:18.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc9d80070d0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 0x7fc9c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:18.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fc9d8120ab0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.836+0000 7fc9e4d2c640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 0x7fc9c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:18.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.840+0000 7fc9e4d2c640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 0x7fc9c4067f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009910 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:18.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.840+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc9d80ee1d0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:18.996+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e0072750 con 0x7fc9e01f5b80 2023-11-06T03:09:18.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9dd7fa640 1 -- 172.21.15.157:0/339550408 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18375 (secure 0 0 0) 0x7fc9d80f17f0 con 0x7fc9e01f5b80 2023-11-06T03:09:18.998 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:18.999 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:02.175228+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.195:6836/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.157:6835/1218323165":"2023-11-07T03:09:02.146924+0000","172.21.15.157:6834/1218323165":"2023-11-07T03:09:02.146924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:19.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 msgr2=0x7fc9c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 0x7fc9c4067f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009910 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2023-11-06T03:09:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 msgr2=0x7fc9e01f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8007040 tx=0x7fc9d0005800 comp rx=0 tx=0).stop 2023-11-06T03:09:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 shutdown_connections 2023-11-06T03:09:19.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9c4065a60 0x7fc9c4067f20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9e01f5b80 0x7fc9e01f6000 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9e015ab20 0x7fc9e015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 --2- 172.21.15.157:0/339550408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0005190 0x7fc9e00b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 >> 172.21.15.157:0/339550408 conn(0x7fc9e00cc2a0 msgr2=0x7fc9e0070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 shutdown_connections 2023-11-06T03:09:19.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.000+0000 7fc9e6f90640 1 -- 172.21.15.157:0/339550408 wait complete. 2023-11-06T03:09:19.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:08:41.869490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:08:42.287789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:09:19.019 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:09:19.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 Processor -- start 2023-11-06T03:09:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 -- start start 2023-11-06T03:09:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76006f470 con 0x7fb7600b1910 2023-11-06T03:09:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7600b1df0 con 0x7fb760160db0 2023-11-06T03:09:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb767712640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76006f6c0 con 0x7fb7600701a0 2023-11-06T03:09:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb7654ae640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49378/0 (socket says 172.21.15.157:49378) 2023-11-06T03:09:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 -- 172.21.15.157:0/3791437986 learned_addr learned my addr 172.21.15.157:0/3791437986 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb7654ae640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:49418/0 (socket says 172.21.15.157:49418) 2023-11-06T03:09:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 -- 172.21.15.157:0/3791437986 >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 msgr2=0x7fb760161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 --2- 172.21.15.157:0/3791437986 >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 -- 172.21.15.157:0/3791437986 >> v1:172.21.15.195:6789/0 conn(0x7fb7600b1910 legacy=0x7fb76006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb765caf640 1 -- 172.21.15.157:0/3791437986 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76006f9a0 con 0x7fb7600701a0 2023-11-06T03:09:19.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.140+0000 7fb7654ae640 1 --2- 172.21.15.157:0/3791437986 >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb765caf640 1 --2- 172.21.15.157:0/3791437986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098a0 tx=0x7fb75801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5684bace6df70a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb757fff640 1 -- 172.21.15.157:0/3791437986 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7580ac600 con 0x7fb7600701a0 2023-11-06T03:09:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb757fff640 1 -- 172.21.15.157:0/3791437986 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7580a8050 con 0x7fb7600701a0 2023-11-06T03:09:19.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb757fff640 1 -- 172.21.15.157:0/3791437986 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7580ac980 con 0x7fb7600701a0 2023-11-06T03:09:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- 172.21.15.157:0/3791437986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 msgr2=0x7fb7600ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 --2- 172.21.15.157:0/3791437986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098a0 tx=0x7fb75801c490 comp rx=0 tx=0).stop 2023-11-06T03:09:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- 172.21.15.157:0/3791437986 shutdown_connections 2023-11-06T03:09:19.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 --2- 172.21.15.157:0/3791437986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7600701a0 0x7fb7600ce660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 --2- 172.21.15.157:0/3791437986 >> v2:172.21.15.195:3300/0 conn(0x7fb760160db0 0x7fb760161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- 172.21.15.157:0/3791437986 >> 172.21.15.157:0/3791437986 conn(0x7fb760065930 msgr2=0x7fb760065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- 172.21.15.157:0/3791437986 shutdown_connections 2023-11-06T03:09:19.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- 172.21.15.157:0/3791437986 wait complete. 2023-11-06T03:09:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 Processor -- start 2023-11-06T03:09:19.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.144+0000 7fb767712640 1 -- start start 2023-11-06T03:09:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7600701a0 0x7fb760151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb760160db0 0x7fb7601561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601fdb60 con 0x7fb7600b1910 2023-11-06T03:09:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601594e0 con 0x7fb7600701a0 2023-11-06T03:09:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7601fcfe0 con 0x7fb760160db0 2023-11-06T03:09:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44430/0 (socket says 172.21.15.157:44430) 2023-11-06T03:09:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 -- 172.21.15.157:0/1417206170 learned_addr learned my addr 172.21.15.157:0/1417206170 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb765caf640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb760160db0 0x7fb7601561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb7654ae640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7600701a0 0x7fb760151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 -- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb760160db0 msgr2=0x7fb7601561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb760160db0 0x7fb7601561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 -- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7600701a0 msgr2=0x7fb760151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7600701a0 0x7fb760151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 -- 172.21.15.157:0/1417206170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c009a70 con 0x7fb7600b1910 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb764cad640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c400 tx=0x7fb75000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7500052d0 con 0x7fb7600b1910 2023-11-06T03:09:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb750005430 con 0x7fb7600b1910 2023-11-06T03:09:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750007260 con 0x7fb7600b1910 2023-11-06T03:09:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7580094d0 con 0x7fb7600b1910 2023-11-06T03:09:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7601fd5a0 con 0x7fb7600b1910 2023-11-06T03:09:19.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb760072750 con 0x7fb7600b1910 2023-11-06T03:09:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7500073c0 con 0x7fb7600b1910 2023-11-06T03:09:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.148+0000 7fb755ffb640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 0x7fb744067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.152+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fb7500828b0 con 0x7fb7600b1910 2023-11-06T03:09:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.152+0000 7fb7654ae640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 0x7fb744067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.152+0000 7fb7654ae640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 0x7fb744067f20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb75c007560 tx=0x7fb75c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.156+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb75004ff30 con 0x7fb7600b1910 2023-11-06T03:09:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.308+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb760071020 con 0x7fb7600b1910 2023-11-06T03:09:19.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.308+0000 7fb755ffb640 1 -- 172.21.15.157:0/1417206170 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb750053550 con 0x7fb7600b1910 2023-11-06T03:09:19.309 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:09:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 msgr2=0x7fb744067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 0x7fb744067f20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb75c007560 tx=0x7fb75c00b040 comp rx=0 tx=0).stop 2023-11-06T03:09:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 msgr2=0x7fb760158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb75000c400 tx=0x7fb75000c8c0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 shutdown_connections 2023-11-06T03:09:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb744065a60 0x7fb744067f20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb760160db0 0x7fb7601561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7600b1910 0x7fb760158fa0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 --2- 172.21.15.157:0/1417206170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7600701a0 0x7fb760151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 >> 172.21.15.157:0/1417206170 conn(0x7fb760065930 msgr2=0x7fb760163510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 shutdown_connections 2023-11-06T03:09:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.312+0000 7fb767712640 1 -- 172.21.15.157:0/1417206170 wait complete. 2023-11-06T03:09:19.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:09:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 Processor -- start 2023-11-06T03:09:19.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 -- start start 2023-11-06T03:09:19.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffbdc157070 0x7ffbdc065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbdc160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc0cf820 con 0x7ffbdc065e70 2023-11-06T03:09:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc06d8f0 con 0x7ffbdc157070 2023-11-06T03:09:19.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe3b60640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc066320 con 0x7ffbdc06d2d0 2023-11-06T03:09:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffbdc065e70 0x7ffbdc06cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60190/0 (socket says 172.21.15.157:60190) 2023-11-06T03:09:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 -- 172.21.15.157:0/3426209313 learned_addr learned my addr 172.21.15.157:0/3426209313 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe20fd640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbdc160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/3426209313 >> v2:172.21.15.195:3300/0 conn(0x7ffbdc157070 0x7ffbdc065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074460449 0 0) 0x7ffbdc0cf820 con 0x7ffbdc065e70 2023-11-06T03:09:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc8003530 con 0x7ffbdc065e70 2023-11-06T03:09:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125232912 0 0) 0x7ffbc8003530 con 0x7ffbdc065e70 2023-11-06T03:09:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 msgr2=0x7ffbdc160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbdc160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 >> v2:172.21.15.195:3300/0 conn(0x7ffbdc157070 msgr2=0x7ffbdc065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 --2- 172.21.15.157:0/3426209313 >> v2:172.21.15.195:3300/0 conn(0x7ffbdc157070 0x7ffbdc065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc15e150 con 0x7ffbdc065e70 2023-11-06T03:09:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ffbcc0016a0 con 0x7ffbdc065e70 2023-11-06T03:09:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/3426209313 >> v2:172.21.15.195:3300/0 conn(0x7ffbdc157070 0x7ffbdc065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 >> v1:172.21.15.157:6789/0 conn(0x7ffbdc065e70 legacy=0x7ffbdc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80039b0 0x7ffbc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc80047d0 0x7ffbc8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbc8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc06d8f0 con 0x7ffbc80047d0 2023-11-06T03:09:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc066320 con 0x7ffbc80039b0 2023-11-06T03:09:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc800ba70 con 0x7ffbdc06d2d0 2023-11-06T03:09:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc80047d0 0x7ffbc8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80039b0 0x7ffbc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe20fd640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbc8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 -- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 msgr2=0x7ffbc8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc06d2d0 0x7ffbc8005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 -- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80039b0 msgr2=0x7ffbc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80039b0 0x7ffbc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe10fb640 1 -- 172.21.15.157:0/3426209313 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8003530 con 0x7ffbc80047d0 2023-11-06T03:09:19.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.444+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80039b0 0x7ffbc8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc80047d0 0x7ffbc8004c50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc001ec0 tx=0x7ffbcc01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ceefccc68898ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc005ae0 con 0x7ffbc80047d0 2023-11-06T03:09:19.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc005c40 con 0x7ffbc80047d0 2023-11-06T03:09:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe08fa640 1 -- 172.21.15.157:0/3426209313 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc007070 con 0x7ffbc80047d0 2023-11-06T03:09:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/3426209313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc80047d0 msgr2=0x7ffbc8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/3426209313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc80047d0 0x7ffbc8004c50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc001ec0 tx=0x7ffbcc01cb80 comp rx=0 tx=0).stop 2023-11-06T03:09:19.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe20fd640 1 -- 172.21.15.157:0/3426209313 reap_dead start 2023-11-06T03:09:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/3426209313 shutdown_connections 2023-11-06T03:09:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/3426209313 >> 172.21.15.157:0/3426209313 conn(0x7ffbdc069d70 msgr2=0x7ffbdc06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/3426209313 shutdown_connections 2023-11-06T03:09:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/3426209313 wait complete. 2023-11-06T03:09:19.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 Processor -- start 2023-11-06T03:09:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- start start 2023-11-06T03:09:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80047d0 0x7ffbdc1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc15a530 0x7ffbdc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc159bd0 con 0x7ffbdc158210 2023-11-06T03:09:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc158c70 con 0x7ffbc80047d0 2023-11-06T03:09:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc158f50 con 0x7ffbdc15a530 2023-11-06T03:09:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44460/0 (socket says 172.21.15.157:44460) 2023-11-06T03:09:19.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 -- 172.21.15.157:0/26638908 learned_addr learned my addr 172.21.15.157:0/26638908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80047d0 0x7ffbdc1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe20fd640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc15a530 0x7ffbdc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 -- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc15a530 msgr2=0x7ffbdc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc15a530 0x7ffbdc0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 -- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80047d0 msgr2=0x7ffbdc1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80047d0 0x7ffbdc1519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 -- 172.21.15.157:0/26638908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc159230 con 0x7ffbdc158210 2023-11-06T03:09:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe10fb640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc01d420 tx=0x7ffbc8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b4070 con 0x7ffbdc158210 2023-11-06T03:09:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc005db0 con 0x7ffbdc158210 2023-11-06T03:09:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc159510 con 0x7ffbdc158210 2023-11-06T03:09:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0015c0 con 0x7ffbdc158210 2023-11-06T03:09:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.448+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc151ef0 con 0x7ffbdc158210 2023-11-06T03:09:19.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc06ad00 con 0x7ffbdc158210 2023-11-06T03:09:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffbcc007040 con 0x7ffbdc158210 2023-11-06T03:09:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbd27fc640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 0x7ffbc0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7ffbcc11cc80 con 0x7ffbdc158210 2023-11-06T03:09:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 0x7ffbc0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.452+0000 7ffbe18fc640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 0x7ffbc0067f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc166a30 tx=0x7ffbd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.456+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffbcc0ea3a0 con 0x7ffbdc158210 2023-11-06T03:09:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffbdc072750 con 0x7ffbdc158210 2023-11-06T03:09:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbd27fc640 1 -- 172.21.15.157:0/26638908 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffbcc0ed9c0 con 0x7ffbdc158210 2023-11-06T03:09:19.607 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:09:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 msgr2=0x7ffbc0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 0x7ffbc0067f50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc166a30 tx=0x7ffbd8008040 comp rx=0 tx=0).stop 2023-11-06T03:09:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 msgr2=0x7ffbdc159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc01d420 tx=0x7ffbc8005800 comp rx=0 tx=0).stop 2023-11-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 shutdown_connections 2023-11-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbc0065a90 0x7ffbc0067f50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbdc15a530 0x7ffbdc0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbdc158210 0x7ffbdc159ff0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 --2- 172.21.15.157:0/26638908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc80047d0 0x7ffbdc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 >> 172.21.15.157:0/26638908 conn(0x7ffbdc069d70 msgr2=0x7ffbdc070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 shutdown_connections 2023-11-06T03:09:19.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.608+0000 7ffbe3b60640 1 -- 172.21.15.157:0/26638908 wait complete. 2023-11-06T03:09:19.625 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:09:19.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 Processor -- start 2023-11-06T03:09:19.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 -- start start 2023-11-06T03:09:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75bc1599c0 0x7f75bc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 0x7f75bc0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc15e440 con 0x7f75bc15dea0 2023-11-06T03:09:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc15adb0 con 0x7f75bc1599c0 2023-11-06T03:09:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75c15f8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc06c240 con 0x7f75bc06d000 2023-11-06T03:09:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75bb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75bc15dea0 0x7f75bc070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60192/0 (socket says 172.21.15.157:60192) 2023-11-06T03:09:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75bb7fe640 1 -- 172.21.15.157:0/4220977904 learned_addr learned my addr 172.21.15.157:0/4220977904 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75ba7fc640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 0x7f75bc0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 --2- 172.21.15.157:0/4220977904 >> v2:172.21.15.195:3300/0 conn(0x7f75bc1599c0 0x7f75bc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824271900 0 0) 0x7f75bc15e440 con 0x7f75bc15dea0 2023-11-06T03:09:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a8005740 con 0x7f75bc15dea0 2023-11-06T03:09:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224498292 0 0) 0x7f75a8005740 con 0x7f75bc15dea0 2023-11-06T03:09:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 msgr2=0x7f75bc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 0x7f75bc0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 >> v2:172.21.15.195:3300/0 conn(0x7f75bc1599c0 msgr2=0x7f75bc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 --2- 172.21.15.157:0/4220977904 >> v2:172.21.15.195:3300/0 conn(0x7f75bc1599c0 0x7f75bc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc06c520 con 0x7f75bc15dea0 2023-11-06T03:09:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f75ac001a00 con 0x7f75bc15dea0 2023-11-06T03:09:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 >> v1:172.21.15.157:6789/0 conn(0x7f75bc15dea0 legacy=0x7f75bc070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75a8005b90 0x7f75a8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 0x7f75a80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a8006f20 0x7f75a80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc15adb0 con 0x7f75a8005b90 2023-11-06T03:09:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc06c240 con 0x7f75a8006f20 2023-11-06T03:09:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a800dba0 con 0x7f75bc06d000 2023-11-06T03:09:19.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75a8005b90 0x7f75a8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 -- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 msgr2=0x7f75a80069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc06d000 0x7f75a80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 -- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a8006f20 msgr2=0x7f75a80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:19.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75a8006f20 0x7f75a80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.736+0000 7f75baffd640 1 -- 172.21.15.157:0/4220977904 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8005740 con 0x7f75a8005b90 2023-11-06T03:09:19.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75a8005b90 0x7f75a8005fb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ae0 tx=0x7f75b401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51fcaa93fe86a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b40a5550 con 0x7f75a8005b90 2023-11-06T03:09:19.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b40a56b0 con 0x7f75a8005b90 2023-11-06T03:09:19.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75b9ffb640 1 -- 172.21.15.157:0/4220977904 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b40a5a60 con 0x7f75a8005b90 2023-11-06T03:09:19.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- 172.21.15.157:0/4220977904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75a8005b90 msgr2=0x7f75a8005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 --2- 172.21.15.157:0/4220977904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75a8005b90 0x7f75a8005fb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009ae0 tx=0x7f75b401ca60 comp rx=0 tx=0).stop 2023-11-06T03:09:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75bb7fe640 1 -- 172.21.15.157:0/4220977904 reap_dead start 2023-11-06T03:09:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- 172.21.15.157:0/4220977904 shutdown_connections 2023-11-06T03:09:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- 172.21.15.157:0/4220977904 >> 172.21.15.157:0/4220977904 conn(0x7f75bc065930 msgr2=0x7f75bc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- 172.21.15.157:0/4220977904 shutdown_connections 2023-11-06T03:09:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- 172.21.15.157:0/4220977904 wait complete. 2023-11-06T03:09:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 Processor -- start 2023-11-06T03:09:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- start start 2023-11-06T03:09:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75bc14ff50 0x7f75bc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc0acdf0 0x7f75bc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0b0160 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0ad2a0 con 0x7f75bc14ff50 2023-11-06T03:09:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75c15f8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0af2d0 con 0x7f75bc0acdf0 2023-11-06T03:09:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44474/0 (socket says 172.21.15.157:44474) 2023-11-06T03:09:19.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 -- 172.21.15.157:0/2234136506 learned_addr learned my addr 172.21.15.157:0/2234136506 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:19.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 -- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc0acdf0 msgr2=0x7f75bc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc0acdf0 0x7f75bc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 -- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75bc14ff50 msgr2=0x7f75bc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.740+0000 7f75baffd640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75bc14ff50 0x7f75bc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75baffd640 1 -- 172.21.15.157:0/2234136506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0af4e0 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75baffd640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f75b401c890 tx=0x7f75b4019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b40a5550 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400f000 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc0af7c0 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004260 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc0afd80 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75b40b4070 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc0cbe60 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 0x7f759c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:19.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f75b411d8c0 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75ba7fc640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 0x7f759c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:19.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.744+0000 7f75ba7fc640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 0x7f759c067f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006aa0 tx=0x7f75a4006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:19.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.748+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f75b40eb070 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.888+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75bc072750 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.888+0000 7f75b37fe640 1 -- 172.21.15.157:0/2234136506 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75b40ee690 con 0x7f75bc1f4ed0 2023-11-06T03:09:19.889 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:09:19.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 msgr2=0x7f759c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 0x7f759c067f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75a4006aa0 tx=0x7f75a4006a30 comp rx=0 tx=0).stop 2023-11-06T03:09:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 msgr2=0x7f75bc0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f75b401c890 tx=0x7f75b4019800 comp rx=0 tx=0).stop 2023-11-06T03:09:19.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 shutdown_connections 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f759c065a60 0x7f759c067f20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75bc0acdf0 0x7f75bc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75bc14ff50 0x7f75bc0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 --2- 172.21.15.157:0/2234136506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75bc1f4ed0 0x7f75bc0aed90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 >> 172.21.15.157:0/2234136506 conn(0x7f75bc065930 msgr2=0x7f75bc15aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:19.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 shutdown_connections 2023-11-06T03:09:19.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:19.892+0000 7f75c15f8640 1 -- 172.21.15.157:0/2234136506 wait complete. 2023-11-06T03:09:19.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:09:20.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 Processor -- start 2023-11-06T03:09:20.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 -- start start 2023-11-06T03:09:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07f0146370 0x7f07f00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07f006bdb0 0x7f07f00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00cf1a0 con 0x7f07f00ce800 2023-11-06T03:09:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f006c3d0 con 0x7f07f0146370 2023-11-06T03:09:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07f5a14640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f006f5a0 con 0x7f07f006bdb0 2023-11-06T03:09:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07f0146370 0x7f07f00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07f0146370 0x7f07f00b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44480/0 (socket says 172.21.15.157:44480) 2023-11-06T03:09:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 -- 172.21.15.157:0/3035904348 learned_addr learned my addr 172.21.15.157:0/3035904348 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07ef7fe640 1 --2- 172.21.15.157:0/3035904348 >> v2:172.21.15.195:3300/0 conn(0x7f07f006bdb0 0x7f07f00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 -- 172.21.15.157:0/3035904348 >> v2:172.21.15.195:3300/0 conn(0x7f07f006bdb0 msgr2=0x7f07f00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:20.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 --2- 172.21.15.157:0/3035904348 >> v2:172.21.15.195:3300/0 conn(0x7f07f006bdb0 0x7f07f00cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 -- 172.21.15.157:0/3035904348 >> v1:172.21.15.195:6789/0 conn(0x7f07f00ce800 legacy=0x7f07f006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:20.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.024+0000 7f07eeffd640 1 -- 172.21.15.157:0/3035904348 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f006f880 con 0x7f07f0146370 2023-11-06T03:09:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07eeffd640 1 --2- 172.21.15.157:0/3035904348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f00b1a10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009870 tx=0x7f07e001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453cbc799801f53d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07edffb640 1 -- 172.21.15.157:0/3035904348 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00ac640 con 0x7f07f0146370 2023-11-06T03:09:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07edffb640 1 -- 172.21.15.157:0/3035904348 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00ac7a0 con 0x7f07f0146370 2023-11-06T03:09:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07edffb640 1 -- 172.21.15.157:0/3035904348 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00acb30 con 0x7f07f0146370 2023-11-06T03:09:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- 172.21.15.157:0/3035904348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 msgr2=0x7f07f00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3035904348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f00b1a10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009870 tx=0x7f07e001c400 comp rx=0 tx=0).stop 2023-11-06T03:09:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- 172.21.15.157:0/3035904348 shutdown_connections 2023-11-06T03:09:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3035904348 >> v2:172.21.15.195:3300/0 conn(0x7f07f006bdb0 0x7f07f00cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3035904348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f00b1a10 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- 172.21.15.157:0/3035904348 >> 172.21.15.157:0/3035904348 conn(0x7f07f0065930 msgr2=0x7f07f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- 172.21.15.157:0/3035904348 shutdown_connections 2023-11-06T03:09:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- 172.21.15.157:0/3035904348 wait complete. 2023-11-06T03:09:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 Processor -- start 2023-11-06T03:09:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.028+0000 7f07f5a14640 1 -- start start 2023-11-06T03:09:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f006bdb0 0x7f07f0157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f00ce800 0x7f07f00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0158090 con 0x7f07f0146370 2023-11-06T03:09:20.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0156ab0 con 0x7f07f006bdb0 2023-11-06T03:09:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0156d90 con 0x7f07f00ce800 2023-11-06T03:09:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44484/0 (socket says 172.21.15.157:44484) 2023-11-06T03:09:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 -- 172.21.15.157:0/3051554 learned_addr learned my addr 172.21.15.157:0/3051554 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07eeffd640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f006bdb0 0x7f07f0157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 -- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f00ce800 msgr2=0x7f07f00aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f00ce800 0x7f07f00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 -- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f006bdb0 msgr2=0x7f07f0157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f006bdb0 0x7f07f0157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 -- 172.21.15.157:0/3051554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00094d0 con 0x7f07f0146370 2023-11-06T03:09:20.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07ef7fe640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f07e800cc30 tx=0x7f07e800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e800ce40 con 0x7f07f0146370 2023-11-06T03:09:20.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e80039a0 con 0x7f07f0146370 2023-11-06T03:09:20.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f0157070 con 0x7f07f0146370 2023-11-06T03:09:20.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8003d30 con 0x7f07f0146370 2023-11-06T03:09:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f0157630 con 0x7f07f0146370 2023-11-06T03:09:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f0157850 con 0x7f07f0146370 2023-11-06T03:09:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f07e8003e90 con 0x7f07f0146370 2023-11-06T03:09:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 0x7f07dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f07e8083800 con 0x7f07f0146370 2023-11-06T03:09:20.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07eeffd640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 0x7f07dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.032+0000 7f07eeffd640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 0x7f07dc067f20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f07e00ae000 tx=0x7f07e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.036+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f07e8050f20 con 0x7f07f0146370 2023-11-06T03:09:20.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.188+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07f00adac0 con 0x7f07f0146370 2023-11-06T03:09:20.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.188+0000 7f07d7fff640 1 -- 172.21.15.157:0/3051554 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07e801d020 con 0x7f07f0146370 2023-11-06T03:09:20.190 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:09:20.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 msgr2=0x7f07dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 0x7f07dc067f20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f07e00ae000 tx=0x7f07e0008040 comp rx=0 tx=0).stop 2023-11-06T03:09:20.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 msgr2=0x7f07f013c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f07e800cc30 tx=0x7f07e800a5c0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 shutdown_connections 2023-11-06T03:09:20.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07dc065a60 0x7f07dc067f20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07f0146370 0x7f07f013c0f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07f00ce800 0x7f07f00aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 --2- 172.21.15.157:0/3051554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07f006bdb0 0x7f07f0157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 >> 172.21.15.157:0/3051554 conn(0x7f07f0065930 msgr2=0x7f07f015ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:20.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 shutdown_connections 2023-11-06T03:09:20.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.192+0000 7f07f5a14640 1 -- 172.21.15.157:0/3051554 wait complete. 2023-11-06T03:09:20.208 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:09:20.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 Processor -- start 2023-11-06T03:09:20.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 -- start start 2023-11-06T03:09:20.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a841551e0 0x7f5a8406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a8415e5e0 0x7f5a840b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8415f580 con 0x7f5a8415ea60 2023-11-06T03:09:20.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8415efc0 con 0x7f5a8415e5e0 2023-11-06T03:09:20.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.320+0000 7f5a8acff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8415f2a0 con 0x7f5a841551e0 2023-11-06T03:09:20.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a8415e5e0 0x7f5a840b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a8415e5e0 0x7f5a840b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44498/0 (socket says 172.21.15.157:44498) 2023-11-06T03:09:20.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 -- 172.21.15.157:0/1452682874 learned_addr learned my addr 172.21.15.157:0/1452682874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:20.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8929c640 1 --1- 172.21.15.157:0/1452682874 >> v1:172.21.15.157:6789/0 conn(0x7f5a8415ea60 0x7f5a8415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60198/0 (socket says 172.21.15.157:60198) 2023-11-06T03:09:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 -- 172.21.15.157:0/1452682874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a841551e0 msgr2=0x7f5a8406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 --2- 172.21.15.157:0/1452682874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a841551e0 0x7f5a8406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 -- 172.21.15.157:0/1452682874 >> v1:172.21.15.157:6789/0 conn(0x7f5a8415ea60 legacy=0x7f5a8415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 -- 172.21.15.157:0/1452682874 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8406f5a0 con 0x7f5a8415e5e0 2023-11-06T03:09:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a837fe640 1 -- 172.21.15.157:0/1452682874 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636558356 0 0) 0x7f5a8415f580 con 0x7f5a8415ea60 2023-11-06T03:09:20.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a83fff640 1 --2- 172.21.15.157:0/1452682874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8415e5e0 0x7f5a840b3410 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009870 tx=0x7f5a7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd9abd283f2a92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a837fe640 1 -- 172.21.15.157:0/1452682874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780ad640 con 0x7f5a8415e5e0 2023-11-06T03:09:20.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a837fe640 1 -- 172.21.15.157:0/1452682874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a780ad7a0 con 0x7f5a8415e5e0 2023-11-06T03:09:20.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a837fe640 1 -- 172.21.15.157:0/1452682874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780adb30 con 0x7f5a8415e5e0 2023-11-06T03:09:20.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 -- 172.21.15.157:0/1452682874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8415e5e0 msgr2=0x7f5a840b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1452682874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8415e5e0 0x7f5a840b3410 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009870 tx=0x7f5a7801c480 comp rx=0 tx=0).stop 2023-11-06T03:09:20.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 -- 172.21.15.157:0/1452682874 shutdown_connections 2023-11-06T03:09:20.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1452682874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8415e5e0 0x7f5a840b3410 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1452682874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a841551e0 0x7f5a8406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 -- 172.21.15.157:0/1452682874 >> 172.21.15.157:0/1452682874 conn(0x7f5a84065930 msgr2=0x7f5a84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:20.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 -- 172.21.15.157:0/1452682874 shutdown_connections 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.324+0000 7f5a8acff640 1 -- 172.21.15.157:0/1452682874 wait complete. 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 Processor -- start 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- start start 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a8415e5e0 0x7f5a84143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a8415ea60 0x7f5a8414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8415dd80 con 0x7f5a841551e0 2023-11-06T03:09:20.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a841448c0 con 0x7f5a8415e5e0 2023-11-06T03:09:20.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840af490 con 0x7f5a8415ea60 2023-11-06T03:09:20.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44506/0 (socket says 172.21.15.157:44506) 2023-11-06T03:09:20.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 -- 172.21.15.157:0/1073577832 learned_addr learned my addr 172.21.15.157:0/1073577832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:20.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 -- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a8415ea60 msgr2=0x7f5a8414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a8415ea60 0x7f5a8414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 -- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a8415e5e0 msgr2=0x7f5a84143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a8415e5e0 0x7f5a84143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 -- 172.21.15.157:0/1073577832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780094d0 con 0x7f5a841551e0 2023-11-06T03:09:20.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a88a9b640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b410 tx=0x7f5a7400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74011830 con 0x7f5a841551e0 2023-11-06T03:09:20.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74011990 con 0x7f5a841551e0 2023-11-06T03:09:20.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a84143cb0 con 0x7f5a841551e0 2023-11-06T03:09:20.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74011d40 con 0x7f5a841551e0 2023-11-06T03:09:20.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a84144270 con 0x7f5a841551e0 2023-11-06T03:09:20.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a7401a400 con 0x7f5a841551e0 2023-11-06T03:09:20.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a84072750 con 0x7f5a841551e0 2023-11-06T03:09:20.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.328+0000 7f5a817fa640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 0x7f5a70067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.332+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f5a74083620 con 0x7f5a841551e0 2023-11-06T03:09:20.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.332+0000 7f5a83fff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 0x7f5a70067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.332+0000 7f5a83fff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 0x7f5a70067f20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5a780af000 tx=0x7f5a78004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.336+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5a74050ca0 con 0x7f5a841551e0 2023-11-06T03:09:20.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.488+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a84071730 con 0x7f5a841551e0 2023-11-06T03:09:20.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.488+0000 7f5a817fa640 1 -- 172.21.15.157:0/1073577832 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v73) v1 ==== 80+0+248 (secure 0 0 0) 0x7f5a74015110 con 0x7f5a841551e0 2023-11-06T03:09:20.489 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.195:6837/3999445109 2023-11-07T03:09:02.175205+0000 2023-11-06T03:09:20.490 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.195:6836/3999445109 2023-11-07T03:09:02.175205+0000 2023-11-06T03:09:20.490 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157:6835/1218323165 2023-11-07T03:09:02.146924+0000 2023-11-06T03:09:20.490 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157:6834/1218323165 2023-11-07T03:09:02.146924+0000 2023-11-06T03:09:20.491 INFO:teuthology.orchestra.run.smithi157.stderr:listed 4 entries 2023-11-06T03:09:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 msgr2=0x7f5a70067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 0x7f5a70067f20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5a780af000 tx=0x7f5a78004740 comp rx=0 tx=0).stop 2023-11-06T03:09:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 msgr2=0x7f5a8415d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b410 tx=0x7f5a7400b8d0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 shutdown_connections 2023-11-06T03:09:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a70065a60 0x7f5a70067f20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a8415ea60 0x7f5a8414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a8415e5e0 0x7f5a84143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 --2- 172.21.15.157:0/1073577832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a841551e0 0x7f5a8415d840 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 >> 172.21.15.157:0/1073577832 conn(0x7f5a84065930 msgr2=0x7f5a8414e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 shutdown_connections 2023-11-06T03:09:20.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.492+0000 7f5a8acff640 1 -- 172.21.15.157:0/1073577832 wait complete. 2023-11-06T03:09:20.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:09:20.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:20.507 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:20.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[65952]: fuse finished with error 0 and tester_r 0 2023-11-06T03:09:20.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:09:20.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:20.520 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:09:20.683 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:09:20.684 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:20.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:09:20.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:09:20.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:20.692 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:09:20.742 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:09:20.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:09:20.744 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:09:20.745 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:20.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:09:20.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:09:20.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:20.752 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:09:20.798 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:09:20.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:09:20.800 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:09:20.800 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:20.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:09:20.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:09:20.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:20.808 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:09:20.854 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:09:20.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:09:20.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.988+0000 7f130ada5640 1 Processor -- start 2023-11-06T03:09:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.988+0000 7f130ada5640 1 -- start start 2023-11-06T03:09:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f130ada5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f130ada5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f130415ea60 0x7f13040b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f130ada5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130415f580 con 0x7f13041551e0 2023-11-06T03:09:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f130ada5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040704b0 con 0x7f130415ea60 2023-11-06T03:09:20.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f130ada5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130415efc0 con 0x7f130415e5e0 2023-11-06T03:09:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1309342640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f130415ea60 0x7f13040b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49472/0 (socket says 172.21.15.157:49472) 2023-11-06T03:09:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 -- 172.21.15.157:0/3694483685 learned_addr learned my addr 172.21.15.157:0/3694483685 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:20.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1308b41640 1 --1- 172.21.15.157:0/3694483685 >> v1:172.21.15.195:6789/0 conn(0x7f13041551e0 0x7f130406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48568/0 (socket says 172.21.15.157:48568) 2023-11-06T03:09:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 -- 172.21.15.157:0/3694483685 >> v2:172.21.15.195:3300/0 conn(0x7f130415ea60 msgr2=0x7f13040b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:20.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 --2- 172.21.15.157:0/3694483685 >> v2:172.21.15.195:3300/0 conn(0x7f130415ea60 0x7f13040b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 -- 172.21.15.157:0/3694483685 >> v1:172.21.15.195:6789/0 conn(0x7f13041551e0 legacy=0x7f130406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.992+0000 7f1303fff640 1 -- 172.21.15.157:0/3694483685 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130415f2a0 con 0x7f130415e5e0 2023-11-06T03:09:20.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1303fff640 1 --2- 172.21.15.157:0/3694483685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009870 tx=0x7f12f801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3dc7b0d30b6f49f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f13037fe640 1 -- 172.21.15.157:0/3694483685 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80ab680 con 0x7f130415e5e0 2023-11-06T03:09:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f13037fe640 1 -- 172.21.15.157:0/3694483685 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f80ab7e0 con 0x7f130415e5e0 2023-11-06T03:09:20.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f13037fe640 1 -- 172.21.15.157:0/3694483685 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80abb70 con 0x7f130415e5e0 2023-11-06T03:09:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/3694483685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 msgr2=0x7f13040b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- 172.21.15.157:0/3694483685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009870 tx=0x7f12f801c480 comp rx=0 tx=0).stop 2023-11-06T03:09:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/3694483685 shutdown_connections 2023-11-06T03:09:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- 172.21.15.157:0/3694483685 >> v2:172.21.15.195:3300/0 conn(0x7f130415ea60 0x7f13040b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- 172.21.15.157:0/3694483685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f130415e5e0 0x7f13040b3410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/3694483685 >> 172.21.15.157:0/3694483685 conn(0x7f1304065930 msgr2=0x7f1304065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/3694483685 shutdown_connections 2023-11-06T03:09:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/3694483685 wait complete. 2023-11-06T03:09:20.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 Processor -- start 2023-11-06T03:09:20.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- start start 2023-11-06T03:09:21.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13041551e0 0x7f130414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 0x7f130414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304150350 con 0x7f130415ea60 2023-11-06T03:09:21.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130414d310 con 0x7f130415e5e0 2023-11-06T03:09:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130414d5b0 con 0x7f13041551e0 2023-11-06T03:09:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44512/0 (socket says 172.21.15.157:44512) 2023-11-06T03:09:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 -- 172.21.15.157:0/1320888662 learned_addr learned my addr 172.21.15.157:0/1320888662 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1308b41640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13041551e0 0x7f130414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1303fff640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 0x7f130414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 -- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13041551e0 msgr2=0x7f130414e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13041551e0 0x7f130414e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 -- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 msgr2=0x7f130414cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 0x7f130414cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 -- 172.21.15.157:0/1320888662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80094d0 con 0x7f130415ea60 2023-11-06T03:09:21.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1303fff640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 0x7f130414cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:21.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f1309342640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00efc0 tx=0x7f12fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc009b20 con 0x7f130415ea60 2023-11-06T03:09:21.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc00f040 con 0x7f130415ea60 2023-11-06T03:09:21.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:20.996+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130414d890 con 0x7f130415ea60 2023-11-06T03:09:21.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0146c0 con 0x7f130415ea60 2023-11-06T03:09:21.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13041505a0 con 0x7f130415ea60 2023-11-06T03:09:21.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13040cbdd0 con 0x7f130415ea60 2023-11-06T03:09:21.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f12fc014820 con 0x7f130415ea60 2023-11-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f13017fa640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 0x7f12f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f12fc0841d0 con 0x7f130415ea60 2023-11-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f1308b41640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 0x7f12f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.000+0000 7f1308b41640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 0x7f12f0067f20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000f70 tx=0x7f12f4007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.004+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f12fc051980 con 0x7f130415ea60 2023-11-06T03:09:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.192+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13040b3e30 con 0x7f130415ea60 2023-11-06T03:09:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.192+0000 7f13017fa640 1 -- 172.21.15.157:0/1320888662 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f12fc009e60 con 0x7f130415ea60 2023-11-06T03:09:21.193 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:21.194 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:05.306735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:21.194 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-11-06T03:09:21.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 msgr2=0x7f12f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 0x7f12f0067f20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f12f4000f70 tx=0x7f12f4007c90 comp rx=0 tx=0).stop 2023-11-06T03:09:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 msgr2=0x7f130414a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00efc0 tx=0x7f12fc00c490 comp rx=0 tx=0).stop 2023-11-06T03:09:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 shutdown_connections 2023-11-06T03:09:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12f0065a60 0x7f12f0067f20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f130415ea60 0x7f130414a770 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f130415e5e0 0x7f130414cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 --2- 172.21.15.157:0/1320888662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f13041551e0 0x7f130414e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 >> 172.21.15.157:0/1320888662 conn(0x7f1304065930 msgr2=0x7f130414bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 shutdown_connections 2023-11-06T03:09:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.196+0000 7f130ada5640 1 -- 172.21.15.157:0/1320888662 wait complete. 2023-11-06T03:09:21.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:21.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 Processor -- start 2023-11-06T03:09:21.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 -- start start 2023-11-06T03:09:21.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 0x7f34b00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b00d2920 con 0x7f34b00ce1a0 2023-11-06T03:09:21.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b00d18a0 con 0x7f34b015ce80 2023-11-06T03:09:21.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.332+0000 7f34b6e42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00d1b60 con 0x7f34b015caa0 2023-11-06T03:09:21.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34affff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 0x7f34b00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49482/0 (socket says 172.21.15.157:49482) 2023-11-06T03:09:21.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 -- 172.21.15.157:0/4245492428 learned_addr learned my addr 172.21.15.157:0/4245492428 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34affff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 0x7f34b00cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:49488/0 (socket says 172.21.15.157:49488) 2023-11-06T03:09:21.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b53df640 1 --1- 172.21.15.157:0/4245492428 >> v1:172.21.15.195:6789/0 conn(0x7f34b00ce1a0 0x7f34b00d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48580/0 (socket says 172.21.15.157:48580) 2023-11-06T03:09:21.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 -- 172.21.15.157:0/4245492428 >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 msgr2=0x7f34b00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:21.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4245492428 >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 0x7f34b00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 -- 172.21.15.157:0/4245492428 >> v1:172.21.15.195:6789/0 conn(0x7f34b00ce1a0 legacy=0x7f34b00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 -- 172.21.15.157:0/4245492428 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00d1e40 con 0x7f34b015caa0 2023-11-06T03:09:21.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4245492428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009870 tx=0x7f34a001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996a05d419a39430 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34af7fe640 1 -- 172.21.15.157:0/4245492428 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a00ab620 con 0x7f34b015caa0 2023-11-06T03:09:21.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34af7fe640 1 -- 172.21.15.157:0/4245492428 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a00ab780 con 0x7f34b015caa0 2023-11-06T03:09:21.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.336+0000 7f34af7fe640 1 -- 172.21.15.157:0/4245492428 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a00abb10 con 0x7f34b015caa0 2023-11-06T03:09:21.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4245492428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 msgr2=0x7f34b00cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4245492428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009870 tx=0x7f34a001c510 comp rx=0 tx=0).stop 2023-11-06T03:09:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4245492428 shutdown_connections 2023-11-06T03:09:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4245492428 >> v2:172.21.15.195:3300/0 conn(0x7f34b015ce80 0x7f34b00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4245492428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b015caa0 0x7f34b00cdc60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4245492428 >> 172.21.15.157:0/4245492428 conn(0x7f34b0070790 msgr2=0x7f34b0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4245492428 shutdown_connections 2023-11-06T03:09:21.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4245492428 wait complete. 2023-11-06T03:09:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 Processor -- start 2023-11-06T03:09:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- start start 2023-11-06T03:09:21.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 0x7f34b00ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b015ce80 0x7f34b01f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00a99a0 con 0x7f34b015caa0 2023-11-06T03:09:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00a9c80 con 0x7f34b015ce80 2023-11-06T03:09:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00a9f60 con 0x7f34b00ce1a0 2023-11-06T03:09:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44526/0 (socket says 172.21.15.157:44526) 2023-11-06T03:09:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 -- 172.21.15.157:0/4060185762 learned_addr learned my addr 172.21.15.157:0/4060185762 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b53df640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b015ce80 0x7f34b01f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 0x7f34b00ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 -- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 msgr2=0x7f34b00ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 0x7f34b00ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 -- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b015ce80 msgr2=0x7f34b01f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b015ce80 0x7f34b01f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 -- 172.21.15.157:0/4060185762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00094d0 con 0x7f34b015caa0 2023-11-06T03:09:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 0x7f34b00ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34affff640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e930 tx=0x7f34a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4009690 con 0x7f34b015caa0 2023-11-06T03:09:21.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400ca00 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b00aa240 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400cd70 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.340+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b00ac650 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b0072750 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f34a4013470 con 0x7f34b015caa0 2023-11-06T03:09:21.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34ad7fa640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 0x7f349c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f34a40836a0 con 0x7f34b015caa0 2023-11-06T03:09:21.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 0x7f349c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.344+0000 7f34b4bde640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 0x7f349c067f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34a00af000 tx=0x7f34a00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.348+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f34a4050eb0 con 0x7f34b015caa0 2023-11-06T03:09:21.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.496+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34b00c93d0 con 0x7f34b015caa0 2023-11-06T03:09:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.500+0000 7f34ad7fa640 1 -- 172.21.15.157:0/4060185762 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18375 (secure 0 0 0) 0x7f34a40544d0 con 0x7f34b015caa0 2023-11-06T03:09:21.501 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:21.501 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:02.175228+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.195:6836/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.157:6835/1218323165":"2023-11-07T03:09:02.146924+0000","172.21.15.157:6834/1218323165":"2023-11-07T03:09:02.146924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 msgr2=0x7f349c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 0x7f349c067f20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34a00af000 tx=0x7f34a00a7000 comp rx=0 tx=0).stop 2023-11-06T03:09:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 msgr2=0x7f34b00c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e930 tx=0x7f34a400edf0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 shutdown_connections 2023-11-06T03:09:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f349c065a60 0x7f349c067f20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b015ce80 0x7f34b01f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b015caa0 0x7f34b00c76a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 --2- 172.21.15.157:0/4060185762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b00ce1a0 0x7f34b00ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 >> 172.21.15.157:0/4060185762 conn(0x7f34b0070790 msgr2=0x7f34b0065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 shutdown_connections 2023-11-06T03:09:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.504+0000 7f34b6e42640 1 -- 172.21.15.157:0/4060185762 wait complete. 2023-11-06T03:09:21.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:08:41.869490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:08:42.287789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:09:21.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:09:21.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.640+0000 7fafdca4a640 1 Processor -- start 2023-11-06T03:09:21.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafdca4a640 1 -- start start 2023-11-06T03:09:21.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafdca4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fafd81551e0 0x7fafd806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafdca4a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd815f580 con 0x7fafd815e5e0 2023-11-06T03:09:21.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafdca4a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd815efc0 con 0x7fafd80b3b20 2023-11-06T03:09:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafdca4a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd815f2a0 con 0x7fafd81551e0 2023-11-06T03:09:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fafd81551e0 0x7fafd806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fafd81551e0 0x7fafd806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44538/0 (socket says 172.21.15.157:44538) 2023-11-06T03:09:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 -- 172.21.15.157:0/4250623199 learned_addr learned my addr 172.21.15.157:0/4250623199 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 -- 172.21.15.157:0/4250623199 >> v1:172.21.15.195:6789/0 conn(0x7fafd80b3b20 legacy=0x7fafd815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:21.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 -- 172.21.15.157:0/4250623199 >> v1:172.21.15.157:6789/0 conn(0x7fafd815e5e0 legacy=0x7fafd80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 -- 172.21.15.157:0/4250623199 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd806f5a0 con 0x7fafd81551e0 2023-11-06T03:09:21.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd659c640 1 --2- 172.21.15.157:0/4250623199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd806fda0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009870 tx=0x7fafc801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909ed0fd1301f26 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd559a640 1 -- 172.21.15.157:0/4250623199 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80ac630 con 0x7fafd81551e0 2023-11-06T03:09:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd559a640 1 -- 172.21.15.157:0/4250623199 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc80ac790 con 0x7fafd81551e0 2023-11-06T03:09:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.644+0000 7fafd559a640 1 -- 172.21.15.157:0/4250623199 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80acb20 con 0x7fafd81551e0 2023-11-06T03:09:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/4250623199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 msgr2=0x7fafd806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 --2- 172.21.15.157:0/4250623199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd806fda0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009870 tx=0x7fafc801c3b0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/4250623199 shutdown_connections 2023-11-06T03:09:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 --2- 172.21.15.157:0/4250623199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd806fda0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/4250623199 >> 172.21.15.157:0/4250623199 conn(0x7fafd8065930 msgr2=0x7fafd8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/4250623199 shutdown_connections 2023-11-06T03:09:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/4250623199 wait complete. 2023-11-06T03:09:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 Processor -- start 2023-11-06T03:09:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- start start 2023-11-06T03:09:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafd80b3b20 0x7fafd814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 0x7fafd81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8153020 con 0x7fafd81551e0 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8153300 con 0x7fafd815e5e0 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81535e0 con 0x7fafd80b3b20 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44542/0 (socket says 172.21.15.157:44542) 2023-11-06T03:09:21.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 -- 172.21.15.157:0/3838963217 learned_addr learned my addr 172.21.15.157:0/3838963217 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd6d9d640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 0x7fafd81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd659c640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafd80b3b20 0x7fafd814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 -- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafd80b3b20 msgr2=0x7fafd814b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafd80b3b20 0x7fafd814b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 -- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 msgr2=0x7fafd81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 0x7fafd81540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 -- 172.21.15.157:0/3838963217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80094d0 con 0x7fafd81551e0 2023-11-06T03:09:21.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd6d9d640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 0x7fafd81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:21.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafd5d9b640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c3c0 tx=0x7fafcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0052d0 con 0x7fafd81551e0 2023-11-06T03:09:21.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc005430 con 0x7fafd81551e0 2023-11-06T03:09:21.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd81538c0 con 0x7fafd81551e0 2023-11-06T03:09:21.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc003840 con 0x7fafd81551e0 2023-11-06T03:09:21.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.648+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd814be50 con 0x7fafd81551e0 2023-11-06T03:09:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd8153de0 con 0x7fafd81551e0 2023-11-06T03:09:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fafcc0039a0 con 0x7fafd81551e0 2023-11-06T03:09:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafc6ffd640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fafcc0828e0 con 0x7fafd81551e0 2023-11-06T03:09:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafd659c640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.652+0000 7fafd659c640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 0x7fafb8067f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.656+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fafd8153de0 con 0x7fafd81551e0 2023-11-06T03:09:21.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafd8153de0 con 0x7fafd81551e0 2023-11-06T03:09:21.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafc6ffd640 1 -- 172.21.15.157:0/3838963217 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafcc017020 con 0x7fafd81551e0 2023-11-06T03:09:21.815 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:09:21.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 msgr2=0x7fafb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 0x7fafb8067f20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ae000 tx=0x7fafc8008040 comp rx=0 tx=0).stop 2023-11-06T03:09:21.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 msgr2=0x7fafd814c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.816+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00c3c0 tx=0x7fafcc00c880 comp rx=0 tx=0).stop 2023-11-06T03:09:21.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 shutdown_connections 2023-11-06T03:09:21.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fafb8065a60 0x7fafb8067f20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafd815e5e0 0x7fafd81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fafd81551e0 0x7fafd814c160 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 --2- 172.21.15.157:0/3838963217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fafd80b3b20 0x7fafd814b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 >> 172.21.15.157:0/3838963217 conn(0x7fafd8065930 msgr2=0x7fafd8150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 shutdown_connections 2023-11-06T03:09:21.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.820+0000 7fafdca4a640 1 -- 172.21.15.157:0/3838963217 wait complete. 2023-11-06T03:09:21.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:09:21.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.956+0000 7f64a1979640 1 Processor -- start 2023-11-06T03:09:21.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.956+0000 7f64a1979640 1 -- start start 2023-11-06T03:09:21.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f649c15e5e0 0x7f649c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f649c15ea60 0x7f649c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c15f580 con 0x7f649c15ea60 2023-11-06T03:09:21.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c15efc0 con 0x7f649c15e5e0 2023-11-06T03:09:21.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f64a1979640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15f2a0 con 0x7f649c1551e0 2023-11-06T03:09:21.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f649c15ea60 0x7f649c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f649c15ea60 0x7f649c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44552/0 (socket says 172.21.15.157:44552) 2023-11-06T03:09:21.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 -- 172.21.15.157:0/4292036370 learned_addr learned my addr 172.21.15.157:0/4292036370 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649affd640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649a7fc640 1 --2- 172.21.15.157:0/4292036370 >> v2:172.21.15.195:3300/0 conn(0x7f649c15e5e0 0x7f649c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 -- 172.21.15.157:0/4292036370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 msgr2=0x7f649c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:21.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 -- 172.21.15.157:0/4292036370 >> v2:172.21.15.195:3300/0 conn(0x7f649c15e5e0 msgr2=0x7f649c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:21.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 --2- 172.21.15.157:0/4292036370 >> v2:172.21.15.195:3300/0 conn(0x7f649c15e5e0 0x7f649c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 -- 172.21.15.157:0/4292036370 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c06f5a0 con 0x7f649c15ea60 2023-11-06T03:09:21.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f649b7fe640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b3950 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f649c0668b0 tx=0x7f649401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca3ef704b158e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f6499ffb640 1 -- 172.21.15.157:0/4292036370 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940a57b0 con 0x7f649c15ea60 2023-11-06T03:09:21.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f6499ffb640 1 -- 172.21.15.157:0/4292036370 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940a5910 con 0x7f649c15ea60 2023-11-06T03:09:21.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.960+0000 7f6499ffb640 1 -- 172.21.15.157:0/4292036370 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940a5ca0 con 0x7f649c15ea60 2023-11-06T03:09:21.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/4292036370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 msgr2=0x7f649c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b3950 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f649c0668b0 tx=0x7f649401c910 comp rx=0 tx=0).stop 2023-11-06T03:09:21.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/4292036370 shutdown_connections 2023-11-06T03:09:21.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b3950 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- 172.21.15.157:0/4292036370 >> v2:172.21.15.195:3300/0 conn(0x7f649c15e5e0 0x7f649c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- 172.21.15.157:0/4292036370 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/4292036370 >> 172.21.15.157:0/4292036370 conn(0x7f649c065930 msgr2=0x7f649c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/4292036370 shutdown_connections 2023-11-06T03:09:21.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/4292036370 wait complete. 2023-11-06T03:09:21.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 Processor -- start 2023-11-06T03:09:21.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- start start 2023-11-06T03:09:21.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649c15e5e0 0x7f649c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1506e0 con 0x7f649c15ea60 2023-11-06T03:09:21.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c150e20 con 0x7f649c15e5e0 2023-11-06T03:09:21.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c151100 con 0x7f649c1551e0 2023-11-06T03:09:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44562/0 (socket says 172.21.15.157:44562) 2023-11-06T03:09:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 -- 172.21.15.157:0/390430293 learned_addr learned my addr 172.21.15.157:0/390430293 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649a7fc640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649c15e5e0 0x7f649c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649affd640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 -- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 msgr2=0x7f649c13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c13eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 -- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649c15e5e0 msgr2=0x7f649c1501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649c15e5e0 0x7f649c1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 -- 172.21.15.157:0/390430293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494009b30 con 0x7f649c15ea60 2023-11-06T03:09:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f649b7fe640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f64940af040 tx=0x7f64940196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b8850 con 0x7f649c15ea60 2023-11-06T03:09:21.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64940b89b0 con 0x7f649c15ea60 2023-11-06T03:09:21.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c1513e0 con 0x7f649c15ea60 2023-11-06T03:09:21.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940b8d20 con 0x7f649c15ea60 2023-11-06T03:09:21.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.964+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c1519a0 con 0x7f649c15ea60 2023-11-06T03:09:21.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c072750 con 0x7f649c15ea60 2023-11-06T03:09:21.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f64940b6410 con 0x7f649c15ea60 2023-11-06T03:09:21.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f6483fff640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 0x7f6488067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:21.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f649411e860 con 0x7f649c15ea60 2023-11-06T03:09:21.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f649affd640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 0x7f6488067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:21.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.968+0000 7f649affd640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 0x7f6488067f20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b7e0 tx=0x7f648c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:21.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:21.972+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f64940ebf00 con 0x7f649c15ea60 2023-11-06T03:09:22.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.116+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f649c151b90 con 0x7f649c15ea60 2023-11-06T03:09:22.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f6483fff640 1 -- 172.21.15.157:0/390430293 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7f64940ef520 con 0x7f649c15ea60 2023-11-06T03:09:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:09:22.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 msgr2=0x7f6488067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 0x7f6488067f20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b7e0 tx=0x7f648c008040 comp rx=0 tx=0).stop 2023-11-06T03:09:22.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 msgr2=0x7f649c0b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f64940af040 tx=0x7f64940196b0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 shutdown_connections 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6488065a60 0x7f6488067f20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f649c15ea60 0x7f649c0b22b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649c15e5e0 0x7f649c1501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 --2- 172.21.15.157:0/390430293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f649c1551e0 0x7f649c13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 >> 172.21.15.157:0/390430293 conn(0x7f649c065930 msgr2=0x7f649c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 shutdown_connections 2023-11-06T03:09:22.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.120+0000 7f64a1979640 1 -- 172.21.15.157:0/390430293 wait complete. 2023-11-06T03:09:22.136 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:09:22.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 Processor -- start 2023-11-06T03:09:22.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 -- start start 2023-11-06T03:09:22.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f840006d2d0 0x7f840015d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84000cf820 con 0x7f8400066040 2023-11-06T03:09:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84000cfb00 con 0x7f8400157070 2023-11-06T03:09:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f840600a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840006d8f0 con 0x7f840006d2d0 2023-11-06T03:09:22.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83f7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8400066040 0x7f840006cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60218/0 (socket says 172.21.15.157:60218) 2023-11-06T03:09:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83f7fff640 1 -- 172.21.15.157:0/1769085720 learned_addr learned my addr 172.21.15.157:0/1769085720 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83fffff640 1 --2- 172.21.15.157:0/1769085720 >> v2:172.21.15.157:3300/0 conn(0x7f840006d2d0 0x7f840015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83fffff640 1 -- 172.21.15.157:0/1769085720 >> v1:172.21.15.195:6789/0 conn(0x7f8400157070 legacy=0x7f8400065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83fffff640 1 -- 172.21.15.157:0/1769085720 >> v1:172.21.15.157:6789/0 conn(0x7f8400066040 legacy=0x7f840006cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83fffff640 1 -- 172.21.15.157:0/1769085720 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000cfde0 con 0x7f840006d2d0 2023-11-06T03:09:22.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83fffff640 1 --2- 172.21.15.157:0/1769085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f840006d2d0 0x7f840015d140 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009d90 tx=0x7f83f801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3931ad4af69385 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.260+0000 7f83feffd640 1 -- 172.21.15.157:0/1769085720 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f80b0600 con 0x7f840006d2d0 2023-11-06T03:09:22.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83feffd640 1 -- 172.21.15.157:0/1769085720 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f80b0760 con 0x7f840006d2d0 2023-11-06T03:09:22.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83feffd640 1 -- 172.21.15.157:0/1769085720 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f80b0af0 con 0x7f840006d2d0 2023-11-06T03:09:22.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/1769085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f840006d2d0 msgr2=0x7f840015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 --2- 172.21.15.157:0/1769085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f840006d2d0 0x7f840015d140 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009d90 tx=0x7f83f801c710 comp rx=0 tx=0).stop 2023-11-06T03:09:22.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/1769085720 shutdown_connections 2023-11-06T03:09:22.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 --2- 172.21.15.157:0/1769085720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f840006d2d0 0x7f840015d140 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/1769085720 >> 172.21.15.157:0/1769085720 conn(0x7f8400069d70 msgr2=0x7f840006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/1769085720 shutdown_connections 2023-11-06T03:09:22.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/1769085720 wait complete. 2023-11-06T03:09:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 Processor -- start 2023-11-06T03:09:22.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- start start 2023-11-06T03:09:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f840006d2d0 0x7f840014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8400157070 0x7f840015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8400161740 con 0x7f8400066040 2023-11-06T03:09:22.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8400157800 con 0x7f8400157070 2023-11-06T03:09:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84001fa350 con 0x7f840006d2d0 2023-11-06T03:09:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44580/0 (socket says 172.21.15.157:44580) 2023-11-06T03:09:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 -- 172.21.15.157:0/2495365443 learned_addr learned my addr 172.21.15.157:0/2495365443 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:22.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 -- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f840006d2d0 msgr2=0x7f840014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f840006d2d0 0x7f840014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 -- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8400157070 msgr2=0x7f840015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8400157070 0x7f840015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 -- 172.21.15.157:0/2495365443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8009a70 con 0x7f8400066040 2023-11-06T03:09:22.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83ff7fe640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c480 tx=0x7f83f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0012830 con 0x7f8400066040 2023-11-06T03:09:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f00129b0 con 0x7f8400066040 2023-11-06T03:09:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8400157a30 con 0x7f8400066040 2023-11-06T03:09:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0012d60 con 0x7f8400066040 2023-11-06T03:09:22.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.264+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8400157ff0 con 0x7f8400066040 2023-11-06T03:09:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840006ad00 con 0x7f8400066040 2023-11-06T03:09:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f83f001c460 con 0x7f8400066040 2023-11-06T03:09:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f83fcff9640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 0x7f83e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f83f0083260 con 0x7f8400066040 2023-11-06T03:09:22.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f83f7fff640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 0x7f83e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.268+0000 7f83f7fff640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 0x7f83e0067f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f840015d1e0 tx=0x7f83e80079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.272+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f83f0050a10 con 0x7f8400066040 2023-11-06T03:09:22.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.416+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8400072750 con 0x7f8400066040 2023-11-06T03:09:22.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.416+0000 7f83fcff9640 1 -- 172.21.15.157:0/2495365443 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83f0054030 con 0x7f8400066040 2023-11-06T03:09:22.417 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:09:22.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 msgr2=0x7f83e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 0x7f83e0067f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f840015d1e0 tx=0x7f83e80079b0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 msgr2=0x7f84001f9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f83f000c480 tx=0x7f83f000c940 comp rx=0 tx=0).stop 2023-11-06T03:09:22.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 shutdown_connections 2023-11-06T03:09:22.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f83e0065a90 0x7f83e0067f50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8400157070 0x7f840015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f840006d2d0 0x7f840014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 --2- 172.21.15.157:0/2495365443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8400066040 0x7f84001f9e10 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 >> 172.21.15.157:0/2495365443 conn(0x7f8400069d70 msgr2=0x7f840015ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 shutdown_connections 2023-11-06T03:09:22.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.420+0000 7f840600a640 1 -- 172.21.15.157:0/2495365443 wait complete. 2023-11-06T03:09:22.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:09:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 Processor -- start 2023-11-06T03:09:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 -- start start 2023-11-06T03:09:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc6640cd730 0x7fc6640cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6640cc2b0 con 0x7fc66415ac80 2023-11-06T03:09:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640b01a0 con 0x7fc6640cd730 2023-11-06T03:09:22.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc66b56e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640cfb10 con 0x7fc6640b2b10 2023-11-06T03:09:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc669b0b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc6640cd730 0x7fc6640cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49524/0 (socket says 172.21.15.157:49524) 2023-11-06T03:09:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 -- 172.21.15.157:0/3794914985 learned_addr learned my addr 172.21.15.157:0/3794914985 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:22.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 -- 172.21.15.157:0/3794914985 >> v2:172.21.15.195:3300/0 conn(0x7fc6640cd730 msgr2=0x7fc6640cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:22.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 --2- 172.21.15.157:0/3794914985 >> v2:172.21.15.195:3300/0 conn(0x7fc6640cd730 0x7fc6640cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 -- 172.21.15.157:0/3794914985 >> v1:172.21.15.195:6789/0 conn(0x7fc66415ac80 legacy=0x7fc6640b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.560+0000 7fc668b09640 1 -- 172.21.15.157:0/3794914985 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640cfdf0 con 0x7fc6640b2b10 2023-11-06T03:09:22.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc668b09640 1 --2- 172.21.15.157:0/3794914985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc654009870 tx=0x7fc65401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650bc441bad5ef53 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc65bfff640 1 -- 172.21.15.157:0/3794914985 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a9400 con 0x7fc6640b2b10 2023-11-06T03:09:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc65bfff640 1 -- 172.21.15.157:0/3794914985 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6540a9560 con 0x7fc6640b2b10 2023-11-06T03:09:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc65bfff640 1 -- 172.21.15.157:0/3794914985 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540a98f0 con 0x7fc6640b2b10 2023-11-06T03:09:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- 172.21.15.157:0/3794914985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 msgr2=0x7fc6640ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- 172.21.15.157:0/3794914985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc654009870 tx=0x7fc65401c540 comp rx=0 tx=0).stop 2023-11-06T03:09:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- 172.21.15.157:0/3794914985 shutdown_connections 2023-11-06T03:09:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- 172.21.15.157:0/3794914985 >> v2:172.21.15.195:3300/0 conn(0x7fc6640cd730 0x7fc6640cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- 172.21.15.157:0/3794914985 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc6640ce550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- 172.21.15.157:0/3794914985 >> 172.21.15.157:0/3794914985 conn(0x7fc66406d560 msgr2=0x7fc66406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- 172.21.15.157:0/3794914985 shutdown_connections 2023-11-06T03:09:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- 172.21.15.157:0/3794914985 wait complete. 2023-11-06T03:09:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 Processor -- start 2023-11-06T03:09:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- start start 2023-11-06T03:09:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc664150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6640cd730 0x7fc664159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c7a50 con 0x7fc66415ac80 2023-11-06T03:09:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641580f0 con 0x7fc6640cd730 2023-11-06T03:09:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.564+0000 7fc66b56e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c7ca0 con 0x7fc6640b2b10 2023-11-06T03:09:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44584/0 (socket says 172.21.15.157:44584) 2023-11-06T03:09:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 -- 172.21.15.157:0/1518183163 learned_addr learned my addr 172.21.15.157:0/1518183163 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc668b09640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6640cd730 0x7fc664159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66930a640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc664150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 -- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 msgr2=0x7fc664150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc664150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 -- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6640cd730 msgr2=0x7fc664159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6640cd730 0x7fc664159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 -- 172.21.15.157:0/1518183163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540094d0 con 0x7fc66415ac80 2023-11-06T03:09:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc669b0b640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00efc0 tx=0x7fc65c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c009ca0 con 0x7fc66415ac80 2023-11-06T03:09:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00f040 con 0x7fc66415ac80 2023-11-06T03:09:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0146c0 con 0x7fc66415ac80 2023-11-06T03:09:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6640c7f80 con 0x7fc66415ac80 2023-11-06T03:09:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc664158620 con 0x7fc66415ac80 2023-11-06T03:09:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc664158810 con 0x7fc66415ac80 2023-11-06T03:09:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc65c014820 con 0x7fc66415ac80 2023-11-06T03:09:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 0x7fc648067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fc65c0842f0 con 0x7fc66415ac80 2023-11-06T03:09:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66930a640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 0x7fc648067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.568+0000 7fc66930a640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 0x7fc648067f20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc660007fd0 tx=0x7fc660009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.572+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc65c051aa0 con 0x7fc66415ac80 2023-11-06T03:09:22.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc664072750 con 0x7fc66415ac80 2023-11-06T03:09:22.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc659ffb640 1 -- 172.21.15.157:0/1518183163 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc65c0550c0 con 0x7fc66415ac80 2023-11-06T03:09:22.727 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:09:22.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 msgr2=0x7fc648067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 0x7fc648067f20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc660007fd0 tx=0x7fc660009040 comp rx=0 tx=0).stop 2023-11-06T03:09:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 msgr2=0x7fc66415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00efc0 tx=0x7fc65c00c490 comp rx=0 tx=0).stop 2023-11-06T03:09:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 shutdown_connections 2023-11-06T03:09:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc648065a60 0x7fc648067f20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc66415ac80 0x7fc66415b060 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6640cd730 0x7fc664159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 --2- 172.21.15.157:0/1518183163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6640b2b10 0x7fc664150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 >> 172.21.15.157:0/1518183163 conn(0x7fc66406d560 msgr2=0x7fc6640d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 shutdown_connections 2023-11-06T03:09:22.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.728+0000 7fc66b56e640 1 -- 172.21.15.157:0/1518183163 wait complete. 2023-11-06T03:09:22.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:22.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 Processor -- start 2023-11-06T03:09:22.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 -- start start 2023-11-06T03:09:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb94070d90 0x7fcb940c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb940ca380 con 0x7fcb9415abd0 2023-11-06T03:09:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb940af440 con 0x7fcb9415a7f0 2023-11-06T03:09:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9bd55640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9415e600 con 0x7fcb94070d90 2023-11-06T03:09:22.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb94070d90 0x7fcb940c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb94070d90 0x7fcb940c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44592/0 (socket says 172.21.15.157:44592) 2023-11-06T03:09:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb992f0640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcb9415abd0 0x7fcb9406c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60230/0 (socket says 172.21.15.157:60230) 2023-11-06T03:09:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 -- 172.21.15.157:0/1626212612 learned_addr learned my addr 172.21.15.157:0/1626212612 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 -- 172.21.15.157:0/1626212612 >> v1:172.21.15.195:6789/0 conn(0x7fcb9415a7f0 legacy=0x7fcb9415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:22.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 -- 172.21.15.157:0/1626212612 >> v1:172.21.15.157:6789/0 conn(0x7fcb9415abd0 legacy=0x7fcb9406c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb9a2f2640 1 -- 172.21.15.157:0/1626212612 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940ca5b0 con 0x7fcb94070d90 2023-11-06T03:09:22.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.864+0000 7fcb98aef640 1 -- 172.21.15.157:0/1626212612 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178762951 0 0) 0x7fcb940ca380 con 0x7fcb9415abd0 2023-11-06T03:09:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9a2f2640 1 --2- 172.21.15.157:0/1626212612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb940c9e40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009870 tx=0x7fcb8401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b5e71019d45266 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb98aef640 1 -- 172.21.15.157:0/1626212612 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840a89f0 con 0x7fcb94070d90 2023-11-06T03:09:22.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb98aef640 1 -- 172.21.15.157:0/1626212612 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb840a8b50 con 0x7fcb94070d90 2023-11-06T03:09:22.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb98aef640 1 -- 172.21.15.157:0/1626212612 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840b0440 con 0x7fcb94070d90 2023-11-06T03:09:22.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- 172.21.15.157:0/1626212612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 msgr2=0x7fcb940c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/1626212612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb940c9e40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009870 tx=0x7fcb8401ecf0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- 172.21.15.157:0/1626212612 shutdown_connections 2023-11-06T03:09:22.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/1626212612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb940c9e40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- 172.21.15.157:0/1626212612 >> 172.21.15.157:0/1626212612 conn(0x7fcb94069d70 msgr2=0x7fcb9406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:22.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- 172.21.15.157:0/1626212612 shutdown_connections 2023-11-06T03:09:22.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- 172.21.15.157:0/1626212612 wait complete. 2023-11-06T03:09:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 Processor -- start 2023-11-06T03:09:22.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- start start 2023-11-06T03:09:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb9415a7f0 0x7fcb9414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9415abd0 0x7fcb941f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb941f47e0 con 0x7fcb94070d90 2023-11-06T03:09:22.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb941f0200 con 0x7fcb9415a7f0 2023-11-06T03:09:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb9bd55640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb941f1150 con 0x7fcb9415abd0 2023-11-06T03:09:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44602/0 (socket says 172.21.15.157:44602) 2023-11-06T03:09:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 -- 172.21.15.157:0/2344694241 learned_addr learned my addr 172.21.15.157:0/2344694241 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:22.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 -- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9415abd0 msgr2=0x7fcb941f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9415abd0 0x7fcb941f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 -- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb9415a7f0 msgr2=0x7fcb9414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb9415a7f0 0x7fcb9414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 -- 172.21.15.157:0/2344694241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840094d0 con 0x7fcb94070d90 2023-11-06T03:09:22.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb99af1640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c7c0 tx=0x7fcb8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.868+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00aa50 con 0x7fcb94070d90 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00abb0 con 0x7fcb94070d90 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c012420 con 0x7fcb94070d90 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb941f4220 con 0x7fcb94070d90 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb941f0410 con 0x7fcb94070d90 2023-11-06T03:09:22.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb94072750 con 0x7fcb94070d90 2023-11-06T03:09:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcb8c012580 con 0x7fcb94070d90 2023-11-06T03:09:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb927fc640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 0x7fcb80067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:22.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fcb8c083510 con 0x7fcb94070d90 2023-11-06T03:09:22.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb992f0640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 0x7fcb80067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:22.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.872+0000 7fcb992f0640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 0x7fcb80067f70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcb940c82a0 tx=0x7fcb88007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:22.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:22.876+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcb8c050da0 con 0x7fcb94070d90 2023-11-06T03:09:23.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.052+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb94070b70 con 0x7fcb94070d90 2023-11-06T03:09:23.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.052+0000 7fcb927fc640 1 -- 172.21.15.157:0/2344694241 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fcb8c0543c0 con 0x7fcb94070d90 2023-11-06T03:09:23.053 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:23.053 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:05.306735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:23.053 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-11-06T03:09:23.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 msgr2=0x7fcb80067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 0x7fcb80067f70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcb940c82a0 tx=0x7fcb88007970 comp rx=0 tx=0).stop 2023-11-06T03:09:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 msgr2=0x7fcb941f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c7c0 tx=0x7fcb8c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:09:23.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 shutdown_connections 2023-11-06T03:09:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb80065ab0 0x7fcb80067f70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb9415abd0 0x7fcb941f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb9415a7f0 0x7fcb9414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 --2- 172.21.15.157:0/2344694241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb94070d90 0x7fcb941f0c10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 >> 172.21.15.157:0/2344694241 conn(0x7fcb94069d70 msgr2=0x7fcb940c4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 shutdown_connections 2023-11-06T03:09:23.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.056+0000 7fcb9bd55640 1 -- 172.21.15.157:0/2344694241 wait complete. 2023-11-06T03:09:23.070 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:23.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.176+0000 7f017011a640 1 Processor -- start 2023-11-06T03:09:23.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.176+0000 7f017011a640 1 -- start start 2023-11-06T03:09:23.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01681599c0 0x7f01680ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f0168159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016815e440 con 0x7f016806d000 2023-11-06T03:09:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016806c240 con 0x7f01681599c0 2023-11-06T03:09:23.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016806c520 con 0x7f01680b2af0 2023-11-06T03:09:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01681599c0 0x7f01680ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016d6b5640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f016806d000 0x7f01680b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60244/0 (socket says 172.21.15.157:60244) 2023-11-06T03:09:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016d6b5640 1 -- 172.21.15.157:0/3158585377 learned_addr learned my addr 172.21.15.157:0/3158585377 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:23.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016e6b7640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f0168159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 -- 172.21.15.157:0/3158585377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 msgr2=0x7f0168159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f0168159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 -- 172.21.15.157:0/3158585377 >> v1:172.21.15.157:6789/0 conn(0x7f016806d000 legacy=0x7f01680b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016ceb4640 1 -- 172.21.15.157:0/3158585377 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518088605 0 0) 0x7f016815e440 con 0x7f016806d000 2023-11-06T03:09:23.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 -- 172.21.15.157:0/3158585377 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016806c800 con 0x7f01681599c0 2023-11-06T03:09:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016deb6640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680ae000 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f01640097c0 tx=0x7f016401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1495859277ec16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016ceb4640 1 -- 172.21.15.157:0/3158585377 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640ab670 con 0x7f01681599c0 2023-11-06T03:09:23.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016ceb4640 1 -- 172.21.15.157:0/3158585377 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640ab7d0 con 0x7f01681599c0 2023-11-06T03:09:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f016ceb4640 1 -- 172.21.15.157:0/3158585377 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640abb60 con 0x7f01681599c0 2023-11-06T03:09:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 -- 172.21.15.157:0/3158585377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 msgr2=0x7f01680ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.180+0000 7f017011a640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680ae000 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f01640097c0 tx=0x7f016401c450 comp rx=0 tx=0).stop 2023-11-06T03:09:23.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/3158585377 shutdown_connections 2023-11-06T03:09:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f0168159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 --2- 172.21.15.157:0/3158585377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680ae000 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/3158585377 >> 172.21.15.157:0/3158585377 conn(0x7f0168065930 msgr2=0x7f0168065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/3158585377 shutdown_connections 2023-11-06T03:09:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/3158585377 wait complete. 2023-11-06T03:09:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 Processor -- start 2023-11-06T03:09:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- start start 2023-11-06T03:09:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016806d000 0x7f016814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f01680ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01680cbaa0 con 0x7f01681599c0 2023-11-06T03:09:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01680cc900 con 0x7f016806d000 2023-11-06T03:09:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01680cbcf0 con 0x7f01680b2af0 2023-11-06T03:09:23.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44614/0 (socket says 172.21.15.157:44614) 2023-11-06T03:09:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 -- 172.21.15.157:0/337041008 learned_addr learned my addr 172.21.15.157:0/337041008 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016d6b5640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f01680ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 -- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 msgr2=0x7f01680ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f01680ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016deb6640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016806d000 0x7f016814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 -- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016806d000 msgr2=0x7f016814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016806d000 0x7f016814b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 -- 172.21.15.157:0/337041008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164009420 con 0x7f01681599c0 2023-11-06T03:09:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016d6b5640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f01680ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f016e6b7640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f016000cd50 tx=0x7f016000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160003ba0 con 0x7f01681599c0 2023-11-06T03:09:23.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0160003d00 con 0x7f01681599c0 2023-11-06T03:09:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01680cbfd0 con 0x7f01681599c0 2023-11-06T03:09:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160004240 con 0x7f01681599c0 2023-11-06T03:09:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01680cc590 con 0x7f01681599c0 2023-11-06T03:09:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.184+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01680cca90 con 0x7f01681599c0 2023-11-06T03:09:23.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.188+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01600043a0 con 0x7f01681599c0 2023-11-06T03:09:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.188+0000 7f015e7fc640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 0x7f014c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.188+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f0160012070 con 0x7f01681599c0 2023-11-06T03:09:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.188+0000 7f016deb6640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 0x7f014c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.188+0000 7f016deb6640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 0x7f014c067f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f01640af000 tx=0x7f0164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.192+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0160051430 con 0x7f01681599c0 2023-11-06T03:09:23.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.344+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0168072750 con 0x7f01681599c0 2023-11-06T03:09:23.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.344+0000 7f015e7fc640 1 -- 172.21.15.157:0/337041008 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18375 (secure 0 0 0) 0x7f0160054a50 con 0x7f01681599c0 2023-11-06T03:09:23.346 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:23.346 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:02.175228+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.195:6836/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.157:6835/1218323165":"2023-11-07T03:09:02.146924+0000","172.21.15.157:6834/1218323165":"2023-11-07T03:09:02.146924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:23.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 msgr2=0x7f014c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 0x7f014c067f20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f01640af000 tx=0x7f0164008040 comp rx=0 tx=0).stop 2023-11-06T03:09:23.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 msgr2=0x7f01680acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f016000cd50 tx=0x7f016000a660 comp rx=0 tx=0).stop 2023-11-06T03:09:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 shutdown_connections 2023-11-06T03:09:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f014c065a60 0x7f014c067f20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01681599c0 0x7f01680acdf0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01680b2af0 0x7f01680ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 --2- 172.21.15.157:0/337041008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f016806d000 0x7f016814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 >> 172.21.15.157:0/337041008 conn(0x7f0168065930 msgr2=0x7f0168151990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 shutdown_connections 2023-11-06T03:09:23.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.348+0000 7f017011a640 1 -- 172.21.15.157:0/337041008 wait complete. 2023-11-06T03:09:23.364 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:09:23.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:23.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.468+0000 7f55759c6640 1 Processor -- start 2023-11-06T03:09:23.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 -- start start 2023-11-06T03:09:23.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5570146370 0x7f55700b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55700ce800 0x7f55700cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55700cf1a0 con 0x7f557006b6a0 2023-11-06T03:09:23.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557006c3d0 con 0x7f55700ce800 2023-11-06T03:09:23.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f55759c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557006f5a0 con 0x7f5570146370 2023-11-06T03:09:23.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55700ce800 0x7f55700cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f557006b6a0 0x7f557006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60250/0 (socket says 172.21.15.157:60250) 2023-11-06T03:09:23.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556f7fe640 1 -- 172.21.15.157:0/3313882975 learned_addr learned my addr 172.21.15.157:0/3313882975 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556effd640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5570146370 0x7f55700b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 -- 172.21.15.157:0/3313882975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5570146370 msgr2=0x7f55700b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5570146370 0x7f55700b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 -- 172.21.15.157:0/3313882975 >> v1:172.21.15.157:6789/0 conn(0x7f557006b6a0 legacy=0x7f557006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 -- 172.21.15.157:0/3313882975 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557006f880 con 0x7f55700ce800 2023-11-06T03:09:23.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556dffb640 1 -- 172.21.15.157:0/3313882975 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468344743 0 0) 0x7f55700cf1a0 con 0x7f557006b6a0 2023-11-06T03:09:23.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556e7fc640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55700ce800 0x7f55700cec60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5558009d90 tx=0x7f555801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c6301d6be957fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556dffb640 1 -- 172.21.15.157:0/3313882975 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55580a5980 con 0x7f55700ce800 2023-11-06T03:09:23.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556dffb640 1 -- 172.21.15.157:0/3313882975 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55580a5ae0 con 0x7f55700ce800 2023-11-06T03:09:23.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.472+0000 7f556dffb640 1 -- 172.21.15.157:0/3313882975 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55580a5e70 con 0x7f55700ce800 2023-11-06T03:09:23.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/3313882975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55700ce800 msgr2=0x7f55700cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55700ce800 0x7f55700cec60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5558009d90 tx=0x7f555801c8c0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/3313882975 shutdown_connections 2023-11-06T03:09:23.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55700ce800 0x7f55700cec60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- 172.21.15.157:0/3313882975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5570146370 0x7f55700b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/3313882975 >> 172.21.15.157:0/3313882975 conn(0x7f5570065930 msgr2=0x7f5570065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/3313882975 shutdown_connections 2023-11-06T03:09:23.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/3313882975 wait complete. 2023-11-06T03:09:23.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 Processor -- start 2023-11-06T03:09:23.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- start start 2023-11-06T03:09:23.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557006b6a0 0x7f5570141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55700ce800 0x7f55700ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55700aea70 con 0x7f5570146370 2023-11-06T03:09:23.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55700af8b0 con 0x7f557006b6a0 2023-11-06T03:09:23.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557013cc80 con 0x7f55700ce800 2023-11-06T03:09:23.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44634/0 (socket says 172.21.15.157:44634) 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 -- 172.21.15.157:0/2140327523 learned_addr learned my addr 172.21.15.157:0/2140327523 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 -- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55700ce800 msgr2=0x7f55700ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55700ce800 0x7f55700ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 -- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557006b6a0 msgr2=0x7f5570141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557006b6a0 0x7f5570141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 -- 172.21.15.157:0/2140327523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009a70 con 0x7f5570146370 2023-11-06T03:09:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f556f7fe640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f556800c3c0 tx=0x7f556800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55680074e0 con 0x7f5570146370 2023-11-06T03:09:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5568007660 con 0x7f5570146370 2023-11-06T03:09:23.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55700aeca0 con 0x7f5570146370 2023-11-06T03:09:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568004a30 con 0x7f5570146370 2023-11-06T03:09:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.476+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55700af260 con 0x7f5570146370 2023-11-06T03:09:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5570072750 con 0x7f5570146370 2023-11-06T03:09:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5568004b90 con 0x7f5570146370 2023-11-06T03:09:23.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f5567fff640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 0x7f5550067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f5568083230 con 0x7f5570146370 2023-11-06T03:09:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f556effd640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 0x7f5550067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.480+0000 7f556effd640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 0x7f5550067f20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f556000a810 tx=0x7f5560008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.484+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5568050950 con 0x7f5570146370 2023-11-06T03:09:23.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.656+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55700af9f0 con 0x7f5570146370 2023-11-06T03:09:23.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f5567fff640 1 -- 172.21.15.157:0/2140327523 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f5568053f70 con 0x7f5570146370 2023-11-06T03:09:23.658 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:23.659 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:05.306735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:23.659 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-11-06T03:09:23.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 msgr2=0x7f5550067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 0x7f5550067f20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f556000a810 tx=0x7f5560008040 comp rx=0 tx=0).stop 2023-11-06T03:09:23.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 msgr2=0x7f5570155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f556800c3c0 tx=0x7f556800c880 comp rx=0 tx=0).stop 2023-11-06T03:09:23.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 shutdown_connections 2023-11-06T03:09:23.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5550065a60 0x7f5550067f20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5570146370 0x7f5570155000 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55700ce800 0x7f55700ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 --2- 172.21.15.157:0/2140327523 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f557006b6a0 0x7f5570141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 >> 172.21.15.157:0/2140327523 conn(0x7f5570065930 msgr2=0x7f55700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 shutdown_connections 2023-11-06T03:09:23.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.660+0000 7f55759c6640 1 -- 172.21.15.157:0/2140327523 wait complete. 2023-11-06T03:09:23.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 Processor -- start 2023-11-06T03:09:23.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 -- start start 2023-11-06T03:09:23.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f042015e5e0 0x7f04200b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042015f580 con 0x7f042015ea90 2023-11-06T03:09:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042015efc0 con 0x7f04201551e0 2023-11-06T03:09:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f04278df640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04200704b0 con 0x7f042015e5e0 2023-11-06T03:09:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0425e7c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f042015ea90 0x7f042015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60262/0 (socket says 172.21.15.157:60262) 2023-11-06T03:09:23.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0425e7c640 1 -- 172.21.15.157:0/3401787262 learned_addr learned my addr 172.21.15.157:0/3401787262 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 --2- 172.21.15.157:0/3401787262 >> v2:172.21.15.195:3300/0 conn(0x7f042015e5e0 0x7f04200b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166616741 0 0) 0x7f042015f580 con 0x7f042015ea90 2023-11-06T03:09:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f040c003530 con 0x7f042015ea90 2023-11-06T03:09:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346282953 0 0) 0x7f040c003530 con 0x7f042015ea90 2023-11-06T03:09:23.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 >> v2:172.21.15.195:3300/0 conn(0x7f042015e5e0 msgr2=0x7f04200b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 --2- 172.21.15.157:0/3401787262 >> v2:172.21.15.195:3300/0 conn(0x7f042015e5e0 0x7f04200b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 >> v1:172.21.15.195:6789/0 conn(0x7f04201551e0 legacy=0x7f042006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042015f2a0 con 0x7f042015ea90 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0418001a00 con 0x7f042015ea90 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 >> v1:172.21.15.157:6789/0 conn(0x7f042015ea90 legacy=0x7f042015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f040c003980 0x7f040c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f040c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f040c005190 0x7f040c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200704b0 con 0x7f040c0047d0 2023-11-06T03:09:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c00c130 con 0x7f040c005190 2023-11-06T03:09:23.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c00c2f0 con 0x7f040c003980 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f040c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0425e7c640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f040c005190 0x7f040c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 -- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f040c003980 msgr2=0x7f040c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f042567b640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f040c003980 0x7f040c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f040c003980 0x7f040c003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 -- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f040c005190 msgr2=0x7f040c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:23.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f040c005190 0x7f040c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0425e7c640 1 -- 172.21.15.157:0/3401787262 reap_dead start 2023-11-06T03:09:23.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.788+0000 7f0424e7a640 1 -- 172.21.15.157:0/3401787262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c003530 con 0x7f040c0047d0 2023-11-06T03:09:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f0424e7a640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f040c004c50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ae0 tx=0x7f041001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0a163d5a4e4071 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100a55f0 con 0x7f040c0047d0 2023-11-06T03:09:23.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04100a5750 con 0x7f040c0047d0 2023-11-06T03:09:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f0417fff640 1 -- 172.21.15.157:0/3401787262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100a5b00 con 0x7f040c0047d0 2023-11-06T03:09:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 msgr2=0x7f040c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f040c004c50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ae0 tx=0x7f041001ca20 comp rx=0 tx=0).stop 2023-11-06T03:09:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- 172.21.15.157:0/3401787262 shutdown_connections 2023-11-06T03:09:23.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 --2- 172.21.15.157:0/3401787262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f040c004c50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- 172.21.15.157:0/3401787262 >> 172.21.15.157:0/3401787262 conn(0x7f0420065930 msgr2=0x7f0420065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- 172.21.15.157:0/3401787262 shutdown_connections 2023-11-06T03:09:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- 172.21.15.157:0/3401787262 wait complete. 2023-11-06T03:09:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 Processor -- start 2023-11-06T03:09:23.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.792+0000 7f04278df640 1 -- start start 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04200cdab0 0x7f04200cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 0x7f042014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200ce470 con 0x7f040c0047d0 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042014f1d0 con 0x7f04200cdab0 2023-11-06T03:09:23.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04201f5230 con 0x7f042014e810 2023-11-06T03:09:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44656/0 (socket says 172.21.15.157:44656) 2023-11-06T03:09:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 -- 172.21.15.157:0/161503758 learned_addr learned my addr 172.21.15.157:0/161503758 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0425e7c640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 0x7f042014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0424e7a640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04200cdab0 0x7f04200cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 -- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 msgr2=0x7f042014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 0x7f042014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 -- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04200cdab0 msgr2=0x7f04200cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04200cdab0 0x7f04200cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 -- 172.21.15.157:0/161503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420151b80 con 0x7f040c0047d0 2023-11-06T03:09:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0425e7c640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 0x7f042014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f042567b640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d7d0 tx=0x7f041c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c005270 con 0x7f040c0047d0 2023-11-06T03:09:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c0053d0 con 0x7f040c0047d0 2023-11-06T03:09:23.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04201f4900 con 0x7f040c0047d0 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c00b8b0 con 0x7f040c0047d0 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201f4ec0 con 0x7f040c0047d0 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0420072750 con 0x7f040c0047d0 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f041c00ba10 con 0x7f040c0047d0 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 0x7f0404067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:23.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f041c083340 con 0x7f040c0047d0 2023-11-06T03:09:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.796+0000 7f0424e7a640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 0x7f0404067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.800+0000 7f0424e7a640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 0x7f0404067f50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04100af000 tx=0x7f0410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:23.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.800+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f041c050a60 con 0x7f040c0047d0 2023-11-06T03:09:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.972+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0420071730 con 0x7f040c0047d0 2023-11-06T03:09:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f0415ffb640 1 -- 172.21.15.157:0/161503758 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f041c019310 con 0x7f040c0047d0 2023-11-06T03:09:23.974 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:23.974 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:08:43.305986+0000","modified":"2023-11-06T03:09:05.306735+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6833/1223634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1223634033},{"type":"v1","addr":"172.21.15.157:6833","nonce":1223634033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-06T03:09:23.974 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 54 2023-11-06T03:09:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 msgr2=0x7f0404067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 0x7f0404067f50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f04100af000 tx=0x7f0410008040 comp rx=0 tx=0).stop 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 msgr2=0x7f0420135360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d7d0 tx=0x7f041c00dc90 comp rx=0 tx=0).stop 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 shutdown_connections 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0404065a90 0x7f0404067f50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042014e810 0x7f042014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04200cdab0 0x7f04200cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 --2- 172.21.15.157:0/161503758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c0047d0 0x7f0420135360 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:23.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 >> 172.21.15.157:0/161503758 conn(0x7f0420065930 msgr2=0x7f04200afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:23.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 shutdown_connections 2023-11-06T03:09:23.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:23.976+0000 7f04278df640 1 -- 172.21.15.157:0/161503758 wait complete. 2023-11-06T03:09:23.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:24.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 Processor -- start 2023-11-06T03:09:24.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 -- start start 2023-11-06T03:09:24.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5cec0b4ab0 0x7f5cec06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec06b8d0 con 0x7f5cec15adc0 2023-11-06T03:09:24.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec0b4f40 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf44e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec06be90 con 0x7f5cec0ca570 2023-11-06T03:09:24.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2a84640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5cec0b4ab0 0x7f5cec06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf1a82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2283640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f5cec15adc0 0x7f5cec070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48644/0 (socket says 172.21.15.157:48644) 2023-11-06T03:09:24.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2283640 1 -- 172.21.15.157:0/1474160697 learned_addr learned my addr 172.21.15.157:0/1474160697 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:24.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2a84640 1 -- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 msgr2=0x7f5cec0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:24.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2a84640 1 --2- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2a84640 1 -- 172.21.15.157:0/1474160697 >> v1:172.21.15.195:6789/0 conn(0x7f5cec15adc0 legacy=0x7f5cec070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.112+0000 7f5cf2a84640 1 -- 172.21.15.157:0/1474160697 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec071880 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf2a84640 1 --2- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec06c320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097c0 tx=0x7f5ce401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15499ad351815320 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf1281640 1 -- 172.21.15.157:0/1474160697 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40ab600 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf1281640 1 -- 172.21.15.157:0/1474160697 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40a7050 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf1281640 1 -- 172.21.15.157:0/1474160697 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40ab980 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec0b4ab0 msgr2=0x7f5cec06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec06c320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097c0 tx=0x7f5ce401c330 comp rx=0 tx=0).stop 2023-11-06T03:09:24.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- 172.21.15.157:0/1474160697 shutdown_connections 2023-11-06T03:09:24.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec06c320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/1474160697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- 172.21.15.157:0/1474160697 >> 172.21.15.157:0/1474160697 conn(0x7f5cec0cb990 msgr2=0x7f5cec0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:24.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- 172.21.15.157:0/1474160697 shutdown_connections 2023-11-06T03:09:24.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- 172.21.15.157:0/1474160697 wait complete. 2023-11-06T03:09:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 Processor -- start 2023-11-06T03:09:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.116+0000 7f5cf44e7640 1 -- start start 2023-11-06T03:09:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec15adc0 0x7f5cec0c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0b01e0 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0b0410 con 0x7f5cec15adc0 2023-11-06T03:09:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c44d0 con 0x7f5cec0ca570 2023-11-06T03:09:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44672/0 (socket says 172.21.15.157:44672) 2023-11-06T03:09:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 -- 172.21.15.157:0/953452994 learned_addr learned my addr 172.21.15.157:0/953452994 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:24.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2a84640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec15adc0 0x7f5cec0c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf1a82640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 -- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 msgr2=0x7f5cec0afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 -- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec15adc0 msgr2=0x7f5cec0c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec15adc0 0x7f5cec0c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 -- 172.21.15.157:0/953452994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009420 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf2283640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cba0 tx=0x7f5cdc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00ad40 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc012400 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec0c47b0 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc00aea0 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec0c4d70 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec072750 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5cdc007070 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 0x7f5cd8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f5cdc0832d0 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.120+0000 7f5cf1a82640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 0x7f5cd8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.124+0000 7f5cf1a82640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 0x7f5cd8067f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0cc600 tx=0x7f5ce0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.124+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5cdc050b30 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.280+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cec0b30f0 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5ceaffd640 1 -- 172.21.15.157:0/953452994 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18375 (secure 0 0 0) 0x7f5cdc054150 con 0x7f5cec0b4ab0 2023-11-06T03:09:24.283 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:24.284 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:02.175228+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:08:41.869490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-06T03:08:42.287789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1e","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.195:6837/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.195:6836/3999445109":"2023-11-07T03:09:02.175205+0000","172.21.15.157:6835/1218323165":"2023-11-07T03:09:02.146924+0000","172.21.15.157:6834/1218323165":"2023-11-07T03:09:02.146924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:24.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 msgr2=0x7f5cd8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 0x7f5cd8067f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0cc600 tx=0x7f5ce0008040 comp rx=0 tx=0).stop 2023-11-06T03:09:24.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 msgr2=0x7f5cec0ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00cba0 tx=0x7f5cdc00a430 comp rx=0 tx=0).stop 2023-11-06T03:09:24.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 shutdown_connections 2023-11-06T03:09:24.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5cd8065a60 0x7f5cd8067f20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cec15adc0 0x7f5cec0c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cec0ca570 0x7f5cec0afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 --2- 172.21.15.157:0/953452994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cec0b4ab0 0x7f5cec0ae820 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 >> 172.21.15.157:0/953452994 conn(0x7f5cec0cb990 msgr2=0x7f5cec0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:24.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 shutdown_connections 2023-11-06T03:09:24.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.284+0000 7f5cf44e7640 1 -- 172.21.15.157:0/953452994 wait complete. 2023-11-06T03:09:24.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:09:24.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 Processor -- start 2023-11-06T03:09:24.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 -- start start 2023-11-06T03:09:24.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71cc1599c0 0x7f71cc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc15e440 con 0x7f71cc06d000 2023-11-06T03:09:24.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc06c240 con 0x7f71cc0b2af0 2023-11-06T03:09:24.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71d163a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc06c520 con 0x7f71cc1599c0 2023-11-06T03:09:24.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71caffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71cc1599c0 0x7f71cc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71ca7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f71cc06d000 0x7f71cc0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60276/0 (socket says 172.21.15.157:60276) 2023-11-06T03:09:24.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71ca7fc640 1 -- 172.21.15.157:0/53623520 learned_addr learned my addr 172.21.15.157:0/53623520 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:24.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71caffd640 1 -- 172.21.15.157:0/53623520 >> v1:172.21.15.195:6789/0 conn(0x7f71cc0b2af0 legacy=0x7f71cc159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:24.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71caffd640 1 -- 172.21.15.157:0/53623520 >> v1:172.21.15.157:6789/0 conn(0x7f71cc06d000 legacy=0x7f71cc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71c9ffb640 1 -- 172.21.15.157:0/53623520 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777330385 0 0) 0x7f71cc15e440 con 0x7f71cc06d000 2023-11-06T03:09:24.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.412+0000 7f71caffd640 1 -- 172.21.15.157:0/53623520 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc06c800 con 0x7f71cc1599c0 2023-11-06T03:09:24.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71caffd640 1 --2- 172.21.15.157:0/53623520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc1599c0 0x7f71cc0ae000 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097c0 tx=0x7f71bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b83282fd18f308 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71c9ffb640 1 -- 172.21.15.157:0/53623520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0a7070 con 0x7f71cc1599c0 2023-11-06T03:09:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71c9ffb640 1 -- 172.21.15.157:0/53623520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc0ab430 con 0x7f71cc1599c0 2023-11-06T03:09:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71c9ffb640 1 -- 172.21.15.157:0/53623520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0ab7c0 con 0x7f71cc1599c0 2023-11-06T03:09:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- 172.21.15.157:0/53623520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc1599c0 msgr2=0x7f71cc0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 --2- 172.21.15.157:0/53623520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc1599c0 0x7f71cc0ae000 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097c0 tx=0x7f71bc01c410 comp rx=0 tx=0).stop 2023-11-06T03:09:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- 172.21.15.157:0/53623520 shutdown_connections 2023-11-06T03:09:24.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 --2- 172.21.15.157:0/53623520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc1599c0 0x7f71cc0ae000 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- 172.21.15.157:0/53623520 >> 172.21.15.157:0/53623520 conn(0x7f71cc065930 msgr2=0x7f71cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:24.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- 172.21.15.157:0/53623520 shutdown_connections 2023-11-06T03:09:24.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- 172.21.15.157:0/53623520 wait complete. 2023-11-06T03:09:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 Processor -- start 2023-11-06T03:09:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.416+0000 7f71d163a640 1 -- start start 2023-11-06T03:09:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71cc06d000 0x7f71cc142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71cc0b2af0 0x7f71cc14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc14c2e0 con 0x7f71cc143b90 2023-11-06T03:09:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc14ce60 con 0x7f71cc0b2af0 2023-11-06T03:09:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc14c530 con 0x7f71cc06d000 2023-11-06T03:09:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44678/0 (socket says 172.21.15.157:44678) 2023-11-06T03:09:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 -- 172.21.15.157:0/1230937360 learned_addr learned my addr 172.21.15.157:0/1230937360 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71ca7fc640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71cc0b2af0 0x7f71cc14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71caffd640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71cc06d000 0x7f71cc142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 -- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71cc06d000 msgr2=0x7f71cc142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71cc06d000 0x7f71cc142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 -- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71cc0b2af0 msgr2=0x7f71cc14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71cc0b2af0 0x7f71cc14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 -- 172.21.15.157:0/1230937360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009420 con 0x7f71cc143b90 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71cb7fe640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f71c400ba10 tx=0x7f71c400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c400cae0 con 0x7f71cc143b90 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c400cc40 con 0x7f71cc143b90 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc14c810 con 0x7f71cc143b90 2023-11-06T03:09:24.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4014440 con 0x7f71cc143b90 2023-11-06T03:09:24.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc144970 con 0x7f71cc143b90 2023-11-06T03:09:24.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc072750 con 0x7f71cc143b90 2023-11-06T03:09:24.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f71c401d3b0 con 0x7f71cc143b90 2023-11-06T03:09:24.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 0x7f71b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6379+0+0 (secure 0 0 0) 0x7f71c4083160 con 0x7f71cc143b90 2023-11-06T03:09:24.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71caffd640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 0x7f71b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.420+0000 7f71caffd640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 0x7f71b8067f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0af000 tx=0x7f71bc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.424+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71c4050910 con 0x7f71cc143b90 2023-11-06T03:09:24.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.600+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f71cc14cfa0 con 0x7f71cc143b90 2023-11-06T03:09:24.601 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:09:24.600+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:09:24.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.612+0000 7f71b37fe640 1 -- 172.21.15.157:0/1230937360 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f71c4053f30 con 0x7f71cc143b90 2023-11-06T03:09:24.613 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:09:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 msgr2=0x7f71b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 0x7f71b8067f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0af000 tx=0x7f71bc0a7000 comp rx=0 tx=0).stop 2023-11-06T03:09:24.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 msgr2=0x7f71cc144040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f71c400ba10 tx=0x7f71c400bed0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 shutdown_connections 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71b8065a60 0x7f71b8067f20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71cc143b90 0x7f71cc144040 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71cc0b2af0 0x7f71cc14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 --2- 172.21.15.157:0/1230937360 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71cc06d000 0x7f71cc142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 >> 172.21.15.157:0/1230937360 conn(0x7f71cc065930 msgr2=0x7f71cc1513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:24.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 shutdown_connections 2023-11-06T03:09:24.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.616+0000 7f71d163a640 1 -- 172.21.15.157:0/1230937360 wait complete. 2023-11-06T03:09:24.623 INFO:tasks.ceph.mds.a.smithi157.stderr: -1719> 2023-11-06T03:09:08.300+0000 7ff34d987640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:09:24.631 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:09:24.652 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:09:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.748+0000 7fa97f32e640 1 Processor -- start 2023-11-06T03:09:24.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.748+0000 7fa97f32e640 1 -- start start 2023-11-06T03:09:24.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97f32e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 0x7fa9780cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97f32e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 0x7fa978160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97f32e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9781612f0 con 0x7fa978160950 2023-11-06T03:09:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97f32e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa978161780 con 0x7fa978160490 2023-11-06T03:09:24.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97f32e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780d0450 con 0x7fa97815f810 2023-11-06T03:09:24.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d0ca640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 0x7fa9780cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d8cb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 0x7fa978160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97c8c9640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa978160950 0x7fa97815d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48674/0 (socket says 172.21.15.157:48674) 2023-11-06T03:09:24.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97c8c9640 1 -- 172.21.15.157:0/1863633478 learned_addr learned my addr 172.21.15.157:0/1863633478 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d8cb640 1 -- 172.21.15.157:0/1863633478 >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 msgr2=0x7fa9780cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d8cb640 1 --2- 172.21.15.157:0/1863633478 >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 0x7fa9780cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d8cb640 1 -- 172.21.15.157:0/1863633478 >> v1:172.21.15.195:6789/0 conn(0x7fa978160950 legacy=0x7fa97815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d8cb640 1 -- 172.21.15.157:0/1863633478 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780cd860 con 0x7fa97815f810 2023-11-06T03:09:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.752+0000 7fa97d0ca640 1 --2- 172.21.15.157:0/1863633478 >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 0x7fa9780cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:24.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97d8cb640 1 --2- 172.21.15.157:0/1863633478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 0x7fa978160db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa970009d90 tx=0x7fa97001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7d22e18f061b3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa96ffff640 1 -- 172.21.15.157:0/1863633478 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97815e280 con 0x7fa97815f810 2023-11-06T03:09:24.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa96ffff640 1 -- 172.21.15.157:0/1863633478 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9700a59e0 con 0x7fa97815f810 2023-11-06T03:09:24.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa96ffff640 1 -- 172.21.15.157:0/1863633478 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9700a5d70 con 0x7fa97815f810 2023-11-06T03:09:24.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- 172.21.15.157:0/1863633478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 msgr2=0x7fa978160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- 172.21.15.157:0/1863633478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 0x7fa978160db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa970009d90 tx=0x7fa97001c920 comp rx=0 tx=0).stop 2023-11-06T03:09:24.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- 172.21.15.157:0/1863633478 shutdown_connections 2023-11-06T03:09:24.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- 172.21.15.157:0/1863633478 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa97815f810 0x7fa978160db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- 172.21.15.157:0/1863633478 >> v2:172.21.15.195:3300/0 conn(0x7fa978160490 0x7fa9780cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- 172.21.15.157:0/1863633478 >> 172.21.15.157:0/1863633478 conn(0x7fa978070790 msgr2=0x7fa978070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:24.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- 172.21.15.157:0/1863633478 shutdown_connections 2023-11-06T03:09:24.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- 172.21.15.157:0/1863633478 wait complete. 2023-11-06T03:09:24.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 Processor -- start 2023-11-06T03:09:24.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- start start 2023-11-06T03:09:24.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa978160490 0x7fa9780cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 0x7fa978152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780d2020 con 0x7fa97815f810 2023-11-06T03:09:24.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780c9ff0 con 0x7fa978160950 2023-11-06T03:09:24.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97f32e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780ca2d0 con 0x7fa978160490 2023-11-06T03:09:24.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97d0ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97d0ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44708/0 (socket says 172.21.15.157:44708) 2023-11-06T03:09:24.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.756+0000 7fa97d0ca640 1 -- 172.21.15.157:0/89307721 learned_addr learned my addr 172.21.15.157:0/89307721 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:24.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d8cb640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 0x7fa978152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97c8c9640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa978160490 0x7fa9780cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 -- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa978160490 msgr2=0x7fa9780cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa978160490 0x7fa9780cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 -- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 msgr2=0x7fa978152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:24.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 0x7fa978152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:24.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 -- 172.21.15.157:0/89307721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa974009420 con 0x7fa97815f810 2023-11-06T03:09:24.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d8cb640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 0x7fa978152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:24.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97d0ca640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e980 tx=0x7fa97400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400ca00 con 0x7fa97815f810 2023-11-06T03:09:24.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97400cb60 con 0x7fa97815f810 2023-11-06T03:09:24.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974013470 con 0x7fa97815f810 2023-11-06T03:09:24.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa970009a70 con 0x7fa97815f810 2023-11-06T03:09:24.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9780ca890 con 0x7fa97815f810 2023-11-06T03:09:24.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa978072750 con 0x7fa97815f810 2023-11-06T03:09:24.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa9740135d0 con 0x7fa97815f810 2023-11-06T03:09:24.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 0x7fa95c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:24.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6465+0+0 (secure 0 0 0) 0x7fa9740836c0 con 0x7fa97815f810 2023-11-06T03:09:24.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97c8c9640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 0x7fa95c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:24.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.760+0000 7fa97c8c9640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 0x7fa95c067f20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d35f0 tx=0x7fa968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:24.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.764+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa974050e10 con 0x7fa97815f810 2023-11-06T03:09:24.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:24.940+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9780ce690 con 0x7fa97815f810 2023-11-06T03:09:25.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.620+0000 7fa96dffb640 1 -- 172.21.15.157:0/89307721 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa974054430 con 0x7fa97815f810 2023-11-06T03:09:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 msgr2=0x7fa95c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 0x7fa95c067f20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa9780d35f0 tx=0x7fa968009040 comp rx=0 tx=0).stop 2023-11-06T03:09:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 msgr2=0x7fa9780d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e980 tx=0x7fa97400ee40 comp rx=0 tx=0).stop 2023-11-06T03:09:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 shutdown_connections 2023-11-06T03:09:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa95c065a60 0x7fa95c067f20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa978160950 0x7fa978152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa978160490 0x7fa9780cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 --2- 172.21.15.157:0/89307721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa97815f810 0x7fa9780d1ae0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 >> 172.21.15.157:0/89307721 conn(0x7fa978070790 msgr2=0x7fa9780b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 shutdown_connections 2023-11-06T03:09:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.624+0000 7fa97f32e640 1 -- 172.21.15.157:0/89307721 wait complete. 2023-11-06T03:09:25.647 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:09:25.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 Processor -- start 2023-11-06T03:09:25.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 -- start start 2023-11-06T03:09:25.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f85bc0cd9c0 0x7f85bc0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0ce250 con 0x7f85bc0ce850 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0cdf70 con 0x7f85bc15afb0 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85c1a6b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0cbdc0 con 0x7f85bc0cd9c0 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85bbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f85bc0ce850 0x7f85bc0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60292/0 (socket says 172.21.15.157:60292) 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85bbfff640 1 -- 172.21.15.157:0/2300809458 learned_addr learned my addr 172.21.15.157:0/2300809458 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:25.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.828+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061937640 0 0) 0x7f85bc0ce250 con 0x7f85bc0ce850 2023-11-06T03:09:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a4003530 con 0x7f85bc0ce850 2023-11-06T03:09:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 --2- 172.21.15.157:0/2300809458 >> v2:172.21.15.195:3300/0 conn(0x7f85bc0cd9c0 0x7f85bc0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544017345 0 0) 0x7f85a4003530 con 0x7f85bc0ce850 2023-11-06T03:09:25.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 >> v2:172.21.15.195:3300/0 conn(0x7f85bc0cd9c0 msgr2=0x7f85bc0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 --2- 172.21.15.157:0/2300809458 >> v2:172.21.15.195:3300/0 conn(0x7f85bc0cd9c0 0x7f85bc0d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 >> v1:172.21.15.195:6789/0 conn(0x7f85bc15afb0 legacy=0x7f85bc0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0cb700 con 0x7f85bc0ce850 2023-11-06T03:09:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f85b4001a00 con 0x7f85bc0ce850 2023-11-06T03:09:25.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 >> v1:172.21.15.157:6789/0 conn(0x7f85bc0ce850 legacy=0x7f85bc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a4003980 0x7f85a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85a40052b0 0x7f85a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0cbdc0 con 0x7f85a40047d0 2023-11-06T03:09:25.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a400c180 con 0x7f85a4003980 2023-11-06T03:09:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a400c340 con 0x7f85a40052b0 2023-11-06T03:09:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85bb7fe640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a4003980 0x7f85a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85bbfff640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85a40052b0 0x7f85a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 -- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85a40052b0 msgr2=0x7f85a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:25.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85a40052b0 0x7f85a400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 -- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a4003980 msgr2=0x7f85a4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:25.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85a4003980 0x7f85a4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85bbfff640 1 -- 172.21.15.157:0/2300809458 reap_dead start 2023-11-06T03:09:25.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 -- 172.21.15.157:0/2300809458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a4003530 con 0x7f85a40047d0 2023-11-06T03:09:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85baffd640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85a4004c50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f85b000ba00 tx=0x7f85b001c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a939aaa56e8bec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00ad3f0 con 0x7f85a40047d0 2023-11-06T03:09:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b00ad550 con 0x7f85a40047d0 2023-11-06T03:09:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.832+0000 7f85ba7fc640 1 -- 172.21.15.157:0/2300809458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00ad900 con 0x7f85a40047d0 2023-11-06T03:09:25.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 msgr2=0x7f85a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85a4004c50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f85b000ba00 tx=0x7f85b001c5f0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/2300809458 shutdown_connections 2023-11-06T03:09:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/2300809458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85a4004c50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/2300809458 >> 172.21.15.157:0/2300809458 conn(0x7f85bc0d0950 msgr2=0x7f85bc0d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:25.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/2300809458 shutdown_connections 2023-11-06T03:09:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/2300809458 wait complete. 2023-11-06T03:09:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 Processor -- start 2023-11-06T03:09:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- start start 2023-11-06T03:09:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85bc0b46b0 0x7f85bc159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85bc0b0ac0 0x7f85bc0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b1480 con 0x7f85a40047d0 2023-11-06T03:09:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc1596c0 con 0x7f85bc0b46b0 2023-11-06T03:09:25.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0c8780 con 0x7f85bc0b0ac0 2023-11-06T03:09:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44734/0 (socket says 172.21.15.157:44734) 2023-11-06T03:09:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 -- 172.21.15.157:0/626440454 learned_addr learned my addr 172.21.15.157:0/626440454 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85baffd640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85bc0b46b0 0x7f85bc159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 -- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85bc0b0ac0 msgr2=0x7f85bc0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85bc0b0ac0 0x7f85bc0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 -- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85bc0b46b0 msgr2=0x7f85bc159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85bc0b46b0 0x7f85bc159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 -- 172.21.15.157:0/626440454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0c8a60 con 0x7f85a40047d0 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85bb7fe640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e3f0 tx=0x7f85ac00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:25.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0052e0 con 0x7f85a40047d0 2023-11-06T03:09:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac005440 con 0x7f85a40047d0 2023-11-06T03:09:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0c7bc0 con 0x7f85a40047d0 2023-11-06T03:09:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac003820 con 0x7f85a40047d0 2023-11-06T03:09:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.836+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc0b4b30 con 0x7f85a40047d0 2023-11-06T03:09:25.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc0c7a00 con 0x7f85a40047d0 2023-11-06T03:09:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f85ac003980 con 0x7f85a40047d0 2023-11-06T03:09:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85a3fff640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 0x7f85a8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f85ac082bc0 con 0x7f85a40047d0 2023-11-06T03:09:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85baffd640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 0x7f85a8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:25.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.840+0000 7f85baffd640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 0x7f85a8067f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f85b00af000 tx=0x7f85b00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:25.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.844+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f85ac0503c0 con 0x7f85a40047d0 2023-11-06T03:09:25.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:25.984+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85bc08b080 con 0x7f85a40047d0 2023-11-06T03:09:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.636+0000 7f85a3fff640 1 -- 172.21.15.157:0/626440454 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 181+0+0 (secure 0 0 0) 0x7f85ac0190d0 con 0x7f85a40047d0 2023-11-06T03:09:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:09:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 msgr2=0x7f85a8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 0x7f85a8067f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f85b00af000 tx=0x7f85b00046d0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 msgr2=0x7f85bc0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e3f0 tx=0x7f85ac00e8b0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 shutdown_connections 2023-11-06T03:09:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85a8065a90 0x7f85a8067f50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85bc0b0ac0 0x7f85bc0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85bc0b46b0 0x7f85bc159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 --2- 172.21.15.157:0/626440454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a40047d0 0x7f85bc0caa00 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 >> 172.21.15.157:0/626440454 conn(0x7f85bc0d0950 msgr2=0x7f85bc0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:26.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 shutdown_connections 2023-11-06T03:09:26.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.640+0000 7f85c1a6b640 1 -- 172.21.15.157:0/626440454 wait complete. 2023-11-06T03:09:26.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:09:26.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.788+0000 7f3885883640 1 Processor -- start 2023-11-06T03:09:26.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.788+0000 7f3885883640 1 -- start start 2023-11-06T03:09:26.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f3885883640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3880155310 0x7f388015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f3885883640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f388006c550 0x7f388015b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f3885883640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38800cb100 con 0x7f388006c170 2023-11-06T03:09:26.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f3885883640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800ce0a0 con 0x7f3880155310 2023-11-06T03:09:26.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f3885883640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388015bef0 con 0x7f388006c550 2023-11-06T03:09:26.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3880155310 0x7f388015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:26.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f388006c170 0x7f3880154c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60304/0 (socket says 172.21.15.157:60304) 2023-11-06T03:09:26.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387effd640 1 -- 172.21.15.157:0/3064134570 learned_addr learned my addr 172.21.15.157:0/3064134570 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:26.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387f7fe640 1 --2- 172.21.15.157:0/3064134570 >> v2:172.21.15.195:3300/0 conn(0x7f388006c550 0x7f388015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:26.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 -- 172.21.15.157:0/3064134570 >> v2:172.21.15.195:3300/0 conn(0x7f388006c550 msgr2=0x7f388015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:26.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 --2- 172.21.15.157:0/3064134570 >> v2:172.21.15.195:3300/0 conn(0x7f388006c550 0x7f388015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 -- 172.21.15.157:0/3064134570 >> v1:172.21.15.157:6789/0 conn(0x7f388006c170 legacy=0x7f3880154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387dffb640 1 -- 172.21.15.157:0/3064134570 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792625696 0 0) 0x7f38800cb100 con 0x7f388006c170 2023-11-06T03:09:26.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 -- 172.21.15.157:0/3064134570 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800cb350 con 0x7f3880155310 2023-11-06T03:09:26.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387e7fc640 1 --2- 172.21.15.157:0/3064134570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f388015b150 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f388006f3c0 tx=0x7f387401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c246a76d501c8446 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:26.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387dffb640 1 -- 172.21.15.157:0/3064134570 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740a7070 con 0x7f3880155310 2023-11-06T03:09:26.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387dffb640 1 -- 172.21.15.157:0/3064134570 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38740ab430 con 0x7f3880155310 2023-11-06T03:09:26.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.792+0000 7f387dffb640 1 -- 172.21.15.157:0/3064134570 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740ab7c0 con 0x7f3880155310 2023-11-06T03:09:26.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/3064134570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 msgr2=0x7f388015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- 172.21.15.157:0/3064134570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f388015b150 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f388006f3c0 tx=0x7f387401c430 comp rx=0 tx=0).stop 2023-11-06T03:09:26.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/3064134570 shutdown_connections 2023-11-06T03:09:26.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- 172.21.15.157:0/3064134570 >> v2:172.21.15.195:3300/0 conn(0x7f388006c550 0x7f388015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- 172.21.15.157:0/3064134570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f388015b150 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/3064134570 >> 172.21.15.157:0/3064134570 conn(0x7f38800b43b0 msgr2=0x7f3880065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:26.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/3064134570 shutdown_connections 2023-11-06T03:09:26.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/3064134570 wait complete. 2023-11-06T03:09:26.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 Processor -- start 2023-11-06T03:09:26.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- start start 2023-11-06T03:09:26.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f388006c170 0x7f388014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388006c550 0x7f388014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388014ea20 con 0x7f3880155310 2023-11-06T03:09:26.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388014ed00 con 0x7f388006c550 2023-11-06T03:09:26.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388014efe0 con 0x7f388006c170 2023-11-06T03:09:26.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:26.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44754/0 (socket says 172.21.15.157:44754) 2023-11-06T03:09:26.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 -- 172.21.15.157:0/1079280085 learned_addr learned my addr 172.21.15.157:0/1079280085 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:26.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 -- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f388006c170 msgr2=0x7f388014a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f388006c170 0x7f388014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 -- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388006c550 msgr2=0x7f388014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:26.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388006c550 0x7f388014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:26.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 -- 172.21.15.157:0/1079280085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009420 con 0x7f3880155310 2023-11-06T03:09:26.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f387f7fe640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f387800cd70 tx=0x7f387800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:26.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878003ba0 con 0x7f3880155310 2023-11-06T03:09:26.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878003d00 con 0x7f3880155310 2023-11-06T03:09:26.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38800cbfb0 con 0x7f3880155310 2023-11-06T03:09:26.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878004240 con 0x7f3880155310 2023-11-06T03:09:26.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.796+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388014f5a0 con 0x7f3880155310 2023-11-06T03:09:26.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388014f790 con 0x7f3880155310 2023-11-06T03:09:26.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38780043a0 con 0x7f3880155310 2023-11-06T03:09:26.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f3867fff640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 0x7f386c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:26.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5998+0+0 (secure 0 0 0) 0x7f3878012070 con 0x7f3880155310 2023-11-06T03:09:26.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f387effd640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 0x7f386c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:26.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.800+0000 7f387effd640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 0x7f386c067f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f38700011f0 tx=0x7f3870007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:26.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.804+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3878051510 con 0x7f3880155310 2023-11-06T03:09:26.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:26.956+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f388004e050 con 0x7f3880155310 2023-11-06T03:09:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.648+0000 7f3867fff640 1 -- 172.21.15.157:0/1079280085 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3878054b30 con 0x7f3880155310 2023-11-06T03:09:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:09:27.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 msgr2=0x7f386c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 0x7f386c067f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f38700011f0 tx=0x7f3870007930 comp rx=0 tx=0).stop 2023-11-06T03:09:27.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 msgr2=0x7f3880159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f387800cd70 tx=0x7f387800a660 comp rx=0 tx=0).stop 2023-11-06T03:09:27.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 shutdown_connections 2023-11-06T03:09:27.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f386c065a60 0x7f386c067f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3880155310 0x7f3880159b80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f388006c550 0x7f388014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 --2- 172.21.15.157:0/1079280085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f388006c170 0x7f388014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 >> 172.21.15.157:0/1079280085 conn(0x7f38800b43b0 msgr2=0x7f3880154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:27.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.652+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 shutdown_connections 2023-11-06T03:09:27.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.656+0000 7f3885883640 1 -- 172.21.15.157:0/1079280085 wait complete. 2023-11-06T03:09:27.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:09:27.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 Processor -- start 2023-11-06T03:09:27.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 -- start start 2023-11-06T03:09:27.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e406c490 con 0x7f29e4156a70 2023-11-06T03:09:27.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e406b950 con 0x7f29e40adf40 2023-11-06T03:09:27.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.808+0000 7f29ebb64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e406fb80 con 0x7f29e40adae0 2023-11-06T03:09:27.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ea101640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e9900640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f29e4156a70 0x7f29e406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48692/0 (socket says 172.21.15.157:48692) 2023-11-06T03:09:27.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ea101640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:49750/0 (socket says 172.21.15.157:49750) 2023-11-06T03:09:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49708/0 (socket says 172.21.15.157:49708) 2023-11-06T03:09:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ea101640 1 -- 172.21.15.157:0/1475570645 learned_addr learned my addr 172.21.15.157:0/1475570645 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 -- 172.21.15.157:0/1475570645 >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 msgr2=0x7f29e40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 --2- 172.21.15.157:0/1475570645 >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 -- 172.21.15.157:0/1475570645 >> v1:172.21.15.195:6789/0 conn(0x7f29e4156a70 legacy=0x7f29e406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 -- 172.21.15.157:0/1475570645 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e406bb60 con 0x7f29e40adae0 2023-11-06T03:09:27.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ea101640 1 --2- 172.21.15.157:0/1475570645 >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:27.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e90ff640 1 --2- 172.21.15.157:0/1475570645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097c0 tx=0x7f29d401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ed445ac8916416 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:27.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e88fe640 1 -- 172.21.15.157:0/1475570645 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40ac630 con 0x7f29e40adae0 2023-11-06T03:09:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e88fe640 1 -- 172.21.15.157:0/1475570645 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d40ac790 con 0x7f29e40adae0 2023-11-06T03:09:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29e88fe640 1 -- 172.21.15.157:0/1475570645 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40acb20 con 0x7f29e40adae0 2023-11-06T03:09:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 -- 172.21.15.157:0/1475570645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 msgr2=0x7f29e40b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 --2- 172.21.15.157:0/1475570645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097c0 tx=0x7f29d401c450 comp rx=0 tx=0).stop 2023-11-06T03:09:27.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 -- 172.21.15.157:0/1475570645 shutdown_connections 2023-11-06T03:09:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 --2- 172.21.15.157:0/1475570645 >> v2:172.21.15.195:3300/0 conn(0x7f29e40adf40 0x7f29e40b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 --2- 172.21.15.157:0/1475570645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b1c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 -- 172.21.15.157:0/1475570645 >> 172.21.15.157:0/1475570645 conn(0x7f29e4065930 msgr2=0x7f29e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 -- 172.21.15.157:0/1475570645 shutdown_connections 2023-11-06T03:09:27.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.812+0000 7f29ebb64640 1 -- 172.21.15.157:0/1475570645 wait complete. 2023-11-06T03:09:27.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 Processor -- start 2023-11-06T03:09:27.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- start start 2023-11-06T03:09:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e40adf40 0x7f29e4151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b0020 con 0x7f29e4156a70 2023-11-06T03:09:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b0300 con 0x7f29e40adf40 2023-11-06T03:09:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b05e0 con 0x7f29e40adae0 2023-11-06T03:09:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44764/0 (socket says 172.21.15.157:44764) 2023-11-06T03:09:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 -- 172.21.15.157:0/269610892 learned_addr learned my addr 172.21.15.157:0/269610892 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29e9900640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29e90ff640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e40adf40 0x7f29e4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 -- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 msgr2=0x7f29e40b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 -- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e40adf40 msgr2=0x7f29e4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e40adf40 0x7f29e4151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 -- 172.21.15.157:0/269610892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc00ac40 con 0x7f29e4156a70 2023-11-06T03:09:27.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ea101640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ec0 tx=0x7f29dc0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:27.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc008260 con 0x7f29e4156a70 2023-11-06T03:09:27.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc010040 con 0x7f29e4156a70 2023-11-06T03:09:27.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0147f0 con 0x7f29e4156a70 2023-11-06T03:09:27.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d4009420 con 0x7f29e4156a70 2023-11-06T03:09:27.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.816+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e40b0ba0 con 0x7f29e4156a70 2023-11-06T03:09:27.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e40b0d90 con 0x7f29e4156a70 2023-11-06T03:09:27.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29dc014950 con 0x7f29e4156a70 2023-11-06T03:09:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29da7fc640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 0x7f29c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f29dc083860 con 0x7f29e4156a70 2023-11-06T03:09:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29e9900640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 0x7f29c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.820+0000 7f29e9900640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 0x7f29c8067f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f29e0000f70 tx=0x7f29e0007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:27.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.824+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f29dc051450 con 0x7f29e4156a70 2023-11-06T03:09:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.976+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f29e4072750 con 0x7f29e4156a70 2023-11-06T03:09:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29da7fc640 1 -- 172.21.15.157:0/269610892 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29dc054a70 con 0x7f29e4156a70 2023-11-06T03:09:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 msgr2=0x7f29c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 0x7f29c8067f20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f29e0000f70 tx=0x7f29e0007e50 comp rx=0 tx=0).stop 2023-11-06T03:09:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 msgr2=0x7f29e4151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ec0 tx=0x7f29dc0075c0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 shutdown_connections 2023-11-06T03:09:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29c8065a60 0x7f29c8067f20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e4156a70 0x7f29e4151890 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e40adf40 0x7f29e4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 --2- 172.21.15.157:0/269610892 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40adae0 0x7f29e40b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 >> 172.21.15.157:0/269610892 conn(0x7f29e4065930 msgr2=0x7f29e4070790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 shutdown_connections 2023-11-06T03:09:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:27.984+0000 7f29ebb64640 1 -- 172.21.15.157:0/269610892 wait complete. 2023-11-06T03:09:28.001 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2023-11-06T03:09:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.112+0000 7fcf3ea3c640 1 Processor -- start 2023-11-06T03:09:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.112+0000 7fcf3ea3c640 1 -- start start 2023-11-06T03:09:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3ea3c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 0x7fcf380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3ea3c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcf3806d000 0x7fcf38159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3ea3c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815e440 con 0x7fcf381599c0 2023-11-06T03:09:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3ea3c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3806c240 con 0x7fcf3806d000 2023-11-06T03:09:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3ea3c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3806c520 con 0x7fcf380b2af0 2023-11-06T03:09:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf37fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcf381599c0 0x7fcf380b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60310/0 (socket says 172.21.15.157:60310) 2023-11-06T03:09:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf37fff640 1 -- 172.21.15.157:0/1279910752 learned_addr learned my addr 172.21.15.157:0/1279910752 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 --2- 172.21.15.157:0/1279910752 >> v2:172.21.15.157:3300/0 conn(0x7fcf3806d000 0x7fcf38159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf36ffd640 1 -- 172.21.15.157:0/1279910752 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393513086 0 0) 0x7fcf3815e440 con 0x7fcf381599c0 2023-11-06T03:09:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf377fe640 1 --2- 172.21.15.157:0/1279910752 >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 0x7fcf380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf36ffd640 1 -- 172.21.15.157:0/1279910752 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003530 con 0x7fcf381599c0 2023-11-06T03:09:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 -- 172.21.15.157:0/1279910752 >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 msgr2=0x7fcf380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 --2- 172.21.15.157:0/1279910752 >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 0x7fcf380ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 -- 172.21.15.157:0/1279910752 >> v1:172.21.15.157:6789/0 conn(0x7fcf381599c0 legacy=0x7fcf380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 -- 172.21.15.157:0/1279910752 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3806c800 con 0x7fcf3806d000 2023-11-06T03:09:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf377fe640 1 --2- 172.21.15.157:0/1279910752 >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 0x7fcf380ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf3cfd9640 1 --2- 172.21.15.157:0/1279910752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3806d000 0x7fcf38159da0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcf380cfa20 tx=0x7fcf3001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85a0bfffe305ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf36ffd640 1 -- 172.21.15.157:0/1279910752 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300ab650 con 0x7fcf3806d000 2023-11-06T03:09:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf36ffd640 1 -- 172.21.15.157:0/1279910752 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf300ab7b0 con 0x7fcf3806d000 2023-11-06T03:09:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.116+0000 7fcf36ffd640 1 -- 172.21.15.157:0/1279910752 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300abb40 con 0x7fcf3806d000 2023-11-06T03:09:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1279910752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3806d000 msgr2=0x7fcf38159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1279910752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3806d000 0x7fcf38159da0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcf380cfa20 tx=0x7fcf3001c430 comp rx=0 tx=0).stop 2023-11-06T03:09:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1279910752 shutdown_connections 2023-11-06T03:09:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1279910752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf3806d000 0x7fcf38159da0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1279910752 >> v2:172.21.15.195:3300/0 conn(0x7fcf380b2af0 0x7fcf380ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1279910752 >> 172.21.15.157:0/1279910752 conn(0x7fcf38065930 msgr2=0x7fcf38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1279910752 shutdown_connections 2023-11-06T03:09:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1279910752 wait complete. 2023-11-06T03:09:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 Processor -- start 2023-11-06T03:09:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- start start 2023-11-06T03:09:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf3806d000 0x7fcf381f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 0x7fcf38152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38150a90 con 0x7fcf380b2af0 2023-11-06T03:09:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381530c0 con 0x7fcf381599c0 2023-11-06T03:09:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381f5720 con 0x7fcf3806d000 2023-11-06T03:09:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44772/0 (socket says 172.21.15.157:44772) 2023-11-06T03:09:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 -- 172.21.15.157:0/1362871773 learned_addr learned my addr 172.21.15.157:0/1362871773 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3cfd9640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 0x7fcf38152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf37fff640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf3806d000 0x7fcf381f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 -- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf3806d000 msgr2=0x7fcf381f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf3806d000 0x7fcf381f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 -- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 msgr2=0x7fcf38152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 0x7fcf38152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 -- 172.21.15.157:0/1362871773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009420 con 0x7fcf380b2af0 2023-11-06T03:09:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3cfd9640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 0x7fcf38152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf377fe640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c7c0 tx=0x7fcf2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20007a30 con 0x7fcf380b2af0 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20007b90 con 0x7fcf380b2af0 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000ce70 con 0x7fcf380b2af0 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf381f5a00 con 0x7fcf380b2af0 2023-11-06T03:09:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.120+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf381f5fc0 con 0x7fcf380b2af0 2023-11-06T03:09:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf38072750 con 0x7fcf380b2af0 2023-11-06T03:09:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcf200113f0 con 0x7fcf380b2af0 2023-11-06T03:09:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf34ff9640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 0x7fcf18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fcf20083530 con 0x7fcf380b2af0 2023-11-06T03:09:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf37fff640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 0x7fcf18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.124+0000 7fcf37fff640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 0x7fcf18067f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001570 tx=0x7fcf28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.128+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcf20051070 con 0x7fcf380b2af0 2023-11-06T03:09:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.320+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fcf381f5b90 con 0x7fcf380b2af0 2023-11-06T03:09:28.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.648+0000 7fcf34ff9640 1 -- 172.21.15.157:0/1362871773 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v273) v1 ==== 143+0+0 (secure 0 0 0) 0x7fcf20054690 con 0x7fcf380b2af0 2023-11-06T03:09:28.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 msgr2=0x7fcf18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 0x7fcf18067f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf28001570 tx=0x7fcf28009040 comp rx=0 tx=0).stop 2023-11-06T03:09:28.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 msgr2=0x7fcf38150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c7c0 tx=0x7fcf2000cc80 comp rx=0 tx=0).stop 2023-11-06T03:09:28.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 shutdown_connections 2023-11-06T03:09:28.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcf18065a60 0x7fcf18067f20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf381599c0 0x7fcf38152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf380b2af0 0x7fcf38150550 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 --2- 172.21.15.157:0/1362871773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf3806d000 0x7fcf381f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 >> 172.21.15.157:0/1362871773 conn(0x7fcf38065930 msgr2=0x7fcf3806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:28.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 shutdown_connections 2023-11-06T03:09:28.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.652+0000 7fcf3ea3c640 1 -- 172.21.15.157:0/1362871773 wait complete. 2023-11-06T03:09:28.673 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2023-11-06T03:09:28.673 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2023-11-06T03:09:28.674 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-11-06T03:09:28.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2023-11-06T03:09:28.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 Processor -- start 2023-11-06T03:09:28.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 -- start start 2023-11-06T03:09:28.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d380b3b20 0x7f2d3815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3815f580 con 0x7f2d3815e5e0 2023-11-06T03:09:28.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3815efc0 con 0x7f2d381551e0 2023-11-06T03:09:28.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3f4a7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d380704b0 con 0x7f2d380b3b20 2023-11-06T03:09:28.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3da44640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d380b3b20 0x7f2d3815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3ca42640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2d3815e5e0 0x7f2d380b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40316/0 (socket says 172.21.15.157:40316) 2023-11-06T03:09:28.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3ca42640 1 -- 172.21.15.157:0/4013752972 learned_addr learned my addr 172.21.15.157:0/4013752972 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:28.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3da44640 1 -- 172.21.15.157:0/4013752972 >> v1:172.21.15.195:6789/0 conn(0x7f2d381551e0 legacy=0x7f2d3806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:28.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3da44640 1 -- 172.21.15.157:0/4013752972 >> v1:172.21.15.157:6789/0 conn(0x7f2d3815e5e0 legacy=0x7f2d380b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.804+0000 7f2d3da44640 1 -- 172.21.15.157:0/4013752972 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3815f2a0 con 0x7f2d380b3b20 2023-11-06T03:09:28.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3da44640 1 --2- 172.21.15.157:0/4013752972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d380b3b20 0x7f2d3815c1b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d3001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae00ef5246bee360 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2ffff640 1 -- 172.21.15.157:0/4013752972 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300a5880 con 0x7f2d380b3b20 2023-11-06T03:09:28.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2ffff640 1 -- 172.21.15.157:0/4013752972 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d300a59e0 con 0x7f2d380b3b20 2023-11-06T03:09:28.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2ffff640 1 -- 172.21.15.157:0/4013752972 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300a5d70 con 0x7f2d380b3b20 2023-11-06T03:09:28.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/4013752972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d380b3b20 msgr2=0x7f2d3815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/4013752972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d380b3b20 0x7f2d3815c1b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009d90 tx=0x7f2d3001c7c0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/4013752972 shutdown_connections 2023-11-06T03:09:28.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/4013752972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d380b3b20 0x7f2d3815c1b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/4013752972 >> 172.21.15.157:0/4013752972 conn(0x7f2d38065930 msgr2=0x7f2d38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:28.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/4013752972 shutdown_connections 2023-11-06T03:09:28.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/4013752972 wait complete. 2023-11-06T03:09:28.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 Processor -- start 2023-11-06T03:09:28.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- start start 2023-11-06T03:09:28.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 0x7f2d3815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 0x7f2d380b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380ce480 con 0x7f2d381551e0 2023-11-06T03:09:28.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380ce690 con 0x7f2d380b3b20 2023-11-06T03:09:28.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3806c6f0 con 0x7f2d3815e5e0 2023-11-06T03:09:28.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35376/0 (socket says 172.21.15.157:35376) 2023-11-06T03:09:28.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 -- 172.21.15.157:0/3381190682 learned_addr learned my addr 172.21.15.157:0/3381190682 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:28.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3da44640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 0x7f2d380b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3d243640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 0x7f2d3815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 -- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 msgr2=0x7f2d380b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 0x7f2d380b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 -- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 msgr2=0x7f2d3815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:28.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 0x7f2d3815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 -- 172.21.15.157:0/3381190682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009a70 con 0x7f2d381551e0 2023-11-06T03:09:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3da44640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 0x7f2d380b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3d243640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 0x7f2d3815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3ca42640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c500 tx=0x7f2d2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28005cf0 con 0x7f2d381551e0 2023-11-06T03:09:28.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d28005e50 con 0x7f2d381551e0 2023-11-06T03:09:28.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3806c9d0 con 0x7f2d381551e0 2023-11-06T03:09:28.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d280072f0 con 0x7f2d381551e0 2023-11-06T03:09:28.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.808+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3814cae0 con 0x7f2d381551e0 2023-11-06T03:09:28.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d38072750 con 0x7f2d381551e0 2023-11-06T03:09:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d28007450 con 0x7f2d381551e0 2023-11-06T03:09:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d2dffb640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 0x7f2d1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f2d28082510 con 0x7f2d381551e0 2023-11-06T03:09:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d3d243640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 0x7f2d1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.812+0000 7f2d3d243640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 0x7f2d1c067f20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d34003e40 tx=0x7f2d34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:28.816+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2d28050100 con 0x7f2d381551e0 2023-11-06T03:09:29.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.012+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f2d38071730 con 0x7f2d381551e0 2023-11-06T03:09:29.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.656+0000 7f2d2dffb640 1 -- 172.21.15.157:0/3381190682 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v274) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2d28053720 con 0x7f2d381551e0 2023-11-06T03:09:29.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 msgr2=0x7f2d1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 0x7f2d1c067f20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d34003e40 tx=0x7f2d34009040 comp rx=0 tx=0).stop 2023-11-06T03:09:29.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 msgr2=0x7f2d3814c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c500 tx=0x7f2d2800c9c0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 shutdown_connections 2023-11-06T03:09:29.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d1c065a60 0x7f2d1c067f20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d3815e5e0 0x7f2d380b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d381551e0 0x7f2d3814c5a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 --2- 172.21.15.157:0/3381190682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d380b3b20 0x7f2d3815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 >> 172.21.15.157:0/3381190682 conn(0x7f2d38065930 msgr2=0x7f2d380b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:29.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 shutdown_connections 2023-11-06T03:09:29.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.660+0000 7f2d3f4a7640 1 -- 172.21.15.157:0/3381190682 wait complete. 2023-11-06T03:09:29.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:09:29.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 Processor -- start 2023-11-06T03:09:29.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 -- start start 2023-11-06T03:09:29.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76681551e0 0x7f766806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:29.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766815f580 con 0x7f76680b3b20 2023-11-06T03:09:29.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766815efc0 con 0x7f766815e5e0 2023-11-06T03:09:29.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766f3fd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766815f2a0 con 0x7f76681551e0 2023-11-06T03:09:29.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d199640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76681551e0 0x7f766806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:29.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d99a640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f76680b3b20 0x7f766815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40318/0 (socket says 172.21.15.157:40318) 2023-11-06T03:09:29.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d99a640 1 -- 172.21.15.157:0/1636458112 learned_addr learned my addr 172.21.15.157:0/1636458112 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:29.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d199640 1 -- 172.21.15.157:0/1636458112 >> v1:172.21.15.195:6789/0 conn(0x7f766815e5e0 legacy=0x7f76680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:29.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d199640 1 -- 172.21.15.157:0/1636458112 >> v1:172.21.15.157:6789/0 conn(0x7f76680b3b20 legacy=0x7f766815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f765ffff640 1 -- 172.21.15.157:0/1636458112 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130512971 0 0) 0x7f766815f580 con 0x7f76680b3b20 2023-11-06T03:09:29.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.816+0000 7f766d199640 1 -- 172.21.15.157:0/1636458112 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766806f5a0 con 0x7f76681551e0 2023-11-06T03:09:29.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766d199640 1 --2- 172.21.15.157:0/1636458112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76681551e0 0x7f766806fda0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f76640097c0 tx=0x7f766401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf7a3bf9861fb76 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:29.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f765ffff640 1 -- 172.21.15.157:0/1636458112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640ac640 con 0x7f76681551e0 2023-11-06T03:09:29.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f765ffff640 1 -- 172.21.15.157:0/1636458112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76640ac7a0 con 0x7f76681551e0 2023-11-06T03:09:29.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f765ffff640 1 -- 172.21.15.157:0/1636458112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640acb30 con 0x7f76681551e0 2023-11-06T03:09:29.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- 172.21.15.157:0/1636458112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76681551e0 msgr2=0x7f766806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 --2- 172.21.15.157:0/1636458112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76681551e0 0x7f766806fda0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f76640097c0 tx=0x7f766401c2f0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- 172.21.15.157:0/1636458112 shutdown_connections 2023-11-06T03:09:29.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 --2- 172.21.15.157:0/1636458112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76681551e0 0x7f766806fda0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- 172.21.15.157:0/1636458112 >> 172.21.15.157:0/1636458112 conn(0x7f7668065930 msgr2=0x7f7668065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:29.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- 172.21.15.157:0/1636458112 shutdown_connections 2023-11-06T03:09:29.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- 172.21.15.157:0/1636458112 wait complete. 2023-11-06T03:09:29.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 Processor -- start 2023-11-06T03:09:29.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.820+0000 7f766f3fd640 1 -- start start 2023-11-06T03:09:29.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:29.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76681551e0 0x7f7668151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:29.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 0x7f76680cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:29.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668153be0 con 0x7f76680b3b20 2023-11-06T03:09:29.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668152fd0 con 0x7f766815e5e0 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76681532b0 con 0x7f76681551e0 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35382/0 (socket says 172.21.15.157:35382) 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 -- 172.21.15.157:0/854615303 learned_addr learned my addr 172.21.15.157:0/854615303 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766c998640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76681551e0 0x7f7668151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:29.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d99a640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 0x7f76680cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:29.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 -- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76681551e0 msgr2=0x7f7668151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76681551e0 0x7f7668151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 -- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 msgr2=0x7f76680cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 0x7f76680cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 -- 172.21.15.157:0/854615303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009420 con 0x7f76680b3b20 2023-11-06T03:09:29.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d99a640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 0x7f76680cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:29.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766d199640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7664018f00 tx=0x7f7664019090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:29.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640ac640 con 0x7f76680b3b20 2023-11-06T03:09:29.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76640b5840 con 0x7f76680b3b20 2023-11-06T03:09:29.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668153590 con 0x7f76680b3b20 2023-11-06T03:09:29.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640b5bf0 con 0x7f76680b3b20 2023-11-06T03:09:29.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76681522f0 con 0x7f76680b3b20 2023-11-06T03:09:29.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7668072750 con 0x7f76680b3b20 2023-11-06T03:09:29.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76640b3040 con 0x7f76680b3b20 2023-11-06T03:09:29.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 0x7f764c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:29.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f766411d930 con 0x7f76680b3b20 2023-11-06T03:09:29.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.824+0000 7f766c998640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 0x7f764c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:29.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.828+0000 7f766c998640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 0x7f764c067f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7668149ba0 tx=0x7f7658009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:29.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.828+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f76640eb340 con 0x7f76680b3b20 2023-11-06T03:09:29.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.980+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f76680cb110 con 0x7f76680b3b20 2023-11-06T03:09:29.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f765dffb640 1 -- 172.21.15.157:0/854615303 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76640ee960 con 0x7f76680b3b20 2023-11-06T03:09:29.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 msgr2=0x7f764c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 0x7f764c067f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7668149ba0 tx=0x7f7658009040 comp rx=0 tx=0).stop 2023-11-06T03:09:29.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 msgr2=0x7f76681529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:29.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7664018f00 tx=0x7f7664019090 comp rx=0 tx=0).stop 2023-11-06T03:09:29.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 shutdown_connections 2023-11-06T03:09:29.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f764c065a60 0x7f764c067f20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f766815e5e0 0x7f76680cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76681551e0 0x7f7668151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 --2- 172.21.15.157:0/854615303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680b3b20 0x7f76681529f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 >> 172.21.15.157:0/854615303 conn(0x7f7668065930 msgr2=0x7f76680b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 shutdown_connections 2023-11-06T03:09:29.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:29.992+0000 7f766f3fd640 1 -- 172.21.15.157:0/854615303 wait complete. 2023-11-06T03:09:30.010 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:09:30.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:09:30.047 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:09:30.096 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:09:30.096 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:09:30.096 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:30.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:09:30.103 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:09:30.103 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:30.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:09:30.109 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:09:30.109 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:30.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:09:30.151 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:09:30.152 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:30.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:09:30.157 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:09:30.158 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.320+0000 7f64f0d60640 1 Processor -- start 2023-11-06T03:09:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.320+0000 7f64f0d60640 1 -- start start 2023-11-06T03:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64f0d60640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64ec1551e0 0x7f64ec06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64f0d60640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f64ec0b3b20 0x7f64ec15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64f0d60640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec15f580 con 0x7f64ec15e5e0 2023-11-06T03:09:30.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64f0d60640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec15efc0 con 0x7f64ec1551e0 2023-11-06T03:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64f0d60640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec15f2a0 con 0x7f64ec0b3b20 2023-11-06T03:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64ec1551e0 0x7f64ec06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64ec1551e0 0x7f64ec06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35398/0 (socket says 172.21.15.157:35398) 2023-11-06T03:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 -- 172.21.15.157:0/1328065284 learned_addr learned my addr 172.21.15.157:0/1328065284 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:30.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ead9d640 1 --2- 172.21.15.157:0/1328065284 >> v2:172.21.15.195:3300/0 conn(0x7f64ec0b3b20 0x7f64ec15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 -- 172.21.15.157:0/1328065284 >> v2:172.21.15.195:3300/0 conn(0x7f64ec0b3b20 msgr2=0x7f64ec15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 --2- 172.21.15.157:0/1328065284 >> v2:172.21.15.195:3300/0 conn(0x7f64ec0b3b20 0x7f64ec15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 -- 172.21.15.157:0/1328065284 >> v1:172.21.15.195:6789/0 conn(0x7f64ec15e5e0 legacy=0x7f64ec0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:30.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 -- 172.21.15.157:0/1328065284 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06f5a0 con 0x7f64ec1551e0 2023-11-06T03:09:30.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64ea59c640 1 --2- 172.21.15.157:0/1328065284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec06fda0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0cbfe0 tx=0x7f64dc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45264b7815b18637 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64e959a640 1 -- 172.21.15.157:0/1328065284 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0ad640 con 0x7f64ec1551e0 2023-11-06T03:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64e959a640 1 -- 172.21.15.157:0/1328065284 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc0ad7a0 con 0x7f64ec1551e0 2023-11-06T03:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.324+0000 7f64e959a640 1 -- 172.21.15.157:0/1328065284 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0adb30 con 0x7f64ec1551e0 2023-11-06T03:09:30.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/1328065284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 msgr2=0x7f64ec06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- 172.21.15.157:0/1328065284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec06fda0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0cbfe0 tx=0x7f64dc01c370 comp rx=0 tx=0).stop 2023-11-06T03:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/1328065284 shutdown_connections 2023-11-06T03:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- 172.21.15.157:0/1328065284 >> v2:172.21.15.195:3300/0 conn(0x7f64ec0b3b20 0x7f64ec15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- 172.21.15.157:0/1328065284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec06fda0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/1328065284 >> 172.21.15.157:0/1328065284 conn(0x7f64ec065930 msgr2=0x7f64ec065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/1328065284 shutdown_connections 2023-11-06T03:09:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/1328065284 wait complete. 2023-11-06T03:09:30.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 Processor -- start 2023-11-06T03:09:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- start start 2023-11-06T03:09:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64ec0b3b20 0x7f64ec14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 0x7f64ec152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec144120 con 0x7f64ec1551e0 2023-11-06T03:09:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec143510 con 0x7f64ec0b3b20 2023-11-06T03:09:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1437f0 con 0x7f64ec15e5e0 2023-11-06T03:09:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35408/0 (socket says 172.21.15.157:35408) 2023-11-06T03:09:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 -- 172.21.15.157:0/2125985451 learned_addr learned my addr 172.21.15.157:0/2125985451 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64ea59c640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64ec0b3b20 0x7f64ec14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64ead9d640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 0x7f64ec152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 -- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 msgr2=0x7f64ec152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 0x7f64ec152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 -- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64ec0b3b20 msgr2=0x7f64ec14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64ec0b3b20 0x7f64ec14cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 -- 172.21.15.157:0/2125985451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0094d0 con 0x7f64ec1551e0 2023-11-06T03:09:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64ead9d640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 0x7f64ec152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:30.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64e9d9b640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b3e0 tx=0x7f64e000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011830 con 0x7f64ec1551e0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e00119b0 con 0x7f64ec1551e0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec143ad0 con 0x7f64ec1551e0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0011d60 con 0x7f64ec1551e0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.328+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec14d270 con 0x7f64ec1551e0 2023-11-06T03:09:30.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec1442d0 con 0x7f64ec1551e0 2023-11-06T03:09:30.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f64e001a400 con 0x7f64ec1551e0 2023-11-06T03:09:30.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64daffd640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 0x7f64cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f64e0083200 con 0x7f64ec1551e0 2023-11-06T03:09:30.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64ea59c640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 0x7f64cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.332+0000 7f64ea59c640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 0x7f64cc067f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0af000 tx=0x7f64dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.336+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f64ec1442d0 con 0x7f64ec1551e0 2023-11-06T03:09:30.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ec0b03a0 con 0x7f64ec1551e0 2023-11-06T03:09:30.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64daffd640 1 -- 172.21.15.157:0/2125985451 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f64e0015df0 con 0x7f64ec1551e0 2023-11-06T03:09:30.523 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:30.523 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5914,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1226708208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1226708208},{"type":"v1","addr":"172.21.15.195:6835","nonce":1226708208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":5923,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2760714073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2760714073},{"type":"v1","addr":"172.21.15.195:6837","nonce":2760714073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7059,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2875132726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2875132726},{"type":"v1","addr":"172.21.15.157:6835","nonce":2875132726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4166105002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4166105002},{"type":"v1","addr":"172.21.15.157:6833","nonce":4166105002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2023-11-06T03:09:30.524 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 56 2023-11-06T03:09:30.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 msgr2=0x7f64cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 0x7f64cc067f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0af000 tx=0x7f64dc008040 comp rx=0 tx=0).stop 2023-11-06T03:09:30.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 msgr2=0x7f64ec152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b3e0 tx=0x7f64e000b8a0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 shutdown_connections 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f64cc065a60 0x7f64cc067f20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64ec15e5e0 0x7f64ec152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1551e0 0x7f64ec152580 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 --2- 172.21.15.157:0/2125985451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64ec0b3b20 0x7f64ec14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 >> 172.21.15.157:0/2125985451 conn(0x7f64ec065930 msgr2=0x7f64ec0af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:30.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 shutdown_connections 2023-11-06T03:09:30.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.524+0000 7f64f0d60640 1 -- 172.21.15.157:0/2125985451 wait complete. 2023-11-06T03:09:30.543 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:09:30.543 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:09:30.544 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:09:30.544 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:30.544 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:09:30.545 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:09:30.545 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:09:30.545 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:30.546 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:09:30.546 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:09:30.546 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:09:30.547 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:30.547 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:09:30.547 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:09:30.548 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:09:30.548 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:09:30.553 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:09:30.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:09:30.555 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:09:30.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:09:30.560 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:09:30.560 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:09:30.565 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:09:30.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:09:30.598 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:09:30.599 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:30.600 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:09:30.600 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:30.611 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:09:30.611 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:30.612 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:09:30.613 DEBUG:teuthology.parallel:result is None 2023-11-06T03:09:30.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:09:30.661 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:09:30.668 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:09:30.684 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:09:30.692 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:09:30.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 Processor -- start 2023-11-06T03:09:30.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 -- start start 2023-11-06T03:09:30.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b40d0ee0 0x7fc9b40cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9b406ac30 0x7fc9b40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b406da10 con 0x7fc9b415ce30 2023-11-06T03:09:30.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b406dcf0 con 0x7fc9b406ac30 2023-11-06T03:09:30.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9baa9a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b406dfd0 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9b406ac30 0x7fc9b40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9b406ac30 0x7fc9b40cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35462/0 (socket says 172.21.15.157:35462) 2023-11-06T03:09:30.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 -- 172.21.15.157:0/1293209314 learned_addr learned my addr 172.21.15.157:0/1293209314 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:30.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b3fff640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b40d0ee0 0x7fc9b40cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 -- 172.21.15.157:0/1293209314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b40d0ee0 msgr2=0x7fc9b40cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:30.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b40d0ee0 0x7fc9b40cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 -- 172.21.15.157:0/1293209314 >> v1:172.21.15.195:6789/0 conn(0x7fc9b415ce30 legacy=0x7fc9b40d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:30.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 -- 172.21.15.157:0/1293209314 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b406e2b0 con 0x7fc9b406ac30 2023-11-06T03:09:30.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.752+0000 7fc9b9037640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b406ac30 0x7fc9b40cc6b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009e50 tx=0x7fc9ac01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3990cc27bd42b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b37fe640 1 -- 172.21.15.157:0/1293209314 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0a57b0 con 0x7fc9b406ac30 2023-11-06T03:09:30.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b37fe640 1 -- 172.21.15.157:0/1293209314 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac0a5910 con 0x7fc9b406ac30 2023-11-06T03:09:30.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b37fe640 1 -- 172.21.15.157:0/1293209314 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0a5ca0 con 0x7fc9b406ac30 2023-11-06T03:09:30.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/1293209314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b406ac30 msgr2=0x7fc9b40cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b406ac30 0x7fc9b40cc6b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009e50 tx=0x7fc9ac01c9d0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/1293209314 shutdown_connections 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b406ac30 0x7fc9b40cc6b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/1293209314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b40d0ee0 0x7fc9b40cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/1293209314 >> 172.21.15.157:0/1293209314 conn(0x7fc9b4071b90 msgr2=0x7fc9b4099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/1293209314 shutdown_connections 2023-11-06T03:09:30.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/1293209314 wait complete. 2023-11-06T03:09:30.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 Processor -- start 2023-11-06T03:09:30.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- start start 2023-11-06T03:09:30.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9b406ac30 0x7fc9b40ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b415ce30 0x7fc9b40b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b41587a0 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b4157b90 con 0x7fc9b406ac30 2023-11-06T03:09:30.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b4157e70 con 0x7fc9b415ce30 2023-11-06T03:09:30.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35474/0 (socket says 172.21.15.157:35474) 2023-11-06T03:09:30.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 -- 172.21.15.157:0/3848534191 learned_addr learned my addr 172.21.15.157:0/3848534191 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:30.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b8836640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9b406ac30 0x7fc9b40ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b9037640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b415ce30 0x7fc9b40b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 -- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b415ce30 msgr2=0x7fc9b40b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b415ce30 0x7fc9b40b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 -- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9b406ac30 msgr2=0x7fc9b40ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:30.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9b406ac30 0x7fc9b40ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:30.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 -- 172.21.15.157:0/3848534191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac009b30 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b3fff640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d800 tx=0x7fc9a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800b6c0 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a800b820 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b4158150 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800bb90 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.756+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b415c250 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b4072750 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc9a800bcf0 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9b17fa640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 0x7fc9a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc9a8083500 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9b8836640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 0x7fc9a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:30.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.760+0000 7fc9b8836640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 0x7fc9a0067f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:30.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.764+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc9a8051050 con 0x7fc9b40d0ee0 2023-11-06T03:09:30.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:30.928+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc9b40b3750 con 0x7fc9b40d0ee0 2023-11-06T03:09:31.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9b17fa640 1 -- 172.21.15.157:0/3848534191 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9a8054670 con 0x7fc9b40d0ee0 2023-11-06T03:09:31.735 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:09:31.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 msgr2=0x7fc9a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 0x7fc9a0067f20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4009060 comp rx=0 tx=0).stop 2023-11-06T03:09:31.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 msgr2=0x7fc9b415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d800 tx=0x7fc9a800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 shutdown_connections 2023-11-06T03:09:31.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc9a0065a60 0x7fc9a0067f20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9b415ce30 0x7fc9b40b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9b40d0ee0 0x7fc9b415bd10 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 --2- 172.21.15.157:0/3848534191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9b406ac30 0x7fc9b40ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.736+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 >> 172.21.15.157:0/3848534191 conn(0x7fc9b4071b90 msgr2=0x7fc9b40aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.740+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 shutdown_connections 2023-11-06T03:09:31.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.740+0000 7fc9baa9a640 1 -- 172.21.15.157:0/3848534191 wait complete. 2023-11-06T03:09:31.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:09:31.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.920+0000 7f2d31858640 1 Processor -- start 2023-11-06T03:09:31.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.920+0000 7f2d31858640 1 -- start start 2023-11-06T03:09:31.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d31858640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d2c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d31858640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c06c490 con 0x7f2d2c156a70 2023-11-06T03:09:31.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d31858640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c06fb80 con 0x7f2d2c0adae0 2023-11-06T03:09:31.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d31858640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06b950 con 0x7f2d2c0b2330 2023-11-06T03:09:31.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2d2c156a70 0x7f2d2c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40346/0 (socket says 172.21.15.157:40346) 2023-11-06T03:09:31.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 -- 172.21.15.157:0/156451096 learned_addr learned my addr 172.21.15.157:0/156451096 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:31.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2b7fe640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d2c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622830028 0 0) 0x7f2d2c06c490 con 0x7f2d2c156a70 2023-11-06T03:09:31.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d18003530 con 0x7f2d2c156a70 2023-11-06T03:09:31.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729176808 0 0) 0x7f2d18003530 con 0x7f2d2c156a70 2023-11-06T03:09:31.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 msgr2=0x7f2d2c06c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:31.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d2c06c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 >> v1:172.21.15.195:6789/0 conn(0x7f2d2c0adae0 legacy=0x7f2d2c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c06bb60 con 0x7f2d2c156a70 2023-11-06T03:09:31.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2d1c001a00 con 0x7f2d2c156a70 2023-11-06T03:09:31.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 >> v1:172.21.15.157:6789/0 conn(0x7f2d2c156a70 legacy=0x7f2d2c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d180039b0 0x7f2d18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d180047d0 0x7f2d18004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06b950 con 0x7f2d180039b0 2023-11-06T03:09:31.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800ba70 con 0x7f2d180047d0 2023-11-06T03:09:31.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800bc30 con 0x7f2d2c0b2330 2023-11-06T03:09:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d180039b0 0x7f2d18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2a7fc640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d180047d0 0x7f2d18004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 -- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 msgr2=0x7f2d18005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2b7fe640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d2c0b2330 0x7f2d18005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 -- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d180047d0 msgr2=0x7f2d18004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:31.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d180047d0 0x7f2d18004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2affd640 1 -- 172.21.15.157:0/156451096 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18003530 con 0x7f2d180039b0 2023-11-06T03:09:31.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.924+0000 7f2d2a7fc640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d180047d0 0x7f2d18004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:31.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2affd640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d180039b0 0x7f2d18003dd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00a410 tx=0x7f2d1c01f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ee53f04a0dd5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:31.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0072d0 con 0x7f2d180039b0 2023-11-06T03:09:31.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c0aa070 con 0x7f2d180039b0 2023-11-06T03:09:31.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d29ffb640 1 -- 172.21.15.157:0/156451096 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c007670 con 0x7f2d180039b0 2023-11-06T03:09:31.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/156451096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d180039b0 msgr2=0x7f2d18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 --2- 172.21.15.157:0/156451096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d180039b0 0x7f2d18003dd0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00a410 tx=0x7f2d1c01f4e0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2b7fe640 1 -- 172.21.15.157:0/156451096 reap_dead start 2023-11-06T03:09:31.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/156451096 shutdown_connections 2023-11-06T03:09:31.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/156451096 >> 172.21.15.157:0/156451096 conn(0x7f2d2c065930 msgr2=0x7f2d2c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:31.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/156451096 shutdown_connections 2023-11-06T03:09:31.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/156451096 wait complete. 2023-11-06T03:09:31.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 Processor -- start 2023-11-06T03:09:31.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- start start 2023-11-06T03:09:31.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d180039b0 0x7f2d2c1464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c14e980 0x7f2d2c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1469f0 con 0x7f2d2c1475b0 2023-11-06T03:09:31.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c14f340 con 0x7f2d2c14e980 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1496a0 con 0x7f2d180039b0 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35486/0 (socket says 172.21.15.157:35486) 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 -- 172.21.15.157:0/855074 learned_addr learned my addr 172.21.15.157:0/855074 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2affd640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d180039b0 0x7f2d2c1464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2b7fe640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c14e980 0x7f2d2c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 -- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d180039b0 msgr2=0x7f2d2c1464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d180039b0 0x7f2d2c1464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 -- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c14e980 msgr2=0x7f2d2c14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:31.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c14e980 0x7f2d2c14ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:31.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 -- 172.21.15.157:0/855074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c149980 con 0x7f2d2c1475b0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d2a7fc640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d770 tx=0x7f2d1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005270 con 0x7f2d2c1475b0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140053d0 con 0x7f2d2c1475b0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c149c60 con 0x7f2d2c1475b0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400b8d0 con 0x7f2d2c1475b0 2023-11-06T03:09:31.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.928+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1492c0 con 0x7f2d2c1475b0 2023-11-06T03:09:31.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c072750 con 0x7f2d2c1475b0 2023-11-06T03:09:31.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d1400ba30 con 0x7f2d2c1475b0 2023-11-06T03:09:31.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d23fff640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 0x7f2d0c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:31.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2d14082f00 con 0x7f2d2c1475b0 2023-11-06T03:09:31.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d2affd640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 0x7f2d0c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:31.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.932+0000 7f2d2affd640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 0x7f2d0c067f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0af000 tx=0x7f2d1c009050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:31.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:31.936+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2d14050ab0 con 0x7f2d2c1475b0 2023-11-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.128+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2d2c157270 con 0x7f2d2c1475b0 2023-11-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.128+0000 7f2d23fff640 1 -- 172.21.15.157:0/855074 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2d14019310 con 0x7f2d2c1475b0 2023-11-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:09:32.132 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:09:32.133 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.133 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.133 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.133 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.133 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:09:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:09:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:09:32.136 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.141 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:09:32.142 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:09:32.143 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.143 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.143 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.143 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.143 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.144 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.145 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.145 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.145 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.145 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.145 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.146 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:09:32.147 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.148 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.148 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.148 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.148 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.148 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:09:32.149 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:09:32.150 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:09:32.151 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.151 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.151 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:09:32.151 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:09:32.151 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.152 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.152 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.152 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:09:32.152 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:09:32.152 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.153 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.154 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.154 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:09:32.154 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:09:32.154 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.154 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.155 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.156 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:09:32.156 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:09:32.156 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.156 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:09:32.156 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.157 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:09:32.158 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.159 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.159 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.159 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.159 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:09:32.159 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.160 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:09:32.161 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:09:32.162 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:09:32.162 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.162 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:32.162 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:09:32.162 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 msgr2=0x7f2d0c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 0x7f2d0c067f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0af000 tx=0x7f2d1c009050 comp rx=0 tx=0).stop 2023-11-06T03:09:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 msgr2=0x7f2d2c147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d770 tx=0x7f2d1400dc30 comp rx=0 tx=0).stop 2023-11-06T03:09:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 shutdown_connections 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d0c065a90 0x7f2d0c067f50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c14e980 0x7f2d2c14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d2c1475b0 0x7f2d2c147a30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 --2- 172.21.15.157:0/855074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d180039b0 0x7f2d2c1464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 >> 172.21.15.157:0/855074 conn(0x7f2d2c065930 msgr2=0x7f2d2c14d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 shutdown_connections 2023-11-06T03:09:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.132+0000 7f2d31858640 1 -- 172.21.15.157:0/855074 wait complete. 2023-11-06T03:09:32.166 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:32.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.288+0000 7f95b599a640 1 Processor -- start 2023-11-06T03:09:32.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.288+0000 7f95b599a640 1 -- start start 2023-11-06T03:09:32.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95b599a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f95b01599c0 0x7f95b00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95b599a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95b00b2af0 0x7f95b0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95b599a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b015e440 con 0x7f95b006d000 2023-11-06T03:09:32.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95b599a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b006c240 con 0x7f95b00b2af0 2023-11-06T03:09:32.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95b599a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b006c520 con 0x7f95b01599c0 2023-11-06T03:09:32.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95b00b2af0 0x7f95b0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95b00b2af0 0x7f95b0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35500/0 (socket says 172.21.15.157:35500) 2023-11-06T03:09:32.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 -- 172.21.15.157:0/2122359389 learned_addr learned my addr 172.21.15.157:0/2122359389 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:32.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95aeffd640 1 --2- 172.21.15.157:0/2122359389 >> v2:172.21.15.195:3300/0 conn(0x7f95b01599c0 0x7f95b00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 -- 172.21.15.157:0/2122359389 >> v2:172.21.15.195:3300/0 conn(0x7f95b01599c0 msgr2=0x7f95b00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:32.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 --2- 172.21.15.157:0/2122359389 >> v2:172.21.15.195:3300/0 conn(0x7f95b01599c0 0x7f95b00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 -- 172.21.15.157:0/2122359389 >> v1:172.21.15.195:6789/0 conn(0x7f95b006d000 legacy=0x7f95b00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:32.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 -- 172.21.15.157:0/2122359389 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b006c800 con 0x7f95b00b2af0 2023-11-06T03:09:32.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.292+0000 7f95af7fe640 1 --2- 172.21.15.157:0/2122359389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b00b2af0 0x7f95b0159da0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f95b0066800 tx=0x7f95a801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9322ad4148886 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95ae7fc640 1 -- 172.21.15.157:0/2122359389 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80a5840 con 0x7f95b00b2af0 2023-11-06T03:09:32.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95ae7fc640 1 -- 172.21.15.157:0/2122359389 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a80a59a0 con 0x7f95b00b2af0 2023-11-06T03:09:32.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95ae7fc640 1 -- 172.21.15.157:0/2122359389 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80a5d30 con 0x7f95b00b2af0 2023-11-06T03:09:32.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- 172.21.15.157:0/2122359389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b00b2af0 msgr2=0x7f95b0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- 172.21.15.157:0/2122359389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b00b2af0 0x7f95b0159da0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f95b0066800 tx=0x7f95a801c960 comp rx=0 tx=0).stop 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- 172.21.15.157:0/2122359389 shutdown_connections 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- 172.21.15.157:0/2122359389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b00b2af0 0x7f95b0159da0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- 172.21.15.157:0/2122359389 >> v2:172.21.15.195:3300/0 conn(0x7f95b01599c0 0x7f95b00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- 172.21.15.157:0/2122359389 >> 172.21.15.157:0/2122359389 conn(0x7f95b0065930 msgr2=0x7f95b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- 172.21.15.157:0/2122359389 shutdown_connections 2023-11-06T03:09:32.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- 172.21.15.157:0/2122359389 wait complete. 2023-11-06T03:09:32.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 Processor -- start 2023-11-06T03:09:32.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- start start 2023-11-06T03:09:32.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 0x7f95b014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b01599c0 0x7f95b01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0154eb0 con 0x7f95b006d000 2023-11-06T03:09:32.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00cbfe0 con 0x7f95b00b2af0 2023-11-06T03:09:32.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95b599a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b0080 con 0x7f95b01599c0 2023-11-06T03:09:32.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35512/0 (socket says 172.21.15.157:35512) 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 -- 172.21.15.157:0/3542836079 learned_addr learned my addr 172.21.15.157:0/3542836079 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95af7fe640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b01599c0 0x7f95b01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95a67fc640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 0x7f95b014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 -- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b01599c0 msgr2=0x7f95b01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b01599c0 0x7f95b01552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 -- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 msgr2=0x7f95b014c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 0x7f95b014c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 -- 172.21.15.157:0/3542836079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009b30 con 0x7f95b006d000 2023-11-06T03:09:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95a67fc640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 0x7f95b014c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95aeffd640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d5f0 tx=0x7f95a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00125f0 con 0x7f95b006d000 2023-11-06T03:09:32.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0012750 con 0x7f95b006d000 2023-11-06T03:09:32.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.296+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0012ae0 con 0x7f95b006d000 2023-11-06T03:09:32.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00b0360 con 0x7f95b006d000 2023-11-06T03:09:32.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b0154ad0 con 0x7f95b006d000 2023-11-06T03:09:32.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b0072750 con 0x7f95b006d000 2023-11-06T03:09:32.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f95a0012c40 con 0x7f95b006d000 2023-11-06T03:09:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95acff9640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 0x7f9590067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f95a0083400 con 0x7f95b006d000 2023-11-06T03:09:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95a67fc640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 0x7f9590067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.300+0000 7f95a67fc640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 0x7f9590067f50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f95b0156e50 tx=0x7f9598009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.304+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f95a0050fb0 con 0x7f95b006d000 2023-11-06T03:09:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.456+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b00cf000 con 0x7f95b006d000 2023-11-06T03:09:32.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.456+0000 7f95acff9640 1 -- 172.21.15.157:0/3542836079 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+14220 (secure 0 0 0) 0x7f95a0004650 con 0x7f95b006d000 2023-11-06T03:09:32.457 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:32.457 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":81,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:31.729965+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:32.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 msgr2=0x7f9590067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 0x7f9590067f50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f95b0156e50 tx=0x7f9598009040 comp rx=0 tx=0).stop 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 msgr2=0x7f95b00cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d5f0 tx=0x7f95a000dab0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 shutdown_connections 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9590065a90 0x7f9590067f50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b01599c0 0x7f95b01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b00b2af0 0x7f95b014c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 --2- 172.21.15.157:0/3542836079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b006d000 0x7f95b00cbaa0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 >> 172.21.15.157:0/3542836079 conn(0x7f95b0065930 msgr2=0x7f95b006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 shutdown_connections 2023-11-06T03:09:32.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.460+0000 7f95b599a640 1 -- 172.21.15.157:0/3542836079 wait complete. 2023-11-06T03:09:32.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:09:32.476 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:09:32.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 Processor -- start 2023-11-06T03:09:32.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 -- start start 2023-11-06T03:09:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb730066040 0x7fb73006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb73006d100 0x7fb730160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300cf820 con 0x7fb730157070 2023-11-06T03:09:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73006d8f0 con 0x7fb730066040 2023-11-06T03:09:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb736a40640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300d05e0 con 0x7fb73006d100 2023-11-06T03:09:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb730066040 0x7fb73006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb730157070 0x7fb730065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40354/0 (socket says 172.21.15.157:40354) 2023-11-06T03:09:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72ffff640 1 -- 172.21.15.157:0/3195489948 learned_addr learned my addr 172.21.15.157:0/3195489948 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72f7fe640 1 -- 172.21.15.157:0/3195489948 >> v2:172.21.15.195:3300/0 conn(0x7fb73006d100 msgr2=0x7fb730160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb734fdd640 1 --2- 172.21.15.157:0/3195489948 >> v2:172.21.15.195:3300/0 conn(0x7fb73006d100 0x7fb730160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72f7fe640 1 --2- 172.21.15.157:0/3195489948 >> v2:172.21.15.195:3300/0 conn(0x7fb73006d100 0x7fb730160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72f7fe640 1 -- 172.21.15.157:0/3195489948 >> v1:172.21.15.157:6789/0 conn(0x7fb730157070 legacy=0x7fb730065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72f7fe640 1 -- 172.21.15.157:0/3195489948 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73015e150 con 0x7fb730066040 2023-11-06T03:09:32.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.596+0000 7fb72effd640 1 -- 172.21.15.157:0/3195489948 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237012466 0 0) 0x7fb7300cf820 con 0x7fb730157070 2023-11-06T03:09:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- 172.21.15.157:0/3195489948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb730066040 0x7fb73006cbc0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb730066950 tx=0x7fb71801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b9fd0b4041123f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72effd640 1 -- 172.21.15.157:0/3195489948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7180ad680 con 0x7fb730066040 2023-11-06T03:09:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72effd640 1 -- 172.21.15.157:0/3195489948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7180ad7e0 con 0x7fb730066040 2023-11-06T03:09:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72effd640 1 -- 172.21.15.157:0/3195489948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7180adb70 con 0x7fb730066040 2023-11-06T03:09:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- 172.21.15.157:0/3195489948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb730066040 msgr2=0x7fb73006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- 172.21.15.157:0/3195489948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb730066040 0x7fb73006cbc0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb730066950 tx=0x7fb71801c380 comp rx=0 tx=0).stop 2023-11-06T03:09:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- 172.21.15.157:0/3195489948 shutdown_connections 2023-11-06T03:09:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- 172.21.15.157:0/3195489948 >> v2:172.21.15.195:3300/0 conn(0x7fb73006d100 0x7fb730160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- 172.21.15.157:0/3195489948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb730066040 0x7fb73006cbc0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- 172.21.15.157:0/3195489948 >> 172.21.15.157:0/3195489948 conn(0x7fb730069d70 msgr2=0x7fb73006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- 172.21.15.157:0/3195489948 shutdown_connections 2023-11-06T03:09:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- 172.21.15.157:0/3195489948 wait complete. 2023-11-06T03:09:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 Processor -- start 2023-11-06T03:09:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- start start 2023-11-06T03:09:32.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb730066040 0x7fb730070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb730157070 0x7fb7301595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015a310 con 0x7fb73006d100 2023-11-06T03:09:32.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015a030 con 0x7fb730157070 2023-11-06T03:09:32.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb736a40640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c8c60 con 0x7fb730066040 2023-11-06T03:09:32.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35532/0 (socket says 172.21.15.157:35532) 2023-11-06T03:09:32.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 -- 172.21.15.157:0/420687838 learned_addr learned my addr 172.21.15.157:0/420687838 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:32.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb734fdd640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb730157070 0x7fb7301595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72ffff640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb730066040 0x7fb730070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 -- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb730066040 msgr2=0x7fb730070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb730066040 0x7fb730070a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 -- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb730157070 msgr2=0x7fb7301595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb730157070 0x7fb7301595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 -- 172.21.15.157:0/420687838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009420 con 0x7fb73006d100 2023-11-06T03:09:32.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.600+0000 7fb72f7fe640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb718019fb0 tx=0x7fb718019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7180ade20 con 0x7fb73006d100 2023-11-06T03:09:32.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb718004020 con 0x7fb73006d100 2023-11-06T03:09:32.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7300c8f40 con 0x7fb73006d100 2023-11-06T03:09:32.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718004390 con 0x7fb73006d100 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730156010 con 0x7fb73006d100 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73006ad00 con 0x7fb73006d100 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7180b4040 con 0x7fb73006d100 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 0x7fb710067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb71811ce20 con 0x7fb73006d100 2023-11-06T03:09:32.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72ffff640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 0x7fb710067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.604+0000 7fb72ffff640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 0x7fb710067f20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb720007fd0 tx=0x7fb720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.608+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7180eab80 con 0x7fb73006d100 2023-11-06T03:09:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.764+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb730072750 con 0x7fb73006d100 2023-11-06T03:09:32.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.764+0000 7fb72cff9640 1 -- 172.21.15.157:0/420687838 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7180b2020 con 0x7fb73006d100 2023-11-06T03:09:32.765 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:09:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 msgr2=0x7fb710067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 0x7fb710067f20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb720007fd0 tx=0x7fb720009040 comp rx=0 tx=0).stop 2023-11-06T03:09:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 msgr2=0x7fb730153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb718019fb0 tx=0x7fb718019120 comp rx=0 tx=0).stop 2023-11-06T03:09:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 shutdown_connections 2023-11-06T03:09:32.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb710065a60 0x7fb710067f20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb730157070 0x7fb7301595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb73006d100 0x7fb730153880 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 --2- 172.21.15.157:0/420687838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb730066040 0x7fb730070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 >> 172.21.15.157:0/420687838 conn(0x7fb730069d70 msgr2=0x7fb730065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 shutdown_connections 2023-11-06T03:09:32.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.768+0000 7fb736a40640 1 -- 172.21.15.157:0/420687838 wait complete. 2023-11-06T03:09:32.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:09:32.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.904+0000 7f3870fef640 1 Processor -- start 2023-11-06T03:09:32.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.904+0000 7f3870fef640 1 -- start start 2023-11-06T03:09:32.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3870fef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 0x7f386c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3870fef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f386c15ea90 0x7f386c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3870fef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c15f580 con 0x7f386c1551e0 2023-11-06T03:09:32.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3870fef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0704b0 con 0x7f386c15ea90 2023-11-06T03:09:32.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3870fef640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c15efc0 con 0x7f386c15e5e0 2023-11-06T03:09:32.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f386c15ea90 0x7f386c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f386c15ea90 0x7f386c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35536/0 (socket says 172.21.15.157:35536) 2023-11-06T03:09:32.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 -- 172.21.15.157:0/2506653252 learned_addr learned my addr 172.21.15.157:0/2506653252 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:32.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3869d9b640 1 --2- 172.21.15.157:0/2506653252 >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 0x7f386c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 -- 172.21.15.157:0/2506653252 >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 msgr2=0x7f386c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:32.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 --2- 172.21.15.157:0/2506653252 >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 0x7f386c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 -- 172.21.15.157:0/2506653252 >> v1:172.21.15.195:6789/0 conn(0x7f386c1551e0 legacy=0x7f386c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:32.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 -- 172.21.15.157:0/2506653252 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c15f2a0 con 0x7f386c15ea90 2023-11-06T03:09:32.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f3869d9b640 1 --2- 172.21.15.157:0/2506653252 >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 0x7f386c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:32.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386ad9d640 1 --2- 172.21.15.157:0/2506653252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15ea90 0x7f386c0b3950 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f386c066880 tx=0x7f386401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4736502f60f2647 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386959a640 1 -- 172.21.15.157:0/2506653252 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640a5790 con 0x7f386c15ea90 2023-11-06T03:09:32.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386959a640 1 -- 172.21.15.157:0/2506653252 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38640a58f0 con 0x7f386c15ea90 2023-11-06T03:09:32.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.908+0000 7f386959a640 1 -- 172.21.15.157:0/2506653252 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640a5c80 con 0x7f386c15ea90 2023-11-06T03:09:32.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/2506653252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15ea90 msgr2=0x7f386c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- 172.21.15.157:0/2506653252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15ea90 0x7f386c0b3950 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f386c066880 tx=0x7f386401c8d0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/2506653252 shutdown_connections 2023-11-06T03:09:32.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- 172.21.15.157:0/2506653252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15ea90 0x7f386c0b3950 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- 172.21.15.157:0/2506653252 >> v2:172.21.15.195:3300/0 conn(0x7f386c15e5e0 0x7f386c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/2506653252 >> 172.21.15.157:0/2506653252 conn(0x7f386c065930 msgr2=0x7f386c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:32.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/2506653252 shutdown_connections 2023-11-06T03:09:32.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/2506653252 wait complete. 2023-11-06T03:09:32.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 Processor -- start 2023-11-06T03:09:32.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- start start 2023-11-06T03:09:32.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f386c1551e0 0x7f386c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f386c15ea90 0x7f386c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c06b9d0 con 0x7f386c15e5e0 2023-11-06T03:09:32.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c14cbe0 con 0x7f386c15ea90 2023-11-06T03:09:32.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c06bc70 con 0x7f386c1551e0 2023-11-06T03:09:32.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35542/0 (socket says 172.21.15.157:35542) 2023-11-06T03:09:32.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 -- 172.21.15.157:0/1663457982 learned_addr learned my addr 172.21.15.157:0/1663457982 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:32.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f386a59c640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f386c1551e0 0x7f386c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f386ad9d640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f386c15ea90 0x7f386c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 -- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f386c1551e0 msgr2=0x7f386c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f386c1551e0 0x7f386c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 -- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f386c15ea90 msgr2=0x7f386c14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:32.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f386c15ea90 0x7f386c14c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:32.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 -- 172.21.15.157:0/1663457982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864009b30 con 0x7f386c15e5e0 2023-11-06T03:09:32.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3869d9b640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f386000c7c0 tx=0x7f386000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860007a30 con 0x7f386c15e5e0 2023-11-06T03:09:32.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860007b90 con 0x7f386c15e5e0 2023-11-06T03:09:32.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c06bf50 con 0x7f386c15e5e0 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386000ce70 con 0x7f386c15e5e0 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.912+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c06c510 con 0x7f386c15e5e0 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c072750 con 0x7f386c15e5e0 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38600113f0 con 0x7f386c15e5e0 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f385affd640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:32.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f38600834e0 con 0x7f386c15e5e0 2023-11-06T03:09:32.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f386a59c640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:32.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.916+0000 7f386a59c640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 0x7f384c067f50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f386c06d510 tx=0x7f385c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:32.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:32.920+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3860050ff0 con 0x7f386c15e5e0 2023-11-06T03:09:33.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f386c071730 con 0x7f386c15e5e0 2023-11-06T03:09:33.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f385affd640 1 -- 172.21.15.157:0/1663457982 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3860054610 con 0x7f386c15e5e0 2023-11-06T03:09:33.063 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:09:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 msgr2=0x7f384c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 0x7f384c067f50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f386c06d510 tx=0x7f385c009040 comp rx=0 tx=0).stop 2023-11-06T03:09:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 msgr2=0x7f386c06b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f386000c7c0 tx=0x7f386000cc80 comp rx=0 tx=0).stop 2023-11-06T03:09:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 shutdown_connections 2023-11-06T03:09:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f384c065a90 0x7f384c067f50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f386c15ea90 0x7f386c14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c15e5e0 0x7f386c06b490 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 --2- 172.21.15.157:0/1663457982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f386c1551e0 0x7f386c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 >> 172.21.15.157:0/1663457982 conn(0x7f386c065930 msgr2=0x7f386c15cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 shutdown_connections 2023-11-06T03:09:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.064+0000 7f3870fef640 1 -- 172.21.15.157:0/1663457982 wait complete. 2023-11-06T03:09:33.081 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:09:33.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 Processor -- start 2023-11-06T03:09:33.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 -- start start 2023-11-06T03:09:33.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f28640ce800 0x7f28640cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640cf1a0 con 0x7f2864146370 2023-11-06T03:09:33.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28640b2120 con 0x7f28640ce800 2023-11-06T03:09:33.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286bad9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286406c3d0 con 0x7f286406b6a0 2023-11-06T03:09:33.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f2869074640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f28640ce800 0x7f28640cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286a076640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286a076640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:38018/0 (socket says 172.21.15.157:38018) 2023-11-06T03:09:33.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286a076640 1 -- 172.21.15.157:0/193787709 learned_addr learned my addr 172.21.15.157:0/193787709 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:33.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f2869074640 1 -- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 msgr2=0x7f286406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:33.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f2869074640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f2869074640 1 -- 172.21.15.157:0/193787709 >> v1:172.21.15.195:6789/0 conn(0x7f2864146370 legacy=0x7f28640b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f2869074640 1 -- 172.21.15.157:0/193787709 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286406f5a0 con 0x7f28640ce800 2023-11-06T03:09:33.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.196+0000 7f286a076640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:33.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869074640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28640ce800 0x7f28640cec60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f28600097c0 tx=0x7f286001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e60b9b5873053c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2868873640 1 -- 172.21.15.157:0/193787709 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600ac620 con 0x7f28640ce800 2023-11-06T03:09:33.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2868873640 1 -- 172.21.15.157:0/193787709 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28600ac780 con 0x7f28640ce800 2023-11-06T03:09:33.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2868873640 1 -- 172.21.15.157:0/193787709 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600acb10 con 0x7f28640ce800 2023-11-06T03:09:33.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28640ce800 msgr2=0x7f28640cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28640ce800 0x7f28640cec60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f28600097c0 tx=0x7f286001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- 172.21.15.157:0/193787709 shutdown_connections 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f286406b6a0 0x7f286406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- 172.21.15.157:0/193787709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28640ce800 0x7f28640cec60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- 172.21.15.157:0/193787709 >> 172.21.15.157:0/193787709 conn(0x7f2864065930 msgr2=0x7f2864065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- 172.21.15.157:0/193787709 shutdown_connections 2023-11-06T03:09:33.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- 172.21.15.157:0/193787709 wait complete. 2023-11-06T03:09:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 Processor -- start 2023-11-06T03:09:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- start start 2023-11-06T03:09:33.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28640ce800 0x7f2864154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2864146370 0x7f2864141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28641589b0 con 0x7f286406b6a0 2023-11-06T03:09:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286415a670 con 0x7f2864146370 2023-11-06T03:09:33.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286bad9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286415a0b0 con 0x7f28640ce800 2023-11-06T03:09:33.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35554/0 (socket says 172.21.15.157:35554) 2023-11-06T03:09:33.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 -- 172.21.15.157:0/1065011108 learned_addr learned my addr 172.21.15.157:0/1065011108 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:33.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f286a076640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2864146370 0x7f2864141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869074640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28640ce800 0x7f2864154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 -- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28640ce800 msgr2=0x7f2864154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28640ce800 0x7f2864154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 -- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2864146370 msgr2=0x7f2864141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.200+0000 7f2869875640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2864146370 0x7f2864141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f2869875640 1 -- 172.21.15.157:0/1065011108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c009b30 con 0x7f286406b6a0 2023-11-06T03:09:33.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f2869875640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f285800c3c0 tx=0x7f285800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580052d0 con 0x7f286406b6a0 2023-11-06T03:09:33.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858005430 con 0x7f286406b6a0 2023-11-06T03:09:33.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2860009420 con 0x7f286406b6a0 2023-11-06T03:09:33.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858003840 con 0x7f286406b6a0 2023-11-06T03:09:33.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286413ab30 con 0x7f286406b6a0 2023-11-06T03:09:33.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864072750 con 0x7f286406b6a0 2023-11-06T03:09:33.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f28580039a0 con 0x7f286406b6a0 2023-11-06T03:09:33.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 0x7f2848067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2858082450 con 0x7f286406b6a0 2023-11-06T03:09:33.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f2869074640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 0x7f2848067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.204+0000 7f2869074640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 0x7f2848067f20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f28600ae000 tx=0x7f2860008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.208+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2858050090 con 0x7f286406b6a0 2023-11-06T03:09:33.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.352+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2864065fb0 con 0x7f286406b6a0 2023-11-06T03:09:33.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.352+0000 7f28567fc640 1 -- 172.21.15.157:0/1065011108 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+126 (secure 0 0 0) 0x7f28580536b0 con 0x7f286406b6a0 2023-11-06T03:09:33.353 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:33.354 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:09:33.354 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:33.354 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:09:33.354 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:09:33.355 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:33.355 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:09:33.355 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:09:33.355 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:09:33.355 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:33.356 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:09:33.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 msgr2=0x7f2848067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 0x7f2848067f20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f28600ae000 tx=0x7f2860008040 comp rx=0 tx=0).stop 2023-11-06T03:09:33.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 msgr2=0x7f2864157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f285800c3c0 tx=0x7f285800c880 comp rx=0 tx=0).stop 2023-11-06T03:09:33.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 shutdown_connections 2023-11-06T03:09:33.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2848065a60 0x7f2848067f20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2864146370 0x7f2864141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28640ce800 0x7f2864154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 --2- 172.21.15.157:0/1065011108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f286406b6a0 0x7f2864157b80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 >> 172.21.15.157:0/1065011108 conn(0x7f2864065930 msgr2=0x7f28640b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 shutdown_connections 2023-11-06T03:09:33.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.356+0000 7f286bad9640 1 -- 172.21.15.157:0/1065011108 wait complete. 2023-11-06T03:09:33.371 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:09:33.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:09:33.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 Processor -- start 2023-11-06T03:09:33.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 -- start start 2023-11-06T03:09:33.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f716c161260 0x7f716c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c0714b0 con 0x7f716c160930 2023-11-06T03:09:33.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c06e270 con 0x7f716c161260 2023-11-06T03:09:33.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f717189e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0708a0 con 0x7f716c161640 2023-11-06T03:09:33.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f716b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f716c160930 0x7f716c0cebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40366/0 (socket says 172.21.15.157:40366) 2023-11-06T03:09:33.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f716b7fe640 1 -- 172.21.15.157:0/3108348477 learned_addr learned my addr 172.21.15.157:0/3108348477 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:33.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f716affd640 1 --2- 172.21.15.157:0/3108348477 >> v2:172.21.15.157:3300/0 conn(0x7f716c161260 0x7f716c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f7169ffb640 1 -- 172.21.15.157:0/3108348477 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286699647 0 0) 0x7f716c0714b0 con 0x7f716c160930 2023-11-06T03:09:33.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f716a7fc640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.484+0000 7f7169ffb640 1 -- 172.21.15.157:0/3108348477 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7158003530 con 0x7f716c160930 2023-11-06T03:09:33.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f716affd640 1 -- 172.21.15.157:0/3108348477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 msgr2=0x7f716c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:33.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f716affd640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f716affd640 1 -- 172.21.15.157:0/3108348477 >> v1:172.21.15.157:6789/0 conn(0x7f716c160930 legacy=0x7f716c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f716affd640 1 -- 172.21.15.157:0/3108348477 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c070b80 con 0x7f716c161260 2023-11-06T03:09:33.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f716affd640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c06d7f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f716401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b736bc9c785963dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f7169ffb640 1 -- 172.21.15.157:0/3108348477 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640a5860 con 0x7f716c161260 2023-11-06T03:09:33.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f7169ffb640 1 -- 172.21.15.157:0/3108348477 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640a59c0 con 0x7f716c161260 2023-11-06T03:09:33.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f7169ffb640 1 -- 172.21.15.157:0/3108348477 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640a5d50 con 0x7f716c161260 2023-11-06T03:09:33.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 -- 172.21.15.157:0/3108348477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 msgr2=0x7f716c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c06d7f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f716401c890 comp rx=0 tx=0).stop 2023-11-06T03:09:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 -- 172.21.15.157:0/3108348477 shutdown_connections 2023-11-06T03:09:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 --2- 172.21.15.157:0/3108348477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c06d7f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 -- 172.21.15.157:0/3108348477 >> 172.21.15.157:0/3108348477 conn(0x7f716c06a170 msgr2=0x7f716c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 -- 172.21.15.157:0/3108348477 shutdown_connections 2023-11-06T03:09:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.488+0000 7f717189e640 1 -- 172.21.15.157:0/3108348477 wait complete. 2023-11-06T03:09:33.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 Processor -- start 2023-11-06T03:09:33.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- start start 2023-11-06T03:09:33.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 0x7f716c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0af9e0 con 0x7f716c161260 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c14c570 con 0x7f716c160930 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c14cff0 con 0x7f716c161640 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35576/0 (socket says 172.21.15.157:35576) 2023-11-06T03:09:33.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 -- 172.21.15.157:0/2480804366 learned_addr learned my addr 172.21.15.157:0/2480804366 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:33.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716affd640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 0x7f716c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716b7fe640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 -- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 msgr2=0x7f716c14a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c14a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 -- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 msgr2=0x7f716c14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 0x7f716c14c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 -- 172.21.15.157:0/2480804366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164009a70 con 0x7f716c161260 2023-11-06T03:09:33.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716affd640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 0x7f716c14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:33.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f716a7fc640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f715400d830 tx=0x7f715400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400b6c0 con 0x7f716c161260 2023-11-06T03:09:33.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400b820 con 0x7f716c161260 2023-11-06T03:09:33.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c153f50 con 0x7f716c161260 2023-11-06T03:09:33.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400bb90 con 0x7f716c161260 2023-11-06T03:09:33.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c14cc80 con 0x7f716c161260 2023-11-06T03:09:33.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c04e050 con 0x7f716c161260 2023-11-06T03:09:33.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f715400bcf0 con 0x7f716c161260 2023-11-06T03:09:33.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 0x7f714c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.492+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7154083460 con 0x7f716c161260 2023-11-06T03:09:33.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.496+0000 7f716affd640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 0x7f714c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.496+0000 7f716affd640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 0x7f714c067f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f71640af000 tx=0x7f71640a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.496+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71540510a0 con 0x7f716c161260 2023-11-06T03:09:33.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.640+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f716c0cefd0 con 0x7f716c161260 2023-11-06T03:09:33.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f7163fff640 1 -- 172.21.15.157:0/2480804366 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f71540546c0 con 0x7f716c161260 2023-11-06T03:09:33.735 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:09:33.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 msgr2=0x7f714c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 0x7f714c067f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f71640af000 tx=0x7f71640a7070 comp rx=0 tx=0).stop 2023-11-06T03:09:33.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 msgr2=0x7f716c14d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f715400d830 tx=0x7f715400dcf0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 shutdown_connections 2023-11-06T03:09:33.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f714c065a90 0x7f714c067f50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f716c161640 0x7f716c14a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c161260 0x7f716c14d480 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 --2- 172.21.15.157:0/2480804366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f716c160930 0x7f716c14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.736+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 >> 172.21.15.157:0/2480804366 conn(0x7f716c06a170 msgr2=0x7f716c15df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.740+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 shutdown_connections 2023-11-06T03:09:33.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.740+0000 7f717189e640 1 -- 172.21.15.157:0/2480804366 wait complete. 2023-11-06T03:09:33.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:09:33.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 Processor -- start 2023-11-06T03:09:33.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 -- start start 2023-11-06T03:09:33.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 0x7f5adc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5adc06d000 0x7f5adc159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc15e440 con 0x7f5adc1599c0 2023-11-06T03:09:33.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc06c240 con 0x7f5adc06d000 2023-11-06T03:09:33.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae2733640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc06c520 con 0x7f5adc0b2af0 2023-11-06T03:09:33.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5adb7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 0x7f5adc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae0cd0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5adc06d000 0x7f5adc159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5adbfff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f5adc1599c0 0x7f5adc0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:41092/0 (socket says 172.21.15.157:41092) 2023-11-06T03:09:33.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5adbfff640 1 -- 172.21.15.157:0/1359097721 learned_addr learned my addr 172.21.15.157:0/1359097721 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:33.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae0cd0640 1 -- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 msgr2=0x7f5adc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:33.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae0cd0640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 0x7f5adc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae0cd0640 1 -- 172.21.15.157:0/1359097721 >> v1:172.21.15.195:6789/0 conn(0x7f5adc1599c0 legacy=0x7f5adc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5ae0cd0640 1 -- 172.21.15.157:0/1359097721 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc06c800 con 0x7f5adc06d000 2023-11-06T03:09:33.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.888+0000 7f5adb7fe640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 0x7f5adc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:33.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae0cd0640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc159da0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ea5ea9c63371eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5adaffd640 1 -- 172.21.15.157:0/1359097721 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40a5a00 con 0x7f5adc06d000 2023-11-06T03:09:33.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5adaffd640 1 -- 172.21.15.157:0/1359097721 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad40a5b60 con 0x7f5adc06d000 2023-11-06T03:09:33.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5adaffd640 1 -- 172.21.15.157:0/1359097721 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40ad440 con 0x7f5adc06d000 2023-11-06T03:09:33.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 -- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 msgr2=0x7f5adc159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc159da0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad401c920 comp rx=0 tx=0).stop 2023-11-06T03:09:33.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 -- 172.21.15.157:0/1359097721 shutdown_connections 2023-11-06T03:09:33.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc159da0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 --2- 172.21.15.157:0/1359097721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc0b2af0 0x7f5adc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 -- 172.21.15.157:0/1359097721 >> 172.21.15.157:0/1359097721 conn(0x7f5adc065930 msgr2=0x7f5adc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:33.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 -- 172.21.15.157:0/1359097721 shutdown_connections 2023-11-06T03:09:33.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.892+0000 7f5ae2733640 1 -- 172.21.15.157:0/1359097721 wait complete. 2023-11-06T03:09:33.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 Processor -- start 2023-11-06T03:09:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- start start 2023-11-06T03:09:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc1599c0 0x7f5adc14ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc1515a0 con 0x7f5adc0b2af0 2023-11-06T03:09:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc14f470 con 0x7f5adc06d000 2023-11-06T03:09:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc154a30 con 0x7f5adc1599c0 2023-11-06T03:09:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35584/0 (socket says 172.21.15.157:35584) 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 -- 172.21.15.157:0/3324028072 learned_addr learned my addr 172.21.15.157:0/3324028072 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adbfff640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae0cd0640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc1599c0 0x7f5adc14ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 -- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc1599c0 msgr2=0x7f5adc14ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc1599c0 0x7f5adc14ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 -- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 msgr2=0x7f5adc1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 -- 172.21.15.157:0/3324028072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad0009420 con 0x7f5adc0b2af0 2023-11-06T03:09:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5adb7fe640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000ebe0 tx=0x7f5ad000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000eee0 con 0x7f5adc0b2af0 2023-11-06T03:09:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad000cc30 con 0x7f5adc0b2af0 2023-11-06T03:09:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0013470 con 0x7f5adc0b2af0 2023-11-06T03:09:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad4009a70 con 0x7f5adc0b2af0 2023-11-06T03:09:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.896+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc14f710 con 0x7f5adc0b2af0 2023-11-06T03:09:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc14f2d0 con 0x7f5adc0b2af0 2023-11-06T03:09:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ad00135d0 con 0x7f5adc0b2af0 2023-11-06T03:09:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5ad8ff9640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 0x7f5ac8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f5ad0083900 con 0x7f5adc0b2af0 2023-11-06T03:09:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5adbfff640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 0x7f5ac8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.900+0000 7f5adbfff640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 0x7f5ac8067f20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005890 tx=0x7f5acc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:33.904+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5ad00514d0 con 0x7f5adc0b2af0 2023-11-06T03:09:34.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.060+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5adc14f2d0 con 0x7f5adc0b2af0 2023-11-06T03:09:34.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.748+0000 7f5ad8ff9640 1 -- 172.21.15.157:0/3324028072 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5adc14f2d0 con 0x7f5adc0b2af0 2023-11-06T03:09:34.746 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:09:34.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 msgr2=0x7f5ac8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 0x7f5ac8067f20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005890 tx=0x7f5acc009040 comp rx=0 tx=0).stop 2023-11-06T03:09:34.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 msgr2=0x7f5adc149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000ebe0 tx=0x7f5ad000c520 comp rx=0 tx=0).stop 2023-11-06T03:09:34.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 shutdown_connections 2023-11-06T03:09:34.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5ac8065a60 0x7f5ac8067f20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5adc1599c0 0x7f5adc14ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5adc0b2af0 0x7f5adc149240 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 --2- 172.21.15.157:0/3324028072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5adc06d000 0x7f5adc1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 >> 172.21.15.157:0/3324028072 conn(0x7f5adc065930 msgr2=0x7f5adc0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:34.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 shutdown_connections 2023-11-06T03:09:34.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.776+0000 7f5ae2733640 1 -- 172.21.15.157:0/3324028072 wait complete. 2023-11-06T03:09:34.797 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:09:34.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 Processor -- start 2023-11-06T03:09:34.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 -- start start 2023-11-06T03:09:34.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f105c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0d2920 con 0x7f105c15caa0 2023-11-06T03:09:34.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0d18a0 con 0x7f105c0d00b0 2023-11-06T03:09:34.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f106269a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d0520 con 0x7f105c15ce80 2023-11-06T03:09:34.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f105c15caa0 0x7f105c0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40368/0 (socket says 172.21.15.157:40368) 2023-11-06T03:09:34.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 -- 172.21.15.157:0/426166874 learned_addr learned my addr 172.21.15.157:0/426166874 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:34.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360752816 0 0) 0x7f105c0d2920 con 0x7f105c15caa0 2023-11-06T03:09:34.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1044003530 con 0x7f105c15caa0 2023-11-06T03:09:34.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105b7fe640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f105c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735509384 0 0) 0x7f1044003530 con 0x7f105c15caa0 2023-11-06T03:09:34.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 msgr2=0x7f105c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:34.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f105c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 >> v1:172.21.15.195:6789/0 conn(0x7f105c0d00b0 legacy=0x7f105c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0d1b60 con 0x7f105c15caa0 2023-11-06T03:09:34.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1050001680 con 0x7f105c15caa0 2023-11-06T03:09:34.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 >> v1:172.21.15.157:6789/0 conn(0x7f105c15caa0 legacy=0x7f105c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f1044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f10440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1044004d10 0x7f1044005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d0520 con 0x7f10440039b0 2023-11-06T03:09:34.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104400bb20 con 0x7f1044004d10 2023-11-06T03:09:34.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104400bce0 con 0x7f105c15ce80 2023-11-06T03:09:34.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f1044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105b7fe640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f10440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f1060c37640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1044004d10 0x7f1044005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 -- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 msgr2=0x7f10440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:34.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f10440047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 -- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1044004d10 msgr2=0x7f1044005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:34.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1044004d10 0x7f1044005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105bfff640 1 -- 172.21.15.157:0/426166874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044003530 con 0x7f10440039b0 2023-11-06T03:09:34.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f105b7fe640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c15ce80 0x7f10440047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:34.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.920+0000 7f1060c37640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1044004d10 0x7f1044005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:34.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f105bfff640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f1044003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f105000c240 tx=0x7f105001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18878d9674a34bd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:34.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050005ae0 con 0x7f10440039b0 2023-11-06T03:09:34.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050005c40 con 0x7f10440039b0 2023-11-06T03:09:34.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f105affd640 1 -- 172.21.15.157:0/426166874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500a5800 con 0x7f10440039b0 2023-11-06T03:09:34.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- 172.21.15.157:0/426166874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 msgr2=0x7f1044003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 --2- 172.21.15.157:0/426166874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f1044003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f105000c240 tx=0x7f105001cc10 comp rx=0 tx=0).stop 2023-11-06T03:09:34.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f1060c37640 1 -- 172.21.15.157:0/426166874 reap_dead start 2023-11-06T03:09:34.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- 172.21.15.157:0/426166874 shutdown_connections 2023-11-06T03:09:34.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- 172.21.15.157:0/426166874 >> 172.21.15.157:0/426166874 conn(0x7f105c070790 msgr2=0x7f105c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:34.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- 172.21.15.157:0/426166874 shutdown_connections 2023-11-06T03:09:34.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- 172.21.15.157:0/426166874 wait complete. 2023-11-06T03:09:34.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 Processor -- start 2023-11-06T03:09:34.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.924+0000 7f106269a640 1 -- start start 2023-11-06T03:09:34.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 0x7f105c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c0c8550 0x7f105c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c160190 con 0x7f10440039b0 2023-11-06T03:09:34.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c15f980 con 0x7f105c15bb60 2023-11-06T03:09:34.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0c7d90 con 0x7f105c0c8550 2023-11-06T03:09:34.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35612/0 (socket says 172.21.15.157:35612) 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 -- 172.21.15.157:0/2967854211 learned_addr learned my addr 172.21.15.157:0/2967854211 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105b7fe640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 0x7f105c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1060c37640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c0c8550 0x7f105c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 -- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c0c8550 msgr2=0x7f105c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c0c8550 0x7f105c0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 -- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 msgr2=0x7f105c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:34.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 0x7f105c161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:34.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 -- 172.21.15.157:0/2967854211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0cbc90 con 0x7f10440039b0 2023-11-06T03:09:34.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105b7fe640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 0x7f105c161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:34.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105bfff640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1050000c10 tx=0x7f1044005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:34.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500b4070 con 0x7f10440039b0 2023-11-06T03:09:34.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050005e70 con 0x7f10440039b0 2023-11-06T03:09:34.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- 172.21.15.157:0/2967854211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c15fbd0 con 0x7f10440039b0 2023-11-06T03:09:34.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050001660 con 0x7f10440039b0 2023-11-06T03:09:34.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- 172.21.15.157:0/2967854211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c0c8030 con 0x7f10440039b0 2023-11-06T03:09:34.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f106269a640 1 -- 172.21.15.157:0/2967854211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c072750 con 0x7f10440039b0 2023-11-06T03:09:34.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f10500a5800 con 0x7f10440039b0 2023-11-06T03:09:34.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 0x7f1048067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:34.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f105011cf10 con 0x7f10440039b0 2023-11-06T03:09:34.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.928+0000 7f105b7fe640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 0x7f1048067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:34.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.932+0000 7f105b7fe640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 0x7f1048067f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f105c1629b0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:34.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:34.932+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f10500ea7c0 con 0x7f10440039b0 2023-11-06T03:09:35.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.124+0000 7f106269a640 1 -- 172.21.15.157:0/2967854211 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f105c0d08b0 con 0x7f10440039b0 2023-11-06T03:09:35.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.132+0000 7f1058ff9640 1 -- 172.21.15.157:0/2967854211 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f10500edde0 con 0x7f10440039b0 2023-11-06T03:09:35.137 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 11 and data pool 12 2023-11-06T03:09:35.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 msgr2=0x7f1048067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 0x7f1048067f20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f105c1629b0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2023-11-06T03:09:35.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 msgr2=0x7f105c0cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1050000c10 tx=0x7f1044005800 comp rx=0 tx=0).stop 2023-11-06T03:09:35.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 shutdown_connections 2023-11-06T03:09:35.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1048065a60 0x7f1048067f20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f105c0c8550 0x7f105c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f105c15bb60 0x7f105c161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 --2- 172.21.15.157:0/2967854211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10440039b0 0x7f105c0cb750 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 >> 172.21.15.157:0/2967854211 conn(0x7f105c070790 msgr2=0x7f105c0cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 shutdown_connections 2023-11-06T03:09:35.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.148+0000 7f10427fc640 1 -- 172.21.15.157:0/2967854211 wait complete. 2023-11-06T03:09:35.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:35.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.288+0000 7f382634c640 1 Processor -- start 2023-11-06T03:09:35.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.288+0000 7f382634c640 1 -- start start 2023-11-06T03:09:35.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3820146370 0x7f38200b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f38200cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382006b6a0 0x7f382006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38200cf1a0 con 0x7f382006b6a0 2023-11-06T03:09:35.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382006c3d0 con 0x7f3820146370 2023-11-06T03:09:35.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f382634c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382006f5a0 con 0x7f38200ce800 2023-11-06T03:09:35.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382006b6a0 0x7f382006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382006b6a0 0x7f382006bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35622/0 (socket says 172.21.15.157:35622) 2023-11-06T03:09:35.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 -- 172.21.15.157:0/3398208080 learned_addr learned my addr 172.21.15.157:0/3398208080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:35.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f381f7fe640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f38200cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f381ffff640 1 --2- 172.21.15.157:0/3398208080 >> v2:172.21.15.195:3300/0 conn(0x7f3820146370 0x7f38200b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 -- 172.21.15.157:0/3398208080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 msgr2=0x7f38200cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f38200cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 -- 172.21.15.157:0/3398208080 >> v2:172.21.15.195:3300/0 conn(0x7f3820146370 msgr2=0x7f38200b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 --2- 172.21.15.157:0/3398208080 >> v2:172.21.15.195:3300/0 conn(0x7f3820146370 0x7f38200b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 -- 172.21.15.157:0/3398208080 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382006f880 con 0x7f382006b6a0 2023-11-06T03:09:35.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f38248e9640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382006b6a0 0x7f382006bb00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f38200ae190 tx=0x7f381801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8191563ad357700 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f381effd640 1 -- 172.21.15.157:0/3398208080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180a5790 con 0x7f382006b6a0 2023-11-06T03:09:35.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f381effd640 1 -- 172.21.15.157:0/3398208080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38180a58f0 con 0x7f382006b6a0 2023-11-06T03:09:35.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.292+0000 7f381effd640 1 -- 172.21.15.157:0/3398208080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180a5c80 con 0x7f382006b6a0 2023-11-06T03:09:35.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/3398208080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382006b6a0 msgr2=0x7f382006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382006b6a0 0x7f382006bb00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f38200ae190 tx=0x7f381801c910 comp rx=0 tx=0).stop 2023-11-06T03:09:35.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/3398208080 shutdown_connections 2023-11-06T03:09:35.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382006b6a0 0x7f382006bb00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- 172.21.15.157:0/3398208080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f38200cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- 172.21.15.157:0/3398208080 >> v2:172.21.15.195:3300/0 conn(0x7f3820146370 0x7f38200b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/3398208080 >> 172.21.15.157:0/3398208080 conn(0x7f3820065930 msgr2=0x7f3820065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/3398208080 shutdown_connections 2023-11-06T03:09:35.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/3398208080 wait complete. 2023-11-06T03:09:35.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 Processor -- start 2023-11-06T03:09:35.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- start start 2023-11-06T03:09:35.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382006b6a0 0x7f38201409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f3820140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38201579c0 con 0x7f3820146370 2023-11-06T03:09:35.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382015a4a0 con 0x7f382006b6a0 2023-11-06T03:09:35.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382015a780 con 0x7f38200ce800 2023-11-06T03:09:35.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35636/0 (socket says 172.21.15.157:35636) 2023-11-06T03:09:35.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 -- 172.21.15.157:0/1948517756 learned_addr learned my addr 172.21.15.157:0/1948517756 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:35.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381ffff640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382006b6a0 0x7f38201409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381f7fe640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f3820140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 -- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 msgr2=0x7f3820140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f3820140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 -- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382006b6a0 msgr2=0x7f38201409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382006b6a0 0x7f38201409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 -- 172.21.15.157:0/1948517756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818009b30 con 0x7f3820146370 2023-11-06T03:09:35.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381f7fe640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f3820140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:35.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f38248e9640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f381801d1b0 tx=0x7f38180196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180b8670 con 0x7f3820146370 2023-11-06T03:09:35.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38180b87f0 con 0x7f3820146370 2023-11-06T03:09:35.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38201452b0 con 0x7f3820146370 2023-11-06T03:09:35.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180b8ba0 con 0x7f3820146370 2023-11-06T03:09:35.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.296+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3820145870 con 0x7f3820146370 2023-11-06T03:09:35.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3820145a60 con 0x7f3820146370 2023-11-06T03:09:35.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38180b8d00 con 0x7f3820146370 2023-11-06T03:09:35.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f381cff9640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 0x7f380c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f381811e860 con 0x7f3820146370 2023-11-06T03:09:35.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f381ffff640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 0x7f380c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.300+0000 7f381ffff640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 0x7f380c067f20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f381000b950 tx=0x7f3810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.304+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38180ec1e0 con 0x7f3820146370 2023-11-06T03:09:35.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.460+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3820072750 con 0x7f3820146370 2023-11-06T03:09:35.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.464+0000 7f381cff9640 1 -- 172.21.15.157:0/1948517756 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17947 (secure 0 0 0) 0x7f38180ef800 con 0x7f3820146370 2023-11-06T03:09:35.464 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:35.464 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":84,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:35.126824+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:35.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.464+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 msgr2=0x7f380c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.464+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 0x7f380c067f20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f381000b950 tx=0x7f3810008040 comp rx=0 tx=0).stop 2023-11-06T03:09:35.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.464+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 msgr2=0x7f3820141440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.464+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f381801d1b0 tx=0x7f38180196b0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 shutdown_connections 2023-11-06T03:09:35.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f380c065a60 0x7f380c067f20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3820146370 0x7f3820141440 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38200ce800 0x7f3820140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 --2- 172.21.15.157:0/1948517756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382006b6a0 0x7f38201409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 >> 172.21.15.157:0/1948517756 conn(0x7f3820065930 msgr2=0x7f38200b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 shutdown_connections 2023-11-06T03:09:35.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.468+0000 7f382634c640 1 -- 172.21.15.157:0/1948517756 wait complete. 2023-11-06T03:09:35.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 Processor -- start 2023-11-06T03:09:35.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 -- start start 2023-11-06T03:09:35.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdca81644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca8154380 con 0x7fdca8153fa0 2023-11-06T03:09:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca80b3ed0 con 0x7fdca80b2630 2023-11-06T03:09:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdcacaf3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80ce520 con 0x7fdca8066450 2023-11-06T03:09:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdca659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdca8153fa0 0x7fdca80cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40384/0 (socket says 172.21.15.157:40384) 2023-11-06T03:09:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdca659c640 1 -- 172.21.15.157:0/1664392774 learned_addr learned my addr 172.21.15.157:0/1664392774 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:35.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153105043 0 0) 0x7fdca8154380 con 0x7fdca8153fa0 2023-11-06T03:09:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc90003530 con 0x7fdca8153fa0 2023-11-06T03:09:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.608+0000 7fdca6d9d640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdca81644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165276635 0 0) 0x7fdc90003530 con 0x7fdca8153fa0 2023-11-06T03:09:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 msgr2=0x7fdca81644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdca81644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 >> v1:172.21.15.195:6789/0 conn(0x7fdca80b2630 legacy=0x7fdca8065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80cb200 con 0x7fdca8153fa0 2023-11-06T03:09:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdc98001680 con 0x7fdca8153fa0 2023-11-06T03:09:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 >> v1:172.21.15.157:6789/0 conn(0x7fdca8153fa0 legacy=0x7fdca80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039b0 0x7fdc90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdc900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc90004d10 0x7fdc90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80ce520 con 0x7fdc900039b0 2023-11-06T03:09:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9000bb20 con 0x7fdc90004d10 2023-11-06T03:09:35.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9000bce0 con 0x7fdca8066450 2023-11-06T03:09:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039b0 0x7fdc90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca5d9b640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdc900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca6d9d640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc90004d10 0x7fdc90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 -- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 msgr2=0x7fdc900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdca8066450 0x7fdc900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 -- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc90004d10 msgr2=0x7fdc90005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc90004d10 0x7fdc90005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 -- 172.21.15.157:0/1664392774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc90003530 con 0x7fdc900039b0 2023-11-06T03:09:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca659c640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039b0 0x7fdc90003dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c240 tx=0x7fdc9801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac6d8d19e709d6c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980a5af0 con 0x7fdc900039b0 2023-11-06T03:09:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc980a5c50 con 0x7fdc900039b0 2023-11-06T03:09:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca559a640 1 -- 172.21.15.157:0/1664392774 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980058e0 con 0x7fdc900039b0 2023-11-06T03:09:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 -- 172.21.15.157:0/1664392774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039b0 msgr2=0x7fdc90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/1664392774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc900039b0 0x7fdc90003dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c240 tx=0x7fdc9801cc10 comp rx=0 tx=0).stop 2023-11-06T03:09:35.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdca6d9d640 1 -- 172.21.15.157:0/1664392774 reap_dead start 2023-11-06T03:09:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 -- 172.21.15.157:0/1664392774 shutdown_connections 2023-11-06T03:09:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 -- 172.21.15.157:0/1664392774 >> 172.21.15.157:0/1664392774 conn(0x7fdca8065930 msgr2=0x7fdca8071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 -- 172.21.15.157:0/1664392774 shutdown_connections 2023-11-06T03:09:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.612+0000 7fdcacaf3640 1 -- 172.21.15.157:0/1664392774 wait complete. 2023-11-06T03:09:35.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 Processor -- start 2023-11-06T03:09:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- start start 2023-11-06T03:09:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 0x7fdca8151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca814fd00 0x7fdca8150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8145160 con 0x7fdca8153120 2023-11-06T03:09:35.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81506c0 con 0x7fdca814fd00 2023-11-06T03:09:35.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8151960 con 0x7fdc900039b0 2023-11-06T03:09:35.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35654/0 (socket says 172.21.15.157:35654) 2023-11-06T03:09:35.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 -- 172.21.15.157:0/3605368673 learned_addr learned my addr 172.21.15.157:0/3605368673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca6d9d640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca814fd00 0x7fdca8150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca659c640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 0x7fdca8151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 -- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 msgr2=0x7fdca8151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 0x7fdca8151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 -- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca814fd00 msgr2=0x7fdca8150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca814fd00 0x7fdca8150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 -- 172.21.15.157:0/3605368673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca81535a0 con 0x7fdca8153120 2023-11-06T03:09:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca659c640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 0x7fdca8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdca5d9b640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e580 tx=0x7fdc9c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0052e0 con 0x7fdca8153120 2023-11-06T03:09:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c005440 con 0x7fdca8153120 2023-11-06T03:09:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca815b110 con 0x7fdca8153120 2023-11-06T03:09:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c003840 con 0x7fdca8153120 2023-11-06T03:09:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca815b6d0 con 0x7fdca8153120 2023-11-06T03:09:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.616+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8072670 con 0x7fdca8153120 2023-11-06T03:09:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.620+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdc9c0039a0 con 0x7fdca8153120 2023-11-06T03:09:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.620+0000 7fdc96ffd640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 0x7fdc88067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.620+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc9c082a40 con 0x7fdca8153120 2023-11-06T03:09:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.620+0000 7fdca659c640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 0x7fdc88067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.620+0000 7fdca659c640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 0x7fdc88067f50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c240 tx=0x7fdc980a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.624+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdc9c084e60 con 0x7fdca8153120 2023-11-06T03:09:35.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.780+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdca81537d0 con 0x7fdca8153120 2023-11-06T03:09:35.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.780+0000 7fdc96ffd640 1 -- 172.21.15.157:0/3605368673 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17947 (secure 0 0 0) 0x7fdc9c050360 con 0x7fdca8153120 2023-11-06T03:09:35.781 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:35.781 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":84,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:35.126824+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:35.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 msgr2=0x7fdc88067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 0x7fdc88067f50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c240 tx=0x7fdc980a7070 comp rx=0 tx=0).stop 2023-11-06T03:09:35.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 msgr2=0x7fdca8144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e580 tx=0x7fdc9c00ea40 comp rx=0 tx=0).stop 2023-11-06T03:09:35.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 shutdown_connections 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc88065a90 0x7fdc88067f50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdca814fd00 0x7fdca8150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca8153120 0x7fdca8144c20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 --2- 172.21.15.157:0/3605368673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc900039b0 0x7fdca8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 >> 172.21.15.157:0/3605368673 conn(0x7fdca8065930 msgr2=0x7fdca815a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 shutdown_connections 2023-11-06T03:09:35.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.784+0000 7fdcacaf3640 1 -- 172.21.15.157:0/3605368673 wait complete. 2023-11-06T03:09:35.800 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:09:35.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 Processor -- start 2023-11-06T03:09:35.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 -- start start 2023-11-06T03:09:35.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59b41551e0 0x7f59b406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b415f580 con 0x7f59b415ea60 2023-11-06T03:09:35.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b415efc0 con 0x7f59b41551e0 2023-11-06T03:09:35.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59bbad9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415f2a0 con 0x7f59b415e5e0 2023-11-06T03:09:35.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59b41551e0 0x7f59b406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59ba076640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f59b415ea60 0x7f59b415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40396/0 (socket says 172.21.15.157:40396) 2023-11-06T03:09:35.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f59b41551e0 0x7f59b406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35668/0 (socket says 172.21.15.157:35668) 2023-11-06T03:09:35.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59ba076640 1 -- 172.21.15.157:0/2808076959 learned_addr learned my addr 172.21.15.157:0/2808076959 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:35.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9074640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 -- 172.21.15.157:0/2808076959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 msgr2=0x7f59b40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:35.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 -- 172.21.15.157:0/2808076959 >> v1:172.21.15.157:6789/0 conn(0x7f59b415ea60 legacy=0x7f59b415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.924+0000 7f59b9875640 1 -- 172.21.15.157:0/2808076959 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b406f5a0 con 0x7f59b41551e0 2023-11-06T03:09:35.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59b9875640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b406fda0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f59b00097c0 tx=0x7f59b001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1986895ab92da124 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59b8873640 1 -- 172.21.15.157:0/2808076959 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00ab600 con 0x7f59b41551e0 2023-11-06T03:09:35.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59b8873640 1 -- 172.21.15.157:0/2808076959 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b00a7050 con 0x7f59b41551e0 2023-11-06T03:09:35.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59b8873640 1 -- 172.21.15.157:0/2808076959 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00ab980 con 0x7f59b41551e0 2023-11-06T03:09:35.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- 172.21.15.157:0/2808076959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 msgr2=0x7f59b406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b406fda0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f59b00097c0 tx=0x7f59b001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- 172.21.15.157:0/2808076959 shutdown_connections 2023-11-06T03:09:35.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2808076959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b406fda0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- 172.21.15.157:0/2808076959 >> 172.21.15.157:0/2808076959 conn(0x7f59b4065930 msgr2=0x7f59b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:35.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- 172.21.15.157:0/2808076959 shutdown_connections 2023-11-06T03:09:35.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- 172.21.15.157:0/2808076959 wait complete. 2023-11-06T03:09:35.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 Processor -- start 2023-11-06T03:09:35.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.928+0000 7f59bbad9640 1 -- start start 2023-11-06T03:09:35.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b41422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59b415ea60 0x7f59b414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40ce2a0 con 0x7f59b41551e0 2023-11-06T03:09:35.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b414d260 con 0x7f59b415ea60 2023-11-06T03:09:35.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b414dc30 con 0x7f59b415e5e0 2023-11-06T03:09:35.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35678/0 (socket says 172.21.15.157:35678) 2023-11-06T03:09:35.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 -- 172.21.15.157:0/2714331192 learned_addr learned my addr 172.21.15.157:0/2714331192 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:35.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59ba076640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59b415ea60 0x7f59b414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9074640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b41422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 -- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 msgr2=0x7f59b41422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b41422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 -- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59b415ea60 msgr2=0x7f59b414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:35.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59b415ea60 0x7f59b414c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:35.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 -- 172.21.15.157:0/2714331192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009420 con 0x7f59b41551e0 2023-11-06T03:09:35.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9875640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f59b00ae040 tx=0x7f59b001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00b7650 con 0x7f59b41551e0 2023-11-06T03:09:35.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b00b77b0 con 0x7f59b41551e0 2023-11-06T03:09:35.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b40cdfc0 con 0x7f59b41551e0 2023-11-06T03:09:35.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00b7b60 con 0x7f59b41551e0 2023-11-06T03:09:35.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b414cef0 con 0x7f59b41551e0 2023-11-06T03:09:35.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b4072750 con 0x7f59b41551e0 2023-11-06T03:09:35.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f59b00b7cc0 con 0x7f59b41551e0 2023-11-06T03:09:35.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 0x7f5998067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:35.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59b011dc10 con 0x7f59b41551e0 2023-11-06T03:09:35.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.932+0000 7f59b9074640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 0x7f5998067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:35.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.936+0000 7f59b9074640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 0x7f5998067f50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f59b414c010 tx=0x7f59a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:35.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:35.936+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f59b00eb620 con 0x7f59b41551e0 2023-11-06T03:09:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.124+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f59b4071730 con 0x7f59b41551e0 2023-11-06T03:09:36.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59aa7fc640 1 -- 172.21.15.157:0/2714331192 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7f59b00bb0b0 con 0x7f59b41551e0 2023-11-06T03:09:36.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 msgr2=0x7f5998067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 0x7f5998067f50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f59b414c010 tx=0x7f59a4008040 comp rx=0 tx=0).stop 2023-11-06T03:09:36.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 msgr2=0x7f59b414d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f59b00ae040 tx=0x7f59b001ac00 comp rx=0 tx=0).stop 2023-11-06T03:09:36.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 shutdown_connections 2023-11-06T03:09:36.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5998065a90 0x7f5998067f50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59b415ea60 0x7f59b414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f59b415e5e0 0x7f59b41422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 --2- 172.21.15.157:0/2714331192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59b41551e0 0x7f59b414d6f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.140+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 >> 172.21.15.157:0/2714331192 conn(0x7f59b4065930 msgr2=0x7f59b40ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:36.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.144+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 shutdown_connections 2023-11-06T03:09:36.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.144+0000 7f59bbad9640 1 -- 172.21.15.157:0/2714331192 wait complete. 2023-11-06T03:09:36.158 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:09:36.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:09:36.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.280+0000 7f0dd86ec640 1 Processor -- start 2023-11-06T03:09:36.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.280+0000 7f0dd86ec640 1 -- start start 2023-11-06T03:09:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd86ec640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0dd015e5e0 0x7f0dd00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd86ec640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd015ea90 0x7f0dd00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd86ec640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd015f580 con 0x7f0dd01551e0 2023-11-06T03:09:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd86ec640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd00704b0 con 0x7f0dd015ea90 2023-11-06T03:09:36.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd86ec640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd015efc0 con 0x7f0dd015e5e0 2023-11-06T03:09:36.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd015ea90 0x7f0dd00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0dd015ea90 0x7f0dd00b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35688/0 (socket says 172.21.15.157:35688) 2023-11-06T03:09:36.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 -- 172.21.15.157:0/827555817 learned_addr learned my addr 172.21.15.157:0/827555817 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:36.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd5c87640 1 --2- 172.21.15.157:0/827555817 >> v2:172.21.15.195:3300/0 conn(0x7f0dd015e5e0 0x7f0dd00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 -- 172.21.15.157:0/827555817 >> v2:172.21.15.195:3300/0 conn(0x7f0dd015e5e0 msgr2=0x7f0dd00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 --2- 172.21.15.157:0/827555817 >> v2:172.21.15.195:3300/0 conn(0x7f0dd015e5e0 0x7f0dd00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 -- 172.21.15.157:0/827555817 >> v1:172.21.15.195:6789/0 conn(0x7f0dd01551e0 legacy=0x7f0dd006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 -- 172.21.15.157:0/827555817 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd015f2a0 con 0x7f0dd015ea90 2023-11-06T03:09:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.284+0000 7f0dd6c89640 1 --2- 172.21.15.157:0/827555817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd00b3950 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0066880 tx=0x7f0dc801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd70571d3f4bd6a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:36.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd5486640 1 -- 172.21.15.157:0/827555817 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80a57d0 con 0x7f0dd015ea90 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd5486640 1 -- 172.21.15.157:0/827555817 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc80a5930 con 0x7f0dd015ea90 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd5486640 1 -- 172.21.15.157:0/827555817 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80a5cc0 con 0x7f0dd015ea90 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/827555817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 msgr2=0x7f0dd00b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/827555817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd00b3950 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0066880 tx=0x7f0dc801c910 comp rx=0 tx=0).stop 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/827555817 shutdown_connections 2023-11-06T03:09:36.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/827555817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd00b3950 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/827555817 >> v2:172.21.15.195:3300/0 conn(0x7f0dd015e5e0 0x7f0dd00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/827555817 >> 172.21.15.157:0/827555817 conn(0x7f0dd0065930 msgr2=0x7f0dd0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:36.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/827555817 shutdown_connections 2023-11-06T03:09:36.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/827555817 wait complete. 2023-11-06T03:09:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 Processor -- start 2023-11-06T03:09:36.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- start start 2023-11-06T03:09:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 0x7f0dd014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dd015e5e0 0x7f0dd013a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd013ad10 con 0x7f0dd015ea90 2023-11-06T03:09:36.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd01502b0 con 0x7f0dd015e5e0 2023-11-06T03:09:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd013df40 con 0x7f0dd01551e0 2023-11-06T03:09:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35698/0 (socket says 172.21.15.157:35698) 2023-11-06T03:09:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 -- 172.21.15.157:0/2014792915 learned_addr learned my addr 172.21.15.157:0/2014792915 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:36.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6488640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 0x7f0dd014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd5c87640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dd015e5e0 0x7f0dd013a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 -- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 msgr2=0x7f0dd014f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 0x7f0dd014f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 -- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dd015e5e0 msgr2=0x7f0dd013a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dd015e5e0 0x7f0dd013a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:36.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 -- 172.21.15.157:0/2014792915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8009b30 con 0x7f0dd015ea90 2023-11-06T03:09:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6488640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 0x7f0dd014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd6c89640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8019520 tx=0x7f0dc80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80b8670 con 0x7f0dd015ea90 2023-11-06T03:09:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc80b87d0 con 0x7f0dd015ea90 2023-11-06T03:09:36.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd014f6a0 con 0x7f0dd015ea90 2023-11-06T03:09:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80b8b60 con 0x7f0dd015ea90 2023-11-06T03:09:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.288+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd014fc60 con 0x7f0dd015ea90 2023-11-06T03:09:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0150420 con 0x7f0dd015ea90 2023-11-06T03:09:36.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0dc80b8cc0 con 0x7f0dd015ea90 2023-11-06T03:09:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dc6ffd640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 0x7f0db4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0dc811e910 con 0x7f0dd015ea90 2023-11-06T03:09:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dd6488640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 0x7f0db4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.292+0000 7f0dd6488640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 0x7f0db4067f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0016f0 tx=0x7f0dcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:36.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.296+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0dc80ec2a0 con 0x7f0dd015ea90 2023-11-06T03:09:36.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:36.480+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0dd0072750 con 0x7f0dd015ea90 2023-11-06T03:09:36.677 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:09:36.676+0000 7fd3166e2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:09:37.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.144+0000 7f0dc6ffd640 1 -- 172.21.15.157:0/2014792915 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0dc80ef8c0 con 0x7f0dd015ea90 2023-11-06T03:09:37.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 msgr2=0x7f0db4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 0x7f0db4067f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0016f0 tx=0x7f0dcc009040 comp rx=0 tx=0).stop 2023-11-06T03:09:37.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 msgr2=0x7f0dd013da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8019520 tx=0x7f0dc80196b0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 shutdown_connections 2023-11-06T03:09:37.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0db4065a60 0x7f0db4067f20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dd015ea90 0x7f0dd013da00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0dd015e5e0 0x7f0dd013a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 --2- 172.21.15.157:0/2014792915 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0dd01551e0 0x7f0dd014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 >> 172.21.15.157:0/2014792915 conn(0x7f0dd0065930 msgr2=0x7f0dd00ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 shutdown_connections 2023-11-06T03:09:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.148+0000 7f0dd86ec640 1 -- 172.21.15.157:0/2014792915 wait complete. 2023-11-06T03:09:37.166 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:09:37.166 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:37.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.288+0000 7f3141b8d640 1 Processor -- start 2023-11-06T03:09:37.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.288+0000 7f3141b8d640 1 -- start start 2023-11-06T03:09:37.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f3141b8d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f313c066090 0x7f313c0707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f3141b8d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f313c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f3141b8d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313c06da60 con 0x7f313c0cc290 2023-11-06T03:09:37.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f3141b8d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313c070e90 con 0x7f313c066090 2023-11-06T03:09:37.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f3141b8d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313c0b3f50 con 0x7f313c15ad40 2023-11-06T03:09:37.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f313c0cc290 0x7f313c0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40402/0 (socket says 172.21.15.157:40402) 2023-11-06T03:09:37.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 -- 172.21.15.157:0/2967192692 learned_addr learned my addr 172.21.15.157:0/2967192692 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:37.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414264561 0 0) 0x7f313c06da60 con 0x7f313c0cc290 2023-11-06T03:09:37.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3124003530 con 0x7f313c0cc290 2023-11-06T03:09:37.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313bfff640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f313c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313affd640 1 --2- 172.21.15.157:0/2967192692 >> v2:172.21.15.195:3300/0 conn(0x7f313c066090 0x7f313c0707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1607985016 0 0) 0x7f3124003530 con 0x7f313c0cc290 2023-11-06T03:09:37.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 msgr2=0x7f313c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f313c0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 >> v2:172.21.15.195:3300/0 conn(0x7f313c066090 msgr2=0x7f313c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 --2- 172.21.15.157:0/2967192692 >> v2:172.21.15.195:3300/0 conn(0x7f313c066090 0x7f313c0707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0b4230 con 0x7f313c0cc290 2023-11-06T03:09:37.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f312c001680 con 0x7f313c0cc290 2023-11-06T03:09:37.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313bfff640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f313c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:37.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 >> v1:172.21.15.157:6789/0 conn(0x7f313c0cc290 legacy=0x7f313c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3124003980 0x7f3124003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f31240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3124004d10 0x7f3124005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c070e90 con 0x7f3124003980 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c0b3f50 con 0x7f3124004d10 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312400ba90 con 0x7f313c15ad40 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313affd640 1 --2- 172.21.15.157:0/2967192692 >> v2:172.21.15.195:3300/0 conn(0x7f313c066090 0x7f313c0707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3124003980 0x7f3124003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313bfff640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3124004d10 0x7f3124005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 -- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 msgr2=0x7f31240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f313c15ad40 0x7f31240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 -- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3124004d10 msgr2=0x7f3124005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3124004d10 0x7f3124005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313b7fe640 1 -- 172.21.15.157:0/2967192692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124003530 con 0x7f3124003980 2023-11-06T03:09:37.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.292+0000 7f313bfff640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3124004d10 0x7f3124005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:37.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313b7fe640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3124003980 0x7f3124003da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c240 tx=0x7f312c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1794483fb20b11a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0a5940 con 0x7f3124003980 2023-11-06T03:09:37.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c0a5aa0 con 0x7f3124003980 2023-11-06T03:09:37.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313a7fc640 1 -- 172.21.15.157:0/2967192692 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0a5e70 con 0x7f3124003980 2023-11-06T03:09:37.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/2967192692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3124003980 msgr2=0x7f3124003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 --2- 172.21.15.157:0/2967192692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3124003980 0x7f3124003da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c240 tx=0x7f312c01cbe0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313bfff640 1 -- 172.21.15.157:0/2967192692 reap_dead start 2023-11-06T03:09:37.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/2967192692 shutdown_connections 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/2967192692 >> 172.21.15.157:0/2967192692 conn(0x7f313c069d70 msgr2=0x7f313c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/2967192692 shutdown_connections 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/2967192692 wait complete. 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 Processor -- start 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- start start 2023-11-06T03:09:37.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3124003980 0x7f313c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 0x7f313c1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1589f0 con 0x7f313c139bc0 2023-11-06T03:09:37.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c159830 con 0x7f313c13a040 2023-11-06T03:09:37.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313c158c20 con 0x7f3124003980 2023-11-06T03:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35718/0 (socket says 172.21.15.157:35718) 2023-11-06T03:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 -- 172.21.15.157:0/3692229113 learned_addr learned my addr 172.21.15.157:0/3692229113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313bfff640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 0x7f313c1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313b7fe640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3124003980 0x7f313c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 -- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3124003980 msgr2=0x7f313c1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3124003980 0x7f313c1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 -- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 msgr2=0x7f313c1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 0x7f313c1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 -- 172.21.15.157:0/3692229113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c158f00 con 0x7f313c139bc0 2023-11-06T03:09:37.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313bfff640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 0x7f313c1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:37.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f313affd640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f313000e810 tx=0x7f313000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313000c8d0 con 0x7f313c139bc0 2023-11-06T03:09:37.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313000ca30 con 0x7f313c139bc0 2023-11-06T03:09:37.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313000cde0 con 0x7f313c139bc0 2023-11-06T03:09:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c1591e0 con 0x7f313c139bc0 2023-11-06T03:09:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.296+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c159cc0 con 0x7f313c139bc0 2023-11-06T03:09:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c1599a0 con 0x7f313c139bc0 2023-11-06T03:09:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3130013410 con 0x7f313c139bc0 2023-11-06T03:09:37.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f3123fff640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 0x7f3128067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3130082fb0 con 0x7f313c139bc0 2023-11-06T03:09:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f313b7fe640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 0x7f3128067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.300+0000 7f313b7fe640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 0x7f3128067f50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f312c0aa000 tx=0x7f312c001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.304+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f31300b4070 con 0x7f313c139bc0 2023-11-06T03:09:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.476+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f313c072750 con 0x7f313c139bc0 2023-11-06T03:09:37.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3123fff640 1 -- 172.21.15.157:0/3692229113 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4653 (secure 0 0 0) 0x7f31300508a0 con 0x7f313c139bc0 2023-11-06T03:09:37.478 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:37.478 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":60,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:09:37.140140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:09:37.479 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 60 2023-11-06T03:09:37.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 msgr2=0x7f3128067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 0x7f3128067f50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f312c0aa000 tx=0x7f312c001a80 comp rx=0 tx=0).stop 2023-11-06T03:09:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 msgr2=0x7f313c1f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f313000e810 tx=0x7f313000ecd0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 shutdown_connections 2023-11-06T03:09:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3128065a90 0x7f3128067f50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f313c13a040 0x7f313c1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f313c139bc0 0x7f313c1f6000 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 --2- 172.21.15.157:0/3692229113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3124003980 0x7f313c1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 >> 172.21.15.157:0/3692229113 conn(0x7f313c069d70 msgr2=0x7f313c0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 shutdown_connections 2023-11-06T03:09:37.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.480+0000 7f3141b8d640 1 -- 172.21.15.157:0/3692229113 wait complete. 2023-11-06T03:09:37.496 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:09:37.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.616+0000 7f735560e640 1 Processor -- start 2023-11-06T03:09:37.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.616+0000 7f735560e640 1 -- start start 2023-11-06T03:09:37.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73501551e0 0x7f735006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f735015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735015f580 con 0x7f735015e5e0 2023-11-06T03:09:37.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735015efc0 con 0x7f73501551e0 2023-11-06T03:09:37.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735015f2a0 con 0x7f73500b3b20 2023-11-06T03:09:37.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73501551e0 0x7f735006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f73501551e0 0x7f735006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35720/0 (socket says 172.21.15.157:35720) 2023-11-06T03:09:37.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 -- 172.21.15.157:0/3797812719 learned_addr learned my addr 172.21.15.157:0/3797812719 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:37.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734f7fe640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f735015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 -- 172.21.15.157:0/3797812719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 msgr2=0x7f735015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f735015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 -- 172.21.15.157:0/3797812719 >> v1:172.21.15.195:6789/0 conn(0x7f735015e5e0 legacy=0x7f73500b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 -- 172.21.15.157:0/3797812719 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735006f5a0 con 0x7f73501551e0 2023-11-06T03:09:37.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734effd640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73501551e0 0x7f735006fda0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9d30949584801 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734dffb640 1 -- 172.21.15.157:0/3797812719 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400ab620 con 0x7f73501551e0 2023-11-06T03:09:37.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734dffb640 1 -- 172.21.15.157:0/3797812719 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73400ab780 con 0x7f73501551e0 2023-11-06T03:09:37.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f734dffb640 1 -- 172.21.15.157:0/3797812719 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400abb10 con 0x7f73501551e0 2023-11-06T03:09:37.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- 172.21.15.157:0/3797812719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73501551e0 msgr2=0x7f735006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73501551e0 0x7f735006fda0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734001c440 comp rx=0 tx=0).stop 2023-11-06T03:09:37.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- 172.21.15.157:0/3797812719 shutdown_connections 2023-11-06T03:09:37.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f735015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 --2- 172.21.15.157:0/3797812719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73501551e0 0x7f735006fda0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- 172.21.15.157:0/3797812719 >> 172.21.15.157:0/3797812719 conn(0x7f7350065930 msgr2=0x7f7350065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- 172.21.15.157:0/3797812719 shutdown_connections 2023-11-06T03:09:37.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.620+0000 7f735560e640 1 -- 172.21.15.157:0/3797812719 wait complete. 2023-11-06T03:09:37.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 Processor -- start 2023-11-06T03:09:37.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- start start 2023-11-06T03:09:37.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f7350142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73501551e0 0x7f7350142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73501438a0 con 0x7f735015e5e0 2023-11-06T03:09:37.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7350143b80 con 0x7f73501551e0 2023-11-06T03:09:37.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7350143e60 con 0x7f73500b3b20 2023-11-06T03:09:37.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35724/0 (socket says 172.21.15.157:35724) 2023-11-06T03:09:37.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 -- 172.21.15.157:0/2381830642 learned_addr learned my addr 172.21.15.157:0/2381830642 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:37.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734effd640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f7350142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734e7fc640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73501551e0 0x7f7350142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 -- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 msgr2=0x7f7350142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f7350142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 -- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73501551e0 msgr2=0x7f7350142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73501551e0 0x7f7350142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 -- 172.21.15.157:0/2381830642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400094d0 con 0x7f735015e5e0 2023-11-06T03:09:37.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f734f7fe640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f734800cd40 tx=0x7f734800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348003ba0 con 0x7f735015e5e0 2023-11-06T03:09:37.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7348003d00 con 0x7f735015e5e0 2023-11-06T03:09:37.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7350144140 con 0x7f735015e5e0 2023-11-06T03:09:37.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348004240 con 0x7f735015e5e0 2023-11-06T03:09:37.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7350142bb0 con 0x7f735015e5e0 2023-11-06T03:09:37.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.624+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7350072750 con 0x7f735015e5e0 2023-11-06T03:09:37.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.628+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73480043a0 con 0x7f735015e5e0 2023-11-06T03:09:37.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.628+0000 7f73377fe640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 0x7f733c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.628+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7348012070 con 0x7f735015e5e0 2023-11-06T03:09:37.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.628+0000 7f734effd640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 0x7f733c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.628+0000 7f734effd640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 0x7f733c067f20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f73400af000 tx=0x7f73400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.632+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7348051560 con 0x7f735015e5e0 2023-11-06T03:09:37.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.784+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f735014bbe0 con 0x7f735015e5e0 2023-11-06T03:09:37.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f73377fe640 1 -- 172.21.15.157:0/2381830642 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17934 (secure 0 0 0) 0x7f7348054b80 con 0x7f735015e5e0 2023-11-06T03:09:37.786 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:37.787 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:09:36.131255+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:09:37.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 msgr2=0x7f733c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 0x7f733c067f20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f73400af000 tx=0x7f73400a7000 comp rx=0 tx=0).stop 2023-11-06T03:09:37.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 msgr2=0x7f7350153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f734800cd40 tx=0x7f734800a660 comp rx=0 tx=0).stop 2023-11-06T03:09:37.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 shutdown_connections 2023-11-06T03:09:37.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f733c065a60 0x7f733c067f20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f735015e5e0 0x7f7350153550 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73501551e0 0x7f7350142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 --2- 172.21.15.157:0/2381830642 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73500b3b20 0x7f7350142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 >> 172.21.15.157:0/2381830642 conn(0x7f7350065930 msgr2=0x7f735014de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 shutdown_connections 2023-11-06T03:09:37.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.788+0000 7f735560e640 1 -- 172.21.15.157:0/2381830642 wait complete. 2023-11-06T03:09:37.806 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:09:37.806 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:09:37.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.920+0000 7fa63b6d1640 1 Processor -- start 2023-11-06T03:09:37.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.920+0000 7fa63b6d1640 1 -- start start 2023-11-06T03:09:37.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 0x7fa6340adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 0x7fa6340c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6340c4230 0x7fa6341607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6340ad660 con 0x7fa6340c4230 2023-11-06T03:09:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63415d010 con 0x7fa6340ae030 2023-11-06T03:09:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63b6d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340aca50 con 0x7fa63415cc30 2023-11-06T03:09:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6340c4230 0x7fa6341607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6340c4230 0x7fa6341607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35740/0 (socket says 172.21.15.157:35740) 2023-11-06T03:09:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 -- 172.21.15.157:0/3817327765 learned_addr learned my addr 172.21.15.157:0/3817327765 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:37.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63946d640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 0x7fa6340adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 -- 172.21.15.157:0/3817327765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 msgr2=0x7fa6340adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa638c6c640 1 --2- 172.21.15.157:0/3817327765 >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 0x7fa6340c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 0x7fa6340adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 -- 172.21.15.157:0/3817327765 >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 msgr2=0x7fa6340c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 --2- 172.21.15.157:0/3817327765 >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 0x7fa6340c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 -- 172.21.15.157:0/3817327765 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340acd30 con 0x7fa6340c4230 2023-11-06T03:09:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa63946d640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 0x7fa6340adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa638c6c640 1 --2- 172.21.15.157:0/3817327765 >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 0x7fa6340c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:37.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa639c6e640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa6341607d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa63406a260 tx=0x7fa62c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cad54bcfcaa198 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa62bfff640 1 -- 172.21.15.157:0/3817327765 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0a5650 con 0x7fa6340c4230 2023-11-06T03:09:37.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa62bfff640 1 -- 172.21.15.157:0/3817327765 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c0a57b0 con 0x7fa6340c4230 2023-11-06T03:09:37.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.924+0000 7fa62bfff640 1 -- 172.21.15.157:0/3817327765 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0a5b40 con 0x7fa6340c4230 2023-11-06T03:09:37.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3817327765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 msgr2=0x7fa6341607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa6341607d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa63406a260 tx=0x7fa62c01c9d0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3817327765 shutdown_connections 2023-11-06T03:09:37.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa6341607d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3817327765 >> v2:172.21.15.195:3300/0 conn(0x7fa6340ae030 0x7fa6340c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3817327765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa63415cc30 0x7fa6340adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3817327765 >> 172.21.15.157:0/3817327765 conn(0x7fa6340ceb60 msgr2=0x7fa6340cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:37.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3817327765 shutdown_connections 2023-11-06T03:09:37.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3817327765 wait complete. 2023-11-06T03:09:37.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 Processor -- start 2023-11-06T03:09:37.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- start start 2023-11-06T03:09:37.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63415cc30 0x7fa6340cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6341f87a0 0x7fa6341f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634152730 con 0x7fa6340c4230 2023-11-06T03:09:37.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634152100 con 0x7fa63415cc30 2023-11-06T03:09:37.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6341f9310 con 0x7fa6341f87a0 2023-11-06T03:09:37.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35754/0 (socket says 172.21.15.157:35754) 2023-11-06T03:09:37.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 -- 172.21.15.157:0/3141320293 learned_addr learned my addr 172.21.15.157:0/3141320293 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:37.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa639c6e640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6341f87a0 0x7fa6341f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa638c6c640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63415cc30 0x7fa6340cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 -- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6341f87a0 msgr2=0x7fa6341f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6341f87a0 0x7fa6341f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 -- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63415cc30 msgr2=0x7fa6340cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63415cc30 0x7fa6340cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 -- 172.21.15.157:0/3141320293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c009b30 con 0x7fa6340c4230 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63946d640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c7f0 tx=0x7fa62400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624007820 con 0x7fa6340c4230 2023-11-06T03:09:37.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624007980 con 0x7fa6340c4230 2023-11-06T03:09:37.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6341f95f0 con 0x7fa6340c4230 2023-11-06T03:09:37.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624007d30 con 0x7fa6340c4230 2023-11-06T03:09:37.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.928+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa634150140 con 0x7fa6340c4230 2023-11-06T03:09:37.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa634072750 con 0x7fa6340c4230 2023-11-06T03:09:37.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6240113f0 con 0x7fa6340c4230 2023-11-06T03:09:37.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa629ffb640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 0x7fa618067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:37.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa624083610 con 0x7fa6340c4230 2023-11-06T03:09:37.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa638c6c640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 0x7fa618067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:37.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.932+0000 7fa638c6c640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 0x7fa618067f20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa6340b4da0 tx=0x7fa630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:37.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:37.936+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa624050e70 con 0x7fa6340c4230 2023-11-06T03:09:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.076+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6341615b0 con 0x7fa618065a60 2023-11-06T03:09:38.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.096+0000 7fa629ffb640 1 -- 172.21.15.157:0/3141320293 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299859 (secure 0 0 0) 0x7fa63000c050 con 0x7fa618065a60 2023-11-06T03:09:38.098 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:38.101 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:09:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 msgr2=0x7fa618067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 0x7fa618067f20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa6340b4da0 tx=0x7fa630008040 comp rx=0 tx=0).stop 2023-11-06T03:09:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 msgr2=0x7fa63414fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c7f0 tx=0x7fa62400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 shutdown_connections 2023-11-06T03:09:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa618065a60 0x7fa618067f20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6341f87a0 0x7fa6341f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa63415cc30 0x7fa6340cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 --2- 172.21.15.157:0/3141320293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6340c4230 0x7fa63414fc00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 >> 172.21.15.157:0/3141320293 conn(0x7fa6340ceb60 msgr2=0x7fa63406f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 shutdown_connections 2023-11-06T03:09:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.100+0000 7fa63b6d1640 1 -- 172.21.15.157:0/3141320293 wait complete. 2023-11-06T03:09:38.116 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2023-11-06T03:09:37.213932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361436,"kb_used_data":8860,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754613284,"statfs":{"total":773094113280,"available":772724002816,"internally_reserved":0,"allocated":9072640,"data_stored":4750241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001109"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743534+0000","last_change":"2023-11-06T03:09:33.743534+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743534+0000","last_undegraded":"2023-11-06T03:09:33.743534+0000","last_fullsized":"2023-11-06T03:09:33.743534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746611+0000","last_change":"2023-11-06T03:09:33.746611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746611+0000","last_undegraded":"2023-11-06T03:09:33.746611+0000","last_fullsized":"2023-11-06T03:09:33.746611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.751332+0000","last_change":"2023-11-06T03:09:33.751332+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.751332+0000","last_undegraded":"2023-11-06T03:09:33.751332+0000","last_fullsized":"2023-11-06T03:09:33.751332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.749455+0000","last_change":"2023-11-06T03:09:33.749455+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.749455+0000","last_undegraded":"2023-11-06T03:09:33.749455+0000","last_fullsized":"2023-11-06T03:09:33.749455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744013+0000","last_change":"2023-11-06T03:09:33.744013+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744013+0000","last_undegraded":"2023-11-06T03:09:33.744013+0000","last_fullsized":"2023-11-06T03:09:33.744013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755195+0000","last_change":"2023-11-06T03:09:33.755195+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755195+0000","last_undegraded":"2023-11-06T03:09:33.755195+0000","last_fullsized":"2023-11-06T03:09:33.755195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740702+0000","last_change":"2023-11-06T03:09:33.740702+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740702+0000","last_undegraded":"2023-11-06T03:09:33.740702+0000","last_fullsized":"2023-11-06T03:09:33.740702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743395+0000","last_change":"2023-11-06T03:09:33.743395+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743395+0000","last_undegraded":"2023-11-06T03:09:33.743395+0000","last_fullsized":"2023-11-06T03:09:33.743395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748920+0000","last_change":"2023-11-06T03:09:33.748920+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748920+0000","last_undegraded":"2023-11-06T03:09:33.748920+0000","last_fullsized":"2023-11-06T03:09:33.748920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755232+0000","last_change":"2023-11-06T03:09:33.755232+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755232+0000","last_undegraded":"2023-11-06T03:09:33.755232+0000","last_fullsized":"2023-11-06T03:09:33.755232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744368+0000","last_change":"2023-11-06T03:09:33.744368+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744368+0000","last_undegraded":"2023-11-06T03:09:33.744368+0000","last_fullsized":"2023-11-06T03:09:33.744368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748273+0000","last_change":"2023-11-06T03:09:33.748273+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748273+0000","last_undegraded":"2023-11-06T03:09:33.748273+0000","last_fullsized":"2023-11-06T03:09:33.748273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755163+0000","last_change":"2023-11-06T03:09:33.755163+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755163+0000","last_undegraded":"2023-11-06T03:09:33.755163+0000","last_fullsized":"2023-11-06T03:09:33.755163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746227+0000","last_change":"2023-11-06T03:09:33.746227+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746227+0000","last_undegraded":"2023-11-06T03:09:33.746227+0000","last_fullsized":"2023-11-06T03:09:33.746227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744803+0000","last_change":"2023-11-06T03:09:33.744803+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744803+0000","last_undegraded":"2023-11-06T03:09:33.744803+0000","last_fullsized":"2023-11-06T03:09:33.744803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740711+0000","last_change":"2023-11-06T03:09:33.740711+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740711+0000","last_undegraded":"2023-11-06T03:09:33.740711+0000","last_fullsized":"2023-11-06T03:09:33.740711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.756784+0000","last_change":"2023-11-06T03:09:33.756784+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.756784+0000","last_undegraded":"2023-11-06T03:09:33.756784+0000","last_fullsized":"2023-11-06T03:09:33.756784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740826+0000","last_change":"2023-11-06T03:09:33.740826+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740826+0000","last_undegraded":"2023-11-06T03:09:33.740826+0000","last_fullsized":"2023-11-06T03:09:33.740826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.750167+0000","last_change":"2023-11-06T03:09:33.750167+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.750167+0000","last_undegraded":"2023-11-06T03:09:33.750167+0000","last_fullsized":"2023-11-06T03:09:33.750167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748000+0000","last_change":"2023-11-06T03:09:33.748000+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748000+0000","last_undegraded":"2023-11-06T03:09:33.748000+0000","last_fullsized":"2023-11-06T03:09:33.748000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746515+0000","last_change":"2023-11-06T03:09:33.746515+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746515+0000","last_undegraded":"2023-11-06T03:09:33.746515+0000","last_fullsized":"2023-11-06T03:09:33.746515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743986+0000","last_change":"2023-11-06T03:09:33.743986+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743986+0000","last_undegraded":"2023-11-06T03:09:33.743986+0000","last_fullsized":"2023-11-06T03:09:33.743986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746372+0000","last_change":"2023-11-06T03:09:33.746372+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746372+0000","last_undegraded":"2023-11-06T03:09:33.746372+0000","last_fullsized":"2023-11-06T03:09:33.746372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747573+0000","last_change":"2023-11-06T03:09:33.747573+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747573+0000","last_undegraded":"2023-11-06T03:09:33.747573+0000","last_fullsized":"2023-11-06T03:09:33.747573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746778+0000","last_change":"2023-11-06T03:09:33.746778+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746778+0000","last_undegraded":"2023-11-06T03:09:33.746778+0000","last_fullsized":"2023-11-06T03:09:33.746778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884198+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:09:31.884198+0000","last_peered":"2023-11-06T03:09:31.884198+0000","last_clean":"2023-11-06T03:09:31.884198+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:09:31.884198+0000","last_undegraded":"2023-11-06T03:09:31.884198+0000","last_fullsized":"2023-11-06T03:09:31.884198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450714+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:09:33.450714+0000","last_peered":"2023-11-06T03:09:33.450714+0000","last_clean":"2023-11-06T03:09:33.450714+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:09:33.450714+0000","last_undegraded":"2023-11-06T03:09:33.450714+0000","last_fullsized":"2023-11-06T03:09:33.450714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746590+0000","last_change":"2023-11-06T03:09:33.746590+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746590+0000","last_undegraded":"2023-11-06T03:09:33.746590+0000","last_fullsized":"2023-11-06T03:09:33.746590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884217+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:09:31.884217+0000","last_peered":"2023-11-06T03:09:31.884217+0000","last_clean":"2023-11-06T03:09:31.884217+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:09:31.884217+0000","last_undegraded":"2023-11-06T03:09:31.884217+0000","last_fullsized":"2023-11-06T03:09:31.884217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746391+0000","last_change":"2023-11-06T03:09:33.746391+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746391+0000","last_undegraded":"2023-11-06T03:09:33.746391+0000","last_fullsized":"2023-11-06T03:09:33.746391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:32.634286+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:09:32.634286+0000","last_peered":"2023-11-06T03:09:32.634286+0000","last_clean":"2023-11-06T03:09:32.634286+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:09:32.634286+0000","last_undegraded":"2023-11-06T03:09:32.634286+0000","last_fullsized":"2023-11-06T03:09:32.634286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746134+0000","last_change":"2023-11-06T03:09:33.746134+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746134+0000","last_undegraded":"2023-11-06T03:09:33.746134+0000","last_fullsized":"2023-11-06T03:09:33.746134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450705+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:09:33.450705+0000","last_peered":"2023-11-06T03:09:33.450705+0000","last_clean":"2023-11-06T03:09:33.450705+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:09:33.450705+0000","last_undegraded":"2023-11-06T03:09:33.450705+0000","last_fullsized":"2023-11-06T03:09:33.450705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743578+0000","last_change":"2023-11-06T03:09:33.743578+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743578+0000","last_undegraded":"2023-11-06T03:09:33.743578+0000","last_fullsized":"2023-11-06T03:09:33.743578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737288+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:09:31.737288+0000","last_peered":"2023-11-06T03:09:31.737288+0000","last_clean":"2023-11-06T03:09:31.737288+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:09:31.737288+0000","last_undegraded":"2023-11-06T03:09:31.737288+0000","last_fullsized":"2023-11-06T03:09:31.737288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746188+0000","last_change":"2023-11-06T03:09:33.746188+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746188+0000","last_undegraded":"2023-11-06T03:09:33.746188+0000","last_fullsized":"2023-11-06T03:09:33.746188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884251+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:09:31.884251+0000","last_peered":"2023-11-06T03:09:31.884251+0000","last_clean":"2023-11-06T03:09:31.884251+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:09:31.884251+0000","last_undegraded":"2023-11-06T03:09:31.884251+0000","last_fullsized":"2023-11-06T03:09:31.884251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747611+0000","last_change":"2023-11-06T03:09:33.747611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747611+0000","last_undegraded":"2023-11-06T03:09:33.747611+0000","last_fullsized":"2023-11-06T03:09:33.747611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737274+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:09:31.737274+0000","last_peered":"2023-11-06T03:09:31.737274+0000","last_clean":"2023-11-06T03:09:31.737274+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:09:31.737274+0000","last_undegraded":"2023-11-06T03:09:31.737274+0000","last_fullsized":"2023-11-06T03:09:31.737274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743765+0000","last_change":"2023-11-06T03:09:33.743765+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743765+0000","last_undegraded":"2023-11-06T03:09:33.743765+0000","last_fullsized":"2023-11-06T03:09:33.743765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450520+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:09:33.450520+0000","last_peered":"2023-11-06T03:09:33.450520+0000","last_clean":"2023-11-06T03:09:33.450520+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:09:33.450520+0000","last_undegraded":"2023-11-06T03:09:33.450520+0000","last_fullsized":"2023-11-06T03:09:33.450520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1581056,"data_stored":1036744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.397,"15min":1.397},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.86099999999999999},"min":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":1.444,"15min":1.444},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.86899999999999999,"15min":0.89600000000000002},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.877,"15min":0.877},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.381,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.216},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.879,"15min":0.95299999999999996},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0629999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.127,"15min":1.0669999999999999},"min":{"1min":0.65600000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":10.596,"5min":10.596,"15min":10.596},"last":1.302}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92500000000000004,"15min":0.97299999999999998},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":1.01},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.282,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.042,"15min":1.05},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.355},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.095},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3240000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79600000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.51,"5min":1.774,"15min":1.774},"last":1.252},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.099,"15min":1.117},"min":{"1min":0.90800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.1830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45960,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325880,"statfs":{"total":96636764160,"available":96589701120,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":0.96199999999999997,"15min":0.84899999999999998},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":3.0750000000000002,"5min":1.256,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":40.905000000000001,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.048,"15min":0.89700000000000002},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":16.012,"5min":16.012,"15min":16.012},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.02,"5min":0.89400000000000002,"15min":0.879},"min":{"1min":0.60499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":1.1120000000000001,"15min":0.90600000000000003},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":27.803999999999998,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.627},{"interface":"front","average":{"1min":1.004,"5min":0.96999999999999997,"15min":0.95699999999999996},"min":{"1min":0.64000000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.621,"15min":1.1919999999999999},"min":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":63.737000000000002,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":3.121,"5min":1.365,"15min":1.0449999999999999},"min":{"1min":0.57199999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":41.079999999999998,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.282,"15min":1.1839999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":24.867999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.359,"15min":1.2370000000000001},"min":{"1min":0.76700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":23.789000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":0.94499999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":1.4359999999999999,"15min":1.1779999999999999},"min":{"1min":0.71599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":40.959000000000003,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1479999999999999,"15min":1.1120000000000001},"min":{"1min":0.90400000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":1.3460000000000001,"15min":1.1899999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.407,"15min":1.22},"min":{"1min":0.81100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":28.164000000000001,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":950272,"data_stored":429420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0580000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81499999999999995,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.145,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2230000000000001,"5min":1.357,"15min":1.357},"last":0.85599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.84199999999999997},"min":{"1min":0.66400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1970000000000001,"5min":1.403,"15min":1.403},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94999999999999996,"15min":0.93600000000000005},"min":{"1min":0.72799999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":1.117}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.91900000000000004},"min":{"1min":0.64900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1060000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2330000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.014,"15min":0.99399999999999999},"min":{"1min":0.63800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.206,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.306,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0269999999999999,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.389,"5min":1.476,"15min":1.476},"last":1.087},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0940000000000001,"15min":1.0840000000000001},"min":{"1min":1.008,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2909999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.82299999999999995},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1619999999999999,"5min":1.484,"15min":1.484},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2989999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.755},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1930000000000001,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85499999999999998,"15min":0.81200000000000006},"min":{"1min":0.55300000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.91600000000000004,"15min":1.04},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.754},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1539999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1919999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.71899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.069,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.365,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5489999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.034},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.1519999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.411,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.095}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.073},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3009999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.139},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.2430000000000001,"15min":1.1870000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6739999999999999,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2749999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50056,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321784,"statfs":{"total":96636764160,"available":96585506816,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.248,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.26700000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.90400000000000003,"5min":1.524,"15min":1.524},"last":0.47999999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1080000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.042,"5min":1.349,"15min":1.349},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.518,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94599999999999995,"15min":0.98399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.536,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.66400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5049999999999999,"5min":1.698,"15min":1.698},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97599999999999998,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4610000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94899999999999995,"15min":1},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.391,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.008,"15min":1.056},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3180000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.024,"15min":1.0649999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4930000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.95299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.24299999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.314},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.255,"5min":0.223,"15min":0.223},"max":{"1min":1.0109999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.62,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.121},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.629,"15min":0.66900000000000004},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.1259999999999999,"5min":1.294,"15min":1.294},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.89000000000000001},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":0.95499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.88400000000000001},"min":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.214,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.626}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.615,"15min":1.615},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.95199999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.006,"15min":1},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.93100000000000005,"15min":0.96199999999999997},"min":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.256,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56299999999999994,"15min":0.59199999999999997},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":1.0629999999999999,"5min":1.157,"15min":1.157},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.26900000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0649999999999999,"5min":1.544,"15min":1.544},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.90700000000000003,"15min":0.71999999999999997},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63900000000000001,"15min":0.68300000000000005},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.03,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1200000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.72599999999999998},"min":{"1min":0.35199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90600000000000003,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.495}]},{"osd":4,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.2809999999999999,"5min":2.258,"15min":2.258},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":14.753,"15min":14.753},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.3919999999999999,"15min":1.1399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":1.367,"15min":1.167},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.272,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.177,"15min":1.0760000000000001},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":23.971,"15min":23.971},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2949999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0149999999999999,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3560000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.79400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1576960,"data_stored":1035725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.28299999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.145,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.69899999999999995,"15min":0.65100000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1120000000000001,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83199999999999996,"5min":1.155,"15min":1.155},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.68500000000000005,"15min":0.65400000000000003},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.876,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.29899999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0700000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.755},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0589999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.96299999999999997,"15min":0.92700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1299999999999999,"5min":4.641,"15min":4.641},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.88500000000000001,"15min":0.93100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1,"5min":2.028,"15min":2.028},"last":0.83999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1140000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.94699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2050000000000001,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.871}]},{"osd":6,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":2.931,"15min":2.931},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.248,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.0289999999999999,"15min":1.0089999999999999},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2649999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.98699999999999999,"15min":0.99199999999999999},"min":{"1min":0.64800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.97,"15min":1.97},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:09:38.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:09:38.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.264+0000 7fd7d5e11640 1 Processor -- start 2023-11-06T03:09:38.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7d5e11640 1 -- start start 2023-11-06T03:09:38.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7d5e11640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 0x7fd7d015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7d5e11640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d015f580 con 0x7fd7d015e5e0 2023-11-06T03:09:38.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7d5e11640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d015efc0 con 0x7fd7d01551e0 2023-11-06T03:09:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7d5e11640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00704b0 con 0x7fd7d00b3b20 2023-11-06T03:09:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd7d015e5e0 0x7fd7d00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:40418/0 (socket says 172.21.15.157:40418) 2023-11-06T03:09:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 -- 172.21.15.157:0/3909867332 learned_addr learned my addr 172.21.15.157:0/3909867332 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7cffff640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 0x7fd7d015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677493460 0 0) 0x7fd7d015f580 con 0x7fd7d015e5e0 2023-11-06T03:09:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc003530 con 0x7fd7d015e5e0 2023-11-06T03:09:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665122270 0 0) 0x7fd7bc003530 con 0x7fd7d015e5e0 2023-11-06T03:09:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 msgr2=0x7fd7d015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 0x7fd7d015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 >> v1:172.21.15.195:6789/0 conn(0x7fd7d01551e0 legacy=0x7fd7d006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d015f2a0 con 0x7fd7d015e5e0 2023-11-06T03:09:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd7b8001680 con 0x7fd7d015e5e0 2023-11-06T03:09:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 >> v1:172.21.15.157:6789/0 conn(0x7fd7d015e5e0 legacy=0x7fd7d00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0039b0 0x7fd7bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7bc0047d0 0x7fd7bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 0x7fd7bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00704b0 con 0x7fd7bc0047d0 2023-11-06T03:09:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc00baf0 con 0x7fd7bc0039b0 2023-11-06T03:09:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc00bcb0 con 0x7fd7d00b3b20 2023-11-06T03:09:38.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7bc0047d0 0x7fd7bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7cf7fe640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0039b0 0x7fd7bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 -- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 msgr2=0x7fd7bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00b3b20 0x7fd7bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 -- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0039b0 msgr2=0x7fd7bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7bc0039b0 0x7fd7bc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.268+0000 7fd7c6ffd640 1 -- 172.21.15.157:0/3909867332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc003530 con 0x7fd7bc0047d0 2023-11-06T03:09:38.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7bc0047d0 0x7fd7bc004c50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c240 tx=0x7fd7b801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93539f96610cd80 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80a5af0 con 0x7fd7bc0047d0 2023-11-06T03:09:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b80a5c50 con 0x7fd7bc0047d0 2023-11-06T03:09:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ceffd640 1 -- 172.21.15.157:0/3909867332 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80058e0 con 0x7fd7bc0047d0 2023-11-06T03:09:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/3909867332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7bc0047d0 msgr2=0x7fd7bc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/3909867332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7bc0047d0 0x7fd7bc004c50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c240 tx=0x7fd7b801cc10 comp rx=0 tx=0).stop 2023-11-06T03:09:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 -- 172.21.15.157:0/3909867332 reap_dead start 2023-11-06T03:09:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/3909867332 shutdown_connections 2023-11-06T03:09:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/3909867332 >> 172.21.15.157:0/3909867332 conn(0x7fd7d0065930 msgr2=0x7fd7d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/3909867332 shutdown_connections 2023-11-06T03:09:38.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/3909867332 wait complete. 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 Processor -- start 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- start start 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 0x7fd7d0146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 0x7fd7d00af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00aff00 con 0x7fd7d00afa50 2023-11-06T03:09:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0149410 con 0x7fd7d00ce950 2023-11-06T03:09:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0148890 con 0x7fd7d00ced30 2023-11-06T03:09:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35778/0 (socket says 172.21.15.157:35778) 2023-11-06T03:09:38.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 -- 172.21.15.157:0/1817754063 learned_addr learned my addr 172.21.15.157:0/1817754063 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cf7fe640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 0x7fd7d0146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 0x7fd7d00af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 -- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 msgr2=0x7fd7d00af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 0x7fd7d00af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 -- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 msgr2=0x7fd7d0146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 0x7fd7d0146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 -- 172.21.15.157:0/1817754063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0148b70 con 0x7fd7d00afa50 2023-11-06T03:09:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cf7fe640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 0x7fd7d0146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7c6ffd640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 0x7fd7d00af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:38.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7cffff640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00cf420 tx=0x7fd7c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8010040 con 0x7fd7d00afa50 2023-11-06T03:09:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c800bae0 con 0x7fd7d00afa50 2023-11-06T03:09:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0148e50 con 0x7fd7d00afa50 2023-11-06T03:09:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800be90 con 0x7fd7d00afa50 2023-11-06T03:09:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.272+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d0148540 con 0x7fd7d00afa50 2023-11-06T03:09:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d00ccce0 con 0x7fd7d00afa50 2023-11-06T03:09:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7c80039a0 con 0x7fd7d00afa50 2023-11-06T03:09:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7ccff9640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 0x7fd7b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7c80835a0 con 0x7fd7d00afa50 2023-11-06T03:09:38.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7cf7fe640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 0x7fd7b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.276+0000 7fd7cf7fe640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 0x7fd7b0067f50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0001600 tx=0x7fd7c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.280+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7d00ccce0 con 0x7fd7d00afa50 2023-11-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.420+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7d015c9c0 con 0x7fd7b0065a90 2023-11-06T03:09:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.444+0000 7fd7ccff9640 1 -- 172.21.15.157:0/1817754063 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299859 (secure 0 0 0) 0x7fd7c000d040 con 0x7fd7b0065a90 2023-11-06T03:09:38.444 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:38.447 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:09:38.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 msgr2=0x7fd7b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 0x7fd7b0067f50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0001600 tx=0x7fd7c0009040 comp rx=0 tx=0).stop 2023-11-06T03:09:38.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 msgr2=0x7fd7d014cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00cf420 tx=0x7fd7c800b480 comp rx=0 tx=0).stop 2023-11-06T03:09:38.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 shutdown_connections 2023-11-06T03:09:38.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7b0065a90 0x7fd7b0067f50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d00afa50 0x7fd7d014cbb0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7d00ced30 0x7fd7d00af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 --2- 172.21.15.157:0/1817754063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7d00ce950 0x7fd7d0146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 >> 172.21.15.157:0/1817754063 conn(0x7fd7d0065930 msgr2=0x7fd7d00b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:38.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 shutdown_connections 2023-11-06T03:09:38.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.448+0000 7fd7d5e11640 1 -- 172.21.15.157:0/1817754063 wait complete. 2023-11-06T03:09:38.463 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2023-11-06T03:09:37.213932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361436,"kb_used_data":8860,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754613284,"statfs":{"total":773094113280,"available":772724002816,"internally_reserved":0,"allocated":9072640,"data_stored":4750241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001109"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743534+0000","last_change":"2023-11-06T03:09:33.743534+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743534+0000","last_undegraded":"2023-11-06T03:09:33.743534+0000","last_fullsized":"2023-11-06T03:09:33.743534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746611+0000","last_change":"2023-11-06T03:09:33.746611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746611+0000","last_undegraded":"2023-11-06T03:09:33.746611+0000","last_fullsized":"2023-11-06T03:09:33.746611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.751332+0000","last_change":"2023-11-06T03:09:33.751332+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.751332+0000","last_undegraded":"2023-11-06T03:09:33.751332+0000","last_fullsized":"2023-11-06T03:09:33.751332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.749455+0000","last_change":"2023-11-06T03:09:33.749455+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.749455+0000","last_undegraded":"2023-11-06T03:09:33.749455+0000","last_fullsized":"2023-11-06T03:09:33.749455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744013+0000","last_change":"2023-11-06T03:09:33.744013+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744013+0000","last_undegraded":"2023-11-06T03:09:33.744013+0000","last_fullsized":"2023-11-06T03:09:33.744013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755195+0000","last_change":"2023-11-06T03:09:33.755195+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755195+0000","last_undegraded":"2023-11-06T03:09:33.755195+0000","last_fullsized":"2023-11-06T03:09:33.755195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740702+0000","last_change":"2023-11-06T03:09:33.740702+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740702+0000","last_undegraded":"2023-11-06T03:09:33.740702+0000","last_fullsized":"2023-11-06T03:09:33.740702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743395+0000","last_change":"2023-11-06T03:09:33.743395+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743395+0000","last_undegraded":"2023-11-06T03:09:33.743395+0000","last_fullsized":"2023-11-06T03:09:33.743395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748920+0000","last_change":"2023-11-06T03:09:33.748920+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748920+0000","last_undegraded":"2023-11-06T03:09:33.748920+0000","last_fullsized":"2023-11-06T03:09:33.748920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755232+0000","last_change":"2023-11-06T03:09:33.755232+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755232+0000","last_undegraded":"2023-11-06T03:09:33.755232+0000","last_fullsized":"2023-11-06T03:09:33.755232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744368+0000","last_change":"2023-11-06T03:09:33.744368+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744368+0000","last_undegraded":"2023-11-06T03:09:33.744368+0000","last_fullsized":"2023-11-06T03:09:33.744368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748273+0000","last_change":"2023-11-06T03:09:33.748273+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748273+0000","last_undegraded":"2023-11-06T03:09:33.748273+0000","last_fullsized":"2023-11-06T03:09:33.748273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755163+0000","last_change":"2023-11-06T03:09:33.755163+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755163+0000","last_undegraded":"2023-11-06T03:09:33.755163+0000","last_fullsized":"2023-11-06T03:09:33.755163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746227+0000","last_change":"2023-11-06T03:09:33.746227+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746227+0000","last_undegraded":"2023-11-06T03:09:33.746227+0000","last_fullsized":"2023-11-06T03:09:33.746227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744803+0000","last_change":"2023-11-06T03:09:33.744803+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744803+0000","last_undegraded":"2023-11-06T03:09:33.744803+0000","last_fullsized":"2023-11-06T03:09:33.744803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740711+0000","last_change":"2023-11-06T03:09:33.740711+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740711+0000","last_undegraded":"2023-11-06T03:09:33.740711+0000","last_fullsized":"2023-11-06T03:09:33.740711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.756784+0000","last_change":"2023-11-06T03:09:33.756784+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.756784+0000","last_undegraded":"2023-11-06T03:09:33.756784+0000","last_fullsized":"2023-11-06T03:09:33.756784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740826+0000","last_change":"2023-11-06T03:09:33.740826+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740826+0000","last_undegraded":"2023-11-06T03:09:33.740826+0000","last_fullsized":"2023-11-06T03:09:33.740826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.750167+0000","last_change":"2023-11-06T03:09:33.750167+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.750167+0000","last_undegraded":"2023-11-06T03:09:33.750167+0000","last_fullsized":"2023-11-06T03:09:33.750167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748000+0000","last_change":"2023-11-06T03:09:33.748000+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748000+0000","last_undegraded":"2023-11-06T03:09:33.748000+0000","last_fullsized":"2023-11-06T03:09:33.748000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746515+0000","last_change":"2023-11-06T03:09:33.746515+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746515+0000","last_undegraded":"2023-11-06T03:09:33.746515+0000","last_fullsized":"2023-11-06T03:09:33.746515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743986+0000","last_change":"2023-11-06T03:09:33.743986+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743986+0000","last_undegraded":"2023-11-06T03:09:33.743986+0000","last_fullsized":"2023-11-06T03:09:33.743986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746372+0000","last_change":"2023-11-06T03:09:33.746372+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746372+0000","last_undegraded":"2023-11-06T03:09:33.746372+0000","last_fullsized":"2023-11-06T03:09:33.746372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747573+0000","last_change":"2023-11-06T03:09:33.747573+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747573+0000","last_undegraded":"2023-11-06T03:09:33.747573+0000","last_fullsized":"2023-11-06T03:09:33.747573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746778+0000","last_change":"2023-11-06T03:09:33.746778+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746778+0000","last_undegraded":"2023-11-06T03:09:33.746778+0000","last_fullsized":"2023-11-06T03:09:33.746778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884198+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:09:31.884198+0000","last_peered":"2023-11-06T03:09:31.884198+0000","last_clean":"2023-11-06T03:09:31.884198+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:09:31.884198+0000","last_undegraded":"2023-11-06T03:09:31.884198+0000","last_fullsized":"2023-11-06T03:09:31.884198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450714+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:09:33.450714+0000","last_peered":"2023-11-06T03:09:33.450714+0000","last_clean":"2023-11-06T03:09:33.450714+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:09:33.450714+0000","last_undegraded":"2023-11-06T03:09:33.450714+0000","last_fullsized":"2023-11-06T03:09:33.450714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746590+0000","last_change":"2023-11-06T03:09:33.746590+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746590+0000","last_undegraded":"2023-11-06T03:09:33.746590+0000","last_fullsized":"2023-11-06T03:09:33.746590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884217+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:09:31.884217+0000","last_peered":"2023-11-06T03:09:31.884217+0000","last_clean":"2023-11-06T03:09:31.884217+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:09:31.884217+0000","last_undegraded":"2023-11-06T03:09:31.884217+0000","last_fullsized":"2023-11-06T03:09:31.884217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746391+0000","last_change":"2023-11-06T03:09:33.746391+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746391+0000","last_undegraded":"2023-11-06T03:09:33.746391+0000","last_fullsized":"2023-11-06T03:09:33.746391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:32.634286+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:09:32.634286+0000","last_peered":"2023-11-06T03:09:32.634286+0000","last_clean":"2023-11-06T03:09:32.634286+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:09:32.634286+0000","last_undegraded":"2023-11-06T03:09:32.634286+0000","last_fullsized":"2023-11-06T03:09:32.634286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746134+0000","last_change":"2023-11-06T03:09:33.746134+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746134+0000","last_undegraded":"2023-11-06T03:09:33.746134+0000","last_fullsized":"2023-11-06T03:09:33.746134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450705+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:09:33.450705+0000","last_peered":"2023-11-06T03:09:33.450705+0000","last_clean":"2023-11-06T03:09:33.450705+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:09:33.450705+0000","last_undegraded":"2023-11-06T03:09:33.450705+0000","last_fullsized":"2023-11-06T03:09:33.450705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743578+0000","last_change":"2023-11-06T03:09:33.743578+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743578+0000","last_undegraded":"2023-11-06T03:09:33.743578+0000","last_fullsized":"2023-11-06T03:09:33.743578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737288+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:09:31.737288+0000","last_peered":"2023-11-06T03:09:31.737288+0000","last_clean":"2023-11-06T03:09:31.737288+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:09:31.737288+0000","last_undegraded":"2023-11-06T03:09:31.737288+0000","last_fullsized":"2023-11-06T03:09:31.737288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746188+0000","last_change":"2023-11-06T03:09:33.746188+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746188+0000","last_undegraded":"2023-11-06T03:09:33.746188+0000","last_fullsized":"2023-11-06T03:09:33.746188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884251+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:09:31.884251+0000","last_peered":"2023-11-06T03:09:31.884251+0000","last_clean":"2023-11-06T03:09:31.884251+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:09:31.884251+0000","last_undegraded":"2023-11-06T03:09:31.884251+0000","last_fullsized":"2023-11-06T03:09:31.884251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747611+0000","last_change":"2023-11-06T03:09:33.747611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747611+0000","last_undegraded":"2023-11-06T03:09:33.747611+0000","last_fullsized":"2023-11-06T03:09:33.747611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737274+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:09:31.737274+0000","last_peered":"2023-11-06T03:09:31.737274+0000","last_clean":"2023-11-06T03:09:31.737274+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:09:31.737274+0000","last_undegraded":"2023-11-06T03:09:31.737274+0000","last_fullsized":"2023-11-06T03:09:31.737274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743765+0000","last_change":"2023-11-06T03:09:33.743765+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743765+0000","last_undegraded":"2023-11-06T03:09:33.743765+0000","last_fullsized":"2023-11-06T03:09:33.743765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450520+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:09:33.450520+0000","last_peered":"2023-11-06T03:09:33.450520+0000","last_clean":"2023-11-06T03:09:33.450520+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:09:33.450520+0000","last_undegraded":"2023-11-06T03:09:33.450520+0000","last_fullsized":"2023-11-06T03:09:33.450520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1581056,"data_stored":1036744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.397,"15min":1.397},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.86099999999999999},"min":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":1.444,"15min":1.444},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.86899999999999999,"15min":0.89600000000000002},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.877,"15min":0.877},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.381,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.216},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.879,"15min":0.95299999999999996},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0629999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.127,"15min":1.0669999999999999},"min":{"1min":0.65600000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":10.596,"5min":10.596,"15min":10.596},"last":1.302}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92500000000000004,"15min":0.97299999999999998},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":1.01},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.282,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.042,"15min":1.05},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.355},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.095},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3240000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79600000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.51,"5min":1.774,"15min":1.774},"last":1.252},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.099,"15min":1.117},"min":{"1min":0.90800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.1830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45960,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325880,"statfs":{"total":96636764160,"available":96589701120,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":0.96199999999999997,"15min":0.84899999999999998},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":3.0750000000000002,"5min":1.256,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":40.905000000000001,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.048,"15min":0.89700000000000002},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":16.012,"5min":16.012,"15min":16.012},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.02,"5min":0.89400000000000002,"15min":0.879},"min":{"1min":0.60499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":1.1120000000000001,"15min":0.90600000000000003},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":27.803999999999998,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.627},{"interface":"front","average":{"1min":1.004,"5min":0.96999999999999997,"15min":0.95699999999999996},"min":{"1min":0.64000000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.621,"15min":1.1919999999999999},"min":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":63.737000000000002,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":3.121,"5min":1.365,"15min":1.0449999999999999},"min":{"1min":0.57199999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":41.079999999999998,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.282,"15min":1.1839999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":24.867999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.359,"15min":1.2370000000000001},"min":{"1min":0.76700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":23.789000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":0.94499999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":1.4359999999999999,"15min":1.1779999999999999},"min":{"1min":0.71599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":40.959000000000003,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1479999999999999,"15min":1.1120000000000001},"min":{"1min":0.90400000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":1.3460000000000001,"15min":1.1899999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.407,"15min":1.22},"min":{"1min":0.81100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":28.164000000000001,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":950272,"data_stored":429420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0580000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81499999999999995,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.145,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2230000000000001,"5min":1.357,"15min":1.357},"last":0.85599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.84199999999999997},"min":{"1min":0.66400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1970000000000001,"5min":1.403,"15min":1.403},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94999999999999996,"15min":0.93600000000000005},"min":{"1min":0.72799999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":1.117}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.91900000000000004},"min":{"1min":0.64900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1060000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2330000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.014,"15min":0.99399999999999999},"min":{"1min":0.63800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.206,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.306,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0269999999999999,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.389,"5min":1.476,"15min":1.476},"last":1.087},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0940000000000001,"15min":1.0840000000000001},"min":{"1min":1.008,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2909999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.82299999999999995},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1619999999999999,"5min":1.484,"15min":1.484},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2989999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.755},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1930000000000001,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85499999999999998,"15min":0.81200000000000006},"min":{"1min":0.55300000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.91600000000000004,"15min":1.04},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.754},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1539999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1919999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.71899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.069,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.365,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5489999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.034},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.1519999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.411,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.095}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.073},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3009999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.139},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.2430000000000001,"15min":1.1870000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6739999999999999,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2749999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50056,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321784,"statfs":{"total":96636764160,"available":96585506816,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.248,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.26700000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.90400000000000003,"5min":1.524,"15min":1.524},"last":0.47999999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1080000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.042,"5min":1.349,"15min":1.349},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.518,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94599999999999995,"15min":0.98399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.536,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.66400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5049999999999999,"5min":1.698,"15min":1.698},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97599999999999998,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4610000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94899999999999995,"15min":1},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.391,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.008,"15min":1.056},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3180000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.024,"15min":1.0649999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4930000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.95299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.24299999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.314},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.255,"5min":0.223,"15min":0.223},"max":{"1min":1.0109999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.62,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.121},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.629,"15min":0.66900000000000004},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.1259999999999999,"5min":1.294,"15min":1.294},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.89000000000000001},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":0.95499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.88400000000000001},"min":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.214,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.626}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.615,"15min":1.615},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.95199999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.006,"15min":1},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.93100000000000005,"15min":0.96199999999999997},"min":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.256,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56299999999999994,"15min":0.59199999999999997},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":1.0629999999999999,"5min":1.157,"15min":1.157},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.26900000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0649999999999999,"5min":1.544,"15min":1.544},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.90700000000000003,"15min":0.71999999999999997},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63900000000000001,"15min":0.68300000000000005},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.03,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1200000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.72599999999999998},"min":{"1min":0.35199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90600000000000003,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.495}]},{"osd":4,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.2809999999999999,"5min":2.258,"15min":2.258},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":14.753,"15min":14.753},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.3919999999999999,"15min":1.1399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":1.367,"15min":1.167},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.272,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.177,"15min":1.0760000000000001},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":23.971,"15min":23.971},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2949999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0149999999999999,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3560000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.79400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1576960,"data_stored":1035725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.28299999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.145,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.69899999999999995,"15min":0.65100000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1120000000000001,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83199999999999996,"5min":1.155,"15min":1.155},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.68500000000000005,"15min":0.65400000000000003},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.876,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.29899999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0700000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.755},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0589999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.96299999999999997,"15min":0.92700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1299999999999999,"5min":4.641,"15min":4.641},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.88500000000000001,"15min":0.93100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1,"5min":2.028,"15min":2.028},"last":0.83999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1140000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.94699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2050000000000001,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.871}]},{"osd":6,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":2.931,"15min":2.931},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.248,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.0289999999999999,"15min":1.0089999999999999},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2649999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.98699999999999999,"15min":0.99199999999999999},"min":{"1min":0.64800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.97,"15min":1.97},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:09:38.470 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:09:38.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 Processor -- start 2023-11-06T03:09:38.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 -- start start 2023-11-06T03:09:38.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31e00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e015e440 con 0x7f31e006d000 2023-11-06T03:09:38.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e006c240 con 0x7f31e00b2af0 2023-11-06T03:09:38.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31e5911640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e006c520 con 0x7f31e01599c0 2023-11-06T03:09:38.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31de7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f31e006d000 0x7f31e00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54426/0 (socket says 172.21.15.157:54426) 2023-11-06T03:09:38.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31de7fc640 1 -- 172.21.15.157:0/1963087064 learned_addr learned my addr 172.21.15.157:0/1963087064 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:38.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31deffd640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31e00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627898536 0 0) 0x7f31e015e440 con 0x7f31e006d000 2023-11-06T03:09:38.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31cc003530 con 0x7f31e006d000 2023-11-06T03:09:38.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180421711 0 0) 0x7f31cc003530 con 0x7f31e006d000 2023-11-06T03:09:38.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 msgr2=0x7f31e00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31e00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 >> v1:172.21.15.195:6789/0 conn(0x7f31e00b2af0 legacy=0x7f31e0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e006c800 con 0x7f31e006d000 2023-11-06T03:09:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f31c8001680 con 0x7f31e006d000 2023-11-06T03:09:38.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 >> v1:172.21.15.157:6789/0 conn(0x7f31e006d000 legacy=0x7f31e00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc004d10 0x7f31cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e006c520 con 0x7f31cc003f30 2023-11-06T03:09:38.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc00ba70 con 0x7f31cc004d10 2023-11-06T03:09:38.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc00bc30 con 0x7f31e01599c0 2023-11-06T03:09:38.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.628+0000 7f31de7fc640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31deffd640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 -- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 msgr2=0x7f31cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01599c0 0x7f31cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31df7fe640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc004d10 0x7f31cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 -- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc004d10 msgr2=0x7f31cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:38.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc004d10 0x7f31cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 -- 172.21.15.157:0/1963087064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc003530 con 0x7f31cc003f30 2023-11-06T03:09:38.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31df7fe640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31cc004d10 0x7f31cc005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:38.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31de7fc640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31cc0047d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c1c0 tx=0x7f31c801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc8f877a73fba8b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8005ae0 con 0x7f31cc003f30 2023-11-06T03:09:38.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80ab070 con 0x7f31cc003f30 2023-11-06T03:09:38.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31ddffb640 1 -- 172.21.15.157:0/1963087064 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8007070 con 0x7f31cc003f30 2023-11-06T03:09:38.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 -- 172.21.15.157:0/1963087064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 msgr2=0x7f31cc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 --2- 172.21.15.157:0/1963087064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31cc0047d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f31c800c1c0 tx=0x7f31c801f390 comp rx=0 tx=0).stop 2023-11-06T03:09:38.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31df7fe640 1 -- 172.21.15.157:0/1963087064 reap_dead start 2023-11-06T03:09:38.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 -- 172.21.15.157:0/1963087064 shutdown_connections 2023-11-06T03:09:38.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 -- 172.21.15.157:0/1963087064 >> 172.21.15.157:0/1963087064 conn(0x7f31e0065930 msgr2=0x7f31e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:38.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 -- 172.21.15.157:0/1963087064 shutdown_connections 2023-11-06T03:09:38.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.632+0000 7f31e5911640 1 -- 172.21.15.157:0/1963087064 wait complete. 2023-11-06T03:09:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 Processor -- start 2023-11-06T03:09:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- start start 2023-11-06T03:09:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 0x7f31e01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 0x7f31e01f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0151a70 con 0x7f31cc003f30 2023-11-06T03:09:38.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e01f5bb0 con 0x7f31e01f6040 2023-11-06T03:09:38.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e01584a0 con 0x7f31e01515f0 2023-11-06T03:09:38.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46830/0 (socket says 172.21.15.157:46830) 2023-11-06T03:09:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 -- 172.21.15.157:0/2628103057 learned_addr learned my addr 172.21.15.157:0/2628103057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31de7fc640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 0x7f31e01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31df7fe640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 0x7f31e01f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 -- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 msgr2=0x7f31e01565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 0x7f31e01565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 -- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 msgr2=0x7f31e01f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 0x7f31e01f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 -- 172.21.15.157:0/2628103057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0157c00 con 0x7f31cc003f30 2023-11-06T03:09:38.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31df7fe640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 0x7f31e01f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31de7fc640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 0x7f31e01565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31deffd640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f31d00115e0 tx=0x7f31d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000f8f0 con 0x7f31cc003f30 2023-11-06T03:09:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d000fa50 con 0x7f31cc003f30 2023-11-06T03:09:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e0157ee0 con 0x7f31cc003f30 2023-11-06T03:09:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000fdc0 con 0x7f31cc003f30 2023-11-06T03:09:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e00ac8b0 con 0x7f31cc003f30 2023-11-06T03:09:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e01f5980 con 0x7f31cc003f30 2023-11-06T03:09:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.636+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f31d0015460 con 0x7f31cc003f30 2023-11-06T03:09:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.640+0000 7f31d7fff640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 0x7f31c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.640+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31d00837d0 con 0x7f31cc003f30 2023-11-06T03:09:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.640+0000 7f31de7fc640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 0x7f31c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:38.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.640+0000 7f31de7fc640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 0x7f31c0067f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31c80af000 tx=0x7f31c80aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:38.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.644+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f31e01f5980 con 0x7f31cc003f30 2023-11-06T03:09:38.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.792+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31e00add00 con 0x7f31c0065a60 2023-11-06T03:09:38.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.816+0000 7f31d7fff640 1 -- 172.21.15.157:0/2628103057 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299859 (secure 0 0 0) 0x7f31c80015c0 con 0x7f31c0065a60 2023-11-06T03:09:38.816 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:38.823 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:09:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 msgr2=0x7f31c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 0x7f31c0067f20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31c80af000 tx=0x7f31c80aa000 comp rx=0 tx=0).stop 2023-11-06T03:09:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 msgr2=0x7f31e0157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f31d00115e0 tx=0x7f31d0011aa0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 shutdown_connections 2023-11-06T03:09:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31c0065a60 0x7f31c0067f20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31e01f6040 0x7f31e01f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31e01515f0 0x7f31e01565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 --2- 172.21.15.157:0/2628103057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31cc003f30 0x7f31e0157590 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 >> 172.21.15.157:0/2628103057 conn(0x7f31e0065930 msgr2=0x7f31e00ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 shutdown_connections 2023-11-06T03:09:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:38.820+0000 7f31e5911640 1 -- 172.21.15.157:0/2628103057 wait complete. 2023-11-06T03:09:38.837 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2023-11-06T03:09:37.213932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361436,"kb_used_data":8860,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754613284,"statfs":{"total":773094113280,"available":772724002816,"internally_reserved":0,"allocated":9072640,"data_stored":4750241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001109"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743534+0000","last_change":"2023-11-06T03:09:33.743534+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743534+0000","last_undegraded":"2023-11-06T03:09:33.743534+0000","last_fullsized":"2023-11-06T03:09:33.743534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746611+0000","last_change":"2023-11-06T03:09:33.746611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746611+0000","last_undegraded":"2023-11-06T03:09:33.746611+0000","last_fullsized":"2023-11-06T03:09:33.746611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.751332+0000","last_change":"2023-11-06T03:09:33.751332+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.751332+0000","last_undegraded":"2023-11-06T03:09:33.751332+0000","last_fullsized":"2023-11-06T03:09:33.751332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.749455+0000","last_change":"2023-11-06T03:09:33.749455+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.749455+0000","last_undegraded":"2023-11-06T03:09:33.749455+0000","last_fullsized":"2023-11-06T03:09:33.749455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744013+0000","last_change":"2023-11-06T03:09:33.744013+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744013+0000","last_undegraded":"2023-11-06T03:09:33.744013+0000","last_fullsized":"2023-11-06T03:09:33.744013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755195+0000","last_change":"2023-11-06T03:09:33.755195+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755195+0000","last_undegraded":"2023-11-06T03:09:33.755195+0000","last_fullsized":"2023-11-06T03:09:33.755195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740702+0000","last_change":"2023-11-06T03:09:33.740702+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740702+0000","last_undegraded":"2023-11-06T03:09:33.740702+0000","last_fullsized":"2023-11-06T03:09:33.740702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743395+0000","last_change":"2023-11-06T03:09:33.743395+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743395+0000","last_undegraded":"2023-11-06T03:09:33.743395+0000","last_fullsized":"2023-11-06T03:09:33.743395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748920+0000","last_change":"2023-11-06T03:09:33.748920+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748920+0000","last_undegraded":"2023-11-06T03:09:33.748920+0000","last_fullsized":"2023-11-06T03:09:33.748920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755232+0000","last_change":"2023-11-06T03:09:33.755232+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755232+0000","last_undegraded":"2023-11-06T03:09:33.755232+0000","last_fullsized":"2023-11-06T03:09:33.755232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744368+0000","last_change":"2023-11-06T03:09:33.744368+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744368+0000","last_undegraded":"2023-11-06T03:09:33.744368+0000","last_fullsized":"2023-11-06T03:09:33.744368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748273+0000","last_change":"2023-11-06T03:09:33.748273+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748273+0000","last_undegraded":"2023-11-06T03:09:33.748273+0000","last_fullsized":"2023-11-06T03:09:33.748273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.755163+0000","last_change":"2023-11-06T03:09:33.755163+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.755163+0000","last_undegraded":"2023-11-06T03:09:33.755163+0000","last_fullsized":"2023-11-06T03:09:33.755163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746227+0000","last_change":"2023-11-06T03:09:33.746227+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746227+0000","last_undegraded":"2023-11-06T03:09:33.746227+0000","last_fullsized":"2023-11-06T03:09:33.746227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.744803+0000","last_change":"2023-11-06T03:09:33.744803+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.744803+0000","last_undegraded":"2023-11-06T03:09:33.744803+0000","last_fullsized":"2023-11-06T03:09:33.744803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740711+0000","last_change":"2023-11-06T03:09:33.740711+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740711+0000","last_undegraded":"2023-11-06T03:09:33.740711+0000","last_fullsized":"2023-11-06T03:09:33.740711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.756784+0000","last_change":"2023-11-06T03:09:33.756784+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.756784+0000","last_undegraded":"2023-11-06T03:09:33.756784+0000","last_fullsized":"2023-11-06T03:09:33.756784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.740826+0000","last_change":"2023-11-06T03:09:33.740826+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.740826+0000","last_undegraded":"2023-11-06T03:09:33.740826+0000","last_fullsized":"2023-11-06T03:09:33.740826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.750167+0000","last_change":"2023-11-06T03:09:33.750167+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.750167+0000","last_undegraded":"2023-11-06T03:09:33.750167+0000","last_fullsized":"2023-11-06T03:09:33.750167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748000+0000","last_change":"2023-11-06T03:09:33.748000+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748000+0000","last_undegraded":"2023-11-06T03:09:33.748000+0000","last_fullsized":"2023-11-06T03:09:33.748000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746515+0000","last_change":"2023-11-06T03:09:33.746515+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746515+0000","last_undegraded":"2023-11-06T03:09:33.746515+0000","last_fullsized":"2023-11-06T03:09:33.746515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743986+0000","last_change":"2023-11-06T03:09:33.743986+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743986+0000","last_undegraded":"2023-11-06T03:09:33.743986+0000","last_fullsized":"2023-11-06T03:09:33.743986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746372+0000","last_change":"2023-11-06T03:09:33.746372+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746372+0000","last_undegraded":"2023-11-06T03:09:33.746372+0000","last_fullsized":"2023-11-06T03:09:33.746372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747573+0000","last_change":"2023-11-06T03:09:33.747573+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747573+0000","last_undegraded":"2023-11-06T03:09:33.747573+0000","last_fullsized":"2023-11-06T03:09:33.747573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746778+0000","last_change":"2023-11-06T03:09:33.746778+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746778+0000","last_undegraded":"2023-11-06T03:09:33.746778+0000","last_fullsized":"2023-11-06T03:09:33.746778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884198+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:09:31.884198+0000","last_peered":"2023-11-06T03:09:31.884198+0000","last_clean":"2023-11-06T03:09:31.884198+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:09:31.884198+0000","last_undegraded":"2023-11-06T03:09:31.884198+0000","last_fullsized":"2023-11-06T03:09:31.884198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450714+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:09:33.450714+0000","last_peered":"2023-11-06T03:09:33.450714+0000","last_clean":"2023-11-06T03:09:33.450714+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:09:33.450714+0000","last_undegraded":"2023-11-06T03:09:33.450714+0000","last_fullsized":"2023-11-06T03:09:33.450714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746590+0000","last_change":"2023-11-06T03:09:33.746590+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746590+0000","last_undegraded":"2023-11-06T03:09:33.746590+0000","last_fullsized":"2023-11-06T03:09:33.746590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884217+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:09:31.884217+0000","last_peered":"2023-11-06T03:09:31.884217+0000","last_clean":"2023-11-06T03:09:31.884217+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:09:31.884217+0000","last_undegraded":"2023-11-06T03:09:31.884217+0000","last_fullsized":"2023-11-06T03:09:31.884217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746391+0000","last_change":"2023-11-06T03:09:33.746391+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746391+0000","last_undegraded":"2023-11-06T03:09:33.746391+0000","last_fullsized":"2023-11-06T03:09:33.746391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:32.634286+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:09:32.634286+0000","last_peered":"2023-11-06T03:09:32.634286+0000","last_clean":"2023-11-06T03:09:32.634286+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:09:32.634286+0000","last_undegraded":"2023-11-06T03:09:32.634286+0000","last_fullsized":"2023-11-06T03:09:32.634286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746134+0000","last_change":"2023-11-06T03:09:33.746134+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746134+0000","last_undegraded":"2023-11-06T03:09:33.746134+0000","last_fullsized":"2023-11-06T03:09:33.746134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450705+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:09:33.450705+0000","last_peered":"2023-11-06T03:09:33.450705+0000","last_clean":"2023-11-06T03:09:33.450705+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:09:33.450705+0000","last_undegraded":"2023-11-06T03:09:33.450705+0000","last_fullsized":"2023-11-06T03:09:33.450705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743578+0000","last_change":"2023-11-06T03:09:33.743578+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743578+0000","last_undegraded":"2023-11-06T03:09:33.743578+0000","last_fullsized":"2023-11-06T03:09:33.743578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737288+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:09:31.737288+0000","last_peered":"2023-11-06T03:09:31.737288+0000","last_clean":"2023-11-06T03:09:31.737288+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:09:31.737288+0000","last_undegraded":"2023-11-06T03:09:31.737288+0000","last_fullsized":"2023-11-06T03:09:31.737288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746188+0000","last_change":"2023-11-06T03:09:33.746188+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746188+0000","last_undegraded":"2023-11-06T03:09:33.746188+0000","last_fullsized":"2023-11-06T03:09:33.746188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884251+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:09:31.884251+0000","last_peered":"2023-11-06T03:09:31.884251+0000","last_clean":"2023-11-06T03:09:31.884251+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:09:31.884251+0000","last_undegraded":"2023-11-06T03:09:31.884251+0000","last_fullsized":"2023-11-06T03:09:31.884251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747611+0000","last_change":"2023-11-06T03:09:33.747611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747611+0000","last_undegraded":"2023-11-06T03:09:33.747611+0000","last_fullsized":"2023-11-06T03:09:33.747611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.737274+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:09:31.737274+0000","last_peered":"2023-11-06T03:09:31.737274+0000","last_clean":"2023-11-06T03:09:31.737274+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:09:31.737274+0000","last_undegraded":"2023-11-06T03:09:31.737274+0000","last_fullsized":"2023-11-06T03:09:31.737274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.743765+0000","last_change":"2023-11-06T03:09:33.743765+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.743765+0000","last_undegraded":"2023-11-06T03:09:33.743765+0000","last_fullsized":"2023-11-06T03:09:33.743765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:33.450520+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:09:33.450520+0000","last_peered":"2023-11-06T03:09:33.450520+0000","last_clean":"2023-11-06T03:09:33.450520+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:09:33.450520+0000","last_undegraded":"2023-11-06T03:09:33.450520+0000","last_fullsized":"2023-11-06T03:09:33.450520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1581056,"data_stored":1036744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.397,"15min":1.397},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.86099999999999999},"min":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":1.444,"15min":1.444},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.86899999999999999,"15min":0.89600000000000002},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.877,"15min":0.877},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.381,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.216},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.879,"15min":0.95299999999999996},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0629999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.127,"15min":1.0669999999999999},"min":{"1min":0.65600000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":10.596,"5min":10.596,"15min":10.596},"last":1.302}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92500000000000004,"15min":0.97299999999999998},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":1.01},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.282,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.042,"15min":1.05},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.355},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.095},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3240000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79600000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.51,"5min":1.774,"15min":1.774},"last":1.252},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.099,"15min":1.117},"min":{"1min":0.90800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.1830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45960,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325880,"statfs":{"total":96636764160,"available":96589701120,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":0.96199999999999997,"15min":0.84899999999999998},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":3.0750000000000002,"5min":1.256,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":40.905000000000001,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.048,"15min":0.89700000000000002},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":16.012,"5min":16.012,"15min":16.012},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.02,"5min":0.89400000000000002,"15min":0.879},"min":{"1min":0.60499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":1.1120000000000001,"15min":0.90600000000000003},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":27.803999999999998,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.627},{"interface":"front","average":{"1min":1.004,"5min":0.96999999999999997,"15min":0.95699999999999996},"min":{"1min":0.64000000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.621,"15min":1.1919999999999999},"min":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":63.737000000000002,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":3.121,"5min":1.365,"15min":1.0449999999999999},"min":{"1min":0.57199999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":41.079999999999998,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.77800000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.282,"15min":1.1839999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":24.867999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.359,"15min":1.2370000000000001},"min":{"1min":0.76700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":23.789000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":0.94499999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":1.4359999999999999,"15min":1.1779999999999999},"min":{"1min":0.71599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":40.959000000000003,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1479999999999999,"15min":1.1120000000000001},"min":{"1min":0.90400000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":1.3460000000000001,"15min":1.1899999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.407,"15min":1.22},"min":{"1min":0.81100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":28.164000000000001,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":950272,"data_stored":429420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0580000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81499999999999995,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.145,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2230000000000001,"5min":1.357,"15min":1.357},"last":0.85599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.84199999999999997},"min":{"1min":0.66400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1970000000000001,"5min":1.403,"15min":1.403},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94999999999999996,"15min":0.93600000000000005},"min":{"1min":0.72799999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":1.117}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.91900000000000004},"min":{"1min":0.64900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1060000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2330000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.014,"15min":0.99399999999999999},"min":{"1min":0.63800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.206,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.306,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0269999999999999,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.389,"5min":1.476,"15min":1.476},"last":1.087},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0940000000000001,"15min":1.0840000000000001},"min":{"1min":1.008,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2909999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.82299999999999995},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1619999999999999,"5min":1.484,"15min":1.484},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2989999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.755},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1930000000000001,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85499999999999998,"15min":0.81200000000000006},"min":{"1min":0.55300000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.91600000000000004,"15min":1.04},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.754},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1539999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1919999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.71899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.069,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.365,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5489999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.034},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.1519999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.411,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.095}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.073},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3009999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.139},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.2430000000000001,"15min":1.1870000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6739999999999999,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.2749999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50056,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321784,"statfs":{"total":96636764160,"available":96585506816,"internally_reserved":0,"allocated":983040,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.248,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.26700000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.90400000000000003,"5min":1.524,"15min":1.524},"last":0.47999999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1080000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.042,"5min":1.349,"15min":1.349},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.518,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94599999999999995,"15min":0.98399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.536,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.66400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5049999999999999,"5min":1.698,"15min":1.698},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97599999999999998,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4610000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94899999999999995,"15min":1},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.391,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.008,"15min":1.056},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3180000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.024,"15min":1.0649999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4930000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.95299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.24299999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.314},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.255,"5min":0.223,"15min":0.223},"max":{"1min":1.0109999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.35399999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.62,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.121},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.629,"15min":0.66900000000000004},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.1259999999999999,"5min":1.294,"15min":1.294},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.89000000000000001},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":0.95499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.88400000000000001},"min":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.214,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.626}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.615,"15min":1.615},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.95199999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.006,"15min":1},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.93100000000000005,"15min":0.96199999999999997},"min":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.256,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":999424,"data_stored":452546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56299999999999994,"15min":0.59199999999999997},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":1.0629999999999999,"5min":1.157,"15min":1.157},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.26900000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0649999999999999,"5min":1.544,"15min":1.544},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.90700000000000003,"15min":0.71999999999999997},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63900000000000001,"15min":0.68300000000000005},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.03,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.43099999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1200000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.72599999999999998},"min":{"1min":0.35199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90600000000000003,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.495}]},{"osd":4,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.2809999999999999,"5min":2.258,"15min":2.258},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":14.753,"15min":14.753},"last":0.89300000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.3919999999999999,"15min":1.1399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":1.367,"15min":1.167},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.272,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.177,"15min":1.0760000000000001},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":23.971,"15min":23.971},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2949999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0149999999999999,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3560000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.79400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313012,"statfs":{"total":96636764160,"available":96576524288,"internally_reserved":0,"allocated":1576960,"data_stored":1035725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.28299999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.145,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.69899999999999995,"15min":0.65100000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1120000000000001,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83199999999999996,"5min":1.155,"15min":1.155},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.68500000000000005,"15min":0.65400000000000003},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.876,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.29899999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0700000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.755},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0589999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.96299999999999997,"15min":0.92700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1299999999999999,"5min":4.641,"15min":4.641},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.88500000000000001,"15min":0.93100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1,"5min":2.028,"15min":2.028},"last":0.83999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1140000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.94699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2050000000000001,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.871}]},{"osd":6,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":2.931,"15min":2.931},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.248,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.0289999999999999,"15min":1.0089999999999999},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2649999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.98699999999999999,"15min":0.99199999999999999},"min":{"1min":0.64800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.97,"15min":1.97},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:09:38.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-11-06T03:09:38.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-11-06T03:09:38.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-11-06T03:09:38.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-11-06T03:09:38.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-11-06T03:09:38.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-11-06T03:09:38.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-11-06T03:09:38.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-11-06T03:09:38.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-11-06T03:09:38.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-11-06T03:09:38.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-11-06T03:09:38.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-11-06T03:09:38.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-11-06T03:09:38.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-11-06T03:09:38.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-11-06T03:09:38.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-11-06T03:09:38.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-11-06T03:09:38.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-11-06T03:09:38.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-11-06T03:09:38.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-11-06T03:09:38.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-11-06T03:09:38.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-11-06T03:09:38.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-11-06T03:09:38.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-11-06T03:09:38.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-11-06T03:09:38.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-11-06T03:09:38.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-11-06T03:09:38.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-11-06T03:09:38.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-11-06T03:09:38.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-11-06T03:09:38.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-11-06T03:09:38.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-11-06T03:09:38.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-11-06T03:09:38.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-06T03:09:38.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.743534+0000', 'last_change': '2023-11-06T03:09:33.743534+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.743534+0000', 'last_undegraded': '2023-11-06T03:09:33.743534+0000', 'last_fullsized': '2023-11-06T03:09:33.743534+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-11-06T03:09:38.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-06T03:09:38.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746611+0000', 'last_change': '2023-11-06T03:09:33.746611+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746611+0000', 'last_undegraded': '2023-11-06T03:09:33.746611+0000', 'last_fullsized': '2023-11-06T03:09:33.746611+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-11-06T03:09:38.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-06T03:09:38.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.751332+0000', 'last_change': '2023-11-06T03:09:33.751332+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.751332+0000', 'last_undegraded': '2023-11-06T03:09:33.751332+0000', 'last_fullsized': '2023-11-06T03:09:33.751332+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-11-06T03:09:38.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-06T03:09:38.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.749455+0000', 'last_change': '2023-11-06T03:09:33.749455+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.749455+0000', 'last_undegraded': '2023-11-06T03:09:33.749455+0000', 'last_fullsized': '2023-11-06T03:09:33.749455+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:09:38.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-11-06T03:09:38.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-06T03:09:38.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.744013+0000', 'last_change': '2023-11-06T03:09:33.744013+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.744013+0000', 'last_undegraded': '2023-11-06T03:09:33.744013+0000', 'last_fullsized': '2023-11-06T03:09:33.744013+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:09:38.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-11-06T03:09:38.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-06T03:09:38.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.755195+0000', 'last_change': '2023-11-06T03:09:33.755195+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.755195+0000', 'last_undegraded': '2023-11-06T03:09:33.755195+0000', 'last_fullsized': '2023-11-06T03:09:33.755195+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:09:38.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-11-06T03:09:38.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-06T03:09:38.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.740702+0000', 'last_change': '2023-11-06T03:09:33.740702+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.740702+0000', 'last_undegraded': '2023-11-06T03:09:33.740702+0000', 'last_fullsized': '2023-11-06T03:09:33.740702+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:09:38.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-11-06T03:09:38.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-06T03:09:38.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.743395+0000', 'last_change': '2023-11-06T03:09:33.743395+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.743395+0000', 'last_undegraded': '2023-11-06T03:09:33.743395+0000', 'last_fullsized': '2023-11-06T03:09:33.743395+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-11-06T03:09:38.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-06T03:09:38.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.748920+0000', 'last_change': '2023-11-06T03:09:33.748920+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.748920+0000', 'last_undegraded': '2023-11-06T03:09:33.748920+0000', 'last_fullsized': '2023-11-06T03:09:33.748920+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-11-06T03:09:38.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-06T03:09:38.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.755232+0000', 'last_change': '2023-11-06T03:09:33.755232+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.755232+0000', 'last_undegraded': '2023-11-06T03:09:33.755232+0000', 'last_fullsized': '2023-11-06T03:09:33.755232+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:09:38.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-11-06T03:09:38.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-06T03:09:38.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.744368+0000', 'last_change': '2023-11-06T03:09:33.744368+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.744368+0000', 'last_undegraded': '2023-11-06T03:09:33.744368+0000', 'last_fullsized': '2023-11-06T03:09:33.744368+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-11-06T03:09:38.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-06T03:09:38.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.748273+0000', 'last_change': '2023-11-06T03:09:33.748273+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.748273+0000', 'last_undegraded': '2023-11-06T03:09:33.748273+0000', 'last_fullsized': '2023-11-06T03:09:33.748273+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:09:38.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-11-06T03:09:38.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-06T03:09:38.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.755163+0000', 'last_change': '2023-11-06T03:09:33.755163+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.755163+0000', 'last_undegraded': '2023-11-06T03:09:33.755163+0000', 'last_fullsized': '2023-11-06T03:09:33.755163+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:09:38.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-11-06T03:09:38.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-06T03:09:38.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746227+0000', 'last_change': '2023-11-06T03:09:33.746227+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746227+0000', 'last_undegraded': '2023-11-06T03:09:33.746227+0000', 'last_fullsized': '2023-11-06T03:09:33.746227+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-11-06T03:09:38.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-06T03:09:38.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.744803+0000', 'last_change': '2023-11-06T03:09:33.744803+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.744803+0000', 'last_undegraded': '2023-11-06T03:09:33.744803+0000', 'last_fullsized': '2023-11-06T03:09:33.744803+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-11-06T03:09:38.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-06T03:09:38.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.740711+0000', 'last_change': '2023-11-06T03:09:33.740711+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.740711+0000', 'last_undegraded': '2023-11-06T03:09:33.740711+0000', 'last_fullsized': '2023-11-06T03:09:33.740711+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:09:38.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-11-06T03:09:38.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-06T03:09:38.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.756784+0000', 'last_change': '2023-11-06T03:09:33.756784+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.756784+0000', 'last_undegraded': '2023-11-06T03:09:33.756784+0000', 'last_fullsized': '2023-11-06T03:09:33.756784+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:09:38.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-11-06T03:09:38.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-06T03:09:38.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.740826+0000', 'last_change': '2023-11-06T03:09:33.740826+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.740826+0000', 'last_undegraded': '2023-11-06T03:09:33.740826+0000', 'last_fullsized': '2023-11-06T03:09:33.740826+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:09:38.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-11-06T03:09:38.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-06T03:09:38.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.750167+0000', 'last_change': '2023-11-06T03:09:33.750167+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.750167+0000', 'last_undegraded': '2023-11-06T03:09:33.750167+0000', 'last_fullsized': '2023-11-06T03:09:33.750167+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-11-06T03:09:38.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-06T03:09:38.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.748000+0000', 'last_change': '2023-11-06T03:09:33.748000+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.748000+0000', 'last_undegraded': '2023-11-06T03:09:33.748000+0000', 'last_fullsized': '2023-11-06T03:09:33.748000+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-11-06T03:09:38.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-06T03:09:38.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746515+0000', 'last_change': '2023-11-06T03:09:33.746515+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746515+0000', 'last_undegraded': '2023-11-06T03:09:33.746515+0000', 'last_fullsized': '2023-11-06T03:09:33.746515+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:09:38.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-11-06T03:09:38.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-06T03:09:38.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.743986+0000', 'last_change': '2023-11-06T03:09:33.743986+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.743986+0000', 'last_undegraded': '2023-11-06T03:09:33.743986+0000', 'last_fullsized': '2023-11-06T03:09:33.743986+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-11-06T03:09:38.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-06T03:09:38.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746372+0000', 'last_change': '2023-11-06T03:09:33.746372+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746372+0000', 'last_undegraded': '2023-11-06T03:09:33.746372+0000', 'last_fullsized': '2023-11-06T03:09:33.746372+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:09:38.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-11-06T03:09:38.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-06T03:09:38.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.747573+0000', 'last_change': '2023-11-06T03:09:33.747573+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.747573+0000', 'last_undegraded': '2023-11-06T03:09:33.747573+0000', 'last_fullsized': '2023-11-06T03:09:33.747573+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:09:38.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-11-06T03:09:38.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-06T03:09:38.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746778+0000', 'last_change': '2023-11-06T03:09:33.746778+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746778+0000', 'last_undegraded': '2023-11-06T03:09:33.746778+0000', 'last_fullsized': '2023-11-06T03:09:33.746778+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:09:38.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-11-06T03:09:38.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-06T03:09:38.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746590+0000', 'last_change': '2023-11-06T03:09:33.746590+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746590+0000', 'last_undegraded': '2023-11-06T03:09:33.746590+0000', 'last_fullsized': '2023-11-06T03:09:33.746590+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:09:38.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-11-06T03:09:38.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-06T03:09:38.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746391+0000', 'last_change': '2023-11-06T03:09:33.746391+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746391+0000', 'last_undegraded': '2023-11-06T03:09:33.746391+0000', 'last_fullsized': '2023-11-06T03:09:33.746391+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:09:38.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-11-06T03:09:38.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-06T03:09:38.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746134+0000', 'last_change': '2023-11-06T03:09:33.746134+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746134+0000', 'last_undegraded': '2023-11-06T03:09:33.746134+0000', 'last_fullsized': '2023-11-06T03:09:33.746134+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:09:38.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-11-06T03:09:38.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-06T03:09:38.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.743578+0000', 'last_change': '2023-11-06T03:09:33.743578+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.743578+0000', 'last_undegraded': '2023-11-06T03:09:33.743578+0000', 'last_fullsized': '2023-11-06T03:09:33.743578+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:09:38.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-11-06T03:09:38.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-06T03:09:38.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.746188+0000', 'last_change': '2023-11-06T03:09:33.746188+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.746188+0000', 'last_undegraded': '2023-11-06T03:09:33.746188+0000', 'last_fullsized': '2023-11-06T03:09:33.746188+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:09:38.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-11-06T03:09:38.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-06T03:09:38.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.747611+0000', 'last_change': '2023-11-06T03:09:33.747611+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.747611+0000', 'last_undegraded': '2023-11-06T03:09:33.747611+0000', 'last_fullsized': '2023-11-06T03:09:33.747611+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:09:38.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-11-06T03:09:38.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:09:34.736732+0000', 'last_change': '2023-11-06T03:09:34.736732+0000', 'last_active': '2023-11-06T03:09:34.736732+0000', 'last_peered': '2023-11-06T03:09:34.736732+0000', 'last_clean': '2023-11-06T03:09:34.736732+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:34.736732+0000', 'last_undegraded': '2023-11-06T03:09:34.736732+0000', 'last_fullsized': '2023-11-06T03:09:34.736732+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:34.736732+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:09:38.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-06T03:09:38.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 82, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:09:33.743765+0000', 'last_change': '2023-11-06T03:09:33.743765+0000', 'last_active': '2023-11-06T03:09:33.733547+0000', 'last_peered': '2023-11-06T03:09:33.733547+0000', 'last_clean': '2023-11-06T03:09:33.733547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:09:33.743765+0000', 'last_undegraded': '2023-11-06T03:09:33.743765+0000', 'last_fullsized': '2023-11-06T03:09:33.743765+0000', 'mapping_epoch': 82, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'last_clean_scrub_stamp': '2023-11-06T03:09:33.733547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:09:38.928 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:09:39.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.056+0000 7f3d5d727640 1 Processor -- start 2023-11-06T03:09:39.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.056+0000 7f3d5d727640 1 -- start start 2023-11-06T03:09:39.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d5815ac80 0x7f3d580b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3d580afb00 0x7f3d580b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580cc2b0 con 0x7f3d580cc720 2023-11-06T03:09:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d580b01a0 con 0x7f3d5815ac80 2023-11-06T03:09:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d580cfb10 con 0x7f3d580afb00 2023-11-06T03:09:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d5815ac80 0x7f3d580b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3d5815ac80 0x7f3d580b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46844/0 (socket says 172.21.15.157:46844) 2023-11-06T03:09:39.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d577fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3d580cc720 0x7f3d580ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54430/0 (socket says 172.21.15.157:54430) 2023-11-06T03:09:39.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 -- 172.21.15.157:0/1198802809 learned_addr learned my addr 172.21.15.157:0/1198802809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:39.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d4e7fc640 1 --2- 172.21.15.157:0/1198802809 >> v2:172.21.15.195:3300/0 conn(0x7f3d580afb00 0x7f3d580b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 -- 172.21.15.157:0/1198802809 >> v2:172.21.15.195:3300/0 conn(0x7f3d580afb00 msgr2=0x7f3d580b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:39.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 --2- 172.21.15.157:0/1198802809 >> v2:172.21.15.195:3300/0 conn(0x7f3d580afb00 0x7f3d580b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 -- 172.21.15.157:0/1198802809 >> v1:172.21.15.157:6789/0 conn(0x7f3d580cc720 legacy=0x7f3d580ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 -- 172.21.15.157:0/1198802809 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580cfdf0 con 0x7f3d5815ac80 2023-11-06T03:09:39.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d567fc640 1 -- 172.21.15.157:0/1198802809 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598363972 0 0) 0x7f3d580cc2b0 con 0x7f3d580cc720 2023-11-06T03:09:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d56ffd640 1 --2- 172.21.15.157:0/1198802809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d5815ac80 0x7f3d580b2400 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3d580b5e40 tx=0x7f3d4001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9ff1cc08fa0841 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d567fc640 1 -- 172.21.15.157:0/1198802809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d580cda90 con 0x7f3d5815ac80 2023-11-06T03:09:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d567fc640 1 -- 172.21.15.157:0/1198802809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d400ab610 con 0x7f3d5815ac80 2023-11-06T03:09:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d567fc640 1 -- 172.21.15.157:0/1198802809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d400ab9a0 con 0x7f3d5815ac80 2023-11-06T03:09:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 -- 172.21.15.157:0/1198802809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d5815ac80 msgr2=0x7f3d580b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1198802809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d5815ac80 0x7f3d580b2400 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3d580b5e40 tx=0x7f3d4001c500 comp rx=0 tx=0).stop 2023-11-06T03:09:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 -- 172.21.15.157:0/1198802809 shutdown_connections 2023-11-06T03:09:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1198802809 >> v2:172.21.15.195:3300/0 conn(0x7f3d580afb00 0x7f3d580b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.060+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1198802809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d5815ac80 0x7f3d580b2400 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1198802809 >> 172.21.15.157:0/1198802809 conn(0x7f3d5806d560 msgr2=0x7f3d5806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1198802809 shutdown_connections 2023-11-06T03:09:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1198802809 wait complete. 2023-11-06T03:09:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 Processor -- start 2023-11-06T03:09:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- start start 2023-11-06T03:09:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d580afb00 0x7f3d58164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d5815ac80 0x7f3d581628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581631f0 con 0x7f3d580cc720 2023-11-06T03:09:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581634d0 con 0x7f3d580afb00 2023-11-06T03:09:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581637b0 con 0x7f3d5815ac80 2023-11-06T03:09:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46856/0 (socket says 172.21.15.157:46856) 2023-11-06T03:09:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 -- 172.21.15.157:0/1022073191 learned_addr learned my addr 172.21.15.157:0/1022073191 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d577fe640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d5815ac80 0x7f3d581628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d56ffd640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d580afb00 0x7f3d58164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 -- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d5815ac80 msgr2=0x7f3d581628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d5815ac80 0x7f3d581628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 -- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d580afb00 msgr2=0x7f3d58164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d580afb00 0x7f3d58164290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 -- 172.21.15.157:0/1022073191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400094d0 con 0x7f3d580cc720 2023-11-06T03:09:39.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4e7fc640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c7f0 tx=0x7f3d4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48004220 con 0x7f3d580cc720 2023-11-06T03:09:39.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d480043a0 con 0x7f3d580cc720 2023-11-06T03:09:39.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d58163a90 con 0x7f3d580cc720 2023-11-06T03:09:39.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48004510 con 0x7f3d580cc720 2023-11-06T03:09:39.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5806b620 con 0x7f3d580cc720 2023-11-06T03:09:39.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.064+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d58072750 con 0x7f3d580cc720 2023-11-06T03:09:39.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.068+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3d48003a70 con 0x7f3d580cc720 2023-11-06T03:09:39.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.068+0000 7f3d4ffff640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 0x7f3d38067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.068+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d48083230 con 0x7f3d580cc720 2023-11-06T03:09:39.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.068+0000 7f3d56ffd640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 0x7f3d38067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.068+0000 7f3d56ffd640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 0x7f3d38067f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d400af000 tx=0x7f3d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.072+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3d48050b20 con 0x7f3d580cc720 2023-11-06T03:09:39.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.276+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3d58163f40 con 0x7f3d580cc720 2023-11-06T03:09:39.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d4ffff640 1 -- 172.21.15.157:0/1022073191 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1529 (secure 0 0 0) 0x7f3d48054140 con 0x7f3d580cc720 2023-11-06T03:09:39.278 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:39.279 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":356,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":370110464,"bytes_avail":772724002816,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":60,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7329}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2023-11-06T03:09:25.210523+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:09:39.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 msgr2=0x7f3d38067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 0x7f3d38067f20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d400af000 tx=0x7f3d40008040 comp rx=0 tx=0).stop 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 msgr2=0x7f3d5806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c7f0 tx=0x7f3d4800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 shutdown_connections 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3d38065a60 0x7f3d38067f20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3d5815ac80 0x7f3d581628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d580cc720 0x7f3d5806b0e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 --2- 172.21.15.157:0/1022073191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3d580afb00 0x7f3d58164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 >> 172.21.15.157:0/1022073191 conn(0x7f3d5806d560 msgr2=0x7f3d580ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:39.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 shutdown_connections 2023-11-06T03:09:39.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.280+0000 7f3d5d727640 1 -- 172.21.15.157:0/1022073191 wait complete. 2023-11-06T03:09:39.296 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:09:39.296 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:09:39.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 Processor -- start 2023-11-06T03:09:39.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 -- start start 2023-11-06T03:09:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae415caa0 0x7f3ae40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40d2920 con 0x7f3ae40d0280 2023-11-06T03:09:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40d18a0 con 0x7f3ae415ce80 2023-11-06T03:09:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3aea44c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40d1b60 con 0x7f3ae415caa0 2023-11-06T03:09:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3ae89e9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3ae40d0280 0x7f3ae40b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54432/0 (socket says 172.21.15.157:54432) 2023-11-06T03:09:39.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3ae89e9640 1 -- 172.21.15.157:0/2554659737 learned_addr learned my addr 172.21.15.157:0/2554659737 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:39.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161036887 0 0) 0x7f3ae40d2920 con 0x7f3ae40d0280 2023-11-06T03:09:39.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3acc003530 con 0x7f3ae40d0280 2023-11-06T03:09:39.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.416+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae415caa0 0x7f3ae40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724265321 0 0) 0x7f3acc003530 con 0x7f3ae40d0280 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae415caa0 msgr2=0x7f3ae40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae415caa0 0x7f3ae40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 >> v1:172.21.15.195:6789/0 conn(0x7f3ae415ce80 legacy=0x7f3ae40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40d1e40 con 0x7f3ae40d0280 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3adc001680 con 0x7f3ae40d0280 2023-11-06T03:09:39.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 >> v1:172.21.15.157:6789/0 conn(0x7f3ae40d0280 legacy=0x7f3ae40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0039b0 0x7f3acc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3acc0047d0 0x7f3acc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3acc005190 0x7f3acc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40d1b60 con 0x7f3acc0047d0 2023-11-06T03:09:39.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc00c060 con 0x7f3acc005190 2023-11-06T03:09:39.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc00c220 con 0x7f3acc0039b0 2023-11-06T03:09:39.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae415caa0 0x7f3ae40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:39.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3acc0047d0 0x7f3acc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae89e9640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3acc005190 0x7f3acc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 -- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0039b0 msgr2=0x7f3acc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:39.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0039b0 0x7f3acc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 -- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3acc005190 msgr2=0x7f3acc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:39.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3acc005190 0x7f3acc00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 -- 172.21.15.157:0/2554659737 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc003530 con 0x7f3acc0047d0 2023-11-06T03:09:39.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae37fe640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3acc0047d0 0x7f3acc004c50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8001780 tx=0x7f3ad801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78c54899dd639d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80052d0 con 0x7f3acc0047d0 2023-11-06T03:09:39.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad80a7050 con 0x7f3acc0047d0 2023-11-06T03:09:39.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae2ffd640 1 -- 172.21.15.157:0/2554659737 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8003600 con 0x7f3acc0047d0 2023-11-06T03:09:39.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 -- 172.21.15.157:0/2554659737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3acc0047d0 msgr2=0x7f3acc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 --2- 172.21.15.157:0/2554659737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3acc0047d0 0x7f3acc004c50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8001780 tx=0x7f3ad801c400 comp rx=0 tx=0).stop 2023-11-06T03:09:39.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3ae89e9640 1 -- 172.21.15.157:0/2554659737 reap_dead start 2023-11-06T03:09:39.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 -- 172.21.15.157:0/2554659737 shutdown_connections 2023-11-06T03:09:39.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 -- 172.21.15.157:0/2554659737 >> 172.21.15.157:0/2554659737 conn(0x7f3ae4070790 msgr2=0x7f3ae4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:39.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 -- 172.21.15.157:0/2554659737 shutdown_connections 2023-11-06T03:09:39.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.420+0000 7f3aea44c640 1 -- 172.21.15.157:0/2554659737 wait complete. 2023-11-06T03:09:39.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 Processor -- start 2023-11-06T03:09:39.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- start start 2023-11-06T03:09:39.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 0x7f3ae41e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae40b3990 0x7f3ae40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40b4310 con 0x7f3ae41f2190 2023-11-06T03:09:39.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41f2f30 con 0x7f3ae40b3990 2023-11-06T03:09:39.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41f64c0 con 0x7f3acc0047d0 2023-11-06T03:09:39.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46862/0 (socket says 172.21.15.157:46862) 2023-11-06T03:09:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 -- 172.21.15.157:0/464676668 learned_addr learned my addr 172.21.15.157:0/464676668 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae37fe640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae40b3990 0x7f3ae40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 0x7f3ae41e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 -- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 msgr2=0x7f3ae41e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 0x7f3ae41e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 -- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae40b3990 msgr2=0x7f3ae40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae40b3990 0x7f3ae40b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 -- 172.21.15.157:0/464676668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae41f5700 con 0x7f3ae41f2190 2023-11-06T03:09:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 0x7f3ae41e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:39.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae89e9640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3adc000c10 tx=0x7f3adc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00eb80 con 0x7f3ae41f2190 2023-11-06T03:09:39.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00ece0 con 0x7f3ae41f2190 2023-11-06T03:09:39.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc015620 con 0x7f3ae41f2190 2023-11-06T03:09:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae41f59e0 con 0x7f3ae41f2190 2023-11-06T03:09:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41f5fa0 con 0x7f3ae41f2190 2023-11-06T03:09:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae41f6190 con 0x7f3ae41f2190 2023-11-06T03:09:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.424+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3adc0143a0 con 0x7f3ae41f2190 2023-11-06T03:09:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.428+0000 7f3ae0ff9640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 0x7f3ad0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.428+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3adc085c20 con 0x7f3ae41f2190 2023-11-06T03:09:39.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.428+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 0x7f3ad0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:39.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.428+0000 7f3ae3fff640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 0x7f3ad0067f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4004b20 tx=0x7f3ad400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:39.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.432+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3adc053510 con 0x7f3ae41f2190 2023-11-06T03:09:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.580+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ae40d2400 con 0x7f3ad0065a60 2023-11-06T03:09:39.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.604+0000 7f3ae0ff9640 1 -- 172.21.15.157:0/464676668 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309290 (secure 0 0 0) 0x7f3ad400f040 con 0x7f3ad0065a60 2023-11-06T03:09:39.606 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:09:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 msgr2=0x7f3ad0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 0x7f3ad0067f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4004b20 tx=0x7f3ad400b040 comp rx=0 tx=0).stop 2023-11-06T03:09:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 msgr2=0x7f3ae41f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3adc000c10 tx=0x7f3adc005760 comp rx=0 tx=0).stop 2023-11-06T03:09:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 shutdown_connections 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ad0065a60 0x7f3ad0067f20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae41f2190 0x7f3ae41f2610 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae40b3990 0x7f3ae40b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 --2- 172.21.15.157:0/464676668 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3acc0047d0 0x7f3ae41e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 >> 172.21.15.157:0/464676668 conn(0x7f3ae4070790 msgr2=0x7f3ae40b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 shutdown_connections 2023-11-06T03:09:39.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:39.608+0000 7f3aea44c640 1 -- 172.21.15.157:0/464676668 wait complete. 2023-11-06T03:09:39.625 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2023-11-06T03:09:39.214543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":198,"acting":198,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361832,"kb_used_data":9256,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754612888,"statfs":{"total":773094113280,"available":772723597312,"internally_reserved":0,"allocated":9478144,"data_stored":4939442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.475040"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454645+0000","last_change":"2023-11-06T03:09:35.151600+0000","last_active":"2023-11-06T03:09:36.454645+0000","last_peered":"2023-11-06T03:09:36.454645+0000","last_clean":"2023-11-06T03:09:36.454645+0000","last_became_active":"2023-11-06T03:09:35.151343+0000","last_became_peered":"2023-11-06T03:09:35.151343+0000","last_unstale":"2023-11-06T03:09:36.454645+0000","last_undegraded":"2023-11-06T03:09:36.454645+0000","last_fullsized":"2023-11-06T03:09:36.454645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:25.279812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142950+0000","last_change":"2023-11-06T03:09:35.152944+0000","last_active":"2023-11-06T03:09:36.142950+0000","last_peered":"2023-11-06T03:09:36.142950+0000","last_clean":"2023-11-06T03:09:36.142950+0000","last_became_active":"2023-11-06T03:09:35.152688+0000","last_became_peered":"2023-11-06T03:09:35.152688+0000","last_unstale":"2023-11-06T03:09:36.142950+0000","last_undegraded":"2023-11-06T03:09:36.142950+0000","last_fullsized":"2023-11-06T03:09:36.142950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237504+0000","last_change":"2023-11-06T03:09:35.196142+0000","last_active":"2023-11-06T03:09:37.237504+0000","last_peered":"2023-11-06T03:09:37.237504+0000","last_clean":"2023-11-06T03:09:37.237504+0000","last_became_active":"2023-11-06T03:09:35.194760+0000","last_became_peered":"2023-11-06T03:09:35.194760+0000","last_unstale":"2023-11-06T03:09:37.237504+0000","last_undegraded":"2023-11-06T03:09:37.237504+0000","last_fullsized":"2023-11-06T03:09:37.237504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736012+0000","last_change":"2023-11-06T03:09:35.179907+0000","last_active":"2023-11-06T03:09:36.736012+0000","last_peered":"2023-11-06T03:09:36.736012+0000","last_clean":"2023-11-06T03:09:36.736012+0000","last_became_active":"2023-11-06T03:09:35.179643+0000","last_became_peered":"2023-11-06T03:09:35.179643+0000","last_unstale":"2023-11-06T03:09:36.736012+0000","last_undegraded":"2023-11-06T03:09:36.736012+0000","last_fullsized":"2023-11-06T03:09:36.736012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:19.415560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235345+0000","last_change":"2023-11-06T03:09:35.172687+0000","last_active":"2023-11-06T03:09:37.235345+0000","last_peered":"2023-11-06T03:09:37.235345+0000","last_clean":"2023-11-06T03:09:37.235345+0000","last_became_active":"2023-11-06T03:09:35.172474+0000","last_became_peered":"2023-11-06T03:09:35.172474+0000","last_unstale":"2023-11-06T03:09:37.235345+0000","last_undegraded":"2023-11-06T03:09:37.235345+0000","last_fullsized":"2023-11-06T03:09:37.235345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:27.705396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781026+0000","last_change":"2023-11-06T03:09:35.148110+0000","last_active":"2023-11-06T03:09:36.781026+0000","last_peered":"2023-11-06T03:09:36.781026+0000","last_clean":"2023-11-06T03:09:36.781026+0000","last_became_active":"2023-11-06T03:09:35.147260+0000","last_became_peered":"2023-11-06T03:09:35.147260+0000","last_unstale":"2023-11-06T03:09:36.781026+0000","last_undegraded":"2023-11-06T03:09:36.781026+0000","last_fullsized":"2023-11-06T03:09:36.781026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:02:55.853075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735559+0000","last_change":"2023-11-06T03:09:35.168034+0000","last_active":"2023-11-06T03:09:36.735559+0000","last_peered":"2023-11-06T03:09:36.735559+0000","last_clean":"2023-11-06T03:09:36.735559+0000","last_became_active":"2023-11-06T03:09:35.167812+0000","last_became_peered":"2023-11-06T03:09:35.167812+0000","last_unstale":"2023-11-06T03:09:36.735559+0000","last_undegraded":"2023-11-06T03:09:36.735559+0000","last_fullsized":"2023-11-06T03:09:36.735559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:45.100372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.147397+0000","last_change":"2023-11-06T03:09:35.172426+0000","last_active":"2023-11-06T03:09:36.147397+0000","last_peered":"2023-11-06T03:09:36.147397+0000","last_clean":"2023-11-06T03:09:36.147397+0000","last_became_active":"2023-11-06T03:09:35.171647+0000","last_became_peered":"2023-11-06T03:09:35.171647+0000","last_unstale":"2023-11-06T03:09:36.147397+0000","last_undegraded":"2023-11-06T03:09:36.147397+0000","last_fullsized":"2023-11-06T03:09:36.147397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:19.690734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237665+0000","last_change":"2023-11-06T03:09:35.193811+0000","last_active":"2023-11-06T03:09:37.237665+0000","last_peered":"2023-11-06T03:09:37.237665+0000","last_clean":"2023-11-06T03:09:37.237665+0000","last_became_active":"2023-11-06T03:09:35.192825+0000","last_became_peered":"2023-11-06T03:09:35.192825+0000","last_unstale":"2023-11-06T03:09:37.237665+0000","last_undegraded":"2023-11-06T03:09:37.237665+0000","last_fullsized":"2023-11-06T03:09:37.237665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:12.925128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.455253+0000","last_change":"2023-11-06T03:09:35.152556+0000","last_active":"2023-11-06T03:09:36.455253+0000","last_peered":"2023-11-06T03:09:36.455253+0000","last_clean":"2023-11-06T03:09:36.455253+0000","last_became_active":"2023-11-06T03:09:35.152136+0000","last_became_peered":"2023-11-06T03:09:35.152136+0000","last_unstale":"2023-11-06T03:09:36.455253+0000","last_undegraded":"2023-11-06T03:09:36.455253+0000","last_fullsized":"2023-11-06T03:09:36.455253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:28.829494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237604+0000","last_change":"2023-11-06T03:09:35.195653+0000","last_active":"2023-11-06T03:09:37.237604+0000","last_peered":"2023-11-06T03:09:37.237604+0000","last_clean":"2023-11-06T03:09:37.237604+0000","last_became_active":"2023-11-06T03:09:35.193804+0000","last_became_peered":"2023-11-06T03:09:35.193804+0000","last_unstale":"2023-11-06T03:09:37.237604+0000","last_undegraded":"2023-11-06T03:09:37.237604+0000","last_fullsized":"2023-11-06T03:09:37.237604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736041+0000","last_change":"2023-11-06T03:09:35.157240+0000","last_active":"2023-11-06T03:09:36.736041+0000","last_peered":"2023-11-06T03:09:36.736041+0000","last_clean":"2023-11-06T03:09:36.736041+0000","last_became_active":"2023-11-06T03:09:35.156975+0000","last_became_peered":"2023-11-06T03:09:35.156975+0000","last_unstale":"2023-11-06T03:09:36.736041+0000","last_undegraded":"2023-11-06T03:09:36.736041+0000","last_fullsized":"2023-11-06T03:09:36.736041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:59.270002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235210+0000","last_change":"2023-11-06T03:09:35.172640+0000","last_active":"2023-11-06T03:09:37.235210+0000","last_peered":"2023-11-06T03:09:37.235210+0000","last_clean":"2023-11-06T03:09:37.235210+0000","last_became_active":"2023-11-06T03:09:35.172364+0000","last_became_peered":"2023-11-06T03:09:35.172364+0000","last_unstale":"2023-11-06T03:09:37.235210+0000","last_undegraded":"2023-11-06T03:09:37.235210+0000","last_fullsized":"2023-11-06T03:09:37.235210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:52:24.016692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454236+0000","last_change":"2023-11-06T03:09:35.151071+0000","last_active":"2023-11-06T03:09:36.454236+0000","last_peered":"2023-11-06T03:09:36.454236+0000","last_clean":"2023-11-06T03:09:36.454236+0000","last_became_active":"2023-11-06T03:09:35.150807+0000","last_became_peered":"2023-11-06T03:09:35.150807+0000","last_unstale":"2023-11-06T03:09:36.454236+0000","last_undegraded":"2023-11-06T03:09:36.454236+0000","last_fullsized":"2023-11-06T03:09:36.454236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:01:07.698040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736430+0000","last_change":"2023-11-06T03:09:35.171453+0000","last_active":"2023-11-06T03:09:36.736430+0000","last_peered":"2023-11-06T03:09:36.736430+0000","last_clean":"2023-11-06T03:09:36.736430+0000","last_became_active":"2023-11-06T03:09:35.171190+0000","last_became_peered":"2023-11-06T03:09:35.171190+0000","last_unstale":"2023-11-06T03:09:36.736430+0000","last_undegraded":"2023-11-06T03:09:36.736430+0000","last_fullsized":"2023-11-06T03:09:36.736430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:24.270703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142805+0000","last_change":"2023-11-06T03:09:35.172514+0000","last_active":"2023-11-06T03:09:36.142805+0000","last_peered":"2023-11-06T03:09:36.142805+0000","last_clean":"2023-11-06T03:09:36.142805+0000","last_became_active":"2023-11-06T03:09:35.172222+0000","last_became_peered":"2023-11-06T03:09:35.172222+0000","last_unstale":"2023-11-06T03:09:36.142805+0000","last_undegraded":"2023-11-06T03:09:36.142805+0000","last_fullsized":"2023-11-06T03:09:36.142805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235439+0000","last_change":"2023-11-06T03:09:35.155783+0000","last_active":"2023-11-06T03:09:37.235439+0000","last_peered":"2023-11-06T03:09:37.235439+0000","last_clean":"2023-11-06T03:09:37.235439+0000","last_became_active":"2023-11-06T03:09:35.155501+0000","last_became_peered":"2023-11-06T03:09:35.155501+0000","last_unstale":"2023-11-06T03:09:37.235439+0000","last_undegraded":"2023-11-06T03:09:37.235439+0000","last_fullsized":"2023-11-06T03:09:37.235439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:23.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736107+0000","last_change":"2023-11-06T03:09:35.142864+0000","last_active":"2023-11-06T03:09:36.736107+0000","last_peered":"2023-11-06T03:09:36.736107+0000","last_clean":"2023-11-06T03:09:36.736107+0000","last_became_active":"2023-11-06T03:09:35.142500+0000","last_became_peered":"2023-11-06T03:09:35.142500+0000","last_unstale":"2023-11-06T03:09:36.736107+0000","last_undegraded":"2023-11-06T03:09:36.736107+0000","last_fullsized":"2023-11-06T03:09:36.736107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:19:54.903256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146609+0000","last_change":"2023-11-06T03:09:35.173385+0000","last_active":"2023-11-06T03:09:36.146609+0000","last_peered":"2023-11-06T03:09:36.146609+0000","last_clean":"2023-11-06T03:09:36.146609+0000","last_became_active":"2023-11-06T03:09:35.172525+0000","last_became_peered":"2023-11-06T03:09:35.172525+0000","last_unstale":"2023-11-06T03:09:36.146609+0000","last_undegraded":"2023-11-06T03:09:36.146609+0000","last_fullsized":"2023-11-06T03:09:36.146609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735770+0000","last_change":"2023-11-06T03:09:35.157771+0000","last_active":"2023-11-06T03:09:36.735770+0000","last_peered":"2023-11-06T03:09:36.735770+0000","last_clean":"2023-11-06T03:09:36.735770+0000","last_became_active":"2023-11-06T03:09:35.157525+0000","last_became_peered":"2023-11-06T03:09:35.157525+0000","last_unstale":"2023-11-06T03:09:36.735770+0000","last_undegraded":"2023-11-06T03:09:36.735770+0000","last_fullsized":"2023-11-06T03:09:36.735770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:27.906246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146325+0000","last_change":"2023-11-06T03:09:35.196860+0000","last_active":"2023-11-06T03:09:36.146325+0000","last_peered":"2023-11-06T03:09:36.146325+0000","last_clean":"2023-11-06T03:09:36.146325+0000","last_became_active":"2023-11-06T03:09:35.196417+0000","last_became_peered":"2023-11-06T03:09:35.196417+0000","last_unstale":"2023-11-06T03:09:36.146325+0000","last_undegraded":"2023-11-06T03:09:36.146325+0000","last_fullsized":"2023-11-06T03:09:36.146325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:13.064955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454387+0000","last_change":"2023-11-06T03:09:35.157458+0000","last_active":"2023-11-06T03:09:36.454387+0000","last_peered":"2023-11-06T03:09:36.454387+0000","last_clean":"2023-11-06T03:09:36.454387+0000","last_became_active":"2023-11-06T03:09:35.157209+0000","last_became_peered":"2023-11-06T03:09:35.157209+0000","last_unstale":"2023-11-06T03:09:36.454387+0000","last_undegraded":"2023-11-06T03:09:36.454387+0000","last_fullsized":"2023-11-06T03:09:36.454387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:02:14.560666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454690+0000","last_change":"2023-11-06T03:09:35.156857+0000","last_active":"2023-11-06T03:09:36.454690+0000","last_peered":"2023-11-06T03:09:36.454690+0000","last_clean":"2023-11-06T03:09:36.454690+0000","last_became_active":"2023-11-06T03:09:35.156610+0000","last_became_peered":"2023-11-06T03:09:35.156610+0000","last_unstale":"2023-11-06T03:09:36.454690+0000","last_undegraded":"2023-11-06T03:09:36.454690+0000","last_fullsized":"2023-11-06T03:09:36.454690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:56:16.942337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237752+0000","last_change":"2023-11-06T03:09:35.193193+0000","last_active":"2023-11-06T03:09:37.237752+0000","last_peered":"2023-11-06T03:09:37.237752+0000","last_clean":"2023-11-06T03:09:37.237752+0000","last_became_active":"2023-11-06T03:09:35.192840+0000","last_became_peered":"2023-11-06T03:09:35.192840+0000","last_unstale":"2023-11-06T03:09:37.237752+0000","last_undegraded":"2023-11-06T03:09:37.237752+0000","last_fullsized":"2023-11-06T03:09:37.237752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736266+0000","last_change":"2023-11-06T03:09:35.194699+0000","last_active":"2023-11-06T03:09:36.736266+0000","last_peered":"2023-11-06T03:09:36.736266+0000","last_clean":"2023-11-06T03:09:36.736266+0000","last_became_active":"2023-11-06T03:09:35.194405+0000","last_became_peered":"2023-11-06T03:09:35.194405+0000","last_unstale":"2023-11-06T03:09:36.736266+0000","last_undegraded":"2023-11-06T03:09:36.736266+0000","last_fullsized":"2023-11-06T03:09:36.736266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:59.280475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454749+0000","last_change":"2023-11-06T03:09:35.173392+0000","last_active":"2023-11-06T03:09:36.454749+0000","last_peered":"2023-11-06T03:09:36.454749+0000","last_clean":"2023-11-06T03:09:36.454749+0000","last_became_active":"2023-11-06T03:09:35.173081+0000","last_became_peered":"2023-11-06T03:09:35.173081+0000","last_unstale":"2023-11-06T03:09:36.454749+0000","last_undegraded":"2023-11-06T03:09:36.454749+0000","last_fullsized":"2023-11-06T03:09:36.454749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:24.553362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142557+0000","last_change":"2023-11-06T03:09:35.173375+0000","last_active":"2023-11-06T03:09:36.142557+0000","last_peered":"2023-11-06T03:09:36.142557+0000","last_clean":"2023-11-06T03:09:36.142557+0000","last_became_active":"2023-11-06T03:09:35.173093+0000","last_became_peered":"2023-11-06T03:09:35.173093+0000","last_unstale":"2023-11-06T03:09:36.142557+0000","last_undegraded":"2023-11-06T03:09:36.142557+0000","last_fullsized":"2023-11-06T03:09:36.142557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:36.739383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142582+0000","last_change":"2023-11-06T03:09:35.174487+0000","last_active":"2023-11-06T03:09:36.142582+0000","last_peered":"2023-11-06T03:09:36.142582+0000","last_clean":"2023-11-06T03:09:36.142582+0000","last_became_active":"2023-11-06T03:09:35.174279+0000","last_became_peered":"2023-11-06T03:09:35.174279+0000","last_unstale":"2023-11-06T03:09:36.142582+0000","last_undegraded":"2023-11-06T03:09:36.142582+0000","last_fullsized":"2023-11-06T03:09:36.142582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:35.444322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"84'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.156865+0000","last_change":"2023-11-06T03:09:34.777260+0000","last_active":"2023-11-06T03:09:35.156865+0000","last_peered":"2023-11-06T03:09:35.156865+0000","last_clean":"2023-11-06T03:09:35.156865+0000","last_became_active":"2023-11-06T03:09:34.776979+0000","last_became_peered":"2023-11-06T03:09:34.776979+0000","last_unstale":"2023-11-06T03:09:35.156865+0000","last_undegraded":"2023-11-06T03:09:35.156865+0000","last_fullsized":"2023-11-06T03:09:35.156865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454397+0000","last_change":"2023-11-06T03:09:35.192364+0000","last_active":"2023-11-06T03:09:36.454397+0000","last_peered":"2023-11-06T03:09:36.454397+0000","last_clean":"2023-11-06T03:09:36.454397+0000","last_became_active":"2023-11-06T03:09:35.192099+0000","last_became_peered":"2023-11-06T03:09:35.192099+0000","last_unstale":"2023-11-06T03:09:36.454397+0000","last_undegraded":"2023-11-06T03:09:36.454397+0000","last_fullsized":"2023-11-06T03:09:36.454397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:33:45.886436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746611+0000","last_change":"2023-11-06T03:09:33.746611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746611+0000","last_undegraded":"2023-11-06T03:09:33.746611+0000","last_fullsized":"2023-11-06T03:09:33.746611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146571+0000","last_change":"2023-11-06T03:09:35.172915+0000","last_active":"2023-11-06T03:09:36.146571+0000","last_peered":"2023-11-06T03:09:36.146571+0000","last_clean":"2023-11-06T03:09:36.146571+0000","last_became_active":"2023-11-06T03:09:35.171804+0000","last_became_peered":"2023-11-06T03:09:35.171804+0000","last_unstale":"2023-11-06T03:09:36.146571+0000","last_undegraded":"2023-11-06T03:09:36.146571+0000","last_fullsized":"2023-11-06T03:09:36.146571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.751332+0000","last_change":"2023-11-06T03:09:33.751332+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.751332+0000","last_undegraded":"2023-11-06T03:09:33.751332+0000","last_fullsized":"2023-11-06T03:09:33.751332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146516+0000","last_change":"2023-11-06T03:09:35.162931+0000","last_active":"2023-11-06T03:09:36.146516+0000","last_peered":"2023-11-06T03:09:36.146516+0000","last_clean":"2023-11-06T03:09:36.146516+0000","last_became_active":"2023-11-06T03:09:35.162176+0000","last_became_peered":"2023-11-06T03:09:35.162176+0000","last_unstale":"2023-11-06T03:09:36.146516+0000","last_undegraded":"2023-11-06T03:09:36.146516+0000","last_fullsized":"2023-11-06T03:09:36.146516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:22.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.191024+0000","last_change":"2023-11-06T03:09:34.791693+0000","last_active":"2023-11-06T03:09:35.191024+0000","last_peered":"2023-11-06T03:09:35.191024+0000","last_clean":"2023-11-06T03:09:35.191024+0000","last_became_active":"2023-11-06T03:09:34.790912+0000","last_became_peered":"2023-11-06T03:09:34.790912+0000","last_unstale":"2023-11-06T03:09:35.191024+0000","last_undegraded":"2023-11-06T03:09:35.191024+0000","last_fullsized":"2023-11-06T03:09:35.191024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454862+0000","last_change":"2023-11-06T03:09:35.156192+0000","last_active":"2023-11-06T03:09:36.454862+0000","last_peered":"2023-11-06T03:09:36.454862+0000","last_clean":"2023-11-06T03:09:36.454862+0000","last_became_active":"2023-11-06T03:09:35.155956+0000","last_became_peered":"2023-11-06T03:09:35.155956+0000","last_unstale":"2023-11-06T03:09:36.454862+0000","last_undegraded":"2023-11-06T03:09:36.454862+0000","last_fullsized":"2023-11-06T03:09:36.454862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.407814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.180761+0000","last_change":"2023-11-06T03:09:34.773862+0000","last_active":"2023-11-06T03:09:35.180761+0000","last_peered":"2023-11-06T03:09:35.180761+0000","last_clean":"2023-11-06T03:09:35.180761+0000","last_became_active":"2023-11-06T03:09:34.773071+0000","last_became_peered":"2023-11-06T03:09:34.773071+0000","last_unstale":"2023-11-06T03:09:35.180761+0000","last_undegraded":"2023-11-06T03:09:35.180761+0000","last_fullsized":"2023-11-06T03:09:35.180761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.455215+0000","last_change":"2023-11-06T03:09:35.152376+0000","last_active":"2023-11-06T03:09:36.455215+0000","last_peered":"2023-11-06T03:09:36.455215+0000","last_clean":"2023-11-06T03:09:36.455215+0000","last_became_active":"2023-11-06T03:09:35.151873+0000","last_became_peered":"2023-11-06T03:09:35.151873+0000","last_unstale":"2023-11-06T03:09:36.455215+0000","last_undegraded":"2023-11-06T03:09:36.455215+0000","last_fullsized":"2023-11-06T03:09:36.455215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:18.193688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.192692+0000","last_change":"2023-11-06T03:09:34.779144+0000","last_active":"2023-11-06T03:09:35.192692+0000","last_peered":"2023-11-06T03:09:35.192692+0000","last_clean":"2023-11-06T03:09:35.192692+0000","last_became_active":"2023-11-06T03:09:34.778339+0000","last_became_peered":"2023-11-06T03:09:34.778339+0000","last_unstale":"2023-11-06T03:09:35.192692+0000","last_undegraded":"2023-11-06T03:09:35.192692+0000","last_fullsized":"2023-11-06T03:09:35.192692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.147065+0000","last_change":"2023-11-06T03:09:35.173260+0000","last_active":"2023-11-06T03:09:36.147065+0000","last_peered":"2023-11-06T03:09:36.147065+0000","last_clean":"2023-11-06T03:09:36.147065+0000","last_became_active":"2023-11-06T03:09:35.171398+0000","last_became_peered":"2023-11-06T03:09:35.171398+0000","last_unstale":"2023-11-06T03:09:36.147065+0000","last_undegraded":"2023-11-06T03:09:36.147065+0000","last_fullsized":"2023-11-06T03:09:36.147065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:37.347892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.148321+0000","last_change":"2023-11-06T03:09:34.780135+0000","last_active":"2023-11-06T03:09:35.148321+0000","last_peered":"2023-11-06T03:09:35.148321+0000","last_clean":"2023-11-06T03:09:35.148321+0000","last_became_active":"2023-11-06T03:09:34.779866+0000","last_became_peered":"2023-11-06T03:09:34.779866+0000","last_unstale":"2023-11-06T03:09:35.148321+0000","last_undegraded":"2023-11-06T03:09:35.148321+0000","last_fullsized":"2023-11-06T03:09:35.148321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781689+0000","last_change":"2023-11-06T03:09:35.171039+0000","last_active":"2023-11-06T03:09:36.781689+0000","last_peered":"2023-11-06T03:09:36.781689+0000","last_clean":"2023-11-06T03:09:36.781689+0000","last_became_active":"2023-11-06T03:09:35.169738+0000","last_became_peered":"2023-11-06T03:09:35.169738+0000","last_unstale":"2023-11-06T03:09:36.781689+0000","last_undegraded":"2023-11-06T03:09:36.781689+0000","last_fullsized":"2023-11-06T03:09:36.781689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:29.176361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.133879+0000","last_change":"2023-11-06T03:09:34.779247+0000","last_active":"2023-11-06T03:09:35.133879+0000","last_peered":"2023-11-06T03:09:35.133879+0000","last_clean":"2023-11-06T03:09:35.133879+0000","last_became_active":"2023-11-06T03:09:34.778750+0000","last_became_peered":"2023-11-06T03:09:34.778750+0000","last_unstale":"2023-11-06T03:09:35.133879+0000","last_undegraded":"2023-11-06T03:09:35.133879+0000","last_fullsized":"2023-11-06T03:09:35.133879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454611+0000","last_change":"2023-11-06T03:09:35.172164+0000","last_active":"2023-11-06T03:09:36.454611+0000","last_peered":"2023-11-06T03:09:36.454611+0000","last_clean":"2023-11-06T03:09:36.454611+0000","last_became_active":"2023-11-06T03:09:35.171854+0000","last_became_peered":"2023-11-06T03:09:35.171854+0000","last_unstale":"2023-11-06T03:09:36.454611+0000","last_undegraded":"2023-11-06T03:09:36.454611+0000","last_fullsized":"2023-11-06T03:09:36.454611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:17.213468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748920+0000","last_change":"2023-11-06T03:09:33.748920+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748920+0000","last_undegraded":"2023-11-06T03:09:33.748920+0000","last_fullsized":"2023-11-06T03:09:33.748920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237258+0000","last_change":"2023-11-06T03:09:35.179533+0000","last_active":"2023-11-06T03:09:37.237258+0000","last_peered":"2023-11-06T03:09:37.237258+0000","last_clean":"2023-11-06T03:09:37.237258+0000","last_became_active":"2023-11-06T03:09:35.178735+0000","last_became_peered":"2023-11-06T03:09:35.178735+0000","last_unstale":"2023-11-06T03:09:37.237258+0000","last_undegraded":"2023-11-06T03:09:37.237258+0000","last_fullsized":"2023-11-06T03:09:37.237258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:35.444322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160351+0000","last_change":"2023-11-06T03:09:34.777403+0000","last_active":"2023-11-06T03:09:35.160351+0000","last_peered":"2023-11-06T03:09:35.160351+0000","last_clean":"2023-11-06T03:09:35.160351+0000","last_became_active":"2023-11-06T03:09:34.776668+0000","last_became_peered":"2023-11-06T03:09:34.776668+0000","last_unstale":"2023-11-06T03:09:35.160351+0000","last_undegraded":"2023-11-06T03:09:35.160351+0000","last_fullsized":"2023-11-06T03:09:35.160351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235400+0000","last_change":"2023-11-06T03:09:35.156712+0000","last_active":"2023-11-06T03:09:37.235400+0000","last_peered":"2023-11-06T03:09:37.235400+0000","last_clean":"2023-11-06T03:09:37.235400+0000","last_became_active":"2023-11-06T03:09:35.156362+0000","last_became_peered":"2023-11-06T03:09:35.156362+0000","last_unstale":"2023-11-06T03:09:37.235400+0000","last_undegraded":"2023-11-06T03:09:37.235400+0000","last_fullsized":"2023-11-06T03:09:37.235400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:24:43.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"84'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.149345+0000","last_change":"2023-11-06T03:09:34.774633+0000","last_active":"2023-11-06T03:09:35.149345+0000","last_peered":"2023-11-06T03:09:35.149345+0000","last_clean":"2023-11-06T03:09:35.149345+0000","last_became_active":"2023-11-06T03:09:34.773898+0000","last_became_peered":"2023-11-06T03:09:34.773898+0000","last_unstale":"2023-11-06T03:09:35.149345+0000","last_undegraded":"2023-11-06T03:09:35.149345+0000","last_fullsized":"2023-11-06T03:09:35.149345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142736+0000","last_change":"2023-11-06T03:09:35.153554+0000","last_active":"2023-11-06T03:09:36.142736+0000","last_peered":"2023-11-06T03:09:36.142736+0000","last_clean":"2023-11-06T03:09:36.142736+0000","last_became_active":"2023-11-06T03:09:35.153292+0000","last_became_peered":"2023-11-06T03:09:35.153292+0000","last_unstale":"2023-11-06T03:09:36.142736+0000","last_undegraded":"2023-11-06T03:09:36.142736+0000","last_fullsized":"2023-11-06T03:09:36.142736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"84'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.198516+0000","last_change":"2023-11-06T03:09:34.791639+0000","last_active":"2023-11-06T03:09:35.198516+0000","last_peered":"2023-11-06T03:09:35.198516+0000","last_clean":"2023-11-06T03:09:35.198516+0000","last_became_active":"2023-11-06T03:09:34.790932+0000","last_became_peered":"2023-11-06T03:09:34.790932+0000","last_unstale":"2023-11-06T03:09:35.198516+0000","last_undegraded":"2023-11-06T03:09:35.198516+0000","last_fullsized":"2023-11-06T03:09:35.198516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237776+0000","last_change":"2023-11-06T03:09:35.193265+0000","last_active":"2023-11-06T03:09:37.237776+0000","last_peered":"2023-11-06T03:09:37.237776+0000","last_clean":"2023-11-06T03:09:37.237776+0000","last_became_active":"2023-11-06T03:09:35.192506+0000","last_became_peered":"2023-11-06T03:09:35.192506+0000","last_unstale":"2023-11-06T03:09:37.237776+0000","last_undegraded":"2023-11-06T03:09:37.237776+0000","last_fullsized":"2023-11-06T03:09:37.237776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:36.739383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.194945+0000","last_change":"2023-11-06T03:09:34.787497+0000","last_active":"2023-11-06T03:09:35.194945+0000","last_peered":"2023-11-06T03:09:35.194945+0000","last_clean":"2023-11-06T03:09:35.194945+0000","last_became_active":"2023-11-06T03:09:34.786691+0000","last_became_peered":"2023-11-06T03:09:34.786691+0000","last_unstale":"2023-11-06T03:09:35.194945+0000","last_undegraded":"2023-11-06T03:09:35.194945+0000","last_fullsized":"2023-11-06T03:09:35.194945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781232+0000","last_change":"2023-11-06T03:09:35.153326+0000","last_active":"2023-11-06T03:09:36.781232+0000","last_peered":"2023-11-06T03:09:36.781232+0000","last_clean":"2023-11-06T03:09:36.781232+0000","last_became_active":"2023-11-06T03:09:35.152580+0000","last_became_peered":"2023-11-06T03:09:35.152580+0000","last_unstale":"2023-11-06T03:09:36.781232+0000","last_undegraded":"2023-11-06T03:09:36.781232+0000","last_fullsized":"2023-11-06T03:09:36.781232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:32.992736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.746227+0000","last_change":"2023-11-06T03:09:33.746227+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.746227+0000","last_undegraded":"2023-11-06T03:09:33.746227+0000","last_fullsized":"2023-11-06T03:09:33.746227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781490+0000","last_change":"2023-11-06T03:09:35.184432+0000","last_active":"2023-11-06T03:09:36.781490+0000","last_peered":"2023-11-06T03:09:36.781490+0000","last_clean":"2023-11-06T03:09:36.781490+0000","last_became_active":"2023-11-06T03:09:35.183743+0000","last_became_peered":"2023-11-06T03:09:35.183743+0000","last_unstale":"2023-11-06T03:09:36.781490+0000","last_undegraded":"2023-11-06T03:09:36.781490+0000","last_fullsized":"2023-11-06T03:09:36.781490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:51.392820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134148+0000","last_change":"2023-11-06T03:09:34.776812+0000","last_active":"2023-11-06T03:09:35.134148+0000","last_peered":"2023-11-06T03:09:35.134148+0000","last_clean":"2023-11-06T03:09:35.134148+0000","last_became_active":"2023-11-06T03:09:34.776519+0000","last_became_peered":"2023-11-06T03:09:34.776519+0000","last_unstale":"2023-11-06T03:09:35.134148+0000","last_undegraded":"2023-11-06T03:09:35.134148+0000","last_fullsized":"2023-11-06T03:09:35.134148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781736+0000","last_change":"2023-11-06T03:09:35.170548+0000","last_active":"2023-11-06T03:09:36.781736+0000","last_peered":"2023-11-06T03:09:36.781736+0000","last_clean":"2023-11-06T03:09:36.781736+0000","last_became_active":"2023-11-06T03:09:35.168906+0000","last_became_peered":"2023-11-06T03:09:35.168906+0000","last_unstale":"2023-11-06T03:09:36.781736+0000","last_undegraded":"2023-11-06T03:09:36.781736+0000","last_fullsized":"2023-11-06T03:09:36.781736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:01.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"84'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.187347+0000","last_change":"2023-11-06T03:09:34.783382+0000","last_active":"2023-11-06T03:09:35.187347+0000","last_peered":"2023-11-06T03:09:35.187347+0000","last_clean":"2023-11-06T03:09:35.187347+0000","last_became_active":"2023-11-06T03:09:34.783101+0000","last_became_peered":"2023-11-06T03:09:34.783101+0000","last_unstale":"2023-11-06T03:09:35.187347+0000","last_undegraded":"2023-11-06T03:09:35.187347+0000","last_fullsized":"2023-11-06T03:09:35.187347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146266+0000","last_change":"2023-11-06T03:09:35.189348+0000","last_active":"2023-11-06T03:09:36.146266+0000","last_peered":"2023-11-06T03:09:36.146266+0000","last_clean":"2023-11-06T03:09:36.146266+0000","last_became_active":"2023-11-06T03:09:35.188417+0000","last_became_peered":"2023-11-06T03:09:35.188417+0000","last_unstale":"2023-11-06T03:09:36.146266+0000","last_undegraded":"2023-11-06T03:09:36.146266+0000","last_fullsized":"2023-11-06T03:09:36.146266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:35.774307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.193140+0000","last_change":"2023-11-06T03:09:34.769704+0000","last_active":"2023-11-06T03:09:35.193140+0000","last_peered":"2023-11-06T03:09:35.193140+0000","last_clean":"2023-11-06T03:09:35.193140+0000","last_became_active":"2023-11-06T03:09:34.768908+0000","last_became_peered":"2023-11-06T03:09:34.768908+0000","last_unstale":"2023-11-06T03:09:35.193140+0000","last_undegraded":"2023-11-06T03:09:35.193140+0000","last_fullsized":"2023-11-06T03:09:35.193140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781215+0000","last_change":"2023-11-06T03:09:35.158291+0000","last_active":"2023-11-06T03:09:36.781215+0000","last_peered":"2023-11-06T03:09:36.781215+0000","last_clean":"2023-11-06T03:09:36.781215+0000","last_became_active":"2023-11-06T03:09:35.157597+0000","last_became_peered":"2023-11-06T03:09:35.157597+0000","last_unstale":"2023-11-06T03:09:36.781215+0000","last_undegraded":"2023-11-06T03:09:36.781215+0000","last_fullsized":"2023-11-06T03:09:36.781215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:31.328291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.143221+0000","last_change":"2023-11-06T03:09:34.775463+0000","last_active":"2023-11-06T03:09:35.143221+0000","last_peered":"2023-11-06T03:09:35.143221+0000","last_clean":"2023-11-06T03:09:35.143221+0000","last_became_active":"2023-11-06T03:09:34.774827+0000","last_became_peered":"2023-11-06T03:09:34.774827+0000","last_unstale":"2023-11-06T03:09:35.143221+0000","last_undegraded":"2023-11-06T03:09:35.143221+0000","last_fullsized":"2023-11-06T03:09:35.143221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146847+0000","last_change":"2023-11-06T03:09:35.165362+0000","last_active":"2023-11-06T03:09:36.146847+0000","last_peered":"2023-11-06T03:09:36.146847+0000","last_clean":"2023-11-06T03:09:36.146847+0000","last_became_active":"2023-11-06T03:09:35.162797+0000","last_became_peered":"2023-11-06T03:09:35.162797+0000","last_unstale":"2023-11-06T03:09:36.146847+0000","last_undegraded":"2023-11-06T03:09:36.146847+0000","last_fullsized":"2023-11-06T03:09:36.146847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.750167+0000","last_change":"2023-11-06T03:09:33.750167+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.750167+0000","last_undegraded":"2023-11-06T03:09:33.750167+0000","last_fullsized":"2023-11-06T03:09:33.750167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454984+0000","last_change":"2023-11-06T03:09:35.180432+0000","last_active":"2023-11-06T03:09:36.454984+0000","last_peered":"2023-11-06T03:09:36.454984+0000","last_clean":"2023-11-06T03:09:36.454984+0000","last_became_active":"2023-11-06T03:09:35.180255+0000","last_became_peered":"2023-11-06T03:09:35.180255+0000","last_unstale":"2023-11-06T03:09:36.454984+0000","last_undegraded":"2023-11-06T03:09:36.454984+0000","last_fullsized":"2023-11-06T03:09:36.454984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:09:42.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.748000+0000","last_change":"2023-11-06T03:09:33.748000+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.748000+0000","last_undegraded":"2023-11-06T03:09:33.748000+0000","last_fullsized":"2023-11-06T03:09:33.748000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781469+0000","last_change":"2023-11-06T03:09:35.169559+0000","last_active":"2023-11-06T03:09:36.781469+0000","last_peered":"2023-11-06T03:09:36.781469+0000","last_clean":"2023-11-06T03:09:36.781469+0000","last_became_active":"2023-11-06T03:09:35.168839+0000","last_became_peered":"2023-11-06T03:09:35.168839+0000","last_unstale":"2023-11-06T03:09:36.781469+0000","last_undegraded":"2023-11-06T03:09:36.781469+0000","last_fullsized":"2023-11-06T03:09:36.781469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:36:39.328805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.140638+0000","last_change":"2023-11-06T03:09:34.788438+0000","last_active":"2023-11-06T03:09:35.140638+0000","last_peered":"2023-11-06T03:09:35.140638+0000","last_clean":"2023-11-06T03:09:35.140638+0000","last_became_active":"2023-11-06T03:09:34.787795+0000","last_became_peered":"2023-11-06T03:09:34.787795+0000","last_unstale":"2023-11-06T03:09:35.140638+0000","last_undegraded":"2023-11-06T03:09:35.140638+0000","last_fullsized":"2023-11-06T03:09:35.140638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781291+0000","last_change":"2023-11-06T03:09:35.148838+0000","last_active":"2023-11-06T03:09:36.781291+0000","last_peered":"2023-11-06T03:09:36.781291+0000","last_clean":"2023-11-06T03:09:36.781291+0000","last_became_active":"2023-11-06T03:09:35.148089+0000","last_became_peered":"2023-11-06T03:09:35.148089+0000","last_unstale":"2023-11-06T03:09:36.781291+0000","last_undegraded":"2023-11-06T03:09:36.781291+0000","last_fullsized":"2023-11-06T03:09:36.781291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:44:38.071325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134853+0000","last_change":"2023-11-06T03:09:34.778026+0000","last_active":"2023-11-06T03:09:35.134853+0000","last_peered":"2023-11-06T03:09:35.134853+0000","last_clean":"2023-11-06T03:09:35.134853+0000","last_became_active":"2023-11-06T03:09:34.777313+0000","last_became_peered":"2023-11-06T03:09:34.777313+0000","last_unstale":"2023-11-06T03:09:35.134853+0000","last_undegraded":"2023-11-06T03:09:35.134853+0000","last_fullsized":"2023-11-06T03:09:35.134853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142904+0000","last_change":"2023-11-06T03:09:35.156187+0000","last_active":"2023-11-06T03:09:36.142904+0000","last_peered":"2023-11-06T03:09:36.142904+0000","last_clean":"2023-11-06T03:09:36.142904+0000","last_became_active":"2023-11-06T03:09:35.155918+0000","last_became_peered":"2023-11-06T03:09:35.155918+0000","last_unstale":"2023-11-06T03:09:36.142904+0000","last_undegraded":"2023-11-06T03:09:36.142904+0000","last_fullsized":"2023-11-06T03:09:36.142904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:22.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.142771+0000","last_change":"2023-11-06T03:09:34.777643+0000","last_active":"2023-11-06T03:09:35.142771+0000","last_peered":"2023-11-06T03:09:35.142771+0000","last_clean":"2023-11-06T03:09:35.142771+0000","last_became_active":"2023-11-06T03:09:34.776923+0000","last_became_peered":"2023-11-06T03:09:34.776923+0000","last_unstale":"2023-11-06T03:09:35.142771+0000","last_undegraded":"2023-11-06T03:09:35.142771+0000","last_fullsized":"2023-11-06T03:09:35.142771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142988+0000","last_change":"2023-11-06T03:09:35.180342+0000","last_active":"2023-11-06T03:09:36.142988+0000","last_peered":"2023-11-06T03:09:36.142988+0000","last_clean":"2023-11-06T03:09:36.142988+0000","last_became_active":"2023-11-06T03:09:35.180015+0000","last_became_peered":"2023-11-06T03:09:35.180015+0000","last_unstale":"2023-11-06T03:09:36.142988+0000","last_undegraded":"2023-11-06T03:09:36.142988+0000","last_fullsized":"2023-11-06T03:09:36.142988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:19.690734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.144442+0000","last_change":"2023-11-06T03:09:34.770016+0000","last_active":"2023-11-06T03:09:35.144442+0000","last_peered":"2023-11-06T03:09:35.144442+0000","last_clean":"2023-11-06T03:09:35.144442+0000","last_became_active":"2023-11-06T03:09:34.769272+0000","last_became_peered":"2023-11-06T03:09:34.769272+0000","last_unstale":"2023-11-06T03:09:35.144442+0000","last_undegraded":"2023-11-06T03:09:35.144442+0000","last_fullsized":"2023-11-06T03:09:35.144442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:09:34.736732+0000","last_change":"2023-11-06T03:09:34.736732+0000","last_active":"2023-11-06T03:09:34.736732+0000","last_peered":"2023-11-06T03:09:34.736732+0000","last_clean":"2023-11-06T03:09:34.736732+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:34.736732+0000","last_undegraded":"2023-11-06T03:09:34.736732+0000","last_fullsized":"2023-11-06T03:09:34.736732+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.146493+0000","last_change":"2023-11-06T03:09:34.792141+0000","last_active":"2023-11-06T03:09:35.146493+0000","last_peered":"2023-11-06T03:09:35.146493+0000","last_clean":"2023-11-06T03:09:35.146493+0000","last_became_active":"2023-11-06T03:09:34.791850+0000","last_became_peered":"2023-11-06T03:09:34.791850+0000","last_unstale":"2023-11-06T03:09:35.146493+0000","last_undegraded":"2023-11-06T03:09:35.146493+0000","last_fullsized":"2023-11-06T03:09:35.146493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884198+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:09:31.884198+0000","last_peered":"2023-11-06T03:09:31.884198+0000","last_clean":"2023-11-06T03:09:31.884198+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:09:31.884198+0000","last_undegraded":"2023-11-06T03:09:31.884198+0000","last_fullsized":"2023-11-06T03:09:31.884198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160119+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:09:35.160119+0000","last_peered":"2023-11-06T03:09:35.160119+0000","last_clean":"2023-11-06T03:09:35.160119+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:09:35.160119+0000","last_undegraded":"2023-11-06T03:09:35.160119+0000","last_fullsized":"2023-11-06T03:09:35.160119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454739+0000","last_change":"2023-11-06T03:09:35.182913+0000","last_active":"2023-11-06T03:09:36.454739+0000","last_peered":"2023-11-06T03:09:36.454739+0000","last_clean":"2023-11-06T03:09:36.454739+0000","last_became_active":"2023-11-06T03:09:35.182611+0000","last_became_peered":"2023-11-06T03:09:35.182611+0000","last_unstale":"2023-11-06T03:09:36.454739+0000","last_undegraded":"2023-11-06T03:09:36.454739+0000","last_fullsized":"2023-11-06T03:09:36.454739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:53.416028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145783+0000","last_change":"2023-11-06T03:09:34.784096+0000","last_active":"2023-11-06T03:09:35.145783+0000","last_peered":"2023-11-06T03:09:35.145783+0000","last_clean":"2023-11-06T03:09:35.145783+0000","last_became_active":"2023-11-06T03:09:34.783680+0000","last_became_peered":"2023-11-06T03:09:34.783680+0000","last_unstale":"2023-11-06T03:09:35.145783+0000","last_undegraded":"2023-11-06T03:09:35.145783+0000","last_fullsized":"2023-11-06T03:09:35.145783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884217+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:09:31.884217+0000","last_peered":"2023-11-06T03:09:31.884217+0000","last_clean":"2023-11-06T03:09:31.884217+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:09:31.884217+0000","last_undegraded":"2023-11-06T03:09:31.884217+0000","last_fullsized":"2023-11-06T03:09:31.884217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781861+0000","last_change":"2023-11-06T03:09:35.193898+0000","last_active":"2023-11-06T03:09:36.781861+0000","last_peered":"2023-11-06T03:09:36.781861+0000","last_clean":"2023-11-06T03:09:36.781861+0000","last_became_active":"2023-11-06T03:09:35.193553+0000","last_became_peered":"2023-11-06T03:09:35.193553+0000","last_unstale":"2023-11-06T03:09:36.781861+0000","last_undegraded":"2023-11-06T03:09:36.781861+0000","last_fullsized":"2023-11-06T03:09:36.781861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:38.069714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145473+0000","last_change":"2023-11-06T03:09:34.783828+0000","last_active":"2023-11-06T03:09:35.145473+0000","last_peered":"2023-11-06T03:09:35.145473+0000","last_clean":"2023-11-06T03:09:35.145473+0000","last_became_active":"2023-11-06T03:09:34.783326+0000","last_became_peered":"2023-11-06T03:09:34.783326+0000","last_unstale":"2023-11-06T03:09:35.145473+0000","last_undegraded":"2023-11-06T03:09:35.145473+0000","last_fullsized":"2023-11-06T03:09:35.145473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134229+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:09:35.134229+0000","last_peered":"2023-11-06T03:09:35.134229+0000","last_clean":"2023-11-06T03:09:35.134229+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:09:35.134229+0000","last_undegraded":"2023-11-06T03:09:35.134229+0000","last_fullsized":"2023-11-06T03:09:35.134229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454985+0000","last_change":"2023-11-06T03:09:35.195675+0000","last_active":"2023-11-06T03:09:36.454985+0000","last_peered":"2023-11-06T03:09:36.454985+0000","last_clean":"2023-11-06T03:09:36.454985+0000","last_became_active":"2023-11-06T03:09:35.195391+0000","last_became_peered":"2023-11-06T03:09:35.195391+0000","last_unstale":"2023-11-06T03:09:36.454985+0000","last_undegraded":"2023-11-06T03:09:36.454985+0000","last_fullsized":"2023-11-06T03:09:36.454985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:19.216099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.155878+0000","last_change":"2023-11-06T03:09:34.767511+0000","last_active":"2023-11-06T03:09:35.155878+0000","last_peered":"2023-11-06T03:09:35.155878+0000","last_clean":"2023-11-06T03:09:35.155878+0000","last_became_active":"2023-11-06T03:09:34.766834+0000","last_became_peered":"2023-11-06T03:09:34.766834+0000","last_unstale":"2023-11-06T03:09:35.155878+0000","last_undegraded":"2023-11-06T03:09:35.155878+0000","last_fullsized":"2023-11-06T03:09:35.155878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160079+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:09:35.160079+0000","last_peered":"2023-11-06T03:09:35.160079+0000","last_clean":"2023-11-06T03:09:35.160079+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:09:35.160079+0000","last_undegraded":"2023-11-06T03:09:35.160079+0000","last_fullsized":"2023-11-06T03:09:35.160079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735938+0000","last_change":"2023-11-06T03:09:35.151930+0000","last_active":"2023-11-06T03:09:36.735938+0000","last_peered":"2023-11-06T03:09:36.735938+0000","last_clean":"2023-11-06T03:09:36.735938+0000","last_became_active":"2023-11-06T03:09:35.151654+0000","last_became_peered":"2023-11-06T03:09:35.151654+0000","last_unstale":"2023-11-06T03:09:36.735938+0000","last_undegraded":"2023-11-06T03:09:36.735938+0000","last_fullsized":"2023-11-06T03:09:36.735938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:03.266444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.156290+0000","last_change":"2023-11-06T03:09:34.785373+0000","last_active":"2023-11-06T03:09:35.156290+0000","last_peered":"2023-11-06T03:09:35.156290+0000","last_clean":"2023-11-06T03:09:35.156290+0000","last_became_active":"2023-11-06T03:09:34.784939+0000","last_became_peered":"2023-11-06T03:09:34.784939+0000","last_unstale":"2023-11-06T03:09:35.156290+0000","last_undegraded":"2023-11-06T03:09:35.156290+0000","last_fullsized":"2023-11-06T03:09:35.156290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145694+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:09:35.145694+0000","last_peered":"2023-11-06T03:09:35.145694+0000","last_clean":"2023-11-06T03:09:35.145694+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:09:35.145694+0000","last_undegraded":"2023-11-06T03:09:35.145694+0000","last_fullsized":"2023-11-06T03:09:35.145694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235243+0000","last_change":"2023-11-06T03:09:35.156345+0000","last_active":"2023-11-06T03:09:37.235243+0000","last_peered":"2023-11-06T03:09:37.235243+0000","last_clean":"2023-11-06T03:09:37.235243+0000","last_became_active":"2023-11-06T03:09:35.156090+0000","last_became_peered":"2023-11-06T03:09:35.156090+0000","last_unstale":"2023-11-06T03:09:37.235243+0000","last_undegraded":"2023-11-06T03:09:37.235243+0000","last_fullsized":"2023-11-06T03:09:37.235243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:35.830319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145622+0000","last_change":"2023-11-06T03:09:34.774319+0000","last_active":"2023-11-06T03:09:35.145622+0000","last_peered":"2023-11-06T03:09:35.145622+0000","last_clean":"2023-11-06T03:09:35.145622+0000","last_became_active":"2023-11-06T03:09:34.774046+0000","last_became_peered":"2023-11-06T03:09:34.774046+0000","last_unstale":"2023-11-06T03:09:35.145622+0000","last_undegraded":"2023-11-06T03:09:35.145622+0000","last_fullsized":"2023-11-06T03:09:35.145622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-06T03:09:31.884251+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:09:31.884251+0000","last_peered":"2023-11-06T03:09:31.884251+0000","last_clean":"2023-11-06T03:09:31.884251+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:09:31.884251+0000","last_undegraded":"2023-11-06T03:09:31.884251+0000","last_fullsized":"2023-11-06T03:09:31.884251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.234980+0000","last_change":"2023-11-06T03:09:35.155835+0000","last_active":"2023-11-06T03:09:37.234980+0000","last_peered":"2023-11-06T03:09:37.234980+0000","last_clean":"2023-11-06T03:09:37.234980+0000","last_became_active":"2023-11-06T03:09:35.155528+0000","last_became_peered":"2023-11-06T03:09:35.155528+0000","last_unstale":"2023-11-06T03:09:37.234980+0000","last_undegraded":"2023-11-06T03:09:37.234980+0000","last_fullsized":"2023-11-06T03:09:37.234980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:21.984806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"creating+peering","last_fresh":"2023-11-06T03:09:33.747611+0000","last_change":"2023-11-06T03:09:33.747611+0000","last_active":"2023-11-06T03:09:33.733547+0000","last_peered":"2023-11-06T03:09:33.733547+0000","last_clean":"2023-11-06T03:09:33.733547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:09:33.747611+0000","last_undegraded":"2023-11-06T03:09:33.747611+0000","last_fullsized":"2023-11-06T03:09:33.747611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145797+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:09:35.145797+0000","last_peered":"2023-11-06T03:09:35.145797+0000","last_clean":"2023-11-06T03:09:35.145797+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:09:35.145797+0000","last_undegraded":"2023-11-06T03:09:35.145797+0000","last_fullsized":"2023-11-06T03:09:35.145797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735586+0000","last_change":"2023-11-06T03:09:35.168659+0000","last_active":"2023-11-06T03:09:36.735586+0000","last_peered":"2023-11-06T03:09:36.735586+0000","last_clean":"2023-11-06T03:09:36.735586+0000","last_became_active":"2023-11-06T03:09:35.168402+0000","last_became_peered":"2023-11-06T03:09:35.168402+0000","last_unstale":"2023-11-06T03:09:36.735586+0000","last_undegraded":"2023-11-06T03:09:36.735586+0000","last_fullsized":"2023-11-06T03:09:36.735586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:36.377101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.182418+0000","last_change":"2023-11-06T03:09:34.777060+0000","last_active":"2023-11-06T03:09:35.182418+0000","last_peered":"2023-11-06T03:09:35.182418+0000","last_clean":"2023-11-06T03:09:35.182418+0000","last_became_active":"2023-11-06T03:09:34.776357+0000","last_became_peered":"2023-11-06T03:09:34.776357+0000","last_unstale":"2023-11-06T03:09:35.182418+0000","last_undegraded":"2023-11-06T03:09:35.182418+0000","last_fullsized":"2023-11-06T03:09:35.182418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.142959+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:09:35.142959+0000","last_peered":"2023-11-06T03:09:35.142959+0000","last_clean":"2023-11-06T03:09:35.142959+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:09:35.142959+0000","last_undegraded":"2023-11-06T03:09:35.142959+0000","last_fullsized":"2023-11-06T03:09:35.142959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1581056,"data_stored":1036744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.397,"15min":1.397},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.86099999999999999},"min":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":1.444,"15min":1.444},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.86899999999999999,"15min":0.89600000000000002},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.877,"15min":0.877},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.381,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.216},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.879,"15min":0.95299999999999996},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0629999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.127,"15min":1.0669999999999999},"min":{"1min":0.65600000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":10.596,"5min":10.596,"15min":10.596},"last":1.302}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92500000000000004,"15min":0.97299999999999998},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":1.01},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.282,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.1060000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.042,"15min":1.05},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.355},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.095},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3240000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79600000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.51,"5min":1.774,"15min":1.774},"last":1.252},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.099,"15min":1.117},"min":{"1min":0.90800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.1830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46020,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325820,"statfs":{"total":96636764160,"available":96589639680,"internally_reserved":0,"allocated":1044480,"data_stored":473500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":0.96199999999999997,"15min":0.84899999999999998},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":1.466},{"interface":"front","average":{"1min":3.0750000000000002,"5min":1.256,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":40.905000000000001,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.048,"15min":0.89700000000000002},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":16.012,"5min":16.012,"15min":16.012},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.02,"5min":0.89400000000000002,"15min":0.879},"min":{"1min":0.60499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.84899999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":1.1120000000000001,"15min":0.90600000000000003},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":27.803999999999998,"5min":27.803999999999998,"15min":27.803999999999998},"last":1.155},{"interface":"front","average":{"1min":1.004,"5min":0.96999999999999997,"15min":0.95699999999999996},"min":{"1min":0.64000000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.542}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.621,"15min":1.1919999999999999},"min":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":63.737000000000002,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":3.121,"5min":1.365,"15min":1.0449999999999999},"min":{"1min":0.57199999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":41.079999999999998,"5min":41.079999999999998,"15min":41.079999999999998},"last":1.0580000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.282,"15min":1.1839999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":24.867999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.373},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.359,"15min":1.2370000000000001},"min":{"1min":0.76700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":23.789000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.103}]},{"osd":5,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":1.4359999999999999,"15min":1.1779999999999999},"min":{"1min":0.71599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":40.959000000000003,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1479999999999999,"15min":1.1120000000000001},"min":{"1min":0.90400000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.6160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":1.3460000000000001,"15min":1.1899999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.234},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.407,"15min":1.22},"min":{"1min":0.81100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":28.164000000000001,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.2589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":1040384,"data_stored":472989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0580000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81499999999999995,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.145,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2230000000000001,"5min":1.357,"15min":1.357},"last":0.85599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.84199999999999997},"min":{"1min":0.66400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1970000000000001,"5min":1.403,"15min":1.403},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94999999999999996,"15min":0.93600000000000005},"min":{"1min":0.72799999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":1.117}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.91900000000000004},"min":{"1min":0.64900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1060000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2330000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.014,"15min":0.99399999999999999},"min":{"1min":0.63800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.206,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.306,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0269999999999999,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.389,"5min":1.476,"15min":1.476},"last":1.087},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0940000000000001,"15min":1.0840000000000001},"min":{"1min":1.008,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2909999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50128,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":1056768,"data_stored":474623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.82299999999999995},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1619999999999999,"5min":1.484,"15min":1.484},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2989999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.755},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1930000000000001,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85499999999999998,"15min":0.81200000000000006},"min":{"1min":0.55300000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.91600000000000004,"15min":1.04},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1539999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1919999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.71899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.069,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.365,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5489999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.1519999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.411,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.94899999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.073},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3009999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.2430000000000001,"15min":1.1870000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6739999999999999,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.0960000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50108,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321732,"statfs":{"total":96636764160,"available":96585453568,"internally_reserved":0,"allocated":1036288,"data_stored":472518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.248,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.26700000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.90400000000000003,"5min":1.524,"15min":1.524},"last":0.622}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1080000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.042,"5min":1.349,"15min":1.349},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.0349999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.518,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94599999999999995,"15min":0.98399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.536,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.66400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5049999999999999,"5min":1.698,"15min":1.698},"last":0.99199999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97599999999999998,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4610000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94899999999999995,"15min":1},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.391,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.008,"15min":1.056},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3180000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.024,"15min":1.0649999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4930000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0029999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":1048576,"data_stored":473697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.24299999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.374},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.255,"5min":0.223,"15min":0.223},"max":{"1min":1.0109999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.62,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.121},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.629,"15min":0.66900000000000004},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.1259999999999999,"5min":1.294,"15min":1.294},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.89000000000000001},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":0.95499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.88400000000000001},"min":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.214,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.615,"15min":1.615},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.95199999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.006,"15min":1},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.93100000000000005,"15min":0.96199999999999997},"min":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.256,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.84999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":473067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56299999999999994,"15min":0.59199999999999997},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":1.0629999999999999,"5min":1.157,"15min":1.157},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.26900000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0649999999999999,"5min":1.544,"15min":1.544},"last":0.371}]},{"osd":2,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.90700000000000003,"15min":0.71999999999999997},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63900000000000001,"15min":0.68300000000000005},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.03,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1200000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.72599999999999998},"min":{"1min":0.35199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90600000000000003,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.2809999999999999,"5min":2.258,"15min":2.258},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":14.753,"15min":14.753},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.3919999999999999,"15min":1.1399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.367,"15min":1.167},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.272,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.755}]},{"osd":6,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.177,"15min":1.0760000000000001},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":23.971,"15min":23.971},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2949999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0149999999999999,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3560000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58876,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312964,"statfs":{"total":96636764160,"available":96576475136,"internally_reserved":0,"allocated":1626112,"data_stored":1062304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.28299999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.145,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.69899999999999995,"15min":0.65100000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1120000000000001,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83199999999999996,"5min":1.155,"15min":1.155},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.68500000000000005,"15min":0.65400000000000003},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.876,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.67000000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.29899999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0700000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0589999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.95499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.96299999999999997,"15min":0.92700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1299999999999999,"5min":4.641,"15min":4.641},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.88500000000000001,"15min":0.93100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1,"5min":2.028,"15min":2.028},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1140000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.94699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2050000000000001,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.117}]},{"osd":6,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":2.931,"15min":2.931},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.248,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.091}]},{"osd":7,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.0289999999999999,"15min":1.0089999999999999},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2649999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.98699999999999999,"15min":0.99199999999999999},"min":{"1min":0.64800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.97,"15min":1.97},"last":1.155}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:09:40.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:09:40.583 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:09:42.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:09:42.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.764+0000 7f3de7ae8640 1 Processor -- start 2023-11-06T03:09:42.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.764+0000 7f3de7ae8640 1 -- start start 2023-11-06T03:09:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de7ae8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de7ae8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3de015e5e0 0x7f3de00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de7ae8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de015f580 con 0x7f3de015ea60 2023-11-06T03:09:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de7ae8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de015efc0 con 0x7f3de015e5e0 2023-11-06T03:09:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de7ae8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015f2a0 con 0x7f3de01551e0 2023-11-06T03:09:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3de015e5e0 0x7f3de00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3de015e5e0 0x7f3de00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46878/0 (socket says 172.21.15.157:46878) 2023-11-06T03:09:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 -- 172.21.15.157:0/3635197728 learned_addr learned my addr 172.21.15.157:0/3635197728 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5884640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 -- 172.21.15.157:0/3635197728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 msgr2=0x7f3de006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 -- 172.21.15.157:0/3635197728 >> v1:172.21.15.195:6789/0 conn(0x7f3de015ea60 legacy=0x7f3de015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 -- 172.21.15.157:0/3635197728 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de006f5a0 con 0x7f3de015e5e0 2023-11-06T03:09:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5884640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de5083640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de00b3410 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d970060d3026f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de4882640 1 -- 172.21.15.157:0/3635197728 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00ac640 con 0x7f3de015e5e0 2023-11-06T03:09:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de4882640 1 -- 172.21.15.157:0/3635197728 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd00ac7a0 con 0x7f3de015e5e0 2023-11-06T03:09:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.768+0000 7f3de4882640 1 -- 172.21.15.157:0/3635197728 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00acb30 con 0x7f3de015e5e0 2023-11-06T03:09:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/3635197728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 msgr2=0x7f3de00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de00b3410 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd001c440 comp rx=0 tx=0).stop 2023-11-06T03:09:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/3635197728 shutdown_connections 2023-11-06T03:09:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de00b3410 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/3635197728 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/3635197728 >> 172.21.15.157:0/3635197728 conn(0x7f3de0065930 msgr2=0x7f3de0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/3635197728 shutdown_connections 2023-11-06T03:09:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/3635197728 wait complete. 2023-11-06T03:09:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 Processor -- start 2023-11-06T03:09:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- start start 2023-11-06T03:09:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de014c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 0x7f3de014ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01534a0 con 0x7f3de015e5e0 2023-11-06T03:09:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01531c0 con 0x7f3de015ea60 2023-11-06T03:09:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00ce610 con 0x7f3de01551e0 2023-11-06T03:09:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46894/0 (socket says 172.21.15.157:46894) 2023-11-06T03:09:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 -- 172.21.15.157:0/898523505 learned_addr learned my addr 172.21.15.157:0/898523505 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de6085640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 0x7f3de014ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5884640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de014c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 -- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 msgr2=0x7f3de014c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de014c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 -- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 msgr2=0x7f3de014ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 0x7f3de014ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 -- 172.21.15.157:0/898523505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00094d0 con 0x7f3de015e5e0 2023-11-06T03:09:42.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de6085640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 0x7f3de014ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:42.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5884640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:42.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de5083640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00af040 tx=0x7f3dd00191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:42.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00ac640 con 0x7f3de015e5e0 2023-11-06T03:09:42.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd00b5870 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de00ccac0 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00b5c20 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.772+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de00ce8f0 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de00ce750 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3dd00b3070 con 0x7f3de015e5e0 2023-11-06T03:09:42.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3dd67fc640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 0x7f3dc4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:42.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3dd011d560 con 0x7f3de015e5e0 2023-11-06T03:09:42.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3de5884640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 0x7f3dc4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:42.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.776+0000 7f3de5884640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 0x7f3dc4067f20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3de0152770 tx=0x7f3ddc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:42.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.780+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3de00ce750 con 0x7f3de015e5e0 2023-11-06T03:09:42.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.924+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3de00b3d60 con 0x7f3dc4065a60 2023-11-06T03:09:42.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.952+0000 7f3dd67fc640 1 -- 172.21.15.157:0/898523505 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310507 (secure 0 0 0) 0x7f3ddc00c040 con 0x7f3dc4065a60 2023-11-06T03:09:42.950 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:42.954 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:09:42.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.952+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 msgr2=0x7f3dc4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:42.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.952+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 0x7f3dc4067f20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3de0152770 tx=0x7f3ddc008040 comp rx=0 tx=0).stop 2023-11-06T03:09:42.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.952+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 msgr2=0x7f3de01f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:42.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.952+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00af040 tx=0x7f3dd00191e0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 shutdown_connections 2023-11-06T03:09:42.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3dc4065a60 0x7f3dc4067f20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3de015ea60 0x7f3de014ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3de015e5e0 0x7f3de01f5f30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 --2- 172.21.15.157:0/898523505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3de01551e0 0x7f3de014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:42.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 >> 172.21.15.157:0/898523505 conn(0x7f3de0065930 msgr2=0x7f3de01536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:42.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 shutdown_connections 2023-11-06T03:09:42.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:42.956+0000 7f3de7ae8640 1 -- 172.21.15.157:0/898523505 wait complete. 2023-11-06T03:09:42.970 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-11-06T03:09:41.215461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361884,"kb_used_data":9308,"kb_used_omap":0,"kb_used_meta":352512,"kb_avail":754612836,"statfs":{"total":773094113280,"available":772723544064,"internally_reserved":0,"allocated":9531392,"data_stored":4966043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":360972288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.470456"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.083139+0000","last_change":"2023-11-06T03:09:35.183982+0000","last_active":"2023-11-06T03:09:37.083139+0000","last_peered":"2023-11-06T03:09:37.083139+0000","last_clean":"2023-11-06T03:09:37.083139+0000","last_became_active":"2023-11-06T03:09:35.183373+0000","last_became_peered":"2023-11-06T03:09:35.183373+0000","last_unstale":"2023-11-06T03:09:37.083139+0000","last_undegraded":"2023-11-06T03:09:37.083139+0000","last_fullsized":"2023-11-06T03:09:37.083139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:42:30.316559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454645+0000","last_change":"2023-11-06T03:09:35.151600+0000","last_active":"2023-11-06T03:09:36.454645+0000","last_peered":"2023-11-06T03:09:36.454645+0000","last_clean":"2023-11-06T03:09:36.454645+0000","last_became_active":"2023-11-06T03:09:35.151343+0000","last_became_peered":"2023-11-06T03:09:35.151343+0000","last_unstale":"2023-11-06T03:09:36.454645+0000","last_undegraded":"2023-11-06T03:09:36.454645+0000","last_fullsized":"2023-11-06T03:09:36.454645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:13:25.279812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142950+0000","last_change":"2023-11-06T03:09:35.152944+0000","last_active":"2023-11-06T03:09:36.142950+0000","last_peered":"2023-11-06T03:09:36.142950+0000","last_clean":"2023-11-06T03:09:36.142950+0000","last_became_active":"2023-11-06T03:09:35.152688+0000","last_became_peered":"2023-11-06T03:09:35.152688+0000","last_unstale":"2023-11-06T03:09:36.142950+0000","last_undegraded":"2023-11-06T03:09:36.142950+0000","last_fullsized":"2023-11-06T03:09:36.142950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237504+0000","last_change":"2023-11-06T03:09:35.196142+0000","last_active":"2023-11-06T03:09:37.237504+0000","last_peered":"2023-11-06T03:09:37.237504+0000","last_clean":"2023-11-06T03:09:37.237504+0000","last_became_active":"2023-11-06T03:09:35.194760+0000","last_became_peered":"2023-11-06T03:09:35.194760+0000","last_unstale":"2023-11-06T03:09:37.237504+0000","last_undegraded":"2023-11-06T03:09:37.237504+0000","last_fullsized":"2023-11-06T03:09:37.237504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736012+0000","last_change":"2023-11-06T03:09:35.179907+0000","last_active":"2023-11-06T03:09:36.736012+0000","last_peered":"2023-11-06T03:09:36.736012+0000","last_clean":"2023-11-06T03:09:36.736012+0000","last_became_active":"2023-11-06T03:09:35.179643+0000","last_became_peered":"2023-11-06T03:09:35.179643+0000","last_unstale":"2023-11-06T03:09:36.736012+0000","last_undegraded":"2023-11-06T03:09:36.736012+0000","last_fullsized":"2023-11-06T03:09:36.736012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:30:19.415560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235345+0000","last_change":"2023-11-06T03:09:35.172687+0000","last_active":"2023-11-06T03:09:37.235345+0000","last_peered":"2023-11-06T03:09:37.235345+0000","last_clean":"2023-11-06T03:09:37.235345+0000","last_became_active":"2023-11-06T03:09:35.172474+0000","last_became_peered":"2023-11-06T03:09:35.172474+0000","last_unstale":"2023-11-06T03:09:37.235345+0000","last_undegraded":"2023-11-06T03:09:37.235345+0000","last_fullsized":"2023-11-06T03:09:37.235345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:27.705396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781026+0000","last_change":"2023-11-06T03:09:35.148110+0000","last_active":"2023-11-06T03:09:36.781026+0000","last_peered":"2023-11-06T03:09:36.781026+0000","last_clean":"2023-11-06T03:09:36.781026+0000","last_became_active":"2023-11-06T03:09:35.147260+0000","last_became_peered":"2023-11-06T03:09:35.147260+0000","last_unstale":"2023-11-06T03:09:36.781026+0000","last_undegraded":"2023-11-06T03:09:36.781026+0000","last_fullsized":"2023-11-06T03:09:36.781026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:02:55.853075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735559+0000","last_change":"2023-11-06T03:09:35.168034+0000","last_active":"2023-11-06T03:09:36.735559+0000","last_peered":"2023-11-06T03:09:36.735559+0000","last_clean":"2023-11-06T03:09:36.735559+0000","last_became_active":"2023-11-06T03:09:35.167812+0000","last_became_peered":"2023-11-06T03:09:35.167812+0000","last_unstale":"2023-11-06T03:09:36.735559+0000","last_undegraded":"2023-11-06T03:09:36.735559+0000","last_fullsized":"2023-11-06T03:09:36.735559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:53:45.100372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.147397+0000","last_change":"2023-11-06T03:09:35.172426+0000","last_active":"2023-11-06T03:09:36.147397+0000","last_peered":"2023-11-06T03:09:36.147397+0000","last_clean":"2023-11-06T03:09:36.147397+0000","last_became_active":"2023-11-06T03:09:35.171647+0000","last_became_peered":"2023-11-06T03:09:35.171647+0000","last_unstale":"2023-11-06T03:09:36.147397+0000","last_undegraded":"2023-11-06T03:09:36.147397+0000","last_fullsized":"2023-11-06T03:09:36.147397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:19.690734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237665+0000","last_change":"2023-11-06T03:09:35.193811+0000","last_active":"2023-11-06T03:09:37.237665+0000","last_peered":"2023-11-06T03:09:37.237665+0000","last_clean":"2023-11-06T03:09:37.237665+0000","last_became_active":"2023-11-06T03:09:35.192825+0000","last_became_peered":"2023-11-06T03:09:35.192825+0000","last_unstale":"2023-11-06T03:09:37.237665+0000","last_undegraded":"2023-11-06T03:09:37.237665+0000","last_fullsized":"2023-11-06T03:09:37.237665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:12.925128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.455253+0000","last_change":"2023-11-06T03:09:35.152556+0000","last_active":"2023-11-06T03:09:36.455253+0000","last_peered":"2023-11-06T03:09:36.455253+0000","last_clean":"2023-11-06T03:09:36.455253+0000","last_became_active":"2023-11-06T03:09:35.152136+0000","last_became_peered":"2023-11-06T03:09:35.152136+0000","last_unstale":"2023-11-06T03:09:36.455253+0000","last_undegraded":"2023-11-06T03:09:36.455253+0000","last_fullsized":"2023-11-06T03:09:36.455253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:28.829494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237604+0000","last_change":"2023-11-06T03:09:35.195653+0000","last_active":"2023-11-06T03:09:37.237604+0000","last_peered":"2023-11-06T03:09:37.237604+0000","last_clean":"2023-11-06T03:09:37.237604+0000","last_became_active":"2023-11-06T03:09:35.193804+0000","last_became_peered":"2023-11-06T03:09:35.193804+0000","last_unstale":"2023-11-06T03:09:37.237604+0000","last_undegraded":"2023-11-06T03:09:37.237604+0000","last_fullsized":"2023-11-06T03:09:37.237604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736041+0000","last_change":"2023-11-06T03:09:35.157240+0000","last_active":"2023-11-06T03:09:36.736041+0000","last_peered":"2023-11-06T03:09:36.736041+0000","last_clean":"2023-11-06T03:09:36.736041+0000","last_became_active":"2023-11-06T03:09:35.156975+0000","last_became_peered":"2023-11-06T03:09:35.156975+0000","last_unstale":"2023-11-06T03:09:36.736041+0000","last_undegraded":"2023-11-06T03:09:36.736041+0000","last_fullsized":"2023-11-06T03:09:36.736041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:59.270002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235210+0000","last_change":"2023-11-06T03:09:35.172640+0000","last_active":"2023-11-06T03:09:37.235210+0000","last_peered":"2023-11-06T03:09:37.235210+0000","last_clean":"2023-11-06T03:09:37.235210+0000","last_became_active":"2023-11-06T03:09:35.172364+0000","last_became_peered":"2023-11-06T03:09:35.172364+0000","last_unstale":"2023-11-06T03:09:37.235210+0000","last_undegraded":"2023-11-06T03:09:37.235210+0000","last_fullsized":"2023-11-06T03:09:37.235210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:52:24.016692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454236+0000","last_change":"2023-11-06T03:09:35.151071+0000","last_active":"2023-11-06T03:09:36.454236+0000","last_peered":"2023-11-06T03:09:36.454236+0000","last_clean":"2023-11-06T03:09:36.454236+0000","last_became_active":"2023-11-06T03:09:35.150807+0000","last_became_peered":"2023-11-06T03:09:35.150807+0000","last_unstale":"2023-11-06T03:09:36.454236+0000","last_undegraded":"2023-11-06T03:09:36.454236+0000","last_fullsized":"2023-11-06T03:09:36.454236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:01:07.698040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.082378+0000","last_change":"2023-11-06T03:09:35.194786+0000","last_active":"2023-11-06T03:09:37.082378+0000","last_peered":"2023-11-06T03:09:37.082378+0000","last_clean":"2023-11-06T03:09:37.082378+0000","last_became_active":"2023-11-06T03:09:35.194382+0000","last_became_peered":"2023-11-06T03:09:35.194382+0000","last_unstale":"2023-11-06T03:09:37.082378+0000","last_undegraded":"2023-11-06T03:09:37.082378+0000","last_fullsized":"2023-11-06T03:09:37.082378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:32.769628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736430+0000","last_change":"2023-11-06T03:09:35.171453+0000","last_active":"2023-11-06T03:09:36.736430+0000","last_peered":"2023-11-06T03:09:36.736430+0000","last_clean":"2023-11-06T03:09:36.736430+0000","last_became_active":"2023-11-06T03:09:35.171190+0000","last_became_peered":"2023-11-06T03:09:35.171190+0000","last_unstale":"2023-11-06T03:09:36.736430+0000","last_undegraded":"2023-11-06T03:09:36.736430+0000","last_fullsized":"2023-11-06T03:09:36.736430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:24.270703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142805+0000","last_change":"2023-11-06T03:09:35.172514+0000","last_active":"2023-11-06T03:09:36.142805+0000","last_peered":"2023-11-06T03:09:36.142805+0000","last_clean":"2023-11-06T03:09:36.142805+0000","last_became_active":"2023-11-06T03:09:35.172222+0000","last_became_peered":"2023-11-06T03:09:35.172222+0000","last_unstale":"2023-11-06T03:09:36.142805+0000","last_undegraded":"2023-11-06T03:09:36.142805+0000","last_fullsized":"2023-11-06T03:09:36.142805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235439+0000","last_change":"2023-11-06T03:09:35.155783+0000","last_active":"2023-11-06T03:09:37.235439+0000","last_peered":"2023-11-06T03:09:37.235439+0000","last_clean":"2023-11-06T03:09:37.235439+0000","last_became_active":"2023-11-06T03:09:35.155501+0000","last_became_peered":"2023-11-06T03:09:35.155501+0000","last_unstale":"2023-11-06T03:09:37.235439+0000","last_undegraded":"2023-11-06T03:09:37.235439+0000","last_fullsized":"2023-11-06T03:09:37.235439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:35:23.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736107+0000","last_change":"2023-11-06T03:09:35.142864+0000","last_active":"2023-11-06T03:09:36.736107+0000","last_peered":"2023-11-06T03:09:36.736107+0000","last_clean":"2023-11-06T03:09:36.736107+0000","last_became_active":"2023-11-06T03:09:35.142500+0000","last_became_peered":"2023-11-06T03:09:35.142500+0000","last_unstale":"2023-11-06T03:09:36.736107+0000","last_undegraded":"2023-11-06T03:09:36.736107+0000","last_fullsized":"2023-11-06T03:09:36.736107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:19:54.903256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146609+0000","last_change":"2023-11-06T03:09:35.173385+0000","last_active":"2023-11-06T03:09:36.146609+0000","last_peered":"2023-11-06T03:09:36.146609+0000","last_clean":"2023-11-06T03:09:36.146609+0000","last_became_active":"2023-11-06T03:09:35.172525+0000","last_became_peered":"2023-11-06T03:09:35.172525+0000","last_unstale":"2023-11-06T03:09:36.146609+0000","last_undegraded":"2023-11-06T03:09:36.146609+0000","last_fullsized":"2023-11-06T03:09:36.146609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:39.287082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735770+0000","last_change":"2023-11-06T03:09:35.157771+0000","last_active":"2023-11-06T03:09:36.735770+0000","last_peered":"2023-11-06T03:09:36.735770+0000","last_clean":"2023-11-06T03:09:36.735770+0000","last_became_active":"2023-11-06T03:09:35.157525+0000","last_became_peered":"2023-11-06T03:09:35.157525+0000","last_unstale":"2023-11-06T03:09:36.735770+0000","last_undegraded":"2023-11-06T03:09:36.735770+0000","last_fullsized":"2023-11-06T03:09:36.735770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:27.906246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146325+0000","last_change":"2023-11-06T03:09:35.196860+0000","last_active":"2023-11-06T03:09:36.146325+0000","last_peered":"2023-11-06T03:09:36.146325+0000","last_clean":"2023-11-06T03:09:36.146325+0000","last_became_active":"2023-11-06T03:09:35.196417+0000","last_became_peered":"2023-11-06T03:09:35.196417+0000","last_unstale":"2023-11-06T03:09:36.146325+0000","last_undegraded":"2023-11-06T03:09:36.146325+0000","last_fullsized":"2023-11-06T03:09:36.146325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:13.064955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454387+0000","last_change":"2023-11-06T03:09:35.157458+0000","last_active":"2023-11-06T03:09:36.454387+0000","last_peered":"2023-11-06T03:09:36.454387+0000","last_clean":"2023-11-06T03:09:36.454387+0000","last_became_active":"2023-11-06T03:09:35.157209+0000","last_became_peered":"2023-11-06T03:09:35.157209+0000","last_unstale":"2023-11-06T03:09:36.454387+0000","last_undegraded":"2023-11-06T03:09:36.454387+0000","last_fullsized":"2023-11-06T03:09:36.454387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:02:14.560666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454690+0000","last_change":"2023-11-06T03:09:35.156857+0000","last_active":"2023-11-06T03:09:36.454690+0000","last_peered":"2023-11-06T03:09:36.454690+0000","last_clean":"2023-11-06T03:09:36.454690+0000","last_became_active":"2023-11-06T03:09:35.156610+0000","last_became_peered":"2023-11-06T03:09:35.156610+0000","last_unstale":"2023-11-06T03:09:36.454690+0000","last_undegraded":"2023-11-06T03:09:36.454690+0000","last_fullsized":"2023-11-06T03:09:36.454690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:56:16.942337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237752+0000","last_change":"2023-11-06T03:09:35.193193+0000","last_active":"2023-11-06T03:09:37.237752+0000","last_peered":"2023-11-06T03:09:37.237752+0000","last_clean":"2023-11-06T03:09:37.237752+0000","last_became_active":"2023-11-06T03:09:35.192840+0000","last_became_peered":"2023-11-06T03:09:35.192840+0000","last_unstale":"2023-11-06T03:09:37.237752+0000","last_undegraded":"2023-11-06T03:09:37.237752+0000","last_fullsized":"2023-11-06T03:09:37.237752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.082797+0000","last_change":"2023-11-06T03:09:35.196683+0000","last_active":"2023-11-06T03:09:37.082797+0000","last_peered":"2023-11-06T03:09:37.082797+0000","last_clean":"2023-11-06T03:09:37.082797+0000","last_became_active":"2023-11-06T03:09:35.195944+0000","last_became_peered":"2023-11-06T03:09:35.195944+0000","last_unstale":"2023-11-06T03:09:37.082797+0000","last_undegraded":"2023-11-06T03:09:37.082797+0000","last_fullsized":"2023-11-06T03:09:37.082797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:39:37.506270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.736266+0000","last_change":"2023-11-06T03:09:35.194699+0000","last_active":"2023-11-06T03:09:36.736266+0000","last_peered":"2023-11-06T03:09:36.736266+0000","last_clean":"2023-11-06T03:09:36.736266+0000","last_became_active":"2023-11-06T03:09:35.194405+0000","last_became_peered":"2023-11-06T03:09:35.194405+0000","last_unstale":"2023-11-06T03:09:36.736266+0000","last_undegraded":"2023-11-06T03:09:36.736266+0000","last_fullsized":"2023-11-06T03:09:36.736266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:59.280475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454749+0000","last_change":"2023-11-06T03:09:35.173392+0000","last_active":"2023-11-06T03:09:36.454749+0000","last_peered":"2023-11-06T03:09:36.454749+0000","last_clean":"2023-11-06T03:09:36.454749+0000","last_became_active":"2023-11-06T03:09:35.173081+0000","last_became_peered":"2023-11-06T03:09:35.173081+0000","last_unstale":"2023-11-06T03:09:36.454749+0000","last_undegraded":"2023-11-06T03:09:36.454749+0000","last_fullsized":"2023-11-06T03:09:36.454749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:51:24.553362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.082970+0000","last_change":"2023-11-06T03:09:35.164241+0000","last_active":"2023-11-06T03:09:37.082970+0000","last_peered":"2023-11-06T03:09:37.082970+0000","last_clean":"2023-11-06T03:09:37.082970+0000","last_became_active":"2023-11-06T03:09:35.163272+0000","last_became_peered":"2023-11-06T03:09:35.163272+0000","last_unstale":"2023-11-06T03:09:37.082970+0000","last_undegraded":"2023-11-06T03:09:37.082970+0000","last_fullsized":"2023-11-06T03:09:37.082970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:20.850182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142557+0000","last_change":"2023-11-06T03:09:35.173375+0000","last_active":"2023-11-06T03:09:36.142557+0000","last_peered":"2023-11-06T03:09:36.142557+0000","last_clean":"2023-11-06T03:09:36.142557+0000","last_became_active":"2023-11-06T03:09:35.173093+0000","last_became_peered":"2023-11-06T03:09:35.173093+0000","last_unstale":"2023-11-06T03:09:36.142557+0000","last_undegraded":"2023-11-06T03:09:36.142557+0000","last_fullsized":"2023-11-06T03:09:36.142557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:36.739383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.082613+0000","last_change":"2023-11-06T03:09:35.184626+0000","last_active":"2023-11-06T03:09:37.082613+0000","last_peered":"2023-11-06T03:09:37.082613+0000","last_clean":"2023-11-06T03:09:37.082613+0000","last_became_active":"2023-11-06T03:09:35.183691+0000","last_became_peered":"2023-11-06T03:09:35.183691+0000","last_unstale":"2023-11-06T03:09:37.082613+0000","last_undegraded":"2023-11-06T03:09:37.082613+0000","last_fullsized":"2023-11-06T03:09:37.082613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:45:34.513604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142582+0000","last_change":"2023-11-06T03:09:35.174487+0000","last_active":"2023-11-06T03:09:36.142582+0000","last_peered":"2023-11-06T03:09:36.142582+0000","last_clean":"2023-11-06T03:09:36.142582+0000","last_became_active":"2023-11-06T03:09:35.174279+0000","last_became_peered":"2023-11-06T03:09:35.174279+0000","last_unstale":"2023-11-06T03:09:36.142582+0000","last_undegraded":"2023-11-06T03:09:36.142582+0000","last_fullsized":"2023-11-06T03:09:36.142582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:35.444322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"84'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.156865+0000","last_change":"2023-11-06T03:09:34.777260+0000","last_active":"2023-11-06T03:09:35.156865+0000","last_peered":"2023-11-06T03:09:35.156865+0000","last_clean":"2023-11-06T03:09:35.156865+0000","last_became_active":"2023-11-06T03:09:34.776979+0000","last_became_peered":"2023-11-06T03:09:34.776979+0000","last_unstale":"2023-11-06T03:09:35.156865+0000","last_undegraded":"2023-11-06T03:09:35.156865+0000","last_fullsized":"2023-11-06T03:09:35.156865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:40.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454397+0000","last_change":"2023-11-06T03:09:35.192364+0000","last_active":"2023-11-06T03:09:36.454397+0000","last_peered":"2023-11-06T03:09:36.454397+0000","last_clean":"2023-11-06T03:09:36.454397+0000","last_became_active":"2023-11-06T03:09:35.192099+0000","last_became_peered":"2023-11-06T03:09:35.192099+0000","last_unstale":"2023-11-06T03:09:36.454397+0000","last_undegraded":"2023-11-06T03:09:36.454397+0000","last_fullsized":"2023-11-06T03:09:36.454397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:33:45.886436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.186467+0000","last_change":"2023-11-06T03:09:34.792775+0000","last_active":"2023-11-06T03:09:35.186467+0000","last_peered":"2023-11-06T03:09:35.186467+0000","last_clean":"2023-11-06T03:09:35.186467+0000","last_became_active":"2023-11-06T03:09:34.791921+0000","last_became_peered":"2023-11-06T03:09:34.791921+0000","last_unstale":"2023-11-06T03:09:35.186467+0000","last_undegraded":"2023-11-06T03:09:35.186467+0000","last_fullsized":"2023-11-06T03:09:35.186467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:12.061770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146571+0000","last_change":"2023-11-06T03:09:35.172915+0000","last_active":"2023-11-06T03:09:36.146571+0000","last_peered":"2023-11-06T03:09:36.146571+0000","last_clean":"2023-11-06T03:09:36.146571+0000","last_became_active":"2023-11-06T03:09:35.171804+0000","last_became_peered":"2023-11-06T03:09:35.171804+0000","last_unstale":"2023-11-06T03:09:36.146571+0000","last_undegraded":"2023-11-06T03:09:36.146571+0000","last_fullsized":"2023-11-06T03:09:36.146571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:35.007844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.180114+0000","last_change":"2023-11-06T03:09:34.777315+0000","last_active":"2023-11-06T03:09:35.180114+0000","last_peered":"2023-11-06T03:09:35.180114+0000","last_clean":"2023-11-06T03:09:35.180114+0000","last_became_active":"2023-11-06T03:09:34.776528+0000","last_became_peered":"2023-11-06T03:09:34.776528+0000","last_unstale":"2023-11-06T03:09:35.180114+0000","last_undegraded":"2023-11-06T03:09:35.180114+0000","last_fullsized":"2023-11-06T03:09:35.180114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:18.687549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146516+0000","last_change":"2023-11-06T03:09:35.162931+0000","last_active":"2023-11-06T03:09:36.146516+0000","last_peered":"2023-11-06T03:09:36.146516+0000","last_clean":"2023-11-06T03:09:36.146516+0000","last_became_active":"2023-11-06T03:09:35.162176+0000","last_became_peered":"2023-11-06T03:09:35.162176+0000","last_unstale":"2023-11-06T03:09:36.146516+0000","last_undegraded":"2023-11-06T03:09:36.146516+0000","last_fullsized":"2023-11-06T03:09:36.146516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:22.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.191024+0000","last_change":"2023-11-06T03:09:34.791693+0000","last_active":"2023-11-06T03:09:35.191024+0000","last_peered":"2023-11-06T03:09:35.191024+0000","last_clean":"2023-11-06T03:09:35.191024+0000","last_became_active":"2023-11-06T03:09:34.790912+0000","last_became_peered":"2023-11-06T03:09:34.790912+0000","last_unstale":"2023-11-06T03:09:35.191024+0000","last_undegraded":"2023-11-06T03:09:35.191024+0000","last_fullsized":"2023-11-06T03:09:35.191024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:54:57.390838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454862+0000","last_change":"2023-11-06T03:09:35.156192+0000","last_active":"2023-11-06T03:09:36.454862+0000","last_peered":"2023-11-06T03:09:36.454862+0000","last_clean":"2023-11-06T03:09:36.454862+0000","last_became_active":"2023-11-06T03:09:35.155956+0000","last_became_peered":"2023-11-06T03:09:35.155956+0000","last_unstale":"2023-11-06T03:09:36.454862+0000","last_undegraded":"2023-11-06T03:09:36.454862+0000","last_fullsized":"2023-11-06T03:09:36.454862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.407814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.180761+0000","last_change":"2023-11-06T03:09:34.773862+0000","last_active":"2023-11-06T03:09:35.180761+0000","last_peered":"2023-11-06T03:09:35.180761+0000","last_clean":"2023-11-06T03:09:35.180761+0000","last_became_active":"2023-11-06T03:09:34.773071+0000","last_became_peered":"2023-11-06T03:09:34.773071+0000","last_unstale":"2023-11-06T03:09:35.180761+0000","last_undegraded":"2023-11-06T03:09:35.180761+0000","last_fullsized":"2023-11-06T03:09:35.180761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:50.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.455215+0000","last_change":"2023-11-06T03:09:35.152376+0000","last_active":"2023-11-06T03:09:36.455215+0000","last_peered":"2023-11-06T03:09:36.455215+0000","last_clean":"2023-11-06T03:09:36.455215+0000","last_became_active":"2023-11-06T03:09:35.151873+0000","last_became_peered":"2023-11-06T03:09:35.151873+0000","last_unstale":"2023-11-06T03:09:36.455215+0000","last_undegraded":"2023-11-06T03:09:36.455215+0000","last_fullsized":"2023-11-06T03:09:36.455215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:18.193688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.192692+0000","last_change":"2023-11-06T03:09:34.779144+0000","last_active":"2023-11-06T03:09:35.192692+0000","last_peered":"2023-11-06T03:09:35.192692+0000","last_clean":"2023-11-06T03:09:35.192692+0000","last_became_active":"2023-11-06T03:09:34.778339+0000","last_became_peered":"2023-11-06T03:09:34.778339+0000","last_unstale":"2023-11-06T03:09:35.192692+0000","last_undegraded":"2023-11-06T03:09:35.192692+0000","last_fullsized":"2023-11-06T03:09:35.192692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:38:17.910893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.147065+0000","last_change":"2023-11-06T03:09:35.173260+0000","last_active":"2023-11-06T03:09:36.147065+0000","last_peered":"2023-11-06T03:09:36.147065+0000","last_clean":"2023-11-06T03:09:36.147065+0000","last_became_active":"2023-11-06T03:09:35.171398+0000","last_became_peered":"2023-11-06T03:09:35.171398+0000","last_unstale":"2023-11-06T03:09:36.147065+0000","last_undegraded":"2023-11-06T03:09:36.147065+0000","last_fullsized":"2023-11-06T03:09:36.147065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:37.347892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.148321+0000","last_change":"2023-11-06T03:09:34.780135+0000","last_active":"2023-11-06T03:09:35.148321+0000","last_peered":"2023-11-06T03:09:35.148321+0000","last_clean":"2023-11-06T03:09:35.148321+0000","last_became_active":"2023-11-06T03:09:34.779866+0000","last_became_peered":"2023-11-06T03:09:34.779866+0000","last_unstale":"2023-11-06T03:09:35.148321+0000","last_undegraded":"2023-11-06T03:09:35.148321+0000","last_fullsized":"2023-11-06T03:09:35.148321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:02.188870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781689+0000","last_change":"2023-11-06T03:09:35.171039+0000","last_active":"2023-11-06T03:09:36.781689+0000","last_peered":"2023-11-06T03:09:36.781689+0000","last_clean":"2023-11-06T03:09:36.781689+0000","last_became_active":"2023-11-06T03:09:35.169738+0000","last_became_peered":"2023-11-06T03:09:35.169738+0000","last_unstale":"2023-11-06T03:09:36.781689+0000","last_undegraded":"2023-11-06T03:09:36.781689+0000","last_fullsized":"2023-11-06T03:09:36.781689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:42:29.176361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.133879+0000","last_change":"2023-11-06T03:09:34.779247+0000","last_active":"2023-11-06T03:09:35.133879+0000","last_peered":"2023-11-06T03:09:35.133879+0000","last_clean":"2023-11-06T03:09:35.133879+0000","last_became_active":"2023-11-06T03:09:34.778750+0000","last_became_peered":"2023-11-06T03:09:34.778750+0000","last_unstale":"2023-11-06T03:09:35.133879+0000","last_undegraded":"2023-11-06T03:09:35.133879+0000","last_fullsized":"2023-11-06T03:09:35.133879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:38:36.107197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454611+0000","last_change":"2023-11-06T03:09:35.172164+0000","last_active":"2023-11-06T03:09:36.454611+0000","last_peered":"2023-11-06T03:09:36.454611+0000","last_clean":"2023-11-06T03:09:36.454611+0000","last_became_active":"2023-11-06T03:09:35.171854+0000","last_became_peered":"2023-11-06T03:09:35.171854+0000","last_unstale":"2023-11-06T03:09:36.454611+0000","last_undegraded":"2023-11-06T03:09:36.454611+0000","last_fullsized":"2023-11-06T03:09:36.454611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:17.213468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.165269+0000","last_change":"2023-11-06T03:09:34.793263+0000","last_active":"2023-11-06T03:09:35.165269+0000","last_peered":"2023-11-06T03:09:35.165269+0000","last_clean":"2023-11-06T03:09:35.165269+0000","last_became_active":"2023-11-06T03:09:34.792607+0000","last_became_peered":"2023-11-06T03:09:34.792607+0000","last_unstale":"2023-11-06T03:09:35.165269+0000","last_undegraded":"2023-11-06T03:09:35.165269+0000","last_fullsized":"2023-11-06T03:09:35.165269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:34.004658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237258+0000","last_change":"2023-11-06T03:09:35.179533+0000","last_active":"2023-11-06T03:09:37.237258+0000","last_peered":"2023-11-06T03:09:37.237258+0000","last_clean":"2023-11-06T03:09:37.237258+0000","last_became_active":"2023-11-06T03:09:35.178735+0000","last_became_peered":"2023-11-06T03:09:35.178735+0000","last_unstale":"2023-11-06T03:09:37.237258+0000","last_undegraded":"2023-11-06T03:09:37.237258+0000","last_fullsized":"2023-11-06T03:09:37.237258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:36:35.444322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160351+0000","last_change":"2023-11-06T03:09:34.777403+0000","last_active":"2023-11-06T03:09:35.160351+0000","last_peered":"2023-11-06T03:09:35.160351+0000","last_clean":"2023-11-06T03:09:35.160351+0000","last_became_active":"2023-11-06T03:09:34.776668+0000","last_became_peered":"2023-11-06T03:09:34.776668+0000","last_unstale":"2023-11-06T03:09:35.160351+0000","last_undegraded":"2023-11-06T03:09:35.160351+0000","last_fullsized":"2023-11-06T03:09:35.160351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:25.401813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235400+0000","last_change":"2023-11-06T03:09:35.156712+0000","last_active":"2023-11-06T03:09:37.235400+0000","last_peered":"2023-11-06T03:09:37.235400+0000","last_clean":"2023-11-06T03:09:37.235400+0000","last_became_active":"2023-11-06T03:09:35.156362+0000","last_became_peered":"2023-11-06T03:09:35.156362+0000","last_unstale":"2023-11-06T03:09:37.235400+0000","last_undegraded":"2023-11-06T03:09:37.235400+0000","last_fullsized":"2023-11-06T03:09:37.235400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:24:43.528030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"84'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.149345+0000","last_change":"2023-11-06T03:09:34.774633+0000","last_active":"2023-11-06T03:09:35.149345+0000","last_peered":"2023-11-06T03:09:35.149345+0000","last_clean":"2023-11-06T03:09:35.149345+0000","last_became_active":"2023-11-06T03:09:34.773898+0000","last_became_peered":"2023-11-06T03:09:34.773898+0000","last_unstale":"2023-11-06T03:09:35.149345+0000","last_undegraded":"2023-11-06T03:09:35.149345+0000","last_fullsized":"2023-11-06T03:09:35.149345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:10.466179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142736+0000","last_change":"2023-11-06T03:09:35.153554+0000","last_active":"2023-11-06T03:09:36.142736+0000","last_peered":"2023-11-06T03:09:36.142736+0000","last_clean":"2023-11-06T03:09:36.142736+0000","last_became_active":"2023-11-06T03:09:35.153292+0000","last_became_peered":"2023-11-06T03:09:35.153292+0000","last_unstale":"2023-11-06T03:09:36.142736+0000","last_undegraded":"2023-11-06T03:09:36.142736+0000","last_fullsized":"2023-11-06T03:09:36.142736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"84'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.198516+0000","last_change":"2023-11-06T03:09:34.791639+0000","last_active":"2023-11-06T03:09:35.198516+0000","last_peered":"2023-11-06T03:09:35.198516+0000","last_clean":"2023-11-06T03:09:35.198516+0000","last_became_active":"2023-11-06T03:09:34.790932+0000","last_became_peered":"2023-11-06T03:09:34.790932+0000","last_unstale":"2023-11-06T03:09:35.198516+0000","last_undegraded":"2023-11-06T03:09:35.198516+0000","last_fullsized":"2023-11-06T03:09:35.198516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:23.267517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.237776+0000","last_change":"2023-11-06T03:09:35.193265+0000","last_active":"2023-11-06T03:09:37.237776+0000","last_peered":"2023-11-06T03:09:37.237776+0000","last_clean":"2023-11-06T03:09:37.237776+0000","last_became_active":"2023-11-06T03:09:35.192506+0000","last_became_peered":"2023-11-06T03:09:35.192506+0000","last_unstale":"2023-11-06T03:09:37.237776+0000","last_undegraded":"2023-11-06T03:09:37.237776+0000","last_fullsized":"2023-11-06T03:09:37.237776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:37:36.739383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.194945+0000","last_change":"2023-11-06T03:09:34.787497+0000","last_active":"2023-11-06T03:09:35.194945+0000","last_peered":"2023-11-06T03:09:35.194945+0000","last_clean":"2023-11-06T03:09:35.194945+0000","last_became_active":"2023-11-06T03:09:34.786691+0000","last_became_peered":"2023-11-06T03:09:34.786691+0000","last_unstale":"2023-11-06T03:09:35.194945+0000","last_undegraded":"2023-11-06T03:09:35.194945+0000","last_fullsized":"2023-11-06T03:09:35.194945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781232+0000","last_change":"2023-11-06T03:09:35.153326+0000","last_active":"2023-11-06T03:09:36.781232+0000","last_peered":"2023-11-06T03:09:36.781232+0000","last_clean":"2023-11-06T03:09:36.781232+0000","last_became_active":"2023-11-06T03:09:35.152580+0000","last_became_peered":"2023-11-06T03:09:35.152580+0000","last_unstale":"2023-11-06T03:09:36.781232+0000","last_undegraded":"2023-11-06T03:09:36.781232+0000","last_fullsized":"2023-11-06T03:09:36.781232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:32.992736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.149070+0000","last_change":"2023-11-06T03:09:34.778124+0000","last_active":"2023-11-06T03:09:35.149070+0000","last_peered":"2023-11-06T03:09:35.149070+0000","last_clean":"2023-11-06T03:09:35.149070+0000","last_became_active":"2023-11-06T03:09:34.777392+0000","last_became_peered":"2023-11-06T03:09:34.777392+0000","last_unstale":"2023-11-06T03:09:35.149070+0000","last_undegraded":"2023-11-06T03:09:35.149070+0000","last_fullsized":"2023-11-06T03:09:35.149070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:34.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781490+0000","last_change":"2023-11-06T03:09:35.184432+0000","last_active":"2023-11-06T03:09:36.781490+0000","last_peered":"2023-11-06T03:09:36.781490+0000","last_clean":"2023-11-06T03:09:36.781490+0000","last_became_active":"2023-11-06T03:09:35.183743+0000","last_became_peered":"2023-11-06T03:09:35.183743+0000","last_unstale":"2023-11-06T03:09:36.781490+0000","last_undegraded":"2023-11-06T03:09:36.781490+0000","last_fullsized":"2023-11-06T03:09:36.781490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:42:51.392820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134148+0000","last_change":"2023-11-06T03:09:34.776812+0000","last_active":"2023-11-06T03:09:35.134148+0000","last_peered":"2023-11-06T03:09:35.134148+0000","last_clean":"2023-11-06T03:09:35.134148+0000","last_became_active":"2023-11-06T03:09:34.776519+0000","last_became_peered":"2023-11-06T03:09:34.776519+0000","last_unstale":"2023-11-06T03:09:35.134148+0000","last_undegraded":"2023-11-06T03:09:35.134148+0000","last_fullsized":"2023-11-06T03:09:35.134148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:47.775244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781736+0000","last_change":"2023-11-06T03:09:35.170548+0000","last_active":"2023-11-06T03:09:36.781736+0000","last_peered":"2023-11-06T03:09:36.781736+0000","last_clean":"2023-11-06T03:09:36.781736+0000","last_became_active":"2023-11-06T03:09:35.168906+0000","last_became_peered":"2023-11-06T03:09:35.168906+0000","last_unstale":"2023-11-06T03:09:36.781736+0000","last_undegraded":"2023-11-06T03:09:36.781736+0000","last_fullsized":"2023-11-06T03:09:36.781736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:01.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"84'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.187347+0000","last_change":"2023-11-06T03:09:34.783382+0000","last_active":"2023-11-06T03:09:35.187347+0000","last_peered":"2023-11-06T03:09:35.187347+0000","last_clean":"2023-11-06T03:09:35.187347+0000","last_became_active":"2023-11-06T03:09:34.783101+0000","last_became_peered":"2023-11-06T03:09:34.783101+0000","last_unstale":"2023-11-06T03:09:35.187347+0000","last_undegraded":"2023-11-06T03:09:35.187347+0000","last_fullsized":"2023-11-06T03:09:35.187347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:48:31.766442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146266+0000","last_change":"2023-11-06T03:09:35.189348+0000","last_active":"2023-11-06T03:09:36.146266+0000","last_peered":"2023-11-06T03:09:36.146266+0000","last_clean":"2023-11-06T03:09:36.146266+0000","last_became_active":"2023-11-06T03:09:35.188417+0000","last_became_peered":"2023-11-06T03:09:35.188417+0000","last_unstale":"2023-11-06T03:09:36.146266+0000","last_undegraded":"2023-11-06T03:09:36.146266+0000","last_fullsized":"2023-11-06T03:09:36.146266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:35.774307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.193140+0000","last_change":"2023-11-06T03:09:34.769704+0000","last_active":"2023-11-06T03:09:35.193140+0000","last_peered":"2023-11-06T03:09:35.193140+0000","last_clean":"2023-11-06T03:09:35.193140+0000","last_became_active":"2023-11-06T03:09:34.768908+0000","last_became_peered":"2023-11-06T03:09:34.768908+0000","last_unstale":"2023-11-06T03:09:35.193140+0000","last_undegraded":"2023-11-06T03:09:35.193140+0000","last_fullsized":"2023-11-06T03:09:35.193140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:06:35.765433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781215+0000","last_change":"2023-11-06T03:09:35.158291+0000","last_active":"2023-11-06T03:09:36.781215+0000","last_peered":"2023-11-06T03:09:36.781215+0000","last_clean":"2023-11-06T03:09:36.781215+0000","last_became_active":"2023-11-06T03:09:35.157597+0000","last_became_peered":"2023-11-06T03:09:35.157597+0000","last_unstale":"2023-11-06T03:09:36.781215+0000","last_undegraded":"2023-11-06T03:09:36.781215+0000","last_fullsized":"2023-11-06T03:09:36.781215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:31.328291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.143221+0000","last_change":"2023-11-06T03:09:34.775463+0000","last_active":"2023-11-06T03:09:35.143221+0000","last_peered":"2023-11-06T03:09:35.143221+0000","last_clean":"2023-11-06T03:09:35.143221+0000","last_became_active":"2023-11-06T03:09:34.774827+0000","last_became_peered":"2023-11-06T03:09:34.774827+0000","last_unstale":"2023-11-06T03:09:35.143221+0000","last_undegraded":"2023-11-06T03:09:35.143221+0000","last_fullsized":"2023-11-06T03:09:35.143221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:06:19.846997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.146847+0000","last_change":"2023-11-06T03:09:35.165362+0000","last_active":"2023-11-06T03:09:36.146847+0000","last_peered":"2023-11-06T03:09:36.146847+0000","last_clean":"2023-11-06T03:09:36.146847+0000","last_became_active":"2023-11-06T03:09:35.162797+0000","last_became_peered":"2023-11-06T03:09:35.162797+0000","last_unstale":"2023-11-06T03:09:36.146847+0000","last_undegraded":"2023-11-06T03:09:36.146847+0000","last_fullsized":"2023-11-06T03:09:36.146847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:02.137082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.150454+0000","last_change":"2023-11-06T03:09:34.777417+0000","last_active":"2023-11-06T03:09:35.150454+0000","last_peered":"2023-11-06T03:09:35.150454+0000","last_clean":"2023-11-06T03:09:35.150454+0000","last_became_active":"2023-11-06T03:09:34.776676+0000","last_became_peered":"2023-11-06T03:09:34.776676+0000","last_unstale":"2023-11-06T03:09:35.150454+0000","last_undegraded":"2023-11-06T03:09:35.150454+0000","last_fullsized":"2023-11-06T03:09:35.150454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:38.283897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454984+0000","last_change":"2023-11-06T03:09:35.180432+0000","last_active":"2023-11-06T03:09:36.454984+0000","last_peered":"2023-11-06T03:09:36.454984+0000","last_clean":"2023-11-06T03:09:36.454984+0000","last_became_active":"2023-11-06T03:09:35.180255+0000","last_became_peered":"2023-11-06T03:09:35.180255+0000","last_unstale":"2023-11-06T03:09:36.454984+0000","last_undegraded":"2023-11-06T03:09:36.454984+0000","last_fullsized":"2023-11-06T03:09:36.454984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:09:42.668606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.148884+0000","last_change":"2023-11-06T03:09:34.799741+0000","last_active":"2023-11-06T03:09:35.148884+0000","last_peered":"2023-11-06T03:09:35.148884+0000","last_clean":"2023-11-06T03:09:35.148884+0000","last_became_active":"2023-11-06T03:09:34.799062+0000","last_became_peered":"2023-11-06T03:09:34.799062+0000","last_unstale":"2023-11-06T03:09:35.148884+0000","last_undegraded":"2023-11-06T03:09:35.148884+0000","last_fullsized":"2023-11-06T03:09:35.148884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:01.133897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781469+0000","last_change":"2023-11-06T03:09:35.169559+0000","last_active":"2023-11-06T03:09:36.781469+0000","last_peered":"2023-11-06T03:09:36.781469+0000","last_clean":"2023-11-06T03:09:36.781469+0000","last_became_active":"2023-11-06T03:09:35.168839+0000","last_became_peered":"2023-11-06T03:09:35.168839+0000","last_unstale":"2023-11-06T03:09:36.781469+0000","last_undegraded":"2023-11-06T03:09:36.781469+0000","last_fullsized":"2023-11-06T03:09:36.781469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:36:39.328805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.140638+0000","last_change":"2023-11-06T03:09:34.788438+0000","last_active":"2023-11-06T03:09:35.140638+0000","last_peered":"2023-11-06T03:09:35.140638+0000","last_clean":"2023-11-06T03:09:35.140638+0000","last_became_active":"2023-11-06T03:09:34.787795+0000","last_became_peered":"2023-11-06T03:09:34.787795+0000","last_unstale":"2023-11-06T03:09:35.140638+0000","last_undegraded":"2023-11-06T03:09:35.140638+0000","last_fullsized":"2023-11-06T03:09:35.140638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:35.373916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781291+0000","last_change":"2023-11-06T03:09:35.148838+0000","last_active":"2023-11-06T03:09:36.781291+0000","last_peered":"2023-11-06T03:09:36.781291+0000","last_clean":"2023-11-06T03:09:36.781291+0000","last_became_active":"2023-11-06T03:09:35.148089+0000","last_became_peered":"2023-11-06T03:09:35.148089+0000","last_unstale":"2023-11-06T03:09:36.781291+0000","last_undegraded":"2023-11-06T03:09:36.781291+0000","last_fullsized":"2023-11-06T03:09:36.781291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:44:38.071325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134853+0000","last_change":"2023-11-06T03:09:34.778026+0000","last_active":"2023-11-06T03:09:35.134853+0000","last_peered":"2023-11-06T03:09:35.134853+0000","last_clean":"2023-11-06T03:09:35.134853+0000","last_became_active":"2023-11-06T03:09:34.777313+0000","last_became_peered":"2023-11-06T03:09:34.777313+0000","last_unstale":"2023-11-06T03:09:35.134853+0000","last_undegraded":"2023-11-06T03:09:35.134853+0000","last_fullsized":"2023-11-06T03:09:35.134853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:14:23.306837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142904+0000","last_change":"2023-11-06T03:09:35.156187+0000","last_active":"2023-11-06T03:09:36.142904+0000","last_peered":"2023-11-06T03:09:36.142904+0000","last_clean":"2023-11-06T03:09:36.142904+0000","last_became_active":"2023-11-06T03:09:35.155918+0000","last_became_peered":"2023-11-06T03:09:35.155918+0000","last_unstale":"2023-11-06T03:09:36.142904+0000","last_undegraded":"2023-11-06T03:09:36.142904+0000","last_fullsized":"2023-11-06T03:09:36.142904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:22.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.142771+0000","last_change":"2023-11-06T03:09:34.777643+0000","last_active":"2023-11-06T03:09:35.142771+0000","last_peered":"2023-11-06T03:09:35.142771+0000","last_clean":"2023-11-06T03:09:35.142771+0000","last_became_active":"2023-11-06T03:09:34.776923+0000","last_became_peered":"2023-11-06T03:09:34.776923+0000","last_unstale":"2023-11-06T03:09:35.142771+0000","last_undegraded":"2023-11-06T03:09:35.142771+0000","last_fullsized":"2023-11-06T03:09:35.142771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:48:58.266816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.142988+0000","last_change":"2023-11-06T03:09:35.180342+0000","last_active":"2023-11-06T03:09:36.142988+0000","last_peered":"2023-11-06T03:09:36.142988+0000","last_clean":"2023-11-06T03:09:36.142988+0000","last_became_active":"2023-11-06T03:09:35.180015+0000","last_became_peered":"2023-11-06T03:09:35.180015+0000","last_unstale":"2023-11-06T03:09:36.142988+0000","last_undegraded":"2023-11-06T03:09:36.142988+0000","last_fullsized":"2023-11-06T03:09:36.142988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:19.690734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.144442+0000","last_change":"2023-11-06T03:09:34.770016+0000","last_active":"2023-11-06T03:09:35.144442+0000","last_peered":"2023-11-06T03:09:35.144442+0000","last_clean":"2023-11-06T03:09:35.144442+0000","last_became_active":"2023-11-06T03:09:34.769272+0000","last_became_peered":"2023-11-06T03:09:34.769272+0000","last_unstale":"2023-11-06T03:09:35.144442+0000","last_undegraded":"2023-11-06T03:09:35.144442+0000","last_fullsized":"2023-11-06T03:09:35.144442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:58:53.377624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.083285+0000","last_change":"2023-11-06T03:09:35.165741+0000","last_active":"2023-11-06T03:09:37.083285+0000","last_peered":"2023-11-06T03:09:37.083285+0000","last_clean":"2023-11-06T03:09:37.083285+0000","last_became_active":"2023-11-06T03:09:35.164974+0000","last_became_peered":"2023-11-06T03:09:35.164974+0000","last_unstale":"2023-11-06T03:09:37.083285+0000","last_undegraded":"2023-11-06T03:09:37.083285+0000","last_fullsized":"2023-11-06T03:09:37.083285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:25:03.192055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.146493+0000","last_change":"2023-11-06T03:09:34.792141+0000","last_active":"2023-11-06T03:09:35.146493+0000","last_peered":"2023-11-06T03:09:35.146493+0000","last_clean":"2023-11-06T03:09:35.146493+0000","last_became_active":"2023-11-06T03:09:34.791850+0000","last_became_peered":"2023-11-06T03:09:34.791850+0000","last_unstale":"2023-11-06T03:09:35.146493+0000","last_undegraded":"2023-11-06T03:09:35.146493+0000","last_fullsized":"2023-11-06T03:09:35.146493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:12:39.070790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.162452+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:09:35.162452+0000","last_peered":"2023-11-06T03:09:35.162452+0000","last_clean":"2023-11-06T03:09:35.162452+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:09:35.162452+0000","last_undegraded":"2023-11-06T03:09:35.162452+0000","last_fullsized":"2023-11-06T03:09:35.162452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160119+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:09:35.160119+0000","last_peered":"2023-11-06T03:09:35.160119+0000","last_clean":"2023-11-06T03:09:35.160119+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:09:35.160119+0000","last_undegraded":"2023-11-06T03:09:35.160119+0000","last_fullsized":"2023-11-06T03:09:35.160119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454739+0000","last_change":"2023-11-06T03:09:35.182913+0000","last_active":"2023-11-06T03:09:36.454739+0000","last_peered":"2023-11-06T03:09:36.454739+0000","last_clean":"2023-11-06T03:09:36.454739+0000","last_became_active":"2023-11-06T03:09:35.182611+0000","last_became_peered":"2023-11-06T03:09:35.182611+0000","last_unstale":"2023-11-06T03:09:36.454739+0000","last_undegraded":"2023-11-06T03:09:36.454739+0000","last_fullsized":"2023-11-06T03:09:36.454739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:44:53.416028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145783+0000","last_change":"2023-11-06T03:09:34.784096+0000","last_active":"2023-11-06T03:09:35.145783+0000","last_peered":"2023-11-06T03:09:35.145783+0000","last_clean":"2023-11-06T03:09:35.145783+0000","last_became_active":"2023-11-06T03:09:34.783680+0000","last_became_peered":"2023-11-06T03:09:34.783680+0000","last_unstale":"2023-11-06T03:09:35.145783+0000","last_undegraded":"2023-11-06T03:09:35.145783+0000","last_fullsized":"2023-11-06T03:09:35.145783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:44.480989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.162217+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:09:35.162217+0000","last_peered":"2023-11-06T03:09:35.162217+0000","last_clean":"2023-11-06T03:09:35.162217+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:09:35.162217+0000","last_undegraded":"2023-11-06T03:09:35.162217+0000","last_fullsized":"2023-11-06T03:09:35.162217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.781861+0000","last_change":"2023-11-06T03:09:35.193898+0000","last_active":"2023-11-06T03:09:36.781861+0000","last_peered":"2023-11-06T03:09:36.781861+0000","last_clean":"2023-11-06T03:09:36.781861+0000","last_became_active":"2023-11-06T03:09:35.193553+0000","last_became_peered":"2023-11-06T03:09:35.193553+0000","last_unstale":"2023-11-06T03:09:36.781861+0000","last_undegraded":"2023-11-06T03:09:36.781861+0000","last_fullsized":"2023-11-06T03:09:36.781861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:38.069714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145473+0000","last_change":"2023-11-06T03:09:34.783828+0000","last_active":"2023-11-06T03:09:35.145473+0000","last_peered":"2023-11-06T03:09:35.145473+0000","last_clean":"2023-11-06T03:09:35.145473+0000","last_became_active":"2023-11-06T03:09:34.783326+0000","last_became_peered":"2023-11-06T03:09:34.783326+0000","last_unstale":"2023-11-06T03:09:35.145473+0000","last_undegraded":"2023-11-06T03:09:35.145473+0000","last_fullsized":"2023-11-06T03:09:35.145473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:11.921942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.134229+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:09:35.134229+0000","last_peered":"2023-11-06T03:09:35.134229+0000","last_clean":"2023-11-06T03:09:35.134229+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:09:35.134229+0000","last_undegraded":"2023-11-06T03:09:35.134229+0000","last_fullsized":"2023-11-06T03:09:35.134229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.454985+0000","last_change":"2023-11-06T03:09:35.195675+0000","last_active":"2023-11-06T03:09:36.454985+0000","last_peered":"2023-11-06T03:09:36.454985+0000","last_clean":"2023-11-06T03:09:36.454985+0000","last_became_active":"2023-11-06T03:09:35.195391+0000","last_became_peered":"2023-11-06T03:09:35.195391+0000","last_unstale":"2023-11-06T03:09:36.454985+0000","last_undegraded":"2023-11-06T03:09:36.454985+0000","last_fullsized":"2023-11-06T03:09:36.454985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:48:19.216099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.155878+0000","last_change":"2023-11-06T03:09:34.767511+0000","last_active":"2023-11-06T03:09:35.155878+0000","last_peered":"2023-11-06T03:09:35.155878+0000","last_clean":"2023-11-06T03:09:35.155878+0000","last_became_active":"2023-11-06T03:09:34.766834+0000","last_became_peered":"2023-11-06T03:09:34.766834+0000","last_unstale":"2023-11-06T03:09:35.155878+0000","last_undegraded":"2023-11-06T03:09:35.155878+0000","last_fullsized":"2023-11-06T03:09:35.155878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:39:15.254924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.160079+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:09:35.160079+0000","last_peered":"2023-11-06T03:09:35.160079+0000","last_clean":"2023-11-06T03:09:35.160079+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:09:35.160079+0000","last_undegraded":"2023-11-06T03:09:35.160079+0000","last_fullsized":"2023-11-06T03:09:35.160079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735938+0000","last_change":"2023-11-06T03:09:35.151930+0000","last_active":"2023-11-06T03:09:36.735938+0000","last_peered":"2023-11-06T03:09:36.735938+0000","last_clean":"2023-11-06T03:09:36.735938+0000","last_became_active":"2023-11-06T03:09:35.151654+0000","last_became_peered":"2023-11-06T03:09:35.151654+0000","last_unstale":"2023-11-06T03:09:36.735938+0000","last_undegraded":"2023-11-06T03:09:36.735938+0000","last_fullsized":"2023-11-06T03:09:36.735938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:12:03.266444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.156290+0000","last_change":"2023-11-06T03:09:34.785373+0000","last_active":"2023-11-06T03:09:35.156290+0000","last_peered":"2023-11-06T03:09:35.156290+0000","last_clean":"2023-11-06T03:09:35.156290+0000","last_became_active":"2023-11-06T03:09:34.784939+0000","last_became_peered":"2023-11-06T03:09:34.784939+0000","last_unstale":"2023-11-06T03:09:35.156290+0000","last_undegraded":"2023-11-06T03:09:35.156290+0000","last_fullsized":"2023-11-06T03:09:35.156290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:10.987052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145694+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:09:35.145694+0000","last_peered":"2023-11-06T03:09:35.145694+0000","last_clean":"2023-11-06T03:09:35.145694+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:09:35.145694+0000","last_undegraded":"2023-11-06T03:09:35.145694+0000","last_fullsized":"2023-11-06T03:09:35.145694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.235243+0000","last_change":"2023-11-06T03:09:35.156345+0000","last_active":"2023-11-06T03:09:37.235243+0000","last_peered":"2023-11-06T03:09:37.235243+0000","last_clean":"2023-11-06T03:09:37.235243+0000","last_became_active":"2023-11-06T03:09:35.156090+0000","last_became_peered":"2023-11-06T03:09:35.156090+0000","last_unstale":"2023-11-06T03:09:37.235243+0000","last_undegraded":"2023-11-06T03:09:37.235243+0000","last_fullsized":"2023-11-06T03:09:37.235243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:35.830319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145622+0000","last_change":"2023-11-06T03:09:34.774319+0000","last_active":"2023-11-06T03:09:35.145622+0000","last_peered":"2023-11-06T03:09:35.145622+0000","last_clean":"2023-11-06T03:09:35.145622+0000","last_became_active":"2023-11-06T03:09:34.774046+0000","last_became_peered":"2023-11-06T03:09:34.774046+0000","last_unstale":"2023-11-06T03:09:35.145622+0000","last_undegraded":"2023-11-06T03:09:35.145622+0000","last_fullsized":"2023-11-06T03:09:35.145622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:55.234985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.151868+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:09:35.151868+0000","last_peered":"2023-11-06T03:09:35.151868+0000","last_clean":"2023-11-06T03:09:35.151868+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:09:35.151868+0000","last_undegraded":"2023-11-06T03:09:35.151868+0000","last_fullsized":"2023-11-06T03:09:35.151868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:37.234980+0000","last_change":"2023-11-06T03:09:35.155835+0000","last_active":"2023-11-06T03:09:37.234980+0000","last_peered":"2023-11-06T03:09:37.234980+0000","last_clean":"2023-11-06T03:09:37.234980+0000","last_became_active":"2023-11-06T03:09:35.155528+0000","last_became_peered":"2023-11-06T03:09:35.155528+0000","last_unstale":"2023-11-06T03:09:37.234980+0000","last_undegraded":"2023-11-06T03:09:37.234980+0000","last_fullsized":"2023-11-06T03:09:37.234980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:21.984806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.183291+0000","last_change":"2023-11-06T03:09:34.776900+0000","last_active":"2023-11-06T03:09:35.183291+0000","last_peered":"2023-11-06T03:09:35.183291+0000","last_clean":"2023-11-06T03:09:35.183291+0000","last_became_active":"2023-11-06T03:09:34.776169+0000","last_became_peered":"2023-11-06T03:09:34.776169+0000","last_unstale":"2023-11-06T03:09:35.183291+0000","last_undegraded":"2023-11-06T03:09:35.183291+0000","last_fullsized":"2023-11-06T03:09:35.183291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:36.344706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.145797+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:09:35.145797+0000","last_peered":"2023-11-06T03:09:35.145797+0000","last_clean":"2023-11-06T03:09:35.145797+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:09:35.145797+0000","last_undegraded":"2023-11-06T03:09:35.145797+0000","last_fullsized":"2023-11-06T03:09:35.145797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-06T03:09:36.735586+0000","last_change":"2023-11-06T03:09:35.168659+0000","last_active":"2023-11-06T03:09:36.735586+0000","last_peered":"2023-11-06T03:09:36.735586+0000","last_clean":"2023-11-06T03:09:36.735586+0000","last_became_active":"2023-11-06T03:09:35.168402+0000","last_became_peered":"2023-11-06T03:09:35.168402+0000","last_unstale":"2023-11-06T03:09:36.735586+0000","last_undegraded":"2023-11-06T03:09:36.735586+0000","last_fullsized":"2023-11-06T03:09:36.735586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:34.736732+0000","last_clean_scrub_stamp":"2023-11-06T03:09:34.736732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:42:36.377101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.182418+0000","last_change":"2023-11-06T03:09:34.777060+0000","last_active":"2023-11-06T03:09:35.182418+0000","last_peered":"2023-11-06T03:09:35.182418+0000","last_clean":"2023-11-06T03:09:35.182418+0000","last_became_active":"2023-11-06T03:09:34.776357+0000","last_became_peered":"2023-11-06T03:09:34.776357+0000","last_unstale":"2023-11-06T03:09:35.182418+0000","last_undegraded":"2023-11-06T03:09:35.182418+0000","last_fullsized":"2023-11-06T03:09:35.182418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:09:33.733547+0000","last_clean_scrub_stamp":"2023-11-06T03:09:33.733547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:00.858051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-06T03:09:35.142959+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:09:35.142959+0000","last_peered":"2023-11-06T03:09:35.142959+0000","last_clean":"2023-11-06T03:09:35.142959+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:09:35.142959+0000","last_undegraded":"2023-11-06T03:09:35.142959+0000","last_fullsized":"2023-11-06T03:09:35.142959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50692,"kb_used_data":1596,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321148,"statfs":{"total":96636764160,"available":96584855552,"internally_reserved":0,"allocated":1634304,"data_stored":1063345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82499999999999996,"15min":0.88300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0760000000000001,"5min":1.397,"15min":1.397},"last":1.3620000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81499999999999995,"15min":0.86099999999999999},"min":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94699999999999995,"5min":1.444,"15min":1.444},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.86899999999999999,"15min":0.89600000000000002},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.877,"15min":0.877},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.381,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.288},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.879,"15min":0.95299999999999996},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0629999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.252}]},{"osd":3,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.127,"15min":1.0669999999999999},"min":{"1min":0.65600000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":10.596,"5min":10.596,"15min":10.596},"last":0.998}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92500000000000004,"15min":0.97299999999999998},"min":{"1min":0.52000000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.208,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.099},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94999999999999996,"15min":1.01},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.282,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.2150000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.042,"15min":1.05},"min":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.095},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3240000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.476}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1060000000000001,"15min":1.1539999999999999},"min":{"1min":0.79600000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.51,"5min":1.774,"15min":1.774},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.099,"15min":1.117},"min":{"1min":0.90800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5430000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.3260000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46020,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325820,"statfs":{"total":96636764160,"available":96589639680,"internally_reserved":0,"allocated":1044480,"data_stored":473500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":0.96199999999999997,"15min":0.84899999999999998},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":16.073,"5min":16.073,"15min":16.073},"last":1.466},{"interface":"front","average":{"1min":3.0750000000000002,"5min":1.256,"15min":0.97599999999999998},"min":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":40.905000000000001,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.048,"15min":0.89700000000000002},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":16.012,"5min":16.012,"15min":16.012},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.02,"5min":0.89400000000000002,"15min":0.879},"min":{"1min":0.60499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.84899999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.331,"5min":1.1120000000000001,"15min":0.90600000000000003},"min":{"1min":0.54300000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":27.803999999999998,"5min":27.803999999999998,"15min":27.803999999999998},"last":1.155},{"interface":"front","average":{"1min":1.004,"5min":0.96999999999999997,"15min":0.95699999999999996},"min":{"1min":0.64000000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.843,"5min":2.843,"15min":2.843},"last":1.542}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":1.621,"15min":1.1919999999999999},"min":{"1min":0.57999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":63.737000000000002,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":3.121,"5min":1.365,"15min":1.0449999999999999},"min":{"1min":0.57199999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":41.079999999999998,"5min":41.079999999999998,"15min":41.079999999999998},"last":1.0580000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.282,"15min":1.1839999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":24.867999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.373},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.359,"15min":1.2370000000000001},"min":{"1min":0.76700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":23.789000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.103}]},{"osd":5,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":3.1789999999999998,"5min":1.4359999999999999,"15min":1.1779999999999999},"min":{"1min":0.71599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":40.959000000000003,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.1479999999999999,"15min":1.1120000000000001},"min":{"1min":0.90400000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.6160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":2.3679999999999999,"5min":1.3460000000000001,"15min":1.1899999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":23.748999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.234},{"interface":"front","average":{"1min":2.6160000000000001,"5min":1.407,"15min":1.22},"min":{"1min":0.81100000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":28.164000000000001,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.2589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":1040384,"data_stored":472989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.81200000000000006},"min":{"1min":0.45500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0580000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.48599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.175,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.81499999999999995,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.145,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2230000000000001,"5min":1.357,"15min":1.357},"last":0.85599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86199999999999999,"15min":0.83799999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2629999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83499999999999996},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.84199999999999997},"min":{"1min":0.66400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1970000000000001,"5min":1.403,"15min":1.403},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94999999999999996,"15min":0.93600000000000005},"min":{"1min":0.72799999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":1.117}]},{"osd":4,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.90300000000000002,"15min":0.91900000000000004},"min":{"1min":0.64900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.1060000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2330000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.014,"15min":0.99399999999999999},"min":{"1min":0.63800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.206,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0029999999999999,"15min":1.0089999999999999},"min":{"1min":0.79900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.306,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:09:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0269999999999999,"15min":0.98899999999999999},"min":{"1min":0.74399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.389,"5min":1.476,"15min":1.476},"last":1.087},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0940000000000001,"15min":1.0840000000000001},"min":{"1min":1.008,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2909999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50128,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321712,"statfs":{"total":96636764160,"available":96585433088,"internally_reserved":0,"allocated":1056768,"data_stored":474623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79400000000000004,"15min":0.82299999999999995},"min":{"1min":0.44600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.1619999999999999,"5min":1.484,"15min":1.484},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2989999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.755},"min":{"1min":0.48099999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1930000000000001,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85499999999999998,"15min":0.81200000000000006},"min":{"1min":0.55300000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.91600000000000004,"15min":1.04},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85699999999999998,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90700000000000003,"15min":0.93899999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1539999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.90200000000000002,"15min":0.89800000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1919999999999999,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.71899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.069,"15min":1.105},"min":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.365,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5489999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0860000000000001,"15min":1.1519999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.411,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":0.94899999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.073},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.3009999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.2430000000000001,"15min":1.1870000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.6739999999999999,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.0960000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50108,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321732,"statfs":{"total":96636764160,"available":96585453568,"internally_reserved":0,"allocated":1036288,"data_stored":472518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.248,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.26700000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.90400000000000003,"5min":1.524,"15min":1.524},"last":0.622}]},{"osd":1,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.38700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.1080000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.91500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.28899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.042,"5min":1.349,"15min":1.349},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.37,"5min":0.255,"15min":0.255},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":1.0349999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86799999999999999,"15min":0.90600000000000003},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.59299999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.518,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.94599999999999995,"15min":0.98399999999999999},"min":{"1min":0.53000000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.536,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.66400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5049999999999999,"5min":1.698,"15min":1.698},"last":0.99199999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97599999999999998,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.4610000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.94899999999999995,"15min":1},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.391,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Mon Nov 6 03:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.008,"15min":1.056},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3180000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.024,"15min":1.0649999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4930000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0029999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":1048576,"data_stored":473697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.24299999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.374},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60599999999999998,"15min":0.64400000000000002},"min":{"1min":0.255,"5min":0.223,"15min":0.223},"max":{"1min":1.0109999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.62,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.121},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.629,"15min":0.66900000000000004},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98599999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.1259999999999999,"5min":1.294,"15min":1.294},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.749},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.89000000000000001},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":0.95499999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2250000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.84599999999999997,"15min":0.88400000000000001},"min":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.214,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93400000000000005,"15min":0.96799999999999997},"min":{"1min":0.62,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.995},"min":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.381,"5min":1.615,"15min":1.615},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.95199999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:09:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.006,"15min":1},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.93100000000000005,"15min":0.96199999999999997},"min":{"1min":0.51500000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.256,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.84999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":473067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.56299999999999994,"15min":0.59199999999999997},"min":{"1min":0.223,"5min":0.19,"15min":0.19},"max":{"1min":1.0629999999999999,"5min":1.157,"15min":1.157},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.26900000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.0649999999999999,"5min":1.544,"15min":1.544},"last":0.371}]},{"osd":2,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.90700000000000003,"15min":0.71999999999999997},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.63900000000000001,"15min":0.68300000000000005},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.03,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69699999999999995,"15min":0.72699999999999998},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.1200000000000001,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.68400000000000005,"15min":0.72599999999999998},"min":{"1min":0.35199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90600000000000003,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.2809999999999999,"5min":2.258,"15min":2.258},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":14.753,"15min":14.753},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.3919999999999999,"15min":1.1399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.161,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.367,"15min":1.167},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.272,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.755}]},{"osd":6,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.177,"15min":1.0760000000000001},"min":{"1min":0.51400000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":23.971,"15min":23.971},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2949999999999999,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0149999999999999,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3560000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.93400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58876,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312964,"statfs":{"total":96636764160,"available":96576475136,"internally_reserved":0,"allocated":1626112,"data_stored":1062304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.28299999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.145,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.69899999999999995,"15min":0.65100000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1120000000000001,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.22,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83199999999999996,"5min":1.155,"15min":1.155},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.68500000000000005,"15min":0.65400000000000003},"min":{"1min":0.318,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.876,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.67000000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.63900000000000001,"15min":0.623},"min":{"1min":0.29899999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0700000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0589999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.95499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.96299999999999997,"15min":0.92700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1299999999999999,"5min":4.641,"15min":4.641},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.88500000000000001,"15min":0.93100000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1,"5min":2.028,"15min":2.028},"last":1.054}]},{"osd":5,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1140000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.94699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2050000000000001,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":1.117}]},{"osd":6,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":2.931,"15min":2.931},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0049999999999999,"15min":1.01},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.248,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.091}]},{"osd":7,"last update":"Mon Nov 6 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.0289999999999999,"15min":1.0089999999999999},"min":{"1min":0.61199999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2649999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.98699999999999999,"15min":0.99199999999999999},"min":{"1min":0.64800000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.97,"15min":1.97},"last":1.155}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:09:42.976 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:09:42.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:09:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 Processor -- start 2023-11-06T03:09:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 -- start start 2023-11-06T03:09:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f488c0b2af0 0x7f488c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f488c06d000 0x7f488c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c15e440 con 0x7f488c1599c0 2023-11-06T03:09:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06c240 con 0x7f488c0b2af0 2023-11-06T03:09:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4890f1f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06c520 con 0x7f488c06d000 2023-11-06T03:09:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f488a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f488c1599c0 0x7f488c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54446/0 (socket says 172.21.15.157:54446) 2023-11-06T03:09:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f488a59c640 1 -- 172.21.15.157:0/1467106185 learned_addr learned my addr 172.21.15.157:0/1467106185 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4889d9b640 1 --2- 172.21.15.157:0/1467106185 >> v2:172.21.15.157:3300/0 conn(0x7f488c0b2af0 0x7f488c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f488ad9d640 1 --2- 172.21.15.157:0/1467106185 >> v2:172.21.15.195:3300/0 conn(0x7f488c06d000 0x7f488c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f488959a640 1 -- 172.21.15.157:0/1467106185 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661073502 0 0) 0x7f488c15e440 con 0x7f488c1599c0 2023-11-06T03:09:43.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f488959a640 1 -- 172.21.15.157:0/1467106185 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4878003530 con 0x7f488c1599c0 2023-11-06T03:09:43.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4889d9b640 1 -- 172.21.15.157:0/1467106185 >> v2:172.21.15.195:3300/0 conn(0x7f488c06d000 msgr2=0x7f488c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:43.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4889d9b640 1 --2- 172.21.15.157:0/1467106185 >> v2:172.21.15.195:3300/0 conn(0x7f488c06d000 0x7f488c159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4889d9b640 1 -- 172.21.15.157:0/1467106185 >> v1:172.21.15.157:6789/0 conn(0x7f488c1599c0 legacy=0x7f488c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.144+0000 7f4889d9b640 1 -- 172.21.15.157:0/1467106185 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c06c800 con 0x7f488c0b2af0 2023-11-06T03:09:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4889d9b640 1 --2- 172.21.15.157:0/1467106185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c0ae000 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f488c06d930 tx=0x7f487401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3271194774a19c35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f488959a640 1 -- 172.21.15.157:0/1467106185 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740ab650 con 0x7f488c0b2af0 2023-11-06T03:09:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f488959a640 1 -- 172.21.15.157:0/1467106185 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740ab7b0 con 0x7f488c0b2af0 2023-11-06T03:09:43.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f488959a640 1 -- 172.21.15.157:0/1467106185 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740abb40 con 0x7f488c0b2af0 2023-11-06T03:09:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 -- 172.21.15.157:0/1467106185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 msgr2=0x7f488c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 --2- 172.21.15.157:0/1467106185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c0ae000 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f488c06d930 tx=0x7f487401c430 comp rx=0 tx=0).stop 2023-11-06T03:09:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 -- 172.21.15.157:0/1467106185 shutdown_connections 2023-11-06T03:09:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 --2- 172.21.15.157:0/1467106185 >> v2:172.21.15.195:3300/0 conn(0x7f488c06d000 0x7f488c159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 --2- 172.21.15.157:0/1467106185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c0ae000 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 -- 172.21.15.157:0/1467106185 >> 172.21.15.157:0/1467106185 conn(0x7f488c065930 msgr2=0x7f488c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 -- 172.21.15.157:0/1467106185 shutdown_connections 2023-11-06T03:09:43.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.148+0000 7f4890f1f640 1 -- 172.21.15.157:0/1467106185 wait complete. 2023-11-06T03:09:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 Processor -- start 2023-11-06T03:09:43.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- start start 2023-11-06T03:09:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f488c06d000 0x7f488c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f488c1599c0 0x7f488c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0cdd20 con 0x7f488c0b2af0 2023-11-06T03:09:43.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c14cd50 con 0x7f488c1599c0 2023-11-06T03:09:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0cdf30 con 0x7f488c06d000 2023-11-06T03:09:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46914/0 (socket says 172.21.15.157:46914) 2023-11-06T03:09:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 -- 172.21.15.157:0/595220615 learned_addr learned my addr 172.21.15.157:0/595220615 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:43.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f488a59c640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f488c06d000 0x7f488c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f488ad9d640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f488c1599c0 0x7f488c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 -- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f488c06d000 msgr2=0x7f488c14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f488c06d000 0x7f488c14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 -- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f488c1599c0 msgr2=0x7f488c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f488c1599c0 0x7f488c156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 -- 172.21.15.157:0/595220615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009420 con 0x7f488c0b2af0 2023-11-06T03:09:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4889d9b640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4874019040 tx=0x7f487401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740bf610 con 0x7f488c0b2af0 2023-11-06T03:09:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740bf790 con 0x7f488c0b2af0 2023-11-06T03:09:43.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740bfb40 con 0x7f488c0b2af0 2023-11-06T03:09:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0ce210 con 0x7f488c0b2af0 2023-11-06T03:09:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c14c930 con 0x7f488c0b2af0 2023-11-06T03:09:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.152+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c0ce730 con 0x7f488c0b2af0 2023-11-06T03:09:43.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.156+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f48740bfca0 con 0x7f488c0b2af0 2023-11-06T03:09:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.156+0000 7f4882ffd640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 0x7f486c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.156+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f487411dd10 con 0x7f488c0b2af0 2023-11-06T03:09:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.156+0000 7f488a59c640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 0x7f486c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.156+0000 7f488a59c640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 0x7f486c067f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4884001570 tx=0x7f4884009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.160+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f488c0ce730 con 0x7f488c0b2af0 2023-11-06T03:09:43.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.372+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f488c0ce730 con 0x7f488c0b2af0 2023-11-06T03:09:43.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4882ffd640 1 -- 172.21.15.157:0/595220615 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f488c0ce730 con 0x7f488c0b2af0 2023-11-06T03:09:43.382 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:09:43.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 msgr2=0x7f486c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 0x7f486c067f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4884001570 tx=0x7f4884009040 comp rx=0 tx=0).stop 2023-11-06T03:09:43.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 msgr2=0x7f488c155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4874019040 tx=0x7f487401ac70 comp rx=0 tx=0).stop 2023-11-06T03:09:43.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 shutdown_connections 2023-11-06T03:09:43.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f486c065a60 0x7f486c067f20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f488c1599c0 0x7f488c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f488c0b2af0 0x7f488c155f50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 --2- 172.21.15.157:0/595220615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f488c06d000 0x7f488c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 >> 172.21.15.157:0/595220615 conn(0x7f488c065930 msgr2=0x7f488c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:43.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 shutdown_connections 2023-11-06T03:09:43.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.384+0000 7f4890f1f640 1 -- 172.21.15.157:0/595220615 wait complete. 2023-11-06T03:09:43.401 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:09:43.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.520+0000 7f6f31b3d640 1 Processor -- start 2023-11-06T03:09:43.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.520+0000 7f6f31b3d640 1 -- start start 2023-11-06T03:09:43.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f31b3d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f2c15caa0 0x7f6f2c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f31b3d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c15ce80 0x7f6f2c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f31b3d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2c0d2920 con 0x7f6f2c0ce1a0 2023-11-06T03:09:43.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f31b3d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2c0d18a0 con 0x7f6f2c15caa0 2023-11-06T03:09:43.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f31b3d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c0d1b60 con 0x7f6f2c15ce80 2023-11-06T03:09:43.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f2c15caa0 0x7f6f2c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6f2c0ce1a0 0x7f6f2c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54452/0 (socket says 172.21.15.157:54452) 2023-11-06T03:09:43.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2bfff640 1 -- 172.21.15.157:0/2278229926 learned_addr learned my addr 172.21.15.157:0/2278229926 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:43.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2affd640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c15ce80 0x7f6f2c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 -- 172.21.15.157:0/2278229926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c15ce80 msgr2=0x7f6f2c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:43.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c15ce80 0x7f6f2c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 -- 172.21.15.157:0/2278229926 >> v1:172.21.15.157:6789/0 conn(0x7f6f2c0ce1a0 legacy=0x7f6f2c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2a7fc640 1 -- 172.21.15.157:0/2278229926 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80907428 0 0) 0x7f6f2c0d2920 con 0x7f6f2c0ce1a0 2023-11-06T03:09:43.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 -- 172.21.15.157:0/2278229926 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0d1e40 con 0x7f6f2c15caa0 2023-11-06T03:09:43.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2b7fe640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c0cdc60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6f200097c0 tx=0x7f6f2001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a641c073b47fb55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2a7fc640 1 -- 172.21.15.157:0/2278229926 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200ad660 con 0x7f6f2c15caa0 2023-11-06T03:09:43.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2a7fc640 1 -- 172.21.15.157:0/2278229926 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f200ad7c0 con 0x7f6f2c15caa0 2023-11-06T03:09:43.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.524+0000 7f6f2a7fc640 1 -- 172.21.15.157:0/2278229926 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200adb50 con 0x7f6f2c15caa0 2023-11-06T03:09:43.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/2278229926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15caa0 msgr2=0x7f6f2c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c0cdc60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6f200097c0 tx=0x7f6f2001c450 comp rx=0 tx=0).stop 2023-11-06T03:09:43.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/2278229926 shutdown_connections 2023-11-06T03:09:43.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c15ce80 0x7f6f2c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/2278229926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c0cdc60 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/2278229926 >> 172.21.15.157:0/2278229926 conn(0x7f6f2c070790 msgr2=0x7f6f2c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/2278229926 shutdown_connections 2023-11-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/2278229926 wait complete. 2023-11-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 Processor -- start 2023-11-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- start start 2023-11-06T03:09:43.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c0ce1a0 0x7f6f2c152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c160550 con 0x7f6f2c15ce80 2023-11-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c15f9d0 con 0x7f6f2c15caa0 2023-11-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c15fcb0 con 0x7f6f2c0ce1a0 2023-11-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46928/0 (socket says 172.21.15.157:46928) 2023-11-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 -- 172.21.15.157:0/1716303816 learned_addr learned my addr 172.21.15.157:0/1716303816 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2affd640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2b7fe640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c0ce1a0 0x7f6f2c152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 -- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c0ce1a0 msgr2=0x7f6f2c152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c0ce1a0 0x7f6f2c152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 -- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f2c15caa0 msgr2=0x7f6f2c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 -- 172.21.15.157:0/1716303816 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20009420 con 0x7f6f2c15ce80 2023-11-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f2bfff640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c6d0 tx=0x7f6f2400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24004d70 con 0x7f6f2c15ce80 2023-11-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400a640 con 0x7f6f2c15ce80 2023-11-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c15ff90 con 0x7f6f2c15ce80 2023-11-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400aa70 con 0x7f6f2c15ce80 2023-11-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.528+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c0c6a20 con 0x7f6f2c15ce80 2023-11-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c072750 con 0x7f6f2c15ce80 2023-11-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6f240070d0 con 0x7f6f2c15ce80 2023-11-06T03:09:43.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f13fff640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 0x7f6f18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:43.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6f24082930 con 0x7f6f2c15ce80 2023-11-06T03:09:43.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f2b7fe640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 0x7f6f18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:43.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.532+0000 7f6f2b7fe640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 0x7f6f18067f20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:43.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.536+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6f24050220 con 0x7f6f2c15ce80 2023-11-06T03:09:43.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f2c152fa0 con 0x7f6f2c15ce80 2023-11-06T03:09:43.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f13fff640 1 -- 172.21.15.157:0/1716303816 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4653 (secure 0 0 0) 0x7f6f24018070 con 0x7f6f2c15ce80 2023-11-06T03:09:43.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:09:43.719 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":60,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:09:37.140140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:09:43.719 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 60 2023-11-06T03:09:43.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 msgr2=0x7f6f18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 0x7f6f18067f20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200a7000 comp rx=0 tx=0).stop 2023-11-06T03:09:43.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 msgr2=0x7f6f2c0b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:43.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c6d0 tx=0x7f6f2400cb90 comp rx=0 tx=0).stop 2023-11-06T03:09:43.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 shutdown_connections 2023-11-06T03:09:43.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6f18065a60 0x7f6f18067f20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f2c15ce80 0x7f6f2c0b36b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f2c15caa0 0x7f6f2c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 --2- 172.21.15.157:0/1716303816 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f2c0ce1a0 0x7f6f2c152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:43.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 >> 172.21.15.157:0/1716303816 conn(0x7f6f2c070790 msgr2=0x7f6f2c15a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:43.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 shutdown_connections 2023-11-06T03:09:43.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:43.720+0000 7f6f31b3d640 1 -- 172.21.15.157:0/1716303816 wait complete. 2023-11-06T03:09:44.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:09:35.127602+0000', 'modified': '2023-11-06T03:09:37.140140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7329}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7329': {'gid': 7329, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/3882288197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3882288197}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3882288197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:09:44.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:09:44.140 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:09:44.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 Processor -- start 2023-11-06T03:09:44.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 -- start start 2023-11-06T03:09:44.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03200ce8c0 0x7f0320164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032015aec0 con 0x7f032006cd50 2023-11-06T03:09:44.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032015a2b0 con 0x7f0320154290 2023-11-06T03:09:44.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0327039640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032015a590 con 0x7f03200ce8c0 2023-11-06T03:09:44.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f032006cd50 0x7f0320070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54468/0 (socket says 172.21.15.157:54468) 2023-11-06T03:09:44.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 -- 172.21.15.157:0/790793707 learned_addr learned my addr 172.21.15.157:0/790793707 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:09:44.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f03255d6640 1 --2- 172.21.15.157:0/790793707 >> v2:172.21.15.195:3300/0 conn(0x7f03200ce8c0 0x7f0320164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600673739 0 0) 0x7f032015aec0 con 0x7f032006cd50 2023-11-06T03:09:44.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0308003530 con 0x7f032006cd50 2023-11-06T03:09:44.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655374347 0 0) 0x7f0308003530 con 0x7f032006cd50 2023-11-06T03:09:44.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 >> v2:172.21.15.195:3300/0 conn(0x7f03200ce8c0 msgr2=0x7f0320164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 --2- 172.21.15.157:0/790793707 >> v2:172.21.15.195:3300/0 conn(0x7f03200ce8c0 0x7f0320164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 >> v1:172.21.15.195:6789/0 conn(0x7f0320154290 legacy=0x7f032006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032015a870 con 0x7f032006cd50 2023-11-06T03:09:44.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0314001a00 con 0x7f032006cd50 2023-11-06T03:09:44.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 >> v1:172.21.15.157:6789/0 conn(0x7f032006cd50 legacy=0x7f0320070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0308003980 0x7f0308003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 0x7f0308004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0308005190 0x7f030800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032015a590 con 0x7f03080047d0 2023-11-06T03:09:44.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030800c030 con 0x7f0308005190 2023-11-06T03:09:44.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030800c1f0 con 0x7f0308003980 2023-11-06T03:09:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 0x7f0308004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f0324dd5640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0308003980 0x7f0308003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 -- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0308003980 msgr2=0x7f0308003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0308003980 0x7f0308003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 -- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0308005190 msgr2=0x7f030800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0308005190 0x7f030800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f031ffff640 1 -- 172.21.15.157:0/790793707 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0308003530 con 0x7f03080047d0 2023-11-06T03:09:44.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.272+0000 7f03255d6640 1 -- 172.21.15.157:0/790793707 reap_dead start 2023-11-06T03:09:44.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f031ffff640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 0x7f0308004c50 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f031400a830 tx=0x7f031401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be5cd1b686423c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140aa6d0 con 0x7f03080047d0 2023-11-06T03:09:44.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140aa830 con 0x7f03080047d0 2023-11-06T03:09:44.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f031f7fe640 1 -- 172.21.15.157:0/790793707 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140aac60 con 0x7f03080047d0 2023-11-06T03:09:44.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 msgr2=0x7f0308004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 0x7f0308004c50 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f031400a830 tx=0x7f031401faa0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- 172.21.15.157:0/790793707 shutdown_connections 2023-11-06T03:09:44.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 --2- 172.21.15.157:0/790793707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03080047d0 0x7f0308004c50 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- 172.21.15.157:0/790793707 >> 172.21.15.157:0/790793707 conn(0x7f03200cf8d0 msgr2=0x7f03200cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- 172.21.15.157:0/790793707 shutdown_connections 2023-11-06T03:09:44.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- 172.21.15.157:0/790793707 wait complete. 2023-11-06T03:09:44.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 Processor -- start 2023-11-06T03:09:44.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.276+0000 7f0327039640 1 -- start start 2023-11-06T03:09:44.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03080047d0 0x7f0320155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03200b0d10 0x7f03201f1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200af730 con 0x7f03200b1b60 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200afa10 con 0x7f03200b0d10 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03200afcf0 con 0x7f03080047d0 2023-11-06T03:09:44.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46940/0 (socket says 172.21.15.157:46940) 2023-11-06T03:09:44.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 -- 172.21.15.157:0/978997343 learned_addr learned my addr 172.21.15.157:0/978997343 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0324dd5640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03080047d0 0x7f0320155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f03255d6640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03200b0d10 0x7f03201f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 -- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03080047d0 msgr2=0x7f0320155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03080047d0 0x7f0320155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 -- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03200b0d10 msgr2=0x7f03201f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03200b0d10 0x7f03201f1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 -- 172.21.15.157:0/978997343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200affd0 con 0x7f03200b1b60 2023-11-06T03:09:44.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031ffff640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f031401f8d0 tx=0x7f0308005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 -- 172.21.15.157:0/978997343 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140017c0 con 0x7f03200b1b60 2023-11-06T03:09:44.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 -- 172.21.15.157:0/978997343 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314001920 con 0x7f03200b1b60 2023-11-06T03:09:44.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03200b02b0 con 0x7f03200b1b60 2023-11-06T03:09:44.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 -- 172.21.15.157:0/978997343 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140aabe0 con 0x7f03200b1b60 2023-11-06T03:09:44.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201f1a80 con 0x7f03200b1b60 2023-11-06T03:09:44.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 -- 172.21.15.157:0/978997343 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f03140aad40 con 0x7f03200b1b60 2023-11-06T03:09:44.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 0x7f030c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f031d7fa640 1 -- 172.21.15.157:0/978997343 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03140b9070 con 0x7f03200b1b60 2023-11-06T03:09:44.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.280+0000 7f0324dd5640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 0x7f030c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 Processor -- start 2023-11-06T03:09:44.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- start start 2023-11-06T03:09:44.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 0x7f030c067f20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f032015bd60 tx=0x7f0318009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aac8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1aac8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a990fc0 con 0x555d1ab126e0 2023-11-06T03:09:44.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a91f1a0 con 0x555d1ab13000 2023-11-06T03:09:44.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1a91c770 con 0x555d1aac8a90 2023-11-06T03:09:44.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46952/0 (socket says 172.21.15.157:46952) 2023-11-06T03:09:44.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 -- 172.21.15.157:0/830465059 learned_addr learned my addr 172.21.15.157:0/830465059 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 -- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 msgr2=0x555d1aac8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f031ffff640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aac8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f03255d6640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1aac8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1aac8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 -- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 msgr2=0x555d1aac8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aac8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 -- 172.21.15.157:0/830465059 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1a614890 con 0x555d1ab126e0 2023-11-06T03:09:44.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f031ffff640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aac8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:09:44.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f0324dd5640 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0318001520 tx=0x7f0318013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3fabfb496c3883 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f03067fc640 1 -- 172.21.15.157:0/830465059 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180146c0 con 0x555d1ab126e0 2023-11-06T03:09:44.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f03067fc640 1 -- 172.21.15.157:0/830465059 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318014820 con 0x555d1ab126e0 2023-11-06T03:09:44.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f03067fc640 1 -- 172.21.15.157:0/830465059 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318014b90 con 0x555d1ab126e0 2023-11-06T03:09:44.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- 172.21.15.157:0/830465059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 msgr2=0x555d1ab12ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0318001520 tx=0x7f0318013000 comp rx=0 tx=0).stop 2023-11-06T03:09:44.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- 172.21.15.157:0/830465059 shutdown_connections 2023-11-06T03:09:44.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1aac8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aac8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- 172.21.15.157:0/830465059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab12ac0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- 172.21.15.157:0/830465059 >> 172.21.15.157:0/830465059 conn(0x555d1aacebf0 msgr2=0x555d1aac6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- 172.21.15.157:0/830465059 shutdown_connections 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- 172.21.15.157:0/830465059 wait complete. 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 -- start start 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 Processor -- start 2023-11-06T03:09:44.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.284+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab126e0 0x555d1aad3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aad4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a918060 con 0x555d1aac8a90 2023-11-06T03:09:44.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a8b7290 con 0x555d1ab13000 2023-11-06T03:09:44.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1a8d56f0 con 0x555d1ab126e0 2023-11-06T03:09:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 10 client.0 _ll_register_callbacks cb 0x555d1aab41d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46960/0 (socket says 172.21.15.157:46960) 2023-11-06T03:09:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 -- 172.21.15.157:0/4126087161 learned_addr learned my addr 172.21.15.157:0/4126087161 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 -- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab126e0 msgr2=0x555d1aad3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f03255d6640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aad4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab126e0 0x555d1aad3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 -- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 msgr2=0x555d1aad4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aad4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 -- 172.21.15.157:0/4126087161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031800f040 con 0x555d1aac8a90 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f0324dd5640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0318001cf0 tx=0x7f03180011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a302aceb7f0deb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031802e630 con 0x555d1aac8a90 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d1a9169b0 con 0x555d1aac8a90 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f031801c050 con 0x555d1aac8a90 2023-11-06T03:09:44.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031802e790 con 0x555d1aac8a90 2023-11-06T03:09:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 10 client.7500 fetch_fsmap learned FSMap version 60 2023-11-06T03:09:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x555d1a8dd1e0 con 0x555d1aac8a90 2023-11-06T03:09:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d1a9169b0 con 0x555d1aac8a90 2023-11-06T03:09:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0318029070 con 0x555d1aac8a90 2023-11-06T03:09:44.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 10 client.7500 fetch_fsmap finished waiting for FSMap version 60 2023-11-06T03:09:44.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 10 client.7500 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:09:44.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 20 client.7500 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:44.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab21980 0x555d1ab23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 4 client.7500 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:09:44.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d1a9474d0 con 0x555d1ab21980 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f031ffff640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab21980 0x555d1ab23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 10 client.7500 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f031ffff640 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab21980 0x555d1ab23d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee08b2d24208ed00 server_cookie=82778271de4e1aad in_seq=0 out_seq=0 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 1 -- 172.21.15.157:0/4126087161 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555d1a9474d0 con 0x555d1ab21980 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 10 client.7500 handle_command_reply: tid=0 2023-11-06T03:09:44.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 client.7500 shutdown 2023-11-06T03:09:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 msgr2=0x555d1aad3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0318001cf0 tx=0x7f03180011a0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 shutdown_connections 2023-11-06T03:09:44.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab21980 0x555d1ab23d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1aad4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab126e0 0x555d1aad3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 --2- 172.21.15.157:0/4126087161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1aac8a90 0x555d1aad3880 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f02ff7fe640 0 client.7500 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:09:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 >> 172.21.15.157:0/4126087161 conn(0x555d1aacebf0 msgr2=0x555d1aac7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 shutdown_connections 2023-11-06T03:09:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 1 -- 172.21.15.157:0/4126087161 wait complete. 2023-11-06T03:09:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.288+0000 7f032a161000 20 client.7500 trim_cache size 0 max 16384 2023-11-06T03:09:44.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 Processor -- start 2023-11-06T03:09:44.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- start start 2023-11-06T03:09:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1ab255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1ab0a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a5d5940 con 0x555d1ab126e0 2023-11-06T03:09:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a9c30b0 con 0x555d1ab13000 2023-11-06T03:09:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1a9139d0 con 0x555d1aac8a90 2023-11-06T03:09:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46976/0 (socket says 172.21.15.157:46976) 2023-11-06T03:09:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 -- 172.21.15.157:0/3012828690 learned_addr learned my addr 172.21.15.157:0/3012828690 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 -- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 msgr2=0x555d1ab255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1ab255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 -- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 msgr2=0x555d1ab0a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1ab0a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 -- 172.21.15.157:0/3012828690 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1a8c3330 con 0x555d1ab126e0 2023-11-06T03:09:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f031ffff640 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f031411f300 tx=0x7f0314001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5642e1bba8d9d36d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f0306ffd640 1 -- 172.21.15.157:0/3012828690 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140f6260 con 0x555d1ab126e0 2023-11-06T03:09:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f0306ffd640 1 -- 172.21.15.157:0/3012828690 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140fa410 con 0x555d1ab126e0 2023-11-06T03:09:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f0306ffd640 1 -- 172.21.15.157:0/3012828690 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140f07d0 con 0x555d1ab126e0 2023-11-06T03:09:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- 172.21.15.157:0/3012828690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 msgr2=0x555d1ab25b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f031411f300 tx=0x7f0314001a90 comp rx=0 tx=0).stop 2023-11-06T03:09:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- 172.21.15.157:0/3012828690 shutdown_connections 2023-11-06T03:09:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1ab13000 0x555d1ab0a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab25b20 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 --2- 172.21.15.157:0/3012828690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1aac8a90 0x555d1ab255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- 172.21.15.157:0/3012828690 >> 172.21.15.157:0/3012828690 conn(0x555d1aacebf0 msgr2=0x555d1ab24ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- 172.21.15.157:0/3012828690 shutdown_connections 2023-11-06T03:09:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.304+0000 7f032a161000 1 -- 172.21.15.157:0/3012828690 wait complete. 2023-11-06T03:09:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- start start 2023-11-06T03:09:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 Processor -- start 2023-11-06T03:09:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1aac8a90 0x555d1ab77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab13000 0x555d1ab0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1aacc570 con 0x555d1ab126e0 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1a916ef0 con 0x555d1aac8a90 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1a91c060 con 0x555d1ab13000 2023-11-06T03:09:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 10 client.0 _ll_register_callbacks cb 0x555d1aa949e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:09:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46992/0 (socket says 172.21.15.157:46992) 2023-11-06T03:09:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 -- 172.21.15.157:0/2058633695 learned_addr learned my addr 172.21.15.157:0/2058633695 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0324dd5640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1aac8a90 0x555d1ab77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 -- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab13000 msgr2=0x555d1ab0d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f03255d6640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab13000 0x555d1ab0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab13000 0x555d1ab0d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 -- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1aac8a90 msgr2=0x555d1ab77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1aac8a90 0x555d1ab77440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 -- 172.21.15.157:0/2058633695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201f1a80 con 0x555d1ab126e0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f031ffff640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f031400a830 tx=0x7f0314107040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd0f292aa87c824 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031410d070 con 0x555d1ab126e0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03140f6260 con 0x555d1ab126e0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d1a8e7b90 con 0x555d1ab126e0 2023-11-06T03:09:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03140f07d0 con 0x555d1ab126e0 2023-11-06T03:09:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 10 client.7506 fetch_fsmap learned FSMap version 60 2023-11-06T03:09:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x555d1a614cc0 con 0x555d1ab126e0 2023-11-06T03:09:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d1a8e7b90 con 0x555d1ab126e0 2023-11-06T03:09:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 60) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f03140b0020 con 0x555d1ab126e0 2023-11-06T03:09:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 10 client.7506 fetch_fsmap finished waiting for FSMap version 60 2023-11-06T03:09:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 10 client.7506 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:09:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 20 client.7506 populate_metadata read hostname 'smithi157' 2023-11-06T03:09:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab0df10 0x555d1ab0e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 4 client.7506 mds_command: new command op to 7329 tid=0[{"prefix": "status"}] 2023-11-06T03:09:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555d1a9110e0 con 0x555d1ab0df10 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0324dd5640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab0df10 0x555d1ab0e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0324dd5640 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab0df10 0x555d1ab0e2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f69c17172a926e server_cookie=672c0a0cdbe0ff2f in_seq=0 out_seq=0 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 10 client.7506 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 1 -- 172.21.15.157:0/2058633695 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x555d1a9110e0 con 0x555d1ab0df10 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 10 client.7506 handle_command_reply: tid=0 2023-11-06T03:09:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 client.7506 shutdown 2023-11-06T03:09:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 msgr2=0x555d1ab77980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f031400a830 tx=0x7f0314107040 comp rx=0 tx=0).stop 2023-11-06T03:09:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 shutdown_connections 2023-11-06T03:09:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x555d1ab0df10 0x555d1ab0e2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555d1ab13000 0x555d1ab0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f02ff7fe640 0 client.7506 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:09:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555d1ab126e0 0x555d1ab77980 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 --2- 172.21.15.157:0/2058633695 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555d1aac8a90 0x555d1ab77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 >> 172.21.15.157:0/2058633695 conn(0x555d1aacebf0 msgr2=0x555d1ab24ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 shutdown_connections 2023-11-06T03:09:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 1 -- 172.21.15.157:0/2058633695 wait complete. 2023-11-06T03:09:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f032a161000 20 client.7506 trim_cache size 0 max 16384 2023-11-06T03:09:44.338 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7329, 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:09:44.339 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:09:44.340 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 9.1813533730000003, 2023-11-06T03:09:44.340 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 60, 2023-11-06T03:09:44.340 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 85, 2023-11-06T03:09:44.340 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 85, 2023-11-06T03:09:44.340 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 13.638792656 2023-11-06T03:09:44.341 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:44.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 msgr2=0x7f030c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 0x7f030c067f20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f032015bd60 tx=0x7f0318009040 comp rx=0 tx=0).stop 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 msgr2=0x7f03200b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f031401f8d0 tx=0x7f0308005800 comp rx=0 tx=0).stop 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 shutdown_connections 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f030c065a60 0x7f030c067f20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03200b0d10 0x7f03201f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03200b1b60 0x7f03200b07d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 --2- 172.21.15.157:0/978997343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03080047d0 0x7f0320155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.308+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 >> 172.21.15.157:0/978997343 conn(0x7f03200cf8d0 msgr2=0x7f03201534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.312+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 shutdown_connections 2023-11-06T03:09:44.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.312+0000 7f0327039640 1 -- 172.21.15.157:0/978997343 wait complete. 2023-11-06T03:09:44.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:09:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.456+0000 7f997bb8f640 1 Processor -- start 2023-11-06T03:09:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 -- start start 2023-11-06T03:09:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9974144430 0x7f99740b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997415c290 0x7f99740661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740cef40 con 0x7f9974065d40 2023-11-06T03:09:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740cd450 con 0x7f997415c290 2023-11-06T03:09:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997bb8f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740cc840 con 0x7f9974144430 2023-11-06T03:09:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997415c290 0x7f99740661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f997415c290 0x7f99740661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47006/0 (socket says 172.21.15.157:47006) 2023-11-06T03:09:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 -- 172.21.15.157:0/609492935 learned_addr learned my addr 172.21.15.157:0/609492935 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997992b640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9974144430 0x7f99740b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 -- 172.21.15.157:0/609492935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9974144430 msgr2=0x7f99740b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9974144430 0x7f99740b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 -- 172.21.15.157:0/609492935 >> v1:172.21.15.195:6789/0 conn(0x7f9974065d40 legacy=0x7f99740cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 -- 172.21.15.157:0/609492935 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974066700 con 0x7f997415c290 2023-11-06T03:09:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f997a12c640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997415c290 0x7f99740661c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f996c009e50 tx=0x7f996c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865fb523d15ddc23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f9978929640 1 -- 172.21.15.157:0/609492935 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0a5660 con 0x7f997415c290 2023-11-06T03:09:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f9978929640 1 -- 172.21.15.157:0/609492935 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0a57c0 con 0x7f997415c290 2023-11-06T03:09:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.460+0000 7f9978929640 1 -- 172.21.15.157:0/609492935 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0a5b50 con 0x7f997415c290 2023-11-06T03:09:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/609492935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997415c290 msgr2=0x7f99740661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997415c290 0x7f99740661c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f996c009e50 tx=0x7f996c01c9d0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/609492935 shutdown_connections 2023-11-06T03:09:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f997415c290 0x7f99740661c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- 172.21.15.157:0/609492935 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9974144430 0x7f99740b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/609492935 >> 172.21.15.157:0/609492935 conn(0x7f9974065930 msgr2=0x7f9974071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/609492935 shutdown_connections 2023-11-06T03:09:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/609492935 wait complete. 2023-11-06T03:09:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 Processor -- start 2023-11-06T03:09:44.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- start start 2023-11-06T03:09:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9974144430 0x7f997413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997415c290 0x7f99740add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974135670 con 0x7f9974065d40 2023-11-06T03:09:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974136470 con 0x7f9974144430 2023-11-06T03:09:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99741358a0 con 0x7f997415c290 2023-11-06T03:09:44.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47012/0 (socket says 172.21.15.157:47012) 2023-11-06T03:09:44.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 -- 172.21.15.157:0/3754327205 learned_addr learned my addr 172.21.15.157:0/3754327205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:44.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997a12c640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997415c290 0x7f99740add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 -- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997415c290 msgr2=0x7f99740add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997912a640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9974144430 0x7f997413ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997415c290 0x7f99740add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 -- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9974144430 msgr2=0x7f997413ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9974144430 0x7f997413ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 -- 172.21.15.157:0/3754327205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009b30 con 0x7f9974065d40 2023-11-06T03:09:44.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997992b640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f997000d800 tx=0x7f997000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000b6c0 con 0x7f9974065d40 2023-11-06T03:09:44.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000b820 con 0x7f9974065d40 2023-11-06T03:09:44.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9974135b40 con 0x7f9974065d40 2023-11-06T03:09:44.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000bb90 con 0x7f9974065d40 2023-11-06T03:09:44.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.464+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974136100 con 0x7f9974065d40 2023-11-06T03:09:44.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974072670 con 0x7f9974065d40 2023-11-06T03:09:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f997000bcf0 con 0x7f9974065d40 2023-11-06T03:09:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f996a7fc640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 0x7f9958067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99700839b0 con 0x7f9974065d40 2023-11-06T03:09:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f997912a640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 0x7f9958067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.468+0000 7f997912a640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 0x7f9958067f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f99741349f0 tx=0x7f9964009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:44.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.472+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f99700510e0 con 0x7f9974065d40 2023-11-06T03:09:44.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.656+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99740b4080 con 0x7f9974065d40 2023-11-06T03:09:44.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.656+0000 7f996a7fc640 1 -- 172.21.15.157:0/3754327205 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9970011340 con 0x7f9974065d40 2023-11-06T03:09:44.657 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:09:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 msgr2=0x7f9958067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 0x7f9958067f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f99741349f0 tx=0x7f9964009040 comp rx=0 tx=0).stop 2023-11-06T03:09:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 msgr2=0x7f99741370b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f997000d800 tx=0x7f997000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 shutdown_connections 2023-11-06T03:09:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9958065a60 0x7f9958067f20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f997415c290 0x7f99740add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9974144430 0x7f997413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 --2- 172.21.15.157:0/3754327205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9974065d40 0x7f99741370b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 >> 172.21.15.157:0/3754327205 conn(0x7f9974065930 msgr2=0x7f997413a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 shutdown_connections 2023-11-06T03:09:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:44.660+0000 7f997bb8f640 1 -- 172.21.15.157:0/3754327205 wait complete. 2023-11-06T03:09:44.676 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:09:44.676 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:09:44.676 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:09:44.677 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:09:44.677 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:44.678 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:09:44.678 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:09:44.679 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:09:44.679 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:44.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.680 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:09:44.686 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:09:44.687 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:09:44.687 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:09:44.687 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.687 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:09:44.687 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.688 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:44.688 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.688 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:09:44.688 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:44.689 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.689 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:09:44.689 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:09:44.689 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.690 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:09:44.690 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.690 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:09:44.690 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42154sec preferred_lft 42154sec 2023-11-06T03:09:44.691 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:09:44.691 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.691 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:09:44.691 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.691 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:09:44.692 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.692 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:09:44.692 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:09:44.692 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:09:44.692 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.693 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:09:44.693 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.693 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:09:44.693 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:09:44.693 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:09:44.694 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.694 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:09:44.694 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:09:44.694 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:09:44.695 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:09:44.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.698 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:09:44.740 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:09:44.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.742 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:09:44.793 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:09:44.793 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:09:44.793 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:09:44.793 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:09:44.794 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:09:44.794 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:09:44.794 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:44.794 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:44.795 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:09:44.795 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:09:44.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.796 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:09:44.796 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:09:44.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:09:44.797 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:09:44.797 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:09:44.797 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:09:44.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.871 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:09:44.918 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:09:44.919 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:09:44.919 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:44.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:44.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:09:44.975 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:09:44.975 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:09:44.976 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:45.023 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:09:45.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:09:45.024 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:45.070 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:09:45.131 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:09:45.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:09:45.185 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:09:45.186 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:09:45.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:45.187 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:09:45.234 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:09:45.235 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:09:45.279 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:09:45.294 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:09:45.294 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:09:45.349 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:09:45.350 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:09:45.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:45.351 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:09:45.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:09:45.356+0000 7f8e3eb62580 -1 init, newargv = 0x558aaa081340 newargc=15 2023-11-06T03:09:45.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[67538]: starting ceph client 2023-11-06T03:09:45.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[67538]: starting fuse 2023-11-06T03:09:46.463 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:09:46.479 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:09:46.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:09:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:09:46.539 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:09:46.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:46.540 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:09:46.586 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:09:46.586 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:09:46.587 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:09:46.587 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:09:46.587 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:09:46.588 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:09:46.588 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:09:46.588 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:46.589 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:09:46.589 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:09:46.590 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:09:46.590 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:09:46.590 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:46.591 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:09:46.591 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:09:46.591 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:09:46.592 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:09:46.592 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:46.592 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:09:46.593 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:09:46.593 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:09:46.593 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:09:46.594 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:09:46.594 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:09:46.595 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:09:46.595 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:09:46.595 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:46.595 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:09:46.596 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:09:46.707 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.67538.asok 2023-11-06T03:09:46.889 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67538.asok 2023-11-06T03:09:46.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:46.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67538.asok status 2023-11-06T03:09:47.047 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:09:47.047 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:09:47.047 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:09:47.048 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:09:47.048 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:09:47.048 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:09:47.048 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:09:47.048 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "67538", 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7518, 2023-11-06T03:09:47.049 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:09:47.050 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:09:47.050 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:09:47.050 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7518 2023-11-06T03:09:47.050 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:47.050 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:09:47.051 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:09:47.051 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:09:47.051 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 898208793 2023-11-06T03:09:47.051 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:09:47.051 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 898208793 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:09:47.052 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7518 192.168.0.1:0/898208793", 2023-11-06T03:09:47.053 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/898208793", 2023-11-06T03:09:47.053 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:09:47.053 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 60, 2023-11-06T03:09:47.053 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 85, 2023-11-06T03:09:47.053 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:09:47.054 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:09:47.054 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:09:47.054 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:09:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:47.062 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:09:47.062 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:47.069 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:09:47.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:47.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:47.070 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:09:47.126 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-06T03:09:47.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 Processor -- start 2023-11-06T03:09:47.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 -- start start 2023-11-06T03:09:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd41597b0 con 0x7f7fd406f290 2023-11-06T03:09:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40b2b20 con 0x7f7fd41593d0 2023-11-06T03:09:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fdb808640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40b2e00 con 0x7f7fd40cb820 2023-11-06T03:09:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fd9da5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fd95a4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fd95a4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:38258/0 (socket says 172.21.15.157:38258) 2023-11-06T03:09:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fd9da5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:47996/0 (socket says 172.21.15.157:47996) 2023-11-06T03:09:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.300+0000 7f7fd95a4640 1 -- 172.21.15.157:0/2471383770 learned_addr learned my addr 172.21.15.157:0/2471383770 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd9da5640 1 -- 172.21.15.157:0/2471383770 >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 msgr2=0x7f7fd40ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:09:47.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd9da5640 1 --2- 172.21.15.157:0/2471383770 >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:47.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd9da5640 1 -- 172.21.15.157:0/2471383770 >> v1:172.21.15.195:6789/0 conn(0x7f7fd406f290 legacy=0x7f7fd40cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:47.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd9da5640 1 -- 172.21.15.157:0/2471383770 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40b30e0 con 0x7f7fd40cb820 2023-11-06T03:09:47.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd95a4640 1 --2- 172.21.15.157:0/2471383770 >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:09:47.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fd9da5640 1 --2- 172.21.15.157:0/2471383770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d90 tx=0x7f7fcc01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be12b88e86174c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:47.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fcbfff640 1 -- 172.21.15.157:0/2471383770 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0a5990 con 0x7f7fd40cb820 2023-11-06T03:09:47.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fcbfff640 1 -- 172.21.15.157:0/2471383770 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0a5af0 con 0x7f7fd40cb820 2023-11-06T03:09:47.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fcbfff640 1 -- 172.21.15.157:0/2471383770 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0a5e80 con 0x7f7fd40cb820 2023-11-06T03:09:47.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 -- 172.21.15.157:0/2471383770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 msgr2=0x7f7fd40cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:47.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 --2- 172.21.15.157:0/2471383770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d90 tx=0x7f7fcc01c950 comp rx=0 tx=0).stop 2023-11-06T03:09:47.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 -- 172.21.15.157:0/2471383770 shutdown_connections 2023-11-06T03:09:47.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 --2- 172.21.15.157:0/2471383770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd40cc730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:47.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 --2- 172.21.15.157:0/2471383770 >> v2:172.21.15.195:3300/0 conn(0x7f7fd41593d0 0x7f7fd40ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:47.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 -- 172.21.15.157:0/2471383770 >> 172.21.15.157:0/2471383770 conn(0x7f7fd40b43b0 msgr2=0x7f7fd4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:47.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 -- 172.21.15.157:0/2471383770 shutdown_connections 2023-11-06T03:09:47.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.304+0000 7f7fdb808640 1 -- 172.21.15.157:0/2471383770 wait complete. 2023-11-06T03:09:47.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 Processor -- start 2023-11-06T03:09:47.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- start start 2023-11-06T03:09:47.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd41599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd41593d0 0x7f7fd414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4150990 con 0x7f7fd406f290 2023-11-06T03:09:47.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4154b80 con 0x7f7fd41593d0 2023-11-06T03:09:47.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4154e60 con 0x7f7fd40cb820 2023-11-06T03:09:47.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47026/0 (socket says 172.21.15.157:47026) 2023-11-06T03:09:47.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 -- 172.21.15.157:0/997175579 learned_addr learned my addr 172.21.15.157:0/997175579 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:09:47.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd8da3640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd41599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd9da5640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd41593d0 0x7f7fd414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 -- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 msgr2=0x7f7fd41599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:47.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd41599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:47.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 -- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd41593d0 msgr2=0x7f7fd414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:47.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd41593d0 0x7f7fd414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:47.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 -- 172.21.15.157:0/997175579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc400b780 con 0x7f7fd406f290 2023-11-06T03:09:47.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd8da3640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd41599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:09:47.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fd95a4640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400dcf0 tx=0x7f7fc400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4009da0 con 0x7f7fd406f290 2023-11-06T03:09:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc4017410 con 0x7f7fd406f290 2023-11-06T03:09:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40177c0 con 0x7f7fd406f290 2023-11-06T03:09:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc009a70 con 0x7f7fd406f290 2023-11-06T03:09:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd4155420 con 0x7f7fd406f290 2023-11-06T03:09:47.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.308+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd41555e0 con 0x7f7fd406f290 2023-11-06T03:09:47.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.312+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7fc4017920 con 0x7f7fd406f290 2023-11-06T03:09:47.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.312+0000 7f7fc9ffb640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 0x7f7fb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:09:47.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.312+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fc400b050 con 0x7f7fd406f290 2023-11-06T03:09:47.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.312+0000 7f7fd8da3640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 0x7f7fb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:09:47.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.312+0000 7f7fd8da3640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 0x7f7fb8067f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4154760 tx=0x7f7fd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:09:47.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.316+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7fc4050ce0 con 0x7f7fd406f290 2023-11-06T03:09:47.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:47.504+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7fd41548d0 con 0x7f7fd406f290 2023-11-06T03:09:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.344+0000 7f7fc9ffb640 1 -- 172.21.15.157:0/997175579 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v61) v1 ==== 167+0+0 (secure 0 0 0) 0x7f7fc4054300 con 0x7f7fd406f290 2023-11-06T03:09:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:enabled new snapshots 2023-11-06T03:09:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 msgr2=0x7f7fb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 0x7f7fb8067f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4154760 tx=0x7f7fd0009040 comp rx=0 tx=0).stop 2023-11-06T03:09:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 msgr2=0x7f7fd413fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:09:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400dcf0 tx=0x7f7fc400bf90 comp rx=0 tx=0).stop 2023-11-06T03:09:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 shutdown_connections 2023-11-06T03:09:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fb8065a90 0x7f7fb8067f50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd41593d0 0x7f7fd414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:48.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40cb820 0x7f7fd41599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 --2- 172.21.15.157:0/997175579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd406f290 0x7f7fd413fec0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:09:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 >> 172.21.15.157:0/997175579 conn(0x7f7fd40b43b0 msgr2=0x7f7fd41579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:09:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 shutdown_connections 2023-11-06T03:09:48.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:09:48.348+0000 7f7fdb808640 1 -- 172.21.15.157:0/997175579 wait complete. 2023-11-06T03:09:48.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:09:48.369 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2023-11-06T03:09:48.385 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:09:48.386 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:09:48.438 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:09:48.439 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:09:48.439 DEBUG:teuthology.orchestra.run.smithi157:> import uuid 2023-11-06T03:09:48.440 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:48.440 DEBUG:teuthology.orchestra.run.smithi157:> n = 2000 2023-11-06T03:09:48.441 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = False 2023-11-06T03:09:48.441 DEBUG:teuthology.orchestra.run.smithi157:> if 3 > 0: 2023-11-06T03:09:48.441 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = True 2023-11-06T03:09:48.442 DEBUG:teuthology.orchestra.run.smithi157:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2023-11-06T03:09:48.442 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:48.442 DEBUG:teuthology.orchestra.run.smithi157:> dpath = os.path.dirname(path) 2023-11-06T03:09:48.443 DEBUG:teuthology.orchestra.run.smithi157:> fnameprefix = os.path.basename(path) 2023-11-06T03:09:48.443 DEBUG:teuthology.orchestra.run.smithi157:> os.makedirs(dpath, exist_ok=True) 2023-11-06T03:09:48.443 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:48.444 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:09:48.444 DEBUG:teuthology.orchestra.run.smithi157:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-11-06T03:09:48.444 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:09:48.445 DEBUG:teuthology.orchestra.run.smithi157:> for i in range(n): 2023-11-06T03:09:48.445 DEBUG:teuthology.orchestra.run.smithi157:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-11-06T03:09:48.445 DEBUG:teuthology.orchestra.run.smithi157:> with open(fpath, '"'"'w'"'"') as f: 2023-11-06T03:09:48.446 DEBUG:teuthology.orchestra.run.smithi157:> f.write(f"{i}") 2023-11-06T03:09:48.446 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:09:48.446 DEBUG:teuthology.orchestra.run.smithi157:> f.flush() 2023-11-06T03:09:48.447 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(f.fileno()) 2023-11-06T03:09:48.447 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:09:48.447 DEBUG:teuthology.orchestra.run.smithi157:> os.unlink(fpath) 2023-11-06T03:09:48.448 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:09:48.448 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:09:48.448 DEBUG:teuthology.orchestra.run.smithi157:> if create_hard_links: 2023-11-06T03:09:48.449 DEBUG:teuthology.orchestra.run.smithi157:> for j in range(3): 2023-11-06T03:09:48.449 DEBUG:teuthology.orchestra.run.smithi157:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-11-06T03:09:48.449 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:09:48.450 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:09:48.450 DEBUG:teuthology.orchestra.run.smithi157:> finally: 2023-11-06T03:09:48.450 DEBUG:teuthology.orchestra.run.smithi157:> os.close(dirfd) 2023-11-06T03:09:48.451 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:10:10.628 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:10:10.631 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:10:11.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:10:11.957 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-06T03:10:11.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:10:11.998 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2023-11-06T03:10:40.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:10:40.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:11:10.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:10.352 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-11-06T03:11:10.452 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2023-11-06T03:11:10.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.575+0000 7f4c37839640 1 Processor -- start 2023-11-06T03:11:10.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.575+0000 7f4c37839640 1 -- start start 2023-11-06T03:11:10.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c37839640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c3006d000 0x7f4c300ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c37839640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3015e440 con 0x7f4c300b2af0 2023-11-06T03:11:10.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c37839640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3006c240 con 0x7f4c301599c0 2023-11-06T03:11:10.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c37839640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3006c520 con 0x7f4c3006d000 2023-11-06T03:11:10.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c34dd4640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4c300b2af0 0x7f4c30159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35930/0 (socket says 172.21.15.157:35930) 2023-11-06T03:11:10.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c34dd4640 1 -- 172.21.15.157:0/4098546129 learned_addr learned my addr 172.21.15.157:0/4098546129 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:10.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/4098546129 >> v2:172.21.15.195:3300/0 conn(0x7f4c3006d000 0x7f4c300ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42875173 0 0) 0x7f4c3015e440 con 0x7f4c300b2af0 2023-11-06T03:11:10.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c1c003530 con 0x7f4c300b2af0 2023-11-06T03:11:10.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3750542567 0 0) 0x7f4c1c003530 con 0x7f4c300b2af0 2023-11-06T03:11:10.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 >> v2:172.21.15.195:3300/0 conn(0x7f4c3006d000 msgr2=0x7f4c300ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 --2- 172.21.15.157:0/4098546129 >> v2:172.21.15.195:3300/0 conn(0x7f4c3006d000 0x7f4c300ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 >> v1:172.21.15.195:6789/0 conn(0x7f4c301599c0 legacy=0x7f4c300b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3006c800 con 0x7f4c300b2af0 2023-11-06T03:11:10.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4c20001680 con 0x7f4c300b2af0 2023-11-06T03:11:10.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 >> v1:172.21.15.157:6789/0 conn(0x7f4c300b2af0 legacy=0x7f4c30159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 0x7f4c1c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c0047d0 0x7f4c1c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c1c005190 0x7f4c1c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3006c520 con 0x7f4c1c003980 2023-11-06T03:11:10.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c00c030 con 0x7f4c1c005190 2023-11-06T03:11:10.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c00c1f0 con 0x7f4c1c0047d0 2023-11-06T03:11:10.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 0x7f4c1c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c1c005190 0x7f4c1c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c0047d0 0x7f4c1c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 -- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c0047d0 msgr2=0x7f4c1c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c0047d0 0x7f4c1c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 -- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c1c005190 msgr2=0x7f4c1c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c1c005190 0x7f4c1c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c35dd6640 1 -- 172.21.15.157:0/4098546129 reap_dead start 2023-11-06T03:11:10.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c355d5640 1 -- 172.21.15.157:0/4098546129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c003530 con 0x7f4c1c003980 2023-11-06T03:11:10.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.579+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c0047d0 0x7f4c1c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:10.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c355d5640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 0x7f4c1c003da0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001780 tx=0x7f4c2c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dd6113f0af08ad server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0052d0 con 0x7f4c1c003980 2023-11-06T03:11:10.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0a7050 con 0x7f4c1c003980 2023-11-06T03:11:10.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c27fff640 1 -- 172.21.15.157:0/4098546129 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c003600 con 0x7f4c1c003980 2023-11-06T03:11:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 msgr2=0x7f4c1c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 0x7f4c1c003da0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001780 tx=0x7f4c2c01c3b0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- 172.21.15.157:0/4098546129 shutdown_connections 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 --2- 172.21.15.157:0/4098546129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c1c003980 0x7f4c1c003da0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- 172.21.15.157:0/4098546129 >> 172.21.15.157:0/4098546129 conn(0x7f4c30065930 msgr2=0x7f4c30065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- 172.21.15.157:0/4098546129 shutdown_connections 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- 172.21.15.157:0/4098546129 wait complete. 2023-11-06T03:11:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 Processor -- start 2023-11-06T03:11:10.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- start start 2023-11-06T03:11:10.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c003980 0x7f4c30151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c300ce450 0x7f4c30143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30143870 con 0x7f4c300cdfd0 2023-11-06T03:11:10.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30150eb0 con 0x7f4c300ce450 2023-11-06T03:11:10.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c37839640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c30151190 con 0x7f4c1c003980 2023-11-06T03:11:10.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51846/0 (socket says 172.21.15.157:51846) 2023-11-06T03:11:10.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c34dd4640 1 -- 172.21.15.157:0/368049 learned_addr learned my addr 172.21.15.157:0/368049 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.583+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c300ce450 0x7f4c30143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c355d5640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c003980 0x7f4c30151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 -- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c003980 msgr2=0x7f4c30151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c003980 0x7f4c30151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 -- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c300ce450 msgr2=0x7f4c30143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c300ce450 0x7f4c30143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 -- 172.21.15.157:0/368049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30151470 con 0x7f4c300cdfd0 2023-11-06T03:11:10.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000d7a0 tx=0x7f4c2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 -- 172.21.15.157:0/368049 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200074e0 con 0x7f4c300cdfd0 2023-11-06T03:11:10.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 -- 172.21.15.157:0/368049 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20007640 con 0x7f4c300cdfd0 2023-11-06T03:11:10.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c30151750 con 0x7f4c300cdfd0 2023-11-06T03:11:10.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 -- 172.21.15.157:0/368049 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20004a30 con 0x7f4c300cdfd0 2023-11-06T03:11:10.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c30152cb0 con 0x7f4c300cdfd0 2023-11-06T03:11:10.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 -- 172.21.15.157:0/368049 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c20004b90 con 0x7f4c300cdfd0 2023-11-06T03:11:10.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c25ffb640 1 -- 172.21.15.157:0/368049 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c20082e90 con 0x7f4c300cdfd0 2023-11-06T03:11:10.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c355d5640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c3a961000 1 Processor -- start 2023-11-06T03:11:10.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.587+0000 7f4c3a961000 1 -- start start 2023-11-06T03:11:10.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c355d5640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001750 tx=0x7f4c2c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17cf65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17bbdbb0 con 0x562b17d409f0 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17b4c380 con 0x562b17d400d0 2023-11-06T03:11:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b17b49950 con 0x562b17cf6200 2023-11-06T03:11:10.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51854/0 (socket says 172.21.15.157:51854) 2023-11-06T03:11:10.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 -- 172.21.15.157:0/3842616150 learned_addr learned my addr 172.21.15.157:0/3842616150 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c355d5640 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 -- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 msgr2=0x562b17cf65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17cf65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17cf65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 -- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 msgr2=0x562b17d404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 -- 172.21.15.157:0/3842616150 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b17841dc0 con 0x562b17d409f0 2023-11-06T03:11:10.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4c20081310 tx=0x7f4c200077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258626967e87168 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c1affd640 1 -- 172.21.15.157:0/3842616150 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20054440 con 0x562b17d409f0 2023-11-06T03:11:10.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c1affd640 1 -- 172.21.15.157:0/3842616150 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20058270 con 0x562b17d409f0 2023-11-06T03:11:10.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c1affd640 1 -- 172.21.15.157:0/3842616150 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2005c420 con 0x562b17d409f0 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- 172.21.15.157:0/3842616150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 msgr2=0x562b17cf5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4c20081310 tx=0x7f4c200077b0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- 172.21.15.157:0/3842616150 shutdown_connections 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17cf65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cf5cc0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- 172.21.15.157:0/3842616150 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- 172.21.15.157:0/3842616150 >> 172.21.15.157:0/3842616150 conn(0x562b17cfc990 msgr2=0x562b17cf45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- 172.21.15.157:0/3842616150 shutdown_connections 2023-11-06T03:11:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- 172.21.15.157:0/3842616150 wait complete. 2023-11-06T03:11:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- start start 2023-11-06T03:11:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 Processor -- start 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d2cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d400d0 0x562b17cfbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17b45240 con 0x562b17d409f0 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17ae4410 con 0x562b17cf6200 2023-11-06T03:11:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b17b02600 con 0x562b17d400d0 2023-11-06T03:11:10.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c3a961000 10 client.0 _ll_register_callbacks cb 0x562b17ce1bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:10.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51868/0 (socket says 172.21.15.157:51868) 2023-11-06T03:11:10.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 -- 172.21.15.157:0/1677033517 learned_addr learned my addr 172.21.15.157:0/1677033517 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c355d5640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d2cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d400d0 0x562b17cfbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 -- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d400d0 msgr2=0x562b17cfbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d400d0 0x562b17cfbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 -- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 msgr2=0x562b17d2cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.591+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d2cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c35dd6640 1 -- 172.21.15.157:0/1677033517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c30152cb0 con 0x562b17d409f0 2023-11-06T03:11:10.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b19a0 tx=0x7f4c28004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb85f563c14de88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:10.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800e040 con 0x562b17d409f0 2023-11-06T03:11:10.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c28004830 con 0x562b17d409f0 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b17b02040 con 0x562b17d409f0 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c28013990 con 0x562b17d409f0 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 10 client.7533 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562b17b0a170 con 0x562b17d409f0 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b17b02040 con 0x562b17d409f0 2023-11-06T03:11:10.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4c28013af0 con 0x562b17d409f0 2023-11-06T03:11:10.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 10 client.7533 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:10.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 10 client.7533 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:10.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 20 client.7533 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:10.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d414b0 0x562b17d418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 4 client.7533 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:11:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b17b43b90 con 0x562b17d414b0 2023-11-06T03:11:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c355d5640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d414b0 0x562b17d418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c355d5640 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d414b0 0x562b17d418c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea489e4ee205bde server_cookie=34102898d172bfee in_seq=0 out_seq=0 2023-11-06T03:11:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 10 client.7533 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 1 -- 172.21.15.157:0/1677033517 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562b17b43b90 con 0x562b17d414b0 2023-11-06T03:11:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 10 client.7533 handle_command_reply: tid=0 2023-11-06T03:11:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 client.7533 shutdown 2023-11-06T03:11:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 msgr2=0x562b17cfc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b19a0 tx=0x7f4c28004700 comp rx=0 tx=0).stop 2023-11-06T03:11:10.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 shutdown_connections 2023-11-06T03:11:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d414b0 0x562b17d418c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17cfc170 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d400d0 0x562b17cfbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 --2- 172.21.15.157:0/1677033517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d2cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c0ffff640 0 client.7533 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 >> 172.21.15.157:0/1677033517 conn(0x562b17cfc990 msgr2=0x562b17cf4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 shutdown_connections 2023-11-06T03:11:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 1 -- 172.21.15.157:0/1677033517 wait complete. 2023-11-06T03:11:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.595+0000 7f4c3a961000 20 client.7533 trim_cache size 0 max 16384 2023-11-06T03:11:10.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 Processor -- start 2023-11-06T03:11:10.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- start start 2023-11-06T03:11:10.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17d2a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d2a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17802a40 con 0x562b17d409f0 2023-11-06T03:11:10.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17bcdc10 con 0x562b17d400d0 2023-11-06T03:11:10.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b17b033a0 con 0x562b17cf6200 2023-11-06T03:11:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51874/0 (socket says 172.21.15.157:51874) 2023-11-06T03:11:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 -- 172.21.15.157:0/2505314828 learned_addr learned my addr 172.21.15.157:0/2505314828 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17d2a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d2a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 -- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 msgr2=0x562b17d2a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17d2a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 -- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 msgr2=0x562b17d2a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d2a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 -- 172.21.15.157:0/2505314828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b17af04c0 con 0x562b17d409f0 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17d2a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4c280070b0 tx=0x7f4c28011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9dfb24132f0681 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c1b7fe640 1 -- 172.21.15.157:0/2505314828 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800e070 con 0x562b17d409f0 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c1b7fe640 1 -- 172.21.15.157:0/2505314828 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28013b90 con 0x562b17d409f0 2023-11-06T03:11:10.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c1b7fe640 1 -- 172.21.15.157:0/2505314828 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28020660 con 0x562b17d409f0 2023-11-06T03:11:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- 172.21.15.157:0/2505314828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 msgr2=0x562b17d2ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4c280070b0 tx=0x7f4c28011000 comp rx=0 tx=0).stop 2023-11-06T03:11:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- 172.21.15.157:0/2505314828 shutdown_connections 2023-11-06T03:11:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d409f0 0x562b17d2ac60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17d400d0 0x562b17d2a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2505314828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17cf6200 0x562b17d2a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- 172.21.15.157:0/2505314828 >> 172.21.15.157:0/2505314828 conn(0x562b17cfc990 msgr2=0x562b17cfb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- 172.21.15.157:0/2505314828 shutdown_connections 2023-11-06T03:11:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- 172.21.15.157:0/2505314828 wait complete. 2023-11-06T03:11:10.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 -- start start 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.611+0000 7f4c3a961000 1 Processor -- start 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d409f0 0x562b17d52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17cf92c0 con 0x562b17d400d0 2023-11-06T03:11:10.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b17b440d0 con 0x562b17cf6200 2023-11-06T03:11:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b17b3f5e0 con 0x562b17d409f0 2023-11-06T03:11:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 10 client.0 _ll_register_callbacks cb 0x562b17b09c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51886/0 (socket says 172.21.15.157:51886) 2023-11-06T03:11:10.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 -- 172.21.15.157:0/2646687147 learned_addr learned my addr 172.21.15.157:0/2646687147 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c35dd6640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d409f0 0x562b17d52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 -- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d409f0 msgr2=0x562b17d52360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d409f0 0x562b17d52360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 -- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 msgr2=0x562b17d518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 -- 172.21.15.157:0/2646687147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b17b0a170 con 0x562b17d400d0 2023-11-06T03:11:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c34dd4640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4c2005bef0 tx=0x7f4c20001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1e6e95bf653367 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2005c190 con 0x562b17d400d0 2023-11-06T03:11:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c20058270 con 0x562b17d400d0 2023-11-06T03:11:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b17b0e5c0 con 0x562b17d400d0 2023-11-06T03:11:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c20050780 con 0x562b17d400d0 2023-11-06T03:11:10.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 10 client.7539 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562b17ae4030 con 0x562b17d400d0 2023-11-06T03:11:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2006f020 con 0x562b17d400d0 2023-11-06T03:11:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4c2005a070 con 0x562b17d400d0 2023-11-06T03:11:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 10 client.7539 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 10 client.7539 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 20 client.7539 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d528a0 0x562b17d52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 4 client.7539 mds_command: new command op to 7329 tid=0[{"prefix": "perf dump"}] 2023-11-06T03:11:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562b17b3e2c0 con 0x562b17d528a0 2023-11-06T03:11:10.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d528a0 0x562b17d52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 10 client.7539 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c355d5640 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d528a0 0x562b17d52cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43dfff1b3e84c90 server_cookie=2a66b664d106d557 in_seq=0 out_seq=0 2023-11-06T03:11:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 1 -- 172.21.15.157:0/2646687147 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17900 (crc 0 0 0) 0x562b17b3e2c0 con 0x562b17d528a0 2023-11-06T03:11:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 10 client.7539 handle_command_reply: tid=0 2023-11-06T03:11:10.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 client.7539 shutdown 2023-11-06T03:11:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 msgr2=0x562b17d51e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4c2005bef0 tx=0x7f4c20001900 comp rx=0 tx=0).stop 2023-11-06T03:11:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 shutdown_connections 2023-11-06T03:11:10.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x562b17d528a0 0x562b17d52cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562b17d409f0 0x562b17d52360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c0ffff640 0 client.7539 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562b17d400d0 0x562b17d51e20 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 --2- 172.21.15.157:0/2646687147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562b17cf6200 0x562b17d518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 >> 172.21.15.157:0/2646687147 conn(0x562b17cfc990 msgr2=0x562b17d271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 shutdown_connections 2023-11-06T03:11:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 1 -- 172.21.15.157:0/2646687147 wait complete. 2023-11-06T03:11:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c3a961000 20 client.7539 trim_cache size 0 max 16384 2023-11-06T03:11:10.641 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:10.642 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T03:11:10.642 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4527, 2023-11-06T03:11:10.642 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 4528, 2023-11-06T03:11:10.642 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1481827, 2023-11-06T03:11:10.642 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 24040930, 2023-11-06T03:11:10.643 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 9, 2023-11-06T03:11:10.643 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2023-11-06T03:11:10.643 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.530057158, 2023-11-06T03:11:10.643 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.291860162, 2023-11-06T03:11:10.643 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.078863624, 2023-11-06T03:11:10.644 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.134093256, 2023-11-06T03:11:10.644 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:10.644 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4527, 2023-11-06T03:11:10.644 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 14.304550052, 2023-11-06T03:11:10.644 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003159829 2023-11-06T03:11:10.645 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.645 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:10.645 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2, 2023-11-06T03:11:10.645 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000220, 2023-11-06T03:11:10.645 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000110 2023-11-06T03:11:10.646 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.646 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T03:11:10.646 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T03:11:10.646 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.646 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T03:11:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 35563, 2023-11-06T03:11:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 61652, 2023-11-06T03:11:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 12618523, 2023-11-06T03:11:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 52735021, 2023-11-06T03:11:10.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 4.520415350, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.247674328, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 2.645490140, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.142037870, 2023-11-06T03:11:10.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:10.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 61650, 2023-11-06T03:11:10.649 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.456947282, 2023-11-06T03:11:10.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000056073 2023-11-06T03:11:10.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.649 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 87300, 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.019953447, 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000228 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T03:11:10.650 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 118752 2023-11-06T03:11:10.651 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.651 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T03:11:10.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 5711, 2023-11-06T03:11:10.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 5696, 2023-11-06T03:11:10.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1668906, 2023-11-06T03:11:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 47706011, 2023-11-06T03:11:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2023-11-06T03:11:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:11:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.612713001, 2023-11-06T03:11:10.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.267299452, 2023-11-06T03:11:10.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.122548424, 2023-11-06T03:11:10.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.191581475, 2023-11-06T03:11:10.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:10.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 5696, 2023-11-06T03:11:10.653 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.413416101, 2023-11-06T03:11:10.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000423703 2023-11-06T03:11:10.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:10.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.654 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 125744, 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 24432 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.655 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-11-06T03:11:10.656 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-11-06T03:11:10.656 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-11-06T03:11:10.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.656 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-11-06T03:11:10.656 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:11:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:11:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8367, 2023-11-06T03:11:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.227854179, 2023-11-06T03:11:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003015161 2023-11-06T03:11:10.657 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1931, 2023-11-06T03:11:10.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.259507002, 2023-11-06T03:11:10.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000652256 2023-11-06T03:11:10.659 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.659 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-11-06T03:11:10.659 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 28832, 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 28832, 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28832, 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 868.719417668, 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.030130390 2023-11-06T03:11:10.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 55, 2023-11-06T03:11:10.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-11-06T03:11:10.662 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-11-06T03:11:10.662 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 18074, 2023-11-06T03:11:10.662 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 10959, 2023-11-06T03:11:10.662 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 7115, 2023-11-06T03:11:10.662 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2023-11-06T03:11:10.663 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 2417, 2023-11-06T03:11:10.663 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2023-11-06T03:11:10.663 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 418, 2023-11-06T03:11:10.663 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 418, 2023-11-06T03:11:10.663 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2023-11-06T03:11:10.664 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 58480, 2023-11-06T03:11:10.664 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 56635, 2023-11-06T03:11:10.664 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-11-06T03:11:10.664 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-11-06T03:11:10.664 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 1533, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 317149, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-11-06T03:11:10.665 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-11-06T03:11:10.666 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-11-06T03:11:10.666 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-11-06T03:11:10.666 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-11-06T03:11:10.666 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2023-11-06T03:11:10.666 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-11-06T03:11:10.667 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2023-11-06T03:11:10.667 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2023-11-06T03:11:10.667 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2023-11-06T03:11:10.667 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2023-11-06T03:11:10.667 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-11-06T03:11:10.668 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T03:11:10.668 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-11-06T03:11:10.668 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2023-11-06T03:11:10.668 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2023-11-06T03:11:10.668 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 7606, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 6185, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T03:11:10.669 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T03:11:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T03:11:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 7604, 2023-11-06T03:11:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 2000, 2023-11-06T03:11:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 41, 2023-11-06T03:11:10.670 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 17961 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-11-06T03:11:10.671 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-11-06T03:11:10.672 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-11-06T03:11:10.672 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 2000, 2023-11-06T03:11:10.672 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 2, 2023-11-06T03:11:10.672 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-11-06T03:11:10.672 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 4165, 2023-11-06T03:11:10.673 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 1584, 2023-11-06T03:11:10.673 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 4395, 2023-11-06T03:11:10.673 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-11-06T03:11:10.673 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-11-06T03:11:10.673 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-11-06T03:11:10.674 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-11-06T03:11:10.674 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-11-06T03:11:10.674 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-11-06T03:11:10.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2023-11-06T03:11:10.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.675 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 23395, 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 21157, 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 20148, 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 3247, 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-11-06T03:11:10.676 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 1009, 2023-11-06T03:11:10.677 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 29, 2023-11-06T03:11:10.677 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 25, 2023-11-06T03:11:10.677 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 24, 2023-11-06T03:11:10.677 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 5, 2023-11-06T03:11:10.677 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-11-06T03:11:10.678 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 1, 2023-11-06T03:11:10.678 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 71329919, 2023-11-06T03:11:10.678 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 83274198, 2023-11-06T03:11:10.678 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2023-11-06T03:11:10.678 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6815, 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 148.147237264, 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.021738406 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2023-11-06T03:11:10.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-11-06T03:11:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 2014, 2023-11-06T03:11:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2014, 2023-11-06T03:11:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2023-11-06T03:11:10.680 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 13, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 13, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 18074, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 20001, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 1927, 2023-11-06T03:11:10.681 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 418, 2023-11-06T03:11:10.682 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2002, 2023-11-06T03:11:10.682 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 1584, 2023-11-06T03:11:10.682 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 110488, 2023-11-06T03:11:10.682 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207144 2023-11-06T03:11:10.682 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.683 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-11-06T03:11:10.683 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 43793, 2023-11-06T03:11:10.683 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-11-06T03:11:10.683 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 28832, 2023-11-06T03:11:10.683 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 7, 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2000, 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.344503724, 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000672251 2023-11-06T03:11:10.684 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.685 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-11-06T03:11:10.685 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 465, 2023-11-06T03:11:10.685 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.181667025, 2023-11-06T03:11:10.685 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000390681 2023-11-06T03:11:10.685 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.686 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-11-06T03:11:10.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.686 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.686 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6000, 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.526959264, 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001587826 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.687 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-11-06T03:11:10.688 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7959, 2023-11-06T03:11:10.688 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 13.479245887, 2023-11-06T03:11:10.688 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001693585 2023-11-06T03:11:10.688 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.688 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-11-06T03:11:10.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.689 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.689 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.689 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-11-06T03:11:10.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.690 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.690 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-11-06T03:11:10.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.691 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.691 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-11-06T03:11:10.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.692 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.692 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.079398187, 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.026466062 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-11-06T03:11:10.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.694 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.694 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-11-06T03:11:10.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.005653255, 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005653255 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006686124, 2023-11-06T03:11:10.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006686124 2023-11-06T03:11:10.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.697 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-11-06T03:11:10.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.697 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.698 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.698 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-11-06T03:11:10.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6, 2023-11-06T03:11:10.698 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.169247532, 2023-11-06T03:11:10.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.028207922 2023-11-06T03:11:10.699 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.699 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-11-06T03:11:10.699 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4395, 2023-11-06T03:11:10.699 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 814.331511245, 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.185285895 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.700 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.701 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.701 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-11-06T03:11:10.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.701 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.702 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.702 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-11-06T03:11:10.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.702 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-11-06T03:11:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:10.703 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006826015, 2023-11-06T03:11:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006826015 2023-11-06T03:11:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-11-06T03:11:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.705 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.705 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-11-06T03:11:10.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:10.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000316404, 2023-11-06T03:11:10.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000316404 2023-11-06T03:11:10.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-11-06T03:11:10.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-11-06T03:11:10.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-11-06T03:11:10.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-11-06T03:11:10.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-11-06T03:11:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-11-06T03:11:10.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8000, 2023-11-06T03:11:10.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.587403006, 2023-11-06T03:11:10.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003698425 2023-11-06T03:11:10.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.712 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.713 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.714 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-11-06T03:11:10.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2023-11-06T03:11:10.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2023-11-06T03:11:10.714 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2023-11-06T03:11:10.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2023-11-06T03:11:10.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-11-06T03:11:10.715 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 15577, 2023-11-06T03:11:10.715 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 15577, 2023-11-06T03:11:10.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 80, 2023-11-06T03:11:10.716 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2023-11-06T03:11:10.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.716 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-11-06T03:11:10.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-11-06T03:11:10.716 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-11-06T03:11:10.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T03:11:10.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-11-06T03:11:10.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T03:11:10.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-11-06T03:11:10.717 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T03:11:10.718 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T03:11:10.718 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T03:11:10.718 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T03:11:10.718 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T03:11:10.718 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-11-06T03:11:10.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T03:11:10.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T03:11:10.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-11-06T03:11:10.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-11-06T03:11:10.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-11-06T03:11:10.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-11-06T03:11:10.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T03:11:10.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T03:11:10.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-11-06T03:11:10.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T03:11:10.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T03:11:10.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-11-06T03:11:10.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-11-06T03:11:10.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-11-06T03:11:10.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-11-06T03:11:10.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T03:11:10.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-11-06T03:11:10.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T03:11:10.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-11-06T03:11:10.723 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 18190418, 2023-11-06T03:11:10.724 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 4631, 2023-11-06T03:11:10.724 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-11-06T03:11:10.724 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-11-06T03:11:10.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-11-06T03:11:10.724 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 6176, 2023-11-06T03:11:10.725 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 78, 2023-11-06T03:11:10.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T03:11:10.726 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-11-06T03:11:10.726 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-11-06T03:11:10.726 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-11-06T03:11:10.726 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 19241980, 2023-11-06T03:11:10.727 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 792590, 2023-11-06T03:11:10.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-11-06T03:11:10.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-11-06T03:11:10.727 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-11-06T03:11:10.728 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 13534, 2023-11-06T03:11:10.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 89922952, 2023-11-06T03:11:10.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 37, 2023-11-06T03:11:10.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 13534, 2023-11-06T03:11:10.729 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 13534, 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 305.700207333, 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.022587572 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-11-06T03:11:10.730 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-11-06T03:11:10.731 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 13534, 2023-11-06T03:11:10.731 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 23172 2023-11-06T03:11:10.731 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 13534, 2023-11-06T03:11:10.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 90, 2023-11-06T03:11:10.732 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 13444, 2023-11-06T03:11:10.732 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-11-06T03:11:10.732 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-11-06T03:11:10.732 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 42, 2023-11-06T03:11:10.732 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 4748, 2023-11-06T03:11:10.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 90, 2023-11-06T03:11:10.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 6905, 2023-11-06T03:11:10.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 73, 2023-11-06T03:11:10.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-11-06T03:11:10.733 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-11-06T03:11:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 1584, 2023-11-06T03:11:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-11-06T03:11:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 46, 2023-11-06T03:11:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-11-06T03:11:10.734 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-11-06T03:11:10.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-11-06T03:11:10.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 0, 2023-11-06T03:11:10.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 9496, 2023-11-06T03:11:10.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-11-06T03:11:10.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-11-06T03:11:10.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-11-06T03:11:10.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-11-06T03:11:10.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-11-06T03:11:10.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-11-06T03:11:10.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-11-06T03:11:10.737 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-11-06T03:11:10.738 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-11-06T03:11:10.738 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-11-06T03:11:10.738 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-11-06T03:11:10.738 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-11-06T03:11:10.738 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-11-06T03:11:10.739 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-11-06T03:11:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-11-06T03:11:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-11-06T03:11:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 85, 2023-11-06T03:11:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-11-06T03:11:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 1, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 8, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 8, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 152, 2023-11-06T03:11:10.741 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2023-11-06T03:11:10.742 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 36 2023-11-06T03:11:10.742 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.742 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-11-06T03:11:10.742 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2023-11-06T03:11:10.742 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-11-06T03:11:10.743 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-11-06T03:11:10.743 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-11-06T03:11:10.743 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.743 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-11-06T03:11:10.743 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 28832 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 32, 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-11-06T03:11:10.744 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 32, 2023-11-06T03:11:10.745 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 1584, 2023-11-06T03:11:10.745 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-11-06T03:11:10.745 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.745 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T03:11:10.746 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:10.746 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:11:10.746 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:10.746 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 45797, 2023-11-06T03:11:10.746 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 11641805, 2023-11-06T03:11:10.747 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:10.747 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 45797, 2023-11-06T03:11:10.747 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:10.747 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:10.747 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 45797, 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 11641805, 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.748 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:10.749 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:11:10.750 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:11:10.750 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:10.753 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:11:10.754 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 13534, 2023-11-06T03:11:10.754 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 90095608, 2023-11-06T03:11:10.754 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 13534, 2023-11-06T03:11:10.754 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 90095608, 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.755 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.756 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-11-06T03:11:10.756 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:10.756 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-11-06T03:11:10.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:10.756 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:11:10.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:11:10.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:10.757 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:11:10.757 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 13534, 2023-11-06T03:11:10.757 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 13534, 2023-11-06T03:11:10.758 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 13534, 2023-11-06T03:11:10.758 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 13534, 2023-11-06T03:11:10.758 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:10.758 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.758 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-11-06T03:11:10.759 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 1584, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 172656, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:10.760 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 1584, 2023-11-06T03:11:10.761 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:10.761 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:10.761 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 90, 2023-11-06T03:11:10.761 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 172656, 2023-11-06T03:11:10.761 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:10.762 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.762 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.762 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.762 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.763 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:10.763 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55d928f2ed20": { 2023-11-06T03:11:10.763 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 84, 2023-11-06T03:11:10.763 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:11:10.763 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:10.764 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 23395, 2023-11-06T03:11:10.764 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 79079894, 2023-11-06T03:11:10.764 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:10.764 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 23395, 2023-11-06T03:11:10.764 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6815, 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 79079810, 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:10.765 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:10.766 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:10.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 msgr2=0x7f4c14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001750 tx=0x7f4c2c0011a0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 msgr2=0x7f4c30142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000d7a0 tx=0x7f4c2000dc60 comp rx=0 tx=0).stop 2023-11-06T03:11:10.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 shutdown_connections 2023-11-06T03:11:10.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c300ce450 0x7f4c30143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c300cdfd0 0x7f4c30142df0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 --2- 172.21.15.157:0/368049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c1c003980 0x7f4c30151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 >> 172.21.15.157:0/368049 conn(0x7f4c30065930 msgr2=0x7f4c300afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 shutdown_connections 2023-11-06T03:11:10.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.615+0000 7f4c37839640 1 -- 172.21.15.157:0/368049 wait complete. 2023-11-06T03:11:10.771 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2023-11-06T03:11:10.771 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2023-11-06T03:11:10.775 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:11:10.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:11:10.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 Processor -- start 2023-11-06T03:11:10.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 -- start start 2023-11-06T03:11:10.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17c80cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17c80b15b0 0x7f17c80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c80d0410 con 0x7f17c815cc70 2023-11-06T03:11:10.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c80d27c0 con 0x7f17c80b15b0 2023-11-06T03:11:10.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17ceb14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80d0f90 con 0x7f17c80cfa90 2023-11-06T03:11:10.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cc8b0640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f17c815cc70 0x7f17c80d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35944/0 (socket says 172.21.15.157:35944) 2023-11-06T03:11:10.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cc8b0640 1 -- 172.21.15.157:0/2896553614 learned_addr learned my addr 172.21.15.157:0/2896553614 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:10.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c7fff640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17c80cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2896553614 >> v2:172.21.15.195:3300/0 conn(0x7f17c80b15b0 0x7f17c80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244521543 0 0) 0x7f17c80d0410 con 0x7f17c815cc70 2023-11-06T03:11:10.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b4003530 con 0x7f17c815cc70 2023-11-06T03:11:10.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672923668 0 0) 0x7f17b4003530 con 0x7f17c815cc70 2023-11-06T03:11:10.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 msgr2=0x7f17c80cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17c80cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 >> v2:172.21.15.195:3300/0 conn(0x7f17c80b15b0 msgr2=0x7f17c80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 --2- 172.21.15.157:0/2896553614 >> v2:172.21.15.195:3300/0 conn(0x7f17c80b15b0 0x7f17c80b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80d1270 con 0x7f17c815cc70 2023-11-06T03:11:10.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f17c0001a00 con 0x7f17c815cc70 2023-11-06T03:11:10.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 >> v1:172.21.15.157:6789/0 conn(0x7f17c815cc70 legacy=0x7f17c80d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4003980 0x7f17b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b4004d10 0x7f17b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80d27c0 con 0x7f17b4004d10 2023-11-06T03:11:10.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80d0f90 con 0x7f17b4003980 2023-11-06T03:11:10.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b400b990 con 0x7f17c80cfa90 2023-11-06T03:11:10.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b4004d10 0x7f17b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4003980 0x7f17b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17c7fff640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 -- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 msgr2=0x7f17b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c80cfa90 0x7f17b40047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 -- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4003980 msgr2=0x7f17b4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4003980 0x7f17b4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.939+0000 7f17cd0b1640 1 -- 172.21.15.157:0/2896553614 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4003530 con 0x7f17b4004d10 2023-11-06T03:11:10.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b4004d10 0x7f17b4005190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009ae0 tx=0x7f17b801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247e1e4815f4418b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80a5680 con 0x7f17b4004d10 2023-11-06T03:11:10.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b80a57e0 con 0x7f17b4004d10 2023-11-06T03:11:10.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17c77fe640 1 -- 172.21.15.157:0/2896553614 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80a5b90 con 0x7f17b4004d10 2023-11-06T03:11:10.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- 172.21.15.157:0/2896553614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b4004d10 msgr2=0x7f17b4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2896553614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17b4004d10 0x7f17b4005190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009ae0 tx=0x7f17b801ca90 comp rx=0 tx=0).stop 2023-11-06T03:11:10.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17cd0b1640 1 -- 172.21.15.157:0/2896553614 reap_dead start 2023-11-06T03:11:10.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- 172.21.15.157:0/2896553614 shutdown_connections 2023-11-06T03:11:10.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- 172.21.15.157:0/2896553614 >> 172.21.15.157:0/2896553614 conn(0x7f17c8070790 msgr2=0x7f17c8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- 172.21.15.157:0/2896553614 shutdown_connections 2023-11-06T03:11:10.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- 172.21.15.157:0/2896553614 wait complete. 2023-11-06T03:11:10.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 Processor -- start 2023-11-06T03:11:10.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- start start 2023-11-06T03:11:10.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 0x7f17c80ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c81f6780 0x7f17c80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c814dab0 con 0x7f17c815d9d0 2023-11-06T03:11:10.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80c9f80 con 0x7f17b4004d10 2023-11-06T03:11:10.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.943+0000 7f17ceb14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80cad60 con 0x7f17c81f6780 2023-11-06T03:11:10.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51904/0 (socket says 172.21.15.157:51904) 2023-11-06T03:11:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 -- 172.21.15.157:0/2051933946 learned_addr learned my addr 172.21.15.157:0/2051933946 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 0x7f17c80ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c81f6780 0x7f17c80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 -- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c81f6780 msgr2=0x7f17c80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c81f6780 0x7f17c80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 -- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 msgr2=0x7f17c80ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 0x7f17c80ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 -- 172.21.15.157:0/2051933946 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c815df20 con 0x7f17c815d9d0 2023-11-06T03:11:10.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 0x7f17c80ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c7fff640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f17b00108c0 tx=0x7f17b0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 -- 172.21.15.157:0/2051933946 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0009630 con 0x7f17c815d9d0 2023-11-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 -- 172.21.15.157:0/2051933946 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000eb60 con 0x7f17c815d9d0 2023-11-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c81f5ca0 con 0x7f17c815d9d0 2023-11-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 -- 172.21.15.157:0/2051933946 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b001c400 con 0x7f17c815d9d0 2023-11-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81f6260 con 0x7f17c815d9d0 2023-11-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 -- 172.21.15.157:0/2051933946 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f17b001c560 con 0x7f17c815d9d0 2023-11-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 0x7f17a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17c57fa640 1 -- 172.21.15.157:0/2051933946 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17b0083810 con 0x7f17c815d9d0 2023-11-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 0x7f17a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17d1c3c000 1 Processor -- start 2023-11-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17d1c3c000 1 -- start start 2023-11-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.947+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 0x7f17a8067f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f17c0002240 tx=0x7f17c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339cbb0 0x5602d339cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d3352d00 0x5602d33530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d321ad10 con 0x5602d339d4d0 2023-11-06T03:11:10.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d31a82b0 con 0x5602d339cbb0 2023-11-06T03:11:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602d31a5880 con 0x5602d3352d00 2023-11-06T03:11:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51908/0 (socket says 172.21.15.157:51908) 2023-11-06T03:11:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 -- 172.21.15.157:0/1494202840 learned_addr learned my addr 172.21.15.157:0/1494202840 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339cbb0 0x5602d339cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d3352d00 0x5602d33530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 -- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d3352d00 msgr2=0x5602d33530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d3352d00 0x5602d33530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 -- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339cbb0 msgr2=0x5602d339cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339cbb0 0x5602d339cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 -- 172.21.15.157:0/1494202840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602d2e9e000 con 0x5602d339d4d0 2023-11-06T03:11:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f17b0081c90 tx=0x7f17b0054e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c62548265b583e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17be7fc640 1 -- 172.21.15.157:0/1494202840 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b006c070 con 0x5602d339d4d0 2023-11-06T03:11:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17be7fc640 1 -- 172.21.15.157:0/1494202840 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b005cda0 con 0x5602d339d4d0 2023-11-06T03:11:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17be7fc640 1 -- 172.21.15.157:0/1494202840 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0051580 con 0x5602d339d4d0 2023-11-06T03:11:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1494202840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 msgr2=0x5602d33527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f17b0081c90 tx=0x7f17b0054e70 comp rx=0 tx=0).stop 2023-11-06T03:11:10.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1494202840 shutdown_connections 2023-11-06T03:11:10.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d3352d00 0x5602d33530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d33527c0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1494202840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339cbb0 0x5602d339cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1494202840 >> 172.21.15.157:0/1494202840 conn(0x5602d3359280 msgr2=0x5602d33510a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1494202840 shutdown_connections 2023-11-06T03:11:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1494202840 wait complete. 2023-11-06T03:11:10.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- start start 2023-11-06T03:11:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 Processor -- start 2023-11-06T03:11:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d31a1170 con 0x5602d339d4d0 2023-11-06T03:11:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d3141060 con 0x5602d3352d00 2023-11-06T03:11:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602d3140d00 con 0x5602d339cbb0 2023-11-06T03:11:10.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17d1c3c000 10 client.0 _ll_register_callbacks cb 0x5602d333e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51924/0 (socket says 172.21.15.157:51924) 2023-11-06T03:11:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 -- 172.21.15.157:0/67162856 learned_addr learned my addr 172.21.15.157:0/67162856 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17c7fff640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 -- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 msgr2=0x5602d33a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 -- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 msgr2=0x5602d33a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:10.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.951+0000 7f17cd0b1640 1 -- 172.21.15.157:0/67162856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81f6260 con 0x5602d339d4d0 2023-11-06T03:11:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009ae0 tx=0x7f17b8019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2055dea149f9d9e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:10.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80a5680 con 0x5602d339d4d0 2023-11-06T03:11:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17b80aec50 con 0x5602d339d4d0 2023-11-06T03:11:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602d3150aa0 con 0x5602d339d4d0 2023-11-06T03:11:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b80aee10 con 0x5602d339d4d0 2023-11-06T03:11:10.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 10 client.7551 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:10.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5602d3166020 con 0x5602d339d4d0 2023-11-06T03:11:10.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602d3150aa0 con 0x5602d339d4d0 2023-11-06T03:11:10.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f17b80ad440 con 0x5602d339d4d0 2023-11-06T03:11:10.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 10 client.7551 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:10.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 10 client.7551 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:10.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 20 client.7551 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:10.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d338b910 0x5602d338bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:10.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 4 client.7551 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:11:10.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602d319fac0 con 0x5602d338b910 2023-11-06T03:11:10.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d338b910 0x5602d338bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:10.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 10 client.7551 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d338b910 0x5602d338bcf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b74d010b66785c server_cookie=6352ef00264a965 in_seq=0 out_seq=0 2023-11-06T03:11:10.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 1 -- 172.21.15.157:0/67162856 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5602d319fac0 con 0x5602d338b910 2023-11-06T03:11:10.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 10 client.7551 handle_command_reply: tid=0 2023-11-06T03:11:10.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 client.7551 shutdown 2023-11-06T03:11:10.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 msgr2=0x5602d338ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:10.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009ae0 tx=0x7f17b8019830 comp rx=0 tx=0).stop 2023-11-06T03:11:10.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 shutdown_connections 2023-11-06T03:11:10.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d338b910 0x5602d338bcf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d338ab70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17a77fe640 0 client.7551 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:10.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/67162856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:10.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 >> 172.21.15.157:0/67162856 conn(0x5602d3359280 msgr2=0x5602d33517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:10.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 shutdown_connections 2023-11-06T03:11:10.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 1 -- 172.21.15.157:0/67162856 wait complete. 2023-11-06T03:11:10.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.955+0000 7f17d1c3c000 20 client.7551 trim_cache size 0 max 16384 2023-11-06T03:11:11.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 Processor -- start 2023-11-06T03:11:11.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- start start 2023-11-06T03:11:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339d4d0 0x5602d33f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d339c770 con 0x5602d3352d00 2023-11-06T03:11:11.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d2e5ec80 con 0x5602d339d4d0 2023-11-06T03:11:11.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602d31a5170 con 0x5602d339cbb0 2023-11-06T03:11:11.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51936/0 (socket says 172.21.15.157:51936) 2023-11-06T03:11:11.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 -- 172.21.15.157:0/1389359057 learned_addr learned my addr 172.21.15.157:0/1389359057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 -- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 msgr2=0x5602d33f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 -- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339d4d0 msgr2=0x5602d33f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339d4d0 0x5602d33f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 -- 172.21.15.157:0/1389359057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602d314c370 con 0x5602d3352d00 2023-11-06T03:11:11.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f17c00154f0 tx=0x7f17c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c512fbe5ae7ffeb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17beffd640 1 -- 172.21.15.157:0/1389359057 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0015e60 con 0x5602d3352d00 2023-11-06T03:11:11.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17beffd640 1 -- 172.21.15.157:0/1389359057 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c00270b0 con 0x5602d3352d00 2023-11-06T03:11:11.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17beffd640 1 -- 172.21.15.157:0/1389359057 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0027420 con 0x5602d3352d00 2023-11-06T03:11:11.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1389359057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 msgr2=0x5602d33f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f17c00154f0 tx=0x7f17c0013000 comp rx=0 tx=0).stop 2023-11-06T03:11:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1389359057 shutdown_connections 2023-11-06T03:11:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d339d4d0 0x5602d33f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/1389359057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d3352d00 0x5602d33f5040 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1389359057 >> 172.21.15.157:0/1389359057 conn(0x5602d3359280 msgr2=0x5602d3387cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1389359057 shutdown_connections 2023-11-06T03:11:11.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- 172.21.15.157:0/1389359057 wait complete. 2023-11-06T03:11:11.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 -- start start 2023-11-06T03:11:11.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.971+0000 7f17d1c3c000 1 Processor -- start 2023-11-06T03:11:11.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d315b1b0 con 0x5602d339d4d0 2023-11-06T03:11:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602d33561e0 con 0x5602d3352d00 2023-11-06T03:11:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602d31a40d0 con 0x5602d339cbb0 2023-11-06T03:11:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 10 client.0 _ll_register_callbacks cb 0x5602d31a42a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:11.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51948/0 (socket says 172.21.15.157:51948) 2023-11-06T03:11:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 -- 172.21.15.157:0/709953852 learned_addr learned my addr 172.21.15.157:0/709953852 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 -- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 msgr2=0x5602d33fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17c7fff640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 -- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 msgr2=0x5602d33fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 -- 172.21.15.157:0/709953852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c001a040 con 0x5602d339d4d0 2023-11-06T03:11:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cd0b1640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f17b801c8c0 tx=0x7f17b4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd499d33e2bce2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80a5680 con 0x5602d339d4d0 2023-11-06T03:11:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17b80ad900 con 0x5602d339d4d0 2023-11-06T03:11:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602d315e4e0 con 0x5602d339d4d0 2023-11-06T03:11:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b800e070 con 0x5602d339d4d0 2023-11-06T03:11:11.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 10 client.7557 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:11.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5602d31400a0 con 0x5602d339d4d0 2023-11-06T03:11:11.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602d315e4e0 con 0x5602d339d4d0 2023-11-06T03:11:11.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f17b80b69f0 con 0x5602d339d4d0 2023-11-06T03:11:11.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 10 client.7557 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:11.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 10 client.7557 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:11.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 20 client.7557 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:11.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d339f2a0 0x5602d339f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 4 client.7557 mds_command: new command op to 7329 tid=0[{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2023-11-06T03:11:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x5602d319a1f0 con 0x5602d339f2a0 2023-11-06T03:11:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d339f2a0 0x5602d339f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17cc8b0640 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d339f2a0 0x5602d339f6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b678b9be89ddd7 server_cookie=c13aab75e0b60728 in_seq=0 out_seq=0 2023-11-06T03:11:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.975+0000 7f17a77fe640 10 client.7557 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17a77fe640 1 -- 172.21.15.157:0/709953852 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5602d319a1f0 con 0x5602d339f2a0 2023-11-06T03:11:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17a77fe640 10 client.7557 handle_command_reply: tid=0 2023-11-06T03:11:11.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 client.7557 shutdown 2023-11-06T03:11:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 msgr2=0x5602d339ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f17b801c8c0 tx=0x7f17b4005800 comp rx=0 tx=0).stop 2023-11-06T03:11:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 shutdown_connections 2023-11-06T03:11:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x5602d339f2a0 0x5602d339f6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5602d339d4d0 0x5602d339ed60 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5602d339cbb0 0x5602d33fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17d1c3c000 1 --2- 172.21.15.157:0/709953852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5602d3352d00 0x5602d33fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.979+0000 7f17a77fe640 0 client.7557 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 >> 172.21.15.157:0/709953852 conn(0x5602d3359280 msgr2=0x5602d3387cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 shutdown_connections 2023-11-06T03:11:11.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17d1c3c000 1 -- 172.21.15.157:0/709953852 wait complete. 2023-11-06T03:11:11.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17d1c3c000 20 client.7557 trim_cache size 0 max 16384 2023-11-06T03:11:11.029 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:11.030 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:11:11.030 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "c717ed31-c4f9-4b2a-9a6b-2344b908f24e", 2023-11-06T03:11:11.030 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:11:11.031 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:11.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 msgr2=0x7f17a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 0x7f17a8067f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f17c0002240 tx=0x7f17c0009040 comp rx=0 tx=0).stop 2023-11-06T03:11:11.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 msgr2=0x7f17c815fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f17b00108c0 tx=0x7f17b0010d80 comp rx=0 tx=0).stop 2023-11-06T03:11:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 shutdown_connections 2023-11-06T03:11:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f17a8065a60 0x7f17a8067f20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17c81f6780 0x7f17c80cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17c815d9d0 0x7f17c815fb30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 --2- 172.21.15.157:0/2051933946 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17b4004d10 0x7f17c80ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 >> 172.21.15.157:0/2051933946 conn(0x7f17c8070790 msgr2=0x7f17c815c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 shutdown_connections 2023-11-06T03:11:11.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:10.983+0000 7f17ceb14640 1 -- 172.21.15.157:0/2051933946 wait complete. 2023-11-06T03:11:11.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-11-06T03:11:11.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 Processor -- start 2023-11-06T03:11:11.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 -- start start 2023-11-06T03:11:11.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c40cc380 0x7f57c4066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c415b620 con 0x7f57c40b1e40 2023-11-06T03:11:11.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c415aff0 con 0x7f57c41585d0 2023-11-06T03:11:11.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57cac59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c406dd60 con 0x7f57c40cc380 2023-11-06T03:11:11.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c40cc380 0x7f57c4066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c40cc380 0x7f57c4066340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51964/0 (socket says 172.21.15.157:51964) 2023-11-06T03:11:11.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 -- 172.21.15.157:0/2784163110 learned_addr learned my addr 172.21.15.157:0/2784163110 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c3fff640 1 --1- 172.21.15.157:0/2784163110 >> v1:172.21.15.157:6789/0 conn(0x7f57c40b1e40 0x7f57c40cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35952/0 (socket says 172.21.15.157:35952) 2023-11-06T03:11:11.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 -- 172.21.15.157:0/2784163110 >> v1:172.21.15.195:6789/0 conn(0x7f57c41585d0 legacy=0x7f57c40b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 -- 172.21.15.157:0/2784163110 >> v1:172.21.15.157:6789/0 conn(0x7f57c40b1e40 legacy=0x7f57c40cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.195+0000 7f57c91f6640 1 -- 172.21.15.157:0/2784163110 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c406e040 con 0x7f57c40cc380 2023-11-06T03:11:11.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57c91f6640 1 --2- 172.21.15.157:0/2784163110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40cc380 0x7f57c4066340 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009e50 tx=0x7f57bc01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d965792e445b426d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57c37fe640 1 -- 172.21.15.157:0/2784163110 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0a58d0 con 0x7f57c40cc380 2023-11-06T03:11:11.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57c37fe640 1 -- 172.21.15.157:0/2784163110 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc0a5a30 con 0x7f57c40cc380 2023-11-06T03:11:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57c37fe640 1 -- 172.21.15.157:0/2784163110 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0a5dc0 con 0x7f57c40cc380 2023-11-06T03:11:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 -- 172.21.15.157:0/2784163110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40cc380 msgr2=0x7f57c4066340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 --2- 172.21.15.157:0/2784163110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40cc380 0x7f57c4066340 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009e50 tx=0x7f57bc01c9e0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 -- 172.21.15.157:0/2784163110 shutdown_connections 2023-11-06T03:11:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 --2- 172.21.15.157:0/2784163110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40cc380 0x7f57c4066340 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 -- 172.21.15.157:0/2784163110 >> 172.21.15.157:0/2784163110 conn(0x7f57c40732f0 msgr2=0x7f57c4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 -- 172.21.15.157:0/2784163110 shutdown_connections 2023-11-06T03:11:11.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.199+0000 7f57cac59640 1 -- 172.21.15.157:0/2784163110 wait complete. 2023-11-06T03:11:11.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 Processor -- start 2023-11-06T03:11:11.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- start start 2023-11-06T03:11:11.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c40cc380 0x7f57c4156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c41585d0 0x7f57c4152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4146360 con 0x7f57c40b1e40 2023-11-06T03:11:11.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c414f5d0 con 0x7f57c41585d0 2023-11-06T03:11:11.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c414f2f0 con 0x7f57c40cc380 2023-11-06T03:11:11.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51974/0 (socket says 172.21.15.157:51974) 2023-11-06T03:11:11.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 -- 172.21.15.157:0/518496621 learned_addr learned my addr 172.21.15.157:0/518496621 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c3fff640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c40cc380 0x7f57c4156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c91f6640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c41585d0 0x7f57c4152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 -- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c40cc380 msgr2=0x7f57c4156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c40cc380 0x7f57c4156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 -- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c41585d0 msgr2=0x7f57c4152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c41585d0 0x7f57c4152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 -- 172.21.15.157:0/518496621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009b30 con 0x7f57c40b1e40 2023-11-06T03:11:11.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c89f5640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f57b8001a80 tx=0x7f57b8014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c17fa640 1 -- 172.21.15.157:0/518496621 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8017070 con 0x7f57c40b1e40 2023-11-06T03:11:11.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c17fa640 1 -- 172.21.15.157:0/518496621 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8014b30 con 0x7f57c40b1e40 2023-11-06T03:11:11.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57c17fa640 1 -- 172.21.15.157:0/518496621 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80050d0 con 0x7f57c40b1e40 2023-11-06T03:11:11.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c4155a60 con 0x7f57c40b1e40 2023-11-06T03:11:11.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.203+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4156020 con 0x7f57c40b1e40 2023-11-06T03:11:11.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.207+0000 7f57c17fa640 1 -- 172.21.15.157:0/518496621 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57b8005230 con 0x7f57c40b1e40 2023-11-06T03:11:11.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.207+0000 7f57c17fa640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 0x7f57a8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.207+0000 7f57c17fa640 1 -- 172.21.15.157:0/518496621 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57b808a620 con 0x7f57c40b1e40 2023-11-06T03:11:11.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.207+0000 7f57cdd81000 1 Processor -- start 2023-11-06T03:11:11.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.207+0000 7f57cdd81000 1 -- start start 2023-11-06T03:11:11.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c3fff640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 0x7f57a8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ec99d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c3fff640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 0x7f57a8067f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f57c41f6cc0 tx=0x7f57b0007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028ec9a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eb61c10 con 0x56028ece4130 2023-11-06T03:11:11.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eaf03e0 con 0x56028ece4a50 2023-11-06T03:11:11.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56028eaed9b0 con 0x56028ec9a260 2023-11-06T03:11:11.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ec99d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51990/0 (socket says 172.21.15.157:51990) 2023-11-06T03:11:11.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 -- 172.21.15.157:0/4137320623 learned_addr learned my addr 172.21.15.157:0/4137320623 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c91f6640 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028ec9a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 -- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 msgr2=0x56028ec9a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028ec9a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 -- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 msgr2=0x56028ec99d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ec99d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 -- 172.21.15.157:0/4137320623 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028e7e5e20 con 0x56028ece4130 2023-11-06T03:11:11.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57c89f5640 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f57b8058170 tx=0x7f57b8001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26250dfffce0b66d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57ae7fc640 1 -- 172.21.15.157:0/4137320623 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80731c0 con 0x56028ece4130 2023-11-06T03:11:11.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57ae7fc640 1 -- 172.21.15.157:0/4137320623 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b805bc70 con 0x56028ece4130 2023-11-06T03:11:11.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57ae7fc640 1 -- 172.21.15.157:0/4137320623 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b805f9f0 con 0x56028ece4130 2023-11-06T03:11:11.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 -- 172.21.15.157:0/4137320623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 msgr2=0x56028ece4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.211+0000 7f57cdd81000 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f57b8058170 tx=0x7f57b8001090 comp rx=0 tx=0).stop 2023-11-06T03:11:11.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 -- 172.21.15.157:0/4137320623 shutdown_connections 2023-11-06T03:11:11.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028ec9a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ec99d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 --2- 172.21.15.157:0/4137320623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4130 0x56028ece4510 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 -- 172.21.15.157:0/4137320623 >> 172.21.15.157:0/4137320623 conn(0x56028eca09f0 msgr2=0x56028ec98600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 -- 172.21.15.157:0/4137320623 shutdown_connections 2023-11-06T03:11:11.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 -- 172.21.15.157:0/4137320623 wait complete. 2023-11-06T03:11:11.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 -- start start 2023-11-06T03:11:11.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 Processor -- start 2023-11-06T03:11:11.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028eceb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.215+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 0x56028eceba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eae92a0 con 0x56028ece4a50 2023-11-06T03:11:11.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028ea88470 con 0x56028ece4130 2023-11-06T03:11:11.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 0x56028eceba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 0x56028eceba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:50442/0 (socket says 172.21.15.157:50442) 2023-11-06T03:11:11.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51994/0 (socket says 172.21.15.157:51994) 2023-11-06T03:11:11.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c3fff640 1 -- 172.21.15.157:0/2312482007 learned_addr learned my addr 172.21.15.157:0/2312482007 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c89f5640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028eceb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56028eaa6660 con 0x56028ec9a260 2023-11-06T03:11:11.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 10 client.0 _ll_register_callbacks cb 0x56028ec85c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:11.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 -- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 msgr2=0x56028eceb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028eceb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 -- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 msgr2=0x56028eceba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 0x56028eceba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 -- 172.21.15.157:0/2312482007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4156020 con 0x56028ece4a50 2023-11-06T03:11:11.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c91f6640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ab000 tx=0x7f57bc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f6a3571ae4781e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57a37fe640 1 -- 172.21.15.157:0/2312482007 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0b64b0 con 0x56028ece4a50 2023-11-06T03:11:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57a37fe640 1 -- 172.21.15.157:0/2312482007 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57bc0c2c30 con 0x56028ece4a50 2023-11-06T03:11:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56028eaa60a0 con 0x56028ece4a50 2023-11-06T03:11:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57a37fe640 1 -- 172.21.15.157:0/2312482007 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57bc0c2e10 con 0x56028ece4a50 2023-11-06T03:11:11.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 10 client.7572 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:11.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({fsmap=70}) v3 -- 0x56028eaae1d0 con 0x56028ece4a50 2023-11-06T03:11:11.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57a37fe640 1 -- 172.21.15.157:0/2312482007 <== mon.0 v2:172.21.15.157:3300/0 4 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f57bc0b6610 con 0x56028ece4a50 2023-11-06T03:11:11.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 10 client.7572 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:11.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 10 client.7572 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:11.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 20 client.7572 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:11.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ece5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 4 client.7572 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:11:11.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56028eaa60a0 con 0x56028ece5b70 2023-11-06T03:11:11.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c89f5640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ece5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.219+0000 7f57c89f5640 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ece5f80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933ff0c2c8b1c3f4 server_cookie=4c84704063bac93b in_seq=0 out_seq=0 2023-11-06T03:11:11.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57a37fe640 10 client.7572 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57a37fe640 1 -- 172.21.15.157:0/2312482007 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56028eaa60a0 con 0x56028ece5b70 2023-11-06T03:11:11.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57a37fe640 10 client.7572 handle_command_reply: tid=0 2023-11-06T03:11:11.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 client.7572 shutdown 2023-11-06T03:11:11.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 msgr2=0x56028ece4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ab000 tx=0x7f57bc004a50 comp rx=0 tx=0).stop 2023-11-06T03:11:11.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 shutdown_connections 2023-11-06T03:11:11.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ece5f80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ece4a50 0x56028ece4e30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57a37fe640 0 client.7572 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4130 0x56028eceba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 --2- 172.21.15.157:0/2312482007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ec9a260 0x56028eceb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 >> 172.21.15.157:0/2312482007 conn(0x56028eca09f0 msgr2=0x56028ec98d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 shutdown_connections 2023-11-06T03:11:11.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 1 -- 172.21.15.157:0/2312482007 wait complete. 2023-11-06T03:11:11.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.223+0000 7f57cdd81000 20 client.7572 trim_cache size 0 max 16384 2023-11-06T03:11:11.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 Processor -- start 2023-11-06T03:11:11.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- start start 2023-11-06T03:11:11.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ece94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ece9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028e7a6aa0 con 0x56028ec9a260 2023-11-06T03:11:11.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eaebe00 con 0x56028ece4a50 2023-11-06T03:11:11.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56028ecfa970 con 0x56028ece4130 2023-11-06T03:11:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51996/0 (socket says 172.21.15.157:51996) 2023-11-06T03:11:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 -- 172.21.15.157:0/1412600868 learned_addr learned my addr 172.21.15.157:0/1412600868 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c3fff640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ece94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c91f6640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ece9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 -- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 msgr2=0x56028ece94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ece94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 -- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 msgr2=0x56028ece9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ece9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 -- 172.21.15.157:0/1412600868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56028ea94520 con 0x56028ec9a260 2023-11-06T03:11:11.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c91f6640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ece9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:11.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57c89f5640 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f57b8058170 tx=0x7f57b805f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e203684f163f84 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57aeffd640 1 -- 172.21.15.157:0/1412600868 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8063ba0 con 0x56028ec9a260 2023-11-06T03:11:11.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57aeffd640 1 -- 172.21.15.157:0/1412600868 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b805bc70 con 0x56028ec9a260 2023-11-06T03:11:11.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57aeffd640 1 -- 172.21.15.157:0/1412600868 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b805f9f0 con 0x56028ec9a260 2023-11-06T03:11:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- 172.21.15.157:0/1412600868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 msgr2=0x56028ece8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f57b8058170 tx=0x7f57b805f000 comp rx=0 tx=0).stop 2023-11-06T03:11:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- 172.21.15.157:0/1412600868 shutdown_connections 2023-11-06T03:11:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ece9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ece94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 --2- 172.21.15.157:0/1412600868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ece8fa0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- 172.21.15.157:0/1412600868 >> 172.21.15.157:0/1412600868 conn(0x56028eca09f0 msgr2=0x56028ec9fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- 172.21.15.157:0/1412600868 shutdown_connections 2023-11-06T03:11:11.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.239+0000 7f57cdd81000 1 -- 172.21.15.157:0/1412600868 wait complete. 2023-11-06T03:11:11.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- start start 2023-11-06T03:11:11.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 Processor -- start 2023-11-06T03:11:11.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ed25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ed257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eae4c10 con 0x56028ec9a260 2023-11-06T03:11:11.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56028eae66a0 con 0x56028ece4a50 2023-11-06T03:11:11.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56028ecf15c0 con 0x56028ece4130 2023-11-06T03:11:11.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 10 client.0 _ll_register_callbacks cb 0x56028eaadcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:11.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52008/0 (socket says 172.21.15.157:52008) 2023-11-06T03:11:11.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 -- 172.21.15.157:0/520804042 learned_addr learned my addr 172.21.15.157:0/520804042 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:11.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c91f6640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ed257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c3fff640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ed25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 -- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 msgr2=0x56028ed25270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ed25270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 -- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 msgr2=0x56028ed257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:11.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ed257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 -- 172.21.15.157:0/520804042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56028e7e5e20 con 0x56028ec9a260 2023-11-06T03:11:11.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c89f5640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f57b8063670 tx=0x7f57b8088940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b5b64e98edfe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:11.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:11.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8063ba0 con 0x56028ec9a260 2023-11-06T03:11:11.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57b8058380 con 0x56028ec9a260 2023-11-06T03:11:11.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56028eab2620 con 0x56028ec9a260 2023-11-06T03:11:11.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b8058560 con 0x56028ec9a260 2023-11-06T03:11:11.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 10 client.7578 fetch_fsmap learned FSMap version 70 2023-11-06T03:11:11.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x56028ea88090 con 0x56028ec9a260 2023-11-06T03:11:11.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8068080 con 0x56028ec9a260 2023-11-06T03:11:11.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 70) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f57b8023060 con 0x56028ec9a260 2023-11-06T03:11:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 10 client.7578 fetch_fsmap finished waiting for FSMap version 70 2023-11-06T03:11:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 10 client.7578 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 20 client.7578 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ed25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 4 client.7578 mds_command: new command op to 7329 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:11:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56028eae7bf0 con 0x56028ece5b70 2023-11-06T03:11:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c3fff640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ed25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 10 client.7578 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57c3fff640 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ed25cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315809bbd06eba42 server_cookie=410a773aa80aafb9 in_seq=0 out_seq=0 2023-11-06T03:11:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 1 -- 172.21.15.157:0/520804042 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x56028eae7bf0 con 0x56028ece5b70 2023-11-06T03:11:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 10 client.7578 handle_command_reply: tid=0 2023-11-06T03:11:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 client.7578 shutdown 2023-11-06T03:11:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 msgr2=0x56028ed24d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f57b8063670 tx=0x7f57b8088940 comp rx=0 tx=0).stop 2023-11-06T03:11:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 shutdown_connections 2023-11-06T03:11:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x56028ece5b70 0x56028ed25cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56028ece4a50 0x56028ed257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56028ece4130 0x56028ed25270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 --2- 172.21.15.157:0/520804042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56028ec9a260 0x56028ed24d30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57a37fe640 0 client.7578 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 >> 172.21.15.157:0/520804042 conn(0x56028eca09f0 msgr2=0x56028ece7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 shutdown_connections 2023-11-06T03:11:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 1 -- 172.21.15.157:0/520804042 wait complete. 2023-11-06T03:11:11.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cdd81000 20 client.7578 trim_cache size 0 max 16384 2023-11-06T03:11:11.266 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:11.266 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:11:11.267 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:11:11.267 INFO:teuthology.orchestra.run.smithi157.stdout: "c717ed31-c4f9-4b2a-9a6b-2344b908f24e": { 2023-11-06T03:11:11.267 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "~mds0", 2023-11-06T03:11:11.267 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "c717ed31-c4f9-4b2a-9a6b-2344b908f24e", 2023-11-06T03:11:11.268 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:11:11.268 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:11.268 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:11.268 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:11.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 msgr2=0x7f57a8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 0x7f57a8067f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f57c41f6cc0 tx=0x7f57b0007e50 comp rx=0 tx=0).stop 2023-11-06T03:11:11.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 msgr2=0x7f57c414e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:11.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f57b8001a80 tx=0x7f57b8014420 comp rx=0 tx=0).stop 2023-11-06T03:11:11.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 shutdown_connections 2023-11-06T03:11:11.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57a8065ab0 0x7f57a8067f70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c41585d0 0x7f57c4152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c40cc380 0x7f57c4156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 --2- 172.21.15.157:0/518496621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c40b1e40 0x7f57c414e240 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.243+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 >> 172.21.15.157:0/518496621 conn(0x7f57c40732f0 msgr2=0x7f57c4065e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.247+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 shutdown_connections 2023-11-06T03:11:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:11.247+0000 7f57cac59640 1 -- 172.21.15.157:0/518496621 wait complete. 2023-11-06T03:11:11.274 INFO:tasks.cephfs.filesystem:scrub status for tag:c717ed31-c4f9-4b2a-9a6b-2344b908f24e - {'path': '~mds0', 'tag': 'c717ed31-c4f9-4b2a-9a6b-2344b908f24e', 'options': 'recursive'} 2023-11-06T03:11:41.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:11:41.093 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:11:41.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-11-06T03:11:41.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 Processor -- start 2023-11-06T03:11:41.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 -- start start 2023-11-06T03:11:41.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb600b2af0 0x7feb600ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feb6006d000 0x7feb60159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6015e440 con 0x7feb601599c0 2023-11-06T03:11:41.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6006c240 con 0x7feb600b2af0 2023-11-06T03:11:41.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.410+0000 7feb65356640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6006c520 con 0x7feb6006d000 2023-11-06T03:11:41.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb600b2af0 0x7feb600ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb600b2af0 0x7feb600ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42830/0 (socket says 172.21.15.157:42830) 2023-11-06T03:11:41.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 -- 172.21.15.157:0/186045194 learned_addr learned my addr 172.21.15.157:0/186045194 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/186045194 >> v2:172.21.15.195:3300/0 conn(0x7feb6006d000 0x7feb60159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 -- 172.21.15.157:0/186045194 >> v2:172.21.15.195:3300/0 conn(0x7feb6006d000 msgr2=0x7feb60159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/186045194 >> v2:172.21.15.195:3300/0 conn(0x7feb6006d000 0x7feb60159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 -- 172.21.15.157:0/186045194 >> v1:172.21.15.195:6789/0 conn(0x7feb601599c0 legacy=0x7feb600b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 -- 172.21.15.157:0/186045194 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6006c800 con 0x7feb600b2af0 2023-11-06T03:11:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/186045194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb600b2af0 0x7feb600ae000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7feb6006d930 tx=0x7feb4801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bb38baea28e0b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5dffb640 1 -- 172.21.15.157:0/186045194 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480aa610 con 0x7feb600b2af0 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5dffb640 1 -- 172.21.15.157:0/186045194 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb480a6040 con 0x7feb600b2af0 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb5dffb640 1 -- 172.21.15.157:0/186045194 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480aa990 con 0x7feb600b2af0 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 -- 172.21.15.157:0/186045194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb600b2af0 msgr2=0x7feb600ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 --2- 172.21.15.157:0/186045194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb600b2af0 0x7feb600ae000 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7feb6006d930 tx=0x7feb4801c490 comp rx=0 tx=0).stop 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 -- 172.21.15.157:0/186045194 shutdown_connections 2023-11-06T03:11:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 --2- 172.21.15.157:0/186045194 >> v2:172.21.15.195:3300/0 conn(0x7feb6006d000 0x7feb60159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 --2- 172.21.15.157:0/186045194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb600b2af0 0x7feb600ae000 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 -- 172.21.15.157:0/186045194 >> 172.21.15.157:0/186045194 conn(0x7feb60065930 msgr2=0x7feb60065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 -- 172.21.15.157:0/186045194 shutdown_connections 2023-11-06T03:11:41.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.414+0000 7feb65356640 1 -- 172.21.15.157:0/186045194 wait complete. 2023-11-06T03:11:41.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 Processor -- start 2023-11-06T03:11:41.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- start start 2023-11-06T03:11:41.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb600b2af0 0x7feb60152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 0x7feb600cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6014efe0 con 0x7feb6006d000 2023-11-06T03:11:41.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb601568a0 con 0x7feb601599c0 2023-11-06T03:11:41.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb601555a0 con 0x7feb600b2af0 2023-11-06T03:11:41.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42832/0 (socket says 172.21.15.157:42832) 2023-11-06T03:11:41.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 -- 172.21.15.157:0/1091831891 learned_addr learned my addr 172.21.15.157:0/1091831891 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 0x7feb600cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb600b2af0 0x7feb60152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 -- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb600b2af0 msgr2=0x7feb60152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb600b2af0 0x7feb60152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 -- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 msgr2=0x7feb600cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 0x7feb600cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 -- 172.21.15.157:0/1091831891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480094d0 con 0x7feb6006d000 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 0x7feb600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb5effd640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b410 tx=0x7feb5000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb577fe640 1 -- 172.21.15.157:0/1091831891 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50004020 con 0x7feb6006d000 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb577fe640 1 -- 172.21.15.157:0/1091831891 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb50004180 con 0x7feb6006d000 2023-11-06T03:11:41.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb60155880 con 0x7feb6006d000 2023-11-06T03:11:41.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb577fe640 1 -- 172.21.15.157:0/1091831891 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50011630 con 0x7feb6006d000 2023-11-06T03:11:41.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.418+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6014f4a0 con 0x7feb6006d000 2023-11-06T03:11:41.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb577fe640 1 -- 172.21.15.157:0/1091831891 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb50011790 con 0x7feb6006d000 2023-11-06T03:11:41.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb577fe640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 0x7feb40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb577fe640 1 -- 172.21.15.157:0/1091831891 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7feb50083030 con 0x7feb6006d000 2023-11-06T03:11:41.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 0x7feb40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 Processor -- start 2023-11-06T03:11:41.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- start start 2023-11-06T03:11:41.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 0x7feb40067f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feb480ae000 tx=0x7feb480a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39a50 0x55b689aeed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689aef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6899b6c10 con 0x55b689b39130 2023-11-06T03:11:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6899453e0 con 0x55b689b39a50 2023-11-06T03:11:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6899429b0 con 0x55b689aef260 2023-11-06T03:11:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42836/0 (socket says 172.21.15.157:42836) 2023-11-06T03:11:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 -- 172.21.15.157:0/2277307322 learned_addr learned my addr 172.21.15.157:0/2277307322 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39a50 0x55b689aeed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689aef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 -- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 msgr2=0x55b689aef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689aef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 -- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39a50 msgr2=0x55b689aeed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39a50 0x55b689aeed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 -- 172.21.15.157:0/2277307322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b68963ae20 con 0x55b689b39130 2023-11-06T03:11:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689aef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb5effd640 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7feb5005bfe0 tx=0x7feb50054d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13645c9a766629b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb54ff9640 1 -- 172.21.15.157:0/2277307322 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50058160 con 0x55b689b39130 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb54ff9640 1 -- 172.21.15.157:0/2277307322 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5005c510 con 0x55b689b39130 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb54ff9640 1 -- 172.21.15.157:0/2277307322 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50021dd0 con 0x55b689b39130 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- 172.21.15.157:0/2277307322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 msgr2=0x55b689b39510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7feb5005bfe0 tx=0x7feb50054d70 comp rx=0 tx=0).stop 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- 172.21.15.157:0/2277307322 shutdown_connections 2023-11-06T03:11:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689aef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39a50 0x55b689aeed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 --2- 172.21.15.157:0/2277307322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b39510 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.422+0000 7feb6847e000 1 -- 172.21.15.157:0/2277307322 >> 172.21.15.157:0/2277307322 conn(0x55b689af59f0 msgr2=0x55b689aed600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/2277307322 shutdown_connections 2023-11-06T03:11:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/2277307322 wait complete. 2023-11-06T03:11:41.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- start start 2023-11-06T03:11:41.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 Processor -- start 2023-11-06T03:11:41.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689af5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b68993e2a0 con 0x55b689b39a50 2023-11-06T03:11:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6898dd470 con 0x55b689b39130 2023-11-06T03:11:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6898fb660 con 0x55b689aef260 2023-11-06T03:11:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 10 client.0 _ll_register_callbacks cb 0x55b689adac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42850/0 (socket says 172.21.15.157:42850) 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 -- 172.21.15.157:0/4189900886 learned_addr learned my addr 172.21.15.157:0/4189900886 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5effd640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689af5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 -- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 msgr2=0x55b689af5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689af5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 -- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 msgr2=0x55b689b245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 -- 172.21.15.157:0/4189900886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6014f4a0 con 0x55b689b39a50 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7feb58007af0 tx=0x7feb58009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c12fa1d737a1589 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:41.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58010b00 con 0x55b689b39a50 2023-11-06T03:11:41.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6898fb0a0 con 0x55b689b39a50 2023-11-06T03:11:41.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7feb58010c60 con 0x55b689b39a50 2023-11-06T03:11:41.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb58010e20 con 0x55b689b39a50 2023-11-06T03:11:41.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 10 client.7590 fetch_fsmap learned FSMap version 73 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55b6899031d0 con 0x55b689b39a50 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b6898fb0a0 con 0x55b689b39a50 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 73) v1 ==== 3106+0+0 (secure 0 0 0) 0x7feb58017430 con 0x55b689b39a50 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 10 client.7590 fetch_fsmap finished waiting for FSMap version 73 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 10 client.7590 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:41.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 20 client.7590 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b3a510 0x55b689b3a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 4 client.7590 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:11:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b68993cbf0 con 0x55b689b3a510 2023-11-06T03:11:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5effd640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b3a510 0x55b689b3a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb5effd640 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b3a510 0x55b689b3a8f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbebc7d8799c84c server_cookie=4760667e7651abe1 in_seq=0 out_seq=0 2023-11-06T03:11:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 10 client.7590 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 1 -- 172.21.15.157:0/4189900886 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b68993cbf0 con 0x55b689b3a510 2023-11-06T03:11:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb3dffb640 10 client.7590 handle_command_reply: tid=0 2023-11-06T03:11:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 client.7590 shutdown 2023-11-06T03:11:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 msgr2=0x55b689b24b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.426+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7feb58007af0 tx=0x7feb58009f60 comp rx=0 tx=0).stop 2023-11-06T03:11:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 shutdown_connections 2023-11-06T03:11:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b3a510 0x55b689b3a8f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b24b20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb3dffb640 0 client.7590 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 --2- 172.21.15.157:0/4189900886 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689af5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 >> 172.21.15.157:0/4189900886 conn(0x55b689af59f0 msgr2=0x55b689aedd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 shutdown_connections 2023-11-06T03:11:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 1 -- 172.21.15.157:0/4189900886 wait complete. 2023-11-06T03:11:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.430+0000 7feb6847e000 20 client.7590 trim_cache size 0 max 16384 2023-11-06T03:11:41.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 Processor -- start 2023-11-06T03:11:41.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 -- start start 2023-11-06T03:11:41.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689b98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6895fbaa0 con 0x55b689b39a50 2023-11-06T03:11:41.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6899c6c70 con 0x55b689b39130 2023-11-06T03:11:41.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6898e37a0 con 0x55b689aef260 2023-11-06T03:11:41.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42866/0 (socket says 172.21.15.157:42866) 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 -- 172.21.15.157:0/3478121733 learned_addr learned my addr 172.21.15.157:0/3478121733 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 -- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 msgr2=0x55b689b98b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689b98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 -- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 msgr2=0x55b689b99080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 -- 172.21.15.157:0/3478121733 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6898e9520 con 0x55b689b39a50 2023-11-06T03:11:41.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f90 tx=0x7feb58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ee3c2d41af5b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb557fa640 1 -- 172.21.15.157:0/3478121733 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb580105f0 con 0x55b689b39a50 2023-11-06T03:11:41.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb557fa640 1 -- 172.21.15.157:0/3478121733 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58017990 con 0x55b689b39a50 2023-11-06T03:11:41.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.442+0000 7feb557fa640 1 -- 172.21.15.157:0/3478121733 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58017e10 con 0x55b689b39a50 2023-11-06T03:11:41.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/3478121733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 msgr2=0x55b689b995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f90 tx=0x7feb58016000 comp rx=0 tx=0).stop 2023-11-06T03:11:41.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/3478121733 shutdown_connections 2023-11-06T03:11:41.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39a50 0x55b689b995c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689b39130 0x55b689b99080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- 172.21.15.157:0/3478121733 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689aef260 0x55b689b98b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/3478121733 >> 172.21.15.157:0/3478121733 conn(0x55b689af59f0 msgr2=0x55b689b4bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/3478121733 shutdown_connections 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/3478121733 wait complete. 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- start start 2023-11-06T03:11:41.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 Processor -- start 2023-11-06T03:11:41.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 0x55b689b33e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689b39a50 0x55b689b348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6899422a0 con 0x55b689b39130 2023-11-06T03:11:41.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b689938640 con 0x55b689aef260 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b68993bab0 con 0x55b689b39a50 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 10 client.0 _ll_register_callbacks cb 0x55b689902cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42870/0 (socket says 172.21.15.157:42870) 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 -- 172.21.15.157:0/2975024613 learned_addr learned my addr 172.21.15.157:0/2975024613 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5f7fe640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689b39a50 0x55b689b348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5effd640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 0x55b689b33e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 -- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689b39a50 msgr2=0x55b689b348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689b39a50 0x55b689b348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 -- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 msgr2=0x55b689b33e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 0x55b689b33e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 -- 172.21.15.157:0/2975024613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6899031d0 con 0x55b689b39130 2023-11-06T03:11:41.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5effd640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 0x55b689b33e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5e7fc640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7feb4801c970 tx=0x7feb48019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994bc66e16740e95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:41.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480c3070 con 0x55b689b39130 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7feb480be910 con 0x55b689b39130 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b689907620 con 0x55b689b39130 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb480bead0 con 0x55b689b39130 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 10 client.7596 fetch_fsmap learned FSMap version 73 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55b6898dd090 con 0x55b689b39130 2023-11-06T03:11:41.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b689907620 con 0x55b689b39130 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 73) v1 ==== 3106+0+0 (secure 0 0 0) 0x7feb480bec30 con 0x55b689b39130 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 10 client.7596 fetch_fsmap finished waiting for FSMap version 73 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 10 client.7596 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 20 client.7596 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b34e10 0x55b689b35220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 4 client.7596 mds_command: new command op to 7329 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:11:41.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b689937320 con 0x55b689b34e10 2023-11-06T03:11:41.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5effd640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b34e10 0x55b689b35220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 10 client.7596 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb5effd640 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b34e10 0x55b689b35220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12555336f63fc91 server_cookie=315f72011cee2d3b in_seq=0 out_seq=0 2023-11-06T03:11:41.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 1 -- 172.21.15.157:0/2975024613 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b689937320 con 0x55b689b34e10 2023-11-06T03:11:41.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb3dffb640 10 client.7596 handle_command_reply: tid=0 2023-11-06T03:11:41.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.446+0000 7feb6847e000 1 client.7596 shutdown 2023-11-06T03:11:41.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 msgr2=0x55b689b34390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7feb4801c970 tx=0x7feb48019230 comp rx=0 tx=0).stop 2023-11-06T03:11:41.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 shutdown_connections 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x55b689b34e10 0x55b689b35220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b689b39a50 0x55b689b348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b689b39130 0x55b689b34390 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb3dffb640 0 client.7596 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 --2- 172.21.15.157:0/2975024613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b689aef260 0x55b689b33e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 >> 172.21.15.157:0/2975024613 conn(0x55b689af59f0 msgr2=0x55b689b4c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 shutdown_connections 2023-11-06T03:11:41.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 1 -- 172.21.15.157:0/2975024613 wait complete. 2023-11-06T03:11:41.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb6847e000 20 client.7596 trim_cache size 0 max 16384 2023-11-06T03:11:41.468 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:41.469 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:11:41.469 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:11:41.469 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:41.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 msgr2=0x7feb40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 0x7feb40067f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feb480ae000 tx=0x7feb480a7000 comp rx=0 tx=0).stop 2023-11-06T03:11:41.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 msgr2=0x7feb60156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b410 tx=0x7feb5000b8d0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 shutdown_connections 2023-11-06T03:11:41.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb40065a60 0x7feb40067f20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb601599c0 0x7feb600cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb600b2af0 0x7feb60152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 --2- 172.21.15.157:0/1091831891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb6006d000 0x7feb60156360 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 >> 172.21.15.157:0/1091831891 conn(0x7feb60065930 msgr2=0x7feb60070890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 shutdown_connections 2023-11-06T03:11:41.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.450+0000 7feb65356640 1 -- 172.21.15.157:0/1091831891 wait complete. 2023-11-06T03:11:41.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:11:41.473 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2023-11-06T03:11:41.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.586+0000 7f2f24c46640 1 Processor -- start 2023-11-06T03:11:41.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.586+0000 7f2f24c46640 1 -- start start 2023-11-06T03:11:41.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f24c46640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f2015e5e0 0x7f2f200b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f24c46640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2015f580 con 0x7f2f201551e0 2023-11-06T03:11:41.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f24c46640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2015efc0 con 0x7f2f2015ea90 2023-11-06T03:11:41.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f24c46640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200704b0 con 0x7f2f2015e5e0 2023-11-06T03:11:41.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2f201551e0 0x7f2f2006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43196/0 (socket says 172.21.15.157:43196) 2023-11-06T03:11:41.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1e59c640 1 -- 172.21.15.157:0/814601947 learned_addr learned my addr 172.21.15.157:0/814601947 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:41.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/814601947 >> v2:172.21.15.157:3300/0 conn(0x7f2f2015e5e0 0x7f2f200b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1d59a640 1 -- 172.21.15.157:0/814601947 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987326168 0 0) 0x7f2f2015f580 con 0x7f2f201551e0 2023-11-06T03:11:41.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1d59a640 1 -- 172.21.15.157:0/814601947 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f08003530 con 0x7f2f201551e0 2023-11-06T03:11:41.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/814601947 >> v1:172.21.15.195:6789/0 conn(0x7f2f2015ea90 legacy=0x7f2f2015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/814601947 >> v1:172.21.15.157:6789/0 conn(0x7f2f201551e0 legacy=0x7f2f2006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/814601947 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2015f2a0 con 0x7f2f2015e5e0 2023-11-06T03:11:41.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/814601947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f2015e5e0 0x7f2f200b3410 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097c0 tx=0x7f2f1401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d1be6965148025 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1d59a640 1 -- 172.21.15.157:0/814601947 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140ab600 con 0x7f2f2015e5e0 2023-11-06T03:11:41.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1d59a640 1 -- 172.21.15.157:0/814601947 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f140a7050 con 0x7f2f2015e5e0 2023-11-06T03:11:41.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.590+0000 7f2f1d59a640 1 -- 172.21.15.157:0/814601947 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140ab960 con 0x7f2f2015e5e0 2023-11-06T03:11:41.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/814601947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f2015e5e0 msgr2=0x7f2f200b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 --2- 172.21.15.157:0/814601947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f2015e5e0 0x7f2f200b3410 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2f140097c0 tx=0x7f2f1401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/814601947 shutdown_connections 2023-11-06T03:11:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 --2- 172.21.15.157:0/814601947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f2015e5e0 0x7f2f200b3410 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/814601947 >> 172.21.15.157:0/814601947 conn(0x7f2f20065930 msgr2=0x7f2f20065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/814601947 shutdown_connections 2023-11-06T03:11:41.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/814601947 wait complete. 2023-11-06T03:11:41.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 Processor -- start 2023-11-06T03:11:41.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- start start 2023-11-06T03:11:41.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2015e5e0 0x7f2f2015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f2015ea90 0x7f2f20149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2006b390 con 0x7f2f201551e0 2023-11-06T03:11:41.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2006b5e0 con 0x7f2f2015e5e0 2023-11-06T03:11:41.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2006b8c0 con 0x7f2f2015ea90 2023-11-06T03:11:41.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42884/0 (socket says 172.21.15.157:42884) 2023-11-06T03:11:41.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 -- 172.21.15.157:0/241432431 learned_addr learned my addr 172.21.15.157:0/241432431 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 -- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f2015ea90 msgr2=0x7f2f20149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f2015ea90 0x7f2f20149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2015e5e0 0x7f2f2015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f2015ea90 0x7f2f20149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 -- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2015e5e0 msgr2=0x7f2f2015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2015e5e0 0x7f2f2015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 -- 172.21.15.157:0/241432431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009420 con 0x7f2f201551e0 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c4d0 tx=0x7f2f1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f0effd640 1 -- 172.21.15.157:0/241432431 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10005cf0 con 0x7f2f201551e0 2023-11-06T03:11:41.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f0effd640 1 -- 172.21.15.157:0/241432431 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f10005e50 con 0x7f2f201551e0 2023-11-06T03:11:41.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2006bba0 con 0x7f2f201551e0 2023-11-06T03:11:41.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f0effd640 1 -- 172.21.15.157:0/241432431 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10007270 con 0x7f2f201551e0 2023-11-06T03:11:41.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.594+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2006c160 con 0x7f2f201551e0 2023-11-06T03:11:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f0effd640 1 -- 172.21.15.157:0/241432431 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2f100073d0 con 0x7f2f201551e0 2023-11-06T03:11:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f0effd640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 0x7f2f00067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f0effd640 1 -- 172.21.15.157:0/241432431 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f2f100827d0 con 0x7f2f201551e0 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 0x7f2f00067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 Processor -- start 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 -- start start 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 0x7f2f00067f50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f140ad000 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6786d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a67734bb0 con 0x556a678b79f0 2023-11-06T03:11:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a676c3380 con 0x556a678b70d0 2023-11-06T03:11:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a676c0950 con 0x556a6786d200 2023-11-06T03:11:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42900/0 (socket says 172.21.15.157:42900) 2023-11-06T03:11:41.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/3054549361 learned_addr learned my addr 172.21.15.157:0/3054549361 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6786d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 msgr2=0x556a6786d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6786d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 msgr2=0x556a678b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/3054549361 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a673b8dc0 con 0x556a678b79f0 2023-11-06T03:11:41.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2f14019ff0 tx=0x7f2f14004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47e67c6bbcb2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2efffff640 1 -- 172.21.15.157:0/3054549361 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004030 con 0x556a678b79f0 2023-11-06T03:11:41.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2efffff640 1 -- 172.21.15.157:0/3054549361 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14004190 con 0x556a678b79f0 2023-11-06T03:11:41.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.598+0000 7f2efffff640 1 -- 172.21.15.157:0/3054549361 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004500 con 0x556a678b79f0 2023-11-06T03:11:41.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3054549361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 msgr2=0x556a6786ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2f14019ff0 tx=0x7f2f14004740 comp rx=0 tx=0).stop 2023-11-06T03:11:41.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3054549361 shutdown_connections 2023-11-06T03:11:41.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6786d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a6786ccc0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3054549361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3054549361 >> 172.21.15.157:0/3054549361 conn(0x556a67873990 msgr2=0x556a6786b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3054549361 shutdown_connections 2023-11-06T03:11:41.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3054549361 wait complete. 2023-11-06T03:11:41.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- start start 2023-11-06T03:11:41.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 Processor -- start 2023-11-06T03:11:41.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a678be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a676bc240 con 0x556a678b79f0 2023-11-06T03:11:41.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a6765b410 con 0x556a678b70d0 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a67679600 con 0x556a6786d200 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 10 client.0 _ll_register_callbacks cb 0x556a67858bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42910/0 (socket says 172.21.15.157:42910) 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 -- 172.21.15.157:0/3747268845 learned_addr learned my addr 172.21.15.157:0/3747268845 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a678be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 -- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 msgr2=0x556a678be580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a678be580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 -- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 msgr2=0x556a678beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 -- 172.21.15.157:0/3747268845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2015f2a0 con 0x556a678b79f0 2023-11-06T03:11:41.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a678be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800f4b0 tx=0x7f2f1800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc27a9674f53167 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18003b90 con 0x556a678b79f0 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f2f18003cf0 con 0x556a678b79f0 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a67679040 con 0x556a678b79f0 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f180050d0 con 0x556a678b79f0 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 10 client.7611 fetch_fsmap learned FSMap version 73 2023-11-06T03:11:41.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556a67681170 con 0x556a678b79f0 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a67679040 con 0x556a678b79f0 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 73) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2f18012070 con 0x556a678b79f0 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 10 client.7611 fetch_fsmap finished waiting for FSMap version 73 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 10 client.7611 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 20 client.7611 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:41.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678b8b70 0x556a678b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 4 client.7611 mds_command: new command op to 7329 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a676bab90 con 0x556a678b8b70 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678b8b70 0x556a678b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2efd7fa640 10 client.7611 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.602+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678b8b70 0x556a678b8f50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c560e4c3a5a2ad server_cookie=5f3d6a51f52a19fe in_seq=0 out_seq=0 2023-11-06T03:11:41.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2efd7fa640 1 -- 172.21.15.157:0/3747268845 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556a676bab90 con 0x556a678b8b70 2023-11-06T03:11:41.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2efd7fa640 10 client.7611 handle_command_reply: tid=0 2023-11-06T03:11:41.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 client.7611 shutdown 2023-11-06T03:11:41.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 msgr2=0x556a678b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800f4b0 tx=0x7f2f1800f970 comp rx=0 tx=0).stop 2023-11-06T03:11:41.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 shutdown_connections 2023-11-06T03:11:41.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678b8b70 0x556a678b8f50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b79f0 0x556a678b7dd0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b70d0 0x556a678beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/3747268845 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a678be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2efd7fa640 0 client.7611 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 >> 172.21.15.157:0/3747268845 conn(0x556a67873990 msgr2=0x556a6786bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 shutdown_connections 2023-11-06T03:11:41.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 1 -- 172.21.15.157:0/3747268845 wait complete. 2023-11-06T03:11:41.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.606+0000 7f2f27d6e000 20 client.7611 trim_cache size 0 max 16384 2023-11-06T03:11:41.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 Processor -- start 2023-11-06T03:11:41.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 -- start start 2023-11-06T03:11:41.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6790e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a6790f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a67379a40 con 0x556a678b70d0 2023-11-06T03:11:41.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a677572d0 con 0x556a678b79f0 2023-11-06T03:11:41.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a676b9640 con 0x556a6786d200 2023-11-06T03:11:41.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42912/0 (socket says 172.21.15.157:42912) 2023-11-06T03:11:41.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/857190684 learned_addr learned my addr 172.21.15.157:0/857190684 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6790e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 msgr2=0x556a6790e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6790e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 msgr2=0x556a6790f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a6790f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 -- 172.21.15.157:0/857190684 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a676674c0 con 0x556a678b70d0 2023-11-06T03:11:41.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2f14000c10 tx=0x7f2f14009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a377dae795d09ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.618+0000 7f2f0cff9640 1 -- 172.21.15.157:0/857190684 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004300 con 0x556a678b70d0 2023-11-06T03:11:41.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f0cff9640 1 -- 172.21.15.157:0/857190684 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14004480 con 0x556a678b70d0 2023-11-06T03:11:41.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f0cff9640 1 -- 172.21.15.157:0/857190684 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14003860 con 0x556a678b70d0 2023-11-06T03:11:41.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/857190684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 msgr2=0x556a6790ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2f14000c10 tx=0x7f2f14009190 comp rx=0 tx=0).stop 2023-11-06T03:11:41.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/857190684 shutdown_connections 2023-11-06T03:11:41.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a6790f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a678b70d0 0x556a6790ebe0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/857190684 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a6786d200 0x556a6790e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/857190684 >> 172.21.15.157:0/857190684 conn(0x556a67873990 msgr2=0x556a67872940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/857190684 shutdown_connections 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/857190684 wait complete. 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- start start 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 Processor -- start 2023-11-06T03:11:41.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a678f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 0x556a678f8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a678c4600 con 0x556a6786d200 2023-11-06T03:11:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a676b7bb0 con 0x556a678b79f0 2023-11-06T03:11:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a67744c10 con 0x556a678f80a0 2023-11-06T03:11:41.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 10 client.0 _ll_register_callbacks cb 0x556a67680c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:11:41.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42918/0 (socket says 172.21.15.157:42918) 2023-11-06T03:11:41.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 -- 172.21.15.157:0/2552014395 learned_addr learned my addr 172.21.15.157:0/2552014395 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:41.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 0x556a678f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 -- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 msgr2=0x556a678f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 0x556a678f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 -- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 msgr2=0x556a678f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:41.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a678f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 -- 172.21.15.157:0/2552014395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a673b8dc0 con 0x556a6786d200 2023-11-06T03:11:41.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1ed9d640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 0x556a678f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:41.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1e59c640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007fd0 tx=0x7f2f10080d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d988993fc80dd91 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:41.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:11:41.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10053270 con 0x556a6786d200 2023-11-06T03:11:41.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f2f100533d0 con 0x556a6786d200 2023-11-06T03:11:41.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a676855c0 con 0x556a6786d200 2023-11-06T03:11:41.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f1006b1a0 con 0x556a6786d200 2023-11-06T03:11:41.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 10 client.7617 fetch_fsmap learned FSMap version 73 2023-11-06T03:11:41.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x556a6765b030 con 0x556a6786d200 2023-11-06T03:11:41.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1007f020 con 0x556a6786d200 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 73) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2f1005b820 con 0x556a6786d200 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 10 client.7617 fetch_fsmap finished waiting for FSMap version 73 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 10 client.7617 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 20 client.7617 populate_metadata read hostname 'smithi157' 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678f89c0 0x556a678f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:41.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 4 client.7617 mds_command: new command op to 7329 tid=0[{"prefix": "perf dump"}] 2023-11-06T03:11:41.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 --> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x556a676b52c0 con 0x556a678f89c0 2023-11-06T03:11:41.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678f89c0 0x556a678f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:41.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2efd7fa640 10 client.7617 ms_handle_connect on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.622+0000 7f2f1dd9b640 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678f89c0 0x556a678f8dd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10420dbd8020439b server_cookie=8824b5d223aaa43 in_seq=0 out_seq=0 2023-11-06T03:11:41.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2efd7fa640 1 -- 172.21.15.157:0/2552014395 <== mds.0 v2:172.21.15.157:6834/3882288197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17906 (crc 0 0 0) 0x556a676b52c0 con 0x556a678f89c0 2023-11-06T03:11:41.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2efd7fa640 10 client.7617 handle_command_reply: tid=0 2023-11-06T03:11:41.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 client.7617 shutdown 2023-11-06T03:11:41.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 msgr2=0x556a678f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2f10007fd0 tx=0x7f2f10080d00 comp rx=0 tx=0).stop 2023-11-06T03:11:41.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 shutdown_connections 2023-11-06T03:11:41.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:6834/3882288197,v1:172.21.15.157:6835/3882288197] conn(0x556a678f89c0 0x556a678f8dd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x556a678f80a0 0x556a678f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556a678b79f0 0x556a678f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2efd7fa640 0 client.7617 ms_handle_reset on v2:172.21.15.157:6834/3882288197 2023-11-06T03:11:41.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 --2- 172.21.15.157:0/2552014395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a6786d200 0x556a678f7230 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 >> 172.21.15.157:0/2552014395 conn(0x556a67873990 msgr2=0x556a678baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 shutdown_connections 2023-11-06T03:11:41.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 1 -- 172.21.15.157:0/2552014395 wait complete. 2023-11-06T03:11:41.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f27d6e000 20 client.7617 trim_cache size 0 max 16384 2023-11-06T03:11:41.646 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2023-11-06T03:11:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 6401, 2023-11-06T03:11:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 6402, 2023-11-06T03:11:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2256783, 2023-11-06T03:11:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 25892937, 2023-11-06T03:11:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 17, 2023-11-06T03:11:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 6, 2023-11-06T03:11:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.688331542, 2023-11-06T03:11:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.357203029, 2023-11-06T03:11:41.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.112958133, 2023-11-06T03:11:41.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.177420484, 2023-11-06T03:11:41.648 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:41.648 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6401, 2023-11-06T03:11:41.648 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 14.807190944, 2023-11-06T03:11:41.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002313262 2023-11-06T03:11:41.649 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.649 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:41.649 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 10, 2023-11-06T03:11:41.649 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000001041, 2023-11-06T03:11:41.650 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000104 2023-11-06T03:11:41.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.650 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-06T03:11:41.650 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2023-11-06T03:11:41.650 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 36530, 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 62592, 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 12927359, 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 53915870, 2023-11-06T03:11:41.651 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 8, 2023-11-06T03:11:41.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:11:41.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 4.610050868, 2023-11-06T03:11:41.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 2.279274285, 2023-11-06T03:11:41.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 2.661576430, 2023-11-06T03:11:41.652 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.172093198, 2023-11-06T03:11:41.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:41.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 62590, 2023-11-06T03:11:41.653 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 3.627544235, 2023-11-06T03:11:41.653 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000057957 2023-11-06T03:11:41.653 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:41.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 87335, 2023-11-06T03:11:41.654 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.019961718, 2023-11-06T03:11:41.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000228 2023-11-06T03:11:41.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 129408 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 7213, 2023-11-06T03:11:41.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 7195, 2023-11-06T03:11:41.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2133631, 2023-11-06T03:11:41.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 48229863, 2023-11-06T03:11:41.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2023-11-06T03:11:41.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2023-11-06T03:11:41.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.731273379, 2023-11-06T03:11:41.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.310830722, 2023-11-06T03:11:41.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.147604361, 2023-11-06T03:11:41.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.230571904, 2023-11-06T03:11:41.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2023-11-06T03:11:41.657 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7195, 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.543469489, 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000353505 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.659 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 133584, 2023-11-06T03:11:41.659 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 31280 2023-11-06T03:11:41.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.660 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2023-11-06T03:11:41.660 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2023-11-06T03:11:41.660 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2023-11-06T03:11:41.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.660 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 9753, 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 25.805434523, 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.002645897 2023-11-06T03:11:41.661 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.662 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.662 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2023-11-06T03:11:41.662 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2023-11-06T03:11:41.662 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2023-11-06T03:11:41.662 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2959, 2023-11-06T03:11:41.663 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.865651267, 2023-11-06T03:11:41.663 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000630500 2023-11-06T03:11:41.663 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.663 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.663 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2023-11-06T03:11:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 28832, 2023-11-06T03:11:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 28832, 2023-11-06T03:11:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2023-11-06T03:11:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 28832, 2023-11-06T03:11:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 868.719417668, 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.030130390 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2023-11-06T03:11:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2023-11-06T03:11:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 67, 2023-11-06T03:11:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2023-11-06T03:11:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2023-11-06T03:11:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 16524, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 10924, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 5600, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 887, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2023-11-06T03:11:41.667 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 2, 2023-11-06T03:11:41.668 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 2, 2023-11-06T03:11:41.668 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2023-11-06T03:11:41.668 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 58481, 2023-11-06T03:11:41.668 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 56636, 2023-11-06T03:11:41.668 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2023-11-06T03:11:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2023-11-06T03:11:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2023-11-06T03:11:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2023-11-06T03:11:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 1533, 2023-11-06T03:11:41.669 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 22718, 2023-11-06T03:11:41.670 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2023-11-06T03:11:41.670 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2023-11-06T03:11:41.670 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2023-11-06T03:11:41.670 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2023-11-06T03:11:41.670 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2023-11-06T03:11:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2023-11-06T03:11:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 1791, 2023-11-06T03:11:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2023-11-06T03:11:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2023-11-06T03:11:41.671 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 1791, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2023-11-06T03:11:41.672 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 11, 2023-11-06T03:11:41.673 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 1, 2023-11-06T03:11:41.673 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 10, 2023-11-06T03:11:41.673 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 1780, 2023-11-06T03:11:41.673 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2023-11-06T03:11:41.673 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 7606, 2023-11-06T03:11:41.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 6185, 2023-11-06T03:11:41.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2023-11-06T03:11:41.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-06T03:11:41.674 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-06T03:11:41.674 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 7604, 2023-11-06T03:11:41.675 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 2000, 2023-11-06T03:11:41.675 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 43, 2023-11-06T03:11:41.675 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 17961 2023-11-06T03:11:41.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.675 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2023-11-06T03:11:41.676 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2023-11-06T03:11:41.676 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2023-11-06T03:11:41.676 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2023-11-06T03:11:41.676 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2023-11-06T03:11:41.676 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 4165, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 3584, 2023-11-06T03:11:41.677 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 4395, 2023-11-06T03:11:41.678 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2023-11-06T03:11:41.678 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2023-11-06T03:11:41.678 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2023-11-06T03:11:41.678 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2023-11-06T03:11:41.678 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 1, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2023-11-06T03:11:41.679 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2023-11-06T03:11:41.680 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2023-11-06T03:11:41.680 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.680 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2023-11-06T03:11:41.680 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 25401, 2023-11-06T03:11:41.680 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 24415, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 24415, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 2010, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 1024, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 30, 2023-11-06T03:11:41.681 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 29, 2023-11-06T03:11:41.682 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 29, 2023-11-06T03:11:41.682 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2023-11-06T03:11:41.682 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2023-11-06T03:11:41.682 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 1, 2023-11-06T03:11:41.682 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 83890749, 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 84486792, 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 0, 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6821, 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 148.288655657, 2023-11-06T03:11:41.683 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.021740016 2023-11-06T03:11:41.684 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.684 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 0 2023-11-06T03:11:41.684 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.684 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2023-11-06T03:11:41.684 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 14, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 2015, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 2001, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 13, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 13, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2023-11-06T03:11:41.685 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 16524, 2023-11-06T03:11:41.686 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 20001, 2023-11-06T03:11:41.686 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 3477, 2023-11-06T03:11:41.686 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 2, 2023-11-06T03:11:41.686 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 2002, 2023-11-06T03:11:41.686 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 2000, 2023-11-06T03:11:41.687 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 88132, 2023-11-06T03:11:41.687 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 207144 2023-11-06T03:11:41.687 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.687 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2023-11-06T03:11:41.687 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 43793, 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 28832, 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 8, 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2023-11-06T03:11:41.688 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2000, 2023-11-06T03:11:41.689 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1.344503724, 2023-11-06T03:11:41.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000672251 2023-11-06T03:11:41.689 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.689 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2023-11-06T03:11:41.689 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 465, 2023-11-06T03:11:41.690 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.181667025, 2023-11-06T03:11:41.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000390681 2023-11-06T03:11:41.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.690 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2023-11-06T03:11:41.690 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.691 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.691 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2023-11-06T03:11:41.691 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6000, 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 9.526959264, 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001587826 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 7959, 2023-11-06T03:11:41.692 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 13.479245887, 2023-11-06T03:11:41.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001693585 2023-11-06T03:11:41.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.693 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2023-11-06T03:11:41.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.695 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2023-11-06T03:11:41.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.695 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.696 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2023-11-06T03:11:41.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.696 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3, 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.079398187, 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.026466062 2023-11-06T03:11:41.697 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.698 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2023-11-06T03:11:41.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.698 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.698 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.698 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.699 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2023-11-06T03:11:41.699 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:41.699 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.005653255, 2023-11-06T03:11:41.699 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.005653255 2023-11-06T03:11:41.699 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.700 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2023-11-06T03:11:41.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:41.701 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006686124, 2023-11-06T03:11:41.701 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006686124 2023-11-06T03:11:41.701 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.701 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2023-11-06T03:11:41.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.702 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.702 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.702 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.702 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2023-11-06T03:11:41.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 6, 2023-11-06T03:11:41.703 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.169247532, 2023-11-06T03:11:41.703 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.028207922 2023-11-06T03:11:41.703 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.703 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 4395, 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 814.331511245, 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.185285895 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2023-11-06T03:11:41.704 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.705 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.705 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.705 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2023-11-06T03:11:41.705 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.706 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.706 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.706 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2023-11-06T03:11:41.706 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.707 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.707 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.707 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2023-11-06T03:11:41.707 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.006826015, 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.006826015 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.708 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.709 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.709 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2023-11-06T03:11:41.709 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1, 2023-11-06T03:11:41.709 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000316404, 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000316404 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.710 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.711 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.711 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2023-11-06T03:11:41.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.711 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.711 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.712 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.713 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2023-11-06T03:11:41.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.713 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.713 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.713 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.714 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2023-11-06T03:11:41.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.714 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.714 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.714 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.715 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2023-11-06T03:11:41.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 8000, 2023-11-06T03:11:41.715 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 29.587403006, 2023-11-06T03:11:41.715 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.003698425 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.716 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2023-11-06T03:11:41.717 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 1, 2023-11-06T03:11:41.718 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 0, 2023-11-06T03:11:41.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 1, 2023-11-06T03:11:41.718 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2023-11-06T03:11:41.718 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 15577, 2023-11-06T03:11:41.718 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 15577, 2023-11-06T03:11:41.719 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 115, 2023-11-06T03:11:41.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2023-11-06T03:11:41.719 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.719 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2023-11-06T03:11:41.719 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2023-11-06T03:11:41.720 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2023-11-06T03:11:41.721 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-06T03:11:41.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2023-11-06T03:11:41.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2023-11-06T03:11:41.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2023-11-06T03:11:41.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2023-11-06T03:11:41.722 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2023-11-06T03:11:41.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-06T03:11:41.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2023-11-06T03:11:41.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-06T03:11:41.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2023-11-06T03:11:41.723 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2023-11-06T03:11:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2023-11-06T03:11:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2023-11-06T03:11:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2023-11-06T03:11:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-06T03:11:41.724 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2023-11-06T03:11:41.725 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2023-11-06T03:11:41.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2023-11-06T03:11:41.726 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2023-11-06T03:11:41.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 251521, 2023-11-06T03:11:41.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 86, 2023-11-06T03:11:41.726 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2023-11-06T03:11:41.727 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2023-11-06T03:11:41.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2023-11-06T03:11:41.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2023-11-06T03:11:41.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2023-11-06T03:11:41.727 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 6280, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 80, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2023-11-06T03:11:41.728 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2023-11-06T03:11:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2023-11-06T03:11:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2023-11-06T03:11:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 12328885, 2023-11-06T03:11:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 645979, 2023-11-06T03:11:41.729 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2023-11-06T03:11:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2023-11-06T03:11:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2023-11-06T03:11:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2023-11-06T03:11:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.730 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 17818, 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 92020519, 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 37, 2023-11-06T03:11:41.731 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 17818, 2023-11-06T03:11:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2023-11-06T03:11:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17818, 2023-11-06T03:11:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 355.567541690, 2023-11-06T03:11:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.019955524 2023-11-06T03:11:41.732 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2023-11-06T03:11:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2023-11-06T03:11:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 17818, 2023-11-06T03:11:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 28378 2023-11-06T03:11:41.733 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 17818, 2023-11-06T03:11:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 1888, 2023-11-06T03:11:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 15930, 2023-11-06T03:11:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2023-11-06T03:11:41.734 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 54, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 5194, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 96, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 6916, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 82, 2023-11-06T03:11:41.735 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2023-11-06T03:11:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2023-11-06T03:11:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 1584, 2023-11-06T03:11:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2023-11-06T03:11:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 2050, 2023-11-06T03:11:41.736 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1792, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 10388, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2023-11-06T03:11:41.737 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2023-11-06T03:11:41.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2023-11-06T03:11:41.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2023-11-06T03:11:41.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2023-11-06T03:11:41.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2023-11-06T03:11:41.738 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2023-11-06T03:11:41.739 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2023-11-06T03:11:41.740 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2023-11-06T03:11:41.741 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2023-11-06T03:11:41.741 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2023-11-06T03:11:41.741 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2023-11-06T03:11:41.741 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2023-11-06T03:11:41.741 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2023-11-06T03:11:41.742 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2023-11-06T03:11:41.742 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2023-11-06T03:11:41.742 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 87, 2023-11-06T03:11:41.742 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2023-11-06T03:11:41.742 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 3, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 8, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 8, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 175, 2023-11-06T03:11:41.743 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 0, 2023-11-06T03:11:41.744 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 47 2023-11-06T03:11:41.744 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.744 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2023-11-06T03:11:41.744 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 1, 2023-11-06T03:11:41.744 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2023-11-06T03:11:41.745 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2023-11-06T03:11:41.745 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2023-11-06T03:11:41.745 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.745 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2023-11-06T03:11:41.745 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 28834 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 32, 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2023-11-06T03:11:41.746 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 32, 2023-11-06T03:11:41.747 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 3584, 2023-11-06T03:11:41.747 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2023-11-06T03:11:41.747 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.747 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-06T03:11:41.747 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:41.748 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:11:41.748 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:41.748 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 50140, 2023-11-06T03:11:41.748 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 12798629, 2023-11-06T03:11:41.748 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 50140, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 50140, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 12798629, 2023-11-06T03:11:41.749 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:41.750 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.750 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.750 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.750 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.750 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2023-11-06T03:11:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2023-11-06T03:11:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:11:41.752 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:11:41.752 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:41.752 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:11:41.752 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 17818, 2023-11-06T03:11:41.752 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 92395175, 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 17818, 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 92395175, 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.753 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.754 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.754 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.754 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2023-11-06T03:11:41.754 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:41.754 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2023-11-06T03:11:41.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:41.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2023-11-06T03:11:41.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2023-11-06T03:11:41.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:41.755 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2023-11-06T03:11:41.756 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 17818, 2023-11-06T03:11:41.756 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 17818, 2023-11-06T03:11:41.756 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 17818, 2023-11-06T03:11:41.756 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 17818, 2023-11-06T03:11:41.756 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.757 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2023-11-06T03:11:41.758 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:41.758 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:11:41.758 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:41.758 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 3584, 2023-11-06T03:11:41.758 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 374656, 2023-11-06T03:11:41.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:41.759 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 3584, 2023-11-06T03:11:41.759 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:41.759 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:41.759 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 95, 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 374656, 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.760 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55d928f2ed20": { 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2023-11-06T03:11:41.761 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 25401, 2023-11-06T03:11:41.762 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 80292488, 2023-11-06T03:11:41.762 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2023-11-06T03:11:41.762 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 25401, 2023-11-06T03:11:41.762 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2023-11-06T03:11:41.762 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2023-11-06T03:11:41.763 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 6821, 2023-11-06T03:11:41.763 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 80292488, 2023-11-06T03:11:41.763 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2023-11-06T03:11:41.763 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2023-11-06T03:11:41.763 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2023-11-06T03:11:41.764 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2023-11-06T03:11:41.764 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.764 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:41.764 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:41.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 msgr2=0x7f2f00067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 0x7f2f00067f50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f140ad000 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2023-11-06T03:11:41.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 msgr2=0x7f2f20149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:41.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c4d0 tx=0x7f2f1000c990 comp rx=0 tx=0).stop 2023-11-06T03:11:41.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 shutdown_connections 2023-11-06T03:11:41.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2f00065a90 0x7f2f00067f50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2f2015ea90 0x7f2f20149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2015e5e0 0x7f2f2015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 --2- 172.21.15.157:0/241432431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f201551e0 0x7f2f20149e10 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:41.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 >> 172.21.15.157:0/241432431 conn(0x7f2f20065930 msgr2=0x7f2f20067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:41.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 shutdown_connections 2023-11-06T03:11:41.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:41.626+0000 7f2f24c46640 1 -- 172.21.15.157:0/241432431 wait complete. 2023-11-06T03:11:41.769 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:11:41.769 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:11:41.823 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:11:41.883 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:11:41.883 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:11:41.884 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:41.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:42.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.058+0000 7f7f92fb4640 1 Processor -- start 2023-11-06T03:11:42.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 -- start start 2023-11-06T03:11:42.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7f8c15e5e0 0x7f7f8c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 0x7f7f8c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c15f580 con 0x7f7f8c1551e0 2023-11-06T03:11:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c0704b0 con 0x7f7f8c15e5e0 2023-11-06T03:11:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f92fb4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15efc0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f90d50640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7f8c1551e0 0x7f7f8c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43202/0 (socket says 172.21.15.157:43202) 2023-11-06T03:11:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f90d50640 1 -- 172.21.15.157:0/2461773152 learned_addr learned my addr 172.21.15.157:0/2461773152 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f91551640 1 --2- 172.21.15.157:0/2461773152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 0x7f7f8c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f8bfff640 1 --2- 172.21.15.157:0/2461773152 >> v2:172.21.15.195:3300/0 conn(0x7f7f8c15e5e0 0x7f7f8c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f8b7fe640 1 -- 172.21.15.157:0/2461773152 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066205498 0 0) 0x7f7f8c15f580 con 0x7f7f8c1551e0 2023-11-06T03:11:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f8b7fe640 1 -- 172.21.15.157:0/2461773152 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f74003530 con 0x7f7f8c1551e0 2023-11-06T03:11:42.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f91551640 1 -- 172.21.15.157:0/2461773152 >> v2:172.21.15.195:3300/0 conn(0x7f7f8c15e5e0 msgr2=0x7f7f8c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:42.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f91551640 1 --2- 172.21.15.157:0/2461773152 >> v2:172.21.15.195:3300/0 conn(0x7f7f8c15e5e0 0x7f7f8c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f91551640 1 -- 172.21.15.157:0/2461773152 >> v1:172.21.15.157:6789/0 conn(0x7f7f8c1551e0 legacy=0x7f7f8c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.062+0000 7f7f91551640 1 -- 172.21.15.157:0/2461773152 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c15f2a0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f91551640 1 --2- 172.21.15.157:0/2461773152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 0x7f7f8c0b3950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009d90 tx=0x7f7f8401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddaa85ce55c91bf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f8b7fe640 1 -- 172.21.15.157:0/2461773152 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f840a5950 con 0x7f7f8c15ea90 2023-11-06T03:11:42.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f8b7fe640 1 -- 172.21.15.157:0/2461773152 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f840a5ab0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f8b7fe640 1 -- 172.21.15.157:0/2461773152 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f840a5e20 con 0x7f7f8c15ea90 2023-11-06T03:11:42.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- 172.21.15.157:0/2461773152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 msgr2=0x7f7f8c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/2461773152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 0x7f7f8c0b3950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009d90 tx=0x7f7f8401c890 comp rx=0 tx=0).stop 2023-11-06T03:11:42.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- 172.21.15.157:0/2461773152 shutdown_connections 2023-11-06T03:11:42.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/2461773152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c15ea90 0x7f7f8c0b3950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/2461773152 >> v2:172.21.15.195:3300/0 conn(0x7f7f8c15e5e0 0x7f7f8c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- 172.21.15.157:0/2461773152 >> 172.21.15.157:0/2461773152 conn(0x7f7f8c065930 msgr2=0x7f7f8c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- 172.21.15.157:0/2461773152 shutdown_connections 2023-11-06T03:11:42.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- 172.21.15.157:0/2461773152 wait complete. 2023-11-06T03:11:42.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 Processor -- start 2023-11-06T03:11:42.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.066+0000 7f7f92fb4640 1 -- start start 2023-11-06T03:11:42.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c1551e0 0x7f7f8c140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f8c15e5e0 0x7f7f8c06bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06d210 con 0x7f7f8c15ea90 2023-11-06T03:11:42.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06c600 con 0x7f7f8c15e5e0 2023-11-06T03:11:42.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06c8e0 con 0x7f7f8c1551e0 2023-11-06T03:11:42.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42928/0 (socket says 172.21.15.157:42928) 2023-11-06T03:11:42.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 -- 172.21.15.157:0/3003911408 learned_addr learned my addr 172.21.15.157:0/3003911408 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:42.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f90d50640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c1551e0 0x7f7f8c140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f8bfff640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f8c15e5e0 0x7f7f8c06bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 -- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c1551e0 msgr2=0x7f7f8c140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c1551e0 0x7f7f8c140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 -- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f8c15e5e0 msgr2=0x7f7f8c06bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f8c15e5e0 0x7f7f8c06bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 -- 172.21.15.157:0/3003911408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84009a70 con 0x7f7f8c15ea90 2023-11-06T03:11:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f91551640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7f8401af40 tx=0x7f7f84019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f840a5950 con 0x7f7f8c15ea90 2023-11-06T03:11:42.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84003db0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84004260 con 0x7f7f8c15ea90 2023-11-06T03:11:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c06cbc0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c140760 con 0x7f7f8c15ea90 2023-11-06T03:11:42.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c06d3c0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7f840b4070 con 0x7f7f8c15ea90 2023-11-06T03:11:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f7f8411ddc0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.070+0000 7f7f90d50640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.074+0000 7f7f90d50640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.078+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7f8c06d3c0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.230+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f8c072750 con 0x7f7f8c15ea90 2023-11-06T03:11:42.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f897fa640 1 -- 172.21.15.157:0/3003911408 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18000 (secure 0 0 0) 0x7f7f840eb6c0 con 0x7f7f8c15ea90 2023-11-06T03:11:42.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:42.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:22.343214+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:42.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 msgr2=0x7f7f78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c009040 comp rx=0 tx=0).stop 2023-11-06T03:11:42.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 msgr2=0x7f7f8c1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7f8401af40 tx=0x7f7f84019630 comp rx=0 tx=0).stop 2023-11-06T03:11:42.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 shutdown_connections 2023-11-06T03:11:42.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7f78065a60 0x7f7f78067f20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f8c15ea90 0x7f7f8c1599d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f8c15e5e0 0x7f7f8c06bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 --2- 172.21.15.157:0/3003911408 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7f8c1551e0 0x7f7f8c140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 >> 172.21.15.157:0/3003911408 conn(0x7f7f8c065930 msgr2=0x7f7f8c15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 shutdown_connections 2023-11-06T03:11:42.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.234+0000 7f7f92fb4640 1 -- 172.21.15.157:0/3003911408 wait complete. 2023-11-06T03:11:42.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:09:33.683286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:09:34.099590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:11:42.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:11:42.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 Processor -- start 2023-11-06T03:11:42.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 -- start start 2023-11-06T03:11:42.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15caa0 0x7f7eec0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 0x7f7eec0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0d2920 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0d18a0 con 0x7f7eec15ce80 2023-11-06T03:11:42.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef42ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0d1b60 con 0x7f7eec15caa0 2023-11-06T03:11:42.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42942/0 (socket says 172.21.15.157:42942) 2023-11-06T03:11:42.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 -- 172.21.15.157:0/480281338 learned_addr learned my addr 172.21.15.157:0/480281338 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:42.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef2089640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15caa0 0x7f7eec0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef1888640 1 --2- 172.21.15.157:0/480281338 >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 0x7f7eec0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 -- 172.21.15.157:0/480281338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15caa0 msgr2=0x7f7eec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:42.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15caa0 0x7f7eec0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 -- 172.21.15.157:0/480281338 >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 msgr2=0x7f7eec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:42.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 --2- 172.21.15.157:0/480281338 >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 0x7f7eec0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef288a640 1 -- 172.21.15.157:0/480281338 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0d1e40 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.394+0000 7f7ef1888640 1 --2- 172.21.15.157:0/480281338 >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 0x7f7eec0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:42.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef288a640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0d3cb0 tx=0x7f7ee401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e6ee9becdb2329 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1087640 1 -- 172.21.15.157:0/480281338 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40a5820 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1087640 1 -- 172.21.15.157:0/480281338 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee40a5980 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1087640 1 -- 172.21.15.157:0/480281338 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40a5d10 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- 172.21.15.157:0/480281338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0ce1a0 msgr2=0x7f7eec0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0d3cb0 tx=0x7f7ee401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- 172.21.15.157:0/480281338 shutdown_connections 2023-11-06T03:11:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec0d00b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/480281338 >> v2:172.21.15.195:3300/0 conn(0x7f7eec15ce80 0x7f7eec0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/480281338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15caa0 0x7f7eec0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- 172.21.15.157:0/480281338 >> 172.21.15.157:0/480281338 conn(0x7f7eec070790 msgr2=0x7f7eec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- 172.21.15.157:0/480281338 shutdown_connections 2023-11-06T03:11:42.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- 172.21.15.157:0/480281338 wait complete. 2023-11-06T03:11:42.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 Processor -- start 2023-11-06T03:11:42.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- start start 2023-11-06T03:11:42.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 0x7f7eec1f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec15d910 con 0x7f7eec15caa0 2023-11-06T03:11:42.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec15dbf0 con 0x7f7eec0ce1a0 2023-11-06T03:11:42.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef42ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec15ded0 con 0x7f7eec15ce80 2023-11-06T03:11:42.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42948/0 (socket says 172.21.15.157:42948) 2023-11-06T03:11:42.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 -- 172.21.15.157:0/1310222135 learned_addr learned my addr 172.21.15.157:0/1310222135 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:42.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef2089640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef288a640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 0x7f7eec1f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 -- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 msgr2=0x7f7eec1f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 0x7f7eec1f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 -- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eec0ce1a0 msgr2=0x7f7eec15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.398+0000 7f7ef1888640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef1888640 1 -- 172.21.15.157:0/1310222135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4009b30 con 0x7f7eec15caa0 2023-11-06T03:11:42.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef288a640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 0x7f7eec1f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:42.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef1888640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c7c0 tx=0x7f7edc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc007a30 con 0x7f7eec15caa0 2023-11-06T03:11:42.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc007b90 con 0x7f7eec15caa0 2023-11-06T03:11:42.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec15e1b0 con 0x7f7eec15caa0 2023-11-06T03:11:42.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00ce70 con 0x7f7eec15caa0 2023-11-06T03:11:42.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec1f5c70 con 0x7f7eec15caa0 2023-11-06T03:11:42.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec072750 con 0x7f7eec15caa0 2023-11-06T03:11:42.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7edc0113f0 con 0x7f7eec15caa0 2023-11-06T03:11:42.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 0x7f7ed0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f7edc083750 con 0x7f7eec15caa0 2023-11-06T03:11:42.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef2089640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 0x7f7ed0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.402+0000 7f7ef2089640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 0x7f7ed0067f20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800ba30 tx=0x7f7ee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.406+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7edc051020 con 0x7f7eec15caa0 2023-11-06T03:11:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7eec0b7950 con 0x7f7eec15caa0 2023-11-06T03:11:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ee2ffd640 1 -- 172.21.15.157:0/1310222135 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7edc054640 con 0x7f7eec15caa0 2023-11-06T03:11:42.562 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:11:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 msgr2=0x7f7ed0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 0x7f7ed0067f20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800ba30 tx=0x7f7ee8008040 comp rx=0 tx=0).stop 2023-11-06T03:11:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 msgr2=0x7f7eec149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c7c0 tx=0x7f7edc00cc80 comp rx=0 tx=0).stop 2023-11-06T03:11:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 shutdown_connections 2023-11-06T03:11:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ed0065a60 0x7f7ed0067f20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7eec15ce80 0x7f7eec1f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7eec15caa0 0x7f7eec149bd0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 --2- 172.21.15.157:0/1310222135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7eec0ce1a0 0x7f7eec15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 >> 172.21.15.157:0/1310222135 conn(0x7f7eec070790 msgr2=0x7f7eec15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 shutdown_connections 2023-11-06T03:11:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.562+0000 7f7ef42ed640 1 -- 172.21.15.157:0/1310222135 wait complete. 2023-11-06T03:11:42.580 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:11:42.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 Processor -- start 2023-11-06T03:11:42.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 -- start start 2023-11-06T03:11:42.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58f415b6a0 0x7f58f40d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f415bbd0 con 0x7f58f40cafb0 2023-11-06T03:11:42.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40b6570 con 0x7f58f415b6a0 2023-11-06T03:11:42.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58fb565640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415beb0 con 0x7f58f40b6190 2023-11-06T03:11:42.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58f415b6a0 0x7f58f40d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58f415b6a0 0x7f58f40d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42952/0 (socket says 172.21.15.157:42952) 2023-11-06T03:11:42.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 -- 172.21.15.157:0/3013837208 learned_addr learned my addr 172.21.15.157:0/3013837208 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:42.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f9301640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 -- 172.21.15.157:0/3013837208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 msgr2=0x7f58f40cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:42.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 -- 172.21.15.157:0/3013837208 >> v1:172.21.15.195:6789/0 conn(0x7f58f40cafb0 legacy=0x7f58f40ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:42.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 -- 172.21.15.157:0/3013837208 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415c190 con 0x7f58f415b6a0 2023-11-06T03:11:42.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f9301640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:42.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.710+0000 7f58f8b00640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f415b6a0 0x7f58f40d0030 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3314d7b671febf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58ebfff640 1 -- 172.21.15.157:0/3013837208 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40ab620 con 0x7f58f415b6a0 2023-11-06T03:11:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58ebfff640 1 -- 172.21.15.157:0/3013837208 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40ab780 con 0x7f58f415b6a0 2023-11-06T03:11:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58ebfff640 1 -- 172.21.15.157:0/3013837208 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40abb10 con 0x7f58f415b6a0 2023-11-06T03:11:42.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- 172.21.15.157:0/3013837208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f415b6a0 msgr2=0x7f58f40d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f415b6a0 0x7f58f40d0030 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- 172.21.15.157:0/3013837208 shutdown_connections 2023-11-06T03:11:42.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f415b6a0 0x7f58f40d0030 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- 172.21.15.157:0/3013837208 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- 172.21.15.157:0/3013837208 >> 172.21.15.157:0/3013837208 conn(0x7f58f40b23e0 msgr2=0x7f58f40b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- 172.21.15.157:0/3013837208 shutdown_connections 2023-11-06T03:11:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- 172.21.15.157:0/3013837208 wait complete. 2023-11-06T03:11:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 Processor -- start 2023-11-06T03:11:42.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- start start 2023-11-06T03:11:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f415b6a0 0x7f58f4148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c7040 con 0x7f58f40cafb0 2023-11-06T03:11:42.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c4920 con 0x7f58f415b6a0 2023-11-06T03:11:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58fb565640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c56d0 con 0x7f58f40b6190 2023-11-06T03:11:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42962/0 (socket says 172.21.15.157:42962) 2023-11-06T03:11:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 -- 172.21.15.157:0/558645415 learned_addr learned my addr 172.21.15.157:0/558645415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f9301640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 -- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 msgr2=0x7f58f40c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 -- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f415b6a0 msgr2=0x7f58f4148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f9b02640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f415b6a0 0x7f58f4148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f415b6a0 0x7f58f4148870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 -- 172.21.15.157:0/558645415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e40094d0 con 0x7f58f40cafb0 2023-11-06T03:11:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f9301640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58f8b00640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f58e401c300 tx=0x7f58e401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.714+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40abe80 con 0x7f58f40cafb0 2023-11-06T03:11:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40be3f0 con 0x7f58f40cafb0 2023-11-06T03:11:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f40c4b50 con 0x7f58f40cafb0 2023-11-06T03:11:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40be7a0 con 0x7f58f40cafb0 2023-11-06T03:11:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f40c5110 con 0x7f58f40cafb0 2023-11-06T03:11:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f4072750 con 0x7f58f40cafb0 2023-11-06T03:11:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f58e40b43f0 con 0x7f58f40cafb0 2023-11-06T03:11:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58e9ffb640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 0x7f58d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f58e411d470 con 0x7f58f40cafb0 2023-11-06T03:11:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58f9301640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 0x7f58d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.718+0000 7f58f9301640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 0x7f58d8067f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f58f0008930 tx=0x7f58f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:42.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.722+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58e40eae40 con 0x7f58f40cafb0 2023-11-06T03:11:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.870+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58f4148db0 con 0x7f58f40cafb0 2023-11-06T03:11:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58e9ffb640 1 -- 172.21.15.157:0/558645415 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f58e40ee460 con 0x7f58f40cafb0 2023-11-06T03:11:42.873 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:11:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 msgr2=0x7f58d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 0x7f58d8067f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f58f0008930 tx=0x7f58f0008040 comp rx=0 tx=0).stop 2023-11-06T03:11:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 msgr2=0x7f58f40c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f58e401c300 tx=0x7f58e401ad10 comp rx=0 tx=0).stop 2023-11-06T03:11:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 shutdown_connections 2023-11-06T03:11:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58d8065a60 0x7f58d8067f20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58f415b6a0 0x7f58f4148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58f40cafb0 0x7f58f40c43e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 --2- 172.21.15.157:0/558645415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58f40b6190 0x7f58f40c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 >> 172.21.15.157:0/558645415 conn(0x7f58f40b23e0 msgr2=0x7f58f4157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 shutdown_connections 2023-11-06T03:11:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:42.874+0000 7f58fb565640 1 -- 172.21.15.157:0/558645415 wait complete. 2023-11-06T03:11:42.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:11:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 Processor -- start 2023-11-06T03:11:43.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 -- start start 2023-11-06T03:11:43.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38a01551e0 0x7f38a006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a015f580 con 0x7f38a015ea60 2023-11-06T03:11:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a015efc0 con 0x7f38a01551e0 2023-11-06T03:11:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f38a5794640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015f2a0 con 0x7f38a015e5e0 2023-11-06T03:11:43.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38a01551e0 0x7f38a006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f3897fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38a01551e0 0x7f38a006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:44074/0 (socket says 172.21.15.157:44074) 2023-11-06T03:11:43.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 -- 172.21.15.157:0/3773562596 learned_addr learned my addr 172.21.15.157:0/3773562596 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:43.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f3897fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:48462/0 (socket says 172.21.15.157:48462) 2023-11-06T03:11:43.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 -- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 msgr2=0x7f38a00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 --2- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 -- 172.21.15.157:0/3773562596 >> v1:172.21.15.195:6789/0 conn(0x7f38a015ea60 legacy=0x7f38a015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.010+0000 7f389effd640 1 -- 172.21.15.157:0/3773562596 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a006f5a0 con 0x7f38a01551e0 2023-11-06T03:11:43.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f389effd640 1 --2- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a01551e0 0x7f38a006fda0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3890009870 tx=0x7f389001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83b1610dd618c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f389e7fc640 1 -- 172.21.15.157:0/3773562596 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900ad640 con 0x7f38a01551e0 2023-11-06T03:11:43.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f389e7fc640 1 -- 172.21.15.157:0/3773562596 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38900ad7a0 con 0x7f38a01551e0 2023-11-06T03:11:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f389e7fc640 1 -- 172.21.15.157:0/3773562596 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900adb30 con 0x7f38a01551e0 2023-11-06T03:11:43.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a01551e0 msgr2=0x7f38a006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 --2- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a01551e0 0x7f38a006fda0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3890009870 tx=0x7f389001c480 comp rx=0 tx=0).stop 2023-11-06T03:11:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- 172.21.15.157:0/3773562596 shutdown_connections 2023-11-06T03:11:43.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 --2- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 --2- 172.21.15.157:0/3773562596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a01551e0 0x7f38a006fda0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- 172.21.15.157:0/3773562596 >> 172.21.15.157:0/3773562596 conn(0x7f38a0065930 msgr2=0x7f38a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- 172.21.15.157:0/3773562596 shutdown_connections 2023-11-06T03:11:43.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- 172.21.15.157:0/3773562596 wait complete. 2023-11-06T03:11:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 Processor -- start 2023-11-06T03:11:43.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.014+0000 7f38a5794640 1 -- start start 2023-11-06T03:11:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a015ea60 0x7f38a014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0147520 con 0x7f38a01551e0 2023-11-06T03:11:43.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006cde0 con 0x7f38a015ea60 2023-11-06T03:11:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a014ea50 con 0x7f38a015e5e0 2023-11-06T03:11:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42964/0 (socket says 172.21.15.157:42964) 2023-11-06T03:11:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 -- 172.21.15.157:0/2376066401 learned_addr learned my addr 172.21.15.157:0/2376066401 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:43.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f3897fff640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389f7fe640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a015ea60 0x7f38a014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 -- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 msgr2=0x7f38a014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a014f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 -- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a015ea60 msgr2=0x7f38a014b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a015ea60 0x7f38a014b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 -- 172.21.15.157:0/2376066401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900094d0 con 0x7f38a01551e0 2023-11-06T03:11:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f389effd640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f389001ab30 tx=0x7f3890019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900b8620 con 0x7f38a01551e0 2023-11-06T03:11:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38900a6040 con 0x7f38a01551e0 2023-11-06T03:11:43.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a014ed30 con 0x7f38a01551e0 2023-11-06T03:11:43.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900b89c0 con 0x7f38a01551e0 2023-11-06T03:11:43.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a014e740 con 0x7f38a01551e0 2023-11-06T03:11:43.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a014f250 con 0x7f38a01551e0 2023-11-06T03:11:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38900b4040 con 0x7f38a01551e0 2023-11-06T03:11:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.018+0000 7f38977fe640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 0x7f3880067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.022+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f389011e150 con 0x7f38a01551e0 2023-11-06T03:11:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.022+0000 7f3897fff640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 0x7f3880067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.022+0000 7f3897fff640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 0x7f3880067f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38a015d0e0 tx=0x7f3888008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.026+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38900eba30 con 0x7f38a01551e0 2023-11-06T03:11:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.182+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38a0072750 con 0x7f38a01551e0 2023-11-06T03:11:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38977fe640 1 -- 172.21.15.157:0/2376066401 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38900ef050 con 0x7f38a01551e0 2023-11-06T03:11:43.185 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:11:43.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 msgr2=0x7f3880067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 0x7f3880067f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38a015d0e0 tx=0x7f3888008040 comp rx=0 tx=0).stop 2023-11-06T03:11:43.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 msgr2=0x7f38a0146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f389001ab30 tx=0x7f3890019220 comp rx=0 tx=0).stop 2023-11-06T03:11:43.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 shutdown_connections 2023-11-06T03:11:43.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3880065a60 0x7f3880067f20 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38a015d0e0 tx=0x7f3888008040 comp rx=0 tx=0).stop 2023-11-06T03:11:43.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38a015ea60 0x7f38a014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38a015e5e0 0x7f38a014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 --2- 172.21.15.157:0/2376066401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38a01551e0 0x7f38a0146fe0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 >> 172.21.15.157:0/2376066401 conn(0x7f38a0065930 msgr2=0x7f38a00ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 shutdown_connections 2023-11-06T03:11:43.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.186+0000 7f38a5794640 1 -- 172.21.15.157:0/2376066401 wait complete. 2023-11-06T03:11:43.203 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:11:43.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.326+0000 7f5159640640 1 Processor -- start 2023-11-06T03:11:43.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.326+0000 7f5159640640 1 -- start start 2023-11-06T03:11:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5159640640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f51540cf6b0 0x7f51540cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5159640640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540d0050 con 0x7f515415cdf0 2023-11-06T03:11:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5159640640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540d1c60 con 0x7f51540b6110 2023-11-06T03:11:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5159640640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540d1f40 con 0x7f51540cf6b0 2023-11-06T03:11:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f515415cdf0 0x7f51540d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43208/0 (socket says 172.21.15.157:43208) 2023-11-06T03:11:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 -- 172.21.15.157:0/954690210 learned_addr learned my addr 172.21.15.157:0/954690210 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799295540 0 0) 0x7f51540d0050 con 0x7f515415cdf0 2023-11-06T03:11:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f51527fc640 1 --2- 172.21.15.157:0/954690210 >> v2:172.21.15.195:3300/0 conn(0x7f51540cf6b0 0x7f51540cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5140003530 con 0x7f515415cdf0 2023-11-06T03:11:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2864369918 0 0) 0x7f5140003530 con 0x7f515415cdf0 2023-11-06T03:11:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 >> v2:172.21.15.195:3300/0 conn(0x7f51540cf6b0 msgr2=0x7f51540cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 --2- 172.21.15.157:0/954690210 >> v2:172.21.15.195:3300/0 conn(0x7f51540cf6b0 0x7f51540cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 >> v1:172.21.15.195:6789/0 conn(0x7f51540b6110 legacy=0x7f51540b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540d2220 con 0x7f515415cdf0 2023-11-06T03:11:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f514c001680 con 0x7f515415cdf0 2023-11-06T03:11:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 >> v1:172.21.15.157:6789/0 conn(0x7f515415cdf0 legacy=0x7f51540d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 0x7f5140003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51400047d0 0x7f5140004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140005190 0x7f514000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540d1f40 con 0x7f5140003980 2023-11-06T03:11:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514000c0e0 con 0x7f51400047d0 2023-11-06T03:11:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514000c2a0 con 0x7f5140005190 2023-11-06T03:11:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 0x7f5140003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f51527fc640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51400047d0 0x7f5140004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 -- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140005190 msgr2=0x7f514000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f51537fe640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140005190 0x7f514000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140005190 0x7f514000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 -- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51400047d0 msgr2=0x7f5140004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51400047d0 0x7f5140004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 -- 172.21.15.157:0/954690210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140003530 con 0x7f5140003980 2023-11-06T03:11:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f51527fc640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51400047d0 0x7f5140004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f51537fe640 1 -- 172.21.15.157:0/954690210 reap_dead start 2023-11-06T03:11:43.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.330+0000 7f5152ffd640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 0x7f5140003da0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c240 tx=0x7f514c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31da02befbad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c005ae0 con 0x7f5140003980 2023-11-06T03:11:43.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c005c40 con 0x7f5140003980 2023-11-06T03:11:43.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5151ffb640 1 -- 172.21.15.157:0/954690210 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c0a5a10 con 0x7f5140003980 2023-11-06T03:11:43.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 msgr2=0x7f5140003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 0x7f5140003da0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c240 tx=0x7f514c01cc10 comp rx=0 tx=0).stop 2023-11-06T03:11:43.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/954690210 shutdown_connections 2023-11-06T03:11:43.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 --2- 172.21.15.157:0/954690210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5140003980 0x7f5140003da0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/954690210 >> 172.21.15.157:0/954690210 conn(0x7f5154065930 msgr2=0x7f5154065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/954690210 shutdown_connections 2023-11-06T03:11:43.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/954690210 wait complete. 2023-11-06T03:11:43.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 Processor -- start 2023-11-06T03:11:43.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- start start 2023-11-06T03:11:43.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140003980 0x7f51541f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51541f5170 0x7f51541f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540ca920 con 0x7f5154147790 2023-11-06T03:11:43.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540cac00 con 0x7f51541f5170 2023-11-06T03:11:43.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540caee0 con 0x7f5140003980 2023-11-06T03:11:43.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42984/0 (socket says 172.21.15.157:42984) 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 -- 172.21.15.157:0/932559118 learned_addr learned my addr 172.21.15.157:0/932559118 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5152ffd640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140003980 0x7f51541f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 -- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140003980 msgr2=0x7f51541f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140003980 0x7f51541f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 -- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51541f5170 msgr2=0x7f51541f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51541f5170 0x7f51541f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 -- 172.21.15.157:0/932559118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540cb1c0 con 0x7f5154147790 2023-11-06T03:11:43.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f51537fe640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f51540d0cc0 tx=0x7f514400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400b070 con 0x7f5154147790 2023-11-06T03:11:43.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51440054e0 con 0x7f5154147790 2023-11-06T03:11:43.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51540cb4a0 con 0x7f5154147790 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144003820 con 0x7f5154147790 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.334+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51541f9770 con 0x7f5154147790 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51540cb6c0 con 0x7f5154147790 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5144003980 con 0x7f5154147790 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f514b7fe640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 0x7f5134067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f5144082990 con 0x7f5154147790 2023-11-06T03:11:43.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f5152ffd640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 0x7f5134067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.338+0000 7f5152ffd640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 0x7f5134067f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f514c0ab000 tx=0x7f514c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.342+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f51440501d0 con 0x7f5154147790 2023-11-06T03:11:43.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.494+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51541f9960 con 0x7f5154147790 2023-11-06T03:11:43.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f514b7fe640 1 -- 172.21.15.157:0/932559118 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5144018020 con 0x7f5154147790 2023-11-06T03:11:43.497 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:11:43.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 msgr2=0x7f5134067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 0x7f5134067f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f514c0ab000 tx=0x7f514c0a7070 comp rx=0 tx=0).stop 2023-11-06T03:11:43.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 msgr2=0x7f51541f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f51540d0cc0 tx=0x7f514400d890 comp rx=0 tx=0).stop 2023-11-06T03:11:43.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 shutdown_connections 2023-11-06T03:11:43.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5134065a60 0x7f5134067f20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5154147790 0x7f51541f9060 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51541f5170 0x7f51541f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 --2- 172.21.15.157:0/932559118 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5140003980 0x7f51541f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 >> 172.21.15.157:0/932559118 conn(0x7f5154065930 msgr2=0x7f515415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 shutdown_connections 2023-11-06T03:11:43.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.498+0000 7f5159640640 1 -- 172.21.15.157:0/932559118 wait complete. 2023-11-06T03:11:43.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:11:43.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 Processor -- start 2023-11-06T03:11:43.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 -- start start 2023-11-06T03:11:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20fc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f20fc0d0280 0x7f20fc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0d2920 con 0x7f20fc15ce80 2023-11-06T03:11:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc0d18a0 con 0x7f20fc0d0280 2023-11-06T03:11:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f21023df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0d1b60 con 0x7f20fc15caa0 2023-11-06T03:11:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f20fc15ce80 0x7f20fc0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43222/0 (socket says 172.21.15.157:43222) 2023-11-06T03:11:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 -- 172.21.15.157:0/3236855119 learned_addr learned my addr 172.21.15.157:0/3236855119 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fbfff640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20fc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f210097c640 1 --2- 172.21.15.157:0/3236855119 >> v2:172.21.15.195:3300/0 conn(0x7f20fc0d0280 0x7f20fc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872576535 0 0) 0x7f20fc0d2920 con 0x7f20fc15ce80 2023-11-06T03:11:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e4003530 con 0x7f20fc15ce80 2023-11-06T03:11:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492107672 0 0) 0x7f20e4003530 con 0x7f20fc15ce80 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 msgr2=0x7f20fc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20fc0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 >> v2:172.21.15.195:3300/0 conn(0x7f20fc0d0280 msgr2=0x7f20fc0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 --2- 172.21.15.157:0/3236855119 >> v2:172.21.15.195:3300/0 conn(0x7f20fc0d0280 0x7f20fc0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0d1e40 con 0x7f20fc15ce80 2023-11-06T03:11:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f20f0001680 con 0x7f20fc15ce80 2023-11-06T03:11:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 >> v1:172.21.15.157:6789/0 conn(0x7f20fc15ce80 legacy=0x7f20fc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20e4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20e4004d10 0x7f20e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0d18a0 con 0x7f20e4003f00 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0d1b60 con 0x7f20e4004d10 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e400b990 con 0x7f20fc15caa0 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fbfff640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20e4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f210097c640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20e4004d10 0x7f20e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 -- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 msgr2=0x7f20e4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc15caa0 0x7f20e4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 -- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20e4004d10 msgr2=0x7f20e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20e4004d10 0x7f20e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f20fb7fe640 1 -- 172.21.15.157:0/3236855119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4003530 con 0x7f20e4003f00 2023-11-06T03:11:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.638+0000 7f210097c640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20e4004d10 0x7f20e4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20e40047d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f20f000c290 tx=0x7f20f001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd2fb02e972aed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0005ae0 con 0x7f20e4003f00 2023-11-06T03:11:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f00ab070 con 0x7f20e4003f00 2023-11-06T03:11:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f20faffd640 1 -- 172.21.15.157:0/3236855119 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0005e80 con 0x7f20e4003f00 2023-11-06T03:11:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- 172.21.15.157:0/3236855119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 msgr2=0x7f20e40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 --2- 172.21.15.157:0/3236855119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20e40047d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f20f000c290 tx=0x7f20f001f3d0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f210097c640 1 -- 172.21.15.157:0/3236855119 reap_dead start 2023-11-06T03:11:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- 172.21.15.157:0/3236855119 shutdown_connections 2023-11-06T03:11:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- 172.21.15.157:0/3236855119 >> 172.21.15.157:0/3236855119 conn(0x7f20fc070790 msgr2=0x7f20fc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- 172.21.15.157:0/3236855119 shutdown_connections 2023-11-06T03:11:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- 172.21.15.157:0/3236855119 wait complete. 2023-11-06T03:11:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 Processor -- start 2023-11-06T03:11:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.642+0000 7f21023df640 1 -- start start 2023-11-06T03:11:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 0x7f20fc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 0x7f20fc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc15ed10 con 0x7f20e4003f00 2023-11-06T03:11:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc155ec0 con 0x7f20fc0c9120 2023-11-06T03:11:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc15e370 con 0x7f20fc1f6960 2023-11-06T03:11:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42990/0 (socket says 172.21.15.157:42990) 2023-11-06T03:11:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 -- 172.21.15.157:0/2293977104 learned_addr learned my addr 172.21.15.157:0/2293977104 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f210097c640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 0x7f20fc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 0x7f20fc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 -- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 msgr2=0x7f20fc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 0x7f20fc0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 -- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 msgr2=0x7f20fc0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 0x7f20fc0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 -- 172.21.15.157:0/2293977104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc15e650 con 0x7f20e4003f00 2023-11-06T03:11:43.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f210097c640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 0x7f20fc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:43.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 0x7f20fc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:43.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fbfff640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0115e0 tx=0x7f20ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00f8f0 con 0x7f20e4003f00 2023-11-06T03:11:43.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec00fa50 con 0x7f20e4003f00 2023-11-06T03:11:43.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc15e930 con 0x7f20e4003f00 2023-11-06T03:11:43.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00fdc0 con 0x7f20e4003f00 2023-11-06T03:11:43.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc152c20 con 0x7f20e4003f00 2023-11-06T03:11:43.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc072750 con 0x7f20e4003f00 2023-11-06T03:11:43.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f20ec015460 con 0x7f20e4003f00 2023-11-06T03:11:43.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 0x7f20e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:43.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f20ec0836a0 con 0x7f20e4003f00 2023-11-06T03:11:43.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 0x7f20e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:43.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.646+0000 7f20fb7fe640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 0x7f20e8067f50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae000 tx=0x7f20f0001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:43.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.650+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f20ec050f70 con 0x7f20e4003f00 2023-11-06T03:11:43.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20fc0c9580 con 0x7f20e4003f00 2023-11-06T03:11:43.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f20f8ff9640 1 -- 172.21.15.157:0/2293977104 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f20ec054590 con 0x7f20e4003f00 2023-11-06T03:11:43.819 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:11:43.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 msgr2=0x7f20e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 0x7f20e8067f50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae000 tx=0x7f20f0001a10 comp rx=0 tx=0).stop 2023-11-06T03:11:43.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 msgr2=0x7f20fc142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:43.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0115e0 tx=0x7f20ec011aa0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 shutdown_connections 2023-11-06T03:11:43.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20e8065a90 0x7f20e8067f50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20fc1f6960 0x7f20fc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20fc0c9120 0x7f20fc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 --2- 172.21.15.157:0/2293977104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e4003f00 0x7f20fc142bd0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:43.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.818+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 >> 172.21.15.157:0/2293977104 conn(0x7f20fc070790 msgr2=0x7f20fc0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:43.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.822+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 shutdown_connections 2023-11-06T03:11:43.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:43.822+0000 7f21023df640 1 -- 172.21.15.157:0/2293977104 wait complete. 2023-11-06T03:11:43.838 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:11:43.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:43.839 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:11:43.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[67538]: fuse finished with error 0 and tester_r 0 2023-11-06T03:11:43.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:11:43.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:43.857 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:11:44.029 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:11:44.029 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:44.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:11:44.036 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:11:44.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:44.037 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:11:44.081 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:11:44.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:11:44.082 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:11:44.083 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:44.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:11:44.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:11:44.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:44.091 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:11:44.137 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:11:44.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:11:44.139 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:11:44.139 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:44.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:11:44.145 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:11:44.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:11:44.146 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:11:44.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:11:44.195 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:11:44.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:11:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.322+0000 7f39197dc640 1 Processor -- start 2023-11-06T03:11:44.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.322+0000 7f39197dc640 1 -- start start 2023-11-06T03:11:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39141551e0 0x7f391406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 0x7f39140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f39140b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391415f580 con 0x7f39141551e0 2023-11-06T03:11:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391415efc0 con 0x7f391415e5e0 2023-11-06T03:11:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39197dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391415f2a0 con 0x7f391415ea90 2023-11-06T03:11:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39141551e0 0x7f391406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39141551e0 0x7f391406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42994/0 (socket says 172.21.15.157:42994) 2023-11-06T03:11:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 -- 172.21.15.157:0/3048381574 learned_addr learned my addr 172.21.15.157:0/3048381574 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39137fe640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f39140b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39127fc640 1 --2- 172.21.15.157:0/3048381574 >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 0x7f39140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 -- 172.21.15.157:0/3048381574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 msgr2=0x7f39140b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f39140b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 -- 172.21.15.157:0/3048381574 >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 msgr2=0x7f39140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 --2- 172.21.15.157:0/3048381574 >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 0x7f39140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 -- 172.21.15.157:0/3048381574 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391406f5a0 con 0x7f39141551e0 2023-11-06T03:11:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f39127fc640 1 --2- 172.21.15.157:0/3048381574 >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 0x7f39140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3912ffd640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39141551e0 0x7f391406fda0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3914066880 tx=0x7f390801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7c2a6d059bd271 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3911ffb640 1 -- 172.21.15.157:0/3048381574 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080ab620 con 0x7f39141551e0 2023-11-06T03:11:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3911ffb640 1 -- 172.21.15.157:0/3048381574 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39080ab780 con 0x7f39141551e0 2023-11-06T03:11:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.326+0000 7f3911ffb640 1 -- 172.21.15.157:0/3048381574 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080abb10 con 0x7f39141551e0 2023-11-06T03:11:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- 172.21.15.157:0/3048381574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39141551e0 msgr2=0x7f391406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39141551e0 0x7f391406fda0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f3914066880 tx=0x7f390801c440 comp rx=0 tx=0).stop 2023-11-06T03:11:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- 172.21.15.157:0/3048381574 shutdown_connections 2023-11-06T03:11:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f39140b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- 172.21.15.157:0/3048381574 >> v2:172.21.15.195:3300/0 conn(0x7f391415e5e0 0x7f39140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- 172.21.15.157:0/3048381574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39141551e0 0x7f391406fda0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- 172.21.15.157:0/3048381574 >> 172.21.15.157:0/3048381574 conn(0x7f3914065930 msgr2=0x7f3914065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- 172.21.15.157:0/3048381574 shutdown_connections 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- 172.21.15.157:0/3048381574 wait complete. 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 Processor -- start 2023-11-06T03:11:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- start start 2023-11-06T03:11:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39141551e0 0x7f39140cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f3914141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914144350 con 0x7f391415e5e0 2023-11-06T03:11:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39141509b0 con 0x7f39141551e0 2023-11-06T03:11:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39197dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39140ce130 con 0x7f391415ea90 2023-11-06T03:11:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42996/0 (socket says 172.21.15.157:42996) 2023-11-06T03:11:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 -- 172.21.15.157:0/1803068736 learned_addr learned my addr 172.21.15.157:0/1803068736 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f3912ffd640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39141551e0 0x7f39140cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39137fe640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f3914141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 -- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 msgr2=0x7f3914141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f3914141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 -- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39141551e0 msgr2=0x7f39140cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.330+0000 7f39127fc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39141551e0 0x7f39140cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f39127fc640 1 -- 172.21.15.157:0/1803068736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080094d0 con 0x7f391415e5e0 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f39127fc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f390400c7c0 tx=0x7f390400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904007a30 con 0x7f391415e5e0 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904007b90 con 0x7f391415e5e0 2023-11-06T03:11:44.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400ce70 con 0x7f391415e5e0 2023-11-06T03:11:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39140cef50 con 0x7f391415e5e0 2023-11-06T03:11:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39140cde50 con 0x7f391415e5e0 2023-11-06T03:11:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3914150b40 con 0x7f391415e5e0 2023-11-06T03:11:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39040113f0 con 0x7f391415e5e0 2023-11-06T03:11:44.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 0x7f3900067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f39040833a0 con 0x7f391415e5e0 2023-11-06T03:11:44.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f3912ffd640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 0x7f3900067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.334+0000 7f3912ffd640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 0x7f3900067f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f39080af000 tx=0x7f39080a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.338+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3904050c70 con 0x7f391415e5e0 2023-11-06T03:11:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.526+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3914072750 con 0x7f391415e5e0 2023-11-06T03:11:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.526+0000 7f38fb7fe640 1 -- 172.21.15.157:0/1803068736 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f3904054290 con 0x7f391415e5e0 2023-11-06T03:11:44.529 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:44.529 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:11:23.420288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":30,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:11:44.529 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-11-06T03:11:44.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 msgr2=0x7f3900067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 0x7f3900067f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f39080af000 tx=0x7f39080a7000 comp rx=0 tx=0).stop 2023-11-06T03:11:44.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 msgr2=0x7f3914142c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f390400c7c0 tx=0x7f390400cc80 comp rx=0 tx=0).stop 2023-11-06T03:11:44.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 shutdown_connections 2023-11-06T03:11:44.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3900065a60 0x7f3900067f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f391415ea90 0x7f3914141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f391415e5e0 0x7f3914142c50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 --2- 172.21.15.157:0/1803068736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39141551e0 0x7f39140cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 >> 172.21.15.157:0/1803068736 conn(0x7f3914065930 msgr2=0x7f391415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:44.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 shutdown_connections 2023-11-06T03:11:44.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.530+0000 7f39197dc640 1 -- 172.21.15.157:0/1803068736 wait complete. 2023-11-06T03:11:44.549 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 Processor -- start 2023-11-06T03:11:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 -- start start 2023-11-06T03:11:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35dc0b3b20 0x7f35dc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc15f580 con 0x7f35dc15e5e0 2023-11-06T03:11:44.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc15efc0 con 0x7f35dc0b3b20 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e2b67640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc15f2a0 con 0x7f35dc1551e0 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35dc0b3b20 0x7f35dc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35dc0b3b20 0x7f35dc15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43012/0 (socket says 172.21.15.157:43012) 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 -- 172.21.15.157:0/1769703635 learned_addr learned my addr 172.21.15.157:0/1769703635 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e0903640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 -- 172.21.15.157:0/1769703635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 msgr2=0x7f35dc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 -- 172.21.15.157:0/1769703635 >> v1:172.21.15.157:6789/0 conn(0x7f35dc15e5e0 legacy=0x7f35dc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.666+0000 7f35e1104640 1 -- 172.21.15.157:0/1769703635 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc06f5a0 con 0x7f35dc0b3b20 2023-11-06T03:11:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e1104640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc15c1b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009e50 tx=0x7f35d401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cc3cd2682c0fea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35db7fe640 1 -- 172.21.15.157:0/1769703635 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d40a5720 con 0x7f35dc0b3b20 2023-11-06T03:11:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35db7fe640 1 -- 172.21.15.157:0/1769703635 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d40a5880 con 0x7f35dc0b3b20 2023-11-06T03:11:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35db7fe640 1 -- 172.21.15.157:0/1769703635 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d40a5c10 con 0x7f35dc0b3b20 2023-11-06T03:11:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- 172.21.15.157:0/1769703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 msgr2=0x7f35dc15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc15c1b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009e50 tx=0x7f35d401c950 comp rx=0 tx=0).stop 2023-11-06T03:11:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- 172.21.15.157:0/1769703635 shutdown_connections 2023-11-06T03:11:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc15c1b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- 172.21.15.157:0/1769703635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- 172.21.15.157:0/1769703635 >> 172.21.15.157:0/1769703635 conn(0x7f35dc065930 msgr2=0x7f35dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- 172.21.15.157:0/1769703635 shutdown_connections 2023-11-06T03:11:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- 172.21.15.157:0/1769703635 wait complete. 2023-11-06T03:11:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 Processor -- start 2023-11-06T03:11:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- start start 2023-11-06T03:11:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35dc15e5e0 0x7f35dc0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc153360 con 0x7f35dc0b3b20 2023-11-06T03:11:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc156a50 con 0x7f35dc15e5e0 2023-11-06T03:11:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e2b67640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc14c930 con 0x7f35dc1551e0 2023-11-06T03:11:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43028/0 (socket says 172.21.15.157:43028) 2023-11-06T03:11:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 -- 172.21.15.157:0/4805942 learned_addr learned my addr 172.21.15.157:0/4805942 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35dbfff640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 -- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 msgr2=0x7f35dc155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 -- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35dc15e5e0 msgr2=0x7f35dc0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35dc15e5e0 0x7f35dc0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.670+0000 7f35e0903640 1 -- 172.21.15.157:0/4805942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009b30 con 0x7f35dc0b3b20 2023-11-06T03:11:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35e0903640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d800 tx=0x7f35d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000b6c0 con 0x7f35dc0b3b20 2023-11-06T03:11:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000b820 con 0x7f35dc0b3b20 2023-11-06T03:11:44.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc156770 con 0x7f35dc0b3b20 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000bb90 con 0x7f35dc0b3b20 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc152ff0 con 0x7f35dc0b3b20 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc152b70 con 0x7f35dc0b3b20 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f35d000bcf0 con 0x7f35dc0b3b20 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 0x7f35c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f35d0083810 con 0x7f35dc0b3b20 2023-11-06T03:11:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35dbfff640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 0x7f35c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.674+0000 7f35dbfff640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 0x7f35c8067f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007fd0 tx=0x7f35cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.678+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f35dc152b70 con 0x7f35dc0b3b20 2023-11-06T03:11:44.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.830+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35dc152b70 con 0x7f35dc0b3b20 2023-11-06T03:11:44.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.830+0000 7f35d97fa640 1 -- 172.21.15.157:0/4805942 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18000 (secure 0 0 0) 0x7f35dc152b70 con 0x7f35dc0b3b20 2023-11-06T03:11:44.832 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:44.832 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:22.343214+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:44.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 msgr2=0x7f35c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 0x7f35c8067f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007fd0 tx=0x7f35cc009040 comp rx=0 tx=0).stop 2023-11-06T03:11:44.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 msgr2=0x7f35dc14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d800 tx=0x7f35d000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 shutdown_connections 2023-11-06T03:11:44.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f35c8065a60 0x7f35c8067f20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35dc15e5e0 0x7f35dc0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35dc1551e0 0x7f35dc155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 --2- 172.21.15.157:0/4805942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35dc0b3b20 0x7f35dc14c3f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 >> 172.21.15.157:0/4805942 conn(0x7f35dc065930 msgr2=0x7f35dc15dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:44.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 shutdown_connections 2023-11-06T03:11:44.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.834+0000 7f35e2b67640 1 -- 172.21.15.157:0/4805942 wait complete. 2023-11-06T03:11:44.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:09:33.683286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:09:34.099590+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:11:44.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:11:44.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 Processor -- start 2023-11-06T03:11:44.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 -- start start 2023-11-06T03:11:44.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1815f580 con 0x7f7d1815ea60 2023-11-06T03:11:44.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1815efc0 con 0x7f7d181551e0 2023-11-06T03:11:44.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1f192640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180704b0 con 0x7f7d1815e5e0 2023-11-06T03:11:44.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7d1815ea60 0x7f7d1815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43240/0 (socket says 172.21.15.157:43240) 2023-11-06T03:11:44.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 -- 172.21.15.157:0/2399684219 learned_addr learned my addr 172.21.15.157:0/2399684219 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:44.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d17fff640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209573545 0 0) 0x7f7d1815f580 con 0x7f7d1815ea60 2023-11-06T03:11:44.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d00003530 con 0x7f7d1815ea60 2023-11-06T03:11:44.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807339733 0 0) 0x7f7d00003530 con 0x7f7d1815ea60 2023-11-06T03:11:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 msgr2=0x7f7d180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d180b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 >> v1:172.21.15.195:6789/0 conn(0x7f7d181551e0 legacy=0x7f7d1806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1815f2a0 con 0x7f7d1815ea60 2023-11-06T03:11:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7d10001a00 con 0x7f7d1815ea60 2023-11-06T03:11:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 >> v1:172.21.15.157:6789/0 conn(0x7f7d1815ea60 legacy=0x7f7d1815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d00003f30 0x7f7d000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d00004d10 0x7f7d00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180704b0 con 0x7f7d00004d10 2023-11-06T03:11:44.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0000bb20 con 0x7f7d00003f30 2023-11-06T03:11:44.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0000bce0 con 0x7f7d1815e5e0 2023-11-06T03:11:44.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d00004d10 0x7f7d00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1cf2e640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d17fff640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d00003f30 0x7f7d000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 -- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 msgr2=0x7f7d000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d1815e5e0 0x7f7d000039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 -- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d00003f30 msgr2=0x7f7d000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d00003f30 0x7f7d000047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.962+0000 7f7d1d72f640 1 -- 172.21.15.157:0/2399684219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00003530 con 0x7f7d00004d10 2023-11-06T03:11:44.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1d72f640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d00004d10 0x7f7d00005190 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000a800 tx=0x7f7d1001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ce460e937d02fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100aa690 con 0x7f7d00004d10 2023-11-06T03:11:44.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d100aa7f0 con 0x7f7d00004d10 2023-11-06T03:11:44.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d177fe640 1 -- 172.21.15.157:0/2399684219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100aabe0 con 0x7f7d00004d10 2023-11-06T03:11:44.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/2399684219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d00004d10 msgr2=0x7f7d00005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 --2- 172.21.15.157:0/2399684219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d00004d10 0x7f7d00005190 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000a800 tx=0x7f7d1001fa60 comp rx=0 tx=0).stop 2023-11-06T03:11:44.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1d72f640 1 -- 172.21.15.157:0/2399684219 reap_dead start 2023-11-06T03:11:44.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/2399684219 shutdown_connections 2023-11-06T03:11:44.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/2399684219 >> 172.21.15.157:0/2399684219 conn(0x7f7d18065930 msgr2=0x7f7d18065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:44.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/2399684219 shutdown_connections 2023-11-06T03:11:44.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/2399684219 wait complete. 2023-11-06T03:11:44.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 Processor -- start 2023-11-06T03:11:44.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- start start 2023-11-06T03:11:44.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 0x7f7d180cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d1815a010 0x7f7d181418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180ccfa0 con 0x7f7d181f2600 2023-11-06T03:11:44.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18142290 con 0x7f7d1815a010 2023-11-06T03:11:44.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18150640 con 0x7f7d18159c30 2023-11-06T03:11:44.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43046/0 (socket says 172.21.15.157:43046) 2023-11-06T03:11:44.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 -- 172.21.15.157:0/4145072386 learned_addr learned my addr 172.21.15.157:0/4145072386 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:44.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1d72f640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d1815a010 0x7f7d181418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1cf2e640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 0x7f7d180cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 -- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 msgr2=0x7f7d180cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 0x7f7d180cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 -- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d1815a010 msgr2=0x7f7d181418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:44.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d1815a010 0x7f7d181418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 -- 172.21.15.157:0/4145072386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1806cb30 con 0x7f7d181f2600 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1cf2e640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 0x7f7d180cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d17fff640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0108c0 tx=0x7f7d0c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c009630 con 0x7f7d181f2600 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00eb60 con 0x7f7d181f2600 2023-11-06T03:11:44.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c01c400 con 0x7f7d181f2600 2023-11-06T03:11:44.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.966+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1806ce10 con 0x7f7d181f2600 2023-11-06T03:11:44.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1806d3d0 con 0x7f7d181f2600 2023-11-06T03:11:44.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18072750 con 0x7f7d181f2600 2023-11-06T03:11:44.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d0c01c560 con 0x7f7d181f2600 2023-11-06T03:11:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d157fa640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 0x7f7d04067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f7d0c0837e0 con 0x7f7d181f2600 2023-11-06T03:11:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d1cf2e640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 0x7f7d04067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:44.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.970+0000 7f7d1cf2e640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 0x7f7d04067f50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007fd0 tx=0x7f7d08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:44.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:44.974+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d0c051020 con 0x7f7d181f2600 2023-11-06T03:11:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d18071730 con 0x7f7d181f2600 2023-11-06T03:11:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d157fa640 1 -- 172.21.15.157:0/4145072386 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d0c054640 con 0x7f7d181f2600 2023-11-06T03:11:45.126 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:11:45.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 msgr2=0x7f7d04067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 0x7f7d04067f50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007fd0 tx=0x7f7d08009040 comp rx=0 tx=0).stop 2023-11-06T03:11:45.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 msgr2=0x7f7d18141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0108c0 tx=0x7f7d0c010d80 comp rx=0 tx=0).stop 2023-11-06T03:11:45.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 shutdown_connections 2023-11-06T03:11:45.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d04065a90 0x7f7d04067f50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d1815a010 0x7f7d181418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d181f2600 0x7f7d18141370 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 --2- 172.21.15.157:0/4145072386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d18159c30 0x7f7d180cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 >> 172.21.15.157:0/4145072386 conn(0x7f7d18065930 msgr2=0x7f7d181532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 shutdown_connections 2023-11-06T03:11:45.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.126+0000 7f7d1f192640 1 -- 172.21.15.157:0/4145072386 wait complete. 2023-11-06T03:11:45.144 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:11:45.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.266+0000 7f8b7f33e640 1 Processor -- start 2023-11-06T03:11:45.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.266+0000 7f8b7f33e640 1 -- start start 2023-11-06T03:11:45.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7f33e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 0x7f8b780cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7f33e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 0x7f8b7806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7f33e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b780cf1a0 con 0x7f8b78146370 2023-11-06T03:11:45.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7f33e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b780b2120 con 0x7f8b780ce800 2023-11-06T03:11:45.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7f33e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7806c3d0 con 0x7f8b7806b6a0 2023-11-06T03:11:45.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 0x7f8b7806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7c8d9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 0x7f8b780cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7c8d9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 0x7f8b780cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:44182/0 (socket says 172.21.15.157:44182) 2023-11-06T03:11:45.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7c8d9640 1 -- 172.21.15.157:0/2382935822 learned_addr learned my addr 172.21.15.157:0/2382935822 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:45.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 -- 172.21.15.157:0/2382935822 >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 msgr2=0x7f8b780cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 --2- 172.21.15.157:0/2382935822 >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 0x7f8b780cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 -- 172.21.15.157:0/2382935822 >> v1:172.21.15.195:6789/0 conn(0x7f8b78146370 legacy=0x7f8b780b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 -- 172.21.15.157:0/2382935822 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7806f5a0 con 0x7f8b7806b6a0 2023-11-06T03:11:45.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.270+0000 7f8b7d8db640 1 --2- 172.21.15.157:0/2382935822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 0x7f8b7806bb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c21f9b6e4c77f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6ffff640 1 -- 172.21.15.157:0/2382935822 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700a5950 con 0x7f8b7806b6a0 2023-11-06T03:11:45.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6ffff640 1 -- 172.21.15.157:0/2382935822 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b700a5ab0 con 0x7f8b7806b6a0 2023-11-06T03:11:45.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6ffff640 1 -- 172.21.15.157:0/2382935822 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700a5e40 con 0x7f8b7806b6a0 2023-11-06T03:11:45.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/2382935822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 msgr2=0x7f8b7806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/2382935822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 0x7f8b7806bb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7001c890 comp rx=0 tx=0).stop 2023-11-06T03:11:45.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/2382935822 shutdown_connections 2023-11-06T03:11:45.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/2382935822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b7806b6a0 0x7f8b7806bb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/2382935822 >> v2:172.21.15.195:3300/0 conn(0x7f8b780ce800 0x7f8b780cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/2382935822 >> 172.21.15.157:0/2382935822 conn(0x7f8b78065930 msgr2=0x7f8b78065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/2382935822 shutdown_connections 2023-11-06T03:11:45.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/2382935822 wait complete. 2023-11-06T03:11:45.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 Processor -- start 2023-11-06T03:11:45.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- start start 2023-11-06T03:11:45.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7806b6a0 0x7f8b78129050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 0x7f8b7815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b781f52e0 con 0x7f8b780ce800 2023-11-06T03:11:45.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b781f5e80 con 0x7f8b7806b6a0 2023-11-06T03:11:45.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78142020 con 0x7f8b78146370 2023-11-06T03:11:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43062/0 (socket says 172.21.15.157:43062) 2023-11-06T03:11:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 -- 172.21.15.157:0/565129861 learned_addr learned my addr 172.21.15.157:0/565129861 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7d0da640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7806b6a0 0x7f8b78129050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7d8db640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 0x7f8b7815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 -- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 msgr2=0x7f8b7815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 0x7f8b7815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 -- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7806b6a0 msgr2=0x7f8b78129050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7806b6a0 0x7f8b78129050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 -- 172.21.15.157:0/565129861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009a70 con 0x7f8b780ce800 2023-11-06T03:11:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7d8db640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 0x7f8b7815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7c8d9640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e8c0 tx=0x7f8b6800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68009620 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800c940 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b78142300 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800ccf0 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.274+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7815f890 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b780cec40 con 0x7f8b780ce800 2023-11-06T03:11:45.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8b68013470 con 0x7f8b780ce800 2023-11-06T03:11:45.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b6dffb640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 0x7f8b5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f8b68083490 con 0x7f8b780ce800 2023-11-06T03:11:45.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b7d0da640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 0x7f8b5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.278+0000 7f8b7d0da640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 0x7f8b5c067f20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8b78141370 tx=0x7f8b74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.282+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8b68050cd0 con 0x7f8b780ce800 2023-11-06T03:11:45.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.434+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b78072750 con 0x7f8b780ce800 2023-11-06T03:11:45.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b6dffb640 1 -- 172.21.15.157:0/565129861 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b680542f0 con 0x7f8b780ce800 2023-11-06T03:11:45.437 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:11:45.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 msgr2=0x7f8b5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 0x7f8b5c067f20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8b78141370 tx=0x7f8b74009040 comp rx=0 tx=0).stop 2023-11-06T03:11:45.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 msgr2=0x7f8b78159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800e8c0 tx=0x7f8b6800ed80 comp rx=0 tx=0).stop 2023-11-06T03:11:45.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 shutdown_connections 2023-11-06T03:11:45.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8b5c065a60 0x7f8b5c067f20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8b78146370 0x7f8b7815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b780ce800 0x7f8b78159c70 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 --2- 172.21.15.157:0/565129861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b7806b6a0 0x7f8b78129050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 >> 172.21.15.157:0/565129861 conn(0x7f8b78065930 msgr2=0x7f8b780b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 shutdown_connections 2023-11-06T03:11:45.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.438+0000 7f8b7f33e640 1 -- 172.21.15.157:0/565129861 wait complete. 2023-11-06T03:11:45.456 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:11:45.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 Processor -- start 2023-11-06T03:11:45.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 -- start start 2023-11-06T03:11:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 0x7f8d2015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d20161780 con 0x7f8d20160490 2023-11-06T03:11:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d200cd860 con 0x7f8d20160950 2023-11-06T03:11:45.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d2625b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200cdb40 con 0x7f8d20160db0 2023-11-06T03:11:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8d20160490 0x7f8d200cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43248/0 (socket says 172.21.15.157:43248) 2023-11-06T03:11:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1f7fe640 1 -- 172.21.15.157:0/3627615721 learned_addr learned my addr 172.21.15.157:0/3627615721 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1ffff640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 0x7f8d2015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445525403 0 0) 0x7f8d20161780 con 0x7f8d20160490 2023-11-06T03:11:45.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d08003530 con 0x7f8d20160490 2023-11-06T03:11:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 840569826 0 0) 0x7f8d08003530 con 0x7f8d20160490 2023-11-06T03:11:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 msgr2=0x7f8d2015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 0x7f8d2015ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 >> v1:172.21.15.195:6789/0 conn(0x7f8d20160950 legacy=0x7f8d2015e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200cde20 con 0x7f8d20160490 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8d10001680 con 0x7f8d20160490 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 >> v1:172.21.15.157:6789/0 conn(0x7f8d20160490 legacy=0x7f8d200cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080039b0 0x7f8d08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 0x7f8d08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200cdb40 con 0x7f8d080047d0 2023-11-06T03:11:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0800ba70 con 0x7f8d080039b0 2023-11-06T03:11:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0800bc30 con 0x7f8d20160db0 2023-11-06T03:11:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1f7fe640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080039b0 0x7f8d08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 -- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 msgr2=0x7f8d08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20160db0 0x7f8d08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 -- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080039b0 msgr2=0x7f8d08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d080039b0 0x7f8d08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.578+0000 7f8d1effd640 1 -- 172.21.15.157:0/3627615721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d08003530 con 0x7f8d080047d0 2023-11-06T03:11:45.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d08004c50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8d14001750 tx=0x7f8d1401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719638c3b47a2fc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140052d0 con 0x7f8d080047d0 2023-11-06T03:11:45.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d140a7050 con 0x7f8d080047d0 2023-11-06T03:11:45.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1e7fc640 1 -- 172.21.15.157:0/3627615721 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14003600 con 0x7f8d080047d0 2023-11-06T03:11:45.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- 172.21.15.157:0/3627615721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 msgr2=0x7f8d08004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3627615721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d08004c50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8d14001750 tx=0x7f8d1401c3a0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1ffff640 1 -- 172.21.15.157:0/3627615721 reap_dead start 2023-11-06T03:11:45.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- 172.21.15.157:0/3627615721 shutdown_connections 2023-11-06T03:11:45.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- 172.21.15.157:0/3627615721 >> 172.21.15.157:0/3627615721 conn(0x7f8d20070790 msgr2=0x7f8d20070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- 172.21.15.157:0/3627615721 shutdown_connections 2023-11-06T03:11:45.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- 172.21.15.157:0/3627615721 wait complete. 2023-11-06T03:11:45.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 Processor -- start 2023-11-06T03:11:45.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- start start 2023-11-06T03:11:45.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c8190 0x7f8d2015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20065930 0x7f8d20065db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015a050 con 0x7f8d080047d0 2023-11-06T03:11:45.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200662f0 con 0x7f8d200c8190 2023-11-06T03:11:45.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d2625b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2015bc90 con 0x7f8d20065930 2023-11-06T03:11:45.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43068/0 (socket says 172.21.15.157:43068) 2023-11-06T03:11:45.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1f7fe640 1 -- 172.21.15.157:0/3529499925 learned_addr learned my addr 172.21.15.157:0/3529499925 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:45.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c8190 0x7f8d2015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.582+0000 7f8d1ffff640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20065930 0x7f8d20065db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 -- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20065930 msgr2=0x7f8d20065db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20065930 0x7f8d20065db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 -- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c8190 msgr2=0x7f8d2015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c8190 0x7f8d2015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 -- 172.21.15.157:0/3529499925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2015bf70 con 0x7f8d080047d0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1f7fe640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8d10011810 tx=0x7f8d10011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000fb00 con 0x7f8d080047d0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000fc60 con 0x7f8d080047d0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2015c250 con 0x7f8d080047d0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10015410 con 0x7f8d080047d0 2023-11-06T03:11:45.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2015c810 con 0x7f8d080047d0 2023-11-06T03:11:45.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20072750 con 0x7f8d080047d0 2023-11-06T03:11:45.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d10015590 con 0x7f8d080047d0 2023-11-06T03:11:45.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 0x7f8d0c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f8d10086620 con 0x7f8d080047d0 2023-11-06T03:11:45.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 0x7f8d0c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.586+0000 7f8d1effd640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 0x7f8d0c067f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8d080057b0 tx=0x7f8d140011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.590+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d10053e60 con 0x7f8d080047d0 2023-11-06T03:11:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.730+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d20157a20 con 0x7f8d080047d0 2023-11-06T03:11:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d07fff640 1 -- 172.21.15.157:0/3529499925 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d10057480 con 0x7f8d080047d0 2023-11-06T03:11:45.733 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:11:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 msgr2=0x7f8d0c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 0x7f8d0c067f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8d080057b0 tx=0x7f8d140011a0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 msgr2=0x7f8d20159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8d10011810 tx=0x7f8d10011cd0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 shutdown_connections 2023-11-06T03:11:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d0c065a60 0x7f8d0c067f20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d20065930 0x7f8d20065db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d200c8190 0x7f8d2015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 --2- 172.21.15.157:0/3529499925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d080047d0 0x7f8d20159b10 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 >> 172.21.15.157:0/3529499925 conn(0x7f8d20070790 msgr2=0x7f8d200b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 shutdown_connections 2023-11-06T03:11:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.734+0000 7f8d2625b640 1 -- 172.21.15.157:0/3529499925 wait complete. 2023-11-06T03:11:45.750 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:11:45.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 Processor -- start 2023-11-06T03:11:45.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 -- start start 2023-11-06T03:11:45.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f151406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f151415e5e0 0x7f15140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151415f580 con 0x7f151415ea60 2023-11-06T03:11:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151415efc0 con 0x7f151415e5e0 2023-11-06T03:11:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f151ad2d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151415f2a0 con 0x7f15141551e0 2023-11-06T03:11:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f151415ea60 0x7f151415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43250/0 (socket says 172.21.15.157:43250) 2023-11-06T03:11:45.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 -- 172.21.15.157:0/3504565140 learned_addr learned my addr 172.21.15.157:0/3504565140 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:45.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f1518ac9640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f151406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041228347 0 0) 0x7f151415f580 con 0x7f151415ea60 2023-11-06T03:11:45.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f1513fff640 1 --2- 172.21.15.157:0/3504565140 >> v2:172.21.15.195:3300/0 conn(0x7f151415e5e0 0x7f15140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14fc003530 con 0x7f151415ea60 2023-11-06T03:11:45.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569469131 0 0) 0x7f14fc003530 con 0x7f151415ea60 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 msgr2=0x7f151406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f151406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 >> v2:172.21.15.195:3300/0 conn(0x7f151415e5e0 msgr2=0x7f15140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 --2- 172.21.15.157:0/3504565140 >> v2:172.21.15.195:3300/0 conn(0x7f151415e5e0 0x7f15140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151406f5a0 con 0x7f151415ea60 2023-11-06T03:11:45.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f150c001a00 con 0x7f151415ea60 2023-11-06T03:11:45.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 >> v1:172.21.15.157:6789/0 conn(0x7f151415ea60 legacy=0x7f151415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14fc003980 0x7f14fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f14fc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14fc004d10 0x7f14fc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151415efc0 con 0x7f14fc004d10 2023-11-06T03:11:45.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151415f2a0 con 0x7f14fc003980 2023-11-06T03:11:45.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc00b990 con 0x7f15141551e0 2023-11-06T03:11:45.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14fc004d10 0x7f14fc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f1513fff640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f14fc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f1518ac9640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14fc003980 0x7f14fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 -- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 msgr2=0x7f14fc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141551e0 0x7f14fc0047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 -- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14fc003980 msgr2=0x7f14fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:45.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14fc003980 0x7f14fc003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.862+0000 7f15192ca640 1 -- 172.21.15.157:0/3504565140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc003530 con 0x7f14fc004d10 2023-11-06T03:11:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f15192ca640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14fc004d10 0x7f14fc005190 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a830 tx=0x7f150c01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312ff5c4c2a18fb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0aa6d0 con 0x7f14fc004d10 2023-11-06T03:11:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c0aa830 con 0x7f14fc004d10 2023-11-06T03:11:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f15137fe640 1 -- 172.21.15.157:0/3504565140 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0aac00 con 0x7f14fc004d10 2023-11-06T03:11:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- 172.21.15.157:0/3504565140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14fc004d10 msgr2=0x7f14fc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3504565140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14fc004d10 0x7f14fc005190 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a830 tx=0x7f150c01faa0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f15192ca640 1 -- 172.21.15.157:0/3504565140 reap_dead start 2023-11-06T03:11:45.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- 172.21.15.157:0/3504565140 shutdown_connections 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- 172.21.15.157:0/3504565140 >> 172.21.15.157:0/3504565140 conn(0x7f1514065930 msgr2=0x7f1514065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- 172.21.15.157:0/3504565140 shutdown_connections 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- 172.21.15.157:0/3504565140 wait complete. 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 Processor -- start 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- start start 2023-11-06T03:11:45.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1514150870 0x7f1514150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141555c0 0x7f1514155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514155fb0 con 0x7f15141574c0 2023-11-06T03:11:45.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514153610 con 0x7f1514150870 2023-11-06T03:11:45.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f151ad2d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15141538f0 con 0x7f15141555c0 2023-11-06T03:11:45.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f1513fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f1513fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43080/0 (socket says 172.21.15.157:43080) 2023-11-06T03:11:45.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f1513fff640 1 -- 172.21.15.157:0/3213421021 learned_addr learned my addr 172.21.15.157:0/3213421021 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:45.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.866+0000 7f1518ac9640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1514150870 0x7f1514150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15192ca640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141555c0 0x7f1514155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 -- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141555c0 msgr2=0x7f1514155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141555c0 0x7f1514155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 -- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1514150870 msgr2=0x7f1514150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:45.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1514150870 0x7f1514150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:45.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 -- 172.21.15.157:0/3213421021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514158d50 con 0x7f15141574c0 2023-11-06T03:11:45.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1513fff640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1504010900 tx=0x7f1504010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040096b0 con 0x7f15141574c0 2023-11-06T03:11:45.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150400ec20 con 0x7f15141574c0 2023-11-06T03:11:45.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150401c400 con 0x7f15141574c0 2023-11-06T03:11:45.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1514159030 con 0x7f15141574c0 2023-11-06T03:11:45.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1514150c50 con 0x7f15141574c0 2023-11-06T03:11:45.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1514158bb0 con 0x7f15141574c0 2023-11-06T03:11:45.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f150401c560 con 0x7f15141574c0 2023-11-06T03:11:45.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:45.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f1504083920 con 0x7f15141574c0 2023-11-06T03:11:45.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1518ac9640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:45.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.870+0000 7f1518ac9640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1508009a30 tx=0x7f1508008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:45.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:45.874+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1514158bb0 con 0x7f15141574c0 2023-11-06T03:11:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1514158bb0 con 0x7f15141574c0 2023-11-06T03:11:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f15117fa640 1 -- 172.21.15.157:0/3213421021 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1514158bb0 con 0x7f15141574c0 2023-11-06T03:11:46.030 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:11:46.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 msgr2=0x7f1500067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1508009a30 tx=0x7f1508008040 comp rx=0 tx=0).stop 2023-11-06T03:11:46.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 msgr2=0x7f1514152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1504010900 tx=0x7f1504010dc0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 shutdown_connections 2023-11-06T03:11:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15141555c0 0x7f1514155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15141574c0 0x7f1514152ca0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 --2- 172.21.15.157:0/3213421021 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1514150870 0x7f1514150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 >> 172.21.15.157:0/3213421021 conn(0x7f1514065930 msgr2=0x7f151406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 shutdown_connections 2023-11-06T03:11:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.030+0000 7f151ad2d640 1 -- 172.21.15.157:0/3213421021 wait complete. 2023-11-06T03:11:46.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:11:46.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.162+0000 7f39fef6c640 1 Processor -- start 2023-11-06T03:11:46.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.162+0000 7f39fef6c640 1 -- start start 2023-11-06T03:11:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fef6c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39f8065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fef6c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f39f8065e70 0x7f39f806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fef6c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80cf820 con 0x7f39f806d100 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fef6c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f806d8f0 con 0x7f39f8065e70 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fef6c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8161740 con 0x7f39f8157070 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fd509640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f39f806d100 0x7f39f8070340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43262/0 (socket says 172.21.15.157:43262) 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fd509640 1 -- 172.21.15.157:0/78044962 learned_addr learned my addr 172.21.15.157:0/78044962 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003276892 0 0) 0x7f39f80cf820 con 0x7f39f806d100 2023-11-06T03:11:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e4003530 con 0x7f39f806d100 2023-11-06T03:11:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39f8065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f7fff640 1 --2- 172.21.15.157:0/78044962 >> v2:172.21.15.195:3300/0 conn(0x7f39f8065e70 0x7f39f806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1502001214 0 0) 0x7f39e4003530 con 0x7f39f806d100 2023-11-06T03:11:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 msgr2=0x7f39f8065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39f8065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 >> v2:172.21.15.195:3300/0 conn(0x7f39f8065e70 msgr2=0x7f39f806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 --2- 172.21.15.157:0/78044962 >> v2:172.21.15.195:3300/0 conn(0x7f39f8065e70 0x7f39f806cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f815e150 con 0x7f39f806d100 2023-11-06T03:11:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f39f0001a00 con 0x7f39f806d100 2023-11-06T03:11:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f7fff640 1 --2- 172.21.15.157:0/78044962 >> v2:172.21.15.195:3300/0 conn(0x7f39f8065e70 0x7f39f806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 >> v1:172.21.15.157:6789/0 conn(0x7f39f806d100 legacy=0x7f39f8070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4003980 0x7f39e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e4004d10 0x7f39e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f806d8f0 con 0x7f39e4003980 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8161740 con 0x7f39e4004d10 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e400baf0 con 0x7f39f8157070 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4003980 0x7f39e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39f7fff640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 -- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 msgr2=0x7f39e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39f8157070 0x7f39e40047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 -- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e4004d10 msgr2=0x7f39e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39e4004d10 0x7f39e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.166+0000 7f39fcd08640 1 -- 172.21.15.157:0/78044962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4003530 con 0x7f39e4003980 2023-11-06T03:11:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fcd08640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4003980 0x7f39e4003da0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f39e800ba30 tx=0x7f39e801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec42eb088bebcb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80ae410 con 0x7f39e4003980 2023-11-06T03:11:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e80a8030 con 0x7f39e4003980 2023-11-06T03:11:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f77fe640 1 -- 172.21.15.157:0/78044962 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e80ae7b0 con 0x7f39e4003980 2023-11-06T03:11:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/78044962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4003980 msgr2=0x7f39e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 --2- 172.21.15.157:0/78044962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4003980 0x7f39e4003da0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f39e800ba30 tx=0x7f39e801c680 comp rx=0 tx=0).stop 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 -- 172.21.15.157:0/78044962 reap_dead start 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/78044962 shutdown_connections 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/78044962 >> 172.21.15.157:0/78044962 conn(0x7f39f8069d70 msgr2=0x7f39f806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/78044962 shutdown_connections 2023-11-06T03:11:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/78044962 wait complete. 2023-11-06T03:11:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 Processor -- start 2023-11-06T03:11:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- start start 2023-11-06T03:11:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e4003980 0x7f39f8143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39f81f60f0 0x7f39f80cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81468f0 con 0x7f39f81f6570 2023-11-06T03:11:46.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81439c0 con 0x7f39f81f60f0 2023-11-06T03:11:46.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8145fc0 con 0x7f39e4003980 2023-11-06T03:11:46.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43094/0 (socket says 172.21.15.157:43094) 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 -- 172.21.15.157:0/314603098 learned_addr learned my addr 172.21.15.157:0/314603098 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f7fff640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39f81f60f0 0x7f39f80cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fcd08640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e4003980 0x7f39f8143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 -- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e4003980 msgr2=0x7f39f8143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e4003980 0x7f39f8143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 -- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39f81f60f0 msgr2=0x7f39f80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39f81f60f0 0x7f39f80cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 -- 172.21.15.157:0/314603098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f81462a0 con 0x7f39f81f6570 2023-11-06T03:11:46.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fd509640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f39f00012f0 tx=0x7f39f000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000a610 con 0x7f39f81f6570 2023-11-06T03:11:46.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f000a7b0 con 0x7f39f81f6570 2023-11-06T03:11:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8146580 con 0x7f39f81f6570 2023-11-06T03:11:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000ab60 con 0x7f39f81f6570 2023-11-06T03:11:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.170+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81f5c10 con 0x7f39f81f6570 2023-11-06T03:11:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f806ad00 con 0x7f39f81f6570 2023-11-06T03:11:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39f000acc0 con 0x7f39f81f6570 2023-11-06T03:11:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39f57fa640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 0x7f39d8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f39f00830e0 con 0x7f39f81f6570 2023-11-06T03:11:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39fcd08640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 0x7f39d8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.174+0000 7f39fcd08640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 0x7f39d8067f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f39e80b0000 tx=0x7f39e80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.178+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39f0050910 con 0x7f39f81f6570 2023-11-06T03:11:46.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39f8072750 con 0x7f39f81f6570 2023-11-06T03:11:46.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39f57fa640 1 -- 172.21.15.157:0/314603098 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f39f0053f30 con 0x7f39f81f6570 2023-11-06T03:11:46.354 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:46.354 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:11:23.420288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":30,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:11:46.354 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-11-06T03:11:46.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 msgr2=0x7f39d8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 0x7f39d8067f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f39e80b0000 tx=0x7f39e80046d0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 msgr2=0x7f39f80cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f39f00012f0 tx=0x7f39f000ce80 comp rx=0 tx=0).stop 2023-11-06T03:11:46.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 shutdown_connections 2023-11-06T03:11:46.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39d8065a90 0x7f39d8067f50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39f81f6570 0x7f39f80cd6f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39f81f60f0 0x7f39f80cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 --2- 172.21.15.157:0/314603098 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39e4003980 0x7f39f8143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 >> 172.21.15.157:0/314603098 conn(0x7f39f8069d70 msgr2=0x7f39f8156350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 shutdown_connections 2023-11-06T03:11:46.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.354+0000 7f39fef6c640 1 -- 172.21.15.157:0/314603098 wait complete. 2023-11-06T03:11:46.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:46.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.482+0000 7f293e877640 1 Processor -- start 2023-11-06T03:11:46.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.482+0000 7f293e877640 1 -- start start 2023-11-06T03:11:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293e877640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29380b2af0 0x7f29380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293e877640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2938159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293e877640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293815e440 con 0x7f29381599c0 2023-11-06T03:11:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293e877640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293806c240 con 0x7f29380b2af0 2023-11-06T03:11:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293e877640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293806c520 con 0x7f293806d000 2023-11-06T03:11:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f2937fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f29381599c0 0x7f29380b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43266/0 (socket says 172.21.15.157:43266) 2023-11-06T03:11:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f2937fff640 1 -- 172.21.15.157:0/470523649 learned_addr learned my addr 172.21.15.157:0/470523649 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709966909 0 0) 0x7f293815e440 con 0x7f29381599c0 2023-11-06T03:11:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2938159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2924003530 con 0x7f29381599c0 2023-11-06T03:11:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f292ffff640 1 --2- 172.21.15.157:0/470523649 >> v2:172.21.15.195:3300/0 conn(0x7f29380b2af0 0x7f29380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4229135518 0 0) 0x7f2924003530 con 0x7f29381599c0 2023-11-06T03:11:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 msgr2=0x7f2938159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2938159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 >> v2:172.21.15.195:3300/0 conn(0x7f29380b2af0 msgr2=0x7f29380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 --2- 172.21.15.157:0/470523649 >> v2:172.21.15.195:3300/0 conn(0x7f29380b2af0 0x7f29380ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293806c800 con 0x7f29381599c0 2023-11-06T03:11:46.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2928001680 con 0x7f29381599c0 2023-11-06T03:11:46.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2938159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:46.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 >> v1:172.21.15.157:6789/0 conn(0x7f29381599c0 legacy=0x7f29380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2924003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2924003f70 0x7f29240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2924004d10 0x7f2924005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293806c240 con 0x7f2924004d10 2023-11-06T03:11:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293806c520 con 0x7f2924003f70 2023-11-06T03:11:46.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292400dc80 con 0x7f293806d000 2023-11-06T03:11:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f292ffff640 1 --2- 172.21.15.157:0/470523649 >> v2:172.21.15.195:3300/0 conn(0x7f29380b2af0 0x7f29380ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2924004d10 0x7f2924005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f2937fff640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2924003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 -- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 msgr2=0x7f2924003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f293806d000 0x7f2924003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 -- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2924003f70 msgr2=0x7f29240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2924003f70 0x7f29240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 -- 172.21.15.157:0/470523649 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924003530 con 0x7f2924004d10 2023-11-06T03:11:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.486+0000 7f293ce14640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2924004d10 0x7f2924005190 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f29380b19c0 tx=0x7f293001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763cdec9539b4446 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300a5d40 con 0x7f2924004d10 2023-11-06T03:11:46.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29300039a0 con 0x7f2924004d10 2023-11-06T03:11:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f29377fe640 1 -- 172.21.15.157:0/470523649 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300a5ea0 con 0x7f2924004d10 2023-11-06T03:11:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 -- 172.21.15.157:0/470523649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2924004d10 msgr2=0x7f2924005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 --2- 172.21.15.157:0/470523649 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2924004d10 0x7f2924005190 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f29380b19c0 tx=0x7f293001ce40 comp rx=0 tx=0).stop 2023-11-06T03:11:46.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293ce14640 1 -- 172.21.15.157:0/470523649 reap_dead start 2023-11-06T03:11:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 -- 172.21.15.157:0/470523649 shutdown_connections 2023-11-06T03:11:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 -- 172.21.15.157:0/470523649 >> 172.21.15.157:0/470523649 conn(0x7f2938065930 msgr2=0x7f2938065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 -- 172.21.15.157:0/470523649 shutdown_connections 2023-11-06T03:11:46.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.490+0000 7f293e877640 1 -- 172.21.15.157:0/470523649 wait complete. 2023-11-06T03:11:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 Processor -- start 2023-11-06T03:11:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- start start 2023-11-06T03:11:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 0x7f29380cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 0x7f29380ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380cf0c0 con 0x7f293815ac00 2023-11-06T03:11:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380ccfe0 con 0x7f293815a820 2023-11-06T03:11:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938153db0 con 0x7f29380ccb10 2023-11-06T03:11:46.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43116/0 (socket says 172.21.15.157:43116) 2023-11-06T03:11:46.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 -- 172.21.15.157:0/35479581 learned_addr learned my addr 172.21.15.157:0/35479581 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:46.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293ce14640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 0x7f29380ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f2937fff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 0x7f29380cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 -- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 msgr2=0x7f29380ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 0x7f29380ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 -- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 msgr2=0x7f29380cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 0x7f29380cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 -- 172.21.15.157:0/35479581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938154090 con 0x7f293815ac00 2023-11-06T03:11:46.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293ce14640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 0x7f29380ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:46.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f2937fff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 0x7f29380cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:46.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f292ffff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f292000e810 tx=0x7f292000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920004020 con 0x7f293815ac00 2023-11-06T03:11:46.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2920004180 con 0x7f293815ac00 2023-11-06T03:11:46.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2938154370 con 0x7f293815ac00 2023-11-06T03:11:46.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000c6d0 con 0x7f293815ac00 2023-11-06T03:11:46.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938159da0 con 0x7f293815ac00 2023-11-06T03:11:46.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938072750 con 0x7f293815ac00 2023-11-06T03:11:46.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f292000c830 con 0x7f293815ac00 2023-11-06T03:11:46.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 0x7f2918067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f2920082d50 con 0x7f293815ac00 2023-11-06T03:11:46.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.494+0000 7f2937fff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 0x7f2918067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.498+0000 7f2937fff640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 0x7f2918067f50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29280079e0 tx=0x7f2928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.498+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2920086030 con 0x7f293815ac00 2023-11-06T03:11:46.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.654+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29380cb110 con 0x7f293815ac00 2023-11-06T03:11:46.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.658+0000 7f29357fa640 1 -- 172.21.15.157:0/35479581 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18000 (secure 0 0 0) 0x7f2920050590 con 0x7f293815ac00 2023-11-06T03:11:46.660 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:46.660 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:22.343214+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 msgr2=0x7f2918067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 0x7f2918067f50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29280079e0 tx=0x7f2928009040 comp rx=0 tx=0).stop 2023-11-06T03:11:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 msgr2=0x7f29380ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f292000e810 tx=0x7f292000ecd0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 shutdown_connections 2023-11-06T03:11:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2918065a90 0x7f2918067f50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29380ccb10 0x7f29380ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f293815ac00 0x7f29380ce640 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 --2- 172.21.15.157:0/35479581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f293815a820 0x7f29380cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 >> 172.21.15.157:0/35479581 conn(0x7f2938065930 msgr2=0x7f29380aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 shutdown_connections 2023-11-06T03:11:46.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.662+0000 7f293e877640 1 -- 172.21.15.157:0/35479581 wait complete. 2023-11-06T03:11:46.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:11:46.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:11:46.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.794+0000 7f6e62c5e640 1 Processor -- start 2023-11-06T03:11:46.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.794+0000 7f6e62c5e640 1 -- start start 2023-11-06T03:11:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e62c5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c1599c0 0x7f6e5c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e62c5e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e5c0b2af0 0x7f6e5c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e62c5e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5c15e440 con 0x7f6e5c06d000 2023-11-06T03:11:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e62c5e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c06c240 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e62c5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c06c520 con 0x7f6e5c1599c0 2023-11-06T03:11:46.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e5c0b2af0 0x7f6e5c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e609fa640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c1599c0 0x7f6e5c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e5c0b2af0 0x7f6e5c159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:44284/0 (socket says 172.21.15.157:44284) 2023-11-06T03:11:46.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 -- 172.21.15.157:0/3654300514 learned_addr learned my addr 172.21.15.157:0/3654300514 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:46.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 -- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c1599c0 msgr2=0x7f6e5c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:46.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 --2- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c1599c0 0x7f6e5c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 -- 172.21.15.157:0/3654300514 >> v1:172.21.15.195:6789/0 conn(0x7f6e5c06d000 legacy=0x7f6e5c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 -- 172.21.15.157:0/3654300514 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c06c800 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.798+0000 7f6e611fb640 1 --2- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c0b2af0 0x7f6e5c159da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6812d61fb908ca7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e5b7fe640 1 -- 172.21.15.157:0/3654300514 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a95f0 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e5b7fe640 1 -- 172.21.15.157:0/3654300514 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e540a9750 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e5b7fe640 1 -- 172.21.15.157:0/3654300514 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a9ae0 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c0b2af0 msgr2=0x7f6e5c159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c0b2af0 0x7f6e5c159da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5401c510 comp rx=0 tx=0).stop 2023-11-06T03:11:46.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/3654300514 shutdown_connections 2023-11-06T03:11:46.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c0b2af0 0x7f6e5c159da0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/3654300514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c1599c0 0x7f6e5c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/3654300514 >> 172.21.15.157:0/3654300514 conn(0x7f6e5c065930 msgr2=0x7f6e5c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/3654300514 shutdown_connections 2023-11-06T03:11:46.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/3654300514 wait complete. 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 Processor -- start 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- start start 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c0b2af0 0x7f6e5c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c1599c0 0x7f6e5c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c155960 con 0x7f6e5c06d000 2023-11-06T03:11:46.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c1550c0 con 0x7f6e5c1599c0 2023-11-06T03:11:46.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c1553a0 con 0x7f6e5c0b2af0 2023-11-06T03:11:46.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43132/0 (socket says 172.21.15.157:43132) 2023-11-06T03:11:46.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 -- 172.21.15.157:0/501658524 learned_addr learned my addr 172.21.15.157:0/501658524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e611fb640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c1599c0 0x7f6e5c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e5bfff640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c0b2af0 0x7f6e5c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 -- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c0b2af0 msgr2=0x7f6e5c152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c0b2af0 0x7f6e5c152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 -- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c1599c0 msgr2=0x7f6e5c152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c1599c0 0x7f6e5c152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 -- 172.21.15.157:0/501658524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540094d0 con 0x7f6e5c06d000 2023-11-06T03:11:46.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e609fa640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00efc0 tx=0x7f6e4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c009b80 con 0x7f6e5c06d000 2023-11-06T03:11:46.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00f040 con 0x7f6e5c06d000 2023-11-06T03:11:46.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.802+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c155680 con 0x7f6e5c06d000 2023-11-06T03:11:46.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0146a0 con 0x7f6e5c06d000 2023-11-06T03:11:46.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c154d40 con 0x7f6e5c06d000 2023-11-06T03:11:46.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c15b550 con 0x7f6e5c06d000 2023-11-06T03:11:46.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6e4c014800 con 0x7f6e5c06d000 2023-11-06T03:11:46.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e597fa640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 0x7f6e3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:46.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f6e4c083d10 con 0x7f6e5c06d000 2023-11-06T03:11:46.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e5bfff640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 0x7f6e3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:46.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.806+0000 7f6e5bfff640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 0x7f6e3c067f20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c1f6d30 tx=0x7f6e44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:46.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.810+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6e4c0515e0 con 0x7f6e5c06d000 2023-11-06T03:11:46.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.982+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e5c1531b0 con 0x7f6e5c06d000 2023-11-06T03:11:46.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.982+0000 7f6e597fa640 1 -- 172.21.15.157:0/501658524 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f6e4c054c00 con 0x7f6e5c06d000 2023-11-06T03:11:46.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:46.984 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:11:23.420288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":30,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:11:46.984 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-11-06T03:11:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 msgr2=0x7f6e3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 0x7f6e3c067f20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c1f6d30 tx=0x7f6e44009040 comp rx=0 tx=0).stop 2023-11-06T03:11:46.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 msgr2=0x7f6e5c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:46.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00efc0 tx=0x7f6e4c00c490 comp rx=0 tx=0).stop 2023-11-06T03:11:46.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 shutdown_connections 2023-11-06T03:11:46.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e3c065a60 0x7f6e3c067f20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e5c1599c0 0x7f6e5c152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e5c0b2af0 0x7f6e5c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 --2- 172.21.15.157:0/501658524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e5c06d000 0x7f6e5c154800 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:46.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 >> 172.21.15.157:0/501658524 conn(0x7f6e5c065930 msgr2=0x7f6e5c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:46.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 shutdown_connections 2023-11-06T03:11:46.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:46.986+0000 7f6e62c5e640 1 -- 172.21.15.157:0/501658524 wait complete. 2023-11-06T03:11:47.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:11:47.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 Processor -- start 2023-11-06T03:11:47.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 -- start start 2023-11-06T03:11:47.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a4815caa0 0x7f4a480cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a4815ce80 0x7f4a480cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480d2920 con 0x7f4a480ce1a0 2023-11-06T03:11:47.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480d18a0 con 0x7f4a4815caa0 2023-11-06T03:11:47.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4cbb9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480d1b60 con 0x7f4a4815ce80 2023-11-06T03:11:47.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a4815caa0 0x7f4a480cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a4815caa0 0x7f4a480cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43134/0 (socket says 172.21.15.157:43134) 2023-11-06T03:11:47.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 -- 172.21.15.157:0/2764091620 learned_addr learned my addr 172.21.15.157:0/2764091620 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:47.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 -- 172.21.15.157:0/2764091620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a4815ce80 msgr2=0x7f4a480cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 --2- 172.21.15.157:0/2764091620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a4815ce80 0x7f4a480cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 -- 172.21.15.157:0/2764091620 >> v1:172.21.15.195:6789/0 conn(0x7f4a480ce1a0 legacy=0x7f4a480d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 -- 172.21.15.157:0/2764091620 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480d1e40 con 0x7f4a4815caa0 2023-11-06T03:11:47.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a4659c640 1 --2- 172.21.15.157:0/2764091620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815caa0 0x7f4a480cdc60 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009870 tx=0x7f4a3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e754a7a3b044df2f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a45d9b640 1 -- 172.21.15.157:0/2764091620 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380ac600 con 0x7f4a4815caa0 2023-11-06T03:11:47.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a45d9b640 1 -- 172.21.15.157:0/2764091620 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a380a8050 con 0x7f4a4815caa0 2023-11-06T03:11:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.110+0000 7f4a45d9b640 1 -- 172.21.15.157:0/2764091620 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380ac980 con 0x7f4a4815caa0 2023-11-06T03:11:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/2764091620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815caa0 msgr2=0x7f4a480cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/2764091620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815caa0 0x7f4a480cdc60 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009870 tx=0x7f4a3801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/2764091620 shutdown_connections 2023-11-06T03:11:47.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/2764091620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a4815ce80 0x7f4a480cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/2764091620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815caa0 0x7f4a480cdc60 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/2764091620 >> 172.21.15.157:0/2764091620 conn(0x7f4a48070790 msgr2=0x7f4a48070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/2764091620 shutdown_connections 2023-11-06T03:11:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/2764091620 wait complete. 2023-11-06T03:11:47.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 Processor -- start 2023-11-06T03:11:47.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- start start 2023-11-06T03:11:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a480ce1a0 0x7f4a48144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4815caa0 0x7f4a480cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480caa10 con 0x7f4a4815ce80 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4815d7f0 con 0x7f4a4815caa0 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48145310 con 0x7f4a480ce1a0 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43142/0 (socket says 172.21.15.157:43142) 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 -- 172.21.15.157:0/1659170610 learned_addr learned my addr 172.21.15.157:0/1659170610 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:47.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 -- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a480ce1a0 msgr2=0x7f4a48144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a480ce1a0 0x7f4a48144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 -- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4815caa0 msgr2=0x7f4a480cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4815caa0 0x7f4a480cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 -- 172.21.15.157:0/1659170610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380094d0 con 0x7f4a4815ce80 2023-11-06T03:11:47.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a46d9d640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b9e0 tx=0x7f4a4000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000cb20 con 0x7f4a4815ce80 2023-11-06T03:11:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000cc80 con 0x7f4a4815ce80 2023-11-06T03:11:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a480ca070 con 0x7f4a4815ce80 2023-11-06T03:11:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40014680 con 0x7f4a4815ce80 2023-11-06T03:11:47.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.114+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a480ca630 con 0x7f4a4815ce80 2023-11-06T03:11:47.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a480b30a0 con 0x7f4a4815ce80 2023-11-06T03:11:47.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a4001d3b0 con 0x7f4a4815ce80 2023-11-06T03:11:47.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a3effd640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 0x7f4a28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f4a40083260 con 0x7f4a4815ce80 2023-11-06T03:11:47.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a4659c640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 0x7f4a28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.118+0000 7f4a4659c640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 0x7f4a28067f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4a380ae000 tx=0x7f4a38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.122+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a40050a90 con 0x7f4a4815ce80 2023-11-06T03:11:47.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.298+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a48072750 con 0x7f4a4815ce80 2023-11-06T03:11:47.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.298+0000 7f4a3effd640 1 -- 172.21.15.157:0/1659170610 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f4a400540b0 con 0x7f4a4815ce80 2023-11-06T03:11:47.300 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:47.300 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":73,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-06T03:09:35.127602+0000","modified":"2023-11-06T03:11:23.420288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7329},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7329":{"gid":7329,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":30,"addr":"172.21.15.157:6835/3882288197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3882288197},{"type":"v1","addr":"172.21.15.157:6835","nonce":3882288197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-06T03:11:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 73 2023-11-06T03:11:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 msgr2=0x7f4a28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 0x7f4a28067f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4a380ae000 tx=0x7f4a38008040 comp rx=0 tx=0).stop 2023-11-06T03:11:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 msgr2=0x7f4a480cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b9e0 tx=0x7f4a4000bea0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 shutdown_connections 2023-11-06T03:11:47.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a28065a60 0x7f4a28067f20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a4815ce80 0x7f4a480cb370 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a4815caa0 0x7f4a480cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 --2- 172.21.15.157:0/1659170610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a480ce1a0 0x7f4a48144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 >> 172.21.15.157:0/1659170610 conn(0x7f4a48070790 msgr2=0x7f4a48070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 shutdown_connections 2023-11-06T03:11:47.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.302+0000 7f4a4cbb9640 1 -- 172.21.15.157:0/1659170610 wait complete. 2023-11-06T03:11:47.319 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:47.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.426+0000 7f35b87e8640 1 Processor -- start 2023-11-06T03:11:47.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.426+0000 7f35b87e8640 1 -- start start 2023-11-06T03:11:47.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b87e8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f35b01599c0 0x7f35b00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b87e8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b006d000 0x7f35b00b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b87e8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b015e440 con 0x7f35b015dea0 2023-11-06T03:11:47.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b87e8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b015adb0 con 0x7f35b006d000 2023-11-06T03:11:47.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b87e8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b006c240 con 0x7f35b01599c0 2023-11-06T03:11:47.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b006d000 0x7f35b00b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35b006d000 0x7f35b00b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43146/0 (socket says 172.21.15.157:43146) 2023-11-06T03:11:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 -- 172.21.15.157:0/2496668112 learned_addr learned my addr 172.21.15.157:0/2496668112 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b6584640 1 --2- 172.21.15.157:0/2496668112 >> v2:172.21.15.195:3300/0 conn(0x7f35b01599c0 0x7f35b00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 -- 172.21.15.157:0/2496668112 >> v2:172.21.15.195:3300/0 conn(0x7f35b01599c0 msgr2=0x7f35b00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 --2- 172.21.15.157:0/2496668112 >> v2:172.21.15.195:3300/0 conn(0x7f35b01599c0 0x7f35b00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 -- 172.21.15.157:0/2496668112 >> v1:172.21.15.195:6789/0 conn(0x7f35b015dea0 legacy=0x7f35b0070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 -- 172.21.15.157:0/2496668112 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b006c520 con 0x7f35b006d000 2023-11-06T03:11:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5d83640 1 --2- 172.21.15.157:0/2496668112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b006d000 0x7f35b00b23e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f35b0066800 tx=0x7f35a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9b4b305260eedc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5582640 1 -- 172.21.15.157:0/2496668112 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00ac640 con 0x7f35b006d000 2023-11-06T03:11:47.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5582640 1 -- 172.21.15.157:0/2496668112 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a00ac7a0 con 0x7f35b006d000 2023-11-06T03:11:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.430+0000 7f35b5582640 1 -- 172.21.15.157:0/2496668112 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00acb30 con 0x7f35b006d000 2023-11-06T03:11:47.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/2496668112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b006d000 msgr2=0x7f35b00b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- 172.21.15.157:0/2496668112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b006d000 0x7f35b00b23e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f35b0066800 tx=0x7f35a001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/2496668112 shutdown_connections 2023-11-06T03:11:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- 172.21.15.157:0/2496668112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b006d000 0x7f35b00b23e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- 172.21.15.157:0/2496668112 >> v2:172.21.15.195:3300/0 conn(0x7f35b01599c0 0x7f35b00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/2496668112 >> 172.21.15.157:0/2496668112 conn(0x7f35b0065930 msgr2=0x7f35b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/2496668112 shutdown_connections 2023-11-06T03:11:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/2496668112 wait complete. 2023-11-06T03:11:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 Processor -- start 2023-11-06T03:11:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- start start 2023-11-06T03:11:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 0x7f35b00ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35b01599c0 0x7f35b014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00ade80 con 0x7f35b015dea0 2023-11-06T03:11:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00ae090 con 0x7f35b006d000 2023-11-06T03:11:47.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00ae370 con 0x7f35b01599c0 2023-11-06T03:11:47.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43158/0 (socket says 172.21.15.157:43158) 2023-11-06T03:11:47.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 -- 172.21.15.157:0/4064301899 learned_addr learned my addr 172.21.15.157:0/4064301899 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:47.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b5d83640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35b01599c0 0x7f35b014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6584640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 0x7f35b00ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 -- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35b01599c0 msgr2=0x7f35b014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35b01599c0 0x7f35b014e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 -- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 msgr2=0x7f35b00ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 0x7f35b00ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 -- 172.21.15.157:0/4064301899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a00094d0 con 0x7f35b015dea0 2023-11-06T03:11:47.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6584640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 0x7f35b00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:47.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b6d85640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c730 tx=0x7f35a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8003b80 con 0x7f35b015dea0 2023-11-06T03:11:47.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8003ce0 con 0x7f35b015dea0 2023-11-06T03:11:47.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800aa30 con 0x7f35b015dea0 2023-11-06T03:11:47.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b00ae650 con 0x7f35b015dea0 2023-11-06T03:11:47.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b00aec10 con 0x7f35b015dea0 2023-11-06T03:11:47.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.434+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b0072750 con 0x7f35b015dea0 2023-11-06T03:11:47.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.438+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f35a800ab90 con 0x7f35b015dea0 2023-11-06T03:11:47.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.438+0000 7f35a6ffd640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 0x7f3594067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.438+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f35a80830b0 con 0x7f35b015dea0 2023-11-06T03:11:47.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.438+0000 7f35b6584640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 0x7f3594067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.438+0000 7f35b6584640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 0x7f3594067f20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009840 tx=0x7f35ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.442+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f35a80508f0 con 0x7f35b015dea0 2023-11-06T03:11:47.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.586+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b00cb110 con 0x7f35b015dea0 2023-11-06T03:11:47.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.590+0000 7f35a6ffd640 1 -- 172.21.15.157:0/4064301899 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18000 (secure 0 0 0) 0x7f35a8053f10 con 0x7f35b015dea0 2023-11-06T03:11:47.592 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:47.592 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:22.343214+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:09:33.683286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-06T03:09:34.099590+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:47.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 msgr2=0x7f3594067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 0x7f3594067f20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009840 tx=0x7f35ac008040 comp rx=0 tx=0).stop 2023-11-06T03:11:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 msgr2=0x7f35b014ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c730 tx=0x7f35a800cbf0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 shutdown_connections 2023-11-06T03:11:47.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3594065a60 0x7f3594067f20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35b015dea0 0x7f35b014ffa0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35b01599c0 0x7f35b014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 --2- 172.21.15.157:0/4064301899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35b006d000 0x7f35b00ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 >> 172.21.15.157:0/4064301899 conn(0x7f35b0065930 msgr2=0x7f35b00b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 shutdown_connections 2023-11-06T03:11:47.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.594+0000 7f35b87e8640 1 -- 172.21.15.157:0/4064301899 wait complete. 2023-11-06T03:11:47.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:11:47.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 Processor -- start 2023-11-06T03:11:47.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 -- start start 2023-11-06T03:11:47.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a380b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9a3815ea60 0x7f9a380b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3815f580 con 0x7f9a381551e0 2023-11-06T03:11:47.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a380704b0 con 0x7f9a3815ea60 2023-11-06T03:11:47.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3cd5f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815efc0 con 0x7f9a3815e5e0 2023-11-06T03:11:47.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9a381551e0 0x7f9a3806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43280/0 (socket says 172.21.15.157:43280) 2023-11-06T03:11:47.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 -- 172.21.15.157:0/2726775694 learned_addr learned my addr 172.21.15.157:0/2726775694 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:47.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a35d9b640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a380b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2726775694 >> v2:172.21.15.195:3300/0 conn(0x7f9a3815ea60 0x7f9a380b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097677703 0 0) 0x7f9a3815f580 con 0x7f9a381551e0 2023-11-06T03:11:47.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a20003530 con 0x7f9a381551e0 2023-11-06T03:11:47.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549436341 0 0) 0x7f9a20003530 con 0x7f9a381551e0 2023-11-06T03:11:47.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 msgr2=0x7f9a380b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a380b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 >> v2:172.21.15.195:3300/0 conn(0x7f9a3815ea60 msgr2=0x7f9a380b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 --2- 172.21.15.157:0/2726775694 >> v2:172.21.15.195:3300/0 conn(0x7f9a3815ea60 0x7f9a380b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3815f2a0 con 0x7f9a381551e0 2023-11-06T03:11:47.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9a2c001680 con 0x7f9a381551e0 2023-11-06T03:11:47.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 >> v1:172.21.15.157:6789/0 conn(0x7f9a381551e0 legacy=0x7f9a3806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 0x7f9a20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380704b0 con 0x7f9a20003980 2023-11-06T03:11:47.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815efc0 con 0x7f9a200047d0 2023-11-06T03:11:47.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2000b9c0 con 0x7f9a3815e5e0 2023-11-06T03:11:47.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a35d9b640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 0x7f9a20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 -- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 msgr2=0x7f9a20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a20005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 -- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 msgr2=0x7f9a20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:47.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 0x7f9a20004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a3659c640 1 -- 172.21.15.157:0/2726775694 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20003530 con 0x7f9a20003980 2023-11-06T03:11:47.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.722+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3815e5e0 0x7f9a20005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:47.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c290 tx=0x7f9a2c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e883fea6c52001 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0058e0 con 0x7f9a20003980 2023-11-06T03:11:47.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c005a40 con 0x7f9a20003980 2023-11-06T03:11:47.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3559a640 1 -- 172.21.15.157:0/2726775694 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c005df0 con 0x7f9a20003980 2023-11-06T03:11:47.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2726775694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 msgr2=0x7f9a20003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2726775694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c290 tx=0x7f9a2c01cc50 comp rx=0 tx=0).stop 2023-11-06T03:11:47.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 -- 172.21.15.157:0/2726775694 reap_dead start 2023-11-06T03:11:47.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2726775694 shutdown_connections 2023-11-06T03:11:47.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2726775694 >> 172.21.15.157:0/2726775694 conn(0x7f9a38065930 msgr2=0x7f9a38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2726775694 shutdown_connections 2023-11-06T03:11:47.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2726775694 wait complete. 2023-11-06T03:11:47.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 Processor -- start 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- start start 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a20003980 0x7f9a3814eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a381518d0 0x7f9a381599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380ccc90 con 0x7f9a3806cfe0 2023-11-06T03:11:47.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3806c260 con 0x7f9a381518d0 2023-11-06T03:11:47.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3cd5f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380a8660 con 0x7f9a20003980 2023-11-06T03:11:47.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43176/0 (socket says 172.21.15.157:43176) 2023-11-06T03:11:47.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 -- 172.21.15.157:0/2807286598 learned_addr learned my addr 172.21.15.157:0/2807286598 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:47.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a35d9b640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a381518d0 0x7f9a381599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a20003980 0x7f9a3814eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 -- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a20003980 msgr2=0x7f9a3814eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a20003980 0x7f9a3814eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 -- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a381518d0 msgr2=0x7f9a381599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a381518d0 0x7f9a381599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.726+0000 7f9a36d9d640 1 -- 172.21.15.157:0/2807286598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380a8940 con 0x7f9a3806cfe0 2023-11-06T03:11:47.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a36d9d640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cf420 tx=0x7f9a3000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30016620 con 0x7f9a3806cfe0 2023-11-06T03:11:47.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a300167a0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3814f0a0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30016b50 con 0x7f9a3806cfe0 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a381f8b40 con 0x7f9a3806cfe0 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a380ccdd0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a30016cb0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 0x7f9a18067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:47.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6090+0+0 (secure 0 0 0) 0x7f9a30083f50 con 0x7f9a3806cfe0 2023-11-06T03:11:47.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 0x7f9a18067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:47.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.730+0000 7f9a3659c640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 0x7f9a18067f50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0aa000 tx=0x7f9a2c001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:47.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.734+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9a380ccdd0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.918+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9a380ccdd0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.921 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:11:47.922+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:11:47.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a26ffd640 1 -- 172.21.15.157:0/2807286598 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9a380ccdd0 con 0x7f9a3806cfe0 2023-11-06T03:11:47.934 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:11:47.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 msgr2=0x7f9a18067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 0x7f9a18067f50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0aa000 tx=0x7f9a2c001a10 comp rx=0 tx=0).stop 2023-11-06T03:11:47.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 msgr2=0x7f9a38159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:47.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cf420 tx=0x7f9a3000f520 comp rx=0 tx=0).stop 2023-11-06T03:11:47.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 shutdown_connections 2023-11-06T03:11:47.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a90 0x7f9a18067f50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3806cfe0 0x7f9a38159f10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a381518d0 0x7f9a381599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 --2- 172.21.15.157:0/2807286598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a20003980 0x7f9a3814eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:47.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.934+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 >> 172.21.15.157:0/2807286598 conn(0x7f9a38065930 msgr2=0x7f9a380b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:47.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.938+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 shutdown_connections 2023-11-06T03:11:47.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:47.938+0000 7f9a3cd5f640 1 -- 172.21.15.157:0/2807286598 wait complete. 2023-11-06T03:11:47.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:11:47.988 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:11:48.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.062+0000 7fcec60dc640 1 Processor -- start 2023-11-06T03:11:48.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.062+0000 7fcec60dc640 1 -- start start 2023-11-06T03:11:48.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcec60dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcec006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcec60dc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00733a0 con 0x7fcec0071150 2023-11-06T03:11:48.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcec60dc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00cc8d0 con 0x7fcec0159ce0 2023-11-06T03:11:48.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcec60dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00730c0 con 0x7fcec00701b0 2023-11-06T03:11:48.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcec0071150 0x7fcec006d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43284/0 (socket says 172.21.15.157:43284) 2023-11-06T03:11:48.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 -- 172.21.15.157:0/2058661664 learned_addr learned my addr 172.21.15.157:0/2058661664 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:48.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500290521 0 0) 0x7fcec00733a0 con 0x7fcec0071150 2023-11-06T03:11:48.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebffff640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcec006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea8003530 con 0x7fcec0071150 2023-11-06T03:11:48.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1409497726 0 0) 0x7fcea8003530 con 0x7fcec0071150 2023-11-06T03:11:48.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 msgr2=0x7fcec006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:48.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcec006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 >> v1:172.21.15.195:6789/0 conn(0x7fcec0159ce0 legacy=0x7fcec0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00cc5f0 con 0x7fcec0071150 2023-11-06T03:11:48.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fceb4001680 con 0x7fcec0071150 2023-11-06T03:11:48.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 >> v1:172.21.15.157:6789/0 conn(0x7fcec0071150 legacy=0x7fcec006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea80039b0 0x7fcea8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea80047d0 0x7fcea8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcea80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00730c0 con 0x7fcea80047d0 2023-11-06T03:11:48.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea800bac0 con 0x7fcea80039b0 2023-11-06T03:11:48.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea800de90 con 0x7fcec00701b0 2023-11-06T03:11:48.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea80047d0 0x7fcea8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebf7fe640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea80039b0 0x7fcea8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebffff640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcea80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 -- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 msgr2=0x7fcea80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:48.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcec00701b0 0x7fcea80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 -- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea80039b0 msgr2=0x7fcea8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:48.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcea80039b0 0x7fcea8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 -- 172.21.15.157:0/2058661664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8003530 con 0x7fcea80047d0 2023-11-06T03:11:48.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.066+0000 7fcebeffd640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea80047d0 0x7fcea8004c50 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c240 tx=0x7fceb401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87a675036437d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:48.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4005ae0 con 0x7fcea80047d0 2023-11-06T03:11:48.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4005c40 con 0x7fcea80047d0 2023-11-06T03:11:48.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebe7fc640 1 -- 172.21.15.157:0/2058661664 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4007070 con 0x7fcea80047d0 2023-11-06T03:11:48.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/2058661664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea80047d0 msgr2=0x7fcea8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 --2- 172.21.15.157:0/2058661664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea80047d0 0x7fcea8004c50 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c240 tx=0x7fceb401cc10 comp rx=0 tx=0).stop 2023-11-06T03:11:48.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebffff640 1 -- 172.21.15.157:0/2058661664 reap_dead start 2023-11-06T03:11:48.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/2058661664 shutdown_connections 2023-11-06T03:11:48.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/2058661664 >> 172.21.15.157:0/2058661664 conn(0x7fcec00cba30 msgr2=0x7fcec00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:48.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/2058661664 shutdown_connections 2023-11-06T03:11:48.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/2058661664 wait complete. 2023-11-06T03:11:48.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 Processor -- start 2023-11-06T03:11:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- start start 2023-11-06T03:11:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea80047d0 0x7fcec014af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec014bed0 0x7fcec014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec014cd50 con 0x7fcec00b1af0 2023-11-06T03:11:48.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b24b0 con 0x7fcec014bed0 2023-11-06T03:11:48.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec014c890 con 0x7fcea80047d0 2023-11-06T03:11:48.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43214/0 (socket says 172.21.15.157:43214) 2023-11-06T03:11:48.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 -- 172.21.15.157:0/872034257 learned_addr learned my addr 172.21.15.157:0/872034257 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebffff640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec014bed0 0x7fcec014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebf7fe640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea80047d0 0x7fcec014af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 -- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea80047d0 msgr2=0x7fcec014af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea80047d0 0x7fcec014af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 -- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec014bed0 msgr2=0x7fcec014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec014bed0 0x7fcec014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 -- 172.21.15.157:0/872034257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b2940 con 0x7fcec00b1af0 2023-11-06T03:11:48.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcebeffd640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fceb401ca40 tx=0x7fcea8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:48.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40b4070 con 0x7fcec00b1af0 2023-11-06T03:11:48.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4005db0 con 0x7fcec00b1af0 2023-11-06T03:11:48.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00b2c20 con 0x7fcec00b1af0 2023-11-06T03:11:48.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4001640 con 0x7fcec00b1af0 2023-11-06T03:11:48.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.070+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec00b31e0 con 0x7fcec00b1af0 2023-11-06T03:11:48.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec00b3400 con 0x7fcec00b1af0 2023-11-06T03:11:48.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fceb4007040 con 0x7fcec00b1af0 2023-11-06T03:11:48.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcea7fff640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 0x7fceac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:48.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fceb411cab0 con 0x7fcec00b1af0 2023-11-06T03:11:48.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcebf7fe640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 0x7fceac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:48.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.074+0000 7fcebf7fe640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 0x7fceac067f20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcec01f61a0 tx=0x7fceb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:48.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.078+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fceb40ea340 con 0x7fcec00b1af0 2023-11-06T03:11:48.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.250+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcec0072750 con 0x7fcec00b1af0 2023-11-06T03:11:48.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcea7fff640 1 -- 172.21.15.157:0/872034257 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fceb40ed960 con 0x7fcec00b1af0 2023-11-06T03:11:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 msgr2=0x7fceac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 0x7fceac067f20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcec01f61a0 tx=0x7fceb0009040 comp rx=0 tx=0).stop 2023-11-06T03:11:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 msgr2=0x7fcec00b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fceb401ca40 tx=0x7fcea8005800 comp rx=0 tx=0).stop 2023-11-06T03:11:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 shutdown_connections 2023-11-06T03:11:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fceac065a60 0x7fceac067f20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec014bed0 0x7fcec014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcec00b1af0 0x7fcec00b1f70 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 --2- 172.21.15.157:0/872034257 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcea80047d0 0x7fcec014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 >> 172.21.15.157:0/872034257 conn(0x7fcec00cba30 msgr2=0x7fcec00b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 shutdown_connections 2023-11-06T03:11:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:48.942+0000 7fcec60dc640 1 -- 172.21.15.157:0/872034257 wait complete. 2023-11-06T03:11:48.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:11:49.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 Processor -- start 2023-11-06T03:11:49.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 -- start start 2023-11-06T03:11:49.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69f406c170 0x7f69f4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f69f415b860 0x7f69f4155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40cb100 con 0x7f69f406c550 2023-11-06T03:11:49.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f415bcf0 con 0x7f69f406c170 2023-11-06T03:11:49.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69fa186640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f40ce0a0 con 0x7f69f415b860 2023-11-06T03:11:49.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69f406c170 0x7f69f4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f69f406c170 0x7f69f4154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49788/0 (socket says 172.21.15.157:49788) 2023-11-06T03:11:49.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 -- 172.21.15.157:0/2014343966 learned_addr learned my addr 172.21.15.157:0/2014343966 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:49.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f3fff640 1 --2- 172.21.15.157:0/2014343966 >> v2:172.21.15.195:3300/0 conn(0x7f69f415b860 0x7f69f4155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 -- 172.21.15.157:0/2014343966 >> v2:172.21.15.195:3300/0 conn(0x7f69f415b860 msgr2=0x7f69f4155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:49.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 --2- 172.21.15.157:0/2014343966 >> v2:172.21.15.195:3300/0 conn(0x7f69f415b860 0x7f69f4155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 -- 172.21.15.157:0/2014343966 >> v1:172.21.15.195:6789/0 conn(0x7f69f406c550 legacy=0x7f69f415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.138+0000 7f69f37fe640 1 -- 172.21.15.157:0/2014343966 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f415bef0 con 0x7f69f406c170 2023-11-06T03:11:49.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69f37fe640 1 --2- 172.21.15.157:0/2014343966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c170 0x7f69f4154c00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f69f406d610 tx=0x7f69e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1836ef271c7a168 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:49.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69f27fc640 1 -- 172.21.15.157:0/2014343966 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40aa430 con 0x7f69f406c170 2023-11-06T03:11:49.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69f27fc640 1 -- 172.21.15.157:0/2014343966 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e40aa590 con 0x7f69f406c170 2023-11-06T03:11:49.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69f27fc640 1 -- 172.21.15.157:0/2014343966 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40aa920 con 0x7f69f406c170 2023-11-06T03:11:49.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- 172.21.15.157:0/2014343966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c170 msgr2=0x7f69f4154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 --2- 172.21.15.157:0/2014343966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c170 0x7f69f4154c00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f69f406d610 tx=0x7f69e401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- 172.21.15.157:0/2014343966 shutdown_connections 2023-11-06T03:11:49.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 --2- 172.21.15.157:0/2014343966 >> v2:172.21.15.195:3300/0 conn(0x7f69f415b860 0x7f69f4155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 --2- 172.21.15.157:0/2014343966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c170 0x7f69f4154c00 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- 172.21.15.157:0/2014343966 >> 172.21.15.157:0/2014343966 conn(0x7f69f40b43b0 msgr2=0x7f69f4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:49.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- 172.21.15.157:0/2014343966 shutdown_connections 2023-11-06T03:11:49.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- 172.21.15.157:0/2014343966 wait complete. 2023-11-06T03:11:49.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 Processor -- start 2023-11-06T03:11:49.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.142+0000 7f69fa186640 1 -- start start 2023-11-06T03:11:49.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f406c170 0x7f69f40ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69f415b860 0x7f69f40a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4150350 con 0x7f69f406c550 2023-11-06T03:11:49.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40a89e0 con 0x7f69f406c170 2023-11-06T03:11:49.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40a9830 con 0x7f69f415b860 2023-11-06T03:11:49.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49802/0 (socket says 172.21.15.157:49802) 2023-11-06T03:11:49.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 -- 172.21.15.157:0/1852369028 learned_addr learned my addr 172.21.15.157:0/1852369028 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:49.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f37fe640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f406c170 0x7f69f40ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f3fff640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69f415b860 0x7f69f40a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 -- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69f415b860 msgr2=0x7f69f40a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69f415b860 0x7f69f40a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 -- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f406c170 msgr2=0x7f69f40ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f406c170 0x7f69f40ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 -- 172.21.15.157:0/1852369028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40094d0 con 0x7f69f406c550 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f2ffd640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c530 tx=0x7f69e800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005220 con 0x7f69f406c550 2023-11-06T03:11:49.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8005380 con 0x7f69f406c550 2023-11-06T03:11:49.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f40cbfb0 con 0x7f69f406c550 2023-11-06T03:11:49.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8007270 con 0x7f69f406c550 2023-11-06T03:11:49.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f40a8f00 con 0x7f69f406c550 2023-11-06T03:11:49.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f40a99c0 con 0x7f69f406c550 2023-11-06T03:11:49.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f69e80073d0 con 0x7f69f406c550 2023-11-06T03:11:49.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 0x7f69e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:49.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f69e80829f0 con 0x7f69f406c550 2023-11-06T03:11:49.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.146+0000 7f69f37fe640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 0x7f69e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:49.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.150+0000 7f69f37fe640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 0x7f69e0067f50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ae000 tx=0x7f69e40b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:49.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.150+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f69e8050330 con 0x7f69f406c550 2023-11-06T03:11:49.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.314+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69f404e050 con 0x7f69f406c550 2023-11-06T03:11:49.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69dbfff640 1 -- 172.21.15.157:0/1852369028 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f69e8053950 con 0x7f69f406c550 2023-11-06T03:11:49.953 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:11:49.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 msgr2=0x7f69e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 0x7f69e0067f50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f69e40ae000 tx=0x7f69e40b5040 comp rx=0 tx=0).stop 2023-11-06T03:11:49.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 msgr2=0x7f69f414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:49.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c530 tx=0x7f69e800c9f0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 shutdown_connections 2023-11-06T03:11:49.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f69e0065a90 0x7f69e0067f50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69f415b860 0x7f69f40a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69f406c550 0x7f69f414fe10 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 --2- 172.21.15.157:0/1852369028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69f406c170 0x7f69f40ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:49.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.954+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 >> 172.21.15.157:0/1852369028 conn(0x7f69f40b43b0 msgr2=0x7f69f415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:49.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.958+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 shutdown_connections 2023-11-06T03:11:49.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:49.958+0000 7f69fa186640 1 -- 172.21.15.157:0/1852369028 wait complete. 2023-11-06T03:11:49.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:11:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.098+0000 7f2e737e4640 1 Processor -- start 2023-11-06T03:11:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e737e4640 1 -- start start 2023-11-06T03:11:50.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e737e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c15cc10 0x7f2e6c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:50.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e737e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0adc00 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e737e4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0adee0 con 0x7f2e6c0b3f70 2023-11-06T03:11:50.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e737e4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0ae880 con 0x7f2e6c15cc10 2023-11-06T03:11:50.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c15cc10 0x7f2e6c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:50.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2e6c15cc10 0x7f2e6c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49806/0 (socket says 172.21.15.157:49806) 2023-11-06T03:11:50.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e70d7f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2e6c0ce0d0 0x7f2e6c0b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51360/0 (socket says 172.21.15.157:51360) 2023-11-06T03:11:50.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 -- 172.21.15.157:0/5396105 learned_addr learned my addr 172.21.15.157:0/5396105 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 -- 172.21.15.157:0/5396105 >> v1:172.21.15.195:6789/0 conn(0x7f2e6c0b3f70 legacy=0x7f2e6c0ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 -- 172.21.15.157:0/5396105 >> v1:172.21.15.157:6789/0 conn(0x7f2e6c0ce0d0 legacy=0x7f2e6c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 -- 172.21.15.157:0/5396105 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0ae1c0 con 0x7f2e6c15cc10 2023-11-06T03:11:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e71580640 1 --2- 172.21.15.157:0/5396105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c06fb80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c01ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a07b1189990cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e63fff640 1 -- 172.21.15.157:0/5396105 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c004230 con 0x7f2e6c15cc10 2023-11-06T03:11:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e63fff640 1 -- 172.21.15.157:0/5396105 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0a7050 con 0x7f2e6c15cc10 2023-11-06T03:11:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.102+0000 7f2e63fff640 1 -- 172.21.15.157:0/5396105 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c003620 con 0x7f2e6c15cc10 2023-11-06T03:11:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/5396105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c15cc10 msgr2=0x7f2e6c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 --2- 172.21.15.157:0/5396105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c06fb80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c01ced0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/5396105 shutdown_connections 2023-11-06T03:11:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 --2- 172.21.15.157:0/5396105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c06fb80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/5396105 >> 172.21.15.157:0/5396105 conn(0x7f2e6c06a910 msgr2=0x7f2e6c0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/5396105 shutdown_connections 2023-11-06T03:11:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/5396105 wait complete. 2023-11-06T03:11:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 Processor -- start 2023-11-06T03:11:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- start start 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0b3f70 0x7f2e6c140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c140ad0 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c147350 con 0x7f2e6c15cc10 2023-11-06T03:11:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c147070 con 0x7f2e6c0b3f70 2023-11-06T03:11:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49814/0 (socket says 172.21.15.157:49814) 2023-11-06T03:11:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 -- 172.21.15.157:0/3939030573 learned_addr learned my addr 172.21.15.157:0/3939030573 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e71d81640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e71580640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0b3f70 0x7f2e6c140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 -- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0b3f70 msgr2=0x7f2e6c140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0b3f70 0x7f2e6c140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 -- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 msgr2=0x7f2e6c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 -- 172.21.15.157:0/3939030573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0094d0 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e71d81640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e70d7f640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c4c0 tx=0x7f2e6800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68007480 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680075e0 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68004a10 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c0cea70 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.106+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c1496f0 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c149270 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e68004b70 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e61ffb640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f2e680822c0 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e71580640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.110+0000 7f2e71580640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 0x7f2e50067f20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ad000 tx=0x7f2e5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.114+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e6c149270 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.270+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2e6c149270 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.958+0000 7f2e61ffb640 1 -- 172.21.15.157:0/3939030573 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2e6c149270 con 0x7f2e6c0ce0d0 2023-11-06T03:11:50.957 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:11:50.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 msgr2=0x7f2e50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 0x7f2e50067f20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ad000 tx=0x7f2e5c008040 comp rx=0 tx=0).stop 2023-11-06T03:11:50.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 msgr2=0x7f2e6c15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:50.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c4c0 tx=0x7f2e6800c980 comp rx=0 tx=0).stop 2023-11-06T03:11:50.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 shutdown_connections 2023-11-06T03:11:50.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e50065a60 0x7f2e50067f20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e6c15cc10 0x7f2e6c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e6c0ce0d0 0x7f2e6c15dd00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 --2- 172.21.15.157:0/3939030573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e6c0b3f70 0x7f2e6c140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:50.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 >> 172.21.15.157:0/3939030573 conn(0x7f2e6c06a910 msgr2=0x7f2e6c0ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:50.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 shutdown_connections 2023-11-06T03:11:50.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:50.962+0000 7f2e737e4640 1 -- 172.21.15.157:0/3939030573 wait complete. 2023-11-06T03:11:50.997 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:11:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 Processor -- start 2023-11-06T03:11:51.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 -- start start 2023-11-06T03:11:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 0x7f0c680b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6815f580 con 0x7f0c681551e0 2023-11-06T03:11:51.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6815efc0 con 0x7f0c6815ea60 2023-11-06T03:11:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6fe24640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c680704b0 con 0x7f0c6815e5e0 2023-11-06T03:11:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6dbc0640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0c681551e0 0x7f0c6806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51364/0 (socket says 172.21.15.157:51364) 2023-11-06T03:11:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6dbc0640 1 -- 172.21.15.157:0/840511358 learned_addr learned my addr 172.21.15.157:0/840511358 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6d3bf640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 0x7f0c680b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296531587 0 0) 0x7f0c6815f580 con 0x7f0c681551e0 2023-11-06T03:11:51.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.118+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c54003530 con 0x7f0c681551e0 2023-11-06T03:11:51.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698713416 0 0) 0x7f0c54003530 con 0x7f0c681551e0 2023-11-06T03:11:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 msgr2=0x7f0c680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 0x7f0c680b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 >> v1:172.21.15.195:6789/0 conn(0x7f0c6815ea60 legacy=0x7f0c6815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6815f2a0 con 0x7f0c681551e0 2023-11-06T03:11:51.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0c64001680 con 0x7f0c681551e0 2023-11-06T03:11:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 >> v1:172.21.15.157:6789/0 conn(0x7f0c681551e0 legacy=0x7f0c6806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c540039b0 0x7f0c54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c540047d0 0x7f0c54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 0x7f0c540051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c680704b0 con 0x7f0c540039b0 2023-11-06T03:11:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5400bb20 con 0x7f0c540047d0 2023-11-06T03:11:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5400bcb0 con 0x7f0c6815e5e0 2023-11-06T03:11:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c540039b0 0x7f0c54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6d3bf640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c540047d0 0x7f0c54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 -- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 msgr2=0x7f0c540051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c6815e5e0 0x7f0c540051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 -- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c540047d0 msgr2=0x7f0c54004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c540047d0 0x7f0c54004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 -- 172.21.15.157:0/840511358 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54003530 con 0x7f0c540039b0 2023-11-06T03:11:51.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c540039b0 0x7f0c54003dd0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c270 tx=0x7f0c6401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d9c6e903c0752e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64005ae0 con 0x7f0c540039b0 2023-11-06T03:11:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64005c40 con 0x7f0c540039b0 2023-11-06T03:11:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6cbbe640 1 -- 172.21.15.157:0/840511358 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640a5800 con 0x7f0c540039b0 2023-11-06T03:11:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 -- 172.21.15.157:0/840511358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c540039b0 msgr2=0x7f0c54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/840511358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c540039b0 0x7f0c54003dd0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c270 tx=0x7f0c6401cc10 comp rx=0 tx=0).stop 2023-11-06T03:11:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6e3c1640 1 -- 172.21.15.157:0/840511358 reap_dead start 2023-11-06T03:11:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 -- 172.21.15.157:0/840511358 shutdown_connections 2023-11-06T03:11:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 -- 172.21.15.157:0/840511358 >> 172.21.15.157:0/840511358 conn(0x7f0c68065930 msgr2=0x7f0c68065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 -- 172.21.15.157:0/840511358 shutdown_connections 2023-11-06T03:11:51.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.122+0000 7f0c6fe24640 1 -- 172.21.15.157:0/840511358 wait complete. 2023-11-06T03:11:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 Processor -- start 2023-11-06T03:11:51.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- start start 2023-11-06T03:11:51.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c540039b0 0x7f0c681599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 0x7f0c68159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6806bd90 con 0x7f0c68160140 2023-11-06T03:11:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c68161750 con 0x7f0c6814db70 2023-11-06T03:11:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c68160eb0 con 0x7f0c540039b0 2023-11-06T03:11:51.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49844/0 (socket says 172.21.15.157:49844) 2023-11-06T03:11:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 -- 172.21.15.157:0/1914912051 learned_addr learned my addr 172.21.15.157:0/1914912051 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6d3bf640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 0x7f0c68159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c540039b0 0x7f0c681599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 -- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c540039b0 msgr2=0x7f0c681599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c540039b0 0x7f0c681599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 -- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 msgr2=0x7f0c68159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 0x7f0c68159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 -- 172.21.15.157:0/1914912051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c68161190 con 0x7f0c68160140 2023-11-06T03:11:51.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6d3bf640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 0x7f0c68159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6e3c1640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0c680cf420 tx=0x7f0c6000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60005d50 con 0x7f0c68160140 2023-11-06T03:11:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6000ced0 con 0x7f0c68160140 2023-11-06T03:11:51.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c68161470 con 0x7f0c68160140 2023-11-06T03:11:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000a850 con 0x7f0c68160140 2023-11-06T03:11:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c681527a0 con 0x7f0c68160140 2023-11-06T03:11:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6806bed0 con 0x7f0c68160140 2023-11-06T03:11:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.126+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0c6000a9b0 con 0x7f0c68160140 2023-11-06T03:11:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.130+0000 7f0c5e7fc640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 0x7f0c4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.130+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c60082440 con 0x7f0c68160140 2023-11-06T03:11:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.130+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 0x7f0c4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.130+0000 7f0c6dbc0640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 0x7f0c4c067f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0c640ab000 tx=0x7f0c640a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.134+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0c6806bed0 con 0x7f0c68160140 2023-11-06T03:11:51.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.290+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0c6806bed0 con 0x7f0c68160140 2023-11-06T03:11:51.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c5e7fc640 1 -- 172.21.15.157:0/1914912051 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c6806bed0 con 0x7f0c68160140 2023-11-06T03:11:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 msgr2=0x7f0c4c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 0x7f0c4c067f20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0c640ab000 tx=0x7f0c640a7070 comp rx=0 tx=0).stop 2023-11-06T03:11:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 msgr2=0x7f0c681605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0c680cf420 tx=0x7f0c6000cb90 comp rx=0 tx=0).stop 2023-11-06T03:11:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 shutdown_connections 2023-11-06T03:11:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0c4c065a60 0x7f0c4c067f20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c68160140 0x7f0c681605c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c6814db70 0x7f0c68159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 --2- 172.21.15.157:0/1914912051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c540039b0 0x7f0c681599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 >> 172.21.15.157:0/1914912051 conn(0x7f0c68065930 msgr2=0x7f0c6815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:51.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 shutdown_connections 2023-11-06T03:11:51.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.298+0000 7f0c6fe24640 1 -- 172.21.15.157:0/1914912051 wait complete. 2023-11-06T03:11:51.317 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2023-11-06T03:11:51.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 Processor -- start 2023-11-06T03:11:51.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 -- start start 2023-11-06T03:11:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 0x7f3c840cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3c840cce90 0x7f3c840cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c840d0140 con 0x7f3c8415e090 2023-11-06T03:11:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c840cc420 con 0x7f3c840cce90 2023-11-06T03:11:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.442+0000 7f3c890f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840d0380 con 0x7f3c840719e0 2023-11-06T03:11:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 0x7f3c840cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8359e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3c840cce90 0x7f3c840cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c82d9d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f3c8415e090 0x7f3c840cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:53152/0 (socket says 172.21.15.157:53152) 2023-11-06T03:11:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c82d9d640 1 -- 172.21.15.157:0/363808546 learned_addr learned my addr 172.21.15.157:0/363808546 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 -- 172.21.15.157:0/363808546 >> v2:172.21.15.195:3300/0 conn(0x7f3c840cce90 msgr2=0x7f3c840cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 --2- 172.21.15.157:0/363808546 >> v2:172.21.15.195:3300/0 conn(0x7f3c840cce90 0x7f3c840cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 -- 172.21.15.157:0/363808546 >> v1:172.21.15.195:6789/0 conn(0x7f3c8415e090 legacy=0x7f3c840cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 -- 172.21.15.157:0/363808546 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840cc690 con 0x7f3c840719e0 2023-11-06T03:11:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c8259c640 1 --2- 172.21.15.157:0/363808546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 0x7f3c840cd320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009d90 tx=0x7f3c7c01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208c58b539f6e801 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c81d9b640 1 -- 172.21.15.157:0/363808546 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0af600 con 0x7f3c840719e0 2023-11-06T03:11:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c81d9b640 1 -- 172.21.15.157:0/363808546 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0ab050 con 0x7f3c840719e0 2023-11-06T03:11:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.446+0000 7f3c81d9b640 1 -- 172.21.15.157:0/363808546 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0af980 con 0x7f3c840719e0 2023-11-06T03:11:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/363808546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 msgr2=0x7f3c840cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- 172.21.15.157:0/363808546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 0x7f3c840cd320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009d90 tx=0x7f3c7c01f130 comp rx=0 tx=0).stop 2023-11-06T03:11:51.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/363808546 shutdown_connections 2023-11-06T03:11:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- 172.21.15.157:0/363808546 >> v2:172.21.15.195:3300/0 conn(0x7f3c840cce90 0x7f3c840cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- 172.21.15.157:0/363808546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c840719e0 0x7f3c840cd320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/363808546 >> 172.21.15.157:0/363808546 conn(0x7f3c8406d3d0 msgr2=0x7f3c8406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/363808546 shutdown_connections 2023-11-06T03:11:51.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/363808546 wait complete. 2023-11-06T03:11:51.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 Processor -- start 2023-11-06T03:11:51.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- start start 2023-11-06T03:11:51.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c840719e0 0x7f3c841f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8415e090 0x7f3c840b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b2d80 con 0x7f3c840cce90 2023-11-06T03:11:51.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b2aa0 con 0x7f3c840719e0 2023-11-06T03:11:51.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8414d350 con 0x7f3c8415e090 2023-11-06T03:11:51.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49850/0 (socket says 172.21.15.157:49850) 2023-11-06T03:11:51.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 -- 172.21.15.157:0/2231583528 learned_addr learned my addr 172.21.15.157:0/2231583528 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:51.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c82d9d640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c840719e0 0x7f3c841f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8359e640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8415e090 0x7f3c840b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 -- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8415e090 msgr2=0x7f3c840b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8415e090 0x7f3c840b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 -- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c840719e0 msgr2=0x7f3c841f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c840719e0 0x7f3c841f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 -- 172.21.15.157:0/2231583528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c009a70 con 0x7f3c840cce90 2023-11-06T03:11:51.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c8259c640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c01cd60 tx=0x7f3c7c01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c009450 con 0x7f3c840cce90 2023-11-06T03:11:51.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0b9410 con 0x7f3c840cce90 2023-11-06T03:11:51.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8414d630 con 0x7f3c840cce90 2023-11-06T03:11:51.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0b97a0 con 0x7f3c840cce90 2023-11-06T03:11:51.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.450+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841f41a0 con 0x7f3c840cce90 2023-11-06T03:11:51.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8414d1b0 con 0x7f3c840cce90 2023-11-06T03:11:51.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3c7c0b6050 con 0x7f3c840cce90 2023-11-06T03:11:51.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c7b7fe640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 0x7f3c64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:51.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c7c1214a0 con 0x7f3c840cce90 2023-11-06T03:11:51.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c82d9d640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 0x7f3c64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:51.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.454+0000 7f3c82d9d640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 0x7f3c64067f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005890 tx=0x7f3c74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:51.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.458+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3c8414d1b0 con 0x7f3c840cce90 2023-11-06T03:11:51.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.662+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f3c8414db50 con 0x7f3c840cce90 2023-11-06T03:11:51.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.962+0000 7f3c7b7fe640 1 -- 172.21.15.157:0/2231583528 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v362) v1 ==== 158+0+0 (secure 0 0 0) 0x7f3c8414db50 con 0x7f3c840cce90 2023-11-06T03:11:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 msgr2=0x7f3c64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 0x7f3c64067f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3c74005890 tx=0x7f3c74009040 comp rx=0 tx=0).stop 2023-11-06T03:11:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 msgr2=0x7f3c840b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c01cd60 tx=0x7f3c7c01bed0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 shutdown_connections 2023-11-06T03:11:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c64065a60 0x7f3c64067f20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c8415e090 0x7f3c840b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c840cce90 0x7f3c840b3500 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 --2- 172.21.15.157:0/2231583528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c840719e0 0x7f3c841f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 >> 172.21.15.157:0/2231583528 conn(0x7f3c8406d3d0 msgr2=0x7f3c840b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 shutdown_connections 2023-11-06T03:11:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:51.966+0000 7f3c890f6640 1 -- 172.21.15.157:0/2231583528 wait complete. 2023-11-06T03:11:51.986 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2023-11-06T03:11:51.987 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2023-11-06T03:11:51.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2023-11-06T03:11:52.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 Processor -- start 2023-11-06T03:11:52.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 -- start start 2023-11-06T03:11:52.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 0x7f935c070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 0x7f935c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c15abb0 con 0x7f935c06c3d0 2023-11-06T03:11:52.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c0cf690 con 0x7f935c15de90 2023-11-06T03:11:52.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9362ad8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c15d1a0 con 0x7f935c070a90 2023-11-06T03:11:52.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f935bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 0x7f935c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f935bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 0x7f935c0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:48332/0 (socket says 172.21.15.157:48332) 2023-11-06T03:11:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f935bfff640 1 -- 172.21.15.157:0/3867974601 learned_addr learned my addr 172.21.15.157:0/3867974601 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9361075640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f935c06c3d0 0x7f935c0b1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:53154/0 (socket says 172.21.15.157:53154) 2023-11-06T03:11:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9360874640 1 --2- 172.21.15.157:0/3867974601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 0x7f935c070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9360874640 1 -- 172.21.15.157:0/3867974601 >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 msgr2=0x7f935c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9360874640 1 --2- 172.21.15.157:0/3867974601 >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 0x7f935c0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9360874640 1 -- 172.21.15.157:0/3867974601 >> v1:172.21.15.195:6789/0 conn(0x7f935c06c3d0 legacy=0x7f935c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.114+0000 7f9360874640 1 -- 172.21.15.157:0/3867974601 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0713b0 con 0x7f935c070a90 2023-11-06T03:11:52.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9360874640 1 --2- 172.21.15.157:0/3867974601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 0x7f935c070e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93440097c0 tx=0x7f934401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180ed6ef0c46c86a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:52.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f935b7fe640 1 -- 172.21.15.157:0/3867974601 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440ac600 con 0x7f935c070a90 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f935b7fe640 1 -- 172.21.15.157:0/3867974601 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440a8050 con 0x7f935c070a90 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f935b7fe640 1 -- 172.21.15.157:0/3867974601 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440ac980 con 0x7f935c070a90 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 -- 172.21.15.157:0/3867974601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 msgr2=0x7f935c070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3867974601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 0x7f935c070e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93440097c0 tx=0x7f934401c480 comp rx=0 tx=0).stop 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 -- 172.21.15.157:0/3867974601 shutdown_connections 2023-11-06T03:11:52.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3867974601 >> v2:172.21.15.195:3300/0 conn(0x7f935c15de90 0x7f935c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3867974601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c070a90 0x7f935c070e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 -- 172.21.15.157:0/3867974601 >> 172.21.15.157:0/3867974601 conn(0x7f935c069d70 msgr2=0x7f935c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:52.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 -- 172.21.15.157:0/3867974601 shutdown_connections 2023-11-06T03:11:52.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.118+0000 7f9362ad8640 1 -- 172.21.15.157:0/3867974601 wait complete. 2023-11-06T03:11:52.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 Processor -- start 2023-11-06T03:11:52.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- start start 2023-11-06T03:11:52.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 0x7f935c141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c15de90 0x7f935c153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0b4c40 con 0x7f935c070a90 2023-11-06T03:11:52.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0b4030 con 0x7f935c06c3d0 2023-11-06T03:11:52.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0b4310 con 0x7f935c15de90 2023-11-06T03:11:52.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49852/0 (socket says 172.21.15.157:49852) 2023-11-06T03:11:52.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 -- 172.21.15.157:0/3920954796 learned_addr learned my addr 172.21.15.157:0/3920954796 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9360874640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 0x7f935c141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9361075640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c15de90 0x7f935c153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 -- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c15de90 msgr2=0x7f935c153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c15de90 0x7f935c153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 -- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 msgr2=0x7f935c141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 0x7f935c141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 -- 172.21.15.157:0/3920954796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009420 con 0x7f935c070a90 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9360874640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 0x7f935c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f935bfff640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f934c00e7f0 tx=0x7f934c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c009580 con 0x7f935c070a90 2023-11-06T03:11:52.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c00c940 con 0x7f935c070a90 2023-11-06T03:11:52.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0b45f0 con 0x7f935c070a90 2023-11-06T03:11:52.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00ccf0 con 0x7f935c070a90 2023-11-06T03:11:52.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1416c0 con 0x7f935c070a90 2023-11-06T03:11:52.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c0b4df0 con 0x7f935c070a90 2023-11-06T03:11:52.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f934c013470 con 0x7f935c070a90 2023-11-06T03:11:52.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 0x7f933c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.122+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f934c0830d0 con 0x7f935c070a90 2023-11-06T03:11:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.126+0000 7f9360874640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 0x7f933c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:52.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.126+0000 7f9360874640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 0x7f933c067f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f93440ae000 tx=0x7f93440a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:52.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.130+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f935c0b4df0 con 0x7f935c070a90 2023-11-06T03:11:52.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.370+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f935c072750 con 0x7f935c070a90 2023-11-06T03:11:52.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.966+0000 7f93597fa640 1 -- 172.21.15.157:0/3920954796 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v363) v1 ==== 147+0+0 (secure 0 0 0) 0x7f934c050c40 con 0x7f935c070a90 2023-11-06T03:11:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 msgr2=0x7f933c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 0x7f933c067f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f93440ae000 tx=0x7f93440a7000 comp rx=0 tx=0).stop 2023-11-06T03:11:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 msgr2=0x7f935c153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:52.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f934c00e7f0 tx=0x7f934c00ecb0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 shutdown_connections 2023-11-06T03:11:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f933c065a60 0x7f933c067f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f935c15de90 0x7f935c153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c070a90 0x7f935c153340 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 --2- 172.21.15.157:0/3920954796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f935c06c3d0 0x7f935c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:52.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 >> 172.21.15.157:0/3920954796 conn(0x7f935c069d70 msgr2=0x7f935c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:52.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 shutdown_connections 2023-11-06T03:11:52.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:52.970+0000 7f9362ad8640 1 -- 172.21.15.157:0/3920954796 wait complete. 2023-11-06T03:11:52.991 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:11:53.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 Processor -- start 2023-11-06T03:11:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 -- start start 2023-11-06T03:11:53.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f496015e5e0 0x7f49600b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f496015ea90 0x7f49600b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496015f580 con 0x7f49601551e0 2023-11-06T03:11:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600704b0 con 0x7f496015e5e0 2023-11-06T03:11:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4967bed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496015efc0 con 0x7f496015ea90 2023-11-06T03:11:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965989640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f49601551e0 0x7f496006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51372/0 (socket says 172.21.15.157:51372) 2023-11-06T03:11:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965989640 1 -- 172.21.15.157:0/3158682035 learned_addr learned my addr 172.21.15.157:0/3158682035 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965188640 1 --2- 172.21.15.157:0/3158682035 >> v2:172.21.15.157:3300/0 conn(0x7f496015e5e0 0x7f49600b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f496618a640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f496015ea90 0x7f49600b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4964987640 1 -- 172.21.15.157:0/3158682035 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666857011 0 0) 0x7f496015f580 con 0x7f49601551e0 2023-11-06T03:11:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4964987640 1 -- 172.21.15.157:0/3158682035 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f494c003530 con 0x7f49601551e0 2023-11-06T03:11:53.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965188640 1 -- 172.21.15.157:0/3158682035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f496015ea90 msgr2=0x7f49600b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:53.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965188640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f496015ea90 0x7f49600b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965188640 1 -- 172.21.15.157:0/3158682035 >> v1:172.21.15.157:6789/0 conn(0x7f49601551e0 legacy=0x7f496006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.122+0000 7f4965188640 1 -- 172.21.15.157:0/3158682035 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496015f2a0 con 0x7f496015e5e0 2023-11-06T03:11:53.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4965188640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015e5e0 0x7f49600b3410 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f49500097c0 tx=0x7f495001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f858035cd4431b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4964987640 1 -- 172.21.15.157:0/3158682035 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500ac620 con 0x7f496015e5e0 2023-11-06T03:11:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4964987640 1 -- 172.21.15.157:0/3158682035 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500ac780 con 0x7f496015e5e0 2023-11-06T03:11:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4964987640 1 -- 172.21.15.157:0/3158682035 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500acb10 con 0x7f496015e5e0 2023-11-06T03:11:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- 172.21.15.157:0/3158682035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015e5e0 msgr2=0x7f49600b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015e5e0 0x7f49600b3410 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f49500097c0 tx=0x7f495001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- 172.21.15.157:0/3158682035 shutdown_connections 2023-11-06T03:11:53.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f496015ea90 0x7f49600b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- 172.21.15.157:0/3158682035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015e5e0 0x7f49600b3410 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- 172.21.15.157:0/3158682035 >> 172.21.15.157:0/3158682035 conn(0x7f4960065930 msgr2=0x7f4960065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:53.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- 172.21.15.157:0/3158682035 shutdown_connections 2023-11-06T03:11:53.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- 172.21.15.157:0/3158682035 wait complete. 2023-11-06T03:11:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 Processor -- start 2023-11-06T03:11:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- start start 2023-11-06T03:11:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49601551e0 0x7f4960156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f496015e5e0 0x7f49600cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960158810 con 0x7f496015ea90 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960156a60 con 0x7f496015e5e0 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4967bed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4960156090 con 0x7f49601551e0 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f496618a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f496618a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49868/0 (socket says 172.21.15.157:49868) 2023-11-06T03:11:53.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f496618a640 1 -- 172.21.15.157:0/1027787757 learned_addr learned my addr 172.21.15.157:0/1027787757 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:53.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4965989640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49601551e0 0x7f4960156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.126+0000 7f4965188640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f496015e5e0 0x7f49600cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 -- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49601551e0 msgr2=0x7f4960156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49601551e0 0x7f4960156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 -- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f496015e5e0 msgr2=0x7f49600cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f496015e5e0 0x7f49600cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 -- 172.21.15.157:0/1027787757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950009420 con 0x7f496015ea90 2023-11-06T03:11:53.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f496618a640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4958007e90 tx=0x7f495800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958014420 con 0x7f496015ea90 2023-11-06T03:11:53.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4958014580 con 0x7f496015ea90 2023-11-06T03:11:53.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4960155760 con 0x7f496015ea90 2023-11-06T03:11:53.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958014930 con 0x7f496015ea90 2023-11-06T03:11:53.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4960155d20 con 0x7f496015ea90 2023-11-06T03:11:53.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4960072750 con 0x7f496015ea90 2023-11-06T03:11:53.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4958014a90 con 0x7f496015ea90 2023-11-06T03:11:53.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 0x7f4944067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f49580838c0 con 0x7f496015ea90 2023-11-06T03:11:53.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f4965989640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 0x7f4944067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.130+0000 7f4965989640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 0x7f4944067f50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f495c0011f0 tx=0x7f495c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.134+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4958051420 con 0x7f496015ea90 2023-11-06T03:11:53.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.290+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4960071730 con 0x7f496015ea90 2023-11-06T03:11:53.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f49567fc640 1 -- 172.21.15.157:0/1027787757 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4958054a40 con 0x7f496015ea90 2023-11-06T03:11:53.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 msgr2=0x7f4944067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 0x7f4944067f50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f495c0011f0 tx=0x7f495c009040 comp rx=0 tx=0).stop 2023-11-06T03:11:53.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 msgr2=0x7f496014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4958007e90 tx=0x7f495800e4b0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 shutdown_connections 2023-11-06T03:11:53.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4944065a90 0x7f4944067f50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f496015ea90 0x7f496014cd00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f496015e5e0 0x7f49600cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 --2- 172.21.15.157:0/1027787757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49601551e0 0x7f4960156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 >> 172.21.15.157:0/1027787757 conn(0x7f4960065930 msgr2=0x7f49600aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:53.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 shutdown_connections 2023-11-06T03:11:53.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.298+0000 7f4967bed640 1 -- 172.21.15.157:0/1027787757 wait complete. 2023-11-06T03:11:53.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:11:53.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:11:53.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:11:53.402 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:11:53.403 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:11:53.403 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:53.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:11:53.418 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:11:53.418 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:53.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:11:53.450 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:11:53.450 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:53.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:11:53.465 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:11:53.465 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:11:53.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:11:53.498 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:11:53.498 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:11:53.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.634+0000 7f5dda29b640 1 Processor -- start 2023-11-06T03:11:53.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dda29b640 1 -- start start 2023-11-06T03:11:53.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dda29b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dda29b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40cc2b0 con 0x7f5dd40b2b10 2023-11-06T03:11:53.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dda29b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40b01a0 con 0x7f5dd415ac80 2023-11-06T03:11:53.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dda29b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40cfb10 con 0x7f5dd40cd730 2023-11-06T03:11:53.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5dd40b2b10 0x7f5dd40ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51378/0 (socket says 172.21.15.157:51378) 2023-11-06T03:11:53.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd37fe640 1 -- 172.21.15.157:0/185391923 learned_addr learned my addr 172.21.15.157:0/185391923 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:53.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd8838640 1 --2- 172.21.15.157:0/185391923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993237770 0 0) 0x7f5dd40cc2b0 con 0x7f5dd40b2b10 2023-11-06T03:11:53.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc0003530 con 0x7f5dd40b2b10 2023-11-06T03:11:53.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400954587 0 0) 0x7f5dc0003530 con 0x7f5dd40b2b10 2023-11-06T03:11:53.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd8838640 1 -- 172.21.15.157:0/185391923 >> v1:172.21.15.195:6789/0 conn(0x7f5dd415ac80 legacy=0x7f5dd40b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd8838640 1 -- 172.21.15.157:0/185391923 >> v1:172.21.15.157:6789/0 conn(0x7f5dd40b2b10 legacy=0x7f5dd40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.638+0000 7f5dd8838640 1 -- 172.21.15.157:0/185391923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40cfdf0 con 0x7f5dd40cd730 2023-11-06T03:11:53.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd8838640 1 --2- 172.21.15.157:0/185391923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd40cec60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009dc0 tx=0x7f5dc401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=768c90663287f2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a59a0 con 0x7f5dd40cd730 2023-11-06T03:11:53.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40a5b00 con 0x7f5dd40cd730 2023-11-06T03:11:53.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd2ffd640 1 -- 172.21.15.157:0/185391923 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a5e70 con 0x7f5dd40cd730 2023-11-06T03:11:53.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- 172.21.15.157:0/185391923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 msgr2=0x7f5dd40cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 --2- 172.21.15.157:0/185391923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd40cec60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009dc0 tx=0x7f5dc401c970 comp rx=0 tx=0).stop 2023-11-06T03:11:53.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- 172.21.15.157:0/185391923 shutdown_connections 2023-11-06T03:11:53.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 --2- 172.21.15.157:0/185391923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd40cec60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- 172.21.15.157:0/185391923 >> 172.21.15.157:0/185391923 conn(0x7f5dd406d560 msgr2=0x7f5dd406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:53.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- 172.21.15.157:0/185391923 shutdown_connections 2023-11-06T03:11:53.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- 172.21.15.157:0/185391923 wait complete. 2023-11-06T03:11:53.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 Processor -- start 2023-11-06T03:11:53.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- start start 2023-11-06T03:11:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 0x7f5dd4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd4145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd41f7df0 con 0x7f5dd415ac80 2023-11-06T03:11:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c9380 con 0x7f5dd40b2b10 2023-11-06T03:11:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dda29b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c9660 con 0x7f5dd40cd730 2023-11-06T03:11:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd8838640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd8838640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49882/0 (socket says 172.21.15.157:49882) 2023-11-06T03:11:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.642+0000 7f5dd8838640 1 -- 172.21.15.157:0/3995037843 learned_addr learned my addr 172.21.15.157:0/3995037843 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:53.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd3fff640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 0x7f5dd4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd37fe640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd4145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 -- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 msgr2=0x7f5dd4145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd4145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 -- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 msgr2=0x7f5dd4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 0x7f5dd4154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 -- 172.21.15.157:0/3995037843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009a70 con 0x7f5dd415ac80 2023-11-06T03:11:53.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd3fff640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 0x7f5dd4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:11:53.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd8838640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4019580 tx=0x7f5dc4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4004240 con 0x7f5dd415ac80 2023-11-06T03:11:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40043a0 con 0x7f5dd415ac80 2023-11-06T03:11:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd40c9940 con 0x7f5dd415ac80 2023-11-06T03:11:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4004710 con 0x7f5dd415ac80 2023-11-06T03:11:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd40c4fa0 con 0x7f5dd415ac80 2023-11-06T03:11:53.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd40c9e60 con 0x7f5dd415ac80 2023-11-06T03:11:53.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5dc40b4030 con 0x7f5dd415ac80 2023-11-06T03:11:53.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 0x7f5db4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:53.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5dc411e020 con 0x7f5dd415ac80 2023-11-06T03:11:53.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd3fff640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 0x7f5db4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:53.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.646+0000 7f5dd3fff640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 0x7f5db4067f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc000f70 tx=0x7f5dcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:53.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.650+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5dd40c9e60 con 0x7f5dd415ac80 2023-11-06T03:11:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dd40c9e60 con 0x7f5dd415ac80 2023-11-06T03:11:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dd0ff9640 1 -- 172.21.15.157:0/3995037843 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f5dc40b2de0 con 0x7f5dd415ac80 2023-11-06T03:11:53.838 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:53.839 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/522926628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":522926628},{"type":"v1","addr":"172.21.15.195:6835","nonce":522926628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6064,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3057023744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3057023744},{"type":"v1","addr":"172.21.15.195:6837","nonce":3057023744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/57036926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":57036926},{"type":"v1","addr":"172.21.15.157:6833","nonce":57036926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":7725,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3176893529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3176893529},{"type":"v1","addr":"172.21.15.157:6835","nonce":3176893529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-11-06T03:11:53.839 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 75 2023-11-06T03:11:53.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 msgr2=0x7f5db4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 0x7f5db4067f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc000f70 tx=0x7f5dcc009040 comp rx=0 tx=0).stop 2023-11-06T03:11:53.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 msgr2=0x7f5dd4146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:53.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4019580 tx=0x7f5dc4019710 comp rx=0 tx=0).stop 2023-11-06T03:11:53.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 shutdown_connections 2023-11-06T03:11:53.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5db4065a60 0x7f5db4067f20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5dd415ac80 0x7f5dd4146260 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5dd40cd730 0x7f5dd4145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 --2- 172.21.15.157:0/3995037843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5dd40b2b10 0x7f5dd4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:53.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 >> 172.21.15.157:0/3995037843 conn(0x7f5dd406d560 msgr2=0x7f5dd40b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:53.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 shutdown_connections 2023-11-06T03:11:53.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:53.838+0000 7f5dda29b640 1 -- 172.21.15.157:0/3995037843 wait complete. 2023-11-06T03:11:53.857 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:11:53.857 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:11:53.858 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:11:53.858 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:11:53.858 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:11:53.859 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:11:53.859 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:11:53.859 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:11:53.860 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:11:53.860 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:11:53.860 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:11:53.860 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:11:53.861 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:11:53.861 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:11:53.861 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:11:53.862 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:11:53.867 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:11:53.867 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:11:53.870 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:11:53.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:11:53.873 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:11:53.873 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:11:53.874 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:11:53.875 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:11:53.913 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:11:53.914 DEBUG:teuthology.parallel:result is None 2023-11-06T03:11:53.915 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:11:53.915 DEBUG:teuthology.parallel:result is None 2023-11-06T03:11:53.918 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:11:53.918 DEBUG:teuthology.parallel:result is None 2023-11-06T03:11:53.919 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:11:53.919 DEBUG:teuthology.parallel:result is None 2023-11-06T03:11:53.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:11:53.974 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:11:53.978 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:11:53.992 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:11:53.993 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:11:54.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 Processor -- start 2023-11-06T03:11:54.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 -- start start 2023-11-06T03:11:54.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ebc065e70 0x7f2ebc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0cf820 con 0x7f2ebc06d100 2023-11-06T03:11:54.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc06d8f0 con 0x7f2ebc065e70 2023-11-06T03:11:54.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ec13d2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc161740 con 0x7f2ebc157070 2023-11-06T03:11:54.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ebc065e70 0x7f2ebc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ebc065e70 0x7f2ebc06cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49912/0 (socket says 172.21.15.157:49912) 2023-11-06T03:11:54.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 -- 172.21.15.157:0/2041861072 learned_addr learned my addr 172.21.15.157:0/2041861072 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:54.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2ebaffd640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 -- 172.21.15.157:0/2041861072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 msgr2=0x7f2ebc065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:54.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 -- 172.21.15.157:0/2041861072 >> v1:172.21.15.195:6789/0 conn(0x7f2ebc06d100 legacy=0x7f2ebc070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:54.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.030+0000 7f2eba7fc640 1 -- 172.21.15.157:0/2041861072 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc15e150 con 0x7f2ebc065e70 2023-11-06T03:11:54.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2eba7fc640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc065e70 0x7f2ebc06cbc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d76f1ba6216bc79 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:54.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2eb9ffb640 1 -- 172.21.15.157:0/2041861072 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ab600 con 0x7f2ebc065e70 2023-11-06T03:11:54.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2eb9ffb640 1 -- 172.21.15.157:0/2041861072 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40a7050 con 0x7f2ebc065e70 2023-11-06T03:11:54.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2eb9ffb640 1 -- 172.21.15.157:0/2041861072 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ab980 con 0x7f2ebc065e70 2023-11-06T03:11:54.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/2041861072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc065e70 msgr2=0x7f2ebc06cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:54.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc065e70 0x7f2ebc06cbc0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea401c4b0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/2041861072 shutdown_connections 2023-11-06T03:11:54.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc065e70 0x7f2ebc06cbc0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/2041861072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/2041861072 >> 172.21.15.157:0/2041861072 conn(0x7f2ebc069d70 msgr2=0x7f2ebc06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:54.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/2041861072 shutdown_connections 2023-11-06T03:11:54.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/2041861072 wait complete. 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 Processor -- start 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- start start 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc065e70 0x7f2ebc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc1558b0 con 0x7f2ebc06d100 2023-11-06T03:11:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc156930 con 0x7f2ebc065e70 2023-11-06T03:11:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2ec13d2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc154ee0 con 0x7f2ebc157070 2023-11-06T03:11:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.034+0000 7f2eba7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ebaffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc065e70 0x7f2ebc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49926/0 (socket says 172.21.15.157:49926) 2023-11-06T03:11:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 -- 172.21.15.157:0/1829604763 learned_addr learned my addr 172.21.15.157:0/1829604763 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ebb7fe640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 -- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 msgr2=0x7f2ebc0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 -- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc065e70 msgr2=0x7f2ebc155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc065e70 0x7f2ebc155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 -- 172.21.15.157:0/1829604763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40094d0 con 0x7f2ebc06d100 2023-11-06T03:11:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eba7fc640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40ae040 tx=0x7f2ea4019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b4420 con 0x7f2ebc06d100 2023-11-06T03:11:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40b45a0 con 0x7f2ebc06d100 2023-11-06T03:11:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc14f130 con 0x7f2ebc06d100 2023-11-06T03:11:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40b4950 con 0x7f2ebc06d100 2023-11-06T03:11:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc154890 con 0x7f2ebc06d100 2023-11-06T03:11:54.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc155090 con 0x7f2ebc06d100 2023-11-06T03:11:54.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ea40b2070 con 0x7f2ebc06d100 2023-11-06T03:11:54.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 0x7f2e9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:54.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ea411c7c0 con 0x7f2ebc06d100 2023-11-06T03:11:54.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ebaffd640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 0x7f2e9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:54.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.038+0000 7f2ebaffd640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 0x7f2e9c067f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40113e0 tx=0x7f2eb400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:54.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.042+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ebc155090 con 0x7f2ebc06d100 2023-11-06T03:11:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:54.198+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2ebc155090 con 0x7f2ebc06d100 2023-11-06T03:11:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.030+0000 7f2eb37fe640 1 -- 172.21.15.157:0/1829604763 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2ebc155090 con 0x7f2ebc06d100 2023-11-06T03:11:55.031 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:11:55.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 msgr2=0x7f2e9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 0x7f2e9c067f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40113e0 tx=0x7f2eb400b040 comp rx=0 tx=0).stop 2023-11-06T03:11:55.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 msgr2=0x7f2ebc14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40ae040 tx=0x7f2ea4019250 comp rx=0 tx=0).stop 2023-11-06T03:11:55.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 shutdown_connections 2023-11-06T03:11:55.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e9c065a60 0x7f2e9c067f20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ebc157070 0x7f2ebc0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ebc06d100 0x7f2ebc14ebf0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 --2- 172.21.15.157:0/1829604763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ebc065e70 0x7f2ebc155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 >> 172.21.15.157:0/1829604763 conn(0x7f2ebc069d70 msgr2=0x7f2ebc0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 shutdown_connections 2023-11-06T03:11:55.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.034+0000 7f2ec13d2640 1 -- 172.21.15.157:0/1829604763 wait complete. 2023-11-06T03:11:55.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:11:55.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.182+0000 7f8a021b3640 1 Processor -- start 2023-11-06T03:11:55.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.182+0000 7f8a021b3640 1 -- start start 2023-11-06T03:11:55.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f8a021b3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f89fc0b3b20 0x7f89fc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f8a021b3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15f580 con 0x7f89fc1551e0 2023-11-06T03:11:55.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f8a021b3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15efc0 con 0x7f89fc15e5e0 2023-11-06T03:11:55.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f8a021b3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc0704b0 con 0x7f89fc0b3b20 2023-11-06T03:11:55.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f89fc1551e0 0x7f89fc06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51412/0 (socket says 172.21.15.157:51412) 2023-11-06T03:11:55.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fb7fe640 1 -- 172.21.15.157:0/4268883622 learned_addr learned my addr 172.21.15.157:0/4268883622 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:55.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 --2- 172.21.15.157:0/4268883622 >> v2:172.21.15.195:3300/0 conn(0x7f89fc0b3b20 0x7f89fc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840346298 0 0) 0x7f89fc15f580 con 0x7f89fc1551e0 2023-11-06T03:11:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e8003530 con 0x7f89fc1551e0 2023-11-06T03:11:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 937591630 0 0) 0x7f89e8003530 con 0x7f89fc1551e0 2023-11-06T03:11:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 >> v2:172.21.15.195:3300/0 conn(0x7f89fc0b3b20 msgr2=0x7f89fc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 --2- 172.21.15.157:0/4268883622 >> v2:172.21.15.195:3300/0 conn(0x7f89fc0b3b20 0x7f89fc15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 >> v1:172.21.15.195:6789/0 conn(0x7f89fc15e5e0 legacy=0x7f89fc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc15f2a0 con 0x7f89fc1551e0 2023-11-06T03:11:55.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f89ec001680 con 0x7f89fc1551e0 2023-11-06T03:11:55.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 >> v1:172.21.15.157:6789/0 conn(0x7f89fc1551e0 legacy=0x7f89fc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e8003980 0x7f89e8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80047d0 0x7f89e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 0x7f89e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0704b0 con 0x7f89e80052b0 2023-11-06T03:11:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800c180 con 0x7f89e80047d0 2023-11-06T03:11:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800c340 con 0x7f89e8003980 2023-11-06T03:11:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 0x7f89e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fb7fe640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e8003980 0x7f89e8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89f3fff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80047d0 0x7f89e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 -- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e8003980 msgr2=0x7f89e8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89e8003980 0x7f89e8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 -- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80047d0 msgr2=0x7f89e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80047d0 0x7f89e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89fbfff640 1 -- 172.21.15.157:0/4268883622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc15f2a0 con 0x7f89e80052b0 2023-11-06T03:11:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.186+0000 7f89f3fff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80047d0 0x7f89e8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:55.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f89fbfff640 1 -- 172.21.15.157:0/4268883622 reap_dead start 2023-11-06T03:11:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f89fbfff640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 0x7f89e800b990 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009f90 tx=0x7f89f401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54fdfaac88def9a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40a5c40 con 0x7f89e80052b0 2023-11-06T03:11:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f400f040 con 0x7f89e80052b0 2023-11-06T03:11:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f89faffd640 1 -- 172.21.15.157:0/4268883622 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40039a0 con 0x7f89e80052b0 2023-11-06T03:11:55.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 msgr2=0x7f89e800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 0x7f89e800b990 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009f90 tx=0x7f89f401cc30 comp rx=0 tx=0).stop 2023-11-06T03:11:55.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- 172.21.15.157:0/4268883622 shutdown_connections 2023-11-06T03:11:55.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 --2- 172.21.15.157:0/4268883622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89e80052b0 0x7f89e800b990 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- 172.21.15.157:0/4268883622 >> 172.21.15.157:0/4268883622 conn(0x7f89fc065930 msgr2=0x7f89fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- 172.21.15.157:0/4268883622 shutdown_connections 2023-11-06T03:11:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- 172.21.15.157:0/4268883622 wait complete. 2023-11-06T03:11:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 Processor -- start 2023-11-06T03:11:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.190+0000 7f8a021b3640 1 -- start start 2023-11-06T03:11:55.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80052b0 0x7f89fc137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 0x7f89fc14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc153920 con 0x7f89fc14ad40 2023-11-06T03:11:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc1398f0 con 0x7f89e80052b0 2023-11-06T03:11:55.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc139bd0 con 0x7f89fc0aef10 2023-11-06T03:11:55.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49950/0 (socket says 172.21.15.157:49950) 2023-11-06T03:11:55.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 -- 172.21.15.157:0/1223137658 learned_addr learned my addr 172.21.15.157:0/1223137658 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:55.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89fb7fe640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80052b0 0x7f89fc137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89fbfff640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 0x7f89fc14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 -- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 msgr2=0x7f89fc14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 0x7f89fc14cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 -- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80052b0 msgr2=0x7f89fc137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80052b0 0x7f89fc137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 -- 172.21.15.157:0/1223137658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc139eb0 con 0x7f89fc14ad40 2023-11-06T03:11:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89fbfff640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 0x7f89fc14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f3fff640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f89e400d770 tx=0x7f89e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4005270 con 0x7f89fc14ad40 2023-11-06T03:11:55.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40053f0 con 0x7f89fc14ad40 2023-11-06T03:11:55.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc13a190 con 0x7f89fc14ad40 2023-11-06T03:11:55.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400b8f0 con 0x7f89fc14ad40 2023-11-06T03:11:55.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc13a750 con 0x7f89fc14ad40 2023-11-06T03:11:55.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc072750 con 0x7f89fc14ad40 2023-11-06T03:11:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f89e400ba50 con 0x7f89fc14ad40 2023-11-06T03:11:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 0x7f89dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f89e4082e70 con 0x7f89fc14ad40 2023-11-06T03:11:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.194+0000 7f89fb7fe640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 0x7f89dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.198+0000 7f89fb7fe640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 0x7f89dc067f20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000fc0 tx=0x7f89ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.198+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f89e4050ab0 con 0x7f89fc14ad40 2023-11-06T03:11:55.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.410+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f89fc0cb110 con 0x7f89fc14ad40 2023-11-06T03:11:55.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.410+0000 7f89f8ff9640 1 -- 172.21.15.157:0/1223137658 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f89e4019310 con 0x7f89fc14ad40 2023-11-06T03:11:55.413 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:55.413 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:11:55.413 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:11:55.413 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.414 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:11:55.414 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:11:55.414 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.414 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.415 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:11:55.415 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:11:55.415 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.415 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.415 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.416 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.416 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:11:55.416 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:11:55.416 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.417 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:11:55.417 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:11:55.417 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.417 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.417 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.418 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.418 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:11:55.418 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:11:55.418 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.418 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.419 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.420 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:11:55.420 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:11:55.420 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:11:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:11:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.421 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.422 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:11:55.422 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:11:55.422 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.422 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.422 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.423 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.423 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.423 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.424 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.425 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.425 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:11:55.425 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:11:55.425 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.426 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.426 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.426 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.426 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.427 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.427 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:11:55.427 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:11:55.427 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.427 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.428 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.428 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.428 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.428 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.428 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.429 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:11:55.429 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:11:55.429 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.429 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.429 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.430 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.431 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:11:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:11:55.432 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.433 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.433 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.433 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.433 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.433 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.434 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.434 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:11:55.434 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:11:55.434 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:11:55.435 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:11:55.435 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.435 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.435 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.435 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.436 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:11:55.436 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:11:55.436 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.436 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.436 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:11:55.437 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:11:55.438 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.438 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.438 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:11:55.438 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:11:55.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:11:55.439 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.439 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.439 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.439 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:11:55.439 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:11:55.440 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.440 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.440 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:11:55.440 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:11:55.440 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:11:55.441 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.441 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.441 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.441 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:11:55.441 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:11:55.442 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.442 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:11:55.442 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:11:55.442 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:11:55.442 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:11:55.443 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.444 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:11:55.444 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:11:55.444 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:11:55.444 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.444 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.445 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.445 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.445 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:11:55.445 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:11:55.445 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.446 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:11:55.446 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:11:55.446 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.446 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:55.447 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:55.447 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:11:55.447 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:11:55.447 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:11:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:11:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:11:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:11:55.449 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:55.450 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:11:55.450 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:11:55.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 msgr2=0x7f89dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 0x7f89dc067f20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000fc0 tx=0x7f89ec009040 comp rx=0 tx=0).stop 2023-11-06T03:11:55.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 msgr2=0x7f89fc1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f89e400d770 tx=0x7f89e400dc30 comp rx=0 tx=0).stop 2023-11-06T03:11:55.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 shutdown_connections 2023-11-06T03:11:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f89dc065a60 0x7f89dc067f20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f89fc0aef10 0x7f89fc14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89fc14ad40 0x7f89fc1533e0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 --2- 172.21.15.157:0/1223137658 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89e80052b0 0x7f89fc137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 >> 172.21.15.157:0/1223137658 conn(0x7f89fc065930 msgr2=0x7f89fc150a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 shutdown_connections 2023-11-06T03:11:55.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.414+0000 7f8a021b3640 1 -- 172.21.15.157:0/1223137658 wait complete. 2023-11-06T03:11:55.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.574+0000 7f643ee93640 1 Processor -- start 2023-11-06T03:11:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.574+0000 7f643ee93640 1 -- start start 2023-11-06T03:11:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643ee93640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6438160490 0x7f64380cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643ee93640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6438160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643ee93640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64381612f0 con 0x7f643815d360 2023-11-06T03:11:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643ee93640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6438161780 con 0x7f6438160490 2023-11-06T03:11:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643ee93640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380d0450 con 0x7f6438160950 2023-11-06T03:11:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f643815d360 0x7f643815d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51416/0 (socket says 172.21.15.157:51416) 2023-11-06T03:11:55.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 -- 172.21.15.157:0/156687574 learned_addr learned my addr 172.21.15.157:0/156687574 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:55.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f6437fff640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6438160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643cc2f640 1 --2- 172.21.15.157:0/156687574 >> v2:172.21.15.195:3300/0 conn(0x7f6438160490 0x7f64380cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292467204 0 0) 0x7f64381612f0 con 0x7f643815d360 2023-11-06T03:11:55.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6420003530 con 0x7f643815d360 2023-11-06T03:11:55.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653734645 0 0) 0x7f6420003530 con 0x7f643815d360 2023-11-06T03:11:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 msgr2=0x7f6438160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6438160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 >> v2:172.21.15.195:3300/0 conn(0x7f6438160490 msgr2=0x7f64380cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 --2- 172.21.15.157:0/156687574 >> v2:172.21.15.195:3300/0 conn(0x7f6438160490 0x7f64380cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380cd860 con 0x7f643815d360 2023-11-06T03:11:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6430001a00 con 0x7f643815d360 2023-11-06T03:11:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 >> v1:172.21.15.157:6789/0 conn(0x7f643815d360 legacy=0x7f643815d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6420003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6420003f00 0x7f64200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6420004d10 0x7f6420005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438161780 con 0x7f6420004d10 2023-11-06T03:11:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380d0450 con 0x7f6420003f00 2023-11-06T03:11:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642000b990 con 0x7f6438160950 2023-11-06T03:11:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6420004d10 0x7f6420005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f6437fff640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6420003f00 0x7f64200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643cc2f640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6420003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 -- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 msgr2=0x7f6420003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6438160950 0x7f6420003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 -- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6420003f00 msgr2=0x7f64200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6420003f00 0x7f64200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 -- 172.21.15.157:0/156687574 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420003530 con 0x7f6420004d10 2023-11-06T03:11:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f6437fff640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6420003f00 0x7f64200047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f643d430640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6420004d10 0x7f6420005190 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f643000a860 tx=0x7f643001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98a6f1898e105cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300ad630 con 0x7f6420004d10 2023-11-06T03:11:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64300a8070 con 0x7f6420004d10 2023-11-06T03:11:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.578+0000 7f64377fe640 1 -- 172.21.15.157:0/156687574 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300ada30 con 0x7f6420004d10 2023-11-06T03:11:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/156687574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6420004d10 msgr2=0x7f6420005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 --2- 172.21.15.157:0/156687574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6420004d10 0x7f6420005190 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f643000a860 tx=0x7f643001d300 comp rx=0 tx=0).stop 2023-11-06T03:11:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 -- 172.21.15.157:0/156687574 reap_dead start 2023-11-06T03:11:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/156687574 shutdown_connections 2023-11-06T03:11:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/156687574 >> 172.21.15.157:0/156687574 conn(0x7f6438070790 msgr2=0x7f6438070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/156687574 shutdown_connections 2023-11-06T03:11:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/156687574 wait complete. 2023-11-06T03:11:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 Processor -- start 2023-11-06T03:11:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- start start 2023-11-06T03:11:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6438156c60 0x7f64380c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64381f7cf0 0x7f6438157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438157040 con 0x7f6438157f40 2023-11-06T03:11:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64381f84d0 con 0x7f6438156c60 2023-11-06T03:11:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64381f87b0 con 0x7f64381f7cf0 2023-11-06T03:11:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49964/0 (socket says 172.21.15.157:49964) 2023-11-06T03:11:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 -- 172.21.15.157:0/2224321077 learned_addr learned my addr 172.21.15.157:0/2224321077 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643cc2f640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6438156c60 0x7f64380c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f6437fff640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64381f7cf0 0x7f6438157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 -- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64381f7cf0 msgr2=0x7f6438157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64381f7cf0 0x7f6438157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 -- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6438156c60 msgr2=0x7f64380c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6438156c60 0x7f64380c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 -- 172.21.15.157:0/2224321077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64381f8a90 con 0x7f6438157f40 2023-11-06T03:11:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643d430640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f643000a860 tx=0x7f6420005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430001fe0 con 0x7f6438157f40 2023-11-06T03:11:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64300011d0 con 0x7f6438157f40 2023-11-06T03:11:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64381f8d70 con 0x7f6438157f40 2023-11-06T03:11:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300ad630 con 0x7f6438157f40 2023-11-06T03:11:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64381f8170 con 0x7f6438157f40 2023-11-06T03:11:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.582+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6438072750 con 0x7f6438157f40 2023-11-06T03:11:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.586+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f64300aa030 con 0x7f6438157f40 2023-11-06T03:11:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.586+0000 7f64357fa640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 0x7f6424067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.586+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f643011cdc0 con 0x7f6438157f40 2023-11-06T03:11:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.586+0000 7f643cc2f640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 0x7f6424067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.586+0000 7f643cc2f640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 0x7f6424067f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6428009ab0 tx=0x7f6428008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.590+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f64300ea9c0 con 0x7f6438157f40 2023-11-06T03:11:55.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.738+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64381f9220 con 0x7f6438157f40 2023-11-06T03:11:55.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.738+0000 7f64357fa640 1 -- 172.21.15.157:0/2224321077 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+14220 (secure 0 0 0) 0x7f64300ba090 con 0x7f6438157f40 2023-11-06T03:11:55.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:55.739 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":95,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:55.026841+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 msgr2=0x7f6424067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 0x7f6424067f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6428009ab0 tx=0x7f6428008040 comp rx=0 tx=0).stop 2023-11-06T03:11:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 msgr2=0x7f64380d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f643000a860 tx=0x7f6420005800 comp rx=0 tx=0).stop 2023-11-06T03:11:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 shutdown_connections 2023-11-06T03:11:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6424065a60 0x7f6424067f20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6438157f40 0x7f64380d1c40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64381f7cf0 0x7f6438157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 --2- 172.21.15.157:0/2224321077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6438156c60 0x7f64380c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 >> 172.21.15.157:0/2224321077 conn(0x7f6438070790 msgr2=0x7f643815c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 shutdown_connections 2023-11-06T03:11:55.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.742+0000 7f643ee93640 1 -- 172.21.15.157:0/2224321077 wait complete. 2023-11-06T03:11:55.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:11:55.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:11:55.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 Processor -- start 2023-11-06T03:11:55.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 -- start start 2023-11-06T03:11:55.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d540ce800 0x7f9d540cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9d5406b6a0 0x7f9d5406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540cf1a0 con 0x7f9d54146370 2023-11-06T03:11:55.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b2120 con 0x7f9d540ce800 2023-11-06T03:11:55.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d59f5b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5406c3d0 con 0x7f9d5406b6a0 2023-11-06T03:11:55.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d52ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d540ce800 0x7f9d540cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d52ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d540ce800 0x7f9d540cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49976/0 (socket says 172.21.15.157:49976) 2023-11-06T03:11:55.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d52ffd640 1 -- 172.21.15.157:0/1470603325 learned_addr learned my addr 172.21.15.157:0/1470603325 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:55.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d53fff640 1 --2- 172.21.15.157:0/1470603325 >> v2:172.21.15.195:3300/0 conn(0x7f9d5406b6a0 0x7f9d5406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d52ffd640 1 -- 172.21.15.157:0/1470603325 >> v2:172.21.15.195:3300/0 conn(0x7f9d5406b6a0 msgr2=0x7f9d5406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:55.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.882+0000 7f9d52ffd640 1 --2- 172.21.15.157:0/1470603325 >> v2:172.21.15.195:3300/0 conn(0x7f9d5406b6a0 0x7f9d5406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d52ffd640 1 -- 172.21.15.157:0/1470603325 >> v1:172.21.15.195:6789/0 conn(0x7f9d54146370 legacy=0x7f9d540b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d52ffd640 1 -- 172.21.15.157:0/1470603325 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5406f5a0 con 0x7f9d540ce800 2023-11-06T03:11:55.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d52ffd640 1 --2- 172.21.15.157:0/1470603325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d540ce800 0x7f9d540cec60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009870 tx=0x7f9d4801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958ca50cb31fc1b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d527fc640 1 -- 172.21.15.157:0/1470603325 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480ab640 con 0x7f9d540ce800 2023-11-06T03:11:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d527fc640 1 -- 172.21.15.157:0/1470603325 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d480a7070 con 0x7f9d540ce800 2023-11-06T03:11:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d527fc640 1 -- 172.21.15.157:0/1470603325 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480ab9c0 con 0x7f9d540ce800 2023-11-06T03:11:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1470603325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d540ce800 msgr2=0x7f9d540cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1470603325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d540ce800 0x7f9d540cec60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009870 tx=0x7f9d4801c440 comp rx=0 tx=0).stop 2023-11-06T03:11:55.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1470603325 shutdown_connections 2023-11-06T03:11:55.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1470603325 >> v2:172.21.15.195:3300/0 conn(0x7f9d5406b6a0 0x7f9d5406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1470603325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d540ce800 0x7f9d540cec60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1470603325 >> 172.21.15.157:0/1470603325 conn(0x7f9d54065930 msgr2=0x7f9d54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:55.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1470603325 shutdown_connections 2023-11-06T03:11:55.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.886+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1470603325 wait complete. 2023-11-06T03:11:55.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 Processor -- start 2023-11-06T03:11:55.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- start start 2023-11-06T03:11:55.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d5406b6a0 0x7f9d5413a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d540ce800 0x7f9d5415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415af30 con 0x7f9d54146370 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5413a710 con 0x7f9d5406b6a0 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540af5c0 con 0x7f9d540ce800 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49988/0 (socket says 172.21.15.157:49988) 2023-11-06T03:11:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 -- 172.21.15.157:0/1924260502 learned_addr learned my addr 172.21.15.157:0/1924260502 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d52ffd640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d540ce800 0x7f9d5415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d537fe640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d5406b6a0 0x7f9d5413a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 -- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d540ce800 msgr2=0x7f9d5415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d540ce800 0x7f9d5415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 -- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d5406b6a0 msgr2=0x7f9d5413a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d5406b6a0 0x7f9d5413a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 -- 172.21.15.157:0/1924260502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480094d0 con 0x7f9d54146370 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d53fff640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c007e60 tx=0x7f9d4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c014420 con 0x7f9d54146370 2023-11-06T03:11:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0145a0 con 0x7f9d54146370 2023-11-06T03:11:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5415a600 con 0x7f9d54146370 2023-11-06T03:11:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c014950 con 0x7f9d54146370 2023-11-06T03:11:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5415abc0 con 0x7f9d54146370 2023-11-06T03:11:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d54072750 con 0x7f9d54146370 2023-11-06T03:11:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d4c014ab0 con 0x7f9d54146370 2023-11-06T03:11:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 0x7f9d40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9d4c083840 con 0x7f9d54146370 2023-11-06T03:11:55.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.890+0000 7f9d537fe640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 0x7f9d40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.894+0000 7f9d537fe640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 0x7f9d40067f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000f70 tx=0x7f9d44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:55.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:55.894+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9d4c051480 con 0x7f9d54146370 2023-11-06T03:11:56.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.050+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d540ac950 con 0x7f9d54146370 2023-11-06T03:11:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.050+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1924260502 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d4c054aa0 con 0x7f9d54146370 2023-11-06T03:11:56.052 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:11:56.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 msgr2=0x7f9d40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 0x7f9d40067f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d44000f70 tx=0x7f9d44009040 comp rx=0 tx=0).stop 2023-11-06T03:11:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 msgr2=0x7f9d5415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c007e60 tx=0x7f9d4c00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 shutdown_connections 2023-11-06T03:11:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d40065a60 0x7f9d40067f20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d54146370 0x7f9d5415e110 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d540ce800 0x7f9d5415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 --2- 172.21.15.157:0/1924260502 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d5406b6a0 0x7f9d5413a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 >> 172.21.15.157:0/1924260502 conn(0x7f9d54065930 msgr2=0x7f9d54070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 shutdown_connections 2023-11-06T03:11:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.054+0000 7f9d59f5b640 1 -- 172.21.15.157:0/1924260502 wait complete. 2023-11-06T03:11:56.072 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:11:56.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 Processor -- start 2023-11-06T03:11:56.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 -- start start 2023-11-06T03:11:56.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1e500b3b20 0x7f1e5015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5015f580 con 0x7f1e5015e5e0 2023-11-06T03:11:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5015efc0 con 0x7f1e500b3b20 2023-11-06T03:11:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e549e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5015f2a0 con 0x7f1e501551e0 2023-11-06T03:11:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4dd9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1e5015e5e0 0x7f1e500b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:53206/0 (socket says 172.21.15.157:53206) 2023-11-06T03:11:56.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:42900/0 (socket says 172.21.15.157:42900) 2023-11-06T03:11:56.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4dd9b640 1 -- 172.21.15.157:0/2172213115 learned_addr learned my addr 172.21.15.157:0/2172213115 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:56.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4ed9d640 1 --2- 172.21.15.157:0/2172213115 >> v2:172.21.15.195:3300/0 conn(0x7f1e500b3b20 0x7f1e5015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 -- 172.21.15.157:0/2172213115 >> v2:172.21.15.195:3300/0 conn(0x7f1e500b3b20 msgr2=0x7f1e5015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 --2- 172.21.15.157:0/2172213115 >> v2:172.21.15.195:3300/0 conn(0x7f1e500b3b20 0x7f1e5015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 -- 172.21.15.157:0/2172213115 >> v1:172.21.15.195:6789/0 conn(0x7f1e5015e5e0 legacy=0x7f1e500b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.194+0000 7f1e4e59c640 1 -- 172.21.15.157:0/2172213115 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5006f5a0 con 0x7f1e501551e0 2023-11-06T03:11:56.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e4e59c640 1 --2- 172.21.15.157:0/2172213115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683fb56d3c42ebaf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e4d59a640 1 -- 172.21.15.157:0/2172213115 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ab600 con 0x7f1e501551e0 2023-11-06T03:11:56.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e4d59a640 1 -- 172.21.15.157:0/2172213115 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400a7050 con 0x7f1e501551e0 2023-11-06T03:11:56.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e4d59a640 1 -- 172.21.15.157:0/2172213115 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ab980 con 0x7f1e501551e0 2023-11-06T03:11:56.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- 172.21.15.157:0/2172213115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 msgr2=0x7f1e5006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- 172.21.15.157:0/2172213115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- 172.21.15.157:0/2172213115 shutdown_connections 2023-11-06T03:11:56.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- 172.21.15.157:0/2172213115 >> v2:172.21.15.195:3300/0 conn(0x7f1e500b3b20 0x7f1e5015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- 172.21.15.157:0/2172213115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e501551e0 0x7f1e5006fda0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- 172.21.15.157:0/2172213115 >> 172.21.15.157:0/2172213115 conn(0x7f1e50065930 msgr2=0x7f1e50065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- 172.21.15.157:0/2172213115 shutdown_connections 2023-11-06T03:11:56.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- 172.21.15.157:0/2172213115 wait complete. 2023-11-06T03:11:56.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 Processor -- start 2023-11-06T03:11:56.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- start start 2023-11-06T03:11:56.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e501551e0 0x7f1e5015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e5015e5e0 0x7f1e50154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5015a7d0 con 0x7f1e500b3b20 2023-11-06T03:11:56.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e50154870 con 0x7f1e501551e0 2023-11-06T03:11:56.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.198+0000 7f1e549e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e50153ea0 con 0x7f1e5015e5e0 2023-11-06T03:11:56.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:49990/0 (socket says 172.21.15.157:49990) 2023-11-06T03:11:56.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 -- 172.21.15.157:0/1612182396 learned_addr learned my addr 172.21.15.157:0/1612182396 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:56.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4ed9d640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e5015e5e0 0x7f1e50154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4dd9b640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e501551e0 0x7f1e5015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 -- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e5015e5e0 msgr2=0x7f1e50154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e5015e5e0 0x7f1e50154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 -- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e501551e0 msgr2=0x7f1e5015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e501551e0 0x7f1e5015a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 -- 172.21.15.157:0/1612182396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40009420 con 0x7f1e500b3b20 2023-11-06T03:11:56.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4e59c640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ae040 tx=0x7f1e4001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b7450 con 0x7f1e500b3b20 2023-11-06T03:11:56.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400b75b0 con 0x7f1e500b3b20 2023-11-06T03:11:56.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b7960 con 0x7f1e500b3b20 2023-11-06T03:11:56.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e50153290 con 0x7f1e500b3b20 2023-11-06T03:11:56.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e50153850 con 0x7f1e500b3b20 2023-11-06T03:11:56.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e50072750 con 0x7f1e500b3b20 2023-11-06T03:11:56.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1e400b7ac0 con 0x7f1e500b3b20 2023-11-06T03:11:56.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 0x7f1e30067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1e4011d750 con 0x7f1e500b3b20 2023-11-06T03:11:56.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4dd9b640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 0x7f1e30067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.202+0000 7f1e4dd9b640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 0x7f1e30067f50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1e50152700 tx=0x7f1e44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.206+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1e400eb390 con 0x7f1e500b3b20 2023-11-06T03:11:56.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e50071730 con 0x7f1e500b3b20 2023-11-06T03:11:56.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e3effd640 1 -- 172.21.15.157:0/1612182396 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v95) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e400ee9b0 con 0x7f1e500b3b20 2023-11-06T03:11:56.353 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:11:56.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 msgr2=0x7f1e30067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 0x7f1e30067f50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1e50152700 tx=0x7f1e44009040 comp rx=0 tx=0).stop 2023-11-06T03:11:56.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 msgr2=0x7f1e5014b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ae040 tx=0x7f1e4001ac00 comp rx=0 tx=0).stop 2023-11-06T03:11:56.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 shutdown_connections 2023-11-06T03:11:56.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1e30065a90 0x7f1e30067f50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e5015e5e0 0x7f1e50154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e501551e0 0x7f1e5015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 --2- 172.21.15.157:0/1612182396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e500b3b20 0x7f1e5014b0c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 >> 172.21.15.157:0/1612182396 conn(0x7f1e50065930 msgr2=0x7f1e500afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 shutdown_connections 2023-11-06T03:11:56.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.354+0000 7f1e549e7640 1 -- 172.21.15.157:0/1612182396 wait complete. 2023-11-06T03:11:56.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:11:56.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.478+0000 7f36f3159640 1 Processor -- start 2023-11-06T03:11:56.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.478+0000 7f36f3159640 1 -- start start 2023-11-06T03:11:56.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15caa0 0x7f36ec0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f36ec15ce80 0x7f36ec0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec0d2920 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec0d18a0 con 0x7f36ec15ce80 2023-11-06T03:11:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f3159640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0d1b60 con 0x7f36ec15caa0 2023-11-06T03:11:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50006/0 (socket says 172.21.15.157:50006) 2023-11-06T03:11:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 -- 172.21.15.157:0/3991621395 learned_addr learned my addr 172.21.15.157:0/3991621395 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36ebfff640 1 --2- 172.21.15.157:0/3991621395 >> v2:172.21.15.195:3300/0 conn(0x7f36ec15ce80 0x7f36ec0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f0ef5640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15caa0 0x7f36ec0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 -- 172.21.15.157:0/3991621395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15caa0 msgr2=0x7f36ec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15caa0 0x7f36ec0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 -- 172.21.15.157:0/3991621395 >> v2:172.21.15.195:3300/0 conn(0x7f36ec15ce80 msgr2=0x7f36ec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 --2- 172.21.15.157:0/3991621395 >> v2:172.21.15.195:3300/0 conn(0x7f36ec15ce80 0x7f36ec0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 -- 172.21.15.157:0/3991621395 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0d1e40 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36f16f6640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0d3cb0 tx=0x7f36e401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179e29f3c3305669 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36eb7fe640 1 -- 172.21.15.157:0/3991621395 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40a5820 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36eb7fe640 1 -- 172.21.15.157:0/3991621395 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e40a5980 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.482+0000 7f36eb7fe640 1 -- 172.21.15.157:0/3991621395 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40a5d10 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/3991621395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 msgr2=0x7f36ec0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0d3cb0 tx=0x7f36e401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/3991621395 shutdown_connections 2023-11-06T03:11:56.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec0d00b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- 172.21.15.157:0/3991621395 >> v2:172.21.15.195:3300/0 conn(0x7f36ec15ce80 0x7f36ec0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- 172.21.15.157:0/3991621395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15caa0 0x7f36ec0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/3991621395 >> 172.21.15.157:0/3991621395 conn(0x7f36ec070790 msgr2=0x7f36ec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/3991621395 shutdown_connections 2023-11-06T03:11:56.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/3991621395 wait complete. 2023-11-06T03:11:56.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 Processor -- start 2023-11-06T03:11:56.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- start start 2023-11-06T03:11:56.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36ec15caa0 0x7f36ec15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15ce80 0x7f36ec0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec15c880 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec15b8f0 con 0x7f36ec15caa0 2023-11-06T03:11:56.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0cc7f0 con 0x7f36ec15ce80 2023-11-06T03:11:56.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50012/0 (socket says 172.21.15.157:50012) 2023-11-06T03:11:56.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 -- 172.21.15.157:0/2538968049 learned_addr learned my addr 172.21.15.157:0/2538968049 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36ebfff640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36ec15caa0 0x7f36ec15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f16f6640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15ce80 0x7f36ec0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 -- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15ce80 msgr2=0x7f36ec0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15ce80 0x7f36ec0ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 -- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36ec15caa0 msgr2=0x7f36ec15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36ec15caa0 0x7f36ec15c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 -- 172.21.15.157:0/2538968049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4009b30 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f0ef5640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d800 tx=0x7f36e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e00096a0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0009800 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0009b70 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec0ccad0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec15be20 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.486+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec072750 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.490+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f36e0009cd0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.490+0000 7f36e97fa640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 0x7f36d8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.490+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f36e00836e0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.490+0000 7f36ebfff640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 0x7f36d8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.490+0000 7f36ebfff640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 0x7f36d8067f50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0c8c20 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.494+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f36e0051290 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36ec0b51f0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36e97fa640 1 -- 172.21.15.157:0/2538968049 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) v1 ==== 84+0+126 (secure 0 0 0) 0x7f36e00548b0 con 0x7f36ec0ce1a0 2023-11-06T03:11:56.638 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:56.639 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:11:56.639 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:56.639 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:11:56.639 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:11:56.640 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:11:56.640 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:11:56.640 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:11:56.640 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:11:56.640 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:11:56.641 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:11:56.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 msgr2=0x7f36d8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 0x7f36d8067f50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0c8c20 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2023-11-06T03:11:56.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 msgr2=0x7f36ec15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d800 tx=0x7f36e000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 shutdown_connections 2023-11-06T03:11:56.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f36d8065a90 0x7f36d8067f50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36ec15ce80 0x7f36ec0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36ec15caa0 0x7f36ec15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 --2- 172.21.15.157:0/2538968049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36ec0ce1a0 0x7f36ec15d310 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 >> 172.21.15.157:0/2538968049 conn(0x7f36ec070790 msgr2=0x7f36ec0b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 shutdown_connections 2023-11-06T03:11:56.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.638+0000 7f36f3159640 1 -- 172.21.15.157:0/2538968049 wait complete. 2023-11-06T03:11:56.656 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:11:56.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:11:56.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 Processor -- start 2023-11-06T03:11:56.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 -- start start 2023-11-06T03:11:56.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f02140b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f02140cfa60 0x7f02140cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02140b6f70 con 0x7f02140b5ad0 2023-11-06T03:11:56.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02140d0430 con 0x7f02140cfa60 2023-11-06T03:11:56.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021c521640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02140b6360 con 0x7f02141600e0 2023-11-06T03:11:56.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f02140b5ad0 0x7f02140d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51432/0 (socket says 172.21.15.157:51432) 2023-11-06T03:11:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 -- 172.21.15.157:0/4145129314 learned_addr learned my addr 172.21.15.157:0/4145129314 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021a2bd640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f02140b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021aabe640 1 --2- 172.21.15.157:0/4145129314 >> v2:172.21.15.195:3300/0 conn(0x7f02140cfa60 0x7f02140cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257329119 0 0) 0x7f02140b6f70 con 0x7f02140b5ad0 2023-11-06T03:11:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0200003530 con 0x7f02140b5ad0 2023-11-06T03:11:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1066209665 0 0) 0x7f0200003530 con 0x7f02140b5ad0 2023-11-06T03:11:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 msgr2=0x7f02140b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f02140b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 >> v2:172.21.15.195:3300/0 conn(0x7f02140cfa60 msgr2=0x7f02140cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 --2- 172.21.15.157:0/4145129314 >> v2:172.21.15.195:3300/0 conn(0x7f02140cfa60 0x7f02140cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140b6640 con 0x7f02140b5ad0 2023-11-06T03:11:56.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0204001680 con 0x7f02140b5ad0 2023-11-06T03:11:56.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 >> v1:172.21.15.157:6789/0 conn(0x7f02140b5ad0 legacy=0x7f02140d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f0200003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f02000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004d10 0x7f0200005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02140d0430 con 0x7f0200003f00 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02140b6360 con 0x7f0200004d10 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020000b9c0 con 0x7f02141600e0 2023-11-06T03:11:56.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f02000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021a2bd640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f0200003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021aabe640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004d10 0x7f0200005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 -- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 msgr2=0x7f0200003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02141600e0 0x7f0200003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 -- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004d10 msgr2=0x7f0200005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:56.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004d10 0x7f0200005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f0219abc640 1 -- 172.21.15.157:0/4145129314 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200003530 con 0x7f0200003f00 2023-11-06T03:11:56.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.770+0000 7f021aabe640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0200004d10 0x7f0200005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:56.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f0219abc640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f02000047d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f020400c290 tx=0x7f020401cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66e176eb1d3ad78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040a5950 con 0x7f0200003f00 2023-11-06T03:11:56.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040a5ab0 con 0x7f0200003f00 2023-11-06T03:11:56.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f02192bb640 1 -- 172.21.15.157:0/4145129314 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040a5e60 con 0x7f0200003f00 2023-11-06T03:11:56.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- 172.21.15.157:0/4145129314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 msgr2=0x7f02000047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 --2- 172.21.15.157:0/4145129314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f02000047d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f020400c290 tx=0x7f020401cce0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021aabe640 1 -- 172.21.15.157:0/4145129314 reap_dead start 2023-11-06T03:11:56.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- 172.21.15.157:0/4145129314 shutdown_connections 2023-11-06T03:11:56.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- 172.21.15.157:0/4145129314 >> 172.21.15.157:0/4145129314 conn(0x7f0214070790 msgr2=0x7f0214070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:56.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- 172.21.15.157:0/4145129314 shutdown_connections 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- 172.21.15.157:0/4145129314 wait complete. 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 Processor -- start 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.774+0000 7f021c521640 1 -- start start 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 0x7f02140b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 0x7f021414fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02141f2660 con 0x7f0200003f00 2023-11-06T03:11:56.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214159de0 con 0x7f0214159950 2023-11-06T03:11:56.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021415a0c0 con 0x7f02140ccac0 2023-11-06T03:11:56.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50040/0 (socket says 172.21.15.157:50040) 2023-11-06T03:11:56.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 -- 172.21.15.157:0/3244777415 learned_addr learned my addr 172.21.15.157:0/3244777415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:56.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f0219abc640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 0x7f02140b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021aabe640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 0x7f021414fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 -- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 msgr2=0x7f021414fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 0x7f021414fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 -- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 msgr2=0x7f02140b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:56.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 0x7f02140b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:56.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 -- 172.21.15.157:0/3244777415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021415a3a0 con 0x7f0200003f00 2023-11-06T03:11:56.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f0219abc640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 0x7f02140b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021aabe640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 0x7f021414fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021a2bd640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f02100115e0 tx=0x7f0210011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000f8f0 con 0x7f0200003f00 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021000fa50 con 0x7f0200003f00 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021415a680 con 0x7f0200003f00 2023-11-06T03:11:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000fdc0 con 0x7f0200003f00 2023-11-06T03:11:56.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02140b37e0 con 0x7f0200003f00 2023-11-06T03:11:56.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021415a8c0 con 0x7f0200003f00 2023-11-06T03:11:56.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0210015460 con 0x7f0200003f00 2023-11-06T03:11:56.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 0x7f01f8067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:56.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f02100837e0 con 0x7f0200003f00 2023-11-06T03:11:56.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.778+0000 7f0219abc640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 0x7f01f8067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:56.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.782+0000 7f0219abc640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 0x7f01f8067ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f02040aa000 tx=0x7f0204001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:56.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.782+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f021415a8c0 con 0x7f0200003f00 2023-11-06T03:11:56.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:56.934+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f021415a8c0 con 0x7f0200003f00 2023-11-06T03:11:57.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.046+0000 7f020affd640 1 -- 172.21.15.157:0/3244777415 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) v1 ==== 138+0+0 (secure 0 0 0) 0x7f021415a8c0 con 0x7f0200003f00 2023-11-06T03:11:57.045 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:11:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 msgr2=0x7f01f8067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 0x7f01f8067ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f02040aa000 tx=0x7f0204001a10 comp rx=0 tx=0).stop 2023-11-06T03:11:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 msgr2=0x7f021415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f02100115e0 tx=0x7f0210011aa0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 shutdown_connections 2023-11-06T03:11:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f01f8065a30 0x7f01f8067ef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f02140ccac0 0x7f021414fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0214159950 0x7f02140b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 --2- 172.21.15.157:0/3244777415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0200003f00 0x7f021415af50 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 >> 172.21.15.157:0/3244777415 conn(0x7f0214070790 msgr2=0x7f02140b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 shutdown_connections 2023-11-06T03:11:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.050+0000 7f021c521640 1 -- 172.21.15.157:0/3244777415 wait complete. 2023-11-06T03:11:57.070 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:11:57.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 Processor -- start 2023-11-06T03:11:57.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 -- start start 2023-11-06T03:11:57.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc1551e0 0x7f4ddc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ddc0b3b20 0x7f4ddc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc15f580 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc15efc0 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de29e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc15f2a0 con 0x7f4ddc1551e0 2023-11-06T03:11:57.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de0f7f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ddc0b3b20 0x7f4ddc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4ddb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4ddc15e5e0 0x7f4ddc0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51444/0 (socket says 172.21.15.157:51444) 2023-11-06T03:11:57.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4ddb7fe640 1 -- 172.21.15.157:0/2062270634 learned_addr learned my addr 172.21.15.157:0/2062270634 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:57.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4ddbfff640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc1551e0 0x7f4ddc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de0f7f640 1 -- 172.21.15.157:0/2062270634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc1551e0 msgr2=0x7f4ddc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:57.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de0f7f640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc1551e0 0x7f4ddc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de0f7f640 1 -- 172.21.15.157:0/2062270634 >> v1:172.21.15.157:6789/0 conn(0x7f4ddc15e5e0 legacy=0x7f4ddc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4ddaffd640 1 -- 172.21.15.157:0/2062270634 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698374630 0 0) 0x7f4ddc15f580 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.194+0000 7f4de0f7f640 1 -- 172.21.15.157:0/2062270634 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc06f5a0 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc0b3b20 0x7f4ddc15c1b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40097c0 tx=0x7f4dd401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0733301549bb98 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:57.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddaffd640 1 -- 172.21.15.157:0/2062270634 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40ab600 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddaffd640 1 -- 172.21.15.157:0/2062270634 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd40a7050 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddaffd640 1 -- 172.21.15.157:0/2062270634 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40ab980 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/2062270634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc0b3b20 msgr2=0x7f4ddc15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc0b3b20 0x7f4ddc15c1b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40097c0 tx=0x7f4dd401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/2062270634 shutdown_connections 2023-11-06T03:11:57.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc0b3b20 0x7f4ddc15c1b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- 172.21.15.157:0/2062270634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc1551e0 0x7f4ddc06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/2062270634 >> 172.21.15.157:0/2062270634 conn(0x7f4ddc065930 msgr2=0x7f4ddc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/2062270634 shutdown_connections 2023-11-06T03:11:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/2062270634 wait complete. 2023-11-06T03:11:57.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 Processor -- start 2023-11-06T03:11:57.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- start start 2023-11-06T03:11:57.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 0x7f4ddc14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ddc1551e0 0x7f4ddc1372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc14c6e0 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc137800 con 0x7f4ddc1551e0 2023-11-06T03:11:57.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc149c50 con 0x7f4ddc0b3b20 2023-11-06T03:11:57.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50070/0 (socket says 172.21.15.157:50070) 2023-11-06T03:11:57.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 -- 172.21.15.157:0/1219101110 learned_addr learned my addr 172.21.15.157:0/1219101110 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:57.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddb7fe640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ddc1551e0 0x7f4ddc1372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddbfff640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 0x7f4ddc14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 -- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 msgr2=0x7f4ddc14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 0x7f4ddc14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 -- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ddc1551e0 msgr2=0x7f4ddc1372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:57.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ddc1551e0 0x7f4ddc1372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:57.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 -- 172.21.15.157:0/1219101110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4009420 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4ddbfff640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 0x7f4ddc14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:11:57.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de0f7f640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4dd401aa70 tx=0x7f4dd4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:57.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4004230 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd4004390 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc14b100 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.198+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4003820 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc14b6c0 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc072750 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4dd40b2040 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4dd8ff9640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 0x7f4dbc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:57.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4dd411c750 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4ddbfff640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 0x7f4dbc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:57.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.202+0000 7f4ddbfff640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 0x7f4dbc067f20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc14c880 tx=0x7f4dcc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:57.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.206+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4dd40ea190 con 0x7f4ddc15e5e0 2023-11-06T03:11:57.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:57.350+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4ddc0b27e0 con 0x7f4ddc15e5e0 2023-11-06T03:11:58.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.058+0000 7f4dd8ff9640 1 -- 172.21.15.157:0/1219101110 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4dd40b00d0 con 0x7f4ddc15e5e0 2023-11-06T03:11:58.076 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:11:58.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 msgr2=0x7f4dbc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 0x7f4dbc067f20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc14c880 tx=0x7f4dcc00b040 comp rx=0 tx=0).stop 2023-11-06T03:11:58.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 msgr2=0x7f4ddc14a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4dd401aa70 tx=0x7f4dd4019160 comp rx=0 tx=0).stop 2023-11-06T03:11:58.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 shutdown_connections 2023-11-06T03:11:58.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4dbc065a60 0x7f4dbc067f20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ddc15e5e0 0x7f4ddc14a570 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ddc1551e0 0x7f4ddc1372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 --2- 172.21.15.157:0/1219101110 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ddc0b3b20 0x7f4ddc14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 >> 172.21.15.157:0/1219101110 conn(0x7f4ddc065930 msgr2=0x7f4ddc15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 shutdown_connections 2023-11-06T03:11:58.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.082+0000 7f4de29e2640 1 -- 172.21.15.157:0/1219101110 wait complete. 2023-11-06T03:11:58.107 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:11:58.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 Processor -- start 2023-11-06T03:11:58.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 -- start start 2023-11-06T03:11:58.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f418c0b1910 0x7f418c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c06f470 con 0x7f418c0701a0 2023-11-06T03:11:58.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418c0b1df0 con 0x7f418c0b1910 2023-11-06T03:11:58.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f41922e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c06f6c0 con 0x7f418c160db0 2023-11-06T03:11:58.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f418c0b1910 0x7f418c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:42954/0 (socket says 172.21.15.157:42954) 2023-11-06T03:11:58.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 -- 172.21.15.157:0/247830564 learned_addr learned my addr 172.21.15.157:0/247830564 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:58.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 -- 172.21.15.157:0/247830564 >> v2:172.21.15.195:3300/0 conn(0x7f418c0b1910 msgr2=0x7f418c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:58.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 --2- 172.21.15.157:0/247830564 >> v2:172.21.15.195:3300/0 conn(0x7f418c0b1910 0x7f418c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 -- 172.21.15.157:0/247830564 >> v1:172.21.15.195:6789/0 conn(0x7f418c0701a0 legacy=0x7f418c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.226+0000 7f418bfff640 1 -- 172.21.15.157:0/247830564 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c06f9a0 con 0x7f418c160db0 2023-11-06T03:11:58.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f418bfff640 1 --2- 172.21.15.157:0/247830564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4180009870 tx=0x7f418001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99481ac4348cb32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f418affd640 1 -- 172.21.15.157:0/247830564 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41800a6040 con 0x7f418c160db0 2023-11-06T03:11:58.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f418affd640 1 -- 172.21.15.157:0/247830564 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41800aa600 con 0x7f418c160db0 2023-11-06T03:11:58.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f418affd640 1 -- 172.21.15.157:0/247830564 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41800aa990 con 0x7f418c160db0 2023-11-06T03:11:58.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- 172.21.15.157:0/247830564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 msgr2=0x7f418c161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- 172.21.15.157:0/247830564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4180009870 tx=0x7f418001c480 comp rx=0 tx=0).stop 2023-11-06T03:11:58.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- 172.21.15.157:0/247830564 shutdown_connections 2023-11-06T03:11:58.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- 172.21.15.157:0/247830564 >> v2:172.21.15.195:3300/0 conn(0x7f418c0b1910 0x7f418c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- 172.21.15.157:0/247830564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c161190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- 172.21.15.157:0/247830564 >> 172.21.15.157:0/247830564 conn(0x7f418c065930 msgr2=0x7f418c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- 172.21.15.157:0/247830564 shutdown_connections 2023-11-06T03:11:58.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- 172.21.15.157:0/247830564 wait complete. 2023-11-06T03:11:58.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 Processor -- start 2023-11-06T03:11:58.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- start start 2023-11-06T03:11:58.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418c0701a0 0x7f418c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c14efe0 con 0x7f418c0b1910 2023-11-06T03:11:58.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c14e360 con 0x7f418c0701a0 2023-11-06T03:11:58.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f41922e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c14e640 con 0x7f418c160db0 2023-11-06T03:11:58.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.230+0000 7f418b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50086/0 (socket says 172.21.15.157:50086) 2023-11-06T03:11:58.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 -- 172.21.15.157:0/2969905882 learned_addr learned my addr 172.21.15.157:0/2969905882 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418bfff640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418c0701a0 0x7f418c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4190881640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 -- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 msgr2=0x7f418c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 -- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418c0701a0 msgr2=0x7f418c14d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418c0701a0 0x7f418c14d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 -- 172.21.15.157:0/2969905882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800094d0 con 0x7f418c0b1910 2023-11-06T03:11:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418b7fe640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e930 tx=0x7f417c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c009690 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00ca00 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cd70 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c14e920 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c14df10 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c14f120 con 0x7f418c0b1910 2023-11-06T03:11:58.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f417c013470 con 0x7f418c0b1910 2023-11-06T03:11:58.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 0x7f4178067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f417c083420 con 0x7f418c0b1910 2023-11-06T03:11:58.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418bfff640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 0x7f4178067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.234+0000 7f418bfff640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 0x7f4178067f20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f41800ae000 tx=0x7f41800b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.238+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f418c14f120 con 0x7f418c0b1910 2023-11-06T03:11:58.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.410+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f418c14f120 con 0x7f418c0b1910 2023-11-06T03:11:58.425 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 13 and data pool 14 2023-11-06T03:11:58.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.426+0000 7f4188ff9640 1 -- 172.21.15.157:0/2969905882 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f418c14f120 con 0x7f418c0b1910 2023-11-06T03:11:58.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 msgr2=0x7f4178067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 0x7f4178067f20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f41800ae000 tx=0x7f41800b5040 comp rx=0 tx=0).stop 2023-11-06T03:11:58.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 msgr2=0x7f418c1532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e930 tx=0x7f417c00edf0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 shutdown_connections 2023-11-06T03:11:58.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4178065a60 0x7f4178067f20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f418c160db0 0x7f418c15e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f418c0b1910 0x7f418c1532e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 --2- 172.21.15.157:0/2969905882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f418c0701a0 0x7f418c14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 >> 172.21.15.157:0/2969905882 conn(0x7f418c065930 msgr2=0x7f418c163870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 shutdown_connections 2023-11-06T03:11:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.430+0000 7f41922e4640 1 -- 172.21.15.157:0/2969905882 wait complete. 2023-11-06T03:11:58.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:58.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 Processor -- start 2023-11-06T03:11:58.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 -- start start 2023-11-06T03:11:58.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f149815caa0 0x7f14980cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f149815ce80 0x7f14980cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14980ce1a0 0x7f14980d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980d2920 con 0x7f149815ce80 2023-11-06T03:11:58.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980d18a0 con 0x7f149815caa0 2023-11-06T03:11:58.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149fb20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14980d1b60 con 0x7f14980ce1a0 2023-11-06T03:11:58.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f149815ce80 0x7f14980cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f149815ce80 0x7f14980cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44124/0 (socket says 172.21.15.157:44124) 2023-11-06T03:11:58.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 -- 172.21.15.157:0/2575374386 learned_addr learned my addr 172.21.15.157:0/2575374386 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:58.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 -- 172.21.15.157:0/2575374386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14980ce1a0 msgr2=0x7f14980d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:58.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 --2- 172.21.15.157:0/2575374386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14980ce1a0 0x7f14980d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 -- 172.21.15.157:0/2575374386 >> v2:172.21.15.195:3300/0 conn(0x7f149815caa0 msgr2=0x7f14980cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:58.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 --2- 172.21.15.157:0/2575374386 >> v2:172.21.15.195:3300/0 conn(0x7f149815caa0 0x7f14980cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 -- 172.21.15.157:0/2575374386 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980d1e40 con 0x7f149815ce80 2023-11-06T03:11:58.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.570+0000 7f149d0bb640 1 --2- 172.21.15.157:0/2575374386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149815ce80 0x7f14980cfb70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f14980b4f80 tx=0x7f148801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d687f1f995dffe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149c8ba640 1 -- 172.21.15.157:0/2575374386 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880aa5f0 con 0x7f149815ce80 2023-11-06T03:11:58.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149c8ba640 1 -- 172.21.15.157:0/2575374386 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880aa750 con 0x7f149815ce80 2023-11-06T03:11:58.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149c8ba640 1 -- 172.21.15.157:0/2575374386 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880aaae0 con 0x7f149815ce80 2023-11-06T03:11:58.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/2575374386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149815ce80 msgr2=0x7f14980cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- 172.21.15.157:0/2575374386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149815ce80 0x7f14980cfb70 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f14980b4f80 tx=0x7f148801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/2575374386 shutdown_connections 2023-11-06T03:11:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- 172.21.15.157:0/2575374386 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14980ce1a0 0x7f14980d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- 172.21.15.157:0/2575374386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149815ce80 0x7f14980cfb70 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- 172.21.15.157:0/2575374386 >> v2:172.21.15.195:3300/0 conn(0x7f149815caa0 0x7f14980cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/2575374386 >> 172.21.15.157:0/2575374386 conn(0x7f1498070790 msgr2=0x7f1498070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/2575374386 shutdown_connections 2023-11-06T03:11:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/2575374386 wait complete. 2023-11-06T03:11:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 Processor -- start 2023-11-06T03:11:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- start start 2023-11-06T03:11:58.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815caa0 0x7f1498155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f149815ce80 0x7f14980c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b6ee0 con 0x7f14980ce1a0 2023-11-06T03:11:58.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b6260 con 0x7f149815caa0 2023-11-06T03:11:58.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b6540 con 0x7f149815ce80 2023-11-06T03:11:58.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44132/0 (socket says 172.21.15.157:44132) 2023-11-06T03:11:58.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 -- 172.21.15.157:0/799478186 learned_addr learned my addr 172.21.15.157:0/799478186 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:58.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 -- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f149815ce80 msgr2=0x7f14980c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f149815ce80 0x7f14980c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 -- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815caa0 msgr2=0x7f1498155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815caa0 0x7f1498155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 -- 172.21.15.157:0/799478186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880094d0 con 0x7f14980ce1a0 2023-11-06T03:11:58.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149d8bc640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f149400b550 tx=0x7f149400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494010870 con 0x7f14980ce1a0 2023-11-06T03:11:58.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14940109f0 con 0x7f14980ce1a0 2023-11-06T03:11:58.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14980b6820 con 0x7f14980ce1a0 2023-11-06T03:11:58.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494010da0 con 0x7f14980ce1a0 2023-11-06T03:11:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.574+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498155d80 con 0x7f14980ce1a0 2023-11-06T03:11:58.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1498072750 con 0x7f14980ce1a0 2023-11-06T03:11:58.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f149400f3c0 con 0x7f14980ce1a0 2023-11-06T03:11:58.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f148e7fc640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 0x7f147c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1494083270 con 0x7f14980ce1a0 2023-11-06T03:11:58.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f149d0bb640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 0x7f147c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.578+0000 7f149d0bb640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 0x7f147c067f20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14880ae000 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.582+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1494050b60 con 0x7f14980ce1a0 2023-11-06T03:11:58.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.734+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14980b1eb0 con 0x7f14980ce1a0 2023-11-06T03:11:58.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.738+0000 7f148e7fc640 1 -- 172.21.15.157:0/799478186 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17887 (secure 0 0 0) 0x7f1494054180 con 0x7f14980ce1a0 2023-11-06T03:11:58.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:58.739 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":98,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:58.417520+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:58.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 msgr2=0x7f147c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 0x7f147c067f20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14880ae000 tx=0x7f1488008040 comp rx=0 tx=0).stop 2023-11-06T03:11:58.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 msgr2=0x7f14980b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f149400b550 tx=0x7f149400ba10 comp rx=0 tx=0).stop 2023-11-06T03:11:58.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 shutdown_connections 2023-11-06T03:11:58.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f147c065a60 0x7f147c067f20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f149815ce80 0x7f14980c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f149815caa0 0x7f1498155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 --2- 172.21.15.157:0/799478186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14980ce1a0 0x7f14980b7300 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 >> 172.21.15.157:0/799478186 conn(0x7f1498070790 msgr2=0x7f14980b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 shutdown_connections 2023-11-06T03:11:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.742+0000 7f149fb20640 1 -- 172.21.15.157:0/799478186 wait complete. 2023-11-06T03:11:58.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:11:58.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.878+0000 7f01af4b2640 1 Processor -- start 2023-11-06T03:11:58.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.878+0000 7f01af4b2640 1 -- start start 2023-11-06T03:11:58.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01af4b2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f01a80adf70 0x7f01a815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01af4b2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01a80cad20 0x7f01a80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01af4b2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a815e100 con 0x7f01a80ccb70 2023-11-06T03:11:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01af4b2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80cb690 con 0x7f01a80cad20 2023-11-06T03:11:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01af4b2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a815cff0 con 0x7f01a80adf70 2023-11-06T03:11:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01a80cad20 0x7f01a80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01aca4d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f01a80ccb70 0x7f01a8161020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44214/0 (socket says 172.21.15.157:44214) 2023-11-06T03:11:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01aca4d640 1 -- 172.21.15.157:0/2166332499 learned_addr learned my addr 172.21.15.157:0/2166332499 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:11:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ad24e640 1 --2- 172.21.15.157:0/2166332499 >> v2:172.21.15.195:3300/0 conn(0x7f01a80adf70 0x7f01a815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 -- 172.21.15.157:0/2166332499 >> v2:172.21.15.195:3300/0 conn(0x7f01a80adf70 msgr2=0x7f01a815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:58.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 --2- 172.21.15.157:0/2166332499 >> v2:172.21.15.195:3300/0 conn(0x7f01a80adf70 0x7f01a815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 -- 172.21.15.157:0/2166332499 >> v1:172.21.15.157:6789/0 conn(0x7f01a80ccb70 legacy=0x7f01a8161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 -- 172.21.15.157:0/2166332499 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80cd130 con 0x7f01a80cad20 2023-11-06T03:11:58.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01a7fff640 1 -- 172.21.15.157:0/2166332499 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804659132 0 0) 0x7f01a815e100 con 0x7f01a80ccb70 2023-11-06T03:11:58.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01ada4f640 1 --2- 172.21.15.157:0/2166332499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80cb150 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f01a8066650 tx=0x7f019801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62466a1870a69ffc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01a7fff640 1 -- 172.21.15.157:0/2166332499 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980ab600 con 0x7f01a80cad20 2023-11-06T03:11:58.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01a7fff640 1 -- 172.21.15.157:0/2166332499 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01980a7050 con 0x7f01a80cad20 2023-11-06T03:11:58.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.882+0000 7f01a7fff640 1 -- 172.21.15.157:0/2166332499 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980ab980 con 0x7f01a80cad20 2023-11-06T03:11:58.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/2166332499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 msgr2=0x7f01a80cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- 172.21.15.157:0/2166332499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80cb150 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f01a8066650 tx=0x7f019801c3d0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/2166332499 shutdown_connections 2023-11-06T03:11:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- 172.21.15.157:0/2166332499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80cb150 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- 172.21.15.157:0/2166332499 >> v2:172.21.15.195:3300/0 conn(0x7f01a80adf70 0x7f01a815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/2166332499 >> 172.21.15.157:0/2166332499 conn(0x7f01a80b0b50 msgr2=0x7f01a80b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:58.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/2166332499 shutdown_connections 2023-11-06T03:11:58.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/2166332499 wait complete. 2023-11-06T03:11:58.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 Processor -- start 2023-11-06T03:11:58.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- start start 2023-11-06T03:11:58.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a80adf70 0x7f01a81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80ccb70 0x7f01a80ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81537e0 con 0x7f01a80cad20 2023-11-06T03:11:58.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8154190 con 0x7f01a80ccb70 2023-11-06T03:11:58.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80c34a0 con 0x7f01a80adf70 2023-11-06T03:11:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44150/0 (socket says 172.21.15.157:44150) 2023-11-06T03:11:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 -- 172.21.15.157:0/762001436 learned_addr learned my addr 172.21.15.157:0/762001436 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01ad24e640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a80adf70 0x7f01a81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01ada4f640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80ccb70 0x7f01a80ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 -- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a80adf70 msgr2=0x7f01a81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a80adf70 0x7f01a81527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 -- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80ccb70 msgr2=0x7f01a80ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:58.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80ccb70 0x7f01a80ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:58.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 -- 172.21.15.157:0/762001436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009420 con 0x7f01a80cad20 2023-11-06T03:11:58.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01aca4d640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f019c005890 tx=0x7f019c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c005cf0 con 0x7f01a80cad20 2023-11-06T03:11:58.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c005e70 con 0x7f01a80cad20 2023-11-06T03:11:58.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a81539d0 con 0x7f01a80cad20 2023-11-06T03:11:58.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c0072f0 con 0x7f01a80cad20 2023-11-06T03:11:58.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.886+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80c3740 con 0x7f01a80cad20 2023-11-06T03:11:58.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8072750 con 0x7f01a80cad20 2023-11-06T03:11:58.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f019c007450 con 0x7f01a80cad20 2023-11-06T03:11:58.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01a5ffb640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 0x7f0194067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:58.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f019c082910 con 0x7f01a80cad20 2023-11-06T03:11:58.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01ad24e640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 0x7f0194067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:58.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.890+0000 7f01ad24e640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 0x7f0194067f50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009f30 tx=0x7f01a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:58.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:58.894+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f019c0500d0 con 0x7f01a80cad20 2023-11-06T03:11:59.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.050+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01a8072fa0 con 0x7f01a80cad20 2023-11-06T03:11:59.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.050+0000 7f01a5ffb640 1 -- 172.21.15.157:0/762001436 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17887 (secure 0 0 0) 0x7f019c017020 con 0x7f01a80cad20 2023-11-06T03:11:59.051 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:11:59.052 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":98,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:58.417520+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:11:59.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 msgr2=0x7f0194067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 0x7f0194067f50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009f30 tx=0x7f01a000b040 comp rx=0 tx=0).stop 2023-11-06T03:11:59.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 msgr2=0x7f01a80c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f019c005890 tx=0x7f019c00c8e0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 shutdown_connections 2023-11-06T03:11:59.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0194065a90 0x7f0194067f50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a80ccb70 0x7f01a80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a80cad20 0x7f01a80c51a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 --2- 172.21.15.157:0/762001436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01a80adf70 0x7f01a81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 >> 172.21.15.157:0/762001436 conn(0x7f01a80b0b50 msgr2=0x7f01a80af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:59.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 shutdown_connections 2023-11-06T03:11:59.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.054+0000 7f01af4b2640 1 -- 172.21.15.157:0/762001436 wait complete. 2023-11-06T03:11:59.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:11:59.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 Processor -- start 2023-11-06T03:11:59.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 -- start start 2023-11-06T03:11:59.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5b300b2af0 0x7f5b300ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b3006d000 0x7f5b30159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3015e440 con 0x7f5b301599c0 2023-11-06T03:11:59.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3006c240 con 0x7f5b3006d000 2023-11-06T03:11:59.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b35cc3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3006c520 con 0x7f5b300b2af0 2023-11-06T03:11:59.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b3006d000 0x7f5b30159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b3006d000 0x7f5b30159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44158/0 (socket says 172.21.15.157:44158) 2023-11-06T03:11:59.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 -- 172.21.15.157:0/3815797969 learned_addr learned my addr 172.21.15.157:0/3815797969 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:59.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2effd640 1 --2- 172.21.15.157:0/3815797969 >> v2:172.21.15.195:3300/0 conn(0x7f5b300b2af0 0x7f5b300ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 -- 172.21.15.157:0/3815797969 >> v2:172.21.15.195:3300/0 conn(0x7f5b300b2af0 msgr2=0x7f5b300ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:59.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 --2- 172.21.15.157:0/3815797969 >> v2:172.21.15.195:3300/0 conn(0x7f5b300b2af0 0x7f5b300ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 -- 172.21.15.157:0/3815797969 >> v1:172.21.15.195:6789/0 conn(0x7f5b301599c0 legacy=0x7f5b300b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:59.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.194+0000 7f5b2ffff640 1 -- 172.21.15.157:0/3815797969 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3006c800 con 0x7f5b3006d000 2023-11-06T03:11:59.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2ffff640 1 --2- 172.21.15.157:0/3815797969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3006d000 0x7f5b30159da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5b300cfa20 tx=0x7f5b2801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6e5e98464fc273 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2e7fc640 1 -- 172.21.15.157:0/3815797969 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a5840 con 0x7f5b3006d000 2023-11-06T03:11:59.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2e7fc640 1 -- 172.21.15.157:0/3815797969 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b280a59a0 con 0x7f5b3006d000 2023-11-06T03:11:59.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2e7fc640 1 -- 172.21.15.157:0/3815797969 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a5d30 con 0x7f5b3006d000 2023-11-06T03:11:59.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3815797969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3006d000 msgr2=0x7f5b30159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3815797969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3006d000 0x7f5b30159da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5b300cfa20 tx=0x7f5b2801c960 comp rx=0 tx=0).stop 2023-11-06T03:11:59.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3815797969 shutdown_connections 2023-11-06T03:11:59.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3815797969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b3006d000 0x7f5b30159da0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3815797969 >> v2:172.21.15.195:3300/0 conn(0x7f5b300b2af0 0x7f5b300ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3815797969 >> 172.21.15.157:0/3815797969 conn(0x7f5b30065930 msgr2=0x7f5b30065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:59.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3815797969 shutdown_connections 2023-11-06T03:11:59.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3815797969 wait complete. 2023-11-06T03:11:59.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 Processor -- start 2023-11-06T03:11:59.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- start start 2023-11-06T03:11:59.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b3006d000 0x7f5b3014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b301599c0 0x7f5b30157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300cc2d0 con 0x7f5b300b2af0 2023-11-06T03:11:59.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300cc5b0 con 0x7f5b3006d000 2023-11-06T03:11:59.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b35cc3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b30157060 con 0x7f5b301599c0 2023-11-06T03:11:59.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44170/0 (socket says 172.21.15.157:44170) 2023-11-06T03:11:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 -- 172.21.15.157:0/3238751419 learned_addr learned my addr 172.21.15.157:0/3238751419 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2f7fe640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b3006d000 0x7f5b3014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2ffff640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b301599c0 0x7f5b30157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 -- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b301599c0 msgr2=0x7f5b30157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b301599c0 0x7f5b30157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 -- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b3006d000 msgr2=0x7f5b3014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b3006d000 0x7f5b3014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.198+0000 7f5b2effd640 1 -- 172.21.15.157:0/3238751419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009b30 con 0x7f5b300b2af0 2023-11-06T03:11:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b2effd640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d800 tx=0x7f5b2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000b6c0 con 0x7f5b300b2af0 2023-11-06T03:11:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2000b820 con 0x7f5b300b2af0 2023-11-06T03:11:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b30157340 con 0x7f5b300b2af0 2023-11-06T03:11:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000bb90 con 0x7f5b300b2af0 2023-11-06T03:11:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3014eb90 con 0x7f5b300b2af0 2023-11-06T03:11:59.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b30072750 con 0x7f5b300b2af0 2023-11-06T03:11:59.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5b2000bcf0 con 0x7f5b300b2af0 2023-11-06T03:11:59.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 0x7f5b1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5b20083850 con 0x7f5b300b2af0 2023-11-06T03:11:59.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b2f7fe640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 0x7f5b1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.202+0000 7f5b2f7fe640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 0x7f5b1c067f20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ad450 tx=0x7f5b24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.206+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5b200510b0 con 0x7f5b300b2af0 2023-11-06T03:11:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.394+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5b300cb110 con 0x7f5b300b2af0 2023-11-06T03:11:59.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b17fff640 1 -- 172.21.15.157:0/3238751419 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5b200546d0 con 0x7f5b300b2af0 2023-11-06T03:11:59.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 msgr2=0x7f5b1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 0x7f5b1c067f20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ad450 tx=0x7f5b24008040 comp rx=0 tx=0).stop 2023-11-06T03:11:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 msgr2=0x7f5b300cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d800 tx=0x7f5b2000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 shutdown_connections 2023-11-06T03:11:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5b1c065a60 0x7f5b1c067f20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5b301599c0 0x7f5b30157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b300b2af0 0x7f5b300cb9d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 --2- 172.21.15.157:0/3238751419 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5b3006d000 0x7f5b3014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 >> 172.21.15.157:0/3238751419 conn(0x7f5b30065930 msgr2=0x7f5b3015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 shutdown_connections 2023-11-06T03:11:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.422+0000 7f5b35cc3640 1 -- 172.21.15.157:0/3238751419 wait complete. 2023-11-06T03:11:59.442 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:11:59.443 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:11:59.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 Processor -- start 2023-11-06T03:11:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 -- start start 2023-11-06T03:11:59.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99c015e090 0x7f99c00cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c00719e0 0x7f99c00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99c00cce90 0x7f99c00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00cc420 con 0x7f99c015e090 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00d0380 con 0x7f99c00cce90 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99c59ff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cc690 con 0x7f99c00719e0 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99c015e090 0x7f99c00cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99c015e090 0x7f99c00cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44180/0 (socket says 172.21.15.157:44180) 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 -- 172.21.15.157:0/4035940296 learned_addr learned my addr 172.21.15.157:0/4035940296 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:59.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 -- 172.21.15.157:0/4035940296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c00719e0 msgr2=0x7f99c00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:59.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 --2- 172.21.15.157:0/4035940296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c00719e0 0x7f99c00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 -- 172.21.15.157:0/4035940296 >> v2:172.21.15.195:3300/0 conn(0x7f99c00cce90 msgr2=0x7f99c00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:11:59.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99bf7fe640 1 --2- 172.21.15.157:0/4035940296 >> v2:172.21.15.195:3300/0 conn(0x7f99c00cce90 0x7f99c00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 --2- 172.21.15.157:0/4035940296 >> v2:172.21.15.195:3300/0 conn(0x7f99c00cce90 0x7f99c00cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.554+0000 7f99beffd640 1 -- 172.21.15.157:0/4035940296 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00cc970 con 0x7f99c015e090 2023-11-06T03:11:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99beffd640 1 --2- 172.21.15.157:0/4035940296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c015e090 0x7f99c00cfa30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c7970 tx=0x7f99b001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8d54d20942bb16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99bdffb640 1 -- 172.21.15.157:0/4035940296 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a89b0 con 0x7f99c015e090 2023-11-06T03:11:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99bdffb640 1 -- 172.21.15.157:0/4035940296 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b00a8b10 con 0x7f99c015e090 2023-11-06T03:11:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99bdffb640 1 -- 172.21.15.157:0/4035940296 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00a8ea0 con 0x7f99c015e090 2023-11-06T03:11:59.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- 172.21.15.157:0/4035940296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c015e090 msgr2=0x7f99c00cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 --2- 172.21.15.157:0/4035940296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c015e090 0x7f99c00cfa30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c7970 tx=0x7f99b001ecf0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- 172.21.15.157:0/4035940296 shutdown_connections 2023-11-06T03:11:59.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 --2- 172.21.15.157:0/4035940296 >> v2:172.21.15.195:3300/0 conn(0x7f99c00cce90 0x7f99c00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 --2- 172.21.15.157:0/4035940296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c00719e0 0x7f99c00cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 --2- 172.21.15.157:0/4035940296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c015e090 0x7f99c00cfa30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- 172.21.15.157:0/4035940296 >> 172.21.15.157:0/4035940296 conn(0x7f99c006d3d0 msgr2=0x7f99c006d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:11:59.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- 172.21.15.157:0/4035940296 shutdown_connections 2023-11-06T03:11:59.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- 172.21.15.157:0/4035940296 wait complete. 2023-11-06T03:11:59.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 Processor -- start 2023-11-06T03:11:59.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.558+0000 7f99c59ff640 1 -- start start 2023-11-06T03:11:59.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c00719e0 0x7f99c0151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c015e090 0x7f99c00ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01523f0 con 0x7f99c00cce90 2023-11-06T03:11:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00aa060 con 0x7f99c00719e0 2023-11-06T03:11:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00a9d80 con 0x7f99c015e090 2023-11-06T03:11:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44182/0 (socket says 172.21.15.157:44182) 2023-11-06T03:11:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 -- 172.21.15.157:0/3595220908 learned_addr learned my addr 172.21.15.157:0/3595220908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 -- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c015e090 msgr2=0x7f99c00ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c015e090 0x7f99c00ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 -- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c00719e0 msgr2=0x7f99c0151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99beffd640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c00719e0 0x7f99c0151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c00719e0 0x7f99c0151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:11:59.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 -- 172.21.15.157:0/3595220908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00094d0 con 0x7f99c00cce90 2023-11-06T03:11:59.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99be7fc640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f99b400b3f0 tx=0x7f99b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4011830 con 0x7f99c00cce90 2023-11-06T03:11:59.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4011990 con 0x7f99c00cce90 2023-11-06T03:11:59.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4011d40 con 0x7f99c00cce90 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00aa560 con 0x7f99c00cce90 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00aab20 con 0x7f99c00cce90 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00aad40 con 0x7f99c00cce90 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f99b401a400 con 0x7f99c00cce90 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 0x7f99ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:11:59.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99b4083370 con 0x7f99c00cce90 2023-11-06T03:11:59.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99beffd640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 0x7f99ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:11:59.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.562+0000 7f99beffd640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 0x7f99ac067f20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b2000 tx=0x7f99b00aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:11:59.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.566+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f99b4050c60 con 0x7f99c00cce90 2023-11-06T03:11:59.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:11:59.738+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f99c0072750 con 0x7f99c00cce90 2023-11-06T03:11:59.991 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:11:59.990+0000 7f7cfe425640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:12:00.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.434+0000 7f99a7fff640 1 -- 172.21.15.157:0/3595220908 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99b4054280 con 0x7f99c00cce90 2023-11-06T03:12:00.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 msgr2=0x7f99ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 0x7f99ac067f20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b2000 tx=0x7f99b00aa070 comp rx=0 tx=0).stop 2023-11-06T03:12:00.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 msgr2=0x7f99c00acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f99b400b3f0 tx=0x7f99b400b8b0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 shutdown_connections 2023-11-06T03:12:00.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f99ac065a60 0x7f99ac067f20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99c015e090 0x7f99c00ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99c00cce90 0x7f99c00acd70 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 --2- 172.21.15.157:0/3595220908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99c00719e0 0x7f99c0151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 >> 172.21.15.157:0/3595220908 conn(0x7f99c006d3d0 msgr2=0x7f99c00c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:00.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 shutdown_connections 2023-11-06T03:12:00.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.438+0000 7f99c59ff640 1 -- 172.21.15.157:0/3595220908 wait complete. 2023-11-06T03:12:00.460 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:12:00.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:00.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.582+0000 7f49202d9640 1 Processor -- start 2023-11-06T03:12:00.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.582+0000 7f49202d9640 1 -- start start 2023-11-06T03:12:00.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49181599c0 0x7f49180ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f491806d000 0x7f49180b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f491815dea0 0x7f49180b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491815e440 con 0x7f491806d000 2023-11-06T03:12:00.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491806c240 con 0x7f491815dea0 2023-11-06T03:12:00.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f49202d9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491806c520 con 0x7f49181599c0 2023-11-06T03:12:00.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f491806d000 0x7f49180b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f491806d000 0x7f49180b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44184/0 (socket says 172.21.15.157:44184) 2023-11-06T03:12:00.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 -- 172.21.15.157:0/479454569 learned_addr learned my addr 172.21.15.157:0/479454569 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:00.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 -- 172.21.15.157:0/479454569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49181599c0 msgr2=0x7f49180ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:00.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 --2- 172.21.15.157:0/479454569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49181599c0 0x7f49180ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 -- 172.21.15.157:0/479454569 >> v2:172.21.15.195:3300/0 conn(0x7f491815dea0 msgr2=0x7f49180b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:00.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 --2- 172.21.15.157:0/479454569 >> v2:172.21.15.195:3300/0 conn(0x7f491815dea0 0x7f49180b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 -- 172.21.15.157:0/479454569 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491806c800 con 0x7f491806d000 2023-11-06T03:12:00.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d874640 1 --2- 172.21.15.157:0/479454569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f491806d000 0x7f49180b23e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f491806bbf0 tx=0x7f490801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445b418cf4ed6679 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d073640 1 -- 172.21.15.157:0/479454569 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080ac400 con 0x7f491806d000 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d073640 1 -- 172.21.15.157:0/479454569 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080a8050 con 0x7f491806d000 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.586+0000 7f491d073640 1 -- 172.21.15.157:0/479454569 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080ac780 con 0x7f491806d000 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/479454569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f491806d000 msgr2=0x7f49180b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- 172.21.15.157:0/479454569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f491806d000 0x7f49180b23e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f491806bbf0 tx=0x7f490801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/479454569 shutdown_connections 2023-11-06T03:12:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- 172.21.15.157:0/479454569 >> v2:172.21.15.195:3300/0 conn(0x7f491815dea0 0x7f49180b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- 172.21.15.157:0/479454569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f491806d000 0x7f49180b23e0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- 172.21.15.157:0/479454569 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49181599c0 0x7f49180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/479454569 >> 172.21.15.157:0/479454569 conn(0x7f4918065930 msgr2=0x7f4918065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/479454569 shutdown_connections 2023-11-06T03:12:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/479454569 wait complete. 2023-11-06T03:12:00.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 Processor -- start 2023-11-06T03:12:00.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- start start 2023-11-06T03:12:00.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491806d000 0x7f49180af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491815dea0 0x7f491814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491814e7a0 con 0x7f49181599c0 2023-11-06T03:12:00.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491814ea80 con 0x7f491815dea0 2023-11-06T03:12:00.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491814ed60 con 0x7f491806d000 2023-11-06T03:12:00.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44194/0 (socket says 172.21.15.157:44194) 2023-11-06T03:12:00.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 -- 172.21.15.157:0/3287953312 learned_addr learned my addr 172.21.15.157:0/3287953312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:00.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491e075640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491806d000 0x7f49180af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 -- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491806d000 msgr2=0x7f49180af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491806d000 0x7f49180af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 -- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491815dea0 msgr2=0x7f491814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491e876640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491815dea0 0x7f491814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491815dea0 0x7f491814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 -- 172.21.15.157:0/3287953312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49080094d0 con 0x7f49181599c0 2023-11-06T03:12:00.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f491d874640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f490801ab80 tx=0x7f490801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080bd070 con 0x7f49181599c0 2023-11-06T03:12:00.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080b7420 con 0x7f49181599c0 2023-11-06T03:12:00.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491814f040 con 0x7f49181599c0 2023-11-06T03:12:00.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080b7790 con 0x7f49181599c0 2023-11-06T03:12:00.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.590+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49181504e0 con 0x7f49181599c0 2023-11-06T03:12:00.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491814f560 con 0x7f49181599c0 2023-11-06T03:12:00.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f49080b3040 con 0x7f49181599c0 2023-11-06T03:12:00.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f490effd640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 0x7f48fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f490811cbd0 con 0x7f49181599c0 2023-11-06T03:12:00.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f491e075640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 0x7f48fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.594+0000 7f491e075640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 0x7f48fc067f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f49180ce2a0 tx=0x7f4914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.598+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f491814f560 con 0x7f49181599c0 2023-11-06T03:12:00.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.786+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f491814f560 con 0x7f49181599c0 2023-11-06T03:12:00.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.786+0000 7f490effd640 1 -- 172.21.15.157:0/3287953312 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f491814f560 con 0x7f49181599c0 2023-11-06T03:12:00.788 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:00.788 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:00.788 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 msgr2=0x7f48fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 0x7f48fc067f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f49180ce2a0 tx=0x7f4914008040 comp rx=0 tx=0).stop 2023-11-06T03:12:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 msgr2=0x7f491814f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f490801ab80 tx=0x7f490801ad10 comp rx=0 tx=0).stop 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 shutdown_connections 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f48fc065a60 0x7f48fc067f20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f491815dea0 0x7f491814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49181599c0 0x7f491814f840 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 --2- 172.21.15.157:0/3287953312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f491806d000 0x7f49180af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 >> 172.21.15.157:0/3287953312 conn(0x7f4918065930 msgr2=0x7f4918155760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 shutdown_connections 2023-11-06T03:12:00.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.790+0000 7f49202d9640 1 -- 172.21.15.157:0/3287953312 wait complete. 2023-11-06T03:12:00.807 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:00.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.926+0000 7fc477a85640 1 Processor -- start 2023-11-06T03:12:00.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 -- start start 2023-11-06T03:12:00.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc4701599c0 0x7fc4700ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc47006d000 0x7fc4700b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47015e440 con 0x7fc47015dea0 2023-11-06T03:12:00.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47015adb0 con 0x7fc47006d000 2023-11-06T03:12:00.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc477a85640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47006c240 con 0x7fc4701599c0 2023-11-06T03:12:00.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc47006d000 0x7fc4700b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc47006d000 0x7fc4700b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44200/0 (socket says 172.21.15.157:44200) 2023-11-06T03:12:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 -- 172.21.15.157:0/3277074235 learned_addr learned my addr 172.21.15.157:0/3277074235 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475821640 1 --2- 172.21.15.157:0/3277074235 >> v2:172.21.15.195:3300/0 conn(0x7fc4701599c0 0x7fc4700ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 -- 172.21.15.157:0/3277074235 >> v2:172.21.15.195:3300/0 conn(0x7fc4701599c0 msgr2=0x7fc4700ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 --2- 172.21.15.157:0/3277074235 >> v2:172.21.15.195:3300/0 conn(0x7fc4701599c0 0x7fc4700ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 -- 172.21.15.157:0/3277074235 >> v1:172.21.15.195:6789/0 conn(0x7fc47015dea0 legacy=0x7fc470070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:00.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 -- 172.21.15.157:0/3277074235 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47006c520 con 0x7fc47006d000 2023-11-06T03:12:00.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc475020640 1 --2- 172.21.15.157:0/3277074235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47006d000 0x7fc4700b23e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc470066800 tx=0x7fc46001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d966fbe68d2d7b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.930+0000 7fc47481f640 1 -- 172.21.15.157:0/3277074235 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600ac600 con 0x7fc47006d000 2023-11-06T03:12:00.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc47481f640 1 -- 172.21.15.157:0/3277074235 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4600a8050 con 0x7fc47006d000 2023-11-06T03:12:00.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc47481f640 1 -- 172.21.15.157:0/3277074235 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600ac980 con 0x7fc47006d000 2023-11-06T03:12:00.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- 172.21.15.157:0/3277074235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47006d000 msgr2=0x7fc4700b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- 172.21.15.157:0/3277074235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47006d000 0x7fc4700b23e0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc470066800 tx=0x7fc46001c490 comp rx=0 tx=0).stop 2023-11-06T03:12:00.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- 172.21.15.157:0/3277074235 shutdown_connections 2023-11-06T03:12:00.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- 172.21.15.157:0/3277074235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc47006d000 0x7fc4700b23e0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- 172.21.15.157:0/3277074235 >> v2:172.21.15.195:3300/0 conn(0x7fc4701599c0 0x7fc4700ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- 172.21.15.157:0/3277074235 >> 172.21.15.157:0/3277074235 conn(0x7fc470065930 msgr2=0x7fc470065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:00.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- 172.21.15.157:0/3277074235 shutdown_connections 2023-11-06T03:12:00.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- 172.21.15.157:0/3277074235 wait complete. 2023-11-06T03:12:00.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 Processor -- start 2023-11-06T03:12:00.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- start start 2023-11-06T03:12:00.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47006d000 0x7fc4700acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47015dea0 0x7fc470158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47014f290 con 0x7fc4701599c0 2023-11-06T03:12:00.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47014fb10 con 0x7fc47006d000 2023-11-06T03:12:00.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc477a85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701579d0 con 0x7fc47015dea0 2023-11-06T03:12:00.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44212/0 (socket says 172.21.15.157:44212) 2023-11-06T03:12:00.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 -- 172.21.15.157:0/3737030725 learned_addr learned my addr 172.21.15.157:0/3737030725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:00.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475821640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47006d000 0x7fc4700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc476022640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47015dea0 0x7fc470158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 -- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47015dea0 msgr2=0x7fc470158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47015dea0 0x7fc470158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 -- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47006d000 msgr2=0x7fc4700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47006d000 0x7fc4700acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.934+0000 7fc475020640 1 -- 172.21.15.157:0/3737030725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4600094d0 con 0x7fc4701599c0 2023-11-06T03:12:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc475020640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c10 tx=0x7fc460019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600b5610 con 0x7fc4701599c0 2023-11-06T03:12:00.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4600b5790 con 0x7fc4701599c0 2023-11-06T03:12:00.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47014fd60 con 0x7fc4701599c0 2023-11-06T03:12:00.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600b5b40 con 0x7fc4701599c0 2023-11-06T03:12:00.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470157f00 con 0x7fc4701599c0 2023-11-06T03:12:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470072750 con 0x7fc4701599c0 2023-11-06T03:12:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4600b3040 con 0x7fc4701599c0 2023-11-06T03:12:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 0x7fc454067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc46011d050 con 0x7fc4701599c0 2023-11-06T03:12:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc475821640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 0x7fc454067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:00.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.938+0000 7fc475821640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 0x7fc454067f20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cf0e0 tx=0x7fc46c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:00.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:00.942+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4600ea8b0 con 0x7fc4701599c0 2023-11-06T03:12:01.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.098+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4700cb110 con 0x7fc4701599c0 2023-11-06T03:12:01.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc4667fc640 1 -- 172.21.15.157:0/3737030725 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fc4600b1090 con 0x7fc4701599c0 2023-11-06T03:12:01.102 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:01.102 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:59.421679+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:01.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 msgr2=0x7fc454067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 0x7fc454067f20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc4700cf0e0 tx=0x7fc46c008040 comp rx=0 tx=0).stop 2023-11-06T03:12:01.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 msgr2=0x7fc470150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c10 tx=0x7fc460019230 comp rx=0 tx=0).stop 2023-11-06T03:12:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 shutdown_connections 2023-11-06T03:12:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc454065a60 0x7fc454067f20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc47015dea0 0x7fc470158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4701599c0 0x7fc470150150 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 --2- 172.21.15.157:0/3737030725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc47006d000 0x7fc4700acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 >> 172.21.15.157:0/3737030725 conn(0x7fc470065930 msgr2=0x7fc47006ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 shutdown_connections 2023-11-06T03:12:01.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.102+0000 7fc477a85640 1 -- 172.21.15.157:0/3737030725 wait complete. 2023-11-06T03:12:01.121 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:12:01.122 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:01.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 Processor -- start 2023-11-06T03:12:01.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 -- start start 2023-11-06T03:12:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa69806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa69815e5e0 0x7fa6980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69815f580 con 0x7fa69815ea60 2023-11-06T03:12:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69815efc0 con 0x7fa69815e5e0 2023-11-06T03:12:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa69da54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815f2a0 con 0x7fa6981551e0 2023-11-06T03:12:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6977fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa69815ea60 0x7fa69815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44220/0 (socket says 172.21.15.157:44220) 2023-11-06T03:12:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6977fe640 1 -- 172.21.15.157:0/2064058022 learned_addr learned my addr 172.21.15.157:0/2064058022 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa69806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa68ffff640 1 --2- 172.21.15.157:0/2064058022 >> v2:172.21.15.195:3300/0 conn(0x7fa69815e5e0 0x7fa6980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611607237 0 0) 0x7fa69815f580 con 0x7fa69815ea60 2023-11-06T03:12:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa678003530 con 0x7fa69815ea60 2023-11-06T03:12:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2716985087 0 0) 0x7fa678003530 con 0x7fa69815ea60 2023-11-06T03:12:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 msgr2=0x7fa69806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa69806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 >> v2:172.21.15.195:3300/0 conn(0x7fa69815e5e0 msgr2=0x7fa6980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 --2- 172.21.15.157:0/2064058022 >> v2:172.21.15.195:3300/0 conn(0x7fa69815e5e0 0x7fa6980b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69806f5a0 con 0x7fa69815ea60 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa690001a00 con 0x7fa69815ea60 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 >> v1:172.21.15.157:6789/0 conn(0x7fa69815ea60 legacy=0x7fa69815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa678003980 0x7fa678003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6780047d0 0x7fa678004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa678005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815efc0 con 0x7fa678003980 2023-11-06T03:12:01.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69815f2a0 con 0x7fa6780047d0 2023-11-06T03:12:01.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67800b990 con 0x7fa6981551e0 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa678003980 0x7fa678003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6977fe640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa678005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 -- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 msgr2=0x7fa678005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa678005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa68ffff640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6780047d0 0x7fa678004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 -- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6780047d0 msgr2=0x7fa678004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6780047d0 0x7fa678004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa696ffd640 1 -- 172.21.15.157:0/2064058022 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678003530 con 0x7fa678003980 2023-11-06T03:12:01.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.242+0000 7fa6977fe640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa6981551e0 0x7fa678005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:01.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa696ffd640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa678003980 0x7fa678003da0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa688009b10 tx=0x7fa68801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb2e27c55d3b5a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880a6430 con 0x7fa678003980 2023-11-06T03:12:01.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6880a6590 con 0x7fa678003980 2023-11-06T03:12:01.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa6967fc640 1 -- 172.21.15.157:0/2064058022 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880a6940 con 0x7fa678003980 2023-11-06T03:12:01.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- 172.21.15.157:0/2064058022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa678003980 msgr2=0x7fa678003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 --2- 172.21.15.157:0/2064058022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa678003980 0x7fa678003da0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa688009b10 tx=0x7fa68801ca20 comp rx=0 tx=0).stop 2023-11-06T03:12:01.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa6977fe640 1 -- 172.21.15.157:0/2064058022 reap_dead start 2023-11-06T03:12:01.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- 172.21.15.157:0/2064058022 shutdown_connections 2023-11-06T03:12:01.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- 172.21.15.157:0/2064058022 >> 172.21.15.157:0/2064058022 conn(0x7fa698065930 msgr2=0x7fa698065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- 172.21.15.157:0/2064058022 shutdown_connections 2023-11-06T03:12:01.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- 172.21.15.157:0/2064058022 wait complete. 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 Processor -- start 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.246+0000 7fa69da54640 1 -- start start 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa698163ae0 0x7fa698161950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa698163ec0 0x7fa698152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698164870 con 0x7fa69814f140 2023-11-06T03:12:01.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698161f30 con 0x7fa698163ec0 2023-11-06T03:12:01.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69814e380 con 0x7fa698163ae0 2023-11-06T03:12:01.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44222/0 (socket says 172.21.15.157:44222) 2023-11-06T03:12:01.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 -- 172.21.15.157:0/3072822415 learned_addr learned my addr 172.21.15.157:0/3072822415 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:01.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa6977fe640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa698163ec0 0x7fa698152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa696ffd640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa698163ae0 0x7fa698161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 -- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa698163ae0 msgr2=0x7fa698161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa698163ae0 0x7fa698161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 -- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa698163ec0 msgr2=0x7fa698152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa698163ec0 0x7fa698152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 -- 172.21.15.157:0/3072822415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69814e660 con 0x7fa69814f140 2023-11-06T03:12:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa68ffff640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa680010900 tx=0x7fa680010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6800096b0 con 0x7fa69814f140 2023-11-06T03:12:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68000ec20 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68001c400 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69814e940 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa698153030 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa698072750 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa68001c560 con 0x7fa69814f140 2023-11-06T03:12:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 0x7fa67c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6800838f0 con 0x7fa69814f140 2023-11-06T03:12:01.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.250+0000 7fa696ffd640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 0x7fa67c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.254+0000 7fa696ffd640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 0x7fa67c067f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad000 tx=0x7fa6880b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.254+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa680051160 con 0x7fa69814f140 2023-11-06T03:12:01.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.406+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6980b3d60 con 0x7fa67c065a60 2023-11-06T03:12:01.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.430+0000 7fa694ff9640 1 -- 172.21.15.157:0/3072822415 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310281 (secure 0 0 0) 0x7fa6880a6700 con 0x7fa67c065a60 2023-11-06T03:12:01.432 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 msgr2=0x7fa67c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 0x7fa67c067f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6880ad000 tx=0x7fa6880b2040 comp rx=0 tx=0).stop 2023-11-06T03:12:01.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 msgr2=0x7fa698164330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa680010900 tx=0x7fa680010dc0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 shutdown_connections 2023-11-06T03:12:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa67c065a60 0x7fa67c067f20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa698163ec0 0x7fa698152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa69814f140 0x7fa698164330 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 --2- 172.21.15.157:0/3072822415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa698163ae0 0x7fa698161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 >> 172.21.15.157:0/3072822415 conn(0x7fa698065930 msgr2=0x7fa6980afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 shutdown_connections 2023-11-06T03:12:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.434+0000 7fa69da54640 1 -- 172.21.15.157:0/3072822415 wait complete. 2023-11-06T03:12:01.451 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2023-11-06T03:12:01.268985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":554088,"kb_used_data":10600,"kb_used_omap":0,"kb_used_meta":543424,"kb_avail":754420632,"statfs":{"total":773094113280,"available":772526727168,"internally_reserved":0,"allocated":10854400,"data_stored":5592083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":556466176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.220806"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451606+0000","last_change":"2023-11-06T03:11:58.450170+0000","last_active":"2023-11-06T03:11:58.451606+0000","last_peered":"2023-11-06T03:11:58.451606+0000","last_clean":"2023-11-06T03:11:58.451606+0000","last_became_active":"2023-11-06T03:11:58.449877+0000","last_became_peered":"2023-11-06T03:11:58.449877+0000","last_unstale":"2023-11-06T03:11:58.451606+0000","last_undegraded":"2023-11-06T03:11:58.451606+0000","last_fullsized":"2023-11-06T03:11:58.451606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479188+0000","last_change":"2023-11-06T03:11:58.465598+0000","last_active":"2023-11-06T03:11:58.479188+0000","last_peered":"2023-11-06T03:11:58.479188+0000","last_clean":"2023-11-06T03:11:58.479188+0000","last_became_active":"2023-11-06T03:11:58.464776+0000","last_became_peered":"2023-11-06T03:11:58.464776+0000","last_unstale":"2023-11-06T03:11:58.479188+0000","last_undegraded":"2023-11-06T03:11:58.479188+0000","last_fullsized":"2023-11-06T03:11:58.479188+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.463767+0000","last_change":"2023-11-06T03:11:58.462034+0000","last_active":"2023-11-06T03:11:58.463767+0000","last_peered":"2023-11-06T03:11:58.463767+0000","last_clean":"2023-11-06T03:11:58.463767+0000","last_became_active":"2023-11-06T03:11:58.461710+0000","last_became_peered":"2023-11-06T03:11:58.461710+0000","last_unstale":"2023-11-06T03:11:58.463767+0000","last_undegraded":"2023-11-06T03:11:58.463767+0000","last_fullsized":"2023-11-06T03:11:58.463767+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469542+0000","last_change":"2023-11-06T03:11:58.469136+0000","last_active":"2023-11-06T03:11:58.469542+0000","last_peered":"2023-11-06T03:11:58.469542+0000","last_clean":"2023-11-06T03:11:58.469542+0000","last_became_active":"2023-11-06T03:11:58.468859+0000","last_became_peered":"2023-11-06T03:11:58.468859+0000","last_unstale":"2023-11-06T03:11:58.469542+0000","last_undegraded":"2023-11-06T03:11:58.469542+0000","last_fullsized":"2023-11-06T03:11:58.469542+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.472053+0000","last_change":"2023-11-06T03:11:58.456581+0000","last_active":"2023-11-06T03:11:58.472053+0000","last_peered":"2023-11-06T03:11:58.472053+0000","last_clean":"2023-11-06T03:11:58.472053+0000","last_became_active":"2023-11-06T03:11:58.455485+0000","last_became_peered":"2023-11-06T03:11:58.455485+0000","last_unstale":"2023-11-06T03:11:58.472053+0000","last_undegraded":"2023-11-06T03:11:58.472053+0000","last_fullsized":"2023-11-06T03:11:58.472053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:46.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458858+0000","last_change":"2023-11-06T03:11:58.457885+0000","last_active":"2023-11-06T03:11:58.458858+0000","last_peered":"2023-11-06T03:11:58.458858+0000","last_clean":"2023-11-06T03:11:58.458858+0000","last_became_active":"2023-11-06T03:11:58.457569+0000","last_became_peered":"2023-11-06T03:11:58.457569+0000","last_unstale":"2023-11-06T03:11:58.458858+0000","last_undegraded":"2023-11-06T03:11:58.458858+0000","last_fullsized":"2023-11-06T03:11:58.458858+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:12.578711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487895+0000","last_change":"2023-11-06T03:11:58.480941+0000","last_active":"2023-11-06T03:11:58.487895+0000","last_peered":"2023-11-06T03:11:58.487895+0000","last_clean":"2023-11-06T03:11:58.487895+0000","last_became_active":"2023-11-06T03:11:58.480329+0000","last_became_peered":"2023-11-06T03:11:58.480329+0000","last_unstale":"2023-11-06T03:11:58.487895+0000","last_undegraded":"2023-11-06T03:11:58.487895+0000","last_fullsized":"2023-11-06T03:11:58.487895+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.482119+0000","last_change":"2023-11-06T03:11:58.480840+0000","last_active":"2023-11-06T03:11:58.482119+0000","last_peered":"2023-11-06T03:11:58.482119+0000","last_clean":"2023-11-06T03:11:58.482119+0000","last_became_active":"2023-11-06T03:11:58.480577+0000","last_became_peered":"2023-11-06T03:11:58.480577+0000","last_unstale":"2023-11-06T03:11:58.482119+0000","last_undegraded":"2023-11-06T03:11:58.482119+0000","last_fullsized":"2023-11-06T03:11:58.482119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479777+0000","last_change":"2023-11-06T03:11:58.465137+0000","last_active":"2023-11-06T03:11:58.479777+0000","last_peered":"2023-11-06T03:11:58.479777+0000","last_clean":"2023-11-06T03:11:58.479777+0000","last_became_active":"2023-11-06T03:11:58.464402+0000","last_became_peered":"2023-11-06T03:11:58.464402+0000","last_unstale":"2023-11-06T03:11:58.479777+0000","last_undegraded":"2023-11-06T03:11:58.479777+0000","last_fullsized":"2023-11-06T03:11:58.479777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:52.571964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455229+0000","last_change":"2023-11-06T03:11:58.452750+0000","last_active":"2023-11-06T03:11:58.455229+0000","last_peered":"2023-11-06T03:11:58.455229+0000","last_clean":"2023-11-06T03:11:58.455229+0000","last_became_active":"2023-11-06T03:11:58.452367+0000","last_became_peered":"2023-11-06T03:11:58.452367+0000","last_unstale":"2023-11-06T03:11:58.455229+0000","last_undegraded":"2023-11-06T03:11:58.455229+0000","last_fullsized":"2023-11-06T03:11:58.455229+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:37.535125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.484212+0000","last_change":"2023-11-06T03:11:58.473516+0000","last_active":"2023-11-06T03:11:58.484212+0000","last_peered":"2023-11-06T03:11:58.484212+0000","last_clean":"2023-11-06T03:11:58.484212+0000","last_became_active":"2023-11-06T03:11:58.471349+0000","last_became_peered":"2023-11-06T03:11:58.471349+0000","last_unstale":"2023-11-06T03:11:58.484212+0000","last_undegraded":"2023-11-06T03:11:58.484212+0000","last_fullsized":"2023-11-06T03:11:58.484212+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.448531+0000","last_change":"2023-11-06T03:11:58.441775+0000","last_active":"2023-11-06T03:11:58.448531+0000","last_peered":"2023-11-06T03:11:58.448531+0000","last_clean":"2023-11-06T03:11:58.448531+0000","last_became_active":"2023-11-06T03:11:58.441507+0000","last_became_peered":"2023-11-06T03:11:58.441507+0000","last_unstale":"2023-11-06T03:11:58.448531+0000","last_undegraded":"2023-11-06T03:11:58.448531+0000","last_fullsized":"2023-11-06T03:11:58.448531+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:38:34.605702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.483799+0000","last_change":"2023-11-06T03:11:58.483415+0000","last_active":"2023-11-06T03:11:58.483799+0000","last_peered":"2023-11-06T03:11:58.483799+0000","last_clean":"2023-11-06T03:11:58.483799+0000","last_became_active":"2023-11-06T03:11:58.483167+0000","last_became_peered":"2023-11-06T03:11:58.483167+0000","last_unstale":"2023-11-06T03:11:58.483799+0000","last_undegraded":"2023-11-06T03:11:58.483799+0000","last_fullsized":"2023-11-06T03:11:58.483799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458485+0000","last_change":"2023-11-06T03:11:58.446077+0000","last_active":"2023-11-06T03:11:58.458485+0000","last_peered":"2023-11-06T03:11:58.458485+0000","last_clean":"2023-11-06T03:11:58.458485+0000","last_became_active":"2023-11-06T03:11:58.445332+0000","last_became_peered":"2023-11-06T03:11:58.445332+0000","last_unstale":"2023-11-06T03:11:58.458485+0000","last_undegraded":"2023-11-06T03:11:58.458485+0000","last_fullsized":"2023-11-06T03:11:58.458485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456940+0000","last_change":"2023-11-06T03:11:58.448291+0000","last_active":"2023-11-06T03:11:58.456940+0000","last_peered":"2023-11-06T03:11:58.456940+0000","last_clean":"2023-11-06T03:11:58.456940+0000","last_became_active":"2023-11-06T03:11:58.447495+0000","last_became_peered":"2023-11-06T03:11:58.447495+0000","last_unstale":"2023-11-06T03:11:58.456940+0000","last_undegraded":"2023-11-06T03:11:58.456940+0000","last_fullsized":"2023-11-06T03:11:58.456940+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:24.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.447389+0000","last_change":"2023-11-06T03:11:58.445588+0000","last_active":"2023-11-06T03:11:58.447389+0000","last_peered":"2023-11-06T03:11:58.447389+0000","last_clean":"2023-11-06T03:11:58.447389+0000","last_became_active":"2023-11-06T03:11:58.445350+0000","last_became_peered":"2023-11-06T03:11:58.445350+0000","last_unstale":"2023-11-06T03:11:58.447389+0000","last_undegraded":"2023-11-06T03:11:58.447389+0000","last_fullsized":"2023-11-06T03:11:58.447389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:28.153196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456471+0000","last_change":"2023-11-06T03:11:58.452436+0000","last_active":"2023-11-06T03:11:58.456471+0000","last_peered":"2023-11-06T03:11:58.456471+0000","last_clean":"2023-11-06T03:11:58.456471+0000","last_became_active":"2023-11-06T03:11:58.451978+0000","last_became_peered":"2023-11-06T03:11:58.451978+0000","last_unstale":"2023-11-06T03:11:58.456471+0000","last_undegraded":"2023-11-06T03:11:58.456471+0000","last_fullsized":"2023-11-06T03:11:58.456471+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:26.239442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461390+0000","last_change":"2023-11-06T03:11:58.460226+0000","last_active":"2023-11-06T03:11:58.461390+0000","last_peered":"2023-11-06T03:11:58.461390+0000","last_clean":"2023-11-06T03:11:58.461390+0000","last_became_active":"2023-11-06T03:11:58.459919+0000","last_became_peered":"2023-11-06T03:11:58.459919+0000","last_unstale":"2023-11-06T03:11:58.461390+0000","last_undegraded":"2023-11-06T03:11:58.461390+0000","last_fullsized":"2023-11-06T03:11:58.461390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:43:36.369931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459561+0000","last_change":"2023-11-06T03:11:58.448520+0000","last_active":"2023-11-06T03:11:58.459561+0000","last_peered":"2023-11-06T03:11:58.459561+0000","last_clean":"2023-11-06T03:11:58.459561+0000","last_became_active":"2023-11-06T03:11:58.447792+0000","last_became_peered":"2023-11-06T03:11:58.447792+0000","last_unstale":"2023-11-06T03:11:58.459561+0000","last_undegraded":"2023-11-06T03:11:58.459561+0000","last_fullsized":"2023-11-06T03:11:58.459561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456724+0000","last_change":"2023-11-06T03:11:58.447321+0000","last_active":"2023-11-06T03:11:58.456724+0000","last_peered":"2023-11-06T03:11:58.456724+0000","last_clean":"2023-11-06T03:11:58.456724+0000","last_became_active":"2023-11-06T03:11:58.447089+0000","last_became_peered":"2023-11-06T03:11:58.447089+0000","last_unstale":"2023-11-06T03:11:58.456724+0000","last_undegraded":"2023-11-06T03:11:58.456724+0000","last_fullsized":"2023-11-06T03:11:58.456724+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:24:02.000380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487611+0000","last_change":"2023-11-06T03:11:58.486085+0000","last_active":"2023-11-06T03:11:58.487611+0000","last_peered":"2023-11-06T03:11:58.487611+0000","last_clean":"2023-11-06T03:11:58.487611+0000","last_became_active":"2023-11-06T03:11:58.485220+0000","last_became_peered":"2023-11-06T03:11:58.485220+0000","last_unstale":"2023-11-06T03:11:58.487611+0000","last_undegraded":"2023-11-06T03:11:58.487611+0000","last_fullsized":"2023-11-06T03:11:58.487611+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:43:36.369931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455720+0000","last_change":"2023-11-06T03:11:58.445001+0000","last_active":"2023-11-06T03:11:58.455720+0000","last_peered":"2023-11-06T03:11:58.455720+0000","last_clean":"2023-11-06T03:11:58.455720+0000","last_became_active":"2023-11-06T03:11:58.444302+0000","last_became_peered":"2023-11-06T03:11:58.444302+0000","last_unstale":"2023-11-06T03:11:58.455720+0000","last_undegraded":"2023-11-06T03:11:58.455720+0000","last_fullsized":"2023-11-06T03:11:58.455720+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.476692+0000","last_change":"2023-11-06T03:11:58.471694+0000","last_active":"2023-11-06T03:11:58.476692+0000","last_peered":"2023-11-06T03:11:58.476692+0000","last_clean":"2023-11-06T03:11:58.476692+0000","last_became_active":"2023-11-06T03:11:58.470412+0000","last_became_peered":"2023-11-06T03:11:58.470412+0000","last_unstale":"2023-11-06T03:11:58.476692+0000","last_undegraded":"2023-11-06T03:11:58.476692+0000","last_fullsized":"2023-11-06T03:11:58.476692+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:28.854478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441664+0000","last_change":"2023-11-06T03:11:58.439042+0000","last_active":"2023-11-06T03:11:58.441664+0000","last_peered":"2023-11-06T03:11:58.441664+0000","last_clean":"2023-11-06T03:11:58.441664+0000","last_became_active":"2023-11-06T03:11:58.438732+0000","last_became_peered":"2023-11-06T03:11:58.438732+0000","last_unstale":"2023-11-06T03:11:58.441664+0000","last_undegraded":"2023-11-06T03:11:58.441664+0000","last_fullsized":"2023-11-06T03:11:58.441664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:52.571964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479454+0000","last_change":"2023-11-06T03:11:58.476845+0000","last_active":"2023-11-06T03:11:58.479454+0000","last_peered":"2023-11-06T03:11:58.479454+0000","last_clean":"2023-11-06T03:11:58.479454+0000","last_became_active":"2023-11-06T03:11:58.476204+0000","last_became_peered":"2023-11-06T03:11:58.476204+0000","last_unstale":"2023-11-06T03:11:58.479454+0000","last_undegraded":"2023-11-06T03:11:58.479454+0000","last_fullsized":"2023-11-06T03:11:58.479454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458778+0000","last_change":"2023-11-06T03:11:58.448344+0000","last_active":"2023-11-06T03:11:58.458778+0000","last_peered":"2023-11-06T03:11:58.458778+0000","last_clean":"2023-11-06T03:11:58.458778+0000","last_became_active":"2023-11-06T03:11:58.447628+0000","last_became_peered":"2023-11-06T03:11:58.447628+0000","last_unstale":"2023-11-06T03:11:58.458778+0000","last_undegraded":"2023-11-06T03:11:58.458778+0000","last_fullsized":"2023-11-06T03:11:58.458778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456041+0000","last_change":"2023-11-06T03:11:58.454421+0000","last_active":"2023-11-06T03:11:58.456041+0000","last_peered":"2023-11-06T03:11:58.456041+0000","last_clean":"2023-11-06T03:11:58.456041+0000","last_became_active":"2023-11-06T03:11:58.454108+0000","last_became_peered":"2023-11-06T03:11:58.454108+0000","last_unstale":"2023-11-06T03:11:58.456041+0000","last_undegraded":"2023-11-06T03:11:58.456041+0000","last_fullsized":"2023-11-06T03:11:58.456041+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:11.707978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.450035+0000","last_change":"2023-11-06T03:11:58.446365+0000","last_active":"2023-11-06T03:11:58.450035+0000","last_peered":"2023-11-06T03:11:58.450035+0000","last_clean":"2023-11-06T03:11:58.450035+0000","last_became_active":"2023-11-06T03:11:58.446139+0000","last_became_peered":"2023-11-06T03:11:58.446139+0000","last_unstale":"2023-11-06T03:11:58.450035+0000","last_undegraded":"2023-11-06T03:11:58.450035+0000","last_fullsized":"2023-11-06T03:11:58.450035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:31.039087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.488077+0000","last_change":"2023-11-06T03:11:58.484257+0000","last_active":"2023-11-06T03:11:58.488077+0000","last_peered":"2023-11-06T03:11:58.488077+0000","last_clean":"2023-11-06T03:11:58.488077+0000","last_became_active":"2023-11-06T03:11:58.481807+0000","last_became_peered":"2023-11-06T03:11:58.481807+0000","last_unstale":"2023-11-06T03:11:58.488077+0000","last_undegraded":"2023-11-06T03:11:58.488077+0000","last_fullsized":"2023-11-06T03:11:58.488077+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479743+0000","last_change":"2023-11-06T03:11:58.464986+0000","last_active":"2023-11-06T03:11:58.479743+0000","last_peered":"2023-11-06T03:11:58.479743+0000","last_clean":"2023-11-06T03:11:58.479743+0000","last_became_active":"2023-11-06T03:11:58.464259+0000","last_became_peered":"2023-11-06T03:11:58.464259+0000","last_unstale":"2023-11-06T03:11:58.479743+0000","last_undegraded":"2023-11-06T03:11:58.479743+0000","last_fullsized":"2023-11-06T03:11:58.479743+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469770+0000","last_change":"2023-11-06T03:11:58.460708+0000","last_active":"2023-11-06T03:11:58.469770+0000","last_peered":"2023-11-06T03:11:58.469770+0000","last_clean":"2023-11-06T03:11:58.469770+0000","last_became_active":"2023-11-06T03:11:58.459932+0000","last_became_peered":"2023-11-06T03:11:58.459932+0000","last_unstale":"2023-11-06T03:11:58.469770+0000","last_undegraded":"2023-11-06T03:11:58.469770+0000","last_fullsized":"2023-11-06T03:11:58.469770+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456494+0000","last_change":"2023-11-06T03:11:58.452473+0000","last_active":"2023-11-06T03:11:58.456494+0000","last_peered":"2023-11-06T03:11:58.456494+0000","last_clean":"2023-11-06T03:11:58.456494+0000","last_became_active":"2023-11-06T03:11:58.452208+0000","last_became_peered":"2023-11-06T03:11:58.452208+0000","last_unstale":"2023-11-06T03:11:58.456494+0000","last_undegraded":"2023-11-06T03:11:58.456494+0000","last_fullsized":"2023-11-06T03:11:58.456494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:17:24.008019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481124+0000","last_change":"2023-11-06T03:11:58.480431+0000","last_active":"2023-11-06T03:11:58.481124+0000","last_peered":"2023-11-06T03:11:58.481124+0000","last_clean":"2023-11-06T03:11:58.481124+0000","last_became_active":"2023-11-06T03:11:58.480161+0000","last_became_peered":"2023-11-06T03:11:58.480161+0000","last_unstale":"2023-11-06T03:11:58.481124+0000","last_undegraded":"2023-11-06T03:11:58.481124+0000","last_fullsized":"2023-11-06T03:11:58.481124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.709058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"98'4","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.480019+0000","last_change":"2023-11-06T03:11:58.086704+0000","last_active":"2023-11-06T03:11:58.480019+0000","last_peered":"2023-11-06T03:11:58.480019+0000","last_clean":"2023-11-06T03:11:58.480019+0000","last_became_active":"2023-11-06T03:11:58.085984+0000","last_became_peered":"2023-11-06T03:11:58.085984+0000","last_unstale":"2023-11-06T03:11:58.480019+0000","last_undegraded":"2023-11-06T03:11:58.480019+0000","last_fullsized":"2023-11-06T03:11:58.480019+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:35.946625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456559+0000","last_change":"2023-11-06T03:11:58.453372+0000","last_active":"2023-11-06T03:11:58.456559+0000","last_peered":"2023-11-06T03:11:58.456559+0000","last_clean":"2023-11-06T03:11:58.456559+0000","last_became_active":"2023-11-06T03:11:58.453101+0000","last_became_peered":"2023-11-06T03:11:58.453101+0000","last_unstale":"2023-11-06T03:11:58.456559+0000","last_undegraded":"2023-11-06T03:11:58.456559+0000","last_fullsized":"2023-11-06T03:11:58.456559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:31:17.741993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.446575+0000","last_change":"2023-11-06T03:11:58.098022+0000","last_active":"2023-11-06T03:11:58.446575+0000","last_peered":"2023-11-06T03:11:58.446575+0000","last_clean":"2023-11-06T03:11:58.446575+0000","last_became_active":"2023-11-06T03:11:58.097424+0000","last_became_peered":"2023-11-06T03:11:58.097424+0000","last_unstale":"2023-11-06T03:11:58.446575+0000","last_undegraded":"2023-11-06T03:11:58.446575+0000","last_fullsized":"2023-11-06T03:11:58.446575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:13.672419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487909+0000","last_change":"2023-11-06T03:11:58.481130+0000","last_active":"2023-11-06T03:11:58.487909+0000","last_peered":"2023-11-06T03:11:58.487909+0000","last_clean":"2023-11-06T03:11:58.487909+0000","last_became_active":"2023-11-06T03:11:58.480424+0000","last_became_peered":"2023-11-06T03:11:58.480424+0000","last_unstale":"2023-11-06T03:11:58.487909+0000","last_undegraded":"2023-11-06T03:11:58.487909+0000","last_fullsized":"2023-11-06T03:11:58.487909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481552+0000","last_change":"2023-11-06T03:11:58.101485+0000","last_active":"2023-11-06T03:11:58.481552+0000","last_peered":"2023-11-06T03:11:58.481552+0000","last_clean":"2023-11-06T03:11:58.481552+0000","last_became_active":"2023-11-06T03:11:58.100699+0000","last_became_peered":"2023-11-06T03:11:58.100699+0000","last_unstale":"2023-11-06T03:11:58.481552+0000","last_undegraded":"2023-11-06T03:11:58.481552+0000","last_fullsized":"2023-11-06T03:11:58.481552+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:40.038408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.475399+0000","last_change":"2023-11-06T03:11:58.475034+0000","last_active":"2023-11-06T03:11:58.475399+0000","last_peered":"2023-11-06T03:11:58.475399+0000","last_clean":"2023-11-06T03:11:58.475399+0000","last_became_active":"2023-11-06T03:11:58.474804+0000","last_became_peered":"2023-11-06T03:11:58.474804+0000","last_unstale":"2023-11-06T03:11:58.475399+0000","last_undegraded":"2023-11-06T03:11:58.475399+0000","last_fullsized":"2023-11-06T03:11:58.475399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:58:35.372951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.471654+0000","last_change":"2023-11-06T03:11:58.097472+0000","last_active":"2023-11-06T03:11:58.471654+0000","last_peered":"2023-11-06T03:11:58.471654+0000","last_clean":"2023-11-06T03:11:58.471654+0000","last_became_active":"2023-11-06T03:11:58.096872+0000","last_became_peered":"2023-11-06T03:11:58.096872+0000","last_unstale":"2023-11-06T03:11:58.471654+0000","last_undegraded":"2023-11-06T03:11:58.471654+0000","last_fullsized":"2023-11-06T03:11:58.471654+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:05.012267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458203+0000","last_change":"2023-11-06T03:11:58.451973+0000","last_active":"2023-11-06T03:11:58.458203+0000","last_peered":"2023-11-06T03:11:58.458203+0000","last_clean":"2023-11-06T03:11:58.458203+0000","last_became_active":"2023-11-06T03:11:58.451676+0000","last_became_peered":"2023-11-06T03:11:58.451676+0000","last_unstale":"2023-11-06T03:11:58.458203+0000","last_undegraded":"2023-11-06T03:11:58.458203+0000","last_fullsized":"2023-11-06T03:11:58.458203+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:08.067884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441502+0000","last_change":"2023-11-06T03:11:58.089763+0000","last_active":"2023-11-06T03:11:58.441502+0000","last_peered":"2023-11-06T03:11:58.441502+0000","last_clean":"2023-11-06T03:11:58.441502+0000","last_became_active":"2023-11-06T03:11:58.089023+0000","last_became_peered":"2023-11-06T03:11:58.089023+0000","last_unstale":"2023-11-06T03:11:58.441502+0000","last_undegraded":"2023-11-06T03:11:58.441502+0000","last_fullsized":"2023-11-06T03:11:58.441502+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:19.805243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.476648+0000","last_change":"2023-11-06T03:11:58.471197+0000","last_active":"2023-11-06T03:11:58.476648+0000","last_peered":"2023-11-06T03:11:58.476648+0000","last_clean":"2023-11-06T03:11:58.476648+0000","last_became_active":"2023-11-06T03:11:58.469584+0000","last_became_peered":"2023-11-06T03:11:58.469584+0000","last_unstale":"2023-11-06T03:11:58.476648+0000","last_undegraded":"2023-11-06T03:11:58.476648+0000","last_fullsized":"2023-11-06T03:11:58.476648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:43:41.328211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.450850+0000","last_change":"2023-11-06T03:11:58.085794+0000","last_active":"2023-11-06T03:11:58.450850+0000","last_peered":"2023-11-06T03:11:58.450850+0000","last_clean":"2023-11-06T03:11:58.450850+0000","last_became_active":"2023-11-06T03:11:58.085427+0000","last_became_peered":"2023-11-06T03:11:58.085427+0000","last_unstale":"2023-11-06T03:11:58.450850+0000","last_undegraded":"2023-11-06T03:11:58.450850+0000","last_fullsized":"2023-11-06T03:11:58.450850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:19.805243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458839+0000","last_change":"2023-11-06T03:11:58.457584+0000","last_active":"2023-11-06T03:11:58.458839+0000","last_peered":"2023-11-06T03:11:58.458839+0000","last_clean":"2023-11-06T03:11:58.458839+0000","last_became_active":"2023-11-06T03:11:58.457315+0000","last_became_peered":"2023-11-06T03:11:58.457315+0000","last_unstale":"2023-11-06T03:11:58.458839+0000","last_undegraded":"2023-11-06T03:11:58.458839+0000","last_fullsized":"2023-11-06T03:11:58.458839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:45:36.660841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.425791+0000","last_change":"2023-11-06T03:11:58.094019+0000","last_active":"2023-11-06T03:11:58.425791+0000","last_peered":"2023-11-06T03:11:58.425791+0000","last_clean":"2023-11-06T03:11:58.425791+0000","last_became_active":"2023-11-06T03:11:58.093712+0000","last_became_peered":"2023-11-06T03:11:58.093712+0000","last_unstale":"2023-11-06T03:11:58.425791+0000","last_undegraded":"2023-11-06T03:11:58.425791+0000","last_fullsized":"2023-11-06T03:11:58.425791+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:24:00.987727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.464828+0000","last_change":"2023-11-06T03:11:58.464112+0000","last_active":"2023-11-06T03:11:58.464828+0000","last_peered":"2023-11-06T03:11:58.464828+0000","last_clean":"2023-11-06T03:11:58.464828+0000","last_became_active":"2023-11-06T03:11:58.463726+0000","last_became_peered":"2023-11-06T03:11:58.463726+0000","last_unstale":"2023-11-06T03:11:58.464828+0000","last_undegraded":"2023-11-06T03:11:58.464828+0000","last_fullsized":"2023-11-06T03:11:58.464828+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.746612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458301+0000","last_change":"2023-11-06T03:11:58.101207+0000","last_active":"2023-11-06T03:11:58.458301+0000","last_peered":"2023-11-06T03:11:58.458301+0000","last_clean":"2023-11-06T03:11:58.458301+0000","last_became_active":"2023-11-06T03:11:58.100369+0000","last_became_peered":"2023-11-06T03:11:58.100369+0000","last_unstale":"2023-11-06T03:11:58.458301+0000","last_undegraded":"2023-11-06T03:11:58.458301+0000","last_fullsized":"2023-11-06T03:11:58.458301+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:07.055232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459241+0000","last_change":"2023-11-06T03:11:58.448711+0000","last_active":"2023-11-06T03:11:58.459241+0000","last_peered":"2023-11-06T03:11:58.459241+0000","last_clean":"2023-11-06T03:11:58.459241+0000","last_became_active":"2023-11-06T03:11:58.448039+0000","last_became_peered":"2023-11-06T03:11:58.448039+0000","last_unstale":"2023-11-06T03:11:58.459241+0000","last_undegraded":"2023-11-06T03:11:58.459241+0000","last_fullsized":"2023-11-06T03:11:58.459241+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:24.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.454700+0000","last_change":"2023-11-06T03:11:58.097033+0000","last_active":"2023-11-06T03:11:58.454700+0000","last_peered":"2023-11-06T03:11:58.454700+0000","last_clean":"2023-11-06T03:11:58.454700+0000","last_became_active":"2023-11-06T03:11:58.096266+0000","last_became_peered":"2023-11-06T03:11:58.096266+0000","last_unstale":"2023-11-06T03:11:58.454700+0000","last_undegraded":"2023-11-06T03:11:58.454700+0000","last_fullsized":"2023-11-06T03:11:58.454700+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:20.359200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.488857+0000","last_change":"2023-11-06T03:11:58.485657+0000","last_active":"2023-11-06T03:11:58.488857+0000","last_peered":"2023-11-06T03:11:58.488857+0000","last_clean":"2023-11-06T03:11:58.488857+0000","last_became_active":"2023-11-06T03:11:58.485127+0000","last_became_peered":"2023-11-06T03:11:58.485127+0000","last_unstale":"2023-11-06T03:11:58.488857+0000","last_undegraded":"2023-11-06T03:11:58.488857+0000","last_fullsized":"2023-11-06T03:11:58.488857+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.426528+0000","last_change":"2023-11-06T03:11:58.084476+0000","last_active":"2023-11-06T03:11:58.426528+0000","last_peered":"2023-11-06T03:11:58.426528+0000","last_clean":"2023-11-06T03:11:58.426528+0000","last_became_active":"2023-11-06T03:11:58.084133+0000","last_became_peered":"2023-11-06T03:11:58.084133+0000","last_unstale":"2023-11-06T03:11:58.426528+0000","last_undegraded":"2023-11-06T03:11:58.426528+0000","last_fullsized":"2023-11-06T03:11:58.426528+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:07.055232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455319+0000","last_change":"2023-11-06T03:11:58.453371+0000","last_active":"2023-11-06T03:11:58.455319+0000","last_peered":"2023-11-06T03:11:58.455319+0000","last_clean":"2023-11-06T03:11:58.455319+0000","last_became_active":"2023-11-06T03:11:58.453115+0000","last_became_peered":"2023-11-06T03:11:58.453115+0000","last_unstale":"2023-11-06T03:11:58.455319+0000","last_undegraded":"2023-11-06T03:11:58.455319+0000","last_fullsized":"2023-11-06T03:11:58.455319+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"98'3","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.471804+0000","last_change":"2023-11-06T03:11:58.086569+0000","last_active":"2023-11-06T03:11:58.471804+0000","last_peered":"2023-11-06T03:11:58.471804+0000","last_clean":"2023-11-06T03:11:58.471804+0000","last_became_active":"2023-11-06T03:11:58.085063+0000","last_became_peered":"2023-11-06T03:11:58.085063+0000","last_unstale":"2023-11-06T03:11:58.471804+0000","last_undegraded":"2023-11-06T03:11:58.471804+0000","last_fullsized":"2023-11-06T03:11:58.471804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:27.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.465859+0000","last_change":"2023-11-06T03:11:58.465346+0000","last_active":"2023-11-06T03:11:58.465859+0000","last_peered":"2023-11-06T03:11:58.465859+0000","last_clean":"2023-11-06T03:11:58.465859+0000","last_became_active":"2023-11-06T03:11:58.465098+0000","last_became_peered":"2023-11-06T03:11:58.465098+0000","last_unstale":"2023-11-06T03:11:58.465859+0000","last_undegraded":"2023-11-06T03:11:58.465859+0000","last_fullsized":"2023-11-06T03:11:58.465859+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"98'6","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.478054+0000","last_change":"2023-11-06T03:11:58.098265+0000","last_active":"2023-11-06T03:11:58.478054+0000","last_peered":"2023-11-06T03:11:58.478054+0000","last_clean":"2023-11-06T03:11:58.478054+0000","last_became_active":"2023-11-06T03:11:58.097518+0000","last_became_peered":"2023-11-06T03:11:58.097518+0000","last_unstale":"2023-11-06T03:11:58.478054+0000","last_undegraded":"2023-11-06T03:11:58.478054+0000","last_fullsized":"2023-11-06T03:11:58.478054+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:03.021218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.482136+0000","last_change":"2023-11-06T03:11:58.480935+0000","last_active":"2023-11-06T03:11:58.482136+0000","last_peered":"2023-11-06T03:11:58.482136+0000","last_clean":"2023-11-06T03:11:58.482136+0000","last_became_active":"2023-11-06T03:11:58.480672+0000","last_became_peered":"2023-11-06T03:11:58.480672+0000","last_unstale":"2023-11-06T03:11:58.482136+0000","last_undegraded":"2023-11-06T03:11:58.482136+0000","last_fullsized":"2023-11-06T03:11:58.482136+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:46.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481122+0000","last_change":"2023-11-06T03:11:58.101451+0000","last_active":"2023-11-06T03:11:58.481122+0000","last_peered":"2023-11-06T03:11:58.481122+0000","last_clean":"2023-11-06T03:11:58.481122+0000","last_became_active":"2023-11-06T03:11:58.100833+0000","last_became_peered":"2023-11-06T03:11:58.100833+0000","last_unstale":"2023-11-06T03:11:58.481122+0000","last_undegraded":"2023-11-06T03:11:58.481122+0000","last_fullsized":"2023-11-06T03:11:58.481122+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:22.994152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.484014+0000","last_change":"2023-11-06T03:11:58.474340+0000","last_active":"2023-11-06T03:11:58.484014+0000","last_peered":"2023-11-06T03:11:58.484014+0000","last_clean":"2023-11-06T03:11:58.484014+0000","last_became_active":"2023-11-06T03:11:58.473656+0000","last_became_peered":"2023-11-06T03:11:58.473656+0000","last_unstale":"2023-11-06T03:11:58.484014+0000","last_undegraded":"2023-11-06T03:11:58.484014+0000","last_fullsized":"2023-11-06T03:11:58.484014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.440595+0000","last_change":"2023-11-06T03:11:58.074405+0000","last_active":"2023-11-06T03:11:58.440595+0000","last_peered":"2023-11-06T03:11:58.440595+0000","last_clean":"2023-11-06T03:11:58.440595+0000","last_became_active":"2023-11-06T03:11:58.074095+0000","last_became_peered":"2023-11-06T03:11:58.074095+0000","last_unstale":"2023-11-06T03:11:58.440595+0000","last_undegraded":"2023-11-06T03:11:58.440595+0000","last_fullsized":"2023-11-06T03:11:58.440595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:29.869220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.446604+0000","last_change":"2023-11-06T03:11:58.440306+0000","last_active":"2023-11-06T03:11:58.446604+0000","last_peered":"2023-11-06T03:11:58.446604+0000","last_clean":"2023-11-06T03:11:58.446604+0000","last_became_active":"2023-11-06T03:11:58.439993+0000","last_became_peered":"2023-11-06T03:11:58.439993+0000","last_unstale":"2023-11-06T03:11:58.446604+0000","last_undegraded":"2023-11-06T03:11:58.446604+0000","last_fullsized":"2023-11-06T03:11:58.446604+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:47.016990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.438867+0000","last_change":"2023-11-06T03:11:58.098171+0000","last_active":"2023-11-06T03:11:58.438867+0000","last_peered":"2023-11-06T03:11:58.438867+0000","last_clean":"2023-11-06T03:11:58.438867+0000","last_became_active":"2023-11-06T03:11:58.097771+0000","last_became_peered":"2023-11-06T03:11:58.097771+0000","last_unstale":"2023-11-06T03:11:58.438867+0000","last_undegraded":"2023-11-06T03:11:58.438867+0000","last_fullsized":"2023-11-06T03:11:58.438867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:27.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.457191+0000","last_change":"2023-11-06T03:11:58.456776+0000","last_active":"2023-11-06T03:11:58.457191+0000","last_peered":"2023-11-06T03:11:58.457191+0000","last_clean":"2023-11-06T03:11:58.457191+0000","last_became_active":"2023-11-06T03:11:58.456337+0000","last_became_peered":"2023-11-06T03:11:58.456337+0000","last_unstale":"2023-11-06T03:11:58.457191+0000","last_undegraded":"2023-11-06T03:11:58.457191+0000","last_fullsized":"2023-11-06T03:11:58.457191+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:47.117552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"98'8","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.454863+0000","last_change":"2023-11-06T03:11:58.087561+0000","last_active":"2023-11-06T03:11:58.454863+0000","last_peered":"2023-11-06T03:11:58.454863+0000","last_clean":"2023-11-06T03:11:58.454863+0000","last_became_active":"2023-11-06T03:11:58.087205+0000","last_became_peered":"2023-11-06T03:11:58.087205+0000","last_unstale":"2023-11-06T03:11:58.454863+0000","last_undegraded":"2023-11-06T03:11:58.454863+0000","last_fullsized":"2023-11-06T03:11:58.454863+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:24.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487119+0000","last_change":"2023-11-06T03:11:58.483414+0000","last_active":"2023-11-06T03:11:58.487119+0000","last_peered":"2023-11-06T03:11:58.487119+0000","last_clean":"2023-11-06T03:11:58.487119+0000","last_became_active":"2023-11-06T03:11:58.482503+0000","last_became_peered":"2023-11-06T03:11:58.482503+0000","last_unstale":"2023-11-06T03:11:58.487119+0000","last_undegraded":"2023-11-06T03:11:58.487119+0000","last_fullsized":"2023-11-06T03:11:58.487119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:25.713522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481892+0000","last_change":"2023-11-06T03:11:58.106262+0000","last_active":"2023-11-06T03:11:58.481892+0000","last_peered":"2023-11-06T03:11:58.481892+0000","last_clean":"2023-11-06T03:11:58.481892+0000","last_became_active":"2023-11-06T03:11:58.105477+0000","last_became_peered":"2023-11-06T03:11:58.105477+0000","last_unstale":"2023-11-06T03:11:58.481892+0000","last_undegraded":"2023-11-06T03:11:58.481892+0000","last_fullsized":"2023-11-06T03:11:58.481892+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:46:54.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.480876+0000","last_change":"2023-11-06T03:11:58.480318+0000","last_active":"2023-11-06T03:11:58.480876+0000","last_peered":"2023-11-06T03:11:58.480876+0000","last_clean":"2023-11-06T03:11:58.480876+0000","last_became_active":"2023-11-06T03:11:58.480065+0000","last_became_peered":"2023-11-06T03:11:58.480065+0000","last_unstale":"2023-11-06T03:11:58.480876+0000","last_undegraded":"2023-11-06T03:11:58.480876+0000","last_fullsized":"2023-11-06T03:11:58.480876+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:10:47.914931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.432440+0000","last_change":"2023-11-06T03:11:58.104867+0000","last_active":"2023-11-06T03:11:58.432440+0000","last_peered":"2023-11-06T03:11:58.432440+0000","last_clean":"2023-11-06T03:11:58.432440+0000","last_became_active":"2023-11-06T03:11:58.104076+0000","last_became_peered":"2023-11-06T03:11:58.104076+0000","last_unstale":"2023-11-06T03:11:58.432440+0000","last_undegraded":"2023-11-06T03:11:58.432440+0000","last_fullsized":"2023-11-06T03:11:58.432440+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:33.467802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456636+0000","last_change":"2023-11-06T03:11:58.453135+0000","last_active":"2023-11-06T03:11:58.456636+0000","last_peered":"2023-11-06T03:11:58.456636+0000","last_clean":"2023-11-06T03:11:58.456636+0000","last_became_active":"2023-11-06T03:11:58.452804+0000","last_became_peered":"2023-11-06T03:11:58.452804+0000","last_unstale":"2023-11-06T03:11:58.456636+0000","last_undegraded":"2023-11-06T03:11:58.456636+0000","last_fullsized":"2023-11-06T03:11:58.456636+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:45.415124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.437505+0000","last_change":"2023-11-06T03:11:58.087002+0000","last_active":"2023-11-06T03:11:58.437505+0000","last_peered":"2023-11-06T03:11:58.437505+0000","last_clean":"2023-11-06T03:11:58.437505+0000","last_became_active":"2023-11-06T03:11:58.085957+0000","last_became_peered":"2023-11-06T03:11:58.085957+0000","last_unstale":"2023-11-06T03:11:58.437505+0000","last_undegraded":"2023-11-06T03:11:58.437505+0000","last_fullsized":"2023-11-06T03:11:58.437505+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:24.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.486340+0000","last_change":"2023-11-06T03:11:58.480906+0000","last_active":"2023-11-06T03:11:58.486340+0000","last_peered":"2023-11-06T03:11:58.486340+0000","last_clean":"2023-11-06T03:11:58.486340+0000","last_became_active":"2023-11-06T03:11:58.480385+0000","last_became_peered":"2023-11-06T03:11:58.480385+0000","last_unstale":"2023-11-06T03:11:58.486340+0000","last_undegraded":"2023-11-06T03:11:58.486340+0000","last_fullsized":"2023-11-06T03:11:58.486340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455390+0000","last_change":"2023-11-06T03:11:58.090180+0000","last_active":"2023-11-06T03:11:58.455390+0000","last_peered":"2023-11-06T03:11:58.455390+0000","last_clean":"2023-11-06T03:11:58.455390+0000","last_became_active":"2023-11-06T03:11:58.089029+0000","last_became_peered":"2023-11-06T03:11:58.089029+0000","last_unstale":"2023-11-06T03:11:58.455390+0000","last_undegraded":"2023-11-06T03:11:58.455390+0000","last_fullsized":"2023-11-06T03:11:58.455390+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:57.997899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455261+0000","last_change":"2023-11-06T03:11:58.452958+0000","last_active":"2023-11-06T03:11:58.455261+0000","last_peered":"2023-11-06T03:11:58.455261+0000","last_clean":"2023-11-06T03:11:58.455261+0000","last_became_active":"2023-11-06T03:11:58.452396+0000","last_became_peered":"2023-11-06T03:11:58.452396+0000","last_unstale":"2023-11-06T03:11:58.455261+0000","last_undegraded":"2023-11-06T03:11:58.455261+0000","last_fullsized":"2023-11-06T03:11:58.455261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:20:53.997665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441064+0000","last_change":"2023-11-06T03:11:58.076687+0000","last_active":"2023-11-06T03:11:58.441064+0000","last_peered":"2023-11-06T03:11:58.441064+0000","last_clean":"2023-11-06T03:11:58.441064+0000","last_became_active":"2023-11-06T03:11:58.076316+0000","last_became_peered":"2023-11-06T03:11:58.076316+0000","last_unstale":"2023-11-06T03:11:58.441064+0000","last_undegraded":"2023-11-06T03:11:58.441064+0000","last_fullsized":"2023-11-06T03:11:58.441064+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:04.021850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451519+0000","last_change":"2023-11-06T03:11:58.449252+0000","last_active":"2023-11-06T03:11:58.451519+0000","last_peered":"2023-11-06T03:11:58.451519+0000","last_clean":"2023-11-06T03:11:58.451519+0000","last_became_active":"2023-11-06T03:11:58.448894+0000","last_became_peered":"2023-11-06T03:11:58.448894+0000","last_unstale":"2023-11-06T03:11:58.451519+0000","last_undegraded":"2023-11-06T03:11:58.451519+0000","last_fullsized":"2023-11-06T03:11:58.451519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:43.895470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.452055+0000","last_change":"2023-11-06T03:11:58.096435+0000","last_active":"2023-11-06T03:11:58.452055+0000","last_peered":"2023-11-06T03:11:58.452055+0000","last_clean":"2023-11-06T03:11:58.452055+0000","last_became_active":"2023-11-06T03:11:58.095759+0000","last_became_peered":"2023-11-06T03:11:58.095759+0000","last_unstale":"2023-11-06T03:11:58.452055+0000","last_undegraded":"2023-11-06T03:11:58.452055+0000","last_fullsized":"2023-11-06T03:11:58.452055+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:30.026434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.472122+0000","last_change":"2023-11-06T03:11:58.466913+0000","last_active":"2023-11-06T03:11:58.472122+0000","last_peered":"2023-11-06T03:11:58.472122+0000","last_clean":"2023-11-06T03:11:58.472122+0000","last_became_active":"2023-11-06T03:11:58.466117+0000","last_became_peered":"2023-11-06T03:11:58.466117+0000","last_unstale":"2023-11-06T03:11:58.472122+0000","last_undegraded":"2023-11-06T03:11:58.472122+0000","last_fullsized":"2023-11-06T03:11:58.472122+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:20.817896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.447234+0000","last_change":"2023-11-06T03:11:58.088798+0000","last_active":"2023-11-06T03:11:58.447234+0000","last_peered":"2023-11-06T03:11:58.447234+0000","last_clean":"2023-11-06T03:11:58.447234+0000","last_became_active":"2023-11-06T03:11:58.088073+0000","last_became_peered":"2023-11-06T03:11:58.088073+0000","last_unstale":"2023-11-06T03:11:58.447234+0000","last_undegraded":"2023-11-06T03:11:58.447234+0000","last_fullsized":"2023-11-06T03:11:58.447234+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:04.000264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451766+0000","last_change":"2023-11-06T03:11:58.446348+0000","last_active":"2023-11-06T03:11:58.451766+0000","last_peered":"2023-11-06T03:11:58.451766+0000","last_clean":"2023-11-06T03:11:58.451766+0000","last_became_active":"2023-11-06T03:11:58.446070+0000","last_became_peered":"2023-11-06T03:11:58.446070+0000","last_unstale":"2023-11-06T03:11:58.451766+0000","last_undegraded":"2023-11-06T03:11:58.451766+0000","last_fullsized":"2023-11-06T03:11:58.451766+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:25.663554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458552+0000","last_change":"2023-11-06T03:11:58.102871+0000","last_active":"2023-11-06T03:11:58.458552+0000","last_peered":"2023-11-06T03:11:58.458552+0000","last_clean":"2023-11-06T03:11:58.458552+0000","last_became_active":"2023-11-06T03:11:58.101984+0000","last_became_peered":"2023-11-06T03:11:58.101984+0000","last_unstale":"2023-11-06T03:11:58.458552+0000","last_undegraded":"2023-11-06T03:11:58.458552+0000","last_fullsized":"2023-11-06T03:11:58.458552+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:24.573628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487964+0000","last_change":"2023-11-06T03:11:58.482516+0000","last_active":"2023-11-06T03:11:58.487964+0000","last_peered":"2023-11-06T03:11:58.487964+0000","last_clean":"2023-11-06T03:11:58.487964+0000","last_became_active":"2023-11-06T03:11:58.481941+0000","last_became_peered":"2023-11-06T03:11:58.481941+0000","last_unstale":"2023-11-06T03:11:58.487964+0000","last_undegraded":"2023-11-06T03:11:58.487964+0000","last_fullsized":"2023-11-06T03:11:58.487964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.438940+0000","last_change":"2023-11-06T03:11:58.102861+0000","last_active":"2023-11-06T03:11:58.438940+0000","last_peered":"2023-11-06T03:11:58.438940+0000","last_clean":"2023-11-06T03:11:58.438940+0000","last_became_active":"2023-11-06T03:11:58.102552+0000","last_became_peered":"2023-11-06T03:11:58.102552+0000","last_unstale":"2023-11-06T03:11:58.438940+0000","last_undegraded":"2023-11-06T03:11:58.438940+0000","last_fullsized":"2023-11-06T03:11:58.438940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:13.672419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461769+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:11:58.461769+0000","last_peered":"2023-11-06T03:11:58.461769+0000","last_clean":"2023-11-06T03:11:58.461769+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:11:58.461769+0000","last_undegraded":"2023-11-06T03:11:58.461769+0000","last_fullsized":"2023-11-06T03:11:58.461769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.085087+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:11:58.085087+0000","last_peered":"2023-11-06T03:11:58.085087+0000","last_clean":"2023-11-06T03:11:58.085087+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:11:58.085087+0000","last_undegraded":"2023-11-06T03:11:58.085087+0000","last_fullsized":"2023-11-06T03:11:58.085087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479732+0000","last_change":"2023-11-06T03:11:58.479363+0000","last_active":"2023-11-06T03:11:58.479732+0000","last_peered":"2023-11-06T03:11:58.479732+0000","last_clean":"2023-11-06T03:11:58.479732+0000","last_became_active":"2023-11-06T03:11:58.479129+0000","last_became_peered":"2023-11-06T03:11:58.479129+0000","last_unstale":"2023-11-06T03:11:58.479732+0000","last_undegraded":"2023-11-06T03:11:58.479732+0000","last_fullsized":"2023-11-06T03:11:58.479732+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:46:39.813525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.485966+0000","last_change":"2023-11-06T03:11:58.096608+0000","last_active":"2023-11-06T03:11:58.485966+0000","last_peered":"2023-11-06T03:11:58.485966+0000","last_clean":"2023-11-06T03:11:58.485966+0000","last_became_active":"2023-11-06T03:11:58.095377+0000","last_became_peered":"2023-11-06T03:11:58.095377+0000","last_unstale":"2023-11-06T03:11:58.485966+0000","last_undegraded":"2023-11-06T03:11:58.485966+0000","last_fullsized":"2023-11-06T03:11:58.485966+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:54.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.086420+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:11:58.086420+0000","last_peered":"2023-11-06T03:11:58.086420+0000","last_clean":"2023-11-06T03:11:58.086420+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:11:58.086420+0000","last_undegraded":"2023-11-06T03:11:58.086420+0000","last_fullsized":"2023-11-06T03:11:58.086420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.467809+0000","last_change":"2023-11-06T03:11:58.460934+0000","last_active":"2023-11-06T03:11:58.467809+0000","last_peered":"2023-11-06T03:11:58.467809+0000","last_clean":"2023-11-06T03:11:58.467809+0000","last_became_active":"2023-11-06T03:11:58.460243+0000","last_became_peered":"2023-11-06T03:11:58.460243+0000","last_unstale":"2023-11-06T03:11:58.467809+0000","last_undegraded":"2023-11-06T03:11:58.467809+0000","last_fullsized":"2023-11-06T03:11:58.467809+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:32.902273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481699+0000","last_change":"2023-11-06T03:11:58.108134+0000","last_active":"2023-11-06T03:11:58.481699+0000","last_peered":"2023-11-06T03:11:58.481699+0000","last_clean":"2023-11-06T03:11:58.481699+0000","last_became_active":"2023-11-06T03:11:58.107476+0000","last_became_peered":"2023-11-06T03:11:58.107476+0000","last_unstale":"2023-11-06T03:11:58.481699+0000","last_undegraded":"2023-11-06T03:11:58.481699+0000","last_fullsized":"2023-11-06T03:11:58.481699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:30.026434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.079840+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:11:58.079840+0000","last_peered":"2023-11-06T03:11:58.079840+0000","last_clean":"2023-11-06T03:11:58.079840+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:11:58.079840+0000","last_undegraded":"2023-11-06T03:11:58.079840+0000","last_fullsized":"2023-11-06T03:11:58.079840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455648+0000","last_change":"2023-11-06T03:11:58.442599+0000","last_active":"2023-11-06T03:11:58.455648+0000","last_peered":"2023-11-06T03:11:58.455648+0000","last_clean":"2023-11-06T03:11:58.455648+0000","last_became_active":"2023-11-06T03:11:58.441817+0000","last_became_peered":"2023-11-06T03:11:58.441817+0000","last_unstale":"2023-11-06T03:11:58.455648+0000","last_undegraded":"2023-11-06T03:11:58.455648+0000","last_fullsized":"2023-11-06T03:11:58.455648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461142+0000","last_change":"2023-11-06T03:11:58.084411+0000","last_active":"2023-11-06T03:11:58.461142+0000","last_peered":"2023-11-06T03:11:58.461142+0000","last_clean":"2023-11-06T03:11:58.461142+0000","last_became_active":"2023-11-06T03:11:58.083771+0000","last_became_peered":"2023-11-06T03:11:58.083771+0000","last_unstale":"2023-11-06T03:11:58.461142+0000","last_undegraded":"2023-11-06T03:11:58.461142+0000","last_fullsized":"2023-11-06T03:11:58.461142+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:49.187169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.085402+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:11:58.085402+0000","last_peered":"2023-11-06T03:11:58.085402+0000","last_clean":"2023-11-06T03:11:58.085402+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:11:58.085402+0000","last_undegraded":"2023-11-06T03:11:58.085402+0000","last_fullsized":"2023-11-06T03:11:58.085402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459664+0000","last_change":"2023-11-06T03:11:58.458773+0000","last_active":"2023-11-06T03:11:58.459664+0000","last_peered":"2023-11-06T03:11:58.459664+0000","last_clean":"2023-11-06T03:11:58.459664+0000","last_became_active":"2023-11-06T03:11:58.458482+0000","last_became_peered":"2023-11-06T03:11:58.458482+0000","last_unstale":"2023-11-06T03:11:58.459664+0000","last_undegraded":"2023-11-06T03:11:58.459664+0000","last_fullsized":"2023-11-06T03:11:58.459664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:49.067646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.465819+0000","last_change":"2023-11-06T03:11:58.095533+0000","last_active":"2023-11-06T03:11:58.465819+0000","last_peered":"2023-11-06T03:11:58.465819+0000","last_clean":"2023-11-06T03:11:58.465819+0000","last_became_active":"2023-11-06T03:11:58.086918+0000","last_became_peered":"2023-11-06T03:11:58.086918+0000","last_unstale":"2023-11-06T03:11:58.465819+0000","last_undegraded":"2023-11-06T03:11:58.465819+0000","last_fullsized":"2023-11-06T03:11:58.465819+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:40.038408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.080804+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:11:58.080804+0000","last_peered":"2023-11-06T03:11:58.080804+0000","last_clean":"2023-11-06T03:11:58.080804+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:11:58.080804+0000","last_undegraded":"2023-11-06T03:11:58.080804+0000","last_fullsized":"2023-11-06T03:11:58.080804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479231+0000","last_change":"2023-11-06T03:11:58.466155+0000","last_active":"2023-11-06T03:11:58.479231+0000","last_peered":"2023-11-06T03:11:58.479231+0000","last_clean":"2023-11-06T03:11:58.479231+0000","last_became_active":"2023-11-06T03:11:58.464064+0000","last_became_peered":"2023-11-06T03:11:58.464064+0000","last_unstale":"2023-11-06T03:11:58.479231+0000","last_undegraded":"2023-11-06T03:11:58.479231+0000","last_fullsized":"2023-11-06T03:11:58.479231+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.435931+0000","last_change":"2023-11-06T03:11:58.084595+0000","last_active":"2023-11-06T03:11:58.435931+0000","last_peered":"2023-11-06T03:11:58.435931+0000","last_clean":"2023-11-06T03:11:58.435931+0000","last_became_active":"2023-11-06T03:11:58.084217+0000","last_became_peered":"2023-11-06T03:11:58.084217+0000","last_unstale":"2023-11-06T03:11:58.435931+0000","last_undegraded":"2023-11-06T03:11:58.435931+0000","last_fullsized":"2023-11-06T03:11:58.435931+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:43:58.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.086562+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:11:58.086562+0000","last_peered":"2023-11-06T03:11:58.086562+0000","last_clean":"2023-11-06T03:11:58.086562+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:11:58.086562+0000","last_undegraded":"2023-11-06T03:11:58.086562+0000","last_fullsized":"2023-11-06T03:11:58.086562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469703+0000","last_change":"2023-11-06T03:11:58.462247+0000","last_active":"2023-11-06T03:11:58.469703+0000","last_peered":"2023-11-06T03:11:58.469703+0000","last_clean":"2023-11-06T03:11:58.469703+0000","last_became_active":"2023-11-06T03:11:58.461382+0000","last_became_peered":"2023-11-06T03:11:58.461382+0000","last_unstale":"2023-11-06T03:11:58.469703+0000","last_undegraded":"2023-11-06T03:11:58.469703+0000","last_fullsized":"2023-11-06T03:11:58.469703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451698+0000","last_change":"2023-11-06T03:11:58.105422+0000","last_active":"2023-11-06T03:11:58.451698+0000","last_peered":"2023-11-06T03:11:58.451698+0000","last_clean":"2023-11-06T03:11:58.451698+0000","last_became_active":"2023-11-06T03:11:58.102932+0000","last_became_peered":"2023-11-06T03:11:58.102932+0000","last_unstale":"2023-11-06T03:11:58.451698+0000","last_undegraded":"2023-11-06T03:11:58.451698+0000","last_fullsized":"2023-11-06T03:11:58.451698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:50:31.695545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.080874+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:11:58.080874+0000","last_peered":"2023-11-06T03:11:58.080874+0000","last_clean":"2023-11-06T03:11:58.080874+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:11:58.080874+0000","last_undegraded":"2023-11-06T03:11:58.080874+0000","last_fullsized":"2023-11-06T03:11:58.080874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451789+0000","last_change":"2023-11-06T03:11:58.446577+0000","last_active":"2023-11-06T03:11:58.451789+0000","last_peered":"2023-11-06T03:11:58.451789+0000","last_clean":"2023-11-06T03:11:58.451789+0000","last_became_active":"2023-11-06T03:11:58.441340+0000","last_became_peered":"2023-11-06T03:11:58.441340+0000","last_unstale":"2023-11-06T03:11:58.451789+0000","last_undegraded":"2023-11-06T03:11:58.451789+0000","last_fullsized":"2023-11-06T03:11:58.451789+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:43.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.432802+0000","last_change":"2023-11-06T03:11:58.095175+0000","last_active":"2023-11-06T03:11:58.432802+0000","last_peered":"2023-11-06T03:11:58.432802+0000","last_clean":"2023-11-06T03:11:58.432802+0000","last_became_active":"2023-11-06T03:11:58.094916+0000","last_became_peered":"2023-11-06T03:11:58.094916+0000","last_unstale":"2023-11-06T03:11:58.432802+0000","last_undegraded":"2023-11-06T03:11:58.432802+0000","last_fullsized":"2023-11-06T03:11:58.432802+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:24:53.783349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.079467+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:11:58.079467+0000","last_peered":"2023-11-06T03:11:58.079467+0000","last_clean":"2023-11-06T03:11:58.079467+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:11:58.079467+0000","last_undegraded":"2023-11-06T03:11:58.079467+0000","last_fullsized":"2023-11-06T03:11:58.079467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86872,"kb_used_data":1744,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284968,"statfs":{"total":96636764160,"available":96547807232,"internally_reserved":0,"allocated":1785856,"data_stored":1135348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.53200000000000003,"5min":0.46300000000000002,"15min":0.42599999999999999},"max":{"1min":1.131,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81499999999999995,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.202,"5min":1.367,"15min":1.397},"last":0.93400000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.85199999999999998},"min":{"1min":0.56000000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0700000000000001,"5min":1.254,"15min":1.444},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.56299999999999994,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.91400000000000003,"15min":0.88400000000000001},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1559999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.88500000000000001,"15min":0.94299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.272,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.76200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.65000000000000002,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.046,"5min":2.512,"15min":2.512},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.135,"15min":1.0620000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60099999999999998},"max":{"1min":1.21,"5min":10.596,"15min":10.596},"last":0.60899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.94699999999999995,"15min":0.97599999999999998},"min":{"1min":0.65900000000000003,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.268,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.032},{"interface":"front","average":{"1min":1.05,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.84499999999999997,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.3480000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0629999999999999,"15min":1.052},"min":{"1min":0.85299999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2689999999999999,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.071,"15min":1.0920000000000001},"min":{"1min":0.76100000000000001,"5min":0.66800000000000004,"15min":0.55200000000000005},"max":{"1min":1.407,"5min":1.476,"15min":1.8839999999999999},"last":1.0820000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1519999999999999,"15min":1.159},"min":{"1min":0.94399999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.472,"5min":2.548,"15min":2.548},"last":1.127},{"interface":"front","average":{"1min":1.123,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.98999999999999999,"5min":0.74299999999999999,"15min":0.53300000000000003},"max":{"1min":1.367,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.107}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76464,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":75264,"kb_avail":94295376,"statfs":{"total":96636764160,"available":96558465024,"internally_reserved":0,"allocated":1220608,"data_stored":553257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77070336},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.97199999999999998,"15min":0.84799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":2.3279999999999998,"5min":16.073,"15min":16.073},"last":0.745},{"interface":"front","average":{"1min":0.92300000000000004,"5min":1.2889999999999999,"15min":0.97999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":4.4000000000000004,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.054,"15min":0.89800000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6359999999999999,"5min":16.012,"15min":16.012},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90000000000000002,"15min":0.879},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.1839999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.1200000000000001,"15min":0.90700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.744,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.95699999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.264,"5min":2.843,"15min":2.843},"last":0.873}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.625,"15min":1.1890000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.1519999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.3899999999999999,"15min":1.0529999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.139,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3440000000000001,"15min":1.1990000000000001},"min":{"1min":0.75800000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.4980000000000002,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.097},{"interface":"front","average":{"1min":1.363,"5min":1.409,"15min":1.2470000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.3970000000000002,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.173}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.4319999999999999,"15min":1.1699999999999999},"min":{"1min":0.64400000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4450000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.05},{"interface":"front","average":{"1min":1.125,"5min":1.159,"15min":1.119},"min":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.524,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.022}]},{"osd":7,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.4079999999999999,"15min":1.2070000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.6399999999999997,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.4219999999999999,"15min":1.2190000000000001},"min":{"1min":0.84399999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.538,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.92800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58464,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313376,"statfs":{"total":96636764160,"available":96576897024,"internally_reserved":0,"allocated":1204224,"data_stored":552108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":5.8019999999999996,"5min":1.792,"15min":1.1399999999999999},"min":{"1min":0.441,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":125.256,"5min":125.256,"15min":125.256},"last":0.83199999999999996},{"interface":"front","average":{"1min":5.8789999999999996,"5min":1.835,"15min":1.167},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":126.98699999999999,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.85699999999999998,"15min":0.76100000000000001},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.339,"5min":2.339,"15min":2.339},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.84999999999999998,"15min":0.83999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.90500000000000003,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.69199999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.004}]},{"osd":4,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.55700000000000005,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1,"15min":0.97199999999999998},"min":{"1min":0.64300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.4560000000000004,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.036,"15min":1.0069999999999999},"min":{"1min":0.78300000000000003,"5min":0.44,"15min":0.378},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0580000000000001,"15min":1.0249999999999999},"min":{"1min":0.755,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.03}]},{"osd":7,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0740000000000001,"15min":1.0089999999999999},"min":{"1min":0.75700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.089},"min":{"1min":0.85299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.157}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86312,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94285528,"statfs":{"total":96636764160,"available":96548380672,"internally_reserved":0,"allocated":1212416,"data_stored":552268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":3.121,"5min":1.2490000000000001,"15min":0.96899999999999997},"min":{"1min":0.51100000000000001,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":42.853999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":3.3340000000000001,"5min":1.306,"15min":0.96099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.41599999999999998},"max":{"1min":44.609000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":3.0830000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78600000000000003,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.1259999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.85799999999999998,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.026,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":1.008},"min":{"1min":0.66700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.123,"5min":1.5820000000000001,"15min":3.2879999999999998},"last":1.452},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.85299999999999998,"15min":0.84999999999999998},"min":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.45900000000000002},"max":{"1min":1.0549999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":3.2639999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.93300000000000005},"min":{"1min":0.57899999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.129,"5min":1.482,"15min":1.6779999999999999},"last":3.198},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.90300000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.452,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.87}]},{"osd":5,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.99199999999999999,"15min":1.0089999999999999},"min":{"1min":0.82899999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.28,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":10.192},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.052,"15min":1.095},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.4339999999999999,"15min":1.9159999999999999},"last":5.3310000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.96499999999999997,"15min":1.0089999999999999},"min":{"1min":0.64800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0609999999999999,"15min":1.1319999999999999},"min":{"1min":0.67600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.226,"5min":1.8120000000000001,"15min":2.6549999999999998},"last":1.506}]},{"osd":7,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.081,"15min":1.075},"min":{"1min":0.82299999999999995,"5min":0.628,"15min":0.628},"max":{"1min":1.345,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":3.3490000000000002},{"interface":"front","average":{"1min":1.099,"5min":1.2310000000000001,"15min":1.181},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.625,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.0069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72412,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94299428,"statfs":{"total":96636764160,"available":96562614272,"internally_reserved":0,"allocated":1200128,"data_stored":551089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.25,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.113,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.84799999999999998,"5min":1.151,"15min":1.524},"last":0.495}]},{"osd":1,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.66100000000000003,"15min":0.62},"min":{"1min":0.28299999999999997,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":1.371,"15min":1.371},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.309,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71299999999999997,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.1970000000000001,"5min":1.284,"15min":1.284},"last":0.57399999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.85399999999999998,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.52700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.52,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.1870000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.94799999999999995,"15min":0.97899999999999998},"min":{"1min":0.57799999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5,"5min":1.536,"15min":1.6639999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":1.01},"min":{"1min":0.58899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.27,"5min":1.5049999999999999,"15min":1.698},"last":0.85299999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97499999999999998,"15min":1.002},"min":{"1min":0.57399999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3620000000000001,"5min":1.649,"15min":1.8620000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.57699999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.748,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.0409999999999999},"min":{"1min":0.55000000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.079,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4119999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.0960000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46176,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325664,"statfs":{"total":96636764160,"available":96589479936,"internally_reserved":0,"allocated":1204224,"data_stored":552108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.189,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.89100000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.437},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":1.1140000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.121},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.65500000000000003},"min":{"1min":0.252,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.871,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.307,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.161,"5min":1.294,"15min":1.294},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.73199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.103,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.244,"5min":1.526,"15min":1.526},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.86599999999999999,"15min":0.93300000000000005},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.329,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.874},"min":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.252,"15min":1.6200000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.95499999999999996},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4359999999999999,"5min":1.5509999999999999,"15min":1.5780000000000001},"last":0.96799999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99099999999999999,"15min":0.98799999999999999},"min":{"1min":0.66500000000000004,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.2490000000000001,"5min":1.615,"15min":1.615},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.93400000000000005,"15min":0.94499999999999995},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":0.98999999999999999},"min":{"1min":0.54200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.4039999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90700000000000003,"15min":0.94999999999999996},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2569999999999999,"5min":1.3540000000000001,"15min":3.4740000000000002},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313372,"statfs":{"total":96636764160,"available":96576892928,"internally_reserved":0,"allocated":1208320,"data_stored":552178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.28499999999999998,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":0.97699999999999998,"5min":1.843,"15min":1.843},"last":0.374},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.2470000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.29799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.71599999999999997},"min":{"1min":0.32900000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.98999999999999999,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63300000000000001,"15min":0.67100000000000004},"min":{"1min":0.377,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1100000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.71699999999999997},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.012,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.71399999999999997},"min":{"1min":0.30499999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.032,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.252,"5min":2.258,"15min":2.258},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.0449999999999999,"15min":0.94499999999999995},"min":{"1min":0.63300000000000001,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3600000000000001,"5min":14.753,"15min":14.753},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.3819999999999999,"15min":1.1240000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.528,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.3680000000000001,"15min":1.149},"min":{"1min":0.73299999999999998,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.3979999999999999,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.1679999999999999,"15min":1.0600000000000001},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.5189999999999999,"5min":23.971,"15min":23.971},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.1759999999999999,"15min":1.087},"min":{"1min":0.69599999999999995,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.4299999999999999,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.995,"15min":1.056},"min":{"1min":0.76400000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4890000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.002,"5min":1.014,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4419999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68920,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":67136,"kb_avail":94302920,"statfs":{"total":96636764160,"available":96566190080,"internally_reserved":0,"allocated":1818624,"data_stored":1143727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68747264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.98299999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.65000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94699999999999995,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.095,"5min":1.155,"15min":1.155},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.147,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.63},"min":{"1min":0.29899999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2909999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.95399999999999996,"15min":0.92200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.306,"5min":4.641,"15min":4.641},"last":0.872},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.91800000000000004},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":2.028,"15min":2.028},"last":0.93200000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":0.97999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.4890000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.98499999999999999,"15min":0.94599999999999995},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":1.3779999999999999,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.91300000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93799999999999994,"15min":0.90700000000000003},"min":{"1min":0.52600000000000002,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2549999999999999,"5min":2.931,"15min":2.931},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.022,"15min":1.0109999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5760000000000001,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0269999999999999,"15min":1.0049999999999999},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.3939999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.998,"15min":0.99099999999999999},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4670000000000001,"5min":1.97,"15min":1.97},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:01.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:01.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.614+0000 7f0ee2f40640 1 Processor -- start 2023-11-06T03:12:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.614+0000 7f0ee2f40640 1 -- start start 2023-11-06T03:12:01.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee2f40640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0edc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee2f40640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc15f580 con 0x7f0edc15ea60 2023-11-06T03:12:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee2f40640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc15efc0 con 0x7f0edc1551e0 2023-11-06T03:12:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee2f40640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0704b0 con 0x7f0edc15e5e0 2023-11-06T03:12:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee14dd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0edc15ea60 0x7f0edc15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44234/0 (socket says 172.21.15.157:44234) 2023-11-06T03:12:01.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee14dd640 1 -- 172.21.15.157:0/4007600292 learned_addr learned my addr 172.21.15.157:0/4007600292 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0edc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471887204 0 0) 0x7f0edc15f580 con 0x7f0edc15ea60 2023-11-06T03:12:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ec4003530 con 0x7f0edc15ea60 2023-11-06T03:12:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289973073 0 0) 0x7f0ec4003530 con 0x7f0edc15ea60 2023-11-06T03:12:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 msgr2=0x7f0edc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0edc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 >> v1:172.21.15.195:6789/0 conn(0x7f0edc1551e0 legacy=0x7f0edc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc15f2a0 con 0x7f0edc15ea60 2023-11-06T03:12:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0ed4001680 con 0x7f0edc15ea60 2023-11-06T03:12:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 >> v1:172.21.15.157:6789/0 conn(0x7f0edc15ea60 legacy=0x7f0edc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0ec40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec4003f30 0x7f0ec40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec4004d10 0x7f0ec4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0704b0 con 0x7f0ec4003f30 2023-11-06T03:12:01.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec400bb20 con 0x7f0ec4004d10 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec400bce0 con 0x7f0edc15e5e0 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec4003f30 0x7f0ec40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee0cdc640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0ec40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee14dd640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec4004d10 0x7f0ec4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 -- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 msgr2=0x7f0ec40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0edc15e5e0 0x7f0ec40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 -- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec4004d10 msgr2=0x7f0ec4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec4004d10 0x7f0ec4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 -- 172.21.15.157:0/4007600292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4003530 con 0x7f0ec4003f30 2023-11-06T03:12:01.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0ee14dd640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ec4004d10 0x7f0ec4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edbfff640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec4003f30 0x7f0ec40047d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009ae0 tx=0x7f0ed001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3a27af69210829 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00a6510 con 0x7f0ec4003f30 2023-11-06T03:12:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed00a6670 con 0x7f0ec4003f30 2023-11-06T03:12:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.618+0000 7f0edb7fe640 1 -- 172.21.15.157:0/4007600292 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00a6a20 con 0x7f0ec4003f30 2023-11-06T03:12:01.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/4007600292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec4003f30 msgr2=0x7f0ec40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/4007600292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ec4003f30 0x7f0ec40047d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009ae0 tx=0x7f0ed001ca20 comp rx=0 tx=0).stop 2023-11-06T03:12:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 -- 172.21.15.157:0/4007600292 reap_dead start 2023-11-06T03:12:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/4007600292 shutdown_connections 2023-11-06T03:12:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/4007600292 >> 172.21.15.157:0/4007600292 conn(0x7f0edc065930 msgr2=0x7f0edc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/4007600292 shutdown_connections 2023-11-06T03:12:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/4007600292 wait complete. 2023-11-06T03:12:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 Processor -- start 2023-11-06T03:12:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- start start 2023-11-06T03:12:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 0x7f0edc159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 0x7f0edc145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc155dc0 con 0x7f0edc155600 2023-11-06T03:12:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc155ae0 con 0x7f0edc1f7e00 2023-11-06T03:12:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1f7320 con 0x7f0ec4003f30 2023-11-06T03:12:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44242/0 (socket says 172.21.15.157:44242) 2023-11-06T03:12:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 -- 172.21.15.157:0/1189514179 learned_addr learned my addr 172.21.15.157:0/1189514179 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee0cdc640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 0x7f0edc159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0edbfff640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 0x7f0edc145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 -- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 msgr2=0x7f0edc159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 0x7f0edc159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 -- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 msgr2=0x7f0edc145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 0x7f0edc145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 -- 172.21.15.157:0/1189514179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc1f7600 con 0x7f0edc155600 2023-11-06T03:12:01.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee0cdc640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 0x7f0edc159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:01.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0edbfff640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 0x7f0edc145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:01.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee14dd640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4001210 tx=0x7f0ed400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4007230 con 0x7f0edc155600 2023-11-06T03:12:01.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed40073b0 con 0x7f0edc155600 2023-11-06T03:12:01.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc1f78e0 con 0x7f0edc155600 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40047f0 con 0x7f0edc155600 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.622+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc153bb0 con 0x7f0edc155600 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc1f7180 con 0x7f0edc155600 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0ed4004950 con 0x7f0edc155600 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ed97fa640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 0x7f0ec8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ed4082a00 con 0x7f0edc155600 2023-11-06T03:12:01.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ee0cdc640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 0x7f0ec8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.626+0000 7f0ee0cdc640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 0x7f0ec8067f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc001dd0 tx=0x7f0ecc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.630+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0edc1f7180 con 0x7f0edc155600 2023-11-06T03:12:01.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.774+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0edc166980 con 0x7f0ec8065a60 2023-11-06T03:12:01.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.798+0000 7f0ed97fa640 1 -- 172.21.15.157:0/1189514179 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310281 (secure 0 0 0) 0x7f0ecc00d050 con 0x7f0ec8065a60 2023-11-06T03:12:01.798 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:01.802 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:01.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 msgr2=0x7f0ec8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 0x7f0ec8067f20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc001dd0 tx=0x7f0ecc009040 comp rx=0 tx=0).stop 2023-11-06T03:12:01.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 msgr2=0x7f0edc154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4001210 tx=0x7f0ed400dc00 comp rx=0 tx=0).stop 2023-11-06T03:12:01.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 shutdown_connections 2023-11-06T03:12:01.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec8065a60 0x7f0ec8067f20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0edc155600 0x7f0edc154100 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0edc1f7e00 0x7f0edc145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 --2- 172.21.15.157:0/1189514179 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ec4003f30 0x7f0edc159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 >> 172.21.15.157:0/1189514179 conn(0x7f0edc065930 msgr2=0x7f0edc0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 shutdown_connections 2023-11-06T03:12:01.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.802+0000 7f0ee2f40640 1 -- 172.21.15.157:0/1189514179 wait complete. 2023-11-06T03:12:01.818 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2023-11-06T03:12:01.268985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":554088,"kb_used_data":10600,"kb_used_omap":0,"kb_used_meta":543424,"kb_avail":754420632,"statfs":{"total":773094113280,"available":772526727168,"internally_reserved":0,"allocated":10854400,"data_stored":5592083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":556466176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.220806"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451606+0000","last_change":"2023-11-06T03:11:58.450170+0000","last_active":"2023-11-06T03:11:58.451606+0000","last_peered":"2023-11-06T03:11:58.451606+0000","last_clean":"2023-11-06T03:11:58.451606+0000","last_became_active":"2023-11-06T03:11:58.449877+0000","last_became_peered":"2023-11-06T03:11:58.449877+0000","last_unstale":"2023-11-06T03:11:58.451606+0000","last_undegraded":"2023-11-06T03:11:58.451606+0000","last_fullsized":"2023-11-06T03:11:58.451606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479188+0000","last_change":"2023-11-06T03:11:58.465598+0000","last_active":"2023-11-06T03:11:58.479188+0000","last_peered":"2023-11-06T03:11:58.479188+0000","last_clean":"2023-11-06T03:11:58.479188+0000","last_became_active":"2023-11-06T03:11:58.464776+0000","last_became_peered":"2023-11-06T03:11:58.464776+0000","last_unstale":"2023-11-06T03:11:58.479188+0000","last_undegraded":"2023-11-06T03:11:58.479188+0000","last_fullsized":"2023-11-06T03:11:58.479188+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.463767+0000","last_change":"2023-11-06T03:11:58.462034+0000","last_active":"2023-11-06T03:11:58.463767+0000","last_peered":"2023-11-06T03:11:58.463767+0000","last_clean":"2023-11-06T03:11:58.463767+0000","last_became_active":"2023-11-06T03:11:58.461710+0000","last_became_peered":"2023-11-06T03:11:58.461710+0000","last_unstale":"2023-11-06T03:11:58.463767+0000","last_undegraded":"2023-11-06T03:11:58.463767+0000","last_fullsized":"2023-11-06T03:11:58.463767+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469542+0000","last_change":"2023-11-06T03:11:58.469136+0000","last_active":"2023-11-06T03:11:58.469542+0000","last_peered":"2023-11-06T03:11:58.469542+0000","last_clean":"2023-11-06T03:11:58.469542+0000","last_became_active":"2023-11-06T03:11:58.468859+0000","last_became_peered":"2023-11-06T03:11:58.468859+0000","last_unstale":"2023-11-06T03:11:58.469542+0000","last_undegraded":"2023-11-06T03:11:58.469542+0000","last_fullsized":"2023-11-06T03:11:58.469542+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.472053+0000","last_change":"2023-11-06T03:11:58.456581+0000","last_active":"2023-11-06T03:11:58.472053+0000","last_peered":"2023-11-06T03:11:58.472053+0000","last_clean":"2023-11-06T03:11:58.472053+0000","last_became_active":"2023-11-06T03:11:58.455485+0000","last_became_peered":"2023-11-06T03:11:58.455485+0000","last_unstale":"2023-11-06T03:11:58.472053+0000","last_undegraded":"2023-11-06T03:11:58.472053+0000","last_fullsized":"2023-11-06T03:11:58.472053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:46.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458858+0000","last_change":"2023-11-06T03:11:58.457885+0000","last_active":"2023-11-06T03:11:58.458858+0000","last_peered":"2023-11-06T03:11:58.458858+0000","last_clean":"2023-11-06T03:11:58.458858+0000","last_became_active":"2023-11-06T03:11:58.457569+0000","last_became_peered":"2023-11-06T03:11:58.457569+0000","last_unstale":"2023-11-06T03:11:58.458858+0000","last_undegraded":"2023-11-06T03:11:58.458858+0000","last_fullsized":"2023-11-06T03:11:58.458858+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:12.578711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487895+0000","last_change":"2023-11-06T03:11:58.480941+0000","last_active":"2023-11-06T03:11:58.487895+0000","last_peered":"2023-11-06T03:11:58.487895+0000","last_clean":"2023-11-06T03:11:58.487895+0000","last_became_active":"2023-11-06T03:11:58.480329+0000","last_became_peered":"2023-11-06T03:11:58.480329+0000","last_unstale":"2023-11-06T03:11:58.487895+0000","last_undegraded":"2023-11-06T03:11:58.487895+0000","last_fullsized":"2023-11-06T03:11:58.487895+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.482119+0000","last_change":"2023-11-06T03:11:58.480840+0000","last_active":"2023-11-06T03:11:58.482119+0000","last_peered":"2023-11-06T03:11:58.482119+0000","last_clean":"2023-11-06T03:11:58.482119+0000","last_became_active":"2023-11-06T03:11:58.480577+0000","last_became_peered":"2023-11-06T03:11:58.480577+0000","last_unstale":"2023-11-06T03:11:58.482119+0000","last_undegraded":"2023-11-06T03:11:58.482119+0000","last_fullsized":"2023-11-06T03:11:58.482119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479777+0000","last_change":"2023-11-06T03:11:58.465137+0000","last_active":"2023-11-06T03:11:58.479777+0000","last_peered":"2023-11-06T03:11:58.479777+0000","last_clean":"2023-11-06T03:11:58.479777+0000","last_became_active":"2023-11-06T03:11:58.464402+0000","last_became_peered":"2023-11-06T03:11:58.464402+0000","last_unstale":"2023-11-06T03:11:58.479777+0000","last_undegraded":"2023-11-06T03:11:58.479777+0000","last_fullsized":"2023-11-06T03:11:58.479777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:52.571964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455229+0000","last_change":"2023-11-06T03:11:58.452750+0000","last_active":"2023-11-06T03:11:58.455229+0000","last_peered":"2023-11-06T03:11:58.455229+0000","last_clean":"2023-11-06T03:11:58.455229+0000","last_became_active":"2023-11-06T03:11:58.452367+0000","last_became_peered":"2023-11-06T03:11:58.452367+0000","last_unstale":"2023-11-06T03:11:58.455229+0000","last_undegraded":"2023-11-06T03:11:58.455229+0000","last_fullsized":"2023-11-06T03:11:58.455229+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:37.535125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.484212+0000","last_change":"2023-11-06T03:11:58.473516+0000","last_active":"2023-11-06T03:11:58.484212+0000","last_peered":"2023-11-06T03:11:58.484212+0000","last_clean":"2023-11-06T03:11:58.484212+0000","last_became_active":"2023-11-06T03:11:58.471349+0000","last_became_peered":"2023-11-06T03:11:58.471349+0000","last_unstale":"2023-11-06T03:11:58.484212+0000","last_undegraded":"2023-11-06T03:11:58.484212+0000","last_fullsized":"2023-11-06T03:11:58.484212+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.448531+0000","last_change":"2023-11-06T03:11:58.441775+0000","last_active":"2023-11-06T03:11:58.448531+0000","last_peered":"2023-11-06T03:11:58.448531+0000","last_clean":"2023-11-06T03:11:58.448531+0000","last_became_active":"2023-11-06T03:11:58.441507+0000","last_became_peered":"2023-11-06T03:11:58.441507+0000","last_unstale":"2023-11-06T03:11:58.448531+0000","last_undegraded":"2023-11-06T03:11:58.448531+0000","last_fullsized":"2023-11-06T03:11:58.448531+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:38:34.605702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.483799+0000","last_change":"2023-11-06T03:11:58.483415+0000","last_active":"2023-11-06T03:11:58.483799+0000","last_peered":"2023-11-06T03:11:58.483799+0000","last_clean":"2023-11-06T03:11:58.483799+0000","last_became_active":"2023-11-06T03:11:58.483167+0000","last_became_peered":"2023-11-06T03:11:58.483167+0000","last_unstale":"2023-11-06T03:11:58.483799+0000","last_undegraded":"2023-11-06T03:11:58.483799+0000","last_fullsized":"2023-11-06T03:11:58.483799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458485+0000","last_change":"2023-11-06T03:11:58.446077+0000","last_active":"2023-11-06T03:11:58.458485+0000","last_peered":"2023-11-06T03:11:58.458485+0000","last_clean":"2023-11-06T03:11:58.458485+0000","last_became_active":"2023-11-06T03:11:58.445332+0000","last_became_peered":"2023-11-06T03:11:58.445332+0000","last_unstale":"2023-11-06T03:11:58.458485+0000","last_undegraded":"2023-11-06T03:11:58.458485+0000","last_fullsized":"2023-11-06T03:11:58.458485+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456940+0000","last_change":"2023-11-06T03:11:58.448291+0000","last_active":"2023-11-06T03:11:58.456940+0000","last_peered":"2023-11-06T03:11:58.456940+0000","last_clean":"2023-11-06T03:11:58.456940+0000","last_became_active":"2023-11-06T03:11:58.447495+0000","last_became_peered":"2023-11-06T03:11:58.447495+0000","last_unstale":"2023-11-06T03:11:58.456940+0000","last_undegraded":"2023-11-06T03:11:58.456940+0000","last_fullsized":"2023-11-06T03:11:58.456940+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:24.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.447389+0000","last_change":"2023-11-06T03:11:58.445588+0000","last_active":"2023-11-06T03:11:58.447389+0000","last_peered":"2023-11-06T03:11:58.447389+0000","last_clean":"2023-11-06T03:11:58.447389+0000","last_became_active":"2023-11-06T03:11:58.445350+0000","last_became_peered":"2023-11-06T03:11:58.445350+0000","last_unstale":"2023-11-06T03:11:58.447389+0000","last_undegraded":"2023-11-06T03:11:58.447389+0000","last_fullsized":"2023-11-06T03:11:58.447389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:28.153196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456471+0000","last_change":"2023-11-06T03:11:58.452436+0000","last_active":"2023-11-06T03:11:58.456471+0000","last_peered":"2023-11-06T03:11:58.456471+0000","last_clean":"2023-11-06T03:11:58.456471+0000","last_became_active":"2023-11-06T03:11:58.451978+0000","last_became_peered":"2023-11-06T03:11:58.451978+0000","last_unstale":"2023-11-06T03:11:58.456471+0000","last_undegraded":"2023-11-06T03:11:58.456471+0000","last_fullsized":"2023-11-06T03:11:58.456471+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:42:26.239442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461390+0000","last_change":"2023-11-06T03:11:58.460226+0000","last_active":"2023-11-06T03:11:58.461390+0000","last_peered":"2023-11-06T03:11:58.461390+0000","last_clean":"2023-11-06T03:11:58.461390+0000","last_became_active":"2023-11-06T03:11:58.459919+0000","last_became_peered":"2023-11-06T03:11:58.459919+0000","last_unstale":"2023-11-06T03:11:58.461390+0000","last_undegraded":"2023-11-06T03:11:58.461390+0000","last_fullsized":"2023-11-06T03:11:58.461390+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:43:36.369931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459561+0000","last_change":"2023-11-06T03:11:58.448520+0000","last_active":"2023-11-06T03:11:58.459561+0000","last_peered":"2023-11-06T03:11:58.459561+0000","last_clean":"2023-11-06T03:11:58.459561+0000","last_became_active":"2023-11-06T03:11:58.447792+0000","last_became_peered":"2023-11-06T03:11:58.447792+0000","last_unstale":"2023-11-06T03:11:58.459561+0000","last_undegraded":"2023-11-06T03:11:58.459561+0000","last_fullsized":"2023-11-06T03:11:58.459561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456724+0000","last_change":"2023-11-06T03:11:58.447321+0000","last_active":"2023-11-06T03:11:58.456724+0000","last_peered":"2023-11-06T03:11:58.456724+0000","last_clean":"2023-11-06T03:11:58.456724+0000","last_became_active":"2023-11-06T03:11:58.447089+0000","last_became_peered":"2023-11-06T03:11:58.447089+0000","last_unstale":"2023-11-06T03:11:58.456724+0000","last_undegraded":"2023-11-06T03:11:58.456724+0000","last_fullsized":"2023-11-06T03:11:58.456724+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:24:02.000380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487611+0000","last_change":"2023-11-06T03:11:58.486085+0000","last_active":"2023-11-06T03:11:58.487611+0000","last_peered":"2023-11-06T03:11:58.487611+0000","last_clean":"2023-11-06T03:11:58.487611+0000","last_became_active":"2023-11-06T03:11:58.485220+0000","last_became_peered":"2023-11-06T03:11:58.485220+0000","last_unstale":"2023-11-06T03:11:58.487611+0000","last_undegraded":"2023-11-06T03:11:58.487611+0000","last_fullsized":"2023-11-06T03:11:58.487611+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:43:36.369931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455720+0000","last_change":"2023-11-06T03:11:58.445001+0000","last_active":"2023-11-06T03:11:58.455720+0000","last_peered":"2023-11-06T03:11:58.455720+0000","last_clean":"2023-11-06T03:11:58.455720+0000","last_became_active":"2023-11-06T03:11:58.444302+0000","last_became_peered":"2023-11-06T03:11:58.444302+0000","last_unstale":"2023-11-06T03:11:58.455720+0000","last_undegraded":"2023-11-06T03:11:58.455720+0000","last_fullsized":"2023-11-06T03:11:58.455720+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.476692+0000","last_change":"2023-11-06T03:11:58.471694+0000","last_active":"2023-11-06T03:11:58.476692+0000","last_peered":"2023-11-06T03:11:58.476692+0000","last_clean":"2023-11-06T03:11:58.476692+0000","last_became_active":"2023-11-06T03:11:58.470412+0000","last_became_peered":"2023-11-06T03:11:58.470412+0000","last_unstale":"2023-11-06T03:11:58.476692+0000","last_undegraded":"2023-11-06T03:11:58.476692+0000","last_fullsized":"2023-11-06T03:11:58.476692+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:28.854478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441664+0000","last_change":"2023-11-06T03:11:58.439042+0000","last_active":"2023-11-06T03:11:58.441664+0000","last_peered":"2023-11-06T03:11:58.441664+0000","last_clean":"2023-11-06T03:11:58.441664+0000","last_became_active":"2023-11-06T03:11:58.438732+0000","last_became_peered":"2023-11-06T03:11:58.438732+0000","last_unstale":"2023-11-06T03:11:58.441664+0000","last_undegraded":"2023-11-06T03:11:58.441664+0000","last_fullsized":"2023-11-06T03:11:58.441664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:57:52.571964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479454+0000","last_change":"2023-11-06T03:11:58.476845+0000","last_active":"2023-11-06T03:11:58.479454+0000","last_peered":"2023-11-06T03:11:58.479454+0000","last_clean":"2023-11-06T03:11:58.479454+0000","last_became_active":"2023-11-06T03:11:58.476204+0000","last_became_peered":"2023-11-06T03:11:58.476204+0000","last_unstale":"2023-11-06T03:11:58.479454+0000","last_undegraded":"2023-11-06T03:11:58.479454+0000","last_fullsized":"2023-11-06T03:11:58.479454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458778+0000","last_change":"2023-11-06T03:11:58.448344+0000","last_active":"2023-11-06T03:11:58.458778+0000","last_peered":"2023-11-06T03:11:58.458778+0000","last_clean":"2023-11-06T03:11:58.458778+0000","last_became_active":"2023-11-06T03:11:58.447628+0000","last_became_peered":"2023-11-06T03:11:58.447628+0000","last_unstale":"2023-11-06T03:11:58.458778+0000","last_undegraded":"2023-11-06T03:11:58.458778+0000","last_fullsized":"2023-11-06T03:11:58.458778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456041+0000","last_change":"2023-11-06T03:11:58.454421+0000","last_active":"2023-11-06T03:11:58.456041+0000","last_peered":"2023-11-06T03:11:58.456041+0000","last_clean":"2023-11-06T03:11:58.456041+0000","last_became_active":"2023-11-06T03:11:58.454108+0000","last_became_peered":"2023-11-06T03:11:58.454108+0000","last_unstale":"2023-11-06T03:11:58.456041+0000","last_undegraded":"2023-11-06T03:11:58.456041+0000","last_fullsized":"2023-11-06T03:11:58.456041+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:11.707978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.450035+0000","last_change":"2023-11-06T03:11:58.446365+0000","last_active":"2023-11-06T03:11:58.450035+0000","last_peered":"2023-11-06T03:11:58.450035+0000","last_clean":"2023-11-06T03:11:58.450035+0000","last_became_active":"2023-11-06T03:11:58.446139+0000","last_became_peered":"2023-11-06T03:11:58.446139+0000","last_unstale":"2023-11-06T03:11:58.450035+0000","last_undegraded":"2023-11-06T03:11:58.450035+0000","last_fullsized":"2023-11-06T03:11:58.450035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:31.039087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.488077+0000","last_change":"2023-11-06T03:11:58.484257+0000","last_active":"2023-11-06T03:11:58.488077+0000","last_peered":"2023-11-06T03:11:58.488077+0000","last_clean":"2023-11-06T03:11:58.488077+0000","last_became_active":"2023-11-06T03:11:58.481807+0000","last_became_peered":"2023-11-06T03:11:58.481807+0000","last_unstale":"2023-11-06T03:11:58.488077+0000","last_undegraded":"2023-11-06T03:11:58.488077+0000","last_fullsized":"2023-11-06T03:11:58.488077+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479743+0000","last_change":"2023-11-06T03:11:58.464986+0000","last_active":"2023-11-06T03:11:58.479743+0000","last_peered":"2023-11-06T03:11:58.479743+0000","last_clean":"2023-11-06T03:11:58.479743+0000","last_became_active":"2023-11-06T03:11:58.464259+0000","last_became_peered":"2023-11-06T03:11:58.464259+0000","last_unstale":"2023-11-06T03:11:58.479743+0000","last_undegraded":"2023-11-06T03:11:58.479743+0000","last_fullsized":"2023-11-06T03:11:58.479743+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469770+0000","last_change":"2023-11-06T03:11:58.460708+0000","last_active":"2023-11-06T03:11:58.469770+0000","last_peered":"2023-11-06T03:11:58.469770+0000","last_clean":"2023-11-06T03:11:58.469770+0000","last_became_active":"2023-11-06T03:11:58.459932+0000","last_became_peered":"2023-11-06T03:11:58.459932+0000","last_unstale":"2023-11-06T03:11:58.469770+0000","last_undegraded":"2023-11-06T03:11:58.469770+0000","last_fullsized":"2023-11-06T03:11:58.469770+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:25.119548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456494+0000","last_change":"2023-11-06T03:11:58.452473+0000","last_active":"2023-11-06T03:11:58.456494+0000","last_peered":"2023-11-06T03:11:58.456494+0000","last_clean":"2023-11-06T03:11:58.456494+0000","last_became_active":"2023-11-06T03:11:58.452208+0000","last_became_peered":"2023-11-06T03:11:58.452208+0000","last_unstale":"2023-11-06T03:11:58.456494+0000","last_undegraded":"2023-11-06T03:11:58.456494+0000","last_fullsized":"2023-11-06T03:11:58.456494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:17:24.008019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481124+0000","last_change":"2023-11-06T03:11:58.480431+0000","last_active":"2023-11-06T03:11:58.481124+0000","last_peered":"2023-11-06T03:11:58.481124+0000","last_clean":"2023-11-06T03:11:58.481124+0000","last_became_active":"2023-11-06T03:11:58.480161+0000","last_became_peered":"2023-11-06T03:11:58.480161+0000","last_unstale":"2023-11-06T03:11:58.481124+0000","last_undegraded":"2023-11-06T03:11:58.481124+0000","last_fullsized":"2023-11-06T03:11:58.481124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:06.709058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"98'4","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.480019+0000","last_change":"2023-11-06T03:11:58.086704+0000","last_active":"2023-11-06T03:11:58.480019+0000","last_peered":"2023-11-06T03:11:58.480019+0000","last_clean":"2023-11-06T03:11:58.480019+0000","last_became_active":"2023-11-06T03:11:58.085984+0000","last_became_peered":"2023-11-06T03:11:58.085984+0000","last_unstale":"2023-11-06T03:11:58.480019+0000","last_undegraded":"2023-11-06T03:11:58.480019+0000","last_fullsized":"2023-11-06T03:11:58.480019+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:35.946625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456559+0000","last_change":"2023-11-06T03:11:58.453372+0000","last_active":"2023-11-06T03:11:58.456559+0000","last_peered":"2023-11-06T03:11:58.456559+0000","last_clean":"2023-11-06T03:11:58.456559+0000","last_became_active":"2023-11-06T03:11:58.453101+0000","last_became_peered":"2023-11-06T03:11:58.453101+0000","last_unstale":"2023-11-06T03:11:58.456559+0000","last_undegraded":"2023-11-06T03:11:58.456559+0000","last_fullsized":"2023-11-06T03:11:58.456559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:31:17.741993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.446575+0000","last_change":"2023-11-06T03:11:58.098022+0000","last_active":"2023-11-06T03:11:58.446575+0000","last_peered":"2023-11-06T03:11:58.446575+0000","last_clean":"2023-11-06T03:11:58.446575+0000","last_became_active":"2023-11-06T03:11:58.097424+0000","last_became_peered":"2023-11-06T03:11:58.097424+0000","last_unstale":"2023-11-06T03:11:58.446575+0000","last_undegraded":"2023-11-06T03:11:58.446575+0000","last_fullsized":"2023-11-06T03:11:58.446575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:13.672419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487909+0000","last_change":"2023-11-06T03:11:58.481130+0000","last_active":"2023-11-06T03:11:58.487909+0000","last_peered":"2023-11-06T03:11:58.487909+0000","last_clean":"2023-11-06T03:11:58.487909+0000","last_became_active":"2023-11-06T03:11:58.480424+0000","last_became_peered":"2023-11-06T03:11:58.480424+0000","last_unstale":"2023-11-06T03:11:58.487909+0000","last_undegraded":"2023-11-06T03:11:58.487909+0000","last_fullsized":"2023-11-06T03:11:58.487909+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:10:16.227388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481552+0000","last_change":"2023-11-06T03:11:58.101485+0000","last_active":"2023-11-06T03:11:58.481552+0000","last_peered":"2023-11-06T03:11:58.481552+0000","last_clean":"2023-11-06T03:11:58.481552+0000","last_became_active":"2023-11-06T03:11:58.100699+0000","last_became_peered":"2023-11-06T03:11:58.100699+0000","last_unstale":"2023-11-06T03:11:58.481552+0000","last_undegraded":"2023-11-06T03:11:58.481552+0000","last_fullsized":"2023-11-06T03:11:58.481552+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:40.038408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.475399+0000","last_change":"2023-11-06T03:11:58.475034+0000","last_active":"2023-11-06T03:11:58.475399+0000","last_peered":"2023-11-06T03:11:58.475399+0000","last_clean":"2023-11-06T03:11:58.475399+0000","last_became_active":"2023-11-06T03:11:58.474804+0000","last_became_peered":"2023-11-06T03:11:58.474804+0000","last_unstale":"2023-11-06T03:11:58.475399+0000","last_undegraded":"2023-11-06T03:11:58.475399+0000","last_fullsized":"2023-11-06T03:11:58.475399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:58:35.372951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.471654+0000","last_change":"2023-11-06T03:11:58.097472+0000","last_active":"2023-11-06T03:11:58.471654+0000","last_peered":"2023-11-06T03:11:58.471654+0000","last_clean":"2023-11-06T03:11:58.471654+0000","last_became_active":"2023-11-06T03:11:58.096872+0000","last_became_peered":"2023-11-06T03:11:58.096872+0000","last_unstale":"2023-11-06T03:11:58.471654+0000","last_undegraded":"2023-11-06T03:11:58.471654+0000","last_fullsized":"2023-11-06T03:11:58.471654+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:21:05.012267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458203+0000","last_change":"2023-11-06T03:11:58.451973+0000","last_active":"2023-11-06T03:11:58.458203+0000","last_peered":"2023-11-06T03:11:58.458203+0000","last_clean":"2023-11-06T03:11:58.458203+0000","last_became_active":"2023-11-06T03:11:58.451676+0000","last_became_peered":"2023-11-06T03:11:58.451676+0000","last_unstale":"2023-11-06T03:11:58.458203+0000","last_undegraded":"2023-11-06T03:11:58.458203+0000","last_fullsized":"2023-11-06T03:11:58.458203+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:08.067884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441502+0000","last_change":"2023-11-06T03:11:58.089763+0000","last_active":"2023-11-06T03:11:58.441502+0000","last_peered":"2023-11-06T03:11:58.441502+0000","last_clean":"2023-11-06T03:11:58.441502+0000","last_became_active":"2023-11-06T03:11:58.089023+0000","last_became_peered":"2023-11-06T03:11:58.089023+0000","last_unstale":"2023-11-06T03:11:58.441502+0000","last_undegraded":"2023-11-06T03:11:58.441502+0000","last_fullsized":"2023-11-06T03:11:58.441502+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:19.805243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.476648+0000","last_change":"2023-11-06T03:11:58.471197+0000","last_active":"2023-11-06T03:11:58.476648+0000","last_peered":"2023-11-06T03:11:58.476648+0000","last_clean":"2023-11-06T03:11:58.476648+0000","last_became_active":"2023-11-06T03:11:58.469584+0000","last_became_peered":"2023-11-06T03:11:58.469584+0000","last_unstale":"2023-11-06T03:11:58.476648+0000","last_undegraded":"2023-11-06T03:11:58.476648+0000","last_fullsized":"2023-11-06T03:11:58.476648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:43:41.328211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.450850+0000","last_change":"2023-11-06T03:11:58.085794+0000","last_active":"2023-11-06T03:11:58.450850+0000","last_peered":"2023-11-06T03:11:58.450850+0000","last_clean":"2023-11-06T03:11:58.450850+0000","last_became_active":"2023-11-06T03:11:58.085427+0000","last_became_peered":"2023-11-06T03:11:58.085427+0000","last_unstale":"2023-11-06T03:11:58.450850+0000","last_undegraded":"2023-11-06T03:11:58.450850+0000","last_fullsized":"2023-11-06T03:11:58.450850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:19.805243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458839+0000","last_change":"2023-11-06T03:11:58.457584+0000","last_active":"2023-11-06T03:11:58.458839+0000","last_peered":"2023-11-06T03:11:58.458839+0000","last_clean":"2023-11-06T03:11:58.458839+0000","last_became_active":"2023-11-06T03:11:58.457315+0000","last_became_peered":"2023-11-06T03:11:58.457315+0000","last_unstale":"2023-11-06T03:11:58.458839+0000","last_undegraded":"2023-11-06T03:11:58.458839+0000","last_fullsized":"2023-11-06T03:11:58.458839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:45:36.660841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.425791+0000","last_change":"2023-11-06T03:11:58.094019+0000","last_active":"2023-11-06T03:11:58.425791+0000","last_peered":"2023-11-06T03:11:58.425791+0000","last_clean":"2023-11-06T03:11:58.425791+0000","last_became_active":"2023-11-06T03:11:58.093712+0000","last_became_peered":"2023-11-06T03:11:58.093712+0000","last_unstale":"2023-11-06T03:11:58.425791+0000","last_undegraded":"2023-11-06T03:11:58.425791+0000","last_fullsized":"2023-11-06T03:11:58.425791+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:24:00.987727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.464828+0000","last_change":"2023-11-06T03:11:58.464112+0000","last_active":"2023-11-06T03:11:58.464828+0000","last_peered":"2023-11-06T03:11:58.464828+0000","last_clean":"2023-11-06T03:11:58.464828+0000","last_became_active":"2023-11-06T03:11:58.463726+0000","last_became_peered":"2023-11-06T03:11:58.463726+0000","last_unstale":"2023-11-06T03:11:58.464828+0000","last_undegraded":"2023-11-06T03:11:58.464828+0000","last_fullsized":"2023-11-06T03:11:58.464828+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.746612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458301+0000","last_change":"2023-11-06T03:11:58.101207+0000","last_active":"2023-11-06T03:11:58.458301+0000","last_peered":"2023-11-06T03:11:58.458301+0000","last_clean":"2023-11-06T03:11:58.458301+0000","last_became_active":"2023-11-06T03:11:58.100369+0000","last_became_peered":"2023-11-06T03:11:58.100369+0000","last_unstale":"2023-11-06T03:11:58.458301+0000","last_undegraded":"2023-11-06T03:11:58.458301+0000","last_fullsized":"2023-11-06T03:11:58.458301+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:07.055232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459241+0000","last_change":"2023-11-06T03:11:58.448711+0000","last_active":"2023-11-06T03:11:58.459241+0000","last_peered":"2023-11-06T03:11:58.459241+0000","last_clean":"2023-11-06T03:11:58.459241+0000","last_became_active":"2023-11-06T03:11:58.448039+0000","last_became_peered":"2023-11-06T03:11:58.448039+0000","last_unstale":"2023-11-06T03:11:58.459241+0000","last_undegraded":"2023-11-06T03:11:58.459241+0000","last_fullsized":"2023-11-06T03:11:58.459241+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:38:24.971917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.454700+0000","last_change":"2023-11-06T03:11:58.097033+0000","last_active":"2023-11-06T03:11:58.454700+0000","last_peered":"2023-11-06T03:11:58.454700+0000","last_clean":"2023-11-06T03:11:58.454700+0000","last_became_active":"2023-11-06T03:11:58.096266+0000","last_became_peered":"2023-11-06T03:11:58.096266+0000","last_unstale":"2023-11-06T03:11:58.454700+0000","last_undegraded":"2023-11-06T03:11:58.454700+0000","last_fullsized":"2023-11-06T03:11:58.454700+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:20.359200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.488857+0000","last_change":"2023-11-06T03:11:58.485657+0000","last_active":"2023-11-06T03:11:58.488857+0000","last_peered":"2023-11-06T03:11:58.488857+0000","last_clean":"2023-11-06T03:11:58.488857+0000","last_became_active":"2023-11-06T03:11:58.485127+0000","last_became_peered":"2023-11-06T03:11:58.485127+0000","last_unstale":"2023-11-06T03:11:58.488857+0000","last_undegraded":"2023-11-06T03:11:58.488857+0000","last_fullsized":"2023-11-06T03:11:58.488857+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:15.728397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.426528+0000","last_change":"2023-11-06T03:11:58.084476+0000","last_active":"2023-11-06T03:11:58.426528+0000","last_peered":"2023-11-06T03:11:58.426528+0000","last_clean":"2023-11-06T03:11:58.426528+0000","last_became_active":"2023-11-06T03:11:58.084133+0000","last_became_peered":"2023-11-06T03:11:58.084133+0000","last_unstale":"2023-11-06T03:11:58.426528+0000","last_undegraded":"2023-11-06T03:11:58.426528+0000","last_fullsized":"2023-11-06T03:11:58.426528+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:13:07.055232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455319+0000","last_change":"2023-11-06T03:11:58.453371+0000","last_active":"2023-11-06T03:11:58.455319+0000","last_peered":"2023-11-06T03:11:58.455319+0000","last_clean":"2023-11-06T03:11:58.455319+0000","last_became_active":"2023-11-06T03:11:58.453115+0000","last_became_peered":"2023-11-06T03:11:58.453115+0000","last_unstale":"2023-11-06T03:11:58.455319+0000","last_undegraded":"2023-11-06T03:11:58.455319+0000","last_fullsized":"2023-11-06T03:11:58.455319+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"98'3","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.471804+0000","last_change":"2023-11-06T03:11:58.086569+0000","last_active":"2023-11-06T03:11:58.471804+0000","last_peered":"2023-11-06T03:11:58.471804+0000","last_clean":"2023-11-06T03:11:58.471804+0000","last_became_active":"2023-11-06T03:11:58.085063+0000","last_became_peered":"2023-11-06T03:11:58.085063+0000","last_unstale":"2023-11-06T03:11:58.471804+0000","last_undegraded":"2023-11-06T03:11:58.471804+0000","last_fullsized":"2023-11-06T03:11:58.471804+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:27.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.465859+0000","last_change":"2023-11-06T03:11:58.465346+0000","last_active":"2023-11-06T03:11:58.465859+0000","last_peered":"2023-11-06T03:11:58.465859+0000","last_clean":"2023-11-06T03:11:58.465859+0000","last_became_active":"2023-11-06T03:11:58.465098+0000","last_became_peered":"2023-11-06T03:11:58.465098+0000","last_unstale":"2023-11-06T03:11:58.465859+0000","last_undegraded":"2023-11-06T03:11:58.465859+0000","last_fullsized":"2023-11-06T03:11:58.465859+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"98'6","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.478054+0000","last_change":"2023-11-06T03:11:58.098265+0000","last_active":"2023-11-06T03:11:58.478054+0000","last_peered":"2023-11-06T03:11:58.478054+0000","last_clean":"2023-11-06T03:11:58.478054+0000","last_became_active":"2023-11-06T03:11:58.097518+0000","last_became_peered":"2023-11-06T03:11:58.097518+0000","last_unstale":"2023-11-06T03:11:58.478054+0000","last_undegraded":"2023-11-06T03:11:58.478054+0000","last_fullsized":"2023-11-06T03:11:58.478054+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:03.021218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.482136+0000","last_change":"2023-11-06T03:11:58.480935+0000","last_active":"2023-11-06T03:11:58.482136+0000","last_peered":"2023-11-06T03:11:58.482136+0000","last_clean":"2023-11-06T03:11:58.482136+0000","last_became_active":"2023-11-06T03:11:58.480672+0000","last_became_peered":"2023-11-06T03:11:58.480672+0000","last_unstale":"2023-11-06T03:11:58.482136+0000","last_undegraded":"2023-11-06T03:11:58.482136+0000","last_fullsized":"2023-11-06T03:11:58.482136+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:34:46.981702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481122+0000","last_change":"2023-11-06T03:11:58.101451+0000","last_active":"2023-11-06T03:11:58.481122+0000","last_peered":"2023-11-06T03:11:58.481122+0000","last_clean":"2023-11-06T03:11:58.481122+0000","last_became_active":"2023-11-06T03:11:58.100833+0000","last_became_peered":"2023-11-06T03:11:58.100833+0000","last_unstale":"2023-11-06T03:11:58.481122+0000","last_undegraded":"2023-11-06T03:11:58.481122+0000","last_fullsized":"2023-11-06T03:11:58.481122+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:22.994152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.484014+0000","last_change":"2023-11-06T03:11:58.474340+0000","last_active":"2023-11-06T03:11:58.484014+0000","last_peered":"2023-11-06T03:11:58.484014+0000","last_clean":"2023-11-06T03:11:58.484014+0000","last_became_active":"2023-11-06T03:11:58.473656+0000","last_became_peered":"2023-11-06T03:11:58.473656+0000","last_unstale":"2023-11-06T03:11:58.484014+0000","last_undegraded":"2023-11-06T03:11:58.484014+0000","last_fullsized":"2023-11-06T03:11:58.484014+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:01.513118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.440595+0000","last_change":"2023-11-06T03:11:58.074405+0000","last_active":"2023-11-06T03:11:58.440595+0000","last_peered":"2023-11-06T03:11:58.440595+0000","last_clean":"2023-11-06T03:11:58.440595+0000","last_became_active":"2023-11-06T03:11:58.074095+0000","last_became_peered":"2023-11-06T03:11:58.074095+0000","last_unstale":"2023-11-06T03:11:58.440595+0000","last_undegraded":"2023-11-06T03:11:58.440595+0000","last_fullsized":"2023-11-06T03:11:58.440595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:29.869220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.446604+0000","last_change":"2023-11-06T03:11:58.440306+0000","last_active":"2023-11-06T03:11:58.446604+0000","last_peered":"2023-11-06T03:11:58.446604+0000","last_clean":"2023-11-06T03:11:58.446604+0000","last_became_active":"2023-11-06T03:11:58.439993+0000","last_became_peered":"2023-11-06T03:11:58.439993+0000","last_unstale":"2023-11-06T03:11:58.446604+0000","last_undegraded":"2023-11-06T03:11:58.446604+0000","last_fullsized":"2023-11-06T03:11:58.446604+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:47.016990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.438867+0000","last_change":"2023-11-06T03:11:58.098171+0000","last_active":"2023-11-06T03:11:58.438867+0000","last_peered":"2023-11-06T03:11:58.438867+0000","last_clean":"2023-11-06T03:11:58.438867+0000","last_became_active":"2023-11-06T03:11:58.097771+0000","last_became_peered":"2023-11-06T03:11:58.097771+0000","last_unstale":"2023-11-06T03:11:58.438867+0000","last_undegraded":"2023-11-06T03:11:58.438867+0000","last_fullsized":"2023-11-06T03:11:58.438867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:56:27.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.457191+0000","last_change":"2023-11-06T03:11:58.456776+0000","last_active":"2023-11-06T03:11:58.457191+0000","last_peered":"2023-11-06T03:11:58.457191+0000","last_clean":"2023-11-06T03:11:58.457191+0000","last_became_active":"2023-11-06T03:11:58.456337+0000","last_became_peered":"2023-11-06T03:11:58.456337+0000","last_unstale":"2023-11-06T03:11:58.457191+0000","last_undegraded":"2023-11-06T03:11:58.457191+0000","last_fullsized":"2023-11-06T03:11:58.457191+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:54:47.117552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"98'8","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.454863+0000","last_change":"2023-11-06T03:11:58.087561+0000","last_active":"2023-11-06T03:11:58.454863+0000","last_peered":"2023-11-06T03:11:58.454863+0000","last_clean":"2023-11-06T03:11:58.454863+0000","last_became_active":"2023-11-06T03:11:58.087205+0000","last_became_peered":"2023-11-06T03:11:58.087205+0000","last_unstale":"2023-11-06T03:11:58.454863+0000","last_undegraded":"2023-11-06T03:11:58.454863+0000","last_fullsized":"2023-11-06T03:11:58.454863+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:24.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487119+0000","last_change":"2023-11-06T03:11:58.483414+0000","last_active":"2023-11-06T03:11:58.487119+0000","last_peered":"2023-11-06T03:11:58.487119+0000","last_clean":"2023-11-06T03:11:58.487119+0000","last_became_active":"2023-11-06T03:11:58.482503+0000","last_became_peered":"2023-11-06T03:11:58.482503+0000","last_unstale":"2023-11-06T03:11:58.487119+0000","last_undegraded":"2023-11-06T03:11:58.487119+0000","last_fullsized":"2023-11-06T03:11:58.487119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:25.713522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481892+0000","last_change":"2023-11-06T03:11:58.106262+0000","last_active":"2023-11-06T03:11:58.481892+0000","last_peered":"2023-11-06T03:11:58.481892+0000","last_clean":"2023-11-06T03:11:58.481892+0000","last_became_active":"2023-11-06T03:11:58.105477+0000","last_became_peered":"2023-11-06T03:11:58.105477+0000","last_unstale":"2023-11-06T03:11:58.481892+0000","last_undegraded":"2023-11-06T03:11:58.481892+0000","last_fullsized":"2023-11-06T03:11:58.481892+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:46:54.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.480876+0000","last_change":"2023-11-06T03:11:58.480318+0000","last_active":"2023-11-06T03:11:58.480876+0000","last_peered":"2023-11-06T03:11:58.480876+0000","last_clean":"2023-11-06T03:11:58.480876+0000","last_became_active":"2023-11-06T03:11:58.480065+0000","last_became_peered":"2023-11-06T03:11:58.480065+0000","last_unstale":"2023-11-06T03:11:58.480876+0000","last_undegraded":"2023-11-06T03:11:58.480876+0000","last_fullsized":"2023-11-06T03:11:58.480876+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:10:47.914931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.432440+0000","last_change":"2023-11-06T03:11:58.104867+0000","last_active":"2023-11-06T03:11:58.432440+0000","last_peered":"2023-11-06T03:11:58.432440+0000","last_clean":"2023-11-06T03:11:58.432440+0000","last_became_active":"2023-11-06T03:11:58.104076+0000","last_became_peered":"2023-11-06T03:11:58.104076+0000","last_unstale":"2023-11-06T03:11:58.432440+0000","last_undegraded":"2023-11-06T03:11:58.432440+0000","last_fullsized":"2023-11-06T03:11:58.432440+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:33.467802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.456636+0000","last_change":"2023-11-06T03:11:58.453135+0000","last_active":"2023-11-06T03:11:58.456636+0000","last_peered":"2023-11-06T03:11:58.456636+0000","last_clean":"2023-11-06T03:11:58.456636+0000","last_became_active":"2023-11-06T03:11:58.452804+0000","last_became_peered":"2023-11-06T03:11:58.452804+0000","last_unstale":"2023-11-06T03:11:58.456636+0000","last_undegraded":"2023-11-06T03:11:58.456636+0000","last_fullsized":"2023-11-06T03:11:58.456636+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:45.415124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.437505+0000","last_change":"2023-11-06T03:11:58.087002+0000","last_active":"2023-11-06T03:11:58.437505+0000","last_peered":"2023-11-06T03:11:58.437505+0000","last_clean":"2023-11-06T03:11:58.437505+0000","last_became_active":"2023-11-06T03:11:58.085957+0000","last_became_peered":"2023-11-06T03:11:58.085957+0000","last_unstale":"2023-11-06T03:11:58.437505+0000","last_undegraded":"2023-11-06T03:11:58.437505+0000","last_fullsized":"2023-11-06T03:11:58.437505+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:13:24.700869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.486340+0000","last_change":"2023-11-06T03:11:58.480906+0000","last_active":"2023-11-06T03:11:58.486340+0000","last_peered":"2023-11-06T03:11:58.486340+0000","last_clean":"2023-11-06T03:11:58.486340+0000","last_became_active":"2023-11-06T03:11:58.480385+0000","last_became_peered":"2023-11-06T03:11:58.480385+0000","last_unstale":"2023-11-06T03:11:58.486340+0000","last_undegraded":"2023-11-06T03:11:58.486340+0000","last_fullsized":"2023-11-06T03:11:58.486340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455390+0000","last_change":"2023-11-06T03:11:58.090180+0000","last_active":"2023-11-06T03:11:58.455390+0000","last_peered":"2023-11-06T03:11:58.455390+0000","last_clean":"2023-11-06T03:11:58.455390+0000","last_became_active":"2023-11-06T03:11:58.089029+0000","last_became_peered":"2023-11-06T03:11:58.089029+0000","last_unstale":"2023-11-06T03:11:58.455390+0000","last_undegraded":"2023-11-06T03:11:58.455390+0000","last_fullsized":"2023-11-06T03:11:58.455390+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:57.997899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455261+0000","last_change":"2023-11-06T03:11:58.452958+0000","last_active":"2023-11-06T03:11:58.455261+0000","last_peered":"2023-11-06T03:11:58.455261+0000","last_clean":"2023-11-06T03:11:58.455261+0000","last_became_active":"2023-11-06T03:11:58.452396+0000","last_became_peered":"2023-11-06T03:11:58.452396+0000","last_unstale":"2023-11-06T03:11:58.455261+0000","last_undegraded":"2023-11-06T03:11:58.455261+0000","last_fullsized":"2023-11-06T03:11:58.455261+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:20:53.997665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.441064+0000","last_change":"2023-11-06T03:11:58.076687+0000","last_active":"2023-11-06T03:11:58.441064+0000","last_peered":"2023-11-06T03:11:58.441064+0000","last_clean":"2023-11-06T03:11:58.441064+0000","last_became_active":"2023-11-06T03:11:58.076316+0000","last_became_peered":"2023-11-06T03:11:58.076316+0000","last_unstale":"2023-11-06T03:11:58.441064+0000","last_undegraded":"2023-11-06T03:11:58.441064+0000","last_fullsized":"2023-11-06T03:11:58.441064+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:04.021850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451519+0000","last_change":"2023-11-06T03:11:58.449252+0000","last_active":"2023-11-06T03:11:58.451519+0000","last_peered":"2023-11-06T03:11:58.451519+0000","last_clean":"2023-11-06T03:11:58.451519+0000","last_became_active":"2023-11-06T03:11:58.448894+0000","last_became_peered":"2023-11-06T03:11:58.448894+0000","last_unstale":"2023-11-06T03:11:58.451519+0000","last_undegraded":"2023-11-06T03:11:58.451519+0000","last_fullsized":"2023-11-06T03:11:58.451519+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:43.895470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.452055+0000","last_change":"2023-11-06T03:11:58.096435+0000","last_active":"2023-11-06T03:11:58.452055+0000","last_peered":"2023-11-06T03:11:58.452055+0000","last_clean":"2023-11-06T03:11:58.452055+0000","last_became_active":"2023-11-06T03:11:58.095759+0000","last_became_peered":"2023-11-06T03:11:58.095759+0000","last_unstale":"2023-11-06T03:11:58.452055+0000","last_undegraded":"2023-11-06T03:11:58.452055+0000","last_fullsized":"2023-11-06T03:11:58.452055+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:30.026434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.472122+0000","last_change":"2023-11-06T03:11:58.466913+0000","last_active":"2023-11-06T03:11:58.472122+0000","last_peered":"2023-11-06T03:11:58.472122+0000","last_clean":"2023-11-06T03:11:58.472122+0000","last_became_active":"2023-11-06T03:11:58.466117+0000","last_became_peered":"2023-11-06T03:11:58.466117+0000","last_unstale":"2023-11-06T03:11:58.472122+0000","last_undegraded":"2023-11-06T03:11:58.472122+0000","last_fullsized":"2023-11-06T03:11:58.472122+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:20:20.817896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.447234+0000","last_change":"2023-11-06T03:11:58.088798+0000","last_active":"2023-11-06T03:11:58.447234+0000","last_peered":"2023-11-06T03:11:58.447234+0000","last_clean":"2023-11-06T03:11:58.447234+0000","last_became_active":"2023-11-06T03:11:58.088073+0000","last_became_peered":"2023-11-06T03:11:58.088073+0000","last_unstale":"2023-11-06T03:11:58.447234+0000","last_undegraded":"2023-11-06T03:11:58.447234+0000","last_fullsized":"2023-11-06T03:11:58.447234+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:04.000264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451766+0000","last_change":"2023-11-06T03:11:58.446348+0000","last_active":"2023-11-06T03:11:58.451766+0000","last_peered":"2023-11-06T03:11:58.451766+0000","last_clean":"2023-11-06T03:11:58.451766+0000","last_became_active":"2023-11-06T03:11:58.446070+0000","last_became_peered":"2023-11-06T03:11:58.446070+0000","last_unstale":"2023-11-06T03:11:58.451766+0000","last_undegraded":"2023-11-06T03:11:58.451766+0000","last_fullsized":"2023-11-06T03:11:58.451766+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:46:25.663554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.458552+0000","last_change":"2023-11-06T03:11:58.102871+0000","last_active":"2023-11-06T03:11:58.458552+0000","last_peered":"2023-11-06T03:11:58.458552+0000","last_clean":"2023-11-06T03:11:58.458552+0000","last_became_active":"2023-11-06T03:11:58.101984+0000","last_became_peered":"2023-11-06T03:11:58.101984+0000","last_unstale":"2023-11-06T03:11:58.458552+0000","last_undegraded":"2023-11-06T03:11:58.458552+0000","last_fullsized":"2023-11-06T03:11:58.458552+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:42:24.573628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.487964+0000","last_change":"2023-11-06T03:11:58.482516+0000","last_active":"2023-11-06T03:11:58.487964+0000","last_peered":"2023-11-06T03:11:58.487964+0000","last_clean":"2023-11-06T03:11:58.487964+0000","last_became_active":"2023-11-06T03:11:58.481941+0000","last_became_peered":"2023-11-06T03:11:58.481941+0000","last_unstale":"2023-11-06T03:11:58.487964+0000","last_undegraded":"2023-11-06T03:11:58.487964+0000","last_fullsized":"2023-11-06T03:11:58.487964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:26:09.289194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.438940+0000","last_change":"2023-11-06T03:11:58.102861+0000","last_active":"2023-11-06T03:11:58.438940+0000","last_peered":"2023-11-06T03:11:58.438940+0000","last_clean":"2023-11-06T03:11:58.438940+0000","last_became_active":"2023-11-06T03:11:58.102552+0000","last_became_peered":"2023-11-06T03:11:58.102552+0000","last_unstale":"2023-11-06T03:11:58.438940+0000","last_undegraded":"2023-11-06T03:11:58.438940+0000","last_fullsized":"2023-11-06T03:11:58.438940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:13.672419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461769+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:11:58.461769+0000","last_peered":"2023-11-06T03:11:58.461769+0000","last_clean":"2023-11-06T03:11:58.461769+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:11:58.461769+0000","last_undegraded":"2023-11-06T03:11:58.461769+0000","last_fullsized":"2023-11-06T03:11:58.461769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.085087+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:11:58.085087+0000","last_peered":"2023-11-06T03:11:58.085087+0000","last_clean":"2023-11-06T03:11:58.085087+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:11:58.085087+0000","last_undegraded":"2023-11-06T03:11:58.085087+0000","last_fullsized":"2023-11-06T03:11:58.085087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479732+0000","last_change":"2023-11-06T03:11:58.479363+0000","last_active":"2023-11-06T03:11:58.479732+0000","last_peered":"2023-11-06T03:11:58.479732+0000","last_clean":"2023-11-06T03:11:58.479732+0000","last_became_active":"2023-11-06T03:11:58.479129+0000","last_became_peered":"2023-11-06T03:11:58.479129+0000","last_unstale":"2023-11-06T03:11:58.479732+0000","last_undegraded":"2023-11-06T03:11:58.479732+0000","last_fullsized":"2023-11-06T03:11:58.479732+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:46:39.813525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.485966+0000","last_change":"2023-11-06T03:11:58.096608+0000","last_active":"2023-11-06T03:11:58.485966+0000","last_peered":"2023-11-06T03:11:58.485966+0000","last_clean":"2023-11-06T03:11:58.485966+0000","last_became_active":"2023-11-06T03:11:58.095377+0000","last_became_peered":"2023-11-06T03:11:58.095377+0000","last_unstale":"2023-11-06T03:11:58.485966+0000","last_undegraded":"2023-11-06T03:11:58.485966+0000","last_fullsized":"2023-11-06T03:11:58.485966+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:26:54.748461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.086420+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:11:58.086420+0000","last_peered":"2023-11-06T03:11:58.086420+0000","last_clean":"2023-11-06T03:11:58.086420+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:11:58.086420+0000","last_undegraded":"2023-11-06T03:11:58.086420+0000","last_fullsized":"2023-11-06T03:11:58.086420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.467809+0000","last_change":"2023-11-06T03:11:58.460934+0000","last_active":"2023-11-06T03:11:58.467809+0000","last_peered":"2023-11-06T03:11:58.467809+0000","last_clean":"2023-11-06T03:11:58.467809+0000","last_became_active":"2023-11-06T03:11:58.460243+0000","last_became_peered":"2023-11-06T03:11:58.460243+0000","last_unstale":"2023-11-06T03:11:58.467809+0000","last_undegraded":"2023-11-06T03:11:58.467809+0000","last_fullsized":"2023-11-06T03:11:58.467809+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:32.902273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.481699+0000","last_change":"2023-11-06T03:11:58.108134+0000","last_active":"2023-11-06T03:11:58.481699+0000","last_peered":"2023-11-06T03:11:58.481699+0000","last_clean":"2023-11-06T03:11:58.481699+0000","last_became_active":"2023-11-06T03:11:58.107476+0000","last_became_peered":"2023-11-06T03:11:58.107476+0000","last_unstale":"2023-11-06T03:11:58.481699+0000","last_undegraded":"2023-11-06T03:11:58.481699+0000","last_fullsized":"2023-11-06T03:11:58.481699+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:12:30.026434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.079840+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:11:58.079840+0000","last_peered":"2023-11-06T03:11:58.079840+0000","last_clean":"2023-11-06T03:11:58.079840+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:11:58.079840+0000","last_undegraded":"2023-11-06T03:11:58.079840+0000","last_fullsized":"2023-11-06T03:11:58.079840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.455648+0000","last_change":"2023-11-06T03:11:58.442599+0000","last_active":"2023-11-06T03:11:58.455648+0000","last_peered":"2023-11-06T03:11:58.455648+0000","last_clean":"2023-11-06T03:11:58.455648+0000","last_became_active":"2023-11-06T03:11:58.441817+0000","last_became_peered":"2023-11-06T03:11:58.441817+0000","last_unstale":"2023-11-06T03:11:58.455648+0000","last_undegraded":"2023-11-06T03:11:58.455648+0000","last_fullsized":"2023-11-06T03:11:58.455648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:33:09.979859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"98'1","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.461142+0000","last_change":"2023-11-06T03:11:58.084411+0000","last_active":"2023-11-06T03:11:58.461142+0000","last_peered":"2023-11-06T03:11:58.461142+0000","last_clean":"2023-11-06T03:11:58.461142+0000","last_became_active":"2023-11-06T03:11:58.083771+0000","last_became_peered":"2023-11-06T03:11:58.083771+0000","last_unstale":"2023-11-06T03:11:58.461142+0000","last_undegraded":"2023-11-06T03:11:58.461142+0000","last_fullsized":"2023-11-06T03:11:58.461142+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:49.187169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.085402+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:11:58.085402+0000","last_peered":"2023-11-06T03:11:58.085402+0000","last_clean":"2023-11-06T03:11:58.085402+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:11:58.085402+0000","last_undegraded":"2023-11-06T03:11:58.085402+0000","last_fullsized":"2023-11-06T03:11:58.085402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.459664+0000","last_change":"2023-11-06T03:11:58.458773+0000","last_active":"2023-11-06T03:11:58.459664+0000","last_peered":"2023-11-06T03:11:58.459664+0000","last_clean":"2023-11-06T03:11:58.459664+0000","last_became_active":"2023-11-06T03:11:58.458482+0000","last_became_peered":"2023-11-06T03:11:58.458482+0000","last_unstale":"2023-11-06T03:11:58.459664+0000","last_undegraded":"2023-11-06T03:11:58.459664+0000","last_fullsized":"2023-11-06T03:11:58.459664+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:49.067646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"98'2","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.465819+0000","last_change":"2023-11-06T03:11:58.095533+0000","last_active":"2023-11-06T03:11:58.465819+0000","last_peered":"2023-11-06T03:11:58.465819+0000","last_clean":"2023-11-06T03:11:58.465819+0000","last_became_active":"2023-11-06T03:11:58.086918+0000","last_became_peered":"2023-11-06T03:11:58.086918+0000","last_unstale":"2023-11-06T03:11:58.465819+0000","last_undegraded":"2023-11-06T03:11:58.465819+0000","last_fullsized":"2023-11-06T03:11:58.465819+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:40.038408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.080804+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:11:58.080804+0000","last_peered":"2023-11-06T03:11:58.080804+0000","last_clean":"2023-11-06T03:11:58.080804+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:11:58.080804+0000","last_undegraded":"2023-11-06T03:11:58.080804+0000","last_fullsized":"2023-11-06T03:11:58.080804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.479231+0000","last_change":"2023-11-06T03:11:58.466155+0000","last_active":"2023-11-06T03:11:58.479231+0000","last_peered":"2023-11-06T03:11:58.479231+0000","last_clean":"2023-11-06T03:11:58.479231+0000","last_became_active":"2023-11-06T03:11:58.464064+0000","last_became_peered":"2023-11-06T03:11:58.464064+0000","last_unstale":"2023-11-06T03:11:58.479231+0000","last_undegraded":"2023-11-06T03:11:58.479231+0000","last_fullsized":"2023-11-06T03:11:58.479231+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:04.123798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.435931+0000","last_change":"2023-11-06T03:11:58.084595+0000","last_active":"2023-11-06T03:11:58.435931+0000","last_peered":"2023-11-06T03:11:58.435931+0000","last_clean":"2023-11-06T03:11:58.435931+0000","last_became_active":"2023-11-06T03:11:58.084217+0000","last_became_peered":"2023-11-06T03:11:58.084217+0000","last_unstale":"2023-11-06T03:11:58.435931+0000","last_undegraded":"2023-11-06T03:11:58.435931+0000","last_fullsized":"2023-11-06T03:11:58.435931+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:43:58.044810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.086562+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:11:58.086562+0000","last_peered":"2023-11-06T03:11:58.086562+0000","last_clean":"2023-11-06T03:11:58.086562+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:11:58.086562+0000","last_undegraded":"2023-11-06T03:11:58.086562+0000","last_fullsized":"2023-11-06T03:11:58.086562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.469703+0000","last_change":"2023-11-06T03:11:58.462247+0000","last_active":"2023-11-06T03:11:58.469703+0000","last_peered":"2023-11-06T03:11:58.469703+0000","last_clean":"2023-11-06T03:11:58.469703+0000","last_became_active":"2023-11-06T03:11:58.461382+0000","last_became_peered":"2023-11-06T03:11:58.461382+0000","last_unstale":"2023-11-06T03:11:58.469703+0000","last_undegraded":"2023-11-06T03:11:58.469703+0000","last_fullsized":"2023-11-06T03:11:58.469703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:53:22.687590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451698+0000","last_change":"2023-11-06T03:11:58.105422+0000","last_active":"2023-11-06T03:11:58.451698+0000","last_peered":"2023-11-06T03:11:58.451698+0000","last_clean":"2023-11-06T03:11:58.451698+0000","last_became_active":"2023-11-06T03:11:58.102932+0000","last_became_peered":"2023-11-06T03:11:58.102932+0000","last_unstale":"2023-11-06T03:11:58.451698+0000","last_undegraded":"2023-11-06T03:11:58.451698+0000","last_fullsized":"2023-11-06T03:11:58.451698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:50:31.695545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.080874+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:11:58.080874+0000","last_peered":"2023-11-06T03:11:58.080874+0000","last_clean":"2023-11-06T03:11:58.080874+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:11:58.080874+0000","last_undegraded":"2023-11-06T03:11:58.080874+0000","last_fullsized":"2023-11-06T03:11:58.080874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.451789+0000","last_change":"2023-11-06T03:11:58.446577+0000","last_active":"2023-11-06T03:11:58.451789+0000","last_peered":"2023-11-06T03:11:58.451789+0000","last_clean":"2023-11-06T03:11:58.451789+0000","last_became_active":"2023-11-06T03:11:58.441340+0000","last_became_peered":"2023-11-06T03:11:58.441340+0000","last_unstale":"2023-11-06T03:11:58.451789+0000","last_undegraded":"2023-11-06T03:11:58.451789+0000","last_fullsized":"2023-11-06T03:11:58.451789+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:58.045440+0000","last_clean_scrub_stamp":"2023-11-06T03:11:58.045440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:43.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.432802+0000","last_change":"2023-11-06T03:11:58.095175+0000","last_active":"2023-11-06T03:11:58.432802+0000","last_peered":"2023-11-06T03:11:58.432802+0000","last_clean":"2023-11-06T03:11:58.432802+0000","last_became_active":"2023-11-06T03:11:58.094916+0000","last_became_peered":"2023-11-06T03:11:58.094916+0000","last_unstale":"2023-11-06T03:11:58.432802+0000","last_undegraded":"2023-11-06T03:11:58.432802+0000","last_fullsized":"2023-11-06T03:11:58.432802+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:11:57.032787+0000","last_clean_scrub_stamp":"2023-11-06T03:11:57.032787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:24:53.783349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-11-06T03:11:58.079467+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:11:58.079467+0000","last_peered":"2023-11-06T03:11:58.079467+0000","last_clean":"2023-11-06T03:11:58.079467+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:11:58.079467+0000","last_undegraded":"2023-11-06T03:11:58.079467+0000","last_fullsized":"2023-11-06T03:11:58.079467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86872,"kb_used_data":1744,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284968,"statfs":{"total":96636764160,"available":96547807232,"internally_reserved":0,"allocated":1785856,"data_stored":1135348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81100000000000005,"15min":0.81399999999999995},"min":{"1min":0.53200000000000003,"5min":0.46300000000000002,"15min":0.42599999999999999},"max":{"1min":1.131,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81499999999999995,"15min":0.871},"min":{"1min":0.56699999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.202,"5min":1.367,"15min":1.397},"last":0.93400000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81599999999999995,"15min":0.85199999999999998},"min":{"1min":0.56000000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0700000000000001,"5min":1.254,"15min":1.444},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.56299999999999994,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.91400000000000003,"15min":0.88400000000000001},"min":{"1min":0.51700000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1559999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.88500000000000001,"15min":0.94299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.272,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.76200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.65000000000000002,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.046,"5min":2.512,"15min":2.512},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.135,"15min":1.0620000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.60099999999999998},"max":{"1min":1.21,"5min":10.596,"15min":10.596},"last":0.60899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.94699999999999995,"15min":0.97599999999999998},"min":{"1min":0.65900000000000003,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.268,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.032},{"interface":"front","average":{"1min":1.05,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.84499999999999997,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.3480000000000001,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0629999999999999,"15min":1.052},"min":{"1min":0.85299999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2689999999999999,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.071,"15min":1.0920000000000001},"min":{"1min":0.76100000000000001,"5min":0.66800000000000004,"15min":0.55200000000000005},"max":{"1min":1.407,"5min":1.476,"15min":1.8839999999999999},"last":1.0820000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1519999999999999,"15min":1.159},"min":{"1min":0.94399999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.472,"5min":2.548,"15min":2.548},"last":1.127},{"interface":"front","average":{"1min":1.123,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.98999999999999999,"5min":0.74299999999999999,"15min":0.53300000000000003},"max":{"1min":1.367,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.107}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76464,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":75264,"kb_avail":94295376,"statfs":{"total":96636764160,"available":96558465024,"internally_reserved":0,"allocated":1220608,"data_stored":553257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77070336},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.97199999999999998,"15min":0.84799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":2.3279999999999998,"5min":16.073,"15min":16.073},"last":0.745},{"interface":"front","average":{"1min":0.92300000000000004,"5min":1.2889999999999999,"15min":0.97999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":4.4000000000000004,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.054,"15min":0.89800000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6359999999999999,"5min":16.012,"15min":16.012},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90000000000000002,"15min":0.879},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.1839999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.1200000000000001,"15min":0.90700000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.744,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.95699999999999996,"15min":0.95199999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.264,"5min":2.843,"15min":2.843},"last":0.873}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.625,"15min":1.1890000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.1519999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.3899999999999999,"15min":1.0529999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.139,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3440000000000001,"15min":1.1990000000000001},"min":{"1min":0.75800000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":7.4980000000000002,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.097},{"interface":"front","average":{"1min":1.363,"5min":1.409,"15min":1.2470000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.3970000000000002,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.173}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.4319999999999999,"15min":1.1699999999999999},"min":{"1min":0.64400000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4450000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.05},{"interface":"front","average":{"1min":1.125,"5min":1.159,"15min":1.119},"min":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.524,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.022}]},{"osd":7,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.4079999999999999,"15min":1.2070000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.6399999999999997,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.4219999999999999,"15min":1.2190000000000001},"min":{"1min":0.84399999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.538,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.92800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58464,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313376,"statfs":{"total":96636764160,"available":96576897024,"internally_reserved":0,"allocated":1204224,"data_stored":552108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":5.8019999999999996,"5min":1.792,"15min":1.1399999999999999},"min":{"1min":0.441,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":125.256,"5min":125.256,"15min":125.256},"last":0.83199999999999996},{"interface":"front","average":{"1min":5.8789999999999996,"5min":1.835,"15min":1.167},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":126.98699999999999,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.85699999999999998,"15min":0.76100000000000001},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.339,"5min":2.339,"15min":2.339},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84799999999999998},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.88600000000000001,"15min":0.84399999999999997},"min":{"1min":0.63900000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.84999999999999998,"15min":0.83999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.90500000000000003,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.96499999999999997,"15min":0.94199999999999995},"min":{"1min":0.69199999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.004}]},{"osd":4,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.92200000000000004},"min":{"1min":0.55700000000000005,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1,"15min":0.97199999999999998},"min":{"1min":0.64300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":4.4560000000000004,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.036,"15min":1.0069999999999999},"min":{"1min":0.78300000000000003,"5min":0.44,"15min":0.378},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0580000000000001,"15min":1.0249999999999999},"min":{"1min":0.755,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.6109999999999998,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.03}]},{"osd":7,"last update":"Mon Nov 6 03:11:10 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0740000000000001,"15min":1.0089999999999999},"min":{"1min":0.75700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.109,"15min":1.089},"min":{"1min":0.85299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.157}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86312,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94285528,"statfs":{"total":96636764160,"available":96548380672,"internally_reserved":0,"allocated":1212416,"data_stored":552268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":3.121,"5min":1.2490000000000001,"15min":0.96899999999999997},"min":{"1min":0.51100000000000001,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":42.853999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":3.3340000000000001,"5min":1.306,"15min":0.96099999999999997},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.41599999999999998},"max":{"1min":44.609000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":3.0830000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78600000000000003,"15min":0.75600000000000001},"min":{"1min":0.54100000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.1259999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.85799999999999998,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.026,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.3100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":1.008},"min":{"1min":0.66700000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.123,"5min":1.5820000000000001,"15min":3.2879999999999998},"last":1.452},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.85299999999999998,"15min":0.84999999999999998},"min":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.45900000000000002},"max":{"1min":1.0549999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":3.2639999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.90400000000000003,"15min":0.93300000000000005},"min":{"1min":0.57899999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.129,"5min":1.482,"15min":1.6779999999999999},"last":3.198},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.90300000000000002,"15min":0.89600000000000002},"min":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.452,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.87}]},{"osd":5,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.99199999999999999,"15min":1.0089999999999999},"min":{"1min":0.82899999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.28,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":10.192},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.052,"15min":1.095},"min":{"1min":0.69599999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.268,"5min":1.4339999999999999,"15min":1.9159999999999999},"last":5.3310000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.96499999999999997,"15min":1.0089999999999999},"min":{"1min":0.64800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5469999999999999,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0609999999999999,"15min":1.1319999999999999},"min":{"1min":0.67600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.226,"5min":1.8120000000000001,"15min":2.6549999999999998},"last":1.506}]},{"osd":7,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.081,"15min":1.075},"min":{"1min":0.82299999999999995,"5min":0.628,"15min":0.628},"max":{"1min":1.345,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":3.3490000000000002},{"interface":"front","average":{"1min":1.099,"5min":1.2310000000000001,"15min":1.181},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.625,"5min":13.582000000000001,"15min":13.582000000000001},"last":1.0069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72412,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":71232,"kb_avail":94299428,"statfs":{"total":96636764160,"available":96562614272,"internally_reserved":0,"allocated":1200128,"data_stored":551089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72941568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.25,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.113,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.84799999999999998,"5min":1.151,"15min":1.524},"last":0.495}]},{"osd":1,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.66100000000000003,"15min":0.62},"min":{"1min":0.28299999999999997,"5min":0.224,"15min":0.224},"max":{"1min":1.1839999999999999,"5min":1.371,"15min":1.371},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.309,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71299999999999997,"15min":0.67300000000000004},"min":{"1min":0.34899999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.1970000000000001,"5min":1.284,"15min":1.284},"last":0.57399999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.85399999999999998,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90100000000000002,"15min":0.94299999999999995},"min":{"1min":0.52700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.52,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.1870000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.94799999999999995,"15min":0.97899999999999998},"min":{"1min":0.57799999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.5,"5min":1.536,"15min":1.6639999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95899999999999996,"15min":1.01},"min":{"1min":0.58899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.27,"5min":1.5049999999999999,"15min":1.698},"last":0.85299999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.97499999999999998,"15min":1.002},"min":{"1min":0.57399999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3620000000000001,"5min":1.649,"15min":1.8620000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.57699999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.748,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Nov 6 03:11:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.0409999999999999},"min":{"1min":0.55000000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.079,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4119999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.0960000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46176,"kb_used_data":1176,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325664,"statfs":{"total":96636764160,"available":96589479936,"internally_reserved":0,"allocated":1204224,"data_stored":552108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.189,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.89100000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.437},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":1.1140000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.121},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.65500000000000003},"min":{"1min":0.252,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.871,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.307,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.161,"5min":1.294,"15min":1.294},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.73199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.103,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.244,"5min":1.526,"15min":1.526},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.86599999999999999,"15min":0.93300000000000005},"min":{"1min":0.45800000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.329,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.89000000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84499999999999997,"15min":0.874},"min":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.252,"15min":1.6200000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.93400000000000005,"15min":0.95499999999999996},"min":{"1min":0.44400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4359999999999999,"5min":1.5509999999999999,"15min":1.5780000000000001},"last":0.96799999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.99099999999999999,"15min":0.98799999999999999},"min":{"1min":0.66500000000000004,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.2490000000000001,"5min":1.615,"15min":1.615},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.93400000000000005,"15min":0.94499999999999995},"min":{"1min":0.503,"5min":0.5,"15min":0.5},"max":{"1min":1.1659999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":0.98999999999999999},"min":{"1min":0.54200000000000004,"5min":0.497,"15min":0.497},"max":{"1min":1.4039999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90700000000000003,"15min":0.94999999999999996},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2569999999999999,"5min":1.3540000000000001,"15min":3.4740000000000002},"last":0.80300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58468,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313372,"statfs":{"total":96636764160,"available":96576892928,"internally_reserved":0,"allocated":1208320,"data_stored":552178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.28499999999999998,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":0.97699999999999998,"5min":1.843,"15min":1.843},"last":0.374},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.2470000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.29799999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.92300000000000004,"15min":0.71599999999999997},"min":{"1min":0.32900000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.98999999999999999,"5min":30.861000000000001,"15min":30.861000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63300000000000001,"15min":0.67100000000000004},"min":{"1min":0.377,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1100000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.71699999999999997},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.012,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68700000000000006,"15min":0.71399999999999997},"min":{"1min":0.30499999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.032,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86299999999999999,"15min":0.87},"min":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.252,"5min":2.258,"15min":2.258},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.0449999999999999,"15min":0.94499999999999995},"min":{"1min":0.63300000000000001,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3600000000000001,"5min":14.753,"15min":14.753},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.3819999999999999,"15min":1.1240000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.528,"5min":43.103999999999999,"15min":43.103999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.3680000000000001,"15min":1.149},"min":{"1min":0.73299999999999998,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.3979999999999999,"5min":44.753999999999998,"15min":44.753999999999998},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.1679999999999999,"15min":1.0600000000000001},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.5189999999999999,"5min":23.971,"15min":23.971},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.1759999999999999,"15min":1.087},"min":{"1min":0.69599999999999995,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.4299999999999999,"5min":14.813000000000001,"15min":14.813000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Nov 6 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.995,"15min":1.056},"min":{"1min":0.76400000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4890000000000001,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.002,"5min":1.014,"15min":1.0409999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4419999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68920,"kb_used_data":1776,"kb_used_omap":0,"kb_used_meta":67136,"kb_avail":94302920,"statfs":{"total":96636764160,"available":96566190080,"internally_reserved":0,"allocated":1818624,"data_stored":1143727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68747264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.98299999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.71299999999999997,"15min":0.65000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.94699999999999995,"5min":7.1230000000000002,"15min":7.1230000000000002},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.61599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.095,"5min":1.155,"15min":1.155},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.147,"5min":4.6920000000000002,"15min":4.6920000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66500000000000004,"15min":0.63},"min":{"1min":0.29899999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2909999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.95399999999999996,"15min":0.92200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.306,"5min":4.641,"15min":4.641},"last":0.872},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.86399999999999999,"15min":0.91800000000000004},"min":{"1min":0.51000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":2.028,"15min":2.028},"last":0.93200000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":0.97999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.4890000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.98499999999999999,"15min":0.94599999999999995},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":1.3779999999999999,"5min":4.7430000000000003,"15min":4.7430000000000003},"last":0.91300000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93799999999999994,"15min":0.90700000000000003},"min":{"1min":0.52600000000000002,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2549999999999999,"5min":2.931,"15min":2.931},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.022,"15min":1.0109999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5760000000000001,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":0.97999999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0269999999999999,"15min":1.0049999999999999},"min":{"1min":0.61899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.3939999999999999,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.998,"15min":0.99099999999999999},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4670000000000001,"5min":1.97,"15min":1.97},"last":0.84799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:01.824 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:12:01.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:12:01.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 Processor -- start 2023-11-06T03:12:01.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 -- start start 2023-11-06T03:12:01.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd668160490 0x7fd6680cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd668160950 0x7fd668160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6681612f0 con 0x7fd66815d360 2023-11-06T03:12:01.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668161780 con 0x7fd668160950 2023-11-06T03:12:01.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66fea0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680d0450 con 0x7fd668160490 2023-11-06T03:12:01.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd668160950 0x7fd668160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd668160950 0x7fd668160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44254/0 (socket says 172.21.15.157:44254) 2023-11-06T03:12:01.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 -- 172.21.15.157:0/1964572433 learned_addr learned my addr 172.21.15.157:0/1964572433 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:01.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66dc3c640 1 --2- 172.21.15.157:0/1964572433 >> v2:172.21.15.195:3300/0 conn(0x7fd668160490 0x7fd6680cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 -- 172.21.15.157:0/1964572433 >> v2:172.21.15.195:3300/0 conn(0x7fd668160490 msgr2=0x7fd6680cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 --2- 172.21.15.157:0/1964572433 >> v2:172.21.15.195:3300/0 conn(0x7fd668160490 0x7fd6680cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 -- 172.21.15.157:0/1964572433 >> v1:172.21.15.195:6789/0 conn(0x7fd66815d360 legacy=0x7fd66815d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:01.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.982+0000 7fd66d43b640 1 -- 172.21.15.157:0/1964572433 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680cd860 con 0x7fd668160950 2023-11-06T03:12:01.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66d43b640 1 --2- 172.21.15.157:0/1964572433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160950 0x7fd668160db0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b50b0 tx=0x7fd65c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651d2c8c283c346e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66cc3a640 1 -- 172.21.15.157:0/1964572433 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0aa5f0 con 0x7fd668160950 2023-11-06T03:12:01.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66cc3a640 1 -- 172.21.15.157:0/1964572433 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c0aa750 con 0x7fd668160950 2023-11-06T03:12:01.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66cc3a640 1 -- 172.21.15.157:0/1964572433 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0aaae0 con 0x7fd668160950 2023-11-06T03:12:01.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- 172.21.15.157:0/1964572433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160950 msgr2=0x7fd668160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 --2- 172.21.15.157:0/1964572433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160950 0x7fd668160db0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b50b0 tx=0x7fd65c01c490 comp rx=0 tx=0).stop 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- 172.21.15.157:0/1964572433 shutdown_connections 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 --2- 172.21.15.157:0/1964572433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160950 0x7fd668160db0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 --2- 172.21.15.157:0/1964572433 >> v2:172.21.15.195:3300/0 conn(0x7fd668160490 0x7fd6680cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- 172.21.15.157:0/1964572433 >> 172.21.15.157:0/1964572433 conn(0x7fd668070790 msgr2=0x7fd668070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:01.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- 172.21.15.157:0/1964572433 shutdown_connections 2023-11-06T03:12:01.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- 172.21.15.157:0/1964572433 wait complete. 2023-11-06T03:12:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 Processor -- start 2023-11-06T03:12:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.986+0000 7fd66fea0640 1 -- start start 2023-11-06T03:12:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd66815d360 0x7fd668152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 0x7fd6680d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815a910 con 0x7fd668160490 2023-11-06T03:12:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681588f0 con 0x7fd668160950 2023-11-06T03:12:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668157a20 con 0x7fd66815d360 2023-11-06T03:12:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44256/0 (socket says 172.21.15.157:44256) 2023-11-06T03:12:01.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 -- 172.21.15.157:0/4081908544 learned_addr learned my addr 172.21.15.157:0/4081908544 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:01.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66e43d640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 0x7fd6680d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 -- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd66815d360 msgr2=0x7fd668152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66dc3c640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd66815d360 0x7fd668152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd66815d360 0x7fd668152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 -- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 msgr2=0x7fd6680d2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:01.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 0x7fd6680d2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:01.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 -- 172.21.15.157:0/4081908544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0094d0 con 0x7fd668160490 2023-11-06T03:12:01.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66e43d640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 0x7fd6680d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:01.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66d43b640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd65c01a0c0 tx=0x7fd65c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0aa5f0 con 0x7fd668160490 2023-11-06T03:12:01.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c0b3800 con 0x7fd668160490 2023-11-06T03:12:01.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668152dc0 con 0x7fd668160490 2023-11-06T03:12:01.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0b3b90 con 0x7fd668160490 2023-11-06T03:12:01.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd668157f50 con 0x7fd668160490 2023-11-06T03:12:01.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668072750 con 0x7fd668160490 2023-11-06T03:12:01.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd65c0b3cf0 con 0x7fd668160490 2023-11-06T03:12:01.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 0x7fd64c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:01.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd65c0a7070 con 0x7fd668160490 2023-11-06T03:12:01.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.990+0000 7fd66dc3c640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 0x7fd64c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:01.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.994+0000 7fd66dc3c640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 0x7fd64c067f20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b35b0 tx=0x7fd664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:01.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:01.994+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd65c0eb580 con 0x7fd668160490 2023-11-06T03:12:02.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.202+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd6680ccf70 con 0x7fd668160490 2023-11-06T03:12:02.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd65a7fc640 1 -- 172.21.15.157:0/4081908544 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd65c0eeba0 con 0x7fd668160490 2023-11-06T03:12:02.210 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:12:02.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 msgr2=0x7fd64c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 0x7fd64c067f20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b35b0 tx=0x7fd664008040 comp rx=0 tx=0).stop 2023-11-06T03:12:02.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 msgr2=0x7fd6680c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd65c01a0c0 tx=0x7fd65c01acd0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 shutdown_connections 2023-11-06T03:12:02.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd64c065a60 0x7fd64c067f20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd668160950 0x7fd6680d2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd668160490 0x7fd6680c6b30 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 --2- 172.21.15.157:0/4081908544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd66815d360 0x7fd668152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.210+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 >> 172.21.15.157:0/4081908544 conn(0x7fd668070790 msgr2=0x7fd66815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:02.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.214+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 shutdown_connections 2023-11-06T03:12:02.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.214+0000 7fd66fea0640 1 -- 172.21.15.157:0/4081908544 wait complete. 2023-11-06T03:12:02.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:02.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.346+0000 7f7a791bb640 1 Processor -- start 2023-11-06T03:12:02.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.346+0000 7f7a791bb640 1 -- start start 2023-11-06T03:12:02.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a791bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a741551e0 0x7f7a7406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a791bb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 0x7f7a7415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a791bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7415f580 con 0x7f7a7415e5e0 2023-11-06T03:12:02.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a791bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7415efc0 con 0x7f7a741551e0 2023-11-06T03:12:02.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a791bb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7415f2a0 con 0x7f7a740b3b20 2023-11-06T03:12:02.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a741551e0 0x7f7a7406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a7259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7a7415e5e0 0x7f7a740b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44242/0 (socket says 172.21.15.157:44242) 2023-11-06T03:12:02.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a7259c640 1 -- 172.21.15.157:0/3173041395 learned_addr learned my addr 172.21.15.157:0/3173041395 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:02.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a741551e0 0x7f7a7406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44260/0 (socket says 172.21.15.157:44260) 2023-11-06T03:12:02.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a7359e640 1 --2- 172.21.15.157:0/3173041395 >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 0x7f7a7415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 -- 172.21.15.157:0/3173041395 >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 msgr2=0x7f7a7415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 --2- 172.21.15.157:0/3173041395 >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 0x7f7a7415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 -- 172.21.15.157:0/3173041395 >> v1:172.21.15.157:6789/0 conn(0x7f7a7415e5e0 legacy=0x7f7a740b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a72d9d640 1 -- 172.21.15.157:0/3173041395 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7406f5a0 con 0x7f7a741551e0 2023-11-06T03:12:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.350+0000 7f7a7359e640 1 --2- 172.21.15.157:0/3173041395 >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 0x7f7a7415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:02.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a72d9d640 1 --2- 172.21.15.157:0/3173041395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a7406fda0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7a740cbfe0 tx=0x7f7a6401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2082fd386d717ed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:02.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a71d9b640 1 -- 172.21.15.157:0/3173041395 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a640a4c90 con 0x7f7a741551e0 2023-11-06T03:12:02.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a71d9b640 1 -- 172.21.15.157:0/3173041395 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a640a4df0 con 0x7f7a741551e0 2023-11-06T03:12:02.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a71d9b640 1 -- 172.21.15.157:0/3173041395 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a640ac660 con 0x7f7a741551e0 2023-11-06T03:12:02.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- 172.21.15.157:0/3173041395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 msgr2=0x7f7a7406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- 172.21.15.157:0/3173041395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a7406fda0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7a740cbfe0 tx=0x7f7a6401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- 172.21.15.157:0/3173041395 shutdown_connections 2023-11-06T03:12:02.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- 172.21.15.157:0/3173041395 >> v2:172.21.15.195:3300/0 conn(0x7f7a740b3b20 0x7f7a7415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- 172.21.15.157:0/3173041395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a7406fda0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- 172.21.15.157:0/3173041395 >> 172.21.15.157:0/3173041395 conn(0x7f7a74065930 msgr2=0x7f7a74065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:02.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- 172.21.15.157:0/3173041395 shutdown_connections 2023-11-06T03:12:02.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- 172.21.15.157:0/3173041395 wait complete. 2023-11-06T03:12:02.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 Processor -- start 2023-11-06T03:12:02.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- start start 2023-11-06T03:12:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a740b3b20 0x7f7a74147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a7415e5e0 0x7f7a7414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7414f8c0 con 0x7f7a741551e0 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74153200 con 0x7f7a740b3b20 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a791bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7414f270 con 0x7f7a7415e5e0 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a7259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a7259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44262/0 (socket says 172.21.15.157:44262) 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.354+0000 7f7a7259c640 1 -- 172.21.15.157:0/711118519 learned_addr learned my addr 172.21.15.157:0/711118519 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:02.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7359e640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a7415e5e0 0x7f7a7414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a72d9d640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a740b3b20 0x7f7a74147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 -- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a7415e5e0 msgr2=0x7f7a7414add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a7415e5e0 0x7f7a7414add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 -- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a740b3b20 msgr2=0x7f7a74147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a740b3b20 0x7f7a74147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 -- 172.21.15.157:0/711118519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009a70 con 0x7f7a741551e0 2023-11-06T03:12:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a7259c640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c780 tx=0x7f7a6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680072d0 con 0x7f7a741551e0 2023-11-06T03:12:02.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68007430 con 0x7f7a741551e0 2023-11-06T03:12:02.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7414f550 con 0x7f7a741551e0 2023-11-06T03:12:02.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680047f0 con 0x7f7a741551e0 2023-11-06T03:12:02.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a741459e0 con 0x7f7a741551e0 2023-11-06T03:12:02.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a74072750 con 0x7f7a741551e0 2023-11-06T03:12:02.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7a68004950 con 0x7f7a741551e0 2023-11-06T03:12:02.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 0x7f7a60067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:02.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a680824c0 con 0x7f7a741551e0 2023-11-06T03:12:02.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a72d9d640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 0x7f7a60067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:02.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.358+0000 7f7a72d9d640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 0x7f7a60067f50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7a640ae000 tx=0x7f7a640a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:02.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.362+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7a6804fdb0 con 0x7f7a741551e0 2023-11-06T03:12:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.538+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a74071730 con 0x7f7a741551e0 2023-11-06T03:12:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.538+0000 7f7a5b7fe640 1 -- 172.21.15.157:0/711118519 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f7a680533d0 con 0x7f7a741551e0 2023-11-06T03:12:02.539 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:02.539 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 msgr2=0x7f7a60067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 0x7f7a60067f50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7a640ae000 tx=0x7f7a640a6070 comp rx=0 tx=0).stop 2023-11-06T03:12:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 msgr2=0x7f7a741454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c780 tx=0x7f7a6800cc40 comp rx=0 tx=0).stop 2023-11-06T03:12:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 shutdown_connections 2023-11-06T03:12:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a60065a90 0x7f7a60067f50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a7415e5e0 0x7f7a7414add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a741551e0 0x7f7a741454a0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 --2- 172.21.15.157:0/711118519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a740b3b20 0x7f7a74147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 >> 172.21.15.157:0/711118519 conn(0x7f7a74065930 msgr2=0x7f7a740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 shutdown_connections 2023-11-06T03:12:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:02.542+0000 7f7a791bb640 1 -- 172.21.15.157:0/711118519 wait complete. 2023-11-06T03:12:02.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:11:58.417992+0000', 'modified': '2023-11-06T03:12:00.431355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7785}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7785': {'gid': 7785, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1762973381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1762973381}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1762973381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:12:02.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:12:02.959 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:12:03.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 Processor -- start 2023-11-06T03:12:03.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 -- start start 2023-11-06T03:12:03.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa39015caa0 0x7fa3900cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa39015ce80 0x7fa3900cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3900ce1a0 0x7fa3900d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3900d2920 con 0x7fa39015caa0 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3900d18a0 con 0x7fa39015ce80 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa395951640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900d1b60 con 0x7fa3900ce1a0 2023-11-06T03:12:03.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa39015caa0 0x7fa3900cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa39015caa0 0x7fa3900cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44264/0 (socket says 172.21.15.157:44264) 2023-11-06T03:12:03.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 -- 172.21.15.157:0/2161462604 learned_addr learned my addr 172.21.15.157:0/2161462604 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3900ce1a0 0x7fa3900d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/2161462604 >> v2:172.21.15.195:3300/0 conn(0x7fa39015ce80 0x7fa3900cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 -- 172.21.15.157:0/2161462604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3900ce1a0 msgr2=0x7fa3900d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3900ce1a0 0x7fa3900d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 -- 172.21.15.157:0/2161462604 >> v2:172.21.15.195:3300/0 conn(0x7fa39015ce80 msgr2=0x7fa3900cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 --2- 172.21.15.157:0/2161462604 >> v2:172.21.15.195:3300/0 conn(0x7fa39015ce80 0x7fa3900cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.098+0000 7fa38effd640 1 -- 172.21.15.157:0/2161462604 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900d1e40 con 0x7fa39015caa0 2023-11-06T03:12:03.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38effd640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015caa0 0x7fa3900cdc60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3900c6720 tx=0x7fa38001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac62572627230c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38dffb640 1 -- 172.21.15.157:0/2161462604 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800aa5f0 con 0x7fa39015caa0 2023-11-06T03:12:03.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38dffb640 1 -- 172.21.15.157:0/2161462604 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800aa750 con 0x7fa39015caa0 2023-11-06T03:12:03.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38dffb640 1 -- 172.21.15.157:0/2161462604 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800aaae0 con 0x7fa39015caa0 2023-11-06T03:12:03.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- 172.21.15.157:0/2161462604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015caa0 msgr2=0x7fa3900cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015caa0 0x7fa3900cdc60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3900c6720 tx=0x7fa38001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- 172.21.15.157:0/2161462604 shutdown_connections 2023-11-06T03:12:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3900ce1a0 0x7fa3900d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- 172.21.15.157:0/2161462604 >> v2:172.21.15.195:3300/0 conn(0x7fa39015ce80 0x7fa3900cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- 172.21.15.157:0/2161462604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015caa0 0x7fa3900cdc60 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- 172.21.15.157:0/2161462604 >> 172.21.15.157:0/2161462604 conn(0x7fa390070790 msgr2=0x7fa390070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- 172.21.15.157:0/2161462604 shutdown_connections 2023-11-06T03:12:03.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- 172.21.15.157:0/2161462604 wait complete. 2023-11-06T03:12:03.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 Processor -- start 2023-11-06T03:12:03.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- start start 2023-11-06T03:12:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3900ce1a0 0x7fa39015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa39015caa0 0x7fa39015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900c6790 con 0x7fa39015ce80 2023-11-06T03:12:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900cac40 con 0x7fa3900ce1a0 2023-11-06T03:12:03.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa395951640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ca680 con 0x7fa39015caa0 2023-11-06T03:12:03.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44280/0 (socket says 172.21.15.157:44280) 2023-11-06T03:12:03.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.102+0000 7fa38f7fe640 1 -- 172.21.15.157:0/40154209 learned_addr learned my addr 172.21.15.157:0/40154209 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38effd640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3900ce1a0 0x7fa39015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa39015caa0 0x7fa39015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 -- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa39015caa0 msgr2=0x7fa39015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa39015caa0 0x7fa39015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 -- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3900ce1a0 msgr2=0x7fa39015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3900ce1a0 0x7fa39015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 -- 172.21.15.157:0/40154209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800094d0 con 0x7fa39015ce80 2023-11-06T03:12:03.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa38800eed0 tx=0x7fa38800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 -- 172.21.15.157:0/40154209 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388010040 con 0x7fa39015ce80 2023-11-06T03:12:03.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 -- 172.21.15.157:0/40154209 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38800bd30 con 0x7fa39015ce80 2023-11-06T03:12:03.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 -- 172.21.15.157:0/40154209 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38800bea0 con 0x7fa39015ce80 2023-11-06T03:12:03.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900ca960 con 0x7fa39015ce80 2023-11-06T03:12:03.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3900ca170 con 0x7fa39015ce80 2023-11-06T03:12:03.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 -- 172.21.15.157:0/40154209 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa388014640 con 0x7fa39015ce80 2023-11-06T03:12:03.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 0x7fa37c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa377fff640 1 -- 172.21.15.157:0/40154209 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa388083b50 con 0x7fa39015ce80 2023-11-06T03:12:03.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa38effd640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 0x7fa37c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa398a79000 1 Processor -- start 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.106+0000 7fa398a79000 1 -- start start 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38effd640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 0x7fa37c067f50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa3800ae000 tx=0x7fa3800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecef6a90 0x55fbecef6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecdbefc0 con 0x55fbecf41000 2023-11-06T03:12:03.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd4d1a0 con 0x55fbecf406e0 2023-11-06T03:12:03.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd4a770 con 0x55fbecef6a90 2023-11-06T03:12:03.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44294/0 (socket says 172.21.15.157:44294) 2023-11-06T03:12:03.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 -- 172.21.15.157:0/661129504 learned_addr learned my addr 172.21.15.157:0/661129504 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38effd640 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 -- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecef6a90 msgr2=0x55fbecef6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecef6a90 0x55fbecef6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 -- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 msgr2=0x55fbecf40ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf40ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 -- 172.21.15.157:0/661129504 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbeca42890 con 0x55fbecf41000 2023-11-06T03:12:03.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa384009b00 tx=0x7fa384009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c0f0a34cebb36e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa3757fa640 1 -- 172.21.15.157:0/661129504 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384011c20 con 0x55fbecf41000 2023-11-06T03:12:03.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa3757fa640 1 -- 172.21.15.157:0/661129504 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384011d80 con 0x55fbecf41000 2023-11-06T03:12:03.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa3757fa640 1 -- 172.21.15.157:0/661129504 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384015660 con 0x55fbecf41000 2023-11-06T03:12:03.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- 172.21.15.157:0/661129504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 msgr2=0x55fbecef6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa384009b00 tx=0x7fa384009190 comp rx=0 tx=0).stop 2023-11-06T03:12:03.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- 172.21.15.157:0/661129504 shutdown_connections 2023-11-06T03:12:03.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecef6a90 0x55fbecef6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecf41000 0x55fbecef6550 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- 172.21.15.157:0/661129504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf40ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- 172.21.15.157:0/661129504 >> 172.21.15.157:0/661129504 conn(0x55fbecefcbf0 msgr2=0x55fbecef4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- 172.21.15.157:0/661129504 shutdown_connections 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- 172.21.15.157:0/661129504 wait complete. 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- start start 2023-11-06T03:12:03.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 Processor -- start 2023-11-06T03:12:03.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf47d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd46060 con 0x55fbecef6a90 2023-11-06T03:12:03.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecce5290 con 0x55fbecf41000 2023-11-06T03:12:03.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd036f0 con 0x55fbecf406e0 2023-11-06T03:12:03.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa398a79000 10 client.0 _ll_register_callbacks cb 0x55fbecee21d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:03.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44298/0 (socket says 172.21.15.157:44298) 2023-11-06T03:12:03.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38effd640 1 -- 172.21.15.157:0/3485049307 learned_addr learned my addr 172.21.15.157:0/3485049307 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.110+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 -- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 msgr2=0x55fbecf47d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf47d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 -- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 msgr2=0x55fbecf016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 -- 172.21.15.157:0/3485049307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa384009730 con 0x55fbecef6a90 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38f7fe640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38effd640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa380097f60 tx=0x7fa38001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae4a0fdc47bf471 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:03.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800c73f0 con 0x55fbecef6a90 2023-11-06T03:12:03.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3800a7070 con 0x55fbecef6a90 2023-11-06T03:12:03.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbecd449b0 con 0x55fbecef6a90 2023-11-06T03:12:03.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3800c7550 con 0x55fbecef6a90 2023-11-06T03:12:03.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 10 client.7926 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:03.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55fbecd0b1e0 con 0x55fbecef6a90 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fbecd449b0 con 0x55fbecef6a90 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa3800a9070 con 0x55fbecef6a90 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 10 client.7926 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 10 client.7926 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 20 client.7926 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf02410 0x55fbecf027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 4 client.7926 mds_command: new command op to 7785 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbecd754d0 con 0x55fbecf02410 2023-11-06T03:12:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf02410 0x55fbecf027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 10 client.7926 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:03.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf02410 0x55fbecf027f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66904dad7daf523 server_cookie=53f3cba085ad5a9b in_seq=0 out_seq=0 2023-11-06T03:12:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 1 -- 172.21.15.157:0/3485049307 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fbecd754d0 con 0x55fbecf02410 2023-11-06T03:12:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 10 client.7926 handle_command_reply: tid=0 2023-11-06T03:12:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 client.7926 shutdown 2023-11-06T03:12:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 msgr2=0x55fbecf47830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa380097f60 tx=0x7fa38001ad10 comp rx=0 tx=0).stop 2023-11-06T03:12:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 shutdown_connections 2023-11-06T03:12:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf02410 0x55fbecf027f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf47d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa36e7fc640 0 client.7926 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 --2- 172.21.15.157:0/3485049307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf47830 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 >> 172.21.15.157:0/3485049307 conn(0x55fbecefcbf0 msgr2=0x55fbecef5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 shutdown_connections 2023-11-06T03:12:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 1 -- 172.21.15.157:0/3485049307 wait complete. 2023-11-06T03:12:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.114+0000 7fa398a79000 20 client.7926 trim_cache size 0 max 16384 2023-11-06T03:12:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 Processor -- start 2023-11-06T03:12:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- start start 2023-11-06T03:12:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf98030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf41000 0x55fbecf98570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbeca03940 con 0x55fbecef6a90 2023-11-06T03:12:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecceb6f0 con 0x55fbecf406e0 2023-11-06T03:12:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fbecf405a0 con 0x55fbecf41000 2023-11-06T03:12:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44310/0 (socket says 172.21.15.157:44310) 2023-11-06T03:12:03.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 -- 172.21.15.157:0/3084795184 learned_addr learned my addr 172.21.15.157:0/3084795184 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 -- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf41000 msgr2=0x55fbecf98570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf41000 0x55fbecf98570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 -- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 msgr2=0x55fbecf98030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf98030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 -- 172.21.15.157:0/3084795184 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbeccf1330 con 0x55fbecef6a90 2023-11-06T03:12:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa38effd640 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa380008010 tx=0x7fa3800c1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdba1464103d28f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa375ffb640 1 -- 172.21.15.157:0/3084795184 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800bfc70 con 0x55fbecef6a90 2023-11-06T03:12:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa375ffb640 1 -- 172.21.15.157:0/3084795184 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800c73f0 con 0x55fbecef6a90 2023-11-06T03:12:03.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa375ffb640 1 -- 172.21.15.157:0/3084795184 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800c77f0 con 0x55fbecef6a90 2023-11-06T03:12:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- 172.21.15.157:0/3084795184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 msgr2=0x55fbecf97af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa380008010 tx=0x7fa3800c1040 comp rx=0 tx=0).stop 2023-11-06T03:12:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- 172.21.15.157:0/3084795184 shutdown_connections 2023-11-06T03:12:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf41000 0x55fbecf98570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf406e0 0x55fbecf98030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 --2- 172.21.15.157:0/3084795184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf97af0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- 172.21.15.157:0/3084795184 >> 172.21.15.157:0/3084795184 conn(0x55fbecefcbf0 msgr2=0x55fbecefbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- 172.21.15.157:0/3084795184 shutdown_connections 2023-11-06T03:12:03.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- 172.21.15.157:0/3084795184 wait complete. 2023-11-06T03:12:03.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 -- start start 2023-11-06T03:12:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.130+0000 7fa398a79000 1 Processor -- start 2023-11-06T03:12:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd89560 con 0x55fbecef6a90 2023-11-06T03:12:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbecd08960 con 0x55fbecf41000 2023-11-06T03:12:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fbeccff0f0 con 0x55fbecf406e0 2023-11-06T03:12:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 10 client.0 _ll_register_callbacks cb 0x55fbecec29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44316/0 (socket says 172.21.15.157:44316) 2023-11-06T03:12:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 -- 172.21.15.157:0/1765922312 learned_addr learned my addr 172.21.15.157:0/1765922312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 -- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 msgr2=0x55fbecf432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 -- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 msgr2=0x55fbecf43820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 -- 172.21.15.157:0/1765922312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbecd0b1e0 con 0x55fbecef6a90 2023-11-06T03:12:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38effd640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa38001c9b0 tx=0x7fa380008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b3d59296471361 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800c7e10 con 0x55fbecef6a90 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3800e0450 con 0x55fbecef6a90 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbecd15b90 con 0x55fbecef6a90 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3800e0630 con 0x55fbecef6a90 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 10 client.7932 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55fbeca42cc0 con 0x55fbecef6a90 2023-11-06T03:12:03.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fbecd15b90 con 0x55fbecef6a90 2023-11-06T03:12:03.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa3800de070 con 0x55fbecef6a90 2023-11-06T03:12:03.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 10 client.7932 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:03.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 10 client.7932 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:12:03.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 20 client.7932 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:03.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf43d60 0x55fbecf44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 4 client.7932 mds_command: new command op to 7785 tid=0[{"prefix": "status"}] 2023-11-06T03:12:03.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fbecd3f0e0 con 0x55fbecf43d60 2023-11-06T03:12:03.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf43d60 0x55fbecf44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 10 client.7932 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:03.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa38e7fc640 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf43d60 0x55fbecf44140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888506cd88f1777b server_cookie=f7119b5c6d9b0dce in_seq=0 out_seq=0 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 1 -- 172.21.15.157:0/1765922312 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fbecd3f0e0 con 0x55fbecf43d60 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 10 client.7932 handle_command_reply: tid=0 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 client.7932 shutdown 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 msgr2=0x55fbecf42da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa38001c9b0 tx=0x7fa380008040 comp rx=0 tx=0).stop 2023-11-06T03:12:03.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 shutdown_connections 2023-11-06T03:12:03.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55fbecf43d60 0x55fbecf44140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbecf41000 0x55fbecf43820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fbecf406e0 0x55fbecf432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa36e7fc640 0 client.7932 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:03.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 --2- 172.21.15.157:0/1765922312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbecef6a90 0x55fbecf42da0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 >> 172.21.15.157:0/1765922312 conn(0x55fbecefcbf0 msgr2=0x55fbecf3c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 shutdown_connections 2023-11-06T03:12:03.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 1 -- 172.21.15.157:0/1765922312 wait complete. 2023-11-06T03:12:03.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa398a79000 20 client.7932 trim_cache size 0 max 16384 2023-11-06T03:12:03.156 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:03.156 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:12:03.156 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:12:03.156 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7785, 2023-11-06T03:12:03.156 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.7181541630000003, 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 79, 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 99, 2023-11-06T03:12:03.157 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 99, 2023-11-06T03:12:03.158 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 9.1554719010000003 2023-11-06T03:12:03.158 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 msgr2=0x7fa37c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 0x7fa37c067f50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa3800ae000 tx=0x7fa3800a7000 comp rx=0 tx=0).stop 2023-11-06T03:12:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 msgr2=0x7fa3900c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa38800eed0 tx=0x7fa38800b480 comp rx=0 tx=0).stop 2023-11-06T03:12:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 shutdown_connections 2023-11-06T03:12:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa37c065a90 0x7fa37c067f50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa39015ce80 0x7fa3900c6bb0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa39015caa0 0x7fa39015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 --2- 172.21.15.157:0/40154209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3900ce1a0 0x7fa39015c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 >> 172.21.15.157:0/40154209 conn(0x7fa390070790 msgr2=0x7fa3900b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 shutdown_connections 2023-11-06T03:12:03.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.134+0000 7fa395951640 1 -- 172.21.15.157:0/40154209 wait complete. 2023-11-06T03:12:03.162 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:12:03.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.270+0000 7f67ae3aa640 1 Processor -- start 2023-11-06T03:12:03.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 -- start start 2023-11-06T03:12:03.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a815e5e0 0x7f67a80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67a815ea90 0x7f67a80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a815f580 con 0x7f67a81551e0 2023-11-06T03:12:03.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a80704b0 con 0x7f67a815e5e0 2023-11-06T03:12:03.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67ae3aa640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a815efc0 con 0x7f67a815ea90 2023-11-06T03:12:03.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f67a815e5e0 0x7f67a80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f67a81551e0 0x7f67a806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44256/0 (socket says 172.21.15.157:44256) 2023-11-06T03:12:03.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a7fff640 1 -- 172.21.15.157:0/105386497 learned_addr learned my addr 172.21.15.157:0/105386497 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:03.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 -- 172.21.15.157:0/105386497 >> v2:172.21.15.195:3300/0 conn(0x7f67a815ea90 msgr2=0x7f67a80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:03.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 --2- 172.21.15.157:0/105386497 >> v2:172.21.15.195:3300/0 conn(0x7f67a815ea90 0x7f67a80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 -- 172.21.15.157:0/105386497 >> v1:172.21.15.157:6789/0 conn(0x7f67a81551e0 legacy=0x7f67a806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 -- 172.21.15.157:0/105386497 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a815f2a0 con 0x7f67a815e5e0 2023-11-06T03:12:03.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a77fe640 1 --2- 172.21.15.157:0/105386497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a815e5e0 0x7f67a80b3410 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f67a80cc2f0 tx=0x7f679001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994a961d7a62e5cf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a6ffd640 1 -- 172.21.15.157:0/105386497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900ac600 con 0x7f67a815e5e0 2023-11-06T03:12:03.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a6ffd640 1 -- 172.21.15.157:0/105386497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900a8050 con 0x7f67a815e5e0 2023-11-06T03:12:03.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.274+0000 7f67a6ffd640 1 -- 172.21.15.157:0/105386497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67900ac980 con 0x7f67a815e5e0 2023-11-06T03:12:03.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/105386497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a815e5e0 msgr2=0x7f67a80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/105386497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a815e5e0 0x7f67a80b3410 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f67a80cc2f0 tx=0x7f679001c3a0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/105386497 shutdown_connections 2023-11-06T03:12:03.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/105386497 >> v2:172.21.15.195:3300/0 conn(0x7f67a815ea90 0x7f67a80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/105386497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a815e5e0 0x7f67a80b3410 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/105386497 >> 172.21.15.157:0/105386497 conn(0x7f67a8065930 msgr2=0x7f67a8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/105386497 shutdown_connections 2023-11-06T03:12:03.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/105386497 wait complete. 2023-11-06T03:12:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 Processor -- start 2023-11-06T03:12:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- start start 2023-11-06T03:12:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67a815e5e0 0x7f67a814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 0x7f67a814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8146d40 con 0x7f67a81551e0 2023-11-06T03:12:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a806c0d0 con 0x7f67a815ea90 2023-11-06T03:12:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a814a980 con 0x7f67a815e5e0 2023-11-06T03:12:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44332/0 (socket says 172.21.15.157:44332) 2023-11-06T03:12:03.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 -- 172.21.15.157:0/949652479 learned_addr learned my addr 172.21.15.157:0/949652479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:03.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ac947640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 0x7f67a814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a77fe640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67a815e5e0 0x7f67a814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 -- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67a815e5e0 msgr2=0x7f67a814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67a815e5e0 0x7f67a814a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 -- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 msgr2=0x7f67a814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 0x7f67a814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 -- 172.21.15.157:0/949652479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790009420 con 0x7f67a81551e0 2023-11-06T03:12:03.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ac947640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 0x7f67a814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:03.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a7fff640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f67a000cbd0 tx=0x7f67a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000ad40 con 0x7f67a81551e0 2023-11-06T03:12:03.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0012400 con 0x7f67a81551e0 2023-11-06T03:12:03.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a806b4c0 con 0x7f67a81551e0 2023-11-06T03:12:03.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000aea0 con 0x7f67a81551e0 2023-11-06T03:12:03.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a806ba80 con 0x7f67a81551e0 2023-11-06T03:12:03.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.278+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a8072750 con 0x7f67a81551e0 2023-11-06T03:12:03.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.282+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f67a0007070 con 0x7f67a81551e0 2023-11-06T03:12:03.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.282+0000 7f67a4ff9640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 0x7f6788067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:03.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.282+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67a00833b0 con 0x7f67a81551e0 2023-11-06T03:12:03.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.282+0000 7f67a77fe640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 0x7f6788067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:03.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.282+0000 7f67a77fe640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 0x7f6788067f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f67900ae000 tx=0x7f6790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:03.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.286+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f67a0050d50 con 0x7f67a81551e0 2023-11-06T03:12:03.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.470+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67a8071730 con 0x7f67a81551e0 2023-11-06T03:12:03.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67a4ff9640 1 -- 172.21.15.157:0/949652479 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f67a0054370 con 0x7f67a81551e0 2023-11-06T03:12:03.474 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:12:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 msgr2=0x7f6788067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 0x7f6788067f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f67900ae000 tx=0x7f6790008040 comp rx=0 tx=0).stop 2023-11-06T03:12:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 msgr2=0x7f67a8146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f67a000cbd0 tx=0x7f67a000a430 comp rx=0 tx=0).stop 2023-11-06T03:12:03.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 shutdown_connections 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6788065a90 0x7f6788067f50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67a815ea90 0x7f67a814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67a815e5e0 0x7f67a814a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 --2- 172.21.15.157:0/949652479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67a81551e0 0x7f67a8146800 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 >> 172.21.15.157:0/949652479 conn(0x7f67a8065930 msgr2=0x7f67a806fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:03.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 shutdown_connections 2023-11-06T03:12:03.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:03.474+0000 7f67ae3aa640 1 -- 172.21.15.157:0/949652479 wait complete. 2023-11-06T03:12:03.492 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:12:03.492 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:12:03.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:12:03.493 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:12:03.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:03.494 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:12:03.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:12:03.494 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:12:03.495 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:03.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.496 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:12:03.502 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:12:03.502 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:12:03.502 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.503 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:12:03.504 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:12:03.504 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:03.504 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.505 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:12:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:12:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42015sec preferred_lft 42015sec 2023-11-06T03:12:03.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:12:03.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.506 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:12:03.506 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.506 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:12:03.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.507 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:12:03.507 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:03.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:12:03.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:12:03.508 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.508 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:12:03.508 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:12:03.508 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:12:03.508 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.509 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:12:03.509 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:12:03.509 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:12:03.509 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:03.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.510 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:12:03.553 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:12:03.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.555 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:12:03.603 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:12:03.604 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:12:03.604 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:12:03.604 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:12:03.604 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:12:03.604 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:12:03.605 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:03.605 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:03.605 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:03.605 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:12:03.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.606 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:12:03.606 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:03.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:12:03.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:12:03.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:12:03.608 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:03.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.678 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:12:03.730 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:12:03.730 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:12:03.730 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:03.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:03.731 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:12:03.786 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:12:03.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:12:03.787 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:03.849 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:03.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:12:03.849 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:03.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:03.955 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:03.955 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:04.009 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:04.010 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:04.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:04.011 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:04.057 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:12:04.058 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:12:04.102 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:04.117 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:04.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:04.175 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:04.175 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:04.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:04.177 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:04.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:12:04.182+0000 7f19801ee580 -1 init, newargv = 0x55e5195a1ff0 newargc=15 2023-11-06T03:12:04.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[81268]: starting ceph client 2023-11-06T03:12:04.190 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[81268]: starting fuse 2023-11-06T03:12:05.289 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:05.306 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:05.307 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:05.361 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:05.362 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:05.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:05.363 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:05.409 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:12:05.410 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:12:05.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:12:05.411 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:12:05.411 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:12:05.411 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:12:05.412 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:12:05.412 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:05.412 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:12:05.413 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:12:05.413 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:12:05.413 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:12:05.414 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:05.414 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:12:05.415 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:12:05.415 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:12:05.415 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:12:05.416 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:05.416 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:12:05.416 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:12:05.417 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:12:05.417 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:12:05.418 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:12:05.418 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:12:05.419 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:12:05.419 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:12:05.420 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:05.420 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:12:05.420 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:12:05.529 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.81268.asok 2023-11-06T03:12:05.712 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81268.asok 2023-11-06T03:12:05.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:05.713 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81268.asok status 2023-11-06T03:12:05.866 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:05.866 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:12:05.866 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:12:05.866 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "81268", 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:12:05.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:05.868 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:12:05.868 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:12:05.868 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 7950, 2023-11-06T03:12:05.868 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:12:05.868 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 7950 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:12:05.869 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3266506908 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:12:05.870 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3266506908 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.7950 192.168.0.1:0/3266506908", 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3266506908", 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:12:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 79, 2023-11-06T03:12:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 99, 2023-11-06T03:12:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:12:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:12:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:12:05.872 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:05.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:05.880 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:12:05.880 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:05.887 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:12:05.887 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:05.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:05.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:05.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:05.941 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:05.942 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:05.942 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.0 2023-11-06T03:12:05.942 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:05.943 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:05.943 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:05.943 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:06.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152504 s, 68.8 MB/s 2023-11-06T03:12:06.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.045 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100473 s, 104 MB/s 2023-11-06T03:12:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016278 s, 64.4 MB/s 2023-11-06T03:12:06.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.077 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107737 s, 97.3 MB/s 2023-11-06T03:12:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.092 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115946 s, 90.4 MB/s 2023-11-06T03:12:06.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.108 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136001 s, 77.1 MB/s 2023-11-06T03:12:06.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.124 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133557 s, 78.5 MB/s 2023-11-06T03:12:06.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152175 s, 68.9 MB/s 2023-11-06T03:12:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168137 s, 62.4 MB/s 2023-11-06T03:12:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126419 s, 82.9 MB/s 2023-11-06T03:12:06.192 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130161 s, 80.6 MB/s 2023-11-06T03:12:06.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.209 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.209 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139209 s, 75.3 MB/s 2023-11-06T03:12:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118739 s, 88.3 MB/s 2023-11-06T03:12:06.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.240 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140389 s, 74.7 MB/s 2023-11-06T03:12:06.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.255 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012789 s, 82.0 MB/s 2023-11-06T03:12:06.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117491 s, 89.2 MB/s 2023-11-06T03:12:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103508 s, 101 MB/s 2023-11-06T03:12:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015442 s, 67.9 MB/s 2023-11-06T03:12:06.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.318 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146415 s, 71.6 MB/s 2023-11-06T03:12:06.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010467 s, 100 MB/s 2023-11-06T03:12:06.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.347 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134489 s, 78.0 MB/s 2023-11-06T03:12:06.359 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.360 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104084 s, 101 MB/s 2023-11-06T03:12:06.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.377 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145739 s, 71.9 MB/s 2023-11-06T03:12:06.392 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.392 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.392 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012329 s, 85.0 MB/s 2023-11-06T03:12:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.408 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128954 s, 81.3 MB/s 2023-11-06T03:12:06.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990431 s, 106 MB/s 2023-11-06T03:12:06.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140466 s, 74.7 MB/s 2023-11-06T03:12:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121293 s, 86.5 MB/s 2023-11-06T03:12:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.468 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129955 s, 80.7 MB/s 2023-11-06T03:12:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120147 s, 87.3 MB/s 2023-11-06T03:12:06.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.498 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123052 s, 85.2 MB/s 2023-11-06T03:12:06.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.514 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122956 s, 85.3 MB/s 2023-11-06T03:12:06.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:06.515 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:06.515 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:06.516 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.1 2023-11-06T03:12:06.516 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:06.516 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:06.517 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:06.517 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.579 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935459 s, 112 MB/s 2023-11-06T03:12:06.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113275 s, 92.6 MB/s 2023-11-06T03:12:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116385 s, 90.1 MB/s 2023-11-06T03:12:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123844 s, 84.7 MB/s 2023-11-06T03:12:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115252 s, 91.0 MB/s 2023-11-06T03:12:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.649 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.650 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110447 s, 94.9 MB/s 2023-11-06T03:12:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858973 s, 122 MB/s 2023-11-06T03:12:06.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120279 s, 87.2 MB/s 2023-11-06T03:12:06.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.689 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120968 s, 86.7 MB/s 2023-11-06T03:12:06.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.705 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.705 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131311 s, 79.9 MB/s 2023-11-06T03:12:06.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110107 s, 95.2 MB/s 2023-11-06T03:12:06.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107785 s, 97.3 MB/s 2023-11-06T03:12:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889272 s, 118 MB/s 2023-11-06T03:12:06.753 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.753 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.753 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841492 s, 125 MB/s 2023-11-06T03:12:06.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792172 s, 132 MB/s 2023-11-06T03:12:06.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767923 s, 137 MB/s 2023-11-06T03:12:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808156 s, 130 MB/s 2023-11-06T03:12:06.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827363 s, 127 MB/s 2023-11-06T03:12:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.808 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108732 s, 96.4 MB/s 2023-11-06T03:12:06.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.821 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105547 s, 99.3 MB/s 2023-11-06T03:12:06.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.833 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100829 s, 104 MB/s 2023-11-06T03:12:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800109 s, 131 MB/s 2023-11-06T03:12:06.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121569 s, 86.3 MB/s 2023-11-06T03:12:06.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.872 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115868 s, 90.5 MB/s 2023-11-06T03:12:06.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.884 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.884 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100889 s, 104 MB/s 2023-11-06T03:12:06.896 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.896 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.896 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101829 s, 103 MB/s 2023-11-06T03:12:06.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110548 s, 94.9 MB/s 2023-11-06T03:12:06.919 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.920 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813881 s, 129 MB/s 2023-11-06T03:12:06.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.932 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100704 s, 104 MB/s 2023-11-06T03:12:06.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110523 s, 94.9 MB/s 2023-11-06T03:12:06.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122973 s, 85.3 MB/s 2023-11-06T03:12:06.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:06.972 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995197 s, 105 MB/s 2023-11-06T03:12:06.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:06.973 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:06.974 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:06.974 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.2 2023-11-06T03:12:06.974 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:06.975 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:06.975 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:06.975 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.039 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010331 s, 101 MB/s 2023-11-06T03:12:07.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.052 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105312 s, 99.6 MB/s 2023-11-06T03:12:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904321 s, 116 MB/s 2023-11-06T03:12:07.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.078 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119434 s, 87.8 MB/s 2023-11-06T03:12:07.090 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.090 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.090 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103826 s, 101 MB/s 2023-11-06T03:12:07.104 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.105 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120911 s, 86.7 MB/s 2023-11-06T03:12:07.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.119 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012305 s, 85.2 MB/s 2023-11-06T03:12:07.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120805 s, 86.8 MB/s 2023-11-06T03:12:07.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.147 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.147 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108421 s, 96.7 MB/s 2023-11-06T03:12:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.161 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121012 s, 86.7 MB/s 2023-11-06T03:12:07.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106331 s, 98.6 MB/s 2023-11-06T03:12:07.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122997 s, 85.3 MB/s 2023-11-06T03:12:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920519 s, 114 MB/s 2023-11-06T03:12:07.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128793 s, 81.4 MB/s 2023-11-06T03:12:07.229 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.229 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.229 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827208 s, 127 MB/s 2023-11-06T03:12:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113279 s, 92.6 MB/s 2023-11-06T03:12:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.258 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130545 s, 80.3 MB/s 2023-11-06T03:12:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.275 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135687 s, 77.3 MB/s 2023-11-06T03:12:07.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012887 s, 81.4 MB/s 2023-11-06T03:12:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127393 s, 82.3 MB/s 2023-11-06T03:12:07.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.321 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139067 s, 75.4 MB/s 2023-11-06T03:12:07.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.336 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126871 s, 82.6 MB/s 2023-11-06T03:12:07.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.350 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118863 s, 88.2 MB/s 2023-11-06T03:12:07.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013864 s, 75.6 MB/s 2023-11-06T03:12:07.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.379 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101547 s, 103 MB/s 2023-11-06T03:12:07.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.391 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974113 s, 108 MB/s 2023-11-06T03:12:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.406 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118055 s, 88.8 MB/s 2023-11-06T03:12:07.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124471 s, 84.2 MB/s 2023-11-06T03:12:07.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112938 s, 92.8 MB/s 2023-11-06T03:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914906 s, 115 MB/s 2023-11-06T03:12:07.458 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108358 s, 96.8 MB/s 2023-11-06T03:12:07.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.472 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106384 s, 98.6 MB/s 2023-11-06T03:12:07.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:07.473 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:07.473 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:07.474 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.3 2023-11-06T03:12:07.474 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:07.474 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:07.475 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:07.475 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:07.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.536 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815337 s, 129 MB/s 2023-11-06T03:12:07.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012486 s, 84.0 MB/s 2023-11-06T03:12:07.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819428 s, 128 MB/s 2023-11-06T03:12:07.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121989 s, 86.0 MB/s 2023-11-06T03:12:07.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.589 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118612 s, 88.4 MB/s 2023-11-06T03:12:07.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112944 s, 92.8 MB/s 2023-11-06T03:12:07.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.617 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118501 s, 88.5 MB/s 2023-11-06T03:12:07.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.633 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128868 s, 81.4 MB/s 2023-11-06T03:12:07.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.645 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868798 s, 121 MB/s 2023-11-06T03:12:07.658 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.659 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115982 s, 90.4 MB/s 2023-11-06T03:12:07.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.673 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111169 s, 94.3 MB/s 2023-11-06T03:12:07.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120636 s, 86.9 MB/s 2023-11-06T03:12:07.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847705 s, 124 MB/s 2023-11-06T03:12:07.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953905 s, 110 MB/s 2023-11-06T03:12:07.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.725 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126288 s, 83.0 MB/s 2023-11-06T03:12:07.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.740 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121836 s, 86.1 MB/s 2023-11-06T03:12:07.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.755 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.755 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119903 s, 87.5 MB/s 2023-11-06T03:12:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118372 s, 88.6 MB/s 2023-11-06T03:12:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923194 s, 114 MB/s 2023-11-06T03:12:07.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988747 s, 106 MB/s 2023-11-06T03:12:07.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.807 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104877 s, 100 MB/s 2023-11-06T03:12:07.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.823 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012773 s, 82.1 MB/s 2023-11-06T03:12:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.837 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120285 s, 87.2 MB/s 2023-11-06T03:12:07.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013303 s, 78.8 MB/s 2023-11-06T03:12:07.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.868 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121859 s, 86.0 MB/s 2023-11-06T03:12:07.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.882 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010407 s, 101 MB/s 2023-11-06T03:12:07.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.895 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104583 s, 100 MB/s 2023-11-06T03:12:07.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.911 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130151 s, 80.6 MB/s 2023-11-06T03:12:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.928 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141246 s, 74.2 MB/s 2023-11-06T03:12:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121033 s, 86.6 MB/s 2023-11-06T03:12:07.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951334 s, 110 MB/s 2023-11-06T03:12:07.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:07.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:07.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136935 s, 76.6 MB/s 2023-11-06T03:12:07.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:07.976 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:07.976 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:07.976 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.4 2023-11-06T03:12:07.977 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:07.977 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:07.977 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:07.978 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:08.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128314 s, 81.7 MB/s 2023-11-06T03:12:08.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.065 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131559 s, 79.7 MB/s 2023-11-06T03:12:08.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.080 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.080 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127689 s, 82.1 MB/s 2023-11-06T03:12:08.094 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.094 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.094 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112809 s, 93.0 MB/s 2023-11-06T03:12:08.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.111 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140304 s, 74.7 MB/s 2023-11-06T03:12:08.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012435 s, 84.3 MB/s 2023-11-06T03:12:08.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135256 s, 77.5 MB/s 2023-11-06T03:12:08.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116902 s, 89.7 MB/s 2023-11-06T03:12:08.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107489 s, 97.6 MB/s 2023-11-06T03:12:08.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.183 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101615 s, 103 MB/s 2023-11-06T03:12:08.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012644 s, 82.9 MB/s 2023-11-06T03:12:08.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.215 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.215 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131662 s, 79.6 MB/s 2023-11-06T03:12:08.229 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.229 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.229 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116914 s, 89.7 MB/s 2023-11-06T03:12:08.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.242 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010407 s, 101 MB/s 2023-11-06T03:12:08.258 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.259 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137305 s, 76.4 MB/s 2023-11-06T03:12:08.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104485 s, 100 MB/s 2023-11-06T03:12:08.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104759 s, 100 MB/s 2023-11-06T03:12:08.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.298 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953481 s, 110 MB/s 2023-11-06T03:12:08.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.312 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110588 s, 94.8 MB/s 2023-11-06T03:12:08.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.328 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135824 s, 77.2 MB/s 2023-11-06T03:12:08.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136574 s, 76.8 MB/s 2023-11-06T03:12:08.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014127 s, 74.2 MB/s 2023-11-06T03:12:08.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.378 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141279 s, 74.2 MB/s 2023-11-06T03:12:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138544 s, 75.7 MB/s 2023-11-06T03:12:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117187 s, 89.5 MB/s 2023-11-06T03:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013009 s, 80.6 MB/s 2023-11-06T03:12:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.439 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970747 s, 108 MB/s 2023-11-06T03:12:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.451 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101349 s, 103 MB/s 2023-11-06T03:12:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.466 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113162 s, 92.7 MB/s 2023-11-06T03:12:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.480 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118309 s, 88.6 MB/s 2023-11-06T03:12:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012891 s, 81.3 MB/s 2023-11-06T03:12:08.511 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.511 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.511 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013031 s, 80.5 MB/s 2023-11-06T03:12:08.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:08.513 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:08.513 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:08.514 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.5 2023-11-06T03:12:08.514 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:08.514 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:08.515 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:08.515 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:08.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.577 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909451 s, 115 MB/s 2023-11-06T03:12:08.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112633 s, 93.1 MB/s 2023-11-06T03:12:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116437 s, 90.1 MB/s 2023-11-06T03:12:08.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011535 s, 90.9 MB/s 2023-11-06T03:12:08.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.633 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105779 s, 99.1 MB/s 2023-11-06T03:12:08.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.646 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.646 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104571 s, 100 MB/s 2023-11-06T03:12:08.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809864 s, 129 MB/s 2023-11-06T03:12:08.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.673 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140019 s, 74.9 MB/s 2023-11-06T03:12:08.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.683 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811498 s, 129 MB/s 2023-11-06T03:12:08.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.697 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011561 s, 90.7 MB/s 2023-11-06T03:12:08.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103859 s, 101 MB/s 2023-11-06T03:12:08.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832896 s, 126 MB/s 2023-11-06T03:12:08.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122698 s, 85.5 MB/s 2023-11-06T03:12:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010766 s, 97.4 MB/s 2023-11-06T03:12:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100846 s, 104 MB/s 2023-11-06T03:12:08.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.773 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848541 s, 124 MB/s 2023-11-06T03:12:08.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.788 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124076 s, 84.5 MB/s 2023-11-06T03:12:08.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011318 s, 92.6 MB/s 2023-11-06T03:12:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.812 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.813 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830661 s, 126 MB/s 2023-11-06T03:12:08.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.823 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778696 s, 135 MB/s 2023-11-06T03:12:08.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.836 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011223 s, 93.4 MB/s 2023-11-06T03:12:08.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.848 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862401 s, 122 MB/s 2023-11-06T03:12:08.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.860 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101946 s, 103 MB/s 2023-11-06T03:12:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121392 s, 86.4 MB/s 2023-11-06T03:12:08.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.890 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119042 s, 88.1 MB/s 2023-11-06T03:12:08.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.903 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.903 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109703 s, 95.6 MB/s 2023-11-06T03:12:08.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.916 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109866 s, 95.4 MB/s 2023-11-06T03:12:08.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.933 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138694 s, 75.6 MB/s 2023-11-06T03:12:08.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107264 s, 97.8 MB/s 2023-11-06T03:12:08.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.959 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104211 s, 101 MB/s 2023-11-06T03:12:08.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.969 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797191 s, 132 MB/s 2023-11-06T03:12:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:08.979 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820239 s, 128 MB/s 2023-11-06T03:12:08.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:08.981 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:08.981 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:08.982 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.6 2023-11-06T03:12:08.982 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:08.982 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:08.983 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:08.983 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:09.050 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.051 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129661 s, 80.9 MB/s 2023-11-06T03:12:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.062 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092633 s, 113 MB/s 2023-11-06T03:12:09.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.074 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928603 s, 113 MB/s 2023-11-06T03:12:09.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.085 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877265 s, 120 MB/s 2023-11-06T03:12:09.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106713 s, 98.3 MB/s 2023-11-06T03:12:09.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.111 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114136 s, 91.9 MB/s 2023-11-06T03:12:09.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.126 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120841 s, 86.8 MB/s 2023-11-06T03:12:09.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115295 s, 90.9 MB/s 2023-11-06T03:12:09.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.152 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100661 s, 104 MB/s 2023-11-06T03:12:09.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848889 s, 124 MB/s 2023-11-06T03:12:09.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011116 s, 94.3 MB/s 2023-11-06T03:12:09.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.191 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123209 s, 85.1 MB/s 2023-11-06T03:12:09.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104973 s, 99.9 MB/s 2023-11-06T03:12:09.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.216 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105665 s, 99.2 MB/s 2023-11-06T03:12:09.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134903 s, 77.7 MB/s 2023-11-06T03:12:09.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120125 s, 87.3 MB/s 2023-11-06T03:12:09.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.260 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112549 s, 93.2 MB/s 2023-11-06T03:12:09.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769417 s, 136 MB/s 2023-11-06T03:12:09.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132403 s, 79.2 MB/s 2023-11-06T03:12:09.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.295 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795225 s, 132 MB/s 2023-11-06T03:12:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.312 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141529 s, 74.1 MB/s 2023-11-06T03:12:09.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.324 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994571 s, 105 MB/s 2023-11-06T03:12:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109621 s, 95.7 MB/s 2023-11-06T03:12:09.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.349 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863985 s, 121 MB/s 2023-11-06T03:12:09.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.360 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089493 s, 117 MB/s 2023-11-06T03:12:09.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.374 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110456 s, 94.9 MB/s 2023-11-06T03:12:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.390 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135907 s, 77.2 MB/s 2023-11-06T03:12:09.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.407 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.407 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138659 s, 75.6 MB/s 2023-11-06T03:12:09.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142343 s, 73.7 MB/s 2023-11-06T03:12:09.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.439 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129777 s, 80.8 MB/s 2023-11-06T03:12:09.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105693 s, 99.2 MB/s 2023-11-06T03:12:09.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.470 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139057 s, 75.4 MB/s 2023-11-06T03:12:09.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:09.471 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:09.471 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:09.472 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.7 2023-11-06T03:12:09.472 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:09.473 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:09.473 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:09.473 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.540 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132029 s, 79.4 MB/s 2023-11-06T03:12:09.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013782 s, 76.1 MB/s 2023-11-06T03:12:09.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108394 s, 96.7 MB/s 2023-11-06T03:12:09.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101453 s, 103 MB/s 2023-11-06T03:12:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.599 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134924 s, 77.7 MB/s 2023-11-06T03:12:09.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122878 s, 85.3 MB/s 2023-11-06T03:12:09.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.631 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123165 s, 85.1 MB/s 2023-11-06T03:12:09.644 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.644 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.644 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103425 s, 101 MB/s 2023-11-06T03:12:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.658 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103042 s, 102 MB/s 2023-11-06T03:12:09.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119164 s, 88.0 MB/s 2023-11-06T03:12:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.688 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131346 s, 79.8 MB/s 2023-11-06T03:12:09.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918401 s, 114 MB/s 2023-11-06T03:12:09.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121475 s, 86.3 MB/s 2023-11-06T03:12:09.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113622 s, 92.3 MB/s 2023-11-06T03:12:09.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131982 s, 79.4 MB/s 2023-11-06T03:12:09.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114557 s, 91.5 MB/s 2023-11-06T03:12:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133039 s, 78.8 MB/s 2023-11-06T03:12:09.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.787 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985497 s, 106 MB/s 2023-11-06T03:12:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012474 s, 84.1 MB/s 2023-11-06T03:12:09.816 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.816 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.816 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109499 s, 95.8 MB/s 2023-11-06T03:12:09.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.832 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012583 s, 83.3 MB/s 2023-11-06T03:12:09.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.846 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117732 s, 89.1 MB/s 2023-11-06T03:12:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128318 s, 81.7 MB/s 2023-11-06T03:12:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127614 s, 82.2 MB/s 2023-11-06T03:12:09.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.892 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115282 s, 91.0 MB/s 2023-11-06T03:12:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104636 s, 100 MB/s 2023-11-06T03:12:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.920 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122154 s, 85.8 MB/s 2023-11-06T03:12:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013044 s, 80.4 MB/s 2023-11-06T03:12:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.949 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102602 s, 102 MB/s 2023-11-06T03:12:09.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.964 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124262 s, 84.4 MB/s 2023-11-06T03:12:09.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103433 s, 101 MB/s 2023-11-06T03:12:09.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:09.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:09.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122102 s, 85.9 MB/s 2023-11-06T03:12:09.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:09.995 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:09.995 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:09.995 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.8 2023-11-06T03:12:09.996 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:09.996 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:09.996 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:09.997 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:10.062 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121471 s, 86.3 MB/s 2023-11-06T03:12:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120394 s, 87.1 MB/s 2023-11-06T03:12:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122161 s, 85.8 MB/s 2023-11-06T03:12:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134823 s, 77.8 MB/s 2023-11-06T03:12:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979003 s, 107 MB/s 2023-11-06T03:12:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127092 s, 82.5 MB/s 2023-11-06T03:12:10.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.152 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970205 s, 108 MB/s 2023-11-06T03:12:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.166 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114756 s, 91.4 MB/s 2023-11-06T03:12:10.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.181 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121095 s, 86.6 MB/s 2023-11-06T03:12:10.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954176 s, 110 MB/s 2023-11-06T03:12:10.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.208 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116166 s, 90.3 MB/s 2023-11-06T03:12:10.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110907 s, 94.5 MB/s 2023-11-06T03:12:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111164 s, 94.3 MB/s 2023-11-06T03:12:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012893 s, 81.3 MB/s 2023-11-06T03:12:10.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129964 s, 80.7 MB/s 2023-11-06T03:12:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121065 s, 86.6 MB/s 2023-11-06T03:12:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108051 s, 97.0 MB/s 2023-11-06T03:12:10.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144177 s, 72.7 MB/s 2023-11-06T03:12:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.331 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130563 s, 80.3 MB/s 2023-11-06T03:12:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.346 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122196 s, 85.8 MB/s 2023-11-06T03:12:10.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.363 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123858 s, 84.7 MB/s 2023-11-06T03:12:10.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.382 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157335 s, 66.6 MB/s 2023-11-06T03:12:10.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.398 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126116 s, 83.1 MB/s 2023-11-06T03:12:10.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.414 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127128 s, 82.5 MB/s 2023-11-06T03:12:10.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.430 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140155 s, 74.8 MB/s 2023-11-06T03:12:10.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.447 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.447 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126619 s, 82.8 MB/s 2023-11-06T03:12:10.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.463 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131392 s, 79.8 MB/s 2023-11-06T03:12:10.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.479 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121215 s, 86.5 MB/s 2023-11-06T03:12:10.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.493 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105985 s, 98.9 MB/s 2023-11-06T03:12:10.508 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.509 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129481 s, 81.0 MB/s 2023-11-06T03:12:10.521 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.522 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990996 s, 106 MB/s 2023-11-06T03:12:10.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101354 s, 103 MB/s 2023-11-06T03:12:10.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:10.536 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:10.537 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:10.537 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.9 2023-11-06T03:12:10.537 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:10.538 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:10.538 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:10.538 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:10.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.609 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137147 s, 76.5 MB/s 2023-11-06T03:12:10.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.623 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100421 s, 104 MB/s 2023-11-06T03:12:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.636 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107175 s, 97.8 MB/s 2023-11-06T03:12:10.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.649 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973723 s, 108 MB/s 2023-11-06T03:12:10.661 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.662 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103731 s, 101 MB/s 2023-11-06T03:12:10.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.677 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124832 s, 84.0 MB/s 2023-11-06T03:12:10.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.693 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126743 s, 82.7 MB/s 2023-11-06T03:12:10.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.707 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111162 s, 94.3 MB/s 2023-11-06T03:12:10.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115432 s, 90.8 MB/s 2023-11-06T03:12:10.735 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.735 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117126 s, 89.5 MB/s 2023-11-06T03:12:10.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.750 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123352 s, 85.0 MB/s 2023-11-06T03:12:10.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095316 s, 110 MB/s 2023-11-06T03:12:10.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.776 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111281 s, 94.2 MB/s 2023-11-06T03:12:10.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.791 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.791 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118332 s, 88.6 MB/s 2023-11-06T03:12:10.805 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.805 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.805 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116889 s, 89.7 MB/s 2023-11-06T03:12:10.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.820 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119304 s, 87.9 MB/s 2023-11-06T03:12:10.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.835 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123351 s, 85.0 MB/s 2023-11-06T03:12:10.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121517 s, 86.3 MB/s 2023-11-06T03:12:10.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.864 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114263 s, 91.8 MB/s 2023-11-06T03:12:10.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126276 s, 83.0 MB/s 2023-11-06T03:12:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011834 s, 88.6 MB/s 2023-11-06T03:12:10.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.909 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127227 s, 82.4 MB/s 2023-11-06T03:12:10.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105683 s, 99.2 MB/s 2023-11-06T03:12:10.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.937 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120512 s, 87.0 MB/s 2023-11-06T03:12:10.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.950 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010096 s, 104 MB/s 2023-11-06T03:12:10.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.963 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990493 s, 106 MB/s 2023-11-06T03:12:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101523 s, 103 MB/s 2023-11-06T03:12:10.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:10.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:10.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012351 s, 84.9 MB/s 2023-11-06T03:12:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124983 s, 83.9 MB/s 2023-11-06T03:12:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.020 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107714 s, 97.3 MB/s 2023-11-06T03:12:11.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.035 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123289 s, 85.1 MB/s 2023-11-06T03:12:11.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.051 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131143 s, 80.0 MB/s 2023-11-06T03:12:11.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:11.053 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:11.053 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:11.053 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.10 2023-11-06T03:12:11.054 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:11.054 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:11.054 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:11.055 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:11.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130291 s, 80.5 MB/s 2023-11-06T03:12:11.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.136 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106424 s, 98.5 MB/s 2023-11-06T03:12:11.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.154 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014835 s, 70.7 MB/s 2023-11-06T03:12:11.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134555 s, 77.9 MB/s 2023-11-06T03:12:11.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101435 s, 103 MB/s 2023-11-06T03:12:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136464 s, 76.8 MB/s 2023-11-06T03:12:11.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.217 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130957 s, 80.1 MB/s 2023-11-06T03:12:11.228 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:12:11.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:12:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120889 s, 86.7 MB/s 2023-11-06T03:12:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.247 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117526 s, 89.2 MB/s 2023-11-06T03:12:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.261 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108577 s, 96.6 MB/s 2023-11-06T03:12:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.274 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101745 s, 103 MB/s 2023-11-06T03:12:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.289 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119653 s, 87.6 MB/s 2023-11-06T03:12:11.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.302 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009479 s, 111 MB/s 2023-11-06T03:12:11.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.317 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121722 s, 86.1 MB/s 2023-11-06T03:12:11.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125091 s, 83.8 MB/s 2023-11-06T03:12:11.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895695 s, 117 MB/s 2023-11-06T03:12:11.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.357 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969977 s, 108 MB/s 2023-11-06T03:12:11.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.369 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.369 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851834 s, 123 MB/s 2023-11-06T03:12:11.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119753 s, 87.6 MB/s 2023-11-06T03:12:11.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.396 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905287 s, 116 MB/s 2023-11-06T03:12:11.407 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.407 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.408 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862706 s, 122 MB/s 2023-11-06T03:12:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116886 s, 89.7 MB/s 2023-11-06T03:12:11.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111335 s, 94.2 MB/s 2023-11-06T03:12:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088033 s, 119 MB/s 2023-11-06T03:12:11.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125295 s, 83.7 MB/s 2023-11-06T03:12:11.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.481 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135807 s, 77.2 MB/s 2023-11-06T03:12:11.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881685 s, 119 MB/s 2023-11-06T03:12:11.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.505 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919748 s, 114 MB/s 2023-11-06T03:12:11.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122704 s, 85.5 MB/s 2023-11-06T03:12:11.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137216 s, 76.4 MB/s 2023-11-06T03:12:11.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.549 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877001 s, 120 MB/s 2023-11-06T03:12:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115769 s, 90.6 MB/s 2023-11-06T03:12:11.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:11.565 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:11.565 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:11.566 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.11 2023-11-06T03:12:11.566 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:11.566 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:11.567 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:11.567 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:11.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.643 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136698 s, 76.7 MB/s 2023-11-06T03:12:11.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102374 s, 102 MB/s 2023-11-06T03:12:11.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.674 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133967 s, 78.3 MB/s 2023-11-06T03:12:11.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.689 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121163 s, 86.5 MB/s 2023-11-06T03:12:11.705 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.705 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.705 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129093 s, 81.2 MB/s 2023-11-06T03:12:11.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.720 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114488 s, 91.6 MB/s 2023-11-06T03:12:11.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.733 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974411 s, 108 MB/s 2023-11-06T03:12:11.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983329 s, 107 MB/s 2023-11-06T03:12:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.760 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964352 s, 109 MB/s 2023-11-06T03:12:11.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.775 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117485 s, 89.3 MB/s 2023-11-06T03:12:11.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.787 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950313 s, 110 MB/s 2023-11-06T03:12:11.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.803 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133919 s, 78.3 MB/s 2023-11-06T03:12:11.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120833 s, 86.8 MB/s 2023-11-06T03:12:11.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.831 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935904 s, 112 MB/s 2023-11-06T03:12:11.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.844 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102184 s, 103 MB/s 2023-11-06T03:12:11.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.856 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953254 s, 110 MB/s 2023-11-06T03:12:11.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.870 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117195 s, 89.5 MB/s 2023-11-06T03:12:11.885 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.885 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.886 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122053 s, 85.9 MB/s 2023-11-06T03:12:11.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.899 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109005 s, 96.2 MB/s 2023-11-06T03:12:11.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011328 s, 92.6 MB/s 2023-11-06T03:12:11.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.930 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126601 s, 82.8 MB/s 2023-11-06T03:12:11.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105951 s, 99.0 MB/s 2023-11-06T03:12:11.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.956 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902486 s, 116 MB/s 2023-11-06T03:12:11.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.971 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120226 s, 87.2 MB/s 2023-11-06T03:12:11.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:11.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:11.985 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103054 s, 102 MB/s 2023-11-06T03:12:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119356 s, 87.9 MB/s 2023-11-06T03:12:12.012 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.013 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.013 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100802 s, 104 MB/s 2023-11-06T03:12:12.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.028 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123986 s, 84.6 MB/s 2023-11-06T03:12:12.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.046 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140238 s, 74.8 MB/s 2023-11-06T03:12:12.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115651 s, 90.7 MB/s 2023-11-06T03:12:12.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.075 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122456 s, 85.6 MB/s 2023-11-06T03:12:12.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.090 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011756 s, 89.2 MB/s 2023-11-06T03:12:12.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:12.091 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:12.092 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:12.092 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.12 2023-11-06T03:12:12.092 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:12.093 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:12.093 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:12.093 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:12.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943823 s, 111 MB/s 2023-11-06T03:12:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941523 s, 111 MB/s 2023-11-06T03:12:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.181 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110796 s, 94.6 MB/s 2023-11-06T03:12:12.197 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.197 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.198 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139208 s, 75.3 MB/s 2023-11-06T03:12:12.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.228 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898474 s, 117 MB/s 2023-11-06T03:12:12.241 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.241 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.241 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113265 s, 92.6 MB/s 2023-11-06T03:12:12.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.257 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120338 s, 87.1 MB/s 2023-11-06T03:12:12.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936408 s, 112 MB/s 2023-11-06T03:12:12.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111618 s, 93.9 MB/s 2023-11-06T03:12:12.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.294 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865788 s, 121 MB/s 2023-11-06T03:12:12.309 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.309 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.309 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131416 s, 79.8 MB/s 2023-11-06T03:12:12.322 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111479 s, 94.1 MB/s 2023-11-06T03:12:12.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122381 s, 85.7 MB/s 2023-11-06T03:12:12.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.352 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118903 s, 88.2 MB/s 2023-11-06T03:12:12.365 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.366 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110451 s, 94.9 MB/s 2023-11-06T03:12:12.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.381 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012656 s, 82.9 MB/s 2023-11-06T03:12:12.392 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.393 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918012 s, 114 MB/s 2023-11-06T03:12:12.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012883 s, 81.4 MB/s 2023-11-06T03:12:12.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095077 s, 110 MB/s 2023-11-06T03:12:12.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126369 s, 83.0 MB/s 2023-11-06T03:12:12.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130538 s, 80.3 MB/s 2023-11-06T03:12:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928772 s, 113 MB/s 2023-11-06T03:12:12.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113541 s, 92.4 MB/s 2023-11-06T03:12:12.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120607 s, 86.9 MB/s 2023-11-06T03:12:12.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.505 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941601 s, 111 MB/s 2023-11-06T03:12:12.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122327 s, 85.7 MB/s 2023-11-06T03:12:12.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.534 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118299 s, 88.6 MB/s 2023-11-06T03:12:12.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.549 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124308 s, 84.4 MB/s 2023-11-06T03:12:12.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.562 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958782 s, 109 MB/s 2023-11-06T03:12:12.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.578 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.578 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013362 s, 78.5 MB/s 2023-11-06T03:12:12.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123622 s, 84.8 MB/s 2023-11-06T03:12:12.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011646 s, 90.0 MB/s 2023-11-06T03:12:12.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:12.609 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:12.609 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:12.610 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.13 2023-11-06T03:12:12.610 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:12.611 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:12.611 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:12.611 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:12.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.678 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123922 s, 84.6 MB/s 2023-11-06T03:12:12.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133404 s, 78.6 MB/s 2023-11-06T03:12:12.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.708 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115712 s, 90.6 MB/s 2023-11-06T03:12:12.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954 s, 110 MB/s 2023-11-06T03:12:12.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132618 s, 79.1 MB/s 2023-11-06T03:12:12.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955639 s, 110 MB/s 2023-11-06T03:12:12.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.764 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.764 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123937 s, 84.6 MB/s 2023-11-06T03:12:12.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.776 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964421 s, 109 MB/s 2023-11-06T03:12:12.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.791 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.791 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119592 s, 87.7 MB/s 2023-11-06T03:12:12.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.803 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926332 s, 113 MB/s 2023-11-06T03:12:12.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.817 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120399 s, 87.1 MB/s 2023-11-06T03:12:12.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.832 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119676 s, 87.6 MB/s 2023-11-06T03:12:12.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.847 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116915 s, 89.7 MB/s 2023-11-06T03:12:12.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.860 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111588 s, 94.0 MB/s 2023-11-06T03:12:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972442 s, 108 MB/s 2023-11-06T03:12:12.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.886 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107693 s, 97.4 MB/s 2023-11-06T03:12:12.897 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.898 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944307 s, 111 MB/s 2023-11-06T03:12:12.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.908 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805588 s, 130 MB/s 2023-11-06T03:12:12.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.921 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106233 s, 98.7 MB/s 2023-11-06T03:12:12.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829987 s, 126 MB/s 2023-11-06T03:12:12.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103112 s, 102 MB/s 2023-11-06T03:12:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105128 s, 99.7 MB/s 2023-11-06T03:12:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103941 s, 101 MB/s 2023-11-06T03:12:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126037 s, 83.2 MB/s 2023-11-06T03:12:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114333 s, 91.7 MB/s 2023-11-06T03:12:13.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811424 s, 129 MB/s 2023-11-06T03:12:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778086 s, 135 MB/s 2023-11-06T03:12:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134376 s, 78.0 MB/s 2023-11-06T03:12:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104676 s, 100 MB/s 2023-11-06T03:12:13.057 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.058 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837448 s, 125 MB/s 2023-11-06T03:12:13.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771016 s, 136 MB/s 2023-11-06T03:12:13.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.081 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116491 s, 90.0 MB/s 2023-11-06T03:12:13.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:13.083 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:13.083 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:13.084 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.14 2023-11-06T03:12:13.084 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:13.084 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:13.085 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:13.085 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:13.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.145 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954915 s, 110 MB/s 2023-11-06T03:12:13.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116902 s, 89.7 MB/s 2023-11-06T03:12:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823248 s, 127 MB/s 2023-11-06T03:12:13.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116456 s, 90.0 MB/s 2023-11-06T03:12:13.195 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102326 s, 102 MB/s 2023-11-06T03:12:13.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119622 s, 87.7 MB/s 2023-11-06T03:12:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807481 s, 130 MB/s 2023-11-06T03:12:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101306 s, 104 MB/s 2023-11-06T03:12:13.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125653 s, 83.5 MB/s 2023-11-06T03:12:13.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.283 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110904 s, 94.5 MB/s 2023-11-06T03:12:13.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115794 s, 90.6 MB/s 2023-11-06T03:12:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.308 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081655 s, 128 MB/s 2023-11-06T03:12:13.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.320 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102691 s, 102 MB/s 2023-11-06T03:12:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.331 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803884 s, 130 MB/s 2023-11-06T03:12:13.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148117 s, 70.8 MB/s 2023-11-06T03:12:13.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.361 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111826 s, 93.8 MB/s 2023-11-06T03:12:13.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.375 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112649 s, 93.1 MB/s 2023-11-06T03:12:13.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.385 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784967 s, 134 MB/s 2023-11-06T03:12:13.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928736 s, 113 MB/s 2023-11-06T03:12:13.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.408 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877262 s, 120 MB/s 2023-11-06T03:12:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142854 s, 73.4 MB/s 2023-11-06T03:12:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119322 s, 87.9 MB/s 2023-11-06T03:12:13.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.449 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806123 s, 130 MB/s 2023-11-06T03:12:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122292 s, 85.7 MB/s 2023-11-06T03:12:13.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.474 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850442 s, 123 MB/s 2023-11-06T03:12:13.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.491 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.491 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131045 s, 80.0 MB/s 2023-11-06T03:12:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112874 s, 92.9 MB/s 2023-11-06T03:12:13.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.523 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144497 s, 72.6 MB/s 2023-11-06T03:12:13.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110681 s, 94.7 MB/s 2023-11-06T03:12:13.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.555 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143035 s, 73.3 MB/s 2023-11-06T03:12:13.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120553 s, 87.0 MB/s 2023-11-06T03:12:13.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.587 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134003 s, 78.3 MB/s 2023-11-06T03:12:13.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:13.589 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:13.589 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:13.589 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.15 2023-11-06T03:12:13.590 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:13.590 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:13.590 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:13.591 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:13.658 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.659 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125817 s, 83.3 MB/s 2023-11-06T03:12:13.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.673 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113104 s, 92.7 MB/s 2023-11-06T03:12:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101596 s, 103 MB/s 2023-11-06T03:12:13.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010986 s, 95.4 MB/s 2023-11-06T03:12:13.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843877 s, 124 MB/s 2023-11-06T03:12:13.723 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.723 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106854 s, 98.1 MB/s 2023-11-06T03:12:13.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888876 s, 118 MB/s 2023-11-06T03:12:13.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890938 s, 118 MB/s 2023-11-06T03:12:13.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910456 s, 115 MB/s 2023-11-06T03:12:13.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914134 s, 115 MB/s 2023-11-06T03:12:13.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.785 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012686 s, 82.7 MB/s 2023-11-06T03:12:13.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.803 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147197 s, 71.2 MB/s 2023-11-06T03:12:13.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959335 s, 109 MB/s 2023-11-06T03:12:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.829 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114561 s, 91.5 MB/s 2023-11-06T03:12:13.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884412 s, 119 MB/s 2023-11-06T03:12:13.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120175 s, 87.3 MB/s 2023-11-06T03:12:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110018 s, 95.3 MB/s 2023-11-06T03:12:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107692 s, 97.4 MB/s 2023-11-06T03:12:13.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938847 s, 112 MB/s 2023-11-06T03:12:13.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.908 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120292 s, 87.2 MB/s 2023-11-06T03:12:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955697 s, 110 MB/s 2023-11-06T03:12:13.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.932 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899497 s, 117 MB/s 2023-11-06T03:12:13.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011072 s, 94.7 MB/s 2023-11-06T03:12:13.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116558 s, 90.0 MB/s 2023-11-06T03:12:13.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.973 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103666 s, 101 MB/s 2023-11-06T03:12:13.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:13.987 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011577 s, 90.6 MB/s 2023-11-06T03:12:14.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.002 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124663 s, 84.1 MB/s 2023-11-06T03:12:14.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133687 s, 78.4 MB/s 2023-11-06T03:12:14.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.035 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146734 s, 71.5 MB/s 2023-11-06T03:12:14.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987565 s, 106 MB/s 2023-11-06T03:12:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942444 s, 111 MB/s 2023-11-06T03:12:14.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.075 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120393 s, 87.1 MB/s 2023-11-06T03:12:14.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:14.076 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:14.077 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:14.077 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.16 2023-11-06T03:12:14.078 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:14.078 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:14.078 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:14.079 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:14.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.138 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827839 s, 127 MB/s 2023-11-06T03:12:14.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.151 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113276 s, 92.6 MB/s 2023-11-06T03:12:14.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834885 s, 126 MB/s 2023-11-06T03:12:14.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.177 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128377 s, 81.7 MB/s 2023-11-06T03:12:14.192 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126349 s, 83.0 MB/s 2023-11-06T03:12:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119189 s, 88.0 MB/s 2023-11-06T03:12:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011965 s, 87.6 MB/s 2023-11-06T03:12:14.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.250 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107175 s, 97.8 MB/s 2023-11-06T03:12:14.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.265 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121144 s, 86.6 MB/s 2023-11-06T03:12:14.275 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.275 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.276 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085754 s, 122 MB/s 2023-11-06T03:12:14.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789016 s, 133 MB/s 2023-11-06T03:12:14.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.298 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108671 s, 96.5 MB/s 2023-11-06T03:12:14.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.308 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797274 s, 132 MB/s 2023-11-06T03:12:14.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.324 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131388 s, 79.8 MB/s 2023-11-06T03:12:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.336 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109428 s, 95.8 MB/s 2023-11-06T03:12:14.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.352 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126314 s, 83.0 MB/s 2023-11-06T03:12:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.363 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944839 s, 111 MB/s 2023-11-06T03:12:14.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.377 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119016 s, 88.1 MB/s 2023-11-06T03:12:14.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.388 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916338 s, 114 MB/s 2023-11-06T03:12:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127418 s, 82.3 MB/s 2023-11-06T03:12:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131648 s, 79.7 MB/s 2023-11-06T03:12:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108751 s, 96.4 MB/s 2023-11-06T03:12:14.442 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.443 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811141 s, 129 MB/s 2023-11-06T03:12:14.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.454 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855657 s, 123 MB/s 2023-11-06T03:12:14.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.467 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105118 s, 99.8 MB/s 2023-11-06T03:12:14.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.483 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013571 s, 77.3 MB/s 2023-11-06T03:12:14.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.498 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130727 s, 80.2 MB/s 2023-11-06T03:12:14.511 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.511 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.511 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111951 s, 93.7 MB/s 2023-11-06T03:12:14.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.526 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129793 s, 80.8 MB/s 2023-11-06T03:12:14.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846188 s, 124 MB/s 2023-11-06T03:12:14.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808204 s, 130 MB/s 2023-11-06T03:12:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.557 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765695 s, 137 MB/s 2023-11-06T03:12:14.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:14.558 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:14.559 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:14.559 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.17 2023-11-06T03:12:14.559 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:14.560 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:14.560 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:14.560 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:14.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.618 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752311 s, 139 MB/s 2023-11-06T03:12:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750334 s, 140 MB/s 2023-11-06T03:12:14.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.640 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104888 s, 100 MB/s 2023-11-06T03:12:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108468 s, 96.7 MB/s 2023-11-06T03:12:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.666 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106518 s, 98.4 MB/s 2023-11-06T03:12:14.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102856 s, 102 MB/s 2023-11-06T03:12:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751187 s, 140 MB/s 2023-11-06T03:12:14.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845117 s, 124 MB/s 2023-11-06T03:12:14.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.711 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100703 s, 104 MB/s 2023-11-06T03:12:14.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082676 s, 127 MB/s 2023-11-06T03:12:14.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102207 s, 103 MB/s 2023-11-06T03:12:14.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772389 s, 136 MB/s 2023-11-06T03:12:14.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120538 s, 87.0 MB/s 2023-11-06T03:12:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076387 s, 137 MB/s 2023-11-06T03:12:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767366 s, 137 MB/s 2023-11-06T03:12:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.787 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770527 s, 136 MB/s 2023-11-06T03:12:14.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.796 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749173 s, 140 MB/s 2023-11-06T03:12:14.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.806 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788038 s, 133 MB/s 2023-11-06T03:12:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123046 s, 85.2 MB/s 2023-11-06T03:12:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077109 s, 136 MB/s 2023-11-06T03:12:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774573 s, 135 MB/s 2023-11-06T03:12:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747788 s, 140 MB/s 2023-11-06T03:12:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106502 s, 98.5 MB/s 2023-11-06T03:12:14.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987134 s, 106 MB/s 2023-11-06T03:12:14.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.887 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114229 s, 91.8 MB/s 2023-11-06T03:12:14.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119782 s, 87.5 MB/s 2023-11-06T03:12:14.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.913 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922729 s, 114 MB/s 2023-11-06T03:12:14.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.928 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124775 s, 84.0 MB/s 2023-11-06T03:12:14.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.941 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115787 s, 90.6 MB/s 2023-11-06T03:12:14.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01254 s, 83.6 MB/s 2023-11-06T03:12:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.966 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.967 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788902 s, 133 MB/s 2023-11-06T03:12:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:14.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:14.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891024 s, 118 MB/s 2023-11-06T03:12:14.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:14.979 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:14.979 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:14.980 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.18 2023-11-06T03:12:14.980 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:14.980 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:14.981 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:14.981 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108893 s, 96.3 MB/s 2023-11-06T03:12:15.057 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131222 s, 79.9 MB/s 2023-11-06T03:12:15.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.068 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832046 s, 126 MB/s 2023-11-06T03:12:15.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.080 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.080 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897638 s, 117 MB/s 2023-11-06T03:12:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.092 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100646 s, 104 MB/s 2023-11-06T03:12:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.103 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898468 s, 117 MB/s 2023-11-06T03:12:15.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.116 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104694 s, 100 MB/s 2023-11-06T03:12:15.129 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.130 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119132 s, 88.0 MB/s 2023-11-06T03:12:15.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804766 s, 130 MB/s 2023-11-06T03:12:15.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.152 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998629 s, 105 MB/s 2023-11-06T03:12:15.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.168 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130818 s, 80.2 MB/s 2023-11-06T03:12:15.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.178 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787496 s, 133 MB/s 2023-11-06T03:12:15.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.191 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011564 s, 90.7 MB/s 2023-11-06T03:12:15.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780904 s, 134 MB/s 2023-11-06T03:12:15.215 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.215 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.215 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116175 s, 90.3 MB/s 2023-11-06T03:12:15.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.241 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101281 s, 104 MB/s 2023-11-06T03:12:15.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.253 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103357 s, 101 MB/s 2023-11-06T03:12:15.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.266 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101362 s, 103 MB/s 2023-11-06T03:12:15.276 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.276 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.276 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818789 s, 128 MB/s 2023-11-06T03:12:15.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.291 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114701 s, 91.4 MB/s 2023-11-06T03:12:15.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.305 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126768 s, 82.7 MB/s 2023-11-06T03:12:15.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.319 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109882 s, 95.4 MB/s 2023-11-06T03:12:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114935 s, 91.2 MB/s 2023-11-06T03:12:15.342 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.343 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845069 s, 124 MB/s 2023-11-06T03:12:15.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779596 s, 135 MB/s 2023-11-06T03:12:15.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.369 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.369 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135807 s, 77.2 MB/s 2023-11-06T03:12:15.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.379 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828047 s, 127 MB/s 2023-11-06T03:12:15.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120048 s, 87.3 MB/s 2023-11-06T03:12:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.404 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.405 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843316 s, 124 MB/s 2023-11-06T03:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122618 s, 85.5 MB/s 2023-11-06T03:12:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116038 s, 90.4 MB/s 2023-11-06T03:12:15.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105349 s, 99.5 MB/s 2023-11-06T03:12:15.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:15.448 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:15.448 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:15.448 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.19 2023-11-06T03:12:15.449 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:15.449 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:15.449 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:15.450 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:15.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121095 s, 86.6 MB/s 2023-11-06T03:12:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803021 s, 131 MB/s 2023-11-06T03:12:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012023 s, 87.2 MB/s 2023-11-06T03:12:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112925 s, 92.9 MB/s 2023-11-06T03:12:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110017 s, 95.3 MB/s 2023-11-06T03:12:15.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.576 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116655 s, 89.9 MB/s 2023-11-06T03:12:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.589 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114207 s, 91.8 MB/s 2023-11-06T03:12:15.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104584 s, 100 MB/s 2023-11-06T03:12:15.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111336 s, 94.2 MB/s 2023-11-06T03:12:15.628 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.628 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.629 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107474 s, 97.6 MB/s 2023-11-06T03:12:15.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.643 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125376 s, 83.6 MB/s 2023-11-06T03:12:15.655 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.655 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103928 s, 101 MB/s 2023-11-06T03:12:15.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.670 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119826 s, 87.5 MB/s 2023-11-06T03:12:15.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819411 s, 128 MB/s 2023-11-06T03:12:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116158 s, 90.3 MB/s 2023-11-06T03:12:15.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.705 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.705 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795815 s, 132 MB/s 2023-11-06T03:12:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121065 s, 86.6 MB/s 2023-11-06T03:12:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964959 s, 109 MB/s 2023-11-06T03:12:15.745 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01251 s, 83.8 MB/s 2023-11-06T03:12:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110088 s, 95.2 MB/s 2023-11-06T03:12:15.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.772 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109685 s, 95.6 MB/s 2023-11-06T03:12:15.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.787 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128947 s, 81.3 MB/s 2023-11-06T03:12:15.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123421 s, 85.0 MB/s 2023-11-06T03:12:15.816 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.816 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.816 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119464 s, 87.8 MB/s 2023-11-06T03:12:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121748 s, 86.1 MB/s 2023-11-06T03:12:15.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.845 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119704 s, 87.6 MB/s 2023-11-06T03:12:15.859 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.859 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114336 s, 91.7 MB/s 2023-11-06T03:12:15.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120482 s, 87.0 MB/s 2023-11-06T03:12:15.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.888 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119796 s, 87.5 MB/s 2023-11-06T03:12:15.903 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.903 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012813 s, 81.8 MB/s 2023-11-06T03:12:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122982 s, 85.3 MB/s 2023-11-06T03:12:15.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.933 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127949 s, 82.0 MB/s 2023-11-06T03:12:15.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:15.935 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:15.935 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:15.935 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.20 2023-11-06T03:12:15.936 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:15.936 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:15.936 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:15.937 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:15.998 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107596 s, 97.5 MB/s 2023-11-06T03:12:16.011 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.011 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.011 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111433 s, 94.1 MB/s 2023-11-06T03:12:16.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.024 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109852 s, 95.5 MB/s 2023-11-06T03:12:16.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.034 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801902 s, 131 MB/s 2023-11-06T03:12:16.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.046 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990657 s, 106 MB/s 2023-11-06T03:12:16.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077985 s, 134 MB/s 2023-11-06T03:12:16.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111737 s, 93.8 MB/s 2023-11-06T03:12:16.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.083 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970195 s, 108 MB/s 2023-11-06T03:12:16.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133443 s, 78.6 MB/s 2023-11-06T03:12:16.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.112 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.112 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113693 s, 92.2 MB/s 2023-11-06T03:12:16.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807823 s, 130 MB/s 2023-11-06T03:12:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122007 s, 85.9 MB/s 2023-11-06T03:12:16.147 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.147 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.147 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826434 s, 127 MB/s 2023-11-06T03:12:16.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130499 s, 80.4 MB/s 2023-11-06T03:12:16.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.178 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127028 s, 82.5 MB/s 2023-11-06T03:12:16.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.192 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115355 s, 90.9 MB/s 2023-11-06T03:12:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.206 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119112 s, 88.0 MB/s 2023-11-06T03:12:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.225 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175049 s, 59.9 MB/s 2023-11-06T03:12:16.245 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.245 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124234 s, 84.4 MB/s 2023-11-06T03:12:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.257 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890724 s, 118 MB/s 2023-11-06T03:12:16.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132554 s, 79.1 MB/s 2023-11-06T03:12:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915969 s, 114 MB/s 2023-11-06T03:12:16.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013221 s, 79.3 MB/s 2023-11-06T03:12:16.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107585 s, 97.5 MB/s 2023-11-06T03:12:16.325 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.325 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.325 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839231 s, 125 MB/s 2023-11-06T03:12:16.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966163 s, 109 MB/s 2023-11-06T03:12:16.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.352 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117608 s, 89.2 MB/s 2023-11-06T03:12:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121647 s, 86.2 MB/s 2023-11-06T03:12:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.382 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120978 s, 86.7 MB/s 2023-11-06T03:12:16.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921905 s, 114 MB/s 2023-11-06T03:12:16.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113047 s, 92.8 MB/s 2023-11-06T03:12:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.423 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012178 s, 86.1 MB/s 2023-11-06T03:12:16.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:16.425 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:16.425 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:16.425 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.21 2023-11-06T03:12:16.426 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:16.426 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:16.426 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:16.426 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130044 s, 80.6 MB/s 2023-11-06T03:12:16.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.507 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110394 s, 95.0 MB/s 2023-11-06T03:12:16.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.518 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875943 s, 120 MB/s 2023-11-06T03:12:16.532 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.533 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.533 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117399 s, 89.3 MB/s 2023-11-06T03:12:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01179 s, 88.9 MB/s 2023-11-06T03:12:16.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129002 s, 81.3 MB/s 2023-11-06T03:12:16.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.576 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010797 s, 97.1 MB/s 2023-11-06T03:12:16.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132745 s, 79.0 MB/s 2023-11-06T03:12:16.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.608 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121081 s, 86.6 MB/s 2023-11-06T03:12:16.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.623 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123967 s, 84.6 MB/s 2023-11-06T03:12:16.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.638 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121027 s, 86.6 MB/s 2023-11-06T03:12:16.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011296 s, 92.8 MB/s 2023-11-06T03:12:16.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.665 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106963 s, 98.0 MB/s 2023-11-06T03:12:16.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.682 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.682 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135017 s, 77.7 MB/s 2023-11-06T03:12:16.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.695 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923246 s, 114 MB/s 2023-11-06T03:12:16.709 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.709 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.709 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118763 s, 88.3 MB/s 2023-11-06T03:12:16.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114965 s, 91.2 MB/s 2023-11-06T03:12:16.739 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.740 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115406 s, 90.9 MB/s 2023-11-06T03:12:16.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129719 s, 80.8 MB/s 2023-11-06T03:12:16.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.773 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129281 s, 81.1 MB/s 2023-11-06T03:12:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.791 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137146 s, 76.5 MB/s 2023-11-06T03:12:16.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.808 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132407 s, 79.2 MB/s 2023-11-06T03:12:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150186 s, 69.8 MB/s 2023-11-06T03:12:16.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.844 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013387 s, 78.3 MB/s 2023-11-06T03:12:16.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107032 s, 98.0 MB/s 2023-11-06T03:12:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136173 s, 77.0 MB/s 2023-11-06T03:12:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119037 s, 88.1 MB/s 2023-11-06T03:12:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122523 s, 85.6 MB/s 2023-11-06T03:12:16.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.919 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.919 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112803 s, 93.0 MB/s 2023-11-06T03:12:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.933 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114195 s, 91.8 MB/s 2023-11-06T03:12:16.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.948 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110178 s, 95.2 MB/s 2023-11-06T03:12:16.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:16.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:16.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988579 s, 106 MB/s 2023-11-06T03:12:16.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:16.962 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:16.962 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:16.962 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.22 2023-11-06T03:12:16.963 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:16.963 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:16.964 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:16.964 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.036 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140019 s, 74.9 MB/s 2023-11-06T03:12:17.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.051 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01128 s, 93.0 MB/s 2023-11-06T03:12:17.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120777 s, 86.8 MB/s 2023-11-06T03:12:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123266 s, 85.1 MB/s 2023-11-06T03:12:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.097 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110155 s, 95.2 MB/s 2023-11-06T03:12:17.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.113 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132859 s, 78.9 MB/s 2023-11-06T03:12:17.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109646 s, 95.6 MB/s 2023-11-06T03:12:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.143 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128677 s, 81.5 MB/s 2023-11-06T03:12:17.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115455 s, 90.8 MB/s 2023-11-06T03:12:17.172 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.173 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.173 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132774 s, 79.0 MB/s 2023-11-06T03:12:17.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125775 s, 83.4 MB/s 2023-11-06T03:12:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123976 s, 84.6 MB/s 2023-11-06T03:12:17.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012776 s, 82.1 MB/s 2023-11-06T03:12:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.261 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140218 s, 74.8 MB/s 2023-11-06T03:12:17.276 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.276 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.277 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011994 s, 87.4 MB/s 2023-11-06T03:12:17.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010644 s, 98.5 MB/s 2023-11-06T03:12:17.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.307 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139577 s, 75.1 MB/s 2023-11-06T03:12:17.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129884 s, 80.7 MB/s 2023-11-06T03:12:17.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.339 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.339 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128716 s, 81.5 MB/s 2023-11-06T03:12:17.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116311 s, 90.2 MB/s 2023-11-06T03:12:17.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108192 s, 96.9 MB/s 2023-11-06T03:12:17.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.381 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103022 s, 102 MB/s 2023-11-06T03:12:17.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.398 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136691 s, 76.7 MB/s 2023-11-06T03:12:17.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.415 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137675 s, 76.2 MB/s 2023-11-06T03:12:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.428 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103752 s, 101 MB/s 2023-11-06T03:12:17.441 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.441 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.441 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101016 s, 104 MB/s 2023-11-06T03:12:17.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.456 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012087 s, 86.8 MB/s 2023-11-06T03:12:17.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126909 s, 82.6 MB/s 2023-11-06T03:12:17.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.484 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101068 s, 104 MB/s 2023-11-06T03:12:17.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.500 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128932 s, 81.3 MB/s 2023-11-06T03:12:17.515 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.515 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.515 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123563 s, 84.9 MB/s 2023-11-06T03:12:17.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.530 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120469 s, 87.0 MB/s 2023-11-06T03:12:17.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:17.532 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:17.532 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:17.532 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.23 2023-11-06T03:12:17.533 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:17.533 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:17.533 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:17.534 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:17.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135758 s, 77.2 MB/s 2023-11-06T03:12:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125449 s, 83.6 MB/s 2023-11-06T03:12:17.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138848 s, 75.5 MB/s 2023-11-06T03:12:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110167 s, 95.2 MB/s 2023-11-06T03:12:17.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.668 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134687 s, 77.9 MB/s 2023-11-06T03:12:17.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.684 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013017 s, 80.6 MB/s 2023-11-06T03:12:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113307 s, 92.5 MB/s 2023-11-06T03:12:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145804 s, 71.9 MB/s 2023-11-06T03:12:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119878 s, 87.5 MB/s 2023-11-06T03:12:17.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130206 s, 80.5 MB/s 2023-11-06T03:12:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.761 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115246 s, 91.0 MB/s 2023-11-06T03:12:17.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.775 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106558 s, 98.4 MB/s 2023-11-06T03:12:17.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.788 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010515 s, 99.7 MB/s 2023-11-06T03:12:17.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.805 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.805 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136039 s, 77.1 MB/s 2023-11-06T03:12:17.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.821 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138707 s, 75.6 MB/s 2023-11-06T03:12:17.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.838 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137343 s, 76.3 MB/s 2023-11-06T03:12:17.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.851 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105888 s, 99.0 MB/s 2023-11-06T03:12:17.865 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.865 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106279 s, 98.7 MB/s 2023-11-06T03:12:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106185 s, 98.8 MB/s 2023-11-06T03:12:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.896 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146723 s, 71.5 MB/s 2023-11-06T03:12:17.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110482 s, 94.9 MB/s 2023-11-06T03:12:17.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.924 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112551 s, 93.2 MB/s 2023-11-06T03:12:17.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.937 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105381 s, 99.5 MB/s 2023-11-06T03:12:17.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.952 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122773 s, 85.4 MB/s 2023-11-06T03:12:17.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.968 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128706 s, 81.5 MB/s 2023-11-06T03:12:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010666 s, 98.3 MB/s 2023-11-06T03:12:17.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:17.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:17.997 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127623 s, 82.2 MB/s 2023-11-06T03:12:18.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.010 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.010 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104891 s, 100 MB/s 2023-11-06T03:12:18.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.025 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126188 s, 83.1 MB/s 2023-11-06T03:12:18.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105646 s, 99.3 MB/s 2023-11-06T03:12:18.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125399 s, 83.6 MB/s 2023-11-06T03:12:18.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105454 s, 99.4 MB/s 2023-11-06T03:12:18.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:18.068 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:18.068 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:18.069 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.24 2023-11-06T03:12:18.069 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:18.069 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:18.070 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:18.070 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:18.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.143 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.143 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106867 s, 98.1 MB/s 2023-11-06T03:12:18.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.160 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013478 s, 77.8 MB/s 2023-11-06T03:12:18.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130641 s, 80.3 MB/s 2023-11-06T03:12:18.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106016 s, 98.9 MB/s 2023-11-06T03:12:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835307 s, 126 MB/s 2023-11-06T03:12:18.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.212 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104485 s, 100 MB/s 2023-11-06T03:12:18.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.230 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155214 s, 67.6 MB/s 2023-11-06T03:12:18.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093716 s, 112 MB/s 2023-11-06T03:12:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117483 s, 89.3 MB/s 2023-11-06T03:12:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117041 s, 89.6 MB/s 2023-11-06T03:12:18.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.301 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125917 s, 83.3 MB/s 2023-11-06T03:12:18.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.318 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137763 s, 76.1 MB/s 2023-11-06T03:12:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011634 s, 90.1 MB/s 2023-11-06T03:12:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117251 s, 89.4 MB/s 2023-11-06T03:12:18.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148215 s, 70.7 MB/s 2023-11-06T03:12:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135884 s, 77.2 MB/s 2023-11-06T03:12:18.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.397 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107826 s, 97.2 MB/s 2023-11-06T03:12:18.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.413 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124867 s, 84.0 MB/s 2023-11-06T03:12:18.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.428 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123522 s, 84.9 MB/s 2023-11-06T03:12:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995361 s, 105 MB/s 2023-11-06T03:12:18.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.456 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127156 s, 82.5 MB/s 2023-11-06T03:12:18.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.472 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130763 s, 80.2 MB/s 2023-11-06T03:12:18.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.487 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120566 s, 87.0 MB/s 2023-11-06T03:12:18.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.503 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130661 s, 80.3 MB/s 2023-11-06T03:12:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.518 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120206 s, 87.2 MB/s 2023-11-06T03:12:18.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014186 s, 73.9 MB/s 2023-11-06T03:12:18.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100024 s, 105 MB/s 2023-11-06T03:12:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104009 s, 101 MB/s 2023-11-06T03:12:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.574 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101185 s, 104 MB/s 2023-11-06T03:12:18.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.587 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106438 s, 98.5 MB/s 2023-11-06T03:12:18.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122966 s, 85.3 MB/s 2023-11-06T03:12:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01309 s, 80.1 MB/s 2023-11-06T03:12:18.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:18.619 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:18.620 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:18.620 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.25 2023-11-06T03:12:18.620 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:18.621 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:18.621 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:18.621 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.689 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120219 s, 87.2 MB/s 2023-11-06T03:12:18.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.706 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143548 s, 73.0 MB/s 2023-11-06T03:12:18.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143356 s, 73.1 MB/s 2023-11-06T03:12:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136345 s, 76.9 MB/s 2023-11-06T03:12:18.755 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.755 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.755 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112834 s, 92.9 MB/s 2023-11-06T03:12:18.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.773 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141282 s, 74.2 MB/s 2023-11-06T03:12:18.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.789 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136251 s, 77.0 MB/s 2023-11-06T03:12:18.805 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137464 s, 76.3 MB/s 2023-11-06T03:12:18.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.821 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128763 s, 81.4 MB/s 2023-11-06T03:12:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.833 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929367 s, 113 MB/s 2023-11-06T03:12:18.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123948 s, 84.6 MB/s 2023-11-06T03:12:18.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.864 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119301 s, 87.9 MB/s 2023-11-06T03:12:18.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116322 s, 90.1 MB/s 2023-11-06T03:12:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012606 s, 83.2 MB/s 2023-11-06T03:12:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.907 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992912 s, 106 MB/s 2023-11-06T03:12:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123567 s, 84.9 MB/s 2023-11-06T03:12:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114311 s, 91.7 MB/s 2023-11-06T03:12:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128939 s, 81.3 MB/s 2023-11-06T03:12:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.964 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906433 s, 116 MB/s 2023-11-06T03:12:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101801 s, 103 MB/s 2023-11-06T03:12:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:18.996 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016014 s, 65.5 MB/s 2023-11-06T03:12:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102232 s, 103 MB/s 2023-11-06T03:12:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.026 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134573 s, 77.9 MB/s 2023-11-06T03:12:19.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.042 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130909 s, 80.1 MB/s 2023-11-06T03:12:19.055 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113522 s, 92.4 MB/s 2023-11-06T03:12:19.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116589 s, 89.9 MB/s 2023-11-06T03:12:19.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.086 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012893 s, 81.3 MB/s 2023-11-06T03:12:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143159 s, 73.2 MB/s 2023-11-06T03:12:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.117 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114415 s, 91.6 MB/s 2023-11-06T03:12:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852696 s, 123 MB/s 2023-11-06T03:12:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963394 s, 109 MB/s 2023-11-06T03:12:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.154 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119424 s, 87.8 MB/s 2023-11-06T03:12:19.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:19.156 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:19.156 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:19.156 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.26 2023-11-06T03:12:19.157 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:19.157 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:19.157 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:19.158 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:19.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.218 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106672 s, 98.3 MB/s 2023-11-06T03:12:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022092 s, 47.5 MB/s 2023-11-06T03:12:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769017 s, 136 MB/s 2023-11-06T03:12:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115441 s, 90.8 MB/s 2023-11-06T03:12:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.284 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115593 s, 90.7 MB/s 2023-11-06T03:12:19.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115015 s, 91.2 MB/s 2023-11-06T03:12:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111462 s, 94.1 MB/s 2023-11-06T03:12:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.321 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857582 s, 122 MB/s 2023-11-06T03:12:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010644 s, 98.5 MB/s 2023-11-06T03:12:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105475 s, 99.4 MB/s 2023-11-06T03:12:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883139 s, 119 MB/s 2023-11-06T03:12:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118212 s, 88.7 MB/s 2023-11-06T03:12:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106374 s, 98.6 MB/s 2023-11-06T03:12:19.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.402 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.402 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121698 s, 86.2 MB/s 2023-11-06T03:12:19.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.410 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832103 s, 126 MB/s 2023-11-06T03:12:19.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011755 s, 89.2 MB/s 2023-11-06T03:12:19.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.435 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846708 s, 124 MB/s 2023-11-06T03:12:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112595 s, 93.1 MB/s 2023-11-06T03:12:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104727 s, 100 MB/s 2023-11-06T03:12:19.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.475 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113823 s, 92.1 MB/s 2023-11-06T03:12:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011013 s, 95.2 MB/s 2023-11-06T03:12:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.499 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864693 s, 121 MB/s 2023-11-06T03:12:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915545 s, 115 MB/s 2023-11-06T03:12:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109643 s, 95.6 MB/s 2023-11-06T03:12:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109571 s, 95.7 MB/s 2023-11-06T03:12:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828153 s, 127 MB/s 2023-11-06T03:12:19.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.558 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832678 s, 126 MB/s 2023-11-06T03:12:19.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.569 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852177 s, 123 MB/s 2023-11-06T03:12:19.580 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.580 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.580 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870507 s, 120 MB/s 2023-11-06T03:12:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123848 s, 84.7 MB/s 2023-11-06T03:12:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106654 s, 98.3 MB/s 2023-11-06T03:12:19.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.623 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125766 s, 83.4 MB/s 2023-11-06T03:12:19.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:19.625 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:19.625 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:19.625 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.27 2023-11-06T03:12:19.626 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:19.626 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:19.626 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:19.627 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012298 s, 85.3 MB/s 2023-11-06T03:12:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125035 s, 83.9 MB/s 2023-11-06T03:12:19.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.722 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118817 s, 88.3 MB/s 2023-11-06T03:12:19.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120608 s, 86.9 MB/s 2023-11-06T03:12:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.751 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112948 s, 92.8 MB/s 2023-11-06T03:12:19.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928745 s, 113 MB/s 2023-11-06T03:12:19.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011749 s, 89.2 MB/s 2023-11-06T03:12:19.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.789 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959979 s, 109 MB/s 2023-11-06T03:12:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.800 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896088 s, 117 MB/s 2023-11-06T03:12:19.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.813 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.813 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814509 s, 129 MB/s 2023-11-06T03:12:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.824 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109164 s, 96.1 MB/s 2023-11-06T03:12:19.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.835 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841905 s, 125 MB/s 2023-11-06T03:12:19.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.849 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011109 s, 94.4 MB/s 2023-11-06T03:12:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126412 s, 82.9 MB/s 2023-11-06T03:12:19.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805042 s, 130 MB/s 2023-11-06T03:12:19.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.888 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121109 s, 86.6 MB/s 2023-11-06T03:12:19.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.898 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775406 s, 135 MB/s 2023-11-06T03:12:19.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136071 s, 77.1 MB/s 2023-11-06T03:12:19.926 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.926 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.927 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106201 s, 98.7 MB/s 2023-11-06T03:12:19.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778318 s, 135 MB/s 2023-11-06T03:12:19.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.947 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770855 s, 136 MB/s 2023-11-06T03:12:19.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.956 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757736 s, 138 MB/s 2023-11-06T03:12:19.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.970 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112182 s, 93.5 MB/s 2023-11-06T03:12:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766367 s, 137 MB/s 2023-11-06T03:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:19.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108831 s, 96.3 MB/s 2023-11-06T03:12:20.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.002 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748929 s, 140 MB/s 2023-11-06T03:12:20.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.014 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106765 s, 98.2 MB/s 2023-11-06T03:12:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877072 s, 120 MB/s 2023-11-06T03:12:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103057 s, 102 MB/s 2023-11-06T03:12:20.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.051 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112637 s, 93.1 MB/s 2023-11-06T03:12:20.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982467 s, 107 MB/s 2023-11-06T03:12:20.076 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.077 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114977 s, 91.2 MB/s 2023-11-06T03:12:20.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:20.078 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:20.079 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:20.079 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.28 2023-11-06T03:12:20.080 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:20.080 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:20.080 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:20.081 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:20.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.144 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012322 s, 85.1 MB/s 2023-11-06T03:12:20.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112091 s, 93.5 MB/s 2023-11-06T03:12:20.173 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115033 s, 91.2 MB/s 2023-11-06T03:12:20.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108339 s, 96.8 MB/s 2023-11-06T03:12:20.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.198 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888293 s, 118 MB/s 2023-11-06T03:12:20.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102328 s, 102 MB/s 2023-11-06T03:12:20.224 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.224 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.224 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113165 s, 92.7 MB/s 2023-11-06T03:12:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135095 s, 77.6 MB/s 2023-11-06T03:12:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.283 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115229 s, 91.0 MB/s 2023-11-06T03:12:20.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109265 s, 96.0 MB/s 2023-11-06T03:12:20.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.311 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010613 s, 98.8 MB/s 2023-11-06T03:12:20.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.325 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114104 s, 91.9 MB/s 2023-11-06T03:12:20.340 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.340 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.341 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116508 s, 90.0 MB/s 2023-11-06T03:12:20.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.351 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817046 s, 128 MB/s 2023-11-06T03:12:20.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100299 s, 105 MB/s 2023-11-06T03:12:20.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.376 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978804 s, 107 MB/s 2023-11-06T03:12:20.386 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.387 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895767 s, 117 MB/s 2023-11-06T03:12:20.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.400 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108591 s, 96.6 MB/s 2023-11-06T03:12:20.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.411 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880366 s, 119 MB/s 2023-11-06T03:12:20.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120127 s, 87.3 MB/s 2023-11-06T03:12:20.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123627 s, 84.8 MB/s 2023-11-06T03:12:20.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074851 s, 140 MB/s 2023-11-06T03:12:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117809 s, 89.0 MB/s 2023-11-06T03:12:20.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121339 s, 86.4 MB/s 2023-11-06T03:12:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.492 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121269 s, 86.5 MB/s 2023-11-06T03:12:20.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.506 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112106 s, 93.5 MB/s 2023-11-06T03:12:20.516 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884077 s, 119 MB/s 2023-11-06T03:12:20.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113807 s, 92.1 MB/s 2023-11-06T03:12:20.541 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.542 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088516 s, 118 MB/s 2023-11-06T03:12:20.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115473 s, 90.8 MB/s 2023-11-06T03:12:20.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112853 s, 92.9 MB/s 2023-11-06T03:12:20.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115351 s, 90.9 MB/s 2023-11-06T03:12:20.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:20.585 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:20.585 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:20.586 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.29 2023-11-06T03:12:20.586 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:20.586 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:20.587 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:20.587 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:20.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.654 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126462 s, 82.9 MB/s 2023-11-06T03:12:20.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.668 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111014 s, 94.5 MB/s 2023-11-06T03:12:20.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.684 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013448 s, 78.0 MB/s 2023-11-06T03:12:20.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113789 s, 92.2 MB/s 2023-11-06T03:12:20.709 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941416 s, 111 MB/s 2023-11-06T03:12:20.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835681 s, 125 MB/s 2023-11-06T03:12:20.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.733 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937203 s, 112 MB/s 2023-11-06T03:12:20.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.745 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917793 s, 114 MB/s 2023-11-06T03:12:20.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113109 s, 92.7 MB/s 2023-11-06T03:12:20.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.773 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129567 s, 80.9 MB/s 2023-11-06T03:12:20.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.788 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.788 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126314 s, 83.0 MB/s 2023-11-06T03:12:20.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.803 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115543 s, 90.8 MB/s 2023-11-06T03:12:20.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.814 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897986 s, 117 MB/s 2023-11-06T03:12:20.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.825 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909811 s, 115 MB/s 2023-11-06T03:12:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.839 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115803 s, 90.5 MB/s 2023-11-06T03:12:20.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088838 s, 118 MB/s 2023-11-06T03:12:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118969 s, 88.1 MB/s 2023-11-06T03:12:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936525 s, 112 MB/s 2023-11-06T03:12:20.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.890 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112404 s, 93.3 MB/s 2023-11-06T03:12:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127473 s, 82.3 MB/s 2023-11-06T03:12:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914382 s, 115 MB/s 2023-11-06T03:12:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.930 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104108 s, 101 MB/s 2023-11-06T03:12:20.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124695 s, 84.1 MB/s 2023-11-06T03:12:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.956 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856394 s, 122 MB/s 2023-11-06T03:12:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.970 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114586 s, 91.5 MB/s 2023-11-06T03:12:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.981 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.982 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883294 s, 119 MB/s 2023-11-06T03:12:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:20.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:20.997 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124933 s, 83.9 MB/s 2023-11-06T03:12:21.011 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.012 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.012 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124132 s, 84.5 MB/s 2023-11-06T03:12:21.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.028 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130245 s, 80.5 MB/s 2023-11-06T03:12:21.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103672 s, 101 MB/s 2023-11-06T03:12:21.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103 s, 102 MB/s 2023-11-06T03:12:21.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.069 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124328 s, 84.3 MB/s 2023-11-06T03:12:21.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:21.071 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:21.071 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:21.071 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.30 2023-11-06T03:12:21.072 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:21.072 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:21.072 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:21.073 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.138 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127676 s, 82.1 MB/s 2023-11-06T03:12:21.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133399 s, 78.6 MB/s 2023-11-06T03:12:21.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.169 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011632 s, 90.1 MB/s 2023-11-06T03:12:21.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.183 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112249 s, 93.4 MB/s 2023-11-06T03:12:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101468 s, 103 MB/s 2023-11-06T03:12:21.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.213 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131115 s, 80.0 MB/s 2023-11-06T03:12:21.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.230 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143581 s, 73.0 MB/s 2023-11-06T03:12:21.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.247 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013441 s, 78.0 MB/s 2023-11-06T03:12:21.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013336 s, 78.6 MB/s 2023-11-06T03:12:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104635 s, 100 MB/s 2023-11-06T03:12:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.316 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129624 s, 80.9 MB/s 2023-11-06T03:12:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.328 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978577 s, 107 MB/s 2023-11-06T03:12:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139388 s, 75.2 MB/s 2023-11-06T03:12:21.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.358 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100642 s, 104 MB/s 2023-11-06T03:12:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.375 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147135 s, 71.3 MB/s 2023-11-06T03:12:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01016 s, 103 MB/s 2023-11-06T03:12:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132089 s, 79.4 MB/s 2023-11-06T03:12:21.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.417 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.417 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105853 s, 99.1 MB/s 2023-11-06T03:12:21.431 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.431 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.431 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118636 s, 88.4 MB/s 2023-11-06T03:12:21.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914794 s, 115 MB/s 2023-11-06T03:12:21.457 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.457 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.457 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113065 s, 92.7 MB/s 2023-11-06T03:12:21.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.473 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012279 s, 85.4 MB/s 2023-11-06T03:12:21.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.485 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928693 s, 113 MB/s 2023-11-06T03:12:21.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875164 s, 120 MB/s 2023-11-06T03:12:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.512 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01357 s, 77.3 MB/s 2023-11-06T03:12:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912569 s, 115 MB/s 2023-11-06T03:12:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.538 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124052 s, 84.5 MB/s 2023-11-06T03:12:21.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.552 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112129 s, 93.5 MB/s 2023-11-06T03:12:21.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.567 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122659 s, 85.5 MB/s 2023-11-06T03:12:21.580 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.581 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108211 s, 96.9 MB/s 2023-11-06T03:12:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911634 s, 115 MB/s 2023-11-06T03:12:21.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.608 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124331 s, 84.3 MB/s 2023-11-06T03:12:21.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:21.610 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:21.610 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:21.610 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.31 2023-11-06T03:12:21.611 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:21.611 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:21.612 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:21.612 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:21.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.676 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981843 s, 107 MB/s 2023-11-06T03:12:21.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.692 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131883 s, 79.5 MB/s 2023-11-06T03:12:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011029 s, 95.1 MB/s 2023-11-06T03:12:21.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010191 s, 103 MB/s 2023-11-06T03:12:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107345 s, 97.7 MB/s 2023-11-06T03:12:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127274 s, 82.4 MB/s 2023-11-06T03:12:21.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122891 s, 85.3 MB/s 2023-11-06T03:12:21.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.779 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01261 s, 83.2 MB/s 2023-11-06T03:12:21.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123086 s, 85.2 MB/s 2023-11-06T03:12:21.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.810 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135945 s, 77.1 MB/s 2023-11-06T03:12:21.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129448 s, 81.0 MB/s 2023-11-06T03:12:21.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121345 s, 86.4 MB/s 2023-11-06T03:12:21.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128992 s, 81.3 MB/s 2023-11-06T03:12:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.873 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127426 s, 82.3 MB/s 2023-11-06T03:12:21.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.889 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127298 s, 82.4 MB/s 2023-11-06T03:12:21.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124111 s, 84.5 MB/s 2023-11-06T03:12:21.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.921 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013437 s, 78.0 MB/s 2023-11-06T03:12:21.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.938 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127133 s, 82.5 MB/s 2023-11-06T03:12:21.953 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135573 s, 77.3 MB/s 2023-11-06T03:12:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.972 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150588 s, 69.6 MB/s 2023-11-06T03:12:21.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:21.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:21.987 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111964 s, 93.7 MB/s 2023-11-06T03:12:22.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.001 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109248 s, 96.0 MB/s 2023-11-06T03:12:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118131 s, 88.8 MB/s 2023-11-06T03:12:22.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.033 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143233 s, 73.2 MB/s 2023-11-06T03:12:22.049 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.049 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.049 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135839 s, 77.2 MB/s 2023-11-06T03:12:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.065 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112578 s, 93.1 MB/s 2023-11-06T03:12:22.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.077 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106932 s, 98.1 MB/s 2023-11-06T03:12:22.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.093 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126259 s, 83.0 MB/s 2023-11-06T03:12:22.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.107 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113045 s, 92.8 MB/s 2023-11-06T03:12:22.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133058 s, 78.8 MB/s 2023-11-06T03:12:22.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126279 s, 83.0 MB/s 2023-11-06T03:12:22.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:22.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:22.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146767 s, 71.4 MB/s 2023-11-06T03:12:22.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2023-11-06T03:12:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 Processor -- start 2023-11-06T03:12:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 -- start start 2023-11-06T03:12:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c06c170 0x7fc95c154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc95c06c550 0x7fc95c155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c0cb100 con 0x7fc95c15b150 2023-11-06T03:12:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95c15bcf0 con 0x7fc95c06c550 2023-11-06T03:12:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc9629b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c0ce0a0 con 0x7fc95c06c170 2023-11-06T03:12:22.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc95b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc95c06c550 0x7fc95c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc960f52640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc95c15b150 0x7fc95c15b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41018/0 (socket says 172.21.15.157:41018) 2023-11-06T03:12:22.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc95b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc95c06c550 0x7fc95c155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41474/0 (socket says 172.21.15.157:41474) 2023-11-06T03:12:22.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.334+0000 7fc960f52640 1 -- 172.21.15.157:0/374129535 learned_addr learned my addr 172.21.15.157:0/374129535 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:22.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95bfff640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c06c170 0x7fc95c154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95b7fe640 1 -- 172.21.15.157:0/374129535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c06c170 msgr2=0x7fc95c154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c06c170 0x7fc95c154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95b7fe640 1 -- 172.21.15.157:0/374129535 >> v1:172.21.15.157:6789/0 conn(0x7fc95c15b150 legacy=0x7fc95c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95b7fe640 1 -- 172.21.15.157:0/374129535 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c15bef0 con 0x7fc95c06c550 2023-11-06T03:12:22.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95affd640 1 -- 172.21.15.157:0/374129535 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039120804 0 0) 0x7fc95c0cb100 con 0x7fc95c15b150 2023-11-06T03:12:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c155140 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc9500097c0 tx=0x7fc95001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24142596bd3be68 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95affd640 1 -- 172.21.15.157:0/374129535 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500ab650 con 0x7fc95c06c550 2023-11-06T03:12:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95affd640 1 -- 172.21.15.157:0/374129535 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500ab7b0 con 0x7fc95c06c550 2023-11-06T03:12:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc95affd640 1 -- 172.21.15.157:0/374129535 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500abb40 con 0x7fc95c06c550 2023-11-06T03:12:22.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 -- 172.21.15.157:0/374129535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 msgr2=0x7fc95c155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c155140 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fc9500097c0 tx=0x7fc95001c450 comp rx=0 tx=0).stop 2023-11-06T03:12:22.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 -- 172.21.15.157:0/374129535 shutdown_connections 2023-11-06T03:12:22.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c155140 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 --2- 172.21.15.157:0/374129535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c06c170 0x7fc95c154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 -- 172.21.15.157:0/374129535 >> 172.21.15.157:0/374129535 conn(0x7fc95c0b43b0 msgr2=0x7fc95c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 -- 172.21.15.157:0/374129535 shutdown_connections 2023-11-06T03:12:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.338+0000 7fc9629b5640 1 -- 172.21.15.157:0/374129535 wait complete. 2023-11-06T03:12:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 Processor -- start 2023-11-06T03:12:22.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- start start 2023-11-06T03:12:22.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 0x7fc95c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c15b150 0x7fc95c15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1544f0 con 0x7fc95c06c550 2023-11-06T03:12:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c1538e0 con 0x7fc95c06c170 2023-11-06T03:12:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c153bc0 con 0x7fc95c15b150 2023-11-06T03:12:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41476/0 (socket says 172.21.15.157:41476) 2023-11-06T03:12:22.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 -- 172.21.15.157:0/2845721456 learned_addr learned my addr 172.21.15.157:0/2845721456 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc960f52640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c15b150 0x7fc95c15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95bfff640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 0x7fc95c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 -- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c15b150 msgr2=0x7fc95c15ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c15b150 0x7fc95c15ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 -- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 msgr2=0x7fc95c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 0x7fc95c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 -- 172.21.15.157:0/2845721456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950009420 con 0x7fc95c06c550 2023-11-06T03:12:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95bfff640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 0x7fc95c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc95001a080 tx=0x7fc95001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc958ff9640 1 -- 172.21.15.157:0/2845721456 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500b5810 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc958ff9640 1 -- 172.21.15.157:0/2845721456 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500b5990 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95c0cbfb0 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc958ff9640 1 -- 172.21.15.157:0/2845721456 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500b5d40 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c154180 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.342+0000 7fc958ff9640 1 -- 172.21.15.157:0/2845721456 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc9500c83f0 con 0x7fc95c06c550 2023-11-06T03:12:22.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc958ff9640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 0x7fc948067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc958ff9640 1 -- 172.21.15.157:0/2845721456 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc95011e310 con 0x7fc95c06c550 2023-11-06T03:12:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95bfff640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 0x7fc948067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 Processor -- start 2023-11-06T03:12:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- start start 2023-11-06T03:12:22.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95bfff640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 0x7fc948067f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006fd0 tx=0x7fc94c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b02ec0 0x55a377b032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377ab9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37797d150 con 0x55a377b037e0 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3779091b0 con 0x55a377ab9440 2023-11-06T03:12:22.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a377906780 con 0x55a377b02ec0 2023-11-06T03:12:22.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41490/0 (socket says 172.21.15.157:41490) 2023-11-06T03:12:22.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 -- 172.21.15.157:0/4126720186 learned_addr learned my addr 172.21.15.157:0/4126720186 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95bfff640 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b02ec0 0x55a377b032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 -- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b02ec0 msgr2=0x55a377b032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b02ec0 0x55a377b032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 -- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 msgr2=0x55a377ab9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc960f52640 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377ab9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377ab9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 -- 172.21.15.157:0/4126720186 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3775feda0 con 0x55a377b037e0 2023-11-06T03:12:22.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc9500f72e0 tx=0x7fc9500191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31a8b3efaf0d5d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc941ffb640 1 -- 172.21.15.157:0/4126720186 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500b5ea0 con 0x55a377b037e0 2023-11-06T03:12:22.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc941ffb640 1 -- 172.21.15.157:0/4126720186 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500f3660 con 0x55a377b037e0 2023-11-06T03:12:22.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc941ffb640 1 -- 172.21.15.157:0/4126720186 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500f7810 con 0x55a377b037e0 2023-11-06T03:12:22.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- 172.21.15.157:0/4126720186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 msgr2=0x55a377ab8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc9500f72e0 tx=0x7fc9500191f0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- 172.21.15.157:0/4126720186 shutdown_connections 2023-11-06T03:12:22.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377ab9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377ab8f00 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 --2- 172.21.15.157:0/4126720186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b02ec0 0x55a377b032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- 172.21.15.157:0/4126720186 >> 172.21.15.157:0/4126720186 conn(0x55a377aac3a0 msgr2=0x55a377ab77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- 172.21.15.157:0/4126720186 shutdown_connections 2023-11-06T03:12:22.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- 172.21.15.157:0/4126720186 wait complete. 2023-11-06T03:12:22.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.346+0000 7fc965add000 1 -- start start 2023-11-06T03:12:22.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 Processor -- start 2023-11-06T03:12:22.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377ab9440 0x55a377af0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377af1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a377902070 con 0x55a377b037e0 2023-11-06T03:12:22.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3778a17c0 con 0x55a377b02ec0 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3778a1460 con 0x55a377ab9440 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 10 client.0 _ll_register_callbacks cb 0x55a3778fb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41504/0 (socket says 172.21.15.157:41504) 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 -- 172.21.15.157:0/1468907982 learned_addr learned my addr 172.21.15.157:0/1468907982 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377ab9440 0x55a377af0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377af1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 -- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377ab9440 msgr2=0x55a377af0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377ab9440 0x55a377af0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 -- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 msgr2=0x55a377af1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377af1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 -- 172.21.15.157:0/1468907982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c154180 con 0x55a377b037e0 2023-11-06T03:12:22.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377af1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:22.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc960f52640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc954007fd0 tx=0x7fc9540019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f806d757322fd658 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400d070 con 0x55a377b037e0 2023-11-06T03:12:22.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9540145f0 con 0x55a377b037e0 2023-11-06T03:12:22.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3778bb840 con 0x55a377b037e0 2023-11-06T03:12:22.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9540147b0 con 0x55a377b037e0 2023-11-06T03:12:22.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 10 client.7965 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55a3778a2410 con 0x55a377b037e0 2023-11-06T03:12:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3778bb840 con 0x55a377b037e0 2023-11-06T03:12:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc954014910 con 0x55a377b037e0 2023-11-06T03:12:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 10 client.7965 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 10 client.7965 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 20 client.7965 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377af20e0 0x55a377b14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 4 client.7965 mds_command: new command op to 7785 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3779009c0 con 0x55a377af20e0 2023-11-06T03:12:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377af20e0 0x55a377b14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377af20e0 0x55a377b14430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c6c6d80f39be37 server_cookie=5673bcf3b726d9cf in_seq=0 out_seq=0 2023-11-06T03:12:22.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 10 client.7965 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 1 -- 172.21.15.157:0/1468907982 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a3779009c0 con 0x55a377af20e0 2023-11-06T03:12:22.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 10 client.7965 handle_command_reply: tid=0 2023-11-06T03:12:22.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 client.7965 shutdown 2023-11-06T03:12:22.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 msgr2=0x55a377af1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc954007fd0 tx=0x7fc9540019f0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 shutdown_connections 2023-11-06T03:12:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377af20e0 0x55a377b14430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b037e0 0x55a377af1840 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc93affd640 0 client.7965 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377af1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 --2- 172.21.15.157:0/1468907982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377ab9440 0x55a377af0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 >> 172.21.15.157:0/1468907982 conn(0x55a377aac3a0 msgr2=0x55a377aba2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 shutdown_connections 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 1 -- 172.21.15.157:0/1468907982 wait complete. 2023-11-06T03:12:22.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.350+0000 7fc965add000 20 client.7965 trim_cache size 0 max 16384 2023-11-06T03:12:22.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 Processor -- start 2023-11-06T03:12:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 -- start start 2023-11-06T03:12:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377ab87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377ab8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3775bfa20 con 0x55a377ab9440 2023-11-06T03:12:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3778fdfe0 con 0x55a377b02ec0 2023-11-06T03:12:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a377900040 con 0x55a377b037e0 2023-11-06T03:12:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41518/0 (socket says 172.21.15.157:41518) 2023-11-06T03:12:22.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 -- 172.21.15.157:0/3258355299 learned_addr learned my addr 172.21.15.157:0/3258355299 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 -- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 msgr2=0x55a377ab8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377ab8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 -- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 msgr2=0x55a377ab87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377ab87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 -- 172.21.15.157:0/3258355299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3778c7370 con 0x55a377ab9440 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc95bfff640 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00dd70 tx=0x7fc94c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db4aa2be950119e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc9427fc640 1 -- 172.21.15.157:0/3258355299 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00b050 con 0x55a377ab9440 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc9427fc640 1 -- 172.21.15.157:0/3258355299 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c01a040 con 0x55a377ab9440 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.366+0000 7fc9427fc640 1 -- 172.21.15.157:0/3258355299 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c025350 con 0x55a377ab9440 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/3258355299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 msgr2=0x55a377aeff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00dd70 tx=0x7fc94c013000 comp rx=0 tx=0).stop 2023-11-06T03:12:22.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/3258355299 shutdown_connections 2023-11-06T03:12:22.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377ab8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377b02ec0 0x55a377ab87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- 172.21.15.157:0/3258355299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377ab9440 0x55a377aeff40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/3258355299 >> 172.21.15.157:0/3258355299 conn(0x55a377aac3a0 msgr2=0x55a377ac02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/3258355299 shutdown_connections 2023-11-06T03:12:22.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/3258355299 wait complete. 2023-11-06T03:12:22.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- start start 2023-11-06T03:12:22.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 Processor -- start 2023-11-06T03:12:22.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377b5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377b13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a377902920 con 0x55a377b02ec0 2023-11-06T03:12:22.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3778c3e00 con 0x55a377ab9440 2023-11-06T03:12:22.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3778cd2a0 con 0x55a377b037e0 2023-11-06T03:12:22.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 10 client.0 _ll_register_callbacks cb 0x55a3778cbf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41532/0 (socket says 172.21.15.157:41532) 2023-11-06T03:12:22.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 -- 172.21.15.157:0/1496347936 learned_addr learned my addr 172.21.15.157:0/1496347936 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377b5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc960f52640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377b13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 -- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 msgr2=0x55a377b13a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377b13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 -- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 msgr2=0x55a377b5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377b5f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 -- 172.21.15.157:0/1496347936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c00da70 con 0x55a377b02ec0 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95b7fe640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc95001c930 tx=0x7fc95011c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bb3b0bd958a7ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95010a020 con 0x55a377b02ec0 2023-11-06T03:12:22.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9500f3660 con 0x55a377b02ec0 2023-11-06T03:12:22.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a37790a990 con 0x55a377b02ec0 2023-11-06T03:12:22.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9500b5ea0 con 0x55a377b02ec0 2023-11-06T03:12:22.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 10 client.7971 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55a3778ad4c0 con 0x55a377b02ec0 2023-11-06T03:12:22.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37790a990 con 0x55a377b02ec0 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc95010d2d0 con 0x55a377b02ec0 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 10 client.7971 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 10 client.7971 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 20 client.7971 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377b143a0 0x55a377b14780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 4 client.7971 mds_command: new command op to 7785 tid=0[{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2023-11-06T03:12:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55a3778fb0f0 con 0x55a377b143a0 2023-11-06T03:12:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377b143a0 0x55a377b14780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc93affd640 10 client.7971 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.370+0000 7fc95bfff640 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377b143a0 0x55a377b14780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a38383891055d584 server_cookie=53fedf8bb4129f3a in_seq=0 out_seq=0 2023-11-06T03:12:22.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc93affd640 1 -- 172.21.15.157:0/1496347936 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a3778fb0f0 con 0x55a377b143a0 2023-11-06T03:12:22.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc93affd640 10 client.7971 handle_command_reply: tid=0 2023-11-06T03:12:22.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 client.7971 shutdown 2023-11-06T03:12:22.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 msgr2=0x55a377b5f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc95001c930 tx=0x7fc95011c5b0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 shutdown_connections 2023-11-06T03:12:22.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55a377b143a0 0x55a377b14780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a377b037e0 0x55a377b13a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a377b02ec0 0x55a377b5f690 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 --2- 172.21.15.157:0/1496347936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a377ab9440 0x55a377b5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 >> 172.21.15.157:0/1496347936 conn(0x55a377aac3a0 msgr2=0x55a377aee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc93affd640 0 client.7971 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 shutdown_connections 2023-11-06T03:12:22.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 1 -- 172.21.15.157:0/1496347936 wait complete. 2023-11-06T03:12:22.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc965add000 20 client.7971 trim_cache size 0 max 16384 2023-11-06T03:12:22.394 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:22.394 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:12:22.394 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "ab5a6874-6fc6-4cbf-b1bf-9c2d95f7fe1d", 2023-11-06T03:12:22.394 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:12:22.394 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:22.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 msgr2=0x7fc948067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 0x7fc948067f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006fd0 tx=0x7fc94c008040 comp rx=0 tx=0).stop 2023-11-06T03:12:22.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 msgr2=0x7fc95c15e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fc95001a080 tx=0x7fc95001ac90 comp rx=0 tx=0).stop 2023-11-06T03:12:22.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 shutdown_connections 2023-11-06T03:12:22.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc948065a60 0x7fc948067f20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95c15b150 0x7fc95c15ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95c06c550 0x7fc95c15e6d0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 --2- 172.21.15.157:0/2845721456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc95c06c170 0x7fc95c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 >> 172.21.15.157:0/2845721456 conn(0x7fc95c0b43b0 msgr2=0x7fc95c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 shutdown_connections 2023-11-06T03:12:22.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.374+0000 7fc9629b5640 1 -- 172.21.15.157:0/2845721456 wait complete. 2023-11-06T03:12:22.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub abort 2023-11-06T03:12:22.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 Processor -- start 2023-11-06T03:12:22.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 -- start start 2023-11-06T03:12:22.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57a80b1e40 0x7f57a8066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a815aff0 con 0x7f57a80cf090 2023-11-06T03:12:22.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a806dd80 con 0x7f57a80cc3d0 2023-11-06T03:12:22.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57afc8c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a806e060 con 0x7f57a80b1e40 2023-11-06T03:12:22.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ae229640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57a80b1e40 0x7f57a8066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ada28640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f57a80cf090 0x7f57a80b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41024/0 (socket says 172.21.15.157:41024) 2023-11-06T03:12:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ada28640 1 -- 172.21.15.157:0/562736529 learned_addr learned my addr 172.21.15.157:0/562736529 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ae229640 1 -- 172.21.15.157:0/562736529 >> v1:172.21.15.195:6789/0 conn(0x7f57a80cc3d0 legacy=0x7f57a80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ae229640 1 -- 172.21.15.157:0/562736529 >> v1:172.21.15.157:6789/0 conn(0x7f57a80cf090 legacy=0x7f57a80b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.546+0000 7f57ae229640 1 -- 172.21.15.157:0/562736529 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a815b2b0 con 0x7f57a80b1e40 2023-11-06T03:12:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 --2- 172.21.15.157:0/562736529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80b1e40 0x7f57a8066340 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009d90 tx=0x7f57a001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ac6513db9086cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57aca26640 1 -- 172.21.15.157:0/562736529 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00a5940 con 0x7f57a80b1e40 2023-11-06T03:12:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57aca26640 1 -- 172.21.15.157:0/562736529 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a00a5aa0 con 0x7f57a80b1e40 2023-11-06T03:12:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57aca26640 1 -- 172.21.15.157:0/562736529 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00a5e30 con 0x7f57a80b1e40 2023-11-06T03:12:22.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- 172.21.15.157:0/562736529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80b1e40 msgr2=0x7f57a8066340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 --2- 172.21.15.157:0/562736529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80b1e40 0x7f57a8066340 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009d90 tx=0x7f57a001c920 comp rx=0 tx=0).stop 2023-11-06T03:12:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- 172.21.15.157:0/562736529 shutdown_connections 2023-11-06T03:12:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 --2- 172.21.15.157:0/562736529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80b1e40 0x7f57a8066340 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- 172.21.15.157:0/562736529 >> 172.21.15.157:0/562736529 conn(0x7f57a80732f0 msgr2=0x7f57a8069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- 172.21.15.157:0/562736529 shutdown_connections 2023-11-06T03:12:22.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- 172.21.15.157:0/562736529 wait complete. 2023-11-06T03:12:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 Processor -- start 2023-11-06T03:12:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- start start 2023-11-06T03:12:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a80b1e40 0x7f57a8157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80cc3d0 0x7f57a80ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8157e90 con 0x7f57a80cf090 2023-11-06T03:12:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8158170 con 0x7f57a80cc3d0 2023-11-06T03:12:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57afc8c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8158450 con 0x7f57a80b1e40 2023-11-06T03:12:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41540/0 (socket says 172.21.15.157:41540) 2023-11-06T03:12:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 -- 172.21.15.157:0/84060446 learned_addr learned my addr 172.21.15.157:0/84060446 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ad227640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80cc3d0 0x7f57a80ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ada28640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a80b1e40 0x7f57a8157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 -- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a80b1e40 msgr2=0x7f57a8157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a80b1e40 0x7f57a8157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 -- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80cc3d0 msgr2=0x7f57a80ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80cc3d0 0x7f57a80ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.550+0000 7f57ae229640 1 -- 172.21.15.157:0/84060446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009a70 con 0x7f57a80cf090 2023-11-06T03:12:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57ae229640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b0040 tx=0x7f57a00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 -- 172.21.15.157:0/84060446 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00ade40 con 0x7f57a80cf090 2023-11-06T03:12:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 -- 172.21.15.157:0/84060446 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a00b9450 con 0x7f57a80cf090 2023-11-06T03:12:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8158730 con 0x7f57a80cf090 2023-11-06T03:12:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 -- 172.21.15.157:0/84060446 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00b9800 con 0x7f57a80cf090 2023-11-06T03:12:22.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a815ca30 con 0x7f57a80cf090 2023-11-06T03:12:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 -- 172.21.15.157:0/84060446 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57a00b9960 con 0x7f57a80cf090 2023-11-06T03:12:22.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 0x7f578c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f579e7fc640 1 -- 172.21.15.157:0/84060446 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57a011ebc0 con 0x7f57a80cf090 2023-11-06T03:12:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57ada28640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 0x7f578c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57b2db4000 1 Processor -- start 2023-11-06T03:12:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57b2db4000 1 -- start start 2023-11-06T03:12:22.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.554+0000 7f57ada28640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 0x7f578c067f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005890 tx=0x7f57a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d91320 0x55ff85d91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d47610 0x55ff85d479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85c0abb0 con 0x55ff85d91c40 2023-11-06T03:12:22.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b99380 con 0x55ff85d91320 2023-11-06T03:12:22.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b96950 con 0x55ff85d47610 2023-11-06T03:12:22.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41542/0 (socket says 172.21.15.157:41542) 2023-11-06T03:12:22.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 -- 172.21.15.157:0/2151265693 learned_addr learned my addr 172.21.15.157:0/2151265693 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ada28640 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d91320 0x55ff85d91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d47610 0x55ff85d479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 -- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d47610 msgr2=0x55ff85d479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d47610 0x55ff85d479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 -- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d91320 msgr2=0x55ff85d91700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d91320 0x55ff85d91700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 -- 172.21.15.157:0/2151265693 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff8588edc0 con 0x55ff85d91c40 2023-11-06T03:12:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5798001550 tx=0x7f5798001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7410967e0b2406 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f578b7fe640 1 -- 172.21.15.157:0/2151265693 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980050d0 con 0x55ff85d91c40 2023-11-06T03:12:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f578b7fe640 1 -- 172.21.15.157:0/2151265693 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798005230 con 0x55ff85d91c40 2023-11-06T03:12:22.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f578b7fe640 1 -- 172.21.15.157:0/2151265693 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800e740 con 0x55ff85d91c40 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/2151265693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 msgr2=0x55ff85d470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5798001550 tx=0x7f5798001a10 comp rx=0 tx=0).stop 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/2151265693 shutdown_connections 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d47610 0x55ff85d479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d470d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- 172.21.15.157:0/2151265693 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d91320 0x55ff85d91700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/2151265693 >> 172.21.15.157:0/2151265693 conn(0x55ff85d49990 msgr2=0x55ff85d415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/2151265693 shutdown_connections 2023-11-06T03:12:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/2151265693 wait complete. 2023-11-06T03:12:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- start start 2023-11-06T03:12:22.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 Processor -- start 2023-11-06T03:12:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d7b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b92240 con 0x55ff85d91c40 2023-11-06T03:12:22.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b31410 con 0x55ff85d47610 2023-11-06T03:12:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b4f600 con 0x55ff85d91320 2023-11-06T03:12:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 10 client.0 _ll_register_callbacks cb 0x55ff85d2ebc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41552/0 (socket says 172.21.15.157:41552) 2023-11-06T03:12:22.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 -- 172.21.15.157:0/1711864840 learned_addr learned my addr 172.21.15.157:0/1711864840 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ada28640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d7b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 -- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 msgr2=0x55ff85d7b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d7b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 -- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 msgr2=0x55ff85d7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 -- 172.21.15.157:0/1711864840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579800f040 con 0x55ff85d91c40 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ad227640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d7b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57ae229640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f57a001ce90 tx=0x7f57a00f0b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec50788cfcf6c44b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f5788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00f62b0 con 0x55ff85d91c40 2023-11-06T03:12:22.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57a00f8330 con 0x55ff85d91c40 2023-11-06T03:12:22.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.558+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff85b4f040 con 0x55ff85d91c40 2023-11-06T03:12:22.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a00f05b0 con 0x55ff85d91c40 2023-11-06T03:12:22.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 10 client.7986 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55ff85b57170 con 0x55ff85d91c40 2023-11-06T03:12:22.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff85b4f040 con 0x55ff85d91c40 2023-11-06T03:12:22.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f57a011b020 con 0x55ff85d91c40 2023-11-06T03:12:22.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 10 client.7986 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 10 client.7986 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 20 client.7986 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85d7c630 0x55ff85d7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 4 client.7986 mds_command: new command op to 7785 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:22.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff85b90b90 con 0x55ff85d7c630 2023-11-06T03:12:22.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57ada28640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85d7c630 0x55ff85d7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57ada28640 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85d7c630 0x55ff85d7ca10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4c99daf6e2eb6 server_cookie=94528a681cb19dbf in_seq=0 out_seq=0 2023-11-06T03:12:22.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 10 client.7986 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 1 -- 172.21.15.157:0/1711864840 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ff85b90b90 con 0x55ff85d7c630 2023-11-06T03:12:22.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 10 client.7986 handle_command_reply: tid=0 2023-11-06T03:12:22.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 client.7986 shutdown 2023-11-06T03:12:22.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 msgr2=0x55ff85d7bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f57a001ce90 tx=0x7f57a00f0b10 comp rx=0 tx=0).stop 2023-11-06T03:12:22.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 shutdown_connections 2023-11-06T03:12:22.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85d7c630 0x55ff85d7ca10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d7bc20 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d7b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f5788ff9640 0 client.7986 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 --2- 172.21.15.157:0/1711864840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 >> 172.21.15.157:0/1711864840 conn(0x55ff85d49990 msgr2=0x55ff85d48980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 shutdown_connections 2023-11-06T03:12:22.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 1 -- 172.21.15.157:0/1711864840 wait complete. 2023-11-06T03:12:22.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.562+0000 7f57b2db4000 20 client.7986 trim_cache size 0 max 16384 2023-11-06T03:12:22.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.574+0000 7f57b2db4000 1 Processor -- start 2023-11-06T03:12:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.574+0000 7f57b2db4000 1 -- start start 2023-11-06T03:12:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d41250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8584fa40 con 0x55ff85d91c40 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85c1ac10 con 0x55ff85d47610 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b96240 con 0x55ff85d91320 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41564/0 (socket says 172.21.15.157:41564) 2023-11-06T03:12:22.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 -- 172.21.15.157:0/391059149 learned_addr learned my addr 172.21.15.157:0/391059149 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 -- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 msgr2=0x55ff85d41250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d41250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 -- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 msgr2=0x55ff85d40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 -- 172.21.15.157:0/391059149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff85b3d4c0 con 0x55ff85d91c40 2023-11-06T03:12:22.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f57a001ce00 tx=0x7f57a00c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b83582eca23831 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f578bfff640 1 -- 172.21.15.157:0/391059149 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00f05b0 con 0x55ff85d91c40 2023-11-06T03:12:22.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f578bfff640 1 -- 172.21.15.157:0/391059149 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a00f62b0 con 0x55ff85d91c40 2023-11-06T03:12:22.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f578bfff640 1 -- 172.21.15.157:0/391059149 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00efa60 con 0x55ff85d91c40 2023-11-06T03:12:22.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- 172.21.15.157:0/391059149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 msgr2=0x55ff85d41790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f57a001ce00 tx=0x7f57a00c4000 comp rx=0 tx=0).stop 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- 172.21.15.157:0/391059149 shutdown_connections 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91c40 0x55ff85d41790 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91320 0x55ff85d41250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- 172.21.15.157:0/391059149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- 172.21.15.157:0/391059149 >> 172.21.15.157:0/391059149 conn(0x55ff85d49990 msgr2=0x55ff85d7b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- 172.21.15.157:0/391059149 shutdown_connections 2023-11-06T03:12:22.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- 172.21.15.157:0/391059149 wait complete. 2023-11-06T03:12:22.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- start start 2023-11-06T03:12:22.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 Processor -- start 2023-11-06T03:12:22.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91c40 0x55ff85dd1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b8fa50 con 0x55ff85d91320 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b1c2d0 con 0x55ff85d47610 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff85b37c40 con 0x55ff85d91c40 2023-11-06T03:12:22.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57b2db4000 10 client.0 _ll_register_callbacks cb 0x55ff85b56c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41580/0 (socket says 172.21.15.157:41580) 2023-11-06T03:12:22.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 -- 172.21.15.157:0/3220009783 learned_addr learned my addr 172.21.15.157:0/3220009783 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ada28640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ae229640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91c40 0x55ff85dd1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 -- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91c40 msgr2=0x55ff85dd1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91c40 0x55ff85dd1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 -- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 msgr2=0x55ff85d484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 -- 172.21.15.157:0/3220009783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579800f040 con 0x55ff85d91320 2023-11-06T03:12:22.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f57ad227640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5798013000 tx=0x7f5798001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370f1c91a5fdb3fb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f5788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.578+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798023070 con 0x55ff85d91320 2023-11-06T03:12:22.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5798007020 con 0x55ff85d91320 2023-11-06T03:12:22.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff85b5b5c0 con 0x55ff85d91320 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57980071e0 con 0x55ff85d91320 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 10 client.7992 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x55ff85b31030 con 0x55ff85d91320 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff85b5b5c0 con 0x55ff85d91320 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5798026020 con 0x55ff85d91320 2023-11-06T03:12:22.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 10 client.7992 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 10 client.7992 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 20 client.7992 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85dd1a50 0x55ff85dd1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 4 client.7992 mds_command: new command op to 7785 tid=0[{"prefix": "scrub abort"}] 2023-11-06T03:12:22.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55ff85b8b2c0 con 0x55ff85dd1a50 2023-11-06T03:12:22.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57ada28640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85dd1a50 0x55ff85dd1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 10 client.7992 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57ada28640 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85dd1a50 0x55ff85dd1e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6640e1c242493935 server_cookie=61e2cd3a5a95ad6a in_seq=0 out_seq=0 2023-11-06T03:12:22.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 1 -- 172.21.15.157:0/3220009783 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55ff85b8b2c0 con 0x55ff85dd1a50 2023-11-06T03:12:22.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 10 client.7992 handle_command_reply: tid=0 2023-11-06T03:12:22.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 client.7992 shutdown 2023-11-06T03:12:22.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 msgr2=0x55ff85dd0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5798013000 tx=0x7f5798001130 comp rx=0 tx=0).stop 2023-11-06T03:12:22.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 shutdown_connections 2023-11-06T03:12:22.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x55ff85dd1a50 0x55ff85dd1e60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ff85d91c40 0x55ff85dd1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff85d91320 0x55ff85dd0c00 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f5788ff9640 0 client.7992 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 --2- 172.21.15.157:0/3220009783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ff85d47610 0x55ff85d484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 >> 172.21.15.157:0/3220009783 conn(0x55ff85d49990 msgr2=0x55ff85d479f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 shutdown_connections 2023-11-06T03:12:22.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 1 -- 172.21.15.157:0/3220009783 wait complete. 2023-11-06T03:12:22.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57b2db4000 20 client.7992 trim_cache size 0 max 16384 2023-11-06T03:12:22.602 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:22.602 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:12:22.603 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:22.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 msgr2=0x7f578c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 0x7f578c067f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57a4005890 tx=0x7f57a4009040 comp rx=0 tx=0).stop 2023-11-06T03:12:22.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 msgr2=0x7f57a814da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f57a00b0040 tx=0x7f57a00196c0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 shutdown_connections 2023-11-06T03:12:22.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f578c065a60 0x7f578c067f20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57a80cf090 0x7f57a814da90 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57a80cc3d0 0x7f57a80ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 --2- 172.21.15.157:0/84060446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57a80b1e40 0x7f57a8157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 >> 172.21.15.157:0/84060446 conn(0x7f57a80732f0 msgr2=0x7f57a8069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 shutdown_connections 2023-11-06T03:12:22.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.582+0000 7f57afc8c640 1 -- 172.21.15.157:0/84060446 wait complete. 2023-11-06T03:12:22.607 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-11-06T03:12:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.722+0000 7faa6f109640 1 Processor -- start 2023-11-06T03:12:22.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.722+0000 7faa6f109640 1 -- start start 2023-11-06T03:12:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6f109640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa6815caa0 0x7faa680cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6f109640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faa680d0280 0x7faa680b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6f109640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa680d2920 con 0x7faa6815ce80 2023-11-06T03:12:22.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6f109640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa680d18a0 con 0x7faa680d0280 2023-11-06T03:12:22.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6f109640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680d1b60 con 0x7faa6815caa0 2023-11-06T03:12:22.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6d6a6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faa680d0280 0x7faa680b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6cea5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa6815caa0 0x7faa680cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa67fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7faa6815ce80 0x7faa680cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:38676/0 (socket says 172.21.15.157:38676) 2023-11-06T03:12:22.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa67fff640 1 -- 172.21.15.157:0/3176705538 learned_addr learned my addr 172.21.15.157:0/3176705538 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:22.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa6815caa0 msgr2=0x7faa680cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa6815caa0 0x7faa680cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3176705538 >> v1:172.21.15.195:6789/0 conn(0x7faa6815ce80 legacy=0x7faa680cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.726+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3176705538 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680d1e40 con 0x7faa680d0280 2023-11-06T03:12:22.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa680d0280 0x7faa680b5d50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a9f795c8d3b998 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa677fe640 1 -- 172.21.15.157:0/3176705538 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600a59e0 con 0x7faa680d0280 2023-11-06T03:12:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa677fe640 1 -- 172.21.15.157:0/3176705538 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa600a5b40 con 0x7faa680d0280 2023-11-06T03:12:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa677fe640 1 -- 172.21.15.157:0/3176705538 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600ad440 con 0x7faa680d0280 2023-11-06T03:12:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa680d0280 msgr2=0x7faa680b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa680d0280 0x7faa680b5d50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6001c920 comp rx=0 tx=0).stop 2023-11-06T03:12:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- 172.21.15.157:0/3176705538 shutdown_connections 2023-11-06T03:12:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa680d0280 0x7faa680b5d50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- 172.21.15.157:0/3176705538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa6815caa0 0x7faa680cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- 172.21.15.157:0/3176705538 >> 172.21.15.157:0/3176705538 conn(0x7faa68070790 msgr2=0x7faa68070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- 172.21.15.157:0/3176705538 shutdown_connections 2023-11-06T03:12:22.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- 172.21.15.157:0/3176705538 wait complete. 2023-11-06T03:12:22.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 Processor -- start 2023-11-06T03:12:22.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- start start 2023-11-06T03:12:22.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa680d0280 0x7faa68151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 0x7faa680ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681582e0 con 0x7faa6815ce80 2023-11-06T03:12:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68158000 con 0x7faa6815caa0 2023-11-06T03:12:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6f109640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815d740 con 0x7faa680d0280 2023-11-06T03:12:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41590/0 (socket says 172.21.15.157:41590) 2023-11-06T03:12:22.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3914493518 learned_addr learned my addr 172.21.15.157:0/3914493518 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa67fff640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 0x7faa680ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa680d0280 0x7faa68151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa680d0280 msgr2=0x7faa68151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa680d0280 0x7faa68151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 msgr2=0x7faa680ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 0x7faa680ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa6d6a6640 1 -- 172.21.15.157:0/3914493518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60009a70 con 0x7faa6815ce80 2023-11-06T03:12:22.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.730+0000 7faa67fff640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 0x7faa680ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:22.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7faa600b0040 tx=0x7faa600196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 -- 172.21.15.157:0/3914493518 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600add90 con 0x7faa6815ce80 2023-11-06T03:12:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 -- 172.21.15.157:0/3914493518 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa60009450 con 0x7faa6815ce80 2023-11-06T03:12:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6815da20 con 0x7faa6815ce80 2023-11-06T03:12:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 -- 172.21.15.157:0/3914493518 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600b9690 con 0x7faa6815ce80 2023-11-06T03:12:22.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa68157cf0 con 0x7faa6815ce80 2023-11-06T03:12:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 -- 172.21.15.157:0/3914493518 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faa600b97f0 con 0x7faa6815ce80 2023-11-06T03:12:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 0x7faa54067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa657fa640 1 -- 172.21.15.157:0/3914493518 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa6011e980 con 0x7faa6815ce80 2023-11-06T03:12:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 0x7faa54067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa72231000 1 Processor -- start 2023-11-06T03:12:22.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa72231000 1 -- start start 2023-11-06T03:12:22.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 0x7faa54067f20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faa58009af0 tx=0x7faa58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.734+0000 7faa72231000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824fa50 0x5555a8204d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a8205260 0x5555a8205640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a80ccc10 con 0x5555a824f130 2023-11-06T03:12:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a805b3e0 con 0x5555a8205260 2023-11-06T03:12:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555a80589b0 con 0x5555a824fa50 2023-11-06T03:12:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41606/0 (socket says 172.21.15.157:41606) 2023-11-06T03:12:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/684104119 learned_addr learned my addr 172.21.15.157:0/684104119 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824fa50 msgr2=0x5555a8204d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824fa50 0x5555a8204d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a8205260 msgr2=0x5555a8205640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a8205260 0x5555a8205640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/684104119 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a7d50e20 con 0x5555a824f130 2023-11-06T03:12:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7faa58011dc0 tx=0x7faa58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4975c4857f9c710a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa4e7fc640 1 -- 172.21.15.157:0/684104119 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58011df0 con 0x5555a824f130 2023-11-06T03:12:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa4e7fc640 1 -- 172.21.15.157:0/684104119 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58021440 con 0x5555a824f130 2023-11-06T03:12:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa4e7fc640 1 -- 172.21.15.157:0/684104119 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580217b0 con 0x5555a824f130 2023-11-06T03:12:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/684104119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 msgr2=0x5555a824f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7faa58011dc0 tx=0x7faa58014000 comp rx=0 tx=0).stop 2023-11-06T03:12:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/684104119 shutdown_connections 2023-11-06T03:12:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a8205260 0x5555a8205640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824fa50 0x5555a8204d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- 172.21.15.157:0/684104119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a824f510 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/684104119 >> 172.21.15.157:0/684104119 conn(0x5555a820b9f0 msgr2=0x5555a8203600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/684104119 shutdown_connections 2023-11-06T03:12:22.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/684104119 wait complete. 2023-11-06T03:12:22.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- start start 2023-11-06T03:12:22.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 Processor -- start 2023-11-06T03:12:22.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 0x5555a8256b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a824fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a80542a0 con 0x5555a8205260 2023-11-06T03:12:22.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a7ff3470 con 0x5555a824fa50 2023-11-06T03:12:22.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555a8011660 con 0x5555a824f130 2023-11-06T03:12:22.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 10 client.0 _ll_register_callbacks cb 0x5555a81f0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41618/0 (socket says 172.21.15.157:41618) 2023-11-06T03:12:22.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/3690047824 learned_addr learned my addr 172.21.15.157:0/3690047824 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa67fff640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 0x5555a8256b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 msgr2=0x5555a8256b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a824fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 0x5555a8256b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 msgr2=0x5555a824fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a824fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 -- 172.21.15.157:0/3690047824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa58011a70 con 0x5555a8205260 2023-11-06T03:12:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa67fff640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 0x5555a8256b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa6cea5640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7faa58023040 tx=0x7faa580046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9f9eed0077e687 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa477fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58009820 con 0x5555a8205260 2023-11-06T03:12:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa58031070 con 0x5555a8205260 2023-11-06T03:12:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555a80110a0 con 0x5555a8205260 2023-11-06T03:12:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa58021b50 con 0x5555a8205260 2023-11-06T03:12:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 10 client.8001 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.738+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5555a80191d0 con 0x5555a8205260 2023-11-06T03:12:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5555a80110a0 con 0x5555a8205260 2023-11-06T03:12:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7faa58021cb0 con 0x5555a8205260 2023-11-06T03:12:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 10 client.8001 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 10 client.8001 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 20 client.8001 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a8250bd0 0x5555a8250fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 4 client.8001 mds_command: new command op to 7785 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555a8052bf0 con 0x5555a8250bd0 2023-11-06T03:12:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa67fff640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a8250bd0 0x5555a8250fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa67fff640 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a8250bd0 0x5555a8250fb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96d2e5e384d3255 server_cookie=cb5aaea02f95bb54 in_seq=0 out_seq=0 2023-11-06T03:12:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 10 client.8001 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 1 -- 172.21.15.157:0/3690047824 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5555a8052bf0 con 0x5555a8250bd0 2023-11-06T03:12:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 10 client.8001 handle_command_reply: tid=0 2023-11-06T03:12:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 client.8001 shutdown 2023-11-06T03:12:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 msgr2=0x5555a82565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7faa58023040 tx=0x7faa580046d0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 shutdown_connections 2023-11-06T03:12:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a8250bd0 0x5555a8250fb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a824fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a824f130 0x5555a8256b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa477fe640 0 client.8001 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 --2- 172.21.15.157:0/3690047824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a8205260 0x5555a82565e0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 >> 172.21.15.157:0/3690047824 conn(0x5555a820b9f0 msgr2=0x5555a8203d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 shutdown_connections 2023-11-06T03:12:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 1 -- 172.21.15.157:0/3690047824 wait complete. 2023-11-06T03:12:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.742+0000 7faa72231000 20 client.8001 trim_cache size 0 max 16384 2023-11-06T03:12:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.754+0000 7faa72231000 1 Processor -- start 2023-11-06T03:12:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.754+0000 7faa72231000 1 -- start start 2023-11-06T03:12:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a8253fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a8254a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a7d11aa0 con 0x5555a824f130 2023-11-06T03:12:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a7ff97a0 con 0x5555a824fa50 2023-11-06T03:12:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555a8012400 con 0x5555a8205260 2023-11-06T03:12:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41620/0 (socket says 172.21.15.157:41620) 2023-11-06T03:12:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 -- 172.21.15.157:0/2031906038 learned_addr learned my addr 172.21.15.157:0/2031906038 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 -- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 msgr2=0x5555a8253fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a8253fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 -- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 msgr2=0x5555a8254a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a8254a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a8254a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 -- 172.21.15.157:0/2031906038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555a7fff520 con 0x5555a824f130 2023-11-06T03:12:22.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005890 tx=0x7faa5c001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10916fb2b632f3be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa4effd640 1 -- 172.21.15.157:0/2031906038 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c001ad0 con 0x5555a824f130 2023-11-06T03:12:22.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa4effd640 1 -- 172.21.15.157:0/2031906038 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c00d040 con 0x5555a824f130 2023-11-06T03:12:22.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa4effd640 1 -- 172.21.15.157:0/2031906038 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c017900 con 0x5555a824f130 2023-11-06T03:12:22.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- 172.21.15.157:0/2031906038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 msgr2=0x5555a8254510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005890 tx=0x7faa5c001570 comp rx=0 tx=0).stop 2023-11-06T03:12:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- 172.21.15.157:0/2031906038 shutdown_connections 2023-11-06T03:12:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824fa50 0x5555a8254a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824f130 0x5555a8254510 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- 172.21.15.157:0/2031906038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a8253fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- 172.21.15.157:0/2031906038 >> 172.21.15.157:0/2031906038 conn(0x5555a820b9f0 msgr2=0x5555a820ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- 172.21.15.157:0/2031906038 shutdown_connections 2023-11-06T03:12:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- 172.21.15.157:0/2031906038 wait complete. 2023-11-06T03:12:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- start start 2023-11-06T03:12:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 Processor -- start 2023-11-06T03:12:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a82a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 0x5555a82a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a80516a0 con 0x5555a824fa50 2023-11-06T03:12:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555a80582a0 con 0x5555a824f130 2023-11-06T03:12:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5555a804fc10 con 0x5555a8205260 2023-11-06T03:12:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa72231000 10 client.0 _ll_register_callbacks cb 0x5555a8018cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41632/0 (socket says 172.21.15.157:41632) 2023-11-06T03:12:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 -- 172.21.15.157:0/1191547570 learned_addr learned my addr 172.21.15.157:0/1191547570 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6cea5640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a82a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa67fff640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 0x5555a82a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 -- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 msgr2=0x5555a82a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a82a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 -- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 msgr2=0x5555a82a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.758+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 0x5555a82a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6d6a6640 1 -- 172.21.15.157:0/1191547570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680d2920 con 0x5555a824fa50 2023-11-06T03:12:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa67fff640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 0x5555a82a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6d6a6640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7faa6001a550 tx=0x7faa600ef950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd395363084092e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600f0390 con 0x5555a824fa50 2023-11-06T03:12:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa600f3d60 con 0x5555a824fa50 2023-11-06T03:12:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555a801d620 con 0x5555a824fa50 2023-11-06T03:12:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa600f7f10 con 0x5555a824fa50 2023-11-06T03:12:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 10 client.8007 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5555a7ff3090 con 0x5555a824fa50 2023-11-06T03:12:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5555a801d620 con 0x5555a824fa50 2023-11-06T03:12:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7faa60005d20 con 0x5555a824fa50 2023-11-06T03:12:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 10 client.8007 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 10 client.8007 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 20 client.8007 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a82a9a10 0x5555a82a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 4 client.8007 mds_command: new command op to 7785 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:12:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555a804d320 con 0x5555a82a9a10 2023-11-06T03:12:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6cea5640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a82a9a10 0x5555a82a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6cea5640 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a82a9a10 0x5555a82a9e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6c80b3bd39aed7 server_cookie=9ecfb71532bd5b29 in_seq=0 out_seq=0 2023-11-06T03:12:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 10 client.8007 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 1 -- 172.21.15.157:0/1191547570 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5555a804d320 con 0x5555a82a9a10 2023-11-06T03:12:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 10 client.8007 handle_command_reply: tid=0 2023-11-06T03:12:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 client.8007 shutdown 2023-11-06T03:12:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 msgr2=0x5555a82a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7faa6001a550 tx=0x7faa600ef950 comp rx=0 tx=0).stop 2023-11-06T03:12:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 shutdown_connections 2023-11-06T03:12:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x5555a82a9a10 0x5555a82a9e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5555a824fa50 0x5555a82a94d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5555a824f130 0x5555a82a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa477fe640 0 client.8007 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 --2- 172.21.15.157:0/1191547570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5555a8205260 0x5555a82a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 >> 172.21.15.157:0/1191547570 conn(0x5555a820b9f0 msgr2=0x5555a8252fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 shutdown_connections 2023-11-06T03:12:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 1 -- 172.21.15.157:0/1191547570 wait complete. 2023-11-06T03:12:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa72231000 20 client.8007 trim_cache size 0 max 16384 2023-11-06T03:12:22.786 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:22.786 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:12:22.786 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:12:22.786 INFO:teuthology.orchestra.run.smithi157.stdout: "ab5a6874-6fc6-4cbf-b1bf-9c2d95f7fe1d": { 2023-11-06T03:12:22.786 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_control_test_path", 2023-11-06T03:12:22.787 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "ab5a6874-6fc6-4cbf-b1bf-9c2d95f7fe1d", 2023-11-06T03:12:22.787 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:12:22.787 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:22.787 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:22.787 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:22.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 msgr2=0x7faa54067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 0x7faa54067f20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faa58009af0 tx=0x7faa58008040 comp rx=0 tx=0).stop 2023-11-06T03:12:22.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 msgr2=0x7faa681577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:22.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7faa600b0040 tx=0x7faa600196c0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 shutdown_connections 2023-11-06T03:12:22.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa54065a60 0x7faa54067f20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6815ce80 0x7faa681577b0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa6815caa0 0x7faa680ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 --2- 172.21.15.157:0/3914493518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faa680d0280 0x7faa68151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:22.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 >> 172.21.15.157:0/3914493518 conn(0x7faa68070790 msgr2=0x7faa68070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:22.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 shutdown_connections 2023-11-06T03:12:22.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:22.762+0000 7faa6f109640 1 -- 172.21.15.157:0/3914493518 wait complete. 2023-11-06T03:12:27.792 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 scrub status 2023-11-06T03:12:27.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 Processor -- start 2023-11-06T03:12:27.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 -- start start 2023-11-06T03:12:27.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7ac1599c0 0x7fa7ac0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ac15e440 con 0x7fa7ac06d000 2023-11-06T03:12:27.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac06c240 con 0x7fa7ac1599c0 2023-11-06T03:12:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7b13e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac06c520 con 0x7fa7ac0b2af0 2023-11-06T03:12:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.926+0000 7fa7aaffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7ac1599c0 0x7fa7ac0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7a27fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa7ac06d000 0x7fa7ac0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41036/0 (socket says 172.21.15.157:41036) 2023-11-06T03:12:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7a27fc640 1 -- 172.21.15.157:0/3573609403 learned_addr learned my addr 172.21.15.157:0/3573609403 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3573609403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 msgr2=0x7fa7ac159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3573609403 >> v1:172.21.15.157:6789/0 conn(0x7fa7ac06d000 legacy=0x7fa7ac0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3573609403 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac06c800 con 0x7fa7ac1599c0 2023-11-06T03:12:27.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aa7fc640 1 -- 172.21.15.157:0/3573609403 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080213263 0 0) 0x7fa7ac15e440 con 0x7fa7ac06d000 2023-11-06T03:12:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0ae000 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0097c0 tx=0x7fa79c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4287da88c04f651 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aa7fc640 1 -- 172.21.15.157:0/3573609403 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0ab670 con 0x7fa7ac1599c0 2023-11-06T03:12:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aa7fc640 1 -- 172.21.15.157:0/3573609403 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c0ab7d0 con 0x7fa7ac1599c0 2023-11-06T03:12:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7aa7fc640 1 -- 172.21.15.157:0/3573609403 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0abb60 con 0x7fa7ac1599c0 2023-11-06T03:12:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 -- 172.21.15.157:0/3573609403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 msgr2=0x7fa7ac0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0ae000 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0097c0 tx=0x7fa79c01c450 comp rx=0 tx=0).stop 2023-11-06T03:12:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 -- 172.21.15.157:0/3573609403 shutdown_connections 2023-11-06T03:12:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/3573609403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0ae000 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 -- 172.21.15.157:0/3573609403 >> 172.21.15.157:0/3573609403 conn(0x7fa7ac065930 msgr2=0x7fa7ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 -- 172.21.15.157:0/3573609403 shutdown_connections 2023-11-06T03:12:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.930+0000 7fa7b13e7640 1 -- 172.21.15.157:0/3573609403 wait complete. 2023-11-06T03:12:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 Processor -- start 2023-11-06T03:12:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- start start 2023-11-06T03:12:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7ac06d000 0x7fa7ac1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac0a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0ce230 con 0x7fa7ac1599c0 2023-11-06T03:12:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac152cc0 con 0x7fa7ac06d000 2023-11-06T03:12:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac152fa0 con 0x7fa7ac0b2af0 2023-11-06T03:12:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41646/0 (socket says 172.21.15.157:41646) 2023-11-06T03:12:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/185111755 learned_addr learned my addr 172.21.15.157:0/185111755 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7ac06d000 0x7fa7ac1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac0a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 msgr2=0x7fa7ac0a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac0a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7ac06d000 msgr2=0x7fa7ac1507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7ac06d000 0x7fa7ac1507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/185111755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c009420 con 0x7fa7ac1599c0 2023-11-06T03:12:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac0a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4007de0 tx=0x7fa7a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a3fff640 1 -- 172.21.15.157:0/185111755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4014620 con 0x7fa7ac1599c0 2023-11-06T03:12:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a3fff640 1 -- 172.21.15.157:0/185111755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a400ee90 con 0x7fa7ac1599c0 2023-11-06T03:12:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac153280 con 0x7fa7ac1599c0 2023-11-06T03:12:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a3fff640 1 -- 172.21.15.157:0/185111755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40149e0 con 0x7fa7ac1599c0 2023-11-06T03:12:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac0a8f60 con 0x7fa7ac1599c0 2023-11-06T03:12:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.934+0000 7fa7a3fff640 1 -- 172.21.15.157:0/185111755 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7a4014b40 con 0x7fa7ac1599c0 2023-11-06T03:12:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a3fff640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 0x7fa78c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a3fff640 1 -- 172.21.15.157:0/185111755 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7a4083d50 con 0x7fa7ac1599c0 2023-11-06T03:12:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 0x7fa78c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 Processor -- start 2023-11-06T03:12:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- start start 2023-11-06T03:12:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 0x7fa78c067f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0af000 tx=0x7fa79c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeeb4bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb4c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eeea13c10 con 0x555eeeb96130 2023-11-06T03:12:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee9a23e0 con 0x555eeeb96a50 2023-11-06T03:12:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eee99f9b0 con 0x555eeeb4c260 2023-11-06T03:12:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41658/0 (socket says 172.21.15.157:41658) 2023-11-06T03:12:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3465132913 learned_addr learned my addr 172.21.15.157:0/3465132913 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb4c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeeb4bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 msgr2=0x555eeeb4c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb4c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 msgr2=0x555eeeb4bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeeb4bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 -- 172.21.15.157:0/3465132913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eee697e20 con 0x555eeeb96130 2023-11-06T03:12:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeeb4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa79c01a080 tx=0x7fa79c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152d268fb9c92d00 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a0ff9640 1 -- 172.21.15.157:0/3465132913 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0c1790 con 0x555eeeb96130 2023-11-06T03:12:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a0ff9640 1 -- 172.21.15.157:0/3465132913 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c0c5070 con 0x555eeeb96130 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7a0ff9640 1 -- 172.21.15.157:0/3465132913 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0c1af0 con 0x555eeeb96130 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- 172.21.15.157:0/3465132913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 msgr2=0x555eeeb96510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa79c01a080 tx=0x7fa79c004740 comp rx=0 tx=0).stop 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- 172.21.15.157:0/3465132913 shutdown_connections 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb4c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeeb4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3465132913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeeb96510 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- 172.21.15.157:0/3465132913 >> 172.21.15.157:0/3465132913 conn(0x555eeeb529f0 msgr2=0x555eeeb4a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- 172.21.15.157:0/3465132913 shutdown_connections 2023-11-06T03:12:27.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- 172.21.15.157:0/3465132913 wait complete. 2023-11-06T03:12:27.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.938+0000 7fa7b450f000 1 -- start start 2023-11-06T03:12:27.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 Processor -- start 2023-11-06T03:12:27.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 0x555eeeb96e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee99b2a0 con 0x555eeeb4c260 2023-11-06T03:12:27.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee93a470 con 0x555eeeb96130 2023-11-06T03:12:27.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eee958660 con 0x555eeeb96a50 2023-11-06T03:12:27.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 10 client.0 _ll_register_callbacks cb 0x555eeeb37c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:27.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41672/0 (socket says 172.21.15.157:41672) 2023-11-06T03:12:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 -- 172.21.15.157:0/1904263485 learned_addr learned my addr 172.21.15.157:0/1904263485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 0x555eeeb96e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 -- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 msgr2=0x555eeeb96e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 0x555eeeb96e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 -- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 msgr2=0x555eeeb9db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 -- 172.21.15.157:0/1904263485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c0bb070 con 0x555eeeb4c260 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 0x555eeeb96e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:27.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0ad010 tx=0x7fa79c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7823309fbc0fb1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:27.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0c1c10 con 0x555eeeb4c260 2023-11-06T03:12:27.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa79c0c1d70 con 0x555eeeb4c260 2023-11-06T03:12:27.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eee9580a0 con 0x555eeeb4c260 2023-11-06T03:12:27.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa79c0ca430 con 0x555eeeb4c260 2023-11-06T03:12:27.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 10 client.8022 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:27.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x555eee9601d0 con 0x555eeeb4c260 2023-11-06T03:12:27.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555eee9580a0 con 0x555eeeb4c260 2023-11-06T03:12:27.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa79c0059a0 con 0x555eeeb4c260 2023-11-06T03:12:27.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 10 client.8022 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 10 client.8022 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 20 client.8022 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeeb97bd0 0x555eeeb97fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 4 client.8022 mds_command: new command op to 7785 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eee999bf0 con 0x555eeeb97bd0 2023-11-06T03:12:27.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeeb97bd0 0x555eeeb97fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeeb97bd0 0x555eeeb97fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cf5d5218b294a8 server_cookie=27f5acefc4833804 in_seq=0 out_seq=0 2023-11-06T03:12:27.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 10 client.8022 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:27.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 1 -- 172.21.15.157:0/1904263485 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555eee999bf0 con 0x555eeeb97bd0 2023-11-06T03:12:27.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 10 client.8022 handle_command_reply: tid=0 2023-11-06T03:12:27.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 client.8022 shutdown 2023-11-06T03:12:27.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 msgr2=0x555eeeb9d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0ad010 tx=0x7fa79c01ac90 comp rx=0 tx=0).stop 2023-11-06T03:12:27.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 shutdown_connections 2023-11-06T03:12:27.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeeb97bd0 0x555eeeb97fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb96a50 0x555eeeb96e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa789ffb640 0 client.8022 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:27.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1904263485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb4c260 0x555eeeb9d600 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 >> 172.21.15.157:0/1904263485 conn(0x555eeeb529f0 msgr2=0x555eeeb4ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 shutdown_connections 2023-11-06T03:12:27.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 1 -- 172.21.15.157:0/1904263485 wait complete. 2023-11-06T03:12:27.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.942+0000 7fa7b450f000 20 client.8022 trim_cache size 0 max 16384 2023-11-06T03:12:27.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 Processor -- start 2023-11-06T03:12:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 -- start start 2023-11-06T03:12:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee658aa0 con 0x555eeeb96a50 2023-11-06T03:12:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee99de00 con 0x555eeeb96130 2023-11-06T03:12:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eeeb4f530 con 0x555eeeb4c260 2023-11-06T03:12:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41684/0 (socket says 172.21.15.157:41684) 2023-11-06T03:12:27.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/1323818828 learned_addr learned my addr 172.21.15.157:0/1323818828 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 msgr2=0x555eeeb9afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb9afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 msgr2=0x555eeeb9b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 -- 172.21.15.157:0/1323818828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eee946520 con 0x555eeeb96a50 2023-11-06T03:12:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4055200 tx=0x7fa7a4054ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373c7afbea6cc6c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7a17fa640 1 -- 172.21.15.157:0/1323818828 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40590a0 con 0x555eeeb96a50 2023-11-06T03:12:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7a17fa640 1 -- 172.21.15.157:0/1323818828 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a405d250 con 0x555eeeb96a50 2023-11-06T03:12:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.958+0000 7fa7a17fa640 1 -- 172.21.15.157:0/1323818828 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4053610 con 0x555eeeb96a50 2023-11-06T03:12:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/1323818828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 msgr2=0x555eeeb9ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4055200 tx=0x7fa7a4054ee0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/1323818828 shutdown_connections 2023-11-06T03:12:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96a50 0x555eeeb9ba50 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96130 0x555eeeb9b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/1323818828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeeb9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/1323818828 >> 172.21.15.157:0/1323818828 conn(0x555eeeb529f0 msgr2=0x555eeeb51a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/1323818828 shutdown_connections 2023-11-06T03:12:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/1323818828 wait complete. 2023-11-06T03:12:27.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- start start 2023-11-06T03:12:27.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 Processor -- start 2023-11-06T03:12:27.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeebf2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeebd6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eeea3fbc0 con 0x555eeeb96130 2023-11-06T03:12:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eee9407a0 con 0x555eeeb96a50 2023-11-06T03:12:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eee995640 con 0x555eeeb4c260 2023-11-06T03:12:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 10 client.0 _ll_register_callbacks cb 0x555eee95fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41700/0 (socket says 172.21.15.157:41700) 2023-11-06T03:12:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 -- 172.21.15.157:0/3848237672 learned_addr learned my addr 172.21.15.157:0/3848237672 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeebf2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeebd6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 -- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 msgr2=0x555eeebf2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeebf2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 -- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 msgr2=0x555eeebd6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeebd6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 -- 172.21.15.157:0/3848237672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac0a8f60 con 0x555eeeb96130 2023-11-06T03:12:27.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7ab7fe640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeebd6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7a27fc640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac06bc20 tx=0x7fa794001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829ab50aeb5f060b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794005cd0 con 0x555eeeb96130 2023-11-06T03:12:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa79400c040 con 0x555eeeb96130 2023-11-06T03:12:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eee964620 con 0x555eeeb96130 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa794005e30 con 0x555eeeb96130 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 10 client.8028 fetch_fsmap learned FSMap version 79 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x555eee93a090 con 0x555eeeb96130 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555eee964620 con 0x555eeeb96130 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa7940047f0 con 0x555eeeb96130 2023-11-06T03:12:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 10 client.8028 fetch_fsmap finished waiting for FSMap version 79 2023-11-06T03:12:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 10 client.8028 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2023-11-06T03:12:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 20 client.8028 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeebd70b0 0x555eeebd74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 4 client.8028 mds_command: new command op to 7785 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:12:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 --> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555eee994320 con 0x555eeebd70b0 2023-11-06T03:12:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeebd70b0 0x555eeebd74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 10 client.8028 ms_handle_connect on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7aaffd640 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeebd70b0 0x555eeebd74c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c971b0bce8512cc1 server_cookie=9d8e46bf5956df03 in_seq=0 out_seq=0 2023-11-06T03:12:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 1 -- 172.21.15.157:0/3848237672 <== mds.0 v2:172.21.15.157:6834/1762973381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555eee994320 con 0x555eeebd70b0 2023-11-06T03:12:27.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 10 client.8028 handle_command_reply: tid=0 2023-11-06T03:12:27.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 client.8028 shutdown 2023-11-06T03:12:27.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 msgr2=0x555eeebf32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac06bc20 tx=0x7fa794001cd0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 shutdown_connections 2023-11-06T03:12:27.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:6834/1762973381,v1:172.21.15.157:6835/1762973381] conn(0x555eeebd70b0 0x555eeebd74c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555eeeb96a50 0x555eeebd6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555eeeb96130 0x555eeebf32f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa789ffb640 0 client.8028 ms_handle_reset on v2:172.21.15.157:6834/1762973381 2023-11-06T03:12:27.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 --2- 172.21.15.157:0/3848237672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555eeeb4c260 0x555eeebf2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 >> 172.21.15.157:0/3848237672 conn(0x555eeeb529f0 msgr2=0x555eeeb99fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 shutdown_connections 2023-11-06T03:12:27.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.962+0000 7fa7b450f000 1 -- 172.21.15.157:0/3848237672 wait complete. 2023-11-06T03:12:27.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b450f000 20 client.8028 trim_cache size 0 max 16384 2023-11-06T03:12:27.991 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:12:27.991 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:12:27.991 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:27.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 msgr2=0x7fa78c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 0x7fa78c067f20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0af000 tx=0x7fa79c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:12:27.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 msgr2=0x7fa7ac0aa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:27.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4007de0 tx=0x7fa7a400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 shutdown_connections 2023-11-06T03:12:27.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa78c065a60 0x7fa78c067f20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7ac1599c0 0x7fa7ac0aa5e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7ac0b2af0 0x7fa7ac0a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 --2- 172.21.15.157:0/185111755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7ac06d000 0x7fa7ac1507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:27.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 >> 172.21.15.157:0/185111755 conn(0x7fa7ac065930 msgr2=0x7fa7ac157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:27.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 shutdown_connections 2023-11-06T03:12:27.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:27.966+0000 7fa7b13e7640 1 -- 172.21.15.157:0/185111755 wait complete. 2023-11-06T03:12:27.995 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:12:27.996 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:28.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.106+0000 7f2ea8f9d640 1 Processor -- start 2023-11-06T03:12:28.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 -- start start 2023-11-06T03:12:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ea40ccf20 0x7f2ea40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 0x7f2ea40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea41617d0 con 0x7f2ea415cb10 2023-11-06T03:12:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea4160fb0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea8f9d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea4161210 con 0x7f2ea415cef0 2023-11-06T03:12:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ea40ccf20 0x7f2ea40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ea40ccf20 0x7f2ea40cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41716/0 (socket says 172.21.15.157:41716) 2023-11-06T03:12:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 -- 172.21.15.157:0/32838089 learned_addr learned my addr 172.21.15.157:0/32838089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea2d9d640 1 --2- 172.21.15.157:0/32838089 >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 0x7f2ea40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 -- 172.21.15.157:0/32838089 >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 msgr2=0x7f2ea40cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/32838089 >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 0x7f2ea40cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 -- 172.21.15.157:0/32838089 >> v1:172.21.15.195:6789/0 conn(0x7f2ea415cb10 legacy=0x7f2ea41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 -- 172.21.15.157:0/32838089 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea41614f0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea2d9d640 1 --2- 172.21.15.157:0/32838089 >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 0x7f2ea40cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/32838089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea40cd390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40cf390 tx=0x7f2e9801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4690a87bec0976d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea159a640 1 -- 172.21.15.157:0/32838089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab600 con 0x7f2ea40ccf20 2023-11-06T03:12:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea159a640 1 -- 172.21.15.157:0/32838089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980ab760 con 0x7f2ea40ccf20 2023-11-06T03:12:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.110+0000 7f2ea159a640 1 -- 172.21.15.157:0/32838089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e980abaf0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/32838089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 msgr2=0x7f2ea40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/32838089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea40cd390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40cf390 tx=0x7f2e9801c400 comp rx=0 tx=0).stop 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/32838089 shutdown_connections 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/32838089 >> v2:172.21.15.195:3300/0 conn(0x7f2ea415cef0 0x7f2ea40cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/32838089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea40cd390 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/32838089 >> 172.21.15.157:0/32838089 conn(0x7f2ea406d560 msgr2=0x7f2ea406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/32838089 shutdown_connections 2023-11-06T03:12:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/32838089 wait complete. 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 Processor -- start 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- start start 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 0x7f2ea415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 0x7f2ea415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea414cd20 con 0x7f2ea40ccf20 2023-11-06T03:12:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40ca8c0 con 0x7f2ea415cef0 2023-11-06T03:12:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40ca300 con 0x7f2ea415cb10 2023-11-06T03:12:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41722/0 (socket says 172.21.15.157:41722) 2023-11-06T03:12:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 -- 172.21.15.157:0/661217809 learned_addr learned my addr 172.21.15.157:0/661217809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea2d9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 0x7f2ea415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 0x7f2ea415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 -- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 msgr2=0x7f2ea415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 0x7f2ea415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 -- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 msgr2=0x7f2ea415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 0x7f2ea415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 -- 172.21.15.157:0/661217809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980094d0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 0x7f2ea415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea2d9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 0x7f2ea415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea259c640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c450 tx=0x7f2e9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94012830 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e940129b0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea40ca5e0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94012d60 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.114+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea4153400 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea4072750 con 0x7f2ea40ccf20 2023-11-06T03:12:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2e9401c460 con 0x7f2ea40ccf20 2023-11-06T03:12:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2e92ffd640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 0x7f2e84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e94083130 con 0x7f2ea40ccf20 2023-11-06T03:12:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 0x7f2e84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.118+0000 7f2ea1d9b640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 0x7f2e84067f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e980af000 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.122+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2e94050990 con 0x7f2ea40ccf20 2023-11-06T03:12:28.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.310+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ea40b6990 con 0x7f2ea40ccf20 2023-11-06T03:12:28.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.310+0000 7f2e92ffd640 1 -- 172.21.15.157:0/661217809 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f2e94053fb0 con 0x7f2ea40ccf20 2023-11-06T03:12:28.312 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:28.312 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:28.312 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 msgr2=0x7f2e84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 0x7f2e84067f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e980af000 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2023-11-06T03:12:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 msgr2=0x7f2ea414c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c450 tx=0x7f2e9400c910 comp rx=0 tx=0).stop 2023-11-06T03:12:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 shutdown_connections 2023-11-06T03:12:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2e84065a60 0x7f2e84067f20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ea415cef0 0x7f2ea415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ea415cb10 0x7f2ea415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 --2- 172.21.15.157:0/661217809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea40ccf20 0x7f2ea414c7e0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 >> 172.21.15.157:0/661217809 conn(0x7f2ea406d560 msgr2=0x7f2ea415c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:28.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 shutdown_connections 2023-11-06T03:12:28.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.314+0000 7f2ea8f9d640 1 -- 172.21.15.157:0/661217809 wait complete. 2023-11-06T03:12:28.330 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:12:28.331 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:12:28.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 Processor -- start 2023-11-06T03:12:28.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 -- start start 2023-11-06T03:12:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f98700ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98700b2af0 0x7f9870159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987015e440 con 0x7f987006d000 2023-11-06T03:12:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987006c240 con 0x7f98700b2af0 2023-11-06T03:12:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.446+0000 7f987600c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987006c520 con 0x7f98701599c0 2023-11-06T03:12:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f987006d000 0x7f98700b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41042/0 (socket says 172.21.15.157:41042) 2023-11-06T03:12:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 -- 172.21.15.157:0/3986074911 learned_addr learned my addr 172.21.15.157:0/3986074911 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:28.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986ffff640 1 --2- 172.21.15.157:0/3986074911 >> v2:172.21.15.195:3300/0 conn(0x7f98700b2af0 0x7f9870159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986f7fe640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f98700ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928290300 0 0) 0x7f987015e440 con 0x7f987006d000 2023-11-06T03:12:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985c003530 con 0x7f987006d000 2023-11-06T03:12:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676500086 0 0) 0x7f985c003530 con 0x7f987006d000 2023-11-06T03:12:28.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 msgr2=0x7f98700ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f98700ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 >> v2:172.21.15.195:3300/0 conn(0x7f98700b2af0 msgr2=0x7f9870159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 --2- 172.21.15.157:0/3986074911 >> v2:172.21.15.195:3300/0 conn(0x7f98700b2af0 0x7f9870159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987006c800 con 0x7f987006d000 2023-11-06T03:12:28.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9858001680 con 0x7f987006d000 2023-11-06T03:12:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 >> v1:172.21.15.157:6789/0 conn(0x7f987006d000 legacy=0x7f98700b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c003980 0x7f985c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985c0047d0 0x7f985c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f985c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987006c240 con 0x7f985c0047d0 2023-11-06T03:12:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987006c520 con 0x7f985c003980 2023-11-06T03:12:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c00b990 con 0x7f98701599c0 2023-11-06T03:12:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985c0047d0 0x7f985c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986f7fe640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c003980 0x7f985c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986ffff640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f985c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 -- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 msgr2=0x7f985c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98701599c0 0x7f985c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 -- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c003980 msgr2=0x7f985c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c003980 0x7f985c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 -- 172.21.15.157:0/3986074911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c003530 con 0x7f985c0047d0 2023-11-06T03:12:28.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986f7fe640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c003980 0x7f985c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f9867fff640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985c0047d0 0x7f985c004c50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f985800c190 tx=0x7f985801f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915b26faa94dd48d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580a8c00 con 0x7f985c0047d0 2023-11-06T03:12:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98580a8d80 con 0x7f985c0047d0 2023-11-06T03:12:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.450+0000 7f986effd640 1 -- 172.21.15.157:0/3986074911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858005b20 con 0x7f985c0047d0 2023-11-06T03:12:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/3986074911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985c0047d0 msgr2=0x7f985c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 --2- 172.21.15.157:0/3986074911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f985c0047d0 0x7f985c004c50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f985800c190 tx=0x7f985801f310 comp rx=0 tx=0).stop 2023-11-06T03:12:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 -- 172.21.15.157:0/3986074911 reap_dead start 2023-11-06T03:12:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/3986074911 shutdown_connections 2023-11-06T03:12:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/3986074911 >> 172.21.15.157:0/3986074911 conn(0x7f9870065930 msgr2=0x7f9870065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/3986074911 shutdown_connections 2023-11-06T03:12:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/3986074911 wait complete. 2023-11-06T03:12:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 Processor -- start 2023-11-06T03:12:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- start start 2023-11-06T03:12:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 0x7f987014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 0x7f987014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987014ed50 con 0x7f987014e8b0 2023-11-06T03:12:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987014f720 con 0x7f985c0047d0 2023-11-06T03:12:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987014fa00 con 0x7f98700cebc0 2023-11-06T03:12:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41738/0 (socket says 172.21.15.157:41738) 2023-11-06T03:12:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 -- 172.21.15.157:0/4002136804 learned_addr learned my addr 172.21.15.157:0/4002136804 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986f7fe640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 0x7f987014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f9867fff640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 0x7f987014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 -- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 msgr2=0x7f987014e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 0x7f987014e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 -- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 msgr2=0x7f987014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 0x7f987014b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 -- 172.21.15.157:0/4002136804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987014fce0 con 0x7f987014e8b0 2023-11-06T03:12:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986f7fe640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 0x7f987014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f9867fff640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 0x7f987014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986ffff640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f98700b1c30 tx=0x7f986800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868017400 con 0x7f987014e8b0 2023-11-06T03:12:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9868011040 con 0x7f987014e8b0 2023-11-06T03:12:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98680177c0 con 0x7f987014e8b0 2023-11-06T03:12:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987014ffc0 con 0x7f987014e8b0 2023-11-06T03:12:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98701504e0 con 0x7f987014e8b0 2023-11-06T03:12:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.454+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98701501b0 con 0x7f987014e8b0 2023-11-06T03:12:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.458+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9868017920 con 0x7f987014e8b0 2023-11-06T03:12:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.458+0000 7f986cff9640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 0x7f9850067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.458+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9868083ff0 con 0x7f987014e8b0 2023-11-06T03:12:28.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.458+0000 7f986f7fe640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 0x7f9850067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.458+0000 7f986f7fe640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 0x7f9850067f50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f98600099e0 tx=0x7f9860008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.462+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9868051850 con 0x7f987014e8b0 2023-11-06T03:12:28.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.674+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f98700adcd0 con 0x7f987014e8b0 2023-11-06T03:12:28.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.674+0000 7f986cff9640 1 -- 172.21.15.157:0/4002136804 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f986801c4c0 con 0x7f987014e8b0 2023-11-06T03:12:28.675 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:28.675 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":526,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1058,"data_bytes":1077938590,"bytes_used":2744799232,"bytes_avail":770349314048,"bytes_total":773094113280,"write_bytes_sec":44492686,"read_op_per_sec":0,"write_op_per_sec":73},"fsmap":{"epoch":79,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7785}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2023-11-06T03:11:23.254243+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:12:28.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 msgr2=0x7f9850067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 0x7f9850067f50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f98600099e0 tx=0x7f9860008040 comp rx=0 tx=0).stop 2023-11-06T03:12:28.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 msgr2=0x7f98700cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f98700b1c30 tx=0x7f986800f4a0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 shutdown_connections 2023-11-06T03:12:28.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9850065a90 0x7f9850067f50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f987014e8b0 0x7f98700cbc70 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98700cebc0 0x7f987014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 --2- 172.21.15.157:0/4002136804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985c0047d0 0x7f987014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 >> 172.21.15.157:0/4002136804 conn(0x7f9870065930 msgr2=0x7f9870070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 shutdown_connections 2023-11-06T03:12:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.678+0000 7f987600c640 1 -- 172.21.15.157:0/4002136804 wait complete. 2023-11-06T03:12:28.693 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:12:28.694 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:12:28.695 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:12:28.742 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:12:28.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:12:28.790 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:12:28.791 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:28.796 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:28.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.958+0000 7f3f8ae06640 1 Processor -- start 2023-11-06T03:12:28.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.958+0000 7f3f8ae06640 1 -- start start 2023-11-06T03:12:28.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f8ae06640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f8415e5e0 0x7f3f840b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f8ae06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415ea90 0x7f3f840b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f8ae06640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8415f580 con 0x7f3f841551e0 2023-11-06T03:12:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f8ae06640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f840704b0 con 0x7f3f8415e5e0 2023-11-06T03:12:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f8ae06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8415efc0 con 0x7f3f8415ea90 2023-11-06T03:12:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f8415e5e0 0x7f3f840b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f8415e5e0 0x7f3f840b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46122/0 (socket says 172.21.15.157:46122) 2023-11-06T03:12:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f88ba2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3f841551e0 0x7f3f8406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37624/0 (socket says 172.21.15.157:37624) 2023-11-06T03:12:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 -- 172.21.15.157:0/2028221883 learned_addr learned my addr 172.21.15.157:0/2028221883 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f893a3640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415ea90 0x7f3f840b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 -- 172.21.15.157:0/2028221883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415ea90 msgr2=0x7f3f840b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:28.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415ea90 0x7f3f840b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 -- 172.21.15.157:0/2028221883 >> v1:172.21.15.157:6789/0 conn(0x7f3f841551e0 legacy=0x7f3f8406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 -- 172.21.15.157:0/2028221883 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8415f2a0 con 0x7f3f8415e5e0 2023-11-06T03:12:28.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f837fe640 1 -- 172.21.15.157:0/2028221883 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978199409 0 0) 0x7f3f8415f580 con 0x7f3f841551e0 2023-11-06T03:12:28.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f83fff640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8415e5e0 0x7f3f840b3410 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009870 tx=0x7f3f7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f88e12208e9b24 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f837fe640 1 -- 172.21.15.157:0/2028221883 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780ab680 con 0x7f3f8415e5e0 2023-11-06T03:12:28.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f837fe640 1 -- 172.21.15.157:0/2028221883 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f780ab7e0 con 0x7f3f8415e5e0 2023-11-06T03:12:28.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.962+0000 7f3f837fe640 1 -- 172.21.15.157:0/2028221883 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f780abb70 con 0x7f3f8415e5e0 2023-11-06T03:12:28.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2028221883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8415e5e0 msgr2=0x7f3f840b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8415e5e0 0x7f3f840b3410 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009870 tx=0x7f3f7801c480 comp rx=0 tx=0).stop 2023-11-06T03:12:28.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2028221883 shutdown_connections 2023-11-06T03:12:28.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415ea90 0x7f3f840b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2028221883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f8415e5e0 0x7f3f840b3410 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2028221883 >> 172.21.15.157:0/2028221883 conn(0x7f3f84065930 msgr2=0x7f3f84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:28.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2028221883 shutdown_connections 2023-11-06T03:12:28.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2028221883 wait complete. 2023-11-06T03:12:28.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 Processor -- start 2023-11-06T03:12:28.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- start start 2023-11-06T03:12:28.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415e5e0 0x7f3f84151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f8415ea90 0x7f3f84138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84153050 con 0x7f3f841551e0 2023-11-06T03:12:28.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84152190 con 0x7f3f8415ea90 2023-11-06T03:12:28.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841f8a00 con 0x7f3f8415e5e0 2023-11-06T03:12:28.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46126/0 (socket says 172.21.15.157:46126) 2023-11-06T03:12:28.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 -- 172.21.15.157:0/2261627040 learned_addr learned my addr 172.21.15.157:0/2261627040 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:28.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 -- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415e5e0 msgr2=0x7f3f84151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f83fff640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415e5e0 0x7f3f84151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415e5e0 0x7f3f84151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 -- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f8415ea90 msgr2=0x7f3f84138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:28.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f893a3640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f8415ea90 0x7f3f84138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f8415ea90 0x7f3f84138e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:28.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 -- 172.21.15.157:0/2261627040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780094d0 con 0x7f3f841551e0 2023-11-06T03:12:28.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f88ba2640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3f740011f0 tx=0x7f3f7400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740074e0 con 0x7f3f841551e0 2023-11-06T03:12:28.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74007640 con 0x7f3f841551e0 2023-11-06T03:12:28.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f841f8ce0 con 0x7f3f841551e0 2023-11-06T03:12:28.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74004a10 con 0x7f3f841551e0 2023-11-06T03:12:28.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.966+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f841f92a0 con 0x7f3f841551e0 2023-11-06T03:12:28.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84072750 con 0x7f3f841551e0 2023-11-06T03:12:28.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3f74004b70 con 0x7f3f841551e0 2023-11-06T03:12:28.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f817fa640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 0x7f3f70067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:28.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f740828a0 con 0x7f3f841551e0 2023-11-06T03:12:28.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f83fff640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 0x7f3f70067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:28.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.970+0000 7f3f83fff640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 0x7f3f70067f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3f780af000 tx=0x7f3f780a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:28.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:28.974+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3f74050100 con 0x7f3f841551e0 2023-11-06T03:12:29.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.126+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f840cca20 con 0x7f3f841551e0 2023-11-06T03:12:29.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.126+0000 7f3f817fa640 1 -- 172.21.15.157:0/2261627040 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17874 (secure 0 0 0) 0x7f3f74018020 con 0x7f3f841551e0 2023-11-06T03:12:29.127 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:29.128 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:59.421679+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:29.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 msgr2=0x7f3f70067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 0x7f3f70067f20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3f780af000 tx=0x7f3f780a7000 comp rx=0 tx=0).stop 2023-11-06T03:12:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 msgr2=0x7f3f84139710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3f740011f0 tx=0x7f3f7400c9b0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 shutdown_connections 2023-11-06T03:12:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3f70065a60 0x7f3f70067f20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f8415ea90 0x7f3f84138e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f8415e5e0 0x7f3f84151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 --2- 172.21.15.157:0/2261627040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f841551e0 0x7f3f84139710 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 >> 172.21.15.157:0/2261627040 conn(0x7f3f84065930 msgr2=0x7f3f8415dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 shutdown_connections 2023-11-06T03:12:29.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.130+0000 7f3f8ae06640 1 -- 172.21.15.157:0/2261627040 wait complete. 2023-11-06T03:12:29.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:11:56.975891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:11:57.391711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:12:29.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:12:29.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 Processor -- start 2023-11-06T03:12:29.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 -- start start 2023-11-06T03:12:29.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78159930 0x7f0e780b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7815a860 con 0x7f0e780b5320 2023-11-06T03:12:29.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780711b0 con 0x7f0e780b3020 2023-11-06T03:12:29.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7cb63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78159d10 con 0x7f0e78159930 2023-11-06T03:12:29.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e76d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0e780b5320 0x7f0e7806c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37638/0 (socket says 172.21.15.157:37638) 2023-11-06T03:12:29.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e76d9d640 1 -- 172.21.15.157:0/80090263 learned_addr learned my addr 172.21.15.157:0/80090263 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:29.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78159930 0x7f0e780b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178549162 0 0) 0x7f0e7815a860 con 0x7f0e780b5320 2023-11-06T03:12:29.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003530 con 0x7f0e780b5320 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4087642954 0 0) 0x7f0e60003530 con 0x7f0e780b5320 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78159930 msgr2=0x7f0e780b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78159930 0x7f0e780b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 >> v1:172.21.15.195:6789/0 conn(0x7f0e780b3020 legacy=0x7f0e780b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78159f30 con 0x7f0e780b5320 2023-11-06T03:12:29.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0e70001a00 con 0x7f0e780b5320 2023-11-06T03:12:29.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 >> v1:172.21.15.157:6789/0 conn(0x7f0e780b5320 legacy=0x7f0e7806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600039b0 0x7f0e60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e600047d0 0x7f0e60004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600052b0 0x7f0e6000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78159d10 con 0x7f0e600039b0 2023-11-06T03:12:29.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6000c0e0 con 0x7f0e600047d0 2023-11-06T03:12:29.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6000e4b0 con 0x7f0e600052b0 2023-11-06T03:12:29.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e78159930 0x7f0e780b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:29.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600039b0 0x7f0e60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e75d9b640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e600047d0 0x7f0e60004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e76d9d640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600052b0 0x7f0e6000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 -- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600052b0 msgr2=0x7f0e6000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600052b0 0x7f0e6000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 -- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e600047d0 msgr2=0x7f0e60004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e600047d0 0x7f0e60004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.270+0000 7f0e7659c640 1 -- 172.21.15.157:0/80090263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60003530 con 0x7f0e600039b0 2023-11-06T03:12:29.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7659c640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600039b0 0x7f0e60003dd0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0e78070990 tx=0x7f0e6801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691b34e64df0b08a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680ab600 con 0x7f0e600039b0 2023-11-06T03:12:29.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e680a7050 con 0x7f0e600039b0 2023-11-06T03:12:29.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7559a640 1 -- 172.21.15.157:0/80090263 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680ab980 con 0x7f0e600039b0 2023-11-06T03:12:29.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 -- 172.21.15.157:0/80090263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600039b0 msgr2=0x7f0e60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/80090263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e600039b0 0x7f0e60003dd0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0e78070990 tx=0x7f0e6801c3f0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e76d9d640 1 -- 172.21.15.157:0/80090263 reap_dead start 2023-11-06T03:12:29.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 -- 172.21.15.157:0/80090263 shutdown_connections 2023-11-06T03:12:29.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 -- 172.21.15.157:0/80090263 >> 172.21.15.157:0/80090263 conn(0x7f0e78069d70 msgr2=0x7f0e7806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 -- 172.21.15.157:0/80090263 shutdown_connections 2023-11-06T03:12:29.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.274+0000 7f0e7cb63640 1 -- 172.21.15.157:0/80090263 wait complete. 2023-11-06T03:12:29.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 Processor -- start 2023-11-06T03:12:29.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- start start 2023-11-06T03:12:29.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600039b0 0x7f0e780afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 0x7f0e781f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780af500 con 0x7f0e780c4720 2023-11-06T03:12:29.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780c4ba0 con 0x7f0e7814cff0 2023-11-06T03:12:29.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780b0260 con 0x7f0e600039b0 2023-11-06T03:12:29.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46144/0 (socket says 172.21.15.157:46144) 2023-11-06T03:12:29.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 -- 172.21.15.157:0/2578641116 learned_addr learned my addr 172.21.15.157:0/2578641116 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:29.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e76d9d640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 0x7f0e781f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7659c640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600039b0 0x7f0e780afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 -- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600039b0 msgr2=0x7f0e780afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600039b0 0x7f0e780afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 -- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 msgr2=0x7f0e781f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 0x7f0e781f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 -- 172.21.15.157:0/2578641116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780ae200 con 0x7f0e780c4720 2023-11-06T03:12:29.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e76d9d640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 0x7f0e781f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:29.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e75d9b640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e550 tx=0x7f0e6c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0052e0 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c005440 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e780ae4e0 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c003820 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e780aeaa0 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e78072750 con 0x7f0e780c4720 2023-11-06T03:12:29.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0e6c003980 con 0x7f0e780c4720 2023-11-06T03:12:29.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 0x7f0e58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.278+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e6c083060 con 0x7f0e780c4720 2023-11-06T03:12:29.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.282+0000 7f0e7659c640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 0x7f0e58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.282+0000 7f0e7659c640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 0x7f0e58067f20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ad000 tx=0x7f0e680046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.282+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0e6c050950 con 0x7f0e780c4720 2023-11-06T03:12:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.430+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e780cd010 con 0x7f0e780c4720 2023-11-06T03:12:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.430+0000 7f0e66ffd640 1 -- 172.21.15.157:0/2578641116 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e6c053f70 con 0x7f0e780c4720 2023-11-06T03:12:29.431 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:12:29.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 msgr2=0x7f0e58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 0x7f0e58067f20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ad000 tx=0x7f0e680046d0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 msgr2=0x7f0e780aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e550 tx=0x7f0e6c00ea10 comp rx=0 tx=0).stop 2023-11-06T03:12:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 shutdown_connections 2023-11-06T03:12:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0e58065a60 0x7f0e58067f20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e7814cff0 0x7f0e781f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e780c4720 0x7f0e780aefc0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 --2- 172.21.15.157:0/2578641116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e600039b0 0x7f0e780afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 >> 172.21.15.157:0/2578641116 conn(0x7f0e78069d70 msgr2=0x7f0e780cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 shutdown_connections 2023-11-06T03:12:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.434+0000 7f0e7cb63640 1 -- 172.21.15.157:0/2578641116 wait complete. 2023-11-06T03:12:29.448 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:12:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 Processor -- start 2023-11-06T03:12:29.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 -- start start 2023-11-06T03:12:29.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbe840b2af0 0x7fbe840ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe84159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8415e440 con 0x7fbe841599c0 2023-11-06T03:12:29.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8406c240 con 0x7fbe840b2af0 2023-11-06T03:12:29.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe8adf7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8406c520 con 0x7fbe8406d000 2023-11-06T03:12:29.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbe841599c0 0x7fbe840b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37650/0 (socket says 172.21.15.157:37650) 2023-11-06T03:12:29.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 -- 172.21.15.157:0/2220937821 learned_addr learned my addr 172.21.15.157:0/2220937821 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:29.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe89394640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe84159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe83fff640 1 --2- 172.21.15.157:0/2220937821 >> v2:172.21.15.195:3300/0 conn(0x7fbe840b2af0 0x7fbe840ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810468542 0 0) 0x7fbe8415e440 con 0x7fbe841599c0 2023-11-06T03:12:29.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe74003530 con 0x7fbe841599c0 2023-11-06T03:12:29.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1290396411 0 0) 0x7fbe74003530 con 0x7fbe841599c0 2023-11-06T03:12:29.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 msgr2=0x7fbe84159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe84159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 >> v2:172.21.15.195:3300/0 conn(0x7fbe840b2af0 msgr2=0x7fbe840ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 --2- 172.21.15.157:0/2220937821 >> v2:172.21.15.195:3300/0 conn(0x7fbe840b2af0 0x7fbe840ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8406c800 con 0x7fbe841599c0 2023-11-06T03:12:29.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fbe78001680 con 0x7fbe841599c0 2023-11-06T03:12:29.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 >> v1:172.21.15.157:6789/0 conn(0x7fbe841599c0 legacy=0x7fbe840b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003980 0x7fbe74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe74004d10 0x7fbe74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8406c240 con 0x7fbe74003980 2023-11-06T03:12:29.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8406c520 con 0x7fbe74004d10 2023-11-06T03:12:29.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7400b990 con 0x7fbe8406d000 2023-11-06T03:12:29.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003980 0x7fbe74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe83fff640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe89394640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe74004d10 0x7fbe74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 -- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 msgr2=0x7fbe740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 -- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe74004d10 msgr2=0x7fbe74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe74004d10 0x7fbe74005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe88b93640 1 -- 172.21.15.157:0/2220937821 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74003530 con 0x7fbe74003980 2023-11-06T03:12:29.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.574+0000 7fbe83fff640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe8406d000 0x7fbe740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003980 0x7fbe74003da0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c210 tx=0x7fbe7801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daec46df30944c28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780a5910 con 0x7fbe74003980 2023-11-06T03:12:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe780a5a70 con 0x7fbe74003980 2023-11-06T03:12:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe837fe640 1 -- 172.21.15.157:0/2220937821 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780a5c20 con 0x7fbe74003980 2023-11-06T03:12:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2220937821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003980 msgr2=0x7fbe74003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2220937821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe74003980 0x7fbe74003da0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c210 tx=0x7fbe7801cc10 comp rx=0 tx=0).stop 2023-11-06T03:12:29.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 -- 172.21.15.157:0/2220937821 reap_dead start 2023-11-06T03:12:29.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2220937821 shutdown_connections 2023-11-06T03:12:29.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2220937821 >> 172.21.15.157:0/2220937821 conn(0x7fbe84065930 msgr2=0x7fbe84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2220937821 shutdown_connections 2023-11-06T03:12:29.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2220937821 wait complete. 2023-11-06T03:12:29.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 Processor -- start 2023-11-06T03:12:29.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- start start 2023-11-06T03:12:29.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe74003980 0x7fbe8414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe84156890 0x7fbe84150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8414c3e0 con 0x7fbe84156d10 2023-11-06T03:12:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840aedb0 con 0x7fbe84156890 2023-11-06T03:12:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe8adf7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840af090 con 0x7fbe74003980 2023-11-06T03:12:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46150/0 (socket says 172.21.15.157:46150) 2023-11-06T03:12:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 -- 172.21.15.157:0/2793080345 learned_addr learned my addr 172.21.15.157:0/2793080345 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe74003980 0x7fbe8414bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe83fff640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe84156890 0x7fbe84150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 -- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe74003980 msgr2=0x7fbe8414bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe74003980 0x7fbe8414bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 -- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe84156890 msgr2=0x7fbe84150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe84156890 0x7fbe84150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 -- 172.21.15.157:0/2793080345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840af370 con 0x7fbe84156d10 2023-11-06T03:12:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.578+0000 7fbe89394640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406d430 tx=0x7fbe7c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017400 con 0x7fbe84156d10 2023-11-06T03:12:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c011040 con 0x7fbe84156d10 2023-11-06T03:12:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840af650 con 0x7fbe84156d10 2023-11-06T03:12:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0177e0 con 0x7fbe84156d10 2023-11-06T03:12:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe840ae9d0 con 0x7fbe84156d10 2023-11-06T03:12:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe84072750 con 0x7fbe84156d10 2023-11-06T03:12:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbe7c017940 con 0x7fbe84156d10 2023-11-06T03:12:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 0x7fbe68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe7c0836d0 con 0x7fbe84156d10 2023-11-06T03:12:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 0x7fbe68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.582+0000 7fbe88b93640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 0x7fbe68067f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe780aa000 tx=0x7fbe78000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.586+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbe7c050fc0 con 0x7fbe84156d10 2023-11-06T03:12:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.738+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe840afb00 con 0x7fbe84156d10 2023-11-06T03:12:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.738+0000 7fbe817fa640 1 -- 172.21.15.157:0/2793080345 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe7c0545e0 con 0x7fbe84156d10 2023-11-06T03:12:29.740 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:12:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 msgr2=0x7fbe68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 0x7fbe68067f20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe780aa000 tx=0x7fbe78000ff0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 msgr2=0x7fbe841513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406d430 tx=0x7fbe7c00f4a0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 shutdown_connections 2023-11-06T03:12:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe68065a60 0x7fbe68067f20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe84156d10 0x7fbe841513c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe84156890 0x7fbe84150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 --2- 172.21.15.157:0/2793080345 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe74003980 0x7fbe8414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 >> 172.21.15.157:0/2793080345 conn(0x7fbe84065930 msgr2=0x7fbe8406fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 shutdown_connections 2023-11-06T03:12:29.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.742+0000 7fbe8adf7640 1 -- 172.21.15.157:0/2793080345 wait complete. 2023-11-06T03:12:29.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:12:29.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.870+0000 7f2dacfb6640 1 Processor -- start 2023-11-06T03:12:29.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.870+0000 7f2dacfb6640 1 -- start start 2023-11-06T03:12:29.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2dacfb6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2da80b3b20 0x7f2da815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2dacfb6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da815f580 con 0x7f2da81551e0 2023-11-06T03:12:29.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2dacfb6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da815efc0 con 0x7f2da815e5e0 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2dacfb6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da80704b0 con 0x7f2da80b3b20 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2da81551e0 0x7f2da806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37660/0 (socket says 172.21.15.157:37660) 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 -- 172.21.15.157:0/3862702085 learned_addr learned my addr 172.21.15.157:0/3862702085 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da6d9d640 1 --2- 172.21.15.157:0/3862702085 >> v2:172.21.15.195:3300/0 conn(0x7f2da80b3b20 0x7f2da815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877658206 0 0) 0x7f2da815f580 con 0x7f2da81551e0 2023-11-06T03:12:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d90003530 con 0x7f2da81551e0 2023-11-06T03:12:29.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453420854 0 0) 0x7f2d90003530 con 0x7f2da81551e0 2023-11-06T03:12:29.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 >> v2:172.21.15.195:3300/0 conn(0x7f2da80b3b20 msgr2=0x7f2da815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 --2- 172.21.15.157:0/3862702085 >> v2:172.21.15.195:3300/0 conn(0x7f2da80b3b20 0x7f2da815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 >> v1:172.21.15.195:6789/0 conn(0x7f2da815e5e0 legacy=0x7f2da80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da815f2a0 con 0x7f2da81551e0 2023-11-06T03:12:29.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2d98001680 con 0x7f2da81551e0 2023-11-06T03:12:29.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 >> v1:172.21.15.157:6789/0 conn(0x7f2da81551e0 legacy=0x7f2da806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 0x7f2d90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d900047d0 0x7f2d90004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d900052b0 0x7f2d9000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80704b0 con 0x7f2d90003980 2023-11-06T03:12:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90010590 con 0x7f2d900052b0 2023-11-06T03:12:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90010750 con 0x7f2d900047d0 2023-11-06T03:12:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 0x7f2d90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da6d9d640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d900052b0 0x7f2d9000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da5d9b640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d900047d0 0x7f2d90004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 -- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d900047d0 msgr2=0x7f2d90004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d900047d0 0x7f2d90004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 -- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d900052b0 msgr2=0x7f2d9000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d900052b0 0x7f2d9000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da6d9d640 1 -- 172.21.15.157:0/3862702085 reap_dead start 2023-11-06T03:12:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.874+0000 7f2da659c640 1 -- 172.21.15.157:0/3862702085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90003530 con 0x7f2d90003980 2023-11-06T03:12:29.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2da659c640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 0x7f2d90003da0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c210 tx=0x7f2d9801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6522f39b9ec4ad6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98005ae0 con 0x7f2d90003980 2023-11-06T03:12:29.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98005c60 con 0x7f2d90003980 2023-11-06T03:12:29.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2da559a640 1 -- 172.21.15.157:0/3862702085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98007070 con 0x7f2d90003980 2023-11-06T03:12:29.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 msgr2=0x7f2d90003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 0x7f2d90003da0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c210 tx=0x7f2d9801cb70 comp rx=0 tx=0).stop 2023-11-06T03:12:29.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- 172.21.15.157:0/3862702085 shutdown_connections 2023-11-06T03:12:29.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/3862702085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d90003980 0x7f2d90003da0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- 172.21.15.157:0/3862702085 >> 172.21.15.157:0/3862702085 conn(0x7f2da8065930 msgr2=0x7f2da8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:29.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- 172.21.15.157:0/3862702085 shutdown_connections 2023-11-06T03:12:29.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- 172.21.15.157:0/3862702085 wait complete. 2023-11-06T03:12:29.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 Processor -- start 2023-11-06T03:12:29.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.878+0000 7f2dacfb6640 1 -- start start 2023-11-06T03:12:29.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 0x7f2da813b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da81e4290 0x7f2da814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da814d850 con 0x7f2da81e4710 2023-11-06T03:12:29.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da814db30 con 0x7f2d90003980 2023-11-06T03:12:29.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da814de10 con 0x7f2da81e4290 2023-11-06T03:12:29.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46174/0 (socket says 172.21.15.157:46174) 2023-11-06T03:12:29.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 -- 172.21.15.157:0/313042538 learned_addr learned my addr 172.21.15.157:0/313042538 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da5d9b640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da81e4290 0x7f2da814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da659c640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 0x7f2da813b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 -- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da81e4290 msgr2=0x7f2da814e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da81e4290 0x7f2da814e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 -- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 msgr2=0x7f2da813b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 0x7f2da813b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 -- 172.21.15.157:0/313042538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da814e0f0 con 0x7f2da81e4710 2023-11-06T03:12:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da659c640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 0x7f2da813b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da6d9d640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2da815e9c0 tx=0x7f2da000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da000e040 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da000f040 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0015430 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da814e3d0 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da813b7d0 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da814e5c0 con 0x7f2da81e4710 2023-11-06T03:12:29.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2da0015590 con 0x7f2da81e4710 2023-11-06T03:12:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 0x7f2d88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2da0083aa0 con 0x7f2da81e4710 2023-11-06T03:12:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.882+0000 7f2da659c640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 0x7f2d88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.886+0000 7f2da659c640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 0x7f2d88067f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2d980ab000 tx=0x7f2d980a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:29.886+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2da0051300 con 0x7f2da81e4710 2023-11-06T03:12:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.030+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2da8072750 con 0x7f2da81e4710 2023-11-06T03:12:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.030+0000 7f2d96ffd640 1 -- 172.21.15.157:0/313042538 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2da0054920 con 0x7f2da81e4710 2023-11-06T03:12:30.031 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:12:30.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 msgr2=0x7f2d88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 0x7f2d88067f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2d980ab000 tx=0x7f2d980a7070 comp rx=0 tx=0).stop 2023-11-06T03:12:30.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 msgr2=0x7f2da813bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2da815e9c0 tx=0x7f2da000f580 comp rx=0 tx=0).stop 2023-11-06T03:12:30.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 shutdown_connections 2023-11-06T03:12:30.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d88065a60 0x7f2d88067f20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2da81e4710 0x7f2da813bdf0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2da81e4290 0x7f2da814e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 --2- 172.21.15.157:0/313042538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d90003980 0x7f2da813b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 >> 172.21.15.157:0/313042538 conn(0x7f2da8065930 msgr2=0x7f2da80ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:30.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 shutdown_connections 2023-11-06T03:12:30.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.034+0000 7f2dacfb6640 1 -- 172.21.15.157:0/313042538 wait complete. 2023-11-06T03:12:30.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:12:30.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 Processor -- start 2023-11-06T03:12:30.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 -- start start 2023-11-06T03:12:30.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a7015e5e0 0x7f5a700b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5a7015ea90 0x7f5a700b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7015f580 con 0x7f5a701551e0 2023-11-06T03:12:30.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a700704b0 con 0x7f5a7015e5e0 2023-11-06T03:12:30.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a779bf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7015efc0 con 0x7f5a7015ea90 2023-11-06T03:12:30.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a7015e5e0 0x7f5a700b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a7015e5e0 0x7f5a700b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46180/0 (socket says 172.21.15.157:46180) 2023-11-06T03:12:30.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 -- 172.21.15.157:0/2837445981 learned_addr learned my addr 172.21.15.157:0/2837445981 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:30.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 -- 172.21.15.157:0/2837445981 >> v2:172.21.15.195:3300/0 conn(0x7f5a7015ea90 msgr2=0x7f5a700b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:30.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 --2- 172.21.15.157:0/2837445981 >> v2:172.21.15.195:3300/0 conn(0x7f5a7015ea90 0x7f5a700b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 -- 172.21.15.157:0/2837445981 >> v1:172.21.15.195:6789/0 conn(0x7f5a701551e0 legacy=0x7f5a7006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:30.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.174+0000 7f5a74f5a640 1 -- 172.21.15.157:0/2837445981 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7015f2a0 con 0x7f5a7015e5e0 2023-11-06T03:12:30.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a74f5a640 1 --2- 172.21.15.157:0/2837445981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7015e5e0 0x7f5a700b3410 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cc2f0 tx=0x7f5a6001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee735e3ff6b0a52 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a67fff640 1 -- 172.21.15.157:0/2837445981 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600ab600 con 0x7f5a7015e5e0 2023-11-06T03:12:30.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a67fff640 1 -- 172.21.15.157:0/2837445981 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a600ab760 con 0x7f5a7015e5e0 2023-11-06T03:12:30.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a67fff640 1 -- 172.21.15.157:0/2837445981 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600abaf0 con 0x7f5a7015e5e0 2023-11-06T03:12:30.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- 172.21.15.157:0/2837445981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7015e5e0 msgr2=0x7f5a700b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2837445981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7015e5e0 0x7f5a700b3410 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cc2f0 tx=0x7f5a6001c400 comp rx=0 tx=0).stop 2023-11-06T03:12:30.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- 172.21.15.157:0/2837445981 shutdown_connections 2023-11-06T03:12:30.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2837445981 >> v2:172.21.15.195:3300/0 conn(0x7f5a7015ea90 0x7f5a700b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2837445981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a7015e5e0 0x7f5a700b3410 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- 172.21.15.157:0/2837445981 >> 172.21.15.157:0/2837445981 conn(0x7f5a70065930 msgr2=0x7f5a70065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:30.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- 172.21.15.157:0/2837445981 shutdown_connections 2023-11-06T03:12:30.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- 172.21.15.157:0/2837445981 wait complete. 2023-11-06T03:12:30.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 Processor -- start 2023-11-06T03:12:30.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- start start 2023-11-06T03:12:30.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a7015e5e0 0x7f5a7014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a7015ea90 0x7f5a7014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a70158c60 con 0x7f5a701551e0 2023-11-06T03:12:30.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7014d6c0 con 0x7f5a7015e5e0 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a779bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7014c8e0 con 0x7f5a7015ea90 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46190/0 (socket says 172.21.15.157:46190) 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 -- 172.21.15.157:0/2423336326 learned_addr learned my addr 172.21.15.157:0/2423336326 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a74f5a640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a7015e5e0 0x7f5a7014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a75f5c640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a7015ea90 0x7f5a7014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 -- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a7015ea90 msgr2=0x7f5a7014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a7015ea90 0x7f5a7014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 -- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a7015e5e0 msgr2=0x7f5a7014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a7015e5e0 0x7f5a7014b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 -- 172.21.15.157:0/2423336326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600094d0 con 0x7f5a701551e0 2023-11-06T03:12:30.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.178+0000 7f5a7575b640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c490 tx=0x7f5a6c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c003600 con 0x7f5a701551e0 2023-11-06T03:12:30.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c003780 con 0x7f5a701551e0 2023-11-06T03:12:30.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7014cbc0 con 0x7f5a701551e0 2023-11-06T03:12:30.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c003b70 con 0x7f5a701551e0 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7014bb40 con 0x7f5a701551e0 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a70158ac0 con 0x7f5a701551e0 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a6c003cd0 con 0x7f5a701551e0 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 0x7f5a54067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a6c082950 con 0x7f5a701551e0 2023-11-06T03:12:30.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a74f5a640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 0x7f5a54067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.182+0000 7f5a74f5a640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 0x7f5a54067f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5a600af000 tx=0x7f5a600a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.186+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5a70158ac0 con 0x7f5a701551e0 2023-11-06T03:12:30.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.326+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a70158ac0 con 0x7f5a701551e0 2023-11-06T03:12:30.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a65ffb640 1 -- 172.21.15.157:0/2423336326 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a70158ac0 con 0x7f5a701551e0 2023-11-06T03:12:30.329 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:12:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 msgr2=0x7f5a54067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 0x7f5a54067f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5a600af000 tx=0x7f5a600a7000 comp rx=0 tx=0).stop 2023-11-06T03:12:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 msgr2=0x7f5a7014d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c490 tx=0x7f5a6c001e80 comp rx=0 tx=0).stop 2023-11-06T03:12:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 shutdown_connections 2023-11-06T03:12:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5a54065a60 0x7f5a54067f20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5a7015ea90 0x7f5a7014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5a7015e5e0 0x7f5a7014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 --2- 172.21.15.157:0/2423336326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a701551e0 0x7f5a7014d180 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 >> 172.21.15.157:0/2423336326 conn(0x7f5a70065930 msgr2=0x7f5a700b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 shutdown_connections 2023-11-06T03:12:30.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.330+0000 7f5a779bf640 1 -- 172.21.15.157:0/2423336326 wait complete. 2023-11-06T03:12:30.348 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:12:30.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 Processor -- start 2023-11-06T03:12:30.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 -- start start 2023-11-06T03:12:30.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe37015e5e0 0x7fe3700b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37015f580 con 0x7fe37015ea90 2023-11-06T03:12:30.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37015efc0 con 0x7fe3701551e0 2023-11-06T03:12:30.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe3753bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3700704b0 con 0x7fe37015e5e0 2023-11-06T03:12:30.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe37015e5e0 0x7fe3700b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe37015ea90 0x7fe37015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37670/0 (socket says 172.21.15.157:37670) 2023-11-06T03:12:30.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36f7fe640 1 -- 172.21.15.157:0/1333255521 learned_addr learned my addr 172.21.15.157:0/1333255521 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:30.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36e7fc640 1 -- 172.21.15.157:0/1333255521 >> v1:172.21.15.195:6789/0 conn(0x7fe3701551e0 legacy=0x7fe37006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36e7fc640 1 -- 172.21.15.157:0/1333255521 >> v1:172.21.15.157:6789/0 conn(0x7fe37015ea90 legacy=0x7fe37015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36e7fc640 1 -- 172.21.15.157:0/1333255521 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37015f2a0 con 0x7fe37015e5e0 2023-11-06T03:12:30.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.466+0000 7fe36dffb640 1 -- 172.21.15.157:0/1333255521 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509900657 0 0) 0x7fe37015f580 con 0x7fe37015ea90 2023-11-06T03:12:30.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe36e7fc640 1 --2- 172.21.15.157:0/1333255521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015e5e0 0x7fe3700b3410 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe3640097c0 tx=0x7fe36401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f31684bcccf6cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe36dffb640 1 -- 172.21.15.157:0/1333255521 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640ab600 con 0x7fe37015e5e0 2023-11-06T03:12:30.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe36dffb640 1 -- 172.21.15.157:0/1333255521 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3640a7050 con 0x7fe37015e5e0 2023-11-06T03:12:30.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe36dffb640 1 -- 172.21.15.157:0/1333255521 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640ab980 con 0x7fe37015e5e0 2023-11-06T03:12:30.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- 172.21.15.157:0/1333255521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015e5e0 msgr2=0x7fe3700b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 --2- 172.21.15.157:0/1333255521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015e5e0 0x7fe3700b3410 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe3640097c0 tx=0x7fe36401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- 172.21.15.157:0/1333255521 shutdown_connections 2023-11-06T03:12:30.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 --2- 172.21.15.157:0/1333255521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015e5e0 0x7fe3700b3410 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- 172.21.15.157:0/1333255521 >> 172.21.15.157:0/1333255521 conn(0x7fe370065930 msgr2=0x7fe370065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:30.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- 172.21.15.157:0/1333255521 shutdown_connections 2023-11-06T03:12:30.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- 172.21.15.157:0/1333255521 wait complete. 2023-11-06T03:12:30.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 Processor -- start 2023-11-06T03:12:30.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- start start 2023-11-06T03:12:30.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3701551e0 0x7fe3701467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe37015e5e0 0x7fe3700cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370152140 con 0x7fe37015ea90 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370152420 con 0x7fe3701551e0 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.470+0000 7fe3753bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe370152700 con 0x7fe37015e5e0 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46206/0 (socket says 172.21.15.157:46206) 2023-11-06T03:12:30.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 -- 172.21.15.157:0/2007977485 learned_addr learned my addr 172.21.15.157:0/2007977485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:30.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36effd640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3701551e0 0x7fe3701467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36e7fc640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe37015e5e0 0x7fe3700cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 -- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe37015e5e0 msgr2=0x7fe3700cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe37015e5e0 0x7fe3700cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 -- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3701551e0 msgr2=0x7fe3701467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3701551e0 0x7fe3701467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 -- 172.21.15.157:0/2007977485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364009420 con 0x7fe37015ea90 2023-11-06T03:12:30.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36f7fe640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c3c0 tx=0x7fe36000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600074e0 con 0x7fe37015ea90 2023-11-06T03:12:30.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360007640 con 0x7fe37015ea90 2023-11-06T03:12:30.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360004a30 con 0x7fe37015ea90 2023-11-06T03:12:30.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3701529e0 con 0x7fe37015ea90 2023-11-06T03:12:30.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe370146d10 con 0x7fe37015ea90 2023-11-06T03:12:30.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe370072750 con 0x7fe37015ea90 2023-11-06T03:12:30.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe360004b90 con 0x7fe37015ea90 2023-11-06T03:12:30.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 0x7fe35c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:30.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe360083060 con 0x7fe37015ea90 2023-11-06T03:12:30.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36effd640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 0x7fe35c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:30.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.474+0000 7fe36effd640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 0x7fe35c067f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe3680012f0 tx=0x7fe368009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:30.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.478+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe3600508c0 con 0x7fe37015ea90 2023-11-06T03:12:30.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe370152e90 con 0x7fe37015ea90 2023-11-06T03:12:30.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3577fe640 1 -- 172.21.15.157:0/2007977485 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe360053ee0 con 0x7fe37015ea90 2023-11-06T03:12:30.634 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:12:30.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 msgr2=0x7fe35c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 0x7fe35c067f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe3680012f0 tx=0x7fe368009040 comp rx=0 tx=0).stop 2023-11-06T03:12:30.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 msgr2=0x7fe3700ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:30.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c3c0 tx=0x7fe36000c880 comp rx=0 tx=0).stop 2023-11-06T03:12:30.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 shutdown_connections 2023-11-06T03:12:30.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe35c065a90 0x7fe35c067f50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe37015ea90 0x7fe3700ce0e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe37015e5e0 0x7fe3700cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 --2- 172.21.15.157:0/2007977485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3701551e0 0x7fe3701467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:30.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 >> 172.21.15.157:0/2007977485 conn(0x7fe370065930 msgr2=0x7fe3700ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:30.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 shutdown_connections 2023-11-06T03:12:30.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:30.634+0000 7fe3753bb640 1 -- 172.21.15.157:0/2007977485 wait complete. 2023-11-06T03:12:30.650 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:12:30.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:30.651 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:30.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[81268]: fuse finished with error 0 and tester_r 0 2023-11-06T03:12:30.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:12:30.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:30.733 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:12:30.913 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:12:30.913 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:30.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:12:30.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:12:30.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:30.921 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:12:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:12:30.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:12:30.970 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:12:30.970 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:30.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:12:30.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:12:30.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:30.977 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:12:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:12:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:12:31.026 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:12:31.027 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:31.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:12:31.033 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:12:31.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:31.034 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:12:31.085 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:12:31.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:12:31.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:31.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 Processor -- start 2023-11-06T03:12:31.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 -- start start 2023-11-06T03:12:31.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160950 0x7fe23c160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe23c15e800 0x7fe23c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c1612f0 con 0x7fe23c160490 2023-11-06T03:12:31.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c161780 con 0x7fe23c15e800 2023-11-06T03:12:31.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe243ff3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0cd860 con 0x7fe23c160950 2023-11-06T03:12:31.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe242590640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe23c15e800 0x7fe23c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe241d8f640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe23c160490 0x7fe23c0cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37686/0 (socket says 172.21.15.157:37686) 2023-11-06T03:12:31.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe241d8f640 1 -- 172.21.15.157:0/208477332 learned_addr learned my addr 172.21.15.157:0/208477332 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:31.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe24158e640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160950 0x7fe23c160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe240d8d640 1 -- 172.21.15.157:0/208477332 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780827234 0 0) 0x7fe23c1612f0 con 0x7fe23c160490 2023-11-06T03:12:31.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe240d8d640 1 -- 172.21.15.157:0/208477332 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003530 con 0x7fe23c160490 2023-11-06T03:12:31.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe242590640 1 -- 172.21.15.157:0/208477332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160950 msgr2=0x7fe23c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe242590640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160950 0x7fe23c160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe242590640 1 -- 172.21.15.157:0/208477332 >> v1:172.21.15.157:6789/0 conn(0x7fe23c160490 legacy=0x7fe23c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.214+0000 7fe242590640 1 -- 172.21.15.157:0/208477332 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0cdb40 con 0x7fe23c15e800 2023-11-06T03:12:31.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe242590640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c15ec90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe234009d90 tx=0x7fe23401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24a0260c89fcba5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe240d8d640 1 -- 172.21.15.157:0/208477332 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340a59e0 con 0x7fe23c15e800 2023-11-06T03:12:31.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe240d8d640 1 -- 172.21.15.157:0/208477332 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2340a5b40 con 0x7fe23c15e800 2023-11-06T03:12:31.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe240d8d640 1 -- 172.21.15.157:0/208477332 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340ad440 con 0x7fe23c15e800 2023-11-06T03:12:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- 172.21.15.157:0/208477332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 msgr2=0x7fe23c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c15ec90 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe234009d90 tx=0x7fe23401c920 comp rx=0 tx=0).stop 2023-11-06T03:12:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- 172.21.15.157:0/208477332 shutdown_connections 2023-11-06T03:12:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c15ec90 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- 172.21.15.157:0/208477332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160950 0x7fe23c160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- 172.21.15.157:0/208477332 >> 172.21.15.157:0/208477332 conn(0x7fe23c070790 msgr2=0x7fe23c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:31.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- 172.21.15.157:0/208477332 shutdown_connections 2023-11-06T03:12:31.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- 172.21.15.157:0/208477332 wait complete. 2023-11-06T03:12:31.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 Processor -- start 2023-11-06T03:12:31.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- start start 2023-11-06T03:12:31.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160490 0x7fe23c148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c160950 0x7fe23c149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c149650 con 0x7fe23c15e800 2023-11-06T03:12:31.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c149800 con 0x7fe23c160950 2023-11-06T03:12:31.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe243ff3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b7350 con 0x7fe23c160490 2023-11-06T03:12:31.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46224/0 (socket says 172.21.15.157:46224) 2023-11-06T03:12:31.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 -- 172.21.15.157:0/2324842141 learned_addr learned my addr 172.21.15.157:0/2324842141 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:31.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe24158e640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160490 0x7fe23c148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 -- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160490 msgr2=0x7fe23c148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160490 0x7fe23c148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 -- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c160950 msgr2=0x7fe23c149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe242590640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c160950 0x7fe23c149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c160950 0x7fe23c149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.218+0000 7fe241d8f640 1 -- 172.21.15.157:0/2324842141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009a70 con 0x7fe23c15e800 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe241d8f640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c3c0 tx=0x7fe22c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c005cf0 con 0x7fe23c15e800 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c005e50 con 0x7fe23c15e800 2023-11-06T03:12:31.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c0b7630 con 0x7fe23c15e800 2023-11-06T03:12:31.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0072f0 con 0x7fe23c15e800 2023-11-06T03:12:31.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c0b7bf0 con 0x7fe23c15e800 2023-11-06T03:12:31.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c149990 con 0x7fe23c15e800 2023-11-06T03:12:31.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe22c007450 con 0x7fe23c15e800 2023-11-06T03:12:31.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 0x7fe220067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe22c0828e0 con 0x7fe23c15e800 2023-11-06T03:12:31.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe24158e640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 0x7fe220067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.222+0000 7fe24158e640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 0x7fe220067f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe23c14ba90 tx=0x7fe238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.226+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe22c050140 con 0x7fe23c15e800 2023-11-06T03:12:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.414+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23c0ce3b0 con 0x7fe23c15e800 2023-11-06T03:12:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.414+0000 7fe2327fc640 1 -- 172.21.15.157:0/2324842141 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fe22c017070 con 0x7fe23c15e800 2023-11-06T03:12:31.415 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:31.415 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:31.415 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.414+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 msgr2=0x7fe220067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.414+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 0x7fe220067f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe23c14ba90 tx=0x7fe238008040 comp rx=0 tx=0).stop 2023-11-06T03:12:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 msgr2=0x7fe23c14e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c3c0 tx=0x7fe22c00c880 comp rx=0 tx=0).stop 2023-11-06T03:12:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 shutdown_connections 2023-11-06T03:12:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe220065a60 0x7fe220067f20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe23c160950 0x7fe23c149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe23c160490 0x7fe23c148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 --2- 172.21.15.157:0/2324842141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15e800 0x7fe23c14e6c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 >> 172.21.15.157:0/2324842141 conn(0x7fe23c070790 msgr2=0x7fe23c067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:31.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 shutdown_connections 2023-11-06T03:12:31.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.418+0000 7fe243ff3640 1 -- 172.21.15.157:0/2324842141 wait complete. 2023-11-06T03:12:31.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:31.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 Processor -- start 2023-11-06T03:12:31.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 -- start start 2023-11-06T03:12:31.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbf580b2af0 0x7fbf580ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5815e440 con 0x7fbf5806d000 2023-11-06T03:12:31.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5806c240 con 0x7fbf581599c0 2023-11-06T03:12:31.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf5dacb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5806c520 con 0x7fbf580b2af0 2023-11-06T03:12:31.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf57fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbf5806d000 0x7fbf58159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37690/0 (socket says 172.21.15.157:37690) 2023-11-06T03:12:31.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf57fff640 1 -- 172.21.15.157:0/60745496 learned_addr learned my addr 172.21.15.157:0/60745496 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:31.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf56ffd640 1 --2- 172.21.15.157:0/60745496 >> v2:172.21.15.195:3300/0 conn(0x7fbf580b2af0 0x7fbf580ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46501423 0 0) 0x7fbf5815e440 con 0x7fbf5806d000 2023-11-06T03:12:31.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf40003530 con 0x7fbf5806d000 2023-11-06T03:12:31.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615073630 0 0) 0x7fbf40003530 con 0x7fbf5806d000 2023-11-06T03:12:31.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 >> v2:172.21.15.195:3300/0 conn(0x7fbf580b2af0 msgr2=0x7fbf580ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 --2- 172.21.15.157:0/60745496 >> v2:172.21.15.195:3300/0 conn(0x7fbf580b2af0 0x7fbf580ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 >> v1:172.21.15.195:6789/0 conn(0x7fbf581599c0 legacy=0x7fbf580b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5806c800 con 0x7fbf5806d000 2023-11-06T03:12:31.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fbf50001a00 con 0x7fbf5806d000 2023-11-06T03:12:31.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 >> v1:172.21.15.157:6789/0 conn(0x7fbf5806d000 legacy=0x7fbf58159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 0x7fbf40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf400047d0 0x7fbf40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf40005190 0x7fbf4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5806c520 con 0x7fbf40003980 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000e240 con 0x7fbf40005190 2023-11-06T03:12:31.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000e400 con 0x7fbf400047d0 2023-11-06T03:12:31.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 0x7fbf40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf57fff640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf40005190 0x7fbf4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 -- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf400047d0 msgr2=0x7fbf40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf400047d0 0x7fbf40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 -- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf40005190 msgr2=0x7fbf4000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf40005190 0x7fbf4000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf577fe640 1 -- 172.21.15.157:0/60745496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40003530 con 0x7fbf40003980 2023-11-06T03:12:31.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf57fff640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf40005190 0x7fbf4000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:31.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.558+0000 7fbf57fff640 1 -- 172.21.15.157:0/60745496 reap_dead start 2023-11-06T03:12:31.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf577fe640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 0x7fbf40003da0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fbf48001780 tx=0x7fbf4801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76ea98d26c43b58 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480050d0 con 0x7fbf40003980 2023-11-06T03:12:31.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480a7050 con 0x7fbf40003980 2023-11-06T03:12:31.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf567fc640 1 -- 172.21.15.157:0/60745496 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48005490 con 0x7fbf40003980 2023-11-06T03:12:31.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 msgr2=0x7fbf40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 0x7fbf40003da0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fbf48001780 tx=0x7fbf4801c3b0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- 172.21.15.157:0/60745496 shutdown_connections 2023-11-06T03:12:31.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/60745496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf40003980 0x7fbf40003da0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- 172.21.15.157:0/60745496 >> 172.21.15.157:0/60745496 conn(0x7fbf58065930 msgr2=0x7fbf58065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:31.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- 172.21.15.157:0/60745496 shutdown_connections 2023-11-06T03:12:31.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- 172.21.15.157:0/60745496 wait complete. 2023-11-06T03:12:31.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 Processor -- start 2023-11-06T03:12:31.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- start start 2023-11-06T03:12:31.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 0x7fbf58152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf580ac8b0 0x7fbf580acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580ad270 con 0x7fbf58154f40 2023-11-06T03:12:31.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58153f90 con 0x7fbf580ac8b0 2023-11-06T03:12:31.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.562+0000 7fbf5dacb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58153380 con 0x7fbf40003980 2023-11-06T03:12:31.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46244/0 (socket says 172.21.15.157:46244) 2023-11-06T03:12:31.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 -- 172.21.15.157:0/545692603 learned_addr learned my addr 172.21.15.157:0/545692603 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:31.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf577fe640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 0x7fbf58152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf57fff640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf580ac8b0 0x7fbf580acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 -- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 msgr2=0x7fbf58152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 0x7fbf58152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 -- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf580ac8b0 msgr2=0x7fbf580acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf580ac8b0 0x7fbf580acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 -- 172.21.15.157:0/545692603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58153660 con 0x7fbf58154f40 2023-11-06T03:12:31.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf577fe640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 0x7fbf58152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:31.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf56ffd640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00e870 tx=0x7fbf4c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c009620 con 0x7fbf58154f40 2023-11-06T03:12:31.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00c940 con 0x7fbf58154f40 2023-11-06T03:12:31.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00ccf0 con 0x7fbf58154f40 2023-11-06T03:12:31.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf58153940 con 0x7fbf58154f40 2023-11-06T03:12:31.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf58153040 con 0x7fbf58154f40 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58154140 con 0x7fbf58154f40 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbf4c013470 con 0x7fbf58154f40 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 0x7fbf44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf4c083370 con 0x7fbf58154f40 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.566+0000 7fbf577fe640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 0x7fbf44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.570+0000 7fbf577fe640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 0x7fbf44067f20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf48001750 tx=0x7fbf480011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.570+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbf58154140 con 0x7fbf58154f40 2023-11-06T03:12:31.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.726+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf58154140 con 0x7fbf58154f40 2023-11-06T03:12:31.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf3ffff640 1 -- 172.21.15.157:0/545692603 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fbf4c054660 con 0x7fbf58154f40 2023-11-06T03:12:31.729 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:31.729 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:59.421679+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:31.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 msgr2=0x7fbf44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 0x7fbf44067f20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf48001750 tx=0x7fbf480011a0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 msgr2=0x7fbf581483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00e870 tx=0x7fbf4c00ed30 comp rx=0 tx=0).stop 2023-11-06T03:12:31.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 shutdown_connections 2023-11-06T03:12:31.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf44065a60 0x7fbf44067f20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf580ac8b0 0x7fbf580acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf58154f40 0x7fbf581483d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 --2- 172.21.15.157:0/545692603 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf40003980 0x7fbf58152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 >> 172.21.15.157:0/545692603 conn(0x7fbf58065930 msgr2=0x7fbf580b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:31.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 shutdown_connections 2023-11-06T03:12:31.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.730+0000 7fbf5dacb640 1 -- 172.21.15.157:0/545692603 wait complete. 2023-11-06T03:12:31.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:11:56.975891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:11:57.391711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:12:31.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:12:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 Processor -- start 2023-11-06T03:12:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 -- start start 2023-11-06T03:12:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 0x7fbc6c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c0cf1a0 con 0x7fbc6c146370 2023-11-06T03:12:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c06c3d0 con 0x7fbc6c06b6a0 2023-11-06T03:12:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc70ab3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b2120 con 0x7fbc6c0ce800 2023-11-06T03:12:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbc6c146370 0x7fbc6c0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37704/0 (socket says 172.21.15.157:37704) 2023-11-06T03:12:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6a59c640 1 -- 172.21.15.157:0/608601980 learned_addr learned my addr 172.21.15.157:0/608601980 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 0x7fbc6c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726231117 0 0) 0x7fbc6c0cf1a0 con 0x7fbc6c146370 2023-11-06T03:12:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc54003530 con 0x7fbc6c146370 2023-11-06T03:12:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063538450 0 0) 0x7fbc6c06c3d0 con 0x7fbc6c06b6a0 2023-11-06T03:12:31.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc6c0cf1a0 con 0x7fbc6c06b6a0 2023-11-06T03:12:31.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331903838 0 0) 0x7fbc54003530 con 0x7fbc6c146370 2023-11-06T03:12:31.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 msgr2=0x7fbc6c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 0x7fbc6c0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 >> v1:172.21.15.195:6789/0 conn(0x7fbc6c06b6a0 legacy=0x7fbc6c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c06f5a0 con 0x7fbc6c146370 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fbc5c001680 con 0x7fbc6c146370 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 >> v1:172.21.15.157:6789/0 conn(0x7fbc6c146370 legacy=0x7fbc6c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 0x7fbc54003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc540040f0 0x7fbc540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc54004d10 0x7fbc54005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b2120 con 0x7fbc540040f0 2023-11-06T03:12:31.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5400bb50 con 0x7fbc54004d10 2023-11-06T03:12:31.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5400bd10 con 0x7fbc6c0ce800 2023-11-06T03:12:31.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc540040f0 0x7fbc540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 -- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 msgr2=0x7fbc54003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc6c0ce800 0x7fbc54003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 -- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc54004d10 msgr2=0x7fbc54005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:31.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc54004d10 0x7fbc54005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.870+0000 7fbc69d9b640 1 -- 172.21.15.157:0/608601980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc54003530 con 0x7fbc540040f0 2023-11-06T03:12:31.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc540040f0 0x7fbc540047d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009ae0 tx=0x7fbc6001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934d5c9b2ac33d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600a5570 con 0x7fbc540040f0 2023-11-06T03:12:31.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc600a56d0 con 0x7fbc540040f0 2023-11-06T03:12:31.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc6959a640 1 -- 172.21.15.157:0/608601980 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600a5a80 con 0x7fbc540040f0 2023-11-06T03:12:31.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- 172.21.15.157:0/608601980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc540040f0 msgr2=0x7fbc540047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/608601980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc540040f0 0x7fbc540047d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009ae0 tx=0x7fbc6001ca90 comp rx=0 tx=0).stop 2023-11-06T03:12:31.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc6ad9d640 1 -- 172.21.15.157:0/608601980 reap_dead start 2023-11-06T03:12:31.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- 172.21.15.157:0/608601980 shutdown_connections 2023-11-06T03:12:31.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- 172.21.15.157:0/608601980 >> 172.21.15.157:0/608601980 conn(0x7fbc6c065930 msgr2=0x7fbc6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:31.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- 172.21.15.157:0/608601980 shutdown_connections 2023-11-06T03:12:31.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- 172.21.15.157:0/608601980 wait complete. 2023-11-06T03:12:31.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 Processor -- start 2023-11-06T03:12:31.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- start start 2023-11-06T03:12:31.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc540040f0 0x7fbc6c1303f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc6c15a550 0x7fbc6c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c145c50 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0ad270 con 0x7fbc6c15a550 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc70ab3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c130930 con 0x7fbc540040f0 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.874+0000 7fbc6ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46264/0 (socket says 172.21.15.157:46264) 2023-11-06T03:12:31.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 -- 172.21.15.157:0/2976384996 learned_addr learned my addr 172.21.15.157:0/2976384996 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:31.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc69d9b640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc6c15a550 0x7fbc6c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6a59c640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc540040f0 0x7fbc6c1303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 -- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc540040f0 msgr2=0x7fbc6c1303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc540040f0 0x7fbc6c1303f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 -- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc6c15a550 msgr2=0x7fbc6c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:31.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc6c15a550 0x7fbc6c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:31.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 -- 172.21.15.157:0/2976384996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c145970 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6ad9d640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0cf9d0 tx=0x7fbc6400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64005d50 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6400a5e0 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c163220 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400a990 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c145640 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c0b0100 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc6400aaf0 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 0x7fbc4c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc64082990 con 0x7fbc6c0acdf0 2023-11-06T03:12:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6a59c640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 0x7fbc4c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:31.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.878+0000 7fbc6a59c640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 0x7fbc4c067f50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001000 tx=0x7fbc5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:31.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:31.882+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbc6c0b0100 con 0x7fbc6c0acdf0 2023-11-06T03:12:32.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.030+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc6c0b0100 con 0x7fbc6c0acdf0 2023-11-06T03:12:32.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.030+0000 7fbc5affd640 1 -- 172.21.15.157:0/2976384996 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc6c0b0100 con 0x7fbc6c0acdf0 2023-11-06T03:12:32.031 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:12:32.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 msgr2=0x7fbc4c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 0x7fbc4c067f50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001000 tx=0x7fbc5c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:32.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 msgr2=0x7fbc6c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0cf9d0 tx=0x7fbc6400cb90 comp rx=0 tx=0).stop 2023-11-06T03:12:32.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 shutdown_connections 2023-11-06T03:12:32.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc4c065a90 0x7fbc4c067f50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc6c0acdf0 0x7fbc6c15c070 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc6c15a550 0x7fbc6c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 --2- 172.21.15.157:0/2976384996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc540040f0 0x7fbc6c1303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 >> 172.21.15.157:0/2976384996 conn(0x7fbc6c065930 msgr2=0x7fbc6c0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 shutdown_connections 2023-11-06T03:12:32.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.034+0000 7fbc70ab3640 1 -- 172.21.15.157:0/2976384996 wait complete. 2023-11-06T03:12:32.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:12:32.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 Processor -- start 2023-11-06T03:12:32.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 -- start start 2023-11-06T03:12:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2ce4161070 0x7f2ce4071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce406d530 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce40ccaf0 con 0x7f2ce4072d60 2023-11-06T03:12:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2cea1b5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce40ccdd0 con 0x7f2ce4161070 2023-11-06T03:12:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2ce40ce9f0 0x7f2ce40c1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37714/0 (socket says 172.21.15.157:37714) 2023-11-06T03:12:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce3fff640 1 -- 172.21.15.157:0/4015671697 learned_addr learned my addr 172.21.15.157:0/4015671697 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069623793 0 0) 0x7f2cd4001120 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/4015671697 >> v2:172.21.15.195:3300/0 conn(0x7f2ce4161070 0x7f2ce4071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd0003530 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086846280 0 0) 0x7f2cd0003530 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 >> v2:172.21.15.195:3300/0 conn(0x7f2ce4161070 msgr2=0x7f2ce4071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 --2- 172.21.15.157:0/4015671697 >> v2:172.21.15.195:3300/0 conn(0x7f2ce4161070 0x7f2ce4071650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 >> v1:172.21.15.195:6789/0 conn(0x7f2ce4072d60 legacy=0x7f2ce40ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40cd0b0 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2cd4001810 con 0x7f2ce40ce9f0 2023-11-06T03:12:32.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/4015671697 >> v2:172.21.15.195:3300/0 conn(0x7f2ce4161070 0x7f2ce4071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:32.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 >> v1:172.21.15.157:6789/0 conn(0x7f2ce40ce9f0 legacy=0x7f2ce40c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cd0003980 0x7f2cd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 0x7f2cd0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd0005190 0x7f2cd000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40ccdd0 con 0x7f2cd00047d0 2023-11-06T03:12:32.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd000c0e0 con 0x7f2cd0003980 2023-11-06T03:12:32.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd000c2a0 con 0x7f2cd0005190 2023-11-06T03:12:32.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 0x7f2cd0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce3fff640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd0005190 0x7f2cd000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cd0003980 0x7f2cd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd0005190 msgr2=0x7f2cd000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd0005190 0x7f2cd000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cd0003980 msgr2=0x7f2cd0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cd0003980 0x7f2cd0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce3fff640 1 -- 172.21.15.157:0/4015671697 reap_dead start 2023-11-06T03:12:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/4015671697 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0003530 con 0x7f2cd00047d0 2023-11-06T03:12:32.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.162+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cd0003980 0x7f2cd0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:32.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 0x7f2cd0004c50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc001750 tx=0x7f2ccc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed64de3a3ce3b068 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0052d0 con 0x7f2cd00047d0 2023-11-06T03:12:32.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0a7050 con 0x7f2cd00047d0 2023-11-06T03:12:32.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2ce27fc640 1 -- 172.21.15.157:0/4015671697 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc003600 con 0x7f2cd00047d0 2023-11-06T03:12:32.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 msgr2=0x7f2cd0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 0x7f2cd0004c50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc001750 tx=0x7f2ccc01c3a0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- 172.21.15.157:0/4015671697 shutdown_connections 2023-11-06T03:12:32.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/4015671697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cd00047d0 0x7f2cd0004c50 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- 172.21.15.157:0/4015671697 >> 172.21.15.157:0/4015671697 conn(0x7f2ce40cb990 msgr2=0x7f2ce40cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- 172.21.15.157:0/4015671697 shutdown_connections 2023-11-06T03:12:32.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- 172.21.15.157:0/4015671697 wait complete. 2023-11-06T03:12:32.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 Processor -- start 2023-11-06T03:12:32.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.166+0000 7f2cea1b5640 1 -- start start 2023-11-06T03:12:32.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd00047d0 0x7f2ce40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce414e260 0x7f2ce4163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40b23e0 con 0x7f2ce41621f0 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40b1b10 con 0x7f2ce414e260 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40c7280 con 0x7f2cd00047d0 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46278/0 (socket says 172.21.15.157:46278) 2023-11-06T03:12:32.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/326253429 learned_addr learned my addr 172.21.15.157:0/326253429 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:32.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd00047d0 0x7f2ce40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd00047d0 msgr2=0x7f2ce40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd00047d0 0x7f2ce40b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce3fff640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce414e260 0x7f2ce4163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce414e260 msgr2=0x7f2ce4163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce414e260 0x7f2ce4163320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 -- 172.21.15.157:0/326253429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce41629d0 con 0x7f2ce41621f0 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce2ffd640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc01c1d0 tx=0x7f2cd0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007440 con 0x7f2ce41621f0 2023-11-06T03:12:32.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0075c0 con 0x7f2ce41621f0 2023-11-06T03:12:32.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce40b1d20 con 0x7f2ce41621f0 2023-11-06T03:12:32.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc006a20 con 0x7f2ce41621f0 2023-11-06T03:12:32.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce4162670 con 0x7f2ce41621f0 2023-11-06T03:12:32.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce40b2520 con 0x7f2ce41621f0 2023-11-06T03:12:32.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ccc006b80 con 0x7f2ce41621f0 2023-11-06T03:12:32.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 0x7f2cc4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ccc11d420 con 0x7f2ce41621f0 2023-11-06T03:12:32.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 0x7f2cc4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.170+0000 7f2ce37fe640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 0x7f2cc4067f50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4063940 tx=0x7f2cdc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.174+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ce40b2520 con 0x7f2ce41621f0 2023-11-06T03:12:32.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2ce40b2520 con 0x7f2ce41621f0 2023-11-06T03:12:32.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cdbfff640 1 -- 172.21.15.157:0/326253429 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ce40b2520 con 0x7f2ce41621f0 2023-11-06T03:12:32.317 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:12:32.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 msgr2=0x7f2cc4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 0x7f2cc4067f50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4063940 tx=0x7f2cdc00b040 comp rx=0 tx=0).stop 2023-11-06T03:12:32.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 msgr2=0x7f2ce40c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc01c1d0 tx=0x7f2cd0005800 comp rx=0 tx=0).stop 2023-11-06T03:12:32.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 shutdown_connections 2023-11-06T03:12:32.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cc4065a90 0x7f2cc4067f50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ce414e260 0x7f2ce4163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce41621f0 0x7f2ce40c6d40 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 --2- 172.21.15.157:0/326253429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cd00047d0 0x7f2ce40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 >> 172.21.15.157:0/326253429 conn(0x7f2ce40cb990 msgr2=0x7f2ce40af570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 shutdown_connections 2023-11-06T03:12:32.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.318+0000 7f2cea1b5640 1 -- 172.21.15.157:0/326253429 wait complete. 2023-11-06T03:12:32.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:12:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.458+0000 7f8ee3f1c640 1 Processor -- start 2023-11-06T03:12:32.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.458+0000 7f8ee3f1c640 1 -- start start 2023-11-06T03:12:32.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee3f1c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8edc06d000 0x7f8edc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee3f1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc15e440 con 0x7f8edc1599c0 2023-11-06T03:12:32.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee3f1c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc06c240 con 0x7f8edc0b2af0 2023-11-06T03:12:32.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee3f1c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc06c520 con 0x7f8edc06d000 2023-11-06T03:12:32.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8edc1599c0 0x7f8edc0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37724/0 (socket says 172.21.15.157:37724) 2023-11-06T03:12:32.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 -- 172.21.15.157:0/1761944217 learned_addr learned my addr 172.21.15.157:0/1761944217 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:32.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee24b9640 1 --2- 172.21.15.157:0/1761944217 >> v2:172.21.15.195:3300/0 conn(0x7f8edc06d000 0x7f8edc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218539584 0 0) 0x7f8edc15e440 con 0x7f8edc1599c0 2023-11-06T03:12:32.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec8003530 con 0x7f8edc1599c0 2023-11-06T03:12:32.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621253469 0 0) 0x7f8ec8003530 con 0x7f8edc1599c0 2023-11-06T03:12:32.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 >> v2:172.21.15.195:3300/0 conn(0x7f8edc06d000 msgr2=0x7f8edc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 --2- 172.21.15.157:0/1761944217 >> v2:172.21.15.195:3300/0 conn(0x7f8edc06d000 0x7f8edc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 >> v1:172.21.15.195:6789/0 conn(0x7f8edc0b2af0 legacy=0x7f8edc159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc06c800 con 0x7f8edc1599c0 2023-11-06T03:12:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8ed8001680 con 0x7f8edc1599c0 2023-11-06T03:12:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 >> v1:172.21.15.157:6789/0 conn(0x7f8edc1599c0 legacy=0x7f8edc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 0x7f8ec8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec80047d0 0x7f8ec8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8005190 0x7f8ec800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc06c520 con 0x7f8ec8003980 2023-11-06T03:12:32.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800c0e0 con 0x7f8ec8005190 2023-11-06T03:12:32.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800c2a0 con 0x7f8ec80047d0 2023-11-06T03:12:32.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 0x7f8ec8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 -- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec80047d0 msgr2=0x7f8ec8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec80047d0 0x7f8ec8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 -- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8005190 msgr2=0x7f8ec800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8005190 0x7f8ec800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee24b9640 1 -- 172.21.15.157:0/1761944217 reap_dead start 2023-11-06T03:12:32.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.462+0000 7f8ee1cb8640 1 -- 172.21.15.157:0/1761944217 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8003530 con 0x7f8ec8003980 2023-11-06T03:12:32.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 0x7f8ec8003da0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c240 tx=0x7f8ed801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42394f8c78472ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8005ae0 con 0x7f8ec8003980 2023-11-06T03:12:32.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8005c40 con 0x7f8ec8003980 2023-11-06T03:12:32.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee0cb6640 1 -- 172.21.15.157:0/1761944217 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80a5a10 con 0x7f8ec8003980 2023-11-06T03:12:32.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 msgr2=0x7f8ec8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 0x7f8ec8003da0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c240 tx=0x7f8ed801cc10 comp rx=0 tx=0).stop 2023-11-06T03:12:32.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/1761944217 shutdown_connections 2023-11-06T03:12:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/1761944217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec8003980 0x7f8ec8003da0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/1761944217 >> 172.21.15.157:0/1761944217 conn(0x7f8edc065930 msgr2=0x7f8edc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/1761944217 shutdown_connections 2023-11-06T03:12:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/1761944217 wait complete. 2023-11-06T03:12:32.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 Processor -- start 2023-11-06T03:12:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- start start 2023-11-06T03:12:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8003980 0x7f8edc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8edc1f3870 0x7f8edc1f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0aa380 con 0x7f8edc0ab420 2023-11-06T03:12:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0aa660 con 0x7f8ec8003980 2023-11-06T03:12:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee3f1c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0aa940 con 0x7f8edc1f3870 2023-11-06T03:12:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46296/0 (socket says 172.21.15.157:46296) 2023-11-06T03:12:32.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 -- 172.21.15.157:0/3240863298 learned_addr learned my addr 172.21.15.157:0/3240863298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 -- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8edc1f3870 msgr2=0x7f8edc1f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee14b7640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8edc1f3870 0x7f8edc1f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8003980 0x7f8edc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8edc1f3870 0x7f8edc1f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 -- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8003980 msgr2=0x7f8edc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8003980 0x7f8edc154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.466+0000 7f8ee24b9640 1 -- 172.21.15.157:0/3240863298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0aac20 con 0x7f8edc0ab420 2023-11-06T03:12:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee24b9640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0b19a0 tx=0x7f8ed4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400ec80 con 0x7f8edc0ab420 2023-11-06T03:12:32.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400ee00 con 0x7f8edc0ab420 2023-11-06T03:12:32.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc0aaf00 con 0x7f8edc0ab420 2023-11-06T03:12:32.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4014660 con 0x7f8edc0ab420 2023-11-06T03:12:32.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc155180 con 0x7f8edc0ab420 2023-11-06T03:12:32.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc0ab140 con 0x7f8edc0ab420 2023-11-06T03:12:32.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ed40147c0 con 0x7f8edc0ab420 2023-11-06T03:12:32.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 0x7f8ec0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ed4083660 con 0x7f8edc0ab420 2023-11-06T03:12:32.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 0x7f8ec0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.470+0000 7f8ee1cb8640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 0x7f8ec0067f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80ab000 tx=0x7f8ed80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.474+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8edc0ab140 con 0x7f8edc0ab420 2023-11-06T03:12:32.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8edc0ab140 con 0x7f8edc0ab420 2023-11-06T03:12:32.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ed27fc640 1 -- 172.21.15.157:0/3240863298 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8edc0ab140 con 0x7f8edc0ab420 2023-11-06T03:12:32.630 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:12:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 msgr2=0x7f8ec0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 0x7f8ec0067f20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80ab000 tx=0x7f8ed80a7070 comp rx=0 tx=0).stop 2023-11-06T03:12:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 msgr2=0x7f8edc0ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0b19a0 tx=0x7f8ed4007f40 comp rx=0 tx=0).stop 2023-11-06T03:12:32.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 shutdown_connections 2023-11-06T03:12:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ec0065a60 0x7f8ec0067f20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0ab420 0x7f8edc0ab8a0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8edc1f3870 0x7f8edc1f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 --2- 172.21.15.157:0/3240863298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec8003980 0x7f8edc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 >> 172.21.15.157:0/3240863298 conn(0x7f8edc065930 msgr2=0x7f8edc070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 shutdown_connections 2023-11-06T03:12:32.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.630+0000 7f8ee3f1c640 1 -- 172.21.15.157:0/3240863298 wait complete. 2023-11-06T03:12:32.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:12:32.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 Processor -- start 2023-11-06T03:12:32.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 -- start start 2023-11-06T03:12:32.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7c4160490 0x7fd7c40cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c415f810 0x7fd7c4160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c41612f0 con 0x7fd7c4160950 2023-11-06T03:12:32.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c4161780 con 0x7fd7c4160490 2023-11-06T03:12:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c8a70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40d0450 con 0x7fd7c415f810 2023-11-06T03:12:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7c4160490 0x7fd7c40cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd7c4160490 0x7fd7c40cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46298/0 (socket says 172.21.15.157:46298) 2023-11-06T03:12:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 -- 172.21.15.157:0/845206827 learned_addr learned my addr 172.21.15.157:0/845206827 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:32.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 -- 172.21.15.157:0/845206827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c415f810 msgr2=0x7fd7c4160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 --2- 172.21.15.157:0/845206827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c415f810 0x7fd7c4160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 -- 172.21.15.157:0/845206827 >> v1:172.21.15.195:6789/0 conn(0x7fd7c4160950 legacy=0x7fd7c415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:32.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.762+0000 7fd7c259c640 1 -- 172.21.15.157:0/845206827 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40cd860 con 0x7fd7c4160490 2023-11-06T03:12:32.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c259c640 1 --2- 172.21.15.157:0/845206827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c4160490 0x7fd7c40cff10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009870 tx=0x7fd7b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100a4bc967c256da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c159a640 1 -- 172.21.15.157:0/845206827 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c415e280 con 0x7fd7c4160490 2023-11-06T03:12:32.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c159a640 1 -- 172.21.15.157:0/845206827 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b40aa5f0 con 0x7fd7c4160490 2023-11-06T03:12:32.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c159a640 1 -- 172.21.15.157:0/845206827 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40aa980 con 0x7fd7c4160490 2023-11-06T03:12:32.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- 172.21.15.157:0/845206827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c4160490 msgr2=0x7fd7c40cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/845206827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c4160490 0x7fd7c40cff10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009870 tx=0x7fd7b401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- 172.21.15.157:0/845206827 shutdown_connections 2023-11-06T03:12:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/845206827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c415f810 0x7fd7c4160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/845206827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c4160490 0x7fd7c40cff10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- 172.21.15.157:0/845206827 >> 172.21.15.157:0/845206827 conn(0x7fd7c4070790 msgr2=0x7fd7c4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- 172.21.15.157:0/845206827 shutdown_connections 2023-11-06T03:12:32.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- 172.21.15.157:0/845206827 wait complete. 2023-11-06T03:12:32.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 Processor -- start 2023-11-06T03:12:32.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- start start 2023-11-06T03:12:32.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c4160490 0x7fd7c4065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c4160950 0x7fd7c414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415c870 con 0x7fd7c415f810 2023-11-06T03:12:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c414eca0 con 0x7fd7c4160950 2023-11-06T03:12:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c8a70640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415ca80 con 0x7fd7c4160490 2023-11-06T03:12:32.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46300/0 (socket says 172.21.15.157:46300) 2023-11-06T03:12:32.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.766+0000 7fd7c259c640 1 -- 172.21.15.157:0/1426774246 learned_addr learned my addr 172.21.15.157:0/1426774246 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:32.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c1d9b640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c4160490 0x7fd7c4065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 -- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c4160490 msgr2=0x7fd7c4065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c2d9d640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c4160950 0x7fd7c414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c4160490 0x7fd7c4065b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 -- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c4160950 msgr2=0x7fd7c414dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c4160950 0x7fd7c414dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 -- 172.21.15.157:0/1426774246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40094d0 con 0x7fd7c415f810 2023-11-06T03:12:32.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c259c640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40ac040 tx=0x7fd7b401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40b3800 con 0x7fd7c415f810 2023-11-06T03:12:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b40b3980 con 0x7fd7c415f810 2023-11-06T03:12:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c415cd60 con 0x7fd7c415f810 2023-11-06T03:12:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40b3d30 con 0x7fd7c415f810 2023-11-06T03:12:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c414f1d0 con 0x7fd7c415f810 2023-11-06T03:12:32.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c40cd400 con 0x7fd7c415f810 2023-11-06T03:12:32.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7b40bd3f0 con 0x7fd7c415f810 2023-11-06T03:12:32.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 0x7fd7a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:32.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd7b40a7070 con 0x7fd7c415f810 2023-11-06T03:12:32.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c1d9b640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 0x7fd7a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:32.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.770+0000 7fd7c1d9b640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 0x7fd7a4067f20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd7c415bee0 tx=0x7fd7b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:32.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.774+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7b40eb5b0 con 0x7fd7c415f810 2023-11-06T03:12:32.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7c4072750 con 0x7fd7c415f810 2023-11-06T03:12:32.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7b2ffd640 1 -- 172.21.15.157:0/1426774246 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7b40eebd0 con 0x7fd7c415f810 2023-11-06T03:12:32.926 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:12:32.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 msgr2=0x7fd7a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 0x7fd7a4067f20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd7c415bee0 tx=0x7fd7b8009040 comp rx=0 tx=0).stop 2023-11-06T03:12:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 msgr2=0x7fd7c40c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40ac040 tx=0x7fd7b401ad10 comp rx=0 tx=0).stop 2023-11-06T03:12:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 shutdown_connections 2023-11-06T03:12:32.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd7a4065a60 0x7fd7a4067f20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7c4160950 0x7fd7c414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7c4160490 0x7fd7c4065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 --2- 172.21.15.157:0/1426774246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7c415f810 0x7fd7c40c91b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.926+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 >> 172.21.15.157:0/1426774246 conn(0x7fd7c4070790 msgr2=0x7fd7c4070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:32.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.930+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 shutdown_connections 2023-11-06T03:12:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:32.930+0000 7fd7c8a70640 1 -- 172.21.15.157:0/1426774246 wait complete. 2023-11-06T03:12:32.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:33.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.050+0000 7f1f5e1fb640 1 Processor -- start 2023-11-06T03:12:33.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.050+0000 7f1f5e1fb640 1 -- start start 2023-11-06T03:12:33.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f5e1fb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1f580b2af0 0x7f1f580ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f5e1fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f5806d000 0x7f1f58159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f5e1fb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5815e440 con 0x7f1f581599c0 2023-11-06T03:12:33.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f5e1fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5806c240 con 0x7f1f5806d000 2023-11-06T03:12:33.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f5e1fb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5806c520 con 0x7f1f580b2af0 2023-11-06T03:12:33.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f5806d000 0x7f1f58159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f5806d000 0x7f1f58159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46306/0 (socket says 172.21.15.157:46306) 2023-11-06T03:12:33.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 -- 172.21.15.157:0/1691252443 learned_addr learned my addr 172.21.15.157:0/1691252443 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:33.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f56ffd640 1 --2- 172.21.15.157:0/1691252443 >> v2:172.21.15.195:3300/0 conn(0x7f1f580b2af0 0x7f1f580ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 -- 172.21.15.157:0/1691252443 >> v2:172.21.15.195:3300/0 conn(0x7f1f580b2af0 msgr2=0x7f1f580ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 --2- 172.21.15.157:0/1691252443 >> v2:172.21.15.195:3300/0 conn(0x7f1f580b2af0 0x7f1f580ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 -- 172.21.15.157:0/1691252443 >> v1:172.21.15.195:6789/0 conn(0x7f1f581599c0 legacy=0x7f1f580b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 -- 172.21.15.157:0/1691252443 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5806c800 con 0x7f1f5806d000 2023-11-06T03:12:33.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f57fff640 1 --2- 172.21.15.157:0/1691252443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f5806d000 0x7f1f58159da0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cfa20 tx=0x7f1f4801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feacee501c0b83d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f567fc640 1 -- 172.21.15.157:0/1691252443 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f480a5840 con 0x7f1f5806d000 2023-11-06T03:12:33.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f567fc640 1 -- 172.21.15.157:0/1691252443 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f480a59a0 con 0x7f1f5806d000 2023-11-06T03:12:33.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.054+0000 7f1f567fc640 1 -- 172.21.15.157:0/1691252443 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f480a5d30 con 0x7f1f5806d000 2023-11-06T03:12:33.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/1691252443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f5806d000 msgr2=0x7f1f58159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/1691252443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f5806d000 0x7f1f58159da0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cfa20 tx=0x7f1f4801c960 comp rx=0 tx=0).stop 2023-11-06T03:12:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/1691252443 shutdown_connections 2023-11-06T03:12:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/1691252443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f5806d000 0x7f1f58159da0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/1691252443 >> v2:172.21.15.195:3300/0 conn(0x7f1f580b2af0 0x7f1f580ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/1691252443 >> 172.21.15.157:0/1691252443 conn(0x7f1f58065930 msgr2=0x7f1f58065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/1691252443 shutdown_connections 2023-11-06T03:12:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/1691252443 wait complete. 2023-11-06T03:12:33.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 Processor -- start 2023-11-06T03:12:33.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- start start 2023-11-06T03:12:33.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f5806d000 0x7f1f5814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 0x7f1f580acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5814c0b0 con 0x7f1f580b2af0 2023-11-06T03:12:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f580aef70 con 0x7f1f5806d000 2023-11-06T03:12:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581589e0 con 0x7f1f581599c0 2023-11-06T03:12:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46308/0 (socket says 172.21.15.157:46308) 2023-11-06T03:12:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 -- 172.21.15.157:0/2631506031 learned_addr learned my addr 172.21.15.157:0/2631506031 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f57fff640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 0x7f1f580acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f577fe640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f5806d000 0x7f1f5814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 -- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 msgr2=0x7f1f580acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 0x7f1f580acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 -- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f5806d000 msgr2=0x7f1f5814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f5806d000 0x7f1f5814bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 -- 172.21.15.157:0/2631506031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009b30 con 0x7f1f580b2af0 2023-11-06T03:12:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f57fff640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 0x7f1f580acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f56ffd640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c7f0 tx=0x7f1f4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40004220 con 0x7f1f580b2af0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f400043a0 con 0x7f1f580b2af0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f580ae9f0 con 0x7f1f580b2af0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40004510 con 0x7f1f580b2af0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.058+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f580aec90 con 0x7f1f580b2af0 2023-11-06T03:12:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f58072750 con 0x7f1f580b2af0 2023-11-06T03:12:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f40003a70 con 0x7f1f580b2af0 2023-11-06T03:12:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f4ffff640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 0x7f1f38067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f400832b0 con 0x7f1f580b2af0 2023-11-06T03:12:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f577fe640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 0x7f1f38067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.062+0000 7f1f577fe640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 0x7f1f38067f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b810 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.066+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f40050b10 con 0x7f1f580b2af0 2023-11-06T03:12:33.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.238+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f580cb110 con 0x7f1f580b2af0 2023-11-06T03:12:33.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.238+0000 7f1f4ffff640 1 -- 172.21.15.157:0/2631506031 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f1f40054130 con 0x7f1f580b2af0 2023-11-06T03:12:33.239 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:33.239 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:33.240 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:33.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 msgr2=0x7f1f38067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 0x7f1f38067f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000b810 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2023-11-06T03:12:33.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 msgr2=0x7f1f580ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c7f0 tx=0x7f1f4000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 shutdown_connections 2023-11-06T03:12:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f38065a60 0x7f1f38067f20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f581599c0 0x7f1f580acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f580b2af0 0x7f1f580ac8b0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 --2- 172.21.15.157:0/2631506031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f5806d000 0x7f1f5814bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 >> 172.21.15.157:0/2631506031 conn(0x7f1f58065930 msgr2=0x7f1f58070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 shutdown_connections 2023-11-06T03:12:33.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.242+0000 7f1f5e1fb640 1 -- 172.21.15.157:0/2631506031 wait complete. 2023-11-06T03:12:33.257 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:33.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.378+0000 7fa3ec91d640 1 Processor -- start 2023-11-06T03:12:33.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.378+0000 7fa3ec91d640 1 -- start start 2023-11-06T03:12:33.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3ec91d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3e80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3ec91d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e815e440 con 0x7fa3e806d000 2023-11-06T03:12:33.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3ec91d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e806c240 con 0x7fa3e81599c0 2023-11-06T03:12:33.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3ec91d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e806c520 con 0x7fa3e80b2af0 2023-11-06T03:12:33.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e6d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa3e806d000 0x7fa3e8159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37740/0 (socket says 172.21.15.157:37740) 2023-11-06T03:12:33.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/3019594486 learned_addr learned my addr 172.21.15.157:0/3019594486 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:33.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3e80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753306534 0 0) 0x7fa3e815e440 con 0x7fa3e806d000 2023-11-06T03:12:33.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d4003530 con 0x7fa3e806d000 2023-11-06T03:12:33.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189268798 0 0) 0x7fa3d4003530 con 0x7fa3e806d000 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 msgr2=0x7fa3e80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3e80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 >> v1:172.21.15.195:6789/0 conn(0x7fa3e81599c0 legacy=0x7fa3e80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e806c800 con 0x7fa3e806d000 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa3e0001a00 con 0x7fa3e806d000 2023-11-06T03:12:33.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 >> v1:172.21.15.157:6789/0 conn(0x7fa3e806d000 legacy=0x7fa3e8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3d40047d0 0x7fa3d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e806c520 con 0x7fa3d40047d0 2023-11-06T03:12:33.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400dc50 con 0x7fa3d40039b0 2023-11-06T03:12:33.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400de10 con 0x7fa3e80b2af0 2023-11-06T03:12:33.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3d40047d0 0x7fa3d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e6d9d640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e659c640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 -- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 msgr2=0x7fa3d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3d4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 -- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d40039b0 msgr2=0x7fa3d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 -- 172.21.15.157:0/3019594486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4003530 con 0x7fa3d40047d0 2023-11-06T03:12:33.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e659c640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:33.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e6d9d640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80b2af0 0x7fa3d4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3dffff640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3d40047d0 0x7fa3d4004c50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ab0 tx=0x7fa3d001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255adc354af36fef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00a6430 con 0x7fa3d40047d0 2023-11-06T03:12:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d00a6590 con 0x7fa3d40047d0 2023-11-06T03:12:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.382+0000 7fa3e5d9b640 1 -- 172.21.15.157:0/3019594486 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00a6940 con 0x7fa3d40047d0 2023-11-06T03:12:33.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/3019594486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3d40047d0 msgr2=0x7fa3d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/3019594486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3d40047d0 0x7fa3d4004c50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009ab0 tx=0x7fa3d001c830 comp rx=0 tx=0).stop 2023-11-06T03:12:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/3019594486 reap_dead start 2023-11-06T03:12:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/3019594486 shutdown_connections 2023-11-06T03:12:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/3019594486 >> 172.21.15.157:0/3019594486 conn(0x7fa3e8065930 msgr2=0x7fa3e8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/3019594486 shutdown_connections 2023-11-06T03:12:33.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/3019594486 wait complete. 2023-11-06T03:12:33.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 Processor -- start 2023-11-06T03:12:33.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- start start 2023-11-06T03:12:33.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8158350 0x7fa3e8158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80aed00 0x7fa3e80af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8144e90 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80af870 con 0x7fa3e8158350 2023-11-06T03:12:33.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80afb50 con 0x7fa3e80aed00 2023-11-06T03:12:33.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46314/0 (socket says 172.21.15.157:46314) 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/2535570893 learned_addr learned my addr 172.21.15.157:0/2535570893 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3dffff640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80aed00 0x7fa3e80af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e659c640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8158350 0x7fa3e8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80aed00 msgr2=0x7fa3e80af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80aed00 0x7fa3e80af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8158350 msgr2=0x7fa3e8158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8158350 0x7fa3e8158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 -- 172.21.15.157:0/2535570893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80ccfa0 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3e6d9d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000f90 tx=0x7fa3e0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000eac0 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000ec40 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e80cd280 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0015400 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.386+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e80ccbc0 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e8072750 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa3e00143a0 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3deffd640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 0x7fa3c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3e0085a40 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3e659c640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 0x7fa3c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.390+0000 7fa3e659c640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 0x7fa3c8067f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8001760 tx=0x7fa3d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.394+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa3e0087e60 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.554+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3e80adcd0 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3deffd640 1 -- 172.21.15.157:0/2535570893 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fa3e001a070 con 0x7fa3e81ed5f0 2023-11-06T03:12:33.558 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:33.558 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:59.421679+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:33.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 msgr2=0x7fa3c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 0x7fa3c8067f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8001760 tx=0x7fa3d8009040 comp rx=0 tx=0).stop 2023-11-06T03:12:33.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 msgr2=0x7fa3e81eda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000f90 tx=0x7fa3e0010cb0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 shutdown_connections 2023-11-06T03:12:33.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3c8065a60 0x7fa3c8067f20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3e81ed5f0 0x7fa3e81eda70 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3e80aed00 0x7fa3e80af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 --2- 172.21.15.157:0/2535570893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3e8158350 0x7fa3e8158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 >> 172.21.15.157:0/2535570893 conn(0x7fa3e8065930 msgr2=0x7fa3e815c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 shutdown_connections 2023-11-06T03:12:33.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.558+0000 7fa3ec91d640 1 -- 172.21.15.157:0/2535570893 wait complete. 2023-11-06T03:12:33.577 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:12:33.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.690+0000 7f641b24a640 1 Processor -- start 2023-11-06T03:12:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.690+0000 7f641b24a640 1 -- start start 2023-11-06T03:12:33.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f641b24a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64141599c0 0x7f64140ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f641b24a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f641406d000 0x7f64140b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f641b24a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641415e440 con 0x7f641415dea0 2023-11-06T03:12:33.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f641b24a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641415adb0 con 0x7f64141599c0 2023-11-06T03:12:33.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f641b24a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641406c240 con 0x7f641406d000 2023-11-06T03:12:33.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64141599c0 0x7f64140ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f64141599c0 0x7f64140ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46324/0 (socket says 172.21.15.157:46324) 2023-11-06T03:12:33.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 -- 172.21.15.157:0/440957051 learned_addr learned my addr 172.21.15.157:0/440957051 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:33.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6413fff640 1 --2- 172.21.15.157:0/440957051 >> v2:172.21.15.195:3300/0 conn(0x7f641406d000 0x7f64140b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 -- 172.21.15.157:0/440957051 >> v2:172.21.15.195:3300/0 conn(0x7f641406d000 msgr2=0x7f64140b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 --2- 172.21.15.157:0/440957051 >> v2:172.21.15.195:3300/0 conn(0x7f641406d000 0x7f64140b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 -- 172.21.15.157:0/440957051 >> v1:172.21.15.195:6789/0 conn(0x7f641415dea0 legacy=0x7f6414070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.694+0000 7f6418fe6640 1 -- 172.21.15.157:0/440957051 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641406c520 con 0x7f64141599c0 2023-11-06T03:12:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 --2- 172.21.15.157:0/440957051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64141599c0 0x7f64140ae000 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f64140cfa20 tx=0x7f640801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba45bb2d872ef00 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f64137fe640 1 -- 172.21.15.157:0/440957051 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080ac600 con 0x7f64141599c0 2023-11-06T03:12:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f64137fe640 1 -- 172.21.15.157:0/440957051 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64080a8050 con 0x7f64141599c0 2023-11-06T03:12:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f64137fe640 1 -- 172.21.15.157:0/440957051 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080ac980 con 0x7f64141599c0 2023-11-06T03:12:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- 172.21.15.157:0/440957051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64141599c0 msgr2=0x7f64140ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- 172.21.15.157:0/440957051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64141599c0 0x7f64140ae000 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f64140cfa20 tx=0x7f640801c490 comp rx=0 tx=0).stop 2023-11-06T03:12:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- 172.21.15.157:0/440957051 shutdown_connections 2023-11-06T03:12:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- 172.21.15.157:0/440957051 >> v2:172.21.15.195:3300/0 conn(0x7f641406d000 0x7f64140b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- 172.21.15.157:0/440957051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64141599c0 0x7f64140ae000 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- 172.21.15.157:0/440957051 >> 172.21.15.157:0/440957051 conn(0x7f6414065930 msgr2=0x7f6414065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- 172.21.15.157:0/440957051 shutdown_connections 2023-11-06T03:12:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- 172.21.15.157:0/440957051 wait complete. 2023-11-06T03:12:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 Processor -- start 2023-11-06T03:12:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- start start 2023-11-06T03:12:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64141599c0 0x7f64141f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f641415dea0 0x7f64141f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141545e0 con 0x7f641406d000 2023-11-06T03:12:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141548c0 con 0x7f641415dea0 2023-11-06T03:12:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f641b24a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6414154ba0 con 0x7f64141599c0 2023-11-06T03:12:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46340/0 (socket says 172.21.15.157:46340) 2023-11-06T03:12:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 -- 172.21.15.157:0/2045935429 learned_addr learned my addr 172.21.15.157:0/2045935429 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f64197e7640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f641415dea0 0x7f64141f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6413fff640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64141599c0 0x7f64141f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 -- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64141599c0 msgr2=0x7f64141f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64141599c0 0x7f64141f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 -- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f641415dea0 msgr2=0x7f64141f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f641415dea0 0x7f64141f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.698+0000 7f6418fe6640 1 -- 172.21.15.157:0/2045935429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080094d0 con 0x7f641406d000 2023-11-06T03:12:33.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f6418fe6640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f640801a0c0 tx=0x7f640801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080b7620 con 0x7f641406d000 2023-11-06T03:12:33.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64080b7780 con 0x7f641406d000 2023-11-06T03:12:33.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6414154e80 con 0x7f641406d000 2023-11-06T03:12:33.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64080b7af0 con 0x7f641406d000 2023-11-06T03:12:33.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6414155440 con 0x7f641406d000 2023-11-06T03:12:33.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414072750 con 0x7f641406d000 2023-11-06T03:12:33.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f64080b3040 con 0x7f641406d000 2023-11-06T03:12:33.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 0x7f63f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:33.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f640811d410 con 0x7f641406d000 2023-11-06T03:12:33.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f6413fff640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 0x7f63f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:33.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.702+0000 7f6413fff640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 0x7f63f8067f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f64140cb060 tx=0x7f6400008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:33.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.706+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f64080ead00 con 0x7f641406d000 2023-11-06T03:12:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.898+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64140cb110 con 0x7f641406d000 2023-11-06T03:12:33.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.898+0000 7f64117fa640 1 -- 172.21.15.157:0/2045935429 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f64080ee320 con 0x7f641406d000 2023-11-06T03:12:33.900 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:33.900 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 msgr2=0x7f63f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 0x7f63f8067f20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f64140cb060 tx=0x7f6400008040 comp rx=0 tx=0).stop 2023-11-06T03:12:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 msgr2=0x7f641414e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f640801a0c0 tx=0x7f640801acd0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 shutdown_connections 2023-11-06T03:12:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f63f8065a60 0x7f63f8067f20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f641415dea0 0x7f64141f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64141599c0 0x7f64141f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 --2- 172.21.15.157:0/2045935429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f641406d000 0x7f641414e070 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 >> 172.21.15.157:0/2045935429 conn(0x7f6414065930 msgr2=0x7f64140b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 shutdown_connections 2023-11-06T03:12:33.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:33.902+0000 7f641b24a640 1 -- 172.21.15.157:0/2045935429 wait complete. 2023-11-06T03:12:33.919 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.034+0000 7f84c6f36640 1 Processor -- start 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 -- start start 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f84c0169740 0x7f84c0169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f84c00cedd0 0x7f84c00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c006dec0 con 0x7f84c00ce220 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00cf7c0 con 0x7f84c00cedd0 2023-11-06T03:12:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.038+0000 7f84c6f36640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c006c1d0 con 0x7f84c0169740 2023-11-06T03:12:34.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.050+0000 7f84bffff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f84c00ce220 0x7f84c006d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:45730/0 (socket says 172.21.15.157:45730) 2023-11-06T03:12:34.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.050+0000 7f84bffff640 1 -- 172.21.15.157:0/2798347971 learned_addr learned my addr 172.21.15.157:0/2798347971 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:34.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.062+0000 7f84c54d3640 1 --2- 172.21.15.157:0/2798347971 >> v2:172.21.15.157:3300/0 conn(0x7f84c00cedd0 0x7f84c00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.062+0000 7f84c54d3640 1 -- 172.21.15.157:0/2798347971 >> v2:172.21.15.195:3300/0 conn(0x7f84c0169740 msgr2=0x7f84c0169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:34.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.062+0000 7f84c54d3640 1 --2- 172.21.15.157:0/2798347971 >> v2:172.21.15.195:3300/0 conn(0x7f84c0169740 0x7f84c0169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.062+0000 7f84c54d3640 1 -- 172.21.15.157:0/2798347971 >> v1:172.21.15.195:6789/0 conn(0x7f84c00ce220 legacy=0x7f84c006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.062+0000 7f84c54d3640 1 -- 172.21.15.157:0/2798347971 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00b2100 con 0x7f84c00cedd0 2023-11-06T03:12:34.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c54d3640 1 --2- 172.21.15.157:0/2798347971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00cedd0 0x7f84c00cf280 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f84c00707f0 tx=0x7f84b401d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7477d3aefc3236d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84bf7fe640 1 -- 172.21.15.157:0/2798347971 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b40ac650 con 0x7f84c00cedd0 2023-11-06T03:12:34.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84bf7fe640 1 -- 172.21.15.157:0/2798347971 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b40ac7b0 con 0x7f84c00cedd0 2023-11-06T03:12:34.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84bf7fe640 1 -- 172.21.15.157:0/2798347971 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b40acb40 con 0x7f84c00cedd0 2023-11-06T03:12:34.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 -- 172.21.15.157:0/2798347971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00cedd0 msgr2=0x7f84c00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 --2- 172.21.15.157:0/2798347971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00cedd0 0x7f84c00cf280 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f84c00707f0 tx=0x7f84b401d570 comp rx=0 tx=0).stop 2023-11-06T03:12:34.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 -- 172.21.15.157:0/2798347971 shutdown_connections 2023-11-06T03:12:34.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 --2- 172.21.15.157:0/2798347971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00cedd0 0x7f84c00cf280 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 --2- 172.21.15.157:0/2798347971 >> v2:172.21.15.195:3300/0 conn(0x7f84c0169740 0x7f84c0169b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 -- 172.21.15.157:0/2798347971 >> 172.21.15.157:0/2798347971 conn(0x7f84c0065d30 msgr2=0x7f84c0066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 -- 172.21.15.157:0/2798347971 shutdown_connections 2023-11-06T03:12:34.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.074+0000 7f84c6f36640 1 -- 172.21.15.157:0/2798347971 wait complete. 2023-11-06T03:12:34.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 Processor -- start 2023-11-06T03:12:34.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- start start 2023-11-06T03:12:34.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c00cedd0 0x7f84c00acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c0169740 0x7f84c00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0084120 con 0x7f84c00ce220 2023-11-06T03:12:34.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0083510 con 0x7f84c00cedd0 2023-11-06T03:12:34.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00837f0 con 0x7f84c0169740 2023-11-06T03:12:34.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46366/0 (socket says 172.21.15.157:46366) 2023-11-06T03:12:34.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 -- 172.21.15.157:0/3570831497 learned_addr learned my addr 172.21.15.157:0/3570831497 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:34.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84bffff640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c00cedd0 0x7f84c00acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 -- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c0169740 msgr2=0x7f84c00ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c0169740 0x7f84c00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 -- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c00cedd0 msgr2=0x7f84c00acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c00cedd0 0x7f84c00acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 -- 172.21.15.157:0/3570831497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009420 con 0x7f84c00ce220 2023-11-06T03:12:34.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c4cd2640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f84b800ba50 tx=0x7f84b800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800cbe0 con 0x7f84c00ce220 2023-11-06T03:12:34.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b800cd60 con 0x7f84c00ce220 2023-11-06T03:12:34.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c0083ad0 con 0x7f84c00ce220 2023-11-06T03:12:34.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8014680 con 0x7f84c00ce220 2023-11-06T03:12:34.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.078+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0157db0 con 0x7f84c00ce220 2023-11-06T03:12:34.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0072750 con 0x7f84c00ce220 2023-11-06T03:12:34.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84b8007370 con 0x7f84c00ce220 2023-11-06T03:12:34.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84bd7fa640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 0x7f84ac067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84b80836e0 con 0x7f84c00ce220 2023-11-06T03:12:34.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84bffff640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 0x7f84ac067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.082+0000 7f84bffff640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 0x7f84ac067f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84c00ae470 tx=0x7f84b0006050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.086+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84b8050fa0 con 0x7f84c00ce220 2023-11-06T03:12:34.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c0071550 con 0x7f84c00ce220 2023-11-06T03:12:34.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84bd7fa640 1 -- 172.21.15.157:0/3570831497 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f84b80545c0 con 0x7f84c00ce220 2023-11-06T03:12:34.282 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:34.282 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":79,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:11:58.417992+0000","modified":"2023-11-06T03:12:00.431355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7785},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7785":{"gid":7785,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1762973381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1762973381},{"type":"v1","addr":"172.21.15.157:6835","nonce":1762973381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-06T03:12:34.282 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 79 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 msgr2=0x7f84ac067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 0x7f84ac067f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84c00ae470 tx=0x7f84b0006050 comp rx=0 tx=0).stop 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 msgr2=0x7f84c0157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f84b800ba50 tx=0x7f84b800bf10 comp rx=0 tx=0).stop 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 shutdown_connections 2023-11-06T03:12:34.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84ac065ab0 0x7f84ac067f70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c0169740 0x7f84c00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c00cedd0 0x7f84c00acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 --2- 172.21.15.157:0/3570831497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c00ce220 0x7f84c0157870 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 >> 172.21.15.157:0/3570831497 conn(0x7f84c0065d30 msgr2=0x7f84c0168160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 shutdown_connections 2023-11-06T03:12:34.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.282+0000 7f84c6f36640 1 -- 172.21.15.157:0/3570831497 wait complete. 2023-11-06T03:12:34.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:34.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.406+0000 7fb06e226640 1 Processor -- start 2023-11-06T03:12:34.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.406+0000 7fb06e226640 1 -- start start 2023-11-06T03:12:34.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb06e226640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 0x7fb0680cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb06e226640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 0x7fb0680b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb06e226640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680d2920 con 0x7fb06815ce80 2023-11-06T03:12:34.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb06e226640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680d18a0 con 0x7fb0680d0280 2023-11-06T03:12:34.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb06e226640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680d1b60 con 0x7fb06815caa0 2023-11-06T03:12:34.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb067fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 0x7fb0680b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb0677fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 0x7fb0680cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb067fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 0x7fb0680b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:54318/0 (socket says 172.21.15.157:54318) 2023-11-06T03:12:34.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb067fff640 1 -- 172.21.15.157:0/1077138379 learned_addr learned my addr 172.21.15.157:0/1077138379 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:34.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb0677fe640 1 -- 172.21.15.157:0/1077138379 >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 msgr2=0x7fb0680b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:34.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb0677fe640 1 --2- 172.21.15.157:0/1077138379 >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 0x7fb0680b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb0677fe640 1 -- 172.21.15.157:0/1077138379 >> v1:172.21.15.195:6789/0 conn(0x7fb06815ce80 legacy=0x7fb0680cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.410+0000 7fb0677fe640 1 -- 172.21.15.157:0/1077138379 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680d1e40 con 0x7fb06815caa0 2023-11-06T03:12:34.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb0677fe640 1 --2- 172.21.15.157:0/1077138379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 0x7fb0680cdc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097c0 tx=0x7fb05801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d70d2ddfe85507e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb066ffd640 1 -- 172.21.15.157:0/1077138379 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580ab650 con 0x7fb06815caa0 2023-11-06T03:12:34.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb066ffd640 1 -- 172.21.15.157:0/1077138379 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0580ab7b0 con 0x7fb06815caa0 2023-11-06T03:12:34.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb066ffd640 1 -- 172.21.15.157:0/1077138379 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580abb40 con 0x7fb06815caa0 2023-11-06T03:12:34.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- 172.21.15.157:0/1077138379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 msgr2=0x7fb0680cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- 172.21.15.157:0/1077138379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 0x7fb0680cdc60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097c0 tx=0x7fb05801c450 comp rx=0 tx=0).stop 2023-11-06T03:12:34.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- 172.21.15.157:0/1077138379 shutdown_connections 2023-11-06T03:12:34.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- 172.21.15.157:0/1077138379 >> v2:172.21.15.195:3300/0 conn(0x7fb0680d0280 0x7fb0680b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- 172.21.15.157:0/1077138379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815caa0 0x7fb0680cdc60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- 172.21.15.157:0/1077138379 >> 172.21.15.157:0/1077138379 conn(0x7fb068070790 msgr2=0x7fb068070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- 172.21.15.157:0/1077138379 shutdown_connections 2023-11-06T03:12:34.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- 172.21.15.157:0/1077138379 wait complete. 2023-11-06T03:12:34.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 Processor -- start 2023-11-06T03:12:34.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- start start 2023-11-06T03:12:34.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06815caa0 0x7fb068151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815ce80 0x7fb06815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068151a70 con 0x7fb0680d0280 2023-11-06T03:12:34.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068151d50 con 0x7fb06815caa0 2023-11-06T03:12:34.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb06e226640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815d7c0 con 0x7fb06815ce80 2023-11-06T03:12:34.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb0677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb0677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46378/0 (socket says 172.21.15.157:46378) 2023-11-06T03:12:34.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb0677fe640 1 -- 172.21.15.157:0/2913872439 learned_addr learned my addr 172.21.15.157:0/2913872439 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:34.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.414+0000 7fb067fff640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815ce80 0x7fb06815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb05ffff640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06815caa0 0x7fb068151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 -- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815ce80 msgr2=0x7fb06815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815ce80 0x7fb06815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 -- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06815caa0 msgr2=0x7fb068151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06815caa0 0x7fb068151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 -- 172.21.15.157:0/2913872439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009420 con 0x7fb0680d0280 2023-11-06T03:12:34.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb0677fe640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb05801a080 tx=0x7fb05801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580bf610 con 0x7fb0680d0280 2023-11-06T03:12:34.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0580bf770 con 0x7fb0680d0280 2023-11-06T03:12:34.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580bfae0 con 0x7fb0680d0280 2023-11-06T03:12:34.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068153270 con 0x7fb0680d0280 2023-11-06T03:12:34.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0680c8720 con 0x7fb0680d0280 2023-11-06T03:12:34.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068072750 con 0x7fb0680d0280 2023-11-06T03:12:34.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb0580b5400 con 0x7fb0680d0280 2023-11-06T03:12:34.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 0x7fb048067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb05811e0d0 con 0x7fb0680d0280 2023-11-06T03:12:34.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb05ffff640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 0x7fb048067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.418+0000 7fb05ffff640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 0x7fb048067f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0681f8bc0 tx=0x7fb050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.422+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb0580eb9c0 con 0x7fb0680d0280 2023-11-06T03:12:34.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.566+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb068159ba0 con 0x7fb0680d0280 2023-11-06T03:12:34.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb064ff9640 1 -- 172.21.15.157:0/2913872439 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17874 (secure 0 0 0) 0x7fb0580eefe0 con 0x7fb0680d0280 2023-11-06T03:12:34.570 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:34.570 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:11:59.421679+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:11:56.975891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-06T03:11:57.391711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:34.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 msgr2=0x7fb048067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 0x7fb048067f20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0681f8bc0 tx=0x7fb050009040 comp rx=0 tx=0).stop 2023-11-06T03:12:34.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 msgr2=0x7fb06815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb05801a080 tx=0x7fb05801ac90 comp rx=0 tx=0).stop 2023-11-06T03:12:34.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 shutdown_connections 2023-11-06T03:12:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb048065a60 0x7fb048067f20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb06815ce80 0x7fb06815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb06815caa0 0x7fb068151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 --2- 172.21.15.157:0/2913872439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0680d0280 0x7fb06815d280 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 >> 172.21.15.157:0/2913872439 conn(0x7fb068070790 msgr2=0x7fb0680d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 shutdown_connections 2023-11-06T03:12:34.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.570+0000 7fb06e226640 1 -- 172.21.15.157:0/2913872439 wait complete. 2023-11-06T03:12:34.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:12:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 Processor -- start 2023-11-06T03:12:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 -- start start 2023-11-06T03:12:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcb4c1551e0 0x7fcb4c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c15f580 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c15efc0 con 0x7fcb4c0b3b20 2023-11-06T03:12:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb514a9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f2a0 con 0x7fcb4c1551e0 2023-11-06T03:12:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb427fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcb4c15e5e0 0x7fcb4c0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37748/0 (socket says 172.21.15.157:37748) 2023-11-06T03:12:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb427fc640 1 -- 172.21.15.157:0/4005137335 learned_addr learned my addr 172.21.15.157:0/4005137335 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4affd640 1 --2- 172.21.15.157:0/4005137335 >> v2:172.21.15.195:3300/0 conn(0x7fcb4c1551e0 0x7fcb4c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568743258 0 0) 0x7fcb4c15f580 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb38003530 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798355697 0 0) 0x7fcb38003530 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 >> v2:172.21.15.195:3300/0 conn(0x7fcb4c1551e0 msgr2=0x7fcb4c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 --2- 172.21.15.157:0/4005137335 >> v2:172.21.15.195:3300/0 conn(0x7fcb4c1551e0 0x7fcb4c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 >> v1:172.21.15.195:6789/0 conn(0x7fcb4c0b3b20 legacy=0x7fcb4c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c06f5a0 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fcb34001680 con 0x7fcb4c15e5e0 2023-11-06T03:12:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 >> v1:172.21.15.157:6789/0 conn(0x7fcb4c15e5e0 legacy=0x7fcb4c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb38003980 0x7fcb38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb380047d0 0x7fcb38004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 0x7fcb3800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f2a0 con 0x7fcb380052b0 2023-11-06T03:12:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3800c180 con 0x7fcb380047d0 2023-11-06T03:12:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.698+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3800c340 con 0x7fcb38003980 2023-11-06T03:12:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 0x7fcb3800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb427fc640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb380047d0 0x7fcb38004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4affd640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb38003980 0x7fcb38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 -- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb38003980 msgr2=0x7fcb38003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:34.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb38003980 0x7fcb38003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 -- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb380047d0 msgr2=0x7fcb38004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:34.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb380047d0 0x7fcb38004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.702+0000 7fcb4b7fe640 1 -- 172.21.15.157:0/4005137335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38003530 con 0x7fcb380052b0 2023-11-06T03:12:34.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4b7fe640 1 -- 172.21.15.157:0/4005137335 reap_dead start 2023-11-06T03:12:34.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4b7fe640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 0x7fcb3800b990 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001e40 tx=0x7fcb4401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17d80c6872f9021 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44003b80 con 0x7fcb380052b0 2023-11-06T03:12:34.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb44003ce0 con 0x7fcb380052b0 2023-11-06T03:12:34.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4a7fc640 1 -- 172.21.15.157:0/4005137335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44005260 con 0x7fcb380052b0 2023-11-06T03:12:34.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 msgr2=0x7fcb3800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 0x7fcb3800b990 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001e40 tx=0x7fcb4401c710 comp rx=0 tx=0).stop 2023-11-06T03:12:34.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- 172.21.15.157:0/4005137335 shutdown_connections 2023-11-06T03:12:34.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 --2- 172.21.15.157:0/4005137335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb380052b0 0x7fcb3800b990 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- 172.21.15.157:0/4005137335 >> 172.21.15.157:0/4005137335 conn(0x7fcb4c065930 msgr2=0x7fcb4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- 172.21.15.157:0/4005137335 shutdown_connections 2023-11-06T03:12:34.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- 172.21.15.157:0/4005137335 wait complete. 2023-11-06T03:12:34.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 Processor -- start 2023-11-06T03:12:34.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- start start 2023-11-06T03:12:34.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb380052b0 0x7fcb4c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb4c1470c0 0x7fcb4c147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c149380 con 0x7fcb4c142c30 2023-11-06T03:12:34.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c148170 con 0x7fcb4c1470c0 2023-11-06T03:12:34.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb514a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c148450 con 0x7fcb380052b0 2023-11-06T03:12:34.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb427fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb427fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46404/0 (socket says 172.21.15.157:46404) 2023-11-06T03:12:34.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb427fc640 1 -- 172.21.15.157:0/3061475116 learned_addr learned my addr 172.21.15.157:0/3061475116 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:34.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4b7fe640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb4c1470c0 0x7fcb4c147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.706+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb380052b0 0x7fcb4c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 -- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb380052b0 msgr2=0x7fcb4c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb380052b0 0x7fcb4c1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 -- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb4c1470c0 msgr2=0x7fcb4c147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb4c1470c0 0x7fcb4c147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 -- 172.21.15.157:0/3061475116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c148730 con 0x7fcb4c142c30 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb427fc640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcb340072c0 tx=0x7fcb34007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400f9c0 con 0x7fcb4c142c30 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400fb20 con 0x7fcb4c142c30 2023-11-06T03:12:34.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c146de0 con 0x7fcb4c142c30 2023-11-06T03:12:34.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400feb0 con 0x7fcb4c142c30 2023-11-06T03:12:34.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c0b3900 con 0x7fcb4c142c30 2023-11-06T03:12:34.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c072750 con 0x7fcb4c142c30 2023-11-06T03:12:34.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcb3401d460 con 0x7fcb4c142c30 2023-11-06T03:12:34.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 0x7fcb2c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:34.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb34083330 con 0x7fcb4c142c30 2023-11-06T03:12:34.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 0x7fcb2c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:34.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.710+0000 7fcb4affd640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 0x7fcb2c067f50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009aa0 tx=0x7fcb3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:34.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.714+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcb34050c20 con 0x7fcb4c142c30 2023-11-06T03:12:34.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.890+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcb4c1488c0 con 0x7fcb4c142c30 2023-11-06T03:12:34.891 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:12:34.890+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:12:34.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.910+0000 7fcb43fff640 1 -- 172.21.15.157:0/3061475116 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcb34054240 con 0x7fcb4c142c30 2023-11-06T03:12:34.911 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:12:34.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 msgr2=0x7fcb2c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 0x7fcb2c067f50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009aa0 tx=0x7fcb3c008040 comp rx=0 tx=0).stop 2023-11-06T03:12:34.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 msgr2=0x7fcb4c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:34.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcb340072c0 tx=0x7fcb34007780 comp rx=0 tx=0).stop 2023-11-06T03:12:34.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 shutdown_connections 2023-11-06T03:12:34.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcb2c065a90 0x7fcb2c067f50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb4c1470c0 0x7fcb4c147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb4c142c30 0x7fcb4c0b31f0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 --2- 172.21.15.157:0/3061475116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb380052b0 0x7fcb4c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:34.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 >> 172.21.15.157:0/3061475116 conn(0x7fcb4c065930 msgr2=0x7fcb4c150150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:34.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 shutdown_connections 2023-11-06T03:12:34.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:34.914+0000 7fcb514a9640 1 -- 172.21.15.157:0/3061475116 wait complete. 2023-11-06T03:12:34.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:12:34.960 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:12:35.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 Processor -- start 2023-11-06T03:12:35.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 -- start start 2023-11-06T03:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2504161020 con 0x7f250415ce60 2023-11-06T03:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2504072020 con 0x7f25040cca40 2023-11-06T03:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f250a996640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2504072300 con 0x7f25040d00c0 2023-11-06T03:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f25037fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f25037fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:54386/0 (socket says 172.21.15.157:54386) 2023-11-06T03:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:48054/0 (socket says 172.21.15.157:48054) 2023-11-06T03:12:35.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f25037fe640 1 -- 172.21.15.157:0/1396312080 learned_addr learned my addr 172.21.15.157:0/1396312080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 -- 172.21.15.157:0/1396312080 >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 msgr2=0x7f25041614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 --2- 172.21.15.157:0/1396312080 >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 -- 172.21.15.157:0/1396312080 >> v1:172.21.15.195:6789/0 conn(0x7f250415ce60 legacy=0x7f25040cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2502ffd640 1 -- 172.21.15.157:0/1396312080 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847065213 0 0) 0x7f2504161020 con 0x7f250415ce60 2023-11-06T03:12:35.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f2508f33640 1 -- 172.21.15.157:0/1396312080 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25041609d0 con 0x7f25040d00c0 2023-11-06T03:12:35.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.042+0000 7f25037fe640 1 --2- 172.21.15.157:0/1396312080 >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:12:35.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f2508f33640 1 --2- 172.21.15.157:0/1396312080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d90 tx=0x7f24fc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298133bd9cd39b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:35.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f2502ffd640 1 -- 172.21.15.157:0/1396312080 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0a5860 con 0x7f25040d00c0 2023-11-06T03:12:35.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f2502ffd640 1 -- 172.21.15.157:0/1396312080 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc0a59c0 con 0x7f25040d00c0 2023-11-06T03:12:35.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f2502ffd640 1 -- 172.21.15.157:0/1396312080 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0a5d50 con 0x7f25040d00c0 2023-11-06T03:12:35.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- 172.21.15.157:0/1396312080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 msgr2=0x7f25040caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- 172.21.15.157:0/1396312080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d90 tx=0x7f24fc01c890 comp rx=0 tx=0).stop 2023-11-06T03:12:35.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- 172.21.15.157:0/1396312080 shutdown_connections 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- 172.21.15.157:0/1396312080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25040caf60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- 172.21.15.157:0/1396312080 >> v2:172.21.15.195:3300/0 conn(0x7f25040cca40 0x7f25041614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- 172.21.15.157:0/1396312080 >> 172.21.15.157:0/1396312080 conn(0x7f250406d3d0 msgr2=0x7f250406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- 172.21.15.157:0/1396312080 shutdown_connections 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- 172.21.15.157:0/1396312080 wait complete. 2023-11-06T03:12:35.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 Processor -- start 2023-11-06T03:12:35.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- start start 2023-11-06T03:12:35.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25041582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f250415ce60 0x7f25040af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504158810 con 0x7f25040cca40 2023-11-06T03:12:35.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25040ae290 con 0x7f250415ce60 2023-11-06T03:12:35.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.046+0000 7f250a996640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25040b3260 con 0x7f25040d00c0 2023-11-06T03:12:35.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46442/0 (socket says 172.21.15.157:46442) 2023-11-06T03:12:35.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 -- 172.21.15.157:0/212960483 learned_addr learned my addr 172.21.15.157:0/212960483 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:35.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f25037fe640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25041582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2508f33640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f250415ce60 0x7f25040af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 -- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 msgr2=0x7f25041582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25041582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 -- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f250415ce60 msgr2=0x7f25040af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f250415ce60 0x7f25040af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 -- 172.21.15.157:0/212960483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0094d0 con 0x7f25040cca40 2023-11-06T03:12:35.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2503fff640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c500 tx=0x7f24f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4003600 con 0x7f25040cca40 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f4003760 con 0x7f25040cca40 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc009a70 con 0x7f25040cca40 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4003b10 con 0x7f25040cca40 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25040aedf0 con 0x7f25040cca40 2023-11-06T03:12:35.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2504072750 con 0x7f25040cca40 2023-11-06T03:12:35.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f24f4003c70 con 0x7f25040cca40 2023-11-06T03:12:35.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 0x7f24e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:35.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f24f40828e0 con 0x7f25040cca40 2023-11-06T03:12:35.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f25037fe640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 0x7f24e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:35.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.050+0000 7f25037fe640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 0x7f24e4067f20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f25040caca0 tx=0x7f24ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:35.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.054+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f24f4050230 con 0x7f25040cca40 2023-11-06T03:12:35.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.230+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2504167ae0 con 0x7f25040cca40 2023-11-06T03:12:35.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.914+0000 7f2500ff9640 1 -- 172.21.15.157:0/212960483 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7f24f4017020 con 0x7f25040cca40 2023-11-06T03:12:35.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 msgr2=0x7f24e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 0x7f24e4067f20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f25040caca0 tx=0x7f24ec008040 comp rx=0 tx=0).stop 2023-11-06T03:12:35.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 msgr2=0x7f25040b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:35.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f24f400c500 tx=0x7f24f400c9c0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 shutdown_connections 2023-11-06T03:12:35.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24e4065a60 0x7f24e4067f20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f250415ce60 0x7f25040af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f25040d00c0 0x7f25041582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 --2- 172.21.15.157:0/212960483 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25040cca40 0x7f25040b2b60 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:35.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 >> 172.21.15.157:0/212960483 conn(0x7f250406d3d0 msgr2=0x7f2504158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:35.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 shutdown_connections 2023-11-06T03:12:35.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:35.918+0000 7f250a996640 1 -- 172.21.15.157:0/212960483 wait complete. 2023-11-06T03:12:35.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:12:36.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 Processor -- start 2023-11-06T03:12:36.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 -- start start 2023-11-06T03:12:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0aed90 0x7ff1cc0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc15af70 con 0x7ff1cc0d1170 2023-11-06T03:12:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cc9e0 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.114+0000 7ff1d4452640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cc140 con 0x7ff1cc0aed90 2023-11-06T03:12:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46454/0 (socket says 172.21.15.157:46454) 2023-11-06T03:12:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 -- 172.21.15.157:0/2646007010 learned_addr learned my addr 172.21.15.157:0/2646007010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d29ef640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0aed90 0x7ff1cc0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 -- 172.21.15.157:0/2646007010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0aed90 msgr2=0x7ff1cc0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0aed90 0x7ff1cc0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 -- 172.21.15.157:0/2646007010 >> v1:172.21.15.195:6789/0 conn(0x7ff1cc0d1170 legacy=0x7ff1cc0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:36.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 -- 172.21.15.157:0/2646007010 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0cc420 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d21ee640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009870 tx=0x7ff1c801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a521d04e096b191 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d11ec640 1 -- 172.21.15.157:0/2646007010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80ab600 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d11ec640 1 -- 172.21.15.157:0/2646007010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c80a7050 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d11ec640 1 -- 172.21.15.157:0/2646007010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80ab980 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 -- 172.21.15.157:0/2646007010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d0d90 msgr2=0x7ff1cc0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:36.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009870 tx=0x7ff1c801c3b0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 -- 172.21.15.157:0/2646007010 shutdown_connections 2023-11-06T03:12:36.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0aed90 0x7ff1cc0cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 --2- 172.21.15.157:0/2646007010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d0d90 0x7ff1cc0cf180 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 -- 172.21.15.157:0/2646007010 >> 172.21.15.157:0/2646007010 conn(0x7ff1cc071c30 msgr2=0x7ff1cc099ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:36.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 -- 172.21.15.157:0/2646007010 shutdown_connections 2023-11-06T03:12:36.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.118+0000 7ff1d4452640 1 -- 172.21.15.157:0/2646007010 wait complete. 2023-11-06T03:12:36.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 Processor -- start 2023-11-06T03:12:36.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- start start 2023-11-06T03:12:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1cc0aed90 0x7ff1cc0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0d0d90 0x7ff1cc14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b64e0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b58d0 con 0x7ff1cc0aed90 2023-11-06T03:12:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b5bb0 con 0x7ff1cc0d0d90 2023-11-06T03:12:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46456/0 (socket says 172.21.15.157:46456) 2023-11-06T03:12:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 -- 172.21.15.157:0/1631797808 learned_addr learned my addr 172.21.15.157:0/1631797808 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d19ed640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0d0d90 0x7ff1cc14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 -- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0d0d90 msgr2=0x7ff1cc14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d21ee640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1cc0aed90 0x7ff1cc0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0d0d90 0x7ff1cc14cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 -- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1cc0aed90 msgr2=0x7ff1cc0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1cc0aed90 0x7ff1cc0b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 -- 172.21.15.157:0/1631797808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80094d0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d29ef640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cc80 tx=0x7ff1c400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400ce90 con 0x7ff1cc0d1170 2023-11-06T03:12:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c400ae80 con 0x7ff1cc0d1170 2023-11-06T03:12:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0b5e90 con 0x7ff1cc0d1170 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4003be0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc0b3cd0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc04e050 con 0x7ff1cc0d1170 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff1c400f3f0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.122+0000 7ff1c2ffd640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 0x7ff1b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.126+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff1c40836d0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.126+0000 7ff1d21ee640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 0x7ff1b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:36.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.126+0000 7ff1d21ee640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 0x7ff1b0067f20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80ad000 tx=0x7ff1c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:36.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.130+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff1c4050ed0 con 0x7ff1cc0d1170 2023-11-06T03:12:36.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.270+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1cc0b6620 con 0x7ff1cc0d1170 2023-11-06T03:12:36.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.926+0000 7ff1c2ffd640 1 -- 172.21.15.157:0/1631797808 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff1cc0b6620 con 0x7ff1cc0d1170 2023-11-06T03:12:36.929 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:12:36.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 msgr2=0x7ff1b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:36.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 0x7ff1b0067f20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80ad000 tx=0x7ff1c8008040 comp rx=0 tx=0).stop 2023-11-06T03:12:36.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 msgr2=0x7ff1cc0b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:36.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cc80 tx=0x7ff1c400a5c0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 shutdown_connections 2023-11-06T03:12:36.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff1b0065a60 0x7ff1b0067f20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1cc0d1170 0x7ff1cc0b3790 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff1cc0d0d90 0x7ff1cc14cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 --2- 172.21.15.157:0/1631797808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff1cc0aed90 0x7ff1cc0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:36.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 >> 172.21.15.157:0/1631797808 conn(0x7ff1cc071c30 msgr2=0x7ff1cc0af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:36.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 shutdown_connections 2023-11-06T03:12:36.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:36.934+0000 7ff1d4452640 1 -- 172.21.15.157:0/1631797808 wait complete. 2023-11-06T03:12:36.958 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:12:37.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.094+0000 7ff04ae34640 1 Processor -- start 2023-11-06T03:12:37.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.094+0000 7ff04ae34640 1 -- start start 2023-11-06T03:12:37.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff04ae34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff044159ac0 0x7ff0440ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff04ae34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff044159ea0 0x7ff0440ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff04ae34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04406d0b0 con 0x7ff0440cf090 2023-11-06T03:12:37.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff04ae34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0440cb440 con 0x7ff044159ac0 2023-11-06T03:12:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff04ae34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440aeeb0 con 0x7ff044159ea0 2023-11-06T03:12:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff044159ac0 0x7ff0440ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff044159ac0 0x7ff0440ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46466/0 (socket says 172.21.15.157:46466) 2023-11-06T03:12:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 -- 172.21.15.157:0/3919975072 learned_addr learned my addr 172.21.15.157:0/3919975072 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:37.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff043fff640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff044159ea0 0x7ff0440ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 -- 172.21.15.157:0/3919975072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff044159ea0 msgr2=0x7ff0440ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:37.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff044159ea0 0x7ff0440ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 -- 172.21.15.157:0/3919975072 >> v1:172.21.15.157:6789/0 conn(0x7ff0440cf090 legacy=0x7ff04415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 -- 172.21.15.157:0/3919975072 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440cb700 con 0x7ff044159ac0 2023-11-06T03:12:37.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff0437fe640 1 -- 172.21.15.157:0/3919975072 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388737877 0 0) 0x7ff04406d0b0 con 0x7ff0440cf090 2023-11-06T03:12:37.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff048bd0640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ac0 0x7ff0440ce610 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff038009870 tx=0x7ff03801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5e4fcc961c7b34 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:37.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff0437fe640 1 -- 172.21.15.157:0/3919975072 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380ac620 con 0x7ff044159ac0 2023-11-06T03:12:37.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff0437fe640 1 -- 172.21.15.157:0/3919975072 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380ac780 con 0x7ff044159ac0 2023-11-06T03:12:37.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.098+0000 7ff0437fe640 1 -- 172.21.15.157:0/3919975072 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380acb10 con 0x7ff044159ac0 2023-11-06T03:12:37.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/3919975072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ac0 msgr2=0x7ff0440ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ac0 0x7ff0440ce610 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff038009870 tx=0x7ff03801c4b0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/3919975072 shutdown_connections 2023-11-06T03:12:37.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff044159ea0 0x7ff0440ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- 172.21.15.157:0/3919975072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ac0 0x7ff0440ce610 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/3919975072 >> 172.21.15.157:0/3919975072 conn(0x7ff0440b1340 msgr2=0x7ff0440b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:37.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/3919975072 shutdown_connections 2023-11-06T03:12:37.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/3919975072 wait complete. 2023-11-06T03:12:37.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 Processor -- start 2023-11-06T03:12:37.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- start start 2023-11-06T03:12:37.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0440cf090 0x7ff0440c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff044159ac0 0x7ff044065b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440b3400 con 0x7ff044159ea0 2023-11-06T03:12:37.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440c79a0 con 0x7ff044159ac0 2023-11-06T03:12:37.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440c6e60 con 0x7ff0440cf090 2023-11-06T03:12:37.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46478/0 (socket says 172.21.15.157:46478) 2023-11-06T03:12:37.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 -- 172.21.15.157:0/2593719388 learned_addr learned my addr 172.21.15.157:0/2593719388 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:37.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff043fff640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff044159ac0 0x7ff044065b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff048bd0640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0440cf090 0x7ff0440c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 -- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0440cf090 msgr2=0x7ff0440c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0440cf090 0x7ff0440c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 -- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff044159ac0 msgr2=0x7ff044065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff044159ac0 0x7ff044065b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 -- 172.21.15.157:0/2593719388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380094d0 con 0x7ff044159ea0 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0493d1640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff03c001570 tx=0x7ff03c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c003db0 con 0x7ff044159ea0 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c005070 con 0x7ff044159ea0 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0440c7100 con 0x7ff044159ea0 2023-11-06T03:12:37.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c005420 con 0x7ff044159ea0 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.102+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0440c76c0 con 0x7ff044159ea0 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04404e050 con 0x7ff044159ea0 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff03c005580 con 0x7ff044159ea0 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff0417fa640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 0x7ff028067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5710+0+0 (secure 0 0 0) 0x7ff03c082a40 con 0x7ff044159ea0 2023-11-06T03:12:37.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff048bd0640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 0x7ff028067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:37.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.106+0000 7ff048bd0640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 0x7ff028067f20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff0380ae000 tx=0x7ff038008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:37.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.110+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff03c084e60 con 0x7ff044159ea0 2023-11-06T03:12:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.262+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0440b57d0 con 0x7ff044159ea0 2023-11-06T03:12:37.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.942+0000 7ff0417fa640 1 -- 172.21.15.157:0/2593719388 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff03c017020 con 0x7ff044159ea0 2023-11-06T03:12:37.941 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:12:37.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 msgr2=0x7ff028067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 0x7ff028067f20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff0380ae000 tx=0x7ff038008040 comp rx=0 tx=0).stop 2023-11-06T03:12:37.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 msgr2=0x7ff044152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:37.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff03c001570 tx=0x7ff03c00cd20 comp rx=0 tx=0).stop 2023-11-06T03:12:37.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 shutdown_connections 2023-11-06T03:12:37.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff028065a60 0x7ff028067f20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044159ea0 0x7ff044152dd0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff044159ac0 0x7ff044065b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 --2- 172.21.15.157:0/2593719388 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0440cf090 0x7ff0440c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:37.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 >> 172.21.15.157:0/2593719388 conn(0x7ff0440b1340 msgr2=0x7ff044072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:37.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 shutdown_connections 2023-11-06T03:12:37.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:37.946+0000 7ff04ae34640 1 -- 172.21.15.157:0/2593719388 wait complete. 2023-11-06T03:12:37.980 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:12:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.102+0000 7f6860a2e640 1 Processor -- start 2023-11-06T03:12:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.102+0000 7f6860a2e640 1 -- start start 2023-11-06T03:12:38.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6860a2e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f685c0adae0 0x7f685c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6860a2e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f685c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6860a2e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c06c490 con 0x7f685c156a70 2023-11-06T03:12:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6860a2e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c06b950 con 0x7f685c0adae0 2023-11-06T03:12:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6860a2e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06fb80 con 0x7f685c0adf40 2023-11-06T03:12:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f685c156a70 0x7f685c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:37786/0 (socket says 172.21.15.157:37786) 2023-11-06T03:12:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 -- 172.21.15.157:0/4102879661 learned_addr learned my addr 172.21.15.157:0/4102879661 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6859d9b640 1 --2- 172.21.15.157:0/4102879661 >> v2:172.21.15.195:3300/0 conn(0x7f685c0adae0 0x7f685c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685ad9d640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f685c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182874848 0 0) 0x7f685c06c490 con 0x7f685c156a70 2023-11-06T03:12:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6844003450 con 0x7f685c156a70 2023-11-06T03:12:38.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176479328 0 0) 0x7f6844003450 con 0x7f685c156a70 2023-11-06T03:12:38.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 msgr2=0x7f685c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f685c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 >> v2:172.21.15.195:3300/0 conn(0x7f685c0adae0 msgr2=0x7f685c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 --2- 172.21.15.157:0/4102879661 >> v2:172.21.15.195:3300/0 conn(0x7f685c0adae0 0x7f685c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c06bb60 con 0x7f685c156a70 2023-11-06T03:12:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f684c001680 con 0x7f685c156a70 2023-11-06T03:12:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 >> v1:172.21.15.157:6789/0 conn(0x7f685c156a70 legacy=0x7f685c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6844003a70 0x7f6844003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6844008860 0x7f684400ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f684400d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06b950 con 0x7f6844003a70 2023-11-06T03:12:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06fb80 con 0x7f6844008860 2023-11-06T03:12:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684400dba0 con 0x7f685c0adf40 2023-11-06T03:12:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6844003a70 0x7f6844003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685ad9d640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f684400d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f6859d9b640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6844008860 0x7f684400ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 -- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 msgr2=0x7f684400d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c0adf40 0x7f684400d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 -- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6844008860 msgr2=0x7f684400ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6844008860 0x7f684400ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.106+0000 7f685a59c640 1 -- 172.21.15.157:0/4102879661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844003450 con 0x7f6844003a70 2023-11-06T03:12:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f685a59c640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6844003a70 0x7f6844003e90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c240 tx=0x7f684c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6937eedc09a49dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0058e0 con 0x7f6844003a70 2023-11-06T03:12:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c005a40 con 0x7f6844003a70 2023-11-06T03:12:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f685959a640 1 -- 172.21.15.157:0/4102879661 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c005e30 con 0x7f6844003a70 2023-11-06T03:12:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f6860a2e640 1 -- 172.21.15.157:0/4102879661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6844003a70 msgr2=0x7f6844003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f6860a2e640 1 --2- 172.21.15.157:0/4102879661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6844003a70 0x7f6844003e90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c240 tx=0x7f684c01cc30 comp rx=0 tx=0).stop 2023-11-06T03:12:38.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f685ad9d640 1 -- 172.21.15.157:0/4102879661 reap_dead start 2023-11-06T03:12:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f6860a2e640 1 -- 172.21.15.157:0/4102879661 shutdown_connections 2023-11-06T03:12:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.110+0000 7f6860a2e640 1 -- 172.21.15.157:0/4102879661 >> 172.21.15.157:0/4102879661 conn(0x7f685c065930 msgr2=0x7f685c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- 172.21.15.157:0/4102879661 shutdown_connections 2023-11-06T03:12:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- 172.21.15.157:0/4102879661 wait complete. 2023-11-06T03:12:38.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 Processor -- start 2023-11-06T03:12:38.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- start start 2023-11-06T03:12:38.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 0x7f685c150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f685c146f80 0x7f685c14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c14b2b0 con 0x7f685c1460f0 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c14a630 con 0x7f685c146f80 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c14a910 con 0x7f685c145d10 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46490/0 (socket says 172.21.15.157:46490) 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 -- 172.21.15.157:0/3361043329 learned_addr learned my addr 172.21.15.157:0/3361043329 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:38.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f685ad9d640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f685c146f80 0x7f685c14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f685a59c640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 0x7f685c150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 -- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 msgr2=0x7f685c150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 0x7f685c150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 -- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f685c146f80 msgr2=0x7f685c14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f685c146f80 0x7f685c14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 -- 172.21.15.157:0/3361043329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c14abf0 con 0x7f685c1460f0 2023-11-06T03:12:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f685a59c640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 0x7f685c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6859d9b640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6850011940 tx=0x7f6850011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850004240 con 0x7f685c1460f0 2023-11-06T03:12:38.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68500043a0 con 0x7f685c1460f0 2023-11-06T03:12:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000fa80 con 0x7f685c1460f0 2023-11-06T03:12:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c14aed0 con 0x7f685c1460f0 2023-11-06T03:12:38.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c146890 con 0x7f685c1460f0 2023-11-06T03:12:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.114+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c14b3f0 con 0x7f685c1460f0 2023-11-06T03:12:38.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.118+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f685000fbe0 con 0x7f685c1460f0 2023-11-06T03:12:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.118+0000 7f684affd640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 0x7f683c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.118+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6850083540 con 0x7f685c1460f0 2023-11-06T03:12:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.118+0000 7f685a59c640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 0x7f683c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.118+0000 7f685a59c640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 0x7f683c067f50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f684c0aa000 tx=0x7f684c001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.122+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f685c14b3f0 con 0x7f685c1460f0 2023-11-06T03:12:38.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.274+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f685c14b3f0 con 0x7f685c1460f0 2023-11-06T03:12:38.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.282+0000 7f684affd640 1 -- 172.21.15.157:0/3361043329 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f685c14b3f0 con 0x7f685c1460f0 2023-11-06T03:12:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 msgr2=0x7f683c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 0x7f683c067f50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f684c0aa000 tx=0x7f684c001a80 comp rx=0 tx=0).stop 2023-11-06T03:12:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 msgr2=0x7f685c151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6850011940 tx=0x7f6850011e00 comp rx=0 tx=0).stop 2023-11-06T03:12:38.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 shutdown_connections 2023-11-06T03:12:38.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f683c065a90 0x7f683c067f50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f685c146f80 0x7f685c14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f685c1460f0 0x7f685c151230 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 --2- 172.21.15.157:0/3361043329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f685c145d10 0x7f685c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 >> 172.21.15.157:0/3361043329 conn(0x7f685c065930 msgr2=0x7f685c15a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:38.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 shutdown_connections 2023-11-06T03:12:38.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.286+0000 7f6860a2e640 1 -- 172.21.15.157:0/3361043329 wait complete. 2023-11-06T03:12:38.303 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2023-11-06T03:12:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 Processor -- start 2023-11-06T03:12:38.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 -- start start 2023-11-06T03:12:38.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa0c1606f0 0x7ffa0c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c15e120 con 0x7ffa0c065930 2023-11-06T03:12:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c0d0590 con 0x7ffa0c1606f0 2023-11-06T03:12:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa1357e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c0cf7d0 con 0x7ffa0c0b0e10 2023-11-06T03:12:38.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa0c1606f0 0x7ffa0c160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa0c1606f0 0x7ffa0c160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46492/0 (socket says 172.21.15.157:46492) 2023-11-06T03:12:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 -- 172.21.15.157:0/566664186 learned_addr learned my addr 172.21.15.157:0/566664186 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa10b19640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 -- 172.21.15.157:0/566664186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 msgr2=0x7ffa0c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 -- 172.21.15.157:0/566664186 >> v1:172.21.15.195:6789/0 conn(0x7ffa0c065930 legacy=0x7ffa0c065d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.426+0000 7ffa11b1b640 1 -- 172.21.15.157:0/566664186 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c0cfab0 con 0x7ffa0c1606f0 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa11b1b640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c160b80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009e50 tx=0x7ffa0401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc0051cfc27737 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa03fff640 1 -- 172.21.15.157:0/566664186 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040a57c0 con 0x7ffa0c1606f0 2023-11-06T03:12:38.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa03fff640 1 -- 172.21.15.157:0/566664186 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa040a5920 con 0x7ffa0c1606f0 2023-11-06T03:12:38.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa03fff640 1 -- 172.21.15.157:0/566664186 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040a5cb0 con 0x7ffa0c1606f0 2023-11-06T03:12:38.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- 172.21.15.157:0/566664186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c1606f0 msgr2=0x7ffa0c160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c160b80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009e50 tx=0x7ffa0401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- 172.21.15.157:0/566664186 shutdown_connections 2023-11-06T03:12:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c160b80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- 172.21.15.157:0/566664186 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- 172.21.15.157:0/566664186 >> 172.21.15.157:0/566664186 conn(0x7ffa0c069d70 msgr2=0x7ffa0c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- 172.21.15.157:0/566664186 shutdown_connections 2023-11-06T03:12:38.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- 172.21.15.157:0/566664186 wait complete. 2023-11-06T03:12:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 Processor -- start 2023-11-06T03:12:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- start start 2023-11-06T03:12:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c157a10 con 0x7ffa0c065930 2023-11-06T03:12:38.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c14d530 con 0x7ffa0c1606f0 2023-11-06T03:12:38.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1357e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c156d90 con 0x7ffa0c0b0e10 2023-11-06T03:12:38.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1131a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1131a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46508/0 (socket says 172.21.15.157:46508) 2023-11-06T03:12:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1131a640 1 -- 172.21.15.157:0/2851226051 learned_addr learned my addr 172.21.15.157:0/2851226051 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa10b19640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa11b1b640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.430+0000 7ffa1131a640 1 -- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 msgr2=0x7ffa0c0cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1131a640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c0cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1131a640 1 -- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 msgr2=0x7ffa0c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1131a640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c0b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1131a640 1 -- 172.21.15.157:0/2851226051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009b30 con 0x7ffa0c065930 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa11b1b640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1131a640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800c3c0 tx=0x7ffa0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080052d0 con 0x7ffa0c065930 2023-11-06T03:12:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08005430 con 0x7ffa0c065930 2023-11-06T03:12:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c157070 con 0x7ffa0c065930 2023-11-06T03:12:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08003840 con 0x7ffa0c065930 2023-11-06T03:12:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c157630 con 0x7ffa0c065930 2023-11-06T03:12:38.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c072750 con 0x7ffa0c065930 2023-11-06T03:12:38.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffa080039a0 con 0x7ffa0c065930 2023-11-06T03:12:38.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 0x7ff9f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:38.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffa08082310 con 0x7ffa0c065930 2023-11-06T03:12:38.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa10b19640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 0x7ff9f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:38.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.434+0000 7ffa10b19640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 0x7ff9f0067f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c155510 tx=0x7ff9fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:38.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.438+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffa0804ff00 con 0x7ffa0c065930 2023-11-06T03:12:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.654+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7ffa0c15e120 con 0x7ffa0c065930 2023-11-06T03:12:38.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.938+0000 7ffa01ffb640 1 -- 172.21.15.157:0/2851226051 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v398) v1 ==== 144+0+0 (secure 0 0 0) 0x7ffa0c15e120 con 0x7ffa0c065930 2023-11-06T03:12:38.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 msgr2=0x7ff9f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 0x7ff9f0067f20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c155510 tx=0x7ff9fc008040 comp rx=0 tx=0).stop 2023-11-06T03:12:38.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 msgr2=0x7ffa0c14cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:38.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800c3c0 tx=0x7ffa0800c880 comp rx=0 tx=0).stop 2023-11-06T03:12:38.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 shutdown_connections 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9f0065a60 0x7ff9f0067f20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa0c1606f0 0x7ffa0c0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa0c0b0e10 0x7ffa0c0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 --2- 172.21.15.157:0/2851226051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa0c065930 0x7ffa0c14cff0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 >> 172.21.15.157:0/2851226051 conn(0x7ffa0c069d70 msgr2=0x7ffa0c15c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:38.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 shutdown_connections 2023-11-06T03:12:38.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:38.942+0000 7ffa1357e640 1 -- 172.21.15.157:0/2851226051 wait complete. 2023-11-06T03:12:38.962 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2023-11-06T03:12:38.963 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2023-11-06T03:12:38.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2023-11-06T03:12:39.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 Processor -- start 2023-11-06T03:12:39.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 -- start start 2023-11-06T03:12:39.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c141551e0 0x7f8c1406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 0x7f8c140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8c1415ea60 0x7f8c140b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1415f580 con 0x7f8c141551e0 2023-11-06T03:12:39.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1415efc0 con 0x7f8c1415ea60 2023-11-06T03:12:39.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c1a288640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1415f2a0 con 0x7f8c1415e5e0 2023-11-06T03:12:39.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c141551e0 0x7f8c1406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8c141551e0 0x7f8c1406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46114/0 (socket says 172.21.15.157:46114) 2023-11-06T03:12:39.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 -- 172.21.15.157:0/2001288400 learned_addr learned my addr 172.21.15.157:0/2001288400 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:39.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c18825640 1 --2- 172.21.15.157:0/2001288400 >> v2:172.21.15.195:3300/0 conn(0x7f8c1415ea60 0x7f8c140b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c137fe640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 0x7f8c140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 -- 172.21.15.157:0/2001288400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 msgr2=0x7f8c140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 0x7f8c140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 -- 172.21.15.157:0/2001288400 >> v2:172.21.15.195:3300/0 conn(0x7f8c1415ea60 msgr2=0x7f8c140b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 --2- 172.21.15.157:0/2001288400 >> v2:172.21.15.195:3300/0 conn(0x7f8c1415ea60 0x7f8c140b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c13fff640 1 -- 172.21.15.157:0/2001288400 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1406f5a0 con 0x7f8c141551e0 2023-11-06T03:12:39.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.090+0000 7f8c137fe640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 0x7f8c140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:39.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c13fff640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c1406fda0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c140cc5d0 tx=0x7f8c0401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d336eae2a815361d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:39.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c12ffd640 1 -- 172.21.15.157:0/2001288400 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040ab620 con 0x7f8c141551e0 2023-11-06T03:12:39.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c12ffd640 1 -- 172.21.15.157:0/2001288400 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c040ab780 con 0x7f8c141551e0 2023-11-06T03:12:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c12ffd640 1 -- 172.21.15.157:0/2001288400 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040abb10 con 0x7f8c141551e0 2023-11-06T03:12:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- 172.21.15.157:0/2001288400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 msgr2=0x7f8c1406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c1406fda0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8c140cc5d0 tx=0x7f8c0401c440 comp rx=0 tx=0).stop 2023-11-06T03:12:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- 172.21.15.157:0/2001288400 shutdown_connections 2023-11-06T03:12:39.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 --2- 172.21.15.157:0/2001288400 >> v2:172.21.15.195:3300/0 conn(0x7f8c1415ea60 0x7f8c140b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415e5e0 0x7f8c140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 --2- 172.21.15.157:0/2001288400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c1406fda0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- 172.21.15.157:0/2001288400 >> 172.21.15.157:0/2001288400 conn(0x7f8c14065930 msgr2=0x7f8c14065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:39.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- 172.21.15.157:0/2001288400 shutdown_connections 2023-11-06T03:12:39.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- 172.21.15.157:0/2001288400 wait complete. 2023-11-06T03:12:39.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 Processor -- start 2023-11-06T03:12:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.094+0000 7f8c1a288640 1 -- start start 2023-11-06T03:12:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c1415e5e0 0x7f8c1414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415ea60 0x7f8c1414c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414d8e0 con 0x7f8c141551e0 2023-11-06T03:12:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14144210 con 0x7f8c1415e5e0 2023-11-06T03:12:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414d450 con 0x7f8c1415ea60 2023-11-06T03:12:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46124/0 (socket says 172.21.15.157:46124) 2023-11-06T03:12:39.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 -- 172.21.15.157:0/4290554828 learned_addr learned my addr 172.21.15.157:0/4290554828 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c18825640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415ea60 0x7f8c1414c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c137fe640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c1415e5e0 0x7f8c1414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 -- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415ea60 msgr2=0x7f8c1414c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415ea60 0x7f8c1414c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 -- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c1415e5e0 msgr2=0x7f8c1414bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c1415e5e0 0x7f8c1414bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 -- 172.21.15.157:0/4290554828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040094d0 con 0x7f8c141551e0 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c13fff640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8c040ad040 tx=0x7f8c0401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040be3f0 con 0x7f8c141551e0 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c040be550 con 0x7f8c141551e0 2023-11-06T03:12:39.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140cde10 con 0x7f8c141551e0 2023-11-06T03:12:39.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040be900 con 0x7f8c141551e0 2023-11-06T03:12:39.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1414d0e0 con 0x7f8c141551e0 2023-11-06T03:12:39.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c14072750 con 0x7f8c141551e0 2023-11-06T03:12:39.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c040b43f0 con 0x7f8c141551e0 2023-11-06T03:12:39.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 0x7f8c00067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:39.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8c0411d2d0 con 0x7f8c141551e0 2023-11-06T03:12:39.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.098+0000 7f8c137fe640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 0x7f8c00067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:39.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.102+0000 7f8c137fe640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 0x7f8c00067f50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8c14165940 tx=0x7f8c08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:39.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.102+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8c040eb000 con 0x7f8c141551e0 2023-11-06T03:12:39.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.314+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f8c14071730 con 0x7f8c141551e0 2023-11-06T03:12:39.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.950+0000 7f8c10ff9640 1 -- 172.21.15.157:0/4290554828 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v399) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8c040ee620 con 0x7f8c141551e0 2023-11-06T03:12:39.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 msgr2=0x7f8c00067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:39.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 0x7f8c00067f50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8c14165940 tx=0x7f8c08008040 comp rx=0 tx=0).stop 2023-11-06T03:12:39.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 msgr2=0x7f8c140ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:39.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8c040ad040 tx=0x7f8c0401ac80 comp rx=0 tx=0).stop 2023-11-06T03:12:39.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 shutdown_connections 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c00065a90 0x7f8c00067f50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c1415ea60 0x7f8c1414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c1415e5e0 0x7f8c1414bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 --2- 172.21.15.157:0/4290554828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c141551e0 0x7f8c140ce8a0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 >> 172.21.15.157:0/4290554828 conn(0x7f8c14065930 msgr2=0x7f8c140b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:39.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 shutdown_connections 2023-11-06T03:12:39.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:39.954+0000 7f8c1a288640 1 -- 172.21.15.157:0/4290554828 wait complete. 2023-11-06T03:12:39.975 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:12:40.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.098+0000 7fdad6a9d640 1 Processor -- start 2023-11-06T03:12:40.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 -- start start 2023-11-06T03:12:40.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdad006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdad00b3b20 0x7fdad015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad015f580 con 0x7fdad015e5e0 2023-11-06T03:12:40.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad015efc0 con 0x7fdad00b3b20 2023-11-06T03:12:40.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad6a9d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad015f2a0 con 0x7fdad01551e0 2023-11-06T03:12:40.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdad015e5e0 0x7fdad00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57904/0 (socket says 172.21.15.157:57904) 2023-11-06T03:12:40.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacffff640 1 -- 172.21.15.157:0/3354027598 learned_addr learned my addr 172.21.15.157:0/3354027598 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:40.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 --2- 172.21.15.157:0/3354027598 >> v2:172.21.15.195:3300/0 conn(0x7fdad00b3b20 0x7fdad015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad4839640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdad006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611311798 0 0) 0x7fdad015f580 con 0x7fdad015e5e0 2023-11-06T03:12:40.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab8003530 con 0x7fdad015e5e0 2023-11-06T03:12:40.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18241719 0 0) 0x7fdab8003530 con 0x7fdad015e5e0 2023-11-06T03:12:40.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 msgr2=0x7fdad006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:40.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdad006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 >> v2:172.21.15.195:3300/0 conn(0x7fdad00b3b20 msgr2=0x7fdad015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 --2- 172.21.15.157:0/3354027598 >> v2:172.21.15.195:3300/0 conn(0x7fdad00b3b20 0x7fdad015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad006f5a0 con 0x7fdad015e5e0 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdac4001680 con 0x7fdad015e5e0 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 >> v1:172.21.15.157:6789/0 conn(0x7fdad015e5e0 legacy=0x7fdad00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdab8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8003f00 0x7fdab80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8004d10 0x7fdab8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad015efc0 con 0x7fdab8004d10 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad015f2a0 con 0x7fdab8003f00 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab800b9e0 con 0x7fdad01551e0 2023-11-06T03:12:40.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8004d10 0x7fdab8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacffff640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8003f00 0x7fdab80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad4839640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdab8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 -- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 msgr2=0x7fdab8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:40.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdab8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 -- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8003f00 msgr2=0x7fdab80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:40.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8003f00 0x7fdab80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad503a640 1 -- 172.21.15.157:0/3354027598 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8003530 con 0x7fdab8004d10 2023-11-06T03:12:40.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdad4839640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdad01551e0 0x7fdab8003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:40.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.102+0000 7fdacffff640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdab8003f00 0x7fdab80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:40.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8004d10 0x7fdab8005190 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009fc0 tx=0x7fdac801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0469c33165b0857 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80a5b60 con 0x7fdab8004d10 2023-11-06T03:12:40.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac800f040 con 0x7fdab8004d10 2023-11-06T03:12:40.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdacf7fe640 1 -- 172.21.15.157:0/3354027598 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80ad440 con 0x7fdab8004d10 2023-11-06T03:12:40.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3354027598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8004d10 msgr2=0x7fdab8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3354027598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab8004d10 0x7fdab8005190 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009fc0 tx=0x7fdac801cc60 comp rx=0 tx=0).stop 2023-11-06T03:12:40.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 -- 172.21.15.157:0/3354027598 reap_dead start 2023-11-06T03:12:40.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3354027598 shutdown_connections 2023-11-06T03:12:40.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3354027598 >> 172.21.15.157:0/3354027598 conn(0x7fdad0065930 msgr2=0x7fdad0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:40.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3354027598 shutdown_connections 2023-11-06T03:12:40.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3354027598 wait complete. 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 Processor -- start 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- start start 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab8004d10 0x7fdad00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 0x7fdad014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad014c4b0 con 0x7fdad01525a0 2023-11-06T03:12:40.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad014c790 con 0x7fdad0143730 2023-11-06T03:12:40.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad6a9d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad014ca70 con 0x7fdab8004d10 2023-11-06T03:12:40.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46152/0 (socket says 172.21.15.157:46152) 2023-11-06T03:12:40.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 -- 172.21.15.157:0/3277414040 learned_addr learned my addr 172.21.15.157:0/3277414040 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:40.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad4839640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab8004d10 0x7fdad00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 -- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab8004d10 msgr2=0x7fdad00ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdacffff640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 0x7fdad014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab8004d10 0x7fdad00ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 -- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 msgr2=0x7fdad014d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 0x7fdad014d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdad503a640 1 -- 172.21.15.157:0/3277414040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad014cd50 con 0x7fdad01525a0 2023-11-06T03:12:40.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.106+0000 7fdacffff640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 0x7fdad014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:40.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad503a640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdac8002240 tx=0x7fdac8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80039a0 con 0x7fdad01525a0 2023-11-06T03:12:40.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac8003b00 con 0x7fdad01525a0 2023-11-06T03:12:40.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad014d030 con 0x7fdad01525a0 2023-11-06T03:12:40.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003e70 con 0x7fdad01525a0 2023-11-06T03:12:40.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad0152a20 con 0x7fdad01525a0 2023-11-06T03:12:40.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad0152c10 con 0x7fdad01525a0 2023-11-06T03:12:40.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdac80b4040 con 0x7fdad01525a0 2023-11-06T03:12:40.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 0x7fdabc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdac811cda0 con 0x7fdad01525a0 2023-11-06T03:12:40.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad4839640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 0x7fdabc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.110+0000 7fdad4839640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 0x7fdabc067f20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdac00099e0 tx=0x7fdac0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.114+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdac80ea9b0 con 0x7fdad01525a0 2023-11-06T03:12:40.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.270+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdad0072750 con 0x7fdad01525a0 2023-11-06T03:12:40.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.278+0000 7fdacd7fa640 1 -- 172.21.15.157:0/3277414040 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdac80b20c0 con 0x7fdad01525a0 2023-11-06T03:12:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 msgr2=0x7fdabc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 0x7fdabc067f20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdac00099e0 tx=0x7fdac0008040 comp rx=0 tx=0).stop 2023-11-06T03:12:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 msgr2=0x7fdad006c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdac8002240 tx=0x7fdac8009750 comp rx=0 tx=0).stop 2023-11-06T03:12:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 shutdown_connections 2023-11-06T03:12:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdabc065a60 0x7fdabc067f20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad01525a0 0x7fdad006c3e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdad0143730 0x7fdad014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 --2- 172.21.15.157:0/3277414040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdab8004d10 0x7fdad00ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 >> 172.21.15.157:0/3277414040 conn(0x7fdad0065930 msgr2=0x7fdad00af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 shutdown_connections 2023-11-06T03:12:40.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.282+0000 7fdad6a9d640 1 -- 172.21.15.157:0/3277414040 wait complete. 2023-11-06T03:12:40.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:12:40.302 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:12:40.345 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:12:40.382 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:12:40.383 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:12:40.383 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:40.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:12:40.393 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:12:40.393 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:40.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:12:40.429 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:12:40.429 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:40.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:12:40.441 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:12:40.442 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:12:40.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:12:40.477 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:12:40.478 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:40.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 Processor -- start 2023-11-06T03:12:40.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 -- start start 2023-11-06T03:12:40.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec15ad30 0x7fdbec0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbec0b2f20 0x7fdbec0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0cef10 con 0x7fdbec0b23b0 2023-11-06T03:12:40.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec0cfb70 con 0x7fdbec0b2f20 2023-11-06T03:12:40.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf4391640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0cf160 con 0x7fdbec15ad30 2023-11-06T03:12:40.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf292e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdbec0b2f20 0x7fdbec0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf192c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdbec0b23b0 0x7fdbec0b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57908/0 (socket says 172.21.15.157:57908) 2023-11-06T03:12:40.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf192c640 1 -- 172.21.15.157:0/320764724 learned_addr learned my addr 172.21.15.157:0/320764724 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf212d640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec15ad30 0x7fdbec0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf292e640 1 -- 172.21.15.157:0/320764724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec15ad30 msgr2=0x7fdbec0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf292e640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec15ad30 0x7fdbec0b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf292e640 1 -- 172.21.15.157:0/320764724 >> v1:172.21.15.157:6789/0 conn(0x7fdbec0b23b0 legacy=0x7fdbec0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.610+0000 7fdbf292e640 1 -- 172.21.15.157:0/320764724 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0cf440 con 0x7fdbec0b2f20 2023-11-06T03:12:40.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf292e640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec0b2f20 0x7fdbec0d0000 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c21acecbd723df server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf112b640 1 -- 172.21.15.157:0/320764724 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40a5740 con 0x7fdbec0b2f20 2023-11-06T03:12:40.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf112b640 1 -- 172.21.15.157:0/320764724 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40a58a0 con 0x7fdbec0b2f20 2023-11-06T03:12:40.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf112b640 1 -- 172.21.15.157:0/320764724 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40a5c30 con 0x7fdbec0b2f20 2023-11-06T03:12:40.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- 172.21.15.157:0/320764724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec0b2f20 msgr2=0x7fdbec0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec0b2f20 0x7fdbec0d0000 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe401c720 comp rx=0 tx=0).stop 2023-11-06T03:12:40.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- 172.21.15.157:0/320764724 shutdown_connections 2023-11-06T03:12:40.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec0b2f20 0x7fdbec0d0000 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- 172.21.15.157:0/320764724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec15ad30 0x7fdbec0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- 172.21.15.157:0/320764724 >> 172.21.15.157:0/320764724 conn(0x7fdbec065d30 msgr2=0x7fdbec066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:40.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- 172.21.15.157:0/320764724 shutdown_connections 2023-11-06T03:12:40.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- 172.21.15.157:0/320764724 wait complete. 2023-11-06T03:12:40.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 Processor -- start 2023-11-06T03:12:40.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- start start 2023-11-06T03:12:40.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 0x7fdbec15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbec0b2f20 0x7fdbec1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec1599f0 con 0x7fdbec15ad30 2023-11-06T03:12:40.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec1552e0 con 0x7fdbec0b2f20 2023-11-06T03:12:40.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.614+0000 7fdbf4391640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec15c8e0 con 0x7fdbec0b23b0 2023-11-06T03:12:40.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46162/0 (socket says 172.21.15.157:46162) 2023-11-06T03:12:40.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 -- 172.21.15.157:0/2299456101 learned_addr learned my addr 172.21.15.157:0/2299456101 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:40.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf212d640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 0x7fdbec15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf192c640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbec0b2f20 0x7fdbec1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 -- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 msgr2=0x7fdbec15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 0x7fdbec15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 -- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbec0b2f20 msgr2=0x7fdbec1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbec0b2f20 0x7fdbec1594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 -- 172.21.15.157:0/2299456101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4009a70 con 0x7fdbec15ad30 2023-11-06T03:12:40.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf212d640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 0x7fdbec15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf292e640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdbe401add0 tx=0x7fdbe40194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40a5740 con 0x7fdbec15ad30 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40ad650 con 0x7fdbec15ad30 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec15cbc0 con 0x7fdbec15ad30 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40ad9e0 con 0x7fdbec15ad30 2023-11-06T03:12:40.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec15d180 con 0x7fdbec15ad30 2023-11-06T03:12:40.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec072750 con 0x7fdbec15ad30 2023-11-06T03:12:40.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdbe40adb40 con 0x7fdbec15ad30 2023-11-06T03:12:40.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 0x7fdbd0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:40.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdbe411e4a0 con 0x7fdbec15ad30 2023-11-06T03:12:40.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf212d640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 0x7fdbd0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:40.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.618+0000 7fdbf212d640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 0x7fdbd0067f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800a990 tx=0x7fdbe8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:40.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.622+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdbe40ec090 con 0x7fdbec15ad30 2023-11-06T03:12:40.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.810+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbec153630 con 0x7fdbec15ad30 2023-11-06T03:12:40.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbe2ffd640 1 -- 172.21.15.157:0/2299456101 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fdbe40c6df0 con 0x7fdbec15ad30 2023-11-06T03:12:40.813 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:40.813 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1832111112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1832111112},{"type":"v1","addr":"172.21.15.157:6833","nonce":1832111112}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1721015555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1721015555},{"type":"v1","addr":"172.21.15.195:6835","nonce":1721015555}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":6367,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1944540886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1944540886},{"type":"v1","addr":"172.21.15.195:6837","nonce":1944540886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":8163,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/985139869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":985139869},{"type":"v1","addr":"172.21.15.157:6835","nonce":985139869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[]} 2023-11-06T03:12:40.813 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 81 2023-11-06T03:12:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 msgr2=0x7fdbd0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 0x7fdbd0067f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800a990 tx=0x7fdbe8008040 comp rx=0 tx=0).stop 2023-11-06T03:12:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 msgr2=0x7fdbec0c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:40.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdbe401add0 tx=0x7fdbe40194c0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 shutdown_connections 2023-11-06T03:12:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdbd0065a60 0x7fdbd0067f20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdbec15ad30 0x7fdbec0c6970 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdbec0b2f20 0x7fdbec1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 --2- 172.21.15.157:0/2299456101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdbec0b23b0 0x7fdbec15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:40.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 >> 172.21.15.157:0/2299456101 conn(0x7fdbec065d30 msgr2=0x7fdbec066110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:40.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 shutdown_connections 2023-11-06T03:12:40.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:40.814+0000 7fdbf4391640 1 -- 172.21.15.157:0/2299456101 wait complete. 2023-11-06T03:12:40.834 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:12:40.834 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:12:40.834 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:12:40.835 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:12:40.835 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:12:40.835 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:12:40.836 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:12:40.836 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:12:40.836 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:12:40.837 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:12:40.837 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:12:40.837 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:12:40.837 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:12:40.838 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:12:40.838 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:12:40.838 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:12:40.845 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:12:40.846 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:12:40.847 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:12:40.847 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:12:40.854 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:12:40.854 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:12:40.857 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:12:40.857 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:12:40.889 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:12:40.889 DEBUG:teuthology.parallel:result is None 2023-11-06T03:12:40.890 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:12:40.890 DEBUG:teuthology.parallel:result is None 2023-11-06T03:12:40.901 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:12:40.901 DEBUG:teuthology.parallel:result is None 2023-11-06T03:12:40.903 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:12:40.903 DEBUG:teuthology.parallel:result is None 2023-11-06T03:12:40.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:12:40.952 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:12:40.953 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:12:40.970 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:12:40.971 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:12:41.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 Processor -- start 2023-11-06T03:12:41.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 -- start start 2023-11-06T03:12:41.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 0x7f72880cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72880d00b0 0x7f72880b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72880d2920 con 0x7f728815caa0 2023-11-06T03:12:41.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72880d0520 con 0x7f72880d00b0 2023-11-06T03:12:41.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728d106640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72880d18a0 con 0x7f728815ce80 2023-11-06T03:12:41.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728759e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72880d00b0 0x7f72880b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f7286d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f728815caa0 0x7f72880cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57946/0 (socket says 172.21.15.157:57946) 2023-11-06T03:12:41.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f7286d9d640 1 -- 172.21.15.157:0/3976559623 learned_addr learned my addr 172.21.15.157:0/3976559623 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:41.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728659c640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 0x7f72880cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728759e640 1 -- 172.21.15.157:0/3976559623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 msgr2=0x7f72880cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:41.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728759e640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 0x7f72880cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:41.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728759e640 1 -- 172.21.15.157:0/3976559623 >> v1:172.21.15.157:6789/0 conn(0x7f728815caa0 legacy=0x7f72880cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:41.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728759e640 1 -- 172.21.15.157:0/3976559623 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880d1b60 con 0x7f72880d00b0 2023-11-06T03:12:41.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.038+0000 7f728659c640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 0x7f72880cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:41.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728759e640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f72880b5d50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7280009d90 tx=0x7f728001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f16b0c419061508 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:41.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7285d9b640 1 -- 172.21.15.157:0/3976559623 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800a59e0 con 0x7f72880d00b0 2023-11-06T03:12:41.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7285d9b640 1 -- 172.21.15.157:0/3976559623 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72800a5b40 con 0x7f72880d00b0 2023-11-06T03:12:41.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7285d9b640 1 -- 172.21.15.157:0/3976559623 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800ad440 con 0x7f72880d00b0 2023-11-06T03:12:41.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- 172.21.15.157:0/3976559623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 msgr2=0x7f72880b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:41.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f72880b5d50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f7280009d90 tx=0x7f728001c920 comp rx=0 tx=0).stop 2023-11-06T03:12:41.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- 172.21.15.157:0/3976559623 shutdown_connections 2023-11-06T03:12:41.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f72880b5d50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- 172.21.15.157:0/3976559623 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815ce80 0x7f72880cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- 172.21.15.157:0/3976559623 >> 172.21.15.157:0/3976559623 conn(0x7f7288070790 msgr2=0x7f7288070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- 172.21.15.157:0/3976559623 shutdown_connections 2023-11-06T03:12:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- 172.21.15.157:0/3976559623 wait complete. 2023-11-06T03:12:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 Processor -- start 2023-11-06T03:12:41.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- start start 2023-11-06T03:12:41.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815caa0 0x7f7288154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 0x7f72880c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880c9630 con 0x7f72880d00b0 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880c7b90 con 0x7f728815ce80 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728d106640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72881542b0 con 0x7f728815caa0 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46220/0 (socket says 172.21.15.157:46220) 2023-11-06T03:12:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 -- 172.21.15.157:0/3979322253 learned_addr learned my addr 172.21.15.157:0/3979322253 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728759e640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 0x7f72880c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728659c640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815caa0 0x7f7288154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 -- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815caa0 msgr2=0x7f7288154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815caa0 0x7f7288154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 -- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 msgr2=0x7f72880c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 0x7f72880c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 -- 172.21.15.157:0/3979322253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280009a70 con 0x7f72880d00b0 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f728759e640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 0x7f72880c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.042+0000 7f7286d9d640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f727800c420 tx=0x7f727800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278005cf0 con 0x7f72880d00b0 2023-11-06T03:12:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278005e50 con 0x7f72880d00b0 2023-11-06T03:12:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72780072f0 con 0x7f72880d00b0 2023-11-06T03:12:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7288153980 con 0x7f72880d00b0 2023-11-06T03:12:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7288153f40 con 0x7f72880d00b0 2023-11-06T03:12:41.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7288072750 con 0x7f72880d00b0 2023-11-06T03:12:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7278007450 con 0x7f72880d00b0 2023-11-06T03:12:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 0x7f7274067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7278082480 con 0x7f72880d00b0 2023-11-06T03:12:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f728659c640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 0x7f7274067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:41.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.046+0000 7f728659c640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 0x7f7274067f50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f72880b7ad0 tx=0x7f727c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:41.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.050+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7278050070 con 0x7f72880d00b0 2023-11-06T03:12:41.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:41.218+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f72881540d0 con 0x7f72880d00b0 2023-11-06T03:12:41.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:12:41.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:12:42.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.022+0000 7f726f7fe640 1 -- 172.21.15.157:0/3979322253 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7278017080 con 0x7f72880d00b0 2023-11-06T03:12:42.024 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:12:42.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 msgr2=0x7f7274067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 0x7f7274067f50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f72880b7ad0 tx=0x7f727c008040 comp rx=0 tx=0).stop 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 msgr2=0x7f7288154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f727800c420 tx=0x7f727800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 shutdown_connections 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7274065a90 0x7f7274067f50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f728815ce80 0x7f72880c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f728815caa0 0x7f7288154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 --2- 172.21.15.157:0/3979322253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72880d00b0 0x7f7288154d90 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 >> 172.21.15.157:0/3979322253 conn(0x7f7288070790 msgr2=0x7f72880cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 shutdown_connections 2023-11-06T03:12:42.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.026+0000 7f728d106640 1 -- 172.21.15.157:0/3979322253 wait complete. 2023-11-06T03:12:42.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:12:42.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.170+0000 7f3aa1664640 1 Processor -- start 2023-11-06T03:12:42.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.170+0000 7f3aa1664640 1 -- start start 2023-11-06T03:12:42.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3aa1664640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0cfc90 0x7f3a9c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3aa1664640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c15d210 con 0x7f3a9c15ce30 2023-11-06T03:12:42.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3aa1664640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0ce090 con 0x7f3a9c0b5f60 2023-11-06T03:12:42.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3aa1664640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0d17f0 con 0x7f3a9c0cfc90 2023-11-06T03:12:42.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3a9c15ce30 0x7f3a9c0cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57952/0 (socket says 172.21.15.157:57952) 2023-11-06T03:12:42.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9affd640 1 -- 172.21.15.157:0/1396119290 learned_addr learned my addr 172.21.15.157:0/1396119290 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:42.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9b7fe640 1 --2- 172.21.15.157:0/1396119290 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0cfc90 0x7f3a9c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804783399 0 0) 0x7f3a9c15d210 con 0x7f3a9c15ce30 2023-11-06T03:12:42.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a88003530 con 0x7f3a9c15ce30 2023-11-06T03:12:42.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477976063 0 0) 0x7f3a88003530 con 0x7f3a9c15ce30 2023-11-06T03:12:42.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0cfc90 msgr2=0x7f3a9c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 --2- 172.21.15.157:0/1396119290 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0cfc90 0x7f3a9c0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 >> v1:172.21.15.195:6789/0 conn(0x7f3a9c0b5f60 legacy=0x7f3a9c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0ce370 con 0x7f3a9c15ce30 2023-11-06T03:12:42.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3a8c001680 con 0x7f3a9c15ce30 2023-11-06T03:12:42.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 >> v1:172.21.15.157:6789/0 conn(0x7f3a9c15ce30 legacy=0x7f3a9c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88003980 0x7f3a88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 0x7f3a88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a88005190 0x7f3a8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0d17f0 con 0x7f3a880047d0 2023-11-06T03:12:42.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8800c130 con 0x7f3a88003980 2023-11-06T03:12:42.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8800c2f0 con 0x7f3a88005190 2023-11-06T03:12:42.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 0x7f3a88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9affd640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88003980 0x7f3a88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9b7fe640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a88005190 0x7f3a8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 -- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a88005190 msgr2=0x7f3a8800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a88005190 0x7f3a8800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 -- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88003980 msgr2=0x7f3a88003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a88003980 0x7f3a88003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a9b7fe640 1 -- 172.21.15.157:0/1396119290 reap_dead start 2023-11-06T03:12:42.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.174+0000 7f3a927fc640 1 -- 172.21.15.157:0/1396119290 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88003530 con 0x7f3a880047d0 2023-11-06T03:12:42.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a927fc640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 0x7f3a88004c50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3a84001780 tx=0x7f3a8401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e2832b24b30d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840052d0 con 0x7f3a880047d0 2023-11-06T03:12:42.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840a7050 con 0x7f3a880047d0 2023-11-06T03:12:42.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9a7fc640 1 -- 172.21.15.157:0/1396119290 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84003600 con 0x7f3a880047d0 2023-11-06T03:12:42.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 msgr2=0x7f3a88004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 0x7f3a88004c50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3a84001780 tx=0x7f3a8401c3e0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/1396119290 shutdown_connections 2023-11-06T03:12:42.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 --2- 172.21.15.157:0/1396119290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a880047d0 0x7f3a88004c50 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/1396119290 >> 172.21.15.157:0/1396119290 conn(0x7f3a9c070990 msgr2=0x7f3a9c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/1396119290 shutdown_connections 2023-11-06T03:12:42.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/1396119290 wait complete. 2023-11-06T03:12:42.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 Processor -- start 2023-11-06T03:12:42.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- start start 2023-11-06T03:12:42.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 0x7f3a9c0aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0cc000 0x7f3a9c0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0c8fe0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c14cc40 con 0x7f3a880047d0 2023-11-06T03:12:42.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c14c960 con 0x7f3a9c0cc000 2023-11-06T03:12:42.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46240/0 (socket says 172.21.15.157:46240) 2023-11-06T03:12:42.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 -- 172.21.15.157:0/3075068570 learned_addr learned my addr 172.21.15.157:0/3075068570 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:42.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a927fc640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0cc000 0x7f3a9c0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9affd640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 0x7f3a9c0aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 -- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0cc000 msgr2=0x7f3a9c0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0cc000 0x7f3a9c0b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 -- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 msgr2=0x7f3a9c0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 0x7f3a9c0aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 -- 172.21.15.157:0/3075068570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0a95a0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9affd640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 0x7f3a9c0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:42.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a9b7fe640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c161a50 tx=0x7f3a940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400e070 con 0x7f3a9c0cc480 2023-11-06T03:12:42.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a94011db0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c0a9880 con 0x7f3a9c0cc480 2023-11-06T03:12:42.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940039a0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.178+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c14bcc0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c072750 con 0x7f3a9c0cc480 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3a94003b00 con 0x7f3a9c0cc480 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3a93fff640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 0x7f3a7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a94083890 con 0x7f3a9c0cc480 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3a9affd640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 0x7f3a7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.182+0000 7f3a9affd640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 0x7f3a7c067f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c001ec0 tx=0x7f3a8c001d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.186+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3a940513a0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.394+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3a9c0a9d30 con 0x7f3a9c0cc480 2023-11-06T03:12:42.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.394+0000 7f3a93fff640 1 -- 172.21.15.157:0/3075068570 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3a940549c0 con 0x7f3a9c0cc480 2023-11-06T03:12:42.395 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:42.396 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:42.396 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:12:42.396 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.396 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:12:42.396 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.397 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.398 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.398 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.398 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:12:42.398 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:12:42.398 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:12:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:12:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.400 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.401 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.402 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:12:42.402 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:12:42.402 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.402 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.402 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.403 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:12:42.404 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:12:42.404 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.404 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.404 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.404 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:12:42.405 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.406 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.406 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.406 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.406 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.407 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.410 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.410 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:12:42.410 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:12:42.410 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.411 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.411 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.411 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.411 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.412 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.413 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.414 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.415 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.415 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.415 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:12:42.415 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:12:42.415 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.416 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:12:42.417 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:12:42.418 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.418 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.418 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.418 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:12:42.418 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:12:42.419 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.420 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.420 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:12:42.420 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:12:42.420 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.421 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.421 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:12:42.421 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:12:42.421 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:12:42.421 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.422 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.422 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.422 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:12:42.422 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:12:42.422 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.423 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.424 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.424 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:12:42.424 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:12:42.424 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.424 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.425 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.426 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:12:42.426 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:12:42.426 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.426 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:12:42.426 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:12:42.427 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.427 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:42.427 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:42.427 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:12:42.428 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.429 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:42.429 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:12:42.429 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:42.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 msgr2=0x7f3a7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 0x7f3a7c067f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c001ec0 tx=0x7f3a8c001d50 comp rx=0 tx=0).stop 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 msgr2=0x7f3a9c0b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c161a50 tx=0x7f3a940115a0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 shutdown_connections 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a7c065a60 0x7f3a7c067f20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0cc480 0x7f3a9c0b51d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0cc000 0x7f3a9c0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 --2- 172.21.15.157:0/3075068570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a880047d0 0x7f3a9c0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 >> 172.21.15.157:0/3075068570 conn(0x7f3a9c070990 msgr2=0x7f3a9c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 shutdown_connections 2023-11-06T03:12:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.398+0000 7f3aa1664640 1 -- 172.21.15.157:0/3075068570 wait complete. 2023-11-06T03:12:42.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:42.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 Processor -- start 2023-11-06T03:12:42.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 -- start start 2023-11-06T03:12:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540b2b10 0x7efe540ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efe540cd730 0x7efe540cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540cc2b0 con 0x7efe5415ac80 2023-11-06T03:12:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540b01a0 con 0x7efe540cd730 2023-11-06T03:12:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe5a60f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540cfb10 con 0x7efe540b2b10 2023-11-06T03:12:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe53fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efe5415ac80 0x7efe540b2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57962/0 (socket says 172.21.15.157:57962) 2023-11-06T03:12:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe53fff640 1 -- 172.21.15.157:0/2633564726 learned_addr learned my addr 172.21.15.157:0/2633564726 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe58bac640 1 --2- 172.21.15.157:0/2633564726 >> v2:172.21.15.157:3300/0 conn(0x7efe540cd730 0x7efe540cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe537fe640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540b2b10 0x7efe540ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe52ffd640 1 -- 172.21.15.157:0/2633564726 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666525372 0 0) 0x7efe540cc2b0 con 0x7efe5415ac80 2023-11-06T03:12:42.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe52ffd640 1 -- 172.21.15.157:0/2633564726 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3c003530 con 0x7efe5415ac80 2023-11-06T03:12:42.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe58bac640 1 -- 172.21.15.157:0/2633564726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540b2b10 msgr2=0x7efe540ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe58bac640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540b2b10 0x7efe540ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe58bac640 1 -- 172.21.15.157:0/2633564726 >> v1:172.21.15.157:6789/0 conn(0x7efe5415ac80 legacy=0x7efe540b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.554+0000 7efe58bac640 1 -- 172.21.15.157:0/2633564726 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540cfdf0 con 0x7efe540cd730 2023-11-06T03:12:42.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe58bac640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540cd730 0x7efe540cea90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009d90 tx=0x7efe4c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b732fe100f8008 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe52ffd640 1 -- 172.21.15.157:0/2633564726 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0a5a10 con 0x7efe540cd730 2023-11-06T03:12:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe52ffd640 1 -- 172.21.15.157:0/2633564726 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c0a5b70 con 0x7efe540cd730 2023-11-06T03:12:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe52ffd640 1 -- 172.21.15.157:0/2633564726 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0ad440 con 0x7efe540cd730 2023-11-06T03:12:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- 172.21.15.157:0/2633564726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540cd730 msgr2=0x7efe540cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540cd730 0x7efe540cea90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009d90 tx=0x7efe4c01c950 comp rx=0 tx=0).stop 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- 172.21.15.157:0/2633564726 shutdown_connections 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe540cd730 0x7efe540cea90 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- 172.21.15.157:0/2633564726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540b2b10 0x7efe540ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- 172.21.15.157:0/2633564726 >> 172.21.15.157:0/2633564726 conn(0x7efe5406d560 msgr2=0x7efe5406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- 172.21.15.157:0/2633564726 shutdown_connections 2023-11-06T03:12:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- 172.21.15.157:0/2633564726 wait complete. 2023-11-06T03:12:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 Processor -- start 2023-11-06T03:12:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- start start 2023-11-06T03:12:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe540b2b10 0x7efe54154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540cd730 0x7efe5414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5414d6c0 con 0x7efe5415ac80 2023-11-06T03:12:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b4d90 con 0x7efe540b2b10 2023-11-06T03:12:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe5a60f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b5070 con 0x7efe540cd730 2023-11-06T03:12:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe58bac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe58bac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46262/0 (socket says 172.21.15.157:46262) 2023-11-06T03:12:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.558+0000 7efe58bac640 1 -- 172.21.15.157:0/4292918291 learned_addr learned my addr 172.21.15.157:0/4292918291 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe537fe640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540cd730 0x7efe5414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe53fff640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe540b2b10 0x7efe54154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 -- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540cd730 msgr2=0x7efe5414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540cd730 0x7efe5414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 -- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe540b2b10 msgr2=0x7efe54154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe540b2b10 0x7efe54154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 -- 172.21.15.157:0/4292918291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c009a70 con 0x7efe5415ac80 2023-11-06T03:12:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe58bac640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b0040 tx=0x7efe4c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0add90 con 0x7efe5415ac80 2023-11-06T03:12:42.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c009450 con 0x7efe5415ac80 2023-11-06T03:12:42.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe540b5350 con 0x7efe5415ac80 2023-11-06T03:12:42.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0b7650 con 0x7efe5415ac80 2023-11-06T03:12:42.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe54154cd0 con 0x7efe5415ac80 2023-11-06T03:12:42.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5414d1b0 con 0x7efe5415ac80 2023-11-06T03:12:42.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efe4c0b4050 con 0x7efe5415ac80 2023-11-06T03:12:42.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 0x7efe40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efe4c11e0c0 con 0x7efe5415ac80 2023-11-06T03:12:42.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe53fff640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 0x7efe40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.562+0000 7efe53fff640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 0x7efe40067f20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efe440011f0 tx=0x7efe44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.566+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efe5414d1b0 con 0x7efe5415ac80 2023-11-06T03:12:42.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.718+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe5414d1b0 con 0x7efe5415ac80 2023-11-06T03:12:42.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe50ff9640 1 -- 172.21.15.157:0/4292918291 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+14221 (secure 0 0 0) 0x7efe5414d1b0 con 0x7efe5415ac80 2023-11-06T03:12:42.721 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:42.721 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":107,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:42.014337+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 msgr2=0x7efe40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 0x7efe40067f20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efe440011f0 tx=0x7efe44009040 comp rx=0 tx=0).stop 2023-11-06T03:12:42.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 msgr2=0x7efe5414e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b0040 tx=0x7efe4c0196f0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 shutdown_connections 2023-11-06T03:12:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe40065a60 0x7efe40067f20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe5415ac80 0x7efe5414e090 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe540cd730 0x7efe5414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 --2- 172.21.15.157:0/4292918291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe540b2b10 0x7efe54154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 >> 172.21.15.157:0/4292918291 conn(0x7efe5406d560 msgr2=0x7efe540b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 shutdown_connections 2023-11-06T03:12:42.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.722+0000 7efe5a60f640 1 -- 172.21.15.157:0/4292918291 wait complete. 2023-11-06T03:12:42.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:12:42.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:12:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 Processor -- start 2023-11-06T03:12:42.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 -- start start 2023-11-06T03:12:42.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34c80b2330 0x7f34c806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c806c490 con 0x7f34c80adae0 2023-11-06T03:12:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c806fb80 con 0x7f34c8156a70 2023-11-06T03:12:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34ce2e8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c806b950 con 0x7f34c80b2330 2023-11-06T03:12:42.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f34c80adae0 0x7f34c80b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57974/0 (socket says 172.21.15.157:57974) 2023-11-06T03:12:42.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 -- 172.21.15.157:0/1249446113 learned_addr learned my addr 172.21.15.157:0/1249446113 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34cc885640 1 --2- 172.21.15.157:0/1249446113 >> v2:172.21.15.195:3300/0 conn(0x7f34c80b2330 0x7f34c806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624094516 0 0) 0x7f34c806c490 con 0x7f34c80adae0 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b4003450 con 0x7f34c80adae0 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 194905224 0 0) 0x7f34b4003450 con 0x7f34c80adae0 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 >> v2:172.21.15.195:3300/0 conn(0x7f34c80b2330 msgr2=0x7f34c806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 --2- 172.21.15.157:0/1249446113 >> v2:172.21.15.195:3300/0 conn(0x7f34c80b2330 0x7f34c806c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 >> v1:172.21.15.195:6789/0 conn(0x7f34c8156a70 legacy=0x7f34c806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c806bb60 con 0x7f34c80adae0 2023-11-06T03:12:42.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f34b0001680 con 0x7f34c80adae0 2023-11-06T03:12:42.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 >> v1:172.21.15.157:6789/0 conn(0x7f34c80adae0 legacy=0x7f34c80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b4003a70 0x7f34b4003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 0x7f34b400ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b400b490 0x7f34b400d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c806b950 con 0x7f34b4008860 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400e230 con 0x7f34b400b490 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400e3f0 con 0x7f34b4003a70 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 0x7f34b400ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34cc885640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b400b490 0x7f34b400d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34c7fff640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b4003a70 0x7f34b4003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 -- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b4003a70 msgr2=0x7f34b4003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34b4003a70 0x7f34b4003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 -- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b400b490 msgr2=0x7f34b400d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b400b490 0x7f34b400d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34cc885640 1 -- 172.21.15.157:0/1249446113 reap_dead start 2023-11-06T03:12:42.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.866+0000 7f34bffff640 1 -- 172.21.15.157:0/1249446113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4003450 con 0x7f34b4008860 2023-11-06T03:12:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34bffff640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 0x7f34b400ac90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c240 tx=0x7f34b001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58cd1e60a444e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0005ae0 con 0x7f34b4008860 2023-11-06T03:12:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b00ab070 con 0x7f34b4008860 2023-11-06T03:12:42.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34c77fe640 1 -- 172.21.15.157:0/1249446113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007070 con 0x7f34b4008860 2023-11-06T03:12:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 msgr2=0x7f34b400ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 0x7f34b400ac90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f34b000c240 tx=0x7f34b001f370 comp rx=0 tx=0).stop 2023-11-06T03:12:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- 172.21.15.157:0/1249446113 shutdown_connections 2023-11-06T03:12:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/1249446113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b4008860 0x7f34b400ac90 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- 172.21.15.157:0/1249446113 >> 172.21.15.157:0/1249446113 conn(0x7f34c8065930 msgr2=0x7f34c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- 172.21.15.157:0/1249446113 shutdown_connections 2023-11-06T03:12:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- 172.21.15.157:0/1249446113 wait complete. 2023-11-06T03:12:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 Processor -- start 2023-11-06T03:12:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- start start 2023-11-06T03:12:42.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b4008860 0x7f34c814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34c8150ae0 0x7f34c8154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8153e00 con 0x7f34c8150680 2023-11-06T03:12:42.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8155540 con 0x7f34b4008860 2023-11-06T03:12:42.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34ce2e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80b00f0 con 0x7f34c8150ae0 2023-11-06T03:12:42.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46280/0 (socket says 172.21.15.157:46280) 2023-11-06T03:12:42.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34bffff640 1 -- 172.21.15.157:0/478123657 learned_addr learned my addr 172.21.15.157:0/478123657 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:42.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34cc885640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34c8150ae0 0x7f34c8154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.870+0000 7f34c7fff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b4008860 0x7f34c814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 -- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34c8150ae0 msgr2=0x7f34c8154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34c8150ae0 0x7f34c8154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 -- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b4008860 msgr2=0x7f34c814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b4008860 0x7f34c814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 -- 172.21.15.157:0/478123657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80b03d0 con 0x7f34c8150680 2023-11-06T03:12:42.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34bffff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f34b001f1a0 tx=0x7f34b4003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0001680 con 0x7f34c8150680 2023-11-06T03:12:42.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0007070 con 0x7f34c8150680 2023-11-06T03:12:42.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80b06b0 con 0x7f34c8150680 2023-11-06T03:12:42.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007440 con 0x7f34c8150680 2023-11-06T03:12:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c814ce80 con 0x7f34c8150680 2023-11-06T03:12:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80b0230 con 0x7f34c8150680 2023-11-06T03:12:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f34b00b9030 con 0x7f34c8150680 2023-11-06T03:12:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 0x7f34a8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:42.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f34b00b7070 con 0x7f34c8150680 2023-11-06T03:12:42.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c7fff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 0x7f34a8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:42.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.874+0000 7f34c7fff640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 0x7f34a8067f50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34b8000c10 tx=0x7f34b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:42.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:42.878+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f34c80b0230 con 0x7f34c8150680 2023-11-06T03:12:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.026+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34c80b0230 con 0x7f34c8150680 2023-11-06T03:12:43.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.026+0000 7f34c57fa640 1 -- 172.21.15.157:0/478123657 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34c80b0230 con 0x7f34c8150680 2023-11-06T03:12:43.027 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:12:43.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 msgr2=0x7f34a8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 0x7f34a8067f50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f34b8000c10 tx=0x7f34b8009040 comp rx=0 tx=0).stop 2023-11-06T03:12:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 msgr2=0x7f34c81549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f34b001f1a0 tx=0x7f34b4003700 comp rx=0 tx=0).stop 2023-11-06T03:12:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 shutdown_connections 2023-11-06T03:12:43.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34a8065a90 0x7f34a8067f50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34c8150ae0 0x7f34c8154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34c8150680 0x7f34c81549f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 --2- 172.21.15.157:0/478123657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b4008860 0x7f34c814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 >> 172.21.15.157:0/478123657 conn(0x7f34c8065930 msgr2=0x7f34c8158060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 shutdown_connections 2023-11-06T03:12:43.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.030+0000 7f34ce2e8640 1 -- 172.21.15.157:0/478123657 wait complete. 2023-11-06T03:12:43.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:12:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 Processor -- start 2023-11-06T03:12:43.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 -- start start 2023-11-06T03:12:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f341815ce80 0x7f34180cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180d00b0 0x7f34180b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34180d2920 con 0x7f341815caa0 2023-11-06T03:12:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34180d0520 con 0x7f341815ce80 2023-11-06T03:12:43.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f34206ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34180d18a0 con 0x7f34180d00b0 2023-11-06T03:12:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341e48b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f341815caa0 0x7f34180cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57990/0 (socket says 172.21.15.157:57990) 2023-11-06T03:12:43.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341e48b640 1 -- 172.21.15.157:0/946577526 learned_addr learned my addr 172.21.15.157:0/946577526 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851099184 0 0) 0x7f34180d2920 con 0x7f341815caa0 2023-11-06T03:12:43.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3404003530 con 0x7f341815caa0 2023-11-06T03:12:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180d00b0 0x7f34180b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341dc8a640 1 --2- 172.21.15.157:0/946577526 >> v2:172.21.15.195:3300/0 conn(0x7f341815ce80 0x7f34180cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 950214353 0 0) 0x7f3404003530 con 0x7f341815caa0 2023-11-06T03:12:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180d00b0 msgr2=0x7f34180b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180d00b0 0x7f34180b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 >> v2:172.21.15.195:3300/0 conn(0x7f341815ce80 msgr2=0x7f34180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 --2- 172.21.15.157:0/946577526 >> v2:172.21.15.195:3300/0 conn(0x7f341815ce80 0x7f34180cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34180d1b60 con 0x7f341815caa0 2023-11-06T03:12:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f34140016a0 con 0x7f341815caa0 2023-11-06T03:12:43.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 >> v1:172.21.15.157:6789/0 conn(0x7f341815caa0 legacy=0x7f34180cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404003980 0x7f3404003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34040047d0 0x7f3404004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3404005190 0x7f340400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34180d0520 con 0x7f3404005190 2023-11-06T03:12:43.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34180d18a0 con 0x7f3404003980 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340400bfd0 con 0x7f34040047d0 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180d00b0 0x7f34180b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341dc8a640 1 --2- 172.21.15.157:0/946577526 >> v2:172.21.15.195:3300/0 conn(0x7f341815ce80 0x7f34180cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3404005190 0x7f340400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341e48b640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404003980 0x7f3404003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341dc8a640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34040047d0 0x7f3404004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 -- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34040047d0 msgr2=0x7f3404004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34040047d0 0x7f3404004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 -- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404003980 msgr2=0x7f3404003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404003980 0x7f3404003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341ec8c640 1 -- 172.21.15.157:0/946577526 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404003530 con 0x7f3404005190 2023-11-06T03:12:43.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.166+0000 7f341e48b640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404003980 0x7f3404003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:43.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341ec8c640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3404005190 0x7f340400b990 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3410009f30 tx=0x7f341001ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9417bd9af51784c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100a5bc0 con 0x7f3404005190 2023-11-06T03:12:43.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34100a5d20 con 0x7f3404005190 2023-11-06T03:12:43.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341d489640 1 -- 172.21.15.157:0/946577526 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410003bc0 con 0x7f3404005190 2023-11-06T03:12:43.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- 172.21.15.157:0/946577526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3404005190 msgr2=0x7f340400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 --2- 172.21.15.157:0/946577526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3404005190 0x7f340400b990 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3410009f30 tx=0x7f341001ccb0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341ec8c640 1 -- 172.21.15.157:0/946577526 reap_dead start 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- 172.21.15.157:0/946577526 shutdown_connections 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- 172.21.15.157:0/946577526 >> 172.21.15.157:0/946577526 conn(0x7f3418070790 msgr2=0x7f3418070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- 172.21.15.157:0/946577526 shutdown_connections 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- 172.21.15.157:0/946577526 wait complete. 2023-11-06T03:12:43.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 Processor -- start 2023-11-06T03:12:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- start start 2023-11-06T03:12:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404005190 0x7f341815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180b3140 0x7f34181f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b35c0 con 0x7f341815a090 2023-11-06T03:12:43.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341815e210 con 0x7f3404005190 2023-11-06T03:12:43.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f34206ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341815e4e0 con 0x7f34180b3140 2023-11-06T03:12:43.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341dc8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341dc8a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46310/0 (socket says 172.21.15.157:46310) 2023-11-06T03:12:43.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.170+0000 7f341dc8a640 1 -- 172.21.15.157:0/4160390799 learned_addr learned my addr 172.21.15.157:0/4160390799 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341e48b640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404005190 0x7f341815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341ec8c640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180b3140 0x7f34181f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 -- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180b3140 msgr2=0x7f34181f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180b3140 0x7f34181f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 -- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404005190 msgr2=0x7f341815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404005190 0x7f341815dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 -- 172.21.15.157:0/4160390799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341815e7c0 con 0x7f341815a090 2023-11-06T03:12:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341dc8a640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f340800f550 tx=0x7f340800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408014890 con 0x7f341815a090 2023-11-06T03:12:43.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800c030 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341815eaa0 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408014c10 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341815f060 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34180cfcd0 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f34080133e0 con 0x7f341815a090 2023-11-06T03:12:43.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 0x7f33fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3408083450 con 0x7f341815a090 2023-11-06T03:12:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341e48b640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 0x7f33fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.174+0000 7f341e48b640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 0x7f33fc067f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34140017d0 tx=0x7f3414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.178+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3408050f60 con 0x7f341815a090 2023-11-06T03:12:43.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.318+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3418072750 con 0x7f341815a090 2023-11-06T03:12:43.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f340effd640 1 -- 172.21.15.157:0/4160390799 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3408054580 con 0x7f341815a090 2023-11-06T03:12:43.320 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:12:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 msgr2=0x7f33fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 0x7f33fc067f20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34140017d0 tx=0x7f3414009040 comp rx=0 tx=0).stop 2023-11-06T03:12:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 msgr2=0x7f34180b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f340800f550 tx=0x7f340800fa10 comp rx=0 tx=0).stop 2023-11-06T03:12:43.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 shutdown_connections 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33fc065a60 0x7f33fc067f20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34180b3140 0x7f34181f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f341815a090 0x7f34180b2c00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 --2- 172.21.15.157:0/4160390799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3404005190 0x7f341815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 >> 172.21.15.157:0/4160390799 conn(0x7f3418070790 msgr2=0x7f34180cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 shutdown_connections 2023-11-06T03:12:43.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.322+0000 7f34206ef640 1 -- 172.21.15.157:0/4160390799 wait complete. 2023-11-06T03:12:43.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:12:43.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 Processor -- start 2023-11-06T03:12:43.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 -- start start 2023-11-06T03:12:43.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8da40adae0 0x7f8da40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8da40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da406c490 con 0x7f8da4156a70 2023-11-06T03:12:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da406b950 con 0x7f8da40adae0 2023-11-06T03:12:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da8ffe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da406fb80 con 0x7f8da40adf40 2023-11-06T03:12:43.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8da4156a70 0x7f8da406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58000/0 (socket says 172.21.15.157:58000) 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da259c640 1 -- 172.21.15.157:0/826738145 learned_addr learned my addr 172.21.15.157:0/826738145 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8d9bfff640 1 --2- 172.21.15.157:0/826738145 >> v2:172.21.15.195:3300/0 conn(0x7f8da40adae0 0x7f8da40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8da40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326191696 0 0) 0x7f8da406c490 con 0x7f8da4156a70 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d90005740 con 0x7f8da4156a70 2023-11-06T03:12:43.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747334620 0 0) 0x7f8d90005740 con 0x7f8da4156a70 2023-11-06T03:12:43.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 msgr2=0x7f8da40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8da40b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 >> v2:172.21.15.195:3300/0 conn(0x7f8da40adae0 msgr2=0x7f8da40b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 --2- 172.21.15.157:0/826738145 >> v2:172.21.15.195:3300/0 conn(0x7f8da40adae0 0x7f8da40b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da406bb60 con 0x7f8da4156a70 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8d94001680 con 0x7f8da4156a70 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 >> v1:172.21.15.157:6789/0 conn(0x7f8da4156a70 legacy=0x7f8da406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d90005b90 0x7f8d90005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8d900069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90006f20 0x7f8d900073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da406b950 con 0x7f8d90006f20 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da406fb80 con 0x7f8d90005b90 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9000dba0 con 0x7f8da40adf40 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90006f20 0x7f8d900073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8d9bfff640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8d900069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da259c640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d90005b90 0x7f8d90005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 -- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 msgr2=0x7f8d900069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8da40adf40 0x7f8d900069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 -- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d90005b90 msgr2=0x7f8d90005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d90005b90 0x7f8d90005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.454+0000 7f8da2d9d640 1 -- 172.21.15.157:0/826738145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90005740 con 0x7f8d90006f20 2023-11-06T03:12:43.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90006f20 0x7f8d900073a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009fc0 tx=0x7f8d9c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335e54a81e97288d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a5c70 con 0x7f8d90006f20 2023-11-06T03:12:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c00f040 con 0x7f8d90006f20 2023-11-06T03:12:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da1d9b640 1 -- 172.21.15.157:0/826738145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0ad440 con 0x7f8d90006f20 2023-11-06T03:12:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- 172.21.15.157:0/826738145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90006f20 msgr2=0x7f8d900073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/826738145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d90006f20 0x7f8d900073a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009fc0 tx=0x7f8d9c01cc60 comp rx=0 tx=0).stop 2023-11-06T03:12:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da2d9d640 1 -- 172.21.15.157:0/826738145 reap_dead start 2023-11-06T03:12:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- 172.21.15.157:0/826738145 shutdown_connections 2023-11-06T03:12:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- 172.21.15.157:0/826738145 >> 172.21.15.157:0/826738145 conn(0x7f8da4065930 msgr2=0x7f8da4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- 172.21.15.157:0/826738145 shutdown_connections 2023-11-06T03:12:43.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- 172.21.15.157:0/826738145 wait complete. 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 Processor -- start 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- start start 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d90006f20 0x7f8da413bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da4153090 0x7f8da413e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da41549f0 con 0x7f8da413e590 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da413ead0 con 0x7f8da4153090 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.458+0000 7f8da8ffe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da413edb0 con 0x7f8d90006f20 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46322/0 (socket says 172.21.15.157:46322) 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 -- 172.21.15.157:0/4106930321 learned_addr learned my addr 172.21.15.157:0/4106930321 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:43.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da259c640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d90006f20 0x7f8da413bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9bfff640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da4153090 0x7f8da413e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 -- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d90006f20 msgr2=0x7f8da413bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d90006f20 0x7f8da413bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 -- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da4153090 msgr2=0x7f8da413e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da4153090 0x7f8da413e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 -- 172.21.15.157:0/4106930321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4154c70 con 0x7f8da413e590 2023-11-06T03:12:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da2d9d640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0b0040 tx=0x7f8d9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0bc070 con 0x7f8da413e590 2023-11-06T03:12:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9c0039a0 con 0x7f8da413e590 2023-11-06T03:12:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c003b30 con 0x7f8da413e590 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da4154f50 con 0x7f8da413e590 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da414d7e0 con 0x7f8da413e590 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da4072750 con 0x7f8da413e590 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d9c0b4040 con 0x7f8da413e590 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 0x7f8d84067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8d9c11d140 con 0x7f8da413e590 2023-11-06T03:12:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.462+0000 7f8da259c640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 0x7f8d84067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.466+0000 7f8da259c640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 0x7f8d84067f50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c10 tx=0x7f8d94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.466+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d9c0ead10 con 0x7f8da413e590 2023-11-06T03:12:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.622+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8da415cfd0 con 0x7f8da413e590 2023-11-06T03:12:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.622+0000 7f8d9affd640 1 -- 172.21.15.157:0/4106930321 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8d9c0b2090 con 0x7f8da413e590 2023-11-06T03:12:43.622 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:43.622 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:12:43.622 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:43.622 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:12:43.623 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:12:43.623 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:43.623 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:12:43.623 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:12:43.623 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:12:43.624 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:43.624 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:12:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 msgr2=0x7f8d84067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 0x7f8d84067f50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c10 tx=0x7f8d94009040 comp rx=0 tx=0).stop 2023-11-06T03:12:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 msgr2=0x7f8da41502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0b0040 tx=0x7f8d9c009750 comp rx=0 tx=0).stop 2023-11-06T03:12:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 shutdown_connections 2023-11-06T03:12:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d84065a90 0x7f8d84067f50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8da413e590 0x7f8da41502c0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8da4153090 0x7f8da413e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 --2- 172.21.15.157:0/4106930321 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d90006f20 0x7f8da413bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 >> 172.21.15.157:0/4106930321 conn(0x7f8da4065930 msgr2=0x7f8da406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 shutdown_connections 2023-11-06T03:12:43.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.626+0000 7f8da8ffe640 1 -- 172.21.15.157:0/4106930321 wait complete. 2023-11-06T03:12:43.640 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:12:43.640 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:12:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 Processor -- start 2023-11-06T03:12:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 -- start start 2023-11-06T03:12:43.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c06a6f0 0x7f411c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f411c0ce040 0x7f411c161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c06db70 con 0x7f411c0cdbe0 2023-11-06T03:12:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c06b010 con 0x7f411c0ce040 2023-11-06T03:12:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f412442c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c06eeb0 con 0x7f411c06a6f0 2023-11-06T03:12:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f411c0ce040 0x7f411c161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f411c0ce040 0x7f411c161220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46334/0 (socket says 172.21.15.157:46334) 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 -- 172.21.15.157:0/70179741 learned_addr learned my addr 172.21.15.157:0/70179741 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41221c8640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c06a6f0 0x7f411c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 -- 172.21.15.157:0/70179741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c06a6f0 msgr2=0x7f411c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c06a6f0 0x7f411c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 -- 172.21.15.157:0/70179741 >> v1:172.21.15.195:6789/0 conn(0x7f411c0cdbe0 legacy=0x7f411c160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.754+0000 7f41229c9640 1 -- 172.21.15.157:0/70179741 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c06de10 con 0x7f411c0ce040 2023-11-06T03:12:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41229c9640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c0ce040 0x7f411c161220 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4114009e50 tx=0x7f411401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6a55af14cb7c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41211c6640 1 -- 172.21.15.157:0/70179741 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140a58a0 con 0x7f411c0ce040 2023-11-06T03:12:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41211c6640 1 -- 172.21.15.157:0/70179741 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140a5a00 con 0x7f411c0ce040 2023-11-06T03:12:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41211c6640 1 -- 172.21.15.157:0/70179741 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140a5d90 con 0x7f411c0ce040 2023-11-06T03:12:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- 172.21.15.157:0/70179741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c0ce040 msgr2=0x7f411c161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c0ce040 0x7f411c161220 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4114009e50 tx=0x7f411401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- 172.21.15.157:0/70179741 shutdown_connections 2023-11-06T03:12:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c0ce040 0x7f411c161220 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- 172.21.15.157:0/70179741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c06a6f0 0x7f411c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- 172.21.15.157:0/70179741 >> 172.21.15.157:0/70179741 conn(0x7f411c0d0950 msgr2=0x7f411c0d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- 172.21.15.157:0/70179741 shutdown_connections 2023-11-06T03:12:43.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- 172.21.15.157:0/70179741 wait complete. 2023-11-06T03:12:43.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 Processor -- start 2023-11-06T03:12:43.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- start start 2023-11-06T03:12:43.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c0cdbe0 0x7f411c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411c0ce040 0x7f411c14b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0c70b0 con 0x7f411c06a6f0 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0c74d0 con 0x7f411c0ce040 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f412442c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0c6430 con 0x7f411c0cdbe0 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41221c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41221c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46338/0 (socket says 172.21.15.157:46338) 2023-11-06T03:12:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.758+0000 7f41221c8640 1 -- 172.21.15.157:0/2740698207 learned_addr learned my addr 172.21.15.157:0/2740698207 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:43.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41219c7640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c0cdbe0 0x7f411c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41229c9640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411c0ce040 0x7f411c14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 -- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c0cdbe0 msgr2=0x7f411c0c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c0cdbe0 0x7f411c0c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 -- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411c0ce040 msgr2=0x7f411c14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411c0ce040 0x7f411c14b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 -- 172.21.15.157:0/2740698207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009b30 con 0x7f411c06a6f0 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41221c8640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f411800e8a0 tx=0x7f411800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118009510 con 0x7f411c06a6f0 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411800c940 con 0x7f411c06a6f0 2023-11-06T03:12:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800ccb0 con 0x7f411c06a6f0 2023-11-06T03:12:43.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c0c6710 con 0x7f411c06a6f0 2023-11-06T03:12:43.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c0c6cd0 con 0x7f411c06a6f0 2023-11-06T03:12:43.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0c7320 con 0x7f411c06a6f0 2023-11-06T03:12:43.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4118013470 con 0x7f411c06a6f0 2023-11-06T03:12:43.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 0x7f4100067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:43.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41180830d0 con 0x7f411c06a6f0 2023-11-06T03:12:43.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41219c7640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 0x7f4100067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.762+0000 7f41219c7640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 0x7f4100067f20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f411c155930 tx=0x7f410c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.766+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4118050c80 con 0x7f411c06a6f0 2023-11-06T03:12:43.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.910+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f411c08b080 con 0x7f411c06a6f0 2023-11-06T03:12:43.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.974+0000 7f4112ffd640 1 -- 172.21.15.157:0/2740698207 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) v1 ==== 138+0+0 (secure 0 0 0) 0x7f41180542a0 con 0x7f411c06a6f0 2023-11-06T03:12:43.974 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:12:43.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 msgr2=0x7f4100067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 0x7f4100067f20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f411c155930 tx=0x7f410c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 msgr2=0x7f411c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f411800e8a0 tx=0x7f411800eca0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 shutdown_connections 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4100065a60 0x7f4100067f20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f411c0ce040 0x7f411c14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f411c0cdbe0 0x7f411c0c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 --2- 172.21.15.157:0/2740698207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411c06a6f0 0x7f411c0cfb00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 >> 172.21.15.157:0/2740698207 conn(0x7f411c0d0950 msgr2=0x7f411c0d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 shutdown_connections 2023-11-06T03:12:43.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:43.978+0000 7f412442c640 1 -- 172.21.15.157:0/2740698207 wait complete. 2023-11-06T03:12:43.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:12:44.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 Processor -- start 2023-11-06T03:12:44.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 -- start start 2023-11-06T03:12:44.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3f1015e090 0x7f3f100cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f100719e0 0x7f3f100cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 0x7f3f100cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f100cc420 con 0x7f3f100719e0 2023-11-06T03:12:44.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f100d0380 con 0x7f3f1015e090 2023-11-06T03:12:44.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f16ee0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100cc690 con 0x7f3f100cce90 2023-11-06T03:12:44.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f100719e0 0x7f3f100cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3f100719e0 0x7f3f100cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46348/0 (socket says 172.21.15.157:46348) 2023-11-06T03:12:44.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 -- 172.21.15.157:0/3732135695 learned_addr learned my addr 172.21.15.157:0/3732135695 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:44.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f1547d640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 0x7f3f100cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f14c7c640 1 --2- 172.21.15.157:0/3732135695 >> v2:172.21.15.195:3300/0 conn(0x7f3f1015e090 0x7f3f100cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 -- 172.21.15.157:0/3732135695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 msgr2=0x7f3f100cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 0x7f3f100cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 -- 172.21.15.157:0/3732135695 >> v2:172.21.15.195:3300/0 conn(0x7f3f1015e090 msgr2=0x7f3f100cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 --2- 172.21.15.157:0/3732135695 >> v2:172.21.15.195:3300/0 conn(0x7f3f1015e090 0x7f3f100cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f0ffff640 1 -- 172.21.15.157:0/3732135695 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100cc970 con 0x7f3f100719e0 2023-11-06T03:12:44.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.122+0000 7f3f1547d640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 0x7f3f100cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:44.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f0ffff640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f100719e0 0x7f3f100cd320 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3f100cd2f0 tx=0x7f3ef801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317fe3fbc0a290eb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:44.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/3732135695 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80a89b0 con 0x7f3f100719e0 2023-11-06T03:12:44.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/3732135695 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80a8b10 con 0x7f3f100719e0 2023-11-06T03:12:44.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f0f7fe640 1 -- 172.21.15.157:0/3732135695 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80a8ea0 con 0x7f3f100719e0 2023-11-06T03:12:44.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- 172.21.15.157:0/3732135695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f100719e0 msgr2=0x7f3f100cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:44.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f100719e0 0x7f3f100cd320 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f3f100cd2f0 tx=0x7f3ef801ecf0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- 172.21.15.157:0/3732135695 shutdown_connections 2023-11-06T03:12:44.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100cce90 0x7f3f100cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/3732135695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f100719e0 0x7f3f100cd320 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/3732135695 >> v2:172.21.15.195:3300/0 conn(0x7f3f1015e090 0x7f3f100cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- 172.21.15.157:0/3732135695 >> 172.21.15.157:0/3732135695 conn(0x7f3f1006d3d0 msgr2=0x7f3f1006d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- 172.21.15.157:0/3732135695 shutdown_connections 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- 172.21.15.157:0/3732135695 wait complete. 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 Processor -- start 2023-11-06T03:12:44.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- start start 2023-11-06T03:12:44.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100719e0 0x7f3f100b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f100cce90 0x7f3f100c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100b3880 con 0x7f3f1015e090 2023-11-06T03:12:44.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100b3b60 con 0x7f3f100cce90 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f16ee0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100b3e40 con 0x7f3f100719e0 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46362/0 (socket says 172.21.15.157:46362) 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 -- 172.21.15.157:0/4261468087 learned_addr learned my addr 172.21.15.157:0/4261468087 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f14c7c640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100719e0 0x7f3f100b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 -- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100719e0 msgr2=0x7f3f100b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:44.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100719e0 0x7f3f100b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f0ffff640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f100cce90 0x7f3f100c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 -- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f100cce90 msgr2=0x7f3f100c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:44.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f100cce90 0x7f3f100c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:44.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.126+0000 7f3f1547d640 1 -- 172.21.15.157:0/4261468087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80094d0 con 0x7f3f1015e090 2023-11-06T03:12:44.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f1547d640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c7f0 tx=0x7f3f0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:44.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00007820 con 0x7f3f1015e090 2023-11-06T03:12:44.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f00007980 con 0x7f3f1015e090 2023-11-06T03:12:44.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f100b4120 con 0x7f3f1015e090 2023-11-06T03:12:44.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00007d70 con 0x7f3f1015e090 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1014d3a0 con 0x7f3f1015e090 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f100723d0 con 0x7f3f1015e090 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3f000113f0 con 0x7f3f1015e090 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 0x7f3ef0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3f00083510 con 0x7f3f1015e090 2023-11-06T03:12:44.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f14c7c640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 0x7f3ef0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:44.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.130+0000 7f3f14c7c640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 0x7f3ef0067f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3f08008ed0 tx=0x7f3f08008420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:44.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.134+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3f00050ec0 con 0x7f3f1015e090 2023-11-06T03:12:44.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.278+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3f10072750 con 0x7f3f1015e090 2023-11-06T03:12:44.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:44.994+0000 7f3f0d7fa640 1 -- 172.21.15.157:0/4261468087 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3f000544e0 con 0x7f3f1015e090 2023-11-06T03:12:44.993 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:12:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 msgr2=0x7f3ef0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 0x7f3ef0067f20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3f08008ed0 tx=0x7f3f08008420 comp rx=0 tx=0).stop 2023-11-06T03:12:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 msgr2=0x7f3f100c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c7f0 tx=0x7f3f0000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 shutdown_connections 2023-11-06T03:12:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ef0065a60 0x7f3ef0067f20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f1015e090 0x7f3f100c9b70 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f100cce90 0x7f3f100c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 --2- 172.21.15.157:0/4261468087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3f100719e0 0x7f3f100b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 >> 172.21.15.157:0/4261468087 conn(0x7f3f1006d3d0 msgr2=0x7f3f100cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.014+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 shutdown_connections 2023-11-06T03:12:45.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.018+0000 7f3f16ee0640 1 -- 172.21.15.157:0/4261468087 wait complete. 2023-11-06T03:12:45.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:12:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 Processor -- start 2023-11-06T03:12:45.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 -- start start 2023-11-06T03:12:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15ce80 0x7f917c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f917c0d00b0 0x7f917c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0d2920 con 0x7f917c15caa0 2023-11-06T03:12:45.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0d0520 con 0x7f917c0d00b0 2023-11-06T03:12:45.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9183948640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0d18a0 con 0x7f917c15ce80 2023-11-06T03:12:45.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f917c0d00b0 0x7f917c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f917c0d00b0 0x7f917c0b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46368/0 (socket says 172.21.15.157:46368) 2023-11-06T03:12:45.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 -- 172.21.15.157:0/3324498492 learned_addr learned my addr 172.21.15.157:0/3324498492 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:45.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9180ee3640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15ce80 0x7f917c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 -- 172.21.15.157:0/3324498492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15ce80 msgr2=0x7f917c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:45.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15ce80 0x7f917c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 -- 172.21.15.157:0/3324498492 >> v1:172.21.15.195:6789/0 conn(0x7f917c15caa0 legacy=0x7f917c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.166+0000 7f9181ee5640 1 -- 172.21.15.157:0/3324498492 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0d1b60 con 0x7f917c0d00b0 2023-11-06T03:12:45.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9181ee5640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c0d00b0 0x7f917c0b5d50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e50 tx=0x7f917401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980f6ed6da3076ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9173fff640 1 -- 172.21.15.157:0/3324498492 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740a58a0 con 0x7f917c0d00b0 2023-11-06T03:12:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9173fff640 1 -- 172.21.15.157:0/3324498492 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91740a5a00 con 0x7f917c0d00b0 2023-11-06T03:12:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9173fff640 1 -- 172.21.15.157:0/3324498492 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740a5d90 con 0x7f917c0d00b0 2023-11-06T03:12:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- 172.21.15.157:0/3324498492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c0d00b0 msgr2=0x7f917c0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c0d00b0 0x7f917c0b5d50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e50 tx=0x7f917401c9e0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- 172.21.15.157:0/3324498492 shutdown_connections 2023-11-06T03:12:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c0d00b0 0x7f917c0b5d50 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 --2- 172.21.15.157:0/3324498492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15ce80 0x7f917c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- 172.21.15.157:0/3324498492 >> 172.21.15.157:0/3324498492 conn(0x7f917c070790 msgr2=0x7f917c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- 172.21.15.157:0/3324498492 shutdown_connections 2023-11-06T03:12:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- 172.21.15.157:0/3324498492 wait complete. 2023-11-06T03:12:45.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 Processor -- start 2023-11-06T03:12:45.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.170+0000 7f9183948640 1 -- start start 2023-11-06T03:12:45.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f917c0d00b0 0x7f917c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15caa0 0x7f917c15c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c152c20 con 0x7f917c15ce80 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c1520a0 con 0x7f917c0d00b0 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c152380 con 0x7f917c15caa0 2023-11-06T03:12:45.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46376/0 (socket says 172.21.15.157:46376) 2023-11-06T03:12:45.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 -- 172.21.15.157:0/3973880410 learned_addr learned my addr 172.21.15.157:0/3973880410 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:45.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9180ee3640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15caa0 0x7f917c15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f91816e4640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f917c0d00b0 0x7f917c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 -- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15caa0 msgr2=0x7f917c15c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15caa0 0x7f917c15c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 -- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f917c0d00b0 msgr2=0x7f917c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f917c0d00b0 0x7f917c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 -- 172.21.15.157:0/3973880410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174009b30 con 0x7f917c15ce80 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9181ee5640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f917401a610 tx=0x7f9174019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740b8670 con 0x7f917c15ce80 2023-11-06T03:12:45.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91740b87d0 con 0x7f917c15ce80 2023-11-06T03:12:45.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c152660 con 0x7f917c15ce80 2023-11-06T03:12:45.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740b8b60 con 0x7f917c15ce80 2023-11-06T03:12:45.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c15c590 con 0x7f917c15ce80 2023-11-06T03:12:45.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c15c780 con 0x7f917c15ce80 2023-11-06T03:12:45.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f91740b8cc0 con 0x7f917c15ce80 2023-11-06T03:12:45.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 0x7f9160067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f917411e770 con 0x7f917c15ce80 2023-11-06T03:12:45.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.174+0000 7f91816e4640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 0x7f9160067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.178+0000 7f91816e4640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 0x7f9160067f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f917c070d40 tx=0x7f916c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.182+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f917c15c780 con 0x7f917c15ce80 2023-11-06T03:12:45.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.366+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f917c0c67a0 con 0x7f917c15ce80 2023-11-06T03:12:45.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.378+0000 7f9171ffb640 1 -- 172.21.15.157:0/3973880410 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7f917c0c67a0 con 0x7f917c15ce80 2023-11-06T03:12:45.378 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 15 and data pool 16 2023-11-06T03:12:45.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 msgr2=0x7f9160067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 0x7f9160067f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f917c070d40 tx=0x7f916c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:45.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 msgr2=0x7f917c14fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f917401a610 tx=0x7f9174019780 comp rx=0 tx=0).stop 2023-11-06T03:12:45.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 shutdown_connections 2023-11-06T03:12:45.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9160065a60 0x7f9160067f20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f917c15ce80 0x7f917c14fd90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f917c15caa0 0x7f917c15c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 --2- 172.21.15.157:0/3973880410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f917c0d00b0 0x7f917c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 >> 172.21.15.157:0/3973880410 conn(0x7f917c070790 msgr2=0x7f917c0b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 shutdown_connections 2023-11-06T03:12:45.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.382+0000 7f9183948640 1 -- 172.21.15.157:0/3973880410 wait complete. 2023-11-06T03:12:45.411 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 Processor -- start 2023-11-06T03:12:45.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 -- start start 2023-11-06T03:12:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff47815e5e0 0x7ff4780b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47815f580 con 0x7ff47815ea90 2023-11-06T03:12:45.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47815efc0 con 0x7ff4781551e0 2023-11-06T03:12:45.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47e85d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4780704b0 con 0x7ff47815e5e0 2023-11-06T03:12:45.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47cdfa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff47815ea90 0x7ff47815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58008/0 (socket says 172.21.15.157:58008) 2023-11-06T03:12:45.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47cdfa640 1 -- 172.21.15.157:0/1049133333 learned_addr learned my addr 172.21.15.157:0/1049133333 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:45.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329020462 0 0) 0x7ff47815f580 con 0x7ff47815ea90 2023-11-06T03:12:45.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff464003530 con 0x7ff47815ea90 2023-11-06T03:12:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> v2:172.21.15.195:3300/0 conn(0x7ff47815e5e0 0x7ff4780b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060642609 0 0) 0x7ff464003530 con 0x7ff47815ea90 2023-11-06T03:12:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 >> v2:172.21.15.195:3300/0 conn(0x7ff47815e5e0 msgr2=0x7ff4780b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 --2- 172.21.15.157:0/1049133333 >> v2:172.21.15.195:3300/0 conn(0x7ff47815e5e0 0x7ff4780b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 >> v1:172.21.15.195:6789/0 conn(0x7ff4781551e0 legacy=0x7ff47806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47815f2a0 con 0x7ff47815ea90 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff470001a00 con 0x7ff47815ea90 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> v2:172.21.15.195:3300/0 conn(0x7ff47815e5e0 0x7ff4780b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 >> v1:172.21.15.157:6789/0 conn(0x7ff47815ea90 legacy=0x7ff47815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff464003980 0x7ff464003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 0x7ff464004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4640052b0 0x7ff46400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780704b0 con 0x7ff4640047d0 2023-11-06T03:12:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46400c180 con 0x7ff464003980 2023-11-06T03:12:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46400c340 con 0x7ff4640052b0 2023-11-06T03:12:45.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 0x7ff464004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff477fff640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff464003980 0x7ff464003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47cdfa640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4640052b0 0x7ff46400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4640052b0 msgr2=0x7ff46400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4640052b0 0x7ff46400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff464003980 msgr2=0x7ff464003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:45.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff464003980 0x7ff464003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1049133333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464003530 con 0x7ff4640047d0 2023-11-06T03:12:45.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff47cdfa640 1 -- 172.21.15.157:0/1049133333 reap_dead start 2023-11-06T03:12:45.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.530+0000 7ff477fff640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff464003980 0x7ff464003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:45.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 0x7ff464004c50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff4680098a0 tx=0x7ff46801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f867c9f7ea5e1aa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4680a9610 con 0x7ff4640047d0 2023-11-06T03:12:45.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4680a9770 con 0x7ff4640047d0 2023-11-06T03:12:45.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff4777fe640 1 -- 172.21.15.157:0/1049133333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4680a9ae0 con 0x7ff4640047d0 2023-11-06T03:12:45.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 msgr2=0x7ff464004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 0x7ff464004c50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff4680098a0 tx=0x7ff46801c520 comp rx=0 tx=0).stop 2023-11-06T03:12:45.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- 172.21.15.157:0/1049133333 shutdown_connections 2023-11-06T03:12:45.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1049133333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4640047d0 0x7ff464004c50 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- 172.21.15.157:0/1049133333 >> 172.21.15.157:0/1049133333 conn(0x7ff478065930 msgr2=0x7ff478065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- 172.21.15.157:0/1049133333 shutdown_connections 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- 172.21.15.157:0/1049133333 wait complete. 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 Processor -- start 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- start start 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4640047d0 0x7ff47814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4781f7db0 0x7ff47815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781f5210 con 0x7ff4781f4d90 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478151360 con 0x7ff4640047d0 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff47e85d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781f83a0 con 0x7ff4781f7db0 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff46f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46390/0 (socket says 172.21.15.157:46390) 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1279478158 learned_addr learned my addr 172.21.15.157:0/1279478158 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.534+0000 7ff477fff640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4640047d0 0x7ff47814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff47cdfa640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4781f7db0 0x7ff47815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4781f7db0 msgr2=0x7ff47815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4781f7db0 0x7ff47815cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4640047d0 msgr2=0x7ff47814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4640047d0 0x7ff47814cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 -- 172.21.15.157:0/1279478158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4781f8680 con 0x7ff4781f4d90 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff46f7fe640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff4680098a0 tx=0x7ff464005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4680b7070 con 0x7ff4781f4d90 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4680a94c0 con 0x7ff4781f4d90 2023-11-06T03:12:45.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4781f4590 con 0x7ff4781f4d90 2023-11-06T03:12:45.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4680bc640 con 0x7ff4781f4d90 2023-11-06T03:12:45.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4781f4280 con 0x7ff4781f4d90 2023-11-06T03:12:45.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff478072750 con 0x7ff4781f4d90 2023-11-06T03:12:45.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4680b13f0 con 0x7ff4781f4d90 2023-11-06T03:12:45.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 0x7ff458067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff46811d6a0 con 0x7ff4781f4d90 2023-11-06T03:12:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff477fff640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 0x7ff458067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.538+0000 7ff477fff640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 0x7ff458067f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff460000c10 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.542+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff4680eaf90 con 0x7ff4781f4d90 2023-11-06T03:12:45.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.698+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4781f5210 con 0x7ff4781f4d90 2023-11-06T03:12:45.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.702+0000 7ff4757fa640 1 -- 172.21.15.157:0/1279478158 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17886 (secure 0 0 0) 0x7ff46800f040 con 0x7ff4781f4d90 2023-11-06T03:12:45.703 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:45.703 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":110,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:45.373798+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:45.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 msgr2=0x7ff458067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 0x7ff458067f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff460000c10 tx=0x7ff460009040 comp rx=0 tx=0).stop 2023-11-06T03:12:45.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 msgr2=0x7ff4781f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff4680098a0 tx=0x7ff464005800 comp rx=0 tx=0).stop 2023-11-06T03:12:45.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 shutdown_connections 2023-11-06T03:12:45.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff458065a60 0x7ff458067f20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4781f7db0 0x7ff47815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4781f4d90 0x7ff4781f7870 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 --2- 172.21.15.157:0/1279478158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4640047d0 0x7ff47814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 >> 172.21.15.157:0/1279478158 conn(0x7ff478065930 msgr2=0x7ff47815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 shutdown_connections 2023-11-06T03:12:45.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.706+0000 7ff47e85d640 1 -- 172.21.15.157:0/1279478158 wait complete. 2023-11-06T03:12:45.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:45.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.842+0000 7fe96f718640 1 Processor -- start 2023-11-06T03:12:45.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.842+0000 7fe96f718640 1 -- start start 2023-11-06T03:12:45.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96f718640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9680b3b20 0x7fe96815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96f718640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96815f580 con 0x7fe9681551e0 2023-11-06T03:12:45.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96f718640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96815efc0 con 0x7fe96815e5e0 2023-11-06T03:12:45.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96f718640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680704b0 con 0x7fe9680b3b20 2023-11-06T03:12:45.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96dcb5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9680b3b20 0x7fe96815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96d4b4640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe9681551e0 0x7fe96806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58024/0 (socket says 172.21.15.157:58024) 2023-11-06T03:12:45.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96d4b4640 1 -- 172.21.15.157:0/714372203 learned_addr learned my addr 172.21.15.157:0/714372203 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:45.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96dcb5640 1 -- 172.21.15.157:0/714372203 >> v1:172.21.15.195:6789/0 conn(0x7fe96815e5e0 legacy=0x7fe9680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:45.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96dcb5640 1 -- 172.21.15.157:0/714372203 >> v1:172.21.15.157:6789/0 conn(0x7fe9681551e0 legacy=0x7fe96806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe95ffff640 1 -- 172.21.15.157:0/714372203 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900874017 0 0) 0x7fe96815f580 con 0x7fe9681551e0 2023-11-06T03:12:45.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96dcb5640 1 -- 172.21.15.157:0/714372203 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96815f2a0 con 0x7fe9680b3b20 2023-11-06T03:12:45.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe96dcb5640 1 --2- 172.21.15.157:0/714372203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9680b3b20 0x7fe96815c1b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe96001c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c334d46c75fc4596 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe95ffff640 1 -- 172.21.15.157:0/714372203 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600ab600 con 0x7fe9680b3b20 2023-11-06T03:12:45.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe95ffff640 1 -- 172.21.15.157:0/714372203 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9600a7050 con 0x7fe9680b3b20 2023-11-06T03:12:45.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.846+0000 7fe95ffff640 1 -- 172.21.15.157:0/714372203 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600ab980 con 0x7fe9680b3b20 2023-11-06T03:12:45.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- 172.21.15.157:0/714372203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9680b3b20 msgr2=0x7fe96815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 --2- 172.21.15.157:0/714372203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9680b3b20 0x7fe96815c1b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097c0 tx=0x7fe96001c2f0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- 172.21.15.157:0/714372203 shutdown_connections 2023-11-06T03:12:45.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 --2- 172.21.15.157:0/714372203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9680b3b20 0x7fe96815c1b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- 172.21.15.157:0/714372203 >> 172.21.15.157:0/714372203 conn(0x7fe968065930 msgr2=0x7fe968065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:45.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- 172.21.15.157:0/714372203 shutdown_connections 2023-11-06T03:12:45.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- 172.21.15.157:0/714372203 wait complete. 2023-11-06T03:12:45.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 Processor -- start 2023-11-06T03:12:45.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- start start 2023-11-06T03:12:45.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9680b3b20 0x7fe96814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9681551e0 0x7fe9681525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814ed90 con 0x7fe96815e5e0 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814de70 con 0x7fe9680b3b20 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96f718640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814e110 con 0x7fe9681551e0 2023-11-06T03:12:45.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46402/0 (socket says 172.21.15.157:46402) 2023-11-06T03:12:45.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 -- 172.21.15.157:0/747699519 learned_addr learned my addr 172.21.15.157:0/747699519 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:45.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96d4b4640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9680b3b20 0x7fe96814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96ccb3640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9681551e0 0x7fe9681525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 -- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9681551e0 msgr2=0x7fe9681525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9681551e0 0x7fe9681525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 -- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9680b3b20 msgr2=0x7fe96814c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9680b3b20 0x7fe96814c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 -- 172.21.15.157:0/747699519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009420 con 0x7fe96815e5e0 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.850+0000 7fe96dcb5640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe960018f00 tx=0x7fe960019090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600b4630 con 0x7fe96815e5e0 2023-11-06T03:12:45.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9600abe90 con 0x7fe96815e5e0 2023-11-06T03:12:45.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600b49d0 con 0x7fe96815e5e0 2023-11-06T03:12:45.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96814e3f0 con 0x7fe96815e5e0 2023-11-06T03:12:45.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96814e9b0 con 0x7fe96815e5e0 2023-11-06T03:12:45.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96814eed0 con 0x7fe96815e5e0 2023-11-06T03:12:45.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9600b2040 con 0x7fe96815e5e0 2023-11-06T03:12:45.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 0x7fe94c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:45.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe96011d2c0 con 0x7fe96815e5e0 2023-11-06T03:12:45.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe96d4b4640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 0x7fe94c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:45.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.854+0000 7fe96d4b4640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 0x7fe94c067f20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe958000f70 tx=0x7fe958007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:45.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:45.858+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9600eabb0 con 0x7fe96815e5e0 2023-11-06T03:12:46.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.014+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe968072750 con 0x7fe96815e5e0 2023-11-06T03:12:46.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe95dffb640 1 -- 172.21.15.157:0/747699519 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17886 (secure 0 0 0) 0x7fe9600ee1d0 con 0x7fe96815e5e0 2023-11-06T03:12:46.017 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:46.018 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":110,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:45.373798+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:46.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 msgr2=0x7fe94c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 0x7fe94c067f20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe958000f70 tx=0x7fe958007b60 comp rx=0 tx=0).stop 2023-11-06T03:12:46.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 msgr2=0x7fe9680b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe960018f00 tx=0x7fe960019090 comp rx=0 tx=0).stop 2023-11-06T03:12:46.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 shutdown_connections 2023-11-06T03:12:46.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe94c065a60 0x7fe94c067f20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe96815e5e0 0x7fe9680b20e0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9681551e0 0x7fe9681525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 --2- 172.21.15.157:0/747699519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9680b3b20 0x7fe96814c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 >> 172.21.15.157:0/747699519 conn(0x7fe968065930 msgr2=0x7fe9680afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:46.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 shutdown_connections 2023-11-06T03:12:46.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.018+0000 7fe96f718640 1 -- 172.21.15.157:0/747699519 wait complete. 2023-11-06T03:12:46.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:12:46.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.158+0000 7fac08209640 1 Processor -- start 2023-11-06T03:12:46.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.158+0000 7fac08209640 1 -- start start 2023-11-06T03:12:46.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac08209640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac0015ce30 0x7fac000b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac08209640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fac000cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac08209640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0015d210 con 0x7fac000cf580 2023-11-06T03:12:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac08209640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000d17f0 con 0x7fac0015ce30 2023-11-06T03:12:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac08209640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac000ce650 con 0x7fac000cd520 2023-11-06T03:12:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac067a6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fac000cf580 0x7fac000cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58040/0 (socket says 172.21.15.157:58040) 2023-11-06T03:12:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac067a6640 1 -- 172.21.15.157:0/3689985939 learned_addr learned my addr 172.21.15.157:0/3689985939 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:46.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fac000cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac05fa5640 1 --2- 172.21.15.157:0/3689985939 >> v2:172.21.15.195:3300/0 conn(0x7fac0015ce30 0x7fac000b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753099489 0 0) 0x7fac0015d210 con 0x7fac000cf580 2023-11-06T03:12:46.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabec003530 con 0x7fac000cf580 2023-11-06T03:12:46.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 415567739 0 0) 0x7fabec003530 con 0x7fac000cf580 2023-11-06T03:12:46.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 msgr2=0x7fac000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fac000cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 >> v2:172.21.15.195:3300/0 conn(0x7fac0015ce30 msgr2=0x7fac000b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 --2- 172.21.15.157:0/3689985939 >> v2:172.21.15.195:3300/0 conn(0x7fac0015ce30 0x7fac000b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000ce090 con 0x7fac000cf580 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fabf8001a00 con 0x7fac000cf580 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 >> v1:172.21.15.157:6789/0 conn(0x7fac000cf580 legacy=0x7fac000cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fabec003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fabec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabec004d10 0x7fabec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000d17f0 con 0x7fabec003f00 2023-11-06T03:12:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000ce650 con 0x7fabec004d10 2023-11-06T03:12:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec00ba70 con 0x7fac000cd520 2023-11-06T03:12:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fabec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac05fa5640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fabec003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac067a6640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabec004d10 0x7fabec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 -- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 msgr2=0x7fabec003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac000cd520 0x7fabec003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 -- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabec004d10 msgr2=0x7fabec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabec004d10 0x7fabec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.162+0000 7fac057a4640 1 -- 172.21.15.157:0/3689985939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec003530 con 0x7fabec003f00 2023-11-06T03:12:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac057a4640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fabec0047d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009ae0 tx=0x7fabf001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9bd44ba0b21a4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00a5510 con 0x7fabec003f00 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf00a5670 con 0x7fabec003f00 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac04fa3640 1 -- 172.21.15.157:0/3689985939 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00a5a20 con 0x7fabec003f00 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- 172.21.15.157:0/3689985939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 msgr2=0x7fabec0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 --2- 172.21.15.157:0/3689985939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fabec0047d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009ae0 tx=0x7fabf001ca50 comp rx=0 tx=0).stop 2023-11-06T03:12:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac067a6640 1 -- 172.21.15.157:0/3689985939 reap_dead start 2023-11-06T03:12:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- 172.21.15.157:0/3689985939 shutdown_connections 2023-11-06T03:12:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- 172.21.15.157:0/3689985939 >> 172.21.15.157:0/3689985939 conn(0x7fac00070990 msgr2=0x7fac00065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- 172.21.15.157:0/3689985939 shutdown_connections 2023-11-06T03:12:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- 172.21.15.157:0/3689985939 wait complete. 2023-11-06T03:12:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 Processor -- start 2023-11-06T03:12:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- start start 2023-11-06T03:12:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac0015d9c0 0x7fac000ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 0x7fac0014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00151b70 con 0x7fabec003f00 2023-11-06T03:12:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac000cb220 con 0x7fac000cc620 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac08209640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac00151da0 con 0x7fac0015d9c0 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46410/0 (socket says 172.21.15.157:46410) 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 -- 172.21.15.157:0/3768324246 learned_addr learned my addr 172.21.15.157:0/3768324246 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac057a4640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac0015d9c0 0x7fac000ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac067a6640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 0x7fac0014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 -- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac0015d9c0 msgr2=0x7fac000ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac0015d9c0 0x7fac000ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 -- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 msgr2=0x7fac0014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 0x7fac0014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 -- 172.21.15.157:0/3768324246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000cac90 con 0x7fabec003f00 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac067a6640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 0x7fac0014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.166+0000 7fac05fa5640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fabfc011660 tx=0x7fabfc011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00f9c0 con 0x7fabec003f00 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00fb20 con 0x7fabec003f00 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac000caf40 con 0x7fabec003f00 2023-11-06T03:12:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00fe90 con 0x7fabec003f00 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac00152320 con 0x7fabec003f00 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac00160bf0 con 0x7fabec003f00 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fabfc015460 con 0x7fabec003f00 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 0x7fabe4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabfc083790 con 0x7fabec003f00 2023-11-06T03:12:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fac057a4640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 0x7fabe4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.170+0000 7fac057a4640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 0x7fabe4067f50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabf00af000 tx=0x7fabf00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.174+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fabfc051080 con 0x7fabec003f00 2023-11-06T03:12:46.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.362+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fac00072750 con 0x7fabec003f00 2023-11-06T03:12:46.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.378+0000 7fabf67fc640 1 -- 172.21.15.157:0/3768324246 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7fabfc0546a0 con 0x7fabec003f00 2023-11-06T03:12:46.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 msgr2=0x7fabe4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 0x7fabe4067f50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabf00af000 tx=0x7fabf00046d0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 msgr2=0x7fac000cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fabfc011660 tx=0x7fabfc011b20 comp rx=0 tx=0).stop 2023-11-06T03:12:46.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 shutdown_connections 2023-11-06T03:12:46.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabe4065a90 0x7fabe4067f50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac000cc620 0x7fac0014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac0015d9c0 0x7fac000ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 --2- 172.21.15.157:0/3768324246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabec003f00 0x7fac000cbda0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 >> 172.21.15.157:0/3768324246 conn(0x7fac00070990 msgr2=0x7fac00158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 shutdown_connections 2023-11-06T03:12:46.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.382+0000 7fac08209640 1 -- 172.21.15.157:0/3768324246 wait complete. 2023-11-06T03:12:46.402 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:12:46.403 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:12:46.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.518+0000 7fc444266640 1 Processor -- start 2023-11-06T03:12:46.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc444266640 1 -- start start 2023-11-06T03:12:46.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc444266640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 0x7fc43c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc444266640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c15f580 con 0x7fc43c0b3b20 2023-11-06T03:12:46.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc444266640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c15efc0 con 0x7fc43c15e5e0 2023-11-06T03:12:46.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc444266640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15f2a0 con 0x7fc43c1551e0 2023-11-06T03:12:46.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc442803640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc43c0b3b20 0x7fc43c15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58056/0 (socket says 172.21.15.157:58056) 2023-11-06T03:12:46.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc442803640 1 -- 172.21.15.157:0/625237531 learned_addr learned my addr 172.21.15.157:0/625237531 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:46.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc442002640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 0x7fc43c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418143514 0 0) 0x7fc43c15f580 con 0x7fc43c0b3b20 2023-11-06T03:12:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc428003530 con 0x7fc43c0b3b20 2023-11-06T03:12:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797936991 0 0) 0x7fc43c15efc0 con 0x7fc43c15e5e0 2023-11-06T03:12:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc43c15f580 con 0x7fc43c15e5e0 2023-11-06T03:12:46.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 416346631 0 0) 0x7fc428003530 con 0x7fc43c0b3b20 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 msgr2=0x7fc43c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 0x7fc43c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 >> v1:172.21.15.195:6789/0 conn(0x7fc43c15e5e0 legacy=0x7fc43c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c06f5a0 con 0x7fc43c0b3b20 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc434001a00 con 0x7fc43c0b3b20 2023-11-06T03:12:46.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 >> v1:172.21.15.157:6789/0 conn(0x7fc43c0b3b20 legacy=0x7fc43c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc428003b40 0x7fc428003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc428004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 0x7fc4280051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15f2a0 con 0x7fc4280047d0 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42800baa0 con 0x7fc428003b40 2023-11-06T03:12:46.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42800bc60 con 0x7fc43c1551e0 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc428004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc442002640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc428003b40 0x7fc428003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 -- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 msgr2=0x7fc4280051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c1551e0 0x7fc4280051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 -- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc428003b40 msgr2=0x7fc428003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:46.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc428003b40 0x7fc428003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.522+0000 7fc441801640 1 -- 172.21.15.157:0/625237531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428003530 con 0x7fc4280047d0 2023-11-06T03:12:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc441801640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc428004c50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0018d0 tx=0x7fc42c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cafb3eff3fa055 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0052d0 con 0x7fc4280047d0 2023-11-06T03:12:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c005450 con 0x7fc4280047d0 2023-11-06T03:12:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc441000640 1 -- 172.21.15.157:0/625237531 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c003840 con 0x7fc4280047d0 2023-11-06T03:12:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/625237531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 msgr2=0x7fc428004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 --2- 172.21.15.157:0/625237531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc428004c50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0018d0 tx=0x7fc42c01c530 comp rx=0 tx=0).stop 2023-11-06T03:12:46.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442803640 1 -- 172.21.15.157:0/625237531 reap_dead start 2023-11-06T03:12:46.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/625237531 shutdown_connections 2023-11-06T03:12:46.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/625237531 >> 172.21.15.157:0/625237531 conn(0x7fc43c065930 msgr2=0x7fc43c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:46.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/625237531 shutdown_connections 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/625237531 wait complete. 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 Processor -- start 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- start start 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc43c151cf0 0x7fc43c150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c14c1d0 0x7fc43c14a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c150560 con 0x7fc4280047d0 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c152260 con 0x7fc43c151cf0 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c152540 con 0x7fc43c14c1d0 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46428/0 (socket says 172.21.15.157:46428) 2023-11-06T03:12:46.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 -- 172.21.15.157:0/652999858 learned_addr learned my addr 172.21.15.157:0/652999858 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:46.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442803640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c14c1d0 0x7fc43c14a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc441801640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc43c151cf0 0x7fc43c150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 -- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c14c1d0 msgr2=0x7fc43c14a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c14c1d0 0x7fc43c14a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 -- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc43c151cf0 msgr2=0x7fc43c150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc43c151cf0 0x7fc43c150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 -- 172.21.15.157:0/652999858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c152820 con 0x7fc4280047d0 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc442002640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc4380119d0 tx=0x7fc438011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800fb50 con 0x7fc4280047d0 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43800fcb0 con 0x7fc4280047d0 2023-11-06T03:12:46.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c152b00 con 0x7fc4280047d0 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438014440 con 0x7fc4280047d0 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.526+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c14c650 con 0x7fc4280047d0 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c14e180 con 0x7fc4280047d0 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4380145a0 con 0x7fc4280047d0 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc4327fc640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 0x7fc420067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:46.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc438083b50 con 0x7fc4280047d0 2023-11-06T03:12:46.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc441801640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 0x7fc420067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:46.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.530+0000 7fc441801640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 0x7fc420067f20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ad000 tx=0x7fc42c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:46.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.534+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc438051470 con 0x7fc4280047d0 2023-11-06T03:12:46.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:46.726+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc43c072750 con 0x7fc4280047d0 2023-11-06T03:12:46.972 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:12:46.974+0000 7f1279cc7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:12:47.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc4327fc640 1 -- 172.21.15.157:0/652999858 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4380220c0 con 0x7fc4280047d0 2023-11-06T03:12:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 msgr2=0x7fc420067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 0x7fc420067f20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ad000 tx=0x7fc42c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:12:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 msgr2=0x7fc43c153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc4380119d0 tx=0x7fc438011e90 comp rx=0 tx=0).stop 2023-11-06T03:12:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 shutdown_connections 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc420065a60 0x7fc420067f20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc43c14c1d0 0x7fc43c14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc43c151cf0 0x7fc43c150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 --2- 172.21.15.157:0/652999858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280047d0 0x7fc43c153300 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 >> 172.21.15.157:0/652999858 conn(0x7fc43c065930 msgr2=0x7fc43c15d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 shutdown_connections 2023-11-06T03:12:47.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.394+0000 7fc444266640 1 -- 172.21.15.157:0/652999858 wait complete. 2023-11-06T03:12:47.417 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:12:47.418 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:47.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 Processor -- start 2023-11-06T03:12:47.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 -- start start 2023-11-06T03:12:47.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74ac0ce800 0x7f74ac0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac06b6a0 0x7f74ac06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac0cf1a0 con 0x7f74ac146370 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac0b2120 con 0x7f74ac0ce800 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b47e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac06c3d0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74ac0ce800 0x7f74ac0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f74ac0ce800 0x7f74ac0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46444/0 (socket says 172.21.15.157:46444) 2023-11-06T03:12:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 -- 172.21.15.157:0/3884122703 learned_addr learned my addr 172.21.15.157:0/3884122703 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b2d84640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac06b6a0 0x7f74ac06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b2583640 1 --1- 172.21.15.157:0/3884122703 >> v1:172.21.15.195:6789/0 conn(0x7f74ac146370 0x7f74ac0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:60952/0 (socket says 172.21.15.157:60952) 2023-11-06T03:12:47.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 -- 172.21.15.157:0/3884122703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac06b6a0 msgr2=0x7f74ac06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:47.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac06b6a0 0x7f74ac06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 -- 172.21.15.157:0/3884122703 >> v1:172.21.15.195:6789/0 conn(0x7f74ac146370 legacy=0x7f74ac0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.542+0000 7f74b1d82640 1 -- 172.21.15.157:0/3884122703 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac06f5a0 con 0x7f74ac0ce800 2023-11-06T03:12:47.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b1d82640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac0ce800 0x7f74ac0cec60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009870 tx=0x7f74a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68be842fc4b837ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b1581640 1 -- 172.21.15.157:0/3884122703 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00ab680 con 0x7f74ac0ce800 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b1581640 1 -- 172.21.15.157:0/3884122703 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a00ab7e0 con 0x7f74ac0ce800 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b1581640 1 -- 172.21.15.157:0/3884122703 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00abb70 con 0x7f74ac0ce800 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- 172.21.15.157:0/3884122703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac0ce800 msgr2=0x7f74ac0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac0ce800 0x7f74ac0cec60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009870 tx=0x7f74a001c480 comp rx=0 tx=0).stop 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- 172.21.15.157:0/3884122703 shutdown_connections 2023-11-06T03:12:47.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac06b6a0 0x7f74ac06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 --2- 172.21.15.157:0/3884122703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac0ce800 0x7f74ac0cec60 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- 172.21.15.157:0/3884122703 >> 172.21.15.157:0/3884122703 conn(0x7f74ac065930 msgr2=0x7f74ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:47.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- 172.21.15.157:0/3884122703 shutdown_connections 2023-11-06T03:12:47.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- 172.21.15.157:0/3884122703 wait complete. 2023-11-06T03:12:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 Processor -- start 2023-11-06T03:12:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.546+0000 7f74b47e7640 1 -- start start 2023-11-06T03:12:47.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac0ce800 0x7f74ac13a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74ac146370 0x7f74ac13a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0af5a0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0afe00 con 0x7f74ac146370 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac15dc10 con 0x7f74ac0ce800 2023-11-06T03:12:47.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46458/0 (socket says 172.21.15.157:46458) 2023-11-06T03:12:47.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 -- 172.21.15.157:0/860101089 learned_addr learned my addr 172.21.15.157:0/860101089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:47.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b1d82640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac0ce800 0x7f74ac13a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 -- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac0ce800 msgr2=0x7f74ac13a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac0ce800 0x7f74ac13a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2d84640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74ac146370 0x7f74ac13a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 -- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74ac146370 msgr2=0x7f74ac13a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74ac146370 0x7f74ac13a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 -- 172.21.15.157:0/860101089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00094d0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b2583640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c8a0 tx=0x7f749c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00a640 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00a7a0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac13c300 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00ab10 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac0afa90 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac072750 con 0x7f74ac06b6a0 2023-11-06T03:12:47.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f749c00ac70 con 0x7f74ac06b6a0 2023-11-06T03:12:47.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 0x7f7498067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f749c082a50 con 0x7f74ac06b6a0 2023-11-06T03:12:47.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b1d82640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 0x7f7498067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.550+0000 7f74b1d82640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 0x7f7498067f20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f74a00af000 tx=0x7f74a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.554+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f749c0503f0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.742+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74ac0b3cb0 con 0x7f74ac06b6a0 2023-11-06T03:12:47.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74aaffd640 1 -- 172.21.15.157:0/860101089 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f749c053a10 con 0x7f74ac06b6a0 2023-11-06T03:12:47.744 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:47.745 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:12:47.745 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:12:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 msgr2=0x7f7498067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 0x7f7498067f20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f74a00af000 tx=0x7f74a0008040 comp rx=0 tx=0).stop 2023-11-06T03:12:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 msgr2=0x7f74ac15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c8a0 tx=0x7f749c00cd60 comp rx=0 tx=0).stop 2023-11-06T03:12:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 shutdown_connections 2023-11-06T03:12:47.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7498065a60 0x7f7498067f20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74ac146370 0x7f74ac13a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f74ac0ce800 0x7f74ac13a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 --2- 172.21.15.157:0/860101089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74ac06b6a0 0x7f74ac15bbf0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 >> 172.21.15.157:0/860101089 conn(0x7f74ac065930 msgr2=0x7f74ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 shutdown_connections 2023-11-06T03:12:47.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.746+0000 7f74b47e7640 1 -- 172.21.15.157:0/860101089 wait complete. 2023-11-06T03:12:47.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:12:47.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.878+0000 7fbe54586640 1 Processor -- start 2023-11-06T03:12:47.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.878+0000 7fbe54586640 1 -- start start 2023-11-06T03:12:47.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe54586640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c1599c0 0x7fbe4c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe54586640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe4c0b2af0 0x7fbe4c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe54586640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4c15e440 con 0x7fbe4c06d000 2023-11-06T03:12:47.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe54586640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c06c240 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe54586640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c06c520 con 0x7fbe4c1599c0 2023-11-06T03:12:47.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe4c0b2af0 0x7fbe4c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbe4c0b2af0 0x7fbe4c159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46460/0 (socket says 172.21.15.157:46460) 2023-11-06T03:12:47.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 -- 172.21.15.157:0/2746448035 learned_addr learned my addr 172.21.15.157:0/2746448035 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:47.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52322640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c1599c0 0x7fbe4c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 -- 172.21.15.157:0/2746448035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c1599c0 msgr2=0x7fbe4c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:47.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c1599c0 0x7fbe4c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 -- 172.21.15.157:0/2746448035 >> v1:172.21.15.195:6789/0 conn(0x7fbe4c06d000 legacy=0x7fbe4c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:47.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 -- 172.21.15.157:0/2746448035 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c06c800 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe52b23640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0b2af0 0x7fbe4c159da0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009870 tx=0x7fbe4401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d6ff755ce15166 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe51320640 1 -- 172.21.15.157:0/2746448035 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe440aa610 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe51320640 1 -- 172.21.15.157:0/2746448035 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440a6040 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.882+0000 7fbe51320640 1 -- 172.21.15.157:0/2746448035 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe440aa990 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/2746448035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0b2af0 msgr2=0x7fbe4c159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0b2af0 0x7fbe4c159da0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009870 tx=0x7fbe4401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/2746448035 shutdown_connections 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0b2af0 0x7fbe4c159da0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- 172.21.15.157:0/2746448035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c1599c0 0x7fbe4c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/2746448035 >> 172.21.15.157:0/2746448035 conn(0x7fbe4c065930 msgr2=0x7fbe4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:47.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/2746448035 shutdown_connections 2023-11-06T03:12:47.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/2746448035 wait complete. 2023-11-06T03:12:47.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 Processor -- start 2023-11-06T03:12:47.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- start start 2023-11-06T03:12:47.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c06d000 0x7fbe4c14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c0b2af0 0x7fbe4c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0cc570 con 0x7fbe4c1599c0 2023-11-06T03:12:47.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0cc850 con 0x7fbe4c06d000 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0ccb30 con 0x7fbe4c0b2af0 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46476/0 (socket says 172.21.15.157:46476) 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 -- 172.21.15.157:0/94513043 learned_addr learned my addr 172.21.15.157:0/94513043 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe51b21640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c0b2af0 0x7fbe4c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52322640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c06d000 0x7fbe4c14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 -- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c0b2af0 msgr2=0x7fbe4c14e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c0b2af0 0x7fbe4c14e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 -- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c06d000 msgr2=0x7fbe4c14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c06d000 0x7fbe4c14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 -- 172.21.15.157:0/94513043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440094d0 con 0x7fbe4c1599c0 2023-11-06T03:12:47.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe52b23640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ac040 tx=0x7fbe4401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe440bd3f0 con 0x7fbe4c1599c0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440bd550 con 0x7fbe4c1599c0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c0cce10 con 0x7fbe4c1599c0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe440bd8e0 con 0x7fbe4c1599c0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.886+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c14d830 con 0x7fbe4c1599c0 2023-11-06T03:12:47.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0ccc70 con 0x7fbe4c1599c0 2023-11-06T03:12:47.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbe440b33f0 con 0x7fbe4c1599c0 2023-11-06T03:12:47.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe42ffd640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 0x7fbe30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:47.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe4411d770 con 0x7fbe4c1599c0 2023-11-06T03:12:47.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe52322640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 0x7fbe30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:47.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.890+0000 7fbe52322640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 0x7fbe30067f20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003fd0 tx=0x7fbe480039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:47.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:47.894+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbe4c0ccc70 con 0x7fbe4c1599c0 2023-11-06T03:12:48.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.038+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe4c0ccc70 con 0x7fbe4c1599c0 2023-11-06T03:12:48.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe42ffd640 1 -- 172.21.15.157:0/94513043 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fbe4c0ccc70 con 0x7fbe4c1599c0 2023-11-06T03:12:48.041 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:48.041 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:46.378969+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:12:48.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 msgr2=0x7fbe30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 0x7fbe30067f20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe48003fd0 tx=0x7fbe480039a0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 msgr2=0x7fbe4c06f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ac040 tx=0x7fbe4401ad10 comp rx=0 tx=0).stop 2023-11-06T03:12:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 shutdown_connections 2023-11-06T03:12:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbe30065a60 0x7fbe30067f20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c1599c0 0x7fbe4c06f470 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbe4c0b2af0 0x7fbe4c14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 --2- 172.21.15.157:0/94513043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe4c06d000 0x7fbe4c14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 >> 172.21.15.157:0/94513043 conn(0x7fbe4c065930 msgr2=0x7fbe4c1567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 shutdown_connections 2023-11-06T03:12:48.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.042+0000 7fbe54586640 1 -- 172.21.15.157:0/94513043 wait complete. 2023-11-06T03:12:48.059 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:12:48.060 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:48.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.174+0000 7f61bc8d6640 1 Processor -- start 2023-11-06T03:12:48.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.174+0000 7f61bc8d6640 1 -- start start 2023-11-06T03:12:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f61b8146370 0x7f61b80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61b80ce800 0x7f61b80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b80cf1a0 con 0x7f61b806b6a0 2023-11-06T03:12:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b806c3d0 con 0x7f61b80ce800 2023-11-06T03:12:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b806f5a0 con 0x7f61b8146370 2023-11-06T03:12:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61b80ce800 0x7f61b80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f61b80ce800 0x7f61b80cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46480/0 (socket says 172.21.15.157:46480) 2023-11-06T03:12:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 -- 172.21.15.157:0/779176145 learned_addr learned my addr 172.21.15.157:0/779176145 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61b659c640 1 --2- 172.21.15.157:0/779176145 >> v2:172.21.15.195:3300/0 conn(0x7f61b8146370 0x7f61b80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 -- 172.21.15.157:0/779176145 >> v2:172.21.15.195:3300/0 conn(0x7f61b8146370 msgr2=0x7f61b80b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 --2- 172.21.15.157:0/779176145 >> v2:172.21.15.195:3300/0 conn(0x7f61b8146370 0x7f61b80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 -- 172.21.15.157:0/779176145 >> v1:172.21.15.195:6789/0 conn(0x7f61b806b6a0 legacy=0x7f61b806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 -- 172.21.15.157:0/779176145 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b806f880 con 0x7f61b80ce800 2023-11-06T03:12:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61add9b640 1 --2- 172.21.15.157:0/779176145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b80cec60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009870 tx=0x7f61a801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf7a9f23b8fadf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61b5d9b640 1 -- 172.21.15.157:0/779176145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ab600 con 0x7f61b80ce800 2023-11-06T03:12:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61b5d9b640 1 -- 172.21.15.157:0/779176145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a80ab760 con 0x7f61b80ce800 2023-11-06T03:12:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61b5d9b640 1 -- 172.21.15.157:0/779176145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80abaf0 con 0x7f61b80ce800 2023-11-06T03:12:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- 172.21.15.157:0/779176145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 msgr2=0x7f61b80cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/779176145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b80cec60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009870 tx=0x7f61a801c400 comp rx=0 tx=0).stop 2023-11-06T03:12:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- 172.21.15.157:0/779176145 shutdown_connections 2023-11-06T03:12:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/779176145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b80cec60 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/779176145 >> v2:172.21.15.195:3300/0 conn(0x7f61b8146370 0x7f61b80b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- 172.21.15.157:0/779176145 >> 172.21.15.157:0/779176145 conn(0x7f61b8065930 msgr2=0x7f61b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- 172.21.15.157:0/779176145 shutdown_connections 2023-11-06T03:12:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.178+0000 7f61bc8d6640 1 -- 172.21.15.157:0/779176145 wait complete. 2023-11-06T03:12:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 Processor -- start 2023-11-06T03:12:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- start start 2023-11-06T03:12:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 0x7f61b8145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8146370 0x7f61b8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b813c100 con 0x7f61b80ce800 2023-11-06T03:12:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b81586b0 con 0x7f61b806b6a0 2023-11-06T03:12:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8158990 con 0x7f61b8146370 2023-11-06T03:12:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46482/0 (socket says 172.21.15.157:46482) 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 -- 172.21.15.157:0/1670206241 learned_addr learned my addr 172.21.15.157:0/1670206241 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 -- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8146370 msgr2=0x7f61b8143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61b659c640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 0x7f61b8145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61b6d9d640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8146370 0x7f61b8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8146370 0x7f61b8143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 -- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 msgr2=0x7f61b8145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 0x7f61b8145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 -- 172.21.15.157:0/1670206241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80094d0 con 0x7f61b80ce800 2023-11-06T03:12:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61b659c640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 0x7f61b8145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61add9b640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f61a80ad040 tx=0x7f61a801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80b4800 con 0x7f61b80ce800 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a80b4980 con 0x7f61b80ce800 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b815a020 con 0x7f61b80ce800 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80b4d30 con 0x7f61b80ce800 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8145820 con 0x7f61b80ce800 2023-11-06T03:12:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.182+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b8072750 con 0x7f61b80ce800 2023-11-06T03:12:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.186+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61a80c7440 con 0x7f61b80ce800 2023-11-06T03:12:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.186+0000 7f61af7fe640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 0x7f619c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.186+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61a811ddb0 con 0x7f61b80ce800 2023-11-06T03:12:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.186+0000 7f61b659c640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 0x7f619c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.186+0000 7f61b659c640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 0x7f619c067f20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.190+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f61a80eb6a0 con 0x7f61b80ce800 2023-11-06T03:12:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.342+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61b8158530 con 0x7f619c065a60 2023-11-06T03:12:48.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.366+0000 7f61af7fe640 1 -- 172.21.15.157:0/1670206241 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299741 (secure 0 0 0) 0x7f61a000c040 con 0x7f619c065a60 2023-11-06T03:12:48.367 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:48.369 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:48.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.366+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 msgr2=0x7f619c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.366+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 0x7f619c067f20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0008040 comp rx=0 tx=0).stop 2023-11-06T03:12:48.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.366+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 msgr2=0x7f61b8143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.366+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f61a80ad040 tx=0x7f61a801ac40 comp rx=0 tx=0).stop 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 shutdown_connections 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f619c065a60 0x7f619c067f20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61b8146370 0x7f61b8143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b80ce800 0x7f61b8143970 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 --2- 172.21.15.157:0/1670206241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61b806b6a0 0x7f61b8145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 >> 172.21.15.157:0/1670206241 conn(0x7f61b8065930 msgr2=0x7f61b80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 shutdown_connections 2023-11-06T03:12:48.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.370+0000 7f61bc8d6640 1 -- 172.21.15.157:0/1670206241 wait complete. 2023-11-06T03:12:48.385 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-11-06T03:12:47.284751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":599196,"kb_used_data":11484,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754375524,"statfs":{"total":773094113280,"available":772480536576,"internally_reserved":0,"allocated":11759616,"data_stored":6017833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001137"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983314+0000","last_change":"2023-11-06T03:12:43.983314+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983314+0000","last_undegraded":"2023-11-06T03:12:43.983314+0000","last_fullsized":"2023-11-06T03:12:43.983314+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983047+0000","last_change":"2023-11-06T03:12:43.983047+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983047+0000","last_undegraded":"2023-11-06T03:12:43.983047+0000","last_fullsized":"2023-11-06T03:12:43.983047+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985377+0000","last_change":"2023-11-06T03:12:43.985377+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985377+0000","last_undegraded":"2023-11-06T03:12:43.985377+0000","last_fullsized":"2023-11-06T03:12:43.985377+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986351+0000","last_change":"2023-11-06T03:12:43.986351+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986351+0000","last_undegraded":"2023-11-06T03:12:43.986351+0000","last_fullsized":"2023-11-06T03:12:43.986351+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989535+0000","last_change":"2023-11-06T03:12:43.989535+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989535+0000","last_undegraded":"2023-11-06T03:12:43.989535+0000","last_fullsized":"2023-11-06T03:12:43.989535+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989081+0000","last_change":"2023-11-06T03:12:43.989081+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989081+0000","last_undegraded":"2023-11-06T03:12:43.989081+0000","last_fullsized":"2023-11-06T03:12:43.989081+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984283+0000","last_change":"2023-11-06T03:12:43.984283+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984283+0000","last_undegraded":"2023-11-06T03:12:43.984283+0000","last_fullsized":"2023-11-06T03:12:43.984283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983426+0000","last_change":"2023-11-06T03:12:43.983426+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983426+0000","last_undegraded":"2023-11-06T03:12:43.983426+0000","last_fullsized":"2023-11-06T03:12:43.983426+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984370+0000","last_change":"2023-11-06T03:12:43.984370+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984370+0000","last_undegraded":"2023-11-06T03:12:43.984370+0000","last_fullsized":"2023-11-06T03:12:43.984370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986505+0000","last_change":"2023-11-06T03:12:43.986505+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986505+0000","last_undegraded":"2023-11-06T03:12:43.986505+0000","last_fullsized":"2023-11-06T03:12:43.986505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990818+0000","last_change":"2023-11-06T03:12:43.990818+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990818+0000","last_undegraded":"2023-11-06T03:12:43.990818+0000","last_fullsized":"2023-11-06T03:12:43.990818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992708+0000","last_change":"2023-11-06T03:12:43.992708+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992708+0000","last_undegraded":"2023-11-06T03:12:43.992708+0000","last_fullsized":"2023-11-06T03:12:43.992708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986256+0000","last_change":"2023-11-06T03:12:43.986256+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986256+0000","last_undegraded":"2023-11-06T03:12:43.986256+0000","last_fullsized":"2023-11-06T03:12:43.986256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983563+0000","last_change":"2023-11-06T03:12:43.983563+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983563+0000","last_undegraded":"2023-11-06T03:12:43.983563+0000","last_fullsized":"2023-11-06T03:12:43.983563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981947+0000","last_change":"2023-11-06T03:12:43.981947+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981947+0000","last_undegraded":"2023-11-06T03:12:43.981947+0000","last_fullsized":"2023-11-06T03:12:43.981947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984220+0000","last_change":"2023-11-06T03:12:43.984220+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984220+0000","last_undegraded":"2023-11-06T03:12:43.984220+0000","last_fullsized":"2023-11-06T03:12:43.984220+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986358+0000","last_change":"2023-11-06T03:12:43.986358+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986358+0000","last_undegraded":"2023-11-06T03:12:43.986358+0000","last_fullsized":"2023-11-06T03:12:43.986358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985941+0000","last_change":"2023-11-06T03:12:43.985941+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985941+0000","last_undegraded":"2023-11-06T03:12:43.985941+0000","last_fullsized":"2023-11-06T03:12:43.985941+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.991077+0000","last_change":"2023-11-06T03:12:43.991077+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.991077+0000","last_undegraded":"2023-11-06T03:12:43.991077+0000","last_fullsized":"2023-11-06T03:12:43.991077+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.987189+0000","last_change":"2023-11-06T03:12:43.987189+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.987189+0000","last_undegraded":"2023-11-06T03:12:43.987189+0000","last_fullsized":"2023-11-06T03:12:43.987189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984408+0000","last_change":"2023-11-06T03:12:43.984408+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984408+0000","last_undegraded":"2023-11-06T03:12:43.984408+0000","last_fullsized":"2023-11-06T03:12:43.984408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980853+0000","last_change":"2023-11-06T03:12:43.980853+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980853+0000","last_undegraded":"2023-11-06T03:12:43.980853+0000","last_fullsized":"2023-11-06T03:12:43.980853+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990913+0000","last_change":"2023-11-06T03:12:43.990913+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990913+0000","last_undegraded":"2023-11-06T03:12:43.990913+0000","last_fullsized":"2023-11-06T03:12:43.990913+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980645+0000","last_change":"2023-11-06T03:12:43.980645+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980645+0000","last_undegraded":"2023-11-06T03:12:43.980645+0000","last_fullsized":"2023-11-06T03:12:43.980645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.982231+0000","last_change":"2023-11-06T03:12:43.982231+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.982231+0000","last_undegraded":"2023-11-06T03:12:43.982231+0000","last_fullsized":"2023-11-06T03:12:43.982231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116826+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:12:43.116826+0000","last_peered":"2023-11-06T03:12:43.116826+0000","last_clean":"2023-11-06T03:12:43.116826+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:12:43.116826+0000","last_undegraded":"2023-11-06T03:12:43.116826+0000","last_fullsized":"2023-11-06T03:12:43.116826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421147+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:12:42.421147+0000","last_peered":"2023-11-06T03:12:42.421147+0000","last_clean":"2023-11-06T03:12:42.421147+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:12:42.421147+0000","last_undegraded":"2023-11-06T03:12:42.421147+0000","last_fullsized":"2023-11-06T03:12:42.421147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989606+0000","last_change":"2023-11-06T03:12:43.989606+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989606+0000","last_undegraded":"2023-11-06T03:12:43.989606+0000","last_fullsized":"2023-11-06T03:12:43.989606+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116807+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:12:43.116807+0000","last_peered":"2023-11-06T03:12:43.116807+0000","last_clean":"2023-11-06T03:12:43.116807+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:12:43.116807+0000","last_undegraded":"2023-11-06T03:12:43.116807+0000","last_fullsized":"2023-11-06T03:12:43.116807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989239+0000","last_change":"2023-11-06T03:12:43.989239+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989239+0000","last_undegraded":"2023-11-06T03:12:43.989239+0000","last_fullsized":"2023-11-06T03:12:43.989239+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.360812+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:12:43.360812+0000","last_peered":"2023-11-06T03:12:43.360812+0000","last_clean":"2023-11-06T03:12:43.360812+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:12:43.360812+0000","last_undegraded":"2023-11-06T03:12:43.360812+0000","last_fullsized":"2023-11-06T03:12:43.360812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992045+0000","last_change":"2023-11-06T03:12:43.992045+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992045+0000","last_undegraded":"2023-11-06T03:12:43.992045+0000","last_fullsized":"2023-11-06T03:12:43.992045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421148+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:12:42.421148+0000","last_peered":"2023-11-06T03:12:42.421148+0000","last_clean":"2023-11-06T03:12:42.421148+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:12:42.421148+0000","last_undegraded":"2023-11-06T03:12:42.421148+0000","last_fullsized":"2023-11-06T03:12:42.421148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992196+0000","last_change":"2023-11-06T03:12:43.992196+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992196+0000","last_undegraded":"2023-11-06T03:12:43.992196+0000","last_fullsized":"2023-11-06T03:12:43.992196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.992408+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:12:42.992408+0000","last_peered":"2023-11-06T03:12:42.992408+0000","last_clean":"2023-11-06T03:12:42.992408+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:12:42.992408+0000","last_undegraded":"2023-11-06T03:12:42.992408+0000","last_fullsized":"2023-11-06T03:12:42.992408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981183+0000","last_change":"2023-11-06T03:12:43.981183+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981183+0000","last_undegraded":"2023-11-06T03:12:43.981183+0000","last_fullsized":"2023-11-06T03:12:43.981183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116732+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:12:43.116732+0000","last_peered":"2023-11-06T03:12:43.116732+0000","last_clean":"2023-11-06T03:12:43.116732+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:12:43.116732+0000","last_undegraded":"2023-11-06T03:12:43.116732+0000","last_fullsized":"2023-11-06T03:12:43.116732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981798+0000","last_change":"2023-11-06T03:12:43.981798+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981798+0000","last_undegraded":"2023-11-06T03:12:43.981798+0000","last_fullsized":"2023-11-06T03:12:43.981798+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.987670+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:12:43.987670+0000","last_peered":"2023-11-06T03:12:43.987670+0000","last_clean":"2023-11-06T03:12:43.987670+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:12:43.987670+0000","last_undegraded":"2023-11-06T03:12:43.987670+0000","last_fullsized":"2023-11-06T03:12:43.987670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981793+0000","last_change":"2023-11-06T03:12:43.981793+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981793+0000","last_undegraded":"2023-11-06T03:12:43.981793+0000","last_fullsized":"2023-11-06T03:12:43.981793+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.993491+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:12:43.993491+0000","last_peered":"2023-11-06T03:12:43.993491+0000","last_clean":"2023-11-06T03:12:43.993491+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:12:43.993491+0000","last_undegraded":"2023-11-06T03:12:43.993491+0000","last_fullsized":"2023-11-06T03:12:43.993491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87000,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284840,"statfs":{"total":96636764160,"available":96547676160,"internally_reserved":0,"allocated":1916928,"data_stored":1195301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.117,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83299999999999996,"15min":0.86599999999999999},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.367,"15min":1.397},"last":1.1020000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.84899999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2490000000000001,"5min":1.254,"15min":1.444},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.218,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.92900000000000005,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1399999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0820000000000001,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.66600000000000004,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":2.512,"15min":2.512},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.1299999999999999,"15min":1.0580000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.1799999999999999,"5min":10.596,"15min":10.596},"last":1.1899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.2709999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.54,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.01}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.056,"15min":1.0469999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.85499999999999998,"5min":0.68600000000000005,"15min":0.55200000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.345}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.171,"15min":1.1579999999999999},"min":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.8149999999999999,"5min":2.548,"15min":2.548},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.7370000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.149}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80668,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":79360,"kb_avail":94291172,"statfs":{"total":96636764160,"available":96554160128,"internally_reserved":0,"allocated":1331200,"data_stored":610084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81264640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.98199999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.1870000000000001,"5min":16.073,"15min":16.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.288,"15min":0.97799999999999998},"min":{"1min":0.53100000000000003,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":40.905000000000001,"15min":40.905000000000001},"last":1.038}]},{"osd":1,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.056,"15min":0.90200000000000002},"min":{"1min":0.55800000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2370000000000001,"5min":16.012,"15min":16.012},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90800000000000003,"15min":0.88200000000000001},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.409,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.137,"15min":0.91000000000000003},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.111,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67500000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.2709999999999999,"5min":2.843,"15min":2.843},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.647,"15min":1.1899999999999999},"min":{"1min":0.67300000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.306,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.377,"15min":1.0529999999999999},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.1830000000000001,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.3440000000000001,"15min":1.1870000000000001},"min":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2290000000000001,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.4219999999999999,"15min":1.2450000000000001},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.593,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.427,"15min":1.1619999999999999},"min":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.1990000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.159,"15min":1.1180000000000001},"min":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.3129999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.4179999999999999,"15min":1.2070000000000001},"min":{"1min":0.749,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7629999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.423,"15min":1.2170000000000001},"min":{"1min":0.86099999999999999,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.3759999999999999,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313260,"statfs":{"total":96636764160,"available":96576778240,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.7849999999999999,"15min":1.1339999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.98499999999999999,"5min":125.256,"15min":125.256},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.833,"15min":1.165},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.373,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.85799999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86599999999999999,"15min":0.77100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2010000000000001,"5min":2.339,"15min":2.339},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86299999999999999,"15min":0.84299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.129,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.83999999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1120000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.86199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.3129999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.94399999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.256,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.121}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.105,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.012},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.004,"15min":0.97399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.4650000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.085}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.02,"15min":1.0029999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.05,"15min":1.02},"min":{"1min":0.432,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.234,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1539999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.077,"15min":1.016},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.4370000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.464,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.26}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90516,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281324,"statfs":{"total":96636764160,"available":96544075776,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.262,"15min":0.97099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3939999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.3200000000000001,"15min":0.96499999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.0830000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.499}]},{"osd":1,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.82199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.872,"15min":0.997},"min":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.452,"5min":1.452,"15min":3.2879999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.88800000000000001,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.746},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.0680000000000001,"15min":1.034},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"max":{"1min":10.192,"5min":10.192,"15min":10.192},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":5.3310000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95699999999999996,"15min":1.002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0609999999999999,"15min":1.125},"min":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.506,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":0.84799999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1120000000000001,"15min":1.175},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.4099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76624,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295216,"statfs":{"total":96636764160,"available":96558301184,"internally_reserved":0,"allocated":1318912,"data_stored":603914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.96799999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.85399999999999998,"5min":1.149,"15min":1.524},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.625},"min":{"1min":0.434,"5min":0.224,"15min":0.224},"max":{"1min":0.95799999999999996,"5min":1.371,"15min":1.371},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.206,"5min":1.532,"15min":1.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.052,"5min":1.284,"15min":1.284},"last":0.84799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.505,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.2609999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.61299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4610000000000001,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.97599999999999998,"15min":0.98299999999999998},"min":{"1min":0.73599999999999999,"5min":0.496,"15min":0.46899999999999997},"max":{"1min":1.319,"5min":1.536,"15min":1.6639999999999999},"last":1.373},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.413,"5min":1.5049999999999999,"15min":1.698},"last":1.294}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.995,"15min":1.0049999999999999},"min":{"1min":0.71399999999999997,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.51,"5min":1.649,"15min":1.8620000000000001},"last":1.069},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96799999999999997,"15min":0.98999999999999999},"min":{"1min":0.67200000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3660000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.2569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3779999999999999,"5min":1.835,"15min":1.835},"last":1.349},{"interface":"front","average":{"1min":1.135,"5min":1.0529999999999999,"15min":1.0609999999999999},"min":{"1min":0.77300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5629999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.097}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68360,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303480,"statfs":{"total":96636764160,"available":96566763520,"internally_reserved":0,"allocated":1310720,"data_stored":599453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.90000000000000002,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.27600000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.85099999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.997,"5min":1.1160000000000001,"15min":1.121},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.67100000000000004},"min":{"1min":0.33300000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.294,"15min":1.294},"last":0.498},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.72799999999999998},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.1100000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.1859999999999999,"5min":1.526,"15min":1.526},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.95199999999999996},"min":{"1min":0.61299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.274,"5min":1.486,"15min":1.5780000000000001},"last":0.622}]},{"osd":6,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98199999999999998,"15min":0.98399999999999999},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.615,"15min":1.615},"last":0.747},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.93300000000000005,"15min":0.94399999999999995},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.238,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91000000000000003,"15min":0.94699999999999995},"min":{"1min":0.65000000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":3.4740000000000002},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46292,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325548,"statfs":{"total":96636764160,"available":96589361152,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.1240000000000001,"5min":1.843,"15min":1.843},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.89800000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.71299999999999997},"min":{"1min":0.27900000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.86099999999999999,"5min":1.169,"15min":30.861000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.35299999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.7330000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.437},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.71099999999999997},"min":{"1min":0.33700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.151,"5min":1.151,"15min":2.3079999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86099999999999999,"15min":0.86899999999999999},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.276,"5min":2.0870000000000002,"15min":2.258},"last":0.751},{"interface":"front","average":{"1min":0.871,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2030000000000001,"5min":1.736,"15min":14.753},"last":0.63}]},{"osd":5,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":1.1160000000000001},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.528,"15min":43.103999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96299999999999997,"15min":1.1459999999999999},"min":{"1min":0.66600000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.2909999999999999,"5min":1.4279999999999999,"15min":44.753999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":1.0529999999999999},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.1639999999999999,"5min":1.6930000000000001,"15min":23.971},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93899999999999995,"15min":1.0800000000000001},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.444,"5min":1.617,"15min":14.813000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.996,"15min":1.052},"min":{"1min":0.67100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.002,"15min":1.0369999999999999},"min":{"1min":0.71899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4470000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91156,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280684,"statfs":{"total":96636764160,"available":96543420416,"internally_reserved":0,"allocated":1912832,"data_stored":1194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0569999999999999,"5min":1.145,"15min":2.6989999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.91800000000000004,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73699999999999999,"15min":0.72799999999999998},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":2.7999999999999998},"last":0.998}]},{"osd":4,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3560000000000001,"5min":1.446,"15min":4.641},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.91300000000000003},"min":{"1min":0.55600000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.131,"5min":1.286,"15min":2.028},"last":0.81699999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.624,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.53200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.27,"5min":1.405,"15min":4.7430000000000003},"last":1.034}]},{"osd":6,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2549999999999999,"15min":2.931},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.004,"5min":0.99199999999999999,"15min":1.01},"min":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.254,"5min":1.802,"15min":2.7789999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96699999999999997,"15min":1.0049999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.306,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:48.392 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:48.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.550+0000 7fa3a87e0640 1 Processor -- start 2023-11-06T03:12:48.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.550+0000 7fa3a87e0640 1 -- start start 2023-11-06T03:12:48.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3a01551e0 0x7fa3a006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa3a015e5e0 0x7fa3a00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a015f580 con 0x7fa3a015ea90 2023-11-06T03:12:48.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a015efc0 con 0x7fa3a01551e0 2023-11-06T03:12:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a015f2a0 con 0x7fa3a015e5e0 2023-11-06T03:12:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3a01551e0 0x7fa3a006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3a01551e0 0x7fa3a006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53728/0 (socket says 172.21.15.157:53728) 2023-11-06T03:12:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 -- 172.21.15.157:0/1729083723 learned_addr learned my addr 172.21.15.157:0/1729083723 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:48.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 -- 172.21.15.157:0/1729083723 >> v2:172.21.15.195:3300/0 conn(0x7fa3a015e5e0 msgr2=0x7fa3a00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a5d7b640 1 --2- 172.21.15.157:0/1729083723 >> v2:172.21.15.195:3300/0 conn(0x7fa3a015e5e0 0x7fa3a00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 --2- 172.21.15.157:0/1729083723 >> v2:172.21.15.195:3300/0 conn(0x7fa3a015e5e0 0x7fa3a00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 -- 172.21.15.157:0/1729083723 >> v1:172.21.15.157:6789/0 conn(0x7fa3a015ea90 legacy=0x7fa3a015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a557a640 1 -- 172.21.15.157:0/1729083723 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523401300 0 0) 0x7fa3a015f580 con 0x7fa3a015ea90 2023-11-06T03:12:48.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 -- 172.21.15.157:0/1729083723 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a006f5a0 con 0x7fa3a01551e0 2023-11-06T03:12:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a657c640 1 --2- 172.21.15.157:0/1729083723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a01551e0 0x7fa3a006fda0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0066880 tx=0x7fa39c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d783b74cb36410 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a557a640 1 -- 172.21.15.157:0/1729083723 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c004230 con 0x7fa3a01551e0 2023-11-06T03:12:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a557a640 1 -- 172.21.15.157:0/1729083723 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0a7050 con 0x7fa3a01551e0 2023-11-06T03:12:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a557a640 1 -- 172.21.15.157:0/1729083723 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003620 con 0x7fa3a01551e0 2023-11-06T03:12:48.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- 172.21.15.157:0/1729083723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a01551e0 msgr2=0x7fa3a006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/1729083723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a01551e0 0x7fa3a006fda0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0066880 tx=0x7fa39c01c440 comp rx=0 tx=0).stop 2023-11-06T03:12:48.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- 172.21.15.157:0/1729083723 shutdown_connections 2023-11-06T03:12:48.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/1729083723 >> v2:172.21.15.195:3300/0 conn(0x7fa3a015e5e0 0x7fa3a00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/1729083723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a01551e0 0x7fa3a006fda0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- 172.21.15.157:0/1729083723 >> 172.21.15.157:0/1729083723 conn(0x7fa3a0065930 msgr2=0x7fa3a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- 172.21.15.157:0/1729083723 shutdown_connections 2023-11-06T03:12:48.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.554+0000 7fa3a87e0640 1 -- 172.21.15.157:0/1729083723 wait complete. 2023-11-06T03:12:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 Processor -- start 2023-11-06T03:12:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- start start 2023-11-06T03:12:48.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 0x7fa3a014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 0x7fa3a00cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0157fd0 con 0x7fa3a015ea90 2023-11-06T03:12:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a014d5f0 con 0x7fa3a01551e0 2023-11-06T03:12:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a01567d0 con 0x7fa3a015e5e0 2023-11-06T03:12:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53740/0 (socket says 172.21.15.157:53740) 2023-11-06T03:12:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 -- 172.21.15.157:0/709454152 learned_addr learned my addr 172.21.15.157:0/709454152 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a657c640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 0x7fa3a014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a5d7b640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 0x7fa3a00cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 -- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 msgr2=0x7fa3a00cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 0x7fa3a00cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 -- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 msgr2=0x7fa3a014b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 0x7fa3a014b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 -- 172.21.15.157:0/709454152 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0094d0 con 0x7fa3a015ea90 2023-11-06T03:12:48.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a657c640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 0x7fa3a014b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:48.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a5d7b640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 0x7fa3a00cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:48.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a6d7d640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa398002240 tx=0x7fa39800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398003db0 con 0x7fa3a015ea90 2023-11-06T03:12:48.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39800ce30 con 0x7fa3a015ea90 2023-11-06T03:12:48.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa398005270 con 0x7fa3a015ea90 2023-11-06T03:12:48.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a01584d0 con 0x7fa3a015ea90 2023-11-06T03:12:48.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a0157030 con 0x7fa3a015ea90 2023-11-06T03:12:48.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.558+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a0158970 con 0x7fa3a015ea90 2023-11-06T03:12:48.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.562+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa3980053d0 con 0x7fa3a015ea90 2023-11-06T03:12:48.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.562+0000 7fa396ffd640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 0x7fa384067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.562+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3980829d0 con 0x7fa3a015ea90 2023-11-06T03:12:48.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.562+0000 7fa3a657c640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 0x7fa384067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.562+0000 7fa3a657c640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 0x7fa384067f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0ad000 tx=0x7fa39c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.566+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa3a0158970 con 0x7fa3a015ea90 2023-11-06T03:12:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.714+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3a00b3d60 con 0x7fa384065a60 2023-11-06T03:12:48.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.738+0000 7fa396ffd640 1 -- 172.21.15.157:0/709454152 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299741 (secure 0 0 0) 0x7fa39c0bb090 con 0x7fa384065a60 2023-11-06T03:12:48.738 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:48.741 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:48.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 msgr2=0x7fa384067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 0x7fa384067f20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0ad000 tx=0x7fa39c008040 comp rx=0 tx=0).stop 2023-11-06T03:12:48.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 msgr2=0x7fa3a00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa398002240 tx=0x7fa39800cbe0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 shutdown_connections 2023-11-06T03:12:48.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa384065a60 0x7fa384067f20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3a015ea90 0x7fa3a00cdfb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3a015e5e0 0x7fa3a00cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 --2- 172.21.15.157:0/709454152 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a01551e0 0x7fa3a014b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 >> 172.21.15.157:0/709454152 conn(0x7fa3a0065930 msgr2=0x7fa3a014e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 shutdown_connections 2023-11-06T03:12:48.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.742+0000 7fa3a87e0640 1 -- 172.21.15.157:0/709454152 wait complete. 2023-11-06T03:12:48.759 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-11-06T03:12:47.284751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":599196,"kb_used_data":11484,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754375524,"statfs":{"total":773094113280,"available":772480536576,"internally_reserved":0,"allocated":11759616,"data_stored":6017833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001137"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983314+0000","last_change":"2023-11-06T03:12:43.983314+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983314+0000","last_undegraded":"2023-11-06T03:12:43.983314+0000","last_fullsized":"2023-11-06T03:12:43.983314+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983047+0000","last_change":"2023-11-06T03:12:43.983047+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983047+0000","last_undegraded":"2023-11-06T03:12:43.983047+0000","last_fullsized":"2023-11-06T03:12:43.983047+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985377+0000","last_change":"2023-11-06T03:12:43.985377+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985377+0000","last_undegraded":"2023-11-06T03:12:43.985377+0000","last_fullsized":"2023-11-06T03:12:43.985377+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986351+0000","last_change":"2023-11-06T03:12:43.986351+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986351+0000","last_undegraded":"2023-11-06T03:12:43.986351+0000","last_fullsized":"2023-11-06T03:12:43.986351+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989535+0000","last_change":"2023-11-06T03:12:43.989535+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989535+0000","last_undegraded":"2023-11-06T03:12:43.989535+0000","last_fullsized":"2023-11-06T03:12:43.989535+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989081+0000","last_change":"2023-11-06T03:12:43.989081+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989081+0000","last_undegraded":"2023-11-06T03:12:43.989081+0000","last_fullsized":"2023-11-06T03:12:43.989081+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984283+0000","last_change":"2023-11-06T03:12:43.984283+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984283+0000","last_undegraded":"2023-11-06T03:12:43.984283+0000","last_fullsized":"2023-11-06T03:12:43.984283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983426+0000","last_change":"2023-11-06T03:12:43.983426+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983426+0000","last_undegraded":"2023-11-06T03:12:43.983426+0000","last_fullsized":"2023-11-06T03:12:43.983426+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984370+0000","last_change":"2023-11-06T03:12:43.984370+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984370+0000","last_undegraded":"2023-11-06T03:12:43.984370+0000","last_fullsized":"2023-11-06T03:12:43.984370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986505+0000","last_change":"2023-11-06T03:12:43.986505+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986505+0000","last_undegraded":"2023-11-06T03:12:43.986505+0000","last_fullsized":"2023-11-06T03:12:43.986505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990818+0000","last_change":"2023-11-06T03:12:43.990818+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990818+0000","last_undegraded":"2023-11-06T03:12:43.990818+0000","last_fullsized":"2023-11-06T03:12:43.990818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992708+0000","last_change":"2023-11-06T03:12:43.992708+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992708+0000","last_undegraded":"2023-11-06T03:12:43.992708+0000","last_fullsized":"2023-11-06T03:12:43.992708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986256+0000","last_change":"2023-11-06T03:12:43.986256+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986256+0000","last_undegraded":"2023-11-06T03:12:43.986256+0000","last_fullsized":"2023-11-06T03:12:43.986256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983563+0000","last_change":"2023-11-06T03:12:43.983563+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983563+0000","last_undegraded":"2023-11-06T03:12:43.983563+0000","last_fullsized":"2023-11-06T03:12:43.983563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981947+0000","last_change":"2023-11-06T03:12:43.981947+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981947+0000","last_undegraded":"2023-11-06T03:12:43.981947+0000","last_fullsized":"2023-11-06T03:12:43.981947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984220+0000","last_change":"2023-11-06T03:12:43.984220+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984220+0000","last_undegraded":"2023-11-06T03:12:43.984220+0000","last_fullsized":"2023-11-06T03:12:43.984220+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986358+0000","last_change":"2023-11-06T03:12:43.986358+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986358+0000","last_undegraded":"2023-11-06T03:12:43.986358+0000","last_fullsized":"2023-11-06T03:12:43.986358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985941+0000","last_change":"2023-11-06T03:12:43.985941+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985941+0000","last_undegraded":"2023-11-06T03:12:43.985941+0000","last_fullsized":"2023-11-06T03:12:43.985941+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.991077+0000","last_change":"2023-11-06T03:12:43.991077+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.991077+0000","last_undegraded":"2023-11-06T03:12:43.991077+0000","last_fullsized":"2023-11-06T03:12:43.991077+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.987189+0000","last_change":"2023-11-06T03:12:43.987189+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.987189+0000","last_undegraded":"2023-11-06T03:12:43.987189+0000","last_fullsized":"2023-11-06T03:12:43.987189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984408+0000","last_change":"2023-11-06T03:12:43.984408+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984408+0000","last_undegraded":"2023-11-06T03:12:43.984408+0000","last_fullsized":"2023-11-06T03:12:43.984408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980853+0000","last_change":"2023-11-06T03:12:43.980853+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980853+0000","last_undegraded":"2023-11-06T03:12:43.980853+0000","last_fullsized":"2023-11-06T03:12:43.980853+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990913+0000","last_change":"2023-11-06T03:12:43.990913+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990913+0000","last_undegraded":"2023-11-06T03:12:43.990913+0000","last_fullsized":"2023-11-06T03:12:43.990913+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980645+0000","last_change":"2023-11-06T03:12:43.980645+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980645+0000","last_undegraded":"2023-11-06T03:12:43.980645+0000","last_fullsized":"2023-11-06T03:12:43.980645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.982231+0000","last_change":"2023-11-06T03:12:43.982231+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.982231+0000","last_undegraded":"2023-11-06T03:12:43.982231+0000","last_fullsized":"2023-11-06T03:12:43.982231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116826+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:12:43.116826+0000","last_peered":"2023-11-06T03:12:43.116826+0000","last_clean":"2023-11-06T03:12:43.116826+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:12:43.116826+0000","last_undegraded":"2023-11-06T03:12:43.116826+0000","last_fullsized":"2023-11-06T03:12:43.116826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421147+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:12:42.421147+0000","last_peered":"2023-11-06T03:12:42.421147+0000","last_clean":"2023-11-06T03:12:42.421147+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:12:42.421147+0000","last_undegraded":"2023-11-06T03:12:42.421147+0000","last_fullsized":"2023-11-06T03:12:42.421147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989606+0000","last_change":"2023-11-06T03:12:43.989606+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989606+0000","last_undegraded":"2023-11-06T03:12:43.989606+0000","last_fullsized":"2023-11-06T03:12:43.989606+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116807+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:12:43.116807+0000","last_peered":"2023-11-06T03:12:43.116807+0000","last_clean":"2023-11-06T03:12:43.116807+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:12:43.116807+0000","last_undegraded":"2023-11-06T03:12:43.116807+0000","last_fullsized":"2023-11-06T03:12:43.116807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989239+0000","last_change":"2023-11-06T03:12:43.989239+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989239+0000","last_undegraded":"2023-11-06T03:12:43.989239+0000","last_fullsized":"2023-11-06T03:12:43.989239+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.360812+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:12:43.360812+0000","last_peered":"2023-11-06T03:12:43.360812+0000","last_clean":"2023-11-06T03:12:43.360812+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:12:43.360812+0000","last_undegraded":"2023-11-06T03:12:43.360812+0000","last_fullsized":"2023-11-06T03:12:43.360812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992045+0000","last_change":"2023-11-06T03:12:43.992045+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992045+0000","last_undegraded":"2023-11-06T03:12:43.992045+0000","last_fullsized":"2023-11-06T03:12:43.992045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421148+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:12:42.421148+0000","last_peered":"2023-11-06T03:12:42.421148+0000","last_clean":"2023-11-06T03:12:42.421148+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:12:42.421148+0000","last_undegraded":"2023-11-06T03:12:42.421148+0000","last_fullsized":"2023-11-06T03:12:42.421148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992196+0000","last_change":"2023-11-06T03:12:43.992196+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992196+0000","last_undegraded":"2023-11-06T03:12:43.992196+0000","last_fullsized":"2023-11-06T03:12:43.992196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.992408+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:12:42.992408+0000","last_peered":"2023-11-06T03:12:42.992408+0000","last_clean":"2023-11-06T03:12:42.992408+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:12:42.992408+0000","last_undegraded":"2023-11-06T03:12:42.992408+0000","last_fullsized":"2023-11-06T03:12:42.992408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981183+0000","last_change":"2023-11-06T03:12:43.981183+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981183+0000","last_undegraded":"2023-11-06T03:12:43.981183+0000","last_fullsized":"2023-11-06T03:12:43.981183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116732+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:12:43.116732+0000","last_peered":"2023-11-06T03:12:43.116732+0000","last_clean":"2023-11-06T03:12:43.116732+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:12:43.116732+0000","last_undegraded":"2023-11-06T03:12:43.116732+0000","last_fullsized":"2023-11-06T03:12:43.116732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981798+0000","last_change":"2023-11-06T03:12:43.981798+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981798+0000","last_undegraded":"2023-11-06T03:12:43.981798+0000","last_fullsized":"2023-11-06T03:12:43.981798+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.987670+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:12:43.987670+0000","last_peered":"2023-11-06T03:12:43.987670+0000","last_clean":"2023-11-06T03:12:43.987670+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:12:43.987670+0000","last_undegraded":"2023-11-06T03:12:43.987670+0000","last_fullsized":"2023-11-06T03:12:43.987670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981793+0000","last_change":"2023-11-06T03:12:43.981793+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981793+0000","last_undegraded":"2023-11-06T03:12:43.981793+0000","last_fullsized":"2023-11-06T03:12:43.981793+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.993491+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:12:43.993491+0000","last_peered":"2023-11-06T03:12:43.993491+0000","last_clean":"2023-11-06T03:12:43.993491+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:12:43.993491+0000","last_undegraded":"2023-11-06T03:12:43.993491+0000","last_fullsized":"2023-11-06T03:12:43.993491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87000,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284840,"statfs":{"total":96636764160,"available":96547676160,"internally_reserved":0,"allocated":1916928,"data_stored":1195301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.117,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83299999999999996,"15min":0.86599999999999999},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.367,"15min":1.397},"last":1.1020000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.84899999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2490000000000001,"5min":1.254,"15min":1.444},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.218,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.92900000000000005,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1399999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0820000000000001,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.66600000000000004,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":2.512,"15min":2.512},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.1299999999999999,"15min":1.0580000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.1799999999999999,"5min":10.596,"15min":10.596},"last":1.1899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.2709999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.54,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.01}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.056,"15min":1.0469999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.85499999999999998,"5min":0.68600000000000005,"15min":0.55200000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.345}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.171,"15min":1.1579999999999999},"min":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.8149999999999999,"5min":2.548,"15min":2.548},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.7370000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.149}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80668,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":79360,"kb_avail":94291172,"statfs":{"total":96636764160,"available":96554160128,"internally_reserved":0,"allocated":1331200,"data_stored":610084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81264640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.98199999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.1870000000000001,"5min":16.073,"15min":16.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.288,"15min":0.97799999999999998},"min":{"1min":0.53100000000000003,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":40.905000000000001,"15min":40.905000000000001},"last":1.038}]},{"osd":1,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.056,"15min":0.90200000000000002},"min":{"1min":0.55800000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2370000000000001,"5min":16.012,"15min":16.012},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90800000000000003,"15min":0.88200000000000001},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.409,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.137,"15min":0.91000000000000003},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.111,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67500000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.2709999999999999,"5min":2.843,"15min":2.843},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.647,"15min":1.1899999999999999},"min":{"1min":0.67300000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.306,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.377,"15min":1.0529999999999999},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.1830000000000001,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.3440000000000001,"15min":1.1870000000000001},"min":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2290000000000001,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.4219999999999999,"15min":1.2450000000000001},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.593,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.427,"15min":1.1619999999999999},"min":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.1990000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.159,"15min":1.1180000000000001},"min":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.3129999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.4179999999999999,"15min":1.2070000000000001},"min":{"1min":0.749,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7629999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.423,"15min":1.2170000000000001},"min":{"1min":0.86099999999999999,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.3759999999999999,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313260,"statfs":{"total":96636764160,"available":96576778240,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.7849999999999999,"15min":1.1339999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.98499999999999999,"5min":125.256,"15min":125.256},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.833,"15min":1.165},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.373,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.85799999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86599999999999999,"15min":0.77100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2010000000000001,"5min":2.339,"15min":2.339},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86299999999999999,"15min":0.84299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.129,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.83999999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1120000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.86199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.3129999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.94399999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.256,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.121}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.105,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.012},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.004,"15min":0.97399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.4650000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.085}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.02,"15min":1.0029999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.05,"15min":1.02},"min":{"1min":0.432,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.234,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1539999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.077,"15min":1.016},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.4370000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.464,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.26}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90516,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281324,"statfs":{"total":96636764160,"available":96544075776,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.262,"15min":0.97099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3939999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.3200000000000001,"15min":0.96499999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.0830000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.499}]},{"osd":1,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.82199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.872,"15min":0.997},"min":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.452,"5min":1.452,"15min":3.2879999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.88800000000000001,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.746},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.0680000000000001,"15min":1.034},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"max":{"1min":10.192,"5min":10.192,"15min":10.192},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":5.3310000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95699999999999996,"15min":1.002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0609999999999999,"15min":1.125},"min":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.506,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":0.84799999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1120000000000001,"15min":1.175},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.4099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76624,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295216,"statfs":{"total":96636764160,"available":96558301184,"internally_reserved":0,"allocated":1318912,"data_stored":603914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.96799999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.85399999999999998,"5min":1.149,"15min":1.524},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.625},"min":{"1min":0.434,"5min":0.224,"15min":0.224},"max":{"1min":0.95799999999999996,"5min":1.371,"15min":1.371},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.206,"5min":1.532,"15min":1.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.052,"5min":1.284,"15min":1.284},"last":0.84799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.505,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.2609999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.61299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4610000000000001,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.97599999999999998,"15min":0.98299999999999998},"min":{"1min":0.73599999999999999,"5min":0.496,"15min":0.46899999999999997},"max":{"1min":1.319,"5min":1.536,"15min":1.6639999999999999},"last":1.373},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.413,"5min":1.5049999999999999,"15min":1.698},"last":1.294}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.995,"15min":1.0049999999999999},"min":{"1min":0.71399999999999997,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.51,"5min":1.649,"15min":1.8620000000000001},"last":1.069},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96799999999999997,"15min":0.98999999999999999},"min":{"1min":0.67200000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3660000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.2569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3779999999999999,"5min":1.835,"15min":1.835},"last":1.349},{"interface":"front","average":{"1min":1.135,"5min":1.0529999999999999,"15min":1.0609999999999999},"min":{"1min":0.77300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5629999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.097}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68360,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303480,"statfs":{"total":96636764160,"available":96566763520,"internally_reserved":0,"allocated":1310720,"data_stored":599453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.90000000000000002,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.27600000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.85099999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.997,"5min":1.1160000000000001,"15min":1.121},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.67100000000000004},"min":{"1min":0.33300000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.294,"15min":1.294},"last":0.498},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.72799999999999998},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.1100000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.1859999999999999,"5min":1.526,"15min":1.526},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.95199999999999996},"min":{"1min":0.61299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.274,"5min":1.486,"15min":1.5780000000000001},"last":0.622}]},{"osd":6,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98199999999999998,"15min":0.98399999999999999},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.615,"15min":1.615},"last":0.747},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.93300000000000005,"15min":0.94399999999999995},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.238,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91000000000000003,"15min":0.94699999999999995},"min":{"1min":0.65000000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":3.4740000000000002},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46292,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325548,"statfs":{"total":96636764160,"available":96589361152,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.1240000000000001,"5min":1.843,"15min":1.843},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.89800000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.71299999999999997},"min":{"1min":0.27900000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.86099999999999999,"5min":1.169,"15min":30.861000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.35299999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.7330000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.437},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.71099999999999997},"min":{"1min":0.33700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.151,"5min":1.151,"15min":2.3079999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86099999999999999,"15min":0.86899999999999999},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.276,"5min":2.0870000000000002,"15min":2.258},"last":0.751},{"interface":"front","average":{"1min":0.871,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2030000000000001,"5min":1.736,"15min":14.753},"last":0.63}]},{"osd":5,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":1.1160000000000001},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.528,"15min":43.103999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96299999999999997,"15min":1.1459999999999999},"min":{"1min":0.66600000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.2909999999999999,"5min":1.4279999999999999,"15min":44.753999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":1.0529999999999999},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.1639999999999999,"5min":1.6930000000000001,"15min":23.971},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93899999999999995,"15min":1.0800000000000001},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.444,"5min":1.617,"15min":14.813000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.996,"15min":1.052},"min":{"1min":0.67100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.002,"15min":1.0369999999999999},"min":{"1min":0.71899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4470000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91156,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280684,"statfs":{"total":96636764160,"available":96543420416,"internally_reserved":0,"allocated":1912832,"data_stored":1194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0569999999999999,"5min":1.145,"15min":2.6989999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.91800000000000004,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73699999999999999,"15min":0.72799999999999998},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":2.7999999999999998},"last":0.998}]},{"osd":4,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3560000000000001,"5min":1.446,"15min":4.641},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.91300000000000003},"min":{"1min":0.55600000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.131,"5min":1.286,"15min":2.028},"last":0.81699999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.624,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.53200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.27,"5min":1.405,"15min":4.7430000000000003},"last":1.034}]},{"osd":6,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2549999999999999,"15min":2.931},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.004,"5min":0.99199999999999999,"15min":1.01},"min":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.254,"5min":1.802,"15min":2.7789999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96699999999999997,"15min":1.0049999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.306,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:48.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:48.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.918+0000 7fdea2b91640 1 Processor -- start 2023-11-06T03:12:48.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.918+0000 7fdea2b91640 1 -- start start 2023-11-06T03:12:48.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea2b91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde9c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea2b91640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fde9c15ea60 0x7fde9c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea2b91640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c15f580 con 0x7fde9c1551e0 2023-11-06T03:12:48.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea2b91640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0704b0 con 0x7fde9c15ea60 2023-11-06T03:12:48.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea2b91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c15efc0 con 0x7fde9c15e5e0 2023-11-06T03:12:48.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fde9c1551e0 0x7fde9c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32810/0 (socket says 172.21.15.157:32810) 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 -- 172.21.15.157:0/79075735 learned_addr learned my addr 172.21.15.157:0/79075735 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea112e640 1 --2- 172.21.15.157:0/79075735 >> v2:172.21.15.195:3300/0 conn(0x7fde9c15ea60 0x7fde9c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9bfff640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde9c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665092634 0 0) 0x7fde9c15f580 con 0x7fde9c1551e0 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde88003530 con 0x7fde9c1551e0 2023-11-06T03:12:48.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1159641055 0 0) 0x7fde88003530 con 0x7fde9c1551e0 2023-11-06T03:12:48.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 msgr2=0x7fde9c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde9c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 >> v2:172.21.15.195:3300/0 conn(0x7fde9c15ea60 msgr2=0x7fde9c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 --2- 172.21.15.157:0/79075735 >> v2:172.21.15.195:3300/0 conn(0x7fde9c15ea60 0x7fde9c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c15f2a0 con 0x7fde9c1551e0 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fde8c001680 con 0x7fde9c1551e0 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 >> v1:172.21.15.157:6789/0 conn(0x7fde9c1551e0 legacy=0x7fde9c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde88004d10 0x7fde88005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0704b0 con 0x7fde88003980 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c15efc0 con 0x7fde88004d10 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8800b9c0 con 0x7fde9c15e5e0 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fde9bfff640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea112e640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde88004d10 0x7fde88005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 -- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 msgr2=0x7fde880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c15e5e0 0x7fde880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 -- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde88004d10 msgr2=0x7fde88005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:48.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde88004d10 0x7fde88005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.922+0000 7fdea092d640 1 -- 172.21.15.157:0/79075735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88003530 con 0x7fde88003980 2023-11-06T03:12:48.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde88003da0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c290 tx=0x7fde8c01ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6532bc02b6e15045 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0058e0 con 0x7fde88003980 2023-11-06T03:12:48.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c005a40 con 0x7fde88003980 2023-11-06T03:12:48.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde9b7fe640 1 -- 172.21.15.157:0/79075735 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c005df0 con 0x7fde88003980 2023-11-06T03:12:48.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- 172.21.15.157:0/79075735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 msgr2=0x7fde88003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 --2- 172.21.15.157:0/79075735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde88003da0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c290 tx=0x7fde8c01ccb0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea112e640 1 -- 172.21.15.157:0/79075735 reap_dead start 2023-11-06T03:12:48.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- 172.21.15.157:0/79075735 shutdown_connections 2023-11-06T03:12:48.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- 172.21.15.157:0/79075735 >> 172.21.15.157:0/79075735 conn(0x7fde9c065930 msgr2=0x7fde9c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- 172.21.15.157:0/79075735 shutdown_connections 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- 172.21.15.157:0/79075735 wait complete. 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 Processor -- start 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- start start 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 0x7fde9c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c145f20 0x7fde9c1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1468e0 con 0x7fde88003980 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1f3930 con 0x7fde9c156ce0 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea2b91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1577a0 con 0x7fde9c145f20 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53758/0 (socket says 172.21.15.157:53758) 2023-11-06T03:12:48.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 -- 172.21.15.157:0/3578632996 learned_addr learned my addr 172.21.15.157:0/3578632996 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:48.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea112e640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c145f20 0x7fde9c1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde9bfff640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 0x7fde9c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 -- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c145f20 msgr2=0x7fde9c1463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c145f20 0x7fde9c1463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 -- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 msgr2=0x7fde9c157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 0x7fde9c157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 -- 172.21.15.157:0/3578632996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c157a80 con 0x7fde88003980 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde9bfff640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 0x7fde9c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fdea092d640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0ab040 tx=0x7fde88005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.926+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0a5810 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c0a5990 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c157d60 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0a5d40 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c155830 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c072750 con 0x7fde88003980 2023-11-06T03:12:48.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fde8c0047f0 con 0x7fde88003980 2023-11-06T03:12:48.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde997fa640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 0x7fde7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:48.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde8c11cf80 con 0x7fde88003980 2023-11-06T03:12:48.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde9bfff640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 0x7fde7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:48.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.930+0000 7fde9bfff640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 0x7fde7c067f20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde9c1569e0 tx=0x7fde84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:48.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:48.934+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fde8c0ea870 con 0x7fde88003980 2023-11-06T03:12:49.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.090+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde9c1f3af0 con 0x7fde7c065a60 2023-11-06T03:12:49.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.110+0000 7fde997fa640 1 -- 172.21.15.157:0/3578632996 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299741 (secure 0 0 0) 0x7fde9c1f3af0 con 0x7fde7c065a60 2023-11-06T03:12:49.114 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:49.116 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:49.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 msgr2=0x7fde7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 0x7fde7c067f20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde9c1569e0 tx=0x7fde84008040 comp rx=0 tx=0).stop 2023-11-06T03:12:49.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 msgr2=0x7fde9c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0ab040 tx=0x7fde88005800 comp rx=0 tx=0).stop 2023-11-06T03:12:49.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 shutdown_connections 2023-11-06T03:12:49.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde7c065a60 0x7fde7c067f20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde9c145f20 0x7fde9c1463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde9c156ce0 0x7fde9c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 --2- 172.21.15.157:0/3578632996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88003980 0x7fde9c0cde00 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 >> 172.21.15.157:0/3578632996 conn(0x7fde9c065930 msgr2=0x7fde9c153990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:49.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 shutdown_connections 2023-11-06T03:12:49.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.114+0000 7fdea2b91640 1 -- 172.21.15.157:0/3578632996 wait complete. 2023-11-06T03:12:49.132 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-11-06T03:12:47.284751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":599196,"kb_used_data":11484,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754375524,"statfs":{"total":773094113280,"available":772480536576,"internally_reserved":0,"allocated":11759616,"data_stored":6017833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001137"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983314+0000","last_change":"2023-11-06T03:12:43.983314+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983314+0000","last_undegraded":"2023-11-06T03:12:43.983314+0000","last_fullsized":"2023-11-06T03:12:43.983314+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983047+0000","last_change":"2023-11-06T03:12:43.983047+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983047+0000","last_undegraded":"2023-11-06T03:12:43.983047+0000","last_fullsized":"2023-11-06T03:12:43.983047+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985377+0000","last_change":"2023-11-06T03:12:43.985377+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985377+0000","last_undegraded":"2023-11-06T03:12:43.985377+0000","last_fullsized":"2023-11-06T03:12:43.985377+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986351+0000","last_change":"2023-11-06T03:12:43.986351+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986351+0000","last_undegraded":"2023-11-06T03:12:43.986351+0000","last_fullsized":"2023-11-06T03:12:43.986351+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989535+0000","last_change":"2023-11-06T03:12:43.989535+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989535+0000","last_undegraded":"2023-11-06T03:12:43.989535+0000","last_fullsized":"2023-11-06T03:12:43.989535+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989081+0000","last_change":"2023-11-06T03:12:43.989081+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989081+0000","last_undegraded":"2023-11-06T03:12:43.989081+0000","last_fullsized":"2023-11-06T03:12:43.989081+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984283+0000","last_change":"2023-11-06T03:12:43.984283+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984283+0000","last_undegraded":"2023-11-06T03:12:43.984283+0000","last_fullsized":"2023-11-06T03:12:43.984283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983426+0000","last_change":"2023-11-06T03:12:43.983426+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983426+0000","last_undegraded":"2023-11-06T03:12:43.983426+0000","last_fullsized":"2023-11-06T03:12:43.983426+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984370+0000","last_change":"2023-11-06T03:12:43.984370+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984370+0000","last_undegraded":"2023-11-06T03:12:43.984370+0000","last_fullsized":"2023-11-06T03:12:43.984370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986505+0000","last_change":"2023-11-06T03:12:43.986505+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986505+0000","last_undegraded":"2023-11-06T03:12:43.986505+0000","last_fullsized":"2023-11-06T03:12:43.986505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990818+0000","last_change":"2023-11-06T03:12:43.990818+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990818+0000","last_undegraded":"2023-11-06T03:12:43.990818+0000","last_fullsized":"2023-11-06T03:12:43.990818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992708+0000","last_change":"2023-11-06T03:12:43.992708+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992708+0000","last_undegraded":"2023-11-06T03:12:43.992708+0000","last_fullsized":"2023-11-06T03:12:43.992708+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986256+0000","last_change":"2023-11-06T03:12:43.986256+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986256+0000","last_undegraded":"2023-11-06T03:12:43.986256+0000","last_fullsized":"2023-11-06T03:12:43.986256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.983563+0000","last_change":"2023-11-06T03:12:43.983563+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.983563+0000","last_undegraded":"2023-11-06T03:12:43.983563+0000","last_fullsized":"2023-11-06T03:12:43.983563+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981947+0000","last_change":"2023-11-06T03:12:43.981947+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981947+0000","last_undegraded":"2023-11-06T03:12:43.981947+0000","last_fullsized":"2023-11-06T03:12:43.981947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984220+0000","last_change":"2023-11-06T03:12:43.984220+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984220+0000","last_undegraded":"2023-11-06T03:12:43.984220+0000","last_fullsized":"2023-11-06T03:12:43.984220+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.986358+0000","last_change":"2023-11-06T03:12:43.986358+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.986358+0000","last_undegraded":"2023-11-06T03:12:43.986358+0000","last_fullsized":"2023-11-06T03:12:43.986358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.985941+0000","last_change":"2023-11-06T03:12:43.985941+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.985941+0000","last_undegraded":"2023-11-06T03:12:43.985941+0000","last_fullsized":"2023-11-06T03:12:43.985941+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.991077+0000","last_change":"2023-11-06T03:12:43.991077+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.991077+0000","last_undegraded":"2023-11-06T03:12:43.991077+0000","last_fullsized":"2023-11-06T03:12:43.991077+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.987189+0000","last_change":"2023-11-06T03:12:43.987189+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.987189+0000","last_undegraded":"2023-11-06T03:12:43.987189+0000","last_fullsized":"2023-11-06T03:12:43.987189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.984408+0000","last_change":"2023-11-06T03:12:43.984408+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.984408+0000","last_undegraded":"2023-11-06T03:12:43.984408+0000","last_fullsized":"2023-11-06T03:12:43.984408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980853+0000","last_change":"2023-11-06T03:12:43.980853+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980853+0000","last_undegraded":"2023-11-06T03:12:43.980853+0000","last_fullsized":"2023-11-06T03:12:43.980853+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.990913+0000","last_change":"2023-11-06T03:12:43.990913+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.990913+0000","last_undegraded":"2023-11-06T03:12:43.990913+0000","last_fullsized":"2023-11-06T03:12:43.990913+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.980645+0000","last_change":"2023-11-06T03:12:43.980645+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.980645+0000","last_undegraded":"2023-11-06T03:12:43.980645+0000","last_fullsized":"2023-11-06T03:12:43.980645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.982231+0000","last_change":"2023-11-06T03:12:43.982231+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.982231+0000","last_undegraded":"2023-11-06T03:12:43.982231+0000","last_fullsized":"2023-11-06T03:12:43.982231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116826+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:12:43.116826+0000","last_peered":"2023-11-06T03:12:43.116826+0000","last_clean":"2023-11-06T03:12:43.116826+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:12:43.116826+0000","last_undegraded":"2023-11-06T03:12:43.116826+0000","last_fullsized":"2023-11-06T03:12:43.116826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421147+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:12:42.421147+0000","last_peered":"2023-11-06T03:12:42.421147+0000","last_clean":"2023-11-06T03:12:42.421147+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:12:42.421147+0000","last_undegraded":"2023-11-06T03:12:42.421147+0000","last_fullsized":"2023-11-06T03:12:42.421147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989606+0000","last_change":"2023-11-06T03:12:43.989606+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989606+0000","last_undegraded":"2023-11-06T03:12:43.989606+0000","last_fullsized":"2023-11-06T03:12:43.989606+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116807+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:12:43.116807+0000","last_peered":"2023-11-06T03:12:43.116807+0000","last_clean":"2023-11-06T03:12:43.116807+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:12:43.116807+0000","last_undegraded":"2023-11-06T03:12:43.116807+0000","last_fullsized":"2023-11-06T03:12:43.116807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.989239+0000","last_change":"2023-11-06T03:12:43.989239+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.989239+0000","last_undegraded":"2023-11-06T03:12:43.989239+0000","last_fullsized":"2023-11-06T03:12:43.989239+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.360812+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:12:43.360812+0000","last_peered":"2023-11-06T03:12:43.360812+0000","last_clean":"2023-11-06T03:12:43.360812+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:12:43.360812+0000","last_undegraded":"2023-11-06T03:12:43.360812+0000","last_fullsized":"2023-11-06T03:12:43.360812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992045+0000","last_change":"2023-11-06T03:12:43.992045+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992045+0000","last_undegraded":"2023-11-06T03:12:43.992045+0000","last_fullsized":"2023-11-06T03:12:43.992045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.421148+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:12:42.421148+0000","last_peered":"2023-11-06T03:12:42.421148+0000","last_clean":"2023-11-06T03:12:42.421148+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:12:42.421148+0000","last_undegraded":"2023-11-06T03:12:42.421148+0000","last_fullsized":"2023-11-06T03:12:42.421148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.992196+0000","last_change":"2023-11-06T03:12:43.992196+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.992196+0000","last_undegraded":"2023-11-06T03:12:43.992196+0000","last_fullsized":"2023-11-06T03:12:43.992196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:42.992408+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:12:42.992408+0000","last_peered":"2023-11-06T03:12:42.992408+0000","last_clean":"2023-11-06T03:12:42.992408+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:12:42.992408+0000","last_undegraded":"2023-11-06T03:12:42.992408+0000","last_fullsized":"2023-11-06T03:12:42.992408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981183+0000","last_change":"2023-11-06T03:12:43.981183+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981183+0000","last_undegraded":"2023-11-06T03:12:43.981183+0000","last_fullsized":"2023-11-06T03:12:43.981183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.116732+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:12:43.116732+0000","last_peered":"2023-11-06T03:12:43.116732+0000","last_clean":"2023-11-06T03:12:43.116732+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:12:43.116732+0000","last_undegraded":"2023-11-06T03:12:43.116732+0000","last_fullsized":"2023-11-06T03:12:43.116732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981798+0000","last_change":"2023-11-06T03:12:43.981798+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981798+0000","last_undegraded":"2023-11-06T03:12:43.981798+0000","last_fullsized":"2023-11-06T03:12:43.981798+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.987670+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:12:43.987670+0000","last_peered":"2023-11-06T03:12:43.987670+0000","last_clean":"2023-11-06T03:12:43.987670+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:12:43.987670+0000","last_undegraded":"2023-11-06T03:12:43.987670+0000","last_fullsized":"2023-11-06T03:12:43.987670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:12:44.977800+0000","last_change":"2023-11-06T03:12:44.977800+0000","last_active":"2023-11-06T03:12:44.977800+0000","last_peered":"2023-11-06T03:12:44.977800+0000","last_clean":"2023-11-06T03:12:44.977800+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:44.977800+0000","last_undegraded":"2023-11-06T03:12:44.977800+0000","last_fullsized":"2023-11-06T03:12:44.977800+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2023-11-06T03:12:43.981793+0000","last_change":"2023-11-06T03:12:43.981793+0000","last_active":"2023-11-06T03:12:43.966612+0000","last_peered":"2023-11-06T03:12:43.966612+0000","last_clean":"2023-11-06T03:12:43.966612+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:12:43.981793+0000","last_undegraded":"2023-11-06T03:12:43.981793+0000","last_fullsized":"2023-11-06T03:12:43.981793+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-06T03:12:43.993491+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:12:43.993491+0000","last_peered":"2023-11-06T03:12:43.993491+0000","last_clean":"2023-11-06T03:12:43.993491+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:12:43.993491+0000","last_undegraded":"2023-11-06T03:12:43.993491+0000","last_fullsized":"2023-11-06T03:12:43.993491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87000,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284840,"statfs":{"total":96636764160,"available":96547676160,"internally_reserved":0,"allocated":1916928,"data_stored":1195301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.117,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83299999999999996,"15min":0.86599999999999999},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.367,"15min":1.397},"last":1.1020000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.84899999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2490000000000001,"5min":1.254,"15min":1.444},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.218,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.95399999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.92900000000000005,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1399999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0820000000000001,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.66600000000000004,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":2.512,"15min":2.512},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.1299999999999999,"15min":1.0580000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.1799999999999999,"5min":10.596,"15min":10.596},"last":1.1899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.2709999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.54,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.01}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.056,"15min":1.0469999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.85499999999999998,"5min":0.68600000000000005,"15min":0.55200000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.345}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.171,"15min":1.1579999999999999},"min":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.8149999999999999,"5min":2.548,"15min":2.548},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.7370000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.149}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80668,"kb_used_data":1300,"kb_used_omap":0,"kb_used_meta":79360,"kb_avail":94291172,"statfs":{"total":96636764160,"available":96554160128,"internally_reserved":0,"allocated":1331200,"data_stored":610084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81264640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.98199999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.1870000000000001,"5min":16.073,"15min":16.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.288,"15min":0.97799999999999998},"min":{"1min":0.53100000000000003,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":40.905000000000001,"15min":40.905000000000001},"last":1.038}]},{"osd":1,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.056,"15min":0.90200000000000002},"min":{"1min":0.55800000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2370000000000001,"5min":16.012,"15min":16.012},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90800000000000003,"15min":0.88200000000000001},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.409,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.137,"15min":0.91000000000000003},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.111,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67500000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.2709999999999999,"5min":2.843,"15min":2.843},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.647,"15min":1.1899999999999999},"min":{"1min":0.67300000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.306,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.377,"15min":1.0529999999999999},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.1830000000000001,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.3440000000000001,"15min":1.1870000000000001},"min":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2290000000000001,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.4219999999999999,"15min":1.2450000000000001},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.593,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.427,"15min":1.1619999999999999},"min":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.1990000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.159,"15min":1.1180000000000001},"min":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.3129999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.4179999999999999,"15min":1.2070000000000001},"min":{"1min":0.749,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7629999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.423,"15min":1.2170000000000001},"min":{"1min":0.86099999999999999,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.3759999999999999,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313260,"statfs":{"total":96636764160,"available":96576778240,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.7849999999999999,"15min":1.1339999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.98499999999999999,"5min":125.256,"15min":125.256},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.833,"15min":1.165},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.373,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.85799999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86599999999999999,"15min":0.77100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2010000000000001,"5min":2.339,"15min":2.339},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86299999999999999,"15min":0.84299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.129,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.83999999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1120000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.86199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.3129999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.94399999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.256,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.121}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.105,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.012},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.004,"15min":0.97399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.4650000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.085}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.02,"15min":1.0029999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.05,"15min":1.02},"min":{"1min":0.432,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.234,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1539999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.077,"15min":1.016},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.4370000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.464,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.26}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90516,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281324,"statfs":{"total":96636764160,"available":96544075776,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.262,"15min":0.97099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3939999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.3200000000000001,"15min":0.96499999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.0830000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.499}]},{"osd":1,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.82199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.872,"15min":0.997},"min":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.452,"5min":1.452,"15min":3.2879999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.88800000000000001,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.746},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.0680000000000001,"15min":1.034},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"max":{"1min":10.192,"5min":10.192,"15min":10.192},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":5.3310000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.90600000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95699999999999996,"15min":1.002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0609999999999999,"15min":1.125},"min":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.506,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":0.84799999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1120000000000001,"15min":1.175},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.4099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76624,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295216,"statfs":{"total":96636764160,"available":96558301184,"internally_reserved":0,"allocated":1318912,"data_stored":603914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.96799999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.85399999999999998,"5min":1.149,"15min":1.524},"last":0.73099999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.625},"min":{"1min":0.434,"5min":0.224,"15min":0.224},"max":{"1min":0.95799999999999996,"5min":1.371,"15min":1.371},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.206,"5min":1.532,"15min":1.532},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.052,"5min":1.284,"15min":1.284},"last":0.84799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.505,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.2609999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.61299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4610000000000001,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.97599999999999998,"15min":0.98299999999999998},"min":{"1min":0.73599999999999999,"5min":0.496,"15min":0.46899999999999997},"max":{"1min":1.319,"5min":1.536,"15min":1.6639999999999999},"last":1.373},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.413,"5min":1.5049999999999999,"15min":1.698},"last":1.294}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.995,"15min":1.0049999999999999},"min":{"1min":0.71399999999999997,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.51,"5min":1.649,"15min":1.8620000000000001},"last":1.069},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96799999999999997,"15min":0.98999999999999999},"min":{"1min":0.67200000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3660000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.2569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3779999999999999,"5min":1.835,"15min":1.835},"last":1.349},{"interface":"front","average":{"1min":1.135,"5min":1.0529999999999999,"15min":1.0609999999999999},"min":{"1min":0.77300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5629999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.097}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68360,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303480,"statfs":{"total":96636764160,"available":96566763520,"internally_reserved":0,"allocated":1310720,"data_stored":599453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.90000000000000002,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.27600000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.85099999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.997,"5min":1.1160000000000001,"15min":1.121},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.67100000000000004},"min":{"1min":0.33300000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.294,"15min":1.294},"last":0.498},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.72799999999999998},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.1100000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.1859999999999999,"5min":1.526,"15min":1.526},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.95199999999999996},"min":{"1min":0.61299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.274,"5min":1.486,"15min":1.5780000000000001},"last":0.622}]},{"osd":6,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98199999999999998,"15min":0.98399999999999999},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.615,"15min":1.615},"last":0.747},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.93300000000000005,"15min":0.94399999999999995},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.238,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91000000000000003,"15min":0.94699999999999995},"min":{"1min":0.65000000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":3.4740000000000002},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46292,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325548,"statfs":{"total":96636764160,"available":96589361152,"internally_reserved":0,"allocated":1323008,"data_stored":604933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.1240000000000001,"5min":1.843,"15min":1.843},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.89800000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.71299999999999997},"min":{"1min":0.27900000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.86099999999999999,"5min":1.169,"15min":30.861000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.35299999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.7330000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.437},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.71099999999999997},"min":{"1min":0.33700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.151,"5min":1.151,"15min":2.3079999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86099999999999999,"15min":0.86899999999999999},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.276,"5min":2.0870000000000002,"15min":2.258},"last":0.751},{"interface":"front","average":{"1min":0.871,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2030000000000001,"5min":1.736,"15min":14.753},"last":0.63}]},{"osd":5,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":1.1160000000000001},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.528,"15min":43.103999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96299999999999997,"15min":1.1459999999999999},"min":{"1min":0.66600000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.2909999999999999,"5min":1.4279999999999999,"15min":44.753999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":1.0529999999999999},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.1639999999999999,"5min":1.6930000000000001,"15min":23.971},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93899999999999995,"15min":1.0800000000000001},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.444,"5min":1.617,"15min":14.813000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.996,"15min":1.052},"min":{"1min":0.67100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.002,"15min":1.0369999999999999},"min":{"1min":0.71899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4470000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91156,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280684,"statfs":{"total":96636764160,"available":96543420416,"internally_reserved":0,"allocated":1912832,"data_stored":1194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0569999999999999,"5min":1.145,"15min":2.6989999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.91800000000000004,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.73699999999999999,"15min":0.72799999999999998},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":2.7999999999999998},"last":0.998}]},{"osd":4,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3560000000000001,"5min":1.446,"15min":4.641},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.91300000000000003},"min":{"1min":0.55600000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.131,"5min":1.286,"15min":2.028},"last":0.81699999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.624,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.53200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.27,"5min":1.405,"15min":4.7430000000000003},"last":1.034}]},{"osd":6,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2549999999999999,"15min":2.931},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.004,"5min":0.99199999999999999,"15min":1.01},"min":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.254,"5min":1.802,"15min":2.7789999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96699999999999997,"15min":1.0049999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.306,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.88700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:49.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-11-06T03:12:49.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2023-11-06T03:12:49.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-11-06T03:12:49.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-11-06T03:12:49.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-11-06T03:12:49.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2023-11-06T03:12:49.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2023-11-06T03:12:49.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-11-06T03:12:49.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2023-11-06T03:12:49.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-11-06T03:12:49.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-11-06T03:12:49.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-11-06T03:12:49.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-11-06T03:12:49.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-11-06T03:12:49.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-11-06T03:12:49.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-11-06T03:12:49.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-11-06T03:12:49.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2023-11-06T03:12:49.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-11-06T03:12:49.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-11-06T03:12:49.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-11-06T03:12:49.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-11-06T03:12:49.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-11-06T03:12:49.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-11-06T03:12:49.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-11-06T03:12:49.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-11-06T03:12:49.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-11-06T03:12:49.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2023-11-06T03:12:49.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-11-06T03:12:49.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-11-06T03:12:49.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2023-11-06T03:12:49.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-11-06T03:12:49.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-11-06T03:12:49.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-11-06T03:12:49.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.983314+0000', 'last_change': '2023-11-06T03:12:43.983314+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.983314+0000', 'last_undegraded': '2023-11-06T03:12:43.983314+0000', 'last_fullsized': '2023-11-06T03:12:43.983314+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:12:49.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-11-06T03:12:49.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.192 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-11-06T03:12:49.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.983047+0000', 'last_change': '2023-11-06T03:12:43.983047+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.983047+0000', 'last_undegraded': '2023-11-06T03:12:43.983047+0000', 'last_fullsized': '2023-11-06T03:12:43.983047+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-11-06T03:12:49.193 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-11-06T03:12:49.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.985377+0000', 'last_change': '2023-11-06T03:12:43.985377+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.985377+0000', 'last_undegraded': '2023-11-06T03:12:43.985377+0000', 'last_fullsized': '2023-11-06T03:12:43.985377+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:12:49.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-11-06T03:12:49.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-11-06T03:12:49.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.986351+0000', 'last_change': '2023-11-06T03:12:43.986351+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.986351+0000', 'last_undegraded': '2023-11-06T03:12:43.986351+0000', 'last_fullsized': '2023-11-06T03:12:43.986351+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:12:49.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-11-06T03:12:49.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-11-06T03:12:49.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.989535+0000', 'last_change': '2023-11-06T03:12:43.989535+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.989535+0000', 'last_undegraded': '2023-11-06T03:12:43.989535+0000', 'last_fullsized': '2023-11-06T03:12:43.989535+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:12:49.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-11-06T03:12:49.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.198 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-11-06T03:12:49.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.989081+0000', 'last_change': '2023-11-06T03:12:43.989081+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.989081+0000', 'last_undegraded': '2023-11-06T03:12:43.989081+0000', 'last_fullsized': '2023-11-06T03:12:43.989081+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:12:49.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-11-06T03:12:49.199 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.199 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-11-06T03:12:49.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.984283+0000', 'last_change': '2023-11-06T03:12:43.984283+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.984283+0000', 'last_undegraded': '2023-11-06T03:12:43.984283+0000', 'last_fullsized': '2023-11-06T03:12:43.984283+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:12:49.200 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-11-06T03:12:49.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-11-06T03:12:49.201 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.983426+0000', 'last_change': '2023-11-06T03:12:43.983426+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.983426+0000', 'last_undegraded': '2023-11-06T03:12:43.983426+0000', 'last_fullsized': '2023-11-06T03:12:43.983426+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:12:49.201 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-11-06T03:12:49.202 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.202 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-11-06T03:12:49.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.984370+0000', 'last_change': '2023-11-06T03:12:43.984370+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.984370+0000', 'last_undegraded': '2023-11-06T03:12:43.984370+0000', 'last_fullsized': '2023-11-06T03:12:43.984370+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:12:49.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-11-06T03:12:49.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-11-06T03:12:49.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.986505+0000', 'last_change': '2023-11-06T03:12:43.986505+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.986505+0000', 'last_undegraded': '2023-11-06T03:12:43.986505+0000', 'last_fullsized': '2023-11-06T03:12:43.986505+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:12:49.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-11-06T03:12:49.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-11-06T03:12:49.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.990818+0000', 'last_change': '2023-11-06T03:12:43.990818+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.990818+0000', 'last_undegraded': '2023-11-06T03:12:43.990818+0000', 'last_fullsized': '2023-11-06T03:12:43.990818+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:12:49.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-11-06T03:12:49.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-11-06T03:12:49.207 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.992708+0000', 'last_change': '2023-11-06T03:12:43.992708+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.992708+0000', 'last_undegraded': '2023-11-06T03:12:43.992708+0000', 'last_fullsized': '2023-11-06T03:12:43.992708+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:12:49.207 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-11-06T03:12:49.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.208 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-11-06T03:12:49.208 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.986256+0000', 'last_change': '2023-11-06T03:12:43.986256+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.986256+0000', 'last_undegraded': '2023-11-06T03:12:43.986256+0000', 'last_fullsized': '2023-11-06T03:12:43.986256+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:12:49.209 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-11-06T03:12:49.209 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-11-06T03:12:49.210 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.983563+0000', 'last_change': '2023-11-06T03:12:43.983563+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.983563+0000', 'last_undegraded': '2023-11-06T03:12:43.983563+0000', 'last_fullsized': '2023-11-06T03:12:43.983563+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:12:49.210 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-11-06T03:12:49.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.211 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-11-06T03:12:49.211 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.981947+0000', 'last_change': '2023-11-06T03:12:43.981947+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.981947+0000', 'last_undegraded': '2023-11-06T03:12:43.981947+0000', 'last_fullsized': '2023-11-06T03:12:43.981947+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.212 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-11-06T03:12:49.212 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-11-06T03:12:49.213 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.984220+0000', 'last_change': '2023-11-06T03:12:43.984220+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.984220+0000', 'last_undegraded': '2023-11-06T03:12:43.984220+0000', 'last_fullsized': '2023-11-06T03:12:43.984220+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:12:49.213 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-11-06T03:12:49.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.214 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-11-06T03:12:49.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.986358+0000', 'last_change': '2023-11-06T03:12:43.986358+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.986358+0000', 'last_undegraded': '2023-11-06T03:12:43.986358+0000', 'last_fullsized': '2023-11-06T03:12:43.986358+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:12:49.215 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-11-06T03:12:49.215 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-11-06T03:12:49.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.985941+0000', 'last_change': '2023-11-06T03:12:43.985941+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.985941+0000', 'last_undegraded': '2023-11-06T03:12:43.985941+0000', 'last_fullsized': '2023-11-06T03:12:43.985941+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:12:49.216 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-11-06T03:12:49.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.217 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-11-06T03:12:49.217 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.991077+0000', 'last_change': '2023-11-06T03:12:43.991077+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.991077+0000', 'last_undegraded': '2023-11-06T03:12:43.991077+0000', 'last_fullsized': '2023-11-06T03:12:43.991077+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:12:49.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-11-06T03:12:49.218 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.218 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-11-06T03:12:49.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.987189+0000', 'last_change': '2023-11-06T03:12:43.987189+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.987189+0000', 'last_undegraded': '2023-11-06T03:12:43.987189+0000', 'last_fullsized': '2023-11-06T03:12:43.987189+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:12:49.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-11-06T03:12:49.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-11-06T03:12:49.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.984408+0000', 'last_change': '2023-11-06T03:12:43.984408+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.984408+0000', 'last_undegraded': '2023-11-06T03:12:43.984408+0000', 'last_fullsized': '2023-11-06T03:12:43.984408+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:12:49.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-11-06T03:12:49.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-11-06T03:12:49.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.980853+0000', 'last_change': '2023-11-06T03:12:43.980853+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.980853+0000', 'last_undegraded': '2023-11-06T03:12:43.980853+0000', 'last_fullsized': '2023-11-06T03:12:43.980853+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-11-06T03:12:49.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-11-06T03:12:49.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.990913+0000', 'last_change': '2023-11-06T03:12:43.990913+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.990913+0000', 'last_undegraded': '2023-11-06T03:12:43.990913+0000', 'last_fullsized': '2023-11-06T03:12:43.990913+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:12:49.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-11-06T03:12:49.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-11-06T03:12:49.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.980645+0000', 'last_change': '2023-11-06T03:12:43.980645+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.980645+0000', 'last_undegraded': '2023-11-06T03:12:43.980645+0000', 'last_fullsized': '2023-11-06T03:12:43.980645+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-11-06T03:12:49.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-11-06T03:12:49.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.982231+0000', 'last_change': '2023-11-06T03:12:43.982231+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.982231+0000', 'last_undegraded': '2023-11-06T03:12:43.982231+0000', 'last_fullsized': '2023-11-06T03:12:43.982231+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-11-06T03:12:49.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-11-06T03:12:49.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.989606+0000', 'last_change': '2023-11-06T03:12:43.989606+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.989606+0000', 'last_undegraded': '2023-11-06T03:12:43.989606+0000', 'last_fullsized': '2023-11-06T03:12:43.989606+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:12:49.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-11-06T03:12:49.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-11-06T03:12:49.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.989239+0000', 'last_change': '2023-11-06T03:12:43.989239+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.989239+0000', 'last_undegraded': '2023-11-06T03:12:43.989239+0000', 'last_fullsized': '2023-11-06T03:12:43.989239+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:12:49.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-11-06T03:12:49.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-11-06T03:12:49.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.992045+0000', 'last_change': '2023-11-06T03:12:43.992045+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.992045+0000', 'last_undegraded': '2023-11-06T03:12:43.992045+0000', 'last_fullsized': '2023-11-06T03:12:43.992045+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:12:49.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-11-06T03:12:49.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-11-06T03:12:49.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.992196+0000', 'last_change': '2023-11-06T03:12:43.992196+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.992196+0000', 'last_undegraded': '2023-11-06T03:12:43.992196+0000', 'last_fullsized': '2023-11-06T03:12:43.992196+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:12:49.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-11-06T03:12:49.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-11-06T03:12:49.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.981183+0000', 'last_change': '2023-11-06T03:12:43.981183+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.981183+0000', 'last_undegraded': '2023-11-06T03:12:43.981183+0000', 'last_fullsized': '2023-11-06T03:12:43.981183+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-11-06T03:12:49.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-11-06T03:12:49.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.981798+0000', 'last_change': '2023-11-06T03:12:43.981798+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.981798+0000', 'last_undegraded': '2023-11-06T03:12:43.981798+0000', 'last_fullsized': '2023-11-06T03:12:43.981798+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-11-06T03:12:49.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:12:44.977800+0000', 'last_change': '2023-11-06T03:12:44.977800+0000', 'last_active': '2023-11-06T03:12:44.977800+0000', 'last_peered': '2023-11-06T03:12:44.977800+0000', 'last_clean': '2023-11-06T03:12:44.977800+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:44.977800+0000', 'last_undegraded': '2023-11-06T03:12:44.977800+0000', 'last_fullsized': '2023-11-06T03:12:44.977800+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:44.977800+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:12:49.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-11-06T03:12:49.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:12:43.981793+0000', 'last_change': '2023-11-06T03:12:43.981793+0000', 'last_active': '2023-11-06T03:12:43.966612+0000', 'last_peered': '2023-11-06T03:12:43.966612+0000', 'last_clean': '2023-11-06T03:12:43.966612+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:12:43.981793+0000', 'last_undegraded': '2023-11-06T03:12:43.981793+0000', 'last_fullsized': '2023-11-06T03:12:43.981793+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'last_clean_scrub_stamp': '2023-11-06T03:12:43.966612+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:12:49.236 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:12:49.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.358+0000 7ffbddaba640 1 Processor -- start 2023-11-06T03:12:49.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.358+0000 7ffbddaba640 1 -- start start 2023-11-06T03:12:49.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbddaba640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffbd80b2af0 0x7ffbd80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbddaba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 0x7ffbd8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbddaba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd815e440 con 0x7ffbd81599c0 2023-11-06T03:12:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbddaba640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd806c240 con 0x7ffbd80b2af0 2023-11-06T03:12:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbddaba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd806c520 con 0x7ffbd806d000 2023-11-06T03:12:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffbd81599c0 0x7ffbd80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32818/0 (socket says 172.21.15.157:32818) 2023-11-06T03:12:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 -- 172.21.15.157:0/1855564147 learned_addr learned my addr 172.21.15.157:0/1855564147 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:49.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721017775 0 0) 0x7ffbd815e440 con 0x7ffbd81599c0 2023-11-06T03:12:49.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbcffff640 1 --2- 172.21.15.157:0/1855564147 >> v2:172.21.15.195:3300/0 conn(0x7ffbd80b2af0 0x7ffbd80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc4003530 con 0x7ffbd81599c0 2023-11-06T03:12:49.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd7fff640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 0x7ffbd8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257793089 0 0) 0x7ffbc4003530 con 0x7ffbd81599c0 2023-11-06T03:12:49.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 msgr2=0x7ffbd8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 0x7ffbd8159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 >> v2:172.21.15.195:3300/0 conn(0x7ffbd80b2af0 msgr2=0x7ffbd80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 --2- 172.21.15.157:0/1855564147 >> v2:172.21.15.195:3300/0 conn(0x7ffbd80b2af0 0x7ffbd80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd806c800 con 0x7ffbd81599c0 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ffbc8001680 con 0x7ffbd81599c0 2023-11-06T03:12:49.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 >> v1:172.21.15.157:6789/0 conn(0x7ffbd81599c0 legacy=0x7ffbd80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbc4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 0x7ffbc4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd806c240 con 0x7ffbc4003980 2023-11-06T03:12:49.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd806c520 con 0x7ffbc40047d0 2023-11-06T03:12:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc400b990 con 0x7ffbd806d000 2023-11-06T03:12:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbc4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbcffff640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 -- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 msgr2=0x7ffbc4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd806d000 0x7ffbc4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 -- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40047d0 msgr2=0x7ffbc4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbd77fe640 1 -- 172.21.15.157:0/1855564147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4003530 con 0x7ffbc4003980 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.362+0000 7ffbcffff640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbc4003da0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c210 tx=0x7ffbc801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1741593329c3f4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80058e0 con 0x7ffbc4003980 2023-11-06T03:12:49.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8005a40 con 0x7ffbc4003980 2023-11-06T03:12:49.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd6ffd640 1 -- 172.21.15.157:0/1855564147 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8005df0 con 0x7ffbc4003980 2023-11-06T03:12:49.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/1855564147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 msgr2=0x7ffbc4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 --2- 172.21.15.157:0/1855564147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbc4003da0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c210 tx=0x7ffbc801cc10 comp rx=0 tx=0).stop 2023-11-06T03:12:49.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd7fff640 1 -- 172.21.15.157:0/1855564147 reap_dead start 2023-11-06T03:12:49.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/1855564147 shutdown_connections 2023-11-06T03:12:49.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/1855564147 >> 172.21.15.157:0/1855564147 conn(0x7ffbd8065930 msgr2=0x7ffbd8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:49.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/1855564147 shutdown_connections 2023-11-06T03:12:49.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/1855564147 wait complete. 2023-11-06T03:12:49.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 Processor -- start 2023-11-06T03:12:49.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- start start 2023-11-06T03:12:49.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd815c2b0 0x7ffbd8152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbd814de70 0x7ffbd80af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81533d0 con 0x7ffbc4003980 2023-11-06T03:12:49.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81535c0 con 0x7ffbd814de70 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8151ec0 con 0x7ffbd815c2b0 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53766/0 (socket says 172.21.15.157:53766) 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 -- 172.21.15.157:0/3369428669 learned_addr learned my addr 172.21.15.157:0/3369428669 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbcffff640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd815c2b0 0x7ffbd8152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd7fff640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbd814de70 0x7ffbd80af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 -- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd815c2b0 msgr2=0x7ffbd8152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd815c2b0 0x7ffbd8152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 -- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbd814de70 msgr2=0x7ffbd80af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbd814de70 0x7ffbd80af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 -- 172.21.15.157:0/3369428669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd81521a0 con 0x7ffbc4003980 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd77fe640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80ab040 tx=0x7ffbc4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8001c80 con 0x7ffbc4003980 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8004e40 con 0x7ffbc4003980 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd815bd90 con 0x7ffbc4003980 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8001440 con 0x7ffbc4003980 2023-11-06T03:12:49.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.366+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd8152480 con 0x7ffbc4003980 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8072750 con 0x7ffbc4003980 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffbc80a5730 con 0x7ffbc4003980 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbd4ff9640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 0x7ffbb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbc811c920 con 0x7ffbc4003980 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbcffff640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 0x7ffbb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.370+0000 7ffbcffff640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 0x7ffbb8067f50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffbd815c690 tx=0x7ffbc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.374+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffbc80ea210 con 0x7ffbc4003980 2023-11-06T03:12:49.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.574+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ffbd815bf20 con 0x7ffbc4003980 2023-11-06T03:12:49.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbd4ff9640 1 -- 172.21.15.157:0/3369428669 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1435 (secure 0 0 0) 0x7ffbc80bd5b0 con 0x7ffbc4003980 2023-11-06T03:12:49.577 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:49.577 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592730,"bytes_used":614035456,"bytes_avail":772480077824,"bytes_total":773094113280,"write_bytes_sec":2702,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2023-11-06T03:12:35.281372+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:12:49.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 msgr2=0x7ffbb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 0x7ffbb8067f50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffbd815c690 tx=0x7ffbc0008040 comp rx=0 tx=0).stop 2023-11-06T03:12:49.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 msgr2=0x7ffbd814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80ab040 tx=0x7ffbc4005800 comp rx=0 tx=0).stop 2023-11-06T03:12:49.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 shutdown_connections 2023-11-06T03:12:49.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffbb8065a90 0x7ffbb8067f50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffbd814de70 0x7ffbd80af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffbd815c2b0 0x7ffbd8152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 --2- 172.21.15.157:0/3369428669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbc4003980 0x7ffbd814b7b0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 >> 172.21.15.157:0/3369428669 conn(0x7ffbd8065930 msgr2=0x7ffbd80ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:49.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 shutdown_connections 2023-11-06T03:12:49.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.578+0000 7ffbddaba640 1 -- 172.21.15.157:0/3369428669 wait complete. 2023-11-06T03:12:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:12:49.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.714+0000 7f8066596640 1 Processor -- start 2023-11-06T03:12:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.714+0000 7f8066596640 1 -- start start 2023-11-06T03:12:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8066596640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80600ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8066596640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806015e440 con 0x7f80600b2af0 2023-11-06T03:12:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8066596640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806006c240 con 0x7f806006d000 2023-11-06T03:12:49.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8066596640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806006c520 con 0x7f80601599c0 2023-11-06T03:12:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f80600b2af0 0x7f8060159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32824/0 (socket says 172.21.15.157:32824) 2023-11-06T03:12:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 -- 172.21.15.157:0/2098845938 learned_addr learned my addr 172.21.15.157:0/2098845938 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805ffff640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80600ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606436591 0 0) 0x7f806015e440 con 0x7f80600b2af0 2023-11-06T03:12:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8048003530 con 0x7f80600b2af0 2023-11-06T03:12:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 344424262 0 0) 0x7f8048003530 con 0x7f80600b2af0 2023-11-06T03:12:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 msgr2=0x7f80600ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80600ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 >> v1:172.21.15.195:6789/0 conn(0x7f806006d000 legacy=0x7f80600b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806006c800 con 0x7f80600b2af0 2023-11-06T03:12:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8058001a00 con 0x7f80600b2af0 2023-11-06T03:12:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 >> v1:172.21.15.157:6789/0 conn(0x7f80600b2af0 legacy=0x7f8060159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8048003f30 0x7f80480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048004d10 0x7f8048005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806006c520 con 0x7f8048004d10 2023-11-06T03:12:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804800ba70 con 0x7f8048003f30 2023-11-06T03:12:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804800bc30 con 0x7f80601599c0 2023-11-06T03:12:49.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048004d10 0x7f8048005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805ffff640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805f7fe640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8048003f30 0x7f80480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 -- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 msgr2=0x7f80480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 -- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8048003f30 msgr2=0x7f80480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8048003f30 0x7f80480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f8064b33640 1 -- 172.21.15.157:0/2098845938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048003530 con 0x7f8048004d10 2023-11-06T03:12:49.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.718+0000 7f805ffff640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80601599c0 0x7f80480039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:49.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8064b33640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048004d10 0x7f8048005190 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f806006f250 tx=0x7f805801d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff1d81f1d841e38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580a5ea0 con 0x7f8048004d10 2023-11-06T03:12:49.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580a8070 con 0x7f8048004d10 2023-11-06T03:12:49.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805effd640 1 -- 172.21.15.157:0/2098845938 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580ad890 con 0x7f8048004d10 2023-11-06T03:12:49.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- 172.21.15.157:0/2098845938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048004d10 msgr2=0x7f8048005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 --2- 172.21.15.157:0/2098845938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8048004d10 0x7f8048005190 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f806006f250 tx=0x7f805801d2c0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8064b33640 1 -- 172.21.15.157:0/2098845938 reap_dead start 2023-11-06T03:12:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- 172.21.15.157:0/2098845938 shutdown_connections 2023-11-06T03:12:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- 172.21.15.157:0/2098845938 >> 172.21.15.157:0/2098845938 conn(0x7f8060065930 msgr2=0x7f8060065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- 172.21.15.157:0/2098845938 shutdown_connections 2023-11-06T03:12:49.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- 172.21.15.157:0/2098845938 wait complete. 2023-11-06T03:12:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 Processor -- start 2023-11-06T03:12:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- start start 2023-11-06T03:12:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8048004d10 0x7f80600ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80601f3a10 0x7f8060147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80601f3ec0 con 0x7f8060156d70 2023-11-06T03:12:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80600ae780 con 0x7f80601f3a10 2023-11-06T03:12:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8066596640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80600aea60 con 0x7f8048004d10 2023-11-06T03:12:49.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53780/0 (socket says 172.21.15.157:53780) 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 -- 172.21.15.157:0/4050469639 learned_addr learned my addr 172.21.15.157:0/4050469639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805ffff640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8048004d10 0x7f80600ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f8064b33640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80601f3a10 0x7f8060147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 -- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8048004d10 msgr2=0x7f80600ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8048004d10 0x7f80600ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 -- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80601f3a10 msgr2=0x7f8060147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80601f3a10 0x7f8060147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 -- 172.21.15.157:0/4050469639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80600aed40 con 0x7f8060156d70 2023-11-06T03:12:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805f7fe640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f805400e4b0 tx=0x7f805400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.722+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540052e0 con 0x7f8060156d70 2023-11-06T03:12:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8054005440 con 0x7f8060156d70 2023-11-06T03:12:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80600af020 con 0x7f8060156d70 2023-11-06T03:12:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054003820 con 0x7f8060156d70 2023-11-06T03:12:49.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80600ad1e0 con 0x7f8060156d70 2023-11-06T03:12:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8060072750 con 0x7f8060156d70 2023-11-06T03:12:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8054003980 con 0x7f8060156d70 2023-11-06T03:12:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805cff9640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 0x7f804c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8054082fe0 con 0x7f8060156d70 2023-11-06T03:12:49.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805ffff640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 0x7f804c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:49.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.726+0000 7f805ffff640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 0x7f804c067f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f80601521a0 tx=0x7f8050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:49.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.730+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80540508b0 con 0x7f8060156d70 2023-11-06T03:12:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.873+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80600add00 con 0x7f804c065a90 2023-11-06T03:12:49.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.897+0000 7f805cff9640 1 -- 172.21.15.157:0/4050469639 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310817 (secure 0 0 0) 0x7f8050013a20 con 0x7f804c065a90 2023-11-06T03:12:49.901 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:49.903 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 msgr2=0x7f804c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 0x7f804c067f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f80601521a0 tx=0x7f8050008040 comp rx=0 tx=0).stop 2023-11-06T03:12:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 msgr2=0x7f80601f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f805400e4b0 tx=0x7f805400e970 comp rx=0 tx=0).stop 2023-11-06T03:12:49.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 shutdown_connections 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f804c065a90 0x7f804c067f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80601f3a10 0x7f8060147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8060156d70 0x7f80601f34d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 --2- 172.21.15.157:0/4050469639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8048004d10 0x7f80600ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 >> 172.21.15.157:0/4050469639 conn(0x7f8060065930 msgr2=0x7f806015bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:49.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 shutdown_connections 2023-11-06T03:12:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:49.901+0000 7f8066596640 1 -- 172.21.15.157:0/4050469639 wait complete. 2023-11-06T03:12:49.918 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2023-11-06T03:12:49.406968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":599644,"kb_used_data":11932,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754375076,"statfs":{"total":773094113280,"available":772480077824,"internally_reserved":0,"allocated":12218368,"data_stored":6234499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":64,"apply_latency_ms":64,"commit_latency_ns":64000000,"apply_latency_ns":64000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.412620"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585920+0000","last_change":"2023-11-06T03:12:45.411100+0000","last_active":"2023-11-06T03:12:46.585920+0000","last_peered":"2023-11-06T03:12:46.585920+0000","last_clean":"2023-11-06T03:12:46.585920+0000","last_became_active":"2023-11-06T03:12:45.410664+0000","last_became_peered":"2023-11-06T03:12:45.410664+0000","last_unstale":"2023-11-06T03:12:46.585920+0000","last_undegraded":"2023-11-06T03:12:46.585920+0000","last_fullsized":"2023-11-06T03:12:46.585920+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:32.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864522+0000","last_change":"2023-11-06T03:12:45.433350+0000","last_active":"2023-11-06T03:12:46.864522+0000","last_peered":"2023-11-06T03:12:46.864522+0000","last_clean":"2023-11-06T03:12:46.864522+0000","last_became_active":"2023-11-06T03:12:45.432683+0000","last_became_peered":"2023-11-06T03:12:45.432683+0000","last_unstale":"2023-11-06T03:12:46.864522+0000","last_undegraded":"2023-11-06T03:12:46.864522+0000","last_fullsized":"2023-11-06T03:12:46.864522+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:30:57.783818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585577+0000","last_change":"2023-11-06T03:12:45.440384+0000","last_active":"2023-11-06T03:12:46.585577+0000","last_peered":"2023-11-06T03:12:46.585577+0000","last_clean":"2023-11-06T03:12:46.585577+0000","last_became_active":"2023-11-06T03:12:45.440062+0000","last_became_peered":"2023-11-06T03:12:45.440062+0000","last_unstale":"2023-11-06T03:12:46.585577+0000","last_undegraded":"2023-11-06T03:12:46.585577+0000","last_fullsized":"2023-11-06T03:12:46.585577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:12:35.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393776+0000","last_change":"2023-11-06T03:12:45.415909+0000","last_active":"2023-11-06T03:12:46.393776+0000","last_peered":"2023-11-06T03:12:46.393776+0000","last_clean":"2023-11-06T03:12:46.393776+0000","last_became_active":"2023-11-06T03:12:45.415628+0000","last_became_peered":"2023-11-06T03:12:45.415628+0000","last_unstale":"2023-11-06T03:12:46.393776+0000","last_undegraded":"2023-11-06T03:12:46.393776+0000","last_fullsized":"2023-11-06T03:12:46.393776+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861369+0000","last_change":"2023-11-06T03:12:45.409517+0000","last_active":"2023-11-06T03:12:46.861369+0000","last_peered":"2023-11-06T03:12:46.861369+0000","last_clean":"2023-11-06T03:12:46.861369+0000","last_became_active":"2023-11-06T03:12:45.409162+0000","last_became_peered":"2023-11-06T03:12:45.409162+0000","last_unstale":"2023-11-06T03:12:46.861369+0000","last_undegraded":"2023-11-06T03:12:46.861369+0000","last_fullsized":"2023-11-06T03:12:46.861369+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:22.231105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863635+0000","last_change":"2023-11-06T03:12:45.406222+0000","last_active":"2023-11-06T03:12:46.863635+0000","last_peered":"2023-11-06T03:12:46.863635+0000","last_clean":"2023-11-06T03:12:46.863635+0000","last_became_active":"2023-11-06T03:12:45.405637+0000","last_became_peered":"2023-11-06T03:12:45.405637+0000","last_unstale":"2023-11-06T03:12:46.863635+0000","last_undegraded":"2023-11-06T03:12:46.863635+0000","last_fullsized":"2023-11-06T03:12:46.863635+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:27:06.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864018+0000","last_change":"2023-11-06T03:12:45.396241+0000","last_active":"2023-11-06T03:12:46.864018+0000","last_peered":"2023-11-06T03:12:46.864018+0000","last_clean":"2023-11-06T03:12:46.864018+0000","last_became_active":"2023-11-06T03:12:45.395412+0000","last_became_peered":"2023-11-06T03:12:45.395412+0000","last_unstale":"2023-11-06T03:12:46.864018+0000","last_undegraded":"2023-11-06T03:12:46.864018+0000","last_fullsized":"2023-11-06T03:12:46.864018+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:32.924101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.860803+0000","last_change":"2023-11-06T03:12:45.400987+0000","last_active":"2023-11-06T03:12:46.860803+0000","last_peered":"2023-11-06T03:12:46.860803+0000","last_clean":"2023-11-06T03:12:46.860803+0000","last_became_active":"2023-11-06T03:12:45.400723+0000","last_became_peered":"2023-11-06T03:12:45.400723+0000","last_unstale":"2023-11-06T03:12:46.860803+0000","last_undegraded":"2023-11-06T03:12:46.860803+0000","last_fullsized":"2023-11-06T03:12:46.860803+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:22:32.347484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.865265+0000","last_change":"2023-11-06T03:12:45.419973+0000","last_active":"2023-11-06T03:12:46.865265+0000","last_peered":"2023-11-06T03:12:46.865265+0000","last_clean":"2023-11-06T03:12:46.865265+0000","last_became_active":"2023-11-06T03:12:45.419264+0000","last_became_peered":"2023-11-06T03:12:45.419264+0000","last_unstale":"2023-11-06T03:12:46.865265+0000","last_undegraded":"2023-11-06T03:12:46.865265+0000","last_fullsized":"2023-11-06T03:12:46.865265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:03.447759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862096+0000","last_change":"2023-11-06T03:12:45.415201+0000","last_active":"2023-11-06T03:12:46.862096+0000","last_peered":"2023-11-06T03:12:46.862096+0000","last_clean":"2023-11-06T03:12:46.862096+0000","last_became_active":"2023-11-06T03:12:45.414958+0000","last_became_peered":"2023-11-06T03:12:45.414958+0000","last_unstale":"2023-11-06T03:12:46.862096+0000","last_undegraded":"2023-11-06T03:12:46.862096+0000","last_fullsized":"2023-11-06T03:12:46.862096+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:43.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862415+0000","last_change":"2023-11-06T03:12:45.442566+0000","last_active":"2023-11-06T03:12:46.862415+0000","last_peered":"2023-11-06T03:12:46.862415+0000","last_clean":"2023-11-06T03:12:46.862415+0000","last_became_active":"2023-11-06T03:12:45.440018+0000","last_became_peered":"2023-11-06T03:12:45.440018+0000","last_unstale":"2023-11-06T03:12:46.862415+0000","last_undegraded":"2023-11-06T03:12:46.862415+0000","last_fullsized":"2023-11-06T03:12:46.862415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401886+0000","last_change":"2023-11-06T03:12:45.407208+0000","last_active":"2023-11-06T03:12:46.401886+0000","last_peered":"2023-11-06T03:12:46.401886+0000","last_clean":"2023-11-06T03:12:46.401886+0000","last_became_active":"2023-11-06T03:12:45.406425+0000","last_became_peered":"2023-11-06T03:12:45.406425+0000","last_unstale":"2023-11-06T03:12:46.401886+0000","last_undegraded":"2023-11-06T03:12:46.401886+0000","last_fullsized":"2023-11-06T03:12:46.401886+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:32:51.357395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393715+0000","last_change":"2023-11-06T03:12:45.443524+0000","last_active":"2023-11-06T03:12:46.393715+0000","last_peered":"2023-11-06T03:12:46.393715+0000","last_clean":"2023-11-06T03:12:46.393715+0000","last_became_active":"2023-11-06T03:12:45.443230+0000","last_became_peered":"2023-11-06T03:12:45.443230+0000","last_unstale":"2023-11-06T03:12:46.393715+0000","last_undegraded":"2023-11-06T03:12:46.393715+0000","last_fullsized":"2023-11-06T03:12:46.393715+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863090+0000","last_change":"2023-11-06T03:12:45.431179+0000","last_active":"2023-11-06T03:12:46.863090+0000","last_peered":"2023-11-06T03:12:46.863090+0000","last_clean":"2023-11-06T03:12:46.863090+0000","last_became_active":"2023-11-06T03:12:45.430649+0000","last_became_peered":"2023-11-06T03:12:45.430649+0000","last_unstale":"2023-11-06T03:12:46.863090+0000","last_undegraded":"2023-11-06T03:12:46.863090+0000","last_fullsized":"2023-11-06T03:12:46.863090+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862976+0000","last_change":"2023-11-06T03:12:45.445000+0000","last_active":"2023-11-06T03:12:46.862976+0000","last_peered":"2023-11-06T03:12:46.862976+0000","last_clean":"2023-11-06T03:12:46.862976+0000","last_became_active":"2023-11-06T03:12:45.444456+0000","last_became_peered":"2023-11-06T03:12:45.444456+0000","last_unstale":"2023-11-06T03:12:46.862976+0000","last_undegraded":"2023-11-06T03:12:46.862976+0000","last_fullsized":"2023-11-06T03:12:46.862976+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863136+0000","last_change":"2023-11-06T03:12:45.430924+0000","last_active":"2023-11-06T03:12:46.863136+0000","last_peered":"2023-11-06T03:12:46.863136+0000","last_clean":"2023-11-06T03:12:46.863136+0000","last_became_active":"2023-11-06T03:12:45.430142+0000","last_became_peered":"2023-11-06T03:12:45.430142+0000","last_unstale":"2023-11-06T03:12:46.863136+0000","last_undegraded":"2023-11-06T03:12:46.863136+0000","last_fullsized":"2023-11-06T03:12:46.863136+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.586021+0000","last_change":"2023-11-06T03:12:45.441412+0000","last_active":"2023-11-06T03:12:46.586021+0000","last_peered":"2023-11-06T03:12:46.586021+0000","last_clean":"2023-11-06T03:12:46.586021+0000","last_became_active":"2023-11-06T03:12:45.441142+0000","last_became_peered":"2023-11-06T03:12:45.441142+0000","last_unstale":"2023-11-06T03:12:46.586021+0000","last_undegraded":"2023-11-06T03:12:46.586021+0000","last_fullsized":"2023-11-06T03:12:46.586021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:43.039951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864755+0000","last_change":"2023-11-06T03:12:45.405413+0000","last_active":"2023-11-06T03:12:46.864755+0000","last_peered":"2023-11-06T03:12:46.864755+0000","last_clean":"2023-11-06T03:12:46.864755+0000","last_became_active":"2023-11-06T03:12:45.404678+0000","last_became_peered":"2023-11-06T03:12:45.404678+0000","last_unstale":"2023-11-06T03:12:46.864755+0000","last_undegraded":"2023-11-06T03:12:46.864755+0000","last_fullsized":"2023-11-06T03:12:46.864755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:54.295847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585763+0000","last_change":"2023-11-06T03:12:45.425060+0000","last_active":"2023-11-06T03:12:46.585763+0000","last_peered":"2023-11-06T03:12:46.585763+0000","last_clean":"2023-11-06T03:12:46.585763+0000","last_became_active":"2023-11-06T03:12:45.424628+0000","last_became_peered":"2023-11-06T03:12:45.424628+0000","last_unstale":"2023-11-06T03:12:46.585763+0000","last_undegraded":"2023-11-06T03:12:46.585763+0000","last_fullsized":"2023-11-06T03:12:46.585763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:20.568857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402030+0000","last_change":"2023-11-06T03:12:45.431500+0000","last_active":"2023-11-06T03:12:46.402030+0000","last_peered":"2023-11-06T03:12:46.402030+0000","last_clean":"2023-11-06T03:12:46.402030+0000","last_became_active":"2023-11-06T03:12:45.430942+0000","last_became_peered":"2023-11-06T03:12:45.430942+0000","last_unstale":"2023-11-06T03:12:46.402030+0000","last_undegraded":"2023-11-06T03:12:46.402030+0000","last_fullsized":"2023-11-06T03:12:46.402030+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:52.901862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862907+0000","last_change":"2023-11-06T03:12:45.440754+0000","last_active":"2023-11-06T03:12:46.862907+0000","last_peered":"2023-11-06T03:12:46.862907+0000","last_clean":"2023-11-06T03:12:46.862907+0000","last_became_active":"2023-11-06T03:12:45.440484+0000","last_became_peered":"2023-11-06T03:12:45.440484+0000","last_unstale":"2023-11-06T03:12:46.862907+0000","last_undegraded":"2023-11-06T03:12:46.862907+0000","last_fullsized":"2023-11-06T03:12:46.862907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:04.674354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861706+0000","last_change":"2023-11-06T03:12:45.439448+0000","last_active":"2023-11-06T03:12:46.861706+0000","last_peered":"2023-11-06T03:12:46.861706+0000","last_clean":"2023-11-06T03:12:46.861706+0000","last_became_active":"2023-11-06T03:12:45.439208+0000","last_became_peered":"2023-11-06T03:12:45.439208+0000","last_unstale":"2023-11-06T03:12:46.861706+0000","last_undegraded":"2023-11-06T03:12:46.861706+0000","last_fullsized":"2023-11-06T03:12:46.861706+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585564+0000","last_change":"2023-11-06T03:12:45.440267+0000","last_active":"2023-11-06T03:12:46.585564+0000","last_peered":"2023-11-06T03:12:46.585564+0000","last_clean":"2023-11-06T03:12:46.585564+0000","last_became_active":"2023-11-06T03:12:45.440019+0000","last_became_peered":"2023-11-06T03:12:45.440019+0000","last_unstale":"2023-11-06T03:12:46.585564+0000","last_undegraded":"2023-11-06T03:12:46.585564+0000","last_fullsized":"2023-11-06T03:12:46.585564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:56.240869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.860792+0000","last_change":"2023-11-06T03:12:45.402001+0000","last_active":"2023-11-06T03:12:46.860792+0000","last_peered":"2023-11-06T03:12:46.860792+0000","last_clean":"2023-11-06T03:12:46.860792+0000","last_became_active":"2023-11-06T03:12:45.401659+0000","last_became_peered":"2023-11-06T03:12:45.401659+0000","last_unstale":"2023-11-06T03:12:46.860792+0000","last_undegraded":"2023-11-06T03:12:46.860792+0000","last_fullsized":"2023-11-06T03:12:46.860792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:18:19.762481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402576+0000","last_change":"2023-11-06T03:12:45.447582+0000","last_active":"2023-11-06T03:12:46.402576+0000","last_peered":"2023-11-06T03:12:46.402576+0000","last_clean":"2023-11-06T03:12:46.402576+0000","last_became_active":"2023-11-06T03:12:45.446982+0000","last_became_peered":"2023-11-06T03:12:45.446982+0000","last_unstale":"2023-11-06T03:12:46.402576+0000","last_undegraded":"2023-11-06T03:12:46.402576+0000","last_fullsized":"2023-11-06T03:12:46.402576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:45:40.848163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402241+0000","last_change":"2023-11-06T03:12:45.429871+0000","last_active":"2023-11-06T03:12:46.402241+0000","last_peered":"2023-11-06T03:12:46.402241+0000","last_clean":"2023-11-06T03:12:46.402241+0000","last_became_active":"2023-11-06T03:12:45.429136+0000","last_became_peered":"2023-11-06T03:12:45.429136+0000","last_unstale":"2023-11-06T03:12:46.402241+0000","last_undegraded":"2023-11-06T03:12:46.402241+0000","last_fullsized":"2023-11-06T03:12:46.402241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:23.719299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862570+0000","last_change":"2023-11-06T03:12:45.440192+0000","last_active":"2023-11-06T03:12:46.862570+0000","last_peered":"2023-11-06T03:12:46.862570+0000","last_clean":"2023-11-06T03:12:46.862570+0000","last_became_active":"2023-11-06T03:12:45.438692+0000","last_became_peered":"2023-11-06T03:12:45.438692+0000","last_unstale":"2023-11-06T03:12:46.862570+0000","last_undegraded":"2023-11-06T03:12:46.862570+0000","last_fullsized":"2023-11-06T03:12:46.862570+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:22.231105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863256+0000","last_change":"2023-11-06T03:12:45.443057+0000","last_active":"2023-11-06T03:12:46.863256+0000","last_peered":"2023-11-06T03:12:46.863256+0000","last_clean":"2023-11-06T03:12:46.863256+0000","last_became_active":"2023-11-06T03:12:45.442480+0000","last_became_peered":"2023-11-06T03:12:45.442480+0000","last_unstale":"2023-11-06T03:12:46.863256+0000","last_undegraded":"2023-11-06T03:12:46.863256+0000","last_fullsized":"2023-11-06T03:12:46.863256+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:20.970476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862741+0000","last_change":"2023-11-06T03:12:45.430274+0000","last_active":"2023-11-06T03:12:46.862741+0000","last_peered":"2023-11-06T03:12:46.862741+0000","last_clean":"2023-11-06T03:12:46.862741+0000","last_became_active":"2023-11-06T03:12:45.429904+0000","last_became_peered":"2023-11-06T03:12:45.429904+0000","last_unstale":"2023-11-06T03:12:46.862741+0000","last_undegraded":"2023-11-06T03:12:46.862741+0000","last_fullsized":"2023-11-06T03:12:46.862741+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:23.593202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401094+0000","last_change":"2023-11-06T03:12:45.445231+0000","last_active":"2023-11-06T03:12:46.401094+0000","last_peered":"2023-11-06T03:12:46.401094+0000","last_clean":"2023-11-06T03:12:46.401094+0000","last_became_active":"2023-11-06T03:12:45.444688+0000","last_became_peered":"2023-11-06T03:12:45.444688+0000","last_unstale":"2023-11-06T03:12:46.401094+0000","last_undegraded":"2023-11-06T03:12:46.401094+0000","last_fullsized":"2023-11-06T03:12:46.401094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:07:41.886545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864262+0000","last_change":"2023-11-06T03:12:45.412475+0000","last_active":"2023-11-06T03:12:46.864262+0000","last_peered":"2023-11-06T03:12:46.864262+0000","last_clean":"2023-11-06T03:12:46.864262+0000","last_became_active":"2023-11-06T03:12:45.411982+0000","last_became_peered":"2023-11-06T03:12:45.411982+0000","last_unstale":"2023-11-06T03:12:46.864262+0000","last_undegraded":"2023-11-06T03:12:46.864262+0000","last_fullsized":"2023-11-06T03:12:46.864262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:20:17.882004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861000+0000","last_change":"2023-11-06T03:12:45.421138+0000","last_active":"2023-11-06T03:12:46.861000+0000","last_peered":"2023-11-06T03:12:46.861000+0000","last_clean":"2023-11-06T03:12:46.861000+0000","last_became_active":"2023-11-06T03:12:45.420849+0000","last_became_peered":"2023-11-06T03:12:45.420849+0000","last_unstale":"2023-11-06T03:12:46.861000+0000","last_undegraded":"2023-11-06T03:12:46.861000+0000","last_fullsized":"2023-11-06T03:12:46.861000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:53:30.582264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585526+0000","last_change":"2023-11-06T03:12:45.412635+0000","last_active":"2023-11-06T03:12:46.585526+0000","last_peered":"2023-11-06T03:12:46.585526+0000","last_clean":"2023-11-06T03:12:46.585526+0000","last_became_active":"2023-11-06T03:12:45.412318+0000","last_became_peered":"2023-11-06T03:12:45.412318+0000","last_unstale":"2023-11-06T03:12:46.585526+0000","last_undegraded":"2023-11-06T03:12:46.585526+0000","last_fullsized":"2023-11-06T03:12:46.585526+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:28.025829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.427553+0000","last_change":"2023-11-06T03:12:45.024196+0000","last_active":"2023-11-06T03:12:45.427553+0000","last_peered":"2023-11-06T03:12:45.427553+0000","last_clean":"2023-11-06T03:12:45.427553+0000","last_became_active":"2023-11-06T03:12:45.023925+0000","last_became_peered":"2023-11-06T03:12:45.023925+0000","last_unstale":"2023-11-06T03:12:45.427553+0000","last_undegraded":"2023-11-06T03:12:45.427553+0000","last_fullsized":"2023-11-06T03:12:45.427553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862430+0000","last_change":"2023-11-06T03:12:45.402369+0000","last_active":"2023-11-06T03:12:46.862430+0000","last_peered":"2023-11-06T03:12:46.862430+0000","last_clean":"2023-11-06T03:12:46.862430+0000","last_became_active":"2023-11-06T03:12:45.401994+0000","last_became_peered":"2023-11-06T03:12:45.401994+0000","last_unstale":"2023-11-06T03:12:46.862430+0000","last_undegraded":"2023-11-06T03:12:46.862430+0000","last_fullsized":"2023-11-06T03:12:46.862430+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:25:41.489850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.429710+0000","last_change":"2023-11-06T03:12:45.021988+0000","last_active":"2023-11-06T03:12:45.429710+0000","last_peered":"2023-11-06T03:12:45.429710+0000","last_clean":"2023-11-06T03:12:45.429710+0000","last_became_active":"2023-11-06T03:12:45.021194+0000","last_became_peered":"2023-11-06T03:12:45.021194+0000","last_unstale":"2023-11-06T03:12:45.429710+0000","last_undegraded":"2023-11-06T03:12:45.429710+0000","last_fullsized":"2023-11-06T03:12:45.429710+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401329+0000","last_change":"2023-11-06T03:12:45.421142+0000","last_active":"2023-11-06T03:12:46.401329+0000","last_peered":"2023-11-06T03:12:46.401329+0000","last_clean":"2023-11-06T03:12:46.401329+0000","last_became_active":"2023-11-06T03:12:45.420482+0000","last_became_peered":"2023-11-06T03:12:45.420482+0000","last_unstale":"2023-11-06T03:12:46.401329+0000","last_undegraded":"2023-11-06T03:12:46.401329+0000","last_fullsized":"2023-11-06T03:12:46.401329+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:32.568478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.446283+0000","last_change":"2023-11-06T03:12:45.035200+0000","last_active":"2023-11-06T03:12:45.446283+0000","last_peered":"2023-11-06T03:12:45.446283+0000","last_clean":"2023-11-06T03:12:45.446283+0000","last_became_active":"2023-11-06T03:12:45.034474+0000","last_became_peered":"2023-11-06T03:12:45.034474+0000","last_unstale":"2023-11-06T03:12:45.446283+0000","last_undegraded":"2023-11-06T03:12:45.446283+0000","last_fullsized":"2023-11-06T03:12:45.446283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863762+0000","last_change":"2023-11-06T03:12:45.440093+0000","last_active":"2023-11-06T03:12:46.863762+0000","last_peered":"2023-11-06T03:12:46.863762+0000","last_clean":"2023-11-06T03:12:46.863762+0000","last_became_active":"2023-11-06T03:12:45.438786+0000","last_became_peered":"2023-11-06T03:12:45.438786+0000","last_unstale":"2023-11-06T03:12:46.863762+0000","last_undegraded":"2023-11-06T03:12:46.863762+0000","last_fullsized":"2023-11-06T03:12:46.863762+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:59.278874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.425517+0000","last_change":"2023-11-06T03:12:45.019690+0000","last_active":"2023-11-06T03:12:45.425517+0000","last_peered":"2023-11-06T03:12:45.425517+0000","last_clean":"2023-11-06T03:12:45.425517+0000","last_became_active":"2023-11-06T03:12:45.019343+0000","last_became_peered":"2023-11-06T03:12:45.019343+0000","last_unstale":"2023-11-06T03:12:45.425517+0000","last_undegraded":"2023-11-06T03:12:45.425517+0000","last_fullsized":"2023-11-06T03:12:45.425517+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393875+0000","last_change":"2023-11-06T03:12:45.418313+0000","last_active":"2023-11-06T03:12:46.393875+0000","last_peered":"2023-11-06T03:12:46.393875+0000","last_clean":"2023-11-06T03:12:46.393875+0000","last_became_active":"2023-11-06T03:12:45.418041+0000","last_became_peered":"2023-11-06T03:12:45.418041+0000","last_unstale":"2023-11-06T03:12:46.393875+0000","last_undegraded":"2023-11-06T03:12:46.393875+0000","last_fullsized":"2023-11-06T03:12:46.393875+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:18:19.762481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.430007+0000","last_change":"2023-11-06T03:12:45.029171+0000","last_active":"2023-11-06T03:12:45.430007+0000","last_peered":"2023-11-06T03:12:45.430007+0000","last_clean":"2023-11-06T03:12:45.430007+0000","last_became_active":"2023-11-06T03:12:45.028406+0000","last_became_peered":"2023-11-06T03:12:45.028406+0000","last_unstale":"2023-11-06T03:12:45.430007+0000","last_undegraded":"2023-11-06T03:12:45.430007+0000","last_fullsized":"2023-11-06T03:12:45.430007+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585910+0000","last_change":"2023-11-06T03:12:45.412584+0000","last_active":"2023-11-06T03:12:46.585910+0000","last_peered":"2023-11-06T03:12:46.585910+0000","last_clean":"2023-11-06T03:12:46.585910+0000","last_became_active":"2023-11-06T03:12:45.412198+0000","last_became_peered":"2023-11-06T03:12:45.412198+0000","last_unstale":"2023-11-06T03:12:46.585910+0000","last_undegraded":"2023-11-06T03:12:46.585910+0000","last_fullsized":"2023-11-06T03:12:46.585910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:32.814892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.442233+0000","last_change":"2023-11-06T03:12:45.034966+0000","last_active":"2023-11-06T03:12:45.442233+0000","last_peered":"2023-11-06T03:12:45.442233+0000","last_clean":"2023-11-06T03:12:45.442233+0000","last_became_active":"2023-11-06T03:12:45.034055+0000","last_became_peered":"2023-11-06T03:12:45.034055+0000","last_unstale":"2023-11-06T03:12:45.442233+0000","last_undegraded":"2023-11-06T03:12:45.442233+0000","last_fullsized":"2023-11-06T03:12:45.442233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864979+0000","last_change":"2023-11-06T03:12:45.439692+0000","last_active":"2023-11-06T03:12:46.864979+0000","last_peered":"2023-11-06T03:12:46.864979+0000","last_clean":"2023-11-06T03:12:46.864979+0000","last_became_active":"2023-11-06T03:12:45.439156+0000","last_became_peered":"2023-11-06T03:12:45.439156+0000","last_unstale":"2023-11-06T03:12:46.864979+0000","last_undegraded":"2023-11-06T03:12:46.864979+0000","last_fullsized":"2023-11-06T03:12:46.864979+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:06.632756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.398617+0000","last_change":"2023-11-06T03:12:45.028815+0000","last_active":"2023-11-06T03:12:45.398617+0000","last_peered":"2023-11-06T03:12:45.398617+0000","last_clean":"2023-11-06T03:12:45.398617+0000","last_became_active":"2023-11-06T03:12:45.028210+0000","last_became_peered":"2023-11-06T03:12:45.028210+0000","last_unstale":"2023-11-06T03:12:45.398617+0000","last_undegraded":"2023-11-06T03:12:45.398617+0000","last_fullsized":"2023-11-06T03:12:45.398617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393869+0000","last_change":"2023-11-06T03:12:45.408597+0000","last_active":"2023-11-06T03:12:46.393869+0000","last_peered":"2023-11-06T03:12:46.393869+0000","last_clean":"2023-11-06T03:12:46.393869+0000","last_became_active":"2023-11-06T03:12:45.407493+0000","last_became_peered":"2023-11-06T03:12:45.407493+0000","last_unstale":"2023-11-06T03:12:46.393869+0000","last_undegraded":"2023-11-06T03:12:46.393869+0000","last_fullsized":"2023-11-06T03:12:46.393869+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:43.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395663+0000","last_change":"2023-11-06T03:12:45.028903+0000","last_active":"2023-11-06T03:12:45.395663+0000","last_peered":"2023-11-06T03:12:45.395663+0000","last_clean":"2023-11-06T03:12:45.395663+0000","last_became_active":"2023-11-06T03:12:45.028293+0000","last_became_peered":"2023-11-06T03:12:45.028293+0000","last_unstale":"2023-11-06T03:12:45.395663+0000","last_undegraded":"2023-11-06T03:12:45.395663+0000","last_fullsized":"2023-11-06T03:12:45.395663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585467+0000","last_change":"2023-11-06T03:12:45.423639+0000","last_active":"2023-11-06T03:12:46.585467+0000","last_peered":"2023-11-06T03:12:46.585467+0000","last_clean":"2023-11-06T03:12:46.585467+0000","last_became_active":"2023-11-06T03:12:45.423343+0000","last_became_peered":"2023-11-06T03:12:45.423343+0000","last_unstale":"2023-11-06T03:12:46.585467+0000","last_undegraded":"2023-11-06T03:12:46.585467+0000","last_fullsized":"2023-11-06T03:12:46.585467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:41.519658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.403245+0000","last_change":"2023-11-06T03:12:45.036332+0000","last_active":"2023-11-06T03:12:45.403245+0000","last_peered":"2023-11-06T03:12:45.403245+0000","last_clean":"2023-11-06T03:12:45.403245+0000","last_became_active":"2023-11-06T03:12:45.035821+0000","last_became_peered":"2023-11-06T03:12:45.035821+0000","last_unstale":"2023-11-06T03:12:45.403245+0000","last_undegraded":"2023-11-06T03:12:45.403245+0000","last_fullsized":"2023-11-06T03:12:45.403245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861196+0000","last_change":"2023-11-06T03:12:45.408527+0000","last_active":"2023-11-06T03:12:46.861196+0000","last_peered":"2023-11-06T03:12:46.861196+0000","last_clean":"2023-11-06T03:12:46.861196+0000","last_became_active":"2023-11-06T03:12:45.405334+0000","last_became_peered":"2023-11-06T03:12:45.405334+0000","last_unstale":"2023-11-06T03:12:46.861196+0000","last_undegraded":"2023-11-06T03:12:46.861196+0000","last_fullsized":"2023-11-06T03:12:46.861196+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.391302+0000","last_change":"2023-11-06T03:12:45.019118+0000","last_active":"2023-11-06T03:12:45.391302+0000","last_peered":"2023-11-06T03:12:45.391302+0000","last_clean":"2023-11-06T03:12:45.391302+0000","last_became_active":"2023-11-06T03:12:45.018401+0000","last_became_peered":"2023-11-06T03:12:45.018401+0000","last_unstale":"2023-11-06T03:12:45.391302+0000","last_undegraded":"2023-11-06T03:12:45.391302+0000","last_fullsized":"2023-11-06T03:12:45.391302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862719+0000","last_change":"2023-11-06T03:12:45.416831+0000","last_active":"2023-11-06T03:12:46.862719+0000","last_peered":"2023-11-06T03:12:46.862719+0000","last_clean":"2023-11-06T03:12:46.862719+0000","last_became_active":"2023-11-06T03:12:45.416542+0000","last_became_peered":"2023-11-06T03:12:45.416542+0000","last_unstale":"2023-11-06T03:12:46.862719+0000","last_undegraded":"2023-11-06T03:12:46.862719+0000","last_fullsized":"2023-11-06T03:12:46.862719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:13.696657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434256+0000","last_change":"2023-11-06T03:12:45.038268+0000","last_active":"2023-11-06T03:12:45.434256+0000","last_peered":"2023-11-06T03:12:45.434256+0000","last_clean":"2023-11-06T03:12:45.434256+0000","last_became_active":"2023-11-06T03:12:45.037639+0000","last_became_peered":"2023-11-06T03:12:45.037639+0000","last_unstale":"2023-11-06T03:12:45.434256+0000","last_undegraded":"2023-11-06T03:12:45.434256+0000","last_fullsized":"2023-11-06T03:12:45.434256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861619+0000","last_change":"2023-11-06T03:12:45.414318+0000","last_active":"2023-11-06T03:12:46.861619+0000","last_peered":"2023-11-06T03:12:46.861619+0000","last_clean":"2023-11-06T03:12:46.861619+0000","last_became_active":"2023-11-06T03:12:45.405366+0000","last_became_peered":"2023-11-06T03:12:45.405366+0000","last_unstale":"2023-11-06T03:12:46.861619+0000","last_undegraded":"2023-11-06T03:12:46.861619+0000","last_fullsized":"2023-11-06T03:12:46.861619+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434121+0000","last_change":"2023-11-06T03:12:45.039214+0000","last_active":"2023-11-06T03:12:45.434121+0000","last_peered":"2023-11-06T03:12:45.434121+0000","last_clean":"2023-11-06T03:12:45.434121+0000","last_became_active":"2023-11-06T03:12:45.038608+0000","last_became_peered":"2023-11-06T03:12:45.038608+0000","last_unstale":"2023-11-06T03:12:45.434121+0000","last_undegraded":"2023-11-06T03:12:45.434121+0000","last_fullsized":"2023-11-06T03:12:45.434121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862277+0000","last_change":"2023-11-06T03:12:45.440352+0000","last_active":"2023-11-06T03:12:46.862277+0000","last_peered":"2023-11-06T03:12:46.862277+0000","last_clean":"2023-11-06T03:12:46.862277+0000","last_became_active":"2023-11-06T03:12:45.438755+0000","last_became_peered":"2023-11-06T03:12:45.438755+0000","last_unstale":"2023-11-06T03:12:46.862277+0000","last_undegraded":"2023-11-06T03:12:46.862277+0000","last_fullsized":"2023-11-06T03:12:46.862277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.418703+0000","last_change":"2023-11-06T03:12:45.031941+0000","last_active":"2023-11-06T03:12:45.418703+0000","last_peered":"2023-11-06T03:12:45.418703+0000","last_clean":"2023-11-06T03:12:45.418703+0000","last_became_active":"2023-11-06T03:12:45.031606+0000","last_became_peered":"2023-11-06T03:12:45.031606+0000","last_unstale":"2023-11-06T03:12:45.418703+0000","last_undegraded":"2023-11-06T03:12:45.418703+0000","last_fullsized":"2023-11-06T03:12:45.418703+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864333+0000","last_change":"2023-11-06T03:12:45.418395+0000","last_active":"2023-11-06T03:12:46.864333+0000","last_peered":"2023-11-06T03:12:46.864333+0000","last_clean":"2023-11-06T03:12:46.864333+0000","last_became_active":"2023-11-06T03:12:45.417772+0000","last_became_peered":"2023-11-06T03:12:45.417772+0000","last_unstale":"2023-11-06T03:12:46.864333+0000","last_undegraded":"2023-11-06T03:12:46.864333+0000","last_fullsized":"2023-11-06T03:12:46.864333+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:46.522287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395028+0000","last_change":"2023-11-06T03:12:45.020168+0000","last_active":"2023-11-06T03:12:45.395028+0000","last_peered":"2023-11-06T03:12:45.395028+0000","last_clean":"2023-11-06T03:12:45.395028+0000","last_became_active":"2023-11-06T03:12:45.019891+0000","last_became_peered":"2023-11-06T03:12:45.019891+0000","last_unstale":"2023-11-06T03:12:45.395028+0000","last_undegraded":"2023-11-06T03:12:45.395028+0000","last_fullsized":"2023-11-06T03:12:45.395028+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863228+0000","last_change":"2023-11-06T03:12:45.425328+0000","last_active":"2023-11-06T03:12:46.863228+0000","last_peered":"2023-11-06T03:12:46.863228+0000","last_clean":"2023-11-06T03:12:46.863228+0000","last_became_active":"2023-11-06T03:12:45.425081+0000","last_became_peered":"2023-11-06T03:12:45.425081+0000","last_unstale":"2023-11-06T03:12:46.863228+0000","last_undegraded":"2023-11-06T03:12:46.863228+0000","last_fullsized":"2023-11-06T03:12:46.863228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:59.511071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.387119+0000","last_change":"2023-11-06T03:12:45.022982+0000","last_active":"2023-11-06T03:12:45.387119+0000","last_peered":"2023-11-06T03:12:45.387119+0000","last_clean":"2023-11-06T03:12:45.387119+0000","last_became_active":"2023-11-06T03:12:45.021582+0000","last_became_peered":"2023-11-06T03:12:45.021582+0000","last_unstale":"2023-11-06T03:12:45.387119+0000","last_undegraded":"2023-11-06T03:12:45.387119+0000","last_fullsized":"2023-11-06T03:12:45.387119+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.586169+0000","last_change":"2023-11-06T03:12:45.441344+0000","last_active":"2023-11-06T03:12:46.586169+0000","last_peered":"2023-11-06T03:12:46.586169+0000","last_clean":"2023-11-06T03:12:46.586169+0000","last_became_active":"2023-11-06T03:12:45.441042+0000","last_became_peered":"2023-11-06T03:12:45.441042+0000","last_unstale":"2023-11-06T03:12:46.586169+0000","last_undegraded":"2023-11-06T03:12:46.586169+0000","last_fullsized":"2023-11-06T03:12:46.586169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:45:54.459354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":19,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.447104+0000","last_change":"2023-11-06T03:12:45.036866+0000","last_active":"2023-11-06T03:12:45.447104+0000","last_peered":"2023-11-06T03:12:45.447104+0000","last_clean":"2023-11-06T03:12:45.447104+0000","last_became_active":"2023-11-06T03:12:45.036202+0000","last_became_peered":"2023-11-06T03:12:45.036202+0000","last_unstale":"2023-11-06T03:12:45.447104+0000","last_undegraded":"2023-11-06T03:12:45.447104+0000","last_fullsized":"2023-11-06T03:12:45.447104+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585385+0000","last_change":"2023-11-06T03:12:45.400081+0000","last_active":"2023-11-06T03:12:46.585385+0000","last_peered":"2023-11-06T03:12:46.585385+0000","last_clean":"2023-11-06T03:12:46.585385+0000","last_became_active":"2023-11-06T03:12:45.399581+0000","last_became_peered":"2023-11-06T03:12:45.399581+0000","last_unstale":"2023-11-06T03:12:46.585385+0000","last_undegraded":"2023-11-06T03:12:46.585385+0000","last_fullsized":"2023-11-06T03:12:46.585385+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:33:44.980111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.417327+0000","last_change":"2023-11-06T03:12:45.014894+0000","last_active":"2023-11-06T03:12:45.417327+0000","last_peered":"2023-11-06T03:12:45.417327+0000","last_clean":"2023-11-06T03:12:45.417327+0000","last_became_active":"2023-11-06T03:12:45.014171+0000","last_became_peered":"2023-11-06T03:12:45.014171+0000","last_unstale":"2023-11-06T03:12:45.417327+0000","last_undegraded":"2023-11-06T03:12:45.417327+0000","last_fullsized":"2023-11-06T03:12:45.417327+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.400451+0000","last_change":"2023-11-06T03:12:45.407200+0000","last_active":"2023-11-06T03:12:46.400451+0000","last_peered":"2023-11-06T03:12:46.400451+0000","last_clean":"2023-11-06T03:12:46.400451+0000","last_became_active":"2023-11-06T03:12:45.405084+0000","last_became_peered":"2023-11-06T03:12:45.405084+0000","last_unstale":"2023-11-06T03:12:46.400451+0000","last_undegraded":"2023-11-06T03:12:46.400451+0000","last_fullsized":"2023-11-06T03:12:46.400451+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:02.356000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.391302+0000","last_change":"2023-11-06T03:12:45.033836+0000","last_active":"2023-11-06T03:12:45.391302+0000","last_peered":"2023-11-06T03:12:45.391302+0000","last_clean":"2023-11-06T03:12:45.391302+0000","last_became_active":"2023-11-06T03:12:45.033383+0000","last_became_peered":"2023-11-06T03:12:45.033383+0000","last_unstale":"2023-11-06T03:12:45.391302+0000","last_undegraded":"2023-11-06T03:12:45.391302+0000","last_fullsized":"2023-11-06T03:12:45.391302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393761+0000","last_change":"2023-11-06T03:12:45.417343+0000","last_active":"2023-11-06T03:12:46.393761+0000","last_peered":"2023-11-06T03:12:46.393761+0000","last_clean":"2023-11-06T03:12:46.393761+0000","last_became_active":"2023-11-06T03:12:45.416913+0000","last_became_peered":"2023-11-06T03:12:45.416913+0000","last_unstale":"2023-11-06T03:12:46.393761+0000","last_undegraded":"2023-11-06T03:12:46.393761+0000","last_fullsized":"2023-11-06T03:12:46.393761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:32.568478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.390717+0000","last_change":"2023-11-06T03:12:45.032631+0000","last_active":"2023-11-06T03:12:45.390717+0000","last_peered":"2023-11-06T03:12:45.390717+0000","last_clean":"2023-11-06T03:12:45.390717+0000","last_became_active":"2023-11-06T03:12:45.032177+0000","last_became_peered":"2023-11-06T03:12:45.032177+0000","last_unstale":"2023-11-06T03:12:45.390717+0000","last_undegraded":"2023-11-06T03:12:45.390717+0000","last_fullsized":"2023-11-06T03:12:45.390717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862517+0000","last_change":"2023-11-06T03:12:45.440426+0000","last_active":"2023-11-06T03:12:46.862517+0000","last_peered":"2023-11-06T03:12:46.862517+0000","last_clean":"2023-11-06T03:12:46.862517+0000","last_became_active":"2023-11-06T03:12:45.439467+0000","last_became_peered":"2023-11-06T03:12:45.439467+0000","last_unstale":"2023-11-06T03:12:46.862517+0000","last_undegraded":"2023-11-06T03:12:46.862517+0000","last_fullsized":"2023-11-06T03:12:46.862517+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:02.833708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.394676+0000","last_change":"2023-11-06T03:12:45.028022+0000","last_active":"2023-11-06T03:12:45.394676+0000","last_peered":"2023-11-06T03:12:45.394676+0000","last_clean":"2023-11-06T03:12:45.394676+0000","last_became_active":"2023-11-06T03:12:45.027325+0000","last_became_peered":"2023-11-06T03:12:45.027325+0000","last_unstale":"2023-11-06T03:12:45.394676+0000","last_undegraded":"2023-11-06T03:12:45.394676+0000","last_fullsized":"2023-11-06T03:12:45.394676+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862809+0000","last_change":"2023-11-06T03:12:45.398468+0000","last_active":"2023-11-06T03:12:46.862809+0000","last_peered":"2023-11-06T03:12:46.862809+0000","last_clean":"2023-11-06T03:12:46.862809+0000","last_became_active":"2023-11-06T03:12:45.394061+0000","last_became_peered":"2023-11-06T03:12:45.394061+0000","last_unstale":"2023-11-06T03:12:46.862809+0000","last_undegraded":"2023-11-06T03:12:46.862809+0000","last_fullsized":"2023-11-06T03:12:46.862809+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:22.305311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.397819+0000","last_change":"2023-11-06T03:12:45.028959+0000","last_active":"2023-11-06T03:12:45.397819+0000","last_peered":"2023-11-06T03:12:45.397819+0000","last_clean":"2023-11-06T03:12:45.397819+0000","last_became_active":"2023-11-06T03:12:45.028406+0000","last_became_peered":"2023-11-06T03:12:45.028406+0000","last_unstale":"2023-11-06T03:12:45.397819+0000","last_undegraded":"2023-11-06T03:12:45.397819+0000","last_fullsized":"2023-11-06T03:12:45.397819+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863106+0000","last_change":"2023-11-06T03:12:45.402029+0000","last_active":"2023-11-06T03:12:46.863106+0000","last_peered":"2023-11-06T03:12:46.863106+0000","last_clean":"2023-11-06T03:12:46.863106+0000","last_became_active":"2023-11-06T03:12:45.401642+0000","last_became_peered":"2023-11-06T03:12:45.401642+0000","last_unstale":"2023-11-06T03:12:46.863106+0000","last_undegraded":"2023-11-06T03:12:46.863106+0000","last_fullsized":"2023-11-06T03:12:46.863106+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:15.085556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.393101+0000","last_change":"2023-11-06T03:12:45.018899+0000","last_active":"2023-11-06T03:12:45.393101+0000","last_peered":"2023-11-06T03:12:45.393101+0000","last_clean":"2023-11-06T03:12:45.393101+0000","last_became_active":"2023-11-06T03:12:45.018234+0000","last_became_peered":"2023-11-06T03:12:45.018234+0000","last_unstale":"2023-11-06T03:12:45.393101+0000","last_undegraded":"2023-11-06T03:12:45.393101+0000","last_fullsized":"2023-11-06T03:12:45.393101+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864930+0000","last_change":"2023-11-06T03:12:45.443717+0000","last_active":"2023-11-06T03:12:46.864930+0000","last_peered":"2023-11-06T03:12:46.864930+0000","last_clean":"2023-11-06T03:12:46.864930+0000","last_became_active":"2023-11-06T03:12:45.443151+0000","last_became_peered":"2023-11-06T03:12:45.443151+0000","last_unstale":"2023-11-06T03:12:46.864930+0000","last_undegraded":"2023-11-06T03:12:46.864930+0000","last_fullsized":"2023-11-06T03:12:46.864930+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:02.833708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.389196+0000","last_change":"2023-11-06T03:12:45.033145+0000","last_active":"2023-11-06T03:12:45.389196+0000","last_peered":"2023-11-06T03:12:45.389196+0000","last_clean":"2023-11-06T03:12:45.389196+0000","last_became_active":"2023-11-06T03:12:45.032354+0000","last_became_peered":"2023-11-06T03:12:45.032354+0000","last_unstale":"2023-11-06T03:12:45.389196+0000","last_undegraded":"2023-11-06T03:12:45.389196+0000","last_fullsized":"2023-11-06T03:12:45.389196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863232+0000","last_change":"2023-11-06T03:12:45.441677+0000","last_active":"2023-11-06T03:12:46.863232+0000","last_peered":"2023-11-06T03:12:46.863232+0000","last_clean":"2023-11-06T03:12:46.863232+0000","last_became_active":"2023-11-06T03:12:45.441016+0000","last_became_peered":"2023-11-06T03:12:45.441016+0000","last_unstale":"2023-11-06T03:12:46.863232+0000","last_undegraded":"2023-11-06T03:12:46.863232+0000","last_fullsized":"2023-11-06T03:12:46.863232+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.386193+0000","last_change":"2023-11-06T03:12:45.024618+0000","last_active":"2023-11-06T03:12:45.386193+0000","last_peered":"2023-11-06T03:12:45.386193+0000","last_clean":"2023-11-06T03:12:45.386193+0000","last_became_active":"2023-11-06T03:12:45.019472+0000","last_became_peered":"2023-11-06T03:12:45.019472+0000","last_unstale":"2023-11-06T03:12:45.386193+0000","last_undegraded":"2023-11-06T03:12:45.386193+0000","last_fullsized":"2023-11-06T03:12:45.386193+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862121+0000","last_change":"2023-11-06T03:12:45.439246+0000","last_active":"2023-11-06T03:12:46.862121+0000","last_peered":"2023-11-06T03:12:46.862121+0000","last_clean":"2023-11-06T03:12:46.862121+0000","last_became_active":"2023-11-06T03:12:45.438665+0000","last_became_peered":"2023-11-06T03:12:45.438665+0000","last_unstale":"2023-11-06T03:12:46.862121+0000","last_undegraded":"2023-11-06T03:12:46.862121+0000","last_fullsized":"2023-11-06T03:12:46.862121+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.410336+0000","last_change":"2023-11-06T03:12:45.025025+0000","last_active":"2023-11-06T03:12:45.410336+0000","last_peered":"2023-11-06T03:12:45.410336+0000","last_clean":"2023-11-06T03:12:45.410336+0000","last_became_active":"2023-11-06T03:12:45.024701+0000","last_became_peered":"2023-11-06T03:12:45.024701+0000","last_unstale":"2023-11-06T03:12:45.410336+0000","last_undegraded":"2023-11-06T03:12:45.410336+0000","last_fullsized":"2023-11-06T03:12:45.410336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.400826+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:12:45.400826+0000","last_peered":"2023-11-06T03:12:45.400826+0000","last_clean":"2023-11-06T03:12:45.400826+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:12:45.400826+0000","last_undegraded":"2023-11-06T03:12:45.400826+0000","last_fullsized":"2023-11-06T03:12:45.400826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.401972+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:12:45.401972+0000","last_peered":"2023-11-06T03:12:45.401972+0000","last_clean":"2023-11-06T03:12:45.401972+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:12:45.401972+0000","last_undegraded":"2023-11-06T03:12:45.401972+0000","last_fullsized":"2023-11-06T03:12:45.401972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863858+0000","last_change":"2023-11-06T03:12:45.412840+0000","last_active":"2023-11-06T03:12:46.863858+0000","last_peered":"2023-11-06T03:12:46.863858+0000","last_clean":"2023-11-06T03:12:46.863858+0000","last_became_active":"2023-11-06T03:12:45.410939+0000","last_became_peered":"2023-11-06T03:12:45.410939+0000","last_unstale":"2023-11-06T03:12:46.863858+0000","last_undegraded":"2023-11-06T03:12:46.863858+0000","last_fullsized":"2023-11-06T03:12:46.863858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:52:20.478173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434333+0000","last_change":"2023-11-06T03:12:45.023312+0000","last_active":"2023-11-06T03:12:45.434333+0000","last_peered":"2023-11-06T03:12:45.434333+0000","last_clean":"2023-11-06T03:12:45.434333+0000","last_became_active":"2023-11-06T03:12:45.022806+0000","last_became_peered":"2023-11-06T03:12:45.022806+0000","last_unstale":"2023-11-06T03:12:45.434333+0000","last_undegraded":"2023-11-06T03:12:45.434333+0000","last_fullsized":"2023-11-06T03:12:45.434333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.400557+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:12:45.400557+0000","last_peered":"2023-11-06T03:12:45.400557+0000","last_clean":"2023-11-06T03:12:45.400557+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:12:45.400557+0000","last_undegraded":"2023-11-06T03:12:45.400557+0000","last_fullsized":"2023-11-06T03:12:45.400557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861482+0000","last_change":"2023-11-06T03:12:45.429678+0000","last_active":"2023-11-06T03:12:46.861482+0000","last_peered":"2023-11-06T03:12:46.861482+0000","last_clean":"2023-11-06T03:12:46.861482+0000","last_became_active":"2023-11-06T03:12:45.429431+0000","last_became_peered":"2023-11-06T03:12:45.429431+0000","last_unstale":"2023-11-06T03:12:46.861482+0000","last_undegraded":"2023-11-06T03:12:46.861482+0000","last_fullsized":"2023-11-06T03:12:46.861482+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.399320+0000","last_change":"2023-11-06T03:12:45.026227+0000","last_active":"2023-11-06T03:12:45.399320+0000","last_peered":"2023-11-06T03:12:45.399320+0000","last_clean":"2023-11-06T03:12:45.399320+0000","last_became_active":"2023-11-06T03:12:45.024563+0000","last_became_peered":"2023-11-06T03:12:45.024563+0000","last_unstale":"2023-11-06T03:12:45.399320+0000","last_undegraded":"2023-11-06T03:12:45.399320+0000","last_fullsized":"2023-11-06T03:12:45.399320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.387210+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:12:45.387210+0000","last_peered":"2023-11-06T03:12:45.387210+0000","last_clean":"2023-11-06T03:12:45.387210+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:12:45.387210+0000","last_undegraded":"2023-11-06T03:12:45.387210+0000","last_fullsized":"2023-11-06T03:12:45.387210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861157+0000","last_change":"2023-11-06T03:12:45.403831+0000","last_active":"2023-11-06T03:12:46.861157+0000","last_peered":"2023-11-06T03:12:46.861157+0000","last_clean":"2023-11-06T03:12:46.861157+0000","last_became_active":"2023-11-06T03:12:45.402589+0000","last_became_peered":"2023-11-06T03:12:45.402589+0000","last_unstale":"2023-11-06T03:12:46.861157+0000","last_undegraded":"2023-11-06T03:12:46.861157+0000","last_fullsized":"2023-11-06T03:12:46.861157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:10.940379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.430547+0000","last_change":"2023-11-06T03:12:45.024881+0000","last_active":"2023-11-06T03:12:45.430547+0000","last_peered":"2023-11-06T03:12:45.430547+0000","last_clean":"2023-11-06T03:12:45.430547+0000","last_became_active":"2023-11-06T03:12:45.024362+0000","last_became_peered":"2023-11-06T03:12:45.024362+0000","last_unstale":"2023-11-06T03:12:45.430547+0000","last_undegraded":"2023-11-06T03:12:45.430547+0000","last_fullsized":"2023-11-06T03:12:45.430547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.401566+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:12:45.401566+0000","last_peered":"2023-11-06T03:12:45.401566+0000","last_clean":"2023-11-06T03:12:45.401566+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:12:45.401566+0000","last_undegraded":"2023-11-06T03:12:45.401566+0000","last_fullsized":"2023-11-06T03:12:45.401566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393609+0000","last_change":"2023-11-06T03:12:45.425084+0000","last_active":"2023-11-06T03:12:46.393609+0000","last_peered":"2023-11-06T03:12:46.393609+0000","last_clean":"2023-11-06T03:12:46.393609+0000","last_became_active":"2023-11-06T03:12:45.424780+0000","last_became_peered":"2023-11-06T03:12:45.424780+0000","last_unstale":"2023-11-06T03:12:46.393609+0000","last_undegraded":"2023-11-06T03:12:46.393609+0000","last_fullsized":"2023-11-06T03:12:46.393609+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:32:51.357395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.388947+0000","last_change":"2023-11-06T03:12:45.025978+0000","last_active":"2023-11-06T03:12:45.388947+0000","last_peered":"2023-11-06T03:12:45.388947+0000","last_clean":"2023-11-06T03:12:45.388947+0000","last_became_active":"2023-11-06T03:12:45.025242+0000","last_became_peered":"2023-11-06T03:12:45.025242+0000","last_unstale":"2023-11-06T03:12:45.388947+0000","last_undegraded":"2023-11-06T03:12:45.388947+0000","last_fullsized":"2023-11-06T03:12:45.388947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.394881+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:12:45.394881+0000","last_peered":"2023-11-06T03:12:45.394881+0000","last_clean":"2023-11-06T03:12:45.394881+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:12:45.394881+0000","last_undegraded":"2023-11-06T03:12:45.394881+0000","last_fullsized":"2023-11-06T03:12:45.394881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862941+0000","last_change":"2023-11-06T03:12:45.398634+0000","last_active":"2023-11-06T03:12:46.862941+0000","last_peered":"2023-11-06T03:12:46.862941+0000","last_clean":"2023-11-06T03:12:46.862941+0000","last_became_active":"2023-11-06T03:12:45.394089+0000","last_became_peered":"2023-11-06T03:12:45.394089+0000","last_unstale":"2023-11-06T03:12:46.862941+0000","last_undegraded":"2023-11-06T03:12:46.862941+0000","last_fullsized":"2023-11-06T03:12:46.862941+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:33:36.000007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.385984+0000","last_change":"2023-11-06T03:12:45.025515+0000","last_active":"2023-11-06T03:12:45.385984+0000","last_peered":"2023-11-06T03:12:45.385984+0000","last_clean":"2023-11-06T03:12:45.385984+0000","last_became_active":"2023-11-06T03:12:45.025188+0000","last_became_peered":"2023-11-06T03:12:45.025188+0000","last_unstale":"2023-11-06T03:12:45.385984+0000","last_undegraded":"2023-11-06T03:12:45.385984+0000","last_fullsized":"2023-11-06T03:12:45.385984+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.398020+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:12:45.398020+0000","last_peered":"2023-11-06T03:12:45.398020+0000","last_clean":"2023-11-06T03:12:45.398020+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:12:45.398020+0000","last_undegraded":"2023-11-06T03:12:45.398020+0000","last_fullsized":"2023-11-06T03:12:45.398020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585620+0000","last_change":"2023-11-06T03:12:45.407853+0000","last_active":"2023-11-06T03:12:46.585620+0000","last_peered":"2023-11-06T03:12:46.585620+0000","last_clean":"2023-11-06T03:12:46.585620+0000","last_became_active":"2023-11-06T03:12:45.407590+0000","last_became_peered":"2023-11-06T03:12:45.407590+0000","last_unstale":"2023-11-06T03:12:46.585620+0000","last_undegraded":"2023-11-06T03:12:46.585620+0000","last_fullsized":"2023-11-06T03:12:46.585620+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:33.279183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.407267+0000","last_change":"2023-11-06T03:12:45.018966+0000","last_active":"2023-11-06T03:12:45.407267+0000","last_peered":"2023-11-06T03:12:45.407267+0000","last_clean":"2023-11-06T03:12:45.407267+0000","last_became_active":"2023-11-06T03:12:45.018670+0000","last_became_peered":"2023-11-06T03:12:45.018670+0000","last_unstale":"2023-11-06T03:12:45.407267+0000","last_undegraded":"2023-11-06T03:12:45.407267+0000","last_fullsized":"2023-11-06T03:12:45.407267+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395242+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:12:45.395242+0000","last_peered":"2023-11-06T03:12:45.395242+0000","last_clean":"2023-11-06T03:12:45.395242+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:12:45.395242+0000","last_undegraded":"2023-11-06T03:12:45.395242+0000","last_fullsized":"2023-11-06T03:12:45.395242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585779+0000","last_change":"2023-11-06T03:12:45.417398+0000","last_active":"2023-11-06T03:12:46.585779+0000","last_peered":"2023-11-06T03:12:46.585779+0000","last_clean":"2023-11-06T03:12:46.585779+0000","last_became_active":"2023-11-06T03:12:45.417063+0000","last_became_peered":"2023-11-06T03:12:45.417063+0000","last_unstale":"2023-11-06T03:12:46.585779+0000","last_undegraded":"2023-11-06T03:12:46.585779+0000","last_fullsized":"2023-11-06T03:12:46.585779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:20.006738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.392934+0000","last_change":"2023-11-06T03:12:45.024770+0000","last_active":"2023-11-06T03:12:45.392934+0000","last_peered":"2023-11-06T03:12:45.392934+0000","last_clean":"2023-11-06T03:12:45.392934+0000","last_became_active":"2023-11-06T03:12:45.024481+0000","last_became_peered":"2023-11-06T03:12:45.024481+0000","last_unstale":"2023-11-06T03:12:45.392934+0000","last_undegraded":"2023-11-06T03:12:45.392934+0000","last_fullsized":"2023-11-06T03:12:45.392934+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.389894+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:12:45.389894+0000","last_peered":"2023-11-06T03:12:45.389894+0000","last_clean":"2023-11-06T03:12:45.389894+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:12:45.389894+0000","last_undegraded":"2023-11-06T03:12:45.389894+0000","last_fullsized":"2023-11-06T03:12:45.389894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87056,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284784,"statfs":{"total":96636764160,"available":96547618816,"internally_reserved":0,"allocated":1974272,"data_stored":1222040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.117,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83299999999999996,"15min":0.86599999999999999},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.367,"15min":1.397},"last":0.93000000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.84899999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2490000000000001,"5min":1.254,"15min":1.444},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.218,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.92900000000000005,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1399999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0820000000000001,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":1.0229999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.66600000000000004,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":2.512,"15min":2.512},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.1299999999999999,"15min":1.0580000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.1799999999999999,"5min":10.596,"15min":10.596},"last":0.874}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.2709999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.54,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.056,"15min":1.0469999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.85499999999999998,"5min":0.68600000000000005,"15min":0.55200000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.081}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.171,"15min":1.1579999999999999},"min":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.8149999999999999,"5min":2.548,"15min":2.548},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.7370000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":0.82699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80724,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":79360,"kb_avail":94291116,"statfs":{"total":96636764160,"available":96554102784,"internally_reserved":0,"allocated":1388544,"data_stored":632661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81264640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.98199999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.1870000000000001,"5min":16.073,"15min":16.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.288,"15min":0.97799999999999998},"min":{"1min":0.53100000000000003,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":40.905000000000001,"15min":40.905000000000001},"last":1.038}]},{"osd":1,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.056,"15min":0.90200000000000002},"min":{"1min":0.55800000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2370000000000001,"5min":16.012,"15min":16.012},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90800000000000003,"15min":0.88200000000000001},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.409,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.137,"15min":0.91000000000000003},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.111,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67500000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.2709999999999999,"5min":2.843,"15min":2.843},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.647,"15min":1.1899999999999999},"min":{"1min":0.67300000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.306,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.377,"15min":1.0529999999999999},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.1830000000000001,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.3440000000000001,"15min":1.1870000000000001},"min":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2290000000000001,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.4219999999999999,"15min":1.2450000000000001},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.593,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.427,"15min":1.1619999999999999},"min":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.1990000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.159,"15min":1.1180000000000001},"min":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.3129999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.4179999999999999,"15min":1.2070000000000001},"min":{"1min":0.749,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7629999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.423,"15min":1.2170000000000001},"min":{"1min":0.86099999999999999,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.3759999999999999,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313212,"statfs":{"total":96636764160,"available":96576729088,"internally_reserved":0,"allocated":1372160,"data_stored":631512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.7849999999999999,"15min":1.1339999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.98499999999999999,"5min":125.256,"15min":125.256},"last":0.996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.833,"15min":1.165},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.373,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86599999999999999,"15min":0.77100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2010000000000001,"5min":2.339,"15min":2.339},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86299999999999999,"15min":0.84299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.129,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.83999999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1120000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.1080000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.86199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.3129999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.077},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.94399999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.256,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.105,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.004,"15min":0.97399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.4650000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.1459999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.02,"15min":1.0029999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.05,"15min":1.02},"min":{"1min":0.432,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.234,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.077,"15min":1.016},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.4370000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.464,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.044}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90572,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281268,"statfs":{"total":96636764160,"available":96544018432,"internally_reserved":0,"allocated":1380352,"data_stored":631604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.262,"15min":0.97099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3939999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.3200000000000001,"15min":0.96499999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.0830000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.68600000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.82199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.80900000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.872,"15min":0.997},"min":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.452,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.88800000000000001,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.76900000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.94999999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.0680000000000001,"15min":1.034},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"max":{"1min":10.192,"5min":10.192,"15min":10.192},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":5.3310000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.84299999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95699999999999996,"15min":1.002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0609999999999999,"15min":1.125},"min":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.506,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.089}]},{"osd":7,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1120000000000001,"15min":1.175},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.1830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76676,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295164,"statfs":{"total":96636764160,"available":96558247936,"internally_reserved":0,"allocated":1372160,"data_stored":631075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.96799999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.85399999999999998,"5min":1.149,"15min":1.524},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.625},"min":{"1min":0.434,"5min":0.224,"15min":0.224},"max":{"1min":0.95799999999999996,"5min":1.371,"15min":1.371},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.206,"5min":1.532,"15min":1.532},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.052,"5min":1.284,"15min":1.284},"last":1.1339999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.505,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.2609999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.048},{"interface":"front","average":{"1min":0.997,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.61299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4610000000000001,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.2210000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.97599999999999998,"15min":0.98299999999999998},"min":{"1min":0.73599999999999999,"5min":0.496,"15min":0.46899999999999997},"max":{"1min":1.319,"5min":1.536,"15min":1.6639999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.413,"5min":1.5049999999999999,"15min":1.698},"last":1.321}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.995,"15min":1.0049999999999999},"min":{"1min":0.71399999999999997,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.51,"5min":1.649,"15min":1.8620000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96799999999999997,"15min":0.98999999999999999},"min":{"1min":0.67200000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3660000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.175}]},{"osd":7,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3779999999999999,"5min":1.835,"15min":1.835},"last":1.26},{"interface":"front","average":{"1min":1.135,"5min":1.0529999999999999,"15min":1.0609999999999999},"min":{"1min":0.77300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5629999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.3620000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1376256,"data_stored":631546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.90000000000000002,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.27600000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.85099999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.29299999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.997,"5min":1.1160000000000001,"15min":1.121},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.67100000000000004},"min":{"1min":0.33300000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.294,"15min":1.294},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.72799999999999998},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.1100000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.1859999999999999,"5min":1.526,"15min":1.526},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.95199999999999996},"min":{"1min":0.61299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.274,"5min":1.486,"15min":1.5780000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98199999999999998,"15min":0.98399999999999999},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.615,"15min":1.615},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.93300000000000005,"15min":0.94399999999999995},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.238,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874}]},{"osd":7,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91000000000000003,"15min":0.94699999999999995},"min":{"1min":0.65000000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":3.4740000000000002},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46344,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325496,"statfs":{"total":96636764160,"available":96589307904,"internally_reserved":0,"allocated":1376256,"data_stored":631546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.1240000000000001,"5min":1.843,"15min":1.843},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.89800000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.71299999999999997},"min":{"1min":0.27900000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.86099999999999999,"5min":1.169,"15min":30.861000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.35299999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.7330000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.71099999999999997},"min":{"1min":0.33700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.151,"5min":1.151,"15min":2.3079999999999998},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86099999999999999,"15min":0.86899999999999999},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.276,"5min":2.0870000000000002,"15min":2.258},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2030000000000001,"5min":1.736,"15min":14.753},"last":0.77700000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":1.1160000000000001},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.528,"15min":43.103999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96299999999999997,"15min":1.1459999999999999},"min":{"1min":0.66600000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.2909999999999999,"5min":1.4279999999999999,"15min":44.753999999999998},"last":1.046}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":1.0529999999999999},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.1639999999999999,"5min":1.6930000000000001,"15min":23.971},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93899999999999995,"15min":1.0800000000000001},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.444,"5min":1.617,"15min":14.813000000000001},"last":1.032}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.996,"15min":1.052},"min":{"1min":0.67100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":1.135},{"interface":"front","average":{"1min":1.008,"5min":1.002,"15min":1.0369999999999999},"min":{"1min":0.71899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4470000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":1.002}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91220,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280620,"statfs":{"total":96636764160,"available":96543354880,"internally_reserved":0,"allocated":1978368,"data_stored":1222515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0569999999999999,"5min":1.145,"15min":2.6989999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.46600000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.91800000000000004,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.73699999999999999,"15min":0.72799999999999998},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":2.7999999999999998},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3560000000000001,"5min":1.446,"15min":4.641},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.91300000000000003},"min":{"1min":0.55600000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.131,"5min":1.286,"15min":2.028},"last":0.63800000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.624,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.53200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.27,"5min":1.405,"15min":4.7430000000000003},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2549999999999999,"15min":2.931},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.004,"5min":0.99199999999999999,"15min":1.01},"min":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.254,"5min":1.802,"15min":2.7789999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96699999999999997,"15min":1.0049999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.306,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:52.925 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:12:53.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 Processor -- start 2023-11-06T03:12:53.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 -- start start 2023-11-06T03:12:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb8ac0b3b20 0x7fb8ac15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac15f580 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac15efc0 con 0x7fb8ac1551e0 2023-11-06T03:12:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.057+0000 7fb8b1f22640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac0704b0 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8ab7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb8ac1551e0 0x7fb8ac06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:41452/0 (socket says 172.21.15.157:41452) 2023-11-06T03:12:53.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8ab7fe640 1 -- 172.21.15.157:0/3426008987 learned_addr learned my addr 172.21.15.157:0/3426008987 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:53.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aaffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb8ac15e5e0 0x7fb8ac0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32834/0 (socket says 172.21.15.157:32834) 2023-11-06T03:12:53.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8abfff640 1 --2- 172.21.15.157:0/3426008987 >> v2:172.21.15.195:3300/0 conn(0x7fb8ac0b3b20 0x7fb8ac15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425623506 0 0) 0x7fb8ac15f580 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb894003530 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851778967 0 0) 0x7fb8ac15efc0 con 0x7fb8ac1551e0 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ac15f580 con 0x7fb8ac1551e0 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813161726 0 0) 0x7fb894003530 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8abfff640 1 -- 172.21.15.157:0/3426008987 >> v1:172.21.15.195:6789/0 conn(0x7fb8ac1551e0 legacy=0x7fb8ac06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8abfff640 1 -- 172.21.15.157:0/3426008987 >> v1:172.21.15.157:6789/0 conn(0x7fb8ac15e5e0 legacy=0x7fb8ac0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8abfff640 1 -- 172.21.15.157:0/3426008987 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac15f2a0 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8abfff640 1 --2- 172.21.15.157:0/3426008987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac15c1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009d90 tx=0x7fb8a401c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d9297a4f02153d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0001680 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40a5560 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.061+0000 7fb8aa7fc640 1 -- 172.21.15.157:0/3426008987 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40a58d0 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/3426008987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 msgr2=0x7fb8ac15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/3426008987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac15c1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009d90 tx=0x7fb8a401c720 comp rx=0 tx=0).stop 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/3426008987 shutdown_connections 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/3426008987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac15c1b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/3426008987 >> 172.21.15.157:0/3426008987 conn(0x7fb8ac065930 msgr2=0x7fb8ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/3426008987 shutdown_connections 2023-11-06T03:12:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/3426008987 wait complete. 2023-11-06T03:12:53.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 Processor -- start 2023-11-06T03:12:53.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- start start 2023-11-06T03:12:53.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac1551e0 0x7fb8ac1383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac14c300 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac138910 con 0x7fb8ac0b3b20 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac14bc70 con 0x7fb8ac1551e0 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53794/0 (socket says 172.21.15.157:53794) 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 -- 172.21.15.157:0/4196191121 learned_addr learned my addr 172.21.15.157:0/4196191121 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8ab7fe640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8aaffd640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac1551e0 0x7fb8ac1383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 -- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac1551e0 msgr2=0x7fb8ac1383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac1551e0 0x7fb8ac1383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 -- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 msgr2=0x7fb8ac0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac0b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 -- 172.21.15.157:0/4196191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4009a70 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8abfff640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009d90 tx=0x7fb8a40194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40ad640 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40ad7a0 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac14b060 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40adb10 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.065+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac14b620 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ac072750 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb8a40b4070 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb893fff640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 0x7fb898067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb8a411def0 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb8ab7fe640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 0x7fb898067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.069+0000 7fb8ab7fe640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 0x7fb898067f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0011f0 tx=0x7fb89c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.073+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb8a40eb8e0 con 0x7fb8ac15e5e0 2023-11-06T03:12:53.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.221+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8ac15b1b0 con 0x7fb898065a60 2023-11-06T03:12:53.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.245+0000 7fb893fff640 1 -- 172.21.15.157:0/4196191121 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310817 (secure 0 0 0) 0x7fb89c001cf0 con 0x7fb898065a60 2023-11-06T03:12:53.249 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:12:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 msgr2=0x7fb898067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 0x7fb898067f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0011f0 tx=0x7fb89c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 msgr2=0x7fb8ac0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009d90 tx=0x7fb8a40194c0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 shutdown_connections 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb898065a60 0x7fb898067f20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8ac15e5e0 0x7fb8ac0b16d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8ac1551e0 0x7fb8ac1383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 --2- 172.21.15.157:0/4196191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8ac0b3b20 0x7fb8ac0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 >> 172.21.15.157:0/4196191121 conn(0x7fb8ac065930 msgr2=0x7fb8ac15dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 shutdown_connections 2023-11-06T03:12:53.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.249+0000 7fb8b1f22640 1 -- 172.21.15.157:0/4196191121 wait complete. 2023-11-06T03:12:53.266 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2023-11-06T03:12:51.286231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":599644,"kb_used_data":11932,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754375076,"statfs":{"total":773094113280,"available":772480077824,"internally_reserved":0,"allocated":12218368,"data_stored":6234499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":64,"apply_latency_ms":64,"commit_latency_ns":64000000,"apply_latency_ns":64000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.902532"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585920+0000","last_change":"2023-11-06T03:12:45.411100+0000","last_active":"2023-11-06T03:12:46.585920+0000","last_peered":"2023-11-06T03:12:46.585920+0000","last_clean":"2023-11-06T03:12:46.585920+0000","last_became_active":"2023-11-06T03:12:45.410664+0000","last_became_peered":"2023-11-06T03:12:45.410664+0000","last_unstale":"2023-11-06T03:12:46.585920+0000","last_undegraded":"2023-11-06T03:12:46.585920+0000","last_fullsized":"2023-11-06T03:12:46.585920+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:32.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864522+0000","last_change":"2023-11-06T03:12:45.433350+0000","last_active":"2023-11-06T03:12:46.864522+0000","last_peered":"2023-11-06T03:12:46.864522+0000","last_clean":"2023-11-06T03:12:46.864522+0000","last_became_active":"2023-11-06T03:12:45.432683+0000","last_became_peered":"2023-11-06T03:12:45.432683+0000","last_unstale":"2023-11-06T03:12:46.864522+0000","last_undegraded":"2023-11-06T03:12:46.864522+0000","last_fullsized":"2023-11-06T03:12:46.864522+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:30:57.783818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585577+0000","last_change":"2023-11-06T03:12:45.440384+0000","last_active":"2023-11-06T03:12:46.585577+0000","last_peered":"2023-11-06T03:12:46.585577+0000","last_clean":"2023-11-06T03:12:46.585577+0000","last_became_active":"2023-11-06T03:12:45.440062+0000","last_became_peered":"2023-11-06T03:12:45.440062+0000","last_unstale":"2023-11-06T03:12:46.585577+0000","last_undegraded":"2023-11-06T03:12:46.585577+0000","last_fullsized":"2023-11-06T03:12:46.585577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:12:35.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393776+0000","last_change":"2023-11-06T03:12:45.415909+0000","last_active":"2023-11-06T03:12:46.393776+0000","last_peered":"2023-11-06T03:12:46.393776+0000","last_clean":"2023-11-06T03:12:46.393776+0000","last_became_active":"2023-11-06T03:12:45.415628+0000","last_became_peered":"2023-11-06T03:12:45.415628+0000","last_unstale":"2023-11-06T03:12:46.393776+0000","last_undegraded":"2023-11-06T03:12:46.393776+0000","last_fullsized":"2023-11-06T03:12:46.393776+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861369+0000","last_change":"2023-11-06T03:12:45.409517+0000","last_active":"2023-11-06T03:12:46.861369+0000","last_peered":"2023-11-06T03:12:46.861369+0000","last_clean":"2023-11-06T03:12:46.861369+0000","last_became_active":"2023-11-06T03:12:45.409162+0000","last_became_peered":"2023-11-06T03:12:45.409162+0000","last_unstale":"2023-11-06T03:12:46.861369+0000","last_undegraded":"2023-11-06T03:12:46.861369+0000","last_fullsized":"2023-11-06T03:12:46.861369+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:22.231105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863635+0000","last_change":"2023-11-06T03:12:45.406222+0000","last_active":"2023-11-06T03:12:46.863635+0000","last_peered":"2023-11-06T03:12:46.863635+0000","last_clean":"2023-11-06T03:12:46.863635+0000","last_became_active":"2023-11-06T03:12:45.405637+0000","last_became_peered":"2023-11-06T03:12:45.405637+0000","last_unstale":"2023-11-06T03:12:46.863635+0000","last_undegraded":"2023-11-06T03:12:46.863635+0000","last_fullsized":"2023-11-06T03:12:46.863635+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:27:06.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864018+0000","last_change":"2023-11-06T03:12:45.396241+0000","last_active":"2023-11-06T03:12:46.864018+0000","last_peered":"2023-11-06T03:12:46.864018+0000","last_clean":"2023-11-06T03:12:46.864018+0000","last_became_active":"2023-11-06T03:12:45.395412+0000","last_became_peered":"2023-11-06T03:12:45.395412+0000","last_unstale":"2023-11-06T03:12:46.864018+0000","last_undegraded":"2023-11-06T03:12:46.864018+0000","last_fullsized":"2023-11-06T03:12:46.864018+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:20:32.924101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.860803+0000","last_change":"2023-11-06T03:12:45.400987+0000","last_active":"2023-11-06T03:12:46.860803+0000","last_peered":"2023-11-06T03:12:46.860803+0000","last_clean":"2023-11-06T03:12:46.860803+0000","last_became_active":"2023-11-06T03:12:45.400723+0000","last_became_peered":"2023-11-06T03:12:45.400723+0000","last_unstale":"2023-11-06T03:12:46.860803+0000","last_undegraded":"2023-11-06T03:12:46.860803+0000","last_fullsized":"2023-11-06T03:12:46.860803+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:22:32.347484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.865265+0000","last_change":"2023-11-06T03:12:45.419973+0000","last_active":"2023-11-06T03:12:46.865265+0000","last_peered":"2023-11-06T03:12:46.865265+0000","last_clean":"2023-11-06T03:12:46.865265+0000","last_became_active":"2023-11-06T03:12:45.419264+0000","last_became_peered":"2023-11-06T03:12:45.419264+0000","last_unstale":"2023-11-06T03:12:46.865265+0000","last_undegraded":"2023-11-06T03:12:46.865265+0000","last_fullsized":"2023-11-06T03:12:46.865265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:03.447759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862096+0000","last_change":"2023-11-06T03:12:45.415201+0000","last_active":"2023-11-06T03:12:46.862096+0000","last_peered":"2023-11-06T03:12:46.862096+0000","last_clean":"2023-11-06T03:12:46.862096+0000","last_became_active":"2023-11-06T03:12:45.414958+0000","last_became_peered":"2023-11-06T03:12:45.414958+0000","last_unstale":"2023-11-06T03:12:46.862096+0000","last_undegraded":"2023-11-06T03:12:46.862096+0000","last_fullsized":"2023-11-06T03:12:46.862096+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:43.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862415+0000","last_change":"2023-11-06T03:12:45.442566+0000","last_active":"2023-11-06T03:12:46.862415+0000","last_peered":"2023-11-06T03:12:46.862415+0000","last_clean":"2023-11-06T03:12:46.862415+0000","last_became_active":"2023-11-06T03:12:45.440018+0000","last_became_peered":"2023-11-06T03:12:45.440018+0000","last_unstale":"2023-11-06T03:12:46.862415+0000","last_undegraded":"2023-11-06T03:12:46.862415+0000","last_fullsized":"2023-11-06T03:12:46.862415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401886+0000","last_change":"2023-11-06T03:12:45.407208+0000","last_active":"2023-11-06T03:12:46.401886+0000","last_peered":"2023-11-06T03:12:46.401886+0000","last_clean":"2023-11-06T03:12:46.401886+0000","last_became_active":"2023-11-06T03:12:45.406425+0000","last_became_peered":"2023-11-06T03:12:45.406425+0000","last_unstale":"2023-11-06T03:12:46.401886+0000","last_undegraded":"2023-11-06T03:12:46.401886+0000","last_fullsized":"2023-11-06T03:12:46.401886+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:32:51.357395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393715+0000","last_change":"2023-11-06T03:12:45.443524+0000","last_active":"2023-11-06T03:12:46.393715+0000","last_peered":"2023-11-06T03:12:46.393715+0000","last_clean":"2023-11-06T03:12:46.393715+0000","last_became_active":"2023-11-06T03:12:45.443230+0000","last_became_peered":"2023-11-06T03:12:45.443230+0000","last_unstale":"2023-11-06T03:12:46.393715+0000","last_undegraded":"2023-11-06T03:12:46.393715+0000","last_fullsized":"2023-11-06T03:12:46.393715+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863090+0000","last_change":"2023-11-06T03:12:45.431179+0000","last_active":"2023-11-06T03:12:46.863090+0000","last_peered":"2023-11-06T03:12:46.863090+0000","last_clean":"2023-11-06T03:12:46.863090+0000","last_became_active":"2023-11-06T03:12:45.430649+0000","last_became_peered":"2023-11-06T03:12:45.430649+0000","last_unstale":"2023-11-06T03:12:46.863090+0000","last_undegraded":"2023-11-06T03:12:46.863090+0000","last_fullsized":"2023-11-06T03:12:46.863090+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862976+0000","last_change":"2023-11-06T03:12:45.445000+0000","last_active":"2023-11-06T03:12:46.862976+0000","last_peered":"2023-11-06T03:12:46.862976+0000","last_clean":"2023-11-06T03:12:46.862976+0000","last_became_active":"2023-11-06T03:12:45.444456+0000","last_became_peered":"2023-11-06T03:12:45.444456+0000","last_unstale":"2023-11-06T03:12:46.862976+0000","last_undegraded":"2023-11-06T03:12:46.862976+0000","last_fullsized":"2023-11-06T03:12:46.862976+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863136+0000","last_change":"2023-11-06T03:12:45.430924+0000","last_active":"2023-11-06T03:12:46.863136+0000","last_peered":"2023-11-06T03:12:46.863136+0000","last_clean":"2023-11-06T03:12:46.863136+0000","last_became_active":"2023-11-06T03:12:45.430142+0000","last_became_peered":"2023-11-06T03:12:45.430142+0000","last_unstale":"2023-11-06T03:12:46.863136+0000","last_undegraded":"2023-11-06T03:12:46.863136+0000","last_fullsized":"2023-11-06T03:12:46.863136+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.586021+0000","last_change":"2023-11-06T03:12:45.441412+0000","last_active":"2023-11-06T03:12:46.586021+0000","last_peered":"2023-11-06T03:12:46.586021+0000","last_clean":"2023-11-06T03:12:46.586021+0000","last_became_active":"2023-11-06T03:12:45.441142+0000","last_became_peered":"2023-11-06T03:12:45.441142+0000","last_unstale":"2023-11-06T03:12:46.586021+0000","last_undegraded":"2023-11-06T03:12:46.586021+0000","last_fullsized":"2023-11-06T03:12:46.586021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:24:43.039951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864755+0000","last_change":"2023-11-06T03:12:45.405413+0000","last_active":"2023-11-06T03:12:46.864755+0000","last_peered":"2023-11-06T03:12:46.864755+0000","last_clean":"2023-11-06T03:12:46.864755+0000","last_became_active":"2023-11-06T03:12:45.404678+0000","last_became_peered":"2023-11-06T03:12:45.404678+0000","last_unstale":"2023-11-06T03:12:46.864755+0000","last_undegraded":"2023-11-06T03:12:46.864755+0000","last_fullsized":"2023-11-06T03:12:46.864755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:54.295847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585763+0000","last_change":"2023-11-06T03:12:45.425060+0000","last_active":"2023-11-06T03:12:46.585763+0000","last_peered":"2023-11-06T03:12:46.585763+0000","last_clean":"2023-11-06T03:12:46.585763+0000","last_became_active":"2023-11-06T03:12:45.424628+0000","last_became_peered":"2023-11-06T03:12:45.424628+0000","last_unstale":"2023-11-06T03:12:46.585763+0000","last_undegraded":"2023-11-06T03:12:46.585763+0000","last_fullsized":"2023-11-06T03:12:46.585763+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:20.568857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402030+0000","last_change":"2023-11-06T03:12:45.431500+0000","last_active":"2023-11-06T03:12:46.402030+0000","last_peered":"2023-11-06T03:12:46.402030+0000","last_clean":"2023-11-06T03:12:46.402030+0000","last_became_active":"2023-11-06T03:12:45.430942+0000","last_became_peered":"2023-11-06T03:12:45.430942+0000","last_unstale":"2023-11-06T03:12:46.402030+0000","last_undegraded":"2023-11-06T03:12:46.402030+0000","last_fullsized":"2023-11-06T03:12:46.402030+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:52.901862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862907+0000","last_change":"2023-11-06T03:12:45.440754+0000","last_active":"2023-11-06T03:12:46.862907+0000","last_peered":"2023-11-06T03:12:46.862907+0000","last_clean":"2023-11-06T03:12:46.862907+0000","last_became_active":"2023-11-06T03:12:45.440484+0000","last_became_peered":"2023-11-06T03:12:45.440484+0000","last_unstale":"2023-11-06T03:12:46.862907+0000","last_undegraded":"2023-11-06T03:12:46.862907+0000","last_fullsized":"2023-11-06T03:12:46.862907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:04.674354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861706+0000","last_change":"2023-11-06T03:12:45.439448+0000","last_active":"2023-11-06T03:12:46.861706+0000","last_peered":"2023-11-06T03:12:46.861706+0000","last_clean":"2023-11-06T03:12:46.861706+0000","last_became_active":"2023-11-06T03:12:45.439208+0000","last_became_peered":"2023-11-06T03:12:45.439208+0000","last_unstale":"2023-11-06T03:12:46.861706+0000","last_undegraded":"2023-11-06T03:12:46.861706+0000","last_fullsized":"2023-11-06T03:12:46.861706+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:18.531765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585564+0000","last_change":"2023-11-06T03:12:45.440267+0000","last_active":"2023-11-06T03:12:46.585564+0000","last_peered":"2023-11-06T03:12:46.585564+0000","last_clean":"2023-11-06T03:12:46.585564+0000","last_became_active":"2023-11-06T03:12:45.440019+0000","last_became_peered":"2023-11-06T03:12:45.440019+0000","last_unstale":"2023-11-06T03:12:46.585564+0000","last_undegraded":"2023-11-06T03:12:46.585564+0000","last_fullsized":"2023-11-06T03:12:46.585564+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:19:56.240869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.860792+0000","last_change":"2023-11-06T03:12:45.402001+0000","last_active":"2023-11-06T03:12:46.860792+0000","last_peered":"2023-11-06T03:12:46.860792+0000","last_clean":"2023-11-06T03:12:46.860792+0000","last_became_active":"2023-11-06T03:12:45.401659+0000","last_became_peered":"2023-11-06T03:12:45.401659+0000","last_unstale":"2023-11-06T03:12:46.860792+0000","last_undegraded":"2023-11-06T03:12:46.860792+0000","last_fullsized":"2023-11-06T03:12:46.860792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:18:19.762481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402576+0000","last_change":"2023-11-06T03:12:45.447582+0000","last_active":"2023-11-06T03:12:46.402576+0000","last_peered":"2023-11-06T03:12:46.402576+0000","last_clean":"2023-11-06T03:12:46.402576+0000","last_became_active":"2023-11-06T03:12:45.446982+0000","last_became_peered":"2023-11-06T03:12:45.446982+0000","last_unstale":"2023-11-06T03:12:46.402576+0000","last_undegraded":"2023-11-06T03:12:46.402576+0000","last_fullsized":"2023-11-06T03:12:46.402576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:45:40.848163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.402241+0000","last_change":"2023-11-06T03:12:45.429871+0000","last_active":"2023-11-06T03:12:46.402241+0000","last_peered":"2023-11-06T03:12:46.402241+0000","last_clean":"2023-11-06T03:12:46.402241+0000","last_became_active":"2023-11-06T03:12:45.429136+0000","last_became_peered":"2023-11-06T03:12:45.429136+0000","last_unstale":"2023-11-06T03:12:46.402241+0000","last_undegraded":"2023-11-06T03:12:46.402241+0000","last_fullsized":"2023-11-06T03:12:46.402241+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:23.719299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862570+0000","last_change":"2023-11-06T03:12:45.440192+0000","last_active":"2023-11-06T03:12:46.862570+0000","last_peered":"2023-11-06T03:12:46.862570+0000","last_clean":"2023-11-06T03:12:46.862570+0000","last_became_active":"2023-11-06T03:12:45.438692+0000","last_became_peered":"2023-11-06T03:12:45.438692+0000","last_unstale":"2023-11-06T03:12:46.862570+0000","last_undegraded":"2023-11-06T03:12:46.862570+0000","last_fullsized":"2023-11-06T03:12:46.862570+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:47:22.231105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863256+0000","last_change":"2023-11-06T03:12:45.443057+0000","last_active":"2023-11-06T03:12:46.863256+0000","last_peered":"2023-11-06T03:12:46.863256+0000","last_clean":"2023-11-06T03:12:46.863256+0000","last_became_active":"2023-11-06T03:12:45.442480+0000","last_became_peered":"2023-11-06T03:12:45.442480+0000","last_unstale":"2023-11-06T03:12:46.863256+0000","last_undegraded":"2023-11-06T03:12:46.863256+0000","last_fullsized":"2023-11-06T03:12:46.863256+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:20.970476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862741+0000","last_change":"2023-11-06T03:12:45.430274+0000","last_active":"2023-11-06T03:12:46.862741+0000","last_peered":"2023-11-06T03:12:46.862741+0000","last_clean":"2023-11-06T03:12:46.862741+0000","last_became_active":"2023-11-06T03:12:45.429904+0000","last_became_peered":"2023-11-06T03:12:45.429904+0000","last_unstale":"2023-11-06T03:12:46.862741+0000","last_undegraded":"2023-11-06T03:12:46.862741+0000","last_fullsized":"2023-11-06T03:12:46.862741+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:23.593202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401094+0000","last_change":"2023-11-06T03:12:45.445231+0000","last_active":"2023-11-06T03:12:46.401094+0000","last_peered":"2023-11-06T03:12:46.401094+0000","last_clean":"2023-11-06T03:12:46.401094+0000","last_became_active":"2023-11-06T03:12:45.444688+0000","last_became_peered":"2023-11-06T03:12:45.444688+0000","last_unstale":"2023-11-06T03:12:46.401094+0000","last_undegraded":"2023-11-06T03:12:46.401094+0000","last_fullsized":"2023-11-06T03:12:46.401094+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:07:41.886545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864262+0000","last_change":"2023-11-06T03:12:45.412475+0000","last_active":"2023-11-06T03:12:46.864262+0000","last_peered":"2023-11-06T03:12:46.864262+0000","last_clean":"2023-11-06T03:12:46.864262+0000","last_became_active":"2023-11-06T03:12:45.411982+0000","last_became_peered":"2023-11-06T03:12:45.411982+0000","last_unstale":"2023-11-06T03:12:46.864262+0000","last_undegraded":"2023-11-06T03:12:46.864262+0000","last_fullsized":"2023-11-06T03:12:46.864262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:20:17.882004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861000+0000","last_change":"2023-11-06T03:12:45.421138+0000","last_active":"2023-11-06T03:12:46.861000+0000","last_peered":"2023-11-06T03:12:46.861000+0000","last_clean":"2023-11-06T03:12:46.861000+0000","last_became_active":"2023-11-06T03:12:45.420849+0000","last_became_peered":"2023-11-06T03:12:45.420849+0000","last_unstale":"2023-11-06T03:12:46.861000+0000","last_undegraded":"2023-11-06T03:12:46.861000+0000","last_fullsized":"2023-11-06T03:12:46.861000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:53:30.582264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585526+0000","last_change":"2023-11-06T03:12:45.412635+0000","last_active":"2023-11-06T03:12:46.585526+0000","last_peered":"2023-11-06T03:12:46.585526+0000","last_clean":"2023-11-06T03:12:46.585526+0000","last_became_active":"2023-11-06T03:12:45.412318+0000","last_became_peered":"2023-11-06T03:12:45.412318+0000","last_unstale":"2023-11-06T03:12:46.585526+0000","last_undegraded":"2023-11-06T03:12:46.585526+0000","last_fullsized":"2023-11-06T03:12:46.585526+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:02:28.025829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.427553+0000","last_change":"2023-11-06T03:12:45.024196+0000","last_active":"2023-11-06T03:12:45.427553+0000","last_peered":"2023-11-06T03:12:45.427553+0000","last_clean":"2023-11-06T03:12:45.427553+0000","last_became_active":"2023-11-06T03:12:45.023925+0000","last_became_peered":"2023-11-06T03:12:45.023925+0000","last_unstale":"2023-11-06T03:12:45.427553+0000","last_undegraded":"2023-11-06T03:12:45.427553+0000","last_fullsized":"2023-11-06T03:12:45.427553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862430+0000","last_change":"2023-11-06T03:12:45.402369+0000","last_active":"2023-11-06T03:12:46.862430+0000","last_peered":"2023-11-06T03:12:46.862430+0000","last_clean":"2023-11-06T03:12:46.862430+0000","last_became_active":"2023-11-06T03:12:45.401994+0000","last_became_peered":"2023-11-06T03:12:45.401994+0000","last_unstale":"2023-11-06T03:12:46.862430+0000","last_undegraded":"2023-11-06T03:12:46.862430+0000","last_fullsized":"2023-11-06T03:12:46.862430+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:25:41.489850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.429710+0000","last_change":"2023-11-06T03:12:45.021988+0000","last_active":"2023-11-06T03:12:45.429710+0000","last_peered":"2023-11-06T03:12:45.429710+0000","last_clean":"2023-11-06T03:12:45.429710+0000","last_became_active":"2023-11-06T03:12:45.021194+0000","last_became_peered":"2023-11-06T03:12:45.021194+0000","last_unstale":"2023-11-06T03:12:45.429710+0000","last_undegraded":"2023-11-06T03:12:45.429710+0000","last_fullsized":"2023-11-06T03:12:45.429710+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:05:51.845353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.401329+0000","last_change":"2023-11-06T03:12:45.421142+0000","last_active":"2023-11-06T03:12:46.401329+0000","last_peered":"2023-11-06T03:12:46.401329+0000","last_clean":"2023-11-06T03:12:46.401329+0000","last_became_active":"2023-11-06T03:12:45.420482+0000","last_became_peered":"2023-11-06T03:12:45.420482+0000","last_unstale":"2023-11-06T03:12:46.401329+0000","last_undegraded":"2023-11-06T03:12:46.401329+0000","last_fullsized":"2023-11-06T03:12:46.401329+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:32.568478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.446283+0000","last_change":"2023-11-06T03:12:45.035200+0000","last_active":"2023-11-06T03:12:45.446283+0000","last_peered":"2023-11-06T03:12:45.446283+0000","last_clean":"2023-11-06T03:12:45.446283+0000","last_became_active":"2023-11-06T03:12:45.034474+0000","last_became_peered":"2023-11-06T03:12:45.034474+0000","last_unstale":"2023-11-06T03:12:45.446283+0000","last_undegraded":"2023-11-06T03:12:45.446283+0000","last_fullsized":"2023-11-06T03:12:45.446283+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863762+0000","last_change":"2023-11-06T03:12:45.440093+0000","last_active":"2023-11-06T03:12:46.863762+0000","last_peered":"2023-11-06T03:12:46.863762+0000","last_clean":"2023-11-06T03:12:46.863762+0000","last_became_active":"2023-11-06T03:12:45.438786+0000","last_became_peered":"2023-11-06T03:12:45.438786+0000","last_unstale":"2023-11-06T03:12:46.863762+0000","last_undegraded":"2023-11-06T03:12:46.863762+0000","last_fullsized":"2023-11-06T03:12:46.863762+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:59.278874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.425517+0000","last_change":"2023-11-06T03:12:45.019690+0000","last_active":"2023-11-06T03:12:45.425517+0000","last_peered":"2023-11-06T03:12:45.425517+0000","last_clean":"2023-11-06T03:12:45.425517+0000","last_became_active":"2023-11-06T03:12:45.019343+0000","last_became_peered":"2023-11-06T03:12:45.019343+0000","last_unstale":"2023-11-06T03:12:45.425517+0000","last_undegraded":"2023-11-06T03:12:45.425517+0000","last_fullsized":"2023-11-06T03:12:45.425517+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:26.256995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393875+0000","last_change":"2023-11-06T03:12:45.418313+0000","last_active":"2023-11-06T03:12:46.393875+0000","last_peered":"2023-11-06T03:12:46.393875+0000","last_clean":"2023-11-06T03:12:46.393875+0000","last_became_active":"2023-11-06T03:12:45.418041+0000","last_became_peered":"2023-11-06T03:12:45.418041+0000","last_unstale":"2023-11-06T03:12:46.393875+0000","last_undegraded":"2023-11-06T03:12:46.393875+0000","last_fullsized":"2023-11-06T03:12:46.393875+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:18:19.762481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.430007+0000","last_change":"2023-11-06T03:12:45.029171+0000","last_active":"2023-11-06T03:12:45.430007+0000","last_peered":"2023-11-06T03:12:45.430007+0000","last_clean":"2023-11-06T03:12:45.430007+0000","last_became_active":"2023-11-06T03:12:45.028406+0000","last_became_peered":"2023-11-06T03:12:45.028406+0000","last_unstale":"2023-11-06T03:12:45.430007+0000","last_undegraded":"2023-11-06T03:12:45.430007+0000","last_fullsized":"2023-11-06T03:12:45.430007+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:56.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585910+0000","last_change":"2023-11-06T03:12:45.412584+0000","last_active":"2023-11-06T03:12:46.585910+0000","last_peered":"2023-11-06T03:12:46.585910+0000","last_clean":"2023-11-06T03:12:46.585910+0000","last_became_active":"2023-11-06T03:12:45.412198+0000","last_became_peered":"2023-11-06T03:12:45.412198+0000","last_unstale":"2023-11-06T03:12:46.585910+0000","last_undegraded":"2023-11-06T03:12:46.585910+0000","last_fullsized":"2023-11-06T03:12:46.585910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:32.814892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.442233+0000","last_change":"2023-11-06T03:12:45.034966+0000","last_active":"2023-11-06T03:12:45.442233+0000","last_peered":"2023-11-06T03:12:45.442233+0000","last_clean":"2023-11-06T03:12:45.442233+0000","last_became_active":"2023-11-06T03:12:45.034055+0000","last_became_peered":"2023-11-06T03:12:45.034055+0000","last_unstale":"2023-11-06T03:12:45.442233+0000","last_undegraded":"2023-11-06T03:12:45.442233+0000","last_fullsized":"2023-11-06T03:12:45.442233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864979+0000","last_change":"2023-11-06T03:12:45.439692+0000","last_active":"2023-11-06T03:12:46.864979+0000","last_peered":"2023-11-06T03:12:46.864979+0000","last_clean":"2023-11-06T03:12:46.864979+0000","last_became_active":"2023-11-06T03:12:45.439156+0000","last_became_peered":"2023-11-06T03:12:45.439156+0000","last_unstale":"2023-11-06T03:12:46.864979+0000","last_undegraded":"2023-11-06T03:12:46.864979+0000","last_fullsized":"2023-11-06T03:12:46.864979+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:06.632756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.398617+0000","last_change":"2023-11-06T03:12:45.028815+0000","last_active":"2023-11-06T03:12:45.398617+0000","last_peered":"2023-11-06T03:12:45.398617+0000","last_clean":"2023-11-06T03:12:45.398617+0000","last_became_active":"2023-11-06T03:12:45.028210+0000","last_became_peered":"2023-11-06T03:12:45.028210+0000","last_unstale":"2023-11-06T03:12:45.398617+0000","last_undegraded":"2023-11-06T03:12:45.398617+0000","last_fullsized":"2023-11-06T03:12:45.398617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:10.898002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393869+0000","last_change":"2023-11-06T03:12:45.408597+0000","last_active":"2023-11-06T03:12:46.393869+0000","last_peered":"2023-11-06T03:12:46.393869+0000","last_clean":"2023-11-06T03:12:46.393869+0000","last_became_active":"2023-11-06T03:12:45.407493+0000","last_became_peered":"2023-11-06T03:12:45.407493+0000","last_unstale":"2023-11-06T03:12:46.393869+0000","last_undegraded":"2023-11-06T03:12:46.393869+0000","last_fullsized":"2023-11-06T03:12:46.393869+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:34:43.886061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395663+0000","last_change":"2023-11-06T03:12:45.028903+0000","last_active":"2023-11-06T03:12:45.395663+0000","last_peered":"2023-11-06T03:12:45.395663+0000","last_clean":"2023-11-06T03:12:45.395663+0000","last_became_active":"2023-11-06T03:12:45.028293+0000","last_became_peered":"2023-11-06T03:12:45.028293+0000","last_unstale":"2023-11-06T03:12:45.395663+0000","last_undegraded":"2023-11-06T03:12:45.395663+0000","last_fullsized":"2023-11-06T03:12:45.395663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:28:13.789716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585467+0000","last_change":"2023-11-06T03:12:45.423639+0000","last_active":"2023-11-06T03:12:46.585467+0000","last_peered":"2023-11-06T03:12:46.585467+0000","last_clean":"2023-11-06T03:12:46.585467+0000","last_became_active":"2023-11-06T03:12:45.423343+0000","last_became_peered":"2023-11-06T03:12:45.423343+0000","last_unstale":"2023-11-06T03:12:46.585467+0000","last_undegraded":"2023-11-06T03:12:46.585467+0000","last_fullsized":"2023-11-06T03:12:46.585467+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:41.519658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.403245+0000","last_change":"2023-11-06T03:12:45.036332+0000","last_active":"2023-11-06T03:12:45.403245+0000","last_peered":"2023-11-06T03:12:45.403245+0000","last_clean":"2023-11-06T03:12:45.403245+0000","last_became_active":"2023-11-06T03:12:45.035821+0000","last_became_peered":"2023-11-06T03:12:45.035821+0000","last_unstale":"2023-11-06T03:12:45.403245+0000","last_undegraded":"2023-11-06T03:12:45.403245+0000","last_fullsized":"2023-11-06T03:12:45.403245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:50:51.523762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861196+0000","last_change":"2023-11-06T03:12:45.408527+0000","last_active":"2023-11-06T03:12:46.861196+0000","last_peered":"2023-11-06T03:12:46.861196+0000","last_clean":"2023-11-06T03:12:46.861196+0000","last_became_active":"2023-11-06T03:12:45.405334+0000","last_became_peered":"2023-11-06T03:12:45.405334+0000","last_unstale":"2023-11-06T03:12:46.861196+0000","last_undegraded":"2023-11-06T03:12:46.861196+0000","last_fullsized":"2023-11-06T03:12:46.861196+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.391302+0000","last_change":"2023-11-06T03:12:45.019118+0000","last_active":"2023-11-06T03:12:45.391302+0000","last_peered":"2023-11-06T03:12:45.391302+0000","last_clean":"2023-11-06T03:12:45.391302+0000","last_became_active":"2023-11-06T03:12:45.018401+0000","last_became_peered":"2023-11-06T03:12:45.018401+0000","last_unstale":"2023-11-06T03:12:45.391302+0000","last_undegraded":"2023-11-06T03:12:45.391302+0000","last_fullsized":"2023-11-06T03:12:45.391302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:06:54.871910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862719+0000","last_change":"2023-11-06T03:12:45.416831+0000","last_active":"2023-11-06T03:12:46.862719+0000","last_peered":"2023-11-06T03:12:46.862719+0000","last_clean":"2023-11-06T03:12:46.862719+0000","last_became_active":"2023-11-06T03:12:45.416542+0000","last_became_peered":"2023-11-06T03:12:45.416542+0000","last_unstale":"2023-11-06T03:12:46.862719+0000","last_undegraded":"2023-11-06T03:12:46.862719+0000","last_fullsized":"2023-11-06T03:12:46.862719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:13.696657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434256+0000","last_change":"2023-11-06T03:12:45.038268+0000","last_active":"2023-11-06T03:12:45.434256+0000","last_peered":"2023-11-06T03:12:45.434256+0000","last_clean":"2023-11-06T03:12:45.434256+0000","last_became_active":"2023-11-06T03:12:45.037639+0000","last_became_peered":"2023-11-06T03:12:45.037639+0000","last_unstale":"2023-11-06T03:12:45.434256+0000","last_undegraded":"2023-11-06T03:12:45.434256+0000","last_fullsized":"2023-11-06T03:12:45.434256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:36:27.303897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861619+0000","last_change":"2023-11-06T03:12:45.414318+0000","last_active":"2023-11-06T03:12:46.861619+0000","last_peered":"2023-11-06T03:12:46.861619+0000","last_clean":"2023-11-06T03:12:46.861619+0000","last_became_active":"2023-11-06T03:12:45.405366+0000","last_became_peered":"2023-11-06T03:12:45.405366+0000","last_unstale":"2023-11-06T03:12:46.861619+0000","last_undegraded":"2023-11-06T03:12:46.861619+0000","last_fullsized":"2023-11-06T03:12:46.861619+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434121+0000","last_change":"2023-11-06T03:12:45.039214+0000","last_active":"2023-11-06T03:12:45.434121+0000","last_peered":"2023-11-06T03:12:45.434121+0000","last_clean":"2023-11-06T03:12:45.434121+0000","last_became_active":"2023-11-06T03:12:45.038608+0000","last_became_peered":"2023-11-06T03:12:45.038608+0000","last_unstale":"2023-11-06T03:12:45.434121+0000","last_undegraded":"2023-11-06T03:12:45.434121+0000","last_fullsized":"2023-11-06T03:12:45.434121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:14.074368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862277+0000","last_change":"2023-11-06T03:12:45.440352+0000","last_active":"2023-11-06T03:12:46.862277+0000","last_peered":"2023-11-06T03:12:46.862277+0000","last_clean":"2023-11-06T03:12:46.862277+0000","last_became_active":"2023-11-06T03:12:45.438755+0000","last_became_peered":"2023-11-06T03:12:45.438755+0000","last_unstale":"2023-11-06T03:12:46.862277+0000","last_undegraded":"2023-11-06T03:12:46.862277+0000","last_fullsized":"2023-11-06T03:12:46.862277+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.418703+0000","last_change":"2023-11-06T03:12:45.031941+0000","last_active":"2023-11-06T03:12:45.418703+0000","last_peered":"2023-11-06T03:12:45.418703+0000","last_clean":"2023-11-06T03:12:45.418703+0000","last_became_active":"2023-11-06T03:12:45.031606+0000","last_became_peered":"2023-11-06T03:12:45.031606+0000","last_unstale":"2023-11-06T03:12:45.418703+0000","last_undegraded":"2023-11-06T03:12:45.418703+0000","last_fullsized":"2023-11-06T03:12:45.418703+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864333+0000","last_change":"2023-11-06T03:12:45.418395+0000","last_active":"2023-11-06T03:12:46.864333+0000","last_peered":"2023-11-06T03:12:46.864333+0000","last_clean":"2023-11-06T03:12:46.864333+0000","last_became_active":"2023-11-06T03:12:45.417772+0000","last_became_peered":"2023-11-06T03:12:45.417772+0000","last_unstale":"2023-11-06T03:12:46.864333+0000","last_undegraded":"2023-11-06T03:12:46.864333+0000","last_fullsized":"2023-11-06T03:12:46.864333+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:46.522287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395028+0000","last_change":"2023-11-06T03:12:45.020168+0000","last_active":"2023-11-06T03:12:45.395028+0000","last_peered":"2023-11-06T03:12:45.395028+0000","last_clean":"2023-11-06T03:12:45.395028+0000","last_became_active":"2023-11-06T03:12:45.019891+0000","last_became_peered":"2023-11-06T03:12:45.019891+0000","last_unstale":"2023-11-06T03:12:45.395028+0000","last_undegraded":"2023-11-06T03:12:45.395028+0000","last_fullsized":"2023-11-06T03:12:45.395028+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:05.919298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863228+0000","last_change":"2023-11-06T03:12:45.425328+0000","last_active":"2023-11-06T03:12:46.863228+0000","last_peered":"2023-11-06T03:12:46.863228+0000","last_clean":"2023-11-06T03:12:46.863228+0000","last_became_active":"2023-11-06T03:12:45.425081+0000","last_became_peered":"2023-11-06T03:12:45.425081+0000","last_unstale":"2023-11-06T03:12:46.863228+0000","last_undegraded":"2023-11-06T03:12:46.863228+0000","last_fullsized":"2023-11-06T03:12:46.863228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:59.511071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.387119+0000","last_change":"2023-11-06T03:12:45.022982+0000","last_active":"2023-11-06T03:12:45.387119+0000","last_peered":"2023-11-06T03:12:45.387119+0000","last_clean":"2023-11-06T03:12:45.387119+0000","last_became_active":"2023-11-06T03:12:45.021582+0000","last_became_peered":"2023-11-06T03:12:45.021582+0000","last_unstale":"2023-11-06T03:12:45.387119+0000","last_undegraded":"2023-11-06T03:12:45.387119+0000","last_fullsized":"2023-11-06T03:12:45.387119+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:04:41.822909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.586169+0000","last_change":"2023-11-06T03:12:45.441344+0000","last_active":"2023-11-06T03:12:46.586169+0000","last_peered":"2023-11-06T03:12:46.586169+0000","last_clean":"2023-11-06T03:12:46.586169+0000","last_became_active":"2023-11-06T03:12:45.441042+0000","last_became_peered":"2023-11-06T03:12:45.441042+0000","last_unstale":"2023-11-06T03:12:46.586169+0000","last_undegraded":"2023-11-06T03:12:46.586169+0000","last_fullsized":"2023-11-06T03:12:46.586169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:45:54.459354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":19,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.447104+0000","last_change":"2023-11-06T03:12:45.036866+0000","last_active":"2023-11-06T03:12:45.447104+0000","last_peered":"2023-11-06T03:12:45.447104+0000","last_clean":"2023-11-06T03:12:45.447104+0000","last_became_active":"2023-11-06T03:12:45.036202+0000","last_became_peered":"2023-11-06T03:12:45.036202+0000","last_unstale":"2023-11-06T03:12:45.447104+0000","last_undegraded":"2023-11-06T03:12:45.447104+0000","last_fullsized":"2023-11-06T03:12:45.447104+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:04.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585385+0000","last_change":"2023-11-06T03:12:45.400081+0000","last_active":"2023-11-06T03:12:46.585385+0000","last_peered":"2023-11-06T03:12:46.585385+0000","last_clean":"2023-11-06T03:12:46.585385+0000","last_became_active":"2023-11-06T03:12:45.399581+0000","last_became_peered":"2023-11-06T03:12:45.399581+0000","last_unstale":"2023-11-06T03:12:46.585385+0000","last_undegraded":"2023-11-06T03:12:46.585385+0000","last_fullsized":"2023-11-06T03:12:46.585385+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:33:44.980111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.417327+0000","last_change":"2023-11-06T03:12:45.014894+0000","last_active":"2023-11-06T03:12:45.417327+0000","last_peered":"2023-11-06T03:12:45.417327+0000","last_clean":"2023-11-06T03:12:45.417327+0000","last_became_active":"2023-11-06T03:12:45.014171+0000","last_became_peered":"2023-11-06T03:12:45.014171+0000","last_unstale":"2023-11-06T03:12:45.417327+0000","last_undegraded":"2023-11-06T03:12:45.417327+0000","last_fullsized":"2023-11-06T03:12:45.417327+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:49:49.976386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.400451+0000","last_change":"2023-11-06T03:12:45.407200+0000","last_active":"2023-11-06T03:12:46.400451+0000","last_peered":"2023-11-06T03:12:46.400451+0000","last_clean":"2023-11-06T03:12:46.400451+0000","last_became_active":"2023-11-06T03:12:45.405084+0000","last_became_peered":"2023-11-06T03:12:45.405084+0000","last_unstale":"2023-11-06T03:12:46.400451+0000","last_undegraded":"2023-11-06T03:12:46.400451+0000","last_fullsized":"2023-11-06T03:12:46.400451+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:02.356000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.391302+0000","last_change":"2023-11-06T03:12:45.033836+0000","last_active":"2023-11-06T03:12:45.391302+0000","last_peered":"2023-11-06T03:12:45.391302+0000","last_clean":"2023-11-06T03:12:45.391302+0000","last_became_active":"2023-11-06T03:12:45.033383+0000","last_became_peered":"2023-11-06T03:12:45.033383+0000","last_unstale":"2023-11-06T03:12:45.391302+0000","last_undegraded":"2023-11-06T03:12:45.391302+0000","last_fullsized":"2023-11-06T03:12:45.391302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:37.240035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393761+0000","last_change":"2023-11-06T03:12:45.417343+0000","last_active":"2023-11-06T03:12:46.393761+0000","last_peered":"2023-11-06T03:12:46.393761+0000","last_clean":"2023-11-06T03:12:46.393761+0000","last_became_active":"2023-11-06T03:12:45.416913+0000","last_became_peered":"2023-11-06T03:12:45.416913+0000","last_unstale":"2023-11-06T03:12:46.393761+0000","last_undegraded":"2023-11-06T03:12:46.393761+0000","last_fullsized":"2023-11-06T03:12:46.393761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:32.568478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.390717+0000","last_change":"2023-11-06T03:12:45.032631+0000","last_active":"2023-11-06T03:12:45.390717+0000","last_peered":"2023-11-06T03:12:45.390717+0000","last_clean":"2023-11-06T03:12:45.390717+0000","last_became_active":"2023-11-06T03:12:45.032177+0000","last_became_peered":"2023-11-06T03:12:45.032177+0000","last_unstale":"2023-11-06T03:12:45.390717+0000","last_undegraded":"2023-11-06T03:12:45.390717+0000","last_fullsized":"2023-11-06T03:12:45.390717+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:09.929191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862517+0000","last_change":"2023-11-06T03:12:45.440426+0000","last_active":"2023-11-06T03:12:46.862517+0000","last_peered":"2023-11-06T03:12:46.862517+0000","last_clean":"2023-11-06T03:12:46.862517+0000","last_became_active":"2023-11-06T03:12:45.439467+0000","last_became_peered":"2023-11-06T03:12:45.439467+0000","last_unstale":"2023-11-06T03:12:46.862517+0000","last_undegraded":"2023-11-06T03:12:46.862517+0000","last_fullsized":"2023-11-06T03:12:46.862517+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:02.833708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.394676+0000","last_change":"2023-11-06T03:12:45.028022+0000","last_active":"2023-11-06T03:12:45.394676+0000","last_peered":"2023-11-06T03:12:45.394676+0000","last_clean":"2023-11-06T03:12:45.394676+0000","last_became_active":"2023-11-06T03:12:45.027325+0000","last_became_peered":"2023-11-06T03:12:45.027325+0000","last_unstale":"2023-11-06T03:12:45.394676+0000","last_undegraded":"2023-11-06T03:12:45.394676+0000","last_fullsized":"2023-11-06T03:12:45.394676+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862809+0000","last_change":"2023-11-06T03:12:45.398468+0000","last_active":"2023-11-06T03:12:46.862809+0000","last_peered":"2023-11-06T03:12:46.862809+0000","last_clean":"2023-11-06T03:12:46.862809+0000","last_became_active":"2023-11-06T03:12:45.394061+0000","last_became_peered":"2023-11-06T03:12:45.394061+0000","last_unstale":"2023-11-06T03:12:46.862809+0000","last_undegraded":"2023-11-06T03:12:46.862809+0000","last_fullsized":"2023-11-06T03:12:46.862809+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:22.305311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.397819+0000","last_change":"2023-11-06T03:12:45.028959+0000","last_active":"2023-11-06T03:12:45.397819+0000","last_peered":"2023-11-06T03:12:45.397819+0000","last_clean":"2023-11-06T03:12:45.397819+0000","last_became_active":"2023-11-06T03:12:45.028406+0000","last_became_peered":"2023-11-06T03:12:45.028406+0000","last_unstale":"2023-11-06T03:12:45.397819+0000","last_undegraded":"2023-11-06T03:12:45.397819+0000","last_fullsized":"2023-11-06T03:12:45.397819+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:58.267686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863106+0000","last_change":"2023-11-06T03:12:45.402029+0000","last_active":"2023-11-06T03:12:46.863106+0000","last_peered":"2023-11-06T03:12:46.863106+0000","last_clean":"2023-11-06T03:12:46.863106+0000","last_became_active":"2023-11-06T03:12:45.401642+0000","last_became_peered":"2023-11-06T03:12:45.401642+0000","last_unstale":"2023-11-06T03:12:46.863106+0000","last_undegraded":"2023-11-06T03:12:46.863106+0000","last_fullsized":"2023-11-06T03:12:46.863106+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:47:15.085556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.393101+0000","last_change":"2023-11-06T03:12:45.018899+0000","last_active":"2023-11-06T03:12:45.393101+0000","last_peered":"2023-11-06T03:12:45.393101+0000","last_clean":"2023-11-06T03:12:45.393101+0000","last_became_active":"2023-11-06T03:12:45.018234+0000","last_became_peered":"2023-11-06T03:12:45.018234+0000","last_unstale":"2023-11-06T03:12:45.393101+0000","last_undegraded":"2023-11-06T03:12:45.393101+0000","last_fullsized":"2023-11-06T03:12:45.393101+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:52:49.955044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.864930+0000","last_change":"2023-11-06T03:12:45.443717+0000","last_active":"2023-11-06T03:12:46.864930+0000","last_peered":"2023-11-06T03:12:46.864930+0000","last_clean":"2023-11-06T03:12:46.864930+0000","last_became_active":"2023-11-06T03:12:45.443151+0000","last_became_peered":"2023-11-06T03:12:45.443151+0000","last_unstale":"2023-11-06T03:12:46.864930+0000","last_undegraded":"2023-11-06T03:12:46.864930+0000","last_fullsized":"2023-11-06T03:12:46.864930+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:50:02.833708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.389196+0000","last_change":"2023-11-06T03:12:45.033145+0000","last_active":"2023-11-06T03:12:45.389196+0000","last_peered":"2023-11-06T03:12:45.389196+0000","last_clean":"2023-11-06T03:12:45.389196+0000","last_became_active":"2023-11-06T03:12:45.032354+0000","last_became_peered":"2023-11-06T03:12:45.032354+0000","last_unstale":"2023-11-06T03:12:45.389196+0000","last_undegraded":"2023-11-06T03:12:45.389196+0000","last_fullsized":"2023-11-06T03:12:45.389196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:21.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863232+0000","last_change":"2023-11-06T03:12:45.441677+0000","last_active":"2023-11-06T03:12:46.863232+0000","last_peered":"2023-11-06T03:12:46.863232+0000","last_clean":"2023-11-06T03:12:46.863232+0000","last_became_active":"2023-11-06T03:12:45.441016+0000","last_became_peered":"2023-11-06T03:12:45.441016+0000","last_unstale":"2023-11-06T03:12:46.863232+0000","last_undegraded":"2023-11-06T03:12:46.863232+0000","last_fullsized":"2023-11-06T03:12:46.863232+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:38.826383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.386193+0000","last_change":"2023-11-06T03:12:45.024618+0000","last_active":"2023-11-06T03:12:45.386193+0000","last_peered":"2023-11-06T03:12:45.386193+0000","last_clean":"2023-11-06T03:12:45.386193+0000","last_became_active":"2023-11-06T03:12:45.019472+0000","last_became_peered":"2023-11-06T03:12:45.019472+0000","last_unstale":"2023-11-06T03:12:45.386193+0000","last_undegraded":"2023-11-06T03:12:45.386193+0000","last_fullsized":"2023-11-06T03:12:45.386193+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:38:56.796109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862121+0000","last_change":"2023-11-06T03:12:45.439246+0000","last_active":"2023-11-06T03:12:46.862121+0000","last_peered":"2023-11-06T03:12:46.862121+0000","last_clean":"2023-11-06T03:12:46.862121+0000","last_became_active":"2023-11-06T03:12:45.438665+0000","last_became_peered":"2023-11-06T03:12:45.438665+0000","last_unstale":"2023-11-06T03:12:46.862121+0000","last_undegraded":"2023-11-06T03:12:46.862121+0000","last_fullsized":"2023-11-06T03:12:46.862121+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.490571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.410336+0000","last_change":"2023-11-06T03:12:45.025025+0000","last_active":"2023-11-06T03:12:45.410336+0000","last_peered":"2023-11-06T03:12:45.410336+0000","last_clean":"2023-11-06T03:12:45.410336+0000","last_became_active":"2023-11-06T03:12:45.024701+0000","last_became_peered":"2023-11-06T03:12:45.024701+0000","last_unstale":"2023-11-06T03:12:45.410336+0000","last_undegraded":"2023-11-06T03:12:45.410336+0000","last_fullsized":"2023-11-06T03:12:45.410336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:34:14.282088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":241,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.400826+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:12:45.400826+0000","last_peered":"2023-11-06T03:12:45.400826+0000","last_clean":"2023-11-06T03:12:45.400826+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:12:45.400826+0000","last_undegraded":"2023-11-06T03:12:45.400826+0000","last_fullsized":"2023-11-06T03:12:45.400826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.401972+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:12:45.401972+0000","last_peered":"2023-11-06T03:12:45.401972+0000","last_clean":"2023-11-06T03:12:45.401972+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:12:45.401972+0000","last_undegraded":"2023-11-06T03:12:45.401972+0000","last_fullsized":"2023-11-06T03:12:45.401972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.863858+0000","last_change":"2023-11-06T03:12:45.412840+0000","last_active":"2023-11-06T03:12:46.863858+0000","last_peered":"2023-11-06T03:12:46.863858+0000","last_clean":"2023-11-06T03:12:46.863858+0000","last_became_active":"2023-11-06T03:12:45.410939+0000","last_became_peered":"2023-11-06T03:12:45.410939+0000","last_unstale":"2023-11-06T03:12:46.863858+0000","last_undegraded":"2023-11-06T03:12:46.863858+0000","last_fullsized":"2023-11-06T03:12:46.863858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:52:20.478173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.434333+0000","last_change":"2023-11-06T03:12:45.023312+0000","last_active":"2023-11-06T03:12:45.434333+0000","last_peered":"2023-11-06T03:12:45.434333+0000","last_clean":"2023-11-06T03:12:45.434333+0000","last_became_active":"2023-11-06T03:12:45.022806+0000","last_became_peered":"2023-11-06T03:12:45.022806+0000","last_unstale":"2023-11-06T03:12:45.434333+0000","last_undegraded":"2023-11-06T03:12:45.434333+0000","last_fullsized":"2023-11-06T03:12:45.434333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:54.093163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.400557+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:12:45.400557+0000","last_peered":"2023-11-06T03:12:45.400557+0000","last_clean":"2023-11-06T03:12:45.400557+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:12:45.400557+0000","last_undegraded":"2023-11-06T03:12:45.400557+0000","last_fullsized":"2023-11-06T03:12:45.400557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861482+0000","last_change":"2023-11-06T03:12:45.429678+0000","last_active":"2023-11-06T03:12:46.861482+0000","last_peered":"2023-11-06T03:12:46.861482+0000","last_clean":"2023-11-06T03:12:46.861482+0000","last_became_active":"2023-11-06T03:12:45.429431+0000","last_became_peered":"2023-11-06T03:12:45.429431+0000","last_unstale":"2023-11-06T03:12:46.861482+0000","last_undegraded":"2023-11-06T03:12:46.861482+0000","last_fullsized":"2023-11-06T03:12:46.861482+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:08.454890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.399320+0000","last_change":"2023-11-06T03:12:45.026227+0000","last_active":"2023-11-06T03:12:45.399320+0000","last_peered":"2023-11-06T03:12:45.399320+0000","last_clean":"2023-11-06T03:12:45.399320+0000","last_became_active":"2023-11-06T03:12:45.024563+0000","last_became_peered":"2023-11-06T03:12:45.024563+0000","last_unstale":"2023-11-06T03:12:45.399320+0000","last_undegraded":"2023-11-06T03:12:45.399320+0000","last_fullsized":"2023-11-06T03:12:45.399320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:28.130604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.387210+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:12:45.387210+0000","last_peered":"2023-11-06T03:12:45.387210+0000","last_clean":"2023-11-06T03:12:45.387210+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:12:45.387210+0000","last_undegraded":"2023-11-06T03:12:45.387210+0000","last_fullsized":"2023-11-06T03:12:45.387210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.861157+0000","last_change":"2023-11-06T03:12:45.403831+0000","last_active":"2023-11-06T03:12:46.861157+0000","last_peered":"2023-11-06T03:12:46.861157+0000","last_clean":"2023-11-06T03:12:46.861157+0000","last_became_active":"2023-11-06T03:12:45.402589+0000","last_became_peered":"2023-11-06T03:12:45.402589+0000","last_unstale":"2023-11-06T03:12:46.861157+0000","last_undegraded":"2023-11-06T03:12:46.861157+0000","last_fullsized":"2023-11-06T03:12:46.861157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:18:10.940379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.430547+0000","last_change":"2023-11-06T03:12:45.024881+0000","last_active":"2023-11-06T03:12:45.430547+0000","last_peered":"2023-11-06T03:12:45.430547+0000","last_clean":"2023-11-06T03:12:45.430547+0000","last_became_active":"2023-11-06T03:12:45.024362+0000","last_became_peered":"2023-11-06T03:12:45.024362+0000","last_unstale":"2023-11-06T03:12:45.430547+0000","last_undegraded":"2023-11-06T03:12:45.430547+0000","last_fullsized":"2023-11-06T03:12:45.430547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:46:22.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.401566+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:12:45.401566+0000","last_peered":"2023-11-06T03:12:45.401566+0000","last_clean":"2023-11-06T03:12:45.401566+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:12:45.401566+0000","last_undegraded":"2023-11-06T03:12:45.401566+0000","last_fullsized":"2023-11-06T03:12:45.401566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.393609+0000","last_change":"2023-11-06T03:12:45.425084+0000","last_active":"2023-11-06T03:12:46.393609+0000","last_peered":"2023-11-06T03:12:46.393609+0000","last_clean":"2023-11-06T03:12:46.393609+0000","last_became_active":"2023-11-06T03:12:45.424780+0000","last_became_peered":"2023-11-06T03:12:45.424780+0000","last_unstale":"2023-11-06T03:12:46.393609+0000","last_undegraded":"2023-11-06T03:12:46.393609+0000","last_fullsized":"2023-11-06T03:12:46.393609+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:32:51.357395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.388947+0000","last_change":"2023-11-06T03:12:45.025978+0000","last_active":"2023-11-06T03:12:45.388947+0000","last_peered":"2023-11-06T03:12:45.388947+0000","last_clean":"2023-11-06T03:12:45.388947+0000","last_became_active":"2023-11-06T03:12:45.025242+0000","last_became_peered":"2023-11-06T03:12:45.025242+0000","last_unstale":"2023-11-06T03:12:45.388947+0000","last_undegraded":"2023-11-06T03:12:45.388947+0000","last_fullsized":"2023-11-06T03:12:45.388947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:32:03.663166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.394881+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:12:45.394881+0000","last_peered":"2023-11-06T03:12:45.394881+0000","last_clean":"2023-11-06T03:12:45.394881+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:12:45.394881+0000","last_undegraded":"2023-11-06T03:12:45.394881+0000","last_fullsized":"2023-11-06T03:12:45.394881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.862941+0000","last_change":"2023-11-06T03:12:45.398634+0000","last_active":"2023-11-06T03:12:46.862941+0000","last_peered":"2023-11-06T03:12:46.862941+0000","last_clean":"2023-11-06T03:12:46.862941+0000","last_became_active":"2023-11-06T03:12:45.394089+0000","last_became_peered":"2023-11-06T03:12:45.394089+0000","last_unstale":"2023-11-06T03:12:46.862941+0000","last_undegraded":"2023-11-06T03:12:46.862941+0000","last_fullsized":"2023-11-06T03:12:46.862941+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:33:36.000007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.385984+0000","last_change":"2023-11-06T03:12:45.025515+0000","last_active":"2023-11-06T03:12:45.385984+0000","last_peered":"2023-11-06T03:12:45.385984+0000","last_clean":"2023-11-06T03:12:45.385984+0000","last_became_active":"2023-11-06T03:12:45.025188+0000","last_became_peered":"2023-11-06T03:12:45.025188+0000","last_unstale":"2023-11-06T03:12:45.385984+0000","last_undegraded":"2023-11-06T03:12:45.385984+0000","last_fullsized":"2023-11-06T03:12:45.385984+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:59.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.398020+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:12:45.398020+0000","last_peered":"2023-11-06T03:12:45.398020+0000","last_clean":"2023-11-06T03:12:45.398020+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:12:45.398020+0000","last_undegraded":"2023-11-06T03:12:45.398020+0000","last_fullsized":"2023-11-06T03:12:45.398020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585620+0000","last_change":"2023-11-06T03:12:45.407853+0000","last_active":"2023-11-06T03:12:46.585620+0000","last_peered":"2023-11-06T03:12:46.585620+0000","last_clean":"2023-11-06T03:12:46.585620+0000","last_became_active":"2023-11-06T03:12:45.407590+0000","last_became_peered":"2023-11-06T03:12:45.407590+0000","last_unstale":"2023-11-06T03:12:46.585620+0000","last_undegraded":"2023-11-06T03:12:46.585620+0000","last_fullsized":"2023-11-06T03:12:46.585620+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:33.279183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.407267+0000","last_change":"2023-11-06T03:12:45.018966+0000","last_active":"2023-11-06T03:12:45.407267+0000","last_peered":"2023-11-06T03:12:45.407267+0000","last_clean":"2023-11-06T03:12:45.407267+0000","last_became_active":"2023-11-06T03:12:45.018670+0000","last_became_peered":"2023-11-06T03:12:45.018670+0000","last_unstale":"2023-11-06T03:12:45.407267+0000","last_undegraded":"2023-11-06T03:12:45.407267+0000","last_fullsized":"2023-11-06T03:12:45.407267+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:48.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.395242+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:12:45.395242+0000","last_peered":"2023-11-06T03:12:45.395242+0000","last_clean":"2023-11-06T03:12:45.395242+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:12:45.395242+0000","last_undegraded":"2023-11-06T03:12:45.395242+0000","last_fullsized":"2023-11-06T03:12:45.395242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-11-06T03:12:46.585779+0000","last_change":"2023-11-06T03:12:45.417398+0000","last_active":"2023-11-06T03:12:46.585779+0000","last_peered":"2023-11-06T03:12:46.585779+0000","last_clean":"2023-11-06T03:12:46.585779+0000","last_became_active":"2023-11-06T03:12:45.417063+0000","last_became_peered":"2023-11-06T03:12:45.417063+0000","last_unstale":"2023-11-06T03:12:46.585779+0000","last_undegraded":"2023-11-06T03:12:46.585779+0000","last_fullsized":"2023-11-06T03:12:46.585779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:44.977800+0000","last_clean_scrub_stamp":"2023-11-06T03:12:44.977800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:44:20.006738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.392934+0000","last_change":"2023-11-06T03:12:45.024770+0000","last_active":"2023-11-06T03:12:45.392934+0000","last_peered":"2023-11-06T03:12:45.392934+0000","last_clean":"2023-11-06T03:12:45.392934+0000","last_became_active":"2023-11-06T03:12:45.024481+0000","last_became_peered":"2023-11-06T03:12:45.024481+0000","last_unstale":"2023-11-06T03:12:45.392934+0000","last_undegraded":"2023-11-06T03:12:45.392934+0000","last_fullsized":"2023-11-06T03:12:45.392934+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:12:43.966612+0000","last_clean_scrub_stamp":"2023-11-06T03:12:43.966612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:53:22.948670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-11-06T03:12:45.389894+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:12:45.389894+0000","last_peered":"2023-11-06T03:12:45.389894+0000","last_clean":"2023-11-06T03:12:45.389894+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:12:45.389894+0000","last_undegraded":"2023-11-06T03:12:45.389894+0000","last_fullsized":"2023-11-06T03:12:45.389894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87056,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284784,"statfs":{"total":96636764160,"available":96547618816,"internally_reserved":0,"allocated":1974272,"data_stored":1222040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.117,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83299999999999996,"15min":0.86599999999999999},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.1659999999999999,"5min":1.367,"15min":1.397},"last":0.93000000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81699999999999995,"15min":0.84899999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2490000000000001,"5min":1.254,"15min":1.444},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.218,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.92900000000000005,"15min":0.89100000000000001},"min":{"1min":0.70699999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1399999999999999,"5min":1.4330000000000001,"15min":1.6080000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.875,"15min":0.93300000000000005},"min":{"1min":0.51800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0820000000000001,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":1.0229999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.66600000000000004,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.3520000000000001,"5min":2.512,"15min":2.512},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.1299999999999999,"15min":1.0580000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.1799999999999999,"5min":10.596,"15min":10.596},"last":0.874}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.94799999999999995,"15min":0.96999999999999997},"min":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.2709999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98899999999999999,"15min":1.0049999999999999},"min":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.54,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.995}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.056,"15min":1.0469999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.85499999999999998,"5min":0.68600000000000005,"15min":0.55200000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.081}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.171,"15min":1.1579999999999999},"min":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.8149999999999999,"5min":2.548,"15min":2.548},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.121,"15min":1.1140000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.7370000000000001,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":0.82699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80724,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":79360,"kb_avail":94291116,"statfs":{"total":96636764160,"available":96554102784,"internally_reserved":0,"allocated":1388544,"data_stored":632661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81264640},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.98199999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.1870000000000001,"5min":16.073,"15min":16.073},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.288,"15min":0.97799999999999998},"min":{"1min":0.53100000000000003,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":40.905000000000001,"15min":40.905000000000001},"last":1.038}]},{"osd":1,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.056,"15min":0.90200000000000002},"min":{"1min":0.55800000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2370000000000001,"5min":16.012,"15min":16.012},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.90800000000000003,"15min":0.88200000000000001},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.409,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.92600000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.137,"15min":0.91000000000000003},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.111,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.67500000000000004,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.2709999999999999,"5min":2.843,"15min":2.843},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.647,"15min":1.1899999999999999},"min":{"1min":0.67300000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.306,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":1.377,"15min":1.0529999999999999},"min":{"1min":0.622,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.1830000000000001,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.3440000000000001,"15min":1.1870000000000001},"min":{"1min":0.69999999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2290000000000001,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.4219999999999999,"15min":1.2450000000000001},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.593,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.139}]},{"osd":5,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.427,"15min":1.1619999999999999},"min":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.1990000000000001,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.159,"15min":1.1180000000000001},"min":{"1min":0.91200000000000003,"5min":0.80500000000000005,"15min":0.71499999999999997},"max":{"1min":1.3129999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.0640000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:12:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.4179999999999999,"15min":1.2070000000000001},"min":{"1min":0.749,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7629999999999999,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.423,"15min":1.2170000000000001},"min":{"1min":0.86099999999999999,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.3759999999999999,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.97199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313212,"statfs":{"total":96636764160,"available":96576729088,"internally_reserved":0,"allocated":1372160,"data_stored":631512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.7849999999999999,"15min":1.1339999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.98499999999999999,"5min":125.256,"15min":125.256},"last":0.996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.833,"15min":1.165},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.373,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86599999999999999,"15min":0.77100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2010000000000001,"5min":2.339,"15min":2.339},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86299999999999999,"15min":0.84299999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.129,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.83999999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1120000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.1080000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89900000000000002,"15min":0.86199999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.3129999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.077},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96899999999999997,"15min":0.94399999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.256,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.60099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89500000000000002,"15min":0.91400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.105,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.004,"15min":0.97399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.4650000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.1459999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.02,"15min":1.0029999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.1850000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.05,"15min":1.02},"min":{"1min":0.432,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.234,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.077,"15min":1.016},"min":{"1min":0.442,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.4370000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.464,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.044}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90572,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281268,"statfs":{"total":96636764160,"available":96544018432,"internally_reserved":0,"allocated":1380352,"data_stored":631604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":1.262,"15min":0.97099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.372},"max":{"1min":1.3939999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.3200000000000001,"15min":0.96499999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.0830000000000002,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.68600000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79300000000000004,"15min":0.76400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.86499999999999999,"15min":0.82199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.80900000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.872,"15min":0.997},"min":{"1min":0.57799999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.452,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.88800000000000001,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.76900000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.198,"5min":3.198,"15min":3.198},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.94999999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.0680000000000001,"15min":1.034},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57799999999999996},"max":{"1min":10.192,"5min":10.192,"15min":10.192},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":5.3310000000000004,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.84299999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.95699999999999996,"15min":1.002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0609999999999999,"15min":1.125},"min":{"1min":0.65300000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.506,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.089}]},{"osd":7,"last update":"Mon Nov 6 03:12:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.0780000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1120000000000001,"15min":1.175},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.1830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76676,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295164,"statfs":{"total":96636764160,"available":96558247936,"internally_reserved":0,"allocated":1372160,"data_stored":631075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.61199999999999999},"min":{"1min":0.32300000000000001,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.96799999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.85399999999999998,"5min":1.149,"15min":1.524},"last":0.84199999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.625},"min":{"1min":0.434,"5min":0.224,"15min":0.224},"max":{"1min":0.95799999999999996,"5min":1.371,"15min":1.371},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.1679999999999999,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.68500000000000005},"min":{"1min":0.48899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.206,"5min":1.532,"15min":1.532},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.052,"5min":1.284,"15min":1.284},"last":1.1339999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.872,"15min":0.89700000000000002},"min":{"1min":0.505,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":1.2609999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.048},{"interface":"front","average":{"1min":0.997,"5min":0.92700000000000005,"15min":0.94499999999999995},"min":{"1min":0.61299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4610000000000001,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.2210000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.97599999999999998,"15min":0.98299999999999998},"min":{"1min":0.73599999999999999,"5min":0.496,"15min":0.46899999999999997},"max":{"1min":1.319,"5min":1.536,"15min":1.6639999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.413,"5min":1.5049999999999999,"15min":1.698},"last":1.321}]},{"osd":6,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.995,"15min":1.0049999999999999},"min":{"1min":0.71399999999999997,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.51,"5min":1.649,"15min":1.8620000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96799999999999997,"15min":0.98999999999999999},"min":{"1min":0.67200000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3660000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.175}]},{"osd":7,"last update":"Mon Nov 6 03:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.63400000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3779999999999999,"5min":1.835,"15min":1.835},"last":1.26},{"interface":"front","average":{"1min":1.135,"5min":1.0529999999999999,"15min":1.0609999999999999},"min":{"1min":0.77300000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5629999999999999,"5min":1.734,"15min":1.8979999999999999},"last":1.3620000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68424,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303416,"statfs":{"total":96636764160,"available":96566697984,"internally_reserved":0,"allocated":1376256,"data_stored":631546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55500000000000005,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.90000000000000002,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.27600000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.85099999999999998,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.29299999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.28499999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.997,"5min":1.1160000000000001,"15min":1.121},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.67100000000000004},"min":{"1min":0.33300000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.294,"15min":1.294},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.72799999999999998},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.1100000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.877},"min":{"1min":0.61699999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.1859999999999999,"5min":1.526,"15min":1.526},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.92500000000000004},"min":{"1min":0.60099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.121,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.84299999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.93100000000000005,"15min":0.95199999999999996},"min":{"1min":0.61299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.274,"5min":1.486,"15min":1.5780000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.98199999999999998,"15min":0.98399999999999999},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.1299999999999999,"5min":1.615,"15min":1.615},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.93300000000000005,"15min":0.94399999999999995},"min":{"1min":0.71999999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.238,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874}]},{"osd":7,"last update":"Mon Nov 6 03:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.98499999999999999},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.6859999999999999,"15min":4.2809999999999997},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91000000000000003,"15min":0.94699999999999995},"min":{"1min":0.65000000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.27,"5min":1.27,"15min":3.4740000000000002},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46344,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325496,"statfs":{"total":96636764160,"available":96589307904,"internally_reserved":0,"allocated":1376256,"data_stored":631546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.313,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.1240000000000001,"5min":1.843,"15min":1.843},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.89800000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.746}]},{"osd":2,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.71299999999999997},"min":{"1min":0.27900000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.86099999999999999,"5min":1.169,"15min":30.861000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63400000000000001,"15min":0.66900000000000004},"min":{"1min":0.35299999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1419999999999999,"5min":1.1819999999999999,"15min":1.7330000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.32600000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.71099999999999997},"min":{"1min":0.33700000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.151,"5min":1.151,"15min":2.3079999999999998},"last":0.67600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86099999999999999,"15min":0.86899999999999999},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.276,"5min":2.0870000000000002,"15min":2.258},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.58999999999999997,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2030000000000001,"5min":1.736,"15min":14.753},"last":0.77700000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":1.1160000000000001},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.47299999999999998},"max":{"1min":1.115,"5min":1.528,"15min":43.103999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96299999999999997,"15min":1.1459999999999999},"min":{"1min":0.66600000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.2909999999999999,"5min":1.4279999999999999,"15min":44.753999999999998},"last":1.046}]},{"osd":6,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91100000000000003,"15min":1.0529999999999999},"min":{"1min":0.54500000000000004,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.1639999999999999,"5min":1.6930000000000001,"15min":23.971},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93899999999999995,"15min":1.0800000000000001},"min":{"1min":0.68700000000000006,"5min":0.50700000000000001,"15min":0.503},"max":{"1min":1.444,"5min":1.617,"15min":14.813000000000001},"last":1.032}]},{"osd":7,"last update":"Mon Nov 6 03:12:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.996,"15min":1.052},"min":{"1min":0.67100000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":1.135},{"interface":"front","average":{"1min":1.008,"5min":1.002,"15min":1.0369999999999999},"min":{"1min":0.71899999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.4470000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":1.002}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91220,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280620,"statfs":{"total":96636764160,"available":96543354880,"internally_reserved":0,"allocated":1978368,"data_stored":1222515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.0569999999999999,"5min":1.145,"15min":2.6989999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.35999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.24199999999999999,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.40999999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.46600000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.38900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.91800000000000004,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.73699999999999999,"15min":0.72799999999999998},"min":{"1min":0.41099999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95899999999999996,"5min":1.2909999999999999,"15min":2.7999999999999998},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.3560000000000001,"5min":1.446,"15min":4.641},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.84899999999999998,"15min":0.91300000000000003},"min":{"1min":0.55600000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.131,"5min":1.286,"15min":2.028},"last":0.63800000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.624,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.3200000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.53200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.27,"5min":1.405,"15min":4.7430000000000003},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89800000000000002,"15min":0.90500000000000003},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2549999999999999,"15min":2.931},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.004,"5min":0.99199999999999999,"15min":1.01},"min":{"1min":0.70299999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.254,"5min":1.802,"15min":2.7789999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.96699999999999997,"15min":1.0049999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.306,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2210000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:12:53.272 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:12:53.273 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:12:53.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.425+0000 7ff053983640 1 Processor -- start 2023-11-06T03:12:53.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.425+0000 7ff053983640 1 -- start start 2023-11-06T03:12:53.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff053983640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff04c0b2b10 0x7ff04c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff053983640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff04c0cd730 0x7ff04c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff053983640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c0cc2b0 con 0x7ff04c15ac80 2023-11-06T03:12:53.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff053983640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0b01a0 con 0x7ff04c0cd730 2023-11-06T03:12:53.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff053983640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0cfb10 con 0x7ff04c0b2b10 2023-11-06T03:12:53.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff04c0cd730 0x7ff04c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff04c0cd730 0x7ff04c0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53806/0 (socket says 172.21.15.157:53806) 2023-11-06T03:12:53.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 -- 172.21.15.157:0/264154977 learned_addr learned my addr 172.21.15.157:0/264154977 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:53.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff050f1e640 1 --2- 172.21.15.157:0/264154977 >> v2:172.21.15.195:3300/0 conn(0x7ff04c0b2b10 0x7ff04c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 -- 172.21.15.157:0/264154977 >> v2:172.21.15.195:3300/0 conn(0x7ff04c0b2b10 msgr2=0x7ff04c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:53.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 --2- 172.21.15.157:0/264154977 >> v2:172.21.15.195:3300/0 conn(0x7ff04c0b2b10 0x7ff04c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 -- 172.21.15.157:0/264154977 >> v1:172.21.15.195:6789/0 conn(0x7ff04c15ac80 legacy=0x7ff04c0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 -- 172.21.15.157:0/264154977 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0cfdf0 con 0x7ff04c0cd730 2023-11-06T03:12:53.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.429+0000 7ff051f20640 1 --2- 172.21.15.157:0/264154977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c0cea90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0b5e40 tx=0x7ff04401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920bf969d819b7fe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff043fff640 1 -- 172.21.15.157:0/264154977 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0440a5890 con 0x7ff04c0cd730 2023-11-06T03:12:53.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff043fff640 1 -- 172.21.15.157:0/264154977 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0440a59f0 con 0x7ff04c0cd730 2023-11-06T03:12:53.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff043fff640 1 -- 172.21.15.157:0/264154977 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0440a5d80 con 0x7ff04c0cd730 2023-11-06T03:12:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- 172.21.15.157:0/264154977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 msgr2=0x7ff04c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 --2- 172.21.15.157:0/264154977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c0cea90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0b5e40 tx=0x7ff04401c9d0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- 172.21.15.157:0/264154977 shutdown_connections 2023-11-06T03:12:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 --2- 172.21.15.157:0/264154977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c0cea90 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 --2- 172.21.15.157:0/264154977 >> v2:172.21.15.195:3300/0 conn(0x7ff04c0b2b10 0x7ff04c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- 172.21.15.157:0/264154977 >> 172.21.15.157:0/264154977 conn(0x7ff04c06d560 msgr2=0x7ff04c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- 172.21.15.157:0/264154977 shutdown_connections 2023-11-06T03:12:53.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- 172.21.15.157:0/264154977 wait complete. 2023-11-06T03:12:53.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 Processor -- start 2023-11-06T03:12:53.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.433+0000 7ff053983640 1 -- start start 2023-11-06T03:12:53.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04c0b2b10 0x7ff04c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 0x7ff04c150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0c4580 con 0x7ff04c0cd730 2023-11-06T03:12:53.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1598d0 con 0x7ff04c0b2b10 2023-11-06T03:12:53.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c0b2260 con 0x7ff04c15ac80 2023-11-06T03:12:53.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53822/0 (socket says 172.21.15.157:53822) 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 -- 172.21.15.157:0/37135848 learned_addr learned my addr 172.21.15.157:0/37135848 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff05171f640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04c0b2b10 0x7ff04c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff051f20640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 0x7ff04c150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 -- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 msgr2=0x7ff04c150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 0x7ff04c150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 -- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04c0b2b10 msgr2=0x7ff04c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04c0b2b10 0x7ff04c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 -- 172.21.15.157:0/37135848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009b30 con 0x7ff04c0cd730 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff051f20640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 0x7ff04c150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff050f1e640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00d800 tx=0x7ff03c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00b660 con 0x7ff04c0cd730 2023-11-06T03:12:53.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00b7c0 con 0x7ff04c0cd730 2023-11-06T03:12:53.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c0b2540 con 0x7ff04c0cd730 2023-11-06T03:12:53.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00bb30 con 0x7ff04c0cd730 2023-11-06T03:12:53.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c0c7dd0 con 0x7ff04c0cd730 2023-11-06T03:12:53.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0cb070 con 0x7ff04c0cd730 2023-11-06T03:12:53.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff03c00bc90 con 0x7ff04c0cd730 2023-11-06T03:12:53.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 0x7ff030067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff03c0838d0 con 0x7ff04c0cd730 2023-11-06T03:12:53.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.437+0000 7ff05171f640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 0x7ff030067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.441+0000 7ff05171f640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 0x7ff030067f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff048000f70 tx=0x7ff048009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.441+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff03c051080 con 0x7ff04c0cd730 2023-11-06T03:12:53.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.637+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff04c072750 con 0x7ff04c0cd730 2023-11-06T03:12:53.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff041ffb640 1 -- 172.21.15.157:0/37135848 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff03c011340 con 0x7ff04c0cd730 2023-11-06T03:12:53.646 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:12:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 msgr2=0x7ff030067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 0x7ff030067f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff048000f70 tx=0x7ff048009040 comp rx=0 tx=0).stop 2023-11-06T03:12:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 msgr2=0x7ff04c150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00d800 tx=0x7ff03c00dcc0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 shutdown_connections 2023-11-06T03:12:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff030065a60 0x7ff030067f20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff04c15ac80 0x7ff04c150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff04c0cd730 0x7ff04c150120 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 --2- 172.21.15.157:0/37135848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff04c0b2b10 0x7ff04c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 >> 172.21.15.157:0/37135848 conn(0x7ff04c06d560 msgr2=0x7ff04c153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 shutdown_connections 2023-11-06T03:12:53.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.645+0000 7ff053983640 1 -- 172.21.15.157:0/37135848 wait complete. 2023-11-06T03:12:53.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:12:53.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 Processor -- start 2023-11-06T03:12:53.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 -- start start 2023-11-06T03:12:53.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f84900ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900cc2b0 con 0x7f84900cd730 2023-11-06T03:12:53.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900b01a0 con 0x7f849015ac80 2023-11-06T03:12:53.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8497db1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900cfb10 con 0x7f84900b2b10 2023-11-06T03:12:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849634e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84900cd730 0x7f84900cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32846/0 (socket says 172.21.15.157:32846) 2023-11-06T03:12:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849634e640 1 -- 172.21.15.157:0/423634823 learned_addr learned my addr 172.21.15.157:0/423634823 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f84900ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656846414 0 0) 0x7f84900cc2b0 con 0x7f84900cd730 2023-11-06T03:12:53.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f847c003530 con 0x7f84900cd730 2023-11-06T03:12:53.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326254824 0 0) 0x7f847c003530 con 0x7f84900cd730 2023-11-06T03:12:53.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 msgr2=0x7f84900ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:53.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f84900ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 >> v1:172.21.15.195:6789/0 conn(0x7f849015ac80 legacy=0x7f84900b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900cfdf0 con 0x7f84900cd730 2023-11-06T03:12:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8488001a00 con 0x7f84900cd730 2023-11-06T03:12:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 >> v1:172.21.15.157:6789/0 conn(0x7f84900cd730 legacy=0x7f84900cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f847c0039b0 0x7f847c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f847c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f847c004d10 0x7f847c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900cfb10 con 0x7f847c004d10 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c00ba40 con 0x7f847c0039b0 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c00bc00 con 0x7f84900b2b10 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849634e640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f847c004d10 0x7f847c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f847c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f8495b4d640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f847c0039b0 0x7f847c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 -- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f847c0039b0 msgr2=0x7f847c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:53.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f847c0039b0 0x7f847c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 -- 172.21.15.157:0/423634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f847c004d10 msgr2=0x7f847c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:53.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f847c004d10 0x7f847c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.785+0000 7f849534c640 1 -- 172.21.15.157:0/423634823 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c003530 con 0x7f84900b2b10 2023-11-06T03:12:53.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f849534c640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f847c0047d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ab0 tx=0x7f848001ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7b70fdfc81f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800a5660 con 0x7f84900b2b10 2023-11-06T03:12:53.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84800a57c0 con 0x7f84900b2b10 2023-11-06T03:12:53.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8494b4b640 1 -- 172.21.15.157:0/423634823 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800a5b70 con 0x7f84900b2b10 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 msgr2=0x7f847c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 --2- 172.21.15.157:0/423634823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900b2b10 0x7f847c0047d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ab0 tx=0x7f848001ca70 comp rx=0 tx=0).stop 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f849634e640 1 -- 172.21.15.157:0/423634823 reap_dead start 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- 172.21.15.157:0/423634823 shutdown_connections 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- 172.21.15.157:0/423634823 >> 172.21.15.157:0/423634823 conn(0x7f849006d560 msgr2=0x7f849006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- 172.21.15.157:0/423634823 shutdown_connections 2023-11-06T03:12:53.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- 172.21.15.157:0/423634823 wait complete. 2023-11-06T03:12:53.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 Processor -- start 2023-11-06T03:12:53.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.789+0000 7f8497db1640 1 -- start start 2023-11-06T03:12:53.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84900b2b10 0x7f84901f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 0x7f849015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901f3340 con 0x7f84900c9010 2023-11-06T03:12:53.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901f04a0 con 0x7f84900b2b10 2023-11-06T03:12:53.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900c8230 con 0x7f84900c7a50 2023-11-06T03:12:53.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53840/0 (socket says 172.21.15.157:53840) 2023-11-06T03:12:53.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 -- 172.21.15.157:0/3418517042 learned_addr learned my addr 172.21.15.157:0/3418517042 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:53.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8495b4d640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84900b2b10 0x7f84901f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849634e640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 0x7f849015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 -- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 msgr2=0x7f849015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 0x7f849015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 -- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84900b2b10 msgr2=0x7f84901f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84900b2b10 0x7f84901f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 -- 172.21.15.157:0/3418517042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900c8510 con 0x7f84900c9010 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849634e640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 0x7f849015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:12:53.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f849534c640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ab0 tx=0x7f847c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800be070 con 0x7f84900c9010 2023-11-06T03:12:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480004020 con 0x7f84900c9010 2023-11-06T03:12:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900c87f0 con 0x7f84900c9010 2023-11-06T03:12:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800043b0 con 0x7f84900c9010 2023-11-06T03:12:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84900c7f00 con 0x7f84900c9010 2023-11-06T03:12:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490072750 con 0x7f84900c9010 2023-11-06T03:12:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84800b4040 con 0x7f84900c9010 2023-11-06T03:12:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f848011d030 con 0x7f84900c9010 2023-11-06T03:12:53.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.793+0000 7f8495b4d640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:53.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.797+0000 7f8495b4d640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8490159470 tx=0x7f848c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:53.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.801+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84800eaad0 con 0x7f84900c9010 2023-11-06T03:12:53.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.985+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f849006e3e0 con 0x7f84900c9010 2023-11-06T03:12:53.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.985+0000 7f84867fc640 1 -- 172.21.15.157:0/3418517042 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f84800ee0f0 con 0x7f84900c9010 2023-11-06T03:12:53.987 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:12:53.987 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:12:53.988 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:12:53.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 msgr2=0x7f8474067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8490159470 tx=0x7f848c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:53.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 msgr2=0x7f84901eff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:53.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ab0 tx=0x7f847c005800 comp rx=0 tx=0).stop 2023-11-06T03:12:53.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 shutdown_connections 2023-11-06T03:12:53.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84900c7a50 0x7f849015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84900c9010 0x7f84901eff60 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 --2- 172.21.15.157:0/3418517042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84900b2b10 0x7f84901f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:53.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 >> 172.21.15.157:0/3418517042 conn(0x7f849006d560 msgr2=0x7f849006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:53.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 shutdown_connections 2023-11-06T03:12:53.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:53.989+0000 7f8497db1640 1 -- 172.21.15.157:0/3418517042 wait complete. 2023-11-06T03:12:54.407 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:12:45.374552+0000', 'modified': '2023-11-06T03:12:47.390020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8226': {'gid': 8226, 'name': 'c', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/2645931923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6833', 'nonce': 2645931923}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2645931923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:12:54.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:12:54.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:12:54.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 Processor -- start 2023-11-06T03:12:54.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 -- start start 2023-11-06T03:12:54.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87700d13b0 0x7f87700c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877015b360 con 0x7f877015af80 2023-11-06T03:12:54.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87700cc780 con 0x7f87700cd9e0 2023-11-06T03:12:54.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.533+0000 7f87782e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700cca60 con 0x7f87700d13b0 2023-11-06T03:12:54.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776080640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f877015af80 0x7f87700cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32856/0 (socket says 172.21.15.157:32856) 2023-11-06T03:12:54.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776080640 1 -- 172.21.15.157:0/1927471620 learned_addr learned my addr 172.21.15.157:0/1927471620 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:54.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776881640 1 --2- 172.21.15.157:0/1927471620 >> v2:172.21.15.195:3300/0 conn(0x7f87700d13b0 0x7f87700c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214394242 0 0) 0x7f877015b360 con 0x7f877015af80 2023-11-06T03:12:54.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f875c003530 con 0x7f877015af80 2023-11-06T03:12:54.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699896282 0 0) 0x7f875c003530 con 0x7f877015af80 2023-11-06T03:12:54.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 >> v2:172.21.15.195:3300/0 conn(0x7f87700d13b0 msgr2=0x7f87700c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 --2- 172.21.15.157:0/1927471620 >> v2:172.21.15.195:3300/0 conn(0x7f87700d13b0 0x7f87700c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 >> v1:172.21.15.195:6789/0 conn(0x7f87700cd9e0 legacy=0x7f87700c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700ccd40 con 0x7f877015af80 2023-11-06T03:12:54.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f876c001a00 con 0x7f877015af80 2023-11-06T03:12:54.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 >> v1:172.21.15.157:6789/0 conn(0x7f877015af80 legacy=0x7f87700cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c003980 0x7f875c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 0x7f875c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f875c005190 0x7f875c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700cca60 con 0x7f875c0047d0 2023-11-06T03:12:54.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c030 con 0x7f875c005190 2023-11-06T03:12:54.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c1f0 con 0x7f875c003980 2023-11-06T03:12:54.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 0x7f875c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776080640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c003980 0x7f875c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776881640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f875c005190 0x7f875c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 -- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c003980 msgr2=0x7f875c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c003980 0x7f875c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 -- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f875c005190 msgr2=0x7f875c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f875c005190 0x7f875c00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f8776881640 1 -- 172.21.15.157:0/1927471620 reap_dead start 2023-11-06T03:12:54.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 -- 172.21.15.157:0/1927471620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c003530 con 0x7f875c0047d0 2023-11-06T03:12:54.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877587f640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 0x7f875c004c50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8760001780 tx=0x7f876001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940d3ac590e870ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87600a4ea0 con 0x7f875c0047d0 2023-11-06T03:12:54.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87600a7050 con 0x7f875c0047d0 2023-11-06T03:12:54.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.537+0000 7f877507e640 1 -- 172.21.15.157:0/1927471620 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760003600 con 0x7f875c0047d0 2023-11-06T03:12:54.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 msgr2=0x7f875c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 0x7f875c004c50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8760001780 tx=0x7f876001c3e0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/1927471620 shutdown_connections 2023-11-06T03:12:54.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 --2- 172.21.15.157:0/1927471620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f875c0047d0 0x7f875c004c50 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/1927471620 >> 172.21.15.157:0/1927471620 conn(0x7f87700d08c0 msgr2=0x7f87700d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/1927471620 shutdown_connections 2023-11-06T03:12:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/1927471620 wait complete. 2023-11-06T03:12:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 Processor -- start 2023-11-06T03:12:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- start start 2023-11-06T03:12:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c0047d0 0x7f8770158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f877015ce60 0x7f877015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877015d820 con 0x7f87700c7c00 2023-11-06T03:12:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87700ca9c0 con 0x7f877015ce60 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700c9db0 con 0x7f875c0047d0 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53870/0 (socket says 172.21.15.157:53870) 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 -- 172.21.15.157:0/2544911332 learned_addr learned my addr 172.21.15.157:0/2544911332 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f8776080640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c0047d0 0x7f8770158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f8776881640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f877015ce60 0x7f877015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 -- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c0047d0 msgr2=0x7f8770158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c0047d0 0x7f8770158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 -- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f877015ce60 msgr2=0x7f877015d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f877015ce60 0x7f877015d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 -- 172.21.15.157:0/2544911332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87700ca090 con 0x7f87700c7c00 2023-11-06T03:12:54.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f877587f640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8760005040 tx=0x7f875c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f8766ffd640 1 -- 172.21.15.157:0/2544911332 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760003e40 con 0x7f87700c7c00 2023-11-06T03:12:54.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f8766ffd640 1 -- 172.21.15.157:0/2544911332 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8760007a40 con 0x7f87700c7c00 2023-11-06T03:12:54.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87700ca370 con 0x7f87700c7c00 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f8766ffd640 1 -- 172.21.15.157:0/2544911332 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760007270 con 0x7f87700c7c00 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.541+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8770158c90 con 0x7f87700c7c00 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8766ffd640 1 -- 172.21.15.157:0/2544911332 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87600073d0 con 0x7f87700c7c00 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8766ffd640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 0x7f8754067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8766ffd640 1 -- 172.21.15.157:0/2544911332 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f876011d6b0 con 0x7f87700c7c00 2023-11-06T03:12:54.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776080640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 0x7f8754067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 Processor -- start 2023-11-06T03:12:54.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776080640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 0x7f8754067f20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f876c000f90 tx=0x7f876c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 -- start start 2023-11-06T03:12:54.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618cf1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618ca7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618b6ffc0 con 0x55a618ca7a90 2023-11-06T03:12:54.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618afe1a0 con 0x55a618cf2000 2023-11-06T03:12:54.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a618afb770 con 0x55a618cf16e0 2023-11-06T03:12:54.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53880/0 (socket says 172.21.15.157:53880) 2023-11-06T03:12:54.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 -- 172.21.15.157:0/3485468996 learned_addr learned my addr 172.21.15.157:0/3485468996 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f877587f640 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618ca7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776080640 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618cf1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 -- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 msgr2=0x55a618cf1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618cf1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 -- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 msgr2=0x55a618ca7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618ca7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 -- 172.21.15.157:0/3485468996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6187f3890 con 0x55a618ca7a90 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8776881640 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f87700b68f0 tx=0x7f8768009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7809d18e39144211 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8753fff640 1 -- 172.21.15.157:0/3485468996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680127a0 con 0x55a618ca7a90 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8753fff640 1 -- 172.21.15.157:0/3485468996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768012900 con 0x55a618ca7a90 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.545+0000 7f8753fff640 1 -- 172.21.15.157:0/3485468996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768012cb0 con 0x55a618ca7a90 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/3485468996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 msgr2=0x55a618ca7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f87700b68f0 tx=0x7f8768009190 comp rx=0 tx=0).stop 2023-11-06T03:12:54.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/3485468996 shutdown_connections 2023-11-06T03:12:54.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618ca7e70 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618ca7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/3485468996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618cf1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/3485468996 >> 172.21.15.157:0/3485468996 conn(0x55a618cadbf0 msgr2=0x55a618ca5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/3485468996 shutdown_connections 2023-11-06T03:12:54.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/3485468996 wait complete. 2023-11-06T03:12:54.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- start start 2023-11-06T03:12:54.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 Processor -- start 2023-11-06T03:12:54.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618cb2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618af7060 con 0x55a618cf16e0 2023-11-06T03:12:54.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618a96290 con 0x55a618cf2000 2023-11-06T03:12:54.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a618ab46f0 con 0x55a618ca7a90 2023-11-06T03:12:54.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 10 client.0 _ll_register_callbacks cb 0x55a618c931d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:54.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53882/0 (socket says 172.21.15.157:53882) 2023-11-06T03:12:54.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 -- 172.21.15.157:0/2506812980 learned_addr learned my addr 172.21.15.157:0/2506812980 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f8776080640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618cb2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 -- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 msgr2=0x55a618cb2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f8776881640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618cb2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 -- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 msgr2=0x55a618cb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cb3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 -- 172.21.15.157:0/2506812980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8768009640 con 0x55a618cf16e0 2023-11-06T03:12:54.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877587f640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f8760097ce0 tx=0x7f87600077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49e74ddab9dd693 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:54.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87600f2940 con 0x55a618cf16e0 2023-11-06T03:12:54.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f87600f6af0 con 0x55a618cf16e0 2023-11-06T03:12:54.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a618af59b0 con 0x55a618cf16e0 2023-11-06T03:12:54.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87600ee000 con 0x55a618cf16e0 2023-11-06T03:12:54.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 10 client.8397 fetch_fsmap learned FSMap version 85 2023-11-06T03:12:54.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55a618abc1e0 con 0x55a618cf16e0 2023-11-06T03:12:54.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a618af59b0 con 0x55a618cf16e0 2023-11-06T03:12:54.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f87600f6660 con 0x55a618cf16e0 2023-11-06T03:12:54.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 10 client.8397 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 10 client.8397 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 20 client.8397 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618cb3de0 0x55a618d02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 4 client.8397 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a618b264d0 con 0x55a618cb3de0 2023-11-06T03:12:54.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f8776080640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618cb3de0 0x55a618d02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f8776080640 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618cb3de0 0x55a618d02c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9ae65fb4ad8af2 server_cookie=22e83370e972171f in_seq=0 out_seq=0 2023-11-06T03:12:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 10 client.8397 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:12:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 1 -- 172.21.15.157:0/2506812980 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a618b264d0 con 0x55a618cb3de0 2023-11-06T03:12:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f87517fa640 10 client.8397 handle_command_reply: tid=0 2023-11-06T03:12:54.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 client.8397 shutdown 2023-11-06T03:12:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 msgr2=0x55a618cb2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.549+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f8760097ce0 tx=0x7f87600077c0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 shutdown_connections 2023-11-06T03:12:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618cb3de0 0x55a618d02c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf16e0 0x55a618cb2fd0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f87517fa640 0 client.8397 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:12:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 --2- 172.21.15.157:0/2506812980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618cb2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 >> 172.21.15.157:0/2506812980 conn(0x55a618cadbf0 msgr2=0x55a618ca6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 shutdown_connections 2023-11-06T03:12:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 1 -- 172.21.15.157:0/2506812980 wait complete. 2023-11-06T03:12:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.553+0000 7f877b40c000 20 client.8397 trim_cache size 0 max 16384 2023-11-06T03:12:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 Processor -- start 2023-11-06T03:12:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 -- start start 2023-11-06T03:12:54.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618d048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6187b4940 con 0x55a618ca7a90 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618cf15a0 con 0x55a618cf2000 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a618ac4bb0 con 0x55a618cf16e0 2023-11-06T03:12:54.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53886/0 (socket says 172.21.15.157:53886) 2023-11-06T03:12:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 -- 172.21.15.157:0/2865165717 learned_addr learned my addr 172.21.15.157:0/2865165717 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 -- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 msgr2=0x55a618d048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618d048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 -- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 msgr2=0x55a618cea370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 -- 172.21.15.157:0/2865165717 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a618aa2330 con 0x55a618ca7a90 2023-11-06T03:12:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8776080640 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f876c005c30 tx=0x7f876c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6876bc4974adbd25 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8764ff9640 1 -- 172.21.15.157:0/2865165717 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c02d030 con 0x55a618ca7a90 2023-11-06T03:12:54.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8764ff9640 1 -- 172.21.15.157:0/2865165717 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0292d0 con 0x55a618ca7a90 2023-11-06T03:12:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.565+0000 7f8764ff9640 1 -- 172.21.15.157:0/2865165717 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c01b040 con 0x55a618ca7a90 2023-11-06T03:12:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/2865165717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 msgr2=0x55a618d04370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f876c005c30 tx=0x7f876c016000 comp rx=0 tx=0).stop 2023-11-06T03:12:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/2865165717 shutdown_connections 2023-11-06T03:12:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf2000 0x55a618cea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618cf16e0 0x55a618d048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/2865165717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618ca7a90 0x55a618d04370 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/2865165717 >> 172.21.15.157:0/2865165717 conn(0x55a618cadbf0 msgr2=0x55a618d02c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/2865165717 shutdown_connections 2023-11-06T03:12:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/2865165717 wait complete. 2023-11-06T03:12:54.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- start start 2023-11-06T03:12:54.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 Processor -- start 2023-11-06T03:12:54.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618d4da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 0x55a618d4df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618ab9960 con 0x55a618cf2000 2023-11-06T03:12:54.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a618abe7d0 con 0x55a618cf16e0 2023-11-06T03:12:54.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a618cab120 con 0x55a618ca7a90 2023-11-06T03:12:54.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 10 client.0 _ll_register_callbacks cb 0x55a618c739e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:12:54.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53892/0 (socket says 172.21.15.157:53892) 2023-11-06T03:12:54.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 -- 172.21.15.157:0/338172590 learned_addr learned my addr 172.21.15.157:0/338172590 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877587f640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 0x55a618d4df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776080640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618d4da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 -- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 msgr2=0x55a618d4da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618d4da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 -- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 msgr2=0x55a618d4df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 0x55a618d4df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 -- 172.21.15.157:0/338172590 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c00f030 con 0x55a618cf2000 2023-11-06T03:12:54.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776080640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618d4da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:54.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877587f640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 0x55a618d4df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:54.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776881640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f87680091d0 tx=0x7f8768004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cfc6bdecc4be06 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:12:54.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768023070 con 0x55a618cf2000 2023-11-06T03:12:54.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8768004490 con 0x55a618cf2000 2023-11-06T03:12:54.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a618ac6b90 con 0x55a618cf2000 2023-11-06T03:12:54.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8768005bb0 con 0x55a618cf2000 2023-11-06T03:12:54.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 10 client.8403 fetch_fsmap learned FSMap version 85 2023-11-06T03:12:54.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55a6187f3cc0 con 0x55a618cf2000 2023-11-06T03:12:54.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a618ac6b90 con 0x55a618cf2000 2023-11-06T03:12:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8768005d10 con 0x55a618cf2000 2023-11-06T03:12:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 10 client.8403 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:12:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 10 client.8403 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:12:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 20 client.8403 populate_metadata read hostname 'smithi157' 2023-11-06T03:12:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618ced4f0 0x55a618ced8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 4 client.8403 mds_command: new command op to 8226 tid=0[{"prefix": "status"}] 2023-11-06T03:12:54.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a618af00e0 con 0x55a618ced4f0 2023-11-06T03:12:54.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776080640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618ced4f0 0x55a618ced8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f8776080640 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618ced4f0 0x55a618ced8d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b072229368bdc58 server_cookie=49e69ae4dded4799 in_seq=0 out_seq=0 2023-11-06T03:12:54.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 10 client.8403 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:12:54.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 1 -- 172.21.15.157:0/338172590 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a618af00e0 con 0x55a618ced4f0 2023-11-06T03:12:54.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 10 client.8403 handle_command_reply: tid=0 2023-11-06T03:12:54.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 client.8403 shutdown 2023-11-06T03:12:54.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 msgr2=0x55a618cecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f87680091d0 tx=0x7f8768004740 comp rx=0 tx=0).stop 2023-11-06T03:12:54.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 shutdown_connections 2023-11-06T03:12:54.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55a618ced4f0 0x55a618ced8d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a618cf2000 0x55a618cecfb0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a618cf16e0 0x55a618d4df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f87517fa640 0 client.8403 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:12:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 --2- 172.21.15.157:0/338172590 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a618ca7a90 0x55a618d4da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 >> 172.21.15.157:0/338172590 conn(0x55a618cadbf0 msgr2=0x55a618cad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 shutdown_connections 2023-11-06T03:12:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.569+0000 7f877b40c000 1 -- 172.21.15.157:0/338172590 wait complete. 2023-11-06T03:12:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f877b40c000 20 client.8403 trim_cache size 0 max 16384 2023-11-06T03:12:54.599 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:12:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:12:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8226, 2023-11-06T03:12:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:12:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:12:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:12:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 9.1983610490000007, 2023-11-06T03:12:54.600 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 85, 2023-11-06T03:12:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 113, 2023-11-06T03:12:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 111, 2023-11-06T03:12:54.601 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 13.617993732 2023-11-06T03:12:54.601 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:54.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 msgr2=0x7f8754067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 0x7f8754067f20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f876c000f90 tx=0x7f876c009040 comp rx=0 tx=0).stop 2023-11-06T03:12:54.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 msgr2=0x7f87700c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8760005040 tx=0x7f875c005800 comp rx=0 tx=0).stop 2023-11-06T03:12:54.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 shutdown_connections 2023-11-06T03:12:54.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8754065a60 0x7f8754067f20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f877015ce60 0x7f877015d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87700c7c00 0x7f87700c8060 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 --2- 172.21.15.157:0/2544911332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f875c0047d0 0x7f8770158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 >> 172.21.15.157:0/2544911332 conn(0x7f87700d08c0 msgr2=0x7f8770159590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 shutdown_connections 2023-11-06T03:12:54.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.573+0000 7f87782e4640 1 -- 172.21.15.157:0/2544911332 wait complete. 2023-11-06T03:12:54.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:12:54.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 Processor -- start 2023-11-06T03:12:54.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 -- start start 2023-11-06T03:12:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6eb41609a0 0x7f6eb40c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6eb40aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40ca1d0 con 0x7f6eb415cea0 2023-11-06T03:12:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb40ce2a0 con 0x7f6eb41609a0 2023-11-06T03:12:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb9661640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41617e0 con 0x7f6eb40c9d70 2023-11-06T03:12:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb2ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6eb415cea0 0x7f6eb40cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32858/0 (socket says 172.21.15.157:32858) 2023-11-06T03:12:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb2ffd640 1 -- 172.21.15.157:0/2877330835 learned_addr learned my addr 172.21.15.157:0/2877330835 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:12:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb37fe640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6eb40aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405434265 0 0) 0x7f6eb40ca1d0 con 0x7f6eb415cea0 2023-11-06T03:12:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e9c003530 con 0x7f6eb415cea0 2023-11-06T03:12:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2877330835 >> v2:172.21.15.195:3300/0 conn(0x7f6eb41609a0 0x7f6eb40c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3020000964 0 0) 0x7f6e9c003530 con 0x7f6eb415cea0 2023-11-06T03:12:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 msgr2=0x7f6eb40aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6eb40aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 >> v2:172.21.15.195:3300/0 conn(0x7f6eb41609a0 msgr2=0x7f6eb40c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 --2- 172.21.15.157:0/2877330835 >> v2:172.21.15.195:3300/0 conn(0x7f6eb41609a0 0x7f6eb40c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40ac9f0 con 0x7f6eb415cea0 2023-11-06T03:12:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6ea4001680 con 0x7f6eb415cea0 2023-11-06T03:12:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 >> v1:172.21.15.157:6789/0 conn(0x7f6eb415cea0 legacy=0x7f6eb40cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6e9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9c003f30 0x7f6e9c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c004d10 0x7f6e9c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41617e0 con 0x7f6e9c003f30 2023-11-06T03:12:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c00b990 con 0x7f6e9c004d10 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c00bb50 con 0x7f6eb40c9d70 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9c003f30 0x7f6e9c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb2ffd640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6e9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 msgr2=0x7f6e9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6e9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c004d10 msgr2=0x7f6e9c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:12:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c004d10 0x7f6e9c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2877330835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c003530 con 0x7f6e9c003f30 2023-11-06T03:12:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.709+0000 7f6eb2ffd640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb40c9d70 0x7f6e9c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:54.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9c003f30 0x7f6e9c0047d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009ab0 tx=0x7f6ea801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c296924f4c80ae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80a5640 con 0x7f6e9c003f30 2023-11-06T03:12:54.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea80a57a0 con 0x7f6e9c003f30 2023-11-06T03:12:54.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb1ffb640 1 -- 172.21.15.157:0/2877330835 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80a5b50 con 0x7f6e9c003f30 2023-11-06T03:12:54.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- 172.21.15.157:0/2877330835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9c003f30 msgr2=0x7f6e9c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2877330835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e9c003f30 0x7f6e9c0047d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009ab0 tx=0x7f6ea801ca70 comp rx=0 tx=0).stop 2023-11-06T03:12:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb37fe640 1 -- 172.21.15.157:0/2877330835 reap_dead start 2023-11-06T03:12:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- 172.21.15.157:0/2877330835 shutdown_connections 2023-11-06T03:12:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- 172.21.15.157:0/2877330835 >> 172.21.15.157:0/2877330835 conn(0x7f6eb40cba30 msgr2=0x7f6eb40cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- 172.21.15.157:0/2877330835 shutdown_connections 2023-11-06T03:12:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- 172.21.15.157:0/2877330835 wait complete. 2023-11-06T03:12:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 Processor -- start 2023-11-06T03:12:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.713+0000 7f6eb9661640 1 -- start start 2023-11-06T03:12:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c003f30 0x7f6eb4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 0x7f6eb415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b3310 con 0x7f6eb40b2950 2023-11-06T03:12:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb415c820 con 0x7f6e9c003f30 2023-11-06T03:12:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b3510 con 0x7f6eb415be50 2023-11-06T03:12:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53912/0 (socket says 172.21.15.157:53912) 2023-11-06T03:12:54.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2952925363 learned_addr learned my addr 172.21.15.157:0/2952925363 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:12:54.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb37fe640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 0x7f6eb415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb2ffd640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c003f30 0x7f6eb4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 msgr2=0x7f6eb415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 0x7f6eb415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c003f30 msgr2=0x7f6eb4156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c003f30 0x7f6eb4156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 -- 172.21.15.157:0/2952925363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40b37f0 con 0x7f6eb40b2950 2023-11-06T03:12:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb37fe640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 0x7f6eb415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:12:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb27fc640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80b0040 tx=0x7f6e9c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80a5e90 con 0x7f6eb40b2950 2023-11-06T03:12:54.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea8004020 con 0x7f6eb40b2950 2023-11-06T03:12:54.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb40b3ad0 con 0x7f6eb40b2950 2023-11-06T03:12:54.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80043d0 con 0x7f6eb40b2950 2023-11-06T03:12:54.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb415f0d0 con 0x7f6eb40b2950 2023-11-06T03:12:54.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb4072750 con 0x7f6eb40b2950 2023-11-06T03:12:54.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ea80b4040 con 0x7f6eb40b2950 2023-11-06T03:12:54.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 0x7f6ea0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:12:54.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6ea811d3f0 con 0x7f6eb40b2950 2023-11-06T03:12:54.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb2ffd640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 0x7f6ea0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:12:54.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.717+0000 7f6eb2ffd640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 0x7f6ea0067f20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40018f0 tx=0x7f6ea4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:12:54.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.721+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6ea80ead70 con 0x7f6eb40b2950 2023-11-06T03:12:54.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6eb40b4f30 con 0x7f6eb40b2950 2023-11-06T03:12:54.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6e9b7fe640 1 -- 172.21.15.157:0/2952925363 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ea80ee390 con 0x7f6eb40b2950 2023-11-06T03:12:54.897 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:12:54.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 msgr2=0x7f6ea0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 0x7f6ea0067f20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40018f0 tx=0x7f6ea4009040 comp rx=0 tx=0).stop 2023-11-06T03:12:54.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 msgr2=0x7f6eb40b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80b0040 tx=0x7f6e9c005800 comp rx=0 tx=0).stop 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 shutdown_connections 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ea0065a60 0x7f6ea0067f20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6eb415be50 0x7f6eb415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6eb40b2950 0x7f6eb40b2dd0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 --2- 172.21.15.157:0/2952925363 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e9c003f30 0x7f6eb4156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:12:54.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 >> 172.21.15.157:0/2952925363 conn(0x7f6eb40cba30 msgr2=0x7f6eb406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:12:54.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 shutdown_connections 2023-11-06T03:12:54.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:12:54.897+0000 7f6eb9661640 1 -- 172.21.15.157:0/2952925363 wait complete. 2023-11-06T03:12:54.914 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:12:54.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:12:54.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:12:54.915 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:12:54.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:54.916 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:12:54.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:12:54.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:12:54.917 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:54.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:54.918 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:12:54.924 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:12:54.924 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:12:54.924 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:12:54.924 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.925 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:12:54.925 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.925 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:54.925 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.925 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:12:54.926 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:54.926 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.926 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:12:54.926 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:12:54.926 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41964sec preferred_lft 41964sec 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:12:54.927 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.928 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:12:54.928 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.928 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:12:54.928 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.928 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:12:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:12:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:12:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:12:54.929 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:12:54.930 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:12:54.931 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:12:54.931 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:12:54.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:54.931 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:12:54.978 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:12:54.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:54.980 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:12:55.031 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:12:55.031 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:12:55.032 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:12:55.032 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:12:55.032 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:12:55.032 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:12:55.032 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:55.033 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:55.033 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:12:55.033 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:12:55.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:55.034 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:12:55.034 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:55.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:12:55.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:12:55.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:12:55.036 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:55.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:55.103 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:12:55.149 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:12:55.149 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:12:55.149 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:55.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:55.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:12:55.207 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:12:55.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:12:55.208 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:55.256 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:12:55.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:12:55.257 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:55.305 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:55.362 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:55.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:55.416 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:55.417 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:55.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:55.418 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:55.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:12:55.465 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:12:55.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:55.524 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:55.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:55.579 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:55.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:55.580 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:55.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:12:55.585+0000 7fed04da3580 -1 init, newargv = 0x561f39e1bc80 newargc=15 2023-11-06T03:12:55.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[84184]: starting ceph client 2023-11-06T03:12:55.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[84184]: starting fuse 2023-11-06T03:12:56.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:12:56.707 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:12:56.708 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:12:56.764 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:12:56.765 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:12:56.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:56.766 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:12:56.813 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:12:56.813 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:12:56.814 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:12:56.814 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:12:56.815 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:12:56.815 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:12:56.815 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:12:56.815 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:56.816 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:12:56.816 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:12:56.816 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:12:56.817 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:12:56.817 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:56.818 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:12:56.818 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:12:56.818 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:12:56.819 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:12:56.819 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:56.819 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:12:56.820 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:12:56.820 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:12:56.820 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:12:56.821 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:12:56.821 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:12:56.821 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:12:56.822 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:12:56.822 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:12:56.822 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:12:56.823 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:12:56.933 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.84184.asok 2023-11-06T03:12:57.115 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84184.asok 2023-11-06T03:12:57.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:57.116 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84184.asok status 2023-11-06T03:12:57.268 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:12:57.269 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:12:57.270 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "84184", 2023-11-06T03:12:57.270 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:12:57.270 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:57.270 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8418, 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:12:57.271 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 8418 2023-11-06T03:12:57.272 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:57.272 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:12:57.272 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:12:57.272 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:12:57.272 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1399707842 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:12:57.273 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1399707842 2023-11-06T03:12:57.274 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:12:57.274 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.8418 192.168.0.1:0/1399707842", 2023-11-06T03:12:57.274 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1399707842", 2023-11-06T03:12:57.274 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:12:57.274 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 85, 2023-11-06T03:12:57.275 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 113, 2023-11-06T03:12:57.275 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:12:57.275 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:12:57.275 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:12:57.275 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:12:57.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:57.283 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:12:57.283 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:57.290 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:12:57.290 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:57.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:57.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:57.346 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2023-11-06T03:12:57.346 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2023-11-06T03:12:57.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:57.347 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:57.348 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:57.349 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.0 2023-11-06T03:12:57.349 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:57.350 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:57.351 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:57.351 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208881 s, 50.2 MB/s 2023-11-06T03:12:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157266 s, 66.7 MB/s 2023-11-06T03:12:57.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.473 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171402 s, 61.2 MB/s 2023-11-06T03:12:57.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.489 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125124 s, 83.8 MB/s 2023-11-06T03:12:57.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.506 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139033 s, 75.4 MB/s 2023-11-06T03:12:57.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.522 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012878 s, 81.4 MB/s 2023-11-06T03:12:57.541 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.541 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.541 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161914 s, 64.8 MB/s 2023-11-06T03:12:57.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119385 s, 87.8 MB/s 2023-11-06T03:12:57.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144649 s, 72.5 MB/s 2023-11-06T03:12:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.590 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143617 s, 73.0 MB/s 2023-11-06T03:12:57.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123403 s, 85.0 MB/s 2023-11-06T03:12:57.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.618 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106388 s, 98.6 MB/s 2023-11-06T03:12:57.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.632 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997289 s, 105 MB/s 2023-11-06T03:12:57.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.648 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137358 s, 76.3 MB/s 2023-11-06T03:12:57.663 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.663 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.663 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123036 s, 85.2 MB/s 2023-11-06T03:12:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.681 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143005 s, 73.3 MB/s 2023-11-06T03:12:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011556 s, 90.7 MB/s 2023-11-06T03:12:57.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122251 s, 85.8 MB/s 2023-11-06T03:12:57.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.727 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136399 s, 76.9 MB/s 2023-11-06T03:12:57.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013917 s, 75.3 MB/s 2023-11-06T03:12:57.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131438 s, 79.8 MB/s 2023-11-06T03:12:57.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138966 s, 75.5 MB/s 2023-11-06T03:12:57.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.797 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169723 s, 61.8 MB/s 2023-11-06T03:12:57.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.812 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.812 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123428 s, 85.0 MB/s 2023-11-06T03:12:57.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.825 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104083 s, 101 MB/s 2023-11-06T03:12:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116963 s, 89.6 MB/s 2023-11-06T03:12:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132578 s, 79.1 MB/s 2023-11-06T03:12:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115243 s, 91.0 MB/s 2023-11-06T03:12:57.885 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.886 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130174 s, 80.6 MB/s 2023-11-06T03:12:57.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.901 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120402 s, 87.1 MB/s 2023-11-06T03:12:57.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137994 s, 76.0 MB/s 2023-11-06T03:12:57.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.931 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104032 s, 101 MB/s 2023-11-06T03:12:57.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:57.932 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:57.932 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:57.933 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.1 2023-11-06T03:12:57.933 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:57.933 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:57.934 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:57.934 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:57.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.958 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109907 s, 95.4 MB/s 2023-11-06T03:12:57.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124877 s, 84.0 MB/s 2023-11-06T03:12:57.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:57.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:57.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121491 s, 86.3 MB/s 2023-11-06T03:12:58.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142677 s, 73.5 MB/s 2023-11-06T03:12:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140438 s, 74.7 MB/s 2023-11-06T03:12:58.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139965 s, 74.9 MB/s 2023-11-06T03:12:58.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145098 s, 72.3 MB/s 2023-11-06T03:12:58.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.076 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.076 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133139 s, 78.8 MB/s 2023-11-06T03:12:58.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.091 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117107 s, 89.5 MB/s 2023-11-06T03:12:58.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113685 s, 92.2 MB/s 2023-11-06T03:12:58.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130969 s, 80.1 MB/s 2023-11-06T03:12:58.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013657 s, 76.8 MB/s 2023-11-06T03:12:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146735 s, 71.5 MB/s 2023-11-06T03:12:58.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.174 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129433 s, 81.0 MB/s 2023-11-06T03:12:58.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109305 s, 95.9 MB/s 2023-11-06T03:12:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132852 s, 78.9 MB/s 2023-11-06T03:12:58.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.221 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128377 s, 81.7 MB/s 2023-11-06T03:12:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.239 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145405 s, 72.1 MB/s 2023-11-06T03:12:58.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137072 s, 76.5 MB/s 2023-11-06T03:12:58.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126483 s, 82.9 MB/s 2023-11-06T03:12:58.292 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.293 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162605 s, 64.5 MB/s 2023-11-06T03:12:58.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.312 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149894 s, 70.0 MB/s 2023-11-06T03:12:58.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146984 s, 71.3 MB/s 2023-11-06T03:12:58.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.352 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143193 s, 73.2 MB/s 2023-11-06T03:12:58.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144359 s, 72.6 MB/s 2023-11-06T03:12:58.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.390 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139426 s, 75.2 MB/s 2023-11-06T03:12:58.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151744 s, 69.1 MB/s 2023-11-06T03:12:58.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124942 s, 83.9 MB/s 2023-11-06T03:12:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123303 s, 85.0 MB/s 2023-11-06T03:12:58.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158666 s, 66.1 MB/s 2023-11-06T03:12:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156298 s, 67.1 MB/s 2023-11-06T03:12:58.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.500 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133107 s, 78.8 MB/s 2023-11-06T03:12:58.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:58.502 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:58.502 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:58.502 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.2 2023-11-06T03:12:58.503 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:58.503 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:58.503 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:58.504 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162772 s, 64.4 MB/s 2023-11-06T03:12:58.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.601 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136834 s, 76.6 MB/s 2023-11-06T03:12:58.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014411 s, 72.8 MB/s 2023-11-06T03:12:58.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.634 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.634 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967348 s, 108 MB/s 2023-11-06T03:12:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115746 s, 90.6 MB/s 2023-11-06T03:12:58.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.664 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124789 s, 84.0 MB/s 2023-11-06T03:12:58.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011184 s, 93.8 MB/s 2023-11-06T03:12:58.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01064 s, 98.6 MB/s 2023-11-06T03:12:58.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.712 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143937 s, 72.8 MB/s 2023-11-06T03:12:58.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135755 s, 77.2 MB/s 2023-11-06T03:12:58.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142894 s, 73.4 MB/s 2023-11-06T03:12:58.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145044 s, 72.3 MB/s 2023-11-06T03:12:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141531 s, 74.1 MB/s 2023-11-06T03:12:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.807 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146753 s, 71.5 MB/s 2023-11-06T03:12:58.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.824 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123579 s, 84.9 MB/s 2023-11-06T03:12:58.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.842 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134579 s, 77.9 MB/s 2023-11-06T03:12:58.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.861 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144292 s, 72.7 MB/s 2023-11-06T03:12:58.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.880 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143485 s, 73.1 MB/s 2023-11-06T03:12:58.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.899 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138987 s, 75.4 MB/s 2023-11-06T03:12:58.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138611 s, 75.6 MB/s 2023-11-06T03:12:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137737 s, 76.1 MB/s 2023-11-06T03:12:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117435 s, 89.3 MB/s 2023-11-06T03:12:58.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.967 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133201 s, 78.7 MB/s 2023-11-06T03:12:58.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:58.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:58.986 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01493 s, 70.2 MB/s 2023-11-06T03:12:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126149 s, 83.1 MB/s 2023-11-06T03:12:59.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.021 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149224 s, 70.3 MB/s 2023-11-06T03:12:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.042 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162228 s, 64.6 MB/s 2023-11-06T03:12:59.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.060 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128976 s, 81.3 MB/s 2023-11-06T03:12:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.080 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156544 s, 67.0 MB/s 2023-11-06T03:12:59.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142059 s, 73.8 MB/s 2023-11-06T03:12:59.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.118 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146598 s, 71.5 MB/s 2023-11-06T03:12:59.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134063 s, 78.2 MB/s 2023-11-06T03:12:59.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:59.138 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:59.139 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:59.139 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.3 2023-11-06T03:12:59.139 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:59.140 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:59.140 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:59.140 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.205 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108691 s, 96.5 MB/s 2023-11-06T03:12:59.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132051 s, 79.4 MB/s 2023-11-06T03:12:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122836 s, 85.4 MB/s 2023-11-06T03:12:59.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.255 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139431 s, 75.2 MB/s 2023-11-06T03:12:59.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133593 s, 78.5 MB/s 2023-11-06T03:12:59.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149288 s, 70.2 MB/s 2023-11-06T03:12:59.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.308 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143819 s, 72.9 MB/s 2023-11-06T03:12:59.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120228 s, 87.2 MB/s 2023-11-06T03:12:59.340 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.340 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.340 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141437 s, 74.1 MB/s 2023-11-06T03:12:59.356 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.356 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.356 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123577 s, 84.9 MB/s 2023-11-06T03:12:59.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.372 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01255 s, 83.6 MB/s 2023-11-06T03:12:59.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126285 s, 83.0 MB/s 2023-11-06T03:12:59.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.404 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131941 s, 79.5 MB/s 2023-11-06T03:12:59.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119347 s, 87.9 MB/s 2023-11-06T03:12:59.435 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138141 s, 75.9 MB/s 2023-11-06T03:12:59.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129986 s, 80.7 MB/s 2023-11-06T03:12:59.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.469 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137015 s, 76.5 MB/s 2023-11-06T03:12:59.487 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.487 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.488 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143488 s, 73.1 MB/s 2023-11-06T03:12:59.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.503 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138173 s, 75.9 MB/s 2023-11-06T03:12:59.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114068 s, 91.9 MB/s 2023-11-06T03:12:59.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106711 s, 98.3 MB/s 2023-11-06T03:12:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133214 s, 78.7 MB/s 2023-11-06T03:12:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.560 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110298 s, 95.1 MB/s 2023-11-06T03:12:59.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.574 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107742 s, 97.3 MB/s 2023-11-06T03:12:59.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143982 s, 72.8 MB/s 2023-11-06T03:12:59.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.606 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127634 s, 82.2 MB/s 2023-11-06T03:12:59.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110143 s, 95.2 MB/s 2023-11-06T03:12:59.634 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.634 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.634 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108979 s, 96.2 MB/s 2023-11-06T03:12:59.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.651 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143187 s, 73.2 MB/s 2023-11-06T03:12:59.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.667 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131327 s, 79.8 MB/s 2023-11-06T03:12:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015186 s, 69.0 MB/s 2023-11-06T03:12:59.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.702 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137063 s, 76.5 MB/s 2023-11-06T03:12:59.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:12:59.703 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:12:59.704 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:12:59.704 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.4 2023-11-06T03:12:59.704 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:12:59.705 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:12:59.705 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:12:59.705 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:12:59.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.772 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128859 s, 81.4 MB/s 2023-11-06T03:12:59.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.790 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138025 s, 76.0 MB/s 2023-11-06T03:12:59.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.804 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121354 s, 86.4 MB/s 2023-11-06T03:12:59.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118317 s, 88.6 MB/s 2023-11-06T03:12:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102819 s, 102 MB/s 2023-11-06T03:12:59.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.847 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120445 s, 87.1 MB/s 2023-11-06T03:12:59.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.864 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147933 s, 70.9 MB/s 2023-11-06T03:12:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123746 s, 84.7 MB/s 2023-11-06T03:12:59.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128694 s, 81.5 MB/s 2023-11-06T03:12:59.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012674 s, 82.7 MB/s 2023-11-06T03:12:59.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105868 s, 99.0 MB/s 2023-11-06T03:12:59.938 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.939 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.939 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130203 s, 80.5 MB/s 2023-11-06T03:12:59.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.956 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137271 s, 76.4 MB/s 2023-11-06T03:12:59.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.973 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143109 s, 73.3 MB/s 2023-11-06T03:12:59.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:12:59.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:12:59.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150039 s, 69.9 MB/s 2023-11-06T03:13:00.010 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.010 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.010 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158006 s, 66.4 MB/s 2023-11-06T03:13:00.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.028 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144996 s, 72.3 MB/s 2023-11-06T03:13:00.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148981 s, 70.4 MB/s 2023-11-06T03:13:00.061 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.061 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.061 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118609 s, 88.4 MB/s 2023-11-06T03:13:00.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.075 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011477 s, 91.4 MB/s 2023-11-06T03:13:00.090 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.090 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.090 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125531 s, 83.5 MB/s 2023-11-06T03:13:00.103 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.103 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.103 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103916 s, 101 MB/s 2023-11-06T03:13:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895871 s, 117 MB/s 2023-11-06T03:13:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142294 s, 73.7 MB/s 2023-11-06T03:13:00.145 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.145 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.146 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119547 s, 87.7 MB/s 2023-11-06T03:13:00.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109213 s, 96.0 MB/s 2023-11-06T03:13:00.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125948 s, 83.3 MB/s 2023-11-06T03:13:00.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128353 s, 81.7 MB/s 2023-11-06T03:13:00.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.206 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134511 s, 78.0 MB/s 2023-11-06T03:13:00.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.223 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135032 s, 77.7 MB/s 2023-11-06T03:13:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131949 s, 79.5 MB/s 2023-11-06T03:13:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013363 s, 78.5 MB/s 2023-11-06T03:13:00.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:00.255 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:00.256 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:00.256 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.5 2023-11-06T03:13:00.256 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:00.257 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:00.257 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:00.257 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:00.322 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010655 s, 98.4 MB/s 2023-11-06T03:13:00.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.337 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122052 s, 85.9 MB/s 2023-11-06T03:13:00.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130504 s, 80.3 MB/s 2023-11-06T03:13:00.365 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.365 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.365 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091804 s, 114 MB/s 2023-11-06T03:13:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.379 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122954 s, 85.3 MB/s 2023-11-06T03:13:00.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114451 s, 91.6 MB/s 2023-11-06T03:13:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.408 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011852 s, 88.5 MB/s 2023-11-06T03:13:00.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914729 s, 115 MB/s 2023-11-06T03:13:00.435 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.435 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.435 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136589 s, 76.8 MB/s 2023-11-06T03:13:00.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126102 s, 83.2 MB/s 2023-11-06T03:13:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111238 s, 94.3 MB/s 2023-11-06T03:13:00.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092969 s, 113 MB/s 2023-11-06T03:13:00.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112028 s, 93.6 MB/s 2023-11-06T03:13:00.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119517 s, 87.7 MB/s 2023-11-06T03:13:00.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.519 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131445 s, 79.8 MB/s 2023-11-06T03:13:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145863 s, 71.9 MB/s 2023-11-06T03:13:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106225 s, 98.7 MB/s 2023-11-06T03:13:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012029 s, 87.2 MB/s 2023-11-06T03:13:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.577 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113558 s, 92.3 MB/s 2023-11-06T03:13:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118381 s, 88.6 MB/s 2023-11-06T03:13:00.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106365 s, 98.6 MB/s 2023-11-06T03:13:00.616 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.616 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.616 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084804 s, 124 MB/s 2023-11-06T03:13:00.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.633 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140817 s, 74.5 MB/s 2023-11-06T03:13:00.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.648 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124614 s, 84.1 MB/s 2023-11-06T03:13:00.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.661 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.661 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108261 s, 96.9 MB/s 2023-11-06T03:13:00.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843219 s, 124 MB/s 2023-11-06T03:13:00.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.686 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120083 s, 87.3 MB/s 2023-11-06T03:13:00.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.700 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118435 s, 88.5 MB/s 2023-11-06T03:13:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875062 s, 120 MB/s 2023-11-06T03:13:00.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.726 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124944 s, 83.9 MB/s 2023-11-06T03:13:00.739 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.739 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.739 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109064 s, 96.1 MB/s 2023-11-06T03:13:00.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.755 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.755 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119746 s, 87.6 MB/s 2023-11-06T03:13:00.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:00.756 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:00.756 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:00.757 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.6 2023-11-06T03:13:00.757 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:00.757 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:00.758 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:00.758 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:00.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.825 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124941 s, 83.9 MB/s 2023-11-06T03:13:00.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.839 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118358 s, 88.6 MB/s 2023-11-06T03:13:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110734 s, 94.7 MB/s 2023-11-06T03:13:00.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.868 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124803 s, 84.0 MB/s 2023-11-06T03:13:00.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.882 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011216 s, 93.5 MB/s 2023-11-06T03:13:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014134 s, 74.2 MB/s 2023-11-06T03:13:00.915 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.915 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.915 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133435 s, 78.6 MB/s 2023-11-06T03:13:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.933 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145382 s, 72.1 MB/s 2023-11-06T03:13:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.948 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125568 s, 83.5 MB/s 2023-11-06T03:13:00.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.962 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011682 s, 89.8 MB/s 2023-11-06T03:13:00.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.979 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013681 s, 76.6 MB/s 2023-11-06T03:13:00.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:00.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:00.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112091 s, 93.5 MB/s 2023-11-06T03:13:01.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013004 s, 80.6 MB/s 2023-11-06T03:13:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103271 s, 102 MB/s 2023-11-06T03:13:01.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.033 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840744 s, 125 MB/s 2023-11-06T03:13:01.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.046 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104423 s, 100 MB/s 2023-11-06T03:13:01.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110988 s, 94.5 MB/s 2023-11-06T03:13:01.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.073 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118669 s, 88.4 MB/s 2023-11-06T03:13:01.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.087 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110328 s, 95.0 MB/s 2023-11-06T03:13:01.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887805 s, 118 MB/s 2023-11-06T03:13:01.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.112 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115557 s, 90.7 MB/s 2023-11-06T03:13:01.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872465 s, 120 MB/s 2023-11-06T03:13:01.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113367 s, 92.5 MB/s 2023-11-06T03:13:01.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.151 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114671 s, 91.4 MB/s 2023-11-06T03:13:01.164 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.164 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.165 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116114 s, 90.3 MB/s 2023-11-06T03:13:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123999 s, 84.6 MB/s 2023-11-06T03:13:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113445 s, 92.4 MB/s 2023-11-06T03:13:01.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113231 s, 92.6 MB/s 2023-11-06T03:13:01.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.221 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117266 s, 89.4 MB/s 2023-11-06T03:13:01.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124421 s, 84.3 MB/s 2023-11-06T03:13:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110551 s, 94.9 MB/s 2023-11-06T03:13:01.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.265 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125227 s, 83.7 MB/s 2023-11-06T03:13:01.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:01.266 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:01.267 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:01.267 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.7 2023-11-06T03:13:01.267 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:01.268 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:01.268 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:01.268 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:01.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.331 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950545 s, 110 MB/s 2023-11-06T03:13:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119476 s, 87.8 MB/s 2023-11-06T03:13:01.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133119 s, 78.8 MB/s 2023-11-06T03:13:01.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.376 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115566 s, 90.7 MB/s 2023-11-06T03:13:01.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.392 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128948 s, 81.3 MB/s 2023-11-06T03:13:01.404 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.405 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984728 s, 106 MB/s 2023-11-06T03:13:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116511 s, 90.0 MB/s 2023-11-06T03:13:01.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.431 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873906 s, 120 MB/s 2023-11-06T03:13:01.447 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.447 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141015 s, 74.4 MB/s 2023-11-06T03:13:01.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137531 s, 76.2 MB/s 2023-11-06T03:13:01.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.479 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124696 s, 84.1 MB/s 2023-11-06T03:13:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108454 s, 96.7 MB/s 2023-11-06T03:13:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118696 s, 88.3 MB/s 2023-11-06T03:13:01.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.519 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880313 s, 119 MB/s 2023-11-06T03:13:01.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134832 s, 77.8 MB/s 2023-11-06T03:13:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882435 s, 119 MB/s 2023-11-06T03:13:01.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.560 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011646 s, 90.0 MB/s 2023-11-06T03:13:01.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115081 s, 91.1 MB/s 2023-11-06T03:13:01.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.585 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877342 s, 120 MB/s 2023-11-06T03:13:01.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144656 s, 72.5 MB/s 2023-11-06T03:13:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.616 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.617 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011312 s, 92.7 MB/s 2023-11-06T03:13:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120557 s, 87.0 MB/s 2023-11-06T03:13:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.646 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120077 s, 87.3 MB/s 2023-11-06T03:13:01.663 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.663 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.663 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137294 s, 76.4 MB/s 2023-11-06T03:13:01.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914108 s, 115 MB/s 2023-11-06T03:13:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968219 s, 108 MB/s 2023-11-06T03:13:01.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889302 s, 118 MB/s 2023-11-06T03:13:01.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121943 s, 86.0 MB/s 2023-11-06T03:13:01.727 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110532 s, 94.9 MB/s 2023-11-06T03:13:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100148 s, 105 MB/s 2023-11-06T03:13:01.753 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.754 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106567 s, 98.4 MB/s 2023-11-06T03:13:01.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135195 s, 77.6 MB/s 2023-11-06T03:13:01.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:01.772 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:01.772 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:01.773 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.8 2023-11-06T03:13:01.773 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:01.773 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:01.774 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:01.774 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:01.838 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.838 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.838 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118156 s, 88.7 MB/s 2023-11-06T03:13:01.852 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113218 s, 92.6 MB/s 2023-11-06T03:13:01.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.868 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118015 s, 88.9 MB/s 2023-11-06T03:13:01.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.884 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.884 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127134 s, 82.5 MB/s 2023-11-06T03:13:01.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.900 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124672 s, 84.1 MB/s 2023-11-06T03:13:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.915 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.916 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123528 s, 84.9 MB/s 2023-11-06T03:13:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.931 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120728 s, 86.9 MB/s 2023-11-06T03:13:01.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112719 s, 93.0 MB/s 2023-11-06T03:13:01.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114407 s, 91.7 MB/s 2023-11-06T03:13:01.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125131 s, 83.8 MB/s 2023-11-06T03:13:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:01.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112155 s, 93.5 MB/s 2023-11-06T03:13:02.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.007 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123652 s, 84.8 MB/s 2023-11-06T03:13:02.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.020 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961831 s, 109 MB/s 2023-11-06T03:13:02.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.035 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110537 s, 94.9 MB/s 2023-11-06T03:13:02.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929326 s, 113 MB/s 2023-11-06T03:13:02.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125493 s, 83.6 MB/s 2023-11-06T03:13:02.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.079 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122013 s, 85.9 MB/s 2023-11-06T03:13:02.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.094 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119508 s, 87.7 MB/s 2023-11-06T03:13:02.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.108 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111699 s, 93.9 MB/s 2023-11-06T03:13:02.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110977 s, 94.5 MB/s 2023-11-06T03:13:02.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.138 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118079 s, 88.8 MB/s 2023-11-06T03:13:02.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.150 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094535 s, 111 MB/s 2023-11-06T03:13:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.165 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117499 s, 89.2 MB/s 2023-11-06T03:13:02.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.181 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125676 s, 83.4 MB/s 2023-11-06T03:13:02.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115001 s, 91.2 MB/s 2023-11-06T03:13:02.211 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.211 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012104 s, 86.6 MB/s 2023-11-06T03:13:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.227 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127997 s, 81.9 MB/s 2023-11-06T03:13:02.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135917 s, 77.1 MB/s 2023-11-06T03:13:02.258 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.258 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.259 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120285 s, 87.2 MB/s 2023-11-06T03:13:02.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107412 s, 97.6 MB/s 2023-11-06T03:13:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122093 s, 85.9 MB/s 2023-11-06T03:13:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01227 s, 85.5 MB/s 2023-11-06T03:13:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:02.304 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:02.304 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:02.305 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.9 2023-11-06T03:13:02.305 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:02.305 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:02.306 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:02.306 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:02.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.376 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011942 s, 87.8 MB/s 2023-11-06T03:13:02.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141219 s, 74.3 MB/s 2023-11-06T03:13:02.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124564 s, 84.2 MB/s 2023-11-06T03:13:02.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113981 s, 92.0 MB/s 2023-11-06T03:13:02.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128957 s, 81.3 MB/s 2023-11-06T03:13:02.454 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.455 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123492 s, 84.9 MB/s 2023-11-06T03:13:02.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.468 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113025 s, 92.8 MB/s 2023-11-06T03:13:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135739 s, 77.2 MB/s 2023-11-06T03:13:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131215 s, 79.9 MB/s 2023-11-06T03:13:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122038 s, 85.9 MB/s 2023-11-06T03:13:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127158 s, 82.5 MB/s 2023-11-06T03:13:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930433 s, 113 MB/s 2023-11-06T03:13:02.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.559 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127046 s, 82.5 MB/s 2023-11-06T03:13:02.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010825 s, 96.9 MB/s 2023-11-06T03:13:02.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.586 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106131 s, 98.8 MB/s 2023-11-06T03:13:02.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.601 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118122 s, 88.8 MB/s 2023-11-06T03:13:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138622 s, 75.6 MB/s 2023-11-06T03:13:02.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.631 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111033 s, 94.4 MB/s 2023-11-06T03:13:02.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.645 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120192 s, 87.2 MB/s 2023-11-06T03:13:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934326 s, 112 MB/s 2023-11-06T03:13:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134847 s, 77.8 MB/s 2023-11-06T03:13:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112133 s, 93.5 MB/s 2023-11-06T03:13:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010279 s, 102 MB/s 2023-11-06T03:13:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112746 s, 93.0 MB/s 2023-11-06T03:13:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012485 s, 84.0 MB/s 2023-11-06T03:13:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133878 s, 78.3 MB/s 2023-11-06T03:13:02.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.760 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011511 s, 91.1 MB/s 2023-11-06T03:13:02.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133932 s, 78.3 MB/s 2023-11-06T03:13:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146534 s, 71.6 MB/s 2023-11-06T03:13:02.809 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.810 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124033 s, 84.5 MB/s 2023-11-06T03:13:02.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.825 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127065 s, 82.5 MB/s 2023-11-06T03:13:02.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134052 s, 78.2 MB/s 2023-11-06T03:13:02.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:02.843 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:02.843 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:02.843 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.10 2023-11-06T03:13:02.844 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:02.844 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:02.844 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:02.845 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:02.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864347 s, 121 MB/s 2023-11-06T03:13:02.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.919 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113852 s, 92.1 MB/s 2023-11-06T03:13:02.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.934 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132222 s, 79.3 MB/s 2023-11-06T03:13:02.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.948 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107034 s, 98.0 MB/s 2023-11-06T03:13:02.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.962 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011331 s, 92.5 MB/s 2023-11-06T03:13:02.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.972 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835752 s, 125 MB/s 2023-11-06T03:13:02.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.983 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863621 s, 121 MB/s 2023-11-06T03:13:02.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:02.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:02.995 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103362 s, 101 MB/s 2023-11-06T03:13:03.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104767 s, 100 MB/s 2023-11-06T03:13:03.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120037 s, 87.4 MB/s 2023-11-06T03:13:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.036 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109796 s, 95.5 MB/s 2023-11-06T03:13:03.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799122 s, 131 MB/s 2023-11-06T03:13:03.061 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.061 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.062 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131826 s, 79.5 MB/s 2023-11-06T03:13:03.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.075 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105668 s, 99.2 MB/s 2023-11-06T03:13:03.090 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.091 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139021 s, 75.4 MB/s 2023-11-06T03:13:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.104 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.105 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114882 s, 91.3 MB/s 2023-11-06T03:13:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.119 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121935 s, 86.0 MB/s 2023-11-06T03:13:03.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123177 s, 85.1 MB/s 2023-11-06T03:13:03.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.149 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127023 s, 82.6 MB/s 2023-11-06T03:13:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824081 s, 127 MB/s 2023-11-06T03:13:03.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130194 s, 80.5 MB/s 2023-11-06T03:13:03.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.189 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116611 s, 89.9 MB/s 2023-11-06T03:13:03.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965611 s, 109 MB/s 2023-11-06T03:13:03.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.213 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943791 s, 111 MB/s 2023-11-06T03:13:03.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.227 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116249 s, 90.2 MB/s 2023-11-06T03:13:03.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.242 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129823 s, 80.8 MB/s 2023-11-06T03:13:03.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.257 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127234 s, 82.4 MB/s 2023-11-06T03:13:03.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.272 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117821 s, 89.0 MB/s 2023-11-06T03:13:03.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.288 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134555 s, 77.9 MB/s 2023-11-06T03:13:03.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.304 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125732 s, 83.4 MB/s 2023-11-06T03:13:03.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.319 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122798 s, 85.4 MB/s 2023-11-06T03:13:03.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.335 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129402 s, 81.0 MB/s 2023-11-06T03:13:03.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:03.336 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:03.337 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:03.337 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.11 2023-11-06T03:13:03.337 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:03.337 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:03.338 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:03.338 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137809 s, 76.1 MB/s 2023-11-06T03:13:03.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.426 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103478 s, 101 MB/s 2023-11-06T03:13:03.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107223 s, 97.8 MB/s 2023-11-06T03:13:03.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.456 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129546 s, 80.9 MB/s 2023-11-06T03:13:03.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126855 s, 82.7 MB/s 2023-11-06T03:13:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.486 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122841 s, 85.4 MB/s 2023-11-06T03:13:03.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.503 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144507 s, 72.6 MB/s 2023-11-06T03:13:03.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116901 s, 89.7 MB/s 2023-11-06T03:13:03.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111042 s, 94.4 MB/s 2023-11-06T03:13:03.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013987 s, 75.0 MB/s 2023-11-06T03:13:03.565 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.566 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145224 s, 72.2 MB/s 2023-11-06T03:13:03.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.582 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126781 s, 82.7 MB/s 2023-11-06T03:13:03.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.598 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135379 s, 77.5 MB/s 2023-11-06T03:13:03.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.632 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976497 s, 107 MB/s 2023-11-06T03:13:03.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.647 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124079 s, 84.5 MB/s 2023-11-06T03:13:03.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.662 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122381 s, 85.7 MB/s 2023-11-06T03:13:03.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145793 s, 71.9 MB/s 2023-11-06T03:13:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121154 s, 86.5 MB/s 2023-11-06T03:13:03.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.711 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137976 s, 76.0 MB/s 2023-11-06T03:13:03.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.727 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.727 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012675 s, 82.7 MB/s 2023-11-06T03:13:03.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127428 s, 82.3 MB/s 2023-11-06T03:13:03.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012665 s, 82.8 MB/s 2023-11-06T03:13:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135033 s, 77.7 MB/s 2023-11-06T03:13:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.790 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012982 s, 80.8 MB/s 2023-11-06T03:13:03.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.806 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130717 s, 80.2 MB/s 2023-11-06T03:13:03.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.824 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.824 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142393 s, 73.6 MB/s 2023-11-06T03:13:03.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.839 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126043 s, 83.2 MB/s 2023-11-06T03:13:03.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128796 s, 81.4 MB/s 2023-11-06T03:13:03.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.872 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132332 s, 79.2 MB/s 2023-11-06T03:13:03.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.888 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130745 s, 80.2 MB/s 2023-11-06T03:13:03.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112062 s, 93.6 MB/s 2023-11-06T03:13:03.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125827 s, 83.3 MB/s 2023-11-06T03:13:03.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:03.918 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:03.919 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:03.919 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.12 2023-11-06T03:13:03.919 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:03.920 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:03.920 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:03.920 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:03.987 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012822 s, 81.8 MB/s 2023-11-06T03:13:04.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.003 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.003 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125078 s, 83.8 MB/s 2023-11-06T03:13:04.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.016 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101069 s, 104 MB/s 2023-11-06T03:13:04.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.031 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128653 s, 81.5 MB/s 2023-11-06T03:13:04.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134294 s, 78.1 MB/s 2023-11-06T03:13:04.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.064 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131493 s, 79.7 MB/s 2023-11-06T03:13:04.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.078 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106128 s, 98.8 MB/s 2023-11-06T03:13:04.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.092 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115885 s, 90.5 MB/s 2023-11-06T03:13:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144246 s, 72.7 MB/s 2023-11-06T03:13:04.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131742 s, 79.6 MB/s 2023-11-06T03:13:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123227 s, 85.1 MB/s 2023-11-06T03:13:04.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129296 s, 81.1 MB/s 2023-11-06T03:13:04.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010873 s, 96.4 MB/s 2023-11-06T03:13:04.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.189 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130839 s, 80.1 MB/s 2023-11-06T03:13:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122757 s, 85.4 MB/s 2023-11-06T03:13:04.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.218 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108821 s, 96.4 MB/s 2023-11-06T03:13:04.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.233 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122253 s, 85.8 MB/s 2023-11-06T03:13:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.250 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140529 s, 74.6 MB/s 2023-11-06T03:13:04.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.266 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133005 s, 78.8 MB/s 2023-11-06T03:13:04.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130683 s, 80.2 MB/s 2023-11-06T03:13:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109548 s, 95.7 MB/s 2023-11-06T03:13:04.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.311 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133165 s, 78.7 MB/s 2023-11-06T03:13:04.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.328 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138193 s, 75.9 MB/s 2023-11-06T03:13:04.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.344 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132293 s, 79.3 MB/s 2023-11-06T03:13:04.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137272 s, 76.4 MB/s 2023-11-06T03:13:04.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.378 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127476 s, 82.3 MB/s 2023-11-06T03:13:04.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.389 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909811 s, 115 MB/s 2023-11-06T03:13:04.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.405 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126147 s, 83.1 MB/s 2023-11-06T03:13:04.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105786 s, 99.1 MB/s 2023-11-06T03:13:04.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115911 s, 90.5 MB/s 2023-11-06T03:13:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129186 s, 81.2 MB/s 2023-11-06T03:13:04.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955069 s, 110 MB/s 2023-11-06T03:13:04.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:04.462 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:04.462 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:04.463 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.13 2023-11-06T03:13:04.463 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:04.463 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:04.464 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:04.464 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:04.527 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.528 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.528 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116781 s, 89.8 MB/s 2023-11-06T03:13:04.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.542 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117598 s, 89.2 MB/s 2023-11-06T03:13:04.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.559 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130023 s, 80.6 MB/s 2023-11-06T03:13:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121839 s, 86.1 MB/s 2023-11-06T03:13:04.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010578 s, 99.1 MB/s 2023-11-06T03:13:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127515 s, 82.2 MB/s 2023-11-06T03:13:04.635 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.636 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135309 s, 77.5 MB/s 2023-11-06T03:13:04.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138267 s, 75.8 MB/s 2023-11-06T03:13:04.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.666 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112154 s, 93.5 MB/s 2023-11-06T03:13:04.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.678 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939376 s, 112 MB/s 2023-11-06T03:13:04.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.691 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104304 s, 101 MB/s 2023-11-06T03:13:04.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.702 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902164 s, 116 MB/s 2023-11-06T03:13:04.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143926 s, 72.9 MB/s 2023-11-06T03:13:04.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121288 s, 86.5 MB/s 2023-11-06T03:13:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012178 s, 86.1 MB/s 2023-11-06T03:13:04.764 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.765 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119711 s, 87.6 MB/s 2023-11-06T03:13:04.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013915 s, 75.4 MB/s 2023-11-06T03:13:04.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.796 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117264 s, 89.4 MB/s 2023-11-06T03:13:04.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122962 s, 85.3 MB/s 2023-11-06T03:13:04.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.827 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140653 s, 74.6 MB/s 2023-11-06T03:13:04.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.842 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120099 s, 87.3 MB/s 2023-11-06T03:13:04.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124473 s, 84.2 MB/s 2023-11-06T03:13:04.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.872 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111243 s, 94.3 MB/s 2023-11-06T03:13:04.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.888 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135966 s, 77.1 MB/s 2023-11-06T03:13:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119095 s, 88.0 MB/s 2023-11-06T03:13:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122373 s, 85.7 MB/s 2023-11-06T03:13:04.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.932 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115082 s, 91.1 MB/s 2023-11-06T03:13:04.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113344 s, 92.5 MB/s 2023-11-06T03:13:04.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119022 s, 88.1 MB/s 2023-11-06T03:13:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.971 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791001 s, 133 MB/s 2023-11-06T03:13:04.980 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.980 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.980 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749633 s, 140 MB/s 2023-11-06T03:13:04.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:04.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:04.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996486 s, 105 MB/s 2023-11-06T03:13:04.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:04.994 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:04.994 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:04.994 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.14 2023-11-06T03:13:04.995 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:04.995 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:04.996 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:04.996 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:05.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117549 s, 89.2 MB/s 2023-11-06T03:13:05.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.068 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759678 s, 138 MB/s 2023-11-06T03:13:05.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.081 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108359 s, 96.8 MB/s 2023-11-06T03:13:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105716 s, 99.2 MB/s 2023-11-06T03:13:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106294 s, 98.6 MB/s 2023-11-06T03:13:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133099 s, 78.8 MB/s 2023-11-06T03:13:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988167 s, 106 MB/s 2023-11-06T03:13:05.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.149 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127601 s, 82.2 MB/s 2023-11-06T03:13:05.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.161 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099556 s, 105 MB/s 2023-11-06T03:13:05.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130261 s, 80.5 MB/s 2023-11-06T03:13:05.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102924 s, 102 MB/s 2023-11-06T03:13:05.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128065 s, 81.9 MB/s 2023-11-06T03:13:05.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.218 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124952 s, 83.9 MB/s 2023-11-06T03:13:05.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125251 s, 83.7 MB/s 2023-11-06T03:13:05.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123457 s, 84.9 MB/s 2023-11-06T03:13:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103523 s, 101 MB/s 2023-11-06T03:13:05.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.273 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120564 s, 87.0 MB/s 2023-11-06T03:13:05.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.287 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112595 s, 93.1 MB/s 2023-11-06T03:13:05.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.299 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102611 s, 102 MB/s 2023-11-06T03:13:05.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.313 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122798 s, 85.4 MB/s 2023-11-06T03:13:05.325 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.325 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.326 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103933 s, 101 MB/s 2023-11-06T03:13:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103502 s, 101 MB/s 2023-11-06T03:13:05.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078036 s, 134 MB/s 2023-11-06T03:13:05.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.360 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102399 s, 102 MB/s 2023-11-06T03:13:05.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.375 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128346 s, 81.7 MB/s 2023-11-06T03:13:05.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.387 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100757 s, 104 MB/s 2023-11-06T03:13:05.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.400 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102168 s, 103 MB/s 2023-11-06T03:13:05.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.413 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110247 s, 95.1 MB/s 2023-11-06T03:13:05.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.428 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118155 s, 88.7 MB/s 2023-11-06T03:13:05.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.440 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980121 s, 107 MB/s 2023-11-06T03:13:05.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.453 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108079 s, 97.0 MB/s 2023-11-06T03:13:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.466 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105486 s, 99.4 MB/s 2023-11-06T03:13:05.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:05.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:05.468 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:05.468 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.15 2023-11-06T03:13:05.469 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:05.469 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:05.469 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:05.470 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:05.528 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.529 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918618 s, 114 MB/s 2023-11-06T03:13:05.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.543 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117224 s, 89.5 MB/s 2023-11-06T03:13:05.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112322 s, 93.4 MB/s 2023-11-06T03:13:05.568 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.569 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104103 s, 101 MB/s 2023-11-06T03:13:05.579 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.579 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.579 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079223 s, 132 MB/s 2023-11-06T03:13:05.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114996 s, 91.2 MB/s 2023-11-06T03:13:05.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.606 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106624 s, 98.3 MB/s 2023-11-06T03:13:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989379 s, 106 MB/s 2023-11-06T03:13:05.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111944 s, 93.7 MB/s 2023-11-06T03:13:05.666 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.667 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012079 s, 86.8 MB/s 2023-11-06T03:13:05.682 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.683 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137788 s, 76.1 MB/s 2023-11-06T03:13:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117229 s, 89.4 MB/s 2023-11-06T03:13:05.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108374 s, 96.8 MB/s 2023-11-06T03:13:05.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117588 s, 89.2 MB/s 2023-11-06T03:13:05.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.735 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821053 s, 128 MB/s 2023-11-06T03:13:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899139 s, 117 MB/s 2023-11-06T03:13:05.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010199 s, 103 MB/s 2023-11-06T03:13:05.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.772 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.772 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109626 s, 95.6 MB/s 2023-11-06T03:13:05.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883623 s, 119 MB/s 2023-11-06T03:13:05.797 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.798 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107458 s, 97.6 MB/s 2023-11-06T03:13:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.812 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121262 s, 86.5 MB/s 2023-11-06T03:13:05.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.823 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858769 s, 122 MB/s 2023-11-06T03:13:05.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.837 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113009 s, 92.8 MB/s 2023-11-06T03:13:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.848 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872988 s, 120 MB/s 2023-11-06T03:13:05.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.864 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130088 s, 80.6 MB/s 2023-11-06T03:13:05.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.878 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115828 s, 90.5 MB/s 2023-11-06T03:13:05.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.891 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107742 s, 97.3 MB/s 2023-11-06T03:13:05.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.907 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131254 s, 79.9 MB/s 2023-11-06T03:13:05.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.921 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116279 s, 90.2 MB/s 2023-11-06T03:13:05.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.935 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116082 s, 90.3 MB/s 2023-11-06T03:13:05.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.949 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.949 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010982 s, 95.5 MB/s 2023-11-06T03:13:05.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:05.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:05.964 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012521 s, 83.7 MB/s 2023-11-06T03:13:05.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:05.965 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:05.965 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:05.966 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.16 2023-11-06T03:13:05.966 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:05.966 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:05.967 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:05.967 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:06.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.093 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013873 s, 75.6 MB/s 2023-11-06T03:13:06.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105626 s, 99.3 MB/s 2023-11-06T03:13:06.120 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.120 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.120 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112783 s, 93.0 MB/s 2023-11-06T03:13:06.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110841 s, 94.6 MB/s 2023-11-06T03:13:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133548 s, 78.5 MB/s 2023-11-06T03:13:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101273 s, 104 MB/s 2023-11-06T03:13:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.177 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119292 s, 87.9 MB/s 2023-11-06T03:13:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.192 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119114 s, 88.0 MB/s 2023-11-06T03:13:06.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.206 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114716 s, 91.4 MB/s 2023-11-06T03:13:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126381 s, 83.0 MB/s 2023-11-06T03:13:06.239 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.239 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.240 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138494 s, 75.7 MB/s 2023-11-06T03:13:06.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117266 s, 89.4 MB/s 2023-11-06T03:13:06.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.273 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130644 s, 80.3 MB/s 2023-11-06T03:13:06.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.291 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01306 s, 80.3 MB/s 2023-11-06T03:13:06.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.306 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119385 s, 87.8 MB/s 2023-11-06T03:13:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118025 s, 88.8 MB/s 2023-11-06T03:13:06.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.335 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011756 s, 89.2 MB/s 2023-11-06T03:13:06.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.349 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121291 s, 86.5 MB/s 2023-11-06T03:13:06.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.361 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889906 s, 118 MB/s 2023-11-06T03:13:06.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.373 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933205 s, 112 MB/s 2023-11-06T03:13:06.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.387 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115414 s, 90.9 MB/s 2023-11-06T03:13:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.402 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123514 s, 84.9 MB/s 2023-11-06T03:13:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.417 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116699 s, 89.9 MB/s 2023-11-06T03:13:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.431 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112356 s, 93.3 MB/s 2023-11-06T03:13:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.445 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116414 s, 90.1 MB/s 2023-11-06T03:13:06.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115863 s, 90.5 MB/s 2023-11-06T03:13:06.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135451 s, 77.4 MB/s 2023-11-06T03:13:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137491 s, 76.3 MB/s 2023-11-06T03:13:06.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.505 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916675 s, 114 MB/s 2023-11-06T03:13:06.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115554 s, 90.7 MB/s 2023-11-06T03:13:06.532 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.532 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.532 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968151 s, 108 MB/s 2023-11-06T03:13:06.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132582 s, 79.1 MB/s 2023-11-06T03:13:06.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:06.550 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:06.550 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:06.551 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.17 2023-11-06T03:13:06.551 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:06.551 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:06.552 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:06.552 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:06.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.574 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084863 s, 124 MB/s 2023-11-06T03:13:06.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.586 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943254 s, 111 MB/s 2023-11-06T03:13:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.601 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120958 s, 86.7 MB/s 2023-11-06T03:13:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011737 s, 89.3 MB/s 2023-11-06T03:13:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944889 s, 111 MB/s 2023-11-06T03:13:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879417 s, 119 MB/s 2023-11-06T03:13:06.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.686 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120137 s, 87.3 MB/s 2023-11-06T03:13:06.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.700 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112376 s, 93.3 MB/s 2023-11-06T03:13:06.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115672 s, 90.7 MB/s 2023-11-06T03:13:06.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.730 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116054 s, 90.4 MB/s 2023-11-06T03:13:06.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110717 s, 94.7 MB/s 2023-11-06T03:13:06.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127415 s, 82.3 MB/s 2023-11-06T03:13:06.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123553 s, 84.9 MB/s 2023-11-06T03:13:06.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.790 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126896 s, 82.6 MB/s 2023-11-06T03:13:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137892 s, 76.0 MB/s 2023-11-06T03:13:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.822 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136465 s, 76.8 MB/s 2023-11-06T03:13:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.835 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102763 s, 102 MB/s 2023-11-06T03:13:06.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.852 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.852 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137783 s, 76.1 MB/s 2023-11-06T03:13:06.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100832 s, 104 MB/s 2023-11-06T03:13:06.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.880 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129385 s, 81.0 MB/s 2023-11-06T03:13:06.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.893 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968594 s, 108 MB/s 2023-11-06T03:13:06.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.909 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131846 s, 79.5 MB/s 2023-11-06T03:13:06.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111665 s, 93.9 MB/s 2023-11-06T03:13:06.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101343 s, 103 MB/s 2023-11-06T03:13:06.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.950 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116721 s, 89.8 MB/s 2023-11-06T03:13:06.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.968 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148091 s, 70.8 MB/s 2023-11-06T03:13:06.982 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.982 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:06.982 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109351 s, 95.9 MB/s 2023-11-06T03:13:06.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:06.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140139 s, 74.8 MB/s 2023-11-06T03:13:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.015 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127218 s, 82.4 MB/s 2023-11-06T03:13:07.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.031 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126578 s, 82.8 MB/s 2023-11-06T03:13:07.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142908 s, 73.4 MB/s 2023-11-06T03:13:07.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.066 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150914 s, 69.5 MB/s 2023-11-06T03:13:07.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:07.068 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:07.068 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:07.069 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.18 2023-11-06T03:13:07.069 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:07.069 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:07.070 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:07.070 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:07.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129941 s, 80.7 MB/s 2023-11-06T03:13:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.151 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132381 s, 79.2 MB/s 2023-11-06T03:13:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.166 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131342 s, 79.8 MB/s 2023-11-06T03:13:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.182 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130404 s, 80.4 MB/s 2023-11-06T03:13:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.198 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130421 s, 80.4 MB/s 2023-11-06T03:13:07.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.216 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149386 s, 70.2 MB/s 2023-11-06T03:13:07.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.231 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118087 s, 88.8 MB/s 2023-11-06T03:13:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119734 s, 87.6 MB/s 2023-11-06T03:13:07.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.264 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013823 s, 75.9 MB/s 2023-11-06T03:13:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.279 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012883 s, 81.4 MB/s 2023-11-06T03:13:07.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.294 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120449 s, 87.1 MB/s 2023-11-06T03:13:07.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147136 s, 71.3 MB/s 2023-11-06T03:13:07.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163476 s, 64.1 MB/s 2023-11-06T03:13:07.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.350 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148147 s, 70.8 MB/s 2023-11-06T03:13:07.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146102 s, 71.8 MB/s 2023-11-06T03:13:07.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.385 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137428 s, 76.3 MB/s 2023-11-06T03:13:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.401 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127597 s, 82.2 MB/s 2023-11-06T03:13:07.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147544 s, 71.1 MB/s 2023-11-06T03:13:07.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137861 s, 76.1 MB/s 2023-11-06T03:13:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132515 s, 79.1 MB/s 2023-11-06T03:13:07.467 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.467 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.467 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121554 s, 86.3 MB/s 2023-11-06T03:13:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.485 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140901 s, 74.4 MB/s 2023-11-06T03:13:07.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136483 s, 76.8 MB/s 2023-11-06T03:13:07.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.519 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014551 s, 72.1 MB/s 2023-11-06T03:13:07.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136711 s, 76.7 MB/s 2023-11-06T03:13:07.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.552 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139085 s, 75.4 MB/s 2023-11-06T03:13:07.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.566 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108617 s, 96.5 MB/s 2023-11-06T03:13:07.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135446 s, 77.4 MB/s 2023-11-06T03:13:07.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.598 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127057 s, 82.5 MB/s 2023-11-06T03:13:07.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01412 s, 74.3 MB/s 2023-11-06T03:13:07.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.654 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125346 s, 83.7 MB/s 2023-11-06T03:13:07.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139529 s, 75.2 MB/s 2023-11-06T03:13:07.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:07.673 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:07.673 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:07.674 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.19 2023-11-06T03:13:07.674 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:07.675 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:07.675 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:07.675 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:07.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117754 s, 89.0 MB/s 2023-11-06T03:13:07.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.756 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994171 s, 105 MB/s 2023-11-06T03:13:07.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011438 s, 91.7 MB/s 2023-11-06T03:13:07.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.786 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120976 s, 86.7 MB/s 2023-11-06T03:13:07.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.800 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116771 s, 89.8 MB/s 2023-11-06T03:13:07.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115639 s, 90.7 MB/s 2023-11-06T03:13:07.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.829 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114361 s, 91.7 MB/s 2023-11-06T03:13:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114494 s, 91.6 MB/s 2023-11-06T03:13:07.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112362 s, 93.3 MB/s 2023-11-06T03:13:07.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011265 s, 93.1 MB/s 2023-11-06T03:13:07.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.886 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118436 s, 88.5 MB/s 2023-11-06T03:13:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124014 s, 84.6 MB/s 2023-11-06T03:13:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100061 s, 105 MB/s 2023-11-06T03:13:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.925 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862684 s, 122 MB/s 2023-11-06T03:13:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.937 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975512 s, 107 MB/s 2023-11-06T03:13:07.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.951 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114678 s, 91.4 MB/s 2023-11-06T03:13:07.965 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.966 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.966 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120824 s, 86.8 MB/s 2023-11-06T03:13:07.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.980 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.980 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111894 s, 93.7 MB/s 2023-11-06T03:13:07.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:07.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:07.995 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124291 s, 84.4 MB/s 2023-11-06T03:13:08.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110276 s, 95.1 MB/s 2023-11-06T03:13:08.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.023 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938072 s, 112 MB/s 2023-11-06T03:13:08.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.037 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104046 s, 101 MB/s 2023-11-06T03:13:08.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141758 s, 74.0 MB/s 2023-11-06T03:13:08.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121886 s, 86.0 MB/s 2023-11-06T03:13:08.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.086 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124304 s, 84.4 MB/s 2023-11-06T03:13:08.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.100 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940459 s, 111 MB/s 2023-11-06T03:13:08.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121202 s, 86.5 MB/s 2023-11-06T03:13:08.129 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.129 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.130 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108682 s, 96.5 MB/s 2023-11-06T03:13:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.143 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113601 s, 92.3 MB/s 2023-11-06T03:13:08.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123102 s, 85.2 MB/s 2023-11-06T03:13:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.174 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119169 s, 88.0 MB/s 2023-11-06T03:13:08.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.191 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013659 s, 76.8 MB/s 2023-11-06T03:13:08.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:08.192 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:08.192 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:08.193 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.20 2023-11-06T03:13:08.193 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:08.193 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:08.194 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:08.194 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:08.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.261 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112394 s, 93.3 MB/s 2023-11-06T03:13:08.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.275 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100809 s, 104 MB/s 2023-11-06T03:13:08.289 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.289 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120648 s, 86.9 MB/s 2023-11-06T03:13:08.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.307 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145809 s, 71.9 MB/s 2023-11-06T03:13:08.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125117 s, 83.8 MB/s 2023-11-06T03:13:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121557 s, 86.3 MB/s 2023-11-06T03:13:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011587 s, 90.5 MB/s 2023-11-06T03:13:08.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118639 s, 88.4 MB/s 2023-11-06T03:13:08.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.383 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123962 s, 84.6 MB/s 2023-11-06T03:13:08.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.399 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130593 s, 80.3 MB/s 2023-11-06T03:13:08.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.414 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116816 s, 89.8 MB/s 2023-11-06T03:13:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.429 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119958 s, 87.4 MB/s 2023-11-06T03:13:08.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133367 s, 78.6 MB/s 2023-11-06T03:13:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109224 s, 96.0 MB/s 2023-11-06T03:13:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157412 s, 66.6 MB/s 2023-11-06T03:13:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.495 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136895 s, 76.6 MB/s 2023-11-06T03:13:08.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.509 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011112 s, 94.4 MB/s 2023-11-06T03:13:08.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.525 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127261 s, 82.4 MB/s 2023-11-06T03:13:08.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.539 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107836 s, 97.2 MB/s 2023-11-06T03:13:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011304 s, 92.8 MB/s 2023-11-06T03:13:08.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133429 s, 78.6 MB/s 2023-11-06T03:13:08.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146893 s, 71.4 MB/s 2023-11-06T03:13:08.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013324 s, 78.7 MB/s 2023-11-06T03:13:08.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132892 s, 78.9 MB/s 2023-11-06T03:13:08.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110915 s, 94.5 MB/s 2023-11-06T03:13:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120508 s, 87.0 MB/s 2023-11-06T03:13:08.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.688 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101128 s, 104 MB/s 2023-11-06T03:13:08.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121088 s, 86.6 MB/s 2023-11-06T03:13:08.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124976 s, 83.9 MB/s 2023-11-06T03:13:08.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129705 s, 80.8 MB/s 2023-11-06T03:13:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125471 s, 83.6 MB/s 2023-11-06T03:13:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.762 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102239 s, 103 MB/s 2023-11-06T03:13:08.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:08.764 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:08.764 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:08.765 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.21 2023-11-06T03:13:08.765 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:08.765 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:08.766 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:08.766 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:08.828 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.829 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117766 s, 89.0 MB/s 2023-11-06T03:13:08.844 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.844 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.845 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131484 s, 79.7 MB/s 2023-11-06T03:13:08.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.856 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916191 s, 114 MB/s 2023-11-06T03:13:08.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.871 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.871 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114885 s, 91.3 MB/s 2023-11-06T03:13:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966129 s, 109 MB/s 2023-11-06T03:13:08.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.898 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.899 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122978 s, 85.3 MB/s 2023-11-06T03:13:08.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135157 s, 77.6 MB/s 2023-11-06T03:13:08.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.931 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132351 s, 79.2 MB/s 2023-11-06T03:13:08.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.947 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131534 s, 79.7 MB/s 2023-11-06T03:13:08.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.962 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120639 s, 86.9 MB/s 2023-11-06T03:13:08.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132613 s, 79.1 MB/s 2023-11-06T03:13:08.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:08.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:08.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117721 s, 89.1 MB/s 2023-11-06T03:13:09.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.007 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114661 s, 91.5 MB/s 2023-11-06T03:13:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129423 s, 81.0 MB/s 2023-11-06T03:13:09.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117384 s, 89.3 MB/s 2023-11-06T03:13:09.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.053 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130699 s, 80.2 MB/s 2023-11-06T03:13:09.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113202 s, 92.6 MB/s 2023-11-06T03:13:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133845 s, 78.3 MB/s 2023-11-06T03:13:09.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121049 s, 86.6 MB/s 2023-11-06T03:13:09.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.114 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128288 s, 81.7 MB/s 2023-11-06T03:13:09.128 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.128 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.128 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105443 s, 99.4 MB/s 2023-11-06T03:13:09.145 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.145 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.145 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134452 s, 78.0 MB/s 2023-11-06T03:13:09.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126292 s, 83.0 MB/s 2023-11-06T03:13:09.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.177 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.177 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012161 s, 86.2 MB/s 2023-11-06T03:13:09.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.194 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013847 s, 75.7 MB/s 2023-11-06T03:13:09.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.209 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119747 s, 87.6 MB/s 2023-11-06T03:13:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110129 s, 95.2 MB/s 2023-11-06T03:13:09.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.237 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118101 s, 88.8 MB/s 2023-11-06T03:13:09.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.253 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128835 s, 81.4 MB/s 2023-11-06T03:13:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124141 s, 84.5 MB/s 2023-11-06T03:13:09.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102649 s, 102 MB/s 2023-11-06T03:13:09.296 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119238 s, 87.9 MB/s 2023-11-06T03:13:09.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:09.298 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:09.298 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:09.299 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.22 2023-11-06T03:13:09.299 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:09.299 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:09.300 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:09.300 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:09.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.365 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125849 s, 83.3 MB/s 2023-11-06T03:13:09.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.380 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122 s, 85.9 MB/s 2023-11-06T03:13:09.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.393 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998159 s, 105 MB/s 2023-11-06T03:13:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129784 s, 80.8 MB/s 2023-11-06T03:13:09.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090649 s, 116 MB/s 2023-11-06T03:13:09.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920588 s, 114 MB/s 2023-11-06T03:13:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118233 s, 88.7 MB/s 2023-11-06T03:13:09.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.463 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120382 s, 87.1 MB/s 2023-11-06T03:13:09.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.479 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130117 s, 80.6 MB/s 2023-11-06T03:13:09.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.492 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105509 s, 99.4 MB/s 2023-11-06T03:13:09.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.509 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122708 s, 85.5 MB/s 2023-11-06T03:13:09.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.522 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123749 s, 84.7 MB/s 2023-11-06T03:13:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.539 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014202 s, 73.8 MB/s 2023-11-06T03:13:09.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.554 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125245 s, 83.7 MB/s 2023-11-06T03:13:09.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.567 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.567 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100249 s, 105 MB/s 2023-11-06T03:13:09.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141921 s, 73.9 MB/s 2023-11-06T03:13:09.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.597 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.597 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103565 s, 101 MB/s 2023-11-06T03:13:09.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.611 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.611 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115538 s, 90.8 MB/s 2023-11-06T03:13:09.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.630 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016004 s, 65.5 MB/s 2023-11-06T03:13:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120276 s, 87.2 MB/s 2023-11-06T03:13:09.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127008 s, 82.6 MB/s 2023-11-06T03:13:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124317 s, 84.3 MB/s 2023-11-06T03:13:09.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01349 s, 77.7 MB/s 2023-11-06T03:13:09.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953112 s, 110 MB/s 2023-11-06T03:13:09.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.730 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117891 s, 88.9 MB/s 2023-11-06T03:13:09.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108755 s, 96.4 MB/s 2023-11-06T03:13:09.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112703 s, 93.0 MB/s 2023-11-06T03:13:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910605 s, 115 MB/s 2023-11-06T03:13:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013214 s, 79.4 MB/s 2023-11-06T03:13:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.796 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892852 s, 117 MB/s 2023-11-06T03:13:09.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121951 s, 86.0 MB/s 2023-11-06T03:13:09.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012573 s, 83.4 MB/s 2023-11-06T03:13:09.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:09.827 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:09.828 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:09.828 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.23 2023-11-06T03:13:09.829 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:09.829 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:09.829 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:09.830 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:09.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.895 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133557 s, 78.5 MB/s 2023-11-06T03:13:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130756 s, 80.2 MB/s 2023-11-06T03:13:09.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.924 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105552 s, 99.3 MB/s 2023-11-06T03:13:09.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.941 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013706 s, 76.5 MB/s 2023-11-06T03:13:09.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.958 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137588 s, 76.2 MB/s 2023-11-06T03:13:09.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131535 s, 79.7 MB/s 2023-11-06T03:13:09.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:09.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:09.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124238 s, 84.4 MB/s 2023-11-06T03:13:10.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133311 s, 78.7 MB/s 2023-11-06T03:13:10.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.021 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120462 s, 87.0 MB/s 2023-11-06T03:13:10.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135712 s, 77.3 MB/s 2023-11-06T03:13:10.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129264 s, 81.1 MB/s 2023-11-06T03:13:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994832 s, 105 MB/s 2023-11-06T03:13:10.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.083 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012845 s, 81.6 MB/s 2023-11-06T03:13:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108505 s, 96.6 MB/s 2023-11-06T03:13:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119464 s, 87.8 MB/s 2023-11-06T03:13:10.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122067 s, 85.9 MB/s 2023-11-06T03:13:10.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110935 s, 94.5 MB/s 2023-11-06T03:13:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.157 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129511 s, 81.0 MB/s 2023-11-06T03:13:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.172 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011784 s, 89.0 MB/s 2023-11-06T03:13:10.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137766 s, 76.1 MB/s 2023-11-06T03:13:10.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126838 s, 82.7 MB/s 2023-11-06T03:13:10.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131156 s, 79.9 MB/s 2023-11-06T03:13:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104508 s, 100 MB/s 2023-11-06T03:13:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129764 s, 80.8 MB/s 2023-11-06T03:13:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.266 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135593 s, 77.3 MB/s 2023-11-06T03:13:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013394 s, 78.3 MB/s 2023-11-06T03:13:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124381 s, 84.3 MB/s 2023-11-06T03:13:10.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134702 s, 77.8 MB/s 2023-11-06T03:13:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119018 s, 88.1 MB/s 2023-11-06T03:13:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130555 s, 80.3 MB/s 2023-11-06T03:13:10.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.359 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.359 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105914 s, 99.0 MB/s 2023-11-06T03:13:10.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.373 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109329 s, 95.9 MB/s 2023-11-06T03:13:10.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:10.374 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:10.375 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:10.375 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.24 2023-11-06T03:13:10.375 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:10.376 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:10.376 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:10.376 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:10.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.444 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123106 s, 85.2 MB/s 2023-11-06T03:13:10.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143318 s, 73.2 MB/s 2023-11-06T03:13:10.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122189 s, 85.8 MB/s 2023-11-06T03:13:10.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.492 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131513 s, 79.7 MB/s 2023-11-06T03:13:10.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891349 s, 118 MB/s 2023-11-06T03:13:10.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130239 s, 80.5 MB/s 2023-11-06T03:13:10.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124965 s, 83.9 MB/s 2023-11-06T03:13:10.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.549 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111273 s, 94.2 MB/s 2023-11-06T03:13:10.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113794 s, 92.1 MB/s 2023-11-06T03:13:10.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.574 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871671 s, 120 MB/s 2023-11-06T03:13:10.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115476 s, 90.8 MB/s 2023-11-06T03:13:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118001 s, 88.9 MB/s 2023-11-06T03:13:10.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.618 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116202 s, 90.2 MB/s 2023-11-06T03:13:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.641 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164635 s, 63.7 MB/s 2023-11-06T03:13:10.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.669 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011466 s, 91.5 MB/s 2023-11-06T03:13:10.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.683 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114166 s, 91.8 MB/s 2023-11-06T03:13:10.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885389 s, 118 MB/s 2023-11-06T03:13:10.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.708 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113164 s, 92.7 MB/s 2023-11-06T03:13:10.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.722 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109046 s, 96.2 MB/s 2023-11-06T03:13:10.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.735 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105415 s, 99.5 MB/s 2023-11-06T03:13:10.751 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.752 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014042 s, 74.7 MB/s 2023-11-06T03:13:10.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.766 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.766 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112775 s, 93.0 MB/s 2023-11-06T03:13:10.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.778 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912356 s, 115 MB/s 2023-11-06T03:13:10.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.789 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890852 s, 118 MB/s 2023-11-06T03:13:10.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986945 s, 106 MB/s 2023-11-06T03:13:10.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108568 s, 96.6 MB/s 2023-11-06T03:13:10.828 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.829 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110667 s, 94.8 MB/s 2023-11-06T03:13:10.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011743 s, 89.3 MB/s 2023-11-06T03:13:10.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012662 s, 82.8 MB/s 2023-11-06T03:13:10.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.875 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136286 s, 76.9 MB/s 2023-11-06T03:13:10.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.892 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141997 s, 73.8 MB/s 2023-11-06T03:13:10.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094959 s, 110 MB/s 2023-11-06T03:13:10.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:10.906 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:10.906 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:10.907 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.25 2023-11-06T03:13:10.907 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:10.907 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:10.908 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:10.908 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:10.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.976 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974617 s, 108 MB/s 2023-11-06T03:13:10.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:10.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:10.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998152 s, 105 MB/s 2023-11-06T03:13:11.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.004 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125256 s, 83.7 MB/s 2023-11-06T03:13:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.019 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012666 s, 82.8 MB/s 2023-11-06T03:13:11.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.033 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112068 s, 93.6 MB/s 2023-11-06T03:13:11.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.048 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124766 s, 84.0 MB/s 2023-11-06T03:13:11.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900637 s, 116 MB/s 2023-11-06T03:13:11.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.074 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120283 s, 87.2 MB/s 2023-11-06T03:13:11.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.089 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120751 s, 86.8 MB/s 2023-11-06T03:13:11.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.102 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106766 s, 98.2 MB/s 2023-11-06T03:13:11.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100638 s, 104 MB/s 2023-11-06T03:13:11.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01064 s, 98.6 MB/s 2023-11-06T03:13:11.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007951 s, 132 MB/s 2023-11-06T03:13:11.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.147 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073168 s, 143 MB/s 2023-11-06T03:13:11.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098251 s, 107 MB/s 2023-11-06T03:13:11.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983445 s, 107 MB/s 2023-11-06T03:13:11.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.183 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010193 s, 103 MB/s 2023-11-06T03:13:11.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.195 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993587 s, 106 MB/s 2023-11-06T03:13:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103187 s, 102 MB/s 2023-11-06T03:13:11.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102464 s, 102 MB/s 2023-11-06T03:13:11.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.229 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746545 s, 140 MB/s 2023-11-06T03:13:11.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.241 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.241 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100297 s, 105 MB/s 2023-11-06T03:13:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.251 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778108 s, 135 MB/s 2023-11-06T03:13:11.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.263 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101947 s, 103 MB/s 2023-11-06T03:13:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.273 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792312 s, 132 MB/s 2023-11-06T03:13:11.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999244 s, 105 MB/s 2023-11-06T03:13:11.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.298 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115495 s, 90.8 MB/s 2023-11-06T03:13:11.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.311 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106775 s, 98.2 MB/s 2023-11-06T03:13:11.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.324 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104026 s, 101 MB/s 2023-11-06T03:13:11.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.336 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100916 s, 104 MB/s 2023-11-06T03:13:11.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.346 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787639 s, 133 MB/s 2023-11-06T03:13:11.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.358 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101941 s, 103 MB/s 2023-11-06T03:13:11.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:11.360 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:11.360 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:11.360 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.26 2023-11-06T03:13:11.361 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:11.361 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:11.361 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:11.362 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:11.414 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:13:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.422 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109629 s, 95.6 MB/s 2023-11-06T03:13:11.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:13:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130121 s, 80.6 MB/s 2023-11-06T03:13:11.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.451 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108061 s, 97.0 MB/s 2023-11-06T03:13:11.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.465 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114614 s, 91.5 MB/s 2023-11-06T03:13:11.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.477 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097783 s, 107 MB/s 2023-11-06T03:13:11.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100809 s, 104 MB/s 2023-11-06T03:13:11.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116177 s, 90.3 MB/s 2023-11-06T03:13:11.516 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105345 s, 99.5 MB/s 2023-11-06T03:13:11.528 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.528 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.528 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934822 s, 112 MB/s 2023-11-06T03:13:11.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.538 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775215 s, 135 MB/s 2023-11-06T03:13:11.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808397 s, 130 MB/s 2023-11-06T03:13:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124547 s, 84.2 MB/s 2023-11-06T03:13:11.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928661 s, 113 MB/s 2023-11-06T03:13:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011103 s, 94.4 MB/s 2023-11-06T03:13:11.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117409 s, 89.3 MB/s 2023-11-06T03:13:11.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.614 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993863 s, 106 MB/s 2023-11-06T03:13:11.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.636 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020112 s, 52.1 MB/s 2023-11-06T03:13:11.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.666 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.666 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105092 s, 99.8 MB/s 2023-11-06T03:13:11.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.681 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127496 s, 82.2 MB/s 2023-11-06T03:13:11.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.691 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789872 s, 133 MB/s 2023-11-06T03:13:11.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.704 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109825 s, 95.5 MB/s 2023-11-06T03:13:11.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113821 s, 92.1 MB/s 2023-11-06T03:13:11.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124905 s, 84.0 MB/s 2023-11-06T03:13:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.750 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124842 s, 84.0 MB/s 2023-11-06T03:13:11.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.766 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106049 s, 98.9 MB/s 2023-11-06T03:13:11.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.779 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010881 s, 96.4 MB/s 2023-11-06T03:13:11.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01183 s, 88.6 MB/s 2023-11-06T03:13:11.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123691 s, 84.8 MB/s 2023-11-06T03:13:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.827 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133072 s, 78.8 MB/s 2023-11-06T03:13:11.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130459 s, 80.4 MB/s 2023-11-06T03:13:11.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130394 s, 80.4 MB/s 2023-11-06T03:13:11.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.876 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137875 s, 76.1 MB/s 2023-11-06T03:13:11.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:11.877 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:11.877 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:11.878 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.27 2023-11-06T03:13:11.878 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:11.879 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:11.879 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:11.880 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:11.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125334 s, 83.7 MB/s 2023-11-06T03:13:11.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113726 s, 92.2 MB/s 2023-11-06T03:13:11.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.975 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122228 s, 85.8 MB/s 2023-11-06T03:13:11.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:11.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:11.990 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012263 s, 85.5 MB/s 2023-11-06T03:13:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.004 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116991 s, 89.6 MB/s 2023-11-06T03:13:12.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102473 s, 102 MB/s 2023-11-06T03:13:12.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117683 s, 89.1 MB/s 2023-11-06T03:13:12.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127593 s, 82.2 MB/s 2023-11-06T03:13:12.062 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124751 s, 84.1 MB/s 2023-11-06T03:13:12.076 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.077 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.077 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108833 s, 96.3 MB/s 2023-11-06T03:13:12.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.090 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100918 s, 104 MB/s 2023-11-06T03:13:12.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128276 s, 81.7 MB/s 2023-11-06T03:13:12.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.119 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01064 s, 98.6 MB/s 2023-11-06T03:13:12.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.135 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124542 s, 84.2 MB/s 2023-11-06T03:13:12.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.148 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111054 s, 94.4 MB/s 2023-11-06T03:13:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.167 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140784 s, 74.5 MB/s 2023-11-06T03:13:12.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142667 s, 73.5 MB/s 2023-11-06T03:13:12.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.200 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119621 s, 87.7 MB/s 2023-11-06T03:13:12.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.212 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969937 s, 108 MB/s 2023-11-06T03:13:12.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.227 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011665 s, 89.9 MB/s 2023-11-06T03:13:12.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119916 s, 87.4 MB/s 2023-11-06T03:13:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.261 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140887 s, 74.4 MB/s 2023-11-06T03:13:12.280 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.280 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.280 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148894 s, 70.4 MB/s 2023-11-06T03:13:12.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.299 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140881 s, 74.4 MB/s 2023-11-06T03:13:12.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.315 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134162 s, 78.2 MB/s 2023-11-06T03:13:12.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136072 s, 77.1 MB/s 2023-11-06T03:13:12.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.347 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125476 s, 83.6 MB/s 2023-11-06T03:13:12.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119387 s, 87.8 MB/s 2023-11-06T03:13:12.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.379 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138642 s, 75.6 MB/s 2023-11-06T03:13:12.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118633 s, 88.4 MB/s 2023-11-06T03:13:12.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011952 s, 87.7 MB/s 2023-11-06T03:13:12.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.422 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108589 s, 96.6 MB/s 2023-11-06T03:13:12.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:12.424 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:12.424 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:12.424 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.28 2023-11-06T03:13:12.425 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:12.425 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:12.425 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:12.426 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:12.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.497 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108133 s, 97.0 MB/s 2023-11-06T03:13:12.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.512 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117123 s, 89.5 MB/s 2023-11-06T03:13:12.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.529 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138746 s, 75.6 MB/s 2023-11-06T03:13:12.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.544 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115697 s, 90.6 MB/s 2023-11-06T03:13:12.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.559 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121775 s, 86.1 MB/s 2023-11-06T03:13:12.571 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.571 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.571 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981561 s, 107 MB/s 2023-11-06T03:13:12.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135996 s, 77.1 MB/s 2023-11-06T03:13:12.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.598 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797681 s, 131 MB/s 2023-11-06T03:13:12.612 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.612 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.612 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115821 s, 90.5 MB/s 2023-11-06T03:13:12.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.628 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.628 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121858 s, 86.0 MB/s 2023-11-06T03:13:12.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.642 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112996 s, 92.8 MB/s 2023-11-06T03:13:12.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012002 s, 87.4 MB/s 2023-11-06T03:13:12.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.693 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011514 s, 91.1 MB/s 2023-11-06T03:13:12.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792763 s, 132 MB/s 2023-11-06T03:13:12.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130266 s, 80.5 MB/s 2023-11-06T03:13:12.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007854 s, 134 MB/s 2023-11-06T03:13:12.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125766 s, 83.4 MB/s 2023-11-06T03:13:12.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120632 s, 86.9 MB/s 2023-11-06T03:13:12.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.771 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109381 s, 95.9 MB/s 2023-11-06T03:13:12.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.785 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119501 s, 87.7 MB/s 2023-11-06T03:13:12.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.799 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111445 s, 94.1 MB/s 2023-11-06T03:13:12.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.809 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784577 s, 134 MB/s 2023-11-06T03:13:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121847 s, 86.1 MB/s 2023-11-06T03:13:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113141 s, 92.7 MB/s 2023-11-06T03:13:12.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.848 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889164 s, 118 MB/s 2023-11-06T03:13:12.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.859 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850501 s, 123 MB/s 2023-11-06T03:13:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129621 s, 80.9 MB/s 2023-11-06T03:13:12.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.887 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103227 s, 102 MB/s 2023-11-06T03:13:12.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128821 s, 81.4 MB/s 2023-11-06T03:13:12.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012564 s, 83.5 MB/s 2023-11-06T03:13:12.927 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.927 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.928 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866646 s, 121 MB/s 2023-11-06T03:13:12.938 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:12.939 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:12.939 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902413 s, 116 MB/s 2023-11-06T03:13:12.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:12.940 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:12.941 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:12.941 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.29 2023-11-06T03:13:12.941 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:12.942 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:12.942 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:12.942 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108692 s, 96.5 MB/s 2023-11-06T03:13:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114128 s, 91.9 MB/s 2023-11-06T03:13:13.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105583 s, 99.3 MB/s 2023-11-06T03:13:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123967 s, 84.6 MB/s 2023-11-06T03:13:13.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.060 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116065 s, 90.3 MB/s 2023-11-06T03:13:13.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.073 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111807 s, 93.8 MB/s 2023-11-06T03:13:13.087 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.087 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.087 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117169 s, 89.5 MB/s 2023-11-06T03:13:13.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.099 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979174 s, 107 MB/s 2023-11-06T03:13:13.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.111 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103624 s, 101 MB/s 2023-11-06T03:13:13.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.125 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010908 s, 96.1 MB/s 2023-11-06T03:13:13.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751024 s, 140 MB/s 2023-11-06T03:13:13.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.147 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.147 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105712 s, 99.2 MB/s 2023-11-06T03:13:13.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101169 s, 104 MB/s 2023-11-06T03:13:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972552 s, 108 MB/s 2023-11-06T03:13:13.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108147 s, 97.0 MB/s 2023-11-06T03:13:13.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.194 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766863 s, 137 MB/s 2023-11-06T03:13:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102403 s, 102 MB/s 2023-11-06T03:13:13.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102022 s, 103 MB/s 2023-11-06T03:13:13.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.231 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102516 s, 102 MB/s 2023-11-06T03:13:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107263 s, 97.8 MB/s 2023-11-06T03:13:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010423 s, 101 MB/s 2023-11-06T03:13:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926047 s, 113 MB/s 2023-11-06T03:13:13.280 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.280 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.280 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104039 s, 101 MB/s 2023-11-06T03:13:13.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.290 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800168 s, 131 MB/s 2023-11-06T03:13:13.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751408 s, 140 MB/s 2023-11-06T03:13:13.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121752 s, 86.1 MB/s 2023-11-06T03:13:13.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.325 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792612 s, 132 MB/s 2023-11-06T03:13:13.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.337 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105154 s, 99.7 MB/s 2023-11-06T03:13:13.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.350 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107845 s, 97.2 MB/s 2023-11-06T03:13:13.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123841 s, 84.7 MB/s 2023-11-06T03:13:13.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.377 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102035 s, 103 MB/s 2023-11-06T03:13:13.386 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.387 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792314 s, 132 MB/s 2023-11-06T03:13:13.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:13.388 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:13.389 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:13.389 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.30 2023-11-06T03:13:13.389 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:13.390 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:13.390 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:13.390 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:13.454 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100077 s, 105 MB/s 2023-11-06T03:13:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122485 s, 85.6 MB/s 2023-11-06T03:13:13.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.483 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109543 s, 95.7 MB/s 2023-11-06T03:13:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817073 s, 128 MB/s 2023-11-06T03:13:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105234 s, 99.6 MB/s 2023-11-06T03:13:13.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122809 s, 85.4 MB/s 2023-11-06T03:13:13.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822887 s, 127 MB/s 2023-11-06T03:13:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012191 s, 86.0 MB/s 2023-11-06T03:13:13.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129123 s, 81.2 MB/s 2023-11-06T03:13:13.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108212 s, 96.9 MB/s 2023-11-06T03:13:13.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.590 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124858 s, 84.0 MB/s 2023-11-06T03:13:13.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.604 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103109 s, 102 MB/s 2023-11-06T03:13:13.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.617 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120804 s, 86.8 MB/s 2023-11-06T03:13:13.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.630 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108095 s, 97.0 MB/s 2023-11-06T03:13:13.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.640 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782422 s, 134 MB/s 2023-11-06T03:13:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123432 s, 85.0 MB/s 2023-11-06T03:13:13.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.695 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776309 s, 135 MB/s 2023-11-06T03:13:13.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.709 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112648 s, 93.1 MB/s 2023-11-06T03:13:13.723 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012428 s, 84.4 MB/s 2023-11-06T03:13:13.737 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.737 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.738 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010945 s, 95.8 MB/s 2023-11-06T03:13:13.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.752 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117992 s, 88.9 MB/s 2023-11-06T03:13:13.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.765 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106564 s, 98.4 MB/s 2023-11-06T03:13:13.780 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.780 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.780 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116748 s, 89.8 MB/s 2023-11-06T03:13:13.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119698 s, 87.6 MB/s 2023-11-06T03:13:13.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.812 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124491 s, 84.2 MB/s 2023-11-06T03:13:13.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112887 s, 92.9 MB/s 2023-11-06T03:13:13.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116154 s, 90.3 MB/s 2023-11-06T03:13:13.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144467 s, 72.6 MB/s 2023-11-06T03:13:13.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130707 s, 80.2 MB/s 2023-11-06T03:13:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011422 s, 91.8 MB/s 2023-11-06T03:13:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885414 s, 118 MB/s 2023-11-06T03:13:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093224 s, 112 MB/s 2023-11-06T03:13:13.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:13.913 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:13.914 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:13.914 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.31 2023-11-06T03:13:13.914 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:13.915 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:13.915 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:13.916 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:13.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:13.984 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144959 s, 72.3 MB/s 2023-11-06T03:13:13.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:13.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012604 s, 83.2 MB/s 2023-11-06T03:13:14.012 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.013 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.013 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103137 s, 102 MB/s 2023-11-06T03:13:14.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.030 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140311 s, 74.7 MB/s 2023-11-06T03:13:14.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097929 s, 107 MB/s 2023-11-06T03:13:14.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.053 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821932 s, 128 MB/s 2023-11-06T03:13:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106973 s, 98.0 MB/s 2023-11-06T03:13:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122965 s, 85.3 MB/s 2023-11-06T03:13:14.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.095 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010488 s, 100 MB/s 2023-11-06T03:13:14.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122623 s, 85.5 MB/s 2023-11-06T03:13:14.119 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.120 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.120 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802488 s, 131 MB/s 2023-11-06T03:13:14.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846412 s, 124 MB/s 2023-11-06T03:13:14.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.145 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113077 s, 92.7 MB/s 2023-11-06T03:13:14.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.156 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928013 s, 113 MB/s 2023-11-06T03:13:14.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120216 s, 87.2 MB/s 2023-11-06T03:13:14.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.186 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138066 s, 75.9 MB/s 2023-11-06T03:13:14.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.199 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990586 s, 106 MB/s 2023-11-06T03:13:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944954 s, 111 MB/s 2023-11-06T03:13:14.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.226 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130808 s, 80.2 MB/s 2023-11-06T03:13:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.237 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894827 s, 117 MB/s 2023-11-06T03:13:14.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.252 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127765 s, 82.1 MB/s 2023-11-06T03:13:14.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.266 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011713 s, 89.5 MB/s 2023-11-06T03:13:14.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.280 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01124 s, 93.3 MB/s 2023-11-06T03:13:14.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.295 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119502 s, 87.7 MB/s 2023-11-06T03:13:14.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.311 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.312 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134824 s, 77.8 MB/s 2023-11-06T03:13:14.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145592 s, 72.0 MB/s 2023-11-06T03:13:14.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.346 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137072 s, 76.5 MB/s 2023-11-06T03:13:14.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131267 s, 79.9 MB/s 2023-11-06T03:13:14.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.414 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137197 s, 76.4 MB/s 2023-11-06T03:13:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.415 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127308 s, 82.4 MB/s 2023-11-06T03:13:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.416 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127623 s, 82.2 MB/s 2023-11-06T03:13:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:14.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117515 s, 89.2 MB/s 2023-11-06T03:13:14.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2023-11-06T03:13:14.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 Processor -- start 2023-11-06T03:13:14.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 -- start start 2023-11-06T03:13:14.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6aec06c170 0x7f6aec154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec15b860 0x7f6aec155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0cb100 con 0x7f6aec06c550 2023-11-06T03:13:14.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec15bcf0 con 0x7f6aec06c170 2023-11-06T03:13:14.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6af10a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0ce0a0 con 0x7f6aec15b860 2023-11-06T03:13:14.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6aec06c170 0x7f6aec154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6aec06c170 0x7f6aec154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58126/0 (socket says 172.21.15.157:58126) 2023-11-06T03:13:14.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 -- 172.21.15.157:0/3084570912 learned_addr learned my addr 172.21.15.157:0/3084570912 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aea59c640 1 --1- 172.21.15.157:0/3084570912 >> v1:172.21.15.157:6789/0 conn(0x7f6aec06c550 0x7f6aec15b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49398/0 (socket says 172.21.15.157:49398) 2023-11-06T03:13:14.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec15b860 0x7f6aec155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 -- 172.21.15.157:0/3084570912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec15b860 msgr2=0x7f6aec155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec15b860 0x7f6aec155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 -- 172.21.15.157:0/3084570912 >> v1:172.21.15.157:6789/0 conn(0x7f6aec06c550 legacy=0x7f6aec15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6aead9d640 1 -- 172.21.15.157:0/3084570912 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec15bef0 con 0x7f6aec06c170 2023-11-06T03:13:14.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.601+0000 7f6ae9d9b640 1 -- 172.21.15.157:0/3084570912 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006793561 0 0) 0x7f6aec0cb100 con 0x7f6aec06c550 2023-11-06T03:13:14.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6aead9d640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c170 0x7f6aec154c00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009870 tx=0x7f6adc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee75dc113ddef72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6ae9d9b640 1 -- 172.21.15.157:0/3084570912 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0aa670 con 0x7f6aec06c170 2023-11-06T03:13:14.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6ae9d9b640 1 -- 172.21.15.157:0/3084570912 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0aa7d0 con 0x7f6aec06c170 2023-11-06T03:13:14.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6ae9d9b640 1 -- 172.21.15.157:0/3084570912 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0aab60 con 0x7f6aec06c170 2023-11-06T03:13:14.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- 172.21.15.157:0/3084570912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c170 msgr2=0x7f6aec154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c170 0x7f6aec154c00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009870 tx=0x7f6adc01c510 comp rx=0 tx=0).stop 2023-11-06T03:13:14.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- 172.21.15.157:0/3084570912 shutdown_connections 2023-11-06T03:13:14.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec15b860 0x7f6aec155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 --2- 172.21.15.157:0/3084570912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c170 0x7f6aec154c00 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- 172.21.15.157:0/3084570912 >> 172.21.15.157:0/3084570912 conn(0x7f6aec0b43b0 msgr2=0x7f6aec065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- 172.21.15.157:0/3084570912 shutdown_connections 2023-11-06T03:13:14.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- 172.21.15.157:0/3084570912 wait complete. 2023-11-06T03:13:14.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 Processor -- start 2023-11-06T03:13:14.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.605+0000 7f6af10a1640 1 -- start start 2023-11-06T03:13:14.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec06c170 0x7f6aec156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 0x7f6aec153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec157410 con 0x7f6aec06c550 2023-11-06T03:13:14.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec14e170 con 0x7f6aec15b860 2023-11-06T03:13:14.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec14e450 con 0x7f6aec06c170 2023-11-06T03:13:14.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58136/0 (socket says 172.21.15.157:58136) 2023-11-06T03:13:14.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 -- 172.21.15.157:0/561686269 learned_addr learned my addr 172.21.15.157:0/561686269 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aead9d640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec06c170 0x7f6aec156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 0x7f6aec153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 -- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec06c170 msgr2=0x7f6aec156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec06c170 0x7f6aec156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 -- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 msgr2=0x7f6aec153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 0x7f6aec153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 -- 172.21.15.157:0/561686269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0094d0 con 0x7f6aec06c550 2023-11-06T03:13:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 0x7f6aec153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aea59c640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00011f0 tx=0x7f6ae000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 -- 172.21.15.157:0/561686269 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00074e0 con 0x7f6aec06c550 2023-11-06T03:13:14.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 -- 172.21.15.157:0/561686269 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0007640 con 0x7f6aec06c550 2023-11-06T03:13:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0cbfb0 con 0x7f6aec06c550 2023-11-06T03:13:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 -- 172.21.15.157:0/561686269 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004a10 con 0x7f6aec06c550 2023-11-06T03:13:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec1535a0 con 0x7f6aec06c550 2023-11-06T03:13:14.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 -- 172.21.15.157:0/561686269 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ae0004b70 con 0x7f6aec06c550 2023-11-06T03:13:14.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 0x7f6ad8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6ad37fe640 1 -- 172.21.15.157:0/561686269 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6ae00827a0 con 0x7f6aec06c550 2023-11-06T03:13:14.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.609+0000 7f6aead9d640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 0x7f6ad8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 Processor -- start 2023-11-06T03:13:14.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- start start 2023-11-06T03:13:14.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 0x7f6ad8067f50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ae000 tx=0x7f6adc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 0x55b813a37f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 0x55b813a38820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8138fc150 con 0x55b813a81ec0 2023-11-06T03:13:14.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8138881b0 con 0x55b813a827e0 2023-11-06T03:13:14.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813885780 con 0x55b813a38440 2023-11-06T03:13:14.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58150/0 (socket says 172.21.15.157:58150) 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 -- 172.21.15.157:0/2522913818 learned_addr learned my addr 172.21.15.157:0/2522913818 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aea59c640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 0x55b813a37f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 0x55b813a38820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 -- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 msgr2=0x55b813a38820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 0x55b813a38820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 -- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 msgr2=0x55b813a37f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 0x55b813a37f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 -- 172.21.15.157:0/2522913818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b81357dda0 con 0x55b813a81ec0 2023-11-06T03:13:14.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aea59c640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 0x55b813a37f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 0x55b813a38820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6aead9d640 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01abc0 tx=0x7f6adc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81f8744fb58c6b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6ad0ff9640 1 -- 172.21.15.157:0/2522913818 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b37b0 con 0x55b813a81ec0 2023-11-06T03:13:14.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6ad0ff9640 1 -- 172.21.15.157:0/2522913818 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0c3070 con 0x55b813a81ec0 2023-11-06T03:13:14.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6ad0ff9640 1 -- 172.21.15.157:0/2522913818 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b3b10 con 0x55b813a81ec0 2023-11-06T03:13:14.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- 172.21.15.157:0/2522913818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 msgr2=0x55b813a822a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01abc0 tx=0x7f6adc004740 comp rx=0 tx=0).stop 2023-11-06T03:13:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- 172.21.15.157:0/2522913818 shutdown_connections 2023-11-06T03:13:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a38440 0x55b813a38820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a827e0 0x55b813a37f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 --2- 172.21.15.157:0/2522913818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a822a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- 172.21.15.157:0/2522913818 >> 172.21.15.157:0/2522913818 conn(0x55b813a3a3a0 msgr2=0x55b813a323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- 172.21.15.157:0/2522913818 shutdown_connections 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- 172.21.15.157:0/2522913818 wait complete. 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 -- start start 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.613+0000 7f6af41c9000 1 Processor -- start 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a81ec0 0x55b813a88750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a827e0 0x55b813a6ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813881070 con 0x55b813a38440 2023-11-06T03:13:14.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8138207c0 con 0x55b813a81ec0 2023-11-06T03:13:14.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813820460 con 0x55b813a827e0 2023-11-06T03:13:14.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 10 client.0 _ll_register_callbacks cb 0x55b81387a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:14.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58160/0 (socket says 172.21.15.157:58160) 2023-11-06T03:13:14.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 -- 172.21.15.157:0/1592824936 learned_addr learned my addr 172.21.15.157:0/1592824936 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aea59c640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a81ec0 0x55b813a88750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 -- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a827e0 msgr2=0x55b813a6ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a827e0 0x55b813a6ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 -- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a81ec0 msgr2=0x55b813a88750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a81ec0 0x55b813a88750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 -- 172.21.15.157:0/1592824936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc0b9070 con 0x55b813a38440 2023-11-06T03:13:14.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aead9d640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01c9f0 tx=0x7f6adc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16452ebfd716ef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:14.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b3890 con 0x55b813a38440 2023-11-06T03:13:14.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6adc0d2c50 con 0x55b813a38440 2023-11-06T03:13:14.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b81383a840 con 0x55b813a38440 2023-11-06T03:13:14.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6adc0d2e50 con 0x55b813a38440 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 10 client.8433 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55b813821410 con 0x55b813a38440 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b81383a840 con 0x55b813a38440 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6adc0a8050 con 0x55b813a38440 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 10 client.8433 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:14.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 10 client.8433 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:14.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 20 client.8433 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:14.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a6f990 0x55b813a6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 4 client.8433 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:14.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b81387f9c0 con 0x55b813a6f990 2023-11-06T03:13:14.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aea59c640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a6f990 0x55b813a6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 10 client.8433 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6aea59c640 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a6f990 0x55b813a6fd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3f06598078f627 server_cookie=9c770ce3a604d1f2 in_seq=0 out_seq=0 2023-11-06T03:13:14.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/1592824936 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b81387f9c0 con 0x55b813a6f990 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 10 client.8433 handle_command_reply: tid=0 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 client.8433 shutdown 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 msgr2=0x55b813a88210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01c9f0 tx=0x7f6adc01ad50 comp rx=0 tx=0).stop 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 shutdown_connections 2023-11-06T03:13:14.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a6f990 0x55b813a6fd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a827e0 0x55b813a6ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a81ec0 0x55b813a88750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6ac9ffb640 0 client.8433 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 --2- 172.21.15.157:0/1592824936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a38440 0x55b813a88210 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 >> 172.21.15.157:0/1592824936 conn(0x55b813a3a3a0 msgr2=0x55b813a31b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 shutdown_connections 2023-11-06T03:13:14.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 1 -- 172.21.15.157:0/1592824936 wait complete. 2023-11-06T03:13:14.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.617+0000 7f6af41c9000 20 client.8433 trim_cache size 0 max 16384 2023-11-06T03:13:14.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 Processor -- start 2023-11-06T03:13:14.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- start start 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813ad9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 0x55b813ad9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81353ea20 con 0x55b813a3a3a0 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81387ff00 con 0x55b813a38440 2023-11-06T03:13:14.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813a3fb60 con 0x55b813a81ec0 2023-11-06T03:13:14.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58176/0 (socket says 172.21.15.157:58176) 2023-11-06T03:13:14.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 -- 172.21.15.157:0/3650179359 learned_addr learned my addr 172.21.15.157:0/3650179359 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aea59c640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813ad9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 0x55b813ad9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 -- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 msgr2=0x55b813ad9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 0x55b813ad9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 -- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 msgr2=0x55b813ad9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813ad9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 -- 172.21.15.157:0/3650179359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b813846370 con 0x55b813a3a3a0 2023-11-06T03:13:14.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 0x55b813ad9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6aead9d640 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01ad90 tx=0x7f6adc00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f7c22ec9fe9810 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6ad17fa640 1 -- 172.21.15.157:0/3650179359 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0e1070 con 0x55b813a3a3a0 2023-11-06T03:13:14.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6ad17fa640 1 -- 172.21.15.157:0/3650179359 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0c3040 con 0x55b813a3a3a0 2023-11-06T03:13:14.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6ad17fa640 1 -- 172.21.15.157:0/3650179359 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b3890 con 0x55b813a3a3a0 2023-11-06T03:13:14.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- 172.21.15.157:0/3650179359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 msgr2=0x55b813ad94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6adc01ad90 tx=0x7f6adc00b050 comp rx=0 tx=0).stop 2023-11-06T03:13:14.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- 172.21.15.157:0/3650179359 shutdown_connections 2023-11-06T03:13:14.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a81ec0 0x55b813ad9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813ad9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 --2- 172.21.15.157:0/3650179359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a3a3a0 0x55b813ad94e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- 172.21.15.157:0/3650179359 >> 172.21.15.157:0/3650179359 conn(0x55b813a8ea60 msgr2=0x55b813a31b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- 172.21.15.157:0/3650179359 shutdown_connections 2023-11-06T03:13:14.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- 172.21.15.157:0/3650179359 wait complete. 2023-11-06T03:13:14.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.633+0000 7f6af41c9000 1 -- start start 2023-11-06T03:13:14.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 Processor -- start 2023-11-06T03:13:14.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813a6beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 0x55b813a6c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813849540 con 0x55b813a81ec0 2023-11-06T03:13:14.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b81383a460 con 0x55b813a38440 2023-11-06T03:13:14.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8138c5670 con 0x55b813a3a3a0 2023-11-06T03:13:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 10 client.0 _ll_register_callbacks cb 0x55b81384af80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58178/0 (socket says 172.21.15.157:58178) 2023-11-06T03:13:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 -- 172.21.15.157:0/628729593 learned_addr learned my addr 172.21.15.157:0/628729593 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aead9d640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813a6beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aea59c640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 0x55b813a6c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 -- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 msgr2=0x55b813a6c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 0x55b813a6c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 -- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 msgr2=0x55b813a6beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813a6beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 -- 172.21.15.157:0/628729593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b813821410 con 0x55b813a81ec0 2023-11-06T03:13:14.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aea59c640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 0x55b813a6c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aeb59e640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4007920 tx=0x7f6ae4007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1c0deba53de4dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:14.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4011ba0 con 0x55b813a81ec0 2023-11-06T03:13:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6ae400e030 con 0x55b813a81ec0 2023-11-06T03:13:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b813889990 con 0x55b813a81ec0 2023-11-06T03:13:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae4011d00 con 0x55b813a81ec0 2023-11-06T03:13:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 10 client.8439 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:14.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55b81382c4c0 con 0x55b813a81ec0 2023-11-06T03:13:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b813889990 con 0x55b813a81ec0 2023-11-06T03:13:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6ae401e440 con 0x55b813a81ec0 2023-11-06T03:13:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 10 client.8439 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 10 client.8439 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:14.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 20 client.8439 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:14.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a827e0 0x55b813ae5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 4 client.8439 mds_command: new command op to 8226 tid=0[{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2023-11-06T03:13:14.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55b81387a0f0 con 0x55b813a827e0 2023-11-06T03:13:14.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aead9d640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a827e0 0x55b813ae5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6ac9ffb640 10 client.8439 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.637+0000 7f6aead9d640 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a827e0 0x55b813ae5770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efedcc8f8a6e5839 server_cookie=9d4bc962bff84c82 in_seq=0 out_seq=0 2023-11-06T03:13:14.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6ac9ffb640 1 -- 172.21.15.157:0/628729593 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b81387a0f0 con 0x55b813a827e0 2023-11-06T03:13:14.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6ac9ffb640 10 client.8439 handle_command_reply: tid=0 2023-11-06T03:13:14.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 client.8439 shutdown 2023-11-06T03:13:14.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 msgr2=0x55b813a6c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4007920 tx=0x7f6ae4007de0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 shutdown_connections 2023-11-06T03:13:14.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b813a827e0 0x55b813ae5770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b813a81ec0 0x55b813a6c930 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b813a3a3a0 0x55b813a6c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 --2- 172.21.15.157:0/628729593 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b813a38440 0x55b813a6beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6ac9ffb640 0 client.8439 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 >> 172.21.15.157:0/628729593 conn(0x55b813a8ea60 msgr2=0x55b813a323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 shutdown_connections 2023-11-06T03:13:14.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 1 -- 172.21.15.157:0/628729593 wait complete. 2023-11-06T03:13:14.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af41c9000 20 client.8439 trim_cache size 0 max 16384 2023-11-06T03:13:14.661 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:14.661 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:13:14.662 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "73e5c3fc-0554-4677-b495-99a29248aef5", 2023-11-06T03:13:14.662 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:13:14.662 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:14.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 msgr2=0x7f6ad8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 0x7f6ad8067f50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ae000 tx=0x7f6adc0a7000 comp rx=0 tx=0).stop 2023-11-06T03:13:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 msgr2=0x7f6aec14d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00011f0 tx=0x7f6ae000c9b0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 shutdown_connections 2023-11-06T03:13:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6ad8065a90 0x7f6ad8067f50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6aec15b860 0x7f6aec153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6aec06c550 0x7f6aec14d550 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 --2- 172.21.15.157:0/561686269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6aec06c170 0x7f6aec156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 >> 172.21.15.157:0/561686269 conn(0x7f6aec0b43b0 msgr2=0x7f6aec070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 shutdown_connections 2023-11-06T03:13:14.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.641+0000 7f6af10a1640 1 -- 172.21.15.157:0/561686269 wait complete. 2023-11-06T03:13:14.666 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub pause 2023-11-06T03:13:14.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 Processor -- start 2023-11-06T03:13:14.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 -- start start 2023-11-06T03:13:14.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b780d31f0 0x7f7b780cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b780d0b70 0x7f7b780b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7815d030 con 0x7f7b7815cc50 2023-11-06T03:13:14.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780d1cd0 con 0x7f7b780d31f0 2023-11-06T03:13:14.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7f88b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780d10c0 con 0x7f7b780d0b70 2023-11-06T03:13:14.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b780d31f0 0x7f7b780cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b780d31f0 0x7f7b780cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58180/0 (socket says 172.21.15.157:58180) 2023-11-06T03:13:14.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3257069640 learned_addr learned my addr 172.21.15.157:0/3257069640 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7d627640 1 --1- 172.21.15.157:0/3257069640 >> v1:172.21.15.157:6789/0 conn(0x7f7b7815cc50 0x7f7b780d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49404/0 (socket says 172.21.15.157:49404) 2023-11-06T03:13:14.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7de28640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b780d0b70 0x7f7b780b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3257069640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b780d0b70 msgr2=0x7f7b780b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b780d0b70 0x7f7b780b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3257069640 >> v1:172.21.15.157:6789/0 conn(0x7f7b7815cc50 legacy=0x7f7b780d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3257069640 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780d13a0 con 0x7f7b780d31f0 2023-11-06T03:13:14.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.809+0000 7f7b6ffff640 1 -- 172.21.15.157:0/3257069640 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181753550 0 0) 0x7f7b74001120 con 0x7f7b7815cc50 2023-11-06T03:13:14.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b780cdde0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009870 tx=0x7f7b6801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510e4fd0c0fc5f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b6ffff640 1 -- 172.21.15.157:0/3257069640 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680aa670 con 0x7f7b780d31f0 2023-11-06T03:13:14.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b6ffff640 1 -- 172.21.15.157:0/3257069640 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680aa7d0 con 0x7f7b780d31f0 2023-11-06T03:13:14.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b6ffff640 1 -- 172.21.15.157:0/3257069640 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680aab60 con 0x7f7b780d31f0 2023-11-06T03:13:14.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- 172.21.15.157:0/3257069640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 msgr2=0x7f7b780cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b780cdde0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009870 tx=0x7f7b6801c510 comp rx=0 tx=0).stop 2023-11-06T03:13:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- 172.21.15.157:0/3257069640 shutdown_connections 2023-11-06T03:13:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b780d0b70 0x7f7b780b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/3257069640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b780cdde0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- 172.21.15.157:0/3257069640 >> 172.21.15.157:0/3257069640 conn(0x7f7b78070790 msgr2=0x7f7b78070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- 172.21.15.157:0/3257069640 shutdown_connections 2023-11-06T03:13:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- 172.21.15.157:0/3257069640 wait complete. 2023-11-06T03:13:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 Processor -- start 2023-11-06T03:13:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.813+0000 7f7b7f88b640 1 -- start start 2023-11-06T03:13:14.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b780d0b70 0x7f7b780c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b7815cc50 0x7f7b780cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780c7760 con 0x7f7b780d31f0 2023-11-06T03:13:14.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780cb910 con 0x7f7b780d0b70 2023-11-06T03:13:14.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780cbbf0 con 0x7f7b7815cc50 2023-11-06T03:13:14.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58190/0 (socket says 172.21.15.157:58190) 2023-11-06T03:13:14.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1452981043 learned_addr learned my addr 172.21.15.157:0/1452981043 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7d627640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b780d0b70 0x7f7b780c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7de28640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b7815cc50 0x7f7b780cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b7815cc50 msgr2=0x7f7b780cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b7815cc50 0x7f7b780cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b780d0b70 msgr2=0x7f7b780c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b780d0b70 0x7f7b780c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1452981043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680094d0 con 0x7f7b780d31f0 2023-11-06T03:13:14.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b680ac040 tx=0x7f7b6801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b6dffb640 1 -- 172.21.15.157:0/1452981043 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680aad50 con 0x7f7b780d31f0 2023-11-06T03:13:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b6dffb640 1 -- 172.21.15.157:0/1452981043 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680b7070 con 0x7f7b780d31f0 2023-11-06T03:13:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b780c6740 con 0x7f7b780d31f0 2023-11-06T03:13:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b6dffb640 1 -- 172.21.15.157:0/1452981043 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004230 con 0x7f7b780d31f0 2023-11-06T03:13:14.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b780c6d00 con 0x7f7b780d31f0 2023-11-06T03:13:14.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b829b3000 1 Processor -- start 2023-11-06T03:13:14.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.817+0000 7f7b829b3000 1 -- start start 2023-11-06T03:13:14.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61ed25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d99bb0 con 0x55eb61ed2200 2023-11-06T03:13:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d28380 con 0x55eb61f1c9f0 2023-11-06T03:13:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d25950 con 0x55eb61f1c0d0 2023-11-06T03:13:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:40814/0 (socket says 172.21.15.157:40814) 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 -- 172.21.15.157:0/2137303492 learned_addr learned my addr 172.21.15.157:0/2137303492 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:50422/0 (socket says 172.21.15.157:50422) 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b6dffb640 1 -- 172.21.15.157:0/1452981043 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7b68004390 con 0x7f7b780d31f0 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61ed25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 -- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 msgr2=0x55eb61ed1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 -- 172.21.15.157:0/2137303492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 msgr2=0x55eb61ed25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61ed25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 -- 172.21.15.157:0/2137303492 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb61a1ddc0 con 0x55eb61f1c0d0 2023-11-06T03:13:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b6dffb640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 0x7f7b5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b6dffb640 1 -- 172.21.15.157:0/1452981043 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7b6811d980 con 0x7f7b780d31f0 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7de28640 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009f90 tx=0x7f7b700077c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7417d822427fe0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 0x7f7b5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b62ffd640 1 -- 172.21.15.157:0/2137303492 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700148a0 con 0x55eb61f1c0d0 2023-11-06T03:13:14.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b62ffd640 1 -- 172.21.15.157:0/2137303492 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000f050 con 0x55eb61f1c0d0 2023-11-06T03:13:14.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b62ffd640 1 -- 172.21.15.157:0/2137303492 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70014c00 con 0x55eb61f1c0d0 2023-11-06T03:13:14.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 msgr2=0x55eb61f1c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009f90 tx=0x7f7b700077c0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- 172.21.15.157:0/2137303492 shutdown_connections 2023-11-06T03:13:14.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61ed25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c9f0 0x55eb61ed1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 --2- 172.21.15.157:0/2137303492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f1c4b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- 172.21.15.157:0/2137303492 >> 172.21.15.157:0/2137303492 conn(0x55eb61ed8990 msgr2=0x55eb61ed05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- 172.21.15.157:0/2137303492 shutdown_connections 2023-11-06T03:13:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b7d627640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 0x7f7b5c067f20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005910 tx=0x7f7b74007f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- 172.21.15.157:0/2137303492 wait complete. 2023-11-06T03:13:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 -- start start 2023-11-06T03:13:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.821+0000 7f7b829b3000 1 Processor -- start 2023-11-06T03:13:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 0x55eb61f1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d21240 con 0x55eb61f1c0d0 2023-11-06T03:13:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61cc0410 con 0x55eb61ed2200 2023-11-06T03:13:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb61cde600 con 0x55eb61f1c9f0 2023-11-06T03:13:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 10 client.0 _ll_register_callbacks cb 0x55eb61ebdbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58208/0 (socket says 172.21.15.157:58208) 2023-11-06T03:13:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3080379881 learned_addr learned my addr 172.21.15.157:0/3080379881 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7d627640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7de28640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 0x55eb61f1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 msgr2=0x55eb61f1cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 0x55eb61f1cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 msgr2=0x55eb61f235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 -- 172.21.15.157:0/3080379881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7400f040 con 0x55eb61f1c0d0 2023-11-06T03:13:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7d627640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7de28640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 0x55eb61f1cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b6801ca80 tx=0x7f7b680ee870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bf3dece5fbfbb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680f2d40 con 0x55eb61f1c0d0 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7b680f6ef0 con 0x55eb61f1c0d0 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb61cde040 con 0x55eb61f1c0d0 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b680bb350 con 0x55eb61f1c0d0 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 10 client.8454 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x7f7b70009bf0 con 0x55eb61f1c0d0 2023-11-06T03:13:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb61cde040 con 0x55eb61f1c0d0 2023-11-06T03:13:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7b680c4060 con 0x55eb61f1c0d0 2023-11-06T03:13:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 10 client.8454 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 10 client.8454 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 20 client.8454 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:14.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1db70 0x55eb61f1df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 4 client.8454 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb61d1fb90 con 0x55eb61f1db70 2023-11-06T03:13:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7d627640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1db70 0x55eb61f1df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b7d627640 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1db70 0x55eb61f1df50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9573a0a15ea17807 server_cookie=5faea3af863fc99f in_seq=0 out_seq=0 2023-11-06T03:13:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 10 client.8454 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 1 -- 172.21.15.157:0/3080379881 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eb61d1fb90 con 0x55eb61f1db70 2023-11-06T03:13:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 10 client.8454 handle_command_reply: tid=0 2023-11-06T03:13:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 client.8454 shutdown 2023-11-06T03:13:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 msgr2=0x55eb61f23ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b6801ca80 tx=0x7f7b680ee870 comp rx=0 tx=0).stop 2023-11-06T03:13:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 shutdown_connections 2023-11-06T03:13:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1db70 0x55eb61f1df50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c9f0 0x55eb61f1cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61f1c0d0 0x55eb61f23ae0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b57fff640 0 client.8454 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 --2- 172.21.15.157:0/3080379881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 >> 172.21.15.157:0/3080379881 conn(0x55eb61ed8990 msgr2=0x55eb61ed0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 shutdown_connections 2023-11-06T03:13:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 1 -- 172.21.15.157:0/3080379881 wait complete. 2023-11-06T03:13:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.825+0000 7f7b829b3000 20 client.8454 trim_cache size 0 max 16384 2023-11-06T03:13:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 Processor -- start 2023-11-06T03:13:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- start start 2023-11-06T03:13:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61ed8990 0x55eb61f20f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c0d0 0x55eb61f219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb619dea40 con 0x55eb61ed2200 2023-11-06T03:13:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d1ea50 con 0x55eb61f1c0d0 2023-11-06T03:13:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d1e640 con 0x55eb61ed8990 2023-11-06T03:13:14.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58212/0 (socket says 172.21.15.157:58212) 2023-11-06T03:13:14.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1906167911 learned_addr learned my addr 172.21.15.157:0/1906167911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61ed8990 msgr2=0x55eb61f20f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61ed8990 0x55eb61f20f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c0d0 msgr2=0x55eb61f219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c0d0 0x55eb61f219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 -- 172.21.15.157:0/1906167911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb61ce6170 con 0x55eb61ed2200 2023-11-06T03:13:14.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7b6801a140 tx=0x7f7b680b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760a1ccf4bac43ce server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b637fe640 1 -- 172.21.15.157:0/1906167911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6811dc40 con 0x55eb61ed2200 2023-11-06T03:13:14.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b637fe640 1 -- 172.21.15.157:0/1906167911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680eece0 con 0x55eb61ed2200 2023-11-06T03:13:14.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b637fe640 1 -- 172.21.15.157:0/1906167911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680f6ef0 con 0x55eb61ed2200 2023-11-06T03:13:14.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- 172.21.15.157:0/1906167911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 msgr2=0x55eb61f214b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7b6801a140 tx=0x7f7b680b5000 comp rx=0 tx=0).stop 2023-11-06T03:13:14.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- 172.21.15.157:0/1906167911 shutdown_connections 2023-11-06T03:13:14.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61f1c0d0 0x55eb61f219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed2200 0x55eb61f214b0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 --2- 172.21.15.157:0/1906167911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61ed8990 0x55eb61f20f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- 172.21.15.157:0/1906167911 >> 172.21.15.157:0/1906167911 conn(0x55eb61f2a020 msgr2=0x55eb61ed7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- 172.21.15.157:0/1906167911 shutdown_connections 2023-11-06T03:13:14.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.841+0000 7f7b829b3000 1 -- 172.21.15.157:0/1906167911 wait complete. 2023-11-06T03:13:14.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- start start 2023-11-06T03:13:14.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 Processor -- start 2023-11-06T03:13:14.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f79e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d23da0 con 0x55eb61ed8990 2023-11-06T03:13:14.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb61d25240 con 0x55eb61ed2200 2023-11-06T03:13:14.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb61da9c10 con 0x55eb61f1c0d0 2023-11-06T03:13:14.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 10 client.0 _ll_register_callbacks cb 0x55eb61ce5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58228/0 (socket says 172.21.15.157:58228) 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 -- 172.21.15.157:0/557811391 learned_addr learned my addr 172.21.15.157:0/557811391 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7d627640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f79e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7de28640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 -- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 msgr2=0x55eb61f7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 -- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 msgr2=0x55eb61f79e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f79e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 -- 172.21.15.157:0/557811391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b70009bf0 con 0x55eb61ed8990 2023-11-06T03:13:14.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7ce26640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7b680eb930 tx=0x7f7b6811bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72491fe874b2e4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680ee600 con 0x55eb61ed8990 2023-11-06T03:13:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7b680f6ef0 con 0x55eb61ed8990 2023-11-06T03:13:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb61cea5c0 con 0x55eb61ed8990 2023-11-06T03:13:14.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b680ef170 con 0x55eb61ed8990 2023-11-06T03:13:14.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 10 client.8460 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:14.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55eb61ccc4c0 con 0x55eb61ed8990 2023-11-06T03:13:14.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6810a080 con 0x55eb61ed8990 2023-11-06T03:13:14.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7b680c4050 con 0x55eb61ed8990 2023-11-06T03:13:14.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 10 client.8460 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:14.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 10 client.8460 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:14.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 20 client.8460 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:14.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1c9f0 0x55eb61f80e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 4 client.8460 mds_command: new command op to 8226 tid=0[{"prefix": "scrub pause"}] 2023-11-06T03:13:14.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55eb61d1a2c0 con 0x55eb61f1c9f0 2023-11-06T03:13:14.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7d627640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1c9f0 0x55eb61f80e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 10 client.8460 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7d627640 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1c9f0 0x55eb61f80e20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d23577c020e942 server_cookie=fd8ea18347e4d0bc in_seq=0 out_seq=0 2023-11-06T03:13:14.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 1 -- 172.21.15.157:0/557811391 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55eb61d1a2c0 con 0x55eb61f1c9f0 2023-11-06T03:13:14.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 10 client.8460 handle_command_reply: tid=0 2023-11-06T03:13:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 client.8460 shutdown 2023-11-06T03:13:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 msgr2=0x55eb61f7c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7b680eb930 tx=0x7f7b6811bc90 comp rx=0 tx=0).stop 2023-11-06T03:13:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 shutdown_connections 2023-11-06T03:13:14.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55eb61f1c9f0 0x55eb61f80e20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55eb61f1c0d0 0x55eb61f7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb61ed8990 0x55eb61f7c380 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b57fff640 0 client.8460 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 --2- 172.21.15.157:0/557811391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb61ed2200 0x55eb61f79e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 >> 172.21.15.157:0/557811391 conn(0x55eb61f2a020 msgr2=0x55eb61f1ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 shutdown_connections 2023-11-06T03:13:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 1 -- 172.21.15.157:0/557811391 wait complete. 2023-11-06T03:13:14.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b829b3000 20 client.8460 trim_cache size 0 max 16384 2023-11-06T03:13:14.868 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:14.869 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:13:14.869 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:14.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 msgr2=0x7f7b5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 0x7f7b5c067f20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005910 tx=0x7f7b74007f90 comp rx=0 tx=0).stop 2023-11-06T03:13:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 msgr2=0x7f7b78158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b680ac040 tx=0x7f7b6801ad50 comp rx=0 tx=0).stop 2023-11-06T03:13:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 shutdown_connections 2023-11-06T03:13:14.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7b5c065a60 0x7f7b5c067f20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b7815cc50 0x7f7b780cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b780d31f0 0x7f7b78158170 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 --2- 172.21.15.157:0/1452981043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b780d0b70 0x7f7b780c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 >> 172.21.15.157:0/1452981043 conn(0x7f7b78070790 msgr2=0x7f7b78159560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:14.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.845+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 shutdown_connections 2023-11-06T03:13:14.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.849+0000 7f7b7f88b640 1 -- 172.21.15.157:0/1452981043 wait complete. 2023-11-06T03:13:14.873 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-11-06T03:13:14.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 Processor -- start 2023-11-06T03:13:14.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 -- start start 2023-11-06T03:13:14.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eb01551e0 0x7f8eb006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb00b3b20 0x7f8eb015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:14.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb015f580 con 0x7f8eb015e5e0 2023-11-06T03:13:14.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb015efc0 con 0x7f8eb01551e0 2023-11-06T03:13:14.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:14.997+0000 7f8eb52cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015f2a0 con 0x7f8eb00b3b20 2023-11-06T03:13:14.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8eb01551e0 0x7f8eb006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eae7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8eb015e5e0 0x7f8eb00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49414/0 (socket says 172.21.15.157:49414) 2023-11-06T03:13:14.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eae7fc640 1 -- 172.21.15.157:0/211404426 learned_addr learned my addr 172.21.15.157:0/211404426 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:14.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb00b3b20 0x7f8eb015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:14.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 -- 172.21.15.157:0/211404426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb00b3b20 msgr2=0x7f8eb015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:14.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb00b3b20 0x7f8eb015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:14.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 -- 172.21.15.157:0/211404426 >> v1:172.21.15.157:6789/0 conn(0x7f8eb015e5e0 legacy=0x7f8eb00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:14.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 -- 172.21.15.157:0/211404426 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb006f5a0 con 0x7f8eb01551e0 2023-11-06T03:13:14.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eadffb640 1 -- 172.21.15.157:0/211404426 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185198819 0 0) 0x7f8eb015f580 con 0x7f8eb015e5e0 2023-11-06T03:13:15.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb006fda0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097c0 tx=0x7f8ea001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6bb51f20d3917d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eadffb640 1 -- 172.21.15.157:0/211404426 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00ac640 con 0x7f8eb01551e0 2023-11-06T03:13:15.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eadffb640 1 -- 172.21.15.157:0/211404426 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00ac7a0 con 0x7f8eb01551e0 2023-11-06T03:13:15.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eadffb640 1 -- 172.21.15.157:0/211404426 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00acb30 con 0x7f8eb01551e0 2023-11-06T03:13:15.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 -- 172.21.15.157:0/211404426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 msgr2=0x7f8eb006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb006fda0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097c0 tx=0x7f8ea001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 -- 172.21.15.157:0/211404426 shutdown_connections 2023-11-06T03:13:15.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb00b3b20 0x7f8eb015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/211404426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb006fda0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 -- 172.21.15.157:0/211404426 >> 172.21.15.157:0/211404426 conn(0x7f8eb0065930 msgr2=0x7f8eb0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 -- 172.21.15.157:0/211404426 shutdown_connections 2023-11-06T03:13:15.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.001+0000 7f8eb52cd640 1 -- 172.21.15.157:0/211404426 wait complete. 2023-11-06T03:13:15.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 Processor -- start 2023-11-06T03:13:15.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- start start 2023-11-06T03:13:15.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb00b3b20 0x7f8eb015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb015e5e0 0x7f8eb014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015abf0 con 0x7f8eb01551e0 2023-11-06T03:13:15.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0152390 con 0x7f8eb00b3b20 2023-11-06T03:13:15.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb014d350 con 0x7f8eb015e5e0 2023-11-06T03:13:15.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58250/0 (socket says 172.21.15.157:58250) 2023-11-06T03:13:15.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 -- 172.21.15.157:0/3480508702 learned_addr learned my addr 172.21.15.157:0/3480508702 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb00b3b20 0x7f8eb015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb015e5e0 0x7f8eb014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 -- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb015e5e0 msgr2=0x7f8eb014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb015e5e0 0x7f8eb014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 -- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb00b3b20 msgr2=0x7f8eb015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb00b3b20 0x7f8eb015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 -- 172.21.15.157:0/3480508702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0009420 con 0x7f8eb01551e0 2023-11-06T03:13:15.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c480 tx=0x7f8e9800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8ea77fe640 1 -- 172.21.15.157:0/3480508702 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e980074e0 con 0x7f8eb01551e0 2023-11-06T03:13:15.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8ea77fe640 1 -- 172.21.15.157:0/3480508702 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98007660 con 0x7f8eb01551e0 2023-11-06T03:13:15.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb01f76c0 con 0x7f8eb01551e0 2023-11-06T03:13:15.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8ea77fe640 1 -- 172.21.15.157:0/3480508702 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004a30 con 0x7f8eb01551e0 2023-11-06T03:13:15.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.005+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb01f7c80 con 0x7f8eb01551e0 2023-11-06T03:13:15.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea77fe640 1 -- 172.21.15.157:0/3480508702 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8e98004b90 con 0x7f8eb01551e0 2023-11-06T03:13:15.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea77fe640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 0x7f8e94067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea77fe640 1 -- 172.21.15.157:0/3480508702 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8e98082450 con 0x7f8eb01551e0 2023-11-06T03:13:15.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 0x7f8e94067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 Processor -- start 2023-11-06T03:13:15.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- start start 2023-11-06T03:13:15.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 0x7f8e94067f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00ae000 tx=0x7f8ea0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7cf9c10 con 0x55b7b7e36670 2023-11-06T03:13:15.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c883e0 con 0x55b7b7e80ca0 2023-11-06T03:13:15.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c859b0 con 0x55b7b7e80380 2023-11-06T03:13:15.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58266/0 (socket says 172.21.15.157:58266) 2023-11-06T03:13:15.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 -- 172.21.15.157:0/1382657635 learned_addr learned my addr 172.21.15.157:0/1382657635 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 -- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 msgr2=0x55b7b7e80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 -- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 msgr2=0x55b7b7e36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 -- 172.21.15.157:0/1382657635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7b797de20 con 0x55b7b7e36670 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80073e0 tx=0x7f8ea80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15dd1178722aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea4ff9640 1 -- 172.21.15.157:0/1382657635 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80147e0 con 0x55b7b7e36670 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea4ff9640 1 -- 172.21.15.157:0/1382657635 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea800f050 con 0x55b7b7e36670 2023-11-06T03:13:15.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8ea4ff9640 1 -- 172.21.15.157:0/1382657635 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8014b60 con 0x55b7b7e36670 2023-11-06T03:13:15.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1382657635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 msgr2=0x55b7b7e36a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80073e0 tx=0x7f8ea80078a0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1382657635 shutdown_connections 2023-11-06T03:13:15.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1382657635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1382657635 >> 172.21.15.157:0/1382657635 conn(0x55b7b7e389f0 msgr2=0x55b7b7e30600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1382657635 shutdown_connections 2023-11-06T03:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.009+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1382657635 wait complete. 2023-11-06T03:13:15.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- start start 2023-11-06T03:13:15.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 Processor -- start 2023-11-06T03:13:15.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 0x55b7b7e69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e6ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c812a0 con 0x55b7b7e80380 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c20470 con 0x55b7b7e80ca0 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c3e660 con 0x55b7b7e36670 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 10 client.0 _ll_register_callbacks cb 0x55b7b7e1dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58270/0 (socket says 172.21.15.157:58270) 2023-11-06T03:13:15.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 -- 172.21.15.157:0/1053040939 learned_addr learned my addr 172.21.15.157:0/1053040939 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e6ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 0x55b7b7e69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 -- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 msgr2=0x55b7b7e69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 0x55b7b7e69df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 -- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 msgr2=0x55b7b7e6ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e6ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 -- 172.21.15.157:0/1053040939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea8009c90 con 0x55b7b7e80380 2023-11-06T03:13:15.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 0x55b7b7e69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:15.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c450 tx=0x7f8e98057810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74296f50c81cafd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:15.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9806b020 con 0x55b7b7e80380 2023-11-06T03:13:15.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8e980530d0 con 0x55b7b7e80380 2023-11-06T03:13:15.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7b7c3e0a0 con 0x55b7b7e80380 2023-11-06T03:13:15.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e98001150 con 0x55b7b7e80380 2023-11-06T03:13:15.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 10 client.8475 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:15.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55b7b7c461d0 con 0x55b7b7e80380 2023-11-06T03:13:15.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7b7c3e0a0 con 0x55b7b7e80380 2023-11-06T03:13:15.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8e9805b530 con 0x55b7b7e80380 2023-11-06T03:13:15.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 10 client.8475 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:15.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 10 client.8475 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:15.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 20 client.8475 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:15.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e6b720 0x55b7b7e6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 4 client.8475 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:15.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7b7c7fbf0 con 0x55b7b7e6b720 2023-11-06T03:13:15.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e6b720 0x55b7b7e6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 10 client.8475 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e6b720 0x55b7b7e6bb00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8740a0fd2b125669 server_cookie=378cfbb1338d5217 in_seq=0 out_seq=0 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 1 -- 172.21.15.157:0/1053040939 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b7b7c7fbf0 con 0x55b7b7e6b720 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 10 client.8475 handle_command_reply: tid=0 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 client.8475 shutdown 2023-11-06T03:13:15.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 msgr2=0x55b7b7e6a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c450 tx=0x7f8e98057810 comp rx=0 tx=0).stop 2023-11-06T03:13:15.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 shutdown_connections 2023-11-06T03:13:15.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e6b720 0x55b7b7e6bb00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e6ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e80380 0x55b7b7e6a7d0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8e9dffb640 0 client.8475 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:15.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/1053040939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e36670 0x55b7b7e69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 >> 172.21.15.157:0/1053040939 conn(0x55b7b7e389f0 msgr2=0x55b7b7e2fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 shutdown_connections 2023-11-06T03:13:15.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 1 -- 172.21.15.157:0/1053040939 wait complete. 2023-11-06T03:13:15.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.013+0000 7f8eb83f5000 20 client.8475 trim_cache size 0 max 16384 2023-11-06T03:13:15.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 Processor -- start 2023-11-06T03:13:15.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 -- start start 2023-11-06T03:13:15.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80380 0x55b7b7e36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80ca0 0x55b7b7e69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b793eaa0 con 0x55b7b7e36670 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c7ffc0 con 0x55b7b7e80380 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7e96ae0 con 0x55b7b7e80ca0 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58282/0 (socket says 172.21.15.157:58282) 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 -- 172.21.15.157:0/2570983391 learned_addr learned my addr 172.21.15.157:0/2570983391 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 -- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80ca0 msgr2=0x55b7b7e69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80ca0 0x55b7b7e69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 -- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80380 msgr2=0x55b7b7e36f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80380 0x55b7b7e36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80380 0x55b7b7e36f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 -- 172.21.15.157:0/2570983391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7b7c2c520 con 0x55b7b7e36670 2023-11-06T03:13:15.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097c0 tx=0x7f8ea0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53896cc3788b1a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8ea57fa640 1 -- 172.21.15.157:0/2570983391 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00bc040 con 0x55b7b7e36670 2023-11-06T03:13:15.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8ea57fa640 1 -- 172.21.15.157:0/2570983391 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00bd040 con 0x55b7b7e36670 2023-11-06T03:13:15.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.029+0000 7f8ea57fa640 1 -- 172.21.15.157:0/2570983391 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00c6420 con 0x55b7b7e36670 2023-11-06T03:13:15.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/2570983391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 msgr2=0x55b7b7e36a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00097c0 tx=0x7f8ea0004740 comp rx=0 tx=0).stop 2023-11-06T03:13:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/2570983391 shutdown_connections 2023-11-06T03:13:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80ca0 0x55b7b7e69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80380 0x55b7b7e36f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/2570983391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7e36a50 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/2570983391 >> 172.21.15.157:0/2570983391 conn(0x55b7b7e389f0 msgr2=0x55b7b7e311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/2570983391 shutdown_connections 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/2570983391 wait complete. 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- start start 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 Processor -- start 2023-11-06T03:13:15.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e82c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e83150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7e7fdd0 con 0x55b7b7e36670 2023-11-06T03:13:15.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7c80130 con 0x55b7b7e80ca0 2023-11-06T03:13:15.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7b7e3dbd0 con 0x55b7b7e80380 2023-11-06T03:13:15.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 10 client.0 _ll_register_callbacks cb 0x55b7b7c45cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:15.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58290/0 (socket says 172.21.15.157:58290) 2023-11-06T03:13:15.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 -- 172.21.15.157:0/435886679 learned_addr learned my addr 172.21.15.157:0/435886679 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 -- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 msgr2=0x55b7b7e82c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaf7fe640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e83150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e82c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 -- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 msgr2=0x55b7b7e83150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e83150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 -- 172.21.15.157:0/435886679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb006f5a0 con 0x55b7b7e36670 2023-11-06T03:13:15.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e82c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:15.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eaeffd640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00adfd0 tx=0x7f8ea001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b02c8803b8342c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:15.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00bc040 con 0x55b7b7e36670 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8ea00c6880 con 0x55b7b7e36670 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7b7c4a620 con 0x55b7b7e36670 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea00c6a40 con 0x55b7b7e36670 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 10 client.8481 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55b7b7c20090 con 0x55b7b7e36670 2023-11-06T03:13:15.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7b7c4a620 con 0x55b7b7e36670 2023-11-06T03:13:15.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8ea00c6ba0 con 0x55b7b7e36670 2023-11-06T03:13:15.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 10 client.8481 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:15.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 10 client.8481 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:15.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 20 client.8481 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:15.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e30480 0x55b7b7e83690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 4 client.8481 mds_command: new command op to 8226 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:13:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7b7c7a320 con 0x55b7b7e30480 2023-11-06T03:13:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e30480 0x55b7b7e83690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eae7fc640 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e30480 0x55b7b7e83690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50837d948f7dc61 server_cookie=56303ba598784d58 in_seq=0 out_seq=0 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 10 client.8481 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 1 -- 172.21.15.157:0/435886679 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55b7b7c7a320 con 0x55b7b7e30480 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 10 client.8481 handle_command_reply: tid=0 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 client.8481 shutdown 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 msgr2=0x55b7b7edcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00adfd0 tx=0x7f8ea001ac00 comp rx=0 tx=0).stop 2023-11-06T03:13:15.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 shutdown_connections 2023-11-06T03:13:15.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x55b7b7e30480 0x55b7b7e83690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b7b7e80ca0 0x55b7b7e83150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b7b7e80380 0x55b7b7e82c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8e9dffb640 0 client.8481 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:15.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 --2- 172.21.15.157:0/435886679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7b7e36670 0x55b7b7edcb40 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 >> 172.21.15.157:0/435886679 conn(0x55b7b7e389f0 msgr2=0x55b7b7e2fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 shutdown_connections 2023-11-06T03:13:15.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.033+0000 7f8eb83f5000 1 -- 172.21.15.157:0/435886679 wait complete. 2023-11-06T03:13:15.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb83f5000 20 client.8481 trim_cache size 0 max 16384 2023-11-06T03:13:15.054 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-11-06T03:13:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:13:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "73e5c3fc-0554-4677-b495-99a29248aef5": { 2023-11-06T03:13:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_control_test_path", 2023-11-06T03:13:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "73e5c3fc-0554-4677-b495-99a29248aef5", 2023-11-06T03:13:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:13:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:15.056 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:15.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 msgr2=0x7f8e94067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 0x7f8e94067f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00ae000 tx=0x7f8ea0008040 comp rx=0 tx=0).stop 2023-11-06T03:13:15.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 msgr2=0x7f8eb01555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c480 tx=0x7f8e9800c940 comp rx=0 tx=0).stop 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 shutdown_connections 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e94065a90 0x7f8e94067f50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8eb015e5e0 0x7f8eb014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8eb01551e0 0x7f8eb01555c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 --2- 172.21.15.157:0/3480508702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8eb00b3b20 0x7f8eb015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 >> 172.21.15.157:0/3480508702 conn(0x7f8eb0065930 msgr2=0x7f8eb014dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 shutdown_connections 2023-11-06T03:13:15.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.037+0000 7f8eb52cd640 1 -- 172.21.15.157:0/3480508702 wait complete. 2023-11-06T03:13:15.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:15.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 Processor -- start 2023-11-06T03:13:15.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 -- start start 2023-11-06T03:13:15.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8cac146370 0x7f8cac0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac0ce800 0x7f8cac0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac0cf1a0 con 0x7f8cac06b6a0 2023-11-06T03:13:15.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac06c3d0 con 0x7f8cac146370 2023-11-06T03:13:15.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cb1fd9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac06f5a0 con 0x7f8cac0ce800 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cab7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8cac146370 0x7f8cac0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cabfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8cac06b6a0 0x7f8cac06bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49426/0 (socket says 172.21.15.157:49426) 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cabfff640 1 -- 172.21.15.157:0/1255236350 learned_addr learned my addr 172.21.15.157:0/1255236350 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8caaffd640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac0ce800 0x7f8cac0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cab7fe640 1 -- 172.21.15.157:0/1255236350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac0ce800 msgr2=0x7f8cac0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8caa7fc640 1 -- 172.21.15.157:0/1255236350 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185064959 0 0) 0x7f8cac0cf1a0 con 0x7f8cac06b6a0 2023-11-06T03:13:15.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cab7fe640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac0ce800 0x7f8cac0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cab7fe640 1 -- 172.21.15.157:0/1255236350 >> v1:172.21.15.157:6789/0 conn(0x7f8cac06b6a0 legacy=0x7f8cac06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.177+0000 7f8cab7fe640 1 -- 172.21.15.157:0/1255236350 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac06f880 con 0x7f8cac146370 2023-11-06T03:13:15.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cab7fe640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac146370 0x7f8cac0b1a10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00097c0 tx=0x7f8ca001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7775cc5dcfa2eff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8caa7fc640 1 -- 172.21.15.157:0/1255236350 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00ab5f0 con 0x7f8cac146370 2023-11-06T03:13:15.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8caa7fc640 1 -- 172.21.15.157:0/1255236350 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca00ab750 con 0x7f8cac146370 2023-11-06T03:13:15.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8caa7fc640 1 -- 172.21.15.157:0/1255236350 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00abae0 con 0x7f8cac146370 2023-11-06T03:13:15.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1255236350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac146370 msgr2=0x7f8cac0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac146370 0x7f8cac0b1a10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00097c0 tx=0x7f8ca001c3f0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1255236350 shutdown_connections 2023-11-06T03:13:15.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac0ce800 0x7f8cac0cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1255236350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac146370 0x7f8cac0b1a10 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1255236350 >> 172.21.15.157:0/1255236350 conn(0x7f8cac065930 msgr2=0x7f8cac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1255236350 shutdown_connections 2023-11-06T03:13:15.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.181+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1255236350 wait complete. 2023-11-06T03:13:15.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 Processor -- start 2023-11-06T03:13:15.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- start start 2023-11-06T03:13:15.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cac06b6a0 0x7f8cac145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac146370 0x7f8cac13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac155a60 con 0x7f8cac0ce800 2023-11-06T03:13:15.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac155130 con 0x7f8cac06b6a0 2023-11-06T03:13:15.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac155410 con 0x7f8cac146370 2023-11-06T03:13:15.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58304/0 (socket says 172.21.15.157:58304) 2023-11-06T03:13:15.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 -- 172.21.15.157:0/1824927974 learned_addr learned my addr 172.21.15.157:0/1824927974 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cabfff640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac146370 0x7f8cac13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cab7fe640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cac06b6a0 0x7f8cac145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 -- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac146370 msgr2=0x7f8cac13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac146370 0x7f8cac13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 -- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cac06b6a0 msgr2=0x7f8cac145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cac06b6a0 0x7f8cac145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 -- 172.21.15.157:0/1824927974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0009420 con 0x7f8cac0ce800 2023-11-06T03:13:15.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8caaffd640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00e870 tx=0x7f8c9c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c009510 con 0x7f8cac0ce800 2023-11-06T03:13:15.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00eec0 con 0x7f8cac0ce800 2023-11-06T03:13:15.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac1556f0 con 0x7f8cac0ce800 2023-11-06T03:13:15.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00cb40 con 0x7f8cac0ce800 2023-11-06T03:13:15.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.185+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac143680 con 0x7f8cac0ce800 2023-11-06T03:13:15.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac155880 con 0x7f8cac0ce800 2023-11-06T03:13:15.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c9c00cca0 con 0x7f8cac0ce800 2023-11-06T03:13:15.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8c93fff640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 0x7f8c98067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8c9c083340 con 0x7f8cac0ce800 2023-11-06T03:13:15.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8cab7fe640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 0x7f8c98067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.189+0000 7f8cab7fe640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 0x7f8c98067f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00af000 tx=0x7f8ca00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.193+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8c9c050c10 con 0x7f8cac0ce800 2023-11-06T03:13:15.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cac072750 con 0x7f8cac0ce800 2023-11-06T03:13:15.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8c93fff640 1 -- 172.21.15.157:0/1824927974 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8c9c054230 con 0x7f8cac0ce800 2023-11-06T03:13:15.378 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:15.379 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:15.379 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:15.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 msgr2=0x7f8c98067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 0x7f8c98067f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00af000 tx=0x7f8ca00a7000 comp rx=0 tx=0).stop 2023-11-06T03:13:15.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 msgr2=0x7f8cac143140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00e870 tx=0x7f8c9c00ec70 comp rx=0 tx=0).stop 2023-11-06T03:13:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 shutdown_connections 2023-11-06T03:13:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c98065a90 0x7f8c98067f50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8cac146370 0x7f8cac13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cac0ce800 0x7f8cac143140 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 --2- 172.21.15.157:0/1824927974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cac06b6a0 0x7f8cac145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.381+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 >> 172.21.15.157:0/1824927974 conn(0x7f8cac065930 msgr2=0x7f8cac0ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.385+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 shutdown_connections 2023-11-06T03:13:15.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.385+0000 7f8cb1fd9640 1 -- 172.21.15.157:0/1824927974 wait complete. 2023-11-06T03:13:15.396 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:15.397 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:15.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.509+0000 7f510fe39640 1 Processor -- start 2023-11-06T03:13:15.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.509+0000 7f510fe39640 1 -- start start 2023-11-06T03:13:15.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f510815ce80 0x7f51080cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f51080d00b0 0x7f51080b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080d2920 con 0x7f510815caa0 2023-11-06T03:13:15.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080d0520 con 0x7f51080d00b0 2023-11-06T03:13:15.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080d18a0 con 0x7f510815ce80 2023-11-06T03:13:15.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510dbd5640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f510815caa0 0x7f51080cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49438/0 (socket says 172.21.15.157:49438) 2023-11-06T03:13:15.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510dbd5640 1 -- 172.21.15.157:0/3951233221 learned_addr learned my addr 172.21.15.157:0/3951233221 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:15.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 --2- 172.21.15.157:0/3951233221 >> v2:172.21.15.157:3300/0 conn(0x7f51080d00b0 0x7f51080b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510d3d4640 1 --2- 172.21.15.157:0/3951233221 >> v2:172.21.15.195:3300/0 conn(0x7f510815ce80 0x7f51080cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510cbd3640 1 -- 172.21.15.157:0/3951233221 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142846439 0 0) 0x7f51080d2920 con 0x7f510815caa0 2023-11-06T03:13:15.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510cbd3640 1 -- 172.21.15.157:0/3951233221 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f4003530 con 0x7f510815caa0 2023-11-06T03:13:15.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 -- 172.21.15.157:0/3951233221 >> v2:172.21.15.195:3300/0 conn(0x7f510815ce80 msgr2=0x7f51080cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:15.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 --2- 172.21.15.157:0/3951233221 >> v2:172.21.15.195:3300/0 conn(0x7f510815ce80 0x7f51080cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 -- 172.21.15.157:0/3951233221 >> v1:172.21.15.157:6789/0 conn(0x7f510815caa0 legacy=0x7f51080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 -- 172.21.15.157:0/3951233221 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080d1b60 con 0x7f51080d00b0 2023-11-06T03:13:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510e3d6640 1 --2- 172.21.15.157:0/3951233221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080d00b0 0x7f51080b5d50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f51080c6720 tx=0x7f510001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92ccf0074e4e95f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510cbd3640 1 -- 172.21.15.157:0/3951233221 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000a59a0 con 0x7f51080d00b0 2023-11-06T03:13:15.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510cbd3640 1 -- 172.21.15.157:0/3951233221 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51000a5b00 con 0x7f51080d00b0 2023-11-06T03:13:15.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510cbd3640 1 -- 172.21.15.157:0/3951233221 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000a5e90 con 0x7f51080d00b0 2023-11-06T03:13:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- 172.21.15.157:0/3951233221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080d00b0 msgr2=0x7f51080b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 --2- 172.21.15.157:0/3951233221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080d00b0 0x7f51080b5d50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f51080c6720 tx=0x7f510001c8e0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- 172.21.15.157:0/3951233221 shutdown_connections 2023-11-06T03:13:15.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 --2- 172.21.15.157:0/3951233221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080d00b0 0x7f51080b5d50 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 --2- 172.21.15.157:0/3951233221 >> v2:172.21.15.195:3300/0 conn(0x7f510815ce80 0x7f51080cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- 172.21.15.157:0/3951233221 >> 172.21.15.157:0/3951233221 conn(0x7f5108070790 msgr2=0x7f5108070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- 172.21.15.157:0/3951233221 shutdown_connections 2023-11-06T03:13:15.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.513+0000 7f510fe39640 1 -- 172.21.15.157:0/3951233221 wait complete. 2023-11-06T03:13:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 Processor -- start 2023-11-06T03:13:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- start start 2023-11-06T03:13:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51080d00b0 0x7f51080a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f510815ce80 0x7f51080d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108156640 con 0x7f510815caa0 2023-11-06T03:13:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108158ae0 con 0x7f510815ce80 2023-11-06T03:13:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5108158650 con 0x7f51080d00b0 2023-11-06T03:13:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58326/0 (socket says 172.21.15.157:58326) 2023-11-06T03:13:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 -- 172.21.15.157:0/4148377429 learned_addr learned my addr 172.21.15.157:0/4148377429 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510e3d6640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f510815ce80 0x7f51080d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510dbd5640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51080d00b0 0x7f51080a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 -- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51080d00b0 msgr2=0x7f51080a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51080d00b0 0x7f51080a9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 -- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f510815ce80 msgr2=0x7f51080d2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f510815ce80 0x7f51080d2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 -- 172.21.15.157:0/4148377429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009a70 con 0x7f510815caa0 2023-11-06T03:13:15.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510d3d4640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f50f800d7d0 tx=0x7f50f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f800b660 con 0x7f510815caa0 2023-11-06T03:13:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f800b7c0 con 0x7f510815caa0 2023-11-06T03:13:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5108157a40 con 0x7f510815caa0 2023-11-06T03:13:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f800bb70 con 0x7f510815caa0 2023-11-06T03:13:15.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081582e0 con 0x7f510815caa0 2023-11-06T03:13:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108072750 con 0x7f510815caa0 2023-11-06T03:13:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.517+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f50f800bcd0 con 0x7f510815caa0 2023-11-06T03:13:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.521+0000 7f50fe7fc640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 0x7f50ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:15.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.521+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f50f80837c0 con 0x7f510815caa0 2023-11-06T03:13:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.521+0000 7f510dbd5640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 0x7f50ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.521+0000 7f510dbd5640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 0x7f50ec067f20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f51040011f0 tx=0x7f5104009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.525+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f50f8051000 con 0x7f510815caa0 2023-11-06T03:13:15.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.729+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5108158790 con 0x7f510815caa0 2023-11-06T03:13:15.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f50fe7fc640 1 -- 172.21.15.157:0/4148377429 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1447 (secure 0 0 0) 0x7f50f8054620 con 0x7f510815caa0 2023-11-06T03:13:15.728 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:15.728 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":573,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1038,"data_bytes":1066433335,"bytes_used":2686144512,"bytes_avail":770407968768,"bytes_total":773094113280,"write_bytes_sec":80056692,"read_op_per_sec":0,"write_op_per_sec":77},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2023-11-06T03:12:35.281372+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:15.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 msgr2=0x7f50ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 0x7f50ec067f20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f51040011f0 tx=0x7f5104009040 comp rx=0 tx=0).stop 2023-11-06T03:13:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 msgr2=0x7f51081571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f50f800d7d0 tx=0x7f50f800dc90 comp rx=0 tx=0).stop 2023-11-06T03:13:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 shutdown_connections 2023-11-06T03:13:15.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50ec065a60 0x7f50ec067f20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f510815ce80 0x7f51080d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f510815caa0 0x7f51081571f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 --2- 172.21.15.157:0/4148377429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51080d00b0 0x7f51080a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 >> 172.21.15.157:0/4148377429 conn(0x7f5108070790 msgr2=0x7f51080cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:15.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 shutdown_connections 2023-11-06T03:13:15.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:15.733+0000 7f510fe39640 1 -- 172.21.15.157:0/4148377429 wait complete. 2023-11-06T03:13:15.746 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:13:16.747 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.877+0000 7f7609b52640 1 Processor -- start 2023-11-06T03:13:16.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7609b52640 1 -- start start 2023-11-06T03:13:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7609b52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f760415e5e0 0x7f76040b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7609b52640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760415f580 con 0x7f760415ea90 2023-11-06T03:13:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7609b52640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760415efc0 con 0x7f76041551e0 2023-11-06T03:13:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7609b52640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040704b0 con 0x7f760415e5e0 2023-11-06T03:13:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7603fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f760415ea90 0x7f760415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49446/0 (socket says 172.21.15.157:49446) 2023-11-06T03:13:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7603fff640 1 -- 172.21.15.157:0/2416596320 learned_addr learned my addr 172.21.15.157:0/2416596320 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f75fbfff640 1 --2- 172.21.15.157:0/2416596320 >> v2:172.21.15.157:3300/0 conn(0x7f760415e5e0 0x7f76040b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190608113 0 0) 0x7f760415f580 con 0x7f760415ea90 2023-11-06T03:13:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f0003530 con 0x7f760415ea90 2023-11-06T03:13:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f75fbfff640 1 -- 172.21.15.157:0/2416596320 >> v1:172.21.15.195:6789/0 conn(0x7f76041551e0 legacy=0x7f760406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f75fbfff640 1 -- 172.21.15.157:0/2416596320 >> v1:172.21.15.157:6789/0 conn(0x7f760415ea90 legacy=0x7f760415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758775970 0 0) 0x7f75f0003530 con 0x7f760415ea90 2023-11-06T03:13:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.881+0000 7f75fbfff640 1 -- 172.21.15.157:0/2416596320 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760415f2a0 con 0x7f760415e5e0 2023-11-06T03:13:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f75fbfff640 1 --2- 172.21.15.157:0/2416596320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415e5e0 0x7f76040b3410 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097c0 tx=0x7f75ec01c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1355a29c68f76c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc001a00 con 0x7f760415e5e0 2023-11-06T03:13:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec0a7050 con 0x7f760415e5e0 2023-11-06T03:13:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7602ffd640 1 -- 172.21.15.157:0/2416596320 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0ab400 con 0x7f760415e5e0 2023-11-06T03:13:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/2416596320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415e5e0 msgr2=0x7f76040b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 --2- 172.21.15.157:0/2416596320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415e5e0 0x7f76040b3410 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097c0 tx=0x7f75ec01c2e0 comp rx=0 tx=0).stop 2023-11-06T03:13:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/2416596320 shutdown_connections 2023-11-06T03:13:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 --2- 172.21.15.157:0/2416596320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415e5e0 0x7f76040b3410 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/2416596320 >> 172.21.15.157:0/2416596320 conn(0x7f7604065930 msgr2=0x7f7604065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/2416596320 shutdown_connections 2023-11-06T03:13:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/2416596320 wait complete. 2023-11-06T03:13:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 Processor -- start 2023-11-06T03:13:16.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- start start 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76041551e0 0x7f7604152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 0x7f7604157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604152db0 con 0x7f760415ea90 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604153090 con 0x7f760415e5e0 2023-11-06T03:13:16.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7604153370 con 0x7f76041551e0 2023-11-06T03:13:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58340/0 (socket says 172.21.15.157:58340) 2023-11-06T03:13:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 -- 172.21.15.157:0/1025897996 learned_addr learned my addr 172.21.15.157:0/1025897996 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f75fbfff640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 0x7f7604157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:16.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 -- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76041551e0 msgr2=0x7f7604152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f76037fe640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76041551e0 0x7f7604152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76041551e0 0x7f7604152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 -- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 msgr2=0x7f7604157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 0x7f7604157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:16.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 -- 172.21.15.157:0/1025897996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009420 con 0x7f760415ea90 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f75fbfff640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 0x7f7604157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7603fff640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00cc00 tx=0x7f75fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00ce10 con 0x7f760415ea90 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc012400 con 0x7f760415ea90 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7604153650 con 0x7f760415ea90 2023-11-06T03:13:16.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc0127b0 con 0x7f760415ea90 2023-11-06T03:13:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.885+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7604156590 con 0x7f760415ea90 2023-11-06T03:13:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7604072750 con 0x7f760415ea90 2023-11-06T03:13:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75fc012910 con 0x7f760415ea90 2023-11-06T03:13:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f7600ff9640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 0x7f75e4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:16.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f75fc0839f0 con 0x7f760415ea90 2023-11-06T03:13:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f76037fe640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 0x7f75e4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.889+0000 7f76037fe640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 0x7f75e4067f50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:16.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:16.893+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f75fc051230 con 0x7f760415ea90 2023-11-06T03:13:17.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.081+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7604153b00 con 0x7f760415ea90 2023-11-06T03:13:17.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.081+0000 7f7600ff9640 1 -- 172.21.15.157:0/1025897996 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f75fc054850 con 0x7f760415ea90 2023-11-06T03:13:17.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:17.079 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:17.080 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 msgr2=0x7f75e4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 0x7f75e4067f50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).stop 2023-11-06T03:13:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 msgr2=0x7f76040b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00cc00 tx=0x7f75fc00a430 comp rx=0 tx=0).stop 2023-11-06T03:13:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 shutdown_connections 2023-11-06T03:13:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f75e4065a90 0x7f75e4067f50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f760415ea90 0x7f76040b1dd0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f760415e5e0 0x7f7604157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 --2- 172.21.15.157:0/1025897996 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76041551e0 0x7f7604152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 >> 172.21.15.157:0/1025897996 conn(0x7f7604065930 msgr2=0x7f7604067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 shutdown_connections 2023-11-06T03:13:17.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.085+0000 7f7609b52640 1 -- 172.21.15.157:0/1025897996 wait complete. 2023-11-06T03:13:17.098 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:17.099 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:17.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.221+0000 7f84de449640 1 Processor -- start 2023-11-06T03:13:17.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.221+0000 7f84de449640 1 -- start start 2023-11-06T03:13:17.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84de449640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 0x7f84d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84de449640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80d2920 con 0x7f84d815caa0 2023-11-06T03:13:17.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84de449640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80d18a0 con 0x7f84d80d00b0 2023-11-06T03:13:17.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84de449640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80d0520 con 0x7f84d815ce80 2023-11-06T03:13:17.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84d815caa0 0x7f84d80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49460/0 (socket says 172.21.15.157:49460) 2023-11-06T03:13:17.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 -- 172.21.15.157:0/2516596396 learned_addr learned my addr 172.21.15.157:0/2516596396 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:17.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84cffff640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 0x7f84d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000683166 0 0) 0x7f84d80d2920 con 0x7f84d815caa0 2023-11-06T03:13:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c4003530 con 0x7f84d815caa0 2023-11-06T03:13:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2022730015 0 0) 0x7f84c4003530 con 0x7f84d815caa0 2023-11-06T03:13:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 msgr2=0x7f84d80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:17.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 0x7f84d80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 >> v1:172.21.15.195:6789/0 conn(0x7f84d80d00b0 legacy=0x7f84d80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80d1b60 con 0x7f84d815caa0 2023-11-06T03:13:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f84c8001680 con 0x7f84d815caa0 2023-11-06T03:13:17.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 >> v1:172.21.15.157:6789/0 conn(0x7f84d815caa0 legacy=0x7f84d80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 0x7f84c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80d0520 con 0x7f84c40039b0 2023-11-06T03:13:17.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400ba70 con 0x7f84c4004d10 2023-11-06T03:13:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400bc30 con 0x7f84d815ce80 2023-11-06T03:13:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84dc9e6640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 -- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 msgr2=0x7f84c40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:17.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d815ce80 0x7f84c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 -- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 msgr2=0x7f84c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.225+0000 7f84d7fff640 1 -- 172.21.15.157:0/2516596396 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4003530 con 0x7f84c40039b0 2023-11-06T03:13:17.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003dd0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c210 tx=0x7f84c801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d41ade87135892 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:17.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8005ae0 con 0x7f84c40039b0 2023-11-06T03:13:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8005c40 con 0x7f84c40039b0 2023-11-06T03:13:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d77fe640 1 -- 172.21.15.157:0/2516596396 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c80a57e0 con 0x7f84c40039b0 2023-11-06T03:13:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/2516596396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 msgr2=0x7f84c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 --2- 172.21.15.157:0/2516596396 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003dd0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c210 tx=0x7f84c801cbf0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84dc9e6640 1 -- 172.21.15.157:0/2516596396 reap_dead start 2023-11-06T03:13:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/2516596396 shutdown_connections 2023-11-06T03:13:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/2516596396 >> 172.21.15.157:0/2516596396 conn(0x7f84d8070790 msgr2=0x7f84d8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/2516596396 shutdown_connections 2023-11-06T03:13:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/2516596396 wait complete. 2023-11-06T03:13:17.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 Processor -- start 2023-11-06T03:13:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- start start 2023-11-06T03:13:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84d81f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d815f720 0x7f84d815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81600e0 con 0x7f84d81f62e0 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81f5500 con 0x7f84d815f720 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80ccb00 con 0x7f84c40039b0 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58358/0 (socket says 172.21.15.157:58358) 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 -- 172.21.15.157:0/168333106 learned_addr learned my addr 172.21.15.157:0/168333106 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:17.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84dc9e6640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d815f720 0x7f84d815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d7fff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84d81f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 -- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 msgr2=0x7f84d81f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84d81f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 -- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d815f720 msgr2=0x7f84d815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d815f720 0x7f84d815fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 -- 172.21.15.157:0/168333106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80cbef0 con 0x7f84d81f62e0 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84cffff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e7f0 tx=0x7f84c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000c940 con 0x7f84d81f62e0 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c000caa0 con 0x7f84d81f62e0 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80cc1d0 con 0x7f84d81f62e0 2023-11-06T03:13:17.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000ce50 con 0x7f84d81f62e0 2023-11-06T03:13:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.229+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d80cc790 con 0x7f84d81f62e0 2023-11-06T03:13:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d80ccc90 con 0x7f84d81f62e0 2023-11-06T03:13:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84c0013470 con 0x7f84d81f62e0 2023-11-06T03:13:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84d57fa640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 0x7f84b8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:17.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f84c0083300 con 0x7f84d81f62e0 2023-11-06T03:13:17.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84d7fff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 0x7f84b8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:17.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.233+0000 7f84d7fff640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 0x7f84b8067f50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f84c80ab000 tx=0x7f84c80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:17.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.237+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84c0050b40 con 0x7f84d81f62e0 2023-11-06T03:13:17.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.457+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f84d80ccf50 con 0x7f84d81f62e0 2023-11-06T03:13:17.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.457+0000 7f84d57fa640 1 -- 172.21.15.157:0/168333106 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1447 (secure 0 0 0) 0x7f84c0054160 con 0x7f84d81f62e0 2023-11-06T03:13:17.455 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:17.455 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":574,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1038,"data_bytes":1066433335,"bytes_used":2686144512,"bytes_avail":770407968768,"bytes_total":773094113280,"write_bytes_sec":80056692,"read_op_per_sec":0,"write_op_per_sec":77},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2023-11-06T03:12:35.281372+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:17.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 msgr2=0x7f84b8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 0x7f84b8067f50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f84c80ab000 tx=0x7f84c80a7070 comp rx=0 tx=0).stop 2023-11-06T03:13:17.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 msgr2=0x7f84d80cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:17.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e7f0 tx=0x7f84c000ecb0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 shutdown_connections 2023-11-06T03:13:17.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a90 0x7f84b8067f50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d815f720 0x7f84d815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d81f62e0 0x7f84d80cb5e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 --2- 172.21.15.157:0/168333106 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84d81f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:17.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 >> 172.21.15.157:0/168333106 conn(0x7f84d8070790 msgr2=0x7f84d80b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:17.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 shutdown_connections 2023-11-06T03:13:17.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:17.461+0000 7f84de449640 1 -- 172.21.15.157:0/168333106 wait complete. 2023-11-06T03:13:17.473 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:13:18.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:18.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.609+0000 7f3abf990640 1 Processor -- start 2023-11-06T03:13:18.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.609+0000 7f3abf990640 1 -- start start 2023-11-06T03:13:18.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.609+0000 7f3abf990640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 0x7f3ab80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.609+0000 7f3abf990640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.609+0000 7f3abf990640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80d2920 con 0x7f3ab80ce1a0 2023-11-06T03:13:18.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abf990640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80d18a0 con 0x7f3ab815caa0 2023-11-06T03:13:18.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abf990640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80d1b60 con 0x7f3ab815ce80 2023-11-06T03:13:18.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abd72c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 0x7f3ab80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abd72c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 0x7f3ab80cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:59896/0 (socket says 172.21.15.157:59896) 2023-11-06T03:13:18.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abdf2d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f3ab80ce1a0 0x7f3ab80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:42272/0 (socket says 172.21.15.157:42272) 2023-11-06T03:13:18.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abd72c640 1 -- 172.21.15.157:0/2922655059 learned_addr learned my addr 172.21.15.157:0/2922655059 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:18.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 -- 172.21.15.157:0/2922655059 >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 msgr2=0x7f3ab80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:18.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 --2- 172.21.15.157:0/2922655059 >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 0x7f3ab80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 -- 172.21.15.157:0/2922655059 >> v1:172.21.15.195:6789/0 conn(0x7f3ab80ce1a0 legacy=0x7f3ab80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 -- 172.21.15.157:0/2922655059 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80d1e40 con 0x7f3ab815ce80 2023-11-06T03:13:18.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3abcf2b640 1 --2- 172.21.15.157:0/2922655059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80cfb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80097c0 tx=0x7f3aa801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c92b4f355035509 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3aaffff640 1 -- 172.21.15.157:0/2922655059 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80ad660 con 0x7f3ab815ce80 2023-11-06T03:13:18.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3aaffff640 1 -- 172.21.15.157:0/2922655059 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80ad7c0 con 0x7f3ab815ce80 2023-11-06T03:13:18.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.613+0000 7f3aaffff640 1 -- 172.21.15.157:0/2922655059 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80adb50 con 0x7f3ab815ce80 2023-11-06T03:13:18.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- 172.21.15.157:0/2922655059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 msgr2=0x7f3ab80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- 172.21.15.157:0/2922655059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80cfb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80097c0 tx=0x7f3aa801c450 comp rx=0 tx=0).stop 2023-11-06T03:13:18.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- 172.21.15.157:0/2922655059 shutdown_connections 2023-11-06T03:13:18.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- 172.21.15.157:0/2922655059 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80cfb70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- 172.21.15.157:0/2922655059 >> v2:172.21.15.195:3300/0 conn(0x7f3ab815caa0 0x7f3ab80cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- 172.21.15.157:0/2922655059 >> 172.21.15.157:0/2922655059 conn(0x7f3ab8070790 msgr2=0x7f3ab8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:18.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- 172.21.15.157:0/2922655059 shutdown_connections 2023-11-06T03:13:18.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- 172.21.15.157:0/2922655059 wait complete. 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 Processor -- start 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- start start 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ab80ce1a0 0x7f3ab815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b02b0 con 0x7f3ab815caa0 2023-11-06T03:13:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80ae390 con 0x7f3ab80ce1a0 2023-11-06T03:13:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abf990640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b04a0 con 0x7f3ab815ce80 2023-11-06T03:13:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50734/0 (socket says 172.21.15.157:50734) 2023-11-06T03:13:18.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 -- 172.21.15.157:0/201495661 learned_addr learned my addr 172.21.15.157:0/201495661 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abd72c640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ab80ce1a0 0x7f3ab815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abdf2d640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 -- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 msgr2=0x7f3ab80ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 -- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ab80ce1a0 msgr2=0x7f3ab815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ab80ce1a0 0x7f3ab815a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 -- 172.21.15.157:0/201495661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8009420 con 0x7f3ab815caa0 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.617+0000 7f3abcf2b640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000c10 tx=0x7f3aa80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b7610 con 0x7f3ab815caa0 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80b77b0 con 0x7f3ab815caa0 2023-11-06T03:13:18.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80b0780 con 0x7f3ab815caa0 2023-11-06T03:13:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b7b60 con 0x7f3ab815caa0 2023-11-06T03:13:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80b0d40 con 0x7f3ab815caa0 2023-11-06T03:13:18.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab8072750 con 0x7f3ab815caa0 2023-11-06T03:13:18.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3aa80b4050 con 0x7f3ab815caa0 2023-11-06T03:13:18.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 0x7f3a9c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3aa811cdf0 con 0x7f3ab815caa0 2023-11-06T03:13:18.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3abd72c640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 0x7f3a9c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.621+0000 7f3abd72c640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 0x7f3a9c067f50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009d80 tx=0x7f3ab400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.625+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3aa80ea750 con 0x7f3ab815caa0 2023-11-06T03:13:18.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.821+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ab80b0f30 con 0x7f3ab815caa0 2023-11-06T03:13:18.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3aadffb640 1 -- 172.21.15.157:0/201495661 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f3aa80edd70 con 0x7f3ab815caa0 2023-11-06T03:13:18.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:18.821 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:18.822 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:18.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 msgr2=0x7f3a9c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 0x7f3a9c067f50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009d80 tx=0x7f3ab400b040 comp rx=0 tx=0).stop 2023-11-06T03:13:18.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 msgr2=0x7f3ab80ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000c10 tx=0x7f3aa80191f0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 shutdown_connections 2023-11-06T03:13:18.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a9c065a90 0x7f3a9c067f50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ab815ce80 0x7f3ab80ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ab815caa0 0x7f3ab80ad350 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 --2- 172.21.15.157:0/201495661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ab80ce1a0 0x7f3ab815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 >> 172.21.15.157:0/201495661 conn(0x7f3ab8070790 msgr2=0x7f3ab80b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:18.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 shutdown_connections 2023-11-06T03:13:18.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.825+0000 7f3abf990640 1 -- 172.21.15.157:0/201495661 wait complete. 2023-11-06T03:13:18.839 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:18.839 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 Processor -- start 2023-11-06T03:13:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 -- start start 2023-11-06T03:13:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcbc41599c0 0x7fcbc40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbc406d000 0x7fcbc40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc415e440 con 0x7fcbc415dea0 2023-11-06T03:13:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc415adb0 con 0x7fcbc406d000 2023-11-06T03:13:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbcb3d3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc406c240 con 0x7fcbc41599c0 2023-11-06T03:13:18.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbc406d000 0x7fcbc40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc9970640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcbc415dea0 0x7fcbc4070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43372/0 (socket says 172.21.15.157:43372) 2023-11-06T03:13:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcbc406d000 0x7fcbc40b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50742/0 (socket says 172.21.15.157:50742) 2023-11-06T03:13:18.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc9970640 1 -- 172.21.15.157:0/3127918404 learned_addr learned my addr 172.21.15.157:0/3127918404 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc916f640 1 --2- 172.21.15.157:0/3127918404 >> v2:172.21.15.195:3300/0 conn(0x7fcbc41599c0 0x7fcbc40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 -- 172.21.15.157:0/3127918404 >> v2:172.21.15.195:3300/0 conn(0x7fcbc41599c0 msgr2=0x7fcbc40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:18.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 --2- 172.21.15.157:0/3127918404 >> v2:172.21.15.195:3300/0 conn(0x7fcbc41599c0 0x7fcbc40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 -- 172.21.15.157:0/3127918404 >> v1:172.21.15.157:6789/0 conn(0x7fcbc415dea0 legacy=0x7fcbc4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbc896e640 1 -- 172.21.15.157:0/3127918404 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc406c520 con 0x7fcbc406d000 2023-11-06T03:13:18.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.957+0000 7fcbbbfff640 1 -- 172.21.15.157:0/3127918404 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913803351 0 0) 0x7fcbc415e440 con 0x7fcbc415dea0 2023-11-06T03:13:18.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbc896e640 1 --2- 172.21.15.157:0/3127918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc40b23e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4066800 tx=0x7fcbb401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b945f66e5b7b3e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbbbfff640 1 -- 172.21.15.157:0/3127918404 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40ab600 con 0x7fcbc406d000 2023-11-06T03:13:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbbbfff640 1 -- 172.21.15.157:0/3127918404 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40a7050 con 0x7fcbc406d000 2023-11-06T03:13:18.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbbbfff640 1 -- 172.21.15.157:0/3127918404 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40ab980 con 0x7fcbc406d000 2023-11-06T03:13:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/3127918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 msgr2=0x7fcbc40b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/3127918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc40b23e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4066800 tx=0x7fcbb401c430 comp rx=0 tx=0).stop 2023-11-06T03:13:18.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/3127918404 shutdown_connections 2023-11-06T03:13:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/3127918404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc40b23e0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/3127918404 >> v2:172.21.15.195:3300/0 conn(0x7fcbc41599c0 0x7fcbc40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/3127918404 >> 172.21.15.157:0/3127918404 conn(0x7fcbc4065930 msgr2=0x7fcbc4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/3127918404 shutdown_connections 2023-11-06T03:13:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.961+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/3127918404 wait complete. 2023-11-06T03:13:18.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 Processor -- start 2023-11-06T03:13:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- start start 2023-11-06T03:13:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc41599c0 0x7fcbc4151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc415dea0 0x7fcbc4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4154ad0 con 0x7fcbc406d000 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc414f560 con 0x7fcbc41599c0 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4153ec0 con 0x7fcbc415dea0 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50756/0 (socket says 172.21.15.157:50756) 2023-11-06T03:13:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 -- 172.21.15.157:0/2930260679 learned_addr learned my addr 172.21.15.157:0/2930260679 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc9970640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc415dea0 0x7fcbc4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc896e640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc41599c0 0x7fcbc4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 -- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc415dea0 msgr2=0x7fcbc4154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc415dea0 0x7fcbc4154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 -- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc41599c0 msgr2=0x7fcbc4151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc41599c0 0x7fcbc4151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 -- 172.21.15.157:0/2930260679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4009420 con 0x7fcbc406d000 2023-11-06T03:13:18.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc916f640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d800 tx=0x7fcbc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000b6c0 con 0x7fcbc406d000 2023-11-06T03:13:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000b820 con 0x7fcbc406d000 2023-11-06T03:13:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc41541a0 con 0x7fcbc406d000 2023-11-06T03:13:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000bb90 con 0x7fcbc406d000 2023-11-06T03:13:18.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc4154760 con 0x7fcbc406d000 2023-11-06T03:13:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcbc000bcf0 con 0x7fcbc406d000 2023-11-06T03:13:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc4072750 con 0x7fcbc406d000 2023-11-06T03:13:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 0x7fcba8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:18.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fcbc0083850 con 0x7fcbc406d000 2023-11-06T03:13:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.965+0000 7fcbc896e640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 0x7fcba8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.969+0000 7fcbc896e640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 0x7fcba8067f50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40b19a0 tx=0x7fcbb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:18.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:18.969+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcbc0051080 con 0x7fcbc406d000 2023-11-06T03:13:19.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcbc41591f0 con 0x7fcbc406d000 2023-11-06T03:13:19.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbb9ffb640 1 -- 172.21.15.157:0/2930260679 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1632 (secure 0 0 0) 0x7fcbc00546a0 con 0x7fcbc406d000 2023-11-06T03:13:19.174 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:19.174 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":576,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1038,"data_bytes":1066433335,"bytes_used":2686144512,"bytes_avail":770407968768,"bytes_total":773094113280,"write_bytes_sec":53640024,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-11-06T03:13:17.296264+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 msgr2=0x7fcba8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 0x7fcba8067f50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40b19a0 tx=0x7fcbb4008040 comp rx=0 tx=0).stop 2023-11-06T03:13:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 msgr2=0x7fcbc414e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d800 tx=0x7fcbc000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 shutdown_connections 2023-11-06T03:13:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcba8065a90 0x7fcba8067f50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcbc415dea0 0x7fcbc4154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbc41599c0 0x7fcbc4151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 --2- 172.21.15.157:0/2930260679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbc406d000 0x7fcbc414e9f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 >> 172.21.15.157:0/2930260679 conn(0x7fcbc4065930 msgr2=0x7fcbc4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 shutdown_connections 2023-11-06T03:13:19.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.177+0000 7fcbcb3d3640 1 -- 172.21.15.157:0/2930260679 wait complete. 2023-11-06T03:13:19.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:13:19.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub resume 2023-11-06T03:13:19.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 Processor -- start 2023-11-06T03:13:19.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 -- start start 2023-11-06T03:13:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd558156a70 0x7fd55806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55806c490 con 0x7fd5580adae0 2023-11-06T03:13:19.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55806bb60 con 0x7fd5580b2330 2023-11-06T03:13:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55f512640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55806b950 con 0x7fd558156a70 2023-11-06T03:13:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55caad640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd5580adae0 0x7fd5580b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43378/0 (socket says 172.21.15.157:43378) 2023-11-06T03:13:19.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55caad640 1 -- 172.21.15.157:0/3590242504 learned_addr learned my addr 172.21.15.157:0/3590242504 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/3590242504 >> v2:172.21.15.157:3300/0 conn(0x7fd558156a70 0x7fd55806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd54ffff640 1 -- 172.21.15.157:0/3590242504 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021963126 0 0) 0x7fd55806c490 con 0x7fd5580adae0 2023-11-06T03:13:19.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd54ffff640 1 -- 172.21.15.157:0/3590242504 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd544003530 con 0x7fd5580adae0 2023-11-06T03:13:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55d2ae640 1 -- 172.21.15.157:0/3590242504 >> v1:172.21.15.195:6789/0 conn(0x7fd5580b2330 legacy=0x7fd55806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55d2ae640 1 -- 172.21.15.157:0/3590242504 >> v1:172.21.15.157:6789/0 conn(0x7fd5580adae0 legacy=0x7fd5580b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.305+0000 7fd55d2ae640 1 -- 172.21.15.157:0/3590242504 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55806be40 con 0x7fd558156a70 2023-11-06T03:13:19.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/3590242504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558156a70 0x7fd55806f470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd5540097c0 tx=0x7fd55401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c7fc7ac005bde1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd54ffff640 1 -- 172.21.15.157:0/3590242504 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540a7070 con 0x7fd558156a70 2023-11-06T03:13:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd54ffff640 1 -- 172.21.15.157:0/3590242504 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540ab430 con 0x7fd558156a70 2023-11-06T03:13:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd54ffff640 1 -- 172.21.15.157:0/3590242504 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540ab7c0 con 0x7fd558156a70 2023-11-06T03:13:19.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 -- 172.21.15.157:0/3590242504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558156a70 msgr2=0x7fd55806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 --2- 172.21.15.157:0/3590242504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558156a70 0x7fd55806f470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd5540097c0 tx=0x7fd55401c410 comp rx=0 tx=0).stop 2023-11-06T03:13:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 -- 172.21.15.157:0/3590242504 shutdown_connections 2023-11-06T03:13:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 --2- 172.21.15.157:0/3590242504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd558156a70 0x7fd55806f470 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 -- 172.21.15.157:0/3590242504 >> 172.21.15.157:0/3590242504 conn(0x7fd558065930 msgr2=0x7fd558065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 -- 172.21.15.157:0/3590242504 shutdown_connections 2023-11-06T03:13:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.309+0000 7fd55f512640 1 -- 172.21.15.157:0/3590242504 wait complete. 2023-11-06T03:13:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 Processor -- start 2023-11-06T03:13:19.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- start start 2023-11-06T03:13:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5580b2330 0x7fd5580b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd558156a70 0x7fd558150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580b1410 con 0x7fd5580adae0 2023-11-06T03:13:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558153fe0 con 0x7fd5580b2330 2023-11-06T03:13:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5580b0b70 con 0x7fd558156a70 2023-11-06T03:13:19.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50762/0 (socket says 172.21.15.157:50762) 2023-11-06T03:13:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 -- 172.21.15.157:0/4233878057 learned_addr learned my addr 172.21.15.157:0/4233878057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 -- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd558156a70 msgr2=0x7fd558150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd558156a70 0x7fd558150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 -- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5580b2330 msgr2=0x7fd5580b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55caad640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5580b2330 0x7fd5580b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5580b2330 0x7fd5580b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 -- 172.21.15.157:0/4233878057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd554009420 con 0x7fd5580adae0 2023-11-06T03:13:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ad040 tx=0x7fd55401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd54dffb640 1 -- 172.21.15.157:0/4233878057 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540bf610 con 0x7fd5580adae0 2023-11-06T03:13:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd54dffb640 1 -- 172.21.15.157:0/4233878057 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540bf790 con 0x7fd5580adae0 2023-11-06T03:13:19.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580b0e50 con 0x7fd5580adae0 2023-11-06T03:13:19.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd54dffb640 1 -- 172.21.15.157:0/4233878057 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540bfb40 con 0x7fd5580adae0 2023-11-06T03:13:19.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55814b7f0 con 0x7fd5580adae0 2023-11-06T03:13:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.313+0000 7fd54dffb640 1 -- 172.21.15.157:0/4233878057 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd5540bfca0 con 0x7fd5580adae0 2023-11-06T03:13:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd54dffb640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd54dffb640 1 -- 172.21.15.157:0/4233878057 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd55411dc60 con 0x7fd5580adae0 2023-11-06T03:13:19.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55caad640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 Processor -- start 2023-11-06T03:13:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- start start 2023-11-06T03:13:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55caad640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 0x7fd53c067f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd558151970 tx=0x7fd548009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07f43d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07e0bc10 con 0x558f07f44260 2023-11-06T03:13:19.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07d9a3e0 con 0x558f07f8e130 2023-11-06T03:13:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f07d979b0 con 0x558f07f8ea50 2023-11-06T03:13:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50766/0 (socket says 172.21.15.157:50766) 2023-11-06T03:13:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 -- 172.21.15.157:0/1262396870 learned_addr learned my addr 172.21.15.157:0/1262396870 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55caad640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07f43d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 -- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 msgr2=0x558f07f43d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07f43d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 -- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 msgr2=0x558f07f8e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 -- 172.21.15.157:0/1262396870 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f07a8fe20 con 0x558f07f44260 2023-11-06T03:13:19.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55caad640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07f43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd550004440 tx=0x7fd550004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07d0ddfe2431982 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd542ffd640 1 -- 172.21.15.157:0/1262396870 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550004d80 con 0x558f07f44260 2023-11-06T03:13:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd542ffd640 1 -- 172.21.15.157:0/1262396870 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000c840 con 0x558f07f44260 2023-11-06T03:13:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd542ffd640 1 -- 172.21.15.157:0/1262396870 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000cbf0 con 0x558f07f44260 2023-11-06T03:13:19.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- 172.21.15.157:0/1262396870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 msgr2=0x558f07f44640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd550004440 tx=0x7fd550004900 comp rx=0 tx=0).stop 2023-11-06T03:13:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- 172.21.15.157:0/1262396870 shutdown_connections 2023-11-06T03:13:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07f44640 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07f43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 --2- 172.21.15.157:0/1262396870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- 172.21.15.157:0/1262396870 >> 172.21.15.157:0/1262396870 conn(0x558f07f4a9f0 msgr2=0x558f07f42600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- 172.21.15.157:0/1262396870 shutdown_connections 2023-11-06T03:13:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- 172.21.15.157:0/1262396870 wait complete. 2023-11-06T03:13:19.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 -- start start 2023-11-06T03:13:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.317+0000 7fd56263a000 1 Processor -- start 2023-11-06T03:13:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07f8d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07d932a0 con 0x558f07f8ea50 2023-11-06T03:13:19.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07d32470 con 0x558f07f8e130 2023-11-06T03:13:19.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f07d50660 con 0x558f07f44260 2023-11-06T03:13:19.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 10 client.0 _ll_register_callbacks cb 0x558f07f2fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50780/0 (socket says 172.21.15.157:50780) 2023-11-06T03:13:19.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 -- 172.21.15.157:0/1694520018 learned_addr learned my addr 172.21.15.157:0/1694520018 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07f8d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55caad640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 -- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 msgr2=0x558f07f8d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07f8d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 -- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 msgr2=0x558f07f8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 -- 172.21.15.157:0/1694520018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd550004070 con 0x558f07f8ea50 2023-11-06T03:13:19.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55caad640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x558f07d95f10 tx=0x7fd550003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55cac8b3918be95 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000ce10 con 0x558f07f8ea50 2023-11-06T03:13:19.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd550013480 con 0x558f07f8ea50 2023-11-06T03:13:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f07d500a0 con 0x558f07f8ea50 2023-11-06T03:13:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd550013640 con 0x558f07f8ea50 2023-11-06T03:13:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 10 client.8541 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x558f07d581d0 con 0x558f07f8ea50 2023-11-06T03:13:19.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f07d500a0 con 0x558f07f8ea50 2023-11-06T03:13:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd55001d040 con 0x558f07f8ea50 2023-11-06T03:13:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 10 client.8541 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 10 client.8541 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 20 client.8541 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:19.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07f8ff90 0x558f07f903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 4 client.8541 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f07d91bf0 con 0x558f07f8ff90 2023-11-06T03:13:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07f8ff90 0x558f07f903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 10 client.8541 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07f8ff90 0x558f07f903a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512312d966ca1932 server_cookie=ecd3750aaa07f998 in_seq=0 out_seq=0 2023-11-06T03:13:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 1 -- 172.21.15.157:0/1694520018 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558f07d91bf0 con 0x558f07f8ff90 2023-11-06T03:13:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 10 client.8541 handle_command_reply: tid=0 2023-11-06T03:13:19.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 client.8541 shutdown 2023-11-06T03:13:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 msgr2=0x558f07f8f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x558f07d95f10 tx=0x7fd550003b60 comp rx=0 tx=0).stop 2023-11-06T03:13:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 shutdown_connections 2023-11-06T03:13:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07f8ff90 0x558f07f903a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07f8f580 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd537fff640 0 client.8541 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07f8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 --2- 172.21.15.157:0/1694520018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07f8d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 >> 172.21.15.157:0/1694520018 conn(0x558f07f4a9f0 msgr2=0x558f07f42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 shutdown_connections 2023-11-06T03:13:19.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 1 -- 172.21.15.157:0/1694520018 wait complete. 2023-11-06T03:13:19.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.321+0000 7fd56263a000 20 client.8541 trim_cache size 0 max 16384 2023-11-06T03:13:19.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.333+0000 7fd56263a000 1 Processor -- start 2023-11-06T03:13:19.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.333+0000 7fd56263a000 1 -- start start 2023-11-06T03:13:19.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07fe5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07fe60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07a50aa0 con 0x558f07f44260 2023-11-06T03:13:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07d906a0 con 0x558f07f8e130 2023-11-06T03:13:19.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f07e37bc0 con 0x558f07f8ea50 2023-11-06T03:13:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50784/0 (socket says 172.21.15.157:50784) 2023-11-06T03:13:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 -- 172.21.15.157:0/1932751474 learned_addr learned my addr 172.21.15.157:0/1932751474 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 -- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 msgr2=0x558f07fe60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07fe60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 -- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 msgr2=0x558f07fe5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07fe5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 -- 172.21.15.157:0/1932751474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f07d3e520 con 0x558f07f44260 2023-11-06T03:13:19.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ef0f0 tx=0x7fd5540f5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865eaaeb48d5fe4d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd5437fe640 1 -- 172.21.15.157:0/1932751474 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540f2f90 con 0x558f07f44260 2023-11-06T03:13:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd5437fe640 1 -- 172.21.15.157:0/1932751474 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540f7140 con 0x558f07f44260 2023-11-06T03:13:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd5437fe640 1 -- 172.21.15.157:0/1932751474 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554009110 con 0x558f07f44260 2023-11-06T03:13:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1932751474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 msgr2=0x558f07fe5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ef0f0 tx=0x7fd5540f5040 comp rx=0 tx=0).stop 2023-11-06T03:13:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1932751474 shutdown_connections 2023-11-06T03:13:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f8ea50 0x558f07fe60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07fe5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- 172.21.15.157:0/1932751474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f44260 0x558f07fe5620 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1932751474 >> 172.21.15.157:0/1932751474 conn(0x558f07f4a9f0 msgr2=0x558f07f48910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1932751474 shutdown_connections 2023-11-06T03:13:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1932751474 wait complete. 2023-11-06T03:13:19.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- start start 2023-11-06T03:13:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 Processor -- start 2023-11-06T03:13:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07feb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07feb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07d92130 con 0x558f07f8ea50 2023-11-06T03:13:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f07fa4720 con 0x558f07f8e130 2023-11-06T03:13:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f07e1bc70 con 0x558f07f44260 2023-11-06T03:13:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 10 client.0 _ll_register_callbacks cb 0x558f07d57cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:19.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50786/0 (socket says 172.21.15.157:50786) 2023-11-06T03:13:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 -- 172.21.15.157:0/1505522300 learned_addr learned my addr 172.21.15.157:0/1505522300 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07feb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 -- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 msgr2=0x558f07feb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07feb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 -- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 msgr2=0x558f07feb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55caad640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07feb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07feb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 -- 172.21.15.157:0/1505522300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55814b7f0 con 0x558f07f8ea50 2023-11-06T03:13:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd55daaf640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c570 tx=0x7fd550003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f56bff59376d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd537fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550013840 con 0x558f07f8ea50 2023-11-06T03:13:19.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd550004290 con 0x558f07f8ea50 2023-11-06T03:13:19.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f07d5c620 con 0x558f07f8ea50 2023-11-06T03:13:19.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd550006b60 con 0x558f07f8ea50 2023-11-06T03:13:19.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 10 client.8547 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:19.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.337+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x558f07d32090 con 0x558f07f8ea50 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f07d5c620 con 0x558f07f8ea50 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fd55001f430 con 0x558f07f8ea50 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 10 client.8547 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 10 client.8547 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 20 client.8547 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:19.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07fcf020 0x558f07fcf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 4 client.8547 mds_command: new command op to 8226 tid=0[{"prefix": "scrub resume"}] 2023-11-06T03:13:19.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x558f07d8c320 con 0x558f07fcf020 2023-11-06T03:13:19.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07fcf020 0x558f07fcf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55d2ae640 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07fcf020 0x558f07fcf400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471c73271d278dd0 server_cookie=73476878dd1bdc78 in_seq=0 out_seq=0 2023-11-06T03:13:19.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 10 client.8547 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 1 -- 172.21.15.157:0/1505522300 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x558f07d8c320 con 0x558f07fcf020 2023-11-06T03:13:19.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 10 client.8547 handle_command_reply: tid=0 2023-11-06T03:13:19.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 client.8547 shutdown 2023-11-06T03:13:19.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 msgr2=0x558f07fceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c570 tx=0x7fd550003af0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 shutdown_connections 2023-11-06T03:13:19.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x558f07fcf020 0x558f07fcf400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f07f8ea50 0x558f07fceae0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558f07f8e130 0x558f07feb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd537fff640 0 client.8547 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 --2- 172.21.15.157:0/1505522300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558f07f44260 0x558f07feb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 >> 172.21.15.157:0/1505522300 conn(0x558f07f4a9f0 msgr2=0x558f07f491a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 shutdown_connections 2023-11-06T03:13:19.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 1 -- 172.21.15.157:0/1505522300 wait complete. 2023-11-06T03:13:19.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd56263a000 20 client.8547 trim_cache size 0 max 16384 2023-11-06T03:13:19.374 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:19.374 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:13:19.374 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:19.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 msgr2=0x7fd53c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 0x7fd53c067f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd558151970 tx=0x7fd548009040 comp rx=0 tx=0).stop 2023-11-06T03:13:19.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 msgr2=0x7fd558153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ad040 tx=0x7fd55401ac50 comp rx=0 tx=0).stop 2023-11-06T03:13:19.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 shutdown_connections 2023-11-06T03:13:19.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd558156a70 0x7fd558150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd5580b2330 0x7fd5580b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 --2- 172.21.15.157:0/4233878057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5580adae0 0x7fd558153aa0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 >> 172.21.15.157:0/4233878057 conn(0x7fd558065930 msgr2=0x7fd558157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 shutdown_connections 2023-11-06T03:13:19.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.341+0000 7fd55f512640 1 -- 172.21.15.157:0/4233878057 wait complete. 2023-11-06T03:13:19.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 scrub status 2023-11-06T03:13:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 Processor -- start 2023-11-06T03:13:19.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 -- start start 2023-11-06T03:13:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5fe0159cd0 0x7f5fe015e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe015e590 con 0x7f5fe01598f0 2023-11-06T03:13:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe00cafa0 con 0x7f5fe00cab10 2023-11-06T03:13:19.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe855c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe00cb280 con 0x7f5fe0159cd0 2023-11-06T03:13:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe62f8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5fe01598f0 0x7f5fe006d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43384/0 (socket says 172.21.15.157:43384) 2023-11-06T03:13:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe62f8640 1 -- 172.21.15.157:0/260307054 learned_addr learned my addr 172.21.15.157:0/260307054 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078581512 0 0) 0x7f5fd00014a0 con 0x7f5fe01598f0 2023-11-06T03:13:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc8003530 con 0x7f5fe01598f0 2023-11-06T03:13:19.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/260307054 >> v2:172.21.15.195:3300/0 conn(0x7f5fe0159cd0 0x7f5fe015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528730240 0 0) 0x7f5fc8003530 con 0x7f5fe01598f0 2023-11-06T03:13:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 >> v2:172.21.15.195:3300/0 conn(0x7f5fe0159cd0 msgr2=0x7f5fe015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 --2- 172.21.15.157:0/260307054 >> v2:172.21.15.195:3300/0 conn(0x7f5fe0159cd0 0x7f5fe015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 >> v1:172.21.15.195:6789/0 conn(0x7f5fe00cab10 legacy=0x7f5fe006f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe006c8a0 con 0x7f5fe01598f0 2023-11-06T03:13:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5fd0001b90 con 0x7f5fe01598f0 2023-11-06T03:13:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 >> v1:172.21.15.157:6789/0 conn(0x7f5fe01598f0 legacy=0x7f5fe006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fc8003980 0x7f5fc8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fc8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fc80052b0 0x7f5fc800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00cb280 con 0x7f5fc80047d0 2023-11-06T03:13:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc800c180 con 0x7f5fc80052b0 2023-11-06T03:13:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc800c340 con 0x7f5fc8003980 2023-11-06T03:13:19.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fc8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fc80052b0 0x7f5fc800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fc8003980 0x7f5fc8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 -- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fc8003980 msgr2=0x7f5fc8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fc8003980 0x7f5fc8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 -- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fc80052b0 msgr2=0x7f5fc800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fc80052b0 0x7f5fc800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe6af9640 1 -- 172.21.15.157:0/260307054 reap_dead start 2023-11-06T03:13:19.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.489+0000 7f5fe5af7640 1 -- 172.21.15.157:0/260307054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8003530 con 0x7f5fc80047d0 2023-11-06T03:13:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fc8004c50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4001750 tx=0x7f5fd401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e668d53a3f6e50f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40052d0 con 0x7f5fc80047d0 2023-11-06T03:13:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40a7050 con 0x7f5fc80047d0 2023-11-06T03:13:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe52f6640 1 -- 172.21.15.157:0/260307054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4003600 con 0x7f5fc80047d0 2023-11-06T03:13:19.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 -- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 msgr2=0x7f5fc8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fc8004c50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4001750 tx=0x7f5fd401c370 comp rx=0 tx=0).stop 2023-11-06T03:13:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 -- 172.21.15.157:0/260307054 shutdown_connections 2023-11-06T03:13:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 --2- 172.21.15.157:0/260307054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fc8004c50 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 -- 172.21.15.157:0/260307054 >> 172.21.15.157:0/260307054 conn(0x7f5fe00b0b50 msgr2=0x7f5fe00b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 -- 172.21.15.157:0/260307054 shutdown_connections 2023-11-06T03:13:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 -- 172.21.15.157:0/260307054 wait complete. 2023-11-06T03:13:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.493+0000 7f5fe855c640 1 Processor -- start 2023-11-06T03:13:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- start start 2023-11-06T03:13:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe00c3b90 0x7f5fe00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe014a2b0 0x7f5fe00ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00c4ea0 con 0x7f5fc80047d0 2023-11-06T03:13:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00c4010 con 0x7f5fe014a2b0 2023-11-06T03:13:19.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe00c4290 con 0x7f5fe00c3b90 2023-11-06T03:13:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50808/0 (socket says 172.21.15.157:50808) 2023-11-06T03:13:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 -- 172.21.15.157:0/4071842187 learned_addr learned my addr 172.21.15.157:0/4071842187 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe00c3b90 0x7f5fe00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe014a2b0 0x7f5fe00ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 -- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe00c3b90 msgr2=0x7f5fe00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe00c3b90 0x7f5fe00c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 -- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe014a2b0 msgr2=0x7f5fe00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe014a2b0 0x7f5fe00ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 -- 172.21.15.157:0/4071842187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00c4570 con 0x7f5fc80047d0 2023-11-06T03:13:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0007a30 tx=0x7f5fd000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 -- 172.21.15.157:0/4071842187 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000ee50 con 0x7f5fc80047d0 2023-11-06T03:13:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 -- 172.21.15.157:0/4071842187 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0013420 con 0x7f5fc80047d0 2023-11-06T03:13:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe00c4850 con 0x7f5fc80047d0 2023-11-06T03:13:19.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 -- 172.21.15.157:0/4071842187 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00137d0 con 0x7f5fc80047d0 2023-11-06T03:13:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe015b590 con 0x7f5fc80047d0 2023-11-06T03:13:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 -- 172.21.15.157:0/4071842187 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5fd0013930 con 0x7f5fc80047d0 2023-11-06T03:13:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 0x7f5fcc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fdeffd640 1 -- 172.21.15.157:0/4071842187 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5fd0083830 con 0x7f5fc80047d0 2023-11-06T03:13:19.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.497+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 0x7f5fcc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 Processor -- start 2023-11-06T03:13:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- start start 2023-11-06T03:13:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 0x7f5fcc067f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004750 tx=0x7f5fd40011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9d6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9d6f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9c36c10 con 0x557de9db9130 2023-11-06T03:13:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9bc53e0 con 0x557de9d6f260 2023-11-06T03:13:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de9bc29b0 con 0x557de9db9a50 2023-11-06T03:13:19.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50812/0 (socket says 172.21.15.157:50812) 2023-11-06T03:13:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 -- 172.21.15.157:0/1350035866 learned_addr learned my addr 172.21.15.157:0/1350035866 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9d6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9d6f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 -- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 msgr2=0x557de9d6ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9d6ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 -- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 msgr2=0x557de9d6f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9d6f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 -- 172.21.15.157:0/1350035866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de98bae20 con 0x557de9db9130 2023-11-06T03:13:19.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9d6ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9d6f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0001640 tx=0x7f5fd0054df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2ffa9ac71f3634 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fc7fff640 1 -- 172.21.15.157:0/1350035866 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd006c070 con 0x557de9db9130 2023-11-06T03:13:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fc7fff640 1 -- 172.21.15.157:0/1350035866 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0001b00 con 0x557de9db9130 2023-11-06T03:13:19.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fc7fff640 1 -- 172.21.15.157:0/1350035866 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd006c1d0 con 0x557de9db9130 2023-11-06T03:13:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- 172.21.15.157:0/1350035866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 msgr2=0x557de9db9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0001640 tx=0x7f5fd0054df0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- 172.21.15.157:0/1350035866 shutdown_connections 2023-11-06T03:13:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9d6f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9d6ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- 172.21.15.157:0/1350035866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9130 0x557de9db9510 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- 172.21.15.157:0/1350035866 >> 172.21.15.157:0/1350035866 conn(0x557de9d759f0 msgr2=0x557de9d6d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- 172.21.15.157:0/1350035866 shutdown_connections 2023-11-06T03:13:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- 172.21.15.157:0/1350035866 wait complete. 2023-11-06T03:13:19.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- start start 2023-11-06T03:13:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 Processor -- start 2023-11-06T03:13:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9dc0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9a50 0x557de9dba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9bbe2a0 con 0x557de9d6f260 2023-11-06T03:13:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9b5d470 con 0x557de9db9a50 2023-11-06T03:13:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de9b7b660 con 0x557de9db9130 2023-11-06T03:13:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5feb684000 10 client.0 _ll_register_callbacks cb 0x557de9d5ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:19.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50814/0 (socket says 172.21.15.157:50814) 2023-11-06T03:13:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe62f8640 1 -- 172.21.15.157:0/2083378627 learned_addr learned my addr 172.21.15.157:0/2083378627 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.501+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9dc0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9a50 0x557de9dba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 -- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 msgr2=0x557de9dc0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9dc0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 -- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9a50 msgr2=0x557de9dba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9a50 0x557de9dba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 -- 172.21.15.157:0/2083378627 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe015b590 con 0x557de9d6f260 2023-11-06T03:13:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ea30 tx=0x7f5fd0001f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414e71d584ee71cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:19.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd006c040 con 0x557de9d6f260 2023-11-06T03:13:19.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5fd0001b00 con 0x557de9d6f260 2023-11-06T03:13:19.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de9b7b0a0 con 0x557de9d6f260 2023-11-06T03:13:19.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd0051440 con 0x557de9d6f260 2023-11-06T03:13:19.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 10 client.8559 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x557de9b831d0 con 0x557de9d6f260 2023-11-06T03:13:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557de9b7b0a0 con 0x557de9d6f260 2023-11-06T03:13:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5fd007bd20 con 0x557de9d6f260 2023-11-06T03:13:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 10 client.8559 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:19.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 10 client.8559 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 20 client.8559 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9dbade0 0x557de9dbb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 4 client.8559 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557de9bbcbf0 con 0x557de9dbade0 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9dbade0 0x557de9dbb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9dbade0 0x557de9dbb1c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcfd03c15a4ab6a server_cookie=491e04ccff239bdb in_seq=0 out_seq=0 2023-11-06T03:13:19.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 10 client.8559 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 1 -- 172.21.15.157:0/2083378627 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557de9bbcbf0 con 0x557de9dbade0 2023-11-06T03:13:19.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 10 client.8559 handle_command_reply: tid=0 2023-11-06T03:13:19.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 client.8559 shutdown 2023-11-06T03:13:19.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 msgr2=0x557de9dc06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ea30 tx=0x7f5fd0001f20 comp rx=0 tx=0).stop 2023-11-06T03:13:19.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 shutdown_connections 2023-11-06T03:13:19.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9dbade0 0x557de9dbb1c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9a50 0x557de9dba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9dc0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 --2- 172.21.15.157:0/2083378627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9dc06e0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5fc57fa640 0 client.8559 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 >> 172.21.15.157:0/2083378627 conn(0x557de9d759f0 msgr2=0x557de9d6dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 shutdown_connections 2023-11-06T03:13:19.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 1 -- 172.21.15.157:0/2083378627 wait complete. 2023-11-06T03:13:19.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.505+0000 7f5feb684000 20 client.8559 trim_cache size 0 max 16384 2023-11-06T03:13:19.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 Processor -- start 2023-11-06T03:13:19.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- start start 2023-11-06T03:13:19.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9130 0x557de9d74d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9dbe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de987baa0 con 0x557de9d6f260 2023-11-06T03:13:19.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9bbbab0 con 0x557de9db9130 2023-11-06T03:13:19.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de9d71880 con 0x557de9db9a50 2023-11-06T03:13:19.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50822/0 (socket says 172.21.15.157:50822) 2023-11-06T03:13:19.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 -- 172.21.15.157:0/3048017379 learned_addr learned my addr 172.21.15.157:0/3048017379 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9130 0x557de9d74d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 -- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 msgr2=0x557de9dbe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9dbe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 -- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9130 msgr2=0x557de9d74d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9130 0x557de9d74d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 -- 172.21.15.157:0/3048017379 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de9b69520 con 0x557de9d6f260 2023-11-06T03:13:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5fd005c7e0 tx=0x7f5fd007c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd941b8956eac605 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fdcff9640 1 -- 172.21.15.157:0/3048017379 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd006c040 con 0x557de9d6f260 2023-11-06T03:13:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fdcff9640 1 -- 172.21.15.157:0/3048017379 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0051070 con 0x557de9d6f260 2023-11-06T03:13:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fdcff9640 1 -- 172.21.15.157:0/3048017379 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0001b00 con 0x557de9d6f260 2023-11-06T03:13:19.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- 172.21.15.157:0/3048017379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 msgr2=0x557de9d74820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5fd005c7e0 tx=0x7f5fd007c000 comp rx=0 tx=0).stop 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- 172.21.15.157:0/3048017379 shutdown_connections 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9a50 0x557de9dbe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9db9130 0x557de9d74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- 172.21.15.157:0/3048017379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9d6f260 0x557de9d74820 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- 172.21.15.157:0/3048017379 >> 172.21.15.157:0/3048017379 conn(0x557de9d759f0 msgr2=0x557de9d74110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- 172.21.15.157:0/3048017379 shutdown_connections 2023-11-06T03:13:19.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- 172.21.15.157:0/3048017379 wait complete. 2023-11-06T03:13:19.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- start start 2023-11-06T03:13:19.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 Processor -- start 2023-11-06T03:13:19.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9e15e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9e18390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9d73550 con 0x557de9db9a50 2023-11-06T03:13:19.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de9bb9c10 con 0x557de9d6f260 2023-11-06T03:13:19.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de9bb8640 con 0x557de9db9130 2023-11-06T03:13:19.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5feb684000 10 client.0 _ll_register_callbacks cb 0x557de9b82cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:19.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.521+0000 7f5fe6af9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50826/0 (socket says 172.21.15.157:50826) 2023-11-06T03:13:19.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 -- 172.21.15.157:0/3110715410 learned_addr learned my addr 172.21.15.157:0/3110715410 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe5af7640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9e18390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9e15e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 -- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 msgr2=0x557de9e18390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9e18390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 -- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 msgr2=0x557de9e15e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9e15e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 -- 172.21.15.157:0/3110715410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe015b590 con 0x557de9db9a50 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9e15e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe6af9640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0066600 tx=0x7f5fd8014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aacaa73f145ce6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:19.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8014ea0 con 0x557de9db9a50 2023-11-06T03:13:19.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de9b87620 con 0x557de9db9a50 2023-11-06T03:13:19.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5fd8017680 con 0x557de9db9a50 2023-11-06T03:13:19.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fd8017840 con 0x557de9db9a50 2023-11-06T03:13:19.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 10 client.8565 fetch_fsmap learned FSMap version 85 2023-11-06T03:13:19.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x557de9b5d090 con 0x557de9db9a50 2023-11-06T03:13:19.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557de9b87620 con 0x557de9db9a50 2023-11-06T03:13:19.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 85) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5fd8004020 con 0x557de9db9a50 2023-11-06T03:13:19.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 10 client.8565 fetch_fsmap finished waiting for FSMap version 85 2023-11-06T03:13:19.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 10 client.8565 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2023-11-06T03:13:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 20 client.8565 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9e1ae20 0x557de9e1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 4 client.8565 mds_command: new command op to 8226 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:13:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 --> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557de9bb7320 con 0x557de9e1ae20 2023-11-06T03:13:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9e1ae20 0x557de9e1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fc57fa640 10 client.8565 ms_handle_connect on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.525+0000 7f5fe62f8640 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9e1ae20 0x557de9e1d210 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7755b8fb5ca4310 server_cookie=ace596b4767eac37 in_seq=0 out_seq=0 2023-11-06T03:13:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fc57fa640 1 -- 172.21.15.157:0/3110715410 <== mds.0 v2:172.21.15.157:6833/2645931923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557de9bb7320 con 0x557de9e1ae20 2023-11-06T03:13:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fc57fa640 10 client.8565 handle_command_reply: tid=0 2023-11-06T03:13:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 client.8565 shutdown 2023-11-06T03:13:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 msgr2=0x557de9e1a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0066600 tx=0x7f5fd8014780 comp rx=0 tx=0).stop 2023-11-06T03:13:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 shutdown_connections 2023-11-06T03:13:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:6833/2645931923,v1:172.21.15.157:6835/2645931923] conn(0x557de9e1ae20 0x557de9e1d210 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557de9db9a50 0x557de9e1a8e0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x557de9db9130 0x557de9e18390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fc57fa640 0 client.8565 ms_handle_reset on v2:172.21.15.157:6833/2645931923 2023-11-06T03:13:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 --2- 172.21.15.157:0/3110715410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557de9d6f260 0x557de9e15e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 >> 172.21.15.157:0/3110715410 conn(0x557de9d759f0 msgr2=0x557de9d74110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 shutdown_connections 2023-11-06T03:13:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 1 -- 172.21.15.157:0/3110715410 wait complete. 2023-11-06T03:13:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5feb684000 20 client.8565 trim_cache size 0 max 16384 2023-11-06T03:13:19.552 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:19.553 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:13:19.553 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:13:19.553 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:19.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 msgr2=0x7f5fcc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 0x7f5fcc067f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004750 tx=0x7f5fd40011a0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 msgr2=0x7f5fe015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0007a30 tx=0x7f5fd000e530 comp rx=0 tx=0).stop 2023-11-06T03:13:19.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 shutdown_connections 2023-11-06T03:13:19.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fcc065a60 0x7f5fcc067f20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe014a2b0 0x7f5fe00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe00c3b90 0x7f5fe00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 --2- 172.21.15.157:0/4071842187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fc80047d0 0x7f5fe015cad0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 >> 172.21.15.157:0/4071842187 conn(0x7f5fe00b0b50 msgr2=0x7f5fe0156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 shutdown_connections 2023-11-06T03:13:19.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.529+0000 7f5fe855c640 1 -- 172.21.15.157:0/4071842187 wait complete. 2023-11-06T03:13:19.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:19.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.677+0000 7fa3bd579640 1 Processor -- start 2023-11-06T03:13:19.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.677+0000 7fa3bd579640 1 -- start start 2023-11-06T03:13:19.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3bd579640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa3b81551e0 0x7fa3b806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3bd579640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b815f580 con 0x7fa3b80b3b20 2023-11-06T03:13:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3bd579640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b815efc0 con 0x7fa3b815e5e0 2023-11-06T03:13:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3bd579640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b815f2a0 con 0x7fa3b81551e0 2023-11-06T03:13:19.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa3b80b3b20 0x7fa3b815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43394/0 (socket says 172.21.15.157:43394) 2023-11-06T03:13:19.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b77fe640 1 -- 172.21.15.157:0/926980735 learned_addr learned my addr 172.21.15.157:0/926980735 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:19.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/926980735 >> v2:172.21.15.157:3300/0 conn(0x7fa3b81551e0 0x7fa3b806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b6ffd640 1 -- 172.21.15.157:0/926980735 >> v1:172.21.15.195:6789/0 conn(0x7fa3b815e5e0 legacy=0x7fa3b80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b6ffd640 1 -- 172.21.15.157:0/926980735 >> v1:172.21.15.157:6789/0 conn(0x7fa3b80b3b20 legacy=0x7fa3b815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b6ffd640 1 -- 172.21.15.157:0/926980735 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b806f5a0 con 0x7fa3b81551e0 2023-11-06T03:13:19.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.681+0000 7fa3b5ffb640 1 -- 172.21.15.157:0/926980735 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191266607 0 0) 0x7fa3b815f580 con 0x7fa3b80b3b20 2023-11-06T03:13:19.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/926980735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b806fda0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0097c0 tx=0x7fa3ac01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4273948a5e1d92 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b5ffb640 1 -- 172.21.15.157:0/926980735 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac004230 con 0x7fa3b81551e0 2023-11-06T03:13:19.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b5ffb640 1 -- 172.21.15.157:0/926980735 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac0a7050 con 0x7fa3b81551e0 2023-11-06T03:13:19.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b5ffb640 1 -- 172.21.15.157:0/926980735 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac003620 con 0x7fa3b81551e0 2023-11-06T03:13:19.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- 172.21.15.157:0/926980735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 msgr2=0x7fa3b806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 --2- 172.21.15.157:0/926980735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b806fda0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0097c0 tx=0x7fa3ac01c330 comp rx=0 tx=0).stop 2023-11-06T03:13:19.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- 172.21.15.157:0/926980735 shutdown_connections 2023-11-06T03:13:19.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 --2- 172.21.15.157:0/926980735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b806fda0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- 172.21.15.157:0/926980735 >> 172.21.15.157:0/926980735 conn(0x7fa3b8065930 msgr2=0x7fa3b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- 172.21.15.157:0/926980735 shutdown_connections 2023-11-06T03:13:19.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- 172.21.15.157:0/926980735 wait complete. 2023-11-06T03:13:19.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 Processor -- start 2023-11-06T03:13:19.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- start start 2023-11-06T03:13:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 0x7fa3b81f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 0x7fa3b80a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b814fe30 con 0x7fa3b81551e0 2023-11-06T03:13:19.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8150110 con 0x7fa3b80b3b20 2023-11-06T03:13:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3bd579640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81503f0 con 0x7fa3b815e5e0 2023-11-06T03:13:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50848/0 (socket says 172.21.15.157:50848) 2023-11-06T03:13:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 -- 172.21.15.157:0/820790783 learned_addr learned my addr 172.21.15.157:0/820790783 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:19.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 0x7fa3b81f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 -- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 msgr2=0x7fa3b80a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b77fe640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 0x7fa3b80a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 0x7fa3b80a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 -- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 msgr2=0x7fa3b81f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 0x7fa3b81f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 -- 172.21.15.157:0/820790783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009420 con 0x7fa3b81551e0 2023-11-06T03:13:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 0x7fa3b81f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b77fe640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 0x7fa3b80a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:19.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.685+0000 7fa3b67fc640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c3c0 tx=0x7fa3a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80052d0 con 0x7fa3b81551e0 2023-11-06T03:13:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a8005430 con 0x7fa3b81551e0 2023-11-06T03:13:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b81506d0 con 0x7fa3b81551e0 2023-11-06T03:13:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8003840 con 0x7fa3b81551e0 2023-11-06T03:13:19.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b81f9760 con 0x7fa3b81551e0 2023-11-06T03:13:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b80cef70 con 0x7fa3b81551e0 2023-11-06T03:13:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa3a80039a0 con 0x7fa3b81551e0 2023-11-06T03:13:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 0x7fa3a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa3a8082820 con 0x7fa3b81551e0 2023-11-06T03:13:19.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 0x7fa3a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.689+0000 7fa3b6ffd640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 0x7fa3a4067f20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0ad000 tx=0x7fa3ac0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:19.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.693+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa3a8050060 con 0x7fa3b81551e0 2023-11-06T03:13:19.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.881+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b8072750 con 0x7fa3b81551e0 2023-11-06T03:13:19.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.881+0000 7fa39f7fe640 1 -- 172.21.15.157:0/820790783 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fa3a8017090 con 0x7fa3b81551e0 2023-11-06T03:13:19.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:19.880 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:19.880 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:19.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 msgr2=0x7fa3a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 0x7fa3a4067f20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0ad000 tx=0x7fa3ac0a6000 comp rx=0 tx=0).stop 2023-11-06T03:13:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 msgr2=0x7fa3b81f9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c3c0 tx=0x7fa3a800c880 comp rx=0 tx=0).stop 2023-11-06T03:13:19.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 shutdown_connections 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3a4065a60 0x7fa3a4067f20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b815e5e0 0x7fa3b80a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3b81551e0 0x7fa3b81f9220 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 --2- 172.21.15.157:0/820790783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b80b3b20 0x7fa3b81f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 >> 172.21.15.157:0/820790783 conn(0x7fa3b8065930 msgr2=0x7fa3b815d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:19.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 shutdown_connections 2023-11-06T03:13:19.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:19.885+0000 7fa3bd579640 1 -- 172.21.15.157:0/820790783 wait complete. 2023-11-06T03:13:19.898 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:19.899 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.021+0000 7fc0fbf2c640 1 Processor -- start 2023-11-06T03:13:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.021+0000 7fc0fbf2c640 1 -- start start 2023-11-06T03:13:20.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc0f415caa0 0x7fc0f40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0f415ce80 0x7fc0f40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f40d2920 con 0x7fc0f40ce1a0 2023-11-06T03:13:20.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40d18a0 con 0x7fc0f415ce80 2023-11-06T03:13:20.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40d1b60 con 0x7fc0f415caa0 2023-11-06T03:13:20.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0f415ce80 0x7fc0f40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0f415ce80 0x7fc0f40cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50854/0 (socket says 172.21.15.157:50854) 2023-11-06T03:13:20.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 -- 172.21.15.157:0/3207344225 learned_addr learned my addr 172.21.15.157:0/3207344225 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:20.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f9cc8640 1 --2- 172.21.15.157:0/3207344225 >> v2:172.21.15.195:3300/0 conn(0x7fc0f415caa0 0x7fc0f40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 -- 172.21.15.157:0/3207344225 >> v2:172.21.15.195:3300/0 conn(0x7fc0f415caa0 msgr2=0x7fc0f40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:20.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 --2- 172.21.15.157:0/3207344225 >> v2:172.21.15.195:3300/0 conn(0x7fc0f415caa0 0x7fc0f40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 -- 172.21.15.157:0/3207344225 >> v1:172.21.15.195:6789/0 conn(0x7fc0f40ce1a0 legacy=0x7fc0f40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:20.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 -- 172.21.15.157:0/3207344225 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40d1e40 con 0x7fc0f415ce80 2023-11-06T03:13:20.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f94c7640 1 --2- 172.21.15.157:0/3207344225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cfb70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40b4f80 tx=0x7fc0e401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280704b2296f1dcf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:20.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f8cc6640 1 -- 172.21.15.157:0/3207344225 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40ac600 con 0x7fc0f415ce80 2023-11-06T03:13:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f8cc6640 1 -- 172.21.15.157:0/3207344225 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40a8050 con 0x7fc0f415ce80 2023-11-06T03:13:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0f8cc6640 1 -- 172.21.15.157:0/3207344225 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40ac980 con 0x7fc0f415ce80 2023-11-06T03:13:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/3207344225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 msgr2=0x7fc0f40cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/3207344225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cfb70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40b4f80 tx=0x7fc0e401c490 comp rx=0 tx=0).stop 2023-11-06T03:13:20.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/3207344225 shutdown_connections 2023-11-06T03:13:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/3207344225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cfb70 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/3207344225 >> v2:172.21.15.195:3300/0 conn(0x7fc0f415caa0 0x7fc0f40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/3207344225 >> 172.21.15.157:0/3207344225 conn(0x7fc0f4070790 msgr2=0x7fc0f4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/3207344225 shutdown_connections 2023-11-06T03:13:20.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.025+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/3207344225 wait complete. 2023-11-06T03:13:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 Processor -- start 2023-11-06T03:13:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- start start 2023-11-06T03:13:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0f40ce1a0 0x7fc0f40c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0f415caa0 0x7fc0f40cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40cc990 con 0x7fc0f415ce80 2023-11-06T03:13:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4155450 con 0x7fc0f415caa0 2023-11-06T03:13:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4155730 con 0x7fc0f40ce1a0 2023-11-06T03:13:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50868/0 (socket says 172.21.15.157:50868) 2023-11-06T03:13:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 -- 172.21.15.157:0/4172624376 learned_addr learned my addr 172.21.15.157:0/4172624376 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0f9cc8640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0f40ce1a0 0x7fc0f40c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0f94c7640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0f415caa0 0x7fc0f40cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 -- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0f40ce1a0 msgr2=0x7fc0f40c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:20.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0f40ce1a0 0x7fc0f40c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 -- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0f415caa0 msgr2=0x7fc0f40cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:20.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0f415caa0 0x7fc0f40cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 -- 172.21.15.157:0/4172624376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40094d0 con 0x7fc0f415ce80 2023-11-06T03:13:20.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fa4c9640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b9e0 tx=0x7fc0ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:20.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec003b80 con 0x7fc0f415ce80 2023-11-06T03:13:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec003ce0 con 0x7fc0f415ce80 2023-11-06T03:13:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00cb60 con 0x7fc0f415ce80 2023-11-06T03:13:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f4155a10 con 0x7fc0f415ce80 2023-11-06T03:13:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.029+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f41604b0 con 0x7fc0f415ce80 2023-11-06T03:13:20.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f4072750 con 0x7fc0f415ce80 2023-11-06T03:13:20.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc0ec00ccc0 con 0x7fc0f415ce80 2023-11-06T03:13:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0ea7fc640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 0x7fc0d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc0ec0830d0 con 0x7fc0f415ce80 2023-11-06T03:13:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0f9cc8640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 0x7fc0d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:20.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.033+0000 7fc0f9cc8640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 0x7fc0d8067f20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40c6370 tx=0x7fc0f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:20.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.037+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc0ec050870 con 0x7fc0f415ce80 2023-11-06T03:13:20.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.237+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc0f4155ec0 con 0x7fc0f415ce80 2023-11-06T03:13:20.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.237+0000 7fc0ea7fc640 1 -- 172.21.15.157:0/4172624376 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1633 (secure 0 0 0) 0x7fc0ec053e90 con 0x7fc0f415ce80 2023-11-06T03:13:20.236 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:20.236 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":577,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1077952432,"bytes_used":2752172032,"bytes_avail":770341941248,"bytes_total":773094113280,"write_bytes_sec":54635000,"read_op_per_sec":0,"write_op_per_sec":171},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-11-06T03:13:17.296264+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:20.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 msgr2=0x7fc0d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 0x7fc0d8067f20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40c6370 tx=0x7fc0f0008040 comp rx=0 tx=0).stop 2023-11-06T03:13:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 msgr2=0x7fc0f40cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:20.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b9e0 tx=0x7fc0ec00bea0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 shutdown_connections 2023-11-06T03:13:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc0d8065a60 0x7fc0d8067f20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f415ce80 0x7fc0f40cc450 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0f415caa0 0x7fc0f40cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 --2- 172.21.15.157:0/4172624376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0f40ce1a0 0x7fc0f40c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:20.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 >> 172.21.15.157:0/4172624376 conn(0x7fc0f4070790 msgr2=0x7fc0f40d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:20.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 shutdown_connections 2023-11-06T03:13:20.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:20.241+0000 7fc0fbf2c640 1 -- 172.21.15.157:0/4172624376 wait complete. 2023-11-06T03:13:20.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:13:21.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:21.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 Processor -- start 2023-11-06T03:13:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 -- start start 2023-11-06T03:13:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 0x7fc3a40cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 0x7fc3a40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40cb6e0 con 0x7fc3a415cb80 2023-11-06T03:13:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a40ce260 con 0x7fc3a40cc910 2023-11-06T03:13:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3abd87640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4071830 con 0x7fc3a415cf60 2023-11-06T03:13:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3a9b23640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fc3a415cb80 0x7fc3a40cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:42326/0 (socket says 172.21.15.157:42326) 2023-11-06T03:13:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3a9b23640 1 -- 172.21.15.157:0/4116296655 learned_addr learned my addr 172.21.15.157:0/4116296655 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3a9322640 1 --2- 172.21.15.157:0/4116296655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 0x7fc3a40cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.389+0000 7fc3aa324640 1 --2- 172.21.15.157:0/4116296655 >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 0x7fc3a40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a8b21640 1 -- 172.21.15.157:0/4116296655 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281610357 0 0) 0x7fc3a40cb6e0 con 0x7fc3a415cb80 2023-11-06T03:13:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a8b21640 1 -- 172.21.15.157:0/4116296655 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003530 con 0x7fc3a415cb80 2023-11-06T03:13:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a9322640 1 -- 172.21.15.157:0/4116296655 >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 msgr2=0x7fc3a40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a9322640 1 --2- 172.21.15.157:0/4116296655 >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 0x7fc3a40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a9322640 1 -- 172.21.15.157:0/4116296655 >> v1:172.21.15.195:6789/0 conn(0x7fc3a415cb80 legacy=0x7fc3a40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a9322640 1 -- 172.21.15.157:0/4116296655 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4071b10 con 0x7fc3a415cf60 2023-11-06T03:13:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3aa324640 1 --2- 172.21.15.157:0/4116296655 >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 0x7fc3a40cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:13:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a9322640 1 --2- 172.21.15.157:0/4116296655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 0x7fc3a40cd7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3940097c0 tx=0x7fc39401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e231387c922eff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a8b21640 1 -- 172.21.15.157:0/4116296655 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940ab680 con 0x7fc3a415cf60 2023-11-06T03:13:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a8b21640 1 -- 172.21.15.157:0/4116296655 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3940ab7e0 con 0x7fc3a415cf60 2023-11-06T03:13:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3a8b21640 1 -- 172.21.15.157:0/4116296655 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940abb50 con 0x7fc3a415cf60 2023-11-06T03:13:21.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 -- 172.21.15.157:0/4116296655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 msgr2=0x7fc3a40cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 --2- 172.21.15.157:0/4116296655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 0x7fc3a40cd7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3940097c0 tx=0x7fc39401c480 comp rx=0 tx=0).stop 2023-11-06T03:13:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 -- 172.21.15.157:0/4116296655 shutdown_connections 2023-11-06T03:13:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 --2- 172.21.15.157:0/4116296655 >> v2:172.21.15.195:3300/0 conn(0x7fc3a40cc910 0x7fc3a40cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 --2- 172.21.15.157:0/4116296655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a415cf60 0x7fc3a40cd7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 -- 172.21.15.157:0/4116296655 >> 172.21.15.157:0/4116296655 conn(0x7fc3a406d560 msgr2=0x7fc3a406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 -- 172.21.15.157:0/4116296655 shutdown_connections 2023-11-06T03:13:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.393+0000 7fc3abd87640 1 -- 172.21.15.157:0/4116296655 wait complete. 2023-11-06T03:13:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 Processor -- start 2023-11-06T03:13:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- start start 2023-11-06T03:13:21.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a40cc910 0x7fc3a41f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3a415cf60 0x7fc3a40b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40ca7b0 con 0x7fc3a415cb80 2023-11-06T03:13:21.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40c9920 con 0x7fc3a415cf60 2023-11-06T03:13:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40c9ba0 con 0x7fc3a40cc910 2023-11-06T03:13:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50878/0 (socket says 172.21.15.157:50878) 2023-11-06T03:13:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 -- 172.21.15.157:0/347736393 learned_addr learned my addr 172.21.15.157:0/347736393 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:21.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9b23640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a40cc910 0x7fc3a41f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3aa324640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3a415cf60 0x7fc3a40b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 -- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a40cc910 msgr2=0x7fc3a41f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a40cc910 0x7fc3a41f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 -- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3a415cf60 msgr2=0x7fc3a40b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3a415cf60 0x7fc3a40b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 -- 172.21.15.157:0/347736393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009a70 con 0x7fc3a415cb80 2023-11-06T03:13:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3a9322640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc3940ad040 tx=0x7fc394019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394004450 con 0x7fc3a415cb80 2023-11-06T03:13:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394009420 con 0x7fc3a415cb80 2023-11-06T03:13:21.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40ca160 con 0x7fc3a415cb80 2023-11-06T03:13:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3940ab950 con 0x7fc3a415cb80 2023-11-06T03:13:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940abce0 con 0x7fc3a415cb80 2023-11-06T03:13:21.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.397+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c005260 con 0x7fc3a415cb80 2023-11-06T03:13:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.401+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc394003620 con 0x7fc3a415cb80 2023-11-06T03:13:21.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.401+0000 7fc39a7fc640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 0x7fc388067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.401+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc39411d3f0 con 0x7fc3a415cb80 2023-11-06T03:13:21.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.401+0000 7fc3a9b23640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 0x7fc388067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.401+0000 7fc3a9b23640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 0x7fc388067f50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007fd0 tx=0x7fc3a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.405+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc3940eac40 con 0x7fc3a415cb80 2023-11-06T03:13:21.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc38c0050d0 con 0x7fc3a415cb80 2023-11-06T03:13:21.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc39a7fc640 1 -- 172.21.15.157:0/347736393 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fc3940ee260 con 0x7fc3a415cb80 2023-11-06T03:13:21.590 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:21.591 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:21.592 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 msgr2=0x7fc388067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 0x7fc388067f50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007fd0 tx=0x7fc3a0009040 comp rx=0 tx=0).stop 2023-11-06T03:13:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 msgr2=0x7fc3a40c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc3940ad040 tx=0x7fc394019220 comp rx=0 tx=0).stop 2023-11-06T03:13:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 shutdown_connections 2023-11-06T03:13:21.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc388065a90 0x7fc388067f50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3a415cf60 0x7fc3a40b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3a415cb80 0x7fc3a40c93e0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 --2- 172.21.15.157:0/347736393 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3a40cc910 0x7fc3a41f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 >> 172.21.15.157:0/347736393 conn(0x7fc3a406d560 msgr2=0x7fc3a40b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 shutdown_connections 2023-11-06T03:13:21.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.593+0000 7fc3abd87640 1 -- 172.21.15.157:0/347736393 wait complete. 2023-11-06T03:13:21.609 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:21.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:21.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.725+0000 7fd07bc9e640 1 Processor -- start 2023-11-06T03:13:21.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 -- start start 2023-11-06T03:13:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd07415caa0 0x7fd0740cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd0740cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740d2920 con 0x7fd0740ce1a0 2023-11-06T03:13:21.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740d18a0 con 0x7fd07415caa0 2023-11-06T03:13:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07bc9e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740d1b60 con 0x7fd07415ce80 2023-11-06T03:13:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd0740ce1a0 0x7fd0740d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43408/0 (socket says 172.21.15.157:43408) 2023-11-06T03:13:21.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 -- 172.21.15.157:0/3873025359 learned_addr learned my addr 172.21.15.157:0/3873025359 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd079239640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd0740cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd079a3a640 1 --2- 172.21.15.157:0/3873025359 >> v2:172.21.15.195:3300/0 conn(0x7fd07415caa0 0x7fd0740cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405150137 0 0) 0x7fd0740d2920 con 0x7fd0740ce1a0 2023-11-06T03:13:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd060003530 con 0x7fd0740ce1a0 2023-11-06T03:13:21.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355284777 0 0) 0x7fd060003530 con 0x7fd0740ce1a0 2023-11-06T03:13:21.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 msgr2=0x7fd0740cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:21.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd0740cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 >> v2:172.21.15.195:3300/0 conn(0x7fd07415caa0 msgr2=0x7fd0740cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 --2- 172.21.15.157:0/3873025359 >> v2:172.21.15.195:3300/0 conn(0x7fd07415caa0 0x7fd0740cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740d1e40 con 0x7fd0740ce1a0 2023-11-06T03:13:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd06c001a00 con 0x7fd0740ce1a0 2023-11-06T03:13:21.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 >> v1:172.21.15.157:6789/0 conn(0x7fd0740ce1a0 legacy=0x7fd0740d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd060003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd060003f00 0x7fd0600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd060004d10 0x7fd060005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740d18a0 con 0x7fd060004d10 2023-11-06T03:13:21.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740d1b60 con 0x7fd060003f00 2023-11-06T03:13:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000b990 con 0x7fd07415ce80 2023-11-06T03:13:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd060004d10 0x7fd060005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd079a3a640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd060003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd079239640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd060003f00 0x7fd0600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 -- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 msgr2=0x7fd060003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd060003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 -- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd060003f00 msgr2=0x7fd0600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd060003f00 0x7fd0600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd07a23b640 1 -- 172.21.15.157:0/3873025359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060003530 con 0x7fd060004d10 2023-11-06T03:13:21.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.729+0000 7fd079a3a640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd07415ce80 0x7fd060003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07a23b640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd060004d10 0x7fd060005190 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ce5d0 tx=0x7fd06c01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0774bad6e3818cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0ad630 con 0x7fd060004d10 2023-11-06T03:13:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0a8070 con 0x7fd060004d10 2023-11-06T03:13:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd078a38640 1 -- 172.21.15.157:0/3873025359 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0ad9d0 con 0x7fd060004d10 2023-11-06T03:13:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 -- 172.21.15.157:0/3873025359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd060004d10 msgr2=0x7fd060005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/3873025359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd060004d10 0x7fd060005190 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ce5d0 tx=0x7fd06c01d340 comp rx=0 tx=0).stop 2023-11-06T03:13:21.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07a23b640 1 -- 172.21.15.157:0/3873025359 reap_dead start 2023-11-06T03:13:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 -- 172.21.15.157:0/3873025359 shutdown_connections 2023-11-06T03:13:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 -- 172.21.15.157:0/3873025359 >> 172.21.15.157:0/3873025359 conn(0x7fd074070790 msgr2=0x7fd074070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 -- 172.21.15.157:0/3873025359 shutdown_connections 2023-11-06T03:13:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 -- 172.21.15.157:0/3873025359 wait complete. 2023-11-06T03:13:21.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.733+0000 7fd07bc9e640 1 Processor -- start 2023-11-06T03:13:21.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- start start 2023-11-06T03:13:21.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 0x7fd0740aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0740ca230 0x7fd0740a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740ab770 con 0x7fd0740ab2f0 2023-11-06T03:13:21.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740cb5f0 con 0x7fd0740ca230 2023-11-06T03:13:21.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740ca9e0 con 0x7fd0741540a0 2023-11-06T03:13:21.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50900/0 (socket says 172.21.15.157:50900) 2023-11-06T03:13:21.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 -- 172.21.15.157:0/4273306202 learned_addr learned my addr 172.21.15.157:0/4273306202 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:21.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079a3a640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 0x7fd0740aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07a23b640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0740ca230 0x7fd0740a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 -- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 msgr2=0x7fd0740aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 0x7fd0740aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 -- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0740ca230 msgr2=0x7fd0740a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0740ca230 0x7fd0740a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 -- 172.21.15.157:0/4273306202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740cacc0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079a3a640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 0x7fd0740aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:21.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd079239640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd064011560 tx=0x7fd064011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064015930 con 0x7fd0740ab2f0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064015a90 con 0x7fd0740ab2f0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0740cafa0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064015e60 con 0x7fd0740ab2f0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0740ca6b0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0740cb7a0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd06400f460 con 0x7fd0740ab2f0 2023-11-06T03:13:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 0x7fd058067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.737+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd064083ae0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.741+0000 7fd079a3a640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 0x7fd058067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.741+0000 7fd079a3a640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 0x7fd058067f50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ccd60 tx=0x7fd070008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:21.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.745+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd0740cb7a0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.953+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd0740cb7a0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.953+0000 7fd06a7fc640 1 -- 172.21.15.157:0/4273306202 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1633 (secure 0 0 0) 0x7fd0740cb7a0 con 0x7fd0740ab2f0 2023-11-06T03:13:21.951 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:21.952 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":579,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1077952432,"bytes_used":2752172032,"bytes_avail":770341941248,"bytes_total":773094113280,"write_bytes_sec":28607629,"read_op_per_sec":0,"write_op_per_sec":146},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2023-11-06T03:13:17.296264+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:21.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 msgr2=0x7fd058067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 0x7fd058067f50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ccd60 tx=0x7fd070008040 comp rx=0 tx=0).stop 2023-11-06T03:13:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 msgr2=0x7fd0740c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd064011560 tx=0x7fd064011a20 comp rx=0 tx=0).stop 2023-11-06T03:13:21.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 shutdown_connections 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd058065a90 0x7fd058067f50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0740ca230 0x7fd0740a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0740ab2f0 0x7fd0740c9cf0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 --2- 172.21.15.157:0/4273306202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd0741540a0 0x7fd0740aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 >> 172.21.15.157:0/4273306202 conn(0x7fd074070790 msgr2=0x7fd0740d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:21.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 shutdown_connections 2023-11-06T03:13:21.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:21.957+0000 7fd07bc9e640 1 -- 172.21.15.157:0/4273306202 wait complete. 2023-11-06T03:13:21.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:13:22.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:23.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.105+0000 7ff532e52640 1 Processor -- start 2023-11-06T03:13:23.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 -- start start 2023-11-06T03:13:23.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff52c15cb80 0x7ff52c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff52c0cdef0 0x7ff52c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c071830 con 0x7ff52c15cf60 2023-11-06T03:13:23.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c071b10 con 0x7ff52c15cb80 2023-11-06T03:13:23.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff532e52640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c071df0 con 0x7ff52c0cdef0 2023-11-06T03:13:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff52c15cb80 0x7ff52c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff52c15cb80 0x7ff52c0cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50908/0 (socket says 172.21.15.157:50908) 2023-11-06T03:13:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 -- 172.21.15.157:0/3251092722 learned_addr learned my addr 172.21.15.157:0/3251092722 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff5313ef640 1 --2- 172.21.15.157:0/3251092722 >> v2:172.21.15.195:3300/0 conn(0x7ff52c0cdef0 0x7ff52c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 -- 172.21.15.157:0/3251092722 >> v2:172.21.15.195:3300/0 conn(0x7ff52c0cdef0 msgr2=0x7ff52c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 --2- 172.21.15.157:0/3251092722 >> v2:172.21.15.195:3300/0 conn(0x7ff52c0cdef0 0x7ff52c0cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 -- 172.21.15.157:0/3251092722 >> v1:172.21.15.195:6789/0 conn(0x7ff52c15cf60 legacy=0x7ff52c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 -- 172.21.15.157:0/3251092722 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0ccfe0 con 0x7ff52c15cb80 2023-11-06T03:13:23.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.109+0000 7ff530bee640 1 --2- 172.21.15.157:0/3251092722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cb80 0x7ff52c0cafd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0c4450 tx=0x7ff51c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876f0870776c587c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff52b7fe640 1 -- 172.21.15.157:0/3251092722 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0a89b0 con 0x7ff52c15cb80 2023-11-06T03:13:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff52b7fe640 1 -- 172.21.15.157:0/3251092722 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c0a8b10 con 0x7ff52c15cb80 2023-11-06T03:13:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff52b7fe640 1 -- 172.21.15.157:0/3251092722 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0a8ea0 con 0x7ff52c15cb80 2023-11-06T03:13:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/3251092722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cb80 msgr2=0x7ff52c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- 172.21.15.157:0/3251092722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cb80 0x7ff52c0cafd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0c4450 tx=0x7ff51c01ecb0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/3251092722 shutdown_connections 2023-11-06T03:13:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- 172.21.15.157:0/3251092722 >> v2:172.21.15.195:3300/0 conn(0x7ff52c0cdef0 0x7ff52c0cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- 172.21.15.157:0/3251092722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cb80 0x7ff52c0cafd0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/3251092722 >> 172.21.15.157:0/3251092722 conn(0x7ff52c06d560 msgr2=0x7ff52c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/3251092722 shutdown_connections 2023-11-06T03:13:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/3251092722 wait complete. 2023-11-06T03:13:23.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 Processor -- start 2023-11-06T03:13:23.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- start start 2023-11-06T03:13:23.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff52c0cdef0 0x7ff52c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff52c15cb80 0x7ff52c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1f6760 con 0x7ff52c15cf60 2023-11-06T03:13:23.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0adaa0 con 0x7ff52c15cb80 2023-11-06T03:13:23.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1f5ae0 con 0x7ff52c0cdef0 2023-11-06T03:13:23.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50912/0 (socket says 172.21.15.157:50912) 2023-11-06T03:13:23.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 -- 172.21.15.157:0/293527446 learned_addr learned my addr 172.21.15.157:0/293527446 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:23.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff530bee640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff52c0cdef0 0x7ff52c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff52bfff640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff52c15cb80 0x7ff52c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 -- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff52c0cdef0 msgr2=0x7ff52c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff52c0cdef0 0x7ff52c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 -- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff52c15cb80 msgr2=0x7ff52c0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff52c15cb80 0x7ff52c0ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 -- 172.21.15.157:0/293527446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0094d0 con 0x7ff52c15cf60 2023-11-06T03:13:23.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5313ef640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff524007e70 tx=0x7ff52400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524014620 con 0x7ff52c15cf60 2023-11-06T03:13:23.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5240147a0 con 0x7ff52c15cf60 2023-11-06T03:13:23.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524014b50 con 0x7ff52c15cf60 2023-11-06T03:13:23.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c1f5dc0 con 0x7ff52c15cf60 2023-11-06T03:13:23.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.113+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1f6380 con 0x7ff52c15cf60 2023-11-06T03:13:23.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c1f68a0 con 0x7ff52c15cf60 2023-11-06T03:13:23.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff524014cb0 con 0x7ff52c15cf60 2023-11-06T03:13:23.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff5297fa640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 0x7ff518067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff524083c00 con 0x7ff52c15cf60 2023-11-06T03:13:23.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff530bee640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 0x7ff518067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.117+0000 7ff530bee640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 0x7ff518067f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0b2000 tx=0x7ff51c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.121+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff52c1f68a0 con 0x7ff52c15cf60 2023-11-06T03:13:23.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.305+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff52c1f68a0 con 0x7ff52c15cf60 2023-11-06T03:13:23.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff5297fa640 1 -- 172.21.15.157:0/293527446 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7ff52c1f68a0 con 0x7ff52c15cf60 2023-11-06T03:13:23.305 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:23.305 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:23.305 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:23.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 msgr2=0x7ff518067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 0x7ff518067f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0b2000 tx=0x7ff51c008040 comp rx=0 tx=0).stop 2023-11-06T03:13:23.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 msgr2=0x7ff52c0ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff524007e70 tx=0x7ff52400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 shutdown_connections 2023-11-06T03:13:23.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff518065a60 0x7ff518067f20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff52c15cf60 0x7ff52c0ad560 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff52c15cb80 0x7ff52c0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 --2- 172.21.15.157:0/293527446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff52c0cdef0 0x7ff52c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 >> 172.21.15.157:0/293527446 conn(0x7ff52c06d560 msgr2=0x7ff52c06ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:23.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 shutdown_connections 2023-11-06T03:13:23.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.309+0000 7ff532e52640 1 -- 172.21.15.157:0/293527446 wait complete. 2023-11-06T03:13:23.322 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:13:23.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:23.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 Processor -- start 2023-11-06T03:13:23.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 -- start start 2023-11-06T03:13:23.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d3c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c0cd170 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3c15af30 con 0x7f9d3c15ab50 2023-11-06T03:13:23.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d41e91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0b3e10 con 0x7f9d3c071820 2023-11-06T03:13:23.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9d3c0cc9b0 0x7f9d3c0b4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43420/0 (socket says 172.21.15.157:43420) 2023-11-06T03:13:23.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1934997544 learned_addr learned my addr 172.21.15.157:0/1934997544 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:23.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d3c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193353769 0 0) 0x7f9d3c0cd170 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d28003530 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869132847 0 0) 0x7f9d28003530 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 msgr2=0x7f9d3c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d3c0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 >> v1:172.21.15.195:6789/0 conn(0x7f9d3c15ab50 legacy=0x7f9d3c0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0b40f0 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9d34001a00 con 0x7f9d3c0cc9b0 2023-11-06T03:13:23.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 >> v1:172.21.15.157:6789/0 conn(0x7f9d3c0cc9b0 legacy=0x7f9d3c0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d28003f30 0x7f9d280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28004d10 0x7f9d28005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0b3e10 con 0x7f9d28003f30 2023-11-06T03:13:23.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2800ba70 con 0x7f9d28004d10 2023-11-06T03:13:23.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2800bc30 con 0x7f9d3c071820 2023-11-06T03:13:23.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d28003f30 0x7f9d280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3b7fe640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d3bfff640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28004d10 0x7f9d28005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 -- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 msgr2=0x7f9d280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c071820 0x7f9d280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 -- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28004d10 msgr2=0x7f9d28005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28004d10 0x7f9d28005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.441+0000 7f9d32ffd640 1 -- 172.21.15.157:0/1934997544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28003530 con 0x7f9d28003f30 2023-11-06T03:13:23.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d28003f30 0x7f9d280047d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009ae0 tx=0x7f9d2401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ece08883628e53 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240a5640 con 0x7f9d28003f30 2023-11-06T03:13:23.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d240a57a0 con 0x7f9d28003f30 2023-11-06T03:13:23.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d3affd640 1 -- 172.21.15.157:0/1934997544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240a5b50 con 0x7f9d28003f30 2023-11-06T03:13:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- 172.21.15.157:0/1934997544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d28003f30 msgr2=0x7f9d280047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 --2- 172.21.15.157:0/1934997544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d28003f30 0x7f9d280047d0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009ae0 tx=0x7f9d2401ca50 comp rx=0 tx=0).stop 2023-11-06T03:13:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d3bfff640 1 -- 172.21.15.157:0/1934997544 reap_dead start 2023-11-06T03:13:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- 172.21.15.157:0/1934997544 shutdown_connections 2023-11-06T03:13:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- 172.21.15.157:0/1934997544 >> 172.21.15.157:0/1934997544 conn(0x7f9d3c065d30 msgr2=0x7f9d3c066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- 172.21.15.157:0/1934997544 shutdown_connections 2023-11-06T03:13:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- 172.21.15.157:0/1934997544 wait complete. 2023-11-06T03:13:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 Processor -- start 2023-11-06T03:13:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.445+0000 7f9d41e91640 1 -- start start 2023-11-06T03:13:23.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28003f30 0x7f9d3c0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c0c9c00 0x7f9d3c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c14c960 con 0x7f9d3c0ca590 2023-11-06T03:13:23.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c14cc10 con 0x7f9d28003f30 2023-11-06T03:13:23.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c0b1080 con 0x7f9d3c0c9c00 2023-11-06T03:13:23.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50942/0 (socket says 172.21.15.157:50942) 2023-11-06T03:13:23.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 -- 172.21.15.157:0/792549113 learned_addr learned my addr 172.21.15.157:0/792549113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d3b7fe640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28003f30 0x7f9d3c0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d3bfff640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c0c9c00 0x7f9d3c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 -- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c0c9c00 msgr2=0x7f9d3c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c0c9c00 0x7f9d3c0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 -- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28003f30 msgr2=0x7f9d3c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28003f30 0x7f9d3c0d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 -- 172.21.15.157:0/792549113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c14cee0 con 0x7f9d3c0ca590 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d32ffd640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009ae0 tx=0x7f9d28005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240a5640 con 0x7f9d3c0ca590 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24004220 con 0x7f9d3c0ca590 2023-11-06T03:13:23.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c14d1c0 con 0x7f9d3c0ca590 2023-11-06T03:13:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003620 con 0x7f9d3c0ca590 2023-11-06T03:13:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c0b0d10 con 0x7f9d3c0ca590 2023-11-06T03:13:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3c072750 con 0x7f9d3c0ca590 2023-11-06T03:13:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d240b4040 con 0x7f9d3c0ca590 2023-11-06T03:13:23.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f9d2411db10 con 0x7f9d3c0ca590 2023-11-06T03:13:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.449+0000 7f9d3b7fe640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.453+0000 7f9d3b7fe640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 0x7f9d1c067f20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c10 tx=0x7f9d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.453+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9d240eb400 con 0x7f9d3c0ca590 2023-11-06T03:13:23.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9d3c14c960 con 0x7f9d3c0ca590 2023-11-06T03:13:23.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d38ff9640 1 -- 172.21.15.157:0/792549113 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1577 (secure 0 0 0) 0x7f9d240eea20 con 0x7f9d3c0ca590 2023-11-06T03:13:23.666 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:23.666 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":581,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1077952432,"bytes_used":2752172032,"bytes_avail":770341941248,"bytes_total":773094113280,"write_bytes_sec":28608507,"read_op_per_sec":0,"write_op_per_sec":146},"fsmap":{"epoch":85,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8226}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-11-06T03:13:23.298350+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:23.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 msgr2=0x7f9d1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 0x7f9d1c067f20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c10 tx=0x7f9d2c009040 comp rx=0 tx=0).stop 2023-11-06T03:13:23.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 msgr2=0x7f9d3c15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009ae0 tx=0x7f9d28005800 comp rx=0 tx=0).stop 2023-11-06T03:13:23.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 shutdown_connections 2023-11-06T03:13:23.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d1c065a60 0x7f9d1c067f20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d3c0c9c00 0x7f9d3c0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d3c0ca590 0x7f9d3c15bbf0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 --2- 172.21.15.157:0/792549113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d28003f30 0x7f9d3c0d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 >> 172.21.15.157:0/792549113 conn(0x7f9d3c065d30 msgr2=0x7f9d3c0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 shutdown_connections 2023-11-06T03:13:23.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.669+0000 7f9d41e91640 1 -- 172.21.15.157:0/792549113 wait complete. 2023-11-06T03:13:23.684 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:13:23.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:13:23.685 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:13:23.726 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:13:23.781 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:13:23.782 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:13:23.782 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:23.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:23.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 Processor -- start 2023-11-06T03:13:23.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 -- start start 2023-11-06T03:13:23.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de815ce80 0x7f7de80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de80d2920 con 0x7f7de815caa0 2023-11-06T03:13:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de80d18a0 con 0x7f7de80d00b0 2023-11-06T03:13:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7ded5c2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de80d0520 con 0x7f7de815ce80 2023-11-06T03:13:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de815ce80 0x7f7de80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de6ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7de815caa0 0x7f7de80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43434/0 (socket says 172.21.15.157:43434) 2023-11-06T03:13:23.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de6ffd640 1 -- 172.21.15.157:0/1028840259 learned_addr learned my addr 172.21.15.157:0/1028840259 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:23.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de67fc640 1 -- 172.21.15.157:0/1028840259 >> v1:172.21.15.195:6789/0 conn(0x7f7de80d00b0 legacy=0x7f7de80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:23.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de67fc640 1 -- 172.21.15.157:0/1028840259 >> v1:172.21.15.157:6789/0 conn(0x7f7de815caa0 legacy=0x7f7de80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de5ffb640 1 -- 172.21.15.157:0/1028840259 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334107326 0 0) 0x7f7de80d2920 con 0x7f7de815caa0 2023-11-06T03:13:23.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.961+0000 7f7de67fc640 1 -- 172.21.15.157:0/1028840259 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80d1b60 con 0x7f7de815ce80 2023-11-06T03:13:23.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 --2- 172.21.15.157:0/1028840259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815ce80 0x7f7de80cfb70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05ad702764b304f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de5ffb640 1 -- 172.21.15.157:0/1028840259 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab650 con 0x7f7de815ce80 2023-11-06T03:13:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de5ffb640 1 -- 172.21.15.157:0/1028840259 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80ab7b0 con 0x7f7de815ce80 2023-11-06T03:13:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de5ffb640 1 -- 172.21.15.157:0/1028840259 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80abb40 con 0x7f7de815ce80 2023-11-06T03:13:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- 172.21.15.157:0/1028840259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815ce80 msgr2=0x7f7de80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/1028840259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815ce80 0x7f7de80cfb70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c450 comp rx=0 tx=0).stop 2023-11-06T03:13:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- 172.21.15.157:0/1028840259 shutdown_connections 2023-11-06T03:13:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/1028840259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815ce80 0x7f7de80cfb70 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- 172.21.15.157:0/1028840259 >> 172.21.15.157:0/1028840259 conn(0x7f7de8070790 msgr2=0x7f7de8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:23.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- 172.21.15.157:0/1028840259 shutdown_connections 2023-11-06T03:13:23.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- 172.21.15.157:0/1028840259 wait complete. 2023-11-06T03:13:23.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 Processor -- start 2023-11-06T03:13:23.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- start start 2023-11-06T03:13:23.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de80d00b0 0x7f7de8153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de815ce80 0x7f7de815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de8154cf0 con 0x7f7de815caa0 2023-11-06T03:13:23.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de8150db0 con 0x7f7de80d00b0 2023-11-06T03:13:23.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7ded5c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de81540e0 con 0x7f7de815ce80 2023-11-06T03:13:23.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50950/0 (socket says 172.21.15.157:50950) 2023-11-06T03:13:23.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 -- 172.21.15.157:0/4219163113 learned_addr learned my addr 172.21.15.157:0/4219163113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:23.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de77fe640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de815ce80 0x7f7de815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de6ffd640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de80d00b0 0x7f7de8153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 -- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de815ce80 msgr2=0x7f7de815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de815ce80 0x7f7de815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 -- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de80d00b0 msgr2=0x7f7de8153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:23.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de80d00b0 0x7f7de8153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:23.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.965+0000 7f7de67fc640 1 -- 172.21.15.157:0/4219163113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009420 con 0x7f7de815caa0 2023-11-06T03:13:23.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7de67fc640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad040 tx=0x7f7dd801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80b5810 con 0x7f7de815caa0 2023-11-06T03:13:23.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80b5990 con 0x7f7de815caa0 2023-11-06T03:13:23.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de81543c0 con 0x7f7de815caa0 2023-11-06T03:13:23.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80b5d40 con 0x7f7de815caa0 2023-11-06T03:13:23.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de8154980 con 0x7f7de815caa0 2023-11-06T03:13:23.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de8072750 con 0x7f7de815caa0 2023-11-06T03:13:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7dd80c83f0 con 0x7f7de815caa0 2023-11-06T03:13:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 0x7f7dd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7dd811e390 con 0x7f7de815caa0 2023-11-06T03:13:23.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7de6ffd640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 0x7f7dd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:23.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.969+0000 7f7de6ffd640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 0x7f7dd4067f20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007fd0 tx=0x7f7ddc007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:23.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:23.973+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7dd80ebd80 con 0x7f7de815caa0 2023-11-06T03:13:24.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.117+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7de80b8150 con 0x7f7de815caa0 2023-11-06T03:13:24.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.121+0000 7f7dcf7fe640 1 -- 172.21.15.157:0/4219163113 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17923 (secure 0 0 0) 0x7f7dd80ef3a0 con 0x7f7de815caa0 2023-11-06T03:13:24.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:24.119 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":113,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:50.403519+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 msgr2=0x7f7dd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 0x7f7dd4067f20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007fd0 tx=0x7f7ddc007ba0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 msgr2=0x7f7de8150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad040 tx=0x7f7dd801ac90 comp rx=0 tx=0).stop 2023-11-06T03:13:24.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 shutdown_connections 2023-11-06T03:13:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd4065a60 0x7f7dd4067f20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de815ce80 0x7f7de815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de815caa0 0x7f7de8150870 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 --2- 172.21.15.157:0/4219163113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de80d00b0 0x7f7de8153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 >> 172.21.15.157:0/4219163113 conn(0x7f7de8070790 msgr2=0x7f7de8070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 shutdown_connections 2023-11-06T03:13:24.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.125+0000 7f7ded5c2640 1 -- 172.21.15.157:0/4219163113 wait complete. 2023-11-06T03:13:24.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:12:43.951782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:12:44.319880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:13:24.137 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:13:24.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 Processor -- start 2023-11-06T03:13:24.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 -- start start 2023-11-06T03:13:24.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58101599c0 0x7f58100ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f58100b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f581015dea0 0x7f58100b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581015e440 con 0x7f58101599c0 2023-11-06T03:13:24.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581006c240 con 0x7f581015dea0 2023-11-06T03:13:24.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f581510a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581006c520 con 0x7f581006d000 2023-11-06T03:13:24.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58101599c0 0x7f58100ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58101599c0 0x7f58100ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50960/0 (socket says 172.21.15.157:50960) 2023-11-06T03:13:24.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 -- 172.21.15.157:0/2035220688 learned_addr learned my addr 172.21.15.157:0/2035220688 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:24.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580e59c640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f58100b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580f59e640 1 --2- 172.21.15.157:0/2035220688 >> v2:172.21.15.195:3300/0 conn(0x7f581015dea0 0x7f58100b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 -- 172.21.15.157:0/2035220688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 msgr2=0x7f58100b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f58100b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 -- 172.21.15.157:0/2035220688 >> v2:172.21.15.195:3300/0 conn(0x7f581015dea0 msgr2=0x7f58100b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 --2- 172.21.15.157:0/2035220688 >> v2:172.21.15.195:3300/0 conn(0x7f581015dea0 0x7f58100b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.257+0000 7f580ed9d640 1 -- 172.21.15.157:0/2035220688 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581006c800 con 0x7f58101599c0 2023-11-06T03:13:24.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580ed9d640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58101599c0 0x7f58100ae000 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f581006bbf0 tx=0x7f580001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6664aa4b1c71d0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580dd9b640 1 -- 172.21.15.157:0/2035220688 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000090d0 con 0x7f58101599c0 2023-11-06T03:13:24.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580dd9b640 1 -- 172.21.15.157:0/2035220688 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5800009230 con 0x7f58101599c0 2023-11-06T03:13:24.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580dd9b640 1 -- 172.21.15.157:0/2035220688 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a5950 con 0x7f58101599c0 2023-11-06T03:13:24.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- 172.21.15.157:0/2035220688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58101599c0 msgr2=0x7f58100ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58101599c0 0x7f58100ae000 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f581006bbf0 tx=0x7f580001c9a0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- 172.21.15.157:0/2035220688 shutdown_connections 2023-11-06T03:13:24.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- 172.21.15.157:0/2035220688 >> v2:172.21.15.195:3300/0 conn(0x7f581015dea0 0x7f58100b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f58100b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- 172.21.15.157:0/2035220688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58101599c0 0x7f58100ae000 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- 172.21.15.157:0/2035220688 >> 172.21.15.157:0/2035220688 conn(0x7f5810065930 msgr2=0x7f5810065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- 172.21.15.157:0/2035220688 shutdown_connections 2023-11-06T03:13:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- 172.21.15.157:0/2035220688 wait complete. 2023-11-06T03:13:24.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 Processor -- start 2023-11-06T03:13:24.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- start start 2023-11-06T03:13:24.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f5810141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58101599c0 0x7f58100ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58101e5730 con 0x7f581015dea0 2023-11-06T03:13:24.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100cb9d0 con 0x7f58101599c0 2023-11-06T03:13:24.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f581510a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58101e5450 con 0x7f581006d000 2023-11-06T03:13:24.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50968/0 (socket says 172.21.15.157:50968) 2023-11-06T03:13:24.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.261+0000 7f580f59e640 1 -- 172.21.15.157:0/3624184055 learned_addr learned my addr 172.21.15.157:0/3624184055 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:24.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 -- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 msgr2=0x7f5810141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580e59c640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58101599c0 0x7f58100ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580ed9d640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f5810141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f5810141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 -- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58101599c0 msgr2=0x7f58100ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58101599c0 0x7f58100ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 -- 172.21.15.157:0/3624184055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800009b30 con 0x7f581015dea0 2023-11-06T03:13:24.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580f59e640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5808009e60 tx=0x7f58080104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808016420 con 0x7f581015dea0 2023-11-06T03:13:24.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080165a0 con 0x7f581015dea0 2023-11-06T03:13:24.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808016950 con 0x7f581015dea0 2023-11-06T03:13:24.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5810142540 con 0x7f581015dea0 2023-11-06T03:13:24.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5810142b00 con 0x7f581015dea0 2023-11-06T03:13:24.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58101423a0 con 0x7f581015dea0 2023-11-06T03:13:24.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5808016ab0 con 0x7f581015dea0 2023-11-06T03:13:24.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 0x7f57fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5808085ac0 con 0x7f581015dea0 2023-11-06T03:13:24.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580ed9d640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 0x7f57fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.265+0000 7f580ed9d640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 0x7f57fc067f20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f58000ae000 tx=0x7f58000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.269+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58101423a0 con 0x7f581015dea0 2023-11-06T03:13:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58101423a0 con 0x7f581015dea0 2023-11-06T03:13:24.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f57f77fe640 1 -- 172.21.15.157:0/3624184055 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5808085e60 con 0x7f581015dea0 2023-11-06T03:13:24.418 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:13:24.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 msgr2=0x7f57fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 0x7f57fc067f20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f58000ae000 tx=0x7f58000a7070 comp rx=0 tx=0).stop 2023-11-06T03:13:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 msgr2=0x7f58100ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5808009e60 tx=0x7f58080104b0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 shutdown_connections 2023-11-06T03:13:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57fc065a60 0x7f57fc067f20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f581015dea0 0x7f58100ce5f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58101599c0 0x7f58100ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 --2- 172.21.15.157:0/3624184055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f581006d000 0x7f5810141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 >> 172.21.15.157:0/3624184055 conn(0x7f5810065930 msgr2=0x7f58100b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 shutdown_connections 2023-11-06T03:13:24.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.421+0000 7f581510a640 1 -- 172.21.15.157:0/3624184055 wait complete. 2023-11-06T03:13:24.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:13:24.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.557+0000 7fea8645f640 1 Processor -- start 2023-11-06T03:13:24.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.557+0000 7fea8645f640 1 -- start start 2023-11-06T03:13:24.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea8645f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fea8015e5e0 0x7fea800b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea8645f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea800b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea8645f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8015f580 con 0x7fea801551e0 2023-11-06T03:13:24.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea8645f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea800704b0 con 0x7fea8015e5e0 2023-11-06T03:13:24.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea8645f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8015efc0 con 0x7fea8015ea90 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fea801551e0 0x7fea8006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43438/0 (socket says 172.21.15.157:43438) 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7ffff640 1 -- 172.21.15.157:0/600831981 learned_addr learned my addr 172.21.15.157:0/600831981 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866190270 0 0) 0x7fea8015f580 con 0x7fea801551e0 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/600831981 >> v2:172.21.15.195:3300/0 conn(0x7fea8015e5e0 0x7fea800b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea849fc640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea800b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68003530 con 0x7fea801551e0 2023-11-06T03:13:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 299545745 0 0) 0x7fea68003530 con 0x7fea801551e0 2023-11-06T03:13:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 msgr2=0x7fea800b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea800b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 >> v2:172.21.15.195:3300/0 conn(0x7fea8015e5e0 msgr2=0x7fea800b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 --2- 172.21.15.157:0/600831981 >> v2:172.21.15.195:3300/0 conn(0x7fea8015e5e0 0x7fea800b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8015f2a0 con 0x7fea801551e0 2023-11-06T03:13:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fea70001680 con 0x7fea801551e0 2023-11-06T03:13:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 >> v1:172.21.15.157:6789/0 conn(0x7fea801551e0 legacy=0x7fea8006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea68003980 0x7fea68003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea680047d0 0x7fea68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea680051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800704b0 con 0x7fea680047d0 2023-11-06T03:13:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8015efc0 con 0x7fea68003980 2023-11-06T03:13:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6800bb20 con 0x7fea8015ea90 2023-11-06T03:13:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea680047d0 0x7fea68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7ffff640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea68003980 0x7fea68003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea849fc640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea680051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 -- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 msgr2=0x7fea680051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea680051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 -- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea68003980 msgr2=0x7fea68003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea68003980 0x7fea68003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea7f7fe640 1 -- 172.21.15.157:0/600831981 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68003530 con 0x7fea680047d0 2023-11-06T03:13:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.561+0000 7fea849fc640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea8015ea90 0x7fea680051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea680047d0 0x7fea68004c50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ba30 tx=0x7fea7401c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd43496d6409b59a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740ac630 con 0x7fea680047d0 2023-11-06T03:13:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740ac790 con 0x7fea680047d0 2023-11-06T03:13:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7effd640 1 -- 172.21.15.157:0/600831981 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740acb40 con 0x7fea680047d0 2023-11-06T03:13:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- 172.21.15.157:0/600831981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea680047d0 msgr2=0x7fea68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 --2- 172.21.15.157:0/600831981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea680047d0 0x7fea68004c50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ba30 tx=0x7fea7401c690 comp rx=0 tx=0).stop 2023-11-06T03:13:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 -- 172.21.15.157:0/600831981 reap_dead start 2023-11-06T03:13:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- 172.21.15.157:0/600831981 shutdown_connections 2023-11-06T03:13:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- 172.21.15.157:0/600831981 >> 172.21.15.157:0/600831981 conn(0x7fea80065930 msgr2=0x7fea80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- 172.21.15.157:0/600831981 shutdown_connections 2023-11-06T03:13:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- 172.21.15.157:0/600831981 wait complete. 2023-11-06T03:13:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 Processor -- start 2023-11-06T03:13:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- start start 2023-11-06T03:13:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea680047d0 0x7fea8014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea80151d70 0x7fea801414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80144ac0 con 0x7fea80148420 2023-11-06T03:13:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80150010 con 0x7fea80151d70 2023-11-06T03:13:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea8645f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea80152270 con 0x7fea680047d0 2023-11-06T03:13:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50980/0 (socket says 172.21.15.157:50980) 2023-11-06T03:13:24.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 -- 172.21.15.157:0/21712288 learned_addr learned my addr 172.21.15.157:0/21712288 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:24.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7ffff640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea680047d0 0x7fea8014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7f7fe640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea80151d70 0x7fea801414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 -- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea680047d0 msgr2=0x7fea8014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea680047d0 0x7fea8014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 -- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea80151d70 msgr2=0x7fea801414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea80151d70 0x7fea801414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 -- 172.21.15.157:0/21712288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8014c1d0 con 0x7fea80148420 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea849fc640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fea8015b010 tx=0x7fea78009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.565+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78007c90 con 0x7fea80148420 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78007e10 con 0x7fea80148420 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8014c4b0 con 0x7fea80148420 2023-11-06T03:13:24.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78014640 con 0x7fea80148420 2023-11-06T03:13:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8014ca70 con 0x7fea80148420 2023-11-06T03:13:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea80072750 con 0x7fea80148420 2023-11-06T03:13:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fea7801d3b0 con 0x7fea80148420 2023-11-06T03:13:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7cff9640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 0x7fea6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fea78082fc0 con 0x7fea80148420 2023-11-06T03:13:24.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7ffff640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 0x7fea6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.569+0000 7fea7ffff640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 0x7fea6c067f20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fea70000c10 tx=0x7fea70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.573+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fea78050760 con 0x7fea80148420 2023-11-06T03:13:24.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.721+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea80071730 con 0x7fea80148420 2023-11-06T03:13:24.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.721+0000 7fea7cff9640 1 -- 172.21.15.157:0/21712288 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea78053d80 con 0x7fea80148420 2023-11-06T03:13:24.719 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:13:24.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 msgr2=0x7fea6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 0x7fea6c067f20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fea70000c10 tx=0x7fea70009040 comp rx=0 tx=0).stop 2023-11-06T03:13:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 msgr2=0x7fea801488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fea8015b010 tx=0x7fea78009d40 comp rx=0 tx=0).stop 2023-11-06T03:13:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 shutdown_connections 2023-11-06T03:13:24.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fea6c065a60 0x7fea6c067f20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea80148420 0x7fea801488d0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fea80151d70 0x7fea801414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 --2- 172.21.15.157:0/21712288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fea680047d0 0x7fea8014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 >> 172.21.15.157:0/21712288 conn(0x7fea80065930 msgr2=0x7fea800afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 shutdown_connections 2023-11-06T03:13:24.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.725+0000 7fea8645f640 1 -- 172.21.15.157:0/21712288 wait complete. 2023-11-06T03:13:24.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:13:24.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 Processor -- start 2023-11-06T03:13:24.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 -- start start 2023-11-06T03:13:24.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f595c0adae0 0x7f595c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f595c0adf40 0x7f595c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c06c490 con 0x7f595c156a70 2023-11-06T03:13:24.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595c06b950 con 0x7f595c0adae0 2023-11-06T03:13:24.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5961199640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595c06fb80 con 0x7f595c0adf40 2023-11-06T03:13:24.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f595c156a70 0x7f595c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43446/0 (socket says 172.21.15.157:43446) 2023-11-06T03:13:24.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f595c0adae0 0x7f595c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595ad9d640 1 -- 172.21.15.157:0/4111434524 learned_addr learned my addr 172.21.15.157:0/4111434524 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:24.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595b59e640 1 --2- 172.21.15.157:0/4111434524 >> v2:172.21.15.195:3300/0 conn(0x7f595c0adf40 0x7f595c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595a59c640 1 -- 172.21.15.157:0/4111434524 >> v2:172.21.15.195:3300/0 conn(0x7f595c0adf40 msgr2=0x7f595c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:24.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595a59c640 1 --2- 172.21.15.157:0/4111434524 >> v2:172.21.15.195:3300/0 conn(0x7f595c0adf40 0x7f595c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595a59c640 1 -- 172.21.15.157:0/4111434524 >> v1:172.21.15.157:6789/0 conn(0x7f595c156a70 legacy=0x7f595c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f595a59c640 1 -- 172.21.15.157:0/4111434524 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c06bb60 con 0x7f595c0adae0 2023-11-06T03:13:24.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.869+0000 7f5959d9b640 1 -- 172.21.15.157:0/4111434524 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939072125 0 0) 0x7f595c06c490 con 0x7f595c156a70 2023-11-06T03:13:24.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- 172.21.15.157:0/4111434524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adae0 0x7f595c0b1c20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ce830 tx=0x7f595001d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2409276619cf3dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5959d9b640 1 -- 172.21.15.157:0/4111434524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a6830 con 0x7f595c0adae0 2023-11-06T03:13:24.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5959d9b640 1 -- 172.21.15.157:0/4111434524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500a6990 con 0x7f595c0adae0 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5959d9b640 1 -- 172.21.15.157:0/4111434524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a6d20 con 0x7f595c0adae0 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/4111434524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adae0 msgr2=0x7f595c0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- 172.21.15.157:0/4111434524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adae0 0x7f595c0b1c20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f595c0ce830 tx=0x7f595001d800 comp rx=0 tx=0).stop 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/4111434524 shutdown_connections 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- 172.21.15.157:0/4111434524 >> v2:172.21.15.195:3300/0 conn(0x7f595c0adf40 0x7f595c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- 172.21.15.157:0/4111434524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adae0 0x7f595c0b1c20 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/4111434524 >> 172.21.15.157:0/4111434524 conn(0x7f595c065930 msgr2=0x7f595c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:24.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/4111434524 shutdown_connections 2023-11-06T03:13:24.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/4111434524 wait complete. 2023-11-06T03:13:24.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 Processor -- start 2023-11-06T03:13:24.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- start start 2023-11-06T03:13:24.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f595c0adae0 0x7f595c162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595c156a70 0x7f595c1f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c1529e0 con 0x7f595c0adf40 2023-11-06T03:13:24.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c151af0 con 0x7f595c0adae0 2023-11-06T03:13:24.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c151dd0 con 0x7f595c156a70 2023-11-06T03:13:24.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50984/0 (socket says 172.21.15.157:50984) 2023-11-06T03:13:24.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 -- 172.21.15.157:0/1936097857 learned_addr learned my addr 172.21.15.157:0/1936097857 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:24.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595ad9d640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f595c0adae0 0x7f595c162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595b59e640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595c156a70 0x7f595c1f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 -- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595c156a70 msgr2=0x7f595c1f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595c156a70 0x7f595c1f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 -- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f595c0adae0 msgr2=0x7f595c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:24.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f595c0adae0 0x7f595c162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:24.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 -- 172.21.15.157:0/1936097857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595000a5c0 con 0x7f595c0adf40 2023-11-06T03:13:24.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f595a59c640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f59500b1040 tx=0x7f595001a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500ae850 con 0x7f595c0adf40 2023-11-06T03:13:24.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500ae9d0 con 0x7f595c0adf40 2023-11-06T03:13:24.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c1520b0 con 0x7f595c0adf40 2023-11-06T03:13:24.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500aed80 con 0x7f595c0adf40 2023-11-06T03:13:24.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.873+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c152670 con 0x7f595c0adf40 2023-11-06T03:13:24.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c152b50 con 0x7f595c0adf40 2023-11-06T03:13:24.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f59500b5040 con 0x7f595c0adf40 2023-11-06T03:13:24.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f59437fe640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 0x7f5948067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:24.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f595011da90 con 0x7f595c0adf40 2023-11-06T03:13:24.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f595ad9d640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 0x7f5948067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:24.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.877+0000 7f595ad9d640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 0x7f5948067f20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f594c000f70 tx=0x7f594c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:24.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:24.881+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f59500eb3f0 con 0x7f595c0adf40 2023-11-06T03:13:25.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.025+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f595c072750 con 0x7f595c0adf40 2023-11-06T03:13:25.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.025+0000 7f59437fe640 1 -- 172.21.15.157:0/1936097857 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59500eea10 con 0x7f595c0adf40 2023-11-06T03:13:25.024 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:13:25.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 msgr2=0x7f5948067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 0x7f5948067f20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f594c000f70 tx=0x7f594c009040 comp rx=0 tx=0).stop 2023-11-06T03:13:25.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 msgr2=0x7f595c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f59500b1040 tx=0x7f595001a5a0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 shutdown_connections 2023-11-06T03:13:25.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5948065a60 0x7f5948067f20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595c156a70 0x7f595c1f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f595c0adf40 0x7f595c155830 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 --2- 172.21.15.157:0/1936097857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f595c0adae0 0x7f595c162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 >> 172.21.15.157:0/1936097857 conn(0x7f595c065930 msgr2=0x7f595c15a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:25.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 shutdown_connections 2023-11-06T03:13:25.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.029+0000 7f5961199640 1 -- 172.21.15.157:0/1936097857 wait complete. 2023-11-06T03:13:25.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:13:25.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.169+0000 7f0c0f663640 1 Processor -- start 2023-11-06T03:13:25.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.169+0000 7f0c0f663640 1 -- start start 2023-11-06T03:13:25.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0f663640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c0815e5e0 0x7f0c080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0f663640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 0x7f0c080b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0f663640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0815f580 con 0x7f0c081551e0 2023-11-06T03:13:25.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0f663640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c080704b0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0f663640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0815efc0 con 0x7f0c0815ea90 2023-11-06T03:13:25.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c0815e5e0 0x7f0c080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0c0815e5e0 0x7f0c080b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50986/0 (socket says 172.21.15.157:50986) 2023-11-06T03:13:25.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/3418698728 learned_addr learned my addr 172.21.15.157:0/3418698728 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:25.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0dc00640 1 --2- 172.21.15.157:0/3418698728 >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 0x7f0c080b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/3418698728 >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 msgr2=0x7f0c080b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:25.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 --2- 172.21.15.157:0/3418698728 >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 0x7f0c080b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/3418698728 >> v1:172.21.15.195:6789/0 conn(0x7f0c081551e0 legacy=0x7f0c0806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:25.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/3418698728 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0815f2a0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0dc00640 1 --2- 172.21.15.157:0/3418698728 >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 0x7f0c080b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:25.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0c0cbfe640 1 --2- 172.21.15.157:0/3418698728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c080b3410 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f0c080cc2f0 tx=0x7f0bf801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e660130fc2d919 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0bfffff640 1 -- 172.21.15.157:0/3418698728 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80ab640 con 0x7f0c0815e5e0 2023-11-06T03:13:25.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0bfffff640 1 -- 172.21.15.157:0/3418698728 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80a7070 con 0x7f0c0815e5e0 2023-11-06T03:13:25.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.173+0000 7f0bfffff640 1 -- 172.21.15.157:0/3418698728 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80ab9c0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/3418698728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 msgr2=0x7f0c080b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- 172.21.15.157:0/3418698728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c080b3410 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f0c080cc2f0 tx=0x7f0bf801c440 comp rx=0 tx=0).stop 2023-11-06T03:13:25.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/3418698728 shutdown_connections 2023-11-06T03:13:25.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- 172.21.15.157:0/3418698728 >> v2:172.21.15.195:3300/0 conn(0x7f0c0815ea90 0x7f0c080b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- 172.21.15.157:0/3418698728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c080b3410 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/3418698728 >> 172.21.15.157:0/3418698728 conn(0x7f0c08065930 msgr2=0x7f0c08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:25.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/3418698728 shutdown_connections 2023-11-06T03:13:25.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/3418698728 wait complete. 2023-11-06T03:13:25.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 Processor -- start 2023-11-06T03:13:25.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- start start 2023-11-06T03:13:25.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c081551e0 0x7f0c0814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c0815ea90 0x7f0c0814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0806c0b0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0806c390 con 0x7f0c0815ea90 2023-11-06T03:13:25.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0806c670 con 0x7f0c081551e0 2023-11-06T03:13:25.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50996/0 (socket says 172.21.15.157:50996) 2023-11-06T03:13:25.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/1269966631 learned_addr learned my addr 172.21.15.157:0/1269966631 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:25.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0d3ff640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c081551e0 0x7f0c0814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0dc00640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c0815ea90 0x7f0c0814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c081551e0 msgr2=0x7f0c0814d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c081551e0 0x7f0c0814d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c0815ea90 msgr2=0x7f0c0814c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c0815ea90 0x7f0c0814c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 -- 172.21.15.157:0/1269966631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf80094d0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0cbfe640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8019050 tx=0x7f0bf801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80be3f0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80be550 con 0x7f0c0815e5e0 2023-11-06T03:13:25.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0806c950 con 0x7f0c0815e5e0 2023-11-06T03:13:25.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80be900 con 0x7f0c0815e5e0 2023-11-06T03:13:25.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.177+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0814cf10 con 0x7f0c0815e5e0 2023-11-06T03:13:25.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0815abf0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0bf80b43f0 con 0x7f0c0815e5e0 2023-11-06T03:13:25.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0bfdffb640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 0x7f0bec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0bf811d260 con 0x7f0c0815e5e0 2023-11-06T03:13:25.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0c0d3ff640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 0x7f0bec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.181+0000 7f0c0d3ff640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 0x7f0bec067f20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0c08152b50 tx=0x7f0c04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.185+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0bf80eaa00 con 0x7f0c0815e5e0 2023-11-06T03:13:25.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.325+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c08072750 con 0x7f0c0815e5e0 2023-11-06T03:13:25.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.325+0000 7f0bfdffb640 1 -- 172.21.15.157:0/1269966631 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bf80ee020 con 0x7f0c0815e5e0 2023-11-06T03:13:25.324 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:13:25.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 msgr2=0x7f0bec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 0x7f0bec067f20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0c08152b50 tx=0x7f0c04009040 comp rx=0 tx=0).stop 2023-11-06T03:13:25.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 msgr2=0x7f0c0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8019050 tx=0x7f0bf801ac80 comp rx=0 tx=0).stop 2023-11-06T03:13:25.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 shutdown_connections 2023-11-06T03:13:25.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0bec065a60 0x7f0bec067f20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c0815ea90 0x7f0c0814c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c0815e5e0 0x7f0c0806d150 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 --2- 172.21.15.157:0/1269966631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0c081551e0 0x7f0c0814d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 >> 172.21.15.157:0/1269966631 conn(0x7f0c08065930 msgr2=0x7f0c080cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:25.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 shutdown_connections 2023-11-06T03:13:25.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.329+0000 7f0c0f663640 1 -- 172.21.15.157:0/1269966631 wait complete. 2023-11-06T03:13:25.341 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:13:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 Processor -- start 2023-11-06T03:13:25.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 -- start start 2023-11-06T03:13:25.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c8146370 0x7f57c80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c806c3d0 con 0x7f57c80ce800 2023-11-06T03:13:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c806f5a0 con 0x7f57c806bdb0 2023-11-06T03:13:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cf31b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c806f880 con 0x7f57c8146370 2023-11-06T03:13:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cd0b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f57c8146370 0x7f57c80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cc8b6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f57c80ce800 0x7f57c806b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43454/0 (socket says 172.21.15.157:43454) 2023-11-06T03:13:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cc8b6640 1 -- 172.21.15.157:0/1481328666 learned_addr learned my addr 172.21.15.157:0/1481328666 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cd0b7640 1 -- 172.21.15.157:0/1481328666 >> v1:172.21.15.195:6789/0 conn(0x7f57c806bdb0 legacy=0x7f57c8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cd0b7640 1 -- 172.21.15.157:0/1481328666 >> v1:172.21.15.157:6789/0 conn(0x7f57c80ce800 legacy=0x7f57c806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57cd0b7640 1 -- 172.21.15.157:0/1481328666 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c806fb60 con 0x7f57c8146370 2023-11-06T03:13:25.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.457+0000 7f57bffff640 1 -- 172.21.15.157:0/1481328666 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687091613 0 0) 0x7f57c806c3d0 con 0x7f57c80ce800 2023-11-06T03:13:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cd0b7640 1 --2- 172.21.15.157:0/1481328666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c8146370 0x7f57c80b1a10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097f0 tx=0x7f57c401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5045e9ddfae38d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57bffff640 1 -- 172.21.15.157:0/1481328666 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40ab600 con 0x7f57c8146370 2023-11-06T03:13:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57bffff640 1 -- 172.21.15.157:0/1481328666 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40a7050 con 0x7f57c8146370 2023-11-06T03:13:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57bffff640 1 -- 172.21.15.157:0/1481328666 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40ab980 con 0x7f57c8146370 2023-11-06T03:13:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- 172.21.15.157:0/1481328666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c8146370 msgr2=0x7f57c80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 --2- 172.21.15.157:0/1481328666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c8146370 0x7f57c80b1a10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097f0 tx=0x7f57c401c380 comp rx=0 tx=0).stop 2023-11-06T03:13:25.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- 172.21.15.157:0/1481328666 shutdown_connections 2023-11-06T03:13:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 --2- 172.21.15.157:0/1481328666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c8146370 0x7f57c80b1a10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- 172.21.15.157:0/1481328666 >> 172.21.15.157:0/1481328666 conn(0x7f57c8065930 msgr2=0x7f57c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- 172.21.15.157:0/1481328666 shutdown_connections 2023-11-06T03:13:25.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- 172.21.15.157:0/1481328666 wait complete. 2023-11-06T03:13:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 Processor -- start 2023-11-06T03:13:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.461+0000 7f57cf31b640 1 -- start start 2023-11-06T03:13:25.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c80ce800 0x7f57c8138fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c8146370 0x7f57c815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8160090 con 0x7f57c806bdb0 2023-11-06T03:13:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c815f410 con 0x7f57c8146370 2023-11-06T03:13:25.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c815f6f0 con 0x7f57c80ce800 2023-11-06T03:13:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51002/0 (socket says 172.21.15.157:51002) 2023-11-06T03:13:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 -- 172.21.15.157:0/2944617243 learned_addr learned my addr 172.21.15.157:0/2944617243 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cc8b6640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c80ce800 0x7f57c8138fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd8b8640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c8146370 0x7f57c815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 -- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c80ce800 msgr2=0x7f57c8138fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c80ce800 0x7f57c8138fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 -- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c8146370 msgr2=0x7f57c815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c8146370 0x7f57c815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 -- 172.21.15.157:0/2944617243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009420 con 0x7f57c806bdb0 2023-11-06T03:13:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cd0b7640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f57c40ae040 tx=0x7f57c401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b7650 con 0x7f57c806bdb0 2023-11-06T03:13:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40b77b0 con 0x7f57c806bdb0 2023-11-06T03:13:25.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c815f9d0 con 0x7f57c806bdb0 2023-11-06T03:13:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40b7b60 con 0x7f57c806bdb0 2023-11-06T03:13:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81394f0 con 0x7f57c806bdb0 2023-11-06T03:13:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c8072750 con 0x7f57c806bdb0 2023-11-06T03:13:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57c40b7cc0 con 0x7f57c806bdb0 2023-11-06T03:13:25.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 0x7f57ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f57c411dc40 con 0x7f57c806bdb0 2023-11-06T03:13:25.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.465+0000 7f57cc8b6640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 0x7f57ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.469+0000 7f57cc8b6640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 0x7f57ac067f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57c8167980 tx=0x7f57b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:25.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.469+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f57c40eb480 con 0x7f57c806bdb0 2023-11-06T03:13:25.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.637+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f57c80afa10 con 0x7f57c806bdb0 2023-11-06T03:13:25.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.637+0000 7f57bdffb640 1 -- 172.21.15.157:0/2944617243 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f57c40bb080 con 0x7f57c806bdb0 2023-11-06T03:13:25.636 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:13:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 msgr2=0x7f57ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 0x7f57ac067f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57c8167980 tx=0x7f57b8009040 comp rx=0 tx=0).stop 2023-11-06T03:13:25.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 msgr2=0x7f57c8162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f57c40ae040 tx=0x7f57c401abc0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 shutdown_connections 2023-11-06T03:13:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f57ac065a60 0x7f57ac067f20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57c8146370 0x7f57c815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f57c80ce800 0x7f57c8138fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 --2- 172.21.15.157:0/2944617243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57c806bdb0 0x7f57c8162270 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 >> 172.21.15.157:0/2944617243 conn(0x7f57c8065930 msgr2=0x7f57c80b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 shutdown_connections 2023-11-06T03:13:25.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:25.641+0000 7f57cf31b640 1 -- 172.21.15.157:0/2944617243 wait complete. 2023-11-06T03:13:25.655 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:13:25.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:25.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:25.666 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[84184]: fuse finished with error 0 and tester_r 0 2023-11-06T03:13:25.741 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:13:25.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:25.742 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:13:25.907 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:13:25.908 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:25.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:13:25.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:13:25.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:25.916 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:13:25.964 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:13:25.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:13:25.965 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:13:25.966 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:25.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:13:25.974 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:13:25.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:25.975 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:13:26.022 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:13:26.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:13:26.024 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:13:26.024 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:26.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:13:26.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:13:26.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:26.039 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:13:26.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:13:26.088 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:13:26.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:26.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.205+0000 7f9556916640 1 Processor -- start 2023-11-06T03:13:26.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.205+0000 7f9556916640 1 -- start start 2023-11-06T03:13:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9556916640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95501551e0 0x7f955006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9556916640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 0x7f95500b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9556916640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955015f580 con 0x7f955015ea60 2023-11-06T03:13:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9556916640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955015efc0 con 0x7f95501551e0 2023-11-06T03:13:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9556916640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955015f2a0 con 0x7f955015e5e0 2023-11-06T03:13:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95501551e0 0x7f955006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9554eb3640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f955015ea60 0x7f955015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43460/0 (socket says 172.21.15.157:43460) 2023-11-06T03:13:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f9554eb3640 1 -- 172.21.15.157:0/1427820654 learned_addr learned my addr 172.21.15.157:0/1427820654 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954f7fe640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 0x7f95500b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 -- 172.21.15.157:0/1427820654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 msgr2=0x7f95500b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 0x7f95500b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 -- 172.21.15.157:0/1427820654 >> v1:172.21.15.157:6789/0 conn(0x7f955015ea60 legacy=0x7f955015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954effd640 1 -- 172.21.15.157:0/1427820654 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537364771 0 0) 0x7f955015f580 con 0x7f955015ea60 2023-11-06T03:13:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 -- 172.21.15.157:0/1427820654 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955006f5a0 con 0x7f95501551e0 2023-11-06T03:13:26.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954f7fe640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 0x7f95500b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:26.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954ffff640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95501551e0 0x7f955006fda0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f95400097c0 tx=0x7f954001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d1e2a537a94b05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954effd640 1 -- 172.21.15.157:0/1427820654 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95400ab600 con 0x7f95501551e0 2023-11-06T03:13:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954effd640 1 -- 172.21.15.157:0/1427820654 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95400a7050 con 0x7f95501551e0 2023-11-06T03:13:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.209+0000 7f954effd640 1 -- 172.21.15.157:0/1427820654 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95400ab980 con 0x7f95501551e0 2023-11-06T03:13:26.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1427820654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95501551e0 msgr2=0x7f955006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95501551e0 0x7f955006fda0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f95400097c0 tx=0x7f954001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1427820654 shutdown_connections 2023-11-06T03:13:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955015e5e0 0x7f95500b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- 172.21.15.157:0/1427820654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95501551e0 0x7f955006fda0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1427820654 >> 172.21.15.157:0/1427820654 conn(0x7f9550065930 msgr2=0x7f9550065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1427820654 shutdown_connections 2023-11-06T03:13:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1427820654 wait complete. 2023-11-06T03:13:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 Processor -- start 2023-11-06T03:13:26.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- start start 2023-11-06T03:13:26.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501551e0 0x7f955015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955015ea60 0x7f9550150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955015ab70 con 0x7f955015e5e0 2023-11-06T03:13:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9550158f50 con 0x7f955015ea60 2023-11-06T03:13:26.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9550159230 con 0x7f95501551e0 2023-11-06T03:13:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51020/0 (socket says 172.21.15.157:51020) 2023-11-06T03:13:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 -- 172.21.15.157:0/1174217727 learned_addr learned my addr 172.21.15.157:0/1174217727 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 -- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501551e0 msgr2=0x7f955015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501551e0 0x7f955015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 -- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955015ea60 msgr2=0x7f9550150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955015ea60 0x7f9550150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 -- 172.21.15.157:0/1174217727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009420 con 0x7f955015e5e0 2023-11-06T03:13:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954f7fe640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f954400c7c0 tx=0x7f954400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544007a30 con 0x7f955015e5e0 2023-11-06T03:13:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9544007b90 con 0x7f955015e5e0 2023-11-06T03:13:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550159510 con 0x7f955015e5e0 2023-11-06T03:13:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400ce70 con 0x7f955015e5e0 2023-11-06T03:13:26.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.213+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9550150a90 con 0x7f955015e5e0 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550159a30 con 0x7f955015e5e0 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f95440113f0 con 0x7f955015e5e0 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f954cff9640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 0x7f953c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f95440837a0 con 0x7f955015e5e0 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f954ffff640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 0x7f953c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.217+0000 7f954ffff640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 0x7f953c067f20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f95400ad000 tx=0x7f9540008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.221+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9550159a30 con 0x7f955015e5e0 2023-11-06T03:13:26.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.409+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9550159a30 con 0x7f955015e5e0 2023-11-06T03:13:26.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.409+0000 7f954cff9640 1 -- 172.21.15.157:0/1174217727 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f9550159a30 con 0x7f955015e5e0 2023-11-06T03:13:26.407 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:26.407 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:26.407 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:26.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 msgr2=0x7f953c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 0x7f953c067f20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f95400ad000 tx=0x7f9540008040 comp rx=0 tx=0).stop 2023-11-06T03:13:26.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 msgr2=0x7f9550159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f954400c7c0 tx=0x7f954400cc80 comp rx=0 tx=0).stop 2023-11-06T03:13:26.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 shutdown_connections 2023-11-06T03:13:26.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f953c065a60 0x7f953c067f20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955015ea60 0x7f9550150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955015e5e0 0x7f9550159d10 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 --2- 172.21.15.157:0/1174217727 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95501551e0 0x7f955015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 >> 172.21.15.157:0/1174217727 conn(0x7f9550065930 msgr2=0x7f95500ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:26.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 shutdown_connections 2023-11-06T03:13:26.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.413+0000 7f9556916640 1 -- 172.21.15.157:0/1174217727 wait complete. 2023-11-06T03:13:26.427 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 Processor -- start 2023-11-06T03:13:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 -- start start 2023-11-06T03:13:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 0x7f4a500b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a500b2d20 0x7f4a500c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a500ccdc0 con 0x7f4a500c3e80 2023-11-06T03:13:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500b0270 con 0x7f4a500b2d20 2023-11-06T03:13:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a562d3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500d0730 con 0x7f4a500afe90 2023-11-06T03:13:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a500b2d20 0x7f4a500c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4a500b2d20 0x7f4a500c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51036/0 (socket says 172.21.15.157:51036) 2023-11-06T03:13:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 -- 172.21.15.157:0/3084083971 learned_addr learned my addr 172.21.15.157:0/3084083971 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a54870640 1 --1- 172.21.15.157:0/3084083971 >> v1:172.21.15.157:6789/0 conn(0x7f4a500c3e80 0x7f4a5006b330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43470/0 (socket says 172.21.15.157:43470) 2023-11-06T03:13:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4ffff640 1 --2- 172.21.15.157:0/3084083971 >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 0x7f4a500b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 -- 172.21.15.157:0/3084083971 >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 msgr2=0x7f4a500b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 --2- 172.21.15.157:0/3084083971 >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 0x7f4a500b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4effd640 1 -- 172.21.15.157:0/3084083971 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081619171 0 0) 0x7f4a500ccdc0 con 0x7f4a500c3e80 2023-11-06T03:13:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 -- 172.21.15.157:0/3084083971 >> v1:172.21.15.157:6789/0 conn(0x7f4a500c3e80 legacy=0x7f4a5006b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4f7fe640 1 -- 172.21.15.157:0/3084083971 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a501617d0 con 0x7f4a500b2d20 2023-11-06T03:13:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.549+0000 7f4a4ffff640 1 --2- 172.21.15.157:0/3084083971 >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 0x7f4a500b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a4f7fe640 1 --2- 172.21.15.157:0/3084083971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a500b2d20 0x7f4a500c3940 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4a500d1a80 tx=0x7f4a4801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4587cd4a174f834f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a4effd640 1 -- 172.21.15.157:0/3084083971 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480a56f0 con 0x7f4a500b2d20 2023-11-06T03:13:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a4effd640 1 -- 172.21.15.157:0/3084083971 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a480a5850 con 0x7f4a500b2d20 2023-11-06T03:13:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a4effd640 1 -- 172.21.15.157:0/3084083971 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480a5be0 con 0x7f4a500b2d20 2023-11-06T03:13:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- 172.21.15.157:0/3084083971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a500b2d20 msgr2=0x7f4a500c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 --2- 172.21.15.157:0/3084083971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a500b2d20 0x7f4a500c3940 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4a500d1a80 tx=0x7f4a4801c940 comp rx=0 tx=0).stop 2023-11-06T03:13:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- 172.21.15.157:0/3084083971 shutdown_connections 2023-11-06T03:13:26.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 --2- 172.21.15.157:0/3084083971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a500b2d20 0x7f4a500c3940 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 --2- 172.21.15.157:0/3084083971 >> v2:172.21.15.195:3300/0 conn(0x7f4a500afe90 0x7f4a500b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- 172.21.15.157:0/3084083971 >> 172.21.15.157:0/3084083971 conn(0x7f4a5006d560 msgr2=0x7f4a5006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:26.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- 172.21.15.157:0/3084083971 shutdown_connections 2023-11-06T03:13:26.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- 172.21.15.157:0/3084083971 wait complete. 2023-11-06T03:13:26.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 Processor -- start 2023-11-06T03:13:26.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.553+0000 7f4a562d3640 1 -- start start 2023-11-06T03:13:26.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a500afe90 0x7f4a50146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a500c3e80 0x7f4a501f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501472b0 con 0x7f4a501572f0 2023-11-06T03:13:26.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501f9db0 con 0x7f4a500c3e80 2023-11-06T03:13:26.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501fa090 con 0x7f4a500afe90 2023-11-06T03:13:26.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51052/0 (socket says 172.21.15.157:51052) 2023-11-06T03:13:26.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 -- 172.21.15.157:0/681131454 learned_addr learned my addr 172.21.15.157:0/681131454 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a4f7fe640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a500c3e80 0x7f4a501f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a4ffff640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a500afe90 0x7f4a50146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 -- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a500afe90 msgr2=0x7f4a50146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a500afe90 0x7f4a50146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 -- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a500c3e80 msgr2=0x7f4a501f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a500c3e80 0x7f4a501f9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 -- 172.21.15.157:0/681131454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009b30 con 0x7f4a501572f0 2023-11-06T03:13:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a54870640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c470 tx=0x7f4a3800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380074e0 con 0x7f4a501572f0 2023-11-06T03:13:26.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5015bd60 con 0x7f4a501572f0 2023-11-06T03:13:26.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a501fa370 con 0x7f4a501572f0 2023-11-06T03:13:26.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.557+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a50072750 con 0x7f4a501572f0 2023-11-06T03:13:26.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38007640 con 0x7f4a501572f0 2023-11-06T03:13:26.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380049f0 con 0x7f4a501572f0 2023-11-06T03:13:26.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4a3801c3b0 con 0x7f4a501572f0 2023-11-06T03:13:26.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4cff9640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 0x7f4a30067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4a38082d40 con 0x7f4a501572f0 2023-11-06T03:13:26.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4ffff640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 0x7f4a30067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.561+0000 7f4a4ffff640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 0x7f4a30067f50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a40006fd0 tx=0x7f4a40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.565+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4a38050590 con 0x7f4a501572f0 2023-11-06T03:13:26.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.721+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a5006e3e0 con 0x7f4a501572f0 2023-11-06T03:13:26.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.721+0000 7f4a4cff9640 1 -- 172.21.15.157:0/681131454 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17923 (secure 0 0 0) 0x7f4a38053bb0 con 0x7f4a501572f0 2023-11-06T03:13:26.719 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:26.720 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":113,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:50.403519+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:26.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 msgr2=0x7f4a30067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 0x7f4a30067f50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a40006fd0 tx=0x7f4a40008040 comp rx=0 tx=0).stop 2023-11-06T03:13:26.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 msgr2=0x7f4a500cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c470 tx=0x7f4a3800c930 comp rx=0 tx=0).stop 2023-11-06T03:13:26.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 shutdown_connections 2023-11-06T03:13:26.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4a30065a90 0x7f4a30067f50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a501572f0 0x7f4a500cb600 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4a500c3e80 0x7f4a501f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 --2- 172.21.15.157:0/681131454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4a500afe90 0x7f4a50146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 >> 172.21.15.157:0/681131454 conn(0x7f4a5006d560 msgr2=0x7f4a50069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:26.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 shutdown_connections 2023-11-06T03:13:26.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.725+0000 7f4a562d3640 1 -- 172.21.15.157:0/681131454 wait complete. 2023-11-06T03:13:26.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:12:43.951782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:12:44.319880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:13:26.740 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:13:26.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 Processor -- start 2023-11-06T03:13:26.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 -- start start 2023-11-06T03:13:26.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f945015e5e0 0x7f94500b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f94500b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945015f580 con 0x7f94501551e0 2023-11-06T03:13:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500704b0 con 0x7f945015e5e0 2023-11-06T03:13:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f94551c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945015efc0 con 0x7f945015ea90 2023-11-06T03:13:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f945015e5e0 0x7f94500b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f945015e5e0 0x7f94500b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51062/0 (socket says 172.21.15.157:51062) 2023-11-06T03:13:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 -- 172.21.15.157:0/2219048203 learned_addr learned my addr 172.21.15.157:0/2219048203 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944f59e640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f94500b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 -- 172.21.15.157:0/2219048203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 msgr2=0x7f94500b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f94500b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 -- 172.21.15.157:0/2219048203 >> v1:172.21.15.195:6789/0 conn(0x7f94501551e0 legacy=0x7f945006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.873+0000 7f944e59c640 1 -- 172.21.15.157:0/2219048203 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945015f2a0 con 0x7f945015e5e0 2023-11-06T03:13:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f944e59c640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f94500b3410 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9444009870 tx=0x7f944401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a780daad5b287603 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f944dd9b640 1 -- 172.21.15.157:0/2219048203 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440ab680 con 0x7f945015e5e0 2023-11-06T03:13:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f944dd9b640 1 -- 172.21.15.157:0/2219048203 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94440ab7e0 con 0x7f945015e5e0 2023-11-06T03:13:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f944dd9b640 1 -- 172.21.15.157:0/2219048203 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440abb70 con 0x7f945015e5e0 2023-11-06T03:13:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 -- 172.21.15.157:0/2219048203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 msgr2=0x7f94500b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f94500b3410 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9444009870 tx=0x7f944401c480 comp rx=0 tx=0).stop 2023-11-06T03:13:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 -- 172.21.15.157:0/2219048203 shutdown_connections 2023-11-06T03:13:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f94500b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 --2- 172.21.15.157:0/2219048203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f94500b3410 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 -- 172.21.15.157:0/2219048203 >> 172.21.15.157:0/2219048203 conn(0x7f9450065930 msgr2=0x7f9450065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 -- 172.21.15.157:0/2219048203 shutdown_connections 2023-11-06T03:13:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.877+0000 7f94551c8640 1 -- 172.21.15.157:0/2219048203 wait complete. 2023-11-06T03:13:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 Processor -- start 2023-11-06T03:13:26.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- start start 2023-11-06T03:13:26.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501551e0 0x7f945006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f945014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945006d510 con 0x7f945015e5e0 2023-11-06T03:13:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500ce190 con 0x7f94501551e0 2023-11-06T03:13:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cdc10 con 0x7f945015ea90 2023-11-06T03:13:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51066/0 (socket says 172.21.15.157:51066) 2023-11-06T03:13:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 -- 172.21.15.157:0/1896043077 learned_addr learned my addr 172.21.15.157:0/1896043077 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944ed9d640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501551e0 0x7f945006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944f59e640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f945014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 -- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 msgr2=0x7f945014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f945014daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 -- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501551e0 msgr2=0x7f945006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501551e0 0x7f945006cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 -- 172.21.15.157:0/1896043077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440094d0 con 0x7f945015e5e0 2023-11-06T03:13:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f944e59c640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f94440ad040 tx=0x7f944401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440b4800 con 0x7f945015e5e0 2023-11-06T03:13:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94440b4960 con 0x7f945015e5e0 2023-11-06T03:13:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94500a8b60 con 0x7f945015e5e0 2023-11-06T03:13:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440b4d10 con 0x7f945015e5e0 2023-11-06T03:13:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94500cdeb0 con 0x7f945015e5e0 2023-11-06T03:13:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450072750 con 0x7f945015e5e0 2023-11-06T03:13:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f94440c7440 con 0x7f945015e5e0 2023-11-06T03:13:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 0x7f943c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.881+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f944411e220 con 0x7f945015e5e0 2023-11-06T03:13:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.885+0000 7f944ed9d640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 0x7f943c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.885+0000 7f944ed9d640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 0x7f943c067f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9440000f70 tx=0x7f9440009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:26.889+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f94440ebaf0 con 0x7f945015e5e0 2023-11-06T03:13:27.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.041+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94500674b0 con 0x7f945015e5e0 2023-11-06T03:13:27.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.041+0000 7f94377fe640 1 -- 172.21.15.157:0/1896043077 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94440ef110 con 0x7f945015e5e0 2023-11-06T03:13:27.038 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:13:27.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 msgr2=0x7f943c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 0x7f943c067f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9440000f70 tx=0x7f9440009040 comp rx=0 tx=0).stop 2023-11-06T03:13:27.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 msgr2=0x7f9450159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f94440ad040 tx=0x7f944401acc0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 shutdown_connections 2023-11-06T03:13:27.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f943c065a60 0x7f943c067f20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945015ea90 0x7f945014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f945015e5e0 0x7f9450159ba0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 --2- 172.21.15.157:0/1896043077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501551e0 0x7f945006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 >> 172.21.15.157:0/1896043077 conn(0x7f9450065930 msgr2=0x7f94500ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 shutdown_connections 2023-11-06T03:13:27.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.045+0000 7f94551c8640 1 -- 172.21.15.157:0/1896043077 wait complete. 2023-11-06T03:13:27.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:13:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.189+0000 7f98e190a640 1 Processor -- start 2023-11-06T03:13:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.189+0000 7f98e190a640 1 -- start start 2023-11-06T03:13:27.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98dc065d40 0x7f98dc0661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98dc15b280 0x7f98dc15b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0cef40 con 0x7f98dc144430 2023-11-06T03:13:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0cd450 con 0x7f98dc065d40 2023-11-06T03:13:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0cc840 con 0x7f98dc15b280 2023-11-06T03:13:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98dc065d40 0x7f98dc0661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f98dc065d40 0x7f98dc0661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51072/0 (socket says 172.21.15.157:51072) 2023-11-06T03:13:27.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 -- 172.21.15.157:0/4084667327 learned_addr learned my addr 172.21.15.157:0/4084667327 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:27.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98db7fe640 1 --2- 172.21.15.157:0/4084667327 >> v2:172.21.15.195:3300/0 conn(0x7f98dc15b280 0x7f98dc15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 -- 172.21.15.157:0/4084667327 >> v2:172.21.15.195:3300/0 conn(0x7f98dc15b280 msgr2=0x7f98dc15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 --2- 172.21.15.157:0/4084667327 >> v2:172.21.15.195:3300/0 conn(0x7f98dc15b280 0x7f98dc15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 -- 172.21.15.157:0/4084667327 >> v1:172.21.15.195:6789/0 conn(0x7f98dc144430 legacy=0x7f98dc0b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 -- 172.21.15.157:0/4084667327 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc066700 con 0x7f98dc065d40 2023-11-06T03:13:27.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98da7fc640 1 --2- 172.21.15.157:0/4084667327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc065d40 0x7f98dc0661c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0aff20 tx=0x7f98d001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dbcf4fa0a2a6e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98d9ffb640 1 -- 172.21.15.157:0/4084667327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00ac410 con 0x7f98dc065d40 2023-11-06T03:13:27.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98d9ffb640 1 -- 172.21.15.157:0/4084667327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d00ac570 con 0x7f98dc065d40 2023-11-06T03:13:27.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98d9ffb640 1 -- 172.21.15.157:0/4084667327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00ac900 con 0x7f98dc065d40 2023-11-06T03:13:27.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- 172.21.15.157:0/4084667327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc065d40 msgr2=0x7f98dc0661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 --2- 172.21.15.157:0/4084667327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc065d40 0x7f98dc0661c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0aff20 tx=0x7f98d001c4c0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- 172.21.15.157:0/4084667327 shutdown_connections 2023-11-06T03:13:27.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 --2- 172.21.15.157:0/4084667327 >> v2:172.21.15.195:3300/0 conn(0x7f98dc15b280 0x7f98dc15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 --2- 172.21.15.157:0/4084667327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc065d40 0x7f98dc0661c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- 172.21.15.157:0/4084667327 >> 172.21.15.157:0/4084667327 conn(0x7f98dc065930 msgr2=0x7f98dc071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- 172.21.15.157:0/4084667327 shutdown_connections 2023-11-06T03:13:27.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.193+0000 7f98e190a640 1 -- 172.21.15.157:0/4084667327 wait complete. 2023-11-06T03:13:27.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 Processor -- start 2023-11-06T03:13:27.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- start start 2023-11-06T03:13:27.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc065d40 0x7f98dc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc144430 0x7f98dc0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0c4c10 con 0x7f98dc15b280 2023-11-06T03:13:27.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc15a880 con 0x7f98dc065d40 2023-11-06T03:13:27.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc158970 con 0x7f98dc144430 2023-11-06T03:13:27.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51076/0 (socket says 172.21.15.157:51076) 2023-11-06T03:13:27.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 -- 172.21.15.157:0/3409506085 learned_addr learned my addr 172.21.15.157:0/3409506085 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:27.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 -- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc144430 msgr2=0x7f98dc0c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98da7fc640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc144430 0x7f98dc0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98daffd640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc065d40 0x7f98dc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc144430 0x7f98dc0c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 -- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc065d40 msgr2=0x7f98dc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc065d40 0x7f98dc158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 -- 172.21.15.157:0/3409506085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00094d0 con 0x7f98dc15b280 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98db7fe640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e60 tx=0x7f98d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014420 con 0x7f98dc15b280 2023-11-06T03:13:27.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d40145a0 con 0x7f98dc15b280 2023-11-06T03:13:27.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0c6b50 con 0x7f98dc15b280 2023-11-06T03:13:27.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014950 con 0x7f98dc15b280 2023-11-06T03:13:27.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc0c4550 con 0x7f98dc15b280 2023-11-06T03:13:27.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.197+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0c4d50 con 0x7f98dc15b280 2023-11-06T03:13:27.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.201+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98d4014ab0 con 0x7f98dc15b280 2023-11-06T03:13:27.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.201+0000 7f98c3fff640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 0x7f98c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.201+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f98d4083c10 con 0x7f98dc15b280 2023-11-06T03:13:27.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.201+0000 7f98daffd640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 0x7f98c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.201+0000 7f98daffd640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 0x7f98c8067f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f98cc003d60 tx=0x7f98cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.205+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f98dc0c4d50 con 0x7f98dc15b280 2023-11-06T03:13:27.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98dc0c4f30 con 0x7f98dc15b280 2023-11-06T03:13:27.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98c3fff640 1 -- 172.21.15.157:0/3409506085 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98dc0c4f30 con 0x7f98dc15b280 2023-11-06T03:13:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:13:27.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 msgr2=0x7f98c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 0x7f98c8067f20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f98cc003d60 tx=0x7f98cc009040 comp rx=0 tx=0).stop 2023-11-06T03:13:27.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 msgr2=0x7f98dc0c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007e60 tx=0x7f98d400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 shutdown_connections 2023-11-06T03:13:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f98c8065a60 0x7f98c8067f20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc15b280 0x7f98dc0c6030 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98dc144430 0x7f98dc0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 --2- 172.21.15.157:0/3409506085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98dc065d40 0x7f98dc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 >> 172.21.15.157:0/3409506085 conn(0x7f98dc065930 msgr2=0x7f98dc06f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.361+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 shutdown_connections 2023-11-06T03:13:27.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.365+0000 7f98e190a640 1 -- 172.21.15.157:0/3409506085 wait complete. 2023-11-06T03:13:27.377 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:13:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.501+0000 7f0ad1416640 1 Processor -- start 2023-11-06T03:13:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.501+0000 7f0ad1416640 1 -- start start 2023-11-06T03:13:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ad1416640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0acc1551e0 0x7f0acc06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ad1416640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 0x7f0acc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ad1416640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc15f580 con 0x7f0acc15ea60 2023-11-06T03:13:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ad1416640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0acc15efc0 con 0x7f0acc1551e0 2023-11-06T03:13:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ad1416640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15f2a0 con 0x7f0acc15e5e0 2023-11-06T03:13:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0acc15ea60 0x7f0acc15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:43474/0 (socket says 172.21.15.157:43474) 2023-11-06T03:13:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acb7fe640 1 -- 172.21.15.157:0/4163195636 learned_addr learned my addr 172.21.15.157:0/4163195636 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 --2- 172.21.15.157:0/4163195636 >> v2:172.21.15.195:3300/0 conn(0x7f0acc1551e0 0x7f0acc06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0ac27fc640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 0x7f0acc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052472244 0 0) 0x7f0acc15f580 con 0x7f0acc15ea60 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab8003530 con 0x7f0acc15ea60 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3731186859 0 0) 0x7f0ab8003530 con 0x7f0acc15ea60 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 msgr2=0x7f0acc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 0x7f0acc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 >> v2:172.21.15.195:3300/0 conn(0x7f0acc1551e0 msgr2=0x7f0acc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 --2- 172.21.15.157:0/4163195636 >> v2:172.21.15.195:3300/0 conn(0x7f0acc1551e0 0x7f0acc06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc06f5a0 con 0x7f0acc15ea60 2023-11-06T03:13:27.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0ac4001a00 con 0x7f0acc15ea60 2023-11-06T03:13:27.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 >> v1:172.21.15.157:6789/0 conn(0x7f0acc15ea60 legacy=0x7f0acc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0ab8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab80047d0 0x7f0ab8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 0x7f0ab8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15efc0 con 0x7f0ab8003980 2023-11-06T03:13:27.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15f2a0 con 0x7f0ab80047d0 2023-11-06T03:13:27.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab800b9c0 con 0x7f0acc15e5e0 2023-11-06T03:13:27.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0ab8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 -- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 msgr2=0x7f0ab8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc15e5e0 0x7f0ab8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 -- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab80047d0 msgr2=0x7f0ab8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ab80047d0 0x7f0ab8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.505+0000 7f0acaffd640 1 -- 172.21.15.157:0/4163195636 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8003530 con 0x7f0ab8003980 2023-11-06T03:13:27.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0ab8003da0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ae0 tx=0x7f0abc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d534b0e997dc5aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0a6470 con 0x7f0ab8003980 2023-11-06T03:13:27.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc0a65d0 con 0x7f0ab8003980 2023-11-06T03:13:27.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0aca7fc640 1 -- 172.21.15.157:0/4163195636 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0a6980 con 0x7f0ab8003980 2023-11-06T03:13:27.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/4163195636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 msgr2=0x7f0ab8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 --2- 172.21.15.157:0/4163195636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0ab8003da0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ae0 tx=0x7f0abc01ca00 comp rx=0 tx=0).stop 2023-11-06T03:13:27.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acb7fe640 1 -- 172.21.15.157:0/4163195636 reap_dead start 2023-11-06T03:13:27.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/4163195636 shutdown_connections 2023-11-06T03:13:27.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/4163195636 >> 172.21.15.157:0/4163195636 conn(0x7f0acc065930 msgr2=0x7f0acc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/4163195636 shutdown_connections 2023-11-06T03:13:27.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/4163195636 wait complete. 2023-11-06T03:13:27.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 Processor -- start 2023-11-06T03:13:27.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- start start 2023-11-06T03:13:27.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0acc1f9330 0x7f0acc1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc159f10 0x7f0acc163960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15a390 con 0x7f0ab8003980 2023-11-06T03:13:27.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc06bd30 con 0x7f0acc1f9330 2023-11-06T03:13:27.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc06c4d0 con 0x7f0acc159f10 2023-11-06T03:13:27.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51104/0 (socket says 172.21.15.157:51104) 2023-11-06T03:13:27.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 -- 172.21.15.157:0/588073443 learned_addr learned my addr 172.21.15.157:0/588073443 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:27.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acb7fe640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc159f10 0x7f0acc163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ac27fc640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0acc1f9330 0x7f0acc1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 -- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc159f10 msgr2=0x7f0acc163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc159f10 0x7f0acc163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 -- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0acc1f9330 msgr2=0x7f0acc1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0acc1f9330 0x7f0acc1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 -- 172.21.15.157:0/588073443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc06c7b0 con 0x7f0ab8003980 2023-11-06T03:13:27.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0acaffd640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ae0 tx=0x7f0ab8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0a6dc0 con 0x7f0ab8003980 2023-11-06T03:13:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc0bc450 con 0x7f0ab8003980 2023-11-06T03:13:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc06ca90 con 0x7f0ab8003980 2023-11-06T03:13:27.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0bc7e0 con 0x7f0ab8003980 2023-11-06T03:13:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.509+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc06d050 con 0x7f0ab8003980 2023-11-06T03:13:27.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc15a2f0 con 0x7f0ab8003980 2023-11-06T03:13:27.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0abc0a7450 con 0x7f0ab8003980 2023-11-06T03:13:27.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ac3fff640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 0x7f0aac067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0abc11d040 con 0x7f0ab8003980 2023-11-06T03:13:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ac27fc640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 0x7f0aac067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.513+0000 7f0ac27fc640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 0x7f0aac067f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0acc15aca0 tx=0x7f0ab4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.517+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0abc0ea930 con 0x7f0ab8003980 2023-11-06T03:13:27.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0acc072750 con 0x7f0ab8003980 2023-11-06T03:13:27.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ac3fff640 1 -- 172.21.15.157:0/588073443 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0abc0edf50 con 0x7f0ab8003980 2023-11-06T03:13:27.669 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:13:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 msgr2=0x7f0aac067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 0x7f0aac067f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0acc15aca0 tx=0x7f0ab4008040 comp rx=0 tx=0).stop 2023-11-06T03:13:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 msgr2=0x7f0acc06b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ae0 tx=0x7f0ab8005800 comp rx=0 tx=0).stop 2023-11-06T03:13:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 shutdown_connections 2023-11-06T03:13:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0aac065a90 0x7f0aac067f50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0acc159f10 0x7f0acc163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0acc1f9330 0x7f0acc1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 --2- 172.21.15.157:0/588073443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ab8003980 0x7f0acc06b7f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 >> 172.21.15.157:0/588073443 conn(0x7f0acc065930 msgr2=0x7f0acc15dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 shutdown_connections 2023-11-06T03:13:27.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.673+0000 7f0ad1416640 1 -- 172.21.15.157:0/588073443 wait complete. 2023-11-06T03:13:27.689 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:13:27.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 Processor -- start 2023-11-06T03:13:27.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 -- start start 2023-11-06T03:13:27.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb80b3b20 0x7f8fb815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb815f580 con 0x7f8fb815e5e0 2023-11-06T03:13:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb815efc0 con 0x7f8fb80b3b20 2023-11-06T03:13:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fc002f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb815f2a0 con 0x7f8fb81551e0 2023-11-06T03:13:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb80b3b20 0x7f8fb815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8fb80b3b20 0x7f8fb815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51112/0 (socket says 172.21.15.157:51112) 2023-11-06T03:13:27.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/252065976 learned_addr learned my addr 172.21.15.157:0/252065976 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbddcb640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/252065976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 msgr2=0x7f8fb806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/252065976 >> v1:172.21.15.195:6789/0 conn(0x7f8fb815e5e0 legacy=0x7f8fb80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/252065976 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb806f5a0 con 0x7f8fb80b3b20 2023-11-06T03:13:27.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.821+0000 7f8fbddcb640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbe5cc640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb80b3b20 0x7f8fb815c1b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009e50 tx=0x7f8fb001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef5ddb4c3871d53 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbcdc9640 1 -- 172.21.15.157:0/252065976 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00a5720 con 0x7f8fb80b3b20 2023-11-06T03:13:27.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbcdc9640 1 -- 172.21.15.157:0/252065976 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb00a5880 con 0x7f8fb80b3b20 2023-11-06T03:13:27.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbcdc9640 1 -- 172.21.15.157:0/252065976 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00a5c10 con 0x7f8fb80b3b20 2023-11-06T03:13:27.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- 172.21.15.157:0/252065976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb80b3b20 msgr2=0x7f8fb815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb80b3b20 0x7f8fb815c1b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009e50 tx=0x7f8fb001c950 comp rx=0 tx=0).stop 2023-11-06T03:13:27.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- 172.21.15.157:0/252065976 shutdown_connections 2023-11-06T03:13:27.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb80b3b20 0x7f8fb815c1b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- 172.21.15.157:0/252065976 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- 172.21.15.157:0/252065976 >> 172.21.15.157:0/252065976 conn(0x7f8fb8065930 msgr2=0x7f8fb8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- 172.21.15.157:0/252065976 shutdown_connections 2023-11-06T03:13:27.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- 172.21.15.157:0/252065976 wait complete. 2023-11-06T03:13:27.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 Processor -- start 2023-11-06T03:13:27.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- start start 2023-11-06T03:13:27.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb80b3b20 0x7f8fb814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb8155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8153360 con 0x7f8fb815e5e0 2023-11-06T03:13:27.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8156a50 con 0x7f8fb80b3b20 2023-11-06T03:13:27.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fc002f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb814c930 con 0x7f8fb81551e0 2023-11-06T03:13:27.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbe5cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbd5ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb8155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbe5cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51126/0 (socket says 172.21.15.157:51126) 2023-11-06T03:13:27.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/3465153158 learned_addr learned my addr 172.21.15.157:0/3465153158 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:27.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.825+0000 7f8fbddcb640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb80b3b20 0x7f8fb814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 msgr2=0x7f8fb8155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb8155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb80b3b20 msgr2=0x7f8fb814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb80b3b20 0x7f8fb814c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 -- 172.21.15.157:0/3465153158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0009b30 con 0x7f8fb815e5e0 2023-11-06T03:13:27.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbe5cc640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8fb001a580 tx=0x7f8fb00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00b8670 con 0x7f8fb815e5e0 2023-11-06T03:13:27.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb00b87d0 con 0x7f8fb815e5e0 2023-11-06T03:13:27.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb8156770 con 0x7f8fb815e5e0 2023-11-06T03:13:27.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00b8b80 con 0x7f8fb815e5e0 2023-11-06T03:13:27.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb8152ff0 con 0x7f8fb815e5e0 2023-11-06T03:13:27.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb8152b70 con 0x7f8fb815e5e0 2023-11-06T03:13:27.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8fb00b8ce0 con 0x7f8fb815e5e0 2023-11-06T03:13:27.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 0x7f8f9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:27.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8fb011e7c0 con 0x7f8fb815e5e0 2023-11-06T03:13:27.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbddcb640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 0x7f8f9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:27.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.829+0000 7f8fbddcb640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 0x7f8f9c067f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cf110 tx=0x7f8fa8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:27.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.833+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8fb8152b70 con 0x7f8fb815e5e0 2023-11-06T03:13:27.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.985+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fb8152b70 con 0x7f8fb815e5e0 2023-11-06T03:13:27.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.985+0000 7f8fae7fc640 1 -- 172.21.15.157:0/3465153158 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fb8152b70 con 0x7f8fb815e5e0 2023-11-06T03:13:27.983 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:13:27.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 msgr2=0x7f8f9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 0x7f8f9c067f20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80cf110 tx=0x7f8fa8008040 comp rx=0 tx=0).stop 2023-11-06T03:13:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 msgr2=0x7f8fb80ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8fb001a580 tx=0x7f8fb00196f0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 shutdown_connections 2023-11-06T03:13:27.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8f9c065a60 0x7f8f9c067f20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fb815e5e0 0x7f8fb80ae890 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fb81551e0 0x7f8fb8155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 --2- 172.21.15.157:0/3465153158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fb80b3b20 0x7f8fb814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 >> 172.21.15.157:0/3465153158 conn(0x7f8fb8065930 msgr2=0x7f8fb815dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:27.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 shutdown_connections 2023-11-06T03:13:27.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:27.989+0000 7f8fc002f640 1 -- 172.21.15.157:0/3465153158 wait complete. 2023-11-06T03:13:28.000 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 Processor -- start 2023-11-06T03:13:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 -- start start 2023-11-06T03:13:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febb00b1eb0 0x7febb00b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb015b3d0 con 0x7febb00b1ad0 2023-11-06T03:13:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb0155610 con 0x7febb00b1eb0 2023-11-06T03:13:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb6b39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c9d30 con 0x7febb015b860 2023-11-06T03:13:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febb00b1eb0 0x7febb00b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7febb00b1eb0 0x7febb00b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51134/0 (socket says 172.21.15.157:51134) 2023-11-06T03:13:28.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 -- 172.21.15.157:0/681546913 learned_addr learned my addr 172.21.15.157:0/681546913 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febb50d6640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 -- 172.21.15.157:0/681546913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 msgr2=0x7febb015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 -- 172.21.15.157:0/681546913 >> v1:172.21.15.195:6789/0 conn(0x7febb00b1ad0 legacy=0x7febb00c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.121+0000 7febaffff640 1 -- 172.21.15.157:0/681546913 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb015b0c0 con 0x7febb00b1eb0 2023-11-06T03:13:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febaffff640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1eb0 0x7febb00b4ac0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009870 tx=0x7feb9c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b401f50d08f94297 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febaf7fe640 1 -- 172.21.15.157:0/681546913 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0aa670 con 0x7febb00b1eb0 2023-11-06T03:13:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febaf7fe640 1 -- 172.21.15.157:0/681546913 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0aa7d0 con 0x7febb00b1eb0 2023-11-06T03:13:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febaf7fe640 1 -- 172.21.15.157:0/681546913 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0aab60 con 0x7febb00b1eb0 2023-11-06T03:13:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/681546913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1eb0 msgr2=0x7febb00b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1eb0 0x7febb00b4ac0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009870 tx=0x7feb9c01c510 comp rx=0 tx=0).stop 2023-11-06T03:13:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/681546913 shutdown_connections 2023-11-06T03:13:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- 172.21.15.157:0/681546913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1eb0 0x7febb00b4ac0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/681546913 >> 172.21.15.157:0/681546913 conn(0x7febb006a170 msgr2=0x7febb006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/681546913 shutdown_connections 2023-11-06T03:13:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/681546913 wait complete. 2023-11-06T03:13:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 Processor -- start 2023-11-06T03:13:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- start start 2023-11-06T03:13:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb00b1eb0 0x7febb01515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb00ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb0151b30 con 0x7febb00b1ad0 2023-11-06T03:13:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00caa60 con 0x7febb00b1eb0 2023-11-06T03:13:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00cad40 con 0x7febb015b860 2023-11-06T03:13:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51142/0 (socket says 172.21.15.157:51142) 2023-11-06T03:13:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 -- 172.21.15.157:0/2019209233 learned_addr learned my addr 172.21.15.157:0/2019209233 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febaffff640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb00b1eb0 0x7febb01515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb50d6640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb00ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 -- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 msgr2=0x7febb00ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb00ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 -- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb00b1eb0 msgr2=0x7febb01515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb00b1eb0 0x7febb01515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 -- 172.21.15.157:0/2019209233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0094d0 con 0x7febb00b1ad0 2023-11-06T03:13:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb48d5640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7feba400c500 tx=0x7feba400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4005220 con 0x7febb00b1ad0 2023-11-06T03:13:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba4005380 con 0x7febb00b1ad0 2023-11-06T03:13:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb00cb020 con 0x7febb00b1ad0 2023-11-06T03:13:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4007270 con 0x7febb00b1ad0 2023-11-06T03:13:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.125+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00cb5e0 con 0x7febb00b1ad0 2023-11-06T03:13:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb00cb7d0 con 0x7febb00b1ad0 2023-11-06T03:13:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feba40073d0 con 0x7febb00b1ad0 2023-11-06T03:13:28.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febad7fa640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 0x7feb94067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7feba4082c30 con 0x7febb00b1ad0 2023-11-06T03:13:28.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febaffff640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 0x7feb94067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.129+0000 7febaffff640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 0x7feb94067f20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0ae000 tx=0x7feb9c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.133+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feba4050480 con 0x7febb00b1ad0 2023-11-06T03:13:28.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.321+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb00b5650 con 0x7febb00b1ad0 2023-11-06T03:13:28.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.321+0000 7febad7fa640 1 -- 172.21.15.157:0/2019209233 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7feba4053aa0 con 0x7febb00b1ad0 2023-11-06T03:13:28.319 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:28.319 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:28.319 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:28.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 msgr2=0x7feb94067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 0x7feb94067f20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0ae000 tx=0x7feb9c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:13:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 msgr2=0x7febb00cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7feba400c500 tx=0x7feba400c9c0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 shutdown_connections 2023-11-06T03:13:28.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb94065a60 0x7feb94067f20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7febb015b860 0x7febb00ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7febb00b1eb0 0x7febb01515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 --2- 172.21.15.157:0/2019209233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febb00b1ad0 0x7febb00cbb00 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 >> 172.21.15.157:0/2019209233 conn(0x7febb006a170 msgr2=0x7febb00ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 shutdown_connections 2023-11-06T03:13:28.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.325+0000 7febb6b39640 1 -- 172.21.15.157:0/2019209233 wait complete. 2023-11-06T03:13:28.338 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.461+0000 7eff754ba640 1 Processor -- start 2023-11-06T03:13:28.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.461+0000 7eff754ba640 1 -- start start 2023-11-06T03:13:28.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff754ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff701599c0 0x7eff700ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff754ba640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7eff7006d000 0x7eff700b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff754ba640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7015e440 con 0x7eff7015dea0 2023-11-06T03:13:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff754ba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7015adb0 con 0x7eff701599c0 2023-11-06T03:13:28.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff754ba640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7006c240 con 0x7eff7006d000 2023-11-06T03:13:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff701599c0 0x7eff700ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7eff701599c0 0x7eff700ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51148/0 (socket says 172.21.15.157:51148) 2023-11-06T03:13:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 -- 172.21.15.157:0/2599945078 learned_addr learned my addr 172.21.15.157:0/2599945078 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff667fc640 1 --2- 172.21.15.157:0/2599945078 >> v2:172.21.15.195:3300/0 conn(0x7eff7006d000 0x7eff700b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 -- 172.21.15.157:0/2599945078 >> v2:172.21.15.195:3300/0 conn(0x7eff7006d000 msgr2=0x7eff700b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 --2- 172.21.15.157:0/2599945078 >> v2:172.21.15.195:3300/0 conn(0x7eff7006d000 0x7eff700b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 -- 172.21.15.157:0/2599945078 >> v1:172.21.15.195:6789/0 conn(0x7eff7015dea0 legacy=0x7eff70070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 -- 172.21.15.157:0/2599945078 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7006c520 con 0x7eff701599c0 2023-11-06T03:13:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.465+0000 7eff6effd640 1 --2- 172.21.15.157:0/2599945078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff701599c0 0x7eff700ae000 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7eff700cfa20 tx=0x7eff6001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f742f165bae4c940 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6e7fc640 1 -- 172.21.15.157:0/2599945078 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600ac640 con 0x7eff701599c0 2023-11-06T03:13:28.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6e7fc640 1 -- 172.21.15.157:0/2599945078 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600ac7a0 con 0x7eff701599c0 2023-11-06T03:13:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6e7fc640 1 -- 172.21.15.157:0/2599945078 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600acb30 con 0x7eff701599c0 2023-11-06T03:13:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/2599945078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff701599c0 msgr2=0x7eff700ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- 172.21.15.157:0/2599945078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff701599c0 0x7eff700ae000 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7eff700cfa20 tx=0x7eff6001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/2599945078 shutdown_connections 2023-11-06T03:13:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- 172.21.15.157:0/2599945078 >> v2:172.21.15.195:3300/0 conn(0x7eff7006d000 0x7eff700b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- 172.21.15.157:0/2599945078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff701599c0 0x7eff700ae000 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/2599945078 >> 172.21.15.157:0/2599945078 conn(0x7eff70065930 msgr2=0x7eff70065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/2599945078 shutdown_connections 2023-11-06T03:13:28.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/2599945078 wait complete. 2023-11-06T03:13:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 Processor -- start 2023-11-06T03:13:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- start start 2023-11-06T03:13:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff701599c0 0x7eff700cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7015dea0 0x7eff700cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70145cf0 con 0x7eff7006d000 2023-11-06T03:13:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70150ad0 con 0x7eff7015dea0 2023-11-06T03:13:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7014d6a0 con 0x7eff701599c0 2023-11-06T03:13:28.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51154/0 (socket says 172.21.15.157:51154) 2023-11-06T03:13:28.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 -- 172.21.15.157:0/85732390 learned_addr learned my addr 172.21.15.157:0/85732390 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:28.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6f7fe640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7015dea0 0x7eff700cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff667fc640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff701599c0 0x7eff700cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 -- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff701599c0 msgr2=0x7eff700cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff701599c0 0x7eff700cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 -- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7015dea0 msgr2=0x7eff700cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7015dea0 0x7eff700cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 -- 172.21.15.157:0/85732390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600094d0 con 0x7eff7006d000 2023-11-06T03:13:28.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff6effd640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7eff600af040 tx=0x7eff6001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600b7830 con 0x7eff7006d000 2023-11-06T03:13:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600b79b0 con 0x7eff7006d000 2023-11-06T03:13:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7014d980 con 0x7eff7006d000 2023-11-06T03:13:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.469+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600b7d60 con 0x7eff7006d000 2023-11-06T03:13:28.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7014d390 con 0x7eff7006d000 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff70072750 con 0x7eff7006d000 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7eff600b3040 con 0x7eff7006d000 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff67fff640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 0x7eff50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7eff6011da90 con 0x7eff7006d000 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff667fc640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 0x7eff50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.473+0000 7eff667fc640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 0x7eff50067f20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7eff7014e880 tx=0x7eff58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.477+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7eff600eb2d0 con 0x7eff7006d000 2023-11-06T03:13:28.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.625+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff700adcd0 con 0x7eff7006d000 2023-11-06T03:13:28.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.629+0000 7eff67fff640 1 -- 172.21.15.157:0/85732390 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17923 (secure 0 0 0) 0x7eff600ac600 con 0x7eff7006d000 2023-11-06T03:13:28.627 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:28.627 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":113,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:50.403519+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:28.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 msgr2=0x7eff50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 0x7eff50067f20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7eff7014e880 tx=0x7eff58008040 comp rx=0 tx=0).stop 2023-11-06T03:13:28.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 msgr2=0x7eff700cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7eff600af040 tx=0x7eff6001ad10 comp rx=0 tx=0).stop 2023-11-06T03:13:28.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 shutdown_connections 2023-11-06T03:13:28.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff50065a60 0x7eff50067f20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff7015dea0 0x7eff700cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff701599c0 0x7eff700cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 --2- 172.21.15.157:0/85732390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff7006d000 0x7eff700cbfa0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 >> 172.21.15.157:0/85732390 conn(0x7eff70065930 msgr2=0x7eff700ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 shutdown_connections 2023-11-06T03:13:28.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.633+0000 7eff754ba640 1 -- 172.21.15.157:0/85732390 wait complete. 2023-11-06T03:13:28.645 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:13:28.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:28.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.761+0000 7ff83be93640 1 Processor -- start 2023-11-06T03:13:28.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.761+0000 7ff83be93640 1 -- start start 2023-11-06T03:13:28.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83be93640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff83415ac80 0x7ff8340b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83be93640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8340cc2b0 con 0x7ff8340cf560 2023-11-06T03:13:28.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83be93640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8340b01a0 con 0x7ff8340afb00 2023-11-06T03:13:28.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83be93640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8340cfb10 con 0x7ff83415ac80 2023-11-06T03:13:28.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83a430640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff8340cf560 0x7ff8340cce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38652/0 (socket says 172.21.15.157:38652) 2023-11-06T03:13:28.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83a430640 1 -- 172.21.15.157:0/3220062081 learned_addr learned my addr 172.21.15.157:0/3220062081 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:28.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff839c2f640 1 --2- 172.21.15.157:0/3220062081 >> v2:172.21.15.195:3300/0 conn(0x7ff83415ac80 0x7ff8340b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054061371 0 0) 0x7ff8340cc2b0 con 0x7ff8340cf560 2023-11-06T03:13:28.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff820003530 con 0x7ff8340cf560 2023-11-06T03:13:28.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606052778 0 0) 0x7ff820003530 con 0x7ff8340cf560 2023-11-06T03:13:28.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 >> v2:172.21.15.195:3300/0 conn(0x7ff83415ac80 msgr2=0x7ff8340b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 --2- 172.21.15.157:0/3220062081 >> v2:172.21.15.195:3300/0 conn(0x7ff83415ac80 0x7ff8340b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 >> v1:172.21.15.195:6789/0 conn(0x7ff8340afb00 legacy=0x7ff8340cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8340cfdf0 con 0x7ff8340cf560 2023-11-06T03:13:28.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff82c001a00 con 0x7ff8340cf560 2023-11-06T03:13:28.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 >> v1:172.21.15.157:6789/0 conn(0x7ff8340cf560 legacy=0x7ff8340cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff820003980 0x7ff820003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 0x7ff820004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff820005190 0x7ff82000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340cfb10 con 0x7ff8200047d0 2023-11-06T03:13:28.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82000c130 con 0x7ff820005190 2023-11-06T03:13:28.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82000c2f0 con 0x7ff820003980 2023-11-06T03:13:28.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 0x7ff820004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff839c2f640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff820003980 0x7ff820003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 -- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff820003980 msgr2=0x7ff820003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff820003980 0x7ff820003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 -- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff820005190 msgr2=0x7ff82000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:28.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff820005190 0x7ff82000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83a430640 1 -- 172.21.15.157:0/3220062081 reap_dead start 2023-11-06T03:13:28.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.765+0000 7ff83942e640 1 -- 172.21.15.157:0/3220062081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff820003530 con 0x7ff8200047d0 2023-11-06T03:13:28.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83942e640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 0x7ff820004c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff824001780 tx=0x7ff82401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2718956ee19368d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240a4ea0 con 0x7ff8200047d0 2023-11-06T03:13:28.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8240a7050 con 0x7ff8200047d0 2023-11-06T03:13:28.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff838c2d640 1 -- 172.21.15.157:0/3220062081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824003600 con 0x7ff8200047d0 2023-11-06T03:13:28.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 msgr2=0x7ff820004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 0x7ff820004c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff824001780 tx=0x7ff82401c3e0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/3220062081 shutdown_connections 2023-11-06T03:13:28.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 --2- 172.21.15.157:0/3220062081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8200047d0 0x7ff820004c50 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/3220062081 >> 172.21.15.157:0/3220062081 conn(0x7ff83406d560 msgr2=0x7ff83406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/3220062081 shutdown_connections 2023-11-06T03:13:28.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/3220062081 wait complete. 2023-11-06T03:13:28.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 Processor -- start 2023-11-06T03:13:28.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- start start 2023-11-06T03:13:28.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8200047d0 0x7ff8340b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8340c58a0 0x7ff8340c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c85f0 con 0x7ff83415c2f0 2023-11-06T03:13:28.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c6260 con 0x7ff8340c58a0 2023-11-06T03:13:28.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83415dfa0 con 0x7ff8200047d0 2023-11-06T03:13:28.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37734/0 (socket says 172.21.15.157:37734) 2023-11-06T03:13:28.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 -- 172.21.15.157:0/4107657172 learned_addr learned my addr 172.21.15.157:0/4107657172 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:28.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83942e640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8340c58a0 0x7ff8340c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff839c2f640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8200047d0 0x7ff8340b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 -- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8200047d0 msgr2=0x7ff8340b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8200047d0 0x7ff8340b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 -- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8340c58a0 msgr2=0x7ff8340c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8340c58a0 0x7ff8340c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 -- 172.21.15.157:0/4107657172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83415cf00 con 0x7ff83415c2f0 2023-11-06T03:13:28.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83a430640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ff82c001570 tx=0x7ff82c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00a610 con 0x7ff83415c2f0 2023-11-06T03:13:28.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00a770 con 0x7ff83415c2f0 2023-11-06T03:13:28.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83415d1e0 con 0x7ff83415c2f0 2023-11-06T03:13:28.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00ab20 con 0x7ff83415c2f0 2023-11-06T03:13:28.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.769+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83415d7a0 con 0x7ff83415c2f0 2023-11-06T03:13:28.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83406e3e0 con 0x7ff83415c2f0 2023-11-06T03:13:28.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff82c00ac80 con 0x7ff83415c2f0 2023-11-06T03:13:28.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff82a7fc640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 0x7ff818067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:28.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff82c082cc0 con 0x7ff83415c2f0 2023-11-06T03:13:28.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff839c2f640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 0x7ff818067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:28.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.773+0000 7ff839c2f640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 0x7ff818067f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff834063940 tx=0x7ff830008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:28.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.777+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff82c089080 con 0x7ff83415c2f0 2023-11-06T03:13:28.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.961+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff834072730 con 0x7ff83415c2f0 2023-11-06T03:13:28.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.961+0000 7ff82a7fc640 1 -- 172.21.15.157:0/4107657172 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7ff82c0504f0 con 0x7ff83415c2f0 2023-11-06T03:13:28.959 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:28.959 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:28.960 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:28.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 msgr2=0x7ff818067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 0x7ff818067f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff834063940 tx=0x7ff830008040 comp rx=0 tx=0).stop 2023-11-06T03:13:28.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 msgr2=0x7ff83415c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:28.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ff82c001570 tx=0x7ff82c00ccd0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 shutdown_connections 2023-11-06T03:13:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff818065a90 0x7ff818067f50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff83415c2f0 0x7ff83415c7a0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8340c58a0 0x7ff8340c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 --2- 172.21.15.157:0/4107657172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8200047d0 0x7ff8340b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 >> 172.21.15.157:0/4107657172 conn(0x7ff83406d560 msgr2=0x7ff8340d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 shutdown_connections 2023-11-06T03:13:28.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:28.965+0000 7ff83be93640 1 -- 172.21.15.157:0/4107657172 wait complete. 2023-11-06T03:13:28.977 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:29.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.097+0000 7fdb915a3640 1 Processor -- start 2023-11-06T03:13:29.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.097+0000 7fdb915a3640 1 -- start start 2023-11-06T03:13:29.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c15d030 con 0x7fdb8c15cc50 2023-11-06T03:13:29.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0d1cd0 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0d10c0 con 0x7fdb8c0d31f0 2023-11-06T03:13:29.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37748/0 (socket says 172.21.15.157:37748) 2023-11-06T03:13:29.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 -- 172.21.15.157:0/666884814 learned_addr learned my addr 172.21.15.157:0/666884814 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8a7fc640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 -- 172.21.15.157:0/666884814 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 msgr2=0x7fdb8c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:29.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 -- 172.21.15.157:0/666884814 >> v1:172.21.15.195:6789/0 conn(0x7fdb8c15cc50 legacy=0x7fdb8c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:29.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 -- 172.21.15.157:0/666884814 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0d13a0 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb8b7fe640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009e50 tx=0x7fdb8401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aab913820e85141 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb89ffb640 1 -- 172.21.15.157:0/666884814 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840a5820 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb89ffb640 1 -- 172.21.15.157:0/666884814 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb840a5980 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb89ffb640 1 -- 172.21.15.157:0/666884814 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840a5d10 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- 172.21.15.157:0/666884814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 msgr2=0x7fdb8c0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009e50 tx=0x7fdb8401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- 172.21.15.157:0/666884814 shutdown_connections 2023-11-06T03:13:29.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c0b5fa0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 --2- 172.21.15.157:0/666884814 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- 172.21.15.157:0/666884814 >> 172.21.15.157:0/666884814 conn(0x7fdb8c070790 msgr2=0x7fdb8c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- 172.21.15.157:0/666884814 shutdown_connections 2023-11-06T03:13:29.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.101+0000 7fdb915a3640 1 -- 172.21.15.157:0/666884814 wait complete. 2023-11-06T03:13:29.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 Processor -- start 2023-11-06T03:13:29.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- start start 2023-11-06T03:13:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb8c15cc50 0x7fdb8c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c153ce0 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0ca590 con 0x7fdb8c15cc50 2023-11-06T03:13:29.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c15a420 con 0x7fdb8c0d31f0 2023-11-06T03:13:29.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37750/0 (socket says 172.21.15.157:37750) 2023-11-06T03:13:29.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 -- 172.21.15.157:0/415305272 learned_addr learned my addr 172.21.15.157:0/415305272 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8a7fc640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 -- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 msgr2=0x7fdb8c15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 -- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb8c15cc50 msgr2=0x7fdb8c0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb8c15cc50 0x7fdb8c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 -- 172.21.15.157:0/415305272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84009b30 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb8affd640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00b410 tx=0x7fdb7c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004020 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c004180 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c153690 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c011630 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c150200 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c072750 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.105+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdb7c011790 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.109+0000 7fdb737fe640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 0x7fdb78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.109+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fdb7c083020 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.109+0000 7fdb8a7fc640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 0x7fdb78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.109+0000 7fdb8a7fc640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 0x7fdb78067f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0b5640 tx=0x7fdb80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.113+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdb7c0508f0 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.297+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb8c15ffc0 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.297+0000 7fdb737fe640 1 -- 172.21.15.157:0/415305272 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fdb7c015090 con 0x7fdb8c0d0b70 2023-11-06T03:13:29.296 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:29.296 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:12:45.374552+0000","modified":"2023-11-06T03:12:47.390020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2645931923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":2645931923},{"type":"v1","addr":"172.21.15.157:6835","nonce":2645931923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-11-06T03:13:29.297 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 85 2023-11-06T03:13:29.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 msgr2=0x7fdb78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 0x7fdb78067f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0b5640 tx=0x7fdb80008040 comp rx=0 tx=0).stop 2023-11-06T03:13:29.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 msgr2=0x7fdb8c14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00b410 tx=0x7fdb7c00b8d0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 shutdown_connections 2023-11-06T03:13:29.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb78065a60 0x7fdb78067f20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb8c15cc50 0x7fdb8c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb8c0d31f0 0x7fdb8c15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 --2- 172.21.15.157:0/415305272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb8c0d0b70 0x7fdb8c14fb00 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 >> 172.21.15.157:0/415305272 conn(0x7fdb8c070790 msgr2=0x7fdb8c159440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 shutdown_connections 2023-11-06T03:13:29.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.301+0000 7fdb915a3640 1 -- 172.21.15.157:0/415305272 wait complete. 2023-11-06T03:13:29.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:29.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.429+0000 7f4d32d59640 1 Processor -- start 2023-11-06T03:13:29.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.429+0000 7f4d32d59640 1 -- start start 2023-11-06T03:13:29.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d32d59640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d32d59640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d32d59640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2c0d2920 con 0x7f4d2c15caa0 2023-11-06T03:13:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d32d59640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c0d0520 con 0x7f4d2c15ce80 2023-11-06T03:13:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d32d59640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0d18a0 con 0x7f4d2c0d00b0 2023-11-06T03:13:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37760/0 (socket says 172.21.15.157:37760) 2023-11-06T03:13:29.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 -- 172.21.15.157:0/137423582 learned_addr learned my addr 172.21.15.157:0/137423582 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d312f6640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 -- 172.21.15.157:0/137423582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 msgr2=0x7f4d2c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:29.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 -- 172.21.15.157:0/137423582 >> v1:172.21.15.195:6789/0 conn(0x7f4d2c15caa0 legacy=0x7f4d2c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 -- 172.21.15.157:0/137423582 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0d1b60 con 0x7f4d2c15ce80 2023-11-06T03:13:29.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2bfff640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009870 tx=0x7f4d1801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d218fa3f8368c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2b7fe640 1 -- 172.21.15.157:0/137423582 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180aa670 con 0x7f4d2c15ce80 2023-11-06T03:13:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2b7fe640 1 -- 172.21.15.157:0/137423582 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d180aa7d0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.433+0000 7f4d2b7fe640 1 -- 172.21.15.157:0/137423582 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180aab60 con 0x7f4d2c15ce80 2023-11-06T03:13:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/137423582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 msgr2=0x7f4d2c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009870 tx=0x7f4d1801c510 comp rx=0 tx=0).stop 2023-11-06T03:13:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/137423582 shutdown_connections 2023-11-06T03:13:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- 172.21.15.157:0/137423582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c0cfb70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/137423582 >> 172.21.15.157:0/137423582 conn(0x7f4d2c070790 msgr2=0x7f4d2c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/137423582 shutdown_connections 2023-11-06T03:13:29.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/137423582 wait complete. 2023-11-06T03:13:29.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 Processor -- start 2023-11-06T03:13:29.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- start start 2023-11-06T03:13:29.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c1565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 0x7f4d2c0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0ca200 con 0x7f4d2c15ce80 2023-11-06T03:13:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c160080 con 0x7f4d2c15caa0 2023-11-06T03:13:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0cbad0 con 0x7f4d2c0d00b0 2023-11-06T03:13:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37764/0 (socket says 172.21.15.157:37764) 2023-11-06T03:13:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 -- 172.21.15.157:0/855240413 learned_addr learned my addr 172.21.15.157:0/855240413 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d2bfff640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 0x7f4d2c0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 -- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 msgr2=0x7f4d2c1565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c1565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 -- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 msgr2=0x7f4d2c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 0x7f4d2c0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 -- 172.21.15.157:0/855240413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180094d0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d2bfff640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 0x7f4d2c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d312f6640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007e60 tx=0x7f4d2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24014420 con 0x7f4d2c15ce80 2023-11-06T03:13:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d240145a0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24014950 con 0x7f4d2c15ce80 2023-11-06T03:13:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0cbdb0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c154d30 con 0x7f4d2c15ce80 2023-11-06T03:13:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.437+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c0c9cf0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.441+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4d24014ab0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.441+0000 7f4d297fa640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 0x7f4d10067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.441+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4d24083c00 con 0x7f4d2c15ce80 2023-11-06T03:13:29.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.441+0000 7f4d30af5640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 0x7f4d10067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.441+0000 7f4d30af5640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 0x7f4d10067f50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d20007fd0 tx=0x7f4d20007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.445+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4d2c0c9cf0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.589+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d2c0c9cf0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d297fa640 1 -- 172.21.15.157:0/855240413 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17923 (secure 0 0 0) 0x7f4d2c0c9cf0 con 0x7f4d2c15ce80 2023-11-06T03:13:29.590 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:29.590 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":113,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:12:50.403519+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:12:43.951782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-06T03:12:44.319880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 msgr2=0x7f4d10067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 0x7f4d10067f50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d20007fd0 tx=0x7f4d20007960 comp rx=0 tx=0).stop 2023-11-06T03:13:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 msgr2=0x7f4d2c1547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4d24007e60 tx=0x7f4d2400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 shutdown_connections 2023-11-06T03:13:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4d10065a90 0x7f4d10067f50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d2c15ce80 0x7f4d2c1547f0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d2c15caa0 0x7f4d2c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 --2- 172.21.15.157:0/855240413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4d2c0d00b0 0x7f4d2c1565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 >> 172.21.15.157:0/855240413 conn(0x7f4d2c070790 msgr2=0x7f4d2c0b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 shutdown_connections 2023-11-06T03:13:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.593+0000 7f4d32d59640 1 -- 172.21.15.157:0/855240413 wait complete. 2023-11-06T03:13:29.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:13:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 Processor -- start 2023-11-06T03:13:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 -- start start 2023-11-06T03:13:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156a70 0x7fee8006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 0x7fee800b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee800adf40 0x7fee800b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8006b950 con 0x7fee80156a70 2023-11-06T03:13:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8006c490 con 0x7fee800adae0 2023-11-06T03:13:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee87ed5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8006bb60 con 0x7fee800adf40 2023-11-06T03:13:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156a70 0x7fee8006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156a70 0x7fee8006f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37774/0 (socket says 172.21.15.157:37774) 2023-11-06T03:13:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 -- 172.21.15.157:0/2219671602 learned_addr learned my addr 172.21.15.157:0/2219671602 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee86472640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee800adf40 0x7fee800b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85470640 1 --2- 172.21.15.157:0/2219671602 >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 0x7fee800b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 -- 172.21.15.157:0/2219671602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee800adf40 msgr2=0x7fee800b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:29.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee800adf40 0x7fee800b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 -- 172.21.15.157:0/2219671602 >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 msgr2=0x7fee800b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:29.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 --2- 172.21.15.157:0/2219671602 >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 0x7fee800b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85c71640 1 -- 172.21.15.157:0/2219671602 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8006be40 con 0x7fee80156a70 2023-11-06T03:13:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.725+0000 7fee85470640 1 --2- 172.21.15.157:0/2219671602 >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 0x7fee800b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee85c71640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156a70 0x7fee8006f470 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fee800cf220 tx=0x7fee7c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911edf4f3f349be2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee84c6f640 1 -- 172.21.15.157:0/2219671602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a7bd0 con 0x7fee80156a70 2023-11-06T03:13:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee84c6f640 1 -- 172.21.15.157:0/2219671602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0a7d30 con 0x7fee80156a70 2023-11-06T03:13:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee84c6f640 1 -- 172.21.15.157:0/2219671602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a7ea0 con 0x7fee80156a70 2023-11-06T03:13:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- 172.21.15.157:0/2219671602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156a70 msgr2=0x7fee8006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156a70 0x7fee8006f470 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fee800cf220 tx=0x7fee7c01eba0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- 172.21.15.157:0/2219671602 shutdown_connections 2023-11-06T03:13:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee800adf40 0x7fee800b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 --2- 172.21.15.157:0/2219671602 >> v2:172.21.15.195:3300/0 conn(0x7fee800adae0 0x7fee800b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 --2- 172.21.15.157:0/2219671602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156a70 0x7fee8006f470 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- 172.21.15.157:0/2219671602 >> 172.21.15.157:0/2219671602 conn(0x7fee80065930 msgr2=0x7fee80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- 172.21.15.157:0/2219671602 shutdown_connections 2023-11-06T03:13:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- 172.21.15.157:0/2219671602 wait complete. 2023-11-06T03:13:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 Processor -- start 2023-11-06T03:13:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.729+0000 7fee87ed5640 1 -- start start 2023-11-06T03:13:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee800adf40 0x7fee8014e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee80156a70 0x7fee80152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8014ecf0 con 0x7fee800adae0 2023-11-06T03:13:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee801514d0 con 0x7fee800adf40 2023-11-06T03:13:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee801517b0 con 0x7fee80156a70 2023-11-06T03:13:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37776/0 (socket says 172.21.15.157:37776) 2023-11-06T03:13:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 -- 172.21.15.157:0/3093068673 learned_addr learned my addr 172.21.15.157:0/3093068673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 -- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee80156a70 msgr2=0x7fee80152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee86472640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee80156a70 0x7fee80152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee80156a70 0x7fee80152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 -- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee800adf40 msgr2=0x7fee8014e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee800adf40 0x7fee8014e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 -- 172.21.15.157:0/3093068673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c00b770 con 0x7fee800adae0 2023-11-06T03:13:29.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85c71640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00b2c0 tx=0x7fee7c01b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a7ea0 con 0x7fee800adae0 2023-11-06T03:13:29.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0afd70 con 0x7fee800adae0 2023-11-06T03:13:29.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee80151a90 con 0x7fee800adae0 2023-11-06T03:13:29.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0ba6a0 con 0x7fee800adae0 2023-11-06T03:13:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8014e850 con 0x7fee800adae0 2023-11-06T03:13:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8006f970 con 0x7fee800adae0 2023-11-06T03:13:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fee7c0ba800 con 0x7fee800adae0 2023-11-06T03:13:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 0x7fee64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:29.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fee7c11e840 con 0x7fee800adae0 2023-11-06T03:13:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.733+0000 7fee85470640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 0x7fee64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.737+0000 7fee85470640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 0x7fee64067f20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fee800ae740 tx=0x7fee70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:29.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.737+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fee7c0ec080 con 0x7fee800adae0 2023-11-06T03:13:29.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.913+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fee8014ea40 con 0x7fee800adae0 2023-11-06T03:13:29.910 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:13:29.913+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:13:29.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.933+0000 7fee767fc640 1 -- 172.21.15.157:0/3093068673 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee7c0b4020 con 0x7fee800adae0 2023-11-06T03:13:29.931 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:13:29.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 msgr2=0x7fee64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 0x7fee64067f20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fee800ae740 tx=0x7fee70008040 comp rx=0 tx=0).stop 2023-11-06T03:13:29.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 msgr2=0x7fee80145d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:29.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00b2c0 tx=0x7fee7c01b940 comp rx=0 tx=0).stop 2023-11-06T03:13:29.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 shutdown_connections 2023-11-06T03:13:29.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a60 0x7fee64067f20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee80156a70 0x7fee80152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee800adf40 0x7fee8014e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 --2- 172.21.15.157:0/3093068673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee800adae0 0x7fee80145d30 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:29.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 >> 172.21.15.157:0/3093068673 conn(0x7fee80065930 msgr2=0x7fee8015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:29.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 shutdown_connections 2023-11-06T03:13:29.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:29.937+0000 7fee87ed5640 1 -- 172.21.15.157:0/3093068673 wait complete. 2023-11-06T03:13:29.949 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:13:30.005 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:13:30.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 Processor -- start 2023-11-06T03:13:30.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 -- start start 2023-11-06T03:13:30.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f625415ce80 0x7f62540cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62540d00b0 0x7f62540b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540d2920 con 0x7f625415caa0 2023-11-06T03:13:30.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540d0520 con 0x7f625415ce80 2023-11-06T03:13:30.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f625b2b7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540d18a0 con 0x7f62540d00b0 2023-11-06T03:13:30.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f625415ce80 0x7f62540cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:30.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6259053640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f625415caa0 0x7f62540cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38668/0 (socket says 172.21.15.157:38668) 2023-11-06T03:13:30.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6259053640 1 -- 172.21.15.157:0/2792094550 learned_addr learned my addr 172.21.15.157:0/2792094550 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:30.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f625415ce80 0x7f62540cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37806/0 (socket says 172.21.15.157:37806) 2023-11-06T03:13:30.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6259854640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62540d00b0 0x7f62540b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:30.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 -- 172.21.15.157:0/2792094550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62540d00b0 msgr2=0x7f62540b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:30.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62540d00b0 0x7f62540b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 -- 172.21.15.157:0/2792094550 >> v1:172.21.15.157:6789/0 conn(0x7f625415caa0 legacy=0x7f62540cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.061+0000 7f6258852640 1 -- 172.21.15.157:0/2792094550 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540d1b60 con 0x7f625415ce80 2023-11-06T03:13:30.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6258852640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415ce80 0x7f62540cfb70 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f62500097c0 tx=0x7f625001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1900dc6aad80e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:30.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f624bfff640 1 -- 172.21.15.157:0/2792094550 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500ab650 con 0x7f625415ce80 2023-11-06T03:13:30.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f624bfff640 1 -- 172.21.15.157:0/2792094550 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62500ab7b0 con 0x7f625415ce80 2023-11-06T03:13:30.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f624bfff640 1 -- 172.21.15.157:0/2792094550 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62500abb40 con 0x7f625415ce80 2023-11-06T03:13:30.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/2792094550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415ce80 msgr2=0x7f62540cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415ce80 0x7f62540cfb70 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f62500097c0 tx=0x7f625001c450 comp rx=0 tx=0).stop 2023-11-06T03:13:30.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/2792094550 shutdown_connections 2023-11-06T03:13:30.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62540d00b0 0x7f62540b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- 172.21.15.157:0/2792094550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f625415ce80 0x7f62540cfb70 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/2792094550 >> 172.21.15.157:0/2792094550 conn(0x7f6254070790 msgr2=0x7f6254070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:30.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/2792094550 shutdown_connections 2023-11-06T03:13:30.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/2792094550 wait complete. 2023-11-06T03:13:30.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 Processor -- start 2023-11-06T03:13:30.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- start start 2023-11-06T03:13:30.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f625415caa0 0x7f62540cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f625415ce80 0x7f62541547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540ca200 con 0x7f62540d00b0 2023-11-06T03:13:30.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254160080 con 0x7f625415ce80 2023-11-06T03:13:30.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540cbad0 con 0x7f625415caa0 2023-11-06T03:13:30.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:30.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37814/0 (socket says 172.21.15.157:37814) 2023-11-06T03:13:30.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 -- 172.21.15.157:0/3736810026 learned_addr learned my addr 172.21.15.157:0/3736810026 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:30.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 -- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f625415caa0 msgr2=0x7f62540cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f625415caa0 0x7f62540cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 -- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f625415ce80 msgr2=0x7f62541547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f625415ce80 0x7f62541547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 -- 172.21.15.157:0/3736810026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250009420 con 0x7f62540d00b0 2023-11-06T03:13:30.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6259053640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6244005890 tx=0x7f624400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:30.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244005cf0 con 0x7f62540d00b0 2023-11-06T03:13:30.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6244005e50 con 0x7f62540d00b0 2023-11-06T03:13:30.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540cbdb0 con 0x7f62540d00b0 2023-11-06T03:13:30.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62440072b0 con 0x7f62540d00b0 2023-11-06T03:13:30.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.065+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6254154d30 con 0x7f62540d00b0 2023-11-06T03:13:30.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62541558e0 con 0x7f62540d00b0 2023-11-06T03:13:30.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6244007410 con 0x7f62540d00b0 2023-11-06T03:13:30.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f6249ffb640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 0x7f6238067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:30.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f6244082850 con 0x7f62540d00b0 2023-11-06T03:13:30.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f6258852640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 0x7f6238067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:30.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.069+0000 7f6258852640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 0x7f6238067f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f62500af000 tx=0x7f6250008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:30.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.073+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6244050030 con 0x7f62540d00b0 2023-11-06T03:13:30.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.245+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6254072750 con 0x7f62540d00b0 2023-11-06T03:13:30.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.933+0000 7f6249ffb640 1 -- 172.21.15.157:0/3736810026 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6244053650 con 0x7f62540d00b0 2023-11-06T03:13:30.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 msgr2=0x7f6238067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 0x7f6238067f50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f62500af000 tx=0x7f6250008040 comp rx=0 tx=0).stop 2023-11-06T03:13:30.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 msgr2=0x7f62541565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:30.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6244005890 tx=0x7f624400c960 comp rx=0 tx=0).stop 2023-11-06T03:13:30.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 shutdown_connections 2023-11-06T03:13:30.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6238065a90 0x7f6238067f50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f625415ce80 0x7f62541547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f625415caa0 0x7f62540cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 --2- 172.21.15.157:0/3736810026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62540d00b0 0x7f62541565e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:30.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 >> 172.21.15.157:0/3736810026 conn(0x7f6254070790 msgr2=0x7f62540b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:30.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 shutdown_connections 2023-11-06T03:13:30.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:30.937+0000 7f625b2b7640 1 -- 172.21.15.157:0/3736810026 wait complete. 2023-11-06T03:13:30.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:13:31.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 Processor -- start 2023-11-06T03:13:31.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 -- start start 2023-11-06T03:13:31.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 0x7f96480d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f96480d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f964806a5f0 0x7f964806aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9648160760 con 0x7f964806a5f0 2023-11-06T03:13:31.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964806afa0 con 0x7f964815ca40 2023-11-06T03:13:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.085+0000 7f964dfa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9648160a20 con 0x7f964815ce20 2023-11-06T03:13:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f964806a5f0 0x7f964806aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f964806a5f0 0x7f964806aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37822/0 (socket says 172.21.15.157:37822) 2023-11-06T03:13:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 -- 172.21.15.157:0/2463004715 learned_addr learned my addr 172.21.15.157:0/2463004715 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:31.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9646ffd640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f96480d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f96477fe640 1 --2- 172.21.15.157:0/2463004715 >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 0x7f96480d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 -- 172.21.15.157:0/2463004715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 msgr2=0x7f96480d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f96480d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 -- 172.21.15.157:0/2463004715 >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 msgr2=0x7f96480d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:31.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 --2- 172.21.15.157:0/2463004715 >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 0x7f96480d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 -- 172.21.15.157:0/2463004715 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648160d00 con 0x7f964806a5f0 2023-11-06T03:13:31.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f96477fe640 1 --2- 172.21.15.157:0/2463004715 >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 0x7f96480d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:31.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f9647fff640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964806a5f0 0x7f964806aa60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f96480cd580 tx=0x7f964001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa9f1bebdd82489 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:31.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f96467fc640 1 -- 172.21.15.157:0/2463004715 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400ac410 con 0x7f964806a5f0 2023-11-06T03:13:31.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f96467fc640 1 -- 172.21.15.157:0/2463004715 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96400ac570 con 0x7f964806a5f0 2023-11-06T03:13:31.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f96467fc640 1 -- 172.21.15.157:0/2463004715 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400ac900 con 0x7f964806a5f0 2023-11-06T03:13:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 -- 172.21.15.157:0/2463004715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964806a5f0 msgr2=0x7f964806aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964806a5f0 0x7f964806aa60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f96480cd580 tx=0x7f964001c500 comp rx=0 tx=0).stop 2023-11-06T03:13:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 -- 172.21.15.157:0/2463004715 shutdown_connections 2023-11-06T03:13:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964806a5f0 0x7f964806aa60 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 --2- 172.21.15.157:0/2463004715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f96480d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 --2- 172.21.15.157:0/2463004715 >> v2:172.21.15.195:3300/0 conn(0x7f964815ca40 0x7f96480d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 -- 172.21.15.157:0/2463004715 >> 172.21.15.157:0/2463004715 conn(0x7f9648071c30 msgr2=0x7f9648099ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:31.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 -- 172.21.15.157:0/2463004715 shutdown_connections 2023-11-06T03:13:31.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.089+0000 7f964dfa4640 1 -- 172.21.15.157:0/2463004715 wait complete. 2023-11-06T03:13:31.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 Processor -- start 2023-11-06T03:13:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- start start 2023-11-06T03:13:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 0x7f9648156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f964815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96481f85b0 con 0x7f964815ca40 2023-11-06T03:13:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96481f6060 con 0x7f964806a5f0 2023-11-06T03:13:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9648159dc0 con 0x7f964815ce20 2023-11-06T03:13:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37830/0 (socket says 172.21.15.157:37830) 2023-11-06T03:13:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 -- 172.21.15.157:0/1432342559 learned_addr learned my addr 172.21.15.157:0/1432342559 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9647fff640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f964815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f96477fe640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 0x7f9648156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 -- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 msgr2=0x7f964815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:31.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f964815ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 -- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 msgr2=0x7f9648156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 0x7f9648156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 -- 172.21.15.157:0/1432342559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400094d0 con 0x7f964815ca40 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f96477fe640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 0x7f9648156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f9646ffd640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d7d0 tx=0x7f963c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:31.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c018070 con 0x7f964815ca40 2023-11-06T03:13:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c004020 con 0x7f964815ca40 2023-11-06T03:13:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964815a0a0 con 0x7f964815ca40 2023-11-06T03:13:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0041d0 con 0x7f964815ca40 2023-11-06T03:13:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964815a660 con 0x7f964815ca40 2023-11-06T03:13:31.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.093+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964804e050 con 0x7f964815ca40 2023-11-06T03:13:31.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.097+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f963c004330 con 0x7f964815ca40 2023-11-06T03:13:31.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.097+0000 7f962ffff640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 0x7f9634067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:31.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.097+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f963c0831f0 con 0x7f964815ca40 2023-11-06T03:13:31.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.097+0000 7f96477fe640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 0x7f9634067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:31.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.097+0000 7f96477fe640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 0x7f9634067f20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f964815f2a0 tx=0x7f9638008530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:31.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.101+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f963c050a40 con 0x7f964815ca40 2023-11-06T03:13:31.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.253+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f964815e5f0 con 0x7f964815ca40 2023-11-06T03:13:31.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.945+0000 7f962ffff640 1 -- 172.21.15.157:0/1432342559 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7f963c054060 con 0x7f964815ca40 2023-11-06T03:13:31.944 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:13:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 msgr2=0x7f9634067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 0x7f9634067f20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f964815f2a0 tx=0x7f9638008530 comp rx=0 tx=0).stop 2023-11-06T03:13:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 msgr2=0x7f964815fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:31.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f963c00d7d0 tx=0x7f963c00dc90 comp rx=0 tx=0).stop 2023-11-06T03:13:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 shutdown_connections 2023-11-06T03:13:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9634065a60 0x7f9634067f20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f964815ce20 0x7f964815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f964815ca40 0x7f964815fa80 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 --2- 172.21.15.157:0/1432342559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f964806a5f0 0x7f9648156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 >> 172.21.15.157:0/1432342559 conn(0x7f9648071c30 msgr2=0x7f964806e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 shutdown_connections 2023-11-06T03:13:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:31.949+0000 7f964dfa4640 1 -- 172.21.15.157:0/1432342559 wait complete. 2023-11-06T03:13:31.970 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:13:32.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.097+0000 7f801997c640 1 Processor -- start 2023-11-06T03:13:32.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.097+0000 7f801997c640 1 -- start start 2023-11-06T03:13:32.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f801997c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 0x7f80140b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f801997c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f801406b2f0 0x7f801415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f801997c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8014070ee0 con 0x7f801415d690 2023-11-06T03:13:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f801997c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801415b060 con 0x7f801406b2f0 2023-11-06T03:13:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f801997c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801406b780 con 0x7f8014159c80 2023-11-06T03:13:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f80137fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f801415d690 0x7f80140b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38676/0 (socket says 172.21.15.157:38676) 2023-11-06T03:13:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f80137fe640 1 -- 172.21.15.157:0/16141923 learned_addr learned my addr 172.21.15.157:0/16141923 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:32.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f80127fc640 1 --2- 172.21.15.157:0/16141923 >> v2:172.21.15.195:3300/0 conn(0x7f801406b2f0 0x7f801415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 0x7f80140b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32418460 0 0) 0x7f8014070ee0 con 0x7f801415d690 2023-11-06T03:13:32.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc005740 con 0x7f801415d690 2023-11-06T03:13:32.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1212589388 0 0) 0x7f7ffc005740 con 0x7f801415d690 2023-11-06T03:13:32.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 msgr2=0x7f80140b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:32.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 0x7f80140b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 >> v2:172.21.15.195:3300/0 conn(0x7f801406b2f0 msgr2=0x7f801415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:32.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 --2- 172.21.15.157:0/16141923 >> v2:172.21.15.195:3300/0 conn(0x7f801406b2f0 0x7f801415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140b2a90 con 0x7f801415d690 2023-11-06T03:13:32.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f800c001680 con 0x7f801415d690 2023-11-06T03:13:32.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 >> v1:172.21.15.157:6789/0 conn(0x7f801415d690 legacy=0x7f80140b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ffc005b90 0x7f7ffc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 0x7f7ffc0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ffc006f20 0x7f7ffc0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801415b060 con 0x7f7ffc005b90 2023-11-06T03:13:32.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801406b780 con 0x7f7ffc006f20 2023-11-06T03:13:32.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc00dba0 con 0x7f8014159c80 2023-11-06T03:13:32.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ffc005b90 0x7f7ffc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f80137fe640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ffc006f20 0x7f7ffc0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 -- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 msgr2=0x7f7ffc0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:32.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159c80 0x7f7ffc0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 -- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ffc006f20 msgr2=0x7f7ffc0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:32.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ffc006f20 0x7f7ffc0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f8012ffd640 1 -- 172.21.15.157:0/16141923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc005740 con 0x7f7ffc005b90 2023-11-06T03:13:32.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.101+0000 7f80137fe640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ffc006f20 0x7f7ffc0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:32.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f8012ffd640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ffc005b90 0x7f7ffc005fb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8004009fc0 tx=0x7f800401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33ac1dd6036beb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:32.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040a5a70 con 0x7f7ffc005b90 2023-11-06T03:13:32.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800400f040 con 0x7f7ffc005b90 2023-11-06T03:13:32.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f8011ffb640 1 -- 172.21.15.157:0/16141923 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040a5e10 con 0x7f7ffc005b90 2023-11-06T03:13:32.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/16141923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ffc005b90 msgr2=0x7f7ffc005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 --2- 172.21.15.157:0/16141923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ffc005b90 0x7f7ffc005fb0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8004009fc0 tx=0x7f800401cc70 comp rx=0 tx=0).stop 2023-11-06T03:13:32.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80137fe640 1 -- 172.21.15.157:0/16141923 reap_dead start 2023-11-06T03:13:32.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/16141923 shutdown_connections 2023-11-06T03:13:32.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/16141923 >> 172.21.15.157:0/16141923 conn(0x7f80140b43b0 msgr2=0x7f8014065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:32.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/16141923 shutdown_connections 2023-11-06T03:13:32.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/16141923 wait complete. 2023-11-06T03:13:32.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 Processor -- start 2023-11-06T03:13:32.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- start start 2023-11-06T03:13:32.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159340 0x7f80140ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80141f5b70 0x7f80141f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141f68f0 con 0x7f80141f9e30 2023-11-06T03:13:32.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141f9350 con 0x7f80141f5b70 2023-11-06T03:13:32.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80141f9630 con 0x7f8014159340 2023-11-06T03:13:32.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37850/0 (socket says 172.21.15.157:37850) 2023-11-06T03:13:32.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 -- 172.21.15.157:0/3275887900 learned_addr learned my addr 172.21.15.157:0/3275887900 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:32.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f8012ffd640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159340 0x7f80140ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80137fe640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80141f5b70 0x7f80141f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 -- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159340 msgr2=0x7f80140ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159340 0x7f80140ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 -- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80141f5b70 msgr2=0x7f80141f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80141f5b70 0x7f80141f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 -- 172.21.15.157:0/3275887900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80141f9910 con 0x7f80141f9e30 2023-11-06T03:13:32.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f80127fc640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f800800e840 tx=0x7f800800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:32.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008009620 con 0x7f80141f9e30 2023-11-06T03:13:32.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800c940 con 0x7f80141f9e30 2023-11-06T03:13:32.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.105+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8014159720 con 0x7f80141f9e30 2023-11-06T03:13:32.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800ccf0 con 0x7f80141f9e30 2023-11-06T03:13:32.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80141e6760 con 0x7f80141f9e30 2023-11-06T03:13:32.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80141f91b0 con 0x7f80141f9e30 2023-11-06T03:13:32.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8008013470 con 0x7f80141f9e30 2023-11-06T03:13:32.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f7ffbfff640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 0x7f8000067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:32.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f80080832a0 con 0x7f80141f9e30 2023-11-06T03:13:32.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f8012ffd640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 0x7f8000067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:32.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.109+0000 7f8012ffd640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 0x7f8000067f20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f80040af000 tx=0x7f8004004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:32.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.113+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80141f91b0 con 0x7f80141f9e30 2023-11-06T03:13:32.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.265+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80141f91b0 con 0x7f80141f9e30 2023-11-06T03:13:32.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.949+0000 7f7ffbfff640 1 -- 172.21.15.157:0/3275887900 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f80141f91b0 con 0x7f80141f9e30 2023-11-06T03:13:32.947 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:13:32.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 msgr2=0x7f8000067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 0x7f8000067f20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f80040af000 tx=0x7f8004004ac0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 msgr2=0x7f80140acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:32.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f800800e840 tx=0x7f800800ed00 comp rx=0 tx=0).stop 2023-11-06T03:13:32.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 shutdown_connections 2023-11-06T03:13:32.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8000065a60 0x7f8000067f20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80141f5b70 0x7f80141f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80141f9e30 0x7f80140acdf0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 --2- 172.21.15.157:0/3275887900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8014159340 0x7f80140ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:32.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 >> 172.21.15.157:0/3275887900 conn(0x7f80140b43b0 msgr2=0x7f801406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:32.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 shutdown_connections 2023-11-06T03:13:32.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:32.953+0000 7f801997c640 1 -- 172.21.15.157:0/3275887900 wait complete. 2023-11-06T03:13:32.981 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:13:33.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 Processor -- start 2023-11-06T03:13:33.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 -- start start 2023-11-06T03:13:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5970159950 0x7f59700b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5970159d30 con 0x7f59700ccb90 2023-11-06T03:13:33.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59700b3f00 con 0x7f597006d7f0 2023-11-06T03:13:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f5974e45640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700702b0 con 0x7f5970159950 2023-11-06T03:13:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f59700ccb90 0x7f597015d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38690/0 (socket says 172.21.15.157:38690) 2023-11-06T03:13:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596ed9d640 1 -- 172.21.15.157:0/552240028 learned_addr learned my addr 172.21.15.157:0/552240028 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596e59c640 1 --2- 172.21.15.157:0/552240028 >> v2:172.21.15.195:3300/0 conn(0x7f5970159950 0x7f59700b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493837972 0 0) 0x7f5970159d30 con 0x7f59700ccb90 2023-11-06T03:13:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5958003530 con 0x7f59700ccb90 2023-11-06T03:13:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3721081686 0 0) 0x7f5958003530 con 0x7f59700ccb90 2023-11-06T03:13:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 >> v2:172.21.15.195:3300/0 conn(0x7f5970159950 msgr2=0x7f59700b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 --2- 172.21.15.157:0/552240028 >> v2:172.21.15.195:3300/0 conn(0x7f5970159950 0x7f59700b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 >> v1:172.21.15.195:6789/0 conn(0x7f597006d7f0 legacy=0x7f59700cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700704c0 con 0x7f59700ccb90 2023-11-06T03:13:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5968001a00 con 0x7f59700ccb90 2023-11-06T03:13:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 >> v1:172.21.15.157:6789/0 conn(0x7f59700ccb90 legacy=0x7f597015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5958003980 0x7f5958003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 0x7f5958004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5958005190 0x7f595800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700702b0 con 0x7f59580047d0 2023-11-06T03:13:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595800c030 con 0x7f5958005190 2023-11-06T03:13:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595800c1f0 con 0x7f5958003980 2023-11-06T03:13:33.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 0x7f5958004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596e59c640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5958003980 0x7f5958003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 -- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5958003980 msgr2=0x7f5958003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5958003980 0x7f5958003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 -- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5958005190 msgr2=0x7f595800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5958005190 0x7f595800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596ed9d640 1 -- 172.21.15.157:0/552240028 reap_dead start 2023-11-06T03:13:33.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.133+0000 7f596dd9b640 1 -- 172.21.15.157:0/552240028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958003530 con 0x7f59580047d0 2023-11-06T03:13:33.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f596dd9b640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 0x7f5958004c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f5960001780 tx=0x7f596001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fb1588d69bb9fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600052d0 con 0x7f59580047d0 2023-11-06T03:13:33.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59600a7050 con 0x7f59580047d0 2023-11-06T03:13:33.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f596d59a640 1 -- 172.21.15.157:0/552240028 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960003600 con 0x7f59580047d0 2023-11-06T03:13:33.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 msgr2=0x7f5958004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 0x7f5958004c50 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f5960001780 tx=0x7f596001c3e0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- 172.21.15.157:0/552240028 shutdown_connections 2023-11-06T03:13:33.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 --2- 172.21.15.157:0/552240028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59580047d0 0x7f5958004c50 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- 172.21.15.157:0/552240028 >> 172.21.15.157:0/552240028 conn(0x7f597006a170 msgr2=0x7f597006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:33.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- 172.21.15.157:0/552240028 shutdown_connections 2023-11-06T03:13:33.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- 172.21.15.157:0/552240028 wait complete. 2023-11-06T03:13:33.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 Processor -- start 2023-11-06T03:13:33.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.137+0000 7f5974e45640 1 -- start start 2023-11-06T03:13:33.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580047d0 0x7f597014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 0x7f59701f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c3810 con 0x7f59701f0980 2023-11-06T03:13:33.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c3af0 con 0x7f59580047d0 2023-11-06T03:13:33.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c3dd0 con 0x7f5970157890 2023-11-06T03:13:33.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37872/0 (socket says 172.21.15.157:37872) 2023-11-06T03:13:33.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 -- 172.21.15.157:0/1567407417 learned_addr learned my addr 172.21.15.157:0/1567407417 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:33.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596e59c640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580047d0 0x7f597014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596dd9b640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 0x7f59701f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 -- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 msgr2=0x7f59701f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 0x7f59701f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 -- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580047d0 msgr2=0x7f597014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580047d0 0x7f597014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 -- 172.21.15.157:0/1567407417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700c40b0 con 0x7f59701f0980 2023-11-06T03:13:33.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596dd9b640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 0x7f59701f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:33.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596ed9d640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5968001340 tx=0x7f596800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800da20 con 0x7f59701f0980 2023-11-06T03:13:33.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800db80 con 0x7f59701f0980 2023-11-06T03:13:33.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700c4390 con 0x7f59701f0980 2023-11-06T03:13:33.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800feb0 con 0x7f59701f0980 2023-11-06T03:13:33.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700c33a0 con 0x7f59701f0980 2023-11-06T03:13:33.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970072750 con 0x7f59701f0980 2023-11-06T03:13:33.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f596800a0d0 con 0x7f59701f0980 2023-11-06T03:13:33.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 0x7f5950067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f59680856a0 con 0x7f59701f0980 2023-11-06T03:13:33.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.141+0000 7f596e59c640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 0x7f5950067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.145+0000 7f596e59c640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 0x7f5950067f20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5970063940 tx=0x7f5964008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.145+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5968053290 con 0x7f59701f0980 2023-11-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.301+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f59700b3cc0 con 0x7f59701f0980 2023-11-06T03:13:33.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f595effd640 1 -- 172.21.15.157:0/1567407417 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f59680568b0 con 0x7f59701f0980 2023-11-06T03:13:33.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 msgr2=0x7f5950067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 0x7f5950067f20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5970063940 tx=0x7f5964008040 comp rx=0 tx=0).stop 2023-11-06T03:13:33.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 msgr2=0x7f59700c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5968001340 tx=0x7f596800fc60 comp rx=0 tx=0).stop 2023-11-06T03:13:33.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 shutdown_connections 2023-11-06T03:13:33.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5950065a60 0x7f5950067f20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59701f0980 0x7f59700c48b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5970157890 0x7f59701f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 --2- 172.21.15.157:0/1567407417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580047d0 0x7f597014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 >> 172.21.15.157:0/1567407417 conn(0x7f597006a170 msgr2=0x7f5970158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:33.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 shutdown_connections 2023-11-06T03:13:33.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.313+0000 7f5974e45640 1 -- 172.21.15.157:0/1567407417 wait complete. 2023-11-06T03:13:33.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2023-11-06T03:13:33.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 Processor -- start 2023-11-06T03:13:33.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 -- start start 2023-11-06T03:13:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8e40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40d2920 con 0x7ff8e415caa0 2023-11-06T03:13:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40d18a0 con 0x7ff8e40d00b0 2023-11-06T03:13:33.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e9f58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40d0520 con 0x7ff8e415ce80 2023-11-06T03:13:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff8e415caa0 0x7ff8e40cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38694/0 (socket says 172.21.15.157:38694) 2023-11-06T03:13:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e37fe640 1 -- 172.21.15.157:0/3857363802 learned_addr learned my addr 172.21.15.157:0/3857363802 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8daffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8e40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116590800 0 0) 0x7ff8e40d2920 con 0x7ff8e415caa0 2023-11-06T03:13:33.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0003530 con 0x7ff8e415caa0 2023-11-06T03:13:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648111775 0 0) 0x7ff8d0003530 con 0x7ff8e415caa0 2023-11-06T03:13:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 msgr2=0x7ff8e40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8e40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 >> v1:172.21.15.195:6789/0 conn(0x7ff8e40d00b0 legacy=0x7ff8e40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40d1b60 con 0x7ff8e415caa0 2023-11-06T03:13:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff8d4001680 con 0x7ff8e415caa0 2023-11-06T03:13:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 >> v1:172.21.15.157:6789/0 conn(0x7ff8e415caa0 legacy=0x7ff8e40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d00039b0 0x7ff8d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d0004d10 0x7ff8d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40d0520 con 0x7ff8d0004d10 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000ba70 con 0x7ff8d00039b0 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000bc30 con 0x7ff8e415ce80 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d0004d10 0x7ff8d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e37fe640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d00039b0 0x7ff8d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8daffd640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 -- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 msgr2=0x7ff8d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e415ce80 0x7ff8d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 -- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d00039b0 msgr2=0x7ff8d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:33.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d00039b0 0x7ff8d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e3fff640 1 -- 172.21.15.157:0/3857363802 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0003530 con 0x7ff8d0004d10 2023-11-06T03:13:33.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.457+0000 7ff8e37fe640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8d00039b0 0x7ff8d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:33.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e3fff640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d0004d10 0x7ff8d0005190 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc001e40 tx=0x7ff8dc01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591725cd190a363d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc003b80 con 0x7ff8d0004d10 2023-11-06T03:13:33.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc003ce0 con 0x7ff8d0004d10 2023-11-06T03:13:33.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e2ffd640 1 -- 172.21.15.157:0/3857363802 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0052c0 con 0x7ff8d0004d10 2023-11-06T03:13:33.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- 172.21.15.157:0/3857363802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d0004d10 msgr2=0x7ff8d0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/3857363802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8d0004d10 0x7ff8d0005190 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc001e40 tx=0x7ff8dc01c740 comp rx=0 tx=0).stop 2023-11-06T03:13:33.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e3fff640 1 -- 172.21.15.157:0/3857363802 reap_dead start 2023-11-06T03:13:33.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- 172.21.15.157:0/3857363802 shutdown_connections 2023-11-06T03:13:33.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- 172.21.15.157:0/3857363802 >> 172.21.15.157:0/3857363802 conn(0x7ff8e4070790 msgr2=0x7ff8e4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:33.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- 172.21.15.157:0/3857363802 shutdown_connections 2023-11-06T03:13:33.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- 172.21.15.157:0/3857363802 wait complete. 2023-11-06T03:13:33.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 Processor -- start 2023-11-06T03:13:33.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- start start 2023-11-06T03:13:33.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e4159810 0x7ff8e415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e4159cd0 0x7ff8e40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41e2340 con 0x7ff8e4159430 2023-11-06T03:13:33.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41e2620 con 0x7ff8e4159cd0 2023-11-06T03:13:33.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e9f58640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41e2900 con 0x7ff8e4159810 2023-11-06T03:13:33.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37894/0 (socket says 172.21.15.157:37894) 2023-11-06T03:13:33.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e37fe640 1 -- 172.21.15.157:0/2982480661 learned_addr learned my addr 172.21.15.157:0/2982480661 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:33.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8daffd640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e4159810 0x7ff8e415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.461+0000 7ff8e3fff640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e4159cd0 0x7ff8e40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 -- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e4159810 msgr2=0x7ff8e415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e4159810 0x7ff8e415a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 -- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e4159cd0 msgr2=0x7ff8e40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e4159cd0 0x7ff8e40cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 -- 172.21.15.157:0/2982480661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e41e2be0 con 0x7ff8e4159430 2023-11-06T03:13:33.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e37fe640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4001bc0 tx=0x7ff8d400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400c870 con 0x7ff8e4159430 2023-11-06T03:13:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d400c9d0 con 0x7ff8e4159430 2023-11-06T03:13:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e41e2ec0 con 0x7ff8e4159430 2023-11-06T03:13:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400cd40 con 0x7ff8e4159430 2023-11-06T03:13:33.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40cc700 con 0x7ff8e4159430 2023-11-06T03:13:33.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e41e3100 con 0x7ff8e4159430 2023-11-06T03:13:33.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff8d401d460 con 0x7ff8e4159430 2023-11-06T03:13:33.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 0x7ff8c4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:33.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff8d4083020 con 0x7ff8e4159430 2023-11-06T03:13:33.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8daffd640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 0x7ff8c4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.465+0000 7ff8daffd640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 0x7ff8c4067f50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40cce10 tx=0x7ff8cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:33.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.469+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff8e41e3100 con 0x7ff8e4159430 2023-11-06T03:13:33.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.681+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7ff8e41e3100 con 0x7ff8e4159430 2023-11-06T03:13:33.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.961+0000 7ff8e0ff9640 1 -- 172.21.15.157:0/2982480661 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v441) v1 ==== 155+0+0 (secure 0 0 0) 0x7ff8e41e3100 con 0x7ff8e4159430 2023-11-06T03:13:33.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.961+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 msgr2=0x7ff8c4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.961+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 0x7ff8c4067f50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40cce10 tx=0x7ff8cc008040 comp rx=0 tx=0).stop 2023-11-06T03:13:33.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.961+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 msgr2=0x7ff8e41f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:33.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.961+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4001bc0 tx=0x7ff8d400ed00 comp rx=0 tx=0).stop 2023-11-06T03:13:33.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 shutdown_connections 2023-11-06T03:13:33.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff8c4065a90 0x7ff8c4067f50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8e4159cd0 0x7ff8e40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8e4159810 0x7ff8e415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 --2- 172.21.15.157:0/2982480661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8e4159430 0x7ff8e41f2240 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:33.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 >> 172.21.15.157:0/2982480661 conn(0x7ff8e4070790 msgr2=0x7ff8e40b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:33.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 shutdown_connections 2023-11-06T03:13:33.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:33.965+0000 7ff8e9f58640 1 -- 172.21.15.157:0/2982480661 wait complete. 2023-11-06T03:13:33.982 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2023-11-06T03:13:33.983 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2023-11-06T03:13:33.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2023-11-06T03:13:34.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 Processor -- start 2023-11-06T03:13:34.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 -- start start 2023-11-06T03:13:34.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f682c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c15e440 con 0x7f682c1599c0 2023-11-06T03:13:34.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c06c240 con 0x7f682c0b2af0 2023-11-06T03:13:34.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6833516640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c06c520 con 0x7f682c06d000 2023-11-06T03:13:34.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f68312b2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f682c1599c0 0x7f682c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38698/0 (socket says 172.21.15.157:38698) 2023-11-06T03:13:34.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f68312b2640 1 -- 172.21.15.157:0/3488005155 learned_addr learned my addr 172.21.15.157:0/3488005155 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:34.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371748222 0 0) 0x7f682c15e440 con 0x7f682c1599c0 2023-11-06T03:13:34.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6818003530 con 0x7f682c1599c0 2023-11-06T03:13:34.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f682c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1813333220 0 0) 0x7f6818003530 con 0x7f682c1599c0 2023-11-06T03:13:34.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 msgr2=0x7f682c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:34.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f682c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 >> v1:172.21.15.195:6789/0 conn(0x7f682c0b2af0 legacy=0x7f682c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c06c800 con 0x7f682c1599c0 2023-11-06T03:13:34.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6828001680 con 0x7f682c1599c0 2023-11-06T03:13:34.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 >> v1:172.21.15.157:6789/0 conn(0x7f682c1599c0 legacy=0x7f682c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f68180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6818003f30 0x7f68180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6818004d10 0x7f6818005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c06c520 con 0x7f6818004d10 2023-11-06T03:13:34.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681800ba70 con 0x7f6818003f30 2023-11-06T03:13:34.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681800bc30 con 0x7f682c06d000 2023-11-06T03:13:34.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6818004d10 0x7f6818005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6830ab1640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6818003f30 0x7f68180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f68312b2640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f68180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 -- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 msgr2=0x7f68180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:34.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f682c06d000 0x7f68180039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 -- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6818003f30 msgr2=0x7f68180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:34.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6818003f30 0x7f68180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6831ab3640 1 -- 172.21.15.157:0/3488005155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6818003530 con 0x7f6818004d10 2023-11-06T03:13:34.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.109+0000 7f6830ab1640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6818003f30 0x7f68180047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:34.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6818004d10 0x7f6818005190 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f90 tx=0x7f682401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8776c3c16e82f0ee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:34.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a5c30 con 0x7f6818004d10 2023-11-06T03:13:34.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682400f040 con 0x7f6818004d10 2023-11-06T03:13:34.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6823fff640 1 -- 172.21.15.157:0/3488005155 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240ad440 con 0x7f6818004d10 2023-11-06T03:13:34.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- 172.21.15.157:0/3488005155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6818004d10 msgr2=0x7f6818005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 --2- 172.21.15.157:0/3488005155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6818004d10 0x7f6818005190 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6824009f90 tx=0x7f682401cc20 comp rx=0 tx=0).stop 2023-11-06T03:13:34.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6831ab3640 1 -- 172.21.15.157:0/3488005155 reap_dead start 2023-11-06T03:13:34.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- 172.21.15.157:0/3488005155 shutdown_connections 2023-11-06T03:13:34.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- 172.21.15.157:0/3488005155 >> 172.21.15.157:0/3488005155 conn(0x7f682c065930 msgr2=0x7f682c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:34.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- 172.21.15.157:0/3488005155 shutdown_connections 2023-11-06T03:13:34.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- 172.21.15.157:0/3488005155 wait complete. 2023-11-06T03:13:34.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 Processor -- start 2023-11-06T03:13:34.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.113+0000 7f6833516640 1 -- start start 2023-11-06T03:13:34.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6818004d10 0x7f682c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 0x7f682c132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c131ba0 con 0x7f682c0af5d0 2023-11-06T03:13:34.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0b0d40 con 0x7f682c1f4170 2023-11-06T03:13:34.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0b1020 con 0x7f6818004d10 2023-11-06T03:13:34.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37918/0 (socket says 172.21.15.157:37918) 2023-11-06T03:13:34.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 -- 172.21.15.157:0/3495313068 learned_addr learned my addr 172.21.15.157:0/3495313068 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:34.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f68312b2640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6818004d10 0x7f682c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 0x7f682c132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 -- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6818004d10 msgr2=0x7f682c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6818004d10 0x7f682c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 -- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 msgr2=0x7f682c132620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 0x7f682c132620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 -- 172.21.15.157:0/3495313068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c131e40 con 0x7f682c0af5d0 2023-11-06T03:13:34.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6831ab3640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 0x7f682c132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6830ab1640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e7b0 tx=0x7f681c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0050d0 con 0x7f682c0af5d0 2023-11-06T03:13:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c005230 con 0x7f682c0af5d0 2023-11-06T03:13:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c132120 con 0x7f682c0af5d0 2023-11-06T03:13:34.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00c6c0 con 0x7f682c0af5d0 2023-11-06T03:13:34.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c1f4620 con 0x7f682c0af5d0 2023-11-06T03:13:34.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c1f4810 con 0x7f682c0af5d0 2023-11-06T03:13:34.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f681c00c820 con 0x7f682c0af5d0 2023-11-06T03:13:34.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 0x7f6810067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:34.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f681c082da0 con 0x7f682c0af5d0 2023-11-06T03:13:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.117+0000 7f68312b2640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 0x7f6810067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.121+0000 7f68312b2640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 0x7f6810067f50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6828000fc0 tx=0x7f6828001480 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:34.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.121+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f681c050a50 con 0x7f682c0af5d0 2023-11-06T03:13:34.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.321+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f682c072750 con 0x7f682c0af5d0 2023-11-06T03:13:34.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6821ffb640 1 -- 172.21.15.157:0/3495313068 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v442) v1 ==== 169+0+0 (secure 0 0 0) 0x7f681c054070 con 0x7f682c0af5d0 2023-11-06T03:13:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 msgr2=0x7f6810067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 0x7f6810067f50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6828000fc0 tx=0x7f6828001480 comp rx=0 tx=0).stop 2023-11-06T03:13:34.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 msgr2=0x7f682c1f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e7b0 tx=0x7f681c00ec70 comp rx=0 tx=0).stop 2023-11-06T03:13:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 shutdown_connections 2023-11-06T03:13:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6810065a90 0x7f6810067f50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f682c1f4170 0x7f682c132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f682c0af5d0 0x7f682c1f3c30 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 --2- 172.21.15.157:0/3495313068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6818004d10 0x7f682c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 >> 172.21.15.157:0/3495313068 conn(0x7f682c065930 msgr2=0x7f682c157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 shutdown_connections 2023-11-06T03:13:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:34.965+0000 7f6833516640 1 -- 172.21.15.157:0/3495313068 wait complete. 2023-11-06T03:13:34.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:13:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.105+0000 7f9d9b84a640 1 Processor -- start 2023-11-06T03:13:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.105+0000 7f9d9b84a640 1 -- start start 2023-11-06T03:13:35.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d9b84a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d9415ce80 0x7f9d940cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d9b84a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9d940d00b0 0x7f9d940b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d9b84a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d940d2920 con 0x7f9d9415caa0 2023-11-06T03:13:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d9b84a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940d0520 con 0x7f9d9415ce80 2023-11-06T03:13:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d9b84a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940d18a0 con 0x7f9d940d00b0 2023-11-06T03:13:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d9415ce80 0x7f9d940cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d9415ce80 0x7f9d940cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37924/0 (socket says 172.21.15.157:37924) 2023-11-06T03:13:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 -- 172.21.15.157:0/3555151638 learned_addr learned my addr 172.21.15.157:0/3555151638 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d99de7640 1 --2- 172.21.15.157:0/3555151638 >> v2:172.21.15.195:3300/0 conn(0x7f9d940d00b0 0x7f9d940b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 -- 172.21.15.157:0/3555151638 >> v2:172.21.15.195:3300/0 conn(0x7f9d940d00b0 msgr2=0x7f9d940b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 --2- 172.21.15.157:0/3555151638 >> v2:172.21.15.195:3300/0 conn(0x7f9d940d00b0 0x7f9d940b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 -- 172.21.15.157:0/3555151638 >> v1:172.21.15.195:6789/0 conn(0x7f9d9415caa0 legacy=0x7f9d940cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 -- 172.21.15.157:0/3555151638 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940d1b60 con 0x7f9d9415ce80 2023-11-06T03:13:35.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d98de5640 1 --2- 172.21.15.157:0/3555151638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415ce80 0x7f9d940cfb70 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9d940cf670 tx=0x7f9d8401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787e5bde7d8b2da8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d8bfff640 1 -- 172.21.15.157:0/3555151638 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d840aa5f0 con 0x7f9d9415ce80 2023-11-06T03:13:35.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d8bfff640 1 -- 172.21.15.157:0/3555151638 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d840aa750 con 0x7f9d9415ce80 2023-11-06T03:13:35.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.109+0000 7f9d8bfff640 1 -- 172.21.15.157:0/3555151638 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d840aaae0 con 0x7f9d9415ce80 2023-11-06T03:13:35.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/3555151638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415ce80 msgr2=0x7f9d940cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/3555151638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415ce80 0x7f9d940cfb70 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9d940cf670 tx=0x7f9d8401c490 comp rx=0 tx=0).stop 2023-11-06T03:13:35.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/3555151638 shutdown_connections 2023-11-06T03:13:35.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/3555151638 >> v2:172.21.15.195:3300/0 conn(0x7f9d940d00b0 0x7f9d940b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/3555151638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415ce80 0x7f9d940cfb70 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/3555151638 >> 172.21.15.157:0/3555151638 conn(0x7f9d94070790 msgr2=0x7f9d94070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/3555151638 shutdown_connections 2023-11-06T03:13:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/3555151638 wait complete. 2023-11-06T03:13:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 Processor -- start 2023-11-06T03:13:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- start start 2023-11-06T03:13:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d940d00b0 0x7f9d9415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9415ce80 0x7f9d940c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c81e0 con 0x7f9d9415caa0 2023-11-06T03:13:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9415de20 con 0x7f9d940d00b0 2023-11-06T03:13:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9415e100 con 0x7f9d9415ce80 2023-11-06T03:13:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37934/0 (socket says 172.21.15.157:37934) 2023-11-06T03:13:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 -- 172.21.15.157:0/2946303530 learned_addr learned my addr 172.21.15.157:0/2946303530 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d99de7640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9415ce80 0x7f9d940c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d995e6640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d940d00b0 0x7f9d9415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 -- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9415ce80 msgr2=0x7f9d940c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9415ce80 0x7f9d940c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 -- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d940d00b0 msgr2=0x7f9d9415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d940d00b0 0x7f9d9415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 -- 172.21.15.157:0/2946303530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840094d0 con 0x7f9d9415caa0 2023-11-06T03:13:35.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d98de5640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9d8401a0c0 tx=0x7f9d8401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b3800 con 0x7f9d9415caa0 2023-11-06T03:13:35.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d840b3960 con 0x7f9d9415caa0 2023-11-06T03:13:35.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b3cf0 con 0x7f9d9415caa0 2023-11-06T03:13:35.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.113+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9415fea0 con 0x7f9d9415caa0 2023-11-06T03:13:35.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d941551a0 con 0x7f9d9415caa0 2023-11-06T03:13:35.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94155390 con 0x7f9d9415caa0 2023-11-06T03:13:35.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d840c73a0 con 0x7f9d9415caa0 2023-11-06T03:13:35.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d89ffb640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 0x7f9d78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9d840a7070 con 0x7f9d9415caa0 2023-11-06T03:13:35.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d995e6640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 0x7f9d78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.117+0000 7f9d995e6640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 0x7f9d78067f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9d90004450 tx=0x7f9d90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.121+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9d94155390 con 0x7f9d9415caa0 2023-11-06T03:13:35.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.273+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9d94155390 con 0x7f9d9415caa0 2023-11-06T03:13:35.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d89ffb640 1 -- 172.21.15.157:0/2946303530 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9d94155390 con 0x7f9d9415caa0 2023-11-06T03:13:35.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 msgr2=0x7f9d78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 0x7f9d78067f20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9d90004450 tx=0x7f9d90009040 comp rx=0 tx=0).stop 2023-11-06T03:13:35.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 msgr2=0x7f9d94154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9d8401a0c0 tx=0x7f9d8401acd0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 shutdown_connections 2023-11-06T03:13:35.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9d78065a60 0x7f9d78067f20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9415ce80 0x7f9d940c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d9415caa0 0x7f9d94154c60 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 --2- 172.21.15.157:0/2946303530 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d940d00b0 0x7f9d9415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.281+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 >> 172.21.15.157:0/2946303530 conn(0x7f9d94070790 msgr2=0x7f9d94158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:35.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.285+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 shutdown_connections 2023-11-06T03:13:35.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.285+0000 7f9d9b84a640 1 -- 172.21.15.157:0/2946303530 wait complete. 2023-11-06T03:13:35.297 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:13:35.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:13:35.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:13:35.398 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:13:35.399 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:13:35.399 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:35.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:13:35.405 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:13:35.406 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:35.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:13:35.412 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:13:35.412 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:35.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:13:35.453 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:13:35.454 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:13:35.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:13:35.460 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:13:35.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 Processor -- start 2023-11-06T03:13:35.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 -- start start 2023-11-06T03:13:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9c180ce800 0x7f9c180cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 0x7f9c1806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c180cf1a0 con 0x7f9c18146370 2023-11-06T03:13:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c180b2120 con 0x7f9c180ce800 2023-11-06T03:13:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1ca1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1806c3d0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9c18146370 0x7f9c180b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38710/0 (socket says 172.21.15.157:38710) 2023-11-06T03:13:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c15d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9c180ce800 0x7f9c180cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1659c640 1 -- 172.21.15.157:0/4161799099 learned_addr learned my addr 172.21.15.157:0/4161799099 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c15d9b640 1 -- 172.21.15.157:0/4161799099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 msgr2=0x7f9c1806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c16d9d640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 0x7f9c1806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c1559a640 1 -- 172.21.15.157:0/4161799099 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930146704 0 0) 0x7f9c180cf1a0 con 0x7f9c18146370 2023-11-06T03:13:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c15d9b640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 0x7f9c1806bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c15d9b640 1 -- 172.21.15.157:0/4161799099 >> v1:172.21.15.157:6789/0 conn(0x7f9c18146370 legacy=0x7f9c180b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c15d9b640 1 -- 172.21.15.157:0/4161799099 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1806f5a0 con 0x7f9c180ce800 2023-11-06T03:13:35.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.625+0000 7f9c16d9d640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 0x7f9c1806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:35.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c15d9b640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c180ce800 0x7f9c180cec60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097c0 tx=0x7f9c0c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16083613120bb2be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1559a640 1 -- 172.21.15.157:0/4161799099 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0ac620 con 0x7f9c180ce800 2023-11-06T03:13:35.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1559a640 1 -- 172.21.15.157:0/4161799099 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0ac780 con 0x7f9c180ce800 2023-11-06T03:13:35.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1559a640 1 -- 172.21.15.157:0/4161799099 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0acb10 con 0x7f9c180ce800 2023-11-06T03:13:35.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/4161799099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c180ce800 msgr2=0x7f9c180cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c180ce800 0x7f9c180cec60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097c0 tx=0x7f9c0c01c3c0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/4161799099 shutdown_connections 2023-11-06T03:13:35.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c1806b6a0 0x7f9c1806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/4161799099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c180ce800 0x7f9c180cec60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/4161799099 >> 172.21.15.157:0/4161799099 conn(0x7f9c18065930 msgr2=0x7f9c18065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:35.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/4161799099 shutdown_connections 2023-11-06T03:13:35.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/4161799099 wait complete. 2023-11-06T03:13:35.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 Processor -- start 2023-11-06T03:13:35.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.629+0000 7f9c1ca1f640 1 -- start start 2023-11-06T03:13:35.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c180ce800 0x7f9c180b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c18146370 0x7f9c180b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1815a020 con 0x7f9c1806b6a0 2023-11-06T03:13:35.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c18159410 con 0x7f9c18146370 2023-11-06T03:13:35.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c181596f0 con 0x7f9c180ce800 2023-11-06T03:13:35.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37958/0 (socket says 172.21.15.157:37958) 2023-11-06T03:13:35.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 -- 172.21.15.157:0/1090097070 learned_addr learned my addr 172.21.15.157:0/1090097070 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:35.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c15d9b640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c180ce800 0x7f9c180b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 -- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c180ce800 msgr2=0x7f9c180b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c180ce800 0x7f9c180b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c16d9d640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c18146370 0x7f9c180b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 -- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c18146370 msgr2=0x7f9c180b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c18146370 0x7f9c180b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 -- 172.21.15.157:0/1090097070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009420 con 0x7f9c1806b6a0 2023-11-06T03:13:35.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1659c640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c480 tx=0x7f9c0800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080074e0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08007660 con 0x7f9c1806b6a0 2023-11-06T03:13:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004a30 con 0x7f9c1806b6a0 2023-11-06T03:13:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c181599d0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1813c7a0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c18072750 con 0x7f9c1806b6a0 2023-11-06T03:13:35.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9c08004b90 con 0x7f9c1806b6a0 2023-11-06T03:13:35.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 0x7f9bf8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:35.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.633+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9c080820e0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.637+0000 7f9c15d9b640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 0x7f9bf8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.637+0000 7f9c15d9b640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 0x7f9bf8067f50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0ae000 tx=0x7f9c0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:35.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.641+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9c08053760 con 0x7f9c1806b6a0 2023-11-06T03:13:35.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.813+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c181601c0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.813+0000 7f9c06ffd640 1 -- 172.21.15.157:0/1090097070 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f9c08018df0 con 0x7f9c1806b6a0 2023-11-06T03:13:35.812 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:35.812 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2936193846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2936193846},{"type":"v1","addr":"172.21.15.195:6837","nonce":2936193846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1984899334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1984899334},{"type":"v1","addr":"172.21.15.195:6835","nonce":1984899334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3139555301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3139555301},{"type":"v1","addr":"172.21.15.157:6834","nonce":3139555301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1567080836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1567080836},{"type":"v1","addr":"172.21.15.157:6835","nonce":1567080836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-11-06T03:13:35.812 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 87 2023-11-06T03:13:35.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 msgr2=0x7f9bf8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 0x7f9bf8067f50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0ae000 tx=0x7f9c0c008040 comp rx=0 tx=0).stop 2023-11-06T03:13:35.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 msgr2=0x7f9c1813c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:35.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c480 tx=0x7f9c0800c940 comp rx=0 tx=0).stop 2023-11-06T03:13:35.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 shutdown_connections 2023-11-06T03:13:35.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9bf8065a90 0x7f9bf8067f50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c18146370 0x7f9c180b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c180ce800 0x7f9c180b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 --2- 172.21.15.157:0/1090097070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c1806b6a0 0x7f9c1813c260 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:35.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 >> 172.21.15.157:0/1090097070 conn(0x7f9c18065930 msgr2=0x7f9c181610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:35.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 shutdown_connections 2023-11-06T03:13:35.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:35.817+0000 7f9c1ca1f640 1 -- 172.21.15.157:0/1090097070 wait complete. 2023-11-06T03:13:35.830 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:13:35.830 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:13:35.831 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:13:35.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:13:35.831 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:13:35.832 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:13:35.832 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:13:35.832 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:13:35.832 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:13:35.833 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:13:35.833 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:13:35.833 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:13:35.833 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:13:35.834 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:13:35.834 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:13:35.834 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:13:35.840 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:13:35.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:13:35.843 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:13:35.843 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:13:35.845 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:13:35.845 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:13:35.846 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:13:35.846 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:13:35.888 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:13:35.889 DEBUG:teuthology.parallel:result is None 2023-11-06T03:13:35.890 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:13:35.891 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:13:35.891 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:13:35.891 DEBUG:teuthology.parallel:result is None 2023-11-06T03:13:35.892 DEBUG:teuthology.parallel:result is None 2023-11-06T03:13:35.892 DEBUG:teuthology.parallel:result is None 2023-11-06T03:13:35.893 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:13:35.947 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:13:35.949 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:13:35.956 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:13:35.959 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:13:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 Processor -- start 2023-11-06T03:13:36.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 -- start start 2023-11-06T03:13:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f889415aec0 0x7f88940cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8894071d00 0x7f88940ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88940cd0d0 con 0x7f88940717b0 2023-11-06T03:13:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889415b2a0 con 0x7f889415aec0 2023-11-06T03:13:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8899743640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88940afc90 con 0x7f8894071d00 2023-11-06T03:13:36.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f889415aec0 0x7f88940cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f889415aec0 0x7f88940cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37994/0 (socket says 172.21.15.157:37994) 2023-11-06T03:13:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 -- 172.21.15.157:0/3407355531 learned_addr learned my addr 172.21.15.157:0/3407355531 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:36.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f88927fc640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8894071d00 0x7f88940ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 -- 172.21.15.157:0/3407355531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8894071d00 msgr2=0x7f88940ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8894071d00 0x7f88940ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 -- 172.21.15.157:0/3407355531 >> v1:172.21.15.195:6789/0 conn(0x7f88940717b0 legacy=0x7f889406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:36.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.009+0000 7f8892ffd640 1 -- 172.21.15.157:0/3407355531 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940aff70 con 0x7f889415aec0 2023-11-06T03:13:36.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8892ffd640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f889415aec0 0x7f88940cc650 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8884006fd0 tx=0x7f888401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4b6682cd416d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:36.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8891ffb640 1 -- 172.21.15.157:0/3407355531 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840aa670 con 0x7f889415aec0 2023-11-06T03:13:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8891ffb640 1 -- 172.21.15.157:0/3407355531 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88840aa7d0 con 0x7f889415aec0 2023-11-06T03:13:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8891ffb640 1 -- 172.21.15.157:0/3407355531 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840aab60 con 0x7f889415aec0 2023-11-06T03:13:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- 172.21.15.157:0/3407355531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f889415aec0 msgr2=0x7f88940cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f889415aec0 0x7f88940cc650 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8884006fd0 tx=0x7f888401c4f0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- 172.21.15.157:0/3407355531 shutdown_connections 2023-11-06T03:13:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8894071d00 0x7f88940ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 --2- 172.21.15.157:0/3407355531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f889415aec0 0x7f88940cc650 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- 172.21.15.157:0/3407355531 >> 172.21.15.157:0/3407355531 conn(0x7f88940cd3c0 msgr2=0x7f88940cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- 172.21.15.157:0/3407355531 shutdown_connections 2023-11-06T03:13:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- 172.21.15.157:0/3407355531 wait complete. 2023-11-06T03:13:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 Processor -- start 2023-11-06T03:13:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.013+0000 7f8899743640 1 -- start start 2023-11-06T03:13:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8894071d00 0x7f8894145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f889415aec0 0x7f88941f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894150350 con 0x7f88940717b0 2023-11-06T03:13:36.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894150630 con 0x7f8894071d00 2023-11-06T03:13:36.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8894150910 con 0x7f889415aec0 2023-11-06T03:13:36.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38000/0 (socket says 172.21.15.157:38000) 2023-11-06T03:13:36.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 -- 172.21.15.157:0/2696829330 learned_addr learned my addr 172.21.15.157:0/2696829330 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:36.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f88937fe640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f889415aec0 0x7f88941f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f88927fc640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8894071d00 0x7f8894145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 -- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f889415aec0 msgr2=0x7f88941f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:36.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f889415aec0 0x7f88941f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 -- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8894071d00 msgr2=0x7f8894145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:36.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8894071d00 0x7f8894145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:36.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 -- 172.21.15.157:0/2696829330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8884009480 con 0x7f88940717b0 2023-11-06T03:13:36.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8892ffd640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f888401aba0 tx=0x7f8884019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:36.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840aad50 con 0x7f88940717b0 2023-11-06T03:13:36.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88840aa630 con 0x7f88940717b0 2023-11-06T03:13:36.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 -- 172.21.15.157:0/2696829330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894150bf0 con 0x7f88940717b0 2023-11-06T03:13:36.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884003860 con 0x7f88940717b0 2023-11-06T03:13:36.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f8899743640 1 -- 172.21.15.157:0/2696829330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88941f7630 con 0x7f88940717b0 2023-11-06T03:13:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c005260 con 0x7f88940717b0 2023-11-06T03:13:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f88840039c0 con 0x7f88940717b0 2023-11-06T03:13:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 0x7f8880067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f888411de40 con 0x7f88940717b0 2023-11-06T03:13:36.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f88927fc640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 0x7f8880067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:36.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.017+0000 7f88927fc640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 0x7f8880067f20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f889414ef90 tx=0x7f8888008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:36.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.021+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f88840ebbe0 con 0x7f88940717b0 2023-11-06T03:13:36.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:36.173+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f886c005510 con 0x7f88940717b0 2023-11-06T03:13:36.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.001+0000 7f887b7fe640 1 -- 172.21.15.157:0/2696829330 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7f88840c3020 con 0x7f88940717b0 2023-11-06T03:13:36.999 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:13:37.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 msgr2=0x7f8880067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 0x7f8880067f20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f889414ef90 tx=0x7f8888008040 comp rx=0 tx=0).stop 2023-11-06T03:13:37.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 msgr2=0x7f88941513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f888401aba0 tx=0x7f8884019290 comp rx=0 tx=0).stop 2023-11-06T03:13:37.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 shutdown_connections 2023-11-06T03:13:37.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8880065a60 0x7f8880067f20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f889415aec0 0x7f88941f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8894071d00 0x7f8894145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 --2- 172.21.15.157:0/2696829330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88940717b0 0x7f88941513f0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 >> 172.21.15.157:0/2696829330 conn(0x7f88940cd3c0 msgr2=0x7f88940c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 shutdown_connections 2023-11-06T03:13:37.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.005+0000 7f88797fa640 1 -- 172.21.15.157:0/2696829330 wait complete. 2023-11-06T03:13:37.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:13:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 Processor -- start 2023-11-06T03:13:37.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 -- start start 2023-11-06T03:13:37.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53480b2af0 0x7f53480ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534815e440 con 0x7f53481599c0 2023-11-06T03:13:37.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534806c240 con 0x7f534806d000 2023-11-06T03:13:37.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f534d84c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534806c520 con 0x7f53480b2af0 2023-11-06T03:13:37.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f5346ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f53481599c0 0x7f53480b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38742/0 (socket says 172.21.15.157:38742) 2023-11-06T03:13:37.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f5346ffd640 1 -- 172.21.15.157:0/841181828 learned_addr learned my addr 172.21.15.157:0/841181828 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:37.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f533e7fc640 1 --2- 172.21.15.157:0/841181828 >> v2:172.21.15.195:3300/0 conn(0x7f53480b2af0 0x7f53480ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652758173 0 0) 0x7f534815e440 con 0x7f53481599c0 2023-11-06T03:13:37.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5334003530 con 0x7f53481599c0 2023-11-06T03:13:37.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4133469757 0 0) 0x7f5334003530 con 0x7f53481599c0 2023-11-06T03:13:37.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 >> v2:172.21.15.195:3300/0 conn(0x7f53480b2af0 msgr2=0x7f53480ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 --2- 172.21.15.157:0/841181828 >> v2:172.21.15.195:3300/0 conn(0x7f53480b2af0 0x7f53480ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 >> v1:172.21.15.195:6789/0 conn(0x7f534806d000 legacy=0x7f5348159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534806c800 con 0x7f53481599c0 2023-11-06T03:13:37.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5338001680 con 0x7f53481599c0 2023-11-06T03:13:37.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 >> v1:172.21.15.157:6789/0 conn(0x7f53481599c0 legacy=0x7f53480b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5334003980 0x7f5334003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53340047d0 0x7f5334004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 0x7f533400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534806c520 con 0x7f5334005190 2023-11-06T03:13:37.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533400c030 con 0x7f5334003980 2023-11-06T03:13:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533400c1f0 con 0x7f53340047d0 2023-11-06T03:13:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 0x7f533400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f5346ffd640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5334003980 0x7f5334003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f533e7fc640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53340047d0 0x7f5334004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 -- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53340047d0 msgr2=0x7f5334004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53340047d0 0x7f5334004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 -- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5334003980 msgr2=0x7f5334003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5334003980 0x7f5334003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f53477fe640 1 -- 172.21.15.157:0/841181828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334003530 con 0x7f5334005190 2023-11-06T03:13:37.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.153+0000 7f5346ffd640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5334003980 0x7f5334003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:37.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f53477fe640 1 -- 172.21.15.157:0/841181828 reap_dead start 2023-11-06T03:13:37.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f53477fe640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 0x7f533400b990 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5340001e40 tx=0x7f534001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5f1e1cdfe8f0bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340003b80 con 0x7f5334005190 2023-11-06T03:13:37.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340003ce0 con 0x7f5334005190 2023-11-06T03:13:37.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f53467fc640 1 -- 172.21.15.157:0/841181828 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340005260 con 0x7f5334005190 2023-11-06T03:13:37.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 msgr2=0x7f533400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 0x7f533400b990 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5340001e40 tx=0x7f534001c740 comp rx=0 tx=0).stop 2023-11-06T03:13:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- 172.21.15.157:0/841181828 shutdown_connections 2023-11-06T03:13:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 --2- 172.21.15.157:0/841181828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5334005190 0x7f533400b990 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- 172.21.15.157:0/841181828 >> 172.21.15.157:0/841181828 conn(0x7f5348065930 msgr2=0x7f5348065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- 172.21.15.157:0/841181828 shutdown_connections 2023-11-06T03:13:37.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- 172.21.15.157:0/841181828 wait complete. 2023-11-06T03:13:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 Processor -- start 2023-11-06T03:13:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.157+0000 7f534d84c640 1 -- start start 2023-11-06T03:13:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5334005190 0x7f534815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53481f1b20 0x7f53480ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481f61f0 con 0x7f53481f1fa0 2023-11-06T03:13:37.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481f5900 con 0x7f53481f1b20 2023-11-06T03:13:37.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5348153a60 con 0x7f5334005190 2023-11-06T03:13:37.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38026/0 (socket says 172.21.15.157:38026) 2023-11-06T03:13:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 -- 172.21.15.157:0/3467099047 learned_addr learned my addr 172.21.15.157:0/3467099047 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f533e7fc640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53481f1b20 0x7f53480ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5346ffd640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5334005190 0x7f534815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 -- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5334005190 msgr2=0x7f534815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5334005190 0x7f534815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 -- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53481f1b20 msgr2=0x7f53480ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53481f1b20 0x7f53480ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 -- 172.21.15.157:0/3467099047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53481f5ba0 con 0x7f53481f1fa0 2023-11-06T03:13:37.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f53477fe640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f534001cfe0 tx=0x7f5334005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340005220 con 0x7f53481f1fa0 2023-11-06T03:13:37.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340005380 con 0x7f53481f1fa0 2023-11-06T03:13:37.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53481f61f0 con 0x7f53481f1fa0 2023-11-06T03:13:37.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340005730 con 0x7f53481f1fa0 2023-11-06T03:13:37.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53481536c0 con 0x7f53481f1fa0 2023-11-06T03:13:37.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53481f6380 con 0x7f53481f1fa0 2023-11-06T03:13:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53400ba460 con 0x7f53481f1fa0 2023-11-06T03:13:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 0x7f5328067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f534011c660 con 0x7f53481f1fa0 2023-11-06T03:13:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.161+0000 7f5346ffd640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 0x7f5328067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.165+0000 7f5346ffd640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 0x7f5328067f20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f53380017b0 tx=0x7f5338000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.165+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f53400ea220 con 0x7f53481f1fa0 2023-11-06T03:13:37.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.365+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f53480adcd0 con 0x7f53481f1fa0 2023-11-06T03:13:37.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f5344ff9640 1 -- 172.21.15.157:0/3467099047 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f53400ed840 con 0x7f53481f1fa0 2023-11-06T03:13:37.365 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:37.365 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:37.365 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:13:37.366 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.366 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:13:37.366 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:13:37.366 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.366 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.367 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:13:37.367 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:13:37.367 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.367 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.367 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.368 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:13:37.369 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:13:37.369 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.369 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.369 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.369 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.370 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:13:37.370 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:13:37.370 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.370 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.370 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.371 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:13:37.372 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:13:37.372 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.372 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.372 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:13:37.372 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.373 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:13:37.374 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:13:37.374 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.374 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.374 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:13:37.375 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:13:37.376 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.376 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.376 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.376 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.376 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.377 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.377 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.377 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:13:37.377 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:13:37.377 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.378 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.378 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.378 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.378 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.379 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.379 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.379 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:13:37.379 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:13:37.380 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.380 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.380 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.380 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.380 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.381 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.381 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.381 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:13:37.381 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:13:37.381 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.382 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.382 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.382 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.382 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.382 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.383 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.383 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:13:37.383 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:13:37.383 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.384 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.384 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.384 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.384 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.384 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:13:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:13:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.385 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.386 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:13:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:13:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:13:37.387 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:13:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.388 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.389 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:13:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:13:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.390 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:13:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:13:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:13:37.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:13:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:13:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:13:37.393 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:13:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:13:37.394 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:13:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:13:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.395 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:13:37.396 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:13:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:13:37.397 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:13:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:13:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:13:37.398 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.399 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:13:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:13:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.400 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:13:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:13:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:13:37.401 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.402 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.403 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:13:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:13:37.404 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:37.405 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:13:37.406 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:37.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 msgr2=0x7f5328067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 0x7f5328067f20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f53380017b0 tx=0x7f5338000ee0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 msgr2=0x7f53480acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f534001cfe0 tx=0x7f5334005800 comp rx=0 tx=0).stop 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 shutdown_connections 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5328065a60 0x7f5328067f20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53481f1fa0 0x7f53480acdf0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53481f1b20 0x7f53480ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 --2- 172.21.15.157:0/3467099047 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5334005190 0x7f534815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 >> 172.21.15.157:0/3467099047 conn(0x7f5348065930 msgr2=0x7f534815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 shutdown_connections 2023-11-06T03:13:37.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.369+0000 7f534d84c640 1 -- 172.21.15.157:0/3467099047 wait complete. 2023-11-06T03:13:37.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:37.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.537+0000 7f84e1d91640 1 Processor -- start 2023-11-06T03:13:37.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.537+0000 7f84e1d91640 1 -- start start 2023-11-06T03:13:37.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84e1d91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84dc0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84e1d91640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0ad880 con 0x7f84dc0ae920 2023-11-06T03:13:37.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84e1d91640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0adb60 con 0x7f84dc0d09b0 2023-11-06T03:13:37.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84e1d91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0ade40 con 0x7f84dc0cf080 2023-11-06T03:13:37.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84dbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84dc0ae920 0x7f84dc0aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38754/0 (socket says 172.21.15.157:38754) 2023-11-06T03:13:37.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84dbfff640 1 -- 172.21.15.157:0/2844673927 learned_addr learned my addr 172.21.15.157:0/2844673927 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:37.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84daffd640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84dc0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164617155 0 0) 0x7f84dc0ad880 con 0x7f84dc0ae920 2023-11-06T03:13:37.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c4003530 con 0x7f84dc0ae920 2023-11-06T03:13:37.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1737123335 0 0) 0x7f84c4003530 con 0x7f84dc0ae920 2023-11-06T03:13:37.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 msgr2=0x7f84dc0cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84dc0cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 >> v1:172.21.15.195:6789/0 conn(0x7f84dc0d09b0 legacy=0x7f84dc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc0ae120 con 0x7f84dc0ae920 2023-11-06T03:13:37.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f84d4001a00 con 0x7f84dc0ae920 2023-11-06T03:13:37.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 >> v1:172.21.15.157:6789/0 conn(0x7f84dc0ae920 legacy=0x7f84dc0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc0ade40 con 0x7f84c40039b0 2023-11-06T03:13:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400ba40 con 0x7f84c4004d10 2023-11-06T03:13:37.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400bc00 con 0x7f84dc0cf080 2023-11-06T03:13:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84dbfff640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84daffd640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 -- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 msgr2=0x7f84c40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84dc0cf080 0x7f84c40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 -- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 msgr2=0x7f84c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4004d10 0x7f84c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.541+0000 7f84db7fe640 1 -- 172.21.15.157:0/2844673927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4003530 con 0x7f84c40039b0 2023-11-06T03:13:37.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84db7fe640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003d90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0018c0 tx=0x7f84cc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334508fea58f8cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0ab600 con 0x7f84c40039b0 2023-11-06T03:13:37.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc0a7050 con 0x7f84c40039b0 2023-11-06T03:13:37.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84da7fc640 1 -- 172.21.15.157:0/2844673927 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0ab9c0 con 0x7f84c40039b0 2023-11-06T03:13:37.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/2844673927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 msgr2=0x7f84c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 --2- 172.21.15.157:0/2844673927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c40039b0 0x7f84c4003d90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0018c0 tx=0x7f84cc01c4b0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84dbfff640 1 -- 172.21.15.157:0/2844673927 reap_dead start 2023-11-06T03:13:37.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/2844673927 shutdown_connections 2023-11-06T03:13:37.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/2844673927 >> 172.21.15.157:0/2844673927 conn(0x7f84dc071b90 msgr2=0x7f84dc099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/2844673927 shutdown_connections 2023-11-06T03:13:37.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/2844673927 wait complete. 2023-11-06T03:13:37.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 Processor -- start 2023-11-06T03:13:37.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- start start 2023-11-06T03:13:37.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84dc0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc152cb0 0x7f84dc156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1531a0 con 0x7f84dc14a170 2023-11-06T03:13:37.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc153480 con 0x7f84dc152cb0 2023-11-06T03:13:37.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc153760 con 0x7f84c40039b0 2023-11-06T03:13:37.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38040/0 (socket says 172.21.15.157:38040) 2023-11-06T03:13:37.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 -- 172.21.15.157:0/3520659400 learned_addr learned my addr 172.21.15.157:0/3520659400 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:37.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84db7fe640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84dc0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84dbfff640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc152cb0 0x7f84dc156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 -- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 msgr2=0x7f84dc0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84dc0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 -- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc152cb0 msgr2=0x7f84dc156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc152cb0 0x7f84dc156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 -- 172.21.15.157:0/3520659400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc0b0370 con 0x7f84dc14a170 2023-11-06T03:13:37.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84daffd640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f84d00115e0 tx=0x7f84d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000f8f0 con 0x7f84dc14a170 2023-11-06T03:13:37.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d000fa50 con 0x7f84dc14a170 2023-11-06T03:13:37.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000fdc0 con 0x7f84dc14a170 2023-11-06T03:13:37.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc164ad0 con 0x7f84dc14a170 2023-11-06T03:13:37.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.545+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc14ad10 con 0x7f84dc14a170 2023-11-06T03:13:37.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc1539a0 con 0x7f84dc14a170 2023-11-06T03:13:37.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84d0015460 con 0x7f84dc14a170 2023-11-06T03:13:37.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84c3fff640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 0x7f84c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f84d00833c0 con 0x7f84dc14a170 2023-11-06T03:13:37.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84db7fe640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 0x7f84c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.549+0000 7f84db7fe640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 0x7f84c8067f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ad000 tx=0x7f84cc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.553+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84dc1539a0 con 0x7f84dc14a170 2023-11-06T03:13:37.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84dc1539a0 con 0x7f84dc14a170 2023-11-06T03:13:37.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84c3fff640 1 -- 172.21.15.157:0/3520659400 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+14229 (secure 0 0 0) 0x7f84dc1539a0 con 0x7f84dc14a170 2023-11-06T03:13:37.709 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:37.710 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":121,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:36.994981+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:37.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 msgr2=0x7f84c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 0x7f84c8067f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ad000 tx=0x7f84cc0a6000 comp rx=0 tx=0).stop 2023-11-06T03:13:37.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 msgr2=0x7f84dc152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f84d00115e0 tx=0x7f84d0011aa0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 shutdown_connections 2023-11-06T03:13:37.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84c8065a60 0x7f84c8067f20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84dc152cb0 0x7f84dc156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84dc14a170 0x7f84dc152770 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 --2- 172.21.15.157:0/3520659400 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40039b0 0x7f84dc0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 >> 172.21.15.157:0/3520659400 conn(0x7f84dc071b90 msgr2=0x7f84dc1553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 shutdown_connections 2023-11-06T03:13:37.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.713+0000 7f84e1d91640 1 -- 172.21.15.157:0/3520659400 wait complete. 2023-11-06T03:13:37.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:13:37.731 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:13:37.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 Processor -- start 2023-11-06T03:13:37.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 -- start start 2023-11-06T03:13:37.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff2b00b3b20 0x7ff2b015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b015f580 con 0x7ff2b01551e0 2023-11-06T03:13:37.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b015efc0 con 0x7ff2b015e5e0 2023-11-06T03:13:37.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2b59e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b00704b0 con 0x7ff2b00b3b20 2023-11-06T03:13:37.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff2b01551e0 0x7ff2b006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38770/0 (socket says 172.21.15.157:38770) 2023-11-06T03:13:37.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 -- 172.21.15.157:0/1899280086 learned_addr learned my addr 172.21.15.157:0/1899280086 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:37.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2af7fe640 1 --2- 172.21.15.157:0/1899280086 >> v2:172.21.15.195:3300/0 conn(0x7ff2b00b3b20 0x7ff2b015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082187399 0 0) 0x7ff2b015f580 con 0x7ff2b01551e0 2023-11-06T03:13:37.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff298003530 con 0x7ff2b01551e0 2023-11-06T03:13:37.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831476796 0 0) 0x7ff298003530 con 0x7ff2b01551e0 2023-11-06T03:13:37.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 >> v2:172.21.15.195:3300/0 conn(0x7ff2b00b3b20 msgr2=0x7ff2b015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 --2- 172.21.15.157:0/1899280086 >> v2:172.21.15.195:3300/0 conn(0x7ff2b00b3b20 0x7ff2b015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 >> v1:172.21.15.195:6789/0 conn(0x7ff2b015e5e0 legacy=0x7ff2b00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b015f2a0 con 0x7ff2b01551e0 2023-11-06T03:13:37.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff2a0001680 con 0x7ff2b01551e0 2023-11-06T03:13:37.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 >> v1:172.21.15.157:6789/0 conn(0x7ff2b01551e0 legacy=0x7ff2b006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2980052b0 0x7ff29800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00704b0 con 0x7ff298003980 2023-11-06T03:13:37.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800c180 con 0x7ff2980047d0 2023-11-06T03:13:37.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800c340 con 0x7ff2980052b0 2023-11-06T03:13:37.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2ae7fc640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 -- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2980052b0 msgr2=0x7ff29800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2980052b0 0x7ff29800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 -- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2980047d0 msgr2=0x7ff298004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:37.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2aeffd640 1 -- 172.21.15.157:0/1899280086 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298003530 con 0x7ff298003980 2023-11-06T03:13:37.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2af7fe640 1 -- 172.21.15.157:0/1899280086 reap_dead start 2023-11-06T03:13:37.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.861+0000 7ff2ae7fc640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2980047d0 0x7ff298004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:37.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 0x7ff298003da0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c210 tx=0x7ff2a001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76293ed143131e82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005ae0 con 0x7ff298003980 2023-11-06T03:13:37.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a0005c40 con 0x7ff298003980 2023-11-06T03:13:37.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2adffb640 1 -- 172.21.15.157:0/1899280086 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00072b0 con 0x7ff298003980 2023-11-06T03:13:37.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 msgr2=0x7ff298003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 0x7ff298003da0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c210 tx=0x7ff2a001cb90 comp rx=0 tx=0).stop 2023-11-06T03:13:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1899280086 shutdown_connections 2023-11-06T03:13:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1899280086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff298003980 0x7ff298003da0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1899280086 >> 172.21.15.157:0/1899280086 conn(0x7ff2b0065930 msgr2=0x7ff2b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:37.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1899280086 shutdown_connections 2023-11-06T03:13:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1899280086 wait complete. 2023-11-06T03:13:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 Processor -- start 2023-11-06T03:13:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- start start 2023-11-06T03:13:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298003980 0x7ff2b01530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 0x7ff2b00b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0153710 con 0x7ff2b014a0b0 2023-11-06T03:13:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01539f0 con 0x7ff2b0152760 2023-11-06T03:13:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.865+0000 7ff2b59e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0151f60 con 0x7ff298003980 2023-11-06T03:13:37.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38054/0 (socket says 172.21.15.157:38054) 2023-11-06T03:13:37.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 -- 172.21.15.157:0/1641938506 learned_addr learned my addr 172.21.15.157:0/1641938506 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:37.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2af7fe640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 0x7ff2b00b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298003980 0x7ff2b01530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 -- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298003980 msgr2=0x7ff2b01530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298003980 0x7ff2b01530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 -- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 msgr2=0x7ff2b00b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 0x7ff2b00b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:37.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 -- 172.21.15.157:0/1641938506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0152240 con 0x7ff2b014a0b0 2023-11-06T03:13:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2af7fe640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 0x7ff2b00b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2ae7fc640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d4a0 tx=0x7ff2a400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4005480 con 0x7ff2b014a0b0 2023-11-06T03:13:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4007030 con 0x7ff2b014a0b0 2023-11-06T03:13:37.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b014a8a0 con 0x7ff2b014a0b0 2023-11-06T03:13:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40073e0 con 0x7ff2b014a0b0 2023-11-06T03:13:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b0149de0 con 0x7ff2b014a0b0 2023-11-06T03:13:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0072750 con 0x7ff2b014a0b0 2023-11-06T03:13:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff2a4007540 con 0x7ff2b014a0b0 2023-11-06T03:13:37.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 0x7ff29c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:37.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff2a4082900 con 0x7ff2b014a0b0 2023-11-06T03:13:37.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 0x7ff29c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:37.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.869+0000 7ff2aeffd640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 0x7ff29c067f50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ab000 tx=0x7ff2a00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:37.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:37.873+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff2a40504e0 con 0x7ff2b014a0b0 2023-11-06T03:13:38.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.025+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2b01523d0 con 0x7ff2b014a0b0 2023-11-06T03:13:38.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.025+0000 7ff297fff640 1 -- 172.21.15.157:0/1641938506 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2a4019070 con 0x7ff2b014a0b0 2023-11-06T03:13:38.024 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:13:38.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 msgr2=0x7ff29c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 0x7ff29c067f50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ab000 tx=0x7ff2a00a7070 comp rx=0 tx=0).stop 2023-11-06T03:13:38.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 msgr2=0x7ff2b01498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d4a0 tx=0x7ff2a400d960 comp rx=0 tx=0).stop 2023-11-06T03:13:38.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 shutdown_connections 2023-11-06T03:13:38.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff29c065a90 0x7ff29c067f50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b0152760 0x7ff2b00b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b014a0b0 0x7ff2b01498a0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 --2- 172.21.15.157:0/1641938506 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff298003980 0x7ff2b01530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 >> 172.21.15.157:0/1641938506 conn(0x7ff2b0065930 msgr2=0x7ff2b00ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 shutdown_connections 2023-11-06T03:13:38.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.029+0000 7ff2b59e4640 1 -- 172.21.15.157:0/1641938506 wait complete. 2023-11-06T03:13:38.041 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:13:38.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.153+0000 7fb1b8e31640 1 Processor -- start 2023-11-06T03:13:38.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 -- start start 2023-11-06T03:13:38.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb1b406d000 0x7fb1b40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b415e440 con 0x7fb1b415dea0 2023-11-06T03:13:38.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b415adb0 con 0x7fb1b406d000 2023-11-06T03:13:38.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b8e31640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b406c240 con 0x7fb1b41599c0 2023-11-06T03:13:38.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1a9d9b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb1b406d000 0x7fb1b40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b2d9d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb1b415dea0 0x7fb1b4070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:57836/0 (socket says 172.21.15.157:57836) 2023-11-06T03:13:38.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b2d9d640 1 -- 172.21.15.157:0/115263972 learned_addr learned my addr 172.21.15.157:0/115263972 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:38.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:44884/0 (socket says 172.21.15.157:44884) 2023-11-06T03:13:38.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 -- 172.21.15.157:0/115263972 >> v2:172.21.15.195:3300/0 conn(0x7fb1b406d000 msgr2=0x7fb1b40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:38.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 --2- 172.21.15.157:0/115263972 >> v2:172.21.15.195:3300/0 conn(0x7fb1b406d000 0x7fb1b40b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 -- 172.21.15.157:0/115263972 >> v1:172.21.15.195:6789/0 conn(0x7fb1b415dea0 legacy=0x7fb1b4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.157+0000 7fb1b259c640 1 -- 172.21.15.157:0/115263972 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b406c520 con 0x7fb1b41599c0 2023-11-06T03:13:38.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- 172.21.15.157:0/115263972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097c0 tx=0x7fb1a401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8721a7fbc8b2975a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b1d9b640 1 -- 172.21.15.157:0/115263972 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40ad660 con 0x7fb1b41599c0 2023-11-06T03:13:38.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b1d9b640 1 -- 172.21.15.157:0/115263972 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a40ad7c0 con 0x7fb1b41599c0 2023-11-06T03:13:38.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b1d9b640 1 -- 172.21.15.157:0/115263972 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40adb50 con 0x7fb1b41599c0 2023-11-06T03:13:38.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- 172.21.15.157:0/115263972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 msgr2=0x7fb1b40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/115263972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097c0 tx=0x7fb1a401c450 comp rx=0 tx=0).stop 2023-11-06T03:13:38.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- 172.21.15.157:0/115263972 shutdown_connections 2023-11-06T03:13:38.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/115263972 >> v2:172.21.15.195:3300/0 conn(0x7fb1b406d000 0x7fb1b40b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/115263972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b41599c0 0x7fb1b40ae000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- 172.21.15.157:0/115263972 >> 172.21.15.157:0/115263972 conn(0x7fb1b4065930 msgr2=0x7fb1b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- 172.21.15.157:0/115263972 shutdown_connections 2023-11-06T03:13:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- 172.21.15.157:0/115263972 wait complete. 2023-11-06T03:13:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 Processor -- start 2023-11-06T03:13:38.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- start start 2023-11-06T03:13:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 0x7fb1b40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b415dea0 0x7fb1b40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b414fc70 con 0x7fb1b406d000 2023-11-06T03:13:38.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b414d510 con 0x7fb1b41599c0 2023-11-06T03:13:38.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b8e31640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4152810 con 0x7fb1b415dea0 2023-11-06T03:13:38.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38056/0 (socket says 172.21.15.157:38056) 2023-11-06T03:13:38.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 -- 172.21.15.157:0/262132120 learned_addr learned my addr 172.21.15.157:0/262132120 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:38.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1a9d9b640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 0x7fb1b40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b2d9d640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b415dea0 0x7fb1b40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 -- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b415dea0 msgr2=0x7fb1b40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b415dea0 0x7fb1b40acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 -- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 msgr2=0x7fb1b40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 0x7fb1b40ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 -- 172.21.15.157:0/262132120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009420 con 0x7fb1b406d000 2023-11-06T03:13:38.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1a9d9b640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 0x7fb1b40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:38.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.161+0000 7fb1b259c640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb1a401ab00 tx=0x7fb1a40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40bd070 con 0x7fb1b406d000 2023-11-06T03:13:38.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a40b6400 con 0x7fb1b406d000 2023-11-06T03:13:38.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b4151c90 con 0x7fb1b406d000 2023-11-06T03:13:38.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b6790 con 0x7fb1b406d000 2023-11-06T03:13:38.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4152250 con 0x7fb1b406d000 2023-11-06T03:13:38.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4072750 con 0x7fb1b406d000 2023-11-06T03:13:38.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb1a40b4040 con 0x7fb1b406d000 2023-11-06T03:13:38.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 0x7fb194067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb1a411c860 con 0x7fb1b406d000 2023-11-06T03:13:38.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1a9d9b640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 0x7fb194067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.165+0000 7fb1a9d9b640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 0x7fb194067f20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1b415dbe0 tx=0x7fb19c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.169+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb1a40ea4a0 con 0x7fb1b406d000 2023-11-06T03:13:38.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.321+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb1b40adcd0 con 0x7fb1b406d000 2023-11-06T03:13:38.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.321+0000 7fb1ab7fe640 1 -- 172.21.15.157:0/262132120 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb1a40edac0 con 0x7fb1b406d000 2023-11-06T03:13:38.319 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:13:38.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 msgr2=0x7fb194067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 0x7fb194067f20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb1b415dbe0 tx=0x7fb19c008040 comp rx=0 tx=0).stop 2023-11-06T03:13:38.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 msgr2=0x7fb1b414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb1a401ab00 tx=0x7fb1a40191f0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 shutdown_connections 2023-11-06T03:13:38.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb194065a60 0x7fb194067f20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb1b415dea0 0x7fb1b40acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb1b41599c0 0x7fb1b40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 --2- 172.21.15.157:0/262132120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b406d000 0x7fb1b414cfd0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 >> 172.21.15.157:0/262132120 conn(0x7fb1b4065930 msgr2=0x7fb1b4153420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 shutdown_connections 2023-11-06T03:13:38.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.325+0000 7fb1b8e31640 1 -- 172.21.15.157:0/262132120 wait complete. 2023-11-06T03:13:38.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:13:38.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 Processor -- start 2023-11-06T03:13:38.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 -- start start 2023-11-06T03:13:38.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f955c15cb80 0x7f955c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c15cf60 0x7f955c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c071830 con 0x7f955c0cc910 2023-11-06T03:13:38.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c071b10 con 0x7f955c15cb80 2023-11-06T03:13:38.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9562a47640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c071df0 con 0x7f955c15cf60 2023-11-06T03:13:38.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f955c15cb80 0x7f955c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955b7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c15cf60 0x7f955c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9560fe4640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f955c0cc910 0x7f955c0cfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:57848/0 (socket says 172.21.15.157:57848) 2023-11-06T03:13:38.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f9560fe4640 1 -- 172.21.15.157:0/2914542148 learned_addr learned my addr 172.21.15.157:0/2914542148 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:38.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955bfff640 1 -- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c15cf60 msgr2=0x7f955c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:38.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955bfff640 1 --2- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c15cf60 0x7f955c0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955bfff640 1 -- 172.21.15.157:0/2914542148 >> v1:172.21.15.195:6789/0 conn(0x7f955c0cc910 legacy=0x7f955c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.461+0000 7f955bfff640 1 -- 172.21.15.157:0/2914542148 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0ccfe0 con 0x7f955c15cb80 2023-11-06T03:13:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f955bfff640 1 --2- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cb80 0x7f955c0cafd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097c0 tx=0x7f954c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fded4a69d3afd8f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f955affd640 1 -- 172.21.15.157:0/2914542148 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0ad640 con 0x7f955c15cb80 2023-11-06T03:13:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f955affd640 1 -- 172.21.15.157:0/2914542148 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c0ad7a0 con 0x7f955c15cb80 2023-11-06T03:13:38.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f955affd640 1 -- 172.21.15.157:0/2914542148 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0adb30 con 0x7f955c15cb80 2023-11-06T03:13:38.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cb80 msgr2=0x7f955c0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 --2- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cb80 0x7f955c0cafd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097c0 tx=0x7f954c01c480 comp rx=0 tx=0).stop 2023-11-06T03:13:38.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- 172.21.15.157:0/2914542148 shutdown_connections 2023-11-06T03:13:38.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 --2- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c15cf60 0x7f955c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 --2- 172.21.15.157:0/2914542148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cb80 0x7f955c0cafd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- 172.21.15.157:0/2914542148 >> 172.21.15.157:0/2914542148 conn(0x7f955c06d560 msgr2=0x7f955c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- 172.21.15.157:0/2914542148 shutdown_connections 2023-11-06T03:13:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- 172.21.15.157:0/2914542148 wait complete. 2023-11-06T03:13:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 Processor -- start 2023-11-06T03:13:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.465+0000 7f9562a47640 1 -- start start 2023-11-06T03:13:38.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c0cc910 0x7f955c1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 0x7f955c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0c8f90 con 0x7f955c15cb80 2023-11-06T03:13:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0ca610 con 0x7f955c15cf60 2023-11-06T03:13:38.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c14c1c0 con 0x7f955c0cc910 2023-11-06T03:13:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38060/0 (socket says 172.21.15.157:38060) 2023-11-06T03:13:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 -- 172.21.15.157:0/3127733458 learned_addr learned my addr 172.21.15.157:0/3127733458 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955bfff640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c0cc910 0x7f955c1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9560fe4640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 0x7f955c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 -- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c0cc910 msgr2=0x7f955c1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c0cc910 0x7f955c1507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 -- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 msgr2=0x7f955c14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 0x7f955c14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 -- 172.21.15.157:0/3127733458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c009420 con 0x7f955c15cb80 2023-11-06T03:13:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9560fe4640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 0x7f955c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955b7fe640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f955000e930 tx=0x7f955000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550009690 con 0x7f955c15cb80 2023-11-06T03:13:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955000ca00 con 0x7f955c15cb80 2023-11-06T03:13:38.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c14c4a0 con 0x7f955c15cb80 2023-11-06T03:13:38.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955000cd70 con 0x7f955c15cb80 2023-11-06T03:13:38.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c150d10 con 0x7f955c15cb80 2023-11-06T03:13:38.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c06bf40 con 0x7f955c15cb80 2023-11-06T03:13:38.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9550013470 con 0x7f955c15cb80 2023-11-06T03:13:38.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 0x7f9548067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9550083220 con 0x7f955c15cb80 2023-11-06T03:13:38.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.469+0000 7f955bfff640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 0x7f9548067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.473+0000 7f955bfff640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 0x7f9548067f20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f954c0af000 tx=0x7f954c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.473+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9550050e90 con 0x7f955c15cb80 2023-11-06T03:13:38.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f955c072750 con 0x7f955c15cb80 2023-11-06T03:13:38.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9558ff9640 1 -- 172.21.15.157:0/3127733458 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) v1 ==== 84+0+126 (secure 0 0 0) 0x7f95500544b0 con 0x7f955c15cb80 2023-11-06T03:13:38.625 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:38.625 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:13:38.626 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:38.626 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:13:38.626 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:13:38.626 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:38.626 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:13:38.627 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:13:38.627 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:13:38.627 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:38.627 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:13:38.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 msgr2=0x7f9548067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 0x7f9548067f20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f954c0af000 tx=0x7f954c0a7070 comp rx=0 tx=0).stop 2023-11-06T03:13:38.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 msgr2=0x7f955c14f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f955000e930 tx=0x7f955000edf0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 shutdown_connections 2023-11-06T03:13:38.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9548065a60 0x7f9548067f20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f955c15cf60 0x7f955c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f955c15cb80 0x7f955c14f810 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 --2- 172.21.15.157:0/3127733458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f955c0cc910 0x7f955c1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 >> 172.21.15.157:0/3127733458 conn(0x7f955c06d560 msgr2=0x7f955c0c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 shutdown_connections 2023-11-06T03:13:38.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.629+0000 7f9562a47640 1 -- 172.21.15.157:0/3127733458 wait complete. 2023-11-06T03:13:38.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:13:38.645 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:13:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 Processor -- start 2023-11-06T03:13:38.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 -- start start 2023-11-06T03:13:38.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fba30155310 0x7fba3015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba3006c550 0x7fba3015b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba300cb100 con 0x7fba3006c170 2023-11-06T03:13:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300ce0a0 con 0x7fba3006c550 2023-11-06T03:13:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba37245640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3015bef0 con 0x7fba30155310 2023-11-06T03:13:38.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba3006c550 0x7fba3015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba3006c550 0x7fba3015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54990/0 (socket says 172.21.15.157:54990) 2023-11-06T03:13:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 -- 172.21.15.157:0/2395239681 learned_addr learned my addr 172.21.15.157:0/2395239681 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba2ffff640 1 --2- 172.21.15.157:0/2395239681 >> v2:172.21.15.195:3300/0 conn(0x7fba30155310 0x7fba3015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 -- 172.21.15.157:0/2395239681 >> v2:172.21.15.195:3300/0 conn(0x7fba30155310 msgr2=0x7fba3015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:38.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 --2- 172.21.15.157:0/2395239681 >> v2:172.21.15.195:3300/0 conn(0x7fba30155310 0x7fba3015b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 -- 172.21.15.157:0/2395239681 >> v1:172.21.15.195:6789/0 conn(0x7fba3006c170 legacy=0x7fba30154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.773+0000 7fba357e2640 1 -- 172.21.15.157:0/2395239681 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300cb350 con 0x7fba3006c550 2023-11-06T03:13:38.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba357e2640 1 --2- 172.21.15.157:0/2395239681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3006c550 0x7fba3015b690 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fba300cc2e0 tx=0x7fba2001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b8d4b0b04861c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba2f7fe640 1 -- 172.21.15.157:0/2395239681 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200aa430 con 0x7fba3006c550 2023-11-06T03:13:38.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba2f7fe640 1 -- 172.21.15.157:0/2395239681 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba200aa590 con 0x7fba3006c550 2023-11-06T03:13:38.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba2f7fe640 1 -- 172.21.15.157:0/2395239681 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200aa920 con 0x7fba3006c550 2023-11-06T03:13:38.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- 172.21.15.157:0/2395239681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3006c550 msgr2=0x7fba3015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 --2- 172.21.15.157:0/2395239681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3006c550 0x7fba3015b690 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fba300cc2e0 tx=0x7fba2001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- 172.21.15.157:0/2395239681 shutdown_connections 2023-11-06T03:13:38.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 --2- 172.21.15.157:0/2395239681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3006c550 0x7fba3015b690 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 --2- 172.21.15.157:0/2395239681 >> v2:172.21.15.195:3300/0 conn(0x7fba30155310 0x7fba3015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- 172.21.15.157:0/2395239681 >> 172.21.15.157:0/2395239681 conn(0x7fba300b43b0 msgr2=0x7fba30065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:38.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- 172.21.15.157:0/2395239681 shutdown_connections 2023-11-06T03:13:38.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- 172.21.15.157:0/2395239681 wait complete. 2023-11-06T03:13:38.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 Processor -- start 2023-11-06T03:13:38.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.777+0000 7fba37245640 1 -- start start 2023-11-06T03:13:38.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 0x7fba30154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3006c550 0x7fba30149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3014a180 con 0x7fba30155310 2023-11-06T03:13:38.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30151d60 con 0x7fba3006c170 2023-11-06T03:13:38.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba301531b0 con 0x7fba3006c550 2023-11-06T03:13:38.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54994/0 (socket says 172.21.15.157:54994) 2023-11-06T03:13:38.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 -- 172.21.15.157:0/122416897 learned_addr learned my addr 172.21.15.157:0/122416897 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:38.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba34fe1640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 0x7fba30154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2ffff640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3006c550 0x7fba30149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 -- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3006c550 msgr2=0x7fba30149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3006c550 0x7fba30149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 -- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 msgr2=0x7fba30154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:38.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 0x7fba30154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:38.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 -- 172.21.15.157:0/122416897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200094d0 con 0x7fba30155310 2023-11-06T03:13:38.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba34fe1640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 0x7fba30154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:38.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba357e2640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fba200ac040 tx=0x7fba2001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200aa430 con 0x7fba30155310 2023-11-06T03:13:38.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba200b3800 con 0x7fba30155310 2023-11-06T03:13:38.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba300cbfb0 con 0x7fba30155310 2023-11-06T03:13:38.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200b3bb0 con 0x7fba30155310 2023-11-06T03:13:38.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba30153450 con 0x7fba30155310 2023-11-06T03:13:38.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3004e050 con 0x7fba30155310 2023-11-06T03:13:38.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fba200b3d10 con 0x7fba30155310 2023-11-06T03:13:38.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 0x7fba1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:38.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba2011ce20 con 0x7fba30155310 2023-11-06T03:13:38.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.781+0000 7fba34fe1640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 0x7fba1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:38.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.785+0000 7fba34fe1640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 0x7fba1c067f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fba28000f70 tx=0x7fba28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:38.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.785+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fba200ea930 con 0x7fba30155310 2023-11-06T03:13:38.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:38.929+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fba300ccbb0 con 0x7fba30155310 2023-11-06T03:13:39.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.025+0000 7fba2d7fa640 1 -- 172.21.15.157:0/122416897 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) v1 ==== 138+0+0 (secure 0 0 0) 0x7fba200edf50 con 0x7fba30155310 2023-11-06T03:13:39.023 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:13:39.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.029+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 msgr2=0x7fba1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:39.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.029+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 0x7fba1c067f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fba28000f70 tx=0x7fba28009040 comp rx=0 tx=0).stop 2023-11-06T03:13:39.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.029+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 msgr2=0x7fba30151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:39.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.029+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fba200ac040 tx=0x7fba2001ad10 comp rx=0 tx=0).stop 2023-11-06T03:13:39.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 shutdown_connections 2023-11-06T03:13:39.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fba1c065a60 0x7fba1c067f20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba30155310 0x7fba30151820 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fba3006c550 0x7fba30149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 --2- 172.21.15.157:0/122416897 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fba3006c170 0x7fba30154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 >> 172.21.15.157:0/122416897 conn(0x7fba300b43b0 msgr2=0x7fba3015c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:39.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 shutdown_connections 2023-11-06T03:13:39.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.033+0000 7fba37245640 1 -- 172.21.15.157:0/122416897 wait complete. 2023-11-06T03:13:39.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:13:39.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.169+0000 7fce12352640 1 Processor -- start 2023-11-06T03:13:39.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.169+0000 7fce12352640 1 -- start start 2023-11-06T03:13:39.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fce0c1551e0 0x7fce0c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce0c15e5e0 0x7fce0c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 0x7fce0c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c15f580 con 0x7fce0c15e5e0 2023-11-06T03:13:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c15efc0 con 0x7fce0c1551e0 2023-11-06T03:13:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c15f2a0 con 0x7fce0c15ea90 2023-11-06T03:13:39.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce0c15e5e0 0x7fce0c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce0c15e5e0 0x7fce0c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55006/0 (socket says 172.21.15.157:55006) 2023-11-06T03:13:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 -- 172.21.15.157:0/1869914639 learned_addr learned my addr 172.21.15.157:0/1869914639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0bfff640 1 --2- 172.21.15.157:0/1869914639 >> v2:172.21.15.195:3300/0 conn(0x7fce0c1551e0 0x7fce0c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce108ef640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 0x7fce0c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 -- 172.21.15.157:0/1869914639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 msgr2=0x7fce0c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 0x7fce0c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 -- 172.21.15.157:0/1869914639 >> v2:172.21.15.195:3300/0 conn(0x7fce0c1551e0 msgr2=0x7fce0c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:39.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/1869914639 >> v2:172.21.15.195:3300/0 conn(0x7fce0c1551e0 0x7fce0c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 -- 172.21.15.157:0/1869914639 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c06f5a0 con 0x7fce0c15e5e0 2023-11-06T03:13:39.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce108ef640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 0x7fce0c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c15e5e0 0x7fce0c0b3410 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fce0c066830 tx=0x7fcdf401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e418cdd4a6e82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0affd640 1 -- 172.21.15.157:0/1869914639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40ab620 con 0x7fce0c15e5e0 2023-11-06T03:13:39.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0affd640 1 -- 172.21.15.157:0/1869914639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf40ab780 con 0x7fce0c15e5e0 2023-11-06T03:13:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce0affd640 1 -- 172.21.15.157:0/1869914639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf40abb10 con 0x7fce0c15e5e0 2023-11-06T03:13:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 -- 172.21.15.157:0/1869914639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c15e5e0 msgr2=0x7fce0c0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.173+0000 7fce12352640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c15e5e0 0x7fce0c0b3410 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fce0c066830 tx=0x7fcdf401c440 comp rx=0 tx=0).stop 2023-11-06T03:13:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/1869914639 shutdown_connections 2023-11-06T03:13:39.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15ea90 0x7fce0c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- 172.21.15.157:0/1869914639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c15e5e0 0x7fce0c0b3410 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- 172.21.15.157:0/1869914639 >> v2:172.21.15.195:3300/0 conn(0x7fce0c1551e0 0x7fce0c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/1869914639 >> 172.21.15.157:0/1869914639 conn(0x7fce0c065930 msgr2=0x7fce0c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/1869914639 shutdown_connections 2023-11-06T03:13:39.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/1869914639 wait complete. 2023-11-06T03:13:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 Processor -- start 2023-11-06T03:13:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- start start 2023-11-06T03:13:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 0x7fce0c1f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 0x7fce0c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c1f7300 con 0x7fce0c1551e0 2023-11-06T03:13:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c15ab80 con 0x7fce0c15ea90 2023-11-06T03:13:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c1f6d40 con 0x7fce0c15e5e0 2023-11-06T03:13:39.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55012/0 (socket says 172.21.15.157:55012) 2023-11-06T03:13:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 -- 172.21.15.157:0/3904773796 learned_addr learned my addr 172.21.15.157:0/3904773796 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce108ef640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 0x7fce0c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 0x7fce0c1f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 -- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 msgr2=0x7fce0c1f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 0x7fce0c1f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 -- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 msgr2=0x7fce0c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 0x7fce0c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 -- 172.21.15.157:0/3904773796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40094d0 con 0x7fce0c1551e0 2023-11-06T03:13:39.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 0x7fce0c1f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce108ef640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 0x7fce0c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce0bfff640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007e60 tx=0x7fcdfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc014620 con 0x7fce0c1551e0 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc00ee90 con 0x7fce0c1551e0 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0149c0 con 0x7fce0c1551e0 2023-11-06T03:13:39.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c1f7020 con 0x7fce0c1551e0 2023-11-06T03:13:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c14bd00 con 0x7fce0c1551e0 2023-11-06T03:13:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.177+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c071730 con 0x7fce0c1551e0 2023-11-06T03:13:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.181+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcdfc014b20 con 0x7fce0c1551e0 2023-11-06T03:13:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.181+0000 7fce08ff9640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 0x7fcdec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:39.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.181+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fcdfc083a00 con 0x7fce0c1551e0 2023-11-06T03:13:39.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.181+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 0x7fcdec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:39.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.181+0000 7fce0b7fe640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 0x7fcdec067f20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:39.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.185+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcdfc0514d0 con 0x7fce0c1551e0 2023-11-06T03:13:39.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:39.329+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce0c0cb110 con 0x7fce0c1551e0 2023-11-06T03:13:40.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.041+0000 7fce08ff9640 1 -- 172.21.15.157:0/3904773796 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcdfc009a20 con 0x7fce0c1551e0 2023-11-06T03:13:40.039 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:13:40.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 msgr2=0x7fcdec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 0x7fcdec067f20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40af000 tx=0x7fcdf4008040 comp rx=0 tx=0).stop 2023-11-06T03:13:40.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 msgr2=0x7fce0c15a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007e60 tx=0x7fcdfc00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 shutdown_connections 2023-11-06T03:13:40.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcdec065a60 0x7fcdec067f20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce0c15ea90 0x7fce0c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce0c15e5e0 0x7fce0c1f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 --2- 172.21.15.157:0/3904773796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce0c1551e0 0x7fce0c15a640 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 >> 172.21.15.157:0/3904773796 conn(0x7fce0c065930 msgr2=0x7fce0c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 shutdown_connections 2023-11-06T03:13:40.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.053+0000 7fce12352640 1 -- 172.21.15.157:0/3904773796 wait complete. 2023-11-06T03:13:40.080 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:13:40.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 Processor -- start 2023-11-06T03:13:40.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 -- start start 2023-11-06T03:13:40.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f49c015ce80 0x7f49c00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00d2920 con 0x7f49c00d00b0 2023-11-06T03:13:40.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00d18a0 con 0x7f49c015caa0 2023-11-06T03:13:40.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49c5217640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00d0520 con 0x7f49c015ce80 2023-11-06T03:13:40.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f49c00d00b0 0x7f49c00b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54942/0 (socket says 172.21.15.157:54942) 2023-11-06T03:13:40.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 -- 172.21.15.157:0/3061002181 learned_addr learned my addr 172.21.15.157:0/3061002181 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:40.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974392629 0 0) 0x7f49c00d2920 con 0x7f49c00d00b0 2023-11-06T03:13:40.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003530 con 0x7f49c00d00b0 2023-11-06T03:13:40.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49b659c640 1 --2- 172.21.15.157:0/3061002181 >> v2:172.21.15.195:3300/0 conn(0x7f49c015ce80 0x7f49c00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894372111 0 0) 0x7f49ac003530 con 0x7f49c00d00b0 2023-11-06T03:13:40.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 >> v2:172.21.15.195:3300/0 conn(0x7f49c015ce80 msgr2=0x7f49c00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 --2- 172.21.15.157:0/3061002181 >> v2:172.21.15.195:3300/0 conn(0x7f49c015ce80 0x7f49c00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 >> v1:172.21.15.195:6789/0 conn(0x7f49c015caa0 legacy=0x7f49c00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00d1b60 con 0x7f49c00d00b0 2023-11-06T03:13:40.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f49b8001a00 con 0x7f49c00d00b0 2023-11-06T03:13:40.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 >> v1:172.21.15.157:6789/0 conn(0x7f49c00d00b0 legacy=0x7f49c00b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49ac003980 0x7f49ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49ac0047d0 0x7f49ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00d0520 con 0x7f49ac005190 2023-11-06T03:13:40.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c030 con 0x7f49ac003980 2023-11-06T03:13:40.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c1f0 con 0x7f49ac0047d0 2023-11-06T03:13:40.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bed9d640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49ac003980 0x7f49ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49b659c640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49ac0047d0 0x7f49ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 -- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49ac0047d0 msgr2=0x7f49ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49ac0047d0 0x7f49ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 -- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49ac003980 msgr2=0x7f49ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49ac003980 0x7f49ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.205+0000 7f49bf59e640 1 -- 172.21.15.157:0/3061002181 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003530 con 0x7f49ac005190 2023-11-06T03:13:40.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49bf59e640 1 -- 172.21.15.157:0/3061002181 reap_dead start 2023-11-06T03:13:40.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49bf59e640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49ac00b990 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f49b800a830 tx=0x7f49b801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c003c2b6edbd01b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80072d0 con 0x7f49ac005190 2023-11-06T03:13:40.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b80a8070 con 0x7f49ac005190 2023-11-06T03:13:40.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49be59c640 1 -- 172.21.15.157:0/3061002181 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80076b0 con 0x7f49ac005190 2023-11-06T03:13:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 msgr2=0x7f49ac00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49ac00b990 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f49b800a830 tx=0x7f49b801d340 comp rx=0 tx=0).stop 2023-11-06T03:13:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- 172.21.15.157:0/3061002181 shutdown_connections 2023-11-06T03:13:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 --2- 172.21.15.157:0/3061002181 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49ac00b990 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- 172.21.15.157:0/3061002181 >> 172.21.15.157:0/3061002181 conn(0x7f49c0070790 msgr2=0x7f49c0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- 172.21.15.157:0/3061002181 shutdown_connections 2023-11-06T03:13:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- 172.21.15.157:0/3061002181 wait complete. 2023-11-06T03:13:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 Processor -- start 2023-11-06T03:13:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.209+0000 7f49c5217640 1 -- start start 2023-11-06T03:13:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49c015adf0 0x7f49c015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49c00c9cf0 0x7f49c00cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01e4ba0 con 0x7f49ac005190 2023-11-06T03:13:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c015ac90 con 0x7f49c00c9cf0 2023-11-06T03:13:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00ca540 con 0x7f49c015adf0 2023-11-06T03:13:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55028/0 (socket says 172.21.15.157:55028) 2023-11-06T03:13:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 -- 172.21.15.157:0/267403090 learned_addr learned my addr 172.21.15.157:0/267403090 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bf59e640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49c00c9cf0 0x7f49c00cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 -- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49c015adf0 msgr2=0x7f49c015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49c015adf0 0x7f49c015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 -- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49c00c9cf0 msgr2=0x7f49c00cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49c00c9cf0 0x7f49c00cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 -- 172.21.15.157:0/267403090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00ca820 con 0x7f49ac005190 2023-11-06T03:13:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49bed9d640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e3c0 tx=0x7f49b000e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00052e0 con 0x7f49ac005190 2023-11-06T03:13:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0005440 con 0x7f49ac005190 2023-11-06T03:13:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00cab00 con 0x7f49ac005190 2023-11-06T03:13:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0003820 con 0x7f49ac005190 2023-11-06T03:13:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00cb0c0 con 0x7f49ac005190 2023-11-06T03:13:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00cb280 con 0x7f49ac005190 2023-11-06T03:13:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f49b0003980 con 0x7f49ac005190 2023-11-06T03:13:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 0x7f49a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f49b00828c0 con 0x7f49ac005190 2023-11-06T03:13:40.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.213+0000 7f49b659c640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 0x7f49a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.217+0000 7f49b659c640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 0x7f49a0067f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b1cb0 tx=0x7f49a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.217+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f49b00502c0 con 0x7f49ac005190 2023-11-06T03:13:40.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.405+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f49c0072750 con 0x7f49ac005190 2023-11-06T03:13:40.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.421+0000 7f49b7fff640 1 -- 172.21.15.157:0/267403090 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v89) v1 ==== 174+0+0 (secure 0 0 0) 0x7f49b0019020 con 0x7f49ac005190 2023-11-06T03:13:40.420 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 17 and data pool 18 2023-11-06T03:13:40.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 msgr2=0x7f49a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 0x7f49a0067f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b1cb0 tx=0x7f49a8008040 comp rx=0 tx=0).stop 2023-11-06T03:13:40.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 msgr2=0x7f49c00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e3c0 tx=0x7f49b000e880 comp rx=0 tx=0).stop 2023-11-06T03:13:40.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 shutdown_connections 2023-11-06T03:13:40.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f49a0065a60 0x7f49a0067f20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49c00c9cf0 0x7f49c00cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f49c015adf0 0x7f49c015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 --2- 172.21.15.157:0/267403090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49ac005190 0x7f49c00cb5e0 secure :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e3c0 tx=0x7f49b000e880 comp rx=0 tx=0).stop 2023-11-06T03:13:40.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.429+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 >> 172.21.15.157:0/267403090 conn(0x7f49c0070790 msgr2=0x7f49c00c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.433+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 shutdown_connections 2023-11-06T03:13:40.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.433+0000 7f49c5217640 1 -- 172.21.15.157:0/267403090 wait complete. 2023-11-06T03:13:40.455 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:40.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.569+0000 7f29c688b640 1 Processor -- start 2023-11-06T03:13:40.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.569+0000 7f29c688b640 1 -- start start 2023-11-06T03:13:40.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29c688b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c01599c0 0x7f29c00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29c688b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29c00b2af0 0x7f29c0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29c688b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c015e440 con 0x7f29c006d000 2023-11-06T03:13:40.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29c688b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c006c240 con 0x7f29c01599c0 2023-11-06T03:13:40.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29c688b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c006c520 con 0x7f29c00b2af0 2023-11-06T03:13:40.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c01599c0 0x7f29c00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29c01599c0 0x7f29c00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55034/0 (socket says 172.21.15.157:55034) 2023-11-06T03:13:40.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 -- 172.21.15.157:0/471016384 learned_addr learned my addr 172.21.15.157:0/471016384 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:40.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 -- 172.21.15.157:0/471016384 >> v2:172.21.15.195:3300/0 conn(0x7f29c00b2af0 msgr2=0x7f29c0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 --2- 172.21.15.157:0/471016384 >> v2:172.21.15.195:3300/0 conn(0x7f29c00b2af0 0x7f29c0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 -- 172.21.15.157:0/471016384 >> v1:172.21.15.195:6789/0 conn(0x7f29c006d000 legacy=0x7f29c00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 -- 172.21.15.157:0/471016384 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c006c800 con 0x7f29c01599c0 2023-11-06T03:13:40.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29bffff640 1 --2- 172.21.15.157:0/471016384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c01599c0 0x7f29c00ae000 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29c00b19a0 tx=0x7f29b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8a5037250a6835 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29beffd640 1 -- 172.21.15.157:0/471016384 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00aa610 con 0x7f29c01599c0 2023-11-06T03:13:40.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29beffd640 1 -- 172.21.15.157:0/471016384 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00a6040 con 0x7f29c01599c0 2023-11-06T03:13:40.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.573+0000 7f29beffd640 1 -- 172.21.15.157:0/471016384 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00aa990 con 0x7f29c01599c0 2023-11-06T03:13:40.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/471016384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c01599c0 msgr2=0x7f29c00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- 172.21.15.157:0/471016384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c01599c0 0x7f29c00ae000 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f29c00b19a0 tx=0x7f29b001c490 comp rx=0 tx=0).stop 2023-11-06T03:13:40.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/471016384 shutdown_connections 2023-11-06T03:13:40.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- 172.21.15.157:0/471016384 >> v2:172.21.15.195:3300/0 conn(0x7f29c00b2af0 0x7f29c0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- 172.21.15.157:0/471016384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c01599c0 0x7f29c00ae000 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/471016384 >> 172.21.15.157:0/471016384 conn(0x7f29c0065930 msgr2=0x7f29c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/471016384 shutdown_connections 2023-11-06T03:13:40.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/471016384 wait complete. 2023-11-06T03:13:40.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 Processor -- start 2023-11-06T03:13:40.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- start start 2023-11-06T03:13:40.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29c00b2af0 0x7f29c014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29c01599c0 0x7f29c014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0139fd0 con 0x7f29c006d000 2023-11-06T03:13:40.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cbf20 con 0x7f29c01599c0 2023-11-06T03:13:40.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0137740 con 0x7f29c00b2af0 2023-11-06T03:13:40.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55040/0 (socket says 172.21.15.157:55040) 2023-11-06T03:13:40.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 -- 172.21.15.157:0/1698684476 learned_addr learned my addr 172.21.15.157:0/1698684476 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:40.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 -- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29c00b2af0 msgr2=0x7f29c014bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29c00b2af0 0x7f29c014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 -- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29c01599c0 msgr2=0x7f29c014c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29c01599c0 0x7f29c014c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 -- 172.21.15.157:0/1698684476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00094d0 con 0x7f29c006d000 2023-11-06T03:13:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bffff640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f29b001a0c0 tx=0x7f29b001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00bd5f0 con 0x7f29c006d000 2023-11-06T03:13:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00bd770 con 0x7f29c006d000 2023-11-06T03:13:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c0139cf0 con 0x7f29c006d000 2023-11-06T03:13:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00bdb20 con 0x7f29c006d000 2023-11-06T03:13:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.577+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c01373d0 con 0x7f29c006d000 2023-11-06T03:13:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0136f50 con 0x7f29c006d000 2023-11-06T03:13:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29b00b33f0 con 0x7f29c006d000 2023-11-06T03:13:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29bcff9640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 0x7f29a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29b011d780 con 0x7f29c006d000 2023-11-06T03:13:40.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29bf7fe640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 0x7f29a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.581+0000 7f29bf7fe640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 0x7f29a0067f20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ad9c0 tx=0x7f29b8007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.585+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f29c0136f50 con 0x7f29c006d000 2023-11-06T03:13:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.741+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29c0136f50 con 0x7f29c006d000 2023-11-06T03:13:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.741+0000 7f29bcff9640 1 -- 172.21.15.157:0/1698684476 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f29c0136f50 con 0x7f29c006d000 2023-11-06T03:13:40.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:40.739 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":124,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:40.409734+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 msgr2=0x7f29a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 0x7f29a0067f20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ad9c0 tx=0x7f29b8007d20 comp rx=0 tx=0).stop 2023-11-06T03:13:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 msgr2=0x7f29c0158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f29b001a0c0 tx=0x7f29b001acd0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 shutdown_connections 2023-11-06T03:13:40.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29a0065a60 0x7f29a0067f20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29c01599c0 0x7f29c014c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29c00b2af0 0x7f29c014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 --2- 172.21.15.157:0/1698684476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c006d000 0x7f29c0158230 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 >> 172.21.15.157:0/1698684476 conn(0x7f29c0065930 msgr2=0x7f29c0070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 shutdown_connections 2023-11-06T03:13:40.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.745+0000 7f29c688b640 1 -- 172.21.15.157:0/1698684476 wait complete. 2023-11-06T03:13:40.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:40.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 Processor -- start 2023-11-06T03:13:40.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 -- start start 2023-11-06T03:13:40.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0146370 0x7fe5e00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5e00ce800 0x7fe5e00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 0x7fe5e006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00cf1a0 con 0x7fe5e00ce800 2023-11-06T03:13:40.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e006c3d0 con 0x7fe5e006b6a0 2023-11-06T03:13:40.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5e4ac8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e006f5a0 con 0x7fe5e0146370 2023-11-06T03:13:40.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5e00ce800 0x7fe5e00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe5e00ce800 0x7fe5e00cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55044/0 (socket says 172.21.15.157:55044) 2023-11-06T03:13:40.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/2681756092 learned_addr learned my addr 172.21.15.157:0/2681756092 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:40.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ded9d640 1 --2- 172.21.15.157:0/2681756092 >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 0x7fe5e006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5de59c640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0146370 0x7fe5e00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/2681756092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0146370 msgr2=0x7fe5e00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0146370 0x7fe5e00b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/2681756092 >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 msgr2=0x7fe5e006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:40.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/2681756092 >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 0x7fe5e006bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/2681756092 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e006f880 con 0x7fe5e00ce800 2023-11-06T03:13:40.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.881+0000 7fe5ded9d640 1 --2- 172.21.15.157:0/2681756092 >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 0x7fe5e006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:40.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e00cec60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe5e006d360 tx=0x7fe5d401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16690809b979c988 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5dd59a640 1 -- 172.21.15.157:0/2681756092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40ab600 con 0x7fe5e00ce800 2023-11-06T03:13:40.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5dd59a640 1 -- 172.21.15.157:0/2681756092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d40ab760 con 0x7fe5e00ce800 2023-11-06T03:13:40.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5dd59a640 1 -- 172.21.15.157:0/2681756092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40abaf0 con 0x7fe5e00ce800 2023-11-06T03:13:40.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/2681756092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 msgr2=0x7fe5e00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e00cec60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe5e006d360 tx=0x7fe5d401c440 comp rx=0 tx=0).stop 2023-11-06T03:13:40.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/2681756092 shutdown_connections 2023-11-06T03:13:40.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/2681756092 >> v2:172.21.15.195:3300/0 conn(0x7fe5e006b6a0 0x7fe5e006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e00cec60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/2681756092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0146370 0x7fe5e00b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/2681756092 >> 172.21.15.157:0/2681756092 conn(0x7fe5e0065930 msgr2=0x7fe5e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:40.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/2681756092 shutdown_connections 2023-11-06T03:13:40.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/2681756092 wait complete. 2023-11-06T03:13:40.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 Processor -- start 2023-11-06T03:13:40.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- start start 2023-11-06T03:13:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 0x7fe5e01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0146370 0x7fe5e0145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0155760 con 0x7fe5e00ce800 2023-11-06T03:13:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0157740 con 0x7fe5e0146370 2023-11-06T03:13:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5e4ac8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0157a20 con 0x7fe5e006b6a0 2023-11-06T03:13:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.885+0000 7fe5ddd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55046/0 (socket says 172.21.15.157:55046) 2023-11-06T03:13:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/3077220647 learned_addr learned my addr 172.21.15.157:0/3077220647 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ded9d640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0146370 0x7fe5e0145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5de59c640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 0x7fe5e01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 msgr2=0x7fe5e01570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 0x7fe5e01570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0146370 msgr2=0x7fe5e0145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:40.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0146370 0x7fe5e0145340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 -- 172.21.15.157:0/3077220647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40094d0 con 0x7fe5e00ce800 2023-11-06T03:13:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5de59c640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 0x7fe5e01570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ddd9b640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5d401aaf0 tx=0x7fe5d401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40b4800 con 0x7fe5e00ce800 2023-11-06T03:13:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d40b4960 con 0x7fe5e00ce800 2023-11-06T03:13:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40b4cf0 con 0x7fe5e00ce800 2023-11-06T03:13:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e0157d00 con 0x7fe5e00ce800 2023-11-06T03:13:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e01582c0 con 0x7fe5e00ce800 2023-11-06T03:13:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e01584b0 con 0x7fe5e00ce800 2023-11-06T03:13:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe5d40c7440 con 0x7fe5e00ce800 2023-11-06T03:13:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 0x7fe5c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5d411de40 con 0x7fe5e00ce800 2023-11-06T03:13:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5de59c640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 0x7fe5c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.889+0000 7fe5de59c640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 0x7fe5c0067f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006fd0 tx=0x7fe5d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:40.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:40.893+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe5d40eb730 con 0x7fe5e00ce800 2023-11-06T03:13:41.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.045+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e0157b60 con 0x7fe5e00ce800 2023-11-06T03:13:41.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.049+0000 7fe5ceffd640 1 -- 172.21.15.157:0/3077220647 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17954 (secure 0 0 0) 0x7fe5e0157b60 con 0x7fe5e00ce800 2023-11-06T03:13:41.045 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:41.045 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":124,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:40.409734+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:41.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 msgr2=0x7fe5c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 0x7fe5c0067f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006fd0 tx=0x7fe5d0008040 comp rx=0 tx=0).stop 2023-11-06T03:13:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 msgr2=0x7fe5e0155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5d401aaf0 tx=0x7fe5d401ac80 comp rx=0 tx=0).stop 2023-11-06T03:13:41.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 shutdown_connections 2023-11-06T03:13:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5c0065a60 0x7fe5c0067f20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0146370 0x7fe5e0145340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00ce800 0x7fe5e0155220 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 --2- 172.21.15.157:0/3077220647 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e006b6a0 0x7fe5e01570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 >> 172.21.15.157:0/3077220647 conn(0x7fe5e0065930 msgr2=0x7fe5e00b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 shutdown_connections 2023-11-06T03:13:41.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.053+0000 7fe5e4ac8640 1 -- 172.21.15.157:0/3077220647 wait complete. 2023-11-06T03:13:41.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:13:41.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.181+0000 7f5867dc6640 1 Processor -- start 2023-11-06T03:13:41.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.181+0000 7f5867dc6640 1 -- start start 2023-11-06T03:13:41.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5867dc6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 0x7f58600ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5867dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 0x7f58600b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5867dc6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586015e440 con 0x7f586015dea0 2023-11-06T03:13:41.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5867dc6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586015adb0 con 0x7f58601599c0 2023-11-06T03:13:41.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5867dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586006c240 con 0x7f586006d000 2023-11-06T03:13:41.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 0x7f58600b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865b62640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 0x7f58600ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5866363640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f586015dea0 0x7f5860070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56766/0 (socket says 172.21.15.157:56766) 2023-11-06T03:13:41.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5866363640 1 -- 172.21.15.157:0/3357439418 learned_addr learned my addr 172.21.15.157:0/3357439418 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:41.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5864b60640 1 -- 172.21.15.157:0/3357439418 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097603187 0 0) 0x7f586015e440 con 0x7f586015dea0 2023-11-06T03:13:41.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5864b60640 1 -- 172.21.15.157:0/3357439418 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584c003530 con 0x7f586015dea0 2023-11-06T03:13:41.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 -- 172.21.15.157:0/3357439418 >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 msgr2=0x7f58600ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:41.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 --2- 172.21.15.157:0/3357439418 >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 0x7f58600ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 -- 172.21.15.157:0/3357439418 >> v1:172.21.15.195:6789/0 conn(0x7f586015dea0 legacy=0x7f5860070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 -- 172.21.15.157:0/3357439418 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586006c520 con 0x7f586006d000 2023-11-06T03:13:41.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865b62640 1 --2- 172.21.15.157:0/3357439418 >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 0x7f58600ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:13:41.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.185+0000 7f5865361640 1 --2- 172.21.15.157:0/3357439418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 0x7f58600b23e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58500097c0 tx=0x7f585001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14de74f3de270197 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5864b60640 1 -- 172.21.15.157:0/3357439418 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500ad660 con 0x7f586006d000 2023-11-06T03:13:41.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5864b60640 1 -- 172.21.15.157:0/3357439418 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58500ad7c0 con 0x7f586006d000 2023-11-06T03:13:41.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5864b60640 1 -- 172.21.15.157:0/3357439418 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500adb50 con 0x7f586006d000 2023-11-06T03:13:41.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- 172.21.15.157:0/3357439418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 msgr2=0x7f58600b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- 172.21.15.157:0/3357439418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 0x7f58600b23e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58500097c0 tx=0x7f585001c450 comp rx=0 tx=0).stop 2023-11-06T03:13:41.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- 172.21.15.157:0/3357439418 shutdown_connections 2023-11-06T03:13:41.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- 172.21.15.157:0/3357439418 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586006d000 0x7f58600b23e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- 172.21.15.157:0/3357439418 >> v2:172.21.15.195:3300/0 conn(0x7f58601599c0 0x7f58600ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- 172.21.15.157:0/3357439418 >> 172.21.15.157:0/3357439418 conn(0x7f5860065930 msgr2=0x7f5860065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:41.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- 172.21.15.157:0/3357439418 shutdown_connections 2023-11-06T03:13:41.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- 172.21.15.157:0/3357439418 wait complete. 2023-11-06T03:13:41.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 Processor -- start 2023-11-06T03:13:41.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- start start 2023-11-06T03:13:41.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58601599c0 0x7f5860155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 0x7f58601678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600cc920 con 0x7f586006d000 2023-11-06T03:13:41.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600cd0e0 con 0x7f58601599c0 2023-11-06T03:13:41.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5867dc6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58600cd3c0 con 0x7f586015dea0 2023-11-06T03:13:41.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5865b62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5865b62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55048/0 (socket says 172.21.15.157:55048) 2023-11-06T03:13:41.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5865b62640 1 -- 172.21.15.157:0/384203908 learned_addr learned my addr 172.21.15.157:0/384203908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:41.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5865361640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58601599c0 0x7f5860155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.189+0000 7f5866363640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 0x7f58601678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 -- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 msgr2=0x7f58601678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 0x7f58601678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 -- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58601599c0 msgr2=0x7f5860155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58601599c0 0x7f5860155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 -- 172.21.15.157:0/384203908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009420 con 0x7f586006d000 2023-11-06T03:13:41.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5866363640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 0x7f58601678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:41.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865b62640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f585c00ebe0 tx=0x7f585c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00eee0 con 0x7f586006d000 2023-11-06T03:13:41.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00cc30 con 0x7f586006d000 2023-11-06T03:13:41.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5850009420 con 0x7f586006d000 2023-11-06T03:13:41.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c013470 con 0x7f586006d000 2023-11-06T03:13:41.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58600cd980 con 0x7f586006d000 2023-11-06T03:13:41.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5860072750 con 0x7f586006d000 2023-11-06T03:13:41.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f585c0135d0 con 0x7f586006d000 2023-11-06T03:13:41.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 0x7f5844067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f585c0835d0 con 0x7f586006d000 2023-11-06T03:13:41.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865361640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 0x7f5844067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.193+0000 7f5865361640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 0x7f5844067f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f58500af000 tx=0x7f5850008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.197+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f585c050e40 con 0x7f586006d000 2023-11-06T03:13:41.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.389+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5860164630 con 0x7f586006d000 2023-11-06T03:13:41.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f58567fc640 1 -- 172.21.15.157:0/384203908 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f585c054460 con 0x7f586006d000 2023-11-06T03:13:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 msgr2=0x7f5844067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 0x7f5844067f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f58500af000 tx=0x7f5850008040 comp rx=0 tx=0).stop 2023-11-06T03:13:41.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 msgr2=0x7f58600cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f585c00ebe0 tx=0x7f585c00c520 comp rx=0 tx=0).stop 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 shutdown_connections 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5844065a60 0x7f5844067f20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f586015dea0 0x7f58601678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58601599c0 0x7f5860155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 --2- 172.21.15.157:0/384203908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f586006d000 0x7f58600cc3e0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 >> 172.21.15.157:0/384203908 conn(0x7f5860065930 msgr2=0x7f58600b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 shutdown_connections 2023-11-06T03:13:41.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.417+0000 7f5867dc6640 1 -- 172.21.15.157:0/384203908 wait complete. 2023-11-06T03:13:41.433 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:13:41.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:13:41.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:13:41.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:13:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 Processor -- start 2023-11-06T03:13:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 -- start start 2023-11-06T03:13:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7ff80701b0 0x7f7ff806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80733a0 con 0x7f7ff8159ce0 2023-11-06T03:13:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80cc8d0 con 0x7f7ff8071150 2023-11-06T03:13:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ffde61640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80730c0 con 0x7f7ff80701b0 2023-11-06T03:13:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7ff8159ce0 0x7f7ff8070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54952/0 (socket says 172.21.15.157:54952) 2023-11-06T03:13:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff77fe640 1 -- 172.21.15.157:0/2902126966 learned_addr learned my addr 172.21.15.157:0/2902126966 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff7fff640 1 --2- 172.21.15.157:0/2902126966 >> v2:172.21.15.195:3300/0 conn(0x7f7ff80701b0 0x7f7ff806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181138008 0 0) 0x7f7ff80733a0 con 0x7f7ff8159ce0 2023-11-06T03:13:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe0003530 con 0x7f7ff8159ce0 2023-11-06T03:13:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487133670 0 0) 0x7f7fe0003530 con 0x7f7ff8159ce0 2023-11-06T03:13:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 >> v2:172.21.15.195:3300/0 conn(0x7f7ff80701b0 msgr2=0x7f7ff806fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 --2- 172.21.15.157:0/2902126966 >> v2:172.21.15.195:3300/0 conn(0x7f7ff80701b0 0x7f7ff806fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 >> v1:172.21.15.195:6789/0 conn(0x7f7ff8071150 legacy=0x7f7ff806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80cc5f0 con 0x7f7ff8159ce0 2023-11-06T03:13:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7fec001680 con 0x7f7ff8159ce0 2023-11-06T03:13:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff7fff640 1 --2- 172.21.15.157:0/2902126966 >> v2:172.21.15.195:3300/0 conn(0x7f7ff80701b0 0x7f7ff806fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:13:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 >> v1:172.21.15.157:6789/0 conn(0x7f7ff8159ce0 legacy=0x7f7ff8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fe0003980 0x7f7fe0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 0x7f7fe0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe0005190 0x7f7fe000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80730c0 con 0x7f7fe00047d0 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000c130 con 0x7f7fe0005190 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe000c2f0 con 0x7f7fe0003980 2023-11-06T03:13:41.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 0x7f7fe0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.557+0000 7f7ff7fff640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe0005190 0x7f7fe000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff77fe640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fe0003980 0x7f7fe0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fe0003980 msgr2=0x7f7fe0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fe0003980 0x7f7fe0003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe0005190 msgr2=0x7f7fe000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:41.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe0005190 0x7f7fe000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2902126966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0003530 con 0x7f7fe00047d0 2023-11-06T03:13:41.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff7fff640 1 -- 172.21.15.157:0/2902126966 reap_dead start 2023-11-06T03:13:41.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 0x7f7fe0004c50 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001780 tx=0x7f7fe801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cdfd74aa403805 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80052d0 con 0x7f7fe00047d0 2023-11-06T03:13:41.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80a7050 con 0x7f7fe00047d0 2023-11-06T03:13:41.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ff67fc640 1 -- 172.21.15.157:0/2902126966 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8003600 con 0x7f7fe00047d0 2023-11-06T03:13:41.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 -- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 msgr2=0x7f7fe0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 0x7f7fe0004c50 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001780 tx=0x7f7fe801c3b0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 -- 172.21.15.157:0/2902126966 shutdown_connections 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2902126966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fe00047d0 0x7f7fe0004c50 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 -- 172.21.15.157:0/2902126966 >> 172.21.15.157:0/2902126966 conn(0x7f7ff80cba30 msgr2=0x7f7ff80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 -- 172.21.15.157:0/2902126966 shutdown_connections 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.561+0000 7f7ffde61640 1 -- 172.21.15.157:0/2902126966 wait complete. 2023-11-06T03:13:41.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 Processor -- start 2023-11-06T03:13:41.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- start start 2023-11-06T03:13:41.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe00047d0 0x7f7ff814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ff80b4050 0x7f7ff80b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80c6ad0 con 0x7f7ff8150d10 2023-11-06T03:13:41.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8153a10 con 0x7f7fe00047d0 2023-11-06T03:13:41.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80c61a0 con 0x7f7ff80b4050 2023-11-06T03:13:41.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55060/0 (socket says 172.21.15.157:55060) 2023-11-06T03:13:41.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2374341866 learned_addr learned my addr 172.21.15.157:0/2374341866 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ff80b4050 msgr2=0x7f7ff80b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ff80b4050 0x7f7ff80b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe00047d0 msgr2=0x7f7ff814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe00047d0 0x7f7ff814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:41.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 -- 172.21.15.157:0/2374341866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80c6480 con 0x7f7ff8150d10 2023-11-06T03:13:41.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ff6ffd640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005040 tx=0x7f7fe0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80070c0 con 0x7f7ff8150d10 2023-11-06T03:13:41.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8007220 con 0x7f7ff8150d10 2023-11-06T03:13:41.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff80c6760 con 0x7f7ff8150d10 2023-11-06T03:13:41.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80075d0 con 0x7f7ff8150d10 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff80c8d50 con 0x7f7ff8150d10 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff80c68f0 con 0x7f7ff8150d10 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.565+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7fe80067e0 con 0x7f7ff8150d10 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.569+0000 7f7fdffff640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.569+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fe811d5d0 con 0x7f7ff8150d10 2023-11-06T03:13:41.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.569+0000 7f7ff77fe640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:41.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.569+0000 7f7ff77fe640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0017b0 tx=0x7f7fec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:41.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.573+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7fe80eaf50 con 0x7f7ff8150d10 2023-11-06T03:13:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:41.761+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7ff80c68f0 con 0x7f7ff8150d10 2023-11-06T03:13:42.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7fdffff640 1 -- 172.21.15.157:0/2374341866 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7fe80ee570 con 0x7f7ff8150d10 2023-11-06T03:13:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 msgr2=0x7f7fe4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0017b0 tx=0x7f7fec009040 comp rx=0 tx=0).stop 2023-11-06T03:13:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 msgr2=0x7f7ff80b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8005040 tx=0x7f7fe0005800 comp rx=0 tx=0).stop 2023-11-06T03:13:42.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 shutdown_connections 2023-11-06T03:13:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fe4065a60 0x7f7fe4067f20 secure :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0017b0 tx=0x7f7fec009040 comp rx=0 tx=0).stop 2023-11-06T03:13:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ff80b4050 0x7f7ff80b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ff8150d10 0x7f7ff80b3b10 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 --2- 172.21.15.157:0/2374341866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fe00047d0 0x7f7ff814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 >> 172.21.15.157:0/2374341866 conn(0x7f7ff80cba30 msgr2=0x7f7ff81578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 shutdown_connections 2023-11-06T03:13:42.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.433+0000 7f7ffde61640 1 -- 172.21.15.157:0/2374341866 wait complete. 2023-11-06T03:13:42.452 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:13:42.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.573+0000 7f4c0cac1640 1 Processor -- start 2023-11-06T03:13:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.573+0000 7f4c0cac1640 1 -- start start 2023-11-06T03:13:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 0x7f4c080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c0815ea90 0x7f4c080b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0815f580 con 0x7f4c081551e0 2023-11-06T03:13:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080704b0 con 0x7f4c0815ea90 2023-11-06T03:13:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0815efc0 con 0x7f4c0815e5e0 2023-11-06T03:13:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c0815ea90 0x7f4c080b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c0815ea90 0x7f4c080b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55076/0 (socket says 172.21.15.157:55076) 2023-11-06T03:13:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 -- 172.21.15.157:0/375929658 learned_addr learned my addr 172.21.15.157:0/375929658 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c05d9b640 1 --2- 172.21.15.157:0/375929658 >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 0x7f4c080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 -- 172.21.15.157:0/375929658 >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 msgr2=0x7f4c080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 --2- 172.21.15.157:0/375929658 >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 0x7f4c080b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 -- 172.21.15.157:0/375929658 >> v1:172.21.15.195:6789/0 conn(0x7f4c081551e0 legacy=0x7f4c0806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 -- 172.21.15.157:0/375929658 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0815f2a0 con 0x7f4c0815ea90 2023-11-06T03:13:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c05d9b640 1 --2- 172.21.15.157:0/375929658 >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 0x7f4c080b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c06d9d640 1 --2- 172.21.15.157:0/375929658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c080b3950 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4c08066880 tx=0x7f4c0001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9869fa2649d078e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0559a640 1 -- 172.21.15.157:0/375929658 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000ab600 con 0x7f4c0815ea90 2023-11-06T03:13:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0559a640 1 -- 172.21.15.157:0/375929658 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000ab760 con 0x7f4c0815ea90 2023-11-06T03:13:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0559a640 1 -- 172.21.15.157:0/375929658 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000abaf0 con 0x7f4c0815ea90 2023-11-06T03:13:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- 172.21.15.157:0/375929658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 msgr2=0x7f4c080b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/375929658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c080b3950 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4c08066880 tx=0x7f4c0001c400 comp rx=0 tx=0).stop 2023-11-06T03:13:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- 172.21.15.157:0/375929658 shutdown_connections 2023-11-06T03:13:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/375929658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c080b3950 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/375929658 >> v2:172.21.15.195:3300/0 conn(0x7f4c0815e5e0 0x7f4c080b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- 172.21.15.157:0/375929658 >> 172.21.15.157:0/375929658 conn(0x7f4c08065930 msgr2=0x7f4c08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- 172.21.15.157:0/375929658 shutdown_connections 2023-11-06T03:13:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.577+0000 7f4c0cac1640 1 -- 172.21.15.157:0/375929658 wait complete. 2023-11-06T03:13:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 Processor -- start 2023-11-06T03:13:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- start start 2023-11-06T03:13:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c081551e0 0x7f4c0814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c0815e5e0 0x7f4c081f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806be60 con 0x7f4c0815ea90 2023-11-06T03:13:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0815dde0 con 0x7f4c0815e5e0 2023-11-06T03:13:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806b530 con 0x7f4c081551e0 2023-11-06T03:13:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55080/0 (socket says 172.21.15.157:55080) 2023-11-06T03:13:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 -- 172.21.15.157:0/1192520763 learned_addr learned my addr 172.21.15.157:0/1192520763 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c05d9b640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c0815e5e0 0x7f4c081f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0659c640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c081551e0 0x7f4c0814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 -- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c081551e0 msgr2=0x7f4c0814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c081551e0 0x7f4c0814d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 -- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c0815e5e0 msgr2=0x7f4c081f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c0815e5e0 0x7f4c081f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 -- 172.21.15.157:0/1192520763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000094d0 con 0x7f4c0815ea90 2023-11-06T03:13:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c06d9d640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4c000ad040 tx=0x7f4c0001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b4800 con 0x7f4c0815ea90 2023-11-06T03:13:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000b4960 con 0x7f4c0815ea90 2023-11-06T03:13:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b4d10 con 0x7f4c0815ea90 2023-11-06T03:13:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0806b810 con 0x7f4c0815ea90 2023-11-06T03:13:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0814d8f0 con 0x7f4c0815ea90 2023-11-06T03:13:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0806c010 con 0x7f4c0815ea90 2023-11-06T03:13:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.581+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c000b3380 con 0x7f4c0815ea90 2023-11-06T03:13:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.585+0000 7f4bfeffd640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 0x7f4be8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.585+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c0011d870 con 0x7f4c0815ea90 2023-11-06T03:13:42.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.585+0000 7f4c0659c640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 0x7f4be8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.585+0000 7f4c0659c640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 0x7f4be8067f50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4c080ae4d0 tx=0x7f4bf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.589+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4c0806c010 con 0x7f4c0815ea90 2023-11-06T03:13:42.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.769+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c08072750 con 0x7f4c0815ea90 2023-11-06T03:13:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.769+0000 7f4bfeffd640 1 -- 172.21.15.157:0/1192520763 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f4c000eb180 con 0x7f4c0815ea90 2023-11-06T03:13:42.766 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:42.767 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:13:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:13:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 msgr2=0x7f4be8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 0x7f4be8067f50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4c080ae4d0 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2023-11-06T03:13:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 msgr2=0x7f4c0814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4c000ad040 tx=0x7f4c0001ac40 comp rx=0 tx=0).stop 2023-11-06T03:13:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 shutdown_connections 2023-11-06T03:13:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4be8065a90 0x7f4be8067f50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0815ea90 0x7f4c0814ed80 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4c0815e5e0 0x7f4c081f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 --2- 172.21.15.157:0/1192520763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4c081551e0 0x7f4c0814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 >> 172.21.15.157:0/1192520763 conn(0x7f4c08065930 msgr2=0x7f4c0815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 shutdown_connections 2023-11-06T03:13:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.773+0000 7f4c0cac1640 1 -- 172.21.15.157:0/1192520763 wait complete. 2023-11-06T03:13:42.784 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:13:42.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 Processor -- start 2023-11-06T03:13:42.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 -- start start 2023-11-06T03:13:42.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f21800cd730 0x7f21800cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800cc2b0 con 0x7f21800b2b10 2023-11-06T03:13:42.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800b01a0 con 0x7f218015ac80 2023-11-06T03:13:42.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f2185162640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800cfb10 con 0x7f21800cd730 2023-11-06T03:13:42.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f21800b2b10 0x7f21800ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54956/0 (socket says 172.21.15.157:54956) 2023-11-06T03:13:42.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 -- 172.21.15.157:0/3708617144 learned_addr learned my addr 172.21.15.157:0/3708617144 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:42.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217f59e640 1 --2- 172.21.15.157:0/3708617144 >> v2:172.21.15.195:3300/0 conn(0x7f21800cd730 0x7f21800cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675087289 0 0) 0x7f21800cc2b0 con 0x7f21800b2b10 2023-11-06T03:13:42.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2168003530 con 0x7f21800b2b10 2023-11-06T03:13:42.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327439936 0 0) 0x7f2168003530 con 0x7f21800b2b10 2023-11-06T03:13:42.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 >> v2:172.21.15.195:3300/0 conn(0x7f21800cd730 msgr2=0x7f21800cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:42.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 --2- 172.21.15.157:0/3708617144 >> v2:172.21.15.195:3300/0 conn(0x7f21800cd730 0x7f21800cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 >> v1:172.21.15.195:6789/0 conn(0x7f218015ac80 legacy=0x7f21800b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800cfdf0 con 0x7f21800b2b10 2023-11-06T03:13:42.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2174001680 con 0x7f21800b2b10 2023-11-06T03:13:42.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 >> v1:172.21.15.157:6789/0 conn(0x7f21800b2b10 legacy=0x7f21800ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168003980 0x7f2168003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168005190 0x7f216800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800cfb10 con 0x7f21680047d0 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216800c030 con 0x7f2168003980 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216800c1f0 con 0x7f2168005190 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217f59e640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168005190 0x7f216800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217ed9d640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168003980 0x7f2168003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 -- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168005190 msgr2=0x7f216800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:42.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168005190 0x7f216800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 -- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168003980 msgr2=0x7f2168003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:42.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168003980 0x7f2168003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217f59e640 1 -- 172.21.15.157:0/3708617144 reap_dead start 2023-11-06T03:13:42.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.897+0000 7f217e59c640 1 -- 172.21.15.157:0/3708617144 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168003530 con 0x7f21680047d0 2023-11-06T03:13:42.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217e59c640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 0x7f2168004c50 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f2174001060 tx=0x7f217401f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87c9d25d8734d89 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740a8c90 con 0x7f21680047d0 2023-11-06T03:13:42.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740a8df0 con 0x7f21680047d0 2023-11-06T03:13:42.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217dd9b640 1 -- 172.21.15.157:0/3708617144 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174005b00 con 0x7f21680047d0 2023-11-06T03:13:42.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 msgr2=0x7f2168004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 0x7f2168004c50 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f2174001060 tx=0x7f217401f270 comp rx=0 tx=0).stop 2023-11-06T03:13:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- 172.21.15.157:0/3708617144 shutdown_connections 2023-11-06T03:13:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 --2- 172.21.15.157:0/3708617144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- 172.21.15.157:0/3708617144 >> 172.21.15.157:0/3708617144 conn(0x7f218006d560 msgr2=0x7f218006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:42.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- 172.21.15.157:0/3708617144 shutdown_connections 2023-11-06T03:13:42.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- 172.21.15.157:0/3708617144 wait complete. 2023-11-06T03:13:42.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 Processor -- start 2023-11-06T03:13:42.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- start start 2023-11-06T03:13:42.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21680047d0 0x7f21801547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21800b15b0 0x7f21800b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218015c880 con 0x7f218015ba70 2023-11-06T03:13:42.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800b1f70 con 0x7f21800b15b0 2023-11-06T03:13:42.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f2185162640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800c8e30 con 0x7f21680047d0 2023-11-06T03:13:42.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55086/0 (socket says 172.21.15.157:55086) 2023-11-06T03:13:42.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217e59c640 1 -- 172.21.15.157:0/2078168567 learned_addr learned my addr 172.21.15.157:0/2078168567 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:42.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217ed9d640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21680047d0 0x7f21801547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.901+0000 7f217f59e640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21800b15b0 0x7f21800b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 -- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21680047d0 msgr2=0x7f21801547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21680047d0 0x7f21801547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 -- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21800b15b0 msgr2=0x7f21800b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21800b15b0 0x7f21800b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:42.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 -- 172.21.15.157:0/2078168567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218015bfe0 con 0x7f218015ba70 2023-11-06T03:13:42.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217e59c640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f21740ae040 tx=0x7f2168005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174004ab0 con 0x7f218015ba70 2023-11-06T03:13:42.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174004c10 con 0x7f218015ba70 2023-11-06T03:13:42.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218015c2c0 con 0x7f218015ba70 2023-11-06T03:13:42.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740070d0 con 0x7f218015ba70 2023-11-06T03:13:42.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800c90d0 con 0x7f218015ba70 2023-11-06T03:13:42.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21800d00b0 con 0x7f218015ba70 2023-11-06T03:13:42.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2174007230 con 0x7f218015ba70 2023-11-06T03:13:42.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 0x7f216c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:42.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21740b7070 con 0x7f218015ba70 2023-11-06T03:13:42.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217ed9d640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 0x7f216c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:42.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.905+0000 7f217ed9d640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 0x7f216c067f20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2170000c10 tx=0x7f2170009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:42.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:42.909+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f21800d00b0 con 0x7f218015ba70 2023-11-06T03:13:43.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.065+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21800d00b0 con 0x7f218015ba70 2023-11-06T03:13:43.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f21677fe640 1 -- 172.21.15.157:0/2078168567 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f21800d00b0 con 0x7f218015ba70 2023-11-06T03:13:43.064 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:43.065 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:41.413681+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:13:43.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 msgr2=0x7f216c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 0x7f216c067f20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2170000c10 tx=0x7f2170009040 comp rx=0 tx=0).stop 2023-11-06T03:13:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 msgr2=0x7f218015da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f21740ae040 tx=0x7f2168005800 comp rx=0 tx=0).stop 2023-11-06T03:13:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 shutdown_connections 2023-11-06T03:13:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f216c065a60 0x7f216c067f20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21800b15b0 0x7f21800b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f218015ba70 0x7f218015da20 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 --2- 172.21.15.157:0/2078168567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f21680047d0 0x7f21801547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 >> 172.21.15.157:0/2078168567 conn(0x7f218006d560 msgr2=0x7f21800b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 shutdown_connections 2023-11-06T03:13:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.069+0000 7f2185162640 1 -- 172.21.15.157:0/2078168567 wait complete. 2023-11-06T03:13:43.083 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:13:43.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:13:43.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.193+0000 7f50dc4bd640 1 Processor -- start 2023-11-06T03:13:43.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.193+0000 7f50dc4bd640 1 -- start start 2023-11-06T03:13:43.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50dc4bd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f50d40b3b20 0x7f50d415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50dc4bd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d415f580 con 0x7f50d41551e0 2023-11-06T03:13:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50dc4bd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d415efc0 con 0x7f50d415e5e0 2023-11-06T03:13:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50dc4bd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d40704b0 con 0x7f50d40b3b20 2023-11-06T03:13:43.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50da259640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f50d41551e0 0x7f50d406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54966/0 (socket says 172.21.15.157:54966) 2023-11-06T03:13:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50da259640 1 -- 172.21.15.157:0/1194030945 learned_addr learned my addr 172.21.15.157:0/1194030945 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 --2- 172.21.15.157:0/1194030945 >> v2:172.21.15.195:3300/0 conn(0x7f50d40b3b20 0x7f50d415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640743840 0 0) 0x7f50d415f580 con 0x7f50d41551e0 2023-11-06T03:13:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c0003530 con 0x7f50d41551e0 2023-11-06T03:13:43.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288918422 0 0) 0x7f50c0003530 con 0x7f50d41551e0 2023-11-06T03:13:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 >> v2:172.21.15.195:3300/0 conn(0x7f50d40b3b20 msgr2=0x7f50d415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 --2- 172.21.15.157:0/1194030945 >> v2:172.21.15.195:3300/0 conn(0x7f50d40b3b20 0x7f50d415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 >> v1:172.21.15.195:6789/0 conn(0x7f50d415e5e0 legacy=0x7f50d40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d415f2a0 con 0x7f50d41551e0 2023-11-06T03:13:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f50d0001680 con 0x7f50d41551e0 2023-11-06T03:13:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 >> v1:172.21.15.157:6789/0 conn(0x7f50d41551e0 legacy=0x7f50d406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 0x7f50c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40704b0 con 0x7f50c00052b0 2023-11-06T03:13:43.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c000c180 con 0x7f50c0003980 2023-11-06T03:13:43.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c000c340 con 0x7f50c00047d0 2023-11-06T03:13:43.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 0x7f50c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50da259640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 -- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 msgr2=0x7f50c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9a58640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 -- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0003980 msgr2=0x7f50c0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 -- 172.21.15.157:0/1194030945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0003530 con 0x7f50c00052b0 2023-11-06T03:13:43.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50da259640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:43.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9a58640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:43.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 -- 172.21.15.157:0/1194030945 reap_dead start 2023-11-06T03:13:43.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50daa5a640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 0x7f50c000b990 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009f90 tx=0x7f50cc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61baafca47ecdae1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.197+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0a5c40 con 0x7f50c00052b0 2023-11-06T03:13:43.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc00f040 con 0x7f50c00052b0 2023-11-06T03:13:43.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9257640 1 -- 172.21.15.157:0/1194030945 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0039a0 con 0x7f50c00052b0 2023-11-06T03:13:43.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 msgr2=0x7f50c000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 0x7f50c000b990 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009f90 tx=0x7f50cc01cc30 comp rx=0 tx=0).stop 2023-11-06T03:13:43.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/1194030945 shutdown_connections 2023-11-06T03:13:43.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/1194030945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c00052b0 0x7f50c000b990 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/1194030945 >> 172.21.15.157:0/1194030945 conn(0x7f50d4065930 msgr2=0x7f50d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/1194030945 shutdown_connections 2023-11-06T03:13:43.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/1194030945 wait complete. 2023-11-06T03:13:43.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 Processor -- start 2023-11-06T03:13:43.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- start start 2023-11-06T03:13:43.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00052b0 0x7f50d406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d413f5b0 0x7f50d40b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d41525e0 con 0x7f50d4151260 2023-11-06T03:13:43.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d41519d0 con 0x7f50d413f5b0 2023-11-06T03:13:43.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d4151cb0 con 0x7f50c00052b0 2023-11-06T03:13:43.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55104/0 (socket says 172.21.15.157:55104) 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 -- 172.21.15.157:0/4051780450 learned_addr learned my addr 172.21.15.157:0/4051780450 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50da259640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00052b0 0x7f50d406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50daa5a640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d413f5b0 0x7f50d40b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 -- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00052b0 msgr2=0x7f50d406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00052b0 0x7f50d406ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 -- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d413f5b0 msgr2=0x7f50d40b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d413f5b0 0x7f50d40b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 -- 172.21.15.157:0/4051780450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4151f90 con 0x7f50d4151260 2023-11-06T03:13:43.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50d9a58640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d770 tx=0x7f50c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4005270 con 0x7f50d4151260 2023-11-06T03:13:43.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c40053d0 con 0x7f50d4151260 2023-11-06T03:13:43.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d4152270 con 0x7f50d4151260 2023-11-06T03:13:43.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c400b8f0 con 0x7f50d4151260 2023-11-06T03:13:43.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.201+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d413f000 con 0x7f50d4151260 2023-11-06T03:13:43.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4151df0 con 0x7f50d4151260 2023-11-06T03:13:43.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f50c400ba50 con 0x7f50d4151260 2023-11-06T03:13:43.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50caffd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 0x7f50b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50c4083310 con 0x7f50d4151260 2023-11-06T03:13:43.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50da259640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 0x7f50b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.205+0000 7f50da259640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 0x7f50b8067f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001210 tx=0x7f50d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.209+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f50d4151df0 con 0x7f50d4151260 2023-11-06T03:13:43.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.349+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50d415b1b0 con 0x7f50b8065a60 2023-11-06T03:13:43.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.369+0000 7f50caffd640 1 -- 172.21.15.157:0/4051780450 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299907 (secure 0 0 0) 0x7f50d000d040 con 0x7f50b8065a60 2023-11-06T03:13:43.371 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:43.372 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:13:43.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 msgr2=0x7f50b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 0x7f50b8067f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001210 tx=0x7f50d0009040 comp rx=0 tx=0).stop 2023-11-06T03:13:43.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 msgr2=0x7f50d413eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d770 tx=0x7f50c400dc30 comp rx=0 tx=0).stop 2023-11-06T03:13:43.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 shutdown_connections 2023-11-06T03:13:43.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50b8065a60 0x7f50b8067f20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d413f5b0 0x7f50d40b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d4151260 0x7f50d413eac0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 --2- 172.21.15.157:0/4051780450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00052b0 0x7f50d406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 >> 172.21.15.157:0/4051780450 conn(0x7f50d4065930 msgr2=0x7f50d40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 shutdown_connections 2023-11-06T03:13:43.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.377+0000 7f50dc4bd640 1 -- 172.21.15.157:0/4051780450 wait complete. 2023-11-06T03:13:43.389 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-11-06T03:13:43.305114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584268,"kb_used_data":12940,"kb_used_omap":0,"kb_used_meta":571264,"kb_avail":754390452,"statfs":{"total":773094113280,"available":772495822848,"internally_reserved":0,"allocated":13250560,"data_stored":6725383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":584974336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277974"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039944+0000","last_change":"2023-11-06T03:13:39.039944+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039944+0000","last_undegraded":"2023-11-06T03:13:39.039944+0000","last_fullsized":"2023-11-06T03:13:39.039944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034813+0000","last_change":"2023-11-06T03:13:39.034813+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034813+0000","last_undegraded":"2023-11-06T03:13:39.034813+0000","last_fullsized":"2023-11-06T03:13:39.034813+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037617+0000","last_change":"2023-11-06T03:13:39.037617+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037617+0000","last_undegraded":"2023-11-06T03:13:39.037617+0000","last_fullsized":"2023-11-06T03:13:39.037617+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034124+0000","last_change":"2023-11-06T03:13:39.034124+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034124+0000","last_undegraded":"2023-11-06T03:13:39.034124+0000","last_fullsized":"2023-11-06T03:13:39.034124+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.031025+0000","last_change":"2023-11-06T03:13:39.031025+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.031025+0000","last_undegraded":"2023-11-06T03:13:39.031025+0000","last_fullsized":"2023-11-06T03:13:39.031025+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030039+0000","last_change":"2023-11-06T03:13:39.030039+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030039+0000","last_undegraded":"2023-11-06T03:13:39.030039+0000","last_fullsized":"2023-11-06T03:13:39.030039+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033922+0000","last_change":"2023-11-06T03:13:39.033922+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033922+0000","last_undegraded":"2023-11-06T03:13:39.033922+0000","last_fullsized":"2023-11-06T03:13:39.033922+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030067+0000","last_change":"2023-11-06T03:13:39.030067+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030067+0000","last_undegraded":"2023-11-06T03:13:39.030067+0000","last_fullsized":"2023-11-06T03:13:39.030067+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033625+0000","last_change":"2023-11-06T03:13:39.033625+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033625+0000","last_undegraded":"2023-11-06T03:13:39.033625+0000","last_fullsized":"2023-11-06T03:13:39.033625+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032230+0000","last_change":"2023-11-06T03:13:39.032230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032230+0000","last_undegraded":"2023-11-06T03:13:39.032230+0000","last_fullsized":"2023-11-06T03:13:39.032230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.036701+0000","last_change":"2023-11-06T03:13:39.036701+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.036701+0000","last_undegraded":"2023-11-06T03:13:39.036701+0000","last_fullsized":"2023-11-06T03:13:39.036701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035894+0000","last_change":"2023-11-06T03:13:39.035894+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035894+0000","last_undegraded":"2023-11-06T03:13:39.035894+0000","last_fullsized":"2023-11-06T03:13:39.035894+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.029843+0000","last_change":"2023-11-06T03:13:39.029843+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.029843+0000","last_undegraded":"2023-11-06T03:13:39.029843+0000","last_fullsized":"2023-11-06T03:13:39.029843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035815+0000","last_change":"2023-11-06T03:13:39.035815+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035815+0000","last_undegraded":"2023-11-06T03:13:39.035815+0000","last_fullsized":"2023-11-06T03:13:39.035815+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032991+0000","last_change":"2023-11-06T03:13:39.032991+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032991+0000","last_undegraded":"2023-11-06T03:13:39.032991+0000","last_fullsized":"2023-11-06T03:13:39.032991+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037034+0000","last_change":"2023-11-06T03:13:39.037034+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037034+0000","last_undegraded":"2023-11-06T03:13:39.037034+0000","last_fullsized":"2023-11-06T03:13:39.037034+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030155+0000","last_change":"2023-11-06T03:13:39.030155+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030155+0000","last_undegraded":"2023-11-06T03:13:39.030155+0000","last_fullsized":"2023-11-06T03:13:39.030155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039010+0000","last_change":"2023-11-06T03:13:39.039010+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039010+0000","last_undegraded":"2023-11-06T03:13:39.039010+0000","last_fullsized":"2023-11-06T03:13:39.039010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039072+0000","last_change":"2023-11-06T03:13:39.039072+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039072+0000","last_undegraded":"2023-11-06T03:13:39.039072+0000","last_fullsized":"2023-11-06T03:13:39.039072+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030042+0000","last_change":"2023-11-06T03:13:39.030042+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030042+0000","last_undegraded":"2023-11-06T03:13:39.030042+0000","last_fullsized":"2023-11-06T03:13:39.030042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030197+0000","last_change":"2023-11-06T03:13:39.030197+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030197+0000","last_undegraded":"2023-11-06T03:13:39.030197+0000","last_fullsized":"2023-11-06T03:13:39.030197+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030538+0000","last_change":"2023-11-06T03:13:39.030538+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030538+0000","last_undegraded":"2023-11-06T03:13:39.030538+0000","last_fullsized":"2023-11-06T03:13:39.030538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030571+0000","last_change":"2023-11-06T03:13:39.030571+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030571+0000","last_undegraded":"2023-11-06T03:13:39.030571+0000","last_fullsized":"2023-11-06T03:13:39.030571+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035852+0000","last_change":"2023-11-06T03:13:39.035852+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035852+0000","last_undegraded":"2023-11-06T03:13:39.035852+0000","last_fullsized":"2023-11-06T03:13:39.035852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032867+0000","last_change":"2023-11-06T03:13:39.032867+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032867+0000","last_undegraded":"2023-11-06T03:13:39.032867+0000","last_fullsized":"2023-11-06T03:13:39.032867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116943+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:13:37.116943+0000","last_peered":"2023-11-06T03:13:37.116943+0000","last_clean":"2023-11-06T03:13:37.116943+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:13:37.116943+0000","last_undegraded":"2023-11-06T03:13:37.116943+0000","last_fullsized":"2023-11-06T03:13:37.116943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114100+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:13:37.114100+0000","last_peered":"2023-11-06T03:13:37.114100+0000","last_clean":"2023-11-06T03:13:37.114100+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:13:37.114100+0000","last_undegraded":"2023-11-06T03:13:37.114100+0000","last_fullsized":"2023-11-06T03:13:37.114100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033827+0000","last_change":"2023-11-06T03:13:39.033827+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033827+0000","last_undegraded":"2023-11-06T03:13:39.033827+0000","last_fullsized":"2023-11-06T03:13:39.033827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033218+0000","last_change":"2023-11-06T03:13:39.033218+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033218+0000","last_undegraded":"2023-11-06T03:13:39.033218+0000","last_fullsized":"2023-11-06T03:13:39.033218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.113041+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:13:37.113041+0000","last_peered":"2023-11-06T03:13:37.113041+0000","last_clean":"2023-11-06T03:13:37.113041+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:13:37.113041+0000","last_undegraded":"2023-11-06T03:13:37.113041+0000","last_fullsized":"2023-11-06T03:13:37.113041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034529+0000","last_change":"2023-11-06T03:13:39.034529+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034529+0000","last_undegraded":"2023-11-06T03:13:39.034529+0000","last_fullsized":"2023-11-06T03:13:39.034529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114094+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:13:37.114094+0000","last_peered":"2023-11-06T03:13:37.114094+0000","last_clean":"2023-11-06T03:13:37.114094+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:13:37.114094+0000","last_undegraded":"2023-11-06T03:13:37.114094+0000","last_fullsized":"2023-11-06T03:13:37.114094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039011+0000","last_change":"2023-11-06T03:13:39.039011+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039011+0000","last_undegraded":"2023-11-06T03:13:39.039011+0000","last_fullsized":"2023-11-06T03:13:39.039011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111589+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:13:37.111589+0000","last_peered":"2023-11-06T03:13:37.111589+0000","last_clean":"2023-11-06T03:13:37.111589+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:13:37.111589+0000","last_undegraded":"2023-11-06T03:13:37.111589+0000","last_fullsized":"2023-11-06T03:13:37.111589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034230+0000","last_change":"2023-11-06T03:13:39.034230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034230+0000","last_undegraded":"2023-11-06T03:13:39.034230+0000","last_fullsized":"2023-11-06T03:13:39.034230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033916+0000","last_change":"2023-11-06T03:13:39.033916+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033916+0000","last_undegraded":"2023-11-06T03:13:39.033916+0000","last_fullsized":"2023-11-06T03:13:39.033916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111653+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:13:37.111653+0000","last_peered":"2023-11-06T03:13:37.111653+0000","last_clean":"2023-11-06T03:13:37.111653+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:13:37.111653+0000","last_undegraded":"2023-11-06T03:13:37.111653+0000","last_fullsized":"2023-11-06T03:13:37.111653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032841+0000","last_change":"2023-11-06T03:13:39.032841+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032841+0000","last_undegraded":"2023-11-06T03:13:39.032841+0000","last_fullsized":"2023-11-06T03:13:39.032841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.115547+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:13:37.115547+0000","last_peered":"2023-11-06T03:13:37.115547+0000","last_clean":"2023-11-06T03:13:37.115547+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:13:37.115547+0000","last_undegraded":"2023-11-06T03:13:37.115547+0000","last_fullsized":"2023-11-06T03:13:37.115547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":2109440,"data_stored":1286916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1830000000000001,"15min":1.4830000000000001},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.85999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.3620000000000001,"15min":1.397},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.147,"5min":1.2490000000000001,"15min":1.444},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.153,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.90700000000000003,"15min":0.88700000000000001},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1539999999999999,"5min":1.381,"15min":1.6080000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.22,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":2.512,"15min":2.512},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.097,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":10.596,"15min":10.596},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.71799999999999997,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.5309999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.99199999999999999,"15min":1.0009999999999999},"min":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.278,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0600000000000001,"15min":1.048},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3720000000000001,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.349,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.123}]},{"osd":6,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.169,"15min":1.1539999999999999},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.454,"5min":2.548,"15min":2.548},"last":1.198},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.1040000000000001,"15min":1.105},"min":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.3049999999999999,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.05}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64460,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307380,"statfs":{"total":96636764160,"available":96570757120,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.97299999999999998,"15min":0.84999999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.081,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.2749999999999999,"15min":0.97299999999999998},"min":{"1min":0.50700000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.0269999999999999,"15min":0.90100000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1359999999999999,"5min":16.012,"15min":16.012},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.1279999999999999,"15min":0.91300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.105,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.94699999999999995},"min":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1619999999999999,"5min":2.843,"15min":2.843},"last":1.2410000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.647,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.2929999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.877,"5min":1.355,"15min":1.052},"min":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.278,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.335,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2609999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.413,"15min":1.2370000000000001},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3540000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.367}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.411,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.294,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1379999999999999,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.3240000000000001,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.399}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.4079999999999999,"15min":1.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.429,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.3999999999999999,"15min":1.212},"min":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.371,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.204}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.7729999999999999,"15min":1.1299999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.0840000000000001,"5min":125.256,"15min":125.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.8129999999999999,"15min":1.159},"min":{"1min":0.495,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.028,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.78000000000000003},"min":{"1min":0.60799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.27,"5min":2.339,"15min":2.339},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.57199999999999995,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.53500000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1799999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0640000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91300000000000003,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.208,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.94499999999999995},"min":{"1min":0.60099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3400000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.66500000000000004,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5149999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.044,"15min":0.98699999999999999},"min":{"1min":0.76200000000000001,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":3.0030000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.008}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0349999999999999,"15min":1.01},"min":{"1min":0.70699999999999996,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.429,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0620000000000001,"15min":1.0249999999999999},"min":{"1min":0.79200000000000004,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3600000000000001,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0960000000000001,"15min":1.0249999999999999},"min":{"1min":0.85599999999999998,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.421,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.94299999999999995,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.514,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90692,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281148,"statfs":{"total":96636764160,"available":96543895552,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.26,"15min":0.96599999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.372},"max":{"1min":1.0429999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.2929999999999999,"15min":0.95899999999999996},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.99099999999999999,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.999}]},{"osd":1,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.51400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1279999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.032,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":1.385}]},{"osd":2,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1779999999999999,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.042,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.59399999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":3.198,"15min":3.198},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.64800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0960000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.081}]},{"osd":5,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0609999999999999,"15min":1.0269999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.143,"5min":10.192,"15min":10.192},"last":1.254},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0600000000000001,"15min":1.089},"min":{"1min":0.80500000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.415}]},{"osd":6,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96099999999999997,"15min":0.998},"min":{"1min":0.66400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.514,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.1140000000000001},"min":{"1min":0.71499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.399,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.512}]},{"osd":7,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.077,"15min":1.075},"min":{"1min":0.76800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.171},"min":{"1min":0.79600000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4099999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.472}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76812,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295028,"statfs":{"total":96636764160,"available":96558108672,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78200000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":1.524},"last":0.313}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.26400000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.89200000000000002,"5min":1.371,"15min":1.371},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93100000000000005,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.23300000000000001},"max":{"1min":0.95099999999999996,"5min":1.532,"15min":1.532},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.34599999999999997,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.206,"5min":1.284,"15min":1.284},"last":0.65000000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45000000000000001},"max":{"1min":1.2270000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.91100000000000003,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.369,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.621}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.626,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.536,"15min":1.6639999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.698},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.999},"min":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.3939999999999999,"5min":1.649,"15min":1.8620000000000001},"last":0.748},{"interface":"front","average":{"1min":0.872,"5min":0.94699999999999995,"15min":0.97999999999999998},"min":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3129999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":1.03},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4410000000000001,"5min":1.835,"15min":1.835},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.623,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.569,"5min":1.734,"15min":1.8979999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.93400000000000005,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.55100000000000005,"15min":0.61399999999999999},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.88100000000000001,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.63400000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.64300000000000002},"min":{"1min":0.32500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.1579999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.294},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65000000000000002,"15min":0.71699999999999997},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.022,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83199999999999996,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.071,"5min":1.526,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.91500000000000004},"min":{"1min":0.52300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82899999999999996,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.248,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.2390000000000001,"5min":1.486,"15min":1.5780000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.381,"15min":1.615},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.94499999999999995},"min":{"1min":0.66100000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.048}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.91400000000000003,"15min":0.97699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3109999999999999,"5min":1.4650000000000001,"15min":4.2809999999999997},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88800000000000001,"15min":0.94199999999999995},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.4740000000000002},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325372,"statfs":{"total":96636764160,"available":96589180928,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.0569999999999999,"5min":1.843,"15min":1.843},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.81200000000000006,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.56499999999999995,"15min":0.70299999999999996},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.76300000000000001,"5min":1.169,"15min":30.861000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.66700000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.0329999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0369999999999999,"5min":1.151,"15min":2.3079999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.82099999999999995,"15min":0.85999999999999999},"min":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.198,"5min":2.0870000000000002,"15min":2.258},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.82499999999999996,"15min":0.92900000000000005},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":1.5549999999999999,"15min":14.753},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":1.101},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47299999999999998},"max":{"1min":1.1499999999999999,"5min":1.528,"15min":43.103999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.92700000000000005,"15min":1.133},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.208,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.88,"15min":1.0409999999999999},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.45100000000000001},"max":{"1min":1.2909999999999999,"5min":1.5189999999999999,"15min":23.971},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":1.0629999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0389999999999999},"min":{"1min":0.67600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.212,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95699999999999996,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.319,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280496,"statfs":{"total":96636764160,"available":96543227904,"internally_reserved":0,"allocated":2105344,"data_stored":1285897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.27500000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.80500000000000005,"5min":1.145,"15min":2.6989999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59999999999999998,"15min":0.64900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.64600000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.82399999999999995,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.872}]},{"osd":3,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.391,"5min":1.391,"15min":2.7999999999999998},"last":0.90700000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.91800000000000004},"min":{"1min":0.54700000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.371,"5min":1.371,"15min":4.641},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.254,"15min":2.028},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.96399999999999997},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.0920000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":1.244},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.93999999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.147}]},{"osd":6,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57399999999999995,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.96399999999999997,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93100000000000005,"15min":0.996},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":1.266,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.101},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.93799999999999994,"15min":0.97999999999999998},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.335,"5min":1.4670000000000001,"15min":1.97},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:13:43.397 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:13:43.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 Processor -- start 2023-11-06T03:13:43.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 -- start start 2023-11-06T03:13:43.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bd415caa0 0x7f5bd40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd40d0280 0x7f5bd40b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40d2920 con 0x7f5bd415ce80 2023-11-06T03:13:43.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40d18a0 con 0x7f5bd415caa0 2023-11-06T03:13:43.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bda1e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40d1b60 con 0x7f5bd40d0280 2023-11-06T03:13:43.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bd415caa0 0x7f5bd40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bd415caa0 0x7f5bd40cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55108/0 (socket says 172.21.15.157:55108) 2023-11-06T03:13:43.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 -- 172.21.15.157:0/1897772377 learned_addr learned my addr 172.21.15.157:0/1897772377 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:43.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd3fff640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd40d0280 0x7f5bd40b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 -- 172.21.15.157:0/1897772377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd40d0280 msgr2=0x7f5bd40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd40d0280 0x7f5bd40b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 -- 172.21.15.157:0/1897772377 >> v1:172.21.15.195:6789/0 conn(0x7f5bd415ce80 legacy=0x7f5bd40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.553+0000 7f5bd37fe640 1 -- 172.21.15.157:0/1897772377 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40d1e40 con 0x7f5bd415caa0 2023-11-06T03:13:43.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd40cdc60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009870 tx=0x7f5bc401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c8f8c9e567adb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd27fc640 1 -- 172.21.15.157:0/1897772377 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40aa670 con 0x7f5bd415caa0 2023-11-06T03:13:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd27fc640 1 -- 172.21.15.157:0/1897772377 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc40aa7d0 con 0x7f5bd415caa0 2023-11-06T03:13:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd27fc640 1 -- 172.21.15.157:0/1897772377 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40aab60 con 0x7f5bd415caa0 2023-11-06T03:13:43.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- 172.21.15.157:0/1897772377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 msgr2=0x7f5bd40cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd40cdc60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009870 tx=0x7f5bc401c510 comp rx=0 tx=0).stop 2023-11-06T03:13:43.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- 172.21.15.157:0/1897772377 shutdown_connections 2023-11-06T03:13:43.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd40d0280 0x7f5bd40b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/1897772377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd40cdc60 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- 172.21.15.157:0/1897772377 >> 172.21.15.157:0/1897772377 conn(0x7f5bd4070790 msgr2=0x7f5bd4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- 172.21.15.157:0/1897772377 shutdown_connections 2023-11-06T03:13:43.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- 172.21.15.157:0/1897772377 wait complete. 2023-11-06T03:13:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 Processor -- start 2023-11-06T03:13:43.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- start start 2023-11-06T03:13:43.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 0x7f5bd4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd415ce80 0x7f5bd415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4155710 con 0x7f5bd415caa0 2023-11-06T03:13:43.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd415d2c0 con 0x7f5bd40d0280 2023-11-06T03:13:43.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bda1e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd415d840 con 0x7f5bd415ce80 2023-11-06T03:13:43.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55114/0 (socket says 172.21.15.157:55114) 2023-11-06T03:13:43.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 -- 172.21.15.157:0/234297129 learned_addr learned my addr 172.21.15.157:0/234297129 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:43.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 0x7f5bd4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd3fff640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd415ce80 0x7f5bd415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 -- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd415ce80 msgr2=0x7f5bd415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd415ce80 0x7f5bd415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 -- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 msgr2=0x7f5bd4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 0x7f5bd4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd2ffd640 1 -- 172.21.15.157:0/234297129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40094d0 con 0x7f5bd415caa0 2023-11-06T03:13:43.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.557+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 0x7f5bd4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:43.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bd2ffd640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c4d0 tx=0x7f5bc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80052d0 con 0x7f5bd415caa0 2023-11-06T03:13:43.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8005430 con 0x7f5bd415caa0 2023-11-06T03:13:43.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4154ed0 con 0x7f5bd415caa0 2023-11-06T03:13:43.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8012830 con 0x7f5bd415caa0 2023-11-06T03:13:43.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd415d4d0 con 0x7f5bd415caa0 2023-11-06T03:13:43.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4072750 con 0x7f5bd415caa0 2023-11-06T03:13:43.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5bc8012990 con 0x7f5bd415caa0 2023-11-06T03:13:43.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 0x7f5bc0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bc8082e00 con 0x7f5bd415caa0 2023-11-06T03:13:43.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 0x7f5bc0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.561+0000 7f5bd37fe640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 0x7f5bc0067f20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ae000 tx=0x7f5bc40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.565+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5bc8050660 con 0x7f5bd415caa0 2023-11-06T03:13:43.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.705+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bd40b1f00 con 0x7f5bc0065a60 2023-11-06T03:13:43.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.725+0000 7f5bbbfff640 1 -- 172.21.15.157:0/234297129 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299907 (secure 0 0 0) 0x7f5bc40aa9c0 con 0x7f5bc0065a60 2023-11-06T03:13:43.728 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:43.730 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:13:43.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 msgr2=0x7f5bc0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 0x7f5bc0067f20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ae000 tx=0x7f5bc40a7000 comp rx=0 tx=0).stop 2023-11-06T03:13:43.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 msgr2=0x7f5bd415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c4d0 tx=0x7f5bc800c990 comp rx=0 tx=0).stop 2023-11-06T03:13:43.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 shutdown_connections 2023-11-06T03:13:43.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bc0065a60 0x7f5bc0067f20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd415ce80 0x7f5bd415dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd415caa0 0x7f5bd415b6c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 --2- 172.21.15.157:0/234297129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd40d0280 0x7f5bd4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 >> 172.21.15.157:0/234297129 conn(0x7f5bd4070790 msgr2=0x7f5bd40b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 shutdown_connections 2023-11-06T03:13:43.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.729+0000 7f5bda1e8640 1 -- 172.21.15.157:0/234297129 wait complete. 2023-11-06T03:13:43.743 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-11-06T03:13:43.305114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584268,"kb_used_data":12940,"kb_used_omap":0,"kb_used_meta":571264,"kb_avail":754390452,"statfs":{"total":773094113280,"available":772495822848,"internally_reserved":0,"allocated":13250560,"data_stored":6725383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":584974336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277974"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039944+0000","last_change":"2023-11-06T03:13:39.039944+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039944+0000","last_undegraded":"2023-11-06T03:13:39.039944+0000","last_fullsized":"2023-11-06T03:13:39.039944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034813+0000","last_change":"2023-11-06T03:13:39.034813+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034813+0000","last_undegraded":"2023-11-06T03:13:39.034813+0000","last_fullsized":"2023-11-06T03:13:39.034813+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037617+0000","last_change":"2023-11-06T03:13:39.037617+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037617+0000","last_undegraded":"2023-11-06T03:13:39.037617+0000","last_fullsized":"2023-11-06T03:13:39.037617+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034124+0000","last_change":"2023-11-06T03:13:39.034124+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034124+0000","last_undegraded":"2023-11-06T03:13:39.034124+0000","last_fullsized":"2023-11-06T03:13:39.034124+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.031025+0000","last_change":"2023-11-06T03:13:39.031025+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.031025+0000","last_undegraded":"2023-11-06T03:13:39.031025+0000","last_fullsized":"2023-11-06T03:13:39.031025+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030039+0000","last_change":"2023-11-06T03:13:39.030039+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030039+0000","last_undegraded":"2023-11-06T03:13:39.030039+0000","last_fullsized":"2023-11-06T03:13:39.030039+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033922+0000","last_change":"2023-11-06T03:13:39.033922+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033922+0000","last_undegraded":"2023-11-06T03:13:39.033922+0000","last_fullsized":"2023-11-06T03:13:39.033922+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030067+0000","last_change":"2023-11-06T03:13:39.030067+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030067+0000","last_undegraded":"2023-11-06T03:13:39.030067+0000","last_fullsized":"2023-11-06T03:13:39.030067+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033625+0000","last_change":"2023-11-06T03:13:39.033625+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033625+0000","last_undegraded":"2023-11-06T03:13:39.033625+0000","last_fullsized":"2023-11-06T03:13:39.033625+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032230+0000","last_change":"2023-11-06T03:13:39.032230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032230+0000","last_undegraded":"2023-11-06T03:13:39.032230+0000","last_fullsized":"2023-11-06T03:13:39.032230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.036701+0000","last_change":"2023-11-06T03:13:39.036701+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.036701+0000","last_undegraded":"2023-11-06T03:13:39.036701+0000","last_fullsized":"2023-11-06T03:13:39.036701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035894+0000","last_change":"2023-11-06T03:13:39.035894+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035894+0000","last_undegraded":"2023-11-06T03:13:39.035894+0000","last_fullsized":"2023-11-06T03:13:39.035894+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.029843+0000","last_change":"2023-11-06T03:13:39.029843+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.029843+0000","last_undegraded":"2023-11-06T03:13:39.029843+0000","last_fullsized":"2023-11-06T03:13:39.029843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035815+0000","last_change":"2023-11-06T03:13:39.035815+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035815+0000","last_undegraded":"2023-11-06T03:13:39.035815+0000","last_fullsized":"2023-11-06T03:13:39.035815+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032991+0000","last_change":"2023-11-06T03:13:39.032991+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032991+0000","last_undegraded":"2023-11-06T03:13:39.032991+0000","last_fullsized":"2023-11-06T03:13:39.032991+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037034+0000","last_change":"2023-11-06T03:13:39.037034+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037034+0000","last_undegraded":"2023-11-06T03:13:39.037034+0000","last_fullsized":"2023-11-06T03:13:39.037034+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030155+0000","last_change":"2023-11-06T03:13:39.030155+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030155+0000","last_undegraded":"2023-11-06T03:13:39.030155+0000","last_fullsized":"2023-11-06T03:13:39.030155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039010+0000","last_change":"2023-11-06T03:13:39.039010+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039010+0000","last_undegraded":"2023-11-06T03:13:39.039010+0000","last_fullsized":"2023-11-06T03:13:39.039010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039072+0000","last_change":"2023-11-06T03:13:39.039072+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039072+0000","last_undegraded":"2023-11-06T03:13:39.039072+0000","last_fullsized":"2023-11-06T03:13:39.039072+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030042+0000","last_change":"2023-11-06T03:13:39.030042+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030042+0000","last_undegraded":"2023-11-06T03:13:39.030042+0000","last_fullsized":"2023-11-06T03:13:39.030042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030197+0000","last_change":"2023-11-06T03:13:39.030197+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030197+0000","last_undegraded":"2023-11-06T03:13:39.030197+0000","last_fullsized":"2023-11-06T03:13:39.030197+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030538+0000","last_change":"2023-11-06T03:13:39.030538+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030538+0000","last_undegraded":"2023-11-06T03:13:39.030538+0000","last_fullsized":"2023-11-06T03:13:39.030538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030571+0000","last_change":"2023-11-06T03:13:39.030571+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030571+0000","last_undegraded":"2023-11-06T03:13:39.030571+0000","last_fullsized":"2023-11-06T03:13:39.030571+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035852+0000","last_change":"2023-11-06T03:13:39.035852+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035852+0000","last_undegraded":"2023-11-06T03:13:39.035852+0000","last_fullsized":"2023-11-06T03:13:39.035852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032867+0000","last_change":"2023-11-06T03:13:39.032867+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032867+0000","last_undegraded":"2023-11-06T03:13:39.032867+0000","last_fullsized":"2023-11-06T03:13:39.032867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116943+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:13:37.116943+0000","last_peered":"2023-11-06T03:13:37.116943+0000","last_clean":"2023-11-06T03:13:37.116943+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:13:37.116943+0000","last_undegraded":"2023-11-06T03:13:37.116943+0000","last_fullsized":"2023-11-06T03:13:37.116943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114100+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:13:37.114100+0000","last_peered":"2023-11-06T03:13:37.114100+0000","last_clean":"2023-11-06T03:13:37.114100+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:13:37.114100+0000","last_undegraded":"2023-11-06T03:13:37.114100+0000","last_fullsized":"2023-11-06T03:13:37.114100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033827+0000","last_change":"2023-11-06T03:13:39.033827+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033827+0000","last_undegraded":"2023-11-06T03:13:39.033827+0000","last_fullsized":"2023-11-06T03:13:39.033827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033218+0000","last_change":"2023-11-06T03:13:39.033218+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033218+0000","last_undegraded":"2023-11-06T03:13:39.033218+0000","last_fullsized":"2023-11-06T03:13:39.033218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.113041+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:13:37.113041+0000","last_peered":"2023-11-06T03:13:37.113041+0000","last_clean":"2023-11-06T03:13:37.113041+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:13:37.113041+0000","last_undegraded":"2023-11-06T03:13:37.113041+0000","last_fullsized":"2023-11-06T03:13:37.113041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034529+0000","last_change":"2023-11-06T03:13:39.034529+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034529+0000","last_undegraded":"2023-11-06T03:13:39.034529+0000","last_fullsized":"2023-11-06T03:13:39.034529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114094+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:13:37.114094+0000","last_peered":"2023-11-06T03:13:37.114094+0000","last_clean":"2023-11-06T03:13:37.114094+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:13:37.114094+0000","last_undegraded":"2023-11-06T03:13:37.114094+0000","last_fullsized":"2023-11-06T03:13:37.114094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039011+0000","last_change":"2023-11-06T03:13:39.039011+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039011+0000","last_undegraded":"2023-11-06T03:13:39.039011+0000","last_fullsized":"2023-11-06T03:13:39.039011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111589+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:13:37.111589+0000","last_peered":"2023-11-06T03:13:37.111589+0000","last_clean":"2023-11-06T03:13:37.111589+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:13:37.111589+0000","last_undegraded":"2023-11-06T03:13:37.111589+0000","last_fullsized":"2023-11-06T03:13:37.111589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034230+0000","last_change":"2023-11-06T03:13:39.034230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034230+0000","last_undegraded":"2023-11-06T03:13:39.034230+0000","last_fullsized":"2023-11-06T03:13:39.034230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033916+0000","last_change":"2023-11-06T03:13:39.033916+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033916+0000","last_undegraded":"2023-11-06T03:13:39.033916+0000","last_fullsized":"2023-11-06T03:13:39.033916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111653+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:13:37.111653+0000","last_peered":"2023-11-06T03:13:37.111653+0000","last_clean":"2023-11-06T03:13:37.111653+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:13:37.111653+0000","last_undegraded":"2023-11-06T03:13:37.111653+0000","last_fullsized":"2023-11-06T03:13:37.111653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032841+0000","last_change":"2023-11-06T03:13:39.032841+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032841+0000","last_undegraded":"2023-11-06T03:13:39.032841+0000","last_fullsized":"2023-11-06T03:13:39.032841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.115547+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:13:37.115547+0000","last_peered":"2023-11-06T03:13:37.115547+0000","last_clean":"2023-11-06T03:13:37.115547+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:13:37.115547+0000","last_undegraded":"2023-11-06T03:13:37.115547+0000","last_fullsized":"2023-11-06T03:13:37.115547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":2109440,"data_stored":1286916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1830000000000001,"15min":1.4830000000000001},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.85999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.3620000000000001,"15min":1.397},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.147,"5min":1.2490000000000001,"15min":1.444},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.153,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.90700000000000003,"15min":0.88700000000000001},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1539999999999999,"5min":1.381,"15min":1.6080000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.22,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":2.512,"15min":2.512},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.097,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":10.596,"15min":10.596},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.71799999999999997,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.5309999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.99199999999999999,"15min":1.0009999999999999},"min":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.278,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0600000000000001,"15min":1.048},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3720000000000001,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.349,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.123}]},{"osd":6,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.169,"15min":1.1539999999999999},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.454,"5min":2.548,"15min":2.548},"last":1.198},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.1040000000000001,"15min":1.105},"min":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.3049999999999999,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.05}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64460,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307380,"statfs":{"total":96636764160,"available":96570757120,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.97299999999999998,"15min":0.84999999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.081,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.2749999999999999,"15min":0.97299999999999998},"min":{"1min":0.50700000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.0269999999999999,"15min":0.90100000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1359999999999999,"5min":16.012,"15min":16.012},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.1279999999999999,"15min":0.91300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.105,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.94699999999999995},"min":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1619999999999999,"5min":2.843,"15min":2.843},"last":1.2410000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.647,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.2929999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.877,"5min":1.355,"15min":1.052},"min":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.278,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.335,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2609999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.413,"15min":1.2370000000000001},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3540000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.367}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.411,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.294,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1379999999999999,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.3240000000000001,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.399}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.4079999999999999,"15min":1.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.429,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.3999999999999999,"15min":1.212},"min":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.371,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.204}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.7729999999999999,"15min":1.1299999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.0840000000000001,"5min":125.256,"15min":125.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.8129999999999999,"15min":1.159},"min":{"1min":0.495,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.028,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.78000000000000003},"min":{"1min":0.60799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.27,"5min":2.339,"15min":2.339},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.57199999999999995,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.53500000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1799999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0640000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91300000000000003,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.208,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.94499999999999995},"min":{"1min":0.60099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3400000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.66500000000000004,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5149999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.044,"15min":0.98699999999999999},"min":{"1min":0.76200000000000001,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":3.0030000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.008}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0349999999999999,"15min":1.01},"min":{"1min":0.70699999999999996,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.429,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0620000000000001,"15min":1.0249999999999999},"min":{"1min":0.79200000000000004,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3600000000000001,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0960000000000001,"15min":1.0249999999999999},"min":{"1min":0.85599999999999998,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.421,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.94299999999999995,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.514,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90692,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281148,"statfs":{"total":96636764160,"available":96543895552,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.26,"15min":0.96599999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.372},"max":{"1min":1.0429999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.2929999999999999,"15min":0.95899999999999996},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.99099999999999999,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.999}]},{"osd":1,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.51400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1279999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.032,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":1.385}]},{"osd":2,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1779999999999999,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.042,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.59399999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":3.198,"15min":3.198},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.64800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0960000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.081}]},{"osd":5,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0609999999999999,"15min":1.0269999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.143,"5min":10.192,"15min":10.192},"last":1.254},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0600000000000001,"15min":1.089},"min":{"1min":0.80500000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.415}]},{"osd":6,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96099999999999997,"15min":0.998},"min":{"1min":0.66400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.514,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.1140000000000001},"min":{"1min":0.71499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.399,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.512}]},{"osd":7,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.077,"15min":1.075},"min":{"1min":0.76800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.171},"min":{"1min":0.79600000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4099999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.472}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76812,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295028,"statfs":{"total":96636764160,"available":96558108672,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78200000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":1.524},"last":0.313}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.26400000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.89200000000000002,"5min":1.371,"15min":1.371},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93100000000000005,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.23300000000000001},"max":{"1min":0.95099999999999996,"5min":1.532,"15min":1.532},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.34599999999999997,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.206,"5min":1.284,"15min":1.284},"last":0.65000000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45000000000000001},"max":{"1min":1.2270000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.91100000000000003,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.369,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.621}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.626,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.536,"15min":1.6639999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.698},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.999},"min":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.3939999999999999,"5min":1.649,"15min":1.8620000000000001},"last":0.748},{"interface":"front","average":{"1min":0.872,"5min":0.94699999999999995,"15min":0.97999999999999998},"min":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3129999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":1.03},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4410000000000001,"5min":1.835,"15min":1.835},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.623,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.569,"5min":1.734,"15min":1.8979999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.93400000000000005,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.55100000000000005,"15min":0.61399999999999999},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.88100000000000001,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.63400000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.64300000000000002},"min":{"1min":0.32500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.1579999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.294},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65000000000000002,"15min":0.71699999999999997},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.022,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83199999999999996,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.071,"5min":1.526,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.91500000000000004},"min":{"1min":0.52300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82899999999999996,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.248,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.2390000000000001,"5min":1.486,"15min":1.5780000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.381,"15min":1.615},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.94499999999999995},"min":{"1min":0.66100000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.048}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.91400000000000003,"15min":0.97699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3109999999999999,"5min":1.4650000000000001,"15min":4.2809999999999997},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88800000000000001,"15min":0.94199999999999995},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.4740000000000002},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325372,"statfs":{"total":96636764160,"available":96589180928,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.0569999999999999,"5min":1.843,"15min":1.843},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.81200000000000006,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.56499999999999995,"15min":0.70299999999999996},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.76300000000000001,"5min":1.169,"15min":30.861000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.66700000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.0329999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0369999999999999,"5min":1.151,"15min":2.3079999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.82099999999999995,"15min":0.85999999999999999},"min":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.198,"5min":2.0870000000000002,"15min":2.258},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.82499999999999996,"15min":0.92900000000000005},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":1.5549999999999999,"15min":14.753},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":1.101},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47299999999999998},"max":{"1min":1.1499999999999999,"5min":1.528,"15min":43.103999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.92700000000000005,"15min":1.133},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.208,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.88,"15min":1.0409999999999999},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.45100000000000001},"max":{"1min":1.2909999999999999,"5min":1.5189999999999999,"15min":23.971},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":1.0629999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0389999999999999},"min":{"1min":0.67600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.212,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95699999999999996,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.319,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280496,"statfs":{"total":96636764160,"available":96543227904,"internally_reserved":0,"allocated":2105344,"data_stored":1285897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.27500000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.80500000000000005,"5min":1.145,"15min":2.6989999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59999999999999998,"15min":0.64900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.64600000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.82399999999999995,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.872}]},{"osd":3,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.391,"5min":1.391,"15min":2.7999999999999998},"last":0.90700000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.91800000000000004},"min":{"1min":0.54700000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.371,"5min":1.371,"15min":4.641},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.254,"15min":2.028},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.96399999999999997},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.0920000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":1.244},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.93999999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.147}]},{"osd":6,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57399999999999995,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.96399999999999997,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93100000000000005,"15min":0.996},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":1.266,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.101},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.93799999999999994,"15min":0.97999999999999998},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.335,"5min":1.4670000000000001,"15min":1.97},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:13:43.751 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:13:43.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 Processor -- start 2023-11-06T03:13:43.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 -- start start 2023-11-06T03:13:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f805c15caa0 0x7f805c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f805c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805c0d2920 con 0x7f805c0ce1a0 2023-11-06T03:13:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c0d18a0 con 0x7f805c15caa0 2023-11-06T03:13:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f80613c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0d1b60 con 0x7f805c15ce80 2023-11-06T03:13:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f805c0ce1a0 0x7f805c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54972/0 (socket says 172.21.15.157:54972) 2023-11-06T03:13:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 -- 172.21.15.157:0/3326663535 learned_addr learned my addr 172.21.15.157:0/3326663535 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805affd640 1 --2- 172.21.15.157:0/3326663535 >> v2:172.21.15.195:3300/0 conn(0x7f805c15caa0 0x7f805c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805a7fc640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f805c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238764514 0 0) 0x7f805c0d2920 con 0x7f805c0ce1a0 2023-11-06T03:13:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8044003530 con 0x7f805c0ce1a0 2023-11-06T03:13:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088890044 0 0) 0x7f8044003530 con 0x7f805c0ce1a0 2023-11-06T03:13:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 msgr2=0x7f805c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f805c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 >> v2:172.21.15.195:3300/0 conn(0x7f805c15caa0 msgr2=0x7f805c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 --2- 172.21.15.157:0/3326663535 >> v2:172.21.15.195:3300/0 conn(0x7f805c15caa0 0x7f805c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0d1e40 con 0x7f805c0ce1a0 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8054001a00 con 0x7f805c0ce1a0 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 >> v1:172.21.15.157:6789/0 conn(0x7f805c0ce1a0 legacy=0x7f805c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f8044003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044003f00 0x7f80440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8044004d10 0x7f8044005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0d18a0 con 0x7f8044004d10 2023-11-06T03:13:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0d1b60 con 0x7f8044003f00 2023-11-06T03:13:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804400b990 con 0x7f805c15ce80 2023-11-06T03:13:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8044004d10 0x7f8044005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805affd640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f8044003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805a7fc640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044003f00 0x7f80440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 -- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 msgr2=0x7f8044003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f805c15ce80 0x7f8044003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 -- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044003f00 msgr2=0x7f80440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8044003f00 0x7f80440047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.913+0000 7f805b7fe640 1 -- 172.21.15.157:0/3326663535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044003530 con 0x7f8044004d10 2023-11-06T03:13:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f805b7fe640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8044004d10 0x7f8044005190 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f805400a830 tx=0x7f805401d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e5ebaa4a0f143e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540ad630 con 0x7f8044004d10 2023-11-06T03:13:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80540a8070 con 0x7f8044004d10 2023-11-06T03:13:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f8059ffb640 1 -- 172.21.15.157:0/3326663535 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540ada50 con 0x7f8044004d10 2023-11-06T03:13:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- 172.21.15.157:0/3326663535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8044004d10 msgr2=0x7f8044005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 --2- 172.21.15.157:0/3326663535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8044004d10 0x7f8044005190 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f805400a830 tx=0x7f805401d2e0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f805b7fe640 1 -- 172.21.15.157:0/3326663535 reap_dead start 2023-11-06T03:13:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- 172.21.15.157:0/3326663535 shutdown_connections 2023-11-06T03:13:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- 172.21.15.157:0/3326663535 >> 172.21.15.157:0/3326663535 conn(0x7f805c070790 msgr2=0x7f805c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- 172.21.15.157:0/3326663535 shutdown_connections 2023-11-06T03:13:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- 172.21.15.157:0/3326663535 wait complete. 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 Processor -- start 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.917+0000 7f80613c8640 1 -- start start 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8044004d10 0x7f805c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f805c1f57b0 0x7f805c1f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c158cc0 con 0x7f805c1f6170 2023-11-06T03:13:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c1f65f0 con 0x7f805c1f57b0 2023-11-06T03:13:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0cc780 con 0x7f8044004d10 2023-11-06T03:13:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55134/0 (socket says 172.21.15.157:55134) 2023-11-06T03:13:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 -- 172.21.15.157:0/170968085 learned_addr learned my addr 172.21.15.157:0/170968085 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805a7fc640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f805c1f57b0 0x7f805c1f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805affd640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8044004d10 0x7f805c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 -- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8044004d10 msgr2=0x7f805c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8044004d10 0x7f805c157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 -- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f805c1f57b0 msgr2=0x7f805c1f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f805c1f57b0 0x7f805c1f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 -- 172.21.15.157:0/170968085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0cca60 con 0x7f805c1f6170 2023-11-06T03:13:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805b7fe640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f805401cd50 tx=0x7f8044005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540017c0 con 0x7f805c1f6170 2023-11-06T03:13:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8054001920 con 0x7f805c1f6170 2023-11-06T03:13:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c0ccd40 con 0x7f805c1f6170 2023-11-06T03:13:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540ad630 con 0x7f805c1f6170 2023-11-06T03:13:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c1581f0 con 0x7f805c1f6170 2023-11-06T03:13:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c0b1ea0 con 0x7f805c1f6170 2023-11-06T03:13:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80540aa030 con 0x7f805c1f6170 2023-11-06T03:13:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 0x7f8048067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f805411cec0 con 0x7f805c1f6170 2023-11-06T03:13:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805affd640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 0x7f8048067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.921+0000 7f805affd640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 0x7f8048067f20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f805c0cb930 tx=0x7f804c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:43.925+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80540ea720 con 0x7f805c1f6170 2023-11-06T03:13:44.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.069+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f805c0d2400 con 0x7f8048065a60 2023-11-06T03:13:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.089+0000 7f80437fe640 1 -- 172.21.15.157:0/170968085 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299907 (secure 0 0 0) 0x7f804c012950 con 0x7f8048065a60 2023-11-06T03:13:44.090 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:13:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 msgr2=0x7f8048067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 0x7f8048067f20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f805c0cb930 tx=0x7f804c008040 comp rx=0 tx=0).stop 2023-11-06T03:13:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 msgr2=0x7f805c159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f805401cd50 tx=0x7f8044005800 comp rx=0 tx=0).stop 2023-11-06T03:13:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 shutdown_connections 2023-11-06T03:13:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8048065a60 0x7f8048067f20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f805c1f6170 0x7f805c159150 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f805c1f57b0 0x7f805c1f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 --2- 172.21.15.157:0/170968085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8044004d10 0x7f805c157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 >> 172.21.15.157:0/170968085 conn(0x7f805c070790 msgr2=0x7f805c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 shutdown_connections 2023-11-06T03:13:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.093+0000 7f80613c8640 1 -- 172.21.15.157:0/170968085 wait complete. 2023-11-06T03:13:44.107 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-11-06T03:13:43.305114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584268,"kb_used_data":12940,"kb_used_omap":0,"kb_used_meta":571264,"kb_avail":754390452,"statfs":{"total":773094113280,"available":772495822848,"internally_reserved":0,"allocated":13250560,"data_stored":6725383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":584974336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277974"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039944+0000","last_change":"2023-11-06T03:13:39.039944+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039944+0000","last_undegraded":"2023-11-06T03:13:39.039944+0000","last_fullsized":"2023-11-06T03:13:39.039944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034813+0000","last_change":"2023-11-06T03:13:39.034813+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034813+0000","last_undegraded":"2023-11-06T03:13:39.034813+0000","last_fullsized":"2023-11-06T03:13:39.034813+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037617+0000","last_change":"2023-11-06T03:13:39.037617+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037617+0000","last_undegraded":"2023-11-06T03:13:39.037617+0000","last_fullsized":"2023-11-06T03:13:39.037617+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034124+0000","last_change":"2023-11-06T03:13:39.034124+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034124+0000","last_undegraded":"2023-11-06T03:13:39.034124+0000","last_fullsized":"2023-11-06T03:13:39.034124+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.031025+0000","last_change":"2023-11-06T03:13:39.031025+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.031025+0000","last_undegraded":"2023-11-06T03:13:39.031025+0000","last_fullsized":"2023-11-06T03:13:39.031025+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030039+0000","last_change":"2023-11-06T03:13:39.030039+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030039+0000","last_undegraded":"2023-11-06T03:13:39.030039+0000","last_fullsized":"2023-11-06T03:13:39.030039+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033922+0000","last_change":"2023-11-06T03:13:39.033922+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033922+0000","last_undegraded":"2023-11-06T03:13:39.033922+0000","last_fullsized":"2023-11-06T03:13:39.033922+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030067+0000","last_change":"2023-11-06T03:13:39.030067+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030067+0000","last_undegraded":"2023-11-06T03:13:39.030067+0000","last_fullsized":"2023-11-06T03:13:39.030067+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033625+0000","last_change":"2023-11-06T03:13:39.033625+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033625+0000","last_undegraded":"2023-11-06T03:13:39.033625+0000","last_fullsized":"2023-11-06T03:13:39.033625+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032230+0000","last_change":"2023-11-06T03:13:39.032230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032230+0000","last_undegraded":"2023-11-06T03:13:39.032230+0000","last_fullsized":"2023-11-06T03:13:39.032230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.036701+0000","last_change":"2023-11-06T03:13:39.036701+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.036701+0000","last_undegraded":"2023-11-06T03:13:39.036701+0000","last_fullsized":"2023-11-06T03:13:39.036701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035894+0000","last_change":"2023-11-06T03:13:39.035894+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035894+0000","last_undegraded":"2023-11-06T03:13:39.035894+0000","last_fullsized":"2023-11-06T03:13:39.035894+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.029843+0000","last_change":"2023-11-06T03:13:39.029843+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.029843+0000","last_undegraded":"2023-11-06T03:13:39.029843+0000","last_fullsized":"2023-11-06T03:13:39.029843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035815+0000","last_change":"2023-11-06T03:13:39.035815+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035815+0000","last_undegraded":"2023-11-06T03:13:39.035815+0000","last_fullsized":"2023-11-06T03:13:39.035815+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032991+0000","last_change":"2023-11-06T03:13:39.032991+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032991+0000","last_undegraded":"2023-11-06T03:13:39.032991+0000","last_fullsized":"2023-11-06T03:13:39.032991+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037034+0000","last_change":"2023-11-06T03:13:39.037034+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037034+0000","last_undegraded":"2023-11-06T03:13:39.037034+0000","last_fullsized":"2023-11-06T03:13:39.037034+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030155+0000","last_change":"2023-11-06T03:13:39.030155+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030155+0000","last_undegraded":"2023-11-06T03:13:39.030155+0000","last_fullsized":"2023-11-06T03:13:39.030155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039010+0000","last_change":"2023-11-06T03:13:39.039010+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039010+0000","last_undegraded":"2023-11-06T03:13:39.039010+0000","last_fullsized":"2023-11-06T03:13:39.039010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039072+0000","last_change":"2023-11-06T03:13:39.039072+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039072+0000","last_undegraded":"2023-11-06T03:13:39.039072+0000","last_fullsized":"2023-11-06T03:13:39.039072+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030042+0000","last_change":"2023-11-06T03:13:39.030042+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030042+0000","last_undegraded":"2023-11-06T03:13:39.030042+0000","last_fullsized":"2023-11-06T03:13:39.030042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030197+0000","last_change":"2023-11-06T03:13:39.030197+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030197+0000","last_undegraded":"2023-11-06T03:13:39.030197+0000","last_fullsized":"2023-11-06T03:13:39.030197+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030538+0000","last_change":"2023-11-06T03:13:39.030538+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030538+0000","last_undegraded":"2023-11-06T03:13:39.030538+0000","last_fullsized":"2023-11-06T03:13:39.030538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030571+0000","last_change":"2023-11-06T03:13:39.030571+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030571+0000","last_undegraded":"2023-11-06T03:13:39.030571+0000","last_fullsized":"2023-11-06T03:13:39.030571+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035852+0000","last_change":"2023-11-06T03:13:39.035852+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035852+0000","last_undegraded":"2023-11-06T03:13:39.035852+0000","last_fullsized":"2023-11-06T03:13:39.035852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032867+0000","last_change":"2023-11-06T03:13:39.032867+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032867+0000","last_undegraded":"2023-11-06T03:13:39.032867+0000","last_fullsized":"2023-11-06T03:13:39.032867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116943+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:13:37.116943+0000","last_peered":"2023-11-06T03:13:37.116943+0000","last_clean":"2023-11-06T03:13:37.116943+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:13:37.116943+0000","last_undegraded":"2023-11-06T03:13:37.116943+0000","last_fullsized":"2023-11-06T03:13:37.116943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114100+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:13:37.114100+0000","last_peered":"2023-11-06T03:13:37.114100+0000","last_clean":"2023-11-06T03:13:37.114100+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:13:37.114100+0000","last_undegraded":"2023-11-06T03:13:37.114100+0000","last_fullsized":"2023-11-06T03:13:37.114100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033827+0000","last_change":"2023-11-06T03:13:39.033827+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033827+0000","last_undegraded":"2023-11-06T03:13:39.033827+0000","last_fullsized":"2023-11-06T03:13:39.033827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033218+0000","last_change":"2023-11-06T03:13:39.033218+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033218+0000","last_undegraded":"2023-11-06T03:13:39.033218+0000","last_fullsized":"2023-11-06T03:13:39.033218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.113041+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:13:37.113041+0000","last_peered":"2023-11-06T03:13:37.113041+0000","last_clean":"2023-11-06T03:13:37.113041+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:13:37.113041+0000","last_undegraded":"2023-11-06T03:13:37.113041+0000","last_fullsized":"2023-11-06T03:13:37.113041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034529+0000","last_change":"2023-11-06T03:13:39.034529+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034529+0000","last_undegraded":"2023-11-06T03:13:39.034529+0000","last_fullsized":"2023-11-06T03:13:39.034529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114094+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:13:37.114094+0000","last_peered":"2023-11-06T03:13:37.114094+0000","last_clean":"2023-11-06T03:13:37.114094+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:13:37.114094+0000","last_undegraded":"2023-11-06T03:13:37.114094+0000","last_fullsized":"2023-11-06T03:13:37.114094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039011+0000","last_change":"2023-11-06T03:13:39.039011+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039011+0000","last_undegraded":"2023-11-06T03:13:39.039011+0000","last_fullsized":"2023-11-06T03:13:39.039011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111589+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:13:37.111589+0000","last_peered":"2023-11-06T03:13:37.111589+0000","last_clean":"2023-11-06T03:13:37.111589+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:13:37.111589+0000","last_undegraded":"2023-11-06T03:13:37.111589+0000","last_fullsized":"2023-11-06T03:13:37.111589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034230+0000","last_change":"2023-11-06T03:13:39.034230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034230+0000","last_undegraded":"2023-11-06T03:13:39.034230+0000","last_fullsized":"2023-11-06T03:13:39.034230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033916+0000","last_change":"2023-11-06T03:13:39.033916+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033916+0000","last_undegraded":"2023-11-06T03:13:39.033916+0000","last_fullsized":"2023-11-06T03:13:39.033916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111653+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:13:37.111653+0000","last_peered":"2023-11-06T03:13:37.111653+0000","last_clean":"2023-11-06T03:13:37.111653+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:13:37.111653+0000","last_undegraded":"2023-11-06T03:13:37.111653+0000","last_fullsized":"2023-11-06T03:13:37.111653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032841+0000","last_change":"2023-11-06T03:13:39.032841+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032841+0000","last_undegraded":"2023-11-06T03:13:39.032841+0000","last_fullsized":"2023-11-06T03:13:39.032841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.115547+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:13:37.115547+0000","last_peered":"2023-11-06T03:13:37.115547+0000","last_clean":"2023-11-06T03:13:37.115547+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:13:37.115547+0000","last_undegraded":"2023-11-06T03:13:37.115547+0000","last_fullsized":"2023-11-06T03:13:37.115547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":2109440,"data_stored":1286916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1830000000000001,"15min":1.4830000000000001},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.85999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.3620000000000001,"15min":1.397},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.147,"5min":1.2490000000000001,"15min":1.444},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.153,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.90700000000000003,"15min":0.88700000000000001},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1539999999999999,"5min":1.381,"15min":1.6080000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.22,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":2.512,"15min":2.512},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.097,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":10.596,"15min":10.596},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.71799999999999997,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.5309999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.99199999999999999,"15min":1.0009999999999999},"min":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.278,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0600000000000001,"15min":1.048},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3720000000000001,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.349,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.123}]},{"osd":6,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.169,"15min":1.1539999999999999},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.454,"5min":2.548,"15min":2.548},"last":1.198},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.1040000000000001,"15min":1.105},"min":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.3049999999999999,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.05}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64460,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307380,"statfs":{"total":96636764160,"available":96570757120,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.97299999999999998,"15min":0.84999999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.081,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.2749999999999999,"15min":0.97299999999999998},"min":{"1min":0.50700000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.0269999999999999,"15min":0.90100000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1359999999999999,"5min":16.012,"15min":16.012},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.1279999999999999,"15min":0.91300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.105,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.94699999999999995},"min":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1619999999999999,"5min":2.843,"15min":2.843},"last":1.2410000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.647,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.2929999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.877,"5min":1.355,"15min":1.052},"min":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.278,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.335,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2609999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.413,"15min":1.2370000000000001},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3540000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.367}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.411,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.294,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1379999999999999,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.3240000000000001,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.399}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.4079999999999999,"15min":1.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.429,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.3999999999999999,"15min":1.212},"min":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.371,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.204}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.7729999999999999,"15min":1.1299999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.0840000000000001,"5min":125.256,"15min":125.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.8129999999999999,"15min":1.159},"min":{"1min":0.495,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.028,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.78000000000000003},"min":{"1min":0.60799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.27,"5min":2.339,"15min":2.339},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.57199999999999995,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.53500000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1799999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0640000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91300000000000003,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.208,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.94499999999999995},"min":{"1min":0.60099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3400000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.66500000000000004,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5149999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.044,"15min":0.98699999999999999},"min":{"1min":0.76200000000000001,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":3.0030000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.008}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0349999999999999,"15min":1.01},"min":{"1min":0.70699999999999996,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.429,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0620000000000001,"15min":1.0249999999999999},"min":{"1min":0.79200000000000004,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3600000000000001,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0960000000000001,"15min":1.0249999999999999},"min":{"1min":0.85599999999999998,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.421,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.94299999999999995,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.514,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90692,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281148,"statfs":{"total":96636764160,"available":96543895552,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.26,"15min":0.96599999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.372},"max":{"1min":1.0429999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.2929999999999999,"15min":0.95899999999999996},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.99099999999999999,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.999}]},{"osd":1,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.51400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1279999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.032,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":1.385}]},{"osd":2,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1779999999999999,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.042,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.59399999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":3.198,"15min":3.198},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.64800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0960000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.081}]},{"osd":5,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0609999999999999,"15min":1.0269999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.143,"5min":10.192,"15min":10.192},"last":1.254},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0600000000000001,"15min":1.089},"min":{"1min":0.80500000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.415}]},{"osd":6,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96099999999999997,"15min":0.998},"min":{"1min":0.66400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.514,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.1140000000000001},"min":{"1min":0.71499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.399,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.512}]},{"osd":7,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.077,"15min":1.075},"min":{"1min":0.76800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.171},"min":{"1min":0.79600000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4099999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.472}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76812,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295028,"statfs":{"total":96636764160,"available":96558108672,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78200000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":1.524},"last":0.313}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.26400000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.89200000000000002,"5min":1.371,"15min":1.371},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93100000000000005,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.23300000000000001},"max":{"1min":0.95099999999999996,"5min":1.532,"15min":1.532},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.34599999999999997,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.206,"5min":1.284,"15min":1.284},"last":0.65000000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45000000000000001},"max":{"1min":1.2270000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.91100000000000003,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.369,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.621}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.626,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.536,"15min":1.6639999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.698},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.999},"min":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.3939999999999999,"5min":1.649,"15min":1.8620000000000001},"last":0.748},{"interface":"front","average":{"1min":0.872,"5min":0.94699999999999995,"15min":0.97999999999999998},"min":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3129999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":1.03},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4410000000000001,"5min":1.835,"15min":1.835},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.623,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.569,"5min":1.734,"15min":1.8979999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.93400000000000005,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.55100000000000005,"15min":0.61399999999999999},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.88100000000000001,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.63400000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.64300000000000002},"min":{"1min":0.32500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.1579999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.294},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65000000000000002,"15min":0.71699999999999997},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.022,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83199999999999996,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.071,"5min":1.526,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.91500000000000004},"min":{"1min":0.52300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82899999999999996,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.248,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.2390000000000001,"5min":1.486,"15min":1.5780000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.381,"15min":1.615},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.94499999999999995},"min":{"1min":0.66100000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.048}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.91400000000000003,"15min":0.97699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3109999999999999,"5min":1.4650000000000001,"15min":4.2809999999999997},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88800000000000001,"15min":0.94199999999999995},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.4740000000000002},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325372,"statfs":{"total":96636764160,"available":96589180928,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.0569999999999999,"5min":1.843,"15min":1.843},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.81200000000000006,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.56499999999999995,"15min":0.70299999999999996},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.76300000000000001,"5min":1.169,"15min":30.861000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.66700000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.0329999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0369999999999999,"5min":1.151,"15min":2.3079999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.82099999999999995,"15min":0.85999999999999999},"min":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.198,"5min":2.0870000000000002,"15min":2.258},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.82499999999999996,"15min":0.92900000000000005},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":1.5549999999999999,"15min":14.753},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":1.101},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47299999999999998},"max":{"1min":1.1499999999999999,"5min":1.528,"15min":43.103999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.92700000000000005,"15min":1.133},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.208,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.88,"15min":1.0409999999999999},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.45100000000000001},"max":{"1min":1.2909999999999999,"5min":1.5189999999999999,"15min":23.971},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":1.0629999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0389999999999999},"min":{"1min":0.67600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.212,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95699999999999996,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.319,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280496,"statfs":{"total":96636764160,"available":96543227904,"internally_reserved":0,"allocated":2105344,"data_stored":1285897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.27500000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.80500000000000005,"5min":1.145,"15min":2.6989999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59999999999999998,"15min":0.64900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.64600000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.82399999999999995,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.872}]},{"osd":3,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.391,"5min":1.391,"15min":2.7999999999999998},"last":0.90700000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.91800000000000004},"min":{"1min":0.54700000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.371,"5min":1.371,"15min":4.641},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.254,"15min":2.028},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.96399999999999997},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.0920000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":1.244},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.93999999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.147}]},{"osd":6,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57399999999999995,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.96399999999999997,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93100000000000005,"15min":0.996},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":1.266,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.101},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.93799999999999994,"15min":0.97999999999999998},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.335,"5min":1.4670000000000001,"15min":1.97},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:13:44.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2023-11-06T03:13:44.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2023-11-06T03:13:44.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2023-11-06T03:13:44.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2023-11-06T03:13:44.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2023-11-06T03:13:44.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2023-11-06T03:13:44.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2023-11-06T03:13:44.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2023-11-06T03:13:44.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2023-11-06T03:13:44.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2023-11-06T03:13:44.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2023-11-06T03:13:44.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2023-11-06T03:13:44.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2023-11-06T03:13:44.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2023-11-06T03:13:44.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2023-11-06T03:13:44.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2023-11-06T03:13:44.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2023-11-06T03:13:44.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2023-11-06T03:13:44.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2023-11-06T03:13:44.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.128 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2023-11-06T03:13:44.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.129 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2023-11-06T03:13:44.129 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2023-11-06T03:13:44.130 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.130 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2023-11-06T03:13:44.131 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.131 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2023-11-06T03:13:44.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2023-11-06T03:13:44.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2023-11-06T03:13:44.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2023-11-06T03:13:44.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2023-11-06T03:13:44.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2023-11-06T03:13:44.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2023-11-06T03:13:44.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2023-11-06T03:13:44.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2023-11-06T03:13:44.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2023-11-06T03:13:44.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-11-06T03:13:44.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.039944+0000', 'last_change': '2023-11-06T03:13:39.039944+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.039944+0000', 'last_undegraded': '2023-11-06T03:13:39.039944+0000', 'last_fullsized': '2023-11-06T03:13:39.039944+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:13:44.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-11-06T03:13:44.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-11-06T03:13:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.034813+0000', 'last_change': '2023-11-06T03:13:39.034813+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.034813+0000', 'last_undegraded': '2023-11-06T03:13:39.034813+0000', 'last_fullsized': '2023-11-06T03:13:39.034813+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:13:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2023-11-06T03:13:44.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-11-06T03:13:44.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.037617+0000', 'last_change': '2023-11-06T03:13:39.037617+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.037617+0000', 'last_undegraded': '2023-11-06T03:13:39.037617+0000', 'last_fullsized': '2023-11-06T03:13:39.037617+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-11-06T03:13:44.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-11-06T03:13:44.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.034124+0000', 'last_change': '2023-11-06T03:13:39.034124+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.034124+0000', 'last_undegraded': '2023-11-06T03:13:39.034124+0000', 'last_fullsized': '2023-11-06T03:13:39.034124+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:13:44.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2023-11-06T03:13:44.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-11-06T03:13:44.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.031025+0000', 'last_change': '2023-11-06T03:13:39.031025+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.031025+0000', 'last_undegraded': '2023-11-06T03:13:39.031025+0000', 'last_fullsized': '2023-11-06T03:13:39.031025+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:13:44.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2023-11-06T03:13:44.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-11-06T03:13:44.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030039+0000', 'last_change': '2023-11-06T03:13:39.030039+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030039+0000', 'last_undegraded': '2023-11-06T03:13:39.030039+0000', 'last_fullsized': '2023-11-06T03:13:39.030039+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2023-11-06T03:13:44.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-11-06T03:13:44.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.033922+0000', 'last_change': '2023-11-06T03:13:39.033922+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.033922+0000', 'last_undegraded': '2023-11-06T03:13:39.033922+0000', 'last_fullsized': '2023-11-06T03:13:39.033922+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:13:44.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2023-11-06T03:13:44.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-11-06T03:13:44.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030067+0000', 'last_change': '2023-11-06T03:13:39.030067+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030067+0000', 'last_undegraded': '2023-11-06T03:13:39.030067+0000', 'last_fullsized': '2023-11-06T03:13:39.030067+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2023-11-06T03:13:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-11-06T03:13:44.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.033625+0000', 'last_change': '2023-11-06T03:13:39.033625+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.033625+0000', 'last_undegraded': '2023-11-06T03:13:39.033625+0000', 'last_fullsized': '2023-11-06T03:13:39.033625+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-06T03:13:44.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-11-06T03:13:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-11-06T03:13:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.032230+0000', 'last_change': '2023-11-06T03:13:39.032230+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.032230+0000', 'last_undegraded': '2023-11-06T03:13:39.032230+0000', 'last_fullsized': '2023-11-06T03:13:39.032230+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-06T03:13:44.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2023-11-06T03:13:44.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-11-06T03:13:44.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.036701+0000', 'last_change': '2023-11-06T03:13:39.036701+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.036701+0000', 'last_undegraded': '2023-11-06T03:13:39.036701+0000', 'last_fullsized': '2023-11-06T03:13:39.036701+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2023-11-06T03:13:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-11-06T03:13:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.035894+0000', 'last_change': '2023-11-06T03:13:39.035894+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.035894+0000', 'last_undegraded': '2023-11-06T03:13:39.035894+0000', 'last_fullsized': '2023-11-06T03:13:39.035894+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-11-06T03:13:44.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-11-06T03:13:44.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.029843+0000', 'last_change': '2023-11-06T03:13:39.029843+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.029843+0000', 'last_undegraded': '2023-11-06T03:13:39.029843+0000', 'last_fullsized': '2023-11-06T03:13:39.029843+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2023-11-06T03:13:44.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-11-06T03:13:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.035815+0000', 'last_change': '2023-11-06T03:13:39.035815+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.035815+0000', 'last_undegraded': '2023-11-06T03:13:39.035815+0000', 'last_fullsized': '2023-11-06T03:13:39.035815+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2023-11-06T03:13:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-11-06T03:13:44.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.032991+0000', 'last_change': '2023-11-06T03:13:39.032991+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.032991+0000', 'last_undegraded': '2023-11-06T03:13:39.032991+0000', 'last_fullsized': '2023-11-06T03:13:39.032991+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2023-11-06T03:13:44.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-11-06T03:13:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.037034+0000', 'last_change': '2023-11-06T03:13:39.037034+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.037034+0000', 'last_undegraded': '2023-11-06T03:13:39.037034+0000', 'last_fullsized': '2023-11-06T03:13:39.037034+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-11-06T03:13:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-11-06T03:13:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030155+0000', 'last_change': '2023-11-06T03:13:39.030155+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030155+0000', 'last_undegraded': '2023-11-06T03:13:39.030155+0000', 'last_fullsized': '2023-11-06T03:13:39.030155+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2023-11-06T03:13:44.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-11-06T03:13:44.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.039010+0000', 'last_change': '2023-11-06T03:13:39.039010+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.039010+0000', 'last_undegraded': '2023-11-06T03:13:39.039010+0000', 'last_fullsized': '2023-11-06T03:13:39.039010+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:13:44.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-11-06T03:13:44.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-11-06T03:13:44.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.039072+0000', 'last_change': '2023-11-06T03:13:39.039072+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.039072+0000', 'last_undegraded': '2023-11-06T03:13:39.039072+0000', 'last_fullsized': '2023-11-06T03:13:39.039072+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:13:44.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2023-11-06T03:13:44.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-11-06T03:13:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030042+0000', 'last_change': '2023-11-06T03:13:39.030042+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030042+0000', 'last_undegraded': '2023-11-06T03:13:39.030042+0000', 'last_fullsized': '2023-11-06T03:13:39.030042+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-11-06T03:13:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-11-06T03:13:44.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030197+0000', 'last_change': '2023-11-06T03:13:39.030197+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030197+0000', 'last_undegraded': '2023-11-06T03:13:39.030197+0000', 'last_fullsized': '2023-11-06T03:13:39.030197+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:13:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2023-11-06T03:13:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-11-06T03:13:44.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030538+0000', 'last_change': '2023-11-06T03:13:39.030538+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030538+0000', 'last_undegraded': '2023-11-06T03:13:39.030538+0000', 'last_fullsized': '2023-11-06T03:13:39.030538+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-06T03:13:44.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-11-06T03:13:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-11-06T03:13:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.030571+0000', 'last_change': '2023-11-06T03:13:39.030571+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.030571+0000', 'last_undegraded': '2023-11-06T03:13:39.030571+0000', 'last_fullsized': '2023-11-06T03:13:39.030571+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-06T03:13:44.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2023-11-06T03:13:44.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-11-06T03:13:44.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.035852+0000', 'last_change': '2023-11-06T03:13:39.035852+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.035852+0000', 'last_undegraded': '2023-11-06T03:13:39.035852+0000', 'last_fullsized': '2023-11-06T03:13:39.035852+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-06T03:13:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-11-06T03:13:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-11-06T03:13:44.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.032867+0000', 'last_change': '2023-11-06T03:13:39.032867+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.032867+0000', 'last_undegraded': '2023-11-06T03:13:39.032867+0000', 'last_fullsized': '2023-11-06T03:13:39.032867+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2023-11-06T03:13:44.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-11-06T03:13:44.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.033827+0000', 'last_change': '2023-11-06T03:13:39.033827+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.033827+0000', 'last_undegraded': '2023-11-06T03:13:39.033827+0000', 'last_fullsized': '2023-11-06T03:13:39.033827+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2023-11-06T03:13:44.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-11-06T03:13:44.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.033218+0000', 'last_change': '2023-11-06T03:13:39.033218+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.033218+0000', 'last_undegraded': '2023-11-06T03:13:39.033218+0000', 'last_fullsized': '2023-11-06T03:13:39.033218+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2023-11-06T03:13:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-11-06T03:13:44.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.034529+0000', 'last_change': '2023-11-06T03:13:39.034529+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.034529+0000', 'last_undegraded': '2023-11-06T03:13:39.034529+0000', 'last_fullsized': '2023-11-06T03:13:39.034529+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2023-11-06T03:13:44.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-11-06T03:13:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.039011+0000', 'last_change': '2023-11-06T03:13:39.039011+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.039011+0000', 'last_undegraded': '2023-11-06T03:13:39.039011+0000', 'last_fullsized': '2023-11-06T03:13:39.039011+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-06T03:13:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2023-11-06T03:13:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-11-06T03:13:44.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.034230+0000', 'last_change': '2023-11-06T03:13:39.034230+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.034230+0000', 'last_undegraded': '2023-11-06T03:13:39.034230+0000', 'last_fullsized': '2023-11-06T03:13:39.034230+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:13:44.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2023-11-06T03:13:44.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-11-06T03:13:44.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.033916+0000', 'last_change': '2023-11-06T03:13:39.033916+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.033916+0000', 'last_undegraded': '2023-11-06T03:13:39.033916+0000', 'last_fullsized': '2023-11-06T03:13:39.033916+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-06T03:13:44.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2023-11-06T03:13:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:13:40.023544+0000', 'last_change': '2023-11-06T03:13:40.023544+0000', 'last_active': '2023-11-06T03:13:40.023544+0000', 'last_peered': '2023-11-06T03:13:40.023544+0000', 'last_clean': '2023-11-06T03:13:40.023544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:40.023544+0000', 'last_undegraded': '2023-11-06T03:13:40.023544+0000', 'last_fullsized': '2023-11-06T03:13:40.023544+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:40.023544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:13:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-11-06T03:13:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 122, 'state': 'creating+peering', 'last_fresh': '2023-11-06T03:13:39.032841+0000', 'last_change': '2023-11-06T03:13:39.032841+0000', 'last_active': '2023-11-06T03:13:39.008912+0000', 'last_peered': '2023-11-06T03:13:39.008912+0000', 'last_clean': '2023-11-06T03:13:39.008912+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:13:39.032841+0000', 'last_undegraded': '2023-11-06T03:13:39.032841+0000', 'last_fullsized': '2023-11-06T03:13:39.032841+0000', 'mapping_epoch': 122, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 122, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'last_clean_scrub_stamp': '2023-11-06T03:13:39.008912+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-06T03:13:44.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:13:44.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 Processor -- start 2023-11-06T03:13:44.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 -- start start 2023-11-06T03:13:44.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4fc41551e0 0x7f4fc406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 0x7f4fc415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc415f580 con 0x7f4fc415e5e0 2023-11-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc415efc0 con 0x7f4fc41551e0 2023-11-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.317+0000 7f4fcb088640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc415f2a0 con 0x7f4fc40b3b20 2023-11-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4fc41551e0 0x7f4fc406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc9625640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 0x7f4fc415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4fc41551e0 0x7f4fc406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:52522/0 (socket says 172.21.15.157:52522) 2023-11-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 -- 172.21.15.157:0/850421249 learned_addr learned my addr 172.21.15.157:0/850421249 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc3fff640 1 --1- 172.21.15.157:0/850421249 >> v1:172.21.15.195:6789/0 conn(0x7f4fc415e5e0 0x7f4fc40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56792/0 (socket says 172.21.15.157:56792) 2023-11-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 -- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 msgr2=0x7f4fc415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 0x7f4fc415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 -- 172.21.15.157:0/850421249 >> v1:172.21.15.195:6789/0 conn(0x7f4fc415e5e0 legacy=0x7f4fc40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc37fe640 1 -- 172.21.15.157:0/850421249 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010638439 0 0) 0x7f4fc415f580 con 0x7f4fc415e5e0 2023-11-06T03:13:44.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 -- 172.21.15.157:0/850421249 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc406f5a0 con 0x7f4fc41551e0 2023-11-06T03:13:44.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc9625640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 0x7f4fc415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:13:44.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc8e24640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc41551e0 0x7f4fc406fda0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009870 tx=0x7f4fb801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af990bec15728 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc37fe640 1 -- 172.21.15.157:0/850421249 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80a6040 con 0x7f4fc41551e0 2023-11-06T03:13:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc37fe640 1 -- 172.21.15.157:0/850421249 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb80aa600 con 0x7f4fc41551e0 2023-11-06T03:13:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.321+0000 7f4fc37fe640 1 -- 172.21.15.157:0/850421249 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80aa990 con 0x7f4fc41551e0 2023-11-06T03:13:44.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc41551e0 msgr2=0x7f4fc406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc41551e0 0x7f4fc406fda0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009870 tx=0x7f4fb801c480 comp rx=0 tx=0).stop 2023-11-06T03:13:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/850421249 shutdown_connections 2023-11-06T03:13:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc40b3b20 0x7f4fc415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- 172.21.15.157:0/850421249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc41551e0 0x7f4fc406fda0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/850421249 >> 172.21.15.157:0/850421249 conn(0x7f4fc4065930 msgr2=0x7f4fc4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/850421249 shutdown_connections 2023-11-06T03:13:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/850421249 wait complete. 2023-11-06T03:13:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 Processor -- start 2023-11-06T03:13:44.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- start start 2023-11-06T03:13:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc40b3b20 0x7f4fc4145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc415e5e0 0x7f4fc415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4145dc0 con 0x7f4fc41551e0 2023-11-06T03:13:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40cca20 con 0x7f4fc40b3b20 2023-11-06T03:13:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc414e750 con 0x7f4fc415e5e0 2023-11-06T03:13:44.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55140/0 (socket says 172.21.15.157:55140) 2023-11-06T03:13:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 -- 172.21.15.157:0/2735192088 learned_addr learned my addr 172.21.15.157:0/2735192088 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc8e24640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc40b3b20 0x7f4fc4145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc9625640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc415e5e0 0x7f4fc415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 -- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc415e5e0 msgr2=0x7f4fc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc415e5e0 0x7f4fc415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 -- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc40b3b20 msgr2=0x7f4fc4145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc40b3b20 0x7f4fc4145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 -- 172.21.15.157:0/2735192088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc009a70 con 0x7f4fc41551e0 2023-11-06T03:13:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc3fff640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c4b0 tx=0x7f4fb000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00074e0 con 0x7f4fc41551e0 2023-11-06T03:13:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0007660 con 0x7f4fc41551e0 2023-11-06T03:13:44.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb80094d0 con 0x7f4fc41551e0 2023-11-06T03:13:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0004a30 con 0x7f4fc41551e0 2023-11-06T03:13:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.325+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40ccc70 con 0x7f4fc41551e0 2023-11-06T03:13:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc4072750 con 0x7f4fc41551e0 2023-11-06T03:13:44.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4fb0004b90 con 0x7f4fc41551e0 2023-11-06T03:13:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fc17fa640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 0x7f4fa8067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fb0082660 con 0x7f4fc41551e0 2023-11-06T03:13:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fc8e24640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 0x7f4fa8067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.329+0000 7f4fc8e24640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 0x7f4fa8067ef0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80ae000 tx=0x7f4fb80b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.333+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4fb0053a90 con 0x7f4fc41551e0 2023-11-06T03:13:44.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.541+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4fc4071730 con 0x7f4fc41551e0 2023-11-06T03:13:44.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fc17fa640 1 -- 172.21.15.157:0/2735192088 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1659 (secure 0 0 0) 0x7f4fb0053620 con 0x7f4fc41551e0 2023-11-06T03:13:44.540 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:44.541 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":598290432,"bytes_avail":772495822848,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-11-06T03:13:23.298350+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:13:44.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 msgr2=0x7f4fa8067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 0x7f4fa8067ef0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80ae000 tx=0x7f4fb80b5040 comp rx=0 tx=0).stop 2023-11-06T03:13:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 msgr2=0x7f4fc4145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c4b0 tx=0x7f4fb000c970 comp rx=0 tx=0).stop 2023-11-06T03:13:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 shutdown_connections 2023-11-06T03:13:44.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fa8065a30 0x7f4fa8067ef0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4fc415e5e0 0x7f4fc415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fc41551e0 0x7f4fc4145880 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 --2- 172.21.15.157:0/2735192088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4fc40b3b20 0x7f4fc4145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 >> 172.21.15.157:0/2735192088 conn(0x7f4fc4065930 msgr2=0x7f4fc415ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:44.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 shutdown_connections 2023-11-06T03:13:44.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.545+0000 7f4fcb088640 1 -- 172.21.15.157:0/2735192088 wait complete. 2023-11-06T03:13:44.558 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:13:44.559 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:13:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 Processor -- start 2023-11-06T03:13:44.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 -- start start 2023-11-06T03:13:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41640cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4164160980 con 0x7f416415cbc0 2023-11-06T03:13:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41640cf0e0 con 0x7f416415cfa0 2023-11-06T03:13:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f416b440640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640cf3c0 con 0x7f4164066040 2023-11-06T03:13:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f416415cbc0 0x7f4164070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54980/0 (socket says 172.21.15.157:54980) 2023-11-06T03:13:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 -- 172.21.15.157:0/2530050772 learned_addr learned my addr 172.21.15.157:0/2530050772 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41699dd640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41640cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370802096 0 0) 0x7f4164160980 con 0x7f416415cbc0 2023-11-06T03:13:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4150003530 con 0x7f416415cbc0 2023-11-06T03:13:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665135617 0 0) 0x7f4150003530 con 0x7f416415cbc0 2023-11-06T03:13:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 msgr2=0x7f41640cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41640cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 >> v1:172.21.15.195:6789/0 conn(0x7f416415cfa0 legacy=0x7f4164065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164160bd0 con 0x7f416415cbc0 2023-11-06T03:13:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4160001680 con 0x7f416415cbc0 2023-11-06T03:13:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 >> v1:172.21.15.157:6789/0 conn(0x7f416415cbc0 legacy=0x7f4164070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41500039b0 0x7f4150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41500047d0 0x7f4150004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640cf3c0 con 0x7f41500039b0 2023-11-06T03:13:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415000baf0 con 0x7f41500047d0 2023-11-06T03:13:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415000bcb0 con 0x7f4164066040 2023-11-06T03:13:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41500039b0 0x7f4150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41689db640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41500047d0 0x7f4150004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41699dd640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 -- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 msgr2=0x7f41500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 -- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41500047d0 msgr2=0x7f4150004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41500047d0 0x7f4150004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41691dc640 1 -- 172.21.15.157:0/2530050772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4150003530 con 0x7f41500039b0 2023-11-06T03:13:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41699dd640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4164066040 0x7f41500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.673+0000 7f41689db640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41500047d0 0x7f4150004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:13:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f41691dc640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41500039b0 0x7f4150003dd0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f416000c240 tx=0x7f416001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a0e47f2d70c682 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160005ae0 con 0x7f41500039b0 2023-11-06T03:13:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160005c40 con 0x7f41500039b0 2023-11-06T03:13:44.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f415bfff640 1 -- 172.21.15.157:0/2530050772 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600a5800 con 0x7f41500039b0 2023-11-06T03:13:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- 172.21.15.157:0/2530050772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41500039b0 msgr2=0x7f4150003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 --2- 172.21.15.157:0/2530050772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41500039b0 0x7f4150003dd0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f416000c240 tx=0x7f416001cc10 comp rx=0 tx=0).stop 2023-11-06T03:13:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f41699dd640 1 -- 172.21.15.157:0/2530050772 reap_dead start 2023-11-06T03:13:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- 172.21.15.157:0/2530050772 shutdown_connections 2023-11-06T03:13:44.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- 172.21.15.157:0/2530050772 >> 172.21.15.157:0/2530050772 conn(0x7f41640cc2a0 msgr2=0x7f41640cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:44.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- 172.21.15.157:0/2530050772 shutdown_connections 2023-11-06T03:13:44.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- 172.21.15.157:0/2530050772 wait complete. 2023-11-06T03:13:44.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 Processor -- start 2023-11-06T03:13:44.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.677+0000 7f416b440640 1 -- start start 2023-11-06T03:13:44.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41500039b0 0x7f41640b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164156110 0x7f416415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640b5010 con 0x7f4164152e70 2023-11-06T03:13:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164155740 con 0x7f4164156110 2023-11-06T03:13:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640ae560 con 0x7f41500039b0 2023-11-06T03:13:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55164/0 (socket says 172.21.15.157:55164) 2023-11-06T03:13:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 -- 172.21.15.157:0/361589090 learned_addr learned my addr 172.21.15.157:0/361589090 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41691dc640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41500039b0 0x7f41640b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 -- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41500039b0 msgr2=0x7f41640b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41500039b0 0x7f41640b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41699dd640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164156110 0x7f416415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 -- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164156110 msgr2=0x7f416415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164156110 0x7f416415d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 -- 172.21.15.157:0/361589090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41641550f0 con 0x7f4164152e70 2023-11-06T03:13:44.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41689db640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f415400e4d0 tx=0x7f415400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540052e0 con 0x7f4164152e70 2023-11-06T03:13:44.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154005440 con 0x7f4164152e70 2023-11-06T03:13:44.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41641553d0 con 0x7f4164152e70 2023-11-06T03:13:44.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154003820 con 0x7f4164152e70 2023-11-06T03:13:44.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416415e1b0 con 0x7f4164152e70 2023-11-06T03:13:44.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416406ad00 con 0x7f4164152e70 2023-11-06T03:13:44.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4154003980 con 0x7f4164152e70 2023-11-06T03:13:44.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 0x7f4148067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:44.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4154083040 con 0x7f4164152e70 2023-11-06T03:13:44.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41691dc640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 0x7f4148067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:44.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.681+0000 7f41691dc640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 0x7f4148067f20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41600ab000 tx=0x7f41600a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:44.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.685+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4154050880 con 0x7f4164152e70 2023-11-06T03:13:44.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.825+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4164065e20 con 0x7f4148065a60 2023-11-06T03:13:44.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.845+0000 7f4159ffb640 1 -- 172.21.15.157:0/361589090 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299907 (secure 0 0 0) 0x7f4164065e20 con 0x7f4148065a60 2023-11-06T03:13:44.846 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:44.848 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:13:44.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 msgr2=0x7f4148067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 0x7f4148067f20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41600ab000 tx=0x7f41600a7070 comp rx=0 tx=0).stop 2023-11-06T03:13:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 msgr2=0x7f4164155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f415400e4d0 tx=0x7f415400e990 comp rx=0 tx=0).stop 2023-11-06T03:13:44.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 shutdown_connections 2023-11-06T03:13:44.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4148065a60 0x7f4148067f20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4164156110 0x7f416415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4164152e70 0x7f4164155bd0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 --2- 172.21.15.157:0/361589090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41500039b0 0x7f41640b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:44.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 >> 172.21.15.157:0/361589090 conn(0x7f41640cc2a0 msgr2=0x7f41640b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:44.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 shutdown_connections 2023-11-06T03:13:44.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:44.849+0000 7f416b440640 1 -- 172.21.15.157:0/361589090 wait complete. 2023-11-06T03:13:44.863 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2023-11-06T03:13:43.305114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584268,"kb_used_data":12940,"kb_used_omap":0,"kb_used_meta":571264,"kb_avail":754390452,"statfs":{"total":773094113280,"available":772495822848,"internally_reserved":0,"allocated":13250560,"data_stored":6725383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":584974336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.277974"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039944+0000","last_change":"2023-11-06T03:13:39.039944+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039944+0000","last_undegraded":"2023-11-06T03:13:39.039944+0000","last_fullsized":"2023-11-06T03:13:39.039944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034813+0000","last_change":"2023-11-06T03:13:39.034813+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034813+0000","last_undegraded":"2023-11-06T03:13:39.034813+0000","last_fullsized":"2023-11-06T03:13:39.034813+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037617+0000","last_change":"2023-11-06T03:13:39.037617+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037617+0000","last_undegraded":"2023-11-06T03:13:39.037617+0000","last_fullsized":"2023-11-06T03:13:39.037617+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034124+0000","last_change":"2023-11-06T03:13:39.034124+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034124+0000","last_undegraded":"2023-11-06T03:13:39.034124+0000","last_fullsized":"2023-11-06T03:13:39.034124+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.031025+0000","last_change":"2023-11-06T03:13:39.031025+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.031025+0000","last_undegraded":"2023-11-06T03:13:39.031025+0000","last_fullsized":"2023-11-06T03:13:39.031025+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030039+0000","last_change":"2023-11-06T03:13:39.030039+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030039+0000","last_undegraded":"2023-11-06T03:13:39.030039+0000","last_fullsized":"2023-11-06T03:13:39.030039+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033922+0000","last_change":"2023-11-06T03:13:39.033922+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033922+0000","last_undegraded":"2023-11-06T03:13:39.033922+0000","last_fullsized":"2023-11-06T03:13:39.033922+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030067+0000","last_change":"2023-11-06T03:13:39.030067+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030067+0000","last_undegraded":"2023-11-06T03:13:39.030067+0000","last_fullsized":"2023-11-06T03:13:39.030067+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033625+0000","last_change":"2023-11-06T03:13:39.033625+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033625+0000","last_undegraded":"2023-11-06T03:13:39.033625+0000","last_fullsized":"2023-11-06T03:13:39.033625+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032230+0000","last_change":"2023-11-06T03:13:39.032230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032230+0000","last_undegraded":"2023-11-06T03:13:39.032230+0000","last_fullsized":"2023-11-06T03:13:39.032230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.036701+0000","last_change":"2023-11-06T03:13:39.036701+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.036701+0000","last_undegraded":"2023-11-06T03:13:39.036701+0000","last_fullsized":"2023-11-06T03:13:39.036701+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035894+0000","last_change":"2023-11-06T03:13:39.035894+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035894+0000","last_undegraded":"2023-11-06T03:13:39.035894+0000","last_fullsized":"2023-11-06T03:13:39.035894+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.029843+0000","last_change":"2023-11-06T03:13:39.029843+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.029843+0000","last_undegraded":"2023-11-06T03:13:39.029843+0000","last_fullsized":"2023-11-06T03:13:39.029843+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035815+0000","last_change":"2023-11-06T03:13:39.035815+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035815+0000","last_undegraded":"2023-11-06T03:13:39.035815+0000","last_fullsized":"2023-11-06T03:13:39.035815+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032991+0000","last_change":"2023-11-06T03:13:39.032991+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032991+0000","last_undegraded":"2023-11-06T03:13:39.032991+0000","last_fullsized":"2023-11-06T03:13:39.032991+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.037034+0000","last_change":"2023-11-06T03:13:39.037034+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.037034+0000","last_undegraded":"2023-11-06T03:13:39.037034+0000","last_fullsized":"2023-11-06T03:13:39.037034+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030155+0000","last_change":"2023-11-06T03:13:39.030155+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030155+0000","last_undegraded":"2023-11-06T03:13:39.030155+0000","last_fullsized":"2023-11-06T03:13:39.030155+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039010+0000","last_change":"2023-11-06T03:13:39.039010+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039010+0000","last_undegraded":"2023-11-06T03:13:39.039010+0000","last_fullsized":"2023-11-06T03:13:39.039010+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039072+0000","last_change":"2023-11-06T03:13:39.039072+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039072+0000","last_undegraded":"2023-11-06T03:13:39.039072+0000","last_fullsized":"2023-11-06T03:13:39.039072+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030042+0000","last_change":"2023-11-06T03:13:39.030042+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030042+0000","last_undegraded":"2023-11-06T03:13:39.030042+0000","last_fullsized":"2023-11-06T03:13:39.030042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030197+0000","last_change":"2023-11-06T03:13:39.030197+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030197+0000","last_undegraded":"2023-11-06T03:13:39.030197+0000","last_fullsized":"2023-11-06T03:13:39.030197+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030538+0000","last_change":"2023-11-06T03:13:39.030538+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030538+0000","last_undegraded":"2023-11-06T03:13:39.030538+0000","last_fullsized":"2023-11-06T03:13:39.030538+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.030571+0000","last_change":"2023-11-06T03:13:39.030571+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.030571+0000","last_undegraded":"2023-11-06T03:13:39.030571+0000","last_fullsized":"2023-11-06T03:13:39.030571+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.035852+0000","last_change":"2023-11-06T03:13:39.035852+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.035852+0000","last_undegraded":"2023-11-06T03:13:39.035852+0000","last_fullsized":"2023-11-06T03:13:39.035852+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032867+0000","last_change":"2023-11-06T03:13:39.032867+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032867+0000","last_undegraded":"2023-11-06T03:13:39.032867+0000","last_fullsized":"2023-11-06T03:13:39.032867+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116943+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:13:37.116943+0000","last_peered":"2023-11-06T03:13:37.116943+0000","last_clean":"2023-11-06T03:13:37.116943+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:13:37.116943+0000","last_undegraded":"2023-11-06T03:13:37.116943+0000","last_fullsized":"2023-11-06T03:13:37.116943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114100+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:13:37.114100+0000","last_peered":"2023-11-06T03:13:37.114100+0000","last_clean":"2023-11-06T03:13:37.114100+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:13:37.114100+0000","last_undegraded":"2023-11-06T03:13:37.114100+0000","last_fullsized":"2023-11-06T03:13:37.114100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033827+0000","last_change":"2023-11-06T03:13:39.033827+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033827+0000","last_undegraded":"2023-11-06T03:13:39.033827+0000","last_fullsized":"2023-11-06T03:13:39.033827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033218+0000","last_change":"2023-11-06T03:13:39.033218+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033218+0000","last_undegraded":"2023-11-06T03:13:39.033218+0000","last_fullsized":"2023-11-06T03:13:39.033218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":95,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.113041+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:13:37.113041+0000","last_peered":"2023-11-06T03:13:37.113041+0000","last_clean":"2023-11-06T03:13:37.113041+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:13:37.113041+0000","last_undegraded":"2023-11-06T03:13:37.113041+0000","last_fullsized":"2023-11-06T03:13:37.113041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034529+0000","last_change":"2023-11-06T03:13:39.034529+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034529+0000","last_undegraded":"2023-11-06T03:13:39.034529+0000","last_fullsized":"2023-11-06T03:13:39.034529+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.114094+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:13:37.114094+0000","last_peered":"2023-11-06T03:13:37.114094+0000","last_clean":"2023-11-06T03:13:37.114094+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:13:37.114094+0000","last_undegraded":"2023-11-06T03:13:37.114094+0000","last_fullsized":"2023-11-06T03:13:37.114094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.039011+0000","last_change":"2023-11-06T03:13:39.039011+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.039011+0000","last_undegraded":"2023-11-06T03:13:39.039011+0000","last_fullsized":"2023-11-06T03:13:39.039011+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111589+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:13:37.111589+0000","last_peered":"2023-11-06T03:13:37.111589+0000","last_clean":"2023-11-06T03:13:37.111589+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:13:37.111589+0000","last_undegraded":"2023-11-06T03:13:37.111589+0000","last_fullsized":"2023-11-06T03:13:37.111589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.034230+0000","last_change":"2023-11-06T03:13:39.034230+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.034230+0000","last_undegraded":"2023-11-06T03:13:39.034230+0000","last_fullsized":"2023-11-06T03:13:39.034230+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.116879+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:13:37.116879+0000","last_peered":"2023-11-06T03:13:37.116879+0000","last_clean":"2023-11-06T03:13:37.116879+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:13:37.116879+0000","last_undegraded":"2023-11-06T03:13:37.116879+0000","last_fullsized":"2023-11-06T03:13:37.116879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.033916+0000","last_change":"2023-11-06T03:13:39.033916+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.033916+0000","last_undegraded":"2023-11-06T03:13:39.033916+0000","last_fullsized":"2023-11-06T03:13:39.033916+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.111653+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:13:37.111653+0000","last_peered":"2023-11-06T03:13:37.111653+0000","last_clean":"2023-11-06T03:13:37.111653+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:13:37.111653+0000","last_undegraded":"2023-11-06T03:13:37.111653+0000","last_fullsized":"2023-11-06T03:13:37.111653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:13:40.023544+0000","last_change":"2023-11-06T03:13:40.023544+0000","last_active":"2023-11-06T03:13:40.023544+0000","last_peered":"2023-11-06T03:13:40.023544+0000","last_clean":"2023-11-06T03:13:40.023544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:40.023544+0000","last_undegraded":"2023-11-06T03:13:40.023544+0000","last_fullsized":"2023-11-06T03:13:40.023544+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-11-06T03:13:39.032841+0000","last_change":"2023-11-06T03:13:39.032841+0000","last_active":"2023-11-06T03:13:39.008912+0000","last_peered":"2023-11-06T03:13:39.008912+0000","last_clean":"2023-11-06T03:13:39.008912+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:13:39.032841+0000","last_undegraded":"2023-11-06T03:13:39.032841+0000","last_fullsized":"2023-11-06T03:13:39.032841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-11-06T03:13:37.115547+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:13:37.115547+0000","last_peered":"2023-11-06T03:13:37.115547+0000","last_clean":"2023-11-06T03:13:37.115547+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:13:37.115547+0000","last_undegraded":"2023-11-06T03:13:37.115547+0000","last_fullsized":"2023-11-06T03:13:37.115547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":2109440,"data_stored":1286916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1830000000000001,"15min":1.4830000000000001},"last":0.626},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.85999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.3620000000000001,"15min":1.397},"last":0.66600000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.147,"5min":1.2490000000000001,"15min":1.444},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.153,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.90700000000000003,"15min":0.88700000000000001},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1539999999999999,"5min":1.381,"15min":1.6080000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.22,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":2.512,"15min":2.512},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.877,"5min":1.097,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":10.596,"15min":10.596},"last":1.028}]},{"osd":4,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.71799999999999997,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.5309999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.99199999999999999,"15min":1.0009999999999999},"min":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.278,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0600000000000001,"15min":1.048},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3720000000000001,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.349,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.123}]},{"osd":6,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.169,"15min":1.1539999999999999},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.454,"5min":2.548,"15min":2.548},"last":1.198},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.1040000000000001,"15min":1.105},"min":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.3049999999999999,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.05}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64460,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307380,"statfs":{"total":96636764160,"available":96570757120,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.97299999999999998,"15min":0.84999999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.081,"5min":16.073,"15min":16.073},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.2749999999999999,"15min":0.97299999999999998},"min":{"1min":0.50700000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.0269999999999999,"15min":0.90100000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1359999999999999,"5min":16.012,"15min":16.012},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.1279999999999999,"15min":0.91300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.105,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.94699999999999995},"min":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1619999999999999,"5min":2.843,"15min":2.843},"last":1.2410000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.647,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.2929999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.877,"5min":1.355,"15min":1.052},"min":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.278,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.335,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2609999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.413,"15min":1.2370000000000001},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3540000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.367}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.411,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.294,"5min":40.959000000000003,"15min":40.959000000000003},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1379999999999999,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.3240000000000001,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.399}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.4079999999999999,"15min":1.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.429,"5min":23.748999999999999,"15min":23.748999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.3999999999999999,"15min":1.212},"min":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.371,"5min":28.164000000000001,"15min":28.164000000000001},"last":1.204}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.7729999999999999,"15min":1.1299999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.0840000000000001,"5min":125.256,"15min":125.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.8129999999999999,"15min":1.159},"min":{"1min":0.495,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.028,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.71799999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.78000000000000003},"min":{"1min":0.60799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.27,"5min":2.339,"15min":2.339},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.57199999999999995,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.53500000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1799999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0640000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91300000000000003,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.208,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.94499999999999995},"min":{"1min":0.60099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3400000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.97799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.66500000000000004,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5149999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.149,"5min":1.044,"15min":0.98699999999999999},"min":{"1min":0.76200000000000001,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":3.0030000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.008}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0349999999999999,"15min":1.01},"min":{"1min":0.70699999999999996,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.429,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0620000000000001,"15min":1.0249999999999999},"min":{"1min":0.79200000000000004,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3600000000000001,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0960000000000001,"15min":1.0249999999999999},"min":{"1min":0.85599999999999998,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.421,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.94299999999999995,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.514,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90692,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281148,"statfs":{"total":96636764160,"available":96543895552,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.26,"15min":0.96599999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.372},"max":{"1min":1.0429999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.2929999999999999,"15min":0.95899999999999996},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.99099999999999999,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.999}]},{"osd":1,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.51400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1279999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.032,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":1.385}]},{"osd":2,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1779999999999999,"5min":1.452,"15min":3.2879999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.042,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.1339999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.59399999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":3.198,"15min":3.198},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.64800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0960000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":1.081}]},{"osd":5,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0609999999999999,"15min":1.0269999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.143,"5min":10.192,"15min":10.192},"last":1.254},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0600000000000001,"15min":1.089},"min":{"1min":0.80500000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.415}]},{"osd":6,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96099999999999997,"15min":0.998},"min":{"1min":0.66400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.514,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.1140000000000001},"min":{"1min":0.71499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.399,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.512}]},{"osd":7,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.077,"15min":1.075},"min":{"1min":0.76800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.171},"min":{"1min":0.79600000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4099999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.472}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76812,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94295028,"statfs":{"total":96636764160,"available":96558108672,"internally_reserved":0,"allocated":1511424,"data_stored":695529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78200000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":1.524},"last":0.313}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.26400000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.89200000000000002,"5min":1.371,"15min":1.371},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93100000000000005,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.23300000000000001},"max":{"1min":0.95099999999999996,"5min":1.532,"15min":1.532},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.34599999999999997,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.206,"5min":1.284,"15min":1.284},"last":0.65000000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45000000000000001},"max":{"1min":1.2270000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.91100000000000003,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.369,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.621}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.626,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.536,"15min":1.6639999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.698},"last":0.53100000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.999},"min":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.3939999999999999,"5min":1.649,"15min":1.8620000000000001},"last":0.748},{"interface":"front","average":{"1min":0.872,"5min":0.94699999999999995,"15min":0.97999999999999998},"min":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3129999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":1.03},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4410000000000001,"5min":1.835,"15min":1.835},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.623,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.569,"5min":1.734,"15min":1.8979999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.93400000000000005,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.55100000000000005,"15min":0.61399999999999999},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.88100000000000001,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.63400000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.64300000000000002},"min":{"1min":0.32500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.1579999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.294},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65000000000000002,"15min":0.71699999999999997},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.022,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83199999999999996,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.071,"5min":1.526,"15min":1.526},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.91500000000000004},"min":{"1min":0.52300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.86499999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82899999999999996,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.248,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.2390000000000001,"5min":1.486,"15min":1.5780000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.381,"15min":1.615},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.94499999999999995},"min":{"1min":0.66100000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.048}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.91400000000000003,"15min":0.97699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3109999999999999,"5min":1.4650000000000001,"15min":4.2809999999999997},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88800000000000001,"15min":0.94199999999999995},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.4740000000000002},"last":0.95999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46468,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325372,"statfs":{"total":96636764160,"available":96589180928,"internally_reserved":0,"allocated":1503232,"data_stored":690378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.0569999999999999,"5min":1.843,"15min":1.843},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.81200000000000006,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.56499999999999995,"15min":0.70299999999999996},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.76300000000000001,"5min":1.169,"15min":30.861000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.66700000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.0329999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0369999999999999,"5min":1.151,"15min":2.3079999999999998},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.82099999999999995,"15min":0.85999999999999999},"min":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.198,"5min":2.0870000000000002,"15min":2.258},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.82499999999999996,"15min":0.92900000000000005},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":1.5549999999999999,"15min":14.753},"last":0.75700000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":1.101},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47299999999999998},"max":{"1min":1.1499999999999999,"5min":1.528,"15min":43.103999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.92700000000000005,"15min":1.133},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.208,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.88,"15min":1.0409999999999999},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.45100000000000001},"max":{"1min":1.2909999999999999,"5min":1.5189999999999999,"15min":23.971},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":1.0629999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0389999999999999},"min":{"1min":0.67600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.212,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95699999999999996,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.319,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91344,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280496,"statfs":{"total":96636764160,"available":96543227904,"internally_reserved":0,"allocated":2105344,"data_stored":1285897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.27500000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.80500000000000005,"5min":1.145,"15min":2.6989999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59999999999999998,"15min":0.64900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.64600000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.82399999999999995,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.872}]},{"osd":3,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.391,"5min":1.391,"15min":2.7999999999999998},"last":0.90700000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.91800000000000004},"min":{"1min":0.54700000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.371,"5min":1.371,"15min":4.641},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.254,"15min":2.028},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.96399999999999997},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.0920000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":1.244},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.93999999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.147}]},{"osd":6,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57399999999999995,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.96399999999999997,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93100000000000005,"15min":0.996},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":1.266,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.101},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.93799999999999994,"15min":0.97999999999999998},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.335,"5min":1.4670000000000001,"15min":1.97},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:13:46.290 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:13:46.293+0000 7fef8a813640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:13:47.871 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:13:48.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 Processor -- start 2023-11-06T03:13:48.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 -- start start 2023-11-06T03:13:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 0x7f6c7c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6c7c0b3b20 0x7f6c7c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c15f580 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c15efc0 con 0x7f6c7c0b3b20 2023-11-06T03:13:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c8314b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c15f2a0 con 0x7f6c7c1551e0 2023-11-06T03:13:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c7bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6c7c15e5e0 0x7f6c7c0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:54992/0 (socket says 172.21.15.157:54992) 2023-11-06T03:13:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c7bfff640 1 -- 172.21.15.157:0/4255730547 learned_addr learned my addr 172.21.15.157:0/4255730547 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c816e8640 1 --2- 172.21.15.157:0/4255730547 >> v2:172.21.15.195:3300/0 conn(0x7f6c7c0b3b20 0x7f6c7c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 0x7f6c7c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478024526 0 0) 0x7f6c7c15f580 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c64003530 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.009+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2650881903 0 0) 0x7f6c64003530 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 msgr2=0x7f6c7c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 0x7f6c7c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 >> v2:172.21.15.195:3300/0 conn(0x7f6c7c0b3b20 msgr2=0x7f6c7c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 --2- 172.21.15.157:0/4255730547 >> v2:172.21.15.195:3300/0 conn(0x7f6c7c0b3b20 0x7f6c7c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c06f5a0 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6c70001a00 con 0x7f6c7c15e5e0 2023-11-06T03:13:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 >> v1:172.21.15.157:6789/0 conn(0x7f6c7c15e5e0 legacy=0x7f6c7c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c64003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 0x7f6c640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c64004d10 0x7f6c64005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c15efc0 con 0x7f6c64003980 2023-11-06T03:13:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c15f2a0 con 0x7f6c64004d10 2023-11-06T03:13:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6400b990 con 0x7f6c7c1551e0 2023-11-06T03:13:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c64003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 -- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 msgr2=0x7f6c640047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1551e0 0x7f6c640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 -- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c64004d10 msgr2=0x7f6c64005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c816e8640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c64004d10 0x7f6c64005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c64004d10 0x7f6c64005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 -- 172.21.15.157:0/4255730547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64003530 con 0x7f6c64003980 2023-11-06T03:13:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c816e8640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c64004d10 0x7f6c64005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:48.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c64003da0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ae0 tx=0x7f6c7401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e6485af597d472 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740a5540 con 0x7f6c64003980 2023-11-06T03:13:48.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c740a56a0 con 0x7f6c64003980 2023-11-06T03:13:48.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c7b7fe640 1 -- 172.21.15.157:0/4255730547 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740a5a50 con 0x7f6c64003980 2023-11-06T03:13:48.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 -- 172.21.15.157:0/4255730547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 msgr2=0x7f6c64003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 --2- 172.21.15.157:0/4255730547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c64003da0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ae0 tx=0x7f6c7401ca60 comp rx=0 tx=0).stop 2023-11-06T03:13:48.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c816e8640 1 -- 172.21.15.157:0/4255730547 reap_dead start 2023-11-06T03:13:48.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 -- 172.21.15.157:0/4255730547 shutdown_connections 2023-11-06T03:13:48.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 -- 172.21.15.157:0/4255730547 >> 172.21.15.157:0/4255730547 conn(0x7f6c7c065930 msgr2=0x7f6c7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 -- 172.21.15.157:0/4255730547 shutdown_connections 2023-11-06T03:13:48.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.013+0000 7f6c8314b640 1 -- 172.21.15.157:0/4255730547 wait complete. 2023-11-06T03:13:48.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 Processor -- start 2023-11-06T03:13:48.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- start start 2023-11-06T03:13:48.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1555c0 0x7f6c7c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 0x7f6c7c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c14f7d0 con 0x7f6c64003980 2023-11-06T03:13:48.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c06cdd0 con 0x7f6c7c1f1d10 2023-11-06T03:13:48.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c06c810 con 0x7f6c7c1555c0 2023-11-06T03:13:48.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55178/0 (socket says 172.21.15.157:55178) 2023-11-06T03:13:48.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 -- 172.21.15.157:0/3718034480 learned_addr learned my addr 172.21.15.157:0/3718034480 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:48.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c816e8640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 0x7f6c7c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c7bfff640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1555c0 0x7f6c7c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 -- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1555c0 msgr2=0x7f6c7c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1555c0 0x7f6c7c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 -- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 msgr2=0x7f6c7c0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 0x7f6c7c0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 -- 172.21.15.157:0/3718034480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c06caf0 con 0x7f6c64003980 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c816e8640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 0x7f6c7c0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:48.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c80ee7640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ae0 tx=0x7f6c64005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740a5e90 con 0x7f6c64003980 2023-11-06T03:13:48.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c74004030 con 0x7f6c64003980 2023-11-06T03:13:48.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c1e6700 con 0x7f6c64003980 2023-11-06T03:13:48.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740043a0 con 0x7f6c64003980 2023-11-06T03:13:48.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c06c2e0 con 0x7f6c64003980 2023-11-06T03:13:48.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.017+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c06cf40 con 0x7f6c64003980 2023-11-06T03:13:48.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.021+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c740b4040 con 0x7f6c64003980 2023-11-06T03:13:48.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.021+0000 7f6c797fa640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 0x7f6c68067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.021+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c7411ca00 con 0x7f6c64003980 2023-11-06T03:13:48.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.021+0000 7f6c7bfff640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 0x7f6c68067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.021+0000 7f6c7bfff640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 0x7f6c68067f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15a7c0 tx=0x7f6c70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.025+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c740ea390 con 0x7f6c64003980 2023-11-06T03:13:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.177+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c7c155d60 con 0x7f6c68065a90 2023-11-06T03:13:48.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.201+0000 7f6c797fa640 1 -- 172.21.15.157:0/3718034480 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310910 (secure 0 0 0) 0x7f6c7c155d60 con 0x7f6c68065a90 2023-11-06T03:13:48.199 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:13:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 msgr2=0x7f6c68067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 0x7f6c68067f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15a7c0 tx=0x7f6c70009040 comp rx=0 tx=0).stop 2023-11-06T03:13:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 msgr2=0x7f6c7c14f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009ae0 tx=0x7f6c64005800 comp rx=0 tx=0).stop 2023-11-06T03:13:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 shutdown_connections 2023-11-06T03:13:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c68065a90 0x7f6c68067f50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c7c1f1d10 0x7f6c7c0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c7c1555c0 0x7f6c7c06bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 --2- 172.21.15.157:0/3718034480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c64003980 0x7f6c7c14f290 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 >> 172.21.15.157:0/3718034480 conn(0x7f6c7c065930 msgr2=0x7f6c7c15daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 shutdown_connections 2023-11-06T03:13:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.205+0000 7f6c8314b640 1 -- 172.21.15.157:0/3718034480 wait complete. 2023-11-06T03:13:48.218 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-11-06T03:13:47.306710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584668,"kb_used_data":13340,"kb_used_omap":0,"kb_used_meta":571264,"kb_avail":754390052,"statfs":{"total":773094113280,"available":772495413248,"internally_reserved":0,"allocated":13660160,"data_stored":6918059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":584974336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":56,"apply_latency_ms":56,"commit_latency_ns":56000000,"apply_latency_ns":56000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.265851"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.434771+0000","last_change":"2023-11-06T03:13:40.481525+0000","last_active":"2023-11-06T03:13:41.434771+0000","last_peered":"2023-11-06T03:13:41.434771+0000","last_clean":"2023-11-06T03:13:41.434771+0000","last_became_active":"2023-11-06T03:13:40.480683+0000","last_became_peered":"2023-11-06T03:13:40.480683+0000","last_unstale":"2023-11-06T03:13:41.434771+0000","last_undegraded":"2023-11-06T03:13:41.434771+0000","last_fullsized":"2023-11-06T03:13:41.434771+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:02:00.714992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794072+0000","last_change":"2023-11-06T03:13:40.477497+0000","last_active":"2023-11-06T03:13:41.794072+0000","last_peered":"2023-11-06T03:13:41.794072+0000","last_clean":"2023-11-06T03:13:41.794072+0000","last_became_active":"2023-11-06T03:13:40.477167+0000","last_became_peered":"2023-11-06T03:13:40.477167+0000","last_unstale":"2023-11-06T03:13:41.794072+0000","last_undegraded":"2023-11-06T03:13:41.794072+0000","last_fullsized":"2023-11-06T03:13:41.794072+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:16:29.708657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794326+0000","last_change":"2023-11-06T03:13:40.472983+0000","last_active":"2023-11-06T03:13:41.794326+0000","last_peered":"2023-11-06T03:13:41.794326+0000","last_clean":"2023-11-06T03:13:41.794326+0000","last_became_active":"2023-11-06T03:13:40.472741+0000","last_became_peered":"2023-11-06T03:13:40.472741+0000","last_unstale":"2023-11-06T03:13:41.794326+0000","last_undegraded":"2023-11-06T03:13:41.794326+0000","last_fullsized":"2023-11-06T03:13:41.794326+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:17.795014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798856+0000","last_change":"2023-11-06T03:13:40.482149+0000","last_active":"2023-11-06T03:13:41.798856+0000","last_peered":"2023-11-06T03:13:41.798856+0000","last_clean":"2023-11-06T03:13:41.798856+0000","last_became_active":"2023-11-06T03:13:40.481561+0000","last_became_peered":"2023-11-06T03:13:40.481561+0000","last_unstale":"2023-11-06T03:13:41.798856+0000","last_undegraded":"2023-11-06T03:13:41.798856+0000","last_fullsized":"2023-11-06T03:13:41.798856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:51:38.729883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.434009+0000","last_change":"2023-11-06T03:13:40.475730+0000","last_active":"2023-11-06T03:13:41.434009+0000","last_peered":"2023-11-06T03:13:41.434009+0000","last_clean":"2023-11-06T03:13:41.434009+0000","last_became_active":"2023-11-06T03:13:40.475010+0000","last_became_peered":"2023-11-06T03:13:40.475010+0000","last_unstale":"2023-11-06T03:13:41.434009+0000","last_undegraded":"2023-11-06T03:13:41.434009+0000","last_fullsized":"2023-11-06T03:13:41.434009+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:59.230674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428827+0000","last_change":"2023-11-06T03:13:40.435374+0000","last_active":"2023-11-06T03:13:41.428827+0000","last_peered":"2023-11-06T03:13:41.428827+0000","last_clean":"2023-11-06T03:13:41.428827+0000","last_became_active":"2023-11-06T03:13:40.434928+0000","last_became_peered":"2023-11-06T03:13:40.434928+0000","last_unstale":"2023-11-06T03:13:41.428827+0000","last_undegraded":"2023-11-06T03:13:41.428827+0000","last_fullsized":"2023-11-06T03:13:41.428827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:17:44.283936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428539+0000","last_change":"2023-11-06T03:13:40.470317+0000","last_active":"2023-11-06T03:13:41.428539+0000","last_peered":"2023-11-06T03:13:41.428539+0000","last_clean":"2023-11-06T03:13:41.428539+0000","last_became_active":"2023-11-06T03:13:40.470024+0000","last_became_peered":"2023-11-06T03:13:40.470024+0000","last_unstale":"2023-11-06T03:13:41.428539+0000","last_undegraded":"2023-11-06T03:13:41.428539+0000","last_fullsized":"2023-11-06T03:13:41.428539+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:23:39.066257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.793465+0000","last_change":"2023-11-06T03:13:40.459975+0000","last_active":"2023-11-06T03:13:41.793465+0000","last_peered":"2023-11-06T03:13:41.793465+0000","last_clean":"2023-11-06T03:13:41.793465+0000","last_became_active":"2023-11-06T03:13:40.459728+0000","last_became_peered":"2023-11-06T03:13:40.459728+0000","last_unstale":"2023-11-06T03:13:41.793465+0000","last_undegraded":"2023-11-06T03:13:41.793465+0000","last_fullsized":"2023-11-06T03:13:41.793465+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:10:38.890501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.799831+0000","last_change":"2023-11-06T03:13:40.469975+0000","last_active":"2023-11-06T03:13:41.799831+0000","last_peered":"2023-11-06T03:13:41.799831+0000","last_clean":"2023-11-06T03:13:41.799831+0000","last_became_active":"2023-11-06T03:13:40.469373+0000","last_became_peered":"2023-11-06T03:13:40.469373+0000","last_unstale":"2023-11-06T03:13:41.799831+0000","last_undegraded":"2023-11-06T03:13:41.799831+0000","last_fullsized":"2023-11-06T03:13:41.799831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:01:15.198135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795320+0000","last_change":"2023-11-06T03:13:40.437154+0000","last_active":"2023-11-06T03:13:41.795320+0000","last_peered":"2023-11-06T03:13:41.795320+0000","last_clean":"2023-11-06T03:13:41.795320+0000","last_became_active":"2023-11-06T03:13:40.436829+0000","last_became_peered":"2023-11-06T03:13:40.436829+0000","last_unstale":"2023-11-06T03:13:41.795320+0000","last_undegraded":"2023-11-06T03:13:41.795320+0000","last_fullsized":"2023-11-06T03:13:41.795320+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:06.894627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.434652+0000","last_change":"2023-11-06T03:13:40.470823+0000","last_active":"2023-11-06T03:13:41.434652+0000","last_peered":"2023-11-06T03:13:41.434652+0000","last_clean":"2023-11-06T03:13:41.434652+0000","last_became_active":"2023-11-06T03:13:40.470078+0000","last_became_peered":"2023-11-06T03:13:40.470078+0000","last_unstale":"2023-11-06T03:13:41.434652+0000","last_undegraded":"2023-11-06T03:13:41.434652+0000","last_fullsized":"2023-11-06T03:13:41.434652+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:10.846329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428996+0000","last_change":"2023-11-06T03:13:40.480232+0000","last_active":"2023-11-06T03:13:41.428996+0000","last_peered":"2023-11-06T03:13:41.428996+0000","last_clean":"2023-11-06T03:13:41.428996+0000","last_became_active":"2023-11-06T03:13:40.479939+0000","last_became_peered":"2023-11-06T03:13:40.479939+0000","last_unstale":"2023-11-06T03:13:41.428996+0000","last_undegraded":"2023-11-06T03:13:41.428996+0000","last_fullsized":"2023-11-06T03:13:41.428996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:03:42.881990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794946+0000","last_change":"2023-11-06T03:13:40.442585+0000","last_active":"2023-11-06T03:13:41.794946+0000","last_peered":"2023-11-06T03:13:41.794946+0000","last_clean":"2023-11-06T03:13:41.794946+0000","last_became_active":"2023-11-06T03:13:40.442292+0000","last_became_peered":"2023-11-06T03:13:40.442292+0000","last_unstale":"2023-11-06T03:13:41.794946+0000","last_undegraded":"2023-11-06T03:13:41.794946+0000","last_fullsized":"2023-11-06T03:13:41.794946+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:01:15.198135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.793891+0000","last_change":"2023-11-06T03:13:40.472924+0000","last_active":"2023-11-06T03:13:41.793891+0000","last_peered":"2023-11-06T03:13:41.793891+0000","last_clean":"2023-11-06T03:13:41.793891+0000","last_became_active":"2023-11-06T03:13:40.472665+0000","last_became_peered":"2023-11-06T03:13:40.472665+0000","last_unstale":"2023-11-06T03:13:41.793891+0000","last_undegraded":"2023-11-06T03:13:41.793891+0000","last_fullsized":"2023-11-06T03:13:41.793891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:50:51.288753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.433804+0000","last_change":"2023-11-06T03:13:40.472436+0000","last_active":"2023-11-06T03:13:41.433804+0000","last_peered":"2023-11-06T03:13:41.433804+0000","last_clean":"2023-11-06T03:13:41.433804+0000","last_became_active":"2023-11-06T03:13:40.471010+0000","last_became_peered":"2023-11-06T03:13:40.471010+0000","last_unstale":"2023-11-06T03:13:41.433804+0000","last_undegraded":"2023-11-06T03:13:41.433804+0000","last_fullsized":"2023-11-06T03:13:41.433804+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:52:13.817989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428489+0000","last_change":"2023-11-06T03:13:40.434322+0000","last_active":"2023-11-06T03:13:41.428489+0000","last_peered":"2023-11-06T03:13:41.428489+0000","last_clean":"2023-11-06T03:13:41.428489+0000","last_became_active":"2023-11-06T03:13:40.433911+0000","last_became_peered":"2023-11-06T03:13:40.433911+0000","last_unstale":"2023-11-06T03:13:41.428489+0000","last_undegraded":"2023-11-06T03:13:41.428489+0000","last_fullsized":"2023-11-06T03:13:41.428489+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:27.725615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795461+0000","last_change":"2023-11-06T03:13:40.451049+0000","last_active":"2023-11-06T03:13:41.795461+0000","last_peered":"2023-11-06T03:13:41.795461+0000","last_clean":"2023-11-06T03:13:41.795461+0000","last_became_active":"2023-11-06T03:13:40.450736+0000","last_became_peered":"2023-11-06T03:13:40.450736+0000","last_unstale":"2023-11-06T03:13:41.795461+0000","last_undegraded":"2023-11-06T03:13:41.795461+0000","last_fullsized":"2023-11-06T03:13:41.795461+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:45.069673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794339+0000","last_change":"2023-11-06T03:13:40.471680+0000","last_active":"2023-11-06T03:13:41.794339+0000","last_peered":"2023-11-06T03:13:41.794339+0000","last_clean":"2023-11-06T03:13:41.794339+0000","last_became_active":"2023-11-06T03:13:40.471446+0000","last_became_peered":"2023-11-06T03:13:40.471446+0000","last_unstale":"2023-11-06T03:13:41.794339+0000","last_undegraded":"2023-11-06T03:13:41.794339+0000","last_fullsized":"2023-11-06T03:13:41.794339+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:57:53.526985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.799901+0000","last_change":"2023-11-06T03:13:40.469907+0000","last_active":"2023-11-06T03:13:41.799901+0000","last_peered":"2023-11-06T03:13:41.799901+0000","last_clean":"2023-11-06T03:13:41.799901+0000","last_became_active":"2023-11-06T03:13:40.468188+0000","last_became_peered":"2023-11-06T03:13:40.468188+0000","last_unstale":"2023-11-06T03:13:41.799901+0000","last_undegraded":"2023-11-06T03:13:41.799901+0000","last_fullsized":"2023-11-06T03:13:41.799901+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:44.183496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796216+0000","last_change":"2023-11-06T03:13:40.436586+0000","last_active":"2023-11-06T03:13:41.796216+0000","last_peered":"2023-11-06T03:13:41.796216+0000","last_clean":"2023-11-06T03:13:41.796216+0000","last_became_active":"2023-11-06T03:13:40.436268+0000","last_became_peered":"2023-11-06T03:13:40.436268+0000","last_unstale":"2023-11-06T03:13:41.796216+0000","last_undegraded":"2023-11-06T03:13:41.796216+0000","last_fullsized":"2023-11-06T03:13:41.796216+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:46:49.505098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798470+0000","last_change":"2023-11-06T03:13:40.451113+0000","last_active":"2023-11-06T03:13:41.798470+0000","last_peered":"2023-11-06T03:13:41.798470+0000","last_clean":"2023-11-06T03:13:41.798470+0000","last_became_active":"2023-11-06T03:13:40.450238+0000","last_became_peered":"2023-11-06T03:13:40.450238+0000","last_unstale":"2023-11-06T03:13:41.798470+0000","last_undegraded":"2023-11-06T03:13:41.798470+0000","last_fullsized":"2023-11-06T03:13:41.798470+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:12:22.584472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.434127+0000","last_change":"2023-11-06T03:13:40.463285+0000","last_active":"2023-11-06T03:13:41.434127+0000","last_peered":"2023-11-06T03:13:41.434127+0000","last_clean":"2023-11-06T03:13:41.434127+0000","last_became_active":"2023-11-06T03:13:40.459189+0000","last_became_peered":"2023-11-06T03:13:40.459189+0000","last_unstale":"2023-11-06T03:13:41.434127+0000","last_undegraded":"2023-11-06T03:13:41.434127+0000","last_fullsized":"2023-11-06T03:13:41.434127+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:50:51.288753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.797744+0000","last_change":"2023-11-06T03:13:40.452878+0000","last_active":"2023-11-06T03:13:41.797744+0000","last_peered":"2023-11-06T03:13:41.797744+0000","last_clean":"2023-11-06T03:13:41.797744+0000","last_became_active":"2023-11-06T03:13:40.452127+0000","last_became_peered":"2023-11-06T03:13:40.452127+0000","last_unstale":"2023-11-06T03:13:41.797744+0000","last_undegraded":"2023-11-06T03:13:41.797744+0000","last_fullsized":"2023-11-06T03:13:41.797744+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:43.862969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796741+0000","last_change":"2023-11-06T03:13:40.474615+0000","last_active":"2023-11-06T03:13:41.796741+0000","last_peered":"2023-11-06T03:13:41.796741+0000","last_clean":"2023-11-06T03:13:41.796741+0000","last_became_active":"2023-11-06T03:13:40.473186+0000","last_became_peered":"2023-11-06T03:13:40.473186+0000","last_unstale":"2023-11-06T03:13:41.796741+0000","last_undegraded":"2023-11-06T03:13:41.796741+0000","last_fullsized":"2023-11-06T03:13:41.796741+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:45.069673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798724+0000","last_change":"2023-11-06T03:13:40.483426+0000","last_active":"2023-11-06T03:13:41.798724+0000","last_peered":"2023-11-06T03:13:41.798724+0000","last_clean":"2023-11-06T03:13:41.798724+0000","last_became_active":"2023-11-06T03:13:40.482795+0000","last_became_peered":"2023-11-06T03:13:40.482795+0000","last_unstale":"2023-11-06T03:13:41.798724+0000","last_undegraded":"2023-11-06T03:13:41.798724+0000","last_fullsized":"2023-11-06T03:13:41.798724+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:37:09.261917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796445+0000","last_change":"2023-11-06T03:13:40.434370+0000","last_active":"2023-11-06T03:13:41.796445+0000","last_peered":"2023-11-06T03:13:41.796445+0000","last_clean":"2023-11-06T03:13:41.796445+0000","last_became_active":"2023-11-06T03:13:40.434105+0000","last_became_peered":"2023-11-06T03:13:40.434105+0000","last_unstale":"2023-11-06T03:13:41.796445+0000","last_undegraded":"2023-11-06T03:13:41.796445+0000","last_fullsized":"2023-11-06T03:13:41.796445+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:13:30.460313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.800018+0000","last_change":"2023-11-06T03:13:40.468275+0000","last_active":"2023-11-06T03:13:41.800018+0000","last_peered":"2023-11-06T03:13:41.800018+0000","last_clean":"2023-11-06T03:13:41.800018+0000","last_became_active":"2023-11-06T03:13:40.467736+0000","last_became_peered":"2023-11-06T03:13:40.467736+0000","last_unstale":"2023-11-06T03:13:41.800018+0000","last_undegraded":"2023-11-06T03:13:41.800018+0000","last_fullsized":"2023-11-06T03:13:41.800018+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:57:29.992302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.429034+0000","last_change":"2023-11-06T03:13:40.450352+0000","last_active":"2023-11-06T03:13:41.429034+0000","last_peered":"2023-11-06T03:13:41.429034+0000","last_clean":"2023-11-06T03:13:41.429034+0000","last_became_active":"2023-11-06T03:13:40.450097+0000","last_became_peered":"2023-11-06T03:13:40.450097+0000","last_unstale":"2023-11-06T03:13:41.429034+0000","last_undegraded":"2023-11-06T03:13:41.429034+0000","last_fullsized":"2023-11-06T03:13:41.429034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:01:38.147959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796023+0000","last_change":"2023-11-06T03:13:40.438519+0000","last_active":"2023-11-06T03:13:41.796023+0000","last_peered":"2023-11-06T03:13:41.796023+0000","last_clean":"2023-11-06T03:13:41.796023+0000","last_became_active":"2023-11-06T03:13:40.438241+0000","last_became_peered":"2023-11-06T03:13:40.438241+0000","last_unstale":"2023-11-06T03:13:41.796023+0000","last_undegraded":"2023-11-06T03:13:41.796023+0000","last_fullsized":"2023-11-06T03:13:41.796023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:08:27.860636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794070+0000","last_change":"2023-11-06T03:13:40.473730+0000","last_active":"2023-11-06T03:13:41.794070+0000","last_peered":"2023-11-06T03:13:41.794070+0000","last_clean":"2023-11-06T03:13:41.794070+0000","last_became_active":"2023-11-06T03:13:40.473460+0000","last_became_peered":"2023-11-06T03:13:40.473460+0000","last_unstale":"2023-11-06T03:13:41.794070+0000","last_undegraded":"2023-11-06T03:13:41.794070+0000","last_fullsized":"2023-11-06T03:13:41.794070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:52:13.817989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.797205+0000","last_change":"2023-11-06T03:13:40.441073+0000","last_active":"2023-11-06T03:13:41.797205+0000","last_peered":"2023-11-06T03:13:41.797205+0000","last_clean":"2023-11-06T03:13:41.797205+0000","last_became_active":"2023-11-06T03:13:40.440206+0000","last_became_peered":"2023-11-06T03:13:40.440206+0000","last_unstale":"2023-11-06T03:13:41.797205+0000","last_undegraded":"2023-11-06T03:13:41.797205+0000","last_fullsized":"2023-11-06T03:13:41.797205+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:52.264376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796761+0000","last_change":"2023-11-06T03:13:40.462308+0000","last_active":"2023-11-06T03:13:41.796761+0000","last_peered":"2023-11-06T03:13:41.796761+0000","last_clean":"2023-11-06T03:13:41.796761+0000","last_became_active":"2023-11-06T03:13:40.461733+0000","last_became_peered":"2023-11-06T03:13:40.461733+0000","last_unstale":"2023-11-06T03:13:41.796761+0000","last_undegraded":"2023-11-06T03:13:41.796761+0000","last_fullsized":"2023-11-06T03:13:41.796761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:06.894627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795547+0000","last_change":"2023-11-06T03:13:40.437333+0000","last_active":"2023-11-06T03:13:41.795547+0000","last_peered":"2023-11-06T03:13:41.795547+0000","last_clean":"2023-11-06T03:13:41.795547+0000","last_became_active":"2023-11-06T03:13:40.437057+0000","last_became_peered":"2023-11-06T03:13:40.437057+0000","last_unstale":"2023-11-06T03:13:41.795547+0000","last_undegraded":"2023-11-06T03:13:41.795547+0000","last_fullsized":"2023-11-06T03:13:41.795547+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:30.313558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"124'4","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.477725+0000","last_change":"2023-11-06T03:13:40.081175+0000","last_active":"2023-11-06T03:13:40.477725+0000","last_peered":"2023-11-06T03:13:40.477725+0000","last_clean":"2023-11-06T03:13:40.477725+0000","last_became_active":"2023-11-06T03:13:40.080420+0000","last_became_peered":"2023-11-06T03:13:40.080420+0000","last_unstale":"2023-11-06T03:13:40.477725+0000","last_undegraded":"2023-11-06T03:13:40.477725+0000","last_fullsized":"2023-11-06T03:13:40.477725+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:23.893489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.429197+0000","last_change":"2023-11-06T03:13:40.435138+0000","last_active":"2023-11-06T03:13:41.429197+0000","last_peered":"2023-11-06T03:13:41.429197+0000","last_clean":"2023-11-06T03:13:41.429197+0000","last_became_active":"2023-11-06T03:13:40.434885+0000","last_became_peered":"2023-11-06T03:13:40.434885+0000","last_unstale":"2023-11-06T03:13:41.429197+0000","last_undegraded":"2023-11-06T03:13:41.429197+0000","last_fullsized":"2023-11-06T03:13:41.429197+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:07:55.130188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.444606+0000","last_change":"2023-11-06T03:13:40.059194+0000","last_active":"2023-11-06T03:13:40.444606+0000","last_peered":"2023-11-06T03:13:40.444606+0000","last_clean":"2023-11-06T03:13:40.444606+0000","last_became_active":"2023-11-06T03:13:40.058785+0000","last_became_peered":"2023-11-06T03:13:40.058785+0000","last_unstale":"2023-11-06T03:13:40.444606+0000","last_undegraded":"2023-11-06T03:13:40.444606+0000","last_fullsized":"2023-11-06T03:13:40.444606+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798310+0000","last_change":"2023-11-06T03:13:40.444219+0000","last_active":"2023-11-06T03:13:41.798310+0000","last_peered":"2023-11-06T03:13:41.798310+0000","last_clean":"2023-11-06T03:13:41.798310+0000","last_became_active":"2023-11-06T03:13:40.443432+0000","last_became_peered":"2023-11-06T03:13:40.443432+0000","last_unstale":"2023-11-06T03:13:41.798310+0000","last_undegraded":"2023-11-06T03:13:41.798310+0000","last_fullsized":"2023-11-06T03:13:41.798310+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:51:38.729883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.477537+0000","last_change":"2023-11-06T03:13:40.086334+0000","last_active":"2023-11-06T03:13:40.477537+0000","last_peered":"2023-11-06T03:13:40.477537+0000","last_clean":"2023-11-06T03:13:40.477537+0000","last_became_active":"2023-11-06T03:13:40.083862+0000","last_became_peered":"2023-11-06T03:13:40.083862+0000","last_unstale":"2023-11-06T03:13:40.477537+0000","last_undegraded":"2023-11-06T03:13:40.477537+0000","last_fullsized":"2023-11-06T03:13:40.477537+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:43.171174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.799956+0000","last_change":"2023-11-06T03:13:40.477484+0000","last_active":"2023-11-06T03:13:41.799956+0000","last_peered":"2023-11-06T03:13:41.799956+0000","last_clean":"2023-11-06T03:13:41.799956+0000","last_became_active":"2023-11-06T03:13:40.476681+0000","last_became_peered":"2023-11-06T03:13:40.476681+0000","last_unstale":"2023-11-06T03:13:41.799956+0000","last_undegraded":"2023-11-06T03:13:41.799956+0000","last_fullsized":"2023-11-06T03:13:41.799956+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:30.313558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.444712+0000","last_change":"2023-11-06T03:13:40.065758+0000","last_active":"2023-11-06T03:13:40.444712+0000","last_peered":"2023-11-06T03:13:40.444712+0000","last_clean":"2023-11-06T03:13:40.444712+0000","last_became_active":"2023-11-06T03:13:40.065377+0000","last_became_peered":"2023-11-06T03:13:40.065377+0000","last_unstale":"2023-11-06T03:13:40.444712+0000","last_undegraded":"2023-11-06T03:13:40.444712+0000","last_fullsized":"2023-11-06T03:13:40.444712+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794888+0000","last_change":"2023-11-06T03:13:40.460104+0000","last_active":"2023-11-06T03:13:41.794888+0000","last_peered":"2023-11-06T03:13:41.794888+0000","last_clean":"2023-11-06T03:13:41.794888+0000","last_became_active":"2023-11-06T03:13:40.459779+0000","last_became_peered":"2023-11-06T03:13:40.459779+0000","last_unstale":"2023-11-06T03:13:41.794888+0000","last_undegraded":"2023-11-06T03:13:41.794888+0000","last_fullsized":"2023-11-06T03:13:41.794888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:12:22.584472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.480836+0000","last_change":"2023-11-06T03:13:40.058048+0000","last_active":"2023-11-06T03:13:40.480836+0000","last_peered":"2023-11-06T03:13:40.480836+0000","last_clean":"2023-11-06T03:13:40.480836+0000","last_became_active":"2023-11-06T03:13:40.057506+0000","last_became_peered":"2023-11-06T03:13:40.057506+0000","last_unstale":"2023-11-06T03:13:40.480836+0000","last_undegraded":"2023-11-06T03:13:40.480836+0000","last_fullsized":"2023-11-06T03:13:40.480836+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:28:29.298925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.433975+0000","last_change":"2023-11-06T03:13:40.472892+0000","last_active":"2023-11-06T03:13:41.433975+0000","last_peered":"2023-11-06T03:13:41.433975+0000","last_clean":"2023-11-06T03:13:41.433975+0000","last_became_active":"2023-11-06T03:13:40.471804+0000","last_became_peered":"2023-11-06T03:13:40.471804+0000","last_unstale":"2023-11-06T03:13:41.433975+0000","last_undegraded":"2023-11-06T03:13:41.433975+0000","last_fullsized":"2023-11-06T03:13:41.433975+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:17.795014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.446267+0000","last_change":"2023-11-06T03:13:40.066966+0000","last_active":"2023-11-06T03:13:40.446267+0000","last_peered":"2023-11-06T03:13:40.446267+0000","last_clean":"2023-11-06T03:13:40.446267+0000","last_became_active":"2023-11-06T03:13:40.066549+0000","last_became_peered":"2023-11-06T03:13:40.066549+0000","last_unstale":"2023-11-06T03:13:40.446267+0000","last_undegraded":"2023-11-06T03:13:40.446267+0000","last_fullsized":"2023-11-06T03:13:40.446267+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:21.993580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794012+0000","last_change":"2023-11-06T03:13:40.422380+0000","last_active":"2023-11-06T03:13:41.794012+0000","last_peered":"2023-11-06T03:13:41.794012+0000","last_clean":"2023-11-06T03:13:41.794012+0000","last_became_active":"2023-11-06T03:13:40.422116+0000","last_became_peered":"2023-11-06T03:13:40.422116+0000","last_unstale":"2023-11-06T03:13:41.794012+0000","last_undegraded":"2023-11-06T03:13:41.794012+0000","last_fullsized":"2023-11-06T03:13:41.794012+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:27:04.877254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.424290+0000","last_change":"2023-11-06T03:13:40.066843+0000","last_active":"2023-11-06T03:13:40.424290+0000","last_peered":"2023-11-06T03:13:40.424290+0000","last_clean":"2023-11-06T03:13:40.424290+0000","last_became_active":"2023-11-06T03:13:40.066267+0000","last_became_peered":"2023-11-06T03:13:40.066267+0000","last_unstale":"2023-11-06T03:13:40.424290+0000","last_undegraded":"2023-11-06T03:13:40.424290+0000","last_fullsized":"2023-11-06T03:13:40.424290+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795158+0000","last_change":"2023-11-06T03:13:40.421721+0000","last_active":"2023-11-06T03:13:41.795158+0000","last_peered":"2023-11-06T03:13:41.795158+0000","last_clean":"2023-11-06T03:13:41.795158+0000","last_became_active":"2023-11-06T03:13:40.421424+0000","last_became_peered":"2023-11-06T03:13:40.421424+0000","last_unstale":"2023-11-06T03:13:41.795158+0000","last_undegraded":"2023-11-06T03:13:41.795158+0000","last_fullsized":"2023-11-06T03:13:41.795158+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:21.894560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.429912+0000","last_change":"2023-11-06T03:13:40.060219+0000","last_active":"2023-11-06T03:13:40.429912+0000","last_peered":"2023-11-06T03:13:40.429912+0000","last_clean":"2023-11-06T03:13:40.429912+0000","last_became_active":"2023-11-06T03:13:40.059951+0000","last_became_peered":"2023-11-06T03:13:40.059951+0000","last_unstale":"2023-11-06T03:13:40.429912+0000","last_undegraded":"2023-11-06T03:13:40.429912+0000","last_fullsized":"2023-11-06T03:13:40.429912+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:31:08.490119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.797680+0000","last_change":"2023-11-06T03:13:40.442021+0000","last_active":"2023-11-06T03:13:41.797680+0000","last_peered":"2023-11-06T03:13:41.797680+0000","last_clean":"2023-11-06T03:13:41.797680+0000","last_became_active":"2023-11-06T03:13:40.441310+0000","last_became_peered":"2023-11-06T03:13:40.441310+0000","last_unstale":"2023-11-06T03:13:41.797680+0000","last_undegraded":"2023-11-06T03:13:41.797680+0000","last_fullsized":"2023-11-06T03:13:41.797680+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:43.862969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.446606+0000","last_change":"2023-11-06T03:13:40.061886+0000","last_active":"2023-11-06T03:13:40.446606+0000","last_peered":"2023-11-06T03:13:40.446606+0000","last_clean":"2023-11-06T03:13:40.446606+0000","last_became_active":"2023-11-06T03:13:40.061431+0000","last_became_peered":"2023-11-06T03:13:40.061431+0000","last_unstale":"2023-11-06T03:13:40.446606+0000","last_undegraded":"2023-11-06T03:13:40.446606+0000","last_fullsized":"2023-11-06T03:13:40.446606+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.434817+0000","last_change":"2023-11-06T03:13:40.470166+0000","last_active":"2023-11-06T03:13:41.434817+0000","last_peered":"2023-11-06T03:13:41.434817+0000","last_clean":"2023-11-06T03:13:41.434817+0000","last_became_active":"2023-11-06T03:13:40.469350+0000","last_became_peered":"2023-11-06T03:13:40.469350+0000","last_unstale":"2023-11-06T03:13:41.434817+0000","last_undegraded":"2023-11-06T03:13:41.434817+0000","last_fullsized":"2023-11-06T03:13:41.434817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:27:04.877254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.446663+0000","last_change":"2023-11-06T03:13:40.060752+0000","last_active":"2023-11-06T03:13:40.446663+0000","last_peered":"2023-11-06T03:13:40.446663+0000","last_clean":"2023-11-06T03:13:40.446663+0000","last_became_active":"2023-11-06T03:13:40.060480+0000","last_became_peered":"2023-11-06T03:13:40.060480+0000","last_unstale":"2023-11-06T03:13:40.446663+0000","last_undegraded":"2023-11-06T03:13:40.446663+0000","last_fullsized":"2023-11-06T03:13:40.446663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:39.011223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.429013+0000","last_change":"2023-11-06T03:13:40.472927+0000","last_active":"2023-11-06T03:13:41.429013+0000","last_peered":"2023-11-06T03:13:41.429013+0000","last_clean":"2023-11-06T03:13:41.429013+0000","last_became_active":"2023-11-06T03:13:40.472647+0000","last_became_peered":"2023-11-06T03:13:40.472647+0000","last_unstale":"2023-11-06T03:13:41.429013+0000","last_undegraded":"2023-11-06T03:13:41.429013+0000","last_fullsized":"2023-11-06T03:13:41.429013+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:01:37.408819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.482408+0000","last_change":"2023-11-06T03:13:40.085647+0000","last_active":"2023-11-06T03:13:40.482408+0000","last_peered":"2023-11-06T03:13:40.482408+0000","last_clean":"2023-11-06T03:13:40.482408+0000","last_became_active":"2023-11-06T03:13:40.085037+0000","last_became_peered":"2023-11-06T03:13:40.085037+0000","last_unstale":"2023-11-06T03:13:40.482408+0000","last_undegraded":"2023-11-06T03:13:40.482408+0000","last_fullsized":"2023-11-06T03:13:40.482408+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:17.428815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798983+0000","last_change":"2023-11-06T03:13:40.473965+0000","last_active":"2023-11-06T03:13:41.798983+0000","last_peered":"2023-11-06T03:13:41.798983+0000","last_clean":"2023-11-06T03:13:41.798983+0000","last_became_active":"2023-11-06T03:13:40.473158+0000","last_became_peered":"2023-11-06T03:13:40.473158+0000","last_unstale":"2023-11-06T03:13:41.798983+0000","last_undegraded":"2023-11-06T03:13:41.798983+0000","last_fullsized":"2023-11-06T03:13:41.798983+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:56.555041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.436782+0000","last_change":"2023-11-06T03:13:40.083705+0000","last_active":"2023-11-06T03:13:40.436782+0000","last_peered":"2023-11-06T03:13:40.436782+0000","last_clean":"2023-11-06T03:13:40.436782+0000","last_became_active":"2023-11-06T03:13:40.083163+0000","last_became_peered":"2023-11-06T03:13:40.083163+0000","last_unstale":"2023-11-06T03:13:40.436782+0000","last_undegraded":"2023-11-06T03:13:40.436782+0000","last_fullsized":"2023-11-06T03:13:40.436782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428478+0000","last_change":"2023-11-06T03:13:40.472915+0000","last_active":"2023-11-06T03:13:41.428478+0000","last_peered":"2023-11-06T03:13:41.428478+0000","last_clean":"2023-11-06T03:13:41.428478+0000","last_became_active":"2023-11-06T03:13:40.472654+0000","last_became_peered":"2023-11-06T03:13:40.472654+0000","last_unstale":"2023-11-06T03:13:41.428478+0000","last_undegraded":"2023-11-06T03:13:41.428478+0000","last_fullsized":"2023-11-06T03:13:41.428478+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:18.547279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.469944+0000","last_change":"2023-11-06T03:13:40.083620+0000","last_active":"2023-11-06T03:13:40.469944+0000","last_peered":"2023-11-06T03:13:40.469944+0000","last_clean":"2023-11-06T03:13:40.469944+0000","last_became_active":"2023-11-06T03:13:40.083316+0000","last_became_peered":"2023-11-06T03:13:40.083316+0000","last_unstale":"2023-11-06T03:13:40.469944+0000","last_undegraded":"2023-11-06T03:13:40.469944+0000","last_fullsized":"2023-11-06T03:13:40.469944+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:29:11.881515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.793543+0000","last_change":"2023-11-06T03:13:40.443743+0000","last_active":"2023-11-06T03:13:41.793543+0000","last_peered":"2023-11-06T03:13:41.793543+0000","last_clean":"2023-11-06T03:13:41.793543+0000","last_became_active":"2023-11-06T03:13:40.443456+0000","last_became_peered":"2023-11-06T03:13:40.443456+0000","last_unstale":"2023-11-06T03:13:41.793543+0000","last_undegraded":"2023-11-06T03:13:41.793543+0000","last_fullsized":"2023-11-06T03:13:41.793543+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:02:00.714992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.437269+0000","last_change":"2023-11-06T03:13:40.087381+0000","last_active":"2023-11-06T03:13:40.437269+0000","last_peered":"2023-11-06T03:13:40.437269+0000","last_clean":"2023-11-06T03:13:40.437269+0000","last_became_active":"2023-11-06T03:13:40.086739+0000","last_became_peered":"2023-11-06T03:13:40.086739+0000","last_unstale":"2023-11-06T03:13:40.437269+0000","last_undegraded":"2023-11-06T03:13:40.437269+0000","last_fullsized":"2023-11-06T03:13:40.437269+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794129+0000","last_change":"2023-11-06T03:13:40.457802+0000","last_active":"2023-11-06T03:13:41.794129+0000","last_peered":"2023-11-06T03:13:41.794129+0000","last_clean":"2023-11-06T03:13:41.794129+0000","last_became_active":"2023-11-06T03:13:40.457560+0000","last_became_peered":"2023-11-06T03:13:40.457560+0000","last_unstale":"2023-11-06T03:13:41.794129+0000","last_undegraded":"2023-11-06T03:13:41.794129+0000","last_fullsized":"2023-11-06T03:13:41.794129+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:10.846329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.417827+0000","last_change":"2023-11-06T03:13:40.068508+0000","last_active":"2023-11-06T03:13:40.417827+0000","last_peered":"2023-11-06T03:13:40.417827+0000","last_clean":"2023-11-06T03:13:40.417827+0000","last_became_active":"2023-11-06T03:13:40.068211+0000","last_became_peered":"2023-11-06T03:13:40.068211+0000","last_unstale":"2023-11-06T03:13:40.417827+0000","last_undegraded":"2023-11-06T03:13:40.417827+0000","last_fullsized":"2023-11-06T03:13:40.417827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:38.921731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.797092+0000","last_change":"2023-11-06T03:13:40.442050+0000","last_active":"2023-11-06T03:13:41.797092+0000","last_peered":"2023-11-06T03:13:41.797092+0000","last_clean":"2023-11-06T03:13:41.797092+0000","last_became_active":"2023-11-06T03:13:40.441117+0000","last_became_peered":"2023-11-06T03:13:40.441117+0000","last_unstale":"2023-11-06T03:13:41.797092+0000","last_undegraded":"2023-11-06T03:13:41.797092+0000","last_fullsized":"2023-11-06T03:13:41.797092+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:21.894560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.433687+0000","last_change":"2023-11-06T03:13:40.083630+0000","last_active":"2023-11-06T03:13:40.433687+0000","last_peered":"2023-11-06T03:13:40.433687+0000","last_clean":"2023-11-06T03:13:40.433687+0000","last_became_active":"2023-11-06T03:13:40.082997+0000","last_became_peered":"2023-11-06T03:13:40.082997+0000","last_unstale":"2023-11-06T03:13:40.433687+0000","last_undegraded":"2023-11-06T03:13:40.433687+0000","last_fullsized":"2023-11-06T03:13:40.433687+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:16.479676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.429086+0000","last_change":"2023-11-06T03:13:40.434294+0000","last_active":"2023-11-06T03:13:41.429086+0000","last_peered":"2023-11-06T03:13:41.429086+0000","last_clean":"2023-11-06T03:13:41.429086+0000","last_became_active":"2023-11-06T03:13:40.433963+0000","last_became_peered":"2023-11-06T03:13:40.433963+0000","last_unstale":"2023-11-06T03:13:41.429086+0000","last_undegraded":"2023-11-06T03:13:41.429086+0000","last_fullsized":"2023-11-06T03:13:41.429086+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:16.796421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.480521+0000","last_change":"2023-11-06T03:13:40.057372+0000","last_active":"2023-11-06T03:13:40.480521+0000","last_peered":"2023-11-06T03:13:40.480521+0000","last_clean":"2023-11-06T03:13:40.480521+0000","last_became_active":"2023-11-06T03:13:40.057039+0000","last_became_peered":"2023-11-06T03:13:40.057039+0000","last_unstale":"2023-11-06T03:13:40.480521+0000","last_undegraded":"2023-11-06T03:13:40.480521+0000","last_fullsized":"2023-11-06T03:13:40.480521+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:56.526361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795023+0000","last_change":"2023-11-06T03:13:40.473114+0000","last_active":"2023-11-06T03:13:41.795023+0000","last_peered":"2023-11-06T03:13:41.795023+0000","last_clean":"2023-11-06T03:13:41.795023+0000","last_became_active":"2023-11-06T03:13:40.472834+0000","last_became_peered":"2023-11-06T03:13:40.472834+0000","last_unstale":"2023-11-06T03:13:41.795023+0000","last_undegraded":"2023-11-06T03:13:41.795023+0000","last_fullsized":"2023-11-06T03:13:41.795023+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:51:38.729883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.444788+0000","last_change":"2023-11-06T03:13:40.065436+0000","last_active":"2023-11-06T03:13:40.444788+0000","last_peered":"2023-11-06T03:13:40.444788+0000","last_clean":"2023-11-06T03:13:40.444788+0000","last_became_active":"2023-11-06T03:13:40.064781+0000","last_became_peered":"2023-11-06T03:13:40.064781+0000","last_unstale":"2023-11-06T03:13:40.444788+0000","last_undegraded":"2023-11-06T03:13:40.444788+0000","last_fullsized":"2023-11-06T03:13:40.444788+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:35:54.008865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.799955+0000","last_change":"2023-11-06T03:13:40.483733+0000","last_active":"2023-11-06T03:13:41.799955+0000","last_peered":"2023-11-06T03:13:41.799955+0000","last_clean":"2023-11-06T03:13:41.799955+0000","last_became_active":"2023-11-06T03:13:40.483130+0000","last_became_peered":"2023-11-06T03:13:40.483130+0000","last_unstale":"2023-11-06T03:13:41.799955+0000","last_undegraded":"2023-11-06T03:13:41.799955+0000","last_fullsized":"2023-11-06T03:13:41.799955+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:21.894560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.445448+0000","last_change":"2023-11-06T03:13:40.066226+0000","last_active":"2023-11-06T03:13:40.445448+0000","last_peered":"2023-11-06T03:13:40.445448+0000","last_clean":"2023-11-06T03:13:40.445448+0000","last_became_active":"2023-11-06T03:13:40.065573+0000","last_became_peered":"2023-11-06T03:13:40.065573+0000","last_unstale":"2023-11-06T03:13:40.445448+0000","last_undegraded":"2023-11-06T03:13:40.445448+0000","last_fullsized":"2023-11-06T03:13:40.445448+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:00:33.875845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796368+0000","last_change":"2023-11-06T03:13:40.451111+0000","last_active":"2023-11-06T03:13:41.796368+0000","last_peered":"2023-11-06T03:13:41.796368+0000","last_clean":"2023-11-06T03:13:41.796368+0000","last_became_active":"2023-11-06T03:13:40.450826+0000","last_became_peered":"2023-11-06T03:13:40.450826+0000","last_unstale":"2023-11-06T03:13:41.796368+0000","last_undegraded":"2023-11-06T03:13:41.796368+0000","last_fullsized":"2023-11-06T03:13:41.796368+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:03:23.071573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.431108+0000","last_change":"2023-11-06T03:13:40.061471+0000","last_active":"2023-11-06T03:13:40.431108+0000","last_peered":"2023-11-06T03:13:40.431108+0000","last_clean":"2023-11-06T03:13:40.431108+0000","last_became_active":"2023-11-06T03:13:40.061191+0000","last_became_peered":"2023-11-06T03:13:40.061191+0000","last_unstale":"2023-11-06T03:13:40.431108+0000","last_undegraded":"2023-11-06T03:13:40.431108+0000","last_fullsized":"2023-11-06T03:13:40.431108+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:29:34.297810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.428600+0000","last_change":"2023-11-06T03:13:40.473117+0000","last_active":"2023-11-06T03:13:41.428600+0000","last_peered":"2023-11-06T03:13:41.428600+0000","last_clean":"2023-11-06T03:13:41.428600+0000","last_became_active":"2023-11-06T03:13:40.472280+0000","last_became_peered":"2023-11-06T03:13:40.472280+0000","last_unstale":"2023-11-06T03:13:41.428600+0000","last_undegraded":"2023-11-06T03:13:41.428600+0000","last_fullsized":"2023-11-06T03:13:41.428600+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:14:51.127910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.458085+0000","last_change":"2023-11-06T03:13:40.057288+0000","last_active":"2023-11-06T03:13:40.458085+0000","last_peered":"2023-11-06T03:13:40.458085+0000","last_clean":"2023-11-06T03:13:40.458085+0000","last_became_active":"2023-11-06T03:13:40.056697+0000","last_became_peered":"2023-11-06T03:13:40.056697+0000","last_unstale":"2023-11-06T03:13:40.458085+0000","last_undegraded":"2023-11-06T03:13:40.458085+0000","last_fullsized":"2023-11-06T03:13:40.458085+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:42.848337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.793567+0000","last_change":"2023-11-06T03:13:40.443203+0000","last_active":"2023-11-06T03:13:41.793567+0000","last_peered":"2023-11-06T03:13:41.793567+0000","last_clean":"2023-11-06T03:13:41.793567+0000","last_became_active":"2023-11-06T03:13:40.442876+0000","last_became_peered":"2023-11-06T03:13:40.442876+0000","last_unstale":"2023-11-06T03:13:41.793567+0000","last_undegraded":"2023-11-06T03:13:41.793567+0000","last_fullsized":"2023-11-06T03:13:41.793567+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:59.230674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.457294+0000","last_change":"2023-11-06T03:13:40.067944+0000","last_active":"2023-11-06T03:13:40.457294+0000","last_peered":"2023-11-06T03:13:40.457294+0000","last_clean":"2023-11-06T03:13:40.457294+0000","last_became_active":"2023-11-06T03:13:40.067270+0000","last_became_peered":"2023-11-06T03:13:40.067270+0000","last_unstale":"2023-11-06T03:13:40.457294+0000","last_undegraded":"2023-11-06T03:13:40.457294+0000","last_fullsized":"2023-11-06T03:13:40.457294+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:28:05.879995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798049+0000","last_change":"2023-11-06T03:13:40.467049+0000","last_active":"2023-11-06T03:13:41.798049+0000","last_peered":"2023-11-06T03:13:41.798049+0000","last_clean":"2023-11-06T03:13:41.798049+0000","last_became_active":"2023-11-06T03:13:40.466378+0000","last_became_peered":"2023-11-06T03:13:40.466378+0000","last_unstale":"2023-11-06T03:13:41.798049+0000","last_undegraded":"2023-11-06T03:13:41.798049+0000","last_fullsized":"2023-11-06T03:13:41.798049+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:26.914969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.428817+0000","last_change":"2023-11-06T03:13:40.058955+0000","last_active":"2023-11-06T03:13:40.428817+0000","last_peered":"2023-11-06T03:13:40.428817+0000","last_clean":"2023-11-06T03:13:40.428817+0000","last_became_active":"2023-11-06T03:13:40.058576+0000","last_became_peered":"2023-11-06T03:13:40.058576+0000","last_unstale":"2023-11-06T03:13:40.428817+0000","last_undegraded":"2023-11-06T03:13:40.428817+0000","last_fullsized":"2023-11-06T03:13:40.428817+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:22.196403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.795072+0000","last_change":"2023-11-06T03:13:40.452618+0000","last_active":"2023-11-06T03:13:41.795072+0000","last_peered":"2023-11-06T03:13:41.795072+0000","last_clean":"2023-11-06T03:13:41.795072+0000","last_became_active":"2023-11-06T03:13:40.452368+0000","last_became_peered":"2023-11-06T03:13:40.452368+0000","last_unstale":"2023-11-06T03:13:41.795072+0000","last_undegraded":"2023-11-06T03:13:41.795072+0000","last_fullsized":"2023-11-06T03:13:41.795072+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:37:09.261917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.433869+0000","last_change":"2023-11-06T03:13:40.085057+0000","last_active":"2023-11-06T03:13:40.433869+0000","last_peered":"2023-11-06T03:13:40.433869+0000","last_clean":"2023-11-06T03:13:40.433869+0000","last_became_active":"2023-11-06T03:13:40.084304+0000","last_became_peered":"2023-11-06T03:13:40.084304+0000","last_unstale":"2023-11-06T03:13:40.433869+0000","last_undegraded":"2023-11-06T03:13:40.433869+0000","last_fullsized":"2023-11-06T03:13:40.433869+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:32:50.624567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798671+0000","last_change":"2023-11-06T03:13:40.466145+0000","last_active":"2023-11-06T03:13:41.798671+0000","last_peered":"2023-11-06T03:13:41.798671+0000","last_clean":"2023-11-06T03:13:41.798671+0000","last_became_active":"2023-11-06T03:13:40.465443+0000","last_became_peered":"2023-11-06T03:13:40.465443+0000","last_unstale":"2023-11-06T03:13:41.798671+0000","last_undegraded":"2023-11-06T03:13:41.798671+0000","last_fullsized":"2023-11-06T03:13:41.798671+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:45.069673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"124'8","reported_seq":19,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.470452+0000","last_change":"2023-11-06T03:13:40.078820+0000","last_active":"2023-11-06T03:13:40.470452+0000","last_peered":"2023-11-06T03:13:40.470452+0000","last_clean":"2023-11-06T03:13:40.470452+0000","last_became_active":"2023-11-06T03:13:40.078386+0000","last_became_peered":"2023-11-06T03:13:40.078386+0000","last_unstale":"2023-11-06T03:13:40.470452+0000","last_undegraded":"2023-11-06T03:13:40.470452+0000","last_fullsized":"2023-11-06T03:13:40.470452+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:51:29.358499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":252,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.436946+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:13:40.436946+0000","last_peered":"2023-11-06T03:13:40.436946+0000","last_clean":"2023-11-06T03:13:40.436946+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:13:40.436946+0000","last_undegraded":"2023-11-06T03:13:40.436946+0000","last_fullsized":"2023-11-06T03:13:40.436946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":97,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.423848+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:13:40.423848+0000","last_peered":"2023-11-06T03:13:40.423848+0000","last_clean":"2023-11-06T03:13:40.423848+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:13:40.423848+0000","last_undegraded":"2023-11-06T03:13:40.423848+0000","last_fullsized":"2023-11-06T03:13:40.423848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798463+0000","last_change":"2023-11-06T03:13:40.449230+0000","last_active":"2023-11-06T03:13:41.798463+0000","last_peered":"2023-11-06T03:13:41.798463+0000","last_clean":"2023-11-06T03:13:41.798463+0000","last_became_active":"2023-11-06T03:13:40.448559+0000","last_became_peered":"2023-11-06T03:13:40.448559+0000","last_unstale":"2023-11-06T03:13:41.798463+0000","last_undegraded":"2023-11-06T03:13:41.798463+0000","last_fullsized":"2023-11-06T03:13:41.798463+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:57:26.649670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.469685+0000","last_change":"2023-11-06T03:13:40.078642+0000","last_active":"2023-11-06T03:13:40.469685+0000","last_peered":"2023-11-06T03:13:40.469685+0000","last_clean":"2023-11-06T03:13:40.469685+0000","last_became_active":"2023-11-06T03:13:40.078264+0000","last_became_peered":"2023-11-06T03:13:40.078264+0000","last_unstale":"2023-11-06T03:13:40.469685+0000","last_undegraded":"2023-11-06T03:13:40.469685+0000","last_fullsized":"2023-11-06T03:13:40.469685+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:07:14.747461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":99,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.434005+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:13:40.434005+0000","last_peered":"2023-11-06T03:13:40.434005+0000","last_clean":"2023-11-06T03:13:40.434005+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:13:40.434005+0000","last_undegraded":"2023-11-06T03:13:40.434005+0000","last_fullsized":"2023-11-06T03:13:40.434005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794798+0000","last_change":"2023-11-06T03:13:40.452675+0000","last_active":"2023-11-06T03:13:41.794798+0000","last_peered":"2023-11-06T03:13:41.794798+0000","last_clean":"2023-11-06T03:13:41.794798+0000","last_became_active":"2023-11-06T03:13:40.452389+0000","last_became_peered":"2023-11-06T03:13:40.452389+0000","last_unstale":"2023-11-06T03:13:41.794798+0000","last_undegraded":"2023-11-06T03:13:41.794798+0000","last_fullsized":"2023-11-06T03:13:41.794798+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:26:52.264376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.418902+0000","last_change":"2023-11-06T03:13:40.072742+0000","last_active":"2023-11-06T03:13:40.418902+0000","last_peered":"2023-11-06T03:13:40.418902+0000","last_clean":"2023-11-06T03:13:40.418902+0000","last_became_active":"2023-11-06T03:13:40.072425+0000","last_became_peered":"2023-11-06T03:13:40.072425+0000","last_unstale":"2023-11-06T03:13:40.418902+0000","last_undegraded":"2023-11-06T03:13:40.418902+0000","last_fullsized":"2023-11-06T03:13:40.418902+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:12:26.910389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.427453+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:13:40.427453+0000","last_peered":"2023-11-06T03:13:40.427453+0000","last_clean":"2023-11-06T03:13:40.427453+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:13:40.427453+0000","last_undegraded":"2023-11-06T03:13:40.427453+0000","last_fullsized":"2023-11-06T03:13:40.427453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.796956+0000","last_change":"2023-11-06T03:13:40.473766+0000","last_active":"2023-11-06T03:13:41.796956+0000","last_peered":"2023-11-06T03:13:41.796956+0000","last_clean":"2023-11-06T03:13:41.796956+0000","last_became_active":"2023-11-06T03:13:40.473415+0000","last_became_peered":"2023-11-06T03:13:40.473415+0000","last_unstale":"2023-11-06T03:13:41.796956+0000","last_undegraded":"2023-11-06T03:13:41.796956+0000","last_fullsized":"2023-11-06T03:13:41.796956+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:25:38.085695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.417536+0000","last_change":"2023-11-06T03:13:40.065212+0000","last_active":"2023-11-06T03:13:40.417536+0000","last_peered":"2023-11-06T03:13:40.417536+0000","last_clean":"2023-11-06T03:13:40.417536+0000","last_became_active":"2023-11-06T03:13:40.064806+0000","last_became_peered":"2023-11-06T03:13:40.064806+0000","last_unstale":"2023-11-06T03:13:40.417536+0000","last_undegraded":"2023-11-06T03:13:40.417536+0000","last_fullsized":"2023-11-06T03:13:40.417536+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:31:19.048685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":97,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.422544+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:13:40.422544+0000","last_peered":"2023-11-06T03:13:40.422544+0000","last_clean":"2023-11-06T03:13:40.422544+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:13:40.422544+0000","last_undegraded":"2023-11-06T03:13:40.422544+0000","last_fullsized":"2023-11-06T03:13:40.422544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.799503+0000","last_change":"2023-11-06T03:13:40.466741+0000","last_active":"2023-11-06T03:13:41.799503+0000","last_peered":"2023-11-06T03:13:41.799503+0000","last_clean":"2023-11-06T03:13:41.799503+0000","last_became_active":"2023-11-06T03:13:40.466067+0000","last_became_peered":"2023-11-06T03:13:40.466067+0000","last_unstale":"2023-11-06T03:13:41.799503+0000","last_undegraded":"2023-11-06T03:13:41.799503+0000","last_fullsized":"2023-11-06T03:13:41.799503+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:55:14.951053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.444227+0000","last_change":"2023-11-06T03:13:40.071858+0000","last_active":"2023-11-06T03:13:40.444227+0000","last_peered":"2023-11-06T03:13:40.444227+0000","last_clean":"2023-11-06T03:13:40.444227+0000","last_became_active":"2023-11-06T03:13:40.071266+0000","last_became_peered":"2023-11-06T03:13:40.071266+0000","last_unstale":"2023-11-06T03:13:40.444227+0000","last_undegraded":"2023-11-06T03:13:40.444227+0000","last_fullsized":"2023-11-06T03:13:40.444227+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:30.620087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":103,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.419877+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:13:40.419877+0000","last_peered":"2023-11-06T03:13:40.419877+0000","last_clean":"2023-11-06T03:13:40.419877+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:13:40.419877+0000","last_undegraded":"2023-11-06T03:13:40.419877+0000","last_fullsized":"2023-11-06T03:13:40.419877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.797996+0000","last_change":"2023-11-06T03:13:40.445225+0000","last_active":"2023-11-06T03:13:41.797996+0000","last_peered":"2023-11-06T03:13:41.797996+0000","last_clean":"2023-11-06T03:13:41.797996+0000","last_became_active":"2023-11-06T03:13:40.444522+0000","last_became_peered":"2023-11-06T03:13:40.444522+0000","last_unstale":"2023-11-06T03:13:41.797996+0000","last_undegraded":"2023-11-06T03:13:41.797996+0000","last_fullsized":"2023-11-06T03:13:41.797996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:01:15.198135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.424741+0000","last_change":"2023-11-06T03:13:40.067876+0000","last_active":"2023-11-06T03:13:40.424741+0000","last_peered":"2023-11-06T03:13:40.424741+0000","last_clean":"2023-11-06T03:13:40.424741+0000","last_became_active":"2023-11-06T03:13:40.067154+0000","last_became_peered":"2023-11-06T03:13:40.067154+0000","last_unstale":"2023-11-06T03:13:40.424741+0000","last_undegraded":"2023-11-06T03:13:40.424741+0000","last_fullsized":"2023-11-06T03:13:40.424741+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:47:45.957577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":99,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.433101+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:13:40.433101+0000","last_peered":"2023-11-06T03:13:40.433101+0000","last_clean":"2023-11-06T03:13:40.433101+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:13:40.433101+0000","last_undegraded":"2023-11-06T03:13:40.433101+0000","last_fullsized":"2023-11-06T03:13:40.433101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.794399+0000","last_change":"2023-11-06T03:13:40.459317+0000","last_active":"2023-11-06T03:13:41.794399+0000","last_peered":"2023-11-06T03:13:41.794399+0000","last_clean":"2023-11-06T03:13:41.794399+0000","last_became_active":"2023-11-06T03:13:40.459123+0000","last_became_peered":"2023-11-06T03:13:40.459123+0000","last_unstale":"2023-11-06T03:13:41.794399+0000","last_undegraded":"2023-11-06T03:13:41.794399+0000","last_fullsized":"2023-11-06T03:13:41.794399+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:57:56.555041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"124'3","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.478453+0000","last_change":"2023-11-06T03:13:40.080431+0000","last_active":"2023-11-06T03:13:40.478453+0000","last_peered":"2023-11-06T03:13:40.478453+0000","last_clean":"2023-11-06T03:13:40.478453+0000","last_became_active":"2023-11-06T03:13:40.079839+0000","last_became_peered":"2023-11-06T03:13:40.079839+0000","last_unstale":"2023-11-06T03:13:40.478453+0000","last_undegraded":"2023-11-06T03:13:40.478453+0000","last_fullsized":"2023-11-06T03:13:40.478453+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:44:27.658662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.419208+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:13:40.419208+0000","last_peered":"2023-11-06T03:13:40.419208+0000","last_clean":"2023-11-06T03:13:40.419208+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:13:40.419208+0000","last_undegraded":"2023-11-06T03:13:40.419208+0000","last_fullsized":"2023-11-06T03:13:40.419208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-06T03:13:41.798984+0000","last_change":"2023-11-06T03:13:40.445060+0000","last_active":"2023-11-06T03:13:41.798984+0000","last_peered":"2023-11-06T03:13:41.798984+0000","last_clean":"2023-11-06T03:13:41.798984+0000","last_became_active":"2023-11-06T03:13:40.443469+0000","last_became_peered":"2023-11-06T03:13:40.443469+0000","last_unstale":"2023-11-06T03:13:41.798984+0000","last_undegraded":"2023-11-06T03:13:41.798984+0000","last_fullsized":"2023-11-06T03:13:41.798984+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:40.023544+0000","last_clean_scrub_stamp":"2023-11-06T03:13:40.023544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:12:22.584472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"124'6","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.487624+0000","last_change":"2023-11-06T03:13:40.065334+0000","last_active":"2023-11-06T03:13:40.487624+0000","last_peered":"2023-11-06T03:13:40.487624+0000","last_clean":"2023-11-06T03:13:40.487624+0000","last_became_active":"2023-11-06T03:13:40.064785+0000","last_became_peered":"2023-11-06T03:13:40.064785+0000","last_unstale":"2023-11-06T03:13:40.487624+0000","last_undegraded":"2023-11-06T03:13:40.487624+0000","last_fullsized":"2023-11-06T03:13:40.487624+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:13:39.008912+0000","last_clean_scrub_stamp":"2023-11-06T03:13:39.008912+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:20.986535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-11-06T03:13:40.444447+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:13:40.444447+0000","last_peered":"2023-11-06T03:13:40.444447+0000","last_clean":"2023-11-06T03:13:40.444447+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:13:40.444447+0000","last_undegraded":"2023-11-06T03:13:40.444447+0000","last_fullsized":"2023-11-06T03:13:40.444447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87240,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284600,"statfs":{"total":96636764160,"available":96547430400,"internally_reserved":0,"allocated":2162688,"data_stored":1308101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1830000000000001,"15min":1.4830000000000001},"last":0.498},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.85999999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.3620000000000001,"15min":1.397},"last":0.88}]},{"osd":1,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.84399999999999997},"min":{"1min":0.46400000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.147,"5min":1.2490000000000001,"15min":1.444},"last":1.048},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.46999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.153,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.754}]},{"osd":2,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.90700000000000003,"15min":0.88700000000000001},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1539999999999999,"5min":1.381,"15min":1.6080000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.61299999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.22,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.877,"15min":0.89700000000000002},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":2.512,"15min":2.512},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.877,"5min":1.097,"15min":1.0469999999999999},"min":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":10.596,"15min":10.596},"last":0.84899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96899999999999997,"15min":0.97099999999999997},"min":{"1min":0.71799999999999997,"5min":0.52000000000000002,"15min":0.499},"max":{"1min":1.5309999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.99199999999999999,"15min":1.0009999999999999},"min":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.47399999999999998},"max":{"1min":1.278,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0600000000000001,"15min":1.048},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3720000000000001,"5min":1.8740000000000001,"15min":1.9139999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.093,"15min":1.091},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.349,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.075}]},{"osd":6,"last update":"Mon Nov 6 03:13:10 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.169,"15min":1.1539999999999999},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.60799999999999998},"max":{"1min":1.454,"5min":2.548,"15min":2.548},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.1040000000000001,"15min":1.105},"min":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.3049999999999999,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":0.98799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64500,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307340,"statfs":{"total":96636764160,"available":96570716160,"internally_reserved":0,"allocated":1552384,"data_stored":716520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.97299999999999998,"15min":0.84999999999999998},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":1.081,"5min":16.073,"15min":16.073},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.2749999999999999,"15min":0.97299999999999998},"min":{"1min":0.50700000000000001,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.0609999999999999,"5min":40.905000000000001,"15min":40.905000000000001},"last":0.624}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.0269999999999999,"15min":0.90100000000000002},"min":{"1min":0.47199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1359999999999999,"5min":16.012,"15min":16.012},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.2889999999999999,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.81999999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.1279999999999999,"15min":0.91300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.105,"5min":27.803999999999998,"15min":27.803999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.94699999999999995},"min":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1619999999999999,"5min":2.843,"15min":2.843},"last":1.0760000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.647,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.2929999999999999,"5min":63.737000000000002,"15min":63.737000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.877,"5min":1.355,"15min":1.052},"min":{"1min":0.57699999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.278,"5min":41.079999999999998,"15min":41.079999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.335,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2609999999999999,"5min":24.867999999999999,"15min":24.867999999999999},"last":1.129},{"interface":"front","average":{"1min":1.056,"5min":1.413,"15min":1.2370000000000001},"min":{"1min":0.83099999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.3540000000000001,"5min":23.789000000000001,"15min":23.789000000000001},"last":1.028}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.411,"15min":1.1559999999999999},"min":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.294,"5min":40.959000000000003,"15min":40.959000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1379999999999999,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.3240000000000001,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.173}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.4079999999999999,"15min":1.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.429,"5min":23.748999999999999,"15min":23.748999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.056,"5min":1.3999999999999999,"15min":1.212},"min":{"1min":0.78200000000000003,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":1.371,"5min":28.164000000000001,"15min":28.164000000000001},"last":0.93700000000000006}]}]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58812,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313028,"statfs":{"total":96636764160,"available":96576540672,"internally_reserved":0,"allocated":1560576,"data_stored":717117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.7729999999999999,"15min":1.1299999999999999},"min":{"1min":0.39800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.0840000000000001,"5min":125.256,"15min":125.256},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.8129999999999999,"15min":1.159},"min":{"1min":0.495,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.028,"5min":126.98699999999999,"15min":126.98699999999999},"last":1.103}]},{"osd":1,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.78000000000000003},"min":{"1min":0.60799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.27,"5min":2.339,"15min":2.339},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.57199999999999995,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.95199999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84599999999999997,"15min":0.84499999999999997},"min":{"1min":0.63700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0609999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.258},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83899999999999997,"15min":0.83999999999999997},"min":{"1min":0.53500000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1799999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.91300000000000003,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.208,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.94499999999999995},"min":{"1min":0.60099999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3400000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.88400000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.66500000000000004,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5149999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.044,"15min":0.98699999999999999},"min":{"1min":0.76200000000000001,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":3.0030000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.171}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0349999999999999,"15min":1.01},"min":{"1min":0.70699999999999996,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.429,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0620000000000001,"15min":1.0249999999999999},"min":{"1min":0.79200000000000004,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3600000000000001,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0960000000000001,"15min":1.0249999999999999},"min":{"1min":0.85599999999999998,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.421,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.94299999999999995,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.514,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.081}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90752,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94281088,"statfs":{"total":96636764160,"available":96543834112,"internally_reserved":0,"allocated":1564672,"data_stored":718521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.26,"15min":0.96599999999999997},"min":{"1min":0.47699999999999998,"5min":0.436,"15min":0.372},"max":{"1min":1.0429999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":1.2929999999999999,"15min":0.95899999999999996},"min":{"1min":0.44700000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.99099999999999999,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.78500000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.51400000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1279999999999999,"5min":1.5029999999999999,"15min":2.3679999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.84499999999999997,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.032,"5min":1.3100000000000001,"15min":1.6719999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.97999999999999998},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1779999999999999,"5min":1.452,"15min":3.2879999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85999999999999999,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.042,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.95399999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.59399999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.091,"5min":3.198,"15min":3.198},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.64800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0960000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.0609999999999999,"15min":1.0269999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.143,"5min":10.192,"15min":10.192},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0600000000000001,"15min":1.089},"min":{"1min":0.80500000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.0489999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96099999999999997,"15min":0.998},"min":{"1min":0.66400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.514,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.87},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0409999999999999,"15min":1.1140000000000001},"min":{"1min":0.71499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.399,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.077,"15min":1.075},"min":{"1min":0.76800000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2010000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.006},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.103,"15min":1.171},"min":{"1min":0.79600000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4099999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":0.97999999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76864,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":75328,"kb_avail":94294976,"statfs":{"total":96636764160,"available":96558055424,"internally_reserved":0,"allocated":1564672,"data_stored":717536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77135872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57599999999999996,"15min":0.60299999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78200000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.84299999999999997,"5min":0.90400000000000003,"15min":1.524},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.26400000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.89200000000000002,"5min":1.371,"15min":1.371},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.93100000000000005,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.23300000000000001},"max":{"1min":0.95099999999999996,"5min":1.532,"15min":1.532},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.34599999999999997,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.206,"5min":1.284,"15min":1.284},"last":0.80200000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85499999999999998,"15min":0.88800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45000000000000001},"max":{"1min":1.2270000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.91100000000000003,"15min":0.93500000000000005},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.369,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":1.246}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":0.97599999999999998},"min":{"1min":0.626,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.536,"15min":1.6639999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96999999999999997,"15min":1.004},"min":{"1min":0.58099999999999996,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.698},"last":1.0509999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.999},"min":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.39300000000000002},"max":{"1min":1.3939999999999999,"5min":1.649,"15min":1.8620000000000001},"last":1.01},{"interface":"front","average":{"1min":0.872,"5min":0.94699999999999995,"15min":0.97999999999999998},"min":{"1min":0.49399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3129999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":1.03},"min":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4410000000000001,"5min":1.835,"15min":1.835},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.623,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.569,"5min":1.734,"15min":1.8979999999999999},"last":1.2010000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68596,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303244,"statfs":{"total":96636764160,"available":96566521856,"internally_reserved":0,"allocated":1552384,"data_stored":716957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.93400000000000005,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.253},{"interface":"front","average":{"1min":0.502,"5min":0.55100000000000005,"15min":0.61399999999999999},"min":{"1min":0.26500000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.88100000000000001,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.307}]},{"osd":1,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59299999999999997,"15min":0.63400000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.64300000000000002},"min":{"1min":0.32500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.0109999999999999,"5min":1.099,"15min":1.1579999999999999},"last":0.376}]},{"osd":3,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.66800000000000004},"min":{"1min":0.35499999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0369999999999999,"5min":1.161,"15min":1.294},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65000000000000002,"15min":0.71699999999999997},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.022,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83199999999999996,"15min":0.86899999999999999},"min":{"1min":0.497,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.071,"5min":1.526,"15min":1.526},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83399999999999996,"15min":0.91500000000000004},"min":{"1min":0.52300000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1759999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82899999999999996,"15min":0.86699999999999999},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.44500000000000001},"max":{"1min":1.248,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.56000000000000005,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.2390000000000001,"5min":1.486,"15min":1.5780000000000001},"last":0.496}]},{"osd":6,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.95499999999999996,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.381,"15min":1.615},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.94499999999999995},"min":{"1min":0.66100000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.2689999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.751}]},{"osd":7,"last update":"Mon Nov 6 03:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.91400000000000003,"15min":0.97699999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3109999999999999,"5min":1.4650000000000001,"15min":4.2809999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88800000000000001,"15min":0.94199999999999995},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.47899999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.4740000000000002},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":1556480,"data_stored":716979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.58299999999999996},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.0569999999999999,"5min":1.843,"15min":1.843},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.81200000000000006,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.56499999999999995,"15min":0.70299999999999996},"min":{"1min":0.26900000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.76300000000000001,"5min":1.169,"15min":30.861000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.60199999999999998,"15min":0.65800000000000003},"min":{"1min":0.30099999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.91300000000000003,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.66700000000000004,"15min":0.70799999999999996},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.24199999999999999},"max":{"1min":1.0329999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.69799999999999995},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.0369999999999999,"5min":1.151,"15min":2.3079999999999998},"last":1.129}]},{"osd":4,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.82099999999999995,"15min":0.85999999999999999},"min":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":1.198,"5min":2.0870000000000002,"15min":2.258},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.82499999999999996,"15min":0.92900000000000005},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":1.5549999999999999,"15min":14.753},"last":0.77400000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":1.101},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47299999999999998},"max":{"1min":1.1499999999999999,"5min":1.528,"15min":43.103999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.92700000000000005,"15min":1.133},"min":{"1min":0.67000000000000004,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.208,"5min":1.3979999999999999,"15min":44.753999999999998},"last":1.0940000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.88,"15min":1.0409999999999999},"min":{"1min":0.54800000000000004,"5min":0.51400000000000001,"15min":0.45100000000000001},"max":{"1min":1.2909999999999999,"5min":1.5189999999999999,"15min":23.971},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":1.0629999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0649999999999999,"5min":1.444,"15min":14.813000000000001},"last":1.0229999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":1.0389999999999999},"min":{"1min":0.67600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.212,"5min":1.4890000000000001,"15min":2.2799999999999998},"last":1.004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.95699999999999996,"15min":1.0229999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.319,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":1.1799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91384,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":89280,"kb_avail":94280456,"statfs":{"total":96636764160,"available":96543186944,"internally_reserved":0,"allocated":2146304,"data_stored":1306328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91422720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.27500000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.80500000000000005,"5min":1.145,"15min":2.6989999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.59999999999999998,"15min":0.64900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.64600000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.82399999999999995,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.84799999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.60399999999999998,"15min":0.626},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.391,"5min":1.391,"15min":2.7999999999999998},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.86199999999999999,"15min":0.91800000000000004},"min":{"1min":0.54700000000000004,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.371,"5min":1.371,"15min":4.641},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82599999999999996,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2330000000000001,"5min":1.254,"15min":2.028},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90900000000000003,"15min":0.96399999999999997},"min":{"1min":0.51400000000000001,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.0920000000000001,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.93999999999999995},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.149}]},{"osd":6,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.57399999999999995,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.96399999999999997,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2949999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.1259999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93100000000000005,"15min":0.996},"min":{"1min":0.56200000000000006,"5min":0.495,"15min":0.495},"max":{"1min":1.266,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.103},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.93799999999999994,"15min":0.97999999999999998},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.335,"5min":1.4670000000000001,"15min":1.97},"last":1.016}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:13:48.225 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:13:48.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:13:48.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 Processor -- start 2023-11-06T03:13:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 -- start start 2023-11-06T03:13:48.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9581599c0 0x7fc9580ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95815e440 con 0x7fc95815dea0 2023-11-06T03:13:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95815adb0 con 0x7fc9581599c0 2023-11-06T03:13:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc95d9e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95806c240 con 0x7fc95806d000 2023-11-06T03:13:48.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9581599c0 0x7fc9580ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc9581599c0 0x7fc9580ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55194/0 (socket says 172.21.15.157:55194) 2023-11-06T03:13:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 -- 172.21.15.157:0/3777956403 learned_addr learned my addr 172.21.15.157:0/3777956403 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 -- 172.21.15.157:0/3777956403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 msgr2=0x7fc9580b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 --2- 172.21.15.157:0/3777956403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 -- 172.21.15.157:0/3777956403 >> v1:172.21.15.195:6789/0 conn(0x7fc95815dea0 legacy=0x7fc958070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.389+0000 7fc956ffd640 1 -- 172.21.15.157:0/3777956403 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95806c520 con 0x7fc9581599c0 2023-11-06T03:13:48.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc956ffd640 1 --2- 172.21.15.157:0/3777956403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9581599c0 0x7fc9580ae000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc948009870 tx=0x7fc94801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7eebee596ea1b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc955ffb640 1 -- 172.21.15.157:0/3777956403 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480ac600 con 0x7fc9581599c0 2023-11-06T03:13:48.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc955ffb640 1 -- 172.21.15.157:0/3777956403 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9480a8050 con 0x7fc9581599c0 2023-11-06T03:13:48.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc955ffb640 1 -- 172.21.15.157:0/3777956403 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480ac980 con 0x7fc9581599c0 2023-11-06T03:13:48.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777956403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9581599c0 msgr2=0x7fc9580ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777956403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9581599c0 0x7fc9580ae000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc948009870 tx=0x7fc94801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777956403 shutdown_connections 2023-11-06T03:13:48.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777956403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777956403 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9581599c0 0x7fc9580ae000 secure :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc948009870 tx=0x7fc94801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777956403 >> 172.21.15.157:0/3777956403 conn(0x7fc958065930 msgr2=0x7fc958065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777956403 shutdown_connections 2023-11-06T03:13:48.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777956403 wait complete. 2023-11-06T03:13:48.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 Processor -- start 2023-11-06T03:13:48.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.393+0000 7fc95d9e1640 1 -- start start 2023-11-06T03:13:48.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9580cb9d0 0x7fc9580cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580cc380 con 0x7fc95815dea0 2023-11-06T03:13:48.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580cdc30 con 0x7fc9580cb9d0 2023-11-06T03:13:48.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580cdf10 con 0x7fc95806d000 2023-11-06T03:13:48.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55206/0 (socket says 172.21.15.157:55206) 2023-11-06T03:13:48.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 -- 172.21.15.157:0/3777116453 learned_addr learned my addr 172.21.15.157:0/3777116453 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc956ffd640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9577fe640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9580cb9d0 0x7fc9580cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 -- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 msgr2=0x7fc9580ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 -- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9580cb9d0 msgr2=0x7fc9580cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9580cb9d0 0x7fc9580cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 -- 172.21.15.157:0/3777116453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480094d0 con 0x7fc95815dea0 2023-11-06T03:13:48.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc9567fc640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00b550 tx=0x7fc94c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c010870 con 0x7fc95815dea0 2023-11-06T03:13:48.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c0109f0 con 0x7fc95815dea0 2023-11-06T03:13:48.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c010da0 con 0x7fc95815dea0 2023-11-06T03:13:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9580ce1f0 con 0x7fc95815dea0 2023-11-06T03:13:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9580cd8e0 con 0x7fc95815dea0 2023-11-06T03:13:48.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9580ce710 con 0x7fc95815dea0 2023-11-06T03:13:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc94c00f3c0 con 0x7fc95815dea0 2023-11-06T03:13:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.397+0000 7fc93ffff640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 0x7fc944067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.401+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc94c083340 con 0x7fc95815dea0 2023-11-06T03:13:48.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.401+0000 7fc956ffd640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 0x7fc944067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.401+0000 7fc956ffd640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 0x7fc944067f20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc9480ae000 tx=0x7fc9480a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.405+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc9580ce710 con 0x7fc95815dea0 2023-11-06T03:13:48.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.609+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc9580ce710 con 0x7fc95815dea0 2023-11-06T03:13:48.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.617+0000 7fc93ffff640 1 -- 172.21.15.157:0/3777116453 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc9580ce710 con 0x7fc95815dea0 2023-11-06T03:13:48.614 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:13:48.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 msgr2=0x7fc944067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 0x7fc944067f20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc9480ae000 tx=0x7fc9480a7000 comp rx=0 tx=0).stop 2023-11-06T03:13:48.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 msgr2=0x7fc958138e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00b550 tx=0x7fc94c00ba10 comp rx=0 tx=0).stop 2023-11-06T03:13:48.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 shutdown_connections 2023-11-06T03:13:48.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc944065a60 0x7fc944067f20 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9580cb9d0 0x7fc9580cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc95815dea0 0x7fc958138e90 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 --2- 172.21.15.157:0/3777116453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc95806d000 0x7fc9580ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 >> 172.21.15.157:0/3777116453 conn(0x7fc958065930 msgr2=0x7fc958167270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 shutdown_connections 2023-11-06T03:13:48.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.621+0000 7fc95d9e1640 1 -- 172.21.15.157:0/3777116453 wait complete. 2023-11-06T03:13:48.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:13:48.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.753+0000 7fc7c6b8b640 1 Processor -- start 2023-11-06T03:13:48.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.753+0000 7fc7c6b8b640 1 -- start start 2023-11-06T03:13:48.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c015e5e0 0x7fc7c00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c015f580 con 0x7fc7c01551e0 2023-11-06T03:13:48.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00704b0 con 0x7fc7c015e5e0 2023-11-06T03:13:48.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c015efc0 con 0x7fc7c015ea90 2023-11-06T03:13:48.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c015e5e0 0x7fc7c00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7c015e5e0 0x7fc7c00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57968/0 (socket says 172.21.15.157:57968) 2023-11-06T03:13:48.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 -- 172.21.15.157:0/366254812 learned_addr learned my addr 172.21.15.157:0/366254812 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:48.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c5128640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 -- 172.21.15.157:0/366254812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 msgr2=0x7fc7c00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 -- 172.21.15.157:0/366254812 >> v1:172.21.15.195:6789/0 conn(0x7fc7c01551e0 legacy=0x7fc7c006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:48.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 -- 172.21.15.157:0/366254812 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c015f2a0 con 0x7fc7c015e5e0 2023-11-06T03:13:48.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bffff640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c015e5e0 0x7fc7c00b3410 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009870 tx=0x7fc7b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1291a92ce7ad42b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bf7fe640 1 -- 172.21.15.157:0/366254812 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40ab680 con 0x7fc7c015e5e0 2023-11-06T03:13:48.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bf7fe640 1 -- 172.21.15.157:0/366254812 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40ab7e0 con 0x7fc7c015e5e0 2023-11-06T03:13:48.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7bf7fe640 1 -- 172.21.15.157:0/366254812 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40abb70 con 0x7fc7c015e5e0 2023-11-06T03:13:48.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/366254812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c015e5e0 msgr2=0x7fc7c00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c015e5e0 0x7fc7c00b3410 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009870 tx=0x7fc7b401c480 comp rx=0 tx=0).stop 2023-11-06T03:13:48.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/366254812 shutdown_connections 2023-11-06T03:13:48.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/366254812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c015e5e0 0x7fc7c00b3410 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/366254812 >> 172.21.15.157:0/366254812 conn(0x7fc7c0065930 msgr2=0x7fc7c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/366254812 shutdown_connections 2023-11-06T03:13:48.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.757+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/366254812 wait complete. 2023-11-06T03:13:48.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 Processor -- start 2023-11-06T03:13:48.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- start start 2023-11-06T03:13:48.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c015e5e0 0x7fc7c0159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c006d510 con 0x7fc7c01551e0 2023-11-06T03:13:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00ce190 con 0x7fc7c015e5e0 2023-11-06T03:13:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00cdc10 con 0x7fc7c015ea90 2023-11-06T03:13:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57976/0 (socket says 172.21.15.157:57976) 2023-11-06T03:13:48.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 -- 172.21.15.157:0/4237327851 learned_addr learned my addr 172.21.15.157:0/4237327851 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:48.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c5128640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7bffff640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c015e5e0 0x7fc7c0159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 -- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 msgr2=0x7fc7c014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 -- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c015e5e0 msgr2=0x7fc7c0159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c015e5e0 0x7fc7c0159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 -- 172.21.15.157:0/4237327851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b40094d0 con 0x7fc7c01551e0 2023-11-06T03:13:48.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c5128640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:48.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c4927640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c530 tx=0x7fc7b000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0005220 con 0x7fc7c01551e0 2023-11-06T03:13:48.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0005380 con 0x7fc7c01551e0 2023-11-06T03:13:48.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0007270 con 0x7fc7c01551e0 2023-11-06T03:13:48.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c00a8b60 con 0x7fc7c01551e0 2023-11-06T03:13:48.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c00cdeb0 con 0x7fc7c01551e0 2023-11-06T03:13:48.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.761+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c0072750 con 0x7fc7c01551e0 2023-11-06T03:13:48.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.765+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7b00073d0 con 0x7fc7c01551e0 2023-11-06T03:13:48.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.765+0000 7fc7bd7fa640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 0x7fc7ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:48.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.765+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7b0082b60 con 0x7fc7c01551e0 2023-11-06T03:13:48.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.765+0000 7fc7bffff640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 0x7fc7ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:48.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.765+0000 7fc7bffff640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 0x7fc7ac067f20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40af000 tx=0x7fc7b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:48.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.769+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7b0050450 con 0x7fc7c01551e0 2023-11-06T03:13:48.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.941+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c00674b0 con 0x7fc7c01551e0 2023-11-06T03:13:48.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7bd7fa640 1 -- 172.21.15.157:0/4237327851 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fc7b0053a70 con 0x7fc7c01551e0 2023-11-06T03:13:48.941 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:13:48.941 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:13:48.941 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:13:48.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 msgr2=0x7fc7ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 0x7fc7ac067f20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40af000 tx=0x7fc7b4008040 comp rx=0 tx=0).stop 2023-11-06T03:13:48.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 msgr2=0x7fc7c006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:48.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c530 tx=0x7fc7b000c9f0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 shutdown_connections 2023-11-06T03:13:48.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7ac065a60 0x7fc7ac067f20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7c015ea90 0x7fc7c014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7c015e5e0 0x7fc7c0159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 --2- 172.21.15.157:0/4237327851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7c01551e0 0x7fc7c006cfd0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 >> 172.21.15.157:0/4237327851 conn(0x7fc7c0065930 msgr2=0x7fc7c00ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:48.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 shutdown_connections 2023-11-06T03:13:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:48.945+0000 7fc7c6b8b640 1 -- 172.21.15.157:0/4237327851 wait complete. 2023-11-06T03:13:49.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:13:40.410329+0000', 'modified': '2023-11-06T03:13:42.426309+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8796}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8796': {'gid': 8796, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/3017914039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3017914039}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3017914039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:13:49.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:13:49.362 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:13:49.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 Processor -- start 2023-11-06T03:13:49.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 -- start start 2023-11-06T03:13:49.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d815caa0 0x7f00d80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815ce80 0x7f00d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80d2920 con 0x7f00d80ce1a0 2023-11-06T03:13:49.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d80d18a0 con 0x7f00d815caa0 2023-11-06T03:13:49.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00e068e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80d1b60 con 0x7f00d815ce80 2023-11-06T03:13:49.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d815caa0 0x7f00d80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00d815caa0 0x7f00d80cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57988/0 (socket says 172.21.15.157:57988) 2023-11-06T03:13:49.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 -- 172.21.15.157:0/53691792 learned_addr learned my addr 172.21.15.157:0/53691792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00ddc29640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815ce80 0x7f00d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 -- 172.21.15.157:0/53691792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815ce80 msgr2=0x7f00d80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815ce80 0x7f00d80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 -- 172.21.15.157:0/53691792 >> v1:172.21.15.195:6789/0 conn(0x7f00d80ce1a0 legacy=0x7f00d80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.497+0000 7f00de42a640 1 -- 172.21.15.157:0/53691792 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80d1e40 con 0x7f00d815caa0 2023-11-06T03:13:49.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00de42a640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815caa0 0x7f00d80cdc60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009870 tx=0x7f00d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7319dd4b849ef8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dd428640 1 -- 172.21.15.157:0/53691792 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40ac600 con 0x7f00d815caa0 2023-11-06T03:13:49.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dd428640 1 -- 172.21.15.157:0/53691792 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d40a8050 con 0x7f00d815caa0 2023-11-06T03:13:49.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dd428640 1 -- 172.21.15.157:0/53691792 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40ac980 con 0x7f00d815caa0 2023-11-06T03:13:49.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- 172.21.15.157:0/53691792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815caa0 msgr2=0x7f00d80cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815caa0 0x7f00d80cdc60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009870 tx=0x7f00d401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- 172.21.15.157:0/53691792 shutdown_connections 2023-11-06T03:13:49.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815ce80 0x7f00d80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- 172.21.15.157:0/53691792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815caa0 0x7f00d80cdc60 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- 172.21.15.157:0/53691792 >> 172.21.15.157:0/53691792 conn(0x7f00d8070790 msgr2=0x7f00d8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- 172.21.15.157:0/53691792 shutdown_connections 2023-11-06T03:13:49.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- 172.21.15.157:0/53691792 wait complete. 2023-11-06T03:13:49.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 Processor -- start 2023-11-06T03:13:49.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- start start 2023-11-06T03:13:49.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 0x7f00d8155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815caa0 0x7f00d8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80ccaf0 con 0x7f00d815ce80 2023-11-06T03:13:49.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8156680 con 0x7f00d80ce1a0 2023-11-06T03:13:49.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00e068e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8156960 con 0x7f00d815caa0 2023-11-06T03:13:49.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58004/0 (socket says 172.21.15.157:58004) 2023-11-06T03:13:49.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 -- 172.21.15.157:0/2609008376 learned_addr learned my addr 172.21.15.157:0/2609008376 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00ddc29640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815caa0 0x7f00d8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00de42a640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 0x7f00d8155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 -- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815caa0 msgr2=0x7f00d8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815caa0 0x7f00d8155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 -- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 msgr2=0x7f00d8155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 0x7f00d8155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 -- 172.21.15.157:0/2609008376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d40094d0 con 0x7f00d815ce80 2023-11-06T03:13:49.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00de42a640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 0x7f00d8155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:49.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00dec2b640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f00d000b9e0 tx=0x7f00d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.501+0000 7f00ceffd640 1 -- 172.21.15.157:0/2609008376 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0003b80 con 0x7f00d815ce80 2023-11-06T03:13:49.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ceffd640 1 -- 172.21.15.157:0/2609008376 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0003ce0 con 0x7f00d815ce80 2023-11-06T03:13:49.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ceffd640 1 -- 172.21.15.157:0/2609008376 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000cb60 con 0x7f00d815ce80 2023-11-06T03:13:49.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d80c7720 con 0x7f00d815ce80 2023-11-06T03:13:49.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d80c7ce0 con 0x7f00d815ce80 2023-11-06T03:13:49.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ceffd640 1 -- 172.21.15.157:0/2609008376 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f00d000ccc0 con 0x7f00d815ce80 2023-11-06T03:13:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ceffd640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 0x7f00bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ceffd640 1 -- 172.21.15.157:0/2609008376 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00d0083010 con 0x7f00d815ce80 2023-11-06T03:13:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00de42a640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 0x7f00bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 Processor -- start 2023-11-06T03:13:49.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 -- start start 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00de42a640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 0x7f00bc067f20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f00d40ae000 tx=0x7f00d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e4fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4ccdfc0 con 0x55fef4e50000 2023-11-06T03:13:49.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c5c1a0 con 0x55fef4e4f6e0 2023-11-06T03:13:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c59770 con 0x55fef4e05a90 2023-11-06T03:13:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ddc29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ddc29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58016/0 (socket says 172.21.15.157:58016) 2023-11-06T03:13:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00ddc29640 1 -- 172.21.15.157:0/2429538174 learned_addr learned my addr 172.21.15.157:0/2429538174 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00dec2b640 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.505+0000 7f00de42a640 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e4fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 -- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 msgr2=0x55fef4e05e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e05e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 -- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 msgr2=0x55fef4e4fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e4fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 -- 172.21.15.157:0/2429538174 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef4951890 con 0x55fef4e50000 2023-11-06T03:13:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ca40 tx=0x7f00c800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d58ae66bce8190 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00bbfff640 1 -- 172.21.15.157:0/2429538174 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800eb40 con 0x55fef4e50000 2023-11-06T03:13:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00bbfff640 1 -- 172.21.15.157:0/2429538174 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c800ecc0 con 0x55fef4e50000 2023-11-06T03:13:49.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00bbfff640 1 -- 172.21.15.157:0/2429538174 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8013450 con 0x55fef4e50000 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/2429538174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 msgr2=0x55fef4e05550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ca40 tx=0x7f00c800cf00 comp rx=0 tx=0).stop 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/2429538174 shutdown_connections 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e05e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e05550 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- 172.21.15.157:0/2429538174 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e4fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/2429538174 >> 172.21.15.157:0/2429538174 conn(0x55fef4e0bbf0 msgr2=0x55fef4e03e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/2429538174 shutdown_connections 2023-11-06T03:13:49.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/2429538174 wait complete. 2023-11-06T03:13:49.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- start start 2023-11-06T03:13:49.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 Processor -- start 2023-11-06T03:13:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e56830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e56d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c55060 con 0x55fef4e50000 2023-11-06T03:13:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4bf4290 con 0x55fef4e4f6e0 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c126f0 con 0x55fef4e05a90 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 10 client.0 _ll_register_callbacks cb 0x55fef4df11d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58032/0 (socket says 172.21.15.157:58032) 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 -- 172.21.15.157:0/3185978254 learned_addr learned my addr 172.21.15.157:0/3185978254 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00de42a640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e56830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00ddc29640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e56d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 -- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 msgr2=0x55fef4e56830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e56830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 -- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 msgr2=0x55fef4e56d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e56d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 -- 172.21.15.157:0/3185978254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c800c6a0 con 0x55fef4e50000 2023-11-06T03:13:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00dec2b640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f00d0081380 tx=0x7f00d0054670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddea94caaff657cd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0053c20 con 0x55fef4e50000 2023-11-06T03:13:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00d005c080 con 0x55fef4e50000 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fef4c539b0 con 0x55fef4e50000 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d0058160 con 0x55fef4e50000 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 10 client.8946 fetch_fsmap learned FSMap version 91 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55fef4c1a1e0 con 0x55fef4e50000 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fef4c539b0 con 0x55fef4e50000 2023-11-06T03:13:49.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f00d005a110 con 0x55fef4e50000 2023-11-06T03:13:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 10 client.8946 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:13:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 10 client.8946 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:13:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 20 client.8946 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e11410 0x55fef4e117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 4 client.8946 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.509+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fef4c844d0 con 0x55fef4e11410 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00de42a640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e11410 0x55fef4e117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00b97fa640 10 client.8946 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00de42a640 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e11410 0x55fef4e117f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe08d1ddf1f51ae2 server_cookie=edfffa56c91ca86a in_seq=0 out_seq=0 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00b97fa640 1 -- 172.21.15.157:0/3185978254 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fef4c844d0 con 0x55fef4e11410 2023-11-06T03:13:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00b97fa640 10 client.8946 handle_command_reply: tid=0 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 client.8946 shutdown 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 msgr2=0x55fef4e106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f00d0081380 tx=0x7f00d0054670 comp rx=0 tx=0).stop 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 shutdown_connections 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e50000 0x55fef4e106a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e11410 0x55fef4e117f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4e56d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 --2- 172.21.15.157:0/3185978254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e05a90 0x55fef4e56830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00b97fa640 0 client.8946 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:13:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 >> 172.21.15.157:0/3185978254 conn(0x55fef4e0bbf0 msgr2=0x55fef4e04530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 shutdown_connections 2023-11-06T03:13:49.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 1 -- 172.21.15.157:0/3185978254 wait complete. 2023-11-06T03:13:49.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.513+0000 7f00e37b6000 20 client.8946 trim_cache size 0 max 16384 2023-11-06T03:13:49.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.525+0000 7f00e37b6000 1 Processor -- start 2023-11-06T03:13:49.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.525+0000 7f00e37b6000 1 -- start start 2023-11-06T03:13:49.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e05a90 0x55fef4ea6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4ea7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4912940 con 0x55fef4e4f6e0 2023-11-06T03:13:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c55910 con 0x55fef4e05a90 2023-11-06T03:13:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c53030 con 0x55fef4e50000 2023-11-06T03:13:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58040/0 (socket says 172.21.15.157:58040) 2023-11-06T03:13:49.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 -- 172.21.15.157:0/161483975 learned_addr learned my addr 172.21.15.157:0/161483975 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 -- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 msgr2=0x55fef4ea7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4ea7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 -- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e05a90 msgr2=0x55fef4ea6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e05a90 0x55fef4ea6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 -- 172.21.15.157:0/161483975 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef4c00330 con 0x55fef4e4f6e0 2023-11-06T03:13:49.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f00c8016040 tx=0x7f00c800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97e9cd20903a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ccff9640 1 -- 172.21.15.157:0/161483975 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800edc0 con 0x55fef4e4f6e0 2023-11-06T03:13:49.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ccff9640 1 -- 172.21.15.157:0/161483975 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c8004040 con 0x55fef4e4f6e0 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ccff9640 1 -- 172.21.15.157:0/161483975 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c80043f0 con 0x55fef4e4f6e0 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- 172.21.15.157:0/161483975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 msgr2=0x55fef4ea7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f00c8016040 tx=0x7f00c800a000 comp rx=0 tx=0).stop 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- 172.21.15.157:0/161483975 shutdown_connections 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4ea7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e4f6e0 0x55fef4ea7450 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- 172.21.15.157:0/161483975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e05a90 0x55fef4ea6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- 172.21.15.157:0/161483975 >> 172.21.15.157:0/161483975 conn(0x55fef4e0bbf0 msgr2=0x55fef4e4b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- 172.21.15.157:0/161483975 shutdown_connections 2023-11-06T03:13:49.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- 172.21.15.157:0/161483975 wait complete. 2023-11-06T03:13:49.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- start start 2023-11-06T03:13:49.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 Processor -- start 2023-11-06T03:13:49.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4eac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4e51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c22bb0 con 0x55fef4e05a90 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef4b8c450 con 0x55fef4e4f6e0 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef4c1f980 con 0x55fef4e50000 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00e37b6000 10 client.0 _ll_register_callbacks cb 0x55fef4dd19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58056/0 (socket says 172.21.15.157:58056) 2023-11-06T03:13:49.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 -- 172.21.15.157:0/4121908954 learned_addr learned my addr 172.21.15.157:0/4121908954 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4eac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00dec2b640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4e51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 -- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 msgr2=0x55fef4e51430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4e51430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 -- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 msgr2=0x55fef4eac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4eac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 -- 172.21.15.157:0/4121908954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fef4951890 con 0x55fef4e05a90 2023-11-06T03:13:49.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00ddc29640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4eac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:13:49.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00de42a640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c10 tx=0x7f00d4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72193bb764dbe6d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:13:49.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.529+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40bb040 con 0x55fef4e05a90 2023-11-06T03:13:49.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00d40c6950 con 0x55fef4e05a90 2023-11-06T03:13:49.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fef4c24b90 con 0x55fef4e05a90 2023-11-06T03:13:49.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d40c6420 con 0x55fef4e05a90 2023-11-06T03:13:49.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 10 client.8952 fetch_fsmap learned FSMap version 91 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55fef4951cc0 con 0x55fef4e05a90 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fef4c24b90 con 0x55fef4e05a90 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f00d40c6b10 con 0x55fef4e05a90 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 10 client.8952 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 10 client.8952 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:13:49.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 20 client.8952 populate_metadata read hostname 'smithi157' 2023-11-06T03:13:49.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e51970 0x55fef4e51d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 4 client.8952 mds_command: new command op to 8796 tid=0[{"prefix": "status"}] 2023-11-06T03:13:49.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fef4c4e0e0 con 0x55fef4e51970 2023-11-06T03:13:49.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00ddc29640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e51970 0x55fef4e51d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 10 client.8952 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:13:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00ddc29640 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e51970 0x55fef4e51d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ae37c6c0e75bcd server_cookie=d47630a7e51372fc in_seq=0 out_seq=0 2023-11-06T03:13:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 1 -- 172.21.15.157:0/4121908954 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fef4c4e0e0 con 0x55fef4e51970 2023-11-06T03:13:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 10 client.8952 handle_command_reply: tid=0 2023-11-06T03:13:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 client.8952 shutdown 2023-11-06T03:13:49.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 msgr2=0x55fef4eac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f00d4000c10 tx=0x7f00d4019270 comp rx=0 tx=0).stop 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 shutdown_connections 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fef4e50000 0x55fef4e51430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55fef4e51970 0x55fef4e51d50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fef4e4f6e0 0x55fef4eac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 --2- 172.21.15.157:0/4121908954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fef4e05a90 0x55fef4eac350 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00b97fa640 0 client.8952 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:13:49.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 >> 172.21.15.157:0/4121908954 conn(0x55fef4e0bbf0 msgr2=0x55fef4e4b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 shutdown_connections 2023-11-06T03:13:49.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 1 -- 172.21.15.157:0/4121908954 wait complete. 2023-11-06T03:13:49.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e37b6000 20 client.8952 trim_cache size 0 max 16384 2023-11-06T03:13:49.552 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:49.552 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:13:49.552 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:13:49.552 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8796, 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 9.1207009289999998, 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 91, 2023-11-06T03:13:49.553 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 125, 2023-11-06T03:13:49.554 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 125, 2023-11-06T03:13:49.554 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 13.578165408 2023-11-06T03:13:49.554 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:49.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 msgr2=0x7f00bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 0x7f00bc067f20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f00d40ae000 tx=0x7f00d4008040 comp rx=0 tx=0).stop 2023-11-06T03:13:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 msgr2=0x7f00d80c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f00d000b9e0 tx=0x7f00d000bea0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 shutdown_connections 2023-11-06T03:13:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00bc065a60 0x7f00bc067f20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00d815ce80 0x7f00d80c9f90 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00d815caa0 0x7f00d8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 --2- 172.21.15.157:0/2609008376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00d80ce1a0 0x7f00d8155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 >> 172.21.15.157:0/2609008376 conn(0x7f00d8070790 msgr2=0x7f00d8070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 shutdown_connections 2023-11-06T03:13:49.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.533+0000 7f00e068e640 1 -- 172.21.15.157:0/2609008376 wait complete. 2023-11-06T03:13:49.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:13:49.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 Processor -- start 2023-11-06T03:13:49.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 -- start start 2023-11-06T03:13:49.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f518006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f518015e5e0 0x7f51800b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518015f580 con 0x7f518015ea60 2023-11-06T03:13:49.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518015efc0 con 0x7f518015e5e0 2023-11-06T03:13:49.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f51852a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015f2a0 con 0x7f51801551e0 2023-11-06T03:13:49.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f518015ea60 0x7f518015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32996/0 (socket says 172.21.15.157:32996) 2023-11-06T03:13:49.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517f7fe640 1 -- 172.21.15.157:0/2136213230 learned_addr learned my addr 172.21.15.157:0/2136213230 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:13:49.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517e7fc640 1 --2- 172.21.15.157:0/2136213230 >> v2:172.21.15.157:3300/0 conn(0x7f518015e5e0 0x7f51800b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517effd640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f518006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517e7fc640 1 -- 172.21.15.157:0/2136213230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 msgr2=0x7f518006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:13:49.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517e7fc640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f518006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517e7fc640 1 -- 172.21.15.157:0/2136213230 >> v1:172.21.15.157:6789/0 conn(0x7f518015ea60 legacy=0x7f518015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517dffb640 1 -- 172.21.15.157:0/2136213230 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949621490 0 0) 0x7f518015f580 con 0x7f518015ea60 2023-11-06T03:13:49.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.677+0000 7f517e7fc640 1 -- 172.21.15.157:0/2136213230 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518006f5a0 con 0x7f518015e5e0 2023-11-06T03:13:49.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517e7fc640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015e5e0 0x7f51800b3410 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f51740097c0 tx=0x7f517401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e1d005d4bc4f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517dffb640 1 -- 172.21.15.157:0/2136213230 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740ab600 con 0x7f518015e5e0 2023-11-06T03:13:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517dffb640 1 -- 172.21.15.157:0/2136213230 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740a7050 con 0x7f518015e5e0 2023-11-06T03:13:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517dffb640 1 -- 172.21.15.157:0/2136213230 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740ab980 con 0x7f518015e5e0 2023-11-06T03:13:49.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- 172.21.15.157:0/2136213230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015e5e0 msgr2=0x7f51800b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015e5e0 0x7f51800b3410 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f51740097c0 tx=0x7f517401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- 172.21.15.157:0/2136213230 shutdown_connections 2023-11-06T03:13:49.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015e5e0 0x7f51800b3410 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- 172.21.15.157:0/2136213230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f518006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- 172.21.15.157:0/2136213230 >> 172.21.15.157:0/2136213230 conn(0x7f5180065930 msgr2=0x7f5180065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- 172.21.15.157:0/2136213230 shutdown_connections 2023-11-06T03:13:49.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- 172.21.15.157:0/2136213230 wait complete. 2023-11-06T03:13:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 Processor -- start 2023-11-06T03:13:49.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- start start 2023-11-06T03:13:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f51801488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518015e5e0 0x7f518006be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518014cf00 con 0x7f518015ea60 2023-11-06T03:13:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518006c810 con 0x7f518015e5e0 2023-11-06T03:13:49.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f51852a2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518014c5d0 con 0x7f51801551e0 2023-11-06T03:13:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58072/0 (socket says 172.21.15.157:58072) 2023-11-06T03:13:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 -- 172.21.15.157:0/3478459822 learned_addr learned my addr 172.21.15.157:0/3478459822 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:13:49.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517e7fc640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518015e5e0 0x7f518006be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517effd640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f51801488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 -- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 msgr2=0x7f51801488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f51801488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 -- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518015e5e0 msgr2=0x7f518006be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518015e5e0 0x7f518006be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 -- 172.21.15.157:0/3478459822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009420 con 0x7f518015ea60 2023-11-06T03:13:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.681+0000 7f517f7fe640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f517800c6d0 tx=0x7f517800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178004d70 con 0x7f518015ea60 2023-11-06T03:13:49.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517800a640 con 0x7f518015ea60 2023-11-06T03:13:49.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518014c8b0 con 0x7f518015ea60 2023-11-06T03:13:49.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800aa70 con 0x7f518015ea60 2023-11-06T03:13:49.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5180148df0 con 0x7f518015ea60 2023-11-06T03:13:49.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5180072750 con 0x7f518015ea60 2023-11-06T03:13:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51780070d0 con 0x7f518015ea60 2023-11-06T03:13:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 0x7f516c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:13:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5178082c90 con 0x7f518015ea60 2023-11-06T03:13:49.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f517effd640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 0x7f516c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:13:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.685+0000 7f517effd640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 0x7f516c067f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5170009790 tx=0x7f5170008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:13:49.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.689+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f51780504f0 con 0x7f518015ea60 2023-11-06T03:13:49.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f518014d040 con 0x7f518015ea60 2023-11-06T03:13:49.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51677fe640 1 -- 172.21.15.157:0/3478459822 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5178018070 con 0x7f518015ea60 2023-11-06T03:13:49.862 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:13:49.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 msgr2=0x7f516c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 0x7f516c067f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5170009790 tx=0x7f5170008040 comp rx=0 tx=0).stop 2023-11-06T03:13:49.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 msgr2=0x7f51801599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:13:49.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f517800c6d0 tx=0x7f517800cb90 comp rx=0 tx=0).stop 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 shutdown_connections 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f516c065a60 0x7f516c067f20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f518015ea60 0x7f51801599d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f518015e5e0 0x7f518006be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 --2- 172.21.15.157:0/3478459822 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51801551e0 0x7f51801488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:13:49.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.865+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 >> 172.21.15.157:0/3478459822 conn(0x7f5180065930 msgr2=0x7f51800ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:13:49.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.869+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 shutdown_connections 2023-11-06T03:13:49.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:13:49.869+0000 7f51852a2640 1 -- 172.21.15.157:0/3478459822 wait complete. 2023-11-06T03:13:49.879 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:13:49.879 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:13:49.880 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:13:49.880 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:13:49.881 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:49.881 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:13:49.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:13:49.882 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:13:49.882 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:13:49.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:49.883 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:13:49.889 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:13:49.890 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:13:49.890 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:13:49.890 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.890 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:13:49.890 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:13:49.891 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.891 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:13:49.891 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:13:49.891 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:13:49.892 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:13:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.892 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:13:49.892 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41909sec preferred_lft 41909sec 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:13:49.893 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.894 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:13:49.894 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.894 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:13:49.894 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:13:49.894 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:13:49.895 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.895 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:13:49.895 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.895 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:13:49.895 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:13:49.896 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:13:49.896 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.896 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:13:49.896 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:13:49.896 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:13:49.897 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:13:49.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:49.897 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:13:49.941 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:13:49.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:49.943 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:13:49.991 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:13:49.991 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:13:49.991 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:13:49.991 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:13:49.991 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:13:49.992 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:13:49.992 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:13:49.992 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:13:49.992 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:13:49.992 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:13:49.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:49.993 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:13:49.993 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:49.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:13:49.994 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:13:49.995 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:13:49.995 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:50.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:50.068 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:13:50.116 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:13:50.117 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:13:50.117 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:13:50.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:50.118 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:13:50.173 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:13:50.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:13:50.174 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:50.224 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:13:50.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:13:50.225 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:50.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:13:50.330 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:13:50.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:13:50.383 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:13:50.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:13:50.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:50.386 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:13:50.433 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:13:50.433 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:13:50.477 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:13:50.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:13:50.493 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:13:50.549 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:13:50.550 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:13:50.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:50.551 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:13:50.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:13:50.557+0000 7f50fe61a580 -1 init, newargv = 0x5615d0a9b560 newargc=15 2023-11-06T03:13:50.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[87403]: starting ceph client 2023-11-06T03:13:50.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[87403]: starting fuse 2023-11-06T03:13:51.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:13:51.678 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:13:51.678 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:13:51.732 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:13:51.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:13:51.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:51.734 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:13:51.780 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:13:51.781 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:13:51.781 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:13:51.782 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:13:51.782 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:13:51.783 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:13:51.783 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:13:51.784 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:13:51.784 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:13:51.784 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:13:51.785 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:13:51.785 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:13:51.785 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:13:51.786 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:13:51.786 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:13:51.787 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:13:51.787 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:13:51.787 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:13:51.788 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:13:51.788 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:13:51.788 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:13:51.789 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:13:51.789 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:13:51.790 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:13:51.790 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:13:51.790 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:13:51.791 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:13:51.791 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:13:51.791 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:13:51.901 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.87403.asok 2023-11-06T03:13:52.083 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87403.asok 2023-11-06T03:13:52.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:52.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87403.asok status 2023-11-06T03:13:52.244 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:13:52.244 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:13:52.245 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:13:52.245 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:13:52.245 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:13:52.245 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:13:52.245 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:13:52.246 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "87403", 2023-11-06T03:13:52.246 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:13:52.246 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:52.246 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:13:52.246 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 8967, 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 8967 2023-11-06T03:13:52.247 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:52.248 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:13:52.248 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:13:52.248 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:13:52.248 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 842602126 2023-11-06T03:13:52.248 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:13:52.249 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:52.249 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:13:52.249 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:13:52.249 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:13:52.249 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 842602126 2023-11-06T03:13:52.250 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:13:52.250 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.8967 192.168.0.1:0/842602126", 2023-11-06T03:13:52.250 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/842602126", 2023-11-06T03:13:52.250 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:13:52.250 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 91, 2023-11-06T03:13:52.251 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 125, 2023-11-06T03:13:52.251 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:13:52.251 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:13:52.251 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:13:52.251 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:13:52.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:52.258 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:13:52.259 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:52.265 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:13:52.266 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:52.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:52.266 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:13:52.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:52.321 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:52.321 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:52.321 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.0 2023-11-06T03:13:52.322 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:52.322 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:52.323 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:52.323 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:52.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.410 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190475 s, 55.1 MB/s 2023-11-06T03:13:52.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.426 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012672 s, 82.7 MB/s 2023-11-06T03:13:52.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162141 s, 64.7 MB/s 2023-11-06T03:13:52.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011438 s, 91.7 MB/s 2023-11-06T03:13:52.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.477 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012816 s, 81.8 MB/s 2023-11-06T03:13:52.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.495 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149895 s, 70.0 MB/s 2023-11-06T03:13:52.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.514 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156273 s, 67.1 MB/s 2023-11-06T03:13:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156136 s, 67.2 MB/s 2023-11-06T03:13:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132325 s, 79.2 MB/s 2023-11-06T03:13:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138891 s, 75.5 MB/s 2023-11-06T03:13:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139627 s, 75.1 MB/s 2023-11-06T03:13:52.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155311 s, 67.5 MB/s 2023-11-06T03:13:52.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150152 s, 69.8 MB/s 2023-11-06T03:13:52.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145723 s, 72.0 MB/s 2023-11-06T03:13:52.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143132 s, 73.3 MB/s 2023-11-06T03:13:52.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155276 s, 67.5 MB/s 2023-11-06T03:13:52.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.695 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015878 s, 66.0 MB/s 2023-11-06T03:13:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146244 s, 71.7 MB/s 2023-11-06T03:13:52.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142393 s, 73.6 MB/s 2023-11-06T03:13:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122944 s, 85.3 MB/s 2023-11-06T03:13:52.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.762 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147682 s, 71.0 MB/s 2023-11-06T03:13:52.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.780 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142876 s, 73.4 MB/s 2023-11-06T03:13:52.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.798 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139165 s, 75.3 MB/s 2023-11-06T03:13:52.813 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.813 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.813 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139579 s, 75.1 MB/s 2023-11-06T03:13:52.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.833 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169281 s, 61.9 MB/s 2023-11-06T03:13:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149484 s, 70.1 MB/s 2023-11-06T03:13:52.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.869 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144204 s, 72.7 MB/s 2023-11-06T03:13:52.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.887 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147665 s, 71.0 MB/s 2023-11-06T03:13:52.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147244 s, 71.2 MB/s 2023-11-06T03:13:52.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139021 s, 75.4 MB/s 2023-11-06T03:13:52.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.938 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134446 s, 78.0 MB/s 2023-11-06T03:13:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.954 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.955 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134768 s, 77.8 MB/s 2023-11-06T03:13:52.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:52.956 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:52.957 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:52.957 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.1 2023-11-06T03:13:52.957 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:52.958 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:52.958 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:52.958 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:52.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:52.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:52.985 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129 s, 81.3 MB/s 2023-11-06T03:13:53.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.002 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142541 s, 73.6 MB/s 2023-11-06T03:13:53.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133817 s, 78.4 MB/s 2023-11-06T03:13:53.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.034 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134523 s, 77.9 MB/s 2023-11-06T03:13:53.050 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.051 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145104 s, 72.3 MB/s 2023-11-06T03:13:53.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013264 s, 79.1 MB/s 2023-11-06T03:13:53.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.081 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112196 s, 93.5 MB/s 2023-11-06T03:13:53.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.096 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.096 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119897 s, 87.5 MB/s 2023-11-06T03:13:53.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.111 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124606 s, 84.2 MB/s 2023-11-06T03:13:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.124 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985641 s, 106 MB/s 2023-11-06T03:13:53.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134593 s, 77.9 MB/s 2023-11-06T03:13:53.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124343 s, 84.3 MB/s 2023-11-06T03:13:53.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.168 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.168 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104505 s, 100 MB/s 2023-11-06T03:13:53.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.182 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116237 s, 90.2 MB/s 2023-11-06T03:13:53.197 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.197 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.197 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125304 s, 83.7 MB/s 2023-11-06T03:13:53.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102691 s, 102 MB/s 2023-11-06T03:13:53.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.224 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104712 s, 100 MB/s 2023-11-06T03:13:53.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.241 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.241 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146571 s, 71.5 MB/s 2023-11-06T03:13:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.255 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118157 s, 88.7 MB/s 2023-11-06T03:13:53.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.271 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130062 s, 80.6 MB/s 2023-11-06T03:13:53.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.287 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130202 s, 80.5 MB/s 2023-11-06T03:13:53.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.299 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991886 s, 106 MB/s 2023-11-06T03:13:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.317 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136552 s, 76.8 MB/s 2023-11-06T03:13:53.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133158 s, 78.7 MB/s 2023-11-06T03:13:53.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.349 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.349 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133781 s, 78.4 MB/s 2023-11-06T03:13:53.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128281 s, 81.7 MB/s 2023-11-06T03:13:53.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.381 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136577 s, 76.8 MB/s 2023-11-06T03:13:53.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.398 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142685 s, 73.5 MB/s 2023-11-06T03:13:53.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.415 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139623 s, 75.1 MB/s 2023-11-06T03:13:53.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.432 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136446 s, 76.8 MB/s 2023-11-06T03:13:53.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143456 s, 73.1 MB/s 2023-11-06T03:13:53.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.466 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137497 s, 76.3 MB/s 2023-11-06T03:13:53.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:53.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:53.468 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:53.469 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.2 2023-11-06T03:13:53.469 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:53.469 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:53.470 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:53.470 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131417 s, 79.8 MB/s 2023-11-06T03:13:53.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.555 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014147 s, 74.1 MB/s 2023-11-06T03:13:53.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106236 s, 98.7 MB/s 2023-11-06T03:13:53.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.585 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125939 s, 83.3 MB/s 2023-11-06T03:13:53.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129577 s, 80.9 MB/s 2023-11-06T03:13:53.618 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.619 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138173 s, 75.9 MB/s 2023-11-06T03:13:53.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148027 s, 70.8 MB/s 2023-11-06T03:13:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.653 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130678 s, 80.2 MB/s 2023-11-06T03:13:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.669 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128698 s, 81.5 MB/s 2023-11-06T03:13:53.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139819 s, 75.0 MB/s 2023-11-06T03:13:53.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.705 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142406 s, 73.6 MB/s 2023-11-06T03:13:53.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.721 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.721 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137309 s, 76.4 MB/s 2023-11-06T03:13:53.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117167 s, 89.5 MB/s 2023-11-06T03:13:53.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.753 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.753 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139633 s, 75.1 MB/s 2023-11-06T03:13:53.767 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.767 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.767 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108649 s, 96.5 MB/s 2023-11-06T03:13:53.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135185 s, 77.6 MB/s 2023-11-06T03:13:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153798 s, 68.2 MB/s 2023-11-06T03:13:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.820 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144893 s, 72.4 MB/s 2023-11-06T03:13:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.837 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.838 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143183 s, 73.2 MB/s 2023-11-06T03:13:53.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142463 s, 73.6 MB/s 2023-11-06T03:13:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124334 s, 84.3 MB/s 2023-11-06T03:13:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.886 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012904 s, 81.3 MB/s 2023-11-06T03:13:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134432 s, 78.0 MB/s 2023-11-06T03:13:53.919 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.920 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145518 s, 72.1 MB/s 2023-11-06T03:13:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.934 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118277 s, 88.7 MB/s 2023-11-06T03:13:53.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.952 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144227 s, 72.7 MB/s 2023-11-06T03:13:53.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.967 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123815 s, 84.7 MB/s 2023-11-06T03:13:53.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.985 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01349 s, 77.7 MB/s 2023-11-06T03:13:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119315 s, 87.9 MB/s 2023-11-06T03:13:54.012 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.013 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.013 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01064 s, 98.6 MB/s 2023-11-06T03:13:54.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.029 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133826 s, 78.4 MB/s 2023-11-06T03:13:54.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.044 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123443 s, 84.9 MB/s 2023-11-06T03:13:54.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:54.045 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:54.046 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:54.046 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.3 2023-11-06T03:13:54.047 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:54.047 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:54.047 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:54.047 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137517 s, 76.3 MB/s 2023-11-06T03:13:54.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127191 s, 82.4 MB/s 2023-11-06T03:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.149 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140078 s, 74.9 MB/s 2023-11-06T03:13:54.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.166 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.166 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146161 s, 71.7 MB/s 2023-11-06T03:13:54.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014283 s, 73.4 MB/s 2023-11-06T03:13:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.198 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117835 s, 89.0 MB/s 2023-11-06T03:13:54.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.215 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013369 s, 78.4 MB/s 2023-11-06T03:13:54.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.230 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122099 s, 85.9 MB/s 2023-11-06T03:13:54.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.248 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146128 s, 71.8 MB/s 2023-11-06T03:13:54.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.265 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140711 s, 74.5 MB/s 2023-11-06T03:13:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140899 s, 74.4 MB/s 2023-11-06T03:13:54.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145838 s, 71.9 MB/s 2023-11-06T03:13:54.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.315 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119226 s, 87.9 MB/s 2023-11-06T03:13:54.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.329 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111877 s, 93.7 MB/s 2023-11-06T03:13:54.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.347 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144441 s, 72.6 MB/s 2023-11-06T03:13:54.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.365 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147857 s, 70.9 MB/s 2023-11-06T03:13:54.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.380 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129618 s, 80.9 MB/s 2023-11-06T03:13:54.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.395 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115307 s, 90.9 MB/s 2023-11-06T03:13:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.410 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125371 s, 83.6 MB/s 2023-11-06T03:13:54.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121226 s, 86.5 MB/s 2023-11-06T03:13:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.441 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132387 s, 79.2 MB/s 2023-11-06T03:13:54.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.455 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111852 s, 93.7 MB/s 2023-11-06T03:13:54.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133009 s, 78.8 MB/s 2023-11-06T03:13:54.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.486 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108852 s, 96.3 MB/s 2023-11-06T03:13:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128114 s, 81.8 MB/s 2023-11-06T03:13:54.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.515 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.515 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107981 s, 97.1 MB/s 2023-11-06T03:13:54.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132235 s, 79.3 MB/s 2023-11-06T03:13:54.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131153 s, 80.0 MB/s 2023-11-06T03:13:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106193 s, 98.7 MB/s 2023-11-06T03:13:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.578 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147576 s, 71.1 MB/s 2023-11-06T03:13:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.594 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134548 s, 77.9 MB/s 2023-11-06T03:13:54.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.610 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128588 s, 81.5 MB/s 2023-11-06T03:13:54.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:54.612 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:54.612 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:54.612 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.4 2023-11-06T03:13:54.613 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:54.613 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:54.613 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:54.614 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:54.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147192 s, 71.2 MB/s 2023-11-06T03:13:54.707 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.708 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171187 s, 61.3 MB/s 2023-11-06T03:13:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.727 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157835 s, 66.4 MB/s 2023-11-06T03:13:54.745 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.745 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150986 s, 69.4 MB/s 2023-11-06T03:13:54.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.762 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135646 s, 77.3 MB/s 2023-11-06T03:13:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.780 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145121 s, 72.3 MB/s 2023-11-06T03:13:54.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.797 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.797 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140246 s, 74.8 MB/s 2023-11-06T03:13:54.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147034 s, 71.3 MB/s 2023-11-06T03:13:54.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.831 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136824 s, 76.6 MB/s 2023-11-06T03:13:54.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.849 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140996 s, 74.4 MB/s 2023-11-06T03:13:54.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130539 s, 80.3 MB/s 2023-11-06T03:13:54.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112424 s, 93.3 MB/s 2023-11-06T03:13:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125975 s, 83.2 MB/s 2023-11-06T03:13:54.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.911 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.911 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140384 s, 74.7 MB/s 2023-11-06T03:13:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.927 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013387 s, 78.3 MB/s 2023-11-06T03:13:54.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151964 s, 69.0 MB/s 2023-11-06T03:13:54.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.962 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130255 s, 80.5 MB/s 2023-11-06T03:13:54.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.976 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113992 s, 92.0 MB/s 2023-11-06T03:13:54.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:54.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:54.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119479 s, 87.8 MB/s 2023-11-06T03:13:55.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013917 s, 75.3 MB/s 2023-11-06T03:13:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.026 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014316 s, 73.2 MB/s 2023-11-06T03:13:55.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147389 s, 71.1 MB/s 2023-11-06T03:13:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133364 s, 78.6 MB/s 2023-11-06T03:13:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143133 s, 73.3 MB/s 2023-11-06T03:13:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012891 s, 81.3 MB/s 2023-11-06T03:13:55.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147707 s, 71.0 MB/s 2023-11-06T03:13:55.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.125 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012047 s, 87.0 MB/s 2023-11-06T03:13:55.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116651 s, 89.9 MB/s 2023-11-06T03:13:55.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144852 s, 72.4 MB/s 2023-11-06T03:13:55.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142227 s, 73.7 MB/s 2023-11-06T03:13:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142728 s, 73.5 MB/s 2023-11-06T03:13:55.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.212 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150317 s, 69.8 MB/s 2023-11-06T03:13:55.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:55.214 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:55.214 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:55.215 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.5 2023-11-06T03:13:55.215 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:55.215 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:55.216 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:55.216 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:55.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.284 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115591 s, 90.7 MB/s 2023-11-06T03:13:55.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129704 s, 80.8 MB/s 2023-11-06T03:13:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.319 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149278 s, 70.2 MB/s 2023-11-06T03:13:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156337 s, 67.1 MB/s 2023-11-06T03:13:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151198 s, 69.4 MB/s 2023-11-06T03:13:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.375 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145319 s, 72.2 MB/s 2023-11-06T03:13:55.392 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.393 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147742 s, 71.0 MB/s 2023-11-06T03:13:55.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.408 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121748 s, 86.1 MB/s 2023-11-06T03:13:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.426 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142315 s, 73.7 MB/s 2023-11-06T03:13:55.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.443 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146138 s, 71.8 MB/s 2023-11-06T03:13:55.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143055 s, 73.3 MB/s 2023-11-06T03:13:55.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142856 s, 73.4 MB/s 2023-11-06T03:13:55.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145261 s, 72.2 MB/s 2023-11-06T03:13:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136058 s, 77.1 MB/s 2023-11-06T03:13:55.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.529 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131019 s, 80.0 MB/s 2023-11-06T03:13:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106889 s, 98.1 MB/s 2023-11-06T03:13:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123059 s, 85.2 MB/s 2023-11-06T03:13:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131797 s, 79.6 MB/s 2023-11-06T03:13:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140642 s, 74.6 MB/s 2023-11-06T03:13:55.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.608 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132632 s, 79.1 MB/s 2023-11-06T03:13:55.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.624 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129312 s, 81.1 MB/s 2023-11-06T03:13:55.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.640 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136943 s, 76.6 MB/s 2023-11-06T03:13:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133656 s, 78.5 MB/s 2023-11-06T03:13:55.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.673 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132141 s, 79.4 MB/s 2023-11-06T03:13:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124019 s, 84.5 MB/s 2023-11-06T03:13:55.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129623 s, 80.9 MB/s 2023-11-06T03:13:55.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.720 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143796 s, 72.9 MB/s 2023-11-06T03:13:55.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134192 s, 78.1 MB/s 2023-11-06T03:13:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152217 s, 68.9 MB/s 2023-11-06T03:13:55.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.768 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114441 s, 91.6 MB/s 2023-11-06T03:13:55.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.785 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138919 s, 75.5 MB/s 2023-11-06T03:13:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.800 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128606 s, 81.5 MB/s 2023-11-06T03:13:55.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:55.802 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:55.802 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:55.802 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.6 2023-11-06T03:13:55.803 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:55.803 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:55.804 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:55.804 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:55.869 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.869 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.869 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121447 s, 86.3 MB/s 2023-11-06T03:13:55.884 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.884 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.884 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116819 s, 89.8 MB/s 2023-11-06T03:13:55.899 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.900 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126548 s, 82.9 MB/s 2023-11-06T03:13:55.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.916 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133434 s, 78.6 MB/s 2023-11-06T03:13:55.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.931 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121319 s, 86.4 MB/s 2023-11-06T03:13:55.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.947 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130345 s, 80.4 MB/s 2023-11-06T03:13:55.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.963 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130303 s, 80.5 MB/s 2023-11-06T03:13:55.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113057 s, 92.7 MB/s 2023-11-06T03:13:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:55.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:55.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122671 s, 85.5 MB/s 2023-11-06T03:13:56.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133172 s, 78.7 MB/s 2023-11-06T03:13:56.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.023 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121721 s, 86.1 MB/s 2023-11-06T03:13:56.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.039 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127645 s, 82.1 MB/s 2023-11-06T03:13:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113232 s, 92.6 MB/s 2023-11-06T03:13:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.064 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885013 s, 118 MB/s 2023-11-06T03:13:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838884 s, 125 MB/s 2023-11-06T03:13:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111176 s, 94.3 MB/s 2023-11-06T03:13:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01102 s, 95.2 MB/s 2023-11-06T03:13:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105925 s, 99.0 MB/s 2023-11-06T03:13:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115711 s, 90.6 MB/s 2023-11-06T03:13:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107958 s, 97.1 MB/s 2023-11-06T03:13:56.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.156 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111959 s, 93.7 MB/s 2023-11-06T03:13:56.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011358 s, 92.3 MB/s 2023-11-06T03:13:56.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.185 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123425 s, 85.0 MB/s 2023-11-06T03:13:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896906 s, 117 MB/s 2023-11-06T03:13:56.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114529 s, 91.6 MB/s 2023-11-06T03:13:56.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.227 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142022 s, 73.8 MB/s 2023-11-06T03:13:56.241 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.242 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117386 s, 89.3 MB/s 2023-11-06T03:13:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100215 s, 105 MB/s 2023-11-06T03:13:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123876 s, 84.6 MB/s 2023-11-06T03:13:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120133 s, 87.3 MB/s 2023-11-06T03:13:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121258 s, 86.5 MB/s 2023-11-06T03:13:56.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124282 s, 84.4 MB/s 2023-11-06T03:13:56.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:56.315 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:56.316 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:56.316 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.7 2023-11-06T03:13:56.317 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:56.317 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:56.317 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:56.318 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:56.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.383 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013912 s, 75.4 MB/s 2023-11-06T03:13:56.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.399 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123926 s, 84.6 MB/s 2023-11-06T03:13:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132227 s, 79.3 MB/s 2023-11-06T03:13:56.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.430 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120953 s, 86.7 MB/s 2023-11-06T03:13:56.444 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.445 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121758 s, 86.1 MB/s 2023-11-06T03:13:56.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.460 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122274 s, 85.8 MB/s 2023-11-06T03:13:56.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.474 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117651 s, 89.1 MB/s 2023-11-06T03:13:56.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.486 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091968 s, 114 MB/s 2023-11-06T03:13:56.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.498 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961527 s, 109 MB/s 2023-11-06T03:13:56.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.513 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122788 s, 85.4 MB/s 2023-11-06T03:13:56.527 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.527 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.528 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114143 s, 91.9 MB/s 2023-11-06T03:13:56.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.541 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110837 s, 94.6 MB/s 2023-11-06T03:13:56.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.554 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107925 s, 97.2 MB/s 2023-11-06T03:13:56.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.569 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129994 s, 80.7 MB/s 2023-11-06T03:13:56.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.585 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.585 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131589 s, 79.7 MB/s 2023-11-06T03:13:56.599 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.600 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.600 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120466 s, 87.0 MB/s 2023-11-06T03:13:56.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.614 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119163 s, 88.0 MB/s 2023-11-06T03:13:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.626 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.627 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968297 s, 108 MB/s 2023-11-06T03:13:56.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.641 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121656 s, 86.2 MB/s 2023-11-06T03:13:56.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123839 s, 84.7 MB/s 2023-11-06T03:13:56.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.668 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086007 s, 122 MB/s 2023-11-06T03:13:56.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.684 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132464 s, 79.2 MB/s 2023-11-06T03:13:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.700 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135892 s, 77.2 MB/s 2023-11-06T03:13:56.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120412 s, 87.1 MB/s 2023-11-06T03:13:56.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010871 s, 96.5 MB/s 2023-11-06T03:13:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117129 s, 89.5 MB/s 2023-11-06T03:13:56.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129862 s, 80.7 MB/s 2023-11-06T03:13:56.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.775 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138953 s, 75.5 MB/s 2023-11-06T03:13:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.789 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116374 s, 90.1 MB/s 2023-11-06T03:13:56.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.804 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012109 s, 86.6 MB/s 2023-11-06T03:13:56.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904526 s, 116 MB/s 2023-11-06T03:13:56.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.832 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139736 s, 75.0 MB/s 2023-11-06T03:13:56.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:56.834 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:56.834 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:56.835 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.8 2023-11-06T03:13:56.835 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:56.835 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:56.836 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:56.836 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:56.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.903 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111646 s, 93.9 MB/s 2023-11-06T03:13:56.919 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.919 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.919 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137099 s, 76.5 MB/s 2023-11-06T03:13:56.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126599 s, 82.8 MB/s 2023-11-06T03:13:56.953 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.953 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.953 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137895 s, 76.0 MB/s 2023-11-06T03:13:56.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149526 s, 70.1 MB/s 2023-11-06T03:13:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:56.987 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122751 s, 85.4 MB/s 2023-11-06T03:13:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.001 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112479 s, 93.2 MB/s 2023-11-06T03:13:57.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.016 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122441 s, 85.6 MB/s 2023-11-06T03:13:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115871 s, 90.5 MB/s 2023-11-06T03:13:57.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915969 s, 114 MB/s 2023-11-06T03:13:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.058 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011736 s, 89.3 MB/s 2023-11-06T03:13:57.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.072 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117124 s, 89.5 MB/s 2023-11-06T03:13:57.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.084 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898154 s, 117 MB/s 2023-11-06T03:13:57.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117448 s, 89.3 MB/s 2023-11-06T03:13:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118623 s, 88.4 MB/s 2023-11-06T03:13:57.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.128 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114514 s, 91.6 MB/s 2023-11-06T03:13:57.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946571 s, 111 MB/s 2023-11-06T03:13:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121989 s, 86.0 MB/s 2023-11-06T03:13:57.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131061 s, 80.0 MB/s 2023-11-06T03:13:57.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.186 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114911 s, 91.3 MB/s 2023-11-06T03:13:57.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.198 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.198 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966488 s, 108 MB/s 2023-11-06T03:13:57.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.214 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013213 s, 79.4 MB/s 2023-11-06T03:13:57.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141143 s, 74.3 MB/s 2023-11-06T03:13:57.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.247 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125164 s, 83.8 MB/s 2023-11-06T03:13:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.263 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128508 s, 81.6 MB/s 2023-11-06T03:13:57.277 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.277 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.278 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117426 s, 89.3 MB/s 2023-11-06T03:13:57.292 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.292 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.292 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115007 s, 91.2 MB/s 2023-11-06T03:13:57.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.310 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125951 s, 83.3 MB/s 2023-11-06T03:13:57.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127533 s, 82.2 MB/s 2023-11-06T03:13:57.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.339 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119504 s, 87.7 MB/s 2023-11-06T03:13:57.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.355 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124035 s, 84.5 MB/s 2023-11-06T03:13:57.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133782 s, 78.4 MB/s 2023-11-06T03:13:57.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:57.372 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:57.373 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:57.373 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.9 2023-11-06T03:13:57.373 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:57.374 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:57.374 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:57.375 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:57.444 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.444 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.444 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01344 s, 78.0 MB/s 2023-11-06T03:13:57.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116926 s, 89.7 MB/s 2023-11-06T03:13:57.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139449 s, 75.2 MB/s 2023-11-06T03:13:57.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102692 s, 102 MB/s 2023-11-06T03:13:57.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.507 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135571 s, 77.3 MB/s 2023-11-06T03:13:57.521 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.522 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117441 s, 89.3 MB/s 2023-11-06T03:13:57.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104413 s, 100 MB/s 2023-11-06T03:13:57.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118472 s, 88.5 MB/s 2023-11-06T03:13:57.564 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.564 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.564 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108979 s, 96.2 MB/s 2023-11-06T03:13:57.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.582 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137343 s, 76.3 MB/s 2023-11-06T03:13:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.597 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128051 s, 81.9 MB/s 2023-11-06T03:13:57.613 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.613 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.613 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127343 s, 82.3 MB/s 2023-11-06T03:13:57.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.630 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135235 s, 77.5 MB/s 2023-11-06T03:13:57.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.645 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123372 s, 85.0 MB/s 2023-11-06T03:13:57.661 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.661 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.661 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128818 s, 81.4 MB/s 2023-11-06T03:13:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141221 s, 74.3 MB/s 2023-11-06T03:13:57.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.696 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014656 s, 71.5 MB/s 2023-11-06T03:13:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.713 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139747 s, 75.0 MB/s 2023-11-06T03:13:57.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129638 s, 80.9 MB/s 2023-11-06T03:13:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146682 s, 71.5 MB/s 2023-11-06T03:13:57.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.765 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140641 s, 74.6 MB/s 2023-11-06T03:13:57.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013972 s, 75.0 MB/s 2023-11-06T03:13:57.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.801 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136474 s, 76.8 MB/s 2023-11-06T03:13:57.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.818 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012491 s, 83.9 MB/s 2023-11-06T03:13:57.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.832 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113869 s, 92.1 MB/s 2023-11-06T03:13:57.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.849 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146229 s, 71.7 MB/s 2023-11-06T03:13:57.865 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.865 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125326 s, 83.7 MB/s 2023-11-06T03:13:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.883 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146043 s, 71.8 MB/s 2023-11-06T03:13:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.897 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111634 s, 93.9 MB/s 2023-11-06T03:13:57.912 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.913 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012615 s, 83.1 MB/s 2023-11-06T03:13:57.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.929 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130501 s, 80.4 MB/s 2023-11-06T03:13:57.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:57.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:57.944 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125795 s, 83.4 MB/s 2023-11-06T03:13:57.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:57.946 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:57.946 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:57.947 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.10 2023-11-06T03:13:57.947 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:57.948 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:57.948 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:57.948 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129552 s, 80.9 MB/s 2023-11-06T03:13:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101216 s, 104 MB/s 2023-11-06T03:13:58.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.042 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120109 s, 87.3 MB/s 2023-11-06T03:13:58.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.057 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.057 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123487 s, 84.9 MB/s 2023-11-06T03:13:58.071 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.072 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122835 s, 85.4 MB/s 2023-11-06T03:13:58.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.084 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958025 s, 109 MB/s 2023-11-06T03:13:58.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.100 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127503 s, 82.2 MB/s 2023-11-06T03:13:58.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.116 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137832 s, 76.1 MB/s 2023-11-06T03:13:58.132 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.132 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.133 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123871 s, 84.7 MB/s 2023-11-06T03:13:58.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.146 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108272 s, 96.8 MB/s 2023-11-06T03:13:58.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.159 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101568 s, 103 MB/s 2023-11-06T03:13:58.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130907 s, 80.1 MB/s 2023-11-06T03:13:58.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126203 s, 83.1 MB/s 2023-11-06T03:13:58.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.206 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126209 s, 83.1 MB/s 2023-11-06T03:13:58.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.223 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139854 s, 75.0 MB/s 2023-11-06T03:13:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123689 s, 84.8 MB/s 2023-11-06T03:13:58.251 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.252 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108164 s, 96.9 MB/s 2023-11-06T03:13:58.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156486 s, 67.0 MB/s 2023-11-06T03:13:58.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127607 s, 82.2 MB/s 2023-11-06T03:13:58.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.303 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131246 s, 79.9 MB/s 2023-11-06T03:13:58.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.317 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121259 s, 86.5 MB/s 2023-11-06T03:13:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131934 s, 79.5 MB/s 2023-11-06T03:13:58.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.350 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.350 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133058 s, 78.8 MB/s 2023-11-06T03:13:58.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013628 s, 76.9 MB/s 2023-11-06T03:13:58.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.383 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013121 s, 79.9 MB/s 2023-11-06T03:13:58.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.400 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131778 s, 79.6 MB/s 2023-11-06T03:13:58.417 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.417 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.417 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140721 s, 74.5 MB/s 2023-11-06T03:13:58.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012497 s, 83.9 MB/s 2023-11-06T03:13:58.448 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.449 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129257 s, 81.1 MB/s 2023-11-06T03:13:58.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.465 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130725 s, 80.2 MB/s 2023-11-06T03:13:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131302 s, 79.9 MB/s 2023-11-06T03:13:58.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.498 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137787 s, 76.1 MB/s 2023-11-06T03:13:58.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:58.500 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:58.500 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:58.501 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.11 2023-11-06T03:13:58.501 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:58.501 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:58.502 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:58.502 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.578 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156792 s, 66.9 MB/s 2023-11-06T03:13:58.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.596 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882135 s, 119 MB/s 2023-11-06T03:13:58.611 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.612 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.612 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132459 s, 79.2 MB/s 2023-11-06T03:13:58.626 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.627 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122678 s, 85.5 MB/s 2023-11-06T03:13:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132991 s, 78.8 MB/s 2023-11-06T03:13:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119774 s, 87.5 MB/s 2023-11-06T03:13:58.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.674 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014618 s, 71.7 MB/s 2023-11-06T03:13:58.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.688 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.688 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112133 s, 93.5 MB/s 2023-11-06T03:13:58.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.704 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123747 s, 84.7 MB/s 2023-11-06T03:13:58.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132502 s, 79.1 MB/s 2023-11-06T03:13:58.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.732 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100613 s, 104 MB/s 2023-11-06T03:13:58.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122096 s, 85.9 MB/s 2023-11-06T03:13:58.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.760 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934125 s, 112 MB/s 2023-11-06T03:13:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119903 s, 87.5 MB/s 2023-11-06T03:13:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115479 s, 90.8 MB/s 2023-11-06T03:13:58.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.808 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123694 s, 84.8 MB/s 2023-11-06T03:13:58.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121122 s, 86.6 MB/s 2023-11-06T03:13:58.834 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.834 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.834 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122145 s, 85.8 MB/s 2023-11-06T03:13:58.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.848 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.848 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110826 s, 94.6 MB/s 2023-11-06T03:13:58.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137994 s, 76.0 MB/s 2023-11-06T03:13:58.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114197 s, 91.8 MB/s 2023-11-06T03:13:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.893 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120986 s, 86.7 MB/s 2023-11-06T03:13:58.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133528 s, 78.5 MB/s 2023-11-06T03:13:58.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.926 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124138 s, 84.5 MB/s 2023-11-06T03:13:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127111 s, 82.5 MB/s 2023-11-06T03:13:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132266 s, 79.3 MB/s 2023-11-06T03:13:58.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.972 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011997 s, 87.4 MB/s 2023-11-06T03:13:58.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:58.987 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:58.987 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123756 s, 84.7 MB/s 2023-11-06T03:13:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.003 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123524 s, 84.9 MB/s 2023-11-06T03:13:59.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.016 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102267 s, 103 MB/s 2023-11-06T03:13:59.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.031 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012408 s, 84.5 MB/s 2023-11-06T03:13:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.046 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121586 s, 86.2 MB/s 2023-11-06T03:13:59.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:59.048 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:59.048 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:59.048 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.12 2023-11-06T03:13:59.049 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:59.049 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:59.049 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:59.050 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:59.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.117 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118225 s, 88.7 MB/s 2023-11-06T03:13:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137269 s, 76.4 MB/s 2023-11-06T03:13:59.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.151 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133726 s, 78.4 MB/s 2023-11-06T03:13:59.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.166 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120134 s, 87.3 MB/s 2023-11-06T03:13:59.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145221 s, 72.2 MB/s 2023-11-06T03:13:59.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.202 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014101 s, 74.4 MB/s 2023-11-06T03:13:59.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139439 s, 75.2 MB/s 2023-11-06T03:13:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111742 s, 93.8 MB/s 2023-11-06T03:13:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010678 s, 98.2 MB/s 2023-11-06T03:13:59.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.265 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133869 s, 78.3 MB/s 2023-11-06T03:13:59.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139362 s, 75.2 MB/s 2023-11-06T03:13:59.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.298 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133308 s, 78.7 MB/s 2023-11-06T03:13:59.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.315 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135958 s, 77.1 MB/s 2023-11-06T03:13:59.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138868 s, 75.5 MB/s 2023-11-06T03:13:59.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126444 s, 82.9 MB/s 2023-11-06T03:13:59.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127338 s, 82.3 MB/s 2023-11-06T03:13:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107613 s, 97.4 MB/s 2023-11-06T03:13:59.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.396 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151991 s, 69.0 MB/s 2023-11-06T03:13:59.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.411 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121167 s, 86.5 MB/s 2023-11-06T03:13:59.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.427 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129135 s, 81.2 MB/s 2023-11-06T03:13:59.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.443 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130746 s, 80.2 MB/s 2023-11-06T03:13:59.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.456 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104998 s, 99.9 MB/s 2023-11-06T03:13:59.470 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123039 s, 85.2 MB/s 2023-11-06T03:13:59.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.488 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133995 s, 78.3 MB/s 2023-11-06T03:13:59.505 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.506 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142959 s, 73.3 MB/s 2023-11-06T03:13:59.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.521 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118654 s, 88.4 MB/s 2023-11-06T03:13:59.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118272 s, 88.7 MB/s 2023-11-06T03:13:59.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.553 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141011 s, 74.4 MB/s 2023-11-06T03:13:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.566 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100751 s, 104 MB/s 2023-11-06T03:13:59.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136835 s, 76.6 MB/s 2023-11-06T03:13:59.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135154 s, 77.6 MB/s 2023-11-06T03:13:59.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.653 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126802 s, 82.7 MB/s 2023-11-06T03:13:59.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:13:59.654 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:13:59.655 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:13:59.655 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.13 2023-11-06T03:13:59.655 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:13:59.656 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:13:59.656 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:13:59.656 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:13:59.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143392 s, 73.1 MB/s 2023-11-06T03:13:59.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106379 s, 98.6 MB/s 2023-11-06T03:13:59.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014137 s, 74.2 MB/s 2023-11-06T03:13:59.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.776 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140635 s, 74.6 MB/s 2023-11-06T03:13:59.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.793 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138007 s, 76.0 MB/s 2023-11-06T03:13:59.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.808 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117278 s, 89.4 MB/s 2023-11-06T03:13:59.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156898 s, 66.8 MB/s 2023-11-06T03:13:59.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134136 s, 78.2 MB/s 2023-11-06T03:13:59.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125927 s, 83.3 MB/s 2023-11-06T03:13:59.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127715 s, 82.1 MB/s 2023-11-06T03:13:59.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.889 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111587 s, 94.0 MB/s 2023-11-06T03:13:59.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.907 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142005 s, 73.8 MB/s 2023-11-06T03:13:59.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.924 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144267 s, 72.7 MB/s 2023-11-06T03:13:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127516 s, 82.2 MB/s 2023-11-06T03:13:59.956 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138522 s, 75.7 MB/s 2023-11-06T03:13:59.972 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.973 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129784 s, 80.8 MB/s 2023-11-06T03:13:59.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:13:59.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:13:59.988 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127181 s, 82.4 MB/s 2023-11-06T03:14:00.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137336 s, 76.4 MB/s 2023-11-06T03:14:00.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.024 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146725 s, 71.5 MB/s 2023-11-06T03:14:00.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114159 s, 91.9 MB/s 2023-11-06T03:14:00.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124496 s, 84.2 MB/s 2023-11-06T03:14:00.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132073 s, 79.4 MB/s 2023-11-06T03:14:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.085 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118336 s, 88.6 MB/s 2023-11-06T03:14:00.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.101 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132401 s, 79.2 MB/s 2023-11-06T03:14:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126647 s, 82.8 MB/s 2023-11-06T03:14:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122154 s, 85.8 MB/s 2023-11-06T03:14:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.144 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102338 s, 102 MB/s 2023-11-06T03:14:00.160 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.160 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.160 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130692 s, 80.2 MB/s 2023-11-06T03:14:00.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129497 s, 81.0 MB/s 2023-11-06T03:14:00.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.194 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140363 s, 74.7 MB/s 2023-11-06T03:14:00.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.211 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.211 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135187 s, 77.6 MB/s 2023-11-06T03:14:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147763 s, 71.0 MB/s 2023-11-06T03:14:00.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:00.231 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:00.231 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:00.231 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.14 2023-11-06T03:14:00.232 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:00.232 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:00.232 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:00.233 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:00.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.302 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150106 s, 69.9 MB/s 2023-11-06T03:14:00.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.318 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115006 s, 91.2 MB/s 2023-11-06T03:14:00.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125185 s, 83.8 MB/s 2023-11-06T03:14:00.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124697 s, 84.1 MB/s 2023-11-06T03:14:00.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131953 s, 79.5 MB/s 2023-11-06T03:14:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.380 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.381 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133233 s, 78.7 MB/s 2023-11-06T03:14:00.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.391 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874514 s, 120 MB/s 2023-11-06T03:14:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123342 s, 85.0 MB/s 2023-11-06T03:14:00.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.420 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107415 s, 97.6 MB/s 2023-11-06T03:14:00.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119427 s, 87.8 MB/s 2023-11-06T03:14:00.447 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.447 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.448 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111272 s, 94.2 MB/s 2023-11-06T03:14:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139032 s, 75.4 MB/s 2023-11-06T03:14:00.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112248 s, 93.4 MB/s 2023-11-06T03:14:00.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.495 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140724 s, 74.5 MB/s 2023-11-06T03:14:00.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.508 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.508 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010915 s, 96.1 MB/s 2023-11-06T03:14:00.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.523 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121411 s, 86.4 MB/s 2023-11-06T03:14:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.536 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110784 s, 94.7 MB/s 2023-11-06T03:14:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010339 s, 101 MB/s 2023-11-06T03:14:00.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109307 s, 95.9 MB/s 2023-11-06T03:14:00.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184904 s, 56.7 MB/s 2023-11-06T03:14:00.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.605 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.605 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118827 s, 88.2 MB/s 2023-11-06T03:14:00.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120833 s, 86.8 MB/s 2023-11-06T03:14:00.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138261 s, 75.8 MB/s 2023-11-06T03:14:00.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124593 s, 84.2 MB/s 2023-11-06T03:14:00.666 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.667 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.667 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111944 s, 93.7 MB/s 2023-11-06T03:14:00.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106688 s, 98.3 MB/s 2023-11-06T03:14:00.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.696 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.696 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127306 s, 82.4 MB/s 2023-11-06T03:14:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151308 s, 69.3 MB/s 2023-11-06T03:14:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122779 s, 85.4 MB/s 2023-11-06T03:14:00.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112451 s, 93.2 MB/s 2023-11-06T03:14:00.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.761 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.761 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140754 s, 74.5 MB/s 2023-11-06T03:14:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116068 s, 90.3 MB/s 2023-11-06T03:14:00.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:00.778 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:00.778 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:00.779 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.15 2023-11-06T03:14:00.779 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:00.779 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:00.780 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:00.780 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:00.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.845 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124944 s, 83.9 MB/s 2023-11-06T03:14:00.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.861 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129158 s, 81.2 MB/s 2023-11-06T03:14:00.876 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.877 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121675 s, 86.2 MB/s 2023-11-06T03:14:00.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.893 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013106 s, 80.0 MB/s 2023-11-06T03:14:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133444 s, 78.6 MB/s 2023-11-06T03:14:00.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.926 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133386 s, 78.6 MB/s 2023-11-06T03:14:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.942 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.943 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013927 s, 75.3 MB/s 2023-11-06T03:14:00.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.959 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131292 s, 79.9 MB/s 2023-11-06T03:14:00.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.976 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140307 s, 74.7 MB/s 2023-11-06T03:14:00.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:00.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:00.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140722 s, 74.5 MB/s 2023-11-06T03:14:01.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125284 s, 83.7 MB/s 2023-11-06T03:14:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119479 s, 87.8 MB/s 2023-11-06T03:14:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.034 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921153 s, 114 MB/s 2023-11-06T03:14:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.050 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137513 s, 76.3 MB/s 2023-11-06T03:14:01.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.066 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129285 s, 81.1 MB/s 2023-11-06T03:14:01.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.080 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113479 s, 92.4 MB/s 2023-11-06T03:14:01.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.093 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104269 s, 101 MB/s 2023-11-06T03:14:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996189 s, 105 MB/s 2023-11-06T03:14:01.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.118 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107438 s, 97.6 MB/s 2023-11-06T03:14:01.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103388 s, 101 MB/s 2023-11-06T03:14:01.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.145 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011298 s, 92.8 MB/s 2023-11-06T03:14:01.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108547 s, 96.6 MB/s 2023-11-06T03:14:01.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106168 s, 98.8 MB/s 2023-11-06T03:14:01.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.182 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841335 s, 125 MB/s 2023-11-06T03:14:01.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.194 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.194 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104562 s, 100 MB/s 2023-11-06T03:14:01.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.209 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115065 s, 91.1 MB/s 2023-11-06T03:14:01.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010645 s, 98.5 MB/s 2023-11-06T03:14:01.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116243 s, 90.2 MB/s 2023-11-06T03:14:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.249 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.250 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119851 s, 87.5 MB/s 2023-11-06T03:14:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.260 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811996 s, 129 MB/s 2023-11-06T03:14:01.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.271 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084092 s, 125 MB/s 2023-11-06T03:14:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.284 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109068 s, 96.1 MB/s 2023-11-06T03:14:01.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:01.286 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:01.286 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:01.287 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.16 2023-11-06T03:14:01.287 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:01.287 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:01.288 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:01.288 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:01.355 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.355 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.355 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133813 s, 78.4 MB/s 2023-11-06T03:14:01.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130178 s, 80.5 MB/s 2023-11-06T03:14:01.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105531 s, 99.4 MB/s 2023-11-06T03:14:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126563 s, 82.9 MB/s 2023-11-06T03:14:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.418 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014904 s, 70.4 MB/s 2023-11-06T03:14:01.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123669 s, 84.8 MB/s 2023-11-06T03:14:01.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.451 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134709 s, 77.8 MB/s 2023-11-06T03:14:01.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.467 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122431 s, 85.6 MB/s 2023-11-06T03:14:01.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.482 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125372 s, 83.6 MB/s 2023-11-06T03:14:01.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.495 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945622 s, 111 MB/s 2023-11-06T03:14:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121413 s, 86.4 MB/s 2023-11-06T03:14:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.526 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134035 s, 78.2 MB/s 2023-11-06T03:14:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128765 s, 81.4 MB/s 2023-11-06T03:14:01.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.558 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125272 s, 83.7 MB/s 2023-11-06T03:14:01.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.574 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132309 s, 79.3 MB/s 2023-11-06T03:14:01.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.601 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156488 s, 67.0 MB/s 2023-11-06T03:14:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125035 s, 83.9 MB/s 2023-11-06T03:14:01.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.636 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131538 s, 79.7 MB/s 2023-11-06T03:14:01.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.649 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989271 s, 106 MB/s 2023-11-06T03:14:01.663 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.664 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126708 s, 82.8 MB/s 2023-11-06T03:14:01.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.681 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141867 s, 73.9 MB/s 2023-11-06T03:14:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140965 s, 74.4 MB/s 2023-11-06T03:14:01.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.712 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106728 s, 98.2 MB/s 2023-11-06T03:14:01.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126883 s, 82.6 MB/s 2023-11-06T03:14:01.745 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.745 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.745 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012481 s, 84.0 MB/s 2023-11-06T03:14:01.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134137 s, 78.2 MB/s 2023-11-06T03:14:01.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.778 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120636 s, 86.9 MB/s 2023-11-06T03:14:01.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.793 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110432 s, 95.0 MB/s 2023-11-06T03:14:01.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.810 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133981 s, 78.3 MB/s 2023-11-06T03:14:01.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118967 s, 88.1 MB/s 2023-11-06T03:14:01.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.844 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135171 s, 77.6 MB/s 2023-11-06T03:14:01.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.861 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.861 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013611 s, 77.0 MB/s 2023-11-06T03:14:01.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:01.863 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:01.863 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:01.864 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.17 2023-11-06T03:14:01.864 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:01.864 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:01.865 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:01.865 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:01.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.930 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122289 s, 85.7 MB/s 2023-11-06T03:14:01.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126675 s, 82.8 MB/s 2023-11-06T03:14:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.961 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111602 s, 94.0 MB/s 2023-11-06T03:14:01.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127635 s, 82.2 MB/s 2023-11-06T03:14:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:01.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117366 s, 89.3 MB/s 2023-11-06T03:14:02.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123309 s, 85.0 MB/s 2023-11-06T03:14:02.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.021 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104444 s, 100 MB/s 2023-11-06T03:14:02.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139038 s, 75.4 MB/s 2023-11-06T03:14:02.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.053 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124513 s, 84.2 MB/s 2023-11-06T03:14:02.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.069 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127154 s, 82.5 MB/s 2023-11-06T03:14:02.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.083 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109021 s, 96.2 MB/s 2023-11-06T03:14:02.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.100 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143584 s, 73.0 MB/s 2023-11-06T03:14:02.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124637 s, 84.1 MB/s 2023-11-06T03:14:02.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126218 s, 83.1 MB/s 2023-11-06T03:14:02.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.145 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117427 s, 89.3 MB/s 2023-11-06T03:14:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142675 s, 73.5 MB/s 2023-11-06T03:14:02.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.178 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129237 s, 81.1 MB/s 2023-11-06T03:14:02.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.191 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946133 s, 111 MB/s 2023-11-06T03:14:02.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988015 s, 106 MB/s 2023-11-06T03:14:02.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138553 s, 75.7 MB/s 2023-11-06T03:14:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.234 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112237 s, 93.4 MB/s 2023-11-06T03:14:02.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.247 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105063 s, 99.8 MB/s 2023-11-06T03:14:02.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.262 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128413 s, 81.7 MB/s 2023-11-06T03:14:02.277 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.277 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.278 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124859 s, 84.0 MB/s 2023-11-06T03:14:02.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.293 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.293 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011967 s, 87.6 MB/s 2023-11-06T03:14:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133904 s, 78.3 MB/s 2023-11-06T03:14:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125561 s, 83.5 MB/s 2023-11-06T03:14:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112294 s, 93.4 MB/s 2023-11-06T03:14:02.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121063 s, 86.6 MB/s 2023-11-06T03:14:02.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128833 s, 81.4 MB/s 2023-11-06T03:14:02.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.383 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126642 s, 82.8 MB/s 2023-11-06T03:14:02.399 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.400 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135763 s, 77.2 MB/s 2023-11-06T03:14:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:02.401 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:02.402 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:02.402 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.18 2023-11-06T03:14:02.402 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:02.403 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:02.403 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:02.404 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:02.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.473 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135792 s, 77.2 MB/s 2023-11-06T03:14:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115989 s, 90.4 MB/s 2023-11-06T03:14:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126996 s, 82.6 MB/s 2023-11-06T03:14:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108842 s, 96.3 MB/s 2023-11-06T03:14:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129555 s, 80.9 MB/s 2023-11-06T03:14:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118237 s, 88.7 MB/s 2023-11-06T03:14:02.564 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.565 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.565 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129849 s, 80.8 MB/s 2023-11-06T03:14:02.579 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.580 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.580 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122288 s, 85.7 MB/s 2023-11-06T03:14:02.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.596 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013517 s, 77.6 MB/s 2023-11-06T03:14:02.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.633 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129086 s, 81.2 MB/s 2023-11-06T03:14:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.647 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116569 s, 90.0 MB/s 2023-11-06T03:14:02.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971624 s, 108 MB/s 2023-11-06T03:14:02.676 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.676 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.677 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145094 s, 72.3 MB/s 2023-11-06T03:14:02.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.692 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012216 s, 85.8 MB/s 2023-11-06T03:14:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971039 s, 108 MB/s 2023-11-06T03:14:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123087 s, 85.2 MB/s 2023-11-06T03:14:02.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121194 s, 86.5 MB/s 2023-11-06T03:14:02.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112998 s, 92.8 MB/s 2023-11-06T03:14:02.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.771 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101297 s, 104 MB/s 2023-11-06T03:14:02.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133541 s, 78.5 MB/s 2023-11-06T03:14:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.792 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124626 s, 84.1 MB/s 2023-11-06T03:14:02.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.808 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129797 s, 80.8 MB/s 2023-11-06T03:14:02.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.822 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111477 s, 94.1 MB/s 2023-11-06T03:14:02.834 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.834 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.834 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970045 s, 108 MB/s 2023-11-06T03:14:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132232 s, 79.3 MB/s 2023-11-06T03:14:02.864 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.865 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.865 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117754 s, 89.0 MB/s 2023-11-06T03:14:02.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.880 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122881 s, 85.3 MB/s 2023-11-06T03:14:02.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117867 s, 89.0 MB/s 2023-11-06T03:14:02.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.908 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114773 s, 91.4 MB/s 2023-11-06T03:14:02.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112843 s, 92.9 MB/s 2023-11-06T03:14:02.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012217 s, 85.8 MB/s 2023-11-06T03:14:02.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:02.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:02.948 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893859 s, 117 MB/s 2023-11-06T03:14:02.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:02.950 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:02.950 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:02.951 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.19 2023-11-06T03:14:02.951 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:02.951 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:02.952 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:02.952 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:03.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.015 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.015 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111248 s, 94.3 MB/s 2023-11-06T03:14:03.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.029 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116062 s, 90.3 MB/s 2023-11-06T03:14:03.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.045 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.045 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135073 s, 77.6 MB/s 2023-11-06T03:14:03.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119259 s, 87.9 MB/s 2023-11-06T03:14:03.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867318 s, 121 MB/s 2023-11-06T03:14:03.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.082 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.082 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990162 s, 106 MB/s 2023-11-06T03:14:03.094 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.095 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104799 s, 100 MB/s 2023-11-06T03:14:03.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.108 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106206 s, 98.7 MB/s 2023-11-06T03:14:03.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.118 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766134 s, 137 MB/s 2023-11-06T03:14:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.130 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010209 s, 103 MB/s 2023-11-06T03:14:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.144 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012088 s, 86.7 MB/s 2023-11-06T03:14:03.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.157 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103401 s, 101 MB/s 2023-11-06T03:14:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.169 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104482 s, 100 MB/s 2023-11-06T03:14:03.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010075 s, 104 MB/s 2023-11-06T03:14:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117421 s, 89.3 MB/s 2023-11-06T03:14:03.209 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.209 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.209 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111867 s, 93.7 MB/s 2023-11-06T03:14:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.223 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118475 s, 88.5 MB/s 2023-11-06T03:14:03.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103046 s, 102 MB/s 2023-11-06T03:14:03.245 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.245 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775264 s, 135 MB/s 2023-11-06T03:14:03.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.259 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115308 s, 90.9 MB/s 2023-11-06T03:14:03.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805923 s, 130 MB/s 2023-11-06T03:14:03.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.283 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102569 s, 102 MB/s 2023-11-06T03:14:03.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.296 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.296 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108932 s, 96.3 MB/s 2023-11-06T03:14:03.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.309 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010399 s, 101 MB/s 2023-11-06T03:14:03.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.322 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.322 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110799 s, 94.6 MB/s 2023-11-06T03:14:03.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796245 s, 132 MB/s 2023-11-06T03:14:03.343 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.344 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980262 s, 107 MB/s 2023-11-06T03:14:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.358 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118989 s, 88.1 MB/s 2023-11-06T03:14:03.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111975 s, 93.6 MB/s 2023-11-06T03:14:03.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109522 s, 95.7 MB/s 2023-11-06T03:14:03.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.398 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110461 s, 94.9 MB/s 2023-11-06T03:14:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119441 s, 87.8 MB/s 2023-11-06T03:14:03.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:03.413 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:03.414 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:03.414 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.20 2023-11-06T03:14:03.415 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:03.415 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:03.415 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:03.416 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:03.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.482 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.482 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117607 s, 89.2 MB/s 2023-11-06T03:14:03.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115446 s, 90.8 MB/s 2023-11-06T03:14:03.508 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.508 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.508 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101093 s, 104 MB/s 2023-11-06T03:14:03.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.523 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125738 s, 83.4 MB/s 2023-11-06T03:14:03.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991013 s, 106 MB/s 2023-11-06T03:14:03.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.548 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011175 s, 93.8 MB/s 2023-11-06T03:14:03.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.562 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119946 s, 87.4 MB/s 2023-11-06T03:14:03.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.576 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109776 s, 95.5 MB/s 2023-11-06T03:14:03.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100058 s, 105 MB/s 2023-11-06T03:14:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106034 s, 98.9 MB/s 2023-11-06T03:14:03.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.630 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834681 s, 126 MB/s 2023-11-06T03:14:03.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.640 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.640 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776485 s, 135 MB/s 2023-11-06T03:14:03.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.654 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123223 s, 85.1 MB/s 2023-11-06T03:14:03.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.670 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110405 s, 95.0 MB/s 2023-11-06T03:14:03.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974656 s, 108 MB/s 2023-11-06T03:14:03.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.692 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111533 s, 94.0 MB/s 2023-11-06T03:14:03.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.706 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.706 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118919 s, 88.2 MB/s 2023-11-06T03:14:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.720 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116564 s, 90.0 MB/s 2023-11-06T03:14:03.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.730 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754784 s, 139 MB/s 2023-11-06T03:14:03.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105564 s, 99.3 MB/s 2023-11-06T03:14:03.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.755 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101459 s, 103 MB/s 2023-11-06T03:14:03.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121338 s, 86.4 MB/s 2023-11-06T03:14:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101922 s, 103 MB/s 2023-11-06T03:14:03.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111361 s, 94.2 MB/s 2023-11-06T03:14:03.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.807 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104468 s, 100 MB/s 2023-11-06T03:14:03.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971397 s, 108 MB/s 2023-11-06T03:14:03.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.829 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771223 s, 136 MB/s 2023-11-06T03:14:03.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982215 s, 107 MB/s 2023-11-06T03:14:03.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100642 s, 104 MB/s 2023-11-06T03:14:03.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.866 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108881 s, 96.3 MB/s 2023-11-06T03:14:03.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.878 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.878 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986586 s, 106 MB/s 2023-11-06T03:14:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011366 s, 92.3 MB/s 2023-11-06T03:14:03.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:03.893 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:03.894 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:03.895 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.21 2023-11-06T03:14:03.895 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:03.896 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:03.896 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:03.896 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:03.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.958 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121188 s, 86.5 MB/s 2023-11-06T03:14:03.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.971 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103118 s, 102 MB/s 2023-11-06T03:14:03.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.984 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114215 s, 91.8 MB/s 2023-11-06T03:14:03.994 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:03.994 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:03.994 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750912 s, 140 MB/s 2023-11-06T03:14:04.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.007 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110162 s, 95.2 MB/s 2023-11-06T03:14:04.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.020 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110503 s, 94.9 MB/s 2023-11-06T03:14:04.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969774 s, 108 MB/s 2023-11-06T03:14:04.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125923 s, 83.3 MB/s 2023-11-06T03:14:04.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972901 s, 108 MB/s 2023-11-06T03:14:04.071 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.072 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107894 s, 97.2 MB/s 2023-11-06T03:14:04.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.086 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115738 s, 90.6 MB/s 2023-11-06T03:14:04.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.098 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105095 s, 99.8 MB/s 2023-11-06T03:14:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988735 s, 106 MB/s 2023-11-06T03:14:04.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.125 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126077 s, 83.2 MB/s 2023-11-06T03:14:04.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.138 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107474 s, 97.6 MB/s 2023-11-06T03:14:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.151 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011151 s, 94.0 MB/s 2023-11-06T03:14:04.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972894 s, 108 MB/s 2023-11-06T03:14:04.173 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.174 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878321 s, 119 MB/s 2023-11-06T03:14:04.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.186 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993543 s, 106 MB/s 2023-11-06T03:14:04.200 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126454 s, 82.9 MB/s 2023-11-06T03:14:04.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.215 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114674 s, 91.4 MB/s 2023-11-06T03:14:04.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.229 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118713 s, 88.3 MB/s 2023-11-06T03:14:04.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.244 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136908 s, 76.6 MB/s 2023-11-06T03:14:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.257 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106815 s, 98.2 MB/s 2023-11-06T03:14:04.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.273 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013102 s, 80.0 MB/s 2023-11-06T03:14:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103998 s, 101 MB/s 2023-11-06T03:14:04.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.297 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101234 s, 104 MB/s 2023-11-06T03:14:04.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.310 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.310 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106 s, 98.9 MB/s 2023-11-06T03:14:04.322 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.322 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.322 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100353 s, 104 MB/s 2023-11-06T03:14:04.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.336 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109486 s, 95.8 MB/s 2023-11-06T03:14:04.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100313 s, 105 MB/s 2023-11-06T03:14:04.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.361 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010852 s, 96.6 MB/s 2023-11-06T03:14:04.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:04.363 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:04.363 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:04.363 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.22 2023-11-06T03:14:04.364 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:04.364 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:04.364 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:04.365 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:04.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105884 s, 99.0 MB/s 2023-11-06T03:14:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103658 s, 101 MB/s 2023-11-06T03:14:04.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755137 s, 139 MB/s 2023-11-06T03:14:04.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127927 s, 82.0 MB/s 2023-11-06T03:14:04.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.471 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782265 s, 134 MB/s 2023-11-06T03:14:04.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.484 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.485 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114976 s, 91.2 MB/s 2023-11-06T03:14:04.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.497 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100312 s, 105 MB/s 2023-11-06T03:14:04.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110959 s, 94.5 MB/s 2023-11-06T03:14:04.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.522 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980811 s, 107 MB/s 2023-11-06T03:14:04.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01261 s, 83.2 MB/s 2023-11-06T03:14:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118434 s, 88.5 MB/s 2023-11-06T03:14:04.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.564 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107775 s, 97.3 MB/s 2023-11-06T03:14:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102493 s, 102 MB/s 2023-11-06T03:14:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930806 s, 113 MB/s 2023-11-06T03:14:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.601 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112294 s, 93.4 MB/s 2023-11-06T03:14:04.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.628 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120897 s, 86.7 MB/s 2023-11-06T03:14:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982375 s, 107 MB/s 2023-11-06T03:14:04.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101502 s, 103 MB/s 2023-11-06T03:14:04.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.665 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111002 s, 94.5 MB/s 2023-11-06T03:14:04.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.677 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.677 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976285 s, 107 MB/s 2023-11-06T03:14:04.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.692 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012576 s, 83.4 MB/s 2023-11-06T03:14:04.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900571 s, 116 MB/s 2023-11-06T03:14:04.717 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124911 s, 83.9 MB/s 2023-11-06T03:14:04.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.733 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129157 s, 81.2 MB/s 2023-11-06T03:14:04.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827873 s, 127 MB/s 2023-11-06T03:14:04.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011474 s, 91.4 MB/s 2023-11-06T03:14:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109303 s, 95.9 MB/s 2023-11-06T03:14:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119726 s, 87.6 MB/s 2023-11-06T03:14:04.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.800 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.800 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135725 s, 77.3 MB/s 2023-11-06T03:14:04.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012698 s, 82.6 MB/s 2023-11-06T03:14:04.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.829 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.830 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121075 s, 86.6 MB/s 2023-11-06T03:14:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124211 s, 84.4 MB/s 2023-11-06T03:14:04.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:04.847 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:04.847 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:04.848 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.23 2023-11-06T03:14:04.848 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:04.848 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:04.848 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:04.849 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:04.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988257 s, 106 MB/s 2023-11-06T03:14:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108019 s, 97.1 MB/s 2023-11-06T03:14:04.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010839 s, 96.7 MB/s 2023-11-06T03:14:04.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.950 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120142 s, 87.3 MB/s 2023-11-06T03:14:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.965 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120305 s, 87.2 MB/s 2023-11-06T03:14:04.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010491 s, 99.9 MB/s 2023-11-06T03:14:04.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:04.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:04.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107329 s, 97.7 MB/s 2023-11-06T03:14:05.003 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.003 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.003 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104052 s, 101 MB/s 2023-11-06T03:14:05.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.016 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106429 s, 98.5 MB/s 2023-11-06T03:14:05.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.030 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.030 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114196 s, 91.8 MB/s 2023-11-06T03:14:05.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111821 s, 93.8 MB/s 2023-11-06T03:14:05.055 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.055 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957661 s, 109 MB/s 2023-11-06T03:14:05.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119934 s, 87.4 MB/s 2023-11-06T03:14:05.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.085 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116863 s, 89.7 MB/s 2023-11-06T03:14:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.101 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115641 s, 90.7 MB/s 2023-11-06T03:14:05.112 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.113 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.113 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115728 s, 90.6 MB/s 2023-11-06T03:14:05.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.127 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118599 s, 88.4 MB/s 2023-11-06T03:14:05.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108758 s, 96.4 MB/s 2023-11-06T03:14:05.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112802 s, 93.0 MB/s 2023-11-06T03:14:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115143 s, 91.1 MB/s 2023-11-06T03:14:05.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.179 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855374 s, 123 MB/s 2023-11-06T03:14:05.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915488 s, 115 MB/s 2023-11-06T03:14:05.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.202 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918939 s, 114 MB/s 2023-11-06T03:14:05.215 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.215 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.216 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113106 s, 92.7 MB/s 2023-11-06T03:14:05.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.231 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127599 s, 82.2 MB/s 2023-11-06T03:14:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112162 s, 93.5 MB/s 2023-11-06T03:14:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.260 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120542 s, 87.0 MB/s 2023-11-06T03:14:05.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.274 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.274 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125815 s, 83.3 MB/s 2023-11-06T03:14:05.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.290 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.291 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137142 s, 76.5 MB/s 2023-11-06T03:14:05.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.305 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.306 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121378 s, 86.4 MB/s 2023-11-06T03:14:05.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.320 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123437 s, 84.9 MB/s 2023-11-06T03:14:05.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.334 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113938 s, 92.0 MB/s 2023-11-06T03:14:05.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:05.336 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:05.336 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:05.337 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.24 2023-11-06T03:14:05.337 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:05.337 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:05.338 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:05.338 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:05.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.404 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117315 s, 89.4 MB/s 2023-11-06T03:14:05.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123233 s, 85.1 MB/s 2023-11-06T03:14:05.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114383 s, 91.7 MB/s 2023-11-06T03:14:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133861 s, 78.3 MB/s 2023-11-06T03:14:05.460 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831941 s, 126 MB/s 2023-11-06T03:14:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.475 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121205 s, 86.5 MB/s 2023-11-06T03:14:05.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124145 s, 84.5 MB/s 2023-11-06T03:14:05.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867989 s, 121 MB/s 2023-11-06T03:14:05.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.518 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145589 s, 72.0 MB/s 2023-11-06T03:14:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.532 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111294 s, 94.2 MB/s 2023-11-06T03:14:05.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.544 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104505 s, 100 MB/s 2023-11-06T03:14:05.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.558 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112758 s, 93.0 MB/s 2023-11-06T03:14:05.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124725 s, 84.1 MB/s 2023-11-06T03:14:05.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.589 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131425 s, 79.8 MB/s 2023-11-06T03:14:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116237 s, 90.2 MB/s 2023-11-06T03:14:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.647 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142082 s, 73.8 MB/s 2023-11-06T03:14:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.662 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115793 s, 90.6 MB/s 2023-11-06T03:14:05.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.679 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131747 s, 79.6 MB/s 2023-11-06T03:14:05.696 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.697 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141448 s, 74.1 MB/s 2023-11-06T03:14:05.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137987 s, 76.0 MB/s 2023-11-06T03:14:05.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155454 s, 67.5 MB/s 2023-11-06T03:14:05.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.752 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.752 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140008 s, 74.9 MB/s 2023-11-06T03:14:05.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143082 s, 73.3 MB/s 2023-11-06T03:14:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127409 s, 82.3 MB/s 2023-11-06T03:14:05.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.803 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140447 s, 74.7 MB/s 2023-11-06T03:14:05.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.820 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013123 s, 79.9 MB/s 2023-11-06T03:14:05.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.833 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106055 s, 98.9 MB/s 2023-11-06T03:14:05.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.850 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.850 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135411 s, 77.4 MB/s 2023-11-06T03:14:05.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.866 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137066 s, 76.5 MB/s 2023-11-06T03:14:05.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.883 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135421 s, 77.4 MB/s 2023-11-06T03:14:05.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.900 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144786 s, 72.4 MB/s 2023-11-06T03:14:05.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.918 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136879 s, 76.6 MB/s 2023-11-06T03:14:05.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:05.920 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:05.921 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:05.922 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.25 2023-11-06T03:14:05.922 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:05.923 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:05.923 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:05.923 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:05.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:05.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:05.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132366 s, 79.2 MB/s 2023-11-06T03:14:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.007 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122102 s, 85.9 MB/s 2023-11-06T03:14:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127096 s, 82.5 MB/s 2023-11-06T03:14:06.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.036 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117918 s, 88.9 MB/s 2023-11-06T03:14:06.051 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.052 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01285 s, 81.6 MB/s 2023-11-06T03:14:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.065 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010453 s, 100 MB/s 2023-11-06T03:14:06.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136354 s, 76.9 MB/s 2023-11-06T03:14:06.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.096 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.096 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111491 s, 94.1 MB/s 2023-11-06T03:14:06.111 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.112 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.112 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136596 s, 76.8 MB/s 2023-11-06T03:14:06.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.126 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103619 s, 101 MB/s 2023-11-06T03:14:06.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101118 s, 104 MB/s 2023-11-06T03:14:06.149 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.150 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010765 s, 97.4 MB/s 2023-11-06T03:14:06.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107232 s, 97.8 MB/s 2023-11-06T03:14:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109953 s, 95.4 MB/s 2023-11-06T03:14:06.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847369 s, 124 MB/s 2023-11-06T03:14:06.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.200 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.200 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107043 s, 98.0 MB/s 2023-11-06T03:14:06.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.213 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01142 s, 91.8 MB/s 2023-11-06T03:14:06.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.228 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.228 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121833 s, 86.1 MB/s 2023-11-06T03:14:06.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.243 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111168 s, 94.3 MB/s 2023-11-06T03:14:06.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.253 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834869 s, 126 MB/s 2023-11-06T03:14:06.265 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.266 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.266 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112322 s, 93.4 MB/s 2023-11-06T03:14:06.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125971 s, 83.2 MB/s 2023-11-06T03:14:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087492 s, 120 MB/s 2023-11-06T03:14:06.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.307 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.307 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116477 s, 90.0 MB/s 2023-11-06T03:14:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.320 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113019 s, 92.8 MB/s 2023-11-06T03:14:06.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.335 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119739 s, 87.6 MB/s 2023-11-06T03:14:06.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010945 s, 95.8 MB/s 2023-11-06T03:14:06.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.361 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106544 s, 98.4 MB/s 2023-11-06T03:14:06.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.374 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110169 s, 95.2 MB/s 2023-11-06T03:14:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.385 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839357 s, 125 MB/s 2023-11-06T03:14:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.398 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105119 s, 99.8 MB/s 2023-11-06T03:14:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.412 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111095 s, 94.4 MB/s 2023-11-06T03:14:06.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:06.414 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:06.414 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:06.415 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.26 2023-11-06T03:14:06.415 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:06.415 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:06.416 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:06.416 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:06.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924349 s, 113 MB/s 2023-11-06T03:14:06.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108997 s, 96.2 MB/s 2023-11-06T03:14:06.499 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.500 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861158 s, 122 MB/s 2023-11-06T03:14:06.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.514 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011305 s, 92.8 MB/s 2023-11-06T03:14:06.527 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.527 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.527 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109 s, 96.2 MB/s 2023-11-06T03:14:06.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.541 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.541 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011345 s, 92.4 MB/s 2023-11-06T03:14:06.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126709 s, 82.8 MB/s 2023-11-06T03:14:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118623 s, 88.4 MB/s 2023-11-06T03:14:06.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.587 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136208 s, 77.0 MB/s 2023-11-06T03:14:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.611 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021431 s, 48.9 MB/s 2023-11-06T03:14:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109335 s, 95.9 MB/s 2023-11-06T03:14:06.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.654 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124431 s, 84.3 MB/s 2023-11-06T03:14:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.668 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115225 s, 91.0 MB/s 2023-11-06T03:14:06.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.683 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125381 s, 83.6 MB/s 2023-11-06T03:14:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.695 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942124 s, 111 MB/s 2023-11-06T03:14:06.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.712 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013483 s, 77.8 MB/s 2023-11-06T03:14:06.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141866 s, 73.9 MB/s 2023-11-06T03:14:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105403 s, 99.5 MB/s 2023-11-06T03:14:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121312 s, 86.4 MB/s 2023-11-06T03:14:06.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.768 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009485 s, 111 MB/s 2023-11-06T03:14:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.784 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012826 s, 81.8 MB/s 2023-11-06T03:14:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126295 s, 83.0 MB/s 2023-11-06T03:14:06.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132802 s, 79.0 MB/s 2023-11-06T03:14:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.830 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119261 s, 87.9 MB/s 2023-11-06T03:14:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.846 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133933 s, 78.3 MB/s 2023-11-06T03:14:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.862 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134201 s, 78.1 MB/s 2023-11-06T03:14:06.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.877 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.878 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125108 s, 83.8 MB/s 2023-11-06T03:14:06.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127327 s, 82.4 MB/s 2023-11-06T03:14:06.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130842 s, 80.1 MB/s 2023-11-06T03:14:06.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.926 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.926 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132081 s, 79.4 MB/s 2023-11-06T03:14:06.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.942 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.942 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123048 s, 85.2 MB/s 2023-11-06T03:14:06.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:06.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:06.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132931 s, 78.9 MB/s 2023-11-06T03:14:06.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:06.959 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:06.959 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:06.960 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.27 2023-11-06T03:14:06.960 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:06.960 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:06.961 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:06.961 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:07.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.027 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126654 s, 82.8 MB/s 2023-11-06T03:14:07.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.043 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130985 s, 80.1 MB/s 2023-11-06T03:14:07.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997516 s, 105 MB/s 2023-11-06T03:14:07.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.073 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130539 s, 80.3 MB/s 2023-11-06T03:14:07.087 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.088 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.088 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120996 s, 86.7 MB/s 2023-11-06T03:14:07.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141286 s, 74.2 MB/s 2023-11-06T03:14:07.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.124 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142078 s, 73.8 MB/s 2023-11-06T03:14:07.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146431 s, 71.6 MB/s 2023-11-06T03:14:07.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121584 s, 86.2 MB/s 2023-11-06T03:14:07.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149929 s, 69.9 MB/s 2023-11-06T03:14:07.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.194 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138395 s, 75.8 MB/s 2023-11-06T03:14:07.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.210 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.210 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140697 s, 74.5 MB/s 2023-11-06T03:14:07.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.226 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120005 s, 87.4 MB/s 2023-11-06T03:14:07.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.240 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114568 s, 91.5 MB/s 2023-11-06T03:14:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114746 s, 91.4 MB/s 2023-11-06T03:14:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.272 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.273 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151992 s, 69.0 MB/s 2023-11-06T03:14:07.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109235 s, 96.0 MB/s 2023-11-06T03:14:07.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.301 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121603 s, 86.2 MB/s 2023-11-06T03:14:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123615 s, 84.8 MB/s 2023-11-06T03:14:07.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.332 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130266 s, 80.5 MB/s 2023-11-06T03:14:07.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101819 s, 103 MB/s 2023-11-06T03:14:07.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.359 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113597 s, 92.3 MB/s 2023-11-06T03:14:07.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.372 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108214 s, 96.9 MB/s 2023-11-06T03:14:07.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.388 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124494 s, 84.2 MB/s 2023-11-06T03:14:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.403 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132188 s, 79.3 MB/s 2023-11-06T03:14:07.415 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.416 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.416 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101787 s, 103 MB/s 2023-11-06T03:14:07.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.431 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.431 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123586 s, 84.8 MB/s 2023-11-06T03:14:07.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123397 s, 85.0 MB/s 2023-11-06T03:14:07.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.462 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127327 s, 82.4 MB/s 2023-11-06T03:14:07.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120346 s, 87.1 MB/s 2023-11-06T03:14:07.491 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.491 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.491 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123025 s, 85.2 MB/s 2023-11-06T03:14:07.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.506 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122966 s, 85.3 MB/s 2023-11-06T03:14:07.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:07.508 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:07.508 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:07.509 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.28 2023-11-06T03:14:07.509 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:07.509 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:07.510 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:07.510 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:07.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.577 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116319 s, 90.1 MB/s 2023-11-06T03:14:07.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118289 s, 88.6 MB/s 2023-11-06T03:14:07.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125844 s, 83.3 MB/s 2023-11-06T03:14:07.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.623 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131237 s, 79.9 MB/s 2023-11-06T03:14:07.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.654 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.655 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119274 s, 87.9 MB/s 2023-11-06T03:14:07.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.668 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112834 s, 92.9 MB/s 2023-11-06T03:14:07.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.684 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128868 s, 81.4 MB/s 2023-11-06T03:14:07.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120065 s, 87.3 MB/s 2023-11-06T03:14:07.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122629 s, 85.5 MB/s 2023-11-06T03:14:07.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.728 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01155 s, 90.8 MB/s 2023-11-06T03:14:07.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.742 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112647 s, 93.1 MB/s 2023-11-06T03:14:07.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122434 s, 85.6 MB/s 2023-11-06T03:14:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967961 s, 108 MB/s 2023-11-06T03:14:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095877 s, 109 MB/s 2023-11-06T03:14:07.791 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.792 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841405 s, 125 MB/s 2023-11-06T03:14:07.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.804 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943 s, 111 MB/s 2023-11-06T03:14:07.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.815 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.816 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907417 s, 116 MB/s 2023-11-06T03:14:07.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.830 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012212 s, 85.9 MB/s 2023-11-06T03:14:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106577 s, 98.4 MB/s 2023-11-06T03:14:07.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122306 s, 85.7 MB/s 2023-11-06T03:14:07.872 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.872 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.873 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120324 s, 87.1 MB/s 2023-11-06T03:14:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.887 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120882 s, 86.7 MB/s 2023-11-06T03:14:07.900 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.901 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109358 s, 95.9 MB/s 2023-11-06T03:14:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107061 s, 97.9 MB/s 2023-11-06T03:14:07.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.929 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125067 s, 83.8 MB/s 2023-11-06T03:14:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.944 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124354 s, 84.3 MB/s 2023-11-06T03:14:07.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105874 s, 99.0 MB/s 2023-11-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125206 s, 83.7 MB/s 2023-11-06T03:14:07.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:07.989 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:07.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128079 s, 81.9 MB/s 2023-11-06T03:14:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120822 s, 86.8 MB/s 2023-11-06T03:14:08.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115273 s, 91.0 MB/s 2023-11-06T03:14:08.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108901 s, 96.3 MB/s 2023-11-06T03:14:08.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:08.033 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:08.034 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:08.034 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.29 2023-11-06T03:14:08.034 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:08.035 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:08.035 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:08.035 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:08.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.102 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124329 s, 84.3 MB/s 2023-11-06T03:14:08.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.116 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116587 s, 89.9 MB/s 2023-11-06T03:14:08.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.132 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125405 s, 83.6 MB/s 2023-11-06T03:14:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.144 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982195 s, 107 MB/s 2023-11-06T03:14:08.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.156 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.156 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956637 s, 110 MB/s 2023-11-06T03:14:08.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.167 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.168 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900646 s, 116 MB/s 2023-11-06T03:14:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.179 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889286 s, 118 MB/s 2023-11-06T03:14:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111017 s, 94.5 MB/s 2023-11-06T03:14:08.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.208 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126614 s, 82.8 MB/s 2023-11-06T03:14:08.221 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110978 s, 94.5 MB/s 2023-11-06T03:14:08.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112016 s, 93.6 MB/s 2023-11-06T03:14:08.248 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.248 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.249 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102107 s, 103 MB/s 2023-11-06T03:14:08.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.262 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107509 s, 97.5 MB/s 2023-11-06T03:14:08.275 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.275 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.276 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104202 s, 101 MB/s 2023-11-06T03:14:08.291 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.291 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.291 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129396 s, 81.0 MB/s 2023-11-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124845 s, 84.0 MB/s 2023-11-06T03:14:08.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.321 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117723 s, 89.1 MB/s 2023-11-06T03:14:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139443 s, 75.2 MB/s 2023-11-06T03:14:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122946 s, 85.3 MB/s 2023-11-06T03:14:08.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.369 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.369 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131624 s, 79.7 MB/s 2023-11-06T03:14:08.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.382 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102999 s, 102 MB/s 2023-11-06T03:14:08.395 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.396 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01088 s, 96.4 MB/s 2023-11-06T03:14:08.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.411 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123294 s, 85.0 MB/s 2023-11-06T03:14:08.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.426 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.426 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112239 s, 93.4 MB/s 2023-11-06T03:14:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.442 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135227 s, 77.5 MB/s 2023-11-06T03:14:08.457 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.458 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129052 s, 81.3 MB/s 2023-11-06T03:14:08.473 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.474 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127898 s, 82.0 MB/s 2023-11-06T03:14:08.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.489 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121282 s, 86.5 MB/s 2023-11-06T03:14:08.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119413 s, 87.8 MB/s 2023-11-06T03:14:08.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130181 s, 80.5 MB/s 2023-11-06T03:14:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.534 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119175 s, 88.0 MB/s 2023-11-06T03:14:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121827 s, 86.1 MB/s 2023-11-06T03:14:08.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:08.552 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:08.553 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:08.554 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.30 2023-11-06T03:14:08.554 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:08.555 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:08.555 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:08.556 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:08.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.628 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.628 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179642 s, 58.4 MB/s 2023-11-06T03:14:08.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108204 s, 96.9 MB/s 2023-11-06T03:14:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012601 s, 83.2 MB/s 2023-11-06T03:14:08.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.689 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114368 s, 91.7 MB/s 2023-11-06T03:14:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113989 s, 92.0 MB/s 2023-11-06T03:14:08.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.720 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139042 s, 75.4 MB/s 2023-11-06T03:14:08.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.735 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113463 s, 92.4 MB/s 2023-11-06T03:14:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.749 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.750 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124473 s, 84.2 MB/s 2023-11-06T03:14:08.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.764 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.764 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116734 s, 89.8 MB/s 2023-11-06T03:14:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116705 s, 89.8 MB/s 2023-11-06T03:14:08.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127413 s, 82.3 MB/s 2023-11-06T03:14:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133034 s, 78.8 MB/s 2023-11-06T03:14:08.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122237 s, 85.8 MB/s 2023-11-06T03:14:08.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116398 s, 90.1 MB/s 2023-11-06T03:14:08.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106345 s, 98.6 MB/s 2023-11-06T03:14:08.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.868 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.868 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104649 s, 100 MB/s 2023-11-06T03:14:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889031 s, 118 MB/s 2023-11-06T03:14:08.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.891 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846426 s, 124 MB/s 2023-11-06T03:14:08.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107489 s, 97.6 MB/s 2023-11-06T03:14:08.916 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993402 s, 106 MB/s 2023-11-06T03:14:08.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.930 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.930 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102496 s, 102 MB/s 2023-11-06T03:14:08.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.944 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120799 s, 86.8 MB/s 2023-11-06T03:14:08.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.961 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135168 s, 77.6 MB/s 2023-11-06T03:14:08.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130637 s, 80.3 MB/s 2023-11-06T03:14:08.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:08.994 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:08.994 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131337 s, 79.8 MB/s 2023-11-06T03:14:09.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118729 s, 88.3 MB/s 2023-11-06T03:14:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.023 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123058 s, 85.2 MB/s 2023-11-06T03:14:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108256 s, 96.9 MB/s 2023-11-06T03:14:09.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131668 s, 79.6 MB/s 2023-11-06T03:14:09.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106091 s, 98.8 MB/s 2023-11-06T03:14:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.083 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130796 s, 80.2 MB/s 2023-11-06T03:14:09.098 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.099 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125705 s, 83.4 MB/s 2023-11-06T03:14:09.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:09.100 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:09.101 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:09.101 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.31 2023-11-06T03:14:09.101 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:09.102 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:09.102 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:09.102 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:09.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128817 s, 81.4 MB/s 2023-11-06T03:14:09.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012433 s, 84.3 MB/s 2023-11-06T03:14:09.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013422 s, 78.1 MB/s 2023-11-06T03:14:09.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.218 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108467 s, 96.7 MB/s 2023-11-06T03:14:09.234 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.234 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.234 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133363 s, 78.6 MB/s 2023-11-06T03:14:09.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.251 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.251 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126261 s, 83.0 MB/s 2023-11-06T03:14:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139242 s, 75.3 MB/s 2023-11-06T03:14:09.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142193 s, 73.7 MB/s 2023-11-06T03:14:09.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.302 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134343 s, 78.1 MB/s 2023-11-06T03:14:09.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.318 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134907 s, 77.7 MB/s 2023-11-06T03:14:09.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.335 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134646 s, 77.9 MB/s 2023-11-06T03:14:09.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104832 s, 100 MB/s 2023-11-06T03:14:09.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.363 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117558 s, 89.2 MB/s 2023-11-06T03:14:09.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.378 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116832 s, 89.8 MB/s 2023-11-06T03:14:09.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130633 s, 80.3 MB/s 2023-11-06T03:14:09.408 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.409 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110953 s, 94.5 MB/s 2023-11-06T03:14:09.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.424 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129216 s, 81.1 MB/s 2023-11-06T03:14:09.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.437 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108676 s, 96.5 MB/s 2023-11-06T03:14:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.453 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130049 s, 80.6 MB/s 2023-11-06T03:14:09.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.468 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123035 s, 85.2 MB/s 2023-11-06T03:14:09.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.484 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127164 s, 82.5 MB/s 2023-11-06T03:14:09.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.498 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122658 s, 85.5 MB/s 2023-11-06T03:14:09.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.515 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133405 s, 78.6 MB/s 2023-11-06T03:14:09.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.530 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012521 s, 83.7 MB/s 2023-11-06T03:14:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131467 s, 79.8 MB/s 2023-11-06T03:14:09.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.559 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.560 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104516 s, 100 MB/s 2023-11-06T03:14:09.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.576 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.576 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131653 s, 79.6 MB/s 2023-11-06T03:14:09.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.593 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140032 s, 74.9 MB/s 2023-11-06T03:14:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.606 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102156 s, 103 MB/s 2023-11-06T03:14:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.619 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100256 s, 105 MB/s 2023-11-06T03:14:09.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.659 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01206 s, 86.9 MB/s 2023-11-06T03:14:09.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:09.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:09.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128423 s, 81.7 MB/s 2023-11-06T03:14:09.676 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2023-11-06T03:14:09.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 Processor -- start 2023-11-06T03:14:09.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 -- start start 2023-11-06T03:14:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 0x7ff47815b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff47806c550 0x7ff47815b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4780cb100 con 0x7ff47806c170 2023-11-06T03:14:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4780ce0a0 con 0x7ff47806c550 2023-11-06T03:14:09.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff47d867640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47815bef0 con 0x7ff478155310 2023-11-06T03:14:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff47806c550 0x7ff47815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff47806c550 0x7ff47815b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39832/0 (socket says 172.21.15.157:39832) 2023-11-06T03:14:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 -- 172.21.15.157:0/3286053967 learned_addr learned my addr 172.21.15.157:0/3286053967 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 0x7ff47815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 -- 172.21.15.157:0/3286053967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 msgr2=0x7ff47815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 0x7ff47815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 -- 172.21.15.157:0/3286053967 >> v1:172.21.15.195:6789/0 conn(0x7ff47806c170 legacy=0x7ff478154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4777fe640 1 -- 172.21.15.157:0/3286053967 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780cb350 con 0x7ff47806c550 2023-11-06T03:14:09.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.853+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 0x7ff47815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:09.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff47806c550 0x7ff47815b690 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e50 tx=0x7ff47001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec3805faf360606 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff475ffb640 1 -- 172.21.15.157:0/3286053967 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700a5820 con 0x7ff47806c550 2023-11-06T03:14:09.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff475ffb640 1 -- 172.21.15.157:0/3286053967 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4700a5980 con 0x7ff47806c550 2023-11-06T03:14:09.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff475ffb640 1 -- 172.21.15.157:0/3286053967 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700a5d10 con 0x7ff47806c550 2023-11-06T03:14:09.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- 172.21.15.157:0/3286053967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff47806c550 msgr2=0x7ff47815b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff47806c550 0x7ff47815b690 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e50 tx=0x7ff47001c9a0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- 172.21.15.157:0/3286053967 shutdown_connections 2023-11-06T03:14:09.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff47806c550 0x7ff47815b690 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 --2- 172.21.15.157:0/3286053967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff478155310 0x7ff47815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- 172.21.15.157:0/3286053967 >> 172.21.15.157:0/3286053967 conn(0x7ff4780b43b0 msgr2=0x7ff478065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- 172.21.15.157:0/3286053967 shutdown_connections 2023-11-06T03:14:09.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- 172.21.15.157:0/3286053967 wait complete. 2023-11-06T03:14:09.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 Processor -- start 2023-11-06T03:14:09.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.857+0000 7ff47d867640 1 -- start start 2023-11-06T03:14:09.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff47806c170 0x7ff47814c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 0x7ff4780b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478153f70 con 0x7ff478155310 2023-11-06T03:14:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780b0bf0 con 0x7ff47806c550 2023-11-06T03:14:09.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781541c0 con 0x7ff47806c170 2023-11-06T03:14:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39838/0 (socket says 172.21.15.157:39838) 2023-11-06T03:14:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 -- 172.21.15.157:0/372896676 learned_addr learned my addr 172.21.15.157:0/372896676 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4767fc640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 0x7ff4780b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff476ffd640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff47806c170 0x7ff47814c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 -- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff47806c170 msgr2=0x7ff47814c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff47806c170 0x7ff47814c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 -- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 msgr2=0x7ff4780b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 0x7ff4780b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 -- 172.21.15.157:0/372896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff470009b30 con 0x7ff478155310 2023-11-06T03:14:09.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4767fc640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 0x7ff4780b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:09.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff4777fe640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff4700af040 tx=0x7ff470019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff45ffff640 1 -- 172.21.15.157:0/372896676 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700b8880 con 0x7ff478155310 2023-11-06T03:14:09.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff45ffff640 1 -- 172.21.15.157:0/372896676 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4700b8a20 con 0x7ff478155310 2023-11-06T03:14:09.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4780cbfb0 con 0x7ff478155310 2023-11-06T03:14:09.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff45ffff640 1 -- 172.21.15.157:0/372896676 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700b8dd0 con 0x7ff478155310 2023-11-06T03:14:09.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.861+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47814bf30 con 0x7ff478155310 2023-11-06T03:14:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 Processor -- start 2023-11-06T03:14:09.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- start start 2023-11-06T03:14:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaa1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca57af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc920150 con 0x55eccca58030 2023-11-06T03:14:09.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8ac1b0 con 0x55ecccaa2380 2023-11-06T03:14:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8a9780 con 0x55ecccaa1a60 2023-11-06T03:14:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45ffff640 1 -- 172.21.15.157:0/372896676 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4700c1410 con 0x7ff478155310 2023-11-06T03:14:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39852/0 (socket says 172.21.15.157:39852) 2023-11-06T03:14:09.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 -- 172.21.15.157:0/3845288721 learned_addr learned my addr 172.21.15.157:0/3845288721 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaa1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca57af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 -- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 msgr2=0x55ecccaa1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaa1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 -- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 msgr2=0x55eccca57af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca57af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 -- 172.21.15.157:0/3845288721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eccc5a1da0 con 0x55eccca58030 2023-11-06T03:14:09.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45ffff640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 0x7ff464067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45ffff640 1 -- 172.21.15.157:0/372896676 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff47011e3b0 con 0x7ff478155310 2023-11-06T03:14:09.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00af90 tx=0x7ff46c00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d3659b7a1ba538 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff476ffd640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 0x7ff464067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45d7fa640 1 -- 172.21.15.157:0/3845288721 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c004030 con 0x55eccca58030 2023-11-06T03:14:09.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45d7fa640 1 -- 172.21.15.157:0/3845288721 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c00c040 con 0x55eccca58030 2023-11-06T03:14:09.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff45d7fa640 1 -- 172.21.15.157:0/3845288721 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0043d0 con 0x55eccca58030 2023-11-06T03:14:09.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- 172.21.15.157:0/3845288721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 msgr2=0x55eccca58410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00af90 tx=0x7ff46c00a4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- 172.21.15.157:0/3845288721 shutdown_connections 2023-11-06T03:14:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca58410 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca57af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff476ffd640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 0x7ff464067f50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff478153d30 tx=0x7ff468009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 --2- 172.21.15.157:0/3845288721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaa1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- 172.21.15.157:0/3845288721 >> 172.21.15.157:0/3845288721 conn(0x55eccca5e3a0 msgr2=0x55eccca563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- 172.21.15.157:0/3845288721 shutdown_connections 2023-11-06T03:14:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- 172.21.15.157:0/3845288721 wait complete. 2023-11-06T03:14:09.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 -- start start 2023-11-06T03:14:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.865+0000 7ff48098f000 1 Processor -- start 2023-11-06T03:14:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55eccca62f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8a5070 con 0x55eccca58030 2023-11-06T03:14:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8447c0 con 0x55ecccaa2380 2023-11-06T03:14:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eccc844460 con 0x55ecccaa1a60 2023-11-06T03:14:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 10 client.0 _ll_register_callbacks cb 0x55eccc89e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:09.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39858/0 (socket says 172.21.15.157:39858) 2023-11-06T03:14:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 -- 172.21.15.157:0/3548660231 learned_addr learned my addr 172.21.15.157:0/3548660231 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55eccca62f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 -- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 msgr2=0x55eccca62f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55eccca62f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 -- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 msgr2=0x55eccca634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 -- 172.21.15.157:0/3548660231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c00abc0 con 0x55eccca58030 2023-11-06T03:14:09.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff468004d40 tx=0x7ff468003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bde7490a419d4b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468004b60 con 0x55eccca58030 2023-11-06T03:14:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff46801d040 con 0x55eccca58030 2023-11-06T03:14:09.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eccc85e840 con 0x55eccca58030 2023-11-06T03:14:09.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff468004750 con 0x55eccca58030 2023-11-06T03:14:09.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 10 client.8979 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:09.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55eccc845410 con 0x55eccca58030 2023-11-06T03:14:09.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eccc85e840 con 0x55eccca58030 2023-11-06T03:14:09.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff468015460 con 0x55eccca58030 2023-11-06T03:14:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 10 client.8979 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 10 client.8979 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 20 client.8979 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55ecccab0af0 0x55ecccab2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 4 client.8979 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eccc8a39c0 con 0x55ecccab0af0 2023-11-06T03:14:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55ecccab0af0 0x55ecccab2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 10 client.8979 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55ecccab0af0 0x55ecccab2ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc149166645d4fd6 server_cookie=97783aa1cec43fbe in_seq=0 out_seq=0 2023-11-06T03:14:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 1 -- 172.21.15.157:0/3548660231 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eccc8a39c0 con 0x55ecccab0af0 2023-11-06T03:14:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 10 client.8979 handle_command_reply: tid=0 2023-11-06T03:14:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 client.8979 shutdown 2023-11-06T03:14:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 msgr2=0x55eccca62a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7ff468004d40 tx=0x7ff468003700 comp rx=0 tx=0).stop 2023-11-06T03:14:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 shutdown_connections 2023-11-06T03:14:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55ecccab0af0 0x55ecccab2ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55eccca634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55eccca62f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff4567fc640 0 client.8979 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 --2- 172.21.15.157:0/3548660231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55eccca62a20 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 >> 172.21.15.157:0/3548660231 conn(0x55eccca5e3a0 msgr2=0x55eccca56ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 shutdown_connections 2023-11-06T03:14:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 1 -- 172.21.15.157:0/3548660231 wait complete. 2023-11-06T03:14:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.869+0000 7ff48098f000 20 client.8979 trim_cache size 0 max 16384 2023-11-06T03:14:09.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 Processor -- start 2023-11-06T03:14:09.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- start start 2023-11-06T03:14:09.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaf9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55ecccafa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc562a20 con 0x55eccca58030 2023-11-06T03:14:09.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8a5ca0 con 0x55ecccaa2380 2023-11-06T03:14:09.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8a4880 con 0x55ecccaa1a60 2023-11-06T03:14:09.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39864/0 (socket says 172.21.15.157:39864) 2023-11-06T03:14:09.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 -- 172.21.15.157:0/168511512 learned_addr learned my addr 172.21.15.157:0/168511512 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 -- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 msgr2=0x55ecccaf9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaf9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 -- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 msgr2=0x55ecccafa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55ecccafa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 -- 172.21.15.157:0/168511512 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47815a530 con 0x55eccca58030 2023-11-06T03:14:09.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff476ffd640 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff4680037b0 tx=0x7ff468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7522ed24bb743c18 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff45dffb640 1 -- 172.21.15.157:0/168511512 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468003c10 con 0x55eccca58030 2023-11-06T03:14:09.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff45dffb640 1 -- 172.21.15.157:0/168511512 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468035070 con 0x55eccca58030 2023-11-06T03:14:09.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff45dffb640 1 -- 172.21.15.157:0/168511512 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468015e40 con 0x55eccca58030 2023-11-06T03:14:09.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- 172.21.15.157:0/168511512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 msgr2=0x55ecccaf99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff4680037b0 tx=0x7ff468017000 comp rx=0 tx=0).stop 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- 172.21.15.157:0/168511512 shutdown_connections 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecccaa2380 0x55ecccafa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccaf9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 --2- 172.21.15.157:0/168511512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eccca58030 0x55ecccaf99a0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.885+0000 7ff48098f000 1 -- 172.21.15.157:0/168511512 >> 172.21.15.157:0/168511512 conn(0x55eccca5e3a0 msgr2=0x55ecccab3770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- 172.21.15.157:0/168511512 shutdown_connections 2023-11-06T03:14:09.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- 172.21.15.157:0/168511512 wait complete. 2023-11-06T03:14:09.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- start start 2023-11-06T03:14:09.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 Processor -- start 2023-11-06T03:14:09.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eccca58030 0x55ecccafdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccafe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecccab76e0 con 0x55ecccaa2380 2023-11-06T03:14:09.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eccc854830 con 0x55eccca58030 2023-11-06T03:14:09.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eccc8a0fe0 con 0x55ecccaa1a60 2023-11-06T03:14:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 10 client.0 _ll_register_callbacks cb 0x55eccc86ef80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39876/0 (socket says 172.21.15.157:39876) 2023-11-06T03:14:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 -- 172.21.15.157:0/3553770038 learned_addr learned my addr 172.21.15.157:0/3553770038 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:09.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eccca58030 0x55ecccafdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4767fc640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccafe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 -- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 msgr2=0x55ecccafe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccafe520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 -- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eccca58030 msgr2=0x55ecccafdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:09.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eccca58030 0x55ecccafdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 -- 172.21.15.157:0/3553770038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eccc845410 con 0x55ecccaa2380 2023-11-06T03:14:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4777fe640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4700ef6e0 tx=0x7ff4700f01a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc585f776a69318d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:09.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700f79c0 con 0x55ecccaa2380 2023-11-06T03:14:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4700efc40 con 0x55ecccaa2380 2023-11-06T03:14:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eccc8ad990 con 0x55ecccaa2380 2023-11-06T03:14:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff470107020 con 0x55ecccaa2380 2023-11-06T03:14:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 10 client.8985 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:09.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55eccc86a370 con 0x55ecccaa2380 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eccc8ad990 con 0x55ecccaa2380 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff470005d20 con 0x55ecccaa2380 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 10 client.8985 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 10 client.8985 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 20 client.8985 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:09.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55eccca9e9d0 0x55eccca9edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 4 client.8985 mds_command: new command op to 8796 tid=0[{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2023-11-06T03:14:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55eccc89e0f0 con 0x55eccca9e9d0 2023-11-06T03:14:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55eccca9e9d0 0x55eccca9edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff4567fc640 10 client.8985 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:09.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.889+0000 7ff476ffd640 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55eccca9e9d0 0x55eccca9edb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43b37d3a6b2c7d2 server_cookie=1b12eeb4039f7974 in_seq=0 out_seq=0 2023-11-06T03:14:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff4567fc640 1 -- 172.21.15.157:0/3553770038 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55eccc89e0f0 con 0x55eccca9e9d0 2023-11-06T03:14:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff4567fc640 10 client.8985 handle_command_reply: tid=0 2023-11-06T03:14:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 client.8985 shutdown 2023-11-06T03:14:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 msgr2=0x55eccca9e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff4700ef6e0 tx=0x7ff4700f01a0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 shutdown_connections 2023-11-06T03:14:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ecccaa2380 0x55eccca9e010 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55ecccaa1a60 0x55ecccafe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55eccca9e9d0 0x55eccca9edb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 --2- 172.21.15.157:0/3553770038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eccca58030 0x55ecccafdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 >> 172.21.15.157:0/3553770038 conn(0x55eccca5e3a0 msgr2=0x55ecccab3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff4567fc640 0 client.8985 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:09.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 shutdown_connections 2023-11-06T03:14:09.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.893+0000 7ff48098f000 1 -- 172.21.15.157:0/3553770038 wait complete. 2023-11-06T03:14:09.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff48098f000 20 client.8985 trim_cache size 0 max 16384 2023-11-06T03:14:09.912 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:09.912 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:14:09.913 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "dfb8ccf2-e836-44cd-a854-d8a1428b7a2c", 2023-11-06T03:14:09.913 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:14:09.913 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 msgr2=0x7ff464067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 0x7ff464067f50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff478153d30 tx=0x7ff468009040 comp rx=0 tx=0).stop 2023-11-06T03:14:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 msgr2=0x7ff4780b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:09.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff4700af040 tx=0x7ff470019740 comp rx=0 tx=0).stop 2023-11-06T03:14:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 shutdown_connections 2023-11-06T03:14:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff464065a90 0x7ff464067f50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff478155310 0x7ff4780b1c80 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff47806c550 0x7ff4780b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 --2- 172.21.15.157:0/372896676 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff47806c170 0x7ff47814c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 >> 172.21.15.157:0/372896676 conn(0x7ff4780b43b0 msgr2=0x7ff4780b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 shutdown_connections 2023-11-06T03:14:09.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:09.897+0000 7ff47d867640 1 -- 172.21.15.157:0/372896676 wait complete. 2023-11-06T03:14:09.918 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub pause 2023-11-06T03:14:10.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 Processor -- start 2023-11-06T03:14:10.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 -- start start 2023-11-06T03:14:10.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb9c15aae0 0x7fdb9c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0b2020 con 0x7fdb9c15a700 2023-11-06T03:14:10.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c0cc220 con 0x7fdb9c066040 2023-11-06T03:14:10.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdba0a98640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b2840 con 0x7fdb9c15aae0 2023-11-06T03:14:10.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdb9c15a700 0x7fdb9c065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58204/0 (socket says 172.21.15.157:58204) 2023-11-06T03:14:10.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 -- 172.21.15.157:0/213420367 learned_addr learned my addr 172.21.15.157:0/213420367 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:10.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342849106 0 0) 0x7fdb9c0b2020 con 0x7fdb9c15a700 2023-11-06T03:14:10.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb84003530 con 0x7fdb9c15a700 2023-11-06T03:14:10.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/213420367 >> v2:172.21.15.195:3300/0 conn(0x7fdb9c15aae0 0x7fdb9c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171653695 0 0) 0x7fdb84003530 con 0x7fdb9c15a700 2023-11-06T03:14:10.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 >> v2:172.21.15.195:3300/0 conn(0x7fdb9c15aae0 msgr2=0x7fdb9c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 --2- 172.21.15.157:0/213420367 >> v2:172.21.15.195:3300/0 conn(0x7fdb9c15aae0 0x7fdb9c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 >> v1:172.21.15.195:6789/0 conn(0x7fdb9c066040 legacy=0x7fdb9c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c15def0 con 0x7fdb9c15a700 2023-11-06T03:14:10.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdb8c0016a0 con 0x7fdb9c15a700 2023-11-06T03:14:10.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/213420367 >> v2:172.21.15.195:3300/0 conn(0x7fdb9c15aae0 0x7fdb9c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:10.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 >> v1:172.21.15.157:6789/0 conn(0x7fdb9c15a700 legacy=0x7fdb9c065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb840047d0 0x7fdb84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb84005190 0x7fdb8400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b2840 con 0x7fdb84003980 2023-11-06T03:14:10.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8400c0e0 con 0x7fdb84005190 2023-11-06T03:14:10.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8400c2a0 con 0x7fdb840047d0 2023-11-06T03:14:10.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb84005190 0x7fdb8400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 -- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb840047d0 msgr2=0x7fdb84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb840047d0 0x7fdb84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 -- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb84005190 msgr2=0x7fdb8400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb84005190 0x7fdb8400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9a59c640 1 -- 172.21.15.157:0/213420367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84003530 con 0x7fdb84003980 2023-11-06T03:14:10.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.057+0000 7fdb9ad9d640 1 -- 172.21.15.157:0/213420367 reap_dead start 2023-11-06T03:14:10.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb84003da0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cbcf0 tx=0x7fdb8c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131b3bb1f56bacb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c005ae0 con 0x7fdb84003980 2023-11-06T03:14:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c005c40 con 0x7fdb84003980 2023-11-06T03:14:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdb9959a640 1 -- 172.21.15.157:0/213420367 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0a58d0 con 0x7fdb84003980 2023-11-06T03:14:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 msgr2=0x7fdb84003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb84003da0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cbcf0 tx=0x7fdb8c01cbd0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- 172.21.15.157:0/213420367 shutdown_connections 2023-11-06T03:14:10.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 --2- 172.21.15.157:0/213420367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb84003da0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- 172.21.15.157:0/213420367 >> 172.21.15.157:0/213420367 conn(0x7fdb9c06a170 msgr2=0x7fdb9c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- 172.21.15.157:0/213420367 shutdown_connections 2023-11-06T03:14:10.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- 172.21.15.157:0/213420367 wait complete. 2023-11-06T03:14:10.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 Processor -- start 2023-11-06T03:14:10.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.061+0000 7fdba0a98640 1 -- start start 2023-11-06T03:14:10.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb9c0b0610 0x7fdb9c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb9c0adc10 0x7fdb9c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0b01f0 con 0x7fdb84003980 2023-11-06T03:14:10.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c158e80 con 0x7fdb9c0b0610 2023-11-06T03:14:10.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0af890 con 0x7fdb9c0adc10 2023-11-06T03:14:10.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39890/0 (socket says 172.21.15.157:39890) 2023-11-06T03:14:10.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 -- 172.21.15.157:0/1270328333 learned_addr learned my addr 172.21.15.157:0/1270328333 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb9c0adc10 0x7fdb9c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb9c0b0610 0x7fdb9c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 -- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb9c0adc10 msgr2=0x7fdb9c15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb9c0adc10 0x7fdb9c15bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 -- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb9c0b0610 msgr2=0x7fdb9c0ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb9c0b0610 0x7fdb9c0ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 -- 172.21.15.157:0/1270328333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0afb30 con 0x7fdb84003980 2023-11-06T03:14:10.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c01c640 tx=0x7fdb84005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 -- 172.21.15.157:0/1270328333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0b4070 con 0x7fdb84003980 2023-11-06T03:14:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 -- 172.21.15.157:0/1270328333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0015e0 con 0x7fdb84003980 2023-11-06T03:14:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c0afe10 con 0x7fdb84003980 2023-11-06T03:14:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 -- 172.21.15.157:0/1270328333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0a58d0 con 0x7fdb84003980 2023-11-06T03:14:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0ae170 con 0x7fdb84003980 2023-11-06T03:14:10.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 -- 172.21.15.157:0/1270328333 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdb8c0a5a30 con 0x7fdb84003980 2023-11-06T03:14:10.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 0x7fdb7c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb8affd640 1 -- 172.21.15.157:0/1270328333 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb8c11cf20 con 0x7fdb84003980 2023-11-06T03:14:10.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.065+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 0x7fdb7c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 Processor -- start 2023-11-06T03:14:10.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- start start 2023-11-06T03:14:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 0x7fdb7c067f50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15cff0 tx=0x7fdb90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffdba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc81bb0 con 0x5626ffe040d0 2023-11-06T03:14:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc10380 con 0x5626ffdba200 2023-11-06T03:14:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc0d950 con 0x5626ffe049f0 2023-11-06T03:14:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39900/0 (socket says 172.21.15.157:39900) 2023-11-06T03:14:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 -- 172.21.15.157:0/3983128565 learned_addr learned my addr 172.21.15.157:0/3983128565 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 -- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 msgr2=0x5626ffdb9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 -- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 msgr2=0x5626ffdba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffdba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 -- 172.21.15.157:0/3983128565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ff905dc0 con 0x5626ffe040d0 2023-11-06T03:14:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ea9d0 tx=0x7fdb8c0ad3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc698e147a3a5de1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb7bfff640 1 -- 172.21.15.157:0/3983128565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0ee4d0 con 0x5626ffe040d0 2023-11-06T03:14:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb7bfff640 1 -- 172.21.15.157:0/3983128565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0f2300 con 0x5626ffe040d0 2023-11-06T03:14:10.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdb7bfff640 1 -- 172.21.15.157:0/3983128565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0f64b0 con 0x5626ffe040d0 2023-11-06T03:14:10.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3983128565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 msgr2=0x5626ffe044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ea9d0 tx=0x7fdb8c0ad3c0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3983128565 shutdown_connections 2023-11-06T03:14:10.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffdba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3983128565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffe044b0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3983128565 >> 172.21.15.157:0/3983128565 conn(0x5626ffdc0990 msgr2=0x5626ffdb85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3983128565 shutdown_connections 2023-11-06T03:14:10.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3983128565 wait complete. 2023-11-06T03:14:10.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- start start 2023-11-06T03:14:10.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 Processor -- start 2023-11-06T03:14:10.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe040d0 0x5626ffe0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc09240 con 0x5626ffe049f0 2023-11-06T03:14:10.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffba8410 con 0x5626ffdba200 2023-11-06T03:14:10.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.069+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626ffbc6600 con 0x5626ffe040d0 2023-11-06T03:14:10.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 10 client.0 _ll_register_callbacks cb 0x5626ffda5bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:10.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39910/0 (socket says 172.21.15.157:39910) 2023-11-06T03:14:10.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 -- 172.21.15.157:0/3460412629 learned_addr learned my addr 172.21.15.157:0/3460412629 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe040d0 0x5626ffe0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 -- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe040d0 msgr2=0x5626ffe0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe040d0 0x5626ffe0b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 -- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 msgr2=0x5626ffe0b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe0b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 -- 172.21.15.157:0/3460412629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0ae170 con 0x5626ffe049f0 2023-11-06T03:14:10.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9ad9d640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c071750 tx=0x7fdb94004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206fb4bf09c8a0c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:10.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94014590 con 0x5626ffe049f0 2023-11-06T03:14:10.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ffbc6040 con 0x5626ffe049f0 2023-11-06T03:14:10.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb940146f0 con 0x5626ffe049f0 2023-11-06T03:14:10.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb940148b0 con 0x5626ffe049f0 2023-11-06T03:14:10.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 10 client.8997 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:10.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5626ffbce170 con 0x5626ffe049f0 2023-11-06T03:14:10.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626ffbc6040 con 0x5626ffe049f0 2023-11-06T03:14:10.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdb94014a10 con 0x5626ffe049f0 2023-11-06T03:14:10.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 10 client.8997 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:10.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 10 client.8997 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:10.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 20 client.8997 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:10.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffdc5e00 0x5626ffdc6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 4 client.8997 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626ffc07b90 con 0x5626ffdc5e00 2023-11-06T03:14:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffdc5e00 0x5626ffdc6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffdc5e00 0x5626ffdc6210 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d2f0a72222d07c server_cookie=92ea29a3cb0157a6 in_seq=0 out_seq=0 2023-11-06T03:14:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 10 client.8997 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 1 -- 172.21.15.157:0/3460412629 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5626ffc07b90 con 0x5626ffdc5e00 2023-11-06T03:14:10.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 10 client.8997 handle_command_reply: tid=0 2023-11-06T03:14:10.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 client.8997 shutdown 2023-11-06T03:14:10.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 msgr2=0x5626ffdc5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c071750 tx=0x7fdb94004410 comp rx=0 tx=0).stop 2023-11-06T03:14:10.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 shutdown_connections 2023-11-06T03:14:10.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe049f0 0x5626ffdc5090 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe040d0 0x5626ffe0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffdc5e00 0x5626ffdc6210 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/3460412629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 >> 172.21.15.157:0/3460412629 conn(0x5626ffdc0990 msgr2=0x5626ffdb8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdb797fa640 0 client.8997 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 shutdown_connections 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 1 -- 172.21.15.157:0/3460412629 wait complete. 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.073+0000 7fdba3bc0000 20 client.8997 trim_cache size 0 max 16384 2023-11-06T03:14:10.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 Processor -- start 2023-11-06T03:14:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- start start 2023-11-06T03:14:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe08420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdbfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff8c6a40 con 0x5626ffe040d0 2023-11-06T03:14:10.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc0bda0 con 0x5626ffdba200 2023-11-06T03:14:10.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc035e0 con 0x5626ffe049f0 2023-11-06T03:14:10.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39926/0 (socket says 172.21.15.157:39926) 2023-11-06T03:14:10.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 -- 172.21.15.157:0/277513200 learned_addr learned my addr 172.21.15.157:0/277513200 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 -- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 msgr2=0x5626ffdbfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdbfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 -- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 msgr2=0x5626ffe08420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe08420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 -- 172.21.15.157:0/277513200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ffbb44c0 con 0x5626ffe040d0 2023-11-06T03:14:10.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdb900052e0 tx=0x7fdb90007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffbf0030aa84fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb88ff9640 1 -- 172.21.15.157:0/277513200 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000d6f0 con 0x5626ffe040d0 2023-11-06T03:14:10.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb88ff9640 1 -- 172.21.15.157:0/277513200 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000d870 con 0x5626ffe040d0 2023-11-06T03:14:10.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb88ff9640 1 -- 172.21.15.157:0/277513200 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000dc20 con 0x5626ffe040d0 2023-11-06T03:14:10.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- 172.21.15.157:0/277513200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 msgr2=0x5626ffdbf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdb900052e0 tx=0x7fdb90007850 comp rx=0 tx=0).stop 2023-11-06T03:14:10.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- 172.21.15.157:0/277513200 shutdown_connections 2023-11-06T03:14:10.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffdbfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffe040d0 0x5626ffdbf8c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/277513200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffdba200 0x5626ffe08420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- 172.21.15.157:0/277513200 >> 172.21.15.157:0/277513200 conn(0x5626ffdc0990 msgr2=0x5626ffe0a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- 172.21.15.157:0/277513200 shutdown_connections 2023-11-06T03:14:10.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- 172.21.15.157:0/277513200 wait complete. 2023-11-06T03:14:10.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- start start 2023-11-06T03:14:10.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 Processor -- start 2023-11-06T03:14:10.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffe040d0 0x5626ffe610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffe449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc05fd0 con 0x5626ffdba200 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc0d950 con 0x5626ffe040d0 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626ffc06a50 con 0x5626ffe049f0 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdba3bc0000 10 client.0 _ll_register_callbacks cb 0x5626ffbcdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39934/0 (socket says 172.21.15.157:39934) 2023-11-06T03:14:10.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 -- 172.21.15.157:0/2836611552 learned_addr learned my addr 172.21.15.157:0/2836611552 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffe040d0 0x5626ffe610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 -- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 msgr2=0x5626ffe449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffe449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 -- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffe040d0 msgr2=0x5626ffe610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffe040d0 0x5626ffe610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 -- 172.21.15.157:0/2836611552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0cc220 con 0x5626ffdba200 2023-11-06T03:14:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb9a59c640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ad570 tx=0x7fdb8c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e795abbe8c1a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:10.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.089+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0f64b0 con 0x5626ffdba200 2023-11-06T03:14:10.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb8c0ec870 con 0x5626ffdba200 2023-11-06T03:14:10.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ffbd25c0 con 0x5626ffdba200 2023-11-06T03:14:10.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8c0a5ce0 con 0x5626ffdba200 2023-11-06T03:14:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 10 client.9003 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5626ffba8030 con 0x5626ffdba200 2023-11-06T03:14:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626ffbd25c0 con 0x5626ffdba200 2023-11-06T03:14:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdb8c0c0060 con 0x5626ffdba200 2023-11-06T03:14:10.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 10 client.9003 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 10 client.9003 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 20 client.9003 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffe44ee0 0x5626ffe452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 4 client.9003 mds_command: new command op to 8796 tid=0[{"prefix": "scrub pause"}] 2023-11-06T03:14:10.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5626ffc022c0 con 0x5626ffe44ee0 2023-11-06T03:14:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffe44ee0 0x5626ffe452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb99d9b640 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffe44ee0 0x5626ffe452f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b33dd7147738cf server_cookie=9cdce95c634836af in_seq=0 out_seq=0 2023-11-06T03:14:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 10 client.9003 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 1 -- 172.21.15.157:0/2836611552 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5626ffc022c0 con 0x5626ffe44ee0 2023-11-06T03:14:10.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 10 client.9003 handle_command_reply: tid=0 2023-11-06T03:14:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 client.9003 shutdown 2023-11-06T03:14:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 msgr2=0x5626ffe60b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ad570 tx=0x7fdb8c01cbd0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 shutdown_connections 2023-11-06T03:14:10.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5626ffe049f0 0x5626ffe449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5626ffe44ee0 0x5626ffe452f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5626ffe040d0 0x5626ffe610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 --2- 172.21.15.157:0/2836611552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626ffdba200 0x5626ffe60b60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdb797fa640 0 client.9003 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 >> 172.21.15.157:0/2836611552 conn(0x5626ffdc0990 msgr2=0x5626ffe0a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 shutdown_connections 2023-11-06T03:14:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 1 -- 172.21.15.157:0/2836611552 wait complete. 2023-11-06T03:14:10.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba3bc0000 20 client.9003 trim_cache size 0 max 16384 2023-11-06T03:14:10.116 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:10.117 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:14:10.117 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:10.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 msgr2=0x7fdb7c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 0x7fdb7c067f50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15cff0 tx=0x7fdb90009040 comp rx=0 tx=0).stop 2023-11-06T03:14:10.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 msgr2=0x7fdb9c158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c01c640 tx=0x7fdb84005800 comp rx=0 tx=0).stop 2023-11-06T03:14:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 shutdown_connections 2023-11-06T03:14:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdb7c065a90 0x7fdb7c067f50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb9c0adc10 0x7fdb9c15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb9c0b0610 0x7fdb9c0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 --2- 172.21.15.157:0/1270328333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb84003980 0x7fdb9c158940 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 >> 172.21.15.157:0/1270328333 conn(0x7fdb9c06a170 msgr2=0x7fdb9c0ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 shutdown_connections 2023-11-06T03:14:10.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.093+0000 7fdba0a98640 1 -- 172.21.15.157:0/1270328333 wait complete. 2023-11-06T03:14:10.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub status 2023-11-06T03:14:10.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.233+0000 7f4ff8e17640 1 Processor -- start 2023-11-06T03:14:10.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.233+0000 7f4ff8e17640 1 -- start start 2023-11-06T03:14:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff8e17640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4ff41599c0 0x7f4ff40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff8e17640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff406d000 0x7f4ff40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff8e17640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff415e440 con 0x7f4ff415dea0 2023-11-06T03:14:10.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff8e17640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff415adb0 con 0x7f4ff406d000 2023-11-06T03:14:10.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff8e17640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff406c240 con 0x7f4ff41599c0 2023-11-06T03:14:10.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff406d000 0x7f4ff40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ff406d000 0x7f4ff40b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39942/0 (socket says 172.21.15.157:39942) 2023-11-06T03:14:10.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 -- 172.21.15.157:0/1636547054 learned_addr learned my addr 172.21.15.157:0/1636547054 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff259c640 1 --2- 172.21.15.157:0/1636547054 >> v2:172.21.15.195:3300/0 conn(0x7f4ff41599c0 0x7f4ff40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 -- 172.21.15.157:0/1636547054 >> v2:172.21.15.195:3300/0 conn(0x7f4ff41599c0 msgr2=0x7f4ff40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/1636547054 >> v2:172.21.15.195:3300/0 conn(0x7f4ff41599c0 0x7f4ff40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 -- 172.21.15.157:0/1636547054 >> v1:172.21.15.195:6789/0 conn(0x7f4ff415dea0 legacy=0x7f4ff4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 -- 172.21.15.157:0/1636547054 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff406c520 con 0x7f4ff406d000 2023-11-06T03:14:10.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/1636547054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40b23e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4066800 tx=0x7f4fe801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d2432b1329f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff159a640 1 -- 172.21.15.157:0/1636547054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80ac600 con 0x7f4ff406d000 2023-11-06T03:14:10.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff159a640 1 -- 172.21.15.157:0/1636547054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe80a8050 con 0x7f4ff406d000 2023-11-06T03:14:10.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.237+0000 7f4ff159a640 1 -- 172.21.15.157:0/1636547054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80ac980 con 0x7f4ff406d000 2023-11-06T03:14:10.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/1636547054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 msgr2=0x7f4ff40b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/1636547054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40b23e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4066800 tx=0x7f4fe801c490 comp rx=0 tx=0).stop 2023-11-06T03:14:10.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/1636547054 shutdown_connections 2023-11-06T03:14:10.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/1636547054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40b23e0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/1636547054 >> v2:172.21.15.195:3300/0 conn(0x7f4ff41599c0 0x7f4ff40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/1636547054 >> 172.21.15.157:0/1636547054 conn(0x7f4ff4065930 msgr2=0x7f4ff4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/1636547054 shutdown_connections 2023-11-06T03:14:10.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/1636547054 wait complete. 2023-11-06T03:14:10.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 Processor -- start 2023-11-06T03:14:10.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- start start 2023-11-06T03:14:10.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ff41599c0 0x7f4ff414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 0x7f4ff415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff414d550 con 0x7f4ff406d000 2023-11-06T03:14:10.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415c730 con 0x7f4ff415dea0 2023-11-06T03:14:10.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415c170 con 0x7f4ff41599c0 2023-11-06T03:14:10.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39948/0 (socket says 172.21.15.157:39948) 2023-11-06T03:14:10.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 -- 172.21.15.157:0/3775676718 learned_addr learned my addr 172.21.15.157:0/3775676718 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 0x7f4ff415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 -- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ff41599c0 msgr2=0x7f4ff414b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ff41599c0 0x7f4ff414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ff41599c0 0x7f4ff414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 -- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 msgr2=0x7f4ff415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 0x7f4ff415b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 -- 172.21.15.157:0/3775676718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80094d0 con 0x7f4ff406d000 2023-11-06T03:14:10.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 0x7f4ff415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:10.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c7f0 tx=0x7f4fe400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4fe2ffd640 1 -- 172.21.15.157:0/3775676718 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4004220 con 0x7f4ff406d000 2023-11-06T03:14:10.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4fe2ffd640 1 -- 172.21.15.157:0/3775676718 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe40043a0 con 0x7f4ff406d000 2023-11-06T03:14:10.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff415c450 con 0x7f4ff406d000 2023-11-06T03:14:10.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4fe2ffd640 1 -- 172.21.15.157:0/3775676718 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4004510 con 0x7f4ff406d000 2023-11-06T03:14:10.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.241+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff414cc50 con 0x7f4ff406d000 2023-11-06T03:14:10.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fe2ffd640 1 -- 172.21.15.157:0/3775676718 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4fe4003a70 con 0x7f4ff406d000 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fe2ffd640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 0x7f4fd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fe2ffd640 1 -- 172.21.15.157:0/3775676718 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fe4083390 con 0x7f4ff406d000 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 0x7f4fd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 Processor -- start 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 -- start start 2023-11-06T03:14:10.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 0x7f4fd4067f20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ae000 tx=0x7f4fe8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af80510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af35d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91adfdc10 con 0x55e91af36260 2023-11-06T03:14:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad8c3e0 con 0x55e91af80130 2023-11-06T03:14:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad899b0 con 0x55e91af80a50 2023-11-06T03:14:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39964/0 (socket says 172.21.15.157:39964) 2023-11-06T03:14:10.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 -- 172.21.15.157:0/487595433 learned_addr learned my addr 172.21.15.157:0/487595433 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af35d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff259c640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af80510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 -- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 msgr2=0x55e91af35d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af35d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 -- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 msgr2=0x55e91af80510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af80510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 -- 172.21.15.157:0/487595433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91aa81e20 con 0x55e91af36260 2023-11-06T03:14:10.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff259c640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af80510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:10.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4fec007f10 tx=0x7f4fec00c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8df87ccf31e86cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fd3fff640 1 -- 172.21.15.157:0/487595433 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00f830 con 0x55e91af36260 2023-11-06T03:14:10.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fd3fff640 1 -- 172.21.15.157:0/487595433 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec00f990 con 0x55e91af36260 2023-11-06T03:14:10.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.245+0000 7f4fd3fff640 1 -- 172.21.15.157:0/487595433 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00fd20 con 0x55e91af36260 2023-11-06T03:14:10.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/487595433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 msgr2=0x55e91af36640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4fec007f10 tx=0x7f4fec00c670 comp rx=0 tx=0).stop 2023-11-06T03:14:10.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/487595433 shutdown_connections 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af36640 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af35d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/487595433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af80510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/487595433 >> 172.21.15.157:0/487595433 conn(0x55e91af3c9f0 msgr2=0x55e91af34600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/487595433 shutdown_connections 2023-11-06T03:14:10.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/487595433 wait complete. 2023-11-06T03:14:10.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- start start 2023-11-06T03:14:10.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 Processor -- start 2023-11-06T03:14:10.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80130 0x55e91af80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80a50 0x55e91af81370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad852a0 con 0x55e91af36260 2023-11-06T03:14:10.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad24470 con 0x55e91af80a50 2023-11-06T03:14:10.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad42660 con 0x55e91af80130 2023-11-06T03:14:10.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 10 client.0 _ll_register_callbacks cb 0x55e91af21c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:10.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39974/0 (socket says 172.21.15.157:39974) 2023-11-06T03:14:10.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 -- 172.21.15.157:0/2470388450 learned_addr learned my addr 172.21.15.157:0/2470388450 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 -- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80130 msgr2=0x55e91af80e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80130 0x55e91af80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff2d9d640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80a50 0x55e91af81370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80130 0x55e91af80e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 -- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80a50 msgr2=0x55e91af81370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80a50 0x55e91af81370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 -- 172.21.15.157:0/2470388450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec007b40 con 0x55e91af36260 2023-11-06T03:14:10.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4fe405c2c0 tx=0x7f4fe4058640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6315bfcc457082d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:10.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4052c70 con 0x55e91af36260 2023-11-06T03:14:10.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fe405c360 con 0x55e91af36260 2023-11-06T03:14:10.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e91ad420a0 con 0x55e91af36260 2023-11-06T03:14:10.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe405c520 con 0x55e91af36260 2023-11-06T03:14:10.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 10 client.9015 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55e91ad4a1d0 con 0x55e91af36260 2023-11-06T03:14:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e91ad420a0 con 0x55e91af36260 2023-11-06T03:14:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4fe4081250 con 0x55e91af36260 2023-11-06T03:14:10.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 10 client.9015 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:10.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 10 client.9015 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:10.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 20 client.9015 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:10.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91af81d80 0x55e91af82160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 4 client.9015 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:10.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e91ad83bf0 con 0x55e91af81d80 2023-11-06T03:14:10.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91af81d80 0x55e91af82160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91af81d80 0x55e91af82160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1eac5869127bec server_cookie=ae35190c072d215a in_seq=0 out_seq=0 2023-11-06T03:14:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 10 client.9015 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 1 -- 172.21.15.157:0/2470388450 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e91ad83bf0 con 0x55e91af81d80 2023-11-06T03:14:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4fd17fa640 10 client.9015 handle_command_reply: tid=0 2023-11-06T03:14:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 client.9015 shutdown 2023-11-06T03:14:10.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 msgr2=0x55e91af7f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.249+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4fe405c2c0 tx=0x7f4fe4058640 comp rx=0 tx=0).stop 2023-11-06T03:14:10.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 shutdown_connections 2023-11-06T03:14:10.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91af81d80 0x55e91af82160 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80a50 0x55e91af81370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80130 0x55e91af80e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4fd17fa640 0 client.9015 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2470388450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7f5b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 >> 172.21.15.157:0/2470388450 conn(0x55e91af3c9f0 msgr2=0x55e91af34d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 shutdown_connections 2023-11-06T03:14:10.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2470388450 wait complete. 2023-11-06T03:14:10.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.253+0000 7f4ffbf3f000 20 client.9015 trim_cache size 0 max 16384 2023-11-06T03:14:10.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 Processor -- start 2023-11-06T03:14:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 -- start start 2023-11-06T03:14:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af7ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af84170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91aa42aa0 con 0x55e91af36260 2023-11-06T03:14:10.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad82ab0 con 0x55e91af80130 2023-11-06T03:14:10.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91af2f080 con 0x55e91af80a50 2023-11-06T03:14:10.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39976/0 (socket says 172.21.15.157:39976) 2023-11-06T03:14:10.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 -- 172.21.15.157:0/2524381629 learned_addr learned my addr 172.21.15.157:0/2524381629 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 -- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 msgr2=0x55e91af84170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 -- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 msgr2=0x55e91af7ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af7ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 -- 172.21.15.157:0/2524381629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91ad30520 con 0x55e91af36260 2023-11-06T03:14:10.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4ff259c640 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40547a0 tx=0x7f4fe4083050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dcc2a5276d5caf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4fe0ff9640 1 -- 172.21.15.157:0/2524381629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4054f80 con 0x55e91af36260 2023-11-06T03:14:10.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4fe0ff9640 1 -- 172.21.15.157:0/2524381629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4050b50 con 0x55e91af36260 2023-11-06T03:14:10.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.265+0000 7f4fe0ff9640 1 -- 172.21.15.157:0/2524381629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4052c70 con 0x55e91af36260 2023-11-06T03:14:10.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2524381629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 msgr2=0x55e91af7e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40547a0 tx=0x7f4fe4083050 comp rx=0 tx=0).stop 2023-11-06T03:14:10.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2524381629 shutdown_connections 2023-11-06T03:14:10.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91af84170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91af7ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/2524381629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91af7e720 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2524381629 >> 172.21.15.157:0/2524381629 conn(0x55e91af3c9f0 msgr2=0x55e91af3aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2524381629 shutdown_connections 2023-11-06T03:14:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/2524381629 wait complete. 2023-11-06T03:14:10.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- start start 2023-11-06T03:14:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 Processor -- start 2023-11-06T03:14:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91afdd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91afd9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad43400 con 0x55e91af36260 2023-11-06T03:14:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad2a7a0 con 0x55e91af80130 2023-11-06T03:14:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91ad7d840 con 0x55e91af80a50 2023-11-06T03:14:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 10 client.0 _ll_register_callbacks cb 0x55e91ad49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39990/0 (socket says 172.21.15.157:39990) 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 -- 172.21.15.157:0/3462940198 learned_addr learned my addr 172.21.15.157:0/3462940198 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 -- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 msgr2=0x55e91afd9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91afd9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 -- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 msgr2=0x55e91afdd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91afdd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 -- 172.21.15.157:0/3462940198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e91ad4a1d0 con 0x55e91af36260 2023-11-06T03:14:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff259c640 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007410 tx=0x7f4fe4081590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aa783692ce10f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4052c70 con 0x55e91af36260 2023-11-06T03:14:10.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fe4054f10 con 0x55e91af36260 2023-11-06T03:14:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e91ad4e620 con 0x55e91af36260 2023-11-06T03:14:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fe4081250 con 0x55e91af36260 2023-11-06T03:14:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 10 client.9021 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55e91ad24090 con 0x55e91af36260 2023-11-06T03:14:10.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e91ad4e620 con 0x55e91af36260 2023-11-06T03:14:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4fe4078020 con 0x55e91af36260 2023-11-06T03:14:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 10 client.9021 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 10 client.9021 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 20 client.9021 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:10.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91afd9e80 0x55e91afda290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 4 client.9021 mds_command: new command op to 8796 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:14:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e91ad7e320 con 0x55e91afd9e80 2023-11-06T03:14:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91afd9e80 0x55e91afda290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ff1d9b640 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91afd9e80 0x55e91afda290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e70e79333582cc server_cookie=45eacc835c87752e in_seq=0 out_seq=0 2023-11-06T03:14:10.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 10 client.9021 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 1 -- 172.21.15.157:0/3462940198 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e91ad7e320 con 0x55e91afd9e80 2023-11-06T03:14:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 10 client.9021 handle_command_reply: tid=0 2023-11-06T03:14:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 client.9021 shutdown 2023-11-06T03:14:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 msgr2=0x55e91afdd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4007410 tx=0x7f4fe4081590 comp rx=0 tx=0).stop 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 shutdown_connections 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e91af80a50 0x55e91afd9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55e91afd9e80 0x55e91afda290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e91af80130 0x55e91afdd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4fd17fa640 0 client.9021 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:10.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 --2- 172.21.15.157:0/3462940198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e91af36260 0x55e91afdd310 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 >> 172.21.15.157:0/3462940198 conn(0x55e91af3c9f0 msgr2=0x55e91af3aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 shutdown_connections 2023-11-06T03:14:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 1 -- 172.21.15.157:0/3462940198 wait complete. 2023-11-06T03:14:10.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.269+0000 7f4ffbf3f000 20 client.9021 trim_cache size 0 max 16384 2023-11-06T03:14:10.292 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:10.292 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-11-06T03:14:10.292 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:14:10.292 INFO:teuthology.orchestra.run.smithi157.stdout: "dfb8ccf2-e836-44cd-a854-d8a1428b7a2c": { 2023-11-06T03:14:10.293 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_control_test_path", 2023-11-06T03:14:10.293 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "dfb8ccf2-e836-44cd-a854-d8a1428b7a2c", 2023-11-06T03:14:10.293 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:14:10.293 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:10.293 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:10.294 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 msgr2=0x7f4fd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 0x7f4fd4067f20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ae000 tx=0x7f4fe8008040 comp rx=0 tx=0).stop 2023-11-06T03:14:10.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 msgr2=0x7f4ff40acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c7f0 tx=0x7f4fe400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 shutdown_connections 2023-11-06T03:14:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fd4065a60 0x7f4fd4067f20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff415dea0 0x7f4ff415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4ff41599c0 0x7f4ff414b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 --2- 172.21.15.157:0/3775676718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ff406d000 0x7f4ff40acb10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 >> 172.21.15.157:0/3775676718 conn(0x7f4ff4065930 msgr2=0x7f4ff4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 shutdown_connections 2023-11-06T03:14:10.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.273+0000 7f4ff8e17640 1 -- 172.21.15.157:0/3775676718 wait complete. 2023-11-06T03:14:10.298 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:10.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 Processor -- start 2023-11-06T03:14:10.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 -- start start 2023-11-06T03:14:10.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa4b406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b415f580 con 0x7fa4b40b3b20 2023-11-06T03:14:10.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b415efc0 con 0x7fa4b415e5e0 2023-11-06T03:14:10.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4bb20e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b415f2a0 con 0x7fa4b41551e0 2023-11-06T03:14:10.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa4b40b3b20 0x7fa4b415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58210/0 (socket says 172.21.15.157:58210) 2023-11-06T03:14:10.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 -- 172.21.15.157:0/4029054417 learned_addr learned my addr 172.21.15.157:0/4029054417 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:10.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b8faa640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa4b406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607041366 0 0) 0x7fa4b415f580 con 0x7fa4b40b3b20 2023-11-06T03:14:10.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa49c003530 con 0x7fa4b40b3b20 2023-11-06T03:14:10.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2118506053 0 0) 0x7fa49c003530 con 0x7fa4b40b3b20 2023-11-06T03:14:10.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 msgr2=0x7fa4b406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa4b406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 >> v1:172.21.15.195:6789/0 conn(0x7fa4b415e5e0 legacy=0x7fa4b40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b406f5a0 con 0x7fa4b40b3b20 2023-11-06T03:14:10.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa4ac001a00 con 0x7fa4b40b3b20 2023-11-06T03:14:10.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 >> v1:172.21.15.157:6789/0 conn(0x7fa4b40b3b20 legacy=0x7fa4b415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa49c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa49c003f30 0x7fa49c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49c004d10 0x7fa49c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b415f2a0 con 0x7fa49c004d10 2023-11-06T03:14:10.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c00bb20 con 0x7fa49c003f30 2023-11-06T03:14:10.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c00bce0 con 0x7fa4b41551e0 2023-11-06T03:14:10.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49c004d10 0x7fa49c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4abfff640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa49c003f30 0x7fa49c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b8faa640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa49c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 -- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 msgr2=0x7fa49c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b41551e0 0x7fa49c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 -- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa49c003f30 msgr2=0x7fa49c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa49c003f30 0x7fa49c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.409+0000 7fa4b97ab640 1 -- 172.21.15.157:0/4029054417 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c003530 con 0x7fa49c004d10 2023-11-06T03:14:10.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b97ab640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49c004d10 0x7fa49c005190 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0013e0 tx=0x7fa4ac01fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221fd58927797ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0aa670 con 0x7fa49c004d10 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac0aa7d0 con 0x7fa49c004d10 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4ab7fe640 1 -- 172.21.15.157:0/4029054417 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0aabc0 con 0x7fa49c004d10 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- 172.21.15.157:0/4029054417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49c004d10 msgr2=0x7fa49c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/4029054417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa49c004d10 0x7fa49c005190 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0013e0 tx=0x7fa4ac01fa40 comp rx=0 tx=0).stop 2023-11-06T03:14:10.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b97ab640 1 -- 172.21.15.157:0/4029054417 reap_dead start 2023-11-06T03:14:10.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- 172.21.15.157:0/4029054417 shutdown_connections 2023-11-06T03:14:10.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- 172.21.15.157:0/4029054417 >> 172.21.15.157:0/4029054417 conn(0x7fa4b4065930 msgr2=0x7fa4b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- 172.21.15.157:0/4029054417 shutdown_connections 2023-11-06T03:14:10.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- 172.21.15.157:0/4029054417 wait complete. 2023-11-06T03:14:10.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 Processor -- start 2023-11-06T03:14:10.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- start start 2023-11-06T03:14:10.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b414f2e0 0x7fa4b40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4b414f760 0x7fa4b40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b414d670 con 0x7fa4b4138680 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b414de80 con 0x7fa4b414f760 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4bb20e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b41f4ba0 con 0x7fa4b414f2e0 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40012/0 (socket says 172.21.15.157:40012) 2023-11-06T03:14:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 -- 172.21.15.157:0/1356760409 learned_addr learned my addr 172.21.15.157:0/1356760409 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b97ab640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4b414f760 0x7fa4b40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4abfff640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b414f2e0 0x7fa4b40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 -- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b414f2e0 msgr2=0x7fa4b40cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b414f2e0 0x7fa4b40cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 -- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4b414f760 msgr2=0x7fa4b40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4b414f760 0x7fa4b40cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 -- 172.21.15.157:0/1356760409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b40cf0c0 con 0x7fa4b4138680 2023-11-06T03:14:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4b8faa640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00115e0 tx=0x7fa4b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.413+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000f8f0 con 0x7fa4b4138680 2023-11-06T03:14:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b000fa50 con 0x7fa4b4138680 2023-11-06T03:14:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b414d8c0 con 0x7fa4b4138680 2023-11-06T03:14:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000fdc0 con 0x7fa4b4138680 2023-11-06T03:14:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b413ec70 con 0x7fa4b4138680 2023-11-06T03:14:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b4072750 con 0x7fa4b4138680 2023-11-06T03:14:10.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4b0015460 con 0x7fa4b4138680 2023-11-06T03:14:10.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4a97fa640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 0x7fa4a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4b0083740 con 0x7fa4b4138680 2023-11-06T03:14:10.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4abfff640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 0x7fa4a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.417+0000 7fa4abfff640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 0x7fa4a0067f20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40b2710 tx=0x7fa4a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.421+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa4b0051030 con 0x7fa4b4138680 2023-11-06T03:14:10.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.593+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4b413aee0 con 0x7fa4b4138680 2023-11-06T03:14:10.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.593+0000 7fa4a97fa640 1 -- 172.21.15.157:0/1356760409 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fa4b0054650 con 0x7fa4b4138680 2023-11-06T03:14:10.592 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:10.592 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:10.592 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:10.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 msgr2=0x7fa4a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 0x7fa4a0067f20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40b2710 tx=0x7fa4a4009040 comp rx=0 tx=0).stop 2023-11-06T03:14:10.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 msgr2=0x7fa4b41f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00115e0 tx=0x7fa4b0011aa0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 shutdown_connections 2023-11-06T03:14:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa4a0065a60 0x7fa4a0067f20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4b414f760 0x7fa4b40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4b414f2e0 0x7fa4b40cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 --2- 172.21.15.157:0/1356760409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4b4138680 0x7fa4b41f4660 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 >> 172.21.15.157:0/1356760409 conn(0x7fa4b4065930 msgr2=0x7fa4b40b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 shutdown_connections 2023-11-06T03:14:10.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.597+0000 7fa4bb20e640 1 -- 172.21.15.157:0/1356760409 wait complete. 2023-11-06T03:14:10.609 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:10.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:10.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 Processor -- start 2023-11-06T03:14:10.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 -- start start 2023-11-06T03:14:10.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc7341551e0 0x7fc73406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73415f580 con 0x7fc7340b3b20 2023-11-06T03:14:10.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73415efc0 con 0x7fc73415e5e0 2023-11-06T03:14:10.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73c391640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73415f2a0 con 0x7fc7341551e0 2023-11-06T03:14:10.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73a92e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc7340b3b20 0x7fc73415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58224/0 (socket says 172.21.15.157:58224) 2023-11-06T03:14:10.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73a92e640 1 -- 172.21.15.157:0/1037281450 learned_addr learned my addr 172.21.15.157:0/1037281450 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:10.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.729+0000 7fc73a12d640 1 --2- 172.21.15.157:0/1037281450 >> v2:172.21.15.157:3300/0 conn(0x7fc7341551e0 0x7fc73406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73a12d640 1 -- 172.21.15.157:0/1037281450 >> v1:172.21.15.195:6789/0 conn(0x7fc73415e5e0 legacy=0x7fc7340b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:10.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73a12d640 1 -- 172.21.15.157:0/1037281450 >> v1:172.21.15.157:6789/0 conn(0x7fc7340b3b20 legacy=0x7fc73415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73a12d640 1 -- 172.21.15.157:0/1037281450 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73406f5a0 con 0x7fc7341551e0 2023-11-06T03:14:10.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73912b640 1 -- 172.21.15.157:0/1037281450 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084800455 0 0) 0x7fc73415f580 con 0x7fc7340b3b20 2023-11-06T03:14:10.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73a12d640 1 --2- 172.21.15.157:0/1037281450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7341551e0 0x7fc73406fda0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fc7300097c0 tx=0x7fc73001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3cc383ab9d8036 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73912b640 1 -- 172.21.15.157:0/1037281450 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730004230 con 0x7fc7341551e0 2023-11-06T03:14:10.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73912b640 1 -- 172.21.15.157:0/1037281450 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7300a7050 con 0x7fc7341551e0 2023-11-06T03:14:10.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73912b640 1 -- 172.21.15.157:0/1037281450 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730003620 con 0x7fc7341551e0 2023-11-06T03:14:10.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 -- 172.21.15.157:0/1037281450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7341551e0 msgr2=0x7fc73406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 --2- 172.21.15.157:0/1037281450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7341551e0 0x7fc73406fda0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fc7300097c0 tx=0x7fc73001c330 comp rx=0 tx=0).stop 2023-11-06T03:14:10.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 -- 172.21.15.157:0/1037281450 shutdown_connections 2023-11-06T03:14:10.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 --2- 172.21.15.157:0/1037281450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7341551e0 0x7fc73406fda0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 -- 172.21.15.157:0/1037281450 >> 172.21.15.157:0/1037281450 conn(0x7fc734065930 msgr2=0x7fc734065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 -- 172.21.15.157:0/1037281450 shutdown_connections 2023-11-06T03:14:10.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.733+0000 7fc73c391640 1 -- 172.21.15.157:0/1037281450 wait complete. 2023-11-06T03:14:10.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 Processor -- start 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- start start 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7340b3b20 0x7fc734151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7341551e0 0x7fc73414b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340ae320 con 0x7fc73415e5e0 2023-11-06T03:14:10.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340add60 con 0x7fc7341551e0 2023-11-06T03:14:10.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340ae040 con 0x7fc7340b3b20 2023-11-06T03:14:10.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40038/0 (socket says 172.21.15.157:40038) 2023-11-06T03:14:10.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 -- 172.21.15.157:0/2875020431 learned_addr learned my addr 172.21.15.157:0/2875020431 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:10.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73992c640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7341551e0 0x7fc73414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 -- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7340b3b20 msgr2=0x7fc734151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7340b3b20 0x7fc734151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 -- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7341551e0 msgr2=0x7fc73414b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7341551e0 0x7fc73414b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 -- 172.21.15.157:0/2875020431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009420 con 0x7fc73415e5e0 2023-11-06T03:14:10.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73a92e640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001570 tx=0x7fc72c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00a640 con 0x7fc73415e5e0 2023-11-06T03:14:10.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00a7c0 con 0x7fc73415e5e0 2023-11-06T03:14:10.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00ab70 con 0x7fc73415e5e0 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc734153890 con 0x7fc73415e5e0 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73414bcb0 con 0x7fc73415e5e0 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7341536f0 con 0x7fc73415e5e0 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.737+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc72c00acd0 con 0x7fc73415e5e0 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.741+0000 7fc72affd640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 0x7fc718067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:10.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.741+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc72c0828d0 con 0x7fc73415e5e0 2023-11-06T03:14:10.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.741+0000 7fc73a12d640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 0x7fc718067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:10.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.741+0000 7fc73a12d640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 0x7fc718067f20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc7300ad000 tx=0x7fc730004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:10.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.745+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc7341536f0 con 0x7fc73415e5e0 2023-11-06T03:14:10.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.953+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc734153db0 con 0x7fc73415e5e0 2023-11-06T03:14:10.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.953+0000 7fc72affd640 1 -- 172.21.15.157:0/2875020431 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1576 (secure 0 0 0) 0x7fc734153db0 con 0x7fc73415e5e0 2023-11-06T03:14:10.952 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:10.952 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":628,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1021,"data_bytes":1048572774,"bytes_used":2677878784,"bytes_avail":770416234496,"bytes_total":773094113280,"write_bytes_sec":78044739,"read_op_per_sec":0,"write_op_per_sec":75},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-11-06T03:13:23.298350+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:10.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 msgr2=0x7fc718067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 0x7fc718067f20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc7300ad000 tx=0x7fc730004740 comp rx=0 tx=0).stop 2023-11-06T03:14:10.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 msgr2=0x7fc7340cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:10.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001570 tx=0x7fc72c00cca0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 shutdown_connections 2023-11-06T03:14:10.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc718065a60 0x7fc718067f20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc73415e5e0 0x7fc7340cdbc0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7341551e0 0x7fc73414b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 --2- 172.21.15.157:0/2875020431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7340b3b20 0x7fc734151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:10.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 >> 172.21.15.157:0/2875020431 conn(0x7fc734065930 msgr2=0x7fc7340aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:10.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 shutdown_connections 2023-11-06T03:14:10.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:10.957+0000 7fc73c391640 1 -- 172.21.15.157:0/2875020431 wait complete. 2023-11-06T03:14:10.978 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:14:11.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:14:11.662 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:14:11.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:12.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.113+0000 7f70cb4a6640 1 Processor -- start 2023-11-06T03:14:12.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.113+0000 7f70cb4a6640 1 -- start start 2023-11-06T03:14:12.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c41551e0 0x7f70c406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 0x7f70c40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70c415ea60 0x7f70c40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c415f580 con 0x7f70c415ea60 2023-11-06T03:14:12.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c415efc0 con 0x7f70c415e5e0 2023-11-06T03:14:12.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70cb4a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c415f2a0 con 0x7f70c41551e0 2023-11-06T03:14:12.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70c415ea60 0x7f70c40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f70c415ea60 0x7f70c40b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40054/0 (socket says 172.21.15.157:40054) 2023-11-06T03:14:12.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 -- 172.21.15.157:0/1775513274 learned_addr learned my addr 172.21.15.157:0/1775513274 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9242640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c41551e0 0x7f70c406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c8a41640 1 --2- 172.21.15.157:0/1775513274 >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 0x7f70c40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 -- 172.21.15.157:0/1775513274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c41551e0 msgr2=0x7f70c406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c41551e0 0x7f70c406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 -- 172.21.15.157:0/1775513274 >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 msgr2=0x7f70c40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 --2- 172.21.15.157:0/1775513274 >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 0x7f70c40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 -- 172.21.15.157:0/1775513274 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c406f5a0 con 0x7f70c415ea60 2023-11-06T03:14:12.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c8a41640 1 --2- 172.21.15.157:0/1775513274 >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 0x7f70c40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:12.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c9a43640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c415ea60 0x7f70c40b3950 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f70c40668b0 tx=0x7f70bc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51e216c25d3aa5f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c3fff640 1 -- 172.21.15.157:0/1775513274 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0a57b0 con 0x7f70c415ea60 2023-11-06T03:14:12.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c3fff640 1 -- 172.21.15.157:0/1775513274 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc0a5910 con 0x7f70c415ea60 2023-11-06T03:14:12.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.117+0000 7f70c3fff640 1 -- 172.21.15.157:0/1775513274 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0a5ca0 con 0x7f70c415ea60 2023-11-06T03:14:12.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1775513274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c415ea60 msgr2=0x7f70c40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c415ea60 0x7f70c40b3950 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f70c40668b0 tx=0x7f70bc01c910 comp rx=0 tx=0).stop 2023-11-06T03:14:12.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1775513274 shutdown_connections 2023-11-06T03:14:12.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c415ea60 0x7f70c40b3950 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1775513274 >> v2:172.21.15.195:3300/0 conn(0x7f70c415e5e0 0x7f70c40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1775513274 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c41551e0 0x7f70c406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1775513274 >> 172.21.15.157:0/1775513274 conn(0x7f70c4065930 msgr2=0x7f70c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1775513274 shutdown_connections 2023-11-06T03:14:12.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1775513274 wait complete. 2023-11-06T03:14:12.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 Processor -- start 2023-11-06T03:14:12.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- start start 2023-11-06T03:14:12.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c415e5e0 0x7f70c40b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c415ea60 0x7f70c4151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40b1f20 con 0x7f70c41551e0 2023-11-06T03:14:12.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c4155dc0 con 0x7f70c415ea60 2023-11-06T03:14:12.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c41560a0 con 0x7f70c415e5e0 2023-11-06T03:14:12.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40058/0 (socket says 172.21.15.157:40058) 2023-11-06T03:14:12.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 -- 172.21.15.157:0/1421377375 learned_addr learned my addr 172.21.15.157:0/1421377375 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c8a41640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c415e5e0 0x7f70c40b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9a43640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c415ea60 0x7f70c4151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 -- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c415e5e0 msgr2=0x7f70c40b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c415e5e0 0x7f70c40b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 -- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c415ea60 msgr2=0x7f70c4151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c415ea60 0x7f70c4151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 -- 172.21.15.157:0/1421377375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc009b30 con 0x7f70c41551e0 2023-11-06T03:14:12.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c9242640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c7f0 tx=0x7f70b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4004220 con 0x7f70c41551e0 2023-11-06T03:14:12.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b4004380 con 0x7f70c41551e0 2023-11-06T03:14:12.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c4156380 con 0x7f70c41551e0 2023-11-06T03:14:12.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4003860 con 0x7f70c41551e0 2023-11-06T03:14:12.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.121+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c40b1c10 con 0x7f70c41551e0 2023-11-06T03:14:12.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c41568a0 con 0x7f70c41551e0 2023-11-06T03:14:12.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f70b4003bd0 con 0x7f70c41551e0 2023-11-06T03:14:12.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70c1ffb640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 0x7f70b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70b40832a0 con 0x7f70c41551e0 2023-11-06T03:14:12.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70c8a41640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 0x7f70b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.125+0000 7f70c8a41640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 0x7f70b0067f50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f70b800a9a0 tx=0x7f70b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.129+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70b4050a60 con 0x7f70c41551e0 2023-11-06T03:14:12.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.317+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70c4072750 con 0x7f70c41551e0 2023-11-06T03:14:12.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.317+0000 7f70c1ffb640 1 -- 172.21.15.157:0/1421377375 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f70b4054080 con 0x7f70c41551e0 2023-11-06T03:14:12.315 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:12.315 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:12.316 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:12.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 msgr2=0x7f70b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 0x7f70b0067f50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f70b800a9a0 tx=0x7f70b8008040 comp rx=0 tx=0).stop 2023-11-06T03:14:12.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 msgr2=0x7f70c4156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f70b400c7f0 tx=0x7f70b400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 shutdown_connections 2023-11-06T03:14:12.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f70b0065a90 0x7f70b0067f50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70c415ea60 0x7f70c4151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70c415e5e0 0x7f70c40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 --2- 172.21.15.157:0/1421377375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70c41551e0 0x7f70c4156b80 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 >> 172.21.15.157:0/1421377375 conn(0x7f70c4065930 msgr2=0x7f70c40af030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 shutdown_connections 2023-11-06T03:14:12.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.321+0000 7f70cb4a6640 1 -- 172.21.15.157:0/1421377375 wait complete. 2023-11-06T03:14:12.334 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:12.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:12.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 Processor -- start 2023-11-06T03:14:12.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 -- start start 2023-11-06T03:14:12.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ea815a700 0x7f3ea80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea8066040 0x7f3ea815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea80b2020 con 0x7f3ea815aae0 2023-11-06T03:14:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea80b2840 con 0x7f3ea815a700 2023-11-06T03:14:12.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.453+0000 7f3eb02b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80cc220 con 0x7f3ea8066040 2023-11-06T03:14:12.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3ea815a700 0x7f3ea80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3ead84b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f3ea815aae0 0x7f3ea8065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:57558/0 (socket says 172.21.15.157:57558) 2023-11-06T03:14:12.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae84d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea8066040 0x7f3ea815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3ead84b640 1 -- 172.21.15.157:0/277455781 learned_addr learned my addr 172.21.15.157:0/277455781 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:12.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 -- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea8066040 msgr2=0x7f3ea815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 --2- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea8066040 0x7f3ea815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 -- 172.21.15.157:0/277455781 >> v1:172.21.15.195:6789/0 conn(0x7f3ea815aae0 legacy=0x7f3ea8065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 -- 172.21.15.157:0/277455781 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea815def0 con 0x7f3ea815a700 2023-11-06T03:14:12.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3eae04c640 1 --2- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea80ccc90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097c0 tx=0x7f3ea401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa3f5e25a3ac79c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3ead04a640 1 -- 172.21.15.157:0/277455781 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40ab670 con 0x7f3ea815a700 2023-11-06T03:14:12.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3ead04a640 1 -- 172.21.15.157:0/277455781 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea40ab7d0 con 0x7f3ea815a700 2023-11-06T03:14:12.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.457+0000 7f3ead04a640 1 -- 172.21.15.157:0/277455781 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40abb60 con 0x7f3ea815a700 2023-11-06T03:14:12.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 msgr2=0x7f3ea80ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea80ccc90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40097c0 tx=0x7f3ea401c450 comp rx=0 tx=0).stop 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/277455781 shutdown_connections 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea8066040 0x7f3ea815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/277455781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea80ccc90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/277455781 >> 172.21.15.157:0/277455781 conn(0x7f3ea806a170 msgr2=0x7f3ea806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/277455781 shutdown_connections 2023-11-06T03:14:12.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/277455781 wait complete. 2023-11-06T03:14:12.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 Processor -- start 2023-11-06T03:14:12.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- start start 2023-11-06T03:14:12.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea8156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea815aae0 0x7f3ea81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81f5fe0 con 0x7f3ea8066040 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81f5360 con 0x7f3ea815a700 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81f5640 con 0x7f3ea815aae0 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40060/0 (socket says 172.21.15.157:40060) 2023-11-06T03:14:12.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 -- 172.21.15.157:0/3478351124 learned_addr learned my addr 172.21.15.157:0/3478351124 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3ead84b640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea8156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae84d640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea815aae0 0x7f3ea81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 -- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea815aae0 msgr2=0x7f3ea81570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea815aae0 0x7f3ea81570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 -- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 msgr2=0x7f3ea8156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea8156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 -- 172.21.15.157:0/3478351124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009420 con 0x7f3ea8066040 2023-11-06T03:14:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eae04c640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3ea401ccf0 tx=0x7f3ea401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40b5810 con 0x7f3ea8066040 2023-11-06T03:14:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea40b5970 con 0x7f3ea8066040 2023-11-06T03:14:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea81f5920 con 0x7f3ea8066040 2023-11-06T03:14:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.461+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40b5ce0 con 0x7f3ea8066040 2023-11-06T03:14:12.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea80c9070 con 0x7f3ea8066040 2023-11-06T03:14:12.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea81f5780 con 0x7f3ea8066040 2023-11-06T03:14:12.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3ea40c83f0 con 0x7f3ea8066040 2023-11-06T03:14:12.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3e9effd640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 0x7f3e8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ea411e2c0 con 0x7f3ea8066040 2023-11-06T03:14:12.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3ead84b640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 0x7f3e8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.465+0000 7f3ead84b640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 0x7f3e8c067f20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000f70 tx=0x7f3e98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.469+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3ea81f5780 con 0x7f3ea8066040 2023-11-06T03:14:12.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.681+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3ea81f6120 con 0x7f3ea8066040 2023-11-06T03:14:12.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3e9effd640 1 -- 172.21.15.157:0/3478351124 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1632 (secure 0 0 0) 0x7f3ea81f6120 con 0x7f3ea8066040 2023-11-06T03:14:12.682 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:12.683 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1021,"data_bytes":1048572774,"bytes_used":2677878784,"bytes_avail":770416234496,"bytes_total":773094113280,"write_bytes_sec":52939015,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-11-06T03:14:11.316119+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:12.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 msgr2=0x7f3e8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 0x7f3e8c067f20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3e98000f70 tx=0x7f3e98009040 comp rx=0 tx=0).stop 2023-11-06T03:14:12.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 msgr2=0x7f3ea8152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3ea401ccf0 tx=0x7f3ea401ac90 comp rx=0 tx=0).stop 2023-11-06T03:14:12.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 shutdown_connections 2023-11-06T03:14:12.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3e8c065a60 0x7f3e8c067f20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ea815aae0 0x7f3ea81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ea815a700 0x7f3ea8156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 --2- 172.21.15.157:0/3478351124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea8066040 0x7f3ea8152f50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 >> 172.21.15.157:0/3478351124 conn(0x7f3ea806a170 msgr2=0x7f3ea80cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 shutdown_connections 2023-11-06T03:14:12.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.685+0000 7f3eb02b0640 1 -- 172.21.15.157:0/3478351124 wait complete. 2023-11-06T03:14:12.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:14:12.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub abort 2023-11-06T03:14:12.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 Processor -- start 2023-11-06T03:14:12.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 -- start start 2023-11-06T03:14:12.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc18406ba10 0x7fc1840b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc184160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc184071d30 con 0x7fc18415cd60 2023-11-06T03:14:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840b23c0 con 0x7fc18406ba10 2023-11-06T03:14:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc189114640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18415d140 con 0x7fc1840713f0 2023-11-06T03:14:12.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc18406ba10 0x7fc1840b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc18406ba10 0x7fc1840b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40064/0 (socket says 172.21.15.157:40064) 2023-11-06T03:14:12.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 -- 172.21.15.157:0/3086817809 learned_addr learned my addr 172.21.15.157:0/3086817809 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 -- 172.21.15.157:0/3086817809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 msgr2=0x7fc184160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18359e640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc184160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc184160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 -- 172.21.15.157:0/3086817809 >> v1:172.21.15.195:6789/0 conn(0x7fc18415cd60 legacy=0x7fc18406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18259c640 1 -- 172.21.15.157:0/3086817809 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184071f80 con 0x7fc18406ba10 2023-11-06T03:14:12.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.825+0000 7fc18359e640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc184160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:12.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc18259c640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18406ba10 0x7fc1840b2e10 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc178009e50 tx=0x7fc17801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c62110c6314bf11 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc181d9b640 1 -- 172.21.15.157:0/3086817809 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780af400 con 0x7fc18406ba10 2023-11-06T03:14:12.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc181d9b640 1 -- 172.21.15.157:0/3086817809 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1780ab050 con 0x7fc18406ba10 2023-11-06T03:14:12.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc181d9b640 1 -- 172.21.15.157:0/3086817809 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780af780 con 0x7fc18406ba10 2023-11-06T03:14:12.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- 172.21.15.157:0/3086817809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18406ba10 msgr2=0x7fc1840b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18406ba10 0x7fc1840b2e10 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc178009e50 tx=0x7fc17801f1f0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- 172.21.15.157:0/3086817809 shutdown_connections 2023-11-06T03:14:12.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc184160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- 172.21.15.157:0/3086817809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18406ba10 0x7fc1840b2e10 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- 172.21.15.157:0/3086817809 >> 172.21.15.157:0/3086817809 conn(0x7fc18406d3f0 msgr2=0x7fc18406d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- 172.21.15.157:0/3086817809 shutdown_connections 2023-11-06T03:14:12.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- 172.21.15.157:0/3086817809 wait complete. 2023-11-06T03:14:12.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 Processor -- start 2023-11-06T03:14:12.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- start start 2023-11-06T03:14:12.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc18406ba10 0x7fc18414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc1840c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840c8640 con 0x7fc18415cd60 2023-11-06T03:14:12.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840c88e0 con 0x7fc18406ba10 2023-11-06T03:14:12.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc189114640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840c8bc0 con 0x7fc1840713f0 2023-11-06T03:14:12.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.829+0000 7fc18359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40080/0 (socket says 172.21.15.157:40080) 2023-11-06T03:14:12.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 -- 172.21.15.157:0/1702378385 learned_addr learned my addr 172.21.15.157:0/1702378385 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc182d9d640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc18406ba10 0x7fc18414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18259c640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc1840c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 -- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 msgr2=0x7fc1840c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc1840c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 -- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc18406ba10 msgr2=0x7fc18414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc18406ba10 0x7fc18414f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 -- 172.21.15.157:0/1702378385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178009b30 con 0x7fc18415cd60 2023-11-06T03:14:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18359e640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c7f0 tx=0x7fc17c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 -- 172.21.15.157:0/1702378385 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c007820 con 0x7fc18415cd60 2023-11-06T03:14:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 -- 172.21.15.157:0/1702378385 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0079c0 con 0x7fc18415cd60 2023-11-06T03:14:12.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840c8ea0 con 0x7fc18415cd60 2023-11-06T03:14:12.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 -- 172.21.15.157:0/1702378385 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c007d70 con 0x7fc18415cd60 2023-11-06T03:14:12.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1840c9460 con 0x7fc18415cd60 2023-11-06T03:14:12.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 -- 172.21.15.157:0/1702378385 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc17c0113f0 con 0x7fc18415cd60 2023-11-06T03:14:12.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 0x7fc170067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc16b7fe640 1 -- 172.21.15.157:0/1702378385 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc17c083720 con 0x7fc18415cd60 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc182d9d640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 0x7fc170067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 Processor -- start 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 -- start start 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc182d9d640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 0x7fc170067f20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc174000f70 tx=0x7fc174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da861cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da729bb0 con 0x5618da8ac0d0 2023-11-06T03:14:12.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da6b8380 con 0x5618da862200 2023-11-06T03:14:12.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.833+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618da6b5950 con 0x5618da8ac9f0 2023-11-06T03:14:12.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40092/0 (socket says 172.21.15.157:40092) 2023-11-06T03:14:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 -- 172.21.15.157:0/3913368491 learned_addr learned my addr 172.21.15.157:0/3913368491 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18259c640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da861cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 -- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 msgr2=0x5618da861cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da861cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 -- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 msgr2=0x5618da8625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 -- 172.21.15.157:0/3913368491 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618da3addc0 con 0x5618da8ac0d0 2023-11-06T03:14:12.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:12.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc182d9d640 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc174005350 tx=0x7fc174010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe3628f2b2cb9e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc168ff9640 1 -- 172.21.15.157:0/3913368491 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17401c070 con 0x5618da8ac0d0 2023-11-06T03:14:12.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc168ff9640 1 -- 172.21.15.157:0/3913368491 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400d4a0 con 0x5618da8ac0d0 2023-11-06T03:14:12.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc168ff9640 1 -- 172.21.15.157:0/3913368491 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400d850 con 0x5618da8ac0d0 2023-11-06T03:14:12.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/3913368491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 msgr2=0x5618da8ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc174005350 tx=0x7fc174010000 comp rx=0 tx=0).stop 2023-11-06T03:14:12.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/3913368491 shutdown_connections 2023-11-06T03:14:12.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da861cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- 172.21.15.157:0/3913368491 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ac4b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/3913368491 >> 172.21.15.157:0/3913368491 conn(0x5618da868990 msgr2=0x5618da8605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/3913368491 shutdown_connections 2023-11-06T03:14:12.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/3913368491 wait complete. 2023-11-06T03:14:12.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- start start 2023-11-06T03:14:12.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 Processor -- start 2023-11-06T03:14:12.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac0d0 0x5618da8b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da6b1240 con 0x5618da8ac9f0 2023-11-06T03:14:12.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da650410 con 0x5618da862200 2023-11-06T03:14:12.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618da66e600 con 0x5618da8ac0d0 2023-11-06T03:14:12.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 10 client.0 _ll_register_callbacks cb 0x5618da84dbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:12.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40100/0 (socket says 172.21.15.157:40100) 2023-11-06T03:14:12.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 -- 172.21.15.157:0/425819052 learned_addr learned my addr 172.21.15.157:0/425819052 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 -- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac0d0 msgr2=0x5618da8b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac0d0 0x5618da8b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 -- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 msgr2=0x5618da8b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 -- 172.21.15.157:0/425819052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc184071d30 con 0x5618da8ac9f0 2023-11-06T03:14:12.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18359e640 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00afd0 tx=0x7fc17c054ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e98ec89393f2db2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:12.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c06c070 con 0x5618da8ac9f0 2023-11-06T03:14:12.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc17c053000 con 0x5618da8ac9f0 2023-11-06T03:14:12.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618da66e040 con 0x5618da8ac9f0 2023-11-06T03:14:12.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc17c06c1d0 con 0x5618da8ac9f0 2023-11-06T03:14:12.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 10 client.9057 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:12.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5618da676170 con 0x5618da8ac9f0 2023-11-06T03:14:12.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618da66e040 con 0x5618da8ac9f0 2023-11-06T03:14:12.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc17c055bb0 con 0x5618da8ac9f0 2023-11-06T03:14:12.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 10 client.9057 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:12.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 10 client.9057 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:12.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.837+0000 7fc18c23c000 20 client.9057 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:12.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8adb70 0x5618da8adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 4 client.9057 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:12.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618da6afb90 con 0x5618da8adb70 2023-11-06T03:14:12.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc182d9d640 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8adb70 0x5618da8adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc161ffb640 10 client.9057 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:12.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc182d9d640 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8adb70 0x5618da8adf50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f624fc1613d8177 server_cookie=4d582638f70b8d69 in_seq=0 out_seq=0 2023-11-06T03:14:12.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc161ffb640 1 -- 172.21.15.157:0/425819052 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5618da6afb90 con 0x5618da8adb70 2023-11-06T03:14:12.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc161ffb640 10 client.9057 handle_command_reply: tid=0 2023-11-06T03:14:12.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 client.9057 shutdown 2023-11-06T03:14:12.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 msgr2=0x5618da8acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00afd0 tx=0x7fc17c054ce0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 shutdown_connections 2023-11-06T03:14:12.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac9f0 0x5618da8acdd0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac0d0 0x5618da8b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8adb70 0x5618da8adf50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 --2- 172.21.15.157:0/425819052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da862200 0x5618da8b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 >> 172.21.15.157:0/425819052 conn(0x5618da868990 msgr2=0x5618da860ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc161ffb640 0 client.9057 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:12.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 shutdown_connections 2023-11-06T03:14:12.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 1 -- 172.21.15.157:0/425819052 wait complete. 2023-11-06T03:14:12.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.841+0000 7fc18c23c000 20 client.9057 trim_cache size 0 max 16384 2023-11-06T03:14:12.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 Processor -- start 2023-11-06T03:14:12.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 -- start start 2023-11-06T03:14:12.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac0d0 0x5618da8b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da8b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da36ea40 con 0x5618da862200 2023-11-06T03:14:12.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da6acbb0 con 0x5618da8ac0d0 2023-11-06T03:14:12.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618da66f3a0 con 0x5618da8ac9f0 2023-11-06T03:14:12.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc182d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc182d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40110/0 (socket says 172.21.15.157:40110) 2023-11-06T03:14:12.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.853+0000 7fc182d9d640 1 -- 172.21.15.157:0/392550803 learned_addr learned my addr 172.21.15.157:0/392550803 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 -- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 msgr2=0x5618da8b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da8b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 -- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac0d0 msgr2=0x5618da8b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac0d0 0x5618da8b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 -- 172.21.15.157:0/392550803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618da65c4c0 con 0x5618da862200 2023-11-06T03:14:12.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc174016000 tx=0x7fc174011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a401e5f4615bdd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc1697fa640 1 -- 172.21.15.157:0/392550803 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174026040 con 0x5618da862200 2023-11-06T03:14:12.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc1697fa640 1 -- 172.21.15.157:0/392550803 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1740069e0 con 0x5618da862200 2023-11-06T03:14:12.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc1697fa640 1 -- 172.21.15.157:0/392550803 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17402a5b0 con 0x5618da862200 2023-11-06T03:14:12.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/392550803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 msgr2=0x5618da8b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fc174016000 tx=0x7fc174011000 comp rx=0 tx=0).stop 2023-11-06T03:14:12.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/392550803 shutdown_connections 2023-11-06T03:14:12.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da8ac9f0 0x5618da8b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac0d0 0x5618da8b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- 172.21.15.157:0/392550803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da862200 0x5618da8b0f70 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/392550803 >> 172.21.15.157:0/392550803 conn(0x5618da868990 msgr2=0x5618da867b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/392550803 shutdown_connections 2023-11-06T03:14:12.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/392550803 wait complete. 2023-11-06T03:14:12.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- start start 2023-11-06T03:14:12.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 Processor -- start 2023-11-06T03:14:12.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da862200 0x5618da8ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac9f0 0x5618da8ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da6b00d0 con 0x5618da8ac0d0 2023-11-06T03:14:12.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618da865700 con 0x5618da8ac9f0 2023-11-06T03:14:12.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618da6f3c70 con 0x5618da862200 2023-11-06T03:14:12.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 10 client.0 _ll_register_callbacks cb 0x5618da675c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:12.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40118/0 (socket says 172.21.15.157:40118) 2023-11-06T03:14:12.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 -- 172.21.15.157:0/2530350169 learned_addr learned my addr 172.21.15.157:0/2530350169 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:12.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da862200 0x5618da8ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18359e640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac9f0 0x5618da8ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 -- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da862200 msgr2=0x5618da8ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da862200 0x5618da8ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 -- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac9f0 msgr2=0x5618da8ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac9f0 0x5618da8ed860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 -- 172.21.15.157:0/2530350169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618da3addc0 con 0x5618da8ac0d0 2023-11-06T03:14:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18259c640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc17801d8a0 tx=0x7fc1780090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c2c81a4ab630d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780ba680 con 0x5618da8ac0d0 2023-11-06T03:14:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc1780ba7e0 con 0x5618da8ac0d0 2023-11-06T03:14:12.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618da67a5c0 con 0x5618da8ac0d0 2023-11-06T03:14:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1780ba9c0 con 0x5618da8ac0d0 2023-11-06T03:14:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 10 client.9063 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5618da650030 con 0x5618da8ac0d0 2023-11-06T03:14:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618da67a5c0 con 0x5618da8ac0d0 2023-11-06T03:14:12.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc1780c2020 con 0x5618da8ac0d0 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 10 client.9063 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 10 client.9063 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 20 client.9063 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8edda0 0x5618da8ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 4 client.9063 mds_command: new command op to 8796 tid=0[{"prefix": "scrub abort"}] 2023-11-06T03:14:12.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5618da6aa2c0 con 0x5618da8edda0 2023-11-06T03:14:12.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8edda0 0x5618da8ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc182d9d640 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8edda0 0x5618da8ee1b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a3e8a2a16dc4ea server_cookie=b09775e18d1d3a03 in_seq=0 out_seq=0 2023-11-06T03:14:12.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.857+0000 7fc161ffb640 10 client.9063 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:12.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc161ffb640 1 -- 172.21.15.157:0/2530350169 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5618da6aa2c0 con 0x5618da8edda0 2023-11-06T03:14:12.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc161ffb640 10 client.9063 handle_command_reply: tid=0 2023-11-06T03:14:12.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 client.9063 shutdown 2023-11-06T03:14:12.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 msgr2=0x5618da8ed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc17801d8a0 tx=0x7fc1780090d0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 shutdown_connections 2023-11-06T03:14:12.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x5618da8edda0 0x5618da8ee1b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5618da8ac9f0 0x5618da8ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc161ffb640 0 client.9063 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:12.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5618da8ac0d0 0x5618da8ed320 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 --2- 172.21.15.157:0/2530350169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5618da862200 0x5618da8ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 >> 172.21.15.157:0/2530350169 conn(0x5618da868990 msgr2=0x5618da8aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 shutdown_connections 2023-11-06T03:14:12.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 1 -- 172.21.15.157:0/2530350169 wait complete. 2023-11-06T03:14:12.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc18c23c000 20 client.9063 trim_cache size 0 max 16384 2023-11-06T03:14:12.879 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:12.880 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:14:12.880 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:12.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 msgr2=0x7fc170067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 0x7fc170067f20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc174000f70 tx=0x7fc174009040 comp rx=0 tx=0).stop 2023-11-06T03:14:12.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 msgr2=0x7fc18415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c7f0 tx=0x7fc17c00ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 shutdown_connections 2023-11-06T03:14:12.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc170065a60 0x7fc170067f20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18415cd60 0x7fc18415f100 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1840713f0 0x7fc1840c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 --2- 172.21.15.157:0/1702378385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc18406ba10 0x7fc18414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 >> 172.21.15.157:0/1702378385 conn(0x7fc18406d3f0 msgr2=0x7fc18406d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:12.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 shutdown_connections 2023-11-06T03:14:12.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.861+0000 7fc189114640 1 -- 172.21.15.157:0/1702378385 wait complete. 2023-11-06T03:14:12.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub status 2023-11-06T03:14:12.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.993+0000 7f0420c7c640 1 Processor -- start 2023-11-06T03:14:12.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.993+0000 7f0420c7c640 1 -- start start 2023-11-06T03:14:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 0x7f041c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f041c0b3b20 0x7f041c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c15f580 con 0x7f041c15e5e0 2023-11-06T03:14:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c15efc0 con 0x7f041c0b3b20 2023-11-06T03:14:12.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c15f2a0 con 0x7f041c1551e0 2023-11-06T03:14:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f041c0b3b20 0x7f041c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0419d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f041c15e5e0 0x7f041c0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58230/0 (socket says 172.21.15.157:58230) 2023-11-06T03:14:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0419d9b640 1 -- 172.21.15.157:0/4097833873 learned_addr learned my addr 172.21.15.157:0/4097833873 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041a59c640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 0x7f041c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:12.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 -- 172.21.15.157:0/4097833873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 msgr2=0x7f041c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 0x7f041c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 -- 172.21.15.157:0/4097833873 >> v1:172.21.15.157:6789/0 conn(0x7f041c15e5e0 legacy=0x7f041c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041959a640 1 -- 172.21.15.157:0/4097833873 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706419153 0 0) 0x7f041c15f580 con 0x7f041c15e5e0 2023-11-06T03:14:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 -- 172.21.15.157:0/4097833873 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c06f5a0 con 0x7f041c0b3b20 2023-11-06T03:14:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041a59c640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 0x7f041c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041ad9d640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c15c1b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f0414009d90 tx=0x7f041401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca8e2b954da3b03 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041959a640 1 -- 172.21.15.157:0/4097833873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140a5970 con 0x7f041c0b3b20 2023-11-06T03:14:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041959a640 1 -- 172.21.15.157:0/4097833873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04140a5ad0 con 0x7f041c0b3b20 2023-11-06T03:14:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f041959a640 1 -- 172.21.15.157:0/4097833873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140a5e60 con 0x7f041c0b3b20 2023-11-06T03:14:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- 172.21.15.157:0/4097833873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 msgr2=0x7f041c15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c15c1b0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f0414009d90 tx=0x7f041401c890 comp rx=0 tx=0).stop 2023-11-06T03:14:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- 172.21.15.157:0/4097833873 shutdown_connections 2023-11-06T03:14:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c15c1b0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 --2- 172.21.15.157:0/4097833873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c1551e0 0x7f041c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- 172.21.15.157:0/4097833873 >> 172.21.15.157:0/4097833873 conn(0x7f041c065930 msgr2=0x7f041c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- 172.21.15.157:0/4097833873 shutdown_connections 2023-11-06T03:14:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:12.997+0000 7f0420c7c640 1 -- 172.21.15.157:0/4097833873 wait complete. 2023-11-06T03:14:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 Processor -- start 2023-11-06T03:14:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- start start 2023-11-06T03:14:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f041c1551e0 0x7f041c14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c15e5e0 0x7f041c142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c147120 con 0x7f041c0b3b20 2023-11-06T03:14:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c147ee0 con 0x7f041c1551e0 2023-11-06T03:14:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c1467c0 con 0x7f041c15e5e0 2023-11-06T03:14:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40148/0 (socket says 172.21.15.157:40148) 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 -- 172.21.15.157:0/930888148 learned_addr learned my addr 172.21.15.157:0/930888148 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0419d9b640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f041c1551e0 0x7f041c14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041ad9d640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c15e5e0 0x7f041c142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 -- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c15e5e0 msgr2=0x7f041c142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c15e5e0 0x7f041c142d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 -- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f041c1551e0 msgr2=0x7f041c14c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f041c1551e0 0x7f041c14c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 -- 172.21.15.157:0/930888148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009a70 con 0x7f041c0b3b20 2023-11-06T03:14:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f041a59c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e810 tx=0x7f040c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f040affd640 1 -- 172.21.15.157:0/930888148 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00c8d0 con 0x7f041c0b3b20 2023-11-06T03:14:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f040affd640 1 -- 172.21.15.157:0/930888148 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00ca30 con 0x7f041c0b3b20 2023-11-06T03:14:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f040affd640 1 -- 172.21.15.157:0/930888148 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00cdc0 con 0x7f041c0b3b20 2023-11-06T03:14:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c146a60 con 0x7f041c0b3b20 2023-11-06T03:14:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.001+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c144c40 con 0x7f041c0b3b20 2023-11-06T03:14:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f040affd640 1 -- 172.21.15.157:0/930888148 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f040c013410 con 0x7f041c0b3b20 2023-11-06T03:14:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f040affd640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 0x7f03fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f040affd640 1 -- 172.21.15.157:0/930888148 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f040c083030 con 0x7f041c0b3b20 2023-11-06T03:14:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0419d9b640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 0x7f03fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 Processor -- start 2023-11-06T03:14:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- start start 2023-11-06T03:14:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0419d9b640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 0x7f03fc067f20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f041c0cad40 tx=0x7f0410007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be741510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741a50 0x55d1be6f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be5bec10 con 0x55d1be6f7260 2023-11-06T03:14:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be54d3e0 con 0x55d1be741130 2023-11-06T03:14:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1be54a9b0 con 0x55d1be741a50 2023-11-06T03:14:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40162/0 (socket says 172.21.15.157:40162) 2023-11-06T03:14:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 -- 172.21.15.157:0/3207549903 learned_addr learned my addr 172.21.15.157:0/3207549903 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041a59c640 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be741510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 -- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741a50 msgr2=0x55d1be6f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741a50 0x55d1be6f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 -- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 msgr2=0x55d1be741510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be741510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 -- 172.21.15.157:0/3207549903 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1be242e20 con 0x55d1be6f7260 2023-11-06T03:14:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041a59c640 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be741510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f04140b0040 tx=0x7f0414004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d53693d28f74403 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f03fbfff640 1 -- 172.21.15.157:0/3207549903 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140ad640 con 0x55d1be6f7260 2023-11-06T03:14:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f03fbfff640 1 -- 172.21.15.157:0/3207549903 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04140ad7a0 con 0x55d1be6f7260 2023-11-06T03:14:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f03fbfff640 1 -- 172.21.15.157:0/3207549903 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140adb30 con 0x55d1be6f7260 2023-11-06T03:14:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- 172.21.15.157:0/3207549903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 msgr2=0x55d1be6f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f04140b0040 tx=0x7f0414004ac0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- 172.21.15.157:0/3207549903 shutdown_connections 2023-11-06T03:14:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be6f7260 0x55d1be6f7640 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741a50 0x55d1be6f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 --2- 172.21.15.157:0/3207549903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be741510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- 172.21.15.157:0/3207549903 >> 172.21.15.157:0/3207549903 conn(0x55d1be6fd9f0 msgr2=0x55d1be6f5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- 172.21.15.157:0/3207549903 shutdown_connections 2023-11-06T03:14:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.005+0000 7f0423da4000 1 -- 172.21.15.157:0/3207549903 wait complete. 2023-11-06T03:14:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- start start 2023-11-06T03:14:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 Processor -- start 2023-11-06T03:14:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be7484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be748a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be5462a0 con 0x55d1be741a50 2023-11-06T03:14:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be4e5470 con 0x55d1be6f7260 2023-11-06T03:14:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1be503660 con 0x55d1be741130 2023-11-06T03:14:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 10 client.0 _ll_register_callbacks cb 0x55d1be6e2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40166/0 (socket says 172.21.15.157:40166) 2023-11-06T03:14:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 -- 172.21.15.157:0/3915458454 learned_addr learned my addr 172.21.15.157:0/3915458454 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041a59c640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be7484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 -- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 msgr2=0x55d1be748a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be748a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 -- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 msgr2=0x55d1be7484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be7484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 -- 172.21.15.157:0/3915458454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c06f5a0 con 0x55d1be741a50 2023-11-06T03:14:13.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041ad9d640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f04140095c0 tx=0x7f0414004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a8d576b20d688c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:13.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414004240 con 0x55d1be741a50 2023-11-06T03:14:13.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04140043a0 con 0x55d1be741a50 2023-11-06T03:14:13.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1be5030a0 con 0x55d1be741a50 2023-11-06T03:14:13.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0414007080 con 0x55d1be741a50 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 10 client.9078 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55d1be50b1d0 con 0x55d1be741a50 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1be5030a0 con 0x55d1be741a50 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0414004560 con 0x55d1be741a50 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 10 client.9078 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 10 client.9078 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 20 client.9078 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be742b70 0x55d1be742f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 4 client.9078 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1be544bf0 con 0x55d1be742b70 2023-11-06T03:14:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041a59c640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be742b70 0x55d1be742f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f041a59c640 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be742b70 0x55d1be742f50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d0e36360c11c5b server_cookie=1016a0f7bae98032 in_seq=0 out_seq=0 2023-11-06T03:14:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 10 client.9078 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 1 -- 172.21.15.157:0/3915458454 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d1be544bf0 con 0x55d1be742b70 2023-11-06T03:14:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 10 client.9078 handle_command_reply: tid=0 2023-11-06T03:14:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 client.9078 shutdown 2023-11-06T03:14:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 msgr2=0x55d1be741e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f04140095c0 tx=0x7f0414004a50 comp rx=0 tx=0).stop 2023-11-06T03:14:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 shutdown_connections 2023-11-06T03:14:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be741e30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be748a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be742b70 0x55d1be742f50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 --2- 172.21.15.157:0/3915458454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be7484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f03f97fa640 0 client.9078 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 >> 172.21.15.157:0/3915458454 conn(0x55d1be6fd9f0 msgr2=0x55d1be6f5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 shutdown_connections 2023-11-06T03:14:13.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 1 -- 172.21.15.157:0/3915458454 wait complete. 2023-11-06T03:14:13.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.009+0000 7f0423da4000 20 client.9078 trim_cache size 0 max 16384 2023-11-06T03:14:13.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 Processor -- start 2023-11-06T03:14:13.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- start start 2023-11-06T03:14:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be745f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be7464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be203aa0 con 0x55d1be741a50 2023-11-06T03:14:13.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be543ab0 con 0x55d1be6f7260 2023-11-06T03:14:13.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1be54a2a0 con 0x55d1be741130 2023-11-06T03:14:13.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40182/0 (socket says 172.21.15.157:40182) 2023-11-06T03:14:13.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 -- 172.21.15.157:0/4035687524 learned_addr learned my addr 172.21.15.157:0/4035687524 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041a59c640 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be745f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 -- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 msgr2=0x55d1be7464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be7464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 -- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 msgr2=0x55d1be745f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be745f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 -- 172.21.15.157:0/4035687524 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1be4f1520 con 0x55d1be741a50 2023-11-06T03:14:13.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041a59c640 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be745f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:13.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f041ad9d640 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f041401c6c0 tx=0x7f04140b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9ff2761350d5f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0408ff9640 1 -- 172.21.15.157:0/4035687524 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140ad620 con 0x55d1be741a50 2023-11-06T03:14:13.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0408ff9640 1 -- 172.21.15.157:0/4035687524 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04140ad780 con 0x55d1be741a50 2023-11-06T03:14:13.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0408ff9640 1 -- 172.21.15.157:0/4035687524 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414003cb0 con 0x55d1be741a50 2023-11-06T03:14:13.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- 172.21.15.157:0/4035687524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 msgr2=0x55d1be7469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f041401c6c0 tx=0x7f04140b7000 comp rx=0 tx=0).stop 2023-11-06T03:14:13.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- 172.21.15.157:0/4035687524 shutdown_connections 2023-11-06T03:14:13.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be7469f0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be741130 0x55d1be7464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 --2- 172.21.15.157:0/4035687524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be6f7260 0x55d1be745f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- 172.21.15.157:0/4035687524 >> 172.21.15.157:0/4035687524 conn(0x55d1be6fd9f0 msgr2=0x55d1be6fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- 172.21.15.157:0/4035687524 shutdown_connections 2023-11-06T03:14:13.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- 172.21.15.157:0/4035687524 wait complete. 2023-11-06T03:14:13.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 -- start start 2023-11-06T03:14:13.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.025+0000 7f0423da4000 1 Processor -- start 2023-11-06T03:14:13.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 0x55d1be780d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be781690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be6fa530 con 0x55d1be741a50 2023-11-06T03:14:13.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1be541c10 con 0x55d1be741130 2023-11-06T03:14:13.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1be6eea70 con 0x55d1be6f7260 2023-11-06T03:14:13.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 10 client.0 _ll_register_callbacks cb 0x55d1be50acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:13.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40196/0 (socket says 172.21.15.157:40196) 2023-11-06T03:14:13.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 -- 172.21.15.157:0/680460178 learned_addr learned my addr 172.21.15.157:0/680460178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0419d9b640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be781690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041a59c640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 0x55d1be780d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 -- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 msgr2=0x55d1be780d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 0x55d1be780d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 -- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 msgr2=0x55d1be781690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be781690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 -- 172.21.15.157:0/680460178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1be50b1d0 con 0x55d1be741a50 2023-11-06T03:14:13.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041a59c640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 0x55d1be780d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:13.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041ad9d640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f04140a7040 tx=0x7f041401b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabb0639ae434e17 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140ad620 con 0x55d1be741a50 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1be50f620 con 0x55d1be741a50 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0414004170 con 0x55d1be741a50 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0414004370 con 0x55d1be741a50 2023-11-06T03:14:13.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 10 client.9084 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:13.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55d1be4e5090 con 0x55d1be741a50 2023-11-06T03:14:13.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04140db020 con 0x55d1be741a50 2023-11-06T03:14:13.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f04140044d0 con 0x55d1be741a50 2023-11-06T03:14:13.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 10 client.9084 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:13.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 10 client.9084 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 20 client.9084 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be782110 0x55d1be7824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 4 client.9084 mds_command: new command op to 8796 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:14:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1be53f320 con 0x55d1be782110 2023-11-06T03:14:13.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041a59c640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be782110 0x55d1be7824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f041a59c640 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be782110 0x55d1be7824f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d69f9a5fb62de47 server_cookie=74c52102a0e1923e in_seq=0 out_seq=0 2023-11-06T03:14:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 10 client.9084 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 1 -- 172.21.15.157:0/680460178 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55d1be53f320 con 0x55d1be782110 2023-11-06T03:14:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 10 client.9084 handle_command_reply: tid=0 2023-11-06T03:14:13.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 client.9084 shutdown 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 msgr2=0x55d1be781bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f04140a7040 tx=0x7f041401b0d0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 shutdown_connections 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1be741a50 0x55d1be781bd0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55d1be782110 0x55d1be7824f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d1be741130 0x55d1be781690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 --2- 172.21.15.157:0/680460178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d1be6f7260 0x55d1be780d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f03f97fa640 0 client.9084 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 >> 172.21.15.157:0/680460178 conn(0x55d1be6fd9f0 msgr2=0x55d1be744f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 shutdown_connections 2023-11-06T03:14:13.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 1 -- 172.21.15.157:0/680460178 wait complete. 2023-11-06T03:14:13.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0423da4000 20 client.9084 trim_cache size 0 max 16384 2023-11-06T03:14:13.050 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:13.050 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-11-06T03:14:13.051 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:14:13.051 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:13.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 msgr2=0x7f03fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 0x7f03fc067f20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f041c0cad40 tx=0x7f0410007c10 comp rx=0 tx=0).stop 2023-11-06T03:14:13.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 msgr2=0x7f041c147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f040c00e810 tx=0x7f040c00ecd0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 shutdown_connections 2023-11-06T03:14:13.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f03fc065a60 0x7f03fc067f20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f041c15e5e0 0x7f041c142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f041c1551e0 0x7f041c14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 --2- 172.21.15.157:0/930888148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f041c0b3b20 0x7f041c147850 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 >> 172.21.15.157:0/930888148 conn(0x7f041c065930 msgr2=0x7f041c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 shutdown_connections 2023-11-06T03:14:13.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.029+0000 7f0420c7c640 1 -- 172.21.15.157:0/930888148 wait complete. 2023-11-06T03:14:13.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:13.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.169+0000 7fa9bb319640 1 Processor -- start 2023-11-06T03:14:13.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.169+0000 7fa9bb319640 1 -- start start 2023-11-06T03:14:13.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b40ce0f0 0x7fa9b40b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b4160980 0x7fa9b40d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b40b5db0 con 0x7fa9b415cd60 2023-11-06T03:14:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40b8c70 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41610d0 con 0x7fa9b4160980 2023-11-06T03:14:13.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b40ce0f0 0x7fa9b40b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa9b40ce0f0 0x7fa9b40b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40198/0 (socket says 172.21.15.157:40198) 2023-11-06T03:14:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 -- 172.21.15.157:0/1217096744 learned_addr learned my addr 172.21.15.157:0/1217096744 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b98b6640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b4160980 0x7fa9b40d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 -- 172.21.15.157:0/1217096744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b4160980 msgr2=0x7fa9b40d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b4160980 0x7fa9b40d1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 -- 172.21.15.157:0/1217096744 >> v1:172.21.15.195:6789/0 conn(0x7fa9b415cd60 legacy=0x7fa9b40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 -- 172.21.15.157:0/1217096744 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b41613b0 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9b88b4640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b40b8730 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40d05d0 tx=0x7fa9b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2bdd7c6d077842 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9abfff640 1 -- 172.21.15.157:0/1217096744 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00ab650 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9abfff640 1 -- 172.21.15.157:0/1217096744 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b00ab7b0 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9abfff640 1 -- 172.21.15.157:0/1217096744 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00abb40 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- 172.21.15.157:0/1217096744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b40ce0f0 msgr2=0x7fa9b40b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b40b8730 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40d05d0 tx=0x7fa9b001c490 comp rx=0 tx=0).stop 2023-11-06T03:14:13.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- 172.21.15.157:0/1217096744 shutdown_connections 2023-11-06T03:14:13.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b4160980 0x7fa9b40d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 --2- 172.21.15.157:0/1217096744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b40b8730 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- 172.21.15.157:0/1217096744 >> 172.21.15.157:0/1217096744 conn(0x7fa9b4065930 msgr2=0x7fa9b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- 172.21.15.157:0/1217096744 shutdown_connections 2023-11-06T03:14:13.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.173+0000 7fa9bb319640 1 -- 172.21.15.157:0/1217096744 wait complete. 2023-11-06T03:14:13.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 Processor -- start 2023-11-06T03:14:13.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- start start 2023-11-06T03:14:13.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b41f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 0x7fa9b414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c6a70 con 0x7fa9b4160980 2023-11-06T03:14:13.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41f8e90 con 0x7fa9b40ce0f0 2023-11-06T03:14:13.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c6740 con 0x7fa9b415cd60 2023-11-06T03:14:13.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40204/0 (socket says 172.21.15.157:40204) 2023-11-06T03:14:13.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 -- 172.21.15.157:0/3371069768 learned_addr learned my addr 172.21.15.157:0/3371069768 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b90b5640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b41f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b88b4640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 0x7fa9b414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 -- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 msgr2=0x7fa9b414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 0x7fa9b414c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 -- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b40ce0f0 msgr2=0x7fa9b41f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b41f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 -- 172.21.15.157:0/3371069768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00094d0 con 0x7fa9b4160980 2023-11-06T03:14:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b88b4640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 0x7fa9b414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:13.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9b98b6640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00cc60 tx=0x7fa9ac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ce70 con 0x7fa9b4160980 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac0039a0 con 0x7fa9b4160980 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b41f9110 con 0x7fa9b4160980 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac003d30 con 0x7fa9b4160980 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41f96d0 con 0x7fa9b4160980 2023-11-06T03:14:13.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.177+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b40c6be0 con 0x7fa9b4160980 2023-11-06T03:14:13.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.181+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa9ac003e90 con 0x7fa9b4160980 2023-11-06T03:14:13.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.181+0000 7fa9a9ffb640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 0x7fa998067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.181+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9ac0835e0 con 0x7fa9b4160980 2023-11-06T03:14:13.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.181+0000 7fa9b90b5640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 0x7fa998067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.181+0000 7fa9b90b5640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 0x7fa998067f20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007fd0 tx=0x7fa9a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.185+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa9ac050ed0 con 0x7fa9b4160980 2023-11-06T03:14:13.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.357+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9b4072750 con 0x7fa9b4160980 2023-11-06T03:14:13.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.357+0000 7fa9a9ffb640 1 -- 172.21.15.157:0/3371069768 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fa9ac01d090 con 0x7fa9b4160980 2023-11-06T03:14:13.356 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:13.356 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:13.356 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:13.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 msgr2=0x7fa998067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 0x7fa998067f20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007fd0 tx=0x7fa9a4009040 comp rx=0 tx=0).stop 2023-11-06T03:14:13.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 msgr2=0x7fa9b40c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00cc60 tx=0x7fa9ac00a5c0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 shutdown_connections 2023-11-06T03:14:13.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa998065a60 0x7fa998067f20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9b4160980 0x7fa9b40c6f00 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa9b415cd60 0x7fa9b414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 --2- 172.21.15.157:0/3371069768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9b40ce0f0 0x7fa9b41f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 >> 172.21.15.157:0/3371069768 conn(0x7fa9b4065930 msgr2=0x7fa9b40b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 shutdown_connections 2023-11-06T03:14:13.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.361+0000 7fa9bb319640 1 -- 172.21.15.157:0/3371069768 wait complete. 2023-11-06T03:14:13.372 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:13.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:13.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 Processor -- start 2023-11-06T03:14:13.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 -- start start 2023-11-06T03:14:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f00540b1ad0 0x7f00540b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f005415b860 0x7f005415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005415b3d0 con 0x7f00540b1eb0 2023-11-06T03:14:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0054155610 con 0x7f005415b860 2023-11-06T03:14:13.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.493+0000 7f0058ce9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00540c9d30 con 0x7f00540b1ad0 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f005415b860 0x7f005415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f005415b860 0x7f005415bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40218/0 (socket says 172.21.15.157:40218) 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 -- 172.21.15.157:0/957937752 learned_addr learned my addr 172.21.15.157:0/957937752 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f005259c640 1 --2- 172.21.15.157:0/957937752 >> v2:172.21.15.195:3300/0 conn(0x7f00540b1ad0 0x7f00540b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 -- 172.21.15.157:0/957937752 >> v2:172.21.15.195:3300/0 conn(0x7f00540b1ad0 msgr2=0x7f00540b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 --2- 172.21.15.157:0/957937752 >> v2:172.21.15.195:3300/0 conn(0x7f00540b1ad0 0x7f00540b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 -- 172.21.15.157:0/957937752 >> v1:172.21.15.195:6789/0 conn(0x7f00540b1eb0 legacy=0x7f00540c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 -- 172.21.15.157:0/957937752 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005415b0c0 con 0x7f005415b860 2023-11-06T03:14:13.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0052d9d640 1 --2- 172.21.15.157:0/957937752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f005415b860 0x7f005415bc80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f00540b2960 tx=0x7f004c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e4ad676ed763af server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f005159a640 1 -- 172.21.15.157:0/957937752 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0a5860 con 0x7f005415b860 2023-11-06T03:14:13.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f005159a640 1 -- 172.21.15.157:0/957937752 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c0a59c0 con 0x7f005415b860 2023-11-06T03:14:13.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f005159a640 1 -- 172.21.15.157:0/957937752 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0a5d50 con 0x7f005415b860 2023-11-06T03:14:13.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 -- 172.21.15.157:0/957937752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f005415b860 msgr2=0x7f005415bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 --2- 172.21.15.157:0/957937752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f005415b860 0x7f005415bc80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f00540b2960 tx=0x7f004c01c9a0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 -- 172.21.15.157:0/957937752 shutdown_connections 2023-11-06T03:14:13.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 --2- 172.21.15.157:0/957937752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f005415b860 0x7f005415bc80 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 --2- 172.21.15.157:0/957937752 >> v2:172.21.15.195:3300/0 conn(0x7f00540b1ad0 0x7f00540b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 -- 172.21.15.157:0/957937752 >> 172.21.15.157:0/957937752 conn(0x7f005406a170 msgr2=0x7f005406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 -- 172.21.15.157:0/957937752 shutdown_connections 2023-11-06T03:14:13.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.497+0000 7f0058ce9640 1 -- 172.21.15.157:0/957937752 wait complete. 2023-11-06T03:14:13.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 Processor -- start 2023-11-06T03:14:13.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- start start 2023-11-06T03:14:13.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00540b1ad0 0x7f0054151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005415b860 0x7f00540c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00541f6550 con 0x7f00540b1eb0 2023-11-06T03:14:13.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00541f5480 con 0x7f005415b860 2023-11-06T03:14:13.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00541f5760 con 0x7f00540b1ad0 2023-11-06T03:14:13.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40230/0 (socket says 172.21.15.157:40230) 2023-11-06T03:14:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 -- 172.21.15.157:0/925752913 learned_addr learned my addr 172.21.15.157:0/925752913 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0052d9d640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005415b860 0x7f00540c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f005259c640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00540b1ad0 0x7f0054151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 -- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00540b1ad0 msgr2=0x7f0054151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00540b1ad0 0x7f0054151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 -- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005415b860 msgr2=0x7f00540c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005415b860 0x7f00540c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 -- 172.21.15.157:0/925752913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009b30 con 0x7f00540b1eb0 2023-11-06T03:14:13.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0051d9b640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f004400c4d0 tx=0x7f004400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440052d0 con 0x7f00540b1eb0 2023-11-06T03:14:13.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044005430 con 0x7f00540b1eb0 2023-11-06T03:14:13.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00541f5a40 con 0x7f00540b1eb0 2023-11-06T03:14:13.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044012830 con 0x7f00540b1eb0 2023-11-06T03:14:13.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00541f6000 con 0x7f00540b1eb0 2023-11-06T03:14:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0054072750 con 0x7f00540b1eb0 2023-11-06T03:14:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0044012990 con 0x7f00540b1eb0 2023-11-06T03:14:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.501+0000 7f0042ffd640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 0x7f0034067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.505+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0044082dd0 con 0x7f00540b1eb0 2023-11-06T03:14:13.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.505+0000 7f005259c640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 0x7f0034067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.505+0000 7f005259c640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 0x7f0034067f20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f004800b9d0 tx=0x7f0048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.509+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0044050630 con 0x7f00540b1eb0 2023-11-06T03:14:13.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.717+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0054070f70 con 0x7f00540b1eb0 2023-11-06T03:14:13.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.717+0000 7f0042ffd640 1 -- 172.21.15.157:0/925752913 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1632 (secure 0 0 0) 0x7f0044058e60 con 0x7f00540b1eb0 2023-11-06T03:14:13.714 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:13.715 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1021,"data_bytes":1048572774,"bytes_used":2677878784,"bytes_avail":770416234496,"bytes_total":773094113280,"write_bytes_sec":52940305,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-11-06T03:14:11.316119+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:13.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 msgr2=0x7f0034067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 0x7f0034067f20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f004800b9d0 tx=0x7f0048008040 comp rx=0 tx=0).stop 2023-11-06T03:14:13.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 msgr2=0x7f00540c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f004400c4d0 tx=0x7f004400c990 comp rx=0 tx=0).stop 2023-11-06T03:14:13.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 shutdown_connections 2023-11-06T03:14:13.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0034065a60 0x7f0034067f20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005415b860 0x7f00540c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b1eb0 0x7f00540c8180 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 --2- 172.21.15.157:0/925752913 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00540b1ad0 0x7f0054151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 >> 172.21.15.157:0/925752913 conn(0x7f005406a170 msgr2=0x7f005406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 shutdown_connections 2023-11-06T03:14:13.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.721+0000 7f0058ce9640 1 -- 172.21.15.157:0/925752913 wait complete. 2023-11-06T03:14:13.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:14:13.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub resume 2023-11-06T03:14:13.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 Processor -- start 2023-11-06T03:14:13.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 -- start start 2023-11-06T03:14:13.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fa41551e0 0x7f1fa406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa415f580 con 0x7f1fa415e5e0 2023-11-06T03:14:13.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa415efc0 con 0x7f1fa40b3b20 2023-11-06T03:14:13.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1faa9e3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa415f2a0 con 0x7f1fa41551e0 2023-11-06T03:14:13.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1fa3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fa41551e0 0x7f1fa406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1fa37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1fa415e5e0 0x7f1fa40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58236/0 (socket says 172.21.15.157:58236) 2023-11-06T03:14:13.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.861+0000 7f1fa37fe640 1 -- 172.21.15.157:0/2277954796 learned_addr learned my addr 172.21.15.157:0/2277954796 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:13.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2277954796 >> v1:172.21.15.195:6789/0 conn(0x7f1fa40b3b20 legacy=0x7f1fa415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2277954796 >> v1:172.21.15.157:6789/0 conn(0x7f1fa415e5e0 legacy=0x7f1fa40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2277954796 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa406f5a0 con 0x7f1fa41551e0 2023-11-06T03:14:13.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2277954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41551e0 0x7f1fa406fda0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1f940097c0 tx=0x7f1f9401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8192f761a93e423 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa2ffd640 1 -- 172.21.15.157:0/2277954796 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94004230 con 0x7f1fa41551e0 2023-11-06T03:14:13.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa2ffd640 1 -- 172.21.15.157:0/2277954796 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940a7050 con 0x7f1fa41551e0 2023-11-06T03:14:13.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1fa2ffd640 1 -- 172.21.15.157:0/2277954796 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94003620 con 0x7f1fa41551e0 2023-11-06T03:14:13.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2277954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41551e0 msgr2=0x7f1fa406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2277954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41551e0 0x7f1fa406fda0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1f940097c0 tx=0x7f1f9401c330 comp rx=0 tx=0).stop 2023-11-06T03:14:13.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2277954796 shutdown_connections 2023-11-06T03:14:13.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2277954796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41551e0 0x7f1fa406fda0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2277954796 >> 172.21.15.157:0/2277954796 conn(0x7f1fa4065930 msgr2=0x7f1fa4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2277954796 shutdown_connections 2023-11-06T03:14:13.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.865+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2277954796 wait complete. 2023-11-06T03:14:13.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 Processor -- start 2023-11-06T03:14:13.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- start start 2023-11-06T03:14:13.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa40b3b20 0x7f1fa4135740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa41551e0 0x7f1fa4134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4135420 con 0x7f1fa415e5e0 2023-11-06T03:14:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa414c9f0 con 0x7f1fa40b3b20 2023-11-06T03:14:13.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4135db0 con 0x7f1fa41551e0 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40260/0 (socket says 172.21.15.157:40260) 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2766096240 learned_addr learned my addr 172.21.15.157:0/2766096240 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa40b3b20 0x7f1fa4135740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa41551e0 0x7f1fa4134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa41551e0 msgr2=0x7f1fa4134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa41551e0 0x7f1fa4134ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa40b3b20 msgr2=0x7f1fa4135740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa40b3b20 0x7f1fa4135740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2766096240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94009420 con 0x7f1fa415e5e0 2023-11-06T03:14:13.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c770 tx=0x7f1f9c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa0ff9640 1 -- 172.21.15.157:0/2766096240 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c003b80 con 0x7f1fa415e5e0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa0ff9640 1 -- 172.21.15.157:0/2766096240 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c003ce0 con 0x7f1fa415e5e0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa4136090 con 0x7f1fa415e5e0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa0ff9640 1 -- 172.21.15.157:0/2766096240 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00aa90 con 0x7f1fa415e5e0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4136650 con 0x7f1fa415e5e0 2023-11-06T03:14:13.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.869+0000 7f1fa0ff9640 1 -- 172.21.15.157:0/2766096240 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f9c00abf0 con 0x7f1fa415e5e0 2023-11-06T03:14:13.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa0ff9640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 0x7f1f90067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa0ff9640 1 -- 172.21.15.157:0/2766096240 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f9c083140 con 0x7f1fa415e5e0 2023-11-06T03:14:13.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 0x7f1f90067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 Processor -- start 2023-11-06T03:14:13.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- start start 2023-11-06T03:14:13.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 0x7f1f90067f20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f940ad000 tx=0x7f1f94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c623510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c5d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c4a0c10 con 0x55c86c5d9260 2023-11-06T03:14:13.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c42f3e0 con 0x55c86c623130 2023-11-06T03:14:13.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c86c42c9b0 con 0x55c86c623a50 2023-11-06T03:14:13.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40264/0 (socket says 172.21.15.157:40264) 2023-11-06T03:14:13.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2631367611 learned_addr learned my addr 172.21.15.157:0/2631367611 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c623510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 msgr2=0x55c86c5d8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c5d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c5d8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 msgr2=0x55c86c623510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c623510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 -- 172.21.15.157:0/2631367611 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c86c124e20 con 0x55c86c5d9260 2023-11-06T03:14:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c050b50 tx=0x7f1f9c054550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d26ae9998b2ec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1f89ffb640 1 -- 172.21.15.157:0/2631367611 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c058480 con 0x55c86c5d9260 2023-11-06T03:14:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1f89ffb640 1 -- 172.21.15.157:0/2631367611 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c05c630 con 0x55c86c5d9260 2023-11-06T03:14:13.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1f89ffb640 1 -- 172.21.15.157:0/2631367611 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0529f0 con 0x55c86c5d9260 2023-11-06T03:14:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2631367611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 msgr2=0x55c86c5d9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c050b50 tx=0x7f1f9c054550 comp rx=0 tx=0).stop 2023-11-06T03:14:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2631367611 shutdown_connections 2023-11-06T03:14:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c5d9640 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c5d8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2631367611 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c623510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2631367611 >> 172.21.15.157:0/2631367611 conn(0x55c86c5df9f0 msgr2=0x55c86c5d7600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2631367611 shutdown_connections 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2631367611 wait complete. 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 -- start start 2023-11-06T03:14:13.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.873+0000 7f1fadb0b000 1 Processor -- start 2023-11-06T03:14:13.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c5d9260 0x55c86c62a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c62a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c4282a0 con 0x55c86c623a50 2023-11-06T03:14:13.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c3c7470 con 0x55c86c623130 2023-11-06T03:14:13.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c86c3e5660 con 0x55c86c5d9260 2023-11-06T03:14:13.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 10 client.0 _ll_register_callbacks cb 0x55c86c5c4c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:13.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40270/0 (socket says 172.21.15.157:40270) 2023-11-06T03:14:13.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 -- 172.21.15.157:0/3174812514 learned_addr learned my addr 172.21.15.157:0/3174812514 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c5d9260 0x55c86c62a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c62a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 -- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c5d9260 msgr2=0x55c86c62a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c5d9260 0x55c86c62a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 -- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 msgr2=0x55c86c62a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c62a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 -- 172.21.15.157:0/3174812514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4136650 con 0x55c86c623a50 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c62a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c01a000 tx=0x7f1f9c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737407dabf9114fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c05c630 con 0x55c86c623a50 2023-11-06T03:14:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f9c058480 con 0x55c86c623a50 2023-11-06T03:14:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c86c3e50a0 con 0x55c86c623a50 2023-11-06T03:14:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f9c0720d0 con 0x55c86c623a50 2023-11-06T03:14:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 10 client.9111 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:13.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55c86c3ed1d0 con 0x55c86c623a50 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c86c3e50a0 con 0x55c86c623a50 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1f9c019020 con 0x55c86c623a50 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 10 client.9111 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 10 client.9111 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 20 client.9111 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:13.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c624bd0 0x55c86c624fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 4 client.9111 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:13.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c86c426bf0 con 0x55c86c624bd0 2023-11-06T03:14:13.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c624bd0 0x55c86c624fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c624bd0 0x55c86c624fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f490469b05d986cd server_cookie=4016c1832b9b4134 in_seq=0 out_seq=0 2023-11-06T03:14:13.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 10 client.9111 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 1 -- 172.21.15.157:0/3174812514 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c86c426bf0 con 0x55c86c624bd0 2023-11-06T03:14:13.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 10 client.9111 handle_command_reply: tid=0 2023-11-06T03:14:13.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 client.9111 shutdown 2023-11-06T03:14:13.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 msgr2=0x55c86c623e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c01a000 tx=0x7f1f9c001260 comp rx=0 tx=0).stop 2023-11-06T03:14:13.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 shutdown_connections 2023-11-06T03:14:13.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623a50 0x55c86c623e30 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c624bd0 0x55c86c624fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c62a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/3174812514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c5d9260 0x55c86c62a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1f82ffd640 0 client.9111 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 >> 172.21.15.157:0/3174812514 conn(0x55c86c5df9f0 msgr2=0x55c86c5d7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 shutdown_connections 2023-11-06T03:14:13.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 1 -- 172.21.15.157:0/3174812514 wait complete. 2023-11-06T03:14:13.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.877+0000 7f1fadb0b000 20 client.9111 trim_cache size 0 max 16384 2023-11-06T03:14:13.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 Processor -- start 2023-11-06T03:14:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- start start 2023-11-06T03:14:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c5d9260 0x55c86c67e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c67f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c0e5aa0 con 0x55c86c623130 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c423c10 con 0x55c86c5d9260 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c86c3ebf90 con 0x55c86c623a50 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40286/0 (socket says 172.21.15.157:40286) 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 -- 172.21.15.157:0/2755173154 learned_addr learned my addr 172.21.15.157:0/2755173154 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c5d9260 0x55c86c67e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa8f80640 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c67f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 -- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 msgr2=0x55c86c67f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c67f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 -- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c5d9260 msgr2=0x55c86c67e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c5d9260 0x55c86c67e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 -- 172.21.15.157:0/2755173154 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c86c3d3520 con 0x55c86c623130 2023-11-06T03:14:13.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f1f980105d0 tx=0x7f1f98010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533e9fe11bee469 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1f8a7fc640 1 -- 172.21.15.157:0/2755173154 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98005d90 con 0x55c86c623130 2023-11-06T03:14:13.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1f8a7fc640 1 -- 172.21.15.157:0/2755173154 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9800d040 con 0x55c86c623130 2023-11-06T03:14:13.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1f8a7fc640 1 -- 172.21.15.157:0/2755173154 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f980072f0 con 0x55c86c623130 2023-11-06T03:14:13.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2755173154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 msgr2=0x55c86c67ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f1f980105d0 tx=0x7f1f98010a90 comp rx=0 tx=0).stop 2023-11-06T03:14:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2755173154 shutdown_connections 2023-11-06T03:14:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c67f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c623130 0x55c86c67ed10 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2755173154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c5d9260 0x55c86c67e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2755173154 >> 172.21.15.157:0/2755173154 conn(0x55c86c5df9f0 msgr2=0x55c86c5de9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2755173154 shutdown_connections 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.893+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2755173154 wait complete. 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- start start 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 Processor -- start 2023-11-06T03:14:13.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c667870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c667db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c4256a0 con 0x55c86c5d9260 2023-11-06T03:14:13.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c86c3cd7a0 con 0x55c86c623130 2023-11-06T03:14:13.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c86c6304e0 con 0x55c86c623a50 2023-11-06T03:14:13.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 10 client.0 _ll_register_callbacks cb 0x55c86c3eccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:13.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40298/0 (socket says 172.21.15.157:40298) 2023-11-06T03:14:13.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2475622145 learned_addr learned my addr 172.21.15.157:0/2475622145 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:13.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c667870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 msgr2=0x55c86c667db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c667db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 msgr2=0x55c86c667870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:13.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c667870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 -- 172.21.15.157:0/2475622145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa415f580 con 0x55c86c5d9260 2023-11-06T03:14:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa3fff640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c10 tx=0x7f1f940046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783ac466537efcb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940c9040 con 0x55c86c5d9260 2023-11-06T03:14:13.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f940d3c40 con 0x55c86c5d9260 2023-11-06T03:14:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c86c3f1620 con 0x55c86c5d9260 2023-11-06T03:14:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f940d3e00 con 0x55c86c5d9260 2023-11-06T03:14:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 10 client.9117 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55c86c3c7090 con 0x55c86c5d9260 2023-11-06T03:14:13.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c86c3f1620 con 0x55c86c5d9260 2023-11-06T03:14:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1f940c9af0 con 0x55c86c5d9260 2023-11-06T03:14:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 10 client.9117 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 10 client.9117 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 20 client.9117 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:13.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c6682f0 0x55c86c683a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 4 client.9117 mds_command: new command op to 8796 tid=0[{"prefix": "scrub resume"}] 2023-11-06T03:14:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55c86c421320 con 0x55c86c6682f0 2023-11-06T03:14:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c6682f0 0x55c86c683a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fa37fe640 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c6682f0 0x55c86c683a30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3143d723f4e718b server_cookie=83cf6df0bf3a2bc4 in_seq=0 out_seq=0 2023-11-06T03:14:13.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 10 client.9117 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 1 -- 172.21.15.157:0/2475622145 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c86c421320 con 0x55c86c6682f0 2023-11-06T03:14:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 10 client.9117 handle_command_reply: tid=0 2023-11-06T03:14:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 client.9117 shutdown 2023-11-06T03:14:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 msgr2=0x55c86c629130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000c10 tx=0x7f1f940046d0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 shutdown_connections 2023-11-06T03:14:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c86c623a50 0x55c86c667db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x55c86c6682f0 0x55c86c683a30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c86c623130 0x55c86c667870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 --2- 172.21.15.157:0/2475622145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c86c5d9260 0x55c86c629130 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1f82ffd640 0 client.9117 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 >> 172.21.15.157:0/2475622145 conn(0x55c86c5df9f0 msgr2=0x55c86c5df0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 shutdown_connections 2023-11-06T03:14:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 1 -- 172.21.15.157:0/2475622145 wait complete. 2023-11-06T03:14:13.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1fadb0b000 20 client.9117 trim_cache size 0 max 16384 2023-11-06T03:14:13.917 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:13.917 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:14:13.917 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 msgr2=0x7f1f90067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 0x7f1f90067f20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1f940ad000 tx=0x7f1f94008040 comp rx=0 tx=0).stop 2023-11-06T03:14:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 msgr2=0x7f1fa4136e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c770 tx=0x7f1f9c00cc30 comp rx=0 tx=0).stop 2023-11-06T03:14:13.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 shutdown_connections 2023-11-06T03:14:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f90065a60 0x7f1f90067f20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa415e5e0 0x7f1fa4136e50 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa41551e0 0x7f1fa4134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 --2- 172.21.15.157:0/2766096240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa40b3b20 0x7f1fa4135740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:13.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 >> 172.21.15.157:0/2766096240 conn(0x7f1fa4065930 msgr2=0x7f1fa40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:13.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 shutdown_connections 2023-11-06T03:14:13.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:13.897+0000 7f1faa9e3640 1 -- 172.21.15.157:0/2766096240 wait complete. 2023-11-06T03:14:13.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 scrub status 2023-11-06T03:14:14.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.037+0000 7f0606473640 1 Processor -- start 2023-11-06T03:14:14.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.037+0000 7f0606473640 1 -- start start 2023-11-06T03:14:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f0606473640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06001551e0 0x7f060006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f0606473640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f060015e5e0 0x7f06000b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f0606473640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060015f580 con 0x7f060015ea60 2023-11-06T03:14:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f0606473640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060015efc0 con 0x7f060015e5e0 2023-11-06T03:14:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f0606473640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060015f2a0 con 0x7f06001551e0 2023-11-06T03:14:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f060015e5e0 0x7f06000b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f060015e5e0 0x7f06000b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40302/0 (socket says 172.21.15.157:40302) 2023-11-06T03:14:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 -- 172.21.15.157:0/1778698412 learned_addr learned my addr 172.21.15.157:0/1778698412 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05fffff640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06001551e0 0x7f060006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 -- 172.21.15.157:0/1778698412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06001551e0 msgr2=0x7f060006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06001551e0 0x7f060006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 -- 172.21.15.157:0/1778698412 >> v1:172.21.15.195:6789/0 conn(0x7f060015ea60 legacy=0x7f060015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 -- 172.21.15.157:0/1778698412 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060006f5a0 con 0x7f060015e5e0 2023-11-06T03:14:14.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060015e5e0 0x7f06000b3410 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009870 tx=0x7f05f401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fa8c3f08316d5d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05feffd640 1 -- 172.21.15.157:0/1778698412 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40ad640 con 0x7f060015e5e0 2023-11-06T03:14:14.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05feffd640 1 -- 172.21.15.157:0/1778698412 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40ad7a0 con 0x7f060015e5e0 2023-11-06T03:14:14.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.041+0000 7f05feffd640 1 -- 172.21.15.157:0/1778698412 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40adb30 con 0x7f060015e5e0 2023-11-06T03:14:14.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/1778698412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060015e5e0 msgr2=0x7f06000b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060015e5e0 0x7f06000b3410 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009870 tx=0x7f05f401c480 comp rx=0 tx=0).stop 2023-11-06T03:14:14.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/1778698412 shutdown_connections 2023-11-06T03:14:14.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f060015e5e0 0x7f06000b3410 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- 172.21.15.157:0/1778698412 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06001551e0 0x7f060006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/1778698412 >> 172.21.15.157:0/1778698412 conn(0x7f0600065930 msgr2=0x7f0600065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/1778698412 shutdown_connections 2023-11-06T03:14:14.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/1778698412 wait complete. 2023-11-06T03:14:14.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 Processor -- start 2023-11-06T03:14:14.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- start start 2023-11-06T03:14:14.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f060015e5e0 0x7f0600147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060015ea60 0x7f06000cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0600143b10 con 0x7f06001551e0 2023-11-06T03:14:14.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000ce580 con 0x7f060015ea60 2023-11-06T03:14:14.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060014c770 con 0x7f060015e5e0 2023-11-06T03:14:14.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40312/0 (socket says 172.21.15.157:40312) 2023-11-06T03:14:14.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 -- 172.21.15.157:0/3001788508 learned_addr learned my addr 172.21.15.157:0/3001788508 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f060015e5e0 0x7f0600147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0604a10640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060015ea60 0x7f06000cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 -- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f060015e5e0 msgr2=0x7f0600147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f060015e5e0 0x7f0600147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 -- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060015ea60 msgr2=0x7f06000cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060015ea60 0x7f06000cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 -- 172.21.15.157:0/3001788508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40094d0 con 0x7f06001551e0 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fffff640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d830 tx=0x7f05f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fcff9640 1 -- 172.21.15.157:0/3001788508 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000b6c0 con 0x7f06001551e0 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fcff9640 1 -- 172.21.15.157:0/3001788508 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000b820 con 0x7f06001551e0 2023-11-06T03:14:14.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06000ced70 con 0x7f06001551e0 2023-11-06T03:14:14.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f05fcff9640 1 -- 172.21.15.157:0/3001788508 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000bb90 con 0x7f06001551e0 2023-11-06T03:14:14.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.045+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000ce7b0 con 0x7f06001551e0 2023-11-06T03:14:14.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05fcff9640 1 -- 172.21.15.157:0/3001788508 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f05f001e460 con 0x7f06001551e0 2023-11-06T03:14:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05fcff9640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 0x7f05ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05fcff9640 1 -- 172.21.15.157:0/3001788508 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05f0083a20 con 0x7f06001551e0 2023-11-06T03:14:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 0x7f05ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 Processor -- start 2023-11-06T03:14:14.054 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 -- start start 2023-11-06T03:14:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 0x7f05ec067f20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f05f40af000 tx=0x7f05f40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb326510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb2dc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb1a3c10 con 0x562ceb326a50 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb1323e0 con 0x562ceb326130 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ceb12f9b0 con 0x562ceb2dc260 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40318/0 (socket says 172.21.15.157:40318) 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 -- 172.21.15.157:0/2130530884 learned_addr learned my addr 172.21.15.157:0/2130530884 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05fffff640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb326510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f0604a10640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb2dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 -- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 msgr2=0x562ceb2dc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb2dc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 -- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 msgr2=0x562ceb326510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb326510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 -- 172.21.15.157:0/2130530884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ceae27e20 con 0x562ceb326a50 2023-11-06T03:14:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05fffff640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb326510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000c10 tx=0x7f05f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381cb27ee7a35291 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05e5ffb640 1 -- 172.21.15.157:0/2130530884 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40be070 con 0x562ceb326a50 2023-11-06T03:14:14.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05e5ffb640 1 -- 172.21.15.157:0/2130530884 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40a6040 con 0x562ceb326a50 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.049+0000 7f05e5ffb640 1 -- 172.21.15.157:0/2130530884 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40adea0 con 0x562ceb326a50 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/2130530884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 msgr2=0x562ceb2dbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f05f4000c10 tx=0x7f05f4004740 comp rx=0 tx=0).stop 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/2130530884 shutdown_connections 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb2dc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb2dbd20 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- 172.21.15.157:0/2130530884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb326510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/2130530884 >> 172.21.15.157:0/2130530884 conn(0x562ceb2e29f0 msgr2=0x562ceb2da600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/2130530884 shutdown_connections 2023-11-06T03:14:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/2130530884 wait complete. 2023-11-06T03:14:14.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- start start 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 Processor -- start 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb325b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb327040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb12b2a0 con 0x562ceb326a50 2023-11-06T03:14:14.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb0ca470 con 0x562ceb326130 2023-11-06T03:14:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ceb0e8660 con 0x562ceb2dc260 2023-11-06T03:14:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 10 client.0 _ll_register_callbacks cb 0x562ceb2c7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40320/0 (socket says 172.21.15.157:40320) 2023-11-06T03:14:14.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 -- 172.21.15.157:0/1540194890 learned_addr learned my addr 172.21.15.157:0/1540194890 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb327040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05fffff640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb325b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 -- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 msgr2=0x562ceb325b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb325b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 -- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 msgr2=0x562ceb327040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb327040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 -- 172.21.15.157:0/1540194890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060006f5a0 con 0x562ceb326a50 2023-11-06T03:14:14.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05fffff640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb325b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:14.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f0604a10640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e270 tx=0x7f05f800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1aef7274540da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:14.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8006d60 con 0x562ceb326a50 2023-11-06T03:14:14.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05f80039a0 con 0x562ceb326a50 2023-11-06T03:14:14.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ceb0e80a0 con 0x562ceb326a50 2023-11-06T03:14:14.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05f8003b60 con 0x562ceb326a50 2023-11-06T03:14:14.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 10 client.9129 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:14.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x562ceb0f01d0 con 0x562ceb326a50 2023-11-06T03:14:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8021020 con 0x562ceb326a50 2023-11-06T03:14:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f05f801b020 con 0x562ceb326a50 2023-11-06T03:14:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 10 client.9129 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 10 client.9129 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:14.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 20 client.9129 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:14.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb328320 0x562ceb338460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 4 client.9129 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:14.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ceb129bf0 con 0x562ceb328320 2023-11-06T03:14:14.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05fffff640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb328320 0x562ceb338460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05fffff640 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb328320 0x562ceb338460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee3641dc9f6ac02 server_cookie=55b8dad65f6c6408 in_seq=0 out_seq=0 2023-11-06T03:14:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.053+0000 7f05deffd640 10 client.9129 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f05deffd640 1 -- 172.21.15.157:0/1540194890 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562ceb129bf0 con 0x562ceb328320 2023-11-06T03:14:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f05deffd640 10 client.9129 handle_command_reply: tid=0 2023-11-06T03:14:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 client.9129 shutdown 2023-11-06T03:14:14.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 msgr2=0x562ceb327580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e270 tx=0x7f05f800e730 comp rx=0 tx=0).stop 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 shutdown_connections 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb327580 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb328320 0x562ceb338460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb327040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f05deffd640 0 client.9129 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:14.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 --2- 172.21.15.157:0/1540194890 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb325b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 >> 172.21.15.157:0/1540194890 conn(0x562ceb2e29f0 msgr2=0x562ceb2dad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 shutdown_connections 2023-11-06T03:14:14.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 1 -- 172.21.15.157:0/1540194890 wait complete. 2023-11-06T03:14:14.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.057+0000 7f060959b000 20 client.9129 trim_cache size 0 max 16384 2023-11-06T03:14:14.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 Processor -- start 2023-11-06T03:14:14.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 -- start start 2023-11-06T03:14:14.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb2e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb326a50 0x562ceb338460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceade8aa0 con 0x562ceb2dc260 2023-11-06T03:14:14.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb0fd540 con 0x562ceb326130 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ceb0d07a0 con 0x562ceb326a50 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40332/0 (socket says 172.21.15.157:40332) 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 -- 172.21.15.157:0/349821951 learned_addr learned my addr 172.21.15.157:0/349821951 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 -- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb326a50 msgr2=0x562ceb338460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb326a50 0x562ceb338460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 -- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 msgr2=0x562ceb2e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb2e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.069+0000 7f05fffff640 1 -- 172.21.15.157:0/349821951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ceb0d6520 con 0x562ceb2dc260 2023-11-06T03:14:14.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05fffff640 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f05f00514d0 tx=0x7f05f005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c89e951325da35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05e67fc640 1 -- 172.21.15.157:0/349821951 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f007f070 con 0x562ceb2dc260 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05e67fc640 1 -- 172.21.15.157:0/349821951 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0081f00 con 0x562ceb2dc260 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05e67fc640 1 -- 172.21.15.157:0/349821951 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0053370 con 0x562ceb2dc260 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/349821951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 msgr2=0x562ceb2e1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f05f00514d0 tx=0x7f05f005b040 comp rx=0 tx=0).stop 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/349821951 shutdown_connections 2023-11-06T03:14:14.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb326a50 0x562ceb338460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb2e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- 172.21.15.157:0/349821951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb2dc260 0x562ceb2e1950 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/349821951 >> 172.21.15.157:0/349821951 conn(0x562ceb2e29f0 msgr2=0x562ceb3338b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/349821951 shutdown_connections 2023-11-06T03:14:14.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/349821951 wait complete. 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- start start 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 Processor -- start 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb320760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb320ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb125640 con 0x562ceb326a50 2023-11-06T03:14:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ceb1b3c70 con 0x562ceb326130 2023-11-06T03:14:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ceb1cfbc0 con 0x562ceb2dc260 2023-11-06T03:14:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 10 client.0 _ll_register_callbacks cb 0x562ceb0efcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40336/0 (socket says 172.21.15.157:40336) 2023-11-06T03:14:14.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 -- 172.21.15.157:0/897166316 learned_addr learned my addr 172.21.15.157:0/897166316 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05fffff640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb320760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 -- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 msgr2=0x562ceb320760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb320760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05ff7fe640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb320ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 -- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 msgr2=0x562ceb320ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb320ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 -- 172.21.15.157:0/897166316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000ce7b0 con 0x562ceb326a50 2023-11-06T03:14:14.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f0604a10640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f05f8007d70 tx=0x7f05f8001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca902d85d4fbd95b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:14.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8013670 con 0x562ceb326a50 2023-11-06T03:14:14.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05f8022080 con 0x562ceb326a50 2023-11-06T03:14:14.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ceb0f4620 con 0x562ceb326a50 2023-11-06T03:14:14.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05f800ecb0 con 0x562ceb326a50 2023-11-06T03:14:14.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 10 client.9135 fetch_fsmap learned FSMap version 91 2023-11-06T03:14:14.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x562ceb0ca090 con 0x562ceb326a50 2023-11-06T03:14:14.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ceb0f4620 con 0x562ceb326a50 2023-11-06T03:14:14.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 91) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f05f8024070 con 0x562ceb326a50 2023-11-06T03:14:14.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 10 client.9135 fetch_fsmap finished waiting for FSMap version 91 2023-11-06T03:14:14.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 10 client.9135 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 20 client.9135 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb321720 0x562ceb321b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 4 client.9135 mds_command: new command op to 8796 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 --> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ceb124320 con 0x562ceb321720 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05fffff640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb321720 0x562ceb321b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05deffd640 10 client.9135 ms_handle_connect on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:14.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.073+0000 7f05fffff640 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb321720 0x562ceb321b30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362c81496f06b69d server_cookie=7e441dae5ae2a9b4 in_seq=0 out_seq=0 2023-11-06T03:14:14.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f05deffd640 1 -- 172.21.15.157:0/897166316 <== mds.0 v2:172.21.15.157:6834/3017914039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562ceb124320 con 0x562ceb321720 2023-11-06T03:14:14.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f05deffd640 10 client.9135 handle_command_reply: tid=0 2023-11-06T03:14:14.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 client.9135 shutdown 2023-11-06T03:14:14.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 msgr2=0x562ceb3211e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f05f8007d70 tx=0x7f05f8001410 comp rx=0 tx=0).stop 2023-11-06T03:14:14.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 shutdown_connections 2023-11-06T03:14:14.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ceb326a50 0x562ceb3211e0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.157:6834/3017914039,v1:172.21.15.157:6835/3017914039] conn(0x562ceb321720 0x562ceb321b30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ceb326130 0x562ceb320ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 --2- 172.21.15.157:0/897166316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ceb2dc260 0x562ceb320760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f05deffd640 0 client.9135 ms_handle_reset on v2:172.21.15.157:6834/3017914039 2023-11-06T03:14:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 >> 172.21.15.157:0/897166316 conn(0x562ceb2e29f0 msgr2=0x562ceb2e11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 shutdown_connections 2023-11-06T03:14:14.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 1 -- 172.21.15.157:0/897166316 wait complete. 2023-11-06T03:14:14.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f060959b000 20 client.9135 trim_cache size 0 max 16384 2023-11-06T03:14:14.091 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:14.091 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:14:14.091 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:14:14.091 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:14.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 msgr2=0x7f05ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 0x7f05ec067f20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f05f40af000 tx=0x7f05f40a7070 comp rx=0 tx=0).stop 2023-11-06T03:14:14.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 msgr2=0x7f06001431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f05f000d830 tx=0x7f05f000dcf0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 shutdown_connections 2023-11-06T03:14:14.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05ec065a60 0x7f05ec067f20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f060015ea60 0x7f06000cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f060015e5e0 0x7f0600147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 --2- 172.21.15.157:0/3001788508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06001551e0 0x7f06001431e0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 >> 172.21.15.157:0/3001788508 conn(0x7f0600065930 msgr2=0x7f06000aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 shutdown_connections 2023-11-06T03:14:14.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.077+0000 7f0606473640 1 -- 172.21.15.157:0/3001788508 wait complete. 2023-11-06T03:14:14.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:14:14.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.209+0000 7f583e81d640 1 Processor -- start 2023-11-06T03:14:14.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.209+0000 7f583e81d640 1 -- start start 2023-11-06T03:14:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58380b2af0 0x7f58380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583815e440 con 0x7f58381599c0 2023-11-06T03:14:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583806c240 con 0x7f583806d000 2023-11-06T03:14:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583806c520 con 0x7f58380b2af0 2023-11-06T03:14:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58380b2af0 0x7f58380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58380b2af0 0x7f58380ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40340/0 (socket says 172.21.15.157:40340) 2023-11-06T03:14:14.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 -- 172.21.15.157:0/4086330618 learned_addr learned my addr 172.21.15.157:0/4086330618 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 -- 172.21.15.157:0/4086330618 >> v1:172.21.15.195:6789/0 conn(0x7f583806d000 legacy=0x7f5838159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 -- 172.21.15.157:0/4086330618 >> v1:172.21.15.157:6789/0 conn(0x7f58381599c0 legacy=0x7f58380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 -- 172.21.15.157:0/4086330618 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583806c800 con 0x7f58380b2af0 2023-11-06T03:14:14.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f58377fe640 1 --2- 172.21.15.157:0/4086330618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f58380ae000 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5820009870 tx=0x7f582001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f34db8011b020d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f5836ffd640 1 -- 172.21.15.157:0/4086330618 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200aa650 con 0x7f58380b2af0 2023-11-06T03:14:14.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f5836ffd640 1 -- 172.21.15.157:0/4086330618 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58200aa7b0 con 0x7f58380b2af0 2023-11-06T03:14:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f5836ffd640 1 -- 172.21.15.157:0/4086330618 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200aab40 con 0x7f58380b2af0 2023-11-06T03:14:14.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 -- 172.21.15.157:0/4086330618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 msgr2=0x7f58380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.213+0000 7f583e81d640 1 --2- 172.21.15.157:0/4086330618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f58380ae000 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5820009870 tx=0x7f582001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/4086330618 shutdown_connections 2023-11-06T03:14:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 --2- 172.21.15.157:0/4086330618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f58380ae000 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/4086330618 >> 172.21.15.157:0/4086330618 conn(0x7f5838065930 msgr2=0x7f5838065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/4086330618 shutdown_connections 2023-11-06T03:14:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/4086330618 wait complete. 2023-11-06T03:14:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 Processor -- start 2023-11-06T03:14:14.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- start start 2023-11-06T03:14:14.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 0x7f583813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 0x7f583814fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838153760 con 0x7f58380b2af0 2023-11-06T03:14:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838154700 con 0x7f58381599c0 2023-11-06T03:14:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5838153940 con 0x7f583806d000 2023-11-06T03:14:14.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40346/0 (socket says 172.21.15.157:40346) 2023-11-06T03:14:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 -- 172.21.15.157:0/256912263 learned_addr learned my addr 172.21.15.157:0/256912263 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 -- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 msgr2=0x7f583813d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583cdba640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 0x7f583814fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f5837fff640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 0x7f583813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 0x7f583813d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 -- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 msgr2=0x7f583814fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 0x7f583814fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 -- 172.21.15.157:0/256912263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200094d0 con 0x7f58380b2af0 2023-11-06T03:14:14.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583cdba640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 0x7f583814fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f5837fff640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 0x7f583813d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f58377fe640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f58200190e0 tx=0x7f582001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200b3800 con 0x7f58380b2af0 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58200b3980 con 0x7f58380b2af0 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200b3d30 con 0x7f58380b2af0 2023-11-06T03:14:14.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838153c20 con 0x7f58380b2af0 2023-11-06T03:14:14.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381541e0 con 0x7f58380b2af0 2023-11-06T03:14:14.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.217+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58381543d0 con 0x7f58380b2af0 2023-11-06T03:14:14.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.221+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f58200b2380 con 0x7f58380b2af0 2023-11-06T03:14:14.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.221+0000 7f5834ff9640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 0x7f5818067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.221+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f582011db20 con 0x7f58380b2af0 2023-11-06T03:14:14.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.221+0000 7f5837fff640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 0x7f5818067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.221+0000 7f5837fff640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 0x7f5818067f20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5828000f70 tx=0x7f5828009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.225+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58200eb4c0 con 0x7f58380b2af0 2023-11-06T03:14:14.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.405+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5838153a80 con 0x7f58380b2af0 2023-11-06T03:14:14.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.405+0000 7f5834ff9640 1 -- 172.21.15.157:0/256912263 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f5838153a80 con 0x7f58380b2af0 2023-11-06T03:14:14.403 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:14.403 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:14.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 msgr2=0x7f5818067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 0x7f5818067f20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5828000f70 tx=0x7f5828009040 comp rx=0 tx=0).stop 2023-11-06T03:14:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 msgr2=0x7f583814f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f58200190e0 tx=0x7f582001ad10 comp rx=0 tx=0).stop 2023-11-06T03:14:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 shutdown_connections 2023-11-06T03:14:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5818065a60 0x7f5818067f20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58381599c0 0x7f583814fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58380b2af0 0x7f583814f650 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 --2- 172.21.15.157:0/256912263 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f583806d000 0x7f583813d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 >> 172.21.15.157:0/256912263 conn(0x7f5838065930 msgr2=0x7f583815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 shutdown_connections 2023-11-06T03:14:14.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.409+0000 7f583e81d640 1 -- 172.21.15.157:0/256912263 wait complete. 2023-11-06T03:14:14.420 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:14.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:14.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 Processor -- start 2023-11-06T03:14:14.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 -- start start 2023-11-06T03:14:14.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f01551e0 0x7ff9f006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9f00b3b20 0x7ff9f015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f015f580 con 0x7ff9f015e5e0 2023-11-06T03:14:14.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f015efc0 con 0x7ff9f00b3b20 2023-11-06T03:14:14.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9f4bc8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f015f2a0 con 0x7ff9f01551e0 2023-11-06T03:14:14.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9eed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9f00b3b20 0x7ff9f015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9edd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff9f015e5e0 0x7ff9f00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58244/0 (socket says 172.21.15.157:58244) 2023-11-06T03:14:14.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9edd9b640 1 -- 172.21.15.157:0/353878517 learned_addr learned my addr 172.21.15.157:0/353878517 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:14.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9ee59c640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f01551e0 0x7ff9f006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9eed9d640 1 -- 172.21.15.157:0/353878517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f01551e0 msgr2=0x7ff9f006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:14.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9eed9d640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f01551e0 0x7ff9f006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9ed59a640 1 -- 172.21.15.157:0/353878517 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297236737 0 0) 0x7ff9f015f580 con 0x7ff9f015e5e0 2023-11-06T03:14:14.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9eed9d640 1 -- 172.21.15.157:0/353878517 >> v1:172.21.15.157:6789/0 conn(0x7ff9f015e5e0 legacy=0x7ff9f00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.541+0000 7ff9eed9d640 1 -- 172.21.15.157:0/353878517 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f006f5a0 con 0x7ff9f00b3b20 2023-11-06T03:14:14.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9eed9d640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f00b3b20 0x7ff9f015c1b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe3a2cecc20d159 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9ed59a640 1 -- 172.21.15.157:0/353878517 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80a5970 con 0x7ff9f00b3b20 2023-11-06T03:14:14.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9ed59a640 1 -- 172.21.15.157:0/353878517 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80a5ad0 con 0x7ff9f00b3b20 2023-11-06T03:14:14.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9ed59a640 1 -- 172.21.15.157:0/353878517 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80a5e60 con 0x7ff9f00b3b20 2023-11-06T03:14:14.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/353878517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f00b3b20 msgr2=0x7ff9f015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f00b3b20 0x7ff9f015c1b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e801c890 comp rx=0 tx=0).stop 2023-11-06T03:14:14.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/353878517 shutdown_connections 2023-11-06T03:14:14.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f00b3b20 0x7ff9f015c1b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/353878517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f01551e0 0x7ff9f006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/353878517 >> 172.21.15.157:0/353878517 conn(0x7ff9f0065930 msgr2=0x7ff9f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/353878517 shutdown_connections 2023-11-06T03:14:14.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/353878517 wait complete. 2023-11-06T03:14:14.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 Processor -- start 2023-11-06T03:14:14.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.545+0000 7ff9f4bc8640 1 -- start start 2023-11-06T03:14:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f00b3b20 0x7ff9f014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9f015e5e0 0x7ff9f014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0150680 con 0x7ff9f01551e0 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01500c0 con 0x7ff9f015e5e0 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01503a0 con 0x7ff9f00b3b20 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40370/0 (socket says 172.21.15.157:40370) 2023-11-06T03:14:14.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 -- 172.21.15.157:0/2314254948 learned_addr learned my addr 172.21.15.157:0/2314254948 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:14.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 -- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f00b3b20 msgr2=0x7ff9f014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f00b3b20 0x7ff9f014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 -- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9f015e5e0 msgr2=0x7ff9f014d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9f015e5e0 0x7ff9f014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 -- 172.21.15.157:0/2314254948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009a70 con 0x7ff9f01551e0 2023-11-06T03:14:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9edd9b640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400c480 tx=0x7ff9e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40074e0 con 0x7ff9f01551e0 2023-11-06T03:14:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e4007660 con 0x7ff9f01551e0 2023-11-06T03:14:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f014a350 con 0x7ff9f01551e0 2023-11-06T03:14:14.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4004a30 con 0x7ff9f01551e0 2023-11-06T03:14:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f0148f10 con 0x7ff9f01551e0 2023-11-06T03:14:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9e4004b90 con 0x7ff9f01551e0 2023-11-06T03:14:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f014a1b0 con 0x7ff9f01551e0 2023-11-06T03:14:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 0x7ff9d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:14.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9e4082420 con 0x7ff9f01551e0 2023-11-06T03:14:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.549+0000 7ff9ee59c640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 0x7ff9d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.553+0000 7ff9ee59c640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 0x7ff9d0067f20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9f015a9c0 tx=0x7ff9e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:14.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.553+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff9f014a1b0 con 0x7ff9f01551e0 2023-11-06T03:14:14.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.765+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff9f014a1b0 con 0x7ff9f01551e0 2023-11-06T03:14:14.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.765+0000 7ff9deffd640 1 -- 172.21.15.157:0/2314254948 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1632 (secure 0 0 0) 0x7ff9f014a1b0 con 0x7ff9f01551e0 2023-11-06T03:14:14.763 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:14.764 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":632,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1021,"data_bytes":1048572774,"bytes_used":2677878784,"bytes_avail":770416234496,"bytes_total":773094113280,"write_bytes_sec":52940305,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-11-06T03:14:11.316119+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:14.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 msgr2=0x7ff9d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 0x7ff9d0067f20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9f015a9c0 tx=0x7ff9e0008040 comp rx=0 tx=0).stop 2023-11-06T03:14:14.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 msgr2=0x7ff9f0152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:14.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400c480 tx=0x7ff9e400c940 comp rx=0 tx=0).stop 2023-11-06T03:14:14.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 shutdown_connections 2023-11-06T03:14:14.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff9d0065a60 0x7ff9d0067f20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9f015e5e0 0x7ff9f014d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9f01551e0 0x7ff9f0152170 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 --2- 172.21.15.157:0/2314254948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9f00b3b20 0x7ff9f014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 >> 172.21.15.157:0/2314254948 conn(0x7ff9f0065930 msgr2=0x7ff9f015dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:14.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 shutdown_connections 2023-11-06T03:14:14.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:14.769+0000 7ff9f4bc8640 1 -- 172.21.15.157:0/2314254948 wait complete. 2023-11-06T03:14:14.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:14:15.783 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:15.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.913+0000 7f22cfee7640 1 Processor -- start 2023-11-06T03:14:15.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 -- start start 2023-11-06T03:14:15.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22c81551e0 0x7f22c806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 0x7f22c815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c815f580 con 0x7f22c815e5e0 2023-11-06T03:14:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c815efc0 con 0x7f22c81551e0 2023-11-06T03:14:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cfee7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815f2a0 con 0x7f22c80b3b20 2023-11-06T03:14:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22c81551e0 0x7f22c806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22c81551e0 0x7f22c806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40376/0 (socket says 172.21.15.157:40376) 2023-11-06T03:14:15.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cd482640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f22c815e5e0 0x7f22c80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58254/0 (socket says 172.21.15.157:58254) 2023-11-06T03:14:15.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 -- 172.21.15.157:0/2489334896 learned_addr learned my addr 172.21.15.157:0/2489334896 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:15.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22ce484640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 0x7f22c815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 -- 172.21.15.157:0/2489334896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 msgr2=0x7f22c815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:15.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 0x7f22c815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:15.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 -- 172.21.15.157:0/2489334896 >> v1:172.21.15.157:6789/0 conn(0x7f22c815e5e0 legacy=0x7f22c80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:15.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22cdc83640 1 -- 172.21.15.157:0/2489334896 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c806f5a0 con 0x7f22c81551e0 2023-11-06T03:14:15.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22ccc81640 1 -- 172.21.15.157:0/2489334896 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842741904 0 0) 0x7f22c815f580 con 0x7f22c815e5e0 2023-11-06T03:14:15.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.917+0000 7f22ce484640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 0x7f22c815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cdc83640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806fda0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009870 tx=0x7f22c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22d8fef11d6502a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22ccc81640 1 -- 172.21.15.157:0/2489334896 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c40a6040 con 0x7f22c81551e0 2023-11-06T03:14:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22ccc81640 1 -- 172.21.15.157:0/2489334896 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c40aa600 con 0x7f22c81551e0 2023-11-06T03:14:15.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22ccc81640 1 -- 172.21.15.157:0/2489334896 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c40aa990 con 0x7f22c81551e0 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- 172.21.15.157:0/2489334896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 msgr2=0x7f22c806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806fda0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009870 tx=0x7f22c401c480 comp rx=0 tx=0).stop 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- 172.21.15.157:0/2489334896 shutdown_connections 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c80b3b20 0x7f22c815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- 172.21.15.157:0/2489334896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806fda0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:15.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- 172.21.15.157:0/2489334896 >> 172.21.15.157:0/2489334896 conn(0x7f22c8065930 msgr2=0x7f22c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:15.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- 172.21.15.157:0/2489334896 shutdown_connections 2023-11-06T03:14:15.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- 172.21.15.157:0/2489334896 wait complete. 2023-11-06T03:14:15.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 Processor -- start 2023-11-06T03:14:15.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- start start 2023-11-06T03:14:15.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 0x7f22c81535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 0x7f22c806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8151f20 con 0x7f22c81551e0 2023-11-06T03:14:15.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8152170 con 0x7f22c80b3b20 2023-11-06T03:14:15.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cfee7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c806b630 con 0x7f22c815e5e0 2023-11-06T03:14:15.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40386/0 (socket says 172.21.15.157:40386) 2023-11-06T03:14:15.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 -- 172.21.15.157:0/3981826129 learned_addr learned my addr 172.21.15.157:0/3981826129 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:15.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cdc83640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 0x7f22c81535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22ce484640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 0x7f22c806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 -- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 msgr2=0x7f22c806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:15.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 0x7f22c806c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:15.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 -- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 msgr2=0x7f22c81535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:15.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 0x7f22c81535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:15.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 -- 172.21.15.157:0/3981826129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c40094d0 con 0x7f22c81551e0 2023-11-06T03:14:15.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22ce484640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 0x7f22c806c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:15.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cdc83640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 0x7f22c81535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:15.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.921+0000 7f22cd482640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c480 tx=0x7f22b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:15.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b80074e0 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8007660 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c806b910 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8004a30 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80ae2e0 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c806b490 con 0x7f22c81551e0 2023-11-06T03:14:15.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22b8004b90 con 0x7f22c81551e0 2023-11-06T03:14:15.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 0x7f22ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:15.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22b8082780 con 0x7f22c81551e0 2023-11-06T03:14:15.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22cdc83640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 0x7f22ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:15.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.925+0000 7f22cdc83640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 0x7f22ac067f20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f22c40ae000 tx=0x7f22c40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:15.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:15.929+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f22c806b490 con 0x7f22c81551e0 2023-11-06T03:14:16.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22c806b490 con 0x7f22c81551e0 2023-11-06T03:14:16.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22be7fc640 1 -- 172.21.15.157:0/3981826129 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f22c806b490 con 0x7f22c81551e0 2023-11-06T03:14:16.125 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:16.126 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:16.126 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:16.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 msgr2=0x7f22ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 0x7f22ac067f20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f22c40ae000 tx=0x7f22c40a7000 comp rx=0 tx=0).stop 2023-11-06T03:14:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 msgr2=0x7f22c806be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c480 tx=0x7f22b800c940 comp rx=0 tx=0).stop 2023-11-06T03:14:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 shutdown_connections 2023-11-06T03:14:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f22ac065a60 0x7f22ac067f20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f22c815e5e0 0x7f22c806c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22c81551e0 0x7f22c806be30 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.128+0000 7f22cfee7640 1 --2- 172.21.15.157:0/3981826129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22c80b3b20 0x7f22c81535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.132+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 >> 172.21.15.157:0/3981826129 conn(0x7f22c8065930 msgr2=0x7f22c80ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.132+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 shutdown_connections 2023-11-06T03:14:16.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.132+0000 7f22cfee7640 1 -- 172.21.15.157:0/3981826129 wait complete. 2023-11-06T03:14:16.145 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:16.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.268+0000 7fa7abf07640 1 Processor -- start 2023-11-06T03:14:16.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.268+0000 7fa7abf07640 1 -- start start 2023-11-06T03:14:16.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7a415caa0 0x7fa7a40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa7a40d0280 0x7fa7a40b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a40d2920 con 0x7fa7a415ce80 2023-11-06T03:14:16.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a40d18a0 con 0x7fa7a415caa0 2023-11-06T03:14:16.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a40d1b60 con 0x7fa7a40d0280 2023-11-06T03:14:16.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7a415caa0 0x7fa7a40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa7a415caa0 0x7fa7a40cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40390/0 (socket says 172.21.15.157:40390) 2023-11-06T03:14:16.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/594941356 learned_addr learned my addr 172.21.15.157:0/594941356 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:16.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/594941356 >> v2:172.21.15.195:3300/0 conn(0x7fa7a40d0280 msgr2=0x7fa7a40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:16.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7aa4a4640 1 --2- 172.21.15.157:0/594941356 >> v2:172.21.15.195:3300/0 conn(0x7fa7a40d0280 0x7fa7a40b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 --2- 172.21.15.157:0/594941356 >> v2:172.21.15.195:3300/0 conn(0x7fa7a40d0280 0x7fa7a40b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/594941356 >> v1:172.21.15.195:6789/0 conn(0x7fa7a415ce80 legacy=0x7fa7a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:16.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/594941356 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40d1e40 con 0x7fa7a415caa0 2023-11-06T03:14:16.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a9ca3640 1 --2- 172.21.15.157:0/594941356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a415caa0 0x7fa7a40cdc60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40cf670 tx=0x7fa79401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045c45567a45d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a8ca1640 1 -- 172.21.15.157:0/594941356 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940aa5f0 con 0x7fa7a415caa0 2023-11-06T03:14:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a8ca1640 1 -- 172.21.15.157:0/594941356 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7940aa750 con 0x7fa7a415caa0 2023-11-06T03:14:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7a8ca1640 1 -- 172.21.15.157:0/594941356 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940aaae0 con 0x7fa7a415caa0 2023-11-06T03:14:16.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- 172.21.15.157:0/594941356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a415caa0 msgr2=0x7fa7a40cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 --2- 172.21.15.157:0/594941356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a415caa0 0x7fa7a40cdc60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40cf670 tx=0x7fa79401c490 comp rx=0 tx=0).stop 2023-11-06T03:14:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- 172.21.15.157:0/594941356 shutdown_connections 2023-11-06T03:14:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 --2- 172.21.15.157:0/594941356 >> v2:172.21.15.195:3300/0 conn(0x7fa7a40d0280 0x7fa7a40b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 --2- 172.21.15.157:0/594941356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a415caa0 0x7fa7a40cdc60 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- 172.21.15.157:0/594941356 >> 172.21.15.157:0/594941356 conn(0x7fa7a4070790 msgr2=0x7fa7a4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.272+0000 7fa7abf07640 1 -- 172.21.15.157:0/594941356 shutdown_connections 2023-11-06T03:14:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- 172.21.15.157:0/594941356 wait complete. 2023-11-06T03:14:16.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 Processor -- start 2023-11-06T03:14:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- start start 2023-11-06T03:14:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7a415caa0 0x7fa7a415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a415ce80 0x7fa7a4154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c7530 con 0x7fa7a40d0280 2023-11-06T03:14:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40cb610 con 0x7fa7a415ce80 2023-11-06T03:14:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a40c7250 con 0x7fa7a415caa0 2023-11-06T03:14:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40404/0 (socket says 172.21.15.157:40404) 2023-11-06T03:14:16.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/449905028 learned_addr learned my addr 172.21.15.157:0/449905028 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7aa4a4640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a415ce80 0x7fa7a4154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a94a2640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7a415caa0 0x7fa7a415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7a415caa0 msgr2=0x7fa7a415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7a415caa0 0x7fa7a415b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a415ce80 msgr2=0x7fa7a4154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a415ce80 0x7fa7a4154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 -- 172.21.15.157:0/449905028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940094d0 con 0x7fa7a40d0280 2023-11-06T03:14:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7a9ca3640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa7940ac040 tx=0x7fa79401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:16.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940b3800 con 0x7fa7a40d0280 2023-11-06T03:14:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7940b3960 con 0x7fa7a40d0280 2023-11-06T03:14:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7940b3cf0 con 0x7fa7a40d0280 2023-11-06T03:14:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a40cafc0 con 0x7fa7a40d0280 2023-11-06T03:14:16.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a415d2c0 con 0x7fa7a40d0280 2023-11-06T03:14:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.276+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4072750 con 0x7fa7a40d0280 2023-11-06T03:14:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.280+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7940b2380 con 0x7fa7a40d0280 2023-11-06T03:14:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.280+0000 7fa79a7fc640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 0x7fa788067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.280+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7940a7070 con 0x7fa7a40d0280 2023-11-06T03:14:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.280+0000 7fa7a94a2640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 0x7fa788067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:16.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.280+0000 7fa7a94a2640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 0x7fa788067f20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4160430 tx=0x7fa7a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:16.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.284+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa7940eb220 con 0x7fa7a40d0280 2023-11-06T03:14:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.492+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa7a40b51f0 con 0x7fa7a40d0280 2023-11-06T03:14:16.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa79a7fc640 1 -- 172.21.15.157:0/449905028 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1633 (secure 0 0 0) 0x7fa7940c3680 con 0x7fa7a40d0280 2023-11-06T03:14:16.493 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:16.493 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1077956881,"bytes_used":2761814016,"bytes_avail":770332299264,"bytes_total":773094113280,"write_bytes_sec":55421645,"read_op_per_sec":0,"write_op_per_sec":170},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2023-11-06T03:14:11.316119+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:16.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 msgr2=0x7fa788067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 0x7fa788067f20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4160430 tx=0x7fa7a0009040 comp rx=0 tx=0).stop 2023-11-06T03:14:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 msgr2=0x7fa7a40c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa7940ac040 tx=0x7fa79401acd0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 shutdown_connections 2023-11-06T03:14:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa788065a60 0x7fa788067f20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7a415ce80 0x7fa7a4154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7a415caa0 0x7fa7a415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 --2- 172.21.15.157:0/449905028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a40d0280 0x7fa7a40c68e0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:16.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 >> 172.21.15.157:0/449905028 conn(0x7fa7a4070790 msgr2=0x7fa7a40b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:16.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 shutdown_connections 2023-11-06T03:14:16.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:16.496+0000 7fa7abf07640 1 -- 172.21.15.157:0/449905028 wait complete. 2023-11-06T03:14:16.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:14:17.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:17.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 Processor -- start 2023-11-06T03:14:17.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 -- start start 2023-11-06T03:14:17.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f00cc15e5e0 0x7f00cc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc15ea90 0x7f00cc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc15f580 con 0x7f00cc1551e0 2023-11-06T03:14:17.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0704b0 con 0x7f00cc15e5e0 2023-11-06T03:14:17.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d2774640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc15efc0 con 0x7f00cc15ea90 2023-11-06T03:14:17.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f00cc1551e0 0x7f00cc06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:58268/0 (socket says 172.21.15.157:58268) 2023-11-06T03:14:17.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cbfff640 1 -- 172.21.15.157:0/549740323 learned_addr learned my addr 172.21.15.157:0/549740323 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:17.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/549740323 >> v2:172.21.15.157:3300/0 conn(0x7f00cc15e5e0 0x7f00cc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00d0d11640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc15ea90 0x7f00cc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00caffd640 1 -- 172.21.15.157:0/549740323 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849659743 0 0) 0x7f00cc15f580 con 0x7f00cc1551e0 2023-11-06T03:14:17.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00caffd640 1 -- 172.21.15.157:0/549740323 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b8003530 con 0x7f00cc1551e0 2023-11-06T03:14:17.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cb7fe640 1 -- 172.21.15.157:0/549740323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc15ea90 msgr2=0x7f00cc0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:17.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc15ea90 0x7f00cc0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cb7fe640 1 -- 172.21.15.157:0/549740323 >> v1:172.21.15.157:6789/0 conn(0x7f00cc1551e0 legacy=0x7f00cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.648+0000 7f00cb7fe640 1 -- 172.21.15.157:0/549740323 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc15f2a0 con 0x7f00cc15e5e0 2023-11-06T03:14:17.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc0b3410 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009d90 tx=0x7f00b401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b82c9e9e1217983 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00caffd640 1 -- 172.21.15.157:0/549740323 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40a5950 con 0x7f00cc15e5e0 2023-11-06T03:14:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00caffd640 1 -- 172.21.15.157:0/549740323 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b40a5ab0 con 0x7f00cc15e5e0 2023-11-06T03:14:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00caffd640 1 -- 172.21.15.157:0/549740323 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40a5e20 con 0x7f00cc15e5e0 2023-11-06T03:14:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- 172.21.15.157:0/549740323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 msgr2=0x7f00cc0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc0b3410 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009d90 tx=0x7f00b401c890 comp rx=0 tx=0).stop 2023-11-06T03:14:17.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- 172.21.15.157:0/549740323 shutdown_connections 2023-11-06T03:14:17.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc15ea90 0x7f00cc0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- 172.21.15.157:0/549740323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc0b3410 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- 172.21.15.157:0/549740323 >> 172.21.15.157:0/549740323 conn(0x7f00cc065930 msgr2=0x7f00cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:17.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- 172.21.15.157:0/549740323 shutdown_connections 2023-11-06T03:14:17.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- 172.21.15.157:0/549740323 wait complete. 2023-11-06T03:14:17.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 Processor -- start 2023-11-06T03:14:17.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- start start 2023-11-06T03:14:17.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc1551e0 0x7f00cc14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc15ea90 0x7f00cc13a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f9790 con 0x7f00cc15e5e0 2023-11-06T03:14:17.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc13e400 con 0x7f00cc15ea90 2023-11-06T03:14:17.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00d2774640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc14f680 con 0x7f00cc1551e0 2023-11-06T03:14:17.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.652+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40428/0 (socket says 172.21.15.157:40428) 2023-11-06T03:14:17.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 -- 172.21.15.157:0/2375095497 learned_addr learned my addr 172.21.15.157:0/2375095497 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:17.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 -- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc1551e0 msgr2=0x7f00cc14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc1551e0 0x7f00cc14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 -- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc15ea90 msgr2=0x7f00cc13a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00d0d11640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc15ea90 0x7f00cc13a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc15ea90 0x7f00cc13a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 -- 172.21.15.157:0/2375095497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4009a70 con 0x7f00cc15e5e0 2023-11-06T03:14:17.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cb7fe640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f00b401af40 tx=0x7f00b4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40aee00 con 0x7f00cc15e5e0 2023-11-06T03:14:17.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4009450 con 0x7f00cc15e5e0 2023-11-06T03:14:17.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b40b7610 con 0x7f00cc15e5e0 2023-11-06T03:14:17.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc13de90 con 0x7f00cc15e5e0 2023-11-06T03:14:17.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc13a250 con 0x7f00cc15e5e0 2023-11-06T03:14:17.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc13a040 con 0x7f00cc15e5e0 2023-11-06T03:14:17.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f00b40b4050 con 0x7f00cc15e5e0 2023-11-06T03:14:17.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 0x7f00ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00b411e860 con 0x7f00cc15e5e0 2023-11-06T03:14:17.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.656+0000 7f00cbfff640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 0x7f00ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.660+0000 7f00cbfff640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 0x7f00ac067f20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f00c4007fd0 tx=0x7f00c4007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.660+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f00b40ec1e0 con 0x7f00cc15e5e0 2023-11-06T03:14:17.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.840+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00cc13de90 con 0x7f00cc15e5e0 2023-11-06T03:14:17.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00c8ff9640 1 -- 172.21.15.157:0/2375095497 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f00cc13de90 con 0x7f00cc15e5e0 2023-11-06T03:14:17.840 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:17.841 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:17.841 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:17.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 msgr2=0x7f00ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 0x7f00ac067f20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f00c4007fd0 tx=0x7f00c4007900 comp rx=0 tx=0).stop 2023-11-06T03:14:17.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 msgr2=0x7f00cc13d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f00b401af40 tx=0x7f00b4019630 comp rx=0 tx=0).stop 2023-11-06T03:14:17.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 shutdown_connections 2023-11-06T03:14:17.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f00ac065a60 0x7f00ac067f20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc15ea90 0x7f00cc13a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00cc15e5e0 0x7f00cc13d950 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 --2- 172.21.15.157:0/2375095497 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00cc1551e0 0x7f00cc14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 >> 172.21.15.157:0/2375095497 conn(0x7f00cc065930 msgr2=0x7f00cc1543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:17.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 shutdown_connections 2023-11-06T03:14:17.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.844+0000 7f00d2774640 1 -- 172.21.15.157:0/2375095497 wait complete. 2023-11-06T03:14:17.858 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:14:17.858 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:14:17.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 Processor -- start 2023-11-06T03:14:17.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 -- start start 2023-11-06T03:14:17.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 0x7f77d00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77d015ce80 0x7f77d00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00d2920 con 0x7f77d00ce1a0 2023-11-06T03:14:17.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d00d18a0 con 0x7f77d015ce80 2023-11-06T03:14:17.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.976+0000 7f77d8303640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00d1b60 con 0x7f77d015caa0 2023-11-06T03:14:17.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77d015ce80 0x7f77d00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77d015ce80 0x7f77d00cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40430/0 (socket says 172.21.15.157:40430) 2023-11-06T03:14:17.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 -- 172.21.15.157:0/1423277001 learned_addr learned my addr 172.21.15.157:0/1423277001 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:17.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d609f640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 0x7f77d00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 -- 172.21.15.157:0/1423277001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 msgr2=0x7f77d00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:17.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 0x7f77d00cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 -- 172.21.15.157:0/1423277001 >> v1:172.21.15.195:6789/0 conn(0x7f77d00ce1a0 legacy=0x7f77d00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 -- 172.21.15.157:0/1423277001 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00d1e40 con 0x7f77d015ce80 2023-11-06T03:14:17.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d609f640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 0x7f77d00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d589e640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015ce80 0x7f77d00cfb70 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009870 tx=0x7f77c001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90d970f123db60e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d509d640 1 -- 172.21.15.157:0/1423277001 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00ab620 con 0x7f77d015ce80 2023-11-06T03:14:17.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d509d640 1 -- 172.21.15.157:0/1423277001 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c00ab780 con 0x7f77d015ce80 2023-11-06T03:14:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d509d640 1 -- 172.21.15.157:0/1423277001 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00abb10 con 0x7f77d015ce80 2023-11-06T03:14:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 -- 172.21.15.157:0/1423277001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015ce80 msgr2=0x7f77d00cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015ce80 0x7f77d00cfb70 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009870 tx=0x7f77c001c510 comp rx=0 tx=0).stop 2023-11-06T03:14:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 -- 172.21.15.157:0/1423277001 shutdown_connections 2023-11-06T03:14:17.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015ce80 0x7f77d00cfb70 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 --2- 172.21.15.157:0/1423277001 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d015caa0 0x7f77d00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 -- 172.21.15.157:0/1423277001 >> 172.21.15.157:0/1423277001 conn(0x7f77d0070790 msgr2=0x7f77d0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 -- 172.21.15.157:0/1423277001 shutdown_connections 2023-11-06T03:14:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.980+0000 7f77d8303640 1 -- 172.21.15.157:0/1423277001 wait complete. 2023-11-06T03:14:17.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 Processor -- start 2023-11-06T03:14:17.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- start start 2023-11-06T03:14:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d00ce1a0 0x7f77d0154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77d015ce80 0x7f77d015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0155690 con 0x7f77d015caa0 2023-11-06T03:14:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d013b7f0 con 0x7f77d015ce80 2023-11-06T03:14:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01553b0 con 0x7f77d00ce1a0 2023-11-06T03:14:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40440/0 (socket says 172.21.15.157:40440) 2023-11-06T03:14:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 -- 172.21.15.157:0/815514862 learned_addr learned my addr 172.21.15.157:0/815514862 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d68a0640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77d015ce80 0x7f77d015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d609f640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d00ce1a0 0x7f77d0154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 -- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d00ce1a0 msgr2=0x7f77d0154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d00ce1a0 0x7f77d0154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 -- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77d015ce80 msgr2=0x7f77d015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77d015ce80 0x7f77d015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 -- 172.21.15.157:0/815514862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00094d0 con 0x7f77d015caa0 2023-11-06T03:14:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d589e640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f77c00ad040 tx=0x7f77c00192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00bf410 con 0x7f77d015caa0 2023-11-06T03:14:17.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c00bf5b0 con 0x7f77d015caa0 2023-11-06T03:14:17.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d013a7d0 con 0x7f77d015caa0 2023-11-06T03:14:17.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00bf960 con 0x7f77d015caa0 2023-11-06T03:14:17.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d013ad90 con 0x7f77d015caa0 2023-11-06T03:14:17.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d0072750 con 0x7f77d015caa0 2023-11-06T03:14:17.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f77c00b5400 con 0x7f77d015caa0 2023-11-06T03:14:17.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.984+0000 7f77c6ffd640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 0x7f77b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:17.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.988+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77c011dd20 con 0x7f77d015caa0 2023-11-06T03:14:17.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.988+0000 7f77d609f640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 0x7f77b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:17.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.988+0000 7f77d609f640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 0x7f77b4067f20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77cc008f80 tx=0x7f77cc0084d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:17.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:17.988+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f77c00eb610 con 0x7f77d015caa0 2023-11-06T03:14:18.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.196+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f77d00b8250 con 0x7f77d015caa0 2023-11-06T03:14:18.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.196+0000 7f77c6ffd640 1 -- 172.21.15.157:0/815514862 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1577 (secure 0 0 0) 0x7f77c00eec30 con 0x7f77d015caa0 2023-11-06T03:14:18.195 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:18.195 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":635,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1077956881,"bytes_used":2761814016,"bytes_avail":770332299264,"bytes_total":773094113280,"write_bytes_sec":28960106,"read_op_per_sec":0,"write_op_per_sec":144},"fsmap":{"epoch":91,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-11-06T03:14:17.318434+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:14:18.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 msgr2=0x7f77b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 0x7f77b4067f20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77cc008f80 tx=0x7f77cc0084d0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 msgr2=0x7f77d013b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f77c00ad040 tx=0x7f77c00192b0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 shutdown_connections 2023-11-06T03:14:18.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77b4065a60 0x7f77b4067f20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77d015ce80 0x7f77d015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77d015caa0 0x7f77d013b2b0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 --2- 172.21.15.157:0/815514862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77d00ce1a0 0x7f77d0154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 >> 172.21.15.157:0/815514862 conn(0x7f77d0070790 msgr2=0x7f77d00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 shutdown_connections 2023-11-06T03:14:18.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.200+0000 7f77d8303640 1 -- 172.21.15.157:0/815514862 wait complete. 2023-11-06T03:14:18.214 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:14:18.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:14:18.215 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:14:18.273 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:14:18.329 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:14:18.330 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:14:18.330 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:18.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:18.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 Processor -- start 2023-11-06T03:14:18.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 -- start start 2023-11-06T03:14:18.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29815ac80 0x7fe2980b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe2980afb00 0x7fe2980b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980cc2b0 con 0x7fe2980cc720 2023-11-06T03:14:18.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2980b01a0 con 0x7fe29815ac80 2023-11-06T03:14:18.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29cab5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2980cfb10 con 0x7fe2980afb00 2023-11-06T03:14:18.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29815ac80 0x7fe2980b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe29815ac80 0x7fe2980b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40446/0 (socket says 172.21.15.157:40446) 2023-11-06T03:14:18.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 -- 172.21.15.157:0/1834100121 learned_addr learned my addr 172.21.15.157:0/1834100121 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:18.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 -- 172.21.15.157:0/1834100121 >> v2:172.21.15.195:3300/0 conn(0x7fe2980afb00 msgr2=0x7fe2980b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:18.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 --2- 172.21.15.157:0/1834100121 >> v2:172.21.15.195:3300/0 conn(0x7fe2980afb00 0x7fe2980b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 -- 172.21.15.157:0/1834100121 >> v1:172.21.15.195:6789/0 conn(0x7fe2980cc720 legacy=0x7fe2980ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:18.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.504+0000 7fe29659c640 1 -- 172.21.15.157:0/1834100121 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2980cfdf0 con 0x7fe29815ac80 2023-11-06T03:14:18.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.508+0000 7fe29659c640 1 --2- 172.21.15.157:0/1834100121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29815ac80 0x7fe2980b2400 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe2980b5e40 tx=0x7fe28801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666196bf8875ee9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.508+0000 7fe29559a640 1 -- 172.21.15.157:0/1834100121 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980cda90 con 0x7fe29815ac80 2023-11-06T03:14:18.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.508+0000 7fe29559a640 1 -- 172.21.15.157:0/1834100121 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2880aa620 con 0x7fe29815ac80 2023-11-06T03:14:18.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.508+0000 7fe29559a640 1 -- 172.21.15.157:0/1834100121 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880aa9b0 con 0x7fe29815ac80 2023-11-06T03:14:18.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1834100121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29815ac80 msgr2=0x7fe2980b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1834100121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29815ac80 0x7fe2980b2400 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe2980b5e40 tx=0x7fe28801c4c0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1834100121 shutdown_connections 2023-11-06T03:14:18.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1834100121 >> v2:172.21.15.195:3300/0 conn(0x7fe2980afb00 0x7fe2980b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1834100121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29815ac80 0x7fe2980b2400 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1834100121 >> 172.21.15.157:0/1834100121 conn(0x7fe29806d560 msgr2=0x7fe29806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:18.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1834100121 shutdown_connections 2023-11-06T03:14:18.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1834100121 wait complete. 2023-11-06T03:14:18.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 Processor -- start 2023-11-06T03:14:18.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- start start 2023-11-06T03:14:18.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2980cc720 0x7fe2980c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 0x7fe298158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980c87c0 con 0x7fe2980afb00 2023-11-06T03:14:18.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298069d70 con 0x7fe29815ac80 2023-11-06T03:14:18.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29806abd0 con 0x7fe2980cc720 2023-11-06T03:14:18.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40462/0 (socket says 172.21.15.157:40462) 2023-11-06T03:14:18.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 -- 172.21.15.157:0/1142621505 learned_addr learned my addr 172.21.15.157:0/1142621505 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:18.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe296d9d640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 0x7fe298158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 -- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2980cc720 msgr2=0x7fe2980c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe295d9b640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2980cc720 0x7fe2980c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2980cc720 0x7fe2980c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 -- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 msgr2=0x7fe298158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 0x7fe298158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 -- 172.21.15.157:0/1142621505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880094d0 con 0x7fe2980afb00 2023-11-06T03:14:18.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe296d9d640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 0x7fe298158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:18.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29659c640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe2880ac040 tx=0x7fe28801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880bd3f0 con 0x7fe2980afb00 2023-11-06T03:14:18.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2880bd570 con 0x7fe2980afb00 2023-11-06T03:14:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe298069fc0 con 0x7fe2980afb00 2023-11-06T03:14:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880bd920 con 0x7fe2980afb00 2023-11-06T03:14:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.512+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29806a580 con 0x7fe2980afb00 2023-11-06T03:14:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe298072750 con 0x7fe2980afb00 2023-11-06T03:14:18.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe2880b33f0 con 0x7fe2980afb00 2023-11-06T03:14:18.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe286ffd640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 0x7fe278067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe28811d9f0 con 0x7fe2980afb00 2023-11-06T03:14:18.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe295d9b640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 0x7fe278067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.516+0000 7fe295d9b640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 0x7fe278067f20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2980c3d20 tx=0x7fe28c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.520+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe2880eb400 con 0x7fe2980afb00 2023-11-06T03:14:18.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.664+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe29806e3e0 con 0x7fe2980afb00 2023-11-06T03:14:18.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe286ffd640 1 -- 172.21.15.157:0/1142621505 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+17941 (secure 0 0 0) 0x7fe2880eea20 con 0x7fe2980afb00 2023-11-06T03:14:18.666 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:18.666 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:41.413681+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:18.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 msgr2=0x7fe278067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 0x7fe278067f20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe2980c3d20 tx=0x7fe28c008040 comp rx=0 tx=0).stop 2023-11-06T03:14:18.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 msgr2=0x7fe2980c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe2880ac040 tx=0x7fe28801ad00 comp rx=0 tx=0).stop 2023-11-06T03:14:18.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 shutdown_connections 2023-11-06T03:14:18.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe278065a60 0x7fe278067f20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe29815ac80 0x7fe298158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe2980cc720 0x7fe2980c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 --2- 172.21.15.157:0/1142621505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2980afb00 0x7fe2980c8280 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 >> 172.21.15.157:0/1142621505 conn(0x7fe29806d560 msgr2=0x7fe298157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:18.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 shutdown_connections 2023-11-06T03:14:18.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.668+0000 7fe29cab5640 1 -- 172.21.15.157:0/1142621505 wait complete. 2023-11-06T03:14:18.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:13:38.971687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:13:39.371284+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:14:18.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:14:18.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.796+0000 7f83a7cbe640 1 Processor -- start 2023-11-06T03:14:18.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 -- start start 2023-11-06T03:14:18.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f83a015ce80 0x7f83a00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83a00d00b0 0x7f83a00b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a00d2920 con 0x7f83a015caa0 2023-11-06T03:14:18.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a00d0520 con 0x7f83a00d00b0 2023-11-06T03:14:18.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a7cbe640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a00d18a0 con 0x7f83a015ce80 2023-11-06T03:14:18.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83a00d00b0 0x7f83a00b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83a00d00b0 0x7f83a00b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42132/0 (socket says 172.21.15.157:42132) 2023-11-06T03:14:18.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 -- 172.21.15.157:0/581349535 learned_addr learned my addr 172.21.15.157:0/581349535 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:18.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a5259640 1 --2- 172.21.15.157:0/581349535 >> v2:172.21.15.195:3300/0 conn(0x7f83a015ce80 0x7f83a00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 -- 172.21.15.157:0/581349535 >> v2:172.21.15.195:3300/0 conn(0x7f83a015ce80 msgr2=0x7f83a00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:18.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 --2- 172.21.15.157:0/581349535 >> v2:172.21.15.195:3300/0 conn(0x7f83a015ce80 0x7f83a00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 -- 172.21.15.157:0/581349535 >> v1:172.21.15.195:6789/0 conn(0x7f83a015caa0 legacy=0x7f83a00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:18.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 -- 172.21.15.157:0/581349535 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00d1b60 con 0x7f83a00d00b0 2023-11-06T03:14:18.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a625b640 1 --2- 172.21.15.157:0/581349535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a00d00b0 0x7f83a00b5d50 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c6720 tx=0x7f839801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7a14bb71e748d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a4a58640 1 -- 172.21.15.157:0/581349535 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980a5820 con 0x7f83a00d00b0 2023-11-06T03:14:18.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a4a58640 1 -- 172.21.15.157:0/581349535 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83980a5980 con 0x7f83a00d00b0 2023-11-06T03:14:18.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.800+0000 7f83a4a58640 1 -- 172.21.15.157:0/581349535 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980a5d10 con 0x7f83a00d00b0 2023-11-06T03:14:18.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/581349535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a00d00b0 msgr2=0x7f83a00b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/581349535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a00d00b0 0x7f83a00b5d50 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c6720 tx=0x7f839801c960 comp rx=0 tx=0).stop 2023-11-06T03:14:18.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/581349535 shutdown_connections 2023-11-06T03:14:18.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/581349535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a00d00b0 0x7f83a00b5d50 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/581349535 >> v2:172.21.15.195:3300/0 conn(0x7f83a015ce80 0x7f83a00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/581349535 >> 172.21.15.157:0/581349535 conn(0x7f83a0070790 msgr2=0x7f83a0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/581349535 shutdown_connections 2023-11-06T03:14:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/581349535 wait complete. 2023-11-06T03:14:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 Processor -- start 2023-11-06T03:14:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- start start 2023-11-06T03:14:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 0x7f83a0152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a015caa0 0x7f83a0153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01540b0 con 0x7f83a015ce80 2023-11-06T03:14:18.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cd190 con 0x7f83a00d00b0 2023-11-06T03:14:18.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00cc250 con 0x7f83a015caa0 2023-11-06T03:14:18.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42148/0 (socket says 172.21.15.157:42148) 2023-11-06T03:14:18.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 -- 172.21.15.157:0/3033042535 learned_addr learned my addr 172.21.15.157:0/3033042535 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:18.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a5a5a640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 0x7f83a0152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a5259640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a015caa0 0x7f83a0153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 -- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a015caa0 msgr2=0x7f83a0153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a015caa0 0x7f83a0153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 -- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 msgr2=0x7f83a0152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 0x7f83a0152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 -- 172.21.15.157:0/3033042535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8398009b30 con 0x7f83a015ce80 2023-11-06T03:14:18.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a5a5a640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 0x7f83a0152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:18.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a625b640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f839801a590 tx=0x7f8398019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980b8670 con 0x7f83a015ce80 2023-11-06T03:14:18.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83980b87f0 con 0x7f83a015ce80 2023-11-06T03:14:18.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a00cc530 con 0x7f83a015ce80 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980b8ba0 con 0x7f83a015ce80 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.804+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a0152550 con 0x7f83a015ce80 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a0072750 con 0x7f83a015ce80 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f83980b8d00 con 0x7f83a015ce80 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83967fc640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 0x7f8384067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:18.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f839811e950 con 0x7f83a015ce80 2023-11-06T03:14:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83a5a5a640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 0x7f8384067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.808+0000 7f83a5a5a640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 0x7f8384067f20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f83a00d31c0 tx=0x7f839c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:18.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.812+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f83980ec2d0 con 0x7f83a015ce80 2023-11-06T03:14:18.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.952+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83a0072750 con 0x7f83a015ce80 2023-11-06T03:14:18.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.952+0000 7f83967fc640 1 -- 172.21.15.157:0/3033042535 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83980ef8f0 con 0x7f83a015ce80 2023-11-06T03:14:18.950 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:14:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 msgr2=0x7f8384067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 0x7f8384067f20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f83a00d31c0 tx=0x7f839c009040 comp rx=0 tx=0).stop 2023-11-06T03:14:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 msgr2=0x7f83a0153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f839801a590 tx=0x7f8398019700 comp rx=0 tx=0).stop 2023-11-06T03:14:18.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 shutdown_connections 2023-11-06T03:14:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8384065a60 0x7f8384067f20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83a015ce80 0x7f83a0153b70 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f83a015caa0 0x7f83a0153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 --2- 172.21.15.157:0/3033042535 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f83a00d00b0 0x7f83a0152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 >> 172.21.15.157:0/3033042535 conn(0x7f83a0070790 msgr2=0x7f83a00cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:18.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 shutdown_connections 2023-11-06T03:14:18.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:18.956+0000 7f83a7cbe640 1 -- 172.21.15.157:0/3033042535 wait complete. 2023-11-06T03:14:18.968 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:14:19.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 Processor -- start 2023-11-06T03:14:19.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 -- start start 2023-11-06T03:14:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75981551e0 0x7f759806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759815f580 con 0x7f75980b3b20 2023-11-06T03:14:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759815efc0 con 0x7f759815e5e0 2023-11-06T03:14:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759eff3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759815f2a0 con 0x7f75981551e0 2023-11-06T03:14:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759cd8f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f75981551e0 0x7f759806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759d590640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75980b3b20 0x7f759815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44368/0 (socket says 172.21.15.157:44368) 2023-11-06T03:14:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759d590640 1 -- 172.21.15.157:0/2372654902 learned_addr learned my addr 172.21.15.157:0/2372654902 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f75977fe640 1 -- 172.21.15.157:0/2372654902 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916484605 0 0) 0x7f759815f580 con 0x7f75980b3b20 2023-11-06T03:14:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f75977fe640 1 -- 172.21.15.157:0/2372654902 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7580003530 con 0x7f75980b3b20 2023-11-06T03:14:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759cd8f640 1 -- 172.21.15.157:0/2372654902 >> v1:172.21.15.195:6789/0 conn(0x7f759815e5e0 legacy=0x7f75980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759cd8f640 1 -- 172.21.15.157:0/2372654902 >> v1:172.21.15.157:6789/0 conn(0x7f75980b3b20 legacy=0x7f759815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.096+0000 7f759cd8f640 1 -- 172.21.15.157:0/2372654902 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759806f5a0 con 0x7f75981551e0 2023-11-06T03:14:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759cd8f640 1 --2- 172.21.15.157:0/2372654902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75981551e0 0x7f759806fda0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f758801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3499bd87e43d16 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f75977fe640 1 -- 172.21.15.157:0/2372654902 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588004230 con 0x7f75981551e0 2023-11-06T03:14:19.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f75977fe640 1 -- 172.21.15.157:0/2372654902 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880a7050 con 0x7f75981551e0 2023-11-06T03:14:19.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f75977fe640 1 -- 172.21.15.157:0/2372654902 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588003620 con 0x7f75981551e0 2023-11-06T03:14:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 -- 172.21.15.157:0/2372654902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75981551e0 msgr2=0x7f759806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 --2- 172.21.15.157:0/2372654902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75981551e0 0x7f759806fda0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f758801c330 comp rx=0 tx=0).stop 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 -- 172.21.15.157:0/2372654902 shutdown_connections 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 --2- 172.21.15.157:0/2372654902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75981551e0 0x7f759806fda0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 -- 172.21.15.157:0/2372654902 >> 172.21.15.157:0/2372654902 conn(0x7f7598065930 msgr2=0x7f7598065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 -- 172.21.15.157:0/2372654902 shutdown_connections 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 -- 172.21.15.157:0/2372654902 wait complete. 2023-11-06T03:14:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.100+0000 7f759eff3640 1 Processor -- start 2023-11-06T03:14:19.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- start start 2023-11-06T03:14:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75980b3b20 0x7f759814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 0x7f759815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759814e280 con 0x7f759815e5e0 2023-11-06T03:14:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75980a94e0 con 0x7f75981551e0 2023-11-06T03:14:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759806ccf0 con 0x7f75980b3b20 2023-11-06T03:14:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42160/0 (socket says 172.21.15.157:42160) 2023-11-06T03:14:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 -- 172.21.15.157:0/3041951439 learned_addr learned my addr 172.21.15.157:0/3041951439 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f7597fff640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 0x7f759815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759cd8f640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75980b3b20 0x7f759814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 -- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75980b3b20 msgr2=0x7f759814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75980b3b20 0x7f759814d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 -- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 msgr2=0x7f759815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 0x7f759815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 -- 172.21.15.157:0/3041951439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588009420 con 0x7f759815e5e0 2023-11-06T03:14:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f7597fff640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 0x7f759815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759d590640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f759000cb40 tx=0x7f759000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759000aca0 con 0x7f759815e5e0 2023-11-06T03:14:19.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759000ae00 con 0x7f759815e5e0 2023-11-06T03:14:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759806cf90 con 0x7f759815e5e0 2023-11-06T03:14:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590012620 con 0x7f759815e5e0 2023-11-06T03:14:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980b2360 con 0x7f759815e5e0 2023-11-06T03:14:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7590007070 con 0x7f759815e5e0 2023-11-06T03:14:19.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7598072750 con 0x7f759815e5e0 2023-11-06T03:14:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 0x7f7584067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7590083340 con 0x7f759815e5e0 2023-11-06T03:14:19.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.104+0000 7f759cd8f640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 0x7f7584067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.108+0000 7f759cd8f640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 0x7f7584067f20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f75880ad000 tx=0x7f7588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.108+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7590050ba0 con 0x7f759815e5e0 2023-11-06T03:14:19.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.252+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7598071730 con 0x7f759815e5e0 2023-11-06T03:14:19.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.252+0000 7f75957fa640 1 -- 172.21.15.157:0/3041951439 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7590017020 con 0x7f759815e5e0 2023-11-06T03:14:19.251 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:14:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 msgr2=0x7f7584067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 0x7f7584067f20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f75880ad000 tx=0x7f7588008040 comp rx=0 tx=0).stop 2023-11-06T03:14:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 msgr2=0x7f759815db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f759000cb40 tx=0x7f759000a060 comp rx=0 tx=0).stop 2023-11-06T03:14:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 shutdown_connections 2023-11-06T03:14:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7584065a60 0x7f7584067f20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f759815e5e0 0x7f759815db40 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75981551e0 0x7f759815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 --2- 172.21.15.157:0/3041951439 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75980b3b20 0x7f759814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 >> 172.21.15.157:0/3041951439 conn(0x7f7598065930 msgr2=0x7f7598152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 shutdown_connections 2023-11-06T03:14:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.256+0000 7f759eff3640 1 -- 172.21.15.157:0/3041951439 wait complete. 2023-11-06T03:14:19.268 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:14:19.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 Processor -- start 2023-11-06T03:14:19.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 -- start start 2023-11-06T03:14:19.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538815e440 con 0x7f53880b2af0 2023-11-06T03:14:19.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538806c240 con 0x7f538806d000 2023-11-06T03:14:19.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538f3e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538806c520 con 0x7f53881599c0 2023-11-06T03:14:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d97e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f53880b2af0 0x7f5388159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44376/0 (socket says 172.21.15.157:44376) 2023-11-06T03:14:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d97e640 1 -- 172.21.15.157:0/3112032041 learned_addr learned my addr 172.21.15.157:0/3112032041 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571575650 0 0) 0x7f538815e440 con 0x7f53880b2af0 2023-11-06T03:14:19.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374003530 con 0x7f53880b2af0 2023-11-06T03:14:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321274719 0 0) 0x7f5374003530 con 0x7f53880b2af0 2023-11-06T03:14:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 msgr2=0x7f53880ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53880ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 >> v1:172.21.15.195:6789/0 conn(0x7f538806d000 legacy=0x7f53880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538806c800 con 0x7f53880b2af0 2023-11-06T03:14:19.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5380001a00 con 0x7f53880b2af0 2023-11-06T03:14:19.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53880ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:19.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 >> v1:172.21.15.157:6789/0 conn(0x7f53880b2af0 legacy=0x7f5388159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53740039b0 0x7f5374003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374004d10 0x7f5374005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538806c520 con 0x7f53740039b0 2023-11-06T03:14:19.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537400baa0 con 0x7f5374004d10 2023-11-06T03:14:19.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537400bc60 con 0x7f53881599c0 2023-11-06T03:14:19.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53740039b0 0x7f5374003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 -- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 msgr2=0x7f53740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538c97c640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d97e640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374004d10 0x7f5374005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53881599c0 0x7f53740047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 -- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374004d10 msgr2=0x7f5374005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:19.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374004d10 0x7f5374005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.384+0000 7f538d17d640 1 -- 172.21.15.157:0/3112032041 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374003530 con 0x7f53740039b0 2023-11-06T03:14:19.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538d17d640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53740039b0 0x7f5374003dd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f53840098a0 tx=0x7f538401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47579e46755323a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840aa430 con 0x7f53740039b0 2023-11-06T03:14:19.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53840aa590 con 0x7f53740039b0 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f537ffff640 1 -- 172.21.15.157:0/3112032041 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840aa900 con 0x7f53740039b0 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- 172.21.15.157:0/3112032041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53740039b0 msgr2=0x7f5374003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3112032041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53740039b0 0x7f5374003dd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f53840098a0 tx=0x7f538401c480 comp rx=0 tx=0).stop 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538d97e640 1 -- 172.21.15.157:0/3112032041 reap_dead start 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- 172.21.15.157:0/3112032041 shutdown_connections 2023-11-06T03:14:19.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- 172.21.15.157:0/3112032041 >> 172.21.15.157:0/3112032041 conn(0x7f5388065930 msgr2=0x7f5388065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- 172.21.15.157:0/3112032041 shutdown_connections 2023-11-06T03:14:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- 172.21.15.157:0/3112032041 wait complete. 2023-11-06T03:14:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 Processor -- start 2023-11-06T03:14:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- start start 2023-11-06T03:14:19.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53740039b0 0x7f5388144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 0x7f53881421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388145e80 con 0x7f5388155f60 2023-11-06T03:14:19.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388146350 con 0x7f53740039b0 2023-11-06T03:14:19.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538f3e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5388145ba0 con 0x7f5388145690 2023-11-06T03:14:19.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42172/0 (socket says 172.21.15.157:42172) 2023-11-06T03:14:19.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 -- 172.21.15.157:0/3622787794 learned_addr learned my addr 172.21.15.157:0/3622787794 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:19.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538d17d640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53740039b0 0x7f5388144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538d97e640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 0x7f53881421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 -- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 msgr2=0x7f53881421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 0x7f53881421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 -- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53740039b0 msgr2=0x7f5388144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53740039b0 0x7f5388144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538c97c640 1 -- 172.21.15.157:0/3622787794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880a8640 con 0x7f5388155f60 2023-11-06T03:14:19.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.388+0000 7f538d97e640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 0x7f53881421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:19.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538c97c640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f537800d770 tx=0x7f537800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378005270 con 0x7f5388155f60 2023-11-06T03:14:19.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780053d0 con 0x7f5388155f60 2023-11-06T03:14:19.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53880a8920 con 0x7f5388155f60 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800b850 con 0x7f5388155f60 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53881428b0 con 0x7f5388155f60 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880a84a0 con 0x7f5388155f60 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f537800b9b0 con 0x7f5388155f60 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 0x7f536c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53780831a0 con 0x7f5388155f60 2023-11-06T03:14:19.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538d17d640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 0x7f536c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.392+0000 7f538d17d640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 0x7f536c067f20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f53840ae000 tx=0x7f53840b5040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.396+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f53880a84a0 con 0x7f5388155f60 2023-11-06T03:14:19.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53880a84a0 con 0x7f5388155f60 2023-11-06T03:14:19.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f537dffb640 1 -- 172.21.15.157:0/3622787794 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53880a84a0 con 0x7f5388155f60 2023-11-06T03:14:19.549 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:14:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 msgr2=0x7f536c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 0x7f536c067f20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f53840ae000 tx=0x7f53840b5040 comp rx=0 tx=0).stop 2023-11-06T03:14:19.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 msgr2=0x7f5388145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f537800d770 tx=0x7f537800dc30 comp rx=0 tx=0).stop 2023-11-06T03:14:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 shutdown_connections 2023-11-06T03:14:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f536c065a60 0x7f536c067f20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5388145690 0x7f53881421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5388155f60 0x7f5388145150 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 --2- 172.21.15.157:0/3622787794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f53740039b0 0x7f5388144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 >> 172.21.15.157:0/3622787794 conn(0x7f5388065930 msgr2=0x7f53880b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 shutdown_connections 2023-11-06T03:14:19.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.552+0000 7f538f3e1640 1 -- 172.21.15.157:0/3622787794 wait complete. 2023-11-06T03:14:19.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:14:19.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 Processor -- start 2023-11-06T03:14:19.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 -- start start 2023-11-06T03:14:19.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d340b3b20 0x7f0d3415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3415f580 con 0x7f0d3415e5e0 2023-11-06T03:14:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3415efc0 con 0x7f0d340b3b20 2023-11-06T03:14:19.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d389e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3415f2a0 con 0x7f0d341551e0 2023-11-06T03:14:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d32d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0d340b3b20 0x7f0d3415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d31d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0d3415e5e0 0x7f0d340b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44378/0 (socket says 172.21.15.157:44378) 2023-11-06T03:14:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d31d9b640 1 -- 172.21.15.157:0/2842709755 learned_addr learned my addr 172.21.15.157:0/2842709755 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:19.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d3259c640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d32d9d640 1 -- 172.21.15.157:0/2842709755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 msgr2=0x7f0d3406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:19.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d32d9d640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d32d9d640 1 -- 172.21.15.157:0/2842709755 >> v1:172.21.15.157:6789/0 conn(0x7f0d3415e5e0 legacy=0x7f0d340b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d3159a640 1 -- 172.21.15.157:0/2842709755 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317257897 0 0) 0x7f0d3415f580 con 0x7f0d3415e5e0 2023-11-06T03:14:19.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.700+0000 7f0d32d9d640 1 -- 172.21.15.157:0/2842709755 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3406f5a0 con 0x7f0d340b3b20 2023-11-06T03:14:19.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d32d9d640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d3415c1b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009d90 tx=0x7f0d2c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453ba35f07d82413 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3159a640 1 -- 172.21.15.157:0/2842709755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0a5970 con 0x7f0d340b3b20 2023-11-06T03:14:19.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3159a640 1 -- 172.21.15.157:0/2842709755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c0a5ad0 con 0x7f0d340b3b20 2023-11-06T03:14:19.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3159a640 1 -- 172.21.15.157:0/2842709755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0a5e60 con 0x7f0d340b3b20 2023-11-06T03:14:19.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2842709755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 msgr2=0x7f0d3415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d3415c1b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009d90 tx=0x7f0d2c01c890 comp rx=0 tx=0).stop 2023-11-06T03:14:19.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2842709755 shutdown_connections 2023-11-06T03:14:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d3415c1b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2842709755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2842709755 >> 172.21.15.157:0/2842709755 conn(0x7f0d34065930 msgr2=0x7f0d34065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2842709755 shutdown_connections 2023-11-06T03:14:19.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2842709755 wait complete. 2023-11-06T03:14:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 Processor -- start 2023-11-06T03:14:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- start start 2023-11-06T03:14:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 0x7f0d34142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34147120 con 0x7f0d340b3b20 2023-11-06T03:14:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34147ee0 con 0x7f0d3415e5e0 2023-11-06T03:14:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341467c0 con 0x7f0d341551e0 2023-11-06T03:14:19.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42186/0 (socket says 172.21.15.157:42186) 2023-11-06T03:14:19.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 -- 172.21.15.157:0/2243050357 learned_addr learned my addr 172.21.15.157:0/2243050357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:19.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d32d9d640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 0x7f0d34142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d31d9b640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 -- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 msgr2=0x7f0d3414c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3414c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 -- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 msgr2=0x7f0d34142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 0x7f0d34142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 -- 172.21.15.157:0/2243050357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c009a70 con 0x7f0d340b3b20 2023-11-06T03:14:19.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d32d9d640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 0x7f0d34142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:19.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d3259c640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c7f0 tx=0x7f0d2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004220 con 0x7f0d340b3b20 2023-11-06T03:14:19.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d240043a0 con 0x7f0d340b3b20 2023-11-06T03:14:19.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d34146a60 con 0x7f0d340b3b20 2023-11-06T03:14:19.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004510 con 0x7f0d340b3b20 2023-11-06T03:14:19.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.704+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d34144c40 con 0x7f0d340b3b20 2023-11-06T03:14:19.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d34072750 con 0x7f0d340b3b20 2023-11-06T03:14:19.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0d24003a70 con 0x7f0d340b3b20 2023-11-06T03:14:19.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d22ffd640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 0x7f0d14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:19.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d24083220 con 0x7f0d340b3b20 2023-11-06T03:14:19.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d31d9b640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 0x7f0d14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:19.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.708+0000 7f0d31d9b640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 0x7f0d14067f20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cad40 tx=0x7f0d28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:19.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.712+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0d24050b10 con 0x7f0d340b3b20 2023-11-06T03:14:19.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.864+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d34071730 con 0x7f0d340b3b20 2023-11-06T03:14:19.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d22ffd640 1 -- 172.21.15.157:0/2243050357 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d2401e020 con 0x7f0d340b3b20 2023-11-06T03:14:19.865 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:14:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 msgr2=0x7f0d14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 0x7f0d14067f20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0d340cad40 tx=0x7f0d28009040 comp rx=0 tx=0).stop 2023-11-06T03:14:19.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 msgr2=0x7f0d34147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400c7f0 tx=0x7f0d2400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 shutdown_connections 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0d14065a60 0x7f0d14067f20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d3415e5e0 0x7f0d34142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0d341551e0 0x7f0d3414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 --2- 172.21.15.157:0/2243050357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d340b3b20 0x7f0d34147850 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 >> 172.21.15.157:0/2243050357 conn(0x7f0d34065930 msgr2=0x7f0d340aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 shutdown_connections 2023-11-06T03:14:19.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:19.868+0000 7f0d389e7640 1 -- 172.21.15.157:0/2243050357 wait complete. 2023-11-06T03:14:19.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:14:20.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 Processor -- start 2023-11-06T03:14:20.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 -- start start 2023-11-06T03:14:20.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815caa0 0x7f17180cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f171815ce80 0x7f17180cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17180ce1a0 0x7f17180d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180d2920 con 0x7f171815ce80 2023-11-06T03:14:20.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180d18a0 con 0x7f17180ce1a0 2023-11-06T03:14:20.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171f75b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17180d1b60 con 0x7f171815caa0 2023-11-06T03:14:20.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f171815ce80 0x7f17180cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f171815ce80 0x7f17180cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42196/0 (socket says 172.21.15.157:42196) 2023-11-06T03:14:20.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 -- 172.21.15.157:0/2041517823 learned_addr learned my addr 172.21.15.157:0/2041517823 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:20.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171d4f7640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815caa0 0x7f17180cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171dcf8640 1 --2- 172.21.15.157:0/2041517823 >> v2:172.21.15.195:3300/0 conn(0x7f17180ce1a0 0x7f17180d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 -- 172.21.15.157:0/2041517823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815caa0 msgr2=0x7f17180cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:20.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815caa0 0x7f17180cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 -- 172.21.15.157:0/2041517823 >> v2:172.21.15.195:3300/0 conn(0x7f17180ce1a0 msgr2=0x7f17180d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:20.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 --2- 172.21.15.157:0/2041517823 >> v2:172.21.15.195:3300/0 conn(0x7f17180ce1a0 0x7f17180d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.008+0000 7f171ccf6640 1 -- 172.21.15.157:0/2041517823 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180d1e40 con 0x7f171815ce80 2023-11-06T03:14:20.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815ce80 0x7f17180cfb70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f17180d0780 tx=0x7f170801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b056ad4f53d341f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170ffff640 1 -- 172.21.15.157:0/2041517823 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080aa5f0 con 0x7f171815ce80 2023-11-06T03:14:20.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170ffff640 1 -- 172.21.15.157:0/2041517823 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17080aa750 con 0x7f171815ce80 2023-11-06T03:14:20.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170ffff640 1 -- 172.21.15.157:0/2041517823 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080aaae0 con 0x7f171815ce80 2023-11-06T03:14:20.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/2041517823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815ce80 msgr2=0x7f17180cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815ce80 0x7f17180cfb70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f17180d0780 tx=0x7f170801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/2041517823 shutdown_connections 2023-11-06T03:14:20.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- 172.21.15.157:0/2041517823 >> v2:172.21.15.195:3300/0 conn(0x7f17180ce1a0 0x7f17180d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815ce80 0x7f17180cfb70 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- 172.21.15.157:0/2041517823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815caa0 0x7f17180cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/2041517823 >> 172.21.15.157:0/2041517823 conn(0x7f1718070790 msgr2=0x7f1718070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:20.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/2041517823 shutdown_connections 2023-11-06T03:14:20.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/2041517823 wait complete. 2023-11-06T03:14:20.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 Processor -- start 2023-11-06T03:14:20.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- start start 2023-11-06T03:14:20.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17180ce1a0 0x7f1718155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815ce80 0x7f17180cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17181f8620 con 0x7f171815caa0 2023-11-06T03:14:20.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17181f8900 con 0x7f17180ce1a0 2023-11-06T03:14:20.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17181f8be0 con 0x7f171815ce80 2023-11-06T03:14:20.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42208/0 (socket says 172.21.15.157:42208) 2023-11-06T03:14:20.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 -- 172.21.15.157:0/317831555 learned_addr learned my addr 172.21.15.157:0/317831555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:20.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171d4f7640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17180ce1a0 0x7f1718155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 -- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815ce80 msgr2=0x7f17180cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815ce80 0x7f17180cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 -- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17180ce1a0 msgr2=0x7f1718155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17180ce1a0 0x7f1718155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 -- 172.21.15.157:0/317831555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080094d0 con 0x7f171815caa0 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171ccf6640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f17080ac040 tx=0x7f1708019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080aa5f0 con 0x7f171815caa0 2023-11-06T03:14:20.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17080b4830 con 0x7f171815caa0 2023-11-06T03:14:20.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17181f8ec0 con 0x7f171815caa0 2023-11-06T03:14:20.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080b4be0 con 0x7f171815caa0 2023-11-06T03:14:20.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.012+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171815b7f0 con 0x7f171815caa0 2023-11-06T03:14:20.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171815b9e0 con 0x7f171815caa0 2023-11-06T03:14:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f17080b4d40 con 0x7f171815caa0 2023-11-06T03:14:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f170dffb640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 0x7f16fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17080a7070 con 0x7f171815caa0 2023-11-06T03:14:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f171d4f7640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 0x7f16fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.016+0000 7f171d4f7640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 0x7f16fc067f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1714009790 tx=0x7f1714008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.020+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f171815b9e0 con 0x7f171815caa0 2023-11-06T03:14:20.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.176+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17181f9370 con 0x7f171815caa0 2023-11-06T03:14:20.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.176+0000 7f170dffb640 1 -- 172.21.15.157:0/317831555 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17181f9370 con 0x7f171815caa0 2023-11-06T03:14:20.176 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:14:20.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 msgr2=0x7f16fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 0x7f16fc067f20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1714009790 tx=0x7f1714008040 comp rx=0 tx=0).stop 2023-11-06T03:14:20.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 msgr2=0x7f171815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f17080ac040 tx=0x7f1708019270 comp rx=0 tx=0).stop 2023-11-06T03:14:20.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 shutdown_connections 2023-11-06T03:14:20.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16fc065a60 0x7f16fc067f20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f171815ce80 0x7f17180cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f171815caa0 0x7f171815b2b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 --2- 172.21.15.157:0/317831555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17180ce1a0 0x7f1718155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 >> 172.21.15.157:0/317831555 conn(0x7f1718070790 msgr2=0x7f17180cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:20.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 shutdown_connections 2023-11-06T03:14:20.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.180+0000 7f171f75b640 1 -- 172.21.15.157:0/317831555 wait complete. 2023-11-06T03:14:20.192 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:14:20.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:20.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:20.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[87403]: fuse finished with error 0 and tester_r 0 2023-11-06T03:14:20.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:14:20.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:20.298 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:14:20.375 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:14:20.376 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:20.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:14:20.383 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:14:20.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:20.384 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:14:20.431 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:14:20.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:14:20.433 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:14:20.434 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:20.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:14:20.440 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:14:20.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:20.442 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:14:20.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:14:20.488 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:14:20.489 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:14:20.489 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:20.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:14:20.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:14:20.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:20.497 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:14:20.544 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:14:20.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:14:20.546 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:20.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.672+0000 7f844fcb7640 1 Processor -- start 2023-11-06T03:14:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844fcb7640 1 -- start start 2023-11-06T03:14:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844fcb7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84480cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844fcb7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84480d2920 con 0x7f844815ce80 2023-11-06T03:14:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844fcb7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84480d18a0 con 0x7f84480d0280 2023-11-06T03:14:20.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844fcb7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84480d1b60 con 0x7f844815caa0 2023-11-06T03:14:20.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844d252640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f844815ce80 0x7f84480cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44392/0 (socket says 172.21.15.157:44392) 2023-11-06T03:14:20.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844d252640 1 -- 172.21.15.157:0/2163415557 learned_addr learned my addr 172.21.15.157:0/2163415557 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:20.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84480cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483871037 0 0) 0x7f84480d2920 con 0x7f844815ce80 2023-11-06T03:14:20.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8434003530 con 0x7f844815ce80 2023-11-06T03:14:20.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861131107 0 0) 0x7f8434003530 con 0x7f844815ce80 2023-11-06T03:14:20.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 msgr2=0x7f84480cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:20.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84480cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 >> v1:172.21.15.195:6789/0 conn(0x7f84480d0280 legacy=0x7f84480b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480d1e40 con 0x7f844815ce80 2023-11-06T03:14:20.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8438001680 con 0x7f844815ce80 2023-11-06T03:14:20.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 >> v1:172.21.15.157:6789/0 conn(0x7f844815ce80 legacy=0x7f84480cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f8434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8434004d10 0x7f8434005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84480d1b60 con 0x7f84340039b0 2023-11-06T03:14:20.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843400ba40 con 0x7f8434004d10 2023-11-06T03:14:20.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843400bc00 con 0x7f844815caa0 2023-11-06T03:14:20.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f8434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844e254640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8434004d10 0x7f8434005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844d252640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 -- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 msgr2=0x7f84340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:20.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f844815caa0 0x7f84340047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 -- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8434004d10 msgr2=0x7f8434005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:20.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8434004d10 0x7f8434005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.676+0000 7f844da53640 1 -- 172.21.15.157:0/2163415557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434003530 con 0x7f84340039b0 2023-11-06T03:14:20.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844da53640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f8434003dd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f8444009ab0 tx=0x7f844401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6115d04cb9e70771 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440a55d0 con 0x7f84340039b0 2023-11-06T03:14:20.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84440a5730 con 0x7f84340039b0 2023-11-06T03:14:20.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844ca51640 1 -- 172.21.15.157:0/2163415557 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440a5ae0 con 0x7f84340039b0 2023-11-06T03:14:20.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- 172.21.15.157:0/2163415557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 msgr2=0x7f8434003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 --2- 172.21.15.157:0/2163415557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f8434003dd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f8444009ab0 tx=0x7f844401ca00 comp rx=0 tx=0).stop 2023-11-06T03:14:20.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844e254640 1 -- 172.21.15.157:0/2163415557 reap_dead start 2023-11-06T03:14:20.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- 172.21.15.157:0/2163415557 shutdown_connections 2023-11-06T03:14:20.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- 172.21.15.157:0/2163415557 >> 172.21.15.157:0/2163415557 conn(0x7f8448070790 msgr2=0x7f8448070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:20.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- 172.21.15.157:0/2163415557 shutdown_connections 2023-11-06T03:14:20.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- 172.21.15.157:0/2163415557 wait complete. 2023-11-06T03:14:20.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 Processor -- start 2023-11-06T03:14:20.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- start start 2023-11-06T03:14:20.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 0x7f844815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 0x7f84480c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84480c7740 con 0x7f84340039b0 2023-11-06T03:14:20.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844815c300 con 0x7f844815be80 2023-11-06T03:14:20.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844fcb7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84480c6b30 con 0x7f84480cc810 2023-11-06T03:14:20.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844da53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844da53640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42228/0 (socket says 172.21.15.157:42228) 2023-11-06T03:14:20.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844da53640 1 -- 172.21.15.157:0/881021835 learned_addr learned my addr 172.21.15.157:0/881021835 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:20.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844d252640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 0x7f844815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.680+0000 7f844e254640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 0x7f84480c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 -- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 msgr2=0x7f844815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 0x7f844815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 -- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 msgr2=0x7f84480c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 0x7f84480c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 -- 172.21.15.157:0/881021835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480c6e10 con 0x7f84340039b0 2023-11-06T03:14:20.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844e254640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 0x7f84480c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:20.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844d252640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 0x7f844815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:20.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844da53640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f84440b0040 tx=0x7f8434005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440a5ea0 con 0x7f84340039b0 2023-11-06T03:14:20.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444004020 con 0x7f84340039b0 2023-11-06T03:14:20.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440043b0 con 0x7f84340039b0 2023-11-06T03:14:20.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84480c70f0 con 0x7f84340039b0 2023-11-06T03:14:20.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84480ccc90 con 0x7f84340039b0 2023-11-06T03:14:20.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8448072750 con 0x7f84340039b0 2023-11-06T03:14:20.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84440b4040 con 0x7f84340039b0 2023-11-06T03:14:20.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 0x7f842c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:20.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f844411d320 con 0x7f84340039b0 2023-11-06T03:14:20.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.684+0000 7f844d252640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 0x7f842c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:20.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.688+0000 7f844d252640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 0x7f842c067f50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f84481f83e0 tx=0x7f8438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:20.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.688+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84440eacc0 con 0x7f84340039b0 2023-11-06T03:14:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.864+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84480c63b0 con 0x7f84340039b0 2023-11-06T03:14:20.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.864+0000 7f843e7fc640 1 -- 172.21.15.157:0/881021835 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f84440ee2e0 con 0x7f84340039b0 2023-11-06T03:14:20.863 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:20.863 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:20.864 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 msgr2=0x7f842c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 0x7f842c067f50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f84481f83e0 tx=0x7f8438009040 comp rx=0 tx=0).stop 2023-11-06T03:14:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 msgr2=0x7f844814d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f84440b0040 tx=0x7f8434005800 comp rx=0 tx=0).stop 2023-11-06T03:14:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 shutdown_connections 2023-11-06T03:14:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f842c065a90 0x7f842c067f50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844815be80 0x7f84480c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84480cc810 0x7f844815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 --2- 172.21.15.157:0/881021835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84340039b0 0x7f844814d150 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:20.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 >> 172.21.15.157:0/881021835 conn(0x7f8448070790 msgr2=0x7f844815aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:20.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 shutdown_connections 2023-11-06T03:14:20.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:20.868+0000 7f844fcb7640 1 -- 172.21.15.157:0/881021835 wait complete. 2023-11-06T03:14:20.882 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:21.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 Processor -- start 2023-11-06T03:14:21.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 -- start start 2023-11-06T03:14:21.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7fd4160db0 0x7f7fd4161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 0x7f7fd406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd406f470 con 0x7f7fd40701a0 2023-11-06T03:14:21.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd40b1df0 con 0x7f7fd4160db0 2023-11-06T03:14:21.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fda8d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406f6c0 con 0x7f7fd40b1910 2023-11-06T03:14:21.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 0x7f7fd406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 0x7f7fd406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54372/0 (socket says 172.21.15.157:54372) 2023-11-06T03:14:21.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fd37fe640 1 -- 172.21.15.157:0/606811244 learned_addr learned my addr 172.21.15.157:0/606811244 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:21.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.016+0000 7f7fd3fff640 1 --2- 172.21.15.157:0/606811244 >> v2:172.21.15.195:3300/0 conn(0x7f7fd4160db0 0x7f7fd4161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.020+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/606811244 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547261221 0 0) 0x7f7fd406f470 con 0x7f7fd40701a0 2023-11-06T03:14:21.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.020+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/606811244 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fbc003530 con 0x7f7fd40701a0 2023-11-06T03:14:21.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.024+0000 7f7fd3fff640 1 -- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 msgr2=0x7f7fd406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.024+0000 7f7fd3fff640 1 --2- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 0x7f7fd406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.024+0000 7f7fd3fff640 1 -- 172.21.15.157:0/606811244 >> v1:172.21.15.195:6789/0 conn(0x7f7fd40701a0 legacy=0x7f7fd40ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.024+0000 7f7fd3fff640 1 -- 172.21.15.157:0/606811244 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd406f9a0 con 0x7f7fd4160db0 2023-11-06T03:14:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.028+0000 7f7fd3fff640 1 --2- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd4160db0 0x7f7fd4161190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d90 tx=0x7f7fcc01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ee47bcd5966533e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.036+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/606811244 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0a5970 con 0x7f7fd4160db0 2023-11-06T03:14:21.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.036+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/606811244 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0a5ad0 con 0x7f7fd4160db0 2023-11-06T03:14:21.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.036+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/606811244 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0a5e60 con 0x7f7fd4160db0 2023-11-06T03:14:21.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.036+0000 7f7fda8d5640 1 -- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd4160db0 msgr2=0x7f7fd4161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.036+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd4160db0 0x7f7fd4161190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009d90 tx=0x7f7fcc01c8b0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- 172.21.15.157:0/606811244 shutdown_connections 2023-11-06T03:14:21.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd40b1910 0x7f7fd406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/606811244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd4160db0 0x7f7fd4161190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- 172.21.15.157:0/606811244 >> 172.21.15.157:0/606811244 conn(0x7f7fd4065930 msgr2=0x7f7fd4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- 172.21.15.157:0/606811244 shutdown_connections 2023-11-06T03:14:21.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- 172.21.15.157:0/606811244 wait complete. 2023-11-06T03:14:21.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 Processor -- start 2023-11-06T03:14:21.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- start start 2023-11-06T03:14:21.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd40701a0 0x7f7fd41fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 0x7f7fd4162080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4162b00 con 0x7f7fd40b1910 2023-11-06T03:14:21.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd41516b0 con 0x7f7fd40701a0 2023-11-06T03:14:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.040+0000 7f7fda8d5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd414ca00 con 0x7f7fd4160db0 2023-11-06T03:14:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54378/0 (socket says 172.21.15.157:54378) 2023-11-06T03:14:21.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 -- 172.21.15.157:0/1623846207 learned_addr learned my addr 172.21.15.157:0/1623846207 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:21.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd37fe640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 0x7f7fd4162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 -- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd40701a0 msgr2=0x7f7fd41fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd40701a0 0x7f7fd41fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 -- 172.21.15.157:0/1623846207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 msgr2=0x7f7fd4162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 0x7f7fd4162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 -- 172.21.15.157:0/1623846207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc009a70 con 0x7f7fd4160db0 2023-11-06T03:14:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd37fe640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 0x7f7fd4162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:14:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd8e72640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c460 tx=0x7f7fc400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40052d0 con 0x7f7fd4160db0 2023-11-06T03:14:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc4005450 con 0x7f7fd4160db0 2023-11-06T03:14:21.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4012870 con 0x7f7fd4160db0 2023-11-06T03:14:21.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.044+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd4151950 con 0x7f7fd4160db0 2023-11-06T03:14:21.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.048+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd4151f10 con 0x7f7fd4160db0 2023-11-06T03:14:21.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.048+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4072750 con 0x7f7fd4160db0 2023-11-06T03:14:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.048+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7fc40129d0 con 0x7f7fd4160db0 2023-11-06T03:14:21.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.052+0000 7f7fd0ff9640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 0x7f7fc0067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.052+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fc40836d0 con 0x7f7fd4160db0 2023-11-06T03:14:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.052+0000 7f7fd3fff640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 0x7f7fc0067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.052+0000 7f7fd3fff640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 0x7f7fc0067f70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0af000 tx=0x7f7fcc004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.068+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7fc4051020 con 0x7f7fd4160db0 2023-11-06T03:14:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.228+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fd4071020 con 0x7f7fd4160db0 2023-11-06T03:14:21.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.232+0000 7f7fd0ff9640 1 -- 172.21.15.157:0/1623846207 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f7fc4054640 con 0x7f7fd4160db0 2023-11-06T03:14:21.230 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:21.231 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:41.413681+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:21.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 msgr2=0x7f7fc0067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 0x7f7fc0067f70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0af000 tx=0x7f7fcc004ac0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 msgr2=0x7f7fd41625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c460 tx=0x7f7fc400c920 comp rx=0 tx=0).stop 2023-11-06T03:14:21.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 shutdown_connections 2023-11-06T03:14:21.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065ab0 0x7f7fc0067f70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fd4160db0 0x7f7fd41625c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fd40b1910 0x7f7fd4162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 --2- 172.21.15.157:0/1623846207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fd40701a0 0x7f7fd41fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 >> 172.21.15.157:0/1623846207 conn(0x7f7fd4065930 msgr2=0x7f7fd415f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 shutdown_connections 2023-11-06T03:14:21.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.236+0000 7f7fda8d5640 1 -- 172.21.15.157:0/1623846207 wait complete. 2023-11-06T03:14:21.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:13:38.971687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:13:39.371284+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:14:21.251 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:14:21.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 Processor -- start 2023-11-06T03:14:21.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 -- start start 2023-11-06T03:14:21.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f469815adc0 0x7f4698070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f46980b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469806d330 con 0x7f46980ca570 2023-11-06T03:14:21.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469806b8d0 con 0x7f469815adc0 2023-11-06T03:14:21.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f46a004c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469806be90 con 0x7f46980713c0 2023-11-06T03:14:21.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469e5e9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f46980ca570 0x7f46980c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44400/0 (socket says 172.21.15.157:44400) 2023-11-06T03:14:21.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469e5e9640 1 -- 172.21.15.157:0/241438948 learned_addr learned my addr 172.21.15.157:0/241438948 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:21.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 --2- 172.21.15.157:0/241438948 >> v2:172.21.15.195:3300/0 conn(0x7f469815adc0 0x7f4698070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469d5e7640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f46980b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139220448 0 0) 0x7f469806d330 con 0x7f46980ca570 2023-11-06T03:14:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4684003530 con 0x7f46980ca570 2023-11-06T03:14:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 404588456 0 0) 0x7f4684003530 con 0x7f46980ca570 2023-11-06T03:14:21.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 msgr2=0x7f46980b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f46980b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 >> v2:172.21.15.195:3300/0 conn(0x7f469815adc0 msgr2=0x7f4698070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 --2- 172.21.15.157:0/241438948 >> v2:172.21.15.195:3300/0 conn(0x7f469815adc0 0x7f4698070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698071880 con 0x7f46980ca570 2023-11-06T03:14:21.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4694001a00 con 0x7f46980ca570 2023-11-06T03:14:21.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 >> v1:172.21.15.157:6789/0 conn(0x7f46980ca570 legacy=0x7f46980c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4684003980 0x7f4684003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46840047d0 0x7f4684004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f4684005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806b8d0 con 0x7f4684003980 2023-11-06T03:14:21.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469806be90 con 0x7f46840047d0 2023-11-06T03:14:21.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468400ba40 con 0x7f46980713c0 2023-11-06T03:14:21.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4684003980 0x7f4684003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469e5e9640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f4684005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469d5e7640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46840047d0 0x7f4684004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 -- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 msgr2=0x7f4684005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f46980713c0 0x7f4684005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 -- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46840047d0 msgr2=0x7f4684004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46840047d0 0x7f4684004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469dde8640 1 -- 172.21.15.157:0/241438948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684003530 con 0x7f4684003980 2023-11-06T03:14:21.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.376+0000 7f469d5e7640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46840047d0 0x7f4684004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f469dde8640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4684003980 0x7f4684003da0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4690009ae0 tx=0x7f469001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de4cc0f389051e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900a5550 con 0x7f4684003980 2023-11-06T03:14:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46900a56b0 con 0x7f4684003980 2023-11-06T03:14:21.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f469cde6640 1 -- 172.21.15.157:0/241438948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900a5a60 con 0x7f4684003980 2023-11-06T03:14:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 -- 172.21.15.157:0/241438948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4684003980 msgr2=0x7f4684003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 --2- 172.21.15.157:0/241438948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4684003980 0x7f4684003da0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4690009ae0 tx=0x7f469001ca60 comp rx=0 tx=0).stop 2023-11-06T03:14:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f469e5e9640 1 -- 172.21.15.157:0/241438948 reap_dead start 2023-11-06T03:14:21.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 -- 172.21.15.157:0/241438948 shutdown_connections 2023-11-06T03:14:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 -- 172.21.15.157:0/241438948 >> 172.21.15.157:0/241438948 conn(0x7f46980cb990 msgr2=0x7f46980cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 -- 172.21.15.157:0/241438948 shutdown_connections 2023-11-06T03:14:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.380+0000 7f46a004c640 1 -- 172.21.15.157:0/241438948 wait complete. 2023-11-06T03:14:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 Processor -- start 2023-11-06T03:14:21.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- start start 2023-11-06T03:14:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003980 0x7f46980c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469815c770 0x7f469815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698159000 con 0x7f4698158640 2023-11-06T03:14:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980c5fe0 con 0x7f469815c770 2023-11-06T03:14:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980c8310 con 0x7f4684003980 2023-11-06T03:14:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42250/0 (socket says 172.21.15.157:42250) 2023-11-06T03:14:21.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 -- 172.21.15.157:0/3398949568 learned_addr learned my addr 172.21.15.157:0/3398949568 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:21.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 -- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003980 msgr2=0x7f46980c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003980 0x7f46980c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469e5e9640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469815c770 0x7f469815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 -- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469815c770 msgr2=0x7f469815cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469815c770 0x7f469815cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 -- 172.21.15.157:0/3398949568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469815d540 con 0x7f4698158640 2023-11-06T03:14:21.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f469d5e7640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4688010890 tx=0x7f4688010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688004220 con 0x7f4698158640 2023-11-06T03:14:21.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688004380 con 0x7f4698158640 2023-11-06T03:14:21.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469815d820 con 0x7f4698158640 2023-11-06T03:14:21.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468800eb80 con 0x7f4698158640 2023-11-06T03:14:21.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469815dde0 con 0x7f4698158640 2023-11-06T03:14:21.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698072750 con 0x7f4698158640 2023-11-06T03:14:21.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.384+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f468800ece0 con 0x7f4698158640 2023-11-06T03:14:21.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.388+0000 7f468e7fc640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 0x7f467c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.388+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46880834b0 con 0x7f4698158640 2023-11-06T03:14:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.388+0000 7f469dde8640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 0x7f467c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.388+0000 7f469dde8640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 0x7f467c067f50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f4690004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.392+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4688050d10 con 0x7f4698158640 2023-11-06T03:14:21.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46980c6150 con 0x7f4698158640 2023-11-06T03:14:21.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f468e7fc640 1 -- 172.21.15.157:0/3398949568 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46880044f0 con 0x7f4698158640 2023-11-06T03:14:21.545 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:14:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 msgr2=0x7f467c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 0x7f467c067f50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f4690004740 comp rx=0 tx=0).stop 2023-11-06T03:14:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 msgr2=0x7f4698158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f4688010890 tx=0x7f4688010d50 comp rx=0 tx=0).stop 2023-11-06T03:14:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 shutdown_connections 2023-11-06T03:14:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f467c065a90 0x7f467c067f50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f469815c770 0x7f469815cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4698158640 0x7f4698158ac0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 --2- 172.21.15.157:0/3398949568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4684003980 0x7f46980c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 >> 172.21.15.157:0/3398949568 conn(0x7f46980cb990 msgr2=0x7f46980b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 shutdown_connections 2023-11-06T03:14:21.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.548+0000 7f46a004c640 1 -- 172.21.15.157:0/3398949568 wait complete. 2023-11-06T03:14:21.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:14:21.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 Processor -- start 2023-11-06T03:14:21.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 -- start start 2023-11-06T03:14:21.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd4066040 0x7f1cd406c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cd406c9a0 0x7f1cd40ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd4070420 con 0x7f1cd415b930 2023-11-06T03:14:21.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd4070700 con 0x7f1cd406c9a0 2023-11-06T03:14:21.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cdb2e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40709e0 con 0x7f1cd4066040 2023-11-06T03:14:21.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1cd406c9a0 0x7f1cd40ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd907e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1cd415b930 0x7f1cd4065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44402/0 (socket says 172.21.15.157:44402) 2023-11-06T03:14:21.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd907e640 1 -- 172.21.15.157:0/600591568 learned_addr learned my addr 172.21.15.157:0/600591568 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:21.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd887d640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd4066040 0x7f1cd406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 -- 172.21.15.157:0/600591568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd4066040 msgr2=0x7f1cd406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:21.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd4066040 0x7f1cd406c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 -- 172.21.15.157:0/600591568 >> v1:172.21.15.157:6789/0 conn(0x7f1cd415b930 legacy=0x7f1cd4065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 -- 172.21.15.157:0/600591568 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4070cc0 con 0x7f1cd406c9a0 2023-11-06T03:14:21.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.696+0000 7f1cd987f640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd406c9a0 0x7f1cd40ce860 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009d90 tx=0x7f1ccc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b3be7ba8714f85 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1ccbfff640 1 -- 172.21.15.157:0/600591568 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0a5960 con 0x7f1cd406c9a0 2023-11-06T03:14:21.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1ccbfff640 1 -- 172.21.15.157:0/600591568 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0a5ac0 con 0x7f1cd406c9a0 2023-11-06T03:14:21.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1ccbfff640 1 -- 172.21.15.157:0/600591568 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0a5e50 con 0x7f1cd406c9a0 2023-11-06T03:14:21.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/600591568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd406c9a0 msgr2=0x7f1cd40ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd406c9a0 0x7f1cd40ce860 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009d90 tx=0x7f1ccc01c880 comp rx=0 tx=0).stop 2023-11-06T03:14:21.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/600591568 shutdown_connections 2023-11-06T03:14:21.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd406c9a0 0x7f1cd40ce860 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/600591568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd4066040 0x7f1cd406c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/600591568 >> 172.21.15.157:0/600591568 conn(0x7f1cd40732f0 msgr2=0x7f1cd4069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/600591568 shutdown_connections 2023-11-06T03:14:21.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/600591568 wait complete. 2023-11-06T03:14:21.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 Processor -- start 2023-11-06T03:14:21.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- start start 2023-11-06T03:14:21.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cd4066040 0x7f1cd4145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd406c9a0 0x7f1cd414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd414d430 con 0x7f1cd415b930 2023-11-06T03:14:21.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd414c390 con 0x7f1cd4066040 2023-11-06T03:14:21.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd414c670 con 0x7f1cd406c9a0 2023-11-06T03:14:21.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42264/0 (socket says 172.21.15.157:42264) 2023-11-06T03:14:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 -- 172.21.15.157:0/4076141428 learned_addr learned my addr 172.21.15.157:0/4076141428 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd907e640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cd4066040 0x7f1cd4145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd887d640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd406c9a0 0x7f1cd414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 -- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd406c9a0 msgr2=0x7f1cd414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd406c9a0 0x7f1cd414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 -- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cd4066040 msgr2=0x7f1cd4145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cd4066040 0x7f1cd4145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 -- 172.21.15.157:0/4076141428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc009a70 con 0x7f1cd415b930 2023-11-06T03:14:21.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cd987f640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009d90 tx=0x7f1ccc019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0aedb0 con 0x7f1cd415b930 2023-11-06T03:14:21.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0b7410 con 0x7f1cd415b930 2023-11-06T03:14:21.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd414c950 con 0x7f1cd415b930 2023-11-06T03:14:21.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b77c0 con 0x7f1cd415b930 2023-11-06T03:14:21.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.700+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd414cf10 con 0x7f1cd415b930 2023-11-06T03:14:21.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd406ad00 con 0x7f1cd415b930 2023-11-06T03:14:21.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ccc0b4050 con 0x7f1cd415b930 2023-11-06T03:14:21.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cc9ffb640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 0x7f1cb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:21.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ccc11e420 con 0x7f1cd415b930 2023-11-06T03:14:21.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cd907e640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 0x7f1cb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:21.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.704+0000 7f1cd907e640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 0x7f1cb8067f50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1cd414da80 tx=0x7f1cd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:21.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.708+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1ccc0ebd10 con 0x7f1cd415b930 2023-11-06T03:14:21.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.860+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1cd4072750 con 0x7f1cd415b930 2023-11-06T03:14:21.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.860+0000 7f1cc9ffb640 1 -- 172.21.15.157:0/4076141428 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ccc0ef330 con 0x7f1cd415b930 2023-11-06T03:14:21.859 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:14:21.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 msgr2=0x7f1cb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 0x7f1cb8067f50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1cd414da80 tx=0x7f1cd0009040 comp rx=0 tx=0).stop 2023-11-06T03:14:21.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 msgr2=0x7f1cd414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:21.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009d90 tx=0x7f1ccc019620 comp rx=0 tx=0).stop 2023-11-06T03:14:21.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 shutdown_connections 2023-11-06T03:14:21.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1cb8065a90 0x7f1cb8067f50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cd415b930 0x7f1cd414f830 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cd406c9a0 0x7f1cd414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 --2- 172.21.15.157:0/4076141428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cd4066040 0x7f1cd4145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:21.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 >> 172.21.15.157:0/4076141428 conn(0x7f1cd40732f0 msgr2=0x7f1cd414e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:21.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 shutdown_connections 2023-11-06T03:14:21.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:21.864+0000 7f1cdb2e2640 1 -- 172.21.15.157:0/4076141428 wait complete. 2023-11-06T03:14:21.879 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:14:22.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 Processor -- start 2023-11-06T03:14:22.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 -- start start 2023-11-06T03:14:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55340ce800 0x7f55340cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340cf1a0 con 0x7f5534146370 2023-11-06T03:14:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55340b2120 con 0x7f55340ce800 2023-11-06T03:14:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5539ffe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553406c3d0 con 0x7f553406b6a0 2023-11-06T03:14:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55340ce800 0x7f55340cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f55340ce800 0x7f55340cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42268/0 (socket says 172.21.15.157:42268) 2023-11-06T03:14:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 -- 172.21.15.157:0/2505911614 learned_addr learned my addr 172.21.15.157:0/2505911614 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 -- 172.21.15.157:0/2505911614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 msgr2=0x7f553406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5533fff640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553406bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 -- 172.21.15.157:0/2505911614 >> v1:172.21.15.195:6789/0 conn(0x7f5534146370 legacy=0x7f55340b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.008+0000 7f5532ffd640 1 -- 172.21.15.157:0/2505911614 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553406f5a0 con 0x7f55340ce800 2023-11-06T03:14:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f55340cec60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5524009870 tx=0x7f552401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ad5dca3b1e8395 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f55327fc640 1 -- 172.21.15.157:0/2505911614 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240ab600 con 0x7f55340ce800 2023-11-06T03:14:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f55327fc640 1 -- 172.21.15.157:0/2505911614 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55240ab760 con 0x7f55340ce800 2023-11-06T03:14:22.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f55327fc640 1 -- 172.21.15.157:0/2505911614 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240abaf0 con 0x7f55340ce800 2023-11-06T03:14:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- 172.21.15.157:0/2505911614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 msgr2=0x7f55340cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f55340cec60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5524009870 tx=0x7f552401c440 comp rx=0 tx=0).stop 2023-11-06T03:14:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- 172.21.15.157:0/2505911614 shutdown_connections 2023-11-06T03:14:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- 172.21.15.157:0/2505911614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f55340cec60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- 172.21.15.157:0/2505911614 >> 172.21.15.157:0/2505911614 conn(0x7f5534065930 msgr2=0x7f5534065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- 172.21.15.157:0/2505911614 shutdown_connections 2023-11-06T03:14:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- 172.21.15.157:0/2505911614 wait complete. 2023-11-06T03:14:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 Processor -- start 2023-11-06T03:14:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- start start 2023-11-06T03:14:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553413ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 0x7f553413e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553413a4d0 con 0x7f55340ce800 2023-11-06T03:14:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55341416c0 con 0x7f5534146370 2023-11-06T03:14:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5539ffe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5534144120 con 0x7f553406b6a0 2023-11-06T03:14:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42278/0 (socket says 172.21.15.157:42278) 2023-11-06T03:14:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 -- 172.21.15.157:0/428441213 learned_addr learned my addr 172.21.15.157:0/428441213 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f55337fe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553413ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5533fff640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 0x7f553413e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 -- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 msgr2=0x7f553413ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553413ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 -- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 msgr2=0x7f553413e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 0x7f553413e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5532ffd640 1 -- 172.21.15.157:0/428441213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240094d0 con 0x7f55340ce800 2023-11-06T03:14:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.012+0000 7f5533fff640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 0x7f553413e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f5532ffd640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f552401aaf0 tx=0x7f552401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240b4800 con 0x7f55340ce800 2023-11-06T03:14:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55240b4960 con 0x7f55340ce800 2023-11-06T03:14:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5534143b10 con 0x7f55340ce800 2023-11-06T03:14:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240b4d10 con 0x7f55340ce800 2023-11-06T03:14:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5534143db0 con 0x7f55340ce800 2023-11-06T03:14:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5534143970 con 0x7f55340ce800 2023-11-06T03:14:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f55240bd390 con 0x7f55340ce800 2023-11-06T03:14:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 0x7f5520067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f552411d950 con 0x7f55340ce800 2023-11-06T03:14:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f55337fe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 0x7f5520067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.016+0000 7f55337fe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 0x7f5520067f20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5534142530 tx=0x7f5528009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.020+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5534143970 con 0x7f55340ce800 2023-11-06T03:14:22.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.172+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5534143950 con 0x7f55340ce800 2023-11-06T03:14:22.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.172+0000 7f551bfff640 1 -- 172.21.15.157:0/428441213 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f55240eb260 con 0x7f55340ce800 2023-11-06T03:14:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:14:22.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 msgr2=0x7f5520067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 0x7f5520067f20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5534142530 tx=0x7f5528009040 comp rx=0 tx=0).stop 2023-11-06T03:14:22.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 msgr2=0x7f553415c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f552401aaf0 tx=0x7f552401ac80 comp rx=0 tx=0).stop 2023-11-06T03:14:22.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 shutdown_connections 2023-11-06T03:14:22.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5520065a60 0x7f5520067f20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5534146370 0x7f553413e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55340ce800 0x7f553415c250 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 --2- 172.21.15.157:0/428441213 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f553406b6a0 0x7f553413ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 >> 172.21.15.157:0/428441213 conn(0x7f5534065930 msgr2=0x7f5534160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 shutdown_connections 2023-11-06T03:14:22.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.176+0000 7f5539ffe640 1 -- 172.21.15.157:0/428441213 wait complete. 2023-11-06T03:14:22.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:14:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.316+0000 7f8c9cd72640 1 Processor -- start 2023-11-06T03:14:22.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.316+0000 7f8c9cd72640 1 -- start start 2023-11-06T03:14:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c9cd72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c9cd72640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9815f580 con 0x7f8c9815ea60 2023-11-06T03:14:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c9cd72640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9815efc0 con 0x7f8c981551e0 2023-11-06T03:14:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c9cd72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980704b0 con 0x7f8c9815e5e0 2023-11-06T03:14:22.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c96d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8c9815ea60 0x7f8c9815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44414/0 (socket says 172.21.15.157:44414) 2023-11-06T03:14:22.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c96d9d640 1 -- 172.21.15.157:0/889866931 learned_addr learned my addr 172.21.15.157:0/889866931 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:22.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190503228 0 0) 0x7f8c9815f580 con 0x7f8c9815ea60 2023-11-06T03:14:22.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c84003530 con 0x7f8c9815ea60 2023-11-06T03:14:22.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3405141285 0 0) 0x7f8c84003530 con 0x7f8c9815ea60 2023-11-06T03:14:22.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 msgr2=0x7f8c980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c980b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 >> v1:172.21.15.195:6789/0 conn(0x7f8c981551e0 legacy=0x7f8c9806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9815f2a0 con 0x7f8c9815ea60 2023-11-06T03:14:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8c90001a00 con 0x7f8c9815ea60 2023-11-06T03:14:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 >> v1:172.21.15.157:6789/0 conn(0x7f8c9815ea60 legacy=0x7f8c9815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c840039b0 0x7f8c84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c840051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980704b0 con 0x7f8c840047d0 2023-11-06T03:14:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400bb20 con 0x7f8c840039b0 2023-11-06T03:14:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400bcb0 con 0x7f8c9815e5e0 2023-11-06T03:14:22.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c9659c640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c840039b0 0x7f8c84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c96d9d640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c840051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 -- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 msgr2=0x7f8c840051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815e5e0 0x7f8c840051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 -- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c840039b0 msgr2=0x7f8c84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c840039b0 0x7f8c84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.320+0000 7f8c8dd9b640 1 -- 172.21.15.157:0/889866931 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84003530 con 0x7f8c840047d0 2023-11-06T03:14:22.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c84004c50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009b10 tx=0x7f8c8001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ea8f5eff2c6a1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800a6510 con 0x7f8c840047d0 2023-11-06T03:14:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c800a6670 con 0x7f8c840047d0 2023-11-06T03:14:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c95d9b640 1 -- 172.21.15.157:0/889866931 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800a6a20 con 0x7f8c840047d0 2023-11-06T03:14:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/889866931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 msgr2=0x7f8c84004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/889866931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c84004c50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009b10 tx=0x7f8c8001ca20 comp rx=0 tx=0).stop 2023-11-06T03:14:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c96d9d640 1 -- 172.21.15.157:0/889866931 reap_dead start 2023-11-06T03:14:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/889866931 shutdown_connections 2023-11-06T03:14:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/889866931 >> 172.21.15.157:0/889866931 conn(0x7f8c98065930 msgr2=0x7f8c98065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/889866931 shutdown_connections 2023-11-06T03:14:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/889866931 wait complete. 2023-11-06T03:14:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 Processor -- start 2023-11-06T03:14:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- start start 2023-11-06T03:14:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815a8f0 0x7f8c981f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c981555c0 0x7f8c980cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98155ec0 con 0x7f8c840047d0 2023-11-06T03:14:22.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981561a0 con 0x7f8c981555c0 2023-11-06T03:14:22.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98156480 con 0x7f8c9815a8f0 2023-11-06T03:14:22.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42294/0 (socket says 172.21.15.157:42294) 2023-11-06T03:14:22.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 -- 172.21.15.157:0/2784667784 learned_addr learned my addr 172.21.15.157:0/2784667784 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:22.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815a8f0 0x7f8c981f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c96d9d640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c981555c0 0x7f8c980cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 -- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815a8f0 msgr2=0x7f8c981f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815a8f0 0x7f8c981f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 -- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c981555c0 msgr2=0x7f8c980cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c981555c0 0x7f8c980cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 -- 172.21.15.157:0/2784667784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98156760 con 0x7f8c840047d0 2023-11-06T03:14:22.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9659c640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e490 tx=0x7f8c8800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c880052e0 con 0x7f8c840047d0 2023-11-06T03:14:22.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c88005440 con 0x7f8c840047d0 2023-11-06T03:14:22.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98156a40 con 0x7f8c840047d0 2023-11-06T03:14:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88003840 con 0x7f8c840047d0 2023-11-06T03:14:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.324+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c98155a40 con 0x7f8c840047d0 2023-11-06T03:14:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98155c60 con 0x7f8c840047d0 2023-11-06T03:14:22.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c880039a0 con 0x7f8c840047d0 2023-11-06T03:14:22.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c8f7fe640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 0x7f8c78067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c88083010 con 0x7f8c840047d0 2023-11-06T03:14:22.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 0x7f8c78067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.328+0000 7f8c8dd9b640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 0x7f8c78067f50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8c800af000 tx=0x7f8c800046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.332+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8c88050930 con 0x7f8c840047d0 2023-11-06T03:14:22.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c98072750 con 0x7f8c840047d0 2023-11-06T03:14:22.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c8f7fe640 1 -- 172.21.15.157:0/2784667784 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c88053f50 con 0x7f8c840047d0 2023-11-06T03:14:22.485 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:14:22.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 msgr2=0x7f8c78067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 0x7f8c78067f50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8c800af000 tx=0x7f8c800046d0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 msgr2=0x7f8c98145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e490 tx=0x7f8c8800e950 comp rx=0 tx=0).stop 2023-11-06T03:14:22.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 shutdown_connections 2023-11-06T03:14:22.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8c78065a90 0x7f8c78067f50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8c981555c0 0x7f8c980cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8c9815a8f0 0x7f8c981f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 --2- 172.21.15.157:0/2784667784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c840047d0 0x7f8c98145530 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 >> 172.21.15.157:0/2784667784 conn(0x7f8c98065930 msgr2=0x7f8c98067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 shutdown_connections 2023-11-06T03:14:22.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.488+0000 7f8c9cd72640 1 -- 172.21.15.157:0/2784667784 wait complete. 2023-11-06T03:14:22.505 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:22.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 Processor -- start 2023-11-06T03:14:22.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 -- start start 2023-11-06T03:14:22.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d500d0280 0x7f8d50167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500d2920 con 0x7f8d5015ce80 2023-11-06T03:14:22.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500d18a0 con 0x7f8d5015caa0 2023-11-06T03:14:22.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d56aa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d500d1b60 con 0x7f8d500d0280 2023-11-06T03:14:22.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d4ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8d5015ce80 0x7f8d500cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44430/0 (socket says 172.21.15.157:44430) 2023-11-06T03:14:22.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d4ffff640 1 -- 172.21.15.157:0/3369747869 learned_addr learned my addr 172.21.15.157:0/3369747869 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:22.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d5503f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d500d0280 0x7f8d50167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d4f7fe640 1 -- 172.21.15.157:0/3369747869 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105521757 0 0) 0x7f8d500d2920 con 0x7f8d5015ce80 2023-11-06T03:14:22.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d4f7fe640 1 -- 172.21.15.157:0/3369747869 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d38003530 con 0x7f8d5015ce80 2023-11-06T03:14:22.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d5503f640 1 -- 172.21.15.157:0/3369747869 >> v1:172.21.15.195:6789/0 conn(0x7f8d5015caa0 legacy=0x7f8d500cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d5503f640 1 -- 172.21.15.157:0/3369747869 >> v1:172.21.15.157:6789/0 conn(0x7f8d5015ce80 legacy=0x7f8d500cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.628+0000 7f8d5503f640 1 -- 172.21.15.157:0/3369747869 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500d1e40 con 0x7f8d500d0280 2023-11-06T03:14:22.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 --2- 172.21.15.157:0/3369747869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d500d0280 0x7f8d50167860 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009d90 tx=0x7f8d4801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b8aa7690a1a56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d4f7fe640 1 -- 172.21.15.157:0/3369747869 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480a59e0 con 0x7f8d500d0280 2023-11-06T03:14:22.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d4f7fe640 1 -- 172.21.15.157:0/3369747869 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480a5b40 con 0x7f8d500d0280 2023-11-06T03:14:22.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d4f7fe640 1 -- 172.21.15.157:0/3369747869 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480ad440 con 0x7f8d500d0280 2023-11-06T03:14:22.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- 172.21.15.157:0/3369747869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d500d0280 msgr2=0x7f8d50167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/3369747869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d500d0280 0x7f8d50167860 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009d90 tx=0x7f8d4801c920 comp rx=0 tx=0).stop 2023-11-06T03:14:22.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- 172.21.15.157:0/3369747869 shutdown_connections 2023-11-06T03:14:22.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/3369747869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d500d0280 0x7f8d50167860 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- 172.21.15.157:0/3369747869 >> 172.21.15.157:0/3369747869 conn(0x7f8d50070790 msgr2=0x7f8d50070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- 172.21.15.157:0/3369747869 shutdown_connections 2023-11-06T03:14:22.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- 172.21.15.157:0/3369747869 wait complete. 2023-11-06T03:14:22.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 Processor -- start 2023-11-06T03:14:22.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- start start 2023-11-06T03:14:22.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d500d0280 0x7f8d50160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5015caa0 0x7f8d500c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500ccf50 con 0x7f8d5015ce80 2023-11-06T03:14:22.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500ca230 con 0x7f8d5015caa0 2023-11-06T03:14:22.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d56aa2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500ca440 con 0x7f8d500d0280 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42312/0 (socket says 172.21.15.157:42312) 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 -- 172.21.15.157:0/2586660714 learned_addr learned my addr 172.21.15.157:0/2586660714 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5483e640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d500d0280 0x7f8d50160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d4ffff640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5015caa0 0x7f8d500c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 -- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d500d0280 msgr2=0x7f8d50160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d500d0280 0x7f8d50160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 -- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5015caa0 msgr2=0x7f8d500c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5015caa0 0x7f8d500c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.632+0000 7f8d5503f640 1 -- 172.21.15.157:0/2586660714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009a70 con 0x7f8d5015ce80 2023-11-06T03:14:22.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d5503f640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801a550 tx=0x7f8d480196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480add10 con 0x7f8d5015ce80 2023-11-06T03:14:22.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480ade70 con 0x7f8d5015ce80 2023-11-06T03:14:22.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d500cc360 con 0x7f8d5015ce80 2023-11-06T03:14:22.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480b9650 con 0x7f8d5015ce80 2023-11-06T03:14:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d500cc890 con 0x7f8d5015ce80 2023-11-06T03:14:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d500b1ec0 con 0x7f8d5015ce80 2023-11-06T03:14:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d480b97b0 con 0x7f8d5015ce80 2023-11-06T03:14:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 0x7f8d3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d4811e9d0 con 0x7f8d5015ce80 2023-11-06T03:14:22.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d5483e640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 0x7f8d3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.636+0000 7f8d5483e640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 0x7f8d3c067f20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f70 tx=0x7f8d44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.640+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d480ec2c0 con 0x7f8d5015ce80 2023-11-06T03:14:22.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.820+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d50072750 con 0x7f8d5015ce80 2023-11-06T03:14:22.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.820+0000 7f8d4d7fa640 1 -- 172.21.15.157:0/2586660714 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8d48009450 con 0x7f8d5015ce80 2023-11-06T03:14:22.819 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:22.819 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:22.820 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 msgr2=0x7f8d3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 0x7f8d3c067f20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f70 tx=0x7f8d44009040 comp rx=0 tx=0).stop 2023-11-06T03:14:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 msgr2=0x7f8d501524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801a550 tx=0x7f8d480196c0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 shutdown_connections 2023-11-06T03:14:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d3c065a60 0x7f8d3c067f20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d5015ce80 0x7f8d501524a0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d5015caa0 0x7f8d500c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 --2- 172.21.15.157:0/2586660714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d500d0280 0x7f8d50160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 >> 172.21.15.157:0/2586660714 conn(0x7f8d50070790 msgr2=0x7f8d500b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 shutdown_connections 2023-11-06T03:14:22.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.824+0000 7f8d56aa2640 1 -- 172.21.15.157:0/2586660714 wait complete. 2023-11-06T03:14:22.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.952+0000 7f36ae587640 1 Processor -- start 2023-11-06T03:14:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.952+0000 7f36ae587640 1 -- start start 2023-11-06T03:14:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36ae587640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36a80b2af0 0x7f36a80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36ae587640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 0x7f36a8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36ae587640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a815e440 con 0x7f36a81599c0 2023-11-06T03:14:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36ae587640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a806c240 con 0x7f36a80b2af0 2023-11-06T03:14:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36ae587640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a806c520 con 0x7f36a806d000 2023-11-06T03:14:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36a80b2af0 0x7f36a80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36a7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f36a81599c0 0x7f36a80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44436/0 (socket says 172.21.15.157:44436) 2023-11-06T03:14:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36a7fff640 1 -- 172.21.15.157:0/2411784707 learned_addr learned my addr 172.21.15.157:0/2411784707 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36acb24640 1 --2- 172.21.15.157:0/2411784707 >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 0x7f36a8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 -- 172.21.15.157:0/2411784707 >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 msgr2=0x7f36a8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 --2- 172.21.15.157:0/2411784707 >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 0x7f36a8159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 -- 172.21.15.157:0/2411784707 >> v1:172.21.15.157:6789/0 conn(0x7f36a81599c0 legacy=0x7f36a80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 -- 172.21.15.157:0/2411784707 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a806c800 con 0x7f36a80b2af0 2023-11-06T03:14:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36acb24640 1 --2- 172.21.15.157:0/2411784707 >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 0x7f36a8159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:22.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f369f7fe640 1 --2- 172.21.15.157:0/2411784707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a80b2af0 0x7f36a80ae000 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f36a806d930 tx=0x7f369001eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec343bbb01deb46 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36a77fe640 1 -- 172.21.15.157:0/2411784707 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36900a7bf0 con 0x7f36a80b2af0 2023-11-06T03:14:22.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36a77fe640 1 -- 172.21.15.157:0/2411784707 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36900a7d50 con 0x7f36a80b2af0 2023-11-06T03:14:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.956+0000 7f36a77fe640 1 -- 172.21.15.157:0/2411784707 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36900af660 con 0x7f36a80b2af0 2023-11-06T03:14:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/2411784707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a80b2af0 msgr2=0x7f36a80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- 172.21.15.157:0/2411784707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a80b2af0 0x7f36a80ae000 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f36a806d930 tx=0x7f369001eb00 comp rx=0 tx=0).stop 2023-11-06T03:14:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/2411784707 shutdown_connections 2023-11-06T03:14:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- 172.21.15.157:0/2411784707 >> v2:172.21.15.195:3300/0 conn(0x7f36a806d000 0x7f36a8159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- 172.21.15.157:0/2411784707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a80b2af0 0x7f36a80ae000 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/2411784707 >> 172.21.15.157:0/2411784707 conn(0x7f36a8065930 msgr2=0x7f36a8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/2411784707 shutdown_connections 2023-11-06T03:14:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/2411784707 wait complete. 2023-11-06T03:14:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 Processor -- start 2023-11-06T03:14:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- start start 2023-11-06T03:14:22.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36a806d000 0x7f36a814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a80b2af0 0x7f36a80aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80af190 con 0x7f36a81599c0 2023-11-06T03:14:22.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a814f0c0 con 0x7f36a80b2af0 2023-11-06T03:14:22.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a814f3a0 con 0x7f36a806d000 2023-11-06T03:14:22.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42326/0 (socket says 172.21.15.157:42326) 2023-11-06T03:14:22.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 -- 172.21.15.157:0/662609416 learned_addr learned my addr 172.21.15.157:0/662609416 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:22.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 -- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36a806d000 msgr2=0x7f36a814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36a806d000 0x7f36a814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 -- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a80b2af0 msgr2=0x7f36a80aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:22.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a80b2af0 0x7f36a80aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:22.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 -- 172.21.15.157:0/662609416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369000b6c0 con 0x7f36a81599c0 2023-11-06T03:14:22.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36acb24640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f36a000cea0 tx=0x7f36a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0003ba0 con 0x7f36a81599c0 2023-11-06T03:14:22.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0003d20 con 0x7f36a81599c0 2023-11-06T03:14:22.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a814f680 con 0x7f36a81599c0 2023-11-06T03:14:22.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000d880 con 0x7f36a81599c0 2023-11-06T03:14:22.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.960+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a80af430 con 0x7f36a81599c0 2023-11-06T03:14:22.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a8072750 con 0x7f36a81599c0 2023-11-06T03:14:22.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f36a000d9e0 con 0x7f36a81599c0 2023-11-06T03:14:22.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36a57fa640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 0x7f3688067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:22.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36a0083b60 con 0x7f36a81599c0 2023-11-06T03:14:22.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36a7fff640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 0x7f3688067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:22.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.964+0000 7f36a7fff640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 0x7f3688067f20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f36a81f4290 tx=0x7f3698007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:22.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:22.968+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f36a0051450 con 0x7f36a81599c0 2023-11-06T03:14:23.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.124+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36a814fb30 con 0x7f36a81599c0 2023-11-06T03:14:23.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36a57fa640 1 -- 172.21.15.157:0/662609416 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f36a0054a70 con 0x7f36a81599c0 2023-11-06T03:14:23.126 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:23.127 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:41.413681+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:23.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 msgr2=0x7f3688067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 0x7f3688067f20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f36a81f4290 tx=0x7f3698007a50 comp rx=0 tx=0).stop 2023-11-06T03:14:23.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 msgr2=0x7f36a80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f36a000cea0 tx=0x7f36a000cf90 comp rx=0 tx=0).stop 2023-11-06T03:14:23.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 shutdown_connections 2023-11-06T03:14:23.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3688065a60 0x7f3688067f20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36a81599c0 0x7f36a80cc480 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a80b2af0 0x7f36a80aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 --2- 172.21.15.157:0/662609416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f36a806d000 0x7f36a814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 >> 172.21.15.157:0/662609416 conn(0x7f36a8065930 msgr2=0x7f36a814def0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 shutdown_connections 2023-11-06T03:14:23.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.128+0000 7f36ae587640 1 -- 172.21.15.157:0/662609416 wait complete. 2023-11-06T03:14:23.144 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:14:23.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:23.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 Processor -- start 2023-11-06T03:14:23.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 -- start start 2023-11-06T03:14:23.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95801599c0 0x7f95800ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95800b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 0x7f95800b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958015e440 con 0x7f95801599c0 2023-11-06T03:14:23.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958006c240 con 0x7f958015dea0 2023-11-06T03:14:23.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9586e76640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958006c520 con 0x7f958006d000 2023-11-06T03:14:23.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95801599c0 0x7f95800ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f95801599c0 0x7f95800ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42328/0 (socket says 172.21.15.157:42328) 2023-11-06T03:14:23.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 -- 172.21.15.157:0/2851083626 learned_addr learned my addr 172.21.15.157:0/2851083626 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:23.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9585413640 1 --2- 172.21.15.157:0/2851083626 >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 0x7f95800b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f957ffff640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95800b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 -- 172.21.15.157:0/2851083626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 msgr2=0x7f95800b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95800b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 -- 172.21.15.157:0/2851083626 >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 msgr2=0x7f95800b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 --2- 172.21.15.157:0/2851083626 >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 0x7f95800b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9584c12640 1 -- 172.21.15.157:0/2851083626 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958006c800 con 0x7f95801599c0 2023-11-06T03:14:23.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.260+0000 7f9585413640 1 --2- 172.21.15.157:0/2851083626 >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 0x7f95800b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9584c12640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f95800ae000 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f958006bbf0 tx=0x7f956c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0ec4e12b07bfe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957f7fe640 1 -- 172.21.15.157:0/2851083626 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0aa610 con 0x7f95801599c0 2023-11-06T03:14:23.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957f7fe640 1 -- 172.21.15.157:0/2851083626 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c0a6040 con 0x7f95801599c0 2023-11-06T03:14:23.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957f7fe640 1 -- 172.21.15.157:0/2851083626 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0aa990 con 0x7f95801599c0 2023-11-06T03:14:23.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- 172.21.15.157:0/2851083626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 msgr2=0x7f95800ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f95800ae000 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f958006bbf0 tx=0x7f956c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- 172.21.15.157:0/2851083626 shutdown_connections 2023-11-06T03:14:23.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- 172.21.15.157:0/2851083626 >> v2:172.21.15.195:3300/0 conn(0x7f958015dea0 0x7f95800b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95800b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- 172.21.15.157:0/2851083626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f95800ae000 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- 172.21.15.157:0/2851083626 >> 172.21.15.157:0/2851083626 conn(0x7f9580065930 msgr2=0x7f9580065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- 172.21.15.157:0/2851083626 shutdown_connections 2023-11-06T03:14:23.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- 172.21.15.157:0/2851083626 wait complete. 2023-11-06T03:14:23.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 Processor -- start 2023-11-06T03:14:23.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- start start 2023-11-06T03:14:23.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95801464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 0x7f9580149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800cd480 con 0x7f95801599c0 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801469f0 con 0x7f958015dea0 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9586e76640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800ccb00 con 0x7f958006d000 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42338/0 (socket says 172.21.15.157:42338) 2023-11-06T03:14:23.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 -- 172.21.15.157:0/1062217967 learned_addr learned my addr 172.21.15.157:0/1062217967 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:23.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9585413640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 0x7f9580149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 -- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 msgr2=0x7f95801464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95801464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 -- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 msgr2=0x7f9580149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 0x7f9580149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 -- 172.21.15.157:0/1062217967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c0094d0 con 0x7f95801599c0 2023-11-06T03:14:23.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f9585413640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 0x7f9580149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957ffff640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f957400c7f0 tx=0x7f957400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574004220 con 0x7f95801599c0 2023-11-06T03:14:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95740043a0 con 0x7f95801599c0 2023-11-06T03:14:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.264+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574004510 con 0x7f95801599c0 2023-11-06T03:14:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9580149e60 con 0x7f95801599c0 2023-11-06T03:14:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95800cd0a0 con 0x7f95801599c0 2023-11-06T03:14:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9580072750 con 0x7f95801599c0 2023-11-06T03:14:23.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9574003a70 con 0x7f95801599c0 2023-11-06T03:14:23.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f957d7fa640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 0x7f9564067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9574083320 con 0x7f95801599c0 2023-11-06T03:14:23.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f9584c12640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 0x7f9564067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.268+0000 7f9584c12640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 0x7f9564067f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ae000 tx=0x7f956c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.272+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9574050ae0 con 0x7f95801599c0 2023-11-06T03:14:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95800cb110 con 0x7f95801599c0 2023-11-06T03:14:23.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f957d7fa640 1 -- 172.21.15.157:0/1062217967 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f9574054100 con 0x7f95801599c0 2023-11-06T03:14:23.449 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:23.449 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 msgr2=0x7f9564067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 0x7f9564067f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f956c0ae000 tx=0x7f956c008040 comp rx=0 tx=0).stop 2023-11-06T03:14:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 msgr2=0x7f9580149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f957400c7f0 tx=0x7f957400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 shutdown_connections 2023-11-06T03:14:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9564065a90 0x7f9564067f50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f958015dea0 0x7f9580149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95801599c0 0x7f9580149310 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 --2- 172.21.15.157:0/1062217967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f958006d000 0x7f95801464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 >> 172.21.15.157:0/1062217967 conn(0x7f9580065930 msgr2=0x7f95801530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 shutdown_connections 2023-11-06T03:14:23.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.452+0000 7f9586e76640 1 -- 172.21.15.157:0/1062217967 wait complete. 2023-11-06T03:14:23.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.576+0000 7fdc21a3c640 1 Processor -- start 2023-11-06T03:14:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.576+0000 7fdc21a3c640 1 -- start start 2023-11-06T03:14:23.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 0x7fdc1c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 0x7fdc1c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0d2920 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0d18a0 con 0x7fdc1c15ce80 2023-11-06T03:14:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc21a3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0d1b60 con 0x7fdc1c15caa0 2023-11-06T03:14:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42354/0 (socket says 172.21.15.157:42354) 2023-11-06T03:14:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 -- 172.21.15.157:0/3917436987 learned_addr learned my addr 172.21.15.157:0/3917436987 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1a7fc640 1 --2- 172.21.15.157:0/3917436987 >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 0x7fdc1c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 -- 172.21.15.157:0/3917436987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 msgr2=0x7fdc1c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 0x7fdc1c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 0x7fdc1c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 -- 172.21.15.157:0/3917436987 >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 msgr2=0x7fdc1c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 --2- 172.21.15.157:0/3917436987 >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 0x7fdc1c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 -- 172.21.15.157:0/3917436987 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0d1e40 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1a7fc640 1 --2- 172.21.15.157:0/3917436987 >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 0x7fdc1c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 0x7fdc1c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc1b7fe640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0d3cb0 tx=0x7fdc1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aefc5987b6e920d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc19ffb640 1 -- 172.21.15.157:0/3917436987 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5820 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc19ffb640 1 -- 172.21.15.157:0/3917436987 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140a5980 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.580+0000 7fdc19ffb640 1 -- 172.21.15.157:0/3917436987 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5d10 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3917436987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0ce1a0 msgr2=0x7fdc1c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0d3cb0 tx=0x7fdc1401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3917436987 shutdown_connections 2023-11-06T03:14:23.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c0ce1a0 0x7fdc1c0d00b0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3917436987 >> v2:172.21.15.195:3300/0 conn(0x7fdc1c15ce80 0x7fdc1c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3917436987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c15caa0 0x7fdc1c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3917436987 >> 172.21.15.157:0/3917436987 conn(0x7fdc1c070790 msgr2=0x7fdc1c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3917436987 shutdown_connections 2023-11-06T03:14:23.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3917436987 wait complete. 2023-11-06T03:14:23.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 Processor -- start 2023-11-06T03:14:23.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- start start 2023-11-06T03:14:23.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 0x7fdc1c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 0x7fdc1c0c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cada0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0c71a0 con 0x7fdc1c15ce80 2023-11-06T03:14:23.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cb010 con 0x7fdc1c0ce1a0 2023-11-06T03:14:23.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42358/0 (socket says 172.21.15.157:42358) 2023-11-06T03:14:23.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 -- 172.21.15.157:0/3642202312 learned_addr learned my addr 172.21.15.157:0/3642202312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:23.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 0x7fdc1c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1b7fe640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 0x7fdc1c0c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 -- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 msgr2=0x7fdc1c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 0x7fdc1c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 -- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 msgr2=0x7fdc1c0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 0x7fdc1c0c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 -- 172.21.15.157:0/3642202312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009b30 con 0x7fdc1c15caa0 2023-11-06T03:14:23.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 0x7fdc1c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1b7fe640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 0x7fdc1c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc1a7fc640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c7c0 tx=0x7fdc1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10007a30 con 0x7fdc1c15caa0 2023-11-06T03:14:23.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10007b90 con 0x7fdc1c15caa0 2023-11-06T03:14:23.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0cb2f0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000ce70 con 0x7fdc1c15caa0 2023-11-06T03:14:23.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.584+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0b3bd0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c0cb530 con 0x7fdc1c15caa0 2023-11-06T03:14:23.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdc100113f0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc03fff640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 0x7fdc08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc10083780 con 0x7fdc1c15caa0 2023-11-06T03:14:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 0x7fdc08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.588+0000 7fdc1affd640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 0x7fdc08067f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.592+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdc1c0cb530 con 0x7fdc1c15caa0 2023-11-06T03:14:23.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.764+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc1c0cb4e0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.764+0000 7fdc03fff640 1 -- 172.21.15.157:0/3642202312 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fdc10050fc0 con 0x7fdc1c15caa0 2023-11-06T03:14:23.764 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:23.764 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:13:40.410329+0000","modified":"2023-11-06T03:13:42.426309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/3017914039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3017914039},{"type":"v1","addr":"172.21.15.157:6835","nonce":3017914039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-06T03:14:23.764 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 91 2023-11-06T03:14:23.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 msgr2=0x7fdc08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 0x7fdc08067f20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2023-11-06T03:14:23.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 msgr2=0x7fdc1c0c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c7c0 tx=0x7fdc1000cc80 comp rx=0 tx=0).stop 2023-11-06T03:14:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 shutdown_connections 2023-11-06T03:14:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdc08065a60 0x7fdc08067f20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdc1c15ce80 0x7fdc1c0c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc1c15caa0 0x7fdc1c0c6750 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 --2- 172.21.15.157:0/3642202312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdc1c0ce1a0 0x7fdc1c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 >> 172.21.15.157:0/3642202312 conn(0x7fdc1c070790 msgr2=0x7fdc1c070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 shutdown_connections 2023-11-06T03:14:23.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.768+0000 7fdc21a3c640 1 -- 172.21.15.157:0/3642202312 wait complete. 2023-11-06T03:14:23.782 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:23.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.900+0000 7f991af7e640 1 Processor -- start 2023-11-06T03:14:23.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.900+0000 7f991af7e640 1 -- start start 2023-11-06T03:14:23.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991af7e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f991415e5e0 0x7f99140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991af7e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991415f580 con 0x7f99141551e0 2023-11-06T03:14:23.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991af7e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991415efc0 con 0x7f991415ea90 2023-11-06T03:14:23.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991af7e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99140704b0 con 0x7f991415e5e0 2023-11-06T03:14:23.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f9918d1a640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99141551e0 0x7f991406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44444/0 (socket says 172.21.15.157:44444) 2023-11-06T03:14:23.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f9918d1a640 1 -- 172.21.15.157:0/888181721 learned_addr learned my addr 172.21.15.157:0/888181721 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:23.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f9913fff640 1 --2- 172.21.15.157:0/888181721 >> v2:172.21.15.195:3300/0 conn(0x7f991415e5e0 0x7f99140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946094775 0 0) 0x7f991415f580 con 0x7f99141551e0 2023-11-06T03:14:23.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98fc003530 con 0x7f99141551e0 2023-11-06T03:14:23.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850169283 0 0) 0x7f98fc003530 con 0x7f99141551e0 2023-11-06T03:14:23.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 >> v2:172.21.15.195:3300/0 conn(0x7f991415e5e0 msgr2=0x7f99140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 --2- 172.21.15.157:0/888181721 >> v2:172.21.15.195:3300/0 conn(0x7f991415e5e0 0x7f99140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 >> v1:172.21.15.195:6789/0 conn(0x7f991415ea90 legacy=0x7f991415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991415f2a0 con 0x7f99141551e0 2023-11-06T03:14:23.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9904001680 con 0x7f99141551e0 2023-11-06T03:14:23.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 >> v1:172.21.15.157:6789/0 conn(0x7f99141551e0 legacy=0x7f991406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98fc003980 0x7f98fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0047d0 0x7f98fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 0x7f98fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140704b0 con 0x7f98fc0052b0 2023-11-06T03:14:23.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc00c180 con 0x7f98fc003980 2023-11-06T03:14:23.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc00c340 con 0x7f98fc0047d0 2023-11-06T03:14:23.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 0x7f98fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f9918d1a640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98fc003980 0x7f98fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f9913fff640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0047d0 0x7f98fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 -- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0047d0 msgr2=0x7f98fc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0047d0 0x7f98fc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 -- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98fc003980 msgr2=0x7f98fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:23.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98fc003980 0x7f98fc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.904+0000 7f991951b640 1 -- 172.21.15.157:0/888181721 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc003530 con 0x7f98fc0052b0 2023-11-06T03:14:23.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991951b640 1 -- 172.21.15.157:0/888181721 reap_dead start 2023-11-06T03:14:23.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991951b640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 0x7f98fc00b990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f990c001e40 tx=0x7f990c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c918583d991bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c003b80 con 0x7f98fc0052b0 2023-11-06T03:14:23.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c003ce0 con 0x7f98fc0052b0 2023-11-06T03:14:23.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f99137fe640 1 -- 172.21.15.157:0/888181721 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0052f0 con 0x7f98fc0052b0 2023-11-06T03:14:23.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 msgr2=0x7f98fc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 0x7f98fc00b990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f990c001e40 tx=0x7f990c01c740 comp rx=0 tx=0).stop 2023-11-06T03:14:23.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- 172.21.15.157:0/888181721 shutdown_connections 2023-11-06T03:14:23.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 --2- 172.21.15.157:0/888181721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc0052b0 0x7f98fc00b990 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- 172.21.15.157:0/888181721 >> 172.21.15.157:0/888181721 conn(0x7f9914065930 msgr2=0x7f9914065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:23.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- 172.21.15.157:0/888181721 shutdown_connections 2023-11-06T03:14:23.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- 172.21.15.157:0/888181721 wait complete. 2023-11-06T03:14:23.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 Processor -- start 2023-11-06T03:14:23.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- start start 2023-11-06T03:14:23.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0052b0 0x7f991413e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 0x7f9914159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991406c0a0 con 0x7f9914149240 2023-11-06T03:14:23.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991406b490 con 0x7f99141e4ca0 2023-11-06T03:14:23.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991af7e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991406b770 con 0x7f98fc0052b0 2023-11-06T03:14:23.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f9913fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f9913fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42376/0 (socket says 172.21.15.157:42376) 2023-11-06T03:14:23.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f9913fff640 1 -- 172.21.15.157:0/2045979539 learned_addr learned my addr 172.21.15.157:0/2045979539 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:23.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f9918d1a640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0052b0 0x7f991413e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.908+0000 7f991951b640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 0x7f9914159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 -- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0052b0 msgr2=0x7f991413e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0052b0 0x7f991413e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 -- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 msgr2=0x7f9914159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:23.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 0x7f9914159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:23.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 -- 172.21.15.157:0/2045979539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991406ba50 con 0x7f9914149240 2023-11-06T03:14:23.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f991951b640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 0x7f9914159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:23.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9913fff640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f990800f580 tx=0x7f990800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908013810 con 0x7f9914149240 2023-11-06T03:14:23.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908013970 con 0x7f9914149240 2023-11-06T03:14:23.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991406bd30 con 0x7f9914149240 2023-11-06T03:14:23.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908013d20 con 0x7f9914149240 2023-11-06T03:14:23.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991414d4e0 con 0x7f9914149240 2023-11-06T03:14:23.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9914072750 con 0x7f9914149240 2023-11-06T03:14:23.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f990801f460 con 0x7f9914149240 2023-11-06T03:14:23.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 0x7f9900067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:23.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9908084170 con 0x7f9914149240 2023-11-06T03:14:23.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9918d1a640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 0x7f9900067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:23.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.912+0000 7f9918d1a640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 0x7f9900067f50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99040011a0 tx=0x7f9904009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:23.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:23.916+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9908051a60 con 0x7f9914149240 2023-11-06T03:14:24.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.060+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99140cb110 con 0x7f9914149240 2023-11-06T03:14:24.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.060+0000 7f99117fa640 1 -- 172.21.15.157:0/2045979539 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f9908055080 con 0x7f9914149240 2023-11-06T03:14:24.059 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:24.060 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:13:41.413681+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:13:38.971687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-06T03:13:39.371284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:24.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 msgr2=0x7f9900067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 0x7f9900067f50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99040011a0 tx=0x7f9904009040 comp rx=0 tx=0).stop 2023-11-06T03:14:24.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 msgr2=0x7f99141599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f990800f580 tx=0x7f990800fa40 comp rx=0 tx=0).stop 2023-11-06T03:14:24.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 shutdown_connections 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9900065a90 0x7f9900067f50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99141e4ca0 0x7f9914159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9914149240 0x7f99141599d0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 --2- 172.21.15.157:0/2045979539 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98fc0052b0 0x7f991413e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 >> 172.21.15.157:0/2045979539 conn(0x7f9914065930 msgr2=0x7f99140af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:24.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 shutdown_connections 2023-11-06T03:14:24.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.064+0000 7f991af7e640 1 -- 172.21.15.157:0/2045979539 wait complete. 2023-11-06T03:14:24.077 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:14:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 Processor -- start 2023-11-06T03:14:24.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 -- start start 2023-11-06T03:14:24.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f082015e5e0 0x7f08200b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082015f580 con 0x7f08201551e0 2023-11-06T03:14:24.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082015efc0 con 0x7f082015ea90 2023-11-06T03:14:24.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0826709640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200704b0 con 0x7f082015e5e0 2023-11-06T03:14:24.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f08201551e0 0x7f082006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44456/0 (socket says 172.21.15.157:44456) 2023-11-06T03:14:24.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081ffff640 1 -- 172.21.15.157:0/4077543635 learned_addr learned my addr 172.21.15.157:0/4077543635 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:24.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081f7fe640 1 --2- 172.21.15.157:0/4077543635 >> v2:172.21.15.195:3300/0 conn(0x7f082015e5e0 0x7f08200b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525609877 0 0) 0x7f082015f580 con 0x7f08201551e0 2023-11-06T03:14:24.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0808003530 con 0x7f08201551e0 2023-11-06T03:14:24.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746791458 0 0) 0x7f0808003530 con 0x7f08201551e0 2023-11-06T03:14:24.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 >> v2:172.21.15.195:3300/0 conn(0x7f082015e5e0 msgr2=0x7f08200b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:24.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 --2- 172.21.15.157:0/4077543635 >> v2:172.21.15.195:3300/0 conn(0x7f082015e5e0 0x7f08200b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 >> v1:172.21.15.195:6789/0 conn(0x7f082015ea90 legacy=0x7f082015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082015f2a0 con 0x7f08201551e0 2023-11-06T03:14:24.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0810001680 con 0x7f08201551e0 2023-11-06T03:14:24.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 >> v1:172.21.15.157:6789/0 conn(0x7f08201551e0 legacy=0x7f082006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0808003980 0x7f0808003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08080047d0 0x7f0808004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f080800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200704b0 con 0x7f0808005190 2023-11-06T03:14:24.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080800c130 con 0x7f0808003980 2023-11-06T03:14:24.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080800c2f0 con 0x7f08080047d0 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f080800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081ffff640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0808003980 0x7f0808003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f081f7fe640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08080047d0 0x7f0808004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 -- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08080047d0 msgr2=0x7f0808004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08080047d0 0x7f0808004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 -- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0808003980 msgr2=0x7f0808003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:24.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0808003980 0x7f0808003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.200+0000 7f0824ca6640 1 -- 172.21.15.157:0/4077543635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808003530 con 0x7f0808005190 2023-11-06T03:14:24.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0824ca6640 1 -- 172.21.15.157:0/4077543635 reap_dead start 2023-11-06T03:14:24.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0824ca6640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f080800b990 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0818001e70 tx=0x7f081801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa46d46a972fb93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818003b80 con 0x7f0808005190 2023-11-06T03:14:24.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818003d00 con 0x7f0808005190 2023-11-06T03:14:24.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f081effd640 1 -- 172.21.15.157:0/4077543635 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818005310 con 0x7f0808005190 2023-11-06T03:14:24.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 msgr2=0x7f080800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f080800b990 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0818001e70 tx=0x7f081801c760 comp rx=0 tx=0).stop 2023-11-06T03:14:24.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- 172.21.15.157:0/4077543635 shutdown_connections 2023-11-06T03:14:24.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 --2- 172.21.15.157:0/4077543635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f080800b990 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- 172.21.15.157:0/4077543635 >> 172.21.15.157:0/4077543635 conn(0x7f0820065930 msgr2=0x7f0820065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:24.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- 172.21.15.157:0/4077543635 shutdown_connections 2023-11-06T03:14:24.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- 172.21.15.157:0/4077543635 wait complete. 2023-11-06T03:14:24.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 Processor -- start 2023-11-06T03:14:24.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.204+0000 7f0826709640 1 -- start start 2023-11-06T03:14:24.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 0x7f08200cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08200ce140 0x7f08200ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820135d00 con 0x7f0808005190 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082014d050 con 0x7f08200cdce0 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082014d330 con 0x7f08200ce140 2023-11-06T03:14:24.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42390/0 (socket says 172.21.15.157:42390) 2023-11-06T03:14:24.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 -- 172.21.15.157:0/1764514308 learned_addr learned my addr 172.21.15.157:0/1764514308 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:24.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081f7fe640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 0x7f08200cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0824ca6640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08200ce140 0x7f08200ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 -- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08200ce140 msgr2=0x7f08200ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08200ce140 0x7f08200ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 -- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 msgr2=0x7f08200cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 0x7f08200cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 -- 172.21.15.157:0/1764514308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820135fa0 con 0x7f0808005190 2023-11-06T03:14:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081f7fe640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 0x7f08200cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:24.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081ffff640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0810011840 tx=0x7f0810011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081000fb60 con 0x7f0808005190 2023-11-06T03:14:24.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081000fd20 con 0x7f0808005190 2023-11-06T03:14:24.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820136280 con 0x7f0808005190 2023-11-06T03:14:24.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810015650 con 0x7f0808005190 2023-11-06T03:14:24.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08200cd4a0 con 0x7f0808005190 2023-11-06T03:14:24.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0820136470 con 0x7f0808005190 2023-11-06T03:14:24.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f08100157b0 con 0x7f0808005190 2023-11-06T03:14:24.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 0x7f080c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08100865e0 con 0x7f0808005190 2023-11-06T03:14:24.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081f7fe640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 0x7f080c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.208+0000 7f081f7fe640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 0x7f080c067f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f08201e4540 tx=0x7f0814008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.212+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0810053e40 con 0x7f0808005190 2023-11-06T03:14:24.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.396+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0820072750 con 0x7f0808005190 2023-11-06T03:14:24.393 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:14:24.396+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:14:24.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.408+0000 7f081cff9640 1 -- 172.21.15.157:0/1764514308 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08100014c0 con 0x7f0808005190 2023-11-06T03:14:24.407 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:14:24.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 msgr2=0x7f080c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 0x7f080c067f20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f08201e4540 tx=0x7f0814008040 comp rx=0 tx=0).stop 2023-11-06T03:14:24.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 msgr2=0x7f08201efcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0810011840 tx=0x7f0810011d00 comp rx=0 tx=0).stop 2023-11-06T03:14:24.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 shutdown_connections 2023-11-06T03:14:24.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f080c065a60 0x7f080c067f20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08200ce140 0x7f08200ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08200cdce0 0x7f08200cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 --2- 172.21.15.157:0/1764514308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808005190 0x7f08201efcc0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 >> 172.21.15.157:0/1764514308 conn(0x7f0820065930 msgr2=0x7f082015de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:24.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 shutdown_connections 2023-11-06T03:14:24.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.412+0000 7f0826709640 1 -- 172.21.15.157:0/1764514308 wait complete. 2023-11-06T03:14:24.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:14:24.458 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:14:24.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 Processor -- start 2023-11-06T03:14:24.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 -- start start 2023-11-06T03:14:24.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe19015ce60 0x7fe1901614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe1900cca40 0x7fe1900cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe190161020 con 0x7fe1900cd9a0 2023-11-06T03:14:24.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe190072020 con 0x7fe1900cca40 2023-11-06T03:14:24.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.552+0000 7fe195ae0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe190072300 con 0x7fe19015ce60 2023-11-06T03:14:24.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe1900cd9a0 0x7fe1900caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44478/0 (socket says 172.21.15.157:44478) 2023-11-06T03:14:24.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 --2- 172.21.15.157:0/593012750 >> v2:172.21.15.157:3300/0 conn(0x7fe1900cca40 0x7fe1900cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18ffff640 1 -- 172.21.15.157:0/593012750 learned_addr learned my addr 172.21.15.157:0/593012750 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:24.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18f7fe640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe19015ce60 0x7fe1901614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 -- 172.21.15.157:0/593012750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe19015ce60 msgr2=0x7fe1901614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:24.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe19015ce60 0x7fe1901614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 -- 172.21.15.157:0/593012750 >> v1:172.21.15.157:6789/0 conn(0x7fe1900cd9a0 legacy=0x7fe1900caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 -- 172.21.15.157:0/593012750 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1901609d0 con 0x7fe1900cca40 2023-11-06T03:14:24.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18e7fc640 1 -- 172.21.15.157:0/593012750 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910741348 0 0) 0x7fe190161020 con 0x7fe1900cd9a0 2023-11-06T03:14:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18effd640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1900cca40 0x7fe1900cf9b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe1800097c0 tx=0x7fe18001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b3500e7fce3c27 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18e7fc640 1 -- 172.21.15.157:0/593012750 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800ad660 con 0x7fe1900cca40 2023-11-06T03:14:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18e7fc640 1 -- 172.21.15.157:0/593012750 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1800ad7c0 con 0x7fe1900cca40 2023-11-06T03:14:24.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe18e7fc640 1 -- 172.21.15.157:0/593012750 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800adb50 con 0x7fe1900cca40 2023-11-06T03:14:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 -- 172.21.15.157:0/593012750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1900cca40 msgr2=0x7fe1900cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1900cca40 0x7fe1900cf9b0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe1800097c0 tx=0x7fe18001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 -- 172.21.15.157:0/593012750 shutdown_connections 2023-11-06T03:14:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1900cca40 0x7fe1900cf9b0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 --2- 172.21.15.157:0/593012750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe19015ce60 0x7fe1901614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 -- 172.21.15.157:0/593012750 >> 172.21.15.157:0/593012750 conn(0x7fe19006d3d0 msgr2=0x7fe19006d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 -- 172.21.15.157:0/593012750 shutdown_connections 2023-11-06T03:14:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.556+0000 7fe195ae0640 1 -- 172.21.15.157:0/593012750 wait complete. 2023-11-06T03:14:24.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 Processor -- start 2023-11-06T03:14:24.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- start start 2023-11-06T03:14:24.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1900cca40 0x7fe190153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1900cd9a0 0x7fe1900c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190155690 con 0x7fe19015ce60 2023-11-06T03:14:24.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190154a10 con 0x7fe1900cd9a0 2023-11-06T03:14:24.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe190154cf0 con 0x7fe1900cca40 2023-11-06T03:14:24.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42416/0 (socket says 172.21.15.157:42416) 2023-11-06T03:14:24.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 -- 172.21.15.157:0/1113045178 learned_addr learned my addr 172.21.15.157:0/1113045178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18effd640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1900cd9a0 0x7fe1900c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 -- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1900cca40 msgr2=0x7fe190153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1900cca40 0x7fe190153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 -- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1900cd9a0 msgr2=0x7fe1900c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:24.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1900cd9a0 0x7fe1900c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 -- 172.21.15.157:0/1113045178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe180009420 con 0x7fe19015ce60 2023-11-06T03:14:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe18ffff640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c3c0 tx=0x7fe17800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780074e0 con 0x7fe19015ce60 2023-11-06T03:14:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178007640 con 0x7fe19015ce60 2023-11-06T03:14:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190154fd0 con 0x7fe19015ce60 2023-11-06T03:14:24.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178004a30 con 0x7fe19015ce60 2023-11-06T03:14:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190156080 con 0x7fe19015ce60 2023-11-06T03:14:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1901557d0 con 0x7fe19015ce60 2023-11-06T03:14:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe178004b90 con 0x7fe19015ce60 2023-11-06T03:14:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 0x7fe170067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:24.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.560+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe178083060 con 0x7fe19015ce60 2023-11-06T03:14:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.564+0000 7fe18f7fe640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 0x7fe170067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.564+0000 7fe18f7fe640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 0x7fe170067f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe18800e190 tx=0x7fe18800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:24.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.568+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe1901557d0 con 0x7fe19015ce60 2023-11-06T03:14:24.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:24.740+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1901557d0 con 0x7fe19015ce60 2023-11-06T03:14:25.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.416+0000 7fe187fff640 1 -- 172.21.15.157:0/1113045178 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1901557d0 con 0x7fe19015ce60 2023-11-06T03:14:25.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 msgr2=0x7fe170067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 0x7fe170067f20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe18800e190 tx=0x7fe18800b040 comp rx=0 tx=0).stop 2023-11-06T03:14:25.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 msgr2=0x7fe190156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c3c0 tx=0x7fe17800c880 comp rx=0 tx=0).stop 2023-11-06T03:14:25.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 shutdown_connections 2023-11-06T03:14:25.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe170065a60 0x7fe170067f20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19015ce60 0x7fe190156e60 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1900cd9a0 0x7fe1900c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 --2- 172.21.15.157:0/1113045178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe1900cca40 0x7fe190153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 >> 172.21.15.157:0/1113045178 conn(0x7fe19006d3d0 msgr2=0x7fe1900c5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:25.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 shutdown_connections 2023-11-06T03:14:25.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.420+0000 7fe195ae0640 1 -- 172.21.15.157:0/1113045178 wait complete. 2023-11-06T03:14:25.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:14:25.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 Processor -- start 2023-11-06T03:14:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 -- start start 2023-11-06T03:14:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f096406c550 0x7f096415b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09640cb100 con 0x7f096406c170 2023-11-06T03:14:25.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09640ce0a0 con 0x7f096406c550 2023-11-06T03:14:25.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096c64d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096415bef0 con 0x7f0964155310 2023-11-06T03:14:25.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:54604/0 (socket says 172.21.15.157:54604) 2023-11-06T03:14:25.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 -- 172.21.15.157:0/351056269 learned_addr learned my addr 172.21.15.157:0/351056269 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:25.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f096abea640 1 --2- 172.21.15.157:0/351056269 >> v2:172.21.15.195:3300/0 conn(0x7f096406c550 0x7f096415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 -- 172.21.15.157:0/351056269 >> v2:172.21.15.195:3300/0 conn(0x7f096406c550 msgr2=0x7f096415b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:25.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 --2- 172.21.15.157:0/351056269 >> v2:172.21.15.195:3300/0 conn(0x7f096406c550 0x7f096415b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 -- 172.21.15.157:0/351056269 >> v1:172.21.15.195:6789/0 conn(0x7f096406c170 legacy=0x7f0964154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.620+0000 7f0969be8640 1 -- 172.21.15.157:0/351056269 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640cb350 con 0x7f0964155310 2023-11-06T03:14:25.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f0969be8640 1 --2- 172.21.15.157:0/351056269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b94a5f091e242080 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f09693e7640 1 -- 172.21.15.157:0/351056269 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540aa670 con 0x7f0964155310 2023-11-06T03:14:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f09693e7640 1 -- 172.21.15.157:0/351056269 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09540aa7d0 con 0x7f0964155310 2023-11-06T03:14:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f09693e7640 1 -- 172.21.15.157:0/351056269 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540aab60 con 0x7f0964155310 2023-11-06T03:14:25.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- 172.21.15.157:0/351056269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 msgr2=0x7f096415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 --2- 172.21.15.157:0/351056269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0954009870 tx=0x7f095401c510 comp rx=0 tx=0).stop 2023-11-06T03:14:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- 172.21.15.157:0/351056269 shutdown_connections 2023-11-06T03:14:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 --2- 172.21.15.157:0/351056269 >> v2:172.21.15.195:3300/0 conn(0x7f096406c550 0x7f096415b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 --2- 172.21.15.157:0/351056269 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096415b150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- 172.21.15.157:0/351056269 >> 172.21.15.157:0/351056269 conn(0x7f09640b43b0 msgr2=0x7f0964065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- 172.21.15.157:0/351056269 shutdown_connections 2023-11-06T03:14:25.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- 172.21.15.157:0/351056269 wait complete. 2023-11-06T03:14:25.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 Processor -- start 2023-11-06T03:14:25.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.624+0000 7f096c64d640 1 -- start start 2023-11-06T03:14:25.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096406c550 0x7f0964149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096414a950 con 0x7f096406c170 2023-11-06T03:14:25.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096414cd90 con 0x7f096406c550 2023-11-06T03:14:25.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096414cab0 con 0x7f0964155310 2023-11-06T03:14:25.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42424/0 (socket says 172.21.15.157:42424) 2023-11-06T03:14:25.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 -- 172.21.15.157:0/3581695092 learned_addr learned my addr 172.21.15.157:0/3581695092 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:25.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096abea640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f0969be8640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096406c550 0x7f0964149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 -- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 msgr2=0x7f096414bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096414bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 -- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096406c550 msgr2=0x7f0964149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:25.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096406c550 0x7f0964149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:25.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 -- 172.21.15.157:0/3581695092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540094d0 con 0x7f096406c170 2023-11-06T03:14:25.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096a3e9640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f096000c420 tx=0x7f096000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:25.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600052d0 con 0x7f096406c170 2023-11-06T03:14:25.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960005430 con 0x7f096406c170 2023-11-06T03:14:25.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09640cbfb0 con 0x7f096406c170 2023-11-06T03:14:25.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960007260 con 0x7f096406c170 2023-11-06T03:14:25.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0964149c10 con 0x7f096406c170 2023-11-06T03:14:25.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096404e050 con 0x7f096406c170 2023-11-06T03:14:25.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09600073c0 con 0x7f096406c170 2023-11-06T03:14:25.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 0x7f0948067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:25.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f09600826c0 con 0x7f096406c170 2023-11-06T03:14:25.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.628+0000 7f0969be8640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 0x7f0948067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:25.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.632+0000 7f0969be8640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 0x7f0948067f20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae000 tx=0x7f09540a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:25.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.632+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f096004ff50 con 0x7f096406c170 2023-11-06T03:14:25.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:25.788+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f096415e310 con 0x7f096406c170 2023-11-06T03:14:26.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.576+0000 7f095affd640 1 -- 172.21.15.157:0/3581695092 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0960053570 con 0x7f096406c170 2023-11-06T03:14:26.573 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:14:26.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.576+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 msgr2=0x7f0948067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.576+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 0x7f0948067f20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae000 tx=0x7f09540a7000 comp rx=0 tx=0).stop 2023-11-06T03:14:26.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.576+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 msgr2=0x7f096414a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.576+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f096000c420 tx=0x7f096000c8e0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 shutdown_connections 2023-11-06T03:14:26.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0948065a60 0x7f0948067f20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0964155310 0x7f096414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f096406c550 0x7f0964149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 --2- 172.21.15.157:0/3581695092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f096406c170 0x7f096414a410 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 >> 172.21.15.157:0/3581695092 conn(0x7f09640b43b0 msgr2=0x7f0964065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:26.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 shutdown_connections 2023-11-06T03:14:26.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.580+0000 7f096c64d640 1 -- 172.21.15.157:0/3581695092 wait complete. 2023-11-06T03:14:26.599 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:14:26.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.728+0000 7f73e8911640 1 Processor -- start 2023-11-06T03:14:26.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.728+0000 7f73e8911640 1 -- start start 2023-11-06T03:14:26.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e8911640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73e40b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e8911640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f73e406b2f0 0x7f73e415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e8911640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e4070ee0 con 0x7f73e415d690 2023-11-06T03:14:26.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e8911640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e415b060 con 0x7f73e406b2f0 2023-11-06T03:14:26.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e8911640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e406b780 con 0x7f73e4159c80 2023-11-06T03:14:26.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e2d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f73e415d690 0x7f73e40b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44480/0 (socket says 172.21.15.157:44480) 2023-11-06T03:14:26.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e2d9d640 1 -- 172.21.15.157:0/312184016 learned_addr learned my addr 172.21.15.157:0/312184016 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:26.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e1d9b640 1 --2- 172.21.15.157:0/312184016 >> v2:172.21.15.195:3300/0 conn(0x7f73e406b2f0 0x7f73e415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73e40b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804263823 0 0) 0x7f73e4070ee0 con 0x7f73e415d690 2023-11-06T03:14:26.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73cc003530 con 0x7f73e415d690 2023-11-06T03:14:26.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571942935 0 0) 0x7f73cc003530 con 0x7f73e415d690 2023-11-06T03:14:26.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 msgr2=0x7f73e40b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:26.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73e40b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 >> v2:172.21.15.195:3300/0 conn(0x7f73e406b2f0 msgr2=0x7f73e415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:26.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 --2- 172.21.15.157:0/312184016 >> v2:172.21.15.195:3300/0 conn(0x7f73e406b2f0 0x7f73e415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40b2a90 con 0x7f73e415d690 2023-11-06T03:14:26.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f73dc001a00 con 0x7f73e415d690 2023-11-06T03:14:26.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 >> v1:172.21.15.157:6789/0 conn(0x7f73e415d690 legacy=0x7f73e40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73cc003980 0x7f73cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73cc0047d0 0x7f73cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415b060 con 0x7f73cc003980 2023-11-06T03:14:26.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e406b780 con 0x7f73cc0047d0 2023-11-06T03:14:26.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc00b990 con 0x7f73e4159c80 2023-11-06T03:14:26.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73cc003980 0x7f73cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e2d9d640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e1d9b640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73cc0047d0 0x7f73cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 -- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 msgr2=0x7f73cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 -- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73cc0047d0 msgr2=0x7f73cc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73cc0047d0 0x7f73cc004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e259c640 1 -- 172.21.15.157:0/312184016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc003530 con 0x7f73cc003980 2023-11-06T03:14:26.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.732+0000 7f73e2d9d640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4159c80 0x7f73cc005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e259c640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73cc003980 0x7f73cc003da0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009b10 tx=0x7f73d801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f6270fc550079 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d80a9430 con 0x7f73cc003980 2023-11-06T03:14:26.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d80a9590 con 0x7f73cc003980 2023-11-06T03:14:26.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e159a640 1 -- 172.21.15.157:0/312184016 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d80a9940 con 0x7f73cc003980 2023-11-06T03:14:26.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- 172.21.15.157:0/312184016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73cc003980 msgr2=0x7f73cc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 --2- 172.21.15.157:0/312184016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73cc003980 0x7f73cc003da0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009b10 tx=0x7f73d801ca20 comp rx=0 tx=0).stop 2023-11-06T03:14:26.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e2d9d640 1 -- 172.21.15.157:0/312184016 reap_dead start 2023-11-06T03:14:26.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- 172.21.15.157:0/312184016 shutdown_connections 2023-11-06T03:14:26.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- 172.21.15.157:0/312184016 >> 172.21.15.157:0/312184016 conn(0x7f73e40b43b0 msgr2=0x7f73e4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:26.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- 172.21.15.157:0/312184016 shutdown_connections 2023-11-06T03:14:26.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- 172.21.15.157:0/312184016 wait complete. 2023-11-06T03:14:26.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 Processor -- start 2023-11-06T03:14:26.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- start start 2023-11-06T03:14:26.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4143440 0x7f73e40a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 0x7f73e40b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40b1d30 con 0x7f73e4143060 2023-11-06T03:14:26.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40a9700 con 0x7f73e40b1370 2023-11-06T03:14:26.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e8911640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e41e6a40 con 0x7f73e4143440 2023-11-06T03:14:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42440/0 (socket says 172.21.15.157:42440) 2023-11-06T03:14:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.736+0000 7f73e259c640 1 -- 172.21.15.157:0/272575307 learned_addr learned my addr 172.21.15.157:0/272575307 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:26.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e2d9d640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 0x7f73e40b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e1d9b640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4143440 0x7f73e40a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 -- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4143440 msgr2=0x7f73e40a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4143440 0x7f73e40a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 -- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 msgr2=0x7f73e40b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 0x7f73e40b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 -- 172.21.15.157:0/272575307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e41e6d20 con 0x7f73e4143060 2023-11-06T03:14:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e2d9d640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 0x7f73e40b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e259c640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ab040 tx=0x7f73d80197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:26.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d80b6070 con 0x7f73e4143060 2023-11-06T03:14:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800f000 con 0x7f73e4143060 2023-11-06T03:14:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e41e7000 con 0x7f73e4143060 2023-11-06T03:14:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d80a9e80 con 0x7f73e4143060 2023-11-06T03:14:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e40a99e0 con 0x7f73e4143060 2023-11-06T03:14:26.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e41e71c0 con 0x7f73e4143060 2023-11-06T03:14:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73d80ba070 con 0x7f73e4143060 2023-11-06T03:14:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 0x7f73c4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f73d811cc90 con 0x7f73e4143060 2023-11-06T03:14:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e1d9b640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 0x7f73c4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:26.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.740+0000 7f73e1d9b640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 0x7f73c4067f50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f73e4157fc0 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:26.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.744+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f73d814e070 con 0x7f73e4143060 2023-11-06T03:14:26.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:26.896+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73e404e050 con 0x7f73e4143060 2023-11-06T03:14:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.584+0000 7f73d2ffd640 1 -- 172.21.15.157:0/272575307 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7f73d80ee290 con 0x7f73e4143060 2023-11-06T03:14:27.583 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:14:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 msgr2=0x7f73c4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 0x7f73c4067f50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f73e4157fc0 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2023-11-06T03:14:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 msgr2=0x7f73e41e62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ab040 tx=0x7f73d80197c0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 shutdown_connections 2023-11-06T03:14:27.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f73c4065a90 0x7f73c4067f50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73e40b1370 0x7f73e40b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73e4143440 0x7f73e40a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 --2- 172.21.15.157:0/272575307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73e4143060 0x7f73e41e62d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.588+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 >> 172.21.15.157:0/272575307 conn(0x7f73e40b43b0 msgr2=0x7f73e40ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:27.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.592+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 shutdown_connections 2023-11-06T03:14:27.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.592+0000 7f73e8911640 1 -- 172.21.15.157:0/272575307 wait complete. 2023-11-06T03:14:27.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:14:27.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 Processor -- start 2023-11-06T03:14:27.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 -- start start 2023-11-06T03:14:27.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 0x7f64980afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f649806da00 0x7f64980b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980af110 con 0x7f6498070330 2023-11-06T03:14:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980afa20 con 0x7f649806da00 2023-11-06T03:14:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649e545640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980b0380 con 0x7f6498159930 2023-11-06T03:14:27.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649cae2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6498070330 0x7f64980cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44496/0 (socket says 172.21.15.157:44496) 2023-11-06T03:14:27.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f649cae2640 1 -- 172.21.15.157:0/2008774300 learned_addr learned my addr 172.21.15.157:0/2008774300 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:27.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f6497fff640 1 --2- 172.21.15.157:0/2008774300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 0x7f64980afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f648f7fe640 1 --2- 172.21.15.157:0/2008774300 >> v2:172.21.15.195:3300/0 conn(0x7f649806da00 0x7f64980b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f6497fff640 1 -- 172.21.15.157:0/2008774300 >> v2:172.21.15.195:3300/0 conn(0x7f649806da00 msgr2=0x7f64980b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:27.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f6497fff640 1 --2- 172.21.15.157:0/2008774300 >> v2:172.21.15.195:3300/0 conn(0x7f649806da00 0x7f64980b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f6497fff640 1 -- 172.21.15.157:0/2008774300 >> v1:172.21.15.157:6789/0 conn(0x7f6498070330 legacy=0x7f64980cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.780+0000 7f6497fff640 1 -- 172.21.15.157:0/2008774300 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649806df50 con 0x7f6498159930 2023-11-06T03:14:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f6497fff640 1 --2- 172.21.15.157:0/2008774300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 0x7f64980afe40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f648800bac0 tx=0x7f6488024e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f22dab375fdc609 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f64977fe640 1 -- 172.21.15.157:0/2008774300 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880adcf0 con 0x7f6498159930 2023-11-06T03:14:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f64977fe640 1 -- 172.21.15.157:0/2008774300 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64880ade50 con 0x7f6498159930 2023-11-06T03:14:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f64977fe640 1 -- 172.21.15.157:0/2008774300 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880b5660 con 0x7f6498159930 2023-11-06T03:14:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f649e545640 1 -- 172.21.15.157:0/2008774300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 msgr2=0x7f64980afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.788+0000 7f649e545640 1 --2- 172.21.15.157:0/2008774300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 0x7f64980afe40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f648800bac0 tx=0x7f6488024e10 comp rx=0 tx=0).stop 2023-11-06T03:14:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.792+0000 7f649e545640 1 -- 172.21.15.157:0/2008774300 shutdown_connections 2023-11-06T03:14:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.792+0000 7f649e545640 1 --2- 172.21.15.157:0/2008774300 >> v2:172.21.15.195:3300/0 conn(0x7f649806da00 0x7f64980b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.792+0000 7f649e545640 1 --2- 172.21.15.157:0/2008774300 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498159930 0x7f64980afe40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.792+0000 7f649e545640 1 -- 172.21.15.157:0/2008774300 >> 172.21.15.157:0/2008774300 conn(0x7f64980732f0 msgr2=0x7f6498069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:27.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.792+0000 7f649e545640 1 -- 172.21.15.157:0/2008774300 shutdown_connections 2023-11-06T03:14:27.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 -- 172.21.15.157:0/2008774300 wait complete. 2023-11-06T03:14:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 Processor -- start 2023-11-06T03:14:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 -- start start 2023-11-06T03:14:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649806da00 0x7f6498151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498070330 0x7f6498152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980cc0a0 con 0x7f6498159930 2023-11-06T03:14:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498151620 con 0x7f649806da00 2023-11-06T03:14:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649e545640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649813b340 con 0x7f6498070330 2023-11-06T03:14:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42450/0 (socket says 172.21.15.157:42450) 2023-11-06T03:14:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 -- 172.21.15.157:0/3987756438 learned_addr learned my addr 172.21.15.157:0/3987756438 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f6497fff640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649806da00 0x7f6498151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f648f7fe640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498070330 0x7f6498152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 -- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498070330 msgr2=0x7f6498152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498070330 0x7f6498152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 -- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649806da00 msgr2=0x7f6498151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649806da00 0x7f6498151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 -- 172.21.15.157:0/3987756438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648800b6c0 con 0x7f6498159930 2023-11-06T03:14:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.796+0000 7f649cae2640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f649000cbf0 tx=0x7f649000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000ad40 con 0x7f6498159930 2023-11-06T03:14:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490012400 con 0x7f6498159930 2023-11-06T03:14:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6498070cd0 con 0x7f6498159930 2023-11-06T03:14:27.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000aea0 con 0x7f6498159930 2023-11-06T03:14:27.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649813acf0 con 0x7f6498159930 2023-11-06T03:14:27.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649813b4f0 con 0x7f6498159930 2023-11-06T03:14:27.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6490007070 con 0x7f6498159930 2023-11-06T03:14:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 0x7f6478067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6490082e00 con 0x7f6498159930 2023-11-06T03:14:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f6497fff640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 0x7f6478067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.800+0000 7f6497fff640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 0x7f6478067f20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f648800bac0 tx=0x7f6488004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.808+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f649813b4f0 con 0x7f6498159930 2023-11-06T03:14:27.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.968+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f649813b4f0 con 0x7f6498159930 2023-11-06T03:14:27.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.976+0000 7f64957fa640 1 -- 172.21.15.157:0/3987756438 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f649813b4f0 con 0x7f6498159930 2023-11-06T03:14:27.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 msgr2=0x7f6478067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 0x7f6478067f20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f648800bac0 tx=0x7f6488004740 comp rx=0 tx=0).stop 2023-11-06T03:14:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 msgr2=0x7f649813ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f649000cbf0 tx=0x7f649000a430 comp rx=0 tx=0).stop 2023-11-06T03:14:27.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 shutdown_connections 2023-11-06T03:14:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6478065a60 0x7f6478067f20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6498159930 0x7f649813ce90 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6498070330 0x7f6498152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 --2- 172.21.15.157:0/3987756438 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f649806da00 0x7f6498151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:27.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 >> 172.21.15.157:0/3987756438 conn(0x7f64980732f0 msgr2=0x7f64980b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 shutdown_connections 2023-11-06T03:14:27.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:27.980+0000 7f649e545640 1 -- 172.21.15.157:0/3987756438 wait complete. 2023-11-06T03:14:27.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2023-11-06T03:14:28.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.108+0000 7f3e0b57d640 1 Processor -- start 2023-11-06T03:14:28.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.108+0000 7f3e0b57d640 1 -- start start 2023-11-06T03:14:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e0b57d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3e040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e0b57d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3e0406d000 0x7f3e04159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e0b57d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0415e440 con 0x7f3e041599c0 2023-11-06T03:14:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e0b57d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0406c240 con 0x7f3e0406d000 2023-11-06T03:14:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e0b57d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0406c520 con 0x7f3e040b2af0 2023-11-06T03:14:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3e041599c0 0x7f3e040b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44512/0 (socket says 172.21.15.157:44512) 2023-11-06T03:14:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 -- 172.21.15.157:0/1117966018 learned_addr learned my addr 172.21.15.157:0/1117966018 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:28.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/1117966018 >> v2:172.21.15.195:3300/0 conn(0x7f3e0406d000 0x7f3e04159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e08b18640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3e040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503724024 0 0) 0x7f3e0415e440 con 0x7f3e041599c0 2023-11-06T03:14:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df0003530 con 0x7f3e041599c0 2023-11-06T03:14:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001576072 0 0) 0x7f3df0003530 con 0x7f3e041599c0 2023-11-06T03:14:28.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 msgr2=0x7f3e040ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3e040ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 >> v2:172.21.15.195:3300/0 conn(0x7f3e0406d000 msgr2=0x7f3e04159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 --2- 172.21.15.157:0/1117966018 >> v2:172.21.15.195:3300/0 conn(0x7f3e0406d000 0x7f3e04159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0406c800 con 0x7f3e041599c0 2023-11-06T03:14:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3df4001680 con 0x7f3e041599c0 2023-11-06T03:14:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 >> v1:172.21.15.157:6789/0 conn(0x7f3e041599c0 legacy=0x7f3e040b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df0003980 0x7f3df0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3df00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0004d10 0x7f3df0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0406c240 con 0x7f3df0003980 2023-11-06T03:14:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0406c520 con 0x7f3df0004d10 2023-11-06T03:14:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df000b990 con 0x7f3e040b2af0 2023-11-06T03:14:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df0003980 0x7f3df0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e08b18640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3df00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0004d10 0x7f3df0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 -- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 msgr2=0x7f3df00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3df00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 -- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0004d10 msgr2=0x7f3df0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:28.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0004d10 0x7f3df0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09319640 1 -- 172.21.15.157:0/1117966018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0003530 con 0x7f3df0003980 2023-11-06T03:14:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0004d10 0x7f3df0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:28.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.112+0000 7f3e08b18640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e040b2af0 0x7f3df00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:28.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09319640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df0003980 0x7f3df0003da0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c210 tx=0x7f3df401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0c737ed0b5dab5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40a5910 con 0x7f3df0003980 2023-11-06T03:14:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df40a5a70 con 0x7f3df0003980 2023-11-06T03:14:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3dfbfff640 1 -- 172.21.15.157:0/1117966018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40a5e20 con 0x7f3df0003980 2023-11-06T03:14:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- 172.21.15.157:0/1117966018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df0003980 msgr2=0x7f3df0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/1117966018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df0003980 0x7f3df0003da0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c210 tx=0x7f3df401cc10 comp rx=0 tx=0).stop 2023-11-06T03:14:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 -- 172.21.15.157:0/1117966018 reap_dead start 2023-11-06T03:14:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- 172.21.15.157:0/1117966018 shutdown_connections 2023-11-06T03:14:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- 172.21.15.157:0/1117966018 >> 172.21.15.157:0/1117966018 conn(0x7f3e04065930 msgr2=0x7f3e04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- 172.21.15.157:0/1117966018 shutdown_connections 2023-11-06T03:14:28.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- 172.21.15.157:0/1117966018 wait complete. 2023-11-06T03:14:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 Processor -- start 2023-11-06T03:14:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- start start 2023-11-06T03:14:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0003980 0x7f3e040cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e04151620 0x7f3e040cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04148770 con 0x7f3e04147db0 2023-11-06T03:14:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04151b90 con 0x7f3df0003980 2023-11-06T03:14:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e0b57d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04151e70 con 0x7f3e04151620 2023-11-06T03:14:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42464/0 (socket says 172.21.15.157:42464) 2023-11-06T03:14:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 -- 172.21.15.157:0/979170316 learned_addr learned my addr 172.21.15.157:0/979170316 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09319640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0003980 0x7f3e040cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e08b18640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e04151620 0x7f3e040cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 -- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e04151620 msgr2=0x7f3e040cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e04151620 0x7f3e040cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 -- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0003980 msgr2=0x7f3e040cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0003980 0x7f3e040cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 -- 172.21.15.157:0/979170316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04152150 con 0x7f3e04147db0 2023-11-06T03:14:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3e09b1a640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3e0406f250 tx=0x7f3dfc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.116+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc017400 con 0x7f3e04147db0 2023-11-06T03:14:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc011040 con 0x7f3e04147db0 2023-11-06T03:14:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e04152430 con 0x7f3e04147db0 2023-11-06T03:14:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc017780 con 0x7f3e04147db0 2023-11-06T03:14:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e040cd410 con 0x7f3e04147db0 2023-11-06T03:14:28.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04072750 con 0x7f3e04147db0 2023-11-06T03:14:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3dfc0203b0 con 0x7f3e04147db0 2023-11-06T03:14:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3df9ffb640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 0x7f3de8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3dfc083100 con 0x7f3e04147db0 2023-11-06T03:14:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3e09319640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 0x7f3de8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.120+0000 7f3e09319640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 0x7f3de8067f50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3df40aa000 tx=0x7f3df4000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.124+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3dfc050c60 con 0x7f3e04147db0 2023-11-06T03:14:28.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.316+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f3e04160200 con 0x7f3e04147db0 2023-11-06T03:14:28.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.584+0000 7f3df9ffb640 1 -- 172.21.15.157:0/979170316 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v483) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3dfc054280 con 0x7f3e04147db0 2023-11-06T03:14:28.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 msgr2=0x7f3de8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 0x7f3de8067f50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3df40aa000 tx=0x7f3df4000ff0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 msgr2=0x7f3e04148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3e0406f250 tx=0x7f3dfc00f4a0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 shutdown_connections 2023-11-06T03:14:28.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3de8065a90 0x7f3de8067f50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e04147db0 0x7f3e04148230 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e04151620 0x7f3e040cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 --2- 172.21.15.157:0/979170316 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3df0003980 0x7f3e040cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 >> 172.21.15.157:0/979170316 conn(0x7f3e04065930 msgr2=0x7f3e040ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:28.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 shutdown_connections 2023-11-06T03:14:28.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.588+0000 7f3e0b57d640 1 -- 172.21.15.157:0/979170316 wait complete. 2023-11-06T03:14:28.607 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2023-11-06T03:14:28.608 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2023-11-06T03:14:28.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2023-11-06T03:14:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 Processor -- start 2023-11-06T03:14:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 -- start start 2023-11-06T03:14:28.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee415e5e0 0x7f9ee40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ee415ea60 0x7f9ee40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee415f580 con 0x7f9ee41551e0 2023-11-06T03:14:28.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40704b0 con 0x7f9ee415ea60 2023-11-06T03:14:28.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee90c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee415efc0 con 0x7f9ee415e5e0 2023-11-06T03:14:28.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee2d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9ee41551e0 0x7f9ee406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49076/0 (socket says 172.21.15.157:49076) 2023-11-06T03:14:28.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee2d9d640 1 -- 172.21.15.157:0/2822359223 learned_addr learned my addr 172.21.15.157:0/2822359223 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:28.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee359e640 1 --2- 172.21.15.157:0/2822359223 >> v2:172.21.15.157:3300/0 conn(0x7f9ee415ea60 0x7f9ee40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee1d9b640 1 -- 172.21.15.157:0/2822359223 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93080855 0 0) 0x7f9ee415f580 con 0x7f9ee41551e0 2023-11-06T03:14:28.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee1d9b640 1 -- 172.21.15.157:0/2822359223 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ecc003530 con 0x7f9ee41551e0 2023-11-06T03:14:28.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee259c640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee415e5e0 0x7f9ee40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee359e640 1 -- 172.21.15.157:0/2822359223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee415e5e0 msgr2=0x7f9ee40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee359e640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee415e5e0 0x7f9ee40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee359e640 1 -- 172.21.15.157:0/2822359223 >> v1:172.21.15.157:6789/0 conn(0x7f9ee41551e0 legacy=0x7f9ee406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.736+0000 7f9ee359e640 1 -- 172.21.15.157:0/2822359223 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee415f2a0 con 0x7f9ee415ea60 2023-11-06T03:14:28.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40b3950 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009d90 tx=0x7f9edc01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691286f6da6cdac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee1d9b640 1 -- 172.21.15.157:0/2822359223 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0a5950 con 0x7f9ee415ea60 2023-11-06T03:14:28.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee1d9b640 1 -- 172.21.15.157:0/2822359223 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc0a5ab0 con 0x7f9ee415ea60 2023-11-06T03:14:28.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee1d9b640 1 -- 172.21.15.157:0/2822359223 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0a5e20 con 0x7f9ee415ea60 2023-11-06T03:14:28.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- 172.21.15.157:0/2822359223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 msgr2=0x7f9ee40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40b3950 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009d90 tx=0x7f9edc01c890 comp rx=0 tx=0).stop 2023-11-06T03:14:28.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- 172.21.15.157:0/2822359223 shutdown_connections 2023-11-06T03:14:28.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40b3950 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/2822359223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee415e5e0 0x7f9ee40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- 172.21.15.157:0/2822359223 >> 172.21.15.157:0/2822359223 conn(0x7f9ee4065930 msgr2=0x7f9ee4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:28.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- 172.21.15.157:0/2822359223 shutdown_connections 2023-11-06T03:14:28.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- 172.21.15.157:0/2822359223 wait complete. 2023-11-06T03:14:28.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 Processor -- start 2023-11-06T03:14:28.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- start start 2023-11-06T03:14:28.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee41551e0 0x7f9ee414c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee415e5e0 0x7f9ee4149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4149ab0 con 0x7f9ee415ea60 2023-11-06T03:14:28.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee414f350 con 0x7f9ee415e5e0 2023-11-06T03:14:28.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee90c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40ce9d0 con 0x7f9ee41551e0 2023-11-06T03:14:28.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50820/0 (socket says 172.21.15.157:50820) 2023-11-06T03:14:28.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 -- 172.21.15.157:0/1526772401 learned_addr learned my addr 172.21.15.157:0/1526772401 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:28.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee2d9d640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee41551e0 0x7f9ee414c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee259c640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee415e5e0 0x7f9ee4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 -- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee41551e0 msgr2=0x7f9ee414c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee41551e0 0x7f9ee414c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 -- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee415e5e0 msgr2=0x7f9ee4149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:28.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee415e5e0 0x7f9ee4149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:28.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.740+0000 7f9ee359e640 1 -- 172.21.15.157:0/1526772401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc009a70 con 0x7f9ee415ea60 2023-11-06T03:14:28.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee359e640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f9edc01a4c0 tx=0x7f9edc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc007000 con 0x7f9ee415ea60 2023-11-06T03:14:28.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc0aede0 con 0x7f9ee415ea60 2023-11-06T03:14:28.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee414ce60 con 0x7f9ee415ea60 2023-11-06T03:14:28.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0b7630 con 0x7f9ee415ea60 2023-11-06T03:14:28.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee414efe0 con 0x7f9ee415ea60 2023-11-06T03:14:28.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4072750 con 0x7f9ee415ea60 2023-11-06T03:14:28.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9edc0b4050 con 0x7f9ee415ea60 2023-11-06T03:14:28.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 0x7f9ed0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:28.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9edc11e1a0 con 0x7f9ee415ea60 2023-11-06T03:14:28.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee2d9d640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 0x7f9ed0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:28.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.744+0000 7f9ee2d9d640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 0x7f9ed0067f20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005890 tx=0x7f9ed4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:28.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.748+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9edc0ebd90 con 0x7f9ee415ea60 2023-11-06T03:14:28.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:28.964+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f9ee4071730 con 0x7f9ee415ea60 2023-11-06T03:14:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.596+0000 7f9ecb7fe640 1 -- 172.21.15.157:0/1526772401 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v484) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9edc0b20f0 con 0x7f9ee415ea60 2023-11-06T03:14:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 msgr2=0x7f9ed0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 0x7f9ed0067f20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005890 tx=0x7f9ed4009040 comp rx=0 tx=0).stop 2023-11-06T03:14:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 msgr2=0x7f9ee40ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f9edc01a4c0 tx=0x7f9edc019630 comp rx=0 tx=0).stop 2023-11-06T03:14:29.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 shutdown_connections 2023-11-06T03:14:29.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9ed0065a60 0x7f9ed0067f20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ee415ea60 0x7f9ee40ce490 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ee415e5e0 0x7f9ee4149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 --2- 172.21.15.157:0/1526772401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ee41551e0 0x7f9ee414c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 >> 172.21.15.157:0/1526772401 conn(0x7f9ee4065930 msgr2=0x7f9ee41543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:29.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 shutdown_connections 2023-11-06T03:14:29.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.600+0000 7f9ee90c3640 1 -- 172.21.15.157:0/1526772401 wait complete. 2023-11-06T03:14:29.619 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:14:29.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 Processor -- start 2023-11-06T03:14:29.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 -- start start 2023-11-06T03:14:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b0806d000 0x7f9b080b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 0x7f9b080b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0815e440 con 0x7f9b0806d000 2023-11-06T03:14:29.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0806c240 con 0x7f9b0815dea0 2023-11-06T03:14:29.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b102eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0806c520 con 0x7f9b081599c0 2023-11-06T03:14:29.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b0806d000 0x7f9b080b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9b0806d000 0x7f9b080b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50836/0 (socket says 172.21.15.157:50836) 2023-11-06T03:14:29.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 -- 172.21.15.157:0/1572758926 learned_addr learned my addr 172.21.15.157:0/1572758926 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:29.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0e888640 1 --2- 172.21.15.157:0/1572758926 >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 0x7f9b080b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0e087640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 -- 172.21.15.157:0/1572758926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 msgr2=0x7f9b080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:29.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 -- 172.21.15.157:0/1572758926 >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 msgr2=0x7f9b080b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:29.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 --2- 172.21.15.157:0/1572758926 >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 0x7f9b080b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0d886640 1 -- 172.21.15.157:0/1572758926 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0806c800 con 0x7f9b0806d000 2023-11-06T03:14:29.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.752+0000 7f9b0e888640 1 --2- 172.21.15.157:0/1572758926 >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 0x7f9b080b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:29.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0d886640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080b23e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9b0806bbf0 tx=0x7f9af801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b5e41a5a5067a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:29.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0d085640 1 -- 172.21.15.157:0/1572758926 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80aa610 con 0x7f9b0806d000 2023-11-06T03:14:29.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0d085640 1 -- 172.21.15.157:0/1572758926 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80a6040 con 0x7f9b0806d000 2023-11-06T03:14:29.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0d085640 1 -- 172.21.15.157:0/1572758926 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80aa990 con 0x7f9b0806d000 2023-11-06T03:14:29.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/1572758926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 msgr2=0x7f9b080b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080b23e0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f9b0806bbf0 tx=0x7f9af801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/1572758926 shutdown_connections 2023-11-06T03:14:29.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- 172.21.15.157:0/1572758926 >> v2:172.21.15.195:3300/0 conn(0x7f9b0815dea0 0x7f9b080b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080b23e0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- 172.21.15.157:0/1572758926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/1572758926 >> 172.21.15.157:0/1572758926 conn(0x7f9b08065930 msgr2=0x7f9b08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:29.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/1572758926 shutdown_connections 2023-11-06T03:14:29.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/1572758926 wait complete. 2023-11-06T03:14:29.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 Processor -- start 2023-11-06T03:14:29.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- start start 2023-11-06T03:14:29.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0815dea0 0x7f9b080b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815b0f0 con 0x7f9b0806d000 2023-11-06T03:14:29.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815a2b0 con 0x7f9b0815dea0 2023-11-06T03:14:29.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815b580 con 0x7f9b081599c0 2023-11-06T03:14:29.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50842/0 (socket says 172.21.15.157:50842) 2023-11-06T03:14:29.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 -- 172.21.15.157:0/413337424 learned_addr learned my addr 172.21.15.157:0/413337424 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:29.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0d886640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 -- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 msgr2=0x7f9b080b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e888640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0815dea0 0x7f9b080b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 -- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0815dea0 msgr2=0x7f9b080b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0815dea0 0x7f9b080b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 -- 172.21.15.157:0/413337424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80094d0 con 0x7f9b0806d000 2023-11-06T03:14:29.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b0e087640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c7f0 tx=0x7f9b0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:29.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04007820 con 0x7f9b0806d000 2023-11-06T03:14:29.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04007980 con 0x7f9b0806d000 2023-11-06T03:14:29.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.756+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b080ce9c0 con 0x7f9b0806d000 2023-11-06T03:14:29.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04007d70 con 0x7f9b0806d000 2023-11-06T03:14:29.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0815a7c0 con 0x7f9b0806d000 2023-11-06T03:14:29.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b08072750 con 0x7f9b0806d000 2023-11-06T03:14:29.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9b040113f0 con 0x7f9b0806d000 2023-11-06T03:14:29.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9afeffd640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 0x7f9aec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:29.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9b04083330 con 0x7f9b0806d000 2023-11-06T03:14:29.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9b0d886640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 0x7f9aec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:29.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.760+0000 7f9b0d886640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 0x7f9aec067f20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ae000 tx=0x7f9af8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:29.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.764+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9b04050f20 con 0x7f9b0806d000 2023-11-06T03:14:29.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.920+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9b080cb110 con 0x7f9b0806d000 2023-11-06T03:14:29.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.924+0000 7f9afeffd640 1 -- 172.21.15.157:0/413337424 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b04054540 con 0x7f9b0806d000 2023-11-06T03:14:29.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 msgr2=0x7f9aec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 0x7f9aec067f20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ae000 tx=0x7f9af8008040 comp rx=0 tx=0).stop 2023-11-06T03:14:29.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 msgr2=0x7f9b080ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:29.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c7f0 tx=0x7f9b0400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 shutdown_connections 2023-11-06T03:14:29.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9aec065a60 0x7f9aec067f20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9b0815dea0 0x7f9b080b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9b081599c0 0x7f9b080b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 --2- 172.21.15.157:0/413337424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b0806d000 0x7f9b080ce480 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:29.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 >> 172.21.15.157:0/413337424 conn(0x7f9b08065930 msgr2=0x7f9b0814fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:29.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 shutdown_connections 2023-11-06T03:14:29.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:29.928+0000 7f9b102eb640 1 -- 172.21.15.157:0/413337424 wait complete. 2023-11-06T03:14:29.942 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:14:29.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:14:30.013 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:14:30.056 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:14:30.057 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:14:30.057 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:30.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:14:30.064 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:14:30.064 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:30.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:14:30.104 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:14:30.104 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:30.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:14:30.112 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:14:30.113 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:14:30.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:14:30.152 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:14:30.153 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 Processor -- start 2023-11-06T03:14:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 -- start start 2023-11-06T03:14:30.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fee80156a00 0x7fee800adc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156de0 0x7fee8006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8006bca0 con 0x7fee8015a410 2023-11-06T03:14:30.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8006bf60 con 0x7fee80156de0 2023-11-06T03:14:30.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee87b8b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8006c240 con 0x7fee80156a00 2023-11-06T03:14:30.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee86128640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156de0 0x7fee8006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee86128640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee80156de0 0x7fee8006f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50848/0 (socket says 172.21.15.157:50848) 2023-11-06T03:14:30.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.292+0000 7fee86128640 1 -- 172.21.15.157:0/1220331108 learned_addr learned my addr 172.21.15.157:0/1220331108 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:30.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee85927640 1 --2- 172.21.15.157:0/1220331108 >> v2:172.21.15.195:3300/0 conn(0x7fee80156a00 0x7fee800adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee86128640 1 -- 172.21.15.157:0/1220331108 >> v2:172.21.15.195:3300/0 conn(0x7fee80156a00 msgr2=0x7fee800adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee86128640 1 --2- 172.21.15.157:0/1220331108 >> v2:172.21.15.195:3300/0 conn(0x7fee80156a00 0x7fee800adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee86128640 1 -- 172.21.15.157:0/1220331108 >> v1:172.21.15.195:6789/0 conn(0x7fee8015a410 legacy=0x7fee8006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee86128640 1 -- 172.21.15.157:0/1220331108 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8006c520 con 0x7fee80156de0 2023-11-06T03:14:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee86128640 1 --2- 172.21.15.157:0/1220331108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee8006f470 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fee800cfa20 tx=0x7fee7801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7966e808c0b82be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee84925640 1 -- 172.21.15.157:0/1220331108 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee780a5770 con 0x7fee80156de0 2023-11-06T03:14:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee84925640 1 -- 172.21.15.157:0/1220331108 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee780a58d0 con 0x7fee80156de0 2023-11-06T03:14:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee84925640 1 -- 172.21.15.157:0/1220331108 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee780a5c60 con 0x7fee80156de0 2023-11-06T03:14:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 -- 172.21.15.157:0/1220331108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 msgr2=0x7fee8006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 --2- 172.21.15.157:0/1220331108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee8006f470 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fee800cfa20 tx=0x7fee7801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 -- 172.21.15.157:0/1220331108 shutdown_connections 2023-11-06T03:14:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 --2- 172.21.15.157:0/1220331108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee8006f470 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 --2- 172.21.15.157:0/1220331108 >> v2:172.21.15.195:3300/0 conn(0x7fee80156a00 0x7fee800adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 -- 172.21.15.157:0/1220331108 >> 172.21.15.157:0/1220331108 conn(0x7fee80065930 msgr2=0x7fee80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 -- 172.21.15.157:0/1220331108 shutdown_connections 2023-11-06T03:14:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.296+0000 7fee87b8b640 1 -- 172.21.15.157:0/1220331108 wait complete. 2023-11-06T03:14:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 Processor -- start 2023-11-06T03:14:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- start start 2023-11-06T03:14:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee80156a00 0x7fee80150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee8015a410 0x7fee800ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee80151320 con 0x7fee80156de0 2023-11-06T03:14:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015af50 con 0x7fee80156a00 2023-11-06T03:14:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015b230 con 0x7fee8015a410 2023-11-06T03:14:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50862/0 (socket says 172.21.15.157:50862) 2023-11-06T03:14:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 -- 172.21.15.157:0/3149257673 learned_addr learned my addr 172.21.15.157:0/3149257673 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85927640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee80156a00 0x7fee80150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee86128640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee8015a410 0x7fee800ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 -- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee8015a410 msgr2=0x7fee800ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee8015a410 0x7fee800ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 -- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee80156a00 msgr2=0x7fee80150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee80156a00 0x7fee80150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 -- 172.21.15.157:0/3149257673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78009b30 con 0x7fee80156de0 2023-11-06T03:14:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee85126640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c3c0 tx=0x7fee7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700052d0 con 0x7fee80156de0 2023-11-06T03:14:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70005430 con 0x7fee80156de0 2023-11-06T03:14:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70003840 con 0x7fee80156de0 2023-11-06T03:14:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8015b510 con 0x7fee80156de0 2023-11-06T03:14:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8015bad0 con 0x7fee80156de0 2023-11-06T03:14:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee80072750 con 0x7fee80156de0 2023-11-06T03:14:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.300+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fee700039a0 con 0x7fee80156de0 2023-11-06T03:14:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.304+0000 7fee767fc640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 0x7fee64067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.304+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee70082460 con 0x7fee80156de0 2023-11-06T03:14:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.304+0000 7fee85927640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 0x7fee64067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.304+0000 7fee85927640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 0x7fee64067f50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009af0 tx=0x7fee7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.308+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fee70050050 con 0x7fee80156de0 2023-11-06T03:14:30.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.500+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee80163c40 con 0x7fee80156de0 2023-11-06T03:14:30.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.500+0000 7fee767fc640 1 -- 172.21.15.157:0/3149257673 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fee70017df0 con 0x7fee80156de0 2023-11-06T03:14:30.499 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:30.499 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2234375162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2234375162},{"type":"v1","addr":"172.21.15.157:6833","nonce":2234375162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3664985021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3664985021},{"type":"v1","addr":"172.21.15.195:6835","nonce":3664985021}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2993880130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2993880130},{"type":"v1","addr":"172.21.15.195:6837","nonce":2993880130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9309,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2659028193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2659028193},{"type":"v1","addr":"172.21.15.157:6835","nonce":2659028193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":93}],"filesystems":[]} 2023-11-06T03:14:30.500 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 93 2023-11-06T03:14:30.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 msgr2=0x7fee64067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 0x7fee64067f50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009af0 tx=0x7fee7c008040 comp rx=0 tx=0).stop 2023-11-06T03:14:30.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 msgr2=0x7fee80154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c3c0 tx=0x7fee7000c880 comp rx=0 tx=0).stop 2023-11-06T03:14:30.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 shutdown_connections 2023-11-06T03:14:30.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fee64065a90 0x7fee64067f50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee8015a410 0x7fee800ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee80156de0 0x7fee80154190 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 --2- 172.21.15.157:0/3149257673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee80156a00 0x7fee80150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 >> 172.21.15.157:0/3149257673 conn(0x7fee80065930 msgr2=0x7fee80166070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 shutdown_connections 2023-11-06T03:14:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.504+0000 7fee87b8b640 1 -- 172.21.15.157:0/3149257673 wait complete. 2023-11-06T03:14:30.519 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:14:30.520 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:14:30.520 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:14:30.520 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:14:30.521 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:14:30.521 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:14:30.521 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:14:30.521 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:14:30.522 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:14:30.522 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:14:30.522 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:14:30.522 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:14:30.523 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:14:30.523 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:14:30.523 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:14:30.524 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:14:30.532 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:14:30.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:14:30.533 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:14:30.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:14:30.537 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:14:30.537 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:14:30.541 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:14:30.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:14:30.576 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:14:30.577 DEBUG:teuthology.parallel:result is None 2023-11-06T03:14:30.578 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:14:30.578 DEBUG:teuthology.parallel:result is None 2023-11-06T03:14:30.584 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:14:30.584 DEBUG:teuthology.parallel:result is None 2023-11-06T03:14:30.585 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:14:30.586 DEBUG:teuthology.parallel:result is None 2023-11-06T03:14:30.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:14:30.642 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:14:30.648 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:14:30.655 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:14:30.656 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:14:30.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 Processor -- start 2023-11-06T03:14:30.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 -- start start 2023-11-06T03:14:30.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9e140b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9e140b1eb0 0x7f9e140c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1415b3d0 con 0x7f9e1415b860 2023-11-06T03:14:30.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e14155610 con 0x7f9e140b1eb0 2023-11-06T03:14:30.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e18dca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1415b0c0 con 0x7f9e140b1ad0 2023-11-06T03:14:30.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e12d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9e1415b860 0x7f9e140cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49106/0 (socket says 172.21.15.157:49106) 2023-11-06T03:14:30.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e12d9d640 1 -- 172.21.15.157:0/595871219 learned_addr learned my addr 172.21.15.157:0/595871219 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:30.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519377234 0 0) 0x7f9e1415b3d0 con 0x7f9e1415b860 2023-11-06T03:14:30.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/595871219 >> v2:172.21.15.195:3300/0 conn(0x7f9e140b1eb0 0x7f9e140c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dfc003530 con 0x7f9e1415b860 2023-11-06T03:14:30.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1259c640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9e140b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607353104 0 0) 0x7f9dfc003530 con 0x7f9e1415b860 2023-11-06T03:14:30.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 msgr2=0x7f9e140b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:30.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9e140b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 >> v2:172.21.15.195:3300/0 conn(0x7f9e140b1eb0 msgr2=0x7f9e140c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:30.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 --2- 172.21.15.157:0/595871219 >> v2:172.21.15.195:3300/0 conn(0x7f9e140b1eb0 0x7f9e140c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e140b50c0 con 0x7f9e1415b860 2023-11-06T03:14:30.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9e0c001a00 con 0x7f9e1415b860 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 >> v1:172.21.15.157:6789/0 conn(0x7f9e1415b860 legacy=0x7f9e140cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dfc003980 0x7f9dfc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0047d0 0x7f9dfc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9dfc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e14155610 con 0x7f9dfc0047d0 2023-11-06T03:14:30.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1415b0c0 con 0x7f9dfc003980 2023-11-06T03:14:30.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc00b990 con 0x7f9e140b1ad0 2023-11-06T03:14:30.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0047d0 0x7f9dfc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1259c640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dfc003980 0x7f9dfc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e12d9d640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9dfc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 -- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 msgr2=0x7f9dfc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:30.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e140b1ad0 0x7f9dfc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 -- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dfc003980 msgr2=0x7f9dfc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:30.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dfc003980 0x7f9dfc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e11d9b640 1 -- 172.21.15.157:0/595871219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc003530 con 0x7f9dfc0047d0 2023-11-06T03:14:30.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.708+0000 7f9e1259c640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9dfc003980 0x7f9dfc003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:14:30.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0047d0 0x7f9dfc004c50 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009ab0 tx=0x7f9e0801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c83310ea012e54 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a6480 con 0x7f9dfc0047d0 2023-11-06T03:14:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080a65e0 con 0x7f9dfc0047d0 2023-11-06T03:14:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e1159a640 1 -- 172.21.15.157:0/595871219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a6990 con 0x7f9dfc0047d0 2023-11-06T03:14:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/595871219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0047d0 msgr2=0x7f9dfc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/595871219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0047d0 0x7f9dfc004c50 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009ab0 tx=0x7f9e0801ca70 comp rx=0 tx=0).stop 2023-11-06T03:14:30.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e12d9d640 1 -- 172.21.15.157:0/595871219 reap_dead start 2023-11-06T03:14:30.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/595871219 shutdown_connections 2023-11-06T03:14:30.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/595871219 >> 172.21.15.157:0/595871219 conn(0x7f9e1406a170 msgr2=0x7f9e1406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:30.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/595871219 shutdown_connections 2023-11-06T03:14:30.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/595871219 wait complete. 2023-11-06T03:14:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 Processor -- start 2023-11-06T03:14:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- start start 2023-11-06T03:14:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dfc0047d0 0x7f9e140ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e140c5ed0 0x7f9e14146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e14151f10 con 0x7f9e141f0940 2023-11-06T03:14:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141521f0 con 0x7f9e140c5ed0 2023-11-06T03:14:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e18dca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141524d0 con 0x7f9dfc0047d0 2023-11-06T03:14:30.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e12d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.712+0000 7f9e12d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50930/0 (socket says 172.21.15.157:50930) 2023-11-06T03:14:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 -- 172.21.15.157:0/1949982941 learned_addr learned my addr 172.21.15.157:0/1949982941 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e11d9b640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e140c5ed0 0x7f9e14146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e1259c640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dfc0047d0 0x7f9e140ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 -- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dfc0047d0 msgr2=0x7f9e140ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dfc0047d0 0x7f9e140ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 -- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e140c5ed0 msgr2=0x7f9e14146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e140c5ed0 0x7f9e14146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:30.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 -- 172.21.15.157:0/1949982941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e141527b0 con 0x7f9e141f0940 2023-11-06T03:14:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e12d9d640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f9e1415bc40 tx=0x7f9e0c0115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c011e20 con 0x7f9e141f0940 2023-11-06T03:14:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c004b70 con 0x7f9e141f0940 2023-11-06T03:14:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e14152a90 con 0x7f9e141f0940 2023-11-06T03:14:30.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c006070 con 0x7f9e141f0940 2023-11-06T03:14:30.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e14151b30 con 0x7f9e141f0940 2023-11-06T03:14:30.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e14152c80 con 0x7f9e141f0940 2023-11-06T03:14:30.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9e0c0061d0 con 0x7f9e141f0940 2023-11-06T03:14:30.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 0x7f9df4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:30.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9e0c085ef0 con 0x7f9e141f0940 2023-11-06T03:14:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e1259c640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 0x7f9df4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.716+0000 7f9e1259c640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 0x7f9df4067f20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099c0 tx=0x7f9e04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:30.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.720+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9e0c053a50 con 0x7f9e141f0940 2023-11-06T03:14:30.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:30.880+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9e14072750 con 0x7f9e141f0940 2023-11-06T03:14:31.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.708+0000 7f9e02ffd640 1 -- 172.21.15.157:0/1949982941 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e0c01c020 con 0x7f9e141f0940 2023-11-06T03:14:31.705 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:14:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 msgr2=0x7f9df4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 0x7f9df4067f20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e040099c0 tx=0x7f9e04008040 comp rx=0 tx=0).stop 2023-11-06T03:14:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 msgr2=0x7f9e141f0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:31.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f9e1415bc40 tx=0x7f9e0c0115c0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 shutdown_connections 2023-11-06T03:14:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9df4065a60 0x7f9df4067f20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e141f0940 0x7f9e141f0df0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e140c5ed0 0x7f9e14146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 --2- 172.21.15.157:0/1949982941 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9dfc0047d0 0x7f9e140ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 >> 172.21.15.157:0/1949982941 conn(0x7f9e1406a170 msgr2=0x7f9e14065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 shutdown_connections 2023-11-06T03:14:31.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.712+0000 7f9e18dca640 1 -- 172.21.15.157:0/1949982941 wait complete. 2023-11-06T03:14:31.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:14:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 Processor -- start 2023-11-06T03:14:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 -- start start 2023-11-06T03:14:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc146370 0x7fdcdc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcdc0ce800 0x7fdcdc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdcdc06b6a0 0x7fdcdc06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0cf1a0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc06c3d0 con 0x7fdcdc06b6a0 2023-11-06T03:14:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdce1228640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06f5a0 con 0x7fdcdc146370 2023-11-06T03:14:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcdc0ce800 0x7fdcdc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdcdc0ce800 0x7fdcdc0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50946/0 (socket says 172.21.15.157:50946) 2023-11-06T03:14:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 -- 172.21.15.157:0/182663312 learned_addr learned my addr 172.21.15.157:0/182663312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 -- 172.21.15.157:0/182663312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc146370 msgr2=0x7fdcdc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 --2- 172.21.15.157:0/182663312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc146370 0x7fdcdc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 -- 172.21.15.157:0/182663312 >> v2:172.21.15.195:3300/0 conn(0x7fdcdc06b6a0 msgr2=0x7fdcdc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 --2- 172.21.15.157:0/182663312 >> v2:172.21.15.195:3300/0 conn(0x7fdcdc06b6a0 0x7fdcdc06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.860+0000 7fdcd259c640 1 -- 172.21.15.157:0/182663312 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc06f880 con 0x7fdcdc0ce800 2023-11-06T03:14:31.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- 172.21.15.157:0/182663312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc0cec60 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc06d360 tx=0x7fdcc401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef73a0cc62b8e85e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcda59c640 1 -- 172.21.15.157:0/182663312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40ab600 con 0x7fdcdc0ce800 2023-11-06T03:14:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcda59c640 1 -- 172.21.15.157:0/182663312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc40ab760 con 0x7fdcdc0ce800 2023-11-06T03:14:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcda59c640 1 -- 172.21.15.157:0/182663312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40abaf0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- 172.21.15.157:0/182663312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 msgr2=0x7fdcdc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:31.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- 172.21.15.157:0/182663312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc0cec60 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc06d360 tx=0x7fdcc401c440 comp rx=0 tx=0).stop 2023-11-06T03:14:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- 172.21.15.157:0/182663312 shutdown_connections 2023-11-06T03:14:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- 172.21.15.157:0/182663312 >> v2:172.21.15.195:3300/0 conn(0x7fdcdc06b6a0 0x7fdcdc06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- 172.21.15.157:0/182663312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc0cec60 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- 172.21.15.157:0/182663312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc146370 0x7fdcdc0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- 172.21.15.157:0/182663312 >> 172.21.15.157:0/182663312 conn(0x7fdcdc065930 msgr2=0x7fdcdc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:31.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- 172.21.15.157:0/182663312 shutdown_connections 2023-11-06T03:14:31.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- 172.21.15.157:0/182663312 wait complete. 2023-11-06T03:14:31.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 Processor -- start 2023-11-06T03:14:31.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- start start 2023-11-06T03:14:31.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 0x7fdcdc1f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 0x7fdcdc0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc167720 con 0x7fdcdc0ce800 2023-11-06T03:14:31.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc13c010 con 0x7fdcdc146370 2023-11-06T03:14:31.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdce1228640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15aa50 con 0x7fdcdc06b6a0 2023-11-06T03:14:31.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:31.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50962/0 (socket says 172.21.15.157:50962) 2023-11-06T03:14:31.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 -- 172.21.15.157:0/2589352537 learned_addr learned my addr 172.21.15.157:0/2589352537 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcdad9d640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 0x7fdcdc1f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 -- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 msgr2=0x7fdcdc1f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcdb59e640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 0x7fdcdc0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 0x7fdcdc1f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 -- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 msgr2=0x7fdcdc0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:31.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 0x7fdcdc0af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:31.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 -- 172.21.15.157:0/2589352537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40094d0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcdad9d640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 0x7fdcdc1f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:31.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcdb59e640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 0x7fdcdc0af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:31.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd259c640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40ad040 tx=0x7fdcc401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40b4800 con 0x7fdcdc0ce800 2023-11-06T03:14:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.864+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc40b4980 con 0x7fdcdc0ce800 2023-11-06T03:14:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc15ad30 con 0x7fdcdc0ce800 2023-11-06T03:14:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40b4d30 con 0x7fdcdc0ce800 2023-11-06T03:14:31.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc15b2f0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc15b4e0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdcc40c7440 con 0x7fdcdc0ce800 2023-11-06T03:14:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcd3fff640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 0x7fdcbc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdcc411dae0 con 0x7fdcdc0ce800 2023-11-06T03:14:31.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcdad9d640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 0x7fdcbc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.868+0000 7fdcdad9d640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 0x7fdcbc067f20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cd8f0 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:31.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:31.872+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdcc40eb720 con 0x7fdcdc0ce800 2023-11-06T03:14:32.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdcdc072750 con 0x7fdcdc0ce800 2023-11-06T03:14:32.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdcd3fff640 1 -- 172.21.15.157:0/2589352537 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fdcc40eed40 con 0x7fdcdc0ce800 2023-11-06T03:14:32.085 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:32.086 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:14:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:14:32.086 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:14:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:14:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:14:32.087 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:14:32.088 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:14:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:14:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.089 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:14:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:14:32.090 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:14:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:14:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.091 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.092 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:14:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:14:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.093 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:14:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:14:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.094 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.095 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:14:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:14:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.096 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.097 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:14:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:14:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.098 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:14:32.099 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.100 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.101 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.101 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:14:32.101 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:14:32.101 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.101 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.102 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.102 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.102 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.102 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.102 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.103 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:14:32.103 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:14:32.103 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.103 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.103 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:14:32.104 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:14:32.105 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.105 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.105 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.105 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.105 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.106 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.106 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.106 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:14:32.106 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:14:32.106 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.107 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.108 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:14:32.108 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:14:32.108 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.108 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.108 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:14:32.109 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:14:32.109 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.109 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.109 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.109 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:14:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:14:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:14:32.110 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:14:32.111 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:14:32.112 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:14:32.112 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:14:32.112 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:14:32.113 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.114 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:14:32.114 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:14:32.114 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:14:32.114 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:14:32.115 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.116 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:14:32.116 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:14:32.116 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:14:32.116 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.116 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.117 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.117 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.117 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:14:32.117 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:14:32.117 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:14:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:14:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.118 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:32.119 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:32.119 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:14:32.119 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:14:32.119 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:14:32.119 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:14:32.120 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:14:32.120 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:14:32.120 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.120 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:32.120 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:14:32.121 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:32.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 msgr2=0x7fdcbc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 0x7fdcbc067f20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cd8f0 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2023-11-06T03:14:32.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 msgr2=0x7fdcdc1671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40ad040 tx=0x7fdcc401ac80 comp rx=0 tx=0).stop 2023-11-06T03:14:32.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 shutdown_connections 2023-11-06T03:14:32.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdcbc065a60 0x7fdcbc067f20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdcdc146370 0x7fdcdc0af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcdc0ce800 0x7fdcdc1671e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 --2- 172.21.15.157:0/2589352537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdcdc06b6a0 0x7fdcdc1f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 >> 172.21.15.157:0/2589352537 conn(0x7fdcdc065930 msgr2=0x7fdcdc0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 shutdown_connections 2023-11-06T03:14:32.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.088+0000 7fdce1228640 1 -- 172.21.15.157:0/2589352537 wait complete. 2023-11-06T03:14:32.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:32.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.232+0000 7f8e3342b640 1 Processor -- start 2023-11-06T03:14:32.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 -- start start 2023-11-06T03:14:32.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8e2c0adae0 0x7f8e2c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e2c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c06c490 con 0x7f8e2c156a70 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c06b950 con 0x7f8e2c0adae0 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e3342b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c06fb80 con 0x7f8e2c0adf40 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8e2c156a70 0x7f8e2c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49118/0 (socket says 172.21.15.157:49118) 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 -- 172.21.15.157:0/1356177102 learned_addr learned my addr 172.21.15.157:0/1356177102 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:32.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601522140 0 0) 0x7f8e2c06c490 con 0x7f8e2c156a70 2023-11-06T03:14:32.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e319c8640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e2c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e18003530 con 0x7f8e2c156a70 2023-11-06T03:14:32.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e309c6640 1 --2- 172.21.15.157:0/1356177102 >> v2:172.21.15.195:3300/0 conn(0x7f8e2c0adae0 0x7f8e2c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203396559 0 0) 0x7f8e18003530 con 0x7f8e2c156a70 2023-11-06T03:14:32.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 msgr2=0x7f8e2c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e2c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 >> v2:172.21.15.195:3300/0 conn(0x7f8e2c0adae0 msgr2=0x7f8e2c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 --2- 172.21.15.157:0/1356177102 >> v2:172.21.15.195:3300/0 conn(0x7f8e2c0adae0 0x7f8e2c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c06bb60 con 0x7f8e2c156a70 2023-11-06T03:14:32.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8e28001680 con 0x7f8e2c156a70 2023-11-06T03:14:32.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 >> v1:172.21.15.157:6789/0 conn(0x7f8e2c156a70 legacy=0x7f8e2c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e18003980 0x7f8e18003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18004d10 0x7f8e18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c06b950 con 0x7f8e18003980 2023-11-06T03:14:32.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c06fb80 con 0x7f8e18004d10 2023-11-06T03:14:32.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1800b990 con 0x7f8e2c0adf40 2023-11-06T03:14:32.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e18003980 0x7f8e18003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 -- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 msgr2=0x7f8e180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e309c6640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e319c8640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18004d10 0x7f8e18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c0adf40 0x7f8e180047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 -- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18004d10 msgr2=0x7f8e18005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18004d10 0x7f8e18005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.236+0000 7f8e311c7640 1 -- 172.21.15.157:0/1356177102 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18003530 con 0x7f8e18003980 2023-11-06T03:14:32.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e18003980 0x7f8e18003da0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c2c0 tx=0x7f8e2801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36fe9bce2f8b101 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e280058e0 con 0x7f8e18003980 2023-11-06T03:14:32.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28005a40 con 0x7f8e18003980 2023-11-06T03:14:32.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e23fff640 1 -- 172.21.15.157:0/1356177102 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28005df0 con 0x7f8e18003980 2023-11-06T03:14:32.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1356177102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e18003980 msgr2=0x7f8e18003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1356177102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e18003980 0x7f8e18003da0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c2c0 tx=0x7f8e2801cc90 comp rx=0 tx=0).stop 2023-11-06T03:14:32.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 -- 172.21.15.157:0/1356177102 reap_dead start 2023-11-06T03:14:32.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1356177102 shutdown_connections 2023-11-06T03:14:32.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1356177102 >> 172.21.15.157:0/1356177102 conn(0x7f8e2c065930 msgr2=0x7f8e2c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1356177102 shutdown_connections 2023-11-06T03:14:32.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1356177102 wait complete. 2023-11-06T03:14:32.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 Processor -- start 2023-11-06T03:14:32.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- start start 2023-11-06T03:14:32.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 0x7f8e2c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c14ac60 0x7f8e2c1381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c153ff0 con 0x7f8e2c138b00 2023-11-06T03:14:32.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1542d0 con 0x7f8e18003980 2023-11-06T03:14:32.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1545b0 con 0x7f8e2c14ac60 2023-11-06T03:14:32.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50988/0 (socket says 172.21.15.157:50988) 2023-11-06T03:14:32.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 -- 172.21.15.157:0/1666783633 learned_addr learned my addr 172.21.15.157:0/1666783633 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:32.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 0x7f8e2c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e309c6640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c14ac60 0x7f8e2c1381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 -- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c14ac60 msgr2=0x7f8e2c1381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c14ac60 0x7f8e2c1381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 -- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 msgr2=0x7f8e2c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 0x7f8e2c153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 -- 172.21.15.157:0/1666783633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c154890 con 0x7f8e2c138b00 2023-11-06T03:14:32.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 0x7f8e2c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:32.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e319c8640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0cf1d0 tx=0x7f8e24007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24003ba0 con 0x7f8e2c138b00 2023-11-06T03:14:32.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e24003d20 con 0x7f8e2c138b00 2023-11-06T03:14:32.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c154b70 con 0x7f8e2c138b00 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24013890 con 0x7f8e2c138b00 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.240+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c1386e0 con 0x7f8e2c138b00 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c154d60 con 0x7f8e2c138b00 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8e240139f0 con 0x7f8e2c138b00 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e21ffb640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 0x7f8e10067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e24085fc0 con 0x7f8e2c138b00 2023-11-06T03:14:32.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 0x7f8e10067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.244+0000 7f8e311c7640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 0x7f8e10067f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e280aa000 tx=0x7f8e28001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.248+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8e24053b70 con 0x7f8e2c138b00 2023-11-06T03:14:32.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.388+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e2c072750 con 0x7f8e2c138b00 2023-11-06T03:14:32.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e21ffb640 1 -- 172.21.15.157:0/1666783633 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+14229 (secure 0 0 0) 0x7f8e24057190 con 0x7f8e2c138b00 2023-11-06T03:14:32.389 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:32.389 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":133,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:14:31.694732+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:32.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 msgr2=0x7f8e10067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 0x7f8e10067f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e280aa000 tx=0x7f8e28001a10 comp rx=0 tx=0).stop 2023-11-06T03:14:32.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 msgr2=0x7f8e2c150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0cf1d0 tx=0x7f8e24007700 comp rx=0 tx=0).stop 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 shutdown_connections 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8e10065a90 0x7f8e10067f50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e2c138b00 0x7f8e2c150ca0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e2c14ac60 0x7f8e2c1381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 --2- 172.21.15.157:0/1666783633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e18003980 0x7f8e2c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 >> 172.21.15.157:0/1666783633 conn(0x7f8e2c065930 msgr2=0x7f8e2c15ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 shutdown_connections 2023-11-06T03:14:32.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.392+0000 7f8e3342b640 1 -- 172.21.15.157:0/1666783633 wait complete. 2023-11-06T03:14:32.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:14:32.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:14:32.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 Processor -- start 2023-11-06T03:14:32.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 -- start start 2023-11-06T03:14:32.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf81599c0 0x7f5bf80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bf806d000 0x7f5bf80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf815e440 con 0x7f5bf815dea0 2023-11-06T03:14:32.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf815adb0 con 0x7f5bf806d000 2023-11-06T03:14:32.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bfd408640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf806c240 con 0x7f5bf81599c0 2023-11-06T03:14:32.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bf67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bf806d000 0x7f5bf80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bf67fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5bf806d000 0x7f5bf80b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51000/0 (socket says 172.21.15.157:51000) 2023-11-06T03:14:32.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bf67fc640 1 -- 172.21.15.157:0/1771112931 learned_addr learned my addr 172.21.15.157:0/1771112931 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:32.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bf6ffd640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf81599c0 0x7f5bf80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.528+0000 7f5bf67fc640 1 -- 172.21.15.157:0/1771112931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf81599c0 msgr2=0x7f5bf80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf67fc640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf81599c0 0x7f5bf80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf67fc640 1 -- 172.21.15.157:0/1771112931 >> v1:172.21.15.195:6789/0 conn(0x7f5bf815dea0 legacy=0x7f5bf8070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf67fc640 1 -- 172.21.15.157:0/1771112931 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf806c520 con 0x7f5bf806d000 2023-11-06T03:14:32.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf67fc640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf806d000 0x7f5bf80b23e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009870 tx=0x7f5bec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7e17ab16916412 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf5ffb640 1 -- 172.21.15.157:0/1771112931 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0ac600 con 0x7f5bf806d000 2023-11-06T03:14:32.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf5ffb640 1 -- 172.21.15.157:0/1771112931 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec0a8050 con 0x7f5bf806d000 2023-11-06T03:14:32.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bf5ffb640 1 -- 172.21.15.157:0/1771112931 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0ac980 con 0x7f5bf806d000 2023-11-06T03:14:32.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 -- 172.21.15.157:0/1771112931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf806d000 msgr2=0x7f5bf80b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf806d000 0x7f5bf80b23e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009870 tx=0x7f5bec01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 -- 172.21.15.157:0/1771112931 shutdown_connections 2023-11-06T03:14:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf806d000 0x7f5bf80b23e0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 --2- 172.21.15.157:0/1771112931 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf81599c0 0x7f5bf80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 -- 172.21.15.157:0/1771112931 >> 172.21.15.157:0/1771112931 conn(0x7f5bf8065930 msgr2=0x7f5bf8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 -- 172.21.15.157:0/1771112931 shutdown_connections 2023-11-06T03:14:32.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.532+0000 7f5bfd408640 1 -- 172.21.15.157:0/1771112931 wait complete. 2023-11-06T03:14:32.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 Processor -- start 2023-11-06T03:14:32.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- start start 2023-11-06T03:14:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf806d000 0x7f5bf8150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bf815dea0 0x7f5bf80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8151490 con 0x7f5bf81599c0 2023-11-06T03:14:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf813a930 con 0x7f5bf815dea0 2023-11-06T03:14:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf8150ac0 con 0x7f5bf806d000 2023-11-06T03:14:32.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51006/0 (socket says 172.21.15.157:51006) 2023-11-06T03:14:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 -- 172.21.15.157:0/871318746 learned_addr learned my addr 172.21.15.157:0/871318746 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf6ffd640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf806d000 0x7f5bf8150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 -- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf806d000 msgr2=0x7f5bf8150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf806d000 0x7f5bf8150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf77fe640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bf815dea0 0x7f5bf80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 -- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bf815dea0 msgr2=0x7f5bf80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bf815dea0 0x7f5bf80b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 -- 172.21.15.157:0/871318746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0094d0 con 0x7f5bf81599c0 2023-11-06T03:14:32.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf67fc640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5bec01cd70 tx=0x7f5bec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0b5830 con 0x7f5bf81599c0 2023-11-06T03:14:32.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec0b5990 con 0x7f5bf81599c0 2023-11-06T03:14:32.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf814feb0 con 0x7f5bf81599c0 2023-11-06T03:14:32.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0b5d20 con 0x7f5bf81599c0 2023-11-06T03:14:32.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf8150470 con 0x7f5bf81599c0 2023-11-06T03:14:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf8150c50 con 0x7f5bf81599c0 2023-11-06T03:14:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5bec0b3070 con 0x7f5bf81599c0 2023-11-06T03:14:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 0x7f5be4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5bec11d620 con 0x7f5bf81599c0 2023-11-06T03:14:32.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.536+0000 7f5bf6ffd640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 0x7f5be4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.540+0000 7f5bf6ffd640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 0x7f5be4067f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5be800b800 tx=0x7f5be8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.540+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5bec0eb1d0 con 0x7f5bf81599c0 2023-11-06T03:14:32.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.684+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bf8072750 con 0x7f5bf81599c0 2023-11-06T03:14:32.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bdf7fe640 1 -- 172.21.15.157:0/871318746 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bec0ee7f0 con 0x7f5bf81599c0 2023-11-06T03:14:32.684 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:14:32.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 msgr2=0x7f5be4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 0x7f5be4067f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5be800b800 tx=0x7f5be8008040 comp rx=0 tx=0).stop 2023-11-06T03:14:32.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 msgr2=0x7f5bf813a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f5bec01cd70 tx=0x7f5bec01ad10 comp rx=0 tx=0).stop 2023-11-06T03:14:32.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 shutdown_connections 2023-11-06T03:14:32.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5be4065a60 0x7f5be4067f20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bf815dea0 0x7f5bf80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf81599c0 0x7f5bf813a3f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 --2- 172.21.15.157:0/871318746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bf806d000 0x7f5bf8150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 >> 172.21.15.157:0/871318746 conn(0x7f5bf8065930 msgr2=0x7f5bf8065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 shutdown_connections 2023-11-06T03:14:32.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.688+0000 7f5bfd408640 1 -- 172.21.15.157:0/871318746 wait complete. 2023-11-06T03:14:32.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:14:32.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 Processor -- start 2023-11-06T03:14:32.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 -- start start 2023-11-06T03:14:32.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f09380b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 0x7f09380cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09380cf1a0 con 0x7f093806b6a0 2023-11-06T03:14:32.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093806c3d0 con 0x7f09380ce800 2023-11-06T03:14:32.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f093d82f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093806f5a0 con 0x7f0938146370 2023-11-06T03:14:32.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f09367fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 0x7f09380cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f0936ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f09380b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f09377fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f093806b6a0 0x7f093806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:51194/0 (socket says 172.21.15.157:51194) 2023-11-06T03:14:32.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f09377fe640 1 -- 172.21.15.157:0/140211072 learned_addr learned my addr 172.21.15.157:0/140211072 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:32.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f0936ffd640 1 -- 172.21.15.157:0/140211072 >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 msgr2=0x7f09380cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:32.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f0936ffd640 1 --2- 172.21.15.157:0/140211072 >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 0x7f09380cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f0936ffd640 1 -- 172.21.15.157:0/140211072 >> v1:172.21.15.195:6789/0 conn(0x7f093806b6a0 legacy=0x7f093806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f0936ffd640 1 -- 172.21.15.157:0/140211072 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093806f880 con 0x7f0938146370 2023-11-06T03:14:32.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.816+0000 7f09367fc640 1 --2- 172.21.15.157:0/140211072 >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 0x7f09380cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:14:32.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f0936ffd640 1 --2- 172.21.15.157:0/140211072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f09380b1a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09300097c0 tx=0x7f093001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6036a1f0c4aa8bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f0935ffb640 1 -- 172.21.15.157:0/140211072 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09300ab5f0 con 0x7f0938146370 2023-11-06T03:14:32.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f0935ffb640 1 -- 172.21.15.157:0/140211072 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300ab750 con 0x7f0938146370 2023-11-06T03:14:32.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f0935ffb640 1 -- 172.21.15.157:0/140211072 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09300abae0 con 0x7f0938146370 2023-11-06T03:14:32.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 -- 172.21.15.157:0/140211072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 msgr2=0x7f09380b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 --2- 172.21.15.157:0/140211072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f09380b1a10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09300097c0 tx=0x7f093001c3f0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 -- 172.21.15.157:0/140211072 shutdown_connections 2023-11-06T03:14:32.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 --2- 172.21.15.157:0/140211072 >> v2:172.21.15.195:3300/0 conn(0x7f09380ce800 0x7f09380cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 --2- 172.21.15.157:0/140211072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f09380b1a10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 -- 172.21.15.157:0/140211072 >> 172.21.15.157:0/140211072 conn(0x7f0938065930 msgr2=0x7f0938065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.820+0000 7f093d82f640 1 -- 172.21.15.157:0/140211072 shutdown_connections 2023-11-06T03:14:32.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- 172.21.15.157:0/140211072 wait complete. 2023-11-06T03:14:32.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 Processor -- start 2023-11-06T03:14:32.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- start start 2023-11-06T03:14:32.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 0x7f09381f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f093815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0938158d70 con 0x7f09380ce800 2023-11-06T03:14:32.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09381f8300 con 0x7f093806b6a0 2023-11-06T03:14:32.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380aff00 con 0x7f0938146370 2023-11-06T03:14:32.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51022/0 (socket says 172.21.15.157:51022) 2023-11-06T03:14:32.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 -- 172.21.15.157:0/1551901309 learned_addr learned my addr 172.21.15.157:0/1551901309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:32.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f0936ffd640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 0x7f09381f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09377fe640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f093815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 -- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 msgr2=0x7f093815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f093815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 -- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 msgr2=0x7f09381f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 0x7f09381f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 -- 172.21.15.157:0/1551901309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009420 con 0x7f09380ce800 2023-11-06T03:14:32.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09377fe640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f093815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:32.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f0936ffd640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 0x7f09381f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:32.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f09367fc640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f092c00e980 tx=0x7f092c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c00ca00 con 0x7f09380ce800 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c00cb60 con 0x7f09380ce800 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0930009420 con 0x7f09380ce800 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c013470 con 0x7f09380ce800 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.824+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09381426b0 con 0x7f09380ce800 2023-11-06T03:14:32.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0938072750 con 0x7f09380ce800 2023-11-06T03:14:32.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f092c0135d0 con 0x7f09380ce800 2023-11-06T03:14:32.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f091ffff640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 0x7f0924067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:32.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f092c0832b0 con 0x7f09380ce800 2023-11-06T03:14:32.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f0936ffd640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 0x7f0924067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:32.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.828+0000 7f0936ffd640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 0x7f0924067f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f09300af000 tx=0x7f0930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:32.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.832+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f092c050dd0 con 0x7f09380ce800 2023-11-06T03:14:32.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.972+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f093806cef0 con 0x7f09380ce800 2023-11-06T03:14:32.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f091ffff640 1 -- 172.21.15.157:0/1551901309 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7f092c0543f0 con 0x7f09380ce800 2023-11-06T03:14:32.972 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:14:32.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 msgr2=0x7f0924067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 0x7f0924067f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f09300af000 tx=0x7f0930008040 comp rx=0 tx=0).stop 2023-11-06T03:14:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 msgr2=0x7f093815a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f092c00e980 tx=0x7f092c00ee40 comp rx=0 tx=0).stop 2023-11-06T03:14:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 shutdown_connections 2023-11-06T03:14:32.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0924065a60 0x7f0924067f20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0938146370 0x7f093815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09380ce800 0x7f093815a470 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 --2- 172.21.15.157:0/1551901309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f093806b6a0 0x7f09381f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 >> 172.21.15.157:0/1551901309 conn(0x7f0938065930 msgr2=0x7f09380b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:32.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 shutdown_connections 2023-11-06T03:14:32.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:32.976+0000 7f093d82f640 1 -- 172.21.15.157:0/1551901309 wait complete. 2023-11-06T03:14:32.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:14:33.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 Processor -- start 2023-11-06T03:14:33.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 -- start start 2023-11-06T03:14:33.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 0x7fad340ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad34071820 0x7fad340b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340cd170 con 0x7fad340cc2a0 2023-11-06T03:14:33.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3415af30 con 0x7fad34071820 2023-11-06T03:14:33.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad3a12f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340b3e10 con 0x7fad3415ab50 2023-11-06T03:14:33.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad34071820 0x7fad340b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fad34071820 0x7fad340b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51030/0 (socket says 172.21.15.157:51030) 2023-11-06T03:14:33.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 -- 172.21.15.157:0/192265298 learned_addr learned my addr 172.21.15.157:0/192265298 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:33.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad337fe640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 0x7fad340ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 -- 172.21.15.157:0/192265298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 msgr2=0x7fad340ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:33.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 0x7fad340ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 -- 172.21.15.157:0/192265298 >> v1:172.21.15.195:6789/0 conn(0x7fad340cc2a0 legacy=0x7fad340cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:33.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad2affd640 1 -- 172.21.15.157:0/192265298 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340b40f0 con 0x7fad34071820 2023-11-06T03:14:33.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.100+0000 7fad337fe640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 0x7fad340ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad2affd640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34071820 0x7fad340b3730 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009870 tx=0x7fad1c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7124f4189d98d4e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad32ffd640 1 -- 172.21.15.157:0/192265298 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0aa5f0 con 0x7fad34071820 2023-11-06T03:14:33.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad32ffd640 1 -- 172.21.15.157:0/192265298 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c0aa750 con 0x7fad34071820 2023-11-06T03:14:33.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad32ffd640 1 -- 172.21.15.157:0/192265298 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0aaae0 con 0x7fad34071820 2023-11-06T03:14:33.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- 172.21.15.157:0/192265298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34071820 msgr2=0x7fad340b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34071820 0x7fad340b3730 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009870 tx=0x7fad1c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- 172.21.15.157:0/192265298 shutdown_connections 2023-11-06T03:14:33.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad34071820 0x7fad340b3730 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- 172.21.15.157:0/192265298 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad3415ab50 0x7fad340ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- 172.21.15.157:0/192265298 >> 172.21.15.157:0/192265298 conn(0x7fad34065d30 msgr2=0x7fad34066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:33.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- 172.21.15.157:0/192265298 shutdown_connections 2023-11-06T03:14:33.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- 172.21.15.157:0/192265298 wait complete. 2023-11-06T03:14:33.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 Processor -- start 2023-11-06T03:14:33.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- start start 2023-11-06T03:14:33.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 0x7fad340c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 0x7fad341541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340a9ad0 con 0x7fad3415ab50 2023-11-06T03:14:33.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad340a8ec0 con 0x7fad340cc2a0 2023-11-06T03:14:33.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad3a12f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340a91a0 con 0x7fad34071820 2023-11-06T03:14:33.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51040/0 (socket says 172.21.15.157:51040) 2023-11-06T03:14:33.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 -- 172.21.15.157:0/1854866597 learned_addr learned my addr 172.21.15.157:0/1854866597 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:33.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad2affd640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 0x7fad341541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad337fe640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 0x7fad340c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 -- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 msgr2=0x7fad340c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 0x7fad340c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 -- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 msgr2=0x7fad341541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 0x7fad341541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 -- 172.21.15.157:0/1854866597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0094d0 con 0x7fad3415ab50 2023-11-06T03:14:33.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad2affd640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 0x7fad341541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad337fe640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 0x7fad340c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.104+0000 7fad33fff640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c780 tx=0x7fad2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c004b70 con 0x7fad3415ab50 2023-11-06T03:14:33.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00ce90 con 0x7fad3415ab50 2023-11-06T03:14:33.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340a9480 con 0x7fad3415ab50 2023-11-06T03:14:33.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00a850 con 0x7fad3415ab50 2023-11-06T03:14:33.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad341559b0 con 0x7fad3415ab50 2023-11-06T03:14:33.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad340b1350 con 0x7fad3415ab50 2023-11-06T03:14:33.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fad2c00a9b0 con 0x7fad3415ab50 2023-11-06T03:14:33.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 0x7fad14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fad2c083160 con 0x7fad3415ab50 2023-11-06T03:14:33.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad337fe640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 0x7fad14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.108+0000 7fad337fe640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 0x7fad14067f20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fad24008890 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.112+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fad340b1350 con 0x7fad3415ab50 2023-11-06T03:14:33.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.268+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fad340b1350 con 0x7fad3415ab50 2023-11-06T03:14:33.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.268+0000 7fad30ff9640 1 -- 172.21.15.157:0/1854866597 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+126 (secure 0 0 0) 0x7fad340b1350 con 0x7fad3415ab50 2023-11-06T03:14:33.267 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:33.267 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:14:33.267 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:33.267 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:14:33.267 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:14:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:14:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:14:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:14:33.268 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:33.269 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:14:33.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 msgr2=0x7fad14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 0x7fad14067f20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fad24008890 tx=0x7fad24008040 comp rx=0 tx=0).stop 2023-11-06T03:14:33.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 msgr2=0x7fad340aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c780 tx=0x7fad2c00cc40 comp rx=0 tx=0).stop 2023-11-06T03:14:33.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 shutdown_connections 2023-11-06T03:14:33.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fad14065a60 0x7fad14067f20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad3415ab50 0x7fad340aa830 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad340cc2a0 0x7fad341541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 --2- 172.21.15.157:0/1854866597 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad34071820 0x7fad340c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 >> 172.21.15.157:0/1854866597 conn(0x7fad34065d30 msgr2=0x7fad340b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:33.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 shutdown_connections 2023-11-06T03:14:33.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.272+0000 7fad3a12f640 1 -- 172.21.15.157:0/1854866597 wait complete. 2023-11-06T03:14:33.285 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:14:33.285 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:14:33.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 Processor -- start 2023-11-06T03:14:33.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 -- start start 2023-11-06T03:14:33.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa04406c170 0x7fa044154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 0x7fa04415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0440cb100 con 0x7fa04406c170 2023-11-06T03:14:33.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04415bef0 con 0x7fa04415b150 2023-11-06T03:14:33.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa049609640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440cb350 con 0x7fa04406c550 2023-11-06T03:14:33.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa04406c170 0x7fa044154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa04406c170 0x7fa044154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51052/0 (socket says 172.21.15.157:51052) 2023-11-06T03:14:33.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 -- 172.21.15.157:0/1472697648 learned_addr learned my addr 172.21.15.157:0/1472697648 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:33.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa0437fe640 1 --2- 172.21.15.157:0/1472697648 >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 0x7fa04415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa03a7fc640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 -- 172.21.15.157:0/1472697648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 msgr2=0x7fa044155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:33.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 -- 172.21.15.157:0/1472697648 >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 msgr2=0x7fa04415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:33.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 --2- 172.21.15.157:0/1472697648 >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 0x7fa04415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa042ffd640 1 -- 172.21.15.157:0/1472697648 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440cb630 con 0x7fa04406c170 2023-11-06T03:14:33.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa03a7fc640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.408+0000 7fa0437fe640 1 --2- 172.21.15.157:0/1472697648 >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 0x7fa04415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa042ffd640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04406c170 0x7fa044154c00 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa0440cc8e0 tx=0x7fa03401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e49decf0e9ab75 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa0427fc640 1 -- 172.21.15.157:0/1472697648 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340ac410 con 0x7fa04406c170 2023-11-06T03:14:33.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa0427fc640 1 -- 172.21.15.157:0/1472697648 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0340ac570 con 0x7fa04406c170 2023-11-06T03:14:33.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa0427fc640 1 -- 172.21.15.157:0/1472697648 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340ac900 con 0x7fa04406c170 2023-11-06T03:14:33.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- 172.21.15.157:0/1472697648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04406c170 msgr2=0x7fa044154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04406c170 0x7fa044154c00 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa0440cc8e0 tx=0x7fa03401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- 172.21.15.157:0/1472697648 shutdown_connections 2023-11-06T03:14:33.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- 172.21.15.157:0/1472697648 >> v2:172.21.15.195:3300/0 conn(0x7fa04415b150 0x7fa04415b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- 172.21.15.157:0/1472697648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04406c170 0x7fa044154c00 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- 172.21.15.157:0/1472697648 >> 172.21.15.157:0/1472697648 conn(0x7fa0440b43b0 msgr2=0x7fa044065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:33.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- 172.21.15.157:0/1472697648 shutdown_connections 2023-11-06T03:14:33.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- 172.21.15.157:0/1472697648 wait complete. 2023-11-06T03:14:33.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 Processor -- start 2023-11-06T03:14:33.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- start start 2023-11-06T03:14:33.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 0x7fa04413df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440b2180 con 0x7fa04415b150 2023-11-06T03:14:33.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440b17e0 con 0x7fa04406c170 2023-11-06T03:14:33.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa049609640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440b1ac0 con 0x7fa04406c550 2023-11-06T03:14:33.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.412+0000 7fa0437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51066/0 (socket says 172.21.15.157:51066) 2023-11-06T03:14:33.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 -- 172.21.15.157:0/3320206568 learned_addr learned my addr 172.21.15.157:0/3320206568 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:33.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03a7fc640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa042ffd640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 0x7fa04413df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 -- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 msgr2=0x7fa044157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 -- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 msgr2=0x7fa04413df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 0x7fa04413df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 -- 172.21.15.157:0/3320206568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340094d0 con 0x7fa04415b150 2023-11-06T03:14:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa042ffd640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 0x7fa04413df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa0437fe640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00cd50 tx=0x7fa03c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00f680 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c00f800 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00fbb0 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0440b1da0 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04413e4c0 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0440b22c0 con 0x7fa04415b150 2023-11-06T03:14:33.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa03c00b040 con 0x7fa04415b150 2023-11-06T03:14:33.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 0x7fa024067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa03c0838e0 con 0x7fa04415b150 2023-11-06T03:14:33.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa042ffd640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 0x7fa024067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.416+0000 7fa042ffd640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 0x7fa024067f20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa0340ae000 tx=0x7fa034008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.420+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa03c051490 con 0x7fa04415b150 2023-11-06T03:14:33.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.564+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa04404e050 con 0x7fa04415b150 2023-11-06T03:14:33.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa03bfff640 1 -- 172.21.15.157:0/3320206568 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa03c01d0b0 con 0x7fa04415b150 2023-11-06T03:14:33.624 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:14:33.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 msgr2=0x7fa024067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 0x7fa024067f20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa0340ae000 tx=0x7fa034008040 comp rx=0 tx=0).stop 2023-11-06T03:14:33.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 msgr2=0x7fa0441581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00cd50 tx=0x7fa03c00a650 comp rx=0 tx=0).stop 2023-11-06T03:14:33.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 shutdown_connections 2023-11-06T03:14:33.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa024065a60 0x7fa024067f20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa04415b150 0x7fa0441581a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa04406c550 0x7fa044157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.628+0000 7fa049609640 1 --2- 172.21.15.157:0/3320206568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa04406c170 0x7fa04413df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.632+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 >> 172.21.15.157:0/3320206568 conn(0x7fa0440b43b0 msgr2=0x7fa0440ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:33.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.632+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 shutdown_connections 2023-11-06T03:14:33.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.632+0000 7fa049609640 1 -- 172.21.15.157:0/3320206568 wait complete. 2023-11-06T03:14:33.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:14:33.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 Processor -- start 2023-11-06T03:14:33.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 -- start start 2023-11-06T03:14:33.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3bcc0ce010 0x7f3bcc071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 0x7f3bcc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0722f0 con 0x7f3bcc0aff30 2023-11-06T03:14:33.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b0310 con 0x7f3bcc0ce010 2023-11-06T03:14:33.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd2ff0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b2e40 con 0x7f3bcc0b2490 2023-11-06T03:14:33.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd0d8c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3bcc0aff30 0x7f3bcc06af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49130/0 (socket says 172.21.15.157:49130) 2023-11-06T03:14:33.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd0d8c640 1 -- 172.21.15.157:0/1955932612 learned_addr learned my addr 172.21.15.157:0/1955932612 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:33.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/1955932612 >> v2:172.21.15.157:3300/0 conn(0x7f3bcc0ce010 0x7f3bcc071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd158d640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 0x7f3bcc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcbfff640 1 -- 172.21.15.157:0/1955932612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 msgr2=0x7f3bcc0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:33.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 0x7f3bcc0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcbfff640 1 -- 172.21.15.157:0/1955932612 >> v1:172.21.15.157:6789/0 conn(0x7f3bcc0aff30 legacy=0x7f3bcc06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcb7fe640 1 -- 172.21.15.157:0/1955932612 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538276525 0 0) 0x7f3bbc001120 con 0x7f3bcc0aff30 2023-11-06T03:14:33.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bcbfff640 1 -- 172.21.15.157:0/1955932612 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc06bda0 con 0x7f3bcc0ce010 2023-11-06T03:14:33.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.776+0000 7f3bd158d640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 0x7f3bcc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:33.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0ce010 0x7f3bcc071db0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097c0 tx=0x7f3bc001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f1a0ad3f12a6be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bcb7fe640 1 -- 172.21.15.157:0/1955932612 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00ac620 con 0x7f3bcc0ce010 2023-11-06T03:14:33.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bcb7fe640 1 -- 172.21.15.157:0/1955932612 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc00ac780 con 0x7f3bcc0ce010 2023-11-06T03:14:33.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bcb7fe640 1 -- 172.21.15.157:0/1955932612 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00acb10 con 0x7f3bcc0ce010 2023-11-06T03:14:33.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/1955932612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0ce010 msgr2=0x7f3bcc071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0ce010 0x7f3bcc071db0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097c0 tx=0x7f3bc001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/1955932612 shutdown_connections 2023-11-06T03:14:33.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0b2490 0x7f3bcc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/1955932612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0ce010 0x7f3bcc071db0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/1955932612 >> 172.21.15.157:0/1955932612 conn(0x7f3bcc0cd490 msgr2=0x7f3bcc0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:33.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/1955932612 shutdown_connections 2023-11-06T03:14:33.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/1955932612 wait complete. 2023-11-06T03:14:33.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.780+0000 7f3bd2ff0640 1 Processor -- start 2023-11-06T03:14:33.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- start start 2023-11-06T03:14:33.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0aff30 0x7f3bcc0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bcc14f9e0 0x7f3bcc154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15c470 con 0x7f3bcc0b2490 2023-11-06T03:14:33.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15bb40 con 0x7f3bcc14f9e0 2023-11-06T03:14:33.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc15be20 con 0x7f3bcc0aff30 2023-11-06T03:14:33.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51076/0 (socket says 172.21.15.157:51076) 2023-11-06T03:14:33.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 -- 172.21.15.157:0/2564253939 learned_addr learned my addr 172.21.15.157:0/2564253939 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:33.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd158d640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bcc14f9e0 0x7f3bcc154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 -- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0aff30 msgr2=0x7f3bcc0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd0d8c640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0aff30 0x7f3bcc0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0aff30 0x7f3bcc0ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 -- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bcc14f9e0 msgr2=0x7f3bcc154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:33.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bcc14f9e0 0x7f3bcc154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:33.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 -- 172.21.15.157:0/2564253939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc0009420 con 0x7f3bcc0b2490 2023-11-06T03:14:33.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bcbfff640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00af040 tx=0x7f3bc0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0004020 con 0x7f3bcc0b2490 2023-11-06T03:14:33.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0004180 con 0x7f3bcc0b2490 2023-11-06T03:14:33.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc15c100 con 0x7f3bcc0b2490 2023-11-06T03:14:33.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00044f0 con 0x7f3bcc0b2490 2023-11-06T03:14:33.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc15ef40 con 0x7f3bcc0b2490 2023-11-06T03:14:33.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc072750 con 0x7f3bcc0b2490 2023-11-06T03:14:33.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3bc00b3040 con 0x7f3bcc0b2490 2023-11-06T03:14:33.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.784+0000 7f3bc97fa640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 0x7f3bb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:33.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.788+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3bc011d1a0 con 0x7f3bcc0b2490 2023-11-06T03:14:33.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.788+0000 7f3bd0d8c640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 0x7f3bb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:33.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.788+0000 7f3bd0d8c640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 0x7f3bb8067f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000f70 tx=0x7f3bbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:33.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.792+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3bc00eac70 con 0x7f3bcc0b2490 2023-11-06T03:14:33.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:33.936+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3bcc150410 con 0x7f3bcc0b2490 2023-11-06T03:14:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.636+0000 7f3bc97fa640 1 -- 172.21.15.157:0/2564253939 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3bc00ee290 con 0x7f3bcc0b2490 2023-11-06T03:14:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:14:34.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.640+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 msgr2=0x7f3bb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.640+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 0x7f3bb8067f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000f70 tx=0x7f3bbc009040 comp rx=0 tx=0).stop 2023-11-06T03:14:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.640+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 msgr2=0x7f3bcc15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.640+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00af040 tx=0x7f3bc0019160 comp rx=0 tx=0).stop 2023-11-06T03:14:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 shutdown_connections 2023-11-06T03:14:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3bb8065a60 0x7f3bb8067f20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3bcc14f9e0 0x7f3bcc154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bcc0b2490 0x7f3bcc15e640 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 --2- 172.21.15.157:0/2564253939 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3bcc0aff30 0x7f3bcc0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 >> 172.21.15.157:0/2564253939 conn(0x7f3bcc0cd490 msgr2=0x7f3bcc0cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 shutdown_connections 2023-11-06T03:14:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.644+0000 7f3bd2ff0640 1 -- 172.21.15.157:0/2564253939 wait complete. 2023-11-06T03:14:34.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:14:34.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.796+0000 7fc5f02e5640 1 Processor -- start 2023-11-06T03:14:34.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.796+0000 7fc5f02e5640 1 -- start start 2023-11-06T03:14:34.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5f02e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5e80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5f02e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e815e440 con 0x7fc5e806d000 2023-11-06T03:14:34.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5f02e5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e806c240 con 0x7fc5e81599c0 2023-11-06T03:14:34.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5f02e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e806c520 con 0x7fc5e80b2af0 2023-11-06T03:14:34.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ee882640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc5e806d000 0x7fc5e8159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49132/0 (socket says 172.21.15.157:49132) 2023-11-06T03:14:34.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ee882640 1 -- 172.21.15.157:0/2681687842 learned_addr learned my addr 172.21.15.157:0/2681687842 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:34.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5e80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652535341 0 0) 0x7fc5e815e440 con 0x7fc5e806d000 2023-11-06T03:14:34.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d4003530 con 0x7fc5e806d000 2023-11-06T03:14:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 616250162 0 0) 0x7fc5d4003530 con 0x7fc5e806d000 2023-11-06T03:14:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 msgr2=0x7fc5e80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5e80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 >> v1:172.21.15.195:6789/0 conn(0x7fc5e81599c0 legacy=0x7fc5e80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e806c800 con 0x7fc5e806d000 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc5e0001a00 con 0x7fc5e806d000 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 >> v1:172.21.15.157:6789/0 conn(0x7fc5e806d000 legacy=0x7fc5e8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5d40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d4003f30 0x7fc5d40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4004d10 0x7fc5d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e806c520 con 0x7fc5d4003f30 2023-11-06T03:14:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d400ba70 con 0x7fc5d4004d10 2023-11-06T03:14:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d400bc30 con 0x7fc5e80b2af0 2023-11-06T03:14:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d4003f30 0x7fc5d40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ee081640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5d40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ee882640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4004d10 0x7fc5d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 -- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 msgr2=0x7fc5d40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:34.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e80b2af0 0x7fc5d40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 -- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4004d10 msgr2=0x7fc5d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:34.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4004d10 0x7fc5d4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ed880640 1 -- 172.21.15.157:0/2681687842 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4003530 con 0x7fc5d4003f30 2023-11-06T03:14:34.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.800+0000 7fc5ee882640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d4004d10 0x7fc5d4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:34.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d4003f30 0x7fc5d40047d0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800bd80 tx=0x7fc5d801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc60250146bd976a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:34.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80ae600 con 0x7fc5d4003f30 2023-11-06T03:14:34.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d80aa050 con 0x7fc5d4003f30 2023-11-06T03:14:34.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed07f640 1 -- 172.21.15.157:0/2681687842 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80ae9a0 con 0x7fc5d4003f30 2023-11-06T03:14:34.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/2681687842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d4003f30 msgr2=0x7fc5d40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/2681687842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d4003f30 0x7fc5d40047d0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800bd80 tx=0x7fc5d801f1c0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ee882640 1 -- 172.21.15.157:0/2681687842 reap_dead start 2023-11-06T03:14:34.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/2681687842 shutdown_connections 2023-11-06T03:14:34.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/2681687842 >> 172.21.15.157:0/2681687842 conn(0x7fc5e8065930 msgr2=0x7fc5e8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:34.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/2681687842 shutdown_connections 2023-11-06T03:14:34.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/2681687842 wait complete. 2023-11-06T03:14:34.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 Processor -- start 2023-11-06T03:14:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- start start 2023-11-06T03:14:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4003f30 0x7fc5e8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e80af4d0 0x7fc5e80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80afe90 con 0x7fc5e80b03b0 2023-11-06T03:14:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80b0b90 con 0x7fc5e80af4d0 2023-11-06T03:14:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e8158160 con 0x7fc5d4003f30 2023-11-06T03:14:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51084/0 (socket says 172.21.15.157:51084) 2023-11-06T03:14:34.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 -- 172.21.15.157:0/1270614762 learned_addr learned my addr 172.21.15.157:0/1270614762 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ee081640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4003f30 0x7fc5e8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ee882640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e80af4d0 0x7fc5e80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 -- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4003f30 msgr2=0x7fc5e8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4003f30 0x7fc5e8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 -- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e80af4d0 msgr2=0x7fc5e80af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e80af4d0 0x7fc5e80af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 -- 172.21.15.157:0/1270614762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8158440 con 0x7fc5e80b03b0 2023-11-06T03:14:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5ed880640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801eff0 tx=0x7fc5d4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b8410 con 0x7fc5e80b03b0 2023-11-06T03:14:34.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d80b8590 con 0x7fc5e80b03b0 2023-11-06T03:14:34.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e8158720 con 0x7fc5e80b03b0 2023-11-06T03:14:34.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80b8940 con 0x7fc5e80b03b0 2023-11-06T03:14:34.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.804+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e8150a00 con 0x7fc5e80b03b0 2023-11-06T03:14:34.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e815de20 con 0x7fc5e80b03b0 2023-11-06T03:14:34.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5d80b5070 con 0x7fc5e80b03b0 2023-11-06T03:14:34.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5deffd640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 0x7fc5cc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:34.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc5d811fa40 con 0x7fc5e80b03b0 2023-11-06T03:14:34.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5ee081640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 0x7fc5cc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:34.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.808+0000 7fc5ee081640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 0x7fc5cc067f50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005510 tx=0x7fc5e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:34.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:34.812+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc5d80ed3e0 con 0x7fc5e80b03b0 2023-11-06T03:14:34.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.000+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc5e8072750 con 0x7fc5e80b03b0 2023-11-06T03:14:35.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5deffd640 1 -- 172.21.15.157:0/1270614762 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc5d80b30c0 con 0x7fc5e80b03b0 2023-11-06T03:14:35.028 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 19 and data pool 20 2023-11-06T03:14:35.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 msgr2=0x7fc5cc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 0x7fc5cc067f50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005510 tx=0x7fc5e4009040 comp rx=0 tx=0).stop 2023-11-06T03:14:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 msgr2=0x7fc5e81504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc5d801eff0 tx=0x7fc5d4005800 comp rx=0 tx=0).stop 2023-11-06T03:14:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 shutdown_connections 2023-11-06T03:14:35.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5cc065a90 0x7fc5cc067f50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e80af4d0 0x7fc5e80af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5e80b03b0 0x7fc5e81504c0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 --2- 172.21.15.157:0/1270614762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d4003f30 0x7fc5e8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 >> 172.21.15.157:0/1270614762 conn(0x7fc5e8065930 msgr2=0x7fc5e815a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 shutdown_connections 2023-11-06T03:14:35.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.032+0000 7fc5f02e5640 1 -- 172.21.15.157:0/1270614762 wait complete. 2023-11-06T03:14:35.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:35.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 Processor -- start 2023-11-06T03:14:35.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 -- start start 2023-11-06T03:14:35.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c415e440 con 0x7f80c41599c0 2023-11-06T03:14:35.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c406c240 con 0x7f80c40b2af0 2023-11-06T03:14:35.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80caefd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c406c520 con 0x7f80c406d000 2023-11-06T03:14:35.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80c8c99640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f80c41599c0 0x7f80c40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49148/0 (socket says 172.21.15.157:49148) 2023-11-06T03:14:35.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80c8c99640 1 -- 172.21.15.157:0/1118368025 learned_addr learned my addr 172.21.15.157:0/1118368025 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:35.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80c949a640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755987443 0 0) 0x7f80c415e440 con 0x7f80c41599c0 2023-11-06T03:14:35.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80ac003530 con 0x7f80c41599c0 2023-11-06T03:14:35.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491307212 0 0) 0x7f80c406c240 con 0x7f80c40b2af0 2023-11-06T03:14:35.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c415e440 con 0x7f80c40b2af0 2023-11-06T03:14:35.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154975503 0 0) 0x7f80ac003530 con 0x7f80c41599c0 2023-11-06T03:14:35.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 msgr2=0x7f80c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80c40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 >> v1:172.21.15.195:6789/0 conn(0x7f80c40b2af0 legacy=0x7f80c4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c406c800 con 0x7f80c41599c0 2023-11-06T03:14:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f80c0001680 con 0x7f80c41599c0 2023-11-06T03:14:35.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.172+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 >> v1:172.21.15.157:6789/0 conn(0x7f80c41599c0 legacy=0x7f80c40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80ac003b40 0x7f80ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac004d10 0x7f80ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c406c520 con 0x7f80ac003b40 2023-11-06T03:14:35.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac00ba40 con 0x7f80ac004d10 2023-11-06T03:14:35.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac00bc00 con 0x7f80c406d000 2023-11-06T03:14:35.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80ac003b40 0x7f80ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bbfff640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c949a640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac004d10 0x7f80ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 -- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 msgr2=0x7f80ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c406d000 0x7f80ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 -- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac004d10 msgr2=0x7f80ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac004d10 0x7f80ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 -- 172.21.15.157:0/1118368025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac003530 con 0x7f80ac003b40 2023-11-06T03:14:35.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c8c99640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80ac003b40 0x7f80ac003f40 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f80c0007fb0 tx=0x7f80c001cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c74fe63182d7e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0005ae0 con 0x7f80ac003b40 2023-11-06T03:14:35.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c0005c60 con 0x7f80ac003b40 2023-11-06T03:14:35.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80bb7fe640 1 -- 172.21.15.157:0/1118368025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c00a57d0 con 0x7f80ac003b40 2023-11-06T03:14:35.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80caefd640 1 -- 172.21.15.157:0/1118368025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80ac003b40 msgr2=0x7f80ac003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80caefd640 1 --2- 172.21.15.157:0/1118368025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80ac003b40 0x7f80ac003f40 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f80c0007fb0 tx=0x7f80c001cad0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80c949a640 1 -- 172.21.15.157:0/1118368025 reap_dead start 2023-11-06T03:14:35.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80caefd640 1 -- 172.21.15.157:0/1118368025 shutdown_connections 2023-11-06T03:14:35.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.176+0000 7f80caefd640 1 -- 172.21.15.157:0/1118368025 >> 172.21.15.157:0/1118368025 conn(0x7f80c4065930 msgr2=0x7f80c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- 172.21.15.157:0/1118368025 shutdown_connections 2023-11-06T03:14:35.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- 172.21.15.157:0/1118368025 wait complete. 2023-11-06T03:14:35.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 Processor -- start 2023-11-06T03:14:35.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- start start 2023-11-06T03:14:35.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 0x7f80c415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c414fce0 0x7f80c41f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40cf180 con 0x7f80c414f880 2023-11-06T03:14:35.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c413bd60 con 0x7f80ac003b40 2023-11-06T03:14:35.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c413acc0 con 0x7f80c414fce0 2023-11-06T03:14:35.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51102/0 (socket says 172.21.15.157:51102) 2023-11-06T03:14:35.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 -- 172.21.15.157:0/754920670 learned_addr learned my addr 172.21.15.157:0/754920670 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:35.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80c949a640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c414fce0 0x7f80c41f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 -- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c414fce0 msgr2=0x7f80c41f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80c8c99640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 0x7f80c415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c414fce0 0x7f80c41f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 -- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 msgr2=0x7f80c415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 0x7f80c415b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 -- 172.21.15.157:0/754920670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c413afa0 con 0x7f80c414f880 2023-11-06T03:14:35.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80c8c99640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 0x7f80c415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:35.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80bbfff640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e7b0 tx=0x7f80b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400ee60 con 0x7f80c414f880 2023-11-06T03:14:35.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b40052e0 con 0x7f80c414f880 2023-11-06T03:14:35.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c413b280 con 0x7f80c414f880 2023-11-06T03:14:35.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400c900 con 0x7f80c414f880 2023-11-06T03:14:35.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c413b840 con 0x7f80c414f880 2023-11-06T03:14:35.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.180+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c4072750 con 0x7f80c414f880 2023-11-06T03:14:35.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.184+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80b400ca60 con 0x7f80c414f880 2023-11-06T03:14:35.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.184+0000 7f80b97fa640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 0x7f80b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.184+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80b4083220 con 0x7f80c414f880 2023-11-06T03:14:35.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.184+0000 7f80c8c99640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 0x7f80b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.184+0000 7f80c8c99640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 0x7f80b0067f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f80c00ab000 tx=0x7f80c00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.188+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80b4050a80 con 0x7f80c414f880 2023-11-06T03:14:35.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.344+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80c40cb110 con 0x7f80c414f880 2023-11-06T03:14:35.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80b97fa640 1 -- 172.21.15.157:0/754920670 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+17902 (secure 0 0 0) 0x7f80b40540a0 con 0x7f80c414f880 2023-11-06T03:14:35.345 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:35.345 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:14:35.004786+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:35.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 msgr2=0x7f80b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 0x7f80b0067f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f80c00ab000 tx=0x7f80c00a7070 comp rx=0 tx=0).stop 2023-11-06T03:14:35.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 msgr2=0x7f80c41f3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e7b0 tx=0x7f80b400ec70 comp rx=0 tx=0).stop 2023-11-06T03:14:35.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 shutdown_connections 2023-11-06T03:14:35.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f80b0065a90 0x7f80b0067f50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80c414fce0 0x7f80c41f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c414f880 0x7f80c41f3b20 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 --2- 172.21.15.157:0/754920670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80ac003b40 0x7f80c415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 >> 172.21.15.157:0/754920670 conn(0x7f80c4065930 msgr2=0x7f80c40b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 shutdown_connections 2023-11-06T03:14:35.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.348+0000 7f80caefd640 1 -- 172.21.15.157:0/754920670 wait complete. 2023-11-06T03:14:35.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:35.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 Processor -- start 2023-11-06T03:14:35.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 -- start start 2023-11-06T03:14:35.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 0x7f08100ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08100b2af0 0x7f0810159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081015e440 con 0x7f081006d000 2023-11-06T03:14:35.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081006c240 con 0x7f08100b2af0 2023-11-06T03:14:35.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f08175c3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081006c520 con 0x7f08101599c0 2023-11-06T03:14:35.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08100b2af0 0x7f0810159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f08100b2af0 0x7f0810159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51118/0 (socket says 172.21.15.157:51118) 2023-11-06T03:14:35.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 -- 172.21.15.157:0/4181443725 learned_addr learned my addr 172.21.15.157:0/4181443725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:35.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f081535f640 1 --2- 172.21.15.157:0/4181443725 >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 0x7f08100ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 -- 172.21.15.157:0/4181443725 >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 msgr2=0x7f08100ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 --2- 172.21.15.157:0/4181443725 >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 0x7f08100ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 -- 172.21.15.157:0/4181443725 >> v1:172.21.15.195:6789/0 conn(0x7f081006d000 legacy=0x7f08100b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f0815b60640 1 -- 172.21.15.157:0/4181443725 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081006c800 con 0x7f08100b2af0 2023-11-06T03:14:35.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.492+0000 7f081535f640 1 --2- 172.21.15.157:0/4181443725 >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 0x7f08100ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:35.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0815b60640 1 --2- 172.21.15.157:0/4181443725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f0810159da0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f0810066800 tx=0x7f080801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e17341f127d83d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0807fff640 1 -- 172.21.15.157:0/4181443725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08080a5840 con 0x7f08100b2af0 2023-11-06T03:14:35.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0807fff640 1 -- 172.21.15.157:0/4181443725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08080a59a0 con 0x7f08100b2af0 2023-11-06T03:14:35.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0807fff640 1 -- 172.21.15.157:0/4181443725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08080a5d30 con 0x7f08100b2af0 2023-11-06T03:14:35.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- 172.21.15.157:0/4181443725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 msgr2=0x7f0810159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- 172.21.15.157:0/4181443725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f0810159da0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f0810066800 tx=0x7f080801c960 comp rx=0 tx=0).stop 2023-11-06T03:14:35.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- 172.21.15.157:0/4181443725 shutdown_connections 2023-11-06T03:14:35.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- 172.21.15.157:0/4181443725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f0810159da0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- 172.21.15.157:0/4181443725 >> v2:172.21.15.195:3300/0 conn(0x7f08101599c0 0x7f08100ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- 172.21.15.157:0/4181443725 >> 172.21.15.157:0/4181443725 conn(0x7f0810065930 msgr2=0x7f0810065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- 172.21.15.157:0/4181443725 shutdown_connections 2023-11-06T03:14:35.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- 172.21.15.157:0/4181443725 wait complete. 2023-11-06T03:14:35.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 Processor -- start 2023-11-06T03:14:35.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- start start 2023-11-06T03:14:35.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f081006d000 0x7f08100cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 0x7f081014c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810150720 con 0x7f08100b2af0 2023-11-06T03:14:35.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081014d570 con 0x7f08101599c0 2023-11-06T03:14:35.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f08175c3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081014d830 con 0x7f081006d000 2023-11-06T03:14:35.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0814b5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0814b5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51128/0 (socket says 172.21.15.157:51128) 2023-11-06T03:14:35.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.496+0000 7f0814b5e640 1 -- 172.21.15.157:0/2247564180 learned_addr learned my addr 172.21.15.157:0/2247564180 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:35.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0815b60640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 0x7f081014c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f081535f640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f081006d000 0x7f08100cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 -- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f081006d000 msgr2=0x7f08100cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f081006d000 0x7f08100cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 -- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 msgr2=0x7f081014c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 0x7f081014c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 -- 172.21.15.157:0/2247564180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808009b30 con 0x7f08100b2af0 2023-11-06T03:14:35.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0815b60640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 0x7f081014c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:35.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0814b5e640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f080000c450 tx=0x7f080000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800012830 con 0x7f08100b2af0 2023-11-06T03:14:35.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000129b0 con 0x7f08100b2af0 2023-11-06T03:14:35.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081014ca50 con 0x7f08100b2af0 2023-11-06T03:14:35.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800012d60 con 0x7f08100b2af0 2023-11-06T03:14:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0810150bf0 con 0x7f08100b2af0 2023-11-06T03:14:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810072750 con 0x7f08100b2af0 2023-11-06T03:14:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f080001c460 con 0x7f08100b2af0 2023-11-06T03:14:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 0x7f07f4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0800083b50 con 0x7f08100b2af0 2023-11-06T03:14:35.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f081535f640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 0x7f07f4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.500+0000 7f081535f640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 0x7f07f4067f50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f080c006fd0 tx=0x7f080c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.504+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0800083e60 con 0x7f08100b2af0 2023-11-06T03:14:35.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.668+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08100cb110 con 0x7f08100b2af0 2023-11-06T03:14:35.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f0805ffb640 1 -- 172.21.15.157:0/2247564180 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+17902 (secure 0 0 0) 0x7f0800054fb0 con 0x7f08100b2af0 2023-11-06T03:14:35.672 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:35.672 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:14:35.004786+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:35.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 msgr2=0x7f07f4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 0x7f07f4067f50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f080c006fd0 tx=0x7f080c008040 comp rx=0 tx=0).stop 2023-11-06T03:14:35.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 msgr2=0x7f08101501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f080000c450 tx=0x7f080000c910 comp rx=0 tx=0).stop 2023-11-06T03:14:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 shutdown_connections 2023-11-06T03:14:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07f4065a90 0x7f07f4067f50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08101599c0 0x7f081014c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08100b2af0 0x7f08101501e0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 --2- 172.21.15.157:0/2247564180 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f081006d000 0x7f08100cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 >> 172.21.15.157:0/2247564180 conn(0x7f0810065930 msgr2=0x7f081015b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 shutdown_connections 2023-11-06T03:14:35.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.676+0000 7f08175c3640 1 -- 172.21.15.157:0/2247564180 wait complete. 2023-11-06T03:14:35.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:14:35.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 Processor -- start 2023-11-06T03:14:35.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 -- start start 2023-11-06T03:14:35.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff99006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99015f580 con 0x7ff99015e5e0 2023-11-06T03:14:35.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99015efc0 con 0x7ff9900b3b20 2023-11-06T03:14:35.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff996339640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99015f2a0 con 0x7ff9901551e0 2023-11-06T03:14:35.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff99015e5e0 0x7ff9900b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49162/0 (socket says 172.21.15.157:49162) 2023-11-06T03:14:35.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 -- 172.21.15.157:0/1198085840 learned_addr learned my addr 172.21.15.157:0/1198085840 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:35.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98ffff640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff99006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511318890 0 0) 0x7ff99015f580 con 0x7ff99015e5e0 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff978003530 con 0x7ff99015e5e0 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895029965 0 0) 0x7ff978003530 con 0x7ff99015e5e0 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 msgr2=0x7ff99006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff99006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 >> v1:172.21.15.195:6789/0 conn(0x7ff9900b3b20 legacy=0x7ff99015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99006f5a0 con 0x7ff99015e5e0 2023-11-06T03:14:35.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff984001680 con 0x7ff99015e5e0 2023-11-06T03:14:35.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98ffff640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff99006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:14:35.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 >> v1:172.21.15.157:6789/0 conn(0x7ff99015e5e0 legacy=0x7ff9900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780039b0 0x7ff978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780047d0 0x7ff978004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff978005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99015f2a0 con 0x7ff9780047d0 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800ba90 con 0x7ff9780039b0 2023-11-06T03:14:35.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800bc50 con 0x7ff9901551e0 2023-11-06T03:14:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780047d0 0x7ff978004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98ffff640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780039b0 0x7ff978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff9948d6640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff978005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 -- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 msgr2=0x7ff978005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff978005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 -- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780039b0 msgr2=0x7ff978003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780039b0 0x7ff978003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff98f7fe640 1 -- 172.21.15.157:0/1198085840 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009420 con 0x7ff9780047d0 2023-11-06T03:14:35.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.820+0000 7ff9948d6640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9901551e0 0x7ff978005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:35.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff98f7fe640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780047d0 0x7ff978004c50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c240 tx=0x7ff98401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c185879345969e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984005ae0 con 0x7ff9780047d0 2023-11-06T03:14:35.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984005c40 con 0x7ff9780047d0 2023-11-06T03:14:35.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff98effd640 1 -- 172.21.15.157:0/1198085840 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840a5830 con 0x7ff9780047d0 2023-11-06T03:14:35.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- 172.21.15.157:0/1198085840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780047d0 msgr2=0x7ff978004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 --2- 172.21.15.157:0/1198085840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9780047d0 0x7ff978004c50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ff98400c240 tx=0x7ff98401cc10 comp rx=0 tx=0).stop 2023-11-06T03:14:35.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff9948d6640 1 -- 172.21.15.157:0/1198085840 reap_dead start 2023-11-06T03:14:35.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- 172.21.15.157:0/1198085840 shutdown_connections 2023-11-06T03:14:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- 172.21.15.157:0/1198085840 >> 172.21.15.157:0/1198085840 conn(0x7ff990065930 msgr2=0x7ff990065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- 172.21.15.157:0/1198085840 shutdown_connections 2023-11-06T03:14:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- 172.21.15.157:0/1198085840 wait complete. 2023-11-06T03:14:35.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 Processor -- start 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.824+0000 7ff996339640 1 -- start start 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780047d0 0x7ff99006c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99014ff00 0x7ff9901531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff990151d20 con 0x7ff990162fa0 2023-11-06T03:14:35.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99006d090 con 0x7ff9780047d0 2023-11-06T03:14:35.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99006d370 con 0x7ff99014ff00 2023-11-06T03:14:35.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51140/0 (socket says 172.21.15.157:51140) 2023-11-06T03:14:35.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 -- 172.21.15.157:0/923562991 learned_addr learned my addr 172.21.15.157:0/923562991 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98f7fe640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99014ff00 0x7ff9901531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98ffff640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780047d0 0x7ff99006c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 -- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99014ff00 msgr2=0x7ff9901531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99014ff00 0x7ff9901531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 -- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780047d0 msgr2=0x7ff99006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780047d0 0x7ff99006c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 -- 172.21.15.157:0/923562991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff990151660 con 0x7ff990162fa0 2023-11-06T03:14:35.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff9948d6640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff9900cc2c0 tx=0x7ff98800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988006d60 con 0x7ff990162fa0 2023-11-06T03:14:35.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988003980 con 0x7ff990162fa0 2023-11-06T03:14:35.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff990151940 con 0x7ff990162fa0 2023-11-06T03:14:35.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988003d30 con 0x7ff990162fa0 2023-11-06T03:14:35.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff990153720 con 0x7ff990162fa0 2023-11-06T03:14:35.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990153910 con 0x7ff990162fa0 2023-11-06T03:14:35.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff988005070 con 0x7ff990162fa0 2023-11-06T03:14:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 0x7ff97c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff988083120 con 0x7ff990162fa0 2023-11-06T03:14:35.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.828+0000 7ff98ffff640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 0x7ff97c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:35.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.832+0000 7ff98ffff640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 0x7ff97c067f20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff980009bc0 tx=0x7ff980008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:35.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:35.832+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff990153910 con 0x7ff990162fa0 2023-11-06T03:14:36.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:36.020+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff990153910 con 0x7ff990162fa0 2023-11-06T03:14:36.613 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:14:36.616+0000 7f2159b1b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:14:37.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.024+0000 7ff98cff9640 1 -- 172.21.15.157:0/923562991 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff990153910 con 0x7ff990162fa0 2023-11-06T03:14:37.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 msgr2=0x7ff97c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:37.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 0x7ff97c067f20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff980009bc0 tx=0x7ff980008040 comp rx=0 tx=0).stop 2023-11-06T03:14:37.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 msgr2=0x7ff990152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:37.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff9900cc2c0 tx=0x7ff98800dc00 comp rx=0 tx=0).stop 2023-11-06T03:14:37.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 shutdown_connections 2023-11-06T03:14:37.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff97c065a60 0x7ff97c067f20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff990162fa0 0x7ff990152120 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff99014ff00 0x7ff9901531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 --2- 172.21.15.157:0/923562991 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9780047d0 0x7ff99006c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 >> 172.21.15.157:0/923562991 conn(0x7ff990065930 msgr2=0x7ff99015b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:37.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 shutdown_connections 2023-11-06T03:14:37.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.028+0000 7ff996339640 1 -- 172.21.15.157:0/923562991 wait complete. 2023-11-06T03:14:37.047 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:14:37.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:14:37.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.180+0000 7fc245503640 1 Processor -- start 2023-11-06T03:14:37.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.180+0000 7fc245503640 1 -- start start 2023-11-06T03:14:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2400b2af0 0x7fc2400ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc24006d000 0x7fc240159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24015e440 con 0x7fc2401599c0 2023-11-06T03:14:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24006c240 con 0x7fc2400b2af0 2023-11-06T03:14:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24006c520 con 0x7fc24006d000 2023-11-06T03:14:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2400b2af0 0x7fc2400ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2400b2af0 0x7fc2400ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51144/0 (socket says 172.21.15.157:51144) 2023-11-06T03:14:37.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 -- 172.21.15.157:0/3719672510 learned_addr learned my addr 172.21.15.157:0/3719672510 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:37.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc23f7fe640 1 --2- 172.21.15.157:0/3719672510 >> v2:172.21.15.195:3300/0 conn(0x7fc24006d000 0x7fc240159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 -- 172.21.15.157:0/3719672510 >> v2:172.21.15.195:3300/0 conn(0x7fc24006d000 msgr2=0x7fc240159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:37.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 --2- 172.21.15.157:0/3719672510 >> v2:172.21.15.195:3300/0 conn(0x7fc24006d000 0x7fc240159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 -- 172.21.15.157:0/3719672510 >> v1:172.21.15.195:6789/0 conn(0x7fc2401599c0 legacy=0x7fc2400b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:37.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 -- 172.21.15.157:0/3719672510 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24006c800 con 0x7fc2400b2af0 2023-11-06T03:14:37.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc237fff640 1 --2- 172.21.15.157:0/3719672510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2400b2af0 0x7fc2400ae000 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fc24006d930 tx=0x7fc22801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61bd31ce116abf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:37.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc23e7fc640 1 -- 172.21.15.157:0/3719672510 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280aa610 con 0x7fc2400b2af0 2023-11-06T03:14:37.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc23e7fc640 1 -- 172.21.15.157:0/3719672510 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2280a6040 con 0x7fc2400b2af0 2023-11-06T03:14:37.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc23e7fc640 1 -- 172.21.15.157:0/3719672510 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280aa990 con 0x7fc2400b2af0 2023-11-06T03:14:37.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 -- 172.21.15.157:0/3719672510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2400b2af0 msgr2=0x7fc2400ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:37.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.184+0000 7fc245503640 1 --2- 172.21.15.157:0/3719672510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2400b2af0 0x7fc2400ae000 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fc24006d930 tx=0x7fc22801c490 comp rx=0 tx=0).stop 2023-11-06T03:14:37.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3719672510 shutdown_connections 2023-11-06T03:14:37.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 --2- 172.21.15.157:0/3719672510 >> v2:172.21.15.195:3300/0 conn(0x7fc24006d000 0x7fc240159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 --2- 172.21.15.157:0/3719672510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2400b2af0 0x7fc2400ae000 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3719672510 >> 172.21.15.157:0/3719672510 conn(0x7fc240065930 msgr2=0x7fc240065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:37.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3719672510 shutdown_connections 2023-11-06T03:14:37.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3719672510 wait complete. 2023-11-06T03:14:37.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 Processor -- start 2023-11-06T03:14:37.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- start start 2023-11-06T03:14:37.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24006d000 0x7fc2401f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 0x7fc240158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc240151e60 con 0x7fc2401599c0 2023-11-06T03:14:37.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc240151900 con 0x7fc24006d000 2023-11-06T03:14:37.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc240151b80 con 0x7fc2400b2af0 2023-11-06T03:14:37.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51152/0 (socket says 172.21.15.157:51152) 2023-11-06T03:14:37.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 -- 172.21.15.157:0/3961948471 learned_addr learned my addr 172.21.15.157:0/3961948471 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:37.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23effd640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24006d000 0x7fc2401f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc237fff640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 0x7fc240158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 -- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 msgr2=0x7fc240158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:37.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 0x7fc240158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 -- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24006d000 msgr2=0x7fc2401f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:37.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24006d000 0x7fc2401f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:37.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 -- 172.21.15.157:0/3961948471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280094d0 con 0x7fc2401599c0 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc237fff640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 0x7fc240158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc23f7fe640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc238007e30 tx=0x7fc23800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238014620 con 0x7fc2401599c0 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800ee90 con 0x7fc2401599c0 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2401f8a80 con 0x7fc2401599c0 2023-11-06T03:14:37.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2380149c0 con 0x7fc2401599c0 2023-11-06T03:14:37.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2401f9040 con 0x7fc2401599c0 2023-11-06T03:14:37.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.188+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2401f9230 con 0x7fc2401599c0 2023-11-06T03:14:37.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.192+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc238014b20 con 0x7fc2401599c0 2023-11-06T03:14:37.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.192+0000 7fc2377fe640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 0x7fc220067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:37.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.192+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc238083bb0 con 0x7fc2401599c0 2023-11-06T03:14:37.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.192+0000 7fc23effd640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 0x7fc220067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:37.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.192+0000 7fc23effd640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 0x7fc220067f50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b810 tx=0x7fc230008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:37.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.196+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc2380514a0 con 0x7fc2401599c0 2023-11-06T03:14:37.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:37.388+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc240072750 con 0x7fc2401599c0 2023-11-06T03:14:38.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.036+0000 7fc2377fe640 1 -- 172.21.15.157:0/3961948471 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc238054ac0 con 0x7fc2401599c0 2023-11-06T03:14:38.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 msgr2=0x7fc220067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 0x7fc220067f50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b810 tx=0x7fc230008040 comp rx=0 tx=0).stop 2023-11-06T03:14:38.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 msgr2=0x7fc2401513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc238007e30 tx=0x7fc23800e4b0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 shutdown_connections 2023-11-06T03:14:38.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc220065a90 0x7fc220067f50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2401599c0 0x7fc2401513c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc2400b2af0 0x7fc240158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 --2- 172.21.15.157:0/3961948471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24006d000 0x7fc2401f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 >> 172.21.15.157:0/3961948471 conn(0x7fc240065930 msgr2=0x7fc2401564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 shutdown_connections 2023-11-06T03:14:38.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.040+0000 7fc245503640 1 -- 172.21.15.157:0/3961948471 wait complete. 2023-11-06T03:14:38.058 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:14:38.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:38.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.184+0000 7f79b7496640 1 Processor -- start 2023-11-06T03:14:38.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.184+0000 7f79b7496640 1 -- start start 2023-11-06T03:14:38.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79b006bdb0 0x7f79b00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00cf1a0 con 0x7f79b0146370 2023-11-06T03:14:38.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006c3d0 con 0x7f79b00ce800 2023-11-06T03:14:38.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00b2120 con 0x7f79b006bdb0 2023-11-06T03:14:38.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5a33640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79b006bdb0 0x7f79b00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5232640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f79b0146370 0x7f79b00b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:49172/0 (socket says 172.21.15.157:49172) 2023-11-06T03:14:38.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5232640 1 -- 172.21.15.157:0/1539521352 learned_addr learned my addr 172.21.15.157:0/1539521352 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:38.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5a33640 1 -- 172.21.15.157:0/1539521352 >> v1:172.21.15.195:6789/0 conn(0x7f79b00ce800 legacy=0x7f79b006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:38.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5a33640 1 -- 172.21.15.157:0/1539521352 >> v1:172.21.15.157:6789/0 conn(0x7f79b0146370 legacy=0x7f79b00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5a33640 1 -- 172.21.15.157:0/1539521352 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b006f5a0 con 0x7f79b006bdb0 2023-11-06T03:14:38.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b5a33640 1 --2- 172.21.15.157:0/1539521352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b006bdb0 0x7f79b00cec60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0097f0 tx=0x7f79ac01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309545776db2f02d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79a7fff640 1 -- 172.21.15.157:0/1539521352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0ad680 con 0x7f79b006bdb0 2023-11-06T03:14:38.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79a7fff640 1 -- 172.21.15.157:0/1539521352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac0ad7e0 con 0x7f79b006bdb0 2023-11-06T03:14:38.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79a7fff640 1 -- 172.21.15.157:0/1539521352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0adb70 con 0x7f79b006bdb0 2023-11-06T03:14:38.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- 172.21.15.157:0/1539521352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b006bdb0 msgr2=0x7f79b00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 --2- 172.21.15.157:0/1539521352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b006bdb0 0x7f79b00cec60 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0097f0 tx=0x7f79ac01c380 comp rx=0 tx=0).stop 2023-11-06T03:14:38.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- 172.21.15.157:0/1539521352 shutdown_connections 2023-11-06T03:14:38.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 --2- 172.21.15.157:0/1539521352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b006bdb0 0x7f79b00cec60 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- 172.21.15.157:0/1539521352 >> 172.21.15.157:0/1539521352 conn(0x7f79b0065930 msgr2=0x7f79b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- 172.21.15.157:0/1539521352 shutdown_connections 2023-11-06T03:14:38.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.188+0000 7f79b7496640 1 -- 172.21.15.157:0/1539521352 wait complete. 2023-11-06T03:14:38.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 Processor -- start 2023-11-06T03:14:38.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- start start 2023-11-06T03:14:38.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 0x7f79b01391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b0146370 0x7f79b00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0157c30 con 0x7f79b00ce800 2023-11-06T03:14:38.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015b170 con 0x7f79b006bdb0 2023-11-06T03:14:38.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0145190 con 0x7f79b0146370 2023-11-06T03:14:38.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51170/0 (socket says 172.21.15.157:51170) 2023-11-06T03:14:38.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 -- 172.21.15.157:0/3062454609 learned_addr learned my addr 172.21.15.157:0/3062454609 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:38.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b5232640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 0x7f79b01391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b5a33640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b0146370 0x7f79b00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 -- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b0146370 msgr2=0x7f79b00afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b0146370 0x7f79b00afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 -- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 msgr2=0x7f79b01391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 0x7f79b01391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 -- 172.21.15.157:0/3062454609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac009420 con 0x7f79b00ce800 2023-11-06T03:14:38.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b5232640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 0x7f79b01391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:38.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b4a31640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c480 tx=0x7f79a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0012830 con 0x7f79b00ce800 2023-11-06T03:14:38.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00129b0 con 0x7f79b00ce800 2023-11-06T03:14:38.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0012d60 con 0x7f79b00ce800 2023-11-06T03:14:38.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b0145470 con 0x7f79b00ce800 2023-11-06T03:14:38.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0145a30 con 0x7f79b00ce800 2023-11-06T03:14:38.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.192+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b015b300 con 0x7f79b00ce800 2023-11-06T03:14:38.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.196+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f79a001c460 con 0x7f79b00ce800 2023-11-06T03:14:38.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.196+0000 7f79a5ffb640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 0x7f7994067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.196+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79a0083120 con 0x7f79b00ce800 2023-11-06T03:14:38.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.196+0000 7f79b5232640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 0x7f7994067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.196+0000 7f79b5232640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 0x7f7994067f50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005c10 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.200+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f79b015b300 con 0x7f79b00ce800 2023-11-06T03:14:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.388+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79b015b2b0 con 0x7f79b00ce800 2023-11-06T03:14:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.388+0000 7f79a5ffb640 1 -- 172.21.15.157:0/3062454609 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f79a0050a10 con 0x7f79b00ce800 2023-11-06T03:14:38.387 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:38.387 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:14:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:14:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 msgr2=0x7f7994067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 0x7f7994067f50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005c10 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2023-11-06T03:14:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 msgr2=0x7f79b00af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c480 tx=0x7f79a000c940 comp rx=0 tx=0).stop 2023-11-06T03:14:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 shutdown_connections 2023-11-06T03:14:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7994065a90 0x7f7994067f50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f79b0146370 0x7f79b00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79b00ce800 0x7f79b00af700 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 --2- 172.21.15.157:0/3062454609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79b006bdb0 0x7f79b01391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 >> 172.21.15.157:0/3062454609 conn(0x7f79b0065930 msgr2=0x7f79b00b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 shutdown_connections 2023-11-06T03:14:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.392+0000 7f79b7496640 1 -- 172.21.15.157:0/3062454609 wait complete. 2023-11-06T03:14:38.406 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:14:38.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 Processor -- start 2023-11-06T03:14:38.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 -- start start 2023-11-06T03:14:38.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71b00b2af0 0x7f71b00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 0x7f71b0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b015e440 con 0x7f71b01599c0 2023-11-06T03:14:38.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b006c240 con 0x7f71b00b2af0 2023-11-06T03:14:38.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71b4e85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b006c520 con 0x7f71b006d000 2023-11-06T03:14:38.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f71b00b2af0 0x7f71b00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71ae59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f71b01599c0 0x7f71b00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51352/0 (socket says 172.21.15.157:51352) 2023-11-06T03:14:38.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 --2- 172.21.15.157:0/1179075818 >> v2:172.21.15.157:3300/0 conn(0x7f71b00b2af0 0x7f71b00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46328/0 (socket says 172.21.15.157:46328) 2023-11-06T03:14:38.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71ae59c640 1 -- 172.21.15.157:0/1179075818 learned_addr learned my addr 172.21.15.157:0/1179075818 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:38.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71aed9d640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 0x7f71b0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 -- 172.21.15.157:0/1179075818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 msgr2=0x7f71b0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:38.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 0x7f71b0159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 -- 172.21.15.157:0/1179075818 >> v1:172.21.15.157:6789/0 conn(0x7f71b01599c0 legacy=0x7f71b00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71add9b640 1 -- 172.21.15.157:0/1179075818 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b006c800 con 0x7f71b00b2af0 2023-11-06T03:14:38.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71ad59a640 1 -- 172.21.15.157:0/1179075818 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020966441 0 0) 0x7f71b015e440 con 0x7f71b01599c0 2023-11-06T03:14:38.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.528+0000 7f71aed9d640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 0x7f71b0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:38.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71add9b640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b00b2af0 0x7f71b00ae000 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f71a400ba60 tx=0x7f71a401eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b881a2e7ac3906 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71ad59a640 1 -- 172.21.15.157:0/1179075818 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40a7c10 con 0x7f71b00b2af0 2023-11-06T03:14:38.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71ad59a640 1 -- 172.21.15.157:0/1179075818 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a40a7d70 con 0x7f71b00b2af0 2023-11-06T03:14:38.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71ad59a640 1 -- 172.21.15.157:0/1179075818 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40af660 con 0x7f71b00b2af0 2023-11-06T03:14:38.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/1179075818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b00b2af0 msgr2=0x7f71b00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b00b2af0 0x7f71b00ae000 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f71a400ba60 tx=0x7f71a401eb20 comp rx=0 tx=0).stop 2023-11-06T03:14:38.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/1179075818 shutdown_connections 2023-11-06T03:14:38.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b006d000 0x7f71b0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- 172.21.15.157:0/1179075818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b00b2af0 0x7f71b00ae000 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/1179075818 >> 172.21.15.157:0/1179075818 conn(0x7f71b0065930 msgr2=0x7f71b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/1179075818 shutdown_connections 2023-11-06T03:14:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/1179075818 wait complete. 2023-11-06T03:14:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 Processor -- start 2023-11-06T03:14:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- start start 2023-11-06T03:14:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b006d000 0x7f71b014cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b00b2af0 0x7f71b014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b014e0a0 con 0x7f71b01599c0 2023-11-06T03:14:38.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b014e940 con 0x7f71b006d000 2023-11-06T03:14:38.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b014da30 con 0x7f71b00b2af0 2023-11-06T03:14:38.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46334/0 (socket says 172.21.15.157:46334) 2023-11-06T03:14:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 -- 172.21.15.157:0/3428794889 learned_addr learned my addr 172.21.15.157:0/3428794889 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71ae59c640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b006d000 0x7f71b014cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71add9b640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b00b2af0 0x7f71b014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 -- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b00b2af0 msgr2=0x7f71b014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b00b2af0 0x7f71b014edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 -- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b006d000 msgr2=0x7f71b014cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b006d000 0x7f71b014cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 -- 172.21.15.157:0/3428794889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a400b6c0 con 0x7f71b01599c0 2023-11-06T03:14:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71aed9d640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f71a800ce70 tx=0x7f71a800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8003ba0 con 0x7f71b01599c0 2023-11-06T03:14:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8003d20 con 0x7f71b01599c0 2023-11-06T03:14:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b014e2f0 con 0x7f71b01599c0 2023-11-06T03:14:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800d880 con 0x7f71b01599c0 2023-11-06T03:14:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.532+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01512d0 con 0x7f71b01599c0 2023-11-06T03:14:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0072750 con 0x7f71b01599c0 2023-11-06T03:14:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f71a800d9e0 con 0x7f71b01599c0 2023-11-06T03:14:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f719effd640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 0x7f7190067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71a8083c40 con 0x7f71b01599c0 2023-11-06T03:14:38.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f71ae59c640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 0x7f7190067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.536+0000 7f71ae59c640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 0x7f7190067f20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f70 tx=0x7f71a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.540+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71a8051400 con 0x7f71b01599c0 2023-11-06T03:14:38.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.696+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b00cb110 con 0x7f71b01599c0 2023-11-06T03:14:38.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f719effd640 1 -- 172.21.15.157:0/3428794889 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17889 (secure 0 0 0) 0x7f71a8054a20 con 0x7f71b01599c0 2023-11-06T03:14:38.696 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:38.696 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":137,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:14:36.009463+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:14:38.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 msgr2=0x7f7190067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 0x7f7190067f20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f70 tx=0x7f71a0009040 comp rx=0 tx=0).stop 2023-11-06T03:14:38.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 msgr2=0x7f71b00cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f71a800ce70 tx=0x7f71a800a710 comp rx=0 tx=0).stop 2023-11-06T03:14:38.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 shutdown_connections 2023-11-06T03:14:38.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7190065a60 0x7f7190067f20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71b01599c0 0x7f71b00cbaa0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71b00b2af0 0x7f71b014edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 --2- 172.21.15.157:0/3428794889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b006d000 0x7f71b014cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 >> 172.21.15.157:0/3428794889 conn(0x7f71b0065930 msgr2=0x7f71b0065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 shutdown_connections 2023-11-06T03:14:38.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.700+0000 7f71b4e85640 1 -- 172.21.15.157:0/3428794889 wait complete. 2023-11-06T03:14:38.715 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:14:38.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:14:38.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.840+0000 7f1b66bb4640 1 Processor -- start 2023-11-06T03:14:38.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.840+0000 7f1b66bb4640 1 -- start start 2023-11-06T03:14:38.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b66bb4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b600b3b20 0x7f1b6015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b66bb4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6015f580 con 0x7f1b6015e5e0 2023-11-06T03:14:38.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b66bb4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6015efc0 con 0x7f1b601551e0 2023-11-06T03:14:38.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b66bb4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b600704b0 con 0x7f1b600b3b20 2023-11-06T03:14:38.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b65151640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b600b3b20 0x7f1b6015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b5ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1b6015e5e0 0x7f1b600b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51362/0 (socket says 172.21.15.157:51362) 2023-11-06T03:14:38.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b5ffff640 1 -- 172.21.15.157:0/3954859791 learned_addr learned my addr 172.21.15.157:0/3954859791 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:38.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b65151640 1 -- 172.21.15.157:0/3954859791 >> v1:172.21.15.195:6789/0 conn(0x7f1b601551e0 legacy=0x7f1b6006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b65151640 1 -- 172.21.15.157:0/3954859791 >> v1:172.21.15.157:6789/0 conn(0x7f1b6015e5e0 legacy=0x7f1b600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b65151640 1 -- 172.21.15.157:0/3954859791 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6015f2a0 con 0x7f1b600b3b20 2023-11-06T03:14:38.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b65151640 1 --2- 172.21.15.157:0/3954859791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b6015c1b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009d90 tx=0x7f1b5801c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce11003ec40d716d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.844+0000 7f1b5f7fe640 1 -- 172.21.15.157:0/3954859791 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580a58c0 con 0x7f1b600b3b20 2023-11-06T03:14:38.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5f7fe640 1 -- 172.21.15.157:0/3954859791 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b580a5a20 con 0x7f1b600b3b20 2023-11-06T03:14:38.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5f7fe640 1 -- 172.21.15.157:0/3954859791 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580a5db0 con 0x7f1b600b3b20 2023-11-06T03:14:38.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/3954859791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 msgr2=0x7f1b6015c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/3954859791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b6015c1b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009d90 tx=0x7f1b5801c800 comp rx=0 tx=0).stop 2023-11-06T03:14:38.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/3954859791 shutdown_connections 2023-11-06T03:14:38.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/3954859791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b6015c1b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/3954859791 >> 172.21.15.157:0/3954859791 conn(0x7f1b60065930 msgr2=0x7f1b60065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:38.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/3954859791 shutdown_connections 2023-11-06T03:14:38.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/3954859791 wait complete. 2023-11-06T03:14:38.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 Processor -- start 2023-11-06T03:14:38.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- start start 2023-11-06T03:14:38.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b601551e0 0x7f1b601589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6015e5e0 0x7f1b60158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600b2430 con 0x7f1b600b3b20 2023-11-06T03:14:38.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601f6b10 con 0x7f1b601551e0 2023-11-06T03:14:38.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60159430 con 0x7f1b6015e5e0 2023-11-06T03:14:38.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46352/0 (socket says 172.21.15.157:46352) 2023-11-06T03:14:38.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 -- 172.21.15.157:0/747154247 learned_addr learned my addr 172.21.15.157:0/747154247 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:38.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5ffff640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b601551e0 0x7f1b601589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b65151640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6015e5e0 0x7f1b60158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 -- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6015e5e0 msgr2=0x7f1b60158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6015e5e0 0x7f1b60158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 -- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b601551e0 msgr2=0x7f1b601589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:38.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b601551e0 0x7f1b601589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:38.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 -- 172.21.15.157:0/747154247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009a70 con 0x7f1b600b3b20 2023-11-06T03:14:38.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b64950640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c530 tx=0x7f1b5000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50005220 con 0x7f1b600b3b20 2023-11-06T03:14:38.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50005380 con 0x7f1b600b3b20 2023-11-06T03:14:38.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b600b2680 con 0x7f1b600b3b20 2023-11-06T03:14:38.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50007270 con 0x7f1b600b3b20 2023-11-06T03:14:38.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.848+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b601f67a0 con 0x7f1b600b3b20 2023-11-06T03:14:38.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b60072750 con 0x7f1b600b3b20 2023-11-06T03:14:38.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b500073d0 con 0x7f1b600b3b20 2023-11-06T03:14:38.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b5d7fa640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 0x7f1b4c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:38.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b50082b90 con 0x7f1b600b3b20 2023-11-06T03:14:38.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b5ffff640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 0x7f1b4c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:38.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.852+0000 7f1b5ffff640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 0x7f1b4c067f50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1b60154580 tx=0x7f1b54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:38.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:38.856+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b50050480 con 0x7f1b600b3b20 2023-11-06T03:14:39.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.008+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b6015b1b0 con 0x7f1b4c065a90 2023-11-06T03:14:39.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.032+0000 7f1b5d7fa640 1 -- 172.21.15.157:0/747154247 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+299372 (secure 0 0 0) 0x7f1b54001c80 con 0x7f1b4c065a90 2023-11-06T03:14:39.031 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:39.035 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:14:39.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 msgr2=0x7f1b4c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 0x7f1b4c067f50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1b60154580 tx=0x7f1b54009040 comp rx=0 tx=0).stop 2023-11-06T03:14:39.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 msgr2=0x7f1b60159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000c530 tx=0x7f1b5000c9f0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 shutdown_connections 2023-11-06T03:14:39.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b4c065a90 0x7f1b4c067f50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b6015e5e0 0x7f1b60158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b601551e0 0x7f1b601589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 --2- 172.21.15.157:0/747154247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b600b3b20 0x7f1b60159ff0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 >> 172.21.15.157:0/747154247 conn(0x7f1b60065930 msgr2=0x7f1b600b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 shutdown_connections 2023-11-06T03:14:39.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.036+0000 7f1b66bb4640 1 -- 172.21.15.157:0/747154247 wait complete. 2023-11-06T03:14:39.054 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2023-11-06T03:14:37.324212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":601904,"kb_used_data":14192,"kb_used_omap":0,"kb_used_meta":587648,"kb_avail":754372816,"statfs":{"total":773094113280,"available":772477763584,"internally_reserved":0,"allocated":14532608,"data_stored":7325901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":601751552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.638681+0000","last_change":"2023-11-06T03:14:33.638681+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.638681+0000","last_undegraded":"2023-11-06T03:14:33.638681+0000","last_fullsized":"2023-11-06T03:14:33.638681+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:21.313270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.631267+0000","last_change":"2023-11-06T03:14:33.631267+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.631267+0000","last_undegraded":"2023-11-06T03:14:33.631267+0000","last_fullsized":"2023-11-06T03:14:33.631267+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:46.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.632908+0000","last_change":"2023-11-06T03:14:33.632908+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.632908+0000","last_undegraded":"2023-11-06T03:14:33.632908+0000","last_fullsized":"2023-11-06T03:14:33.632908+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:16:54.094248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.633502+0000","last_change":"2023-11-06T03:14:33.633502+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.633502+0000","last_undegraded":"2023-11-06T03:14:33.633502+0000","last_fullsized":"2023-11-06T03:14:33.633502+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:40:00.983362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.637699+0000","last_change":"2023-11-06T03:14:33.637699+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.637699+0000","last_undegraded":"2023-11-06T03:14:33.637699+0000","last_fullsized":"2023-11-06T03:14:33.637699+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:07:55.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.631613+0000","last_change":"2023-11-06T03:14:33.631613+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.631613+0000","last_undegraded":"2023-11-06T03:14:33.631613+0000","last_fullsized":"2023-11-06T03:14:33.631613+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:55.945675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.635065+0000","last_change":"2023-11-06T03:14:33.635065+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.635065+0000","last_undegraded":"2023-11-06T03:14:33.635065+0000","last_fullsized":"2023-11-06T03:14:33.635065+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:51.350027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.636595+0000","last_change":"2023-11-06T03:14:33.636595+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.636595+0000","last_undegraded":"2023-11-06T03:14:33.636595+0000","last_fullsized":"2023-11-06T03:14:33.636595+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:18.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.642255+0000","last_change":"2023-11-06T03:14:33.642255+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.642255+0000","last_undegraded":"2023-11-06T03:14:33.642255+0000","last_fullsized":"2023-11-06T03:14:33.642255+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:27.565777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.638631+0000","last_change":"2023-11-06T03:14:33.638631+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.638631+0000","last_undegraded":"2023-11-06T03:14:33.638631+0000","last_fullsized":"2023-11-06T03:14:33.638631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:12.134933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634444+0000","last_change":"2023-11-06T03:14:33.634444+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634444+0000","last_undegraded":"2023-11-06T03:14:33.634444+0000","last_fullsized":"2023-11-06T03:14:33.634444+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:44.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.631665+0000","last_change":"2023-11-06T03:14:33.631665+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.631665+0000","last_undegraded":"2023-11-06T03:14:33.631665+0000","last_fullsized":"2023-11-06T03:14:33.631665+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:50.813648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.636679+0000","last_change":"2023-11-06T03:14:33.636679+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.636679+0000","last_undegraded":"2023-11-06T03:14:33.636679+0000","last_fullsized":"2023-11-06T03:14:33.636679+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:16:54.094248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.635921+0000","last_change":"2023-11-06T03:14:33.635921+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.635921+0000","last_undegraded":"2023-11-06T03:14:33.635921+0000","last_fullsized":"2023-11-06T03:14:33.635921+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:11:31.739036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.632112+0000","last_change":"2023-11-06T03:14:33.632112+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.632112+0000","last_undegraded":"2023-11-06T03:14:33.632112+0000","last_fullsized":"2023-11-06T03:14:33.632112+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:06:44.199655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634324+0000","last_change":"2023-11-06T03:14:33.634324+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634324+0000","last_undegraded":"2023-11-06T03:14:33.634324+0000","last_fullsized":"2023-11-06T03:14:33.634324+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:51.235823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.635427+0000","last_change":"2023-11-06T03:14:33.635427+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.635427+0000","last_undegraded":"2023-11-06T03:14:33.635427+0000","last_fullsized":"2023-11-06T03:14:33.635427+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:18:38.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.637014+0000","last_change":"2023-11-06T03:14:33.637014+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.637014+0000","last_undegraded":"2023-11-06T03:14:33.637014+0000","last_fullsized":"2023-11-06T03:14:33.637014+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:41.796339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.638265+0000","last_change":"2023-11-06T03:14:33.638265+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.638265+0000","last_undegraded":"2023-11-06T03:14:33.638265+0000","last_fullsized":"2023-11-06T03:14:33.638265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:40:00.983362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634418+0000","last_change":"2023-11-06T03:14:33.634418+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634418+0000","last_undegraded":"2023-11-06T03:14:33.634418+0000","last_fullsized":"2023-11-06T03:14:33.634418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:06:44.199655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.642858+0000","last_change":"2023-11-06T03:14:33.642858+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.642858+0000","last_undegraded":"2023-11-06T03:14:33.642858+0000","last_fullsized":"2023-11-06T03:14:33.642858+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:46.148170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.632050+0000","last_change":"2023-11-06T03:14:33.632050+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.632050+0000","last_undegraded":"2023-11-06T03:14:33.632050+0000","last_fullsized":"2023-11-06T03:14:33.632050+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:51.235823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634358+0000","last_change":"2023-11-06T03:14:33.634358+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634358+0000","last_undegraded":"2023-11-06T03:14:33.634358+0000","last_fullsized":"2023-11-06T03:14:33.634358+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:37.138948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.630929+0000","last_change":"2023-11-06T03:14:33.630929+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.630929+0000","last_undegraded":"2023-11-06T03:14:33.630929+0000","last_fullsized":"2023-11-06T03:14:33.630929+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:42.016610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.632936+0000","last_change":"2023-11-06T03:14:33.632936+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.632936+0000","last_undegraded":"2023-11-06T03:14:33.632936+0000","last_fullsized":"2023-11-06T03:14:33.632936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:22.459648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.796974+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:14:31.796974+0000","last_peered":"2023-11-06T03:14:31.796974+0000","last_clean":"2023-11-06T03:14:31.796974+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:14:31.796974+0000","last_undegraded":"2023-11-06T03:14:31.796974+0000","last_fullsized":"2023-11-06T03:14:31.796974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.798301+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:14:31.798301+0000","last_peered":"2023-11-06T03:14:31.798301+0000","last_clean":"2023-11-06T03:14:31.798301+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:14:31.798301+0000","last_undegraded":"2023-11-06T03:14:31.798301+0000","last_fullsized":"2023-11-06T03:14:31.798301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.636528+0000","last_change":"2023-11-06T03:14:33.636528+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.636528+0000","last_undegraded":"2023-11-06T03:14:33.636528+0000","last_fullsized":"2023-11-06T03:14:33.636528+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:02:31.735613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.796754+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:14:31.796754+0000","last_peered":"2023-11-06T03:14:31.796754+0000","last_clean":"2023-11-06T03:14:31.796754+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:14:31.796754+0000","last_undegraded":"2023-11-06T03:14:31.796754+0000","last_fullsized":"2023-11-06T03:14:31.796754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634789+0000","last_change":"2023-11-06T03:14:33.634789+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634789+0000","last_undegraded":"2023-11-06T03:14:33.634789+0000","last_fullsized":"2023-11-06T03:14:33.634789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:22.459648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":105,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.780018+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:14:31.780018+0000","last_peered":"2023-11-06T03:14:31.780018+0000","last_clean":"2023-11-06T03:14:31.780018+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:14:31.780018+0000","last_undegraded":"2023-11-06T03:14:31.780018+0000","last_fullsized":"2023-11-06T03:14:31.780018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.635983+0000","last_change":"2023-11-06T03:14:33.635983+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.635983+0000","last_undegraded":"2023-11-06T03:14:33.635983+0000","last_fullsized":"2023-11-06T03:14:33.635983+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:20.305844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":102,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.798271+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:14:31.798271+0000","last_peered":"2023-11-06T03:14:31.798271+0000","last_clean":"2023-11-06T03:14:31.798271+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:14:31.798271+0000","last_undegraded":"2023-11-06T03:14:31.798271+0000","last_fullsized":"2023-11-06T03:14:31.798271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.631931+0000","last_change":"2023-11-06T03:14:33.631931+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.631931+0000","last_undegraded":"2023-11-06T03:14:33.631931+0000","last_fullsized":"2023-11-06T03:14:33.631931+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:07:55.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":110,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.716368+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:14:31.716368+0000","last_peered":"2023-11-06T03:14:31.716368+0000","last_clean":"2023-11-06T03:14:31.716368+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:14:31.716368+0000","last_undegraded":"2023-11-06T03:14:31.716368+0000","last_fullsized":"2023-11-06T03:14:31.716368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.632242+0000","last_change":"2023-11-06T03:14:33.632242+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.632242+0000","last_undegraded":"2023-11-06T03:14:33.632242+0000","last_fullsized":"2023-11-06T03:14:33.632242+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:37.138948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.797029+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:14:31.797029+0000","last_peered":"2023-11-06T03:14:31.797029+0000","last_clean":"2023-11-06T03:14:31.797029+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:14:31.797029+0000","last_undegraded":"2023-11-06T03:14:31.797029+0000","last_fullsized":"2023-11-06T03:14:31.797029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.643191+0000","last_change":"2023-11-06T03:14:33.643191+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.643191+0000","last_undegraded":"2023-11-06T03:14:33.643191+0000","last_fullsized":"2023-11-06T03:14:33.643191+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:47:07.237326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.716418+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:14:31.716418+0000","last_peered":"2023-11-06T03:14:31.716418+0000","last_clean":"2023-11-06T03:14:31.716418+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:14:31.716418+0000","last_undegraded":"2023-11-06T03:14:31.716418+0000","last_fullsized":"2023-11-06T03:14:31.716418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:14:34.623868+0000","last_change":"2023-11-06T03:14:34.623868+0000","last_active":"2023-11-06T03:14:34.623868+0000","last_peered":"2023-11-06T03:14:34.623868+0000","last_clean":"2023-11-06T03:14:34.623868+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:34.623868+0000","last_undegraded":"2023-11-06T03:14:34.623868+0000","last_fullsized":"2023-11-06T03:14:34.623868+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2023-11-06T03:14:33.634834+0000","last_change":"2023-11-06T03:14:33.634834+0000","last_active":"2023-11-06T03:14:33.611199+0000","last_peered":"2023-11-06T03:14:33.611199+0000","last_clean":"2023-11-06T03:14:33.611199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:14:33.634834+0000","last_undegraded":"2023-11-06T03:14:33.634834+0000","last_fullsized":"2023-11-06T03:14:33.634834+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:17.547269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":100,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-06T03:14:31.798072+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:14:31.798072+0000","last_peered":"2023-11-06T03:14:31.798072+0000","last_clean":"2023-11-06T03:14:31.798072+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:14:31.798072+0000","last_undegraded":"2023-11-06T03:14:31.798072+0000","last_fullsized":"2023-11-06T03:14:31.798072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574979,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87340,"kb_used_data":2212,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284500,"statfs":{"total":96636764160,"available":96547328000,"internally_reserved":0,"allocated":2265088,"data_stored":1359667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.4830000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.3620000000000001,"15min":1.397},"last":0.52300000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.86499999999999999,"15min":0.88100000000000001},"min":{"1min":0.54400000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1890000000000001,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.89000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.2170000000000001,"5min":1.288,"15min":1.6080000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.879,"15min":0.91600000000000004},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.151,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.89300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":2.512},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.0409999999999999},"min":{"1min":0.63100000000000001,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.139,"5min":1.381,"15min":10.596},"last":1.1100000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.96899999999999997,"15min":0.96399999999999997},"min":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.2509999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.71399999999999997,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.0669999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.036,"15min":1.044},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2929999999999999,"5min":1.4339999999999999,"15min":1.9139999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1,"5min":1.081,"15min":1.0840000000000001},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.1899999999999999,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":0.99399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.232,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":9.6159999999999997,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.0660000000000001,"15min":1.097},"min":{"1min":0.754,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":1.1579999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64612,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307228,"statfs":{"total":96636764160,"available":96570601472,"internally_reserved":0,"allocated":1667072,"data_stored":768280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.85199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":2.2429999999999999,"5min":2.3279999999999998,"15min":16.073},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.97499999999999998},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":2.1850000000000001,"5min":4.4000000000000004,"15min":40.905000000000001},"last":0.92200000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83299999999999996,"15min":0.90100000000000002},"min":{"1min":0.53500000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.351,"5min":1.6359999999999999,"15min":16.012},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.51800000000000002,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.081,"5min":1.409,"15min":3.3050000000000002},"last":0.78200000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82999999999999996,"15min":0.91500000000000004},"min":{"1min":0.56599999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.702,"5min":1.744,"15min":27.803999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.92600000000000005,"15min":0.94899999999999995},"min":{"1min":0.60099999999999998,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.429,"5min":1.542,"15min":2.843},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.94099999999999995,"15min":1.1799999999999999},"min":{"1min":0.61699999999999999,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.264,"5min":1.306,"15min":63.737000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.91900000000000004,"15min":1.056},"min":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.88400000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0960000000000001,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.58699999999999997},"max":{"1min":1.502,"5min":7.4980000000000002,"15min":24.867999999999999},"last":1.034},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1639999999999999,"15min":1.2330000000000001},"min":{"1min":0.93000000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4530000000000001,"5min":6.3970000000000002,"15min":23.789000000000001},"last":1.1679999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.98799999999999999,"15min":1.157},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.103,"15min":1.115},"min":{"1min":0.79200000000000004,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.5620000000000001,"5min":1.6160000000000001,"15min":4.1159999999999997},"last":1.101}]},{"osd":7,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.1870000000000001,"15min":1.2150000000000001},"min":{"1min":0.81299999999999994,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":4.2389999999999999,"5min":4.6399999999999997,"15min":23.748999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.1359999999999999,"15min":1.2230000000000001},"min":{"1min":0.89400000000000002,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.1379999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308824,"statfs":{"total":96636764160,"available":96572235776,"internally_reserved":0,"allocated":1671168,"data_stored":769299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.804,"15min":1.1379999999999999},"min":{"1min":0.53300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":4.0510000000000002,"5min":125.256,"15min":125.256},"last":0.80700000000000005},{"interface":"front","average":{"1min":2.222,"5min":2.1059999999999999,"15min":1.2509999999999999},"min":{"1min":0.57399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":26.518000000000001,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.749}]},{"osd":1,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.89700000000000002,"15min":0.80400000000000005},"min":{"1min":0.59699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.6390000000000002,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":3.1579999999999999,"5min":1.284,"15min":0.997},"min":{"1min":0.53600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":43.799999999999997,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.2030000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":3.274,"5min":1.3360000000000001,"15min":1.008},"min":{"1min":0.626,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":43.741,"5min":43.741,"15min":43.741},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.85799999999999998,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.385,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.89500000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92400000000000004,"15min":0.875},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.349,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.173},{"interface":"front","average":{"1min":3.3159999999999998,"5min":1.4330000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":43.841999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.91900000000000004,"15min":0.91600000000000004},"min":{"1min":0.47399999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.321,"5min":1.121,"15min":1.012},"min":{"1min":0.70299999999999996,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":4.4020000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.0329999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0489999999999999,"15min":1.016},"min":{"1min":0.501,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":2,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.095},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.113,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":4.1230000000000002,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":0.95799999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.155,"15min":1.0509999999999999},"min":{"1min":0.53300000000000003,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.131},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.145,"15min":1.1060000000000001},"min":{"1min":0.66500000000000004,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.2749999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90856,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280984,"statfs":{"total":96636764160,"available":96543727616,"internally_reserved":0,"allocated":1671168,"data_stored":769299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":1.25,"15min":0.96099999999999997},"min":{"1min":0.496,"5min":0.436,"15min":0.372},"max":{"1min":1.4319999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.2869999999999999,"15min":0.95699999999999996},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.3100000000000001,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.89300000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77600000000000002,"15min":0.76900000000000002},"min":{"1min":0.47799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.628},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.85799999999999998,"15min":0.83399999999999996},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.6719999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88800000000000001,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.53300000000000003},"max":{"1min":1.2230000000000001,"5min":1.452,"15min":3.2879999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.876,"15min":0.86199999999999999},"min":{"1min":0.51200000000000001,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.262,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.0620000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":0.92200000000000004},"min":{"1min":0.58499999999999996,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.3779999999999999,"5min":3.198,"15min":3.198},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89300000000000002,"15min":0.89700000000000002},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.288,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0800000000000001,"15min":1.03},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5429999999999999,"5min":10.192,"15min":10.192},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0880000000000001},"min":{"1min":0.70299999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.415,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.1060000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.97699999999999998,"15min":0.998},"min":{"1min":0.64200000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3640000000000001,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.119,"5min":1.0569999999999999,"15min":1.109},"min":{"1min":0.72399999999999998,"5min":0.65300000000000002,"15min":0.57799999999999996},"max":{"1min":1.512,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.089,"15min":1.077},"min":{"1min":0.66000000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5289999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.151},{"interface":"front","average":{"1min":1.155,"5min":1.109,"15min":1.171},"min":{"1min":0.745,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.526,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.125}]}]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81048,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":79424,"kb_avail":94290792,"statfs":{"total":96636764160,"available":96553771008,"internally_reserved":0,"allocated":1654784,"data_stored":762110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81330176},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54800000000000004,"15min":0.59599999999999997},"min":{"1min":0.23499999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.93600000000000005,"5min":1.113,"15min":1.4199999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.27800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.80700000000000005,"5min":0.85399999999999998,"15min":1.524},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.251,"5min":0.224,"15min":0.224},"max":{"1min":0.754,"5min":1.3420000000000001,"15min":1.371},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.29999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.96999999999999997,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69699999999999995,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.95899999999999996,"5min":1.532,"15min":1.532},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.284},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83499999999999996,"15min":0.876},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0580000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89400000000000002,"15min":0.92600000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3129999999999999,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.53900000000000003,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.2529999999999999,"5min":1.5,"15min":1.6639999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95399999999999996,"15min":0.99299999999999999},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.169,"5min":1.573,"15min":1.698},"last":0.79400000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95699999999999996,"15min":0.98699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.26,"5min":1.649,"15min":1.8620000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.93600000000000005,"15min":0.96999999999999997},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3120000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":1.018},"min":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.194,"5min":1.835,"15min":1.835},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.014,"15min":1.0409999999999999},"min":{"1min":0.53900000000000003,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":1.359,"5min":1.734,"15min":1.8979999999999999},"last":0.70399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68712,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94303128,"statfs":{"total":96636764160,"available":96566403072,"internally_reserved":0,"allocated":1671168,"data_stored":769299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.51000000000000001,"15min":0.53900000000000003},"min":{"1min":0.23200000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.751,"5min":0.93400000000000005,"15min":1.7270000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53600000000000003,"15min":0.60199999999999998},"min":{"1min":0.27400000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.79300000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57899999999999996,"15min":0.623},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.88,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57599999999999996,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":1.099,"15min":1.1579999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.66200000000000003},"min":{"1min":0.34899999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.087,"5min":1.161,"15min":1.294},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.70399999999999996},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2010000000000001,"5min":1.244,"15min":1.526},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.90500000000000003},"min":{"1min":0.55000000000000004,"5min":0.439,"15min":0.41399999999999998},"max":{"1min":1.2809999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83199999999999996,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.627},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.872,"15min":0.93000000000000005},"min":{"1min":0.496,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.355,"5min":1.4359999999999999,"15min":1.5780000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93799999999999994,"15min":0.97499999999999998},"min":{"1min":0.54100000000000004,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.615},"last":0.752},{"interface":"front","average":{"1min":0.871,"5min":0.92100000000000004,"15min":0.93899999999999995},"min":{"1min":0.68999999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.93300000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89800000000000002,"15min":0.96699999999999997},"min":{"1min":0.50900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90000000000000002,"15min":0.93600000000000005},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50728,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321112,"statfs":{"total":96636764160,"available":96584818688,"internally_reserved":0,"allocated":1671168,"data_stored":769299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.006,"5min":1.843,"15min":1.843},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.79600000000000004,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55400000000000005,"15min":0.69599999999999995},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.83699999999999997,"5min":1.169,"15min":30.861000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59799999999999998,"15min":0.64700000000000002},"min":{"1min":0.30599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.874,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.80100000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65600000000000003,"15min":0.69899999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.1240000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.129,"5min":1.151,"15min":2.3079999999999998},"last":0.97899999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.434},"max":{"1min":1.0800000000000001,"5min":2.0870000000000002,"15min":2.258},"last":1.119},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.5549999999999999,"15min":14.753},"last":1.1419999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":1.089},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.173,"5min":1.528,"15min":43.103999999999999},"last":1.351},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.93100000000000005,"15min":1.1220000000000001},"min":{"1min":0.64600000000000002,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.274,"5min":1.3979999999999999,"15min":44.753999999999998},"last":1.091}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.89100000000000001,"15min":1.0309999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":23.971},"last":1.3240000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":1.0469999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1950000000000001,"5min":1.444,"15min":14.813000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97599999999999998,"15min":1.028},"min":{"1min":0.65800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.581,"5min":1.581,"15min":2.2799999999999998},"last":1.25},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":1.016},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3680000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":1.2190000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95592,"kb_used_data":2208,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94276248,"statfs":{"total":96636764160,"available":96538877952,"internally_reserved":0,"allocated":2260992,"data_stored":1358648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57399999999999995},"min":{"1min":0.28199999999999997,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.91800000000000004,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60399999999999998,"15min":0.64300000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.35699999999999998,"5min":0.22800000000000001,"15min":0.20599999999999999},"max":{"1min":0.997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.64500000000000002},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1779999999999999,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.71399999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.95599999999999996,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.622},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.33400000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.099,"5min":1.391,"15min":2.7999999999999998},"last":0.755}]},{"osd":4,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.262,"5min":1.371,"15min":4.641},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.89000000000000001},"min":{"1min":0.51300000000000001,"5min":0.46899999999999997,"15min":0.42899999999999999},"max":{"1min":1.0860000000000001,"5min":1.254,"15min":2.028},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.2929999999999999,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89800000000000002,"15min":0.93700000000000006},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.401,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":1.0720000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.88100000000000001,"15min":0.89700000000000002},"min":{"1min":0.503,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.1559999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97699999999999998,"15min":0.998},"min":{"1min":0.60199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2969999999999999,"5min":1.802,"15min":2.7789999999999999},"last":1.0269999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93899999999999995,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.3520000000000001,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.125},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3320000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.98899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:14:39.063 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:14:39.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 Processor -- start 2023-11-06T03:14:39.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 -- start start 2023-11-06T03:14:39.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44200b2330 0x7f442006c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442006c490 con 0x7f44200adae0 2023-11-06T03:14:39.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442006fb80 con 0x7f4420156a70 2023-11-06T03:14:39.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4426a54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442006b950 con 0x7f44200b2330 2023-11-06T03:14:39.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f44177fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f44200adae0 0x7f44200b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51364/0 (socket says 172.21.15.157:51364) 2023-11-06T03:14:39.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f44177fe640 1 -- 172.21.15.157:0/4265770860 learned_addr learned my addr 172.21.15.157:0/4265770860 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:39.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4424ff1640 1 --2- 172.21.15.157:0/4265770860 >> v2:172.21.15.157:3300/0 conn(0x7f44200b2330 0x7f442006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f441f7fe640 1 -- 172.21.15.157:0/4265770860 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894000512 0 0) 0x7f442006c490 con 0x7f44200adae0 2023-11-06T03:14:39.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f441f7fe640 1 -- 172.21.15.157:0/4265770860 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c003450 con 0x7f44200adae0 2023-11-06T03:14:39.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4424ff1640 1 -- 172.21.15.157:0/4265770860 >> v1:172.21.15.195:6789/0 conn(0x7f4420156a70 legacy=0x7f442006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4424ff1640 1 -- 172.21.15.157:0/4265770860 >> v1:172.21.15.157:6789/0 conn(0x7f44200adae0 legacy=0x7f44200b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.228+0000 7f4424ff1640 1 -- 172.21.15.157:0/4265770860 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442006bb60 con 0x7f44200b2330 2023-11-06T03:14:39.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4424ff1640 1 --2- 172.21.15.157:0/4265770860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44200b2330 0x7f442006c920 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4418009d90 tx=0x7f441801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5231091359d2f50b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f441f7fe640 1 -- 172.21.15.157:0/4265770860 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44180a59e0 con 0x7f44200b2330 2023-11-06T03:14:39.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f441f7fe640 1 -- 172.21.15.157:0/4265770860 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180a5b40 con 0x7f44200b2330 2023-11-06T03:14:39.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f441f7fe640 1 -- 172.21.15.157:0/4265770860 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44180ad440 con 0x7f44200b2330 2023-11-06T03:14:39.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- 172.21.15.157:0/4265770860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44200b2330 msgr2=0x7f442006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 --2- 172.21.15.157:0/4265770860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44200b2330 0x7f442006c920 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f4418009d90 tx=0x7f441801c920 comp rx=0 tx=0).stop 2023-11-06T03:14:39.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- 172.21.15.157:0/4265770860 shutdown_connections 2023-11-06T03:14:39.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 --2- 172.21.15.157:0/4265770860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44200b2330 0x7f442006c920 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- 172.21.15.157:0/4265770860 >> 172.21.15.157:0/4265770860 conn(0x7f4420065930 msgr2=0x7f4420065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- 172.21.15.157:0/4265770860 shutdown_connections 2023-11-06T03:14:39.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- 172.21.15.157:0/4265770860 wait complete. 2023-11-06T03:14:39.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 Processor -- start 2023-11-06T03:14:39.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- start start 2023-11-06T03:14:39.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44200adae0 0x7f44200af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 0x7f442014e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b0600 con 0x7f4420156a70 2023-11-06T03:14:39.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200af9f0 con 0x7f44200b2330 2023-11-06T03:14:39.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4426a54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200afcd0 con 0x7f44200adae0 2023-11-06T03:14:39.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4424ff1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4424ff1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46362/0 (socket says 172.21.15.157:46362) 2023-11-06T03:14:39.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f4424ff1640 1 -- 172.21.15.157:0/1651040052 learned_addr learned my addr 172.21.15.157:0/1651040052 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:39.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f441ffff640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44200adae0 0x7f44200af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.232+0000 7f44177fe640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 0x7f442014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 -- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44200adae0 msgr2=0x7f44200af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44200adae0 0x7f44200af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 -- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 msgr2=0x7f442014e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 0x7f442014e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 -- 172.21.15.157:0/1651040052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009a70 con 0x7f4420156a70 2023-11-06T03:14:39.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f44177fe640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 0x7f442014e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:39.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4424ff1640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f4418019530 tx=0x7f44180196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44180aee00 con 0x7f4420156a70 2023-11-06T03:14:39.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418009450 con 0x7f4420156a70 2023-11-06T03:14:39.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44200affb0 con 0x7f4420156a70 2023-11-06T03:14:39.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44200af620 con 0x7f4420156a70 2023-11-06T03:14:39.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418003bc0 con 0x7f4420156a70 2023-11-06T03:14:39.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420153990 con 0x7f4420156a70 2023-11-06T03:14:39.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f44180b4050 con 0x7f4420156a70 2023-11-06T03:14:39.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 0x7f4400067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f441811d9e0 con 0x7f4420156a70 2023-11-06T03:14:39.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441ffff640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 0x7f4400067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.236+0000 7f441ffff640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 0x7f4400067f20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4410000f70 tx=0x7f4410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.240+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4420153990 con 0x7f4420156a70 2023-11-06T03:14:39.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.392+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4420156e50 con 0x7f4400065a60 2023-11-06T03:14:39.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.416+0000 7f441d7fa640 1 -- 172.21.15.157:0/1651040052 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310448 (secure 0 0 0) 0x7f4410001370 con 0x7f4400065a60 2023-11-06T03:14:39.413 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:39.416 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:14:39.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 msgr2=0x7f4400067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 0x7f4400067f20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4410000f70 tx=0x7f4410009040 comp rx=0 tx=0).stop 2023-11-06T03:14:39.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 msgr2=0x7f442014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f4418019530 tx=0x7f44180196c0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 shutdown_connections 2023-11-06T03:14:39.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4400065a60 0x7f4400067f20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4420156a70 0x7f442014c2b0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44200b2330 0x7f442014e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 --2- 172.21.15.157:0/1651040052 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44200adae0 0x7f44200af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 >> 172.21.15.157:0/1651040052 conn(0x7f4420065930 msgr2=0x7f442015a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 shutdown_connections 2023-11-06T03:14:39.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.420+0000 7f4426a54640 1 -- 172.21.15.157:0/1651040052 wait complete. 2023-11-06T03:14:39.433 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2023-11-06T03:14:39.324829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":606364,"kb_used_data":14556,"kb_used_omap":0,"kb_used_meta":591744,"kb_avail":754368356,"statfs":{"total":773094113280,"available":772473196544,"internally_reserved":0,"allocated":14905344,"data_stored":7500003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":605945856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.695952"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848761+0000","last_change":"2023-11-06T03:14:35.030163+0000","last_active":"2023-11-06T03:14:37.848761+0000","last_peered":"2023-11-06T03:14:37.848761+0000","last_clean":"2023-11-06T03:14:37.848761+0000","last_became_active":"2023-11-06T03:14:35.029833+0000","last_became_peered":"2023-11-06T03:14:35.029833+0000","last_unstale":"2023-11-06T03:14:37.848761+0000","last_undegraded":"2023-11-06T03:14:37.848761+0000","last_fullsized":"2023-11-06T03:14:37.848761+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:08:34.509664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.849419+0000","last_change":"2023-11-06T03:14:35.050996+0000","last_active":"2023-11-06T03:14:37.849419+0000","last_peered":"2023-11-06T03:14:37.849419+0000","last_clean":"2023-11-06T03:14:37.849419+0000","last_became_active":"2023-11-06T03:14:35.050265+0000","last_became_peered":"2023-11-06T03:14:35.050265+0000","last_unstale":"2023-11-06T03:14:37.849419+0000","last_undegraded":"2023-11-06T03:14:37.849419+0000","last_fullsized":"2023-11-06T03:14:37.849419+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:55:54.441063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148759+0000","last_change":"2023-11-06T03:14:35.074611+0000","last_active":"2023-11-06T03:14:36.148759+0000","last_peered":"2023-11-06T03:14:36.148759+0000","last_clean":"2023-11-06T03:14:36.148759+0000","last_became_active":"2023-11-06T03:14:35.074132+0000","last_became_peered":"2023-11-06T03:14:35.074132+0000","last_unstale":"2023-11-06T03:14:36.148759+0000","last_undegraded":"2023-11-06T03:14:36.148759+0000","last_fullsized":"2023-11-06T03:14:36.148759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:28.578446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848946+0000","last_change":"2023-11-06T03:14:35.053366+0000","last_active":"2023-11-06T03:14:37.848946+0000","last_peered":"2023-11-06T03:14:37.848946+0000","last_clean":"2023-11-06T03:14:37.848946+0000","last_became_active":"2023-11-06T03:14:35.052447+0000","last_became_peered":"2023-11-06T03:14:35.052447+0000","last_unstale":"2023-11-06T03:14:37.848946+0000","last_undegraded":"2023-11-06T03:14:37.848946+0000","last_fullsized":"2023-11-06T03:14:37.848946+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:42.809008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023371+0000","last_change":"2023-11-06T03:14:35.049070+0000","last_active":"2023-11-06T03:14:36.023371+0000","last_peered":"2023-11-06T03:14:36.023371+0000","last_clean":"2023-11-06T03:14:36.023371+0000","last_became_active":"2023-11-06T03:14:35.048804+0000","last_became_peered":"2023-11-06T03:14:35.048804+0000","last_unstale":"2023-11-06T03:14:36.023371+0000","last_undegraded":"2023-11-06T03:14:36.023371+0000","last_fullsized":"2023-11-06T03:14:36.023371+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:21:15.156668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594164+0000","last_change":"2023-11-06T03:14:35.072820+0000","last_active":"2023-11-06T03:14:37.594164+0000","last_peered":"2023-11-06T03:14:37.594164+0000","last_clean":"2023-11-06T03:14:37.594164+0000","last_became_active":"2023-11-06T03:14:35.072510+0000","last_became_peered":"2023-11-06T03:14:35.072510+0000","last_unstale":"2023-11-06T03:14:37.594164+0000","last_undegraded":"2023-11-06T03:14:37.594164+0000","last_fullsized":"2023-11-06T03:14:37.594164+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:25.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.538733+0000","last_change":"2023-11-06T03:14:35.078147+0000","last_active":"2023-11-06T03:14:36.538733+0000","last_peered":"2023-11-06T03:14:36.538733+0000","last_clean":"2023-11-06T03:14:36.538733+0000","last_became_active":"2023-11-06T03:14:35.077143+0000","last_became_peered":"2023-11-06T03:14:35.077143+0000","last_unstale":"2023-11-06T03:14:36.538733+0000","last_undegraded":"2023-11-06T03:14:36.538733+0000","last_fullsized":"2023-11-06T03:14:36.538733+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:08.893032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.027890+0000","last_change":"2023-11-06T03:14:35.044567+0000","last_active":"2023-11-06T03:14:36.027890+0000","last_peered":"2023-11-06T03:14:36.027890+0000","last_clean":"2023-11-06T03:14:36.027890+0000","last_became_active":"2023-11-06T03:14:35.042130+0000","last_became_peered":"2023-11-06T03:14:35.042130+0000","last_unstale":"2023-11-06T03:14:36.027890+0000","last_undegraded":"2023-11-06T03:14:36.027890+0000","last_fullsized":"2023-11-06T03:14:36.027890+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:49:14.774782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594719+0000","last_change":"2023-11-06T03:14:35.057026+0000","last_active":"2023-11-06T03:14:37.594719+0000","last_peered":"2023-11-06T03:14:37.594719+0000","last_clean":"2023-11-06T03:14:37.594719+0000","last_became_active":"2023-11-06T03:14:35.056787+0000","last_became_peered":"2023-11-06T03:14:35.056787+0000","last_unstale":"2023-11-06T03:14:37.594719+0000","last_undegraded":"2023-11-06T03:14:37.594719+0000","last_fullsized":"2023-11-06T03:14:37.594719+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:00.628533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.218081+0000","last_change":"2023-11-06T03:14:35.040592+0000","last_active":"2023-11-06T03:14:37.218081+0000","last_peered":"2023-11-06T03:14:37.218081+0000","last_clean":"2023-11-06T03:14:37.218081+0000","last_became_active":"2023-11-06T03:14:35.040319+0000","last_became_peered":"2023-11-06T03:14:35.040319+0000","last_unstale":"2023-11-06T03:14:37.218081+0000","last_undegraded":"2023-11-06T03:14:37.218081+0000","last_fullsized":"2023-11-06T03:14:37.218081+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:26:34.956731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848991+0000","last_change":"2023-11-06T03:14:35.078294+0000","last_active":"2023-11-06T03:14:37.848991+0000","last_peered":"2023-11-06T03:14:37.848991+0000","last_clean":"2023-11-06T03:14:37.848991+0000","last_became_active":"2023-11-06T03:14:35.077682+0000","last_became_peered":"2023-11-06T03:14:35.077682+0000","last_unstale":"2023-11-06T03:14:37.848991+0000","last_undegraded":"2023-11-06T03:14:37.848991+0000","last_fullsized":"2023-11-06T03:14:37.848991+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:02:38.768908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848553+0000","last_change":"2023-11-06T03:14:35.032362+0000","last_active":"2023-11-06T03:14:37.848553+0000","last_peered":"2023-11-06T03:14:37.848553+0000","last_clean":"2023-11-06T03:14:37.848553+0000","last_became_active":"2023-11-06T03:14:35.032094+0000","last_became_peered":"2023-11-06T03:14:35.032094+0000","last_unstale":"2023-11-06T03:14:37.848553+0000","last_undegraded":"2023-11-06T03:14:37.848553+0000","last_fullsized":"2023-11-06T03:14:37.848553+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:34.790641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594664+0000","last_change":"2023-11-06T03:14:35.073302+0000","last_active":"2023-11-06T03:14:37.594664+0000","last_peered":"2023-11-06T03:14:37.594664+0000","last_clean":"2023-11-06T03:14:37.594664+0000","last_became_active":"2023-11-06T03:14:35.073031+0000","last_became_peered":"2023-11-06T03:14:35.073031+0000","last_unstale":"2023-11-06T03:14:37.594664+0000","last_undegraded":"2023-11-06T03:14:37.594664+0000","last_fullsized":"2023-11-06T03:14:37.594664+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:33.779930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848242+0000","last_change":"2023-11-06T03:14:35.045127+0000","last_active":"2023-11-06T03:14:37.848242+0000","last_peered":"2023-11-06T03:14:37.848242+0000","last_clean":"2023-11-06T03:14:37.848242+0000","last_became_active":"2023-11-06T03:14:35.044842+0000","last_became_peered":"2023-11-06T03:14:35.044842+0000","last_unstale":"2023-11-06T03:14:37.848242+0000","last_undegraded":"2023-11-06T03:14:37.848242+0000","last_fullsized":"2023-11-06T03:14:37.848242+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:19.944588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848463+0000","last_change":"2023-11-06T03:14:35.062596+0000","last_active":"2023-11-06T03:14:37.848463+0000","last_peered":"2023-11-06T03:14:37.848463+0000","last_clean":"2023-11-06T03:14:37.848463+0000","last_became_active":"2023-11-06T03:14:35.062360+0000","last_became_peered":"2023-11-06T03:14:35.062360+0000","last_unstale":"2023-11-06T03:14:37.848463+0000","last_undegraded":"2023-11-06T03:14:37.848463+0000","last_fullsized":"2023-11-06T03:14:37.848463+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:04:37.482313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.593871+0000","last_change":"2023-11-06T03:14:35.029194+0000","last_active":"2023-11-06T03:14:37.593871+0000","last_peered":"2023-11-06T03:14:37.593871+0000","last_clean":"2023-11-06T03:14:37.593871+0000","last_became_active":"2023-11-06T03:14:35.028868+0000","last_became_peered":"2023-11-06T03:14:35.028868+0000","last_unstale":"2023-11-06T03:14:37.593871+0000","last_undegraded":"2023-11-06T03:14:37.593871+0000","last_fullsized":"2023-11-06T03:14:37.593871+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:46.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594856+0000","last_change":"2023-11-06T03:14:35.029879+0000","last_active":"2023-11-06T03:14:37.594856+0000","last_peered":"2023-11-06T03:14:37.594856+0000","last_clean":"2023-11-06T03:14:37.594856+0000","last_became_active":"2023-11-06T03:14:35.029615+0000","last_became_peered":"2023-11-06T03:14:35.029615+0000","last_unstale":"2023-11-06T03:14:37.594856+0000","last_undegraded":"2023-11-06T03:14:37.594856+0000","last_fullsized":"2023-11-06T03:14:37.594856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:42.456780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023627+0000","last_change":"2023-11-06T03:14:35.059801+0000","last_active":"2023-11-06T03:14:36.023627+0000","last_peered":"2023-11-06T03:14:36.023627+0000","last_clean":"2023-11-06T03:14:36.023627+0000","last_became_active":"2023-11-06T03:14:35.059394+0000","last_became_peered":"2023-11-06T03:14:35.059394+0000","last_unstale":"2023-11-06T03:14:36.023627+0000","last_undegraded":"2023-11-06T03:14:36.023627+0000","last_fullsized":"2023-11-06T03:14:36.023627+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:17:02.847610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539375+0000","last_change":"2023-11-06T03:14:35.074840+0000","last_active":"2023-11-06T03:14:36.539375+0000","last_peered":"2023-11-06T03:14:36.539375+0000","last_clean":"2023-11-06T03:14:36.539375+0000","last_became_active":"2023-11-06T03:14:35.073803+0000","last_became_peered":"2023-11-06T03:14:35.073803+0000","last_unstale":"2023-11-06T03:14:36.539375+0000","last_undegraded":"2023-11-06T03:14:36.539375+0000","last_fullsized":"2023-11-06T03:14:36.539375+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:49.915994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023709+0000","last_change":"2023-11-06T03:14:35.064051+0000","last_active":"2023-11-06T03:14:36.023709+0000","last_peered":"2023-11-06T03:14:36.023709+0000","last_clean":"2023-11-06T03:14:36.023709+0000","last_became_active":"2023-11-06T03:14:35.063784+0000","last_became_peered":"2023-11-06T03:14:35.063784+0000","last_unstale":"2023-11-06T03:14:36.023709+0000","last_undegraded":"2023-11-06T03:14:36.023709+0000","last_fullsized":"2023-11-06T03:14:36.023709+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:40:49.973131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848922+0000","last_change":"2023-11-06T03:14:35.052330+0000","last_active":"2023-11-06T03:14:37.848922+0000","last_peered":"2023-11-06T03:14:37.848922+0000","last_clean":"2023-11-06T03:14:37.848922+0000","last_became_active":"2023-11-06T03:14:35.051631+0000","last_became_peered":"2023-11-06T03:14:35.051631+0000","last_unstale":"2023-11-06T03:14:37.848922+0000","last_undegraded":"2023-11-06T03:14:37.848922+0000","last_fullsized":"2023-11-06T03:14:37.848922+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:07:56.316669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848039+0000","last_change":"2023-11-06T03:14:35.052324+0000","last_active":"2023-11-06T03:14:37.848039+0000","last_peered":"2023-11-06T03:14:37.848039+0000","last_clean":"2023-11-06T03:14:37.848039+0000","last_became_active":"2023-11-06T03:14:35.052014+0000","last_became_peered":"2023-11-06T03:14:35.052014+0000","last_unstale":"2023-11-06T03:14:37.848039+0000","last_undegraded":"2023-11-06T03:14:37.848039+0000","last_fullsized":"2023-11-06T03:14:37.848039+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:32:12.327290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848887+0000","last_change":"2023-11-06T03:14:35.064886+0000","last_active":"2023-11-06T03:14:37.848887+0000","last_peered":"2023-11-06T03:14:37.848887+0000","last_clean":"2023-11-06T03:14:37.848887+0000","last_became_active":"2023-11-06T03:14:35.064334+0000","last_became_peered":"2023-11-06T03:14:35.064334+0000","last_unstale":"2023-11-06T03:14:37.848887+0000","last_undegraded":"2023-11-06T03:14:37.848887+0000","last_fullsized":"2023-11-06T03:14:37.848887+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:33:57.915651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.593943+0000","last_change":"2023-11-06T03:14:35.052059+0000","last_active":"2023-11-06T03:14:37.593943+0000","last_peered":"2023-11-06T03:14:37.593943+0000","last_clean":"2023-11-06T03:14:37.593943+0000","last_became_active":"2023-11-06T03:14:35.051810+0000","last_became_peered":"2023-11-06T03:14:35.051810+0000","last_unstale":"2023-11-06T03:14:37.593943+0000","last_undegraded":"2023-11-06T03:14:37.593943+0000","last_fullsized":"2023-11-06T03:14:37.593943+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:33.984315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148955+0000","last_change":"2023-11-06T03:14:35.075633+0000","last_active":"2023-11-06T03:14:36.148955+0000","last_peered":"2023-11-06T03:14:36.148955+0000","last_clean":"2023-11-06T03:14:36.148955+0000","last_became_active":"2023-11-06T03:14:35.075000+0000","last_became_peered":"2023-11-06T03:14:35.075000+0000","last_unstale":"2023-11-06T03:14:36.148955+0000","last_undegraded":"2023-11-06T03:14:36.148955+0000","last_fullsized":"2023-11-06T03:14:36.148955+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:18.119294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148579+0000","last_change":"2023-11-06T03:14:35.076959+0000","last_active":"2023-11-06T03:14:36.148579+0000","last_peered":"2023-11-06T03:14:36.148579+0000","last_clean":"2023-11-06T03:14:36.148579+0000","last_became_active":"2023-11-06T03:14:35.076449+0000","last_became_peered":"2023-11-06T03:14:35.076449+0000","last_unstale":"2023-11-06T03:14:36.148579+0000","last_undegraded":"2023-11-06T03:14:36.148579+0000","last_fullsized":"2023-11-06T03:14:36.148579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:52.362696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539354+0000","last_change":"2023-11-06T03:14:35.076165+0000","last_active":"2023-11-06T03:14:36.539354+0000","last_peered":"2023-11-06T03:14:36.539354+0000","last_clean":"2023-11-06T03:14:36.539354+0000","last_became_active":"2023-11-06T03:14:35.074432+0000","last_became_peered":"2023-11-06T03:14:35.074432+0000","last_unstale":"2023-11-06T03:14:36.539354+0000","last_undegraded":"2023-11-06T03:14:36.539354+0000","last_fullsized":"2023-11-06T03:14:36.539354+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:42.456780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.218063+0000","last_change":"2023-11-06T03:14:35.040738+0000","last_active":"2023-11-06T03:14:37.218063+0000","last_peered":"2023-11-06T03:14:37.218063+0000","last_clean":"2023-11-06T03:14:37.218063+0000","last_became_active":"2023-11-06T03:14:35.040315+0000","last_became_peered":"2023-11-06T03:14:35.040315+0000","last_unstale":"2023-11-06T03:14:37.218063+0000","last_undegraded":"2023-11-06T03:14:37.218063+0000","last_fullsized":"2023-11-06T03:14:37.218063+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:00:58.289685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148698+0000","last_change":"2023-11-06T03:14:35.059777+0000","last_active":"2023-11-06T03:14:36.148698+0000","last_peered":"2023-11-06T03:14:36.148698+0000","last_clean":"2023-11-06T03:14:36.148698+0000","last_became_active":"2023-11-06T03:14:35.059061+0000","last_became_peered":"2023-11-06T03:14:35.059061+0000","last_unstale":"2023-11-06T03:14:36.148698+0000","last_undegraded":"2023-11-06T03:14:36.148698+0000","last_fullsized":"2023-11-06T03:14:36.148698+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:00:53.423180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148160+0000","last_change":"2023-11-06T03:14:35.074387+0000","last_active":"2023-11-06T03:14:36.148160+0000","last_peered":"2023-11-06T03:14:36.148160+0000","last_clean":"2023-11-06T03:14:36.148160+0000","last_became_active":"2023-11-06T03:14:35.073767+0000","last_became_peered":"2023-11-06T03:14:35.073767+0000","last_unstale":"2023-11-06T03:14:36.148160+0000","last_undegraded":"2023-11-06T03:14:36.148160+0000","last_fullsized":"2023-11-06T03:14:36.148160+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:38:38.385231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.027933+0000","last_change":"2023-11-06T03:14:35.025992+0000","last_active":"2023-11-06T03:14:36.027933+0000","last_peered":"2023-11-06T03:14:36.027933+0000","last_clean":"2023-11-06T03:14:36.027933+0000","last_became_active":"2023-11-06T03:14:35.025205+0000","last_became_peered":"2023-11-06T03:14:35.025205+0000","last_unstale":"2023-11-06T03:14:36.027933+0000","last_undegraded":"2023-11-06T03:14:36.027933+0000","last_fullsized":"2023-11-06T03:14:36.027933+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:47.339408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.217870+0000","last_change":"2023-11-06T03:14:35.032426+0000","last_active":"2023-11-06T03:14:37.217870+0000","last_peered":"2023-11-06T03:14:37.217870+0000","last_clean":"2023-11-06T03:14:37.217870+0000","last_became_active":"2023-11-06T03:14:35.032169+0000","last_became_peered":"2023-11-06T03:14:35.032169+0000","last_unstale":"2023-11-06T03:14:37.217870+0000","last_undegraded":"2023-11-06T03:14:37.217870+0000","last_fullsized":"2023-11-06T03:14:37.217870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:41:25.380727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.028230+0000","last_change":"2023-11-06T03:14:35.065186+0000","last_active":"2023-11-06T03:14:36.028230+0000","last_peered":"2023-11-06T03:14:36.028230+0000","last_clean":"2023-11-06T03:14:36.028230+0000","last_became_active":"2023-11-06T03:14:35.064465+0000","last_became_peered":"2023-11-06T03:14:35.064465+0000","last_unstale":"2023-11-06T03:14:36.028230+0000","last_undegraded":"2023-11-06T03:14:36.028230+0000","last_fullsized":"2023-11-06T03:14:36.028230+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:33.779930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"136'4","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.083453+0000","last_change":"2023-11-06T03:14:34.666887+0000","last_active":"2023-11-06T03:14:35.083453+0000","last_peered":"2023-11-06T03:14:35.083453+0000","last_clean":"2023-11-06T03:14:35.083453+0000","last_became_active":"2023-11-06T03:14:34.666263+0000","last_became_peered":"2023-11-06T03:14:34.666263+0000","last_unstale":"2023-11-06T03:14:35.083453+0000","last_undegraded":"2023-11-06T03:14:35.083453+0000","last_fullsized":"2023-11-06T03:14:35.083453+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:21.313270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848285+0000","last_change":"2023-11-06T03:14:35.051123+0000","last_active":"2023-11-06T03:14:37.848285+0000","last_peered":"2023-11-06T03:14:37.848285+0000","last_clean":"2023-11-06T03:14:37.848285+0000","last_became_active":"2023-11-06T03:14:35.050839+0000","last_became_peered":"2023-11-06T03:14:35.050839+0000","last_unstale":"2023-11-06T03:14:37.848285+0000","last_undegraded":"2023-11-06T03:14:37.848285+0000","last_fullsized":"2023-11-06T03:14:37.848285+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:45:25.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.059579+0000","last_change":"2023-11-06T03:14:34.663918+0000","last_active":"2023-11-06T03:14:35.059579+0000","last_peered":"2023-11-06T03:14:35.059579+0000","last_clean":"2023-11-06T03:14:35.059579+0000","last_became_active":"2023-11-06T03:14:34.663630+0000","last_became_peered":"2023-11-06T03:14:34.663630+0000","last_unstale":"2023-11-06T03:14:35.059579+0000","last_undegraded":"2023-11-06T03:14:35.059579+0000","last_fullsized":"2023-11-06T03:14:35.059579+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:46.326740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848933+0000","last_change":"2023-11-06T03:14:35.064993+0000","last_active":"2023-11-06T03:14:37.848933+0000","last_peered":"2023-11-06T03:14:37.848933+0000","last_clean":"2023-11-06T03:14:37.848933+0000","last_became_active":"2023-11-06T03:14:35.064474+0000","last_became_peered":"2023-11-06T03:14:35.064474+0000","last_unstale":"2023-11-06T03:14:37.848933+0000","last_undegraded":"2023-11-06T03:14:37.848933+0000","last_fullsized":"2023-11-06T03:14:37.848933+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:51.826317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.073408+0000","last_change":"2023-11-06T03:14:34.678506+0000","last_active":"2023-11-06T03:14:35.073408+0000","last_peered":"2023-11-06T03:14:35.073408+0000","last_clean":"2023-11-06T03:14:35.073408+0000","last_became_active":"2023-11-06T03:14:34.677877+0000","last_became_peered":"2023-11-06T03:14:34.677877+0000","last_unstale":"2023-11-06T03:14:35.073408+0000","last_undegraded":"2023-11-06T03:14:35.073408+0000","last_fullsized":"2023-11-06T03:14:35.073408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:16:54.094248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848083+0000","last_change":"2023-11-06T03:14:35.062903+0000","last_active":"2023-11-06T03:14:37.848083+0000","last_peered":"2023-11-06T03:14:37.848083+0000","last_clean":"2023-11-06T03:14:37.848083+0000","last_became_active":"2023-11-06T03:14:35.062582+0000","last_became_peered":"2023-11-06T03:14:35.062582+0000","last_unstale":"2023-11-06T03:14:37.848083+0000","last_undegraded":"2023-11-06T03:14:37.848083+0000","last_fullsized":"2023-11-06T03:14:37.848083+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.072503+0000","last_change":"2023-11-06T03:14:34.678440+0000","last_active":"2023-11-06T03:14:35.072503+0000","last_peered":"2023-11-06T03:14:35.072503+0000","last_clean":"2023-11-06T03:14:35.072503+0000","last_became_active":"2023-11-06T03:14:34.677826+0000","last_became_peered":"2023-11-06T03:14:34.677826+0000","last_unstale":"2023-11-06T03:14:35.072503+0000","last_undegraded":"2023-11-06T03:14:35.072503+0000","last_fullsized":"2023-11-06T03:14:35.072503+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:40:00.983362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.028681+0000","last_change":"2023-11-06T03:14:35.027195+0000","last_active":"2023-11-06T03:14:36.028681+0000","last_peered":"2023-11-06T03:14:36.028681+0000","last_clean":"2023-11-06T03:14:36.028681+0000","last_became_active":"2023-11-06T03:14:35.026439+0000","last_became_peered":"2023-11-06T03:14:35.026439+0000","last_unstale":"2023-11-06T03:14:36.028681+0000","last_undegraded":"2023-11-06T03:14:36.028681+0000","last_fullsized":"2023-11-06T03:14:36.028681+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:00.628533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.041299+0000","last_change":"2023-11-06T03:14:34.664350+0000","last_active":"2023-11-06T03:14:35.041299+0000","last_peered":"2023-11-06T03:14:35.041299+0000","last_clean":"2023-11-06T03:14:35.041299+0000","last_became_active":"2023-11-06T03:14:34.663771+0000","last_became_peered":"2023-11-06T03:14:34.663771+0000","last_unstale":"2023-11-06T03:14:35.041299+0000","last_undegraded":"2023-11-06T03:14:35.041299+0000","last_fullsized":"2023-11-06T03:14:35.041299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:07:55.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.028545+0000","last_change":"2023-11-06T03:14:35.027632+0000","last_active":"2023-11-06T03:14:36.028545+0000","last_peered":"2023-11-06T03:14:36.028545+0000","last_clean":"2023-11-06T03:14:36.028545+0000","last_became_active":"2023-11-06T03:14:35.027062+0000","last_became_peered":"2023-11-06T03:14:35.027062+0000","last_unstale":"2023-11-06T03:14:36.028545+0000","last_undegraded":"2023-11-06T03:14:36.028545+0000","last_fullsized":"2023-11-06T03:14:36.028545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:07:42.456780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.040214+0000","last_change":"2023-11-06T03:14:34.657499+0000","last_active":"2023-11-06T03:14:35.040214+0000","last_peered":"2023-11-06T03:14:35.040214+0000","last_clean":"2023-11-06T03:14:35.040214+0000","last_became_active":"2023-11-06T03:14:34.657233+0000","last_became_peered":"2023-11-06T03:14:34.657233+0000","last_unstale":"2023-11-06T03:14:35.040214+0000","last_undegraded":"2023-11-06T03:14:35.040214+0000","last_fullsized":"2023-11-06T03:14:35.040214+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:43:55.945675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023841+0000","last_change":"2023-11-06T03:14:35.063099+0000","last_active":"2023-11-06T03:14:36.023841+0000","last_peered":"2023-11-06T03:14:36.023841+0000","last_clean":"2023-11-06T03:14:36.023841+0000","last_became_active":"2023-11-06T03:14:35.062810+0000","last_became_peered":"2023-11-06T03:14:35.062810+0000","last_unstale":"2023-11-06T03:14:36.023841+0000","last_undegraded":"2023-11-06T03:14:36.023841+0000","last_fullsized":"2023-11-06T03:14:36.023841+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:13:18.684397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.011705+0000","last_change":"2023-11-06T03:14:34.650172+0000","last_active":"2023-11-06T03:14:35.011705+0000","last_peered":"2023-11-06T03:14:35.011705+0000","last_clean":"2023-11-06T03:14:35.011705+0000","last_became_active":"2023-11-06T03:14:34.649895+0000","last_became_peered":"2023-11-06T03:14:34.649895+0000","last_unstale":"2023-11-06T03:14:35.011705+0000","last_undegraded":"2023-11-06T03:14:35.011705+0000","last_fullsized":"2023-11-06T03:14:35.011705+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:32:51.350027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539268+0000","last_change":"2023-11-06T03:14:35.072899+0000","last_active":"2023-11-06T03:14:36.539268+0000","last_peered":"2023-11-06T03:14:36.539268+0000","last_clean":"2023-11-06T03:14:36.539268+0000","last_became_active":"2023-11-06T03:14:35.072397+0000","last_became_peered":"2023-11-06T03:14:35.072397+0000","last_unstale":"2023-11-06T03:14:36.539268+0000","last_undegraded":"2023-11-06T03:14:36.539268+0000","last_fullsized":"2023-11-06T03:14:36.539268+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:31:46.218122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.035478+0000","last_change":"2023-11-06T03:14:34.659537+0000","last_active":"2023-11-06T03:14:35.035478+0000","last_peered":"2023-11-06T03:14:35.035478+0000","last_clean":"2023-11-06T03:14:35.035478+0000","last_became_active":"2023-11-06T03:14:34.658766+0000","last_became_peered":"2023-11-06T03:14:34.658766+0000","last_unstale":"2023-11-06T03:14:35.035478+0000","last_undegraded":"2023-11-06T03:14:35.035478+0000","last_fullsized":"2023-11-06T03:14:35.035478+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:43:18.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148176+0000","last_change":"2023-11-06T03:14:35.059782+0000","last_active":"2023-11-06T03:14:36.148176+0000","last_peered":"2023-11-06T03:14:36.148176+0000","last_clean":"2023-11-06T03:14:36.148176+0000","last_became_active":"2023-11-06T03:14:35.059009+0000","last_became_peered":"2023-11-06T03:14:35.059009+0000","last_unstale":"2023-11-06T03:14:36.148176+0000","last_undegraded":"2023-11-06T03:14:36.148176+0000","last_fullsized":"2023-11-06T03:14:36.148176+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:47.327592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.057811+0000","last_change":"2023-11-06T03:14:34.656747+0000","last_active":"2023-11-06T03:14:35.057811+0000","last_peered":"2023-11-06T03:14:35.057811+0000","last_clean":"2023-11-06T03:14:35.057811+0000","last_became_active":"2023-11-06T03:14:34.656008+0000","last_became_peered":"2023-11-06T03:14:34.656008+0000","last_unstale":"2023-11-06T03:14:35.057811+0000","last_undegraded":"2023-11-06T03:14:35.057811+0000","last_fullsized":"2023-11-06T03:14:35.057811+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:27.565777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.538740+0000","last_change":"2023-11-06T03:14:35.072888+0000","last_active":"2023-11-06T03:14:36.538740+0000","last_peered":"2023-11-06T03:14:36.538740+0000","last_clean":"2023-11-06T03:14:36.538740+0000","last_became_active":"2023-11-06T03:14:35.072154+0000","last_became_peered":"2023-11-06T03:14:35.072154+0000","last_unstale":"2023-11-06T03:14:36.538740+0000","last_undegraded":"2023-11-06T03:14:36.538740+0000","last_fullsized":"2023-11-06T03:14:36.538740+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:05.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.083263+0000","last_change":"2023-11-06T03:14:34.666480+0000","last_active":"2023-11-06T03:14:35.083263+0000","last_peered":"2023-11-06T03:14:35.083263+0000","last_clean":"2023-11-06T03:14:35.083263+0000","last_became_active":"2023-11-06T03:14:34.665414+0000","last_became_peered":"2023-11-06T03:14:34.665414+0000","last_unstale":"2023-11-06T03:14:35.083263+0000","last_undegraded":"2023-11-06T03:14:35.083263+0000","last_fullsized":"2023-11-06T03:14:35.083263+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:19:12.134933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539297+0000","last_change":"2023-11-06T03:14:35.072648+0000","last_active":"2023-11-06T03:14:36.539297+0000","last_peered":"2023-11-06T03:14:36.539297+0000","last_clean":"2023-11-06T03:14:36.539297+0000","last_became_active":"2023-11-06T03:14:35.072021+0000","last_became_peered":"2023-11-06T03:14:35.072021+0000","last_unstale":"2023-11-06T03:14:36.539297+0000","last_undegraded":"2023-11-06T03:14:36.539297+0000","last_fullsized":"2023-11-06T03:14:36.539297+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:49:14.774782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"136'1","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.035597+0000","last_change":"2023-11-06T03:14:34.650219+0000","last_active":"2023-11-06T03:14:35.035597+0000","last_peered":"2023-11-06T03:14:35.035597+0000","last_clean":"2023-11-06T03:14:35.035597+0000","last_became_active":"2023-11-06T03:14:34.649624+0000","last_became_peered":"2023-11-06T03:14:34.649624+0000","last_unstale":"2023-11-06T03:14:35.035597+0000","last_undegraded":"2023-11-06T03:14:35.035597+0000","last_fullsized":"2023-11-06T03:14:35.035597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:44.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.217997+0000","last_change":"2023-11-06T03:14:35.073491+0000","last_active":"2023-11-06T03:14:37.217997+0000","last_peered":"2023-11-06T03:14:37.217997+0000","last_clean":"2023-11-06T03:14:37.217997+0000","last_became_active":"2023-11-06T03:14:35.073094+0000","last_became_peered":"2023-11-06T03:14:35.073094+0000","last_unstale":"2023-11-06T03:14:37.217997+0000","last_undegraded":"2023-11-06T03:14:37.217997+0000","last_fullsized":"2023-11-06T03:14:37.217997+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:11:09.573733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.025336+0000","last_change":"2023-11-06T03:14:34.659684+0000","last_active":"2023-11-06T03:14:35.025336+0000","last_peered":"2023-11-06T03:14:35.025336+0000","last_clean":"2023-11-06T03:14:35.025336+0000","last_became_active":"2023-11-06T03:14:34.659352+0000","last_became_peered":"2023-11-06T03:14:34.659352+0000","last_unstale":"2023-11-06T03:14:35.025336+0000","last_undegraded":"2023-11-06T03:14:35.025336+0000","last_fullsized":"2023-11-06T03:14:35.025336+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:43:50.813648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023645+0000","last_change":"2023-11-06T03:14:35.032886+0000","last_active":"2023-11-06T03:14:36.023645+0000","last_peered":"2023-11-06T03:14:36.023645+0000","last_clean":"2023-11-06T03:14:36.023645+0000","last_became_active":"2023-11-06T03:14:35.032275+0000","last_became_peered":"2023-11-06T03:14:35.032275+0000","last_unstale":"2023-11-06T03:14:36.023645+0000","last_undegraded":"2023-11-06T03:14:36.023645+0000","last_fullsized":"2023-11-06T03:14:36.023645+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:50:38.643562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.061351+0000","last_change":"2023-11-06T03:14:34.667950+0000","last_active":"2023-11-06T03:14:35.061351+0000","last_peered":"2023-11-06T03:14:35.061351+0000","last_clean":"2023-11-06T03:14:35.061351+0000","last_became_active":"2023-11-06T03:14:34.667453+0000","last_became_peered":"2023-11-06T03:14:34.667453+0000","last_unstale":"2023-11-06T03:14:35.061351+0000","last_undegraded":"2023-11-06T03:14:35.061351+0000","last_fullsized":"2023-11-06T03:14:35.061351+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:16:54.094248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.848783+0000","last_change":"2023-11-06T03:14:35.067050+0000","last_active":"2023-11-06T03:14:37.848783+0000","last_peered":"2023-11-06T03:14:37.848783+0000","last_clean":"2023-11-06T03:14:37.848783+0000","last_became_active":"2023-11-06T03:14:35.066792+0000","last_became_peered":"2023-11-06T03:14:35.066792+0000","last_unstale":"2023-11-06T03:14:37.848783+0000","last_undegraded":"2023-11-06T03:14:37.848783+0000","last_fullsized":"2023-11-06T03:14:37.848783+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:33.882755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.019091+0000","last_change":"2023-11-06T03:14:34.660954+0000","last_active":"2023-11-06T03:14:35.019091+0000","last_peered":"2023-11-06T03:14:35.019091+0000","last_clean":"2023-11-06T03:14:35.019091+0000","last_became_active":"2023-11-06T03:14:34.660664+0000","last_became_peered":"2023-11-06T03:14:34.660664+0000","last_unstale":"2023-11-06T03:14:35.019091+0000","last_undegraded":"2023-11-06T03:14:35.019091+0000","last_fullsized":"2023-11-06T03:14:35.019091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:11:31.739036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.024062+0000","last_change":"2023-11-06T03:14:35.075608+0000","last_active":"2023-11-06T03:14:36.024062+0000","last_peered":"2023-11-06T03:14:36.024062+0000","last_clean":"2023-11-06T03:14:36.024062+0000","last_became_active":"2023-11-06T03:14:35.075318+0000","last_became_peered":"2023-11-06T03:14:35.075318+0000","last_unstale":"2023-11-06T03:14:36.024062+0000","last_undegraded":"2023-11-06T03:14:36.024062+0000","last_fullsized":"2023-11-06T03:14:36.024062+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:35:20.586417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.042654+0000","last_change":"2023-11-06T03:14:34.678670+0000","last_active":"2023-11-06T03:14:35.042654+0000","last_peered":"2023-11-06T03:14:35.042654+0000","last_clean":"2023-11-06T03:14:35.042654+0000","last_became_active":"2023-11-06T03:14:34.677999+0000","last_became_peered":"2023-11-06T03:14:34.677999+0000","last_unstale":"2023-11-06T03:14:35.042654+0000","last_undegraded":"2023-11-06T03:14:35.042654+0000","last_fullsized":"2023-11-06T03:14:35.042654+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:06:44.199655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.849247+0000","last_change":"2023-11-06T03:14:35.080101+0000","last_active":"2023-11-06T03:14:37.849247+0000","last_peered":"2023-11-06T03:14:37.849247+0000","last_clean":"2023-11-06T03:14:37.849247+0000","last_became_active":"2023-11-06T03:14:35.079487+0000","last_became_peered":"2023-11-06T03:14:35.079487+0000","last_unstale":"2023-11-06T03:14:37.849247+0000","last_undegraded":"2023-11-06T03:14:37.849247+0000","last_fullsized":"2023-11-06T03:14:37.849247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:40:01.996031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.012230+0000","last_change":"2023-11-06T03:14:34.654308+0000","last_active":"2023-11-06T03:14:35.012230+0000","last_peered":"2023-11-06T03:14:35.012230+0000","last_clean":"2023-11-06T03:14:35.012230+0000","last_became_active":"2023-11-06T03:14:34.654021+0000","last_became_peered":"2023-11-06T03:14:34.654021+0000","last_unstale":"2023-11-06T03:14:35.012230+0000","last_undegraded":"2023-11-06T03:14:35.012230+0000","last_fullsized":"2023-11-06T03:14:35.012230+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:51.235823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023043+0000","last_change":"2023-11-06T03:14:35.063115+0000","last_active":"2023-11-06T03:14:36.023043+0000","last_peered":"2023-11-06T03:14:36.023043+0000","last_clean":"2023-11-06T03:14:36.023043+0000","last_became_active":"2023-11-06T03:14:35.062859+0000","last_became_peered":"2023-11-06T03:14:35.062859+0000","last_unstale":"2023-11-06T03:14:36.023043+0000","last_undegraded":"2023-11-06T03:14:36.023043+0000","last_fullsized":"2023-11-06T03:14:36.023043+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:47:38.211041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.064591+0000","last_change":"2023-11-06T03:14:34.668561+0000","last_active":"2023-11-06T03:14:35.064591+0000","last_peered":"2023-11-06T03:14:35.064591+0000","last_clean":"2023-11-06T03:14:35.064591+0000","last_became_active":"2023-11-06T03:14:34.668134+0000","last_became_peered":"2023-11-06T03:14:34.668134+0000","last_unstale":"2023-11-06T03:14:35.064591+0000","last_undegraded":"2023-11-06T03:14:35.064591+0000","last_fullsized":"2023-11-06T03:14:35.064591+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:18:38.238696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.147942+0000","last_change":"2023-11-06T03:14:35.060559+0000","last_active":"2023-11-06T03:14:36.147942+0000","last_peered":"2023-11-06T03:14:36.147942+0000","last_clean":"2023-11-06T03:14:36.147942+0000","last_became_active":"2023-11-06T03:14:35.058397+0000","last_became_peered":"2023-11-06T03:14:35.058397+0000","last_unstale":"2023-11-06T03:14:36.147942+0000","last_undegraded":"2023-11-06T03:14:36.147942+0000","last_fullsized":"2023-11-06T03:14:36.147942+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:47:08.249995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.021910+0000","last_change":"2023-11-06T03:14:34.655433+0000","last_active":"2023-11-06T03:14:35.021910+0000","last_peered":"2023-11-06T03:14:35.021910+0000","last_clean":"2023-11-06T03:14:35.021910+0000","last_became_active":"2023-11-06T03:14:34.654913+0000","last_became_peered":"2023-11-06T03:14:34.654913+0000","last_unstale":"2023-11-06T03:14:35.021910+0000","last_undegraded":"2023-11-06T03:14:35.021910+0000","last_fullsized":"2023-11-06T03:14:35.021910+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:39:41.796339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.538731+0000","last_change":"2023-11-06T03:14:35.077700+0000","last_active":"2023-11-06T03:14:36.538731+0000","last_peered":"2023-11-06T03:14:36.538731+0000","last_clean":"2023-11-06T03:14:36.538731+0000","last_became_active":"2023-11-06T03:14:35.077113+0000","last_became_peered":"2023-11-06T03:14:35.077113+0000","last_unstale":"2023-11-06T03:14:36.538731+0000","last_undegraded":"2023-11-06T03:14:36.538731+0000","last_fullsized":"2023-11-06T03:14:36.538731+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:33.779930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.021909+0000","last_change":"2023-11-06T03:14:34.661252+0000","last_active":"2023-11-06T03:14:35.021909+0000","last_peered":"2023-11-06T03:14:35.021909+0000","last_clean":"2023-11-06T03:14:35.021909+0000","last_became_active":"2023-11-06T03:14:34.660622+0000","last_became_peered":"2023-11-06T03:14:34.660622+0000","last_unstale":"2023-11-06T03:14:35.021909+0000","last_undegraded":"2023-11-06T03:14:35.021909+0000","last_fullsized":"2023-11-06T03:14:35.021909+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:40:00.983362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.218103+0000","last_change":"2023-11-06T03:14:35.040735+0000","last_active":"2023-11-06T03:14:37.218103+0000","last_peered":"2023-11-06T03:14:37.218103+0000","last_clean":"2023-11-06T03:14:37.218103+0000","last_became_active":"2023-11-06T03:14:35.040446+0000","last_became_peered":"2023-11-06T03:14:35.040446+0000","last_unstale":"2023-11-06T03:14:37.218103+0000","last_undegraded":"2023-11-06T03:14:37.218103+0000","last_fullsized":"2023-11-06T03:14:37.218103+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:59:21.943273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.012244+0000","last_change":"2023-11-06T03:14:34.662426+0000","last_active":"2023-11-06T03:14:35.012244+0000","last_peered":"2023-11-06T03:14:35.012244+0000","last_clean":"2023-11-06T03:14:35.012244+0000","last_became_active":"2023-11-06T03:14:34.661802+0000","last_became_peered":"2023-11-06T03:14:34.661802+0000","last_unstale":"2023-11-06T03:14:35.012244+0000","last_undegraded":"2023-11-06T03:14:35.012244+0000","last_fullsized":"2023-11-06T03:14:35.012244+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:06:44.199655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594507+0000","last_change":"2023-11-06T03:14:35.032587+0000","last_active":"2023-11-06T03:14:37.594507+0000","last_peered":"2023-11-06T03:14:37.594507+0000","last_clean":"2023-11-06T03:14:37.594507+0000","last_became_active":"2023-11-06T03:14:35.032316+0000","last_became_peered":"2023-11-06T03:14:35.032316+0000","last_unstale":"2023-11-06T03:14:37.594507+0000","last_undegraded":"2023-11-06T03:14:37.594507+0000","last_fullsized":"2023-11-06T03:14:37.594507+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:02:49.915994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.034718+0000","last_change":"2023-11-06T03:14:34.664704+0000","last_active":"2023-11-06T03:14:35.034718+0000","last_peered":"2023-11-06T03:14:35.034718+0000","last_clean":"2023-11-06T03:14:35.034718+0000","last_became_active":"2023-11-06T03:14:34.660345+0000","last_became_peered":"2023-11-06T03:14:34.660345+0000","last_unstale":"2023-11-06T03:14:35.034718+0000","last_undegraded":"2023-11-06T03:14:35.034718+0000","last_fullsized":"2023-11-06T03:14:35.034718+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:46.148170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.595062+0000","last_change":"2023-11-06T03:14:35.021139+0000","last_active":"2023-11-06T03:14:37.595062+0000","last_peered":"2023-11-06T03:14:37.595062+0000","last_clean":"2023-11-06T03:14:37.595062+0000","last_became_active":"2023-11-06T03:14:35.020871+0000","last_became_peered":"2023-11-06T03:14:35.020871+0000","last_unstale":"2023-11-06T03:14:37.595062+0000","last_undegraded":"2023-11-06T03:14:37.595062+0000","last_fullsized":"2023-11-06T03:14:37.595062+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:08.893032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.045052+0000","last_change":"2023-11-06T03:14:34.678678+0000","last_active":"2023-11-06T03:14:35.045052+0000","last_peered":"2023-11-06T03:14:35.045052+0000","last_clean":"2023-11-06T03:14:35.045052+0000","last_became_active":"2023-11-06T03:14:34.677921+0000","last_became_peered":"2023-11-06T03:14:34.677921+0000","last_unstale":"2023-11-06T03:14:35.045052+0000","last_undegraded":"2023-11-06T03:14:35.045052+0000","last_fullsized":"2023-11-06T03:14:35.045052+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:51.235823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594441+0000","last_change":"2023-11-06T03:14:35.042044+0000","last_active":"2023-11-06T03:14:37.594441+0000","last_peered":"2023-11-06T03:14:37.594441+0000","last_clean":"2023-11-06T03:14:37.594441+0000","last_became_active":"2023-11-06T03:14:35.041769+0000","last_became_peered":"2023-11-06T03:14:35.041769+0000","last_unstale":"2023-11-06T03:14:37.594441+0000","last_undegraded":"2023-11-06T03:14:37.594441+0000","last_fullsized":"2023-11-06T03:14:37.594441+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:05.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.012108+0000","last_change":"2023-11-06T03:14:34.665818+0000","last_active":"2023-11-06T03:14:35.012108+0000","last_peered":"2023-11-06T03:14:35.012108+0000","last_clean":"2023-11-06T03:14:35.012108+0000","last_became_active":"2023-11-06T03:14:34.665571+0000","last_became_peered":"2023-11-06T03:14:34.665571+0000","last_unstale":"2023-11-06T03:14:35.012108+0000","last_undegraded":"2023-11-06T03:14:35.012108+0000","last_fullsized":"2023-11-06T03:14:35.012108+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:37.138948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539682+0000","last_change":"2023-11-06T03:14:35.084457+0000","last_active":"2023-11-06T03:14:36.539682+0000","last_peered":"2023-11-06T03:14:36.539682+0000","last_clean":"2023-11-06T03:14:36.539682+0000","last_became_active":"2023-11-06T03:14:35.084100+0000","last_became_peered":"2023-11-06T03:14:35.084100+0000","last_unstale":"2023-11-06T03:14:36.539682+0000","last_undegraded":"2023-11-06T03:14:36.539682+0000","last_fullsized":"2023-11-06T03:14:36.539682+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:25.397606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.026771+0000","last_change":"2023-11-06T03:14:34.665953+0000","last_active":"2023-11-06T03:14:35.026771+0000","last_peered":"2023-11-06T03:14:35.026771+0000","last_clean":"2023-11-06T03:14:35.026771+0000","last_became_active":"2023-11-06T03:14:34.665475+0000","last_became_peered":"2023-11-06T03:14:34.665475+0000","last_unstale":"2023-11-06T03:14:35.026771+0000","last_undegraded":"2023-11-06T03:14:35.026771+0000","last_fullsized":"2023-11-06T03:14:35.026771+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:06:42.016610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148623+0000","last_change":"2023-11-06T03:14:35.059073+0000","last_active":"2023-11-06T03:14:36.148623+0000","last_peered":"2023-11-06T03:14:36.148623+0000","last_clean":"2023-11-06T03:14:36.148623+0000","last_became_active":"2023-11-06T03:14:35.058363+0000","last_became_peered":"2023-11-06T03:14:35.058363+0000","last_unstale":"2023-11-06T03:14:36.148623+0000","last_undegraded":"2023-11-06T03:14:36.148623+0000","last_fullsized":"2023-11-06T03:14:36.148623+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:47:50.759932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.042911+0000","last_change":"2023-11-06T03:14:34.680891+0000","last_active":"2023-11-06T03:14:35.042911+0000","last_peered":"2023-11-06T03:14:35.042911+0000","last_clean":"2023-11-06T03:14:35.042911+0000","last_became_active":"2023-11-06T03:14:34.680233+0000","last_became_peered":"2023-11-06T03:14:34.680233+0000","last_unstale":"2023-11-06T03:14:35.042911+0000","last_undegraded":"2023-11-06T03:14:35.042911+0000","last_fullsized":"2023-11-06T03:14:35.042911+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:22.459648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.041288+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:14:35.041288+0000","last_peered":"2023-11-06T03:14:35.041288+0000","last_clean":"2023-11-06T03:14:35.041288+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:14:35.041288+0000","last_undegraded":"2023-11-06T03:14:35.041288+0000","last_fullsized":"2023-11-06T03:14:35.041288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":105,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.034929+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:14:35.034929+0000","last_peered":"2023-11-06T03:14:35.034929+0000","last_clean":"2023-11-06T03:14:35.034929+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:14:35.034929+0000","last_undegraded":"2023-11-06T03:14:35.034929+0000","last_fullsized":"2023-11-06T03:14:35.034929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.539702+0000","last_change":"2023-11-06T03:14:35.072903+0000","last_active":"2023-11-06T03:14:36.539702+0000","last_peered":"2023-11-06T03:14:36.539702+0000","last_clean":"2023-11-06T03:14:36.539702+0000","last_became_active":"2023-11-06T03:14:35.072186+0000","last_became_peered":"2023-11-06T03:14:35.072186+0000","last_unstale":"2023-11-06T03:14:36.539702+0000","last_undegraded":"2023-11-06T03:14:36.539702+0000","last_fullsized":"2023-11-06T03:14:36.539702+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.452944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.038363+0000","last_change":"2023-11-06T03:14:34.653580+0000","last_active":"2023-11-06T03:14:35.038363+0000","last_peered":"2023-11-06T03:14:35.038363+0000","last_clean":"2023-11-06T03:14:35.038363+0000","last_became_active":"2023-11-06T03:14:34.653146+0000","last_became_peered":"2023-11-06T03:14:34.653146+0000","last_unstale":"2023-11-06T03:14:35.038363+0000","last_undegraded":"2023-11-06T03:14:35.038363+0000","last_fullsized":"2023-11-06T03:14:35.038363+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:02:31.735613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.041042+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:14:35.041042+0000","last_peered":"2023-11-06T03:14:35.041042+0000","last_clean":"2023-11-06T03:14:35.041042+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:14:35.041042+0000","last_undegraded":"2023-11-06T03:14:35.041042+0000","last_fullsized":"2023-11-06T03:14:35.041042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.023751+0000","last_change":"2023-11-06T03:14:35.036142+0000","last_active":"2023-11-06T03:14:36.023751+0000","last_peered":"2023-11-06T03:14:36.023751+0000","last_clean":"2023-11-06T03:14:36.023751+0000","last_became_active":"2023-11-06T03:14:35.035864+0000","last_became_peered":"2023-11-06T03:14:35.035864+0000","last_unstale":"2023-11-06T03:14:36.023751+0000","last_undegraded":"2023-11-06T03:14:36.023751+0000","last_fullsized":"2023-11-06T03:14:36.023751+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:00:02.452944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"136'8","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.060626+0000","last_change":"2023-11-06T03:14:34.666104+0000","last_active":"2023-11-06T03:14:35.060626+0000","last_peered":"2023-11-06T03:14:35.060626+0000","last_clean":"2023-11-06T03:14:35.060626+0000","last_became_active":"2023-11-06T03:14:34.665490+0000","last_became_peered":"2023-11-06T03:14:34.665490+0000","last_unstale":"2023-11-06T03:14:35.060626+0000","last_undegraded":"2023-11-06T03:14:35.060626+0000","last_fullsized":"2023-11-06T03:14:35.060626+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:22.459648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.018979+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:14:35.018979+0000","last_peered":"2023-11-06T03:14:35.018979+0000","last_clean":"2023-11-06T03:14:35.018979+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:14:35.018979+0000","last_undegraded":"2023-11-06T03:14:35.018979+0000","last_fullsized":"2023-11-06T03:14:35.018979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.218163+0000","last_change":"2023-11-06T03:14:35.030797+0000","last_active":"2023-11-06T03:14:37.218163+0000","last_peered":"2023-11-06T03:14:37.218163+0000","last_clean":"2023-11-06T03:14:37.218163+0000","last_became_active":"2023-11-06T03:14:35.030409+0000","last_became_peered":"2023-11-06T03:14:35.030409+0000","last_unstale":"2023-11-06T03:14:37.218163+0000","last_undegraded":"2023-11-06T03:14:37.218163+0000","last_fullsized":"2023-11-06T03:14:37.218163+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:01:59.026953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"136'2","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.029954+0000","last_change":"2023-11-06T03:14:34.662390+0000","last_active":"2023-11-06T03:14:35.029954+0000","last_peered":"2023-11-06T03:14:35.029954+0000","last_clean":"2023-11-06T03:14:35.029954+0000","last_became_active":"2023-11-06T03:14:34.662071+0000","last_became_peered":"2023-11-06T03:14:34.662071+0000","last_unstale":"2023-11-06T03:14:35.029954+0000","last_undegraded":"2023-11-06T03:14:35.029954+0000","last_fullsized":"2023-11-06T03:14:35.029954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:03:20.305844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":105,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.028731+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:14:35.028731+0000","last_peered":"2023-11-06T03:14:35.028731+0000","last_clean":"2023-11-06T03:14:35.028731+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:14:35.028731+0000","last_undegraded":"2023-11-06T03:14:35.028731+0000","last_fullsized":"2023-11-06T03:14:35.028731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148467+0000","last_change":"2023-11-06T03:14:35.073923+0000","last_active":"2023-11-06T03:14:36.148467+0000","last_peered":"2023-11-06T03:14:36.148467+0000","last_clean":"2023-11-06T03:14:36.148467+0000","last_became_active":"2023-11-06T03:14:35.073375+0000","last_became_peered":"2023-11-06T03:14:35.073375+0000","last_unstale":"2023-11-06T03:14:36.148467+0000","last_undegraded":"2023-11-06T03:14:36.148467+0000","last_fullsized":"2023-11-06T03:14:36.148467+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:47:45.246712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"136'3","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.084137+0000","last_change":"2023-11-06T03:14:34.679671+0000","last_active":"2023-11-06T03:14:35.084137+0000","last_peered":"2023-11-06T03:14:35.084137+0000","last_clean":"2023-11-06T03:14:35.084137+0000","last_became_active":"2023-11-06T03:14:34.679105+0000","last_became_peered":"2023-11-06T03:14:34.679105+0000","last_unstale":"2023-11-06T03:14:35.084137+0000","last_undegraded":"2023-11-06T03:14:35.084137+0000","last_fullsized":"2023-11-06T03:14:35.084137+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:07:55.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":113,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.012357+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:14:35.012357+0000","last_peered":"2023-11-06T03:14:35.012357+0000","last_clean":"2023-11-06T03:14:35.012357+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:14:35.012357+0000","last_undegraded":"2023-11-06T03:14:35.012357+0000","last_fullsized":"2023-11-06T03:14:35.012357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:36.148839+0000","last_change":"2023-11-06T03:14:35.073997+0000","last_active":"2023-11-06T03:14:36.148839+0000","last_peered":"2023-11-06T03:14:36.148839+0000","last_clean":"2023-11-06T03:14:36.148839+0000","last_became_active":"2023-11-06T03:14:35.073353+0000","last_became_peered":"2023-11-06T03:14:35.073353+0000","last_unstale":"2023-11-06T03:14:36.148839+0000","last_undegraded":"2023-11-06T03:14:36.148839+0000","last_fullsized":"2023-11-06T03:14:36.148839+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:11:47.160839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"136'6","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.085588+0000","last_change":"2023-11-06T03:14:34.679002+0000","last_active":"2023-11-06T03:14:35.085588+0000","last_peered":"2023-11-06T03:14:35.085588+0000","last_clean":"2023-11-06T03:14:35.085588+0000","last_became_active":"2023-11-06T03:14:34.678473+0000","last_became_peered":"2023-11-06T03:14:34.678473+0000","last_unstale":"2023-11-06T03:14:35.085588+0000","last_undegraded":"2023-11-06T03:14:35.085588+0000","last_fullsized":"2023-11-06T03:14:35.085588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:06:37.138948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.046994+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:14:35.046994+0000","last_peered":"2023-11-06T03:14:35.046994+0000","last_clean":"2023-11-06T03:14:35.046994+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:14:35.046994+0000","last_undegraded":"2023-11-06T03:14:35.046994+0000","last_fullsized":"2023-11-06T03:14:35.046994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.847903+0000","last_change":"2023-11-06T03:14:35.059075+0000","last_active":"2023-11-06T03:14:37.847903+0000","last_peered":"2023-11-06T03:14:37.847903+0000","last_clean":"2023-11-06T03:14:37.847903+0000","last_became_active":"2023-11-06T03:14:35.058802+0000","last_became_peered":"2023-11-06T03:14:35.058802+0000","last_unstale":"2023-11-06T03:14:37.847903+0000","last_undegraded":"2023-11-06T03:14:37.847903+0000","last_fullsized":"2023-11-06T03:14:37.847903+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:26:58.408138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.037126+0000","last_change":"2023-11-06T03:14:34.652830+0000","last_active":"2023-11-06T03:14:35.037126+0000","last_peered":"2023-11-06T03:14:35.037126+0000","last_clean":"2023-11-06T03:14:35.037126+0000","last_became_active":"2023-11-06T03:14:34.652088+0000","last_became_peered":"2023-11-06T03:14:34.652088+0000","last_unstale":"2023-11-06T03:14:35.037126+0000","last_undegraded":"2023-11-06T03:14:35.037126+0000","last_fullsized":"2023-11-06T03:14:35.037126+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:47:07.237326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.012928+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:14:35.012928+0000","last_peered":"2023-11-06T03:14:35.012928+0000","last_clean":"2023-11-06T03:14:35.012928+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:14:35.012928+0000","last_undegraded":"2023-11-06T03:14:35.012928+0000","last_fullsized":"2023-11-06T03:14:35.012928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-11-06T03:14:37.594203+0000","last_change":"2023-11-06T03:14:35.027734+0000","last_active":"2023-11-06T03:14:37.594203+0000","last_peered":"2023-11-06T03:14:37.594203+0000","last_clean":"2023-11-06T03:14:37.594203+0000","last_became_active":"2023-11-06T03:14:35.027285+0000","last_became_peered":"2023-11-06T03:14:35.027285+0000","last_unstale":"2023-11-06T03:14:37.594203+0000","last_undegraded":"2023-11-06T03:14:37.594203+0000","last_fullsized":"2023-11-06T03:14:37.594203+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:34.623868+0000","last_clean_scrub_stamp":"2023-11-06T03:14:34.623868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:49:14.774782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.025532+0000","last_change":"2023-11-06T03:14:34.662531+0000","last_active":"2023-11-06T03:14:35.025532+0000","last_peered":"2023-11-06T03:14:35.025532+0000","last_clean":"2023-11-06T03:14:35.025532+0000","last_became_active":"2023-11-06T03:14:34.662238+0000","last_became_peered":"2023-11-06T03:14:34.662238+0000","last_unstale":"2023-11-06T03:14:35.025532+0000","last_undegraded":"2023-11-06T03:14:35.025532+0000","last_fullsized":"2023-11-06T03:14:35.025532+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:14:33.611199+0000","last_clean_scrub_stamp":"2023-11-06T03:14:33.611199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:30:17.547269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-11-06T03:14:35.039691+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:14:35.039691+0000","last_peered":"2023-11-06T03:14:35.039691+0000","last_clean":"2023-11-06T03:14:35.039691+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:14:35.039691+0000","last_undegraded":"2023-11-06T03:14:35.039691+0000","last_fullsized":"2023-11-06T03:14:35.039691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87392,"kb_used_data":2264,"kb_used_omap":0,"kb_used_meta":85120,"kb_avail":94284448,"statfs":{"total":96636764160,"available":96547274752,"internally_reserved":0,"allocated":2318336,"data_stored":1381722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87162880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.39400000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.4830000000000001},"last":1.109},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97799999999999998,"5min":1.3620000000000001,"15min":1.397},"last":0.74199999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.86499999999999999,"15min":0.88100000000000001},"min":{"1min":0.54400000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1890000000000001,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.2170000000000001,"5min":1.288,"15min":1.6080000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.879,"15min":0.91600000000000004},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.151,"5min":1.3240000000000001,"15min":1.6259999999999999},"last":1.036}]},{"osd":3,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.89300000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":2.512},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95999999999999996,"15min":1.0409999999999999},"min":{"1min":0.63100000000000001,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.139,"5min":1.381,"15min":10.596},"last":1.431}]},{"osd":4,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.96899999999999997,"15min":0.96399999999999997},"min":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.2509999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.71399999999999997,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.90900000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.036,"15min":1.044},"min":{"1min":0.69399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2929999999999999,"5min":1.4339999999999999,"15min":1.9139999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1,"5min":1.081,"15min":1.0840000000000001},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.55200000000000005},"max":{"1min":1.1899999999999999,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":1.1459999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.232,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":9.6159999999999997,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.25},{"interface":"front","average":{"1min":0.999,"5min":1.0660000000000001,"15min":1.097},"min":{"1min":0.754,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.55,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":1.2290000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64656,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94307184,"statfs":{"total":96636764160,"available":96570556416,"internally_reserved":0,"allocated":1712128,"data_stored":789353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80100000000000005,"15min":0.85199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":2.2429999999999999,"5min":2.3279999999999998,"15min":16.073},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83299999999999996,"15min":0.97499999999999998},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":2.1850000000000001,"5min":4.4000000000000004,"15min":40.905000000000001},"last":1.1930000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83299999999999996,"15min":0.90100000000000002},"min":{"1min":0.53500000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.351,"5min":1.6359999999999999,"15min":16.012},"last":1.135},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.51800000000000002,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.081,"5min":1.409,"15min":3.3050000000000002},"last":1.0740000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.82999999999999996,"15min":0.91500000000000004},"min":{"1min":0.56599999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.702,"5min":1.744,"15min":27.803999999999998},"last":1.101},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.92600000000000005,"15min":0.94899999999999995},"min":{"1min":0.60099999999999998,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.429,"5min":1.542,"15min":2.843},"last":1.028}]},{"osd":3,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.94099999999999995,"15min":1.1799999999999999},"min":{"1min":0.61699999999999999,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.264,"5min":1.306,"15min":63.737000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.91900000000000004,"15min":1.056},"min":{"1min":0.57099999999999995,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":41.079999999999998},"last":1.163}]},{"osd":4,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0960000000000001,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.58699999999999997},"max":{"1min":1.502,"5min":7.4980000000000002,"15min":24.867999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1639999999999999,"15min":1.2330000000000001},"min":{"1min":0.93000000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4530000000000001,"5min":6.3970000000000002,"15min":23.789000000000001},"last":0.98999999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.98799999999999999,"15min":1.157},"min":{"1min":0.73499999999999999,"5min":0.64400000000000002,"15min":0.51800000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.103,"15min":1.115},"min":{"1min":0.79200000000000004,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.5620000000000001,"5min":1.6160000000000001,"15min":4.1159999999999997},"last":1.2889999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:14:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.1870000000000001,"15min":1.2150000000000001},"min":{"1min":0.81299999999999994,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":4.2389999999999999,"5min":4.6399999999999997,"15min":23.748999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.1359999999999999,"15min":1.2230000000000001},"min":{"1min":0.89400000000000002,"5min":0.76600000000000001,"15min":0.56499999999999995},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.264}]}]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308784,"statfs":{"total":96636764160,"available":96572194816,"internally_reserved":0,"allocated":1712128,"data_stored":789722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.804,"15min":1.1379999999999999},"min":{"1min":0.53300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":4.0510000000000002,"5min":125.256,"15min":125.256},"last":0.63900000000000001},{"interface":"front","average":{"1min":2.222,"5min":2.1059999999999999,"15min":1.2509999999999999},"min":{"1min":0.57399999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":26.518000000000001,"5min":126.98699999999999,"15min":126.98699999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.89700000000000002,"15min":0.80400000000000005},"min":{"1min":0.59699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.6390000000000002,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":3.1579999999999999,"5min":1.284,"15min":0.997},"min":{"1min":0.53600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":43.799999999999997,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.74099999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":3.274,"5min":1.3360000000000001,"15min":1.008},"min":{"1min":0.626,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":43.741,"5min":43.741,"15min":43.741},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.85799999999999998,"15min":0.84299999999999997},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.385,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.82999999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.92400000000000004,"15min":0.875},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.349,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":3.3159999999999998,"5min":1.4330000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":43.841999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.048}]},{"osd":4,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.91900000000000004,"15min":0.91600000000000004},"min":{"1min":0.47399999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.5,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.121,"15min":1.012},"min":{"1min":0.70299999999999996,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":4.4020000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.014}]},{"osd":6,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0489999999999999,"15min":1.016},"min":{"1min":0.501,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":2,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.113,"15min":1.042},"min":{"1min":0.58099999999999996,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":4.1230000000000002,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1180000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:14:16 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.155,"15min":1.0509999999999999},"min":{"1min":0.53300000000000003,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.145,"15min":1.1060000000000001},"min":{"1min":0.66500000000000004,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.236}]}]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90892,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280948,"statfs":{"total":96636764160,"available":96543690752,"internally_reserved":0,"allocated":1708032,"data_stored":789700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":1.25,"15min":0.96099999999999997},"min":{"1min":0.496,"5min":0.436,"15min":0.372},"max":{"1min":1.4319999999999999,"5min":42.853999999999999,"15min":42.853999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.2869999999999999,"15min":0.95699999999999996},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.3100000000000001,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.73599999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77600000000000002,"15min":0.76900000000000002},"min":{"1min":0.47799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.85799999999999998,"15min":0.83399999999999996},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.639,"5min":1.639,"15min":1.6719999999999999},"last":1.151}]},{"osd":2,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88800000000000001,"15min":0.96799999999999997},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.53300000000000003},"max":{"1min":1.2230000000000001,"5min":1.452,"15min":3.2879999999999998},"last":1.056},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.876,"15min":0.86199999999999999},"min":{"1min":0.51200000000000001,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.262,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.0169999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90500000000000003,"15min":0.92200000000000004},"min":{"1min":0.58499999999999996,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.3779999999999999,"5min":3.198,"15min":3.198},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89300000000000002,"15min":0.89700000000000002},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.288,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0800000000000001,"15min":1.03},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.5429999999999999,"5min":10.192,"15min":10.192},"last":1.238},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0740000000000001,"15min":1.0880000000000001},"min":{"1min":0.70299999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.415,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.0960000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.97699999999999998,"15min":0.998},"min":{"1min":0.64200000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3640000000000001,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":1.54},{"interface":"front","average":{"1min":1.119,"5min":1.0569999999999999,"15min":1.109},"min":{"1min":0.72399999999999998,"5min":0.65300000000000002,"15min":0.57799999999999996},"max":{"1min":1.512,"5min":1.6200000000000001,"15min":2.6549999999999998},"last":1.3109999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.089,"15min":1.077},"min":{"1min":0.66000000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5289999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.109,"15min":1.171},"min":{"1min":0.745,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.526,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.1879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81112,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":79424,"kb_avail":94290728,"statfs":{"total":96636764160,"available":96553705472,"internally_reserved":0,"allocated":1720320,"data_stored":790315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":81330176},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54800000000000004,"15min":0.59599999999999997},"min":{"1min":0.23499999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.93600000000000005,"5min":1.113,"15min":1.4199999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.27800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.80700000000000005,"5min":0.85399999999999998,"15min":1.524},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.251,"5min":0.224,"15min":0.224},"max":{"1min":0.754,"5min":1.3420000000000001,"15min":1.371},"last":0.624},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.29999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.96999999999999997,"5min":1.2050000000000001,"15min":1.4099999999999999},"last":0.73699999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.69699999999999995,"15min":0.68000000000000005},"min":{"1min":0.32400000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.95899999999999996,"5min":1.532,"15min":1.532},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.255},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.284},"last":0.79900000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.83499999999999996,"15min":0.876},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0580000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89400000000000002,"15min":0.92600000000000005},"min":{"1min":0.49299999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3129999999999999,"5min":1.5309999999999999,"15min":1.8300000000000001},"last":0.92600000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.95199999999999996,"15min":0.96699999999999997},"min":{"1min":0.53900000000000003,"5min":0.5,"15min":0.46899999999999997},"max":{"1min":1.2529999999999999,"5min":1.5,"15min":1.6639999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95399999999999996,"15min":0.99299999999999999},"min":{"1min":0.504,"5min":0.49399999999999999,"15min":0.35999999999999999},"max":{"1min":1.169,"5min":1.573,"15min":1.698},"last":0.88700000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.95699999999999996,"15min":0.98699999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.26,"5min":1.649,"15min":1.8620000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.93600000000000005,"15min":0.96999999999999997},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3120000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0760000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":1.018},"min":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.194,"5min":1.835,"15min":1.835},"last":1.125},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.014,"15min":1.0409999999999999},"min":{"1min":0.53900000000000003,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":1.359,"5min":1.734,"15min":1.8979999999999999},"last":1.202}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72852,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298988,"statfs":{"total":96636764160,"available":96562163712,"internally_reserved":0,"allocated":1716224,"data_stored":790316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.51000000000000001,"15min":0.53900000000000003},"min":{"1min":0.23200000000000001,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.751,"5min":0.93400000000000005,"15min":1.7270000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53600000000000003,"15min":0.60199999999999998},"min":{"1min":0.27400000000000002,"5min":0.191,"15min":0.191},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.41199999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57899999999999996,"15min":0.623},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.88,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.57599999999999996,"15min":0.63100000000000001},"min":{"1min":0.311,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":1.099,"15min":1.1579999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.66200000000000003},"min":{"1min":0.34899999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.087,"5min":1.161,"15min":1.294},"last":0.504},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.63400000000000001,"15min":0.70399999999999996},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.2010000000000001,"5min":1.244,"15min":1.526},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.90500000000000003},"min":{"1min":0.55000000000000004,"5min":0.439,"15min":0.41399999999999998},"max":{"1min":1.2809999999999999,"5min":1.329,"15min":1.5229999999999999},"last":0.872}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83199999999999996,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.872,"15min":0.93000000000000005},"min":{"1min":0.496,"5min":0.44400000000000001,"15min":0.35999999999999999},"max":{"1min":1.355,"5min":1.4359999999999999,"15min":1.5780000000000001},"last":0.84799999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93799999999999994,"15min":0.97499999999999998},"min":{"1min":0.54100000000000004,"5min":0.504,"15min":0.46600000000000003},"max":{"1min":1.542,"5min":1.542,"15min":1.615},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.92100000000000004,"15min":0.93899999999999995},"min":{"1min":0.68999999999999995,"5min":0.5,"15min":0.5},"max":{"1min":1.091,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89800000000000002,"15min":0.96699999999999997},"min":{"1min":0.50900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.90000000000000002,"15min":0.93600000000000005},"min":{"1min":0.60999999999999999,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50768,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321072,"statfs":{"total":96636764160,"available":96584777728,"internally_reserved":0,"allocated":1712128,"data_stored":789770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.23999999999999999,"5min":0.19700000000000001,"15min":0.19},"max":{"1min":1.006,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.20399999999999999},"max":{"1min":0.79600000000000004,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.244}]},{"osd":2,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55400000000000005,"15min":0.69599999999999995},"min":{"1min":0.27200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.83699999999999997,"5min":1.169,"15min":30.861000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59799999999999998,"15min":0.64700000000000002},"min":{"1min":0.30599999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.874,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.64000000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65600000000000003,"15min":0.69899999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.1240000000000001,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63700000000000001,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.129,"5min":1.151,"15min":2.3079999999999998},"last":1.3979999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.85399999999999998},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.434},"max":{"1min":1.0800000000000001,"5min":2.0870000000000002,"15min":2.258},"last":1.133},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.5549999999999999,"15min":14.753},"last":1.107}]},{"osd":5,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":1.089},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.173,"5min":1.528,"15min":43.103999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.93100000000000005,"15min":1.1220000000000001},"min":{"1min":0.64600000000000002,"5min":0.47799999999999998,"15min":0.39100000000000001},"max":{"1min":1.274,"5min":1.3979999999999999,"15min":44.753999999999998},"last":1.1930000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.89100000000000001,"15min":1.0309999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":23.971},"last":1.323},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":1.0469999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1950000000000001,"5min":1.444,"15min":14.813000000000001},"last":1.262}]},{"osd":7,"last update":"Mon Nov 6 03:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97599999999999998,"15min":1.028},"min":{"1min":0.65800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.581,"5min":1.581,"15min":2.2799999999999998},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96599999999999997,"15min":1.016},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3680000000000001,"5min":2.0499999999999998,"15min":2.3559999999999999},"last":1.835}]}]},{"osd":0,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95636,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94276204,"statfs":{"total":96636764160,"available":96538832896,"internally_reserved":0,"allocated":2306048,"data_stored":1379105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57399999999999995},"min":{"1min":0.28199999999999997,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.91800000000000004,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.317},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.60399999999999998,"15min":0.64300000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.747,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.35699999999999998,"5min":0.22800000000000001,"15min":0.20599999999999999},"max":{"1min":0.997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.64500000000000002},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1779999999999999,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.42899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.95599999999999996,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71199999999999997,"15min":0.72099999999999997},"min":{"1min":0.33400000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.099,"5min":1.391,"15min":2.7999999999999998},"last":0.439}]},{"osd":4,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.262,"5min":1.371,"15min":4.641},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.83099999999999996,"15min":0.89000000000000001},"min":{"1min":0.51300000000000001,"5min":0.46899999999999997,"15min":0.42899999999999999},"max":{"1min":1.0860000000000001,"5min":1.254,"15min":2.028},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.91800000000000004,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.2929999999999999,"5min":1.7190000000000001,"15min":2.8559999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89800000000000002,"15min":0.93700000000000006},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.401,"5min":1.4219999999999999,"15min":4.7430000000000003},"last":0.749}]},{"osd":6,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.88100000000000001,"15min":0.89700000000000002},"min":{"1min":0.503,"5min":0.497,"15min":0.44800000000000001},"max":{"1min":1.1559999999999999,"5min":1.2609999999999999,"15min":2.931},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.97699999999999998,"15min":0.998},"min":{"1min":0.60199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2969999999999999,"5min":1.802,"15min":2.7789999999999999},"last":0.56799999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93899999999999995,"15min":0.98899999999999999},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.3520000000000001,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":0.627},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":0.97499999999999998},"min":{"1min":0.622,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3320000000000001,"5min":1.4670000000000001,"15min":1.97},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:14:39.439 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:14:39.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:14:39.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 Processor -- start 2023-11-06T03:14:39.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 -- start start 2023-11-06T03:14:39.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f24e41551e0 0x7f24e406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e415e5e0 0x7f24e40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415ea90 0x7f24e40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e415f580 con 0x7f24e415e5e0 2023-11-06T03:14:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e415efc0 con 0x7f24e41551e0 2023-11-06T03:14:39.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e9432640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e415f2a0 con 0x7f24e415ea90 2023-11-06T03:14:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e27fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e415e5e0 0x7f24e40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e27fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f24e415e5e0 0x7f24e40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46366/0 (socket says 172.21.15.157:46366) 2023-11-06T03:14:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e27fc640 1 -- 172.21.15.157:0/1743295520 learned_addr learned my addr 172.21.15.157:0/1743295520 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e2ffd640 1 --2- 172.21.15.157:0/1743295520 >> v2:172.21.15.195:3300/0 conn(0x7f24e41551e0 0x7f24e406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.608+0000 7f24e37fe640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415ea90 0x7f24e40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 -- 172.21.15.157:0/1743295520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415ea90 msgr2=0x7f24e40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415ea90 0x7f24e40b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 -- 172.21.15.157:0/1743295520 >> v2:172.21.15.195:3300/0 conn(0x7f24e41551e0 msgr2=0x7f24e406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:39.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 --2- 172.21.15.157:0/1743295520 >> v2:172.21.15.195:3300/0 conn(0x7f24e41551e0 0x7f24e406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 -- 172.21.15.157:0/1743295520 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e406f5a0 con 0x7f24e415e5e0 2023-11-06T03:14:39.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e27fc640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415e5e0 0x7f24e40b3410 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f24e4066830 tx=0x7f24d401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a034adecc43cfd62 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e1ffb640 1 -- 172.21.15.157:0/1743295520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40ab620 con 0x7f24e415e5e0 2023-11-06T03:14:39.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e1ffb640 1 -- 172.21.15.157:0/1743295520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40ab780 con 0x7f24e415e5e0 2023-11-06T03:14:39.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e1ffb640 1 -- 172.21.15.157:0/1743295520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40abb10 con 0x7f24e415e5e0 2023-11-06T03:14:39.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 -- 172.21.15.157:0/1743295520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415e5e0 msgr2=0x7f24e40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415e5e0 0x7f24e40b3410 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f24e4066830 tx=0x7f24d401c440 comp rx=0 tx=0).stop 2023-11-06T03:14:39.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 -- 172.21.15.157:0/1743295520 shutdown_connections 2023-11-06T03:14:39.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415ea90 0x7f24e40b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 --2- 172.21.15.157:0/1743295520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415e5e0 0x7f24e40b3410 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 --2- 172.21.15.157:0/1743295520 >> v2:172.21.15.195:3300/0 conn(0x7f24e41551e0 0x7f24e406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 -- 172.21.15.157:0/1743295520 >> 172.21.15.157:0/1743295520 conn(0x7f24e4065930 msgr2=0x7f24e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 -- 172.21.15.157:0/1743295520 shutdown_connections 2023-11-06T03:14:39.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.612+0000 7f24e9432640 1 -- 172.21.15.157:0/1743295520 wait complete. 2023-11-06T03:14:39.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 Processor -- start 2023-11-06T03:14:39.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- start start 2023-11-06T03:14:39.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 0x7f24e4155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415e5e0 0x7f24e41555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4155b00 con 0x7f24e415ea90 2023-11-06T03:14:39.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4156490 con 0x7f24e41551e0 2023-11-06T03:14:39.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40cdb60 con 0x7f24e415e5e0 2023-11-06T03:14:39.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46378/0 (socket says 172.21.15.157:46378) 2023-11-06T03:14:39.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 -- 172.21.15.157:0/3777637311 learned_addr learned my addr 172.21.15.157:0/3777637311 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e2ffd640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 0x7f24e4155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e27fc640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415e5e0 0x7f24e41555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 -- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415e5e0 msgr2=0x7f24e41555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415e5e0 0x7f24e41555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 -- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 msgr2=0x7f24e4155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 0x7f24e4155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 -- 172.21.15.157:0/3777637311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40094d0 con 0x7f24e415ea90 2023-11-06T03:14:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e2ffd640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 0x7f24e4155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:14:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e37fe640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00d560 tx=0x7f24cc00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0125f0 con 0x7f24e415ea90 2023-11-06T03:14:39.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc012770 con 0x7f24e415ea90 2023-11-06T03:14:39.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e40cde40 con 0x7f24e415ea90 2023-11-06T03:14:39.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc012b20 con 0x7f24e415ea90 2023-11-06T03:14:39.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4156770 con 0x7f24e415ea90 2023-11-06T03:14:39.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40ce360 con 0x7f24e415ea90 2023-11-06T03:14:39.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.616+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f24cc012c80 con 0x7f24e415ea90 2023-11-06T03:14:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.620+0000 7f24db7fe640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 0x7f24c4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.620+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24cc0836d0 con 0x7f24e415ea90 2023-11-06T03:14:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.620+0000 7f24e2ffd640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 0x7f24c4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.620+0000 7f24e2ffd640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 0x7f24c4067f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003fd0 tx=0x7f24dc003a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.624+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f24e40ce360 con 0x7f24e415ea90 2023-11-06T03:14:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.824+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f24e40ce360 con 0x7f24e415ea90 2023-11-06T03:14:39.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.832+0000 7f24db7fe640 1 -- 172.21.15.157:0/3777637311 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f24e40ce360 con 0x7f24e415ea90 2023-11-06T03:14:39.831 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:14:39.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 msgr2=0x7f24c4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 0x7f24c4067f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f24dc003fd0 tx=0x7f24dc003a00 comp rx=0 tx=0).stop 2023-11-06T03:14:39.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 msgr2=0x7f24e413a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00d560 tx=0x7f24cc00da20 comp rx=0 tx=0).stop 2023-11-06T03:14:39.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 shutdown_connections 2023-11-06T03:14:39.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f24c4065a90 0x7f24c4067f50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24e415ea90 0x7f24e413a1f0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f24e415e5e0 0x7f24e41555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 --2- 172.21.15.157:0/3777637311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24e41551e0 0x7f24e4155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 >> 172.21.15.157:0/3777637311 conn(0x7f24e4065930 msgr2=0x7f24e40b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 shutdown_connections 2023-11-06T03:14:39.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.836+0000 7f24e9432640 1 -- 172.21.15.157:0/3777637311 wait complete. 2023-11-06T03:14:39.850 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:39.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.972+0000 7fcd20474640 1 Processor -- start 2023-11-06T03:14:39.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.972+0000 7fcd20474640 1 -- start start 2023-11-06T03:14:39.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd20474640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd180b3b20 0x7fcd1815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd20474640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1815f580 con 0x7fcd181551e0 2023-11-06T03:14:39.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd20474640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1815efc0 con 0x7fcd1815e5e0 2023-11-06T03:14:39.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd20474640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180704b0 con 0x7fcd180b3b20 2023-11-06T03:14:39.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1ea11640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd180b3b20 0x7fcd1815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1e210640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fcd181551e0 0x7fcd1806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51366/0 (socket says 172.21.15.157:51366) 2023-11-06T03:14:39.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1e210640 1 -- 172.21.15.157:0/332254594 learned_addr learned my addr 172.21.15.157:0/332254594 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:39.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1ea11640 1 -- 172.21.15.157:0/332254594 >> v1:172.21.15.195:6789/0 conn(0x7fcd1815e5e0 legacy=0x7fcd180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:39.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1ea11640 1 -- 172.21.15.157:0/332254594 >> v1:172.21.15.157:6789/0 conn(0x7fcd181551e0 legacy=0x7fcd1806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1ea11640 1 -- 172.21.15.157:0/332254594 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1815f2a0 con 0x7fcd180b3b20 2023-11-06T03:14:39.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1ea11640 1 --2- 172.21.15.157:0/332254594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd180b3b20 0x7fcd1815c1b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009d90 tx=0x7fcd1001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad4b13a982ed66b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1d20e640 1 -- 172.21.15.157:0/332254594 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100a5880 con 0x7fcd180b3b20 2023-11-06T03:14:39.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1d20e640 1 -- 172.21.15.157:0/332254594 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd100a59e0 con 0x7fcd180b3b20 2023-11-06T03:14:39.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.976+0000 7fcd1d20e640 1 -- 172.21.15.157:0/332254594 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100a5d70 con 0x7fcd180b3b20 2023-11-06T03:14:39.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/332254594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd180b3b20 msgr2=0x7fcd1815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 --2- 172.21.15.157:0/332254594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd180b3b20 0x7fcd1815c1b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009d90 tx=0x7fcd1001c7c0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/332254594 shutdown_connections 2023-11-06T03:14:39.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 --2- 172.21.15.157:0/332254594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd180b3b20 0x7fcd1815c1b0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/332254594 >> 172.21.15.157:0/332254594 conn(0x7fcd18065930 msgr2=0x7fcd18065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:39.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/332254594 shutdown_connections 2023-11-06T03:14:39.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/332254594 wait complete. 2023-11-06T03:14:39.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 Processor -- start 2023-11-06T03:14:39.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- start start 2023-11-06T03:14:39.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd180b3b20 0x7fcd18152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd1815e5e0 0x7fcd1814f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1814f2c0 con 0x7fcd181551e0 2023-11-06T03:14:39.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1814fb60 con 0x7fcd1815e5e0 2023-11-06T03:14:39.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1814dd70 con 0x7fcd180b3b20 2023-11-06T03:14:39.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46384/0 (socket says 172.21.15.157:46384) 2023-11-06T03:14:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 -- 172.21.15.157:0/2755938765 learned_addr learned my addr 172.21.15.157:0/2755938765 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 -- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd180b3b20 msgr2=0x7fcd18152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd180b3b20 0x7fcd18152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 -- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd1815e5e0 msgr2=0x7fcd1814f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:39.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd1815e5e0 0x7fcd1814f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:39.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 -- 172.21.15.157:0/2755938765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009a70 con 0x7fcd181551e0 2023-11-06T03:14:39.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd1da0f640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800b3e0 tx=0x7fcd0800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011830 con 0x7fcd181551e0 2023-11-06T03:14:39.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd080119b0 con 0x7fcd181551e0 2023-11-06T03:14:39.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1814dfc0 con 0x7fcd181551e0 2023-11-06T03:14:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08011d60 con 0x7fcd181551e0 2023-11-06T03:14:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1814e580 con 0x7fcd181551e0 2023-11-06T03:14:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.980+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18072750 con 0x7fcd181551e0 2023-11-06T03:14:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.984+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcd0801a400 con 0x7fcd181551e0 2023-11-06T03:14:39.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.984+0000 7fcd0effd640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 0x7fccfc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.984+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd08083340 con 0x7fcd181551e0 2023-11-06T03:14:39.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.984+0000 7fcd1e210640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 0x7fccfc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.984+0000 7fcd1e210640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 0x7fccfc067f20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001600 tx=0x7fcd14007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:39.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:39.988+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcd08050c30 con 0x7fcd181551e0 2023-11-06T03:14:40.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.176+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd1814f400 con 0x7fcd181551e0 2023-11-06T03:14:40.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.176+0000 7fcd0effd640 1 -- 172.21.15.157:0/2755938765 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fcd08054250 con 0x7fcd181551e0 2023-11-06T03:14:40.174 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:40.174 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:14:40.174 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:14:40.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 msgr2=0x7fccfc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 0x7fccfc067f20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001600 tx=0x7fcd14007a50 comp rx=0 tx=0).stop 2023-11-06T03:14:40.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 msgr2=0x7fcd1814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800b3e0 tx=0x7fcd0800b8a0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 shutdown_connections 2023-11-06T03:14:40.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fccfc065a60 0x7fccfc067f20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd1815e5e0 0x7fcd1814f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd181551e0 0x7fcd1814ed80 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 --2- 172.21.15.157:0/2755938765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd180b3b20 0x7fcd18152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 >> 172.21.15.157:0/2755938765 conn(0x7fcd18065930 msgr2=0x7fcd1815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 shutdown_connections 2023-11-06T03:14:40.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.180+0000 7fcd20474640 1 -- 172.21.15.157:0/2755938765 wait complete. 2023-11-06T03:14:40.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:14:35.005631+0000', 'modified': '2023-11-06T03:14:38.023366+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9375': {'gid': 9375, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1004239715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1004239715}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1004239715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:14:40.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:14:40.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:14:40.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 Processor -- start 2023-11-06T03:14:40.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 -- start start 2023-11-06T03:14:40.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f86700cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86700cb6e0 con 0x7f867015cb80 2023-11-06T03:14:40.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8670071830 con 0x7f867015cf60 2023-11-06T03:14:40.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8675379640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8670071b10 con 0x7f86700cdef0 2023-11-06T03:14:40.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f867015cb80 0x7f86700cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51380/0 (socket says 172.21.15.157:51380) 2023-11-06T03:14:40.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866effd640 1 -- 172.21.15.157:0/2729291247 learned_addr learned my addr 172.21.15.157:0/2729291247 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:40.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f86700cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172540381 0 0) 0x7f86700cb6e0 con 0x7f867015cb80 2023-11-06T03:14:40.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c003530 con 0x7f867015cb80 2023-11-06T03:14:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31640874 0 0) 0x7f865c003530 con 0x7f867015cb80 2023-11-06T03:14:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 msgr2=0x7f86700cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f86700cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 >> v1:172.21.15.195:6789/0 conn(0x7f867015cf60 legacy=0x7f86700cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670071df0 con 0x7f867015cb80 2023-11-06T03:14:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8660001680 con 0x7f867015cb80 2023-11-06T03:14:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 >> v1:172.21.15.157:6789/0 conn(0x7f867015cb80 legacy=0x7f86700cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f865c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c003f30 0x7f865c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c004d10 0x7f865c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8670071b10 con 0x7f865c004d10 2023-11-06T03:14:40.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c00bb00 con 0x7f865c003f30 2023-11-06T03:14:40.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c00bcc0 con 0x7f86700cdef0 2023-11-06T03:14:40.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c004d10 0x7f865c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866effd640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f865c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f8667fff640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c003f30 0x7f865c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 -- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 msgr2=0x7f865c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700cdef0 0x7f865c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 -- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c003f30 msgr2=0x7f865c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c003f30 0x7f865c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.736+0000 7f866f7fe640 1 -- 172.21.15.157:0/2729291247 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c003530 con 0x7f865c004d10 2023-11-06T03:14:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f866f7fe640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c004d10 0x7f865c005190 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8668009fc0 tx=0x7f866801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292e317d0aadf2ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680a5c70 con 0x7f865c004d10 2023-11-06T03:14:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866800f040 con 0x7f865c004d10 2023-11-06T03:14:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f866e7fc640 1 -- 172.21.15.157:0/2729291247 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680ad440 con 0x7f865c004d10 2023-11-06T03:14:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- 172.21.15.157:0/2729291247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c004d10 msgr2=0x7f865c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 --2- 172.21.15.157:0/2729291247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865c004d10 0x7f865c005190 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f8668009fc0 tx=0x7f866801cc60 comp rx=0 tx=0).stop 2023-11-06T03:14:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f866f7fe640 1 -- 172.21.15.157:0/2729291247 reap_dead start 2023-11-06T03:14:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- 172.21.15.157:0/2729291247 shutdown_connections 2023-11-06T03:14:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- 172.21.15.157:0/2729291247 >> 172.21.15.157:0/2729291247 conn(0x7f867006d560 msgr2=0x7f867006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- 172.21.15.157:0/2729291247 shutdown_connections 2023-11-06T03:14:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- 172.21.15.157:0/2729291247 wait complete. 2023-11-06T03:14:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 Processor -- start 2023-11-06T03:14:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- start start 2023-11-06T03:14:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 0x7f867014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700d0c10 0x7f86700d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700d15d0 con 0x7f867014be70 2023-11-06T03:14:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c8140 con 0x7f865c004d10 2023-11-06T03:14:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8675379640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867014bc80 con 0x7f86700d0c10 2023-11-06T03:14:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.740+0000 7f8667fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46392/0 (socket says 172.21.15.157:46392) 2023-11-06T03:14:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 -- 172.21.15.157:0/1593133242 learned_addr learned my addr 172.21.15.157:0/1593133242 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f866f7fe640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700d0c10 0x7f86700d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f866effd640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 0x7f867014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 -- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700d0c10 msgr2=0x7f86700d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700d0c10 0x7f86700d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 -- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 msgr2=0x7f867014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 0x7f867014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 -- 172.21.15.157:0/1593133242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700b5520 con 0x7f867014be70 2023-11-06T03:14:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f866effd640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 0x7f867014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8667fff640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f865800e540 tx=0x7f865800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 -- 172.21.15.157:0/1593133242 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580052e0 con 0x7f867014be70 2023-11-06T03:14:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 -- 172.21.15.157:0/1593133242 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658005440 con 0x7f867014be70 2023-11-06T03:14:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86700b57a0 con 0x7f867014be70 2023-11-06T03:14:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 -- 172.21.15.157:0/1593133242 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658003820 con 0x7f867014be70 2023-11-06T03:14:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700c7af0 con 0x7f867014be70 2023-11-06T03:14:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 -- 172.21.15.157:0/1593133242 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8658003980 con 0x7f867014be70 2023-11-06T03:14:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 0x7f8650067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86677fe640 1 -- 172.21.15.157:0/1593133242 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8658083140 con 0x7f867014be70 2023-11-06T03:14:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f866effd640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 0x7f8650067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86784a1000 1 Processor -- start 2023-11-06T03:14:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.744+0000 7f86784a1000 1 -- start start 2023-11-06T03:14:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 0x7f8650067f20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8660001210 tx=0x7f8660001160 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845952c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x55845952ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584593f4fc0 con 0x5584595766e0 2023-11-06T03:14:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584593831a0 con 0x55845952ca90 2023-11-06T03:14:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558459380770 con 0x558459577000 2023-11-06T03:14:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46394/0 (socket says 172.21.15.157:46394) 2023-11-06T03:14:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 -- 172.21.15.157:0/1712765867 learned_addr learned my addr 172.21.15.157:0/1712765867 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 -- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 msgr2=0x55845952c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845952c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 -- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 msgr2=0x55845952ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866f7fe640 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x55845952ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x55845952ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 -- 172.21.15.157:0/1712765867 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558459078890 con 0x5584595766e0 2023-11-06T03:14:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f86600049d0 tx=0x7f866000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c8ac2f3e1f8028 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f8664ff9640 1 -- 172.21.15.157:0/1712765867 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866000f7c0 con 0x5584595766e0 2023-11-06T03:14:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f8664ff9640 1 -- 172.21.15.157:0/1712765867 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866000f920 con 0x5584595766e0 2023-11-06T03:14:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f8664ff9640 1 -- 172.21.15.157:0/1712765867 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866000fc90 con 0x5584595766e0 2023-11-06T03:14:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- 172.21.15.157:0/1712765867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 msgr2=0x558459576ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f86600049d0 tx=0x7f866000d000 comp rx=0 tx=0).stop 2023-11-06T03:14:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- 172.21.15.157:0/1712765867 shutdown_connections 2023-11-06T03:14:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x55845952ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845952c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- 172.21.15.157:0/1712765867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x558459576ac0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- 172.21.15.157:0/1712765867 >> 172.21.15.157:0/1712765867 conn(0x558459532bf0 msgr2=0x55845952ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- 172.21.15.157:0/1712765867 shutdown_connections 2023-11-06T03:14:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- 172.21.15.157:0/1712765867 wait complete. 2023-11-06T03:14:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- start start 2023-11-06T03:14:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 Processor -- start 2023-11-06T03:14:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 0x558459537fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x558459538510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845937c060 con 0x55845952ca90 2023-11-06T03:14:40.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845931b290 con 0x5584595766e0 2023-11-06T03:14:40.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584593396f0 con 0x558459577000 2023-11-06T03:14:40.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f86784a1000 10 client.0 _ll_register_callbacks cb 0x5584595181d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:40.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46402/0 (socket says 172.21.15.157:46402) 2023-11-06T03:14:40.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866effd640 1 -- 172.21.15.157:0/1052166441 learned_addr learned my addr 172.21.15.157:0/1052166441 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f8667fff640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 0x558459537fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.748+0000 7f866f7fe640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x558459538510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 -- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 msgr2=0x558459538510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x558459538510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 -- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 msgr2=0x558459537fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 0x558459537fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 -- 172.21.15.157:0/1052166441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8670071df0 con 0x55845952ca90 2023-11-06T03:14:40.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866f7fe640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x558459538510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:40.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f8667fff640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 0x558459537fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:40.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f866effd640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x5584593fae40 tx=0x7f8660001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d832dd9086186b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:40.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660029070 con 0x55845952ca90 2023-11-06T03:14:40.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8660014430 con 0x55845952ca90 2023-11-06T03:14:40.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55845937a9b0 con 0x55845952ca90 2023-11-06T03:14:40.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86600145f0 con 0x55845952ca90 2023-11-06T03:14:40.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 10 client.9531 fetch_fsmap learned FSMap version 99 2023-11-06T03:14:40.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5584593411e0 con 0x55845952ca90 2023-11-06T03:14:40.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55845937a9b0 con 0x55845952ca90 2023-11-06T03:14:40.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8660020030 con 0x55845952ca90 2023-11-06T03:14:40.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 10 client.9531 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:14:40.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 10 client.9531 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:14:40.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 20 client.9531 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459538de0 0x558459587c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 4 client.9531 mds_command: new command op to 9375 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584593ab4d0 con 0x558459538de0 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f8667fff640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459538de0 0x558459587c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 10 client.9531 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:14:40.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f8667fff640 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459538de0 0x558459587c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b972b0d54d1ccc server_cookie=da8cdff7c358165b in_seq=0 out_seq=0 2023-11-06T03:14:40.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 1 -- 172.21.15.157:0/1052166441 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5584593ab4d0 con 0x558459538de0 2023-11-06T03:14:40.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 10 client.9531 handle_command_reply: tid=0 2023-11-06T03:14:40.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 client.9531 shutdown 2023-11-06T03:14:40.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 msgr2=0x558459537a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x5584593fae40 tx=0x7f8660001a10 comp rx=0 tx=0).stop 2023-11-06T03:14:40.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 shutdown_connections 2023-11-06T03:14:40.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459538de0 0x558459587c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x558459538510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5584595766e0 0x558459537fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 --2- 172.21.15.157:0/1052166441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x558459537a90 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f864dffb640 0 client.9531 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:14:40.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 >> 172.21.15.157:0/1052166441 conn(0x558459532bf0 msgr2=0x55845952b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 shutdown_connections 2023-11-06T03:14:40.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 1 -- 172.21.15.157:0/1052166441 wait complete. 2023-11-06T03:14:40.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.752+0000 7f86784a1000 20 client.9531 trim_cache size 0 max 16384 2023-11-06T03:14:40.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 Processor -- start 2023-11-06T03:14:40.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- start start 2023-11-06T03:14:40.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x558459589370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845956f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558459039940 con 0x5584595766e0 2023-11-06T03:14:40.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845958c460 con 0x55845952ca90 2023-11-06T03:14:40.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584595242a0 con 0x558459577000 2023-11-06T03:14:40.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46410/0 (socket says 172.21.15.157:46410) 2023-11-06T03:14:40.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 -- 172.21.15.157:0/3458786168 learned_addr learned my addr 172.21.15.157:0/3458786168 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 -- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 msgr2=0x55845956f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845956f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 -- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 msgr2=0x558459589370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x558459589370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 -- 172.21.15.157:0/3458786168 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558459327330 con 0x5584595766e0 2023-11-06T03:14:40.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f8667fff640 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8658050ac0 tx=0x7f8658000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6568f99a09aba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86657fa640 1 -- 172.21.15.157:0/3458786168 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865805a040 con 0x5584595766e0 2023-11-06T03:14:40.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86657fa640 1 -- 172.21.15.157:0/3458786168 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865805c5a0 con 0x5584595766e0 2023-11-06T03:14:40.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86657fa640 1 -- 172.21.15.157:0/3458786168 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658058120 con 0x5584595766e0 2023-11-06T03:14:40.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- 172.21.15.157:0/3458786168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 msgr2=0x5584595898b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8658050ac0 tx=0x7f8658000f80 comp rx=0 tx=0).stop 2023-11-06T03:14:40.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- 172.21.15.157:0/3458786168 shutdown_connections 2023-11-06T03:14:40.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558459577000 0x55845956f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584595766e0 0x5584595898b0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- 172.21.15.157:0/3458786168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55845952ca90 0x558459589370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- 172.21.15.157:0/3458786168 >> 172.21.15.157:0/3458786168 conn(0x558459532bf0 msgr2=0x558459587c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- 172.21.15.157:0/3458786168 shutdown_connections 2023-11-06T03:14:40.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- 172.21.15.157:0/3458786168 wait complete. 2023-11-06T03:14:40.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- start start 2023-11-06T03:14:40.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 Processor -- start 2023-11-06T03:14:40.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584595766e0 0x5584595d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558459577000 0x558459572af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584595324a0 con 0x55845952ca90 2023-11-06T03:14:40.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558459537d10 con 0x558459577000 2023-11-06T03:14:40.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558459530da0 con 0x5584595766e0 2023-11-06T03:14:40.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.768+0000 7f86784a1000 10 client.0 _ll_register_callbacks cb 0x5584594f89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:14:40.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46426/0 (socket says 172.21.15.157:46426) 2023-11-06T03:14:40.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 -- 172.21.15.157:0/3358032604 learned_addr learned my addr 172.21.15.157:0/3358032604 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866f7fe640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558459577000 0x558459572af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8667fff640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584595766e0 0x5584595d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 -- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584595766e0 msgr2=0x5584595d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584595766e0 0x5584595d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 -- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558459577000 msgr2=0x558459572af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558459577000 0x558459572af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 -- 172.21.15.157:0/3358032604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700c7af0 con 0x55845952ca90 2023-11-06T03:14:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f866effd640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8660005850 tx=0x7f8660001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2a496de7d2971 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:14:40.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866000fad0 con 0x55845952ca90 2023-11-06T03:14:40.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f866000fe80 con 0x55845952ca90 2023-11-06T03:14:40.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55845934bb90 con 0x55845952ca90 2023-11-06T03:14:40.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8660014930 con 0x55845952ca90 2023-11-06T03:14:40.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 10 client.9537 fetch_fsmap learned FSMap version 99 2023-11-06T03:14:40.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x558459078cc0 con 0x55845952ca90 2023-11-06T03:14:40.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660045020 con 0x55845952ca90 2023-11-06T03:14:40.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8660020070 con 0x55845952ca90 2023-11-06T03:14:40.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 10 client.9537 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:14:40.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 10 client.9537 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:14:40.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 20 client.9537 populate_metadata read hostname 'smithi157' 2023-11-06T03:14:40.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459573030 0x558459573410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 4 client.9537 mds_command: new command op to 9375 tid=0[{"prefix": "status"}] 2023-11-06T03:14:40.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584593750e0 con 0x558459573030 2023-11-06T03:14:40.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8667fff640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459573030 0x558459573410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8667fff640 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459573030 0x558459573410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673f21a326791c9 server_cookie=4ae61df967dcaa0c in_seq=0 out_seq=0 2023-11-06T03:14:40.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 10 client.9537 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:14:40.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 1 -- 172.21.15.157:0/3358032604 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5584593750e0 con 0x558459573030 2023-11-06T03:14:40.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 10 client.9537 handle_command_reply: tid=0 2023-11-06T03:14:40.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 client.9537 shutdown 2023-11-06T03:14:40.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 msgr2=0x5584595d3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8660005850 tx=0x7f8660001c30 comp rx=0 tx=0).stop 2023-11-06T03:14:40.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 shutdown_connections 2023-11-06T03:14:40.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x558459573030 0x558459573410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558459577000 0x558459572af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5584595766e0 0x5584595d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f864dffb640 0 client.9537 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:14:40.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 --2- 172.21.15.157:0/3358032604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55845952ca90 0x5584595d3590 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 >> 172.21.15.157:0/3358032604 conn(0x558459532bf0 msgr2=0x558459587c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 shutdown_connections 2023-11-06T03:14:40.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 1 -- 172.21.15.157:0/3358032604 wait complete. 2023-11-06T03:14:40.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f86784a1000 20 client.9537 trim_cache size 0 max 16384 2023-11-06T03:14:40.802 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:40.802 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:14:40.802 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:14:40.803 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 9375, 2023-11-06T03:14:40.803 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:14:40.803 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:14:40.803 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:14:40.803 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 5.7658008650000001, 2023-11-06T03:14:40.804 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 99, 2023-11-06T03:14:40.804 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 137, 2023-11-06T03:14:40.804 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 137, 2023-11-06T03:14:40.804 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 10.11944302 2023-11-06T03:14:40.804 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 msgr2=0x7f8650067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 0x7f8650067f20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8660001210 tx=0x7f8660001160 comp rx=0 tx=0).stop 2023-11-06T03:14:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 msgr2=0x7f86700b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f865800e540 tx=0x7f865800ea00 comp rx=0 tx=0).stop 2023-11-06T03:14:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 shutdown_connections 2023-11-06T03:14:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8650065a60 0x7f8650067f20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f86700d0c10 0x7f86700d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867014be70 0x7f86700b4fe0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 --2- 172.21.15.157:0/1593133242 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f865c004d10 0x7f867014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 >> 172.21.15.157:0/1593133242 conn(0x7f867006d560 msgr2=0x7f867006af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 shutdown_connections 2023-11-06T03:14:40.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.772+0000 7f8675379640 1 -- 172.21.15.157:0/1593133242 wait complete. 2023-11-06T03:14:40.808 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:14:40.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.916+0000 7f50df205640 1 Processor -- start 2023-11-06T03:14:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50df205640 1 -- start start 2023-11-06T03:14:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50df205640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f50d80b6190 0x7f50d80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50df205640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d80cf2a0 con 0x7f50d80cdad0 2023-11-06T03:14:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50df205640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d815bbd0 con 0x7f50d815b6a0 2023-11-06T03:14:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50df205640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d80b6570 con 0x7f50d80b6190 2023-11-06T03:14:40.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dd7a2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f50d80cdad0 0x7f50d80ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51382/0 (socket says 172.21.15.157:51382) 2023-11-06T03:14:40.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dd7a2640 1 -- 172.21.15.157:0/311364185 learned_addr learned my addr 172.21.15.157:0/311364185 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:40.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/311364185 >> v2:172.21.15.195:3300/0 conn(0x7f50d80b6190 0x7f50d80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814053952 0 0) 0x7f50d80cf2a0 con 0x7f50d80cdad0 2023-11-06T03:14:40.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c0003530 con 0x7f50d80cdad0 2023-11-06T03:14:40.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346049125 0 0) 0x7f50c0003530 con 0x7f50d80cdad0 2023-11-06T03:14:40.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 >> v2:172.21.15.195:3300/0 conn(0x7f50d80b6190 msgr2=0x7f50d80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 --2- 172.21.15.157:0/311364185 >> v2:172.21.15.195:3300/0 conn(0x7f50d80b6190 0x7f50d80cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 >> v1:172.21.15.195:6789/0 conn(0x7f50d815b6a0 legacy=0x7f50d80ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d815beb0 con 0x7f50d80cdad0 2023-11-06T03:14:40.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f50d0001a00 con 0x7f50d80cdad0 2023-11-06T03:14:40.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 >> v1:172.21.15.157:6789/0 conn(0x7f50d80cdad0 legacy=0x7f50d80ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0005190 0x7f50c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b6570 con 0x7f50c0003980 2023-11-06T03:14:40.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c000c030 con 0x7f50c0005190 2023-11-06T03:14:40.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c000c1f0 con 0x7f50c00047d0 2023-11-06T03:14:40.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dd7a2640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0005190 0x7f50c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 -- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 msgr2=0x7f50c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c00047d0 0x7f50c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 -- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0005190 msgr2=0x7f50c000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0005190 0x7f50c000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dcfa1640 1 -- 172.21.15.157:0/311364185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0003530 con 0x7f50c0003980 2023-11-06T03:14:40.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dd7a2640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c0005190 0x7f50c000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:40.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.920+0000 7f50dd7a2640 1 -- 172.21.15.157:0/311364185 reap_dead start 2023-11-06T03:14:40.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009ae0 tx=0x7f50c801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4998e79404b4fedf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80a55f0 con 0x7f50c0003980 2023-11-06T03:14:40.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c80a5750 con 0x7f50c0003980 2023-11-06T03:14:40.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d77fe640 1 -- 172.21.15.157:0/311364185 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80a5b00 con 0x7f50c0003980 2023-11-06T03:14:40.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 msgr2=0x7f50c0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009ae0 tx=0x7f50c801ca20 comp rx=0 tx=0).stop 2023-11-06T03:14:40.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/311364185 shutdown_connections 2023-11-06T03:14:40.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 --2- 172.21.15.157:0/311364185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c0003980 0x7f50c0003da0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/311364185 >> 172.21.15.157:0/311364185 conn(0x7f50d80b23e0 msgr2=0x7f50d80b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:40.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/311364185 shutdown_connections 2023-11-06T03:14:40.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/311364185 wait complete. 2023-11-06T03:14:40.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 Processor -- start 2023-11-06T03:14:40.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- start start 2023-11-06T03:14:40.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c0003980 0x7f50d8147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80b08f0 0x7f50d80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b12b0 con 0x7f50d8144950 2023-11-06T03:14:40.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8145220 con 0x7f50d80b08f0 2023-11-06T03:14:40.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80c9e90 con 0x7f50c0003980 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46436/0 (socket says 172.21.15.157:46436) 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 -- 172.21.15.157:0/456985073 learned_addr learned my addr 172.21.15.157:0/456985073 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c0003980 0x7f50d8147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50dd7a2640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80b08f0 0x7f50d80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 -- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c0003980 msgr2=0x7f50d8147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c0003980 0x7f50d8147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 -- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80b08f0 msgr2=0x7f50d80b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:40.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80b08f0 0x7f50d80b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:40.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 -- 172.21.15.157:0/456985073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80ca170 con 0x7f50d8144950 2023-11-06T03:14:40.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d7fff640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c500 tx=0x7f50cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0052d0 con 0x7f50d8144950 2023-11-06T03:14:40.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc005430 con 0x7f50d8144950 2023-11-06T03:14:40.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc012870 con 0x7f50d8144950 2023-11-06T03:14:40.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.924+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80ca450 con 0x7f50d8144950 2023-11-06T03:14:40.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80c9b30 con 0x7f50d8144950 2023-11-06T03:14:40.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d8072750 con 0x7f50d8144950 2023-11-06T03:14:40.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f50cc0129d0 con 0x7f50d8144950 2023-11-06T03:14:40.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50d57fa640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 0x7f50c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:40.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50cc082ed0 con 0x7f50d8144950 2023-11-06T03:14:40.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 0x7f50c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:40.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.928+0000 7f50dcfa1640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 0x7f50c4067f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50c80af000 tx=0x7f50c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:40.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:40.932+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f50cc050730 con 0x7f50d8144950 2023-11-06T03:14:41.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50d81453d0 con 0x7f50d8144950 2023-11-06T03:14:41.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50d57fa640 1 -- 172.21.15.157:0/456985073 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50cc053d50 con 0x7f50d8144950 2023-11-06T03:14:41.105 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:14:41.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 msgr2=0x7f50c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:41.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 0x7f50c4067f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50c80af000 tx=0x7f50c8008040 comp rx=0 tx=0).stop 2023-11-06T03:14:41.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 msgr2=0x7f50d80c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:41.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c500 tx=0x7f50cc00c9c0 comp rx=0 tx=0).stop 2023-11-06T03:14:41.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 shutdown_connections 2023-11-06T03:14:41.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50c4065a60 0x7f50c4067f20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:41.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50d80b08f0 0x7f50d80b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:41.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50d8144950 0x7f50d80c95f0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:41.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 --2- 172.21.15.157:0/456985073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c0003980 0x7f50d8147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:41.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 >> 172.21.15.157:0/456985073 conn(0x7f50d80b23e0 msgr2=0x7f50d8071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:41.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 shutdown_connections 2023-11-06T03:14:41.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:41.108+0000 7f50df205640 1 -- 172.21.15.157:0/456985073 wait complete. 2023-11-06T03:14:41.122 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:14:41.123 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:14:41.123 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:14:41.124 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:14:41.124 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:41.124 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:14:41.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:14:41.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:14:41.126 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:14:41.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.128 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:14:41.134 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:14:41.134 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:14:41.134 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:14:41.135 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.135 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:14:41.135 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.135 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:14:41.136 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:14:41.137 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.137 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:14:41.137 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.137 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:14:41.137 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41858sec preferred_lft 41858sec 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:14:41.138 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.139 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:14:41.139 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:14:41.139 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:14:41.139 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.139 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.140 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:14:41.141 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:14:41.141 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:14:41.141 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:14:41.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.142 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:14:41.189 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:14:41.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.191 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:14:41.243 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:14:41.243 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:14:41.244 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:14:41.244 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:14:41.244 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:14:41.244 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:14:41.244 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:14:41.245 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:14:41.245 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:14:41.245 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:14:41.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.246 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:14:41.246 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:41.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:14:41.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:14:41.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:14:41.248 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:41.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.320 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:14:41.368 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:14:41.369 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:14:41.369 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:14:41.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.370 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:14:41.425 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:14:41.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:14:41.426 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:41.473 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:14:41.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:14:41.474 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:41.520 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:14:41.578 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:14:41.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:14:41.636 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:14:41.637 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:14:41.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.638 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:14:41.684 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:14:41.685 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:14:41.729 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:14:41.744 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:14:41.745 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:14:41.799 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:14:41.800 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:14:41.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:41.801 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:14:41.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:14:41.812+0000 7f63b5919580 -1 init, newargv = 0x55dde6a96fc0 newargc=15 2023-11-06T03:14:41.808 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[90618]: starting ceph client 2023-11-06T03:14:41.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[90618]: starting fuse 2023-11-06T03:14:41.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:14:41.843 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:14:42.914 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:14:42.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:14:42.931 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:14:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:14:42.985 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:14:42.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:42.986 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:14:43.032 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:14:43.033 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:14:43.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:14:43.033 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:14:43.034 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:14:43.034 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:14:43.035 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:14:43.035 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:14:43.035 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:14:43.036 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:14:43.036 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:14:43.037 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:14:43.037 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:14:43.037 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:14:43.038 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:14:43.038 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:14:43.038 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:14:43.039 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:14:43.039 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:14:43.040 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:14:43.040 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:14:43.040 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:14:43.041 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:14:43.041 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:14:43.042 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:14:43.042 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:14:43.043 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:14:43.043 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:14:43.043 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:14:43.151 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.90618.asok 2023-11-06T03:14:43.333 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.90618.asok 2023-11-06T03:14:43.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:43.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.90618.asok status 2023-11-06T03:14:43.496 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:14:43.496 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:14:43.496 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:14:43.496 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:14:43.497 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:14:43.497 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:14:43.497 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:14:43.497 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "90618", 2023-11-06T03:14:43.497 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 9552, 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:14:43.498 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:14:43.499 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:14:43.499 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 9552 2023-11-06T03:14:43.499 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:43.499 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:14:43.499 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:14:43.500 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:14:43.500 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 903558861 2023-11-06T03:14:43.500 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:14:43.500 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:43.500 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 903558861 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.9552 192.168.0.1:0/903558861", 2023-11-06T03:14:43.501 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/903558861", 2023-11-06T03:14:43.502 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:14:43.502 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 99, 2023-11-06T03:14:43.502 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 137, 2023-11-06T03:14:43.502 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:14:43.502 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:14:43.503 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:14:43.503 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:14:43.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:43.510 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:14:43.510 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:43.517 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:14:43.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:43.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:43.518 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:14:43.571 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:43.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 Processor -- start 2023-11-06T03:14:43.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 -- start start 2023-11-06T03:14:43.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ad40b2400 0x7f2ad40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:43.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad406d2b0 con 0x7f2ad415a600 2023-11-06T03:14:43.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad40b2d20 con 0x7f2ad415d4d0 2023-11-06T03:14:43.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad91a1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad406c820 con 0x7f2ad40b2400 2023-11-06T03:14:43.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ad40b2400 0x7f2ad40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:43.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad359e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2ad415a600 0x7f2ad415aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51390/0 (socket says 172.21.15.157:51390) 2023-11-06T03:14:43.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad359e640 1 -- 172.21.15.157:0/2845225628 learned_addr learned my addr 172.21.15.157:0/2845225628 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:14:43.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/2845225628 >> v1:172.21.15.195:6789/0 conn(0x7f2ad415d4d0 legacy=0x7f2ad406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:43.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/2845225628 >> v1:172.21.15.157:6789/0 conn(0x7f2ad415a600 legacy=0x7f2ad415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad259c640 1 -- 172.21.15.157:0/2845225628 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358117557 0 0) 0x7f2ad406d2b0 con 0x7f2ad415a600 2023-11-06T03:14:43.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.744+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/2845225628 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad406ca10 con 0x7f2ad40b2400 2023-11-06T03:14:43.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- 172.21.15.157:0/2845225628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad40b27e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0097c0 tx=0x7f2abc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b15cd579c62d7e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:43.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad259c640 1 -- 172.21.15.157:0/2845225628 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0ab610 con 0x7f2ad40b2400 2023-11-06T03:14:43.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad259c640 1 -- 172.21.15.157:0/2845225628 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc0ab770 con 0x7f2ad40b2400 2023-11-06T03:14:43.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad259c640 1 -- 172.21.15.157:0/2845225628 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0abb00 con 0x7f2ad40b2400 2023-11-06T03:14:43.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/2845225628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 msgr2=0x7f2ad40b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/2845225628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad40b27e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0097c0 tx=0x7f2abc01c410 comp rx=0 tx=0).stop 2023-11-06T03:14:43.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/2845225628 shutdown_connections 2023-11-06T03:14:43.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/2845225628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad40b27e0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/2845225628 >> 172.21.15.157:0/2845225628 conn(0x7f2ad4065930 msgr2=0x7f2ad4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:43.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/2845225628 shutdown_connections 2023-11-06T03:14:43.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/2845225628 wait complete. 2023-11-06T03:14:43.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 Processor -- start 2023-11-06T03:14:43.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- start start 2023-11-06T03:14:43.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:43.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad415a600 0x7f2ad4150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:43.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ad415d4d0 0x7f2ad4153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:43.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad414b230 con 0x7f2ad40b2400 2023-11-06T03:14:43.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40af680 con 0x7f2ad415a600 2023-11-06T03:14:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4151480 con 0x7f2ad415d4d0 2023-11-06T03:14:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46466/0 (socket says 172.21.15.157:46466) 2023-11-06T03:14:43.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/3829875294 learned_addr learned my addr 172.21.15.157:0/3829875294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:43.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad359e640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ad415d4d0 0x7f2ad4153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:43.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2acbfff640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad415a600 0x7f2ad4150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:43.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ad415d4d0 msgr2=0x7f2ad4153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ad415d4d0 0x7f2ad4153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad415a600 msgr2=0x7f2ad4150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad415a600 0x7f2ad4150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 -- 172.21.15.157:0/3829875294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc009420 con 0x7f2ad40b2400 2023-11-06T03:14:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad2d9d640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ad040 tx=0x7f2abc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b5810 con 0x7f2ad40b2400 2023-11-06T03:14:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc0b59b0 con 0x7f2ad40b2400 2023-11-06T03:14:43.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.748+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad40af3a0 con 0x7f2ad40b2400 2023-11-06T03:14:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b5d60 con 0x7f2ad40b2400 2023-11-06T03:14:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad414f670 con 0x7f2ad40b2400 2023-11-06T03:14:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad4159d80 con 0x7f2ad40b2400 2023-11-06T03:14:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2abc0b4390 con 0x7f2ad40b2400 2023-11-06T03:14:43.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acb7fe640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 0x7f2ab4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2abc11dd60 con 0x7f2ad40b2400 2023-11-06T03:14:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acbfff640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 0x7f2ab4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:43.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.752+0000 7f2acbfff640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 0x7f2ab4067f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ad415cbd0 tx=0x7f2ac4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:43.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.756+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ad4159d80 con 0x7f2ad40b2400 2023-11-06T03:14:43.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.944+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ad41533b0 con 0x7f2ad40b2400 2023-11-06T03:14:43.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.944+0000 7f2acb7fe640 1 -- 172.21.15.157:0/3829875294 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f2ad41533b0 con 0x7f2ad40b2400 2023-11-06T03:14:43.942 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:43.942 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:14:43.942 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:14:43.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 msgr2=0x7f2ab4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 0x7f2ab4067f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ad415cbd0 tx=0x7f2ac4008040 comp rx=0 tx=0).stop 2023-11-06T03:14:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 msgr2=0x7f2ad414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ad040 tx=0x7f2abc01ac50 comp rx=0 tx=0).stop 2023-11-06T03:14:43.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 shutdown_connections 2023-11-06T03:14:43.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2ab4065a90 0x7f2ab4067f50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2ad415d4d0 0x7f2ad4153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2ad415a600 0x7f2ad4150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 --2- 172.21.15.157:0/3829875294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ad40b2400 0x7f2ad414acf0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:43.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 >> 172.21.15.157:0/3829875294 conn(0x7f2ad4065930 msgr2=0x7f2ad415aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 shutdown_connections 2023-11-06T03:14:43.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:43.948+0000 7f2ad91a1640 1 -- 172.21.15.157:0/3829875294 wait complete. 2023-11-06T03:14:43.962 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:14:44.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.080+0000 7f90a8155640 1 Processor -- start 2023-11-06T03:14:44.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.080+0000 7f90a8155640 1 -- start start 2023-11-06T03:14:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 0x7f90a015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90a00ce200 0x7f90a006c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00704d0 0x7f90a00cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a015e220 con 0x7f90a00ce200 2023-11-06T03:14:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a015d980 con 0x7f90a00cde20 2023-11-06T03:14:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a8155640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a015dc60 con 0x7f90a00704d0 2023-11-06T03:14:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90a00ce200 0x7f90a006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f90a00ce200 0x7f90a006c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46468/0 (socket says 172.21.15.157:46468) 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 -- 172.21.15.157:0/951352142 learned_addr learned my addr 172.21.15.157:0/951352142 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a5ef1640 1 --2- 172.21.15.157:0/951352142 >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 0x7f90a015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a66f2640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00704d0 0x7f90a00cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 -- 172.21.15.157:0/951352142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00704d0 msgr2=0x7f90a00cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00704d0 0x7f90a00cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 -- 172.21.15.157:0/951352142 >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 msgr2=0x7f90a015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:14:44.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 --2- 172.21.15.157:0/951352142 >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 0x7f90a015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 -- 172.21.15.157:0/951352142 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a015df40 con 0x7f90a00ce200 2023-11-06T03:14:44.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a5ef1640 1 --2- 172.21.15.157:0/951352142 >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 0x7f90a015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:44.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a56f0640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00ce200 0x7f90a006c710 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f9090009870 tx=0x7f909001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533a4f26e55d323e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:44.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a4eef640 1 -- 172.21.15.157:0/951352142 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900ab650 con 0x7f90a00ce200 2023-11-06T03:14:44.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a4eef640 1 -- 172.21.15.157:0/951352142 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90900ab7b0 con 0x7f90a00ce200 2023-11-06T03:14:44.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.084+0000 7f90a4eef640 1 -- 172.21.15.157:0/951352142 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90900abb40 con 0x7f90a00ce200 2023-11-06T03:14:44.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/951352142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00ce200 msgr2=0x7f90a006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00ce200 0x7f90a006c710 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f9090009870 tx=0x7f909001c470 comp rx=0 tx=0).stop 2023-11-06T03:14:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/951352142 shutdown_connections 2023-11-06T03:14:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00704d0 0x7f90a00cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- 172.21.15.157:0/951352142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00ce200 0x7f90a006c710 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- 172.21.15.157:0/951352142 >> v2:172.21.15.195:3300/0 conn(0x7f90a00cde20 0x7f90a015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/951352142 >> 172.21.15.157:0/951352142 conn(0x7f90a006a4d0 msgr2=0x7f90a006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/951352142 shutdown_connections 2023-11-06T03:14:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/951352142 wait complete. 2023-11-06T03:14:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 Processor -- start 2023-11-06T03:14:44.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- start start 2023-11-06T03:14:44.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 0x7f90a014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a00ce200 0x7f90a0152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a01500c0 con 0x7f90a00704d0 2023-11-06T03:14:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a014f7d0 con 0x7f90a00ce200 2023-11-06T03:14:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a014fa70 con 0x7f90a00cde20 2023-11-06T03:14:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46478/0 (socket says 172.21.15.157:46478) 2023-11-06T03:14:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 -- 172.21.15.157:0/570445449 learned_addr learned my addr 172.21.15.157:0/570445449 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:14:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a66f2640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a00ce200 0x7f90a0152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a56f0640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 0x7f90a014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 -- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 msgr2=0x7f90a014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 0x7f90a014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 -- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a00ce200 msgr2=0x7f90a0152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a00ce200 0x7f90a0152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 -- 172.21.15.157:0/570445449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900094d0 con 0x7f90a00704d0 2023-11-06T03:14:44.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a56f0640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 0x7f90a014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:14:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a5ef1640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c7f0 tx=0x7f909c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c005ba0 con 0x7f90a00704d0 2023-11-06T03:14:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c005d00 con 0x7f90a00704d0 2023-11-06T03:14:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a014fd50 con 0x7f90a00704d0 2023-11-06T03:14:44.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c007a50 con 0x7f90a00704d0 2023-11-06T03:14:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.088+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a01613b0 con 0x7f90a00704d0 2023-11-06T03:14:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a0072750 con 0x7f90a00704d0 2023-11-06T03:14:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f909c007bb0 con 0x7f90a00704d0 2023-11-06T03:14:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90967fc640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 0x7f9084067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:14:44.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f909c083210 con 0x7f90a00704d0 2023-11-06T03:14:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90a56f0640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 0x7f9084067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:14:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.092+0000 7f90a56f0640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 0x7f9084067f20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f90900af000 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:14:44.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.096+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f909c050a60 con 0x7f90a00704d0 2023-11-06T03:14:44.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.284+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90a0167530 con 0x7f90a00704d0 2023-11-06T03:14:44.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.284+0000 7f90967fc640 1 -- 172.21.15.157:0/570445449 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f909c007e60 con 0x7f90a00704d0 2023-11-06T03:14:44.283 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:14:44.283 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:14:44.284 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:14:44.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 msgr2=0x7f9084067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:44.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 0x7f9084067f20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f90900af000 tx=0x7f9090008040 comp rx=0 tx=0).stop 2023-11-06T03:14:44.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 msgr2=0x7f90a0150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:14:44.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c7f0 tx=0x7f909c00cbf0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 shutdown_connections 2023-11-06T03:14:44.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9084065a60 0x7f9084067f20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90a00ce200 0x7f90a0152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90a00cde20 0x7f90a014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 --2- 172.21.15.157:0/570445449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a00704d0 0x7f90a0150550 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:14:44.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 >> 172.21.15.157:0/570445449 conn(0x7f90a006a4d0 msgr2=0x7f90a015ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:14:44.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 shutdown_connections 2023-11-06T03:14:44.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:14:44.288+0000 7f90a8155640 1 -- 172.21.15.157:0/570445449 wait complete. 2023-11-06T03:14:44.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:44.304 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:44.304 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:44.305 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.0 2023-11-06T03:14:44.306 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:44.306 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:44.307 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:44.307 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:44.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.353 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184838 s, 56.7 MB/s 2023-11-06T03:14:44.367 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136276 s, 76.9 MB/s 2023-11-06T03:14:44.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.383 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116639 s, 89.9 MB/s 2023-11-06T03:14:44.395 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.395 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.396 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103668 s, 101 MB/s 2023-11-06T03:14:44.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.407 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874456 s, 120 MB/s 2023-11-06T03:14:44.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.421 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111037 s, 94.4 MB/s 2023-11-06T03:14:44.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.432 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.432 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941705 s, 111 MB/s 2023-11-06T03:14:44.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109942 s, 95.4 MB/s 2023-11-06T03:14:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106436 s, 98.5 MB/s 2023-11-06T03:14:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.475 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139914 s, 74.9 MB/s 2023-11-06T03:14:44.489 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120691 s, 86.9 MB/s 2023-11-06T03:14:44.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121351 s, 86.4 MB/s 2023-11-06T03:14:44.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.519 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118653 s, 88.4 MB/s 2023-11-06T03:14:44.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134844 s, 77.8 MB/s 2023-11-06T03:14:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.547 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949571 s, 110 MB/s 2023-11-06T03:14:44.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109762 s, 95.5 MB/s 2023-11-06T03:14:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113774 s, 92.2 MB/s 2023-11-06T03:14:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01336 s, 78.5 MB/s 2023-11-06T03:14:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128601 s, 81.5 MB/s 2023-11-06T03:14:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112188 s, 93.5 MB/s 2023-11-06T03:14:44.634 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.634 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.635 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116667 s, 89.9 MB/s 2023-11-06T03:14:44.646 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.647 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009098 s, 115 MB/s 2023-11-06T03:14:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107634 s, 97.4 MB/s 2023-11-06T03:14:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010497 s, 99.9 MB/s 2023-11-06T03:14:44.684 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.685 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915537 s, 115 MB/s 2023-11-06T03:14:44.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113755 s, 92.2 MB/s 2023-11-06T03:14:44.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.715 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013058 s, 80.3 MB/s 2023-11-06T03:14:44.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.725 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794734 s, 132 MB/s 2023-11-06T03:14:44.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125421 s, 83.6 MB/s 2023-11-06T03:14:44.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.756 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.756 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123639 s, 84.8 MB/s 2023-11-06T03:14:44.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.770 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.770 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112937 s, 92.8 MB/s 2023-11-06T03:14:44.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.782 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.782 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890469 s, 118 MB/s 2023-11-06T03:14:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:44.783 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:44.784 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:44.784 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.1 2023-11-06T03:14:44.784 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:44.785 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:44.785 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:44.785 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132315 s, 79.2 MB/s 2023-11-06T03:14:44.872 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.873 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158442 s, 66.2 MB/s 2023-11-06T03:14:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133115 s, 78.8 MB/s 2023-11-06T03:14:44.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.906 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133979 s, 78.3 MB/s 2023-11-06T03:14:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132939 s, 78.9 MB/s 2023-11-06T03:14:44.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.937 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012166 s, 86.2 MB/s 2023-11-06T03:14:44.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.951 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.951 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107403 s, 97.6 MB/s 2023-11-06T03:14:44.966 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.967 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125832 s, 83.3 MB/s 2023-11-06T03:14:44.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:44.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:44.984 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132058 s, 79.4 MB/s 2023-11-06T03:14:44.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119329 s, 87.9 MB/s 2023-11-06T03:14:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.017 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117555 s, 89.2 MB/s 2023-11-06T03:14:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.035 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.036 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144628 s, 72.5 MB/s 2023-11-06T03:14:45.055 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152094 s, 68.9 MB/s 2023-11-06T03:14:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143811 s, 72.9 MB/s 2023-11-06T03:14:45.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.095 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146233 s, 71.7 MB/s 2023-11-06T03:14:45.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.117 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161873 s, 64.8 MB/s 2023-11-06T03:14:45.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125657 s, 83.4 MB/s 2023-11-06T03:14:45.151 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.152 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132573 s, 79.1 MB/s 2023-11-06T03:14:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.172 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152316 s, 68.8 MB/s 2023-11-06T03:14:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.191 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144191 s, 72.7 MB/s 2023-11-06T03:14:45.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.208 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140175 s, 74.8 MB/s 2023-11-06T03:14:45.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.226 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141901 s, 73.9 MB/s 2023-11-06T03:14:45.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158681 s, 66.1 MB/s 2023-11-06T03:14:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141975 s, 73.9 MB/s 2023-11-06T03:14:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.282 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141595 s, 74.1 MB/s 2023-11-06T03:14:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.301 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157693 s, 66.5 MB/s 2023-11-06T03:14:45.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.319 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.319 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136118 s, 77.0 MB/s 2023-11-06T03:14:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.334 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.335 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119559 s, 87.7 MB/s 2023-11-06T03:14:45.351 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.352 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.352 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138393 s, 75.8 MB/s 2023-11-06T03:14:45.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135753 s, 77.2 MB/s 2023-11-06T03:14:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.387 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155128 s, 67.6 MB/s 2023-11-06T03:14:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146123 s, 71.8 MB/s 2023-11-06T03:14:45.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:45.408 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:45.408 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:45.409 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.2 2023-11-06T03:14:45.409 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:45.410 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:45.410 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:45.410 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:45.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.480 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127265 s, 82.4 MB/s 2023-11-06T03:14:45.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013354 s, 78.5 MB/s 2023-11-06T03:14:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.514 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150905 s, 69.5 MB/s 2023-11-06T03:14:45.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013599 s, 77.1 MB/s 2023-11-06T03:14:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.547 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.548 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136138 s, 77.0 MB/s 2023-11-06T03:14:45.565 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.565 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.565 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014751 s, 71.1 MB/s 2023-11-06T03:14:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141102 s, 74.3 MB/s 2023-11-06T03:14:45.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.598 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129109 s, 81.2 MB/s 2023-11-06T03:14:45.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134953 s, 77.7 MB/s 2023-11-06T03:14:45.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.631 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128264 s, 81.8 MB/s 2023-11-06T03:14:45.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.648 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142112 s, 73.8 MB/s 2023-11-06T03:14:45.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.665 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136211 s, 77.0 MB/s 2023-11-06T03:14:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133476 s, 78.6 MB/s 2023-11-06T03:14:45.696 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.696 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.696 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124971 s, 83.9 MB/s 2023-11-06T03:14:45.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.713 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143622 s, 73.0 MB/s 2023-11-06T03:14:45.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.729 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132117 s, 79.4 MB/s 2023-11-06T03:14:45.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131437 s, 79.8 MB/s 2023-11-06T03:14:45.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.760 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.760 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126312 s, 83.0 MB/s 2023-11-06T03:14:45.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131602 s, 79.7 MB/s 2023-11-06T03:14:45.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.793 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134994 s, 77.7 MB/s 2023-11-06T03:14:45.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139822 s, 75.0 MB/s 2023-11-06T03:14:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013491 s, 77.7 MB/s 2023-11-06T03:14:45.843 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.844 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.844 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134007 s, 78.2 MB/s 2023-11-06T03:14:45.859 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.860 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.860 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130453 s, 80.4 MB/s 2023-11-06T03:14:45.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112257 s, 93.4 MB/s 2023-11-06T03:14:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137823 s, 76.1 MB/s 2023-11-06T03:14:45.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.907 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.907 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134958 s, 77.7 MB/s 2023-11-06T03:14:45.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124779 s, 84.0 MB/s 2023-11-06T03:14:45.938 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.938 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.938 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129929 s, 80.7 MB/s 2023-11-06T03:14:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138371 s, 75.8 MB/s 2023-11-06T03:14:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156142 s, 67.2 MB/s 2023-11-06T03:14:45.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:45.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:45.992 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148998 s, 70.4 MB/s 2023-11-06T03:14:45.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:45.994 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:45.994 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:45.994 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.3 2023-11-06T03:14:45.995 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:45.995 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:45.996 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:45.996 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:46.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.066 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146027 s, 71.8 MB/s 2023-11-06T03:14:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.085 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.086 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153463 s, 68.3 MB/s 2023-11-06T03:14:46.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.101 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138322 s, 75.8 MB/s 2023-11-06T03:14:46.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.117 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01232 s, 85.1 MB/s 2023-11-06T03:14:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155602 s, 67.4 MB/s 2023-11-06T03:14:46.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.154 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152625 s, 68.7 MB/s 2023-11-06T03:14:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118329 s, 88.6 MB/s 2023-11-06T03:14:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.184 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116354 s, 90.1 MB/s 2023-11-06T03:14:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141771 s, 74.0 MB/s 2023-11-06T03:14:46.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144256 s, 72.7 MB/s 2023-11-06T03:14:46.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.231 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972408 s, 108 MB/s 2023-11-06T03:14:46.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.248 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.248 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128477 s, 81.6 MB/s 2023-11-06T03:14:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119937 s, 87.4 MB/s 2023-11-06T03:14:46.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.279 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130311 s, 80.5 MB/s 2023-11-06T03:14:46.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.296 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.296 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125579 s, 83.5 MB/s 2023-11-06T03:14:46.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148043 s, 70.8 MB/s 2023-11-06T03:14:46.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122634 s, 85.5 MB/s 2023-11-06T03:14:46.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144434 s, 72.6 MB/s 2023-11-06T03:14:46.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.366 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.367 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133085 s, 78.8 MB/s 2023-11-06T03:14:46.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.385 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136065 s, 77.1 MB/s 2023-11-06T03:14:46.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.402 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.402 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130444 s, 80.4 MB/s 2023-11-06T03:14:46.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.422 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157212 s, 66.7 MB/s 2023-11-06T03:14:46.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.440 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.441 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135395 s, 77.4 MB/s 2023-11-06T03:14:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011795 s, 88.9 MB/s 2023-11-06T03:14:46.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.476 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.476 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162246 s, 64.6 MB/s 2023-11-06T03:14:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.495 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153158 s, 68.5 MB/s 2023-11-06T03:14:46.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.512 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139172 s, 75.3 MB/s 2023-11-06T03:14:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.531 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146831 s, 71.4 MB/s 2023-11-06T03:14:46.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147447 s, 71.1 MB/s 2023-11-06T03:14:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.566 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132622 s, 79.1 MB/s 2023-11-06T03:14:46.582 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131308 s, 79.9 MB/s 2023-11-06T03:14:46.601 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142698 s, 73.5 MB/s 2023-11-06T03:14:46.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:46.603 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:46.604 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:46.604 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.4 2023-11-06T03:14:46.605 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:46.605 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:46.605 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:46.606 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:46.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014782 s, 70.9 MB/s 2023-11-06T03:14:46.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.701 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139391 s, 75.2 MB/s 2023-11-06T03:14:46.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014816 s, 70.8 MB/s 2023-11-06T03:14:46.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.736 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.736 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142403 s, 73.6 MB/s 2023-11-06T03:14:46.750 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.751 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112724 s, 93.0 MB/s 2023-11-06T03:14:46.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.766 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119373 s, 87.8 MB/s 2023-11-06T03:14:46.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.782 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.782 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130241 s, 80.5 MB/s 2023-11-06T03:14:46.797 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.798 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133437 s, 78.6 MB/s 2023-11-06T03:14:46.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.812 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107561 s, 97.5 MB/s 2023-11-06T03:14:46.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.825 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107304 s, 97.7 MB/s 2023-11-06T03:14:46.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.842 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.843 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142409 s, 73.6 MB/s 2023-11-06T03:14:46.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.859 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.859 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133166 s, 78.7 MB/s 2023-11-06T03:14:46.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.875 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01331 s, 78.8 MB/s 2023-11-06T03:14:46.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.891 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012713 s, 82.5 MB/s 2023-11-06T03:14:46.903 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106154 s, 98.8 MB/s 2023-11-06T03:14:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106345 s, 98.6 MB/s 2023-11-06T03:14:46.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.934 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139288 s, 75.3 MB/s 2023-11-06T03:14:46.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.948 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118402 s, 88.6 MB/s 2023-11-06T03:14:46.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.964 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128901 s, 81.3 MB/s 2023-11-06T03:14:46.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.979 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119884 s, 87.5 MB/s 2023-11-06T03:14:46.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:46.996 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:46.996 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144851 s, 72.4 MB/s 2023-11-06T03:14:47.011 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.011 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.011 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123466 s, 84.9 MB/s 2023-11-06T03:14:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013158 s, 79.7 MB/s 2023-11-06T03:14:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.044 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133432 s, 78.6 MB/s 2023-11-06T03:14:47.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.059 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.059 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125042 s, 83.9 MB/s 2023-11-06T03:14:47.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.073 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111021 s, 94.4 MB/s 2023-11-06T03:14:47.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.091 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150399 s, 69.7 MB/s 2023-11-06T03:14:47.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.108 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138617 s, 75.6 MB/s 2023-11-06T03:14:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.125 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01385 s, 75.7 MB/s 2023-11-06T03:14:47.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143364 s, 73.1 MB/s 2023-11-06T03:14:47.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134763 s, 77.8 MB/s 2023-11-06T03:14:47.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.176 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145491 s, 72.1 MB/s 2023-11-06T03:14:47.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:47.178 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:47.178 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:47.179 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.5 2023-11-06T03:14:47.179 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:47.179 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:47.180 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:47.180 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:47.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.252 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126258 s, 83.1 MB/s 2023-11-06T03:14:47.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133033 s, 78.8 MB/s 2023-11-06T03:14:47.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147055 s, 71.3 MB/s 2023-11-06T03:14:47.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.302 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130181 s, 80.5 MB/s 2023-11-06T03:14:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.316 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114034 s, 92.0 MB/s 2023-11-06T03:14:47.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137599 s, 76.2 MB/s 2023-11-06T03:14:47.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116136 s, 90.3 MB/s 2023-11-06T03:14:47.361 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111432 s, 94.1 MB/s 2023-11-06T03:14:47.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.380 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149245 s, 70.3 MB/s 2023-11-06T03:14:47.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.397 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137934 s, 76.0 MB/s 2023-11-06T03:14:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116854 s, 89.7 MB/s 2023-11-06T03:14:47.428 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.429 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141144 s, 74.3 MB/s 2023-11-06T03:14:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.447 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142159 s, 73.8 MB/s 2023-11-06T03:14:47.463 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.464 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133382 s, 78.6 MB/s 2023-11-06T03:14:47.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.480 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134462 s, 78.0 MB/s 2023-11-06T03:14:47.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01251 s, 83.8 MB/s 2023-11-06T03:14:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142664 s, 73.5 MB/s 2023-11-06T03:14:47.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.530 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.530 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134686 s, 77.9 MB/s 2023-11-06T03:14:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.544 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115913 s, 90.5 MB/s 2023-11-06T03:14:47.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.560 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013395 s, 78.3 MB/s 2023-11-06T03:14:47.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.577 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132525 s, 79.1 MB/s 2023-11-06T03:14:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.594 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142885 s, 73.4 MB/s 2023-11-06T03:14:47.609 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.610 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129967 s, 80.7 MB/s 2023-11-06T03:14:47.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.627 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.627 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143053 s, 73.3 MB/s 2023-11-06T03:14:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.644 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126159 s, 83.1 MB/s 2023-11-06T03:14:47.658 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.658 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.659 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111253 s, 94.3 MB/s 2023-11-06T03:14:47.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.674 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129556 s, 80.9 MB/s 2023-11-06T03:14:47.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.690 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136348 s, 76.9 MB/s 2023-11-06T03:14:47.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103073 s, 102 MB/s 2023-11-06T03:14:47.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120628 s, 86.9 MB/s 2023-11-06T03:14:47.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129401 s, 81.0 MB/s 2023-11-06T03:14:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099758 s, 105 MB/s 2023-11-06T03:14:47.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:47.748 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:47.748 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:47.749 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.6 2023-11-06T03:14:47.749 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:47.750 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:47.750 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:47.750 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:47.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.823 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132397 s, 79.2 MB/s 2023-11-06T03:14:47.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.840 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142138 s, 73.8 MB/s 2023-11-06T03:14:47.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112385 s, 93.3 MB/s 2023-11-06T03:14:47.873 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.874 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159367 s, 65.8 MB/s 2023-11-06T03:14:47.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.888 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111679 s, 93.9 MB/s 2023-11-06T03:14:47.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133057 s, 78.8 MB/s 2023-11-06T03:14:47.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146709 s, 71.5 MB/s 2023-11-06T03:14:47.939 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.940 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143996 s, 72.8 MB/s 2023-11-06T03:14:47.954 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.955 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118256 s, 88.7 MB/s 2023-11-06T03:14:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.968 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010752 s, 97.5 MB/s 2023-11-06T03:14:47.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:47.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:47.985 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138345 s, 75.8 MB/s 2023-11-06T03:14:48.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.001 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126198 s, 83.1 MB/s 2023-11-06T03:14:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136332 s, 76.9 MB/s 2023-11-06T03:14:48.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.034 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.034 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013727 s, 76.4 MB/s 2023-11-06T03:14:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100676 s, 104 MB/s 2023-11-06T03:14:48.062 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.063 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125518 s, 83.5 MB/s 2023-11-06T03:14:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.079 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128336 s, 81.7 MB/s 2023-11-06T03:14:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.093 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105353 s, 99.5 MB/s 2023-11-06T03:14:48.108 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.109 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124353 s, 84.3 MB/s 2023-11-06T03:14:48.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.126 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134216 s, 78.1 MB/s 2023-11-06T03:14:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.142 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131734 s, 79.6 MB/s 2023-11-06T03:14:48.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126578 s, 82.8 MB/s 2023-11-06T03:14:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105968 s, 99.0 MB/s 2023-11-06T03:14:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118602 s, 88.4 MB/s 2023-11-06T03:14:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102345 s, 102 MB/s 2023-11-06T03:14:48.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.217 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123471 s, 84.9 MB/s 2023-11-06T03:14:48.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.233 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130524 s, 80.3 MB/s 2023-11-06T03:14:48.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.251 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.251 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128504 s, 81.6 MB/s 2023-11-06T03:14:48.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.265 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112105 s, 93.5 MB/s 2023-11-06T03:14:48.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.281 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.281 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132608 s, 79.1 MB/s 2023-11-06T03:14:48.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.299 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133615 s, 78.5 MB/s 2023-11-06T03:14:48.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.315 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.315 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132038 s, 79.4 MB/s 2023-11-06T03:14:48.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:48.317 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:48.317 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:48.318 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.7 2023-11-06T03:14:48.318 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:48.319 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:48.319 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:48.319 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:48.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.389 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130879 s, 80.1 MB/s 2023-11-06T03:14:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.405 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128779 s, 81.4 MB/s 2023-11-06T03:14:48.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.420 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.421 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116377 s, 90.1 MB/s 2023-11-06T03:14:48.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.437 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130905 s, 80.1 MB/s 2023-11-06T03:14:48.454 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.455 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150632 s, 69.6 MB/s 2023-11-06T03:14:48.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.468 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104878 s, 100 MB/s 2023-11-06T03:14:48.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.483 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116878 s, 89.7 MB/s 2023-11-06T03:14:48.499 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.499 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.499 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120567 s, 87.0 MB/s 2023-11-06T03:14:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142163 s, 73.8 MB/s 2023-11-06T03:14:48.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.535 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151149 s, 69.4 MB/s 2023-11-06T03:14:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.549 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.550 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104137 s, 101 MB/s 2023-11-06T03:14:48.562 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.563 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107376 s, 97.7 MB/s 2023-11-06T03:14:48.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.577 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.578 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125144 s, 83.8 MB/s 2023-11-06T03:14:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100983 s, 104 MB/s 2023-11-06T03:14:48.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.607 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138064 s, 75.9 MB/s 2023-11-06T03:14:48.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.622 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122687 s, 85.5 MB/s 2023-11-06T03:14:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122435 s, 85.6 MB/s 2023-11-06T03:14:48.655 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155495 s, 67.4 MB/s 2023-11-06T03:14:48.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.671 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.672 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129252 s, 81.1 MB/s 2023-11-06T03:14:48.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125265 s, 83.7 MB/s 2023-11-06T03:14:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.702 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121642 s, 86.2 MB/s 2023-11-06T03:14:48.715 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.716 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.716 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108189 s, 96.9 MB/s 2023-11-06T03:14:48.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124871 s, 84.0 MB/s 2023-11-06T03:14:48.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126054 s, 83.2 MB/s 2023-11-06T03:14:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.762 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125134 s, 83.8 MB/s 2023-11-06T03:14:48.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.779 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135325 s, 77.5 MB/s 2023-11-06T03:14:48.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124994 s, 83.9 MB/s 2023-11-06T03:14:48.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.810 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130181 s, 80.5 MB/s 2023-11-06T03:14:48.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.827 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130775 s, 80.2 MB/s 2023-11-06T03:14:48.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109159 s, 96.1 MB/s 2023-11-06T03:14:48.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.857 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125353 s, 83.7 MB/s 2023-11-06T03:14:48.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.870 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107687 s, 97.4 MB/s 2023-11-06T03:14:48.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:48.872 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:48.872 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:48.872 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.8 2023-11-06T03:14:48.873 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:48.873 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:48.873 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:48.874 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:48.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140135 s, 74.8 MB/s 2023-11-06T03:14:48.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.963 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135403 s, 77.4 MB/s 2023-11-06T03:14:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115292 s, 90.9 MB/s 2023-11-06T03:14:48.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:48.995 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:48.995 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147823 s, 70.9 MB/s 2023-11-06T03:14:49.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.010 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011357 s, 92.3 MB/s 2023-11-06T03:14:49.026 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.027 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.027 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139371 s, 75.2 MB/s 2023-11-06T03:14:49.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.042 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.042 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126741 s, 82.7 MB/s 2023-11-06T03:14:49.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.057 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105669 s, 99.2 MB/s 2023-11-06T03:14:49.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.073 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.073 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135916 s, 77.1 MB/s 2023-11-06T03:14:49.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.089 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.089 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127145 s, 82.5 MB/s 2023-11-06T03:14:49.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.107 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147063 s, 71.3 MB/s 2023-11-06T03:14:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111615 s, 93.9 MB/s 2023-11-06T03:14:49.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012907 s, 81.2 MB/s 2023-11-06T03:14:49.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130943 s, 80.1 MB/s 2023-11-06T03:14:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126082 s, 83.2 MB/s 2023-11-06T03:14:49.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138681 s, 75.6 MB/s 2023-11-06T03:14:49.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124492 s, 84.2 MB/s 2023-11-06T03:14:49.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134093 s, 78.2 MB/s 2023-11-06T03:14:49.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137786 s, 76.1 MB/s 2023-11-06T03:14:49.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138862 s, 75.5 MB/s 2023-11-06T03:14:49.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127154 s, 82.5 MB/s 2023-11-06T03:14:49.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.287 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.287 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135992 s, 77.1 MB/s 2023-11-06T03:14:49.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.303 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134665 s, 77.9 MB/s 2023-11-06T03:14:49.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.321 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139426 s, 75.2 MB/s 2023-11-06T03:14:49.335 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.336 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.336 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122373 s, 85.7 MB/s 2023-11-06T03:14:49.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.354 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144541 s, 72.5 MB/s 2023-11-06T03:14:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115461 s, 90.8 MB/s 2023-11-06T03:14:49.386 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.386 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.386 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148609 s, 70.6 MB/s 2023-11-06T03:14:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.400 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.401 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108161 s, 96.9 MB/s 2023-11-06T03:14:49.417 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.417 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.417 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137919 s, 76.0 MB/s 2023-11-06T03:14:49.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136896 s, 76.6 MB/s 2023-11-06T03:14:49.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129421 s, 81.0 MB/s 2023-11-06T03:14:49.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:49.451 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:49.452 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:49.452 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.9 2023-11-06T03:14:49.452 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:49.453 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:49.453 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:49.453 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:49.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101172 s, 104 MB/s 2023-11-06T03:14:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011274 s, 93.0 MB/s 2023-11-06T03:14:49.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.552 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117272 s, 89.4 MB/s 2023-11-06T03:14:49.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130461 s, 80.4 MB/s 2023-11-06T03:14:49.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.587 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.587 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149666 s, 70.1 MB/s 2023-11-06T03:14:49.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132403 s, 79.2 MB/s 2023-11-06T03:14:49.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142022 s, 73.8 MB/s 2023-11-06T03:14:49.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136335 s, 76.9 MB/s 2023-11-06T03:14:49.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.653 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01183 s, 88.6 MB/s 2023-11-06T03:14:49.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.670 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142588 s, 73.5 MB/s 2023-11-06T03:14:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.685 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125328 s, 83.7 MB/s 2023-11-06T03:14:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.703 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014546 s, 72.1 MB/s 2023-11-06T03:14:49.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131041 s, 80.0 MB/s 2023-11-06T03:14:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114446 s, 91.6 MB/s 2023-11-06T03:14:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010937 s, 95.9 MB/s 2023-11-06T03:14:49.764 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.765 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013523 s, 77.5 MB/s 2023-11-06T03:14:49.782 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.782 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.783 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136096 s, 77.0 MB/s 2023-11-06T03:14:49.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.799 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.799 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127886 s, 82.0 MB/s 2023-11-06T03:14:49.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.817 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153593 s, 68.3 MB/s 2023-11-06T03:14:49.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132187 s, 79.3 MB/s 2023-11-06T03:14:49.857 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.858 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.858 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140471 s, 74.6 MB/s 2023-11-06T03:14:49.874 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142676 s, 73.5 MB/s 2023-11-06T03:14:49.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.892 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.892 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013495 s, 77.7 MB/s 2023-11-06T03:14:49.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131027 s, 80.0 MB/s 2023-11-06T03:14:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.926 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129507 s, 81.0 MB/s 2023-11-06T03:14:49.942 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.942 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.942 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136914 s, 76.6 MB/s 2023-11-06T03:14:49.958 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.959 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132826 s, 78.9 MB/s 2023-11-06T03:14:49.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.975 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01352 s, 77.6 MB/s 2023-11-06T03:14:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:49.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012992 s, 80.7 MB/s 2023-11-06T03:14:50.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.006 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.006 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123531 s, 84.9 MB/s 2023-11-06T03:14:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124048 s, 84.5 MB/s 2023-11-06T03:14:50.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.037 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123373 s, 85.0 MB/s 2023-11-06T03:14:50.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:50.039 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:50.040 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:50.040 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.10 2023-11-06T03:14:50.041 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:50.041 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:50.041 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:50.042 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:50.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124027 s, 84.5 MB/s 2023-11-06T03:14:50.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132804 s, 79.0 MB/s 2023-11-06T03:14:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147861 s, 70.9 MB/s 2023-11-06T03:14:50.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.156 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136529 s, 76.8 MB/s 2023-11-06T03:14:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.172 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132236 s, 79.3 MB/s 2023-11-06T03:14:50.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.186 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109992 s, 95.3 MB/s 2023-11-06T03:14:50.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.199 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.199 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010709 s, 97.9 MB/s 2023-11-06T03:14:50.213 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.214 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.214 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122784 s, 85.4 MB/s 2023-11-06T03:14:50.226 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.227 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.227 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101061 s, 104 MB/s 2023-11-06T03:14:50.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.242 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.242 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125645 s, 83.5 MB/s 2023-11-06T03:14:50.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.257 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.258 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130376 s, 80.4 MB/s 2023-11-06T03:14:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.274 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128941 s, 81.3 MB/s 2023-11-06T03:14:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.288 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120629 s, 86.9 MB/s 2023-11-06T03:14:50.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.304 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.304 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129531 s, 81.0 MB/s 2023-11-06T03:14:50.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.317 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.317 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106138 s, 98.8 MB/s 2023-11-06T03:14:50.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.329 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102503 s, 102 MB/s 2023-11-06T03:14:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118598 s, 88.4 MB/s 2023-11-06T03:14:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.359 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121947 s, 86.0 MB/s 2023-11-06T03:14:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.372 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.373 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106991 s, 98.0 MB/s 2023-11-06T03:14:50.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.390 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014287 s, 73.4 MB/s 2023-11-06T03:14:50.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110274 s, 95.1 MB/s 2023-11-06T03:14:50.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129923 s, 80.7 MB/s 2023-11-06T03:14:50.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.433 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.433 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122147 s, 85.8 MB/s 2023-11-06T03:14:50.449 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138327 s, 75.8 MB/s 2023-11-06T03:14:50.464 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.465 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122437 s, 85.6 MB/s 2023-11-06T03:14:50.480 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.481 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135666 s, 77.3 MB/s 2023-11-06T03:14:50.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.494 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109366 s, 95.9 MB/s 2023-11-06T03:14:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133914 s, 78.3 MB/s 2023-11-06T03:14:50.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.526 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122957 s, 85.3 MB/s 2023-11-06T03:14:50.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.542 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.542 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135711 s, 77.3 MB/s 2023-11-06T03:14:50.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.558 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.558 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129599 s, 80.9 MB/s 2023-11-06T03:14:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.571 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103892 s, 101 MB/s 2023-11-06T03:14:50.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:50.572 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:50.573 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:50.573 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.11 2023-11-06T03:14:50.574 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:50.574 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:50.575 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:50.575 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:50.645 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.646 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.646 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137153 s, 76.5 MB/s 2023-11-06T03:14:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.663 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136902 s, 76.6 MB/s 2023-11-06T03:14:50.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.678 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129772 s, 80.8 MB/s 2023-11-06T03:14:50.690 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.691 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.691 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100923 s, 104 MB/s 2023-11-06T03:14:50.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.704 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.705 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108465 s, 96.7 MB/s 2023-11-06T03:14:50.718 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.719 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.719 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117667 s, 89.1 MB/s 2023-11-06T03:14:50.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.733 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113068 s, 92.7 MB/s 2023-11-06T03:14:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.749 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126591 s, 82.8 MB/s 2023-11-06T03:14:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768108 s, 137 MB/s 2023-11-06T03:14:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.771 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103618 s, 101 MB/s 2023-11-06T03:14:50.780 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.781 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.781 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757208 s, 138 MB/s 2023-11-06T03:14:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107876 s, 97.2 MB/s 2023-11-06T03:14:50.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.807 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.807 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113601 s, 92.3 MB/s 2023-11-06T03:14:50.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.821 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116278 s, 90.2 MB/s 2023-11-06T03:14:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105365 s, 99.5 MB/s 2023-11-06T03:14:50.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.856 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.856 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765113 s, 137 MB/s 2023-11-06T03:14:50.869 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.870 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119175 s, 88.0 MB/s 2023-11-06T03:14:50.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.883 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111541 s, 94.0 MB/s 2023-11-06T03:14:50.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.896 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010729 s, 97.7 MB/s 2023-11-06T03:14:50.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120609 s, 86.9 MB/s 2023-11-06T03:14:50.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102687 s, 102 MB/s 2023-11-06T03:14:50.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107987 s, 97.1 MB/s 2023-11-06T03:14:50.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.949 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109513 s, 95.7 MB/s 2023-11-06T03:14:50.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.961 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101275 s, 104 MB/s 2023-11-06T03:14:50.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.975 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122926 s, 85.3 MB/s 2023-11-06T03:14:50.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:50.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:50.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112101 s, 93.5 MB/s 2023-11-06T03:14:51.001 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.002 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106324 s, 98.6 MB/s 2023-11-06T03:14:51.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144089 s, 72.8 MB/s 2023-11-06T03:14:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106204 s, 98.7 MB/s 2023-11-06T03:14:51.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.044 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.044 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106374 s, 98.6 MB/s 2023-11-06T03:14:51.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.057 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.057 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101922 s, 103 MB/s 2023-11-06T03:14:51.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804686 s, 130 MB/s 2023-11-06T03:14:51.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:51.069 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:51.069 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:51.069 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.12 2023-11-06T03:14:51.070 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:51.070 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:51.071 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:51.071 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.140 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125653 s, 83.4 MB/s 2023-11-06T03:14:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132078 s, 79.4 MB/s 2023-11-06T03:14:51.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.169 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.169 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118968 s, 88.1 MB/s 2023-11-06T03:14:51.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.181 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.181 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934209 s, 112 MB/s 2023-11-06T03:14:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.196 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123476 s, 84.9 MB/s 2023-11-06T03:14:51.211 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.212 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.212 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118936 s, 88.2 MB/s 2023-11-06T03:14:51.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.225 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.226 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115894 s, 90.5 MB/s 2023-11-06T03:14:51.239 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.240 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.240 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115022 s, 91.2 MB/s 2023-11-06T03:14:51.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.255 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.255 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131691 s, 79.6 MB/s 2023-11-06T03:14:51.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.268 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104113 s, 101 MB/s 2023-11-06T03:14:51.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134428 s, 78.0 MB/s 2023-11-06T03:14:51.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125262 s, 83.7 MB/s 2023-11-06T03:14:51.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.313 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106546 s, 98.4 MB/s 2023-11-06T03:14:51.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.330 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142502 s, 73.6 MB/s 2023-11-06T03:14:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.346 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131054 s, 80.0 MB/s 2023-11-06T03:14:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.362 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132586 s, 79.1 MB/s 2023-11-06T03:14:51.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.379 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.379 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131361 s, 79.8 MB/s 2023-11-06T03:14:51.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.397 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140972 s, 74.4 MB/s 2023-11-06T03:14:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.414 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133381 s, 78.6 MB/s 2023-11-06T03:14:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.430 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127222 s, 82.4 MB/s 2023-11-06T03:14:51.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137209 s, 76.4 MB/s 2023-11-06T03:14:51.462 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.462 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.463 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137414 s, 76.3 MB/s 2023-11-06T03:14:51.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.479 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.479 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137077 s, 76.5 MB/s 2023-11-06T03:14:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.494 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116728 s, 89.8 MB/s 2023-11-06T03:14:51.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.509 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121161 s, 86.5 MB/s 2023-11-06T03:14:51.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.524 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.524 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114336 s, 91.7 MB/s 2023-11-06T03:14:51.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.539 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.540 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128965 s, 81.3 MB/s 2023-11-06T03:14:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.557 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148052 s, 70.8 MB/s 2023-11-06T03:14:51.574 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.575 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.575 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140527 s, 74.6 MB/s 2023-11-06T03:14:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.589 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107447 s, 97.6 MB/s 2023-11-06T03:14:51.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.604 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127739 s, 82.1 MB/s 2023-11-06T03:14:51.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.620 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.620 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012938 s, 81.0 MB/s 2023-11-06T03:14:51.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:51.622 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:51.622 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:51.623 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.13 2023-11-06T03:14:51.623 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:51.624 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:51.624 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:51.624 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:51.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.707 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.707 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119452 s, 87.8 MB/s 2023-11-06T03:14:51.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.711 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127116 s, 82.5 MB/s 2023-11-06T03:14:51.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.727 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.727 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132253 s, 79.3 MB/s 2023-11-06T03:14:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117567 s, 89.2 MB/s 2023-11-06T03:14:51.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.758 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134491 s, 78.0 MB/s 2023-11-06T03:14:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.771 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104357 s, 100 MB/s 2023-11-06T03:14:51.785 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118502 s, 88.5 MB/s 2023-11-06T03:14:51.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.798 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.799 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100709 s, 104 MB/s 2023-11-06T03:14:51.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.814 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.815 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129989 s, 80.7 MB/s 2023-11-06T03:14:51.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.831 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141641 s, 74.0 MB/s 2023-11-06T03:14:51.852 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.852 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01013 s, 104 MB/s 2023-11-06T03:14:51.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.866 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111814 s, 93.8 MB/s 2023-11-06T03:14:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124719 s, 84.1 MB/s 2023-11-06T03:14:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.895 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106977 s, 98.0 MB/s 2023-11-06T03:14:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861426 s, 122 MB/s 2023-11-06T03:14:51.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.921 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127071 s, 82.5 MB/s 2023-11-06T03:14:51.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.936 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122011 s, 85.9 MB/s 2023-11-06T03:14:51.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.950 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.950 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011347 s, 92.4 MB/s 2023-11-06T03:14:51.964 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.965 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117886 s, 88.9 MB/s 2023-11-06T03:14:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.979 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111595 s, 94.0 MB/s 2023-11-06T03:14:51.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:51.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:51.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117604 s, 89.2 MB/s 2023-11-06T03:14:52.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123427 s, 85.0 MB/s 2023-11-06T03:14:52.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.025 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112261 s, 93.4 MB/s 2023-11-06T03:14:52.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.040 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146707 s, 71.5 MB/s 2023-11-06T03:14:52.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.055 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112073 s, 93.6 MB/s 2023-11-06T03:14:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.069 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.070 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121504 s, 86.3 MB/s 2023-11-06T03:14:52.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.087 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135221 s, 77.5 MB/s 2023-11-06T03:14:52.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.101 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011868 s, 88.4 MB/s 2023-11-06T03:14:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.117 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012781 s, 82.0 MB/s 2023-11-06T03:14:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.130 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957757 s, 109 MB/s 2023-11-06T03:14:52.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.147 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.147 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136501 s, 76.8 MB/s 2023-11-06T03:14:52.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127427 s, 82.3 MB/s 2023-11-06T03:14:52.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:52.165 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:52.165 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:52.165 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.14 2023-11-06T03:14:52.166 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:52.166 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:52.166 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:52.167 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.234 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113033 s, 92.8 MB/s 2023-11-06T03:14:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.248 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128914 s, 81.3 MB/s 2023-11-06T03:14:52.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.264 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.264 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115543 s, 90.8 MB/s 2023-11-06T03:14:52.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.279 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122384 s, 85.7 MB/s 2023-11-06T03:14:52.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.295 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.296 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123626 s, 84.8 MB/s 2023-11-06T03:14:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140941 s, 74.4 MB/s 2023-11-06T03:14:52.327 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.327 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.327 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011808 s, 88.8 MB/s 2023-11-06T03:14:52.342 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.342 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.343 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121774 s, 86.1 MB/s 2023-11-06T03:14:52.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.358 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.359 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125986 s, 83.2 MB/s 2023-11-06T03:14:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.374 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125495 s, 83.6 MB/s 2023-11-06T03:14:52.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.390 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125028 s, 83.9 MB/s 2023-11-06T03:14:52.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.407 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126062 s, 83.2 MB/s 2023-11-06T03:14:52.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.423 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128446 s, 81.6 MB/s 2023-11-06T03:14:52.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.439 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127851 s, 82.0 MB/s 2023-11-06T03:14:52.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.455 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.455 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132327 s, 79.2 MB/s 2023-11-06T03:14:52.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.473 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013625 s, 77.0 MB/s 2023-11-06T03:14:52.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.490 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.490 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140925 s, 74.4 MB/s 2023-11-06T03:14:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103741 s, 101 MB/s 2023-11-06T03:14:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131771 s, 79.6 MB/s 2023-11-06T03:14:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117534 s, 89.2 MB/s 2023-11-06T03:14:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117955 s, 88.9 MB/s 2023-11-06T03:14:52.568 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.568 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.568 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138227 s, 75.9 MB/s 2023-11-06T03:14:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128497 s, 81.6 MB/s 2023-11-06T03:14:52.600 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.600 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.600 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136526 s, 76.8 MB/s 2023-11-06T03:14:52.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119962 s, 87.4 MB/s 2023-11-06T03:14:52.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.632 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131665 s, 79.6 MB/s 2023-11-06T03:14:52.647 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.648 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124123 s, 84.5 MB/s 2023-11-06T03:14:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.665 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140412 s, 74.7 MB/s 2023-11-06T03:14:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.678 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01003 s, 105 MB/s 2023-11-06T03:14:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.694 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.694 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127515 s, 82.2 MB/s 2023-11-06T03:14:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012944 s, 81.0 MB/s 2023-11-06T03:14:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122969 s, 85.3 MB/s 2023-11-06T03:14:52.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:52.727 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:52.727 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:52.728 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.15 2023-11-06T03:14:52.728 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:52.729 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:52.729 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:52.729 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:52.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137884 s, 76.0 MB/s 2023-11-06T03:14:52.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.811 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131388 s, 79.8 MB/s 2023-11-06T03:14:52.825 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.826 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122274 s, 85.8 MB/s 2023-11-06T03:14:52.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.863 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.863 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117813 s, 89.0 MB/s 2023-11-06T03:14:52.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.881 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.881 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014491 s, 72.4 MB/s 2023-11-06T03:14:52.895 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.896 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.896 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116522 s, 90.0 MB/s 2023-11-06T03:14:52.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.908 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.909 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989893 s, 106 MB/s 2023-11-06T03:14:52.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.920 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.920 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900625 s, 116 MB/s 2023-11-06T03:14:52.934 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.935 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011975 s, 87.6 MB/s 2023-11-06T03:14:52.948 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.949 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.949 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110726 s, 94.7 MB/s 2023-11-06T03:14:52.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.963 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114821 s, 91.3 MB/s 2023-11-06T03:14:52.975 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.976 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010028 s, 105 MB/s 2023-11-06T03:14:52.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:52.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:52.989 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983634 s, 107 MB/s 2023-11-06T03:14:53.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.005 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.005 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137251 s, 76.4 MB/s 2023-11-06T03:14:53.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.018 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.018 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101993 s, 103 MB/s 2023-11-06T03:14:53.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.033 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.033 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124432 s, 84.3 MB/s 2023-11-06T03:14:53.046 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.047 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106228 s, 98.7 MB/s 2023-11-06T03:14:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.060 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105311 s, 99.6 MB/s 2023-11-06T03:14:53.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.075 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.076 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126593 s, 82.8 MB/s 2023-11-06T03:14:53.091 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.092 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.092 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130463 s, 80.4 MB/s 2023-11-06T03:14:53.104 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.104 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.105 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010477 s, 100 MB/s 2023-11-06T03:14:53.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014415 s, 72.7 MB/s 2023-11-06T03:14:53.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.138 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137 s, 76.5 MB/s 2023-11-06T03:14:53.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135553 s, 77.4 MB/s 2023-11-06T03:14:53.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013323 s, 78.7 MB/s 2023-11-06T03:14:53.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.188 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132699 s, 79.0 MB/s 2023-11-06T03:14:53.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130728 s, 80.2 MB/s 2023-11-06T03:14:53.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135319 s, 77.5 MB/s 2023-11-06T03:14:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.235 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124085 s, 84.5 MB/s 2023-11-06T03:14:53.250 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.251 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127603 s, 82.2 MB/s 2023-11-06T03:14:53.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.267 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.267 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130071 s, 80.6 MB/s 2023-11-06T03:14:53.282 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.283 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.283 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130207 s, 80.5 MB/s 2023-11-06T03:14:53.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:53.284 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:53.285 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:53.285 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.16 2023-11-06T03:14:53.286 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:53.286 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:53.287 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:53.287 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:53.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.355 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.355 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011683 s, 89.8 MB/s 2023-11-06T03:14:53.369 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.370 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121442 s, 86.3 MB/s 2023-11-06T03:14:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.384 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113819 s, 92.1 MB/s 2023-11-06T03:14:53.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.396 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.396 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095534 s, 110 MB/s 2023-11-06T03:14:53.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.412 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.413 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131559 s, 79.7 MB/s 2023-11-06T03:14:53.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.424 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092744 s, 113 MB/s 2023-11-06T03:14:53.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.437 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922422 s, 114 MB/s 2023-11-06T03:14:53.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126419 s, 82.9 MB/s 2023-11-06T03:14:53.467 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.468 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.468 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128966 s, 81.3 MB/s 2023-11-06T03:14:53.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.482 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111779 s, 93.8 MB/s 2023-11-06T03:14:53.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.497 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011989 s, 87.5 MB/s 2023-11-06T03:14:53.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102529 s, 102 MB/s 2023-11-06T03:14:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.522 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.523 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986875 s, 106 MB/s 2023-11-06T03:14:53.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.538 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122812 s, 85.4 MB/s 2023-11-06T03:14:53.553 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.554 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.554 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132659 s, 79.0 MB/s 2023-11-06T03:14:53.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125965 s, 83.2 MB/s 2023-11-06T03:14:53.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.584 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108919 s, 96.3 MB/s 2023-11-06T03:14:53.599 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.599 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.600 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131302 s, 79.9 MB/s 2023-11-06T03:14:53.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.616 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130809 s, 80.2 MB/s 2023-11-06T03:14:53.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.632 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.632 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134076 s, 78.2 MB/s 2023-11-06T03:14:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.648 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.649 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137292 s, 76.4 MB/s 2023-11-06T03:14:53.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.665 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.665 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134806 s, 77.8 MB/s 2023-11-06T03:14:53.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.681 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.681 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126012 s, 83.2 MB/s 2023-11-06T03:14:53.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.698 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013746 s, 76.3 MB/s 2023-11-06T03:14:53.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.712 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119139 s, 88.0 MB/s 2023-11-06T03:14:53.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.728 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.729 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134587 s, 77.9 MB/s 2023-11-06T03:14:53.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.744 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136643 s, 76.7 MB/s 2023-11-06T03:14:53.761 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.761 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.761 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141342 s, 74.2 MB/s 2023-11-06T03:14:53.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.777 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.778 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134977 s, 77.7 MB/s 2023-11-06T03:14:53.793 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.794 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.794 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013537 s, 77.5 MB/s 2023-11-06T03:14:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.810 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013612 s, 77.0 MB/s 2023-11-06T03:14:53.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.830 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168054 s, 62.4 MB/s 2023-11-06T03:14:53.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:53.832 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:53.832 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:53.833 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.17 2023-11-06T03:14:53.833 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:53.834 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:53.834 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:53.834 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:53.901 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127134 s, 82.5 MB/s 2023-11-06T03:14:53.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.918 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134745 s, 77.8 MB/s 2023-11-06T03:14:53.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.931 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.931 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010648 s, 98.5 MB/s 2023-11-06T03:14:53.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.946 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122005 s, 85.9 MB/s 2023-11-06T03:14:53.959 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114058 s, 91.9 MB/s 2023-11-06T03:14:53.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118569 s, 88.4 MB/s 2023-11-06T03:14:53.987 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.988 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:53.988 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105634 s, 99.3 MB/s 2023-11-06T03:14:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:53.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958479 s, 109 MB/s 2023-11-06T03:14:54.013 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.013 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.014 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115391 s, 90.9 MB/s 2023-11-06T03:14:54.028 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.029 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.029 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111728 s, 93.9 MB/s 2023-11-06T03:14:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.042 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114024 s, 92.0 MB/s 2023-11-06T03:14:54.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.054 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975627 s, 107 MB/s 2023-11-06T03:14:54.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.065 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.065 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876338 s, 120 MB/s 2023-11-06T03:14:54.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.081 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.081 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119101 s, 88.0 MB/s 2023-11-06T03:14:54.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.095 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126007 s, 83.2 MB/s 2023-11-06T03:14:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121178 s, 86.5 MB/s 2023-11-06T03:14:54.124 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.125 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116799 s, 89.8 MB/s 2023-11-06T03:14:54.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100665 s, 104 MB/s 2023-11-06T03:14:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.149 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867649 s, 121 MB/s 2023-11-06T03:14:54.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.163 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.163 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119793 s, 87.5 MB/s 2023-11-06T03:14:54.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.179 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111239 s, 94.3 MB/s 2023-11-06T03:14:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010402 s, 101 MB/s 2023-11-06T03:14:54.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.202 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981133 s, 107 MB/s 2023-11-06T03:14:54.216 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.217 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.217 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114532 s, 91.6 MB/s 2023-11-06T03:14:54.231 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.232 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.232 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122344 s, 85.7 MB/s 2023-11-06T03:14:54.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.246 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.246 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118886 s, 88.2 MB/s 2023-11-06T03:14:54.261 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.262 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131984 s, 79.4 MB/s 2023-11-06T03:14:54.278 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.279 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143244 s, 73.2 MB/s 2023-11-06T03:14:54.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.294 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120679 s, 86.9 MB/s 2023-11-06T03:14:54.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.309 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117195 s, 89.5 MB/s 2023-11-06T03:14:54.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113156 s, 92.7 MB/s 2023-11-06T03:14:54.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011614 s, 90.3 MB/s 2023-11-06T03:14:54.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:54.339 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:54.339 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:54.339 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.18 2023-11-06T03:14:54.340 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:54.340 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:54.340 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:54.341 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:54.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.406 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.406 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138526 s, 75.7 MB/s 2023-11-06T03:14:54.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.423 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133967 s, 78.3 MB/s 2023-11-06T03:14:54.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.438 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128919 s, 81.3 MB/s 2023-11-06T03:14:54.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.452 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105396 s, 99.5 MB/s 2023-11-06T03:14:54.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.466 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113565 s, 92.3 MB/s 2023-11-06T03:14:54.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.481 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.482 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129259 s, 81.1 MB/s 2023-11-06T03:14:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.496 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115932 s, 90.4 MB/s 2023-11-06T03:14:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116554 s, 90.0 MB/s 2023-11-06T03:14:54.523 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.524 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.524 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109815 s, 95.5 MB/s 2023-11-06T03:14:54.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.537 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.537 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105582 s, 99.3 MB/s 2023-11-06T03:14:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.552 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117068 s, 89.6 MB/s 2023-11-06T03:14:54.567 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.567 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.567 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01214 s, 86.4 MB/s 2023-11-06T03:14:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.584 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139154 s, 75.4 MB/s 2023-11-06T03:14:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.598 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.599 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121777 s, 86.1 MB/s 2023-11-06T03:14:54.612 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.612 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.612 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110399 s, 95.0 MB/s 2023-11-06T03:14:54.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.630 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144493 s, 72.6 MB/s 2023-11-06T03:14:54.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.643 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.644 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111759 s, 93.8 MB/s 2023-11-06T03:14:54.658 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.659 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.659 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124268 s, 84.4 MB/s 2023-11-06T03:14:54.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.675 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128579 s, 81.6 MB/s 2023-11-06T03:14:54.692 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.693 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149145 s, 70.3 MB/s 2023-11-06T03:14:54.708 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.709 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.709 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013474 s, 77.8 MB/s 2023-11-06T03:14:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012665 s, 82.8 MB/s 2023-11-06T03:14:54.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141612 s, 74.0 MB/s 2023-11-06T03:14:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.757 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129923 s, 80.7 MB/s 2023-11-06T03:14:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.771 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104212 s, 101 MB/s 2023-11-06T03:14:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.786 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120149 s, 87.3 MB/s 2023-11-06T03:14:54.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.802 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.802 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128681 s, 81.5 MB/s 2023-11-06T03:14:54.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141172 s, 74.3 MB/s 2023-11-06T03:14:54.835 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.836 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.836 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141937 s, 73.9 MB/s 2023-11-06T03:14:54.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.867 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137586 s, 76.2 MB/s 2023-11-06T03:14:54.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.883 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.883 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134073 s, 78.2 MB/s 2023-11-06T03:14:54.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148979 s, 70.4 MB/s 2023-11-06T03:14:54.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:54.907 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:54.908 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:54.908 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.19 2023-11-06T03:14:54.908 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:54.909 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:54.909 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:54.910 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:54.980 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.980 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.981 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014855 s, 70.6 MB/s 2023-11-06T03:14:54.998 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:54.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:54.999 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152292 s, 68.9 MB/s 2023-11-06T03:14:55.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.016 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.016 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142409 s, 73.6 MB/s 2023-11-06T03:14:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125106 s, 83.8 MB/s 2023-11-06T03:14:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124639 s, 84.1 MB/s 2023-11-06T03:14:55.063 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132818 s, 78.9 MB/s 2023-11-06T03:14:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125546 s, 83.5 MB/s 2023-11-06T03:14:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123705 s, 84.8 MB/s 2023-11-06T03:14:55.109 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118853 s, 88.2 MB/s 2023-11-06T03:14:55.125 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.126 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.126 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129239 s, 81.1 MB/s 2023-11-06T03:14:55.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.141 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126394 s, 83.0 MB/s 2023-11-06T03:14:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.157 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013018 s, 80.5 MB/s 2023-11-06T03:14:55.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.174 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136469 s, 76.8 MB/s 2023-11-06T03:14:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142162 s, 73.8 MB/s 2023-11-06T03:14:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.208 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114405 s, 91.7 MB/s 2023-11-06T03:14:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109325 s, 95.9 MB/s 2023-11-06T03:14:55.237 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128547 s, 81.6 MB/s 2023-11-06T03:14:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130979 s, 80.1 MB/s 2023-11-06T03:14:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122703 s, 85.5 MB/s 2023-11-06T03:14:55.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136109 s, 77.0 MB/s 2023-11-06T03:14:55.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.301 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117041 s, 89.6 MB/s 2023-11-06T03:14:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.318 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140712 s, 74.5 MB/s 2023-11-06T03:14:55.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110611 s, 94.8 MB/s 2023-11-06T03:14:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125794 s, 83.4 MB/s 2023-11-06T03:14:55.363 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.364 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.364 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129104 s, 81.2 MB/s 2023-11-06T03:14:55.377 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.378 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108758 s, 96.4 MB/s 2023-11-06T03:14:55.393 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.394 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.394 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127421 s, 82.3 MB/s 2023-11-06T03:14:55.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.411 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138974 s, 75.5 MB/s 2023-11-06T03:14:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.425 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011484 s, 91.3 MB/s 2023-11-06T03:14:55.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.439 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.439 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112499 s, 93.2 MB/s 2023-11-06T03:14:55.454 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.454 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.454 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117371 s, 89.3 MB/s 2023-11-06T03:14:55.470 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.470 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.471 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136385 s, 76.9 MB/s 2023-11-06T03:14:55.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:55.472 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:55.473 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:55.473 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.20 2023-11-06T03:14:55.473 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:55.474 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:55.474 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:55.474 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:55.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116791 s, 89.8 MB/s 2023-11-06T03:14:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012521 s, 83.7 MB/s 2023-11-06T03:14:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121572 s, 86.3 MB/s 2023-11-06T03:14:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125834 s, 83.3 MB/s 2023-11-06T03:14:55.603 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.604 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.604 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114876 s, 91.3 MB/s 2023-11-06T03:14:55.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.622 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.622 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141948 s, 73.9 MB/s 2023-11-06T03:14:55.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134234 s, 78.1 MB/s 2023-11-06T03:14:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127821 s, 82.0 MB/s 2023-11-06T03:14:55.675 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.676 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.676 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173913 s, 60.3 MB/s 2023-11-06T03:14:55.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.693 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.693 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145324 s, 72.2 MB/s 2023-11-06T03:14:55.712 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.713 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167353 s, 62.7 MB/s 2023-11-06T03:14:55.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.726 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115095 s, 91.1 MB/s 2023-11-06T03:14:55.740 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120803 s, 86.8 MB/s 2023-11-06T03:14:55.754 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.755 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113193 s, 92.6 MB/s 2023-11-06T03:14:55.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.769 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.769 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122879 s, 85.3 MB/s 2023-11-06T03:14:55.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.783 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.783 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113116 s, 92.7 MB/s 2023-11-06T03:14:55.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.796 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.796 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104397 s, 100 MB/s 2023-11-06T03:14:55.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.809 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106888 s, 98.1 MB/s 2023-11-06T03:14:55.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.822 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.822 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114348 s, 91.7 MB/s 2023-11-06T03:14:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016663 s, 62.9 MB/s 2023-11-06T03:14:55.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.867 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110094 s, 95.2 MB/s 2023-11-06T03:14:55.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.880 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.880 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109543 s, 95.7 MB/s 2023-11-06T03:14:55.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.894 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115336 s, 90.9 MB/s 2023-11-06T03:14:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132014 s, 79.4 MB/s 2023-11-06T03:14:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.922 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.922 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978976 s, 107 MB/s 2023-11-06T03:14:55.936 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.937 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.937 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120451 s, 87.1 MB/s 2023-11-06T03:14:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.952 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.953 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129623 s, 80.9 MB/s 2023-11-06T03:14:55.968 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.969 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135745 s, 77.2 MB/s 2023-11-06T03:14:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:55.985 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133779 s, 78.4 MB/s 2023-11-06T03:14:56.002 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.003 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.003 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146596 s, 71.5 MB/s 2023-11-06T03:14:56.019 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.020 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138602 s, 75.7 MB/s 2023-11-06T03:14:56.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.036 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.037 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138206 s, 75.9 MB/s 2023-11-06T03:14:56.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:56.038 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:56.039 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:56.039 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.21 2023-11-06T03:14:56.040 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:56.040 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:56.040 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:56.041 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136639 s, 76.7 MB/s 2023-11-06T03:14:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125238 s, 83.7 MB/s 2023-11-06T03:14:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142191 s, 73.7 MB/s 2023-11-06T03:14:56.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.155 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.155 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013163 s, 79.7 MB/s 2023-11-06T03:14:56.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.171 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123215 s, 85.1 MB/s 2023-11-06T03:14:56.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.185 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.185 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114417 s, 91.6 MB/s 2023-11-06T03:14:56.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015049 s, 69.7 MB/s 2023-11-06T03:14:56.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132719 s, 79.0 MB/s 2023-11-06T03:14:56.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.237 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134804 s, 77.8 MB/s 2023-11-06T03:14:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137381 s, 76.3 MB/s 2023-11-06T03:14:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143553 s, 73.0 MB/s 2023-11-06T03:14:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013597 s, 77.1 MB/s 2023-11-06T03:14:56.303 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.304 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.304 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123503 s, 84.9 MB/s 2023-11-06T03:14:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.320 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133328 s, 78.6 MB/s 2023-11-06T03:14:56.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.337 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012961 s, 80.9 MB/s 2023-11-06T03:14:56.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.354 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143608 s, 73.0 MB/s 2023-11-06T03:14:56.370 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.371 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.371 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140641 s, 74.6 MB/s 2023-11-06T03:14:56.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.385 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.386 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112585 s, 93.1 MB/s 2023-11-06T03:14:56.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.401 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.401 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125787 s, 83.4 MB/s 2023-11-06T03:14:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146135 s, 71.8 MB/s 2023-11-06T03:14:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.434 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115057 s, 91.1 MB/s 2023-11-06T03:14:56.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.450 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131802 s, 79.6 MB/s 2023-11-06T03:14:56.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.466 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.467 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130865 s, 80.1 MB/s 2023-11-06T03:14:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.483 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.484 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138946 s, 75.5 MB/s 2023-11-06T03:14:56.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141075 s, 74.3 MB/s 2023-11-06T03:14:56.516 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.516 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.517 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127204 s, 82.4 MB/s 2023-11-06T03:14:56.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.534 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.534 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145133 s, 72.2 MB/s 2023-11-06T03:14:56.550 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.551 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.551 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135008 s, 77.7 MB/s 2023-11-06T03:14:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.564 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.565 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114254 s, 91.8 MB/s 2023-11-06T03:14:56.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.581 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.581 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139367 s, 75.2 MB/s 2023-11-06T03:14:56.596 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.597 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.597 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127061 s, 82.5 MB/s 2023-11-06T03:14:56.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.611 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.611 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111508 s, 94.0 MB/s 2023-11-06T03:14:56.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:56.612 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:56.613 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:56.613 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.22 2023-11-06T03:14:56.613 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:56.614 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:56.614 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:56.614 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:56.679 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118036 s, 88.8 MB/s 2023-11-06T03:14:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.697 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.698 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140839 s, 74.5 MB/s 2023-11-06T03:14:56.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131626 s, 79.7 MB/s 2023-11-06T03:14:56.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136629 s, 76.7 MB/s 2023-11-06T03:14:56.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.746 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.746 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127063 s, 82.5 MB/s 2023-11-06T03:14:56.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134337 s, 78.1 MB/s 2023-11-06T03:14:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.779 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133814 s, 78.4 MB/s 2023-11-06T03:14:56.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.795 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.795 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013568 s, 77.3 MB/s 2023-11-06T03:14:56.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.811 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.812 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131196 s, 79.9 MB/s 2023-11-06T03:14:56.826 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.827 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.827 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118032 s, 88.8 MB/s 2023-11-06T03:14:56.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.849 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.849 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191651 s, 54.7 MB/s 2023-11-06T03:14:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.885 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133854 s, 78.3 MB/s 2023-11-06T03:14:56.904 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154808 s, 67.7 MB/s 2023-11-06T03:14:56.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.924 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.925 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161747 s, 64.8 MB/s 2023-11-06T03:14:56.943 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.945 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.945 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147295 s, 71.2 MB/s 2023-11-06T03:14:56.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.960 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.960 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012598 s, 83.2 MB/s 2023-11-06T03:14:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137131 s, 76.5 MB/s 2023-11-06T03:14:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:56.993 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:56.994 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135454 s, 77.4 MB/s 2023-11-06T03:14:57.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.009 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.009 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124551 s, 84.2 MB/s 2023-11-06T03:14:57.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.024 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121242 s, 86.5 MB/s 2023-11-06T03:14:57.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107207 s, 97.8 MB/s 2023-11-06T03:14:57.052 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.053 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121889 s, 86.0 MB/s 2023-11-06T03:14:57.070 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.071 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.071 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144951 s, 72.3 MB/s 2023-11-06T03:14:57.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.086 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.087 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128702 s, 81.5 MB/s 2023-11-06T03:14:57.101 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.102 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.102 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118421 s, 88.5 MB/s 2023-11-06T03:14:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131425 s, 79.8 MB/s 2023-11-06T03:14:57.133 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.134 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.134 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120376 s, 87.1 MB/s 2023-11-06T03:14:57.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.154 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.154 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148439 s, 70.6 MB/s 2023-11-06T03:14:57.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.170 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.171 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145257 s, 72.2 MB/s 2023-11-06T03:14:57.186 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127444 s, 82.3 MB/s 2023-11-06T03:14:57.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.205 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.205 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01469 s, 71.4 MB/s 2023-11-06T03:14:57.223 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.224 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.224 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152333 s, 68.8 MB/s 2023-11-06T03:14:57.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:57.225 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:57.226 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:57.226 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.23 2023-11-06T03:14:57.227 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:57.227 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:57.228 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:57.228 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:57.291 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.291 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.292 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011738 s, 89.3 MB/s 2023-11-06T03:14:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.306 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120246 s, 87.2 MB/s 2023-11-06T03:14:57.320 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.321 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.321 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119758 s, 87.6 MB/s 2023-11-06T03:14:57.332 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.333 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.333 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946167 s, 111 MB/s 2023-11-06T03:14:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.347 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.348 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121821 s, 86.1 MB/s 2023-11-06T03:14:57.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.362 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.363 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122476 s, 85.6 MB/s 2023-11-06T03:14:57.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.376 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.377 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115515 s, 90.8 MB/s 2023-11-06T03:14:57.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.391 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.391 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119109 s, 88.0 MB/s 2023-11-06T03:14:57.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.405 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.406 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116812 s, 89.8 MB/s 2023-11-06T03:14:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137391 s, 76.3 MB/s 2023-11-06T03:14:57.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124186 s, 84.4 MB/s 2023-11-06T03:14:57.452 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.453 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125834 s, 83.3 MB/s 2023-11-06T03:14:57.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.472 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161813 s, 64.8 MB/s 2023-11-06T03:14:57.487 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.488 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.488 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121923 s, 86.0 MB/s 2023-11-06T03:14:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.503 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.504 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123671 s, 84.8 MB/s 2023-11-06T03:14:57.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.518 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.519 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120508 s, 87.0 MB/s 2023-11-06T03:14:57.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.532 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105959 s, 99.0 MB/s 2023-11-06T03:14:57.545 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.545 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.546 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111527 s, 94.0 MB/s 2023-11-06T03:14:57.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.560 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.561 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125452 s, 83.6 MB/s 2023-11-06T03:14:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144105 s, 72.8 MB/s 2023-11-06T03:14:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.591 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101236 s, 104 MB/s 2023-11-06T03:14:57.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.606 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.607 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128916 s, 81.3 MB/s 2023-11-06T03:14:57.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.621 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.621 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118845 s, 88.2 MB/s 2023-11-06T03:14:57.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.637 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.637 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134137 s, 78.2 MB/s 2023-11-06T03:14:57.651 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.652 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.652 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115353 s, 90.9 MB/s 2023-11-06T03:14:57.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.670 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150305 s, 69.8 MB/s 2023-11-06T03:14:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143394 s, 73.1 MB/s 2023-11-06T03:14:57.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.703 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.704 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139399 s, 75.2 MB/s 2023-11-06T03:14:57.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.720 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.720 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133815 s, 78.4 MB/s 2023-11-06T03:14:57.733 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.734 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.734 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108363 s, 96.8 MB/s 2023-11-06T03:14:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.747 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108362 s, 96.8 MB/s 2023-11-06T03:14:57.764 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.765 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.765 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145003 s, 72.3 MB/s 2023-11-06T03:14:57.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:57.770 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:57.770 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:57.770 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.24 2023-11-06T03:14:57.771 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:57.771 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:57.771 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:57.772 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.839 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144263 s, 72.7 MB/s 2023-11-06T03:14:57.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116745 s, 89.8 MB/s 2023-11-06T03:14:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.893 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012282 s, 85.4 MB/s 2023-11-06T03:14:57.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.906 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.906 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979243 s, 107 MB/s 2023-11-06T03:14:57.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.923 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.923 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140793 s, 74.5 MB/s 2023-11-06T03:14:57.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.941 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146613 s, 71.5 MB/s 2023-11-06T03:14:57.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.957 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.957 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135066 s, 77.6 MB/s 2023-11-06T03:14:57.973 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.974 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.974 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133387 s, 78.6 MB/s 2023-11-06T03:14:57.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:57.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:57.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142453 s, 73.6 MB/s 2023-11-06T03:14:58.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.007 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133614 s, 78.5 MB/s 2023-11-06T03:14:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115445 s, 90.8 MB/s 2023-11-06T03:14:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128407 s, 81.7 MB/s 2023-11-06T03:14:58.054 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.055 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.055 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140151 s, 74.8 MB/s 2023-11-06T03:14:58.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.072 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.072 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147858 s, 70.9 MB/s 2023-11-06T03:14:58.088 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.088 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.089 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128524 s, 81.6 MB/s 2023-11-06T03:14:58.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.106 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138538 s, 75.7 MB/s 2023-11-06T03:14:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.123 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134951 s, 77.7 MB/s 2023-11-06T03:14:58.140 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.141 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.141 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135119 s, 77.6 MB/s 2023-11-06T03:14:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.158 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137882 s, 76.0 MB/s 2023-11-06T03:14:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.172 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108025 s, 97.1 MB/s 2023-11-06T03:14:58.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.189 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138794 s, 75.5 MB/s 2023-11-06T03:14:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.204 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124701 s, 84.1 MB/s 2023-11-06T03:14:58.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120579 s, 87.0 MB/s 2023-11-06T03:14:58.233 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.234 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.234 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120524 s, 87.0 MB/s 2023-11-06T03:14:58.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.247 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.248 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110533 s, 94.9 MB/s 2023-11-06T03:14:58.262 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.263 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.264 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118601 s, 88.4 MB/s 2023-11-06T03:14:58.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.279 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.279 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135757 s, 77.2 MB/s 2023-11-06T03:14:58.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.294 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.295 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117582 s, 89.2 MB/s 2023-11-06T03:14:58.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.308 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.308 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106459 s, 98.5 MB/s 2023-11-06T03:14:58.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.323 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.323 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121345 s, 86.4 MB/s 2023-11-06T03:14:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125675 s, 83.4 MB/s 2023-11-06T03:14:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121005 s, 86.7 MB/s 2023-11-06T03:14:58.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:58.356 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:58.356 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:58.356 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.25 2023-11-06T03:14:58.357 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:58.357 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:58.358 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:58.358 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:58.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.429 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130748 s, 80.2 MB/s 2023-11-06T03:14:58.443 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.444 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.444 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117418 s, 89.3 MB/s 2023-11-06T03:14:58.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.459 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.459 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125933 s, 83.3 MB/s 2023-11-06T03:14:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.472 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100857 s, 104 MB/s 2023-11-06T03:14:58.485 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.486 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.486 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108202 s, 96.9 MB/s 2023-11-06T03:14:58.500 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.501 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.501 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119916 s, 87.4 MB/s 2023-11-06T03:14:58.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.513 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.513 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099871 s, 105 MB/s 2023-11-06T03:14:58.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.529 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.529 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133844 s, 78.3 MB/s 2023-11-06T03:14:58.543 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.543 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.543 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111933 s, 93.7 MB/s 2023-11-06T03:14:58.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.556 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.556 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010209 s, 103 MB/s 2023-11-06T03:14:58.569 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.570 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.570 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113566 s, 92.3 MB/s 2023-11-06T03:14:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.583 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106448 s, 98.5 MB/s 2023-11-06T03:14:58.595 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.595 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.595 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958899 s, 109 MB/s 2023-11-06T03:14:58.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.610 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.610 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121786 s, 86.1 MB/s 2023-11-06T03:14:58.625 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.625 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.625 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126651 s, 82.8 MB/s 2023-11-06T03:14:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139362 s, 75.2 MB/s 2023-11-06T03:14:58.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.657 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011935 s, 87.9 MB/s 2023-11-06T03:14:58.673 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.674 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.674 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140899 s, 74.4 MB/s 2023-11-06T03:14:58.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.689 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.690 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135711 s, 77.3 MB/s 2023-11-06T03:14:58.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.701 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903172 s, 116 MB/s 2023-11-06T03:14:58.717 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.717 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134543 s, 77.9 MB/s 2023-11-06T03:14:58.730 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102288 s, 103 MB/s 2023-11-06T03:14:58.742 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.743 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101097 s, 104 MB/s 2023-11-06T03:14:58.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131649 s, 79.6 MB/s 2023-11-06T03:14:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012591 s, 83.3 MB/s 2023-11-06T03:14:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123218 s, 85.1 MB/s 2023-11-06T03:14:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012492 s, 83.9 MB/s 2023-11-06T03:14:58.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.817 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.818 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109344 s, 95.9 MB/s 2023-11-06T03:14:58.832 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.833 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129369 s, 81.1 MB/s 2023-11-06T03:14:58.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.847 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115185 s, 91.0 MB/s 2023-11-06T03:14:58.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.867 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178901 s, 58.6 MB/s 2023-11-06T03:14:58.889 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.890 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133588 s, 78.5 MB/s 2023-11-06T03:14:58.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:58.892 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:58.892 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:58.892 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.26 2023-11-06T03:14:58.893 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:58.893 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:58.893 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:58.894 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125057 s, 83.8 MB/s 2023-11-06T03:14:58.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.978 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.978 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141789 s, 74.0 MB/s 2023-11-06T03:14:58.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:58.992 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:58.993 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116313 s, 90.2 MB/s 2023-11-06T03:14:59.007 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.008 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012304 s, 85.2 MB/s 2023-11-06T03:14:59.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.024 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.024 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013614 s, 77.0 MB/s 2023-11-06T03:14:59.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.041 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137474 s, 76.3 MB/s 2023-11-06T03:14:59.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.058 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.058 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148064 s, 70.8 MB/s 2023-11-06T03:14:59.071 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.071 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.072 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104374 s, 100 MB/s 2023-11-06T03:14:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.084 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992936 s, 106 MB/s 2023-11-06T03:14:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130359 s, 80.4 MB/s 2023-11-06T03:14:59.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.115 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.115 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121744 s, 86.1 MB/s 2023-11-06T03:14:59.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.131 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133929 s, 78.3 MB/s 2023-11-06T03:14:59.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.148 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.148 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134102 s, 78.2 MB/s 2023-11-06T03:14:59.161 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.162 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.162 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111239 s, 94.3 MB/s 2023-11-06T03:14:59.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.176 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107757 s, 97.3 MB/s 2023-11-06T03:14:59.192 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.193 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.193 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139715 s, 75.1 MB/s 2023-11-06T03:14:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107813 s, 97.3 MB/s 2023-11-06T03:14:59.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.222 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.223 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131329 s, 79.8 MB/s 2023-11-06T03:14:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130087 s, 80.6 MB/s 2023-11-06T03:14:59.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.253 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.254 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125246 s, 83.7 MB/s 2023-11-06T03:14:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.271 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147372 s, 71.2 MB/s 2023-11-06T03:14:59.286 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.287 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.287 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128656 s, 81.5 MB/s 2023-11-06T03:14:59.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.301 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.301 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111774 s, 93.8 MB/s 2023-11-06T03:14:59.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.316 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.317 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125007 s, 83.9 MB/s 2023-11-06T03:14:59.330 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.331 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.331 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106328 s, 98.6 MB/s 2023-11-06T03:14:59.344 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.345 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.345 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111581 s, 94.0 MB/s 2023-11-06T03:14:59.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.357 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.357 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968451 s, 108 MB/s 2023-11-06T03:14:59.373 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.374 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132471 s, 79.2 MB/s 2023-11-06T03:14:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117029 s, 89.6 MB/s 2023-11-06T03:14:59.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.404 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126141 s, 83.1 MB/s 2023-11-06T03:14:59.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.419 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124487 s, 84.2 MB/s 2023-11-06T03:14:59.435 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.436 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.436 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139148 s, 75.4 MB/s 2023-11-06T03:14:59.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:14:59.437 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:14:59.438 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:14:59.438 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.27 2023-11-06T03:14:59.438 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:14:59.439 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:14:59.439 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:14:59.439 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:14:59.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.510 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121742 s, 86.1 MB/s 2023-11-06T03:14:59.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.526 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.526 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131984 s, 79.4 MB/s 2023-11-06T03:14:59.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.540 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.540 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115924 s, 90.5 MB/s 2023-11-06T03:14:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119245 s, 87.9 MB/s 2023-11-06T03:14:59.572 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.573 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.573 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142393 s, 73.6 MB/s 2023-11-06T03:14:59.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.586 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.586 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108167 s, 96.9 MB/s 2023-11-06T03:14:59.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.602 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.603 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132013 s, 79.4 MB/s 2023-11-06T03:14:59.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.617 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.618 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123008 s, 85.2 MB/s 2023-11-06T03:14:59.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.633 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.633 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127246 s, 82.4 MB/s 2023-11-06T03:14:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.650 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135928 s, 77.1 MB/s 2023-11-06T03:14:59.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.664 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.664 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114369 s, 91.7 MB/s 2023-11-06T03:14:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128629 s, 81.5 MB/s 2023-11-06T03:14:59.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.695 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.696 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117933 s, 88.9 MB/s 2023-11-06T03:14:59.710 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.711 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.711 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130093 s, 80.6 MB/s 2023-11-06T03:14:59.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.726 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.726 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123363 s, 85.0 MB/s 2023-11-06T03:14:59.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.743 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.744 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149562 s, 70.1 MB/s 2023-11-06T03:14:59.758 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.759 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.759 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120012 s, 87.4 MB/s 2023-11-06T03:14:59.774 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.775 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.775 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131849 s, 79.5 MB/s 2023-11-06T03:14:59.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.787 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.788 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100298 s, 105 MB/s 2023-11-06T03:14:59.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.801 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.801 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105622 s, 99.3 MB/s 2023-11-06T03:14:59.818 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.819 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.819 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142323 s, 73.7 MB/s 2023-11-06T03:14:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.834 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.834 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121387 s, 86.4 MB/s 2023-11-06T03:14:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.851 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.852 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014006 s, 74.9 MB/s 2023-11-06T03:14:59.866 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124237 s, 84.4 MB/s 2023-11-06T03:14:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.902 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126719 s, 82.7 MB/s 2023-11-06T03:14:59.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.913 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.914 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919544 s, 114 MB/s 2023-11-06T03:14:59.928 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.929 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.929 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122294 s, 85.7 MB/s 2023-11-06T03:14:59.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.941 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.941 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946076 s, 111 MB/s 2023-11-06T03:14:59.954 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.955 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114109 s, 91.9 MB/s 2023-11-06T03:14:59.969 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.970 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121619 s, 86.2 MB/s 2023-11-06T03:14:59.983 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:14:59.984 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:14:59.984 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116334 s, 90.1 MB/s 2023-11-06T03:15:00.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.000 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.001 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139098 s, 75.4 MB/s 2023-11-06T03:15:00.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:00.002 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:15:00.003 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:15:00.003 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.28 2023-11-06T03:15:00.003 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:15:00.004 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:15:00.004 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:15:00.004 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:15:00.066 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.067 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.067 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115475 s, 90.8 MB/s 2023-11-06T03:15:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.078 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948315 s, 111 MB/s 2023-11-06T03:15:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.093 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.094 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124303 s, 84.4 MB/s 2023-11-06T03:15:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956494 s, 110 MB/s 2023-11-06T03:15:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082947 s, 126 MB/s 2023-11-06T03:15:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116844 s, 89.7 MB/s 2023-11-06T03:15:00.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.146 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.146 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134679 s, 77.9 MB/s 2023-11-06T03:15:00.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.159 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.160 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111364 s, 94.2 MB/s 2023-11-06T03:15:00.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.175 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.175 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130404 s, 80.4 MB/s 2023-11-06T03:15:00.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.190 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.190 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116135 s, 90.3 MB/s 2023-11-06T03:15:00.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.203 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.203 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110112 s, 95.2 MB/s 2023-11-06T03:15:00.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.220 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.220 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139857 s, 75.0 MB/s 2023-11-06T03:15:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135001 s, 77.7 MB/s 2023-11-06T03:15:00.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.252 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.252 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131199 s, 79.9 MB/s 2023-11-06T03:15:00.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.268 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.269 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138901 s, 75.5 MB/s 2023-11-06T03:15:00.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.284 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.285 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130591 s, 80.3 MB/s 2023-11-06T03:15:00.297 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.298 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.298 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110294 s, 95.1 MB/s 2023-11-06T03:15:00.312 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.313 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.313 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124447 s, 84.3 MB/s 2023-11-06T03:15:00.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.328 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131374 s, 79.8 MB/s 2023-11-06T03:15:00.343 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.343 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.343 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117629 s, 89.1 MB/s 2023-11-06T03:15:00.359 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.360 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.360 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133926 s, 78.3 MB/s 2023-11-06T03:15:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.375 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120755 s, 86.8 MB/s 2023-11-06T03:15:00.389 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.390 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120455 s, 87.1 MB/s 2023-11-06T03:15:00.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.403 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.403 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108545 s, 96.6 MB/s 2023-11-06T03:15:00.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.418 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.418 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125036 s, 83.9 MB/s 2023-11-06T03:15:00.429 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.430 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.430 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894366 s, 117 MB/s 2023-11-06T03:15:00.445 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.446 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.446 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129117 s, 81.2 MB/s 2023-11-06T03:15:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.461 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128479 s, 81.6 MB/s 2023-11-06T03:15:00.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137387 s, 76.3 MB/s 2023-11-06T03:15:00.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.493 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.493 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122846 s, 85.4 MB/s 2023-11-06T03:15:00.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.509 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.509 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142336 s, 73.7 MB/s 2023-11-06T03:15:00.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.525 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.525 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130651 s, 80.3 MB/s 2023-11-06T03:15:00.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:00.527 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:15:00.527 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:15:00.528 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.29 2023-11-06T03:15:00.528 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:15:00.528 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:15:00.528 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:15:00.529 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:15:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.592 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128177 s, 81.8 MB/s 2023-11-06T03:15:00.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.608 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.608 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130491 s, 80.4 MB/s 2023-11-06T03:15:00.623 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.624 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.624 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125441 s, 83.6 MB/s 2023-11-06T03:15:00.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.639 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.639 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118007 s, 88.9 MB/s 2023-11-06T03:15:00.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.653 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.653 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118039 s, 88.8 MB/s 2023-11-06T03:15:00.668 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.669 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.669 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122331 s, 85.7 MB/s 2023-11-06T03:15:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.683 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112061 s, 93.6 MB/s 2023-11-06T03:15:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135587 s, 77.3 MB/s 2023-11-06T03:15:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118817 s, 88.3 MB/s 2023-11-06T03:15:00.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.731 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141209 s, 74.3 MB/s 2023-11-06T03:15:00.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142068 s, 73.8 MB/s 2023-11-06T03:15:00.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.763 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127659 s, 82.1 MB/s 2023-11-06T03:15:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.776 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.777 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111313 s, 94.2 MB/s 2023-11-06T03:15:00.789 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.790 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107713 s, 97.3 MB/s 2023-11-06T03:15:00.803 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.804 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.804 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011343 s, 92.4 MB/s 2023-11-06T03:15:00.820 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.821 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.821 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143891 s, 72.9 MB/s 2023-11-06T03:15:00.830 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.831 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.831 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800587 s, 131 MB/s 2023-11-06T03:15:00.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.842 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851683 s, 123 MB/s 2023-11-06T03:15:00.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.854 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.855 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010934 s, 95.9 MB/s 2023-11-06T03:15:00.875 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.876 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.876 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185682 s, 56.5 MB/s 2023-11-06T03:15:00.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.891 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.891 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105827 s, 99.1 MB/s 2023-11-06T03:15:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115404 s, 90.9 MB/s 2023-11-06T03:15:00.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.918 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.919 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109109 s, 96.1 MB/s 2023-11-06T03:15:00.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.933 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.933 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124417 s, 84.3 MB/s 2023-11-06T03:15:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111434 s, 94.1 MB/s 2023-11-06T03:15:00.962 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.963 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.963 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135163 s, 77.6 MB/s 2023-11-06T03:15:00.976 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.977 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108926 s, 96.3 MB/s 2023-11-06T03:15:00.990 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:00.991 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:00.991 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114242 s, 91.8 MB/s 2023-11-06T03:15:01.003 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.004 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.004 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107475 s, 97.6 MB/s 2023-11-06T03:15:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.017 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011089 s, 94.6 MB/s 2023-11-06T03:15:01.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.032 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.032 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123673 s, 84.8 MB/s 2023-11-06T03:15:01.047 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.049 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120153 s, 87.3 MB/s 2023-11-06T03:15:01.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:01.051 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:15:01.051 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:15:01.052 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.30 2023-11-06T03:15:01.052 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:15:01.053 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:15:01.053 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:15:01.053 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:15:01.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.121 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.121 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124845 s, 84.0 MB/s 2023-11-06T03:15:01.136 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.137 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.137 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130094 s, 80.6 MB/s 2023-11-06T03:15:01.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.152 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.153 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116966 s, 89.6 MB/s 2023-11-06T03:15:01.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.165 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.165 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108805 s, 96.4 MB/s 2023-11-06T03:15:01.178 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.179 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115486 s, 90.8 MB/s 2023-11-06T03:15:01.191 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.192 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104364 s, 100 MB/s 2023-11-06T03:15:01.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.206 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121475 s, 86.3 MB/s 2023-11-06T03:15:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101366 s, 103 MB/s 2023-11-06T03:15:01.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.230 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.230 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872401 s, 120 MB/s 2023-11-06T03:15:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.244 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.245 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119521 s, 87.7 MB/s 2023-11-06T03:15:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.259 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111853 s, 93.7 MB/s 2023-11-06T03:15:01.269 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.270 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.270 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820622 s, 128 MB/s 2023-11-06T03:15:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.285 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.286 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137727 s, 76.1 MB/s 2023-11-06T03:15:01.299 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.300 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.300 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112033 s, 93.6 MB/s 2023-11-06T03:15:01.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.314 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.314 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011929 s, 87.9 MB/s 2023-11-06T03:15:01.326 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.326 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.326 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905346 s, 116 MB/s 2023-11-06T03:15:01.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.338 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.338 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928288 s, 113 MB/s 2023-11-06T03:15:01.353 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.354 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.354 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126012 s, 83.2 MB/s 2023-11-06T03:15:01.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.368 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.368 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123198 s, 85.1 MB/s 2023-11-06T03:15:01.381 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.382 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.382 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110575 s, 94.8 MB/s 2023-11-06T03:15:01.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.397 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.397 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132418 s, 79.2 MB/s 2023-11-06T03:15:01.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.410 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.411 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110604 s, 94.8 MB/s 2023-11-06T03:15:01.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.423 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.423 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101414 s, 103 MB/s 2023-11-06T03:15:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.437 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.438 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114965 s, 91.2 MB/s 2023-11-06T03:15:01.450 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.451 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.451 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112126 s, 93.5 MB/s 2023-11-06T03:15:01.462 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.462 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.462 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880727 s, 119 MB/s 2023-11-06T03:15:01.477 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.478 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.478 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013249 s, 79.1 MB/s 2023-11-06T03:15:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.492 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119667 s, 87.6 MB/s 2023-11-06T03:15:01.506 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.507 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125416 s, 83.6 MB/s 2023-11-06T03:15:01.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.519 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.520 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103269 s, 102 MB/s 2023-11-06T03:15:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.531 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.532 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954621 s, 110 MB/s 2023-11-06T03:15:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122461 s, 85.6 MB/s 2023-11-06T03:15:01.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:01.548 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2023-11-06T03:15:01.548 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:15:01.549 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p scrub_control_test_path/dir.31 2023-11-06T03:15:01.549 DEBUG:teuthology.orchestra.run.smithi157:> for ((i = 0; i < 32; i++)); do 2023-11-06T03:15:01.550 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2023-11-06T03:15:01.550 DEBUG:teuthology.orchestra.run.smithi157:> done 2023-11-06T03:15:01.551 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:15:01.614 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.615 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.615 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011511 s, 91.1 MB/s 2023-11-06T03:15:01.630 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.631 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011201 s, 93.6 MB/s 2023-11-06T03:15:01.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.641 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.642 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010352 s, 101 MB/s 2023-11-06T03:15:01.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.656 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.656 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124829 s, 84.0 MB/s 2023-11-06T03:15:01.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.670 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.671 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112953 s, 92.8 MB/s 2023-11-06T03:15:01.685 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.686 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128909 s, 81.3 MB/s 2023-11-06T03:15:01.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.701 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.702 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131036 s, 80.0 MB/s 2023-11-06T03:15:01.717 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.717 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.718 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127105 s, 82.5 MB/s 2023-11-06T03:15:01.731 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.732 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.732 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123749 s, 84.7 MB/s 2023-11-06T03:15:01.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.748 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.748 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134284 s, 78.1 MB/s 2023-11-06T03:15:01.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.762 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.763 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118611 s, 88.4 MB/s 2023-11-06T03:15:01.778 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.779 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.780 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126288 s, 83.0 MB/s 2023-11-06T03:15:01.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.792 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.793 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116709 s, 89.8 MB/s 2023-11-06T03:15:01.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.808 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.809 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129278 s, 81.1 MB/s 2023-11-06T03:15:01.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.823 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.824 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121244 s, 86.5 MB/s 2023-11-06T03:15:01.840 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.841 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138936 s, 75.5 MB/s 2023-11-06T03:15:01.852 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.853 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.853 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925575 s, 113 MB/s 2023-11-06T03:15:01.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.870 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.871 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151324 s, 69.3 MB/s 2023-11-06T03:15:01.909 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.910 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133997 s, 78.3 MB/s 2023-11-06T03:15:01.925 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.926 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130587 s, 80.3 MB/s 2023-11-06T03:15:01.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.940 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.940 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120014 s, 87.4 MB/s 2023-11-06T03:15:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.955 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.956 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011851 s, 88.5 MB/s 2023-11-06T03:15:01.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.970 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.971 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114885 s, 91.3 MB/s 2023-11-06T03:15:01.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.986 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:01.986 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132056 s, 79.4 MB/s 2023-11-06T03:15:01.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:01.999 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.000 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993771 s, 106 MB/s 2023-11-06T03:15:02.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.014 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.015 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114068 s, 91.9 MB/s 2023-11-06T03:15:02.026 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.026 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.026 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904797 s, 116 MB/s 2023-11-06T03:15:02.040 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.041 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.041 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121565 s, 86.3 MB/s 2023-11-06T03:15:02.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.056 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.056 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125312 s, 83.7 MB/s 2023-11-06T03:15:02.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.068 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.068 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921202 s, 114 MB/s 2023-11-06T03:15:02.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.083 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.084 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122194 s, 85.8 MB/s 2023-11-06T03:15:02.099 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2023-11-06T03:15:02.100 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2023-11-06T03:15:02.100 INFO:teuthology.orchestra.run.smithi157.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139154 s, 75.4 MB/s 2023-11-06T03:15:02.101 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2023-11-06T03:15:02.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 Processor -- start 2023-11-06T03:15:02.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 -- start start 2023-11-06T03:15:02.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29e4154cc0 0x7f29e415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29e41550a0 0x7f29e40cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40ccf80 0x7f29e406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e406c720 con 0x7f29e41550a0 2023-11-06T03:15:02.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e415ea10 con 0x7f29e4154cc0 2023-11-06T03:15:02.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29ea873640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40d0640 con 0x7f29e40ccf80 2023-11-06T03:15:02.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29e41550a0 0x7f29e40cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29e41550a0 0x7f29e40cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53964/0 (socket says 172.21.15.157:53964) 2023-11-06T03:15:02.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 -- 172.21.15.157:0/2018751525 learned_addr learned my addr 172.21.15.157:0/2018751525 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 -- 172.21.15.157:0/2018751525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40ccf80 msgr2=0x7f29e406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 --2- 172.21.15.157:0/2018751525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40ccf80 0x7f29e406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 -- 172.21.15.157:0/2018751525 >> v2:172.21.15.195:3300/0 conn(0x7f29e4154cc0 msgr2=0x7f29e415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e3fff640 1 --2- 172.21.15.157:0/2018751525 >> v2:172.21.15.195:3300/0 conn(0x7f29e4154cc0 0x7f29e415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 --2- 172.21.15.157:0/2018751525 >> v2:172.21.15.195:3300/0 conn(0x7f29e4154cc0 0x7f29e415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.276+0000 7f29e37fe640 1 -- 172.21.15.157:0/2018751525 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e40cf790 con 0x7f29e41550a0 2023-11-06T03:15:02.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e37fe640 1 --2- 172.21.15.157:0/2018751525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e41550a0 0x7f29e40cca40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f29e40d0840 tx=0x7f29d801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe633e1e51b0b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e2ffd640 1 -- 172.21.15.157:0/2018751525 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80ac600 con 0x7f29e41550a0 2023-11-06T03:15:02.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e2ffd640 1 -- 172.21.15.157:0/2018751525 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d80a8050 con 0x7f29e41550a0 2023-11-06T03:15:02.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e2ffd640 1 -- 172.21.15.157:0/2018751525 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80ac980 con 0x7f29e41550a0 2023-11-06T03:15:02.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- 172.21.15.157:0/2018751525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e41550a0 msgr2=0x7f29e40cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- 172.21.15.157:0/2018751525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e41550a0 0x7f29e40cca40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f29e40d0840 tx=0x7f29d801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- 172.21.15.157:0/2018751525 shutdown_connections 2023-11-06T03:15:02.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- 172.21.15.157:0/2018751525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e40ccf80 0x7f29e406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- 172.21.15.157:0/2018751525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e41550a0 0x7f29e40cca40 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- 172.21.15.157:0/2018751525 >> v2:172.21.15.195:3300/0 conn(0x7f29e4154cc0 0x7f29e415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- 172.21.15.157:0/2018751525 >> 172.21.15.157:0/2018751525 conn(0x7f29e4069d70 msgr2=0x7f29e406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- 172.21.15.157:0/2018751525 shutdown_connections 2023-11-06T03:15:02.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- 172.21.15.157:0/2018751525 wait complete. 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 Processor -- start 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- start start 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 0x7f29e414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e41550a0 0x7f29e4147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e41506d0 con 0x7f29e40ccf80 2023-11-06T03:15:02.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e414d240 con 0x7f29e4154cc0 2023-11-06T03:15:02.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29ea873640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e41508c0 con 0x7f29e41550a0 2023-11-06T03:15:02.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53966/0 (socket says 172.21.15.157:53966) 2023-11-06T03:15:02.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.280+0000 7f29e3fff640 1 -- 172.21.15.157:0/1982742565 learned_addr learned my addr 172.21.15.157:0/1982742565 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 0x7f29e414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 -- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e41550a0 msgr2=0x7f29e4147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e41550a0 0x7f29e4147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 -- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 msgr2=0x7f29e414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 0x7f29e414cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 -- 172.21.15.157:0/1982742565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80094d0 con 0x7f29e40ccf80 2023-11-06T03:15:02.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 0x7f29e414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00ce40 tx=0x7f29dc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 -- 172.21.15.157:0/1982742565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc003ba0 con 0x7f29e40ccf80 2023-11-06T03:15:02.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 -- 172.21.15.157:0/1982742565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc003d00 con 0x7f29e40ccf80 2023-11-06T03:15:02.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 -- 172.21.15.157:0/1982742565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc004240 con 0x7f29e40ccf80 2023-11-06T03:15:02.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4150ba0 con 0x7f29e40ccf80 2023-11-06T03:15:02.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e4151160 con 0x7f29e40ccf80 2023-11-06T03:15:02.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 -- 172.21.15.157:0/1982742565 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29dc0043a0 con 0x7f29e40ccf80 2023-11-06T03:15:02.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 0x7f29d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e0ff9640 1 -- 172.21.15.157:0/1982742565 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29dc012070 con 0x7f29e40ccf80 2023-11-06T03:15:02.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 0x7f29d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29ed99b000 1 Processor -- start 2023-11-06T03:15:02.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29ed99b000 1 -- start start 2023-11-06T03:15:02.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.284+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 0x7f29d0067f20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29d80ae000 tx=0x7f29d80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1d380 0x55a950ed2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950ed3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950d9b150 con 0x55a950f1ca60 2023-11-06T03:15:02.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950d271b0 con 0x55a950ed3030 2023-11-06T03:15:02.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a950d24780 con 0x55a950f1d380 2023-11-06T03:15:02.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53978/0 (socket says 172.21.15.157:53978) 2023-11-06T03:15:02.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 -- 172.21.15.157:0/1181105467 learned_addr learned my addr 172.21.15.157:0/1181105467 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1d380 0x55a950ed2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950ed3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 -- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1d380 msgr2=0x55a950ed2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1d380 0x55a950ed2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 -- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 msgr2=0x55a950ed3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950ed3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 -- 172.21.15.157:0/1181105467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a950a1cda0 con 0x55a950f1ca60 2023-11-06T03:15:02.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950ed3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f29dc082120 tx=0x7f29dc00aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f066d5979d0a0d05 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c9ffb640 1 -- 172.21.15.157:0/1181105467 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc055250 con 0x55a950f1ca60 2023-11-06T03:15:02.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c9ffb640 1 -- 172.21.15.157:0/1181105467 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc059080 con 0x55a950f1ca60 2023-11-06T03:15:02.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c9ffb640 1 -- 172.21.15.157:0/1181105467 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc05d230 con 0x55a950f1ca60 2023-11-06T03:15:02.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- 172.21.15.157:0/1181105467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 msgr2=0x55a950f1ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f29dc082120 tx=0x7f29dc00aef0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- 172.21.15.157:0/1181105467 shutdown_connections 2023-11-06T03:15:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950ed3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1d380 0x55a950ed2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1181105467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1ca60 0x55a950f1ce40 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- 172.21.15.157:0/1181105467 >> 172.21.15.157:0/1181105467 conn(0x55a950ed93a0 msgr2=0x55a950ed13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- 172.21.15.157:0/1181105467 shutdown_connections 2023-11-06T03:15:02.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- 172.21.15.157:0/1181105467 wait complete. 2023-11-06T03:15:02.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- start start 2023-11-06T03:15:02.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 Processor -- start 2023-11-06T03:15:02.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950ed3030 0x55a950ed7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1ca60 0x55a950edda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950d20070 con 0x55a950f1d380 2023-11-06T03:15:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950cbf7c0 con 0x55a950f1ca60 2023-11-06T03:15:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a950cbf460 con 0x55a950ed3030 2023-11-06T03:15:02.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29ed99b000 10 client.0 _ll_register_callbacks cb 0x55a950d199b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53986/0 (socket says 172.21.15.157:53986) 2023-11-06T03:15:02.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 -- 172.21.15.157:0/1395033149 learned_addr learned my addr 172.21.15.157:0/1395033149 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950ed3030 0x55a950ed7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e37fe640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1ca60 0x55a950edda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 -- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950ed3030 msgr2=0x55a950ed7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950ed3030 0x55a950ed7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 -- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1ca60 msgr2=0x55a950edda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1ca60 0x55a950edda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 -- 172.21.15.157:0/1395033149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e4151160 con 0x55a950f1d380 2023-11-06T03:15:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29e8e10640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f29d400af50 tx=0x7f29d400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dc11f4d0cf4de2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40115f0 con 0x55a950f1d380 2023-11-06T03:15:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.288+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29d4011750 con 0x55a950f1d380 2023-11-06T03:15:02.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a950cd9840 con 0x55a950f1d380 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d4011910 con 0x55a950f1d380 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 10 client.9579 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55a950cc0410 con 0x55a950f1d380 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a950cd9840 con 0x55a950f1d380 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f29d4011a70 con 0x55a950f1d380 2023-11-06T03:15:02.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 10 client.9579 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 10 client.9579 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 20 client.9579 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f22c80 0x55a950f23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 4 client.9579 mds_command: new command op to 9375 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:02.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a950d1e9c0 con 0x55a950f22c80 2023-11-06T03:15:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f22c80 0x55a950f23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29e3fff640 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f22c80 0x55a950f23090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a791a5edf80579df server_cookie=4b972d2435ced33 in_seq=0 out_seq=0 2023-11-06T03:15:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 10 client.9579 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 1 -- 172.21.15.157:0/1395033149 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a950d1e9c0 con 0x55a950f22c80 2023-11-06T03:15:02.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 10 client.9579 handle_command_reply: tid=0 2023-11-06T03:15:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 client.9579 shutdown 2023-11-06T03:15:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 msgr2=0x55a950eddf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f29d400af50 tx=0x7f29d400c5e0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 shutdown_connections 2023-11-06T03:15:02.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f22c80 0x55a950f23090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950eddf60 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1ca60 0x55a950edda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 --2- 172.21.15.157:0/1395033149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950ed3030 0x55a950ed7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29c2ffd640 0 client.9579 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 >> 172.21.15.157:0/1395033149 conn(0x55a950ed93a0 msgr2=0x55a950ed1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 shutdown_connections 2023-11-06T03:15:02.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 1 -- 172.21.15.157:0/1395033149 wait complete. 2023-11-06T03:15:02.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.292+0000 7f29ed99b000 20 client.9579 trim_cache size 0 max 16384 2023-11-06T03:15:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 Processor -- start 2023-11-06T03:15:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- start start 2023-11-06T03:15:02.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1d380 0x55a950f169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9509dda20 con 0x55a950ed3030 2023-11-06T03:15:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950dd3ee0 con 0x55a950f1d380 2023-11-06T03:15:02.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a950ed5640 con 0x55a950f1ca60 2023-11-06T03:15:02.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54002/0 (socket says 172.21.15.157:54002) 2023-11-06T03:15:02.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 -- 172.21.15.157:0/3906108135 learned_addr learned my addr 172.21.15.157:0/3906108135 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 -- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 msgr2=0x55a950f213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 -- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1d380 msgr2=0x55a950f169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1d380 0x55a950f169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 -- 172.21.15.157:0/3906108135 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a950ce5370 con 0x55a950ed3030 2023-11-06T03:15:02.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29e3fff640 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f29dc05cd00 tx=0x7f29dc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1768e95773e7af0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ca7fc640 1 -- 172.21.15.157:0/3906108135 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc071040 con 0x55a950ed3030 2023-11-06T03:15:02.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ca7fc640 1 -- 172.21.15.157:0/3906108135 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc07c030 con 0x55a950ed3030 2023-11-06T03:15:02.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ca7fc640 1 -- 172.21.15.157:0/3906108135 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc055020 con 0x55a950ed3030 2023-11-06T03:15:02.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- 172.21.15.157:0/3906108135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 msgr2=0x55a950f20e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f29dc05cd00 tx=0x7f29dc00f000 comp rx=0 tx=0).stop 2023-11-06T03:15:02.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- 172.21.15.157:0/3906108135 shutdown_connections 2023-11-06T03:15:02.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950f1d380 0x55a950f169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- 172.21.15.157:0/3906108135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950ed3030 0x55a950f20e90 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- 172.21.15.157:0/3906108135 >> 172.21.15.157:0/3906108135 conn(0x55a950ed93a0 msgr2=0x55a950f1f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- 172.21.15.157:0/3906108135 shutdown_connections 2023-11-06T03:15:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- 172.21.15.157:0/3906108135 wait complete. 2023-11-06T03:15:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 -- start start 2023-11-06T03:15:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 Processor -- start 2023-11-06T03:15:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.308+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950f80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950ebe2d0 con 0x55a950f1d380 2023-11-06T03:15:02.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950d1f880 con 0x55a950ed3030 2023-11-06T03:15:02.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a950cc8960 con 0x55a950f1ca60 2023-11-06T03:15:02.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 10 client.0 _ll_register_callbacks cb 0x55a950ce9f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54010/0 (socket says 172.21.15.157:54010) 2023-11-06T03:15:02.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 -- 172.21.15.157:0/2929392548 learned_addr learned my addr 172.21.15.157:0/2929392548 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e3fff640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950f80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 -- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 msgr2=0x55a950f814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 -- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 msgr2=0x55a950f80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950f80f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 -- 172.21.15.157:0/2929392548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a950a1cda0 con 0x55a950f1d380 2023-11-06T03:15:02.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e8e10640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f29d4001ec0 tx=0x7f29d40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2e16e9e70e2336 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40113f0 con 0x55a950f1d380 2023-11-06T03:15:02.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29d402a070 con 0x55a950f1d380 2023-11-06T03:15:02.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a950d28990 con 0x55a950f1d380 2023-11-06T03:15:02.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d4011550 con 0x55a950f1d380 2023-11-06T03:15:02.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 10 client.9585 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55a950ccb4c0 con 0x55a950f1d380 2023-11-06T03:15:02.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a950d28990 con 0x55a950f1d380 2023-11-06T03:15:02.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f29d40116b0 con 0x55a950f1d380 2023-11-06T03:15:02.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 10 client.9585 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 10 client.9585 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 20 client.9585 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f1a480 0x55a950f1a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 4 client.9585 mds_command: new command op to 9375 tid=0[{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2023-11-06T03:15:02.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55a950d190f0 con 0x55a950f1a480 2023-11-06T03:15:02.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e3fff640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f1a480 0x55a950f1a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29e3fff640 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f1a480 0x55a950f1a890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b201432c19d53d0 server_cookie=a263200aaa0f564b in_seq=0 out_seq=0 2023-11-06T03:15:02.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.312+0000 7f29c2ffd640 10 client.9585 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29c2ffd640 1 -- 172.21.15.157:0/2929392548 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a950d190f0 con 0x55a950f1a480 2023-11-06T03:15:02.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29c2ffd640 10 client.9585 handle_command_reply: tid=0 2023-11-06T03:15:02.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 client.9585 shutdown 2023-11-06T03:15:02.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 msgr2=0x55a950f19600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f29d4001ec0 tx=0x7f29d40047b0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 shutdown_connections 2023-11-06T03:15:02.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55a950f1a480 0x55a950f1a890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a950f1d380 0x55a950f19600 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a950f1ca60 0x55a950f814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 --2- 172.21.15.157:0/2929392548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a950ed3030 0x55a950f80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 >> 172.21.15.157:0/2929392548 conn(0x55a950ed93a0 msgr2=0x55a950f21930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29c2ffd640 0 client.9585 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 shutdown_connections 2023-11-06T03:15:02.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 1 -- 172.21.15.157:0/2929392548 wait complete. 2023-11-06T03:15:02.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ed99b000 20 client.9585 trim_cache size 0 max 16384 2023-11-06T03:15:02.335 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:02.335 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:15:02.335 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "fab56f51-4cc6-4ed1-b7a1-cac0ea9e6c10", 2023-11-06T03:15:02.335 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:15:02.336 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:02.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 msgr2=0x7f29d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 0x7f29d0067f20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29d80ae000 tx=0x7f29d80a7000 comp rx=0 tx=0).stop 2023-11-06T03:15:02.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 msgr2=0x7f29e4150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00ce40 tx=0x7f29dc00a660 comp rx=0 tx=0).stop 2023-11-06T03:15:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 shutdown_connections 2023-11-06T03:15:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f29d0065a60 0x7f29d0067f20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29e41550a0 0x7f29e4147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29e4154cc0 0x7f29e414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 --2- 172.21.15.157:0/1982742565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e40ccf80 0x7f29e4150190 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 >> 172.21.15.157:0/1982742565 conn(0x7f29e4069d70 msgr2=0x7f29e40b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 shutdown_connections 2023-11-06T03:15:02.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.316+0000 7f29ea873640 1 -- 172.21.15.157:0/1982742565 wait complete. 2023-11-06T03:15:02.340 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.9:0 scrub pause 2023-11-06T03:15:02.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 Processor -- start 2023-11-06T03:15:02.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 -- start start 2023-11-06T03:15:02.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff18015cf10 0x7ff1800b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1800660f0 0x7ff1800b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18006c880 con 0x7ff18015cb30 2023-11-06T03:15:02.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18006d500 con 0x7ff1800660f0 2023-11-06T03:15:02.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff186390640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18006cb60 con 0x7ff18015cf10 2023-11-06T03:15:02.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1800660f0 0x7ff1800b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff1800660f0 0x7ff1800b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54024/0 (socket says 172.21.15.157:54024) 2023-11-06T03:15:02.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 -- 172.21.15.157:0/2937717110 learned_addr learned my addr 172.21.15.157:0/2937717110 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/2937717110 >> v2:172.21.15.195:3300/0 conn(0x7ff18015cf10 0x7ff1800b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 -- 172.21.15.157:0/2937717110 >> v2:172.21.15.195:3300/0 conn(0x7ff18015cf10 msgr2=0x7ff1800b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 --2- 172.21.15.157:0/2937717110 >> v2:172.21.15.195:3300/0 conn(0x7ff18015cf10 0x7ff1800b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 -- 172.21.15.157:0/2937717110 >> v1:172.21.15.195:6789/0 conn(0x7ff18015cb30 legacy=0x7ff1800659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.480+0000 7ff18492d640 1 -- 172.21.15.157:0/2937717110 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18006ce40 con 0x7ff1800660f0 2023-11-06T03:15:02.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff18492d640 1 --2- 172.21.15.157:0/2937717110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800b2e30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7ff180071060 tx=0x7ff17801f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46792b87e04e3a8d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff17effd640 1 -- 172.21.15.157:0/2937717110 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780af400 con 0x7ff1800660f0 2023-11-06T03:15:02.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff17effd640 1 -- 172.21.15.157:0/2937717110 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1780ab050 con 0x7ff1800660f0 2023-11-06T03:15:02.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff17effd640 1 -- 172.21.15.157:0/2937717110 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780af780 con 0x7ff1800660f0 2023-11-06T03:15:02.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- 172.21.15.157:0/2937717110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 msgr2=0x7ff1800b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 --2- 172.21.15.157:0/2937717110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800b2e30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7ff180071060 tx=0x7ff17801f180 comp rx=0 tx=0).stop 2023-11-06T03:15:02.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- 172.21.15.157:0/2937717110 shutdown_connections 2023-11-06T03:15:02.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 --2- 172.21.15.157:0/2937717110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800b2e30 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 --2- 172.21.15.157:0/2937717110 >> v2:172.21.15.195:3300/0 conn(0x7ff18015cf10 0x7ff1800b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- 172.21.15.157:0/2937717110 >> 172.21.15.157:0/2937717110 conn(0x7ff18006a170 msgr2=0x7ff18006a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- 172.21.15.157:0/2937717110 shutdown_connections 2023-11-06T03:15:02.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- 172.21.15.157:0/2937717110 wait complete. 2023-11-06T03:15:02.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 Processor -- start 2023-11-06T03:15:02.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.484+0000 7ff186390640 1 -- start start 2023-11-06T03:15:02.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff18015cb30 0x7ff18015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff18015cf10 0x7ff1800c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015fd40 con 0x7ff1800660f0 2023-11-06T03:15:02.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800c4660 con 0x7ff18015cb30 2023-11-06T03:15:02.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015d9f0 con 0x7ff18015cf10 2023-11-06T03:15:02.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54038/0 (socket says 172.21.15.157:54038) 2023-11-06T03:15:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 -- 172.21.15.157:0/804381430 learned_addr learned my addr 172.21.15.157:0/804381430 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 -- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff18015cf10 msgr2=0x7ff1800c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff18015cb30 0x7ff18015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff18015cf10 0x7ff1800c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 -- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff18015cb30 msgr2=0x7ff18015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff18015cb30 0x7ff18015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 -- 172.21.15.157:0/804381430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178009b30 con 0x7ff1800660f0 2023-11-06T03:15:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17ffff640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c450 tx=0x7ff17000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 -- 172.21.15.157:0/804381430 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170012830 con 0x7ff1800660f0 2023-11-06T03:15:02.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 -- 172.21.15.157:0/804381430 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1700129b0 con 0x7ff1800660f0 2023-11-06T03:15:02.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18015dcd0 con 0x7ff1800660f0 2023-11-06T03:15:02.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 -- 172.21.15.157:0/804381430 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170012d60 con 0x7ff1800660f0 2023-11-06T03:15:02.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18015e290 con 0x7ff1800660f0 2023-11-06T03:15:02.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 -- 172.21.15.157:0/804381430 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff17001c460 con 0x7ff1800660f0 2023-11-06T03:15:02.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 0x7ff16c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17cff9640 1 -- 172.21.15.157:0/804381430 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff170083150 con 0x7ff1800660f0 2023-11-06T03:15:02.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.488+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 0x7ff16c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 Processor -- start 2023-11-06T03:15:02.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- start start 2023-11-06T03:15:02.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 0x7ff16c067f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff1800c2ce0 tx=0x7ff174008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804ef974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804ef4d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804ee14bb0 con 0x55804ef979f0 2023-11-06T03:15:02.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804eda3380 con 0x55804ef4d200 2023-11-06T03:15:02.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55804eda0950 con 0x55804ef970d0 2023-11-06T03:15:02.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54054/0 (socket says 172.21.15.157:54054) 2023-11-06T03:15:02.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 -- 172.21.15.157:0/22147254 learned_addr learned my addr 172.21.15.157:0/22147254 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff18492d640 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804ef4d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 -- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 msgr2=0x55804ef974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804ef974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 -- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 msgr2=0x55804ef4d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804ef4d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 -- 172.21.15.157:0/22147254 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55804ea98dc0 con 0x55804ef979f0 2023-11-06T03:15:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff18492d640 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804ef4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff17400d9a0 tx=0x7ff174010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b86af46c0a111f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff165ffb640 1 -- 172.21.15.157:0/22147254 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400b050 con 0x55804ef979f0 2023-11-06T03:15:02.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff165ffb640 1 -- 172.21.15.157:0/22147254 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174017070 con 0x55804ef979f0 2023-11-06T03:15:02.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff165ffb640 1 -- 172.21.15.157:0/22147254 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174012b30 con 0x55804ef979f0 2023-11-06T03:15:02.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- 172.21.15.157:0/22147254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 msgr2=0x55804ef4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff17400d9a0 tx=0x7ff174010000 comp rx=0 tx=0).stop 2023-11-06T03:15:02.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- 172.21.15.157:0/22147254 shutdown_connections 2023-11-06T03:15:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804ef4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804ef4ccc0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- 172.21.15.157:0/22147254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804ef974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- 172.21.15.157:0/22147254 >> 172.21.15.157:0/22147254 conn(0x55804ef53990 msgr2=0x55804ef4b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- 172.21.15.157:0/22147254 shutdown_connections 2023-11-06T03:15:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- 172.21.15.157:0/22147254 wait complete. 2023-11-06T03:15:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 -- start start 2023-11-06T03:15:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 Processor -- start 2023-11-06T03:15:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.492+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef4d200 0x55804ef9e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 0x55804ef97dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804ed9c240 con 0x55804ef970d0 2023-11-06T03:15:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804ed3b410 con 0x55804ef979f0 2023-11-06T03:15:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55804ed59600 con 0x55804ef4d200 2023-11-06T03:15:02.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 10 client.0 _ll_register_callbacks cb 0x55804ef38bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54060/0 (socket says 172.21.15.157:54060) 2023-11-06T03:15:02.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 -- 172.21.15.157:0/1148775942 learned_addr learned my addr 172.21.15.157:0/1148775942 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff18492d640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 0x55804ef97dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17ffff640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef4d200 0x55804ef9e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 -- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef4d200 msgr2=0x55804ef9e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef4d200 0x55804ef9e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 -- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 msgr2=0x55804ef97dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 0x55804ef97dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 -- 172.21.15.157:0/1148775942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17400be50 con 0x55804ef970d0 2023-11-06T03:15:02.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff18492d640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 0x55804ef97dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e010 tx=0x7ff174004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb990cc56cf16e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17401e910 con 0x55804ef970d0 2023-11-06T03:15:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff17401ea70 con 0x55804ef970d0 2023-11-06T03:15:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55804ed59040 con 0x55804ef970d0 2023-11-06T03:15:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff17401ec30 con 0x55804ef970d0 2023-11-06T03:15:02.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 10 client.9597 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55804ed61170 con 0x55804ef970d0 2023-11-06T03:15:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55804ed59040 con 0x55804ef970d0 2023-11-06T03:15:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff174031410 con 0x55804ef970d0 2023-11-06T03:15:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 10 client.9597 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 10 client.9597 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 20 client.9597 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef98b70 0x55804ef98f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 4 client.9597 mds_command: new command op to 9375 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:02.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55804ed9ab90 con 0x55804ef98b70 2023-11-06T03:15:02.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17ffff640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef98b70 0x55804ef98f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff17ffff640 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef98b70 0x55804ef98f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311e1da31bdfafcc server_cookie=fda994e31a394a98 in_seq=0 out_seq=0 2023-11-06T03:15:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 10 client.9597 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 1 -- 172.21.15.157:0/1148775942 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55804ed9ab90 con 0x55804ef98b70 2023-11-06T03:15:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 10 client.9597 handle_command_reply: tid=0 2023-11-06T03:15:02.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 client.9597 shutdown 2023-11-06T03:15:02.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 msgr2=0x55804ef9ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e010 tx=0x7ff174004740 comp rx=0 tx=0).stop 2023-11-06T03:15:02.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 shutdown_connections 2023-11-06T03:15:02.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef98b70 0x55804ef98f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef979f0 0x55804ef97dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff15effd640 0 client.9597 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef9ea20 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 --2- 172.21.15.157:0/1148775942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef4d200 0x55804ef9e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 >> 172.21.15.157:0/1148775942 conn(0x55804ef53990 msgr2=0x55804ef4bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 shutdown_connections 2023-11-06T03:15:02.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 1 -- 172.21.15.157:0/1148775942 wait complete. 2023-11-06T03:15:02.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.496+0000 7ff1894b8000 20 client.9597 trim_cache size 0 max 16384 2023-11-06T03:15:02.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 Processor -- start 2023-11-06T03:15:02.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- start start 2023-11-06T03:15:02.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804efeeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804ea59a40 con 0x55804ef979f0 2023-11-06T03:15:02.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804ee24c10 con 0x55804ef4d200 2023-11-06T03:15:02.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55804ed5a3a0 con 0x55804ef970d0 2023-11-06T03:15:02.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54062/0 (socket says 172.21.15.157:54062) 2023-11-06T03:15:02.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 -- 172.21.15.157:0/443108111 learned_addr learned my addr 172.21.15.157:0/443108111 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff17ffff640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804efeeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 -- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 msgr2=0x55804efeeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804efeeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 -- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 msgr2=0x55804efee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 -- 172.21.15.157:0/443108111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55804ed474c0 con 0x55804ef979f0 2023-11-06T03:15:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804efeeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff18492d640 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7ff17801cdb0 tx=0x7ff178004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc64620d9334f49b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1667fc640 1 -- 172.21.15.157:0/443108111 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780ba680 con 0x55804ef979f0 2023-11-06T03:15:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1667fc640 1 -- 172.21.15.157:0/443108111 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1780ba7e0 con 0x55804ef979f0 2023-11-06T03:15:02.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1667fc640 1 -- 172.21.15.157:0/443108111 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780bab70 con 0x55804ef979f0 2023-11-06T03:15:02.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- 172.21.15.157:0/443108111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 msgr2=0x55804efef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7ff17801cdb0 tx=0x7ff178004a50 comp rx=0 tx=0).stop 2023-11-06T03:15:02.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- 172.21.15.157:0/443108111 shutdown_connections 2023-11-06T03:15:02.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef979f0 0x55804efef080 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef970d0 0x55804efeeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 --2- 172.21.15.157:0/443108111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- 172.21.15.157:0/443108111 >> 172.21.15.157:0/443108111 conn(0x55804ef53990 msgr2=0x55804ef52970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- 172.21.15.157:0/443108111 shutdown_connections 2023-11-06T03:15:02.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- 172.21.15.157:0/443108111 wait complete. 2023-11-06T03:15:02.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 -- start start 2023-11-06T03:15:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.512+0000 7ff1894b8000 1 Processor -- start 2023-11-06T03:15:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efeddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef979f0 0x55804ef92150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804eda0240 con 0x55804ef970d0 2023-11-06T03:15:02.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804eddec70 con 0x55804ef4d200 2023-11-06T03:15:02.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55804ed99640 con 0x55804ef979f0 2023-11-06T03:15:02.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 10 client.0 _ll_register_callbacks cb 0x55804ed60c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54068/0 (socket says 172.21.15.157:54068) 2023-11-06T03:15:02.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 -- 172.21.15.157:0/3152885895 learned_addr learned my addr 172.21.15.157:0/3152885895 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 -- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef979f0 msgr2=0x55804ef92150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef979f0 0x55804ef92150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 -- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 msgr2=0x55804efeddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efeddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 -- 172.21.15.157:0/3152885895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18006ce40 con 0x55804ef970d0 2023-11-06T03:15:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17f7fe640 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff174031f50 tx=0x7ff174013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209346ad47bd99b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174004240 con 0x55804ef970d0 2023-11-06T03:15:02.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1740043a0 con 0x55804ef970d0 2023-11-06T03:15:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55804ed655c0 con 0x55804ef970d0 2023-11-06T03:15:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff174004560 con 0x55804ef970d0 2023-11-06T03:15:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 10 client.9603 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55804ed3b030 con 0x55804ef970d0 2023-11-06T03:15:02.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55804ed655c0 con 0x55804ef970d0 2023-11-06T03:15:02.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff17401e710 con 0x55804ef970d0 2023-11-06T03:15:02.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 10 client.9603 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 10 client.9603 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 20 client.9603 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef92690 0x55804ef92aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 4 client.9603 mds_command: new command op to 9375 tid=0[{"prefix": "scrub pause"}] 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55804ed952c0 con 0x55804ef92690 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17ffff640 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef92690 0x55804ef92aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 10 client.9603 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff17ffff640 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef92690 0x55804ef92aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ca619c0875d0b4 server_cookie=8705f42993ac268a in_seq=0 out_seq=0 2023-11-06T03:15:02.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 1 -- 172.21.15.157:0/3152885895 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55804ed952c0 con 0x55804ef92690 2023-11-06T03:15:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 10 client.9603 handle_command_reply: tid=0 2023-11-06T03:15:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 client.9603 shutdown 2023-11-06T03:15:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 msgr2=0x55804ef91c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff174031f50 tx=0x7ff174013000 comp rx=0 tx=0).stop 2023-11-06T03:15:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 shutdown_connections 2023-11-06T03:15:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x55804ef92690 0x55804ef92aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55804ef979f0 0x55804ef92150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55804ef970d0 0x55804ef91c10 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff15effd640 0 client.9603 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 --2- 172.21.15.157:0/3152885895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55804ef4d200 0x55804efeddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 >> 172.21.15.157:0/3152885895 conn(0x55804ef53990 msgr2=0x55804ef53070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 shutdown_connections 2023-11-06T03:15:02.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 1 -- 172.21.15.157:0/3152885895 wait complete. 2023-11-06T03:15:02.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff1894b8000 20 client.9603 trim_cache size 0 max 16384 2023-11-06T03:15:02.545 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:02.545 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0 2023-11-06T03:15:02.546 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 msgr2=0x7ff16c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 0x7ff16c067f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff1800c2ce0 tx=0x7ff174008040 comp rx=0 tx=0).stop 2023-11-06T03:15:02.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 msgr2=0x7ff1800c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c450 tx=0x7ff17000c910 comp rx=0 tx=0).stop 2023-11-06T03:15:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 shutdown_connections 2023-11-06T03:15:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff16c065a60 0x7ff16c067f20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff18015cf10 0x7ff1800c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff18015cb30 0x7ff18015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 --2- 172.21.15.157:0/804381430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1800660f0 0x7ff1800c6360 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 >> 172.21.15.157:0/804381430 conn(0x7ff18006a170 msgr2=0x7ff1800ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 shutdown_connections 2023-11-06T03:15:02.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.516+0000 7ff186390640 1 -- 172.21.15.157:0/804381430 wait complete. 2023-11-06T03:15:02.550 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.9:0 scrub status 2023-11-06T03:15:02.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 Processor -- start 2023-11-06T03:15:02.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 -- start start 2023-11-06T03:15:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab01551e0 0x7faab006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faab015e5e0 0x7faab00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab015f580 con 0x7faab015ea60 2023-11-06T03:15:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab015efc0 con 0x7faab015e5e0 2023-11-06T03:15:02.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.668+0000 7faab7710640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab015f2a0 con 0x7faab01551e0 2023-11-06T03:15:02.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faab015e5e0 0x7faab00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab5cad640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faab015ea60 0x7faab015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51310/0 (socket says 172.21.15.157:51310) 2023-11-06T03:15:02.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab5cad640 1 -- 172.21.15.157:0/3682916856 learned_addr learned my addr 172.21.15.157:0/3682916856 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:02.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab54ac640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab01551e0 0x7faab006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 -- 172.21.15.157:0/3682916856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab01551e0 msgr2=0x7faab006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab01551e0 0x7faab006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 -- 172.21.15.157:0/3682916856 >> v1:172.21.15.157:6789/0 conn(0x7faab015ea60 legacy=0x7faab015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faaaffff640 1 -- 172.21.15.157:0/3682916856 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501373553 0 0) 0x7faab015f580 con 0x7faab015ea60 2023-11-06T03:15:02.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 -- 172.21.15.157:0/3682916856 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab006f5a0 con 0x7faab015e5e0 2023-11-06T03:15:02.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab4cab640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab00b3410 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009d90 tx=0x7faaa401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac6f006ea564a53 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faaaffff640 1 -- 172.21.15.157:0/3682916856 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40a5860 con 0x7faab015e5e0 2023-11-06T03:15:02.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faaaffff640 1 -- 172.21.15.157:0/3682916856 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40a59c0 con 0x7faab015e5e0 2023-11-06T03:15:02.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faaaffff640 1 -- 172.21.15.157:0/3682916856 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40a5d50 con 0x7faab015e5e0 2023-11-06T03:15:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 -- 172.21.15.157:0/3682916856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 msgr2=0x7faab00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab00b3410 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009d90 tx=0x7faaa401c890 comp rx=0 tx=0).stop 2023-11-06T03:15:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 -- 172.21.15.157:0/3682916856 shutdown_connections 2023-11-06T03:15:02.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab00b3410 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 --2- 172.21.15.157:0/3682916856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab01551e0 0x7faab006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 -- 172.21.15.157:0/3682916856 >> 172.21.15.157:0/3682916856 conn(0x7faab0065930 msgr2=0x7faab0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 -- 172.21.15.157:0/3682916856 shutdown_connections 2023-11-06T03:15:02.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.672+0000 7faab7710640 1 -- 172.21.15.157:0/3682916856 wait complete. 2023-11-06T03:15:02.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 Processor -- start 2023-11-06T03:15:02.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- start start 2023-11-06T03:15:02.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 0x7faab0146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab015ea60 0x7faab01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab014c1a0 con 0x7faab015e5e0 2023-11-06T03:15:02.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab0146880 con 0x7faab01551e0 2023-11-06T03:15:02.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab014bec0 con 0x7faab015ea60 2023-11-06T03:15:02.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54096/0 (socket says 172.21.15.157:54096) 2023-11-06T03:15:02.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 -- 172.21.15.157:0/840804556 learned_addr learned my addr 172.21.15.157:0/840804556 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab54ac640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 0x7faab0146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 -- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab015ea60 msgr2=0x7faab01540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab015ea60 0x7faab01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 -- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 msgr2=0x7faab0146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 0x7faab0146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 -- 172.21.15.157:0/840804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa4009a70 con 0x7faab015e5e0 2023-11-06T03:15:02.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab54ac640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 0x7faab0146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab4cab640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7faaa401cd70 tx=0x7faaa4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faaadffb640 1 -- 172.21.15.157:0/840804556 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40ad910 con 0x7faab015e5e0 2023-11-06T03:15:02.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faaadffb640 1 -- 172.21.15.157:0/840804556 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40ada70 con 0x7faab015e5e0 2023-11-06T03:15:02.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab014c670 con 0x7faab015e5e0 2023-11-06T03:15:02.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faaadffb640 1 -- 172.21.15.157:0/840804556 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40adde0 con 0x7faab015e5e0 2023-11-06T03:15:02.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab014cc30 con 0x7faab015e5e0 2023-11-06T03:15:02.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.676+0000 7faaadffb640 1 -- 172.21.15.157:0/840804556 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faaa40b4070 con 0x7faab015e5e0 2023-11-06T03:15:02.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaadffb640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 0x7faa9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaadffb640 1 -- 172.21.15.157:0/840804556 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faaa411dfa0 con 0x7faab015e5e0 2023-11-06T03:15:02.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 0x7faa9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 Processor -- start 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- start start 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 0x7faa9c067f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036a50 0x565499febd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x565499fec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499eb3c10 con 0x56549a036130 2023-11-06T03:15:02.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499e423e0 con 0x565499fec260 2023-11-06T03:15:02.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x565499e3f9b0 con 0x56549a036a50 2023-11-06T03:15:02.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54108/0 (socket says 172.21.15.157:54108) 2023-11-06T03:15:02.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 -- 172.21.15.157:0/3404599500 learned_addr learned my addr 172.21.15.157:0/3404599500 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab4cab640 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036a50 0x565499febd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab5cad640 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x565499fec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 -- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036a50 msgr2=0x565499febd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036a50 0x565499febd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 -- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 msgr2=0x565499fec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x565499fec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 -- 172.21.15.157:0/3404599500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565499b37e20 con 0x56549a036130 2023-11-06T03:15:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faab54ac640 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7faaa800db00 tx=0x7faaa8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a88be36c5050d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faa96ffd640 1 -- 172.21.15.157:0/3404599500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800b050 con 0x56549a036130 2023-11-06T03:15:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faa96ffd640 1 -- 172.21.15.157:0/3404599500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8017070 con 0x56549a036130 2023-11-06T03:15:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faa96ffd640 1 -- 172.21.15.157:0/3404599500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8012cb0 con 0x56549a036130 2023-11-06T03:15:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- 172.21.15.157:0/3404599500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 msgr2=0x56549a036510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7faaa800db00 tx=0x7faaa8010000 comp rx=0 tx=0).stop 2023-11-06T03:15:02.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- 172.21.15.157:0/3404599500 shutdown_connections 2023-11-06T03:15:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x565499fec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036a50 0x565499febd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 --2- 172.21.15.157:0/3404599500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a036510 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- 172.21.15.157:0/3404599500 >> 172.21.15.157:0/3404599500 conn(0x565499ff29f0 msgr2=0x565499fea600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- 172.21.15.157:0/3404599500 shutdown_connections 2023-11-06T03:15:02.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- 172.21.15.157:0/3404599500 wait complete. 2023-11-06T03:15:02.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.680+0000 7faaba838000 1 -- start start 2023-11-06T03:15:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 Processor -- start 2023-11-06T03:15:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x56549a0355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a036e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499e3b2a0 con 0x56549a036a50 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499dda470 con 0x565499fec260 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x565499df8660 con 0x56549a036130 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 10 client.0 _ll_register_callbacks cb 0x565499fd7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54112/0 (socket says 172.21.15.157:54112) 2023-11-06T03:15:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 -- 172.21.15.157:0/3883658036 learned_addr learned my addr 172.21.15.157:0/3883658036 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab54ac640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x56549a0355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab4cab640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a036e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 -- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 msgr2=0x56549a036e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a036e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 -- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 msgr2=0x56549a0355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x56549a0355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 -- 172.21.15.157:0/3883658036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa800d840 con 0x56549a036a50 2023-11-06T03:15:02.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab5cad640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001cd0 tx=0x7faaa000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f178d800a3a1761 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa00072d0 con 0x56549a036a50 2023-11-06T03:15:02.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faaa000de40 con 0x56549a036a50 2023-11-06T03:15:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565499df80a0 con 0x56549a036a50 2023-11-06T03:15:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa0007430 con 0x56549a036a50 2023-11-06T03:15:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 10 client.9618 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x565499e001d0 con 0x56549a036a50 2023-11-06T03:15:02.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565499df80a0 con 0x56549a036a50 2023-11-06T03:15:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7faaa00047f0 con 0x56549a036a50 2023-11-06T03:15:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 10 client.9618 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 10 client.9618 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 20 client.9618 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a037d80 0x56549a038190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 4 client.9618 mds_command: new command op to 9375 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565499e39bf0 con 0x56549a037d80 2023-11-06T03:15:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab54ac640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a037d80 0x56549a038190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 10 client.9618 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faab54ac640 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a037d80 0x56549a038190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922ec597b0a405c8 server_cookie=b1ec56cf5ec2591 in_seq=0 out_seq=0 2023-11-06T03:15:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 1 -- 172.21.15.157:0/3883658036 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x565499e39bf0 con 0x56549a037d80 2023-11-06T03:15:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 10 client.9618 handle_command_reply: tid=0 2023-11-06T03:15:02.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 client.9618 shutdown 2023-11-06T03:15:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 msgr2=0x56549a037370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001cd0 tx=0x7faaa000d520 comp rx=0 tx=0).stop 2023-11-06T03:15:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 shutdown_connections 2023-11-06T03:15:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a037d80 0x56549a038190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036a50 0x56549a037370 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a036e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faa8ffff640 0 client.9618 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 --2- 172.21.15.157:0/3883658036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x565499fec260 0x56549a0355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 >> 172.21.15.157:0/3883658036 conn(0x565499ff29f0 msgr2=0x565499fead00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 shutdown_connections 2023-11-06T03:15:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 1 -- 172.21.15.157:0/3883658036 wait complete. 2023-11-06T03:15:02.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.684+0000 7faaba838000 20 client.9618 trim_cache size 0 max 16384 2023-11-06T03:15:02.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 Processor -- start 2023-11-06T03:15:02.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- start start 2023-11-06T03:15:02.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565499fec260 0x56549a0431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a0332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499af8aa0 con 0x56549a036130 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499ec3c70 con 0x56549a036a50 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x565499ff0700 con 0x565499fec260 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54128/0 (socket says 172.21.15.157:54128) 2023-11-06T03:15:02.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 -- 172.21.15.157:0/18544482 learned_addr learned my addr 172.21.15.157:0/18544482 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 -- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565499fec260 msgr2=0x56549a0431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565499fec260 0x56549a0431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 -- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 msgr2=0x56549a0332d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a0332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 -- 172.21.15.157:0/18544482 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565499de6520 con 0x56549a036130 2023-11-06T03:15:02.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faab4cab640 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7faaa401af40 tx=0x7faaa40a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd14050d8662c140 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faa977fe640 1 -- 172.21.15.157:0/18544482 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40f3410 con 0x56549a036130 2023-11-06T03:15:02.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faa977fe640 1 -- 172.21.15.157:0/18544482 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40f75c0 con 0x56549a036130 2023-11-06T03:15:02.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faa977fe640 1 -- 172.21.15.157:0/18544482 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40ebcf0 con 0x56549a036130 2023-11-06T03:15:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- 172.21.15.157:0/18544482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 msgr2=0x56549a043700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7faaa401af40 tx=0x7faaa40a5f90 comp rx=0 tx=0).stop 2023-11-06T03:15:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- 172.21.15.157:0/18544482 shutdown_connections 2023-11-06T03:15:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a0332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56549a036130 0x56549a043700 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 --2- 172.21.15.157:0/18544482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x565499fec260 0x56549a0431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.700+0000 7faaba838000 1 -- 172.21.15.157:0/18544482 >> 172.21.15.157:0/18544482 conn(0x565499ff29f0 msgr2=0x565499ff1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/18544482 shutdown_connections 2023-11-06T03:15:02.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/18544482 wait complete. 2023-11-06T03:15:02.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- start start 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 Processor -- start 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a095750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a097ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499df9400 con 0x565499fec260 2023-11-06T03:15:02.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x565499e7dcd0 con 0x56549a036a50 2023-11-06T03:15:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x565499de07a0 con 0x56549a036130 2023-11-06T03:15:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 10 client.0 _ll_register_callbacks cb 0x565499dffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54138/0 (socket says 172.21.15.157:54138) 2023-11-06T03:15:02.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 -- 172.21.15.157:0/3164705434 learned_addr learned my addr 172.21.15.157:0/3164705434 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 -- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 msgr2=0x56549a095750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a095750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 -- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 msgr2=0x56549a097ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab5cad640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a097ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a097ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 -- 172.21.15.157:0/3164705434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab014cc30 con 0x565499fec260 2023-11-06T03:15:02.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab54ac640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7faaa801f000 tx=0x7faaa80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683e82a29eec9da5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:02.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800b040 con 0x565499fec260 2023-11-06T03:15:02.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faaa8012e50 con 0x565499fec260 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565499e04620 con 0x565499fec260 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa80316a0 con 0x565499fec260 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 10 client.9624 fetch_fsmap learned FSMap version 99 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x565499dda090 con 0x565499fec260 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565499e04620 con 0x565499fec260 2023-11-06T03:15:02.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 99) v1 ==== 3106+0+0 (secure 0 0 0) 0x7faaa8031800 con 0x565499fec260 2023-11-06T03:15:02.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 10 client.9624 fetch_fsmap finished waiting for FSMap version 99 2023-11-06T03:15:02.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 10 client.9624 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2023-11-06T03:15:02.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 20 client.9624 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:02.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a0981e0 0x56549a09a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 4 client.9624 mds_command: new command op to 9375 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:15:02.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 --> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565499e34320 con 0x56549a0981e0 2023-11-06T03:15:02.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab4cab640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a0981e0 0x56549a09a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 10 client.9624 ms_handle_connect on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faab4cab640 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a0981e0 0x56549a09a5d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748f86082314961e server_cookie=536874e39003906a in_seq=0 out_seq=0 2023-11-06T03:15:02.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 1 -- 172.21.15.157:0/3164705434 <== mds.0 v2:172.21.15.157:6834/1004239715 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x565499e34320 con 0x56549a0981e0 2023-11-06T03:15:02.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 10 client.9624 handle_command_reply: tid=0 2023-11-06T03:15:02.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 client.9624 shutdown 2023-11-06T03:15:02.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 msgr2=0x56549a08f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7faaa801f000 tx=0x7faaa80046d0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 shutdown_connections 2023-11-06T03:15:02.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:6834/1004239715,v1:172.21.15.157:6835/1004239715] conn(0x56549a0981e0 0x56549a09a5d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56549a036a50 0x56549a097ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56549a036130 0x56549a095750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faa8ffff640 0 client.9624 ms_handle_reset on v2:172.21.15.157:6834/1004239715 2023-11-06T03:15:02.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 --2- 172.21.15.157:0/3164705434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565499fec260 0x56549a08f090 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 >> 172.21.15.157:0/3164705434 conn(0x565499ff29f0 msgr2=0x565499ff1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 shutdown_connections 2023-11-06T03:15:02.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 1 -- 172.21.15.157:0/3164705434 wait complete. 2023-11-06T03:15:02.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.704+0000 7faaba838000 20 client.9624 trim_cache size 0 max 16384 2023-11-06T03:15:02.727 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:02.727 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "PAUSED (0 inodes in the stack)", 2023-11-06T03:15:02.727 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:15:02.727 INFO:teuthology.orchestra.run.smithi157.stdout: "fab56f51-4cc6-4ed1-b7a1-cac0ea9e6c10": { 2023-11-06T03:15:02.727 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/scrub_control_test_path", 2023-11-06T03:15:02.728 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "fab56f51-4cc6-4ed1-b7a1-cac0ea9e6c10", 2023-11-06T03:15:02.728 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:15:02.728 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:02.728 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:02.728 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 msgr2=0x7faa9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 0x7faa9c067f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 msgr2=0x7faab0153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7faaa401cd70 tx=0x7faaa4019630 comp rx=0 tx=0).stop 2023-11-06T03:15:02.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 shutdown_connections 2023-11-06T03:15:02.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faa9c065a60 0x7faa9c067f20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faab015ea60 0x7faab01540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faab015e5e0 0x7faab0153b90 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 --2- 172.21.15.157:0/840804556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faab01551e0 0x7faab0146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 >> 172.21.15.157:0/840804556 conn(0x7faab0065930 msgr2=0x7faab00ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 shutdown_connections 2023-11-06T03:15:02.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.708+0000 7faab7710640 1 -- 172.21.15.157:0/840804556 wait complete. 2023-11-06T03:15:02.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:02.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.848+0000 7f5c4e37f640 1 Processor -- start 2023-11-06T03:15:02.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.848+0000 7f5c4e37f640 1 -- start start 2023-11-06T03:15:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c4e37f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c48156a70 0x7f5c4806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c4e37f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5c480adae0 0x7f5c480b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c4e37f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4806c490 con 0x7f5c480adf40 2023-11-06T03:15:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c4e37f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4806b950 con 0x7f5c48156a70 2023-11-06T03:15:02.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c4e37f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4806bb60 con 0x7f5c480adae0 2023-11-06T03:15:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c48156a70 0x7f5c4806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c48156a70 0x7f5c4806f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54150/0 (socket says 172.21.15.157:54150) 2023-11-06T03:15:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 -- 172.21.15.157:0/1080094204 learned_addr learned my addr 172.21.15.157:0/1080094204 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c477fe640 1 --2- 172.21.15.157:0/1080094204 >> v2:172.21.15.195:3300/0 conn(0x7f5c480adae0 0x7f5c480b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 -- 172.21.15.157:0/1080094204 >> v2:172.21.15.195:3300/0 conn(0x7f5c480adae0 msgr2=0x7f5c480b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 --2- 172.21.15.157:0/1080094204 >> v2:172.21.15.195:3300/0 conn(0x7f5c480adae0 0x7f5c480b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 -- 172.21.15.157:0/1080094204 >> v1:172.21.15.195:6789/0 conn(0x7f5c480adf40 legacy=0x7f5c4806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:02.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 -- 172.21.15.157:0/1080094204 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4806be40 con 0x7f5c48156a70 2023-11-06T03:15:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c47fff640 1 --2- 172.21.15.157:0/1080094204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c48156a70 0x7f5c4806f470 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5c480cf220 tx=0x7f5c3801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fd462cf42103f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.852+0000 7f5c46ffd640 1 -- 172.21.15.157:0/1080094204 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380a7c90 con 0x7f5c48156a70 2023-11-06T03:15:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c46ffd640 1 -- 172.21.15.157:0/1080094204 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380ab070 con 0x7f5c48156a70 2023-11-06T03:15:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c46ffd640 1 -- 172.21.15.157:0/1080094204 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380b1400 con 0x7f5c48156a70 2023-11-06T03:15:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/1080094204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c48156a70 msgr2=0x7f5c4806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/1080094204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c48156a70 0x7f5c4806f470 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5c480cf220 tx=0x7f5c3801eba0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/1080094204 shutdown_connections 2023-11-06T03:15:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/1080094204 >> v2:172.21.15.195:3300/0 conn(0x7f5c480adae0 0x7f5c480b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/1080094204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c48156a70 0x7f5c4806f470 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/1080094204 >> 172.21.15.157:0/1080094204 conn(0x7f5c48065930 msgr2=0x7f5c48065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:02.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/1080094204 shutdown_connections 2023-11-06T03:15:02.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/1080094204 wait complete. 2023-11-06T03:15:02.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 Processor -- start 2023-11-06T03:15:02.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- start start 2023-11-06T03:15:02.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 0x7f5c4814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 0x7f5c480cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4814f040 con 0x7f5c480adae0 2023-11-06T03:15:02.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48155940 con 0x7f5c480adf40 2023-11-06T03:15:02.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48155c20 con 0x7f5c48156a70 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54154/0 (socket says 172.21.15.157:54154) 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 -- 172.21.15.157:0/2703258053 learned_addr learned my addr 172.21.15.157:0/2703258053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4c91c640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 0x7f5c480cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c477fe640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 0x7f5c4814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 -- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 msgr2=0x7f5c480cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 0x7f5c480cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 -- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 msgr2=0x7f5c4814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:02.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 0x7f5c4814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:02.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 -- 172.21.15.157:0/2703258053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3800bd40 con 0x7f5c480adae0 2023-11-06T03:15:02.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4c91c640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 0x7f5c480cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c477fe640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 0x7f5c4814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:02.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c47fff640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f5c38000f90 tx=0x7f5c3801b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380bc910 con 0x7f5c480adae0 2023-11-06T03:15:02.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380bca70 con 0x7f5c480adae0 2023-11-06T03:15:02.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c48155f00 con 0x7f5c480adae0 2023-11-06T03:15:02.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380bce00 con 0x7f5c480adae0 2023-11-06T03:15:02.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.856+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c480b1010 con 0x7f5c480adae0 2023-11-06T03:15:02.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c48072750 con 0x7f5c480adae0 2023-11-06T03:15:02.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5c380ba410 con 0x7f5c480adae0 2023-11-06T03:15:02.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c44ff9640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 0x7f5c34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:02.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c38120560 con 0x7f5c480adae0 2023-11-06T03:15:02.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c477fe640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 0x7f5c34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:02.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.860+0000 7f5c477fe640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 0x7f5c34067f50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c481f4600 tx=0x7f5c3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:02.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:02.864+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5c380ede50 con 0x7f5c480adae0 2023-11-06T03:15:03.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c481577c0 con 0x7f5c480adae0 2023-11-06T03:15:03.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c44ff9640 1 -- 172.21.15.157:0/2703258053 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f5c380f1470 con 0x7f5c480adae0 2023-11-06T03:15:03.045 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:03.045 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:03.046 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 msgr2=0x7f5c34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 0x7f5c34067f50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c481f4600 tx=0x7f5c3c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 msgr2=0x7f5c4814b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f5c38000f90 tx=0x7f5c3801b940 comp rx=0 tx=0).stop 2023-11-06T03:15:03.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 shutdown_connections 2023-11-06T03:15:03.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5c34065a90 0x7f5c34067f50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5c48156a70 0x7f5c480cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5c480adf40 0x7f5c4814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 --2- 172.21.15.157:0/2703258053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c480adae0 0x7f5c4814b870 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 >> 172.21.15.157:0/2703258053 conn(0x7f5c48065930 msgr2=0x7f5c48163290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:03.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 shutdown_connections 2023-11-06T03:15:03.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.048+0000 7f5c4e37f640 1 -- 172.21.15.157:0/2703258053 wait complete. 2023-11-06T03:15:03.062 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:15:03.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:15:03.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 Processor -- start 2023-11-06T03:15:03.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 -- start start 2023-11-06T03:15:03.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f84d80b1d40 0x7f84d80cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80c94f0 con 0x7f84d815aa80 2023-11-06T03:15:03.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80c97b0 con 0x7f84d80b17f0 2023-11-06T03:15:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.180+0000 7f84deb0d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d815ae60 con 0x7f84d80b1d40 2023-11-06T03:15:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84d815aa80 0x7f84d80b30e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51318/0 (socket says 172.21.15.157:51318) 2023-11-06T03:15:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 -- 172.21.15.157:0/4705875 learned_addr learned my addr 172.21.15.157:0/4705875 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d7fff640 1 --2- 172.21.15.157:0/4705875 >> v2:172.21.15.195:3300/0 conn(0x7f84d80b1d40 0x7f84d80cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205117402 0 0) 0x7f84d80c94f0 con 0x7f84d815aa80 2023-11-06T03:15:03.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c4003530 con 0x7f84d815aa80 2023-11-06T03:15:03.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3541662018 0 0) 0x7f84c4003530 con 0x7f84d815aa80 2023-11-06T03:15:03.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 >> v2:172.21.15.195:3300/0 conn(0x7f84d80b1d40 msgr2=0x7f84d80cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:03.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 --2- 172.21.15.157:0/4705875 >> v2:172.21.15.195:3300/0 conn(0x7f84d80b1d40 0x7f84d80cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 >> v1:172.21.15.195:6789/0 conn(0x7f84d80b17f0 legacy=0x7f84d815e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80c9a90 con 0x7f84d815aa80 2023-11-06T03:15:03.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f84c8001680 con 0x7f84d815aa80 2023-11-06T03:15:03.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 >> v1:172.21.15.157:6789/0 conn(0x7f84d815aa80 legacy=0x7f84d80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84c4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40047d0 0x7f84c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4005190 0x7f84c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d815ae60 con 0x7f84c4003980 2023-11-06T03:15:03.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400c030 con 0x7f84c4005190 2023-11-06T03:15:03.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400c1f0 con 0x7f84c40047d0 2023-11-06T03:15:03.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84c4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dd0aa640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4005190 0x7f84c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84d7fff640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40047d0 0x7f84c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 -- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40047d0 msgr2=0x7f84c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:03.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84c40047d0 0x7f84c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 -- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4005190 msgr2=0x7f84c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:03.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4005190 0x7f84c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dc8a9640 1 -- 172.21.15.157:0/4705875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4003530 con 0x7f84c4003980 2023-11-06T03:15:03.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dd0aa640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84c4005190 0x7f84c400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:03.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.184+0000 7f84dd0aa640 1 -- 172.21.15.157:0/4705875 reap_dead start 2023-11-06T03:15:03.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84c4003da0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c240 tx=0x7f84c801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b3048c97149397 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:03.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8005ae0 con 0x7f84c4003980 2023-11-06T03:15:03.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8005c40 con 0x7f84c4003980 2023-11-06T03:15:03.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d77fe640 1 -- 172.21.15.157:0/4705875 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c80a5a40 con 0x7f84c4003980 2023-11-06T03:15:03.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 msgr2=0x7f84c4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84c4003da0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c240 tx=0x7f84c801cc10 comp rx=0 tx=0).stop 2023-11-06T03:15:03.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/4705875 shutdown_connections 2023-11-06T03:15:03.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 --2- 172.21.15.157:0/4705875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84c4003da0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/4705875 >> 172.21.15.157:0/4705875 conn(0x7f84d806a170 msgr2=0x7f84d806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:03.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/4705875 shutdown_connections 2023-11-06T03:15:03.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/4705875 wait complete. 2023-11-06T03:15:03.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 Processor -- start 2023-11-06T03:15:03.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- start start 2023-11-06T03:15:03.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d81f3030 0x7f84d80a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d80a9400 0x7f84d80cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80aae50 con 0x7f84c4003980 2023-11-06T03:15:03.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81f66a0 con 0x7f84d81f3030 2023-11-06T03:15:03.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80aa4e0 con 0x7f84d80a9400 2023-11-06T03:15:03.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54170/0 (socket says 172.21.15.157:54170) 2023-11-06T03:15:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 -- 172.21.15.157:0/2996033156 learned_addr learned my addr 172.21.15.157:0/2996033156 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 -- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d80a9400 msgr2=0x7f84d80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dd0aa640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d80a9400 0x7f84d80cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d80a9400 0x7f84d80cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d81f3030 0x7f84d80a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 -- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d81f3030 msgr2=0x7f84d80a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d81f3030 0x7f84d80a8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 -- 172.21.15.157:0/2996033156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80a9960 con 0x7f84c4003980 2023-11-06T03:15:03.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84dc8a9640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c240 tx=0x7f84c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c80b4070 con 0x7f84c4003980 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8001480 con 0x7f84c4003980 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80a9c40 con 0x7f84c4003980 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c80a5a40 con 0x7f84c4003980 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.188+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d80aa200 con 0x7f84c4003980 2023-11-06T03:15:03.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d8072750 con 0x7f84c4003980 2023-11-06T03:15:03.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f84c80a5ba0 con 0x7f84c4003980 2023-11-06T03:15:03.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84d57fa640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 0x7f84b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:03.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84c811c500 con 0x7f84c4003980 2023-11-06T03:15:03.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 0x7f84b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:03.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.192+0000 7f84d7fff640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 0x7f84b8067f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f84d806cb10 tx=0x7f84c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:03.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.196+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84c80e9d60 con 0x7f84c4003980 2023-11-06T03:15:03.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f84d8070f50 con 0x7f84c4003980 2023-11-06T03:15:03.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84d57fa640 1 -- 172.21.15.157:0/2996033156 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1445 (secure 0 0 0) 0x7f84c80ed380 con 0x7f84c4003980 2023-11-06T03:15:03.408 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:03.409 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":877,"data_bytes":896957257,"bytes_used":2360823808,"bytes_avail":770733289472,"bytes_total":773094113280,"write_bytes_sec":50549662,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":99,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9375}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-11-06T03:14:25.321050+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:15:03.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 msgr2=0x7f84b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 0x7f84b8067f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f84d806cb10 tx=0x7f84c0008040 comp rx=0 tx=0).stop 2023-11-06T03:15:03.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 msgr2=0x7f84d81f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c240 tx=0x7f84c4005800 comp rx=0 tx=0).stop 2023-11-06T03:15:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 shutdown_connections 2023-11-06T03:15:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f84b8065a60 0x7f84b8067f20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84d80a9400 0x7f84d80cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f84d81f3030 0x7f84d80a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 --2- 172.21.15.157:0/2996033156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84c4003980 0x7f84d81f2af0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 >> 172.21.15.157:0/2996033156 conn(0x7f84d806a170 msgr2=0x7f84d80b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 shutdown_connections 2023-11-06T03:15:03.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:03.412+0000 7f84deb0d640 1 -- 172.21.15.157:0/2996033156 wait complete. 2023-11-06T03:15:03.427 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:15:04.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:04.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.564+0000 7efea95af640 1 Processor -- start 2023-11-06T03:15:04.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.564+0000 7efea95af640 1 -- start start 2023-11-06T03:15:04.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea95af640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efea415e5e0 0x7efea40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea95af640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efea415ea90 0x7efea40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea95af640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea415f580 con 0x7efea41551e0 2023-11-06T03:15:04.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea95af640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea40704b0 con 0x7efea415ea90 2023-11-06T03:15:04.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea95af640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea415efc0 con 0x7efea415e5e0 2023-11-06T03:15:04.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efea415ea90 0x7efea40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efea415ea90 0x7efea40b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54186/0 (socket says 172.21.15.157:54186) 2023-11-06T03:15:04.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 -- 172.21.15.157:0/3876652 learned_addr learned my addr 172.21.15.157:0/3876652 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:04.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea27fc640 1 --2- 172.21.15.157:0/3876652 >> v2:172.21.15.195:3300/0 conn(0x7efea415e5e0 0x7efea40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 -- 172.21.15.157:0/3876652 >> v2:172.21.15.195:3300/0 conn(0x7efea415e5e0 msgr2=0x7efea40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:04.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 --2- 172.21.15.157:0/3876652 >> v2:172.21.15.195:3300/0 conn(0x7efea415e5e0 0x7efea40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 -- 172.21.15.157:0/3876652 >> v1:172.21.15.195:6789/0 conn(0x7efea41551e0 legacy=0x7efea406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 -- 172.21.15.157:0/3876652 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea415f2a0 con 0x7efea415ea90 2023-11-06T03:15:04.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea37fe640 1 --2- 172.21.15.157:0/3876652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415ea90 0x7efea40b3950 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7efea4066880 tx=0x7efe9c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb1a0e58eae96a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea1ffb640 1 -- 172.21.15.157:0/3876652 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0a57d0 con 0x7efea415ea90 2023-11-06T03:15:04.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea1ffb640 1 -- 172.21.15.157:0/3876652 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c0a5930 con 0x7efea415ea90 2023-11-06T03:15:04.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.568+0000 7efea1ffb640 1 -- 172.21.15.157:0/3876652 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0a5cc0 con 0x7efea415ea90 2023-11-06T03:15:04.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3876652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415ea90 msgr2=0x7efea40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- 172.21.15.157:0/3876652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415ea90 0x7efea40b3950 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7efea4066880 tx=0x7efe9c01c910 comp rx=0 tx=0).stop 2023-11-06T03:15:04.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3876652 shutdown_connections 2023-11-06T03:15:04.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- 172.21.15.157:0/3876652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415ea90 0x7efea40b3950 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- 172.21.15.157:0/3876652 >> v2:172.21.15.195:3300/0 conn(0x7efea415e5e0 0x7efea40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3876652 >> 172.21.15.157:0/3876652 conn(0x7efea4065930 msgr2=0x7efea4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:04.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3876652 shutdown_connections 2023-11-06T03:15:04.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3876652 wait complete. 2023-11-06T03:15:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 Processor -- start 2023-11-06T03:15:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- start start 2023-11-06T03:15:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 0x7efea4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efea415ea90 0x7efea41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41589d0 con 0x7efea415e5e0 2023-11-06T03:15:04.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea415a460 con 0x7efea41551e0 2023-11-06T03:15:04.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea41556f0 con 0x7efea415ea90 2023-11-06T03:15:04.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54188/0 (socket says 172.21.15.157:54188) 2023-11-06T03:15:04.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 -- 172.21.15.157:0/3248309911 learned_addr learned my addr 172.21.15.157:0/3248309911 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:04.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea2ffd640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 0x7efea4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 -- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efea415ea90 msgr2=0x7efea41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea37fe640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efea415ea90 0x7efea41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efea415ea90 0x7efea41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 -- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 msgr2=0x7efea4157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 0x7efea4157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 -- 172.21.15.157:0/3248309911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c009b30 con 0x7efea415e5e0 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea2ffd640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 0x7efea4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:04.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea27fc640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d830 tx=0x7efe8c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00b6c0 con 0x7efea415e5e0 2023-11-06T03:15:04.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c00b820 con 0x7efea415e5e0 2023-11-06T03:15:04.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00bb90 con 0x7efea415e5e0 2023-11-06T03:15:04.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea41559d0 con 0x7efea415e5e0 2023-11-06T03:15:04.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.572+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea4155f90 con 0x7efea415e5e0 2023-11-06T03:15:04.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea4156120 con 0x7efea415e5e0 2023-11-06T03:15:04.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efe8c01e460 con 0x7efea415e5e0 2023-11-06T03:15:04.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efe9b7fe640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 0x7efe84067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe8c083a20 con 0x7efea415e5e0 2023-11-06T03:15:04.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efea2ffd640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 0x7efe84067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.576+0000 7efea2ffd640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 0x7efe84067f20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efea414eaa0 tx=0x7efe94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.580+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efe8c051310 con 0x7efea415e5e0 2023-11-06T03:15:04.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.768+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efea4072750 con 0x7efea415e5e0 2023-11-06T03:15:04.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.768+0000 7efe9b7fe640 1 -- 172.21.15.157:0/3248309911 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7efe8c054930 con 0x7efea415e5e0 2023-11-06T03:15:04.767 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:04.767 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:04.768 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:04.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 msgr2=0x7efe84067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 0x7efe84067f20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efea414eaa0 tx=0x7efe94009040 comp rx=0 tx=0).stop 2023-11-06T03:15:04.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 msgr2=0x7efea41564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d830 tx=0x7efe8c00dcf0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 shutdown_connections 2023-11-06T03:15:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efe84065a60 0x7efe84067f20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efea415ea90 0x7efea41599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea415e5e0 0x7efea41564b0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 --2- 172.21.15.157:0/3248309911 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efea41551e0 0x7efea4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 >> 172.21.15.157:0/3248309911 conn(0x7efea4065930 msgr2=0x7efea40b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:04.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 shutdown_connections 2023-11-06T03:15:04.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.772+0000 7efea95af640 1 -- 172.21.15.157:0/3248309911 wait complete. 2023-11-06T03:15:04.785 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:15:04.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:15:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 Processor -- start 2023-11-06T03:15:04.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 -- start start 2023-11-06T03:15:04.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f50c415aeb0 0x7f50c4071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50c40d0400 0x7f50c40b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40b3c80 con 0x7f50c415b290 2023-11-06T03:15:04.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c40ce890 con 0x7f50c40d0400 2023-11-06T03:15:04.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.912+0000 7f50cac71640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c40af640 con 0x7f50c415aeb0 2023-11-06T03:15:04.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f50c40d0400 0x7f50c40b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f50c415b290 0x7f50c40cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51320/0 (socket says 172.21.15.157:51320) 2023-11-06T03:15:04.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c3fff640 1 -- 172.21.15.157:0/1783640341 learned_addr learned my addr 172.21.15.157:0/1783640341 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:04.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c8a0d640 1 --2- 172.21.15.157:0/1783640341 >> v2:172.21.15.195:3300/0 conn(0x7f50c415aeb0 0x7f50c4071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 -- 172.21.15.157:0/1783640341 >> v2:172.21.15.195:3300/0 conn(0x7f50c415aeb0 msgr2=0x7f50c4071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:04.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 --2- 172.21.15.157:0/1783640341 >> v2:172.21.15.195:3300/0 conn(0x7f50c415aeb0 0x7f50c4071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 -- 172.21.15.157:0/1783640341 >> v1:172.21.15.157:6789/0 conn(0x7f50c415b290 legacy=0x7f50c40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c37fe640 1 -- 172.21.15.157:0/1783640341 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593724376 0 0) 0x7f50c40b3c80 con 0x7f50c415b290 2023-11-06T03:15:04.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 -- 172.21.15.157:0/1783640341 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40af920 con 0x7f50c40d0400 2023-11-06T03:15:04.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c920e640 1 --2- 172.21.15.157:0/1783640341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c40d0400 0x7f50c40b3740 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f50c4066930 tx=0x7f50bc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f6eb354bbd1484 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c37fe640 1 -- 172.21.15.157:0/1783640341 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0a59a0 con 0x7f50c40d0400 2023-11-06T03:15:04.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c37fe640 1 -- 172.21.15.157:0/1783640341 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc0a5b00 con 0x7f50c40d0400 2023-11-06T03:15:04.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50c37fe640 1 -- 172.21.15.157:0/1783640341 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0a5e90 con 0x7f50c40d0400 2023-11-06T03:15:04.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 -- 172.21.15.157:0/1783640341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c40d0400 msgr2=0x7f50c40b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 --2- 172.21.15.157:0/1783640341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c40d0400 0x7f50c40b3740 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f50c4066930 tx=0x7f50bc01c8e0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 -- 172.21.15.157:0/1783640341 shutdown_connections 2023-11-06T03:15:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 --2- 172.21.15.157:0/1783640341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c40d0400 0x7f50c40b3740 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 --2- 172.21.15.157:0/1783640341 >> v2:172.21.15.195:3300/0 conn(0x7f50c415aeb0 0x7f50c4071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 -- 172.21.15.157:0/1783640341 >> 172.21.15.157:0/1783640341 conn(0x7f50c40cd370 msgr2=0x7f50c40cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 -- 172.21.15.157:0/1783640341 shutdown_connections 2023-11-06T03:15:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.916+0000 7f50cac71640 1 -- 172.21.15.157:0/1783640341 wait complete. 2023-11-06T03:15:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 Processor -- start 2023-11-06T03:15:04.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- start start 2023-11-06T03:15:04.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 0x7f50c4153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 0x7f50c415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40adcf0 con 0x7f50c415aeb0 2023-11-06T03:15:04.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40ad090 con 0x7f50c40d0400 2023-11-06T03:15:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40ad370 con 0x7f50c415b290 2023-11-06T03:15:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54192/0 (socket says 172.21.15.157:54192) 2023-11-06T03:15:04.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 -- 172.21.15.157:0/2190349374 learned_addr learned my addr 172.21.15.157:0/2190349374 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c920e640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 0x7f50c415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c8a0d640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 0x7f50c4153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 -- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 msgr2=0x7f50c415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 0x7f50c415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 -- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 msgr2=0x7f50c4153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:04.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 0x7f50c4153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 -- 172.21.15.157:0/2190349374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc009a70 con 0x7f50c415aeb0 2023-11-06T03:15:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c8a0d640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 0x7f50c4153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c920e640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 0x7f50c415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c3fff640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f50b000c450 tx=0x7f50b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00074e0 con 0x7f50c415aeb0 2023-11-06T03:15:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0007640 con 0x7f50c415aeb0 2023-11-06T03:15:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c40ad6a0 con 0x7f50c415aeb0 2023-11-06T03:15:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00049f0 con 0x7f50c415aeb0 2023-11-06T03:15:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4154030 con 0x7f50c415aeb0 2023-11-06T03:15:04.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c406cc80 con 0x7f50c415aeb0 2023-11-06T03:15:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.920+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f50b0004b50 con 0x7f50c415aeb0 2023-11-06T03:15:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.924+0000 7f50c17fa640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 0x7f50a8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.924+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50b00828f0 con 0x7f50c415aeb0 2023-11-06T03:15:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.924+0000 7f50c8a0d640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 0x7f50a8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.924+0000 7f50c8a0d640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 0x7f50a8067f50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009840 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:04.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:04.928+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f50b0053b40 con 0x7f50c415aeb0 2023-11-06T03:15:05.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.132+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f50c4072750 con 0x7f50c415aeb0 2023-11-06T03:15:05.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50c17fa640 1 -- 172.21.15.157:0/2190349374 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1445 (secure 0 0 0) 0x7f50b0018020 con 0x7f50c415aeb0 2023-11-06T03:15:05.132 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:05.132 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":682,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":877,"data_bytes":896957257,"bytes_used":2360823808,"bytes_avail":770733289472,"bytes_total":773094113280,"write_bytes_sec":50551529,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":99,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9375}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-11-06T03:14:25.321050+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:15:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 msgr2=0x7f50a8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 0x7f50a8067f50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009840 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:05.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 msgr2=0x7f50c415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f50b000c450 tx=0x7f50b000c910 comp rx=0 tx=0).stop 2023-11-06T03:15:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 shutdown_connections 2023-11-06T03:15:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f50a8065a90 0x7f50a8067f50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50c415b290 0x7f50c415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:05.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50c415aeb0 0x7f50c415dd80 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:05.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 --2- 172.21.15.157:0/2190349374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50c40d0400 0x7f50c4153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:05.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 >> 172.21.15.157:0/2190349374 conn(0x7f50c40cd370 msgr2=0x7f50c40b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:05.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 shutdown_connections 2023-11-06T03:15:05.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:05.136+0000 7f50cac71640 1 -- 172.21.15.157:0/2190349374 wait complete. 2023-11-06T03:15:05.149 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:15:06.150 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:06.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.280+0000 7f45263e5640 1 Processor -- start 2023-11-06T03:15:06.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.280+0000 7f45263e5640 1 -- start start 2023-11-06T03:15:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f45263e5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f45200b2af0 0x7f45200ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f45263e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452015e440 con 0x7f45201599c0 2023-11-06T03:15:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f45263e5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452006c240 con 0x7f452006d000 2023-11-06T03:15:06.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f45263e5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452006c520 con 0x7f45200b2af0 2023-11-06T03:15:06.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f45201599c0 0x7f45200b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51330/0 (socket says 172.21.15.157:51330) 2023-11-06T03:15:06.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451ffff640 1 -- 172.21.15.157:0/3896144744 learned_addr learned my addr 172.21.15.157:0/3896144744 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:06.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451f7fe640 1 --2- 172.21.15.157:0/3896144744 >> v2:172.21.15.195:3300/0 conn(0x7f45200b2af0 0x7f45200ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560271412 0 0) 0x7f452015e440 con 0x7f45201599c0 2023-11-06T03:15:06.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4508005740 con 0x7f45201599c0 2023-11-06T03:15:06.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879307047 0 0) 0x7f4508005740 con 0x7f45201599c0 2023-11-06T03:15:06.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 >> v2:172.21.15.195:3300/0 conn(0x7f45200b2af0 msgr2=0x7f45200ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:06.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 --2- 172.21.15.157:0/3896144744 >> v2:172.21.15.195:3300/0 conn(0x7f45200b2af0 0x7f45200ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 >> v1:172.21.15.195:6789/0 conn(0x7f452006d000 legacy=0x7f4520159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452006c800 con 0x7f45201599c0 2023-11-06T03:15:06.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4514001680 con 0x7f45201599c0 2023-11-06T03:15:06.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 >> v1:172.21.15.157:6789/0 conn(0x7f45201599c0 legacy=0x7f45200b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4508005b90 0x7f4508005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45080069e0 0x7f4508006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 0x7f450800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452006c520 con 0x7f45080073a0 2023-11-06T03:15:06.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450800e240 con 0x7f45080069e0 2023-11-06T03:15:06.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450800e400 con 0x7f4508005b90 2023-11-06T03:15:06.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 0x7f450800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451f7fe640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45080069e0 0x7f4508006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451ffff640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4508005b90 0x7f4508005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 -- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4508005b90 msgr2=0x7f4508005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4508005b90 0x7f4508005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 -- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45080069e0 msgr2=0x7f4508006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45080069e0 0x7f4508006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f4524982640 1 -- 172.21.15.157:0/3896144744 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508005740 con 0x7f45080073a0 2023-11-06T03:15:06.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.284+0000 7f451ffff640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4508005b90 0x7f4508005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:06.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 -- 172.21.15.157:0/3896144744 reap_dead start 2023-11-06T03:15:06.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 0x7f450800dba0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f4518001e40 tx=0x7f451801c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16490eabd3a09a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518003b80 con 0x7f45080073a0 2023-11-06T03:15:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518003ce0 con 0x7f45080073a0 2023-11-06T03:15:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f451effd640 1 -- 172.21.15.157:0/3896144744 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518005260 con 0x7f45080073a0 2023-11-06T03:15:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 msgr2=0x7f450800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 0x7f450800dba0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f4518001e40 tx=0x7f451801c740 comp rx=0 tx=0).stop 2023-11-06T03:15:06.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- 172.21.15.157:0/3896144744 shutdown_connections 2023-11-06T03:15:06.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 --2- 172.21.15.157:0/3896144744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45080073a0 0x7f450800dba0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- 172.21.15.157:0/3896144744 >> 172.21.15.157:0/3896144744 conn(0x7f4520065930 msgr2=0x7f4520065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:06.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- 172.21.15.157:0/3896144744 shutdown_connections 2023-11-06T03:15:06.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- 172.21.15.157:0/3896144744 wait complete. 2023-11-06T03:15:06.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 Processor -- start 2023-11-06T03:15:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- start start 2023-11-06T03:15:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45080073a0 0x7f45201523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45200acd20 0x7f45200cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520159fd0 con 0x7f45200cd260 2023-11-06T03:15:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45200cdc30 con 0x7f45200acd20 2023-11-06T03:15:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f45263e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45200ccaa0 con 0x7f45080073a0 2023-11-06T03:15:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54210/0 (socket says 172.21.15.157:54210) 2023-11-06T03:15:06.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 -- 172.21.15.157:0/3310005429 learned_addr learned my addr 172.21.15.157:0/3310005429 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f451f7fe640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45200acd20 0x7f45200cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f451ffff640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45080073a0 0x7f45201523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 -- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45080073a0 msgr2=0x7f45201523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45080073a0 0x7f45201523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 -- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45200acd20 msgr2=0x7f45200cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45200acd20 0x7f45200cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.288+0000 7f4524982640 1 -- 172.21.15.157:0/3310005429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520152920 con 0x7f45200cd260 2023-11-06T03:15:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f4524982640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f451801cfe0 tx=0x7f4508007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518005430 con 0x7f45200cd260 2023-11-06T03:15:06.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518005590 con 0x7f45200cd260 2023-11-06T03:15:06.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45200cc200 con 0x7f45200cd260 2023-11-06T03:15:06.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45180012e0 con 0x7f45200cd260 2023-11-06T03:15:06.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45200cc7c0 con 0x7f45200cd260 2023-11-06T03:15:06.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4520072750 con 0x7f45200cd260 2023-11-06T03:15:06.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45180ba460 con 0x7f45200cd260 2023-11-06T03:15:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 0x7f450c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f451811c9d0 con 0x7f45200cd260 2023-11-06T03:15:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451ffff640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 0x7f450c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.292+0000 7f451ffff640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 0x7f450c067f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f45140017b0 tx=0x7f4514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.296+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45180ea2c0 con 0x7f45200cd260 2023-11-06T03:15:06.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.484+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45200adcd0 con 0x7f45200cd260 2023-11-06T03:15:06.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.484+0000 7f451cff9640 1 -- 172.21.15.157:0/3310005429 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f45180ed8e0 con 0x7f45200cd260 2023-11-06T03:15:06.482 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:06.483 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:06.483 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:06.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 msgr2=0x7f450c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 0x7f450c067f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f45140017b0 tx=0x7f4514009040 comp rx=0 tx=0).stop 2023-11-06T03:15:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 msgr2=0x7f45200cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f451801cfe0 tx=0x7f4508007a10 comp rx=0 tx=0).stop 2023-11-06T03:15:06.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 shutdown_connections 2023-11-06T03:15:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f450c065a60 0x7f450c067f20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45200cd260 0x7f45200cd6f0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45200acd20 0x7f45200cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 --2- 172.21.15.157:0/3310005429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45080073a0 0x7f45201523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 >> 172.21.15.157:0/3310005429 conn(0x7f4520065930 msgr2=0x7f45200afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 shutdown_connections 2023-11-06T03:15:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.488+0000 7f45263e5640 1 -- 172.21.15.157:0/3310005429 wait complete. 2023-11-06T03:15:06.500 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:15:06.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:15:06.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 Processor -- start 2023-11-06T03:15:06.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 -- start start 2023-11-06T03:15:06.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58ec15caa0 0x7f58ec0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58ec0d0280 0x7f58ec0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec0d2920 con 0x7f58ec15ce80 2023-11-06T03:15:06.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec0d18a0 con 0x7f58ec0d0280 2023-11-06T03:15:06.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58f1e43640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec0d1b60 con 0x7f58ec15caa0 2023-11-06T03:15:06.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58ebfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f58ec0d0280 0x7f58ec0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58eaffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f58ec15ce80 0x7f58ec0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51340/0 (socket says 172.21.15.157:51340) 2023-11-06T03:15:06.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58eaffd640 1 -- 172.21.15.157:0/3538017780 learned_addr learned my addr 172.21.15.157:0/3538017780 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:06.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.616+0000 7f58eb7fe640 1 --2- 172.21.15.157:0/3538017780 >> v2:172.21.15.195:3300/0 conn(0x7f58ec15caa0 0x7f58ec0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ebfff640 1 -- 172.21.15.157:0/3538017780 >> v2:172.21.15.195:3300/0 conn(0x7f58ec15caa0 msgr2=0x7f58ec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:06.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ebfff640 1 --2- 172.21.15.157:0/3538017780 >> v2:172.21.15.195:3300/0 conn(0x7f58ec15caa0 0x7f58ec0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ebfff640 1 -- 172.21.15.157:0/3538017780 >> v1:172.21.15.157:6789/0 conn(0x7f58ec15ce80 legacy=0x7f58ec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ebfff640 1 -- 172.21.15.157:0/3538017780 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0d1e40 con 0x7f58ec0d0280 2023-11-06T03:15:06.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ebfff640 1 --2- 172.21.15.157:0/3538017780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec0d0280 0x7f58ec0b5d50 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0b4f80 tx=0x7f58e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3334710ca302caef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ea7fc640 1 -- 172.21.15.157:0/3538017780 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40a59a0 con 0x7f58ec0d0280 2023-11-06T03:15:06.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ea7fc640 1 -- 172.21.15.157:0/3538017780 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40a5b00 con 0x7f58ec0d0280 2023-11-06T03:15:06.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58ea7fc640 1 -- 172.21.15.157:0/3538017780 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e40a5e90 con 0x7f58ec0d0280 2023-11-06T03:15:06.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 -- 172.21.15.157:0/3538017780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec0d0280 msgr2=0x7f58ec0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 --2- 172.21.15.157:0/3538017780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec0d0280 0x7f58ec0b5d50 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0b4f80 tx=0x7f58e401c8e0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 -- 172.21.15.157:0/3538017780 shutdown_connections 2023-11-06T03:15:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 --2- 172.21.15.157:0/3538017780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec0d0280 0x7f58ec0b5d50 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 --2- 172.21.15.157:0/3538017780 >> v2:172.21.15.195:3300/0 conn(0x7f58ec15caa0 0x7f58ec0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 -- 172.21.15.157:0/3538017780 >> 172.21.15.157:0/3538017780 conn(0x7f58ec070790 msgr2=0x7f58ec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 -- 172.21.15.157:0/3538017780 shutdown_connections 2023-11-06T03:15:06.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.620+0000 7f58f1e43640 1 -- 172.21.15.157:0/3538017780 wait complete. 2023-11-06T03:15:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 Processor -- start 2023-11-06T03:15:06.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- start start 2023-11-06T03:15:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ec0d0280 0x7f58ec1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ec15ce80 0x7f58ec0b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec152ca0 con 0x7f58ec15caa0 2023-11-06T03:15:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1526e0 con 0x7f58ec15ce80 2023-11-06T03:15:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1529c0 con 0x7f58ec0d0280 2023-11-06T03:15:06.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54234/0 (socket says 172.21.15.157:54234) 2023-11-06T03:15:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 -- 172.21.15.157:0/70014344 learned_addr learned my addr 172.21.15.157:0/70014344 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58ebfff640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ec15ce80 0x7f58ec0b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eb7fe640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ec0d0280 0x7f58ec1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 -- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ec0d0280 msgr2=0x7f58ec1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ec0d0280 0x7f58ec1553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 -- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ec15ce80 msgr2=0x7f58ec0b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ec15ce80 0x7f58ec0b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 -- 172.21.15.157:0/70014344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e4009a70 con 0x7f58ec15caa0 2023-11-06T03:15:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58eaffd640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c4d0 tx=0x7f58d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4005cf0 con 0x7f58ec15caa0 2023-11-06T03:15:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4005e50 con 0x7f58ec15caa0 2023-11-06T03:15:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4007270 con 0x7f58ec15caa0 2023-11-06T03:15:06.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec0c6860 con 0x7f58ec15caa0 2023-11-06T03:15:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1516d0 con 0x7f58ec15caa0 2023-11-06T03:15:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec072750 con 0x7f58ec15caa0 2023-11-06T03:15:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.624+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f58d40073d0 con 0x7f58ec15caa0 2023-11-06T03:15:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.628+0000 7f58e3fff640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 0x7f58cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:06.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.628+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58d4082780 con 0x7f58ec15caa0 2023-11-06T03:15:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.628+0000 7f58eb7fe640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 0x7f58cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.628+0000 7f58eb7fe640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 0x7f58cc067f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0c60b0 tx=0x7f58dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:06.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.632+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58d4050090 con 0x7f58ec15caa0 2023-11-06T03:15:06.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f58ec0ce5b0 con 0x7f58ec15caa0 2023-11-06T03:15:06.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58e3fff640 1 -- 172.21.15.157:0/70014344 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1447 (secure 0 0 0) 0x7f58d40536b0 con 0x7f58ec15caa0 2023-11-06T03:15:06.840 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:06.841 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1048,"data_bytes":1077451700,"bytes_used":2772144128,"bytes_avail":770321969152,"bytes_total":773094113280,"write_bytes_sec":65589338,"read_op_per_sec":0,"write_op_per_sec":64},"fsmap":{"epoch":99,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9375}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-11-06T03:14:25.321050+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:15:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 msgr2=0x7f58cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 0x7f58cc067f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0c60b0 tx=0x7f58dc008040 comp rx=0 tx=0).stop 2023-11-06T03:15:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 msgr2=0x7f58ec151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:06.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c4d0 tx=0x7f58d400c990 comp rx=0 tx=0).stop 2023-11-06T03:15:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 shutdown_connections 2023-11-06T03:15:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f58cc065a60 0x7f58cc067f20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58ec15ce80 0x7f58ec0b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58ec15caa0 0x7f58ec151190 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 --2- 172.21.15.157:0/70014344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58ec0d0280 0x7f58ec1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 >> 172.21.15.157:0/70014344 conn(0x7f58ec070790 msgr2=0x7f58ec0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 shutdown_connections 2023-11-06T03:15:06.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:06.844+0000 7f58f1e43640 1 -- 172.21.15.157:0/70014344 wait complete. 2023-11-06T03:15:06.859 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:15:07.860 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:07.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 Processor -- start 2023-11-06T03:15:07.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 -- start start 2023-11-06T03:15:07.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f978c1599c0 0x7f978c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:07.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 0x7f978c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:07.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c15e440 con 0x7f978c15dea0 2023-11-06T03:15:07.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c15adb0 con 0x7f978c1599c0 2023-11-06T03:15:07.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9790ea0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c06c240 con 0x7f978c06d000 2023-11-06T03:15:07.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978ad9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f978c15dea0 0x7f978c070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51356/0 (socket says 172.21.15.157:51356) 2023-11-06T03:15:08.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978ad9d640 1 -- 172.21.15.157:0/653180865 learned_addr learned my addr 172.21.15.157:0/653180865 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:08.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 --2- 172.21.15.157:0/653180865 >> v2:172.21.15.195:3300/0 conn(0x7f978c1599c0 0x7f978c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9789d9b640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 0x7f978c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617598268 0 0) 0x7f978c15e440 con 0x7f978c15dea0 2023-11-06T03:15:08.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9774003530 con 0x7f978c15dea0 2023-11-06T03:15:08.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408301185 0 0) 0x7f9774003530 con 0x7f978c15dea0 2023-11-06T03:15:08.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 msgr2=0x7f978c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 0x7f978c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 >> v2:172.21.15.195:3300/0 conn(0x7f978c1599c0 msgr2=0x7f978c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 --2- 172.21.15.157:0/653180865 >> v2:172.21.15.195:3300/0 conn(0x7f978c1599c0 0x7f978c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c06c520 con 0x7f978c15dea0 2023-11-06T03:15:08.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9784001a00 con 0x7f978c15dea0 2023-11-06T03:15:08.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 >> v1:172.21.15.157:6789/0 conn(0x7f978c15dea0 legacy=0x7f978c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f9774003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97740047d0 0x7f9774004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 0x7f9774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c15adb0 con 0x7f9774003980 2023-11-06T03:15:08.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c06c240 con 0x7f97740047d0 2023-11-06T03:15:08.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977400b990 con 0x7f978c06d000 2023-11-06T03:15:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f9774003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f9789d9b640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97740047d0 0x7f9774004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 -- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 msgr2=0x7f9774005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c06d000 0x7f9774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 -- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97740047d0 msgr2=0x7f9774004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97740047d0 0x7f9774004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.000+0000 7f978a59c640 1 -- 172.21.15.157:0/653180865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774003530 con 0x7f9774003980 2023-11-06T03:15:08.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f978a59c640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f9774003da0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f977c009ae0 tx=0x7f977c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f64efe71a277e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0a6460 con 0x7f9774003980 2023-11-06T03:15:08.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c0a65c0 con 0x7f9774003980 2023-11-06T03:15:08.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f978959a640 1 -- 172.21.15.157:0/653180865 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0a6970 con 0x7f9774003980 2023-11-06T03:15:08.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- 172.21.15.157:0/653180865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 msgr2=0x7f9774003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 --2- 172.21.15.157:0/653180865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f9774003da0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f977c009ae0 tx=0x7f977c01ca60 comp rx=0 tx=0).stop 2023-11-06T03:15:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f978ad9d640 1 -- 172.21.15.157:0/653180865 reap_dead start 2023-11-06T03:15:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- 172.21.15.157:0/653180865 shutdown_connections 2023-11-06T03:15:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- 172.21.15.157:0/653180865 >> 172.21.15.157:0/653180865 conn(0x7f978c065930 msgr2=0x7f978c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- 172.21.15.157:0/653180865 shutdown_connections 2023-11-06T03:15:08.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- 172.21.15.157:0/653180865 wait complete. 2023-11-06T03:15:08.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 Processor -- start 2023-11-06T03:15:08.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.004+0000 7f9790ea0640 1 -- start start 2023-11-06T03:15:08.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 0x7f978c1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 0x7f978c0cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0ce8f0 con 0x7f9774003980 2023-11-06T03:15:08.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0ce2e0 con 0x7f978c1527c0 2023-11-06T03:15:08.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c0cbd90 con 0x7f978c0cd610 2023-11-06T03:15:08.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54248/0 (socket says 172.21.15.157:54248) 2023-11-06T03:15:08.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 -- 172.21.15.157:0/894048058 learned_addr learned my addr 172.21.15.157:0/894048058 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:08.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9789d9b640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 0x7f978c1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978ad9d640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 0x7f978c0cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 -- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 msgr2=0x7f978c0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 0x7f978c0cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 -- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 msgr2=0x7f978c1f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 0x7f978c1f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 -- 172.21.15.157:0/894048058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0ce580 con 0x7f9774003980 2023-11-06T03:15:08.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9789d9b640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 0x7f978c1f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:08.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978ad9d640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 0x7f978c0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:08.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f978a59c640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f977c009ae0 tx=0x7f9774005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0a6db0 con 0x7f9774003980 2023-11-06T03:15:08.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c0bc450 con 0x7f9774003980 2023-11-06T03:15:08.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c1f3200 con 0x7f9774003980 2023-11-06T03:15:08.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0bc7e0 con 0x7f9774003980 2023-11-06T03:15:08.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c0cba10 con 0x7f9774003980 2023-11-06T03:15:08.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c072750 con 0x7f9774003980 2023-11-06T03:15:08.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f977c0a7440 con 0x7f9774003980 2023-11-06T03:15:08.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 0x7f976c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f977c11d010 con 0x7f9774003980 2023-11-06T03:15:08.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.008+0000 7f9789d9b640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 0x7f976c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.012+0000 7f9789d9b640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 0x7f976c067f50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f978c150d50 tx=0x7f9780008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.012+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f977c0ea870 con 0x7f9774003980 2023-11-06T03:15:08.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.200+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f978c0cb110 con 0x7f9774003980 2023-11-06T03:15:08.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.200+0000 7f977affd640 1 -- 172.21.15.157:0/894048058 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f977c0ede90 con 0x7f9774003980 2023-11-06T03:15:08.198 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:08.199 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:08.199 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:08.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 msgr2=0x7f976c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 0x7f976c067f50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f978c150d50 tx=0x7f9780008040 comp rx=0 tx=0).stop 2023-11-06T03:15:08.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 msgr2=0x7f978c152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f977c009ae0 tx=0x7f9774005800 comp rx=0 tx=0).stop 2023-11-06T03:15:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 shutdown_connections 2023-11-06T03:15:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f976c065a90 0x7f976c067f50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f978c0cd610 0x7f978c0cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f978c1527c0 0x7f978c1f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 --2- 172.21.15.157:0/894048058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9774003980 0x7f978c152c90 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 >> 172.21.15.157:0/894048058 conn(0x7f978c065930 msgr2=0x7f978c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:08.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 shutdown_connections 2023-11-06T03:15:08.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.204+0000 7f9790ea0640 1 -- 172.21.15.157:0/894048058 wait complete. 2023-11-06T03:15:08.217 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2023-11-06T03:15:08.218 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:15:08.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 Processor -- start 2023-11-06T03:15:08.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 -- start start 2023-11-06T03:15:08.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7a041599c0 0x7f7a040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f7a04159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0415e440 con 0x7f7a0406d000 2023-11-06T03:15:08.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0406c240 con 0x7f7a041599c0 2023-11-06T03:15:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a0a73b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0406c520 con 0x7f7a040b2af0 2023-11-06T03:15:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a037fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7a0406d000 0x7f7a040b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51372/0 (socket says 172.21.15.157:51372) 2023-11-06T03:15:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a037fe640 1 -- 172.21.15.157:0/1800229605 learned_addr learned my addr 172.21.15.157:0/1800229605 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a03fff640 1 --2- 172.21.15.157:0/1800229605 >> v2:172.21.15.195:3300/0 conn(0x7f7a041599c0 0x7f7a040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f7a04159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114461238 0 0) 0x7f7a0415e440 con 0x7f7a0406d000 2023-11-06T03:15:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec003530 con 0x7f7a0406d000 2023-11-06T03:15:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739048167 0 0) 0x7f79ec003530 con 0x7f7a0406d000 2023-11-06T03:15:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 msgr2=0x7f7a04159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f7a04159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 >> v2:172.21.15.195:3300/0 conn(0x7f7a041599c0 msgr2=0x7f7a040ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 --2- 172.21.15.157:0/1800229605 >> v2:172.21.15.195:3300/0 conn(0x7f7a041599c0 0x7f7a040ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0406c800 con 0x7f7a0406d000 2023-11-06T03:15:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f79f80016a0 con 0x7f7a0406d000 2023-11-06T03:15:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 >> v1:172.21.15.157:6789/0 conn(0x7f7a0406d000 legacy=0x7f7a040b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79ec003980 0x7f79ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f79ec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f79ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0406c240 con 0x7f79ec004d10 2023-11-06T03:15:08.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0406c520 con 0x7f79ec003980 2023-11-06T03:15:08.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.336+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec00b990 con 0x7f7a040b2af0 2023-11-06T03:15:08.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f79ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a037fe640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f79ec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a03fff640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79ec003980 0x7f79ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 -- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 msgr2=0x7f79ec0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040b2af0 0x7f79ec0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 -- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79ec003980 msgr2=0x7f79ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f79ec003980 0x7f79ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 -- 172.21.15.157:0/1800229605 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec003530 con 0x7f79ec004d10 2023-11-06T03:15:08.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f79ec005190 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009f90 tx=0x7f79fc01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c883d9cf70702e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0a5bf0 con 0x7f79ec004d10 2023-11-06T03:15:08.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc00f040 con 0x7f79ec004d10 2023-11-06T03:15:08.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a02ffd640 1 -- 172.21.15.157:0/1800229605 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0ad440 con 0x7f79ec004d10 2023-11-06T03:15:08.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 -- 172.21.15.157:0/1800229605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 msgr2=0x7f79ec005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/1800229605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f79ec005190 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009f90 tx=0x7f79fc01cbe0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a08cd8640 1 -- 172.21.15.157:0/1800229605 reap_dead start 2023-11-06T03:15:08.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 -- 172.21.15.157:0/1800229605 shutdown_connections 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 -- 172.21.15.157:0/1800229605 >> 172.21.15.157:0/1800229605 conn(0x7f7a04065930 msgr2=0x7f7a04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 -- 172.21.15.157:0/1800229605 shutdown_connections 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.340+0000 7f7a0a73b640 1 -- 172.21.15.157:0/1800229605 wait complete. 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 Processor -- start 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- start start 2023-11-06T03:15:08.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 0x7f7a040cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040cd330 0x7f7a040cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040cdcf0 con 0x7f79ec004d10 2023-11-06T03:15:08.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040cd100 con 0x7f7a040cba10 2023-11-06T03:15:08.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a041ee190 con 0x7f7a040cd330 2023-11-06T03:15:08.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54266/0 (socket says 172.21.15.157:54266) 2023-11-06T03:15:08.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 -- 172.21.15.157:0/284730333 learned_addr learned my addr 172.21.15.157:0/284730333 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:08.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a08cd8640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040cd330 0x7f7a040cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a037fe640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 0x7f7a040cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 -- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040cd330 msgr2=0x7f7a040cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040cd330 0x7f7a040cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 -- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 msgr2=0x7f7a040cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 0x7f7a040cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 -- 172.21.15.157:0/284730333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0414de10 con 0x7f79ec004d10 2023-11-06T03:15:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a037fe640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 0x7f7a040cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a03fff640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f79f4011d80 tx=0x7f79f400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4016070 con 0x7f79ec004d10 2023-11-06T03:15:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f400ee40 con 0x7f79ec004d10 2023-11-06T03:15:08.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a040cc580 con 0x7f79ec004d10 2023-11-06T03:15:08.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400f7e0 con 0x7f79ec004d10 2023-11-06T03:15:08.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a040ccb40 con 0x7f79ec004d10 2023-11-06T03:15:08.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a04072750 con 0x7f79ec004d10 2023-11-06T03:15:08.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.344+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f79f400f940 con 0x7f79ec004d10 2023-11-06T03:15:08.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.348+0000 7f7a00ff9640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 0x7f79f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.348+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79f4085d00 con 0x7f79ec004d10 2023-11-06T03:15:08.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.348+0000 7f7a037fe640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 0x7f79f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.348+0000 7f7a037fe640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 0x7f79f0067f20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001cf0 tx=0x7f79f8001c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.352+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f79f40535f0 con 0x7f79ec004d10 2023-11-06T03:15:08.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.548+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7a040cb110 con 0x7f79ec004d10 2023-11-06T03:15:08.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a00ff9640 1 -- 172.21.15.157:0/284730333 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1632 (secure 0 0 0) 0x7f79f4056c10 con 0x7f79ec004d10 2023-11-06T03:15:08.549 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:08.550 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":686,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1048,"data_bytes":1077451700,"bytes_used":2772144128,"bytes_avail":770321969152,"bytes_total":773094113280,"write_bytes_sec":39763843,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":99,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9375}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2023-11-06T03:15:07.335662+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:15:08.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 msgr2=0x7f79f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 0x7f79f0067f20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001cf0 tx=0x7f79f8001c80 comp rx=0 tx=0).stop 2023-11-06T03:15:08.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 msgr2=0x7f7a041edc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f79f4011d80 tx=0x7f79f400e730 comp rx=0 tx=0).stop 2023-11-06T03:15:08.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 shutdown_connections 2023-11-06T03:15:08.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f79f0065a60 0x7f79f0067f20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.552+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a040cd330 0x7f7a040cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.556+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a040cba10 0x7f7a040cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.556+0000 7f7a0a73b640 1 --2- 172.21.15.157:0/284730333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ec004d10 0x7f7a041edc50 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.556+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 >> 172.21.15.157:0/284730333 conn(0x7f7a04065930 msgr2=0x7f7a04070010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.556+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 shutdown_connections 2023-11-06T03:15:08.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.556+0000 7f7a0a73b640 1 -- 172.21.15.157:0/284730333 wait complete. 2023-11-06T03:15:08.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2023-11-06T03:15:08.567 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:15:08.568 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:15:08.581 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:15:08.581 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'b', 'a'} 2023-11-06T03:15:08.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-06T03:15:08.748 INFO:teuthology.orchestra.run.smithi157.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-06T03:15:08.748 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-06T03:15:08.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:08.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 Processor -- start 2023-11-06T03:15:08.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 -- start start 2023-11-06T03:15:08.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85341551e0 0x7f853406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f85340b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853415f580 con 0x7f853415ea60 2023-11-06T03:15:08.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853415efc0 con 0x7f85341551e0 2023-11-06T03:15:08.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f853a267640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853415f2a0 con 0x7f853415e5e0 2023-11-06T03:15:08.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85341551e0 0x7f853406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85341551e0 0x7f853406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35404/0 (socket says 172.21.15.157:35404) 2023-11-06T03:15:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 -- 172.21.15.157:0/3789091529 learned_addr learned my addr 172.21.15.157:0/3789091529 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f85337fe640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f85340b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 -- 172.21.15.157:0/3789091529 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 msgr2=0x7f85340b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f85340b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 -- 172.21.15.157:0/3789091529 >> v1:172.21.15.195:6789/0 conn(0x7f853415ea60 legacy=0x7f853415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:08.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 -- 172.21.15.157:0/3789091529 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853406f5a0 con 0x7f85341551e0 2023-11-06T03:15:08.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f85337fe640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f85340b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:08.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8533fff640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85341551e0 0x7f853406fda0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f8524009870 tx=0x7f852401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2b8dab42188655 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.876+0000 7f8532ffd640 1 -- 172.21.15.157:0/3789091529 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240ac640 con 0x7f85341551e0 2023-11-06T03:15:08.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8532ffd640 1 -- 172.21.15.157:0/3789091529 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85240ac7a0 con 0x7f85341551e0 2023-11-06T03:15:08.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8532ffd640 1 -- 172.21.15.157:0/3789091529 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240acb30 con 0x7f85341551e0 2023-11-06T03:15:08.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3789091529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85341551e0 msgr2=0x7f853406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85341551e0 0x7f853406fda0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f8524009870 tx=0x7f852401c440 comp rx=0 tx=0).stop 2023-11-06T03:15:08.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3789091529 shutdown_connections 2023-11-06T03:15:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f85340b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- 172.21.15.157:0/3789091529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85341551e0 0x7f853406fda0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3789091529 >> 172.21.15.157:0/3789091529 conn(0x7f8534065930 msgr2=0x7f8534065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3789091529 shutdown_connections 2023-11-06T03:15:08.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3789091529 wait complete. 2023-11-06T03:15:08.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 Processor -- start 2023-11-06T03:15:08.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- start start 2023-11-06T03:15:08.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f8534144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8534142cd0 0x7f853414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534145430 con 0x7f853415ea60 2023-11-06T03:15:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340ce020 con 0x7f8534142cd0 2023-11-06T03:15:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340ce300 con 0x7f853415e5e0 2023-11-06T03:15:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35418/0 (socket says 172.21.15.157:35418) 2023-11-06T03:15:08.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 -- 172.21.15.157:0/3754748354 learned_addr learned my addr 172.21.15.157:0/3754748354 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:08.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8533fff640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f8534144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 -- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 msgr2=0x7f8534144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f8534144ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8538804640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8534142cd0 0x7f853414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 -- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8534142cd0 msgr2=0x7f853414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:08.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8534142cd0 0x7f853414ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:08.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 -- 172.21.15.157:0/3754748354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240094d0 con 0x7f853415ea60 2023-11-06T03:15:08.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8533fff640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f8534144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:08.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f85337fe640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f852800c7c0 tx=0x7f852800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528007a30 con 0x7f853415ea60 2023-11-06T03:15:08.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528007b90 con 0x7f853415ea60 2023-11-06T03:15:08.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800ce70 con 0x7f853415ea60 2023-11-06T03:15:08.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340ce5e0 con 0x7f853415ea60 2023-11-06T03:15:08.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.880+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534152870 con 0x7f853415ea60 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534072750 con 0x7f853415ea60 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f85280113f0 con 0x7f853415ea60 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f8530ff9640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 0x7f8520067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8528083730 con 0x7f853415ea60 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f8533fff640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 0x7f8520067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:08.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.884+0000 7f8533fff640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 0x7f8520067f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f85240ae000 tx=0x7f8524008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:08.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:08.888+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8528051020 con 0x7f853415ea60 2023-11-06T03:15:09.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.076+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85340aeb30 con 0x7f853415ea60 2023-11-06T03:15:09.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.076+0000 7f8530ff9640 1 -- 172.21.15.157:0/3754748354 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f8528054640 con 0x7f853415ea60 2023-11-06T03:15:09.075 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:09.075 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:09.076 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:09.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 msgr2=0x7f8520067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:09.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 0x7f8520067f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f85240ae000 tx=0x7f8524008040 comp rx=0 tx=0).stop 2023-11-06T03:15:09.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 msgr2=0x7f8534152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:09.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f852800c7c0 tx=0x7f852800cc80 comp rx=0 tx=0).stop 2023-11-06T03:15:09.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 shutdown_connections 2023-11-06T03:15:09.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8520065a60 0x7f8520067f20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:09.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8534142cd0 0x7f853414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:09.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f853415ea60 0x7f8534152d90 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:09.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 --2- 172.21.15.157:0/3754748354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f853415e5e0 0x7f8534144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:09.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 >> 172.21.15.157:0/3754748354 conn(0x7f8534065930 msgr2=0x7f85340b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:09.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 shutdown_connections 2023-11-06T03:15:09.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:09.080+0000 7f853a267640 1 -- 172.21.15.157:0/3754748354 wait complete. 2023-11-06T03:15:09.093 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-06T03:15:11.964 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:15:11.967 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:15:14.095 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:14.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.228+0000 7fd2a51e9640 1 Processor -- start 2023-11-06T03:15:14.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.228+0000 7fd2a51e9640 1 -- start start 2023-11-06T03:15:14.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd2a51e9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2a015e5e0 0x7fd2a00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd2a51e9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2a015ea90 0x7fd2a00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd2a51e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a015f580 con 0x7fd2a01551e0 2023-11-06T03:15:14.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd2a51e9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a00704b0 con 0x7fd2a015ea90 2023-11-06T03:15:14.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd2a51e9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a015efc0 con 0x7fd2a015e5e0 2023-11-06T03:15:14.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd2a01551e0 0x7fd2a006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:35384/0 (socket says 172.21.15.157:35384) 2023-11-06T03:15:14.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd2a015ea90 0x7fd2a00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29ed9d640 1 -- 172.21.15.157:0/3912260520 learned_addr learned my addr 172.21.15.157:0/3912260520 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:14.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29e59c640 1 --2- 172.21.15.157:0/3912260520 >> v2:172.21.15.195:3300/0 conn(0x7fd2a015e5e0 0x7fd2a00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 -- 172.21.15.157:0/3912260520 >> v2:172.21.15.195:3300/0 conn(0x7fd2a015e5e0 msgr2=0x7fd2a00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:14.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 --2- 172.21.15.157:0/3912260520 >> v2:172.21.15.195:3300/0 conn(0x7fd2a015e5e0 0x7fd2a00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 -- 172.21.15.157:0/3912260520 >> v1:172.21.15.157:6789/0 conn(0x7fd2a01551e0 legacy=0x7fd2a006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 -- 172.21.15.157:0/3912260520 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a015f2a0 con 0x7fd2a015ea90 2023-11-06T03:15:14.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29dd9b640 1 -- 172.21.15.157:0/3912260520 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890302416 0 0) 0x7fd2a015f580 con 0x7fd2a01551e0 2023-11-06T03:15:14.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29f59e640 1 --2- 172.21.15.157:0/3912260520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a015ea90 0x7fd2a00b3950 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0066880 tx=0x7fd29801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c10ba192c25fbe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29dd9b640 1 -- 172.21.15.157:0/3912260520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980a5910 con 0x7fd2a015ea90 2023-11-06T03:15:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29dd9b640 1 -- 172.21.15.157:0/3912260520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2980a5a70 con 0x7fd2a015ea90 2023-11-06T03:15:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.232+0000 7fd29dd9b640 1 -- 172.21.15.157:0/3912260520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980a5e00 con 0x7fd2a015ea90 2023-11-06T03:15:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/3912260520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a015ea90 msgr2=0x7fd2a00b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/3912260520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a015ea90 0x7fd2a00b3950 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0066880 tx=0x7fd29801c850 comp rx=0 tx=0).stop 2023-11-06T03:15:14.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/3912260520 shutdown_connections 2023-11-06T03:15:14.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/3912260520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a015ea90 0x7fd2a00b3950 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/3912260520 >> v2:172.21.15.195:3300/0 conn(0x7fd2a015e5e0 0x7fd2a00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/3912260520 >> 172.21.15.157:0/3912260520 conn(0x7fd2a0065930 msgr2=0x7fd2a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:14.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/3912260520 shutdown_connections 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/3912260520 wait complete. 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 Processor -- start 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- start start 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 0x7fd2a014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a015ea90 0x7fd2a015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a014e4f0 con 0x7fd2a01551e0 2023-11-06T03:15:14.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a0158430 con 0x7fd2a015e5e0 2023-11-06T03:15:14.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01578b0 con 0x7fd2a015ea90 2023-11-06T03:15:14.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35424/0 (socket says 172.21.15.157:35424) 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 -- 172.21.15.157:0/2988069750 learned_addr learned my addr 172.21.15.157:0/2988069750 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29f59e640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a015ea90 0x7fd2a015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29e59c640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 0x7fd2a014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 -- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a015ea90 msgr2=0x7fd2a015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a015ea90 0x7fd2a015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 -- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 msgr2=0x7fd2a014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 0x7fd2a014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 -- 172.21.15.157:0/2988069750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd298009a70 con 0x7fd2a01551e0 2023-11-06T03:15:14.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29e59c640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 0x7fd2a014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:14.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd29ed9d640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd2900011f0 tx=0x7fd29000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900074e0 con 0x7fd2a01551e0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290007640 con 0x7fd2a01551e0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a0157b90 con 0x7fd2a01551e0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290004a10 con 0x7fd2a01551e0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.236+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a0158150 con 0x7fd2a01551e0 2023-11-06T03:15:14.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a0072750 con 0x7fd2a01551e0 2023-11-06T03:15:14.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd29001b360 con 0x7fd2a01551e0 2023-11-06T03:15:14.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd2877fe640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 0x7fd28c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:14.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd2900828d0 con 0x7fd2a01551e0 2023-11-06T03:15:14.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd29e59c640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 0x7fd28c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:14.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.240+0000 7fd29e59c640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 0x7fd28c067f20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd2a01f6af0 tx=0x7fd294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:14.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.244+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd2900502a0 con 0x7fd2a01551e0 2023-11-06T03:15:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2a0071730 con 0x7fd2a01551e0 2023-11-06T03:15:14.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2877fe640 1 -- 172.21.15.157:0/2988069750 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fd290018020 con 0x7fd2a01551e0 2023-11-06T03:15:14.429 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:14.429 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:14.429 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:14.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 msgr2=0x7fd28c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 0x7fd28c067f20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd2a01f6af0 tx=0x7fd294008040 comp rx=0 tx=0).stop 2023-11-06T03:15:14.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 msgr2=0x7fd2a01572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd2900011f0 tx=0x7fd29000c9b0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 shutdown_connections 2023-11-06T03:15:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd28c065a60 0x7fd28c067f20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a015ea90 0x7fd2a015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2a015e5e0 0x7fd2a014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 --2- 172.21.15.157:0/2988069750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2a01551e0 0x7fd2a01572d0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:14.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 >> 172.21.15.157:0/2988069750 conn(0x7fd2a0065930 msgr2=0x7fd2a015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:14.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 shutdown_connections 2023-11-06T03:15:14.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:14.432+0000 7fd2a51e9640 1 -- 172.21.15.157:0/2988069750 wait complete. 2023-11-06T03:15:14.448 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-06T03:15:19.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:19.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 Processor -- start 2023-11-06T03:15:19.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 -- start start 2023-11-06T03:15:19.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efda00ca570 0x7efda00b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efda00b4ab0 0x7efda006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00b4f20 con 0x7efda015adc0 2023-11-06T03:15:19.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda006b8d0 con 0x7efda00b4ab0 2023-11-06T03:15:19.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efda4ca2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda006be90 con 0x7efda00ca570 2023-11-06T03:15:19.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efda00b4ab0 0x7efda006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efda00b4ab0 0x7efda006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44622/0 (socket says 172.21.15.157:44622) 2023-11-06T03:15:19.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 -- 172.21.15.157:0/2231432816 learned_addr learned my addr 172.21.15.157:0/2231432816 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:19.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9dd9b640 1 --2- 172.21.15.157:0/2231432816 >> v2:172.21.15.195:3300/0 conn(0x7efda00ca570 0x7efda00b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 -- 172.21.15.157:0/2231432816 >> v2:172.21.15.195:3300/0 conn(0x7efda00ca570 msgr2=0x7efda00b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:19.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 --2- 172.21.15.157:0/2231432816 >> v2:172.21.15.195:3300/0 conn(0x7efda00ca570 0x7efda00b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 -- 172.21.15.157:0/2231432816 >> v1:172.21.15.195:6789/0 conn(0x7efda015adc0 legacy=0x7efda0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:19.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.592+0000 7efd9ed9d640 1 -- 172.21.15.157:0/2231432816 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0071880 con 0x7efda00b4ab0 2023-11-06T03:15:19.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efd9ed9d640 1 --2- 172.21.15.157:0/2231432816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00b4ab0 0x7efda006c320 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719ffca9b6c4f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efd9d59a640 1 -- 172.21.15.157:0/2231432816 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980a5700 con 0x7efda00b4ab0 2023-11-06T03:15:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efd9d59a640 1 -- 172.21.15.157:0/2231432816 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980a5860 con 0x7efda00b4ab0 2023-11-06T03:15:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efd9d59a640 1 -- 172.21.15.157:0/2231432816 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980a5bf0 con 0x7efda00b4ab0 2023-11-06T03:15:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- 172.21.15.157:0/2231432816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00b4ab0 msgr2=0x7efda006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:19.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 --2- 172.21.15.157:0/2231432816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00b4ab0 0x7efda006c320 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9801c840 comp rx=0 tx=0).stop 2023-11-06T03:15:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- 172.21.15.157:0/2231432816 shutdown_connections 2023-11-06T03:15:19.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 --2- 172.21.15.157:0/2231432816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda00b4ab0 0x7efda006c320 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 --2- 172.21.15.157:0/2231432816 >> v2:172.21.15.195:3300/0 conn(0x7efda00ca570 0x7efda00b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- 172.21.15.157:0/2231432816 >> 172.21.15.157:0/2231432816 conn(0x7efda00cb990 msgr2=0x7efda00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:19.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- 172.21.15.157:0/2231432816 shutdown_connections 2023-11-06T03:15:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- 172.21.15.157:0/2231432816 wait complete. 2023-11-06T03:15:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 Processor -- start 2023-11-06T03:15:19.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.596+0000 7efda4ca2640 1 -- start start 2023-11-06T03:15:19.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda00b4ab0 0x7efda0146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 0x7efda015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0146cc0 con 0x7efda015adc0 2023-11-06T03:15:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda015d0d0 con 0x7efda00b4ab0 2023-11-06T03:15:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda014f4b0 con 0x7efda00ca570 2023-11-06T03:15:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44632/0 (socket says 172.21.15.157:44632) 2023-11-06T03:15:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 -- 172.21.15.157:0/1338815036 learned_addr learned my addr 172.21.15.157:0/1338815036 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9e59c640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda00b4ab0 0x7efda0146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9dd9b640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 0x7efda015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 -- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 msgr2=0x7efda015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:19.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 0x7efda015c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 -- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda00b4ab0 msgr2=0x7efda0146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda00b4ab0 0x7efda0146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 -- 172.21.15.157:0/1338815036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009b30 con 0x7efda015adc0 2023-11-06T03:15:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9dd9b640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 0x7efda015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:19.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9ed9d640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7efd980af040 tx=0x7efd980195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980b8670 con 0x7efda015adc0 2023-11-06T03:15:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980b87d0 con 0x7efda015adc0 2023-11-06T03:15:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980b8b60 con 0x7efda015adc0 2023-11-06T03:15:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda014f6c0 con 0x7efda015adc0 2023-11-06T03:15:19.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda014fc80 con 0x7efda015adc0 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda0072750 con 0x7efda015adc0 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd980b8cc0 con 0x7efda015adc0 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 0x7efd80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd9811e920 con 0x7efda015adc0 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.600+0000 7efd9e59c640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 0x7efd80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:19.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.604+0000 7efd9e59c640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 0x7efd80067f20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efd90004e30 tx=0x7efd90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:19.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.604+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efd980ec070 con 0x7efda015adc0 2023-11-06T03:15:19.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.792+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda01575b0 con 0x7efda015adc0 2023-11-06T03:15:19.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efd8effd640 1 -- 172.21.15.157:0/1338815036 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4659 (secure 0 0 0) 0x7efd980ef690 con 0x7efda015adc0 2023-11-06T03:15:19.792 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:19.793 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":99,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:14:38.023366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1004239715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1004239715},{"type":"v1","addr":"172.21.15.157:6835","nonce":1004239715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 99 2023-11-06T03:15:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 msgr2=0x7efd80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 0x7efd80067f20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efd90004e30 tx=0x7efd90009040 comp rx=0 tx=0).stop 2023-11-06T03:15:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 msgr2=0x7efda015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:19.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7efd980af040 tx=0x7efd980195e0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 shutdown_connections 2023-11-06T03:15:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd80065a60 0x7efd80067f20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efda015adc0 0x7efda015c5f0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efda00ca570 0x7efda015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 --2- 172.21.15.157:0/1338815036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda00b4ab0 0x7efda0146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 >> 172.21.15.157:0/1338815036 conn(0x7efda00cb990 msgr2=0x7efda015b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 shutdown_connections 2023-11-06T03:15:19.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:19.796+0000 7efda4ca2640 1 -- 172.21.15.157:0/1338815036 wait complete. 2023-11-06T03:15:19.812 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-06T03:15:24.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 Processor -- start 2023-11-06T03:15:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 -- start start 2023-11-06T03:15:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 0x7f29a00b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a015e440 con 0x7f29a015dea0 2023-11-06T03:15:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a015adb0 con 0x7f29a006d000 2023-11-06T03:15:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a69a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a006c240 con 0x7f29a01599c0 2023-11-06T03:15:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f2997fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 0x7f29a00b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a4f43640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f29a015dea0 0x7f29a0070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:35946/0 (socket says 172.21.15.157:35946) 2023-11-06T03:15:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f29a4f43640 1 -- 172.21.15.157:0/2746933288 learned_addr learned my addr 172.21.15.157:0/2746933288 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43110/0 (socket says 172.21.15.157:43110) 2023-11-06T03:15:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 -- 172.21.15.157:0/2746933288 >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 msgr2=0x7f29a00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 --2- 172.21.15.157:0/2746933288 >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 0x7f29a00b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 -- 172.21.15.157:0/2746933288 >> v1:172.21.15.195:6789/0 conn(0x7f29a015dea0 legacy=0x7f29a0070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f299ffff640 1 -- 172.21.15.157:0/2746933288 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a006c520 con 0x7f29a01599c0 2023-11-06T03:15:24.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.948+0000 7f2997fff640 1 --2- 172.21.15.157:0/2746933288 >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 0x7f29a00b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:15:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f299ffff640 1 --2- 172.21.15.157:0/2746933288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29900097c0 tx=0x7f299001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afac52e5763e283a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f299f7fe640 1 -- 172.21.15.157:0/2746933288 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900ad660 con 0x7f29a01599c0 2023-11-06T03:15:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f299f7fe640 1 -- 172.21.15.157:0/2746933288 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29900ad7c0 con 0x7f29a01599c0 2023-11-06T03:15:24.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f299f7fe640 1 -- 172.21.15.157:0/2746933288 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900adb50 con 0x7f29a01599c0 2023-11-06T03:15:24.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- 172.21.15.157:0/2746933288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 msgr2=0x7f29a00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 --2- 172.21.15.157:0/2746933288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29900097c0 tx=0x7f299001c450 comp rx=0 tx=0).stop 2023-11-06T03:15:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- 172.21.15.157:0/2746933288 shutdown_connections 2023-11-06T03:15:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 --2- 172.21.15.157:0/2746933288 >> v2:172.21.15.195:3300/0 conn(0x7f29a006d000 0x7f29a00b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 --2- 172.21.15.157:0/2746933288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a01599c0 0x7f29a00ae000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- 172.21.15.157:0/2746933288 >> 172.21.15.157:0/2746933288 conn(0x7f29a0065930 msgr2=0x7f29a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- 172.21.15.157:0/2746933288 shutdown_connections 2023-11-06T03:15:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- 172.21.15.157:0/2746933288 wait complete. 2023-11-06T03:15:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 Processor -- start 2023-11-06T03:15:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.952+0000 7f29a69a6640 1 -- start start 2023-11-06T03:15:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a006d000 0x7f29a00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 0x7f29a014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a014e7f0 con 0x7f29a015dea0 2023-11-06T03:15:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0070570 con 0x7f29a01599c0 2023-11-06T03:15:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00cecd0 con 0x7f29a006d000 2023-11-06T03:15:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44646/0 (socket says 172.21.15.157:44646) 2023-11-06T03:15:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 -- 172.21.15.157:0/3422809916 learned_addr learned my addr 172.21.15.157:0/3422809916 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f2997fff640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 0x7f29a014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299ffff640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a006d000 0x7f29a00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 -- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a006d000 msgr2=0x7f29a00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a006d000 0x7f29a00ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:24.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 -- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 msgr2=0x7f29a014d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 0x7f29a014d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 -- 172.21.15.157:0/3422809916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009420 con 0x7f29a015dea0 2023-11-06T03:15:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f2997fff640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 0x7f29a014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a4f43640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f299800ca80 tx=0x7f299800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:24.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800ac30 con 0x7f29a015dea0 2023-11-06T03:15:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299800ad90 con 0x7f29a015dea0 2023-11-06T03:15:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998012600 con 0x7f29a015dea0 2023-11-06T03:15:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990009420 con 0x7f29a015dea0 2023-11-06T03:15:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a006ffb0 con 0x7f29a015dea0 2023-11-06T03:15:24.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0072750 con 0x7f29a015dea0 2023-11-06T03:15:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2998007070 con 0x7f29a015dea0 2023-11-06T03:15:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 0x7f2980067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f2998083090 con 0x7f29a015dea0 2023-11-06T03:15:24.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.956+0000 7f299ffff640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 0x7f2980067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:24.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.960+0000 7f299ffff640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 0x7f2980067f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f29900af000 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:24.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:24.960+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f29980509d0 con 0x7f29a015dea0 2023-11-06T03:15:25.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.148+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a00adcd0 con 0x7f29a015dea0 2023-11-06T03:15:25.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.148+0000 7f299d7fa640 1 -- 172.21.15.157:0/3422809916 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+3905 (secure 0 0 0) 0x7f2998017090 con 0x7f29a015dea0 2023-11-06T03:15:25.148 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:25.148 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":102,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:24.605667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:25.149 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 102 2023-11-06T03:15:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 msgr2=0x7f2980067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 0x7f2980067f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f29900af000 tx=0x7f2990008040 comp rx=0 tx=0).stop 2023-11-06T03:15:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 msgr2=0x7f29a015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:25.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f299800ca80 tx=0x7f299800cf40 comp rx=0 tx=0).stop 2023-11-06T03:15:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 shutdown_connections 2023-11-06T03:15:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2980065a60 0x7f2980067f20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29a015dea0 0x7f29a015c160 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:25.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29a01599c0 0x7f29a014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 --2- 172.21.15.157:0/3422809916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f29a006d000 0x7f29a00ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 >> 172.21.15.157:0/3422809916 conn(0x7f29a0065930 msgr2=0x7f29a0158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:25.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 shutdown_connections 2023-11-06T03:15:25.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:25.152+0000 7f29a69a6640 1 -- 172.21.15.157:0/3422809916 wait complete. 2023-11-06T03:15:25.168 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-06T03:15:26.612 INFO:tasks.ceph.mds.b.smithi195.stderr:2023-11-06T03:15:26.617+0000 7f9809721640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:15:30.169 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 Processor -- start 2023-11-06T03:15:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 -- start start 2023-11-06T03:15:30.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa28c15ea60 0x7fa28c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c15f580 con 0x7fa28c1551e0 2023-11-06T03:15:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c0704b0 con 0x7fa28c15ea60 2023-11-06T03:15:30.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa290ecf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c15efc0 con 0x7fa28c15e5e0 2023-11-06T03:15:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa28ad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa28c15ea60 0x7fa28c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa289d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa28a59c640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa28c1551e0 0x7fa28c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:37508/0 (socket says 172.21.15.157:37508) 2023-11-06T03:15:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa28ad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa28c15ea60 0x7fa28c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:46508/0 (socket says 172.21.15.157:46508) 2023-11-06T03:15:30.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.304+0000 7fa28a59c640 1 -- 172.21.15.157:0/2849575451 learned_addr learned my addr 172.21.15.157:0/2849575451 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 msgr2=0x7fa28c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:30.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28ad9d640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2849575451 >> v1:172.21.15.195:6789/0 conn(0x7fa28c1551e0 legacy=0x7fa28c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2849575451 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c15f2a0 con 0x7fa28c15ea60 2023-11-06T03:15:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa289d9b640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:15:30.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28ad9d640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa284009d90 tx=0x7fa28401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308dc6f795ccdf25 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28959a640 1 -- 172.21.15.157:0/2849575451 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840a5950 con 0x7fa28c15ea60 2023-11-06T03:15:30.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28959a640 1 -- 172.21.15.157:0/2849575451 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840a5ab0 con 0x7fa28c15ea60 2023-11-06T03:15:30.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa28959a640 1 -- 172.21.15.157:0/2849575451 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840a5e40 con 0x7fa28c15ea60 2023-11-06T03:15:30.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 -- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c15ea60 msgr2=0x7fa28c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3950 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa284009d90 tx=0x7fa28401c890 comp rx=0 tx=0).stop 2023-11-06T03:15:30.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 -- 172.21.15.157:0/2849575451 shutdown_connections 2023-11-06T03:15:30.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3950 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2849575451 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 -- 172.21.15.157:0/2849575451 >> 172.21.15.157:0/2849575451 conn(0x7fa28c065930 msgr2=0x7fa28c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:30.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 -- 172.21.15.157:0/2849575451 shutdown_connections 2023-11-06T03:15:30.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.308+0000 7fa290ecf640 1 -- 172.21.15.157:0/2849575451 wait complete. 2023-11-06T03:15:30.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 Processor -- start 2023-11-06T03:15:30.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- start start 2023-11-06T03:15:30.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c1551e0 0x7fa28c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c15a470 con 0x7fa28c15ea60 2023-11-06T03:15:30.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c06c8b0 con 0x7fa28c1551e0 2023-11-06T03:15:30.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c159f10 con 0x7fa28c15e5e0 2023-11-06T03:15:30.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52802/0 (socket says 172.21.15.157:52802) 2023-11-06T03:15:30.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2611037113 learned_addr learned my addr 172.21.15.157:0/2611037113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:30.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa289d9b640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28a59c640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c1551e0 0x7fa28c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 msgr2=0x7fa28c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c1551e0 msgr2=0x7fa28c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c1551e0 0x7fa28c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 -- 172.21.15.157:0/2611037113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274009420 con 0x7fa28c15ea60 2023-11-06T03:15:30.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa289d9b640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:30.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa28ad9d640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa28401a4c0 tx=0x7fa284019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284007000 con 0x7fa28c15ea60 2023-11-06T03:15:30.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840aede0 con 0x7fa28c15ea60 2023-11-06T03:15:30.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa284009a70 con 0x7fa28c15ea60 2023-11-06T03:15:30.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840b7650 con 0x7fa28c15ea60 2023-11-06T03:15:30.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c06c260 con 0x7fa28c15ea60 2023-11-06T03:15:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.312+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c06c0c0 con 0x7fa28c15ea60 2023-11-06T03:15:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.316+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa2840b4050 con 0x7fa28c15ea60 2023-11-06T03:15:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.316+0000 7fa282ffd640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 0x7fa26c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.316+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa28411e730 con 0x7fa28c15ea60 2023-11-06T03:15:30.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.316+0000 7fa28a59c640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 0x7fa26c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.316+0000 7fa28a59c640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 0x7fa26c067f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000f70 tx=0x7fa27c007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.320+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa2840ec0c0 con 0x7fa28c15ea60 2023-11-06T03:15:30.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa28c06c0c0 con 0x7fa28c15ea60 2023-11-06T03:15:30.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa282ffd640 1 -- 172.21.15.157:0/2611037113 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7fa28c06c0c0 con 0x7fa28c15ea60 2023-11-06T03:15:30.504 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:30.505 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:30.505 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:30.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 msgr2=0x7fa26c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 0x7fa26c067f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000f70 tx=0x7fa27c007e90 comp rx=0 tx=0).stop 2023-11-06T03:15:30.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 msgr2=0x7fa28c0b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa28401a4c0 tx=0x7fa284019630 comp rx=0 tx=0).stop 2023-11-06T03:15:30.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 shutdown_connections 2023-11-06T03:15:30.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa26c065a60 0x7fa26c067f20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa28c15ea60 0x7fa28c0b3400 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa28c15e5e0 0x7fa28c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 --2- 172.21.15.157:0/2611037113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa28c1551e0 0x7fa28c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 >> 172.21.15.157:0/2611037113 conn(0x7fa28c065930 msgr2=0x7fa28c15de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:30.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 shutdown_connections 2023-11-06T03:15:30.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.508+0000 7fa290ecf640 1 -- 172.21.15.157:0/2611037113 wait complete. 2023-11-06T03:15:30.524 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-06T03:15:30.524 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:30.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 Processor -- start 2023-11-06T03:15:30.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 -- start start 2023-11-06T03:15:30.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f78500cefc0 0x7f78500cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78500708b0 0x7f785015d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78500cf8e0 con 0x7f7850070420 2023-11-06T03:15:30.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785015e220 con 0x7f78500cefc0 2023-11-06T03:15:30.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f78550f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785015dc60 con 0x7f78500708b0 2023-11-06T03:15:30.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7850070420 0x7f785015cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33212/0 (socket says 172.21.15.157:33212) 2023-11-06T03:15:30.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784e59c640 1 -- 172.21.15.157:0/1885952689 learned_addr learned my addr 172.21.15.157:0/1885952689 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:30.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1885952689 >> v2:172.21.15.157:3300/0 conn(0x7f78500cefc0 0x7f78500cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784f59e640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78500708b0 0x7f785015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784dd9b640 1 -- 172.21.15.157:0/1885952689 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256893548 0 0) 0x7f78500cf8e0 con 0x7f7850070420 2023-11-06T03:15:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784dd9b640 1 -- 172.21.15.157:0/1885952689 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7838003530 con 0x7f7850070420 2023-11-06T03:15:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784ed9d640 1 -- 172.21.15.157:0/1885952689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78500708b0 msgr2=0x7f785015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:30.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78500708b0 0x7f785015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784ed9d640 1 -- 172.21.15.157:0/1885952689 >> v1:172.21.15.157:6789/0 conn(0x7f7850070420 legacy=0x7f785015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.648+0000 7f784ed9d640 1 -- 172.21.15.157:0/1885952689 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785015df40 con 0x7f78500cefc0 2023-11-06T03:15:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500cefc0 0x7f78500cf3a0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f78400097a0 tx=0x7f784001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827c8cd1f4123a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784dd9b640 1 -- 172.21.15.157:0/1885952689 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ac600 con 0x7f78500cefc0 2023-11-06T03:15:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784dd9b640 1 -- 172.21.15.157:0/1885952689 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400a8050 con 0x7f78500cefc0 2023-11-06T03:15:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784dd9b640 1 -- 172.21.15.157:0/1885952689 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ac960 con 0x7f78500cefc0 2023-11-06T03:15:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- 172.21.15.157:0/1885952689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500cefc0 msgr2=0x7f78500cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500cefc0 0x7f78500cf3a0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f78400097a0 tx=0x7f784001c3a0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- 172.21.15.157:0/1885952689 shutdown_connections 2023-11-06T03:15:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78500708b0 0x7f785015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- 172.21.15.157:0/1885952689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500cefc0 0x7f78500cf3a0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- 172.21.15.157:0/1885952689 >> 172.21.15.157:0/1885952689 conn(0x7f785006a4d0 msgr2=0x7f785006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- 172.21.15.157:0/1885952689 shutdown_connections 2023-11-06T03:15:30.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- 172.21.15.157:0/1885952689 wait complete. 2023-11-06T03:15:30.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 Processor -- start 2023-11-06T03:15:30.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- start start 2023-11-06T03:15:30.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7850070420 0x7f785014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 0x7f785015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785014b150 con 0x7f78500708b0 2023-11-06T03:15:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785014b430 con 0x7f78500cefc0 2023-11-06T03:15:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f78550f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785014b710 con 0x7f7850070420 2023-11-06T03:15:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52830/0 (socket says 172.21.15.157:52830) 2023-11-06T03:15:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 -- 172.21.15.157:0/1418157571 learned_addr learned my addr 172.21.15.157:0/1418157571 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7850070420 0x7f785014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784f59e640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 0x7f785015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 -- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7850070420 msgr2=0x7f785014a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7850070420 0x7f785014a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 -- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 msgr2=0x7f785015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 0x7f785015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 -- 172.21.15.157:0/1418157571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400093b0 con 0x7f78500708b0 2023-11-06T03:15:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784f59e640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 0x7f785015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:30.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.652+0000 7f784e59c640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f784400cb40 tx=0x7f784400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400aca0 con 0x7f78500708b0 2023-11-06T03:15:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784400ae00 con 0x7f78500708b0 2023-11-06T03:15:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78500b1e00 con 0x7f78500708b0 2023-11-06T03:15:30.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844012620 con 0x7f78500708b0 2023-11-06T03:15:30.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785014bdf0 con 0x7f78500708b0 2023-11-06T03:15:30.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78500b1ff0 con 0x7f78500708b0 2023-11-06T03:15:30.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7844007070 con 0x7f78500708b0 2023-11-06T03:15:30.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 0x7f783c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:30.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f78440833a0 con 0x7f78500708b0 2023-11-06T03:15:30.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 0x7f783c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:30.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.656+0000 7f784ed9d640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 0x7f783c067f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f78400ae000 tx=0x7f7840008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:30.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.660+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7844050ba0 con 0x7f78500708b0 2023-11-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.884+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7850072750 con 0x7f78500708b0 2023-11-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.884+0000 7f78377fe640 1 -- 172.21.15.157:0/1418157571 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7f7844017090 con 0x7f78500708b0 2023-11-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 msgr2=0x7f783c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 0x7f783c067f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f78400ae000 tx=0x7f7840008040 comp rx=0 tx=0).stop 2023-11-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 msgr2=0x7f785014a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f784400cb40 tx=0x7f784400a060 comp rx=0 tx=0).stop 2023-11-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 shutdown_connections 2023-11-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f783c065a60 0x7f783c067f20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78500cefc0 0x7f785015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78500708b0 0x7f785014a700 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 --2- 172.21.15.157:0/1418157571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7850070420 0x7f785014a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 >> 172.21.15.157:0/1418157571 conn(0x7f785006a4d0 msgr2=0x7f78500ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 shutdown_connections 2023-11-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:30.888+0000 7f78550f6640 1 -- 172.21.15.157:0/1418157571 wait complete. 2023-11-06T03:15:30.902 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2023-11-06T03:15:30.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:15:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 Processor -- start 2023-11-06T03:15:31.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 -- start start 2023-11-06T03:15:31.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 0x7f327006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f32700b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f327015ea60 0x7f32700b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327015f580 con 0x7f327015ea60 2023-11-06T03:15:31.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327015efc0 con 0x7f32701551e0 2023-11-06T03:15:31.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f3277cdf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327015f2a0 con 0x7f327015e5e0 2023-11-06T03:15:31.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f327627c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f327015ea60 0x7f32700b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f327627c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f327015ea60 0x7f32700b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52836/0 (socket says 172.21.15.157:52836) 2023-11-06T03:15:31.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.016+0000 7f327627c640 1 -- 172.21.15.157:0/381167310 learned_addr learned my addr 172.21.15.157:0/381167310 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:31.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3275a7b640 1 --2- 172.21.15.157:0/381167310 >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 0x7f327006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327527a640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f32700b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 -- 172.21.15.157:0/381167310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 msgr2=0x7f32700b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:31.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f32700b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 -- 172.21.15.157:0/381167310 >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 msgr2=0x7f327006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:31.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 --2- 172.21.15.157:0/381167310 >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 0x7f327006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 -- 172.21.15.157:0/381167310 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327006f5a0 con 0x7f327015ea60 2023-11-06T03:15:31.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3275a7b640 1 --2- 172.21.15.157:0/381167310 >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 0x7f327006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:31.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f327627c640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700b3950 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3270066830 tx=0x7f326801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287b64ee5851a39e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3274a79640 1 -- 172.21.15.157:0/381167310 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680a57b0 con 0x7f327015ea60 2023-11-06T03:15:31.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3274a79640 1 -- 172.21.15.157:0/381167310 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32680a5910 con 0x7f327015ea60 2023-11-06T03:15:31.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3274a79640 1 -- 172.21.15.157:0/381167310 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680a5ca0 con 0x7f327015ea60 2023-11-06T03:15:31.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 -- 172.21.15.157:0/381167310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 msgr2=0x7f32700b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700b3950 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3270066830 tx=0x7f326801c910 comp rx=0 tx=0).stop 2023-11-06T03:15:31.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 -- 172.21.15.157:0/381167310 shutdown_connections 2023-11-06T03:15:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700b3950 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 --2- 172.21.15.157:0/381167310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f32700b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 --2- 172.21.15.157:0/381167310 >> v2:172.21.15.195:3300/0 conn(0x7f32701551e0 0x7f327006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 -- 172.21.15.157:0/381167310 >> 172.21.15.157:0/381167310 conn(0x7f3270065930 msgr2=0x7f3270065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:31.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 -- 172.21.15.157:0/381167310 shutdown_connections 2023-11-06T03:15:31.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.020+0000 7f3277cdf640 1 -- 172.21.15.157:0/381167310 wait complete. 2023-11-06T03:15:31.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 Processor -- start 2023-11-06T03:15:31.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- start start 2023-11-06T03:15:31.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32701551e0 0x7f327015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f3270138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701548f0 con 0x7f327015ea60 2023-11-06T03:15:31.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701f5dd0 con 0x7f32701551e0 2023-11-06T03:15:31.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32701f60b0 con 0x7f327015e5e0 2023-11-06T03:15:31.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52838/0 (socket says 172.21.15.157:52838) 2023-11-06T03:15:31.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 -- 172.21.15.157:0/419317832 learned_addr learned my addr 172.21.15.157:0/419317832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:31.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3275a7b640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32701551e0 0x7f327015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327527a640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f3270138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 -- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 msgr2=0x7f3270138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f3270138e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 -- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32701551e0 msgr2=0x7f327015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32701551e0 0x7f327015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 -- 172.21.15.157:0/419317832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009b30 con 0x7f327015ea60 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f327627c640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f326801afc0 tx=0x7f32680196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680b8670 con 0x7f327015ea60 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32680b87d0 con 0x7f327015ea60 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32701599d0 con 0x7f327015ea60 2023-11-06T03:15:31.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680b8b60 con 0x7f327015ea60 2023-11-06T03:15:31.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3270159f90 con 0x7f327015ea60 2023-11-06T03:15:31.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.024+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3270072750 con 0x7f327015ea60 2023-11-06T03:15:31.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.028+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f32680b8cc0 con 0x7f327015ea60 2023-11-06T03:15:31.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.028+0000 7f32667fc640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 0x7f3254067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.028+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f326811e8f0 con 0x7f327015ea60 2023-11-06T03:15:31.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.028+0000 7f3275a7b640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 0x7f3254067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.028+0000 7f3275a7b640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 0x7f3254067f20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3270151c00 tx=0x7f326c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.032+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f32680ec0f0 con 0x7f327015ea60 2023-11-06T03:15:31.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.244+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3270071730 con 0x7f327015ea60 2023-11-06T03:15:31.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.244+0000 7f32667fc640 1 -- 172.21.15.157:0/419317832 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1583 (secure 0 0 0) 0x7f32680bc640 con 0x7f327015ea60 2023-11-06T03:15:31.243 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:31.243 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_WARN","checks":{"MDS_TRIM":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs behind on trimming","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":138,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":1070,"data_bytes":1078527966,"bytes_used":2811285504,"bytes_avail":770282827776,"bytes_total":773094113280,"read_bytes_sec":392251,"write_bytes_sec":87519,"read_op_per_sec":7,"write_op_per_sec":155},"fsmap":{"epoch":104,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":6455}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2023-11-06T03:15:23.341572+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:15:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 msgr2=0x7f3254067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 0x7f3254067f20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3270151c00 tx=0x7f326c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 msgr2=0x7f32700cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f326801afc0 tx=0x7f32680196b0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 shutdown_connections 2023-11-06T03:15:31.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3254065a60 0x7f3254067f20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f327015ea60 0x7f32700cdba0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f327015e5e0 0x7f3270138e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 --2- 172.21.15.157:0/419317832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32701551e0 0x7f327015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 >> 172.21.15.157:0/419317832 conn(0x7f3270065930 msgr2=0x7f327015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:31.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 shutdown_connections 2023-11-06T03:15:31.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.248+0000 7f3277cdf640 1 -- 172.21.15.157:0/419317832 wait complete. 2023-11-06T03:15:31.262 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2023-11-06T03:15:31.262 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:15:31.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:15:31.313 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:15:31.352 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:15:31.353 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:15:31.353 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:31.361 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:31.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 Processor -- start 2023-11-06T03:15:31.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 -- start start 2023-11-06T03:15:31.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff4a015e5e0 0x7ff4a00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff4a015ea90 0x7ff4a00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a015f580 con 0x7ff4a01551e0 2023-11-06T03:15:31.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a00704b0 con 0x7ff4a015ea90 2023-11-06T03:15:31.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff4a4e9b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a015efc0 con 0x7ff4a015e5e0 2023-11-06T03:15:31.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff4a01551e0 0x7ff4a006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33222/0 (socket says 172.21.15.157:33222) 2023-11-06T03:15:31.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49e59c640 1 -- 172.21.15.157:0/625652589 learned_addr learned my addr 172.21.15.157:0/625652589 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:31.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49ed9d640 1 --2- 172.21.15.157:0/625652589 >> v2:172.21.15.157:3300/0 conn(0x7ff4a015ea90 0x7ff4a00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49d59a640 1 -- 172.21.15.157:0/625652589 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155853437 0 0) 0x7ff4a015f580 con 0x7ff4a01551e0 2023-11-06T03:15:31.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49d59a640 1 -- 172.21.15.157:0/625652589 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff488003530 con 0x7ff4a01551e0 2023-11-06T03:15:31.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49dd9b640 1 --2- 172.21.15.157:0/625652589 >> v2:172.21.15.195:3300/0 conn(0x7ff4a015e5e0 0x7ff4a00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49ed9d640 1 -- 172.21.15.157:0/625652589 >> v2:172.21.15.195:3300/0 conn(0x7ff4a015e5e0 msgr2=0x7ff4a00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:31.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49ed9d640 1 --2- 172.21.15.157:0/625652589 >> v2:172.21.15.195:3300/0 conn(0x7ff4a015e5e0 0x7ff4a00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49ed9d640 1 -- 172.21.15.157:0/625652589 >> v1:172.21.15.157:6789/0 conn(0x7ff4a01551e0 legacy=0x7ff4a006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.532+0000 7ff49ed9d640 1 -- 172.21.15.157:0/625652589 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a015f2a0 con 0x7ff4a015ea90 2023-11-06T03:15:31.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49ed9d640 1 --2- 172.21.15.157:0/625652589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a015ea90 0x7ff4a00b3950 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0066880 tx=0x7ff49801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fcf04d52162fbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49d59a640 1 -- 172.21.15.157:0/625652589 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980a5910 con 0x7ff4a015ea90 2023-11-06T03:15:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49d59a640 1 -- 172.21.15.157:0/625652589 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4980a5a70 con 0x7ff4a015ea90 2023-11-06T03:15:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49d59a640 1 -- 172.21.15.157:0/625652589 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980a5e00 con 0x7ff4a015ea90 2023-11-06T03:15:31.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/625652589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a015ea90 msgr2=0x7ff4a00b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/625652589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a015ea90 0x7ff4a00b3950 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0066880 tx=0x7ff49801c850 comp rx=0 tx=0).stop 2023-11-06T03:15:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/625652589 shutdown_connections 2023-11-06T03:15:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/625652589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a015ea90 0x7ff4a00b3950 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/625652589 >> v2:172.21.15.195:3300/0 conn(0x7ff4a015e5e0 0x7ff4a00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/625652589 >> 172.21.15.157:0/625652589 conn(0x7ff4a0065930 msgr2=0x7ff4a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/625652589 shutdown_connections 2023-11-06T03:15:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/625652589 wait complete. 2023-11-06T03:15:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 Processor -- start 2023-11-06T03:15:31.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- start start 2023-11-06T03:15:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a015e5e0 0x7ff4a0153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 0x7ff4a00ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a006cb30 con 0x7ff4a01551e0 2023-11-06T03:15:31.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a014e6c0 con 0x7ff4a015ea90 2023-11-06T03:15:31.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff4a4e9b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0152cb0 con 0x7ff4a015e5e0 2023-11-06T03:15:31.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52856/0 (socket says 172.21.15.157:52856) 2023-11-06T03:15:31.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 -- 172.21.15.157:0/2776811908 learned_addr learned my addr 172.21.15.157:0/2776811908 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:31.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49ed9d640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 0x7ff4a00ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49dd9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a015e5e0 0x7ff4a0153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 -- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a015e5e0 msgr2=0x7ff4a0153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a015e5e0 0x7ff4a0153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 -- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 msgr2=0x7ff4a00ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 0x7ff4a00ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49e59c640 1 -- 172.21.15.157:0/2776811908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009a70 con 0x7ff4a01551e0 2023-11-06T03:15:31.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.536+0000 7ff49ed9d640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 0x7ff4a00ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:31.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff49e59c640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c4d0 tx=0x7ff49000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490005cf0 con 0x7ff4a01551e0 2023-11-06T03:15:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490005e50 con 0x7ff4a01551e0 2023-11-06T03:15:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a0152f90 con 0x7ff4a01551e0 2023-11-06T03:15:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490007270 con 0x7ff4a01551e0 2023-11-06T03:15:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a0153550 con 0x7ff4a01551e0 2023-11-06T03:15:31.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a0153770 con 0x7ff4a01551e0 2023-11-06T03:15:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff4900073d0 con 0x7ff4a01551e0 2023-11-06T03:15:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 0x7ff480067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff4900827d0 con 0x7ff4a01551e0 2023-11-06T03:15:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff49dd9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 0x7ff480067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.540+0000 7ff49dd9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 0x7ff480067f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff4a01549b0 tx=0x7ff494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.544+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff490050110 con 0x7ff4a01551e0 2023-11-06T03:15:31.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.700+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4a0072750 con 0x7ff4a01551e0 2023-11-06T03:15:31.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.704+0000 7ff48effd640 1 -- 172.21.15.157:0/2776811908 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18020 (secure 0 0 0) 0x7ff490017090 con 0x7ff4a01551e0 2023-11-06T03:15:31.702 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:31.702 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":138,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:22.475708+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/1004239715":"2023-11-07T03:15:22.475619+0000","172.21.15.157:6834/1004239715":"2023-11-07T03:15:22.475619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:31.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 msgr2=0x7ff480067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 0x7ff480067f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff4a01549b0 tx=0x7ff494008040 comp rx=0 tx=0).stop 2023-11-06T03:15:31.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 msgr2=0x7ff4a014d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c4d0 tx=0x7ff49000c990 comp rx=0 tx=0).stop 2023-11-06T03:15:31.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 shutdown_connections 2023-11-06T03:15:31.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff480065a60 0x7ff480067f20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff4a015ea90 0x7ff4a00ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff4a015e5e0 0x7ff4a0153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 --2- 172.21.15.157:0/2776811908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4a01551e0 0x7ff4a014d640 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 >> 172.21.15.157:0/2776811908 conn(0x7ff4a0065930 msgr2=0x7ff4a015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 shutdown_connections 2023-11-06T03:15:31.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.708+0000 7ff4a4e9b640 1 -- 172.21.15.157:0/2776811908 wait complete. 2023-11-06T03:15:31.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:14:33.607794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:14:33.978137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:15:31.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:15:31.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.844+0000 7f05f2f7c640 1 Processor -- start 2023-11-06T03:15:31.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f2f7c640 1 -- start start 2023-11-06T03:15:31.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f2f7c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05ec0d0280 0x7f05ec167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f2f7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0d2920 con 0x7f05ec15caa0 2023-11-06T03:15:31.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f2f7c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0d18a0 con 0x7f05ec15ce80 2023-11-06T03:15:31.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f2f7c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0d1b60 con 0x7f05ec0d0280 2023-11-06T03:15:31.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f1519640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f05ec0d0280 0x7f05ec167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f0d18640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f05ec15caa0 0x7f05ec0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33238/0 (socket says 172.21.15.157:33238) 2023-11-06T03:15:31.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f0d18640 1 -- 172.21.15.157:0/788328083 learned_addr learned my addr 172.21.15.157:0/788328083 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:31.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f1519640 1 -- 172.21.15.157:0/788328083 >> v1:172.21.15.195:6789/0 conn(0x7f05ec15ce80 legacy=0x7f05ec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:31.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f1519640 1 -- 172.21.15.157:0/788328083 >> v1:172.21.15.157:6789/0 conn(0x7f05ec15caa0 legacy=0x7f05ec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f1519640 1 -- 172.21.15.157:0/788328083 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0d1e40 con 0x7f05ec0d0280 2023-11-06T03:15:31.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05f1519640 1 --2- 172.21.15.157:0/788328083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec0d0280 0x7f05ec167860 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e401c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eadb702e2e071f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.848+0000 7f05e37fe640 1 -- 172.21.15.157:0/788328083 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a57f0 con 0x7f05ec0d0280 2023-11-06T03:15:31.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e37fe640 1 -- 172.21.15.157:0/788328083 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40a5950 con 0x7f05ec0d0280 2023-11-06T03:15:31.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e37fe640 1 -- 172.21.15.157:0/788328083 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a5ce0 con 0x7f05ec0d0280 2023-11-06T03:15:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/788328083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec0d0280 msgr2=0x7f05ec167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/788328083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec0d0280 0x7f05ec167860 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e401c770 comp rx=0 tx=0).stop 2023-11-06T03:15:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/788328083 shutdown_connections 2023-11-06T03:15:31.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/788328083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec0d0280 0x7f05ec167860 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/788328083 >> 172.21.15.157:0/788328083 conn(0x7f05ec070790 msgr2=0x7f05ec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/788328083 shutdown_connections 2023-11-06T03:15:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/788328083 wait complete. 2023-11-06T03:15:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 Processor -- start 2023-11-06T03:15:31.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- start start 2023-11-06T03:15:31.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05ec0d0280 0x7f05ec156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05ec15ce80 0x7f05ec152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec153160 con 0x7f05ec15caa0 2023-11-06T03:15:31.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec156fd0 con 0x7f05ec15ce80 2023-11-06T03:15:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec1533b0 con 0x7f05ec0d0280 2023-11-06T03:15:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52866/0 (socket says 172.21.15.157:52866) 2023-11-06T03:15:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 -- 172.21.15.157:0/800112227 learned_addr learned my addr 172.21.15.157:0/800112227 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:31.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f0d18640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05ec0d0280 0x7f05ec156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f1519640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05ec15ce80 0x7f05ec152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 -- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05ec0d0280 msgr2=0x7f05ec156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05ec0d0280 0x7f05ec156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 -- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05ec15ce80 msgr2=0x7f05ec152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05ec15ce80 0x7f05ec152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 -- 172.21.15.157:0/800112227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009a70 con 0x7f05ec15caa0 2023-11-06T03:15:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e3fff640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c410 tx=0x7f05dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011790 con 0x7f05ec15caa0 2023-11-06T03:15:31.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc0118f0 con 0x7f05ec15caa0 2023-11-06T03:15:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec153690 con 0x7f05ec15caa0 2023-11-06T03:15:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011ce0 con 0x7f05ec15caa0 2023-11-06T03:15:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.852+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec153c50 con 0x7f05ec15caa0 2023-11-06T03:15:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec072750 con 0x7f05ec15caa0 2023-11-06T03:15:31.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f05dc01c460 con 0x7f05ec15caa0 2023-11-06T03:15:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05e17fa640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 0x7f05d8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f05dc083030 con 0x7f05ec15caa0 2023-11-06T03:15:31.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05f0d18640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 0x7f05d8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.856+0000 7f05f0d18640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 0x7f05d8067f50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f70 tx=0x7f05e8007c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:31.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:31.860+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f05dc050830 con 0x7f05ec15caa0 2023-11-06T03:15:32.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05ec0b17a0 con 0x7f05ec15caa0 2023-11-06T03:15:32.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05e17fa640 1 -- 172.21.15.157:0/800112227 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05dc053e50 con 0x7f05ec15caa0 2023-11-06T03:15:32.012 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:15:32.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 msgr2=0x7f05d8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 0x7f05d8067f50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f05e8000f70 tx=0x7f05e8007c30 comp rx=0 tx=0).stop 2023-11-06T03:15:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 msgr2=0x7f05ec152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c410 tx=0x7f05dc00c8d0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 shutdown_connections 2023-11-06T03:15:32.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f05d8065a90 0x7f05d8067f50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05ec15ce80 0x7f05ec152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05ec15caa0 0x7f05ec152610 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 --2- 172.21.15.157:0/800112227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f05ec0d0280 0x7f05ec156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 >> 172.21.15.157:0/800112227 conn(0x7f05ec070790 msgr2=0x7f05ec0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 shutdown_connections 2023-11-06T03:15:32.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.016+0000 7f05f2f7c640 1 -- 172.21.15.157:0/800112227 wait complete. 2023-11-06T03:15:32.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:15:32.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 Processor -- start 2023-11-06T03:15:32.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 -- start start 2023-11-06T03:15:32.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e41599c0 0x7f85e40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85e40b2af0 0x7f85e4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e415e440 con 0x7f85e406d000 2023-11-06T03:15:32.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e406c240 con 0x7f85e40b2af0 2023-11-06T03:15:32.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85eb42a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e406c520 con 0x7f85e41599c0 2023-11-06T03:15:32.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85e40b2af0 0x7f85e4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e89c5640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f85e406d000 0x7f85e40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33246/0 (socket says 172.21.15.157:33246) 2023-11-06T03:15:32.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e89c5640 1 -- 172.21.15.157:0/1199496063 learned_addr learned my addr 172.21.15.157:0/1199496063 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:32.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e91c6640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e41599c0 0x7f85e40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 -- 172.21.15.157:0/1199496063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e41599c0 msgr2=0x7f85e40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:32.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e41599c0 0x7f85e40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 -- 172.21.15.157:0/1199496063 >> v1:172.21.15.157:6789/0 conn(0x7f85e406d000 legacy=0x7f85e40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 -- 172.21.15.157:0/1199496063 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e406c800 con 0x7f85e40b2af0 2023-11-06T03:15:32.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85dbfff640 1 -- 172.21.15.157:0/1199496063 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308138105 0 0) 0x7f85e415e440 con 0x7f85e406d000 2023-11-06T03:15:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85e99c7640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e40b2af0 0x7f85e4159da0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009d90 tx=0x7f85dc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c1283c2f6fbbf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85dbfff640 1 -- 172.21.15.157:0/1199496063 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0a5a00 con 0x7f85e40b2af0 2023-11-06T03:15:32.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85dbfff640 1 -- 172.21.15.157:0/1199496063 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0a5b60 con 0x7f85e40b2af0 2023-11-06T03:15:32.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.152+0000 7f85dbfff640 1 -- 172.21.15.157:0/1199496063 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0ad440 con 0x7f85e40b2af0 2023-11-06T03:15:32.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1199496063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e40b2af0 msgr2=0x7f85e4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e40b2af0 0x7f85e4159da0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009d90 tx=0x7f85dc01c920 comp rx=0 tx=0).stop 2023-11-06T03:15:32.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1199496063 shutdown_connections 2023-11-06T03:15:32.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e40b2af0 0x7f85e4159da0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1199496063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e41599c0 0x7f85e40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1199496063 >> 172.21.15.157:0/1199496063 conn(0x7f85e4065930 msgr2=0x7f85e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1199496063 shutdown_connections 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1199496063 wait complete. 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 Processor -- start 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- start start 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e40b2af0 0x7f85e40cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85e41599c0 0x7f85e4158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e414efb0 con 0x7f85e406d000 2023-11-06T03:15:32.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40cd220 con 0x7f85e41599c0 2023-11-06T03:15:32.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40cc610 con 0x7f85e40b2af0 2023-11-06T03:15:32.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52880/0 (socket says 172.21.15.157:52880) 2023-11-06T03:15:32.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 -- 172.21.15.157:0/1285775037 learned_addr learned my addr 172.21.15.157:0/1285775037 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:32.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 -- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e40b2af0 msgr2=0x7f85e40cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e40b2af0 0x7f85e40cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 -- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85e41599c0 msgr2=0x7f85e4158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e99c7640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85e41599c0 0x7f85e4158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85e41599c0 0x7f85e4158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 -- 172.21.15.157:0/1285775037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc009a70 con 0x7f85e406d000 2023-11-06T03:15:32.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85e91c6640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c7f0 tx=0x7f85e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004220 con 0x7f85e406d000 2023-11-06T03:15:32.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e00043a0 con 0x7f85e406d000 2023-11-06T03:15:32.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e40cc8f0 con 0x7f85e406d000 2023-11-06T03:15:32.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004510 con 0x7f85e406d000 2023-11-06T03:15:32.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.156+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e40cceb0 con 0x7f85e406d000 2023-11-06T03:15:32.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e4072750 con 0x7f85e406d000 2023-11-06T03:15:32.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f85e0003a70 con 0x7f85e406d000 2023-11-06T03:15:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85d9ffb640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 0x7f85c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f85e00833a0 con 0x7f85e406d000 2023-11-06T03:15:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85e89c5640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 0x7f85c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.160+0000 7f85e89c5640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 0x7f85c8067f20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f85e406fcc0 tx=0x7f85d4007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.164+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f85e0050ba0 con 0x7f85e406d000 2023-11-06T03:15:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.304+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85e40cb110 con 0x7f85e406d000 2023-11-06T03:15:32.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.304+0000 7f85d9ffb640 1 -- 172.21.15.157:0/1285775037 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85e001e020 con 0x7f85e406d000 2023-11-06T03:15:32.303 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:15:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 msgr2=0x7f85c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 0x7f85c8067f20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f85e406fcc0 tx=0x7f85d4007970 comp rx=0 tx=0).stop 2023-11-06T03:15:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 msgr2=0x7f85e40cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c7f0 tx=0x7f85e000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 shutdown_connections 2023-11-06T03:15:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f85c8065a60 0x7f85c8067f20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85e41599c0 0x7f85e4158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f85e40b2af0 0x7f85e40cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 --2- 172.21.15.157:0/1285775037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85e406d000 0x7f85e40cbaa0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 >> 172.21.15.157:0/1285775037 conn(0x7f85e4065930 msgr2=0x7f85e40b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 shutdown_connections 2023-11-06T03:15:32.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.308+0000 7f85eb42a640 1 -- 172.21.15.157:0/1285775037 wait complete. 2023-11-06T03:15:32.320 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:15:32.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.440+0000 7fae20558640 1 Processor -- start 2023-11-06T03:15:32.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 -- start start 2023-11-06T03:15:32.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae181551e0 0x7fae1806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae180b3b20 0x7fae1815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1815f580 con 0x7fae1815e5e0 2023-11-06T03:15:32.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1815efc0 con 0x7fae181551e0 2023-11-06T03:15:32.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae20558640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1815f2a0 con 0x7fae180b3b20 2023-11-06T03:15:32.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae181551e0 0x7fae1806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fae181551e0 0x7fae1806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52892/0 (socket says 172.21.15.157:52892) 2023-11-06T03:15:32.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 -- 172.21.15.157:0/3491853862 learned_addr learned my addr 172.21.15.157:0/3491853862 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:32.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1eaf5640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae180b3b20 0x7fae1815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 -- 172.21.15.157:0/3491853862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae180b3b20 msgr2=0x7fae1815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:32.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae180b3b20 0x7fae1815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 -- 172.21.15.157:0/3491853862 >> v1:172.21.15.195:6789/0 conn(0x7fae1815e5e0 legacy=0x7fae180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 -- 172.21.15.157:0/3491853862 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1806f5a0 con 0x7fae181551e0 2023-11-06T03:15:32.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.444+0000 7fae1e2f4640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae181551e0 0x7fae1806fda0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fae14009870 tx=0x7fae1401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bf0d90452be5a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1d2f2640 1 -- 172.21.15.157:0/3491853862 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140a6040 con 0x7fae181551e0 2023-11-06T03:15:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1d2f2640 1 -- 172.21.15.157:0/3491853862 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae140aa600 con 0x7fae181551e0 2023-11-06T03:15:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1d2f2640 1 -- 172.21.15.157:0/3491853862 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140aa990 con 0x7fae181551e0 2023-11-06T03:15:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/3491853862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae181551e0 msgr2=0x7fae1806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae181551e0 0x7fae1806fda0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fae14009870 tx=0x7fae1401c480 comp rx=0 tx=0).stop 2023-11-06T03:15:32.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/3491853862 shutdown_connections 2023-11-06T03:15:32.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae180b3b20 0x7fae1815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- 172.21.15.157:0/3491853862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae181551e0 0x7fae1806fda0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/3491853862 >> 172.21.15.157:0/3491853862 conn(0x7fae18065930 msgr2=0x7fae18065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/3491853862 shutdown_connections 2023-11-06T03:15:32.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/3491853862 wait complete. 2023-11-06T03:15:32.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 Processor -- start 2023-11-06T03:15:32.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- start start 2023-11-06T03:15:32.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae181551e0 0x7fae1814d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae1815e5e0 0x7fae18142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18142d90 con 0x7fae180b3b20 2023-11-06T03:15:32.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180cd110 con 0x7fae1815e5e0 2023-11-06T03:15:32.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae18143f40 con 0x7fae181551e0 2023-11-06T03:15:32.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52900/0 (socket says 172.21.15.157:52900) 2023-11-06T03:15:32.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 -- 172.21.15.157:0/1929359160 learned_addr learned my addr 172.21.15.157:0/1929359160 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:32.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1daf3640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae181551e0 0x7fae1814d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 -- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae181551e0 msgr2=0x7fae1814d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae181551e0 0x7fae1814d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 -- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae1815e5e0 msgr2=0x7fae18142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae1815e5e0 0x7fae18142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 -- 172.21.15.157:0/1929359160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140094d0 con 0x7fae180b3b20 2023-11-06T03:15:32.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae1e2f4640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fae1401a0b0 tx=0x7fae1401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140b3800 con 0x7fae180b3b20 2023-11-06T03:15:32.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae140b3960 con 0x7fae180b3b20 2023-11-06T03:15:32.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae18144220 con 0x7fae180b3b20 2023-11-06T03:15:32.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140b3cd0 con 0x7fae180b3b20 2023-11-06T03:15:32.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.448+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae180ccda0 con 0x7fae180b3b20 2023-11-06T03:15:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae18072750 con 0x7fae180b3b20 2023-11-06T03:15:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fae140bd3f0 con 0x7fae180b3b20 2023-11-06T03:15:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae0effd640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 0x7fadfc067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fae1411d2c0 con 0x7fae180b3b20 2023-11-06T03:15:32.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae1daf3640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 0x7fadfc067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.452+0000 7fae1daf3640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 0x7fadfc067ef0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fae18143e70 tx=0x7fae08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.456+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fae140eabe0 con 0x7fae180b3b20 2023-11-06T03:15:32.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.600+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae18071730 con 0x7fae180b3b20 2023-11-06T03:15:32.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.600+0000 7fae0effd640 1 -- 172.21.15.157:0/1929359160 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae140ee200 con 0x7fae180b3b20 2023-11-06T03:15:32.597 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:15:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.600+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 msgr2=0x7fadfc067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.600+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 0x7fadfc067ef0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fae18143e70 tx=0x7fae08009040 comp rx=0 tx=0).stop 2023-11-06T03:15:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 msgr2=0x7fae18143930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fae1401a0b0 tx=0x7fae1401acc0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 shutdown_connections 2023-11-06T03:15:32.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fadfc065a30 0x7fadfc067ef0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae1815e5e0 0x7fae18142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae181551e0 0x7fae1814d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 --2- 172.21.15.157:0/1929359160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae180b3b20 0x7fae18143930 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 >> 172.21.15.157:0/1929359160 conn(0x7fae18065930 msgr2=0x7fae180b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 shutdown_connections 2023-11-06T03:15:32.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.604+0000 7fae20558640 1 -- 172.21.15.157:0/1929359160 wait complete. 2023-11-06T03:15:32.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:15:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.776+0000 7f204dd61640 1 Processor -- start 2023-11-06T03:15:32.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.776+0000 7f204dd61640 1 -- start start 2023-11-06T03:15:32.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f204dd61640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f20480b3b20 0x7f204815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f204dd61640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204815f580 con 0x7f20481551e0 2023-11-06T03:15:32.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f204dd61640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204815efc0 con 0x7f204815e5e0 2023-11-06T03:15:32.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f204dd61640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20480704b0 con 0x7f20480b3b20 2023-11-06T03:15:32.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20477fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f20481551e0 0x7f204806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33256/0 (socket says 172.21.15.157:33256) 2023-11-06T03:15:32.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20477fe640 1 -- 172.21.15.157:0/3729283026 learned_addr learned my addr 172.21.15.157:0/3729283026 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:32.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 --2- 172.21.15.157:0/3729283026 >> v2:172.21.15.195:3300/0 conn(0x7f20480b3b20 0x7f204815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306772975 0 0) 0x7f204815f580 con 0x7f20481551e0 2023-11-06T03:15:32.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2030003530 con 0x7f20481551e0 2023-11-06T03:15:32.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203522171 0 0) 0x7f2030003530 con 0x7f20481551e0 2023-11-06T03:15:32.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 >> v2:172.21.15.195:3300/0 conn(0x7f20480b3b20 msgr2=0x7f204815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:32.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 --2- 172.21.15.157:0/3729283026 >> v2:172.21.15.195:3300/0 conn(0x7f20480b3b20 0x7f204815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 >> v1:172.21.15.195:6789/0 conn(0x7f204815e5e0 legacy=0x7f20480b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204815f2a0 con 0x7f20481551e0 2023-11-06T03:15:32.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2038001680 con 0x7f20481551e0 2023-11-06T03:15:32.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 >> v1:172.21.15.157:6789/0 conn(0x7f20481551e0 legacy=0x7f204806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2030003980 0x7f2030003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300047d0 0x7f2030004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 0x7f203000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20480704b0 con 0x7f20300052b0 2023-11-06T03:15:32.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203000c180 con 0x7f20300047d0 2023-11-06T03:15:32.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203000c340 con 0x7f2030003980 2023-11-06T03:15:32.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 0x7f203000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2046ffd640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300047d0 0x7f2030004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f20477fe640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2030003980 0x7f2030003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 -- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2030003980 msgr2=0x7f2030003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:32.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2030003980 0x7f2030003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 -- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300047d0 msgr2=0x7f2030004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:32.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300047d0 0x7f2030004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2047fff640 1 -- 172.21.15.157:0/3729283026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030003530 con 0x7f20300052b0 2023-11-06T03:15:32.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.780+0000 7f2046ffd640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300047d0 0x7f2030004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:32.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f2047fff640 1 -- 172.21.15.157:0/3729283026 reap_dead start 2023-11-06T03:15:32.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f2047fff640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 0x7f203000b990 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2040009f90 tx=0x7f204001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9724bd837dcd38f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400a5c40 con 0x7f20300052b0 2023-11-06T03:15:32.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204000f040 con 0x7f20300052b0 2023-11-06T03:15:32.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f20467fc640 1 -- 172.21.15.157:0/3729283026 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400039a0 con 0x7f20300052b0 2023-11-06T03:15:32.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 msgr2=0x7f203000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 0x7f203000b990 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2040009f90 tx=0x7f204001cc30 comp rx=0 tx=0).stop 2023-11-06T03:15:32.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- 172.21.15.157:0/3729283026 shutdown_connections 2023-11-06T03:15:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 --2- 172.21.15.157:0/3729283026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20300052b0 0x7f203000b990 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- 172.21.15.157:0/3729283026 >> 172.21.15.157:0/3729283026 conn(0x7f2048065930 msgr2=0x7f2048065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- 172.21.15.157:0/3729283026 shutdown_connections 2023-11-06T03:15:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- 172.21.15.157:0/3729283026 wait complete. 2023-11-06T03:15:32.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 Processor -- start 2023-11-06T03:15:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 -- start start 2023-11-06T03:15:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.784+0000 7f204dd61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300052b0 0x7f2048150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f204813a2b0 0x7f204814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204814c220 con 0x7f204814bda0 2023-11-06T03:15:32.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2048149ea0 con 0x7f20300052b0 2023-11-06T03:15:32.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204814c4b0 con 0x7f204813a2b0 2023-11-06T03:15:32.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52918/0 (socket says 172.21.15.157:52918) 2023-11-06T03:15:32.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 -- 172.21.15.157:0/2995261202 learned_addr learned my addr 172.21.15.157:0/2995261202 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2046ffd640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f204813a2b0 0x7f204814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f20477fe640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300052b0 0x7f2048150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 -- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f204813a2b0 msgr2=0x7f204814b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f204813a2b0 0x7f204814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 -- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300052b0 msgr2=0x7f2048150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300052b0 0x7f2048150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 -- 172.21.15.157:0/2995261202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20480b08e0 con 0x7f204814bda0 2023-11-06T03:15:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f2047fff640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f204001d4d0 tx=0x7f2040009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040003ba0 con 0x7f204814bda0 2023-11-06T03:15:32.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040003d00 con 0x7f204814bda0 2023-11-06T03:15:32.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20480b0bc0 con 0x7f204814bda0 2023-11-06T03:15:32.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040004260 con 0x7f204814bda0 2023-11-06T03:15:32.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204814c790 con 0x7f204814bda0 2023-11-06T03:15:32.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204815c1b0 con 0x7f204814bda0 2023-11-06T03:15:32.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f20400043c0 con 0x7f204814bda0 2023-11-06T03:15:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 0x7f2034067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f204011e580 con 0x7f204814bda0 2023-11-06T03:15:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.788+0000 7f20477fe640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 0x7f2034067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:32.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.792+0000 7f20477fe640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 0x7f2034067f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f20380010f0 tx=0x7f2038000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:32.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.792+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f20400ebd80 con 0x7f204814bda0 2023-11-06T03:15:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.932+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2048072750 con 0x7f204814bda0 2023-11-06T03:15:32.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f202ffff640 1 -- 172.21.15.157:0/2995261202 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20400ef3a0 con 0x7f204814bda0 2023-11-06T03:15:32.932 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:15:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 msgr2=0x7f2034067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 0x7f2034067f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f20380010f0 tx=0x7f2038000ee0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 msgr2=0x7f204806b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f204001d4d0 tx=0x7f2040009750 comp rx=0 tx=0).stop 2023-11-06T03:15:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 shutdown_connections 2023-11-06T03:15:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2034065a60 0x7f2034067f20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f204814bda0 0x7f204806b490 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f204813a2b0 0x7f204814b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 --2- 172.21.15.157:0/2995261202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20300052b0 0x7f2048150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 >> 172.21.15.157:0/2995261202 conn(0x7f2048065930 msgr2=0x7f20480ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 shutdown_connections 2023-11-06T03:15:32.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:32.936+0000 7f204dd61640 1 -- 172.21.15.157:0/2995261202 wait complete. 2023-11-06T03:15:32.950 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:15:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 Processor -- start 2023-11-06T03:15:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 -- start start 2023-11-06T03:15:33.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65c415e5e0 0x7f65c40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c415f580 con 0x7f65c41551e0 2023-11-06T03:15:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c415efc0 con 0x7f65c415ea90 2023-11-06T03:15:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65ca275640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c40704b0 con 0x7f65c415e5e0 2023-11-06T03:15:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65c415e5e0 0x7f65c40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f65c41551e0 0x7f65c406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33270/0 (socket says 172.21.15.157:33270) 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c3fff640 1 -- 172.21.15.157:0/2250315366 learned_addr learned my addr 172.21.15.157:0/2250315366 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c2ffd640 1 -- 172.21.15.157:0/2250315366 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28858593 0 0) 0x7f65c415f580 con 0x7f65c41551e0 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c2ffd640 1 -- 172.21.15.157:0/2250315366 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65ac003530 con 0x7f65c41551e0 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c37fe640 1 -- 172.21.15.157:0/2250315366 >> v1:172.21.15.195:6789/0 conn(0x7f65c415ea90 legacy=0x7f65c415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c37fe640 1 -- 172.21.15.157:0/2250315366 >> v1:172.21.15.157:6789/0 conn(0x7f65c41551e0 legacy=0x7f65c406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.060+0000 7f65c37fe640 1 -- 172.21.15.157:0/2250315366 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c415f2a0 con 0x7f65c415e5e0 2023-11-06T03:15:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65c37fe640 1 --2- 172.21.15.157:0/2250315366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40b3410 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f65b80097c0 tx=0x7f65b801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f5143b1635da88 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65c2ffd640 1 -- 172.21.15.157:0/2250315366 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80ab600 con 0x7f65c415e5e0 2023-11-06T03:15:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65c2ffd640 1 -- 172.21.15.157:0/2250315366 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80a7050 con 0x7f65c415e5e0 2023-11-06T03:15:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65c2ffd640 1 -- 172.21.15.157:0/2250315366 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80ab980 con 0x7f65c415e5e0 2023-11-06T03:15:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- 172.21.15.157:0/2250315366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 msgr2=0x7f65c40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 --2- 172.21.15.157:0/2250315366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40b3410 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f65b80097c0 tx=0x7f65b801c380 comp rx=0 tx=0).stop 2023-11-06T03:15:33.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- 172.21.15.157:0/2250315366 shutdown_connections 2023-11-06T03:15:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 --2- 172.21.15.157:0/2250315366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40b3410 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- 172.21.15.157:0/2250315366 >> 172.21.15.157:0/2250315366 conn(0x7f65c4065930 msgr2=0x7f65c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- 172.21.15.157:0/2250315366 shutdown_connections 2023-11-06T03:15:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- 172.21.15.157:0/2250315366 wait complete. 2023-11-06T03:15:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 Processor -- start 2023-11-06T03:15:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- start start 2023-11-06T03:15:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65c41551e0 0x7f65c4151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c415ea90 0x7f65c4148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c4153490 con 0x7f65c415e5e0 2023-11-06T03:15:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c4152880 con 0x7f65c415ea90 2023-11-06T03:15:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.064+0000 7f65ca275640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c4152b60 con 0x7f65c41551e0 2023-11-06T03:15:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52932/0 (socket says 172.21.15.157:52932) 2023-11-06T03:15:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 -- 172.21.15.157:0/3411040726 learned_addr learned my addr 172.21.15.157:0/3411040726 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c3fff640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65c41551e0 0x7f65c4151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 -- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65c41551e0 msgr2=0x7f65c4151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c8812640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c415ea90 0x7f65c4148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65c41551e0 0x7f65c4151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 -- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c415ea90 msgr2=0x7f65c4148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c415ea90 0x7f65c4148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 -- 172.21.15.157:0/3411040726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b8009420 con 0x7f65c415e5e0 2023-11-06T03:15:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c37fe640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f65b80ae040 tx=0x7f65b8019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80b4840 con 0x7f65c415e5e0 2023-11-06T03:15:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80b49a0 con 0x7f65c415e5e0 2023-11-06T03:15:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c4152e40 con 0x7f65c415e5e0 2023-11-06T03:15:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80b4d50 con 0x7f65c415e5e0 2023-11-06T03:15:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c4152420 con 0x7f65c415e5e0 2023-11-06T03:15:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c4072750 con 0x7f65c415e5e0 2023-11-06T03:15:33.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f65b80b2040 con 0x7f65c415e5e0 2023-11-06T03:15:33.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 0x7f65b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f65b811d6c0 con 0x7f65c415e5e0 2023-11-06T03:15:33.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c3fff640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 0x7f65b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.068+0000 7f65c3fff640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 0x7f65b0067f20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f65b40011f0 tx=0x7f65b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.072+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f65b80eaec0 con 0x7f65c415e5e0 2023-11-06T03:15:33.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65c40cb110 con 0x7f65c415e5e0 2023-11-06T03:15:33.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65c0ff9640 1 -- 172.21.15.157:0/3411040726 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v138) v1 ==== 80+0+124 (secure 0 0 0) 0x7f65b80b8090 con 0x7f65c415e5e0 2023-11-06T03:15:33.229 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157:6835/1004239715 2023-11-07T03:15:22.475619+0000 2023-11-06T03:15:33.229 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157:6834/1004239715 2023-11-07T03:15:22.475619+0000 2023-11-06T03:15:33.229 INFO:teuthology.orchestra.run.smithi157.stderr:listed 2 entries 2023-11-06T03:15:33.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 msgr2=0x7f65b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 0x7f65b0067f20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f65b40011f0 tx=0x7f65b4009040 comp rx=0 tx=0).stop 2023-11-06T03:15:33.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 msgr2=0x7f65c40aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f65b80ae040 tx=0x7f65b8019120 comp rx=0 tx=0).stop 2023-11-06T03:15:33.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 shutdown_connections 2023-11-06T03:15:33.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65b0065a60 0x7f65b0067f20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65c415ea90 0x7f65c4148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c415e5e0 0x7f65c40aed80 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 --2- 172.21.15.157:0/3411040726 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65c41551e0 0x7f65c4151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 >> 172.21.15.157:0/3411040726 conn(0x7f65c4065930 msgr2=0x7f65c40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:33.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 shutdown_connections 2023-11-06T03:15:33.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.232+0000 7f65ca275640 1 -- 172.21.15.157:0/3411040726 wait complete. 2023-11-06T03:15:33.247 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:15:33.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:33.247 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:33.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[90618]: fuse finished with error 0 and tester_r 0 2023-11-06T03:15:33.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:15:33.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:33.259 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:15:33.421 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:15:33.422 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:33.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:15:33.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:15:33.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:33.430 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:15:33.475 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:15:33.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:15:33.477 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:15:33.477 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:33.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:15:33.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:15:33.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:33.484 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:15:33.531 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:15:33.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:15:33.533 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:15:33.533 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:33.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:15:33.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:15:33.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:33.540 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:15:33.587 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:15:33.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:15:33.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:33.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 Processor -- start 2023-11-06T03:15:33.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 -- start start 2023-11-06T03:15:33.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0fb00d0280 0x7f0fb00b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb00d2920 con 0x7f0fb015ce80 2023-11-06T03:15:33.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00d18a0 con 0x7f0fb00d0280 2023-11-06T03:15:33.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb6715640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00d1b60 con 0x7f0fb015caa0 2023-11-06T03:15:33.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0fb00d0280 0x7f0fb00b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0fb00d0280 0x7f0fb00b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52942/0 (socket says 172.21.15.157:52942) 2023-11-06T03:15:33.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/2351796080 learned_addr learned my addr 172.21.15.157:0/2351796080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:33.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0faffff640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/2351796080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 msgr2=0x7f0fb00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:33.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb00cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/2351796080 >> v1:172.21.15.195:6789/0 conn(0x7f0fb015ce80 legacy=0x7f0fb00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:33.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/2351796080 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00d1e40 con 0x7f0fb00d0280 2023-11-06T03:15:33.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.716+0000 7f0faffff640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:33.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb00d0280 0x7f0fb00b5d50 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009e50 tx=0x7f0fa801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da80aa63006b81d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0faf7fe640 1 -- 172.21.15.157:0/2351796080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80a5820 con 0x7f0fb00d0280 2023-11-06T03:15:33.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0faf7fe640 1 -- 172.21.15.157:0/2351796080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa80a5980 con 0x7f0fb00d0280 2023-11-06T03:15:33.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0faf7fe640 1 -- 172.21.15.157:0/2351796080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80a5d10 con 0x7f0fb00d0280 2023-11-06T03:15:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/2351796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb00d0280 msgr2=0x7f0fb00b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb00d0280 0x7f0fb00b5d50 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009e50 tx=0x7f0fa801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/2351796080 shutdown_connections 2023-11-06T03:15:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb00d0280 0x7f0fb00b5d50 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- 172.21.15.157:0/2351796080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/2351796080 >> 172.21.15.157:0/2351796080 conn(0x7f0fb0070790 msgr2=0x7f0fb0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/2351796080 shutdown_connections 2023-11-06T03:15:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/2351796080 wait complete. 2023-11-06T03:15:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 Processor -- start 2023-11-06T03:15:33.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- start start 2023-11-06T03:15:33.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 0x7f0fb0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb0152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00ca000 con 0x7f0fb015ce80 2023-11-06T03:15:33.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0158030 con 0x7f0fb00d0280 2023-11-06T03:15:33.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0157a70 con 0x7f0fb015caa0 2023-11-06T03:15:33.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52948/0 (socket says 172.21.15.157:52948) 2023-11-06T03:15:33.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/3170070017 learned_addr learned my addr 172.21.15.157:0/3170070017 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:33.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fa7fff640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb0152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0faffff640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 0x7f0fb0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 msgr2=0x7f0fb0152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb0152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 msgr2=0x7f0fb0154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 0x7f0fb0154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 -- 172.21.15.157:0/3170070017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8009b30 con 0x7f0fb015ce80 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0faffff640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 0x7f0fb0154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb4cb2640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80af040 tx=0x7f0fa8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80b8670 con 0x7f0fb015ce80 2023-11-06T03:15:33.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa80b8810 con 0x7f0fb015ce80 2023-11-06T03:15:33.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb0157d50 con 0x7f0fb015ce80 2023-11-06T03:15:33.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80b8bc0 con 0x7f0fb015ce80 2023-11-06T03:15:33.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.720+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb015d800 con 0x7f0fb015ce80 2023-11-06T03:15:33.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb015d9f0 con 0x7f0fb015ce80 2023-11-06T03:15:33.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0fa80b8d20 con 0x7f0fb015ce80 2023-11-06T03:15:33.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0fad7fa640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 0x7f0f90067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:33.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0fa811e5c0 con 0x7f0fb015ce80 2023-11-06T03:15:33.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0faffff640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 0x7f0f90067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:33.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.724+0000 7f0faffff640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 0x7f0f90067f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0008930 tx=0x7f0fa0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:33.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.728+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0fa80ebdc0 con 0x7f0fb015ce80 2023-11-06T03:15:33.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.900+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fb015d9f0 con 0x7f0fb015ce80 2023-11-06T03:15:33.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.900+0000 7f0fad7fa640 1 -- 172.21.15.157:0/3170070017 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7f0fa80ef3e0 con 0x7f0fb015ce80 2023-11-06T03:15:33.898 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:33.898 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:33.898 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 msgr2=0x7f0f90067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 0x7f0f90067f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0008930 tx=0x7f0fa0008040 comp rx=0 tx=0).stop 2023-11-06T03:15:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 msgr2=0x7f0fb0152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80af040 tx=0x7f0fa8019740 comp rx=0 tx=0).stop 2023-11-06T03:15:33.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 shutdown_connections 2023-11-06T03:15:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0f90065a90 0x7f0f90067f50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fb015ce80 0x7f0fb0152ea0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fb015caa0 0x7f0fb0152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 --2- 172.21.15.157:0/3170070017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fb00d0280 0x7f0fb0154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 >> 172.21.15.157:0/3170070017 conn(0x7f0fb0070790 msgr2=0x7f0fb00b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:33.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 shutdown_connections 2023-11-06T03:15:33.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:33.904+0000 7f0fb6715640 1 -- 172.21.15.157:0/3170070017 wait complete. 2023-11-06T03:15:33.915 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:34.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.024+0000 7f676df01640 1 Processor -- start 2023-11-06T03:15:34.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.024+0000 7f676df01640 1 -- start start 2023-11-06T03:15:34.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f676df01640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67681551e0 0x7f676806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f676df01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f676815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f676df01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676815f580 con 0x7f676815e5e0 2023-11-06T03:15:34.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f676df01640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676815efc0 con 0x7f67681551e0 2023-11-06T03:15:34.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f676df01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676815f2a0 con 0x7f67680b3b20 2023-11-06T03:15:34.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f675effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f676815e5e0 0x7f67680b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33276/0 (socket says 172.21.15.157:33276) 2023-11-06T03:15:34.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f675effd640 1 -- 172.21.15.157:0/3642622082 learned_addr learned my addr 172.21.15.157:0/3642622082 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:34.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6767fff640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f676815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 --2- 172.21.15.157:0/3642622082 >> v2:172.21.15.195:3300/0 conn(0x7f67681551e0 0x7f676806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238068748 0 0) 0x7f676815f580 con 0x7f676815e5e0 2023-11-06T03:15:34.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6754003530 con 0x7f676815e5e0 2023-11-06T03:15:34.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651303566 0 0) 0x7f6754003530 con 0x7f676815e5e0 2023-11-06T03:15:34.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 msgr2=0x7f676815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f676815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 >> v2:172.21.15.195:3300/0 conn(0x7f67681551e0 msgr2=0x7f676806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 --2- 172.21.15.157:0/3642622082 >> v2:172.21.15.195:3300/0 conn(0x7f67681551e0 0x7f676806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676806f5a0 con 0x7f676815e5e0 2023-11-06T03:15:34.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6750001680 con 0x7f676815e5e0 2023-11-06T03:15:34.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 >> v1:172.21.15.157:6789/0 conn(0x7f676815e5e0 legacy=0x7f67680b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6754003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f67540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6754004d10 0x7f6754005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676815efc0 con 0x7f6754003980 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676815f2a0 con 0x7f6754004d10 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675400b9c0 con 0x7f67680b3b20 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6754003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f675effd640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f67540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 -- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 msgr2=0x7f67540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680b3b20 0x7f67540047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 -- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6754004d10 msgr2=0x7f6754005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f6767fff640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6754004d10 0x7f6754005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6754004d10 0x7f6754005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.028+0000 7f67677fe640 1 -- 172.21.15.157:0/3642622082 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754003530 con 0x7f6754003980 2023-11-06T03:15:34.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6754003da0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ae0 tx=0x7f675801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf7404805602b6c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580a5650 con 0x7f6754003980 2023-11-06T03:15:34.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67580a57b0 con 0x7f6754003980 2023-11-06T03:15:34.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6766ffd640 1 -- 172.21.15.157:0/3642622082 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580a5b60 con 0x7f6754003980 2023-11-06T03:15:34.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/3642622082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 msgr2=0x7f6754003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 --2- 172.21.15.157:0/3642622082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6754003da0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ae0 tx=0x7f675801ca60 comp rx=0 tx=0).stop 2023-11-06T03:15:34.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6767fff640 1 -- 172.21.15.157:0/3642622082 reap_dead start 2023-11-06T03:15:34.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/3642622082 shutdown_connections 2023-11-06T03:15:34.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/3642622082 >> 172.21.15.157:0/3642622082 conn(0x7f6768065930 msgr2=0x7f6768065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/3642622082 shutdown_connections 2023-11-06T03:15:34.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/3642622082 wait complete. 2023-11-06T03:15:34.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 Processor -- start 2023-11-06T03:15:34.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- start start 2023-11-06T03:15:34.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 0x7f676814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680cd430 0x7f676814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768162440 con 0x7f6754003980 2023-11-06T03:15:34.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768162720 con 0x7f67681634e0 2023-11-06T03:15:34.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6768162a00 con 0x7f67680cd430 2023-11-06T03:15:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52966/0 (socket says 172.21.15.157:52966) 2023-11-06T03:15:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 -- 172.21.15.157:0/1320950459 learned_addr learned my addr 172.21.15.157:0/1320950459 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f675effd640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 0x7f676814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6767fff640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680cd430 0x7f676814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 -- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680cd430 msgr2=0x7f676814d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680cd430 0x7f676814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 -- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 msgr2=0x7f676814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 0x7f676814fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 -- 172.21.15.157:0/1320950459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768162ce0 con 0x7f6754003980 2023-11-06T03:15:34.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f675effd640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 0x7f676814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:34.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f67677fe640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ae0 tx=0x7f6754005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580be070 con 0x7f6754003980 2023-11-06T03:15:34.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675800f000 con 0x7f6754003980 2023-11-06T03:15:34.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6768162fc0 con 0x7f6754003980 2023-11-06T03:15:34.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758004220 con 0x7f6754003980 2023-11-06T03:15:34.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.032+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676814d660 con 0x7f6754003980 2023-11-06T03:15:34.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6768072750 con 0x7f6754003980 2023-11-06T03:15:34.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f67580b4040 con 0x7f6754003980 2023-11-06T03:15:34.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f6764ff9640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 0x7f6748067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f675811cb20 con 0x7f6754003980 2023-11-06T03:15:34.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f675effd640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 0x7f6748067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.036+0000 7f675effd640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 0x7f6748067f20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f67681f7ac0 tx=0x7f6750009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.040+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f67580ea3b0 con 0x7f6754003980 2023-11-06T03:15:34.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.196+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6768071730 con 0x7f6754003980 2023-11-06T03:15:34.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.196+0000 7f6764ff9640 1 -- 172.21.15.157:0/1320950459 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18020 (secure 0 0 0) 0x7f67580ed9d0 con 0x7f6754003980 2023-11-06T03:15:34.195 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:34.195 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":138,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:22.475708+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/1004239715":"2023-11-07T03:15:22.475619+0000","172.21.15.157:6834/1004239715":"2023-11-07T03:15:22.475619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:34.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 msgr2=0x7f6748067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 0x7f6748067f20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f67681f7ac0 tx=0x7f6750009040 comp rx=0 tx=0).stop 2023-11-06T03:15:34.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 msgr2=0x7f6768161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ae0 tx=0x7f6754005800 comp rx=0 tx=0).stop 2023-11-06T03:15:34.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 shutdown_connections 2023-11-06T03:15:34.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6748065a60 0x7f6748067f20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67680cd430 0x7f676814d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67681634e0 0x7f676814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 --2- 172.21.15.157:0/1320950459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6754003980 0x7f6768161e40 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 >> 172.21.15.157:0/1320950459 conn(0x7f6768065930 msgr2=0x7f67680cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 shutdown_connections 2023-11-06T03:15:34.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.200+0000 7f676df01640 1 -- 172.21.15.157:0/1320950459 wait complete. 2023-11-06T03:15:34.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:14:33.607794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:14:33.978137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:15:34.215 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:15:34.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 Processor -- start 2023-11-06T03:15:34.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 -- start start 2023-11-06T03:15:34.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3501599c0 0x7fc3500ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35006d000 0x7fc3500b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc35015dea0 0x7fc3500b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35015e440 con 0x7fc3501599c0 2023-11-06T03:15:34.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35006c240 con 0x7fc35015dea0 2023-11-06T03:15:34.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc356857640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35006c520 con 0x7fc35006d000 2023-11-06T03:15:34.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3501599c0 0x7fc3500ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc3501599c0 0x7fc3500ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52982/0 (socket says 172.21.15.157:52982) 2023-11-06T03:15:34.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 -- 172.21.15.157:0/2202556511 learned_addr learned my addr 172.21.15.157:0/2202556511 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc354df4640 1 --2- 172.21.15.157:0/2202556511 >> v2:172.21.15.195:3300/0 conn(0x7fc35015dea0 0x7fc3500b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc347fff640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35006d000 0x7fc3500b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 -- 172.21.15.157:0/2202556511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35006d000 msgr2=0x7fc3500b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35006d000 0x7fc3500b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 -- 172.21.15.157:0/2202556511 >> v2:172.21.15.195:3300/0 conn(0x7fc35015dea0 msgr2=0x7fc3500b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 --2- 172.21.15.157:0/2202556511 >> v2:172.21.15.195:3300/0 conn(0x7fc35015dea0 0x7fc3500b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 -- 172.21.15.157:0/2202556511 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35006c800 con 0x7fc3501599c0 2023-11-06T03:15:34.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.344+0000 7fc34ffff640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3501599c0 0x7fc3500ae000 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc35006bbf0 tx=0x7fc34001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ccd7cb81b14ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34f7fe640 1 -- 172.21.15.157:0/2202556511 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400aa610 con 0x7fc3501599c0 2023-11-06T03:15:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34f7fe640 1 -- 172.21.15.157:0/2202556511 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3400a6040 con 0x7fc3501599c0 2023-11-06T03:15:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34f7fe640 1 -- 172.21.15.157:0/2202556511 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400aa990 con 0x7fc3501599c0 2023-11-06T03:15:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/2202556511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3501599c0 msgr2=0x7fc3500ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3501599c0 0x7fc3500ae000 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc35006bbf0 tx=0x7fc34001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/2202556511 shutdown_connections 2023-11-06T03:15:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- 172.21.15.157:0/2202556511 >> v2:172.21.15.195:3300/0 conn(0x7fc35015dea0 0x7fc3500b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35006d000 0x7fc3500b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- 172.21.15.157:0/2202556511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3501599c0 0x7fc3500ae000 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/2202556511 >> 172.21.15.157:0/2202556511 conn(0x7fc350065930 msgr2=0x7fc350065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/2202556511 shutdown_connections 2023-11-06T03:15:34.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/2202556511 wait complete. 2023-11-06T03:15:34.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 Processor -- start 2023-11-06T03:15:34.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- start start 2023-11-06T03:15:34.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3501599c0 0x7fc3501f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35015dea0 0x7fc3500ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350141930 con 0x7fc35006d000 2023-11-06T03:15:34.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500cc6c0 con 0x7fc3501599c0 2023-11-06T03:15:34.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501f6150 con 0x7fc35015dea0 2023-11-06T03:15:34.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52990/0 (socket says 172.21.15.157:52990) 2023-11-06T03:15:34.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 -- 172.21.15.157:0/1220734378 learned_addr learned my addr 172.21.15.157:0/1220734378 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 -- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35015dea0 msgr2=0x7fc3500ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35015dea0 0x7fc3500ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 -- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3501599c0 msgr2=0x7fc3501f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3501599c0 0x7fc3501f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 -- 172.21.15.157:0/1220734378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400094d0 con 0x7fc35006d000 2023-11-06T03:15:34.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34ffff640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc34001cd70 tx=0x7fc34001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400aaea0 con 0x7fc35006d000 2023-11-06T03:15:34.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3400bd610 con 0x7fc35006d000 2023-11-06T03:15:34.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500cbe70 con 0x7fc35006d000 2023-11-06T03:15:34.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400bd9c0 con 0x7fc35006d000 2023-11-06T03:15:34.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.348+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3500ce1c0 con 0x7fc35006d000 2023-11-06T03:15:34.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3500cdd40 con 0x7fc35006d000 2023-11-06T03:15:34.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc3400b33f0 con 0x7fc35006d000 2023-11-06T03:15:34.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc34d7fa640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 0x7fc330067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc34011d530 con 0x7fc35006d000 2023-11-06T03:15:34.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc347fff640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 0x7fc330067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.352+0000 7fc347fff640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 0x7fc330067f20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098f0 tx=0x7fc338008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.356+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc3500cdd40 con 0x7fc35006d000 2023-11-06T03:15:34.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3500cdd40 con 0x7fc35006d000 2023-11-06T03:15:34.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc34d7fa640 1 -- 172.21.15.157:0/1220734378 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3500cdd40 con 0x7fc35006d000 2023-11-06T03:15:34.494 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:15:34.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 msgr2=0x7fc330067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 0x7fc330067f20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc3380098f0 tx=0x7fc338008040 comp rx=0 tx=0).stop 2023-11-06T03:15:34.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 msgr2=0x7fc3501413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc34001cd70 tx=0x7fc34001ad10 comp rx=0 tx=0).stop 2023-11-06T03:15:34.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 shutdown_connections 2023-11-06T03:15:34.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc330065a60 0x7fc330067f20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc35015dea0 0x7fc3500ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3501599c0 0x7fc3501f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 --2- 172.21.15.157:0/1220734378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc35006d000 0x7fc3501413f0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.496+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 >> 172.21.15.157:0/1220734378 conn(0x7fc350065930 msgr2=0x7fc350158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.500+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 shutdown_connections 2023-11-06T03:15:34.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.500+0000 7fc356857640 1 -- 172.21.15.157:0/1220734378 wait complete. 2023-11-06T03:15:34.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:15:34.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.628+0000 7fb48fb2c640 1 Processor -- start 2023-11-06T03:15:34.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.628+0000 7fb48fb2c640 1 -- start start 2023-11-06T03:15:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48fb2c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4880ce800 0x7fb4880cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48fb2c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb48806b6a0 0x7fb48806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48fb2c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4880cf1a0 con 0x7fb488146370 2023-11-06T03:15:34.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48fb2c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4880b2120 con 0x7fb4880ce800 2023-11-06T03:15:34.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48fb2c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48806c3d0 con 0x7fb48806b6a0 2023-11-06T03:15:34.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb4880ce800 0x7fb4880cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d8c8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb488146370 0x7fb4880b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33280/0 (socket says 172.21.15.157:33280) 2023-11-06T03:15:34.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d8c8640 1 -- 172.21.15.157:0/645424575 learned_addr learned my addr 172.21.15.157:0/645424575 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:34.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48e0c9640 1 --2- 172.21.15.157:0/645424575 >> v2:172.21.15.195:3300/0 conn(0x7fb48806b6a0 0x7fb48806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 -- 172.21.15.157:0/645424575 >> v2:172.21.15.195:3300/0 conn(0x7fb48806b6a0 msgr2=0x7fb48806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 --2- 172.21.15.157:0/645424575 >> v2:172.21.15.195:3300/0 conn(0x7fb48806b6a0 0x7fb48806bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 -- 172.21.15.157:0/645424575 >> v1:172.21.15.157:6789/0 conn(0x7fb488146370 legacy=0x7fb4880b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 -- 172.21.15.157:0/645424575 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48806f5a0 con 0x7fb4880ce800 2023-11-06T03:15:34.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48c8c6640 1 -- 172.21.15.157:0/645424575 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153672595 0 0) 0x7fb4880cf1a0 con 0x7fb488146370 2023-11-06T03:15:34.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48d0c7640 1 --2- 172.21.15.157:0/645424575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4880ce800 0x7fb4880cec60 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097c0 tx=0x7fb47801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5218b65c60ae898 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48c8c6640 1 -- 172.21.15.157:0/645424575 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780ad680 con 0x7fb4880ce800 2023-11-06T03:15:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48c8c6640 1 -- 172.21.15.157:0/645424575 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4780ad7e0 con 0x7fb4880ce800 2023-11-06T03:15:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.632+0000 7fb48c8c6640 1 -- 172.21.15.157:0/645424575 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780adb70 con 0x7fb4880ce800 2023-11-06T03:15:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/645424575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4880ce800 msgr2=0x7fb4880cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/645424575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4880ce800 0x7fb4880cec60 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097c0 tx=0x7fb47801c380 comp rx=0 tx=0).stop 2023-11-06T03:15:34.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/645424575 shutdown_connections 2023-11-06T03:15:34.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/645424575 >> v2:172.21.15.195:3300/0 conn(0x7fb48806b6a0 0x7fb48806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/645424575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4880ce800 0x7fb4880cec60 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/645424575 >> 172.21.15.157:0/645424575 conn(0x7fb488065930 msgr2=0x7fb488065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/645424575 shutdown_connections 2023-11-06T03:15:34.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/645424575 wait complete. 2023-11-06T03:15:34.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 Processor -- start 2023-11-06T03:15:34.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- start start 2023-11-06T03:15:34.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb488146370 0x7fb48813b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb48813d2c0 0x7fb48815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488140ec0 con 0x7fb48806b6a0 2023-11-06T03:15:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488140210 con 0x7fb488146370 2023-11-06T03:15:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb488140590 con 0x7fb48813d2c0 2023-11-06T03:15:34.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53008/0 (socket says 172.21.15.157:53008) 2023-11-06T03:15:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 -- 172.21.15.157:0/2935594864 learned_addr learned my addr 172.21.15.157:0/2935594864 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d0c7640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb488146370 0x7fb48813b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 -- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb48813d2c0 msgr2=0x7fb48815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb48813d2c0 0x7fb48815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 -- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb488146370 msgr2=0x7fb48813b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb488146370 0x7fb48813b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 -- 172.21.15.157:0/2935594864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009420 con 0x7fb48806b6a0 2023-11-06T03:15:34.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48d8c8640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fb4840011f0 tx=0x7fb48400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840074e0 con 0x7fb48806b6a0 2023-11-06T03:15:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484007640 con 0x7fb48806b6a0 2023-11-06T03:15:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb488140870 con 0x7fb48806b6a0 2023-11-06T03:15:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484004a10 con 0x7fb48806b6a0 2023-11-06T03:15:34.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.636+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48813be70 con 0x7fb48806b6a0 2023-11-06T03:15:34.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb488072750 con 0x7fb48806b6a0 2023-11-06T03:15:34.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb484004b70 con 0x7fb48806b6a0 2023-11-06T03:15:34.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb47e7fc640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 0x7fb46c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb484082870 con 0x7fb48806b6a0 2023-11-06T03:15:34.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb48d0c7640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 0x7fb46c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.640+0000 7fb48d0c7640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 0x7fb46c067f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb4780af000 tx=0x7fb478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.644+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb484050070 con 0x7fb48806b6a0 2023-11-06T03:15:34.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.800+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb4880b2aa0 con 0x7fb48806b6a0 2023-11-06T03:15:34.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.800+0000 7fb47e7fc640 1 -- 172.21.15.157:0/2935594864 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb4840180a0 con 0x7fb48806b6a0 2023-11-06T03:15:34.798 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:15:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 msgr2=0x7fb46c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 0x7fb46c067f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb4780af000 tx=0x7fb478008040 comp rx=0 tx=0).stop 2023-11-06T03:15:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 msgr2=0x7fb48815f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fb4840011f0 tx=0x7fb48400c9b0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 shutdown_connections 2023-11-06T03:15:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb46c065a60 0x7fb46c067f20 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb48813d2c0 0x7fb48815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb488146370 0x7fb48813b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 --2- 172.21.15.157:0/2935594864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb48806b6a0 0x7fb48815f480 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 >> 172.21.15.157:0/2935594864 conn(0x7fb488065930 msgr2=0x7fb488070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 shutdown_connections 2023-11-06T03:15:34.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.804+0000 7fb48fb2c640 1 -- 172.21.15.157:0/2935594864 wait complete. 2023-11-06T03:15:34.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:15:34.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 Processor -- start 2023-11-06T03:15:34.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 -- start start 2023-11-06T03:15:34.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 0x7f18b006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b00b1c50 0x7f18b006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18b00b20b0 0x7f18b0164c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b01614a0 con 0x7f18b00b20b0 2023-11-06T03:15:34.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00b00d0 con 0x7f18b01610c0 2023-11-06T03:15:34.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b6a50640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00b03b0 con 0x7f18b00b1c50 2023-11-06T03:15:34.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18b00b20b0 0x7f18b0164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18b00b20b0 0x7f18b0164c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53022/0 (socket says 172.21.15.157:53022) 2023-11-06T03:15:34.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 -- 172.21.15.157:0/1671510178 learned_addr learned my addr 172.21.15.157:0/1671510178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18af7fe640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b00b1c50 0x7f18b006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18affff640 1 --2- 172.21.15.157:0/1671510178 >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 0x7f18b006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 -- 172.21.15.157:0/1671510178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b00b1c50 msgr2=0x7f18b006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b00b1c50 0x7f18b006cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 -- 172.21.15.157:0/1671510178 >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 msgr2=0x7f18b006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:34.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 --2- 172.21.15.157:0/1671510178 >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 0x7f18b006dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18b4fed640 1 -- 172.21.15.157:0/1671510178 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b006c9a0 con 0x7f18b00b20b0 2023-11-06T03:15:34.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.944+0000 7f18affff640 1 --2- 172.21.15.157:0/1671510178 >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 0x7f18b006dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:34.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b4fed640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b20b0 0x7f18b0164c30 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009870 tx=0x7f18a401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c13caeebe9361ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18aeffd640 1 -- 172.21.15.157:0/1671510178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40aa640 con 0x7f18b00b20b0 2023-11-06T03:15:34.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18aeffd640 1 -- 172.21.15.157:0/1671510178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40aa7a0 con 0x7f18b00b20b0 2023-11-06T03:15:34.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18aeffd640 1 -- 172.21.15.157:0/1671510178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40aab30 con 0x7f18b00b20b0 2023-11-06T03:15:34.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- 172.21.15.157:0/1671510178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b20b0 msgr2=0x7f18b0164c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b20b0 0x7f18b0164c30 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009870 tx=0x7f18a401c500 comp rx=0 tx=0).stop 2023-11-06T03:15:34.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- 172.21.15.157:0/1671510178 shutdown_connections 2023-11-06T03:15:34.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b20b0 0x7f18b0164c30 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- 172.21.15.157:0/1671510178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b00b1c50 0x7f18b006cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- 172.21.15.157:0/1671510178 >> v2:172.21.15.195:3300/0 conn(0x7f18b01610c0 0x7f18b006dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- 172.21.15.157:0/1671510178 >> 172.21.15.157:0/1671510178 conn(0x7f18b00732f0 msgr2=0x7f18b0069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:34.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- 172.21.15.157:0/1671510178 shutdown_connections 2023-11-06T03:15:34.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- 172.21.15.157:0/1671510178 wait complete. 2023-11-06T03:15:34.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 Processor -- start 2023-11-06T03:15:34.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- start start 2023-11-06T03:15:34.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 0x7f18b0163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b01610c0 0x7f18b00b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0157760 con 0x7f18b00b1c50 2023-11-06T03:15:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0151eb0 con 0x7f18b00b20b0 2023-11-06T03:15:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b6a50640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0150fc0 con 0x7f18b01610c0 2023-11-06T03:15:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53034/0 (socket says 172.21.15.157:53034) 2023-11-06T03:15:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 -- 172.21.15.157:0/2862920789 learned_addr learned my addr 172.21.15.157:0/2862920789 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:34.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18b4fed640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b01610c0 0x7f18b00b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18af7fe640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 0x7f18b0163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 -- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b01610c0 msgr2=0x7f18b00b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b01610c0 0x7f18b00b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 -- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 msgr2=0x7f18b0163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 0x7f18b0163dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18affff640 1 -- 172.21.15.157:0/2862920789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40094d0 con 0x7f18b00b1c50 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.948+0000 7f18af7fe640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 0x7f18b0163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18affff640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c7f0 tx=0x7f18a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007820 con 0x7f18b00b1c50 2023-11-06T03:15:34.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8007980 con 0x7f18b00b1c50 2023-11-06T03:15:34.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b01512a0 con 0x7f18b00b1c50 2023-11-06T03:15:34.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007d30 con 0x7f18b00b1c50 2023-11-06T03:15:34.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0151860 con 0x7f18b00b1c50 2023-11-06T03:15:34.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b0072750 con 0x7f18b00b1c50 2023-11-06T03:15:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f18a80113f0 con 0x7f18b00b1c50 2023-11-06T03:15:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 0x7f189c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f18a8083670 con 0x7f18b00b1c50 2023-11-06T03:15:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18af7fe640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 0x7f189c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.952+0000 7f18af7fe640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 0x7f189c067f50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f18b00cc0a0 tx=0x7f18a0008460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:34.956+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f18a8050e70 con 0x7f18b00b1c50 2023-11-06T03:15:35.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.100+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18b00d05d0 con 0x7f18b00b1c50 2023-11-06T03:15:35.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18acff9640 1 -- 172.21.15.157:0/2862920789 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18a8054490 con 0x7f18b00b1c50 2023-11-06T03:15:35.100 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:15:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 msgr2=0x7f189c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 0x7f189c067f50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f18b00cc0a0 tx=0x7f18a0008460 comp rx=0 tx=0).stop 2023-11-06T03:15:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 msgr2=0x7f18b014d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c7f0 tx=0x7f18a800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 shutdown_connections 2023-11-06T03:15:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f189c065a90 0x7f189c067f50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18b01610c0 0x7f18b00b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18b00b20b0 0x7f18b0163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 --2- 172.21.15.157:0/2862920789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18b00b1c50 0x7f18b014d430 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 >> 172.21.15.157:0/2862920789 conn(0x7f18b00732f0 msgr2=0x7f18b0070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 shutdown_connections 2023-11-06T03:15:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.104+0000 7f18b6a50640 1 -- 172.21.15.157:0/2862920789 wait complete. 2023-11-06T03:15:35.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:15:35.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 Processor -- start 2023-11-06T03:15:35.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 -- start start 2023-11-06T03:15:35.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 0x7fd7080cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd70815ce80 0x7fd7080cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080d2920 con 0x7fd7080ce1a0 2023-11-06T03:15:35.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7080d18a0 con 0x7fd70815ce80 2023-11-06T03:15:35.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70cdce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080d1b60 con 0x7fd70815caa0 2023-11-06T03:15:35.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 0x7fd7080cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd70815ce80 0x7fd7080cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd70815ce80 0x7fd7080cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:46728/0 (socket says 172.21.15.157:46728) 2023-11-06T03:15:35.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd70659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 0x7fd7080cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37464/0 (socket says 172.21.15.157:37464) 2023-11-06T03:15:35.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 -- 172.21.15.157:0/1420064745 learned_addr learned my addr 172.21.15.157:0/1420064745 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:35.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd706d9d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fd7080ce1a0 0x7fd7080d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:37560/0 (socket says 172.21.15.157:37560) 2023-11-06T03:15:35.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 -- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 msgr2=0x7fd7080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 --2- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 0x7fd7080cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 -- 172.21.15.157:0/1420064745 >> v1:172.21.15.195:6789/0 conn(0x7fd7080ce1a0 legacy=0x7fd7080d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.244+0000 7fd6fffff640 1 -- 172.21.15.157:0/1420064745 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080d1e40 con 0x7fd70815ce80 2023-11-06T03:15:35.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd6fffff640 1 --2- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815ce80 0x7fd7080cfb70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52a3316d659fe9c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd705d9b640 1 -- 172.21.15.157:0/1420064745 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00ab620 con 0x7fd70815ce80 2023-11-06T03:15:35.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd705d9b640 1 -- 172.21.15.157:0/1420064745 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00ab780 con 0x7fd70815ce80 2023-11-06T03:15:35.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd705d9b640 1 -- 172.21.15.157:0/1420064745 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00abb10 con 0x7fd70815ce80 2023-11-06T03:15:35.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815ce80 msgr2=0x7fd7080cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815ce80 0x7fd7080cfb70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009870 tx=0x7fd6f001c510 comp rx=0 tx=0).stop 2023-11-06T03:15:35.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- 172.21.15.157:0/1420064745 shutdown_connections 2023-11-06T03:15:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815ce80 0x7fd7080cfb70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- 172.21.15.157:0/1420064745 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815caa0 0x7fd7080cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- 172.21.15.157:0/1420064745 >> 172.21.15.157:0/1420064745 conn(0x7fd708070790 msgr2=0x7fd708070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- 172.21.15.157:0/1420064745 shutdown_connections 2023-11-06T03:15:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- 172.21.15.157:0/1420064745 wait complete. 2023-11-06T03:15:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 Processor -- start 2023-11-06T03:15:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- start start 2023-11-06T03:15:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7080ce1a0 0x7fd70815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815caa0 0x7fd708065b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70814b590 con 0x7fd70814aba0 2023-11-06T03:15:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815c8b0 con 0x7fd70815caa0 2023-11-06T03:15:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd70cdce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080a9640 con 0x7fd7080ce1a0 2023-11-06T03:15:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd706d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd706d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53040/0 (socket says 172.21.15.157:53040) 2023-11-06T03:15:35.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.248+0000 7fd706d9d640 1 -- 172.21.15.157:0/4019819147 learned_addr learned my addr 172.21.15.157:0/4019819147 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:35.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6fffff640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815caa0 0x7fd708065b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70659c640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7080ce1a0 0x7fd70815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 -- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7080ce1a0 msgr2=0x7fd70815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7080ce1a0 0x7fd70815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 -- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815caa0 msgr2=0x7fd708065b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815caa0 0x7fd708065b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 -- 172.21.15.157:0/4019819147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00094d0 con 0x7fd70814aba0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd706d9d640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c7a0 tx=0x7fd70000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000a640 con 0x7fd70814aba0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70000a7a0 con 0x7fd70814aba0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70815c050 con 0x7fd70814aba0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000ab10 con 0x7fd70814aba0 2023-11-06T03:15:35.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70815c5d0 con 0x7fd70814aba0 2023-11-06T03:15:35.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd708072750 con 0x7fd70814aba0 2023-11-06T03:15:35.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd70000ac70 con 0x7fd70814aba0 2023-11-06T03:15:35.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 0x7fd6e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd700082a70 con 0x7fd70814aba0 2023-11-06T03:15:35.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70659c640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 0x7fd6e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.252+0000 7fd70659c640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 0x7fd6e8067f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009af0 tx=0x7fd6f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.256+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7000503b0 con 0x7fd70814aba0 2023-11-06T03:15:35.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.408+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7080c9ab0 con 0x7fd70814aba0 2023-11-06T03:15:35.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.408+0000 7fd6feffd640 1 -- 172.21.15.157:0/4019819147 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7000170a0 con 0x7fd70814aba0 2023-11-06T03:15:35.407 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:15:35.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 msgr2=0x7fd6e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 0x7fd6e8067f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009af0 tx=0x7fd6f8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:35.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 msgr2=0x7fd70814b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c7a0 tx=0x7fd70000cc60 comp rx=0 tx=0).stop 2023-11-06T03:15:35.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 shutdown_connections 2023-11-06T03:15:35.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6e8065a60 0x7fd6e8067f20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd70814aba0 0x7fd70814b050 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd70815caa0 0x7fd708065b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 --2- 172.21.15.157:0/4019819147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd7080ce1a0 0x7fd70815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 >> 172.21.15.157:0/4019819147 conn(0x7fd708070790 msgr2=0x7fd708070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 shutdown_connections 2023-11-06T03:15:35.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.412+0000 7fd70cdce640 1 -- 172.21.15.157:0/4019819147 wait complete. 2023-11-06T03:15:35.426 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:35.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 Processor -- start 2023-11-06T03:15:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 -- start start 2023-11-06T03:15:35.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36640ce800 0x7f36640cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f366406b6a0 0x7f366406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36640cf1a0 con 0x7f3664146370 2023-11-06T03:15:35.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640b2120 con 0x7f36640ce800 2023-11-06T03:15:35.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3669e3f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366406c3d0 con 0x7f366406b6a0 2023-11-06T03:15:35.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36640ce800 0x7f36640cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f36640ce800 0x7f36640cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53048/0 (socket says 172.21.15.157:53048) 2023-11-06T03:15:35.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 -- 172.21.15.157:0/166582337 learned_addr learned my addr 172.21.15.157:0/166582337 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:35.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f3663fff640 1 --2- 172.21.15.157:0/166582337 >> v2:172.21.15.195:3300/0 conn(0x7f366406b6a0 0x7f366406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 -- 172.21.15.157:0/166582337 >> v2:172.21.15.195:3300/0 conn(0x7f366406b6a0 msgr2=0x7f366406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 --2- 172.21.15.157:0/166582337 >> v2:172.21.15.195:3300/0 conn(0x7f366406b6a0 0x7f366406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 -- 172.21.15.157:0/166582337 >> v1:172.21.15.195:6789/0 conn(0x7f3664146370 legacy=0x7f36640b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.552+0000 7f365bfff640 1 -- 172.21.15.157:0/166582337 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366406f5a0 con 0x7f36640ce800 2023-11-06T03:15:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f365bfff640 1 --2- 172.21.15.157:0/166582337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36640ce800 0x7f36640cec60 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a1d4255c578daa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3662ffd640 1 -- 172.21.15.157:0/166582337 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0ab600 con 0x7f36640ce800 2023-11-06T03:15:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3662ffd640 1 -- 172.21.15.157:0/166582337 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0ab760 con 0x7f36640ce800 2023-11-06T03:15:35.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3662ffd640 1 -- 172.21.15.157:0/166582337 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0abaf0 con 0x7f36640ce800 2023-11-06T03:15:35.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- 172.21.15.157:0/166582337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36640ce800 msgr2=0x7f36640cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- 172.21.15.157:0/166582337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36640ce800 0x7f36640cec60 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c01c400 comp rx=0 tx=0).stop 2023-11-06T03:15:35.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- 172.21.15.157:0/166582337 shutdown_connections 2023-11-06T03:15:35.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- 172.21.15.157:0/166582337 >> v2:172.21.15.195:3300/0 conn(0x7f366406b6a0 0x7f366406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- 172.21.15.157:0/166582337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36640ce800 0x7f36640cec60 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- 172.21.15.157:0/166582337 >> 172.21.15.157:0/166582337 conn(0x7f3664065930 msgr2=0x7f3664065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- 172.21.15.157:0/166582337 shutdown_connections 2023-11-06T03:15:35.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- 172.21.15.157:0/166582337 wait complete. 2023-11-06T03:15:35.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 Processor -- start 2023-11-06T03:15:35.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- start start 2023-11-06T03:15:35.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 0x7f366415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3664146370 0x7f366413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b0050 con 0x7f366406b6a0 2023-11-06T03:15:35.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641403f0 con 0x7f36640ce800 2023-11-06T03:15:35.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3669e3f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36641406d0 con 0x7f3664146370 2023-11-06T03:15:35.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53060/0 (socket says 172.21.15.157:53060) 2023-11-06T03:15:35.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 -- 172.21.15.157:0/1428680681 learned_addr learned my addr 172.21.15.157:0/1428680681 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:35.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f365bfff640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 0x7f366415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 -- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3664146370 msgr2=0x7f366413d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3664146370 0x7f366413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 -- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 msgr2=0x7f366415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 0x7f366415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 -- 172.21.15.157:0/1428680681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0094d0 con 0x7f366406b6a0 2023-11-06T03:15:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f365bfff640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 0x7f366415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f36637fe640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f365400c450 tx=0x7f365400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654012830 con 0x7f366406b6a0 2023-11-06T03:15:35.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540129b0 con 0x7f366406b6a0 2023-11-06T03:15:35.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.556+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654012d60 con 0x7f366406b6a0 2023-11-06T03:15:35.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36641409b0 con 0x7f366406b6a0 2023-11-06T03:15:35.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3664140f70 con 0x7f366406b6a0 2023-11-06T03:15:35.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664072750 con 0x7f366406b6a0 2023-11-06T03:15:35.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f365401c460 con 0x7f366406b6a0 2023-11-06T03:15:35.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3660ff9640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 0x7f3644067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3654083260 con 0x7f366406b6a0 2023-11-06T03:15:35.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f365bfff640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 0x7f3644067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.560+0000 7f365bfff640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 0x7f3644067f20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f364c0af000 tx=0x7f364c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.564+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3654050a40 con 0x7f366406b6a0 2023-11-06T03:15:35.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.756+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36640b1b90 con 0x7f366406b6a0 2023-11-06T03:15:35.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.756+0000 7f3660ff9640 1 -- 172.21.15.157:0/1428680681 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7f3654054060 con 0x7f366406b6a0 2023-11-06T03:15:35.754 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:35.754 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:35.754 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:35.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 msgr2=0x7f3644067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 0x7f3644067f20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f364c0af000 tx=0x7f364c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:35.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 msgr2=0x7f366413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f365400c450 tx=0x7f365400c910 comp rx=0 tx=0).stop 2023-11-06T03:15:35.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 shutdown_connections 2023-11-06T03:15:35.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3644065a60 0x7f3644067f20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3664146370 0x7f366413d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36640ce800 0x7f366415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 --2- 172.21.15.157:0/1428680681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f366406b6a0 0x7f366413c630 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 >> 172.21.15.157:0/1428680681 conn(0x7f3664065930 msgr2=0x7f36640ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 shutdown_connections 2023-11-06T03:15:35.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.760+0000 7f3669e3f640 1 -- 172.21.15.157:0/1428680681 wait complete. 2023-11-06T03:15:35.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:35.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 Processor -- start 2023-11-06T03:15:35.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 -- start start 2023-11-06T03:15:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f58880cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58880d0280 0x7f58880b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880d2920 con 0x7f588815ce80 2023-11-06T03:15:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58880d18a0 con 0x7f58880d0280 2023-11-06T03:15:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588f93f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880d1b60 con 0x7f588815caa0 2023-11-06T03:15:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f588815ce80 0x7f58880cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33288/0 (socket says 172.21.15.157:33288) 2023-11-06T03:15:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 -- 172.21.15.157:0/2383005531 learned_addr learned my addr 172.21.15.157:0/2383005531 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588dedc640 1 --2- 172.21.15.157:0/2383005531 >> v2:172.21.15.195:3300/0 conn(0x7f58880d0280 0x7f58880b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588d6db640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f58880cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33726523 0 0) 0x7f58880d2920 con 0x7f588815ce80 2023-11-06T03:15:35.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5874003530 con 0x7f588815ce80 2023-11-06T03:15:35.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172695656 0 0) 0x7f5874003530 con 0x7f588815ce80 2023-11-06T03:15:35.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 msgr2=0x7f58880cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f58880cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 >> v2:172.21.15.195:3300/0 conn(0x7f58880d0280 msgr2=0x7f58880b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 --2- 172.21.15.157:0/2383005531 >> v2:172.21.15.195:3300/0 conn(0x7f58880d0280 0x7f58880b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880d1e40 con 0x7f588815ce80 2023-11-06T03:15:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5878001680 con 0x7f588815ce80 2023-11-06T03:15:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 >> v1:172.21.15.157:6789/0 conn(0x7f588815ce80 legacy=0x7f58880cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5874003980 0x7f5874003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58740047d0 0x7f5874004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f5874005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880d18a0 con 0x7f58740047d0 2023-11-06T03:15:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880d1b60 con 0x7f5874003980 2023-11-06T03:15:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587400b990 con 0x7f588815caa0 2023-11-06T03:15:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58740047d0 0x7f5874004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588d6db640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5874003980 0x7f5874003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588dedc640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f5874005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 -- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 msgr2=0x7f5874005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f588815caa0 0x7f5874005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 -- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5874003980 msgr2=0x7f5874003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5874003980 0x7f5874003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.888+0000 7f588ceda640 1 -- 172.21.15.157:0/2383005531 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5874003530 con 0x7f58740047d0 2023-11-06T03:15:35.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588ceda640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58740047d0 0x7f5874004c50 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f587800c240 tx=0x7f587801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cae54809fbd07f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780a8d00 con 0x7f58740047d0 2023-11-06T03:15:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780a8e60 con 0x7f58740047d0 2023-11-06T03:15:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f587ffff640 1 -- 172.21.15.157:0/2383005531 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878005ae0 con 0x7f58740047d0 2023-11-06T03:15:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- 172.21.15.157:0/2383005531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58740047d0 msgr2=0x7f5874004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 --2- 172.21.15.157:0/2383005531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58740047d0 0x7f5874004c50 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f587800c240 tx=0x7f587801f3d0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588dedc640 1 -- 172.21.15.157:0/2383005531 reap_dead start 2023-11-06T03:15:35.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- 172.21.15.157:0/2383005531 shutdown_connections 2023-11-06T03:15:35.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- 172.21.15.157:0/2383005531 >> 172.21.15.157:0/2383005531 conn(0x7f5888070790 msgr2=0x7f5888070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:35.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- 172.21.15.157:0/2383005531 shutdown_connections 2023-11-06T03:15:35.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- 172.21.15.157:0/2383005531 wait complete. 2023-11-06T03:15:35.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 Processor -- start 2023-11-06T03:15:35.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.892+0000 7f588f93f640 1 -- start start 2023-11-06T03:15:35.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58740047d0 0x7f5888152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 0x7f5888153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588815ec90 con 0x7f588815e2b0 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588815ef70 con 0x7f588815e710 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588815f250 con 0x7f58740047d0 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53078/0 (socket says 172.21.15.157:53078) 2023-11-06T03:15:35.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 -- 172.21.15.157:0/1620413826 learned_addr learned my addr 172.21.15.157:0/1620413826 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:35.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588d6db640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58740047d0 0x7f5888152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588dedc640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 0x7f5888153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 -- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58740047d0 msgr2=0x7f5888152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58740047d0 0x7f5888152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 -- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 msgr2=0x7f5888153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:35.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 0x7f5888153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:35.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 -- 172.21.15.157:0/1620413826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588815f530 con 0x7f588815e2b0 2023-11-06T03:15:35.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588dedc640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 0x7f5888153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:35.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588ceda640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f58780ae040 tx=0x7f5874005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780049f0 con 0x7f588815e2b0 2023-11-06T03:15:35.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780b6070 con 0x7f588815e2b0 2023-11-06T03:15:35.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588815f810 con 0x7f588815e2b0 2023-11-06T03:15:35.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878004d90 con 0x7f588815e2b0 2023-11-06T03:15:35.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5888152940 con 0x7f588815e2b0 2023-11-06T03:15:35.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58881605a0 con 0x7f588815e2b0 2023-11-06T03:15:35.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5878007070 con 0x7f588815e2b0 2023-11-06T03:15:35.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 0x7f586c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:35.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f58780b8070 con 0x7f588815e2b0 2023-11-06T03:15:35.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588d6db640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 0x7f586c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:35.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.896+0000 7f588d6db640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 0x7f586c067f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f58840098d0 tx=0x7f5884008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:35.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:35.900+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f58780eddf0 con 0x7f588815e2b0 2023-11-06T03:15:36.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.056+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5888072750 con 0x7f588815e2b0 2023-11-06T03:15:36.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.056+0000 7f587dffb640 1 -- 172.21.15.157:0/1620413826 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18020 (secure 0 0 0) 0x7f58780f1410 con 0x7f588815e2b0 2023-11-06T03:15:36.053 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:36.054 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":138,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:22.475708+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/1004239715":"2023-11-07T03:15:22.475619+0000","172.21.15.157:6834/1004239715":"2023-11-07T03:15:22.475619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:36.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 msgr2=0x7f586c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 0x7f586c067f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f58840098d0 tx=0x7f5884008040 comp rx=0 tx=0).stop 2023-11-06T03:15:36.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 msgr2=0x7f5888152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f58780ae040 tx=0x7f5874005800 comp rx=0 tx=0).stop 2023-11-06T03:15:36.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 shutdown_connections 2023-11-06T03:15:36.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f586c065a60 0x7f586c067f20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f588815e710 0x7f5888153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588815e2b0 0x7f5888152af0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 --2- 172.21.15.157:0/1620413826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58740047d0 0x7f5888152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 >> 172.21.15.157:0/1620413826 conn(0x7f5888070790 msgr2=0x7f58880cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 shutdown_connections 2023-11-06T03:15:36.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.060+0000 7f588f93f640 1 -- 172.21.15.157:0/1620413826 wait complete. 2023-11-06T03:15:36.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:15:36.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:36.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 Processor -- start 2023-11-06T03:15:36.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 -- start start 2023-11-06T03:15:36.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5f815caa0 0x7fe5f80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80d2920 con 0x7fe5f80d0280 2023-11-06T03:15:36.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80d18a0 con 0x7fe5f815ce80 2023-11-06T03:15:36.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5fd08f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80d1b60 con 0x7fe5f815caa0 2023-11-06T03:15:36.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f759e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe5f80d0280 0x7fe5f80b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33292/0 (socket says 172.21.15.157:33292) 2023-11-06T03:15:36.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f759e640 1 -- 172.21.15.157:0/203902600 learned_addr learned my addr 172.21.15.157:0/203902600 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:36.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884400242 0 0) 0x7fe5f80d2920 con 0x7fe5f80d0280 2023-11-06T03:15:36.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e0003530 con 0x7fe5f80d0280 2023-11-06T03:15:36.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/203902600 >> v2:172.21.15.195:3300/0 conn(0x7fe5f815caa0 0x7fe5f80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1173239065 0 0) 0x7fe5e0003530 con 0x7fe5f80d0280 2023-11-06T03:15:36.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 >> v2:172.21.15.195:3300/0 conn(0x7fe5f815caa0 msgr2=0x7fe5f80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 --2- 172.21.15.157:0/203902600 >> v2:172.21.15.195:3300/0 conn(0x7fe5f815caa0 0x7fe5f80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 >> v1:172.21.15.195:6789/0 conn(0x7fe5f815ce80 legacy=0x7fe5f80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80d1e40 con 0x7fe5f80d0280 2023-11-06T03:15:36.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe5e8001a00 con 0x7fe5f80d0280 2023-11-06T03:15:36.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 >> v1:172.21.15.157:6789/0 conn(0x7fe5f80d0280 legacy=0x7fe5f80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0003980 0x7fe5e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 0x7fe5e0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0005190 0x7fe5e000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80d1b60 con 0x7fe5e00047d0 2023-11-06T03:15:36.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e000c0e0 con 0x7fe5e0003980 2023-11-06T03:15:36.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e000c2a0 con 0x7fe5e0005190 2023-11-06T03:15:36.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 0x7fe5e0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0003980 0x7fe5e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 -- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0005190 msgr2=0x7fe5e000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5e0005190 0x7fe5e000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 -- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0003980 msgr2=0x7fe5e0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0003980 0x7fe5e0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f659c640 1 -- 172.21.15.157:0/203902600 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0003530 con 0x7fe5e00047d0 2023-11-06T03:15:36.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f759e640 1 -- 172.21.15.157:0/203902600 reap_dead start 2023-11-06T03:15:36.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.184+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e0003980 0x7fe5e0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:36.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f659c640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 0x7fe5e0004c50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec001750 tx=0x7fe5ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34a13dfdb9875e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0052d0 con 0x7fe5e00047d0 2023-11-06T03:15:36.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec0a7050 con 0x7fe5e00047d0 2023-11-06T03:15:36.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f5d9b640 1 -- 172.21.15.157:0/203902600 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003600 con 0x7fe5e00047d0 2023-11-06T03:15:36.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 msgr2=0x7fe5e0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 0x7fe5e0004c50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec001750 tx=0x7fe5ec01c3c0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/203902600 shutdown_connections 2023-11-06T03:15:36.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/203902600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5e00047d0 0x7fe5e0004c50 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/203902600 >> 172.21.15.157:0/203902600 conn(0x7fe5f8070790 msgr2=0x7fe5f8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/203902600 shutdown_connections 2023-11-06T03:15:36.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/203902600 wait complete. 2023-11-06T03:15:36.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 Processor -- start 2023-11-06T03:15:36.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- start start 2023-11-06T03:15:36.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e00047d0 0x7fe5f81f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5f815fc70 0x7fe5f815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815d4a0 con 0x7fe5f815ea80 2023-11-06T03:15:36.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815d780 con 0x7fe5e00047d0 2023-11-06T03:15:36.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815da60 con 0x7fe5f815fc70 2023-11-06T03:15:36.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.196 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53102/0 (socket says 172.21.15.157:53102) 2023-11-06T03:15:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 -- 172.21.15.157:0/3664799296 learned_addr learned my addr 172.21.15.157:0/3664799296 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e00047d0 0x7fe5f81f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 -- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5f815fc70 msgr2=0x7fe5f815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5f815fc70 0x7fe5f815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 -- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e00047d0 msgr2=0x7fe5f81f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e00047d0 0x7fe5f81f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 -- 172.21.15.157:0/3664799296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f815dd40 con 0x7fe5f815ea80 2023-11-06T03:15:36.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5f759e640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b49b0 tx=0x7fe5e800fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800d9c0 con 0x7fe5f815ea80 2023-11-06T03:15:36.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e800db40 con 0x7fe5f815ea80 2023-11-06T03:15:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.188+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f815e020 con 0x7fe5f815ea80 2023-11-06T03:15:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800a030 con 0x7fe5f815ea80 2023-11-06T03:15:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f80b6a00 con 0x7fe5f815ea80 2023-11-06T03:15:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f8072750 con 0x7fe5f815ea80 2023-11-06T03:15:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe5e800a190 con 0x7fe5f815ea80 2023-11-06T03:15:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5df7fe640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 0x7fe5e4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe5e8085ce0 con 0x7fe5f815ea80 2023-11-06T03:15:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 0x7fe5e4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.192+0000 7fe5f6d9d640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 0x7fe5e4067f50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80cbd70 tx=0x7fe5f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.196+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe5e8053450 con 0x7fe5f815ea80 2023-11-06T03:15:36.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.368+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f80c9ab0 con 0x7fe5f815ea80 2023-11-06T03:15:36.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5df7fe640 1 -- 172.21.15.157:0/3664799296 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7fe5e8056a70 con 0x7fe5f815ea80 2023-11-06T03:15:36.368 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:36.368 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:36.369 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:36.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 msgr2=0x7fe5e4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 0x7fe5e4067f50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80cbd70 tx=0x7fe5f0008040 comp rx=0 tx=0).stop 2023-11-06T03:15:36.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 msgr2=0x7fe5f80b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80b49b0 tx=0x7fe5e800fcf0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 shutdown_connections 2023-11-06T03:15:36.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5e4065a90 0x7fe5e4067f50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5f815ea80 0x7fe5f80b64c0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5f815fc70 0x7fe5f815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 --2- 172.21.15.157:0/3664799296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5e00047d0 0x7fe5f81f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 >> 172.21.15.157:0/3664799296 conn(0x7fe5f8070790 msgr2=0x7fe5f815c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 shutdown_connections 2023-11-06T03:15:36.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.372+0000 7fe5fd08f640 1 -- 172.21.15.157:0/3664799296 wait complete. 2023-11-06T03:15:36.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:36.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 Processor -- start 2023-11-06T03:15:36.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 -- start start 2023-11-06T03:15:36.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e00b2af0 0x7f45e00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f45e006d000 0x7f45e0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e015e440 con 0x7f45e01599c0 2023-11-06T03:15:36.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e006c240 con 0x7f45e00b2af0 2023-11-06T03:15:36.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45e5923640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e006c520 con 0x7f45e006d000 2023-11-06T03:15:36.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e00b2af0 0x7f45e00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f45e00b2af0 0x7f45e00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53112/0 (socket says 172.21.15.157:53112) 2023-11-06T03:15:36.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 -- 172.21.15.157:0/630498728 learned_addr learned my addr 172.21.15.157:0/630498728 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:36.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45df7fe640 1 --2- 172.21.15.157:0/630498728 >> v2:172.21.15.195:3300/0 conn(0x7f45e006d000 0x7f45e0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 -- 172.21.15.157:0/630498728 >> v2:172.21.15.195:3300/0 conn(0x7f45e006d000 msgr2=0x7f45e0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 --2- 172.21.15.157:0/630498728 >> v2:172.21.15.195:3300/0 conn(0x7f45e006d000 0x7f45e0159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 -- 172.21.15.157:0/630498728 >> v1:172.21.15.195:6789/0 conn(0x7f45e01599c0 legacy=0x7f45e00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.508+0000 7f45de7fc640 1 -- 172.21.15.157:0/630498728 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e006c800 con 0x7f45e00b2af0 2023-11-06T03:15:36.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- 172.21.15.157:0/630498728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e00ae000 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f45e006d930 tx=0x7f45c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5624d75940da6b55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45ddffb640 1 -- 172.21.15.157:0/630498728 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80aa610 con 0x7f45e00b2af0 2023-11-06T03:15:36.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45ddffb640 1 -- 172.21.15.157:0/630498728 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80a6040 con 0x7f45e00b2af0 2023-11-06T03:15:36.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45ddffb640 1 -- 172.21.15.157:0/630498728 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80aa990 con 0x7f45e00b2af0 2023-11-06T03:15:36.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/630498728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 msgr2=0x7f45e00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- 172.21.15.157:0/630498728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e00ae000 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f45e006d930 tx=0x7f45c801c490 comp rx=0 tx=0).stop 2023-11-06T03:15:36.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/630498728 shutdown_connections 2023-11-06T03:15:36.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- 172.21.15.157:0/630498728 >> v2:172.21.15.195:3300/0 conn(0x7f45e006d000 0x7f45e0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- 172.21.15.157:0/630498728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e00ae000 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/630498728 >> 172.21.15.157:0/630498728 conn(0x7f45e0065930 msgr2=0x7f45e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/630498728 shutdown_connections 2023-11-06T03:15:36.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/630498728 wait complete. 2023-11-06T03:15:36.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 Processor -- start 2023-11-06T03:15:36.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- start start 2023-11-06T03:15:36.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e006d000 0x7f45e0157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e01599c0 0x7f45e0151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0154570 con 0x7f45e00b2af0 2023-11-06T03:15:36.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0158350 con 0x7f45e01599c0 2023-11-06T03:15:36.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01585a0 con 0x7f45e006d000 2023-11-06T03:15:36.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53120/0 (socket says 172.21.15.157:53120) 2023-11-06T03:15:36.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 -- 172.21.15.157:0/1628637214 learned_addr learned my addr 172.21.15.157:0/1628637214 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:36.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45df7fe640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e01599c0 0x7f45e0151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45deffd640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e006d000 0x7f45e0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 -- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e006d000 msgr2=0x7f45e0157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e006d000 0x7f45e0157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 -- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e01599c0 msgr2=0x7f45e0151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e01599c0 0x7f45e0151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 -- 172.21.15.157:0/1628637214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80094d0 con 0x7f45e00b2af0 2023-11-06T03:15:36.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45de7fc640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a0c0 tx=0x7f45c801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80bd5f0 con 0x7f45e00b2af0 2023-11-06T03:15:36.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80bd750 con 0x7f45e00b2af0 2023-11-06T03:15:36.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0166310 con 0x7f45e00b2af0 2023-11-06T03:15:36.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80bdb00 con 0x7f45e00b2af0 2023-11-06T03:15:36.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.512+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e01668d0 con 0x7f45e00b2af0 2023-11-06T03:15:36.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0152240 con 0x7f45e00b2af0 2023-11-06T03:15:36.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45c80b33f0 con 0x7f45e00b2af0 2023-11-06T03:15:36.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45d7fff640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 0x7f45c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f45c811e2c0 con 0x7f45e00b2af0 2023-11-06T03:15:36.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45deffd640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 0x7f45c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.516+0000 7f45deffd640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 0x7f45c0067f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f45e00ace70 tx=0x7f45d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.520+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f45c80eba20 con 0x7f45e00b2af0 2023-11-06T03:15:36.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.696+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e0166ac0 con 0x7f45e00b2af0 2023-11-06T03:15:36.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.696+0000 7f45d7fff640 1 -- 172.21.15.157:0/1628637214 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3905 (secure 0 0 0) 0x7f45c80ef040 con 0x7f45e00b2af0 2023-11-06T03:15:36.694 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:36.695 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":104,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:14:35.005631+0000","modified":"2023-11-06T03:15:29.639284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6455":{"gid":6455,"name":"b","rank":0,"incarnation":100,"state":"up:active","state_seq":17,"addr":"172.21.15.195:6837/1960706313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":1960706313},{"type":"v1","addr":"172.21.15.195:6837","nonce":1960706313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-11-06T03:15:36.695 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 104 2023-11-06T03:15:36.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 msgr2=0x7f45c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 0x7f45c0067f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f45e00ace70 tx=0x7f45d8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 msgr2=0x7f45e0151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a0c0 tx=0x7f45c801acd0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 shutdown_connections 2023-11-06T03:15:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45c0065a60 0x7f45c0067f20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e01599c0 0x7f45e0151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45e00b2af0 0x7f45e0151000 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 --2- 172.21.15.157:0/1628637214 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45e006d000 0x7f45e0157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 >> 172.21.15.157:0/1628637214 conn(0x7f45e0065930 msgr2=0x7f45e0070890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 shutdown_connections 2023-11-06T03:15:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.700+0000 7f45e5923640 1 -- 172.21.15.157:0/1628637214 wait complete. 2023-11-06T03:15:36.713 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:36.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.824+0000 7f9e81c7d640 1 Processor -- start 2023-11-06T03:15:36.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.824+0000 7f9e81c7d640 1 -- start start 2023-11-06T03:15:36.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 0x7f9e7c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15ce80 0x7f9e7c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c0d2920 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c0d18a0 con 0x7f9e7c15caa0 2023-11-06T03:15:36.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e81c7d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0d1b60 con 0x7f9e7c15ce80 2023-11-06T03:15:36.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53134/0 (socket says 172.21.15.157:53134) 2023-11-06T03:15:36.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 -- 172.21.15.157:0/70901937 learned_addr learned my addr 172.21.15.157:0/70901937 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:36.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7b7fe640 1 --2- 172.21.15.157:0/70901937 >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 0x7f9e7c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e73fff640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15ce80 0x7f9e7c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 -- 172.21.15.157:0/70901937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15ce80 msgr2=0x7f9e7c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15ce80 0x7f9e7c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 -- 172.21.15.157:0/70901937 >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 msgr2=0x7f9e7c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:36.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 --2- 172.21.15.157:0/70901937 >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 0x7f9e7c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 -- 172.21.15.157:0/70901937 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c0d1e40 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7b7fe640 1 --2- 172.21.15.157:0/70901937 >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 0x7f9e7c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:36.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7bfff640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0b4f80 tx=0x7f9e7401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cda4af70ba3502 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7affd640 1 -- 172.21.15.157:0/70901937 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740a5820 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7affd640 1 -- 172.21.15.157:0/70901937 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e740a5980 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.828+0000 7f9e7affd640 1 -- 172.21.15.157:0/70901937 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740a5d10 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/70901937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 msgr2=0x7f9e7c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0b4f80 tx=0x7f9e7401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/70901937 shutdown_connections 2023-11-06T03:15:36.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c0d00b0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/70901937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15ce80 0x7f9e7c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/70901937 >> v2:172.21.15.195:3300/0 conn(0x7f9e7c15caa0 0x7f9e7c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/70901937 >> 172.21.15.157:0/70901937 conn(0x7f9e7c070790 msgr2=0x7f9e7c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/70901937 shutdown_connections 2023-11-06T03:15:36.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/70901937 wait complete. 2023-11-06T03:15:36.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 Processor -- start 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- start start 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15caa0 0x7f9e7c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 0x7f9e7c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1585f0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c15f960 con 0x7f9e7c15ce80 2023-11-06T03:15:36.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0c6e90 con 0x7f9e7c15caa0 2023-11-06T03:15:36.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53142/0 (socket says 172.21.15.157:53142) 2023-11-06T03:15:36.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 -- 172.21.15.157:0/3811760487 learned_addr learned my addr 172.21.15.157:0/3811760487 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:36.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7bfff640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 0x7f9e7c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e73fff640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15caa0 0x7f9e7c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 -- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15caa0 msgr2=0x7f9e7c14d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15caa0 0x7f9e7c14d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 -- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 msgr2=0x7f9e7c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 0x7f9e7c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 -- 172.21.15.157:0/3811760487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e74009b30 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7bfff640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 0x7f9e7c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:36.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e7b7fe640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c7c0 tx=0x7f9e6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0079d0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c007b30 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c0c68d0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00ce70 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c15f5f0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.832+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c072750 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.836+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9e6c0113f0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.836+0000 7f9e78ff9640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 0x7f9e60067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:36.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.836+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9e6c083390 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.836+0000 7f9e73fff640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 0x7f9e60067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:36.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.836+0000 7f9e73fff640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 0x7f9e60067f50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0b36a0 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:36.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.840+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9e6c050c20 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.984+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e7c0b51f0 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e78ff9640 1 -- 172.21.15.157:0/3811760487 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18020 (secure 0 0 0) 0x7f9e6c054240 con 0x7f9e7c0ce1a0 2023-11-06T03:15:36.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:36.984 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":138,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:22.475708+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:14:33.607794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-06T03:14:33.978137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/1004239715":"2023-11-07T03:15:22.475619+0000","172.21.15.157:6834/1004239715":"2023-11-07T03:15:22.475619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 msgr2=0x7f9e60067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 0x7f9e60067f50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0b36a0 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2023-11-06T03:15:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 msgr2=0x7f9e7c1580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00c7c0 tx=0x7f9e6c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:15:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 shutdown_connections 2023-11-06T03:15:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9e60065a90 0x7f9e60067f50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9e7c15ce80 0x7f9e7c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9e7c15caa0 0x7f9e7c14d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 --2- 172.21.15.157:0/3811760487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e7c0ce1a0 0x7f9e7c1580b0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:36.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 >> 172.21.15.157:0/3811760487 conn(0x7f9e7c070790 msgr2=0x7f9e7c0cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:36.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 shutdown_connections 2023-11-06T03:15:36.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:36.988+0000 7f9e81c7d640 1 -- 172.21.15.157:0/3811760487 wait complete. 2023-11-06T03:15:37.002 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:15:37.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 Processor -- start 2023-11-06T03:15:37.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 -- start start 2023-11-06T03:15:37.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f867c15ea60 0x7f867c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c15f580 con 0x7f867c1551e0 2023-11-06T03:15:37.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c0704b0 con 0x7f867c15ea60 2023-11-06T03:15:37.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f8681d61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15efc0 con 0x7f867c15e5e0 2023-11-06T03:15:37.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f867c15ea60 0x7f867c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867affd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867b7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f867c1551e0 0x7f867c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:37600/0 (socket says 172.21.15.157:37600) 2023-11-06T03:15:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867b7fe640 1 -- 172.21.15.157:0/691912983 learned_addr learned my addr 172.21.15.157:0/691912983 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867bfff640 1 -- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 msgr2=0x7f867c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867bfff640 1 --2- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867bfff640 1 -- 172.21.15.157:0/691912983 >> v1:172.21.15.195:6789/0 conn(0x7f867c1551e0 legacy=0x7f867c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.128+0000 7f867bfff640 1 -- 172.21.15.157:0/691912983 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15f2a0 con 0x7f867c15ea60 2023-11-06T03:15:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f867bfff640 1 --2- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c15ea60 0x7f867c0b3950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f866c009d90 tx=0x7f866c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd1468c8b9c0f2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f867a7fc640 1 -- 172.21.15.157:0/691912983 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0a5950 con 0x7f867c15ea60 2023-11-06T03:15:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f867a7fc640 1 -- 172.21.15.157:0/691912983 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0a5ab0 con 0x7f867c15ea60 2023-11-06T03:15:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f867a7fc640 1 -- 172.21.15.157:0/691912983 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0a5e40 con 0x7f867c15ea60 2023-11-06T03:15:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c15ea60 msgr2=0x7f867c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 --2- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c15ea60 0x7f867c0b3950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f866c009d90 tx=0x7f866c01c890 comp rx=0 tx=0).stop 2023-11-06T03:15:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- 172.21.15.157:0/691912983 shutdown_connections 2023-11-06T03:15:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 --2- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c15ea60 0x7f867c0b3950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 --2- 172.21.15.157:0/691912983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- 172.21.15.157:0/691912983 >> 172.21.15.157:0/691912983 conn(0x7f867c065930 msgr2=0x7f867c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- 172.21.15.157:0/691912983 shutdown_connections 2023-11-06T03:15:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- 172.21.15.157:0/691912983 wait complete. 2023-11-06T03:15:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 Processor -- start 2023-11-06T03:15:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.132+0000 7f8681d61640 1 -- start start 2023-11-06T03:15:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c1551e0 0x7f867c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c1593e0 con 0x7f867c15ea60 2023-11-06T03:15:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0afd50 con 0x7f867c1551e0 2023-11-06T03:15:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0b0030 con 0x7f867c15e5e0 2023-11-06T03:15:37.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53158/0 (socket says 172.21.15.157:53158) 2023-11-06T03:15:37.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 -- 172.21.15.157:0/346613081 learned_addr learned my addr 172.21.15.157:0/346613081 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:37.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867affd640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867b7fe640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c1551e0 0x7f867c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 -- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 msgr2=0x7f867c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 -- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c1551e0 msgr2=0x7f867c14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c1551e0 0x7f867c14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 -- 172.21.15.157:0/346613081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c009a70 con 0x7f867c15ea60 2023-11-06T03:15:37.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867bfff640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f866c01a4c0 tx=0x7f866c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c007000 con 0x7f867c15ea60 2023-11-06T03:15:37.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0aede0 con 0x7f867c15ea60 2023-11-06T03:15:37.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c0b0310 con 0x7f867c15ea60 2023-11-06T03:15:37.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0b7650 con 0x7f867c15ea60 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c155b00 con 0x7f867c15ea60 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c158ed0 con 0x7f867c15ea60 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f866c0b4050 con 0x7f867c15ea60 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 0x7f865c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f866c11e600 con 0x7f867c15ea60 2023-11-06T03:15:37.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.136+0000 7f867b7fe640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 0x7f865c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.140+0000 7f867b7fe640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 0x7f865c067f20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f867c0b2e60 tx=0x7f8674009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.140+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f867c158ed0 con 0x7f867c15ea60 2023-11-06T03:15:37.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.312+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f867c158ed0 con 0x7f867c15ea60 2023-11-06T03:15:37.313 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:15:37.316+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:15:37.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8673fff640 1 -- 172.21.15.157:0/346613081 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f867c158ed0 con 0x7f867c15ea60 2023-11-06T03:15:37.332 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:15:37.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 msgr2=0x7f865c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 0x7f865c067f20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f867c0b2e60 tx=0x7f8674009040 comp rx=0 tx=0).stop 2023-11-06T03:15:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 msgr2=0x7f867c1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f866c01a4c0 tx=0x7f866c019630 comp rx=0 tx=0).stop 2023-11-06T03:15:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 shutdown_connections 2023-11-06T03:15:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f865c065a60 0x7f865c067f20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f867c15ea60 0x7f867c1555c0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f867c15e5e0 0x7f867c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 --2- 172.21.15.157:0/346613081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f867c1551e0 0x7f867c14df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 >> 172.21.15.157:0/346613081 conn(0x7f867c065930 msgr2=0x7f867c15de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:37.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 shutdown_connections 2023-11-06T03:15:37.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.336+0000 7f8681d61640 1 -- 172.21.15.157:0/346613081 wait complete. 2023-11-06T03:15:37.350 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:15:37.411 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:15:37.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 Processor -- start 2023-11-06T03:15:37.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 -- start start 2023-11-06T03:15:37.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd6c01599c0 0x7fd6c00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6c006d000 0x7fd6c00b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c015e440 con 0x7fd6c015dea0 2023-11-06T03:15:37.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c015adb0 con 0x7fd6c006d000 2023-11-06T03:15:37.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6c53f9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c006c240 con 0x7fd6c01599c0 2023-11-06T03:15:37.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6be7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6c006d000 0x7fd6c00b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6be7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6c006d000 0x7fd6c00b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53162/0 (socket says 172.21.15.157:53162) 2023-11-06T03:15:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.476+0000 7fd6be7fc640 1 -- 172.21.15.157:0/1074997853 learned_addr learned my addr 172.21.15.157:0/1074997853 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6beffd640 1 --2- 172.21.15.157:0/1074997853 >> v2:172.21.15.195:3300/0 conn(0x7fd6c01599c0 0x7fd6c00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6be7fc640 1 -- 172.21.15.157:0/1074997853 >> v2:172.21.15.195:3300/0 conn(0x7fd6c01599c0 msgr2=0x7fd6c00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6be7fc640 1 --2- 172.21.15.157:0/1074997853 >> v2:172.21.15.195:3300/0 conn(0x7fd6c01599c0 0x7fd6c00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6be7fc640 1 -- 172.21.15.157:0/1074997853 >> v1:172.21.15.195:6789/0 conn(0x7fd6c015dea0 legacy=0x7fd6c0070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:37.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6be7fc640 1 -- 172.21.15.157:0/1074997853 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c006c520 con 0x7fd6c006d000 2023-11-06T03:15:37.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6be7fc640 1 --2- 172.21.15.157:0/1074997853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c006d000 0x7fd6c00b23e0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0066800 tx=0x7fd6b401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6494220ee076cd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6bdffb640 1 -- 172.21.15.157:0/1074997853 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40ac600 con 0x7fd6c006d000 2023-11-06T03:15:37.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6bdffb640 1 -- 172.21.15.157:0/1074997853 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b40a8050 con 0x7fd6c006d000 2023-11-06T03:15:37.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.480+0000 7fd6bdffb640 1 -- 172.21.15.157:0/1074997853 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40ac980 con 0x7fd6c006d000 2023-11-06T03:15:37.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/1074997853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c006d000 msgr2=0x7fd6c00b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/1074997853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c006d000 0x7fd6c00b23e0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0066800 tx=0x7fd6b401c490 comp rx=0 tx=0).stop 2023-11-06T03:15:37.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/1074997853 shutdown_connections 2023-11-06T03:15:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/1074997853 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c006d000 0x7fd6c00b23e0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/1074997853 >> v2:172.21.15.195:3300/0 conn(0x7fd6c01599c0 0x7fd6c00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/1074997853 >> 172.21.15.157:0/1074997853 conn(0x7fd6c0065930 msgr2=0x7fd6c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/1074997853 shutdown_connections 2023-11-06T03:15:37.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/1074997853 wait complete. 2023-11-06T03:15:37.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 Processor -- start 2023-11-06T03:15:37.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- start start 2023-11-06T03:15:37.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 0x7fd6c00cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c015dea0 0x7fd6c014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01502e0 con 0x7fd6c01599c0 2023-11-06T03:15:37.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0150530 con 0x7fd6c015dea0 2023-11-06T03:15:37.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0150810 con 0x7fd6c006d000 2023-11-06T03:15:37.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53168/0 (socket says 172.21.15.157:53168) 2023-11-06T03:15:37.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 -- 172.21.15.157:0/541519746 learned_addr learned my addr 172.21.15.157:0/541519746 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:37.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6beffd640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 0x7fd6c00cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 -- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 msgr2=0x7fd6c00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 0x7fd6c00cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6bf7fe640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c015dea0 0x7fd6c014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 -- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c015dea0 msgr2=0x7fd6c014ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:37.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c015dea0 0x7fd6c014ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:37.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 -- 172.21.15.157:0/541519746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40094d0 con 0x7fd6c01599c0 2023-11-06T03:15:37.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6beffd640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 0x7fd6c00cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:37.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6be7fc640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401a0c0 tx=0x7fd6b401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40bc070 con 0x7fd6c01599c0 2023-11-06T03:15:37.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b40ace70 con 0x7fd6c01599c0 2023-11-06T03:15:37.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0150af0 con 0x7fd6c01599c0 2023-11-06T03:15:37.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40b7650 con 0x7fd6c01599c0 2023-11-06T03:15:37.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.484+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c014b710 con 0x7fd6c01599c0 2023-11-06T03:15:37.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c0072750 con 0x7fd6c01599c0 2023-11-06T03:15:37.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd6b40b3070 con 0x7fd6c01599c0 2023-11-06T03:15:37.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6a77fe640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 0x7fd6ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:37.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fd6b411ce70 con 0x7fd6c01599c0 2023-11-06T03:15:37.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6beffd640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 0x7fd6ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:37.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.488+0000 7fd6beffd640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 0x7fd6ac067f20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:37.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.492+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd6b40ea620 con 0x7fd6c01599c0 2023-11-06T03:15:37.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:37.664+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6c00cb110 con 0x7fd6c01599c0 2023-11-06T03:15:38.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.348+0000 7fd6a77fe640 1 -- 172.21.15.157:0/541519746 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd6b40edc40 con 0x7fd6c01599c0 2023-11-06T03:15:38.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 msgr2=0x7fd6ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 0x7fd6ac067f20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0008040 comp rx=0 tx=0).stop 2023-11-06T03:15:38.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 msgr2=0x7fd6c014b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401a0c0 tx=0x7fd6b401acd0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 shutdown_connections 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6ac065a60 0x7fd6ac067f20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6c015dea0 0x7fd6c014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6c01599c0 0x7fd6c014b1d0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 --2- 172.21.15.157:0/541519746 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6c006d000 0x7fd6c00cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 >> 172.21.15.157:0/541519746 conn(0x7fd6c0065930 msgr2=0x7fd6c00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:38.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 shutdown_connections 2023-11-06T03:15:38.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.352+0000 7fd6c53f9640 1 -- 172.21.15.157:0/541519746 wait complete. 2023-11-06T03:15:38.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:15:38.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 Processor -- start 2023-11-06T03:15:38.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 -- start start 2023-11-06T03:15:38.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f3258154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f325815b860 0x7f3258155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32580cb100 con 0x7f325806c550 2023-11-06T03:15:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325815bcf0 con 0x7f325815b860 2023-11-06T03:15:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325cea3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580ce0a0 con 0x7f325806c170 2023-11-06T03:15:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f325815b860 0x7f3258155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f325815b860 0x7f3258155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53174/0 (socket says 172.21.15.157:53174) 2023-11-06T03:15:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 -- 172.21.15.157:0/689516049 learned_addr learned my addr 172.21.15.157:0/689516049 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3255d9b640 1 --1- 172.21.15.157:0/689516049 >> v1:172.21.15.157:6789/0 conn(0x7f325806c550 0x7f325815b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33300/0 (socket says 172.21.15.157:33300) 2023-11-06T03:15:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325659c640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f3258154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 -- 172.21.15.157:0/689516049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 msgr2=0x7f3258154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:38.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f3258154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f325559a640 1 -- 172.21.15.157:0/689516049 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241580887 0 0) 0x7f32580cb100 con 0x7f325806c550 2023-11-06T03:15:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 -- 172.21.15.157:0/689516049 >> v1:172.21.15.157:6789/0 conn(0x7f325806c550 legacy=0x7f325815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.504+0000 7f3256d9d640 1 -- 172.21.15.157:0/689516049 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325815bef0 con 0x7f325815b860 2023-11-06T03:15:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f3256d9d640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325815b860 0x7f3258155140 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f3250009e50 tx=0x7f325001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7087fc651d785fde server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:38.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325559a640 1 -- 172.21.15.157:0/689516049 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500a58a0 con 0x7f325815b860 2023-11-06T03:15:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325559a640 1 -- 172.21.15.157:0/689516049 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32500a5a00 con 0x7f325815b860 2023-11-06T03:15:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325559a640 1 -- 172.21.15.157:0/689516049 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500a5d90 con 0x7f325815b860 2023-11-06T03:15:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- 172.21.15.157:0/689516049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325815b860 msgr2=0x7f3258155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325815b860 0x7f3258155140 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f3250009e50 tx=0x7f325001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- 172.21.15.157:0/689516049 shutdown_connections 2023-11-06T03:15:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325815b860 0x7f3258155140 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 --2- 172.21.15.157:0/689516049 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f3258154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- 172.21.15.157:0/689516049 >> 172.21.15.157:0/689516049 conn(0x7f32580b43b0 msgr2=0x7f3258065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:38.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- 172.21.15.157:0/689516049 shutdown_connections 2023-11-06T03:15:38.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- 172.21.15.157:0/689516049 wait complete. 2023-11-06T03:15:38.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 Processor -- start 2023-11-06T03:15:38.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.508+0000 7f325cea3640 1 -- start start 2023-11-06T03:15:38.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f325814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f325815b860 0x7f325806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325814a470 con 0x7f325806c550 2023-11-06T03:15:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325814a750 con 0x7f325815b860 2023-11-06T03:15:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325814aa30 con 0x7f325806c170 2023-11-06T03:15:38.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53180/0 (socket says 172.21.15.157:53180) 2023-11-06T03:15:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 -- 172.21.15.157:0/2683090834 learned_addr learned my addr 172.21.15.157:0/2683090834 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3256d9d640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f325815b860 0x7f325806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325659c640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f325814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 -- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 msgr2=0x7f325814bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f325814bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 -- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f325815b860 msgr2=0x7f325806fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f325815b860 0x7f325806fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 -- 172.21.15.157:0/2683090834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3250009b30 con 0x7f325806c550 2023-11-06T03:15:38.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3255d9b640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f324c0011f0 tx=0x7f324c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c0074e0 con 0x7f325806c550 2023-11-06T03:15:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c007640 con 0x7f325806c550 2023-11-06T03:15:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c004a10 con 0x7f325806c550 2023-11-06T03:15:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32580cbfb0 con 0x7f325806c550 2023-11-06T03:15:38.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325814aff0 con 0x7f325806c550 2023-11-06T03:15:38.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325804e050 con 0x7f325806c550 2023-11-06T03:15:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f324c004b70 con 0x7f325806c550 2023-11-06T03:15:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 0x7f3238067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f324c082740 con 0x7f325806c550 2023-11-06T03:15:38.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.512+0000 7f325659c640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 0x7f3238067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.516+0000 7f325659c640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 0x7f3238067f20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f324800b9d0 tx=0x7f3248008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:38.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.516+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f324c04fe60 con 0x7f325806c550 2023-11-06T03:15:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:38.672+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32580ccbb0 con 0x7f325806c550 2023-11-06T03:15:39.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.356+0000 7f3246ffd640 1 -- 172.21.15.157:0/2683090834 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7f324c018020 con 0x7f325806c550 2023-11-06T03:15:39.354 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:15:39.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 msgr2=0x7f3238067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:39.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 0x7f3238067f20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f324800b9d0 tx=0x7f3248008040 comp rx=0 tx=0).stop 2023-11-06T03:15:39.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 msgr2=0x7f3258144df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:39.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f324c0011f0 tx=0x7f324c00c9b0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 shutdown_connections 2023-11-06T03:15:39.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3238065a60 0x7f3238067f20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f325815b860 0x7f325806fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f325806c550 0x7f3258144df0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 --2- 172.21.15.157:0/2683090834 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f325806c170 0x7f325814bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 >> 172.21.15.157:0/2683090834 conn(0x7f32580b43b0 msgr2=0x7f3258065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:39.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 shutdown_connections 2023-11-06T03:15:39.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.360+0000 7f325cea3640 1 -- 172.21.15.157:0/2683090834 wait complete. 2023-11-06T03:15:39.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:15:39.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.512+0000 7fb2f9e7d640 1 Processor -- start 2023-11-06T03:15:39.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.512+0000 7fb2f9e7d640 1 -- start start 2023-11-06T03:15:39.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f9e7d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb2f415cc10 0x7fb2f406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f9e7d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2f40b3f70 0x7fb2f40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f9e7d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f40adc00 con 0x7fb2f40ce0d0 2023-11-06T03:15:39.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f9e7d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40ae880 con 0x7fb2f40b3f70 2023-11-06T03:15:39.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f9e7d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40b43e0 con 0x7fb2f415cc10 2023-11-06T03:15:39.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2f40b3f70 0x7fb2f40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2f40b3f70 0x7fb2f40ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38026/0 (socket says 172.21.15.157:38026) 2023-11-06T03:15:39.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1767195129 learned_addr learned my addr 172.21.15.157:0/1767195129 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:39.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f37fe640 1 --2- 172.21.15.157:0/1767195129 >> v2:172.21.15.195:3300/0 conn(0x7fb2f415cc10 0x7fb2f406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1767195129 >> v2:172.21.15.195:3300/0 conn(0x7fb2f415cc10 msgr2=0x7fb2f406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:39.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 --2- 172.21.15.157:0/1767195129 >> v2:172.21.15.195:3300/0 conn(0x7fb2f415cc10 0x7fb2f406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1767195129 >> v1:172.21.15.195:6789/0 conn(0x7fb2f40ce0d0 legacy=0x7fb2f40b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:39.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1767195129 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40adee0 con 0x7fb2f40b3f70 2023-11-06T03:15:39.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f3fff640 1 --2- 172.21.15.157:0/1767195129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f40b3f70 0x7fb2f40ce560 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b45b0 tx=0x7fb2ec01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901aadca4ea73b65 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:39.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f27fc640 1 -- 172.21.15.157:0/1767195129 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0a56b0 con 0x7fb2f40b3f70 2023-11-06T03:15:39.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f27fc640 1 -- 172.21.15.157:0/1767195129 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0a5810 con 0x7fb2f40b3f70 2023-11-06T03:15:39.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.516+0000 7fb2f27fc640 1 -- 172.21.15.157:0/1767195129 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0a5ba0 con 0x7fb2f40b3f70 2023-11-06T03:15:39.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1767195129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f40b3f70 msgr2=0x7fb2f40ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:39.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1767195129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f40b3f70 0x7fb2f40ce560 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40b45b0 tx=0x7fb2ec01c900 comp rx=0 tx=0).stop 2023-11-06T03:15:39.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1767195129 shutdown_connections 2023-11-06T03:15:39.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1767195129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f40b3f70 0x7fb2f40ce560 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1767195129 >> v2:172.21.15.195:3300/0 conn(0x7fb2f415cc10 0x7fb2f406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1767195129 >> 172.21.15.157:0/1767195129 conn(0x7fb2f406a910 msgr2=0x7fb2f40cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:39.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1767195129 shutdown_connections 2023-11-06T03:15:39.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1767195129 wait complete. 2023-11-06T03:15:39.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 Processor -- start 2023-11-06T03:15:39.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- start start 2023-11-06T03:15:39.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 0x7fb2f4142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2f40ce0d0 0x7fb2f4148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f414a900 con 0x7fb2f415cc10 2023-11-06T03:15:39.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4143720 con 0x7fb2f40ce0d0 2023-11-06T03:15:39.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f9e7d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f414a620 con 0x7fb2f40b3f70 2023-11-06T03:15:39.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38032/0 (socket says 172.21.15.157:38032) 2023-11-06T03:15:39.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.520+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1063988986 learned_addr learned my addr 172.21.15.157:0/1063988986 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:39.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f2ffd640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2f40ce0d0 0x7fb2f4148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f37fe640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 0x7fb2f4142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 msgr2=0x7fb2f4142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 0x7fb2f4142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2f40ce0d0 msgr2=0x7fb2f4148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:39.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2f40ce0d0 0x7fb2f4148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 -- 172.21.15.157:0/1063988986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009b30 con 0x7fb2f415cc10 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f37fe640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 0x7fb2f4142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f3fff640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0af040 tx=0x7fb2ec0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0b8670 con 0x7fb2f415cc10 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0b87d0 con 0x7fb2f415cc10 2023-11-06T03:15:39.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f41430d0 con 0x7fb2f415cc10 2023-11-06T03:15:39.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0b8b60 con 0x7fb2f415cc10 2023-11-06T03:15:39.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f414a2b0 con 0x7fb2f415cc10 2023-11-06T03:15:39.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f4143890 con 0x7fb2f415cc10 2023-11-06T03:15:39.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2ec0b8cc0 con 0x7fb2f415cc10 2023-11-06T03:15:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 0x7fb2e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fb2ec11e4f0 con 0x7fb2f415cc10 2023-11-06T03:15:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f37fe640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 0x7fb2e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:39.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.524+0000 7fb2f37fe640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 0x7fb2e0067f50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb2f406d3c0 tx=0x7fb2e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:39.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.528+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb2ec0ebfa0 con 0x7fb2f415cc10 2023-11-06T03:15:39.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:39.684+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2f404e050 con 0x7fb2f415cc10 2023-11-06T03:15:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.360+0000 7fb2dbfff640 1 -- 172.21.15.157:0/1063988986 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb2ec009510 con 0x7fb2f415cc10 2023-11-06T03:15:40.357 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:15:40.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 msgr2=0x7fb2e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 0x7fb2e0067f50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb2f406d3c0 tx=0x7fb2e8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 msgr2=0x7fb2f4148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0af040 tx=0x7fb2ec0196a0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 shutdown_connections 2023-11-06T03:15:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2e0065a90 0x7fb2e0067f50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2f415cc10 0x7fb2f4148ca0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2f40ce0d0 0x7fb2f4148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 --2- 172.21.15.157:0/1063988986 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2f40b3f70 0x7fb2f4142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.364+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 >> 172.21.15.157:0/1063988986 conn(0x7fb2f406a910 msgr2=0x7fb2f4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.368+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 shutdown_connections 2023-11-06T03:15:40.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.368+0000 7fb2f9e7d640 1 -- 172.21.15.157:0/1063988986 wait complete. 2023-11-06T03:15:40.397 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:15:40.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 Processor -- start 2023-11-06T03:15:40.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 -- start start 2023-11-06T03:15:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3144146370 0x7f31440b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f31440cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440cf1a0 con 0x7f31440ce800 2023-11-06T03:15:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314406c3d0 con 0x7f3144146370 2023-11-06T03:15:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3149c9d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314406f5a0 con 0x7f314406bdb0 2023-11-06T03:15:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3142ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f31440ce800 0x7f314406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59542/0 (socket says 172.21.15.157:59542) 2023-11-06T03:15:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3142ffd640 1 -- 172.21.15.157:0/3095832805 learned_addr learned my addr 172.21.15.157:0/3095832805 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:40.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f31437fe640 1 --2- 172.21.15.157:0/3095832805 >> v2:172.21.15.195:3300/0 conn(0x7f3144146370 0x7f31440b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639226958 0 0) 0x7f31440cf1a0 con 0x7f31440ce800 2023-11-06T03:15:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f312c003530 con 0x7f31440ce800 2023-11-06T03:15:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.520+0000 7f3143fff640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f31440cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 222034466 0 0) 0x7f312c003530 con 0x7f31440ce800 2023-11-06T03:15:40.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 msgr2=0x7f31440cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f31440cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 >> v2:172.21.15.195:3300/0 conn(0x7f3144146370 msgr2=0x7f31440b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 --2- 172.21.15.157:0/3095832805 >> v2:172.21.15.195:3300/0 conn(0x7f3144146370 0x7f31440b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314406f880 con 0x7f31440ce800 2023-11-06T03:15:40.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3138001680 con 0x7f31440ce800 2023-11-06T03:15:40.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 >> v1:172.21.15.157:6789/0 conn(0x7f31440ce800 legacy=0x7f314406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f312c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f312c003f00 0x7f312c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312c004d10 0x7f312c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314406c3d0 con 0x7f312c004d10 2023-11-06T03:15:40.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314406f5a0 con 0x7f312c003f00 2023-11-06T03:15:40.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c00b9e0 con 0x7f314406bdb0 2023-11-06T03:15:40.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312c004d10 0x7f312c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3142ffd640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f312c003f00 0x7f312c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31437fe640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f312c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 -- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 msgr2=0x7f312c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:40.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f314406bdb0 0x7f312c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 -- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f312c003f00 msgr2=0x7f312c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:40.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f312c003f00 0x7f312c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 -- 172.21.15.157:0/3095832805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c003530 con 0x7f312c004d10 2023-11-06T03:15:40.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3142ffd640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f312c003f00 0x7f312c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:40.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312c004d10 0x7f312c005190 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f313c00e170 tx=0x7f313c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7cc49bd4af77a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0a5c50 con 0x7f312c004d10 2023-11-06T03:15:40.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00f040 con 0x7f312c004d10 2023-11-06T03:15:40.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f31427fc640 1 -- 172.21.15.157:0/3095832805 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0ad440 con 0x7f312c004d10 2023-11-06T03:15:40.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 -- 172.21.15.157:0/3095832805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312c004d10 msgr2=0x7f312c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3095832805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f312c004d10 0x7f312c005190 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f313c00e170 tx=0x7f313c01cc40 comp rx=0 tx=0).stop 2023-11-06T03:15:40.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3143fff640 1 -- 172.21.15.157:0/3095832805 reap_dead start 2023-11-06T03:15:40.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 -- 172.21.15.157:0/3095832805 shutdown_connections 2023-11-06T03:15:40.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 -- 172.21.15.157:0/3095832805 >> 172.21.15.157:0/3095832805 conn(0x7f3144065930 msgr2=0x7f3144065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:40.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 -- 172.21.15.157:0/3095832805 shutdown_connections 2023-11-06T03:15:40.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.524+0000 7f3149c9d640 1 -- 172.21.15.157:0/3095832805 wait complete. 2023-11-06T03:15:40.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 Processor -- start 2023-11-06T03:15:40.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- start start 2023-11-06T03:15:40.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 0x7f31441454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 0x7f314415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314415c5c0 con 0x7f31440b3460 2023-11-06T03:15:40.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31441f5740 con 0x7f31440b38e0 2023-11-06T03:15:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31441f5990 con 0x7f312c004d10 2023-11-06T03:15:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38052/0 (socket says 172.21.15.157:38052) 2023-11-06T03:15:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 -- 172.21.15.157:0/3233045146 learned_addr learned my addr 172.21.15.157:0/3233045146 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:40.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f31437fe640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 0x7f31441454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3143fff640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 0x7f314415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 -- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 msgr2=0x7f31441454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 0x7f31441454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 -- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 msgr2=0x7f314415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 0x7f314415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 -- 172.21.15.157:0/3233045146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31441f5c70 con 0x7f31440b3460 2023-11-06T03:15:40.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3143fff640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 0x7f314415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:40.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f31437fe640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 0x7f31441454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:40.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3142ffd640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f3138011790 tx=0x7f3138011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138011ea0 con 0x7f31440b3460 2023-11-06T03:15:40.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800fb80 con 0x7f31440b3460 2023-11-06T03:15:40.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138015410 con 0x7f31440b3460 2023-11-06T03:15:40.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31441f5f50 con 0x7f31440b3460 2023-11-06T03:15:40.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31441f6510 con 0x7f31440b3460 2023-11-06T03:15:40.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.528+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314415d190 con 0x7f31440b3460 2023-11-06T03:15:40.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.532+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3138015570 con 0x7f31440b3460 2023-11-06T03:15:40.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.532+0000 7f312bfff640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 0x7f3130067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.532+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3138086330 con 0x7f31440b3460 2023-11-06T03:15:40.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.532+0000 7f31437fe640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 0x7f3130067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.532+0000 7f31437fe640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 0x7f3130067f20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3134009b70 tx=0x7f3134008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.536+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3138053e40 con 0x7f31440b3460 2023-11-06T03:15:40.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.688+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3144072750 con 0x7f31440b3460 2023-11-06T03:15:40.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.696+0000 7f312bfff640 1 -- 172.21.15.157:0/3233045146 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3138057460 con 0x7f31440b3460 2023-11-06T03:15:40.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.696+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 msgr2=0x7f3130067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.696+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 0x7f3130067f20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3134009b70 tx=0x7f3134008040 comp rx=0 tx=0).stop 2023-11-06T03:15:40.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.696+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 msgr2=0x7f314415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.696+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f3138011790 tx=0x7f3138011c50 comp rx=0 tx=0).stop 2023-11-06T03:15:40.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 shutdown_connections 2023-11-06T03:15:40.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3130065a60 0x7f3130067f20 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31440b38e0 0x7f314415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31440b3460 0x7f314415bb40 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 --2- 172.21.15.157:0/3233045146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f312c004d10 0x7f31441454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 >> 172.21.15.157:0/3233045146 conn(0x7f3144065930 msgr2=0x7f31440ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:40.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 shutdown_connections 2023-11-06T03:15:40.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.700+0000 7f3149c9d640 1 -- 172.21.15.157:0/3233045146 wait complete. 2023-11-06T03:15:40.713 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2023-11-06T03:15:40.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.836+0000 7f44cf7e0640 1 Processor -- start 2023-11-06T03:15:40.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 -- start start 2023-11-06T03:15:40.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44c81551e0 0x7f44c806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f44c815e5e0 0x7f44c80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c815f580 con 0x7f44c815ea90 2023-11-06T03:15:40.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c815efc0 con 0x7f44c81551e0 2023-11-06T03:15:40.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cf7e0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c815f2a0 con 0x7f44c815e5e0 2023-11-06T03:15:40.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cdd7d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f44c815ea90 0x7f44c815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59546/0 (socket says 172.21.15.157:59546) 2023-11-06T03:15:40.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cdd7d640 1 -- 172.21.15.157:0/1688952974 learned_addr learned my addr 172.21.15.157:0/1688952974 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:40.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 --2- 172.21.15.157:0/1688952974 >> v2:172.21.15.157:3300/0 conn(0x7f44c81551e0 0x7f44c806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44ccd7b640 1 --2- 172.21.15.157:0/1688952974 >> v2:172.21.15.195:3300/0 conn(0x7f44c815e5e0 0x7f44c80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 -- 172.21.15.157:0/1688952974 >> v2:172.21.15.195:3300/0 conn(0x7f44c815e5e0 msgr2=0x7f44c80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:40.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 --2- 172.21.15.157:0/1688952974 >> v2:172.21.15.195:3300/0 conn(0x7f44c815e5e0 0x7f44c80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 -- 172.21.15.157:0/1688952974 >> v1:172.21.15.157:6789/0 conn(0x7f44c815ea90 legacy=0x7f44c815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44bffff640 1 -- 172.21.15.157:0/1688952974 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949348028 0 0) 0x7f44c815f580 con 0x7f44c815ea90 2023-11-06T03:15:40.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 -- 172.21.15.157:0/1688952974 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c806f5a0 con 0x7f44c81551e0 2023-11-06T03:15:40.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.840+0000 7f44cd57c640 1 --2- 172.21.15.157:0/1688952974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c81551e0 0x7f44c806fda0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f44c8066880 tx=0x7f44c401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcc1ad6eebd1d8c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bffff640 1 -- 172.21.15.157:0/1688952974 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40ab600 con 0x7f44c81551e0 2023-11-06T03:15:40.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bffff640 1 -- 172.21.15.157:0/1688952974 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c40a7050 con 0x7f44c81551e0 2023-11-06T03:15:40.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bffff640 1 -- 172.21.15.157:0/1688952974 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40ab980 con 0x7f44c81551e0 2023-11-06T03:15:40.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/1688952974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c81551e0 msgr2=0x7f44c806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/1688952974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c81551e0 0x7f44c806fda0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f44c8066880 tx=0x7f44c401c3a0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/1688952974 shutdown_connections 2023-11-06T03:15:40.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/1688952974 >> v2:172.21.15.195:3300/0 conn(0x7f44c815e5e0 0x7f44c80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/1688952974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c81551e0 0x7f44c806fda0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/1688952974 >> 172.21.15.157:0/1688952974 conn(0x7f44c8065930 msgr2=0x7f44c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:40.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/1688952974 shutdown_connections 2023-11-06T03:15:40.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/1688952974 wait complete. 2023-11-06T03:15:40.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 Processor -- start 2023-11-06T03:15:40.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- start start 2023-11-06T03:15:40.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44c81551e0 0x7f44c814da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44c815ea90 0x7f44c80afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8151b80 con 0x7f44c815e5e0 2023-11-06T03:15:40.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c806cf40 con 0x7f44c815ea90 2023-11-06T03:15:40.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c81531d0 con 0x7f44c81551e0 2023-11-06T03:15:40.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38066/0 (socket says 172.21.15.157:38066) 2023-11-06T03:15:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 -- 172.21.15.157:0/3475285565 learned_addr learned my addr 172.21.15.157:0/3475285565 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cdd7d640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44c815ea90 0x7f44c80afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cd57c640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44c81551e0 0x7f44c814da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 -- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44c81551e0 msgr2=0x7f44c814da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44c81551e0 0x7f44c814da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 -- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44c815ea90 msgr2=0x7f44c80afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:40.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44c815ea90 0x7f44c80afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:40.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 -- 172.21.15.157:0/3475285565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009420 con 0x7f44c815e5e0 2023-11-06T03:15:40.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44ccd7b640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d830 tx=0x7f44b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800b6c0 con 0x7f44c815e5e0 2023-11-06T03:15:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800b820 con 0x7f44c815e5e0 2023-11-06T03:15:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800bb90 con 0x7f44c815e5e0 2023-11-06T03:15:40.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c814e0c0 con 0x7f44c815e5e0 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.844+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c8152ef0 con 0x7f44c815e5e0 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c8151d30 con 0x7f44c815e5e0 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f44b801e460 con 0x7f44c815e5e0 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44bdffb640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 0x7f44ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f44b8083750 con 0x7f44c815e5e0 2023-11-06T03:15:40.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44cd57c640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 0x7f44ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:40.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.848+0000 7f44cd57c640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 0x7f44ac067f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ad000 tx=0x7f44c40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:40.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:40.852+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f44c8151d30 con 0x7f44c815e5e0 2023-11-06T03:15:41.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.084+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f44c8151d30 con 0x7f44c815e5e0 2023-11-06T03:15:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.376+0000 7f44bdffb640 1 -- 172.21.15.157:0/3475285565 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v536) v1 ==== 166+0+0 (secure 0 0 0) 0x7f44c8151d30 con 0x7f44c815e5e0 2023-11-06T03:15:41.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 msgr2=0x7f44ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 0x7f44ac067f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ad000 tx=0x7f44c40a6000 comp rx=0 tx=0).stop 2023-11-06T03:15:41.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 msgr2=0x7f44c806ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d830 tx=0x7f44b800dcf0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 shutdown_connections 2023-11-06T03:15:41.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f44ac065a60 0x7f44ac067f20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44c815ea90 0x7f44c80afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44c815e5e0 0x7f44c806ca00 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 --2- 172.21.15.157:0/3475285565 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44c81551e0 0x7f44c814da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 >> 172.21.15.157:0/3475285565 conn(0x7f44c8065930 msgr2=0x7f44c814fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 shutdown_connections 2023-11-06T03:15:41.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.380+0000 7f44cf7e0640 1 -- 172.21.15.157:0/3475285565 wait complete. 2023-11-06T03:15:41.407 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2023-11-06T03:15:41.407 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2023-11-06T03:15:41.408 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2023-11-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 Processor -- start 2023-11-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 -- start start 2023-11-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 0x7f241c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f241c15ce80 0x7f241c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c0d2920 con 0x7f241c0ce1a0 2023-11-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0d18a0 con 0x7f241c15ce80 2023-11-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f2422311640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0d1b60 con 0x7f241c15caa0 2023-11-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f241c15ce80 0x7f241c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f24208ae640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f241c0ce1a0 0x7f241c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59560/0 (socket says 172.21.15.157:59560) 2023-11-06T03:15:41.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f241c15ce80 0x7f241c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38072/0 (socket says 172.21.15.157:38072) 2023-11-06T03:15:41.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f24208ae640 1 -- 172.21.15.157:0/1938998867 learned_addr learned my addr 172.21.15.157:0/1938998867 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:41.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241bfff640 1 --2- 172.21.15.157:0/1938998867 >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 0x7f241c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 -- 172.21.15.157:0/1938998867 >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 msgr2=0x7f241c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 --2- 172.21.15.157:0/1938998867 >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 0x7f241c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 -- 172.21.15.157:0/1938998867 >> v1:172.21.15.157:6789/0 conn(0x7f241c0ce1a0 legacy=0x7f241c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241b7fe640 1 -- 172.21.15.157:0/1938998867 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0d1e40 con 0x7f241c15ce80 2023-11-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.528+0000 7f241bfff640 1 --2- 172.21.15.157:0/1938998867 >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 0x7f241c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f241b7fe640 1 --2- 172.21.15.157:0/1938998867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c0cfb70 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f241c0b4f80 tx=0x7f241001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1e9aa409fe7427 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f241affd640 1 -- 172.21.15.157:0/1938998867 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100ab600 con 0x7f241c15ce80 2023-11-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f241affd640 1 -- 172.21.15.157:0/1938998867 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24100a7050 con 0x7f241c15ce80 2023-11-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f241affd640 1 -- 172.21.15.157:0/1938998867 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100ab980 con 0x7f241c15ce80 2023-11-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- 172.21.15.157:0/1938998867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 msgr2=0x7f241c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 --2- 172.21.15.157:0/1938998867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c0cfb70 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f241c0b4f80 tx=0x7f241001c430 comp rx=0 tx=0).stop 2023-11-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- 172.21.15.157:0/1938998867 shutdown_connections 2023-11-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 --2- 172.21.15.157:0/1938998867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c0cfb70 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 --2- 172.21.15.157:0/1938998867 >> v2:172.21.15.195:3300/0 conn(0x7f241c15caa0 0x7f241c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- 172.21.15.157:0/1938998867 >> 172.21.15.157:0/1938998867 conn(0x7f241c070790 msgr2=0x7f241c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- 172.21.15.157:0/1938998867 shutdown_connections 2023-11-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- 172.21.15.157:0/1938998867 wait complete. 2023-11-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 Processor -- start 2023-11-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.532+0000 7f2422311640 1 -- start start 2023-11-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f241c0ce1a0 0x7f241c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f241c15caa0 0x7f241c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c151130 con 0x7f241c15ce80 2023-11-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c157140 con 0x7f241c0ce1a0 2023-11-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c156190 con 0x7f241c15caa0 2023-11-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38082/0 (socket says 172.21.15.157:38082) 2023-11-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 -- 172.21.15.157:0/2317749378 learned_addr learned my addr 172.21.15.157:0/2317749378 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f241bfff640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f241c0ce1a0 0x7f241c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f241b7fe640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f241c15caa0 0x7f241c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 -- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f241c15caa0 msgr2=0x7f241c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f241c15caa0 0x7f241c156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 -- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f241c0ce1a0 msgr2=0x7f241c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f241c0ce1a0 0x7f241c0c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 -- 172.21.15.157:0/2317749378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410009420 con 0x7f241c15ce80 2023-11-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f24208ae640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f241400caa0 tx=0x7f241400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400ab40 con 0x7f241c15ce80 2023-11-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400aca0 con 0x7f241c15ce80 2023-11-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c154f00 con 0x7f241c15ce80 2023-11-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414012600 con 0x7f241c15ce80 2023-11-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c155450 con 0x7f241c15ce80 2023-11-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c072750 con 0x7f241c15ce80 2023-11-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2414007070 con 0x7f241c15ce80 2023-11-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 0x7f2408067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f2414082cf0 con 0x7f241c15ce80 2023-11-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.536+0000 7f241bfff640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 0x7f2408067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.540+0000 7f241bfff640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 0x7f2408067f20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f240c00a9a0 tx=0x7f240c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.540+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2414087070 con 0x7f241c15ce80 2023-11-06T03:15:41.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:41.752+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f241c0c9ab0 con 0x7f241c15ce80 2023-11-06T03:15:42.020 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:15:42.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:15:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2418ff9640 1 -- 172.21.15.157:0/2317749378 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v537) v1 ==== 143+0+0 (secure 0 0 0) 0x7f2414017020 con 0x7f241c15ce80 2023-11-06T03:15:42.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 msgr2=0x7f2408067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 0x7f2408067f20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f240c00a9a0 tx=0x7f240c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:42.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 msgr2=0x7f241c155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f241400caa0 tx=0x7f241400cf60 comp rx=0 tx=0).stop 2023-11-06T03:15:42.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.380+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 shutdown_connections 2023-11-06T03:15:42.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2408065a60 0x7f2408067f20 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f241c15ce80 0x7f241c155c50 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f241c15caa0 0x7f241c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 --2- 172.21.15.157:0/2317749378 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f241c0ce1a0 0x7f241c0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 >> 172.21.15.157:0/2317749378 conn(0x7f241c070790 msgr2=0x7f241c070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:42.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 shutdown_connections 2023-11-06T03:15:42.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.384+0000 7f2422311640 1 -- 172.21.15.157:0/2317749378 wait complete. 2023-11-06T03:15:42.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:15:42.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 Processor -- start 2023-11-06T03:15:42.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 -- start start 2023-11-06T03:15:42.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f898c15ae40 0x7f898c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c15b220 0x7f898c0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c0cf920 con 0x7f898c0afb00 2023-11-06T03:15:42.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0cff50 con 0x7f898c15b220 2023-11-06T03:15:42.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f89944bc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0cdf80 con 0x7f898c15ae40 2023-11-06T03:15:42.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c15b220 0x7f898c0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:42.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f898c15b220 0x7f898c0cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38088/0 (socket says 172.21.15.157:38088) 2023-11-06T03:15:42.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 -- 172.21.15.157:0/495570345 learned_addr learned my addr 172.21.15.157:0/495570345 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:42.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8992258640 1 --2- 172.21.15.157:0/495570345 >> v2:172.21.15.195:3300/0 conn(0x7f898c15ae40 0x7f898c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:42.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 -- 172.21.15.157:0/495570345 >> v2:172.21.15.195:3300/0 conn(0x7f898c15ae40 msgr2=0x7f898c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:42.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 --2- 172.21.15.157:0/495570345 >> v2:172.21.15.195:3300/0 conn(0x7f898c15ae40 0x7f898c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 -- 172.21.15.157:0/495570345 >> v1:172.21.15.195:6789/0 conn(0x7f898c0afb00 legacy=0x7f898c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:42.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.528+0000 7f8991a57640 1 -- 172.21.15.157:0/495570345 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0ce260 con 0x7f898c15b220 2023-11-06T03:15:42.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- 172.21.15.157:0/495570345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15b220 0x7f898c0cc7d0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f898c066930 tx=0x7f897c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfef0fa8d044377 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:42.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991256640 1 -- 172.21.15.157:0/495570345 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0ab620 con 0x7f898c15b220 2023-11-06T03:15:42.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991256640 1 -- 172.21.15.157:0/495570345 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c0ab780 con 0x7f898c15b220 2023-11-06T03:15:42.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991256640 1 -- 172.21.15.157:0/495570345 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0abb10 con 0x7f898c15b220 2023-11-06T03:15:42.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/495570345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15b220 msgr2=0x7f898c0cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- 172.21.15.157:0/495570345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15b220 0x7f898c0cc7d0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f898c066930 tx=0x7f897c01c400 comp rx=0 tx=0).stop 2023-11-06T03:15:42.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/495570345 shutdown_connections 2023-11-06T03:15:42.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- 172.21.15.157:0/495570345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15b220 0x7f898c0cc7d0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- 172.21.15.157:0/495570345 >> v2:172.21.15.195:3300/0 conn(0x7f898c15ae40 0x7f898c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/495570345 >> 172.21.15.157:0/495570345 conn(0x7f898c0cd490 msgr2=0x7f898c0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:42.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/495570345 shutdown_connections 2023-11-06T03:15:42.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/495570345 wait complete. 2023-11-06T03:15:42.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 Processor -- start 2023-11-06T03:15:42.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- start start 2023-11-06T03:15:42.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f898c0afb00 0x7f898c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f898c15b220 0x7f898c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c15c670 con 0x7f898c15ae40 2023-11-06T03:15:42.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c15cf10 con 0x7f898c0afb00 2023-11-06T03:15:42.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c15d120 con 0x7f898c15b220 2023-11-06T03:15:42.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:42.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38092/0 (socket says 172.21.15.157:38092) 2023-11-06T03:15:42.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 -- 172.21.15.157:0/3539000555 learned_addr learned my addr 172.21.15.157:0/3539000555 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:42.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8992258640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f898c0afb00 0x7f898c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:42.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 -- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f898c15b220 msgr2=0x7f898c156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f898c15b220 0x7f898c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 -- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f898c0afb00 msgr2=0x7f898c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f898c0afb00 0x7f898c15c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 -- 172.21.15.157:0/3539000555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0094d0 con 0x7f898c15ae40 2023-11-06T03:15:42.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8991a57640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f897c01a030 tx=0x7f897c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:42.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0be3f0 con 0x7f898c15ae40 2023-11-06T03:15:42.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c0be570 con 0x7f898c15ae40 2023-11-06T03:15:42.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c15d400 con 0x7f898c15ae40 2023-11-06T03:15:42.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0be920 con 0x7f898c15ae40 2023-11-06T03:15:42.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.532+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c15d9c0 con 0x7f898c15ae40 2023-11-06T03:15:42.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c072750 con 0x7f898c15ae40 2023-11-06T03:15:42.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f897c0b43f0 con 0x7f898c15ae40 2023-11-06T03:15:42.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f8982ffd640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 0x7f8970067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:42.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f897c11d680 con 0x7f898c15ae40 2023-11-06T03:15:42.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f8992258640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 0x7f8970067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:42.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.536+0000 7f8992258640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 0x7f8970067f20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8988009790 tx=0x7f8988008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:42.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.540+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f897c0eb1a0 con 0x7f898c15ae40 2023-11-06T03:15:42.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.696+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f898c067440 con 0x7f898c15ae40 2023-11-06T03:15:42.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.704+0000 7f8982ffd640 1 -- 172.21.15.157:0/3539000555 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f897c0ee7c0 con 0x7f898c15ae40 2023-11-06T03:15:42.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 msgr2=0x7f8970067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 0x7f8970067f20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8988009790 tx=0x7f8988008040 comp rx=0 tx=0).stop 2023-11-06T03:15:42.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 msgr2=0x7f898c15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:42.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f897c01a030 tx=0x7f897c01ac40 comp rx=0 tx=0).stop 2023-11-06T03:15:42.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 shutdown_connections 2023-11-06T03:15:42.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8970065a60 0x7f8970067f20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f898c15b220 0x7f898c156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f898c15ae40 0x7f898c15c9d0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 --2- 172.21.15.157:0/3539000555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f898c0afb00 0x7f898c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:42.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 >> 172.21.15.157:0/3539000555 conn(0x7f898c0cd490 msgr2=0x7f898c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:42.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 shutdown_connections 2023-11-06T03:15:42.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:42.708+0000 7f89944bc640 1 -- 172.21.15.157:0/3539000555 wait complete. 2023-11-06T03:15:42.723 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:15:42.723 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:15:42.769 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:15:42.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:15:42.829 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:15:42.829 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:42.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:15:42.836 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:15:42.836 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:42.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:15:42.843 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:15:42.843 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:42.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:15:42.884 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:15:42.884 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:15:42.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:15:42.891 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:15:42.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:43.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.056+0000 7fab10f85640 1 Processor -- start 2023-11-06T03:15:43.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.056+0000 7fab10f85640 1 -- start start 2023-11-06T03:15:43.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab10f85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c0b2af0 0x7fab0c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab10f85640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab0c06d000 0x7fab0c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab10f85640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c15e440 con 0x7fab0c1599c0 2023-11-06T03:15:43.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab10f85640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0c06c240 con 0x7fab0c06d000 2023-11-06T03:15:43.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab10f85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c06c520 con 0x7fab0c0b2af0 2023-11-06T03:15:43.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0a59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fab0c1599c0 0x7fab0c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59576/0 (socket says 172.21.15.157:59576) 2023-11-06T03:15:43.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0a59c640 1 -- 172.21.15.157:0/1422203577 learned_addr learned my addr 172.21.15.157:0/1422203577 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:43.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0ad9d640 1 --2- 172.21.15.157:0/1422203577 >> v2:172.21.15.157:3300/0 conn(0x7fab0c06d000 0x7fab0c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab09d9b640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c0b2af0 0x7fab0c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0959a640 1 -- 172.21.15.157:0/1422203577 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135071550 0 0) 0x7fab0c15e440 con 0x7fab0c1599c0 2023-11-06T03:15:43.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0959a640 1 -- 172.21.15.157:0/1422203577 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf4003530 con 0x7fab0c1599c0 2023-11-06T03:15:43.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0ad9d640 1 -- 172.21.15.157:0/1422203577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c0b2af0 msgr2=0x7fab0c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:43.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0ad9d640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c0b2af0 0x7fab0c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0ad9d640 1 -- 172.21.15.157:0/1422203577 >> v1:172.21.15.157:6789/0 conn(0x7fab0c1599c0 legacy=0x7fab0c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.060+0000 7fab0ad9d640 1 -- 172.21.15.157:0/1422203577 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c06c800 con 0x7fab0c06d000 2023-11-06T03:15:43.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0ad9d640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c159da0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fab04009d90 tx=0x7fab0401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28070db72e0f4841 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0959a640 1 -- 172.21.15.157:0/1422203577 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040a5a00 con 0x7fab0c06d000 2023-11-06T03:15:43.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0959a640 1 -- 172.21.15.157:0/1422203577 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab040a5b60 con 0x7fab0c06d000 2023-11-06T03:15:43.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0959a640 1 -- 172.21.15.157:0/1422203577 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040ad440 con 0x7fab0c06d000 2023-11-06T03:15:43.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- 172.21.15.157:0/1422203577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 msgr2=0x7fab0c159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c159da0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fab04009d90 tx=0x7fab0401c920 comp rx=0 tx=0).stop 2023-11-06T03:15:43.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- 172.21.15.157:0/1422203577 shutdown_connections 2023-11-06T03:15:43.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c159da0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- 172.21.15.157:0/1422203577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c0b2af0 0x7fab0c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- 172.21.15.157:0/1422203577 >> 172.21.15.157:0/1422203577 conn(0x7fab0c065930 msgr2=0x7fab0c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:43.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- 172.21.15.157:0/1422203577 shutdown_connections 2023-11-06T03:15:43.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- 172.21.15.157:0/1422203577 wait complete. 2023-11-06T03:15:43.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 Processor -- start 2023-11-06T03:15:43.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- start start 2023-11-06T03:15:43.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 0x7fab0c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 0x7fab0c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0cf060 con 0x7fab0c06d000 2023-11-06T03:15:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0ae0f0 con 0x7fab0c0b2af0 2023-11-06T03:15:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab10f85640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0ae3d0 con 0x7fab0c1599c0 2023-11-06T03:15:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38108/0 (socket says 172.21.15.157:38108) 2023-11-06T03:15:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 -- 172.21.15.157:0/2486128035 learned_addr learned my addr 172.21.15.157:0/2486128035 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0ad9d640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 0x7fab0c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab09d9b640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 0x7fab0c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 -- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 msgr2=0x7fab0c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 0x7fab0c0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 -- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 msgr2=0x7fab0c0cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 0x7fab0c0cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0a59c640 1 -- 172.21.15.157:0/2486128035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04009a70 con 0x7fab0c06d000 2023-11-06T03:15:43.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.064+0000 7fab0ad9d640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 0x7fab0c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab09d9b640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 0x7fab0c0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab0a59c640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c420 tx=0x7faafc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc005cf0 con 0x7fab0c06d000 2023-11-06T03:15:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc005e50 con 0x7fab0c06d000 2023-11-06T03:15:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c0ae6b0 con 0x7fab0c06d000 2023-11-06T03:15:43.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0072d0 con 0x7fab0c06d000 2023-11-06T03:15:43.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c139670 con 0x7fab0c06d000 2023-11-06T03:15:43.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c0b2930 con 0x7fab0c06d000 2023-11-06T03:15:43.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faafc007430 con 0x7fab0c06d000 2023-11-06T03:15:43.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 0x7faaec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5382+0+0 (secure 0 0 0) 0x7faafc082520 con 0x7fab0c06d000 2023-11-06T03:15:43.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab09d9b640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 0x7faaec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.068+0000 7fab09d9b640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 0x7faaec067f20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fab0c1f6f20 tx=0x7fab00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.072+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7faafc050160 con 0x7fab0c06d000 2023-11-06T03:15:43.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab0c072750 con 0x7fab0c06d000 2023-11-06T03:15:43.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7faafaffd640 1 -- 172.21.15.157:0/2486128035 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+2780 (secure 0 0 0) 0x7faafc017af0 con 0x7fab0c06d000 2023-11-06T03:15:43.261 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:43.261 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/1264708947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":1264708947},{"type":"v1","addr":"172.21.15.157:6833","nonce":1264708947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":6452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/252835837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":252835837},{"type":"v1","addr":"172.21.15.195:6835","nonce":252835837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7447,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/3185412443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":3185412443},{"type":"v1","addr":"172.21.15.195:6837","nonce":3185412443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[]} 2023-11-06T03:15:43.261 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 107 2023-11-06T03:15:43.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 msgr2=0x7faaec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 0x7faaec067f20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fab0c1f6f20 tx=0x7fab00008040 comp rx=0 tx=0).stop 2023-11-06T03:15:43.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 msgr2=0x7fab0c139130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c420 tx=0x7faafc00c8e0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 shutdown_connections 2023-11-06T03:15:43.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7faaec065a60 0x7faaec067f20 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab0c1599c0 0x7fab0c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab0c0b2af0 0x7fab0c0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 --2- 172.21.15.157:0/2486128035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab0c06d000 0x7fab0c139130 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 >> 172.21.15.157:0/2486128035 conn(0x7fab0c065930 msgr2=0x7fab0c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:43.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.264+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 shutdown_connections 2023-11-06T03:15:43.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.268+0000 7fab10f85640 1 -- 172.21.15.157:0/2486128035 wait complete. 2023-11-06T03:15:43.281 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:15:43.281 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:15:43.282 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:15:43.282 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:15:43.282 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:15:43.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:15:43.283 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:15:43.283 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:15:43.284 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:15:43.284 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:15:43.284 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:15:43.285 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:15:43.285 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:15:43.285 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:15:43.287 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:15:43.287 DEBUG:teuthology.parallel:result is None 2023-11-06T03:15:43.295 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:15:43.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:15:43.299 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:15:43.300 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:15:43.302 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:15:43.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:15:43.340 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:15:43.341 DEBUG:teuthology.parallel:result is None 2023-11-06T03:15:43.347 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:15:43.347 DEBUG:teuthology.parallel:result is None 2023-11-06T03:15:43.348 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:15:43.348 DEBUG:teuthology.parallel:result is None 2023-11-06T03:15:43.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:15:43.359 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:15:43.404 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:15:43.420 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:15:43.421 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:15:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.472+0000 7f461244b640 1 Processor -- start 2023-11-06T03:15:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.472+0000 7f461244b640 1 -- start start 2023-11-06T03:15:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f461244b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f460c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f461244b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c15f580 con 0x7f460c15e5e0 2023-11-06T03:15:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f461244b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c15efc0 con 0x7f460c1551e0 2023-11-06T03:15:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f461244b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0704b0 con 0x7f460c0b3b20 2023-11-06T03:15:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f460c15e5e0 0x7f460c0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59590/0 (socket says 172.21.15.157:59590) 2023-11-06T03:15:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 -- 172.21.15.157:0/2380981701 learned_addr learned my addr 172.21.15.157:0/2380981701 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f46109e8640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f460c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388224560 0 0) 0x7f460c15f580 con 0x7f460c15e5e0 2023-11-06T03:15:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f8003530 con 0x7f460c15e5e0 2023-11-06T03:15:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989894833 0 0) 0x7f460c15efc0 con 0x7f460c1551e0 2023-11-06T03:15:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f460c15f580 con 0x7f460c1551e0 2023-11-06T03:15:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2012950590 0 0) 0x7f45f8003530 con 0x7f460c15e5e0 2023-11-06T03:15:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 msgr2=0x7f460c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f460c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 >> v1:172.21.15.195:6789/0 conn(0x7f460c1551e0 legacy=0x7f460c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c15f2a0 con 0x7f460c15e5e0 2023-11-06T03:15:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f45f4001680 con 0x7f460c15e5e0 2023-11-06T03:15:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 >> v1:172.21.15.157:6789/0 conn(0x7f460c15e5e0 legacy=0x7f460c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f45f8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45f80040a0 0x7f45f80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f8004d10 0x7f45f8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0704b0 con 0x7f45f80040a0 2023-11-06T03:15:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f800bb20 con 0x7f45f8004d10 2023-11-06T03:15:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f800bcb0 con 0x7f460c0b3b20 2023-11-06T03:15:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45f80040a0 0x7f45f80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460bfff640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f45f8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f46109e8640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f8004d10 0x7f45f8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 -- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 msgr2=0x7f45f8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:43.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f45f8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 -- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f8004d10 msgr2=0x7f45f8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:43.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f8004d10 0x7f45f8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 -- 172.21.15.157:0/2380981701 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8003530 con 0x7f45f80040a0 2023-11-06T03:15:43.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f46109e8640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f8004d10 0x7f45f8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460bfff640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0b3b20 0x7f45f8003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.476+0000 7f460b7fe640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45f80040a0 0x7f45f80047d0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c160 tx=0x7f45f401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0f809577490ec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f40058e0 con 0x7f45f80040a0 2023-11-06T03:15:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4005a60 con 0x7f45f80040a0 2023-11-06T03:15:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f460affd640 1 -- 172.21.15.157:0/2380981701 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4005e50 con 0x7f45f80040a0 2023-11-06T03:15:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2380981701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45f80040a0 msgr2=0x7f45f80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 --2- 172.21.15.157:0/2380981701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45f80040a0 0x7f45f80047d0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c160 tx=0x7f45f401cb70 comp rx=0 tx=0).stop 2023-11-06T03:15:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 -- 172.21.15.157:0/2380981701 reap_dead start 2023-11-06T03:15:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2380981701 shutdown_connections 2023-11-06T03:15:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2380981701 >> 172.21.15.157:0/2380981701 conn(0x7f460c065930 msgr2=0x7f460c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2380981701 shutdown_connections 2023-11-06T03:15:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2380981701 wait complete. 2023-11-06T03:15:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 Processor -- start 2023-11-06T03:15:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- start start 2023-11-06T03:15:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f80040a0 0x7f460c144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0ce500 0x7f460c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1454e0 con 0x7f460c0ce980 2023-11-06T03:15:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1481e0 con 0x7f45f80040a0 2023-11-06T03:15:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1484c0 con 0x7f460c0ce500 2023-11-06T03:15:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38170/0 (socket says 172.21.15.157:38170) 2023-11-06T03:15:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f460bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f80040a0 0x7f460c144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 -- 172.21.15.157:0/2175239285 learned_addr learned my addr 172.21.15.157:0/2175239285 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:43.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f460b7fe640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0ce500 0x7f460c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 -- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0ce500 msgr2=0x7f460c1524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0ce500 0x7f460c1524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 -- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f80040a0 msgr2=0x7f460c144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:43.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f80040a0 0x7f460c144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:43.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 -- 172.21.15.157:0/2175239285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c1487a0 con 0x7f460c0ce980 2023-11-06T03:15:43.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f46109e8640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f460c15e9c0 tx=0x7f4604010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460401b070 con 0x7f460c0ce980 2023-11-06T03:15:43.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400f040 con 0x7f460c0ce980 2023-11-06T03:15:43.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c148a80 con 0x7f460c0ce980 2023-11-06T03:15:43.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400e050 con 0x7f460c0ce980 2023-11-06T03:15:43.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.480+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c147e90 con 0x7f460c0ce980 2023-11-06T03:15:43.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c072750 con 0x7f460c0ce980 2023-11-06T03:15:43.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f46040039a0 con 0x7f460c0ce980 2023-11-06T03:15:43.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f4608ff9640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 0x7f45ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:43.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f4604083a80 con 0x7f460c0ce980 2023-11-06T03:15:43.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f460bfff640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 0x7f45ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:43.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.484+0000 7f460bfff640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 0x7f45ec067f20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:43.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.488+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f46040517a0 con 0x7f460c0ce980 2023-11-06T03:15:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:43.656+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f460c148f30 con 0x7f460c0ce980 2023-11-06T03:15:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.460+0000 7f4608ff9640 1 -- 172.21.15.157:0/2175239285 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4604054dc0 con 0x7f460c0ce980 2023-11-06T03:15:44.457 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:15:44.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 msgr2=0x7f45ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 0x7f45ec067f20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).stop 2023-11-06T03:15:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 msgr2=0x7f460c152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f460c15e9c0 tx=0x7f4604010590 comp rx=0 tx=0).stop 2023-11-06T03:15:44.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 shutdown_connections 2023-11-06T03:15:44.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f45ec065a60 0x7f45ec067f20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460c0ce980 0x7f460c152a30 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f460c0ce500 0x7f460c1524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 --2- 172.21.15.157:0/2175239285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f80040a0 0x7f460c144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 >> 172.21.15.157:0/2175239285 conn(0x7f460c065930 msgr2=0x7f460c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 shutdown_connections 2023-11-06T03:15:44.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.464+0000 7f461244b640 1 -- 172.21.15.157:0/2175239285 wait complete. 2023-11-06T03:15:44.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:15:44.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.656+0000 7f114050d640 1 Processor -- start 2023-11-06T03:15:44.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.656+0000 7f114050d640 1 -- start start 2023-11-06T03:15:44.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f114050d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f11381599c0 0x7f11380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f114050d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1138159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f114050d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113815e440 con 0x7f113806d000 2023-11-06T03:15:44.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f114050d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113806c240 con 0x7f11381599c0 2023-11-06T03:15:44.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f114050d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113806c520 con 0x7f11380b2af0 2023-11-06T03:15:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113daa8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f113806d000 0x7f11380b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59598/0 (socket says 172.21.15.157:59598) 2023-11-06T03:15:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113daa8640 1 -- 172.21.15.157:0/1406956331 learned_addr learned my addr 172.21.15.157:0/1406956331 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586756442 0 0) 0x7f113815e440 con 0x7f113806d000 2023-11-06T03:15:44.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113e2a9640 1 --2- 172.21.15.157:0/1406956331 >> v2:172.21.15.195:3300/0 conn(0x7f11381599c0 0x7f11380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1124003530 con 0x7f113806d000 2023-11-06T03:15:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1138159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482580482 0 0) 0x7f1124003530 con 0x7f113806d000 2023-11-06T03:15:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 msgr2=0x7f1138159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:44.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1138159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 >> v2:172.21.15.195:3300/0 conn(0x7f11381599c0 msgr2=0x7f11380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 --2- 172.21.15.157:0/1406956331 >> v2:172.21.15.195:3300/0 conn(0x7f11381599c0 0x7f11380ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113806c800 con 0x7f113806d000 2023-11-06T03:15:44.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f11280016a0 con 0x7f113806d000 2023-11-06T03:15:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 >> v1:172.21.15.157:6789/0 conn(0x7f113806d000 legacy=0x7f11380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1124003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1124003f00 0x7f11240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124004d10 0x7f1124005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113806c240 con 0x7f1124004d10 2023-11-06T03:15:44.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113806c520 con 0x7f1124003f00 2023-11-06T03:15:44.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112400b990 con 0x7f11380b2af0 2023-11-06T03:15:44.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124004d10 0x7f1124005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113daa8640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1124003f00 0x7f11240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113e2a9640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1124003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 -- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 msgr2=0x7f1124003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1124003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 -- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1124003f00 msgr2=0x7f11240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1124003f00 0x7f11240047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113eaaa640 1 -- 172.21.15.157:0/1406956331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124003530 con 0x7f1124004d10 2023-11-06T03:15:44.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.660+0000 7f113e2a9640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11380b2af0 0x7f1124003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:44.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113eaaa640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124004d10 0x7f1124005190 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f113806f250 tx=0x7f113001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ae17377235d965 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:44.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300a5c10 con 0x7f1124004d10 2023-11-06T03:15:44.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113000f040 con 0x7f1124004d10 2023-11-06T03:15:44.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113d2a7640 1 -- 172.21.15.157:0/1406956331 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300ad440 con 0x7f1124004d10 2023-11-06T03:15:44.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/1406956331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124004d10 msgr2=0x7f1124005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 --2- 172.21.15.157:0/1406956331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1124004d10 0x7f1124005190 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f113806f250 tx=0x7f113001cba0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113eaaa640 1 -- 172.21.15.157:0/1406956331 reap_dead start 2023-11-06T03:15:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/1406956331 shutdown_connections 2023-11-06T03:15:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/1406956331 >> 172.21.15.157:0/1406956331 conn(0x7f1138065930 msgr2=0x7f1138065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:44.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/1406956331 shutdown_connections 2023-11-06T03:15:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/1406956331 wait complete. 2023-11-06T03:15:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 Processor -- start 2023-11-06T03:15:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- start start 2023-11-06T03:15:44.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 0x7f1138151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11381523c0 0x7f11381f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1138152850 con 0x7f11380cec10 2023-11-06T03:15:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11381559f0 con 0x7f11380ceff0 2023-11-06T03:15:44.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11381f0d90 con 0x7f11381523c0 2023-11-06T03:15:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38192/0 (socket says 172.21.15.157:38192) 2023-11-06T03:15:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 -- 172.21.15.157:0/2766788245 learned_addr learned my addr 172.21.15.157:0/2766788245 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113daa8640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 0x7f1138151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 -- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11381523c0 msgr2=0x7f11381f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11381523c0 0x7f11381f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 -- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 msgr2=0x7f1138151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 0x7f1138151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 -- 172.21.15.157:0/2766788245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11381f1070 con 0x7f11380cec10 2023-11-06T03:15:44.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113daa8640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 0x7f1138151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f113e2a9640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f1134010980 tx=0x7f1134010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134004220 con 0x7f11380cec10 2023-11-06T03:15:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134004380 con 0x7f11380cec10 2023-11-06T03:15:44.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1138158250 con 0x7f11380cec10 2023-11-06T03:15:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.664+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400eba0 con 0x7f11380cec10 2023-11-06T03:15:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1138158810 con 0x7f11380cec10 2023-11-06T03:15:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11381589d0 con 0x7f11380cec10 2023-11-06T03:15:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f113400ed00 con 0x7f11380cec10 2023-11-06T03:15:44.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f112effd640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 0x7f111c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1134083710 con 0x7f11380cec10 2023-11-06T03:15:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f113daa8640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 0x7f111c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.668+0000 7f113daa8640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 0x7f111c067f50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11380acb30 tx=0x7f1128005e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:44.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.672+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f11340512a0 con 0x7f11380cec10 2023-11-06T03:15:44.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f11381580b0 con 0x7f11380cec10 2023-11-06T03:15:44.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f112effd640 1 -- 172.21.15.157:0/2766788245 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f11381580b0 con 0x7f11380cec10 2023-11-06T03:15:44.881 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:44.881 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:44.881 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:15:44.881 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.881 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:15:44.882 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:15:44.882 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.882 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.882 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:15:44.882 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:15:44.883 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.883 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.883 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.883 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.883 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:15:44.884 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:15:44.884 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.884 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.884 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:15:44.884 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:15:44.885 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.885 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.885 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.885 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.885 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:15:44.886 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:15:44.886 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.886 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.886 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:15:44.887 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:15:44.887 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.887 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.887 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.887 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:15:44.888 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:15:44.889 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.889 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.889 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.889 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.889 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:15:44.890 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:15:44.890 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.890 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.890 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.890 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.891 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.891 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.891 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.891 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:15:44.891 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.892 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:15:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:15:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.893 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.894 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.894 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.894 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.894 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.894 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.895 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:15:44.895 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:15:44.895 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.895 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.895 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.896 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:15:44.897 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:15:44.897 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.897 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.897 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.897 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.898 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.898 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.898 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.898 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:15:44.898 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.899 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.900 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.900 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:15:44.900 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:15:44.900 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.900 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.901 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:15:44.902 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:15:44.902 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.902 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:15:44.902 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:15:44.902 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.903 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.903 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.903 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.903 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:15:44.903 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.904 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.905 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.905 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:15:44.905 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:15:44.905 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.905 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.906 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:15:44.906 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:15:44.906 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:15:44.906 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.907 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:15:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:15:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:15:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.908 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:15:44.909 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:15:44.910 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:15:44.910 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.910 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.910 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.910 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:15:44.911 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:15:44.911 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.911 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:15:44.911 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:15:44.911 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:15:44.912 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.912 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.912 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.912 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:15:44.913 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.914 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.914 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:44.914 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:44.914 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:15:44.914 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:15:44.915 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:15:44.915 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:15:44.915 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:15:44.915 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:15:44.915 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.916 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:44.916 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:15:44.916 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 msgr2=0x7f111c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 0x7f111c067f50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f11380acb30 tx=0x7f1128005e30 comp rx=0 tx=0).stop 2023-11-06T03:15:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 msgr2=0x7f1138158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f1134010980 tx=0x7f1134010e40 comp rx=0 tx=0).stop 2023-11-06T03:15:44.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 shutdown_connections 2023-11-06T03:15:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f111c065a90 0x7f111c067f50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f11381523c0 0x7f11381f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f11380ceff0 0x7f1138151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 --2- 172.21.15.157:0/2766788245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11380cec10 0x7f1138158d30 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.884+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 >> 172.21.15.157:0/2766788245 conn(0x7f1138065930 msgr2=0x7f113806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.888+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 shutdown_connections 2023-11-06T03:15:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:44.888+0000 7f114050d640 1 -- 172.21.15.157:0/2766788245 wait complete. 2023-11-06T03:15:44.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:45.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.044+0000 7f5fffbbf640 1 Processor -- start 2023-11-06T03:15:45.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5fffbbf640 1 -- start start 2023-11-06T03:15:45.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5fffbbf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5ff80b2af0 0x7f5ff80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5fffbbf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff815e440 con 0x7f5ff806d000 2023-11-06T03:15:45.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5fffbbf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff806c240 con 0x7f5ff81599c0 2023-11-06T03:15:45.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5fffbbf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff806c520 con 0x7f5ff80b2af0 2023-11-06T03:15:45.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ff806d000 0x7f5ff8159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59610/0 (socket says 172.21.15.157:59610) 2023-11-06T03:15:45.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 -- 172.21.15.157:0/95472099 learned_addr learned my addr 172.21.15.157:0/95472099 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:45.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/95472099 >> v2:172.21.15.195:3300/0 conn(0x7f5ff80b2af0 0x7f5ff80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926253915 0 0) 0x7f5ff815e440 con 0x7f5ff806d000 2023-11-06T03:15:45.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe4005740 con 0x7f5ff806d000 2023-11-06T03:15:45.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277109988 0 0) 0x7f5fe4005740 con 0x7f5ff806d000 2023-11-06T03:15:45.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 >> v2:172.21.15.195:3300/0 conn(0x7f5ff80b2af0 msgr2=0x7f5ff80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 --2- 172.21.15.157:0/95472099 >> v2:172.21.15.195:3300/0 conn(0x7f5ff80b2af0 0x7f5ff80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 >> v1:172.21.15.195:6789/0 conn(0x7f5ff81599c0 legacy=0x7f5ff80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff806c800 con 0x7f5ff806d000 2023-11-06T03:15:45.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5ff0001a00 con 0x7f5ff806d000 2023-11-06T03:15:45.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 >> v1:172.21.15.157:6789/0 conn(0x7f5ff806d000 legacy=0x7f5ff8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe4005b90 0x7f5fe4005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40069e0 0x7f5fe4006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 0x7f5fe400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff806c520 con 0x7f5fe40073a0 2023-11-06T03:15:45.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe400e240 con 0x7f5fe40069e0 2023-11-06T03:15:45.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe400e400 con 0x7f5fe4005b90 2023-11-06T03:15:45.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 0x7f5fe400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffd95b640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe4005b90 0x7f5fe4005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40069e0 0x7f5fe4006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 -- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe4005b90 msgr2=0x7f5fe4005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5fe4005b90 0x7f5fe4005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 -- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40069e0 msgr2=0x7f5fe4006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40069e0 0x7f5fe4006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffe15c640 1 -- 172.21.15.157:0/95472099 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4005740 con 0x7f5fe40073a0 2023-11-06T03:15:45.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.048+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40069e0 0x7f5fe4006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:45.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5ffe15c640 1 -- 172.21.15.157:0/95472099 reap_dead start 2023-11-06T03:15:45.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5ffe15c640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 0x7f5fe400dba0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000a800 tx=0x7f5ff001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c2f2ae828b31f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00072d0 con 0x7f5fe40073a0 2023-11-06T03:15:45.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00a8070 con 0x7f5fe40073a0 2023-11-06T03:15:45.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5ffc959640 1 -- 172.21.15.157:0/95472099 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0007690 con 0x7f5fe40073a0 2023-11-06T03:15:45.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 msgr2=0x7f5fe400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 0x7f5fe400dba0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000a800 tx=0x7f5ff001d300 comp rx=0 tx=0).stop 2023-11-06T03:15:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- 172.21.15.157:0/95472099 shutdown_connections 2023-11-06T03:15:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/95472099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5fe40073a0 0x7f5fe400dba0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- 172.21.15.157:0/95472099 >> 172.21.15.157:0/95472099 conn(0x7f5ff8065930 msgr2=0x7f5ff8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- 172.21.15.157:0/95472099 shutdown_connections 2023-11-06T03:15:45.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- 172.21.15.157:0/95472099 wait complete. 2023-11-06T03:15:45.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 Processor -- start 2023-11-06T03:15:45.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- start start 2023-11-06T03:15:45.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 0x7f5ff81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815a6c0 0x7f5ff815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8155ae0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80ad270 con 0x7f5fe40073a0 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.052+0000 7f5fffbbf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815b080 con 0x7f5ff815a6c0 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38214/0 (socket says 172.21.15.157:38214) 2023-11-06T03:15:45.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 -- 172.21.15.157:0/3520296914 learned_addr learned my addr 172.21.15.157:0/3520296914 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:45.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd95b640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 0x7f5ff81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffe15c640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815a6c0 0x7f5ff815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 -- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815a6c0 msgr2=0x7f5ff815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815a6c0 0x7f5ff815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 -- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 msgr2=0x7f5ff81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 0x7f5ff81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 -- 172.21.15.157:0/3520296914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff81effc0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd95b640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 0x7f5ff81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:45.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd15a640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80108d0 tx=0x7f5fe8010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004220 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8004380 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff81f32e0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe800eba0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81f3820 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8072750 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5fe800ed00 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 0x7f5fdc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5fe8082f00 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd95b640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 0x7f5fdc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.056+0000 7f5ffd95b640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 0x7f5fdc067f50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8071580 tx=0x7f5ff4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.060+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5fe8050ab0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.216+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ff80cb110 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.216+0000 7f5fee7fc640 1 -- 172.21.15.157:0/3520296914 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+14229 (secure 0 0 0) 0x7f5fe80540d0 con 0x7f5ff80ac8b0 2023-11-06T03:15:45.215 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:45.215 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":145,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:44.454716+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:45.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 msgr2=0x7f5fdc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 0x7f5fdc067f50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8071580 tx=0x7f5ff4009040 comp rx=0 tx=0).stop 2023-11-06T03:15:45.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 msgr2=0x7f5ff80acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80108d0 tx=0x7f5fe8010d90 comp rx=0 tx=0).stop 2023-11-06T03:15:45.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 shutdown_connections 2023-11-06T03:15:45.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5fdc065a90 0x7f5fdc067f50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ff815a6c0 0x7f5ff815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ff80ac8b0 0x7f5ff80acd30 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 --2- 172.21.15.157:0/3520296914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5fe40073a0 0x7f5ff81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 >> 172.21.15.157:0/3520296914 conn(0x7f5ff8065930 msgr2=0x7f5ff815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 shutdown_connections 2023-11-06T03:15:45.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.220+0000 7f5fffbbf640 1 -- 172.21.15.157:0/3520296914 wait complete. 2023-11-06T03:15:45.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:15:45.235 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:15:45.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 Processor -- start 2023-11-06T03:15:45.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 -- start start 2023-11-06T03:15:45.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe77415e5e0 0x7fe7740b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77415f580 con 0x7fe7741551e0 2023-11-06T03:15:45.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77415efc0 con 0x7fe77415ea90 2023-11-06T03:15:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7792b0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7740704b0 con 0x7fe77415e5e0 2023-11-06T03:15:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7727fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe77415e5e0 0x7fe7740b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe772ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe7741551e0 0x7fe77406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59612/0 (socket says 172.21.15.157:59612) 2023-11-06T03:15:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe772ffd640 1 -- 172.21.15.157:0/1993818410 learned_addr learned my addr 172.21.15.157:0/1993818410 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:45.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7727fc640 1 -- 172.21.15.157:0/1993818410 >> v1:172.21.15.195:6789/0 conn(0x7fe77415ea90 legacy=0x7fe77415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7727fc640 1 -- 172.21.15.157:0/1993818410 >> v1:172.21.15.157:6789/0 conn(0x7fe7741551e0 legacy=0x7fe77406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.352+0000 7fe7727fc640 1 -- 172.21.15.157:0/1993818410 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77415f2a0 con 0x7fe77415e5e0 2023-11-06T03:15:45.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 --2- 172.21.15.157:0/1993818410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe7740b3410 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fe7680097c0 tx=0x7fe76801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5827e00ce32ecbc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe771ffb640 1 -- 172.21.15.157:0/1993818410 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ab600 con 0x7fe77415e5e0 2023-11-06T03:15:45.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe771ffb640 1 -- 172.21.15.157:0/1993818410 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680a7050 con 0x7fe77415e5e0 2023-11-06T03:15:45.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe771ffb640 1 -- 172.21.15.157:0/1993818410 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ab980 con 0x7fe77415e5e0 2023-11-06T03:15:45.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- 172.21.15.157:0/1993818410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 msgr2=0x7fe7740b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 --2- 172.21.15.157:0/1993818410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe7740b3410 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fe7680097c0 tx=0x7fe76801c380 comp rx=0 tx=0).stop 2023-11-06T03:15:45.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- 172.21.15.157:0/1993818410 shutdown_connections 2023-11-06T03:15:45.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 --2- 172.21.15.157:0/1993818410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe7740b3410 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- 172.21.15.157:0/1993818410 >> 172.21.15.157:0/1993818410 conn(0x7fe774065930 msgr2=0x7fe774065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- 172.21.15.157:0/1993818410 shutdown_connections 2023-11-06T03:15:45.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- 172.21.15.157:0/1993818410 wait complete. 2023-11-06T03:15:45.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 Processor -- start 2023-11-06T03:15:45.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- start start 2023-11-06T03:15:45.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7741551e0 0x7fe77415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 0x7fe774149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77414c000 con 0x7fe77415e5e0 2023-11-06T03:15:45.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77414b380 con 0x7fe77415ea90 2023-11-06T03:15:45.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7792b0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77414b660 con 0x7fe7741551e0 2023-11-06T03:15:45.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38232/0 (socket says 172.21.15.157:38232) 2023-11-06T03:15:45.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 -- 172.21.15.157:0/3238328790 learned_addr learned my addr 172.21.15.157:0/3238328790 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:45.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe772ffd640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7741551e0 0x7fe77415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 -- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7741551e0 msgr2=0x7fe77415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7741551e0 0x7fe77415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 -- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 msgr2=0x7fe774149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7737fe640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 0x7fe774149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 0x7fe774149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7727fc640 1 -- 172.21.15.157:0/3238328790 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009420 con 0x7fe77415e5e0 2023-11-06T03:15:45.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.356+0000 7fe7737fe640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 0x7fe774149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:45.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe7727fc640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ae040 tx=0x7fe768019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b4840 con 0x7fe77415e5e0 2023-11-06T03:15:45.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680b49a0 con 0x7fe77415e5e0 2023-11-06T03:15:45.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b4d50 con 0x7fe77415e5e0 2023-11-06T03:15:45.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77414b940 con 0x7fe77415e5e0 2023-11-06T03:15:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77415a990 con 0x7fe77415e5e0 2023-11-06T03:15:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77414c140 con 0x7fe77415e5e0 2023-11-06T03:15:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe7680b2040 con 0x7fe77415e5e0 2023-11-06T03:15:45.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 0x7fe760067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe76811d280 con 0x7fe77415e5e0 2023-11-06T03:15:45.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe772ffd640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 0x7fe760067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.360+0000 7fe772ffd640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 0x7fe760067f20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe764005890 tx=0x7fe764009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.364+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe77414c140 con 0x7fe77415e5e0 2023-11-06T03:15:45.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.520+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe77414c140 con 0x7fe77415e5e0 2023-11-06T03:15:45.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.520+0000 7fe75b7fe640 1 -- 172.21.15.157:0/3238328790 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7680b8020 con 0x7fe77415e5e0 2023-11-06T03:15:45.517 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:15:45.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.520+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 msgr2=0x7fe760067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.520+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 0x7fe760067f20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe764005890 tx=0x7fe764009040 comp rx=0 tx=0).stop 2023-11-06T03:15:45.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 msgr2=0x7fe77414c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ae040 tx=0x7fe768019120 comp rx=0 tx=0).stop 2023-11-06T03:15:45.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 shutdown_connections 2023-11-06T03:15:45.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe760065a60 0x7fe760067f20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe77415ea90 0x7fe774149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe77415e5e0 0x7fe77414c420 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 --2- 172.21.15.157:0/3238328790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7741551e0 0x7fe77415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 >> 172.21.15.157:0/3238328790 conn(0x7fe774065930 msgr2=0x7fe77415ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 shutdown_connections 2023-11-06T03:15:45.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.524+0000 7fe7792b0640 1 -- 172.21.15.157:0/3238328790 wait complete. 2023-11-06T03:15:45.536 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:15:45.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 Processor -- start 2023-11-06T03:15:45.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 -- start start 2023-11-06T03:15:45.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f98540b2af0 0x7f98540ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985415e440 con 0x7f98541599c0 2023-11-06T03:15:45.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985406c240 con 0x7f985406d000 2023-11-06T03:15:45.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f985a801640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985406c520 con 0x7f98540b2af0 2023-11-06T03:15:45.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9853fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f98541599c0 0x7f98540b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59624/0 (socket says 172.21.15.157:59624) 2023-11-06T03:15:45.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9853fff640 1 -- 172.21.15.157:0/3510054881 learned_addr learned my addr 172.21.15.157:0/3510054881 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:45.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017038049 0 0) 0x7f985415e440 con 0x7f98541599c0 2023-11-06T03:15:45.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9840005740 con 0x7f98541599c0 2023-11-06T03:15:45.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f98537fe640 1 --2- 172.21.15.157:0/3510054881 >> v2:172.21.15.195:3300/0 conn(0x7f98540b2af0 0x7f98540ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621978147 0 0) 0x7f9840005740 con 0x7f98541599c0 2023-11-06T03:15:45.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 >> v2:172.21.15.195:3300/0 conn(0x7f98540b2af0 msgr2=0x7f98540ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 --2- 172.21.15.157:0/3510054881 >> v2:172.21.15.195:3300/0 conn(0x7f98540b2af0 0x7f98540ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 >> v1:172.21.15.195:6789/0 conn(0x7f985406d000 legacy=0x7f9854159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985406c800 con 0x7f98541599c0 2023-11-06T03:15:45.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9844001680 con 0x7f98541599c0 2023-11-06T03:15:45.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 >> v1:172.21.15.157:6789/0 conn(0x7f98541599c0 legacy=0x7f98540b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9840005b90 0x7f9840005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400069e0 0x7f9840006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 0x7f984000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985406c520 con 0x7f98400073a0 2023-11-06T03:15:45.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984000e2f0 con 0x7f9840005b90 2023-11-06T03:15:45.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984000e4b0 con 0x7f98400069e0 2023-11-06T03:15:45.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 0x7f984000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9853fff640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9840005b90 0x7f9840005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f98537fe640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400069e0 0x7f9840006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 -- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400069e0 msgr2=0x7f9840006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400069e0 0x7f9840006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 -- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9840005b90 msgr2=0x7f9840005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9840005b90 0x7f9840005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.668+0000 7f9858d9e640 1 -- 172.21.15.157:0/3510054881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840005740 con 0x7f98400073a0 2023-11-06T03:15:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 -- 172.21.15.157:0/3510054881 reap_dead start 2023-11-06T03:15:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 0x7f984000dba0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f984c001e40 tx=0x7f984c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d05316fec02405 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c003b80 con 0x7f98400073a0 2023-11-06T03:15:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c003ce0 con 0x7f98400073a0 2023-11-06T03:15:45.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9852ffd640 1 -- 172.21.15.157:0/3510054881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c005260 con 0x7f98400073a0 2023-11-06T03:15:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 msgr2=0x7f984000dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 0x7f984000dba0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f984c001e40 tx=0x7f984c01c740 comp rx=0 tx=0).stop 2023-11-06T03:15:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/3510054881 shutdown_connections 2023-11-06T03:15:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 --2- 172.21.15.157:0/3510054881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98400073a0 0x7f984000dba0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/3510054881 >> 172.21.15.157:0/3510054881 conn(0x7f9854065930 msgr2=0x7f9854065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/3510054881 shutdown_connections 2023-11-06T03:15:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/3510054881 wait complete. 2023-11-06T03:15:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 Processor -- start 2023-11-06T03:15:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- start start 2023-11-06T03:15:45.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 0x7f985415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 0x7f98540cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541ee380 con 0x7f98540cbc30 2023-11-06T03:15:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854159e40 con 0x7f98541edf20 2023-11-06T03:15:45.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985415a120 con 0x7f98400073a0 2023-11-06T03:15:45.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38248/0 (socket says 172.21.15.157:38248) 2023-11-06T03:15:45.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 -- 172.21.15.157:0/2413689639 learned_addr learned my addr 172.21.15.157:0/2413689639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:45.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f98537fe640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 0x7f98540cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 -- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 msgr2=0x7f985415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9853fff640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 0x7f985415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 0x7f985415ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 -- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 msgr2=0x7f98540cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 0x7f98540cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 -- 172.21.15.157:0/2413689639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985415a400 con 0x7f98540cbc30 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f98537fe640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 0x7f98540cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9853fff640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 0x7f985415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9858d9e640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f984c01c570 tx=0x7f9840007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c004fb0 con 0x7f98540cbc30 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c005110 con 0x7f98540cbc30 2023-11-06T03:15:45.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985415a6e0 con 0x7f98540cbc30 2023-11-06T03:15:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.672+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0054a0 con 0x7f98540cbc30 2023-11-06T03:15:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98540ce3c0 con 0x7f98540cbc30 2023-11-06T03:15:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98541ee300 con 0x7f98540cbc30 2023-11-06T03:15:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f984c005600 con 0x7f98540cbc30 2023-11-06T03:15:45.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f9850ff9640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 0x7f9834067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f984c11c030 con 0x7f98540cbc30 2023-11-06T03:15:45.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f9853fff640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 0x7f9834067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.676+0000 7f9853fff640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 0x7f9834067f20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9844001ec0 tx=0x7f9844000f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.680+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f984c0e9c70 con 0x7f98540cbc30 2023-11-06T03:15:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f985415a8d0 con 0x7f98540cbc30 2023-11-06T03:15:45.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f9850ff9640 1 -- 172.21.15.157:0/2413689639 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v145) v1 ==== 92+0+10 (secure 0 0 0) 0x7f984c0b6020 con 0x7f98540cbc30 2023-11-06T03:15:45.829 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:15:45.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 msgr2=0x7f9834067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 0x7f9834067f20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9844001ec0 tx=0x7f9844000f60 comp rx=0 tx=0).stop 2023-11-06T03:15:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 msgr2=0x7f98541f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f984c01c570 tx=0x7f9840007a10 comp rx=0 tx=0).stop 2023-11-06T03:15:45.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 shutdown_connections 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9834065a60 0x7f9834067f20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98540cbc30 0x7f98541f0890 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f98541edf20 0x7f98540cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 --2- 172.21.15.157:0/2413689639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98400073a0 0x7f985415ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 >> 172.21.15.157:0/2413689639 conn(0x7f9854065930 msgr2=0x7f985406f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 shutdown_connections 2023-11-06T03:15:45.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.832+0000 7f985a801640 1 -- 172.21.15.157:0/2413689639 wait complete. 2023-11-06T03:15:45.846 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:15:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.964+0000 7f3572995640 1 Processor -- start 2023-11-06T03:15:45.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.964+0000 7f3572995640 1 -- start start 2023-11-06T03:15:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3572995640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f356c15cb10 0x7f356c1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3572995640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f356c0ccf20 0x7f356c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3572995640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c1617d0 con 0x7f356c15cef0 2023-11-06T03:15:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3572995640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c161210 con 0x7f356c0ccf20 2023-11-06T03:15:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3572995640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c1614f0 con 0x7f356c15cb10 2023-11-06T03:15:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f356c0ccf20 0x7f356c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f356c0ccf20 0x7f356c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38254/0 (socket says 172.21.15.157:38254) 2023-11-06T03:15:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 -- 172.21.15.157:0/2784692892 learned_addr learned my addr 172.21.15.157:0/2784692892 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f3570f32640 1 --1- 172.21.15.157:0/2784692892 >> v1:172.21.15.157:6789/0 conn(0x7f356c15cef0 0x7f356c0b2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59628/0 (socket says 172.21.15.157:59628) 2023-11-06T03:15:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356bfff640 1 --2- 172.21.15.157:0/2784692892 >> v2:172.21.15.195:3300/0 conn(0x7f356c15cb10 0x7f356c1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 -- 172.21.15.157:0/2784692892 >> v2:172.21.15.195:3300/0 conn(0x7f356c15cb10 msgr2=0x7f356c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:45.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 --2- 172.21.15.157:0/2784692892 >> v2:172.21.15.195:3300/0 conn(0x7f356c15cb10 0x7f356c1608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 -- 172.21.15.157:0/2784692892 >> v1:172.21.15.157:6789/0 conn(0x7f356c15cef0 legacy=0x7f356c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 -- 172.21.15.157:0/2784692892 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0cb190 con 0x7f356c0ccf20 2023-11-06T03:15:45.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356affd640 1 -- 172.21.15.157:0/2784692892 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265218747 0 0) 0x7f356c1617d0 con 0x7f356c15cef0 2023-11-06T03:15:45.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356b7fe640 1 --2- 172.21.15.157:0/2784692892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c0ccf20 0x7f356c0cd390 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f356c0d0970 tx=0x7f356001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b7457a198b5278 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356affd640 1 -- 172.21.15.157:0/2784692892 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600ab660 con 0x7f356c0ccf20 2023-11-06T03:15:45.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356affd640 1 -- 172.21.15.157:0/2784692892 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35600ab7c0 con 0x7f356c0ccf20 2023-11-06T03:15:45.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.968+0000 7f356affd640 1 -- 172.21.15.157:0/2784692892 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600abb50 con 0x7f356c0ccf20 2023-11-06T03:15:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/2784692892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c0ccf20 msgr2=0x7f356c0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- 172.21.15.157:0/2784692892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c0ccf20 0x7f356c0cd390 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f356c0d0970 tx=0x7f356001c440 comp rx=0 tx=0).stop 2023-11-06T03:15:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/2784692892 shutdown_connections 2023-11-06T03:15:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- 172.21.15.157:0/2784692892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c0ccf20 0x7f356c0cd390 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- 172.21.15.157:0/2784692892 >> v2:172.21.15.195:3300/0 conn(0x7f356c15cb10 0x7f356c1608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/2784692892 >> 172.21.15.157:0/2784692892 conn(0x7f356c06d560 msgr2=0x7f356c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:45.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/2784692892 shutdown_connections 2023-11-06T03:15:45.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/2784692892 wait complete. 2023-11-06T03:15:45.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 Processor -- start 2023-11-06T03:15:45.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- start start 2023-11-06T03:15:45.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f356c0ccf20 0x7f356c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f356c15cb10 0x7f356c154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c15b730 con 0x7f356c15cef0 2023-11-06T03:15:45.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0b44b0 con 0x7f356c15cb10 2023-11-06T03:15:45.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c1546c0 con 0x7f356c0ccf20 2023-11-06T03:15:45.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38258/0 (socket says 172.21.15.157:38258) 2023-11-06T03:15:45.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 -- 172.21.15.157:0/1403171789 learned_addr learned my addr 172.21.15.157:0/1403171789 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:45.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 -- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f356c0ccf20 msgr2=0x7f356c0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f356bfff640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f356c0ccf20 0x7f356c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f356c0ccf20 0x7f356c0b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 -- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f356c15cb10 msgr2=0x7f356c154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:45.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f356b7fe640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f356c15cb10 0x7f356c154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f356c15cb10 0x7f356c154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:45.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 -- 172.21.15.157:0/1403171789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600094d0 con 0x7f356c15cef0 2023-11-06T03:15:45.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3570f32640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f356400c7b0 tx=0x7f356400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400a640 con 0x7f356c15cef0 2023-11-06T03:15:45.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356400a7c0 con 0x7f356c15cef0 2023-11-06T03:15:45.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c15d860 con 0x7f356c15cef0 2023-11-06T03:15:45.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400ab70 con 0x7f356c15cef0 2023-11-06T03:15:45.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c15de20 con 0x7f356c15cef0 2023-11-06T03:15:45.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.972+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c072750 con 0x7f356c15cef0 2023-11-06T03:15:45.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.976+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f356400acd0 con 0x7f356c15cef0 2023-11-06T03:15:45.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.976+0000 7f3568ff9640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:45.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.976+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3564082340 con 0x7f356c15cef0 2023-11-06T03:15:45.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.976+0000 7f356bfff640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:45.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.976+0000 7f356bfff640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 0x7f3558067f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f356c0c72b0 tx=0x7f355c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:45.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:45.980+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f356404ff80 con 0x7f356c15cef0 2023-11-06T03:15:46.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f356c0b6990 con 0x7f356c15cef0 2023-11-06T03:15:46.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3568ff9640 1 -- 172.21.15.157:0/1403171789 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3564018090 con 0x7f356c15cef0 2023-11-06T03:15:46.120 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:46.120 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:15:46.120 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:15:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:15:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:15:46.121 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:15:46.122 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:15:46.122 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:46.122 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:15:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 msgr2=0x7f3558067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 0x7f3558067f20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f356c0c72b0 tx=0x7f355c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:46.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 msgr2=0x7f356c15f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f356400c7b0 tx=0x7f356400cc70 comp rx=0 tx=0).stop 2023-11-06T03:15:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 shutdown_connections 2023-11-06T03:15:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3558065a60 0x7f3558067f20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356c15cef0 0x7f356c15f030 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f356c15cb10 0x7f356c154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 --2- 172.21.15.157:0/1403171789 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f356c0ccf20 0x7f356c0b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 >> 172.21.15.157:0/1403171789 conn(0x7f356c06d560 msgr2=0x7f356c069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 shutdown_connections 2023-11-06T03:15:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.124+0000 7f3572995640 1 -- 172.21.15.157:0/1403171789 wait complete. 2023-11-06T03:15:46.137 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:15:46.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:15:46.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 Processor -- start 2023-11-06T03:15:46.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 -- start start 2023-11-06T03:15:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f42841593d0 0x7f42840ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f428406f290 0x7f42840cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42840cb650 0x7f42840b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840cbb00 con 0x7f428406f290 2023-11-06T03:15:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42841597b0 con 0x7f42841593d0 2023-11-06T03:15:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4288be7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42840b2b20 con 0x7f42840cb650 2023-11-06T03:15:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f428406f290 0x7f42840cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f428406f290 0x7f42840cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38260/0 (socket says 172.21.15.157:38260) 2023-11-06T03:15:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 -- 172.21.15.157:0/1451630354 learned_addr learned my addr 172.21.15.157:0/1451630354 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4282d9d640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42840cb650 0x7f42840b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f428259c640 1 --2- 172.21.15.157:0/1451630354 >> v2:172.21.15.195:3300/0 conn(0x7f42841593d0 0x7f42840ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 -- 172.21.15.157:0/1451630354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42840cb650 msgr2=0x7f42840b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42840cb650 0x7f42840b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 -- 172.21.15.157:0/1451630354 >> v2:172.21.15.195:3300/0 conn(0x7f42841593d0 msgr2=0x7f42840ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1451630354 >> v2:172.21.15.195:3300/0 conn(0x7f42841593d0 0x7f42840ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.260+0000 7f4281d9b640 1 -- 172.21.15.157:0/1451630354 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840b2e00 con 0x7f428406f290 2023-11-06T03:15:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840cb110 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f4284066640 tx=0x7f427801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea036cc165e53ef server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f428159a640 1 -- 172.21.15.157:0/1451630354 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780ab640 con 0x7f428406f290 2023-11-06T03:15:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f428159a640 1 -- 172.21.15.157:0/1451630354 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42780a7070 con 0x7f428406f290 2023-11-06T03:15:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f428159a640 1 -- 172.21.15.157:0/1451630354 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780ab9c0 con 0x7f428406f290 2023-11-06T03:15:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 -- 172.21.15.157:0/1451630354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 msgr2=0x7f42840cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840cb110 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f4284066640 tx=0x7f427801c500 comp rx=0 tx=0).stop 2023-11-06T03:15:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 -- 172.21.15.157:0/1451630354 shutdown_connections 2023-11-06T03:15:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42840cb650 0x7f42840b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 --2- 172.21.15.157:0/1451630354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840cb110 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 --2- 172.21.15.157:0/1451630354 >> v2:172.21.15.195:3300/0 conn(0x7f42841593d0 0x7f42840ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 -- 172.21.15.157:0/1451630354 >> 172.21.15.157:0/1451630354 conn(0x7f42840b43b0 msgr2=0x7f4284065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 -- 172.21.15.157:0/1451630354 shutdown_connections 2023-11-06T03:15:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.264+0000 7f4288be7640 1 -- 172.21.15.157:0/1451630354 wait complete. 2023-11-06T03:15:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 Processor -- start 2023-11-06T03:15:46.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- start start 2023-11-06T03:15:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 0x7f4284153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42841593d0 0x7f428414eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428415ade0 con 0x7f428406f290 2023-11-06T03:15:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428415b0c0 con 0x7f42840cb650 2023-11-06T03:15:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428414ccc0 con 0x7f42841593d0 2023-11-06T03:15:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38276/0 (socket says 172.21.15.157:38276) 2023-11-06T03:15:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 -- 172.21.15.157:0/1688366516 learned_addr learned my addr 172.21.15.157:0/1688366516 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 0x7f4284153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 -- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42841593d0 msgr2=0x7f428414eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42841593d0 0x7f428414eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 -- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 msgr2=0x7f4284153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 0x7f4284153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 -- 172.21.15.157:0/1688366516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780094d0 con 0x7f428406f290 2023-11-06T03:15:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 0x7f4284153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f428259c640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f427400d5f0 tx=0x7f427400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42740125f0 con 0x7f428406f290 2023-11-06T03:15:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274012750 con 0x7f428406f290 2023-11-06T03:15:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428414cfa0 con 0x7f428406f290 2023-11-06T03:15:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274012ae0 con 0x7f428406f290 2023-11-06T03:15:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4284140ec0 con 0x7f428406f290 2023-11-06T03:15:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.268+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428404e050 con 0x7f428406f290 2023-11-06T03:15:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.272+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4274012c40 con 0x7f428406f290 2023-11-06T03:15:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.272+0000 7f4272ffd640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 0x7f4264067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.272+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4274083460 con 0x7f428406f290 2023-11-06T03:15:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.272+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 0x7f4264067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.272+0000 7f4281d9b640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 0x7f4264067f20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42780af000 tx=0x7f4278008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.276+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4274051010 con 0x7f428406f290 2023-11-06T03:15:46.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.416+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42840b2410 con 0x7f428406f290 2023-11-06T03:15:46.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.464+0000 7f4272ffd640 1 -- 172.21.15.157:0/1688366516 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4274007380 con 0x7f428406f290 2023-11-06T03:15:46.461 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:15:46.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 msgr2=0x7f4264067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 0x7f4264067f20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42780af000 tx=0x7f4278008040 comp rx=0 tx=0).stop 2023-11-06T03:15:46.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 msgr2=0x7f42840ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f427400d5f0 tx=0x7f427400dab0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 shutdown_connections 2023-11-06T03:15:46.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4264065a60 0x7f4264067f20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42841593d0 0x7f428414eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42840cb650 0x7f4284153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 --2- 172.21.15.157:0/1688366516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f428406f290 0x7f42840ccc00 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 >> 172.21.15.157:0/1688366516 conn(0x7f42840b43b0 msgr2=0x7f42840cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:46.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 shutdown_connections 2023-11-06T03:15:46.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.468+0000 7f4288be7640 1 -- 172.21.15.157:0/1688366516 wait complete. 2023-11-06T03:15:46.484 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:15:46.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 Processor -- start 2023-11-06T03:15:46.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 -- start start 2023-11-06T03:15:46.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 0x7fde880ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 0x7fde8815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde880b2020 con 0x7fde8815aae0 2023-11-06T03:15:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880b2840 con 0x7fde8815a700 2023-11-06T03:15:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8c849640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880cc220 con 0x7fde88066040 2023-11-06T03:15:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde86d9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 0x7fde8815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8659c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 0x7fde880ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde85d9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fde8815aae0 0x7fde88065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48486/0 (socket says 172.21.15.157:48486) 2023-11-06T03:15:46.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde85d9b640 1 -- 172.21.15.157:0/142448362 learned_addr learned my addr 172.21.15.157:0/142448362 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:46.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde86d9d640 1 -- 172.21.15.157:0/142448362 >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 msgr2=0x7fde880ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:46.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde86d9d640 1 --2- 172.21.15.157:0/142448362 >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 0x7fde880ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde86d9d640 1 -- 172.21.15.157:0/142448362 >> v1:172.21.15.195:6789/0 conn(0x7fde8815aae0 legacy=0x7fde88065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde86d9d640 1 -- 172.21.15.157:0/142448362 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8815def0 con 0x7fde88066040 2023-11-06T03:15:46.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.604+0000 7fde8659c640 1 --2- 172.21.15.157:0/142448362 >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 0x7fde880ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:15:46.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde86d9d640 1 --2- 172.21.15.157:0/142448362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 0x7fde8815d2e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2c3fefe985092c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8559a640 1 -- 172.21.15.157:0/142448362 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800a5a00 con 0x7fde88066040 2023-11-06T03:15:46.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8559a640 1 -- 172.21.15.157:0/142448362 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde800a5b60 con 0x7fde88066040 2023-11-06T03:15:46.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8559a640 1 -- 172.21.15.157:0/142448362 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800ad440 con 0x7fde88066040 2023-11-06T03:15:46.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- 172.21.15.157:0/142448362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 msgr2=0x7fde8815d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 --2- 172.21.15.157:0/142448362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 0x7fde8815d2e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde80009d90 tx=0x7fde8001c920 comp rx=0 tx=0).stop 2023-11-06T03:15:46.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- 172.21.15.157:0/142448362 shutdown_connections 2023-11-06T03:15:46.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 --2- 172.21.15.157:0/142448362 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde88066040 0x7fde8815d2e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 --2- 172.21.15.157:0/142448362 >> v2:172.21.15.195:3300/0 conn(0x7fde8815a700 0x7fde880ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- 172.21.15.157:0/142448362 >> 172.21.15.157:0/142448362 conn(0x7fde8806a170 msgr2=0x7fde8806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:46.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- 172.21.15.157:0/142448362 shutdown_connections 2023-11-06T03:15:46.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- 172.21.15.157:0/142448362 wait complete. 2023-11-06T03:15:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 Processor -- start 2023-11-06T03:15:46.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.608+0000 7fde8c849640 1 -- start start 2023-11-06T03:15:46.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde8815a700 0x7fde88153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde8815aae0 0x7fde88158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c7550 con 0x7fde88066040 2023-11-06T03:15:46.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c6940 con 0x7fde8815a700 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880c6c20 con 0x7fde8815aae0 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38284/0 (socket says 172.21.15.157:38284) 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 -- 172.21.15.157:0/1904152756 learned_addr learned my addr 172.21.15.157:0/1904152756 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde85d9b640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde8815a700 0x7fde88153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde86d9d640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde8815aae0 0x7fde88158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 -- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde8815aae0 msgr2=0x7fde88158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde8815aae0 0x7fde88158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 -- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde8815a700 msgr2=0x7fde88153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde8815a700 0x7fde88153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 -- 172.21.15.157:0/1904152756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c009420 con 0x7fde88066040 2023-11-06T03:15:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8659c640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00ebe0 tx=0x7fde7c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00eee0 con 0x7fde88066040 2023-11-06T03:15:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c00cc30 con 0x7fde88066040 2023-11-06T03:15:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80009a70 con 0x7fde88066040 2023-11-06T03:15:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c013470 con 0x7fde88066040 2023-11-06T03:15:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde880c71e0 con 0x7fde88066040 2023-11-06T03:15:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde88072750 con 0x7fde88066040 2023-11-06T03:15:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fde7c0135d0 con 0x7fde88066040 2023-11-06T03:15:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 0x7fde68067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fde7c083c20 con 0x7fde88066040 2023-11-06T03:15:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.612+0000 7fde85d9b640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 0x7fde68067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.616+0000 7fde85d9b640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 0x7fde68067f50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fde88157c30 tx=0x7fde78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.616+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fde7c051660 con 0x7fde88066040 2023-11-06T03:15:46.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:46.764+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fde88070f70 con 0x7fde88066040 2023-11-06T03:15:47.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.476+0000 7fde76ffd640 1 -- 172.21.15.157:0/1904152756 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) v1 ==== 170+0+0 (secure 0 0 0) 0x7fde7c054c80 con 0x7fde88066040 2023-11-06T03:15:47.472 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:15:47.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 msgr2=0x7fde68067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 0x7fde68067f50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fde88157c30 tx=0x7fde78009040 comp rx=0 tx=0).stop 2023-11-06T03:15:47.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 msgr2=0x7fde88154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00ebe0 tx=0x7fde7c00c520 comp rx=0 tx=0).stop 2023-11-06T03:15:47.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 shutdown_connections 2023-11-06T03:15:47.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fde68065a90 0x7fde68067f50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fde8815aae0 0x7fde88158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde8815a700 0x7fde88153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 --2- 172.21.15.157:0/1904152756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde88066040 0x7fde88154700 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.484+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 >> 172.21.15.157:0/1904152756 conn(0x7fde8806a170 msgr2=0x7fde880cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:47.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.492+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 shutdown_connections 2023-11-06T03:15:47.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.492+0000 7fde8c849640 1 -- 172.21.15.157:0/1904152756 wait complete. 2023-11-06T03:15:47.513 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:15:47.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 Processor -- start 2023-11-06T03:15:47.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 -- start start 2023-11-06T03:15:47.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f766c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f766c0d0280 0x7f766c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0d2920 con 0x7f766c15ce80 2023-11-06T03:15:47.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0d18a0 con 0x7f766c0d0280 2023-11-06T03:15:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f767305a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0d1b60 con 0x7f766c15caa0 2023-11-06T03:15:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f766c15ce80 0x7f766c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59642/0 (socket says 172.21.15.157:59642) 2023-11-06T03:15:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766bfff640 1 -- 172.21.15.157:0/1240710454 learned_addr learned my addr 172.21.15.157:0/1240710454 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f7670df6640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f766c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 --2- 172.21.15.157:0/1240710454 >> v2:172.21.15.195:3300/0 conn(0x7f766c0d0280 0x7f766c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813020949 0 0) 0x7f766c0d2920 con 0x7f766c15ce80 2023-11-06T03:15:47.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c003530 con 0x7f766c15ce80 2023-11-06T03:15:47.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2251725299 0 0) 0x7f764c003530 con 0x7f766c15ce80 2023-11-06T03:15:47.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 msgr2=0x7f766c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:47.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f766c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 >> v2:172.21.15.195:3300/0 conn(0x7f766c0d0280 msgr2=0x7f766c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:47.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 --2- 172.21.15.157:0/1240710454 >> v2:172.21.15.195:3300/0 conn(0x7f766c0d0280 0x7f766c0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0d1e40 con 0x7f766c15ce80 2023-11-06T03:15:47.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7654001a00 con 0x7f766c15ce80 2023-11-06T03:15:47.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 >> v1:172.21.15.157:6789/0 conn(0x7f766c15ce80 legacy=0x7f766c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f764c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c003f00 0x7f764c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764c004d10 0x7f764c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0d18a0 con 0x7f764c004d10 2023-11-06T03:15:47.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0d1b60 con 0x7f764c003f00 2023-11-06T03:15:47.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00b990 con 0x7f766c15caa0 2023-11-06T03:15:47.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764c004d10 0x7f764c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f766bfff640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c003f00 0x7f764c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f7670df6640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f764c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 -- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 msgr2=0x7f764c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:47.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f764c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 -- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c003f00 msgr2=0x7f764c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c003f00 0x7f764c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f76715f7640 1 -- 172.21.15.157:0/1240710454 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c003530 con 0x7f764c004d10 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.636+0000 7f7670df6640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c15caa0 0x7f764c003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f76715f7640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764c004d10 0x7f764c005190 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ae0 tx=0x7f766401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab257d0ba7d5c491 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640a5680 con 0x7f764c004d10 2023-11-06T03:15:47.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76640a57e0 con 0x7f764c004d10 2023-11-06T03:15:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f766b7fe640 1 -- 172.21.15.157:0/1240710454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640a5b90 con 0x7f764c004d10 2023-11-06T03:15:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- 172.21.15.157:0/1240710454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764c004d10 msgr2=0x7f764c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 --2- 172.21.15.157:0/1240710454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f764c004d10 0x7f764c005190 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ae0 tx=0x7f766401ca30 comp rx=0 tx=0).stop 2023-11-06T03:15:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f76715f7640 1 -- 172.21.15.157:0/1240710454 reap_dead start 2023-11-06T03:15:47.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- 172.21.15.157:0/1240710454 shutdown_connections 2023-11-06T03:15:47.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- 172.21.15.157:0/1240710454 >> 172.21.15.157:0/1240710454 conn(0x7f766c070790 msgr2=0x7f766c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:47.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- 172.21.15.157:0/1240710454 shutdown_connections 2023-11-06T03:15:47.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- 172.21.15.157:0/1240710454 wait complete. 2023-11-06T03:15:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 Processor -- start 2023-11-06T03:15:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.640+0000 7f767305a640 1 -- start start 2023-11-06T03:15:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c004d10 0x7f766c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c0caba0 0x7f766c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c15a330 con 0x7f766c159ea0 2023-11-06T03:15:47.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c1f32b0 con 0x7f764c004d10 2023-11-06T03:15:47.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0c9e20 con 0x7f766c0caba0 2023-11-06T03:15:47.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38294/0 (socket says 172.21.15.157:38294) 2023-11-06T03:15:47.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 -- 172.21.15.157:0/3938925914 learned_addr learned my addr 172.21.15.157:0/3938925914 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:47.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f7670df6640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c004d10 0x7f766c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f766bfff640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c0caba0 0x7f766c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 -- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c0caba0 msgr2=0x7f766c159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c0caba0 0x7f766c159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 -- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c004d10 msgr2=0x7f766c1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c004d10 0x7f766c1f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 -- 172.21.15.157:0/3938925914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c1f2ff0 con 0x7f766c159ea0 2023-11-06T03:15:47.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76715f7640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ae0 tx=0x7f764c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:47.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640ba070 con 0x7f766c159ea0 2023-11-06T03:15:47.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7664004020 con 0x7f766c159ea0 2023-11-06T03:15:47.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c0ca0c0 con 0x7f766c159ea0 2023-11-06T03:15:47.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640a5680 con 0x7f766c159ea0 2023-11-06T03:15:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c0ca680 con 0x7f766c159ea0 2023-11-06T03:15:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766c0ca870 con 0x7f766c159ea0 2023-11-06T03:15:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76640b4070 con 0x7f766c159ea0 2023-11-06T03:15:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 0x7f7650067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:47.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.644+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f766411db50 con 0x7f766c159ea0 2023-11-06T03:15:47.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.648+0000 7f7670df6640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 0x7f7650067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:47.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.648+0000 7f7670df6640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 0x7f7650067f50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f765c009b70 tx=0x7f765c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:47.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.648+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f76640eb530 con 0x7f766c159ea0 2023-11-06T03:15:47.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.839+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f766c072750 con 0x7f766c159ea0 2023-11-06T03:15:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.851+0000 7f76697fa640 1 -- 172.21.15.157:0/3938925914 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7f76640eeb50 con 0x7f766c159ea0 2023-11-06T03:15:47.850 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 21 and data pool 22 2023-11-06T03:15:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 msgr2=0x7f7650067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 0x7f7650067f50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f765c009b70 tx=0x7f765c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 msgr2=0x7f766c0aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ae0 tx=0x7f764c005800 comp rx=0 tx=0).stop 2023-11-06T03:15:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 shutdown_connections 2023-11-06T03:15:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7650065a90 0x7f7650067f50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f766c159ea0 0x7f766c0aa070 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f766c0caba0 0x7f766c159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 --2- 172.21.15.157:0/3938925914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f764c004d10 0x7f766c1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.859+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 >> 172.21.15.157:0/3938925914 conn(0x7f766c070790 msgr2=0x7f766c0b63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.863+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 shutdown_connections 2023-11-06T03:15:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:47.863+0000 7f767305a640 1 -- 172.21.15.157:0/3938925914 wait complete. 2023-11-06T03:15:47.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:48.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.007+0000 7fce83d52640 1 Processor -- start 2023-11-06T03:15:48.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.007+0000 7fce83d52640 1 -- start start 2023-11-06T03:15:48.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce83d52640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fce7c1551e0 0x7fce7c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce83d52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce83d52640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c15f580 con 0x7fce7c15e5e0 2023-11-06T03:15:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce83d52640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c15efc0 con 0x7fce7c1551e0 2023-11-06T03:15:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce83d52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c15f2a0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce822ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce822ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:53276/0 (socket says 172.21.15.157:53276) 2023-11-06T03:15:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce822ef640 1 -- 172.21.15.157:0/1907734899 learned_addr learned my addr 172.21.15.157:0/1907734899 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:48.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 --2- 172.21.15.157:0/1907734899 >> v2:172.21.15.195:3300/0 conn(0x7fce7c1551e0 0x7fce7c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 -- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 msgr2=0x7fce7c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 -- 172.21.15.157:0/1907734899 >> v1:172.21.15.195:6789/0 conn(0x7fce7c15e5e0 legacy=0x7fce7c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 -- 172.21.15.157:0/1907734899 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c06f5a0 con 0x7fce7c1551e0 2023-11-06T03:15:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce822ef640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:15:48.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce81aee640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c1551e0 0x7fce7c06fda0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce780097c0 tx=0x7fce7801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf8a80d47e29e85 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce80aec640 1 -- 172.21.15.157:0/1907734899 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780ac640 con 0x7fce7c1551e0 2023-11-06T03:15:48.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce80aec640 1 -- 172.21.15.157:0/1907734899 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce780ac7a0 con 0x7fce7c1551e0 2023-11-06T03:15:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.011+0000 7fce80aec640 1 -- 172.21.15.157:0/1907734899 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780acb30 con 0x7fce7c1551e0 2023-11-06T03:15:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c1551e0 msgr2=0x7fce7c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c1551e0 0x7fce7c06fda0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fce780097c0 tx=0x7fce7801c3c0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/1907734899 shutdown_connections 2023-11-06T03:15:48.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c0b3b20 0x7fce7c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- 172.21.15.157:0/1907734899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c1551e0 0x7fce7c06fda0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/1907734899 >> 172.21.15.157:0/1907734899 conn(0x7fce7c065930 msgr2=0x7fce7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/1907734899 shutdown_connections 2023-11-06T03:15:48.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/1907734899 wait complete. 2023-11-06T03:15:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 Processor -- start 2023-11-06T03:15:48.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- start start 2023-11-06T03:15:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1551e0 0x7fce7c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c15e5e0 0x7fce7c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0cdcc0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c144790 con 0x7fce7c15e5e0 2023-11-06T03:15:48.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c12f8d0 con 0x7fce7c1551e0 2023-11-06T03:15:48.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38296/0 (socket says 172.21.15.157:38296) 2023-11-06T03:15:48.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 -- 172.21.15.157:0/3768378979 learned_addr learned my addr 172.21.15.157:0/3768378979 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:48.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce822ef640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c15e5e0 0x7fce7c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce812ed640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1551e0 0x7fce7c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 -- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1551e0 msgr2=0x7fce7c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1551e0 0x7fce7c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 -- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c15e5e0 msgr2=0x7fce7c1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c15e5e0 0x7fce7c1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 -- 172.21.15.157:0/3768378979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009b30 con 0x7fce7c0b3b20 2023-11-06T03:15:48.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce81aee640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fce78019ff0 tx=0x7fce78019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780bb070 con 0x7fce7c0b3b20 2023-11-06T03:15:48.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce780b5400 con 0x7fce7c0b3b20 2023-11-06T03:15:48.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780b57d0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78009420 con 0x7fce7c0b3b20 2023-11-06T03:15:48.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.015+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1444b0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c072750 con 0x7fce7c0b3b20 2023-11-06T03:15:48.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fce780b3040 con 0x7fce7c0b3b20 2023-11-06T03:15:48.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce727fc640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce7811cb40 con 0x7fce7c0b3b20 2023-11-06T03:15:48.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce812ed640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.019+0000 7fce812ed640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fce7c145cb0 tx=0x7fce6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.023+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fce780ea3a0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.179+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce7c0ae7b0 con 0x7fce7c0b3b20 2023-11-06T03:15:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce727fc640 1 -- 172.21.15.157:0/3768378979 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17894 (secure 0 0 0) 0x7fce780b1090 con 0x7fce7c0b3b20 2023-11-06T03:15:48.179 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:48.180 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":148,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:47.842799+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 msgr2=0x7fce60067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fce7c145cb0 tx=0x7fce6c009040 comp rx=0 tx=0).stop 2023-11-06T03:15:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 msgr2=0x7fce7c12f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fce78019ff0 tx=0x7fce78019160 comp rx=0 tx=0).stop 2023-11-06T03:15:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 shutdown_connections 2023-11-06T03:15:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce60065a60 0x7fce60067f20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce7c15e5e0 0x7fce7c1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce7c1551e0 0x7fce7c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 --2- 172.21.15.157:0/3768378979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce7c0b3b20 0x7fce7c12f390 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 >> 172.21.15.157:0/3768378979 conn(0x7fce7c065930 msgr2=0x7fce7c0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 shutdown_connections 2023-11-06T03:15:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.183+0000 7fce83d52640 1 -- 172.21.15.157:0/3768378979 wait complete. 2023-11-06T03:15:48.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:48.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 Processor -- start 2023-11-06T03:15:48.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 -- start start 2023-11-06T03:15:48.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f932815cc70 0x7f93280d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f93280d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9328160830 con 0x7f93280d25a0 2023-11-06T03:15:48.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93281614b0 con 0x7f932815cc70 2023-11-06T03:15:48.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932eb34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9328160b10 con 0x7f93280d00a0 2023-11-06T03:15:48.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f9327fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f93280d25a0 0x7f93280cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59650/0 (socket says 172.21.15.157:59650) 2023-11-06T03:15:48.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f9327fff640 1 -- 172.21.15.157:0/2352796537 learned_addr learned my addr 172.21.15.157:0/2352796537 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932d0d1640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f93280d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2352796537 >> v2:172.21.15.195:3300/0 conn(0x7f932815cc70 0x7f93280d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358735185 0 0) 0x7f9328160830 con 0x7f93280d25a0 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9310003530 con 0x7f93280d25a0 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1980805655 0 0) 0x7f9310003530 con 0x7f93280d25a0 2023-11-06T03:15:48.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 msgr2=0x7f93280d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f93280d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 >> v2:172.21.15.195:3300/0 conn(0x7f932815cc70 msgr2=0x7f93280d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 --2- 172.21.15.157:0/2352796537 >> v2:172.21.15.195:3300/0 conn(0x7f932815cc70 0x7f93280d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328160df0 con 0x7f93280d25a0 2023-11-06T03:15:48.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f931c001680 con 0x7f93280d25a0 2023-11-06T03:15:48.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 >> v1:172.21.15.157:6789/0 conn(0x7f93280d25a0 legacy=0x7f93280cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9310003980 0x7f9310003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93100047d0 0x7f9310004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f9310005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281614b0 con 0x7f9310003980 2023-11-06T03:15:48.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328160b10 con 0x7f93100047d0 2023-11-06T03:15:48.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931000b9c0 con 0x7f93280d00a0 2023-11-06T03:15:48.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9310003980 0x7f9310003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f9327fff640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93100047d0 0x7f9310004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932d0d1640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f9310005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 -- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 msgr2=0x7f9310005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f9310005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 -- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93100047d0 msgr2=0x7f9310004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93100047d0 0x7f9310004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932c8d0640 1 -- 172.21.15.157:0/2352796537 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310003530 con 0x7f9310003980 2023-11-06T03:15:48.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f9327fff640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93100047d0 0x7f9310004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:48.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.319+0000 7f932d0d1640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93280d00a0 0x7f9310005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:48.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9310003980 0x7f9310003da0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f9318009ae0 tx=0x7f931801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3640ed83e84fef1b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180a64a0 con 0x7f9310003980 2023-11-06T03:15:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93180a6600 con 0x7f9310003980 2023-11-06T03:15:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f93277fe640 1 -- 172.21.15.157:0/2352796537 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180a69b0 con 0x7f9310003980 2023-11-06T03:15:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- 172.21.15.157:0/2352796537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9310003980 msgr2=0x7f9310003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 --2- 172.21.15.157:0/2352796537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9310003980 0x7f9310003da0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f9318009ae0 tx=0x7f931801ca90 comp rx=0 tx=0).stop 2023-11-06T03:15:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932d0d1640 1 -- 172.21.15.157:0/2352796537 reap_dead start 2023-11-06T03:15:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- 172.21.15.157:0/2352796537 shutdown_connections 2023-11-06T03:15:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- 172.21.15.157:0/2352796537 >> 172.21.15.157:0/2352796537 conn(0x7f9328070560 msgr2=0x7f9328070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- 172.21.15.157:0/2352796537 shutdown_connections 2023-11-06T03:15:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- 172.21.15.157:0/2352796537 wait complete. 2023-11-06T03:15:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 Processor -- start 2023-11-06T03:15:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.323+0000 7f932eb34640 1 -- start start 2023-11-06T03:15:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9310003980 0x7f932815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93280cba10 0x7f93280c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932815bac0 con 0x7f93281f1fd0 2023-11-06T03:15:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cbe90 con 0x7f93280cba10 2023-11-06T03:15:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cac00 con 0x7f9310003980 2023-11-06T03:15:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38328/0 (socket says 172.21.15.157:38328) 2023-11-06T03:15:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 -- 172.21.15.157:0/2383720087 learned_addr learned my addr 172.21.15.157:0/2383720087 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9310003980 0x7f932815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932d0d1640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93280cba10 0x7f93280c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 -- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9310003980 msgr2=0x7f932815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9310003980 0x7f932815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 -- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93280cba10 msgr2=0x7f93280c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93280cba10 0x7f93280c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 -- 172.21.15.157:0/2383720087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280ca920 con 0x7f93281f1fd0 2023-11-06T03:15:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f9327fff640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f931c000fc0 tx=0x7f931c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00eb60 con 0x7f93281f1fd0 2023-11-06T03:15:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00ecc0 con 0x7f93281f1fd0 2023-11-06T03:15:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932815d180 con 0x7f93281f1fd0 2023-11-06T03:15:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0145e0 con 0x7f93281f1fd0 2023-11-06T03:15:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932815d740 con 0x7f93281f1fd0 2023-11-06T03:15:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932815d930 con 0x7f93281f1fd0 2023-11-06T03:15:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f931c014740 con 0x7f93281f1fd0 2023-11-06T03:15:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 0x7f9314067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f931c085c90 con 0x7f93281f1fd0 2023-11-06T03:15:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.327+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 0x7f9314067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.331+0000 7f932c8d0640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 0x7f9314067f50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f93180ad000 tx=0x7f93180b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.331+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f931c053580 con 0x7f93281f1fd0 2023-11-06T03:15:48.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.483+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9328072750 con 0x7f93281f1fd0 2023-11-06T03:15:48.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.483+0000 7f93257fa640 1 -- 172.21.15.157:0/2383720087 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17894 (secure 0 0 0) 0x7f931c01b090 con 0x7f93281f1fd0 2023-11-06T03:15:48.483 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:48.483 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":148,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:47.842799+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:48.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 msgr2=0x7f9314067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 0x7f9314067f50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f93180ad000 tx=0x7f93180b2040 comp rx=0 tx=0).stop 2023-11-06T03:15:48.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 msgr2=0x7f93280cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f931c000fc0 tx=0x7f931c010ef0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 shutdown_connections 2023-11-06T03:15:48.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9314065a90 0x7f9314067f50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93280cba10 0x7f93280c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93281f1fd0 0x7f93280cb4d0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 --2- 172.21.15.157:0/2383720087 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9310003980 0x7f932815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 >> 172.21.15.157:0/2383720087 conn(0x7f9328070560 msgr2=0x7f932815c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 shutdown_connections 2023-11-06T03:15:48.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.487+0000 7f932eb34640 1 -- 172.21.15.157:0/2383720087 wait complete. 2023-11-06T03:15:48.501 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:15:48.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 Processor -- start 2023-11-06T03:15:48.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 -- start start 2023-11-06T03:15:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f885c0cdef0 0x7f885c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c0cb6e0 con 0x7f885c15cf60 2023-11-06T03:15:48.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c071830 con 0x7f885c15cb80 2023-11-06T03:15:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f886257c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c071b10 con 0x7f885c0cdef0 2023-11-06T03:15:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8853fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f885c15cf60 0x7f885c0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44700/0 (socket says 172.21.15.157:44700) 2023-11-06T03:15:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8853fff640 1 -- 172.21.15.157:0/1666222983 learned_addr learned my addr 172.21.15.157:0/1666222983 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 --2- 172.21.15.157:0/1666222983 >> v2:172.21.15.195:3300/0 conn(0x7f885c0cdef0 0x7f885c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178584203 0 0) 0x7f885c0cb6e0 con 0x7f885c15cf60 2023-11-06T03:15:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8848003530 con 0x7f885c15cf60 2023-11-06T03:15:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196778227 0 0) 0x7f8848003530 con 0x7f885c15cf60 2023-11-06T03:15:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 >> v2:172.21.15.195:3300/0 conn(0x7f885c0cdef0 msgr2=0x7f885c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 --2- 172.21.15.157:0/1666222983 >> v2:172.21.15.195:3300/0 conn(0x7f885c0cdef0 0x7f885c0cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 >> v1:172.21.15.195:6789/0 conn(0x7f885c15cb80 legacy=0x7f885c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c071df0 con 0x7f885c15cf60 2023-11-06T03:15:48.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8844001680 con 0x7f885c15cf60 2023-11-06T03:15:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 >> v1:172.21.15.157:6789/0 conn(0x7f885c15cf60 legacy=0x7f885c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848003980 0x7f8848003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88480047d0 0x7f8848004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 0x7f884800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c071b10 con 0x7f8848005190 2023-11-06T03:15:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884800c0c0 con 0x7f88480047d0 2023-11-06T03:15:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884800c280 con 0x7f8848003980 2023-11-06T03:15:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 0x7f884800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8853fff640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88480047d0 0x7f8848004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885bfff640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848003980 0x7f8848003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 -- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848003980 msgr2=0x7f8848003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848003980 0x7f8848003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 -- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88480047d0 msgr2=0x7f8848004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88480047d0 0x7f8848004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f8860b19640 1 -- 172.21.15.157:0/1666222983 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8848003530 con 0x7f8848005190 2023-11-06T03:15:48.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.627+0000 7f885bfff640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848003980 0x7f8848003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 -- 172.21.15.157:0/1666222983 reap_dead start 2023-11-06T03:15:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 0x7f884800b990 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8854007360 tx=0x7f885401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09361f9f5c1092c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88540a5d60 con 0x7f8848005190 2023-11-06T03:15:48.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88540039a0 con 0x7f8848005190 2023-11-06T03:15:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f885b7fe640 1 -- 172.21.15.157:0/1666222983 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854003d50 con 0x7f8848005190 2023-11-06T03:15:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 msgr2=0x7f884800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 0x7f884800b990 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f8854007360 tx=0x7f885401ce60 comp rx=0 tx=0).stop 2023-11-06T03:15:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- 172.21.15.157:0/1666222983 shutdown_connections 2023-11-06T03:15:48.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 --2- 172.21.15.157:0/1666222983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8848005190 0x7f884800b990 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- 172.21.15.157:0/1666222983 >> 172.21.15.157:0/1666222983 conn(0x7f885c06d560 msgr2=0x7f885c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- 172.21.15.157:0/1666222983 shutdown_connections 2023-11-06T03:15:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- 172.21.15.157:0/1666222983 wait complete. 2023-11-06T03:15:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 Processor -- start 2023-11-06T03:15:48.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- start start 2023-11-06T03:15:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848005190 0x7f885c152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885c0ae030 0x7f885c1f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0acc70 con 0x7f885c0ae4b0 2023-11-06T03:15:48.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c1532e0 con 0x7f885c0ae030 2023-11-06T03:15:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f886257c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0b3860 con 0x7f8848005190 2023-11-06T03:15:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37240/0 (socket says 172.21.15.157:37240) 2023-11-06T03:15:48.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 -- 172.21.15.157:0/3546930893 learned_addr learned my addr 172.21.15.157:0/3546930893 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:48.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8853fff640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885c0ae030 0x7f885c1f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f885bfff640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848005190 0x7f885c152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 -- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848005190 msgr2=0x7f885c152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848005190 0x7f885c152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 -- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885c0ae030 msgr2=0x7f885c1f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885c0ae030 0x7f885c1f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.631+0000 7f8860b19640 1 -- 172.21.15.157:0/3546930893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0b3b40 con 0x7f885c0ae4b0 2023-11-06T03:15:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f8860b19640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f885401cc90 tx=0x7f8854019c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88540adda0 con 0x7f885c0ae4b0 2023-11-06T03:15:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88540a5d70 con 0x7f885c0ae4b0 2023-11-06T03:15:48.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c0b3e20 con 0x7f885c0ae4b0 2023-11-06T03:15:48.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88540b5640 con 0x7f885c0ae4b0 2023-11-06T03:15:48.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1f6500 con 0x7f885c0ae4b0 2023-11-06T03:15:48.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c06bf00 con 0x7f885c0ae4b0 2023-11-06T03:15:48.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f88540b57a0 con 0x7f885c0ae4b0 2023-11-06T03:15:48.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 0x7f883c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f885411de20 con 0x7f885c0ae4b0 2023-11-06T03:15:48.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f885bfff640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 0x7f883c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.635+0000 7f885bfff640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 0x7f883c067f50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f884c007fd0 tx=0x7f884c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.639+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f88540eb680 con 0x7f885c0ae4b0 2023-11-06T03:15:48.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.827+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f885c0b2e90 con 0x7f885c0ae4b0 2023-11-06T03:15:48.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f88597fa640 1 -- 172.21.15.157:0/3546930893 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v110) v1 ==== 118+0+0 (secure 0 0 0) 0x7f88540eeca0 con 0x7f885c0ae4b0 2023-11-06T03:15:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 msgr2=0x7f883c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 0x7f883c067f50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f884c007fd0 tx=0x7f884c009040 comp rx=0 tx=0).stop 2023-11-06T03:15:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 msgr2=0x7f885c1f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f885401cc90 tx=0x7f8854019c00 comp rx=0 tx=0).stop 2023-11-06T03:15:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 shutdown_connections 2023-11-06T03:15:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f883c065a90 0x7f883c067f50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f885c0ae4b0 0x7f885c1f5dc0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885c0ae030 0x7f885c1f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 --2- 172.21.15.157:0/3546930893 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8848005190 0x7f885c152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 >> 172.21.15.157:0/3546930893 conn(0x7f885c06d560 msgr2=0x7f885c0c6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:48.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 shutdown_connections 2023-11-06T03:15:48.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.851+0000 7f886257c640 1 -- 172.21.15.157:0/3546930893 wait complete. 2023-11-06T03:15:48.869 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:15:48.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:15:48.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 Processor -- start 2023-11-06T03:15:48.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 -- start start 2023-11-06T03:15:48.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4bb815e5e0 0x7f4bb80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bb815ea60 0x7f4bb80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb815f580 con 0x7f4bb815ea60 2023-11-06T03:15:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb815efc0 con 0x7f4bb815e5e0 2023-11-06T03:15:48.992 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.991+0000 7f4bbd3b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb815f2a0 con 0x7f4bb81551e0 2023-11-06T03:15:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bb815ea60 0x7f4bb80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4bb815ea60 0x7f4bb80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37242/0 (socket says 172.21.15.157:37242) 2023-11-06T03:15:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 -- 172.21.15.157:0/311798790 learned_addr learned my addr 172.21.15.157:0/311798790 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bae7fc640 1 --2- 172.21.15.157:0/311798790 >> v2:172.21.15.195:3300/0 conn(0x7f4bb815e5e0 0x7f4bb80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb6ffd640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:48.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 -- 172.21.15.157:0/311798790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 msgr2=0x7f4bb806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 -- 172.21.15.157:0/311798790 >> v2:172.21.15.195:3300/0 conn(0x7f4bb815e5e0 msgr2=0x7f4bb80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:48.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 --2- 172.21.15.157:0/311798790 >> v2:172.21.15.195:3300/0 conn(0x7f4bb815e5e0 0x7f4bb80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:48.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 -- 172.21.15.157:0/311798790 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb806f5a0 con 0x7f4bb815ea60 2023-11-06T03:15:48.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb77fe640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815ea60 0x7f4bb80b3950 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80668b0 tx=0x7f4bb001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707b338f8f04257 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:48.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb67fc640 1 -- 172.21.15.157:0/311798790 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00a57b0 con 0x7f4bb815ea60 2023-11-06T03:15:48.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb67fc640 1 -- 172.21.15.157:0/311798790 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb00a5910 con 0x7f4bb815ea60 2023-11-06T03:15:48.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bb67fc640 1 -- 172.21.15.157:0/311798790 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00a5ca0 con 0x7f4bb815ea60 2023-11-06T03:15:48.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/311798790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815ea60 msgr2=0x7f4bb80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:48.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815ea60 0x7f4bb80b3950 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80668b0 tx=0x7f4bb001c910 comp rx=0 tx=0).stop 2023-11-06T03:15:48.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/311798790 shutdown_connections 2023-11-06T03:15:48.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815ea60 0x7f4bb80b3950 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/311798790 >> v2:172.21.15.195:3300/0 conn(0x7f4bb815e5e0 0x7f4bb80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/311798790 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/311798790 >> 172.21.15.157:0/311798790 conn(0x7f4bb8065930 msgr2=0x7f4bb8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:49.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/311798790 shutdown_connections 2023-11-06T03:15:49.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.995+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/311798790 wait complete. 2023-11-06T03:15:49.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 Processor -- start 2023-11-06T03:15:49.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- start start 2023-11-06T03:15:49.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb80ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:49.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:49.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 0x7f4bb814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:49.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8156640 con 0x7f4bb815e5e0 2023-11-06T03:15:49.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8155a30 con 0x7f4bb815ea60 2023-11-06T03:15:49.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8155d10 con 0x7f4bb81551e0 2023-11-06T03:15:49.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:49.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37252/0 (socket says 172.21.15.157:37252) 2023-11-06T03:15:49.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 -- 172.21.15.157:0/1335559347 learned_addr learned my addr 172.21.15.157:0/1335559347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:49.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bb77fe640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 0x7f4bb814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:49.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bb6ffd640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb80ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:49.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 -- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 msgr2=0x7f4bb80ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:49.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb80ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 -- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 msgr2=0x7f4bb814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:49.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 0x7f4bb814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 -- 172.21.15.157:0/1335559347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009b30 con 0x7f4bb815e5e0 2023-11-06T03:15:49.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bb6ffd640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb80ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:49.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bb77fe640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 0x7f4bb814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:49.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bae7fc640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d800 tx=0x7f4ba000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:49.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000b6c0 con 0x7f4bb815e5e0 2023-11-06T03:15:49.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000b820 con 0x7f4bb815e5e0 2023-11-06T03:15:49.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb8155ff0 con 0x7f4bb815e5e0 2023-11-06T03:15:49.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000bb90 con 0x7f4bb815e5e0 2023-11-06T03:15:49.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80aed20 con 0x7f4bb815e5e0 2023-11-06T03:15:49.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:48.999+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb8072750 con 0x7f4bb815e5e0 2023-11-06T03:15:49.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.003+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4ba000bcf0 con 0x7f4bb815e5e0 2023-11-06T03:15:49.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.003+0000 7f4baffff640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 0x7f4b98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:49.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.003+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ba0083890 con 0x7f4bb815e5e0 2023-11-06T03:15:49.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.003+0000 7f4bb6ffd640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 0x7f4b98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:49.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.003+0000 7f4bb6ffd640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 0x7f4b98067f20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80af320 tx=0x7f4ba8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:49.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.007+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4ba00510f0 con 0x7f4bb815e5e0 2023-11-06T03:15:49.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.195+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4bb8071730 con 0x7f4bb815e5e0 2023-11-06T03:15:49.387 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:15:49.391+0000 7fa23e408640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:15:49.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.871+0000 7f4baffff640 1 -- 172.21.15.157:0/1335559347 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v111) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ba0054710 con 0x7f4bb815e5e0 2023-11-06T03:15:49.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 msgr2=0x7f4b98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:49.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 0x7f4b98067f20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80af320 tx=0x7f4ba8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:49.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 msgr2=0x7f4bb8156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:49.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d800 tx=0x7f4ba000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 shutdown_connections 2023-11-06T03:15:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4b98065a60 0x7f4b98067f20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4bb815ea60 0x7f4bb814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bb815e5e0 0x7f4bb8156ad0 secure :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d800 tx=0x7f4ba000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 --2- 172.21.15.157:0/1335559347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4bb81551e0 0x7f4bb80ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:49.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 >> 172.21.15.157:0/1335559347 conn(0x7f4bb8065930 msgr2=0x7f4bb80afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:49.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 shutdown_connections 2023-11-06T03:15:49.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:49.875+0000 7f4bbd3b8640 1 -- 172.21.15.157:0/1335559347 wait complete. 2023-11-06T03:15:49.895 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:15:49.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:50.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 Processor -- start 2023-11-06T03:15:50.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 -- start start 2023-11-06T03:15:50.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43f81551e0 0x7f43f806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 0x7f43f815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f815f580 con 0x7f43f815e5e0 2023-11-06T03:15:50.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f815efc0 con 0x7f43f81551e0 2023-11-06T03:15:50.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43ff8be640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f815f2a0 con 0x7f43f80b3b20 2023-11-06T03:15:50.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f43f81551e0 0x7f43f806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fce59640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f43f815e5e0 0x7f43f80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44714/0 (socket says 172.21.15.157:44714) 2023-11-06T03:15:50.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fce59640 1 -- 172.21.15.157:0/1984772906 learned_addr learned my addr 172.21.15.157:0/1984772906 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:50.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 --2- 172.21.15.157:0/1984772906 >> v2:172.21.15.157:3300/0 conn(0x7f43f81551e0 0x7f43f806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37254/0 (socket says 172.21.15.157:37254) 2023-11-06T03:15:50.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 -- 172.21.15.157:0/1984772906 >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 msgr2=0x7f43f815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fde5b640 1 --2- 172.21.15.157:0/1984772906 >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 0x7f43f815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 --2- 172.21.15.157:0/1984772906 >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 0x7f43f815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 -- 172.21.15.157:0/1984772906 >> v1:172.21.15.157:6789/0 conn(0x7f43f815e5e0 legacy=0x7f43f80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fd65a640 1 -- 172.21.15.157:0/1984772906 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f806f5a0 con 0x7f43f81551e0 2023-11-06T03:15:50.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43effff640 1 -- 172.21.15.157:0/1984772906 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573749799 0 0) 0x7f43f815f580 con 0x7f43f815e5e0 2023-11-06T03:15:50.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.011+0000 7f43fde5b640 1 --2- 172.21.15.157:0/1984772906 >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 0x7f43f815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:50.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fd65a640 1 --2- 172.21.15.157:0/1984772906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f81551e0 0x7f43f806fda0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f43f80cbfe0 tx=0x7f43e801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e697fa1d6cef23b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43effff640 1 -- 172.21.15.157:0/1984772906 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80ab600 con 0x7f43f81551e0 2023-11-06T03:15:50.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43effff640 1 -- 172.21.15.157:0/1984772906 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e80a7050 con 0x7f43f81551e0 2023-11-06T03:15:50.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43effff640 1 -- 172.21.15.157:0/1984772906 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80ab980 con 0x7f43f81551e0 2023-11-06T03:15:50.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- 172.21.15.157:0/1984772906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f81551e0 msgr2=0x7f43f806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- 172.21.15.157:0/1984772906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f81551e0 0x7f43f806fda0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f43f80cbfe0 tx=0x7f43e801c310 comp rx=0 tx=0).stop 2023-11-06T03:15:50.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- 172.21.15.157:0/1984772906 shutdown_connections 2023-11-06T03:15:50.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- 172.21.15.157:0/1984772906 >> v2:172.21.15.195:3300/0 conn(0x7f43f80b3b20 0x7f43f815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- 172.21.15.157:0/1984772906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f81551e0 0x7f43f806fda0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- 172.21.15.157:0/1984772906 >> 172.21.15.157:0/1984772906 conn(0x7f43f8065930 msgr2=0x7f43f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- 172.21.15.157:0/1984772906 shutdown_connections 2023-11-06T03:15:50.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- 172.21.15.157:0/1984772906 wait complete. 2023-11-06T03:15:50.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 Processor -- start 2023-11-06T03:15:50.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- start start 2023-11-06T03:15:50.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f80b3b20 0x7f43f80aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 0x7f43f814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f815e270 con 0x7f43f815e5e0 2023-11-06T03:15:50.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f814d4b0 con 0x7f43f80b3b20 2023-11-06T03:15:50.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43ff8be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f814d1d0 con 0x7f43f81551e0 2023-11-06T03:15:50.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37256/0 (socket says 172.21.15.157:37256) 2023-11-06T03:15:50.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 -- 172.21.15.157:0/2838231432 learned_addr learned my addr 172.21.15.157:0/2838231432 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fd65a640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f80b3b20 0x7f43f80aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fce59640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 0x7f43f814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 -- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 msgr2=0x7f43f814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 0x7f43f814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 -- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f80b3b20 msgr2=0x7f43f80aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f80b3b20 0x7f43f80aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fde5b640 1 -- 172.21.15.157:0/2838231432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009420 con 0x7f43f815e5e0 2023-11-06T03:15:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.015+0000 7f43fce59640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 0x7f43f814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43fde5b640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f43f000cef0 tx=0x7f43f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000d840 con 0x7f43f815e5e0 2023-11-06T03:15:50.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f000d9c0 con 0x7f43f815e5e0 2023-11-06T03:15:50.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f8159110 con 0x7f43f815e5e0 2023-11-06T03:15:50.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000dd70 con 0x7f43f815e5e0 2023-11-06T03:15:50.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f81596d0 con 0x7f43f815e5e0 2023-11-06T03:15:50.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8072750 con 0x7f43f815e5e0 2023-11-06T03:15:50.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f43f001e3b0 con 0x7f43f815e5e0 2023-11-06T03:15:50.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 0x7f43dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f43f0083e10 con 0x7f43f815e5e0 2023-11-06T03:15:50.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43fd65a640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 0x7f43dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.019+0000 7f43fd65a640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 0x7f43dc067f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ad000 tx=0x7f43e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.023+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f43f00516d0 con 0x7f43f815e5e0 2023-11-06T03:15:50.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43f8071730 con 0x7f43f815e5e0 2023-11-06T03:15:50.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43edffb640 1 -- 172.21.15.157:0/2838231432 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f43f001a090 con 0x7f43f815e5e0 2023-11-06T03:15:50.201 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:50.201 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:15:50.202 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:15:50.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 msgr2=0x7f43dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 0x7f43dc067f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ad000 tx=0x7f43e80a6000 comp rx=0 tx=0).stop 2023-11-06T03:15:50.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 msgr2=0x7f43f814c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f43f000cef0 tx=0x7f43f000a710 comp rx=0 tx=0).stop 2023-11-06T03:15:50.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 shutdown_connections 2023-11-06T03:15:50.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f43dc065a60 0x7f43dc067f20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f43f815e5e0 0x7f43f814c740 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f43f81551e0 0x7f43f814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 --2- 172.21.15.157:0/2838231432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f43f80b3b20 0x7f43f80aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.203+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 >> 172.21.15.157:0/2838231432 conn(0x7f43f8065930 msgr2=0x7f43f8154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.207+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 shutdown_connections 2023-11-06T03:15:50.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.207+0000 7f43ff8be640 1 -- 172.21.15.157:0/2838231432 wait complete. 2023-11-06T03:15:50.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:15:50.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 Processor -- start 2023-11-06T03:15:50.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 -- start start 2023-11-06T03:15:50.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f06e815caa0 0x7f06e80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06e80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80d2920 con 0x7f06e80ce1a0 2023-11-06T03:15:50.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e80d18a0 con 0x7f06e815caa0 2023-11-06T03:15:50.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ee995640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80d1b60 con 0x7f06e815ce80 2023-11-06T03:15:50.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ecf32640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f06e80ce1a0 0x7f06e80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44720/0 (socket says 172.21.15.157:44720) 2023-11-06T03:15:50.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ecf32640 1 -- 172.21.15.157:0/3584229851 learned_addr learned my addr 172.21.15.157:0/3584229851 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:50.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e77fe640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06e80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 --2- 172.21.15.157:0/3584229851 >> v2:172.21.15.195:3300/0 conn(0x7f06e815caa0 0x7f06e80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793657713 0 0) 0x7f06e80d2920 con 0x7f06e80ce1a0 2023-11-06T03:15:50.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06d0003530 con 0x7f06e80ce1a0 2023-11-06T03:15:50.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837997271 0 0) 0x7f06d0003530 con 0x7f06e80ce1a0 2023-11-06T03:15:50.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 msgr2=0x7f06e80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06e80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 >> v2:172.21.15.195:3300/0 conn(0x7f06e815caa0 msgr2=0x7f06e80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 --2- 172.21.15.157:0/3584229851 >> v2:172.21.15.195:3300/0 conn(0x7f06e815caa0 0x7f06e80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80d1e40 con 0x7f06e80ce1a0 2023-11-06T03:15:50.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f06e0001a00 con 0x7f06e80ce1a0 2023-11-06T03:15:50.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 >> v1:172.21.15.157:6789/0 conn(0x7f06e80ce1a0 legacy=0x7f06e80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06d0003980 0x7f06d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d00047d0 0x7f06d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80d18a0 con 0x7f06d0003980 2023-11-06T03:15:50.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80d1b60 con 0x7f06d00047d0 2023-11-06T03:15:50.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d000b990 con 0x7f06e815ce80 2023-11-06T03:15:50.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06d0003980 0x7f06d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06ecf32640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 -- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 msgr2=0x7f06d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e815ce80 0x7f06d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 -- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d00047d0 msgr2=0x7f06d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d00047d0 0x7f06d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.339+0000 7f06e7fff640 1 -- 172.21.15.157:0/3584229851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0003530 con 0x7f06d0003980 2023-11-06T03:15:50.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e7fff640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06d0003980 0x7f06d0003da0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009ae0 tx=0x7f06d801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9394598f92ec81f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80a6460 con 0x7f06d0003980 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d80a65c0 con 0x7f06d0003980 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e6ffd640 1 -- 172.21.15.157:0/3584229851 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80a6970 con 0x7f06d0003980 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/3584229851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06d0003980 msgr2=0x7f06d0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 --2- 172.21.15.157:0/3584229851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06d0003980 0x7f06d0003da0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009ae0 tx=0x7f06d801ca60 comp rx=0 tx=0).stop 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ecf32640 1 -- 172.21.15.157:0/3584229851 reap_dead start 2023-11-06T03:15:50.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/3584229851 shutdown_connections 2023-11-06T03:15:50.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/3584229851 >> 172.21.15.157:0/3584229851 conn(0x7f06e8070790 msgr2=0x7f06e8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/3584229851 shutdown_connections 2023-11-06T03:15:50.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/3584229851 wait complete. 2023-11-06T03:15:50.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 Processor -- start 2023-11-06T03:15:50.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- start start 2023-11-06T03:15:50.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d0003980 0x7f06e8158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e80cb1a0 0x7f06e815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cb720 con 0x7f06e80cad40 2023-11-06T03:15:50.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cba00 con 0x7f06d0003980 2023-11-06T03:15:50.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cbce0 con 0x7f06e80cb1a0 2023-11-06T03:15:50.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37268/0 (socket says 172.21.15.157:37268) 2023-11-06T03:15:50.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 -- 172.21.15.157:0/2258790109 learned_addr learned my addr 172.21.15.157:0/2258790109 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:50.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 -- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e80cb1a0 msgr2=0x7f06e815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e80cb1a0 0x7f06e815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 -- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d0003980 msgr2=0x7f06e8158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d0003980 0x7f06e8158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 -- 172.21.15.157:0/2258790109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80cbfc0 con 0x7f06e80cad40 2023-11-06T03:15:50.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e77fe640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00e7c0 tx=0x7f06dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00eed0 con 0x7f06e80cad40 2023-11-06T03:15:50.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc00c940 con 0x7f06e80cad40 2023-11-06T03:15:50.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e80cc2a0 con 0x7f06e80cad40 2023-11-06T03:15:50.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00ccf0 con 0x7f06e80cad40 2023-11-06T03:15:50.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.343+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e8159340 con 0x7f06e80cad40 2023-11-06T03:15:50.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e8072750 con 0x7f06e80cad40 2023-11-06T03:15:50.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f06dc013470 con 0x7f06e80cad40 2023-11-06T03:15:50.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06e4ff9640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 0x7f06d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f06dc0833b0 con 0x7f06e80cad40 2023-11-06T03:15:50.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06e7fff640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 0x7f06d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.347+0000 7f06e7fff640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 0x7f06d4067f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f06d80ad000 tx=0x7f06d8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.351+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f06dc050c40 con 0x7f06e80cad40 2023-11-06T03:15:50.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.507+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06e80cc460 con 0x7f06e80cad40 2023-11-06T03:15:50.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06e4ff9640 1 -- 172.21.15.157:0/2258790109 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+18086 (secure 0 0 0) 0x7f06dc054260 con 0x7f06e80cad40 2023-11-06T03:15:50.507 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:50.507 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":150,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:49.858333+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:15:50.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 msgr2=0x7f06d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 0x7f06d4067f20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f06d80ad000 tx=0x7f06d8004740 comp rx=0 tx=0).stop 2023-11-06T03:15:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 msgr2=0x7f06e815f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00e7c0 tx=0x7f06dc00ec80 comp rx=0 tx=0).stop 2023-11-06T03:15:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 shutdown_connections 2023-11-06T03:15:50.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f06d4065a60 0x7f06d4067f20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06e80cb1a0 0x7f06e815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06e80cad40 0x7f06e815f010 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 --2- 172.21.15.157:0/2258790109 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f06d0003980 0x7f06e8158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 >> 172.21.15.157:0/2258790109 conn(0x7f06e8070790 msgr2=0x7f06e815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 shutdown_connections 2023-11-06T03:15:50.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.511+0000 7f06ee995640 1 -- 172.21.15.157:0/2258790109 wait complete. 2023-11-06T03:15:50.527 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:15:50.527 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:15:50.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 Processor -- start 2023-11-06T03:15:50.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 -- start start 2023-11-06T03:15:50.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce50156a70 0x7fce5006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce500b2330 0x7fce5006c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5006c490 con 0x7fce500adae0 2023-11-06T03:15:50.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5006b950 con 0x7fce50156a70 2023-11-06T03:15:50.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce56091640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5006bb60 con 0x7fce500b2330 2023-11-06T03:15:50.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce50156a70 0x7fce5006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fce50156a70 0x7fce5006f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37272/0 (socket says 172.21.15.157:37272) 2023-11-06T03:15:50.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 -- 172.21.15.157:0/1963121395 learned_addr learned my addr 172.21.15.157:0/1963121395 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:50.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 -- 172.21.15.157:0/1963121395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce500b2330 msgr2=0x7fce5006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 --2- 172.21.15.157:0/1963121395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce500b2330 0x7fce5006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 -- 172.21.15.157:0/1963121395 >> v1:172.21.15.195:6789/0 conn(0x7fce500adae0 legacy=0x7fce500b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:50.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.651+0000 7fce4f7fe640 1 -- 172.21.15.157:0/1963121395 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5006be40 con 0x7fce50156a70 2023-11-06T03:15:50.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4f7fe640 1 --2- 172.21.15.157:0/1963121395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50156a70 0x7fce5006f470 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fce4000aa10 tx=0x7fce4001d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9987a6eb4ba32b9a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4effd640 1 -- 172.21.15.157:0/1963121395 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400a6970 con 0x7fce50156a70 2023-11-06T03:15:50.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4effd640 1 -- 172.21.15.157:0/1963121395 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce400a6ad0 con 0x7fce50156a70 2023-11-06T03:15:50.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4effd640 1 -- 172.21.15.157:0/1963121395 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400a6e60 con 0x7fce50156a70 2023-11-06T03:15:50.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1963121395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50156a70 msgr2=0x7fce5006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- 172.21.15.157:0/1963121395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50156a70 0x7fce5006f470 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fce4000aa10 tx=0x7fce4001d8c0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1963121395 shutdown_connections 2023-11-06T03:15:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- 172.21.15.157:0/1963121395 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce500b2330 0x7fce5006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- 172.21.15.157:0/1963121395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce50156a70 0x7fce5006f470 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1963121395 >> 172.21.15.157:0/1963121395 conn(0x7fce50065930 msgr2=0x7fce50065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1963121395 shutdown_connections 2023-11-06T03:15:50.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1963121395 wait complete. 2023-11-06T03:15:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 Processor -- start 2023-11-06T03:15:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- start start 2023-11-06T03:15:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce500adae0 0x7fce50155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce50156a70 0x7fce500704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500ae8d0 con 0x7fce500b2330 2023-11-06T03:15:50.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501f62b0 con 0x7fce500adae0 2023-11-06T03:15:50.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce501f6590 con 0x7fce50156a70 2023-11-06T03:15:50.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37278/0 (socket says 172.21.15.157:37278) 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 -- 172.21.15.157:0/1381489456 learned_addr learned my addr 172.21.15.157:0/1381489456 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4f7fe640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce500adae0 0x7fce50155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4ffff640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce50156a70 0x7fce500704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 -- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce50156a70 msgr2=0x7fce500704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce50156a70 0x7fce500704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 -- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce500adae0 msgr2=0x7fce50155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce500adae0 0x7fce50155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 -- 172.21.15.157:0/1381489456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4000a670 con 0x7fce500b2330 2023-11-06T03:15:50.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce47fff640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c490 tx=0x7fce38001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38003600 con 0x7fce500b2330 2023-11-06T03:15:50.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38003780 con 0x7fce500b2330 2023-11-06T03:15:50.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500b0260 con 0x7fce500b2330 2023-11-06T03:15:50.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38003b70 con 0x7fce500b2330 2023-11-06T03:15:50.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.655+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce500b0820 con 0x7fce500b2330 2023-11-06T03:15:50.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce500b00c0 con 0x7fce500b2330 2023-11-06T03:15:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fce38003cd0 con 0x7fce500b2330 2023-11-06T03:15:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce4cff9640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 0x7fce30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fce38082500 con 0x7fce500b2330 2023-11-06T03:15:50.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce4f7fe640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 0x7fce30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:50.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.659+0000 7fce4f7fe640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 0x7fce30067f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fce400b0000 tx=0x7fce40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:50.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.663+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fce500b00c0 con 0x7fce500b2330 2023-11-06T03:15:50.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.815+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce5006d270 con 0x7fce30065a60 2023-11-06T03:15:50.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce4cff9640 1 -- 172.21.15.157:0/1381489456 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310800 (secure 0 0 0) 0x7fce400ae440 con 0x7fce30065a60 2023-11-06T03:15:50.841 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:50.843 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:15:50.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 msgr2=0x7fce30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 0x7fce30067f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fce400b0000 tx=0x7fce40008040 comp rx=0 tx=0).stop 2023-11-06T03:15:50.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 msgr2=0x7fce500b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:50.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c490 tx=0x7fce38001e80 comp rx=0 tx=0).stop 2023-11-06T03:15:50.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 shutdown_connections 2023-11-06T03:15:50.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fce30065a60 0x7fce30067f20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fce50156a70 0x7fce500704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce500b2330 0x7fce500b0d40 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 --2- 172.21.15.157:0/1381489456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce500adae0 0x7fce50155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 >> 172.21.15.157:0/1381489456 conn(0x7fce50065930 msgr2=0x7fce50163190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 shutdown_connections 2023-11-06T03:15:50.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:50.843+0000 7fce56091640 1 -- 172.21.15.157:0/1381489456 wait complete. 2023-11-06T03:15:50.859 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2023-11-06T03:15:49.872264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":641096,"kb_used_data":15688,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754333624,"statfs":{"total":773094113280,"available":772437630976,"internally_reserved":0,"allocated":16064512,"data_stored":8047632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.405379"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921664+0000","last_change":"2023-11-06T03:15:47.914528+0000","last_active":"2023-11-06T03:15:47.921664+0000","last_peered":"2023-11-06T03:15:47.921664+0000","last_clean":"2023-11-06T03:15:47.921664+0000","last_became_active":"2023-11-06T03:15:47.913761+0000","last_became_peered":"2023-11-06T03:15:47.913761+0000","last_unstale":"2023-11-06T03:15:47.921664+0000","last_undegraded":"2023-11-06T03:15:47.921664+0000","last_fullsized":"2023-11-06T03:15:47.921664+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:04.522411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.917266+0000","last_change":"2023-11-06T03:15:47.914268+0000","last_active":"2023-11-06T03:15:47.917266+0000","last_peered":"2023-11-06T03:15:47.917266+0000","last_clean":"2023-11-06T03:15:47.917266+0000","last_became_active":"2023-11-06T03:15:47.913945+0000","last_became_peered":"2023-11-06T03:15:47.913945+0000","last_unstale":"2023-11-06T03:15:47.917266+0000","last_undegraded":"2023-11-06T03:15:47.917266+0000","last_fullsized":"2023-11-06T03:15:47.917266+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:00:02.426149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.882888+0000","last_change":"2023-11-06T03:15:47.882337+0000","last_active":"2023-11-06T03:15:47.882888+0000","last_peered":"2023-11-06T03:15:47.882888+0000","last_clean":"2023-11-06T03:15:47.882888+0000","last_became_active":"2023-11-06T03:15:47.882047+0000","last_became_peered":"2023-11-06T03:15:47.882047+0000","last_unstale":"2023-11-06T03:15:47.882888+0000","last_undegraded":"2023-11-06T03:15:47.882888+0000","last_fullsized":"2023-11-06T03:15:47.882888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922311+0000","last_change":"2023-11-06T03:15:47.901861+0000","last_active":"2023-11-06T03:15:47.922311+0000","last_peered":"2023-11-06T03:15:47.922311+0000","last_clean":"2023-11-06T03:15:47.922311+0000","last_became_active":"2023-11-06T03:15:47.901260+0000","last_became_peered":"2023-11-06T03:15:47.901260+0000","last_unstale":"2023-11-06T03:15:47.922311+0000","last_undegraded":"2023-11-06T03:15:47.922311+0000","last_fullsized":"2023-11-06T03:15:47.922311+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:40:50.438603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860496+0000","last_change":"2023-11-06T03:15:47.915338+0000","last_active":"2023-11-06T03:15:48.860496+0000","last_peered":"2023-11-06T03:15:48.860496+0000","last_clean":"2023-11-06T03:15:48.860496+0000","last_became_active":"2023-11-06T03:15:47.915077+0000","last_became_peered":"2023-11-06T03:15:47.915077+0000","last_unstale":"2023-11-06T03:15:48.860496+0000","last_undegraded":"2023-11-06T03:15:48.860496+0000","last_fullsized":"2023-11-06T03:15:48.860496+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:06.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860644+0000","last_change":"2023-11-06T03:15:47.915812+0000","last_active":"2023-11-06T03:15:48.860644+0000","last_peered":"2023-11-06T03:15:48.860644+0000","last_clean":"2023-11-06T03:15:48.860644+0000","last_became_active":"2023-11-06T03:15:47.915553+0000","last_became_peered":"2023-11-06T03:15:47.915553+0000","last_unstale":"2023-11-06T03:15:48.860644+0000","last_undegraded":"2023-11-06T03:15:48.860644+0000","last_fullsized":"2023-11-06T03:15:48.860644+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:13.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.923812+0000","last_change":"2023-11-06T03:15:47.916792+0000","last_active":"2023-11-06T03:15:47.923812+0000","last_peered":"2023-11-06T03:15:47.923812+0000","last_clean":"2023-11-06T03:15:47.923812+0000","last_became_active":"2023-11-06T03:15:47.916287+0000","last_became_peered":"2023-11-06T03:15:47.916287+0000","last_unstale":"2023-11-06T03:15:47.923812+0000","last_undegraded":"2023-11-06T03:15:47.923812+0000","last_fullsized":"2023-11-06T03:15:47.923812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:53.927074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.864921+0000","last_change":"2023-11-06T03:15:47.869483+0000","last_active":"2023-11-06T03:15:48.864921+0000","last_peered":"2023-11-06T03:15:48.864921+0000","last_clean":"2023-11-06T03:15:48.864921+0000","last_became_active":"2023-11-06T03:15:47.868593+0000","last_became_peered":"2023-11-06T03:15:47.868593+0000","last_unstale":"2023-11-06T03:15:48.864921+0000","last_undegraded":"2023-11-06T03:15:48.864921+0000","last_fullsized":"2023-11-06T03:15:48.864921+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860462+0000","last_change":"2023-11-06T03:15:47.877455+0000","last_active":"2023-11-06T03:15:48.860462+0000","last_peered":"2023-11-06T03:15:48.860462+0000","last_clean":"2023-11-06T03:15:48.860462+0000","last_became_active":"2023-11-06T03:15:47.877182+0000","last_became_peered":"2023-11-06T03:15:47.877182+0000","last_unstale":"2023-11-06T03:15:48.860462+0000","last_undegraded":"2023-11-06T03:15:48.860462+0000","last_fullsized":"2023-11-06T03:15:48.860462+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:08.027457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860728+0000","last_change":"2023-11-06T03:15:47.870971+0000","last_active":"2023-11-06T03:15:48.860728+0000","last_peered":"2023-11-06T03:15:48.860728+0000","last_clean":"2023-11-06T03:15:48.860728+0000","last_became_active":"2023-11-06T03:15:47.870692+0000","last_became_peered":"2023-11-06T03:15:47.870692+0000","last_unstale":"2023-11-06T03:15:48.860728+0000","last_undegraded":"2023-11-06T03:15:48.860728+0000","last_fullsized":"2023-11-06T03:15:48.860728+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:27.345172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876394+0000","last_change":"2023-11-06T03:15:47.873108+0000","last_active":"2023-11-06T03:15:47.876394+0000","last_peered":"2023-11-06T03:15:47.876394+0000","last_clean":"2023-11-06T03:15:47.876394+0000","last_became_active":"2023-11-06T03:15:47.872837+0000","last_became_peered":"2023-11-06T03:15:47.872837+0000","last_unstale":"2023-11-06T03:15:47.876394+0000","last_undegraded":"2023-11-06T03:15:47.876394+0000","last_fullsized":"2023-11-06T03:15:47.876394+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:55.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924465+0000","last_change":"2023-11-06T03:15:47.916323+0000","last_active":"2023-11-06T03:15:47.924465+0000","last_peered":"2023-11-06T03:15:47.924465+0000","last_clean":"2023-11-06T03:15:47.924465+0000","last_became_active":"2023-11-06T03:15:47.915726+0000","last_became_peered":"2023-11-06T03:15:47.915726+0000","last_unstale":"2023-11-06T03:15:47.924465+0000","last_undegraded":"2023-11-06T03:15:47.924465+0000","last_fullsized":"2023-11-06T03:15:47.924465+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:19.311071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860397+0000","last_change":"2023-11-06T03:15:47.874540+0000","last_active":"2023-11-06T03:15:48.860397+0000","last_peered":"2023-11-06T03:15:48.860397+0000","last_clean":"2023-11-06T03:15:48.860397+0000","last_became_active":"2023-11-06T03:15:47.874280+0000","last_became_peered":"2023-11-06T03:15:47.874280+0000","last_unstale":"2023-11-06T03:15:48.860397+0000","last_undegraded":"2023-11-06T03:15:48.860397+0000","last_fullsized":"2023-11-06T03:15:48.860397+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:38.919598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922329+0000","last_change":"2023-11-06T03:15:47.902264+0000","last_active":"2023-11-06T03:15:47.922329+0000","last_peered":"2023-11-06T03:15:47.922329+0000","last_clean":"2023-11-06T03:15:47.922329+0000","last_became_active":"2023-11-06T03:15:47.901381+0000","last_became_peered":"2023-11-06T03:15:47.901381+0000","last_unstale":"2023-11-06T03:15:47.922329+0000","last_undegraded":"2023-11-06T03:15:47.922329+0000","last_fullsized":"2023-11-06T03:15:47.922329+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:12:22.532910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905697+0000","last_change":"2023-11-06T03:15:47.904996+0000","last_active":"2023-11-06T03:15:47.905697+0000","last_peered":"2023-11-06T03:15:47.905697+0000","last_clean":"2023-11-06T03:15:47.905697+0000","last_became_active":"2023-11-06T03:15:47.904440+0000","last_became_peered":"2023-11-06T03:15:47.904440+0000","last_unstale":"2023-11-06T03:15:47.905697+0000","last_undegraded":"2023-11-06T03:15:47.905697+0000","last_fullsized":"2023-11-06T03:15:47.905697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:03:14.405112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860090+0000","last_change":"2023-11-06T03:15:47.890692+0000","last_active":"2023-11-06T03:15:48.860090+0000","last_peered":"2023-11-06T03:15:48.860090+0000","last_clean":"2023-11-06T03:15:48.860090+0000","last_became_active":"2023-11-06T03:15:47.890249+0000","last_became_peered":"2023-11-06T03:15:47.890249+0000","last_unstale":"2023-11-06T03:15:48.860090+0000","last_undegraded":"2023-11-06T03:15:48.860090+0000","last_fullsized":"2023-11-06T03:15:48.860090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:59.046349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922251+0000","last_change":"2023-11-06T03:15:47.902039+0000","last_active":"2023-11-06T03:15:47.922251+0000","last_peered":"2023-11-06T03:15:47.922251+0000","last_clean":"2023-11-06T03:15:47.922251+0000","last_became_active":"2023-11-06T03:15:47.901310+0000","last_became_peered":"2023-11-06T03:15:47.901310+0000","last_unstale":"2023-11-06T03:15:47.922251+0000","last_undegraded":"2023-11-06T03:15:47.922251+0000","last_fullsized":"2023-11-06T03:15:47.922251+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:10.208146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916637+0000","last_change":"2023-11-06T03:15:47.915102+0000","last_active":"2023-11-06T03:15:47.916637+0000","last_peered":"2023-11-06T03:15:47.916637+0000","last_clean":"2023-11-06T03:15:47.916637+0000","last_became_active":"2023-11-06T03:15:47.914327+0000","last_became_peered":"2023-11-06T03:15:47.914327+0000","last_unstale":"2023-11-06T03:15:47.916637+0000","last_undegraded":"2023-11-06T03:15:47.916637+0000","last_fullsized":"2023-11-06T03:15:47.916637+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:30:48.976495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.912271+0000","last_change":"2023-11-06T03:15:47.911358+0000","last_active":"2023-11-06T03:15:47.912271+0000","last_peered":"2023-11-06T03:15:47.912271+0000","last_clean":"2023-11-06T03:15:47.912271+0000","last_became_active":"2023-11-06T03:15:47.910674+0000","last_became_peered":"2023-11-06T03:15:47.910674+0000","last_unstale":"2023-11-06T03:15:47.912271+0000","last_undegraded":"2023-11-06T03:15:47.912271+0000","last_fullsized":"2023-11-06T03:15:47.912271+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877608+0000","last_change":"2023-11-06T03:15:47.876549+0000","last_active":"2023-11-06T03:15:47.877608+0000","last_peered":"2023-11-06T03:15:47.877608+0000","last_clean":"2023-11-06T03:15:47.877608+0000","last_became_active":"2023-11-06T03:15:47.876271+0000","last_became_peered":"2023-11-06T03:15:47.876271+0000","last_unstale":"2023-11-06T03:15:47.877608+0000","last_undegraded":"2023-11-06T03:15:47.877608+0000","last_fullsized":"2023-11-06T03:15:47.877608+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:33:44.484721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860452+0000","last_change":"2023-11-06T03:15:47.873052+0000","last_active":"2023-11-06T03:15:48.860452+0000","last_peered":"2023-11-06T03:15:48.860452+0000","last_clean":"2023-11-06T03:15:48.860452+0000","last_became_active":"2023-11-06T03:15:47.872531+0000","last_became_peered":"2023-11-06T03:15:47.872531+0000","last_unstale":"2023-11-06T03:15:48.860452+0000","last_undegraded":"2023-11-06T03:15:48.860452+0000","last_fullsized":"2023-11-06T03:15:48.860452+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:53.838120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921426+0000","last_change":"2023-11-06T03:15:47.913987+0000","last_active":"2023-11-06T03:15:47.921426+0000","last_peered":"2023-11-06T03:15:47.921426+0000","last_clean":"2023-11-06T03:15:47.921426+0000","last_became_active":"2023-11-06T03:15:47.913553+0000","last_became_peered":"2023-11-06T03:15:47.913553+0000","last_unstale":"2023-11-06T03:15:47.921426+0000","last_undegraded":"2023-11-06T03:15:47.921426+0000","last_fullsized":"2023-11-06T03:15:47.921426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:03:08.155507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.864876+0000","last_change":"2023-11-06T03:15:47.919565+0000","last_active":"2023-11-06T03:15:48.864876+0000","last_peered":"2023-11-06T03:15:48.864876+0000","last_clean":"2023-11-06T03:15:48.864876+0000","last_became_active":"2023-11-06T03:15:47.918943+0000","last_became_peered":"2023-11-06T03:15:47.918943+0000","last_unstale":"2023-11-06T03:15:48.864876+0000","last_undegraded":"2023-11-06T03:15:48.864876+0000","last_fullsized":"2023-11-06T03:15:48.864876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:25:50.652489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876113+0000","last_change":"2023-11-06T03:15:47.871715+0000","last_active":"2023-11-06T03:15:47.876113+0000","last_peered":"2023-11-06T03:15:47.876113+0000","last_clean":"2023-11-06T03:15:47.876113+0000","last_became_active":"2023-11-06T03:15:47.871221+0000","last_became_peered":"2023-11-06T03:15:47.871221+0000","last_unstale":"2023-11-06T03:15:47.876113+0000","last_undegraded":"2023-11-06T03:15:47.876113+0000","last_fullsized":"2023-11-06T03:15:47.876113+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:51.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.923788+0000","last_change":"2023-11-06T03:15:47.915785+0000","last_active":"2023-11-06T03:15:47.923788+0000","last_peered":"2023-11-06T03:15:47.923788+0000","last_clean":"2023-11-06T03:15:47.923788+0000","last_became_active":"2023-11-06T03:15:47.914490+0000","last_became_peered":"2023-11-06T03:15:47.914490+0000","last_unstale":"2023-11-06T03:15:47.923788+0000","last_undegraded":"2023-11-06T03:15:47.923788+0000","last_fullsized":"2023-11-06T03:15:47.923788+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:54.247168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921593+0000","last_change":"2023-11-06T03:15:47.914864+0000","last_active":"2023-11-06T03:15:47.921593+0000","last_peered":"2023-11-06T03:15:47.921593+0000","last_clean":"2023-11-06T03:15:47.921593+0000","last_became_active":"2023-11-06T03:15:47.914170+0000","last_became_peered":"2023-11-06T03:15:47.914170+0000","last_unstale":"2023-11-06T03:15:47.921593+0000","last_undegraded":"2023-11-06T03:15:47.921593+0000","last_fullsized":"2023-11-06T03:15:47.921593+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:25:50.652489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860793+0000","last_change":"2023-11-06T03:15:47.877611+0000","last_active":"2023-11-06T03:15:48.860793+0000","last_peered":"2023-11-06T03:15:48.860793+0000","last_clean":"2023-11-06T03:15:48.860793+0000","last_became_active":"2023-11-06T03:15:47.877291+0000","last_became_peered":"2023-11-06T03:15:47.877291+0000","last_unstale":"2023-11-06T03:15:48.860793+0000","last_undegraded":"2023-11-06T03:15:48.860793+0000","last_fullsized":"2023-11-06T03:15:48.860793+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:57.616562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877493+0000","last_change":"2023-11-06T03:15:47.865193+0000","last_active":"2023-11-06T03:15:47.877493+0000","last_peered":"2023-11-06T03:15:47.877493+0000","last_clean":"2023-11-06T03:15:47.877493+0000","last_became_active":"2023-11-06T03:15:47.864926+0000","last_became_peered":"2023-11-06T03:15:47.864926+0000","last_unstale":"2023-11-06T03:15:47.877493+0000","last_undegraded":"2023-11-06T03:15:47.877493+0000","last_fullsized":"2023-11-06T03:15:47.877493+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:22.812985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866514+0000","last_change":"2023-11-06T03:15:47.881506+0000","last_active":"2023-11-06T03:15:48.866514+0000","last_peered":"2023-11-06T03:15:48.866514+0000","last_clean":"2023-11-06T03:15:48.866514+0000","last_became_active":"2023-11-06T03:15:47.879999+0000","last_became_peered":"2023-11-06T03:15:47.879999+0000","last_unstale":"2023-11-06T03:15:48.866514+0000","last_undegraded":"2023-11-06T03:15:48.866514+0000","last_fullsized":"2023-11-06T03:15:48.866514+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.902180+0000","last_change":"2023-11-06T03:15:47.901364+0000","last_active":"2023-11-06T03:15:47.902180+0000","last_peered":"2023-11-06T03:15:47.902180+0000","last_clean":"2023-11-06T03:15:47.902180+0000","last_became_active":"2023-11-06T03:15:47.901067+0000","last_became_peered":"2023-11-06T03:15:47.901067+0000","last_unstale":"2023-11-06T03:15:47.902180+0000","last_undegraded":"2023-11-06T03:15:47.902180+0000","last_fullsized":"2023-11-06T03:15:47.902180+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:47.771294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860178+0000","last_change":"2023-11-06T03:15:47.880028+0000","last_active":"2023-11-06T03:15:48.860178+0000","last_peered":"2023-11-06T03:15:48.860178+0000","last_clean":"2023-11-06T03:15:48.860178+0000","last_became_active":"2023-11-06T03:15:47.879545+0000","last_became_peered":"2023-11-06T03:15:47.879545+0000","last_unstale":"2023-11-06T03:15:48.860178+0000","last_undegraded":"2023-11-06T03:15:48.860178+0000","last_fullsized":"2023-11-06T03:15:48.860178+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:38:45.703881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865168+0000","last_change":"2023-11-06T03:15:47.869086+0000","last_active":"2023-11-06T03:15:48.865168+0000","last_peered":"2023-11-06T03:15:48.865168+0000","last_clean":"2023-11-06T03:15:48.865168+0000","last_became_active":"2023-11-06T03:15:47.868388+0000","last_became_peered":"2023-11-06T03:15:47.868388+0000","last_unstale":"2023-11-06T03:15:48.865168+0000","last_undegraded":"2023-11-06T03:15:48.865168+0000","last_fullsized":"2023-11-06T03:15:48.865168+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:04.522411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.883197+0000","last_change":"2023-11-06T03:15:47.882716+0000","last_active":"2023-11-06T03:15:47.883197+0000","last_peered":"2023-11-06T03:15:47.883197+0000","last_clean":"2023-11-06T03:15:47.883197+0000","last_became_active":"2023-11-06T03:15:47.882295+0000","last_became_peered":"2023-11-06T03:15:47.882295+0000","last_unstale":"2023-11-06T03:15:47.883197+0000","last_undegraded":"2023-11-06T03:15:47.883197+0000","last_fullsized":"2023-11-06T03:15:47.883197+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"148'4","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860650+0000","last_change":"2023-11-06T03:15:47.495275+0000","last_active":"2023-11-06T03:15:48.860650+0000","last_peered":"2023-11-06T03:15:48.860650+0000","last_clean":"2023-11-06T03:15:48.860650+0000","last_became_active":"2023-11-06T03:15:47.494945+0000","last_became_peered":"2023-11-06T03:15:47.494945+0000","last_unstale":"2023-11-06T03:15:48.860650+0000","last_undegraded":"2023-11-06T03:15:48.860650+0000","last_fullsized":"2023-11-06T03:15:48.860650+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:45.807010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916629+0000","last_change":"2023-11-06T03:15:47.914148+0000","last_active":"2023-11-06T03:15:47.916629+0000","last_peered":"2023-11-06T03:15:47.916629+0000","last_clean":"2023-11-06T03:15:47.916629+0000","last_became_active":"2023-11-06T03:15:47.913871+0000","last_became_peered":"2023-11-06T03:15:47.913871+0000","last_unstale":"2023-11-06T03:15:47.916629+0000","last_undegraded":"2023-11-06T03:15:47.916629+0000","last_fullsized":"2023-11-06T03:15:47.916629+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:55.317634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.899403+0000","last_change":"2023-11-06T03:15:47.512311+0000","last_active":"2023-11-06T03:15:47.899403+0000","last_peered":"2023-11-06T03:15:47.899403+0000","last_clean":"2023-11-06T03:15:47.899403+0000","last_became_active":"2023-11-06T03:15:47.511689+0000","last_became_peered":"2023-11-06T03:15:47.511689+0000","last_unstale":"2023-11-06T03:15:47.899403+0000","last_undegraded":"2023-11-06T03:15:47.899403+0000","last_fullsized":"2023-11-06T03:15:47.899403+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:53.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866182+0000","last_change":"2023-11-06T03:15:47.919361+0000","last_active":"2023-11-06T03:15:48.866182+0000","last_peered":"2023-11-06T03:15:48.866182+0000","last_clean":"2023-11-06T03:15:48.866182+0000","last_became_active":"2023-11-06T03:15:47.918905+0000","last_became_peered":"2023-11-06T03:15:47.918905+0000","last_unstale":"2023-11-06T03:15:48.866182+0000","last_undegraded":"2023-11-06T03:15:48.866182+0000","last_fullsized":"2023-11-06T03:15:48.866182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:53.927074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.899881+0000","last_change":"2023-11-06T03:15:47.510786+0000","last_active":"2023-11-06T03:15:47.899881+0000","last_peered":"2023-11-06T03:15:47.899881+0000","last_clean":"2023-11-06T03:15:47.899881+0000","last_became_active":"2023-11-06T03:15:47.510214+0000","last_became_peered":"2023-11-06T03:15:47.510214+0000","last_unstale":"2023-11-06T03:15:47.899881+0000","last_undegraded":"2023-11-06T03:15:47.899881+0000","last_fullsized":"2023-11-06T03:15:47.899881+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:34.697001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.919680+0000","last_change":"2023-11-06T03:15:47.917509+0000","last_active":"2023-11-06T03:15:47.919680+0000","last_peered":"2023-11-06T03:15:47.919680+0000","last_clean":"2023-11-06T03:15:47.919680+0000","last_became_active":"2023-11-06T03:15:47.916912+0000","last_became_peered":"2023-11-06T03:15:47.916912+0000","last_unstale":"2023-11-06T03:15:47.919680+0000","last_undegraded":"2023-11-06T03:15:47.919680+0000","last_fullsized":"2023-11-06T03:15:47.919680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.908101+0000","last_change":"2023-11-06T03:15:47.525307+0000","last_active":"2023-11-06T03:15:47.908101+0000","last_peered":"2023-11-06T03:15:47.908101+0000","last_clean":"2023-11-06T03:15:47.908101+0000","last_became_active":"2023-11-06T03:15:47.524915+0000","last_became_peered":"2023-11-06T03:15:47.524915+0000","last_unstale":"2023-11-06T03:15:47.908101+0000","last_undegraded":"2023-11-06T03:15:47.908101+0000","last_fullsized":"2023-11-06T03:15:47.908101+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:48.117488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.869959+0000","last_change":"2023-11-06T03:15:47.867593+0000","last_active":"2023-11-06T03:15:47.869959+0000","last_peered":"2023-11-06T03:15:47.869959+0000","last_clean":"2023-11-06T03:15:47.869959+0000","last_became_active":"2023-11-06T03:15:47.867181+0000","last_became_peered":"2023-11-06T03:15:47.867181+0000","last_unstale":"2023-11-06T03:15:47.869959+0000","last_undegraded":"2023-11-06T03:15:47.869959+0000","last_fullsized":"2023-11-06T03:15:47.869959+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:51.382858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"148'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865915+0000","last_change":"2023-11-06T03:15:47.518422+0000","last_active":"2023-11-06T03:15:48.865915+0000","last_peered":"2023-11-06T03:15:48.865915+0000","last_clean":"2023-11-06T03:15:48.865915+0000","last_became_active":"2023-11-06T03:15:47.516127+0000","last_became_peered":"2023-11-06T03:15:47.516127+0000","last_unstale":"2023-11-06T03:15:48.865915+0000","last_undegraded":"2023-11-06T03:15:48.865915+0000","last_fullsized":"2023-11-06T03:15:48.865915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:29.083744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922348+0000","last_change":"2023-11-06T03:15:47.904066+0000","last_active":"2023-11-06T03:15:47.922348+0000","last_peered":"2023-11-06T03:15:47.922348+0000","last_clean":"2023-11-06T03:15:47.922348+0000","last_became_active":"2023-11-06T03:15:47.903431+0000","last_became_peered":"2023-11-06T03:15:47.903431+0000","last_unstale":"2023-11-06T03:15:47.922348+0000","last_undegraded":"2023-11-06T03:15:47.922348+0000","last_fullsized":"2023-11-06T03:15:47.922348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:02.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"148'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860773+0000","last_change":"2023-11-06T03:15:47.509422+0000","last_active":"2023-11-06T03:15:48.860773+0000","last_peered":"2023-11-06T03:15:48.860773+0000","last_clean":"2023-11-06T03:15:48.860773+0000","last_became_active":"2023-11-06T03:15:47.508910+0000","last_became_peered":"2023-11-06T03:15:47.508910+0000","last_unstale":"2023-11-06T03:15:48.860773+0000","last_undegraded":"2023-11-06T03:15:48.860773+0000","last_fullsized":"2023-11-06T03:15:48.860773+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:52.924237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878684+0000","last_change":"2023-11-06T03:15:47.875146+0000","last_active":"2023-11-06T03:15:47.878684+0000","last_peered":"2023-11-06T03:15:47.878684+0000","last_clean":"2023-11-06T03:15:47.878684+0000","last_became_active":"2023-11-06T03:15:47.874725+0000","last_became_peered":"2023-11-06T03:15:47.874725+0000","last_unstale":"2023-11-06T03:15:47.878684+0000","last_undegraded":"2023-11-06T03:15:47.878684+0000","last_fullsized":"2023-11-06T03:15:47.878684+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897432+0000","last_change":"2023-11-06T03:15:47.492282+0000","last_active":"2023-11-06T03:15:47.897432+0000","last_peered":"2023-11-06T03:15:47.897432+0000","last_clean":"2023-11-06T03:15:47.897432+0000","last_became_active":"2023-11-06T03:15:47.491702+0000","last_became_peered":"2023-11-06T03:15:47.491702+0000","last_unstale":"2023-11-06T03:15:47.897432+0000","last_undegraded":"2023-11-06T03:15:47.897432+0000","last_fullsized":"2023-11-06T03:15:47.897432+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:01.984847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921401+0000","last_change":"2023-11-06T03:15:47.913766+0000","last_active":"2023-11-06T03:15:47.921401+0000","last_peered":"2023-11-06T03:15:47.921401+0000","last_clean":"2023-11-06T03:15:47.921401+0000","last_became_active":"2023-11-06T03:15:47.913186+0000","last_became_peered":"2023-11-06T03:15:47.913186+0000","last_unstale":"2023-11-06T03:15:47.921401+0000","last_undegraded":"2023-11-06T03:15:47.921401+0000","last_fullsized":"2023-11-06T03:15:47.921401+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860178+0000","last_change":"2023-11-06T03:15:47.509099+0000","last_active":"2023-11-06T03:15:48.860178+0000","last_peered":"2023-11-06T03:15:48.860178+0000","last_clean":"2023-11-06T03:15:48.860178+0000","last_became_active":"2023-11-06T03:15:47.508623+0000","last_became_peered":"2023-11-06T03:15:47.508623+0000","last_unstale":"2023-11-06T03:15:48.860178+0000","last_undegraded":"2023-11-06T03:15:48.860178+0000","last_fullsized":"2023-11-06T03:15:48.860178+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:35.361093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905541+0000","last_change":"2023-11-06T03:15:47.893008+0000","last_active":"2023-11-06T03:15:47.905541+0000","last_peered":"2023-11-06T03:15:47.905541+0000","last_clean":"2023-11-06T03:15:47.905541+0000","last_became_active":"2023-11-06T03:15:47.892227+0000","last_became_peered":"2023-11-06T03:15:47.892227+0000","last_unstale":"2023-11-06T03:15:47.905541+0000","last_undegraded":"2023-11-06T03:15:47.905541+0000","last_fullsized":"2023-11-06T03:15:47.905541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878369+0000","last_change":"2023-11-06T03:15:47.507458+0000","last_active":"2023-11-06T03:15:47.878369+0000","last_peered":"2023-11-06T03:15:47.878369+0000","last_clean":"2023-11-06T03:15:47.878369+0000","last_became_active":"2023-11-06T03:15:47.507035+0000","last_became_peered":"2023-11-06T03:15:47.507035+0000","last_unstale":"2023-11-06T03:15:47.878369+0000","last_undegraded":"2023-11-06T03:15:47.878369+0000","last_fullsized":"2023-11-06T03:15:47.878369+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:04:30.873334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.907468+0000","last_change":"2023-11-06T03:15:47.905474+0000","last_active":"2023-11-06T03:15:47.907468+0000","last_peered":"2023-11-06T03:15:47.907468+0000","last_clean":"2023-11-06T03:15:47.907468+0000","last_became_active":"2023-11-06T03:15:47.904286+0000","last_became_peered":"2023-11-06T03:15:47.904286+0000","last_unstale":"2023-11-06T03:15:47.907468+0000","last_undegraded":"2023-11-06T03:15:47.907468+0000","last_fullsized":"2023-11-06T03:15:47.907468+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886377+0000","last_change":"2023-11-06T03:15:47.507568+0000","last_active":"2023-11-06T03:15:47.886377+0000","last_peered":"2023-11-06T03:15:47.886377+0000","last_clean":"2023-11-06T03:15:47.886377+0000","last_became_active":"2023-11-06T03:15:47.507005+0000","last_became_peered":"2023-11-06T03:15:47.507005+0000","last_unstale":"2023-11-06T03:15:47.886377+0000","last_undegraded":"2023-11-06T03:15:47.886377+0000","last_fullsized":"2023-11-06T03:15:47.886377+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:31.541526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921622+0000","last_change":"2023-11-06T03:15:47.914949+0000","last_active":"2023-11-06T03:15:47.921622+0000","last_peered":"2023-11-06T03:15:47.921622+0000","last_clean":"2023-11-06T03:15:47.921622+0000","last_became_active":"2023-11-06T03:15:47.914385+0000","last_became_peered":"2023-11-06T03:15:47.914385+0000","last_unstale":"2023-11-06T03:15:47.921622+0000","last_undegraded":"2023-11-06T03:15:47.921622+0000","last_fullsized":"2023-11-06T03:15:47.921622+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:27.745003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886949+0000","last_change":"2023-11-06T03:15:47.509075+0000","last_active":"2023-11-06T03:15:47.886949+0000","last_peered":"2023-11-06T03:15:47.886949+0000","last_clean":"2023-11-06T03:15:47.886949+0000","last_became_active":"2023-11-06T03:15:47.508821+0000","last_became_peered":"2023-11-06T03:15:47.508821+0000","last_unstale":"2023-11-06T03:15:47.886949+0000","last_undegraded":"2023-11-06T03:15:47.886949+0000","last_fullsized":"2023-11-06T03:15:47.886949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:45:06.497188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924055+0000","last_change":"2023-11-06T03:15:47.920514+0000","last_active":"2023-11-06T03:15:47.924055+0000","last_peered":"2023-11-06T03:15:47.924055+0000","last_clean":"2023-11-06T03:15:47.924055+0000","last_became_active":"2023-11-06T03:15:47.920040+0000","last_became_peered":"2023-11-06T03:15:47.920040+0000","last_unstale":"2023-11-06T03:15:47.924055+0000","last_undegraded":"2023-11-06T03:15:47.924055+0000","last_fullsized":"2023-11-06T03:15:47.924055+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:46.809847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.887413+0000","last_change":"2023-11-06T03:15:47.527699+0000","last_active":"2023-11-06T03:15:47.887413+0000","last_peered":"2023-11-06T03:15:47.887413+0000","last_clean":"2023-11-06T03:15:47.887413+0000","last_became_active":"2023-11-06T03:15:47.527035+0000","last_became_peered":"2023-11-06T03:15:47.527035+0000","last_unstale":"2023-11-06T03:15:47.887413+0000","last_undegraded":"2023-11-06T03:15:47.887413+0000","last_fullsized":"2023-11-06T03:15:47.887413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:35:08.162833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.906758+0000","last_change":"2023-11-06T03:15:47.903415+0000","last_active":"2023-11-06T03:15:47.906758+0000","last_peered":"2023-11-06T03:15:47.906758+0000","last_clean":"2023-11-06T03:15:47.906758+0000","last_became_active":"2023-11-06T03:15:47.902700+0000","last_became_peered":"2023-11-06T03:15:47.902700+0000","last_unstale":"2023-11-06T03:15:47.906758+0000","last_undegraded":"2023-11-06T03:15:47.906758+0000","last_fullsized":"2023-11-06T03:15:47.906758+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.879626+0000","last_change":"2023-11-06T03:15:47.525119+0000","last_active":"2023-11-06T03:15:47.879626+0000","last_peered":"2023-11-06T03:15:47.879626+0000","last_clean":"2023-11-06T03:15:47.879626+0000","last_became_active":"2023-11-06T03:15:47.524759+0000","last_became_peered":"2023-11-06T03:15:47.524759+0000","last_unstale":"2023-11-06T03:15:47.879626+0000","last_undegraded":"2023-11-06T03:15:47.879626+0000","last_fullsized":"2023-11-06T03:15:47.879626+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:50.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.917316+0000","last_change":"2023-11-06T03:15:47.915837+0000","last_active":"2023-11-06T03:15:47.917316+0000","last_peered":"2023-11-06T03:15:47.917316+0000","last_clean":"2023-11-06T03:15:47.917316+0000","last_became_active":"2023-11-06T03:15:47.915562+0000","last_became_peered":"2023-11-06T03:15:47.915562+0000","last_unstale":"2023-11-06T03:15:47.917316+0000","last_undegraded":"2023-11-06T03:15:47.917316+0000","last_fullsized":"2023-11-06T03:15:47.917316+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865355+0000","last_change":"2023-11-06T03:15:47.514671+0000","last_active":"2023-11-06T03:15:48.865355+0000","last_peered":"2023-11-06T03:15:48.865355+0000","last_clean":"2023-11-06T03:15:48.865355+0000","last_became_active":"2023-11-06T03:15:47.513777+0000","last_became_peered":"2023-11-06T03:15:47.513777+0000","last_unstale":"2023-11-06T03:15:48.865355+0000","last_undegraded":"2023-11-06T03:15:48.865355+0000","last_fullsized":"2023-11-06T03:15:48.865355+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:01:05.855936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.891996+0000","last_change":"2023-11-06T03:15:47.880623+0000","last_active":"2023-11-06T03:15:47.891996+0000","last_peered":"2023-11-06T03:15:47.891996+0000","last_clean":"2023-11-06T03:15:47.891996+0000","last_became_active":"2023-11-06T03:15:47.879820+0000","last_became_peered":"2023-11-06T03:15:47.879820+0000","last_unstale":"2023-11-06T03:15:47.891996+0000","last_undegraded":"2023-11-06T03:15:47.891996+0000","last_fullsized":"2023-11-06T03:15:47.891996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.857915+0000","last_change":"2023-11-06T03:15:47.525153+0000","last_active":"2023-11-06T03:15:47.857915+0000","last_peered":"2023-11-06T03:15:47.857915+0000","last_clean":"2023-11-06T03:15:47.857915+0000","last_became_active":"2023-11-06T03:15:47.524724+0000","last_became_peered":"2023-11-06T03:15:47.524724+0000","last_unstale":"2023-11-06T03:15:47.857915+0000","last_undegraded":"2023-11-06T03:15:47.857915+0000","last_fullsized":"2023-11-06T03:15:47.857915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:46.699184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.902546+0000","last_change":"2023-11-06T03:15:47.901780+0000","last_active":"2023-11-06T03:15:47.902546+0000","last_peered":"2023-11-06T03:15:47.902546+0000","last_clean":"2023-11-06T03:15:47.902546+0000","last_became_active":"2023-11-06T03:15:47.901421+0000","last_became_peered":"2023-11-06T03:15:47.901421+0000","last_unstale":"2023-11-06T03:15:47.902546+0000","last_undegraded":"2023-11-06T03:15:47.902546+0000","last_fullsized":"2023-11-06T03:15:47.902546+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:15.438881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865509+0000","last_change":"2023-11-06T03:15:47.526863+0000","last_active":"2023-11-06T03:15:48.865509+0000","last_peered":"2023-11-06T03:15:48.865509+0000","last_clean":"2023-11-06T03:15:48.865509+0000","last_became_active":"2023-11-06T03:15:47.526344+0000","last_became_peered":"2023-11-06T03:15:47.526344+0000","last_unstale":"2023-11-06T03:15:48.865509+0000","last_undegraded":"2023-11-06T03:15:48.865509+0000","last_fullsized":"2023-11-06T03:15:48.865509+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:19:26.518153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860601+0000","last_change":"2023-11-06T03:15:47.912331+0000","last_active":"2023-11-06T03:15:48.860601+0000","last_peered":"2023-11-06T03:15:48.860601+0000","last_clean":"2023-11-06T03:15:48.860601+0000","last_became_active":"2023-11-06T03:15:47.912122+0000","last_became_peered":"2023-11-06T03:15:47.912122+0000","last_unstale":"2023-11-06T03:15:48.860601+0000","last_undegraded":"2023-11-06T03:15:48.860601+0000","last_fullsized":"2023-11-06T03:15:48.860601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:14:18.764251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.915613+0000","last_change":"2023-11-06T03:15:47.527814+0000","last_active":"2023-11-06T03:15:47.915613+0000","last_peered":"2023-11-06T03:15:47.915613+0000","last_clean":"2023-11-06T03:15:47.915613+0000","last_became_active":"2023-11-06T03:15:47.527003+0000","last_became_peered":"2023-11-06T03:15:47.527003+0000","last_unstale":"2023-11-06T03:15:47.915613+0000","last_undegraded":"2023-11-06T03:15:47.915613+0000","last_fullsized":"2023-11-06T03:15:47.915613+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:00.368386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924443+0000","last_change":"2023-11-06T03:15:47.915067+0000","last_active":"2023-11-06T03:15:47.924443+0000","last_peered":"2023-11-06T03:15:47.924443+0000","last_clean":"2023-11-06T03:15:47.924443+0000","last_became_active":"2023-11-06T03:15:47.914683+0000","last_became_peered":"2023-11-06T03:15:47.914683+0000","last_unstale":"2023-11-06T03:15:47.924443+0000","last_undegraded":"2023-11-06T03:15:47.924443+0000","last_fullsized":"2023-11-06T03:15:47.924443+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:35.479381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.898500+0000","last_change":"2023-11-06T03:15:47.509688+0000","last_active":"2023-11-06T03:15:47.898500+0000","last_peered":"2023-11-06T03:15:47.898500+0000","last_clean":"2023-11-06T03:15:47.898500+0000","last_became_active":"2023-11-06T03:15:47.509023+0000","last_became_peered":"2023-11-06T03:15:47.509023+0000","last_unstale":"2023-11-06T03:15:47.898500+0000","last_undegraded":"2023-11-06T03:15:47.898500+0000","last_fullsized":"2023-11-06T03:15:47.898500+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:12:21.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921563+0000","last_change":"2023-11-06T03:15:47.913697+0000","last_active":"2023-11-06T03:15:47.921563+0000","last_peered":"2023-11-06T03:15:47.921563+0000","last_clean":"2023-11-06T03:15:47.921563+0000","last_became_active":"2023-11-06T03:15:47.913139+0000","last_became_peered":"2023-11-06T03:15:47.913139+0000","last_unstale":"2023-11-06T03:15:47.921563+0000","last_undegraded":"2023-11-06T03:15:47.921563+0000","last_fullsized":"2023-11-06T03:15:47.921563+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:32.150958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.866543+0000","last_change":"2023-11-06T03:15:47.525263+0000","last_active":"2023-11-06T03:15:47.866543+0000","last_peered":"2023-11-06T03:15:47.866543+0000","last_clean":"2023-11-06T03:15:47.866543+0000","last_became_active":"2023-11-06T03:15:47.524956+0000","last_became_peered":"2023-11-06T03:15:47.524956+0000","last_unstale":"2023-11-06T03:15:47.866543+0000","last_undegraded":"2023-11-06T03:15:47.866543+0000","last_fullsized":"2023-11-06T03:15:47.866543+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:30.870788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860577+0000","last_change":"2023-11-06T03:15:47.915229+0000","last_active":"2023-11-06T03:15:48.860577+0000","last_peered":"2023-11-06T03:15:48.860577+0000","last_clean":"2023-11-06T03:15:48.860577+0000","last_became_active":"2023-11-06T03:15:47.914946+0000","last_became_peered":"2023-11-06T03:15:47.914946+0000","last_unstale":"2023-11-06T03:15:48.860577+0000","last_undegraded":"2023-11-06T03:15:48.860577+0000","last_fullsized":"2023-11-06T03:15:48.860577+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:56.264390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860472+0000","last_change":"2023-11-06T03:15:47.525331+0000","last_active":"2023-11-06T03:15:48.860472+0000","last_peered":"2023-11-06T03:15:48.860472+0000","last_clean":"2023-11-06T03:15:48.860472+0000","last_became_active":"2023-11-06T03:15:47.525007+0000","last_became_peered":"2023-11-06T03:15:47.525007+0000","last_unstale":"2023-11-06T03:15:48.860472+0000","last_undegraded":"2023-11-06T03:15:48.860472+0000","last_fullsized":"2023-11-06T03:15:48.860472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:03.519574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865407+0000","last_change":"2023-11-06T03:15:47.881406+0000","last_active":"2023-11-06T03:15:48.865407+0000","last_peered":"2023-11-06T03:15:48.865407+0000","last_clean":"2023-11-06T03:15:48.865407+0000","last_became_active":"2023-11-06T03:15:47.878599+0000","last_became_peered":"2023-11-06T03:15:47.878599+0000","last_unstale":"2023-11-06T03:15:48.865407+0000","last_undegraded":"2023-11-06T03:15:48.865407+0000","last_fullsized":"2023-11-06T03:15:48.865407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897846+0000","last_change":"2023-11-06T03:15:47.511309+0000","last_active":"2023-11-06T03:15:47.897846+0000","last_peered":"2023-11-06T03:15:47.897846+0000","last_clean":"2023-11-06T03:15:47.897846+0000","last_became_active":"2023-11-06T03:15:47.510724+0000","last_became_peered":"2023-11-06T03:15:47.510724+0000","last_unstale":"2023-11-06T03:15:47.897846+0000","last_undegraded":"2023-11-06T03:15:47.897846+0000","last_fullsized":"2023-11-06T03:15:47.897846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:29:05.547526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922510+0000","last_change":"2023-11-06T03:15:47.908992+0000","last_active":"2023-11-06T03:15:47.922510+0000","last_peered":"2023-11-06T03:15:47.922510+0000","last_clean":"2023-11-06T03:15:47.922510+0000","last_became_active":"2023-11-06T03:15:47.903685+0000","last_became_peered":"2023-11-06T03:15:47.903685+0000","last_unstale":"2023-11-06T03:15:47.922510+0000","last_undegraded":"2023-11-06T03:15:47.922510+0000","last_fullsized":"2023-11-06T03:15:47.922510+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:17.797481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.851749+0000","last_change":"2023-11-06T03:15:47.518655+0000","last_active":"2023-11-06T03:15:47.851749+0000","last_peered":"2023-11-06T03:15:47.851749+0000","last_clean":"2023-11-06T03:15:47.851749+0000","last_became_active":"2023-11-06T03:15:47.518326+0000","last_became_peered":"2023-11-06T03:15:47.518326+0000","last_unstale":"2023-11-06T03:15:47.851749+0000","last_undegraded":"2023-11-06T03:15:47.851749+0000","last_fullsized":"2023-11-06T03:15:47.851749+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:56.547181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878651+0000","last_change":"2023-11-06T03:15:47.874653+0000","last_active":"2023-11-06T03:15:47.878651+0000","last_peered":"2023-11-06T03:15:47.878651+0000","last_clean":"2023-11-06T03:15:47.878651+0000","last_became_active":"2023-11-06T03:15:47.872756+0000","last_became_peered":"2023-11-06T03:15:47.872756+0000","last_unstale":"2023-11-06T03:15:47.878651+0000","last_undegraded":"2023-11-06T03:15:47.878651+0000","last_fullsized":"2023-11-06T03:15:47.878651+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897471+0000","last_change":"2023-11-06T03:15:47.494228+0000","last_active":"2023-11-06T03:15:47.897471+0000","last_peered":"2023-11-06T03:15:47.897471+0000","last_clean":"2023-11-06T03:15:47.897471+0000","last_became_active":"2023-11-06T03:15:47.493507+0000","last_became_peered":"2023-11-06T03:15:47.493507+0000","last_unstale":"2023-11-06T03:15:47.897471+0000","last_undegraded":"2023-11-06T03:15:47.897471+0000","last_fullsized":"2023-11-06T03:15:47.897471+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:34.476544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865505+0000","last_change":"2023-11-06T03:15:47.881947+0000","last_active":"2023-11-06T03:15:48.865505+0000","last_peered":"2023-11-06T03:15:48.865505+0000","last_clean":"2023-11-06T03:15:48.865505+0000","last_became_active":"2023-11-06T03:15:47.881192+0000","last_became_peered":"2023-11-06T03:15:47.881192+0000","last_unstale":"2023-11-06T03:15:48.865505+0000","last_undegraded":"2023-11-06T03:15:48.865505+0000","last_fullsized":"2023-11-06T03:15:48.865505+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.873724+0000","last_change":"2023-11-06T03:15:47.507220+0000","last_active":"2023-11-06T03:15:47.873724+0000","last_peered":"2023-11-06T03:15:47.873724+0000","last_clean":"2023-11-06T03:15:47.873724+0000","last_became_active":"2023-11-06T03:15:47.506684+0000","last_became_peered":"2023-11-06T03:15:47.506684+0000","last_unstale":"2023-11-06T03:15:47.873724+0000","last_undegraded":"2023-11-06T03:15:47.873724+0000","last_fullsized":"2023-11-06T03:15:47.873724+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:48.117488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866302+0000","last_change":"2023-11-06T03:15:47.889415+0000","last_active":"2023-11-06T03:15:48.866302+0000","last_peered":"2023-11-06T03:15:48.866302+0000","last_clean":"2023-11-06T03:15:48.866302+0000","last_became_active":"2023-11-06T03:15:47.869850+0000","last_became_peered":"2023-11-06T03:15:47.869850+0000","last_unstale":"2023-11-06T03:15:48.866302+0000","last_undegraded":"2023-11-06T03:15:48.866302+0000","last_fullsized":"2023-11-06T03:15:48.866302+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"148'6","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921137+0000","last_change":"2023-11-06T03:15:47.512744+0000","last_active":"2023-11-06T03:15:47.921137+0000","last_peered":"2023-11-06T03:15:47.921137+0000","last_clean":"2023-11-06T03:15:47.921137+0000","last_became_active":"2023-11-06T03:15:47.512370+0000","last_became_peered":"2023-11-06T03:15:47.512370+0000","last_unstale":"2023-11-06T03:15:47.921137+0000","last_undegraded":"2023-11-06T03:15:47.921137+0000","last_fullsized":"2023-11-06T03:15:47.921137+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:21.567227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":270,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490712+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:15:47.490712+0000","last_peered":"2023-11-06T03:15:47.490712+0000","last_clean":"2023-11-06T03:15:47.490712+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:15:47.490712+0000","last_undegraded":"2023-11-06T03:15:47.490712+0000","last_fullsized":"2023-11-06T03:15:47.490712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":114,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.491754+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:15:47.491754+0000","last_peered":"2023-11-06T03:15:47.491754+0000","last_clean":"2023-11-06T03:15:47.491754+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:15:47.491754+0000","last_undegraded":"2023-11-06T03:15:47.491754+0000","last_fullsized":"2023-11-06T03:15:47.491754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865855+0000","last_change":"2023-11-06T03:15:47.895227+0000","last_active":"2023-11-06T03:15:48.865855+0000","last_peered":"2023-11-06T03:15:48.865855+0000","last_clean":"2023-11-06T03:15:48.865855+0000","last_became_active":"2023-11-06T03:15:47.889409+0000","last_became_peered":"2023-11-06T03:15:47.889409+0000","last_unstale":"2023-11-06T03:15:48.865855+0000","last_undegraded":"2023-11-06T03:15:48.865855+0000","last_fullsized":"2023-11-06T03:15:48.865855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.880053+0000","last_change":"2023-11-06T03:15:47.508753+0000","last_active":"2023-11-06T03:15:47.880053+0000","last_peered":"2023-11-06T03:15:47.880053+0000","last_clean":"2023-11-06T03:15:47.880053+0000","last_became_active":"2023-11-06T03:15:47.508109+0000","last_became_peered":"2023-11-06T03:15:47.508109+0000","last_unstale":"2023-11-06T03:15:47.880053+0000","last_undegraded":"2023-11-06T03:15:47.880053+0000","last_fullsized":"2023-11-06T03:15:47.880053+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:49:07.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":117,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490530+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:15:47.490530+0000","last_peered":"2023-11-06T03:15:47.490530+0000","last_clean":"2023-11-06T03:15:47.490530+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:15:47.490530+0000","last_undegraded":"2023-11-06T03:15:47.490530+0000","last_fullsized":"2023-11-06T03:15:47.490530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876358+0000","last_change":"2023-11-06T03:15:47.870180+0000","last_active":"2023-11-06T03:15:47.876358+0000","last_peered":"2023-11-06T03:15:47.876358+0000","last_clean":"2023-11-06T03:15:47.876358+0000","last_became_active":"2023-11-06T03:15:47.869900+0000","last_became_peered":"2023-11-06T03:15:47.869900+0000","last_unstale":"2023-11-06T03:15:47.876358+0000","last_undegraded":"2023-11-06T03:15:47.876358+0000","last_fullsized":"2023-11-06T03:15:47.876358+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:40.548877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.898235+0000","last_change":"2023-11-06T03:15:47.499101+0000","last_active":"2023-11-06T03:15:47.898235+0000","last_peered":"2023-11-06T03:15:47.898235+0000","last_clean":"2023-11-06T03:15:47.898235+0000","last_became_active":"2023-11-06T03:15:47.498544+0000","last_became_peered":"2023-11-06T03:15:47.498544+0000","last_unstale":"2023-11-06T03:15:47.898235+0000","last_undegraded":"2023-11-06T03:15:47.898235+0000","last_fullsized":"2023-11-06T03:15:47.898235+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:16.794644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.489167+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:15:47.489167+0000","last_peered":"2023-11-06T03:15:47.489167+0000","last_clean":"2023-11-06T03:15:47.489167+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:15:47.489167+0000","last_undegraded":"2023-11-06T03:15:47.489167+0000","last_fullsized":"2023-11-06T03:15:47.489167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916619+0000","last_change":"2023-11-06T03:15:47.914994+0000","last_active":"2023-11-06T03:15:47.916619+0000","last_peered":"2023-11-06T03:15:47.916619+0000","last_clean":"2023-11-06T03:15:47.916619+0000","last_became_active":"2023-11-06T03:15:47.914194+0000","last_became_peered":"2023-11-06T03:15:47.914194+0000","last_unstale":"2023-11-06T03:15:47.916619+0000","last_undegraded":"2023-11-06T03:15:47.916619+0000","last_fullsized":"2023-11-06T03:15:47.916619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:50:40.242424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"148'3","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877232+0000","last_change":"2023-11-06T03:15:47.491520+0000","last_active":"2023-11-06T03:15:47.877232+0000","last_peered":"2023-11-06T03:15:47.877232+0000","last_clean":"2023-11-06T03:15:47.877232+0000","last_became_active":"2023-11-06T03:15:47.491014+0000","last_became_peered":"2023-11-06T03:15:47.491014+0000","last_unstale":"2023-11-06T03:15:47.877232+0000","last_undegraded":"2023-11-06T03:15:47.877232+0000","last_fullsized":"2023-11-06T03:15:47.877232+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:45:27.664756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":114,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490703+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:15:47.490703+0000","last_peered":"2023-11-06T03:15:47.490703+0000","last_clean":"2023-11-06T03:15:47.490703+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:15:47.490703+0000","last_undegraded":"2023-11-06T03:15:47.490703+0000","last_fullsized":"2023-11-06T03:15:47.490703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865297+0000","last_change":"2023-11-06T03:15:47.890055+0000","last_active":"2023-11-06T03:15:48.865297+0000","last_peered":"2023-11-06T03:15:48.865297+0000","last_clean":"2023-11-06T03:15:48.865297+0000","last_became_active":"2023-11-06T03:15:47.882281+0000","last_became_peered":"2023-11-06T03:15:47.882281+0000","last_unstale":"2023-11-06T03:15:48.865297+0000","last_undegraded":"2023-11-06T03:15:48.865297+0000","last_fullsized":"2023-11-06T03:15:48.865297+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"148'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860546+0000","last_change":"2023-11-06T03:15:47.495663+0000","last_active":"2023-11-06T03:15:48.860546+0000","last_peered":"2023-11-06T03:15:48.860546+0000","last_clean":"2023-11-06T03:15:48.860546+0000","last_became_active":"2023-11-06T03:15:47.495420+0000","last_became_peered":"2023-11-06T03:15:47.495420+0000","last_unstale":"2023-11-06T03:15:48.860546+0000","last_undegraded":"2023-11-06T03:15:48.860546+0000","last_fullsized":"2023-11-06T03:15:48.860546+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:31.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":122,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.473182+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:15:47.473182+0000","last_peered":"2023-11-06T03:15:47.473182+0000","last_clean":"2023-11-06T03:15:47.473182+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:15:47.473182+0000","last_undegraded":"2023-11-06T03:15:47.473182+0000","last_fullsized":"2023-11-06T03:15:47.473182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905522+0000","last_change":"2023-11-06T03:15:47.904915+0000","last_active":"2023-11-06T03:15:47.905522+0000","last_peered":"2023-11-06T03:15:47.905522+0000","last_clean":"2023-11-06T03:15:47.905522+0000","last_became_active":"2023-11-06T03:15:47.904404+0000","last_became_peered":"2023-11-06T03:15:47.904404+0000","last_unstale":"2023-11-06T03:15:47.905522+0000","last_undegraded":"2023-11-06T03:15:47.905522+0000","last_fullsized":"2023-11-06T03:15:47.905522+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:47.287086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886537+0000","last_change":"2023-11-06T03:15:47.528591+0000","last_active":"2023-11-06T03:15:47.886537+0000","last_peered":"2023-11-06T03:15:47.886537+0000","last_clean":"2023-11-06T03:15:47.886537+0000","last_became_active":"2023-11-06T03:15:47.527997+0000","last_became_peered":"2023-11-06T03:15:47.527997+0000","last_unstale":"2023-11-06T03:15:47.886537+0000","last_undegraded":"2023-11-06T03:15:47.886537+0000","last_fullsized":"2023-11-06T03:15:47.886537+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:05.594496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.489956+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:15:47.489956+0000","last_peered":"2023-11-06T03:15:47.489956+0000","last_clean":"2023-11-06T03:15:47.489956+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:15:47.489956+0000","last_undegraded":"2023-11-06T03:15:47.489956+0000","last_fullsized":"2023-11-06T03:15:47.489956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860659+0000","last_change":"2023-11-06T03:15:47.878228+0000","last_active":"2023-11-06T03:15:48.860659+0000","last_peered":"2023-11-06T03:15:48.860659+0000","last_clean":"2023-11-06T03:15:48.860659+0000","last_became_active":"2023-11-06T03:15:47.877147+0000","last_became_peered":"2023-11-06T03:15:47.877147+0000","last_unstale":"2023-11-06T03:15:48.860659+0000","last_undegraded":"2023-11-06T03:15:48.860659+0000","last_fullsized":"2023-11-06T03:15:48.860659+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:03:14.405112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.887095+0000","last_change":"2023-11-06T03:15:47.509631+0000","last_active":"2023-11-06T03:15:47.887095+0000","last_peered":"2023-11-06T03:15:47.887095+0000","last_clean":"2023-11-06T03:15:47.887095+0000","last_became_active":"2023-11-06T03:15:47.508958+0000","last_became_peered":"2023-11-06T03:15:47.508958+0000","last_unstale":"2023-11-06T03:15:47.887095+0000","last_undegraded":"2023-11-06T03:15:47.887095+0000","last_fullsized":"2023-11-06T03:15:47.887095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:48:01.412718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":122,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.473309+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:15:47.473309+0000","last_peered":"2023-11-06T03:15:47.473309+0000","last_clean":"2023-11-06T03:15:47.473309+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:15:47.473309+0000","last_undegraded":"2023-11-06T03:15:47.473309+0000","last_fullsized":"2023-11-06T03:15:47.473309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878348+0000","last_change":"2023-11-06T03:15:47.874637+0000","last_active":"2023-11-06T03:15:47.878348+0000","last_peered":"2023-11-06T03:15:47.878348+0000","last_clean":"2023-11-06T03:15:47.878348+0000","last_became_active":"2023-11-06T03:15:47.874338+0000","last_became_peered":"2023-11-06T03:15:47.874338+0000","last_unstale":"2023-11-06T03:15:47.878348+0000","last_undegraded":"2023-11-06T03:15:47.878348+0000","last_fullsized":"2023-11-06T03:15:47.878348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"148'8","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916020+0000","last_change":"2023-11-06T03:15:47.503853+0000","last_active":"2023-11-06T03:15:47.916020+0000","last_peered":"2023-11-06T03:15:47.916020+0000","last_clean":"2023-11-06T03:15:47.916020+0000","last_became_active":"2023-11-06T03:15:47.503218+0000","last_became_peered":"2023-11-06T03:15:47.503218+0000","last_unstale":"2023-11-06T03:15:47.916020+0000","last_undegraded":"2023-11-06T03:15:47.916020+0000","last_fullsized":"2023-11-06T03:15:47.916020+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.308233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":113,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.498828+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:15:47.498828+0000","last_peered":"2023-11-06T03:15:47.498828+0000","last_clean":"2023-11-06T03:15:47.498828+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:15:47.498828+0000","last_undegraded":"2023-11-06T03:15:47.498828+0000","last_fullsized":"2023-11-06T03:15:47.498828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91616,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280224,"statfs":{"total":96636764160,"available":96542949376,"internally_reserved":0,"allocated":2449408,"data_stored":1447712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.238,"15min":1.4830000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78900000000000003,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.157,"5min":1.202,"15min":1.397},"last":0.63800000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52500000000000002,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.167,"5min":1.627,"15min":1.627},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.875},"min":{"1min":0.56499999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1910000000000001,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86199999999999999,"15min":0.879},"min":{"1min":0.625,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.105,"5min":1.2170000000000001,"15min":1.6080000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.59799999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.147,"5min":1.272,"15min":1.6259999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.86899999999999999,"15min":0.89100000000000001},"min":{"1min":0.66900000000000004,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.159,"5min":1.4099999999999999,"15min":2.512},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.94999999999999996,"15min":1.038},"min":{"1min":0.67800000000000005,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.431,"5min":1.431,"15min":10.596},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.72799999999999998,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":1.54,"15min":1.9399999999999999},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0329999999999999,"15min":1.044},"min":{"1min":0.71499999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2889999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.071,"15min":1.0780000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.327,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.204,"15min":1.1659999999999999},"min":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":1.399,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.008},{"interface":"front","average":{"1min":1.101,"5min":1.0629999999999999,"15min":1.095},"min":{"1min":0.80800000000000005,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.329,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":0.93400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68888,"kb_used_data":1808,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302952,"statfs":{"total":96636764160,"available":96566222848,"internally_reserved":0,"allocated":1851392,"data_stored":856373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.85399999999999998},"min":{"1min":0.64900000000000002,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2310000000000001,"5min":2.3279999999999998,"15min":16.073},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.85299999999999998,"15min":0.97999999999999998},"min":{"1min":0.59099999999999997,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.2370000000000001,"5min":4.4000000000000004,"15min":40.905000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84599999999999997,"15min":0.90700000000000003},"min":{"1min":0.60699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.222,"5min":1.6359999999999999,"15min":16.012},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.85899999999999999,"15min":0.876},"min":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.151,"5min":1.409,"15min":3.3050000000000002},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.91600000000000004},"min":{"1min":0.65200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.107,"5min":1.744,"15min":27.803999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.94799999999999995},"min":{"1min":0.71999999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.429,"15min":2.843},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.93500000000000005,"15min":1.179},"min":{"1min":0.70499999999999996,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.3049999999999999,"5min":1.306,"15min":63.737000000000002},"last":0.75},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90900000000000003,"15min":1.0589999999999999},"min":{"1min":0.70299999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.173,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.71399999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.075,"15min":1.1619999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.512,"5min":7.4980000000000002,"15min":24.867999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.109,"5min":1.1559999999999999,"15min":1.228},"min":{"1min":0.91000000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2969999999999999,"5min":6.3970000000000002,"15min":23.789000000000001},"last":0.96899999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98199999999999998,"15min":1.1539999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.51800000000000002},"max":{"1min":1.694,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.103,"15min":1.1200000000000001},"min":{"1min":0.85599999999999998,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.02}]},{"osd":7,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.71699999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4690000000000001,"5min":4.6399999999999997,"15min":23.748999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1359999999999999,"15min":1.2250000000000001},"min":{"1min":0.92500000000000004,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.452,"5min":5.2709999999999999,"15min":28.164000000000001},"last":0.93999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1808,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1851392,"data_stored":857322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":1.804,"15min":1.1339999999999999},"min":{"1min":0.46300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.1679999999999999,"5min":125.256,"15min":125.256},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":2.0960000000000001,"15min":1.25},"min":{"1min":0.64900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1140000000000001,"5min":126.98699999999999,"15min":126.98699999999999},"last":1.089}]},{"osd":1,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.88300000000000001,"15min":0.80900000000000005},"min":{"1min":0.55200000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.034,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.292,"15min":0.996},"min":{"1min":0.54900000000000004,"5min":0.376,"15min":0.376},"max":{"1min":1.2030000000000001,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.347,"15min":1.012},"min":{"1min":0.63900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.363,"5min":43.741,"15min":43.741},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":0.84299999999999997},"min":{"1min":0.51200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0669999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0169999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92600000000000005,"15min":0.88300000000000001},"min":{"1min":0.61699999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.173,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.429,"15min":1.1040000000000001},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.296,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.1559999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.089,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.1020000000000001,"15min":1.0129999999999999},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.25,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.1799999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0549999999999999,"15min":1.0229999999999999},"min":{"1min":0.68899999999999995,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.647,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.0449999999999999},"min":{"1min":0.65500000000000003,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.2509999999999999,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1319999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.145,"15min":1.0589999999999999},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.3220000000000001,"5min":4.569,"15min":4.569},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.1479999999999999,"15min":1.109},"min":{"1min":0.73899999999999999,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.595,"5min":2.121,"15min":2.121},"last":1.2450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113200,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258640,"statfs":{"total":96636764160,"available":96520847360,"internally_reserved":0,"allocated":1875968,"data_stored":865539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":1.258,"15min":0.95699999999999996},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.125},{"interface":"front","average":{"1min":0.75600000000000001,"5min":1.2829999999999999,"15min":0.95399999999999996},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":1.103,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.83399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77600000000000002,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1539999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":1.159},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.83899999999999997},"min":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2390000000000001,"5min":1.639,"15min":1.6719999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89000000000000001,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.55700000000000005,"15min":0.53300000000000003},"max":{"1min":1.3200000000000001,"5min":1.452,"15min":3.2879999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.872,"15min":0.85999999999999999},"min":{"1min":0.497,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.103,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.081}]},{"osd":3,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89200000000000002,"15min":0.91800000000000004},"min":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.2150000000000001,"5min":3.198,"15min":3.198},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.59099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1919999999999999,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.98399999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.073,"15min":1.03},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2490000000000001,"5min":10.192,"15min":10.192},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0700000000000001,"15min":1.085},"min":{"1min":0.86399999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.292,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.2190000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98099999999999998,"15min":0.997},"min":{"1min":0.77700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.54,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.88},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.98799999999999999,"5min":0.65300000000000002,"15min":0.57799999999999996},"max":{"1min":1.3759999999999999,"5min":1.512,"15min":2.6549999999999998},"last":1.1890000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0900000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.383,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.107,"15min":1.1679999999999999},"min":{"1min":0.879,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.49,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82908,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288932,"statfs":{"total":96636764160,"available":96551866368,"internally_reserved":0,"allocated":1855488,"data_stored":856919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.58899999999999997},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.94399999999999995,"5min":1.113,"15min":1.4199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.524},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.36399999999999999,"5min":0.251,"15min":0.224},"max":{"1min":1.024,"5min":1.1839999999999999,"15min":1.371},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":1.421,"5min":1.532,"15min":1.532},"last":0.496},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40799999999999997,"5min":0.32000000000000001,"15min":0.255},"max":{"1min":1.052,"5min":1.206,"15min":1.284},"last":0.433}]},{"osd":4,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.86899999999999999},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2529999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.89500000000000002,"15min":0.91700000000000004},"min":{"1min":0.58599999999999997,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.52,"15min":1.8300000000000001},"last":0.628}]},{"osd":5,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.302,"5min":1.5,"15min":1.6639999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.95999999999999996,"15min":0.98499999999999999},"min":{"1min":0.69799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.2050000000000001,"5min":1.573,"15min":1.698},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.95699999999999996,"15min":0.97899999999999998},"min":{"1min":0.64800000000000002,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.387,"5min":1.51,"15min":1.8620000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.96099999999999997},"min":{"1min":0.62,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.1140000000000001,"5min":1.748,"15min":1.8740000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.97399999999999998,"15min":1.01},"min":{"1min":0.65800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.24,"5min":1.835,"15min":1.835},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0089999999999999,"15min":1.0329999999999999},"min":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.569,"15min":1.8979999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72992,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298848,"statfs":{"total":96636764160,"available":96562020352,"internally_reserved":0,"allocated":1859584,"data_stored":857434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.51200000000000001,"15min":0.53300000000000003},"min":{"1min":0.253,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.69799999999999995,"5min":0.93400000000000005,"15min":1.7270000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.58999999999999997},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":0.79300000000000004,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.61299999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.86899999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57599999999999996,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.22800000000000001},"max":{"1min":0.82399999999999995,"5min":1.0509999999999999,"15min":1.1579999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.33900000000000002,"5min":0.307,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.161,"15min":1.294},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.625,"15min":0.68899999999999995},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.95299999999999996,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79900000000000004,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.097,"5min":1.244,"15min":1.526},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83099999999999996,"15min":0.89300000000000002},"min":{"1min":0.53300000000000003,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":1.0860000000000001,"5min":1.329,"15min":1.5229999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82099999999999995,"15min":0.85199999999999998},"min":{"1min":0.56200000000000006,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":1.012,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.91500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.4359999999999999,"15min":1.5780000000000001},"last":1.026}]},{"osd":6,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.194,"5min":1.542,"15min":1.615},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90700000000000003,"15min":0.93000000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.5},"max":{"1min":1.1839999999999999,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89400000000000002,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2050000000000001,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":1.071},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.92500000000000004},"min":{"1min":0.54200000000000004,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.1120000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1855488,"data_stored":857392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.19},"max":{"1min":1.042,"5min":1.1240000000000001,"15min":1.843},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.244,"5min":1.2470000000000001,"15min":2.1150000000000002},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.69199999999999995},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":30.861000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.64100000000000001},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.98399999999999999,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66200000000000003,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0629999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.68300000000000005},"min":{"1min":0.34300000000000003,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81799999999999995,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1339999999999999,"5min":1.276,"15min":2.258},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.83699999999999997,"15min":0.92100000000000004},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.262,"5min":1.3600000000000001,"15min":14.753},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.872,"15min":1.081},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.351,"5min":1.528,"15min":43.103999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94599999999999995,"15min":1.115},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.39100000000000001},"max":{"1min":1.333,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":1.026},"min":{"1min":0.59499999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.3240000000000001,"5min":1.5189999999999999,"15min":23.971},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.90400000000000003,"15min":1.04},"min":{"1min":0.56399999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2949999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.377,"5min":1.581,"15min":2.2799999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95999999999999996,"15min":1.0089999999999999},"min":{"1min":0.58099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.835,"5min":1.835,"15min":2.3559999999999999},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95792,"kb_used_data":2408,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94276048,"statfs":{"total":96636764160,"available":96538673152,"internally_reserved":0,"allocated":2465792,"data_stored":1448941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.23899999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.84599999999999997,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.246,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.90200000000000002,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.72799999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22800000000000001,"15min":0.20599999999999999},"max":{"1min":0.98899999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.014},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.97499999999999998,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.91200000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.87,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67300000000000004,"15min":0.71199999999999997},"min":{"1min":0.32500000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1000000000000001,"5min":1.391,"15min":2.7999999999999998},"last":1.157}]},{"osd":4,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.83099999999999996,"15min":0.90000000000000002},"min":{"1min":0.52000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1619999999999999,"5min":1.371,"15min":4.641},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82299999999999995,"15min":0.88100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":2.028},"last":0.878}]},{"osd":5,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88900000000000001,"15min":0.94899999999999995},"min":{"1min":0.57399999999999995,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.379,"5min":1.4890000000000001,"15min":2.8559999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.879,"15min":0.93300000000000005},"min":{"1min":0.59699999999999998,"5min":0.44,"15min":0.432},"max":{"1min":1.496,"5min":1.496,"15min":4.7430000000000003},"last":1.1259999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.89500000000000002},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":2.931},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.98699999999999999},"min":{"1min":0.56599999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.98099999999999998},"min":{"1min":0.61599999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.3740000000000001,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.96799999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2330000000000001,"5min":1.4670000000000001,"15min":1.97},"last":1.0369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:15:50.866 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:15:51.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.027+0000 7f936cd02640 1 Processor -- start 2023-11-06T03:15:51.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.027+0000 7f936cd02640 1 -- start start 2023-11-06T03:15:51.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93680adae0 0x7f93680b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f93680adf40 0x7f93680b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936806c490 con 0x7f9368156a70 2023-11-06T03:15:51.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936806b950 con 0x7f93680adae0 2023-11-06T03:15:51.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936806fb80 con 0x7f93680adf40 2023-11-06T03:15:51.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93680adae0 0x7f93680b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9368156a70 0x7f936806f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44732/0 (socket says 172.21.15.157:44732) 2023-11-06T03:15:51.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936659c640 1 -- 172.21.15.157:0/1047953560 learned_addr learned my addr 172.21.15.157:0/1047953560 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:51.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9366d9d640 1 --2- 172.21.15.157:0/1047953560 >> v2:172.21.15.195:3300/0 conn(0x7f93680adf40 0x7f93680b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936559a640 1 -- 172.21.15.157:0/1047953560 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104935417 0 0) 0x7f936806c490 con 0x7f9368156a70 2023-11-06T03:15:51.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936559a640 1 -- 172.21.15.157:0/1047953560 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9354003530 con 0x7f9368156a70 2023-11-06T03:15:51.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 -- 172.21.15.157:0/1047953560 >> v2:172.21.15.195:3300/0 conn(0x7f93680adf40 msgr2=0x7f93680b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:51.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 --2- 172.21.15.157:0/1047953560 >> v2:172.21.15.195:3300/0 conn(0x7f93680adf40 0x7f93680b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 -- 172.21.15.157:0/1047953560 >> v1:172.21.15.157:6789/0 conn(0x7f9368156a70 legacy=0x7f936806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 -- 172.21.15.157:0/1047953560 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936806bb60 con 0x7f93680adae0 2023-11-06T03:15:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f9365d9b640 1 --2- 172.21.15.157:0/1047953560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93680b1c20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f93680ce830 tx=0x7f935001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c30347090d97bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936559a640 1 -- 172.21.15.157:0/1047953560 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500ac410 con 0x7f93680adae0 2023-11-06T03:15:51.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936559a640 1 -- 172.21.15.157:0/1047953560 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500ac570 con 0x7f93680adae0 2023-11-06T03:15:51.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936559a640 1 -- 172.21.15.157:0/1047953560 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500ac8e0 con 0x7f93680adae0 2023-11-06T03:15:51.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- 172.21.15.157:0/1047953560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 msgr2=0x7f93680b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 --2- 172.21.15.157:0/1047953560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93680b1c20 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f93680ce830 tx=0x7f935001c430 comp rx=0 tx=0).stop 2023-11-06T03:15:51.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- 172.21.15.157:0/1047953560 shutdown_connections 2023-11-06T03:15:51.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 --2- 172.21.15.157:0/1047953560 >> v2:172.21.15.195:3300/0 conn(0x7f93680adf40 0x7f93680b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 --2- 172.21.15.157:0/1047953560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93680b1c20 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- 172.21.15.157:0/1047953560 >> 172.21.15.157:0/1047953560 conn(0x7f9368065930 msgr2=0x7f9368065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- 172.21.15.157:0/1047953560 shutdown_connections 2023-11-06T03:15:51.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.031+0000 7f936cd02640 1 -- 172.21.15.157:0/1047953560 wait complete. 2023-11-06T03:15:51.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 Processor -- start 2023-11-06T03:15:51.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- start start 2023-11-06T03:15:51.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93680adf40 0x7f9368158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9368156a70 0x7f9368149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93681f8b50 con 0x7f93680adae0 2023-11-06T03:15:51.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93681465f0 con 0x7f9368156a70 2023-11-06T03:15:51.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93681573b0 con 0x7f93680adf40 2023-11-06T03:15:51.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37296/0 (socket says 172.21.15.157:37296) 2023-11-06T03:15:51.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 -- 172.21.15.157:0/1799637349 learned_addr learned my addr 172.21.15.157:0/1799637349 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:51.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f9366d9d640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9368156a70 0x7f9368149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 -- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93680adf40 msgr2=0x7f9368158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f9365d9b640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93680adf40 0x7f9368158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93680adf40 0x7f9368158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 -- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9368156a70 msgr2=0x7f9368149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9368156a70 0x7f9368149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 -- 172.21.15.157:0/1799637349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009420 con 0x7f93680adae0 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936659c640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f936000caa0 tx=0x7f936000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000ac00 con 0x7f93680adae0 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000ad60 con 0x7f93680adae0 2023-11-06T03:15:51.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9368157690 con 0x7f93680adae0 2023-11-06T03:15:51.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360012620 con 0x7f93680adae0 2023-11-06T03:15:51.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9368157c50 con 0x7f93680adae0 2023-11-06T03:15:51.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.035+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368072750 con 0x7f93680adae0 2023-11-06T03:15:51.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.039+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9360007070 con 0x7f93680adae0 2023-11-06T03:15:51.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.039+0000 7f935effd640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 0x7f9348067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.039+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9360083260 con 0x7f93680adae0 2023-11-06T03:15:51.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.039+0000 7f9365d9b640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 0x7f9348067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.039+0000 7f9365d9b640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 0x7f9348067f20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f93500ae000 tx=0x7f93500a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.043+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9360050af0 con 0x7f93680adae0 2023-11-06T03:15:51.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.191+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f936815aed0 con 0x7f9348065a60 2023-11-06T03:15:51.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.215+0000 7f935effd640 1 -- 172.21.15.157:0/1799637349 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310800 (secure 0 0 0) 0x7f936815aed0 con 0x7f9348065a60 2023-11-06T03:15:51.217 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:51.219 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:15:51.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 msgr2=0x7f9348067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 0x7f9348067f20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f93500ae000 tx=0x7f93500a6040 comp rx=0 tx=0).stop 2023-11-06T03:15:51.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 msgr2=0x7f93681f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f936000caa0 tx=0x7f936000cf60 comp rx=0 tx=0).stop 2023-11-06T03:15:51.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 shutdown_connections 2023-11-06T03:15:51.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9348065a60 0x7f9348067f20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9368156a70 0x7f9368149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f93680adf40 0x7f9368158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 --2- 172.21.15.157:0/1799637349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93680adae0 0x7f93681f8610 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 >> 172.21.15.157:0/1799637349 conn(0x7f9368065930 msgr2=0x7f936815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 shutdown_connections 2023-11-06T03:15:51.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.223+0000 7f936cd02640 1 -- 172.21.15.157:0/1799637349 wait complete. 2023-11-06T03:15:51.237 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2023-11-06T03:15:50.874003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":641096,"kb_used_data":15688,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754333624,"statfs":{"total":773094113280,"available":772437630976,"internally_reserved":0,"allocated":16064512,"data_stored":8047632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.524043"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921664+0000","last_change":"2023-11-06T03:15:47.914528+0000","last_active":"2023-11-06T03:15:47.921664+0000","last_peered":"2023-11-06T03:15:47.921664+0000","last_clean":"2023-11-06T03:15:47.921664+0000","last_became_active":"2023-11-06T03:15:47.913761+0000","last_became_peered":"2023-11-06T03:15:47.913761+0000","last_unstale":"2023-11-06T03:15:47.921664+0000","last_undegraded":"2023-11-06T03:15:47.921664+0000","last_fullsized":"2023-11-06T03:15:47.921664+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:04.522411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.917266+0000","last_change":"2023-11-06T03:15:47.914268+0000","last_active":"2023-11-06T03:15:47.917266+0000","last_peered":"2023-11-06T03:15:47.917266+0000","last_clean":"2023-11-06T03:15:47.917266+0000","last_became_active":"2023-11-06T03:15:47.913945+0000","last_became_peered":"2023-11-06T03:15:47.913945+0000","last_unstale":"2023-11-06T03:15:47.917266+0000","last_undegraded":"2023-11-06T03:15:47.917266+0000","last_fullsized":"2023-11-06T03:15:47.917266+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:00:02.426149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.882888+0000","last_change":"2023-11-06T03:15:47.882337+0000","last_active":"2023-11-06T03:15:47.882888+0000","last_peered":"2023-11-06T03:15:47.882888+0000","last_clean":"2023-11-06T03:15:47.882888+0000","last_became_active":"2023-11-06T03:15:47.882047+0000","last_became_peered":"2023-11-06T03:15:47.882047+0000","last_unstale":"2023-11-06T03:15:47.882888+0000","last_undegraded":"2023-11-06T03:15:47.882888+0000","last_fullsized":"2023-11-06T03:15:47.882888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922311+0000","last_change":"2023-11-06T03:15:47.901861+0000","last_active":"2023-11-06T03:15:47.922311+0000","last_peered":"2023-11-06T03:15:47.922311+0000","last_clean":"2023-11-06T03:15:47.922311+0000","last_became_active":"2023-11-06T03:15:47.901260+0000","last_became_peered":"2023-11-06T03:15:47.901260+0000","last_unstale":"2023-11-06T03:15:47.922311+0000","last_undegraded":"2023-11-06T03:15:47.922311+0000","last_fullsized":"2023-11-06T03:15:47.922311+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:40:50.438603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860496+0000","last_change":"2023-11-06T03:15:47.915338+0000","last_active":"2023-11-06T03:15:48.860496+0000","last_peered":"2023-11-06T03:15:48.860496+0000","last_clean":"2023-11-06T03:15:48.860496+0000","last_became_active":"2023-11-06T03:15:47.915077+0000","last_became_peered":"2023-11-06T03:15:47.915077+0000","last_unstale":"2023-11-06T03:15:48.860496+0000","last_undegraded":"2023-11-06T03:15:48.860496+0000","last_fullsized":"2023-11-06T03:15:48.860496+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:06.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860644+0000","last_change":"2023-11-06T03:15:47.915812+0000","last_active":"2023-11-06T03:15:48.860644+0000","last_peered":"2023-11-06T03:15:48.860644+0000","last_clean":"2023-11-06T03:15:48.860644+0000","last_became_active":"2023-11-06T03:15:47.915553+0000","last_became_peered":"2023-11-06T03:15:47.915553+0000","last_unstale":"2023-11-06T03:15:48.860644+0000","last_undegraded":"2023-11-06T03:15:48.860644+0000","last_fullsized":"2023-11-06T03:15:48.860644+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:43:13.205842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.923812+0000","last_change":"2023-11-06T03:15:47.916792+0000","last_active":"2023-11-06T03:15:47.923812+0000","last_peered":"2023-11-06T03:15:47.923812+0000","last_clean":"2023-11-06T03:15:47.923812+0000","last_became_active":"2023-11-06T03:15:47.916287+0000","last_became_peered":"2023-11-06T03:15:47.916287+0000","last_unstale":"2023-11-06T03:15:47.923812+0000","last_undegraded":"2023-11-06T03:15:47.923812+0000","last_fullsized":"2023-11-06T03:15:47.923812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:53.927074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.864921+0000","last_change":"2023-11-06T03:15:47.869483+0000","last_active":"2023-11-06T03:15:48.864921+0000","last_peered":"2023-11-06T03:15:48.864921+0000","last_clean":"2023-11-06T03:15:48.864921+0000","last_became_active":"2023-11-06T03:15:47.868593+0000","last_became_peered":"2023-11-06T03:15:47.868593+0000","last_unstale":"2023-11-06T03:15:48.864921+0000","last_undegraded":"2023-11-06T03:15:48.864921+0000","last_fullsized":"2023-11-06T03:15:48.864921+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860462+0000","last_change":"2023-11-06T03:15:47.877455+0000","last_active":"2023-11-06T03:15:48.860462+0000","last_peered":"2023-11-06T03:15:48.860462+0000","last_clean":"2023-11-06T03:15:48.860462+0000","last_became_active":"2023-11-06T03:15:47.877182+0000","last_became_peered":"2023-11-06T03:15:47.877182+0000","last_unstale":"2023-11-06T03:15:48.860462+0000","last_undegraded":"2023-11-06T03:15:48.860462+0000","last_fullsized":"2023-11-06T03:15:48.860462+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:27:08.027457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860728+0000","last_change":"2023-11-06T03:15:47.870971+0000","last_active":"2023-11-06T03:15:48.860728+0000","last_peered":"2023-11-06T03:15:48.860728+0000","last_clean":"2023-11-06T03:15:48.860728+0000","last_became_active":"2023-11-06T03:15:47.870692+0000","last_became_peered":"2023-11-06T03:15:47.870692+0000","last_unstale":"2023-11-06T03:15:48.860728+0000","last_undegraded":"2023-11-06T03:15:48.860728+0000","last_fullsized":"2023-11-06T03:15:48.860728+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:27.345172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876394+0000","last_change":"2023-11-06T03:15:47.873108+0000","last_active":"2023-11-06T03:15:47.876394+0000","last_peered":"2023-11-06T03:15:47.876394+0000","last_clean":"2023-11-06T03:15:47.876394+0000","last_became_active":"2023-11-06T03:15:47.872837+0000","last_became_peered":"2023-11-06T03:15:47.872837+0000","last_unstale":"2023-11-06T03:15:47.876394+0000","last_undegraded":"2023-11-06T03:15:47.876394+0000","last_fullsized":"2023-11-06T03:15:47.876394+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:55.672805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924465+0000","last_change":"2023-11-06T03:15:47.916323+0000","last_active":"2023-11-06T03:15:47.924465+0000","last_peered":"2023-11-06T03:15:47.924465+0000","last_clean":"2023-11-06T03:15:47.924465+0000","last_became_active":"2023-11-06T03:15:47.915726+0000","last_became_peered":"2023-11-06T03:15:47.915726+0000","last_unstale":"2023-11-06T03:15:47.924465+0000","last_undegraded":"2023-11-06T03:15:47.924465+0000","last_fullsized":"2023-11-06T03:15:47.924465+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:19.311071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860397+0000","last_change":"2023-11-06T03:15:47.874540+0000","last_active":"2023-11-06T03:15:48.860397+0000","last_peered":"2023-11-06T03:15:48.860397+0000","last_clean":"2023-11-06T03:15:48.860397+0000","last_became_active":"2023-11-06T03:15:47.874280+0000","last_became_peered":"2023-11-06T03:15:47.874280+0000","last_unstale":"2023-11-06T03:15:48.860397+0000","last_undegraded":"2023-11-06T03:15:48.860397+0000","last_fullsized":"2023-11-06T03:15:48.860397+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:38.919598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922329+0000","last_change":"2023-11-06T03:15:47.902264+0000","last_active":"2023-11-06T03:15:47.922329+0000","last_peered":"2023-11-06T03:15:47.922329+0000","last_clean":"2023-11-06T03:15:47.922329+0000","last_became_active":"2023-11-06T03:15:47.901381+0000","last_became_peered":"2023-11-06T03:15:47.901381+0000","last_unstale":"2023-11-06T03:15:47.922329+0000","last_undegraded":"2023-11-06T03:15:47.922329+0000","last_fullsized":"2023-11-06T03:15:47.922329+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:12:22.532910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905697+0000","last_change":"2023-11-06T03:15:47.904996+0000","last_active":"2023-11-06T03:15:47.905697+0000","last_peered":"2023-11-06T03:15:47.905697+0000","last_clean":"2023-11-06T03:15:47.905697+0000","last_became_active":"2023-11-06T03:15:47.904440+0000","last_became_peered":"2023-11-06T03:15:47.904440+0000","last_unstale":"2023-11-06T03:15:47.905697+0000","last_undegraded":"2023-11-06T03:15:47.905697+0000","last_fullsized":"2023-11-06T03:15:47.905697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:03:14.405112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860090+0000","last_change":"2023-11-06T03:15:47.890692+0000","last_active":"2023-11-06T03:15:48.860090+0000","last_peered":"2023-11-06T03:15:48.860090+0000","last_clean":"2023-11-06T03:15:48.860090+0000","last_became_active":"2023-11-06T03:15:47.890249+0000","last_became_peered":"2023-11-06T03:15:47.890249+0000","last_unstale":"2023-11-06T03:15:48.860090+0000","last_undegraded":"2023-11-06T03:15:48.860090+0000","last_fullsized":"2023-11-06T03:15:48.860090+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:59.046349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922251+0000","last_change":"2023-11-06T03:15:47.902039+0000","last_active":"2023-11-06T03:15:47.922251+0000","last_peered":"2023-11-06T03:15:47.922251+0000","last_clean":"2023-11-06T03:15:47.922251+0000","last_became_active":"2023-11-06T03:15:47.901310+0000","last_became_peered":"2023-11-06T03:15:47.901310+0000","last_unstale":"2023-11-06T03:15:47.922251+0000","last_undegraded":"2023-11-06T03:15:47.922251+0000","last_fullsized":"2023-11-06T03:15:47.922251+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:10.208146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916637+0000","last_change":"2023-11-06T03:15:47.915102+0000","last_active":"2023-11-06T03:15:47.916637+0000","last_peered":"2023-11-06T03:15:47.916637+0000","last_clean":"2023-11-06T03:15:47.916637+0000","last_became_active":"2023-11-06T03:15:47.914327+0000","last_became_peered":"2023-11-06T03:15:47.914327+0000","last_unstale":"2023-11-06T03:15:47.916637+0000","last_undegraded":"2023-11-06T03:15:47.916637+0000","last_fullsized":"2023-11-06T03:15:47.916637+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:30:48.976495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.912271+0000","last_change":"2023-11-06T03:15:47.911358+0000","last_active":"2023-11-06T03:15:47.912271+0000","last_peered":"2023-11-06T03:15:47.912271+0000","last_clean":"2023-11-06T03:15:47.912271+0000","last_became_active":"2023-11-06T03:15:47.910674+0000","last_became_peered":"2023-11-06T03:15:47.910674+0000","last_unstale":"2023-11-06T03:15:47.912271+0000","last_undegraded":"2023-11-06T03:15:47.912271+0000","last_fullsized":"2023-11-06T03:15:47.912271+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877608+0000","last_change":"2023-11-06T03:15:47.876549+0000","last_active":"2023-11-06T03:15:47.877608+0000","last_peered":"2023-11-06T03:15:47.877608+0000","last_clean":"2023-11-06T03:15:47.877608+0000","last_became_active":"2023-11-06T03:15:47.876271+0000","last_became_peered":"2023-11-06T03:15:47.876271+0000","last_unstale":"2023-11-06T03:15:47.877608+0000","last_undegraded":"2023-11-06T03:15:47.877608+0000","last_fullsized":"2023-11-06T03:15:47.877608+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:33:44.484721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860452+0000","last_change":"2023-11-06T03:15:47.873052+0000","last_active":"2023-11-06T03:15:48.860452+0000","last_peered":"2023-11-06T03:15:48.860452+0000","last_clean":"2023-11-06T03:15:48.860452+0000","last_became_active":"2023-11-06T03:15:47.872531+0000","last_became_peered":"2023-11-06T03:15:47.872531+0000","last_unstale":"2023-11-06T03:15:48.860452+0000","last_undegraded":"2023-11-06T03:15:48.860452+0000","last_fullsized":"2023-11-06T03:15:48.860452+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:17:53.838120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921426+0000","last_change":"2023-11-06T03:15:47.913987+0000","last_active":"2023-11-06T03:15:47.921426+0000","last_peered":"2023-11-06T03:15:47.921426+0000","last_clean":"2023-11-06T03:15:47.921426+0000","last_became_active":"2023-11-06T03:15:47.913553+0000","last_became_peered":"2023-11-06T03:15:47.913553+0000","last_unstale":"2023-11-06T03:15:47.921426+0000","last_undegraded":"2023-11-06T03:15:47.921426+0000","last_fullsized":"2023-11-06T03:15:47.921426+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:03:08.155507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.864876+0000","last_change":"2023-11-06T03:15:47.919565+0000","last_active":"2023-11-06T03:15:48.864876+0000","last_peered":"2023-11-06T03:15:48.864876+0000","last_clean":"2023-11-06T03:15:48.864876+0000","last_became_active":"2023-11-06T03:15:47.918943+0000","last_became_peered":"2023-11-06T03:15:47.918943+0000","last_unstale":"2023-11-06T03:15:48.864876+0000","last_undegraded":"2023-11-06T03:15:48.864876+0000","last_fullsized":"2023-11-06T03:15:48.864876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:25:50.652489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876113+0000","last_change":"2023-11-06T03:15:47.871715+0000","last_active":"2023-11-06T03:15:47.876113+0000","last_peered":"2023-11-06T03:15:47.876113+0000","last_clean":"2023-11-06T03:15:47.876113+0000","last_became_active":"2023-11-06T03:15:47.871221+0000","last_became_peered":"2023-11-06T03:15:47.871221+0000","last_unstale":"2023-11-06T03:15:47.876113+0000","last_undegraded":"2023-11-06T03:15:47.876113+0000","last_fullsized":"2023-11-06T03:15:47.876113+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:08:51.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.923788+0000","last_change":"2023-11-06T03:15:47.915785+0000","last_active":"2023-11-06T03:15:47.923788+0000","last_peered":"2023-11-06T03:15:47.923788+0000","last_clean":"2023-11-06T03:15:47.923788+0000","last_became_active":"2023-11-06T03:15:47.914490+0000","last_became_peered":"2023-11-06T03:15:47.914490+0000","last_unstale":"2023-11-06T03:15:47.923788+0000","last_undegraded":"2023-11-06T03:15:47.923788+0000","last_fullsized":"2023-11-06T03:15:47.923788+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:54.247168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921593+0000","last_change":"2023-11-06T03:15:47.914864+0000","last_active":"2023-11-06T03:15:47.921593+0000","last_peered":"2023-11-06T03:15:47.921593+0000","last_clean":"2023-11-06T03:15:47.921593+0000","last_became_active":"2023-11-06T03:15:47.914170+0000","last_became_peered":"2023-11-06T03:15:47.914170+0000","last_unstale":"2023-11-06T03:15:47.921593+0000","last_undegraded":"2023-11-06T03:15:47.921593+0000","last_fullsized":"2023-11-06T03:15:47.921593+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:25:50.652489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860793+0000","last_change":"2023-11-06T03:15:47.877611+0000","last_active":"2023-11-06T03:15:48.860793+0000","last_peered":"2023-11-06T03:15:48.860793+0000","last_clean":"2023-11-06T03:15:48.860793+0000","last_became_active":"2023-11-06T03:15:47.877291+0000","last_became_peered":"2023-11-06T03:15:47.877291+0000","last_unstale":"2023-11-06T03:15:48.860793+0000","last_undegraded":"2023-11-06T03:15:48.860793+0000","last_fullsized":"2023-11-06T03:15:48.860793+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:57.616562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877493+0000","last_change":"2023-11-06T03:15:47.865193+0000","last_active":"2023-11-06T03:15:47.877493+0000","last_peered":"2023-11-06T03:15:47.877493+0000","last_clean":"2023-11-06T03:15:47.877493+0000","last_became_active":"2023-11-06T03:15:47.864926+0000","last_became_peered":"2023-11-06T03:15:47.864926+0000","last_unstale":"2023-11-06T03:15:47.877493+0000","last_undegraded":"2023-11-06T03:15:47.877493+0000","last_fullsized":"2023-11-06T03:15:47.877493+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:09:22.812985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866514+0000","last_change":"2023-11-06T03:15:47.881506+0000","last_active":"2023-11-06T03:15:48.866514+0000","last_peered":"2023-11-06T03:15:48.866514+0000","last_clean":"2023-11-06T03:15:48.866514+0000","last_became_active":"2023-11-06T03:15:47.879999+0000","last_became_peered":"2023-11-06T03:15:47.879999+0000","last_unstale":"2023-11-06T03:15:48.866514+0000","last_undegraded":"2023-11-06T03:15:48.866514+0000","last_fullsized":"2023-11-06T03:15:48.866514+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.902180+0000","last_change":"2023-11-06T03:15:47.901364+0000","last_active":"2023-11-06T03:15:47.902180+0000","last_peered":"2023-11-06T03:15:47.902180+0000","last_clean":"2023-11-06T03:15:47.902180+0000","last_became_active":"2023-11-06T03:15:47.901067+0000","last_became_peered":"2023-11-06T03:15:47.901067+0000","last_unstale":"2023-11-06T03:15:47.902180+0000","last_undegraded":"2023-11-06T03:15:47.902180+0000","last_fullsized":"2023-11-06T03:15:47.902180+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:47:47.771294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860178+0000","last_change":"2023-11-06T03:15:47.880028+0000","last_active":"2023-11-06T03:15:48.860178+0000","last_peered":"2023-11-06T03:15:48.860178+0000","last_clean":"2023-11-06T03:15:48.860178+0000","last_became_active":"2023-11-06T03:15:47.879545+0000","last_became_peered":"2023-11-06T03:15:47.879545+0000","last_unstale":"2023-11-06T03:15:48.860178+0000","last_undegraded":"2023-11-06T03:15:48.860178+0000","last_fullsized":"2023-11-06T03:15:48.860178+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:38:45.703881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865168+0000","last_change":"2023-11-06T03:15:47.869086+0000","last_active":"2023-11-06T03:15:48.865168+0000","last_peered":"2023-11-06T03:15:48.865168+0000","last_clean":"2023-11-06T03:15:48.865168+0000","last_became_active":"2023-11-06T03:15:47.868388+0000","last_became_peered":"2023-11-06T03:15:47.868388+0000","last_unstale":"2023-11-06T03:15:48.865168+0000","last_undegraded":"2023-11-06T03:15:48.865168+0000","last_fullsized":"2023-11-06T03:15:48.865168+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:04.522411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.883197+0000","last_change":"2023-11-06T03:15:47.882716+0000","last_active":"2023-11-06T03:15:47.883197+0000","last_peered":"2023-11-06T03:15:47.883197+0000","last_clean":"2023-11-06T03:15:47.883197+0000","last_became_active":"2023-11-06T03:15:47.882295+0000","last_became_peered":"2023-11-06T03:15:47.882295+0000","last_unstale":"2023-11-06T03:15:47.883197+0000","last_undegraded":"2023-11-06T03:15:47.883197+0000","last_fullsized":"2023-11-06T03:15:47.883197+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"148'4","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860650+0000","last_change":"2023-11-06T03:15:47.495275+0000","last_active":"2023-11-06T03:15:48.860650+0000","last_peered":"2023-11-06T03:15:48.860650+0000","last_clean":"2023-11-06T03:15:48.860650+0000","last_became_active":"2023-11-06T03:15:47.494945+0000","last_became_peered":"2023-11-06T03:15:47.494945+0000","last_unstale":"2023-11-06T03:15:48.860650+0000","last_undegraded":"2023-11-06T03:15:48.860650+0000","last_fullsized":"2023-11-06T03:15:48.860650+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:45.807010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916629+0000","last_change":"2023-11-06T03:15:47.914148+0000","last_active":"2023-11-06T03:15:47.916629+0000","last_peered":"2023-11-06T03:15:47.916629+0000","last_clean":"2023-11-06T03:15:47.916629+0000","last_became_active":"2023-11-06T03:15:47.913871+0000","last_became_peered":"2023-11-06T03:15:47.913871+0000","last_unstale":"2023-11-06T03:15:47.916629+0000","last_undegraded":"2023-11-06T03:15:47.916629+0000","last_fullsized":"2023-11-06T03:15:47.916629+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:55.317634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.899403+0000","last_change":"2023-11-06T03:15:47.512311+0000","last_active":"2023-11-06T03:15:47.899403+0000","last_peered":"2023-11-06T03:15:47.899403+0000","last_clean":"2023-11-06T03:15:47.899403+0000","last_became_active":"2023-11-06T03:15:47.511689+0000","last_became_peered":"2023-11-06T03:15:47.511689+0000","last_unstale":"2023-11-06T03:15:47.899403+0000","last_undegraded":"2023-11-06T03:15:47.899403+0000","last_fullsized":"2023-11-06T03:15:47.899403+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:35:53.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866182+0000","last_change":"2023-11-06T03:15:47.919361+0000","last_active":"2023-11-06T03:15:48.866182+0000","last_peered":"2023-11-06T03:15:48.866182+0000","last_clean":"2023-11-06T03:15:48.866182+0000","last_became_active":"2023-11-06T03:15:47.918905+0000","last_became_peered":"2023-11-06T03:15:47.918905+0000","last_unstale":"2023-11-06T03:15:48.866182+0000","last_undegraded":"2023-11-06T03:15:48.866182+0000","last_fullsized":"2023-11-06T03:15:48.866182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:53.927074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.899881+0000","last_change":"2023-11-06T03:15:47.510786+0000","last_active":"2023-11-06T03:15:47.899881+0000","last_peered":"2023-11-06T03:15:47.899881+0000","last_clean":"2023-11-06T03:15:47.899881+0000","last_became_active":"2023-11-06T03:15:47.510214+0000","last_became_peered":"2023-11-06T03:15:47.510214+0000","last_unstale":"2023-11-06T03:15:47.899881+0000","last_undegraded":"2023-11-06T03:15:47.899881+0000","last_fullsized":"2023-11-06T03:15:47.899881+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:53:34.697001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.919680+0000","last_change":"2023-11-06T03:15:47.917509+0000","last_active":"2023-11-06T03:15:47.919680+0000","last_peered":"2023-11-06T03:15:47.919680+0000","last_clean":"2023-11-06T03:15:47.919680+0000","last_became_active":"2023-11-06T03:15:47.916912+0000","last_became_peered":"2023-11-06T03:15:47.916912+0000","last_unstale":"2023-11-06T03:15:47.919680+0000","last_undegraded":"2023-11-06T03:15:47.919680+0000","last_fullsized":"2023-11-06T03:15:47.919680+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.908101+0000","last_change":"2023-11-06T03:15:47.525307+0000","last_active":"2023-11-06T03:15:47.908101+0000","last_peered":"2023-11-06T03:15:47.908101+0000","last_clean":"2023-11-06T03:15:47.908101+0000","last_became_active":"2023-11-06T03:15:47.524915+0000","last_became_peered":"2023-11-06T03:15:47.524915+0000","last_unstale":"2023-11-06T03:15:47.908101+0000","last_undegraded":"2023-11-06T03:15:47.908101+0000","last_fullsized":"2023-11-06T03:15:47.908101+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:48.117488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.869959+0000","last_change":"2023-11-06T03:15:47.867593+0000","last_active":"2023-11-06T03:15:47.869959+0000","last_peered":"2023-11-06T03:15:47.869959+0000","last_clean":"2023-11-06T03:15:47.869959+0000","last_became_active":"2023-11-06T03:15:47.867181+0000","last_became_peered":"2023-11-06T03:15:47.867181+0000","last_unstale":"2023-11-06T03:15:47.869959+0000","last_undegraded":"2023-11-06T03:15:47.869959+0000","last_fullsized":"2023-11-06T03:15:47.869959+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:01:51.382858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"148'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865915+0000","last_change":"2023-11-06T03:15:47.518422+0000","last_active":"2023-11-06T03:15:48.865915+0000","last_peered":"2023-11-06T03:15:48.865915+0000","last_clean":"2023-11-06T03:15:48.865915+0000","last_became_active":"2023-11-06T03:15:47.516127+0000","last_became_peered":"2023-11-06T03:15:47.516127+0000","last_unstale":"2023-11-06T03:15:48.865915+0000","last_undegraded":"2023-11-06T03:15:48.865915+0000","last_fullsized":"2023-11-06T03:15:48.865915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:34:29.083744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922348+0000","last_change":"2023-11-06T03:15:47.904066+0000","last_active":"2023-11-06T03:15:47.922348+0000","last_peered":"2023-11-06T03:15:47.922348+0000","last_clean":"2023-11-06T03:15:47.922348+0000","last_became_active":"2023-11-06T03:15:47.903431+0000","last_became_peered":"2023-11-06T03:15:47.903431+0000","last_unstale":"2023-11-06T03:15:47.922348+0000","last_undegraded":"2023-11-06T03:15:47.922348+0000","last_fullsized":"2023-11-06T03:15:47.922348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:02.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"148'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860773+0000","last_change":"2023-11-06T03:15:47.509422+0000","last_active":"2023-11-06T03:15:48.860773+0000","last_peered":"2023-11-06T03:15:48.860773+0000","last_clean":"2023-11-06T03:15:48.860773+0000","last_became_active":"2023-11-06T03:15:47.508910+0000","last_became_peered":"2023-11-06T03:15:47.508910+0000","last_unstale":"2023-11-06T03:15:48.860773+0000","last_undegraded":"2023-11-06T03:15:48.860773+0000","last_fullsized":"2023-11-06T03:15:48.860773+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:51:52.924237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878684+0000","last_change":"2023-11-06T03:15:47.875146+0000","last_active":"2023-11-06T03:15:47.878684+0000","last_peered":"2023-11-06T03:15:47.878684+0000","last_clean":"2023-11-06T03:15:47.878684+0000","last_became_active":"2023-11-06T03:15:47.874725+0000","last_became_peered":"2023-11-06T03:15:47.874725+0000","last_unstale":"2023-11-06T03:15:47.878684+0000","last_undegraded":"2023-11-06T03:15:47.878684+0000","last_fullsized":"2023-11-06T03:15:47.878684+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897432+0000","last_change":"2023-11-06T03:15:47.492282+0000","last_active":"2023-11-06T03:15:47.897432+0000","last_peered":"2023-11-06T03:15:47.897432+0000","last_clean":"2023-11-06T03:15:47.897432+0000","last_became_active":"2023-11-06T03:15:47.491702+0000","last_became_peered":"2023-11-06T03:15:47.491702+0000","last_unstale":"2023-11-06T03:15:47.897432+0000","last_undegraded":"2023-11-06T03:15:47.897432+0000","last_fullsized":"2023-11-06T03:15:47.897432+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:36:01.984847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921401+0000","last_change":"2023-11-06T03:15:47.913766+0000","last_active":"2023-11-06T03:15:47.921401+0000","last_peered":"2023-11-06T03:15:47.921401+0000","last_clean":"2023-11-06T03:15:47.921401+0000","last_became_active":"2023-11-06T03:15:47.913186+0000","last_became_peered":"2023-11-06T03:15:47.913186+0000","last_unstale":"2023-11-06T03:15:47.921401+0000","last_undegraded":"2023-11-06T03:15:47.921401+0000","last_fullsized":"2023-11-06T03:15:47.921401+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860178+0000","last_change":"2023-11-06T03:15:47.509099+0000","last_active":"2023-11-06T03:15:48.860178+0000","last_peered":"2023-11-06T03:15:48.860178+0000","last_clean":"2023-11-06T03:15:48.860178+0000","last_became_active":"2023-11-06T03:15:47.508623+0000","last_became_peered":"2023-11-06T03:15:47.508623+0000","last_unstale":"2023-11-06T03:15:48.860178+0000","last_undegraded":"2023-11-06T03:15:48.860178+0000","last_fullsized":"2023-11-06T03:15:48.860178+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:35.361093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905541+0000","last_change":"2023-11-06T03:15:47.893008+0000","last_active":"2023-11-06T03:15:47.905541+0000","last_peered":"2023-11-06T03:15:47.905541+0000","last_clean":"2023-11-06T03:15:47.905541+0000","last_became_active":"2023-11-06T03:15:47.892227+0000","last_became_peered":"2023-11-06T03:15:47.892227+0000","last_unstale":"2023-11-06T03:15:47.905541+0000","last_undegraded":"2023-11-06T03:15:47.905541+0000","last_fullsized":"2023-11-06T03:15:47.905541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:09:36.363931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878369+0000","last_change":"2023-11-06T03:15:47.507458+0000","last_active":"2023-11-06T03:15:47.878369+0000","last_peered":"2023-11-06T03:15:47.878369+0000","last_clean":"2023-11-06T03:15:47.878369+0000","last_became_active":"2023-11-06T03:15:47.507035+0000","last_became_peered":"2023-11-06T03:15:47.507035+0000","last_unstale":"2023-11-06T03:15:47.878369+0000","last_undegraded":"2023-11-06T03:15:47.878369+0000","last_fullsized":"2023-11-06T03:15:47.878369+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:04:30.873334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.907468+0000","last_change":"2023-11-06T03:15:47.905474+0000","last_active":"2023-11-06T03:15:47.907468+0000","last_peered":"2023-11-06T03:15:47.907468+0000","last_clean":"2023-11-06T03:15:47.907468+0000","last_became_active":"2023-11-06T03:15:47.904286+0000","last_became_peered":"2023-11-06T03:15:47.904286+0000","last_unstale":"2023-11-06T03:15:47.907468+0000","last_undegraded":"2023-11-06T03:15:47.907468+0000","last_fullsized":"2023-11-06T03:15:47.907468+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886377+0000","last_change":"2023-11-06T03:15:47.507568+0000","last_active":"2023-11-06T03:15:47.886377+0000","last_peered":"2023-11-06T03:15:47.886377+0000","last_clean":"2023-11-06T03:15:47.886377+0000","last_became_active":"2023-11-06T03:15:47.507005+0000","last_became_peered":"2023-11-06T03:15:47.507005+0000","last_unstale":"2023-11-06T03:15:47.886377+0000","last_undegraded":"2023-11-06T03:15:47.886377+0000","last_fullsized":"2023-11-06T03:15:47.886377+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:16:31.541526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921622+0000","last_change":"2023-11-06T03:15:47.914949+0000","last_active":"2023-11-06T03:15:47.921622+0000","last_peered":"2023-11-06T03:15:47.921622+0000","last_clean":"2023-11-06T03:15:47.921622+0000","last_became_active":"2023-11-06T03:15:47.914385+0000","last_became_peered":"2023-11-06T03:15:47.914385+0000","last_unstale":"2023-11-06T03:15:47.921622+0000","last_undegraded":"2023-11-06T03:15:47.921622+0000","last_fullsized":"2023-11-06T03:15:47.921622+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:27.745003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886949+0000","last_change":"2023-11-06T03:15:47.509075+0000","last_active":"2023-11-06T03:15:47.886949+0000","last_peered":"2023-11-06T03:15:47.886949+0000","last_clean":"2023-11-06T03:15:47.886949+0000","last_became_active":"2023-11-06T03:15:47.508821+0000","last_became_peered":"2023-11-06T03:15:47.508821+0000","last_unstale":"2023-11-06T03:15:47.886949+0000","last_undegraded":"2023-11-06T03:15:47.886949+0000","last_fullsized":"2023-11-06T03:15:47.886949+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:45:06.497188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924055+0000","last_change":"2023-11-06T03:15:47.920514+0000","last_active":"2023-11-06T03:15:47.924055+0000","last_peered":"2023-11-06T03:15:47.924055+0000","last_clean":"2023-11-06T03:15:47.924055+0000","last_became_active":"2023-11-06T03:15:47.920040+0000","last_became_peered":"2023-11-06T03:15:47.920040+0000","last_unstale":"2023-11-06T03:15:47.924055+0000","last_undegraded":"2023-11-06T03:15:47.924055+0000","last_fullsized":"2023-11-06T03:15:47.924055+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:27:46.809847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.887413+0000","last_change":"2023-11-06T03:15:47.527699+0000","last_active":"2023-11-06T03:15:47.887413+0000","last_peered":"2023-11-06T03:15:47.887413+0000","last_clean":"2023-11-06T03:15:47.887413+0000","last_became_active":"2023-11-06T03:15:47.527035+0000","last_became_peered":"2023-11-06T03:15:47.527035+0000","last_unstale":"2023-11-06T03:15:47.887413+0000","last_undegraded":"2023-11-06T03:15:47.887413+0000","last_fullsized":"2023-11-06T03:15:47.887413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:35:08.162833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.906758+0000","last_change":"2023-11-06T03:15:47.903415+0000","last_active":"2023-11-06T03:15:47.906758+0000","last_peered":"2023-11-06T03:15:47.906758+0000","last_clean":"2023-11-06T03:15:47.906758+0000","last_became_active":"2023-11-06T03:15:47.902700+0000","last_became_peered":"2023-11-06T03:15:47.902700+0000","last_unstale":"2023-11-06T03:15:47.906758+0000","last_undegraded":"2023-11-06T03:15:47.906758+0000","last_fullsized":"2023-11-06T03:15:47.906758+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.879626+0000","last_change":"2023-11-06T03:15:47.525119+0000","last_active":"2023-11-06T03:15:47.879626+0000","last_peered":"2023-11-06T03:15:47.879626+0000","last_clean":"2023-11-06T03:15:47.879626+0000","last_became_active":"2023-11-06T03:15:47.524759+0000","last_became_peered":"2023-11-06T03:15:47.524759+0000","last_unstale":"2023-11-06T03:15:47.879626+0000","last_undegraded":"2023-11-06T03:15:47.879626+0000","last_fullsized":"2023-11-06T03:15:47.879626+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:50.081738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.917316+0000","last_change":"2023-11-06T03:15:47.915837+0000","last_active":"2023-11-06T03:15:47.917316+0000","last_peered":"2023-11-06T03:15:47.917316+0000","last_clean":"2023-11-06T03:15:47.917316+0000","last_became_active":"2023-11-06T03:15:47.915562+0000","last_became_peered":"2023-11-06T03:15:47.915562+0000","last_unstale":"2023-11-06T03:15:47.917316+0000","last_undegraded":"2023-11-06T03:15:47.917316+0000","last_fullsized":"2023-11-06T03:15:47.917316+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865355+0000","last_change":"2023-11-06T03:15:47.514671+0000","last_active":"2023-11-06T03:15:48.865355+0000","last_peered":"2023-11-06T03:15:48.865355+0000","last_clean":"2023-11-06T03:15:48.865355+0000","last_became_active":"2023-11-06T03:15:47.513777+0000","last_became_peered":"2023-11-06T03:15:47.513777+0000","last_unstale":"2023-11-06T03:15:48.865355+0000","last_undegraded":"2023-11-06T03:15:48.865355+0000","last_fullsized":"2023-11-06T03:15:48.865355+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:01:05.855936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.891996+0000","last_change":"2023-11-06T03:15:47.880623+0000","last_active":"2023-11-06T03:15:47.891996+0000","last_peered":"2023-11-06T03:15:47.891996+0000","last_clean":"2023-11-06T03:15:47.891996+0000","last_became_active":"2023-11-06T03:15:47.879820+0000","last_became_peered":"2023-11-06T03:15:47.879820+0000","last_unstale":"2023-11-06T03:15:47.891996+0000","last_undegraded":"2023-11-06T03:15:47.891996+0000","last_fullsized":"2023-11-06T03:15:47.891996+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:21:50.119788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.857915+0000","last_change":"2023-11-06T03:15:47.525153+0000","last_active":"2023-11-06T03:15:47.857915+0000","last_peered":"2023-11-06T03:15:47.857915+0000","last_clean":"2023-11-06T03:15:47.857915+0000","last_became_active":"2023-11-06T03:15:47.524724+0000","last_became_peered":"2023-11-06T03:15:47.524724+0000","last_unstale":"2023-11-06T03:15:47.857915+0000","last_undegraded":"2023-11-06T03:15:47.857915+0000","last_fullsized":"2023-11-06T03:15:47.857915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:20:46.699184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.902546+0000","last_change":"2023-11-06T03:15:47.901780+0000","last_active":"2023-11-06T03:15:47.902546+0000","last_peered":"2023-11-06T03:15:47.902546+0000","last_clean":"2023-11-06T03:15:47.902546+0000","last_became_active":"2023-11-06T03:15:47.901421+0000","last_became_peered":"2023-11-06T03:15:47.901421+0000","last_unstale":"2023-11-06T03:15:47.902546+0000","last_undegraded":"2023-11-06T03:15:47.902546+0000","last_fullsized":"2023-11-06T03:15:47.902546+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:15.438881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865509+0000","last_change":"2023-11-06T03:15:47.526863+0000","last_active":"2023-11-06T03:15:48.865509+0000","last_peered":"2023-11-06T03:15:48.865509+0000","last_clean":"2023-11-06T03:15:48.865509+0000","last_became_active":"2023-11-06T03:15:47.526344+0000","last_became_peered":"2023-11-06T03:15:47.526344+0000","last_unstale":"2023-11-06T03:15:48.865509+0000","last_undegraded":"2023-11-06T03:15:48.865509+0000","last_fullsized":"2023-11-06T03:15:48.865509+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:19:26.518153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860601+0000","last_change":"2023-11-06T03:15:47.912331+0000","last_active":"2023-11-06T03:15:48.860601+0000","last_peered":"2023-11-06T03:15:48.860601+0000","last_clean":"2023-11-06T03:15:48.860601+0000","last_became_active":"2023-11-06T03:15:47.912122+0000","last_became_peered":"2023-11-06T03:15:47.912122+0000","last_unstale":"2023-11-06T03:15:48.860601+0000","last_undegraded":"2023-11-06T03:15:48.860601+0000","last_fullsized":"2023-11-06T03:15:48.860601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:14:18.764251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.915613+0000","last_change":"2023-11-06T03:15:47.527814+0000","last_active":"2023-11-06T03:15:47.915613+0000","last_peered":"2023-11-06T03:15:47.915613+0000","last_clean":"2023-11-06T03:15:47.915613+0000","last_became_active":"2023-11-06T03:15:47.527003+0000","last_became_peered":"2023-11-06T03:15:47.527003+0000","last_unstale":"2023-11-06T03:15:47.915613+0000","last_undegraded":"2023-11-06T03:15:47.915613+0000","last_fullsized":"2023-11-06T03:15:47.915613+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:43:00.368386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.924443+0000","last_change":"2023-11-06T03:15:47.915067+0000","last_active":"2023-11-06T03:15:47.924443+0000","last_peered":"2023-11-06T03:15:47.924443+0000","last_clean":"2023-11-06T03:15:47.924443+0000","last_became_active":"2023-11-06T03:15:47.914683+0000","last_became_peered":"2023-11-06T03:15:47.914683+0000","last_unstale":"2023-11-06T03:15:47.924443+0000","last_undegraded":"2023-11-06T03:15:47.924443+0000","last_fullsized":"2023-11-06T03:15:47.924443+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:35.479381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.898500+0000","last_change":"2023-11-06T03:15:47.509688+0000","last_active":"2023-11-06T03:15:47.898500+0000","last_peered":"2023-11-06T03:15:47.898500+0000","last_clean":"2023-11-06T03:15:47.898500+0000","last_became_active":"2023-11-06T03:15:47.509023+0000","last_became_peered":"2023-11-06T03:15:47.509023+0000","last_unstale":"2023-11-06T03:15:47.898500+0000","last_undegraded":"2023-11-06T03:15:47.898500+0000","last_fullsized":"2023-11-06T03:15:47.898500+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:12:21.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921563+0000","last_change":"2023-11-06T03:15:47.913697+0000","last_active":"2023-11-06T03:15:47.921563+0000","last_peered":"2023-11-06T03:15:47.921563+0000","last_clean":"2023-11-06T03:15:47.921563+0000","last_became_active":"2023-11-06T03:15:47.913139+0000","last_became_peered":"2023-11-06T03:15:47.913139+0000","last_unstale":"2023-11-06T03:15:47.921563+0000","last_undegraded":"2023-11-06T03:15:47.921563+0000","last_fullsized":"2023-11-06T03:15:47.921563+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:32.150958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.866543+0000","last_change":"2023-11-06T03:15:47.525263+0000","last_active":"2023-11-06T03:15:47.866543+0000","last_peered":"2023-11-06T03:15:47.866543+0000","last_clean":"2023-11-06T03:15:47.866543+0000","last_became_active":"2023-11-06T03:15:47.524956+0000","last_became_peered":"2023-11-06T03:15:47.524956+0000","last_unstale":"2023-11-06T03:15:47.866543+0000","last_undegraded":"2023-11-06T03:15:47.866543+0000","last_fullsized":"2023-11-06T03:15:47.866543+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:13:30.870788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860577+0000","last_change":"2023-11-06T03:15:47.915229+0000","last_active":"2023-11-06T03:15:48.860577+0000","last_peered":"2023-11-06T03:15:48.860577+0000","last_clean":"2023-11-06T03:15:48.860577+0000","last_became_active":"2023-11-06T03:15:47.914946+0000","last_became_peered":"2023-11-06T03:15:47.914946+0000","last_unstale":"2023-11-06T03:15:48.860577+0000","last_undegraded":"2023-11-06T03:15:48.860577+0000","last_fullsized":"2023-11-06T03:15:48.860577+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:04:56.264390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860472+0000","last_change":"2023-11-06T03:15:47.525331+0000","last_active":"2023-11-06T03:15:48.860472+0000","last_peered":"2023-11-06T03:15:48.860472+0000","last_clean":"2023-11-06T03:15:48.860472+0000","last_became_active":"2023-11-06T03:15:47.525007+0000","last_became_peered":"2023-11-06T03:15:47.525007+0000","last_unstale":"2023-11-06T03:15:48.860472+0000","last_undegraded":"2023-11-06T03:15:48.860472+0000","last_fullsized":"2023-11-06T03:15:48.860472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:03.519574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865407+0000","last_change":"2023-11-06T03:15:47.881406+0000","last_active":"2023-11-06T03:15:48.865407+0000","last_peered":"2023-11-06T03:15:48.865407+0000","last_clean":"2023-11-06T03:15:48.865407+0000","last_became_active":"2023-11-06T03:15:47.878599+0000","last_became_peered":"2023-11-06T03:15:47.878599+0000","last_unstale":"2023-11-06T03:15:48.865407+0000","last_undegraded":"2023-11-06T03:15:48.865407+0000","last_fullsized":"2023-11-06T03:15:48.865407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897846+0000","last_change":"2023-11-06T03:15:47.511309+0000","last_active":"2023-11-06T03:15:47.897846+0000","last_peered":"2023-11-06T03:15:47.897846+0000","last_clean":"2023-11-06T03:15:47.897846+0000","last_became_active":"2023-11-06T03:15:47.510724+0000","last_became_peered":"2023-11-06T03:15:47.510724+0000","last_unstale":"2023-11-06T03:15:47.897846+0000","last_undegraded":"2023-11-06T03:15:47.897846+0000","last_fullsized":"2023-11-06T03:15:47.897846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:29:05.547526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.922510+0000","last_change":"2023-11-06T03:15:47.908992+0000","last_active":"2023-11-06T03:15:47.922510+0000","last_peered":"2023-11-06T03:15:47.922510+0000","last_clean":"2023-11-06T03:15:47.922510+0000","last_became_active":"2023-11-06T03:15:47.903685+0000","last_became_peered":"2023-11-06T03:15:47.903685+0000","last_unstale":"2023-11-06T03:15:47.922510+0000","last_undegraded":"2023-11-06T03:15:47.922510+0000","last_fullsized":"2023-11-06T03:15:47.922510+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:17.797481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.851749+0000","last_change":"2023-11-06T03:15:47.518655+0000","last_active":"2023-11-06T03:15:47.851749+0000","last_peered":"2023-11-06T03:15:47.851749+0000","last_clean":"2023-11-06T03:15:47.851749+0000","last_became_active":"2023-11-06T03:15:47.518326+0000","last_became_peered":"2023-11-06T03:15:47.518326+0000","last_unstale":"2023-11-06T03:15:47.851749+0000","last_undegraded":"2023-11-06T03:15:47.851749+0000","last_fullsized":"2023-11-06T03:15:47.851749+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:56.547181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878651+0000","last_change":"2023-11-06T03:15:47.874653+0000","last_active":"2023-11-06T03:15:47.878651+0000","last_peered":"2023-11-06T03:15:47.878651+0000","last_clean":"2023-11-06T03:15:47.878651+0000","last_became_active":"2023-11-06T03:15:47.872756+0000","last_became_peered":"2023-11-06T03:15:47.872756+0000","last_unstale":"2023-11-06T03:15:47.878651+0000","last_undegraded":"2023-11-06T03:15:47.878651+0000","last_fullsized":"2023-11-06T03:15:47.878651+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:29:39.792854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.897471+0000","last_change":"2023-11-06T03:15:47.494228+0000","last_active":"2023-11-06T03:15:47.897471+0000","last_peered":"2023-11-06T03:15:47.897471+0000","last_clean":"2023-11-06T03:15:47.897471+0000","last_became_active":"2023-11-06T03:15:47.493507+0000","last_became_peered":"2023-11-06T03:15:47.493507+0000","last_unstale":"2023-11-06T03:15:47.897471+0000","last_undegraded":"2023-11-06T03:15:47.897471+0000","last_fullsized":"2023-11-06T03:15:47.897471+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:44:34.476544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865505+0000","last_change":"2023-11-06T03:15:47.881947+0000","last_active":"2023-11-06T03:15:48.865505+0000","last_peered":"2023-11-06T03:15:48.865505+0000","last_clean":"2023-11-06T03:15:48.865505+0000","last_became_active":"2023-11-06T03:15:47.881192+0000","last_became_peered":"2023-11-06T03:15:47.881192+0000","last_unstale":"2023-11-06T03:15:48.865505+0000","last_undegraded":"2023-11-06T03:15:48.865505+0000","last_fullsized":"2023-11-06T03:15:48.865505+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:33:45.104284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.873724+0000","last_change":"2023-11-06T03:15:47.507220+0000","last_active":"2023-11-06T03:15:47.873724+0000","last_peered":"2023-11-06T03:15:47.873724+0000","last_clean":"2023-11-06T03:15:47.873724+0000","last_became_active":"2023-11-06T03:15:47.506684+0000","last_became_peered":"2023-11-06T03:15:47.506684+0000","last_unstale":"2023-11-06T03:15:47.873724+0000","last_undegraded":"2023-11-06T03:15:47.873724+0000","last_fullsized":"2023-11-06T03:15:47.873724+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:48.117488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.866302+0000","last_change":"2023-11-06T03:15:47.889415+0000","last_active":"2023-11-06T03:15:48.866302+0000","last_peered":"2023-11-06T03:15:48.866302+0000","last_clean":"2023-11-06T03:15:48.866302+0000","last_became_active":"2023-11-06T03:15:47.869850+0000","last_became_peered":"2023-11-06T03:15:47.869850+0000","last_unstale":"2023-11-06T03:15:48.866302+0000","last_undegraded":"2023-11-06T03:15:48.866302+0000","last_fullsized":"2023-11-06T03:15:48.866302+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:34:39.339629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"148'6","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.921137+0000","last_change":"2023-11-06T03:15:47.512744+0000","last_active":"2023-11-06T03:15:47.921137+0000","last_peered":"2023-11-06T03:15:47.921137+0000","last_clean":"2023-11-06T03:15:47.921137+0000","last_became_active":"2023-11-06T03:15:47.512370+0000","last_became_peered":"2023-11-06T03:15:47.512370+0000","last_unstale":"2023-11-06T03:15:47.921137+0000","last_undegraded":"2023-11-06T03:15:47.921137+0000","last_fullsized":"2023-11-06T03:15:47.921137+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:51:21.567227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":270,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490712+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:15:47.490712+0000","last_peered":"2023-11-06T03:15:47.490712+0000","last_clean":"2023-11-06T03:15:47.490712+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:15:47.490712+0000","last_undegraded":"2023-11-06T03:15:47.490712+0000","last_fullsized":"2023-11-06T03:15:47.490712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":114,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.491754+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:15:47.491754+0000","last_peered":"2023-11-06T03:15:47.491754+0000","last_clean":"2023-11-06T03:15:47.491754+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:15:47.491754+0000","last_undegraded":"2023-11-06T03:15:47.491754+0000","last_fullsized":"2023-11-06T03:15:47.491754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865855+0000","last_change":"2023-11-06T03:15:47.895227+0000","last_active":"2023-11-06T03:15:48.865855+0000","last_peered":"2023-11-06T03:15:48.865855+0000","last_clean":"2023-11-06T03:15:48.865855+0000","last_became_active":"2023-11-06T03:15:47.889409+0000","last_became_peered":"2023-11-06T03:15:47.889409+0000","last_unstale":"2023-11-06T03:15:48.865855+0000","last_undegraded":"2023-11-06T03:15:48.865855+0000","last_fullsized":"2023-11-06T03:15:48.865855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:44:17.047593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.880053+0000","last_change":"2023-11-06T03:15:47.508753+0000","last_active":"2023-11-06T03:15:47.880053+0000","last_peered":"2023-11-06T03:15:47.880053+0000","last_clean":"2023-11-06T03:15:47.880053+0000","last_became_active":"2023-11-06T03:15:47.508109+0000","last_became_peered":"2023-11-06T03:15:47.508109+0000","last_unstale":"2023-11-06T03:15:47.880053+0000","last_undegraded":"2023-11-06T03:15:47.880053+0000","last_fullsized":"2023-11-06T03:15:47.880053+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:49:07.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":117,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490530+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:15:47.490530+0000","last_peered":"2023-11-06T03:15:47.490530+0000","last_clean":"2023-11-06T03:15:47.490530+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:15:47.490530+0000","last_undegraded":"2023-11-06T03:15:47.490530+0000","last_fullsized":"2023-11-06T03:15:47.490530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.876358+0000","last_change":"2023-11-06T03:15:47.870180+0000","last_active":"2023-11-06T03:15:47.876358+0000","last_peered":"2023-11-06T03:15:47.876358+0000","last_clean":"2023-11-06T03:15:47.876358+0000","last_became_active":"2023-11-06T03:15:47.869900+0000","last_became_peered":"2023-11-06T03:15:47.869900+0000","last_unstale":"2023-11-06T03:15:47.876358+0000","last_undegraded":"2023-11-06T03:15:47.876358+0000","last_fullsized":"2023-11-06T03:15:47.876358+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:40.548877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.898235+0000","last_change":"2023-11-06T03:15:47.499101+0000","last_active":"2023-11-06T03:15:47.898235+0000","last_peered":"2023-11-06T03:15:47.898235+0000","last_clean":"2023-11-06T03:15:47.898235+0000","last_became_active":"2023-11-06T03:15:47.498544+0000","last_became_peered":"2023-11-06T03:15:47.498544+0000","last_unstale":"2023-11-06T03:15:47.898235+0000","last_undegraded":"2023-11-06T03:15:47.898235+0000","last_fullsized":"2023-11-06T03:15:47.898235+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:21:16.794644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.489167+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:15:47.489167+0000","last_peered":"2023-11-06T03:15:47.489167+0000","last_clean":"2023-11-06T03:15:47.489167+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:15:47.489167+0000","last_undegraded":"2023-11-06T03:15:47.489167+0000","last_fullsized":"2023-11-06T03:15:47.489167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916619+0000","last_change":"2023-11-06T03:15:47.914994+0000","last_active":"2023-11-06T03:15:47.916619+0000","last_peered":"2023-11-06T03:15:47.916619+0000","last_clean":"2023-11-06T03:15:47.916619+0000","last_became_active":"2023-11-06T03:15:47.914194+0000","last_became_peered":"2023-11-06T03:15:47.914194+0000","last_unstale":"2023-11-06T03:15:47.916619+0000","last_undegraded":"2023-11-06T03:15:47.916619+0000","last_fullsized":"2023-11-06T03:15:47.916619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:50:40.242424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"148'3","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.877232+0000","last_change":"2023-11-06T03:15:47.491520+0000","last_active":"2023-11-06T03:15:47.877232+0000","last_peered":"2023-11-06T03:15:47.877232+0000","last_clean":"2023-11-06T03:15:47.877232+0000","last_became_active":"2023-11-06T03:15:47.491014+0000","last_became_peered":"2023-11-06T03:15:47.491014+0000","last_unstale":"2023-11-06T03:15:47.877232+0000","last_undegraded":"2023-11-06T03:15:47.877232+0000","last_fullsized":"2023-11-06T03:15:47.877232+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:45:27.664756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":114,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.490703+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:15:47.490703+0000","last_peered":"2023-11-06T03:15:47.490703+0000","last_clean":"2023-11-06T03:15:47.490703+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:15:47.490703+0000","last_undegraded":"2023-11-06T03:15:47.490703+0000","last_fullsized":"2023-11-06T03:15:47.490703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.865297+0000","last_change":"2023-11-06T03:15:47.890055+0000","last_active":"2023-11-06T03:15:48.865297+0000","last_peered":"2023-11-06T03:15:48.865297+0000","last_clean":"2023-11-06T03:15:48.865297+0000","last_became_active":"2023-11-06T03:15:47.882281+0000","last_became_peered":"2023-11-06T03:15:47.882281+0000","last_unstale":"2023-11-06T03:15:48.865297+0000","last_undegraded":"2023-11-06T03:15:48.865297+0000","last_fullsized":"2023-11-06T03:15:48.865297+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"148'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860546+0000","last_change":"2023-11-06T03:15:47.495663+0000","last_active":"2023-11-06T03:15:48.860546+0000","last_peered":"2023-11-06T03:15:48.860546+0000","last_clean":"2023-11-06T03:15:48.860546+0000","last_became_active":"2023-11-06T03:15:47.495420+0000","last_became_peered":"2023-11-06T03:15:47.495420+0000","last_unstale":"2023-11-06T03:15:48.860546+0000","last_undegraded":"2023-11-06T03:15:48.860546+0000","last_fullsized":"2023-11-06T03:15:48.860546+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:31.148120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":122,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.473182+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:15:47.473182+0000","last_peered":"2023-11-06T03:15:47.473182+0000","last_clean":"2023-11-06T03:15:47.473182+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:15:47.473182+0000","last_undegraded":"2023-11-06T03:15:47.473182+0000","last_fullsized":"2023-11-06T03:15:47.473182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.905522+0000","last_change":"2023-11-06T03:15:47.904915+0000","last_active":"2023-11-06T03:15:47.905522+0000","last_peered":"2023-11-06T03:15:47.905522+0000","last_clean":"2023-11-06T03:15:47.905522+0000","last_became_active":"2023-11-06T03:15:47.904404+0000","last_became_peered":"2023-11-06T03:15:47.904404+0000","last_unstale":"2023-11-06T03:15:47.905522+0000","last_undegraded":"2023-11-06T03:15:47.905522+0000","last_fullsized":"2023-11-06T03:15:47.905522+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:47.287086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.886537+0000","last_change":"2023-11-06T03:15:47.528591+0000","last_active":"2023-11-06T03:15:47.886537+0000","last_peered":"2023-11-06T03:15:47.886537+0000","last_clean":"2023-11-06T03:15:47.886537+0000","last_became_active":"2023-11-06T03:15:47.527997+0000","last_became_peered":"2023-11-06T03:15:47.527997+0000","last_unstale":"2023-11-06T03:15:47.886537+0000","last_undegraded":"2023-11-06T03:15:47.886537+0000","last_fullsized":"2023-11-06T03:15:47.886537+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:41:05.594496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.489956+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:15:47.489956+0000","last_peered":"2023-11-06T03:15:47.489956+0000","last_clean":"2023-11-06T03:15:47.489956+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:15:47.489956+0000","last_undegraded":"2023-11-06T03:15:47.489956+0000","last_fullsized":"2023-11-06T03:15:47.489956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-11-06T03:15:48.860659+0000","last_change":"2023-11-06T03:15:47.878228+0000","last_active":"2023-11-06T03:15:48.860659+0000","last_peered":"2023-11-06T03:15:48.860659+0000","last_clean":"2023-11-06T03:15:48.860659+0000","last_became_active":"2023-11-06T03:15:47.877147+0000","last_became_peered":"2023-11-06T03:15:47.877147+0000","last_unstale":"2023-11-06T03:15:48.860659+0000","last_undegraded":"2023-11-06T03:15:48.860659+0000","last_fullsized":"2023-11-06T03:15:48.860659+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:03:14.405112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.887095+0000","last_change":"2023-11-06T03:15:47.509631+0000","last_active":"2023-11-06T03:15:47.887095+0000","last_peered":"2023-11-06T03:15:47.887095+0000","last_clean":"2023-11-06T03:15:47.887095+0000","last_became_active":"2023-11-06T03:15:47.508958+0000","last_became_peered":"2023-11-06T03:15:47.508958+0000","last_unstale":"2023-11-06T03:15:47.887095+0000","last_undegraded":"2023-11-06T03:15:47.887095+0000","last_fullsized":"2023-11-06T03:15:47.887095+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:48:01.412718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":122,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.473309+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:15:47.473309+0000","last_peered":"2023-11-06T03:15:47.473309+0000","last_clean":"2023-11-06T03:15:47.473309+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:15:47.473309+0000","last_undegraded":"2023-11-06T03:15:47.473309+0000","last_fullsized":"2023-11-06T03:15:47.473309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.878348+0000","last_change":"2023-11-06T03:15:47.874637+0000","last_active":"2023-11-06T03:15:47.878348+0000","last_peered":"2023-11-06T03:15:47.878348+0000","last_clean":"2023-11-06T03:15:47.878348+0000","last_became_active":"2023-11-06T03:15:47.874338+0000","last_became_peered":"2023-11-06T03:15:47.874338+0000","last_unstale":"2023-11-06T03:15:47.878348+0000","last_undegraded":"2023-11-06T03:15:47.878348+0000","last_fullsized":"2023-11-06T03:15:47.878348+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:47.462885+0000","last_clean_scrub_stamp":"2023-11-06T03:15:47.462885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:27.356125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"148'8","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.916020+0000","last_change":"2023-11-06T03:15:47.503853+0000","last_active":"2023-11-06T03:15:47.916020+0000","last_peered":"2023-11-06T03:15:47.916020+0000","last_clean":"2023-11-06T03:15:47.916020+0000","last_became_active":"2023-11-06T03:15:47.503218+0000","last_became_peered":"2023-11-06T03:15:47.503218+0000","last_unstale":"2023-11-06T03:15:47.916020+0000","last_undegraded":"2023-11-06T03:15:47.916020+0000","last_fullsized":"2023-11-06T03:15:47.916020+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:15:46.460048+0000","last_clean_scrub_stamp":"2023-11-06T03:15:46.460048+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:36:18.308233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":113,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-11-06T03:15:47.498828+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:15:47.498828+0000","last_peered":"2023-11-06T03:15:47.498828+0000","last_clean":"2023-11-06T03:15:47.498828+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:15:47.498828+0000","last_undegraded":"2023-11-06T03:15:47.498828+0000","last_fullsized":"2023-11-06T03:15:47.498828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91616,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280224,"statfs":{"total":96636764160,"available":96542949376,"internally_reserved":0,"allocated":2449408,"data_stored":1447712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.238,"15min":1.4830000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78900000000000003,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.157,"5min":1.202,"15min":1.397},"last":0.63800000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52500000000000002,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.167,"5min":1.627,"15min":1.627},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.84799999999999998,"15min":0.875},"min":{"1min":0.56499999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1910000000000001,"5min":1.3149999999999999,"15min":1.6899999999999999},"last":0.52200000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.86199999999999999,"15min":0.879},"min":{"1min":0.625,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.105,"5min":1.2170000000000001,"15min":1.6080000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85899999999999999,"15min":0.90700000000000003},"min":{"1min":0.59799999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.147,"5min":1.272,"15min":1.6259999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.86899999999999999,"15min":0.89100000000000001},"min":{"1min":0.66900000000000004,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.159,"5min":1.4099999999999999,"15min":2.512},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.94999999999999996,"15min":1.038},"min":{"1min":0.67800000000000005,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.431,"5min":1.431,"15min":10.596},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.72799999999999998,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":1.54,"15min":1.9399999999999999},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0329999999999999,"15min":1.044},"min":{"1min":0.71499999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2889999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.071,"15min":1.0780000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.327,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.204,"15min":1.1659999999999999},"min":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":1.399,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.008},{"interface":"front","average":{"1min":1.101,"5min":1.0629999999999999,"15min":1.095},"min":{"1min":0.80800000000000005,"5min":0.69499999999999995,"15min":0.53300000000000003},"max":{"1min":1.329,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":0.93400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68888,"kb_used_data":1808,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302952,"statfs":{"total":96636764160,"available":96566222848,"internally_reserved":0,"allocated":1851392,"data_stored":856373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80700000000000005,"15min":0.85399999999999998},"min":{"1min":0.64900000000000002,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2310000000000001,"5min":2.3279999999999998,"15min":16.073},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.85299999999999998,"15min":0.97999999999999998},"min":{"1min":0.59099999999999997,"5min":0.41699999999999998,"15min":0.39900000000000002},"max":{"1min":1.2370000000000001,"5min":4.4000000000000004,"15min":40.905000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84599999999999997,"15min":0.90700000000000003},"min":{"1min":0.60699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.222,"5min":1.6359999999999999,"15min":16.012},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.85899999999999999,"15min":0.876},"min":{"1min":0.56100000000000005,"5min":0.49199999999999999,"15min":0.45600000000000002},"max":{"1min":1.151,"5min":1.409,"15min":3.3050000000000002},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.91600000000000004},"min":{"1min":0.65200000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.107,"5min":1.744,"15min":27.803999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92500000000000004,"15min":0.94799999999999995},"min":{"1min":0.71999999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"max":{"1min":1.1479999999999999,"5min":1.429,"15min":2.843},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.93500000000000005,"15min":1.179},"min":{"1min":0.70499999999999996,"5min":0.55400000000000005,"15min":0.48699999999999999},"max":{"1min":1.3049999999999999,"5min":1.306,"15min":63.737000000000002},"last":0.75},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90900000000000003,"15min":1.0589999999999999},"min":{"1min":0.70299999999999996,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":1.173,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.71399999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.075,"15min":1.1619999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.512,"5min":7.4980000000000002,"15min":24.867999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.109,"5min":1.1559999999999999,"15min":1.228},"min":{"1min":0.91000000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.2969999999999999,"5min":6.3970000000000002,"15min":23.789000000000001},"last":0.96899999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.98199999999999998,"15min":1.1539999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.51800000000000002},"max":{"1min":1.694,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.103,"15min":1.1200000000000001},"min":{"1min":0.85599999999999998,"5min":0.76100000000000001,"15min":0.71499999999999997},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.02}]},{"osd":7,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.71699999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4690000000000001,"5min":4.6399999999999997,"15min":23.748999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1359999999999999,"15min":1.2250000000000001},"min":{"1min":0.92500000000000004,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.452,"5min":5.2709999999999999,"15min":28.164000000000001},"last":0.93999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1808,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1851392,"data_stored":857322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":1.804,"15min":1.1339999999999999},"min":{"1min":0.46300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.1679999999999999,"5min":125.256,"15min":125.256},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":2.0960000000000001,"15min":1.25},"min":{"1min":0.64900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1140000000000001,"5min":126.98699999999999,"15min":126.98699999999999},"last":1.089}]},{"osd":1,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.88300000000000001,"15min":0.80900000000000005},"min":{"1min":0.55200000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.034,"5min":4.6390000000000002,"15min":4.6390000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.292,"15min":0.996},"min":{"1min":0.54900000000000004,"5min":0.376,"15min":0.376},"max":{"1min":1.2030000000000001,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.0580000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.347,"15min":1.012},"min":{"1min":0.63900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.363,"5min":43.741,"15min":43.741},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84999999999999998,"15min":0.84299999999999997},"min":{"1min":0.51200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0669999999999999,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.0169999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92600000000000005,"15min":0.88300000000000001},"min":{"1min":0.61699999999999999,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":1.173,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.429,"15min":1.1040000000000001},"min":{"1min":0.65800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.296,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.1559999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.90200000000000002,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.089,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.1020000000000001,"15min":1.0129999999999999},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":1.25,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.1799999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0549999999999999,"15min":1.0229999999999999},"min":{"1min":0.68899999999999995,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.647,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.0449999999999999},"min":{"1min":0.65500000000000003,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.2509999999999999,"5min":5.6109999999999998,"15min":5.6109999999999998},"last":1.1319999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:15:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.145,"15min":1.0589999999999999},"min":{"1min":0.52500000000000002,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":1.3220000000000001,"5min":4.569,"15min":4.569},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.1479999999999999,"15min":1.109},"min":{"1min":0.73899999999999999,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":1.595,"5min":2.121,"15min":2.121},"last":1.2450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113200,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258640,"statfs":{"total":96636764160,"available":96520847360,"internally_reserved":0,"allocated":1875968,"data_stored":865539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":1.258,"15min":0.95699999999999996},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.372},"max":{"1min":1.3120000000000001,"5min":42.853999999999999,"15min":42.853999999999999},"last":1.125},{"interface":"front","average":{"1min":0.75600000000000001,"5min":1.2829999999999999,"15min":0.95399999999999996},"min":{"1min":0.437,"5min":0.308,"15min":0.308},"max":{"1min":1.103,"5min":44.609000000000002,"15min":44.609000000000002},"last":0.83399999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77600000000000002,"15min":0.76900000000000002},"min":{"1min":0.51300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1539999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":1.159},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.83899999999999997},"min":{"1min":0.48199999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2390000000000001,"5min":1.639,"15min":1.6719999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89000000000000001,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.55700000000000005,"15min":0.53300000000000003},"max":{"1min":1.3200000000000001,"5min":1.452,"15min":3.2879999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.872,"15min":0.85999999999999999},"min":{"1min":0.497,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.103,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":1.081}]},{"osd":3,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89200000000000002,"15min":0.91800000000000004},"min":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.2150000000000001,"5min":3.198,"15min":3.198},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.59099999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1919999999999999,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.98399999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.073,"15min":1.03},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2490000000000001,"5min":10.192,"15min":10.192},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0700000000000001,"15min":1.085},"min":{"1min":0.86399999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.292,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":1.2190000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98099999999999998,"15min":0.997},"min":{"1min":0.77700000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.54,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.88},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.98799999999999999,"5min":0.65300000000000002,"15min":0.57799999999999996},"max":{"1min":1.3759999999999999,"5min":1.512,"15min":2.6549999999999998},"last":1.1890000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0900000000000001,"15min":1.0800000000000001},"min":{"1min":0.754,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.383,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.107,"15min":1.1679999999999999},"min":{"1min":0.879,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.49,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82908,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288932,"statfs":{"total":96636764160,"available":96551866368,"internally_reserved":0,"allocated":1855488,"data_stored":856919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.58899999999999997},"min":{"1min":0.30299999999999999,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.94399999999999995,"5min":1.113,"15min":1.4199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":1.524},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.36399999999999999,"5min":0.251,"15min":0.224},"max":{"1min":1.024,"5min":1.1839999999999999,"15min":1.371},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":1.421,"5min":1.532,"15min":1.532},"last":0.496},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40799999999999997,"5min":0.32000000000000001,"15min":0.255},"max":{"1min":1.052,"5min":1.206,"15min":1.284},"last":0.433}]},{"osd":4,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.86899999999999999},"min":{"1min":0.51000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2529999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.89500000000000002,"15min":0.91700000000000004},"min":{"1min":0.58599999999999997,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.1739999999999999,"5min":1.52,"15min":1.8300000000000001},"last":0.628}]},{"osd":5,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.95899999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.302,"5min":1.5,"15min":1.6639999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.95999999999999996,"15min":0.98499999999999999},"min":{"1min":0.69799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.2050000000000001,"5min":1.573,"15min":1.698},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.95699999999999996,"15min":0.97899999999999998},"min":{"1min":0.64800000000000002,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.387,"5min":1.51,"15min":1.8620000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.96099999999999997},"min":{"1min":0.62,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.1140000000000001,"5min":1.748,"15min":1.8740000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.97399999999999998,"15min":1.01},"min":{"1min":0.65800000000000003,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.24,"5min":1.835,"15min":1.835},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0089999999999999,"15min":1.0329999999999999},"min":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":1.349,"5min":1.569,"15min":1.8979999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72992,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298848,"statfs":{"total":96636764160,"available":96562020352,"internally_reserved":0,"allocated":1859584,"data_stored":857434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.51200000000000001,"15min":0.53300000000000003},"min":{"1min":0.253,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.69799999999999995,"5min":0.93400000000000005,"15min":1.7270000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.58999999999999997},"min":{"1min":0.28799999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":0.79300000000000004,"5min":1.1140000000000001,"15min":1.4590000000000001},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.61299999999999999},"min":{"1min":0.29599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.86899999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57599999999999996,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.245,"15min":0.22800000000000001},"max":{"1min":0.82399999999999995,"5min":1.0509999999999999,"15min":1.1579999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.33900000000000002,"5min":0.307,"15min":0.23200000000000001},"max":{"1min":1.1379999999999999,"5min":1.161,"15min":1.294},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.625,"15min":0.68899999999999995},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.95299999999999996,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.79900000000000004,"15min":0.84799999999999998},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.097,"5min":1.244,"15min":1.526},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83099999999999996,"15min":0.89300000000000002},"min":{"1min":0.53300000000000003,"5min":0.45800000000000002,"15min":0.41399999999999998},"max":{"1min":1.0860000000000001,"5min":1.329,"15min":1.5229999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82099999999999995,"15min":0.85199999999999998},"min":{"1min":0.56200000000000006,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":1.012,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.91500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.4359999999999999,"15min":1.5780000000000001},"last":1.026}]},{"osd":6,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.92800000000000005,"15min":0.96599999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.194,"5min":1.542,"15min":1.615},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.90700000000000003,"15min":0.93000000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.5},"max":{"1min":1.1839999999999999,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89400000000000002,"15min":0.95999999999999996},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2050000000000001,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":1.071},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.92500000000000004},"min":{"1min":0.54200000000000004,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.1120000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1855488,"data_stored":857392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.19},"max":{"1min":1.042,"5min":1.1240000000000001,"15min":1.843},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.244,"5min":1.2470000000000001,"15min":2.1150000000000002},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.69199999999999995},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":30.861000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.64100000000000001},"min":{"1min":0.32200000000000001,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.98399999999999999,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66200000000000003,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0629999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.68300000000000005},"min":{"1min":0.34300000000000003,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81799999999999995,"15min":0.84999999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1339999999999999,"5min":1.276,"15min":2.258},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.83699999999999997,"15min":0.92100000000000004},"min":{"1min":0.52000000000000002,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.262,"5min":1.3600000000000001,"15min":14.753},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.872,"15min":1.081},"min":{"1min":0.58099999999999996,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.351,"5min":1.528,"15min":43.103999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94599999999999995,"15min":1.115},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.39100000000000001},"max":{"1min":1.333,"5min":1.3979999999999999,"15min":44.753999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.90100000000000002,"15min":1.026},"min":{"1min":0.59499999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.3240000000000001,"5min":1.5189999999999999,"15min":23.971},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.90400000000000003,"15min":1.04},"min":{"1min":0.56399999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2949999999999999,"5min":1.444,"15min":14.813000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97999999999999998,"15min":1.018},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.377,"5min":1.581,"15min":2.2799999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95999999999999996,"15min":1.0089999999999999},"min":{"1min":0.58099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.835,"5min":1.835,"15min":2.3559999999999999},"last":0.77400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95792,"kb_used_data":2408,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94276048,"statfs":{"total":96636764160,"available":96538673152,"internally_reserved":0,"allocated":2465792,"data_stored":1448941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54200000000000004,"15min":0.56599999999999995},"min":{"1min":0.23899999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.84599999999999997,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.246,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.90200000000000002,"5min":1.1459999999999999,"15min":7.1230000000000002},"last":0.72799999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22800000000000001,"15min":0.20599999999999999},"max":{"1min":0.98899999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.014},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.64000000000000001},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":0.97499999999999998,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.91200000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.87,"5min":1.1779999999999999,"15min":2.1680000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.67300000000000004,"15min":0.71199999999999997},"min":{"1min":0.32500000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1000000000000001,"5min":1.391,"15min":2.7999999999999998},"last":1.157}]},{"osd":4,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.83099999999999996,"15min":0.90000000000000002},"min":{"1min":0.52000000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1619999999999999,"5min":1.371,"15min":4.641},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82299999999999995,"15min":0.88100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.337,"5min":1.337,"15min":2.028},"last":0.878}]},{"osd":5,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88900000000000001,"15min":0.94899999999999995},"min":{"1min":0.57399999999999995,"5min":0.46000000000000002,"15min":0.45300000000000001},"max":{"1min":1.379,"5min":1.4890000000000001,"15min":2.8559999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.879,"15min":0.93300000000000005},"min":{"1min":0.59699999999999998,"5min":0.44,"15min":0.432},"max":{"1min":1.496,"5min":1.496,"15min":4.7430000000000003},"last":1.1259999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86599999999999999,"15min":0.89500000000000002},"min":{"1min":0.59799999999999998,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":2.931},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.93100000000000005,"15min":0.98699999999999999},"min":{"1min":0.56599999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91600000000000004,"15min":0.98099999999999998},"min":{"1min":0.61599999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.3740000000000001,"5min":1.3939999999999999,"15min":4.8339999999999996},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.96799999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2330000000000001,"5min":1.4670000000000001,"15min":1.97},"last":1.0369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:15:51.243 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:15:51.244 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:15:51.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 Processor -- start 2023-11-06T03:15:51.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 -- start start 2023-11-06T03:15:51.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07e41551e0 0x7f07e406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e415e5e0 0x7f07e40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07e415ea60 0x7f07e40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e415f580 con 0x7f07e41551e0 2023-11-06T03:15:51.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e415efc0 con 0x7f07e415ea60 2023-11-06T03:15:51.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e993e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415f2a0 con 0x7f07e415e5e0 2023-11-06T03:15:51.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07e41551e0 0x7f07e406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07e41551e0 0x7f07e406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37300/0 (socket says 172.21.15.157:37300) 2023-11-06T03:15:51.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1982754855 learned_addr learned my addr 172.21.15.157:0/1982754855 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:51.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e37fe640 1 --2- 172.21.15.157:0/1982754855 >> v2:172.21.15.195:3300/0 conn(0x7f07e415ea60 0x7f07e40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e27fc640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e415e5e0 0x7f07e40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1982754855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e415e5e0 msgr2=0x7f07e40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e415e5e0 0x7f07e40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1982754855 >> v2:172.21.15.195:3300/0 conn(0x7f07e415ea60 msgr2=0x7f07e40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1982754855 >> v2:172.21.15.195:3300/0 conn(0x7f07e415ea60 0x7f07e40b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.411+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1982754855 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e406f5a0 con 0x7f07e41551e0 2023-11-06T03:15:51.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e41551e0 0x7f07e406fda0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f07e40cc5d0 tx=0x7f07d401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db3fbd90fe9349f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e1ffb640 1 -- 172.21.15.157:0/1982754855 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40ab620 con 0x7f07e41551e0 2023-11-06T03:15:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e1ffb640 1 -- 172.21.15.157:0/1982754855 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d40ab780 con 0x7f07e41551e0 2023-11-06T03:15:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e1ffb640 1 -- 172.21.15.157:0/1982754855 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40abb10 con 0x7f07e41551e0 2023-11-06T03:15:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- 172.21.15.157:0/1982754855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e41551e0 msgr2=0x7f07e406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e41551e0 0x7f07e406fda0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f07e40cc5d0 tx=0x7f07d401c440 comp rx=0 tx=0).stop 2023-11-06T03:15:51.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- 172.21.15.157:0/1982754855 shutdown_connections 2023-11-06T03:15:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 --2- 172.21.15.157:0/1982754855 >> v2:172.21.15.195:3300/0 conn(0x7f07e415ea60 0x7f07e40b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e415e5e0 0x7f07e40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 --2- 172.21.15.157:0/1982754855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e41551e0 0x7f07e406fda0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- 172.21.15.157:0/1982754855 >> 172.21.15.157:0/1982754855 conn(0x7f07e4065930 msgr2=0x7f07e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- 172.21.15.157:0/1982754855 shutdown_connections 2023-11-06T03:15:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- 172.21.15.157:0/1982754855 wait complete. 2023-11-06T03:15:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 Processor -- start 2023-11-06T03:15:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.415+0000 7f07e993e640 1 -- start start 2023-11-06T03:15:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 0x7f07e4146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e414ad50 0x7f07e414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4149f90 con 0x7f07e415e5e0 2023-11-06T03:15:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e414a270 con 0x7f07e415ea60 2023-11-06T03:15:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e414a550 con 0x7f07e414ad50 2023-11-06T03:15:51.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37304/0 (socket says 172.21.15.157:37304) 2023-11-06T03:15:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1052241587 learned_addr learned my addr 172.21.15.157:0/1052241587 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e27fc640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 0x7f07e4146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e37fe640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e414ad50 0x7f07e414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e414ad50 msgr2=0x7f07e414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e414ad50 0x7f07e414d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 msgr2=0x7f07e4146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 0x7f07e4146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 -- 172.21.15.157:0/1052241587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40094d0 con 0x7f07e415e5e0 2023-11-06T03:15:51.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e27fc640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 0x7f07e4146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:51.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e2ffd640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f07d40ad040 tx=0x7f07d401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40be3f0 con 0x7f07e415e5e0 2023-11-06T03:15:51.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d40be570 con 0x7f07e415e5e0 2023-11-06T03:15:51.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e414a830 con 0x7f07e415e5e0 2023-11-06T03:15:51.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40be920 con 0x7f07e415e5e0 2023-11-06T03:15:51.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4146980 con 0x7f07e415e5e0 2023-11-06T03:15:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e4072750 con 0x7f07e415e5e0 2023-11-06T03:15:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f07d40b43f0 con 0x7f07e415e5e0 2023-11-06T03:15:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 0x7f07d0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f07d411d510 con 0x7f07e415e5e0 2023-11-06T03:15:51.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.419+0000 7f07e27fc640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 0x7f07d0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.423+0000 7f07e27fc640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 0x7f07d0067f50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f07e4154c00 tx=0x7f07d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.423+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f07d40eaf70 con 0x7f07e415e5e0 2023-11-06T03:15:51.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.639+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f07e414a0d0 con 0x7f07e415e5e0 2023-11-06T03:15:51.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07cbfff640 1 -- 172.21.15.157:0/1052241587 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f07e414a0d0 con 0x7f07e415e5e0 2023-11-06T03:15:51.644 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:15:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 msgr2=0x7f07d0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 0x7f07d0067f50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f07e4154c00 tx=0x7f07d8008040 comp rx=0 tx=0).stop 2023-11-06T03:15:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 msgr2=0x7f07e41543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f07d40ad040 tx=0x7f07d401ac80 comp rx=0 tx=0).stop 2023-11-06T03:15:51.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 shutdown_connections 2023-11-06T03:15:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f07d0065a90 0x7f07d0067f50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e414ad50 0x7f07e414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e415ea60 0x7f07e4146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 --2- 172.21.15.157:0/1052241587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07e415e5e0 0x7f07e41543b0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 >> 172.21.15.157:0/1052241587 conn(0x7f07e4065930 msgr2=0x7f07e415d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 shutdown_connections 2023-11-06T03:15:51.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.647+0000 7f07e993e640 1 -- 172.21.15.157:0/1052241587 wait complete. 2023-11-06T03:15:51.663 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:15:51.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 Processor -- start 2023-11-06T03:15:51.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 -- start start 2023-11-06T03:15:51.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6c1551e0 0x7f3a6c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c15e5e0 0x7f3a6c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 0x7f3a6c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c15f580 con 0x7f3a6c1551e0 2023-11-06T03:15:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c15efc0 con 0x7f3a6c15ea60 2023-11-06T03:15:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a71cd9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c15f2a0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a6b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6c1551e0 0x7f3a6c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a6b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a6c1551e0 0x7f3a6c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37316/0 (socket says 172.21.15.157:37316) 2023-11-06T03:15:51.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.779+0000 7f3a6b7fe640 1 -- 172.21.15.157:0/175070872 learned_addr learned my addr 172.21.15.157:0/175070872 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6affd640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c15e5e0 0x7f3a6c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6bfff640 1 --2- 172.21.15.157:0/175070872 >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 0x7f3a6c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 -- 172.21.15.157:0/175070872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c15e5e0 msgr2=0x7f3a6c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:51.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c15e5e0 0x7f3a6c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 -- 172.21.15.157:0/175070872 >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 msgr2=0x7f3a6c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:51.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/175070872 >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 0x7f3a6c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 -- 172.21.15.157:0/175070872 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c06f5a0 con 0x7f3a6c1551e0 2023-11-06T03:15:51.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6bfff640 1 --2- 172.21.15.157:0/175070872 >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 0x7f3a6c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:51.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c1551e0 0x7f3a6c06fda0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cc5d0 tx=0x7f3a5c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699168916c8c6317 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6a7fc640 1 -- 172.21.15.157:0/175070872 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0ab620 con 0x7f3a6c1551e0 2023-11-06T03:15:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6a7fc640 1 -- 172.21.15.157:0/175070872 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c0ab780 con 0x7f3a6c1551e0 2023-11-06T03:15:51.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a6a7fc640 1 -- 172.21.15.157:0/175070872 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0abb10 con 0x7f3a6c1551e0 2023-11-06T03:15:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 -- 172.21.15.157:0/175070872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c1551e0 msgr2=0x7f3a6c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c1551e0 0x7f3a6c06fda0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0cc5d0 tx=0x7f3a5c01c440 comp rx=0 tx=0).stop 2023-11-06T03:15:51.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 -- 172.21.15.157:0/175070872 shutdown_connections 2023-11-06T03:15:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/175070872 >> v2:172.21.15.195:3300/0 conn(0x7f3a6c15ea60 0x7f3a6c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c15e5e0 0x7f3a6c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/175070872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c1551e0 0x7f3a6c06fda0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.783+0000 7f3a71cd9640 1 -- 172.21.15.157:0/175070872 >> 172.21.15.157:0/175070872 conn(0x7f3a6c065930 msgr2=0x7f3a6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- 172.21.15.157:0/175070872 shutdown_connections 2023-11-06T03:15:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- 172.21.15.157:0/175070872 wait complete. 2023-11-06T03:15:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 Processor -- start 2023-11-06T03:15:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- start start 2023-11-06T03:15:51.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c1551e0 0x7f3a6c1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a6c15ea60 0x7f3a6c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ce380 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0cdae0 con 0x7f3a6c15ea60 2023-11-06T03:15:51.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0cddc0 con 0x7f3a6c1551e0 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37322/0 (socket says 172.21.15.157:37322) 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 -- 172.21.15.157:0/1934464882 learned_addr learned my addr 172.21.15.157:0/1934464882 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c1551e0 0x7f3a6c1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 -- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c1551e0 msgr2=0x7f3a6c1420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6bfff640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a6c15ea60 0x7f3a6c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c1551e0 0x7f3a6c1420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 -- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a6c15ea60 msgr2=0x7f3a6c142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a6c15ea60 0x7f3a6c142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 -- 172.21.15.157:0/1934464882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0094d0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a6affd640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e8b0 tx=0x7f3a6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000c940 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6000caa0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c0ce0a0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000ce30 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c143b40 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.787+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c1523e0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.791+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3a60013470 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.791+0000 7f3a53fff640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 0x7f3a58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:51.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.791+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f3a60083480 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.791+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 0x7f3a58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:51.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.791+0000 7f3a6b7fe640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 0x7f3a58067f20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0af000 tx=0x7f3a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:51.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.795+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3a6c1523e0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.967+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a6c1523e0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.967+0000 7f3a53fff640 1 -- 172.21.15.157:0/1934464882 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f3a6c1523e0 con 0x7f3a6c15e5e0 2023-11-06T03:15:51.967 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:51.967 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:15:51.968 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:15:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 msgr2=0x7f3a58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 0x7f3a58067f20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0af000 tx=0x7f3a5c008040 comp rx=0 tx=0).stop 2023-11-06T03:15:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 msgr2=0x7f3a6c142870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e8b0 tx=0x7f3a6000ed70 comp rx=0 tx=0).stop 2023-11-06T03:15:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 shutdown_connections 2023-11-06T03:15:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3a58065a60 0x7f3a58067f20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a6c15ea60 0x7f3a6c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c15e5e0 0x7f3a6c142870 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 --2- 172.21.15.157:0/1934464882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a6c1551e0 0x7f3a6c1420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:51.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 >> 172.21.15.157:0/1934464882 conn(0x7f3a6c065930 msgr2=0x7f3a6c14e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:51.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 shutdown_connections 2023-11-06T03:15:51.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:51.971+0000 7f3a71cd9640 1 -- 172.21.15.157:0/1934464882 wait complete. 2023-11-06T03:15:52.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:15:47.843550+0000', 'modified': '2023-11-06T03:15:49.858188+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9918': {'gid': 9918, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1485464139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1485464139}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1485464139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:15:52.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:15:52.388 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:15:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 Processor -- start 2023-11-06T03:15:52.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 -- start start 2023-11-06T03:15:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a41599c0 0x7f03a40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03a406d000 0x7f03a40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a415e440 con 0x7f03a415dea0 2023-11-06T03:15:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a415adb0 con 0x7f03a406d000 2023-11-06T03:15:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a92a5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a406c240 con 0x7f03a41599c0 2023-11-06T03:15:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f03a415dea0 0x7f03a4070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44742/0 (socket says 172.21.15.157:44742) 2023-11-06T03:15:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a37fe640 1 -- 172.21.15.157:0/3494747109 learned_addr learned my addr 172.21.15.157:0/3494747109 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a27fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03a406d000 0x7f03a40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a41599c0 0x7f03a40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a1ffb640 1 -- 172.21.15.157:0/3494747109 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764803332 0 0) 0x7f03a415e440 con 0x7f03a415dea0 2023-11-06T03:15:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a1ffb640 1 -- 172.21.15.157:0/3494747109 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c003530 con 0x7f03a415dea0 2023-11-06T03:15:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a27fc640 1 -- 172.21.15.157:0/3494747109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a41599c0 msgr2=0x7f03a40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a27fc640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a41599c0 0x7f03a40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a27fc640 1 -- 172.21.15.157:0/3494747109 >> v1:172.21.15.157:6789/0 conn(0x7f03a415dea0 legacy=0x7f03a4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.527+0000 7f03a27fc640 1 -- 172.21.15.157:0/3494747109 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a406c520 con 0x7f03a406d000 2023-11-06T03:15:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a27fc640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a406d000 0x7f03a40b23e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f03980097c0 tx=0x7f039801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c704fa9593ae4c59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a1ffb640 1 -- 172.21.15.157:0/3494747109 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980ad660 con 0x7f03a406d000 2023-11-06T03:15:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a1ffb640 1 -- 172.21.15.157:0/3494747109 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03980ad7c0 con 0x7f03a406d000 2023-11-06T03:15:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a1ffb640 1 -- 172.21.15.157:0/3494747109 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980adb50 con 0x7f03a406d000 2023-11-06T03:15:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- 172.21.15.157:0/3494747109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a406d000 msgr2=0x7f03a40b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a406d000 0x7f03a40b23e0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f03980097c0 tx=0x7f039801c450 comp rx=0 tx=0).stop 2023-11-06T03:15:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- 172.21.15.157:0/3494747109 shutdown_connections 2023-11-06T03:15:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a406d000 0x7f03a40b23e0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- 172.21.15.157:0/3494747109 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a41599c0 0x7f03a40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- 172.21.15.157:0/3494747109 >> 172.21.15.157:0/3494747109 conn(0x7f03a4065930 msgr2=0x7f03a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- 172.21.15.157:0/3494747109 shutdown_connections 2023-11-06T03:15:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- 172.21.15.157:0/3494747109 wait complete. 2023-11-06T03:15:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 Processor -- start 2023-11-06T03:15:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- start start 2023-11-06T03:15:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 0x7f03a41f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a415dea0 0x7f03a414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a414ee40 con 0x7f03a41599c0 2023-11-06T03:15:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41f6570 con 0x7f03a406d000 2023-11-06T03:15:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a92a5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a414e830 con 0x7f03a415dea0 2023-11-06T03:15:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37334/0 (socket says 172.21.15.157:37334) 2023-11-06T03:15:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.531+0000 7f03a27fc640 1 -- 172.21.15.157:0/41424829 learned_addr learned my addr 172.21.15.157:0/41424829 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 0x7f03a41f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a37fe640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a415dea0 0x7f03a414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 -- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a415dea0 msgr2=0x7f03a414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a415dea0 0x7f03a414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 -- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 msgr2=0x7f03a41f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 0x7f03a41f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 -- 172.21.15.157:0/41424829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009420 con 0x7f03a41599c0 2023-11-06T03:15:52.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 0x7f03a41f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:52.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a27fc640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f03980b0040 tx=0x7f03980191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 -- 172.21.15.157:0/41424829 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980b6610 con 0x7f03a41599c0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 -- 172.21.15.157:0/41424829 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03980b6770 con 0x7f03a41599c0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40cd210 con 0x7f03a41599c0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 -- 172.21.15.157:0/41424829 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980b6b20 con 0x7f03a41599c0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40cd4f0 con 0x7f03a41599c0 2023-11-06T03:15:52.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 -- 172.21.15.157:0/41424829 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f03980b4040 con 0x7f03a41599c0 2023-11-06T03:15:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 0x7f0390067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f038b7fe640 1 -- 172.21.15.157:0/41424829 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f039811d8c0 con 0x7f03a41599c0 2023-11-06T03:15:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 0x7f0390067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03ac3cd000 1 Processor -- start 2023-11-06T03:15:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03ac3cd000 1 -- start start 2023-11-06T03:15:52.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.535+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 0x7f0390067f20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0394009840 tx=0x7f0394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df456550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df456e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df31efc0 con 0x5614df4a06e0 2023-11-06T03:15:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df2ad1a0 con 0x5614df456a90 2023-11-06T03:15:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614df2aa770 con 0x5614df4a1000 2023-11-06T03:15:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37350/0 (socket says 172.21.15.157:37350) 2023-11-06T03:15:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3538382720 learned_addr learned my addr 172.21.15.157:0/3538382720 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df456550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 msgr2=0x5614df456550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df456550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 msgr2=0x5614df456e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a37fe640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df456e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df456e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3538382720 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614defa2890 con 0x5614df4a06e0 2023-11-06T03:15:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df456550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0394004380 tx=0x7f0394011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799b0451fa06faaf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f0388ff9640 1 -- 172.21.15.157:0/3538382720 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039401d070 con 0x5614df4a06e0 2023-11-06T03:15:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f0388ff9640 1 -- 172.21.15.157:0/3538382720 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03940044f0 con 0x5614df4a06e0 2023-11-06T03:15:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f0388ff9640 1 -- 172.21.15.157:0/3538382720 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394003840 con 0x5614df4a06e0 2023-11-06T03:15:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3538382720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 msgr2=0x5614df4a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0394004380 tx=0x7f0394011000 comp rx=0 tx=0).stop 2023-11-06T03:15:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3538382720 shutdown_connections 2023-11-06T03:15:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df456e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df456550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3538382720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a0ac0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3538382720 >> 172.21.15.157:0/3538382720 conn(0x5614df45cbf0 msgr2=0x5614df454e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3538382720 shutdown_connections 2023-11-06T03:15:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3538382720 wait complete. 2023-11-06T03:15:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- start start 2023-11-06T03:15:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 Processor -- start 2023-11-06T03:15:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 0x5614df4a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df2a6060 con 0x5614df4a06e0 2023-11-06T03:15:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df245290 con 0x5614df4a1000 2023-11-06T03:15:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614df2636f0 con 0x5614df456a90 2023-11-06T03:15:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03ac3cd000 10 client.0 _ll_register_callbacks cb 0x5614df4421d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37362/0 (socket says 172.21.15.157:37362) 2023-11-06T03:15:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 -- 172.21.15.157:0/251430160 learned_addr learned my addr 172.21.15.157:0/251430160 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a37fe640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 0x5614df4a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 -- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 msgr2=0x5614df4a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 0x5614df4a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 -- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 msgr2=0x5614df4616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 -- 172.21.15.157:0/251430160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0394004030 con 0x5614df4a06e0 2023-11-06T03:15:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 0x5614df4a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f03a27fc640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f039811bc60 tx=0x7f03980ef5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d93f76da48a5ba server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.539+0000 7f0381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980f6c00 con 0x5614df4a06e0 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f03980f2ce0 con 0x5614df4a06e0 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614df2a49b0 con 0x5614df4a06e0 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03980f6e90 con 0x5614df4a06e0 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 10 client.10065 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x5614df26b1e0 con 0x5614df4a06e0 2023-11-06T03:15:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614df2a49b0 con 0x5614df4a06e0 2023-11-06T03:15:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f039810b0f0 con 0x5614df4a06e0 2023-11-06T03:15:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 10 client.10065 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 10 client.10065 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:15:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 20 client.10065 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4623e0 0x5614df4627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 4 client.10065 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614df2d54d0 con 0x5614df4623e0 2023-11-06T03:15:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4623e0 0x5614df4627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 10 client.10065 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4623e0 0x5614df4627c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f99b4b449e80256 server_cookie=e5de16b797d63577 in_seq=0 out_seq=0 2023-11-06T03:15:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 1 -- 172.21.15.157:0/251430160 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5614df2d54d0 con 0x5614df4623e0 2023-11-06T03:15:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 10 client.10065 handle_command_reply: tid=0 2023-11-06T03:15:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 client.10065 shutdown 2023-11-06T03:15:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 msgr2=0x5614df4a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f039811bc60 tx=0x7f03980ef5b0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 shutdown_connections 2023-11-06T03:15:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4623e0 0x5614df4627c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a7d70 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f0381ffb640 0 client.10065 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/251430160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df456a90 0x5614df4a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 >> 172.21.15.157:0/251430160 conn(0x5614df45cbf0 msgr2=0x5614df455530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 shutdown_connections 2023-11-06T03:15:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 1 -- 172.21.15.157:0/251430160 wait complete. 2023-11-06T03:15:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.543+0000 7f03ac3cd000 20 client.10065 trim_cache size 0 max 16384 2023-11-06T03:15:52.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 Processor -- start 2023-11-06T03:15:52.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- start start 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a06e0 0x5614df4f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4f8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614def63940 con 0x5614df456a90 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df2a19d0 con 0x5614df4a1000 2023-11-06T03:15:52.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614df273bb0 con 0x5614df4a06e0 2023-11-06T03:15:52.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37372/0 (socket says 172.21.15.157:37372) 2023-11-06T03:15:52.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3270009486 learned_addr learned my addr 172.21.15.157:0/3270009486 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a06e0 msgr2=0x5614df4f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a06e0 0x5614df4f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 msgr2=0x5614df4f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 -- 172.21.15.157:0/3270009486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614df251330 con 0x5614df456a90 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f039400e540 tx=0x7f0394012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2e55b11141c97a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03897fa640 1 -- 172.21.15.157:0/3270009486 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039401d040 con 0x5614df456a90 2023-11-06T03:15:52.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03897fa640 1 -- 172.21.15.157:0/3270009486 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039402b3d0 con 0x5614df456a90 2023-11-06T03:15:52.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03897fa640 1 -- 172.21.15.157:0/3270009486 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039402b7b0 con 0x5614df456a90 2023-11-06T03:15:52.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3270009486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 msgr2=0x5614df4f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f039400e540 tx=0x7f0394012000 comp rx=0 tx=0).stop 2023-11-06T03:15:52.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3270009486 shutdown_connections 2023-11-06T03:15:52.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df4a1000 0x5614df4f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a06e0 0x5614df4f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/3270009486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df456a90 0x5614df4f7d00 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3270009486 >> 172.21.15.157:0/3270009486 conn(0x5614df45cbf0 msgr2=0x5614df45bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3270009486 shutdown_connections 2023-11-06T03:15:52.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- 172.21.15.157:0/3270009486 wait complete. 2023-11-06T03:15:52.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 -- start start 2023-11-06T03:15:52.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.559+0000 7f03ac3cd000 1 Processor -- start 2023-11-06T03:15:52.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df4a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df4a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df275090 con 0x5614df4a06e0 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614df2a4030 con 0x5614df456a90 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614df269ff0 con 0x5614df4a1000 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 10 client.0 _ll_register_callbacks cb 0x5614df4229e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:52.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37378/0 (socket says 172.21.15.157:37378) 2023-11-06T03:15:52.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 -- 172.21.15.157:0/4162223715 learned_addr learned my addr 172.21.15.157:0/4162223715 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a37fe640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df4a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df4a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 -- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 msgr2=0x5614df4a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df4a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 -- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 msgr2=0x5614df4a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df4a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 -- 172.21.15.157:0/4162223715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0394026040 con 0x5614df4a06e0 2023-11-06T03:15:52.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a37fe640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df4a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:52.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a27fc640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f0398006cf0 tx=0x7f039811bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1e9dff7fb730be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:52.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980eb010 con 0x5614df4a06e0 2023-11-06T03:15:52.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f03980f2ce0 con 0x5614df4a06e0 2023-11-06T03:15:52.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614df275b90 con 0x5614df4a06e0 2023-11-06T03:15:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03980f6e90 con 0x5614df4a06e0 2023-11-06T03:15:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 10 client.10071 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x5614defa2cc0 con 0x5614df4a06e0 2023-11-06T03:15:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614df275b90 con 0x5614df4a06e0 2023-11-06T03:15:52.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f03980ede10 con 0x5614df4a06e0 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 10 client.10071 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 10 client.10071 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 20 client.10071 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4a3d60 0x5614df4a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 4 client.10071 mds_command: new command op to 9918 tid=0[{"prefix": "status"}] 2023-11-06T03:15:52.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5614df29f0e0 con 0x5614df4a3d60 2023-11-06T03:15:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4a3d60 0x5614df4a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 10 client.10071 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a2ffd640 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4a3d60 0x5614df4a4140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1685fe8114223d1 server_cookie=df53ad29dfd001b5 in_seq=0 out_seq=0 2023-11-06T03:15:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 1 -- 172.21.15.157:0/4162223715 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x7f039401a160 con 0x5614df4a3d60 2023-11-06T03:15:52.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 10 client.10071 handle_command_reply: tid=0 2023-11-06T03:15:52.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 client.10071 shutdown 2023-11-06T03:15:52.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 msgr2=0x5614df4a32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f0398006cf0 tx=0x7f039811bb90 comp rx=0 tx=0).stop 2023-11-06T03:15:52.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 shutdown_connections 2023-11-06T03:15:52.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5614df4a3d60 0x5614df4a4140 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5614df4a1000 0x5614df4a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5614df4a06e0 0x5614df4a32e0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 --2- 172.21.15.157:0/4162223715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5614df456a90 0x5614df4a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f0381ffb640 0 client.10071 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 >> 172.21.15.157:0/4162223715 conn(0x5614df45cbf0 msgr2=0x5614df49c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 shutdown_connections 2023-11-06T03:15:52.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 1 -- 172.21.15.157:0/4162223715 wait complete. 2023-11-06T03:15:52.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03ac3cd000 20 client.10071 trim_cache size 0 max 16384 2023-11-06T03:15:52.588 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 9918, 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:15:52.589 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:15:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.7196774039999996, 2023-11-06T03:15:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 111, 2023-11-06T03:15:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 151, 2023-11-06T03:15:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 149, 2023-11-06T03:15:52.590 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 9.1553129720000008 2023-11-06T03:15:52.591 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:52.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 msgr2=0x7f0390067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 0x7f0390067f20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0394009840 tx=0x7f0394008040 comp rx=0 tx=0).stop 2023-11-06T03:15:52.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 msgr2=0x7f03a40cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f03980b0040 tx=0x7f03980191f0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 shutdown_connections 2023-11-06T03:15:52.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0390065a60 0x7f0390067f20 secure :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0394009840 tx=0x7f0394008040 comp rx=0 tx=0).stop 2023-11-06T03:15:52.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03a415dea0 0x7f03a414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03a41599c0 0x7f03a40cc910 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 --2- 172.21.15.157:0/41424829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03a406d000 0x7f03a41f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 >> 172.21.15.157:0/41424829 conn(0x7f03a4065930 msgr2=0x7f03a4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 shutdown_connections 2023-11-06T03:15:52.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.563+0000 7f03a92a5640 1 -- 172.21.15.157:0/41424829 wait complete. 2023-11-06T03:15:52.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:15:52.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.707+0000 7f466859f640 1 Processor -- start 2023-11-06T03:15:52.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.707+0000 7f466859f640 1 -- start start 2023-11-06T03:15:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46601551e0 0x7f466006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f46600b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f466015ea60 0x7f46600b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466015f580 con 0x7f46601551e0 2023-11-06T03:15:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466015efc0 con 0x7f466015ea60 2023-11-06T03:15:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466859f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466015f2a0 con 0x7f466015e5e0 2023-11-06T03:15:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46601551e0 0x7f466006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46601551e0 0x7f466006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37394/0 (socket says 172.21.15.157:37394) 2023-11-06T03:15:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 -- 172.21.15.157:0/1460195430 learned_addr learned my addr 172.21.15.157:0/1460195430 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f4666b3c640 1 --2- 172.21.15.157:0/1460195430 >> v2:172.21.15.195:3300/0 conn(0x7f466015ea60 0x7f46600b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f4665b3a640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f46600b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 -- 172.21.15.157:0/1460195430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 msgr2=0x7f46600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f46600b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 -- 172.21.15.157:0/1460195430 >> v2:172.21.15.195:3300/0 conn(0x7f466015ea60 msgr2=0x7f46600b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 --2- 172.21.15.157:0/1460195430 >> v2:172.21.15.195:3300/0 conn(0x7f466015ea60 0x7f46600b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 -- 172.21.15.157:0/1460195430 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466006f5a0 con 0x7f46601551e0 2023-11-06T03:15:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f466633b640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f466006fda0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f46600cc5d0 tx=0x7f465c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49f8350d6512ee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f4665339640 1 -- 172.21.15.157:0/1460195430 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0ab620 con 0x7f46601551e0 2023-11-06T03:15:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f4665339640 1 -- 172.21.15.157:0/1460195430 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0ab780 con 0x7f46601551e0 2023-11-06T03:15:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.711+0000 7f4665339640 1 -- 172.21.15.157:0/1460195430 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0abb10 con 0x7f46601551e0 2023-11-06T03:15:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1460195430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 msgr2=0x7f466006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f466006fda0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f46600cc5d0 tx=0x7f465c01c440 comp rx=0 tx=0).stop 2023-11-06T03:15:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1460195430 shutdown_connections 2023-11-06T03:15:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- 172.21.15.157:0/1460195430 >> v2:172.21.15.195:3300/0 conn(0x7f466015ea60 0x7f46600b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f46600b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- 172.21.15.157:0/1460195430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f466006fda0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1460195430 >> 172.21.15.157:0/1460195430 conn(0x7f4660065930 msgr2=0x7f4660065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1460195430 shutdown_connections 2023-11-06T03:15:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1460195430 wait complete. 2023-11-06T03:15:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 Processor -- start 2023-11-06T03:15:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- start start 2023-11-06T03:15:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f4660142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f466015ea60 0x7f4660142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600ce380 con 0x7f46601551e0 2023-11-06T03:15:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600cdae0 con 0x7f466015ea60 2023-11-06T03:15:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600cddc0 con 0x7f466015e5e0 2023-11-06T03:15:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37400/0 (socket says 172.21.15.157:37400) 2023-11-06T03:15:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 -- 172.21.15.157:0/1807439166 learned_addr learned my addr 172.21.15.157:0/1807439166 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f4665b3a640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f4660142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f4666b3c640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f466015ea60 0x7f4660142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 -- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 msgr2=0x7f4660142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f4660142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 -- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f466015ea60 msgr2=0x7f4660142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f466015ea60 0x7f4660142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 -- 172.21.15.157:0/1807439166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0094d0 con 0x7f46601551e0 2023-11-06T03:15:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466633b640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f465c01aaf0 tx=0x7f465c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0abe80 con 0x7f46601551e0 2023-11-06T03:15:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0be3f0 con 0x7f46601551e0 2023-11-06T03:15:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46600ce0a0 con 0x7f46601551e0 2023-11-06T03:15:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0be780 con 0x7f46601551e0 2023-11-06T03:15:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.715+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4660143b40 con 0x7f46601551e0 2023-11-06T03:15:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46601523e0 con 0x7f46601551e0 2023-11-06T03:15:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f465c0be8e0 con 0x7f46601551e0 2023-11-06T03:15:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f4656ffd640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 0x7f4644067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f465c11da10 con 0x7f46601551e0 2023-11-06T03:15:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f4665b3a640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 0x7f4644067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.719+0000 7f4665b3a640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 0x7f4644067f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f46601442d0 tx=0x7f4650008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.723+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f46601523e0 con 0x7f46601551e0 2023-11-06T03:15:52.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46601527c0 con 0x7f46601551e0 2023-11-06T03:15:52.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f4656ffd640 1 -- 172.21.15.157:0/1807439166 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46601527c0 con 0x7f46601551e0 2023-11-06T03:15:52.897 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:15:52.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 msgr2=0x7f4644067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 0x7f4644067f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f46601442d0 tx=0x7f4650008040 comp rx=0 tx=0).stop 2023-11-06T03:15:52.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 msgr2=0x7f46601420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:52.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f465c01aaf0 tx=0x7f465c01ac80 comp rx=0 tx=0).stop 2023-11-06T03:15:52.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.899+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 shutdown_connections 2023-11-06T03:15:52.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4644065a60 0x7f4644067f20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f466015ea60 0x7f4660142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f466015e5e0 0x7f4660142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 --2- 172.21.15.157:0/1807439166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46601551e0 0x7f46601420b0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:52.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 >> 172.21.15.157:0/1807439166 conn(0x7f4660065930 msgr2=0x7f466014e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:52.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 shutdown_connections 2023-11-06T03:15:52.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:52.903+0000 7f466859f640 1 -- 172.21.15.157:0/1807439166 wait complete. 2023-11-06T03:15:52.914 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:15:52.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:15:52.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:15:52.915 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:15:52.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:52.916 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:15:52.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:15:52.917 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:15:52.917 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:15:52.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:52.918 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:15:52.925 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:15:52.925 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:15:52.925 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:15:52.926 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.926 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:15:52.926 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.926 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:15:52.926 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.927 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:15:52.927 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:15:52.927 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.927 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:15:52.927 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:15:52.928 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.928 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:15:52.928 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.928 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:15:52.928 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41786sec preferred_lft 41786sec 2023-11-06T03:15:52.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:15:52.929 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.929 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:15:52.929 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.929 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:15:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.930 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:15:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:15:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:15:52.930 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:15:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.931 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:15:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:15:52.931 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:15:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.932 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:15:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:15:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:15:52.932 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:15:52.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:52.933 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:15:52.977 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:15:52.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:52.979 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:15:53.031 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:15:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:15:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:15:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:15:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:15:53.032 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:15:53.033 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:15:53.033 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:15:53.033 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:15:53.033 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:15:53.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:53.034 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:15:53.035 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:15:53.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:15:53.035 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:15:53.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:15:53.036 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:15:53.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:53.144 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:15:53.157 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:15:53.158 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:15:53.158 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:15:53.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:53.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:15:53.211 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:15:53.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:15:53.212 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:53.259 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:15:53.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:15:53.261 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:53.308 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:15:53.366 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:15:53.366 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:15:53.418 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:15:53.419 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:15:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:53.420 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:15:53.468 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:15:53.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:15:53.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:15:53.531 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:15:53.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:15:53.586 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:15:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:15:53.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:53.588 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:15:53.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:15:53.599+0000 7fa3bbc83580 -1 init, newargv = 0x55c16ff2f6b0 newargc=15 2023-11-06T03:15:53.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[93757]: starting ceph client 2023-11-06T03:15:53.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[93757]: starting fuse 2023-11-06T03:15:54.704 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:15:54.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:15:54.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:15:54.779 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:15:54.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:15:54.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:54.781 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:15:54.828 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:15:54.828 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:15:54.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:15:54.829 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:15:54.829 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:15:54.830 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:15:54.830 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:15:54.831 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:15:54.831 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:15:54.831 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:15:54.832 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:15:54.832 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:15:54.832 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:15:54.833 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:15:54.833 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:15:54.833 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:15:54.834 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:15:54.834 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:15:54.835 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:15:54.835 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:15:54.835 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:15:54.836 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:15:54.836 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:15:54.836 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:15:54.837 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:15:54.837 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:15:54.837 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:15:54.838 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:15:54.838 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:15:54.947 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.93757.asok 2023-11-06T03:15:55.130 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93757.asok 2023-11-06T03:15:55.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:55.131 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93757.asok status 2023-11-06T03:15:55.283 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:55.284 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:15:55.284 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:15:55.285 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:15:55.285 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:15:55.285 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:15:55.285 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:15:55.286 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "93757", 2023-11-06T03:15:55.286 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:15:55.286 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:55.286 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:15:55.287 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:15:55.287 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 10086, 2023-11-06T03:15:55.287 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:15:55.287 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:15:55.287 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:15:55.288 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 10086 2023-11-06T03:15:55.288 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:55.288 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:15:55.288 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:15:55.288 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:15:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3569496078 2023-11-06T03:15:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:15:55.289 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:15:55.290 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:15:55.290 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3569496078 2023-11-06T03:15:55.290 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:15:55.290 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.10086 192.168.0.1:0/3569496078", 2023-11-06T03:15:55.290 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3569496078", 2023-11-06T03:15:55.291 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:15:55.291 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 111, 2023-11-06T03:15:55.291 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 151, 2023-11-06T03:15:55.292 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:15:55.292 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:15:55.292 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:15:55.292 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:55.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:55.298 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:15:55.299 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:55.305 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:15:55.306 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:55.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:15:55.307 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:15:55.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 damage ls 2023-11-06T03:15:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 Processor -- start 2023-11-06T03:15:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 -- start start 2023-11-06T03:15:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7adc1458d0 0x7f7adc1611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc0ac8b0 0x7f7adc065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc145cb0 con 0x7f7adc065e70 2023-11-06T03:15:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0662e0 con 0x7f7adc1458d0 2023-11-06T03:15:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ae0a01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0ada30 con 0x7f7adc0ac8b0 2023-11-06T03:15:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ada59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7adc1458d0 0x7f7adc1611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ad9d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc0ac8b0 0x7f7adc065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ada59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7adc1458d0 0x7f7adc1611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:46734/0 (socket says 172.21.15.157:46734) 2023-11-06T03:15:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7ada59c640 1 -- 172.21.15.157:0/1717325792 learned_addr learned my addr 172.21.15.157:0/1717325792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.543+0000 7f7adad9d640 1 --1- 172.21.15.157:0/1717325792 >> v1:172.21.15.195:6789/0 conn(0x7f7adc065e70 0x7f7adc070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:45224/0 (socket says 172.21.15.157:45224) 2023-11-06T03:15:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ada59c640 1 -- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc0ac8b0 msgr2=0x7f7adc065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ada59c640 1 --2- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc0ac8b0 0x7f7adc065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ada59c640 1 -- 172.21.15.157:0/1717325792 >> v1:172.21.15.195:6789/0 conn(0x7f7adc065e70 legacy=0x7f7adc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ada59c640 1 -- 172.21.15.157:0/1717325792 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc161700 con 0x7f7adc1458d0 2023-11-06T03:15:55.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ada59c640 1 --2- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc1458d0 0x7f7adc1611c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009870 tx=0x7f7acc01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda8eb31e331f5e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ad959a640 1 -- 172.21.15.157:0/1717325792 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0a9400 con 0x7f7adc1458d0 2023-11-06T03:15:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ad959a640 1 -- 172.21.15.157:0/1717325792 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc0a9560 con 0x7f7adc1458d0 2023-11-06T03:15:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ad959a640 1 -- 172.21.15.157:0/1717325792 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0a98f0 con 0x7f7adc1458d0 2023-11-06T03:15:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 -- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc1458d0 msgr2=0x7f7adc1611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc1458d0 0x7f7adc1611c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009870 tx=0x7f7acc01c540 comp rx=0 tx=0).stop 2023-11-06T03:15:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 -- 172.21.15.157:0/1717325792 shutdown_connections 2023-11-06T03:15:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc0ac8b0 0x7f7adc065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/1717325792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc1458d0 0x7f7adc1611c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 -- 172.21.15.157:0/1717325792 >> 172.21.15.157:0/1717325792 conn(0x7f7adc0732f0 msgr2=0x7f7adc069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 -- 172.21.15.157:0/1717325792 shutdown_connections 2023-11-06T03:15:55.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.547+0000 7f7ae0a01640 1 -- 172.21.15.157:0/1717325792 wait complete. 2023-11-06T03:15:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 Processor -- start 2023-11-06T03:15:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- start start 2023-11-06T03:15:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc065e70 0x7f7adc15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 0x7f7adc13df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc13f2a0 con 0x7f7adc1458d0 2023-11-06T03:15:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc13e480 con 0x7f7adc0ac8b0 2023-11-06T03:15:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc13e720 con 0x7f7adc065e70 2023-11-06T03:15:55.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37404/0 (socket says 172.21.15.157:37404) 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 -- 172.21.15.157:0/190617873 learned_addr learned my addr 172.21.15.157:0/190617873 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ada59c640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc065e70 0x7f7adc15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 0x7f7adc13df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 -- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc065e70 msgr2=0x7f7adc15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc065e70 0x7f7adc15c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 -- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 msgr2=0x7f7adc13df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 0x7f7adc13df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 -- 172.21.15.157:0/190617873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0094d0 con 0x7f7adc1458d0 2023-11-06T03:15:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 0x7f7adc13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:15:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7adad9d640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c410 tx=0x7f7ad000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7acaffd640 1 -- 172.21.15.157:0/190617873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00074e0 con 0x7f7adc1458d0 2023-11-06T03:15:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7acaffd640 1 -- 172.21.15.157:0/190617873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0007640 con 0x7f7adc1458d0 2023-11-06T03:15:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc13ea00 con 0x7f7adc1458d0 2023-11-06T03:15:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7acaffd640 1 -- 172.21.15.157:0/190617873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0004a10 con 0x7f7adc1458d0 2023-11-06T03:15:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.551+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc13efc0 con 0x7f7adc1458d0 2023-11-06T03:15:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7acaffd640 1 -- 172.21.15.157:0/190617873 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7ad0004b70 con 0x7f7adc1458d0 2023-11-06T03:15:55.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7acaffd640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 0x7f7abc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7acaffd640 1 -- 172.21.15.157:0/190617873 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7ad0082c60 con 0x7f7adc1458d0 2023-11-06T03:15:55.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ada59c640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 0x7f7abc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 Processor -- start 2023-11-06T03:15:55.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- start start 2023-11-06T03:15:55.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ada59c640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 0x7f7abc067f20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0ae000 tx=0x7f7acc0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997cce5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce6530 0x55997cc9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997cb63d60 con 0x55997cc9bf90 2023-11-06T03:15:55.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997caf23b0 con 0x55997cce6530 2023-11-06T03:15:55.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997caef980 con 0x55997cce5c10 2023-11-06T03:15:55.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37416/0 (socket says 172.21.15.157:37416) 2023-11-06T03:15:55.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 -- 172.21.15.157:0/1292162377 learned_addr learned my addr 172.21.15.157:0/1292162377 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ada59c640 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997cce5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce6530 0x55997cc9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 -- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 msgr2=0x55997cce5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997cce5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 -- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce6530 msgr2=0x55997cc9ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce6530 0x55997cc9ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 -- 172.21.15.157:0/1292162377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ca8c110 con 0x55997cc9bf90 2023-11-06T03:15:55.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7adad9d640 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0081060 tx=0x7f7ad0054970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb41f4cae18e51c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7abbfff640 1 -- 172.21.15.157:0/1292162377 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad005c190 con 0x55997cc9bf90 2023-11-06T03:15:55.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7abbfff640 1 -- 172.21.15.157:0/1292162377 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad00522f0 con 0x55997cc9bf90 2023-11-06T03:15:55.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7abbfff640 1 -- 172.21.15.157:0/1292162377 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00508c0 con 0x55997cc9bf90 2023-11-06T03:15:55.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- 172.21.15.157:0/1292162377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 msgr2=0x55997cc9c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0081060 tx=0x7f7ad0054970 comp rx=0 tx=0).stop 2023-11-06T03:15:55.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- 172.21.15.157:0/1292162377 shutdown_connections 2023-11-06T03:15:55.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cc9c370 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce6530 0x55997cc9ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/1292162377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997cce5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- 172.21.15.157:0/1292162377 >> 172.21.15.157:0/1292162377 conn(0x55997cc97f40 msgr2=0x55997cc9a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- 172.21.15.157:0/1292162377 shutdown_connections 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.555+0000 7f7ae3b29000 1 -- 172.21.15.157:0/1292162377 wait complete. 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- start start 2023-11-06T03:15:55.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 Processor -- start 2023-11-06T03:15:55.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997cca12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce6530 0x55997ccd3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997caeb270 con 0x55997cc9bf90 2023-11-06T03:15:55.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ca8a5a0 con 0x55997cce5c10 2023-11-06T03:15:55.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997caa87c0 con 0x55997cce6530 2023-11-06T03:15:55.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 10 client.0 _ll_register_callbacks cb 0x55997cadf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:55.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37418/0 (socket says 172.21.15.157:37418) 2023-11-06T03:15:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 -- 172.21.15.157:0/4060297485 learned_addr learned my addr 172.21.15.157:0/4060297485 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7adad9d640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce6530 0x55997ccd3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997cca12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 -- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce6530 msgr2=0x55997ccd3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce6530 0x55997ccd3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 -- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 msgr2=0x55997cca12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997cca12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 -- 172.21.15.157:0/4060297485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc13efc0 con 0x55997cc9bf90 2023-11-06T03:15:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ada59c640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7acc01a170 tx=0x7f7acc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415d31b795628f30 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0c1040 con 0x55997cc9bf90 2023-11-06T03:15:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7acc0bcc60 con 0x55997cc9bf90 2023-11-06T03:15:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55997caaf350 con 0x55997cc9bf90 2023-11-06T03:15:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7acc0b6040 con 0x55997cc9bf90 2023-11-06T03:15:55.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 10 client.10095 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:55.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55997c7e7e20 con 0x55997cc9bf90 2023-11-06T03:15:55.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997caaf350 con 0x55997cc9bf90 2023-11-06T03:15:55.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7acc0cf3f0 con 0x55997cc9bf90 2023-11-06T03:15:55.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 10 client.10095 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:55.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 10 client.10095 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 20 client.10095 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997ccd38f0 0x55997ccd48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 4 client.10095 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55997cae9bc0 con 0x55997ccd38f0 2023-11-06T03:15:55.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997ccd38f0 0x55997ccd48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 10 client.10095 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997ccd38f0 0x55997ccd48f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2fa2428fcfff9e server_cookie=85632242c9324316 in_seq=0 out_seq=0 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 1 -- 172.21.15.157:0/4060297485 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55997cae9bc0 con 0x55997ccd38f0 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 10 client.10095 handle_command_reply: tid=0 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 client.10095 shutdown 2023-11-06T03:15:55.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 msgr2=0x55997cca0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7acc01a170 tx=0x7f7acc01ad80 comp rx=0 tx=0).stop 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 shutdown_connections 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997ccd38f0 0x55997ccd48f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce6530 0x55997ccd3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997cca12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ab97fa640 0 client.10095 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/4060297485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc9bf90 0x55997cca0d60 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.559+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 >> 172.21.15.157:0/4060297485 conn(0x55997cc97f40 msgr2=0x55997cc9abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.563+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 shutdown_connections 2023-11-06T03:15:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.563+0000 7f7ae3b29000 1 -- 172.21.15.157:0/4060297485 wait complete. 2023-11-06T03:15:55.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.563+0000 7f7ae3b29000 20 client.10095 trim_cache size 0 max 16384 2023-11-06T03:15:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 Processor -- start 2023-11-06T03:15:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 -- start start 2023-11-06T03:15:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cc9bf90 0x55997cc9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997ccf5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997c7a8aa0 con 0x55997cc97f40 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997caeddd0 con 0x55997cc9bf90 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997cca05b0 con 0x55997cce5c10 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37434/0 (socket says 172.21.15.157:37434) 2023-11-06T03:15:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.571+0000 7f7ada59c640 1 -- 172.21.15.157:0/2205526406 learned_addr learned my addr 172.21.15.157:0/2205526406 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 msgr2=0x55997ccf5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997ccf5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cc9bf90 0x55997cc9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cc9bf90 msgr2=0x55997cc9b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cc9bf90 0x55997cc9b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/2205526406 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997cab01a0 con 0x55997cc97f40 2023-11-06T03:15:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c10 tx=0x7f7acc0c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d985eec469e901ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ac8ff9640 1 -- 172.21.15.157:0/2205526406 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0c1040 con 0x55997cc97f40 2023-11-06T03:15:55.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ac8ff9640 1 -- 172.21.15.157:0/2205526406 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc009240 con 0x55997cc97f40 2023-11-06T03:15:55.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ac8ff9640 1 -- 172.21.15.157:0/2205526406 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0bcc80 con 0x55997cc97f40 2023-11-06T03:15:55.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/2205526406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 msgr2=0x55997cc9b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c10 tx=0x7f7acc0c4000 comp rx=0 tx=0).stop 2023-11-06T03:15:55.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/2205526406 shutdown_connections 2023-11-06T03:15:55.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cce5c10 0x55997ccf5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cc9bf90 0x55997cc9b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/2205526406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cc9b340 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/2205526406 >> 172.21.15.157:0/2205526406 conn(0x55997ccd4040 msgr2=0x55997ccd2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/2205526406 shutdown_connections 2023-11-06T03:15:55.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/2205526406 wait complete. 2023-11-06T03:15:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- start start 2023-11-06T03:15:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 Processor -- start 2023-11-06T03:15:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cc9bf90 0x55997ccf9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997ccf9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997cc978e0 con 0x55997cc97f40 2023-11-06T03:15:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997cae6be0 con 0x55997cce5c10 2023-11-06T03:15:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55997cc9f8b0 con 0x55997cc9bf90 2023-11-06T03:15:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 10 client.0 _ll_register_callbacks cb 0x55997ca99780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:55.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37450/0 (socket says 172.21.15.157:37450) 2023-11-06T03:15:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/603880639 learned_addr learned my addr 172.21.15.157:0/603880639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cc9bf90 0x55997ccf9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7adad9d640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997ccf9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cc9bf90 msgr2=0x55997ccf9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cc9bf90 0x55997ccf9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 msgr2=0x55997ccf9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997ccf9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 -- 172.21.15.157:0/603880639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55997c7e7e20 con 0x55997cc97f40 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ada59c640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f7acc098010 tx=0x7f7acc0a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905416b6bff0957d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0c1040 con 0x55997cc97f40 2023-11-06T03:15:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7acc0e2440 con 0x55997cc97f40 2023-11-06T03:15:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55997ca9ac20 con 0x55997cc97f40 2023-11-06T03:15:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7acc0e2620 con 0x55997cc97f40 2023-11-06T03:15:55.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 10 client.10101 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55997ca964f0 con 0x55997cc97f40 2023-11-06T03:15:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997ca9ac20 con 0x55997cc97f40 2023-11-06T03:15:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7acc0e2780 con 0x55997cc97f40 2023-11-06T03:15:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 10 client.10101 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:55.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 10 client.10101 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 20 client.10101 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997cce6530 0x55997ccf9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 4 client.10101 mds_command: new command op to 9918 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:15:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55997cae42f0 con 0x55997cce6530 2023-11-06T03:15:55.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.575+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997cce6530 0x55997ccf9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ab97fa640 10 client.10101 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ad9d9b640 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997cce6530 0x55997ccf9eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1c7575a4566b2f server_cookie=90e2d6e0f589a033 in_seq=0 out_seq=0 2023-11-06T03:15:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ab97fa640 1 -- 172.21.15.157:0/603880639 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55997cae42f0 con 0x55997cce6530 2023-11-06T03:15:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ab97fa640 10 client.10101 handle_command_reply: tid=0 2023-11-06T03:15:55.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 client.10101 shutdown 2023-11-06T03:15:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 msgr2=0x55997cd3f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f7acc098010 tx=0x7f7acc0a9000 comp rx=0 tx=0).stop 2023-11-06T03:15:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 shutdown_connections 2023-11-06T03:15:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55997cce6530 0x55997ccf9eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55997cce5c10 0x55997ccf9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55997cc9bf90 0x55997ccf9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ab97fa640 0 client.10101 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:55.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 --2- 172.21.15.157:0/603880639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55997cc97f40 0x55997cd3f580 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 >> 172.21.15.157:0/603880639 conn(0x55997ccd4040 msgr2=0x55997ccd2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 shutdown_connections 2023-11-06T03:15:55.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 1 -- 172.21.15.157:0/603880639 wait complete. 2023-11-06T03:15:55.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae3b29000 20 client.10101 trim_cache size 0 max 16384 2023-11-06T03:15:55.599 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2023-11-06T03:15:55.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 msgr2=0x7f7abc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 0x7f7abc067f20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0ae000 tx=0x7f7acc0b3040 comp rx=0 tx=0).stop 2023-11-06T03:15:55.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 msgr2=0x7f7adc0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c410 tx=0x7f7ad000c8d0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 shutdown_connections 2023-11-06T03:15:55.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7abc065a60 0x7f7abc067f20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7adc1458d0 0x7f7adc0b0f40 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7adc0ac8b0 0x7f7adc13df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 --2- 172.21.15.157:0/190617873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7adc065e70 0x7f7adc15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 >> 172.21.15.157:0/190617873 conn(0x7f7adc0732f0 msgr2=0x7f7adc142790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 shutdown_connections 2023-11-06T03:15:55.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.579+0000 7f7ae0a01640 1 -- 172.21.15.157:0/190617873 wait complete. 2023-11-06T03:15:55.603 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-06T03:15:55.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 Processor -- start 2023-11-06T03:15:55.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 -- start start 2023-11-06T03:15:55.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015caa0 0x7ffa200cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa2015ce80 0x7ffa200cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa200d2920 con 0x7ffa200ce1a0 2023-11-06T03:15:55.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa200d18a0 con 0x7ffa2015ce80 2023-11-06T03:15:55.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa2487b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200d1b60 con 0x7ffa2015caa0 2023-11-06T03:15:55.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1dd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa2015ce80 0x7ffa200cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1ed9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffa200ce1a0 0x7ffa200d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44750/0 (socket says 172.21.15.157:44750) 2023-11-06T03:15:55.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1ed9d640 1 -- 172.21.15.157:0/3686531534 learned_addr learned my addr 172.21.15.157:0/3686531534 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:55.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1dd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa2015ce80 0x7ffa200cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37466/0 (socket says 172.21.15.157:37466) 2023-11-06T03:15:55.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1e59c640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015caa0 0x7ffa200cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1dd9b640 1 -- 172.21.15.157:0/3686531534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015caa0 msgr2=0x7ffa200cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:55.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.719+0000 7ffa1dd9b640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015caa0 0x7ffa200cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1dd9b640 1 -- 172.21.15.157:0/3686531534 >> v1:172.21.15.157:6789/0 conn(0x7ffa200ce1a0 legacy=0x7ffa200d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1d59a640 1 -- 172.21.15.157:0/3686531534 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750413843 0 0) 0x7ffa200d2920 con 0x7ffa200ce1a0 2023-11-06T03:15:55.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1dd9b640 1 -- 172.21.15.157:0/3686531534 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa200d1e40 con 0x7ffa2015ce80 2023-11-06T03:15:55.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1dd9b640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2015ce80 0x7ffa200cfb70 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7ffa140097c0 tx=0x7ffa1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49e3581cb20161a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1d59a640 1 -- 172.21.15.157:0/3686531534 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa140ad660 con 0x7ffa2015ce80 2023-11-06T03:15:55.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1d59a640 1 -- 172.21.15.157:0/3686531534 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa140ad7c0 con 0x7ffa2015ce80 2023-11-06T03:15:55.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa1d59a640 1 -- 172.21.15.157:0/3686531534 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa140adb50 con 0x7ffa2015ce80 2023-11-06T03:15:55.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 -- 172.21.15.157:0/3686531534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2015ce80 msgr2=0x7ffa200cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2015ce80 0x7ffa200cfb70 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7ffa140097c0 tx=0x7ffa1401c450 comp rx=0 tx=0).stop 2023-11-06T03:15:55.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 -- 172.21.15.157:0/3686531534 shutdown_connections 2023-11-06T03:15:55.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2015ce80 0x7ffa200cfb70 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 --2- 172.21.15.157:0/3686531534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015caa0 0x7ffa200cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 -- 172.21.15.157:0/3686531534 >> 172.21.15.157:0/3686531534 conn(0x7ffa20070790 msgr2=0x7ffa20070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 -- 172.21.15.157:0/3686531534 shutdown_connections 2023-11-06T03:15:55.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.723+0000 7ffa2487b640 1 -- 172.21.15.157:0/3686531534 wait complete. 2023-11-06T03:15:55.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 Processor -- start 2023-11-06T03:15:55.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- start start 2023-11-06T03:15:55.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa2015caa0 0x7ffa200cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015ce80 0x7ffa201566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20156230 con 0x7ffa200ce1a0 2023-11-06T03:15:55.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20151890 con 0x7ffa2015caa0 2023-11-06T03:15:55.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20155620 con 0x7ffa2015ce80 2023-11-06T03:15:55.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37472/0 (socket says 172.21.15.157:37472) 2023-11-06T03:15:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 -- 172.21.15.157:0/565656050 learned_addr learned my addr 172.21.15.157:0/565656050 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:55.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1dd9b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa2015caa0 0x7ffa200cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1ed9d640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015ce80 0x7ffa201566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 -- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015ce80 msgr2=0x7ffa201566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015ce80 0x7ffa201566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 -- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa2015caa0 msgr2=0x7ffa200cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa2015caa0 0x7ffa200cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 -- 172.21.15.157:0/565656050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14009420 con 0x7ffa200ce1a0 2023-11-06T03:15:55.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa1e59c640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9b0 tx=0x7ffa1000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa100096b0 con 0x7ffa200ce1a0 2023-11-06T03:15:55.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1000c9a0 con 0x7ffa200ce1a0 2023-11-06T03:15:55.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa20155900 con 0x7ffa200ce1a0 2023-11-06T03:15:55.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000cd30 con 0x7ffa200ce1a0 2023-11-06T03:15:55.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa20155ec0 con 0x7ffa200ce1a0 2023-11-06T03:15:55.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa20072750 con 0x7ffa200ce1a0 2023-11-06T03:15:55.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffa10013470 con 0x7ffa200ce1a0 2023-11-06T03:15:55.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 0x7ffa00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:55.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.727+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ffa10083650 con 0x7ffa200ce1a0 2023-11-06T03:15:55.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.731+0000 7ffa1dd9b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 0x7ffa00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:55.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.731+0000 7ffa1dd9b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 0x7ffa00067f20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffa140af000 tx=0x7ffa140a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:55.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.735+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffa10050e50 con 0x7ffa200ce1a0 2023-11-06T03:15:55.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.919+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffa200cf180 con 0x7ffa200ce1a0 2023-11-06T03:15:55.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.919+0000 7ffa0effd640 1 -- 172.21.15.157:0/565656050 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffa10054470 con 0x7ffa200ce1a0 2023-11-06T03:15:55.917 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:15:55.917 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T03:15:55.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 msgr2=0x7ffa00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 0x7ffa00067f20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffa140af000 tx=0x7ffa140a7000 comp rx=0 tx=0).stop 2023-11-06T03:15:55.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 msgr2=0x7ffa2014f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:55.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000e9b0 tx=0x7ffa1000ee70 comp rx=0 tx=0).stop 2023-11-06T03:15:55.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 shutdown_connections 2023-11-06T03:15:55.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffa00065a60 0x7ffa00067f20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa2015ce80 0x7ffa201566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa2015caa0 0x7ffa200cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 --2- 172.21.15.157:0/565656050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa200ce1a0 0x7ffa2014f7a0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 >> 172.21.15.157:0/565656050 conn(0x7ffa20070790 msgr2=0x7ffa200d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 shutdown_connections 2023-11-06T03:15:55.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:55.923+0000 7ffa2487b640 1 -- 172.21.15.157:0/565656050 wait complete. 2023-11-06T03:15:55.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 scrub start '~mdsdir' recursive 2023-11-06T03:15:56.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 Processor -- start 2023-11-06T03:15:56.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 -- start start 2023-11-06T03:15:56.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff01c0b6e80 0x7ff01c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01c0d0410 con 0x7ff01c15cc70 2023-11-06T03:15:56.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01c0d27c0 con 0x7ff01c0cfa90 2023-11-06T03:15:56.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff023a90640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01c0d0f90 con 0x7ff01c0b6e80 2023-11-06T03:15:56.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02202d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff01c0b6e80 0x7ff01c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02182c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff01c15cc70 0x7ff01c0d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44766/0 (socket says 172.21.15.157:44766) 2023-11-06T03:15:56.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02202d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff01c0b6e80 0x7ff01c0cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37484/0 (socket says 172.21.15.157:37484) 2023-11-06T03:15:56.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02182c640 1 -- 172.21.15.157:0/134365321 learned_addr learned my addr 172.21.15.157:0/134365321 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:56.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02202d640 1 -- 172.21.15.157:0/134365321 >> v1:172.21.15.195:6789/0 conn(0x7ff01c0cfa90 legacy=0x7ff01c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02202d640 1 -- 172.21.15.157:0/134365321 >> v1:172.21.15.157:6789/0 conn(0x7ff01c15cc70 legacy=0x7ff01c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.047+0000 7ff02202d640 1 -- 172.21.15.157:0/134365321 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c0d1270 con 0x7ff01c0b6e80 2023-11-06T03:15:56.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff02202d640 1 --2- 172.21.15.157:0/134365321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0b6e80 0x7ff01c0cfed0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0097c0 tx=0x7ff00c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4270bfefe6d0547a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff02082a640 1 -- 172.21.15.157:0/134365321 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0ad680 con 0x7ff01c0b6e80 2023-11-06T03:15:56.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff02082a640 1 -- 172.21.15.157:0/134365321 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c0ad7e0 con 0x7ff01c0b6e80 2023-11-06T03:15:56.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff02082a640 1 -- 172.21.15.157:0/134365321 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0adb70 con 0x7ff01c0b6e80 2023-11-06T03:15:56.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- 172.21.15.157:0/134365321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0b6e80 msgr2=0x7ff01c0cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 --2- 172.21.15.157:0/134365321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0b6e80 0x7ff01c0cfed0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0097c0 tx=0x7ff00c01c380 comp rx=0 tx=0).stop 2023-11-06T03:15:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- 172.21.15.157:0/134365321 shutdown_connections 2023-11-06T03:15:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 --2- 172.21.15.157:0/134365321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0b6e80 0x7ff01c0cfed0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- 172.21.15.157:0/134365321 >> 172.21.15.157:0/134365321 conn(0x7ff01c070790 msgr2=0x7ff01c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- 172.21.15.157:0/134365321 shutdown_connections 2023-11-06T03:15:56.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- 172.21.15.157:0/134365321 wait complete. 2023-11-06T03:15:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 Processor -- start 2023-11-06T03:15:56.055 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.051+0000 7ff023a90640 1 -- start start 2023-11-06T03:15:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff01c0b6e80 0x7ff01c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff01c15cc70 0x7ff01c0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c1604d0 con 0x7ff01c0cfa90 2023-11-06T03:15:56.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c15de90 con 0x7ff01c15cc70 2023-11-06T03:15:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c15f270 con 0x7ff01c0b6e80 2023-11-06T03:15:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37498/0 (socket says 172.21.15.157:37498) 2023-11-06T03:15:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 -- 172.21.15.157:0/1955152320 learned_addr learned my addr 172.21.15.157:0/1955152320 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 -- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff01c0b6e80 msgr2=0x7ff01c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff01c0b6e80 0x7ff01c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 -- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff01c15cc70 msgr2=0x7ff01c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff01c15cc70 0x7ff01c0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 -- 172.21.15.157:0/1955152320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009420 con 0x7ff01c0cfa90 2023-11-06T03:15:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02102b640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b410 tx=0x7ff01000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 -- 172.21.15.157:0/1955152320 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010004020 con 0x7ff01c0cfa90 2023-11-06T03:15:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 -- 172.21.15.157:0/1955152320 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010004180 con 0x7ff01c0cfa90 2023-11-06T03:15:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01c15f4f0 con 0x7ff01c0cfa90 2023-11-06T03:15:56.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 -- 172.21.15.157:0/1955152320 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010011630 con 0x7ff01c0cfa90 2023-11-06T03:15:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c0c6000 con 0x7ff01c0cfa90 2023-11-06T03:15:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 -- 172.21.15.157:0/1955152320 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff010011790 con 0x7ff01c0cfa90 2023-11-06T03:15:56.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 0x7ff008067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff01a7fc640 1 -- 172.21.15.157:0/1955152320 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff010083030 con 0x7ff01c0cfa90 2023-11-06T03:15:56.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.055+0000 7ff02182c640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 0x7ff008067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 Processor -- start 2023-11-06T03:15:56.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- start start 2023-11-06T03:15:56.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 0x7ff008067f20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff014002080 tx=0x7ff014001670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818f4d0 0x55e2981447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29800cd10 con 0x55e29818ebb0 2023-11-06T03:15:56.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297f9a2b0 con 0x55e29818f4d0 2023-11-06T03:15:56.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e297f97880 con 0x55e298144d00 2023-11-06T03:15:56.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37512/0 (socket says 172.21.15.157:37512) 2023-11-06T03:15:56.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 -- 172.21.15.157:0/803655719 learned_addr learned my addr 172.21.15.157:0/803655719 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 -- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 msgr2=0x55e2981450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02102b640 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818f4d0 0x55e2981447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 -- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818f4d0 msgr2=0x55e2981447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818f4d0 0x55e2981447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 -- 172.21.15.157:0/803655719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e297c90000 con 0x55e29818ebb0 2023-11-06T03:15:56.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff02182c640 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff014001d10 tx=0x7ff01400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b9bc809d28a4cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff0037fe640 1 -- 172.21.15.157:0/803655719 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01401b070 con 0x55e29818ebb0 2023-11-06T03:15:56.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff0037fe640 1 -- 172.21.15.157:0/803655719 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff014016110 con 0x55e29818ebb0 2023-11-06T03:15:56.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff0037fe640 1 -- 172.21.15.157:0/803655719 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014016ea0 con 0x55e29818ebb0 2023-11-06T03:15:56.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- 172.21.15.157:0/803655719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 msgr2=0x55e29818ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff014001d10 tx=0x7ff01400f000 comp rx=0 tx=0).stop 2023-11-06T03:15:56.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- 172.21.15.157:0/803655719 shutdown_connections 2023-11-06T03:15:56.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818f4d0 0x55e2981447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- 172.21.15.157:0/803655719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e29818ef90 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- 172.21.15.157:0/803655719 >> 172.21.15.157:0/803655719 conn(0x55e29814b280 msgr2=0x55e2981430a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- 172.21.15.157:0/803655719 shutdown_connections 2023-11-06T03:15:56.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- 172.21.15.157:0/803655719 wait complete. 2023-11-06T03:15:56.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- start start 2023-11-06T03:15:56.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 Processor -- start 2023-11-06T03:15:56.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e298195d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 0x55e29814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297f93170 con 0x55e29818ebb0 2023-11-06T03:15:56.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297f33060 con 0x55e298144d00 2023-11-06T03:15:56.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e297f32d00 con 0x55e29818f4d0 2023-11-06T03:15:56.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.059+0000 7ff026bb8000 10 client.0 _ll_register_callbacks cb 0x55e2981306a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:56.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37526/0 (socket says 172.21.15.157:37526) 2023-11-06T03:15:56.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 -- 172.21.15.157:0/4247004200 learned_addr learned my addr 172.21.15.157:0/4247004200 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02182c640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e298195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02202d640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 0x55e29814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 -- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 msgr2=0x55e29814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 0x55e29814fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 -- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 msgr2=0x55e298195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e298195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 -- 172.21.15.157:0/4247004200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c0d0410 con 0x55e29818ebb0 2023-11-06T03:15:56.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02182c640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e298195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:56.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02202d640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 0x55e29814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:56.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02102b640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7ff010050a80 tx=0x7ff010054dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0f1e7fccfba7ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:56.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010058160 con 0x55e29818ebb0 2023-11-06T03:15:56.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff01005c0d0 con 0x55e29818ebb0 2023-11-06T03:15:56.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e297f42aa0 con 0x55e29818ebb0 2023-11-06T03:15:56.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff01005c290 con 0x55e29818ebb0 2023-11-06T03:15:56.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 10 client.10125 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:56.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55e297f58020 con 0x55e29818ebb0 2023-11-06T03:15:56.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e297f42aa0 con 0x55e29818ebb0 2023-11-06T03:15:56.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff01005a160 con 0x55e29818ebb0 2023-11-06T03:15:56.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 10 client.10125 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 10 client.10125 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 20 client.10125 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e2981508e0 0x55e298150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 4 client.10125 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e297f91ac0 con 0x55e2981508e0 2023-11-06T03:15:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02182c640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e2981508e0 0x55e298150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff02182c640 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e2981508e0 0x55e298150cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825912584224a6af server_cookie=337b36d0eeab8385 in_seq=0 out_seq=0 2023-11-06T03:15:56.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 10 client.10125 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 1 -- 172.21.15.157:0/4247004200 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e297f91ac0 con 0x55e2981508e0 2023-11-06T03:15:56.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 10 client.10125 handle_command_reply: tid=0 2023-11-06T03:15:56.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 client.10125 shutdown 2023-11-06T03:15:56.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 msgr2=0x55e298196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7ff010050a80 tx=0x7ff010054dc0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 shutdown_connections 2023-11-06T03:15:56.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e2981508e0 0x55e298150cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29818f4d0 0x55e29814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e298196240 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff000ff9640 0 client.10125 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4247004200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e298195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 >> 172.21.15.157:0/4247004200 conn(0x55e29814b280 msgr2=0x55e2981437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 shutdown_connections 2023-11-06T03:15:56.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 1 -- 172.21.15.157:0/4247004200 wait complete. 2023-11-06T03:15:56.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.063+0000 7ff026bb8000 20 client.10125 trim_cache size 0 max 16384 2023-11-06T03:15:56.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 Processor -- start 2023-11-06T03:15:56.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- start start 2023-11-06T03:15:56.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29814b280 0x55e2981e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e2981e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e298147db0 con 0x55e29818ebb0 2023-11-06T03:15:56.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297c50c80 con 0x55e298144d00 2023-11-06T03:15:56.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e297f97170 con 0x55e29814b280 2023-11-06T03:15:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37536/0 (socket says 172.21.15.157:37536) 2023-11-06T03:15:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 -- 172.21.15.157:0/4042075435 learned_addr learned my addr 172.21.15.157:0/4042075435 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 -- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29814b280 msgr2=0x55e2981e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02102b640 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e2981e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02182c640 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29814b280 0x55e2981e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29814b280 0x55e2981e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 -- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 msgr2=0x55e2981e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e2981e7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 -- 172.21.15.157:0/4042075435 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e297f3e370 con 0x55e29818ebb0 2023-11-06T03:15:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02202d640 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff00c018f90 tx=0x7ff00c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bed47c7be7d6a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff003fff640 1 -- 172.21.15.157:0/4042075435 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0adb50 con 0x55e29818ebb0 2023-11-06T03:15:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff003fff640 1 -- 172.21.15.157:0/4042075435 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c0adcb0 con 0x55e29818ebb0 2023-11-06T03:15:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff003fff640 1 -- 172.21.15.157:0/4042075435 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c0b8640 con 0x55e29818ebb0 2023-11-06T03:15:56.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- 172.21.15.157:0/4042075435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 msgr2=0x55e2981e76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff00c018f90 tx=0x7ff00c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- 172.21.15.157:0/4042075435 shutdown_connections 2023-11-06T03:15:56.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29818ebb0 0x55e2981e76a0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e298144d00 0x55e2981e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- 172.21.15.157:0/4042075435 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e29814b280 0x55e2981e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- 172.21.15.157:0/4042075435 >> 172.21.15.157:0/4042075435 conn(0x55e298149fb0 msgr2=0x55e29818b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- 172.21.15.157:0/4042075435 shutdown_connections 2023-11-06T03:15:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- 172.21.15.157:0/4042075435 wait complete. 2023-11-06T03:15:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- start start 2023-11-06T03:15:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 Processor -- start 2023-11-06T03:15:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818ebb0 0x55e2981cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e298144820 con 0x55e29814b280 2023-11-06T03:15:56.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e297f8eae0 con 0x55e29818ebb0 2023-11-06T03:15:56.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e297f960d0 con 0x55e298144d00 2023-11-06T03:15:56.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff026bb8000 10 client.0 _ll_register_callbacks cb 0x55e297f962a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:56.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02102b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37552/0 (socket says 172.21.15.157:37552) 2023-11-06T03:15:56.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.079+0000 7ff02102b640 1 -- 172.21.15.157:0/120479347 learned_addr learned my addr 172.21.15.157:0/120479347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02202d640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818ebb0 0x55e2981cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02182c640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 -- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 msgr2=0x55e2981ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 -- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818ebb0 msgr2=0x55e2981cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818ebb0 0x55e2981cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 -- 172.21.15.157:0/120479347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c0d1270 con 0x55e29814b280 2023-11-06T03:15:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02102b640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b3e0 tx=0x7ff010081300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65aac98fc8beda6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:56.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010058160 con 0x55e29814b280 2023-11-06T03:15:56.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff0100583b0 con 0x55e29814b280 2023-11-06T03:15:56.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e298148680 con 0x55e29814b280 2023-11-06T03:15:56.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff01000d040 con 0x55e29814b280 2023-11-06T03:15:56.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 10 client.10131 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55e297f320a0 con 0x55e29814b280 2023-11-06T03:15:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e298148680 con 0x55e29814b280 2023-11-06T03:15:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff010078060 con 0x55e29814b280 2023-11-06T03:15:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 10 client.10131 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:56.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 10 client.10131 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 20 client.10131 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e29818f4d0 0x55e2981d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 4 client.10131 mds_command: new command op to 9918 tid=0[{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2023-11-06T03:15:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55e297f504e0 con 0x55e29818f4d0 2023-11-06T03:15:56.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02182c640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e29818f4d0 0x55e2981d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 10 client.10131 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff02182c640 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e29818f4d0 0x55e2981d03a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bc62b062f58494 server_cookie=834ad8c8c09395ac in_seq=0 out_seq=0 2023-11-06T03:15:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 1 -- 172.21.15.157:0/120479347 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e297f504e0 con 0x55e29818f4d0 2023-11-06T03:15:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 10 client.10131 handle_command_reply: tid=0 2023-11-06T03:15:56.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 client.10131 shutdown 2023-11-06T03:15:56.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 msgr2=0x55e2981ec7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b3e0 tx=0x7ff010081300 comp rx=0 tx=0).stop 2023-11-06T03:15:56.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 shutdown_connections 2023-11-06T03:15:56.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e29818f4d0 0x55e2981d03a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e29818ebb0 0x55e2981cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e29814b280 0x55e2981ec7b0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff000ff9640 0 client.10131 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 --2- 172.21.15.157:0/120479347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e298144d00 0x55e2981ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 >> 172.21.15.157:0/120479347 conn(0x55e298149fb0 msgr2=0x55e29818be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 shutdown_connections 2023-11-06T03:15:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 1 -- 172.21.15.157:0/120479347 wait complete. 2023-11-06T03:15:56.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff026bb8000 20 client.10131 trim_cache size 0 max 16384 2023-11-06T03:15:56.118 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:56.118 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:15:56.118 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "185c72d6-9c41-410b-8677-4d3deb0c4c27", 2023-11-06T03:15:56.118 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:15:56.119 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 msgr2=0x7ff008067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 0x7ff008067f20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff014002080 tx=0x7ff014001670 comp rx=0 tx=0).stop 2023-11-06T03:15:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 msgr2=0x7ff01c1537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b410 tx=0x7ff01000b8d0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 shutdown_connections 2023-11-06T03:15:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff008065a60 0x7ff008067f20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff01c15cc70 0x7ff01c0c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c0cfa90 0x7ff01c1537b0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 --2- 172.21.15.157:0/1955152320 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff01c0b6e80 0x7ff01c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 >> 172.21.15.157:0/1955152320 conn(0x7ff01c070790 msgr2=0x7ff01c1583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 shutdown_connections 2023-11-06T03:15:56.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.083+0000 7ff023a90640 1 -- 172.21.15.157:0/1955152320 wait complete. 2023-11-06T03:15:56.123 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 scrub status 2023-11-06T03:15:56.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 Processor -- start 2023-11-06T03:15:56.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 -- start start 2023-11-06T03:15:56.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2a415caa0 0x7ff2a40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 0x7ff2a40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40d2920 con 0x7ff2a40ce1a0 2023-11-06T03:15:56.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40d18a0 con 0x7ff2a415caa0 2023-11-06T03:15:56.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2abb94640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40d1b60 con 0x7ff2a415ce80 2023-11-06T03:15:56.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a9930640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2a415caa0 0x7ff2a40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2aa131640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff2a40ce1a0 0x7ff2a40d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:44774/0 (socket says 172.21.15.157:44774) 2023-11-06T03:15:56.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2aa131640 1 -- 172.21.15.157:0/287072896 learned_addr learned my addr 172.21.15.157:0/287072896 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:15:56.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a912f640 1 --2- 172.21.15.157:0/287072896 >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 0x7ff2a40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a9930640 1 -- 172.21.15.157:0/287072896 >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 msgr2=0x7ff2a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a9930640 1 --2- 172.21.15.157:0/287072896 >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 0x7ff2a40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a9930640 1 -- 172.21.15.157:0/287072896 >> v1:172.21.15.157:6789/0 conn(0x7ff2a40ce1a0 legacy=0x7ff2a40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a9930640 1 -- 172.21.15.157:0/287072896 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40d1e40 con 0x7ff2a415caa0 2023-11-06T03:15:56.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a892e640 1 -- 172.21.15.157:0/287072896 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526828240 0 0) 0x7ff2a40d2920 con 0x7ff2a40ce1a0 2023-11-06T03:15:56.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.239+0000 7ff2a912f640 1 --2- 172.21.15.157:0/287072896 >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 0x7ff2a40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:56.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a9930640 1 --2- 172.21.15.157:0/287072896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415caa0 0x7ff2a40cdc60 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40c6720 tx=0x7ff2a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f79a4cdd87f6f7c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a892e640 1 -- 172.21.15.157:0/287072896 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00ab600 con 0x7ff2a415caa0 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a892e640 1 -- 172.21.15.157:0/287072896 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00a7050 con 0x7ff2a415caa0 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a892e640 1 -- 172.21.15.157:0/287072896 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00ab980 con 0x7ff2a415caa0 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/287072896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415caa0 msgr2=0x7ff2a40cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- 172.21.15.157:0/287072896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415caa0 0x7ff2a40cdc60 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40c6720 tx=0x7ff2a001c430 comp rx=0 tx=0).stop 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/287072896 shutdown_connections 2023-11-06T03:15:56.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- 172.21.15.157:0/287072896 >> v2:172.21.15.195:3300/0 conn(0x7ff2a415ce80 0x7ff2a40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- 172.21.15.157:0/287072896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415caa0 0x7ff2a40cdc60 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/287072896 >> 172.21.15.157:0/287072896 conn(0x7ff2a4070790 msgr2=0x7ff2a4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/287072896 shutdown_connections 2023-11-06T03:15:56.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/287072896 wait complete. 2023-11-06T03:15:56.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 Processor -- start 2023-11-06T03:15:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- start start 2023-11-06T03:15:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a40ce1a0 0x7ff2a4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 0x7ff2a40cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4154e60 con 0x7ff2a415ce80 2023-11-06T03:15:56.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4155140 con 0x7ff2a40ce1a0 2023-11-06T03:15:56.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4155420 con 0x7ff2a415caa0 2023-11-06T03:15:56.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37572/0 (socket says 172.21.15.157:37572) 2023-11-06T03:15:56.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 -- 172.21.15.157:0/3359374433 learned_addr learned my addr 172.21.15.157:0/3359374433 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a9930640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a40ce1a0 0x7ff2a4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a912f640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 0x7ff2a40cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 -- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 msgr2=0x7ff2a40cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 0x7ff2a40cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 -- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a40ce1a0 msgr2=0x7ff2a4150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a40ce1a0 0x7ff2a4150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 -- 172.21.15.157:0/3359374433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009420 con 0x7ff2a415ce80 2023-11-06T03:15:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2a912f640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 0x7ff2a40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2aa131640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c750 tx=0x7ff29c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff29a7fc640 1 -- 172.21.15.157:0/3359374433 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00ce60 con 0x7ff2a415ce80 2023-11-06T03:15:56.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff29a7fc640 1 -- 172.21.15.157:0/3359374433 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00a640 con 0x7ff2a415ce80 2023-11-06T03:15:56.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a40ca9c0 con 0x7ff2a415ce80 2023-11-06T03:15:56.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff29a7fc640 1 -- 172.21.15.157:0/3359374433 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00a9b0 con 0x7ff2a415ce80 2023-11-06T03:15:56.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.243+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40caf80 con 0x7ff2a415ce80 2023-11-06T03:15:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff29a7fc640 1 -- 172.21.15.157:0/3359374433 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff29c00ab10 con 0x7ff2a415ce80 2023-11-06T03:15:56.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff29a7fc640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 0x7ff288067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff29a7fc640 1 -- 172.21.15.157:0/3359374433 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff29c082ca0 con 0x7ff2a415ce80 2023-11-06T03:15:56.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 Processor -- start 2023-11-06T03:15:56.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 -- start start 2023-11-06T03:15:56.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a9930640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 0x7ff288067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456130 0x555ca7456510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 0x555ca740c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca72d3c10 con 0x555ca7456a50 2023-11-06T03:15:56.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca72623e0 con 0x555ca7456130 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ca725f9b0 con 0x555ca740c260 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a9930640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 0x7ff288067f20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cf670 tx=0x7ff2a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37578/0 (socket says 172.21.15.157:37578) 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 -- 172.21.15.157:0/1722098520 learned_addr learned my addr 172.21.15.157:0/1722098520 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a9930640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456130 0x555ca7456510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aa131640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 0x555ca740c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 -- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 msgr2=0x555ca740c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 0x555ca740c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 -- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456130 msgr2=0x555ca7456510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456130 0x555ca7456510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 -- 172.21.15.157:0/1722098520 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ca6f57e20 con 0x555ca7456a50 2023-11-06T03:15:56.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2aa131640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 0x555ca740c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:15:56.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7ff29400baa0 tx=0x7ff29400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec02a47992f1ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2877fe640 1 -- 172.21.15.157:0/1722098520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400dd00 con 0x555ca7456a50 2023-11-06T03:15:56.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2877fe640 1 -- 172.21.15.157:0/1722098520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400c040 con 0x555ca7456a50 2023-11-06T03:15:56.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.247+0000 7ff2877fe640 1 -- 172.21.15.157:0/1722098520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940126a0 con 0x555ca7456a50 2023-11-06T03:15:56.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1722098520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 msgr2=0x555ca740bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7ff29400baa0 tx=0x7ff29400bf60 comp rx=0 tx=0).stop 2023-11-06T03:15:56.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1722098520 shutdown_connections 2023-11-06T03:15:56.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca740c260 0x555ca740c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456a50 0x555ca740bd20 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1722098520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456130 0x555ca7456510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1722098520 >> 172.21.15.157:0/1722098520 conn(0x555ca74129f0 msgr2=0x555ca740a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1722098520 shutdown_connections 2023-11-06T03:15:56.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1722098520 wait complete. 2023-11-06T03:15:56.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- start start 2023-11-06T03:15:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 Processor -- start 2023-11-06T03:15:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7456e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca725b2a0 con 0x555ca7456130 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca71fa470 con 0x555ca740c260 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ca7218660 con 0x555ca7456a50 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 10 client.0 _ll_register_callbacks cb 0x555ca73f7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37580/0 (socket says 172.21.15.157:37580) 2023-11-06T03:15:56.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 -- 172.21.15.157:0/1550719507 learned_addr learned my addr 172.21.15.157:0/1550719507 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a9930640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7456e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aa131640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 -- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 msgr2=0x555ca74578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 -- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 msgr2=0x555ca7456e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7456e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 -- 172.21.15.157:0/1550719507 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29400b6d0 con 0x555ca7456130 2023-11-06T03:15:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a912f640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7ff29400bfa0 tx=0x7ff294004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0d2bb36f2dd690 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:56.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294020070 con 0x555ca7456130 2023-11-06T03:15:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff2940129a0 con 0x555ca7456130 2023-11-06T03:15:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ca72180a0 con 0x555ca7456130 2023-11-06T03:15:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff294012b60 con 0x555ca7456130 2023-11-06T03:15:56.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 10 client.10146 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x555ca72201d0 con 0x555ca7456130 2023-11-06T03:15:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555ca72180a0 con 0x555ca7456130 2023-11-06T03:15:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff294012cc0 con 0x555ca7456130 2023-11-06T03:15:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 10 client.10146 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:56.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 10 client.10146 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:56.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 20 client.10146 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:56.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca7465cf0 0x555ca74680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 4 client.10146 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:15:56.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ca7259bf0 con 0x555ca7465cf0 2023-11-06T03:15:56.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a9930640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca7465cf0 0x555ca74680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff284ff9640 10 client.10146 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.251+0000 7ff2a9930640 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca7465cf0 0x555ca74680e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803d570bf5485ae8 server_cookie=b1235dcd4d3c08cb in_seq=0 out_seq=0 2023-11-06T03:15:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff284ff9640 1 -- 172.21.15.157:0/1550719507 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555ca7259bf0 con 0x555ca7465cf0 2023-11-06T03:15:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff284ff9640 10 client.10146 handle_command_reply: tid=0 2023-11-06T03:15:56.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 client.10146 shutdown 2023-11-06T03:15:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 msgr2=0x555ca7457370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7ff29400bfa0 tx=0x7ff294004740 comp rx=0 tx=0).stop 2023-11-06T03:15:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 shutdown_connections 2023-11-06T03:15:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca7465cf0 0x555ca74680e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca7456130 0x555ca7457370 secure :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7ff29400bfa0 tx=0x7ff294004740 comp rx=0 tx=0).stop 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/1550719507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7456e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff284ff9640 0 client.10146 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 >> 172.21.15.157:0/1550719507 conn(0x555ca74129f0 msgr2=0x555ca740ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 shutdown_connections 2023-11-06T03:15:56.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 1 -- 172.21.15.157:0/1550719507 wait complete. 2023-11-06T03:15:56.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.255+0000 7ff2aecbc000 20 client.10146 trim_cache size 0 max 16384 2023-11-06T03:15:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 Processor -- start 2023-11-06T03:15:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 -- start start 2023-11-06T03:15:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456a50 0x555ca7468620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca746a250 0x555ca7468b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca7457a60 con 0x555ca740c260 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca6f18aa0 con 0x555ca7456a50 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ca72e3c70 con 0x555ca746a250 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37592/0 (socket says 172.21.15.157:37592) 2023-11-06T03:15:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 -- 172.21.15.157:0/2182230685 learned_addr learned my addr 172.21.15.157:0/2182230685 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 -- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca746a250 msgr2=0x555ca7468b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca746a250 0x555ca7468b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 -- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456a50 msgr2=0x555ca7468620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a912f640 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456a50 0x555ca7468620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456a50 0x555ca7468620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 -- 172.21.15.157:0/2182230685 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ca7206520 con 0x555ca740c260 2023-11-06T03:15:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.267+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001c470 tx=0x7ff2a0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67d12cec0ca2537 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff287fff640 1 -- 172.21.15.157:0/2182230685 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00abcc0 con 0x555ca740c260 2023-11-06T03:15:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff287fff640 1 -- 172.21.15.157:0/2182230685 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00bc040 con 0x555ca740c260 2023-11-06T03:15:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff287fff640 1 -- 172.21.15.157:0/2182230685 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00c7360 con 0x555ca740c260 2023-11-06T03:15:56.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2182230685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 msgr2=0x555ca74680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001c470 tx=0x7ff2a0004740 comp rx=0 tx=0).stop 2023-11-06T03:15:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2182230685 shutdown_connections 2023-11-06T03:15:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca746a250 0x555ca7468b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca7456a50 0x555ca7468620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2182230685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca740c260 0x555ca74680e0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2182230685 >> 172.21.15.157:0/2182230685 conn(0x555ca74129f0 msgr2=0x555ca7410ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2182230685 shutdown_connections 2023-11-06T03:15:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2182230685 wait complete. 2023-11-06T03:15:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- start start 2023-11-06T03:15:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 Processor -- start 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7469760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca72007a0 con 0x555ca746a250 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca7219400 con 0x555ca740c260 2023-11-06T03:15:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ca7256c10 con 0x555ca7456a50 2023-11-06T03:15:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 10 client.0 _ll_register_callbacks cb 0x555ca721fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:15:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37602/0 (socket says 172.21.15.157:37602) 2023-11-06T03:15:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 -- 172.21.15.157:0/2812518812 learned_addr learned my addr 172.21.15.157:0/2812518812 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:15:56.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2a912f640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7469760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 -- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 msgr2=0x555ca74b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 -- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 msgr2=0x555ca7469760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:15:56.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7469760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 -- 172.21.15.157:0/2812518812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40d1e40 con 0x555ca746a250 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aa131640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff29c001910 tx=0x7ff29c054180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d178bb6754e2c57 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c05bc10 con 0x555ca746a250 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff29c05bd70 con 0x555ca746a250 2023-11-06T03:15:56.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ca7224620 con 0x555ca746a250 2023-11-06T03:15:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff29c067af0 con 0x555ca746a250 2023-11-06T03:15:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 10 client.10152 fetch_fsmap learned FSMap version 111 2023-11-06T03:15:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x555ca71fa090 con 0x555ca746a250 2023-11-06T03:15:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c07d020 con 0x555ca746a250 2023-11-06T03:15:56.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff29c067c50 con 0x555ca746a250 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 10 client.10152 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 10 client.10152 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 20 client.10152 populate_metadata read hostname 'smithi157' 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca74b10f0 0x555ca74ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 4 client.10152 mds_command: new command op to 9918 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:15:56.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ca7254320 con 0x555ca74b10f0 2023-11-06T03:15:56.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca74b10f0 0x555ca74ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:15:56.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 10 client.10152 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2a9930640 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca74b10f0 0x555ca74ba990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976364341f03f9dc server_cookie=909e7eb2e8a898a1 in_seq=0 out_seq=0 2023-11-06T03:15:56.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 1 -- 172.21.15.157:0/2812518812 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x555ca7254320 con 0x555ca74b10f0 2023-11-06T03:15:56.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 10 client.10152 handle_command_reply: tid=0 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 client.10152 shutdown 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 msgr2=0x555ca74b8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff29c001910 tx=0x7ff29c054180 comp rx=0 tx=0).stop 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 shutdown_connections 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x555ca74b10f0 0x555ca74ba990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.292 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ca746a250 0x555ca74b8440 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x555ca7456a50 0x555ca74b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 --2- 172.21.15.157:0/2812518812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555ca740c260 0x555ca7469760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff284ff9640 0 client.10152 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:15:56.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.271+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 >> 172.21.15.157:0/2812518812 conn(0x555ca74129f0 msgr2=0x555ca7411350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.293 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 shutdown_connections 2023-11-06T03:15:56.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2aecbc000 1 -- 172.21.15.157:0/2812518812 wait complete. 2023-11-06T03:15:56.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2aecbc000 20 client.10152 trim_cache size 0 max 16384 2023-11-06T03:15:56.294 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:15:56.295 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:15:56.295 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:15:56.295 INFO:teuthology.orchestra.run.smithi157.stdout: "185c72d6-9c41-410b-8677-4d3deb0c4c27": { 2023-11-06T03:15:56.295 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "~mds0", 2023-11-06T03:15:56.295 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "185c72d6-9c41-410b-8677-4d3deb0c4c27", 2023-11-06T03:15:56.296 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive" 2023-11-06T03:15:56.296 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:56.296 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:15:56.296 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:15:56.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 msgr2=0x7ff288067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 0x7ff288067f20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40cf670 tx=0x7ff2a00a6000 comp rx=0 tx=0).stop 2023-11-06T03:15:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 msgr2=0x7ff2a41568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:15:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c750 tx=0x7ff29c00cc10 comp rx=0 tx=0).stop 2023-11-06T03:15:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 shutdown_connections 2023-11-06T03:15:56.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff288065a60 0x7ff288067f20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a415ce80 0x7ff2a41568e0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a415caa0 0x7ff2a40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 --2- 172.21.15.157:0/3359374433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2a40ce1a0 0x7ff2a4150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:15:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 >> 172.21.15.157:0/3359374433 conn(0x7ff2a4070790 msgr2=0x7ff2a4070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:15:56.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 shutdown_connections 2023-11-06T03:15:56.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:15:56.275+0000 7ff2abb94640 1 -- 172.21.15.157:0/3359374433 wait complete. 2023-11-06T03:15:56.300 INFO:tasks.cephfs.filesystem:scrub status for tag:185c72d6-9c41-410b-8677-4d3deb0c4c27 - {'path': '~mds0', 'tag': '185c72d6-9c41-410b-8677-4d3deb0c4c27', 'options': 'recursive'} 2023-11-06T03:16:12.148 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:16:12.151 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:16:26.301 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 scrub status 2023-11-06T03:16:26.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.439+0000 7f03226eb640 1 Processor -- start 2023-11-06T03:16:26.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.439+0000 7f03226eb640 1 -- start start 2023-11-06T03:16:26.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f03226eb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f031c15caa0 0x7f031c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f03226eb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c0d2920 con 0x7f031c15ce80 2023-11-06T03:16:26.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f03226eb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c0d18a0 con 0x7f031c0d0280 2023-11-06T03:16:26.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f03226eb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c0d1b60 con 0x7f031c15caa0 2023-11-06T03:16:26.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f031c15ce80 0x7f031c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55032/0 (socket says 172.21.15.157:55032) 2023-11-06T03:16:26.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031b7fe640 1 -- 172.21.15.157:0/3707671327 learned_addr learned my addr 172.21.15.157:0/3707671327 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:26.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81928655 0 0) 0x7f031c0d2920 con 0x7f031c15ce80 2023-11-06T03:16:26.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0304003530 con 0x7f031c15ce80 2023-11-06T03:16:26.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031bfff640 1 --2- 172.21.15.157:0/3707671327 >> v2:172.21.15.195:3300/0 conn(0x7f031c15caa0 0x7f031c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674754097 0 0) 0x7f0304003530 con 0x7f031c15ce80 2023-11-06T03:16:26.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 >> v2:172.21.15.195:3300/0 conn(0x7f031c15caa0 msgr2=0x7f031c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 --2- 172.21.15.157:0/3707671327 >> v2:172.21.15.195:3300/0 conn(0x7f031c15caa0 0x7f031c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 >> v1:172.21.15.195:6789/0 conn(0x7f031c0d0280 legacy=0x7f031c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0d1e40 con 0x7f031c15ce80 2023-11-06T03:16:26.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0310001680 con 0x7f031c15ce80 2023-11-06T03:16:26.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 >> v1:172.21.15.157:6789/0 conn(0x7f031c15ce80 legacy=0x7f031c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304003980 0x7f0304003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03040047d0 0x7f0304004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 0x7f030400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0d1b60 con 0x7f0304005190 2023-11-06T03:16:26.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030400c0e0 con 0x7f03040047d0 2023-11-06T03:16:26.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030400c2a0 con 0x7f0304003980 2023-11-06T03:16:26.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 0x7f030400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031b7fe640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03040047d0 0x7f0304004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f031bfff640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304003980 0x7f0304003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 -- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304003980 msgr2=0x7f0304003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0304003980 0x7f0304003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 -- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03040047d0 msgr2=0x7f0304004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03040047d0 0x7f0304004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.443+0000 7f0320c88640 1 -- 172.21.15.157:0/3707671327 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0304003530 con 0x7f0304005190 2023-11-06T03:16:26.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f0320c88640 1 -- 172.21.15.157:0/3707671327 reap_dead start 2023-11-06T03:16:26.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f0320c88640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 0x7f030400b990 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f0314001e70 tx=0x7f031401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4489f8b46a274cee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314003b80 con 0x7f0304005190 2023-11-06T03:16:26.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314003d20 con 0x7f0304005190 2023-11-06T03:16:26.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f031affd640 1 -- 172.21.15.157:0/3707671327 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314005260 con 0x7f0304005190 2023-11-06T03:16:26.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 msgr2=0x7f030400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 0x7f030400b990 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f0314001e70 tx=0x7f031401c760 comp rx=0 tx=0).stop 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- 172.21.15.157:0/3707671327 shutdown_connections 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 --2- 172.21.15.157:0/3707671327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0304005190 0x7f030400b990 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- 172.21.15.157:0/3707671327 >> 172.21.15.157:0/3707671327 conn(0x7f031c070790 msgr2=0x7f031c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- 172.21.15.157:0/3707671327 shutdown_connections 2023-11-06T03:16:26.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- 172.21.15.157:0/3707671327 wait complete. 2023-11-06T03:16:26.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 Processor -- start 2023-11-06T03:16:26.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.447+0000 7f03226eb640 1 -- start start 2023-11-06T03:16:26.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005190 0x7f031c14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f031c15beb0 0x7f031c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c15c8a0 con 0x7f031c0cc380 2023-11-06T03:16:26.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c14f5b0 con 0x7f0304005190 2023-11-06T03:16:26.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c1e69d0 con 0x7f031c15beb0 2023-11-06T03:16:26.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44620/0 (socket says 172.21.15.157:44620) 2023-11-06T03:16:26.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 -- 172.21.15.157:0/113058332 learned_addr learned my addr 172.21.15.157:0/113058332 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031bfff640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005190 0x7f031c14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0320c88640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f031c15beb0 0x7f031c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 -- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f031c15beb0 msgr2=0x7f031c15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f031c15beb0 0x7f031c15c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 -- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005190 msgr2=0x7f031c14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005190 0x7f031c14f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 -- 172.21.15.157:0/113058332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c1e6cb0 con 0x7f031c0cc380 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031b7fe640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f0310007290 tx=0x7f0310007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 -- 172.21.15.157:0/113058332 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000f9c0 con 0x7f031c0cc380 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 -- 172.21.15.157:0/113058332 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031000fb20 con 0x7f031c0cc380 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c1e6f90 con 0x7f031c0cc380 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 -- 172.21.15.157:0/113058332 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000feb0 con 0x7f031c0cc380 2023-11-06T03:16:26.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c1e7550 con 0x7f031c0cc380 2023-11-06T03:16:26.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 -- 172.21.15.157:0/113058332 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f031001d460 con 0x7f031c0cc380 2023-11-06T03:16:26.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 0x7f0308067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f0318ff9640 1 -- 172.21.15.157:0/113058332 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f0310083630 con 0x7f031c0cc380 2023-11-06T03:16:26.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.451+0000 7f031bfff640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 0x7f0308067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 Processor -- start 2023-11-06T03:16:26.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- start start 2023-11-06T03:16:26.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 0x7f0308067f50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f030c0045f0 tx=0x7f030c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb181d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb182640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bb049c10 con 0x5571bb1cc130 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bafd83e0 con 0x5571bb1cca50 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571bafd59b0 con 0x5571bb182260 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44632/0 (socket says 172.21.15.157:44632) 2023-11-06T03:16:26.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 -- 172.21.15.157:0/3956042359 learned_addr learned my addr 172.21.15.157:0/3956042359 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031b7fe640 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb181d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0320c88640 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb182640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 -- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 msgr2=0x5571bb182640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb182640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 -- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 msgr2=0x5571bb181d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb181d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 -- 172.21.15.157:0/3956042359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571baccde20 con 0x5571bb1cc130 2023-11-06T03:16:26.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f031bfff640 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b8e0 tx=0x7f030c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b5bbaead969fdd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0301ffb640 1 -- 172.21.15.157:0/3956042359 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00be60 con 0x5571bb1cc130 2023-11-06T03:16:26.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0301ffb640 1 -- 172.21.15.157:0/3956042359 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0159d0 con 0x5571bb1cc130 2023-11-06T03:16:26.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0301ffb640 1 -- 172.21.15.157:0/3956042359 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c015d40 con 0x5571bb1cc130 2023-11-06T03:16:26.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- 172.21.15.157:0/3956042359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 msgr2=0x5571bb1cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f030c00b8e0 tx=0x7f030c004620 comp rx=0 tx=0).stop 2023-11-06T03:16:26.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- 172.21.15.157:0/3956042359 shutdown_connections 2023-11-06T03:16:26.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb182640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb181d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 --2- 172.21.15.157:0/3956042359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb1cc510 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- 172.21.15.157:0/3956042359 >> 172.21.15.157:0/3956042359 conn(0x5571bb1889f0 msgr2=0x5571bb180600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- 172.21.15.157:0/3956042359 shutdown_connections 2023-11-06T03:16:26.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- 172.21.15.157:0/3956042359 wait complete. 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 -- start start 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.455+0000 7f0325813000 1 Processor -- start 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb1d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb1cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bafd12a0 con 0x5571bb182260 2023-11-06T03:16:26.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571baf70470 con 0x5571bb1cca50 2023-11-06T03:16:26.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571baf8e660 con 0x5571bb1cc130 2023-11-06T03:16:26.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 10 client.0 _ll_register_callbacks cb 0x5571bb16dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:26.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44644/0 (socket says 172.21.15.157:44644) 2023-11-06T03:16:26.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 -- 172.21.15.157:0/281424335 learned_addr learned my addr 172.21.15.157:0/281424335 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0320c88640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb1cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031b7fe640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb1d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 -- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 msgr2=0x5571bb1d3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb1d3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 -- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 msgr2=0x5571bb1cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb1cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 -- 172.21.15.157:0/281424335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c00e040 con 0x5571bb182260 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0320c88640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb1cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031b7fe640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb1d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:26.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031bfff640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f030c008010 tx=0x7f030c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ba7a63168ace23 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:26.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c015ea0 con 0x5571bb182260 2023-11-06T03:16:26.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571baf8e0a0 con 0x5571bb182260 2023-11-06T03:16:26.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f030c00be60 con 0x5571bb182260 2023-11-06T03:16:26.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f030c022a80 con 0x5571bb182260 2023-11-06T03:16:26.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 10 client.10164 fetch_fsmap learned FSMap version 111 2023-11-06T03:16:26.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x5571baf961d0 con 0x5571bb182260 2023-11-06T03:16:26.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571baf8e0a0 con 0x5571bb182260 2023-11-06T03:16:26.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f030c022be0 con 0x5571bb182260 2023-11-06T03:16:26.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 10 client.10164 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 10 client.10164 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 20 client.10164 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb1cdbd0 0x5571bb1cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 4 client.10164 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571bafcfbf0 con 0x5571bb1cdbd0 2023-11-06T03:16:26.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031b7fe640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb1cdbd0 0x5571bb1cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f031b7fe640 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb1cdbd0 0x5571bb1cdfb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be2a197083358e7 server_cookie=850bad7bef28eddc in_seq=0 out_seq=0 2023-11-06T03:16:26.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 10 client.10164 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 1 -- 172.21.15.157:0/281424335 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5571bafcfbf0 con 0x5571bb1cdbd0 2023-11-06T03:16:26.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 10 client.10164 handle_command_reply: tid=0 2023-11-06T03:16:26.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 client.10164 shutdown 2023-11-06T03:16:26.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 msgr2=0x5571bb1d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f030c008010 tx=0x7f030c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 shutdown_connections 2023-11-06T03:16:26.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb1cdbd0 0x5571bb1cdfb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb1cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb1d3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f02faffd640 0 client.10164 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 --2- 172.21.15.157:0/281424335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb182260 0x5571bb1d3540 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 >> 172.21.15.157:0/281424335 conn(0x5571bb1889f0 msgr2=0x5571bb180d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 shutdown_connections 2023-11-06T03:16:26.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 1 -- 172.21.15.157:0/281424335 wait complete. 2023-11-06T03:16:26.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.459+0000 7f0325813000 20 client.10164 trim_cache size 0 max 16384 2023-11-06T03:16:26.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 Processor -- start 2023-11-06T03:16:26.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- start start 2023-11-06T03:16:26.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 0x5571bb223660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb223ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bac8eaa0 con 0x5571bb1cca50 2023-11-06T03:16:26.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bb013cd0 con 0x5571bb182260 2023-11-06T03:16:26.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571bafd3e00 con 0x5571bb1cc130 2023-11-06T03:16:26.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44648/0 (socket says 172.21.15.157:44648) 2023-11-06T03:16:26.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 -- 172.21.15.157:0/880662501 learned_addr learned my addr 172.21.15.157:0/880662501 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f031bfff640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 0x5571bb223660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f031b7fe640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb223ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 -- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 msgr2=0x5571bb223ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb223ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 -- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 msgr2=0x5571bb223660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 0x5571bb223660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 -- 172.21.15.157:0/880662501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571baf7c520 con 0x5571bb1cca50 2023-11-06T03:16:26.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f031bfff640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 0x5571bb223660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:26.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0320c88640 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f031401d000 tx=0x7f0314001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14db24352adfed5d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f03027fc640 1 -- 172.21.15.157:0/880662501 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314001260 con 0x5571bb1cca50 2023-11-06T03:16:26.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f03027fc640 1 -- 172.21.15.157:0/880662501 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314003980 con 0x5571bb1cca50 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f03027fc640 1 -- 172.21.15.157:0/880662501 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314006b60 con 0x5571bb1cca50 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- 172.21.15.157:0/880662501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 msgr2=0x5571bb2240e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f031401d000 tx=0x7f0314001520 comp rx=0 tx=0).stop 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- 172.21.15.157:0/880662501 shutdown_connections 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cca50 0x5571bb2240e0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb1cc130 0x5571bb223ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 --2- 172.21.15.157:0/880662501 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb182260 0x5571bb223660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- 172.21.15.157:0/880662501 >> 172.21.15.157:0/880662501 conn(0x5571bb1889f0 msgr2=0x5571bb187a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- 172.21.15.157:0/880662501 shutdown_connections 2023-11-06T03:16:26.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.475+0000 7f0325813000 1 -- 172.21.15.157:0/880662501 wait complete. 2023-11-06T03:16:26.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- start start 2023-11-06T03:16:26.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 Processor -- start 2023-11-06T03:16:26.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb229260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb22e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bafccc10 con 0x5571bb1cc130 2023-11-06T03:16:26.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571bafceab0 con 0x5571bb1cca50 2023-11-06T03:16:26.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571baf767a0 con 0x5571bb182260 2023-11-06T03:16:26.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 10 client.0 _ll_register_callbacks cb 0x5571baf95cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:26.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44654/0 (socket says 172.21.15.157:44654) 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 -- 172.21.15.157:0/2699678045 learned_addr learned my addr 172.21.15.157:0/2699678045 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031bfff640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb229260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0320c88640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb22e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 -- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 msgr2=0x5571bb229260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb229260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 -- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 msgr2=0x5571bb22e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb22e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 -- 172.21.15.157:0/2699678045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03140af040 con 0x5571bb1cc130 2023-11-06T03:16:26.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031b7fe640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f031005c500 tx=0x7f0310004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85152f944fa20d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:26.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031005ca30 con 0x5571bb1cc130 2023-11-06T03:16:26.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f03100510d0 con 0x5571bb1cc130 2023-11-06T03:16:26.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571baf9a620 con 0x5571bb1cc130 2023-11-06T03:16:26.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0310051290 con 0x5571bb1cc130 2023-11-06T03:16:26.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 10 client.10170 fetch_fsmap learned FSMap version 111 2023-11-06T03:16:26.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x5571baf70090 con 0x5571bb1cc130 2023-11-06T03:16:26.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5571baf9a620 con 0x5571bb1cc130 2023-11-06T03:16:26.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0310012070 con 0x5571bb1cc130 2023-11-06T03:16:26.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 10 client.10170 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:16:26.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 10 client.10170 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:16:26.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 20 client.10170 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:26.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb22e660 0x5571bb230a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 4 client.10170 mds_command: new command op to 9918 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:16:26.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571bafca320 con 0x5571bb22e660 2023-11-06T03:16:26.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031bfff640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb22e660 0x5571bb230a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f031bfff640 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb22e660 0x5571bb230a50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fc490c425145d9 server_cookie=c33da09af9031690 in_seq=0 out_seq=0 2023-11-06T03:16:26.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 10 client.10170 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 1 -- 172.21.15.157:0/2699678045 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5571bafca320 con 0x5571bb22e660 2023-11-06T03:16:26.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 10 client.10170 handle_command_reply: tid=0 2023-11-06T03:16:26.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 client.10170 shutdown 2023-11-06T03:16:26.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 msgr2=0x5571bb2297a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f031005c500 tx=0x7f0310004f90 comp rx=0 tx=0).stop 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 shutdown_connections 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x5571bb22e660 0x5571bb230a50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5571bb1cca50 0x5571bb22e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f02faffd640 0 client.10170 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5571bb1cc130 0x5571bb2297a0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 --2- 172.21.15.157:0/2699678045 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5571bb182260 0x5571bb229260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 >> 172.21.15.157:0/2699678045 conn(0x5571bb1889f0 msgr2=0x5571bb1cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 shutdown_connections 2023-11-06T03:16:26.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 1 -- 172.21.15.157:0/2699678045 wait complete. 2023-11-06T03:16:26.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f0325813000 20 client.10170 trim_cache size 0 max 16384 2023-11-06T03:16:26.503 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:26.503 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:16:26.503 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:16:26.504 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 msgr2=0x7f0308067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 0x7f0308067f50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f030c0045f0 tx=0x7f030c008040 comp rx=0 tx=0).stop 2023-11-06T03:16:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 msgr2=0x7f031c0cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.479+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f0310007290 tx=0x7f0310007750 comp rx=0 tx=0).stop 2023-11-06T03:16:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 shutdown_connections 2023-11-06T03:16:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0308065a90 0x7f0308067f50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f031c15beb0 0x7f031c15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f031c0cc380 0x7f031c0cc7e0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 --2- 172.21.15.157:0/113058332 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0304005190 0x7f031c14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 >> 172.21.15.157:0/113058332 conn(0x7f031c070790 msgr2=0x7f031c0661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 shutdown_connections 2023-11-06T03:16:26.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.483+0000 7f03226eb640 1 -- 172.21.15.157:0/113058332 wait complete. 2023-11-06T03:16:26.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:16:26.508 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.10:0 damage ls 2023-11-06T03:16:26.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.631+0000 7fef39a91640 1 Processor -- start 2023-11-06T03:16:26.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.631+0000 7fef39a91640 1 -- start start 2023-11-06T03:16:26.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef39a91640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef341551e0 0x7fef3406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef39a91640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3415f580 con 0x7fef340b3b20 2023-11-06T03:16:26.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef39a91640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3415efc0 con 0x7fef3415e5e0 2023-11-06T03:16:26.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef39a91640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3415f2a0 con 0x7fef341551e0 2023-11-06T03:16:26.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef33fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fef340b3b20 0x7fef3415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55038/0 (socket says 172.21.15.157:55038) 2023-11-06T03:16:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef33fff640 1 -- 172.21.15.157:0/1690867322 learned_addr learned my addr 172.21.15.157:0/1690867322 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263590460 0 0) 0x7fef3415f580 con 0x7fef340b3b20 2023-11-06T03:16:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef1c003530 con 0x7fef340b3b20 2023-11-06T03:16:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef337fe640 1 --2- 172.21.15.157:0/1690867322 >> v2:172.21.15.195:3300/0 conn(0x7fef341551e0 0x7fef3406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4083117992 0 0) 0x7fef1c003530 con 0x7fef340b3b20 2023-11-06T03:16:26.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 >> v2:172.21.15.195:3300/0 conn(0x7fef341551e0 msgr2=0x7fef3406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 --2- 172.21.15.157:0/1690867322 >> v2:172.21.15.195:3300/0 conn(0x7fef341551e0 0x7fef3406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 >> v1:172.21.15.195:6789/0 conn(0x7fef3415e5e0 legacy=0x7fef340b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3406f5a0 con 0x7fef340b3b20 2023-11-06T03:16:26.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fef2c001a00 con 0x7fef340b3b20 2023-11-06T03:16:26.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 >> v1:172.21.15.157:6789/0 conn(0x7fef340b3b20 legacy=0x7fef3415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c003980 0x7fef1c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef1c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c005390 0x7fef1c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3415f2a0 con 0x7fef1c0047d0 2023-11-06T03:16:26.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c00c220 con 0x7fef1c003980 2023-11-06T03:16:26.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c00c3e0 con 0x7fef1c005390 2023-11-06T03:16:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef1c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef337fe640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c003980 0x7fef1c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef33fff640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c005390 0x7fef1c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 -- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c005390 msgr2=0x7fef1c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef1c005390 0x7fef1c00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 -- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c003980 msgr2=0x7fef1c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef1c003980 0x7fef1c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef33fff640 1 -- 172.21.15.157:0/1690867322 reap_dead start 2023-11-06T03:16:26.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 -- 172.21.15.157:0/1690867322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c003530 con 0x7fef1c0047d0 2023-11-06T03:16:26.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef32ffd640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef1c004c50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fef28001780 tx=0x7fef2801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3011026319240e38 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280052d0 con 0x7fef1c0047d0 2023-11-06T03:16:26.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280a7050 con 0x7fef1c0047d0 2023-11-06T03:16:26.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.635+0000 7fef327fc640 1 -- 172.21.15.157:0/1690867322 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28003600 con 0x7fef1c0047d0 2023-11-06T03:16:26.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 msgr2=0x7fef1c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef1c004c50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fef28001780 tx=0x7fef2801c3d0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/1690867322 shutdown_connections 2023-11-06T03:16:26.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 --2- 172.21.15.157:0/1690867322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef1c004c50 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/1690867322 >> 172.21.15.157:0/1690867322 conn(0x7fef34065930 msgr2=0x7fef34065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/1690867322 shutdown_connections 2023-11-06T03:16:26.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/1690867322 wait complete. 2023-11-06T03:16:26.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 Processor -- start 2023-11-06T03:16:26.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- start start 2023-11-06T03:16:26.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340a84a0 0x7fef340a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 0x7fef34158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34159520 con 0x7fef1c0047d0 2023-11-06T03:16:26.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340a8e40 con 0x7fef34157ce0 2023-11-06T03:16:26.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef341589a0 con 0x7fef340a84a0 2023-11-06T03:16:26.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44682/0 (socket says 172.21.15.157:44682) 2023-11-06T03:16:26.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 -- 172.21.15.157:0/2255692094 learned_addr learned my addr 172.21.15.157:0/2255692094 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef33fff640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 0x7fef34158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef32ffd640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340a84a0 0x7fef340a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 -- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340a84a0 msgr2=0x7fef340a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340a84a0 0x7fef340a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 -- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 msgr2=0x7fef34158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 0x7fef34158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 -- 172.21.15.157:0/2255692094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34158c80 con 0x7fef1c0047d0 2023-11-06T03:16:26.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef33fff640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 0x7fef34158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:26.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef337fe640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fef2400f530 tx=0x7fef2400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef1bfff640 1 -- 172.21.15.157:0/2255692094 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24004020 con 0x7fef1c0047d0 2023-11-06T03:16:26.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef1bfff640 1 -- 172.21.15.157:0/2255692094 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef24004180 con 0x7fef1c0047d0 2023-11-06T03:16:26.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef1bfff640 1 -- 172.21.15.157:0/2255692094 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24004510 con 0x7fef1c0047d0 2023-11-06T03:16:26.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef34158f60 con 0x7fef1c0047d0 2023-11-06T03:16:26.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.639+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef34158680 con 0x7fef1c0047d0 2023-11-06T03:16:26.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef1bfff640 1 -- 172.21.15.157:0/2255692094 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fef24014670 con 0x7fef1c0047d0 2023-11-06T03:16:26.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef1bfff640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 0x7fef20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef1bfff640 1 -- 172.21.15.157:0/2255692094 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fef24083800 con 0x7fef1c0047d0 2023-11-06T03:16:26.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef32ffd640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 0x7fef20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 Processor -- start 2023-11-06T03:16:26.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 -- start start 2023-11-06T03:16:26.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef32ffd640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 0x7fef20067f20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fef28001750 tx=0x7fef280011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643803ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 0x55e6437b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e643681d60 con 0x55e6437b9f90 2023-11-06T03:16:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6436103b0 con 0x55e643804530 2023-11-06T03:16:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e64360d980 con 0x55e643803c10 2023-11-06T03:16:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44690/0 (socket says 172.21.15.157:44690) 2023-11-06T03:16:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 -- 172.21.15.157:0/4191622015 learned_addr learned my addr 172.21.15.157:0/4191622015 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef337fe640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643803ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef32ffd640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 0x55e6437b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 -- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 msgr2=0x55e643803ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643803ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 -- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 msgr2=0x55e6437b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 0x55e6437b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 -- 172.21.15.157:0/4191622015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6435aa110 con 0x55e6437b9f90 2023-11-06T03:16:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef32ffd640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 0x55e6437b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef33fff640 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fef2c000f90 tx=0x7fef2c005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e6ca65504b368a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef197fa640 1 -- 172.21.15.157:0/4191622015 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0079f0 con 0x55e6437b9f90 2023-11-06T03:16:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef197fa640 1 -- 172.21.15.157:0/4191622015 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00d070 con 0x55e6437b9f90 2023-11-06T03:16:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.643+0000 7fef197fa640 1 -- 172.21.15.157:0/4191622015 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c013630 con 0x55e6437b9f90 2023-11-06T03:16:26.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4191622015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 msgr2=0x55e6437ba370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fef2c000f90 tx=0x7fef2c005f50 comp rx=0 tx=0).stop 2023-11-06T03:16:26.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4191622015 shutdown_connections 2023-11-06T03:16:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437ba370 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643804530 0x55e6437b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4191622015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643803ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4191622015 >> 172.21.15.157:0/4191622015 conn(0x55e6437b5f40 msgr2=0x55e6437b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4191622015 shutdown_connections 2023-11-06T03:16:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4191622015 wait complete. 2023-11-06T03:16:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- start start 2023-11-06T03:16:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 Processor -- start 2023-11-06T03:16:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643804530 0x55e6437f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e643609270 con 0x55e6437b9f90 2023-11-06T03:16:26.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6435a85a0 con 0x55e643803c10 2023-11-06T03:16:26.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6435c67c0 con 0x55e643804530 2023-11-06T03:16:26.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 10 client.0 _ll_register_callbacks cb 0x55e6435fd130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:26.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44702/0 (socket says 172.21.15.157:44702) 2023-11-06T03:16:26.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 -- 172.21.15.157:0/4025741884 learned_addr learned my addr 172.21.15.157:0/4025741884 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef33fff640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643804530 0x55e6437f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef32ffd640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 -- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643804530 msgr2=0x55e6437f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643804530 0x55e6437f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 -- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 msgr2=0x55e6437bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 -- 172.21.15.157:0/4025741884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3406f5a0 con 0x55e6437b9f90 2023-11-06T03:16:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef337fe640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fef2405c800 tx=0x7fef24054e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8771f9da7a07c9d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2406c070 con 0x55e6437b9f90 2023-11-06T03:16:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fef2406c1d0 con 0x55e6437b9f90 2023-11-06T03:16:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6435cd350 con 0x55e6437b9f90 2023-11-06T03:16:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef2405c8a0 con 0x55e6437b9f90 2023-11-06T03:16:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 10 client.10182 fetch_fsmap learned FSMap version 111 2023-11-06T03:16:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55e643305e20 con 0x55e6437b9f90 2023-11-06T03:16:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6435cd350 con 0x55e6437b9f90 2023-11-06T03:16:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fef24050b40 con 0x55e6437b9f90 2023-11-06T03:16:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 10 client.10182 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:16:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 10 client.10182 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:16:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 20 client.10182 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437f28f0 0x55e643815cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 4 client.10182 mds_command: new command op to 9918 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:16:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e643607bc0 con 0x55e6437f28f0 2023-11-06T03:16:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef32ffd640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437f28f0 0x55e643815cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef127fc640 10 client.10182 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.647+0000 7fef32ffd640 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437f28f0 0x55e643815cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d3f1c4a3fdc51e server_cookie=8d0f09d5f863b618 in_seq=0 out_seq=0 2023-11-06T03:16:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef127fc640 1 -- 172.21.15.157:0/4025741884 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e643607bc0 con 0x55e6437f28f0 2023-11-06T03:16:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef127fc640 10 client.10182 handle_command_reply: tid=0 2023-11-06T03:16:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 client.10182 shutdown 2023-11-06T03:16:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 msgr2=0x55e6437bed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fef2405c800 tx=0x7fef24054e10 comp rx=0 tx=0).stop 2023-11-06T03:16:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 shutdown_connections 2023-11-06T03:16:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437f28f0 0x55e643815cc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643804530 0x55e6437f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef127fc640 0 client.10182 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/4025741884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e6437b9f90 0x55e6437bed80 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 >> 172.21.15.157:0/4025741884 conn(0x55e6437b5f40 msgr2=0x55e6437b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 shutdown_connections 2023-11-06T03:16:26.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 1 -- 172.21.15.157:0/4025741884 wait complete. 2023-11-06T03:16:26.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.651+0000 7fef3cbb9000 20 client.10182 trim_cache size 0 max 16384 2023-11-06T03:16:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.659+0000 7fef3cbb9000 1 Processor -- start 2023-11-06T03:16:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.659+0000 7fef3cbb9000 1 -- start start 2023-11-06T03:16:26.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6437b9f90 0x55e6437b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6432c6aa0 con 0x55e643804530 2023-11-06T03:16:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6437c0580 con 0x55e643803c10 2023-11-06T03:16:26.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6435a6330 con 0x55e6437b9f90 2023-11-06T03:16:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44718/0 (socket says 172.21.15.157:44718) 2023-11-06T03:16:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/572511018 learned_addr learned my addr 172.21.15.157:0/572511018 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6437b9f90 msgr2=0x55e6437b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6437b9f90 0x55e6437b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 msgr2=0x55e6437b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef32ffd640 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437b9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/572511018 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6435ce1a0 con 0x55e643804530 2023-11-06T03:16:26.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009010 tx=0x7fef2c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ed390398c2b483 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef19ffb640 1 -- 172.21.15.157:0/572511018 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c027480 con 0x55e643804530 2023-11-06T03:16:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef19ffb640 1 -- 172.21.15.157:0/572511018 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c022070 con 0x55e643804530 2023-11-06T03:16:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef19ffb640 1 -- 172.21.15.157:0/572511018 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c027860 con 0x55e643804530 2023-11-06T03:16:26.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/572511018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 msgr2=0x55e643815cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009010 tx=0x7fef2c00d000 comp rx=0 tx=0).stop 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/572511018 shutdown_connections 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e643815cc0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e643803c10 0x55e6437b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/572511018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e6437b9f90 0x55e6437b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/572511018 >> 172.21.15.157:0/572511018 conn(0x55e6437b5f40 msgr2=0x55e643811200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/572511018 shutdown_connections 2023-11-06T03:16:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/572511018 wait complete. 2023-11-06T03:16:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- start start 2023-11-06T03:16:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 Processor -- start 2023-11-06T03:16:26.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6437b9f90 0x55e643860f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643861460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6437b58e0 con 0x55e643804530 2023-11-06T03:16:26.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e643603610 con 0x55e6437b9f90 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e64360d270 con 0x55e643803c10 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 10 client.0 _ll_register_callbacks cb 0x55e6435b7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44730/0 (socket says 172.21.15.157:44730) 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/2405646221 learned_addr learned my addr 172.21.15.157:0/2405646221 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef337fe640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6437b9f90 0x55e643860f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef32ffd640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643861460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 msgr2=0x55e643861460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643861460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6437b9f90 msgr2=0x55e643860f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6437b9f90 0x55e643860f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 -- 172.21.15.157:0/2405646221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6435aa110 con 0x55e643804530 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef33fff640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fef2c001380 tx=0x7fef2c001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af984dfeb1f2885f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef127fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c027890 con 0x55e643804530 2023-11-06T03:16:26.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fef2c0279f0 con 0x55e643804530 2023-11-06T03:16:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.663+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6435b8c20 con 0x55e643804530 2023-11-06T03:16:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef2c027bb0 con 0x55e643804530 2023-11-06T03:16:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 10 client.10188 fetch_fsmap learned FSMap version 111 2023-11-06T03:16:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) v3 -- 0x55e6435b44f0 con 0x55e643804530 2023-11-06T03:16:26.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6435b8c20 con 0x55e643804530 2023-11-06T03:16:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 111) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fef2c012070 con 0x55e643804530 2023-11-06T03:16:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 10 client.10188 fetch_fsmap finished waiting for FSMap version 111 2023-11-06T03:16:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 10 client.10188 resolve_mds: resolved 10:0 to role '10:0' aka daemon mds.a 2023-11-06T03:16:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 20 client.10188 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:26.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437ee090 0x55e6437ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 4 client.10188 mds_command: new command op to 9918 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:16:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 --> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6436022f0 con 0x55e6437ee090 2023-11-06T03:16:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef337fe640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437ee090 0x55e6437ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 10 client.10188 ms_handle_connect on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef337fe640 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437ee090 0x55e6437ee4a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4367411dd22c36c3 server_cookie=b16dacb7691896bb in_seq=0 out_seq=0 2023-11-06T03:16:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 1 -- 172.21.15.157:0/2405646221 <== mds.0 v2:172.21.15.157:6834/1485464139 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6436022f0 con 0x55e6437ee090 2023-11-06T03:16:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 10 client.10188 handle_command_reply: tid=0 2023-11-06T03:16:26.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 client.10188 shutdown 2023-11-06T03:16:26.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 msgr2=0x55e6437edb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fef2c001380 tx=0x7fef2c001d90 comp rx=0 tx=0).stop 2023-11-06T03:16:26.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 shutdown_connections 2023-11-06T03:16:26.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:6834/1485464139,v1:172.21.15.157:6835/1485464139] conn(0x55e6437ee090 0x55e6437ee4a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e643804530 0x55e6437edb50 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e643803c10 0x55e643861460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef127fc640 0 client.10188 ms_handle_reset on v2:172.21.15.157:6834/1485464139 2023-11-06T03:16:26.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 --2- 172.21.15.157:0/2405646221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6437b9f90 0x55e643860f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 >> 172.21.15.157:0/2405646221 conn(0x55e6437b5f40 msgr2=0x55e643811200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 shutdown_connections 2023-11-06T03:16:26.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 1 -- 172.21.15.157:0/2405646221 wait complete. 2023-11-06T03:16:26.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef3cbb9000 20 client.10188 trim_cache size 0 max 16384 2023-11-06T03:16:26.698 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2023-11-06T03:16:26.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 msgr2=0x7fef20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 0x7fef20067f20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fef28001750 tx=0x7fef280011a0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 msgr2=0x7fef34150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fef2400f530 tx=0x7fef2400f9f0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 shutdown_connections 2023-11-06T03:16:26.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef20065a60 0x7fef20067f20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34157ce0 0x7fef34158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef340a84a0 0x7fef340a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 --2- 172.21.15.157:0/2255692094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef1c0047d0 0x7fef34150de0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 >> 172.21.15.157:0/2255692094 conn(0x7fef34065930 msgr2=0x7fef34151740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 shutdown_connections 2023-11-06T03:16:26.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.667+0000 7fef39a91640 1 -- 172.21.15.157:0/2255692094 wait complete. 2023-11-06T03:16:26.703 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-06T03:16:26.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 Processor -- start 2023-11-06T03:16:26.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 -- start start 2023-11-06T03:16:26.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32b80b2af0 0x7f32b80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 0x7f32b8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b815e440 con 0x7f32b81599c0 2023-11-06T03:16:26.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b806c240 con 0x7f32b80b2af0 2023-11-06T03:16:26.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.819+0000 7f32c048a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b806c520 con 0x7f32b806d000 2023-11-06T03:16:26.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32b80b2af0 0x7f32b80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f32b80b2af0 0x7f32b80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44742/0 (socket says 172.21.15.157:44742) 2023-11-06T03:16:26.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 -- 172.21.15.157:0/2439332276 learned_addr learned my addr 172.21.15.157:0/2439332276 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32be226640 1 --1- 172.21.15.157:0/2439332276 >> v1:172.21.15.157:6789/0 conn(0x7f32b81599c0 0x7f32b80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55040/0 (socket says 172.21.15.157:55040) 2023-11-06T03:16:26.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bea27640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 0x7f32b8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bd224640 1 -- 172.21.15.157:0/2439332276 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014857434 0 0) 0x7f32b815e440 con 0x7f32b81599c0 2023-11-06T03:16:26.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bd224640 1 -- 172.21.15.157:0/2439332276 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4005740 con 0x7f32b81599c0 2023-11-06T03:16:26.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 -- 172.21.15.157:0/2439332276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 msgr2=0x7f32b8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:26.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 0x7f32b8159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 -- 172.21.15.157:0/2439332276 >> v1:172.21.15.157:6789/0 conn(0x7f32b81599c0 legacy=0x7f32b80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 -- 172.21.15.157:0/2439332276 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b806c800 con 0x7f32b80b2af0 2023-11-06T03:16:26.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bea27640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 0x7f32b8159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:26.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bda25640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b2af0 0x7f32b80ae000 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009870 tx=0x7f32a801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e3c73ca2cf5b58 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bd224640 1 -- 172.21.15.157:0/2439332276 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80a95f0 con 0x7f32b80b2af0 2023-11-06T03:16:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bd224640 1 -- 172.21.15.157:0/2439332276 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a80a9750 con 0x7f32b80b2af0 2023-11-06T03:16:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32bd224640 1 -- 172.21.15.157:0/2439332276 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80a9ac0 con 0x7f32b80b2af0 2023-11-06T03:16:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32c048a640 1 -- 172.21.15.157:0/2439332276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b2af0 msgr2=0x7f32b80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.823+0000 7f32c048a640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b2af0 0x7f32b80ae000 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009870 tx=0x7f32a801c510 comp rx=0 tx=0).stop 2023-11-06T03:16:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2439332276 shutdown_connections 2023-11-06T03:16:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b806d000 0x7f32b8159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 --2- 172.21.15.157:0/2439332276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b80b2af0 0x7f32b80ae000 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2439332276 >> 172.21.15.157:0/2439332276 conn(0x7f32b8065930 msgr2=0x7f32b8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2439332276 shutdown_connections 2023-11-06T03:16:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2439332276 wait complete. 2023-11-06T03:16:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 Processor -- start 2023-11-06T03:16:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- start start 2023-11-06T03:16:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b80b2af0 0x7f32b814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32b81599c0 0x7f32b8153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8152cb0 con 0x7f32b806d000 2023-11-06T03:16:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8152f90 con 0x7f32b81599c0 2023-11-06T03:16:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8153270 con 0x7f32b80b2af0 2023-11-06T03:16:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44750/0 (socket says 172.21.15.157:44750) 2023-11-06T03:16:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 -- 172.21.15.157:0/2176783681 learned_addr learned my addr 172.21.15.157:0/2176783681 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32bea27640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32b81599c0 0x7f32b8153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 -- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b80b2af0 msgr2=0x7f32b814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32bda25640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b80b2af0 0x7f32b814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b80b2af0 0x7f32b814e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 -- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32b81599c0 msgr2=0x7f32b8153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:26.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32b81599c0 0x7f32b8153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:26.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 -- 172.21.15.157:0/2176783681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a80094d0 con 0x7f32b806d000 2023-11-06T03:16:26.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32be226640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c430 tx=0x7f32b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40074e0 con 0x7f32b806d000 2023-11-06T03:16:26.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4007660 con 0x7f32b806d000 2023-11-06T03:16:26.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b8153550 con 0x7f32b806d000 2023-11-06T03:16:26.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4004a30 con 0x7f32b806d000 2023-11-06T03:16:26.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b8152400 con 0x7f32b806d000 2023-11-06T03:16:26.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.827+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8072750 con 0x7f32b806d000 2023-11-06T03:16:26.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.831+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f32b4004b90 con 0x7f32b806d000 2023-11-06T03:16:26.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.831+0000 7f32aeffd640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 0x7f329c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:26.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.831+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f32b4082df0 con 0x7f32b806d000 2023-11-06T03:16:26.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.831+0000 7f32bda25640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 0x7f329c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:26.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.831+0000 7f32bda25640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 0x7f329c067f20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ae000 tx=0x7f32a80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:26.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:26.835+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f32b4050680 con 0x7f32b806d000 2023-11-06T03:16:27.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.023+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f32b80cb110 con 0x7f32b806d000 2023-11-06T03:16:27.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32aeffd640 1 -- 172.21.15.157:0/2176783681 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f32b4053ca0 con 0x7f32b806d000 2023-11-06T03:16:27.023 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:27.023 INFO:teuthology.orchestra.run.smithi157.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-06T03:16:27.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 msgr2=0x7f329c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 0x7f329c067f20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ae000 tx=0x7f32a80a7000 comp rx=0 tx=0).stop 2023-11-06T03:16:27.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 msgr2=0x7f32b81516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c430 tx=0x7f32b400c8f0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 shutdown_connections 2023-11-06T03:16:27.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f329c065a60 0x7f329c067f20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32b81599c0 0x7f32b8153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32b80b2af0 0x7f32b814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 --2- 172.21.15.157:0/2176783681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32b806d000 0x7f32b81516b0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 >> 172.21.15.157:0/2176783681 conn(0x7f32b8065930 msgr2=0x7f32b8065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 shutdown_connections 2023-11-06T03:16:27.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.027+0000 7f32c048a640 1 -- 172.21.15.157:0/2176783681 wait complete. 2023-11-06T03:16:27.042 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:16:27.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:16:27.096 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:16:27.144 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:16:27.145 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:16:27.145 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:27.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:27.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.319+0000 7f420bfe7640 1 Processor -- start 2023-11-06T03:16:27.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.319+0000 7f420bfe7640 1 -- start start 2023-11-06T03:16:27.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42041599c0 0x7f42040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f42040b2af0 0x7f4204159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420415e440 con 0x7f420406d000 2023-11-06T03:16:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420406c240 con 0x7f42041599c0 2023-11-06T03:16:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420406c520 con 0x7f42040b2af0 2023-11-06T03:16:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42041599c0 0x7f42040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f42041599c0 0x7f42040ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44764/0 (socket says 172.21.15.157:44764) 2023-11-06T03:16:27.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 -- 172.21.15.157:0/1148956613 learned_addr learned my addr 172.21.15.157:0/1148956613 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:27.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 -- 172.21.15.157:0/1148956613 >> v2:172.21.15.195:3300/0 conn(0x7f42040b2af0 msgr2=0x7f4204159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:27.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420a584640 1 --2- 172.21.15.157:0/1148956613 >> v2:172.21.15.195:3300/0 conn(0x7f42040b2af0 0x7f4204159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 --2- 172.21.15.157:0/1148956613 >> v2:172.21.15.195:3300/0 conn(0x7f42040b2af0 0x7f4204159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 -- 172.21.15.157:0/1148956613 >> v1:172.21.15.195:6789/0 conn(0x7f420406d000 legacy=0x7f42040b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:27.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 -- 172.21.15.157:0/1148956613 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420406c800 con 0x7f42041599c0 2023-11-06T03:16:27.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4209d83640 1 --2- 172.21.15.157:0/1148956613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041599c0 0x7f42040ae000 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f42040b19a0 tx=0x7f420001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7123d2fa0952a690 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4208d81640 1 -- 172.21.15.157:0/1148956613 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000aa610 con 0x7f42041599c0 2023-11-06T03:16:27.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4208d81640 1 -- 172.21.15.157:0/1148956613 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42000a6040 con 0x7f42041599c0 2023-11-06T03:16:27.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f4208d81640 1 -- 172.21.15.157:0/1148956613 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000aa990 con 0x7f42041599c0 2023-11-06T03:16:27.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- 172.21.15.157:0/1148956613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041599c0 msgr2=0x7f42040ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 --2- 172.21.15.157:0/1148956613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041599c0 0x7f42040ae000 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f42040b19a0 tx=0x7f420001c490 comp rx=0 tx=0).stop 2023-11-06T03:16:27.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- 172.21.15.157:0/1148956613 shutdown_connections 2023-11-06T03:16:27.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 --2- 172.21.15.157:0/1148956613 >> v2:172.21.15.195:3300/0 conn(0x7f42040b2af0 0x7f4204159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 --2- 172.21.15.157:0/1148956613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42041599c0 0x7f42040ae000 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- 172.21.15.157:0/1148956613 >> 172.21.15.157:0/1148956613 conn(0x7f4204065930 msgr2=0x7f4204065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- 172.21.15.157:0/1148956613 shutdown_connections 2023-11-06T03:16:27.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.323+0000 7f420bfe7640 1 -- 172.21.15.157:0/1148956613 wait complete. 2023-11-06T03:16:27.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 Processor -- start 2023-11-06T03:16:27.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- start start 2023-11-06T03:16:27.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 0x7f42041f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42041599c0 0x7f420414e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cf240 con 0x7f420406d000 2023-11-06T03:16:27.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cec80 con 0x7f42041599c0 2023-11-06T03:16:27.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cef60 con 0x7f42040b2af0 2023-11-06T03:16:27.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44770/0 (socket says 172.21.15.157:44770) 2023-11-06T03:16:27.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 -- 172.21.15.157:0/2572526241 learned_addr learned my addr 172.21.15.157:0/2572526241 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:27.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420a584640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42041599c0 0x7f420414e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209582640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 0x7f42041f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 -- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 msgr2=0x7f42041f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 0x7f42041f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 -- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42041599c0 msgr2=0x7f420414e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42041599c0 0x7f420414e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 -- 172.21.15.157:0/2572526241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000094d0 con 0x7f420406d000 2023-11-06T03:16:27.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209582640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 0x7f42041f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:27.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f4209d83640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f420001a0c0 tx=0x7f420001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000bd5f0 con 0x7f420406d000 2023-11-06T03:16:27.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42000bd770 con 0x7f420406d000 2023-11-06T03:16:27.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42041f8ab0 con 0x7f420406d000 2023-11-06T03:16:27.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000bdb20 con 0x7f420406d000 2023-11-06T03:16:27.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42041f97f0 con 0x7f420406d000 2023-11-06T03:16:27.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.327+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4204072750 con 0x7f420406d000 2023-11-06T03:16:27.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.331+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f42000b33f0 con 0x7f420406d000 2023-11-06T03:16:27.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.331+0000 7f41fa7fc640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 0x7f41e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.331+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f420011de50 con 0x7f420406d000 2023-11-06T03:16:27.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.331+0000 7f4209582640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 0x7f41e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.331+0000 7f4209582640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 0x7f41e8067f20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4204158980 tx=0x7f41f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.335+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f42000eb700 con 0x7f420406d000 2023-11-06T03:16:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.491+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42040cb110 con 0x7f420406d000 2023-11-06T03:16:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.491+0000 7f41fa7fc640 1 -- 172.21.15.157:0/2572526241 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+18086 (secure 0 0 0) 0x7f42000eed20 con 0x7f420406d000 2023-11-06T03:16:27.490 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":151,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:50.867833+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:27.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 msgr2=0x7f41e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 0x7f41e8067f20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4204158980 tx=0x7f41f4009040 comp rx=0 tx=0).stop 2023-11-06T03:16:27.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 msgr2=0x7f42040ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f420001a0c0 tx=0x7f420001acd0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 shutdown_connections 2023-11-06T03:16:27.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f41e8065a60 0x7f41e8067f20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42041599c0 0x7f420414e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f42040b2af0 0x7f42041f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 --2- 172.21.15.157:0/2572526241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f420406d000 0x7f42040ce5c0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 >> 172.21.15.157:0/2572526241 conn(0x7f4204065930 msgr2=0x7f4204070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 shutdown_connections 2023-11-06T03:16:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.495+0000 7f420bfe7640 1 -- 172.21.15.157:0/2572526241 wait complete. 2023-11-06T03:16:27.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:15:46.459717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:15:46.807462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:16:27.512 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:16:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 Processor -- start 2023-11-06T03:16:27.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 -- start start 2023-11-06T03:16:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7281551e0 0x7fb72806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7280b3b20 0x7fb72815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72815f580 con 0x7fb72815e5e0 2023-11-06T03:16:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72815efc0 con 0x7fb7281551e0 2023-11-06T03:16:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72fcce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72815f2a0 con 0x7fb7280b3b20 2023-11-06T03:16:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7281551e0 0x7fb72806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72d269640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb72815e5e0 0x7fb7280b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:55042/0 (socket says 172.21.15.157:55042) 2023-11-06T03:16:27.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72d269640 1 -- 172.21.15.157:0/328873006 learned_addr learned my addr 172.21.15.157:0/328873006 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72e26b640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7280b3b20 0x7fb72815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 -- 172.21.15.157:0/328873006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7280b3b20 msgr2=0x7fb72815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7280b3b20 0x7fb72815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 -- 172.21.15.157:0/328873006 >> v1:172.21.15.157:6789/0 conn(0x7fb72815e5e0 legacy=0x7fb7280b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 -- 172.21.15.157:0/328873006 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72806f5a0 con 0x7fb7281551e0 2023-11-06T03:16:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72ca68640 1 -- 172.21.15.157:0/328873006 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358339735 0 0) 0x7fb72815f580 con 0x7fb72815e5e0 2023-11-06T03:16:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.639+0000 7fb72da6a640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7281551e0 0x7fb72806fda0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb7240097c0 tx=0x7fb72401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b36e22c139c5ff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72ca68640 1 -- 172.21.15.157:0/328873006 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240ac640 con 0x7fb7281551e0 2023-11-06T03:16:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72ca68640 1 -- 172.21.15.157:0/328873006 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7240ac7a0 con 0x7fb7281551e0 2023-11-06T03:16:27.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72ca68640 1 -- 172.21.15.157:0/328873006 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240acb30 con 0x7fb7281551e0 2023-11-06T03:16:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/328873006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7281551e0 msgr2=0x7fb72806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7281551e0 0x7fb72806fda0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fb7240097c0 tx=0x7fb72401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/328873006 shutdown_connections 2023-11-06T03:16:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7280b3b20 0x7fb72815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- 172.21.15.157:0/328873006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7281551e0 0x7fb72806fda0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/328873006 >> 172.21.15.157:0/328873006 conn(0x7fb728065930 msgr2=0x7fb728065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/328873006 shutdown_connections 2023-11-06T03:16:27.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/328873006 wait complete. 2023-11-06T03:16:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 Processor -- start 2023-11-06T03:16:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- start start 2023-11-06T03:16:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 0x7fb72806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7281551e0 0x7fb72806b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72806d4b0 con 0x7fb72815e5e0 2023-11-06T03:16:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72806bc30 con 0x7fb7280b3b20 2023-11-06T03:16:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72806bed0 con 0x7fb7281551e0 2023-11-06T03:16:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44800/0 (socket says 172.21.15.157:44800) 2023-11-06T03:16:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 -- 172.21.15.157:0/3944948742 learned_addr learned my addr 172.21.15.157:0/3944948742 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72da6a640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 0x7fb72806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72d269640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7281551e0 0x7fb72806b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 -- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7281551e0 msgr2=0x7fb72806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7281551e0 0x7fb72806b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 -- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 msgr2=0x7fb72806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 0x7fb72806cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 -- 172.21.15.157:0/3944948742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724009420 con 0x7fb72815e5e0 2023-11-06T03:16:27.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72da6a640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 0x7fb72806cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:27.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72e26b640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fb720007e30 tx=0x7fb72000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720014620 con 0x7fb72815e5e0 2023-11-06T03:16:27.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000ee90 con 0x7fb72815e5e0 2023-11-06T03:16:27.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72806c1b0 con 0x7fb72815e5e0 2023-11-06T03:16:27.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200149c0 con 0x7fb72815e5e0 2023-11-06T03:16:27.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.643+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72806c770 con 0x7fb72815e5e0 2023-11-06T03:16:27.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb728072750 con 0x7fb72815e5e0 2023-11-06T03:16:27.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb720014b20 con 0x7fb72815e5e0 2023-11-06T03:16:27.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb71e7fc640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 0x7fb70c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fb720083c20 con 0x7fb72815e5e0 2023-11-06T03:16:27.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb72da6a640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 0x7fb70c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.647+0000 7fb72da6a640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 0x7fb70c067f20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7240ae000 tx=0x7fb724008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.651+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb7200514b0 con 0x7fb72815e5e0 2023-11-06T03:16:27.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.803+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb728071730 con 0x7fb72815e5e0 2023-11-06T03:16:27.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.803+0000 7fb71e7fc640 1 -- 172.21.15.157:0/3944948742 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb720054ad0 con 0x7fb72815e5e0 2023-11-06T03:16:27.802 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:16:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 msgr2=0x7fb70c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 0x7fb70c067f20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7240ae000 tx=0x7fb724008040 comp rx=0 tx=0).stop 2023-11-06T03:16:27.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 msgr2=0x7fb728152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fb720007e30 tx=0x7fb72000e4b0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 shutdown_connections 2023-11-06T03:16:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb70c065a60 0x7fb70c067f20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72815e5e0 0x7fb728152da0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7281551e0 0x7fb72806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 --2- 172.21.15.157:0/3944948742 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7280b3b20 0x7fb72806cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 >> 172.21.15.157:0/3944948742 conn(0x7fb728065930 msgr2=0x7fb72814dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 shutdown_connections 2023-11-06T03:16:27.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.807+0000 7fb72fcce640 1 -- 172.21.15.157:0/3944948742 wait complete. 2023-11-06T03:16:27.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:16:27.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.939+0000 7f2cf3c82640 1 Processor -- start 2023-11-06T03:16:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.939+0000 7f2cf3c82640 1 -- start start 2023-11-06T03:16:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf3c82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf3c82640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf3c82640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0d2920 con 0x7f2cec15caa0 2023-11-06T03:16:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf3c82640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0d0520 con 0x7f2cec0d00b0 2023-11-06T03:16:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf3c82640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0d18a0 con 0x7f2cec15ce80 2023-11-06T03:16:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf221f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf221f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:36870/0 (socket says 172.21.15.157:36870) 2023-11-06T03:16:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:42536/0 (socket says 172.21.15.157:42536) 2023-11-06T03:16:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf1a1e640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f2cec15caa0 0x7f2cec0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:33694/0 (socket says 172.21.15.157:33694) 2023-11-06T03:16:27.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf221f640 1 -- 172.21.15.157:0/3286492227 learned_addr learned my addr 172.21.15.157:0/3286492227 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:27.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 -- 172.21.15.157:0/3286492227 >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 msgr2=0x7f2cec0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 --2- 172.21.15.157:0/3286492227 >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 -- 172.21.15.157:0/3286492227 >> v1:172.21.15.195:6789/0 conn(0x7f2cec15caa0 legacy=0x7f2cec0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf121d640 1 -- 172.21.15.157:0/3286492227 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0d1b60 con 0x7f2cec15ce80 2023-11-06T03:16:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.943+0000 7f2cf221f640 1 --2- 172.21.15.157:0/3286492227 >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:27.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf121d640 1 --2- 172.21.15.157:0/3286492227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097c0 tx=0x7f2cdc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e851067e9f7442 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf0a1c640 1 -- 172.21.15.157:0/3286492227 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0ab650 con 0x7f2cec15ce80 2023-11-06T03:16:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf0a1c640 1 -- 172.21.15.157:0/3286492227 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc0ab7b0 con 0x7f2cec15ce80 2023-11-06T03:16:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf0a1c640 1 -- 172.21.15.157:0/3286492227 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0abb40 con 0x7f2cec15ce80 2023-11-06T03:16:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3286492227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 msgr2=0x7f2cec0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3286492227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097c0 tx=0x7f2cdc01c450 comp rx=0 tx=0).stop 2023-11-06T03:16:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3286492227 shutdown_connections 2023-11-06T03:16:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3286492227 >> v2:172.21.15.195:3300/0 conn(0x7f2cec0d00b0 0x7f2cec0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3286492227 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec15ce80 0x7f2cec0cfb70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3286492227 >> 172.21.15.157:0/3286492227 conn(0x7f2cec070790 msgr2=0x7f2cec070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:27.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3286492227 shutdown_connections 2023-11-06T03:16:27.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3286492227 wait complete. 2023-11-06T03:16:27.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 Processor -- start 2023-11-06T03:16:27.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- start start 2023-11-06T03:16:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec0d00b0 0x7f2cec14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cec15caa0 0x7f2cec154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c8fe0 con 0x7f2cec15ce80 2023-11-06T03:16:27.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c80f0 con 0x7f2cec15caa0 2023-11-06T03:16:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c83d0 con 0x7f2cec0d00b0 2023-11-06T03:16:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44804/0 (socket says 172.21.15.157:44804) 2023-11-06T03:16:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 -- 172.21.15.157:0/3259352033 learned_addr learned my addr 172.21.15.157:0/3259352033 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:27.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf1a1e640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec0d00b0 0x7f2cec14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf121d640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cec15caa0 0x7f2cec154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 -- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec0d00b0 msgr2=0x7f2cec14cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec0d00b0 0x7f2cec14cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 -- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cec15caa0 msgr2=0x7f2cec154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:27.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cec15caa0 0x7f2cec154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 -- 172.21.15.157:0/3259352033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009b30 con 0x7f2cec15ce80 2023-11-06T03:16:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf221f640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400efc0 tx=0x7f2ce400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4018070 con 0x7f2cec15ce80 2023-11-06T03:16:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce400f040 con 0x7f2cec15ce80 2023-11-06T03:16:27.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc009420 con 0x7f2cec15ce80 2023-11-06T03:16:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400ce00 con 0x7f2cec15ce80 2023-11-06T03:16:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.947+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec0c8990 con 0x7f2cec15ce80 2023-11-06T03:16:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec0c9150 con 0x7f2cec15ce80 2023-11-06T03:16:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2ce4014480 con 0x7f2cec15ce80 2023-11-06T03:16:27.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2ce27fc640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 0x7f2cd0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f2ce4083660 con 0x7f2cec15ce80 2023-11-06T03:16:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2cf1a1e640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 0x7f2cd0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:27.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.951+0000 7f2cf1a1e640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 0x7f2cd0067f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000f70 tx=0x7f2ce8007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:27.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:27.955+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2ce4050e70 con 0x7f2cec15ce80 2023-11-06T03:16:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.095+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2cec072750 con 0x7f2cec15ce80 2023-11-06T03:16:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.095+0000 7f2ce27fc640 1 -- 172.21.15.157:0/3259352033 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ce4054490 con 0x7f2cec15ce80 2023-11-06T03:16:28.094 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:16:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 msgr2=0x7f2cd0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 0x7f2cd0067f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8000f70 tx=0x7f2ce8007c90 comp rx=0 tx=0).stop 2023-11-06T03:16:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 msgr2=0x7f2cec154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400efc0 tx=0x7f2ce400c490 comp rx=0 tx=0).stop 2023-11-06T03:16:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 shutdown_connections 2023-11-06T03:16:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2cd0065a60 0x7f2cd0067f20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cec15ce80 0x7f2cec154d90 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2cec15caa0 0x7f2cec154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 --2- 172.21.15.157:0/3259352033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2cec0d00b0 0x7f2cec14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 >> 172.21.15.157:0/3259352033 conn(0x7f2cec070790 msgr2=0x7f2cec0b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 shutdown_connections 2023-11-06T03:16:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.099+0000 7f2cf3c82640 1 -- 172.21.15.157:0/3259352033 wait complete. 2023-11-06T03:16:28.112 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:16:28.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.223+0000 7f276a6cd640 1 Processor -- start 2023-11-06T03:16:28.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.223+0000 7f276a6cd640 1 -- start start 2023-11-06T03:16:28.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27641551e0 0x7f276406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f276415e5e0 0x7f27640b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f276415ea90 0x7f27640b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276415f580 con 0x7f27641551e0 2023-11-06T03:16:28.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276415efc0 con 0x7f276415e5e0 2023-11-06T03:16:28.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f276a6cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276415f2a0 con 0x7f276415ea90 2023-11-06T03:16:28.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27641551e0 0x7f276406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f27641551e0 0x7f276406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44814/0 (socket says 172.21.15.157:44814) 2023-11-06T03:16:28.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 -- 172.21.15.157:0/215492532 learned_addr learned my addr 172.21.15.157:0/215492532 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:28.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2768c6a640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f276415ea90 0x7f27640b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f27637fe640 1 --2- 172.21.15.157:0/215492532 >> v2:172.21.15.195:3300/0 conn(0x7f276415e5e0 0x7f27640b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 -- 172.21.15.157:0/215492532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f276415ea90 msgr2=0x7f27640b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:28.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f276415ea90 0x7f27640b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 -- 172.21.15.157:0/215492532 >> v2:172.21.15.195:3300/0 conn(0x7f276415e5e0 msgr2=0x7f27640b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:28.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 --2- 172.21.15.157:0/215492532 >> v2:172.21.15.195:3300/0 conn(0x7f276415e5e0 0x7f27640b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 -- 172.21.15.157:0/215492532 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276406f5a0 con 0x7f27641551e0 2023-11-06T03:16:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2763fff640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27641551e0 0x7f276406fda0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f2764066880 tx=0x7f275401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f07ba964ce35fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2762ffd640 1 -- 172.21.15.157:0/215492532 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27540ab620 con 0x7f27641551e0 2023-11-06T03:16:28.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2762ffd640 1 -- 172.21.15.157:0/215492532 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27540ab780 con 0x7f27641551e0 2023-11-06T03:16:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.227+0000 7f2762ffd640 1 -- 172.21.15.157:0/215492532 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27540abb10 con 0x7f27641551e0 2023-11-06T03:16:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/215492532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27641551e0 msgr2=0x7f276406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27641551e0 0x7f276406fda0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f2764066880 tx=0x7f275401c440 comp rx=0 tx=0).stop 2023-11-06T03:16:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/215492532 shutdown_connections 2023-11-06T03:16:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f276415ea90 0x7f27640b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- 172.21.15.157:0/215492532 >> v2:172.21.15.195:3300/0 conn(0x7f276415e5e0 0x7f27640b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- 172.21.15.157:0/215492532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27641551e0 0x7f276406fda0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/215492532 >> 172.21.15.157:0/215492532 conn(0x7f2764065930 msgr2=0x7f2764065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/215492532 shutdown_connections 2023-11-06T03:16:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/215492532 wait complete. 2023-11-06T03:16:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 Processor -- start 2023-11-06T03:16:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- start start 2023-11-06T03:16:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 0x7f27641530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276415ea90 0x7f2764147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27641543f0 con 0x7f276415e5e0 2023-11-06T03:16:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27641450a0 con 0x7f276415ea90 2023-11-06T03:16:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2764153770 con 0x7f27641551e0 2023-11-06T03:16:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44826/0 (socket says 172.21.15.157:44826) 2023-11-06T03:16:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 -- 172.21.15.157:0/2731675674 learned_addr learned my addr 172.21.15.157:0/2731675674 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2768c6a640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276415ea90 0x7f2764147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2763fff640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 0x7f27641530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 -- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 msgr2=0x7f27641530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 0x7f27641530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 -- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276415ea90 msgr2=0x7f2764147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276415ea90 0x7f2764147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 -- 172.21.15.157:0/2731675674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540094d0 con 0x7f276415e5e0 2023-11-06T03:16:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2763fff640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 0x7f27641530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f27637fe640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f275800d800 tx=0x7f275800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27580096a0 con 0x7f276415e5e0 2023-11-06T03:16:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2758009800 con 0x7f276415e5e0 2023-11-06T03:16:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2764153a50 con 0x7f276415e5e0 2023-11-06T03:16:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758009b70 con 0x7f276415e5e0 2023-11-06T03:16:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.231+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2764154010 con 0x7f276415e5e0 2023-11-06T03:16:28.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2764154530 con 0x7f276415e5e0 2023-11-06T03:16:28.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2758009cd0 con 0x7f276415e5e0 2023-11-06T03:16:28.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f2760ff9640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 0x7f2750067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f27580839c0 con 0x7f276415e5e0 2023-11-06T03:16:28.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f2763fff640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 0x7f2750067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.235+0000 7f2763fff640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 0x7f2750067f50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27540af000 tx=0x7f2754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.239+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2764154530 con 0x7f276415e5e0 2023-11-06T03:16:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.387+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2764154530 con 0x7f276415e5e0 2023-11-06T03:16:28.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.387+0000 7f2760ff9640 1 -- 172.21.15.157:0/2731675674 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2764154530 con 0x7f276415e5e0 2023-11-06T03:16:28.387 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:16:28.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 msgr2=0x7f2750067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 0x7f2750067f50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f27540af000 tx=0x7f2754008040 comp rx=0 tx=0).stop 2023-11-06T03:16:28.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 msgr2=0x7f2764144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f275800d800 tx=0x7f275800dcc0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 shutdown_connections 2023-11-06T03:16:28.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2750065a90 0x7f2750067f50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f276415ea90 0x7f2764147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f276415e5e0 0x7f2764144b60 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 --2- 172.21.15.157:0/2731675674 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f27641551e0 0x7f27641530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 >> 172.21.15.157:0/2731675674 conn(0x7f2764065930 msgr2=0x7f276415b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 shutdown_connections 2023-11-06T03:16:28.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.391+0000 7f276a6cd640 1 -- 172.21.15.157:0/2731675674 wait complete. 2023-11-06T03:16:28.405 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:16:28.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 Processor -- start 2023-11-06T03:16:28.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 -- start start 2023-11-06T03:16:28.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3a006d000 0x7fe3a0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a015e440 con 0x7fe3a01599c0 2023-11-06T03:16:28.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a006c240 con 0x7fe3a006d000 2023-11-06T03:16:28.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe3a619c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a006c520 con 0x7fe3a00b2af0 2023-11-06T03:16:28.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3a006d000 0x7fe3a0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe3a01599c0 0x7fe3a00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38054/0 (socket says 172.21.15.157:38054) 2023-11-06T03:16:28.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39f7fe640 1 -- 172.21.15.157:0/3953777630 learned_addr learned my addr 172.21.15.157:0/3953777630 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:28.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39effd640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39ffff640 1 -- 172.21.15.157:0/3953777630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 msgr2=0x7fe3a00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:28.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39ffff640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39ffff640 1 -- 172.21.15.157:0/3953777630 >> v1:172.21.15.157:6789/0 conn(0x7fe3a01599c0 legacy=0x7fe3a00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39ffff640 1 -- 172.21.15.157:0/3953777630 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a006c800 con 0x7fe3a006d000 2023-11-06T03:16:28.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39e7fc640 1 -- 172.21.15.157:0/3953777630 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279676870 0 0) 0x7fe3a015e440 con 0x7fe3a01599c0 2023-11-06T03:16:28.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.523+0000 7fe39effd640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:28.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a006d000 0x7fe3a0159da0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa36757c31e07e48 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39e7fc640 1 -- 172.21.15.157:0/3953777630 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980a5a00 con 0x7fe3a006d000 2023-11-06T03:16:28.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39e7fc640 1 -- 172.21.15.157:0/3953777630 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980a5b60 con 0x7fe3a006d000 2023-11-06T03:16:28.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39e7fc640 1 -- 172.21.15.157:0/3953777630 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980ad440 con 0x7fe3a006d000 2023-11-06T03:16:28.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- 172.21.15.157:0/3953777630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a006d000 msgr2=0x7fe3a0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a006d000 0x7fe3a0159da0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c920 comp rx=0 tx=0).stop 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- 172.21.15.157:0/3953777630 shutdown_connections 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a006d000 0x7fe3a0159da0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- 172.21.15.157:0/3953777630 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- 172.21.15.157:0/3953777630 >> 172.21.15.157:0/3953777630 conn(0x7fe3a0065930 msgr2=0x7fe3a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- 172.21.15.157:0/3953777630 shutdown_connections 2023-11-06T03:16:28.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- 172.21.15.157:0/3953777630 wait complete. 2023-11-06T03:16:28.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 Processor -- start 2023-11-06T03:16:28.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- start start 2023-11-06T03:16:28.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3a006d000 0x7fe3a01430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a01520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00aef00 con 0x7fe3a01599c0 2023-11-06T03:16:28.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0156870 con 0x7fe3a006d000 2023-11-06T03:16:28.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe3a619c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a014f4f0 con 0x7fe3a00b2af0 2023-11-06T03:16:28.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56384/0 (socket says 172.21.15.157:56384) 2023-11-06T03:16:28.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 -- 172.21.15.157:0/4158389498 learned_addr learned my addr 172.21.15.157:0/4158389498 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:28.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39effd640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a01520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39f7fe640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3a006d000 0x7fe3a01430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 -- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 msgr2=0x7fe3a01520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a01520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 -- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3a006d000 msgr2=0x7fe3a01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3a006d000 0x7fe3a01430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.527+0000 7fe39ffff640 1 -- 172.21.15.157:0/4158389498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009a70 con 0x7fe3a01599c0 2023-11-06T03:16:28.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe39ffff640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fe39801afd0 tx=0x7fe3980196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980add90 con 0x7fe3a01599c0 2023-11-06T03:16:28.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00ae140 con 0x7fe3a01599c0 2023-11-06T03:16:28.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980b7410 con 0x7fe3a01599c0 2023-11-06T03:16:28.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980b77a0 con 0x7fe3a01599c0 2023-11-06T03:16:28.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a00ae700 con 0x7fe3a01599c0 2023-11-06T03:16:28.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0072750 con 0x7fe3a01599c0 2023-11-06T03:16:28.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe3980b4050 con 0x7fe3a01599c0 2023-11-06T03:16:28.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 0x7fe38c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fe39811f0c0 con 0x7fe3a01599c0 2023-11-06T03:16:28.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe39f7fe640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 0x7fe38c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.531+0000 7fe39f7fe640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 0x7fe38c067f20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f70 tx=0x7fe390009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.535+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe3980ec980 con 0x7fe3a01599c0 2023-11-06T03:16:28.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.679+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3a0070360 con 0x7fe3a01599c0 2023-11-06T03:16:28.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.679+0000 7fe387fff640 1 -- 172.21.15.157:0/4158389498 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3980b20a0 con 0x7fe3a01599c0 2023-11-06T03:16:28.677 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:16:28.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 msgr2=0x7fe38c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 0x7fe38c067f20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f70 tx=0x7fe390009040 comp rx=0 tx=0).stop 2023-11-06T03:16:28.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 msgr2=0x7fe3a01525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fe39801afd0 tx=0x7fe3980196c0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 shutdown_connections 2023-11-06T03:16:28.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe38c065a60 0x7fe38c067f20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3a01599c0 0x7fe3a01525f0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3a00b2af0 0x7fe3a01520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 --2- 172.21.15.157:0/4158389498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe3a006d000 0x7fe3a01430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 >> 172.21.15.157:0/4158389498 conn(0x7fe3a0065930 msgr2=0x7fe3a00b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 shutdown_connections 2023-11-06T03:16:28.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.683+0000 7fe3a619c640 1 -- 172.21.15.157:0/4158389498 wait complete. 2023-11-06T03:16:28.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:16:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 Processor -- start 2023-11-06T03:16:28.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 -- start start 2023-11-06T03:16:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 0x7fb2d806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2d815ea60 0x7fb2d80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815f580 con 0x7fb2d815ea60 2023-11-06T03:16:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815efc0 con 0x7fb2d81551e0 2023-11-06T03:16:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2df733640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815f2a0 con 0x7fb2d815e5e0 2023-11-06T03:16:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2d815ea60 0x7fb2d80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb2d815ea60 0x7fb2d80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56388/0 (socket says 172.21.15.157:56388) 2023-11-06T03:16:28.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 -- 172.21.15.157:0/760168643 learned_addr learned my addr 172.21.15.157:0/760168643 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:28.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2dccce640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2dd4cf640 1 --2- 172.21.15.157:0/760168643 >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 0x7fb2d806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 -- 172.21.15.157:0/760168643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 msgr2=0x7fb2d80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:28.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 -- 172.21.15.157:0/760168643 >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 msgr2=0x7fb2d806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:28.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 --2- 172.21.15.157:0/760168643 >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 0x7fb2d806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2ddcd0640 1 -- 172.21.15.157:0/760168643 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d806f5a0 con 0x7fb2d815ea60 2023-11-06T03:16:28.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.819+0000 7fb2dd4cf640 1 --2- 172.21.15.157:0/760168643 >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 0x7fb2d806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:28.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2ddcd0640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d815ea60 0x7fb2d80b3950 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8066830 tx=0x7fb2d001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717d93c410ebbf50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2cffff640 1 -- 172.21.15.157:0/760168643 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a57b0 con 0x7fb2d815ea60 2023-11-06T03:16:28.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2cffff640 1 -- 172.21.15.157:0/760168643 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d00a5910 con 0x7fb2d815ea60 2023-11-06T03:16:28.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2cffff640 1 -- 172.21.15.157:0/760168643 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a5ca0 con 0x7fb2d815ea60 2023-11-06T03:16:28.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- 172.21.15.157:0/760168643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d815ea60 msgr2=0x7fb2d80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d815ea60 0x7fb2d80b3950 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8066830 tx=0x7fb2d001c910 comp rx=0 tx=0).stop 2023-11-06T03:16:28.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- 172.21.15.157:0/760168643 shutdown_connections 2023-11-06T03:16:28.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d815ea60 0x7fb2d80b3950 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- 172.21.15.157:0/760168643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- 172.21.15.157:0/760168643 >> v2:172.21.15.195:3300/0 conn(0x7fb2d81551e0 0x7fb2d806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- 172.21.15.157:0/760168643 >> 172.21.15.157:0/760168643 conn(0x7fb2d8065930 msgr2=0x7fb2d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- 172.21.15.157:0/760168643 shutdown_connections 2023-11-06T03:16:28.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- 172.21.15.157:0/760168643 wait complete. 2023-11-06T03:16:28.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 Processor -- start 2023-11-06T03:16:28.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- start start 2023-11-06T03:16:28.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2d815ea60 0x7fb2d806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8152d00 con 0x7fb2d81551e0 2023-11-06T03:16:28.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d806ce10 con 0x7fb2d815ea60 2023-11-06T03:16:28.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2df733640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d814eb30 con 0x7fb2d815e5e0 2023-11-06T03:16:28.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56398/0 (socket says 172.21.15.157:56398) 2023-11-06T03:16:28.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 -- 172.21.15.157:0/4043592010 learned_addr learned my addr 172.21.15.157:0/4043592010 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:28.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dccce640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2ddcd0640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2d815ea60 0x7fb2d806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 -- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 msgr2=0x7fb2d81527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d81527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 -- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2d815ea60 msgr2=0x7fb2d806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2d815ea60 0x7fb2d806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 -- 172.21.15.157:0/4043592010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0009b30 con 0x7fb2d81551e0 2023-11-06T03:16:28.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.823+0000 7fb2dd4cf640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c7f0 tx=0x7fb2c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800cea0 con 0x7fb2d81551e0 2023-11-06T03:16:28.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8007a30 con 0x7fb2d81551e0 2023-11-06T03:16:28.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d814c6f0 con 0x7fb2d81551e0 2023-11-06T03:16:28.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8007dc0 con 0x7fb2d81551e0 2023-11-06T03:16:28.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d814f080 con 0x7fb2d81551e0 2023-11-06T03:16:28.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8072750 con 0x7fb2d81551e0 2023-11-06T03:16:28.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2c80113f0 con 0x7fb2d81551e0 2023-11-06T03:16:28.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 0x7fb2bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:28.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fb2c8083410 con 0x7fb2d81551e0 2023-11-06T03:16:28.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2dccce640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 0x7fb2bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:28.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.827+0000 7fb2dccce640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 0x7fb2bc067f20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb2d81519f0 tx=0x7fb2d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:28.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.831+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb2c8050ca0 con 0x7fb2d81551e0 2023-11-06T03:16:28.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb2d8071730 con 0x7fb2d81551e0 2023-11-06T03:16:28.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2cdffb640 1 -- 172.21.15.157:0/4043592010 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb2c80542c0 con 0x7fb2d81551e0 2023-11-06T03:16:28.984 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:16:28.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 msgr2=0x7fb2bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 0x7fb2bc067f20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb2d81519f0 tx=0x7fb2d4008040 comp rx=0 tx=0).stop 2023-11-06T03:16:28.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 msgr2=0x7fb2d814f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:28.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c7f0 tx=0x7fb2c800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 shutdown_connections 2023-11-06T03:16:28.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb2bc065a60 0x7fb2bc067f20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2d815ea60 0x7fb2d806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2d815e5e0 0x7fb2d81527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 --2- 172.21.15.157:0/4043592010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d81551e0 0x7fb2d814f5a0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:28.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 >> 172.21.15.157:0/4043592010 conn(0x7fb2d8065930 msgr2=0x7fb2d815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:28.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 shutdown_connections 2023-11-06T03:16:28.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:28.987+0000 7fb2df733640 1 -- 172.21.15.157:0/4043592010 wait complete. 2023-11-06T03:16:29.003 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:16:29.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:29.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:29.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[93757]: fuse finished with error 0 and tester_r 0 2023-11-06T03:16:29.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:16:29.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:29.016 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:16:29.078 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:16:29.078 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:29.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:16:29.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:16:29.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:29.086 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:16:29.131 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:16:29.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:16:29.133 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:16:29.134 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:29.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:16:29.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:16:29.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:29.141 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:16:29.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:16:29.188 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:16:29.188 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:16:29.189 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:29.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:16:29.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:16:29.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:29.196 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:16:29.246 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:16:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:16:29.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 Processor -- start 2023-11-06T03:16:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 -- start start 2023-11-06T03:16:29.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1838144430 0x7f18380b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380cef40 con 0x7f183815c290 2023-11-06T03:16:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380cd450 con 0x7f1838065d40 2023-11-06T03:16:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183e466640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380cbf80 con 0x7f1838144430 2023-11-06T03:16:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183ca03640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f183815c290 0x7f18380cb870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38064/0 (socket says 172.21.15.157:38064) 2023-11-06T03:16:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183ca03640 1 -- 172.21.15.157:0/2179830813 learned_addr learned my addr 172.21.15.157:0/2179830813 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799984357 0 0) 0x7f18380cef40 con 0x7f183815c290 2023-11-06T03:16:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1820003530 con 0x7f183815c290 2023-11-06T03:16:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 --2- 172.21.15.157:0/2179830813 >> v2:172.21.15.195:3300/0 conn(0x7f1838144430 0x7f18380b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641975818 0 0) 0x7f1820003530 con 0x7f183815c290 2023-11-06T03:16:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 >> v2:172.21.15.195:3300/0 conn(0x7f1838144430 msgr2=0x7f18380b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2179830813 >> v2:172.21.15.195:3300/0 conn(0x7f1838144430 0x7f18380b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 >> v1:172.21.15.195:6789/0 conn(0x7f1838065d40 legacy=0x7f18380cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380cc840 con 0x7f183815c290 2023-11-06T03:16:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1830001a00 con 0x7f183815c290 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 >> v1:172.21.15.157:6789/0 conn(0x7f183815c290 legacy=0x7f18380cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f1820003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18200047d0 0x7f1820004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1820005190 0x7f182000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380cbf80 con 0x7f1820003980 2023-11-06T03:16:29.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182000c0e0 con 0x7f1820005190 2023-11-06T03:16:29.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182000c2a0 con 0x7f18200047d0 2023-11-06T03:16:29.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f1820003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 -- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18200047d0 msgr2=0x7f1820004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:29.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18200047d0 0x7f1820004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 -- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1820005190 msgr2=0x7f182000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1820005190 0x7f182000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f1837fff640 1 -- 172.21.15.157:0/2179830813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820003530 con 0x7f1820003980 2023-11-06T03:16:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.375+0000 7f183ca03640 1 -- 172.21.15.157:0/2179830813 reap_dead start 2023-11-06T03:16:29.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f1837fff640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f1820003da0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f182800ba30 tx=0x7f182801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608b9831573cbc70 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280ac620 con 0x7f1820003980 2023-11-06T03:16:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18280a6040 con 0x7f1820003980 2023-11-06T03:16:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f1836ffd640 1 -- 172.21.15.157:0/2179830813 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280ac9c0 con 0x7f1820003980 2023-11-06T03:16:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 msgr2=0x7f1820003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f1820003da0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f182800ba30 tx=0x7f182801c6a0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- 172.21.15.157:0/2179830813 shutdown_connections 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 --2- 172.21.15.157:0/2179830813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f1820003da0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- 172.21.15.157:0/2179830813 >> 172.21.15.157:0/2179830813 conn(0x7f1838065930 msgr2=0x7f1838071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- 172.21.15.157:0/2179830813 shutdown_connections 2023-11-06T03:16:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- 172.21.15.157:0/2179830813 wait complete. 2023-11-06T03:16:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 Processor -- start 2023-11-06T03:16:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- start start 2023-11-06T03:16:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 0x7f183815e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f183815ee50 0x7f18380a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380b2c00 con 0x7f1820003980 2023-11-06T03:16:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183815f2d0 con 0x7f18380b3020 2023-11-06T03:16:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.379+0000 7f183e466640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18380b1f80 con 0x7f183815ee50 2023-11-06T03:16:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56418/0 (socket says 172.21.15.157:56418) 2023-11-06T03:16:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 -- 172.21.15.157:0/2692717487 learned_addr learned my addr 172.21.15.157:0/2692717487 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f183ca03640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f183815ee50 0x7f18380a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f18377fe640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 0x7f183815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 -- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f183815ee50 msgr2=0x7f18380a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f183815ee50 0x7f18380a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 -- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 msgr2=0x7f183815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 0x7f183815e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 -- 172.21.15.157:0/2692717487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380b2260 con 0x7f1820003980 2023-11-06T03:16:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f18377fe640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 0x7f183815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1837fff640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f182800b950 tx=0x7f1820005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828004220 con 0x7f1820003980 2023-11-06T03:16:29.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18280b9070 con 0x7f1820003980 2023-11-06T03:16:29.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280ac7c0 con 0x7f1820003980 2023-11-06T03:16:29.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18380b2540 con 0x7f1820003980 2023-11-06T03:16:29.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838140530 con 0x7f1820003980 2023-11-06T03:16:29.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1838072670 con 0x7f1820003980 2023-11-06T03:16:29.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f18280ac920 con 0x7f1820003980 2023-11-06T03:16:29.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 0x7f1824067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f182811d640 con 0x7f1820003980 2023-11-06T03:16:29.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f18377fe640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 0x7f1824067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.383+0000 7f18377fe640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 0x7f1824067f50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f18380aafe0 tx=0x7f182c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.387+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f18280eaed0 con 0x7f1820003980 2023-11-06T03:16:29.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18380b0820 con 0x7f1820003980 2023-11-06T03:16:29.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f1834ff9640 1 -- 172.21.15.157:0/2692717487 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f18280ee4f0 con 0x7f1820003980 2023-11-06T03:16:29.557 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:29.557 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:16:29.558 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:16:29.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 msgr2=0x7f1824067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 0x7f1824067f50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f18380aafe0 tx=0x7f182c009040 comp rx=0 tx=0).stop 2023-11-06T03:16:29.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 msgr2=0x7f183813fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.559+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f182800b950 tx=0x7f1820005800 comp rx=0 tx=0).stop 2023-11-06T03:16:29.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 shutdown_connections 2023-11-06T03:16:29.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1824065a90 0x7f1824067f50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f183815ee50 0x7f18380a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18380b3020 0x7f183815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 --2- 172.21.15.157:0/2692717487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1820003980 0x7f183813fff0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 >> 172.21.15.157:0/2692717487 conn(0x7f1838065930 msgr2=0x7f18380704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:29.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 shutdown_connections 2023-11-06T03:16:29.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.563+0000 7f183e466640 1 -- 172.21.15.157:0/2692717487 wait complete. 2023-11-06T03:16:29.574 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:29.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 Processor -- start 2023-11-06T03:16:29.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 -- start start 2023-11-06T03:16:29.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fad0015e5e0 0x7fad000b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0015f580 con 0x7fad0015ea60 2023-11-06T03:16:29.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0015efc0 con 0x7fad0015e5e0 2023-11-06T03:16:29.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.687+0000 7fad084c1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015f2a0 con 0x7fad001551e0 2023-11-06T03:16:29.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad05a5c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fad0015e5e0 0x7fad000b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:49620/0 (socket says 172.21.15.157:49620) 2023-11-06T03:16:29.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 -- 172.21.15.157:0/2863558384 learned_addr learned my addr 172.21.15.157:0/2863558384 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:29.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 -- 172.21.15.157:0/2863558384 >> v2:172.21.15.195:3300/0 conn(0x7fad0015e5e0 msgr2=0x7fad000b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:29.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 --2- 172.21.15.157:0/2863558384 >> v2:172.21.15.195:3300/0 conn(0x7fad0015e5e0 0x7fad000b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 -- 172.21.15.157:0/2863558384 >> v1:172.21.15.195:6789/0 conn(0x7fad0015ea60 legacy=0x7fad0015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 -- 172.21.15.157:0/2863558384 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0006f5a0 con 0x7fad001551e0 2023-11-06T03:16:29.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0625d640 1 --2- 172.21.15.157:0/2863558384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facfc009870 tx=0x7facfc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9161fcba2aa89c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0525b640 1 -- 172.21.15.157:0/2863558384 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0ad640 con 0x7fad001551e0 2023-11-06T03:16:29.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0525b640 1 -- 172.21.15.157:0/2863558384 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc0ad7a0 con 0x7fad001551e0 2023-11-06T03:16:29.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.691+0000 7fad0525b640 1 -- 172.21.15.157:0/2863558384 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0adb30 con 0x7fad001551e0 2023-11-06T03:16:29.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/2863558384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 msgr2=0x7fad0006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- 172.21.15.157:0/2863558384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facfc009870 tx=0x7facfc01c480 comp rx=0 tx=0).stop 2023-11-06T03:16:29.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/2863558384 shutdown_connections 2023-11-06T03:16:29.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- 172.21.15.157:0/2863558384 >> v2:172.21.15.195:3300/0 conn(0x7fad0015e5e0 0x7fad000b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- 172.21.15.157:0/2863558384 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad001551e0 0x7fad0006fda0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/2863558384 >> 172.21.15.157:0/2863558384 conn(0x7fad00065930 msgr2=0x7fad00065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:29.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/2863558384 shutdown_connections 2023-11-06T03:16:29.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/2863558384 wait complete. 2023-11-06T03:16:29.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 Processor -- start 2023-11-06T03:16:29.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- start start 2023-11-06T03:16:29.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 0x7fad000aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 0x7fad0014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00154830 con 0x7fad001551e0 2023-11-06T03:16:29.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000ce690 con 0x7fad0015ea60 2023-11-06T03:16:29.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad00153f00 con 0x7fad0015e5e0 2023-11-06T03:16:29.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56420/0 (socket says 172.21.15.157:56420) 2023-11-06T03:16:29.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 -- 172.21.15.157:0/1429286331 learned_addr learned my addr 172.21.15.157:0/1429286331 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:29.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad05a5c640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 0x7fad000aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad06a5e640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 0x7fad0014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 -- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 msgr2=0x7fad000aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 0x7fad000aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 -- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 msgr2=0x7fad0014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 0x7fad0014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 -- 172.21.15.157:0/1429286331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0094d0 con 0x7fad001551e0 2023-11-06T03:16:29.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad06a5e640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 0x7fad0014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:29.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad05a5c640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 0x7fad000aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:29.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad0625d640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7facfc01ab30 tx=0x7facfc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b8620 con 0x7fad001551e0 2023-11-06T03:16:29.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc0a6040 con 0x7fad001551e0 2023-11-06T03:16:29.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad001541e0 con 0x7fad001551e0 2023-11-06T03:16:29.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0b89c0 con 0x7fad001551e0 2023-11-06T03:16:29.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.695+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0014d1f0 con 0x7fad001551e0 2023-11-06T03:16:29.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad000af890 con 0x7fad001551e0 2023-11-06T03:16:29.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7facfc0b4040 con 0x7fad001551e0 2023-11-06T03:16:29.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7facf6ffd640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 0x7face4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:29.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7facfc11df90 con 0x7fad001551e0 2023-11-06T03:16:29.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7fad05a5c640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 0x7face4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:29.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.699+0000 7fad05a5c640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 0x7face4067f20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fad001f7550 tx=0x7facf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:29.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.703+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7facfc0eb8b0 con 0x7fad001551e0 2023-11-06T03:16:29.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.859+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad00072750 con 0x7fad001551e0 2023-11-06T03:16:29.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.859+0000 7facf6ffd640 1 -- 172.21.15.157:0/1429286331 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+18086 (secure 0 0 0) 0x7facfc0eeed0 con 0x7fad001551e0 2023-11-06T03:16:29.858 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:29.858 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":151,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:50.867833+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:29.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 msgr2=0x7face4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 0x7face4067f20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fad001f7550 tx=0x7facf0008040 comp rx=0 tx=0).stop 2023-11-06T03:16:29.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 msgr2=0x7fad0014d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:29.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7facfc01ab30 tx=0x7facfc019220 comp rx=0 tx=0).stop 2023-11-06T03:16:29.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 shutdown_connections 2023-11-06T03:16:29.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7face4065a60 0x7face4067f20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fad0015ea60 0x7fad0014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fad0015e5e0 0x7fad000aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 --2- 172.21.15.157:0/1429286331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad001551e0 0x7fad0014d500 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:29.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 >> 172.21.15.157:0/1429286331 conn(0x7fad00065930 msgr2=0x7fad000afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:29.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 shutdown_connections 2023-11-06T03:16:29.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.863+0000 7fad084c1640 1 -- 172.21.15.157:0/1429286331 wait complete. 2023-11-06T03:16:29.876 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:15:46.459717+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:15:46.807462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:16:29.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:16:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.999+0000 7f7aa2d3f640 1 Processor -- start 2023-11-06T03:16:29.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:29.999+0000 7f7aa2d3f640 1 -- start start 2023-11-06T03:16:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa2d3f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a9c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa2d3f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c0cc5f0 con 0x7f7a9c1597f0 2023-11-06T03:16:30.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa2d3f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c0cc8d0 con 0x7f7a9c0b0ce0 2023-11-06T03:16:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa2d3f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0cd170 con 0x7f7a9c159bd0 2023-11-06T03:16:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa0adb640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7a9c1597f0 0x7f7a9c15d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38078/0 (socket says 172.21.15.157:38078) 2023-11-06T03:16:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa0adb640 1 -- 172.21.15.157:0/3012687201 learned_addr learned my addr 172.21.15.157:0/3012687201 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a9c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295729628 0 0) 0x7f7a9c0cc5f0 con 0x7f7a9c1597f0 2023-11-06T03:16:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8c003530 con 0x7f7a9c1597f0 2023-11-06T03:16:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1148521941 0 0) 0x7f7a8c003530 con 0x7f7a9c1597f0 2023-11-06T03:16:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 msgr2=0x7f7a9c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a9c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 >> v1:172.21.15.195:6789/0 conn(0x7f7a9c0b0ce0 legacy=0x7f7a9c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0ccbb0 con 0x7f7a9c1597f0 2023-11-06T03:16:30.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7a90001680 con 0x7f7a9c1597f0 2023-11-06T03:16:30.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a9c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:30.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 >> v1:172.21.15.157:6789/0 conn(0x7f7a9c1597f0 legacy=0x7f7a9c15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c0039b0 0x7f7a8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a8c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004d10 0x7f7a8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0cd170 con 0x7f7a8c004d10 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c00b990 con 0x7f7a8c0039b0 2023-11-06T03:16:30.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c00bb00 con 0x7f7a9c159bd0 2023-11-06T03:16:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004d10 0x7f7a8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a8c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa0adb640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c0039b0 0x7f7a8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 -- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 msgr2=0x7f7a8c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c159bd0 0x7f7a8c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 -- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c0039b0 msgr2=0x7f7a8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c0039b0 0x7f7a8c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.003+0000 7f7aa12dc640 1 -- 172.21.15.157:0/3012687201 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a88009420 con 0x7f7a8c004d10 2023-11-06T03:16:30.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa12dc640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004d10 0x7f7a8c005190 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0b1130 tx=0x7f7a9401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32339d9aec929699 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940a6610 con 0x7f7a8c004d10 2023-11-06T03:16:30.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940a6770 con 0x7f7a8c004d10 2023-11-06T03:16:30.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9b7fe640 1 -- 172.21.15.157:0/3012687201 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940a6b20 con 0x7f7a8c004d10 2023-11-06T03:16:30.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/3012687201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004d10 msgr2=0x7f7a8c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/3012687201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8c004d10 0x7f7a8c005190 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0b1130 tx=0x7f7a9401c820 comp rx=0 tx=0).stop 2023-11-06T03:16:30.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa12dc640 1 -- 172.21.15.157:0/3012687201 reap_dead start 2023-11-06T03:16:30.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/3012687201 shutdown_connections 2023-11-06T03:16:30.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/3012687201 >> 172.21.15.157:0/3012687201 conn(0x7f7a9c069d70 msgr2=0x7f7a9c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/3012687201 shutdown_connections 2023-11-06T03:16:30.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/3012687201 wait complete. 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 Processor -- start 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- start start 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 0x7f7a9c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0c4c60 0x7f7a9c147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c1e3d90 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c1e3110 con 0x7f7a8c004d10 2023-11-06T03:16:30.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa2d3f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c1e33f0 con 0x7f7a9c0c4c60 2023-11-06T03:16:30.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56438/0 (socket says 172.21.15.157:56438) 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 -- 172.21.15.157:0/299821472 learned_addr learned my addr 172.21.15.157:0/299821472 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa12dc640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0c4c60 0x7f7a9c147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa0adb640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 0x7f7a9c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 -- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0c4c60 msgr2=0x7f7a9c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0c4c60 0x7f7a9c147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 -- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 msgr2=0x7f7a9c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 0x7f7a9c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 -- 172.21.15.157:0/299821472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c1e36d0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7aa0adb640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 0x7f7a9c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.007+0000 7f7a9bfff640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7a88011900 tx=0x7f7a88011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800fbd0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8800fd30 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88015660 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c1e39b0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c154a90 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c0b54f0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7a880157c0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 0x7f7a80067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7a880839a0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7aa0adb640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 0x7f7a80067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.011+0000 7f7aa0adb640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 0x7f7a80067f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7a900017b0 tx=0x7f7a90001160 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.015+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7a88051230 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.155+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a9c0b4ec0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7a997fa640 1 -- 172.21.15.157:0/299821472 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a880220a0 con 0x7f7a9c1e41b0 2023-11-06T03:16:30.155 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:16:30.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 msgr2=0x7f7a80067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 0x7f7a80067f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7a900017b0 tx=0x7f7a90001160 comp rx=0 tx=0).stop 2023-11-06T03:16:30.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 msgr2=0x7f7a9c0c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f7a88011900 tx=0x7f7a88011dc0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 shutdown_connections 2023-11-06T03:16:30.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7a80065a90 0x7f7a80067f50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7a9c0c4c60 0x7f7a9c147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a9c1e41b0 0x7f7a9c0c4720 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 --2- 172.21.15.157:0/299821472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7a8c004d10 0x7f7a9c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 >> 172.21.15.157:0/299821472 conn(0x7f7a9c069d70 msgr2=0x7f7a9c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 shutdown_connections 2023-11-06T03:16:30.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.159+0000 7f7aa2d3f640 1 -- 172.21.15.157:0/299821472 wait complete. 2023-11-06T03:16:30.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:16:30.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 Processor -- start 2023-11-06T03:16:30.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 -- start start 2023-11-06T03:16:30.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc5c146370 0x7fbc5c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 0x7fbc5c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0cf1a0 con 0x7fbc5c06b6a0 2023-11-06T03:16:30.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c06c3d0 con 0x7fbc5c146370 2023-11-06T03:16:30.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc63792640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c06f5a0 con 0x7fbc5c0ce800 2023-11-06T03:16:30.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc6152e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc5c146370 0x7fbc5c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc61d2f640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fbc5c06b6a0 0x7fbc5c06bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:60254/0 (socket says 172.21.15.157:60254) 2023-11-06T03:16:30.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc60d2d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 0x7fbc5c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc61d2f640 1 -- 172.21.15.157:0/4145710698 learned_addr learned my addr 172.21.15.157:0/4145710698 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:30.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc60d2d640 1 -- 172.21.15.157:0/4145710698 >> v2:172.21.15.195:3300/0 conn(0x7fbc5c146370 msgr2=0x7fbc5c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc60d2d640 1 --2- 172.21.15.157:0/4145710698 >> v2:172.21.15.195:3300/0 conn(0x7fbc5c146370 0x7fbc5c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc60d2d640 1 -- 172.21.15.157:0/4145710698 >> v1:172.21.15.195:6789/0 conn(0x7fbc5c06b6a0 legacy=0x7fbc5c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.287+0000 7fbc60d2d640 1 -- 172.21.15.157:0/4145710698 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c06f880 con 0x7fbc5c0ce800 2023-11-06T03:16:30.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc60d2d640 1 --2- 172.21.15.157:0/4145710698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 0x7fbc5c0cec60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc580097c0 tx=0x7fbc5801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5341002e5409ede server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc53fff640 1 -- 172.21.15.157:0/4145710698 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580ab5f0 con 0x7fbc5c0ce800 2023-11-06T03:16:30.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc53fff640 1 -- 172.21.15.157:0/4145710698 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580ab750 con 0x7fbc5c0ce800 2023-11-06T03:16:30.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc53fff640 1 -- 172.21.15.157:0/4145710698 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580abae0 con 0x7fbc5c0ce800 2023-11-06T03:16:30.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- 172.21.15.157:0/4145710698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 msgr2=0x7fbc5c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- 172.21.15.157:0/4145710698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 0x7fbc5c0cec60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbc580097c0 tx=0x7fbc5801c3f0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- 172.21.15.157:0/4145710698 shutdown_connections 2023-11-06T03:16:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- 172.21.15.157:0/4145710698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c0ce800 0x7fbc5c0cec60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- 172.21.15.157:0/4145710698 >> v2:172.21.15.195:3300/0 conn(0x7fbc5c146370 0x7fbc5c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- 172.21.15.157:0/4145710698 >> 172.21.15.157:0/4145710698 conn(0x7fbc5c065930 msgr2=0x7fbc5c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- 172.21.15.157:0/4145710698 shutdown_connections 2023-11-06T03:16:30.291 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- 172.21.15.157:0/4145710698 wait complete. 2023-11-06T03:16:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 Processor -- start 2023-11-06T03:16:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- start start 2023-11-06T03:16:30.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc5c06b6a0 0x7fbc5c1f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c146370 0x7fbc5c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15b5f0 con 0x7fbc5c0ce800 2023-11-06T03:16:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15b8d0 con 0x7fbc5c06b6a0 2023-11-06T03:16:30.295 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.291+0000 7fbc63792640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1f8430 con 0x7fbc5c146370 2023-11-06T03:16:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56446/0 (socket says 172.21.15.157:56446) 2023-11-06T03:16:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 -- 172.21.15.157:0/908106686 learned_addr learned my addr 172.21.15.157:0/908106686 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc61d2f640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c146370 0x7fbc5c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.297 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc6152e640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc5c06b6a0 0x7fbc5c1f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 -- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c146370 msgr2=0x7fbc5c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c146370 0x7fbc5c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 -- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc5c06b6a0 msgr2=0x7fbc5c1f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc5c06b6a0 0x7fbc5c1f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.298 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 -- 172.21.15.157:0/908106686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009420 con 0x7fbc5c0ce800 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc60d2d640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ad040 tx=0x7fbc5801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580bf610 con 0x7fbc5c0ce800 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580bf770 con 0x7fbc5c0ce800 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c1f8710 con 0x7fbc5c0ce800 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580bfae0 con 0x7fbc5c0ce800 2023-11-06T03:16:30.299 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c15b300 con 0x7fbc5c0ce800 2023-11-06T03:16:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c15b730 con 0x7fbc5c0ce800 2023-11-06T03:16:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc580bfc40 con 0x7fbc5c0ce800 2023-11-06T03:16:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 0x7fbc40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fbc580a8050 con 0x7fbc5c0ce800 2023-11-06T03:16:30.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc6152e640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 0x7fbc40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.295+0000 7fbc6152e640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 0x7fbc40067f20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c141610 tx=0x7fbc4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.299+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbc5c15b730 con 0x7fbc5c0ce800 2023-11-06T03:16:30.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc5c1f8c30 con 0x7fbc5c0ce800 2023-11-06T03:16:30.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc51ffb640 1 -- 172.21.15.157:0/908106686 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc5c1f8c30 con 0x7fbc5c0ce800 2023-11-06T03:16:30.449 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:16:30.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 msgr2=0x7fbc40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 0x7fbc40067f20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c141610 tx=0x7fbc4c008040 comp rx=0 tx=0).stop 2023-11-06T03:16:30.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 msgr2=0x7fbc5c15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.451+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fbc580ad040 tx=0x7fbc5801ac30 comp rx=0 tx=0).stop 2023-11-06T03:16:30.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 shutdown_connections 2023-11-06T03:16:30.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbc40065a60 0x7fbc40067f20 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc5c146370 0x7fbc5c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbc5c0ce800 0x7fbc5c15adc0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 --2- 172.21.15.157:0/908106686 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc5c06b6a0 0x7fbc5c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 >> 172.21.15.157:0/908106686 conn(0x7fbc5c065930 msgr2=0x7fbc5c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 shutdown_connections 2023-11-06T03:16:30.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.455+0000 7fbc63792640 1 -- 172.21.15.157:0/908106686 wait complete. 2023-11-06T03:16:30.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:16:30.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 Processor -- start 2023-11-06T03:16:30.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 -- start start 2023-11-06T03:16:30.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cb10 0x7f57641608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 0x7f57640cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f576415cef0 0x7f57640cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57641617d0 con 0x7f576415cef0 2023-11-06T03:16:30.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5764161210 con 0x7f57640ccf20 2023-11-06T03:16:30.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5768f69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57641614f0 con 0x7f576415cb10 2023-11-06T03:16:30.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f576415cef0 0x7f57640cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f576415cef0 0x7f57640cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56448/0 (socket says 172.21.15.157:56448) 2023-11-06T03:16:30.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 -- 172.21.15.157:0/624146631 learned_addr learned my addr 172.21.15.157:0/624146631 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:30.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f576259c640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cb10 0x7f57641608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f575bfff640 1 --2- 172.21.15.157:0/624146631 >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 0x7f57640cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 -- 172.21.15.157:0/624146631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cb10 msgr2=0x7f57641608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cb10 0x7f57641608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 -- 172.21.15.157:0/624146631 >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 msgr2=0x7f57640cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:30.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 --2- 172.21.15.157:0/624146631 >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 0x7f57640cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 -- 172.21.15.157:0/624146631 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640cb190 con 0x7f576415cef0 2023-11-06T03:16:30.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f575bfff640 1 --2- 172.21.15.157:0/624146631 >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 0x7f57640cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:30.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5762d9d640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cef0 0x7f57640cd8d0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f576406c260 tx=0x7f575c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ad9eabca6558ac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5761d9b640 1 -- 172.21.15.157:0/624146631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a57b0 con 0x7f576415cef0 2023-11-06T03:16:30.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5761d9b640 1 -- 172.21.15.157:0/624146631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0a5910 con 0x7f576415cef0 2023-11-06T03:16:30.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.583+0000 7f5761d9b640 1 -- 172.21.15.157:0/624146631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a5ca0 con 0x7f576415cef0 2023-11-06T03:16:30.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/624146631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cef0 msgr2=0x7f57640cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cef0 0x7f57640cd8d0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f576406c260 tx=0x7f575c01c910 comp rx=0 tx=0).stop 2023-11-06T03:16:30.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/624146631 shutdown_connections 2023-11-06T03:16:30.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cef0 0x7f57640cd8d0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- 172.21.15.157:0/624146631 >> v2:172.21.15.195:3300/0 conn(0x7f57640ccf20 0x7f57640cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- 172.21.15.157:0/624146631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cb10 0x7f57641608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/624146631 >> 172.21.15.157:0/624146631 conn(0x7f576406d560 msgr2=0x7f576406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/624146631 shutdown_connections 2023-11-06T03:16:30.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/624146631 wait complete. 2023-11-06T03:16:30.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 Processor -- start 2023-11-06T03:16:30.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- start start 2023-11-06T03:16:30.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57640ccf20 0x7f5764152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cef0 0x7f576415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764160220 con 0x7f576415cb10 2023-11-06T03:16:30.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576415d8e0 con 0x7f57640ccf20 2023-11-06T03:16:30.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576415fc60 con 0x7f576415cef0 2023-11-06T03:16:30.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56452/0 (socket says 172.21.15.157:56452) 2023-11-06T03:16:30.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 -- 172.21.15.157:0/935600436 learned_addr learned my addr 172.21.15.157:0/935600436 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:30.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f576259c640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57640ccf20 0x7f5764152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5762d9d640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cef0 0x7f576415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 -- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cef0 msgr2=0x7f576415b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cef0 0x7f576415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 -- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57640ccf20 msgr2=0x7f5764152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57640ccf20 0x7f5764152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 -- 172.21.15.157:0/935600436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009b30 con 0x7f576415cb10 2023-11-06T03:16:30.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575bfff640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c7f0 tx=0x7f574c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c00cea0 con 0x7f576415cb10 2023-11-06T03:16:30.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c004230 con 0x7f576415cb10 2023-11-06T03:16:30.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576415ff40 con 0x7f576415cb10 2023-11-06T03:16:30.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c007830 con 0x7f576415cb10 2023-11-06T03:16:30.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5764159e00 con 0x7f576415cb10 2023-11-06T03:16:30.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.587+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5764072750 con 0x7f576415cb10 2023-11-06T03:16:30.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.591+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f574c007990 con 0x7f576415cb10 2023-11-06T03:16:30.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.591+0000 7f575affd640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 0x7f5744067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.591+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f574c083280 con 0x7f576415cb10 2023-11-06T03:16:30.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.591+0000 7f576259c640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 0x7f5744067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.591+0000 7f576259c640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 0x7f5744067f20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f575400b9d0 tx=0x7f5754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.595+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f574c050b10 con 0x7f576415cb10 2023-11-06T03:16:30.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.739+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f576415c020 con 0x7f576415cb10 2023-11-06T03:16:30.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.739+0000 7f575affd640 1 -- 172.21.15.157:0/935600436 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7f574c054130 con 0x7f576415cb10 2023-11-06T03:16:30.737 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:16:30.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 msgr2=0x7f5744067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 0x7f5744067f20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f575400b9d0 tx=0x7f5754008040 comp rx=0 tx=0).stop 2023-11-06T03:16:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 msgr2=0x7f576415d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c7f0 tx=0x7f574c00ccb0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 shutdown_connections 2023-11-06T03:16:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5744065a60 0x7f5744067f20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f576415cef0 0x7f576415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f576415cb10 0x7f576415d3a0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 --2- 172.21.15.157:0/935600436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f57640ccf20 0x7f5764152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 >> 172.21.15.157:0/935600436 conn(0x7f576406d560 msgr2=0x7f5764155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 shutdown_connections 2023-11-06T03:16:30.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.743+0000 7f5768f69640 1 -- 172.21.15.157:0/935600436 wait complete. 2023-11-06T03:16:30.755 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:16:30.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.879+0000 7f3ef43bc640 1 Processor -- start 2023-11-06T03:16:30.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef43bc640 1 -- start start 2023-11-06T03:16:30.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef43bc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3eec15e5e0 0x7f3eec0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef43bc640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec15f580 con 0x7f3eec1551e0 2023-11-06T03:16:30.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef43bc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec15efc0 con 0x7f3eec15ea90 2023-11-06T03:16:30.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef43bc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec0704b0 con 0x7f3eec15e5e0 2023-11-06T03:16:30.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef2158640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f3eec1551e0 0x7f3eec06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38088/0 (socket says 172.21.15.157:38088) 2023-11-06T03:16:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef2158640 1 -- 172.21.15.157:0/3892245053 learned_addr learned my addr 172.21.15.157:0/3892245053 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1957640 1 --2- 172.21.15.157:0/3892245053 >> v2:172.21.15.157:3300/0 conn(0x7f3eec15e5e0 0x7f3eec0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1156640 1 -- 172.21.15.157:0/3892245053 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614916523 0 0) 0x7f3eec15f580 con 0x7f3eec1551e0 2023-11-06T03:16:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1156640 1 -- 172.21.15.157:0/3892245053 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003530 con 0x7f3eec1551e0 2023-11-06T03:16:30.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1957640 1 -- 172.21.15.157:0/3892245053 >> v1:172.21.15.195:6789/0 conn(0x7f3eec15ea90 legacy=0x7f3eec15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1957640 1 -- 172.21.15.157:0/3892245053 >> v1:172.21.15.157:6789/0 conn(0x7f3eec1551e0 legacy=0x7f3eec06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.883+0000 7f3ef1957640 1 -- 172.21.15.157:0/3892245053 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec15f2a0 con 0x7f3eec15e5e0 2023-11-06T03:16:30.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef1957640 1 --2- 172.21.15.157:0/3892245053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15e5e0 0x7f3eec0b3410 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0097c0 tx=0x7f3edc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7237d49615edf2c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef1156640 1 -- 172.21.15.157:0/3892245053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0ab600 con 0x7f3eec15e5e0 2023-11-06T03:16:30.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef1156640 1 -- 172.21.15.157:0/3892245053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc0a7050 con 0x7f3eec15e5e0 2023-11-06T03:16:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef1156640 1 -- 172.21.15.157:0/3892245053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0ab980 con 0x7f3eec15e5e0 2023-11-06T03:16:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/3892245053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15e5e0 msgr2=0x7f3eec0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/3892245053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15e5e0 0x7f3eec0b3410 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0097c0 tx=0x7f3edc01c3c0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/3892245053 shutdown_connections 2023-11-06T03:16:30.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/3892245053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15e5e0 0x7f3eec0b3410 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/3892245053 >> 172.21.15.157:0/3892245053 conn(0x7f3eec065930 msgr2=0x7f3eec065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:30.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/3892245053 shutdown_connections 2023-11-06T03:16:30.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/3892245053 wait complete. 2023-11-06T03:16:30.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 Processor -- start 2023-11-06T03:16:30.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- start start 2023-11-06T03:16:30.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec1551e0 0x7f3eec15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec15e5e0 0x7f3eec146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec153b00 con 0x7f3eec15ea90 2023-11-06T03:16:30.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec15a930 con 0x7f3eec1551e0 2023-11-06T03:16:30.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec14ac50 con 0x7f3eec15e5e0 2023-11-06T03:16:30.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56462/0 (socket says 172.21.15.157:56462) 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 -- 172.21.15.157:0/96318776 learned_addr learned my addr 172.21.15.157:0/96318776 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef1957640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec15e5e0 0x7f3eec146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2158640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec1551e0 0x7f3eec15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 -- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec15e5e0 msgr2=0x7f3eec146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec15e5e0 0x7f3eec146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 -- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec1551e0 msgr2=0x7f3eec15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:30.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec1551e0 0x7f3eec15a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:30.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 -- 172.21.15.157:0/96318776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc009420 con 0x7f3eec15ea90 2023-11-06T03:16:30.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef2959640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c730 tx=0x7f3ee400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003b80 con 0x7f3eec15ea90 2023-11-06T03:16:30.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4003ce0 con 0x7f3eec15ea90 2023-11-06T03:16:30.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec14aef0 con 0x7f3eec15ea90 2023-11-06T03:16:30.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400aa30 con 0x7f3eec15ea90 2023-11-06T03:16:30.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.887+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec14b4b0 con 0x7f3eec15ea90 2023-11-06T03:16:30.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec14b6a0 con 0x7f3eec15ea90 2023-11-06T03:16:30.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3ee400ab90 con 0x7f3eec15ea90 2023-11-06T03:16:30.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ee2ffd640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 0x7f3ed0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:30.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f3ee40830f0 con 0x7f3eec15ea90 2023-11-06T03:16:30.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ef2158640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 0x7f3ed0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:30.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.891+0000 7f3ef2158640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 0x7f3ed0067f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80011f0 tx=0x7f3ee8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:30.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:30.895+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3ee40508f0 con 0x7f3eec15ea90 2023-11-06T03:16:31.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3eec072750 con 0x7f3eec15ea90 2023-11-06T03:16:31.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ee2ffd640 1 -- 172.21.15.157:0/96318776 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ee4053f10 con 0x7f3eec15ea90 2023-11-06T03:16:31.036 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:16:31.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 msgr2=0x7f3ed0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 0x7f3ed0067f20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80011f0 tx=0x7f3ee8009040 comp rx=0 tx=0).stop 2023-11-06T03:16:31.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 msgr2=0x7f3eec0cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c730 tx=0x7f3ee400cbf0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 shutdown_connections 2023-11-06T03:16:31.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3ed0065a60 0x7f3ed0067f20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3eec15ea90 0x7f3eec0cdba0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3eec15e5e0 0x7f3eec146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 --2- 172.21.15.157:0/96318776 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3eec1551e0 0x7f3eec15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 >> 172.21.15.157:0/96318776 conn(0x7f3eec065930 msgr2=0x7f3eec14d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 shutdown_connections 2023-11-06T03:16:31.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.039+0000 7f3ef43bc640 1 -- 172.21.15.157:0/96318776 wait complete. 2023-11-06T03:16:31.055 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:31.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 Processor -- start 2023-11-06T03:16:31.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 -- start start 2023-11-06T03:16:31.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f486415ce80 0x7f48640cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48640b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48640d2920 con 0x7f486415caa0 2023-11-06T03:16:31.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48640d0520 con 0x7f486415ce80 2023-11-06T03:16:31.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486c1e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48640d18a0 con 0x7f48640d00b0 2023-11-06T03:16:31.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869f83640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f486415caa0 0x7f48640cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38098/0 (socket says 172.21.15.157:38098) 2023-11-06T03:16:31.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869f83640 1 -- 172.21.15.157:0/1438823493 learned_addr learned my addr 172.21.15.157:0/1438823493 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:31.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634323255 0 0) 0x7f48640d2920 con 0x7f486415caa0 2023-11-06T03:16:31.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869782640 1 --2- 172.21.15.157:0/1438823493 >> v2:172.21.15.195:3300/0 conn(0x7f486415ce80 0x7f48640cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4850003530 con 0x7f486415caa0 2023-11-06T03:16:31.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48640b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2483109295 0 0) 0x7f4850003530 con 0x7f486415caa0 2023-11-06T03:16:31.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 msgr2=0x7f48640b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48640b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 >> v2:172.21.15.195:3300/0 conn(0x7f486415ce80 msgr2=0x7f48640cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 --2- 172.21.15.157:0/1438823493 >> v2:172.21.15.195:3300/0 conn(0x7f486415ce80 0x7f48640cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640d1b60 con 0x7f486415caa0 2023-11-06T03:16:31.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f48600016a0 con 0x7f486415caa0 2023-11-06T03:16:31.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 >> v1:172.21.15.157:6789/0 conn(0x7f486415caa0 legacy=0x7f48640cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4850003980 0x7f4850003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4850004d10 0x7f4850005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640d0520 con 0x7f4850004d10 2023-11-06T03:16:31.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640d18a0 con 0x7f4850003980 2023-11-06T03:16:31.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485000b990 con 0x7f48640d00b0 2023-11-06T03:16:31.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4850004d10 0x7f4850005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869782640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 -- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 msgr2=0x7f48500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f48640d00b0 0x7f48500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 -- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4850003980 msgr2=0x7f4850003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869f83640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4850003980 0x7f4850003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4850003980 0x7f4850003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f486a784640 1 -- 172.21.15.157:0/1438823493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4850003530 con 0x7f4850004d10 2023-11-06T03:16:31.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.163+0000 7f4869f83640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4850003980 0x7f4850003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:31.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486a784640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4850004d10 0x7f4850005190 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f485c00e170 tx=0x7f485c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366ec1cae92fa0d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c0a5c50 con 0x7f4850004d10 2023-11-06T03:16:31.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c00f040 con 0x7f4850004d10 2023-11-06T03:16:31.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f4868f81640 1 -- 172.21.15.157:0/1438823493 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c0ad440 con 0x7f4850004d10 2023-11-06T03:16:31.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- 172.21.15.157:0/1438823493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4850004d10 msgr2=0x7f4850005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1438823493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4850004d10 0x7f4850005190 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f485c00e170 tx=0x7f485c01cc40 comp rx=0 tx=0).stop 2023-11-06T03:16:31.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486a784640 1 -- 172.21.15.157:0/1438823493 reap_dead start 2023-11-06T03:16:31.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- 172.21.15.157:0/1438823493 shutdown_connections 2023-11-06T03:16:31.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- 172.21.15.157:0/1438823493 >> 172.21.15.157:0/1438823493 conn(0x7f4864070790 msgr2=0x7f4864070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- 172.21.15.157:0/1438823493 shutdown_connections 2023-11-06T03:16:31.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- 172.21.15.157:0/1438823493 wait complete. 2023-11-06T03:16:31.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 Processor -- start 2023-11-06T03:16:31.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.167+0000 7f486c1e7640 1 -- start start 2023-11-06T03:16:31.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4864158190 0x7f48640b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48641e3560 0x7f48641e4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640b1780 con 0x7f48640cc500 2023-11-06T03:16:31.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48641e39e0 con 0x7f48641e3560 2023-11-06T03:16:31.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48640cd2f0 con 0x7f4864158190 2023-11-06T03:16:31.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56486/0 (socket says 172.21.15.157:56486) 2023-11-06T03:16:31.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 -- 172.21.15.157:0/1451630881 learned_addr learned my addr 172.21.15.157:0/1451630881 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:31.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869f83640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4864158190 0x7f48640b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486a784640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48641e3560 0x7f48641e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 -- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4864158190 msgr2=0x7f48640b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4864158190 0x7f48640b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 -- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48641e3560 msgr2=0x7f48641e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48641e3560 0x7f48641e4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 -- 172.21.15.157:0/1451630881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640ccd30 con 0x7f48640cc500 2023-11-06T03:16:31.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869782640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4858011630 tx=0x7f4858011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858004220 con 0x7f48640cc500 2023-11-06T03:16:31.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858004380 con 0x7f48640cc500 2023-11-06T03:16:31.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48640cd010 con 0x7f48640cc500 2023-11-06T03:16:31.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485800f960 con 0x7f48640cc500 2023-11-06T03:16:31.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641e5850 con 0x7f48640cc500 2023-11-06T03:16:31.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48640d20d0 con 0x7f48640cc500 2023-11-06T03:16:31.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f485800fac0 con 0x7f48640cc500 2023-11-06T03:16:31.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 0x7f4848067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f48580835a0 con 0x7f48640cc500 2023-11-06T03:16:31.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869f83640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 0x7f4848067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.171+0000 7f4869f83640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 0x7f4848067f20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4860001ec0 tx=0x7f4860009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.175+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4858050d80 con 0x7f48640cc500 2023-11-06T03:16:31.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.347+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4864072750 con 0x7f48640cc500 2023-11-06T03:16:31.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f48567fc640 1 -- 172.21.15.157:0/1451630881 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7f48580044f0 con 0x7f48640cc500 2023-11-06T03:16:31.348 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:31.348 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:16:31.348 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:16:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 msgr2=0x7f4848067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 0x7f4848067f20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4860001ec0 tx=0x7f4860009040 comp rx=0 tx=0).stop 2023-11-06T03:16:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 msgr2=0x7f48641e3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4858011630 tx=0x7f4858011af0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 shutdown_connections 2023-11-06T03:16:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4848065a60 0x7f4848067f20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48641e3560 0x7f48641e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640cc500 0x7f48641e3020 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 --2- 172.21.15.157:0/1451630881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4864158190 0x7f48640b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 >> 172.21.15.157:0/1451630881 conn(0x7f4864070790 msgr2=0x7f48640d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 shutdown_connections 2023-11-06T03:16:31.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.351+0000 7f486c1e7640 1 -- 172.21.15.157:0/1451630881 wait complete. 2023-11-06T03:16:31.366 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:31.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.475+0000 7fb5039bf640 1 Processor -- start 2023-11-06T03:16:31.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 -- start start 2023-11-06T03:16:31.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb4fc15ce80 0x7fb4fc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4fc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0d2920 con 0x7fb4fc15caa0 2023-11-06T03:16:31.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4fc0d0520 con 0x7fb4fc15ce80 2023-11-06T03:16:31.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb5039bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0d18a0 con 0x7fb4fc0d00b0 2023-11-06T03:16:31.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb50175b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb4fc15caa0 0x7fb4fc0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38102/0 (socket says 172.21.15.157:38102) 2023-11-06T03:16:31.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb50175b640 1 -- 172.21.15.157:0/493245832 learned_addr learned my addr 172.21.15.157:0/493245832 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:31.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb501f5c640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4fc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 --2- 172.21.15.157:0/493245832 >> v2:172.21.15.195:3300/0 conn(0x7fb4fc15ce80 0x7fb4fc0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203416653 0 0) 0x7fb4fc0d2920 con 0x7fb4fc15caa0 2023-11-06T03:16:31.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e8003530 con 0x7fb4fc15caa0 2023-11-06T03:16:31.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1739253270 0 0) 0x7fb4e8003530 con 0x7fb4fc15caa0 2023-11-06T03:16:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 msgr2=0x7fb4fc0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4fc0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 >> v2:172.21.15.195:3300/0 conn(0x7fb4fc15ce80 msgr2=0x7fb4fc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 --2- 172.21.15.157:0/493245832 >> v2:172.21.15.195:3300/0 conn(0x7fb4fc15ce80 0x7fb4fc0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0d1b60 con 0x7fb4fc15caa0 2023-11-06T03:16:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb4f4001a20 con 0x7fb4fc15caa0 2023-11-06T03:16:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 >> v1:172.21.15.157:6789/0 conn(0x7fb4fc15caa0 legacy=0x7fb4fc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4e8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003f00 0x7fb4e80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8004d10 0x7fb4e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0d0520 con 0x7fb4e8003f00 2023-11-06T03:16:31.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0d18a0 con 0x7fb4e8004d10 2023-11-06T03:16:31.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e800b990 con 0x7fb4fc0d00b0 2023-11-06T03:16:31.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003f00 0x7fb4e80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb50175b640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4e8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb501f5c640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8004d10 0x7fb4e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 -- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 msgr2=0x7fb4e8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc0d00b0 0x7fb4e8003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 -- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8004d10 msgr2=0x7fb4e8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8004d10 0x7fb4e8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb500f5a640 1 -- 172.21.15.157:0/493245832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8003530 con 0x7fb4e8003f00 2023-11-06T03:16:31.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.479+0000 7fb501f5c640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8004d10 0x7fb4e8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:31.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb500f5a640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003f00 0x7fb4e80047d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009ab0 tx=0x7fb4ec01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc6f28cffa3a64c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0a6490 con 0x7fb4e8003f00 2023-11-06T03:16:31.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0a65f0 con 0x7fb4e8003f00 2023-11-06T03:16:31.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f3fff640 1 -- 172.21.15.157:0/493245832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0a69a0 con 0x7fb4e8003f00 2023-11-06T03:16:31.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/493245832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003f00 msgr2=0x7fb4e80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 --2- 172.21.15.157:0/493245832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4e8003f00 0x7fb4e80047d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009ab0 tx=0x7fb4ec01c830 comp rx=0 tx=0).stop 2023-11-06T03:16:31.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 -- 172.21.15.157:0/493245832 reap_dead start 2023-11-06T03:16:31.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/493245832 shutdown_connections 2023-11-06T03:16:31.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/493245832 >> 172.21.15.157:0/493245832 conn(0x7fb4fc070790 msgr2=0x7fb4fc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/493245832 shutdown_connections 2023-11-06T03:16:31.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/493245832 wait complete. 2023-11-06T03:16:31.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 Processor -- start 2023-11-06T03:16:31.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- start start 2023-11-06T03:16:31.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8003f00 0x7fb4fc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc15f510 0x7fb4fc0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0caee0 con 0x7fb4fc159fc0 2023-11-06T03:16:31.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc15a960 con 0x7fb4e8003f00 2023-11-06T03:16:31.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0ca2d0 con 0x7fb4fc15f510 2023-11-06T03:16:31.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56502/0 (socket says 172.21.15.157:56502) 2023-11-06T03:16:31.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 -- 172.21.15.157:0/496589972 learned_addr learned my addr 172.21.15.157:0/496589972 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:31.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb500f5a640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc15f510 0x7fb4fc0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb50175b640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8003f00 0x7fb4fc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 -- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc15f510 msgr2=0x7fb4fc0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc15f510 0x7fb4fc0b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 -- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8003f00 msgr2=0x7fb4fc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8003f00 0x7fb4fc0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 -- 172.21.15.157:0/496589972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0ca5b0 con 0x7fb4fc159fc0 2023-11-06T03:16:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb501f5c640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0c66f0 tx=0x7fb4f8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f801b070 con 0x7fb4fc159fc0 2023-11-06T03:16:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f8004020 con 0x7fb4fc159fc0 2023-11-06T03:16:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc0ca890 con 0x7fb4fc159fc0 2023-11-06T03:16:31.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80043b0 con 0x7fb4fc159fc0 2023-11-06T03:16:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.483+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc15f990 con 0x7fb4fc159fc0 2023-11-06T03:16:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc0cb050 con 0x7fb4fc159fc0 2023-11-06T03:16:31.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4f800e6a0 con 0x7fb4fc159fc0 2023-11-06T03:16:31.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb4f1ffb640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 0x7fb4e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fb4f80826a0 con 0x7fb4fc159fc0 2023-11-06T03:16:31.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb50175b640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 0x7fb4e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.487+0000 7fb50175b640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 0x7fb4e0067f20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0b5790 tx=0x7fb4f4001af0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.491+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb4f804ff30 con 0x7fb4fc159fc0 2023-11-06T03:16:31.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.635+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4fc072750 con 0x7fb4fc159fc0 2023-11-06T03:16:31.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb4f1ffb640 1 -- 172.21.15.157:0/496589972 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+18086 (secure 0 0 0) 0x7fb4f8053550 con 0x7fb4fc159fc0 2023-11-06T03:16:31.636 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:31.637 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":151,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:50.867833+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:31.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 msgr2=0x7fb4e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 0x7fb4e0067f20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0b5790 tx=0x7fb4f4001af0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 msgr2=0x7fb4fc15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0c66f0 tx=0x7fb4f8010c20 comp rx=0 tx=0).stop 2023-11-06T03:16:31.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 shutdown_connections 2023-11-06T03:16:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4e0065a60 0x7fb4e0067f20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4fc159fc0 0x7fb4fc15a420 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4fc15f510 0x7fb4fc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 --2- 172.21.15.157:0/496589972 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4e8003f00 0x7fb4fc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 >> 172.21.15.157:0/496589972 conn(0x7fb4fc070790 msgr2=0x7fb4fc0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 shutdown_connections 2023-11-06T03:16:31.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.639+0000 7fb5039bf640 1 -- 172.21.15.157:0/496589972 wait complete. 2023-11-06T03:16:31.655 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:16:31.656 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:31.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.763+0000 7fd70d9af640 1 Processor -- start 2023-11-06T03:16:31.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.763+0000 7fd70d9af640 1 -- start start 2023-11-06T03:16:31.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd70d9af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd7080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd70d9af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70815f580 con 0x7fd7081551e0 2023-11-06T03:16:31.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd70d9af640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70815efc0 con 0x7fd70815ea60 2023-11-06T03:16:31.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd70d9af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080704b0 con 0x7fd70815e5e0 2023-11-06T03:16:31.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd7081551e0 0x7fd70806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38118/0 (socket says 172.21.15.157:38118) 2023-11-06T03:16:31.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 -- 172.21.15.157:0/464893681 learned_addr learned my addr 172.21.15.157:0/464893681 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:31.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd7067fc640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd7080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187548112 0 0) 0x7fd70815f580 con 0x7fd7081551e0 2023-11-06T03:16:31.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f0003530 con 0x7fd7081551e0 2023-11-06T03:16:31.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 924093945 0 0) 0x7fd6f0003530 con 0x7fd7081551e0 2023-11-06T03:16:31.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 msgr2=0x7fd7080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd7080b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 >> v1:172.21.15.195:6789/0 conn(0x7fd70815ea60 legacy=0x7fd70815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70815f2a0 con 0x7fd7081551e0 2023-11-06T03:16:31.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd6f8001680 con 0x7fd7081551e0 2023-11-06T03:16:31.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 >> v1:172.21.15.157:6789/0 conn(0x7fd7081551e0 legacy=0x7fd70806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f00047d0 0x7fd6f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd6f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080704b0 con 0x7fd6f00039b0 2023-11-06T03:16:31.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0005520 con 0x7fd6f00047d0 2023-11-06T03:16:31.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f000bed0 con 0x7fd70815e5e0 2023-11-06T03:16:31.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd7067fc640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f00047d0 0x7fd6f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 -- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 msgr2=0x7fd6f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd7077fe640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd6f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd6f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 -- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f00047d0 msgr2=0x7fd6f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:31.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f00047d0 0x7fd6f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 -- 172.21.15.157:0/464893681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0003530 con 0x7fd6f00039b0 2023-11-06T03:16:31.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd7067fc640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6f00047d0 0x7fd6f0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:31.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd7077fe640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd70815e5e0 0x7fd6f000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:31.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd706ffd640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c240 tx=0x7fd6f801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d39ddf6cf7cadb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8005ae0 con 0x7fd6f00039b0 2023-11-06T03:16:31.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8005c40 con 0x7fd6f00039b0 2023-11-06T03:16:31.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.767+0000 7fd705ffb640 1 -- 172.21.15.157:0/464893681 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80a5800 con 0x7fd6f00039b0 2023-11-06T03:16:31.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/464893681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f00039b0 msgr2=0x7fd6f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 --2- 172.21.15.157:0/464893681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6f00039b0 0x7fd6f0003dd0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c240 tx=0x7fd6f801cc10 comp rx=0 tx=0).stop 2023-11-06T03:16:31.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 -- 172.21.15.157:0/464893681 reap_dead start 2023-11-06T03:16:31.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/464893681 shutdown_connections 2023-11-06T03:16:31.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/464893681 >> 172.21.15.157:0/464893681 conn(0x7fd708065930 msgr2=0x7fd708065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/464893681 shutdown_connections 2023-11-06T03:16:31.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/464893681 wait complete. 2023-11-06T03:16:31.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 Processor -- start 2023-11-06T03:16:31.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- start start 2023-11-06T03:16:31.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 0x7fd708151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd708163dc0 0x7fd7081653b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708156cf0 con 0x7fd708156330 2023-11-06T03:16:31.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708164310 con 0x7fd708150910 2023-11-06T03:16:31.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081645f0 con 0x7fd708163dc0 2023-11-06T03:16:31.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56528/0 (socket says 172.21.15.157:56528) 2023-11-06T03:16:31.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 -- 172.21.15.157:0/691144267 learned_addr learned my addr 172.21.15.157:0/691144267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:31.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7067fc640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd708163dc0 0x7fd7081653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd706ffd640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 0x7fd708151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 -- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd708163dc0 msgr2=0x7fd7081653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd708163dc0 0x7fd7081653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 -- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 msgr2=0x7fd708151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 0x7fd708151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 -- 172.21.15.157:0/691144267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7081648d0 con 0x7fd708156330 2023-11-06T03:16:31.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd706ffd640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 0x7fd708151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:31.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd7077fe640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cc5a0 tx=0x7fd70000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700006d60 con 0x7fd708156330 2023-11-06T03:16:31.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700003980 con 0x7fd708156330 2023-11-06T03:16:31.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700003d70 con 0x7fd708156330 2023-11-06T03:16:31.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708164bb0 con 0x7fd708156330 2023-11-06T03:16:31.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.771+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708151780 con 0x7fd708156330 2023-11-06T03:16:31.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7081658f0 con 0x7fd708156330 2023-11-06T03:16:31.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd700005070 con 0x7fd708156330 2023-11-06T03:16:31.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd6effff640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 0x7fd6f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:31.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fd700083010 con 0x7fd708156330 2023-11-06T03:16:31.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd706ffd640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 0x7fd6f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:31.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.775+0000 7fd706ffd640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 0x7fd6f4067f20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80ab000 tx=0x7fd6f80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:31.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.779+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7000508a0 con 0x7fd708156330 2023-11-06T03:16:31.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.951+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd708072750 con 0x7fd708156330 2023-11-06T03:16:31.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.951+0000 7fd6effff640 1 -- 172.21.15.157:0/691144267 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7fd700053ec0 con 0x7fd708156330 2023-11-06T03:16:31.949 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:31.950 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:16:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:16:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 msgr2=0x7fd6f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 0x7fd6f4067f20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80ab000 tx=0x7fd6f80a7070 comp rx=0 tx=0).stop 2023-11-06T03:16:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 msgr2=0x7fd7081567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fd7080cc5a0 tx=0x7fd70000dc00 comp rx=0 tx=0).stop 2023-11-06T03:16:31.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 shutdown_connections 2023-11-06T03:16:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd6f4065a60 0x7fd6f4067f20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd708156330 0x7fd7081567b0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd708163dc0 0x7fd7081653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 --2- 172.21.15.157:0/691144267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd708150910 0x7fd708151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 >> 172.21.15.157:0/691144267 conn(0x7fd708065930 msgr2=0x7fd7080ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:31.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 shutdown_connections 2023-11-06T03:16:31.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:31.955+0000 7fd70d9af640 1 -- 172.21.15.157:0/691144267 wait complete. 2023-11-06T03:16:31.968 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:32.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.075+0000 7ff2a59c2640 1 Processor -- start 2023-11-06T03:16:32.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.075+0000 7ff2a59c2640 1 -- start start 2023-11-06T03:16:32.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff2a59c2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff2a00cdaa0 0x7ff2a0167bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff2a59c2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a01617d0 con 0x7ff2a015cb10 2023-11-06T03:16:32.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff2a59c2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a0160fb0 con 0x7ff2a00ccf20 2023-11-06T03:16:32.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff2a59c2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a0161210 con 0x7ff2a00cdaa0 2023-11-06T03:16:32.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff2a015cb10 0x7ff2a01608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38124/0 (socket says 172.21.15.157:38124) 2023-11-06T03:16:32.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29effd640 1 -- 172.21.15.157:0/4047837116 learned_addr learned my addr 172.21.15.157:0/4047837116 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:32.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384971990 0 0) 0x7ff2a01617d0 con 0x7ff2a015cb10 2023-11-06T03:16:32.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff28c003530 con 0x7ff2a015cb10 2023-11-06T03:16:32.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29f7fe640 1 --2- 172.21.15.157:0/4047837116 >> v2:172.21.15.195:3300/0 conn(0x7ff2a00cdaa0 0x7ff2a0167bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2341838117 0 0) 0x7ff28c003530 con 0x7ff2a015cb10 2023-11-06T03:16:32.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 >> v2:172.21.15.195:3300/0 conn(0x7ff2a00cdaa0 msgr2=0x7ff2a0167bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 --2- 172.21.15.157:0/4047837116 >> v2:172.21.15.195:3300/0 conn(0x7ff2a00cdaa0 0x7ff2a0167bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 >> v1:172.21.15.195:6789/0 conn(0x7ff2a00ccf20 legacy=0x7ff2a00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a01614f0 con 0x7ff2a015cb10 2023-11-06T03:16:32.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7ff290001a00 con 0x7ff2a015cb10 2023-11-06T03:16:32.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 >> v1:172.21.15.157:6789/0 conn(0x7ff2a015cb10 legacy=0x7ff2a01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff28c003980 0x7ff28c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 0x7ff28c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c005190 0x7ff28c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a0161210 con 0x7ff28c0047d0 2023-11-06T03:16:32.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c00c130 con 0x7ff28c005190 2023-11-06T03:16:32.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c00c2f0 con 0x7ff28c003980 2023-11-06T03:16:32.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 0x7ff28c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29effd640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff28c003980 0x7ff28c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 -- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff28c003980 msgr2=0x7ff28c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff28c003980 0x7ff28c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 -- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c005190 msgr2=0x7ff28c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c005190 0x7ff28c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29f7fe640 1 -- 172.21.15.157:0/4047837116 reap_dead start 2023-11-06T03:16:32.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 -- 172.21.15.157:0/4047837116 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c003530 con 0x7ff28c0047d0 2023-11-06T03:16:32.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29effd640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff28c003980 0x7ff28c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:32.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 0x7ff28c004c50 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff288001750 tx=0x7ff28801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a437cc56083a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880052d0 con 0x7ff28c0047d0 2023-11-06T03:16:32.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2880a7050 con 0x7ff28c0047d0 2023-11-06T03:16:32.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.079+0000 7ff29dffb640 1 -- 172.21.15.157:0/4047837116 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288003820 con 0x7ff28c0047d0 2023-11-06T03:16:32.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 msgr2=0x7ff28c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 0x7ff28c004c50 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff288001750 tx=0x7ff28801c370 comp rx=0 tx=0).stop 2023-11-06T03:16:32.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/4047837116 shutdown_connections 2023-11-06T03:16:32.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/4047837116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff28c0047d0 0x7ff28c004c50 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/4047837116 >> 172.21.15.157:0/4047837116 conn(0x7ff2a006d560 msgr2=0x7ff2a006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/4047837116 shutdown_connections 2023-11-06T03:16:32.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/4047837116 wait complete. 2023-11-06T03:16:32.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 Processor -- start 2023-11-06T03:16:32.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- start start 2023-11-06T03:16:32.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c0047d0 0x7ff2a00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a014cd60 0x7ff2a015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a014f080 con 0x7ff2a00b4040 2023-11-06T03:16:32.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a01504f0 con 0x7ff28c0047d0 2023-11-06T03:16:32.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00b4600 con 0x7ff2a014cd60 2023-11-06T03:16:32.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56548/0 (socket says 172.21.15.157:56548) 2023-11-06T03:16:32.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 -- 172.21.15.157:0/3687193881 learned_addr learned my addr 172.21.15.157:0/3687193881 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:32.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29f7fe640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a014cd60 0x7ff2a015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29effd640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c0047d0 0x7ff2a00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 -- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a014cd60 msgr2=0x7ff2a015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a014cd60 0x7ff2a015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 -- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c0047d0 msgr2=0x7ff2a00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c0047d0 0x7ff2a00b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 -- 172.21.15.157:0/3687193881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00b5fe0 con 0x7ff2a00b4040 2023-11-06T03:16:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff29e7fc640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff288005040 tx=0x7ff28c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880069e0 con 0x7ff2a00b4040 2023-11-06T03:16:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288006b40 con 0x7ff2a00b4040 2023-11-06T03:16:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a014f080 con 0x7ff2a00b4040 2023-11-06T03:16:32.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288007020 con 0x7ff2a00b4040 2023-11-06T03:16:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.083+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a00c4a60 con 0x7ff2a00b4040 2023-11-06T03:16:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a006bf00 con 0x7ff2a00b4040 2023-11-06T03:16:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff288007180 con 0x7ff2a00b4040 2023-11-06T03:16:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff297fff640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 0x7ff280067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7ff28811d2a0 con 0x7ff2a00b4040 2023-11-06T03:16:32.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff29effd640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 0x7ff280067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.087+0000 7ff29effd640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 0x7ff280067f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff290000f90 tx=0x7ff290009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.091+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff2880eab30 con 0x7ff2a00b4040 2023-11-06T03:16:32.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a0072750 con 0x7ff2a00b4040 2023-11-06T03:16:32.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff297fff640 1 -- 172.21.15.157:0/3687193881 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4664 (secure 0 0 0) 0x7ff2880ee150 con 0x7ff2a00b4040 2023-11-06T03:16:32.269 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:32.270 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":111,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:15:47.843550+0000","modified":"2023-11-06T03:15:49.858188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9918":{"gid":9918,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1485464139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1485464139},{"type":"v1","addr":"172.21.15.157:6835","nonce":1485464139}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-11-06T03:16:32.271 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 111 2023-11-06T03:16:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 msgr2=0x7ff280067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 0x7ff280067f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff290000f90 tx=0x7ff290009040 comp rx=0 tx=0).stop 2023-11-06T03:16:32.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 msgr2=0x7ff2a00c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff288005040 tx=0x7ff28c005800 comp rx=0 tx=0).stop 2023-11-06T03:16:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 shutdown_connections 2023-11-06T03:16:32.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff280065a60 0x7ff280067f20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2a014cd60 0x7ff2a015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2a00b4040 0x7ff2a00c4520 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 --2- 172.21.15.157:0/3687193881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff28c0047d0 0x7ff2a00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 >> 172.21.15.157:0/3687193881 conn(0x7ff2a006d560 msgr2=0x7ff2a006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 shutdown_connections 2023-11-06T03:16:32.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.271+0000 7ff2a59c2640 1 -- 172.21.15.157:0/3687193881 wait complete. 2023-11-06T03:16:32.286 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:32.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.411+0000 7f1524926640 1 Processor -- start 2023-11-06T03:16:32.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.411+0000 7f1524926640 1 -- start start 2023-11-06T03:16:32.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f1524926640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1520159930 0x7f15200b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f1524926640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152015a860 con 0x7f15200b3020 2023-11-06T03:16:32.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f1524926640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200711b0 con 0x7f15200b5320 2023-11-06T03:16:32.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f1524926640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1520159d10 con 0x7f1520159930 2023-11-06T03:16:32.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f15200b3020 0x7f15200b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38134/0 (socket says 172.21.15.157:38134) 2023-11-06T03:16:32.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151dd9b640 1 -- 172.21.15.157:0/3737772025 learned_addr learned my addr 172.21.15.157:0/3737772025 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:32.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151e59c640 1 --2- 172.21.15.157:0/3737772025 >> v2:172.21.15.195:3300/0 conn(0x7f1520159930 0x7f15200b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245997653 0 0) 0x7f152015a860 con 0x7f15200b3020 2023-11-06T03:16:32.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1508005740 con 0x7f15200b3020 2023-11-06T03:16:32.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383821393 0 0) 0x7f1508005740 con 0x7f15200b3020 2023-11-06T03:16:32.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 >> v2:172.21.15.195:3300/0 conn(0x7f1520159930 msgr2=0x7f15200b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 --2- 172.21.15.157:0/3737772025 >> v2:172.21.15.195:3300/0 conn(0x7f1520159930 0x7f15200b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 >> v1:172.21.15.195:6789/0 conn(0x7f15200b5320 legacy=0x7f152006c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520159f30 con 0x7f15200b3020 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1514001680 con 0x7f15200b3020 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 >> v1:172.21.15.157:6789/0 conn(0x7f15200b3020 legacy=0x7f15200b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1508005b90 0x7f1508005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080069e0 0x7f1508006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 0x7f150800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520159d10 con 0x7f15080073a0 2023-11-06T03:16:32.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150800e240 con 0x7f1508005b90 2023-11-06T03:16:32.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150800e400 con 0x7f15080069e0 2023-11-06T03:16:32.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 0x7f150800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151dd9b640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080069e0 0x7f1508006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 -- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080069e0 msgr2=0x7f1508006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080069e0 0x7f1508006e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 -- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1508005b90 msgr2=0x7f1508005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1508005b90 0x7f1508005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 -- 172.21.15.157:0/3737772025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508005740 con 0x7f15080073a0 2023-11-06T03:16:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 -- 172.21.15.157:0/3737772025 reap_dead start 2023-11-06T03:16:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.415+0000 7f151ed9d640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 0x7f150800dba0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f1518001e70 tx=0x7f151801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54be701e726d61d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518003b80 con 0x7f15080073a0 2023-11-06T03:16:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518003d20 con 0x7f15080073a0 2023-11-06T03:16:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151d59a640 1 -- 172.21.15.157:0/3737772025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518005260 con 0x7f15080073a0 2023-11-06T03:16:32.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 msgr2=0x7f150800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 0x7f150800dba0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f1518001e70 tx=0x7f151801c760 comp rx=0 tx=0).stop 2023-11-06T03:16:32.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/3737772025 shutdown_connections 2023-11-06T03:16:32.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 --2- 172.21.15.157:0/3737772025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15080073a0 0x7f150800dba0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/3737772025 >> 172.21.15.157:0/3737772025 conn(0x7f1520069d70 msgr2=0x7f152006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/3737772025 shutdown_connections 2023-11-06T03:16:32.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/3737772025 wait complete. 2023-11-06T03:16:32.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 Processor -- start 2023-11-06T03:16:32.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- start start 2023-11-06T03:16:32.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080073a0 0x7f1520156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 0x7f15200c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520157200 con 0x7f1520143e70 2023-11-06T03:16:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15201574e0 con 0x7f15201401c0 2023-11-06T03:16:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15201577c0 con 0x7f15080073a0 2023-11-06T03:16:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56562/0 (socket says 172.21.15.157:56562) 2023-11-06T03:16:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 -- 172.21.15.157:0/668863222 learned_addr learned my addr 172.21.15.157:0/668863222 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151e59c640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080073a0 0x7f1520156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151dd9b640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 0x7f15200c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 -- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080073a0 msgr2=0x7f1520156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080073a0 0x7f1520156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 -- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 msgr2=0x7f15200c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 0x7f15200c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 -- 172.21.15.157:0/668863222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520157aa0 con 0x7f1520143e70 2023-11-06T03:16:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151dd9b640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 0x7f15200c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f151ed9d640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f151801cc40 tx=0x7f1508007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180ac070 con 0x7f1520143e70 2023-11-06T03:16:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518003e90 con 0x7f1520143e70 2023-11-06T03:16:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1520157d80 con 0x7f1520143e70 2023-11-06T03:16:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518006d60 con 0x7f1520143e70 2023-11-06T03:16:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.419+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1520140640 con 0x7f1520143e70 2023-11-06T03:16:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520072750 con 0x7f1520143e70 2023-11-06T03:16:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1518007030 con 0x7f1520143e70 2023-11-06T03:16:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f150effd640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f151811c270 con 0x7f1520143e70 2023-11-06T03:16:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f151e59c640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.423+0000 7f151e59c640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15100047c0 tx=0x7f1510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.427+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f15180e9a70 con 0x7f1520143e70 2023-11-06T03:16:32.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.583+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1520070a30 con 0x7f1520143e70 2023-11-06T03:16:32.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f150effd640 1 -- 172.21.15.157:0/668863222 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+18086 (secure 0 0 0) 0x7f15180ed090 con 0x7f1520143e70 2023-11-06T03:16:32.584 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:32.584 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":151,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:15:50.867833+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:15:46.459717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-06T03:15:46.807462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:32.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 msgr2=0x7f1500067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15100047c0 tx=0x7f1510008040 comp rx=0 tx=0).stop 2023-11-06T03:16:32.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 msgr2=0x7f1520144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f151801cc40 tx=0x7f1508007a10 comp rx=0 tx=0).stop 2023-11-06T03:16:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 shutdown_connections 2023-11-06T03:16:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1500065a90 0x7f1500067f50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1520143e70 0x7f1520144320 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15201401c0 0x7f15200c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 --2- 172.21.15.157:0/668863222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15080073a0 0x7f1520156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 >> 172.21.15.157:0/668863222 conn(0x7f1520069d70 msgr2=0x7f152006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 shutdown_connections 2023-11-06T03:16:32.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.587+0000 7f1524926640 1 -- 172.21.15.157:0/668863222 wait complete. 2023-11-06T03:16:32.604 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:16:32.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 Processor -- start 2023-11-06T03:16:32.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 -- start start 2023-11-06T03:16:32.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f923815e5e0 0x7f92380b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923815f580 con 0x7f923815ea90 2023-11-06T03:16:32.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923815efc0 con 0x7f92381551e0 2023-11-06T03:16:32.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f923da68640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92380704b0 con 0x7f923815e5e0 2023-11-06T03:16:32.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9236ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f923815e5e0 0x7f92380b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9237fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f923815ea90 0x7f923815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38148/0 (socket says 172.21.15.157:38148) 2023-11-06T03:16:32.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9237fff640 1 -- 172.21.15.157:0/1811754234 learned_addr learned my addr 172.21.15.157:0/1811754234 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:32.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9236ffd640 1 -- 172.21.15.157:0/1811754234 >> v1:172.21.15.195:6789/0 conn(0x7f92381551e0 legacy=0x7f923806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9236ffd640 1 -- 172.21.15.157:0/1811754234 >> v1:172.21.15.157:6789/0 conn(0x7f923815ea90 legacy=0x7f923815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.723+0000 7f9236ffd640 1 -- 172.21.15.157:0/1811754234 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923815f2a0 con 0x7f923815e5e0 2023-11-06T03:16:32.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f9236ffd640 1 --2- 172.21.15.157:0/1811754234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815e5e0 0x7f92380b3410 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f92200097c0 tx=0x7f922001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7715b2be8a536d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f92367fc640 1 -- 172.21.15.157:0/1811754234 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200ab600 con 0x7f923815e5e0 2023-11-06T03:16:32.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f92367fc640 1 -- 172.21.15.157:0/1811754234 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92200a7050 con 0x7f923815e5e0 2023-11-06T03:16:32.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f92367fc640 1 -- 172.21.15.157:0/1811754234 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200ab980 con 0x7f923815e5e0 2023-11-06T03:16:32.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- 172.21.15.157:0/1811754234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815e5e0 msgr2=0x7f92380b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 --2- 172.21.15.157:0/1811754234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815e5e0 0x7f92380b3410 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f92200097c0 tx=0x7f922001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- 172.21.15.157:0/1811754234 shutdown_connections 2023-11-06T03:16:32.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 --2- 172.21.15.157:0/1811754234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815e5e0 0x7f92380b3410 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- 172.21.15.157:0/1811754234 >> 172.21.15.157:0/1811754234 conn(0x7f9238065930 msgr2=0x7f9238065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- 172.21.15.157:0/1811754234 shutdown_connections 2023-11-06T03:16:32.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- 172.21.15.157:0/1811754234 wait complete. 2023-11-06T03:16:32.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 Processor -- start 2023-11-06T03:16:32.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.727+0000 7f923da68640 1 -- start start 2023-11-06T03:16:32.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 0x7f9238152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 0x7f9238148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238151ec0 con 0x7f923815ea90 2023-11-06T03:16:32.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92381521a0 con 0x7f923815e5e0 2023-11-06T03:16:32.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9238152480 con 0x7f92381551e0 2023-11-06T03:16:32.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56582/0 (socket says 172.21.15.157:56582) 2023-11-06T03:16:32.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 -- 172.21.15.157:0/351612664 learned_addr learned my addr 172.21.15.157:0/351612664 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:32.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 -- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 msgr2=0x7f9238152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f92377fe640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 0x7f9238152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9236ffd640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 0x7f9238148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 0x7f9238152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 -- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 msgr2=0x7f9238148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 0x7f9238148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 -- 172.21.15.157:0/351612664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009420 con 0x7f923815ea90 2023-11-06T03:16:32.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f92377fe640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 0x7f9238152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:32.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9236ffd640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 0x7f9238148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:32.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f9237fff640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f923000ca80 tx=0x7f923000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000ac30 con 0x7f923815ea90 2023-11-06T03:16:32.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000ad90 con 0x7f923815ea90 2023-11-06T03:16:32.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9238152760 con 0x7f923815ea90 2023-11-06T03:16:32.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230012600 con 0x7f923815ea90 2023-11-06T03:16:32.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9238149280 con 0x7f923815ea90 2023-11-06T03:16:32.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9238152c80 con 0x7f923815ea90 2023-11-06T03:16:32.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9230007070 con 0x7f923815ea90 2023-11-06T03:16:32.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 0x7f9218067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:32.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9230083150 con 0x7f923815ea90 2023-11-06T03:16:32.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.731+0000 7f92377fe640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 0x7f9218067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:32.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.735+0000 7f92377fe640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 0x7f9218067f20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9228000f70 tx=0x7f9228009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:32.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.735+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9238152c80 con 0x7f923815ea90 2023-11-06T03:16:32.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.919+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9238152c80 con 0x7f923815ea90 2023-11-06T03:16:32.917 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:16:32.919+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:16:32.933 INFO:tasks.ceph.mds.a.smithi157.stderr: -2154> 2023-11-06T03:15:49.391+0000 7fa23e408640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:16:32.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.935+0000 7f922ffff640 1 -- 172.21.15.157:0/351612664 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9238152c80 con 0x7f923815ea90 2023-11-06T03:16:32.935 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:16:32.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 msgr2=0x7f9218067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 0x7f9218067f20 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9228000f70 tx=0x7f9228009040 comp rx=0 tx=0).stop 2023-11-06T03:16:32.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 msgr2=0x7f923814cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:32.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f923000ca80 tx=0x7f923000cf40 comp rx=0 tx=0).stop 2023-11-06T03:16:32.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 shutdown_connections 2023-11-06T03:16:32.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9218065a60 0x7f9218067f20 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f923815ea90 0x7f923814cde0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f923815e5e0 0x7f9238148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 --2- 172.21.15.157:0/351612664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92381551e0 0x7f9238152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 >> 172.21.15.157:0/351612664 conn(0x7f9238065930 msgr2=0x7f92380b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 shutdown_connections 2023-11-06T03:16:32.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:32.939+0000 7f923da68640 1 -- 172.21.15.157:0/351612664 wait complete. 2023-11-06T03:16:32.955 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:16:32.959 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:16:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.067+0000 7fa33cc4b640 1 Processor -- start 2023-11-06T03:16:33.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.067+0000 7fa33cc4b640 1 -- start start 2023-11-06T03:16:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa33815caa0 0x7fa3380cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 0x7fa3380cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa3380d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3380d2920 con 0x7fa33815caa0 2023-11-06T03:16:33.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3380d18a0 con 0x7fa33815ce80 2023-11-06T03:16:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33cc4b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380d1b60 con 0x7fa3380ce1a0 2023-11-06T03:16:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa33815caa0 0x7fa3380cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa33815caa0 0x7fa3380cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56624/0 (socket says 172.21.15.157:56624) 2023-11-06T03:16:33.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 -- 172.21.15.157:0/3724823888 learned_addr learned my addr 172.21.15.157:0/3724823888 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa32ffff640 1 --2- 172.21.15.157:0/3724823888 >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 0x7fa3380cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa336d9d640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa3380d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 -- 172.21.15.157:0/3724823888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 msgr2=0x7fa3380d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa3380d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 -- 172.21.15.157:0/3724823888 >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 msgr2=0x7fa3380cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 --2- 172.21.15.157:0/3724823888 >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 0x7fa3380cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 -- 172.21.15.157:0/3724823888 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3380d1e40 con 0x7fa33815caa0 2023-11-06T03:16:33.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa32ffff640 1 --2- 172.21.15.157:0/3724823888 >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 0x7fa3380cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:33.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa33659c640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815caa0 0x7fa3380cdc60 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fa3380c6720 tx=0x7fa32801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7061a3c475f4ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:33.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa335d9b640 1 -- 172.21.15.157:0/3724823888 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280aa5f0 con 0x7fa33815caa0 2023-11-06T03:16:33.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa335d9b640 1 -- 172.21.15.157:0/3724823888 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3280aa750 con 0x7fa33815caa0 2023-11-06T03:16:33.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.071+0000 7fa335d9b640 1 -- 172.21.15.157:0/3724823888 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280aaae0 con 0x7fa33815caa0 2023-11-06T03:16:33.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/3724823888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815caa0 msgr2=0x7fa3380cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:33.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815caa0 0x7fa3380cdc60 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fa3380c6720 tx=0x7fa32801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/3724823888 shutdown_connections 2023-11-06T03:16:33.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa3380d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/3724823888 >> v2:172.21.15.195:3300/0 conn(0x7fa33815ce80 0x7fa3380cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/3724823888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815caa0 0x7fa3380cdc60 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/3724823888 >> 172.21.15.157:0/3724823888 conn(0x7fa338070790 msgr2=0x7fa338070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:33.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/3724823888 shutdown_connections 2023-11-06T03:16:33.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/3724823888 wait complete. 2023-11-06T03:16:33.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 Processor -- start 2023-11-06T03:16:33.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- start start 2023-11-06T03:16:33.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa33815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa33815caa0 0x7fa338151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33815a3f0 con 0x7fa33815ce80 2023-11-06T03:16:33.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33815b810 con 0x7fa33815caa0 2023-11-06T03:16:33.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa338151680 con 0x7fa3380ce1a0 2023-11-06T03:16:33.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56638/0 (socket says 172.21.15.157:56638) 2023-11-06T03:16:33.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 -- 172.21.15.157:0/4138726172 learned_addr learned my addr 172.21.15.157:0/4138726172 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:33.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa32ffff640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa33815caa0 0x7fa338151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33659c640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa33815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 -- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 msgr2=0x7fa33815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:33.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa33815b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 -- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa33815caa0 msgr2=0x7fa338151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:33.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa33815caa0 0x7fa338151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 -- 172.21.15.157:0/4138726172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3280094d0 con 0x7fa33815ce80 2023-11-06T03:16:33.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa336d9d640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa33000cb80 tx=0x7fa33000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:33.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000ce70 con 0x7fa33815ce80 2023-11-06T03:16:33.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33000abd0 con 0x7fa33815ce80 2023-11-06T03:16:33.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330011410 con 0x7fa33815ce80 2023-11-06T03:16:33.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33815bfc0 con 0x7fa33815ce80 2023-11-06T03:16:33.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33815ba00 con 0x7fa33815ce80 2023-11-06T03:16:33.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.075+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33815a530 con 0x7fa33815ce80 2023-11-06T03:16:33.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.079+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa330011570 con 0x7fa33815ce80 2023-11-06T03:16:33.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.079+0000 7fa32effd640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 0x7fa318067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:33.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.079+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fa3300831b0 con 0x7fa33815ce80 2023-11-06T03:16:33.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.079+0000 7fa33659c640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 0x7fa318067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:33.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.079+0000 7fa33659c640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 0x7fa318067f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa3280ae000 tx=0x7fa328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:33.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.083+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa33815a530 con 0x7fa33815ce80 2023-11-06T03:16:33.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.255+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa33815a530 con 0x7fa33815ce80 2023-11-06T03:16:33.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.939+0000 7fa32effd640 1 -- 172.21.15.157:0/4138726172 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa33815a530 con 0x7fa33815ce80 2023-11-06T03:16:33.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 msgr2=0x7fa318067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:33.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 0x7fa318067f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa3280ae000 tx=0x7fa328008040 comp rx=0 tx=0).stop 2023-11-06T03:16:33.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 msgr2=0x7fa33814d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:33.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa33000cb80 tx=0x7fa33000a4c0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 shutdown_connections 2023-11-06T03:16:33.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa318065a60 0x7fa318067f20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa33815ce80 0x7fa33814d170 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa33815caa0 0x7fa338151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 --2- 172.21.15.157:0/4138726172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3380ce1a0 0x7fa33815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:33.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 >> 172.21.15.157:0/4138726172 conn(0x7fa338070790 msgr2=0x7fa3380b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:33.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 shutdown_connections 2023-11-06T03:16:33.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:33.943+0000 7fa33cc4b640 1 -- 172.21.15.157:0/4138726172 wait complete. 2023-11-06T03:16:33.961 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:16:34.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 Processor -- start 2023-11-06T03:16:34.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 -- start start 2023-11-06T03:16:34.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38b015b800 0x7f38b01622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38b00cb850 0x7f38b00b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00cbfc0 con 0x7f38b006bf20 2023-11-06T03:16:34.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b006d0d0 con 0x7f38b00cb850 2023-11-06T03:16:34.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38b5729640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b0162820 con 0x7f38b015b800 2023-11-06T03:16:34.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38b00cb850 0x7f38b00b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38ae7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f38b006bf20 0x7f38b00cb140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38158/0 (socket says 172.21.15.157:38158) 2023-11-06T03:16:34.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f38b00cb850 0x7f38b00b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56642/0 (socket says 172.21.15.157:56642) 2023-11-06T03:16:34.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38ae7fc640 1 -- 172.21.15.157:0/2874986454 learned_addr learned my addr 172.21.15.157:0/2874986454 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:34.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38aeffd640 1 --2- 172.21.15.157:0/2874986454 >> v2:172.21.15.195:3300/0 conn(0x7f38b015b800 0x7f38b01622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 -- 172.21.15.157:0/2874986454 >> v2:172.21.15.195:3300/0 conn(0x7f38b015b800 msgr2=0x7f38b01622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:34.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 --2- 172.21.15.157:0/2874986454 >> v2:172.21.15.195:3300/0 conn(0x7f38b015b800 0x7f38b01622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 -- 172.21.15.157:0/2874986454 >> v1:172.21.15.157:6789/0 conn(0x7f38b006bf20 legacy=0x7f38b00cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.135+0000 7f38af7fe640 1 -- 172.21.15.157:0/2874986454 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b006c4c0 con 0x7f38b00cb850 2023-11-06T03:16:34.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38af7fe640 1 --2- 172.21.15.157:0/2874986454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b00cb850 0x7f38b00b1950 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cc370 tx=0x7f38a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842de3fa30ef2764 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:34.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38adffb640 1 -- 172.21.15.157:0/2874986454 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00a7070 con 0x7f38b00cb850 2023-11-06T03:16:34.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38adffb640 1 -- 172.21.15.157:0/2874986454 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a00ab430 con 0x7f38b00cb850 2023-11-06T03:16:34.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38adffb640 1 -- 172.21.15.157:0/2874986454 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00ab7c0 con 0x7f38b00cb850 2023-11-06T03:16:34.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- 172.21.15.157:0/2874986454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b00cb850 msgr2=0x7f38b00b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 --2- 172.21.15.157:0/2874986454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b00cb850 0x7f38b00b1950 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cc370 tx=0x7f38a001c430 comp rx=0 tx=0).stop 2023-11-06T03:16:34.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- 172.21.15.157:0/2874986454 shutdown_connections 2023-11-06T03:16:34.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 --2- 172.21.15.157:0/2874986454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b00cb850 0x7f38b00b1950 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 --2- 172.21.15.157:0/2874986454 >> v2:172.21.15.195:3300/0 conn(0x7f38b015b800 0x7f38b01622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- 172.21.15.157:0/2874986454 >> 172.21.15.157:0/2874986454 conn(0x7f38b00b43b0 msgr2=0x7f38b0065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:34.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- 172.21.15.157:0/2874986454 shutdown_connections 2023-11-06T03:16:34.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- 172.21.15.157:0/2874986454 wait complete. 2023-11-06T03:16:34.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 Processor -- start 2023-11-06T03:16:34.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.139+0000 7f38b5729640 1 -- start start 2023-11-06T03:16:34.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b006bf20 0x7f38b00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b00cb850 0x7f38b00af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0153a00 con 0x7f38b015b800 2023-11-06T03:16:34.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00adf40 con 0x7f38b00cb850 2023-11-06T03:16:34.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00ae220 con 0x7f38b006bf20 2023-11-06T03:16:34.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56656/0 (socket says 172.21.15.157:56656) 2023-11-06T03:16:34.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 -- 172.21.15.157:0/3370191121 learned_addr learned my addr 172.21.15.157:0/3370191121 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:34.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38ae7fc640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b00cb850 0x7f38b00af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38aeffd640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b006bf20 0x7f38b00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 -- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b006bf20 msgr2=0x7f38b00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b006bf20 0x7f38b00ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 -- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b00cb850 msgr2=0x7f38b00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b00cb850 0x7f38b00af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 -- 172.21.15.157:0/3370191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009420 con 0x7f38b015b800 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38af7fe640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f38a00ad040 tx=0x7f38a001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00bf610 con 0x7f38b015b800 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a00bf770 con 0x7f38b015b800 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b00ae500 con 0x7f38b015b800 2023-11-06T03:16:34.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00bfb00 con 0x7f38b015b800 2023-11-06T03:16:34.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b00ae7e0 con 0x7f38b015b800 2023-11-06T03:16:34.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b01593c0 con 0x7f38b015b800 2023-11-06T03:16:34.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38a00b5400 con 0x7f38b015b800 2023-11-06T03:16:34.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 0x7f3890067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:34.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.143+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f38a011db00 con 0x7f38b015b800 2023-11-06T03:16:34.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.147+0000 7f38aeffd640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 0x7f3890067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:34.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.147+0000 7f38aeffd640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 0x7f3890067f20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e90 tx=0x7f38a800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:34.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.147+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38b01593c0 con 0x7f38b015b800 2023-11-06T03:16:34.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.291+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38b01593c0 con 0x7f38b015b800 2023-11-06T03:16:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.955+0000 7f38a77fe640 1 -- 172.21.15.157:0/3370191121 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38b01593c0 con 0x7f38b015b800 2023-11-06T03:16:34.952 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:16:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 msgr2=0x7f3890067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 0x7f3890067f20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009e90 tx=0x7f38a800b040 comp rx=0 tx=0).stop 2023-11-06T03:16:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 msgr2=0x7f38b0155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:34.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f38a00ad040 tx=0x7f38a001ac70 comp rx=0 tx=0).stop 2023-11-06T03:16:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 shutdown_connections 2023-11-06T03:16:34.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3890065a60 0x7f3890067f20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38b015b800 0x7f38b0155f60 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b00cb850 0x7f38b00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 --2- 172.21.15.157:0/3370191121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b006bf20 0x7f38b00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 >> 172.21.15.157:0/3370191121 conn(0x7f38b00b43b0 msgr2=0x7f38b00707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:34.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 shutdown_connections 2023-11-06T03:16:34.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:34.959+0000 7f38b5729640 1 -- 172.21.15.157:0/3370191121 wait complete. 2023-11-06T03:16:34.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:16:35.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 Processor -- start 2023-11-06T03:16:35.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 -- start start 2023-11-06T03:16:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efd780b1ee0 0x7efd780d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd780cfc10 0x7efd780b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd780b5b20 con 0x7efd780d2180 2023-11-06T03:16:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd780ce1d0 con 0x7efd780cfc10 2023-11-06T03:16:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7f7c8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd780d1dc0 con 0x7efd780b1ee0 2023-11-06T03:16:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7dd65640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd780cfc10 0x7efd780b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:35.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7cd63640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7efd780d2180 0x7efd780cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38168/0 (socket says 172.21.15.157:38168) 2023-11-06T03:16:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7cd63640 1 -- 172.21.15.157:0/1726701051 learned_addr learned my addr 172.21.15.157:0/1726701051 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7d564640 1 --2- 172.21.15.157:0/1726701051 >> v2:172.21.15.195:3300/0 conn(0x7efd780b1ee0 0x7efd780d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7dd65640 1 -- 172.21.15.157:0/1726701051 >> v2:172.21.15.195:3300/0 conn(0x7efd780b1ee0 msgr2=0x7efd780d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7dd65640 1 --2- 172.21.15.157:0/1726701051 >> v2:172.21.15.195:3300/0 conn(0x7efd780b1ee0 0x7efd780d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7dd65640 1 -- 172.21.15.157:0/1726701051 >> v1:172.21.15.157:6789/0 conn(0x7efd780d2180 legacy=0x7efd780cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd6ffff640 1 -- 172.21.15.157:0/1726701051 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061861657 0 0) 0x7efd780b5b20 con 0x7efd780d2180 2023-11-06T03:16:35.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.103+0000 7efd7dd65640 1 -- 172.21.15.157:0/1726701051 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780d1280 con 0x7efd780cfc10 2023-11-06T03:16:35.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7dd65640 1 --2- 172.21.15.157:0/1726701051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780cfc10 0x7efd780b6250 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7efd780b56e0 tx=0x7efd7001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae196afe551be8d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd6ffff640 1 -- 172.21.15.157:0/1726701051 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700a5910 con 0x7efd780cfc10 2023-11-06T03:16:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd6ffff640 1 -- 172.21.15.157:0/1726701051 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700a5a70 con 0x7efd780cfc10 2023-11-06T03:16:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd6ffff640 1 -- 172.21.15.157:0/1726701051 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700a5e00 con 0x7efd780cfc10 2023-11-06T03:16:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1726701051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780cfc10 msgr2=0x7efd780b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1726701051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780cfc10 0x7efd780b6250 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7efd780b56e0 tx=0x7efd7001c850 comp rx=0 tx=0).stop 2023-11-06T03:16:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1726701051 shutdown_connections 2023-11-06T03:16:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1726701051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780cfc10 0x7efd780b6250 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1726701051 >> v2:172.21.15.195:3300/0 conn(0x7efd780b1ee0 0x7efd780d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1726701051 >> 172.21.15.157:0/1726701051 conn(0x7efd78065930 msgr2=0x7efd78065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:35.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1726701051 shutdown_connections 2023-11-06T03:16:35.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1726701051 wait complete. 2023-11-06T03:16:35.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.107+0000 7efd7f7c8640 1 Processor -- start 2023-11-06T03:16:35.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- start start 2023-11-06T03:16:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd780b1ee0 0x7efd7814f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd780cfc10 0x7efd7814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7814dfa0 con 0x7efd780d2180 2023-11-06T03:16:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7814f4f0 con 0x7efd780cfc10 2023-11-06T03:16:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7814e8e0 con 0x7efd780b1ee0 2023-11-06T03:16:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56676/0 (socket says 172.21.15.157:56676) 2023-11-06T03:16:35.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 -- 172.21.15.157:0/1377133305 learned_addr learned my addr 172.21.15.157:0/1377133305 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 -- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd780b1ee0 msgr2=0x7efd7814f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd780b1ee0 0x7efd7814f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 -- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd780cfc10 msgr2=0x7efd7814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd780cfc10 0x7efd7814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 -- 172.21.15.157:0/1377133305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70009a70 con 0x7efd780d2180 2023-11-06T03:16:35.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7dd65640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7efd70019460 tx=0x7efd700195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700b9920 con 0x7efd780d2180 2023-11-06T03:16:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700b9aa0 con 0x7efd780d2180 2023-11-06T03:16:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700b9e50 con 0x7efd780d2180 2023-11-06T03:16:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7814ebc0 con 0x7efd780d2180 2023-11-06T03:16:35.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7814f180 con 0x7efd780d2180 2023-11-06T03:16:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7804e050 con 0x7efd780d2180 2023-11-06T03:16:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.111+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd700ad440 con 0x7efd780d2180 2023-11-06T03:16:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.115+0000 7efd6dffb640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 0x7efd5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.115+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5797+0+0 (secure 0 0 0) 0x7efd7011e6f0 con 0x7efd780d2180 2023-11-06T03:16:35.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.115+0000 7efd7d564640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 0x7efd5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.115+0000 7efd7d564640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 0x7efd5c067f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7efd740098f0 tx=0x7efd74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:35.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.119+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efd700ec170 con 0x7efd780d2180 2023-11-06T03:16:35.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.283+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd7814f660 con 0x7efd780d2180 2023-11-06T03:16:35.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.967+0000 7efd6dffb640 1 -- 172.21.15.157:0/1377133305 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 169+0+0 (secure 0 0 0) 0x7efd700ef790 con 0x7efd780d2180 2023-11-06T03:16:35.963 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:16:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 msgr2=0x7efd5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 0x7efd5c067f20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7efd740098f0 tx=0x7efd74008040 comp rx=0 tx=0).stop 2023-11-06T03:16:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 msgr2=0x7efd7815abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:35.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7efd70019460 tx=0x7efd700195f0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 shutdown_connections 2023-11-06T03:16:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd5c065a60 0x7efd5c067f20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd780d2180 0x7efd7815abf0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd780cfc10 0x7efd7814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 --2- 172.21.15.157:0/1377133305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd780b1ee0 0x7efd7814f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 >> 172.21.15.157:0/1377133305 conn(0x7efd78065930 msgr2=0x7efd780ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 shutdown_connections 2023-11-06T03:16:35.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:35.971+0000 7efd7f7c8640 1 -- 172.21.15.157:0/1377133305 wait complete. 2023-11-06T03:16:35.989 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:16:36.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.115+0000 7f9201c20640 1 Processor -- start 2023-11-06T03:16:36.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.115+0000 7f9201c20640 1 -- start start 2023-11-06T03:16:36.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f9201c20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc15cc70 0x7f91fc0d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f9201c20640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f9201c20640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc160830 con 0x7f91fc0d25a0 2023-11-06T03:16:36.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f9201c20640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc1614b0 con 0x7f91fc0d00a0 2023-11-06T03:16:36.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f9201c20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc160b10 con 0x7f91fc15cc70 2023-11-06T03:16:36.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56692/0 (socket says 172.21.15.157:56692) 2023-11-06T03:16:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 -- 172.21.15.157:0/453614105 learned_addr learned my addr 172.21.15.157:0/453614105 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc15cc70 0x7f91fc0d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 -- 172.21.15.157:0/453614105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc15cc70 msgr2=0x7f91fc0d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:36.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc15cc70 0x7f91fc0d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 -- 172.21.15.157:0/453614105 >> v1:172.21.15.195:6789/0 conn(0x7f91fc0d25a0 legacy=0x7f91fc0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:36.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 -- 172.21.15.157:0/453614105 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc160df0 con 0x7f91fc0d00a0 2023-11-06T03:16:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fbfff640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009e50 tx=0x7f91f401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63946631b9349a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fa7fc640 1 -- 172.21.15.157:0/453614105 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40af400 con 0x7f91fc0d00a0 2023-11-06T03:16:36.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fa7fc640 1 -- 172.21.15.157:0/453614105 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f40ab050 con 0x7f91fc0d00a0 2023-11-06T03:16:36.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.119+0000 7f91fa7fc640 1 -- 172.21.15.157:0/453614105 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40af780 con 0x7f91fc0d00a0 2023-11-06T03:16:36.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/453614105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d00a0 msgr2=0x7f91fc0d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009e50 tx=0x7f91f401f1c0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/453614105 shutdown_connections 2023-11-06T03:16:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d00a0 0x7f91fc0d0bb0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- 172.21.15.157:0/453614105 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc15cc70 0x7f91fc0d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/453614105 >> 172.21.15.157:0/453614105 conn(0x7f91fc070560 msgr2=0x7f91fc070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/453614105 shutdown_connections 2023-11-06T03:16:36.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/453614105 wait complete. 2023-11-06T03:16:36.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 Processor -- start 2023-11-06T03:16:36.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- start start 2023-11-06T03:16:36.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0d00a0 0x7f91fc0cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 0x7f91fc0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc15fe60 con 0x7f91fc0d25a0 2023-11-06T03:16:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc15dc80 con 0x7f91fc15cc70 2023-11-06T03:16:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc160280 con 0x7f91fc0d00a0 2023-11-06T03:16:36.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56704/0 (socket says 172.21.15.157:56704) 2023-11-06T03:16:36.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 -- 172.21.15.157:0/2968385306 learned_addr learned my addr 172.21.15.157:0/2968385306 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:36.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0d00a0 0x7f91fc0cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91fbfff640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 0x7f91fc0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 -- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0d00a0 msgr2=0x7f91fc0cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0d00a0 0x7f91fc0cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 -- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 msgr2=0x7f91fc0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 0x7f91fc0b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 -- 172.21.15.157:0/2968385306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4009b30 con 0x7f91fc0d25a0 2023-11-06T03:16:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91fbfff640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 0x7f91fc0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91faffd640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c480 tx=0x7f91f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0012830 con 0x7f91fc0d25a0 2023-11-06T03:16:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00129b0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc15f1e0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0012d60 con 0x7f91fc0d25a0 2023-11-06T03:16:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.123+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc15f7a0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc15ffa0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f91f001c460 con 0x7f91fc0d25a0 2023-11-06T03:16:36.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f91e3fff640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 0x7f91e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f91f0082e40 con 0x7f91fc0d25a0 2023-11-06T03:16:36.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 0x7f91e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.127+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 0x7f91e8067f20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00b930 tx=0x7f91ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.131+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f91fc15ffa0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.296 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.299+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f91fc15ffa0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f91e3fff640 1 -- 172.21.15.157:0/2968385306 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f91fc15ffa0 con 0x7f91fc0d25a0 2023-11-06T03:16:36.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 msgr2=0x7f91e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 0x7f91e8067f20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00b930 tx=0x7f91ec008040 comp rx=0 tx=0).stop 2023-11-06T03:16:36.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 msgr2=0x7f91fc0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c480 tx=0x7f91f000c940 comp rx=0 tx=0).stop 2023-11-06T03:16:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 shutdown_connections 2023-11-06T03:16:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91e8065a60 0x7f91e8067f20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc15cc70 0x7f91fc0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0d25a0 0x7f91fc0b70c0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 --2- 172.21.15.157:0/2968385306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0d00a0 0x7f91fc0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 >> 172.21.15.157:0/2968385306 conn(0x7f91fc070560 msgr2=0x7f91fc0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 shutdown_connections 2023-11-06T03:16:36.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.307+0000 7f9201c20640 1 -- 172.21.15.157:0/2968385306 wait complete. 2023-11-06T03:16:36.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2023-11-06T03:16:36.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 Processor -- start 2023-11-06T03:16:36.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 -- start start 2023-11-06T03:16:36.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15ea90 0x7f3c5c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c15f580 con 0x7f3c5c1551e0 2023-11-06T03:16:36.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5c0704b0 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c62ea1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c15efc0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56710/0 (socket says 172.21.15.157:56710) 2023-11-06T03:16:36.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 -- 172.21.15.157:0/510531119 learned_addr learned my addr 172.21.15.157:0/510531119 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:36.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c6143e640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15ea90 0x7f3c5c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 -- 172.21.15.157:0/510531119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15ea90 msgr2=0x7f3c5c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:36.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15ea90 0x7f3c5c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 -- 172.21.15.157:0/510531119 >> v1:172.21.15.195:6789/0 conn(0x7f3c5c1551e0 legacy=0x7f3c5c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:36.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.451+0000 7f3c5bfff640 1 -- 172.21.15.157:0/510531119 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c15f2a0 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c5bfff640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009870 tx=0x7f3c5001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccddd9213e40092b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c5b7fe640 1 -- 172.21.15.157:0/510531119 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500ab600 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c5b7fe640 1 -- 172.21.15.157:0/510531119 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c500ab760 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c5b7fe640 1 -- 172.21.15.157:0/510531119 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500abaf0 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- 172.21.15.157:0/510531119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15e5e0 msgr2=0x7f3c5c0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009870 tx=0x7f3c5001c440 comp rx=0 tx=0).stop 2023-11-06T03:16:36.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- 172.21.15.157:0/510531119 shutdown_connections 2023-11-06T03:16:36.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15ea90 0x7f3c5c0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/510531119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15e5e0 0x7f3c5c0b3410 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- 172.21.15.157:0/510531119 >> 172.21.15.157:0/510531119 conn(0x7f3c5c065930 msgr2=0x7f3c5c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:36.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- 172.21.15.157:0/510531119 shutdown_connections 2023-11-06T03:16:36.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- 172.21.15.157:0/510531119 wait complete. 2023-11-06T03:16:36.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 Processor -- start 2023-11-06T03:16:36.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- start start 2023-11-06T03:16:36.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c5c1551e0 0x7f3c5c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15e5e0 0x7f3c5c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c155d20 con 0x7f3c5c15ea90 2023-11-06T03:16:36.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c158470 con 0x7f3c5c1551e0 2023-11-06T03:16:36.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c62ea1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c1586f0 con 0x7f3c5c15e5e0 2023-11-06T03:16:36.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c6143e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c6143e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56718/0 (socket says 172.21.15.157:56718) 2023-11-06T03:16:36.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.455+0000 7f3c6143e640 1 -- 172.21.15.157:0/228777089 learned_addr learned my addr 172.21.15.157:0/228777089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:36.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c60c3d640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c5c1551e0 0x7f3c5c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c5bfff640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15e5e0 0x7f3c5c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 -- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15e5e0 msgr2=0x7f3c5c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15e5e0 0x7f3c5c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 -- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c5c1551e0 msgr2=0x7f3c5c14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c5c1551e0 0x7f3c5c14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 -- 172.21.15.157:0/228777089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c500094d0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c6143e640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f3c54007e60 tx=0x7f3c5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014420 con 0x7f3c5c15ea90 2023-11-06T03:16:36.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c540145a0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014950 con 0x7f3c5c15ea90 2023-11-06T03:16:36.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c155660 con 0x7f3c5c15ea90 2023-11-06T03:16:36.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c1589d0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c155e60 con 0x7f3c5c15ea90 2023-11-06T03:16:36.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3c54014ab0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 0x7f3c40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:36.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c54083890 con 0x7f3c5c15ea90 2023-11-06T03:16:36.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.459+0000 7f3c60c3d640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 0x7f3c40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:36.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.463+0000 7f3c60c3d640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 0x7f3c40067f20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004e30 tx=0x7f3c48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:36.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.463+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3c54051480 con 0x7f3c5c15ea90 2023-11-06T03:16:36.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.675+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f3c5c072750 con 0x7f3c5c15ea90 2023-11-06T03:16:36.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.967+0000 7f3c597fa640 1 -- 172.21.15.157:0/228777089 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v576) v1 ==== 140+0+0 (secure 0 0 0) 0x7f3c54054aa0 con 0x7f3c5c15ea90 2023-11-06T03:16:36.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 msgr2=0x7f3c40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 0x7f3c40067f20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004e30 tx=0x7f3c48009040 comp rx=0 tx=0).stop 2023-11-06T03:16:36.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 msgr2=0x7f3c5c14d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f3c54007e60 tx=0x7f3c5400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 shutdown_connections 2023-11-06T03:16:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3c40065a60 0x7f3c40067f20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c5c15ea90 0x7f3c5c14d9c0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3c5c15e5e0 0x7f3c5c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 --2- 172.21.15.157:0/228777089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3c5c1551e0 0x7f3c5c14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 >> 172.21.15.157:0/228777089 conn(0x7f3c5c065930 msgr2=0x7f3c5c0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 shutdown_connections 2023-11-06T03:16:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:36.971+0000 7f3c62ea1640 1 -- 172.21.15.157:0/228777089 wait complete. 2023-11-06T03:16:36.987 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2023-11-06T03:16:36.988 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2023-11-06T03:16:36.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2023-11-06T03:16:37.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.115+0000 7f1a0ce3c640 1 Processor -- start 2023-11-06T03:16:37.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.115+0000 7f1a0ce3c640 1 -- start start 2023-11-06T03:16:37.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0ce3c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1a0806f470 0x7f1a080adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0ce3c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0806c490 con 0x7f1a08156a70 2023-11-06T03:16:37.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0ce3c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0806bb60 con 0x7f1a080ae020 2023-11-06T03:16:37.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0ce3c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0806b950 con 0x7f1a0806f470 2023-11-06T03:16:37.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1a08156a70 0x7f1a080ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38170/0 (socket says 172.21.15.157:38170) 2023-11-06T03:16:37.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 -- 172.21.15.157:0/2574717666 learned_addr learned my addr 172.21.15.157:0/2574717666 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:37.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890438185 0 0) 0x7f1a0806c490 con 0x7f1a08156a70 2023-11-06T03:16:37.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003530 con 0x7f1a08156a70 2023-11-06T03:16:37.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a05d9b640 1 --2- 172.21.15.157:0/2574717666 >> v2:172.21.15.195:3300/0 conn(0x7f1a0806f470 0x7f1a080adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046888155 0 0) 0x7f19f0003530 con 0x7f1a08156a70 2023-11-06T03:16:37.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 >> v2:172.21.15.195:3300/0 conn(0x7f1a0806f470 msgr2=0x7f1a080adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:37.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 --2- 172.21.15.157:0/2574717666 >> v2:172.21.15.195:3300/0 conn(0x7f1a0806f470 0x7f1a080adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 >> v1:172.21.15.195:6789/0 conn(0x7f1a080ae020 legacy=0x7f1a080b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0806be40 con 0x7f1a08156a70 2023-11-06T03:16:37.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f19f8001680 con 0x7f1a08156a70 2023-11-06T03:16:37.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 >> v1:172.21.15.157:6789/0 conn(0x7f1a08156a70 legacy=0x7f1a080ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f19f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19f00047d0 0x7f19f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19f0005190 0x7f19f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0806b950 con 0x7f19f0003980 2023-11-06T03:16:37.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f000c0e0 con 0x7f19f00047d0 2023-11-06T03:16:37.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f000c2a0 con 0x7f19f0005190 2023-11-06T03:16:37.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f19f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a05d9b640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19f00047d0 0x7f19f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a06d9d640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19f0005190 0x7f19f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 -- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19f0005190 msgr2=0x7f19f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:37.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19f0005190 0x7f19f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 -- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19f00047d0 msgr2=0x7f19f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:37.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19f00047d0 0x7f19f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 -- 172.21.15.157:0/2574717666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0003530 con 0x7f19f0003980 2023-11-06T03:16:37.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a06d9d640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f19f0005190 0x7f19f000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:37.123 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a06d9d640 1 -- 172.21.15.157:0/2574717666 reap_dead start 2023-11-06T03:16:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.119+0000 7f1a0659c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f19f0003da0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c240 tx=0x7f19f801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b456fb62a7a76b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8005ae0 con 0x7f19f0003980 2023-11-06T03:16:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8005c60 con 0x7f19f0003980 2023-11-06T03:16:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0559a640 1 -- 172.21.15.157:0/2574717666 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80a5890 con 0x7f19f0003980 2023-11-06T03:16:37.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 msgr2=0x7f19f0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f19f0003da0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c240 tx=0x7f19f801cc10 comp rx=0 tx=0).stop 2023-11-06T03:16:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/2574717666 shutdown_connections 2023-11-06T03:16:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/2574717666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f19f0003da0 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/2574717666 >> 172.21.15.157:0/2574717666 conn(0x7f1a08065930 msgr2=0x7f1a08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:37.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/2574717666 shutdown_connections 2023-11-06T03:16:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/2574717666 wait complete. 2023-11-06T03:16:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 Processor -- start 2023-11-06T03:16:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- start start 2023-11-06T03:16:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1a0813f7b0 0x7f1a0814dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 0x7f1a081574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0814bfc0 con 0x7f19f0003980 2023-11-06T03:16:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0813cc50 con 0x7f1a08157060 2023-11-06T03:16:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08157a20 con 0x7f1a0813f7b0 2023-11-06T03:16:37.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56734/0 (socket says 172.21.15.157:56734) 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 -- 172.21.15.157:0/3246802178 learned_addr learned my addr 172.21.15.157:0/3246802178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a05d9b640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1a0813f7b0 0x7f1a0814dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a06d9d640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 0x7f1a081574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 -- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1a0813f7b0 msgr2=0x7f1a0814dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1a0813f7b0 0x7f1a0814dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 -- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 msgr2=0x7f1a081574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 0x7f1a081574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 -- 172.21.15.157:0/3246802178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0814b7b0 con 0x7f19f0003980 2023-11-06T03:16:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a06d9d640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 0x7f1a081574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:37.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0659c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f19f8001940 tx=0x7f19f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:37.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80b4070 con 0x7f19f0003980 2023-11-06T03:16:37.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8001500 con 0x7f19f0003980 2023-11-06T03:16:37.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0814ba00 con 0x7f19f0003980 2023-11-06T03:16:37.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80a5ad0 con 0x7f19f0003980 2023-11-06T03:16:37.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.123+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0813ce60 con 0x7f19f0003980 2023-11-06T03:16:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a08072750 con 0x7f19f0003980 2023-11-06T03:16:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f19f80a5c30 con 0x7f19f0003980 2023-11-06T03:16:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f19f6ffd640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 0x7f19e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f19f811cc20 con 0x7f19f0003980 2023-11-06T03:16:37.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f1a05d9b640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 0x7f19e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.127+0000 7f1a05d9b640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 0x7f19e8067f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1a0814ab60 tx=0x7f19fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:37.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.131+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f19f80ea780 con 0x7f19f0003980 2023-11-06T03:16:37.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.339+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f1a080704c0 con 0x7f19f0003980 2023-11-06T03:16:37.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.975+0000 7f19f6ffd640 1 -- 172.21.15.157:0/3246802178 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v577) v1 ==== 136+0+0 (secure 0 0 0) 0x7f19f80edda0 con 0x7f19f0003980 2023-11-06T03:16:37.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 msgr2=0x7f19e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 0x7f19e8067f20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1a0814ab60 tx=0x7f19fc008040 comp rx=0 tx=0).stop 2023-11-06T03:16:37.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 msgr2=0x7f1a0814ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:37.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f19f8001940 tx=0x7f19f0005800 comp rx=0 tx=0).stop 2023-11-06T03:16:37.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 shutdown_connections 2023-11-06T03:16:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f19e8065a60 0x7f19e8067f20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1a08157060 0x7f1a081574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1a0813f7b0 0x7f1a0814dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 --2- 172.21.15.157:0/3246802178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19f0003980 0x7f1a0814ae40 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 >> 172.21.15.157:0/3246802178 conn(0x7f1a08065930 msgr2=0x7f1a08159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 shutdown_connections 2023-11-06T03:16:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:37.979+0000 7f1a0ce3c640 1 -- 172.21.15.157:0/3246802178 wait complete. 2023-11-06T03:16:37.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:16:38.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 Processor -- start 2023-11-06T03:16:38.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 -- start start 2023-11-06T03:16:38.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe05015ce80 0x7fe0500cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0500d2920 con 0x7fe05015caa0 2023-11-06T03:16:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0500d18a0 con 0x7fe0500d00b0 2023-11-06T03:16:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe05877b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0500d0520 con 0x7fe05015ce80 2023-11-06T03:16:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe055d16640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe05015ce80 0x7fe0500cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe056517640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe05015caa0 0x7fe0500cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38178/0 (socket says 172.21.15.157:38178) 2023-11-06T03:16:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe056517640 1 -- 172.21.15.157:0/3090590080 learned_addr learned my addr 172.21.15.157:0/3090590080 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe055d16640 1 -- 172.21.15.157:0/3090590080 >> v1:172.21.15.195:6789/0 conn(0x7fe0500d00b0 legacy=0x7fe0500b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe055d16640 1 -- 172.21.15.157:0/3090590080 >> v1:172.21.15.157:6789/0 conn(0x7fe05015caa0 legacy=0x7fe0500cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.131+0000 7fe055d16640 1 -- 172.21.15.157:0/3090590080 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0500d1b60 con 0x7fe05015ce80 2023-11-06T03:16:38.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe055d16640 1 --2- 172.21.15.157:0/3090590080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0500cfb70 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0097c0 tx=0x7fe04c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fa11a57d5182f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe055515640 1 -- 172.21.15.157:0/3090590080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0ab610 con 0x7fe05015ce80 2023-11-06T03:16:38.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe055515640 1 -- 172.21.15.157:0/3090590080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c0ab770 con 0x7fe05015ce80 2023-11-06T03:16:38.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe055515640 1 -- 172.21.15.157:0/3090590080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0abb00 con 0x7fe05015ce80 2023-11-06T03:16:38.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- 172.21.15.157:0/3090590080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 msgr2=0x7fe0500cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 --2- 172.21.15.157:0/3090590080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0500cfb70 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0097c0 tx=0x7fe04c01c410 comp rx=0 tx=0).stop 2023-11-06T03:16:38.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- 172.21.15.157:0/3090590080 shutdown_connections 2023-11-06T03:16:38.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 --2- 172.21.15.157:0/3090590080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0500cfb70 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- 172.21.15.157:0/3090590080 >> 172.21.15.157:0/3090590080 conn(0x7fe050070790 msgr2=0x7fe050070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:38.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- 172.21.15.157:0/3090590080 shutdown_connections 2023-11-06T03:16:38.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- 172.21.15.157:0/3090590080 wait complete. 2023-11-06T03:16:38.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 Processor -- start 2023-11-06T03:16:38.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- start start 2023-11-06T03:16:38.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0500d00b0 0x7fe050158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe05015caa0 0x7fe0501549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0501586f0 con 0x7fe05015ce80 2023-11-06T03:16:38.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05015de40 con 0x7fe0500d00b0 2023-11-06T03:16:38.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe05877b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05015e120 con 0x7fe05015caa0 2023-11-06T03:16:38.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe056d18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe056d18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56756/0 (socket says 172.21.15.157:56756) 2023-11-06T03:16:38.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe056d18640 1 -- 172.21.15.157:0/1797645000 learned_addr learned my addr 172.21.15.157:0/1797645000 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:38.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe055d16640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe05015caa0 0x7fe0501549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.135+0000 7fe056517640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0500d00b0 0x7fe050158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 -- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe05015caa0 msgr2=0x7fe0501549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe05015caa0 0x7fe0501549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 -- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0500d00b0 msgr2=0x7fe050158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0500d00b0 0x7fe050158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 -- 172.21.15.157:0/1797645000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c009420 con 0x7fe05015ce80 2023-11-06T03:16:38.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056d18640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b970 tx=0x7fe04800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800cb40 con 0x7fe05015ce80 2023-11-06T03:16:38.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800cca0 con 0x7fe05015ce80 2023-11-06T03:16:38.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05015e400 con 0x7fe05015ce80 2023-11-06T03:16:38.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048014680 con 0x7fe05015ce80 2023-11-06T03:16:38.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0501591a0 con 0x7fe05015ce80 2023-11-06T03:16:38.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05015dca0 con 0x7fe05015ce80 2023-11-06T03:16:38.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe04801d3b0 con 0x7fe05015ce80 2023-11-06T03:16:38.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 0x7fe034067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe048082e80 con 0x7fe05015ce80 2023-11-06T03:16:38.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056517640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 0x7fe034067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.139+0000 7fe056517640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 0x7fe034067f20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe040000f70 tx=0x7fe040007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.143+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe05015dca0 con 0x7fe05015ce80 2023-11-06T03:16:38.294 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.295+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe05015dca0 con 0x7fe05015ce80 2023-11-06T03:16:38.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.303+0000 7fe046ffd640 1 -- 172.21.15.157:0/1797645000 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe05015dca0 con 0x7fe05015ce80 2023-11-06T03:16:38.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 msgr2=0x7fe034067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 0x7fe034067f20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe040000f70 tx=0x7fe040007e90 comp rx=0 tx=0).stop 2023-11-06T03:16:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 msgr2=0x7fe0501596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b970 tx=0x7fe04800be30 comp rx=0 tx=0).stop 2023-11-06T03:16:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 shutdown_connections 2023-11-06T03:16:38.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe034065a60 0x7fe034067f20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe05015ce80 0x7fe0501596b0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe05015caa0 0x7fe0501549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 --2- 172.21.15.157:0/1797645000 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0500d00b0 0x7fe050158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 >> 172.21.15.157:0/1797645000 conn(0x7fe050070790 msgr2=0x7fe0500b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:38.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 shutdown_connections 2023-11-06T03:16:38.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.307+0000 7fe05877b640 1 -- 172.21.15.157:0/1797645000 wait complete. 2023-11-06T03:16:38.321 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:16:38.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:16:38.368 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:16:38.428 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:16:38.428 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:16:38.429 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:38.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:16:38.434 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:16:38.435 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:38.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:16:38.442 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:16:38.442 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:38.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:16:38.484 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:16:38.484 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:38.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:16:38.491 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:16:38.491 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:38.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 Processor -- start 2023-11-06T03:16:38.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 -- start start 2023-11-06T03:16:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 0x7f015806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015815f580 con 0x7f015815e5e0 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015815efc0 con 0x7f01581551e0 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015fd4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015815f2a0 con 0x7f01580b3b20 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015e2eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015e2eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:44424/0 (socket says 172.21.15.157:44424) 2023-11-06T03:16:38.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015e2eb640 1 -- 172.21.15.157:0/2720479875 learned_addr learned my addr 172.21.15.157:0/2720479875 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:38.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.655+0000 7f015daea640 1 --2- 172.21.15.157:0/2720479875 >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 0x7f015806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015cae8640 1 -- 172.21.15.157:0/2720479875 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539204766 0 0) 0x7f015815f580 con 0x7f015815e5e0 2023-11-06T03:16:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015cae8640 1 -- 172.21.15.157:0/2720479875 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0144003530 con 0x7f015815e5e0 2023-11-06T03:16:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015e2eb640 1 -- 172.21.15.157:0/2720479875 >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 msgr2=0x7f015806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015e2eb640 1 --2- 172.21.15.157:0/2720479875 >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 0x7f015806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015e2eb640 1 -- 172.21.15.157:0/2720479875 >> v1:172.21.15.195:6789/0 conn(0x7f015815e5e0 legacy=0x7f01580b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015e2eb640 1 -- 172.21.15.157:0/2720479875 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015806f5a0 con 0x7f01580b3b20 2023-11-06T03:16:38.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015daea640 1 --2- 172.21.15.157:0/2720479875 >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 0x7f015806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:38.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015e2eb640 1 --2- 172.21.15.157:0/2720479875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94cb461f07a9585 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015cae8640 1 -- 172.21.15.157:0/2720479875 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5830 con 0x7f01580b3b20 2023-11-06T03:16:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015cae8640 1 -- 172.21.15.157:0/2720479875 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01500a5990 con 0x7f01580b3b20 2023-11-06T03:16:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015cae8640 1 -- 172.21.15.157:0/2720479875 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5d20 con 0x7f01580b3b20 2023-11-06T03:16:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015fd4e640 1 -- 172.21.15.157:0/2720479875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 msgr2=0x7f015815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.659+0000 7f015fd4e640 1 --2- 172.21.15.157:0/2720479875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015001c950 comp rx=0 tx=0).stop 2023-11-06T03:16:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/2720479875 shutdown_connections 2023-11-06T03:16:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 --2- 172.21.15.157:0/2720479875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01580b3b20 0x7f015815c1b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 --2- 172.21.15.157:0/2720479875 >> v2:172.21.15.195:3300/0 conn(0x7f01581551e0 0x7f015806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/2720479875 >> 172.21.15.157:0/2720479875 conn(0x7f0158065930 msgr2=0x7f0158065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/2720479875 shutdown_connections 2023-11-06T03:16:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/2720479875 wait complete. 2023-11-06T03:16:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 Processor -- start 2023-11-06T03:16:38.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- start start 2023-11-06T03:16:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01581551e0 0x7f0158152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 0x7f015806b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158134e90 con 0x7f01580b3b20 2023-11-06T03:16:38.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158135100 con 0x7f015815e5e0 2023-11-06T03:16:38.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01581353e0 con 0x7f01581551e0 2023-11-06T03:16:38.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38588/0 (socket says 172.21.15.157:38588) 2023-11-06T03:16:38.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 -- 172.21.15.157:0/802340633 learned_addr learned my addr 172.21.15.157:0/802340633 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:38.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015e2eb640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 0x7f015806b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015d2e9640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01581551e0 0x7f0158152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 -- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01581551e0 msgr2=0x7f0158152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01581551e0 0x7f0158152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 -- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 msgr2=0x7f015806b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 0x7f015806b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 -- 172.21.15.157:0/802340633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0148009420 con 0x7f01580b3b20 2023-11-06T03:16:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015e2eb640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 0x7f015806b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015daea640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f014800ebe0 tx=0x7f014800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800eee0 con 0x7f01580b3b20 2023-11-06T03:16:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014800cc30 con 0x7f01580b3b20 2023-11-06T03:16:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0150009b30 con 0x7f01580b3b20 2023-11-06T03:16:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148013470 con 0x7f01580b3b20 2023-11-06T03:16:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0158152bf0 con 0x7f01580b3b20 2023-11-06T03:16:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.663+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158072750 con 0x7f01580b3b20 2023-11-06T03:16:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.667+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01480135d0 con 0x7f01580b3b20 2023-11-06T03:16:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.667+0000 7f014e7fc640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 0x7f013c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.667+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0148083b60 con 0x7f01580b3b20 2023-11-06T03:16:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.667+0000 7f015d2e9640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 0x7f013c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.667+0000 7f015d2e9640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 0x7f013c067f20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f01581468b0 tx=0x7f0154009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.671+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f01480516c0 con 0x7f01580b3b20 2023-11-06T03:16:38.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0158071730 con 0x7f01580b3b20 2023-11-06T03:16:38.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f014e7fc640 1 -- 172.21.15.157:0/802340633 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+3541 (secure 0 0 0) 0x7f0148017df0 con 0x7f01580b3b20 2023-11-06T03:16:38.856 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:38.856 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/633623674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":633623674},{"type":"v1","addr":"172.21.15.195:6835","nonce":633623674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/131430013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":131430013},{"type":"v1","addr":"172.21.15.195:6837","nonce":131430013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":9912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4117500760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4117500760},{"type":"v1","addr":"172.21.15.157:6833","nonce":4117500760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":10317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3810079363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3810079363},{"type":"v1","addr":"172.21.15.157:6835","nonce":3810079363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":113}],"filesystems":[]} 2023-11-06T03:16:38.856 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 113 2023-11-06T03:16:38.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 msgr2=0x7f013c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 0x7f013c067f20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f01581468b0 tx=0x7f0154009040 comp rx=0 tx=0).stop 2023-11-06T03:16:38.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 msgr2=0x7f015814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:38.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f014800ebe0 tx=0x7f014800c520 comp rx=0 tx=0).stop 2023-11-06T03:16:38.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 shutdown_connections 2023-11-06T03:16:38.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f013c065a60 0x7f013c067f20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f015815e5e0 0x7f015806b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f01581551e0 0x7f0158152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 --2- 172.21.15.157:0/802340633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01580b3b20 0x7f015814ef10 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:38.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 >> 172.21.15.157:0/802340633 conn(0x7f0158065930 msgr2=0x7f01580afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:38.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 shutdown_connections 2023-11-06T03:16:38.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:38.859+0000 7f015fd4e640 1 -- 172.21.15.157:0/802340633 wait complete. 2023-11-06T03:16:38.876 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:16:38.876 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:16:38.877 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:16:38.877 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:16:38.877 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:16:38.878 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:16:38.878 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:16:38.878 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:16:38.879 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:16:38.879 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:16:38.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:16:38.879 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:16:38.880 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:16:38.880 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:16:38.880 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:16:38.880 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:16:38.888 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:16:38.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:16:38.889 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:16:38.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:16:38.896 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:16:38.897 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:16:38.898 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:16:38.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:16:38.932 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:16:38.932 DEBUG:teuthology.parallel:result is None 2023-11-06T03:16:38.933 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:16:38.933 DEBUG:teuthology.parallel:result is None 2023-11-06T03:16:38.939 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:16:38.940 DEBUG:teuthology.parallel:result is None 2023-11-06T03:16:38.940 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:16:38.941 DEBUG:teuthology.parallel:result is None 2023-11-06T03:16:38.941 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:16:38.993 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:16:38.996 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:16:39.011 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:16:39.012 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:16:39.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 Processor -- start 2023-11-06T03:16:39.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 -- start start 2023-11-06T03:16:39.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 0x7f6e6c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0b5440 con 0x7f6e6c065930 2023-11-06T03:16:39.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c15e910 con 0x7f6e6c0b4ac0 2023-11-06T03:16:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e71167640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15ebf0 con 0x7f6e6c1444b0 2023-11-06T03:16:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6a59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 0x7f6e6c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:44474/0 (socket says 172.21.15.157:44474) 2023-11-06T03:16:39.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/135206453 learned_addr learned my addr 172.21.15.157:0/135206453 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6b59e640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6e6c065930 0x7f6e6c065d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48774/0 (socket says 172.21.15.157:48774) 2023-11-06T03:16:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/135206453 >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 msgr2=0x7f6e6c0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:39.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 --2- 172.21.15.157:0/135206453 >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 0x7f6e6c0b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:39.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.059+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/135206453 >> v1:172.21.15.195:6789/0 conn(0x7f6e6c065930 legacy=0x7f6e6c065d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:39.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/135206453 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c15eed0 con 0x7f6e6c1444b0 2023-11-06T03:16:39.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e6a59c640 1 --2- 172.21.15.157:0/135206453 >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 0x7f6e6c0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:39.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e6ad9d640 1 --2- 172.21.15.157:0/135206453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009870 tx=0x7f6e6401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f97d0fb1212305 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e69d9b640 1 -- 172.21.15.157:0/135206453 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640ab610 con 0x7f6e6c1444b0 2023-11-06T03:16:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e69d9b640 1 -- 172.21.15.157:0/135206453 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640ab770 con 0x7f6e6c1444b0 2023-11-06T03:16:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e69d9b640 1 -- 172.21.15.157:0/135206453 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640abb00 con 0x7f6e6c1444b0 2023-11-06T03:16:39.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 -- 172.21.15.157:0/135206453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 msgr2=0x7f6e6c06c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 --2- 172.21.15.157:0/135206453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009870 tx=0x7f6e6401c540 comp rx=0 tx=0).stop 2023-11-06T03:16:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 -- 172.21.15.157:0/135206453 shutdown_connections 2023-11-06T03:16:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 --2- 172.21.15.157:0/135206453 >> v2:172.21.15.195:3300/0 conn(0x7f6e6c0b4ac0 0x7f6e6c0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:39.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 --2- 172.21.15.157:0/135206453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c1444b0 0x7f6e6c06c3d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 -- 172.21.15.157:0/135206453 >> 172.21.15.157:0/135206453 conn(0x7f6e6c069d70 msgr2=0x7f6e6c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 -- 172.21.15.157:0/135206453 shutdown_connections 2023-11-06T03:16:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.063+0000 7f6e71167640 1 -- 172.21.15.157:0/135206453 wait complete. 2023-11-06T03:16:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 Processor -- start 2023-11-06T03:16:39.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- start start 2023-11-06T03:16:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 0x7f6e6c13a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 0x7f6e6c13c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c13c690 con 0x7f6e6c1444b0 2023-11-06T03:16:39.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1560e0 con 0x7f6e6c0b4ac0 2023-11-06T03:16:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1563c0 con 0x7f6e6c065930 2023-11-06T03:16:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6a59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 0x7f6e6c13c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 0x7f6e6c13a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38630/0 (socket says 172.21.15.157:38630) 2023-11-06T03:16:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6b59e640 1 -- 172.21.15.157:0/1940118560 learned_addr learned my addr 172.21.15.157:0/1940118560 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 msgr2=0x7f6e6c13c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 0x7f6e6c13c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/1940118560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 msgr2=0x7f6e6c1407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:39.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 -- 172.21.15.157:0/1940118560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e60009420 con 0x7f6e6c065930 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6a59c640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 0x7f6e6c13c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6b59e640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6ad9d640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 0x7f6e6c13a330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e640ac000 tx=0x7f6e6401ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640b4800 con 0x7f6e6c065930 2023-11-06T03:16:39.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640b4960 con 0x7f6e6c065930 2023-11-06T03:16:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640094d0 con 0x7f6e6c065930 2023-11-06T03:16:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640b4d10 con 0x7f6e6c065930 2023-11-06T03:16:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c156980 con 0x7f6e6c065930 2023-11-06T03:16:39.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c072750 con 0x7f6e6c065930 2023-11-06T03:16:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6e640c83a0 con 0x7f6e6c065930 2023-11-06T03:16:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 0x7f6e58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6e6411d880 con 0x7f6e6c065930 2023-11-06T03:16:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.067+0000 7f6e6a59c640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 0x7f6e58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:39.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.071+0000 7f6e6a59c640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 0x7f6e58067f20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b0ea0 tx=0x7f6e60009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:39.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.075+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6e640eb520 con 0x7f6e6c065930 2023-11-06T03:16:39.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:39.235+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6e6c0d00f0 con 0x7f6e6c065930 2023-11-06T03:16:40.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.063+0000 7f6e537fe640 1 -- 172.21.15.157:0/1940118560 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6e640ab5d0 con 0x7f6e6c065930 2023-11-06T03:16:40.061 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:16:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 msgr2=0x7f6e58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 0x7f6e58067f20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b0ea0 tx=0x7f6e60009190 comp rx=0 tx=0).stop 2023-11-06T03:16:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 msgr2=0x7f6e6c13a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 0x7f6e6c13a330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e640ac000 tx=0x7f6e6401ad80 comp rx=0 tx=0).stop 2023-11-06T03:16:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 shutdown_connections 2023-11-06T03:16:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6e58065a60 0x7f6e58067f20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e6c1444b0 0x7f6e6c1407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6e6c0b4ac0 0x7f6e6c13c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 --2- 172.21.15.157:0/1940118560 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6e6c065930 0x7f6e6c13a330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 >> 172.21.15.157:0/1940118560 conn(0x7f6e6c069d70 msgr2=0x7f6e6c0b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 shutdown_connections 2023-11-06T03:16:40.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.067+0000 7f6e71167640 1 -- 172.21.15.157:0/1940118560 wait complete. 2023-11-06T03:16:40.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:16:40.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 Processor -- start 2023-11-06T03:16:40.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 -- start start 2023-11-06T03:16:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5ea40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5ea406d000 0x7f5ea4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea415e440 con 0x7f5ea41599c0 2023-11-06T03:16:40.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea406c240 con 0x7f5ea406d000 2023-11-06T03:16:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5eab692640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406c520 con 0x7f5ea40b2af0 2023-11-06T03:16:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea942e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5ea41599c0 0x7f5ea40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45014/0 (socket says 172.21.15.157:45014) 2023-11-06T03:16:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea942e640 1 -- 172.21.15.157:0/157228125 learned_addr learned my addr 172.21.15.157:0/157228125 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71265782 0 0) 0x7f5ea415e440 con 0x7f5ea41599c0 2023-11-06T03:16:40.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5ea40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e90005740 con 0x7f5ea41599c0 2023-11-06T03:16:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> v2:172.21.15.195:3300/0 conn(0x7f5ea406d000 0x7f5ea4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1316277936 0 0) 0x7f5e90005740 con 0x7f5ea41599c0 2023-11-06T03:16:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 msgr2=0x7f5ea40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5ea40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 >> v2:172.21.15.195:3300/0 conn(0x7f5ea406d000 msgr2=0x7f5ea4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 --2- 172.21.15.157:0/157228125 >> v2:172.21.15.195:3300/0 conn(0x7f5ea406d000 0x7f5ea4159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea406c800 con 0x7f5ea41599c0 2023-11-06T03:16:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5ea0001680 con 0x7f5ea41599c0 2023-11-06T03:16:40.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 >> v1:172.21.15.157:6789/0 conn(0x7f5ea41599c0 legacy=0x7f5ea40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e90005b90 0x7f5e90005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5e900069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e90006f20 0x7f5e900073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406c240 con 0x7f5e90006f20 2023-11-06T03:16:40.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406c520 con 0x7f5e90005b90 2023-11-06T03:16:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9000dcd0 con 0x7f5ea40b2af0 2023-11-06T03:16:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> v2:172.21.15.195:3300/0 conn(0x7f5ea406d000 0x7f5ea4159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e90006f20 0x7f5e900073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5e900069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea942e640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e90005b90 0x7f5e90005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 -- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 msgr2=0x7f5e900069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5e900069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 -- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e90005b90 msgr2=0x7f5e90005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e90005b90 0x7f5e90005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea9c2f640 1 -- 172.21.15.157:0/157228125 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90005740 con 0x7f5e90006f20 2023-11-06T03:16:40.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea942e640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e90005b90 0x7f5e90005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:40.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.215+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea40b2af0 0x7f5e900069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e90006f20 0x7f5e900073a0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f5ea406d430 tx=0x7f5e9c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da85077b3d717f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0a5dc0 con 0x7f5e90006f20 2023-11-06T03:16:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0ad440 con 0x7f5e90006f20 2023-11-06T03:16:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5e9bfff640 1 -- 172.21.15.157:0/157228125 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0ad7b0 con 0x7f5e90006f20 2023-11-06T03:16:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- 172.21.15.157:0/157228125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e90006f20 msgr2=0x7f5e900073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 --2- 172.21.15.157:0/157228125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e90006f20 0x7f5e900073a0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f5ea406d430 tx=0x7f5e9c01cec0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5ea9c2f640 1 -- 172.21.15.157:0/157228125 reap_dead start 2023-11-06T03:16:40.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- 172.21.15.157:0/157228125 shutdown_connections 2023-11-06T03:16:40.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- 172.21.15.157:0/157228125 >> 172.21.15.157:0/157228125 conn(0x7f5ea4065930 msgr2=0x7f5ea4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- 172.21.15.157:0/157228125 shutdown_connections 2023-11-06T03:16:40.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- 172.21.15.157:0/157228125 wait complete. 2023-11-06T03:16:40.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 Processor -- start 2023-11-06T03:16:40.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- start start 2023-11-06T03:16:40.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ea4152490 0x7f5ea4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 0x7f5ea415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41f22d0 con 0x7f5ea4151df0 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4154650 con 0x7f5ea4152490 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5eab692640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4153b30 con 0x7f5ea4155020 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5ea942e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5ea942e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38638/0 (socket says 172.21.15.157:38638) 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.219+0000 7f5ea942e640 1 -- 172.21.15.157:0/1154348299 learned_addr learned my addr 172.21.15.157:0/1154348299 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:40.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 0x7f5ea415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 -- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 msgr2=0x7f5ea415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 0x7f5ea415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ea4152490 0x7f5ea4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 -- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ea4152490 msgr2=0x7f5ea4154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ea4152490 0x7f5ea4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 -- 172.21.15.157:0/1154348299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40cc110 con 0x7f5ea4151df0 2023-11-06T03:16:40.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea9c2f640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 0x7f5ea415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:40.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea942e640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000eb20 tx=0x7f5ea0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000c8d0 con 0x7f5ea4151df0 2023-11-06T03:16:40.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000ca30 con 0x7f5ea4151df0 2023-11-06T03:16:40.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea40cc440 con 0x7f5ea4151df0 2023-11-06T03:16:40.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000cdc0 con 0x7f5ea4151df0 2023-11-06T03:16:40.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea40cca00 con 0x7f5ea4151df0 2023-11-06T03:16:40.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea40ccbf0 con 0x7f5ea4151df0 2023-11-06T03:16:40.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ea001d460 con 0x7f5ea4151df0 2023-11-06T03:16:40.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 0x7f5e88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5ea0082ef0 con 0x7f5ea4151df0 2023-11-06T03:16:40.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 0x7f5e88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.223+0000 7f5ea8c2d640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 0x7f5e88067f20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40cee20 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.227+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5ea0050b30 con 0x7f5ea4151df0 2023-11-06T03:16:40.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.427+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5ea4072750 con 0x7f5ea4151df0 2023-11-06T03:16:40.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5e99ffb640 1 -- 172.21.15.157:0/1154348299 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5ea0054150 con 0x7f5ea4151df0 2023-11-06T03:16:40.427 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:40.427 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:40.428 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:16:40.428 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.428 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:16:40.428 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:16:40.429 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.429 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.429 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:16:40.429 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:16:40.429 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.430 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.430 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:16:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:16:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.431 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.432 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:16:40.432 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:16:40.432 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.432 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.432 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.433 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.433 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:16:40.433 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:16:40.433 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.433 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:16:40.434 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:16:40.434 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.434 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.434 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:16:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:16:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.435 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:16:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:16:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.436 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.437 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.438 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:16:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:16:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.439 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:16:40.440 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:16:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.441 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:16:40.442 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:16:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.443 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:16:40.444 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.445 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:16:40.446 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:16:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.447 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:16:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:16:40.448 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.450 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.450 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:16:40.450 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:16:40.450 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.451 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:16:40.451 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:16:40.451 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.451 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.451 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.452 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.452 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:16:40.452 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:16:40.452 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.452 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.453 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:16:40.453 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:16:40.453 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.453 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.453 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.454 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:16:40.454 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:16:40.454 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.454 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.454 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.455 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:16:40.456 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:16:40.456 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.456 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.456 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:16:40.456 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:16:40.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:16:40.457 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.457 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.457 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.457 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:16:40.458 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:16:40.458 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:16:40.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:16:40.458 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:16:40.459 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:16:40.460 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.460 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:16:40.460 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:16:40.460 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:16:40.460 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.461 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.461 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.461 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.461 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:16:40.461 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:16:40.462 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.462 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:16:40.462 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:16:40.462 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.462 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.463 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:40.463 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:40.463 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:16:40.463 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:16:40.464 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:16:40.464 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:16:40.464 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:16:40.464 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:16:40.464 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.465 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:40.465 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:16:40.465 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 msgr2=0x7f5e88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 0x7f5e88067f20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40cee20 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2023-11-06T03:16:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 msgr2=0x7f5ea40cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000eb20 tx=0x7f5ea0001640 comp rx=0 tx=0).stop 2023-11-06T03:16:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 shutdown_connections 2023-11-06T03:16:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e88065a60 0x7f5e88067f20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5ea4155020 0x7f5ea415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ea4152490 0x7f5ea4154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 --2- 172.21.15.157:0/1154348299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ea4151df0 0x7f5ea40cba10 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 >> 172.21.15.157:0/1154348299 conn(0x7f5ea4065930 msgr2=0x7f5ea406fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 shutdown_connections 2023-11-06T03:16:40.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.431+0000 7f5eab692640 1 -- 172.21.15.157:0/1154348299 wait complete. 2023-11-06T03:16:40.470 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:40.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.583+0000 7fa131980640 1 Processor -- start 2023-11-06T03:16:40.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.583+0000 7fa131980640 1 -- start start 2023-11-06T03:16:40.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa131980640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0adae0 0x7fa12c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa131980640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa12c0adf40 0x7fa12c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa131980640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c06c490 con 0x7fa12c156a70 2023-11-06T03:16:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa131980640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c06b950 con 0x7fa12c0adf40 2023-11-06T03:16:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa131980640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c06fb80 con 0x7fa12c0adae0 2023-11-06T03:16:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12affd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa12c156a70 0x7fa12c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45016/0 (socket says 172.21.15.157:45016) 2023-11-06T03:16:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12affd640 1 -- 172.21.15.157:0/2600568284 learned_addr learned my addr 172.21.15.157:0/2600568284 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:40.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa123fff640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0adae0 0x7fa12c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/2600568284 >> v2:172.21.15.195:3300/0 conn(0x7fa12c0adf40 0x7fa12c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922932229 0 0) 0x7fa12c06c490 con 0x7fa12c156a70 2023-11-06T03:16:40.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa118003530 con 0x7fa12c156a70 2023-11-06T03:16:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540911955 0 0) 0x7fa118003530 con 0x7fa12c156a70 2023-11-06T03:16:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0adae0 msgr2=0x7fa12c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0adae0 0x7fa12c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 >> v2:172.21.15.195:3300/0 conn(0x7fa12c0adf40 msgr2=0x7fa12c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 --2- 172.21.15.157:0/2600568284 >> v2:172.21.15.195:3300/0 conn(0x7fa12c0adf40 0x7fa12c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c06bb60 con 0x7fa12c156a70 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa11c001680 con 0x7fa12c156a70 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 >> v1:172.21.15.157:6789/0 conn(0x7fa12c156a70 legacy=0x7fa12c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa118003980 0x7fa118003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004370 0x7fa118004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004f70 0x7fa11800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c06b950 con 0x7fa118004f70 2023-11-06T03:16:40.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c06fb80 con 0x7fa118004370 2023-11-06T03:16:40.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11800bf00 con 0x7fa118003980 2023-11-06T03:16:40.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa123fff640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0adae0 0x7fa12c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004f70 0x7fa11800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12affd640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa118003980 0x7fa118003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 -- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa118003980 msgr2=0x7fa118003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa118003980 0x7fa118003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 -- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004370 msgr2=0x7fa118004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa123fff640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004370 0x7fa118004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004370 0x7fa118004850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.587+0000 7fa12b7fe640 1 -- 172.21.15.157:0/2600568284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118003530 con 0x7fa118004f70 2023-11-06T03:16:40.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004f70 0x7fa11800b990 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa124009fc0 tx=0x7fa12401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186d44b2f9bdcf73 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1240a5cb0 con 0x7fa118004f70 2023-11-06T03:16:40.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12400f040 con 0x7fa118004f70 2023-11-06T03:16:40.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12a7fc640 1 -- 172.21.15.157:0/2600568284 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1240ad680 con 0x7fa118004f70 2023-11-06T03:16:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- 172.21.15.157:0/2600568284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004f70 msgr2=0x7fa11800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 --2- 172.21.15.157:0/2600568284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa118004f70 0x7fa11800b990 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa124009fc0 tx=0x7fa12401cca0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12b7fe640 1 -- 172.21.15.157:0/2600568284 reap_dead start 2023-11-06T03:16:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- 172.21.15.157:0/2600568284 shutdown_connections 2023-11-06T03:16:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- 172.21.15.157:0/2600568284 >> 172.21.15.157:0/2600568284 conn(0x7fa12c065930 msgr2=0x7fa12c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- 172.21.15.157:0/2600568284 shutdown_connections 2023-11-06T03:16:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- 172.21.15.157:0/2600568284 wait complete. 2023-11-06T03:16:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 Processor -- start 2023-11-06T03:16:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- start start 2023-11-06T03:16:40.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004f70 0x7fa12c145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0ac8b0 0x7fa12c0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0ad270 con 0x7fa12c15bd30 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c149510 con 0x7fa118004f70 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa131980640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1497f0 con 0x7fa12c0ac8b0 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa123fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa123fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38654/0 (socket says 172.21.15.157:38654) 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa123fff640 1 -- 172.21.15.157:0/3362855137 learned_addr learned my addr 172.21.15.157:0/3362855137 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:40.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.591+0000 7fa12b7fe640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0ac8b0 0x7fa12c0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 -- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0ac8b0 msgr2=0x7fa12c0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0ac8b0 0x7fa12c0acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 -- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004f70 msgr2=0x7fa12c145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa12affd640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004f70 0x7fa12c145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004f70 0x7fa12c145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 -- 172.21.15.157:0/3362855137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c14a510 con 0x7fa12c15bd30 2023-11-06T03:16:40.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa123fff640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa11400f490 tx=0x7fa11400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114014890 con 0x7fa12c15bd30 2023-11-06T03:16:40.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11400c030 con 0x7fa12c15bd30 2023-11-06T03:16:40.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c14a7f0 con 0x7fa12c15bd30 2023-11-06T03:16:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114014c10 con 0x7fa12c15bd30 2023-11-06T03:16:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c145950 con 0x7fa12c15bd30 2023-11-06T03:16:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c072750 con 0x7fa12c15bd30 2023-11-06T03:16:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa11401d400 con 0x7fa12c15bd30 2023-11-06T03:16:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 0x7fa10c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa1140834a0 con 0x7fa12c15bd30 2023-11-06T03:16:40.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa12affd640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 0x7fa10c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.595+0000 7fa12affd640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 0x7fa10c067f50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f70 tx=0x7fa11c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.599+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa1140510e0 con 0x7fa12c15bd30 2023-11-06T03:16:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.759+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa12c0cef80 con 0x7fa12c15bd30 2023-11-06T03:16:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.759+0000 7fa128ff9640 1 -- 172.21.15.157:0/3362855137 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+14229 (secure 0 0 0) 0x7fa114009340 con 0x7fa12c15bd30 2023-11-06T03:16:40.757 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:40.757 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":159,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:40.051394+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 msgr2=0x7fa10c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 0x7fa10c067f50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000f70 tx=0x7fa11c009040 comp rx=0 tx=0).stop 2023-11-06T03:16:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 msgr2=0x7fa12c148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa11400f490 tx=0x7fa11400f950 comp rx=0 tx=0).stop 2023-11-06T03:16:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 shutdown_connections 2023-11-06T03:16:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa10c065a90 0x7fa10c067f50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa12c0ac8b0 0x7fa12c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa12c15bd30 0x7fa12c148cc0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 --2- 172.21.15.157:0/3362855137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa118004f70 0x7fa12c145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 >> 172.21.15.157:0/3362855137 conn(0x7fa12c065930 msgr2=0x7fa12c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 shutdown_connections 2023-11-06T03:16:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.763+0000 7fa131980640 1 -- 172.21.15.157:0/3362855137 wait complete. 2023-11-06T03:16:40.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:16:40.777 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:16:40.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 Processor -- start 2023-11-06T03:16:40.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 -- start start 2023-11-06T03:16:40.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 0x7f512006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f51200b3b20 0x7f512015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512015f580 con 0x7f512015e5e0 2023-11-06T03:16:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512015efc0 con 0x7f51200b3b20 2023-11-06T03:16:40.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f5125127640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512015f2a0 con 0x7f51201551e0 2023-11-06T03:16:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f512015e5e0 0x7f51200b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45018/0 (socket says 172.21.15.157:45018) 2023-11-06T03:16:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511e59c640 1 -- 172.21.15.157:0/3070489100 learned_addr learned my addr 172.21.15.157:0/3070489100 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756788943 0 0) 0x7f512015f580 con 0x7f512015e5e0 2023-11-06T03:16:40.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 0x7f512006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511f59e640 1 --2- 172.21.15.157:0/3070489100 >> v2:172.21.15.195:3300/0 conn(0x7f51200b3b20 0x7f512015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5108003530 con 0x7f512015e5e0 2023-11-06T03:16:40.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3256139119 0 0) 0x7f5108003530 con 0x7f512015e5e0 2023-11-06T03:16:40.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 msgr2=0x7f512006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 0x7f512006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 >> v2:172.21.15.195:3300/0 conn(0x7f51200b3b20 msgr2=0x7f512015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 --2- 172.21.15.157:0/3070489100 >> v2:172.21.15.195:3300/0 conn(0x7f51200b3b20 0x7f512015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512006f5a0 con 0x7f512015e5e0 2023-11-06T03:16:40.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5114001680 con 0x7f512015e5e0 2023-11-06T03:16:40.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 >> v1:172.21.15.157:6789/0 conn(0x7f512015e5e0 legacy=0x7f51200b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5108003980 0x7f5108003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51080047d0 0x7f5108004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 0x7f5108005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512015efc0 con 0x7f5108003980 2023-11-06T03:16:40.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512015f2a0 con 0x7f51080047d0 2023-11-06T03:16:40.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510800b990 con 0x7f51201551e0 2023-11-06T03:16:40.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5108003980 0x7f5108003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511e59c640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51080047d0 0x7f5108004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 -- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 msgr2=0x7f5108005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201551e0 0x7f5108005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 -- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51080047d0 msgr2=0x7f5108004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:40.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51080047d0 0x7f5108004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511ed9d640 1 -- 172.21.15.157:0/3070489100 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108003530 con 0x7f5108003980 2023-11-06T03:16:40.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.895+0000 7f511e59c640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51080047d0 0x7f5108004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:40.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5108003980 0x7f5108003da0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f5110009b10 tx=0x7f511001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5cadd0bdc0097d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100a6470 con 0x7f5108003980 2023-11-06T03:16:40.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51100a65d0 con 0x7f5108003980 2023-11-06T03:16:40.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f511dd9b640 1 -- 172.21.15.157:0/3070489100 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100a6980 con 0x7f5108003980 2023-11-06T03:16:40.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- 172.21.15.157:0/3070489100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5108003980 msgr2=0x7f5108003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 --2- 172.21.15.157:0/3070489100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5108003980 0x7f5108003da0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f5110009b10 tx=0x7f511001ca60 comp rx=0 tx=0).stop 2023-11-06T03:16:40.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f511f59e640 1 -- 172.21.15.157:0/3070489100 reap_dead start 2023-11-06T03:16:40.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- 172.21.15.157:0/3070489100 shutdown_connections 2023-11-06T03:16:40.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- 172.21.15.157:0/3070489100 >> 172.21.15.157:0/3070489100 conn(0x7f5120065930 msgr2=0x7f5120065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:40.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- 172.21.15.157:0/3070489100 shutdown_connections 2023-11-06T03:16:40.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- 172.21.15.157:0/3070489100 wait complete. 2023-11-06T03:16:40.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 Processor -- start 2023-11-06T03:16:40.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 -- start start 2023-11-06T03:16:40.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.899+0000 7f5125127640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 0x7f5120150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201f9340 0x7f51201f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51201529f0 con 0x7f51201f5d90 2023-11-06T03:16:40.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512014d980 con 0x7f512014ae90 2023-11-06T03:16:40.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5120138b90 con 0x7f51201f9340 2023-11-06T03:16:40.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38674/0 (socket says 172.21.15.157:38674) 2023-11-06T03:16:40.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 -- 172.21.15.157:0/3462071012 learned_addr learned my addr 172.21.15.157:0/3462071012 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:40.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 0x7f5120150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511f59e640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201f9340 0x7f51201f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 -- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201f9340 msgr2=0x7f51201f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201f9340 0x7f51201f97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 -- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 msgr2=0x7f5120150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 0x7f5120150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:40.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 -- 172.21.15.157:0/3462071012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51201f6750 con 0x7f51201f5d90 2023-11-06T03:16:40.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 0x7f5120150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:40.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511e59c640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f5114000fc0 tx=0x7f511400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114004cc0 con 0x7f51201f5d90 2023-11-06T03:16:40.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5114004e20 con 0x7f51201f5d90 2023-11-06T03:16:40.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512014b270 con 0x7f51201f5d90 2023-11-06T03:16:40.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140072f0 con 0x7f51201f5d90 2023-11-06T03:16:40.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512014dbd0 con 0x7f51201f5d90 2023-11-06T03:16:40.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5120152750 con 0x7f51201f5d90 2023-11-06T03:16:40.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5114007450 con 0x7f51201f5d90 2023-11-06T03:16:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 0x7f510c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5114082c50 con 0x7f51201f5d90 2023-11-06T03:16:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 0x7f510c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.903+0000 7f511ed9d640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 0x7f510c067f20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f51100ad000 tx=0x7f51100a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:40.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:40.907+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5120152750 con 0x7f51201f5d90 2023-11-06T03:16:41.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5120152b30 con 0x7f51201f5d90 2023-11-06T03:16:41.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f51077fe640 1 -- 172.21.15.157:0/3462071012 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5120152b30 con 0x7f51201f5d90 2023-11-06T03:16:41.057 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:16:41.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 msgr2=0x7f510c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 0x7f510c067f20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f51100ad000 tx=0x7f51100a9040 comp rx=0 tx=0).stop 2023-11-06T03:16:41.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 msgr2=0x7f51201f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.059+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f5114000fc0 tx=0x7f511400dd20 comp rx=0 tx=0).stop 2023-11-06T03:16:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 shutdown_connections 2023-11-06T03:16:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f510c065a60 0x7f510c067f20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51201f9340 0x7f51201f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51201f5d90 0x7f51201f6210 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 --2- 172.21.15.157:0/3462071012 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f512014ae90 0x7f5120150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 >> 172.21.15.157:0/3462071012 conn(0x7f5120065930 msgr2=0x7f512014ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 shutdown_connections 2023-11-06T03:16:41.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.063+0000 7f5125127640 1 -- 172.21.15.157:0/3462071012 wait complete. 2023-11-06T03:16:41.075 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:16:41.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 Processor -- start 2023-11-06T03:16:41.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 -- start start 2023-11-06T03:16:41.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbf815caa0 0x7fbbf80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 0x7fbbf80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80d2920 con 0x7fbbf80ce1a0 2023-11-06T03:16:41.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80d18a0 con 0x7fbbf815caa0 2023-11-06T03:16:41.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbfe080640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80d1b60 con 0x7fbbf815ce80 2023-11-06T03:16:41.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf77fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbbf815caa0 0x7fbbf80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbbf80ce1a0 0x7fbbf80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45020/0 (socket says 172.21.15.157:45020) 2023-11-06T03:16:41.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf7fff640 1 -- 172.21.15.157:0/1386394196 learned_addr learned my addr 172.21.15.157:0/1386394196 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:41.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf77fe640 1 -- 172.21.15.157:0/1386394196 >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 msgr2=0x7fbbf80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf6ffd640 1 --2- 172.21.15.157:0/1386394196 >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 0x7fbbf80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf67fc640 1 -- 172.21.15.157:0/1386394196 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582069636 0 0) 0x7fbbf80d2920 con 0x7fbbf80ce1a0 2023-11-06T03:16:41.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf77fe640 1 --2- 172.21.15.157:0/1386394196 >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 0x7fbbf80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf77fe640 1 -- 172.21.15.157:0/1386394196 >> v1:172.21.15.157:6789/0 conn(0x7fbbf80ce1a0 legacy=0x7fbbf80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf77fe640 1 -- 172.21.15.157:0/1386394196 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80d1e40 con 0x7fbbf815caa0 2023-11-06T03:16:41.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.203+0000 7fbbf6ffd640 1 --2- 172.21.15.157:0/1386394196 >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 0x7fbbf80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:41.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf77fe640 1 --2- 172.21.15.157:0/1386394196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cdc60 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80c6720 tx=0x7fbbe801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203d1f8518cccfd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf67fc640 1 -- 172.21.15.157:0/1386394196 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80ab600 con 0x7fbbf815caa0 2023-11-06T03:16:41.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf67fc640 1 -- 172.21.15.157:0/1386394196 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe80a7050 con 0x7fbbf815caa0 2023-11-06T03:16:41.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf67fc640 1 -- 172.21.15.157:0/1386394196 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80ab980 con 0x7fbbf815caa0 2023-11-06T03:16:41.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- 172.21.15.157:0/1386394196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 msgr2=0x7fbbf80cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- 172.21.15.157:0/1386394196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cdc60 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80c6720 tx=0x7fbbe801c430 comp rx=0 tx=0).stop 2023-11-06T03:16:41.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- 172.21.15.157:0/1386394196 shutdown_connections 2023-11-06T03:16:41.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- 172.21.15.157:0/1386394196 >> v2:172.21.15.195:3300/0 conn(0x7fbbf815ce80 0x7fbbf80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- 172.21.15.157:0/1386394196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cdc60 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- 172.21.15.157:0/1386394196 >> 172.21.15.157:0/1386394196 conn(0x7fbbf8070790 msgr2=0x7fbbf8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- 172.21.15.157:0/1386394196 shutdown_connections 2023-11-06T03:16:41.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- 172.21.15.157:0/1386394196 wait complete. 2023-11-06T03:16:41.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 Processor -- start 2023-11-06T03:16:41.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- start start 2023-11-06T03:16:41.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf80ce1a0 0x7fbbf8150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 0x7fbbf81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8154e60 con 0x7fbbf815caa0 2023-11-06T03:16:41.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8155140 con 0x7fbbf815ce80 2023-11-06T03:16:41.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbfe080640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8155420 con 0x7fbbf80ce1a0 2023-11-06T03:16:41.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38706/0 (socket says 172.21.15.157:38706) 2023-11-06T03:16:41.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf6ffd640 1 -- 172.21.15.157:0/4235439761 learned_addr learned my addr 172.21.15.157:0/4235439761 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:41.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.207+0000 7fbbf77fe640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf80ce1a0 0x7fbbf8150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf7fff640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 0x7fbbf81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 -- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf80ce1a0 msgr2=0x7fbbf8150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf80ce1a0 0x7fbbf8150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 -- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 msgr2=0x7fbbf81568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 0x7fbbf81568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 -- 172.21.15.157:0/4235439761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe8009420 con 0x7fbbf815caa0 2023-11-06T03:16:41.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf7fff640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 0x7fbbf81568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:41.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf6ffd640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c7f0 tx=0x7fbbec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00cea0 con 0x7fbbf815caa0 2023-11-06T03:16:41.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec007a30 con 0x7fbbf815caa0 2023-11-06T03:16:41.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf80ca9c0 con 0x7fbbf815caa0 2023-11-06T03:16:41.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec007dc0 con 0x7fbbf815caa0 2023-11-06T03:16:41.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf80caf80 con 0x7fbbf815caa0 2023-11-06T03:16:41.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf80ca820 con 0x7fbbf815caa0 2023-11-06T03:16:41.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbbec0113f0 con 0x7fbbf815caa0 2023-11-06T03:16:41.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 0x7fbbe4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbbec083140 con 0x7fbbf815caa0 2023-11-06T03:16:41.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf77fe640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 0x7fbbe4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.211+0000 7fbbf77fe640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 0x7fbbe4067f20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80cf670 tx=0x7fbbe80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.215+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbbf80ca820 con 0x7fbbf815caa0 2023-11-06T03:16:41.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbbf80ca820 con 0x7fbbf815caa0 2023-11-06T03:16:41.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbdffff640 1 -- 172.21.15.157:0/4235439761 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v159) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbf80ca820 con 0x7fbbf815caa0 2023-11-06T03:16:41.368 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:16:41.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 msgr2=0x7fbbe4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 0x7fbbe4067f20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80cf670 tx=0x7fbbe80a6000 comp rx=0 tx=0).stop 2023-11-06T03:16:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 msgr2=0x7fbbf80cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c7f0 tx=0x7fbbec00ccb0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 shutdown_connections 2023-11-06T03:16:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbbe4065a60 0x7fbbe4067f20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbbf815ce80 0x7fbbf81568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf815caa0 0x7fbbf80cc120 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 --2- 172.21.15.157:0/4235439761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbbf80ce1a0 0x7fbbf8150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 >> 172.21.15.157:0/4235439761 conn(0x7fbbf8070790 msgr2=0x7fbbf8070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 shutdown_connections 2023-11-06T03:16:41.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.371+0000 7fbbfe080640 1 -- 172.21.15.157:0/4235439761 wait complete. 2023-11-06T03:16:41.387 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:16:41.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 Processor -- start 2023-11-06T03:16:41.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 -- start start 2023-11-06T03:16:41.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bd806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd815f580 con 0x7f2bd815e5e0 2023-11-06T03:16:41.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd815efc0 con 0x7f2bd80b3b20 2023-11-06T03:16:41.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bddd62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd815f2a0 con 0x7f2bd81551e0 2023-11-06T03:16:41.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bcffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2bd815e5e0 0x7f2bd80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45024/0 (socket says 172.21.15.157:45024) 2023-11-06T03:16:41.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bcffff640 1 -- 172.21.15.157:0/14314631 learned_addr learned my addr 172.21.15.157:0/14314631 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:41.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bd806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473618157 0 0) 0x7f2bd815f580 con 0x7f2bd815e5e0 2023-11-06T03:16:41.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc4003530 con 0x7f2bd815e5e0 2023-11-06T03:16:41.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017016600 0 0) 0x7f2bc4003530 con 0x7f2bd815e5e0 2023-11-06T03:16:41.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 msgr2=0x7f2bd806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bd806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 >> v1:172.21.15.195:6789/0 conn(0x7f2bd80b3b20 legacy=0x7f2bd815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd806f5a0 con 0x7f2bd815e5e0 2023-11-06T03:16:41.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2bc0001680 con 0x7f2bd815e5e0 2023-11-06T03:16:41.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 >> v1:172.21.15.157:6789/0 conn(0x7f2bd815e5e0 legacy=0x7f2bd80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bc40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bc4004d10 0x7f2bc4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd815f2a0 con 0x7f2bc40039b0 2023-11-06T03:16:41.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc400bac0 con 0x7f2bc4004d10 2023-11-06T03:16:41.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc400bc80 con 0x7f2bd81551e0 2023-11-06T03:16:41.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.507+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bcffff640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bc40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd7fff640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bc4004d10 0x7f2bc4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 -- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 msgr2=0x7f2bc40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd81551e0 0x7f2bc40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 -- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bc4004d10 msgr2=0x7f2bc4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bc4004d10 0x7f2bc4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 -- 172.21.15.157:0/14314631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4003530 con 0x7f2bc40039b0 2023-11-06T03:16:41.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bc4003dd0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009ae0 tx=0x7f2bc801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd313cfee298518 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80a6470 con 0x7f2bc40039b0 2023-11-06T03:16:41.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc80a65d0 con 0x7f2bc40039b0 2023-11-06T03:16:41.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd6ffd640 1 -- 172.21.15.157:0/14314631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80a6980 con 0x7f2bc40039b0 2023-11-06T03:16:41.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 -- 172.21.15.157:0/14314631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 msgr2=0x7f2bc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 --2- 172.21.15.157:0/14314631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bc4003dd0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009ae0 tx=0x7f2bc801c980 comp rx=0 tx=0).stop 2023-11-06T03:16:41.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bd7fff640 1 -- 172.21.15.157:0/14314631 reap_dead start 2023-11-06T03:16:41.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 -- 172.21.15.157:0/14314631 shutdown_connections 2023-11-06T03:16:41.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 -- 172.21.15.157:0/14314631 >> 172.21.15.157:0/14314631 conn(0x7f2bd8065930 msgr2=0x7f2bd8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 -- 172.21.15.157:0/14314631 shutdown_connections 2023-11-06T03:16:41.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.511+0000 7f2bddd62640 1 -- 172.21.15.157:0/14314631 wait complete. 2023-11-06T03:16:41.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 Processor -- start 2023-11-06T03:16:41.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- start start 2023-11-06T03:16:41.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd8147ed0 0x7f2bd8147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80b16c0 0x7f2bd80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8165c30 con 0x7f2bc40039b0 2023-11-06T03:16:41.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b2080 con 0x7f2bd8147ed0 2023-11-06T03:16:41.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd8166120 con 0x7f2bd80b16c0 2023-11-06T03:16:41.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bcffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd8147ed0 0x7f2bd8147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd7fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80b16c0 0x7f2bd80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38732/0 (socket says 172.21.15.157:38732) 2023-11-06T03:16:41.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 -- 172.21.15.157:0/3319985724 learned_addr learned my addr 172.21.15.157:0/3319985724 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:41.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 -- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80b16c0 msgr2=0x7f2bd80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80b16c0 0x7f2bd80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 -- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd8147ed0 msgr2=0x7f2bd8147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd8147ed0 0x7f2bd8147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 -- 172.21.15.157:0/3319985724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8150aa0 con 0x7f2bc40039b0 2023-11-06T03:16:41.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd77fe640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009ae0 tx=0x7f2bc4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800f000 con 0x7f2bc40039b0 2023-11-06T03:16:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8004020 con 0x7f2bc40039b0 2023-11-06T03:16:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd8147cb0 con 0x7f2bc40039b0 2023-11-06T03:16:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80043f0 con 0x7f2bc40039b0 2023-11-06T03:16:41.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd8150d40 con 0x7f2bc40039b0 2023-11-06T03:16:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd8072750 con 0x7f2bc40039b0 2023-11-06T03:16:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2bc8003620 con 0x7f2bc40039b0 2023-11-06T03:16:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 0x7f2bb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.515+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2bc811d080 con 0x7f2bc40039b0 2023-11-06T03:16:41.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.519+0000 7f2bcffff640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 0x7f2bb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.519+0000 7f2bcffff640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 0x7f2bb8067f50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001ca0 tx=0x7f2bc0007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.519+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2bc80ead70 con 0x7f2bc40039b0 2023-11-06T03:16:41.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2bd8071730 con 0x7f2bc40039b0 2023-11-06T03:16:41.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bd4ff9640 1 -- 172.21.15.157:0/3319985724 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2bc80ee390 con 0x7f2bc40039b0 2023-11-06T03:16:41.664 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:41.665 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:16:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:16:41.665 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:16:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:16:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:16:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:16:41.666 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:41.667 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:16:41.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 msgr2=0x7f2bb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 0x7f2bb8067f50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001ca0 tx=0x7f2bc0007040 comp rx=0 tx=0).stop 2023-11-06T03:16:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 msgr2=0x7f2bd8149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009ae0 tx=0x7f2bc4005800 comp rx=0 tx=0).stop 2023-11-06T03:16:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 shutdown_connections 2023-11-06T03:16:41.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2bb8065a90 0x7f2bb8067f50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bd80b16c0 0x7f2bd80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bd8147ed0 0x7f2bd8147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 --2- 172.21.15.157:0/3319985724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc40039b0 0x7f2bd8149f90 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 >> 172.21.15.157:0/3319985724 conn(0x7f2bd8065930 msgr2=0x7f2bd814f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 shutdown_connections 2023-11-06T03:16:41.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.667+0000 7f2bddd62640 1 -- 172.21.15.157:0/3319985724 wait complete. 2023-11-06T03:16:41.681 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:16:41.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:16:41.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.799+0000 7f7fe4611640 1 Processor -- start 2023-11-06T03:16:41.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.799+0000 7f7fe4611640 1 -- start start 2023-11-06T03:16:41.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe4611640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fdc0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe4611640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc15ae80 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe4611640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0cb370 con 0x7f7fdc0cebc0 2023-11-06T03:16:41.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe4611640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc06be20 con 0x7f7fdc0ce780 2023-11-06T03:16:41.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe23ad640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7fdc15aaa0 0x7f7fdc06c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45032/0 (socket says 172.21.15.157:45032) 2023-11-06T03:16:41.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe23ad640 1 -- 172.21.15.157:0/2914445210 learned_addr learned my addr 172.21.15.157:0/2914445210 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:41.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637178485 0 0) 0x7f7fdc15ae80 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc8003530 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fdc0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2715051546 0 0) 0x7f7fc8003530 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 msgr2=0x7f7fdc0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fdc0cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 >> v1:172.21.15.195:6789/0 conn(0x7f7fdc0cebc0 legacy=0x7f7fdc0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0cb650 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7fd8001680 con 0x7f7fdc15aaa0 2023-11-06T03:16:41.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 >> v1:172.21.15.157:6789/0 conn(0x7f7fdc15aaa0 legacy=0x7f7fdc06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fc80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8003f30 0x7f7fc80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8004d10 0x7f7fc8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc06be20 con 0x7f7fc8004d10 2023-11-06T03:16:41.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800ba70 con 0x7f7fc8003f30 2023-11-06T03:16:41.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800bc30 con 0x7f7fdc0ce780 2023-11-06T03:16:41.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8004d10 0x7f7fc8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe23ad640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fc80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8003f30 0x7f7fc80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 -- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 msgr2=0x7f7fc80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0ce780 0x7f7fc80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 -- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8003f30 msgr2=0x7f7fc80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:41.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8003f30 0x7f7fc80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe2bae640 1 -- 172.21.15.157:0/2914445210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8003530 con 0x7f7fc8004d10 2023-11-06T03:16:41.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.803+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8003f30 0x7f7fc80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:41.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe2bae640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8004d10 0x7f7fc8005190 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4001e70 tx=0x7f7fd401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbc4595d00a3c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003b80 con 0x7f7fc8004d10 2023-11-06T03:16:41.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4003ce0 con 0x7f7fc8004d10 2023-11-06T03:16:41.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe13ab640 1 -- 172.21.15.157:0/2914445210 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40052c0 con 0x7f7fc8004d10 2023-11-06T03:16:41.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/2914445210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8004d10 msgr2=0x7f7fc8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 --2- 172.21.15.157:0/2914445210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fc8004d10 0x7f7fc8005190 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4001e70 tx=0x7f7fd401c760 comp rx=0 tx=0).stop 2023-11-06T03:16:41.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe2bae640 1 -- 172.21.15.157:0/2914445210 reap_dead start 2023-11-06T03:16:41.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/2914445210 shutdown_connections 2023-11-06T03:16:41.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/2914445210 >> 172.21.15.157:0/2914445210 conn(0x7f7fdc0b0c30 msgr2=0x7f7fdc0b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:41.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/2914445210 shutdown_connections 2023-11-06T03:16:41.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/2914445210 wait complete. 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 Processor -- start 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- start start 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8004d10 0x7f7fdc0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0b21c0 0x7f7fdc0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc070b10 con 0x7f7fdc070150 2023-11-06T03:16:41.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c83e0 con 0x7f7fc8004d10 2023-11-06T03:16:41.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c7690 con 0x7f7fdc0b21c0 2023-11-06T03:16:41.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38742/0 (socket says 172.21.15.157:38742) 2023-11-06T03:16:41.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 -- 172.21.15.157:0/1358463851 learned_addr learned my addr 172.21.15.157:0/1358463851 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:41.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 -- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0b21c0 msgr2=0x7f7fdc0b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0b21c0 0x7f7fdc0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 -- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8004d10 msgr2=0x7f7fdc0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:41.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe23ad640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8004d10 0x7f7fdc0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8004d10 0x7f7fdc0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:41.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 -- 172.21.15.157:0/1358463851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0c7970 con 0x7f7fdc070150 2023-11-06T03:16:41.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe1bac640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010810 tx=0x7f7fcc010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004220 con 0x7f7fdc070150 2023-11-06T03:16:41.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc004380 con 0x7f7fdc070150 2023-11-06T03:16:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0c7d90 con 0x7f7fdc070150 2023-11-06T03:16:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00eb80 con 0x7f7fdc070150 2023-11-06T03:16:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.807+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc0b2b80 con 0x7f7fdc070150 2023-11-06T03:16:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc0c7be0 con 0x7f7fdc070150 2023-11-06T03:16:41.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7fcc00ece0 con 0x7f7fdc070150 2023-11-06T03:16:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fd2ffd640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 0x7f7fc0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7fcc0830d0 con 0x7f7fdc070150 2023-11-06T03:16:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fe23ad640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 0x7f7fc0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.811+0000 7f7fe23ad640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 0x7f7fc0067f20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8001ec0 tx=0x7f7fd8001d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:41.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.815+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7fcc050d10 con 0x7f7fdc070150 2023-11-06T03:16:41.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:41.959+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7fdc04e050 con 0x7f7fdc070150 2023-11-06T03:16:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.059+0000 7f7fd2ffd640 1 -- 172.21.15.157:0/1358463851 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7fcc0044f0 con 0x7f7fdc070150 2023-11-06T03:16:42.056 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:16:42.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 msgr2=0x7f7fc0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 0x7f7fc0067f20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8001ec0 tx=0x7f7fd8001d50 comp rx=0 tx=0).stop 2023-11-06T03:16:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 msgr2=0x7f7fdc0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010810 tx=0x7f7fcc010cd0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 shutdown_connections 2023-11-06T03:16:42.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7fc0065a60 0x7f7fc0067f20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fdc0b21c0 0x7f7fdc0b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fdc070150 0x7f7fdc0705d0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 --2- 172.21.15.157:0/1358463851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc8004d10 0x7f7fdc0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 >> 172.21.15.157:0/1358463851 conn(0x7f7fdc0b0c30 msgr2=0x7f7fdc14ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:42.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.063+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 shutdown_connections 2023-11-06T03:16:42.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.067+0000 7f7fe4611640 1 -- 172.21.15.157:0/1358463851 wait complete. 2023-11-06T03:16:42.083 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:16:42.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 Processor -- start 2023-11-06T03:16:42.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 -- start start 2023-11-06T03:16:42.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f099c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c15e440 con 0x7f099c1599c0 2023-11-06T03:16:42.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c06c240 con 0x7f099c0b2af0 2023-11-06T03:16:42.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a3923640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c06c520 con 0x7f099c06d000 2023-11-06T03:16:42.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a16bf640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f099c1599c0 0x7f099c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45042/0 (socket says 172.21.15.157:45042) 2023-11-06T03:16:42.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a16bf640 1 -- 172.21.15.157:0/4272259222 learned_addr learned my addr 172.21.15.157:0/4272259222 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:42.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a1ec0640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f099c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260339426 0 0) 0x7f099c15e440 con 0x7f099c1599c0 2023-11-06T03:16:42.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0984003530 con 0x7f099c1599c0 2023-11-06T03:16:42.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359832394 0 0) 0x7f0984003530 con 0x7f099c1599c0 2023-11-06T03:16:42.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 msgr2=0x7f099c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:42.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f099c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 >> v1:172.21.15.195:6789/0 conn(0x7f099c0b2af0 legacy=0x7f099c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c06c800 con 0x7f099c1599c0 2023-11-06T03:16:42.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f098c001680 con 0x7f099c1599c0 2023-11-06T03:16:42.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 >> v1:172.21.15.157:6789/0 conn(0x7f099c1599c0 legacy=0x7f099c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f0984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f09840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0984004d10 0x7f0984005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c06c520 con 0x7f09840039b0 2023-11-06T03:16:42.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098400ba40 con 0x7f0984004d10 2023-11-06T03:16:42.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098400de10 con 0x7f099c06d000 2023-11-06T03:16:42.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a16bf640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f0984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a1ec0640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0984004d10 0x7f0984005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f09840047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.211+0000 7f09a16bf640 1 -- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 msgr2=0x7f09840047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:42.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a16bf640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f09840047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a16bf640 1 -- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0984004d10 msgr2=0x7f0984005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:42.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a16bf640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0984004d10 0x7f0984005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a16bf640 1 -- 172.21.15.157:0/4272259222 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984003530 con 0x7f09840039b0 2023-11-06T03:16:42.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a1ec0640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0984004d10 0x7f0984005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:42.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c06d000 0x7f09840047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:42.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a16bf640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f0984003dd0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c190 tx=0x7f098c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49082500477511e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:42.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c005ae0 con 0x7f09840039b0 2023-11-06T03:16:42.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c005c40 con 0x7f09840039b0 2023-11-06T03:16:42.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f099bfff640 1 -- 172.21.15.157:0/4272259222 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0a58d0 con 0x7f09840039b0 2023-11-06T03:16:42.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 -- 172.21.15.157:0/4272259222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 msgr2=0x7f0984003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 --2- 172.21.15.157:0/4272259222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f0984003dd0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c190 tx=0x7f098c01cbd0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a1ec0640 1 -- 172.21.15.157:0/4272259222 reap_dead start 2023-11-06T03:16:42.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 -- 172.21.15.157:0/4272259222 shutdown_connections 2023-11-06T03:16:42.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 -- 172.21.15.157:0/4272259222 >> 172.21.15.157:0/4272259222 conn(0x7f099c065930 msgr2=0x7f099c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:42.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 -- 172.21.15.157:0/4272259222 shutdown_connections 2023-11-06T03:16:42.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.215+0000 7f09a3923640 1 -- 172.21.15.157:0/4272259222 wait complete. 2023-11-06T03:16:42.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 Processor -- start 2023-11-06T03:16:42.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- start start 2023-11-06T03:16:42.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 0x7f099c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c149050 0x7f099c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c15b800 con 0x7f09840039b0 2023-11-06T03:16:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c149d40 con 0x7f099c149050 2023-11-06T03:16:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c14a020 con 0x7f099c15ae40 2023-11-06T03:16:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38752/0 (socket says 172.21.15.157:38752) 2023-11-06T03:16:42.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 -- 172.21.15.157:0/920843878 learned_addr learned my addr 172.21.15.157:0/920843878 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:42.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a1ec0640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c149050 0x7f099c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 0x7f099c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 -- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 msgr2=0x7f099c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 0x7f099c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 -- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c149050 msgr2=0x7f099c1494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:42.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c149050 0x7f099c1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:42.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 -- 172.21.15.157:0/920843878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c14a300 con 0x7f09840039b0 2023-11-06T03:16:42.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 0x7f099c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:42.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a16bf640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f098c01cc10 tx=0x7f0984005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:42.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0b4070 con 0x7f09840039b0 2023-11-06T03:16:42.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c0013c0 con 0x7f09840039b0 2023-11-06T03:16:42.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c14e240 con 0x7f09840039b0 2023-11-06T03:16:42.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c001750 con 0x7f09840039b0 2023-11-06T03:16:42.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c149a10 con 0x7f09840039b0 2023-11-06T03:16:42.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c072750 con 0x7f09840039b0 2023-11-06T03:16:42.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f098c0a58d0 con 0x7f09840039b0 2023-11-06T03:16:42.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.219+0000 7f0999ffb640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 0x7f0988067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:42.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.223+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f098c11ce40 con 0x7f09840039b0 2023-11-06T03:16:42.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.223+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 0x7f0988067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:42.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.223+0000 7f09a0ebe640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 0x7f0988067f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f099c15ba30 tx=0x7f0990009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:42.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.223+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f098c0ea960 con 0x7f09840039b0 2023-11-06T03:16:42.365 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:16:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:42.367+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f099c0cb110 con 0x7f09840039b0 2023-11-06T03:16:42.368 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:16:43.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.067+0000 7f0999ffb640 1 -- 172.21.15.157:0/920843878 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7f098c0edf80 con 0x7f09840039b0 2023-11-06T03:16:43.070 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:16:43.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.083+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 msgr2=0x7f0988067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.083+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 0x7f0988067f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f099c15ba30 tx=0x7f0990009040 comp rx=0 tx=0).stop 2023-11-06T03:16:43.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.083+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 msgr2=0x7f099c1506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.083+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f098c01cc10 tx=0x7f0984005800 comp rx=0 tx=0).stop 2023-11-06T03:16:43.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 shutdown_connections 2023-11-06T03:16:43.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0988065a90 0x7f0988067f50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c149050 0x7f099c1494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c15ae40 0x7f099c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 --2- 172.21.15.157:0/920843878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09840039b0 0x7f099c1506a0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 >> 172.21.15.157:0/920843878 conn(0x7f099c065930 msgr2=0x7f099c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 shutdown_connections 2023-11-06T03:16:43.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.087+0000 7f09a3923640 1 -- 172.21.15.157:0/920843878 wait complete. 2023-11-06T03:16:43.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:16:43.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.231+0000 7f7d166e1640 1 Processor -- start 2023-11-06T03:16:43.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.231+0000 7f7d166e1640 1 -- start start 2023-11-06T03:16:43.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.231+0000 7f7d166e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d100ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.231+0000 7f7d166e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d100b2af0 0x7f7d10159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.231+0000 7f7d166e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1015e440 con 0x7f7d1006d000 2023-11-06T03:16:43.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1006c240 con 0x7f7d100b2af0 2023-11-06T03:16:43.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1006c520 con 0x7f7d101599c0 2023-11-06T03:16:43.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d100b2af0 0x7f7d10159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7d1006d000 0x7f7d100b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45054/0 (socket says 172.21.15.157:45054) 2023-11-06T03:16:43.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0f7fe640 1 -- 172.21.15.157:0/1646503589 learned_addr learned my addr 172.21.15.157:0/1646503589 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:43.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0ffff640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d100ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 -- 172.21.15.157:0/1646503589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 msgr2=0x7f7d100ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d100ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 -- 172.21.15.157:0/1646503589 >> v1:172.21.15.157:6789/0 conn(0x7f7d1006d000 legacy=0x7f7d100b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 -- 172.21.15.157:0/1646503589 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1006c800 con 0x7f7d100b2af0 2023-11-06T03:16:43.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0ffff640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d100ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:43.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d14c7e640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d100b2af0 0x7f7d10159da0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009d90 tx=0x7f7d0801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae175ebe483f9e55 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0effd640 1 -- 172.21.15.157:0/1646503589 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080a5a00 con 0x7f7d100b2af0 2023-11-06T03:16:43.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0effd640 1 -- 172.21.15.157:0/1646503589 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d080a5b60 con 0x7f7d100b2af0 2023-11-06T03:16:43.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d0effd640 1 -- 172.21.15.157:0/1646503589 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080ad440 con 0x7f7d100b2af0 2023-11-06T03:16:43.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- 172.21.15.157:0/1646503589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d100b2af0 msgr2=0x7f7d10159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d100b2af0 0x7f7d10159da0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009d90 tx=0x7f7d0801c920 comp rx=0 tx=0).stop 2023-11-06T03:16:43.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- 172.21.15.157:0/1646503589 shutdown_connections 2023-11-06T03:16:43.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d100b2af0 0x7f7d10159da0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 --2- 172.21.15.157:0/1646503589 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d100ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- 172.21.15.157:0/1646503589 >> 172.21.15.157:0/1646503589 conn(0x7f7d10065930 msgr2=0x7f7d10065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- 172.21.15.157:0/1646503589 shutdown_connections 2023-11-06T03:16:43.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.235+0000 7f7d166e1640 1 -- 172.21.15.157:0/1646503589 wait complete. 2023-11-06T03:16:43.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 Processor -- start 2023-11-06T03:16:43.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- start start 2023-11-06T03:16:43.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d100b2af0 0x7f7d100cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d1014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1013f0a0 con 0x7f7d1006d000 2023-11-06T03:16:43.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1013c440 con 0x7f7d100b2af0 2023-11-06T03:16:43.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1013c720 con 0x7f7d101599c0 2023-11-06T03:16:43.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38758/0 (socket says 172.21.15.157:38758) 2023-11-06T03:16:43.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 -- 172.21.15.157:0/2899989677 learned_addr learned my addr 172.21.15.157:0/2899989677 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:43.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 -- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 msgr2=0x7f7d1014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0f7fe640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d100b2af0 0x7f7d100cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d14c7e640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d1014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d1014fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 -- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d100b2af0 msgr2=0x7f7d100cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d100b2af0 0x7f7d100cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 -- 172.21.15.157:0/2899989677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009a70 con 0x7f7d1006d000 2023-11-06T03:16:43.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0ffff640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c7f0 tx=0x7f7d0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007820 con 0x7f7d1006d000 2023-11-06T03:16:43.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00007980 con 0x7f7d1006d000 2023-11-06T03:16:43.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1013ca00 con 0x7f7d1006d000 2023-11-06T03:16:43.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00007d30 con 0x7f7d1006d000 2023-11-06T03:16:43.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1013cfc0 con 0x7f7d1006d000 2023-11-06T03:16:43.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10072750 con 0x7f7d1006d000 2023-11-06T03:16:43.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.239+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d000113f0 con 0x7f7d1006d000 2023-11-06T03:16:43.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.243+0000 7f7d0cff9640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 0x7f7cfc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.243+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f7d00083570 con 0x7f7d1006d000 2023-11-06T03:16:43.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.243+0000 7f7d0f7fe640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 0x7f7cfc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.243+0000 7f7d0f7fe640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 0x7f7cfc067f20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7d1006b780 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.247+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d00050e30 con 0x7f7d1006d000 2023-11-06T03:16:43.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.431+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7d100cb110 con 0x7f7d1006d000 2023-11-06T03:16:43.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.439+0000 7f7d0cff9640 1 -- 172.21.15.157:0/2899989677 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v115) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7d00054450 con 0x7f7d1006d000 2023-11-06T03:16:43.441 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 23 and data pool 24 2023-11-06T03:16:43.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 msgr2=0x7f7cfc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 0x7f7cfc067f20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7d1006b780 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2023-11-06T03:16:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 msgr2=0x7f7d1013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c7f0 tx=0x7f7d0000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 shutdown_connections 2023-11-06T03:16:43.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cfc065a60 0x7f7cfc067f20 secure :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7d1006b780 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2023-11-06T03:16:43.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d101599c0 0x7f7d1014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d100b2af0 0x7f7d100cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 --2- 172.21.15.157:0/2899989677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d1006d000 0x7f7d1013eb60 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.447+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 >> 172.21.15.157:0/2899989677 conn(0x7f7d10065930 msgr2=0x7f7d100b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.451+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 shutdown_connections 2023-11-06T03:16:43.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.451+0000 7f7d166e1640 1 -- 172.21.15.157:0/2899989677 wait complete. 2023-11-06T03:16:43.482 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:43.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 Processor -- start 2023-11-06T03:16:43.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 -- start start 2023-11-06T03:16:43.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f84881599c0 0x7f84880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848815e440 con 0x7f84880b2af0 2023-11-06T03:16:43.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848806c240 con 0x7f848806d000 2023-11-06T03:16:43.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848efa6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848806c520 con 0x7f84881599c0 2023-11-06T03:16:43.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848d543640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f84880b2af0 0x7f8488159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45066/0 (socket says 172.21.15.157:45066) 2023-11-06T03:16:43.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848d543640 1 -- 172.21.15.157:0/1824715066 learned_addr learned my addr 172.21.15.157:0/1824715066 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:43.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848cd42640 1 --2- 172.21.15.157:0/1824715066 >> v2:172.21.15.195:3300/0 conn(0x7f84881599c0 0x7f84880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543237964 0 0) 0x7f848815e440 con 0x7f84880b2af0 2023-11-06T03:16:43.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8470003530 con 0x7f84880b2af0 2023-11-06T03:16:43.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625425584 0 0) 0x7f8470003530 con 0x7f84880b2af0 2023-11-06T03:16:43.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 >> v2:172.21.15.195:3300/0 conn(0x7f84881599c0 msgr2=0x7f84880ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 --2- 172.21.15.157:0/1824715066 >> v2:172.21.15.195:3300/0 conn(0x7f84881599c0 0x7f84880ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 >> v1:172.21.15.195:6789/0 conn(0x7f848806d000 legacy=0x7f84880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848806c800 con 0x7f84880b2af0 2023-11-06T03:16:43.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8480001a00 con 0x7f84880b2af0 2023-11-06T03:16:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 >> v1:172.21.15.157:6789/0 conn(0x7f84880b2af0 legacy=0x7f8488159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8470003980 0x7f8470003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 0x7f8470004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8470005190 0x7f847000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848806c520 con 0x7f84700047d0 2023-11-06T03:16:43.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847000c030 con 0x7f8470005190 2023-11-06T03:16:43.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847000c1f0 con 0x7f8470003980 2023-11-06T03:16:43.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 0x7f8470004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848cd42640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8470003980 0x7f8470003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 -- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8470003980 msgr2=0x7f8470003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848d543640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8470005190 0x7f847000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8470003980 0x7f8470003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 -- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8470005190 msgr2=0x7f847000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8470005190 0x7f847000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f848d543640 1 -- 172.21.15.157:0/1824715066 reap_dead start 2023-11-06T03:16:43.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.599+0000 7f8487fff640 1 -- 172.21.15.157:0/1824715066 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470003530 con 0x7f84700047d0 2023-11-06T03:16:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f8487fff640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 0x7f8470004c50 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f8478001780 tx=0x7f847801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fe44c1acb09759 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780052d0 con 0x7f84700047d0 2023-11-06T03:16:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780a7050 con 0x7f84700047d0 2023-11-06T03:16:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f84877fe640 1 -- 172.21.15.157:0/1824715066 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478003600 con 0x7f84700047d0 2023-11-06T03:16:43.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 msgr2=0x7f8470004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 0x7f8470004c50 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f8478001780 tx=0x7f847801c3e0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- 172.21.15.157:0/1824715066 shutdown_connections 2023-11-06T03:16:43.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 --2- 172.21.15.157:0/1824715066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84700047d0 0x7f8470004c50 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- 172.21.15.157:0/1824715066 >> 172.21.15.157:0/1824715066 conn(0x7f8488065930 msgr2=0x7f8488065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- 172.21.15.157:0/1824715066 shutdown_connections 2023-11-06T03:16:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- 172.21.15.157:0/1824715066 wait complete. 2023-11-06T03:16:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 Processor -- start 2023-11-06T03:16:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- start start 2023-11-06T03:16:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84700047d0 0x7f8488148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8488146020 0x7f84880acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488149090 con 0x7f8488147660 2023-11-06T03:16:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881465c0 con 0x7f8488146020 2023-11-06T03:16:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.603+0000 7f848efa6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84881468a0 con 0x7f84700047d0 2023-11-06T03:16:43.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38776/0 (socket says 172.21.15.157:38776) 2023-11-06T03:16:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 -- 172.21.15.157:0/2201130989 learned_addr learned my addr 172.21.15.157:0/2201130989 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848cd42640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84700047d0 0x7f8488148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f8487fff640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8488146020 0x7f84880acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 -- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84700047d0 msgr2=0x7f8488148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84700047d0 0x7f8488148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 -- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8488146020 msgr2=0x7f84880acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8488146020 0x7f84880acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 -- 172.21.15.157:0/2201130989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488146b80 con 0x7f8488147660 2023-11-06T03:16:43.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848d543640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f84880b3b00 tx=0x7f8480010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480010ee0 con 0x7f8488147660 2023-11-06T03:16:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848000eac0 con 0x7f8488147660 2023-11-06T03:16:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848000ee70 con 0x7f8488147660 2023-11-06T03:16:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8488146e60 con 0x7f8488147660 2023-11-06T03:16:43.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880ad2c0 con 0x7f8488147660 2023-11-06T03:16:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488072750 con 0x7f8488147660 2023-11-06T03:16:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f848001e3b0 con 0x7f8488147660 2023-11-06T03:16:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8480085e60 con 0x7f8488147660 2023-11-06T03:16:43.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.607+0000 7f848cd42640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.611+0000 7f848cd42640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f847c0099f0 tx=0x7f847c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.611+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f84800536c0 con 0x7f8488147660 2023-11-06T03:16:43.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.767+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84880adcd0 con 0x7f8488147660 2023-11-06T03:16:43.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.767+0000 7f84857fa640 1 -- 172.21.15.157:0/2201130989 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17886 (secure 0 0 0) 0x7f848001a020 con 0x7f8488147660 2023-11-06T03:16:43.766 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:43.766 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":162,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:43.438084+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 msgr2=0x7f8474067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f847c0099f0 tx=0x7f847c008040 comp rx=0 tx=0).stop 2023-11-06T03:16:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 msgr2=0x7f8488147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f84880b3b00 tx=0x7f8480010cf0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 shutdown_connections 2023-11-06T03:16:43.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8474065a90 0x7f8474067f50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8488147660 0x7f8488147ae0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8488146020 0x7f84880acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 --2- 172.21.15.157:0/2201130989 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f84700047d0 0x7f8488148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 >> 172.21.15.157:0/2201130989 conn(0x7f8488065930 msgr2=0x7f8488159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 shutdown_connections 2023-11-06T03:16:43.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.771+0000 7f848efa6640 1 -- 172.21.15.157:0/2201130989 wait complete. 2023-11-06T03:16:43.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:43.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 Processor -- start 2023-11-06T03:16:43.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 -- start start 2023-11-06T03:16:43.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f013c0d0280 0x7f013c167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c0d2920 con 0x7f013c15ce80 2023-11-06T03:16:43.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013c0d18a0 con 0x7f013c15caa0 2023-11-06T03:16:43.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0142719640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f013c0d1b60 con 0x7f013c0d0280 2023-11-06T03:16:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f01337fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f013c15ce80 0x7f013c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45074/0 (socket says 172.21.15.157:45074) 2023-11-06T03:16:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f01337fe640 1 -- 172.21.15.157:0/1672670172 learned_addr learned my addr 172.21.15.157:0/1672670172 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0140cb6640 1 --2- 172.21.15.157:0/1672670172 >> v2:172.21.15.195:3300/0 conn(0x7f013c0d0280 0x7f013c167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140823703 0 0) 0x7f013c0d2920 con 0x7f013c15ce80 2023-11-06T03:16:43.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0128003530 con 0x7f013c15ce80 2023-11-06T03:16:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113947568 0 0) 0x7f0128003530 con 0x7f013c15ce80 2023-11-06T03:16:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 >> v2:172.21.15.195:3300/0 conn(0x7f013c0d0280 msgr2=0x7f013c167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 --2- 172.21.15.157:0/1672670172 >> v2:172.21.15.195:3300/0 conn(0x7f013c0d0280 0x7f013c167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 >> v1:172.21.15.195:6789/0 conn(0x7f013c15caa0 legacy=0x7f013c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c0d1e40 con 0x7f013c15ce80 2023-11-06T03:16:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f0124001680 con 0x7f013c15ce80 2023-11-06T03:16:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 >> v1:172.21.15.157:6789/0 conn(0x7f013c15ce80 legacy=0x7f013c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 0x7f0128003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01280047d0 0x7f0128004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0128005190 0x7f012800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0d1b60 con 0x7f0128003980 2023-11-06T03:16:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012800c030 con 0x7f01280047d0 2023-11-06T03:16:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012800c1f0 con 0x7f0128005190 2023-11-06T03:16:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 0x7f0128003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f01337fe640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01280047d0 0x7f0128004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 -- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0128005190 msgr2=0x7f012800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0128005190 0x7f012800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 -- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01280047d0 msgr2=0x7f0128004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01280047d0 0x7f0128004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f013bfff640 1 -- 172.21.15.157:0/1672670172 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128003530 con 0x7f0128003980 2023-11-06T03:16:43.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.911+0000 7f0140cb6640 1 -- 172.21.15.157:0/1672670172 reap_dead start 2023-11-06T03:16:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f013bfff640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 0x7f0128003da0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f012c001780 tx=0x7f012c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67275498f1aa7148 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0052d0 con 0x7f0128003980 2023-11-06T03:16:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c005450 con 0x7f0128003980 2023-11-06T03:16:43.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f013b7fe640 1 -- 172.21.15.157:0/1672670172 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c003840 con 0x7f0128003980 2023-11-06T03:16:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 msgr2=0x7f0128003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 0x7f0128003da0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f012c001780 tx=0x7f012c01c3b0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- 172.21.15.157:0/1672670172 shutdown_connections 2023-11-06T03:16:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 --2- 172.21.15.157:0/1672670172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003980 0x7f0128003da0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- 172.21.15.157:0/1672670172 >> 172.21.15.157:0/1672670172 conn(0x7f013c070790 msgr2=0x7f013c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- 172.21.15.157:0/1672670172 shutdown_connections 2023-11-06T03:16:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- 172.21.15.157:0/1672670172 wait complete. 2023-11-06T03:16:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 Processor -- start 2023-11-06T03:16:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.915+0000 7f0142719640 1 -- start start 2023-11-06T03:16:43.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0128003980 0x7f013c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 0x7f013c158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c1595c0 con 0x7f013c159140 2023-11-06T03:16:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c150620 con 0x7f0128003980 2023-11-06T03:16:43.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c0c67c0 con 0x7f013c0c7520 2023-11-06T03:16:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38786/0 (socket says 172.21.15.157:38786) 2023-11-06T03:16:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 -- 172.21.15.157:0/3568274160 learned_addr learned my addr 172.21.15.157:0/3568274160 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01337fe640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 0x7f013c158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f013bfff640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0128003980 0x7f013c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 -- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 msgr2=0x7f013c158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 0x7f013c158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 -- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0128003980 msgr2=0x7f013c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0128003980 0x7f013c14fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:43.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 -- 172.21.15.157:0/3568274160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c0c6a60 con 0x7f013c159140 2023-11-06T03:16:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01337fe640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 0x7f013c158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0140cb6640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f013c0ceb20 tx=0x7f013400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134010040 con 0x7f013c159140 2023-11-06T03:16:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400bba0 con 0x7f013c159140 2023-11-06T03:16:43.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013c0c6d40 con 0x7f013c159140 2023-11-06T03:16:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134014400 con 0x7f013c159140 2023-11-06T03:16:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c15c540 con 0x7f013c159140 2023-11-06T03:16:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c072750 con 0x7f013c159140 2023-11-06T03:16:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0134014560 con 0x7f013c159140 2023-11-06T03:16:43.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 0x7f011c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0134083580 con 0x7f013c159140 2023-11-06T03:16:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.919+0000 7f013bfff640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 0x7f011c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:43.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.923+0000 7f013bfff640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 0x7f011c067f50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f012c001750 tx=0x7f012c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:43.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:43.923+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0134050d40 con 0x7f013c159140 2023-11-06T03:16:44.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.079+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f013c0c9ab0 con 0x7f013c159140 2023-11-06T03:16:44.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.079+0000 7f01397fa640 1 -- 172.21.15.157:0/3568274160 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17886 (secure 0 0 0) 0x7f0134054360 con 0x7f013c159140 2023-11-06T03:16:44.079 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:44.079 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":162,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:43.438084+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:44.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 msgr2=0x7f011c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 0x7f011c067f50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f012c001750 tx=0x7f012c0011a0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 msgr2=0x7f013c15b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f013c0ceb20 tx=0x7f013400ef30 comp rx=0 tx=0).stop 2023-11-06T03:16:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 shutdown_connections 2023-11-06T03:16:44.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f011c065a90 0x7f011c067f50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f013c159140 0x7f013c15b960 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f013c0c7520 0x7f013c158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 --2- 172.21.15.157:0/3568274160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0128003980 0x7f013c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 >> 172.21.15.157:0/3568274160 conn(0x7f013c070790 msgr2=0x7f013c15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 shutdown_connections 2023-11-06T03:16:44.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.083+0000 7f0142719640 1 -- 172.21.15.157:0/3568274160 wait complete. 2023-11-06T03:16:44.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:16:44.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 Processor -- start 2023-11-06T03:16:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 -- start start 2023-11-06T03:16:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 0x7f9a340cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9a340d00b0 0x7f9a340b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a340d2920 con 0x7f9a3415caa0 2023-11-06T03:16:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340d0520 con 0x7f9a340d00b0 2023-11-06T03:16:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3ba6d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d18a0 con 0x7f9a3415ce80 2023-11-06T03:16:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a39809640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f9a3415caa0 0x7f9a340cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45088/0 (socket says 172.21.15.157:45088) 2023-11-06T03:16:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a39809640 1 -- 172.21.15.157:0/536729054 learned_addr learned my addr 172.21.15.157:0/536729054 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621222765 0 0) 0x7f9a340d2920 con 0x7f9a3415caa0 2023-11-06T03:16:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a3a00a640 1 --2- 172.21.15.157:0/536729054 >> v2:172.21.15.195:3300/0 conn(0x7f9a340d00b0 0x7f9a340b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a20003530 con 0x7f9a3415caa0 2023-11-06T03:16:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a39008640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 0x7f9a340cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 548780034 0 0) 0x7f9a20003530 con 0x7f9a3415caa0 2023-11-06T03:16:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 msgr2=0x7f9a340cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 0x7f9a340cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 >> v2:172.21.15.195:3300/0 conn(0x7f9a340d00b0 msgr2=0x7f9a340b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 --2- 172.21.15.157:0/536729054 >> v2:172.21.15.195:3300/0 conn(0x7f9a340d00b0 0x7f9a340b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340d1b60 con 0x7f9a3415caa0 2023-11-06T03:16:44.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9a30001680 con 0x7f9a3415caa0 2023-11-06T03:16:44.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 >> v1:172.21.15.157:6789/0 conn(0x7f9a3415caa0 legacy=0x7f9a340cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 0x7f9a20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 0x7f9a20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d0520 con 0x7f9a20003980 2023-11-06T03:16:44.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d18a0 con 0x7f9a200047d0 2023-11-06T03:16:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2000b9c0 con 0x7f9a3415ce80 2023-11-06T03:16:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.223+0000 7f9a39809640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 -- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 msgr2=0x7f9a20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415ce80 0x7f9a20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 -- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 msgr2=0x7f9a20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a200047d0 0x7f9a20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 -- 172.21.15.157:0/536729054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20003530 con 0x7f9a20003980 2023-11-06T03:16:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a39809640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c240 tx=0x7f9a3001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b6ba11920b23b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a300058e0 con 0x7f9a20003980 2023-11-06T03:16:44.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30005a40 con 0x7f9a20003980 2023-11-06T03:16:44.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a38807640 1 -- 172.21.15.157:0/536729054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30005dd0 con 0x7f9a20003980 2023-11-06T03:16:44.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/536729054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 msgr2=0x7f9a20003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/536729054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a20003980 0x7f9a20003da0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c240 tx=0x7f9a3001cc30 comp rx=0 tx=0).stop 2023-11-06T03:16:44.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3a00a640 1 -- 172.21.15.157:0/536729054 reap_dead start 2023-11-06T03:16:44.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/536729054 shutdown_connections 2023-11-06T03:16:44.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/536729054 >> 172.21.15.157:0/536729054 conn(0x7f9a34070790 msgr2=0x7f9a34070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:44.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/536729054 shutdown_connections 2023-11-06T03:16:44.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.227+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/536729054 wait complete. 2023-11-06T03:16:44.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 Processor -- start 2023-11-06T03:16:44.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- start start 2023-11-06T03:16:44.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a20003980 0x7f9a34156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 0x7f9a341625b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a341583a0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34157790 con 0x7f9a20003980 2023-11-06T03:16:44.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34157a70 con 0x7f9a3415e720 2023-11-06T03:16:44.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38800/0 (socket says 172.21.15.157:38800) 2023-11-06T03:16:44.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 -- 172.21.15.157:0/2101650962 learned_addr learned my addr 172.21.15.157:0/2101650962 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:44.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39809640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a20003980 0x7f9a34156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3a00a640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 0x7f9a341625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 -- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 msgr2=0x7f9a341625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 0x7f9a341625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 -- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a20003980 msgr2=0x7f9a34156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a20003980 0x7f9a34156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 -- 172.21.15.157:0/2101650962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34157d50 con 0x7f9a3415e2c0 2023-11-06T03:16:44.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3a00a640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 0x7f9a341625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:44.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a39008640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400d800 tx=0x7f9a2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2400b6c0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2400b840 con 0x7f9a3415e2c0 2023-11-06T03:16:44.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a34158030 con 0x7f9a3415e2c0 2023-11-06T03:16:44.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2400bbf0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a341573b0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34072750 con 0x7f9a3415e2c0 2023-11-06T03:16:44.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a2400bd50 con 0x7f9a3415e2c0 2023-11-06T03:16:44.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.231+0000 7f9a2a7fc640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 0x7f9a18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.235+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9a24083ac0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.235+0000 7f9a39809640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 0x7f9a18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.235+0000 7f9a39809640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 0x7f9a18067f20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9a300aa000 tx=0x7f9a30001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.239+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9a24051270 con 0x7f9a3415e2c0 2023-11-06T03:16:44.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.427+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9a340cdcf0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.455+0000 7f9a2a7fc640 1 -- 172.21.15.157:0/2101650962 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v116) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9a240180b0 con 0x7f9a3415e2c0 2023-11-06T03:16:44.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 msgr2=0x7f9a18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 0x7f9a18067f20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9a300aa000 tx=0x7f9a30001a80 comp rx=0 tx=0).stop 2023-11-06T03:16:44.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 msgr2=0x7f9a34162070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400d800 tx=0x7f9a2400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 shutdown_connections 2023-11-06T03:16:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9a18065a60 0x7f9a18067f20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9a3415e720 0x7f9a341625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a3415e2c0 0x7f9a34162070 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 --2- 172.21.15.157:0/2101650962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a20003980 0x7f9a34156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 >> 172.21.15.157:0/2101650962 conn(0x7f9a34070790 msgr2=0x7f9a34163020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 shutdown_connections 2023-11-06T03:16:44.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.459+0000 7f9a3ba6d640 1 -- 172.21.15.157:0/2101650962 wait complete. 2023-11-06T03:16:44.473 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:16:44.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:16:44.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 Processor -- start 2023-11-06T03:16:44.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 -- start start 2023-11-06T03:16:44.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd741551e0 0x7fcd7406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd7415e5e0 0x7fcd740b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7415f580 con 0x7fcd7415ea60 2023-11-06T03:16:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7415efc0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd7b2bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7415f2a0 con 0x7fcd741551e0 2023-11-06T03:16:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd7415e5e0 0x7fcd740b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd7415e5e0 0x7fcd740b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38812/0 (socket says 172.21.15.157:38812) 2023-11-06T03:16:44.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 -- 172.21.15.157:0/1412119322 learned_addr learned my addr 172.21.15.157:0/1412119322 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd79057640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd741551e0 0x7fcd7406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 -- 172.21.15.157:0/1412119322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd741551e0 msgr2=0x7fcd7406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd741551e0 0x7fcd7406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 -- 172.21.15.157:0/1412119322 >> v1:172.21.15.195:6789/0 conn(0x7fcd7415ea60 legacy=0x7fcd7415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 -- 172.21.15.157:0/1412119322 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7406f5a0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.599+0000 7fcd78856640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd740b3410 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009870 tx=0x7fcd6401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9837c016076b300a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd6bfff640 1 -- 172.21.15.157:0/1412119322 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640ac640 con 0x7fcd7415e5e0 2023-11-06T03:16:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd6bfff640 1 -- 172.21.15.157:0/1412119322 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640ac7a0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd6bfff640 1 -- 172.21.15.157:0/1412119322 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640acb30 con 0x7fcd7415e5e0 2023-11-06T03:16:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1412119322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 msgr2=0x7fcd740b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd740b3410 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009870 tx=0x7fcd6401c440 comp rx=0 tx=0).stop 2023-11-06T03:16:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1412119322 shutdown_connections 2023-11-06T03:16:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd740b3410 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1412119322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd741551e0 0x7fcd7406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1412119322 >> 172.21.15.157:0/1412119322 conn(0x7fcd74065930 msgr2=0x7fcd74065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1412119322 shutdown_connections 2023-11-06T03:16:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1412119322 wait complete. 2023-11-06T03:16:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 Processor -- start 2023-11-06T03:16:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- start start 2023-11-06T03:16:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd741551e0 0x7fcd7414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7415ea60 0x7fcd74159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74154aa0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7406bc70 con 0x7fcd741551e0 2023-11-06T03:16:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74154200 con 0x7fcd7415ea60 2023-11-06T03:16:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38824/0 (socket says 172.21.15.157:38824) 2023-11-06T03:16:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 -- 172.21.15.157:0/1597749185 learned_addr learned my addr 172.21.15.157:0/1597749185 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd79858640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7415ea60 0x7fcd74159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 -- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7415ea60 msgr2=0x7fcd74159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd79057640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd741551e0 0x7fcd7414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7415ea60 0x7fcd74159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 -- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd741551e0 msgr2=0x7fcd7414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd741551e0 0x7fcd7414b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 -- 172.21.15.157:0/1597749185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640094d0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd78856640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fcd640af040 tx=0x7fcd640191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640ac640 con 0x7fcd7415e5e0 2023-11-06T03:16:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640b5870 con 0x7fcd7415e5e0 2023-11-06T03:16:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd741544e0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640b5c20 con 0x7fcd7415e5e0 2023-11-06T03:16:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.603+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd741534b0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7406bde0 con 0x7fcd7415e5e0 2023-11-06T03:16:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcd640b3070 con 0x7fcd7415e5e0 2023-11-06T03:16:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd69ffb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 0x7fcd58067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd6411d560 con 0x7fcd7415e5e0 2023-11-06T03:16:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd79057640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 0x7fcd58067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.607+0000 7fcd79057640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 0x7fcd58067f20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b7c0 tx=0x7fcd70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:44.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.611+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcd640eae30 con 0x7fcd7415e5e0 2023-11-06T03:16:44.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:44.799+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcd74072750 con 0x7fcd7415e5e0 2023-11-06T03:16:44.995 INFO:tasks.ceph.mds.d.smithi195.stderr:2023-11-06T03:16:45.000+0000 7fc5ffb5a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:16:45.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.459+0000 7fcd69ffb640 1 -- 172.21.15.157:0/1597749185 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v117) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd640ee450 con 0x7fcd7415e5e0 2023-11-06T03:16:45.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 msgr2=0x7fcd58067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 0x7fcd58067f20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b7c0 tx=0x7fcd70008040 comp rx=0 tx=0).stop 2023-11-06T03:16:45.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 msgr2=0x7fcd741599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fcd640af040 tx=0x7fcd640191e0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 shutdown_connections 2023-11-06T03:16:45.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd58065a60 0x7fcd58067f20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd7415ea60 0x7fcd74159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd7415e5e0 0x7fcd741599d0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 --2- 172.21.15.157:0/1597749185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd741551e0 0x7fcd7414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 >> 172.21.15.157:0/1597749185 conn(0x7fcd74065930 msgr2=0x7fcd740b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:45.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 shutdown_connections 2023-11-06T03:16:45.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.463+0000 7fcd7b2bb640 1 -- 172.21.15.157:0/1597749185 wait complete. 2023-11-06T03:16:45.482 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:16:45.482 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:45.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 Processor -- start 2023-11-06T03:16:45.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 -- start start 2023-11-06T03:16:45.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 0x7fdb080cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb080d0280 0x7fdb080b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb080d2920 con 0x7fdb0815ce80 2023-11-06T03:16:45.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080d18a0 con 0x7fdb080d0280 2023-11-06T03:16:45.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb0d708640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080d1b60 con 0x7fdb0815caa0 2023-11-06T03:16:45.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb06ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 0x7fdb080cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb067fc640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fdb0815ce80 0x7fdb080cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48830/0 (socket says 172.21.15.157:48830) 2023-11-06T03:16:45.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb077fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb080d0280 0x7fdb080b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb067fc640 1 -- 172.21.15.157:0/2534621571 learned_addr learned my addr 172.21.15.157:0/2534621571 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:45.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb05ffb640 1 -- 172.21.15.157:0/2534621571 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791658306 0 0) 0x7fdb080d2920 con 0x7fdb0815ce80 2023-11-06T03:16:45.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb05ffb640 1 -- 172.21.15.157:0/2534621571 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf4003530 con 0x7fdb0815ce80 2023-11-06T03:16:45.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb077fe640 1 -- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 msgr2=0x7fdb080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:45.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb077fe640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 0x7fdb080cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb077fe640 1 -- 172.21.15.157:0/2534621571 >> v1:172.21.15.195:6789/0 conn(0x7fdb0815ce80 legacy=0x7fdb080cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb077fe640 1 -- 172.21.15.157:0/2534621571 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080d1e40 con 0x7fdb080d0280 2023-11-06T03:16:45.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.611+0000 7fdb06ffd640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 0x7fdb080cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:45.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb077fe640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080b5d50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009d90 tx=0x7fdb0001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e52ac933236d4e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb05ffb640 1 -- 172.21.15.157:0/2534621571 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000a59e0 con 0x7fdb080d0280 2023-11-06T03:16:45.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb05ffb640 1 -- 172.21.15.157:0/2534621571 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb000a5b40 con 0x7fdb080d0280 2023-11-06T03:16:45.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb05ffb640 1 -- 172.21.15.157:0/2534621571 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000ad440 con 0x7fdb080d0280 2023-11-06T03:16:45.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 msgr2=0x7fdb080b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080b5d50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009d90 tx=0x7fdb0001c920 comp rx=0 tx=0).stop 2023-11-06T03:16:45.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- 172.21.15.157:0/2534621571 shutdown_connections 2023-11-06T03:16:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080b5d50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2534621571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815caa0 0x7fdb080cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- 172.21.15.157:0/2534621571 >> 172.21.15.157:0/2534621571 conn(0x7fdb08070790 msgr2=0x7fdb08070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- 172.21.15.157:0/2534621571 shutdown_connections 2023-11-06T03:16:45.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- 172.21.15.157:0/2534621571 wait complete. 2023-11-06T03:16:45.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 Processor -- start 2023-11-06T03:16:45.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- start start 2023-11-06T03:16:45.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815ce80 0x7fdb08152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0815fbd0 con 0x7fdb0815caa0 2023-11-06T03:16:45.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0815e290 con 0x7fdb080d0280 2023-11-06T03:16:45.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb0d708640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb081523f0 con 0x7fdb0815ce80 2023-11-06T03:16:45.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb067fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb067fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38826/0 (socket says 172.21.15.157:38826) 2023-11-06T03:16:45.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb067fc640 1 -- 172.21.15.157:0/2666923583 learned_addr learned my addr 172.21.15.157:0/2666923583 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:45.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb06ffd640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.615+0000 7fdb077fe640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815ce80 0x7fdb08152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 -- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815ce80 msgr2=0x7fdb08152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815ce80 0x7fdb08152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 -- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 msgr2=0x7fdb080c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 -- 172.21.15.157:0/2666923583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8009420 con 0x7fdb0815caa0 2023-11-06T03:16:45.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb06ffd640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:45.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb067fc640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c480 tx=0x7fdaf000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005cf0 con 0x7fdb0815caa0 2023-11-06T03:16:45.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0005e50 con 0x7fdb0815caa0 2023-11-06T03:16:45.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb00009a70 con 0x7fdb0815caa0 2023-11-06T03:16:45.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0007290 con 0x7fdb0815caa0 2023-11-06T03:16:45.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb08151500 con 0x7fdb0815caa0 2023-11-06T03:16:45.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0815b5d0 con 0x7fdb0815caa0 2023-11-06T03:16:45.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdaf00073f0 con 0x7fdb0815caa0 2023-11-06T03:16:45.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 0x7fdae8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdaf00828a0 con 0x7fdb0815caa0 2023-11-06T03:16:45.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb06ffd640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 0x7fdae8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.619+0000 7fdb06ffd640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 0x7fdae8067f20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b4210 tx=0x7fdaf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.623+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdaf0050190 con 0x7fdb0815caa0 2023-11-06T03:16:45.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.811+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb08072750 con 0x7fdb0815caa0 2023-11-06T03:16:45.811 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdaff7fe640 1 -- 172.21.15.157:0/2666923583 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7fdaf0017070 con 0x7fdb0815caa0 2023-11-06T03:16:45.811 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:45.812 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:16:45.812 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:16:45.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 msgr2=0x7fdae8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 0x7fdae8067f20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdb080b4210 tx=0x7fdaf8008040 comp rx=0 tx=0).stop 2023-11-06T03:16:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 msgr2=0x7fdb0815f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c480 tx=0x7fdaf000c940 comp rx=0 tx=0).stop 2023-11-06T03:16:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 shutdown_connections 2023-11-06T03:16:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdae8065a60 0x7fdae8067f20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb0815ce80 0x7fdb08152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb0815caa0 0x7fdb0815f690 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 --2- 172.21.15.157:0/2666923583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb080d0280 0x7fdb080c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 >> 172.21.15.157:0/2666923583 conn(0x7fdb08070790 msgr2=0x7fdb080b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 shutdown_connections 2023-11-06T03:16:45.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.815+0000 7fdb0d708640 1 -- 172.21.15.157:0/2666923583 wait complete. 2023-11-06T03:16:45.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:16:45.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 Processor -- start 2023-11-06T03:16:45.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 -- start start 2023-11-06T03:16:45.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa44806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4480b3b20 0x7fa44815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44815f580 con 0x7fa44815e5e0 2023-11-06T03:16:45.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44815efc0 con 0x7fa4480b3b20 2023-11-06T03:16:45.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44f681640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44815f2a0 con 0x7fa4481551e0 2023-11-06T03:16:45.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4480b3b20 0x7fa44815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa4480b3b20 0x7fa44815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38832/0 (socket says 172.21.15.157:38832) 2023-11-06T03:16:45.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 -- 172.21.15.157:0/4104684422 learned_addr learned my addr 172.21.15.157:0/4104684422 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44d41d640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa44806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 -- 172.21.15.157:0/4104684422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 msgr2=0x7fa44806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa44806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 -- 172.21.15.157:0/4104684422 >> v1:172.21.15.195:6789/0 conn(0x7fa44815e5e0 legacy=0x7fa4480b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:45.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.955+0000 7fa44dc1e640 1 -- 172.21.15.157:0/4104684422 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44806f5a0 con 0x7fa4480b3b20 2023-11-06T03:16:45.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44dc1e640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44815c1b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fa444009e50 tx=0x7fa44401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11458dd115ad44c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43ffff640 1 -- 172.21.15.157:0/4104684422 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440a56a0 con 0x7fa4480b3b20 2023-11-06T03:16:45.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43ffff640 1 -- 172.21.15.157:0/4104684422 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4440a5800 con 0x7fa4480b3b20 2023-11-06T03:16:45.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43ffff640 1 -- 172.21.15.157:0/4104684422 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440a5b90 con 0x7fa4480b3b20 2023-11-06T03:16:45.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/4104684422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 msgr2=0x7fa44815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44815c1b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fa444009e50 tx=0x7fa44401c910 comp rx=0 tx=0).stop 2023-11-06T03:16:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/4104684422 shutdown_connections 2023-11-06T03:16:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44815c1b0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- 172.21.15.157:0/4104684422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa44806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/4104684422 >> 172.21.15.157:0/4104684422 conn(0x7fa448065930 msgr2=0x7fa448065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:45.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/4104684422 shutdown_connections 2023-11-06T03:16:45.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/4104684422 wait complete. 2023-11-06T03:16:45.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 Processor -- start 2023-11-06T03:16:45.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- start start 2023-11-06T03:16:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa4480cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa44815e5e0 0x7fa44814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448144a00 con 0x7fa4480b3b20 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480cd0b0 con 0x7fa44815e5e0 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480cd390 con 0x7fa4481551e0 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38842/0 (socket says 172.21.15.157:38842) 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 -- 172.21.15.157:0/2468320197 learned_addr learned my addr 172.21.15.157:0/2468320197 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:45.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44dc1e640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa44815e5e0 0x7fa44814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44cc1c640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa4480cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 -- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 msgr2=0x7fa4480cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa4480cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 -- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa44815e5e0 msgr2=0x7fa44814b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa44815e5e0 0x7fa44814b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 -- 172.21.15.157:0/2468320197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009b30 con 0x7fa4480b3b20 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44d41d640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d800 tx=0x7fa44000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000b6c0 con 0x7fa4480b3b20 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44000b820 con 0x7fa4480b3b20 2023-11-06T03:16:45.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4480cd670 con 0x7fa4480b3b20 2023-11-06T03:16:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000bb90 con 0x7fa4480b3b20 2023-11-06T03:16:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.959+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4480cdc30 con 0x7fa4480b3b20 2023-11-06T03:16:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4480cddf0 con 0x7fa4480b3b20 2023-11-06T03:16:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa44000bcf0 con 0x7fa4480b3b20 2023-11-06T03:16:45.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa43dffb640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 0x7fa42c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa440083760 con 0x7fa4480b3b20 2023-11-06T03:16:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa44cc1c640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 0x7fa42c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.963+0000 7fa44cc1c640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 0x7fa42c067f50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa4480b3660 tx=0x7fa438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:45.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:45.967+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa440051050 con 0x7fa4480b3b20 2023-11-06T03:16:46.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.123+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa448072750 con 0x7fa4480b3b20 2023-11-06T03:16:46.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.123+0000 7fa43dffb640 1 -- 172.21.15.157:0/2468320197 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fa440054670 con 0x7fa4480b3b20 2023-11-06T03:16:46.122 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:46.122 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":163,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:44.441779+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:16:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 msgr2=0x7fa42c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 0x7fa42c067f50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa4480b3660 tx=0x7fa438008040 comp rx=0 tx=0).stop 2023-11-06T03:16:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 msgr2=0x7fa44814a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fa44000d800 tx=0x7fa44000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 shutdown_connections 2023-11-06T03:16:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa42c065a90 0x7fa42c067f50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa44815e5e0 0x7fa44814b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4481551e0 0x7fa4480cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 --2- 172.21.15.157:0/2468320197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4480b3b20 0x7fa44814a9d0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 >> 172.21.15.157:0/2468320197 conn(0x7fa448065930 msgr2=0x7fa44815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 shutdown_connections 2023-11-06T03:16:46.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.127+0000 7fa44f681640 1 -- 172.21.15.157:0/2468320197 wait complete. 2023-11-06T03:16:46.142 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:16:46.143 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:16:46.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.259+0000 7fa2a2696640 1 Processor -- start 2023-11-06T03:16:46.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.259+0000 7fa2a2696640 1 -- start start 2023-11-06T03:16:46.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a2696640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa29c0b2af0 0x7fa29c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a2696640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 0x7fa29c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a2696640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29c15e440 con 0x7fa29c1599c0 2023-11-06T03:16:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a2696640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29c06c240 con 0x7fa29c0b2af0 2023-11-06T03:16:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a2696640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c06c520 con 0x7fa29c06d000 2023-11-06T03:16:46.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa29c0b2af0 0x7fa29c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29bfff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fa29c1599c0 0x7fa29c0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:48854/0 (socket says 172.21.15.157:48854) 2023-11-06T03:16:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a0c33640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 0x7fa29c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29bfff640 1 -- 172.21.15.157:0/3334785818 learned_addr learned my addr 172.21.15.157:0/3334785818 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:46.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29b7fe640 1 -- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 msgr2=0x7fa29c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:46.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29b7fe640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 0x7fa29c159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29b7fe640 1 -- 172.21.15.157:0/3334785818 >> v1:172.21.15.195:6789/0 conn(0x7fa29c1599c0 legacy=0x7fa29c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa29b7fe640 1 -- 172.21.15.157:0/3334785818 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c06c800 con 0x7fa29c0b2af0 2023-11-06T03:16:46.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.263+0000 7fa2a0c33640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 0x7fa29c159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:16:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa29b7fe640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c0b2af0 0x7fa29c0ae000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097c0 tx=0x7fa29001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6469dc3cb24e4e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa29affd640 1 -- 172.21.15.157:0/3334785818 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900ab670 con 0x7fa29c0b2af0 2023-11-06T03:16:46.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa29affd640 1 -- 172.21.15.157:0/3334785818 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2900ab7d0 con 0x7fa29c0b2af0 2023-11-06T03:16:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa29affd640 1 -- 172.21.15.157:0/3334785818 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900abb60 con 0x7fa29c0b2af0 2023-11-06T03:16:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c0b2af0 msgr2=0x7fa29c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c0b2af0 0x7fa29c0ae000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa2900097c0 tx=0x7fa29001c450 comp rx=0 tx=0).stop 2023-11-06T03:16:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- 172.21.15.157:0/3334785818 shutdown_connections 2023-11-06T03:16:46.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c06d000 0x7fa29c159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- 172.21.15.157:0/3334785818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c0b2af0 0x7fa29c0ae000 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- 172.21.15.157:0/3334785818 >> 172.21.15.157:0/3334785818 conn(0x7fa29c065930 msgr2=0x7fa29c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- 172.21.15.157:0/3334785818 shutdown_connections 2023-11-06T03:16:46.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- 172.21.15.157:0/3334785818 wait complete. 2023-11-06T03:16:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 Processor -- start 2023-11-06T03:16:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- start start 2023-11-06T03:16:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c06d000 0x7fa29c140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 0x7fa29c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c150e00 con 0x7fa29c1599c0 2023-11-06T03:16:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c1409b0 con 0x7fa29c06d000 2023-11-06T03:16:46.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.267+0000 7fa2a2696640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c14fcd0 con 0x7fa29c0b2af0 2023-11-06T03:16:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38858/0 (socket says 172.21.15.157:38858) 2023-11-06T03:16:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 -- 172.21.15.157:0/2420724325 learned_addr learned my addr 172.21.15.157:0/2420724325 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa29b7fe640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 0x7fa29c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa29bfff640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c06d000 0x7fa29c140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 -- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 msgr2=0x7fa29c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 0x7fa29c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 -- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c06d000 msgr2=0x7fa29c140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c06d000 0x7fa29c140470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 -- 172.21.15.157:0/2420724325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294009a70 con 0x7fa29c1599c0 2023-11-06T03:16:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa29b7fe640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 0x7fa29c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a0c33640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fa29400efc0 tx=0x7fa29400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294014480 con 0x7fa29c1599c0 2023-11-06T03:16:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400f040 con 0x7fa29c1599c0 2023-11-06T03:16:46.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa290009420 con 0x7fa29c1599c0 2023-11-06T03:16:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294014840 con 0x7fa29c1599c0 2023-11-06T03:16:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c150740 con 0x7fa29c1599c0 2023-11-06T03:16:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c150f40 con 0x7fa29c1599c0 2023-11-06T03:16:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa2940149a0 con 0x7fa29c1599c0 2023-11-06T03:16:46.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 0x7fa288067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2940841b0 con 0x7fa29c1599c0 2023-11-06T03:16:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.271+0000 7fa29bfff640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 0x7fa288067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.275+0000 7fa29bfff640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 0x7fa288067f20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000f70 tx=0x7fa28c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.275+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa29c150f40 con 0x7fa29c1599c0 2023-11-06T03:16:46.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.427+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa29c150f40 con 0x7fa288065a60 2023-11-06T03:16:46.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.451+0000 7fa298ff9640 1 -- 172.21.15.157:0/2420724325 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310932 (secure 0 0 0) 0x7fa29c150f40 con 0x7fa288065a60 2023-11-06T03:16:46.450 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:46.454 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:16:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 msgr2=0x7fa288067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 0x7fa288067f20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa28c000f70 tx=0x7fa28c009040 comp rx=0 tx=0).stop 2023-11-06T03:16:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 msgr2=0x7fa29c0acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fa29400efc0 tx=0x7fa29400c490 comp rx=0 tx=0).stop 2023-11-06T03:16:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 shutdown_connections 2023-11-06T03:16:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa288065a60 0x7fa288067f20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa29c1599c0 0x7fa29c0acdf0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa29c0b2af0 0x7fa29c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 --2- 172.21.15.157:0/2420724325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa29c06d000 0x7fa29c140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 >> 172.21.15.157:0/2420724325 conn(0x7fa29c065930 msgr2=0x7fa29c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 shutdown_connections 2023-11-06T03:16:46.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.455+0000 7fa2a2696640 1 -- 172.21.15.157:0/2420724325 wait complete. 2023-11-06T03:16:46.472 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2023-11-06T03:16:45.370681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":642452,"kb_used_data":17044,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754332268,"statfs":{"total":773094113280,"available":772436242432,"internally_reserved":0,"allocated":17453056,"data_stored":8712175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001301"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490301+0000","last_change":"2023-11-06T03:16:43.476040+0000","last_active":"2023-11-06T03:16:43.490301+0000","last_peered":"2023-11-06T03:16:43.490301+0000","last_clean":"2023-11-06T03:16:43.490301+0000","last_became_active":"2023-11-06T03:16:43.475488+0000","last_became_peered":"2023-11-06T03:16:43.475488+0000","last_unstale":"2023-11-06T03:16:43.490301+0000","last_undegraded":"2023-11-06T03:16:43.490301+0000","last_fullsized":"2023-11-06T03:16:43.490301+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:08.080154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481857+0000","last_change":"2023-11-06T03:16:43.471406+0000","last_active":"2023-11-06T03:16:43.481857+0000","last_peered":"2023-11-06T03:16:43.481857+0000","last_clean":"2023-11-06T03:16:43.481857+0000","last_became_active":"2023-11-06T03:16:43.471033+0000","last_became_peered":"2023-11-06T03:16:43.471033+0000","last_unstale":"2023-11-06T03:16:43.481857+0000","last_undegraded":"2023-11-06T03:16:43.481857+0000","last_fullsized":"2023-11-06T03:16:43.481857+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:36:04.624576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485753+0000","last_change":"2023-11-06T03:16:43.466824+0000","last_active":"2023-11-06T03:16:43.485753+0000","last_peered":"2023-11-06T03:16:43.485753+0000","last_clean":"2023-11-06T03:16:43.485753+0000","last_became_active":"2023-11-06T03:16:43.466369+0000","last_became_peered":"2023-11-06T03:16:43.466369+0000","last_unstale":"2023-11-06T03:16:43.485753+0000","last_undegraded":"2023-11-06T03:16:43.485753+0000","last_fullsized":"2023-11-06T03:16:43.485753+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:23.704213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487913+0000","last_change":"2023-11-06T03:16:43.475776+0000","last_active":"2023-11-06T03:16:43.487913+0000","last_peered":"2023-11-06T03:16:43.487913+0000","last_clean":"2023-11-06T03:16:43.487913+0000","last_became_active":"2023-11-06T03:16:43.475484+0000","last_became_peered":"2023-11-06T03:16:43.475484+0000","last_unstale":"2023-11-06T03:16:43.487913+0000","last_undegraded":"2023-11-06T03:16:43.487913+0000","last_fullsized":"2023-11-06T03:16:43.487913+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:47.537838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459749+0000","last_change":"2023-11-06T03:16:43.457255+0000","last_active":"2023-11-06T03:16:43.459749+0000","last_peered":"2023-11-06T03:16:43.459749+0000","last_clean":"2023-11-06T03:16:43.459749+0000","last_became_active":"2023-11-06T03:16:43.456893+0000","last_became_peered":"2023-11-06T03:16:43.456893+0000","last_unstale":"2023-11-06T03:16:43.459749+0000","last_undegraded":"2023-11-06T03:16:43.459749+0000","last_fullsized":"2023-11-06T03:16:43.459749+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:03:05.620215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490487+0000","last_change":"2023-11-06T03:16:43.486781+0000","last_active":"2023-11-06T03:16:43.490487+0000","last_peered":"2023-11-06T03:16:43.490487+0000","last_clean":"2023-11-06T03:16:43.490487+0000","last_became_active":"2023-11-06T03:16:43.486367+0000","last_became_peered":"2023-11-06T03:16:43.486367+0000","last_unstale":"2023-11-06T03:16:43.490487+0000","last_undegraded":"2023-11-06T03:16:43.490487+0000","last_fullsized":"2023-11-06T03:16:43.490487+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:42.904810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491390+0000","last_change":"2023-11-06T03:16:43.483187+0000","last_active":"2023-11-06T03:16:43.491390+0000","last_peered":"2023-11-06T03:16:43.491390+0000","last_clean":"2023-11-06T03:16:43.491390+0000","last_became_active":"2023-11-06T03:16:43.482698+0000","last_became_peered":"2023-11-06T03:16:43.482698+0000","last_unstale":"2023-11-06T03:16:43.491390+0000","last_undegraded":"2023-11-06T03:16:43.491390+0000","last_fullsized":"2023-11-06T03:16:43.491390+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:10.825573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493416+0000","last_change":"2023-11-06T03:16:43.490867+0000","last_active":"2023-11-06T03:16:43.493416+0000","last_peered":"2023-11-06T03:16:43.493416+0000","last_clean":"2023-11-06T03:16:43.493416+0000","last_became_active":"2023-11-06T03:16:43.487629+0000","last_became_peered":"2023-11-06T03:16:43.487629+0000","last_unstale":"2023-11-06T03:16:43.493416+0000","last_undegraded":"2023-11-06T03:16:43.493416+0000","last_fullsized":"2023-11-06T03:16:43.493416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:18.321258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491382+0000","last_change":"2023-11-06T03:16:43.474722+0000","last_active":"2023-11-06T03:16:43.491382+0000","last_peered":"2023-11-06T03:16:43.491382+0000","last_clean":"2023-11-06T03:16:43.491382+0000","last_became_active":"2023-11-06T03:16:43.474374+0000","last_became_peered":"2023-11-06T03:16:43.474374+0000","last_unstale":"2023-11-06T03:16:43.491382+0000","last_undegraded":"2023-11-06T03:16:43.491382+0000","last_fullsized":"2023-11-06T03:16:43.491382+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:38.206951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493401+0000","last_change":"2023-11-06T03:16:43.490825+0000","last_active":"2023-11-06T03:16:43.493401+0000","last_peered":"2023-11-06T03:16:43.493401+0000","last_clean":"2023-11-06T03:16:43.493401+0000","last_became_active":"2023-11-06T03:16:43.490470+0000","last_became_peered":"2023-11-06T03:16:43.490470+0000","last_unstale":"2023-11-06T03:16:43.493401+0000","last_undegraded":"2023-11-06T03:16:43.493401+0000","last_fullsized":"2023-11-06T03:16:43.493401+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:25:44.976366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.472377+0000","last_change":"2023-11-06T03:16:43.471178+0000","last_active":"2023-11-06T03:16:43.472377+0000","last_peered":"2023-11-06T03:16:43.472377+0000","last_clean":"2023-11-06T03:16:43.472377+0000","last_became_active":"2023-11-06T03:16:43.470917+0000","last_became_peered":"2023-11-06T03:16:43.470917+0000","last_unstale":"2023-11-06T03:16:43.472377+0000","last_undegraded":"2023-11-06T03:16:43.472377+0000","last_fullsized":"2023-11-06T03:16:43.472377+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:20.794448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490053+0000","last_change":"2023-11-06T03:16:43.486441+0000","last_active":"2023-11-06T03:16:43.490053+0000","last_peered":"2023-11-06T03:16:43.490053+0000","last_clean":"2023-11-06T03:16:43.490053+0000","last_became_active":"2023-11-06T03:16:43.486154+0000","last_became_peered":"2023-11-06T03:16:43.486154+0000","last_unstale":"2023-11-06T03:16:43.490053+0000","last_undegraded":"2023-11-06T03:16:43.490053+0000","last_fullsized":"2023-11-06T03:16:43.490053+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:26.350831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477551+0000","last_change":"2023-11-06T03:16:43.465429+0000","last_active":"2023-11-06T03:16:43.477551+0000","last_peered":"2023-11-06T03:16:43.477551+0000","last_clean":"2023-11-06T03:16:43.477551+0000","last_became_active":"2023-11-06T03:16:43.465147+0000","last_became_peered":"2023-11-06T03:16:43.465147+0000","last_unstale":"2023-11-06T03:16:43.477551+0000","last_undegraded":"2023-11-06T03:16:43.477551+0000","last_fullsized":"2023-11-06T03:16:43.477551+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:21:34.480405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459831+0000","last_change":"2023-11-06T03:16:43.457674+0000","last_active":"2023-11-06T03:16:43.459831+0000","last_peered":"2023-11-06T03:16:43.459831+0000","last_clean":"2023-11-06T03:16:43.459831+0000","last_became_active":"2023-11-06T03:16:43.457458+0000","last_became_peered":"2023-11-06T03:16:43.457458+0000","last_unstale":"2023-11-06T03:16:43.459831+0000","last_undegraded":"2023-11-06T03:16:43.459831+0000","last_fullsized":"2023-11-06T03:16:43.459831+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.656829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490014+0000","last_change":"2023-11-06T03:16:43.487181+0000","last_active":"2023-11-06T03:16:43.490014+0000","last_peered":"2023-11-06T03:16:43.490014+0000","last_clean":"2023-11-06T03:16:43.490014+0000","last_became_active":"2023-11-06T03:16:43.486713+0000","last_became_peered":"2023-11-06T03:16:43.486713+0000","last_unstale":"2023-11-06T03:16:43.490014+0000","last_undegraded":"2023-11-06T03:16:43.490014+0000","last_fullsized":"2023-11-06T03:16:43.490014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:35.085986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487913+0000","last_change":"2023-11-06T03:16:43.475305+0000","last_active":"2023-11-06T03:16:43.487913+0000","last_peered":"2023-11-06T03:16:43.487913+0000","last_clean":"2023-11-06T03:16:43.487913+0000","last_became_active":"2023-11-06T03:16:43.474973+0000","last_became_peered":"2023-11-06T03:16:43.474973+0000","last_unstale":"2023-11-06T03:16:43.487913+0000","last_undegraded":"2023-11-06T03:16:43.487913+0000","last_fullsized":"2023-11-06T03:16:43.487913+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:44.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491345+0000","last_change":"2023-11-06T03:16:43.473039+0000","last_active":"2023-11-06T03:16:43.491345+0000","last_peered":"2023-11-06T03:16:43.491345+0000","last_clean":"2023-11-06T03:16:43.491345+0000","last_became_active":"2023-11-06T03:16:43.472293+0000","last_became_peered":"2023-11-06T03:16:43.472293+0000","last_unstale":"2023-11-06T03:16:43.491345+0000","last_undegraded":"2023-11-06T03:16:43.491345+0000","last_fullsized":"2023-11-06T03:16:43.491345+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:40.792037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.488561+0000","last_change":"2023-11-06T03:16:43.487853+0000","last_active":"2023-11-06T03:16:43.488561+0000","last_peered":"2023-11-06T03:16:43.488561+0000","last_clean":"2023-11-06T03:16:43.488561+0000","last_became_active":"2023-11-06T03:16:43.487445+0000","last_became_peered":"2023-11-06T03:16:43.487445+0000","last_unstale":"2023-11-06T03:16:43.488561+0000","last_undegraded":"2023-11-06T03:16:43.488561+0000","last_fullsized":"2023-11-06T03:16:43.488561+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:42:49.978034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490395+0000","last_change":"2023-11-06T03:16:43.489781+0000","last_active":"2023-11-06T03:16:43.490395+0000","last_peered":"2023-11-06T03:16:43.490395+0000","last_clean":"2023-11-06T03:16:43.490395+0000","last_became_active":"2023-11-06T03:16:43.489301+0000","last_became_peered":"2023-11-06T03:16:43.489301+0000","last_unstale":"2023-11-06T03:16:43.490395+0000","last_undegraded":"2023-11-06T03:16:43.490395+0000","last_fullsized":"2023-11-06T03:16:43.490395+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:51.344363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468035+0000","last_change":"2023-11-06T03:16:43.464404+0000","last_active":"2023-11-06T03:16:43.468035+0000","last_peered":"2023-11-06T03:16:43.468035+0000","last_clean":"2023-11-06T03:16:43.468035+0000","last_became_active":"2023-11-06T03:16:43.464109+0000","last_became_peered":"2023-11-06T03:16:43.464109+0000","last_unstale":"2023-11-06T03:16:43.468035+0000","last_undegraded":"2023-11-06T03:16:43.468035+0000","last_fullsized":"2023-11-06T03:16:43.468035+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:59.046167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489521+0000","last_change":"2023-11-06T03:16:43.486937+0000","last_active":"2023-11-06T03:16:43.489521+0000","last_peered":"2023-11-06T03:16:43.489521+0000","last_clean":"2023-11-06T03:16:43.489521+0000","last_became_active":"2023-11-06T03:16:43.486425+0000","last_became_peered":"2023-11-06T03:16:43.486425+0000","last_unstale":"2023-11-06T03:16:43.489521+0000","last_undegraded":"2023-11-06T03:16:43.489521+0000","last_fullsized":"2023-11-06T03:16:43.489521+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:24.829598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481636+0000","last_change":"2023-11-06T03:16:43.464963+0000","last_active":"2023-11-06T03:16:43.481636+0000","last_peered":"2023-11-06T03:16:43.481636+0000","last_clean":"2023-11-06T03:16:43.481636+0000","last_became_active":"2023-11-06T03:16:43.464639+0000","last_became_peered":"2023-11-06T03:16:43.464639+0000","last_unstale":"2023-11-06T03:16:43.481636+0000","last_undegraded":"2023-11-06T03:16:43.481636+0000","last_fullsized":"2023-11-06T03:16:43.481636+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:51.344363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480305+0000","last_change":"2023-11-06T03:16:43.479329+0000","last_active":"2023-11-06T03:16:43.480305+0000","last_peered":"2023-11-06T03:16:43.480305+0000","last_clean":"2023-11-06T03:16:43.480305+0000","last_became_active":"2023-11-06T03:16:43.478933+0000","last_became_peered":"2023-11-06T03:16:43.478933+0000","last_unstale":"2023-11-06T03:16:43.480305+0000","last_undegraded":"2023-11-06T03:16:43.480305+0000","last_fullsized":"2023-11-06T03:16:43.480305+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:36.998194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485490+0000","last_change":"2023-11-06T03:16:43.460587+0000","last_active":"2023-11-06T03:16:43.485490+0000","last_peered":"2023-11-06T03:16:43.485490+0000","last_clean":"2023-11-06T03:16:43.485490+0000","last_became_active":"2023-11-06T03:16:43.460304+0000","last_became_peered":"2023-11-06T03:16:43.460304+0000","last_unstale":"2023-11-06T03:16:43.485490+0000","last_undegraded":"2023-11-06T03:16:43.485490+0000","last_fullsized":"2023-11-06T03:16:43.485490+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477887+0000","last_change":"2023-11-06T03:16:43.472196+0000","last_active":"2023-11-06T03:16:43.477887+0000","last_peered":"2023-11-06T03:16:43.477887+0000","last_clean":"2023-11-06T03:16:43.477887+0000","last_became_active":"2023-11-06T03:16:43.471885+0000","last_became_peered":"2023-11-06T03:16:43.471885+0000","last_unstale":"2023-11-06T03:16:43.477887+0000","last_undegraded":"2023-11-06T03:16:43.477887+0000","last_fullsized":"2023-11-06T03:16:43.477887+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:02.501410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.466582+0000","last_change":"2023-11-06T03:16:43.462862+0000","last_active":"2023-11-06T03:16:43.466582+0000","last_peered":"2023-11-06T03:16:43.466582+0000","last_clean":"2023-11-06T03:16:43.466582+0000","last_became_active":"2023-11-06T03:16:43.462531+0000","last_became_peered":"2023-11-06T03:16:43.462531+0000","last_unstale":"2023-11-06T03:16:43.466582+0000","last_undegraded":"2023-11-06T03:16:43.466582+0000","last_fullsized":"2023-11-06T03:16:43.466582+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:07.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484091+0000","last_change":"2023-11-06T03:16:43.473397+0000","last_active":"2023-11-06T03:16:43.484091+0000","last_peered":"2023-11-06T03:16:43.484091+0000","last_clean":"2023-11-06T03:16:43.484091+0000","last_became_active":"2023-11-06T03:16:43.472808+0000","last_became_peered":"2023-11-06T03:16:43.472808+0000","last_unstale":"2023-11-06T03:16:43.484091+0000","last_undegraded":"2023-11-06T03:16:43.484091+0000","last_fullsized":"2023-11-06T03:16:43.484091+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:56.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489502+0000","last_change":"2023-11-06T03:16:43.486783+0000","last_active":"2023-11-06T03:16:43.489502+0000","last_peered":"2023-11-06T03:16:43.489502+0000","last_clean":"2023-11-06T03:16:43.489502+0000","last_became_active":"2023-11-06T03:16:43.486478+0000","last_became_peered":"2023-11-06T03:16:43.486478+0000","last_unstale":"2023-11-06T03:16:43.489502+0000","last_undegraded":"2023-11-06T03:16:43.489502+0000","last_fullsized":"2023-11-06T03:16:43.489502+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:20.219291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490988+0000","last_change":"2023-11-06T03:16:43.490219+0000","last_active":"2023-11-06T03:16:43.490988+0000","last_peered":"2023-11-06T03:16:43.490988+0000","last_clean":"2023-11-06T03:16:43.490988+0000","last_became_active":"2023-11-06T03:16:43.489683+0000","last_became_peered":"2023-11-06T03:16:43.489683+0000","last_unstale":"2023-11-06T03:16:43.490988+0000","last_undegraded":"2023-11-06T03:16:43.490988+0000","last_fullsized":"2023-11-06T03:16:43.490988+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:23.704213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.482271+0000","last_change":"2023-11-06T03:16:43.475695+0000","last_active":"2023-11-06T03:16:43.482271+0000","last_peered":"2023-11-06T03:16:43.482271+0000","last_clean":"2023-11-06T03:16:43.482271+0000","last_became_active":"2023-11-06T03:16:43.475144+0000","last_became_peered":"2023-11-06T03:16:43.475144+0000","last_unstale":"2023-11-06T03:16:43.482271+0000","last_undegraded":"2023-11-06T03:16:43.482271+0000","last_fullsized":"2023-11-06T03:16:43.482271+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:06.262155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481987+0000","last_change":"2023-11-06T03:16:43.481347+0000","last_active":"2023-11-06T03:16:43.481987+0000","last_peered":"2023-11-06T03:16:43.481987+0000","last_clean":"2023-11-06T03:16:43.481987+0000","last_became_active":"2023-11-06T03:16:43.480931+0000","last_became_peered":"2023-11-06T03:16:43.480931+0000","last_unstale":"2023-11-06T03:16:43.481987+0000","last_undegraded":"2023-11-06T03:16:43.481987+0000","last_fullsized":"2023-11-06T03:16:43.481987+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485234+0000","last_change":"2023-11-06T03:16:43.484322+0000","last_active":"2023-11-06T03:16:43.485234+0000","last_peered":"2023-11-06T03:16:43.485234+0000","last_clean":"2023-11-06T03:16:43.485234+0000","last_became_active":"2023-11-06T03:16:43.483535+0000","last_became_peered":"2023-11-06T03:16:43.483535+0000","last_unstale":"2023-11-06T03:16:43.485234+0000","last_undegraded":"2023-11-06T03:16:43.485234+0000","last_fullsized":"2023-11-06T03:16:43.485234+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:25.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484304+0000","last_change":"2023-11-06T03:16:43.477939+0000","last_active":"2023-11-06T03:16:43.484304+0000","last_peered":"2023-11-06T03:16:43.484304+0000","last_clean":"2023-11-06T03:16:43.484304+0000","last_became_active":"2023-11-06T03:16:43.477382+0000","last_became_peered":"2023-11-06T03:16:43.477382+0000","last_unstale":"2023-11-06T03:16:43.484304+0000","last_undegraded":"2023-11-06T03:16:43.484304+0000","last_fullsized":"2023-11-06T03:16:43.484304+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:35.085986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"162'4","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480230+0000","last_change":"2023-11-06T03:16:43.080958+0000","last_active":"2023-11-06T03:16:43.480230+0000","last_peered":"2023-11-06T03:16:43.480230+0000","last_clean":"2023-11-06T03:16:43.480230+0000","last_became_active":"2023-11-06T03:16:43.080227+0000","last_became_peered":"2023-11-06T03:16:43.080227+0000","last_unstale":"2023-11-06T03:16:43.480230+0000","last_undegraded":"2023-11-06T03:16:43.480230+0000","last_fullsized":"2023-11-06T03:16:43.480230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:25.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483378+0000","last_change":"2023-11-06T03:16:43.479053+0000","last_active":"2023-11-06T03:16:43.483378+0000","last_peered":"2023-11-06T03:16:43.483378+0000","last_clean":"2023-11-06T03:16:43.483378+0000","last_became_active":"2023-11-06T03:16:43.478733+0000","last_became_peered":"2023-11-06T03:16:43.478733+0000","last_unstale":"2023-11-06T03:16:43.483378+0000","last_undegraded":"2023-11-06T03:16:43.483378+0000","last_fullsized":"2023-11-06T03:16:43.483378+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:24.678686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491308+0000","last_change":"2023-11-06T03:16:43.097292+0000","last_active":"2023-11-06T03:16:43.491308+0000","last_peered":"2023-11-06T03:16:43.491308+0000","last_clean":"2023-11-06T03:16:43.491308+0000","last_became_active":"2023-11-06T03:16:43.096950+0000","last_became_peered":"2023-11-06T03:16:43.096950+0000","last_unstale":"2023-11-06T03:16:43.491308+0000","last_undegraded":"2023-11-06T03:16:43.491308+0000","last_fullsized":"2023-11-06T03:16:43.491308+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:10.928900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459812+0000","last_change":"2023-11-06T03:16:43.456925+0000","last_active":"2023-11-06T03:16:43.459812+0000","last_peered":"2023-11-06T03:16:43.459812+0000","last_clean":"2023-11-06T03:16:43.459812+0000","last_became_active":"2023-11-06T03:16:43.456671+0000","last_became_peered":"2023-11-06T03:16:43.456671+0000","last_unstale":"2023-11-06T03:16:43.459812+0000","last_undegraded":"2023-11-06T03:16:43.459812+0000","last_fullsized":"2023-11-06T03:16:43.459812+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:32.050039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.469846+0000","last_change":"2023-11-06T03:16:43.099126+0000","last_active":"2023-11-06T03:16:43.469846+0000","last_peered":"2023-11-06T03:16:43.469846+0000","last_clean":"2023-11-06T03:16:43.469846+0000","last_became_active":"2023-11-06T03:16:43.098242+0000","last_became_peered":"2023-11-06T03:16:43.098242+0000","last_unstale":"2023-11-06T03:16:43.469846+0000","last_undegraded":"2023-11-06T03:16:43.469846+0000","last_fullsized":"2023-11-06T03:16:43.469846+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:38:45.180079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484313+0000","last_change":"2023-11-06T03:16:43.479949+0000","last_active":"2023-11-06T03:16:43.484313+0000","last_peered":"2023-11-06T03:16:43.484313+0000","last_clean":"2023-11-06T03:16:43.484313+0000","last_became_active":"2023-11-06T03:16:43.479461+0000","last_became_peered":"2023-11-06T03:16:43.479461+0000","last_unstale":"2023-11-06T03:16:43.484313+0000","last_undegraded":"2023-11-06T03:16:43.484313+0000","last_fullsized":"2023-11-06T03:16:43.484313+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:23:15.277978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480991+0000","last_change":"2023-11-06T03:16:43.092523+0000","last_active":"2023-11-06T03:16:43.480991+0000","last_peered":"2023-11-06T03:16:43.480991+0000","last_clean":"2023-11-06T03:16:43.480991+0000","last_became_active":"2023-11-06T03:16:43.092202+0000","last_became_peered":"2023-11-06T03:16:43.092202+0000","last_unstale":"2023-11-06T03:16:43.480991+0000","last_undegraded":"2023-11-06T03:16:43.480991+0000","last_fullsized":"2023-11-06T03:16:43.480991+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:14.063072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493301+0000","last_change":"2023-11-06T03:16:43.490322+0000","last_active":"2023-11-06T03:16:43.493301+0000","last_peered":"2023-11-06T03:16:43.493301+0000","last_clean":"2023-11-06T03:16:43.493301+0000","last_became_active":"2023-11-06T03:16:43.487453+0000","last_became_peered":"2023-11-06T03:16:43.487453+0000","last_unstale":"2023-11-06T03:16:43.493301+0000","last_undegraded":"2023-11-06T03:16:43.493301+0000","last_fullsized":"2023-11-06T03:16:43.493301+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:28.239699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485807+0000","last_change":"2023-11-06T03:16:43.095795+0000","last_active":"2023-11-06T03:16:43.485807+0000","last_peered":"2023-11-06T03:16:43.485807+0000","last_clean":"2023-11-06T03:16:43.485807+0000","last_became_active":"2023-11-06T03:16:43.095310+0000","last_became_peered":"2023-11-06T03:16:43.095310+0000","last_unstale":"2023-11-06T03:16:43.485807+0000","last_undegraded":"2023-11-06T03:16:43.485807+0000","last_fullsized":"2023-11-06T03:16:43.485807+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:40.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477618+0000","last_change":"2023-11-06T03:16:43.473861+0000","last_active":"2023-11-06T03:16:43.477618+0000","last_peered":"2023-11-06T03:16:43.477618+0000","last_clean":"2023-11-06T03:16:43.477618+0000","last_became_active":"2023-11-06T03:16:43.472880+0000","last_became_peered":"2023-11-06T03:16:43.472880+0000","last_unstale":"2023-11-06T03:16:43.477618+0000","last_undegraded":"2023-11-06T03:16:43.477618+0000","last_fullsized":"2023-11-06T03:16:43.477618+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:10:34.131430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.472142+0000","last_change":"2023-11-06T03:16:43.089805+0000","last_active":"2023-11-06T03:16:43.472142+0000","last_peered":"2023-11-06T03:16:43.472142+0000","last_clean":"2023-11-06T03:16:43.472142+0000","last_became_active":"2023-11-06T03:16:43.089449+0000","last_became_peered":"2023-11-06T03:16:43.089449+0000","last_unstale":"2023-11-06T03:16:43.472142+0000","last_undegraded":"2023-11-06T03:16:43.472142+0000","last_fullsized":"2023-11-06T03:16:43.472142+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:05:01.412501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480097+0000","last_change":"2023-11-06T03:16:43.475509+0000","last_active":"2023-11-06T03:16:43.480097+0000","last_peered":"2023-11-06T03:16:43.480097+0000","last_clean":"2023-11-06T03:16:43.480097+0000","last_became_active":"2023-11-06T03:16:43.475135+0000","last_became_peered":"2023-11-06T03:16:43.475135+0000","last_unstale":"2023-11-06T03:16:43.480097+0000","last_undegraded":"2023-11-06T03:16:43.480097+0000","last_fullsized":"2023-11-06T03:16:43.480097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:09:31.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.452696+0000","last_change":"2023-11-06T03:16:43.096549+0000","last_active":"2023-11-06T03:16:43.452696+0000","last_peered":"2023-11-06T03:16:43.452696+0000","last_clean":"2023-11-06T03:16:43.452696+0000","last_became_active":"2023-11-06T03:16:43.096223+0000","last_became_peered":"2023-11-06T03:16:43.096223+0000","last_unstale":"2023-11-06T03:16:43.452696+0000","last_undegraded":"2023-11-06T03:16:43.452696+0000","last_fullsized":"2023-11-06T03:16:43.452696+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:13.949940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485956+0000","last_change":"2023-11-06T03:16:43.468576+0000","last_active":"2023-11-06T03:16:43.485956+0000","last_peered":"2023-11-06T03:16:43.485956+0000","last_clean":"2023-11-06T03:16:43.485956+0000","last_became_active":"2023-11-06T03:16:43.467864+0000","last_became_peered":"2023-11-06T03:16:43.467864+0000","last_unstale":"2023-11-06T03:16:43.485956+0000","last_undegraded":"2023-11-06T03:16:43.485956+0000","last_fullsized":"2023-11-06T03:16:43.485956+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:05.877838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.461104+0000","last_change":"2023-11-06T03:16:43.089090+0000","last_active":"2023-11-06T03:16:43.461104+0000","last_peered":"2023-11-06T03:16:43.461104+0000","last_clean":"2023-11-06T03:16:43.461104+0000","last_became_active":"2023-11-06T03:16:43.088728+0000","last_became_peered":"2023-11-06T03:16:43.088728+0000","last_unstale":"2023-11-06T03:16:43.461104+0000","last_undegraded":"2023-11-06T03:16:43.461104+0000","last_fullsized":"2023-11-06T03:16:43.461104+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:04.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493211+0000","last_change":"2023-11-06T03:16:43.484089+0000","last_active":"2023-11-06T03:16:43.493211+0000","last_peered":"2023-11-06T03:16:43.493211+0000","last_clean":"2023-11-06T03:16:43.493211+0000","last_became_active":"2023-11-06T03:16:43.483085+0000","last_became_peered":"2023-11-06T03:16:43.483085+0000","last_unstale":"2023-11-06T03:16:43.493211+0000","last_undegraded":"2023-11-06T03:16:43.493211+0000","last_fullsized":"2023-11-06T03:16:43.493211+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:03.768113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489111+0000","last_change":"2023-11-06T03:16:43.103705+0000","last_active":"2023-11-06T03:16:43.489111+0000","last_peered":"2023-11-06T03:16:43.489111+0000","last_clean":"2023-11-06T03:16:43.489111+0000","last_became_active":"2023-11-06T03:16:43.103387+0000","last_became_peered":"2023-11-06T03:16:43.103387+0000","last_unstale":"2023-11-06T03:16:43.489111+0000","last_undegraded":"2023-11-06T03:16:43.489111+0000","last_fullsized":"2023-11-06T03:16:43.489111+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:24:21.731897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483458+0000","last_change":"2023-11-06T03:16:43.478663+0000","last_active":"2023-11-06T03:16:43.483458+0000","last_peered":"2023-11-06T03:16:43.483458+0000","last_clean":"2023-11-06T03:16:43.483458+0000","last_became_active":"2023-11-06T03:16:43.478300+0000","last_became_peered":"2023-11-06T03:16:43.478300+0000","last_unstale":"2023-11-06T03:16:43.483458+0000","last_undegraded":"2023-11-06T03:16:43.483458+0000","last_fullsized":"2023-11-06T03:16:43.483458+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:06.501292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491242+0000","last_change":"2023-11-06T03:16:43.093755+0000","last_active":"2023-11-06T03:16:43.491242+0000","last_peered":"2023-11-06T03:16:43.491242+0000","last_clean":"2023-11-06T03:16:43.491242+0000","last_became_active":"2023-11-06T03:16:43.092656+0000","last_became_peered":"2023-11-06T03:16:43.092656+0000","last_unstale":"2023-11-06T03:16:43.491242+0000","last_undegraded":"2023-11-06T03:16:43.491242+0000","last_fullsized":"2023-11-06T03:16:43.491242+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:08.411222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489615+0000","last_change":"2023-11-06T03:16:43.486878+0000","last_active":"2023-11-06T03:16:43.489615+0000","last_peered":"2023-11-06T03:16:43.489615+0000","last_clean":"2023-11-06T03:16:43.489615+0000","last_became_active":"2023-11-06T03:16:43.486365+0000","last_became_peered":"2023-11-06T03:16:43.486365+0000","last_unstale":"2023-11-06T03:16:43.489615+0000","last_undegraded":"2023-11-06T03:16:43.489615+0000","last_fullsized":"2023-11-06T03:16:43.489615+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:26.350831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485239+0000","last_change":"2023-11-06T03:16:43.106242+0000","last_active":"2023-11-06T03:16:43.485239+0000","last_peered":"2023-11-06T03:16:43.485239+0000","last_clean":"2023-11-06T03:16:43.485239+0000","last_became_active":"2023-11-06T03:16:43.106002+0000","last_became_peered":"2023-11-06T03:16:43.106002+0000","last_unstale":"2023-11-06T03:16:43.485239+0000","last_undegraded":"2023-11-06T03:16:43.485239+0000","last_fullsized":"2023-11-06T03:16:43.485239+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:49.361616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.473110+0000","last_change":"2023-11-06T03:16:43.471910+0000","last_active":"2023-11-06T03:16:43.473110+0000","last_peered":"2023-11-06T03:16:43.473110+0000","last_clean":"2023-11-06T03:16:43.473110+0000","last_became_active":"2023-11-06T03:16:43.471588+0000","last_became_peered":"2023-11-06T03:16:43.471588+0000","last_unstale":"2023-11-06T03:16:43.473110+0000","last_undegraded":"2023-11-06T03:16:43.473110+0000","last_fullsized":"2023-11-06T03:16:43.473110+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:56.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.464816+0000","last_change":"2023-11-06T03:16:43.093443+0000","last_active":"2023-11-06T03:16:43.464816+0000","last_peered":"2023-11-06T03:16:43.464816+0000","last_clean":"2023-11-06T03:16:43.464816+0000","last_became_active":"2023-11-06T03:16:43.093105+0000","last_became_peered":"2023-11-06T03:16:43.093105+0000","last_unstale":"2023-11-06T03:16:43.464816+0000","last_undegraded":"2023-11-06T03:16:43.464816+0000","last_fullsized":"2023-11-06T03:16:43.464816+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:25.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489538+0000","last_change":"2023-11-06T03:16:43.486962+0000","last_active":"2023-11-06T03:16:43.489538+0000","last_peered":"2023-11-06T03:16:43.489538+0000","last_clean":"2023-11-06T03:16:43.489538+0000","last_became_active":"2023-11-06T03:16:43.486406+0000","last_became_peered":"2023-11-06T03:16:43.486406+0000","last_unstale":"2023-11-06T03:16:43.489538+0000","last_undegraded":"2023-11-06T03:16:43.489538+0000","last_fullsized":"2023-11-06T03:16:43.489538+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:11.932240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480667+0000","last_change":"2023-11-06T03:16:43.100179+0000","last_active":"2023-11-06T03:16:43.480667+0000","last_peered":"2023-11-06T03:16:43.480667+0000","last_clean":"2023-11-06T03:16:43.480667+0000","last_became_active":"2023-11-06T03:16:43.099911+0000","last_became_peered":"2023-11-06T03:16:43.099911+0000","last_unstale":"2023-11-06T03:16:43.480667+0000","last_undegraded":"2023-11-06T03:16:43.480667+0000","last_fullsized":"2023-11-06T03:16:43.480667+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:23.675346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487697+0000","last_change":"2023-11-06T03:16:43.485938+0000","last_active":"2023-11-06T03:16:43.487697+0000","last_peered":"2023-11-06T03:16:43.487697+0000","last_clean":"2023-11-06T03:16:43.487697+0000","last_became_active":"2023-11-06T03:16:43.485583+0000","last_became_peered":"2023-11-06T03:16:43.485583+0000","last_unstale":"2023-11-06T03:16:43.487697+0000","last_undegraded":"2023-11-06T03:16:43.487697+0000","last_fullsized":"2023-11-06T03:16:43.487697+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:53.978884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.462592+0000","last_change":"2023-11-06T03:16:43.099380+0000","last_active":"2023-11-06T03:16:43.462592+0000","last_peered":"2023-11-06T03:16:43.462592+0000","last_clean":"2023-11-06T03:16:43.462592+0000","last_became_active":"2023-11-06T03:16:43.098626+0000","last_became_peered":"2023-11-06T03:16:43.098626+0000","last_unstale":"2023-11-06T03:16:43.462592+0000","last_undegraded":"2023-11-06T03:16:43.462592+0000","last_fullsized":"2023-11-06T03:16:43.462592+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:05.839972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487683+0000","last_change":"2023-11-06T03:16:43.485468+0000","last_active":"2023-11-06T03:16:43.487683+0000","last_peered":"2023-11-06T03:16:43.487683+0000","last_clean":"2023-11-06T03:16:43.487683+0000","last_became_active":"2023-11-06T03:16:43.484815+0000","last_became_peered":"2023-11-06T03:16:43.484815+0000","last_unstale":"2023-11-06T03:16:43.487683+0000","last_undegraded":"2023-11-06T03:16:43.487683+0000","last_fullsized":"2023-11-06T03:16:43.487683+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:22:07.885491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.452940+0000","last_change":"2023-11-06T03:16:43.104549+0000","last_active":"2023-11-06T03:16:43.452940+0000","last_peered":"2023-11-06T03:16:43.452940+0000","last_clean":"2023-11-06T03:16:43.452940+0000","last_became_active":"2023-11-06T03:16:43.104183+0000","last_became_peered":"2023-11-06T03:16:43.104183+0000","last_unstale":"2023-11-06T03:16:43.452940+0000","last_undegraded":"2023-11-06T03:16:43.452940+0000","last_fullsized":"2023-11-06T03:16:43.452940+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:45.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484096+0000","last_change":"2023-11-06T03:16:43.472629+0000","last_active":"2023-11-06T03:16:43.484096+0000","last_peered":"2023-11-06T03:16:43.484096+0000","last_clean":"2023-11-06T03:16:43.484096+0000","last_became_active":"2023-11-06T03:16:43.472360+0000","last_became_peered":"2023-11-06T03:16:43.472360+0000","last_unstale":"2023-11-06T03:16:43.484096+0000","last_undegraded":"2023-11-06T03:16:43.484096+0000","last_fullsized":"2023-11-06T03:16:43.484096+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:36:16.402184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.458001+0000","last_change":"2023-11-06T03:16:43.108530+0000","last_active":"2023-11-06T03:16:43.458001+0000","last_peered":"2023-11-06T03:16:43.458001+0000","last_clean":"2023-11-06T03:16:43.458001+0000","last_became_active":"2023-11-06T03:16:43.107937+0000","last_became_peered":"2023-11-06T03:16:43.107937+0000","last_unstale":"2023-11-06T03:16:43.458001+0000","last_undegraded":"2023-11-06T03:16:43.458001+0000","last_fullsized":"2023-11-06T03:16:43.458001+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:42.521582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.479967+0000","last_change":"2023-11-06T03:16:43.478875+0000","last_active":"2023-11-06T03:16:43.479967+0000","last_peered":"2023-11-06T03:16:43.479967+0000","last_clean":"2023-11-06T03:16:43.479967+0000","last_became_active":"2023-11-06T03:16:43.478523+0000","last_became_peered":"2023-11-06T03:16:43.478523+0000","last_unstale":"2023-11-06T03:16:43.479967+0000","last_undegraded":"2023-11-06T03:16:43.479967+0000","last_fullsized":"2023-11-06T03:16:43.479967+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:23:15.277978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.474068+0000","last_change":"2023-11-06T03:16:43.102037+0000","last_active":"2023-11-06T03:16:43.474068+0000","last_peered":"2023-11-06T03:16:43.474068+0000","last_clean":"2023-11-06T03:16:43.474068+0000","last_became_active":"2023-11-06T03:16:43.101740+0000","last_became_peered":"2023-11-06T03:16:43.101740+0000","last_unstale":"2023-11-06T03:16:43.474068+0000","last_undegraded":"2023-11-06T03:16:43.474068+0000","last_fullsized":"2023-11-06T03:16:43.474068+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:19.215950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491363+0000","last_change":"2023-11-06T03:16:43.482622+0000","last_active":"2023-11-06T03:16:43.491363+0000","last_peered":"2023-11-06T03:16:43.491363+0000","last_clean":"2023-11-06T03:16:43.491363+0000","last_became_active":"2023-11-06T03:16:43.482360+0000","last_became_peered":"2023-11-06T03:16:43.482360+0000","last_unstale":"2023-11-06T03:16:43.491363+0000","last_undegraded":"2023-11-06T03:16:43.491363+0000","last_fullsized":"2023-11-06T03:16:43.491363+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:29:54.461377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.455190+0000","last_change":"2023-11-06T03:16:43.085436+0000","last_active":"2023-11-06T03:16:43.455190+0000","last_peered":"2023-11-06T03:16:43.455190+0000","last_clean":"2023-11-06T03:16:43.455190+0000","last_became_active":"2023-11-06T03:16:43.085137+0000","last_became_peered":"2023-11-06T03:16:43.085137+0000","last_unstale":"2023-11-06T03:16:43.455190+0000","last_undegraded":"2023-11-06T03:16:43.455190+0000","last_fullsized":"2023-11-06T03:16:43.455190+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:41.901469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.479859+0000","last_change":"2023-11-06T03:16:43.475339+0000","last_active":"2023-11-06T03:16:43.479859+0000","last_peered":"2023-11-06T03:16:43.479859+0000","last_clean":"2023-11-06T03:16:43.479859+0000","last_became_active":"2023-11-06T03:16:43.475015+0000","last_became_peered":"2023-11-06T03:16:43.475015+0000","last_unstale":"2023-11-06T03:16:43.479859+0000","last_undegraded":"2023-11-06T03:16:43.479859+0000","last_fullsized":"2023-11-06T03:16:43.479859+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:57.287183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.455280+0000","last_change":"2023-11-06T03:16:43.095767+0000","last_active":"2023-11-06T03:16:43.455280+0000","last_peered":"2023-11-06T03:16:43.455280+0000","last_clean":"2023-11-06T03:16:43.455280+0000","last_became_active":"2023-11-06T03:16:43.095485+0000","last_became_peered":"2023-11-06T03:16:43.095485+0000","last_unstale":"2023-11-06T03:16:43.455280+0000","last_undegraded":"2023-11-06T03:16:43.455280+0000","last_fullsized":"2023-11-06T03:16:43.455280+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:39.788696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493153+0000","last_change":"2023-11-06T03:16:43.485595+0000","last_active":"2023-11-06T03:16:43.493153+0000","last_peered":"2023-11-06T03:16:43.493153+0000","last_clean":"2023-11-06T03:16:43.493153+0000","last_became_active":"2023-11-06T03:16:43.484988+0000","last_became_peered":"2023-11-06T03:16:43.484988+0000","last_unstale":"2023-11-06T03:16:43.493153+0000","last_undegraded":"2023-11-06T03:16:43.493153+0000","last_fullsized":"2023-11-06T03:16:43.493153+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:04.578036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454723+0000","last_change":"2023-11-06T03:16:43.088714+0000","last_active":"2023-11-06T03:16:43.454723+0000","last_peered":"2023-11-06T03:16:43.454723+0000","last_clean":"2023-11-06T03:16:43.454723+0000","last_became_active":"2023-11-06T03:16:43.088150+0000","last_became_peered":"2023-11-06T03:16:43.088150+0000","last_unstale":"2023-11-06T03:16:43.454723+0000","last_undegraded":"2023-11-06T03:16:43.454723+0000","last_fullsized":"2023-11-06T03:16:43.454723+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:39.075788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485712+0000","last_change":"2023-11-06T03:16:43.466416+0000","last_active":"2023-11-06T03:16:43.485712+0000","last_peered":"2023-11-06T03:16:43.485712+0000","last_clean":"2023-11-06T03:16:43.485712+0000","last_became_active":"2023-11-06T03:16:43.466224+0000","last_became_peered":"2023-11-06T03:16:43.466224+0000","last_unstale":"2023-11-06T03:16:43.485712+0000","last_undegraded":"2023-11-06T03:16:43.485712+0000","last_fullsized":"2023-11-06T03:16:43.485712+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:42.904810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.448563+0000","last_change":"2023-11-06T03:16:43.099793+0000","last_active":"2023-11-06T03:16:43.448563+0000","last_peered":"2023-11-06T03:16:43.448563+0000","last_clean":"2023-11-06T03:16:43.448563+0000","last_became_active":"2023-11-06T03:16:43.099058+0000","last_became_peered":"2023-11-06T03:16:43.099058+0000","last_unstale":"2023-11-06T03:16:43.448563+0000","last_undegraded":"2023-11-06T03:16:43.448563+0000","last_fullsized":"2023-11-06T03:16:43.448563+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:34:21.593316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484461+0000","last_change":"2023-11-06T03:16:43.480136+0000","last_active":"2023-11-06T03:16:43.484461+0000","last_peered":"2023-11-06T03:16:43.484461+0000","last_clean":"2023-11-06T03:16:43.484461+0000","last_became_active":"2023-11-06T03:16:43.479739+0000","last_became_peered":"2023-11-06T03:16:43.479739+0000","last_unstale":"2023-11-06T03:16:43.484461+0000","last_undegraded":"2023-11-06T03:16:43.484461+0000","last_fullsized":"2023-11-06T03:16:43.484461+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:20.794448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454716+0000","last_change":"2023-11-06T03:16:43.092098+0000","last_active":"2023-11-06T03:16:43.454716+0000","last_peered":"2023-11-06T03:16:43.454716+0000","last_clean":"2023-11-06T03:16:43.454716+0000","last_became_active":"2023-11-06T03:16:43.091694+0000","last_became_peered":"2023-11-06T03:16:43.091694+0000","last_unstale":"2023-11-06T03:16:43.454716+0000","last_undegraded":"2023-11-06T03:16:43.454716+0000","last_fullsized":"2023-11-06T03:16:43.454716+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:07.076814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489472+0000","last_change":"2023-11-06T03:16:43.485229+0000","last_active":"2023-11-06T03:16:43.489472+0000","last_peered":"2023-11-06T03:16:43.489472+0000","last_clean":"2023-11-06T03:16:43.489472+0000","last_became_active":"2023-11-06T03:16:43.484905+0000","last_became_peered":"2023-11-06T03:16:43.484905+0000","last_unstale":"2023-11-06T03:16:43.489472+0000","last_undegraded":"2023-11-06T03:16:43.489472+0000","last_fullsized":"2023-11-06T03:16:43.489472+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:50.364957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.463233+0000","last_change":"2023-11-06T03:16:43.098520+0000","last_active":"2023-11-06T03:16:43.463233+0000","last_peered":"2023-11-06T03:16:43.463233+0000","last_clean":"2023-11-06T03:16:43.463233+0000","last_became_active":"2023-11-06T03:16:43.097827+0000","last_became_peered":"2023-11-06T03:16:43.097827+0000","last_unstale":"2023-11-06T03:16:43.463233+0000","last_undegraded":"2023-11-06T03:16:43.463233+0000","last_fullsized":"2023-11-06T03:16:43.463233+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:48.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468108+0000","last_change":"2023-11-06T03:16:43.465861+0000","last_active":"2023-11-06T03:16:43.468108+0000","last_peered":"2023-11-06T03:16:43.468108+0000","last_clean":"2023-11-06T03:16:43.468108+0000","last_became_active":"2023-11-06T03:16:43.465512+0000","last_became_peered":"2023-11-06T03:16:43.465512+0000","last_unstale":"2023-11-06T03:16:43.468108+0000","last_undegraded":"2023-11-06T03:16:43.468108+0000","last_fullsized":"2023-11-06T03:16:43.468108+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:42.932955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.446641+0000","last_change":"2023-11-06T03:16:43.103658+0000","last_active":"2023-11-06T03:16:43.446641+0000","last_peered":"2023-11-06T03:16:43.446641+0000","last_clean":"2023-11-06T03:16:43.446641+0000","last_became_active":"2023-11-06T03:16:43.103299+0000","last_became_peered":"2023-11-06T03:16:43.103299+0000","last_unstale":"2023-11-06T03:16:43.446641+0000","last_undegraded":"2023-11-06T03:16:43.446641+0000","last_fullsized":"2023-11-06T03:16:43.446641+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:21:33.477065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484308+0000","last_change":"2023-11-06T03:16:43.481220+0000","last_active":"2023-11-06T03:16:43.484308+0000","last_peered":"2023-11-06T03:16:43.484308+0000","last_clean":"2023-11-06T03:16:43.484308+0000","last_became_active":"2023-11-06T03:16:43.480896+0000","last_became_peered":"2023-11-06T03:16:43.480896+0000","last_unstale":"2023-11-06T03:16:43.484308+0000","last_undegraded":"2023-11-06T03:16:43.484308+0000","last_fullsized":"2023-11-06T03:16:43.484308+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:05.877838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454808+0000","last_change":"2023-11-06T03:16:43.085834+0000","last_active":"2023-11-06T03:16:43.454808+0000","last_peered":"2023-11-06T03:16:43.454808+0000","last_clean":"2023-11-06T03:16:43.454808+0000","last_became_active":"2023-11-06T03:16:43.082598+0000","last_became_peered":"2023-11-06T03:16:43.082598+0000","last_unstale":"2023-11-06T03:16:43.454808+0000","last_undegraded":"2023-11-06T03:16:43.454808+0000","last_fullsized":"2023-11-06T03:16:43.454808+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:43.137932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097986+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:16:43.097986+0000","last_peered":"2023-11-06T03:16:43.097986+0000","last_clean":"2023-11-06T03:16:43.097986+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:16:43.097986+0000","last_undegraded":"2023-11-06T03:16:43.097986+0000","last_fullsized":"2023-11-06T03:16:43.097986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.090803+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:16:43.090803+0000","last_peered":"2023-11-06T03:16:43.090803+0000","last_clean":"2023-11-06T03:16:43.090803+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:16:43.090803+0000","last_undegraded":"2023-11-06T03:16:43.090803+0000","last_fullsized":"2023-11-06T03:16:43.090803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489445+0000","last_change":"2023-11-06T03:16:43.484652+0000","last_active":"2023-11-06T03:16:43.489445+0000","last_peered":"2023-11-06T03:16:43.489445+0000","last_clean":"2023-11-06T03:16:43.489445+0000","last_became_active":"2023-11-06T03:16:43.484314+0000","last_became_peered":"2023-11-06T03:16:43.484314+0000","last_unstale":"2023-11-06T03:16:43.489445+0000","last_undegraded":"2023-11-06T03:16:43.489445+0000","last_fullsized":"2023-11-06T03:16:43.489445+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:08.123116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"162'3","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.461827+0000","last_change":"2023-11-06T03:16:43.105567+0000","last_active":"2023-11-06T03:16:43.461827+0000","last_peered":"2023-11-06T03:16:43.461827+0000","last_clean":"2023-11-06T03:16:43.461827+0000","last_became_active":"2023-11-06T03:16:43.104944+0000","last_became_peered":"2023-11-06T03:16:43.104944+0000","last_unstale":"2023-11-06T03:16:43.461827+0000","last_undegraded":"2023-11-06T03:16:43.461827+0000","last_fullsized":"2023-11-06T03:16:43.461827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:40.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097756+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:16:43.097756+0000","last_peered":"2023-11-06T03:16:43.097756+0000","last_clean":"2023-11-06T03:16:43.097756+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:16:43.097756+0000","last_undegraded":"2023-11-06T03:16:43.097756+0000","last_fullsized":"2023-11-06T03:16:43.097756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468006+0000","last_change":"2023-11-06T03:16:43.464990+0000","last_active":"2023-11-06T03:16:43.468006+0000","last_peered":"2023-11-06T03:16:43.468006+0000","last_clean":"2023-11-06T03:16:43.468006+0000","last_became_active":"2023-11-06T03:16:43.464706+0000","last_became_peered":"2023-11-06T03:16:43.464706+0000","last_unstale":"2023-11-06T03:16:43.468006+0000","last_undegraded":"2023-11-06T03:16:43.468006+0000","last_fullsized":"2023-11-06T03:16:43.468006+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:20.346917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"162'6","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493602+0000","last_change":"2023-11-06T03:16:43.086521+0000","last_active":"2023-11-06T03:16:43.493602+0000","last_peered":"2023-11-06T03:16:43.493602+0000","last_clean":"2023-11-06T03:16:43.493602+0000","last_became_active":"2023-11-06T03:16:43.082586+0000","last_became_peered":"2023-11-06T03:16:43.082586+0000","last_unstale":"2023-11-06T03:16:43.493602+0000","last_undegraded":"2023-11-06T03:16:43.493602+0000","last_fullsized":"2023-11-06T03:16:43.493602+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:22.700873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.074584+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:16:43.074584+0000","last_peered":"2023-11-06T03:16:43.074584+0000","last_clean":"2023-11-06T03:16:43.074584+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:16:43.074584+0000","last_undegraded":"2023-11-06T03:16:43.074584+0000","last_fullsized":"2023-11-06T03:16:43.074584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.486809+0000","last_change":"2023-11-06T03:16:43.481383+0000","last_active":"2023-11-06T03:16:43.486809+0000","last_peered":"2023-11-06T03:16:43.486809+0000","last_clean":"2023-11-06T03:16:43.486809+0000","last_became_active":"2023-11-06T03:16:43.479960+0000","last_became_peered":"2023-11-06T03:16:43.479960+0000","last_unstale":"2023-11-06T03:16:43.486809+0000","last_undegraded":"2023-11-06T03:16:43.486809+0000","last_fullsized":"2023-11-06T03:16:43.486809+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459627+0000","last_change":"2023-11-06T03:16:43.082428+0000","last_active":"2023-11-06T03:16:43.459627+0000","last_peered":"2023-11-06T03:16:43.459627+0000","last_clean":"2023-11-06T03:16:43.459627+0000","last_became_active":"2023-11-06T03:16:43.081818+0000","last_became_peered":"2023-11-06T03:16:43.081818+0000","last_unstale":"2023-11-06T03:16:43.459627+0000","last_undegraded":"2023-11-06T03:16:43.459627+0000","last_fullsized":"2023-11-06T03:16:43.459627+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:04.874498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.090485+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:16:43.090485+0000","last_peered":"2023-11-06T03:16:43.090485+0000","last_clean":"2023-11-06T03:16:43.090485+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:16:43.090485+0000","last_undegraded":"2023-11-06T03:16:43.090485+0000","last_fullsized":"2023-11-06T03:16:43.090485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483745+0000","last_change":"2023-11-06T03:16:43.482676+0000","last_active":"2023-11-06T03:16:43.483745+0000","last_peered":"2023-11-06T03:16:43.483745+0000","last_clean":"2023-11-06T03:16:43.483745+0000","last_became_active":"2023-11-06T03:16:43.482379+0000","last_became_peered":"2023-11-06T03:16:43.482379+0000","last_unstale":"2023-11-06T03:16:43.483745+0000","last_undegraded":"2023-11-06T03:16:43.483745+0000","last_fullsized":"2023-11-06T03:16:43.483745+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:05.234569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.463067+0000","last_change":"2023-11-06T03:16:43.103738+0000","last_active":"2023-11-06T03:16:43.463067+0000","last_peered":"2023-11-06T03:16:43.463067+0000","last_clean":"2023-11-06T03:16:43.463067+0000","last_became_active":"2023-11-06T03:16:43.103042+0000","last_became_peered":"2023-11-06T03:16:43.103042+0000","last_unstale":"2023-11-06T03:16:43.463067+0000","last_undegraded":"2023-11-06T03:16:43.463067+0000","last_fullsized":"2023-11-06T03:16:43.463067+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:14.765834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.075588+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:16:43.075588+0000","last_peered":"2023-11-06T03:16:43.075588+0000","last_clean":"2023-11-06T03:16:43.075588+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:16:43.075588+0000","last_undegraded":"2023-11-06T03:16:43.075588+0000","last_fullsized":"2023-11-06T03:16:43.075588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468085+0000","last_change":"2023-11-06T03:16:43.465362+0000","last_active":"2023-11-06T03:16:43.468085+0000","last_peered":"2023-11-06T03:16:43.468085+0000","last_clean":"2023-11-06T03:16:43.468085+0000","last_became_active":"2023-11-06T03:16:43.463856+0000","last_became_peered":"2023-11-06T03:16:43.463856+0000","last_unstale":"2023-11-06T03:16:43.468085+0000","last_undegraded":"2023-11-06T03:16:43.468085+0000","last_fullsized":"2023-11-06T03:16:43.468085+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:13:49.045571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"162'8","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491509+0000","last_change":"2023-11-06T03:16:43.100740+0000","last_active":"2023-11-06T03:16:43.491509+0000","last_peered":"2023-11-06T03:16:43.491509+0000","last_clean":"2023-11-06T03:16:43.491509+0000","last_became_active":"2023-11-06T03:16:43.100459+0000","last_became_peered":"2023-11-06T03:16:43.100459+0000","last_unstale":"2023-11-06T03:16:43.491509+0000","last_undegraded":"2023-11-06T03:16:43.491509+0000","last_fullsized":"2023-11-06T03:16:43.491509+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:50.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.099887+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:16:43.099887+0000","last_peered":"2023-11-06T03:16:43.099887+0000","last_clean":"2023-11-06T03:16:43.099887+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:16:43.099887+0000","last_undegraded":"2023-11-06T03:16:43.099887+0000","last_fullsized":"2023-11-06T03:16:43.099887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485209+0000","last_change":"2023-11-06T03:16:43.483997+0000","last_active":"2023-11-06T03:16:43.485209+0000","last_peered":"2023-11-06T03:16:43.485209+0000","last_clean":"2023-11-06T03:16:43.485209+0000","last_became_active":"2023-11-06T03:16:43.483497+0000","last_became_peered":"2023-11-06T03:16:43.483497+0000","last_unstale":"2023-11-06T03:16:43.485209+0000","last_undegraded":"2023-11-06T03:16:43.485209+0000","last_fullsized":"2023-11-06T03:16:43.485209+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:19.842451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.469650+0000","last_change":"2023-11-06T03:16:43.103535+0000","last_active":"2023-11-06T03:16:43.469650+0000","last_peered":"2023-11-06T03:16:43.469650+0000","last_clean":"2023-11-06T03:16:43.469650+0000","last_became_active":"2023-11-06T03:16:43.103216+0000","last_became_peered":"2023-11-06T03:16:43.103216+0000","last_unstale":"2023-11-06T03:16:43.469650+0000","last_undegraded":"2023-11-06T03:16:43.469650+0000","last_fullsized":"2023-11-06T03:16:43.469650+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:07.119775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.076953+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:16:43.076953+0000","last_peered":"2023-11-06T03:16:43.076953+0000","last_clean":"2023-11-06T03:16:43.076953+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:16:43.076953+0000","last_undegraded":"2023-11-06T03:16:43.076953+0000","last_fullsized":"2023-11-06T03:16:43.076953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493114+0000","last_change":"2023-11-06T03:16:43.484970+0000","last_active":"2023-11-06T03:16:43.493114+0000","last_peered":"2023-11-06T03:16:43.493114+0000","last_clean":"2023-11-06T03:16:43.493114+0000","last_became_active":"2023-11-06T03:16:43.484301+0000","last_became_peered":"2023-11-06T03:16:43.484301+0000","last_unstale":"2023-11-06T03:16:43.493114+0000","last_undegraded":"2023-11-06T03:16:43.493114+0000","last_fullsized":"2023-11-06T03:16:43.493114+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:25.861365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454607+0000","last_change":"2023-11-06T03:16:43.094062+0000","last_active":"2023-11-06T03:16:43.454607+0000","last_peered":"2023-11-06T03:16:43.454607+0000","last_clean":"2023-11-06T03:16:43.454607+0000","last_became_active":"2023-11-06T03:16:43.093750+0000","last_became_peered":"2023-11-06T03:16:43.093750+0000","last_unstale":"2023-11-06T03:16:43.454607+0000","last_undegraded":"2023-11-06T03:16:43.454607+0000","last_fullsized":"2023-11-06T03:16:43.454607+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:47.002128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097221+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:16:43.097221+0000","last_peered":"2023-11-06T03:16:43.097221+0000","last_clean":"2023-11-06T03:16:43.097221+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:16:43.097221+0000","last_undegraded":"2023-11-06T03:16:43.097221+0000","last_fullsized":"2023-11-06T03:16:43.097221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91776,"kb_used_data":2552,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280064,"statfs":{"total":96636764160,"available":96542785536,"internally_reserved":0,"allocated":2613248,"data_stored":1526339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.81000000000000005,"15min":0.80800000000000005},"min":{"1min":0.61099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.82899999999999996},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.397},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.83399999999999996},"min":{"1min":0.501,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.1180000000000001,"5min":1.627,"15min":1.627},"last":0.62},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.137,"5min":1.218,"15min":1.6899999999999999},"last":0.92100000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.873,"15min":0.88300000000000001},"min":{"1min":0.628,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.6080000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.872,"15min":0.90300000000000002},"min":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.48999999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85999999999999999,"15min":0.88600000000000001},"min":{"1min":0.58599999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.103,"5min":1.4099999999999999,"15min":2.512},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95399999999999996,"15min":1.034},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.319,"5min":1.431,"15min":10.596},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93700000000000006,"15min":0.95599999999999996},"min":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.163,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.877},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.95299999999999996,"15min":0.98099999999999998},"min":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.304,"5min":1.54,"15min":1.9399999999999999},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.042},"min":{"1min":0.71199999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0740000000000001},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2929999999999999,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.179,"15min":1.159},"min":{"1min":0.82499999999999996,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":1.3460000000000001,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":0.75},{"interface":"front","average":{"1min":1.016,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.246,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":0.94699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69060,"kb_used_data":1980,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302780,"statfs":{"total":96636764160,"available":96566046720,"internally_reserved":0,"allocated":2027520,"data_stored":942080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.86099999999999999},"min":{"1min":0.50600000000000001,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.1830000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83999999999999997,"15min":0.98099999999999998},"min":{"1min":0.56100000000000005,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.581,"5min":2.1850000000000001,"15min":40.905000000000001},"last":0.90000000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.90800000000000003},"min":{"1min":0.53300000000000003,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.1719999999999999,"5min":1.351,"15min":16.012},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.873,"15min":0.88},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.3400000000000001,"5min":1.409,"15min":3.3050000000000002},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.175,"5min":1.702,"15min":27.803999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.248,"5min":1.429,"15min":2.843},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93899999999999995,"15min":1.1759999999999999},"min":{"1min":0.64700000000000002,"5min":0.61699999999999999,"15min":0.48699999999999999},"max":{"1min":1.204,"5min":1.306,"15min":63.737000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":1.0600000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.1399999999999999,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":1.151},"min":{"1min":0.70699999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2330000000000001,"5min":1.512,"15min":24.867999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1100000000000001,"15min":1.2250000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.65800000000000003},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.97899999999999998,"15min":1.1459999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.206,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0820000000000001,"15min":1.115},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3200000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":0.86099999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.131,"15min":1.2130000000000001},"min":{"1min":0.79700000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5309999999999999,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.137,"15min":1.2230000000000001},"min":{"1min":0.85999999999999999,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.331,"5min":5.2709999999999999,"15min":28.164000000000001},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46992,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324848,"statfs":{"total":96636764160,"available":96588644352,"internally_reserved":0,"allocated":2039808,"data_stored":943223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":1.133},"min":{"1min":0.58499999999999996,"5min":0.309,"15min":0.309},"max":{"1min":1.123,"5min":4.0510000000000002,"15min":125.256},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.127,"15min":1.2629999999999999},"min":{"1min":0.52400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.7050000000000001,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.91400000000000003,"15min":0.83299999999999996},"min":{"1min":0.65900000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.306,"15min":0.998},"min":{"1min":0.60699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.6459999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.3400000000000001,"15min":1.0149999999999999},"min":{"1min":0.60999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3959999999999999,"5min":43.741,"15min":43.741},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.86299999999999999,"15min":0.84899999999999998},"min":{"1min":0.60499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2470000000000001,"5min":1.385,"15min":2.9700000000000002},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.96299999999999997,"15min":0.90300000000000002},"min":{"1min":0.629,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":4.7800000000000002,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.4370000000000001,"15min":1.111},"min":{"1min":0.57599999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.796,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.107}]},{"osd":4,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.90500000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.595,"5min":1.595,"15min":3.1349999999999998},"last":1.333},{"interface":"front","average":{"1min":1.163,"5min":1.1200000000000001,"15min":1.0269999999999999},"min":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":4.6200000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.393}]},{"osd":6,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0409999999999999,"15min":1.026},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.72,"5min":2,"15min":2.6909999999999998},"last":1.365},{"interface":"front","average":{"1min":1.044,"5min":1.077,"15min":1.0469999999999999},"min":{"1min":0.69099999999999995,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3979999999999999,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.413}]},{"osd":7,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.177,"15min":1.0780000000000001},"min":{"1min":0.59099999999999997,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":4.9160000000000004,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.262},{"interface":"front","average":{"1min":1.248,"5min":1.1719999999999999,"15min":1.121},"min":{"1min":0.65300000000000002,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":1.45}]}]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113340,"kb_used_data":1972,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258500,"statfs":{"total":96636764160,"available":96520704000,"internally_reserved":0,"allocated":2019328,"data_stored":935971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79500000000000004,"15min":0.95499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.258,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.79600000000000004,"15min":0.96099999999999997},"min":{"1min":0.505,"5min":0.308,"15min":0.308},"max":{"1min":1.4750000000000001,"5min":3.0830000000000002,"15min":44.609000000000002},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.77400000000000002},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2350000000000001,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3109999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.90000000000000002,"15min":0.94099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2410000000000001,"5min":1.452,"15min":3.2879999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.90300000000000002,"15min":0.86699999999999999},"min":{"1min":0.65100000000000002,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.27,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.91700000000000004},"min":{"1min":0.61899999999999999,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.3660000000000001,"5min":3.198,"15min":3.198},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.90900000000000003,"15min":0.90100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4510000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2210000000000001,"5min":10.192,"15min":10.192},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.0780000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.494,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6140000000000001,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0840000000000001,"15min":1.099},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.92500000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.095,"15min":1.0820000000000001},"min":{"1min":0.72599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6000000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1140000000000001,"15min":1.1659999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6819999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.0389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2048000,"data_stored":944248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.27400000000000002,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78100000000000003,"5min":0.96799999999999997,"15min":1.4199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":1.524},"last":0.33800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.251,"15min":0.224},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.371},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.089,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":1.052,"5min":1.421,"15min":1.532},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.0109999999999999,"5min":1.206,"15min":1.284},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82499999999999996,"15min":0.86499999999999999},"min":{"1min":0.65500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.034,"5min":1.2609999999999999,"15min":1.7989999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.878,"15min":0.91200000000000003},"min":{"1min":0.628,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.218,"5min":1.4610000000000001,"15min":1.8300000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.95599999999999996},"min":{"1min":0.68100000000000005,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.4430000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"min":{"1min":0.61499999999999999,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.573,"15min":1.698},"last":0.83399999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94599999999999995,"15min":0.97399999999999998},"min":{"1min":0.70599999999999996,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.367,"5min":1.51,"15min":1.8620000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.92200000000000004,"15min":0.95599999999999996},"min":{"1min":0.623,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.3140000000000001,"5min":1.3660000000000001,"15min":1.8740000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":1.006},"min":{"1min":0.71899999999999997,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.387,"5min":1.4410000000000001,"15min":1.835},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0029999999999999,"15min":1.03},"min":{"1min":0.75600000000000001,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.3600000000000001,"5min":1.569,"15min":1.8979999999999999},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73160,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298680,"statfs":{"total":96636764160,"available":96561848320,"internally_reserved":0,"allocated":2031616,"data_stored":943099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.53300000000000003,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.48399999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.28499999999999998,"15min":0.23000000000000001},"max":{"1min":1.002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.62},"min":{"1min":0.27800000000000002,"5min":0.245,"15min":0.22800000000000001},"max":{"1min":0.89400000000000002,"5min":1.0509999999999999,"15min":1.1579999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63900000000000001,"15min":0.65700000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.121,"5min":1.1459999999999999,"15min":1.294},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.626,"15min":0.68500000000000005},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.107,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.81299999999999994,"15min":0.84799999999999998},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.0840000000000001,"5min":1.2010000000000001,"15min":1.526},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.85799999999999998,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.1930000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.87}]},{"osd":5,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83199999999999996,"15min":0.84999999999999998},"min":{"1min":0.58699999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.1699999999999999,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.86599999999999999,"15min":0.91500000000000004},"min":{"1min":0.64200000000000002,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.298,"5min":1.355,"15min":1.5780000000000001},"last":1.032}]},{"osd":6,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.93300000000000005,"15min":0.96599999999999997},"min":{"1min":0.73799999999999999,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.2689999999999999,"5min":1.542,"15min":1.615},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.22,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.752}]},{"osd":7,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.90400000000000003,"15min":0.95899999999999996},"min":{"1min":0.73499999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3380000000000001,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.89400000000000002,"15min":0.92300000000000004},"min":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1699999999999999,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69076,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302764,"statfs":{"total":96636764160,"available":96566030336,"internally_reserved":0,"allocated":2043904,"data_stored":944663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.90200000000000002,"5min":1.1240000000000001,"15min":1.843},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.59099999999999997},"min":{"1min":0.28299999999999997,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.90100000000000002,"5min":1.244,"15min":2.1150000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56399999999999995,"15min":0.69499999999999995},"min":{"1min":0.30399999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.91000000000000003,"5min":1.143,"15min":30.861000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.61599999999999999,"15min":0.64200000000000002},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":1.117,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.32200000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":1.016},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.284,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":2.258},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.83699999999999997,"15min":0.92200000000000004},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":14.753},"last":1.135}]},{"osd":5,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.876,"15min":1.0780000000000001},"min":{"1min":0.74299999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2649999999999999,"5min":1.351,"15min":43.103999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.96299999999999997,"15min":1.115},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.468,"5min":1.468,"15min":44.753999999999998},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.91200000000000003,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":23.971},"last":1.2789999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.90300000000000002,"15min":1.034},"min":{"1min":0.53500000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3979999999999999,"5min":1.444,"15min":14.813000000000001},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.55,"5min":1.581,"15min":2.2799999999999998},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.97299999999999998,"15min":1.0089999999999999},"min":{"1min":0.77400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4059999999999999,"5min":1.835,"15min":2.3559999999999999},"last":1.347}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95952,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275888,"statfs":{"total":96636764160,"available":96538509312,"internally_reserved":0,"allocated":2629632,"data_stored":1532552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.214,"15min":0.19600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.437},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.61599999999999999,"15min":0.64700000000000002},"min":{"1min":0.44400000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.54700000000000004,"5min":0.24199999999999999,"15min":0.20599999999999999},"max":{"1min":1.014,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.52100000000000002,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.083,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.41199999999999998,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.69899999999999995,"15min":0.72099999999999997},"min":{"1min":0.54800000000000004,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.2829999999999999,"5min":1.391,"15min":2.7999999999999998},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.85399999999999998,"15min":0.90300000000000002},"min":{"1min":0.67900000000000005,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.1870000000000001,"5min":1.371,"15min":4.641},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.79000000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.028},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.91200000000000003,"15min":0.95099999999999996},"min":{"1min":0.77300000000000002,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.492,"5min":1.492,"15min":2.8559999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.70699999999999996,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.377,"5min":1.496,"15min":4.7430000000000003},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.84199999999999997,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":2.931},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.94999999999999996,"15min":0.98899999999999999},"min":{"1min":0.77800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.381,"5min":1.381,"15min":4.8339999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.089,"5min":0.94399999999999995,"15min":0.97399999999999998},"min":{"1min":0.78200000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3260000000000001,"5min":1.335,"15min":1.97},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:16:46.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:16:46.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 Processor -- start 2023-11-06T03:16:46.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 -- start start 2023-11-06T03:16:46.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f311c15ce80 0x7f311c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0d2920 con 0x7f311c0d00b0 2023-11-06T03:16:46.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0d18a0 con 0x7f311c15caa0 2023-11-06T03:16:46.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3122be7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0d0520 con 0x7f311c15ce80 2023-11-06T03:16:46.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f311c0d00b0 0x7f311c0b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45098/0 (socket says 172.21.15.157:45098) 2023-11-06T03:16:46.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 -- 172.21.15.157:0/1873463292 learned_addr learned my addr 172.21.15.157:0/1873463292 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:46.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311bfff640 1 --2- 172.21.15.157:0/1873463292 >> v2:172.21.15.195:3300/0 conn(0x7f311c15ce80 0x7f311c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419299817 0 0) 0x7f311c0d2920 con 0x7f311c0d00b0 2023-11-06T03:16:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3104003530 con 0x7f311c0d00b0 2023-11-06T03:16:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890689790 0 0) 0x7f3104003530 con 0x7f311c0d00b0 2023-11-06T03:16:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 >> v2:172.21.15.195:3300/0 conn(0x7f311c15ce80 msgr2=0x7f311c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:46.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 --2- 172.21.15.157:0/1873463292 >> v2:172.21.15.195:3300/0 conn(0x7f311c15ce80 0x7f311c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 >> v1:172.21.15.195:6789/0 conn(0x7f311c15caa0 legacy=0x7f311c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0d1b60 con 0x7f311c0d00b0 2023-11-06T03:16:46.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f3114001a00 con 0x7f311c0d00b0 2023-11-06T03:16:46.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 >> v1:172.21.15.157:6789/0 conn(0x7f311c0d00b0 legacy=0x7f311c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104003980 0x7f3104003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31040047d0 0x7f3104004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 0x7f310400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0d0520 con 0x7f3104005190 2023-11-06T03:16:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310400c030 con 0x7f3104003980 2023-11-06T03:16:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310400c1f0 con 0x7f31040047d0 2023-11-06T03:16:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 0x7f310400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3120983640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104003980 0x7f3104003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 -- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31040047d0 msgr2=0x7f3104004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f311bfff640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31040047d0 0x7f3104004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31040047d0 0x7f3104004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 -- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104003980 msgr2=0x7f3104003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:46.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104003980 0x7f3104003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3121184640 1 -- 172.21.15.157:0/1873463292 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104003530 con 0x7f3104005190 2023-11-06T03:16:46.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.639+0000 7f3120983640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104003980 0x7f3104003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:46.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3121184640 1 -- 172.21.15.157:0/1873463292 reap_dead start 2023-11-06T03:16:46.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3121184640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 0x7f310400b990 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f311400a830 tx=0x7f311401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9641fe5334728bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140072d0 con 0x7f3104005190 2023-11-06T03:16:46.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31140a8070 con 0x7f3104005190 2023-11-06T03:16:46.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311b7fe640 1 -- 172.21.15.157:0/1873463292 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140076b0 con 0x7f3104005190 2023-11-06T03:16:46.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 msgr2=0x7f310400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 0x7f310400b990 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f311400a830 tx=0x7f311401d340 comp rx=0 tx=0).stop 2023-11-06T03:16:46.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- 172.21.15.157:0/1873463292 shutdown_connections 2023-11-06T03:16:46.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 --2- 172.21.15.157:0/1873463292 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3104005190 0x7f310400b990 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- 172.21.15.157:0/1873463292 >> 172.21.15.157:0/1873463292 conn(0x7f311c070790 msgr2=0x7f311c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:46.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- 172.21.15.157:0/1873463292 shutdown_connections 2023-11-06T03:16:46.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- 172.21.15.157:0/1873463292 wait complete. 2023-11-06T03:16:46.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 Processor -- start 2023-11-06T03:16:46.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- start start 2023-11-06T03:16:46.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104005190 0x7f311c1e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 0x7f311c1e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1e25b0 con 0x7f311c15ad60 2023-11-06T03:16:46.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1e2890 con 0x7f3104005190 2023-11-06T03:16:46.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3122be7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1e2b70 con 0x7f311c15a540 2023-11-06T03:16:46.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38868/0 (socket says 172.21.15.157:38868) 2023-11-06T03:16:46.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f311bfff640 1 -- 172.21.15.157:0/2325704092 learned_addr learned my addr 172.21.15.157:0/2325704092 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:46.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3121184640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 0x7f311c1e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.643+0000 7f3120983640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104005190 0x7f311c1e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 -- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 msgr2=0x7f311c1e46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 0x7f311c1e46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 -- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104005190 msgr2=0x7f311c1e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104005190 0x7f311c1e1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 -- 172.21.15.157:0/2325704092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c1e2e50 con 0x7f311c15ad60 2023-11-06T03:16:46.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3121184640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 0x7f311c1e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:46.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f311bfff640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3110010820 tx=0x7f3110010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31100095d0 con 0x7f311c15ad60 2023-11-06T03:16:46.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311000eb60 con 0x7f311c15ad60 2023-11-06T03:16:46.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c1e3130 con 0x7f311c15ad60 2023-11-06T03:16:46.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311001c400 con 0x7f311c15ad60 2023-11-06T03:16:46.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c15a9f0 con 0x7f311c15ad60 2023-11-06T03:16:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c072750 con 0x7f311c15ad60 2023-11-06T03:16:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f311001c560 con 0x7f311c15ad60 2023-11-06T03:16:46.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 0x7f3108067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3110083820 con 0x7f311c15ad60 2023-11-06T03:16:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3120983640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 0x7f3108067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.647+0000 7f3120983640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 0x7f3108067f50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f310c0014a0 tx=0x7f310c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:46.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.651+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3110051130 con 0x7f311c15ad60 2023-11-06T03:16:46.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.803+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f311c0d1e40 con 0x7f3108065a90 2023-11-06T03:16:46.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.827+0000 7f31197fa640 1 -- 172.21.15.157:0/2325704092 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310932 (secure 0 0 0) 0x7f310c00d050 con 0x7f3108065a90 2023-11-06T03:16:46.826 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:46.827 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:16:46.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 msgr2=0x7f3108067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 0x7f3108067f50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f310c0014a0 tx=0x7f310c009040 comp rx=0 tx=0).stop 2023-11-06T03:16:46.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 msgr2=0x7f311c1e3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:46.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3110010820 tx=0x7f3110010ce0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 shutdown_connections 2023-11-06T03:16:46.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3108065a90 0x7f3108067f50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f311c15a540 0x7f311c1e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f311c15ad60 0x7f311c1e3650 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 --2- 172.21.15.157:0/2325704092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3104005190 0x7f311c1e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:46.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 >> 172.21.15.157:0/2325704092 conn(0x7f311c070790 msgr2=0x7f311c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:46.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 shutdown_connections 2023-11-06T03:16:46.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:46.831+0000 7f3122be7640 1 -- 172.21.15.157:0/2325704092 wait complete. 2023-11-06T03:16:46.845 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2023-11-06T03:16:45.370681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":642452,"kb_used_data":17044,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754332268,"statfs":{"total":773094113280,"available":772436242432,"internally_reserved":0,"allocated":17453056,"data_stored":8712175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001301"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490301+0000","last_change":"2023-11-06T03:16:43.476040+0000","last_active":"2023-11-06T03:16:43.490301+0000","last_peered":"2023-11-06T03:16:43.490301+0000","last_clean":"2023-11-06T03:16:43.490301+0000","last_became_active":"2023-11-06T03:16:43.475488+0000","last_became_peered":"2023-11-06T03:16:43.475488+0000","last_unstale":"2023-11-06T03:16:43.490301+0000","last_undegraded":"2023-11-06T03:16:43.490301+0000","last_fullsized":"2023-11-06T03:16:43.490301+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:08.080154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481857+0000","last_change":"2023-11-06T03:16:43.471406+0000","last_active":"2023-11-06T03:16:43.481857+0000","last_peered":"2023-11-06T03:16:43.481857+0000","last_clean":"2023-11-06T03:16:43.481857+0000","last_became_active":"2023-11-06T03:16:43.471033+0000","last_became_peered":"2023-11-06T03:16:43.471033+0000","last_unstale":"2023-11-06T03:16:43.481857+0000","last_undegraded":"2023-11-06T03:16:43.481857+0000","last_fullsized":"2023-11-06T03:16:43.481857+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:36:04.624576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485753+0000","last_change":"2023-11-06T03:16:43.466824+0000","last_active":"2023-11-06T03:16:43.485753+0000","last_peered":"2023-11-06T03:16:43.485753+0000","last_clean":"2023-11-06T03:16:43.485753+0000","last_became_active":"2023-11-06T03:16:43.466369+0000","last_became_peered":"2023-11-06T03:16:43.466369+0000","last_unstale":"2023-11-06T03:16:43.485753+0000","last_undegraded":"2023-11-06T03:16:43.485753+0000","last_fullsized":"2023-11-06T03:16:43.485753+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:23.704213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487913+0000","last_change":"2023-11-06T03:16:43.475776+0000","last_active":"2023-11-06T03:16:43.487913+0000","last_peered":"2023-11-06T03:16:43.487913+0000","last_clean":"2023-11-06T03:16:43.487913+0000","last_became_active":"2023-11-06T03:16:43.475484+0000","last_became_peered":"2023-11-06T03:16:43.475484+0000","last_unstale":"2023-11-06T03:16:43.487913+0000","last_undegraded":"2023-11-06T03:16:43.487913+0000","last_fullsized":"2023-11-06T03:16:43.487913+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:20:47.537838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459749+0000","last_change":"2023-11-06T03:16:43.457255+0000","last_active":"2023-11-06T03:16:43.459749+0000","last_peered":"2023-11-06T03:16:43.459749+0000","last_clean":"2023-11-06T03:16:43.459749+0000","last_became_active":"2023-11-06T03:16:43.456893+0000","last_became_peered":"2023-11-06T03:16:43.456893+0000","last_unstale":"2023-11-06T03:16:43.459749+0000","last_undegraded":"2023-11-06T03:16:43.459749+0000","last_fullsized":"2023-11-06T03:16:43.459749+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:03:05.620215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490487+0000","last_change":"2023-11-06T03:16:43.486781+0000","last_active":"2023-11-06T03:16:43.490487+0000","last_peered":"2023-11-06T03:16:43.490487+0000","last_clean":"2023-11-06T03:16:43.490487+0000","last_became_active":"2023-11-06T03:16:43.486367+0000","last_became_peered":"2023-11-06T03:16:43.486367+0000","last_unstale":"2023-11-06T03:16:43.490487+0000","last_undegraded":"2023-11-06T03:16:43.490487+0000","last_fullsized":"2023-11-06T03:16:43.490487+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:42.904810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491390+0000","last_change":"2023-11-06T03:16:43.483187+0000","last_active":"2023-11-06T03:16:43.491390+0000","last_peered":"2023-11-06T03:16:43.491390+0000","last_clean":"2023-11-06T03:16:43.491390+0000","last_became_active":"2023-11-06T03:16:43.482698+0000","last_became_peered":"2023-11-06T03:16:43.482698+0000","last_unstale":"2023-11-06T03:16:43.491390+0000","last_undegraded":"2023-11-06T03:16:43.491390+0000","last_fullsized":"2023-11-06T03:16:43.491390+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:10.825573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493416+0000","last_change":"2023-11-06T03:16:43.490867+0000","last_active":"2023-11-06T03:16:43.493416+0000","last_peered":"2023-11-06T03:16:43.493416+0000","last_clean":"2023-11-06T03:16:43.493416+0000","last_became_active":"2023-11-06T03:16:43.487629+0000","last_became_peered":"2023-11-06T03:16:43.487629+0000","last_unstale":"2023-11-06T03:16:43.493416+0000","last_undegraded":"2023-11-06T03:16:43.493416+0000","last_fullsized":"2023-11-06T03:16:43.493416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:45:18.321258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491382+0000","last_change":"2023-11-06T03:16:43.474722+0000","last_active":"2023-11-06T03:16:43.491382+0000","last_peered":"2023-11-06T03:16:43.491382+0000","last_clean":"2023-11-06T03:16:43.491382+0000","last_became_active":"2023-11-06T03:16:43.474374+0000","last_became_peered":"2023-11-06T03:16:43.474374+0000","last_unstale":"2023-11-06T03:16:43.491382+0000","last_undegraded":"2023-11-06T03:16:43.491382+0000","last_fullsized":"2023-11-06T03:16:43.491382+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:38.206951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493401+0000","last_change":"2023-11-06T03:16:43.490825+0000","last_active":"2023-11-06T03:16:43.493401+0000","last_peered":"2023-11-06T03:16:43.493401+0000","last_clean":"2023-11-06T03:16:43.493401+0000","last_became_active":"2023-11-06T03:16:43.490470+0000","last_became_peered":"2023-11-06T03:16:43.490470+0000","last_unstale":"2023-11-06T03:16:43.493401+0000","last_undegraded":"2023-11-06T03:16:43.493401+0000","last_fullsized":"2023-11-06T03:16:43.493401+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:25:44.976366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.472377+0000","last_change":"2023-11-06T03:16:43.471178+0000","last_active":"2023-11-06T03:16:43.472377+0000","last_peered":"2023-11-06T03:16:43.472377+0000","last_clean":"2023-11-06T03:16:43.472377+0000","last_became_active":"2023-11-06T03:16:43.470917+0000","last_became_peered":"2023-11-06T03:16:43.470917+0000","last_unstale":"2023-11-06T03:16:43.472377+0000","last_undegraded":"2023-11-06T03:16:43.472377+0000","last_fullsized":"2023-11-06T03:16:43.472377+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:20.794448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490053+0000","last_change":"2023-11-06T03:16:43.486441+0000","last_active":"2023-11-06T03:16:43.490053+0000","last_peered":"2023-11-06T03:16:43.490053+0000","last_clean":"2023-11-06T03:16:43.490053+0000","last_became_active":"2023-11-06T03:16:43.486154+0000","last_became_peered":"2023-11-06T03:16:43.486154+0000","last_unstale":"2023-11-06T03:16:43.490053+0000","last_undegraded":"2023-11-06T03:16:43.490053+0000","last_fullsized":"2023-11-06T03:16:43.490053+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:26.350831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477551+0000","last_change":"2023-11-06T03:16:43.465429+0000","last_active":"2023-11-06T03:16:43.477551+0000","last_peered":"2023-11-06T03:16:43.477551+0000","last_clean":"2023-11-06T03:16:43.477551+0000","last_became_active":"2023-11-06T03:16:43.465147+0000","last_became_peered":"2023-11-06T03:16:43.465147+0000","last_unstale":"2023-11-06T03:16:43.477551+0000","last_undegraded":"2023-11-06T03:16:43.477551+0000","last_fullsized":"2023-11-06T03:16:43.477551+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:21:34.480405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459831+0000","last_change":"2023-11-06T03:16:43.457674+0000","last_active":"2023-11-06T03:16:43.459831+0000","last_peered":"2023-11-06T03:16:43.459831+0000","last_clean":"2023-11-06T03:16:43.459831+0000","last_became_active":"2023-11-06T03:16:43.457458+0000","last_became_peered":"2023-11-06T03:16:43.457458+0000","last_unstale":"2023-11-06T03:16:43.459831+0000","last_undegraded":"2023-11-06T03:16:43.459831+0000","last_fullsized":"2023-11-06T03:16:43.459831+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:23:33.656829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490014+0000","last_change":"2023-11-06T03:16:43.487181+0000","last_active":"2023-11-06T03:16:43.490014+0000","last_peered":"2023-11-06T03:16:43.490014+0000","last_clean":"2023-11-06T03:16:43.490014+0000","last_became_active":"2023-11-06T03:16:43.486713+0000","last_became_peered":"2023-11-06T03:16:43.486713+0000","last_unstale":"2023-11-06T03:16:43.490014+0000","last_undegraded":"2023-11-06T03:16:43.490014+0000","last_fullsized":"2023-11-06T03:16:43.490014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:35.085986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487913+0000","last_change":"2023-11-06T03:16:43.475305+0000","last_active":"2023-11-06T03:16:43.487913+0000","last_peered":"2023-11-06T03:16:43.487913+0000","last_clean":"2023-11-06T03:16:43.487913+0000","last_became_active":"2023-11-06T03:16:43.474973+0000","last_became_peered":"2023-11-06T03:16:43.474973+0000","last_unstale":"2023-11-06T03:16:43.487913+0000","last_undegraded":"2023-11-06T03:16:43.487913+0000","last_fullsized":"2023-11-06T03:16:43.487913+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:44.141272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491345+0000","last_change":"2023-11-06T03:16:43.473039+0000","last_active":"2023-11-06T03:16:43.491345+0000","last_peered":"2023-11-06T03:16:43.491345+0000","last_clean":"2023-11-06T03:16:43.491345+0000","last_became_active":"2023-11-06T03:16:43.472293+0000","last_became_peered":"2023-11-06T03:16:43.472293+0000","last_unstale":"2023-11-06T03:16:43.491345+0000","last_undegraded":"2023-11-06T03:16:43.491345+0000","last_fullsized":"2023-11-06T03:16:43.491345+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:40.792037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.488561+0000","last_change":"2023-11-06T03:16:43.487853+0000","last_active":"2023-11-06T03:16:43.488561+0000","last_peered":"2023-11-06T03:16:43.488561+0000","last_clean":"2023-11-06T03:16:43.488561+0000","last_became_active":"2023-11-06T03:16:43.487445+0000","last_became_peered":"2023-11-06T03:16:43.487445+0000","last_unstale":"2023-11-06T03:16:43.488561+0000","last_undegraded":"2023-11-06T03:16:43.488561+0000","last_fullsized":"2023-11-06T03:16:43.488561+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:42:49.978034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490395+0000","last_change":"2023-11-06T03:16:43.489781+0000","last_active":"2023-11-06T03:16:43.490395+0000","last_peered":"2023-11-06T03:16:43.490395+0000","last_clean":"2023-11-06T03:16:43.490395+0000","last_became_active":"2023-11-06T03:16:43.489301+0000","last_became_peered":"2023-11-06T03:16:43.489301+0000","last_unstale":"2023-11-06T03:16:43.490395+0000","last_undegraded":"2023-11-06T03:16:43.490395+0000","last_fullsized":"2023-11-06T03:16:43.490395+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:51.344363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468035+0000","last_change":"2023-11-06T03:16:43.464404+0000","last_active":"2023-11-06T03:16:43.468035+0000","last_peered":"2023-11-06T03:16:43.468035+0000","last_clean":"2023-11-06T03:16:43.468035+0000","last_became_active":"2023-11-06T03:16:43.464109+0000","last_became_peered":"2023-11-06T03:16:43.464109+0000","last_unstale":"2023-11-06T03:16:43.468035+0000","last_undegraded":"2023-11-06T03:16:43.468035+0000","last_fullsized":"2023-11-06T03:16:43.468035+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:33:59.046167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489521+0000","last_change":"2023-11-06T03:16:43.486937+0000","last_active":"2023-11-06T03:16:43.489521+0000","last_peered":"2023-11-06T03:16:43.489521+0000","last_clean":"2023-11-06T03:16:43.489521+0000","last_became_active":"2023-11-06T03:16:43.486425+0000","last_became_peered":"2023-11-06T03:16:43.486425+0000","last_unstale":"2023-11-06T03:16:43.489521+0000","last_undegraded":"2023-11-06T03:16:43.489521+0000","last_fullsized":"2023-11-06T03:16:43.489521+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:24.829598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481636+0000","last_change":"2023-11-06T03:16:43.464963+0000","last_active":"2023-11-06T03:16:43.481636+0000","last_peered":"2023-11-06T03:16:43.481636+0000","last_clean":"2023-11-06T03:16:43.481636+0000","last_became_active":"2023-11-06T03:16:43.464639+0000","last_became_peered":"2023-11-06T03:16:43.464639+0000","last_unstale":"2023-11-06T03:16:43.481636+0000","last_undegraded":"2023-11-06T03:16:43.481636+0000","last_fullsized":"2023-11-06T03:16:43.481636+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:51.344363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480305+0000","last_change":"2023-11-06T03:16:43.479329+0000","last_active":"2023-11-06T03:16:43.480305+0000","last_peered":"2023-11-06T03:16:43.480305+0000","last_clean":"2023-11-06T03:16:43.480305+0000","last_became_active":"2023-11-06T03:16:43.478933+0000","last_became_peered":"2023-11-06T03:16:43.478933+0000","last_unstale":"2023-11-06T03:16:43.480305+0000","last_undegraded":"2023-11-06T03:16:43.480305+0000","last_fullsized":"2023-11-06T03:16:43.480305+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:49:36.998194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485490+0000","last_change":"2023-11-06T03:16:43.460587+0000","last_active":"2023-11-06T03:16:43.485490+0000","last_peered":"2023-11-06T03:16:43.485490+0000","last_clean":"2023-11-06T03:16:43.485490+0000","last_became_active":"2023-11-06T03:16:43.460304+0000","last_became_peered":"2023-11-06T03:16:43.460304+0000","last_unstale":"2023-11-06T03:16:43.485490+0000","last_undegraded":"2023-11-06T03:16:43.485490+0000","last_fullsized":"2023-11-06T03:16:43.485490+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477887+0000","last_change":"2023-11-06T03:16:43.472196+0000","last_active":"2023-11-06T03:16:43.477887+0000","last_peered":"2023-11-06T03:16:43.477887+0000","last_clean":"2023-11-06T03:16:43.477887+0000","last_became_active":"2023-11-06T03:16:43.471885+0000","last_became_peered":"2023-11-06T03:16:43.471885+0000","last_unstale":"2023-11-06T03:16:43.477887+0000","last_undegraded":"2023-11-06T03:16:43.477887+0000","last_fullsized":"2023-11-06T03:16:43.477887+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:02.501410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.466582+0000","last_change":"2023-11-06T03:16:43.462862+0000","last_active":"2023-11-06T03:16:43.466582+0000","last_peered":"2023-11-06T03:16:43.466582+0000","last_clean":"2023-11-06T03:16:43.466582+0000","last_became_active":"2023-11-06T03:16:43.462531+0000","last_became_peered":"2023-11-06T03:16:43.462531+0000","last_unstale":"2023-11-06T03:16:43.466582+0000","last_undegraded":"2023-11-06T03:16:43.466582+0000","last_fullsized":"2023-11-06T03:16:43.466582+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:31:07.589454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484091+0000","last_change":"2023-11-06T03:16:43.473397+0000","last_active":"2023-11-06T03:16:43.484091+0000","last_peered":"2023-11-06T03:16:43.484091+0000","last_clean":"2023-11-06T03:16:43.484091+0000","last_became_active":"2023-11-06T03:16:43.472808+0000","last_became_peered":"2023-11-06T03:16:43.472808+0000","last_unstale":"2023-11-06T03:16:43.484091+0000","last_undegraded":"2023-11-06T03:16:43.484091+0000","last_fullsized":"2023-11-06T03:16:43.484091+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:56.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489502+0000","last_change":"2023-11-06T03:16:43.486783+0000","last_active":"2023-11-06T03:16:43.489502+0000","last_peered":"2023-11-06T03:16:43.489502+0000","last_clean":"2023-11-06T03:16:43.489502+0000","last_became_active":"2023-11-06T03:16:43.486478+0000","last_became_peered":"2023-11-06T03:16:43.486478+0000","last_unstale":"2023-11-06T03:16:43.489502+0000","last_undegraded":"2023-11-06T03:16:43.489502+0000","last_fullsized":"2023-11-06T03:16:43.489502+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:20.219291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.490988+0000","last_change":"2023-11-06T03:16:43.490219+0000","last_active":"2023-11-06T03:16:43.490988+0000","last_peered":"2023-11-06T03:16:43.490988+0000","last_clean":"2023-11-06T03:16:43.490988+0000","last_became_active":"2023-11-06T03:16:43.489683+0000","last_became_peered":"2023-11-06T03:16:43.489683+0000","last_unstale":"2023-11-06T03:16:43.490988+0000","last_undegraded":"2023-11-06T03:16:43.490988+0000","last_fullsized":"2023-11-06T03:16:43.490988+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:23.704213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.482271+0000","last_change":"2023-11-06T03:16:43.475695+0000","last_active":"2023-11-06T03:16:43.482271+0000","last_peered":"2023-11-06T03:16:43.482271+0000","last_clean":"2023-11-06T03:16:43.482271+0000","last_became_active":"2023-11-06T03:16:43.475144+0000","last_became_peered":"2023-11-06T03:16:43.475144+0000","last_unstale":"2023-11-06T03:16:43.482271+0000","last_undegraded":"2023-11-06T03:16:43.482271+0000","last_fullsized":"2023-11-06T03:16:43.482271+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:10:06.262155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.481987+0000","last_change":"2023-11-06T03:16:43.481347+0000","last_active":"2023-11-06T03:16:43.481987+0000","last_peered":"2023-11-06T03:16:43.481987+0000","last_clean":"2023-11-06T03:16:43.481987+0000","last_became_active":"2023-11-06T03:16:43.480931+0000","last_became_peered":"2023-11-06T03:16:43.480931+0000","last_unstale":"2023-11-06T03:16:43.481987+0000","last_undegraded":"2023-11-06T03:16:43.481987+0000","last_fullsized":"2023-11-06T03:16:43.481987+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485234+0000","last_change":"2023-11-06T03:16:43.484322+0000","last_active":"2023-11-06T03:16:43.485234+0000","last_peered":"2023-11-06T03:16:43.485234+0000","last_clean":"2023-11-06T03:16:43.485234+0000","last_became_active":"2023-11-06T03:16:43.483535+0000","last_became_peered":"2023-11-06T03:16:43.483535+0000","last_unstale":"2023-11-06T03:16:43.485234+0000","last_undegraded":"2023-11-06T03:16:43.485234+0000","last_fullsized":"2023-11-06T03:16:43.485234+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:18:25.085192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484304+0000","last_change":"2023-11-06T03:16:43.477939+0000","last_active":"2023-11-06T03:16:43.484304+0000","last_peered":"2023-11-06T03:16:43.484304+0000","last_clean":"2023-11-06T03:16:43.484304+0000","last_became_active":"2023-11-06T03:16:43.477382+0000","last_became_peered":"2023-11-06T03:16:43.477382+0000","last_unstale":"2023-11-06T03:16:43.484304+0000","last_undegraded":"2023-11-06T03:16:43.484304+0000","last_fullsized":"2023-11-06T03:16:43.484304+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:05:35.085986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"162'4","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480230+0000","last_change":"2023-11-06T03:16:43.080958+0000","last_active":"2023-11-06T03:16:43.480230+0000","last_peered":"2023-11-06T03:16:43.480230+0000","last_clean":"2023-11-06T03:16:43.480230+0000","last_became_active":"2023-11-06T03:16:43.080227+0000","last_became_peered":"2023-11-06T03:16:43.080227+0000","last_unstale":"2023-11-06T03:16:43.480230+0000","last_undegraded":"2023-11-06T03:16:43.480230+0000","last_fullsized":"2023-11-06T03:16:43.480230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:25.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483378+0000","last_change":"2023-11-06T03:16:43.479053+0000","last_active":"2023-11-06T03:16:43.483378+0000","last_peered":"2023-11-06T03:16:43.483378+0000","last_clean":"2023-11-06T03:16:43.483378+0000","last_became_active":"2023-11-06T03:16:43.478733+0000","last_became_peered":"2023-11-06T03:16:43.478733+0000","last_unstale":"2023-11-06T03:16:43.483378+0000","last_undegraded":"2023-11-06T03:16:43.483378+0000","last_fullsized":"2023-11-06T03:16:43.483378+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:24.678686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491308+0000","last_change":"2023-11-06T03:16:43.097292+0000","last_active":"2023-11-06T03:16:43.491308+0000","last_peered":"2023-11-06T03:16:43.491308+0000","last_clean":"2023-11-06T03:16:43.491308+0000","last_became_active":"2023-11-06T03:16:43.096950+0000","last_became_peered":"2023-11-06T03:16:43.096950+0000","last_unstale":"2023-11-06T03:16:43.491308+0000","last_undegraded":"2023-11-06T03:16:43.491308+0000","last_fullsized":"2023-11-06T03:16:43.491308+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:10.928900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459812+0000","last_change":"2023-11-06T03:16:43.456925+0000","last_active":"2023-11-06T03:16:43.459812+0000","last_peered":"2023-11-06T03:16:43.459812+0000","last_clean":"2023-11-06T03:16:43.459812+0000","last_became_active":"2023-11-06T03:16:43.456671+0000","last_became_peered":"2023-11-06T03:16:43.456671+0000","last_unstale":"2023-11-06T03:16:43.459812+0000","last_undegraded":"2023-11-06T03:16:43.459812+0000","last_fullsized":"2023-11-06T03:16:43.459812+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:39:32.050039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.469846+0000","last_change":"2023-11-06T03:16:43.099126+0000","last_active":"2023-11-06T03:16:43.469846+0000","last_peered":"2023-11-06T03:16:43.469846+0000","last_clean":"2023-11-06T03:16:43.469846+0000","last_became_active":"2023-11-06T03:16:43.098242+0000","last_became_peered":"2023-11-06T03:16:43.098242+0000","last_unstale":"2023-11-06T03:16:43.469846+0000","last_undegraded":"2023-11-06T03:16:43.469846+0000","last_fullsized":"2023-11-06T03:16:43.469846+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:38:45.180079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484313+0000","last_change":"2023-11-06T03:16:43.479949+0000","last_active":"2023-11-06T03:16:43.484313+0000","last_peered":"2023-11-06T03:16:43.484313+0000","last_clean":"2023-11-06T03:16:43.484313+0000","last_became_active":"2023-11-06T03:16:43.479461+0000","last_became_peered":"2023-11-06T03:16:43.479461+0000","last_unstale":"2023-11-06T03:16:43.484313+0000","last_undegraded":"2023-11-06T03:16:43.484313+0000","last_fullsized":"2023-11-06T03:16:43.484313+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:23:15.277978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480991+0000","last_change":"2023-11-06T03:16:43.092523+0000","last_active":"2023-11-06T03:16:43.480991+0000","last_peered":"2023-11-06T03:16:43.480991+0000","last_clean":"2023-11-06T03:16:43.480991+0000","last_became_active":"2023-11-06T03:16:43.092202+0000","last_became_peered":"2023-11-06T03:16:43.092202+0000","last_unstale":"2023-11-06T03:16:43.480991+0000","last_undegraded":"2023-11-06T03:16:43.480991+0000","last_fullsized":"2023-11-06T03:16:43.480991+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:28:14.063072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493301+0000","last_change":"2023-11-06T03:16:43.490322+0000","last_active":"2023-11-06T03:16:43.493301+0000","last_peered":"2023-11-06T03:16:43.493301+0000","last_clean":"2023-11-06T03:16:43.493301+0000","last_became_active":"2023-11-06T03:16:43.487453+0000","last_became_peered":"2023-11-06T03:16:43.487453+0000","last_unstale":"2023-11-06T03:16:43.493301+0000","last_undegraded":"2023-11-06T03:16:43.493301+0000","last_fullsized":"2023-11-06T03:16:43.493301+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:15:28.239699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485807+0000","last_change":"2023-11-06T03:16:43.095795+0000","last_active":"2023-11-06T03:16:43.485807+0000","last_peered":"2023-11-06T03:16:43.485807+0000","last_clean":"2023-11-06T03:16:43.485807+0000","last_became_active":"2023-11-06T03:16:43.095310+0000","last_became_peered":"2023-11-06T03:16:43.095310+0000","last_unstale":"2023-11-06T03:16:43.485807+0000","last_undegraded":"2023-11-06T03:16:43.485807+0000","last_fullsized":"2023-11-06T03:16:43.485807+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:40.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.477618+0000","last_change":"2023-11-06T03:16:43.473861+0000","last_active":"2023-11-06T03:16:43.477618+0000","last_peered":"2023-11-06T03:16:43.477618+0000","last_clean":"2023-11-06T03:16:43.477618+0000","last_became_active":"2023-11-06T03:16:43.472880+0000","last_became_peered":"2023-11-06T03:16:43.472880+0000","last_unstale":"2023-11-06T03:16:43.477618+0000","last_undegraded":"2023-11-06T03:16:43.477618+0000","last_fullsized":"2023-11-06T03:16:43.477618+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:10:34.131430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.472142+0000","last_change":"2023-11-06T03:16:43.089805+0000","last_active":"2023-11-06T03:16:43.472142+0000","last_peered":"2023-11-06T03:16:43.472142+0000","last_clean":"2023-11-06T03:16:43.472142+0000","last_became_active":"2023-11-06T03:16:43.089449+0000","last_became_peered":"2023-11-06T03:16:43.089449+0000","last_unstale":"2023-11-06T03:16:43.472142+0000","last_undegraded":"2023-11-06T03:16:43.472142+0000","last_fullsized":"2023-11-06T03:16:43.472142+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:05:01.412501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480097+0000","last_change":"2023-11-06T03:16:43.475509+0000","last_active":"2023-11-06T03:16:43.480097+0000","last_peered":"2023-11-06T03:16:43.480097+0000","last_clean":"2023-11-06T03:16:43.480097+0000","last_became_active":"2023-11-06T03:16:43.475135+0000","last_became_peered":"2023-11-06T03:16:43.475135+0000","last_unstale":"2023-11-06T03:16:43.480097+0000","last_undegraded":"2023-11-06T03:16:43.480097+0000","last_fullsized":"2023-11-06T03:16:43.480097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:09:31.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.452696+0000","last_change":"2023-11-06T03:16:43.096549+0000","last_active":"2023-11-06T03:16:43.452696+0000","last_peered":"2023-11-06T03:16:43.452696+0000","last_clean":"2023-11-06T03:16:43.452696+0000","last_became_active":"2023-11-06T03:16:43.096223+0000","last_became_peered":"2023-11-06T03:16:43.096223+0000","last_unstale":"2023-11-06T03:16:43.452696+0000","last_undegraded":"2023-11-06T03:16:43.452696+0000","last_fullsized":"2023-11-06T03:16:43.452696+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:26:13.949940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485956+0000","last_change":"2023-11-06T03:16:43.468576+0000","last_active":"2023-11-06T03:16:43.485956+0000","last_peered":"2023-11-06T03:16:43.485956+0000","last_clean":"2023-11-06T03:16:43.485956+0000","last_became_active":"2023-11-06T03:16:43.467864+0000","last_became_peered":"2023-11-06T03:16:43.467864+0000","last_unstale":"2023-11-06T03:16:43.485956+0000","last_undegraded":"2023-11-06T03:16:43.485956+0000","last_fullsized":"2023-11-06T03:16:43.485956+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:05.877838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.461104+0000","last_change":"2023-11-06T03:16:43.089090+0000","last_active":"2023-11-06T03:16:43.461104+0000","last_peered":"2023-11-06T03:16:43.461104+0000","last_clean":"2023-11-06T03:16:43.461104+0000","last_became_active":"2023-11-06T03:16:43.088728+0000","last_became_peered":"2023-11-06T03:16:43.088728+0000","last_unstale":"2023-11-06T03:16:43.461104+0000","last_undegraded":"2023-11-06T03:16:43.461104+0000","last_fullsized":"2023-11-06T03:16:43.461104+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:04.231228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493211+0000","last_change":"2023-11-06T03:16:43.484089+0000","last_active":"2023-11-06T03:16:43.493211+0000","last_peered":"2023-11-06T03:16:43.493211+0000","last_clean":"2023-11-06T03:16:43.493211+0000","last_became_active":"2023-11-06T03:16:43.483085+0000","last_became_peered":"2023-11-06T03:16:43.483085+0000","last_unstale":"2023-11-06T03:16:43.493211+0000","last_undegraded":"2023-11-06T03:16:43.493211+0000","last_fullsized":"2023-11-06T03:16:43.493211+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:28:03.768113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489111+0000","last_change":"2023-11-06T03:16:43.103705+0000","last_active":"2023-11-06T03:16:43.489111+0000","last_peered":"2023-11-06T03:16:43.489111+0000","last_clean":"2023-11-06T03:16:43.489111+0000","last_became_active":"2023-11-06T03:16:43.103387+0000","last_became_peered":"2023-11-06T03:16:43.103387+0000","last_unstale":"2023-11-06T03:16:43.489111+0000","last_undegraded":"2023-11-06T03:16:43.489111+0000","last_fullsized":"2023-11-06T03:16:43.489111+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:24:21.731897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483458+0000","last_change":"2023-11-06T03:16:43.478663+0000","last_active":"2023-11-06T03:16:43.483458+0000","last_peered":"2023-11-06T03:16:43.483458+0000","last_clean":"2023-11-06T03:16:43.483458+0000","last_became_active":"2023-11-06T03:16:43.478300+0000","last_became_peered":"2023-11-06T03:16:43.478300+0000","last_unstale":"2023-11-06T03:16:43.483458+0000","last_undegraded":"2023-11-06T03:16:43.483458+0000","last_fullsized":"2023-11-06T03:16:43.483458+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:15:06.501292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491242+0000","last_change":"2023-11-06T03:16:43.093755+0000","last_active":"2023-11-06T03:16:43.491242+0000","last_peered":"2023-11-06T03:16:43.491242+0000","last_clean":"2023-11-06T03:16:43.491242+0000","last_became_active":"2023-11-06T03:16:43.092656+0000","last_became_peered":"2023-11-06T03:16:43.092656+0000","last_unstale":"2023-11-06T03:16:43.491242+0000","last_undegraded":"2023-11-06T03:16:43.491242+0000","last_fullsized":"2023-11-06T03:16:43.491242+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:20:08.411222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489615+0000","last_change":"2023-11-06T03:16:43.486878+0000","last_active":"2023-11-06T03:16:43.489615+0000","last_peered":"2023-11-06T03:16:43.489615+0000","last_clean":"2023-11-06T03:16:43.489615+0000","last_became_active":"2023-11-06T03:16:43.486365+0000","last_became_peered":"2023-11-06T03:16:43.486365+0000","last_unstale":"2023-11-06T03:16:43.489615+0000","last_undegraded":"2023-11-06T03:16:43.489615+0000","last_fullsized":"2023-11-06T03:16:43.489615+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:26.350831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485239+0000","last_change":"2023-11-06T03:16:43.106242+0000","last_active":"2023-11-06T03:16:43.485239+0000","last_peered":"2023-11-06T03:16:43.485239+0000","last_clean":"2023-11-06T03:16:43.485239+0000","last_became_active":"2023-11-06T03:16:43.106002+0000","last_became_peered":"2023-11-06T03:16:43.106002+0000","last_unstale":"2023-11-06T03:16:43.485239+0000","last_undegraded":"2023-11-06T03:16:43.485239+0000","last_fullsized":"2023-11-06T03:16:43.485239+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:49.361616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.473110+0000","last_change":"2023-11-06T03:16:43.471910+0000","last_active":"2023-11-06T03:16:43.473110+0000","last_peered":"2023-11-06T03:16:43.473110+0000","last_clean":"2023-11-06T03:16:43.473110+0000","last_became_active":"2023-11-06T03:16:43.471588+0000","last_became_peered":"2023-11-06T03:16:43.471588+0000","last_unstale":"2023-11-06T03:16:43.473110+0000","last_undegraded":"2023-11-06T03:16:43.473110+0000","last_fullsized":"2023-11-06T03:16:43.473110+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:46:56.398286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.464816+0000","last_change":"2023-11-06T03:16:43.093443+0000","last_active":"2023-11-06T03:16:43.464816+0000","last_peered":"2023-11-06T03:16:43.464816+0000","last_clean":"2023-11-06T03:16:43.464816+0000","last_became_active":"2023-11-06T03:16:43.093105+0000","last_became_peered":"2023-11-06T03:16:43.093105+0000","last_unstale":"2023-11-06T03:16:43.464816+0000","last_undegraded":"2023-11-06T03:16:43.464816+0000","last_fullsized":"2023-11-06T03:16:43.464816+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:39:25.347490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489538+0000","last_change":"2023-11-06T03:16:43.486962+0000","last_active":"2023-11-06T03:16:43.489538+0000","last_peered":"2023-11-06T03:16:43.489538+0000","last_clean":"2023-11-06T03:16:43.489538+0000","last_became_active":"2023-11-06T03:16:43.486406+0000","last_became_peered":"2023-11-06T03:16:43.486406+0000","last_unstale":"2023-11-06T03:16:43.489538+0000","last_undegraded":"2023-11-06T03:16:43.489538+0000","last_fullsized":"2023-11-06T03:16:43.489538+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:57:11.932240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.480667+0000","last_change":"2023-11-06T03:16:43.100179+0000","last_active":"2023-11-06T03:16:43.480667+0000","last_peered":"2023-11-06T03:16:43.480667+0000","last_clean":"2023-11-06T03:16:43.480667+0000","last_became_active":"2023-11-06T03:16:43.099911+0000","last_became_peered":"2023-11-06T03:16:43.099911+0000","last_unstale":"2023-11-06T03:16:43.480667+0000","last_undegraded":"2023-11-06T03:16:43.480667+0000","last_fullsized":"2023-11-06T03:16:43.480667+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:23.675346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487697+0000","last_change":"2023-11-06T03:16:43.485938+0000","last_active":"2023-11-06T03:16:43.487697+0000","last_peered":"2023-11-06T03:16:43.487697+0000","last_clean":"2023-11-06T03:16:43.487697+0000","last_became_active":"2023-11-06T03:16:43.485583+0000","last_became_peered":"2023-11-06T03:16:43.485583+0000","last_unstale":"2023-11-06T03:16:43.487697+0000","last_undegraded":"2023-11-06T03:16:43.487697+0000","last_fullsized":"2023-11-06T03:16:43.487697+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:15:53.978884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.462592+0000","last_change":"2023-11-06T03:16:43.099380+0000","last_active":"2023-11-06T03:16:43.462592+0000","last_peered":"2023-11-06T03:16:43.462592+0000","last_clean":"2023-11-06T03:16:43.462592+0000","last_became_active":"2023-11-06T03:16:43.098626+0000","last_became_peered":"2023-11-06T03:16:43.098626+0000","last_unstale":"2023-11-06T03:16:43.462592+0000","last_undegraded":"2023-11-06T03:16:43.462592+0000","last_fullsized":"2023-11-06T03:16:43.462592+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:05.839972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.487683+0000","last_change":"2023-11-06T03:16:43.485468+0000","last_active":"2023-11-06T03:16:43.487683+0000","last_peered":"2023-11-06T03:16:43.487683+0000","last_clean":"2023-11-06T03:16:43.487683+0000","last_became_active":"2023-11-06T03:16:43.484815+0000","last_became_peered":"2023-11-06T03:16:43.484815+0000","last_unstale":"2023-11-06T03:16:43.487683+0000","last_undegraded":"2023-11-06T03:16:43.487683+0000","last_fullsized":"2023-11-06T03:16:43.487683+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:22:07.885491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.452940+0000","last_change":"2023-11-06T03:16:43.104549+0000","last_active":"2023-11-06T03:16:43.452940+0000","last_peered":"2023-11-06T03:16:43.452940+0000","last_clean":"2023-11-06T03:16:43.452940+0000","last_became_active":"2023-11-06T03:16:43.104183+0000","last_became_peered":"2023-11-06T03:16:43.104183+0000","last_unstale":"2023-11-06T03:16:43.452940+0000","last_undegraded":"2023-11-06T03:16:43.452940+0000","last_fullsized":"2023-11-06T03:16:43.452940+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:02:45.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484096+0000","last_change":"2023-11-06T03:16:43.472629+0000","last_active":"2023-11-06T03:16:43.484096+0000","last_peered":"2023-11-06T03:16:43.484096+0000","last_clean":"2023-11-06T03:16:43.484096+0000","last_became_active":"2023-11-06T03:16:43.472360+0000","last_became_peered":"2023-11-06T03:16:43.472360+0000","last_unstale":"2023-11-06T03:16:43.484096+0000","last_undegraded":"2023-11-06T03:16:43.484096+0000","last_fullsized":"2023-11-06T03:16:43.484096+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:36:16.402184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.458001+0000","last_change":"2023-11-06T03:16:43.108530+0000","last_active":"2023-11-06T03:16:43.458001+0000","last_peered":"2023-11-06T03:16:43.458001+0000","last_clean":"2023-11-06T03:16:43.458001+0000","last_became_active":"2023-11-06T03:16:43.107937+0000","last_became_peered":"2023-11-06T03:16:43.107937+0000","last_unstale":"2023-11-06T03:16:43.458001+0000","last_undegraded":"2023-11-06T03:16:43.458001+0000","last_fullsized":"2023-11-06T03:16:43.458001+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:22:42.521582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.479967+0000","last_change":"2023-11-06T03:16:43.478875+0000","last_active":"2023-11-06T03:16:43.479967+0000","last_peered":"2023-11-06T03:16:43.479967+0000","last_clean":"2023-11-06T03:16:43.479967+0000","last_became_active":"2023-11-06T03:16:43.478523+0000","last_became_peered":"2023-11-06T03:16:43.478523+0000","last_unstale":"2023-11-06T03:16:43.479967+0000","last_undegraded":"2023-11-06T03:16:43.479967+0000","last_fullsized":"2023-11-06T03:16:43.479967+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:23:15.277978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.474068+0000","last_change":"2023-11-06T03:16:43.102037+0000","last_active":"2023-11-06T03:16:43.474068+0000","last_peered":"2023-11-06T03:16:43.474068+0000","last_clean":"2023-11-06T03:16:43.474068+0000","last_became_active":"2023-11-06T03:16:43.101740+0000","last_became_peered":"2023-11-06T03:16:43.101740+0000","last_unstale":"2023-11-06T03:16:43.474068+0000","last_undegraded":"2023-11-06T03:16:43.474068+0000","last_fullsized":"2023-11-06T03:16:43.474068+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:22:19.215950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491363+0000","last_change":"2023-11-06T03:16:43.482622+0000","last_active":"2023-11-06T03:16:43.491363+0000","last_peered":"2023-11-06T03:16:43.491363+0000","last_clean":"2023-11-06T03:16:43.491363+0000","last_became_active":"2023-11-06T03:16:43.482360+0000","last_became_peered":"2023-11-06T03:16:43.482360+0000","last_unstale":"2023-11-06T03:16:43.491363+0000","last_undegraded":"2023-11-06T03:16:43.491363+0000","last_fullsized":"2023-11-06T03:16:43.491363+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:29:54.461377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.455190+0000","last_change":"2023-11-06T03:16:43.085436+0000","last_active":"2023-11-06T03:16:43.455190+0000","last_peered":"2023-11-06T03:16:43.455190+0000","last_clean":"2023-11-06T03:16:43.455190+0000","last_became_active":"2023-11-06T03:16:43.085137+0000","last_became_peered":"2023-11-06T03:16:43.085137+0000","last_unstale":"2023-11-06T03:16:43.455190+0000","last_undegraded":"2023-11-06T03:16:43.455190+0000","last_fullsized":"2023-11-06T03:16:43.455190+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:41.901469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.479859+0000","last_change":"2023-11-06T03:16:43.475339+0000","last_active":"2023-11-06T03:16:43.479859+0000","last_peered":"2023-11-06T03:16:43.479859+0000","last_clean":"2023-11-06T03:16:43.479859+0000","last_became_active":"2023-11-06T03:16:43.475015+0000","last_became_peered":"2023-11-06T03:16:43.475015+0000","last_unstale":"2023-11-06T03:16:43.479859+0000","last_undegraded":"2023-11-06T03:16:43.479859+0000","last_fullsized":"2023-11-06T03:16:43.479859+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:56:57.287183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.455280+0000","last_change":"2023-11-06T03:16:43.095767+0000","last_active":"2023-11-06T03:16:43.455280+0000","last_peered":"2023-11-06T03:16:43.455280+0000","last_clean":"2023-11-06T03:16:43.455280+0000","last_became_active":"2023-11-06T03:16:43.095485+0000","last_became_peered":"2023-11-06T03:16:43.095485+0000","last_unstale":"2023-11-06T03:16:43.455280+0000","last_undegraded":"2023-11-06T03:16:43.455280+0000","last_fullsized":"2023-11-06T03:16:43.455280+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:57:39.788696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493153+0000","last_change":"2023-11-06T03:16:43.485595+0000","last_active":"2023-11-06T03:16:43.493153+0000","last_peered":"2023-11-06T03:16:43.493153+0000","last_clean":"2023-11-06T03:16:43.493153+0000","last_became_active":"2023-11-06T03:16:43.484988+0000","last_became_peered":"2023-11-06T03:16:43.484988+0000","last_unstale":"2023-11-06T03:16:43.493153+0000","last_undegraded":"2023-11-06T03:16:43.493153+0000","last_fullsized":"2023-11-06T03:16:43.493153+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:45:04.578036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454723+0000","last_change":"2023-11-06T03:16:43.088714+0000","last_active":"2023-11-06T03:16:43.454723+0000","last_peered":"2023-11-06T03:16:43.454723+0000","last_clean":"2023-11-06T03:16:43.454723+0000","last_became_active":"2023-11-06T03:16:43.088150+0000","last_became_peered":"2023-11-06T03:16:43.088150+0000","last_unstale":"2023-11-06T03:16:43.454723+0000","last_undegraded":"2023-11-06T03:16:43.454723+0000","last_fullsized":"2023-11-06T03:16:43.454723+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:34:39.075788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485712+0000","last_change":"2023-11-06T03:16:43.466416+0000","last_active":"2023-11-06T03:16:43.485712+0000","last_peered":"2023-11-06T03:16:43.485712+0000","last_clean":"2023-11-06T03:16:43.485712+0000","last_became_active":"2023-11-06T03:16:43.466224+0000","last_became_peered":"2023-11-06T03:16:43.466224+0000","last_unstale":"2023-11-06T03:16:43.485712+0000","last_undegraded":"2023-11-06T03:16:43.485712+0000","last_fullsized":"2023-11-06T03:16:43.485712+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:46:42.904810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.448563+0000","last_change":"2023-11-06T03:16:43.099793+0000","last_active":"2023-11-06T03:16:43.448563+0000","last_peered":"2023-11-06T03:16:43.448563+0000","last_clean":"2023-11-06T03:16:43.448563+0000","last_became_active":"2023-11-06T03:16:43.099058+0000","last_became_peered":"2023-11-06T03:16:43.099058+0000","last_unstale":"2023-11-06T03:16:43.448563+0000","last_undegraded":"2023-11-06T03:16:43.448563+0000","last_fullsized":"2023-11-06T03:16:43.448563+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:34:21.593316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484461+0000","last_change":"2023-11-06T03:16:43.480136+0000","last_active":"2023-11-06T03:16:43.484461+0000","last_peered":"2023-11-06T03:16:43.484461+0000","last_clean":"2023-11-06T03:16:43.484461+0000","last_became_active":"2023-11-06T03:16:43.479739+0000","last_became_peered":"2023-11-06T03:16:43.479739+0000","last_unstale":"2023-11-06T03:16:43.484461+0000","last_undegraded":"2023-11-06T03:16:43.484461+0000","last_fullsized":"2023-11-06T03:16:43.484461+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:43:20.794448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454716+0000","last_change":"2023-11-06T03:16:43.092098+0000","last_active":"2023-11-06T03:16:43.454716+0000","last_peered":"2023-11-06T03:16:43.454716+0000","last_clean":"2023-11-06T03:16:43.454716+0000","last_became_active":"2023-11-06T03:16:43.091694+0000","last_became_peered":"2023-11-06T03:16:43.091694+0000","last_unstale":"2023-11-06T03:16:43.454716+0000","last_undegraded":"2023-11-06T03:16:43.454716+0000","last_fullsized":"2023-11-06T03:16:43.454716+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:07.076814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489472+0000","last_change":"2023-11-06T03:16:43.485229+0000","last_active":"2023-11-06T03:16:43.489472+0000","last_peered":"2023-11-06T03:16:43.489472+0000","last_clean":"2023-11-06T03:16:43.489472+0000","last_became_active":"2023-11-06T03:16:43.484905+0000","last_became_peered":"2023-11-06T03:16:43.484905+0000","last_unstale":"2023-11-06T03:16:43.489472+0000","last_undegraded":"2023-11-06T03:16:43.489472+0000","last_fullsized":"2023-11-06T03:16:43.489472+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:37:50.364957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.463233+0000","last_change":"2023-11-06T03:16:43.098520+0000","last_active":"2023-11-06T03:16:43.463233+0000","last_peered":"2023-11-06T03:16:43.463233+0000","last_clean":"2023-11-06T03:16:43.463233+0000","last_became_active":"2023-11-06T03:16:43.097827+0000","last_became_peered":"2023-11-06T03:16:43.097827+0000","last_unstale":"2023-11-06T03:16:43.463233+0000","last_undegraded":"2023-11-06T03:16:43.463233+0000","last_fullsized":"2023-11-06T03:16:43.463233+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:19:48.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468108+0000","last_change":"2023-11-06T03:16:43.465861+0000","last_active":"2023-11-06T03:16:43.468108+0000","last_peered":"2023-11-06T03:16:43.468108+0000","last_clean":"2023-11-06T03:16:43.468108+0000","last_became_active":"2023-11-06T03:16:43.465512+0000","last_became_peered":"2023-11-06T03:16:43.465512+0000","last_unstale":"2023-11-06T03:16:43.468108+0000","last_undegraded":"2023-11-06T03:16:43.468108+0000","last_fullsized":"2023-11-06T03:16:43.468108+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:59:42.932955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.446641+0000","last_change":"2023-11-06T03:16:43.103658+0000","last_active":"2023-11-06T03:16:43.446641+0000","last_peered":"2023-11-06T03:16:43.446641+0000","last_clean":"2023-11-06T03:16:43.446641+0000","last_became_active":"2023-11-06T03:16:43.103299+0000","last_became_peered":"2023-11-06T03:16:43.103299+0000","last_unstale":"2023-11-06T03:16:43.446641+0000","last_undegraded":"2023-11-06T03:16:43.446641+0000","last_fullsized":"2023-11-06T03:16:43.446641+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:21:33.477065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.484308+0000","last_change":"2023-11-06T03:16:43.481220+0000","last_active":"2023-11-06T03:16:43.484308+0000","last_peered":"2023-11-06T03:16:43.484308+0000","last_clean":"2023-11-06T03:16:43.484308+0000","last_became_active":"2023-11-06T03:16:43.480896+0000","last_became_peered":"2023-11-06T03:16:43.480896+0000","last_unstale":"2023-11-06T03:16:43.484308+0000","last_undegraded":"2023-11-06T03:16:43.484308+0000","last_fullsized":"2023-11-06T03:16:43.484308+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:05.877838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454808+0000","last_change":"2023-11-06T03:16:43.085834+0000","last_active":"2023-11-06T03:16:43.454808+0000","last_peered":"2023-11-06T03:16:43.454808+0000","last_clean":"2023-11-06T03:16:43.454808+0000","last_became_active":"2023-11-06T03:16:43.082598+0000","last_became_peered":"2023-11-06T03:16:43.082598+0000","last_unstale":"2023-11-06T03:16:43.454808+0000","last_undegraded":"2023-11-06T03:16:43.454808+0000","last_fullsized":"2023-11-06T03:16:43.454808+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:16:43.137932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":282,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097986+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:16:43.097986+0000","last_peered":"2023-11-06T03:16:43.097986+0000","last_clean":"2023-11-06T03:16:43.097986+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:16:43.097986+0000","last_undegraded":"2023-11-06T03:16:43.097986+0000","last_fullsized":"2023-11-06T03:16:43.097986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.090803+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:16:43.090803+0000","last_peered":"2023-11-06T03:16:43.090803+0000","last_clean":"2023-11-06T03:16:43.090803+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:16:43.090803+0000","last_undegraded":"2023-11-06T03:16:43.090803+0000","last_fullsized":"2023-11-06T03:16:43.090803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.489445+0000","last_change":"2023-11-06T03:16:43.484652+0000","last_active":"2023-11-06T03:16:43.489445+0000","last_peered":"2023-11-06T03:16:43.489445+0000","last_clean":"2023-11-06T03:16:43.489445+0000","last_became_active":"2023-11-06T03:16:43.484314+0000","last_became_peered":"2023-11-06T03:16:43.484314+0000","last_unstale":"2023-11-06T03:16:43.489445+0000","last_undegraded":"2023-11-06T03:16:43.489445+0000","last_fullsized":"2023-11-06T03:16:43.489445+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:08.123116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"162'3","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.461827+0000","last_change":"2023-11-06T03:16:43.105567+0000","last_active":"2023-11-06T03:16:43.461827+0000","last_peered":"2023-11-06T03:16:43.461827+0000","last_clean":"2023-11-06T03:16:43.461827+0000","last_became_active":"2023-11-06T03:16:43.104944+0000","last_became_peered":"2023-11-06T03:16:43.104944+0000","last_unstale":"2023-11-06T03:16:43.461827+0000","last_undegraded":"2023-11-06T03:16:43.461827+0000","last_fullsized":"2023-11-06T03:16:43.461827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:40.787249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097756+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:16:43.097756+0000","last_peered":"2023-11-06T03:16:43.097756+0000","last_clean":"2023-11-06T03:16:43.097756+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:16:43.097756+0000","last_undegraded":"2023-11-06T03:16:43.097756+0000","last_fullsized":"2023-11-06T03:16:43.097756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468006+0000","last_change":"2023-11-06T03:16:43.464990+0000","last_active":"2023-11-06T03:16:43.468006+0000","last_peered":"2023-11-06T03:16:43.468006+0000","last_clean":"2023-11-06T03:16:43.468006+0000","last_became_active":"2023-11-06T03:16:43.464706+0000","last_became_peered":"2023-11-06T03:16:43.464706+0000","last_unstale":"2023-11-06T03:16:43.468006+0000","last_undegraded":"2023-11-06T03:16:43.468006+0000","last_fullsized":"2023-11-06T03:16:43.468006+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:20.346917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"162'6","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493602+0000","last_change":"2023-11-06T03:16:43.086521+0000","last_active":"2023-11-06T03:16:43.493602+0000","last_peered":"2023-11-06T03:16:43.493602+0000","last_clean":"2023-11-06T03:16:43.493602+0000","last_became_active":"2023-11-06T03:16:43.082586+0000","last_became_peered":"2023-11-06T03:16:43.082586+0000","last_unstale":"2023-11-06T03:16:43.493602+0000","last_undegraded":"2023-11-06T03:16:43.493602+0000","last_fullsized":"2023-11-06T03:16:43.493602+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:22.700873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":128,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.074584+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:16:43.074584+0000","last_peered":"2023-11-06T03:16:43.074584+0000","last_clean":"2023-11-06T03:16:43.074584+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:16:43.074584+0000","last_undegraded":"2023-11-06T03:16:43.074584+0000","last_fullsized":"2023-11-06T03:16:43.074584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.486809+0000","last_change":"2023-11-06T03:16:43.481383+0000","last_active":"2023-11-06T03:16:43.486809+0000","last_peered":"2023-11-06T03:16:43.486809+0000","last_clean":"2023-11-06T03:16:43.486809+0000","last_became_active":"2023-11-06T03:16:43.479960+0000","last_became_peered":"2023-11-06T03:16:43.479960+0000","last_unstale":"2023-11-06T03:16:43.486809+0000","last_undegraded":"2023-11-06T03:16:43.486809+0000","last_fullsized":"2023-11-06T03:16:43.486809+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:17:08.363447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.459627+0000","last_change":"2023-11-06T03:16:43.082428+0000","last_active":"2023-11-06T03:16:43.459627+0000","last_peered":"2023-11-06T03:16:43.459627+0000","last_clean":"2023-11-06T03:16:43.459627+0000","last_became_active":"2023-11-06T03:16:43.081818+0000","last_became_peered":"2023-11-06T03:16:43.081818+0000","last_unstale":"2023-11-06T03:16:43.459627+0000","last_undegraded":"2023-11-06T03:16:43.459627+0000","last_fullsized":"2023-11-06T03:16:43.459627+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:53:04.874498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.090485+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:16:43.090485+0000","last_peered":"2023-11-06T03:16:43.090485+0000","last_clean":"2023-11-06T03:16:43.090485+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:16:43.090485+0000","last_undegraded":"2023-11-06T03:16:43.090485+0000","last_fullsized":"2023-11-06T03:16:43.090485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.483745+0000","last_change":"2023-11-06T03:16:43.482676+0000","last_active":"2023-11-06T03:16:43.483745+0000","last_peered":"2023-11-06T03:16:43.483745+0000","last_clean":"2023-11-06T03:16:43.483745+0000","last_became_active":"2023-11-06T03:16:43.482379+0000","last_became_peered":"2023-11-06T03:16:43.482379+0000","last_unstale":"2023-11-06T03:16:43.483745+0000","last_undegraded":"2023-11-06T03:16:43.483745+0000","last_fullsized":"2023-11-06T03:16:43.483745+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:05.234569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.463067+0000","last_change":"2023-11-06T03:16:43.103738+0000","last_active":"2023-11-06T03:16:43.463067+0000","last_peered":"2023-11-06T03:16:43.463067+0000","last_clean":"2023-11-06T03:16:43.463067+0000","last_became_active":"2023-11-06T03:16:43.103042+0000","last_became_peered":"2023-11-06T03:16:43.103042+0000","last_unstale":"2023-11-06T03:16:43.463067+0000","last_undegraded":"2023-11-06T03:16:43.463067+0000","last_fullsized":"2023-11-06T03:16:43.463067+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:14.765834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.075588+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:16:43.075588+0000","last_peered":"2023-11-06T03:16:43.075588+0000","last_clean":"2023-11-06T03:16:43.075588+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:16:43.075588+0000","last_undegraded":"2023-11-06T03:16:43.075588+0000","last_fullsized":"2023-11-06T03:16:43.075588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.468085+0000","last_change":"2023-11-06T03:16:43.465362+0000","last_active":"2023-11-06T03:16:43.468085+0000","last_peered":"2023-11-06T03:16:43.468085+0000","last_clean":"2023-11-06T03:16:43.468085+0000","last_became_active":"2023-11-06T03:16:43.463856+0000","last_became_peered":"2023-11-06T03:16:43.463856+0000","last_unstale":"2023-11-06T03:16:43.468085+0000","last_undegraded":"2023-11-06T03:16:43.468085+0000","last_fullsized":"2023-11-06T03:16:43.468085+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:13:49.045571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"162'8","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.491509+0000","last_change":"2023-11-06T03:16:43.100740+0000","last_active":"2023-11-06T03:16:43.491509+0000","last_peered":"2023-11-06T03:16:43.491509+0000","last_clean":"2023-11-06T03:16:43.491509+0000","last_became_active":"2023-11-06T03:16:43.100459+0000","last_became_peered":"2023-11-06T03:16:43.100459+0000","last_unstale":"2023-11-06T03:16:43.491509+0000","last_undegraded":"2023-11-06T03:16:43.491509+0000","last_fullsized":"2023-11-06T03:16:43.491509+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:41:50.341023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.099887+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:16:43.099887+0000","last_peered":"2023-11-06T03:16:43.099887+0000","last_clean":"2023-11-06T03:16:43.099887+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:16:43.099887+0000","last_undegraded":"2023-11-06T03:16:43.099887+0000","last_fullsized":"2023-11-06T03:16:43.099887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.485209+0000","last_change":"2023-11-06T03:16:43.483997+0000","last_active":"2023-11-06T03:16:43.485209+0000","last_peered":"2023-11-06T03:16:43.485209+0000","last_clean":"2023-11-06T03:16:43.485209+0000","last_became_active":"2023-11-06T03:16:43.483497+0000","last_became_peered":"2023-11-06T03:16:43.483497+0000","last_unstale":"2023-11-06T03:16:43.485209+0000","last_undegraded":"2023-11-06T03:16:43.485209+0000","last_fullsized":"2023-11-06T03:16:43.485209+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:19.842451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.469650+0000","last_change":"2023-11-06T03:16:43.103535+0000","last_active":"2023-11-06T03:16:43.469650+0000","last_peered":"2023-11-06T03:16:43.469650+0000","last_clean":"2023-11-06T03:16:43.469650+0000","last_became_active":"2023-11-06T03:16:43.103216+0000","last_became_peered":"2023-11-06T03:16:43.103216+0000","last_unstale":"2023-11-06T03:16:43.469650+0000","last_undegraded":"2023-11-06T03:16:43.469650+0000","last_fullsized":"2023-11-06T03:16:43.469650+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:07.119775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.076953+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:16:43.076953+0000","last_peered":"2023-11-06T03:16:43.076953+0000","last_clean":"2023-11-06T03:16:43.076953+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:16:43.076953+0000","last_undegraded":"2023-11-06T03:16:43.076953+0000","last_fullsized":"2023-11-06T03:16:43.076953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.493114+0000","last_change":"2023-11-06T03:16:43.484970+0000","last_active":"2023-11-06T03:16:43.493114+0000","last_peered":"2023-11-06T03:16:43.493114+0000","last_clean":"2023-11-06T03:16:43.493114+0000","last_became_active":"2023-11-06T03:16:43.484301+0000","last_became_peered":"2023-11-06T03:16:43.484301+0000","last_unstale":"2023-11-06T03:16:43.493114+0000","last_undegraded":"2023-11-06T03:16:43.493114+0000","last_fullsized":"2023-11-06T03:16:43.493114+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:43.057292+0000","last_clean_scrub_stamp":"2023-11-06T03:16:43.057292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:46:25.861365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.454607+0000","last_change":"2023-11-06T03:16:43.094062+0000","last_active":"2023-11-06T03:16:43.454607+0000","last_peered":"2023-11-06T03:16:43.454607+0000","last_clean":"2023-11-06T03:16:43.454607+0000","last_became_active":"2023-11-06T03:16:43.093750+0000","last_became_peered":"2023-11-06T03:16:43.093750+0000","last_unstale":"2023-11-06T03:16:43.454607+0000","last_undegraded":"2023-11-06T03:16:43.454607+0000","last_fullsized":"2023-11-06T03:16:43.454607+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:16:42.053952+0000","last_clean_scrub_stamp":"2023-11-06T03:16:42.053952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:26:47.002128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-11-06T03:16:43.097221+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:16:43.097221+0000","last_peered":"2023-11-06T03:16:43.097221+0000","last_clean":"2023-11-06T03:16:43.097221+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:16:43.097221+0000","last_undegraded":"2023-11-06T03:16:43.097221+0000","last_fullsized":"2023-11-06T03:16:43.097221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91776,"kb_used_data":2552,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94280064,"statfs":{"total":96636764160,"available":96542785536,"internally_reserved":0,"allocated":2613248,"data_stored":1526339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.81000000000000005,"15min":0.80800000000000005},"min":{"1min":0.61099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.78600000000000003,"15min":0.82899999999999996},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.397},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82999999999999996,"15min":0.83399999999999996},"min":{"1min":0.501,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.1180000000000001,"5min":1.627,"15min":1.627},"last":0.62},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.137,"5min":1.218,"15min":1.6899999999999999},"last":0.92100000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.873,"15min":0.88300000000000001},"min":{"1min":0.628,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.6080000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.872,"15min":0.90300000000000002},"min":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.48999999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85999999999999999,"15min":0.88600000000000001},"min":{"1min":0.58599999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.103,"5min":1.4099999999999999,"15min":2.512},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95399999999999996,"15min":1.034},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.60099999999999998},"max":{"1min":1.319,"5min":1.431,"15min":10.596},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.93700000000000006,"15min":0.95599999999999996},"min":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.499},"max":{"1min":1.163,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.877},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.95299999999999996,"15min":0.98099999999999998},"min":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.47399999999999998},"max":{"1min":1.304,"5min":1.54,"15min":1.9399999999999999},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0249999999999999,"15min":1.042},"min":{"1min":0.71199999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.0589999999999999,"15min":1.0740000000000001},"min":{"1min":0.68400000000000005,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2929999999999999,"5min":1.8460000000000001,"15min":1.8839999999999999},"last":0.79000000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:16:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.179,"15min":1.159},"min":{"1min":0.82499999999999996,"5min":0.81699999999999995,"15min":0.60799999999999998},"max":{"1min":1.3460000000000001,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":0.75},{"interface":"front","average":{"1min":1.016,"5min":1.042,"15min":1.0880000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.246,"5min":1.7370000000000001,"15min":2.5430000000000001},"last":0.94699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69060,"kb_used_data":1980,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302780,"statfs":{"total":96636764160,"available":96566046720,"internally_reserved":0,"allocated":2027520,"data_stored":942080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.86099999999999999},"min":{"1min":0.50600000000000001,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.1830000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.83999999999999997,"15min":0.98099999999999998},"min":{"1min":0.56100000000000005,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.581,"5min":2.1850000000000001,"15min":40.905000000000001},"last":0.90000000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.90800000000000003},"min":{"1min":0.53300000000000003,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.1719999999999999,"5min":1.351,"15min":16.012},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.873,"15min":0.88},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.3400000000000001,"5min":1.409,"15min":3.3050000000000002},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.92100000000000004},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.175,"5min":1.702,"15min":27.803999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.248,"5min":1.429,"15min":2.843},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93899999999999995,"15min":1.1759999999999999},"min":{"1min":0.64700000000000002,"5min":0.61699999999999999,"15min":0.48699999999999999},"max":{"1min":1.204,"5min":1.306,"15min":63.737000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":1.0600000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.1399999999999999,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.99299999999999999,"15min":1.151},"min":{"1min":0.70699999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.2330000000000001,"5min":1.512,"15min":24.867999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1100000000000001,"15min":1.2250000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.65800000000000003},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.0580000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.97899999999999998,"15min":1.1459999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.206,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0820000000000001,"15min":1.115},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3200000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":0.86099999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.131,"15min":1.2130000000000001},"min":{"1min":0.79700000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5309999999999999,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.137,"15min":1.2230000000000001},"min":{"1min":0.85999999999999999,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.331,"5min":5.2709999999999999,"15min":28.164000000000001},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46992,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324848,"statfs":{"total":96636764160,"available":96588644352,"internally_reserved":0,"allocated":2039808,"data_stored":943223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.80600000000000005,"15min":1.133},"min":{"1min":0.58499999999999996,"5min":0.309,"15min":0.309},"max":{"1min":1.123,"5min":4.0510000000000002,"15min":125.256},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.127,"15min":1.2629999999999999},"min":{"1min":0.52400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.7050000000000001,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.91400000000000003,"15min":0.83299999999999996},"min":{"1min":0.65900000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":4.8490000000000002,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.306,"15min":0.998},"min":{"1min":0.60699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.6459999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":1.3400000000000001,"15min":1.0149999999999999},"min":{"1min":0.60999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.3959999999999999,"5min":43.741,"15min":43.741},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.86299999999999999,"15min":0.84899999999999998},"min":{"1min":0.60499999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2470000000000001,"5min":1.385,"15min":2.9700000000000002},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.96299999999999997,"15min":0.90300000000000002},"min":{"1min":0.629,"5min":0.41899999999999998,"15min":0.40400000000000003},"max":{"1min":4.7800000000000002,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.4370000000000001,"15min":1.111},"min":{"1min":0.57599999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.796,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.107}]},{"osd":4,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.90500000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":1.595,"5min":1.595,"15min":3.1349999999999998},"last":1.333},{"interface":"front","average":{"1min":1.163,"5min":1.1200000000000001,"15min":1.0269999999999999},"min":{"1min":0.54000000000000004,"5min":0.46600000000000003,"15min":0.33300000000000002},"max":{"1min":4.6200000000000001,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.393}]},{"osd":6,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0409999999999999,"15min":1.026},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.378},"max":{"1min":1.72,"5min":2,"15min":2.6909999999999998},"last":1.365},{"interface":"front","average":{"1min":1.044,"5min":1.077,"15min":1.0469999999999999},"min":{"1min":0.69099999999999995,"5min":0.432,"15min":0.35699999999999998},"max":{"1min":1.3979999999999999,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.413}]},{"osd":7,"last update":"Mon Nov 6 03:16:21 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.177,"15min":1.0780000000000001},"min":{"1min":0.59099999999999997,"5min":0.442,"15min":0.42699999999999999},"max":{"1min":4.9160000000000004,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.262},{"interface":"front","average":{"1min":1.248,"5min":1.1719999999999999,"15min":1.121},"min":{"1min":0.65300000000000002,"5min":0.40899999999999997,"15min":0.38700000000000001},"max":{"1min":3.738,"5min":3.738,"15min":3.738},"last":1.45}]}]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113340,"kb_used_data":1972,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258500,"statfs":{"total":96636764160,"available":96520704000,"internally_reserved":0,"allocated":2019328,"data_stored":935971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79500000000000004,"15min":0.95499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.258,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.79600000000000004,"15min":0.96099999999999997},"min":{"1min":0.505,"5min":0.308,"15min":0.308},"max":{"1min":1.4750000000000001,"5min":3.0830000000000002,"15min":44.609000000000002},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.77400000000000002},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2350000000000001,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.89800000000000002,"15min":0.85099999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3109999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.90000000000000002,"15min":0.94099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2410000000000001,"5min":1.452,"15min":3.2879999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.90300000000000002,"15min":0.86699999999999999},"min":{"1min":0.65100000000000002,"5min":0.47499999999999998,"15min":0.45900000000000002},"max":{"1min":1.27,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90700000000000003,"15min":0.91700000000000004},"min":{"1min":0.61899999999999999,"5min":0.53500000000000003,"15min":0.47599999999999998},"max":{"1min":1.3660000000000001,"5min":3.198,"15min":3.198},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.90900000000000003,"15min":0.90100000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4510000000000001,"5min":1.5980000000000001,"15min":2.4500000000000002},"last":0.83499999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2210000000000001,"5min":10.192,"15min":10.192},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.0780000000000001,"15min":1.0800000000000001},"min":{"1min":0.60299999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.494,"5min":5.3310000000000004,"15min":5.3310000000000004},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6140000000000001,"5min":1.6779999999999999,"15min":2.5099999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0840000000000001,"15min":1.099},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.92500000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.095,"15min":1.0820000000000001},"min":{"1min":0.72599999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6000000000000001,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1140000000000001,"15min":1.1659999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6819999999999999,"5min":1.7490000000000001,"15min":13.582000000000001},"last":1.0389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83096,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288744,"statfs":{"total":96636764160,"available":96551673856,"internally_reserved":0,"allocated":2048000,"data_stored":944248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.27400000000000002,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.78100000000000003,"5min":0.96799999999999997,"15min":1.4199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.29999999999999999,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":1.524},"last":0.33800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.251,"15min":0.224},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.371},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.089,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":1.052,"5min":1.421,"15min":1.532},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.70499999999999996},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.0109999999999999,"5min":1.206,"15min":1.284},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82499999999999996,"15min":0.86499999999999999},"min":{"1min":0.65500000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.034,"5min":1.2609999999999999,"15min":1.7989999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.878,"15min":0.91200000000000003},"min":{"1min":0.628,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.218,"5min":1.4610000000000001,"15min":1.8300000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.93600000000000005,"15min":0.95599999999999996},"min":{"1min":0.68100000000000005,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.4430000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"min":{"1min":0.61499999999999999,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.486,"5min":1.573,"15min":1.698},"last":0.83399999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94599999999999995,"15min":0.97399999999999998},"min":{"1min":0.70599999999999996,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.367,"5min":1.51,"15min":1.8620000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.92200000000000004,"15min":0.95599999999999996},"min":{"1min":0.623,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.3140000000000001,"5min":1.3660000000000001,"15min":1.8740000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.97399999999999998,"15min":1.006},"min":{"1min":0.71899999999999997,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.387,"5min":1.4410000000000001,"15min":1.835},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0029999999999999,"15min":1.03},"min":{"1min":0.75600000000000001,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.3600000000000001,"5min":1.569,"15min":1.8979999999999999},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73160,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298680,"statfs":{"total":96636764160,"available":96561848320,"internally_reserved":0,"allocated":2031616,"data_stored":943099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53000000000000003,"15min":0.53700000000000003},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.53300000000000003,"15min":0.58899999999999997},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.48399999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.60999999999999999},"min":{"1min":0.34200000000000003,"5min":0.28499999999999998,"15min":0.23000000000000001},"max":{"1min":1.002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58599999999999997,"15min":0.62},"min":{"1min":0.27800000000000002,"5min":0.245,"15min":0.22800000000000001},"max":{"1min":0.89400000000000002,"5min":1.0509999999999999,"15min":1.1579999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63900000000000001,"15min":0.65700000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.121,"5min":1.1459999999999999,"15min":1.294},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.626,"15min":0.68500000000000005},"min":{"1min":0.35899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.107,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.81299999999999994,"15min":0.84799999999999998},"min":{"1min":0.60999999999999999,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.0840000000000001,"5min":1.2010000000000001,"15min":1.526},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.85799999999999998,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.1930000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.87}]},{"osd":5,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83199999999999996,"15min":0.84999999999999998},"min":{"1min":0.58699999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.1699999999999999,"5min":1.3300000000000001,"15min":1.6200000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.86599999999999999,"15min":0.91500000000000004},"min":{"1min":0.64200000000000002,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.298,"5min":1.355,"15min":1.5780000000000001},"last":1.032}]},{"osd":6,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.93300000000000005,"15min":0.96599999999999997},"min":{"1min":0.73799999999999999,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.2689999999999999,"5min":1.542,"15min":1.615},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.93200000000000005},"min":{"1min":0.63,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.22,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.752}]},{"osd":7,"last update":"Mon Nov 6 03:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.90400000000000003,"15min":0.95899999999999996},"min":{"1min":0.73499999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3380000000000001,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.89400000000000002,"15min":0.92300000000000004},"min":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1699999999999999,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69076,"kb_used_data":1996,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302764,"statfs":{"total":96636764160,"available":96566030336,"internally_reserved":0,"allocated":2043904,"data_stored":944663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.90200000000000002,"5min":1.1240000000000001,"15min":1.843},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.59099999999999997},"min":{"1min":0.28299999999999997,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.90100000000000002,"5min":1.244,"15min":2.1150000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.56399999999999995,"15min":0.69499999999999995},"min":{"1min":0.30399999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.91000000000000003,"5min":1.143,"15min":30.861000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.61599999999999999,"15min":0.64200000000000002},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":1.117,"5min":1.1419999999999999,"15min":1.7330000000000001},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.32200000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.9339999999999999},"last":1.016},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.377,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.284,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.98299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":2.258},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.83699999999999997,"15min":0.92200000000000004},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":14.753},"last":1.135}]},{"osd":5,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.876,"15min":1.0780000000000001},"min":{"1min":0.74299999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2649999999999999,"5min":1.351,"15min":43.103999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.96299999999999997,"15min":1.115},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.468,"5min":1.468,"15min":44.753999999999998},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.91200000000000003,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":23.971},"last":1.2789999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.90300000000000002,"15min":1.034},"min":{"1min":0.53500000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3979999999999999,"5min":1.444,"15min":14.813000000000001},"last":1.212}]},{"osd":7,"last update":"Mon Nov 6 03:16:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.55,"5min":1.581,"15min":2.2799999999999998},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.97299999999999998,"15min":1.0089999999999999},"min":{"1min":0.77400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4059999999999999,"5min":1.835,"15min":2.3559999999999999},"last":1.347}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95952,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275888,"statfs":{"total":96636764160,"available":96538509312,"internally_reserved":0,"allocated":2629632,"data_stored":1532552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.56000000000000005,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.214,"15min":0.19600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":2.6989999999999998},"last":0.437},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.61599999999999999,"15min":0.64700000000000002},"min":{"1min":0.44400000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.54700000000000004,"5min":0.24199999999999999,"15min":0.20599999999999999},"max":{"1min":1.014,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.52100000000000002,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.083,"5min":1.3140000000000001,"15min":4.6920000000000002},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.41199999999999998,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.69899999999999995,"15min":0.72099999999999997},"min":{"1min":0.54800000000000004,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.2829999999999999,"5min":1.391,"15min":2.7999999999999998},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.85399999999999998,"15min":0.90300000000000002},"min":{"1min":0.67900000000000005,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.1870000000000001,"5min":1.371,"15min":4.641},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.79000000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":2.028},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.91200000000000003,"15min":0.95099999999999996},"min":{"1min":0.77300000000000002,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.492,"5min":1.492,"15min":2.8559999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.70699999999999996,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.377,"5min":1.496,"15min":4.7430000000000003},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.84199999999999997,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":2.931},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.68999999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.575,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.94999999999999996,"15min":0.98899999999999999},"min":{"1min":0.77800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.381,"5min":1.381,"15min":4.8339999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.089,"5min":0.94399999999999995,"15min":0.97399999999999998},"min":{"1min":0.78200000000000003,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.3260000000000001,"5min":1.335,"15min":1.97},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:16:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:16:46.855 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:16:47.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 Processor -- start 2023-11-06T03:16:47.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 -- start start 2023-11-06T03:16:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f73341551e0 0x7f733406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733415f580 con 0x7f733415e5e0 2023-11-06T03:16:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733415efc0 con 0x7f73340b3b20 2023-11-06T03:16:47.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f73390d1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733415f2a0 con 0x7f73341551e0 2023-11-06T03:16:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f733415e5e0 0x7f73340b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45102/0 (socket says 172.21.15.157:45102) 2023-11-06T03:16:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733259c640 1 -- 172.21.15.157:0/1012804200 learned_addr learned my addr 172.21.15.157:0/1012804200 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7332d9d640 1 --2- 172.21.15.157:0/1012804200 >> v2:172.21.15.195:3300/0 conn(0x7f73341551e0 0x7f733406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650634464 0 0) 0x7f733415f580 con 0x7f733415e5e0 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7320003530 con 0x7f733415e5e0 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357893328 0 0) 0x7f7320003530 con 0x7f733415e5e0 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 >> v2:172.21.15.195:3300/0 conn(0x7f73341551e0 msgr2=0x7f733406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 --2- 172.21.15.157:0/1012804200 >> v2:172.21.15.195:3300/0 conn(0x7f73341551e0 0x7f733406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 >> v1:172.21.15.195:6789/0 conn(0x7f73340b3b20 legacy=0x7f733415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733406f5a0 con 0x7f733415e5e0 2023-11-06T03:16:47.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f731c001680 con 0x7f733415e5e0 2023-11-06T03:16:47.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 >> v1:172.21.15.157:6789/0 conn(0x7f733415e5e0 legacy=0x7f73340b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7320003980 0x7f7320003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200047d0 0x7f7320004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 0x7f732000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733415f2a0 con 0x7f73200052b0 2023-11-06T03:16:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732000c180 con 0x7f73200047d0 2023-11-06T03:16:47.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732000c340 con 0x7f7320003980 2023-11-06T03:16:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733359e640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 0x7f732000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733259c640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200047d0 0x7f7320004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f7332d9d640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7320003980 0x7f7320003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733359e640 1 -- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7320003980 msgr2=0x7f7320003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:47.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.027+0000 7f733359e640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7320003980 0x7f7320003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f733359e640 1 -- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200047d0 msgr2=0x7f7320004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f733359e640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200047d0 0x7f7320004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f733359e640 1 -- 172.21.15.157:0/1012804200 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7320003530 con 0x7f73200052b0 2023-11-06T03:16:47.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f733359e640 1 -- 172.21.15.157:0/1012804200 reap_dead start 2023-11-06T03:16:47.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f733359e640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 0x7f732000b990 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f732c001e40 tx=0x7f732c01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a3ce774c03ea83 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c003b80 con 0x7f73200052b0 2023-11-06T03:16:47.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c003ce0 con 0x7f73200052b0 2023-11-06T03:16:47.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f7331d9b640 1 -- 172.21.15.157:0/1012804200 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c005260 con 0x7f73200052b0 2023-11-06T03:16:47.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 -- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 msgr2=0x7f732000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 0x7f732000b990 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f732c001e40 tx=0x7f732c01c710 comp rx=0 tx=0).stop 2023-11-06T03:16:47.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 -- 172.21.15.157:0/1012804200 shutdown_connections 2023-11-06T03:16:47.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 --2- 172.21.15.157:0/1012804200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f73200052b0 0x7f732000b990 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 -- 172.21.15.157:0/1012804200 >> 172.21.15.157:0/1012804200 conn(0x7f7334065930 msgr2=0x7f7334065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:47.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 -- 172.21.15.157:0/1012804200 shutdown_connections 2023-11-06T03:16:47.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.031+0000 7f73390d1640 1 -- 172.21.15.157:0/1012804200 wait complete. 2023-11-06T03:16:47.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 Processor -- start 2023-11-06T03:16:47.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- start start 2023-11-06T03:16:47.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200052b0 0x7f73340ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f733414c640 0x7f7334159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733414a180 con 0x7f7334148480 2023-11-06T03:16:47.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733414bb60 con 0x7f73200052b0 2023-11-06T03:16:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733414be40 con 0x7f733414c640 2023-11-06T03:16:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38888/0 (socket says 172.21.15.157:38888) 2023-11-06T03:16:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 -- 172.21.15.157:0/704054536 learned_addr learned my addr 172.21.15.157:0/704054536 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:47.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f7332d9d640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200052b0 0x7f73340ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733259c640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f733414c640 0x7f7334159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 -- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f733414c640 msgr2=0x7f7334159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f733414c640 0x7f7334159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 -- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200052b0 msgr2=0x7f73340ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200052b0 0x7f73340ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 -- 172.21.15.157:0/704054536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733414c120 con 0x7f7334148480 2023-11-06T03:16:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f733359e640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f732c01c540 tx=0x7f7320005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c005720 con 0x7f7334148480 2023-11-06T03:16:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c003980 con 0x7f7334148480 2023-11-06T03:16:47.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7334149e10 con 0x7f7334148480 2023-11-06T03:16:47.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c005880 con 0x7f7334148480 2023-11-06T03:16:47.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733415a170 con 0x7f7334148480 2023-11-06T03:16:47.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733414b970 con 0x7f7334148480 2023-11-06T03:16:47.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f732c0a6410 con 0x7f7334148480 2023-11-06T03:16:47.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 0x7f7314067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.035+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f732c11c340 con 0x7f7334148480 2023-11-06T03:16:47.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.039+0000 7f7332d9d640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 0x7f7314067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.039+0000 7f7332d9d640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 0x7f7314067f20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f73240099b0 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.039+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f732c0e9ba0 con 0x7f7334148480 2023-11-06T03:16:47.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.255+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7334072750 con 0x7f7334148480 2023-11-06T03:16:47.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.263+0000 7f732b7fe640 1 -- 172.21.15.157:0/704054536 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f732c0ed1c0 con 0x7f7334148480 2023-11-06T03:16:47.261 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:16:47.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 msgr2=0x7f7314067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 0x7f7314067f20 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f73240099b0 tx=0x7f7324008040 comp rx=0 tx=0).stop 2023-11-06T03:16:47.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 msgr2=0x7f7334139ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f732c01c540 tx=0x7f7320005800 comp rx=0 tx=0).stop 2023-11-06T03:16:47.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 shutdown_connections 2023-11-06T03:16:47.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7314065a60 0x7f7314067f20 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7334148480 0x7f7334139ed0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f733414c640 0x7f7334159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 --2- 172.21.15.157:0/704054536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f73200052b0 0x7f73340ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 >> 172.21.15.157:0/704054536 conn(0x7f7334065930 msgr2=0x7f733415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:47.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 shutdown_connections 2023-11-06T03:16:47.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.267+0000 7f73390d1640 1 -- 172.21.15.157:0/704054536 wait complete. 2023-11-06T03:16:47.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:47.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.399+0000 7fd1895dd640 1 Processor -- start 2023-11-06T03:16:47.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.399+0000 7fd1895dd640 1 -- start start 2023-11-06T03:16:47.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1895dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd1840b3b20 0x7fd18415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1895dd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18415f580 con 0x7fd1841551e0 2023-11-06T03:16:47.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1895dd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18415efc0 con 0x7fd18415e5e0 2023-11-06T03:16:47.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1895dd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1840704b0 con 0x7fd1840b3b20 2023-11-06T03:16:47.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd182ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd1841551e0 0x7fd18406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45106/0 (socket says 172.21.15.157:45106) 2023-11-06T03:16:47.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd182ffd640 1 -- 172.21.15.157:0/2742586342 learned_addr learned my addr 172.21.15.157:0/2742586342 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:47.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1837fe640 1 --2- 172.21.15.157:0/2742586342 >> v2:172.21.15.157:3300/0 conn(0x7fd1840b3b20 0x7fd18415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd181ffb640 1 -- 172.21.15.157:0/2742586342 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084766149 0 0) 0x7fd18415f580 con 0x7fd1841551e0 2023-11-06T03:16:47.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd181ffb640 1 -- 172.21.15.157:0/2742586342 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003530 con 0x7fd1841551e0 2023-11-06T03:16:47.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1837fe640 1 -- 172.21.15.157:0/2742586342 >> v1:172.21.15.195:6789/0 conn(0x7fd18415e5e0 legacy=0x7fd1840b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1837fe640 1 -- 172.21.15.157:0/2742586342 >> v1:172.21.15.157:6789/0 conn(0x7fd1841551e0 legacy=0x7fd18406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1837fe640 1 -- 172.21.15.157:0/2742586342 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18415f2a0 con 0x7fd1840b3b20 2023-11-06T03:16:47.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd1837fe640 1 --2- 172.21.15.157:0/2742586342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1840b3b20 0x7fd18415c1b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d90 tx=0x7fd17c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7648b532e5bf3b51 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd181ffb640 1 -- 172.21.15.157:0/2742586342 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a58c0 con 0x7fd1840b3b20 2023-11-06T03:16:47.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd181ffb640 1 -- 172.21.15.157:0/2742586342 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0a5a20 con 0x7fd1840b3b20 2023-11-06T03:16:47.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.403+0000 7fd181ffb640 1 -- 172.21.15.157:0/2742586342 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a5db0 con 0x7fd1840b3b20 2023-11-06T03:16:47.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/2742586342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1840b3b20 msgr2=0x7fd18415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 --2- 172.21.15.157:0/2742586342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1840b3b20 0x7fd18415c1b0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d90 tx=0x7fd17c01c800 comp rx=0 tx=0).stop 2023-11-06T03:16:47.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/2742586342 shutdown_connections 2023-11-06T03:16:47.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 --2- 172.21.15.157:0/2742586342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1840b3b20 0x7fd18415c1b0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/2742586342 >> 172.21.15.157:0/2742586342 conn(0x7fd184065930 msgr2=0x7fd184065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:47.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/2742586342 shutdown_connections 2023-11-06T03:16:47.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/2742586342 wait complete. 2023-11-06T03:16:47.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 Processor -- start 2023-11-06T03:16:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- start start 2023-11-06T03:16:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1840b3b20 0x7fd1840b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 0x7fd1840ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840b2620 con 0x7fd1841551e0 2023-11-06T03:16:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18414c7d0 con 0x7fd1840b3b20 2023-11-06T03:16:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd184151620 con 0x7fd18415e5e0 2023-11-06T03:16:47.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38900/0 (socket says 172.21.15.157:38900) 2023-11-06T03:16:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 -- 172.21.15.157:0/703927155 learned_addr learned my addr 172.21.15.157:0/703927155 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd182ffd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1840b3b20 0x7fd1840b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1837fe640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 0x7fd1840ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 -- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 msgr2=0x7fd1840ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 0x7fd1840ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 -- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1840b3b20 msgr2=0x7fd1840b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1840b3b20 0x7fd1840b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 -- 172.21.15.157:0/703927155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009a70 con 0x7fd1841551e0 2023-11-06T03:16:47.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1837fe640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 0x7fd1840ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:47.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1827fc640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c530 tx=0x7fd17800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178005220 con 0x7fd1841551e0 2023-11-06T03:16:47.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178005380 con 0x7fd1841551e0 2023-11-06T03:16:47.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178007270 con 0x7fd1841551e0 2023-11-06T03:16:47.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18414cd00 con 0x7fd1841551e0 2023-11-06T03:16:47.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.407+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd184159f90 con 0x7fd1841551e0 2023-11-06T03:16:47.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184072750 con 0x7fd1841551e0 2023-11-06T03:16:47.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd1780073d0 con 0x7fd1841551e0 2023-11-06T03:16:47.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd16b7fe640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 0x7fd170067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:47.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd178082b70 con 0x7fd1841551e0 2023-11-06T03:16:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd182ffd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 0x7fd170067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.411+0000 7fd182ffd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 0x7fd170067f20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd1740011f0 tx=0x7fd174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:47.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.415+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd178050460 con 0x7fd1841551e0 2023-11-06T03:16:47.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.603+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd184151760 con 0x7fd1841551e0 2023-11-06T03:16:47.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.603+0000 7fd16b7fe640 1 -- 172.21.15.157:0/703927155 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7fd178053a80 con 0x7fd1841551e0 2023-11-06T03:16:47.602 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:47.603 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:16:47.603 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:16:47.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 msgr2=0x7fd170067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 0x7fd170067f20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd1740011f0 tx=0x7fd174009040 comp rx=0 tx=0).stop 2023-11-06T03:16:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 msgr2=0x7fd18414acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c530 tx=0x7fd17800c9f0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 shutdown_connections 2023-11-06T03:16:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd170065a60 0x7fd170067f20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd18415e5e0 0x7fd1840ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1841551e0 0x7fd18414acb0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 --2- 172.21.15.157:0/703927155 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1840b3b20 0x7fd1840b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 >> 172.21.15.157:0/703927155 conn(0x7fd184065930 msgr2=0x7fd1840afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 shutdown_connections 2023-11-06T03:16:47.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:47.607+0000 7fd1895dd640 1 -- 172.21.15.157:0/703927155 wait complete. 2023-11-06T03:16:48.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 117, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:16:43.438847+0000', 'modified': '2023-11-06T03:16:45.454769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10392}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10392': {'gid': 10392, 'name': 'd', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6837/3466838470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6835', 'nonce': 3466838470}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 3466838470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:16:48.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:16:48.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-06T03:16:48.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 Processor -- start 2023-11-06T03:16:48.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 -- start start 2023-11-06T03:16:48.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f23700b2af0 0x7f23700ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237015e440 con 0x7f23701599c0 2023-11-06T03:16:48.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237006c240 con 0x7f237006d000 2023-11-06T03:16:48.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2375f26640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237006c520 con 0x7f23700b2af0 2023-11-06T03:16:48.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f236f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f23701599c0 0x7f23700b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45120/0 (socket says 172.21.15.157:45120) 2023-11-06T03:16:48.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f236f7fe640 1 -- 172.21.15.157:0/248451378 learned_addr learned my addr 172.21.15.157:0/248451378 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:48.160 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.159+0000 7f2366ffd640 1 --2- 172.21.15.157:0/248451378 >> v2:172.21.15.157:3300/0 conn(0x7f23700b2af0 0x7f23700ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f236effd640 1 -- 172.21.15.157:0/248451378 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900738629 0 0) 0x7f237015e440 con 0x7f23701599c0 2023-11-06T03:16:48.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f236effd640 1 -- 172.21.15.157:0/248451378 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c003530 con 0x7f23701599c0 2023-11-06T03:16:48.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2366ffd640 1 -- 172.21.15.157:0/248451378 >> v1:172.21.15.195:6789/0 conn(0x7f237006d000 legacy=0x7f2370159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2366ffd640 1 -- 172.21.15.157:0/248451378 >> v1:172.21.15.157:6789/0 conn(0x7f23701599c0 legacy=0x7f23700b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2366ffd640 1 -- 172.21.15.157:0/248451378 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237006c800 con 0x7f23700b2af0 2023-11-06T03:16:48.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2366ffd640 1 --2- 172.21.15.157:0/248451378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23700b2af0 0x7f23700ae000 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f23580097c0 tx=0x7f235801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8382a6b3e04dbc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f236effd640 1 -- 172.21.15.157:0/248451378 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580ab670 con 0x7f23700b2af0 2023-11-06T03:16:48.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f236effd640 1 -- 172.21.15.157:0/248451378 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580ab7d0 con 0x7f23700b2af0 2023-11-06T03:16:48.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f236effd640 1 -- 172.21.15.157:0/248451378 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580abb40 con 0x7f23700b2af0 2023-11-06T03:16:48.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 -- 172.21.15.157:0/248451378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23700b2af0 msgr2=0x7f23700ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 --2- 172.21.15.157:0/248451378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23700b2af0 0x7f23700ae000 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f23580097c0 tx=0x7f235801c450 comp rx=0 tx=0).stop 2023-11-06T03:16:48.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 -- 172.21.15.157:0/248451378 shutdown_connections 2023-11-06T03:16:48.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 --2- 172.21.15.157:0/248451378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23700b2af0 0x7f23700ae000 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 -- 172.21.15.157:0/248451378 >> 172.21.15.157:0/248451378 conn(0x7f2370065930 msgr2=0x7f2370065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 -- 172.21.15.157:0/248451378 shutdown_connections 2023-11-06T03:16:48.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.163+0000 7f2375f26640 1 -- 172.21.15.157:0/248451378 wait complete. 2023-11-06T03:16:48.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 Processor -- start 2023-11-06T03:16:48.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- start start 2023-11-06T03:16:48.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23700b2af0 0x7f23700cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23701599c0 0x7f2370153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700cdbf0 con 0x7f237006d000 2023-11-06T03:16:48.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2370145950 con 0x7f23700b2af0 2023-11-06T03:16:48.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23700ccf70 con 0x7f23701599c0 2023-11-06T03:16:48.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38916/0 (socket says 172.21.15.157:38916) 2023-11-06T03:16:48.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 -- 172.21.15.157:0/2077554645 learned_addr learned my addr 172.21.15.157:0/2077554645 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2366ffd640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23700b2af0 0x7f23700cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236ffff640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23701599c0 0x7f2370153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 -- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23701599c0 msgr2=0x7f2370153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23701599c0 0x7f2370153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 -- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23700b2af0 msgr2=0x7f23700cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23700b2af0 0x7f23700cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 -- 172.21.15.157:0/2077554645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009420 con 0x7f237006d000 2023-11-06T03:16:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f236000c420 tx=0x7f236000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236cff9640 1 -- 172.21.15.157:0/2077554645 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360005cf0 con 0x7f237006d000 2023-11-06T03:16:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236cff9640 1 -- 172.21.15.157:0/2077554645 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360005e50 con 0x7f237006d000 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23700cd250 con 0x7f237006d000 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236cff9640 1 -- 172.21.15.157:0/2077554645 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600072d0 con 0x7f237006d000 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23700cd810 con 0x7f237006d000 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.167+0000 7f236cff9640 1 -- 172.21.15.157:0/2077554645 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2360007430 con 0x7f237006d000 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236cff9640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 0x7f2350067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236cff9640 1 -- 172.21.15.157:0/2077554645 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23600827e0 con 0x7f237006d000 2023-11-06T03:16:48.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f2366ffd640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 0x7f2350067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 Processor -- start 2023-11-06T03:16:48.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- start start 2023-11-06T03:16:48.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f2366ffd640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 0x7f2350067f20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f23580af000 tx=0x7f23580a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c6355000 0x55e8c630a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c630ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c61d2fc0 con 0x55e8c63546e0 2023-11-06T03:16:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c61611a0 con 0x55e8c6355000 2023-11-06T03:16:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c615e770 con 0x55e8c630aa90 2023-11-06T03:16:48.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38928/0 (socket says 172.21.15.157:38928) 2023-11-06T03:16:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 -- 172.21.15.157:0/2499190688 learned_addr learned my addr 172.21.15.157:0/2499190688 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f2366ffd640 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c6355000 0x55e8c630a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 -- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 msgr2=0x55e8c630ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.180 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c630ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 -- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c6355000 msgr2=0x55e8c630a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c6355000 0x55e8c630a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 -- 172.21.15.157:0/2499190688 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c5e56890 con 0x55e8c63546e0 2023-11-06T03:16:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f236f7fe640 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f236005b7a0 tx=0x7f2360053ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a47f6366be2804 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f23657fa640 1 -- 172.21.15.157:0/2499190688 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600076e0 con 0x55e8c63546e0 2023-11-06T03:16:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f23657fa640 1 -- 172.21.15.157:0/2499190688 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360052140 con 0x55e8c63546e0 2023-11-06T03:16:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f23657fa640 1 -- 172.21.15.157:0/2499190688 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600504b0 con 0x55e8c63546e0 2023-11-06T03:16:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- 172.21.15.157:0/2499190688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 msgr2=0x55e8c6354ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f236005b7a0 tx=0x7f2360053ee0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- 172.21.15.157:0/2499190688 shutdown_connections 2023-11-06T03:16:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c630ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c6355000 0x55e8c630a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 --2- 172.21.15.157:0/2499190688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6354ac0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- 172.21.15.157:0/2499190688 >> 172.21.15.157:0/2499190688 conn(0x55e8c6310bf0 msgr2=0x55e8c6308e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- 172.21.15.157:0/2499190688 shutdown_connections 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.171+0000 7f237904e000 1 -- 172.21.15.157:0/2499190688 wait complete. 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- start start 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 Processor -- start 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c630aa90 0x55e8c6315880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c6316300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c615a060 con 0x55e8c63546e0 2023-11-06T03:16:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c60f9290 con 0x55e8c630aa90 2023-11-06T03:16:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c61176f0 con 0x55e8c6355000 2023-11-06T03:16:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 10 client.0 _ll_register_callbacks cb 0x55e8c62f61d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38930/0 (socket says 172.21.15.157:38930) 2023-11-06T03:16:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 -- 172.21.15.157:0/3123713074 learned_addr learned my addr 172.21.15.157:0/3123713074 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f236f7fe640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c630aa90 0x55e8c6315880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f236ffff640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c6316300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 -- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 msgr2=0x55e8c6316300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c6316300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 -- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c630aa90 msgr2=0x55e8c6315880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c630aa90 0x55e8c6315880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 -- 172.21.15.157:0/3123713074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23700cd810 con 0x55e8c63546e0 2023-11-06T03:16:48.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f2366ffd640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f235801c930 tx=0x7f235801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aed7bdf4f098a61 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358003b70 con 0x55e8c63546e0 2023-11-06T03:16:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2358003cd0 con 0x55e8c63546e0 2023-11-06T03:16:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8c61589b0 con 0x55e8c63546e0 2023-11-06T03:16:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2358003e90 con 0x55e8c63546e0 2023-11-06T03:16:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 10 client.10521 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55e8c611f1e0 con 0x55e8c63546e0 2023-11-06T03:16:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8c61589b0 con 0x55e8c63546e0 2023-11-06T03:16:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f23580cc470 con 0x55e8c63546e0 2023-11-06T03:16:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 10 client.10521 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 10 client.10521 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T03:16:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 20 client.10521 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c6363980 0x55e8c6365d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 4 client.10521 mds_command: new command op to 10392 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:16:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8c61894d0 con 0x55e8c6363980 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f236f7fe640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c6363980 0x55e8c6365d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f234e7fc640 10 client.10521 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.175+0000 7f236f7fe640 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c6363980 0x55e8c6365d70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755e2b9b9d8aee15 server_cookie=1e4a3612bb1fbb37 in_seq=0 out_seq=0 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f234e7fc640 1 -- 172.21.15.157:0/3123713074 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e8c61894d0 con 0x55e8c6363980 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f234e7fc640 10 client.10521 handle_command_reply: tid=0 2023-11-06T03:16:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 client.10521 shutdown 2023-11-06T03:16:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 msgr2=0x55e8c6315dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f235801c930 tx=0x7f235801ac90 comp rx=0 tx=0).stop 2023-11-06T03:16:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 shutdown_connections 2023-11-06T03:16:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c6363980 0x55e8c6365d70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c6316300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f234e7fc640 0 client.10521 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c63546e0 0x55e8c6315dc0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 --2- 172.21.15.157:0/3123713074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c630aa90 0x55e8c6315880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 >> 172.21.15.157:0/3123713074 conn(0x55e8c6310bf0 msgr2=0x55e8c6309530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 shutdown_connections 2023-11-06T03:16:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 1 -- 172.21.15.157:0/3123713074 wait complete. 2023-11-06T03:16:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.179+0000 7f237904e000 20 client.10521 trim_cache size 0 max 16384 2023-11-06T03:16:48.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 Processor -- start 2023-11-06T03:16:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- start start 2023-11-06T03:16:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c6353670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c6366d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c5e17940 con 0x55e8c6355000 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c615e060 con 0x55e8c63546e0 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c614ed30 con 0x55e8c630aa90 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38938/0 (socket says 172.21.15.157:38938) 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 -- 172.21.15.157:0/983100197 learned_addr learned my addr 172.21.15.157:0/983100197 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 -- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 msgr2=0x55e8c6353670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236f7fe640 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c6353670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c6353670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 -- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 msgr2=0x55e8c6366d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f2366ffd640 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c6366d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c6366d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 -- 172.21.15.157:0/983100197 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c6105330 con 0x55e8c6355000 2023-11-06T03:16:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f236ffff640 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f2368001aa0 tx=0x7f2368001f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cda57cc320c7cde server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f2365ffb640 1 -- 172.21.15.157:0/983100197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368003b90 con 0x55e8c6355000 2023-11-06T03:16:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f2365ffb640 1 -- 172.21.15.157:0/983100197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236800d050 con 0x55e8c6355000 2023-11-06T03:16:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f2365ffb640 1 -- 172.21.15.157:0/983100197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236800f7b0 con 0x55e8c6355000 2023-11-06T03:16:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- 172.21.15.157:0/983100197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 msgr2=0x55e8c63672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f2368001aa0 tx=0x7f2368001f60 comp rx=0 tx=0).stop 2023-11-06T03:16:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- 172.21.15.157:0/983100197 shutdown_connections 2023-11-06T03:16:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c6355000 0x55e8c63672c0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c6366d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 --2- 172.21.15.157:0/983100197 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c630aa90 0x55e8c6353670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- 172.21.15.157:0/983100197 >> 172.21.15.157:0/983100197 conn(0x55e8c6310bf0 msgr2=0x55e8c630faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- 172.21.15.157:0/983100197 shutdown_connections 2023-11-06T03:16:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- 172.21.15.157:0/983100197 wait complete. 2023-11-06T03:16:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 -- start start 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.195+0000 7f237904e000 1 Processor -- start 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c63b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c63b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c6154400 con 0x55e8c630aa90 2023-11-06T03:16:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c6155fd0 con 0x55e8c63546e0 2023-11-06T03:16:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c630df10 con 0x55e8c6355000 2023-11-06T03:16:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 10 client.0 _ll_register_callbacks cb 0x55e8c62d69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38950/0 (socket says 172.21.15.157:38950) 2023-11-06T03:16:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 -- 172.21.15.157:0/748570542 learned_addr learned my addr 172.21.15.157:0/748570542 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236ffff640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c63b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2366ffd640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c63b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 -- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 msgr2=0x55e8c63b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c63b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 -- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 msgr2=0x55e8c63b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c63b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 -- 172.21.15.157:0/748570542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237006c240 con 0x55e8c630aa90 2023-11-06T03:16:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2366ffd640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c63b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f236f7fe640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f236005be90 tx=0x7f236000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afb1db2eccae4b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:48.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360080040 con 0x55e8c630aa90 2023-11-06T03:16:48.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f236007f040 con 0x55e8c630aa90 2023-11-06T03:16:48.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8c6129b90 con 0x55e8c630aa90 2023-11-06T03:16:48.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2360053b70 con 0x55e8c630aa90 2023-11-06T03:16:48.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 10 client.10527 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55e8c5e56cc0 con 0x55e8c630aa90 2023-11-06T03:16:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8c6129b90 con 0x55e8c630aa90 2023-11-06T03:16:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f236006b170 con 0x55e8c630aa90 2023-11-06T03:16:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 10 client.10527 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:48.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 10 client.10527 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 20 client.10527 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c63b6860 0x55e8c63b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 4 client.10527 mds_command: new command op to 10392 tid=0[{"prefix": "status"}] 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8c61530e0 con 0x55e8c63b6860 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2366ffd640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c63b6860 0x55e8c63b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2366ffd640 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c63b6860 0x55e8c63b8c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45ea70c12db3a44 server_cookie=18109015ee8542b2 in_seq=0 out_seq=0 2023-11-06T03:16:48.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 10 client.10527 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:48.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 1 -- 172.21.15.157:0/748570542 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55e8c61530e0 con 0x55e8c63b6860 2023-11-06T03:16:48.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 10 client.10527 handle_command_reply: tid=0 2023-11-06T03:16:48.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 client.10527 shutdown 2023-11-06T03:16:48.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 msgr2=0x55e8c63b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f236005be90 tx=0x7f236000cf90 comp rx=0 tx=0).stop 2023-11-06T03:16:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 shutdown_connections 2023-11-06T03:16:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55e8c63b6860 0x55e8c63b8c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55e8c6355000 0x55e8c63b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e8c63546e0 0x55e8c63b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 --2- 172.21.15.157:0/748570542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8c630aa90 0x55e8c63b1880 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f234e7fc640 0 client.10527 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 >> 172.21.15.157:0/748570542 conn(0x55e8c6310bf0 msgr2=0x55e8c6365d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 shutdown_connections 2023-11-06T03:16:48.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 1 -- 172.21.15.157:0/748570542 wait complete. 2023-11-06T03:16:48.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f237904e000 20 client.10527 trim_cache size 0 max 16384 2023-11-06T03:16:48.216 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:16:48.216 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:16:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 10392, 2023-11-06T03:16:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:16:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:16:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:16:48.217 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.7617545149999998, 2023-11-06T03:16:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 117, 2023-11-06T03:16:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 163, 2023-11-06T03:16:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 163, 2023-11-06T03:16:48.218 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 9.1841636789999992 2023-11-06T03:16:48.218 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:48.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 msgr2=0x7f2350067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 0x7f2350067f20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f23580af000 tx=0x7f23580a7000 comp rx=0 tx=0).stop 2023-11-06T03:16:48.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 msgr2=0x7f2370145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f236000c420 tx=0x7f236000c8e0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 shutdown_connections 2023-11-06T03:16:48.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2350065a60 0x7f2350067f20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23701599c0 0x7f2370153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23700b2af0 0x7f23700cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 --2- 172.21.15.157:0/2077554645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f237006d000 0x7f2370145410 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.199+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 >> 172.21.15.157:0/2077554645 conn(0x7f2370065930 msgr2=0x7f2370156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.203+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 shutdown_connections 2023-11-06T03:16:48.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.203+0000 7f2375f26640 1 -- 172.21.15.157:0/2077554645 wait complete. 2023-11-06T03:16:48.222 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:16:48.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.331+0000 7f5010478640 1 Processor -- start 2023-11-06T03:16:48.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.331+0000 7f5010478640 1 -- start start 2023-11-06T03:16:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f500815ce80 0x7f50080cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 0x7f50080b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50080d2920 con 0x7f500815caa0 2023-11-06T03:16:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080d0520 con 0x7f500815ce80 2023-11-06T03:16:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080d18a0 con 0x7f50080d00b0 2023-11-06T03:16:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f500815ce80 0x7f50080cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f500815ce80 0x7f50080cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38962/0 (socket says 172.21.15.157:38962) 2023-11-06T03:16:48.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 -- 172.21.15.157:0/178828144 learned_addr learned my addr 172.21.15.157:0/178828144 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500ea15640 1 --2- 172.21.15.157:0/178828144 >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 0x7f50080b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 -- 172.21.15.157:0/178828144 >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 msgr2=0x7f50080b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 --2- 172.21.15.157:0/178828144 >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 0x7f50080b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 -- 172.21.15.157:0/178828144 >> v1:172.21.15.195:6789/0 conn(0x7f500815caa0 legacy=0x7f50080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:48.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 -- 172.21.15.157:0/178828144 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080d1b60 con 0x7f500815ce80 2023-11-06T03:16:48.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500ea15640 1 --2- 172.21.15.157:0/178828144 >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 0x7f50080b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500da13640 1 --2- 172.21.15.157:0/178828144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815ce80 0x7f50080cfb70 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f50080cf670 tx=0x7f4ff801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ef2af6381f7dc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500d212640 1 -- 172.21.15.157:0/178828144 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80aa5f0 con 0x7f500815ce80 2023-11-06T03:16:48.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500d212640 1 -- 172.21.15.157:0/178828144 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff80aa750 con 0x7f500815ce80 2023-11-06T03:16:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f500d212640 1 -- 172.21.15.157:0/178828144 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80aaae0 con 0x7f500815ce80 2023-11-06T03:16:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- 172.21.15.157:0/178828144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815ce80 msgr2=0x7f50080cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 --2- 172.21.15.157:0/178828144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815ce80 0x7f50080cfb70 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f50080cf670 tx=0x7f4ff801c490 comp rx=0 tx=0).stop 2023-11-06T03:16:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- 172.21.15.157:0/178828144 shutdown_connections 2023-11-06T03:16:48.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 --2- 172.21.15.157:0/178828144 >> v2:172.21.15.195:3300/0 conn(0x7f50080d00b0 0x7f50080b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 --2- 172.21.15.157:0/178828144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815ce80 0x7f50080cfb70 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- 172.21.15.157:0/178828144 >> 172.21.15.157:0/178828144 conn(0x7f5008070790 msgr2=0x7f5008070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- 172.21.15.157:0/178828144 shutdown_connections 2023-11-06T03:16:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.335+0000 7f5010478640 1 -- 172.21.15.157:0/178828144 wait complete. 2023-11-06T03:16:48.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 Processor -- start 2023-11-06T03:16:48.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- start start 2023-11-06T03:16:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 0x7f50081520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 0x7f500814cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008151c40 con 0x7f500815caa0 2023-11-06T03:16:48.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008150d90 con 0x7f500815ce80 2023-11-06T03:16:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008151030 con 0x7f50080d00b0 2023-11-06T03:16:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38966/0 (socket says 172.21.15.157:38966) 2023-11-06T03:16:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 -- 172.21.15.157:0/3835803702 learned_addr learned my addr 172.21.15.157:0/3835803702 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:48.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500ea15640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 0x7f500814cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500e214640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 0x7f50081520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 -- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 msgr2=0x7f50081520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 0x7f50081520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 -- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 msgr2=0x7f500814cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 0x7f500814cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 -- 172.21.15.157:0/3835803702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80094d0 con 0x7f500815caa0 2023-11-06T03:16:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500ea15640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 0x7f500814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500e214640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 0x7f50081520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f500da13640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f4ff801a0c0 tx=0x7f4ff801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80b3800 con 0x7f500815caa0 2023-11-06T03:16:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff80b3960 con 0x7f500815caa0 2023-11-06T03:16:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5008151310 con 0x7f500815caa0 2023-11-06T03:16:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80b3cf0 con 0x7f500815caa0 2023-11-06T03:16:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50081518d0 con 0x7f500815caa0 2023-11-06T03:16:48.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.339+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008151db0 con 0x7f500815caa0 2023-11-06T03:16:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.343+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4ff80b2380 con 0x7f500815caa0 2023-11-06T03:16:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.343+0000 7f4ffeffd640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 0x7f4fec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.343+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ff80a7070 con 0x7f500815caa0 2023-11-06T03:16:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.343+0000 7f500e214640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 0x7f4fec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:48.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.343+0000 7f500e214640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 0x7f4fec067f20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5008158ec0 tx=0x7f5004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:48.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.347+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4ff80eb270 con 0x7f500815caa0 2023-11-06T03:16:48.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.523+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5008072750 con 0x7f500815caa0 2023-11-06T03:16:48.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.523+0000 7f4ffeffd640 1 -- 172.21.15.157:0/3835803702 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v117) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4ff80ee890 con 0x7f500815caa0 2023-11-06T03:16:48.521 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:16:48.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 msgr2=0x7f4fec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 0x7f4fec067f20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5008158ec0 tx=0x7f5004009040 comp rx=0 tx=0).stop 2023-11-06T03:16:48.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 msgr2=0x7f5008153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:48.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f4ff801a0c0 tx=0x7f4ff801acd0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 shutdown_connections 2023-11-06T03:16:48.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4fec065a60 0x7f4fec067f20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f500815ce80 0x7f500814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f500815caa0 0x7f5008153370 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 --2- 172.21.15.157:0/3835803702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50080d00b0 0x7f50081520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:48.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 >> 172.21.15.157:0/3835803702 conn(0x7f5008070790 msgr2=0x7f50080b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:48.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 shutdown_connections 2023-11-06T03:16:48.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:48.527+0000 7f5010478640 1 -- 172.21.15.157:0/3835803702 wait complete. 2023-11-06T03:16:48.538 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:16:48.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:16:48.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:16:48.539 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:16:48.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:48.540 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:16:48.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:16:48.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:16:48.541 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:16:48.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.542 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:16:48.548 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:16:48.548 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:16:48.548 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:16:48.549 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.549 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:16:48.549 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.549 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:16:48.549 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.550 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:16:48.550 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:16:48.550 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.550 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:16:48.551 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41730sec preferred_lft 41730sec 2023-11-06T03:16:48.552 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:16:48.552 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.552 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:16:48.552 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.552 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:16:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.553 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:16:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:16:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:16:48.553 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.554 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:16:48.554 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.554 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:16:48.554 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:16:48.554 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:16:48.555 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.555 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:16:48.555 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:16:48.555 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:16:48.555 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:16:48.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.556 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:16:48.603 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:16:48.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.604 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:16:48.654 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:16:48.654 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:16:48.654 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:16:48.655 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:16:48.655 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:16:48.655 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:16:48.655 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:16:48.655 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:16:48.656 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:16:48.656 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:16:48.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.657 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:16:48.657 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:16:48.657 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:16:48.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:16:48.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:16:48.659 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:16:48.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.730 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:16:48.783 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:16:48.783 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:16:48.784 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:16:48.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:48.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:16:48.839 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:16:48.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:16:48.840 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:48.887 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:16:48.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:16:48.888 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:48.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:16:48.992 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:16:48.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:16:49.046 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:16:49.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:16:49.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:49.049 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:16:49.095 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:16:49.096 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:16:49.144 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:16:49.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:16:49.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:16:49.214 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:16:49.215 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:16:49.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:49.216 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:16:49.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:16:49.223+0000 7fd20cc73580 -1 init, newargv = 0x55cf71e47d50 newargc=15 2023-11-06T03:16:49.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[95482]: starting ceph client 2023-11-06T03:16:49.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[95482]: starting fuse 2023-11-06T03:16:50.327 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:16:50.344 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:16:50.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:16:50.399 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:16:50.400 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:16:50.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:50.401 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:16:50.447 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:16:50.448 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:16:50.448 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:16:50.449 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:16:50.449 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:16:50.449 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:16:50.450 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:16:50.450 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:50.451 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:16:50.451 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:16:50.451 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:16:50.452 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:16:50.452 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:50.453 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:16:50.453 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:16:50.454 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:16:50.454 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:16:50.454 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:50.455 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:16:50.455 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:16:50.456 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:16:50.456 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:16:50.456 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:16:50.457 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:16:50.457 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:16:50.457 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:16:50.458 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:50.458 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:16:50.459 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:16:50.568 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.95482.asok 2023-11-06T03:16:50.751 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95482.asok 2023-11-06T03:16:50.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:50.752 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95482.asok status 2023-11-06T03:16:50.911 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:50.911 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:16:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:16:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:16:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:16:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:16:50.912 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:16:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "95482", 2023-11-06T03:16:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:16:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:16:50.913 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:16:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 10539, 2023-11-06T03:16:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:16:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:16:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:16:50.914 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 10539 2023-11-06T03:16:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:16:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:16:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:16:50.915 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3126308000 2023-11-06T03:16:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:16:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:16:50.916 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:16:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3126308000 2023-11-06T03:16:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:16:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.10539 192.168.0.1:0/3126308000", 2023-11-06T03:16:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/3126308000", 2023-11-06T03:16:50.917 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:16:50.918 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 117, 2023-11-06T03:16:50.918 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 163, 2023-11-06T03:16:50.918 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:16:50.918 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:16:50.918 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:16:50.919 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:50.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:50.926 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:16:50.926 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:50.932 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:16:50.932 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:50.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:50.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:16:50.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:50.989 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-11-06T03:16:51.048 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:16:51.048 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:16:51.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:16:51.096 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T03:16:51.218 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records in 2023-11-06T03:16:51.219 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records out 2023-11-06T03:16:51.219 INFO:teuthology.orchestra.run.smithi157.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0700232 s, 89.8 MB/s 2023-11-06T03:16:51.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:16:51.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:16:51.251 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-06T03:16:51.252 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-06T03:16:51.252 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:16:51.252 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-06T03:16:51.252 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.253 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-06T03:16:51.253 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.253 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-06T03:16:51.253 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.254 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-06T03:16:51.254 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-06T03:16:51.254 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.254 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-06T03:16:51.254 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-06T03:16:51.255 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.255 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:16:51.255 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-06T03:16:51.255 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.256 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:16:51.256 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:16:51.256 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.260 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-06T03:16:51.261 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-06T03:16:51.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-06T03:16:51.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:16:51.261 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:16:51.262 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-06T03:16:51.262 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:16:51.262 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-06T03:16:51.263 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:16:51.263 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:16:51.263 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-06T03:16:51.263 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.264 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:16:51.264 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.264 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-06T03:16:51.264 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.265 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-06T03:16:51.265 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.265 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-06T03:16:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-06T03:16:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-06T03:16:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:16:51.267 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:16:51.267 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.267 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-06T03:16:51.267 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-06T03:16:51.268 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.268 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-06T03:16:51.268 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-06T03:16:51.268 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:16:51.269 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.269 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-06T03:16:51.269 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-06T03:16:51.269 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.269 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-06T03:16:51.270 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-06T03:16:51.270 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-06T03:16:51.271 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.271 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-06T03:16:51.272 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-06T03:16:51.272 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.273 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:16:51.273 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.274 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-06T03:16:51.274 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.274 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-06T03:16:51.274 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.275 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:16:51.275 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-06T03:16:51.275 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.275 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-06T03:16:51.275 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-06T03:16:51.276 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-06T03:16:51.277 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-06T03:16:51.277 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.277 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-06T03:16:51.277 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-06T03:16:51.277 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:16:51.278 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-06T03:16:51.278 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:16:51.278 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:16:51.278 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-06T03:16:51.278 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:16:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.279 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-06T03:16:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-06T03:16:51.279 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:16:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:16:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-06T03:16:51.280 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.281 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-06T03:16:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:16:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-06T03:16:51.281 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-06T03:16:51.282 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:16:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:16:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:16:51.283 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:16:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:16:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:16:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.284 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-06T03:16:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:16:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:16:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-06T03:16:51.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.286 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-06T03:16:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-06T03:16:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-06T03:16:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-06T03:16:51.286 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:16:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:16:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-06T03:16:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-06T03:16:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.287 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-06T03:16:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.288 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.288 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-06T03:16:51.288 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:16:51.289 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-06T03:16:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-06T03:16:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-06T03:16:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: mds cache max size = 1000 2023-11-06T03:16:51.290 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: mds log max segments = 1 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = False 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:16:51.291 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.292 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.293 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-06T03:16:51.293 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-06T03:16:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:16:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-06T03:16:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.294 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-06T03:16:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-06T03:16:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-06T03:16:51.295 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:16:51.296 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-06T03:16:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:16:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:16:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.297 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-06T03:16:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-06T03:16:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-06T03:16:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:16:51.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:16:51.299 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-06T03:16:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:16:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-06T03:16:51.300 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-06T03:16:51.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:16:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:16:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.302 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-06T03:16:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-06T03:16:51.302 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-06T03:16:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-06T03:16:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:16:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.303 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-06T03:16:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-06T03:16:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-06T03:16:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-06T03:16:51.304 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-06T03:16:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-06T03:16:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-06T03:16:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.305 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:16:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-06T03:16:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-06T03:16:51.306 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-06T03:16:51.307 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-06T03:16:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-06T03:16:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2023-11-06T03:16:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-06T03:16:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-06T03:16:51.308 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-06T03:16:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-06T03:16:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-06T03:16:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-06T03:16:51.309 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:16:51.310 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.311 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-06T03:16:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-06T03:16:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:16:51.311 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:16:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-06T03:16:51.312 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.312 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-06T03:16:51.313 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-06T03:16:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-06T03:16:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-06T03:16:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.314 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:16:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:16:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:16:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.315 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:16:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:16:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:16:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-06T03:16:51.316 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-06T03:16:51.317 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-06T03:16:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-06T03:16:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-06T03:16:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:16:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:16:51.318 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.319 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:16:51.320 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:16:51.321 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2023-11-06T03:16:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2023-11-06T03:16:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2023-11-06T03:16:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.321 INFO:teuthology.orchestra.run.smithi195.stdout: mds cache max size = 1000 2023-11-06T03:16:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2023-11-06T03:16:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2023-11-06T03:16:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: mds log max segments = 1 2023-11-06T03:16:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2023-11-06T03:16:51.322 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2023-11-06T03:16:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:16:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.323 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:16:51.340 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:16:51.369 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-06T03:16:51.369 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-06T03:16:51.369 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:16:51.369 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-06T03:16:51.370 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.370 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-06T03:16:51.370 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.370 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-06T03:16:51.370 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.371 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-06T03:16:51.371 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-06T03:16:51.371 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.371 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-06T03:16:51.371 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-06T03:16:51.372 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.372 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:16:51.372 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-06T03:16:51.372 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.372 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:16:51.373 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:16:51.373 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.373 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-06T03:16:51.373 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.374 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-06T03:16:51.374 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-06T03:16:51.374 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:16:51.374 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:16:51.375 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-06T03:16:51.375 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:16:51.375 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-06T03:16:51.375 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:16:51.375 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:16:51.376 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-06T03:16:51.376 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.376 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:16:51.376 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.376 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-06T03:16:51.377 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-06T03:16:51.378 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:16:51.378 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:16:51.378 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.378 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-06T03:16:51.378 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-06T03:16:51.379 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.379 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-06T03:16:51.379 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-06T03:16:51.379 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:16:51.379 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-06T03:16:51.380 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:16:51.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.382 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-06T03:16:51.382 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.382 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-06T03:16:51.382 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.382 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:16:51.383 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-06T03:16:51.383 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.383 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-06T03:16:51.383 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-06T03:16:51.383 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-06T03:16:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-06T03:16:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-06T03:16:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-06T03:16:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-06T03:16:51.384 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:16:51.385 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-06T03:16:51.386 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:16:51.386 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:16:51.386 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-06T03:16:51.386 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:16:51.386 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.387 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-06T03:16:51.387 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.387 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-06T03:16:51.387 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:16:51.387 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-06T03:16:51.388 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.389 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:16:51.389 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-06T03:16:51.389 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:16:51.389 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-06T03:16:51.389 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-06T03:16:51.390 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.390 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-06T03:16:51.390 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-06T03:16:51.390 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.390 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:16:51.391 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:16:51.392 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:16:51.392 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.392 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-06T03:16:51.392 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:16:51.392 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-06T03:16:51.393 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-06T03:16:51.394 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-06T03:16:51.394 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:16:51.394 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:16:51.394 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-06T03:16:51.394 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-06T03:16:51.395 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.395 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-06T03:16:51.395 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.395 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.395 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:16:51.396 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:16:51.397 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-06T03:16:51.397 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-06T03:16:51.397 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-06T03:16:51.397 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:16:51.397 INFO:teuthology.orchestra.run.smithi157.stdout: mds cache max size = 1000 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = False 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: mds log max segments = 1 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = False 2023-11-06T03:16:51.398 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:16:51.399 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.399 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.400 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-06T03:16:51.401 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-06T03:16:51.401 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:16:51.401 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-06T03:16:51.401 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.401 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-06T03:16:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-06T03:16:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-06T03:16:51.402 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-06T03:16:51.403 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.403 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-06T03:16:51.403 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-06T03:16:51.403 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.403 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.404 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-06T03:16:51.405 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.405 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-06T03:16:51.405 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-06T03:16:51.405 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:16:51.405 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:16:51.406 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-06T03:16:51.406 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:16:51.406 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-06T03:16:51.406 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:16:51.406 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:16:51.407 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-06T03:16:51.407 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.407 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:16:51.407 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.407 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-06T03:16:51.408 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-06T03:16:51.409 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:16:51.409 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:16:51.409 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.409 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-06T03:16:51.409 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-06T03:16:51.410 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.410 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-06T03:16:51.410 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-06T03:16:51.410 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:16:51.410 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-06T03:16:51.411 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-06T03:16:51.412 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.412 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-06T03:16:51.412 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-06T03:16:51.412 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.412 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.413 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:16:51.414 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-06T03:16:51.414 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.414 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-06T03:16:51.414 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-06T03:16:51.414 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-06T03:16:51.415 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-06T03:16:51.415 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2023-11-06T03:16:51.415 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-06T03:16:51.415 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-06T03:16:51.415 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:16:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-06T03:16:51.417 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:16:51.417 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:16:51.417 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-06T03:16:51.417 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:16:51.417 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.418 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-06T03:16:51.418 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.418 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-06T03:16:51.418 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:16:51.418 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-06T03:16:51.419 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.420 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:16:51.420 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-06T03:16:51.420 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:16:51.420 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-06T03:16:51.420 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.421 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:16:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:16:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:16:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:16:51.422 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:16:51.423 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:16:51.423 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.423 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-06T03:16:51.423 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:16:51.423 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.424 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:16:51.424 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2023-11-06T03:16:51.424 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:16:51.424 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-06T03:16:51.424 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2023-11-06T03:16:51.425 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2023-11-06T03:16:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2023-11-06T03:16:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.426 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.426 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-06T03:16:51.427 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-06T03:16:51.427 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-06T03:16:51.427 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:16:51.427 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:16:51.427 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:16:51.428 INFO:teuthology.orchestra.run.smithi195.stdout: mds cache max size = 1000 2023-11-06T03:16:51.429 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2023-11-06T03:16:51.429 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = False 2023-11-06T03:16:51.429 INFO:teuthology.orchestra.run.smithi195.stdout: mds log max segments = 1 2023-11-06T03:16:51.429 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2023-11-06T03:16:51.429 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2023-11-06T03:16:51.430 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:16:51.430 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:16:51.430 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:16:51.431 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:16:51.431 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:16:51.431 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:16:51.432 DEBUG:teuthology.orchestra.run.smithi157:> import json 2023-11-06T03:16:51.432 DEBUG:teuthology.orchestra.run.smithi157:> import sys 2023-11-06T03:16:51.432 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:51.433 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:16:51.433 DEBUG:teuthology.orchestra.run.smithi157:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-11-06T03:16:51.433 DEBUG:teuthology.orchestra.run.smithi157:> except OSError as e: 2023-11-06T03:16:51.434 DEBUG:teuthology.orchestra.run.smithi157:> sys.exit(e.errno) 2023-11-06T03:16:51.434 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:16:51.434 DEBUG:teuthology.orchestra.run.smithi157:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-06T03:16:51.435 DEBUG:teuthology.orchestra.run.smithi157:> print(json.dumps( 2023-11-06T03:16:51.435 DEBUG:teuthology.orchestra.run.smithi157:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-06T03:16:51.436 DEBUG:teuthology.orchestra.run.smithi157:> indent=2)) 2023-11-06T03:16:51.436 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:16:51.512 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:51.513 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mode": 33204, 2023-11-06T03:16:51.513 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ino": 1099511627777, 2023-11-06T03:16:51.513 INFO:teuthology.orchestra.run.smithi157.stdout: "st_dev": 45, 2023-11-06T03:16:51.513 INFO:teuthology.orchestra.run.smithi157.stdout: "st_nlink": 1, 2023-11-06T03:16:51.513 INFO:teuthology.orchestra.run.smithi157.stdout: "st_uid": 1000, 2023-11-06T03:16:51.514 INFO:teuthology.orchestra.run.smithi157.stdout: "st_gid": 1271, 2023-11-06T03:16:51.514 INFO:teuthology.orchestra.run.smithi157.stdout: "st_size": 6291456, 2023-11-06T03:16:51.514 INFO:teuthology.orchestra.run.smithi157.stdout: "st_atime": 1699240611.1528065, 2023-11-06T03:16:51.515 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mtime": 1699240611.1985438, 2023-11-06T03:16:51.515 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ctime": 1699240611.1985438 2023-11-06T03:16:51.515 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:51.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:16:51.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 Processor -- start 2023-11-06T03:16:51.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 -- start start 2023-11-06T03:16:51.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bf81551e0 0x7f6bf806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6bf815e5e0 0x7f6bf80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf815f580 con 0x7f6bf815ea90 2023-11-06T03:16:51.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf815efc0 con 0x7f6bf81551e0 2023-11-06T03:16:51.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfffd8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf815f2a0 con 0x7f6bf815e5e0 2023-11-06T03:16:51.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bf81551e0 0x7f6bf806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6bf81551e0 0x7f6bf806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55036/0 (socket says 172.21.15.157:55036) 2023-11-06T03:16:51.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2910277347 learned_addr learned my addr 172.21.15.157:0/2910277347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:51.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfd573640 1 --2- 172.21.15.157:0/2910277347 >> v2:172.21.15.195:3300/0 conn(0x7f6bf815e5e0 0x7f6bf80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2910277347 >> v2:172.21.15.195:3300/0 conn(0x7f6bf815e5e0 msgr2=0x7f6bf80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:51.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 --2- 172.21.15.157:0/2910277347 >> v2:172.21.15.195:3300/0 conn(0x7f6bf815e5e0 0x7f6bf80b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:51.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2910277347 >> v1:172.21.15.195:6789/0 conn(0x7f6bf815ea90 legacy=0x7f6bf815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:51.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.871+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2910277347 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf806f5a0 con 0x7f6bf81551e0 2023-11-06T03:16:51.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- 172.21.15.157:0/2910277347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf806fda0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8066880 tx=0x7f6bf401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5dad4de2e456 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfcd72640 1 -- 172.21.15.157:0/2910277347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40ac640 con 0x7f6bf81551e0 2023-11-06T03:16:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfcd72640 1 -- 172.21.15.157:0/2910277347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf40ac7a0 con 0x7f6bf81551e0 2023-11-06T03:16:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfcd72640 1 -- 172.21.15.157:0/2910277347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40acb30 con 0x7f6bf81551e0 2023-11-06T03:16:51.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2910277347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 msgr2=0x7f6bf806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2910277347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf806fda0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8066880 tx=0x7f6bf401c400 comp rx=0 tx=0).stop 2023-11-06T03:16:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2910277347 shutdown_connections 2023-11-06T03:16:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2910277347 >> v2:172.21.15.195:3300/0 conn(0x7f6bf815e5e0 0x7f6bf80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:51.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2910277347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf806fda0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2910277347 >> 172.21.15.157:0/2910277347 conn(0x7f6bf8065930 msgr2=0x7f6bf8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2910277347 shutdown_connections 2023-11-06T03:16:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2910277347 wait complete. 2023-11-06T03:16:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 Processor -- start 2023-11-06T03:16:51.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- start start 2023-11-06T03:16:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bf815e5e0 0x7f6bf814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 0x7f6bf80cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf8157d10 con 0x7f6bf81551e0 2023-11-06T03:16:51.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf8150be0 con 0x7f6bf815ea90 2023-11-06T03:16:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf814c2b0 con 0x7f6bf815e5e0 2023-11-06T03:16:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55038/0 (socket says 172.21.15.157:55038) 2023-11-06T03:16:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2733528564 learned_addr learned my addr 172.21.15.157:0/2733528564 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:51.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfd573640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bf815e5e0 0x7f6bf814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfe575640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 0x7f6bf80cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bf815e5e0 msgr2=0x7f6bf814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bf815e5e0 0x7f6bf814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 msgr2=0x7f6bf80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:51.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 0x7f6bf80cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 -- 172.21.15.157:0/2733528564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40094d0 con 0x7f6bf81551e0 2023-11-06T03:16:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfe575640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 0x7f6bf80cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfdd74640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f6bf401aab0 tx=0x7f6bf40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b6850 con 0x7f6bf81551e0 2023-11-06T03:16:51.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf40b69d0 con 0x7f6bf81551e0 2023-11-06T03:16:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf814fb40 con 0x7f6bf81551e0 2023-11-06T03:16:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40b6d80 con 0x7f6bf81551e0 2023-11-06T03:16:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.875+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf8150100 con 0x7f6bf81551e0 2023-11-06T03:16:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf8072750 con 0x7f6bf81551e0 2023-11-06T03:16:51.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6bf40b53a0 con 0x7f6bf81551e0 2023-11-06T03:16:51.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bea7fc640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 0x7f6bdc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:51.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bf411db50 con 0x7f6bf81551e0 2023-11-06T03:16:51.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bfd573640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 0x7f6bdc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:51.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.879+0000 7f6bfd573640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 0x7f6bdc067f20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80cc1f0 tx=0x7f6bec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:51.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:51.883+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6bf40eb3b0 con 0x7f6bf81551e0 2023-11-06T03:16:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.075+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bf8071730 con 0x7f6bf81551e0 2023-11-06T03:16:52.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.075+0000 7f6bea7fc640 1 -- 172.21.15.157:0/2733528564 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f6bf40ee9d0 con 0x7f6bf81551e0 2023-11-06T03:16:52.074 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:16:52.074 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:16:52.074 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:16:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 msgr2=0x7f6bdc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 0x7f6bdc067f20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80cc1f0 tx=0x7f6bec008040 comp rx=0 tx=0).stop 2023-11-06T03:16:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 msgr2=0x7f6bf814bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f6bf401aab0 tx=0x7f6bf40191a0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 shutdown_connections 2023-11-06T03:16:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6bdc065a60 0x7f6bdc067f20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bf815ea90 0x7f6bf80cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bf815e5e0 0x7f6bf814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 --2- 172.21.15.157:0/2733528564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bf81551e0 0x7f6bf814bd70 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 >> 172.21.15.157:0/2733528564 conn(0x7f6bf8065930 msgr2=0x7f6bf80aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 shutdown_connections 2023-11-06T03:16:52.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.079+0000 7f6bfffd8640 1 -- 172.21.15.157:0/2733528564 wait complete. 2023-11-06T03:16:52.094 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-06T03:16:52.255 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2023-11-06T03:16:52.256 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T03:16:52.257 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2023-11-06T03:16:52.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 Processor -- start 2023-11-06T03:16:52.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- start start 2023-11-06T03:16:52.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 0x55c1b79de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7939dd0 0x55c1b794d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1b7851450 con 0x55c1b7957a10 2023-11-06T03:16:52.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1b7851150 con 0x55c1b79399f0 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1b7847dc0 con 0x55c1b7939dd0 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55046/0 (socket says 172.21.15.157:55046) 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 -- 172.21.15.157:0/2317830871 learned_addr learned my addr 172.21.15.157:0/2317830871 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff6247b640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 0x55c1b79de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 -- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7939dd0 msgr2=0x55c1b794d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff61c7a640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7939dd0 0x55c1b794d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7939dd0 0x55c1b794d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 -- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 msgr2=0x55c1b79de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 0x55c1b79de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 -- 172.21.15.157:0/2317830871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58005d30 con 0x55c1b7957a10 2023-11-06T03:16:52.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff6247b640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 0x55c1b79de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:52.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7eff54009640 tx=0x7eff54009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12cf74c198e127c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff60c78640 1 -- 172.21.15.157:0/2317830871 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400e070 con 0x55c1b7957a10 2023-11-06T03:16:52.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff60c78640 1 -- 172.21.15.157:0/2317830871 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff540066c0 con 0x55c1b7957a10 2023-11-06T03:16:52.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff60c78640 1 -- 172.21.15.157:0/2317830871 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54006a50 con 0x55c1b7957a10 2023-11-06T03:16:52.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- 172.21.15.157:0/2317830871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 msgr2=0x55c1b7908b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7eff54009640 tx=0x7eff54009b00 comp rx=0 tx=0).stop 2023-11-06T03:16:52.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- 172.21.15.157:0/2317830871 shutdown_connections 2023-11-06T03:16:52.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b7957a10 0x55c1b7908b00 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7939dd0 0x55c1b794d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 --2- 172.21.15.157:0/2317830871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b79399f0 0x55c1b79de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- 172.21.15.157:0/2317830871 >> 172.21.15.157:0/2317830871 conn(0x55c1b7938360 msgr2=0x55c1b793b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- 172.21.15.157:0/2317830871 shutdown_connections 2023-11-06T03:16:52.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.303+0000 7eff64961080 1 -- 172.21.15.157:0/2317830871 wait complete. 2023-11-06T03:16:52.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 Processor -- start 2023-11-06T03:16:52.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- start start 2023-11-06T03:16:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 0x55c1b7908ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b7939dd0 0x55c1b79c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7957a10 0x55c1b79e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1b79c1200 con 0x55c1b79399f0 2023-11-06T03:16:52.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1b79c14a0 con 0x55c1b7939dd0 2023-11-06T03:16:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1b79c1740 con 0x55c1b7957a10 2023-11-06T03:16:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 0x55c1b7908ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 0x55c1b7908ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55052/0 (socket says 172.21.15.157:55052) 2023-11-06T03:16:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 -- 172.21.15.157:0/2117127559 learned_addr learned my addr 172.21.15.157:0/2117127559 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff61c7a640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b7939dd0 0x55c1b79c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7957a10 0x55c1b79e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 -- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7957a10 msgr2=0x55c1b79e0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c1b7957a10 0x55c1b79e0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 -- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b7939dd0 msgr2=0x55c1b79c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b7939dd0 0x55c1b79c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 -- 172.21.15.157:0/2117127559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1b785fff0 con 0x55c1b79399f0 2023-11-06T03:16:52.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff61c7a640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c1b7939dd0 0x55c1b79c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:52.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff6247b640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 0x55c1b7908ff0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c480 tx=0x7eff5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff537fe640 1 -- 172.21.15.157:0/2117127559 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c003660 con 0x55c1b79399f0 2023-11-06T03:16:52.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff537fe640 1 -- 172.21.15.157:0/2117127559 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c0037c0 con 0x55c1b79399f0 2023-11-06T03:16:52.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c1b7853430 con 0x55c1b79399f0 2023-11-06T03:16:52.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff537fe640 1 -- 172.21.15.157:0/2117127559 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c003b30 con 0x55c1b79399f0 2023-11-06T03:16:52.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1b7852e90 con 0x55c1b79399f0 2023-11-06T03:16:52.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.307+0000 7eff537fe640 1 -- 172.21.15.157:0/2117127559 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7eff5c003c90 con 0x55c1b79399f0 2023-11-06T03:16:52.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff537fe640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff44064440 0x7eff44066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff537fe640 1 -- 172.21.15.157:0/2117127559 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff5c0820e0 con 0x55c1b79399f0 2023-11-06T03:16:52.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff61c7a640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff44064440 0x7eff44066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55c1b79e08d0 0x55c1b7950b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- osd_op(unknown.0.0:1 24.7 24:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e163) v8 -- 0x55c1b79510a0 con 0x55c1b79e08d0 2023-11-06T03:16:52.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55c1b79e08d0 0x55c1b7950b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff62c7c640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55c1b79e08d0 0x55c1b7950b60 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff61c7a640 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff44064440 0x7eff44066900 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7eff580ac000 tx=0x7eff580a5000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff62c7c640 1 -- 172.21.15.157:0/2117127559 <== osd.7 v2:172.21.15.195:6800/38427 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v163'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7eff54019040 con 0x55c1b79e08d0 2023-11-06T03:16:52.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55c1b79e08d0 msgr2=0x55c1b7950b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55c1b79e08d0 0x55c1b7950b60 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff44064440 msgr2=0x7eff44066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7eff44064440 0x7eff44066900 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7eff580ac000 tx=0x7eff580a5000 comp rx=0 tx=0).stop 2023-11-06T03:16:52.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 msgr2=0x55c1b7908ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 --2- 172.21.15.157:0/2117127559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1b79399f0 0x55c1b7908ff0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c480 tx=0x7eff5c00c940 comp rx=0 tx=0).stop 2023-11-06T03:16:52.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff62c7c640 1 -- 172.21.15.157:0/2117127559 reap_dead start 2023-11-06T03:16:52.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 shutdown_connections 2023-11-06T03:16:52.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 >> 172.21.15.157:0/2117127559 conn(0x55c1b7938360 msgr2=0x55c1b794e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 shutdown_connections 2023-11-06T03:16:52.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.311+0000 7eff64961080 1 -- 172.21.15.157:0/2117127559 wait complete. 2023-11-06T03:16:52.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 scrub start / recursive,repair 2023-11-06T03:16:52.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 Processor -- start 2023-11-06T03:16:52.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 -- start start 2023-11-06T03:16:52.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe61006bfb0 0x7fe610164990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100b37a0 con 0x7fe61006c4b0 2023-11-06T03:16:52.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6100b1b10 con 0x7fe610154230 2023-11-06T03:16:52.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.499+0000 7fe615c02640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6100b39d0 con 0x7fe61006bfb0 2023-11-06T03:16:52.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe61006bfb0 0x7fe610164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe606ffd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe61006c4b0 0x7fe61006b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59940/0 (socket says 172.21.15.157:59940) 2023-11-06T03:16:52.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe606ffd640 1 -- 172.21.15.157:0/2488482275 learned_addr learned my addr 172.21.15.157:0/2488482275 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:16:52.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60effd640 1 -- 172.21.15.157:0/2488482275 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199363204 0 0) 0x7fe6100b37a0 con 0x7fe61006c4b0 2023-11-06T03:16:52.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60effd640 1 -- 172.21.15.157:0/2488482275 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc003530 con 0x7fe61006c4b0 2023-11-06T03:16:52.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60ffff640 1 -- 172.21.15.157:0/2488482275 >> v1:172.21.15.195:6789/0 conn(0x7fe610154230 legacy=0x7fe61015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60ffff640 1 -- 172.21.15.157:0/2488482275 >> v1:172.21.15.157:6789/0 conn(0x7fe61006c4b0 legacy=0x7fe61006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60ffff640 1 -- 172.21.15.157:0/2488482275 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100b3cb0 con 0x7fe61006bfb0 2023-11-06T03:16:52.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60ffff640 1 --2- 172.21.15.157:0/2488482275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61006bfb0 0x7fe610164990 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fe608009d90 tx=0x7fe60801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcdd610720cbfae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60effd640 1 -- 172.21.15.157:0/2488482275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a5910 con 0x7fe61006bfb0 2023-11-06T03:16:52.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60effd640 1 -- 172.21.15.157:0/2488482275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6100ccdb0 con 0x7fe61006bfb0 2023-11-06T03:16:52.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe60effd640 1 -- 172.21.15.157:0/2488482275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080a5c90 con 0x7fe61006bfb0 2023-11-06T03:16:52.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 -- 172.21.15.157:0/2488482275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61006bfb0 msgr2=0x7fe610164990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 --2- 172.21.15.157:0/2488482275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61006bfb0 0x7fe610164990 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fe608009d90 tx=0x7fe60801c830 comp rx=0 tx=0).stop 2023-11-06T03:16:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 -- 172.21.15.157:0/2488482275 shutdown_connections 2023-11-06T03:16:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 --2- 172.21.15.157:0/2488482275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe61006bfb0 0x7fe610164990 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 -- 172.21.15.157:0/2488482275 >> 172.21.15.157:0/2488482275 conn(0x7fe610065930 msgr2=0x7fe610065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 -- 172.21.15.157:0/2488482275 shutdown_connections 2023-11-06T03:16:52.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.503+0000 7fe615c02640 1 -- 172.21.15.157:0/2488482275 wait complete. 2023-11-06T03:16:52.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 Processor -- start 2023-11-06T03:16:52.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- start start 2023-11-06T03:16:52.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe61006bfb0 0x7fe610153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 0x7fe6100b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101fce20 con 0x7fe610154230 2023-11-06T03:16:52.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101fd070 con 0x7fe61006c4b0 2023-11-06T03:16:52.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6101fd350 con 0x7fe61006bfb0 2023-11-06T03:16:52.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55066/0 (socket says 172.21.15.157:55066) 2023-11-06T03:16:52.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 -- 172.21.15.157:0/3959797602 learned_addr learned my addr 172.21.15.157:0/3959797602 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe606ffd640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 0x7fe6100b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe61006bfb0 0x7fe610153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 -- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe61006bfb0 msgr2=0x7fe610153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe61006bfb0 0x7fe610153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 -- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 msgr2=0x7fe6100b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 0x7fe6100b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 -- 172.21.15.157:0/3959797602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009a70 con 0x7fe610154230 2023-11-06T03:16:52.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe606ffd640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 0x7fe6100b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:52.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60ffff640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fe60801a460 tx=0x7fe6080195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60cff9640 1 -- 172.21.15.157:0/3959797602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080b9650 con 0x7fe610154230 2023-11-06T03:16:52.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60cff9640 1 -- 172.21.15.157:0/3959797602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6080b97b0 con 0x7fe610154230 2023-11-06T03:16:52.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe60cff9640 1 -- 172.21.15.157:0/3959797602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080b9b20 con 0x7fe610154230 2023-11-06T03:16:52.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6101fd630 con 0x7fe610154230 2023-11-06T03:16:52.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.507+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe610153770 con 0x7fe610154230 2023-11-06T03:16:52.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60cff9640 1 -- 172.21.15.157:0/3959797602 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe6080ad440 con 0x7fe610154230 2023-11-06T03:16:52.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60cff9640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 0x7fe5f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60cff9640 1 -- 172.21.15.157:0/3959797602 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe60811e5f0 con 0x7fe610154230 2023-11-06T03:16:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 0x7fe5f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 0x7fe5f0067f20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f70 tx=0x7fe600009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 Processor -- start 2023-11-06T03:16:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 -- start start 2023-11-06T03:16:52.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f27741cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27741d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2772b6a90 con 0x55f2774386a0 2023-11-06T03:16:52.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f277244490 con 0x55f277438fc0 2023-11-06T03:16:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f277241a60 con 0x55f27741d400 2023-11-06T03:16:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55074/0 (socket says 172.21.15.157:55074) 2023-11-06T03:16:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.511+0000 7fe60f7fe640 1 -- 172.21.15.157:0/3381086704 learned_addr learned my addr 172.21.15.157:0/3381086704 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60ffff640 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27741d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 msgr2=0x55f27741d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27741d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 msgr2=0x55f27741cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f27741cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/3381086704 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f276f3a070 con 0x55f2774386a0 2023-11-06T03:16:52.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fe600001560 tx=0x7fe600011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc0c5621ebc9146 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe6057fa640 1 -- 172.21.15.157:0/3381086704 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600003600 con 0x55f2774386a0 2023-11-06T03:16:52.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe6057fa640 1 -- 172.21.15.157:0/3381086704 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe600003760 con 0x55f2774386a0 2023-11-06T03:16:52.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe6057fa640 1 -- 172.21.15.157:0/3381086704 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600003ad0 con 0x55f2774386a0 2023-11-06T03:16:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/3381086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 msgr2=0x55f277438a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fe600001560 tx=0x7fe600011000 comp rx=0 tx=0).stop 2023-11-06T03:16:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/3381086704 shutdown_connections 2023-11-06T03:16:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27741d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f27741cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- 172.21.15.157:0/3381086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277438a80 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/3381086704 >> 172.21.15.157:0/3381086704 conn(0x55f2773e5bc0 msgr2=0x55f2773f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/3381086704 shutdown_connections 2023-11-06T03:16:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/3381086704 wait complete. 2023-11-06T03:16:52.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- start start 2023-11-06T03:16:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 Processor -- start 2023-11-06T03:16:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2774386a0 0x55f2773f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f2773f2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27723d350 con 0x55f27741d400 2023-11-06T03:16:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2771dc610 con 0x55f277438fc0 2023-11-06T03:16:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2771fa800 con 0x55f2774386a0 2023-11-06T03:16:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 10 client.0 _ll_register_callbacks cb 0x55f277232960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:52.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55088/0 (socket says 172.21.15.157:55088) 2023-11-06T03:16:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/548849114 learned_addr learned my addr 172.21.15.157:0/548849114 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe606ffd640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2774386a0 0x55f2773f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60ffff640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f2773f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2774386a0 msgr2=0x55f2773f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2774386a0 0x55f2773f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 msgr2=0x55f2773f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f2773f2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 -- 172.21.15.157:0/548849114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60000f040 con 0x55f27741d400 2023-11-06T03:16:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fe600001ec0 tx=0x7fe600013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401d68cb1521740b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe5ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600028070 con 0x55f27741d400 2023-11-06T03:16:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe600003c80 con 0x55f27741d400 2023-11-06T03:16:52.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f27720cc80 con 0x55f27741d400 2023-11-06T03:16:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe600003e40 con 0x55f27741d400 2023-11-06T03:16:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 10 client.10572 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.515+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55f2772022d0 con 0x55f27741d400 2023-11-06T03:16:52.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f27720cc80 con 0x55f27741d400 2023-11-06T03:16:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe600021450 con 0x55f27741d400 2023-11-06T03:16:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 10 client.10572 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 10 client.10572 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:16:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 20 client.10572 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:52.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f277447940 0x55f277449d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 4 client.10572 mds_command: new command op to 10392 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:16:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f27723bca0 con 0x55f277447940 2023-11-06T03:16:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe606ffd640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f277447940 0x55f277449d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe606ffd640 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f277447940 0x55f277449d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b482d1689efbc80e server_cookie=bca62d2d112a265a in_seq=0 out_seq=0 2023-11-06T03:16:52.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 10 client.10572 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/548849114 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f27723bca0 con 0x55f277447940 2023-11-06T03:16:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 10 client.10572 handle_command_reply: tid=0 2023-11-06T03:16:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 client.10572 shutdown 2023-11-06T03:16:52.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 msgr2=0x55f2773f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fe600001ec0 tx=0x7fe600013000 comp rx=0 tx=0).stop 2023-11-06T03:16:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 shutdown_connections 2023-11-06T03:16:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f277447940 0x55f277449d30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f2773f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f2774386a0 0x55f2773f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe5ee7fc640 0 client.10572 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 --2- 172.21.15.157:0/548849114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f27741d400 0x55f2773f15d0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 >> 172.21.15.157:0/548849114 conn(0x55f2773e5bc0 msgr2=0x55f27741e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 shutdown_connections 2023-11-06T03:16:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 1 -- 172.21.15.157:0/548849114 wait complete. 2023-11-06T03:16:52.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.519+0000 7fe618d2a000 20 client.10572 trim_cache size 0 max 16384 2023-11-06T03:16:52.544 INFO:tasks.ceph.mds.d.smithi195.stderr:2023-11-06T03:16:52.548+0000 7fc5fc353640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2023-11-06T03:16:51.191915+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10539=0-12582912@1} caps={10539=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@4},l=10539 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55a8f3c83b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(24)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-06T03:16:52.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 Processor -- start 2023-11-06T03:16:52.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- start start 2023-11-06T03:16:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27744ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277417750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f276efacf0 con 0x55f2774386a0 2023-11-06T03:16:52.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27720c8e0 con 0x55f277438fc0 2023-11-06T03:16:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f277204440 con 0x55f27741d400 2023-11-06T03:16:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55094/0 (socket says 172.21.15.157:55094) 2023-11-06T03:16:52.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 -- 172.21.15.157:0/234543615 learned_addr learned my addr 172.21.15.157:0/234543615 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 -- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 msgr2=0x55f27744ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27744ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 -- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 msgr2=0x55f277417750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe60ffff640 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277417750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277417750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 -- 172.21.15.157:0/234543615 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2771e8620 con 0x55f2774386a0 2023-11-06T03:16:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe606ffd640 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8001720 tx=0x7fe5f8001c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c1fefd9990293e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe605ffb640 1 -- 172.21.15.157:0/234543615 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8005d90 con 0x55f2774386a0 2023-11-06T03:16:52.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe605ffb640 1 -- 172.21.15.157:0/234543615 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f801a070 con 0x55f2774386a0 2023-11-06T03:16:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe605ffb640 1 -- 172.21.15.157:0/234543615 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8016ad0 con 0x55f2774386a0 2023-11-06T03:16:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- 172.21.15.157:0/234543615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 msgr2=0x55f27744b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8001720 tx=0x7fe5f8001c20 comp rx=0 tx=0).stop 2023-11-06T03:16:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- 172.21.15.157:0/234543615 shutdown_connections 2023-11-06T03:16:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277417750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f27744b280 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- 172.21.15.157:0/234543615 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f27744ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- 172.21.15.157:0/234543615 >> 172.21.15.157:0/234543615 conn(0x55f2773e5bc0 msgr2=0x55f27741e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- 172.21.15.157:0/234543615 shutdown_connections 2023-11-06T03:16:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- 172.21.15.157:0/234543615 wait complete. 2023-11-06T03:16:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 -- start start 2023-11-06T03:16:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 Processor -- start 2023-11-06T03:16:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.539+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f2774951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277419e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27720c180 con 0x55f2774386a0 2023-11-06T03:16:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f277238cc0 con 0x55f277438fc0 2023-11-06T03:16:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2771ea560 con 0x55f27741d400 2023-11-06T03:16:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 10 client.0 _ll_register_callbacks cb 0x55f2772430a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55098/0 (socket says 172.21.15.157:55098) 2023-11-06T03:16:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 -- 172.21.15.157:0/2105425033 learned_addr learned my addr 172.21.15.157:0/2105425033 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f2774951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe60ffff640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277419e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 -- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 msgr2=0x55f2774951e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f2774951e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 -- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 msgr2=0x55f277419e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277419e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 -- 172.21.15.157:0/2105425033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc003530 con 0x55f2774386a0 2023-11-06T03:16:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe606ffd640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80011f0 tx=0x7fe5f8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a84e4aacb75b93 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8026070 con 0x55f2774386a0 2023-11-06T03:16:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5f80169d0 con 0x55f2774386a0 2023-11-06T03:16:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2773ef730 con 0x55f2774386a0 2023-11-06T03:16:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5f8016b90 con 0x55f2774386a0 2023-11-06T03:16:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 10 client.10578 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55f2771dc190 con 0x55f2774386a0 2023-11-06T03:16:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f2773ef730 con 0x55f2774386a0 2023-11-06T03:16:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe5f8012070 con 0x55f2774386a0 2023-11-06T03:16:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 10 client.10578 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 10 client.10578 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:16:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 20 client.10578 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f27741ac60 0x55f27741b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 4 client.10578 mds_command: new command op to 10392 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2023-11-06T03:16:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55f2771ecd50 con 0x55f27741ac60 2023-11-06T03:16:52.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f27741ac60 0x55f27741b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 10 client.10578 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe60f7fe640 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f27741ac60 0x55f27741b040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3f49f0b923784e server_cookie=2d1f11e571244e6c in_seq=0 out_seq=0 2023-11-06T03:16:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 1 -- 172.21.15.157:0/2105425033 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f2771ecd50 con 0x55f27741ac60 2023-11-06T03:16:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 10 client.10578 handle_command_reply: tid=0 2023-11-06T03:16:52.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 client.10578 shutdown 2023-11-06T03:16:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 msgr2=0x55f277495720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80011f0 tx=0x7fe5f8000ee0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 shutdown_connections 2023-11-06T03:16:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55f27741ac60 0x55f27741b040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f277438fc0 0x55f277419e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2774386a0 0x55f277495720 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe5ee7fc640 0 client.10578 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 --2- 172.21.15.157:0/2105425033 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55f27741d400 0x55f2774951e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 >> 172.21.15.157:0/2105425033 conn(0x55f2773e5bc0 msgr2=0x55f27741edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 shutdown_connections 2023-11-06T03:16:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.543+0000 7fe618d2a000 1 -- 172.21.15.157:0/2105425033 wait complete. 2023-11-06T03:16:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe618d2a000 20 client.10578 trim_cache size 0 max 16384 2023-11-06T03:16:52.563 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:52.563 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:16:52.563 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "97ca0a94-97a0-4639-8a75-4581b49923aa", 2023-11-06T03:16:52.563 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:16:52.564 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 msgr2=0x7fe5f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 0x7fe5f0067f20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f70 tx=0x7fe600009040 comp rx=0 tx=0).stop 2023-11-06T03:16:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 msgr2=0x7fe61015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fe60801a460 tx=0x7fe6080195d0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 shutdown_connections 2023-11-06T03:16:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe5f0065a60 0x7fe5f0067f20 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe610154230 0x7fe61015d4e0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe61006c4b0 0x7fe6100b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 --2- 172.21.15.157:0/3959797602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe61006bfb0 0x7fe610153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 >> 172.21.15.157:0/3959797602 conn(0x7fe610065930 msgr2=0x7fe610065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 shutdown_connections 2023-11-06T03:16:52.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.547+0000 7fe615c02640 1 -- 172.21.15.157:0/3959797602 wait complete. 2023-11-06T03:16:52.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 scrub status 2023-11-06T03:16:52.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.687+0000 7fe6aab84640 1 Processor -- start 2023-11-06T03:16:52.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.687+0000 7fe6aab84640 1 -- start start 2023-11-06T03:16:52.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6aab84640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe6a41551e0 0x7fe6a406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6aab84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6aab84640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a415f580 con 0x7fe6a415e5e0 2023-11-06T03:16:52.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6aab84640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a415efc0 con 0x7fe6a41551e0 2023-11-06T03:16:52.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6aab84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a415f2a0 con 0x7fe6a40b3b20 2023-11-06T03:16:52.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:50294/0 (socket says 172.21.15.157:50294) 2023-11-06T03:16:52.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 -- 172.21.15.157:0/1048941410 learned_addr learned my addr 172.21.15.157:0/1048941410 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a8920640 1 --2- 172.21.15.157:0/1048941410 >> v2:172.21.15.195:3300/0 conn(0x7fe6a41551e0 0x7fe6a406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 -- 172.21.15.157:0/1048941410 >> v2:172.21.15.195:3300/0 conn(0x7fe6a41551e0 msgr2=0x7fe6a406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 --2- 172.21.15.157:0/1048941410 >> v2:172.21.15.195:3300/0 conn(0x7fe6a41551e0 0x7fe6a406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 -- 172.21.15.157:0/1048941410 >> v1:172.21.15.195:6789/0 conn(0x7fe6a415e5e0 legacy=0x7fe6a40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.691+0000 7fe6a9121640 1 -- 172.21.15.157:0/1048941410 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a406f5a0 con 0x7fe6a40b3b20 2023-11-06T03:16:52.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6a9121640 1 --2- 172.21.15.157:0/1048941410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009e50 tx=0x7fe69c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb798b7c6385d4fb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1048941410 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0a5830 con 0x7fe6a40b3b20 2023-11-06T03:16:52.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1048941410 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0a5990 con 0x7fe6a40b3b20 2023-11-06T03:16:52.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1048941410 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0a5d20 con 0x7fe6a40b3b20 2023-11-06T03:16:52.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- 172.21.15.157:0/1048941410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 msgr2=0x7fe6a415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- 172.21.15.157:0/1048941410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009e50 tx=0x7fe69c01c950 comp rx=0 tx=0).stop 2023-11-06T03:16:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- 172.21.15.157:0/1048941410 shutdown_connections 2023-11-06T03:16:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- 172.21.15.157:0/1048941410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a415c1b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- 172.21.15.157:0/1048941410 >> v2:172.21.15.195:3300/0 conn(0x7fe6a41551e0 0x7fe6a406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- 172.21.15.157:0/1048941410 >> 172.21.15.157:0/1048941410 conn(0x7fe6a4065930 msgr2=0x7fe6a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- 172.21.15.157:0/1048941410 shutdown_connections 2023-11-06T03:16:52.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- 172.21.15.157:0/1048941410 wait complete. 2023-11-06T03:16:52.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 Processor -- start 2023-11-06T03:16:52.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- start start 2023-11-06T03:16:52.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a41447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a41551e0 0x7fe6a40b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41528d0 con 0x7fe6a415e5e0 2023-11-06T03:16:52.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a406ca80 con 0x7fe6a41551e0 2023-11-06T03:16:52.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6aab84640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a406cd60 con 0x7fe6a40b3b20 2023-11-06T03:16:52.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.695+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55110/0 (socket says 172.21.15.157:55110) 2023-11-06T03:16:52.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 -- 172.21.15.157:0/3340522068 learned_addr learned my addr 172.21.15.157:0/3340522068 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a8920640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a41447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a41551e0 0x7fe6a40b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 -- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 msgr2=0x7fe6a41447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a41447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 -- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a41551e0 msgr2=0x7fe6a40b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a41551e0 0x7fe6a40b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 -- 172.21.15.157:0/3340522068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009b30 con 0x7fe6a415e5e0 2023-11-06T03:16:52.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a9121640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fe69c01a580 tx=0x7fe69c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3340522068 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0b8670 con 0x7fe6a415e5e0 2023-11-06T03:16:52.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3340522068 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0b87d0 con 0x7fe6a415e5e0 2023-11-06T03:16:52.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a406d040 con 0x7fe6a415e5e0 2023-11-06T03:16:52.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3340522068 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0b8b60 con 0x7fe6a415e5e0 2023-11-06T03:16:52.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a4144cf0 con 0x7fe6a415e5e0 2023-11-06T03:16:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3340522068 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe69c0b8cc0 con 0x7fe6a415e5e0 2023-11-06T03:16:52.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 0x7fe690067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3340522068 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe69c11e760 con 0x7fe6a415e5e0 2023-11-06T03:16:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a8920640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 0x7fe690067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6adcac000 1 Processor -- start 2023-11-06T03:16:52.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6adcac000 1 -- start start 2023-11-06T03:16:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.699+0000 7fe6a8920640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 0x7fe690067f20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe694009af0 tx=0x7fe694008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4b79640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4a40c10 con 0x55d2f4bc3a50 2023-11-06T03:16:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49cf3e0 con 0x55d2f4b79260 2023-11-06T03:16:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49cc9b0 con 0x55d2f4bc3130 2023-11-06T03:16:52.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55126/0 (socket says 172.21.15.157:55126) 2023-11-06T03:16:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3524272662 learned_addr learned my addr 172.21.15.157:0/3524272662 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a8920640 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 msgr2=0x55d2f4bc3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 msgr2=0x55d2f4b79640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4b79640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3524272662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f46c4e20 con 0x55d2f4bc3a50 2023-11-06T03:16:52.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41f9320 tx=0x7fe698001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c184389a9a816b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe68a7fc640 1 -- 172.21.15.157:0/3524272662 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6980052e0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe68a7fc640 1 -- 172.21.15.157:0/3524272662 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe698005440 con 0x55d2f4bc3a50 2023-11-06T03:16:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe68a7fc640 1 -- 172.21.15.157:0/3524272662 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698003820 con 0x55d2f4bc3a50 2023-11-06T03:16:52.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/3524272662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 msgr2=0x55d2f4b78d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41f9320 tx=0x7fe698001940 comp rx=0 tx=0).stop 2023-11-06T03:16:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/3524272662 shutdown_connections 2023-11-06T03:16:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4b79640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4b78d20 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- 172.21.15.157:0/3524272662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/3524272662 >> 172.21.15.157:0/3524272662 conn(0x55d2f4b7f9f0 msgr2=0x55d2f4b77600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/3524272662 shutdown_connections 2023-11-06T03:16:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/3524272662 wait complete. 2023-11-06T03:16:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- start start 2023-11-06T03:16:52.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 Processor -- start 2023-11-06T03:16:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4bc2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49c82a0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4967470 con 0x55d2f4b79260 2023-11-06T03:16:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4985660 con 0x55d2f4bc3130 2023-11-06T03:16:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 10 client.0 _ll_register_callbacks cb 0x55d2f4b64c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55138/0 (socket says 172.21.15.157:55138) 2023-11-06T03:16:52.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 -- 172.21.15.157:0/4213360540 learned_addr learned my addr 172.21.15.157:0/4213360540 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a8920640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4bc2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 -- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 msgr2=0x55d2f4bc4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 -- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 msgr2=0x55d2f4bc2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4bc2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 -- 172.21.15.157:0/4213360540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69800f040 con 0x55d2f4bc3a50 2023-11-06T03:16:52.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6a9121640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0ec240 tx=0x7fe69c0f0580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586fc013bf992318 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6837fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0f01a0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe69c0f7d20 con 0x55d2f4bc3a50 2023-11-06T03:16:52.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2f49850a0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe69c0ef230 con 0x55d2f4bc3a50 2023-11-06T03:16:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.703+0000 7fe6adcac000 10 client.10587 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55d2f498d1d0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2f49850a0 con 0x55d2f4bc3a50 2023-11-06T03:16:52.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe69c005d20 con 0x55d2f4bc3a50 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 10 client.10587 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 10 client.10587 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 20 client.10587 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bc4f90 0x55d2f4bc5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 4 client.10587 mds_command: new command op to 10392 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:16:52.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2f49c6bf0 con 0x55d2f4bc4f90 2023-11-06T03:16:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6a8920640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bc4f90 0x55d2f4bc5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6a8920640 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bc4f90 0x55d2f4bc5370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3555083fedcddc81 server_cookie=fd3a9721ae49ebb7 in_seq=0 out_seq=0 2023-11-06T03:16:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 10 client.10587 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 1 -- 172.21.15.157:0/4213360540 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d2f49c6bf0 con 0x55d2f4bc4f90 2023-11-06T03:16:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 10 client.10587 handle_command_reply: tid=0 2023-11-06T03:16:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 client.10587 shutdown 2023-11-06T03:16:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 msgr2=0x55d2f4bc4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0ec240 tx=0x7fe69c0f0580 comp rx=0 tx=0).stop 2023-11-06T03:16:52.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 shutdown_connections 2023-11-06T03:16:52.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bc4f90 0x55d2f4bc5370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3a50 0x55d2f4bc4580 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4bc4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6837fe640 0 client.10587 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 --2- 172.21.15.157:0/4213360540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4bc2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 >> 172.21.15.157:0/4213360540 conn(0x55d2f4b7f9f0 msgr2=0x55d2f4b77d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 shutdown_connections 2023-11-06T03:16:52.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 1 -- 172.21.15.157:0/4213360540 wait complete. 2023-11-06T03:16:52.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.707+0000 7fe6adcac000 20 client.10587 trim_cache size 0 max 16384 2023-11-06T03:16:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 Processor -- start 2023-11-06T03:16:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- start start 2023-11-06T03:16:52.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4c1ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4bc3a50 0x55d2f4c1b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4685aa0 con 0x55d2f4b79260 2023-11-06T03:16:52.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49c2820 con 0x55d2f4bc3a50 2023-11-06T03:16:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4986400 con 0x55d2f4bc3130 2023-11-06T03:16:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55154/0 (socket says 172.21.15.157:55154) 2023-11-06T03:16:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 -- 172.21.15.157:0/131644894 learned_addr learned my addr 172.21.15.157:0/131644894 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 -- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 msgr2=0x55d2f4c1ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4c1ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4c1ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 -- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4bc3a50 msgr2=0x55d2f4c1b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4bc3a50 0x55d2f4c1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 -- 172.21.15.157:0/131644894 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f4973520 con 0x55d2f4b79260 2023-11-06T03:16:52.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6a8920640 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe694011b10 tx=0x7fe694017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84020950335cf413 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe68affd640 1 -- 172.21.15.157:0/131644894 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694028040 con 0x55d2f4b79260 2023-11-06T03:16:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe68affd640 1 -- 172.21.15.157:0/131644894 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe694029240 con 0x55d2f4b79260 2023-11-06T03:16:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe68affd640 1 -- 172.21.15.157:0/131644894 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694029620 con 0x55d2f4b79260 2023-11-06T03:16:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- 172.21.15.157:0/131644894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 msgr2=0x55d2f4c1a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe694011b10 tx=0x7fe694017000 comp rx=0 tx=0).stop 2023-11-06T03:16:52.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- 172.21.15.157:0/131644894 shutdown_connections 2023-11-06T03:16:52.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4bc3a50 0x55d2f4c1b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3130 0x55d2f4c1ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 --2- 172.21.15.157:0/131644894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4b79260 0x55d2f4c1a8c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- 172.21.15.157:0/131644894 >> 172.21.15.157:0/131644894 conn(0x55d2f4b7f9f0 msgr2=0x55d2f4bc02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- 172.21.15.157:0/131644894 shutdown_connections 2023-11-06T03:16:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- 172.21.15.157:0/131644894 wait complete. 2023-11-06T03:16:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 -- start start 2023-11-06T03:16:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.723+0000 7fe6adcac000 1 Processor -- start 2023-11-06T03:16:52.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4c20200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 0x55d2f4bd5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49c7130 con 0x55d2f4bc3130 2023-11-06T03:16:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f4b7c320 con 0x55d2f4b79260 2023-11-06T03:16:52.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f49c5030 con 0x55d2f4bc3a50 2023-11-06T03:16:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 10 client.0 _ll_register_callbacks cb 0x55d2f498ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:16:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55158/0 (socket says 172.21.15.157:55158) 2023-11-06T03:16:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 -- 172.21.15.157:0/26991872 learned_addr learned my addr 172.21.15.157:0/26991872 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:16:52.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a9121640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 0x55d2f4bd5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a8920640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4c20200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 -- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 msgr2=0x55d2f4bd5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 0x55d2f4bd5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 -- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 msgr2=0x55d2f4c20200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:16:52.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4c20200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 -- 172.21.15.157:0/26991872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe694011d90 con 0x55d2f4bc3130 2023-11-06T03:16:52.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a8920640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4c20200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:16:52.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a9121640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 0x55d2f4bd5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:16:52.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe698001980 tx=0x7fe698000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42718f4f51651fc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:16:52.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:16:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698003810 con 0x55d2f4bc3130 2023-11-06T03:16:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe698003970 con 0x55d2f4bc3130 2023-11-06T03:16:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2f4991620 con 0x55d2f4bc3130 2023-11-06T03:16:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe698003b30 con 0x55d2f4bc3130 2023-11-06T03:16:52.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 10 client.10593 fetch_fsmap learned FSMap version 117 2023-11-06T03:16:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x55d2f4967090 con 0x55d2f4bc3130 2023-11-06T03:16:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2f4991620 con 0x55d2f4bc3130 2023-11-06T03:16:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe698003c90 con 0x55d2f4bc3130 2023-11-06T03:16:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 10 client.10593 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:16:52.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 10 client.10593 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:16:52.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 20 client.10593 populate_metadata read hostname 'smithi157' 2023-11-06T03:16:52.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bd5d60 0x55d2f4bd6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:16:52.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 4 client.10593 mds_command: new command op to 10392 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:16:52.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2f49c1320 con 0x55d2f4bd5d60 2023-11-06T03:16:52.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a8920640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bd5d60 0x55d2f4bd6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:16:52.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 10 client.10593 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6a8920640 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bd5d60 0x55d2f4bd6140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626d75ef76ff52f0 server_cookie=a0cc7b8d9e57d188 in_seq=0 out_seq=0 2023-11-06T03:16:52.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 1 -- 172.21.15.157:0/26991872 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55d2f49c1320 con 0x55d2f4bd5d60 2023-11-06T03:16:52.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 10 client.10593 handle_command_reply: tid=0 2023-11-06T03:16:52.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 client.10593 shutdown 2023-11-06T03:16:52.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 msgr2=0x55d2f4c20740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe698001980 tx=0x7fe698000ee0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 shutdown_connections 2023-11-06T03:16:52.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x55d2f4bd5d60 0x55d2f4bd6140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d2f4bc3a50 0x55d2f4bd5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d2f4bc3130 0x55d2f4c20740 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6837fe640 0 client.10593 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:16:52.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 --2- 172.21.15.157:0/26991872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2f4b79260 0x55d2f4c20200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 >> 172.21.15.157:0/26991872 conn(0x55d2f4b7f9f0 msgr2=0x55d2f4b7e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 shutdown_connections 2023-11-06T03:16:52.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 1 -- 172.21.15.157:0/26991872 wait complete. 2023-11-06T03:16:52.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.727+0000 7fe6adcac000 20 client.10593 trim_cache size 0 max 16384 2023-11-06T03:16:52.750 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:16:52.751 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:16:52.751 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:16:52.751 INFO:teuthology.orchestra.run.smithi157.stdout: "97ca0a94-97a0-4639-8a75-4581b49923aa": { 2023-11-06T03:16:52.751 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/", 2023-11-06T03:16:52.751 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "97ca0a94-97a0-4639-8a75-4581b49923aa", 2023-11-06T03:16:52.752 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive,repair" 2023-11-06T03:16:52.752 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:52.752 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:16:52.752 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:16:52.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 msgr2=0x7fe690067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 0x7fe690067f20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe694009af0 tx=0x7fe694008040 comp rx=0 tx=0).stop 2023-11-06T03:16:52.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 msgr2=0x7fe6a40b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:16:52.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fe69c01a580 tx=0x7fe69c0196f0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 shutdown_connections 2023-11-06T03:16:52.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe690065a60 0x7fe690067f20 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415e5e0 0x7fe6a40b1c10 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a41551e0 0x7fe6a40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 --2- 172.21.15.157:0/3340522068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40b3b20 0x7fe6a41447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:16:52.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 >> 172.21.15.157:0/3340522068 conn(0x7fe6a4065930 msgr2=0x7fe6a40afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:16:52.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 shutdown_connections 2023-11-06T03:16:52.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:16:52.731+0000 7fe6aab84640 1 -- 172.21.15.157:0/3340522068 wait complete. 2023-11-06T03:16:52.756 INFO:tasks.cephfs.filesystem:scrub status for tag:97ca0a94-97a0-4639-8a75-4581b49923aa - {'path': '/', 'tag': '97ca0a94-97a0-4639-8a75-4581b49923aa', 'options': 'recursive,repair'} 2023-11-06T03:17:12.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:17:12.431 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:17:22.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 scrub status 2023-11-06T03:17:22.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.891+0000 7fdd814c4640 1 Processor -- start 2023-11-06T03:17:22.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.891+0000 7fdd814c4640 1 -- start start 2023-11-06T03:17:22.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd814c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 0x7fdd7c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd814c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd814c4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c0d2920 con 0x7fdd7c15ce80 2023-11-06T03:17:22.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd814c4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0d18a0 con 0x7fdd7c0d0280 2023-11-06T03:17:22.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd814c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0d1b60 con 0x7fdd7c15caa0 2023-11-06T03:17:22.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38148/0 (socket says 172.21.15.157:38148) 2023-11-06T03:17:22.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/2527388760 learned_addr learned my addr 172.21.15.157:0/2527388760 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7affd640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 0x7fdd7c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/2527388760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 msgr2=0x7fdd7c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 0x7fdd7c0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/2527388760 >> v1:172.21.15.195:6789/0 conn(0x7fdd7c15ce80 legacy=0x7fdd7c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/2527388760 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0d1e40 con 0x7fdd7c0d0280 2023-11-06T03:17:22.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7affd640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 0x7fdd7c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:22.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009e50 tx=0x7fdd7401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4a75397b8301e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.895+0000 7fdd79ffb640 1 -- 172.21.15.157:0/2527388760 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740a5820 con 0x7fdd7c0d0280 2023-11-06T03:17:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd79ffb640 1 -- 172.21.15.157:0/2527388760 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd740a5980 con 0x7fdd7c0d0280 2023-11-06T03:17:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd79ffb640 1 -- 172.21.15.157:0/2527388760 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740a5d10 con 0x7fdd7c0d0280 2023-11-06T03:17:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/2527388760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c0d0280 msgr2=0x7fdd7c0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009e50 tx=0x7fdd7401c9a0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/2527388760 shutdown_connections 2023-11-06T03:17:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c0d0280 0x7fdd7c0b5d50 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- 172.21.15.157:0/2527388760 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c15caa0 0x7fdd7c0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/2527388760 >> 172.21.15.157:0/2527388760 conn(0x7fdd7c070790 msgr2=0x7fdd7c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/2527388760 shutdown_connections 2023-11-06T03:17:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/2527388760 wait complete. 2023-11-06T03:17:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 Processor -- start 2023-11-06T03:17:22.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- start start 2023-11-06T03:17:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c0d0280 0x7fdd7c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c15caa0 0x7fdd7c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0c9d90 con 0x7fdd7c15ce80 2023-11-06T03:17:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0ca030 con 0x7fdd7c15caa0 2023-11-06T03:17:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0ca310 con 0x7fdd7c0d0280 2023-11-06T03:17:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38154/0 (socket says 172.21.15.157:38154) 2023-11-06T03:17:22.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/96829055 learned_addr learned my addr 172.21.15.157:0/96829055 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7affd640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c0d0280 0x7fdd7c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c15caa0 0x7fdd7c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c0d0280 msgr2=0x7fdd7c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c0d0280 0x7fdd7c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c15caa0 msgr2=0x7fdd7c156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c15caa0 0x7fdd7c156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 -- 172.21.15.157:0/96829055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74009b30 con 0x7fdd7c15ce80 2023-11-06T03:17:22.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fdd740af040 tx=0x7fdd74019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd637fe640 1 -- 172.21.15.157:0/96829055 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740b8670 con 0x7fdd7c15ce80 2023-11-06T03:17:22.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd637fe640 1 -- 172.21.15.157:0/96829055 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd740b8810 con 0x7fdd7c15ce80 2023-11-06T03:17:22.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c0ca5f0 con 0x7fdd7c15ce80 2023-11-06T03:17:22.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd637fe640 1 -- 172.21.15.157:0/96829055 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740b8bc0 con 0x7fdd7c15ce80 2023-11-06T03:17:22.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.899+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c14c800 con 0x7fdd7c15ce80 2023-11-06T03:17:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd637fe640 1 -- 172.21.15.157:0/96829055 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdd740b8d20 con 0x7fdd7c15ce80 2023-11-06T03:17:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd637fe640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 0x7fdd68067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd637fe640 1 -- 172.21.15.157:0/96829055 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd7411e590 con 0x7fdd7c15ce80 2023-11-06T03:17:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 0x7fdd68067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 0x7fdd68067f50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c15ff70 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 Processor -- start 2023-11-06T03:17:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 -- start start 2023-11-06T03:17:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843ce0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce1260 0x562843ce1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843ba8c10 con 0x562843d2b130 2023-11-06T03:17:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843b373e0 con 0x562843d2ba50 2023-11-06T03:17:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562843b349b0 con 0x562843ce1260 2023-11-06T03:17:22.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38156/0 (socket says 172.21.15.157:38156) 2023-11-06T03:17:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 -- 172.21.15.157:0/3853219629 learned_addr learned my addr 172.21.15.157:0/3853219629 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843ce0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce1260 0x562843ce1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 -- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce1260 msgr2=0x562843ce1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce1260 0x562843ce1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 -- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 msgr2=0x562843ce0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843ce0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7affd640 1 -- 172.21.15.157:0/3853219629 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284382ce20 con 0x562843d2b130 2023-11-06T03:17:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.903+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843ce0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00f9f0 tx=0x7fdd6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347fb2da859a8bd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd60ff9640 1 -- 172.21.15.157:0/3853219629 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c014be0 con 0x562843d2b130 2023-11-06T03:17:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd60ff9640 1 -- 172.21.15.157:0/3853219629 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c019070 con 0x562843d2b130 2023-11-06T03:17:22.918 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd60ff9640 1 -- 172.21.15.157:0/3853219629 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c020440 con 0x562843d2b130 2023-11-06T03:17:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/3853219629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 msgr2=0x562843d2b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00f9f0 tx=0x7fdd6c012000 comp rx=0 tx=0).stop 2023-11-06T03:17:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/3853219629 shutdown_connections 2023-11-06T03:17:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce1260 0x562843ce1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843ce0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- 172.21.15.157:0/3853219629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843d2b130 0x562843d2b510 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/3853219629 >> 172.21.15.157:0/3853219629 conn(0x562843ce79f0 msgr2=0x562843cdf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/3853219629 shutdown_connections 2023-11-06T03:17:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/3853219629 wait complete. 2023-11-06T03:17:22.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- start start 2023-11-06T03:17:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 Processor -- start 2023-11-06T03:17:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843d2be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843b302a0 con 0x562843ce1260 2023-11-06T03:17:22.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843acf470 con 0x562843d2ba50 2023-11-06T03:17:22.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562843aed660 con 0x562843d2b130 2023-11-06T03:17:22.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 10 client.0 _ll_register_callbacks cb 0x562843cccc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38166/0 (socket says 172.21.15.157:38166) 2023-11-06T03:17:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 -- 172.21.15.157:0/1055416823 learned_addr learned my addr 172.21.15.157:0/1055416823 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843d2be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 -- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 msgr2=0x562843d32b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d32b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 -- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 msgr2=0x562843d2be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843d2be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 -- 172.21.15.157:0/1055416823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c00e040 con 0x562843ce1260 2023-11-06T03:17:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7b7fe640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843d2be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd7affd640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c022040 tx=0x7fdd6c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508ebe6ad4b52529 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c006240 con 0x562843ce1260 2023-11-06T03:17:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd6c0063a0 con 0x562843ce1260 2023-11-06T03:17:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562843aed0a0 con 0x562843ce1260 2023-11-06T03:17:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6c006560 con 0x562843ce1260 2023-11-06T03:17:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 10 client.10605 fetch_fsmap learned FSMap version 117 2023-11-06T03:17:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x562843af51d0 con 0x562843ce1260 2023-11-06T03:17:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562843aed0a0 con 0x562843ce1260 2023-11-06T03:17:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdd6c030020 con 0x562843ce1260 2023-11-06T03:17:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 10 client.10605 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:17:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 10 client.10605 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:17:22.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 20 client.10605 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:22.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2cbd0 0x562843d2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 4 client.10605 mds_command: new command op to 10392 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:17:22.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.907+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562843b2ebf0 con 0x562843d2cbd0 2023-11-06T03:17:22.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2cbd0 0x562843d2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd59ffb640 10 client.10605 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:17:22.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2cbd0 0x562843d2cfe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a429ec40e828a05 server_cookie=cedd1a78f078f6a8 in_seq=0 out_seq=0 2023-11-06T03:17:22.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd59ffb640 1 -- 172.21.15.157:0/1055416823 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562843b2ebf0 con 0x562843d2cbd0 2023-11-06T03:17:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd59ffb640 10 client.10605 handle_command_reply: tid=0 2023-11-06T03:17:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 client.10605 shutdown 2023-11-06T03:17:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 msgr2=0x562843d32600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c022040 tx=0x7fdd6c004740 comp rx=0 tx=0).stop 2023-11-06T03:17:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 shutdown_connections 2023-11-06T03:17:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2cbd0 0x562843d2cfe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2ba50 0x562843d2be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d32b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd59ffb640 0 client.10605 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:17:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 --2- 172.21.15.157:0/1055416823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d32600 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 >> 172.21.15.157:0/1055416823 conn(0x562843ce79f0 msgr2=0x562843cdfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 shutdown_connections 2023-11-06T03:17:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 1 -- 172.21.15.157:0/1055416823 wait complete. 2023-11-06T03:17:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.911+0000 7fdd845ec000 20 client.10605 trim_cache size 0 max 16384 2023-11-06T03:17:22.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 Processor -- start 2023-11-06T03:17:22.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- start start 2023-11-06T03:17:22.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce79f0 0x562843d826b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2b130 0x562843d83130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628437edaa0 con 0x562843ce1260 2023-11-06T03:17:22.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843bb8c70 con 0x562843d2b130 2023-11-06T03:17:22.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562843b28840 con 0x562843ce79f0 2023-11-06T03:17:22.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38178/0 (socket says 172.21.15.157:38178) 2023-11-06T03:17:22.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 -- 172.21.15.157:0/2847901307 learned_addr learned my addr 172.21.15.157:0/2847901307 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 -- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce79f0 msgr2=0x562843d826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce79f0 0x562843d826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 -- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2b130 msgr2=0x562843d83130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2b130 0x562843d83130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 -- 172.21.15.157:0/2847901307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562843adb520 con 0x562843ce1260 2023-11-06T03:17:22.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0b5f80 tx=0x7fdd7001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834e907b28adf35 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd617fa640 1 -- 172.21.15.157:0/2847901307 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001a0b0 con 0x562843ce1260 2023-11-06T03:17:22.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd617fa640 1 -- 172.21.15.157:0/2847901307 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7000d070 con 0x562843ce1260 2023-11-06T03:17:22.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd617fa640 1 -- 172.21.15.157:0/2847901307 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70017430 con 0x562843ce1260 2023-11-06T03:17:22.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- 172.21.15.157:0/2847901307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 msgr2=0x562843d82bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0b5f80 tx=0x7fdd7001ad60 comp rx=0 tx=0).stop 2023-11-06T03:17:22.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- 172.21.15.157:0/2847901307 shutdown_connections 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843d2b130 0x562843d83130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d82bf0 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2847901307 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843ce79f0 0x562843d826b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- 172.21.15.157:0/2847901307 >> 172.21.15.157:0/2847901307 conn(0x562843d39080 msgr2=0x562843ce6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- 172.21.15.157:0/2847901307 shutdown_connections 2023-11-06T03:17:22.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- 172.21.15.157:0/2847901307 wait complete. 2023-11-06T03:17:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 -- start start 2023-11-06T03:17:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.927+0000 7fdd845ec000 1 Processor -- start 2023-11-06T03:17:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 0x562843d84f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843d419c0 con 0x562843ce1260 2023-11-06T03:17:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843b2bc10 con 0x562843ce79f0 2023-11-06T03:17:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562843d38f40 con 0x562843d2b130 2023-11-06T03:17:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 10 client.0 _ll_register_callbacks cb 0x562843af4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:22.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38184/0 (socket says 172.21.15.157:38184) 2023-11-06T03:17:22.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 -- 172.21.15.157:0/2457109121 learned_addr learned my addr 172.21.15.157:0/2457109121 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:22.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 0x562843d84f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 -- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 msgr2=0x562843d85e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 -- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 msgr2=0x562843d84f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 0x562843d84f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.948 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 -- 172.21.15.157:0/2457109121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7000f040 con 0x562843ce1260 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 0x562843d84f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7affd640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006aa0 tx=0x7fdd6c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de83d2bb46f4516f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c020710 con 0x562843ce1260 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdd6c0067b0 con 0x562843ce1260 2023-11-06T03:17:22.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562843af9620 con 0x562843ce1260 2023-11-06T03:17:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6c020870 con 0x562843ce1260 2023-11-06T03:17:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 10 client.10611 fetch_fsmap learned FSMap version 117 2023-11-06T03:17:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=117,monmap=2+}) v3 -- 0x562843acf090 con 0x562843ce1260 2023-11-06T03:17:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562843af9620 con 0x562843ce1260 2023-11-06T03:17:22.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 117) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdd6c030040 con 0x562843ce1260 2023-11-06T03:17:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 10 client.10611 fetch_fsmap finished waiting for FSMap version 117 2023-11-06T03:17:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 10 client.10611 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2023-11-06T03:17:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 20 client.10611 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2ba50 0x562843d86390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:22.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 4 client.10611 mds_command: new command op to 10392 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 --> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562843b29320 con 0x562843d2ba50 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2ba50 0x562843d86390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd7a7fc640 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2ba50 0x562843d86390 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e01ee5189b77a1 server_cookie=41daa2e76a9095eb in_seq=0 out_seq=0 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 10 client.10611 ms_handle_connect on v2:172.21.15.195:6835/3466838470 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 1 -- 172.21.15.157:0/2457109121 <== mds.0 v2:172.21.15.195:6835/3466838470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562843b29320 con 0x562843d2ba50 2023-11-06T03:17:22.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 10 client.10611 handle_command_reply: tid=0 2023-11-06T03:17:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 client.10611 shutdown 2023-11-06T03:17:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 msgr2=0x562843d849c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006aa0 tx=0x7fdd6c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 shutdown_connections 2023-11-06T03:17:22.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:6835/3466838470,v1:172.21.15.195:6837/3466838470] conn(0x562843d2ba50 0x562843d86390 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562843d2b130 0x562843d85e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562843ce79f0 0x562843d84f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 --2- 172.21.15.157:0/2457109121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562843ce1260 0x562843d849c0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd59ffb640 0 client.10611 ms_handle_reset on v2:172.21.15.195:6835/3466838470 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 >> 172.21.15.157:0/2457109121 conn(0x562843d39080 msgr2=0x562843d2efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 shutdown_connections 2023-11-06T03:17:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 1 -- 172.21.15.157:0/2457109121 wait complete. 2023-11-06T03:17:22.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd845ec000 20 client.10611 trim_cache size 0 max 16384 2023-11-06T03:17:22.955 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:22.955 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:17:22.956 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:17:22.956 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 msgr2=0x7fdd68067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 0x7fdd68067f50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c15ff70 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2023-11-06T03:17:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 msgr2=0x7fdd7c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:22.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fdd740af040 tx=0x7fdd74019740 comp rx=0 tx=0).stop 2023-11-06T03:17:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 shutdown_connections 2023-11-06T03:17:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdd68065a90 0x7fdd68067f50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c15ce80 0x7fdd7c15bff0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c15caa0 0x7fdd7c156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 --2- 172.21.15.157:0/96829055 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdd7c0d0280 0x7fdd7c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 >> 172.21.15.157:0/96829055 conn(0x7fdd7c070790 msgr2=0x7fdd7c0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 shutdown_connections 2023-11-06T03:17:22.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:22.931+0000 7fdd814c4640 1 -- 172.21.15.157:0/96829055 wait complete. 2023-11-06T03:17:22.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:17:22.961 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:17:22.961 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:17:22.962 DEBUG:teuthology.orchestra.run.smithi157:> import stat 2023-11-06T03:17:22.962 DEBUG:teuthology.orchestra.run.smithi157:> import json 2023-11-06T03:17:22.963 DEBUG:teuthology.orchestra.run.smithi157:> import sys 2023-11-06T03:17:22.963 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:22.963 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:17:22.964 DEBUG:teuthology.orchestra.run.smithi157:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-11-06T03:17:22.964 DEBUG:teuthology.orchestra.run.smithi157:> except OSError as e: 2023-11-06T03:17:22.964 DEBUG:teuthology.orchestra.run.smithi157:> sys.exit(e.errno) 2023-11-06T03:17:22.965 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:22.965 DEBUG:teuthology.orchestra.run.smithi157:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-06T03:17:22.965 DEBUG:teuthology.orchestra.run.smithi157:> print(json.dumps( 2023-11-06T03:17:22.966 DEBUG:teuthology.orchestra.run.smithi157:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-06T03:17:22.966 DEBUG:teuthology.orchestra.run.smithi157:> indent=2)) 2023-11-06T03:17:22.966 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:17:23.027 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:23.028 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mode": 33204, 2023-11-06T03:17:23.028 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ino": 1099511627777, 2023-11-06T03:17:23.029 INFO:teuthology.orchestra.run.smithi157.stdout: "st_dev": 45, 2023-11-06T03:17:23.029 INFO:teuthology.orchestra.run.smithi157.stdout: "st_nlink": 1, 2023-11-06T03:17:23.030 INFO:teuthology.orchestra.run.smithi157.stdout: "st_uid": 1000, 2023-11-06T03:17:23.030 INFO:teuthology.orchestra.run.smithi157.stdout: "st_gid": 1271, 2023-11-06T03:17:23.030 INFO:teuthology.orchestra.run.smithi157.stdout: "st_size": 6291456, 2023-11-06T03:17:23.031 INFO:teuthology.orchestra.run.smithi157.stdout: "st_atime": 1699240611.1528065, 2023-11-06T03:17:23.031 INFO:teuthology.orchestra.run.smithi157.stdout: "st_mtime": 1699240611.1985438, 2023-11-06T03:17:23.032 INFO:teuthology.orchestra.run.smithi157.stdout: "st_ctime": 1699240611.1985438 2023-11-06T03:17:23.032 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:23.215 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:23.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 Processor -- start 2023-11-06T03:17:23.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 -- start start 2023-11-06T03:17:23.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16d00ce800 0x7f16d00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 0x7f16d006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d00cf1a0 con 0x7f16d0146370 2023-11-06T03:17:23.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d00b2120 con 0x7f16d00ce800 2023-11-06T03:17:23.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16d4f45640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d006c3d0 con 0x7f16d006b6a0 2023-11-06T03:17:23.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16d00ce800 0x7f16d00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f16d00ce800 0x7f16d00cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38198/0 (socket says 172.21.15.157:38198) 2023-11-06T03:17:23.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 -- 172.21.15.157:0/915306681 learned_addr learned my addr 172.21.15.157:0/915306681 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:23.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16ced9d640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 0x7f16d006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 -- 172.21.15.157:0/915306681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 msgr2=0x7f16d006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:23.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 0x7f16d006bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 -- 172.21.15.157:0/915306681 >> v1:172.21.15.195:6789/0 conn(0x7f16d0146370 legacy=0x7f16d00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16cdd9b640 1 -- 172.21.15.157:0/915306681 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d006f5a0 con 0x7f16d00ce800 2023-11-06T03:17:23.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.391+0000 7f16ced9d640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 0x7f16d006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:23.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cdd9b640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d00cec60 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009e50 tx=0x7f16c001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c9f1abeb0ea1aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cd59a640 1 -- 172.21.15.157:0/915306681 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00a5810 con 0x7f16d00ce800 2023-11-06T03:17:23.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cd59a640 1 -- 172.21.15.157:0/915306681 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00a5970 con 0x7f16d00ce800 2023-11-06T03:17:23.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cd59a640 1 -- 172.21.15.157:0/915306681 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00a5d00 con 0x7f16d00ce800 2023-11-06T03:17:23.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- 172.21.15.157:0/915306681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 msgr2=0x7f16d00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d00cec60 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009e50 tx=0x7f16c001c950 comp rx=0 tx=0).stop 2023-11-06T03:17:23.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- 172.21.15.157:0/915306681 shutdown_connections 2023-11-06T03:17:23.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d006b6a0 0x7f16d006bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- 172.21.15.157:0/915306681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d00cec60 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- 172.21.15.157:0/915306681 >> 172.21.15.157:0/915306681 conn(0x7f16d0065930 msgr2=0x7f16d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:23.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- 172.21.15.157:0/915306681 shutdown_connections 2023-11-06T03:17:23.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- 172.21.15.157:0/915306681 wait complete. 2023-11-06T03:17:23.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 Processor -- start 2023-11-06T03:17:23.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- start start 2023-11-06T03:17:23.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16d006b6a0 0x7f16d013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d0146370 0x7f16d013dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d01404a0 con 0x7f16d00ce800 2023-11-06T03:17:23.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d01406f0 con 0x7f16d006b6a0 2023-11-06T03:17:23.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16d4f45640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d01409d0 con 0x7f16d0146370 2023-11-06T03:17:23.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cdd9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38202/0 (socket says 172.21.15.157:38202) 2023-11-06T03:17:23.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16cdd9b640 1 -- 172.21.15.157:0/103547968 learned_addr learned my addr 172.21.15.157:0/103547968 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:23.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16ce59c640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16d006b6a0 0x7f16d013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.395+0000 7f16ced9d640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d0146370 0x7f16d013dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 -- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d0146370 msgr2=0x7f16d013dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d0146370 0x7f16d013dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 -- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16d006b6a0 msgr2=0x7f16d013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16d006b6a0 0x7f16d013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 -- 172.21.15.157:0/103547968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009b30 con 0x7f16d00ce800 2023-11-06T03:17:23.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16cdd9b640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f16c00af040 tx=0x7f16c00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00b8670 con 0x7f16d00ce800 2023-11-06T03:17:23.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00b87d0 con 0x7f16d00ce800 2023-11-06T03:17:23.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d0140cb0 con 0x7f16d00ce800 2023-11-06T03:17:23.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00b8b80 con 0x7f16d00ce800 2023-11-06T03:17:23.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d013c9c0 con 0x7f16d00ce800 2023-11-06T03:17:23.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d01411d0 con 0x7f16d00ce800 2023-11-06T03:17:23.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f16c00b8ce0 con 0x7f16d00ce800 2023-11-06T03:17:23.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 0x7f16b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16c011e800 con 0x7f16d00ce800 2023-11-06T03:17:23.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16ce59c640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 0x7f16b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.399+0000 7f16ce59c640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 0x7f16b0067f50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007fd0 tx=0x7f16c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.403+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f16d01411d0 con 0x7f16d00ce800 2023-11-06T03:17:23.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16d01411d0 con 0x7f16d00ce800 2023-11-06T03:17:23.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16beffd640 1 -- 172.21.15.157:0/103547968 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f16d01411d0 con 0x7f16d00ce800 2023-11-06T03:17:23.596 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:23.596 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:17:23.596 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:17:23.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 msgr2=0x7f16b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 0x7f16b0067f50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f16c4007fd0 tx=0x7f16c4009040 comp rx=0 tx=0).stop 2023-11-06T03:17:23.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 msgr2=0x7f16d013d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f16c00af040 tx=0x7f16c00196f0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 shutdown_connections 2023-11-06T03:17:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f16b0065a90 0x7f16b0067f50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f16d0146370 0x7f16d013dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16d00ce800 0x7f16d013d6f0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 --2- 172.21.15.157:0/103547968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16d006b6a0 0x7f16d013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 >> 172.21.15.157:0/103547968 conn(0x7f16d0065930 msgr2=0x7f16d00b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:23.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 shutdown_connections 2023-11-06T03:17:23.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.599+0000 7f16d4f45640 1 -- 172.21.15.157:0/103547968 wait complete. 2023-11-06T03:17:23.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-06T03:17:23.770 INFO:teuthology.orchestra.run.smithi195.stdout:{"message":"","return_code":0} 2023-11-06T03:17:23.771 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T03:17:23.771 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-06T03:17:23.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 Processor -- start 2023-11-06T03:17:23.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 -- start start 2023-11-06T03:17:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc0b8e0 0x55b09dce14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dcdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09db51450 con 0x55b09dc534b0 2023-11-06T03:17:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09db51150 con 0x55b09dc0b8e0 2023-11-06T03:17:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f51afd080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09db47dc0 con 0x55b09dc0bcc0 2023-11-06T03:17:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38220/0 (socket says 172.21.15.157:38220) 2023-11-06T03:17:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 -- 172.21.15.157:0/1273031919 learned_addr learned my addr 172.21.15.157:0/1273031919 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4f617640 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc0b8e0 0x55b09dce14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4ee16640 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dcdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 -- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 msgr2=0x55b09dcdf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dcdf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 -- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc0b8e0 msgr2=0x55b09dce14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc0b8e0 0x55b09dce14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 -- 172.21.15.157:0/1273031919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48005df0 con 0x55b09dc534b0 2023-11-06T03:17:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.815+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f9f4801b910 tx=0x7f9f48003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7236f8f22549895a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4de14640 1 -- 172.21.15.157:0/1273031919 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480aa680 con 0x55b09dc534b0 2023-11-06T03:17:23.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4de14640 1 -- 172.21.15.157:0/1273031919 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f480b7030 con 0x55b09dc534b0 2023-11-06T03:17:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4de14640 1 -- 172.21.15.157:0/1273031919 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480aaa00 con 0x55b09dc534b0 2023-11-06T03:17:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/1273031919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 msgr2=0x55b09dc4a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f9f4801b910 tx=0x7f9f48003030 comp rx=0 tx=0).stop 2023-11-06T03:17:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/1273031919 shutdown_connections 2023-11-06T03:17:23.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc534b0 0x55b09dc4a340 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dcdf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- 172.21.15.157:0/1273031919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc0b8e0 0x55b09dce14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/1273031919 >> 172.21.15.157:0/1273031919 conn(0x55b09dc35ef0 msgr2=0x55b09dc090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/1273031919 shutdown_connections 2023-11-06T03:17:23.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/1273031919 wait complete. 2023-11-06T03:17:23.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 Processor -- start 2023-11-06T03:17:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- start start 2023-11-06T03:17:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 0x55b09dc4a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dce85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc534b0 0x55b09dce8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09dcc1a00 con 0x55b09dc0b8e0 2023-11-06T03:17:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09dcc44d0 con 0x55b09dc534b0 2023-11-06T03:17:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09dcc4770 con 0x55b09dc0bcc0 2023-11-06T03:17:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 0x55b09dc4a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 0x55b09dc4a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38222/0 (socket says 172.21.15.157:38222) 2023-11-06T03:17:23.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 -- 172.21.15.157:0/67257446 learned_addr learned my addr 172.21.15.157:0/67257446 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:23.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc534b0 0x55b09dce8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4ee16640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dce85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 -- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 msgr2=0x55b09dce85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55b09dc0bcc0 0x55b09dce85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 -- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc534b0 msgr2=0x55b09dce8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b09dc534b0 0x55b09dce8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 -- 172.21.15.157:0/67257446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b09db7a190 con 0x55b09dc0b8e0 2023-11-06T03:17:23.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4f617640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 0x55b09dc4a730 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d3c0 tx=0x7f9f4000d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4ce12640 1 -- 172.21.15.157:0/67257446 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400125f0 con 0x55b09dc0b8e0 2023-11-06T03:17:23.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4ce12640 1 -- 172.21.15.157:0/67257446 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40012750 con 0x55b09dc0b8e0 2023-11-06T03:17:23.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b09db5fff0 con 0x55b09dc0b8e0 2023-11-06T03:17:23.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f4ce12640 1 -- 172.21.15.157:0/67257446 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40012ae0 con 0x55b09dc0b8e0 2023-11-06T03:17:23.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.819+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b09db53430 con 0x55b09dc0b8e0 2023-11-06T03:17:23.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4ce12640 1 -- 172.21.15.157:0/67257446 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9f40012c40 con 0x55b09dc0b8e0 2023-11-06T03:17:23.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4ce12640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9f30064410 0x7f9f300668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4ce12640 1 -- 172.21.15.157:0/67257446 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f40082090 con 0x55b09dc0b8e0 2023-11-06T03:17:23.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4ee16640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9f30064410 0x7f9f300668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55b09dd7b2a0 0x55b09dcc3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:23.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- osd_op(unknown.0.0:1 24.7 24:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e163) v8 -- 0x55b09dcc39a0 con 0x55b09dd7b2a0 2023-11-06T03:17:23.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55b09dd7b2a0 0x55b09dcc3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:23.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4ee16640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9f30064410 0x7f9f300668d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005fd0 tx=0x7f9f38005b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4fe18640 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55b09dd7b2a0 0x55b09dcc3460 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:23.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4fe18640 1 -- 172.21.15.157:0/67257446 <== osd.7 v2:172.21.15.195:6800/38427 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f9f480bc640 con 0x55b09dd7b2a0 2023-11-06T03:17:23.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55b09dd7b2a0 msgr2=0x55b09dcc3460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x55b09dd7b2a0 0x55b09dcc3460 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:23.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9f30064410 msgr2=0x7f9f300668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9f30064410 0x7f9f300668d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005fd0 tx=0x7f9f38005b90 comp rx=0 tx=0).stop 2023-11-06T03:17:23.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 msgr2=0x55b09dc4a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:23.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 --2- 172.21.15.157:0/67257446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b09dc0b8e0 0x55b09dc4a730 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d3c0 tx=0x7f9f4000d880 comp rx=0 tx=0).stop 2023-11-06T03:17:23.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f4fe18640 1 -- 172.21.15.157:0/67257446 reap_dead start 2023-11-06T03:17:23.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 shutdown_connections 2023-11-06T03:17:23.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 >> 172.21.15.157:0/67257446 conn(0x55b09dc35ef0 msgr2=0x55b09dce0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:23.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 shutdown_connections 2023-11-06T03:17:23.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:23.823+0000 7f9f51afd080 1 -- 172.21.15.157:0/67257446 wait complete. 2023-11-06T03:17:23.837 INFO:teuthology.orchestra.run.smithi157.stdout:Ysixmegssubdir  2023-11-06T03:17:23.837 DEBUG:teuthology.orchestra.run.smithi157:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-06T03:17:24.016 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:24.017 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 1099511627777, 2023-11-06T03:17:24.017 INFO:teuthology.orchestra.run.smithi157.stdout: "ancestors": [ 2023-11-06T03:17:24.017 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:24.017 INFO:teuthology.orchestra.run.smithi157.stdout: "dirino": 1099511627776, 2023-11-06T03:17:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: "dname": "sixmegs", 2023-11-06T03:17:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: "version": 8 2023-11-06T03:17:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:24.018 INFO:teuthology.orchestra.run.smithi157.stdout: "dirino": 1, 2023-11-06T03:17:24.019 INFO:teuthology.orchestra.run.smithi157.stdout: "dname": "subdir", 2023-11-06T03:17:24.019 INFO:teuthology.orchestra.run.smithi157.stdout: "version": 11 2023-11-06T03:17:24.019 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:24.019 INFO:teuthology.orchestra.run.smithi157.stdout: ], 2023-11-06T03:17:24.019 INFO:teuthology.orchestra.run.smithi157.stdout: "pool": 24, 2023-11-06T03:17:24.020 INFO:teuthology.orchestra.run.smithi157.stdout: "old_pools": [] 2023-11-06T03:17:24.020 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:24.020 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:24.022 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:17:24.022 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:24.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:24.127 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:17:24.128 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:24.128 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:24.135 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:24.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 Processor -- start 2023-11-06T03:17:24.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 -- start start 2023-11-06T03:17:24.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbba80aff30 0x7fbba80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba806bda0 con 0x7fbba80b2490 2023-11-06T03:17:24.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80b0310 con 0x7fbba80aff30 2023-11-06T03:17:24.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbb07b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba806b7e0 con 0x7fbba8071db0 2023-11-06T03:17:24.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbae552640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbba80aff30 0x7fbba80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:48008/0 (socket says 172.21.15.157:48008) 2023-11-06T03:17:24.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 -- 172.21.15.157:0/2450692776 learned_addr learned my addr 172.21.15.157:0/2450692776 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:24.310 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 -- 172.21.15.157:0/2450692776 >> v2:172.21.15.195:3300/0 conn(0x7fbba80aff30 msgr2=0x7fbba80ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:24.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 --2- 172.21.15.157:0/2450692776 >> v2:172.21.15.195:3300/0 conn(0x7fbba80aff30 0x7fbba80ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 -- 172.21.15.157:0/2450692776 >> v1:172.21.15.195:6789/0 conn(0x7fbba80b2490 legacy=0x7fbba8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.310+0000 7fbbadd51640 1 -- 172.21.15.157:0/2450692776 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba806bac0 con 0x7fbba8071db0 2023-11-06T03:17:24.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbadd51640 1 --2- 172.21.15.157:0/2450692776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009870 tx=0x7fbb9801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a38ee18b677516 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbad550640 1 -- 172.21.15.157:0/2450692776 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980ab680 con 0x7fbba8071db0 2023-11-06T03:17:24.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbad550640 1 -- 172.21.15.157:0/2450692776 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb980ab7e0 con 0x7fbba8071db0 2023-11-06T03:17:24.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbad550640 1 -- 172.21.15.157:0/2450692776 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980abb70 con 0x7fbba8071db0 2023-11-06T03:17:24.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- 172.21.15.157:0/2450692776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 msgr2=0x7fbba806af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/2450692776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009870 tx=0x7fbb9801c480 comp rx=0 tx=0).stop 2023-11-06T03:17:24.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- 172.21.15.157:0/2450692776 shutdown_connections 2023-11-06T03:17:24.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/2450692776 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba8071db0 0x7fbba806af90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/2450692776 >> v2:172.21.15.195:3300/0 conn(0x7fbba80aff30 0x7fbba80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- 172.21.15.157:0/2450692776 >> 172.21.15.157:0/2450692776 conn(0x7fbba80cd490 msgr2=0x7fbba80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:24.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- 172.21.15.157:0/2450692776 shutdown_connections 2023-11-06T03:17:24.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- 172.21.15.157:0/2450692776 wait complete. 2023-11-06T03:17:24.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 Processor -- start 2023-11-06T03:17:24.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.314+0000 7fbbb07b6640 1 -- start start 2023-11-06T03:17:24.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba80aff30 0x7fbba80b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba80b2490 0x7fbba8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80ac950 con 0x7fbba8071db0 2023-11-06T03:17:24.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81f9930 con 0x7fbba80aff30 2023-11-06T03:17:24.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81f9c10 con 0x7fbba80b2490 2023-11-06T03:17:24.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38232/0 (socket says 172.21.15.157:38232) 2023-11-06T03:17:24.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 -- 172.21.15.157:0/738099645 learned_addr learned my addr 172.21.15.157:0/738099645 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:24.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbaed53640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba80b2490 0x7fbba8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbadd51640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba80aff30 0x7fbba80b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 -- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba80b2490 msgr2=0x7fbba8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba80b2490 0x7fbba8156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 -- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba80aff30 msgr2=0x7fbba80b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba80aff30 0x7fbba80b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 -- 172.21.15.157:0/738099645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980094d0 con 0x7fbba8071db0 2023-11-06T03:17:24.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbae552640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e870 tx=0x7fbba400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4009510 con 0x7fbba8071db0 2023-11-06T03:17:24.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400eec0 con 0x7fbba8071db0 2023-11-06T03:17:24.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba81f9ef0 con 0x7fbba8071db0 2023-11-06T03:17:24.321 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cb40 con 0x7fbba8071db0 2023-11-06T03:17:24.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba80af170 con 0x7fbba8071db0 2023-11-06T03:17:24.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8072750 con 0x7fbba8071db0 2023-11-06T03:17:24.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbba400cca0 con 0x7fbba8071db0 2023-11-06T03:17:24.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 0x7fbb8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbba4083440 con 0x7fbba8071db0 2023-11-06T03:17:24.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.318+0000 7fbbadd51640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 0x7fbb8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.322+0000 7fbbadd51640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 0x7fbb8c067f20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbb980af000 tx=0x7fbb980a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.322+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbba4050ca0 con 0x7fbba8071db0 2023-11-06T03:17:24.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.478+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbba80b6990 con 0x7fbba8071db0 2023-11-06T03:17:24.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbb9effd640 1 -- 172.21.15.157:0/738099645 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fbba40542c0 con 0x7fbba8071db0 2023-11-06T03:17:24.479 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:24.480 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":163,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:44.441779+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:24.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 msgr2=0x7fbb8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 0x7fbb8c067f20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbb980af000 tx=0x7fbb980a7000 comp rx=0 tx=0).stop 2023-11-06T03:17:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 msgr2=0x7fbba80b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e870 tx=0x7fbba400ec70 comp rx=0 tx=0).stop 2023-11-06T03:17:24.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 shutdown_connections 2023-11-06T03:17:24.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb8c065a60 0x7fbb8c067f20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbba80b2490 0x7fbba8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbba80aff30 0x7fbba80b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 --2- 172.21.15.157:0/738099645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbba8071db0 0x7fbba80b0970 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 >> 172.21.15.157:0/738099645 conn(0x7fbba80cd490 msgr2=0x7fbba80cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:24.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 shutdown_connections 2023-11-06T03:17:24.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.482+0000 7fbbb07b6640 1 -- 172.21.15.157:0/738099645 wait complete. 2023-11-06T03:17:24.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:16:42.003695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:16:42.405981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:17:24.500 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:17:24.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.626+0000 7fe94066a640 1 Processor -- start 2023-11-06T03:17:24.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.626+0000 7fe94066a640 1 -- start start 2023-11-06T03:17:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe94066a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9380d0280 0x7fe938167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe94066a640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9380d2920 con 0x7fe93815ce80 2023-11-06T03:17:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe94066a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9380d18a0 con 0x7fe93815caa0 2023-11-06T03:17:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe94066a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9380d1b60 con 0x7fe9380d0280 2023-11-06T03:17:24.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93ec07640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9380d0280 0x7fe938167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93dc05640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe93815ce80 0x7fe9380cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41308/0 (socket says 172.21.15.157:41308) 2023-11-06T03:17:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93dc05640 1 -- 172.21.15.157:0/2906275767 learned_addr learned my addr 172.21.15.157:0/2906275767 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:24.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93ec07640 1 -- 172.21.15.157:0/2906275767 >> v1:172.21.15.195:6789/0 conn(0x7fe93815caa0 legacy=0x7fe9380cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93ec07640 1 -- 172.21.15.157:0/2906275767 >> v1:172.21.15.157:6789/0 conn(0x7fe93815ce80 legacy=0x7fe9380cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93ec07640 1 -- 172.21.15.157:0/2906275767 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9380d1e40 con 0x7fe9380d0280 2023-11-06T03:17:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93ec07640 1 --2- 172.21.15.157:0/2906275767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe938167860 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fe930009d90 tx=0x7fe93001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3b105d6e7655ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93d404640 1 -- 172.21.15.157:0/2906275767 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300a59e0 con 0x7fe9380d0280 2023-11-06T03:17:24.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93d404640 1 -- 172.21.15.157:0/2906275767 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9300a5b40 con 0x7fe9380d0280 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.630+0000 7fe93d404640 1 -- 172.21.15.157:0/2906275767 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300ad440 con 0x7fe9380d0280 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/2906275767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 msgr2=0x7fe938167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 --2- 172.21.15.157:0/2906275767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe938167860 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fe930009d90 tx=0x7fe93001c920 comp rx=0 tx=0).stop 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/2906275767 shutdown_connections 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 --2- 172.21.15.157:0/2906275767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe938167860 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/2906275767 >> 172.21.15.157:0/2906275767 conn(0x7fe938070790 msgr2=0x7fe938070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/2906275767 shutdown_connections 2023-11-06T03:17:24.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/2906275767 wait complete. 2023-11-06T03:17:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 Processor -- start 2023-11-06T03:17:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- start start 2023-11-06T03:17:24.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 0x7fe93815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93815ce80 0x7fe93815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9380c7f70 con 0x7fe9380d0280 2023-11-06T03:17:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93815e9f0 con 0x7fe93815caa0 2023-11-06T03:17:24.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93815ecd0 con 0x7fe93815ce80 2023-11-06T03:17:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38250/0 (socket says 172.21.15.157:38250) 2023-11-06T03:17:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 -- 172.21.15.157:0/3770120357 learned_addr learned my addr 172.21.15.157:0/3770120357 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93ec07640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93815ce80 0x7fe93815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93dc05640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 0x7fe93815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 -- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93815ce80 msgr2=0x7fe93815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93815ce80 0x7fe93815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 -- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 msgr2=0x7fe93815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 0x7fe93815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 -- 172.21.15.157:0/3770120357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930009a70 con 0x7fe9380d0280 2023-11-06T03:17:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93dc05640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 0x7fe93815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe93e406640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fe93400c420 tx=0x7fe93400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340052d0 con 0x7fe9380d0280 2023-11-06T03:17:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe934005430 con 0x7fe9380d0280 2023-11-06T03:17:24.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93815efb0 con 0x7fe9380d0280 2023-11-06T03:17:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934007260 con 0x7fe9380d0280 2023-11-06T03:17:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93815f570 con 0x7fe9380d0280 2023-11-06T03:17:24.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.634+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9380d0850 con 0x7fe9380d0280 2023-11-06T03:17:24.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.638+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9340073c0 con 0x7fe9380d0280 2023-11-06T03:17:24.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.638+0000 7fe92effd640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 0x7fe91c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.638+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe934082ee0 con 0x7fe9380d0280 2023-11-06T03:17:24.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.638+0000 7fe93dc05640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 0x7fe91c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.638+0000 7fe93dc05640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 0x7fe91c067f20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe9381591c0 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.642+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe934050820 con 0x7fe9380d0280 2023-11-06T03:17:24.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.794+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9380b1ed0 con 0x7fe9380d0280 2023-11-06T03:17:24.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe92effd640 1 -- 172.21.15.157:0/3770120357 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9340200a0 con 0x7fe9380d0280 2023-11-06T03:17:24.795 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:17:24.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 msgr2=0x7fe91c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 0x7fe91c067f20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe9381591c0 tx=0x7fe928009040 comp rx=0 tx=0).stop 2023-11-06T03:17:24.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 msgr2=0x7fe9380c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7fe93400c420 tx=0x7fe93400c8e0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 shutdown_connections 2023-11-06T03:17:24.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe91c065a60 0x7fe91c067f20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93815ce80 0x7fe93815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe93815caa0 0x7fe93815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 --2- 172.21.15.157:0/3770120357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9380d0280 0x7fe9380c7a30 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 >> 172.21.15.157:0/3770120357 conn(0x7fe938070790 msgr2=0x7fe938070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:24.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 shutdown_connections 2023-11-06T03:17:24.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.798+0000 7fe94066a640 1 -- 172.21.15.157:0/3770120357 wait complete. 2023-11-06T03:17:24.814 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:17:24.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 Processor -- start 2023-11-06T03:17:24.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 -- start start 2023-11-06T03:17:24.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f247015caa0 0x7f24700cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f24700cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700d2920 con 0x7f24700ce1a0 2023-11-06T03:17:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700d18a0 con 0x7f247015caa0 2023-11-06T03:17:24.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f24783e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700d1b60 con 0x7f247015ce80 2023-11-06T03:17:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247617d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f247015caa0 0x7f24700cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247697e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f24700ce1a0 0x7f24700d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41318/0 (socket says 172.21.15.157:41318) 2023-11-06T03:17:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247697e640 1 -- 172.21.15.157:0/202761140 learned_addr learned my addr 172.21.15.157:0/202761140 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247597c640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f24700cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247617d640 1 -- 172.21.15.157:0/202761140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 msgr2=0x7f24700cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247617d640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f24700cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247617d640 1 -- 172.21.15.157:0/202761140 >> v1:172.21.15.157:6789/0 conn(0x7f24700ce1a0 legacy=0x7f24700d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247517b640 1 -- 172.21.15.157:0/202761140 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756905794 0 0) 0x7f24700d2920 con 0x7f24700ce1a0 2023-11-06T03:17:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247617d640 1 -- 172.21.15.157:0/202761140 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700d1e40 con 0x7f247015caa0 2023-11-06T03:17:24.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.942+0000 7f247597c640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f24700cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f247617d640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247015caa0 0x7f24700cdc60 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f246c0097c0 tx=0x7f246c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e515a4f44bce55c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f247517b640 1 -- 172.21.15.157:0/202761140 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0ad660 con 0x7f247015caa0 2023-11-06T03:17:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f247517b640 1 -- 172.21.15.157:0/202761140 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c0ad7c0 con 0x7f247015caa0 2023-11-06T03:17:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f247517b640 1 -- 172.21.15.157:0/202761140 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0adb50 con 0x7f247015caa0 2023-11-06T03:17:24.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 -- 172.21.15.157:0/202761140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247015caa0 msgr2=0x7f24700cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247015caa0 0x7f24700cdc60 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f246c0097c0 tx=0x7f246c01c450 comp rx=0 tx=0).stop 2023-11-06T03:17:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 -- 172.21.15.157:0/202761140 shutdown_connections 2023-11-06T03:17:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f24700cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 --2- 172.21.15.157:0/202761140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f247015caa0 0x7f24700cdc60 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 -- 172.21.15.157:0/202761140 >> 172.21.15.157:0/202761140 conn(0x7f2470070790 msgr2=0x7f2470070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 -- 172.21.15.157:0/202761140 shutdown_connections 2023-11-06T03:17:24.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.946+0000 7f24783e1640 1 -- 172.21.15.157:0/202761140 wait complete. 2023-11-06T03:17:24.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 Processor -- start 2023-11-06T03:17:24.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- start start 2023-11-06T03:17:24.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f247015caa0 0x7f247015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f247015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247014fe00 con 0x7f24700ce1a0 2023-11-06T03:17:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24700c8810 con 0x7f247015caa0 2023-11-06T03:17:24.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2470151160 con 0x7f247015ce80 2023-11-06T03:17:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38264/0 (socket says 172.21.15.157:38264) 2023-11-06T03:17:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 -- 172.21.15.157:0/4262111455 learned_addr learned my addr 172.21.15.157:0/4262111455 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 -- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 msgr2=0x7f247015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247597c640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f247015caa0 0x7f247015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247697e640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f247015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f247015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 -- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f247015caa0 msgr2=0x7f247015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f247015caa0 0x7f247015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:24.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 -- 172.21.15.157:0/4262111455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c009420 con 0x7f24700ce1a0 2023-11-06T03:17:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247617d640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f246c0b0040 tx=0x7f246c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0b9860 con 0x7f24700ce1a0 2023-11-06T03:17:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c0b99e0 con 0x7f24700ce1a0 2023-11-06T03:17:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2470150550 con 0x7f24700ce1a0 2023-11-06T03:17:24.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0b9d90 con 0x7f24700ce1a0 2023-11-06T03:17:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2470150b10 con 0x7f24700ce1a0 2023-11-06T03:17:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24700b1ec0 con 0x7f24700ce1a0 2023-11-06T03:17:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f246c0c4460 con 0x7f24700ce1a0 2023-11-06T03:17:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 0x7f2454067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:24.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f246c11dff0 con 0x7f24700ce1a0 2023-11-06T03:17:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.950+0000 7f247597c640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 0x7f2454067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.954+0000 7f247597c640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 0x7f2454067f20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2460006fd0 tx=0x7f2460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:24.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:24.954+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f246c0eb850 con 0x7f24700ce1a0 2023-11-06T03:17:25.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.114+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2470072750 con 0x7f24700ce1a0 2023-11-06T03:17:25.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.114+0000 7f2466ffd640 1 -- 172.21.15.157:0/4262111455 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7f246c0eee70 con 0x7f24700ce1a0 2023-11-06T03:17:25.113 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:17:25.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 msgr2=0x7f2454067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 0x7f2454067f20 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2460006fd0 tx=0x7f2460008040 comp rx=0 tx=0).stop 2023-11-06T03:17:25.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 msgr2=0x7f247014ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f246c0b0040 tx=0x7f246c01ac90 comp rx=0 tx=0).stop 2023-11-06T03:17:25.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 shutdown_connections 2023-11-06T03:17:25.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2454065a60 0x7f2454067f20 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f247015ce80 0x7f247015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f247015caa0 0x7f247015fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 --2- 172.21.15.157:0/4262111455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24700ce1a0 0x7f247014ff70 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 >> 172.21.15.157:0/4262111455 conn(0x7f2470070790 msgr2=0x7f24700b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 shutdown_connections 2023-11-06T03:17:25.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.118+0000 7f24783e1640 1 -- 172.21.15.157:0/4262111455 wait complete. 2023-11-06T03:17:25.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:17:25.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.262+0000 7ffb26384640 1 Processor -- start 2023-11-06T03:17:25.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.262+0000 7ffb26384640 1 -- start start 2023-11-06T03:17:25.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb26384640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb2015ac80 0x7ffb200b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb26384640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffb200cf560 0x7ffb200b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb26384640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200cc2b0 con 0x7ffb200afb00 2023-11-06T03:17:25.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb26384640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200b01a0 con 0x7ffb2015ac80 2023-11-06T03:17:25.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb26384640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200cfb10 con 0x7ffb200cf560 2023-11-06T03:17:25.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb2015ac80 0x7ffb200b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb177fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffb200afb00 0x7ffb200cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41332/0 (socket says 172.21.15.157:41332) 2023-11-06T03:17:25.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb177fe640 1 -- 172.21.15.157:0/4286061883 learned_addr learned my addr 172.21.15.157:0/4286061883 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:25.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/4286061883 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441658665 0 0) 0x7ffb200cc2b0 con 0x7ffb200afb00 2023-11-06T03:17:25.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/4286061883 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c003530 con 0x7ffb200afb00 2023-11-06T03:17:25.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 -- 172.21.15.157:0/4286061883 >> v2:172.21.15.195:3300/0 conn(0x7ffb200cf560 msgr2=0x7ffb200b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:25.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/4286061883 >> v2:172.21.15.195:3300/0 conn(0x7ffb200cf560 0x7ffb200b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 -- 172.21.15.157:0/4286061883 >> v1:172.21.15.157:6789/0 conn(0x7ffb200afb00 legacy=0x7ffb200cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 -- 172.21.15.157:0/4286061883 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200cfdf0 con 0x7ffb2015ac80 2023-11-06T03:17:25.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/4286061883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb200b2400 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070fe0 tx=0x7ffb1001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3129b72c8f574 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/4286061883 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100ab600 con 0x7ffb2015ac80 2023-11-06T03:17:25.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/4286061883 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100a7050 con 0x7ffb2015ac80 2023-11-06T03:17:25.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.266+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/4286061883 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100ab960 con 0x7ffb2015ac80 2023-11-06T03:17:25.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/4286061883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 msgr2=0x7ffb200b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- 172.21.15.157:0/4286061883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb200b2400 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070fe0 tx=0x7ffb1001c3d0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/4286061883 shutdown_connections 2023-11-06T03:17:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- 172.21.15.157:0/4286061883 >> v2:172.21.15.195:3300/0 conn(0x7ffb200cf560 0x7ffb200b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- 172.21.15.157:0/4286061883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb200b2400 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/4286061883 >> 172.21.15.157:0/4286061883 conn(0x7ffb2006d560 msgr2=0x7ffb2006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/4286061883 shutdown_connections 2023-11-06T03:17:25.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/4286061883 wait complete. 2023-11-06T03:17:25.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 Processor -- start 2023-11-06T03:17:25.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- start start 2023-11-06T03:17:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 0x7ffb20149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb200cf560 0x7ffb20151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2014a470 con 0x7ffb2015ac80 2023-11-06T03:17:25.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20151b60 con 0x7ffb200afb00 2023-11-06T03:17:25.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20151e40 con 0x7ffb200cf560 2023-11-06T03:17:25.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38280/0 (socket says 172.21.15.157:38280) 2023-11-06T03:17:25.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 -- 172.21.15.157:0/1638161854 learned_addr learned my addr 172.21.15.157:0/1638161854 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:25.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 0x7ffb20149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb177fe640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb200cf560 0x7ffb20151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 -- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb200cf560 msgr2=0x7ffb20151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb200cf560 0x7ffb20151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 -- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 msgr2=0x7ffb20149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 0x7ffb20149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 -- 172.21.15.157:0/1638161854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10009420 con 0x7ffb2015ac80 2023-11-06T03:17:25.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 0x7ffb20149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb24921640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b970 tx=0x7ffb1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800cb40 con 0x7ffb2015ac80 2023-11-06T03:17:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1800ccc0 con 0x7ffb2015ac80 2023-11-06T03:17:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb20152120 con 0x7ffb2015ac80 2023-11-06T03:17:25.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18014680 con 0x7ffb2015ac80 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.270+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201526e0 con 0x7ffb2015ac80 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb20152900 con 0x7ffb2015ac80 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffb1801d3b0 con 0x7ffb2015ac80 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb1d7fa640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 0x7ffb00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb180831d0 con 0x7ffb2015ac80 2023-11-06T03:17:25.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 0x7ffb00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.274+0000 7ffb1ffff640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 0x7ffb00067f20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffb100ad000 tx=0x7ffb100a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.278+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffb18050ac0 con 0x7ffb2015ac80 2023-11-06T03:17:25.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.430+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb20072750 con 0x7ffb2015ac80 2023-11-06T03:17:25.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.430+0000 7ffb1d7fa640 1 -- 172.21.15.157:0/1638161854 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v163) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb180540e0 con 0x7ffb2015ac80 2023-11-06T03:17:25.430 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:17:25.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 msgr2=0x7ffb00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 0x7ffb00067f20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffb100ad000 tx=0x7ffb100a6000 comp rx=0 tx=0).stop 2023-11-06T03:17:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 msgr2=0x7ffb201590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b970 tx=0x7ffb1800be30 comp rx=0 tx=0).stop 2023-11-06T03:17:25.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 shutdown_connections 2023-11-06T03:17:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb00065a60 0x7ffb00067f20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb2015ac80 0x7ffb201590a0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb200cf560 0x7ffb20151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 --2- 172.21.15.157:0/1638161854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb200afb00 0x7ffb20149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 >> 172.21.15.157:0/1638161854 conn(0x7ffb2006d560 msgr2=0x7ffb2006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 shutdown_connections 2023-11-06T03:17:25.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.434+0000 7ffb26384640 1 -- 172.21.15.157:0/1638161854 wait complete. 2023-11-06T03:17:25.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:17:25.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 Processor -- start 2023-11-06T03:17:25.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 -- start start 2023-11-06T03:17:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 0x7ff5480b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5480ce800 0x7ff5480cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5480cf1a0 con 0x7ff54806b6a0 2023-11-06T03:17:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54806c3d0 con 0x7ff5480ce800 2023-11-06T03:17:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff54e5c2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54806f5a0 con 0x7ff548146370 2023-11-06T03:17:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5480ce800 0x7ff5480cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5480ce800 0x7ff5480cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38284/0 (socket says 172.21.15.157:38284) 2023-11-06T03:17:25.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 -- 172.21.15.157:0/651756805 learned_addr learned my addr 172.21.15.157:0/651756805 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff547fff640 1 --2- 172.21.15.157:0/651756805 >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 0x7ff5480b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 -- 172.21.15.157:0/651756805 >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 msgr2=0x7ff5480b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 --2- 172.21.15.157:0/651756805 >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 0x7ff5480b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 -- 172.21.15.157:0/651756805 >> v1:172.21.15.195:6789/0 conn(0x7ff54806b6a0 legacy=0x7ff54806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:25.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 -- 172.21.15.157:0/651756805 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54806f880 con 0x7ff5480ce800 2023-11-06T03:17:25.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff547fff640 1 --2- 172.21.15.157:0/651756805 >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 0x7ff5480b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:25.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff53f7fe640 1 --2- 172.21.15.157:0/651756805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5480ce800 0x7ff5480cec60 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7ff530009870 tx=0x7ff53001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e4cdbd176ee874 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff5477fe640 1 -- 172.21.15.157:0/651756805 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300ab600 con 0x7ff5480ce800 2023-11-06T03:17:25.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff5477fe640 1 -- 172.21.15.157:0/651756805 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5300ab760 con 0x7ff5480ce800 2023-11-06T03:17:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.570+0000 7ff5477fe640 1 -- 172.21.15.157:0/651756805 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300abaf0 con 0x7ff5480ce800 2023-11-06T03:17:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/651756805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5480ce800 msgr2=0x7ff5480cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/651756805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5480ce800 0x7ff5480cec60 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7ff530009870 tx=0x7ff53001c400 comp rx=0 tx=0).stop 2023-11-06T03:17:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/651756805 shutdown_connections 2023-11-06T03:17:25.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/651756805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5480ce800 0x7ff5480cec60 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/651756805 >> v2:172.21.15.195:3300/0 conn(0x7ff548146370 0x7ff5480b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/651756805 >> 172.21.15.157:0/651756805 conn(0x7ff548065930 msgr2=0x7ff548065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/651756805 shutdown_connections 2023-11-06T03:17:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/651756805 wait complete. 2023-11-06T03:17:25.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 Processor -- start 2023-11-06T03:17:25.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- start start 2023-11-06T03:17:25.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5480ce800 0x7ff548144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff548146370 0x7ff5481588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff548145280 con 0x7ff54806b6a0 2023-11-06T03:17:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff548158de0 con 0x7ff5480ce800 2023-11-06T03:17:25.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5481f8310 con 0x7ff548146370 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38286/0 (socket says 172.21.15.157:38286) 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 -- 172.21.15.157:0/1888121580 learned_addr learned my addr 172.21.15.157:0/1888121580 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54cb5f640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff548146370 0x7ff5481588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff53f7fe640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5480ce800 0x7ff548144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 -- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff548146370 msgr2=0x7ff5481588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff548146370 0x7ff5481588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 -- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5480ce800 msgr2=0x7ff548144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5480ce800 0x7ff548144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 -- 172.21.15.157:0/1888121580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300094d0 con 0x7ff54806b6a0 2023-11-06T03:17:25.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff547fff640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff53800c7f0 tx=0x7ff53800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800cea0 con 0x7ff54806b6a0 2023-11-06T03:17:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538007a30 con 0x7ff54806b6a0 2023-11-06T03:17:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5481f88d0 con 0x7ff54806b6a0 2023-11-06T03:17:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538007de0 con 0x7ff54806b6a0 2023-11-06T03:17:25.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.574+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff548145750 con 0x7ff54806b6a0 2023-11-06T03:17:25.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff548072750 con 0x7ff54806b6a0 2023-11-06T03:17:25.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff5380113f0 con 0x7ff54806b6a0 2023-11-06T03:17:25.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff5457fa640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 0x7ff528067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5380834a0 con 0x7ff54806b6a0 2023-11-06T03:17:25.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff53f7fe640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 0x7ff528067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.578+0000 7ff53f7fe640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 0x7ff528067f20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff5300af000 tx=0x7ff5300a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.582+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff538050c60 con 0x7ff54806b6a0 2023-11-06T03:17:25.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff548160cf0 con 0x7ff54806b6a0 2023-11-06T03:17:25.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff5457fa640 1 -- 172.21.15.157:0/1888121580 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v163) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff538054280 con 0x7ff54806b6a0 2023-11-06T03:17:25.724 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:17:25.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 msgr2=0x7ff528067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 0x7ff528067f20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff5300af000 tx=0x7ff5300a7000 comp rx=0 tx=0).stop 2023-11-06T03:17:25.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 msgr2=0x7ff5481f8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff53800c7f0 tx=0x7ff53800ccb0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 shutdown_connections 2023-11-06T03:17:25.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff528065a60 0x7ff528067f20 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff548146370 0x7ff5481588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5480ce800 0x7ff548144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 --2- 172.21.15.157:0/1888121580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff54806b6a0 0x7ff5481f8d60 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 >> 172.21.15.157:0/1888121580 conn(0x7ff548065930 msgr2=0x7ff5480b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 shutdown_connections 2023-11-06T03:17:25.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.726+0000 7ff54e5c2640 1 -- 172.21.15.157:0/1888121580 wait complete. 2023-11-06T03:17:25.742 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:17:25.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 Processor -- start 2023-11-06T03:17:25.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 -- start start 2023-11-06T03:17:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4081599c0 0x7fa4080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa40806d000 0x7fa4080b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40815e440 con 0x7fa40815dea0 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40815adb0 con 0x7fa40806d000 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40f1ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40806c240 con 0x7fa4081599c0 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa40806d000 0x7fa4080b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa40806d000 0x7fa4080b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38288/0 (socket says 172.21.15.157:38288) 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 -- 172.21.15.157:0/563275937 learned_addr learned my addr 172.21.15.157:0/563275937 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:25.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa40cf66640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4081599c0 0x7fa4080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 -- 172.21.15.157:0/563275937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4081599c0 msgr2=0x7fa4080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4081599c0 0x7fa4080ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 -- 172.21.15.157:0/563275937 >> v1:172.21.15.195:6789/0 conn(0x7fa40815dea0 legacy=0x7fa408070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.862+0000 7fa407fff640 1 -- 172.21.15.157:0/563275937 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40806c520 con 0x7fa40806d000 2023-11-06T03:17:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa407fff640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40806d000 0x7fa4080b23e0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009e50 tx=0x7fa3f801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31aefe4fe8b92963 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa4077fe640 1 -- 172.21.15.157:0/563275937 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a57c0 con 0x7fa40806d000 2023-11-06T03:17:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa4077fe640 1 -- 172.21.15.157:0/563275937 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f80a5920 con 0x7fa40806d000 2023-11-06T03:17:25.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa4077fe640 1 -- 172.21.15.157:0/563275937 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80a5cb0 con 0x7fa40806d000 2023-11-06T03:17:25.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- 172.21.15.157:0/563275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40806d000 msgr2=0x7fa4080b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40806d000 0x7fa4080b23e0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009e50 tx=0x7fa3f801c9e0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- 172.21.15.157:0/563275937 shutdown_connections 2023-11-06T03:17:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40806d000 0x7fa4080b23e0 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/563275937 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa4081599c0 0x7fa4080ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- 172.21.15.157:0/563275937 >> 172.21.15.157:0/563275937 conn(0x7fa408065930 msgr2=0x7fa408065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- 172.21.15.157:0/563275937 shutdown_connections 2023-11-06T03:17:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- 172.21.15.157:0/563275937 wait complete. 2023-11-06T03:17:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 Processor -- start 2023-11-06T03:17:25.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.866+0000 7fa40f1ca640 1 -- start start 2023-11-06T03:17:25.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40806d000 0x7fa4081567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4081599c0 0x7fa40814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408156ce0 con 0x7fa40815dea0 2023-11-06T03:17:25.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40814e970 con 0x7fa4081599c0 2023-11-06T03:17:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4081523f0 con 0x7fa40806d000 2023-11-06T03:17:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38298/0 (socket says 172.21.15.157:38298) 2023-11-06T03:17:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 -- 172.21.15.157:0/1180977919 learned_addr learned my addr 172.21.15.157:0/1180977919 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:25.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 -- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40806d000 msgr2=0x7fa4081567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40806d000 0x7fa4081567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 -- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4081599c0 msgr2=0x7fa40814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:25.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4081599c0 0x7fa40814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:25.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 -- 172.21.15.157:0/1180977919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8009b30 con 0x7fa40815dea0 2023-11-06T03:17:25.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40d767640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c340 tx=0x7fa3f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0005220 con 0x7fa40815dea0 2023-11-06T03:17:25.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005380 con 0x7fa40815dea0 2023-11-06T03:17:25.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4080cbed0 con 0x7fa40815dea0 2023-11-06T03:17:25.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007270 con 0x7fa40815dea0 2023-11-06T03:17:25.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4080cc490 con 0x7fa40815dea0 2023-11-06T03:17:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408072750 con 0x7fa40815dea0 2023-11-06T03:17:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa3f00073d0 con 0x7fa40815dea0 2023-11-06T03:17:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 0x7fa3e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3f00827d0 con 0x7fa40815dea0 2023-11-06T03:17:25.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.870+0000 7fa40cf66640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 0x7fa3e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.874+0000 7fa40cf66640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 0x7fa3e8067f20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa4000098f0 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:25.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:25.874+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa3f00500c0 con 0x7fa40815dea0 2023-11-06T03:17:26.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.030+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4080cb110 con 0x7fa40815dea0 2023-11-06T03:17:26.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.030+0000 7fa4057fa640 1 -- 172.21.15.157:0/1180977919 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v163) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3f00536e0 con 0x7fa40815dea0 2023-11-06T03:17:26.030 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:17:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 msgr2=0x7fa3e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 0x7fa3e8067f20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa4000098f0 tx=0x7fa400008040 comp rx=0 tx=0).stop 2023-11-06T03:17:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 msgr2=0x7fa40814eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c340 tx=0x7fa3f000c800 comp rx=0 tx=0).stop 2023-11-06T03:17:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 shutdown_connections 2023-11-06T03:17:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3e8065a60 0x7fa3e8067f20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa40815dea0 0x7fa40814eb40 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4081599c0 0x7fa40814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 --2- 172.21.15.157:0/1180977919 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa40806d000 0x7fa4081567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 >> 172.21.15.157:0/1180977919 conn(0x7fa408065930 msgr2=0x7fa408065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:26.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 shutdown_connections 2023-11-06T03:17:26.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.034+0000 7fa40f1ca640 1 -- 172.21.15.157:0/1180977919 wait complete. 2023-11-06T03:17:26.047 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:17:26.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:26.049 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:26.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[95482]: fuse finished with error 0 and tester_r 0 2023-11-06T03:17:26.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:17:26.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:26.064 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:17:26.232 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:26.232 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:26.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:26.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:17:26.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:26.241 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:17:26.286 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:17:26.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:26.288 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:26.289 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:26.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:17:26.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:17:26.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:26.297 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:17:26.342 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:17:26.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:26.344 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:26.345 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:26.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:26.351 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:17:26.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:26.352 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:17:26.402 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:17:26.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:26.404 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:26.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 Processor -- start 2023-11-06T03:17:26.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 -- start start 2023-11-06T03:17:26.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7b080cf6b0 0x7f7b080cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 0x7f7b080b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080d0050 con 0x7f7b0815cdf0 2023-11-06T03:17:26.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b080d1c60 con 0x7f7b080cf6b0 2023-11-06T03:17:26.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0c958640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080d1f40 con 0x7f7b080b6110 2023-11-06T03:17:26.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7b0815cdf0 0x7f7b080d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41340/0 (socket says 172.21.15.157:41340) 2023-11-06T03:17:26.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 -- 172.21.15.157:0/4137068275 learned_addr learned my addr 172.21.15.157:0/4137068275 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:26.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b05d9b640 1 --2- 172.21.15.157:0/4137068275 >> v2:172.21.15.195:3300/0 conn(0x7f7b080cf6b0 0x7f7b080cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523899909 0 0) 0x7f7b080d0050 con 0x7f7b0815cdf0 2023-11-06T03:17:26.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af0003530 con 0x7f7b0815cdf0 2023-11-06T03:17:26.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b06d9d640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 0x7f7b080b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2941371120 0 0) 0x7f7af0003530 con 0x7f7b0815cdf0 2023-11-06T03:17:26.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 msgr2=0x7f7b080b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 0x7f7b080b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 >> v2:172.21.15.195:3300/0 conn(0x7f7b080cf6b0 msgr2=0x7f7b080cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 --2- 172.21.15.157:0/4137068275 >> v2:172.21.15.195:3300/0 conn(0x7f7b080cf6b0 0x7f7b080cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080d2220 con 0x7f7b0815cdf0 2023-11-06T03:17:26.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7af8001680 con 0x7f7b0815cdf0 2023-11-06T03:17:26.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 >> v1:172.21.15.157:6789/0 conn(0x7f7b0815cdf0 legacy=0x7f7b080d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0003980 0x7f7af0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 0x7f7af00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0004d10 0x7f7af0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080d1c60 con 0x7f7af0003980 2023-11-06T03:17:26.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080d1f40 con 0x7f7af0004d10 2023-11-06T03:17:26.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af000b990 con 0x7f7b080b6110 2023-11-06T03:17:26.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0003980 0x7f7af0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b06d9d640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0004d10 0x7f7af0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 -- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 msgr2=0x7f7af00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b080b6110 0x7f7af00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 -- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0004d10 msgr2=0x7f7af0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0004d10 0x7f7af0005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.534+0000 7f7b0659c640 1 -- 172.21.15.157:0/4137068275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0003530 con 0x7f7af0003980 2023-11-06T03:17:26.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0659c640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0003980 0x7f7af0003da0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c290 tx=0x7f7af801ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59997d1df165e72d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80a5900 con 0x7f7af0003980 2023-11-06T03:17:26.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af80a5a60 con 0x7f7af0003980 2023-11-06T03:17:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0559a640 1 -- 172.21.15.157:0/4137068275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80a5e10 con 0x7f7af0003980 2023-11-06T03:17:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- 172.21.15.157:0/4137068275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0003980 msgr2=0x7f7af0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 --2- 172.21.15.157:0/4137068275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7af0003980 0x7f7af0003da0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c290 tx=0x7f7af801ccb0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b06d9d640 1 -- 172.21.15.157:0/4137068275 reap_dead start 2023-11-06T03:17:26.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- 172.21.15.157:0/4137068275 shutdown_connections 2023-11-06T03:17:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- 172.21.15.157:0/4137068275 >> 172.21.15.157:0/4137068275 conn(0x7f7b08065930 msgr2=0x7f7b08065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- 172.21.15.157:0/4137068275 shutdown_connections 2023-11-06T03:17:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- 172.21.15.157:0/4137068275 wait complete. 2023-11-06T03:17:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 Processor -- start 2023-11-06T03:17:26.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- start start 2023-11-06T03:17:26.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0003980 0x7f7b080c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b08154e30 0x7f7b081552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080ca3b0 con 0x7f7b080c8810 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081557f0 con 0x7f7af0003980 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.538+0000 7f7b0c958640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081591a0 con 0x7f7b08154e30 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38318/0 (socket says 172.21.15.157:38318) 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 -- 172.21.15.157:0/2703631813 learned_addr learned my addr 172.21.15.157:0/2703631813 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:26.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b0659c640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0003980 0x7f7b080c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b06d9d640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b08154e30 0x7f7b081552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 -- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b08154e30 msgr2=0x7f7b081552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b08154e30 0x7f7b081552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 -- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0003980 msgr2=0x7f7b080c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0003980 0x7f7b080c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 -- 172.21.15.157:0/2703631813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b081f5140 con 0x7f7b080c8810 2023-11-06T03:17:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b05d9b640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e840 tx=0x7f7afc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc009620 con 0x7f7b080c8810 2023-11-06T03:17:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00c8e0 con 0x7f7b080c8810 2023-11-06T03:17:26.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b081f5420 con 0x7f7b080c8810 2023-11-06T03:17:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00cc90 con 0x7f7b080c8810 2023-11-06T03:17:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b081f59e0 con 0x7f7b080c8810 2023-11-06T03:17:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b081f5ba0 con 0x7f7b080c8810 2023-11-06T03:17:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7afc013470 con 0x7f7b080c8810 2023-11-06T03:17:26.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 0x7f7ae8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7afc083320 con 0x7f7b080c8810 2023-11-06T03:17:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.542+0000 7f7b0659c640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 0x7f7ae8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.546+0000 7f7b0659c640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 0x7f7ae8067f20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7af80aa000 tx=0x7f7af8001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.546+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7afc050b80 con 0x7f7b080c8810 2023-11-06T03:17:26.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.738+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b08072750 con 0x7f7b080c8810 2023-11-06T03:17:26.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.738+0000 7f7af6ffd640 1 -- 172.21.15.157:0/2703631813 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f7afc0541a0 con 0x7f7b080c8810 2023-11-06T03:17:26.737 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:26.737 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:17:26.738 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:17:26.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 msgr2=0x7f7ae8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 0x7f7ae8067f20 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7af80aa000 tx=0x7f7af8001a10 comp rx=0 tx=0).stop 2023-11-06T03:17:26.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 msgr2=0x7f7b08158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00e840 tx=0x7f7afc00ed00 comp rx=0 tx=0).stop 2023-11-06T03:17:26.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 shutdown_connections 2023-11-06T03:17:26.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7ae8065a60 0x7f7ae8067f20 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b08154e30 0x7f7b081552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080c8810 0x7f7b08158c60 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 --2- 172.21.15.157:0/2703631813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7af0003980 0x7f7b080c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 >> 172.21.15.157:0/2703631813 conn(0x7f7b08065930 msgr2=0x7f7b0815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 shutdown_connections 2023-11-06T03:17:26.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.742+0000 7f7b0c958640 1 -- 172.21.15.157:0/2703631813 wait complete. 2023-11-06T03:17:26.756 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:26.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.870+0000 7f7201eba640 1 Processor -- start 2023-11-06T03:17:26.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.870+0000 7f7201eba640 1 -- start start 2023-11-06T03:17:26.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f7201eba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 0x7f71fc167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f7201eba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0d2920 con 0x7f71fc15caa0 2023-11-06T03:17:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f7201eba640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0d18a0 con 0x7f71fc15ce80 2023-11-06T03:17:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f7201eba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d1b60 con 0x7f71fc0d0280 2023-11-06T03:17:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f71fc15caa0 0x7f71fc0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41354/0 (socket says 172.21.15.157:41354) 2023-11-06T03:17:26.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 -- 172.21.15.157:0/1371201577 learned_addr learned my addr 172.21.15.157:0/1371201577 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:26.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690871016 0 0) 0x7f71fc0d2920 con 0x7f71fc15caa0 2023-11-06T03:17:26.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fbfff640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 0x7f71fc167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e4003530 con 0x7f71fc15caa0 2023-11-06T03:17:26.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881008908 0 0) 0x7f71e4003530 con 0x7f71fc15caa0 2023-11-06T03:17:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 msgr2=0x7f71fc167860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 0x7f71fc167860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 >> v1:172.21.15.195:6789/0 conn(0x7f71fc15ce80 legacy=0x7f71fc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0d1e40 con 0x7f71fc15caa0 2023-11-06T03:17:26.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f71ec001680 con 0x7f71fc15caa0 2023-11-06T03:17:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 >> v1:172.21.15.157:6789/0 conn(0x7f71fc15caa0 legacy=0x7f71fc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e40039b0 0x7f71e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 0x7f71e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e4004d10 0x7f71e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0d1b60 con 0x7f71e40039b0 2023-11-06T03:17:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e400ba70 con 0x7f71e4004d10 2023-11-06T03:17:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e400bc30 con 0x7f71fc0d0280 2023-11-06T03:17:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e40039b0 0x7f71e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 -- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 msgr2=0x7f71e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0d0280 0x7f71e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 -- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e4004d10 msgr2=0x7f71e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e4004d10 0x7f71e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 -- 172.21.15.157:0/1371201577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4003530 con 0x7f71e40039b0 2023-11-06T03:17:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e40039b0 0x7f71e4003dd0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00c240 tx=0x7f71ec01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9415baae6cd1a562 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0a5af0 con 0x7f71e40039b0 2023-11-06T03:17:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec0a5c50 con 0x7f71e40039b0 2023-11-06T03:17:26.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.874+0000 7f71fa7fc640 1 -- 172.21.15.157:0/1371201577 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0058e0 con 0x7f71e40039b0 2023-11-06T03:17:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/1371201577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e40039b0 msgr2=0x7f71e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 --2- 172.21.15.157:0/1371201577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71e40039b0 0x7f71e4003dd0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00c240 tx=0x7f71ec01cc10 comp rx=0 tx=0).stop 2023-11-06T03:17:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 -- 172.21.15.157:0/1371201577 reap_dead start 2023-11-06T03:17:26.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/1371201577 shutdown_connections 2023-11-06T03:17:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/1371201577 >> 172.21.15.157:0/1371201577 conn(0x7f71fc070790 msgr2=0x7f71fc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/1371201577 shutdown_connections 2023-11-06T03:17:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/1371201577 wait complete. 2023-11-06T03:17:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 Processor -- start 2023-11-06T03:17:26.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- start start 2023-11-06T03:17:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e40039b0 0x7f71fc1420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 0x7f71fc15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc156520 con 0x7f71fc0c8160 2023-11-06T03:17:26.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc142630 con 0x7f71e40039b0 2023-11-06T03:17:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc155910 con 0x7f71fc0c7ce0 2023-11-06T03:17:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38328/0 (socket says 172.21.15.157:38328) 2023-11-06T03:17:26.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 -- 172.21.15.157:0/2349745275 learned_addr learned my addr 172.21.15.157:0/2349745275 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71faffd640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 0x7f71fc15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e40039b0 0x7f71fc1420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 -- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 msgr2=0x7f71fc15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 0x7f71fc15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 -- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e40039b0 msgr2=0x7f71fc1420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:26.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e40039b0 0x7f71fc1420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:26.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 -- 172.21.15.157:0/2349745275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc155bf0 con 0x7f71fc0c8160 2023-11-06T03:17:26.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71faffd640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 0x7f71fc15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:26.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71fbfff640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c66f0 tx=0x7f71f400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4010040 con 0x7f71fc0c8160 2023-11-06T03:17:26.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400bae0 con 0x7f71fc0c8160 2023-11-06T03:17:26.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc155ed0 con 0x7f71fc0c8160 2023-11-06T03:17:26.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4014400 con 0x7f71fc0c8160 2023-11-06T03:17:26.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.878+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc153fb0 con 0x7f71fc0c8160 2023-11-06T03:17:26.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc0b6d20 con 0x7f71fc0c8160 2023-11-06T03:17:26.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f71f4014560 con 0x7f71fc0c8160 2023-11-06T03:17:26.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f71e3fff640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 0x7f71e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:26.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71f40834f0 con 0x7f71fc0c8160 2023-11-06T03:17:26.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 0x7f71e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:26.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.882+0000 7f71fb7fe640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 0x7f71e8067f20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0ab000 tx=0x7f71ec0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:26.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:26.886+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f71f4050d50 con 0x7f71fc0c8160 2023-11-06T03:17:27.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.030+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71fc072750 con 0x7f71fc0c8160 2023-11-06T03:17:27.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.030+0000 7f71e3fff640 1 -- 172.21.15.157:0/2349745275 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17873 (secure 0 0 0) 0x7f71f4054370 con 0x7f71fc0c8160 2023-11-06T03:17:27.031 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:27.031 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":163,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:44.441779+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:27.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 msgr2=0x7f71e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 0x7f71e8067f20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0ab000 tx=0x7f71ec0a7070 comp rx=0 tx=0).stop 2023-11-06T03:17:27.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 msgr2=0x7f71fc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0c66f0 tx=0x7f71f400b480 comp rx=0 tx=0).stop 2023-11-06T03:17:27.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 shutdown_connections 2023-11-06T03:17:27.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f71e8065a60 0x7f71e8067f20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71fc0c8160 0x7f71fc15ad10 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f71fc0c7ce0 0x7f71fc15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 --2- 172.21.15.157:0/2349745275 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71e40039b0 0x7f71fc1420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 >> 172.21.15.157:0/2349745275 conn(0x7f71fc070790 msgr2=0x7f71fc0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 shutdown_connections 2023-11-06T03:17:27.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.034+0000 7f7201eba640 1 -- 172.21.15.157:0/2349745275 wait complete. 2023-11-06T03:17:27.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:16:42.003695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:16:42.405981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:17:27.047 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:17:27.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 Processor -- start 2023-11-06T03:17:27.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 -- start start 2023-11-06T03:17:27.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f172415caa0 0x7f17240cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.172 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240d2920 con 0x7f172415ce80 2023-11-06T03:17:27.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240d18a0 con 0x7f172415caa0 2023-11-06T03:17:27.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172bede640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240d1b60 con 0x7f17240d0280 2023-11-06T03:17:27.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f1729c7a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f172415caa0 0x7f17240cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.173 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:48166/0 (socket says 172.21.15.157:48166) 2023-11-06T03:17:27.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 -- 172.21.15.157:0/152607508 learned_addr learned my addr 172.21.15.157:0/152607508 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:27.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 -- 172.21.15.157:0/152607508 >> v2:172.21.15.195:3300/0 conn(0x7f172415caa0 msgr2=0x7f17240cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 --2- 172.21.15.157:0/152607508 >> v2:172.21.15.195:3300/0 conn(0x7f172415caa0 0x7f17240cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.174 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 -- 172.21.15.157:0/152607508 >> v1:172.21.15.195:6789/0 conn(0x7f172415ce80 legacy=0x7f17240cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.175 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.174+0000 7f172a47b640 1 -- 172.21.15.157:0/152607508 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240d1e40 con 0x7f17240d0280 2023-11-06T03:17:27.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172a47b640 1 --2- 172.21.15.157:0/152607508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f171c009e50 tx=0x7f171c01c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfafce865f9693e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1728c78640 1 -- 172.21.15.157:0/152607508 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0a58a0 con 0x7f17240d0280 2023-11-06T03:17:27.176 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1728c78640 1 -- 172.21.15.157:0/152607508 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c0a5a00 con 0x7f17240d0280 2023-11-06T03:17:27.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1728c78640 1 -- 172.21.15.157:0/152607508 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0a5d90 con 0x7f17240d0280 2023-11-06T03:17:27.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- 172.21.15.157:0/152607508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 msgr2=0x7f17240b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- 172.21.15.157:0/152607508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f171c009e50 tx=0x7f171c01c9e0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.177 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- 172.21.15.157:0/152607508 shutdown_connections 2023-11-06T03:17:27.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- 172.21.15.157:0/152607508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f17240b5d50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- 172.21.15.157:0/152607508 >> v2:172.21.15.195:3300/0 conn(0x7f172415caa0 0x7f17240cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- 172.21.15.157:0/152607508 >> 172.21.15.157:0/152607508 conn(0x7f1724070790 msgr2=0x7f1724070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.178 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- 172.21.15.157:0/152607508 shutdown_connections 2023-11-06T03:17:27.179 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- 172.21.15.157:0/152607508 wait complete. 2023-11-06T03:17:27.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 Processor -- start 2023-11-06T03:17:27.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- start start 2023-11-06T03:17:27.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f172415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.182 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f172415ce80 0x7f1724150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724152990 con 0x7f172415caa0 2023-11-06T03:17:27.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724153200 con 0x7f172415ce80 2023-11-06T03:17:27.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f172bede640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1724151d10 con 0x7f17240d0280 2023-11-06T03:17:27.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1729479640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1729479640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38338/0 (socket says 172.21.15.157:38338) 2023-11-06T03:17:27.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.178+0000 7f1729479640 1 -- 172.21.15.157:0/1017251194 learned_addr learned my addr 172.21.15.157:0/1017251194 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:27.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f172a47b640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f172415ce80 0x7f1724150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729c7a640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f172415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 -- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 msgr2=0x7f172415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f172415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 -- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f172415ce80 msgr2=0x7f1724150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f172415ce80 0x7f1724150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 -- 172.21.15.157:0/1017251194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c009b30 con 0x7f172415caa0 2023-11-06T03:17:27.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729479640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f171400c4d0 tx=0x7f171400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140052d0 con 0x7f172415caa0 2023-11-06T03:17:27.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714005430 con 0x7f172415caa0 2023-11-06T03:17:27.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1724151ff0 con 0x7f172415caa0 2023-11-06T03:17:27.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714003800 con 0x7f172415caa0 2023-11-06T03:17:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241525b0 con 0x7f172415caa0 2023-11-06T03:17:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1724152ad0 con 0x7f172415caa0 2023-11-06T03:17:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1714003960 con 0x7f172415caa0 2023-11-06T03:17:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 0x7f1708067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1714082640 con 0x7f172415caa0 2023-11-06T03:17:27.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729c7a640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 0x7f1708067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.182+0000 7f1729c7a640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 0x7f1708067f20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1720009af0 tx=0x7f1720008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.186+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1724152ad0 con 0x7f172415caa0 2023-11-06T03:17:27.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.342+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1724152ad0 con 0x7f172415caa0 2023-11-06T03:17:27.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.342+0000 7f171a7fc640 1 -- 172.21.15.157:0/1017251194 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1724152ad0 con 0x7f172415caa0 2023-11-06T03:17:27.340 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:17:27.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 msgr2=0x7f1708067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 0x7f1708067f20 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1720009af0 tx=0x7f1720008040 comp rx=0 tx=0).stop 2023-11-06T03:17:27.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 msgr2=0x7f17241563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f171400c4d0 tx=0x7f171400c990 comp rx=0 tx=0).stop 2023-11-06T03:17:27.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 shutdown_connections 2023-11-06T03:17:27.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1708065a60 0x7f1708067f20 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f172415ce80 0x7f1724150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172415caa0 0x7f17241563a0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 --2- 172.21.15.157:0/1017251194 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17240d0280 0x7f172415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 >> 172.21.15.157:0/1017251194 conn(0x7f1724070790 msgr2=0x7f17240d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 shutdown_connections 2023-11-06T03:17:27.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.346+0000 7f172bede640 1 -- 172.21.15.157:0/1017251194 wait complete. 2023-11-06T03:17:27.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:17:27.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.482+0000 7f6dd29b8640 1 Processor -- start 2023-11-06T03:17:27.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.482+0000 7f6dd29b8640 1 -- start start 2023-11-06T03:17:27.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd29b8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6dcc0cf6b0 0x7f6dcc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd29b8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0d0050 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd29b8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0d1c60 con 0x7f6dcc0b6110 2023-11-06T03:17:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd29b8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc0d1f40 con 0x7f6dcc0cf6b0 2023-11-06T03:17:27.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6dcc15cdf0 0x7f6dcc0d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41364/0 (socket says 172.21.15.157:41364) 2023-11-06T03:17:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcbfff640 1 -- 172.21.15.157:0/1807918262 learned_addr learned my addr 172.21.15.157:0/1807918262 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867014059 0 0) 0x7f6dcc0d0050 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/1807918262 >> v2:172.21.15.195:3300/0 conn(0x7f6dcc0cf6b0 0x7f6dcc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db8003530 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289859764 0 0) 0x7f6db8003530 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 >> v2:172.21.15.195:3300/0 conn(0x7f6dcc0cf6b0 msgr2=0x7f6dcc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 --2- 172.21.15.157:0/1807918262 >> v2:172.21.15.195:3300/0 conn(0x7f6dcc0cf6b0 0x7f6dcc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 >> v1:172.21.15.195:6789/0 conn(0x7f6dcc0b6110 legacy=0x7f6dcc0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0d2220 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6dbc001680 con 0x7f6dcc15cdf0 2023-11-06T03:17:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 >> v1:172.21.15.157:6789/0 conn(0x7f6dcc15cdf0 legacy=0x7f6dcc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6db8003980 0x7f6db8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6db8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6db8005190 0x7f6db800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0d1f40 con 0x7f6db80047d0 2023-11-06T03:17:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db800c030 con 0x7f6db8005190 2023-11-06T03:17:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db800c1f0 con 0x7f6db8003980 2023-11-06T03:17:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6db8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 -- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6db8003980 msgr2=0x7f6db8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6db8003980 0x7f6db8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 -- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6db8005190 msgr2=0x7f6db800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd0f55640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6db8005190 0x7f6db800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6db8005190 0x7f6db800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dcb7fe640 1 -- 172.21.15.157:0/1807918262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db8003530 con 0x7f6db80047d0 2023-11-06T03:17:27.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd0f55640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6db8005190 0x7f6db800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:27.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.486+0000 7f6dd0f55640 1 -- 172.21.15.157:0/1807918262 reap_dead start 2023-11-06T03:17:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6db8004c50 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009ae0 tx=0x7f6db401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292911bbe9f48bd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db40a5640 con 0x7f6db80047d0 2023-11-06T03:17:27.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db40a57a0 con 0x7f6db80047d0 2023-11-06T03:17:27.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcaffd640 1 -- 172.21.15.157:0/1807918262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db40a5b50 con 0x7f6db80047d0 2023-11-06T03:17:27.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 msgr2=0x7f6db8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6db8004c50 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009ae0 tx=0x7f6db401ca50 comp rx=0 tx=0).stop 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/1807918262 shutdown_connections 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/1807918262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6db8004c50 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/1807918262 >> 172.21.15.157:0/1807918262 conn(0x7f6dcc065930 msgr2=0x7f6dcc065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/1807918262 shutdown_connections 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/1807918262 wait complete. 2023-11-06T03:17:27.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 Processor -- start 2023-11-06T03:17:27.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- start start 2023-11-06T03:17:27.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dcc0c9350 0x7f6dcc15fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 0x7f6dcc0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c8f30 con 0x7f6db80047d0 2023-11-06T03:17:27.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c82b0 con 0x7f6dcc0c6620 2023-11-06T03:17:27.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0c8590 con 0x7f6dcc0c9350 2023-11-06T03:17:27.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38360/0 (socket says 172.21.15.157:38360) 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 -- 172.21.15.157:0/2801910567 learned_addr learned my addr 172.21.15.157:0/2801910567 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd0f55640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 0x7f6dcc0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dcc0c9350 0x7f6dcc15fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 -- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dcc0c9350 msgr2=0x7f6dcc15fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dcc0c9350 0x7f6dcc15fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.499 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 -- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 msgr2=0x7f6dcc0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 0x7f6dcc0b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 -- 172.21.15.157:0/2801910567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0c8870 con 0x7f6db80047d0 2023-11-06T03:17:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd0f55640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 0x7f6dcc0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dcbfff640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0011f0 tx=0x7f6dbc00f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc007d00 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc007e60 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc0c8b50 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00a310 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.490+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0c6aa0 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc0c6cf0 con 0x7f6db80047d0 2023-11-06T03:17:27.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6dbc00a470 con 0x7f6db80047d0 2023-11-06T03:17:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dc8ff9640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 0x7f6dac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6dbc085780 con 0x7f6db80047d0 2023-11-06T03:17:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 0x7f6dac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.494+0000 7f6dcb7fe640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 0x7f6dac067f20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6db40af000 tx=0x7f6db4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.498+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6dcc0c6cf0 con 0x7f6db80047d0 2023-11-06T03:17:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.642+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dcc0c6cf0 con 0x7f6db80047d0 2023-11-06T03:17:27.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.642+0000 7f6dc8ff9640 1 -- 172.21.15.157:0/2801910567 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dcc0c6cf0 con 0x7f6db80047d0 2023-11-06T03:17:27.640 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:17:27.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 msgr2=0x7f6dac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 0x7f6dac067f20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6db40af000 tx=0x7f6db4008040 comp rx=0 tx=0).stop 2023-11-06T03:17:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 msgr2=0x7f6dcc142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0011f0 tx=0x7f6dbc00f960 comp rx=0 tx=0).stop 2023-11-06T03:17:27.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 shutdown_connections 2023-11-06T03:17:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6dac065a60 0x7f6dac067f20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dcc0c6620 0x7f6dcc0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dcc0c9350 0x7f6dcc15fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 --2- 172.21.15.157:0/2801910567 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6db80047d0 0x7f6dcc142cf0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 >> 172.21.15.157:0/2801910567 conn(0x7f6dcc065930 msgr2=0x7f6dcc0b7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 shutdown_connections 2023-11-06T03:17:27.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.646+0000 7f6dd29b8640 1 -- 172.21.15.157:0/2801910567 wait complete. 2023-11-06T03:17:27.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:17:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 Processor -- start 2023-11-06T03:17:27.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 -- start start 2023-11-06T03:17:27.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb71415ce80 0x7fb7140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 0x7fb7140b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140d2920 con 0x7fb71415caa0 2023-11-06T03:17:27.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7140d0520 con 0x7fb71415ce80 2023-11-06T03:17:27.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb71a038640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140d18a0 con 0x7fb7140d00b0 2023-11-06T03:17:27.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7137fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb71415caa0 0x7fb7140cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41376/0 (socket says 172.21.15.157:41376) 2023-11-06T03:17:27.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7137fe640 1 -- 172.21.15.157:0/4227662631 learned_addr learned my addr 172.21.15.157:0/4227662631 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:27.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb712ffd640 1 --2- 172.21.15.157:0/4227662631 >> v2:172.21.15.195:3300/0 conn(0x7fb71415ce80 0x7fb7140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 0x7fb7140b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565104982 0 0) 0x7fb7140d2920 con 0x7fb71415caa0 2023-11-06T03:17:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb700003530 con 0x7fb71415caa0 2023-11-06T03:17:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1993459619 0 0) 0x7fb700003530 con 0x7fb71415caa0 2023-11-06T03:17:27.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 msgr2=0x7fb7140b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 0x7fb7140b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 >> v2:172.21.15.195:3300/0 conn(0x7fb71415ce80 msgr2=0x7fb7140cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 --2- 172.21.15.157:0/4227662631 >> v2:172.21.15.195:3300/0 conn(0x7fb71415ce80 0x7fb7140cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140d1b60 con 0x7fb71415caa0 2023-11-06T03:17:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb704001a20 con 0x7fb71415caa0 2023-11-06T03:17:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 >> v1:172.21.15.157:6789/0 conn(0x7fb71415caa0 legacy=0x7fb7140cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 0x7fb700003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb700003f00 0x7fb7000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb700005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140d0520 con 0x7fb700004d10 2023-11-06T03:17:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140d18a0 con 0x7fb700003f00 2023-11-06T03:17:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70000b990 con 0x7fb7140d00b0 2023-11-06T03:17:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb700005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb712ffd640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb700003f00 0x7fb7000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 -- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 msgr2=0x7fb700003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140d00b0 0x7fb700003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 -- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb700003f00 msgr2=0x7fb7000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb700003f00 0x7fb7000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb713fff640 1 -- 172.21.15.157:0/4227662631 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700003530 con 0x7fb700004d10 2023-11-06T03:17:27.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.782+0000 7fb712ffd640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb700003f00 0x7fb7000047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb713fff640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb700005190 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009ae0 tx=0x7fb70c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c2703617a0c6ec server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0a5680 con 0x7fb700004d10 2023-11-06T03:17:27.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c0a57e0 con 0x7fb700004d10 2023-11-06T03:17:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb7127fc640 1 -- 172.21.15.157:0/4227662631 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0a5b90 con 0x7fb700004d10 2023-11-06T03:17:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- 172.21.15.157:0/4227662631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 msgr2=0x7fb700005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 --2- 172.21.15.157:0/4227662631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb700005190 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009ae0 tx=0x7fb70c01ca30 comp rx=0 tx=0).stop 2023-11-06T03:17:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb713fff640 1 -- 172.21.15.157:0/4227662631 reap_dead start 2023-11-06T03:17:27.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- 172.21.15.157:0/4227662631 shutdown_connections 2023-11-06T03:17:27.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- 172.21.15.157:0/4227662631 >> 172.21.15.157:0/4227662631 conn(0x7fb714070790 msgr2=0x7fb714070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- 172.21.15.157:0/4227662631 shutdown_connections 2023-11-06T03:17:27.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- 172.21.15.157:0/4227662631 wait complete. 2023-11-06T03:17:27.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 Processor -- start 2023-11-06T03:17:27.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.786+0000 7fb71a038640 1 -- start start 2023-11-06T03:17:27.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140c77e0 0x7fb7141e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7140c7c60 0x7fb7141e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71415d8f0 con 0x7fb700004d10 2023-11-06T03:17:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71415e8a0 con 0x7fb7140c7c60 2023-11-06T03:17:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71415d2c0 con 0x7fb7140c77e0 2023-11-06T03:17:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38376/0 (socket says 172.21.15.157:38376) 2023-11-06T03:17:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 -- 172.21.15.157:0/872384277 learned_addr learned my addr 172.21.15.157:0/872384277 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 -- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140c77e0 msgr2=0x7fb7141e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140c77e0 0x7fb7141e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 -- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7140c7c60 msgr2=0x7fb7141e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7140c7c60 0x7fb7141e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 -- 172.21.15.157:0/872384277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714151f60 con 0x7fb700004d10 2023-11-06T03:17:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb7137fe640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fb70400cb10 tx=0x7fb70400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400a780 con 0x7fb700004d10 2023-11-06T03:17:27.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400a8e0 con 0x7fb700004d10 2023-11-06T03:17:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb714152460 con 0x7fb700004d10 2023-11-06T03:17:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400ac90 con 0x7fb700004d10 2023-11-06T03:17:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7141602e0 con 0x7fb700004d10 2023-11-06T03:17:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb714072750 con 0x7fb700004d10 2023-11-06T03:17:27.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7040070d0 con 0x7fb700004d10 2023-11-06T03:17:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 0x7fb6f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb704082d30 con 0x7fb700004d10 2023-11-06T03:17:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb712ffd640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 0x7fb6f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:27.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.790+0000 7fb712ffd640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 0x7fb6f4067f20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb7141e7470 tx=0x7fb6fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:27.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.794+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb704050620 con 0x7fb700004d10 2023-11-06T03:17:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.938+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb71415be50 con 0x7fb700004d10 2023-11-06T03:17:27.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.938+0000 7fb70bfff640 1 -- 172.21.15.157:0/872384277 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v163) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb704018020 con 0x7fb700004d10 2023-11-06T03:17:27.937 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:17:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 msgr2=0x7fb6f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 0x7fb6f4067f20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb7141e7470 tx=0x7fb6fc008040 comp rx=0 tx=0).stop 2023-11-06T03:17:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 msgr2=0x7fb714151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:27.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fb70400cb10 tx=0x7fb70400a060 comp rx=0 tx=0).stop 2023-11-06T03:17:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 shutdown_connections 2023-11-06T03:17:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb6f4065a60 0x7fb6f4067f20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7140c7c60 0x7fb7141e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb7140c77e0 0x7fb7141e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 --2- 172.21.15.157:0/872384277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb700004d10 0x7fb714151a20 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 >> 172.21.15.157:0/872384277 conn(0x7fb714070790 msgr2=0x7fb71415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 shutdown_connections 2023-11-06T03:17:27.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:27.942+0000 7fb71a038640 1 -- 172.21.15.157:0/872384277 wait complete. 2023-11-06T03:17:27.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:17:28.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 Processor -- start 2023-11-06T03:17:28.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 -- start start 2023-11-06T03:17:28.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f541c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c15f580 con 0x7f541c1551e0 2023-11-06T03:17:28.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c15efc0 con 0x7f541c15e5e0 2023-11-06T03:17:28.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f5422237640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0704b0 con 0x7f541c0b3b20 2023-11-06T03:17:28.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f541c1551e0 0x7f541c06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41392/0 (socket says 172.21.15.157:41392) 2023-11-06T03:17:28.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541b7fe640 1 -- 172.21.15.157:0/2927198229 learned_addr learned my addr 172.21.15.157:0/2927198229 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:28.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853727159 0 0) 0x7f541c15f580 con 0x7f541c1551e0 2023-11-06T03:17:28.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f541c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5404003530 con 0x7f541c1551e0 2023-11-06T03:17:28.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336566502 0 0) 0x7f5404003530 con 0x7f541c1551e0 2023-11-06T03:17:28.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 msgr2=0x7f541c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f541c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 >> v1:172.21.15.195:6789/0 conn(0x7f541c15e5e0 legacy=0x7f541c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c15f2a0 con 0x7f541c1551e0 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f540c001680 con 0x7f541c1551e0 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f541c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 >> v1:172.21.15.157:6789/0 conn(0x7f541c1551e0 legacy=0x7f541c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54040039b0 0x7f5404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f54040047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5404004d10 0x7f5404005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0704b0 con 0x7f5404004d10 2023-11-06T03:17:28.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540400bb50 con 0x7f54040039b0 2023-11-06T03:17:28.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540400bd10 con 0x7f541c0b3b20 2023-11-06T03:17:28.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5404004d10 0x7f5404005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541affd640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f54040047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541b7fe640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54040039b0 0x7f5404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 -- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 msgr2=0x7f54040047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f54040047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 -- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54040039b0 msgr2=0x7f5404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54040039b0 0x7f5404003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541bfff640 1 -- 172.21.15.157:0/2927198229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404003530 con 0x7f5404004d10 2023-11-06T03:17:28.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541affd640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c0b3b20 0x7f54040047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:28.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.082+0000 7f541b7fe640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54040039b0 0x7f5404003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:28.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541bfff640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5404004d10 0x7f5404005190 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f54140073b0 tx=0x7f541401ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcf607c7053546c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140a5d80 con 0x7f5404004d10 2023-11-06T03:17:28.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c001d80 con 0x7f5404004d10 2023-11-06T03:17:28.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541a7fc640 1 -- 172.21.15.157:0/2927198229 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54140ad640 con 0x7f5404004d10 2023-11-06T03:17:28.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- 172.21.15.157:0/2927198229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5404004d10 msgr2=0x7f5404005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 --2- 172.21.15.157:0/2927198229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5404004d10 0x7f5404005190 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f54140073b0 tx=0x7f541401ce80 comp rx=0 tx=0).stop 2023-11-06T03:17:28.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541bfff640 1 -- 172.21.15.157:0/2927198229 reap_dead start 2023-11-06T03:17:28.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- 172.21.15.157:0/2927198229 shutdown_connections 2023-11-06T03:17:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- 172.21.15.157:0/2927198229 >> 172.21.15.157:0/2927198229 conn(0x7f541c065930 msgr2=0x7f541c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- 172.21.15.157:0/2927198229 shutdown_connections 2023-11-06T03:17:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- 172.21.15.157:0/2927198229 wait complete. 2023-11-06T03:17:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 Processor -- start 2023-11-06T03:17:28.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- start start 2023-11-06T03:17:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f541c06c300 0x7f541c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c06d150 0x7f541c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c06b930 con 0x7f541c150190 2023-11-06T03:17:28.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c1f34b0 con 0x7f541c06c300 2023-11-06T03:17:28.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f5422237640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c150570 con 0x7f541c06d150 2023-11-06T03:17:28.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38406/0 (socket says 172.21.15.157:38406) 2023-11-06T03:17:28.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.086+0000 7f541b7fe640 1 -- 172.21.15.157:0/3475049080 learned_addr learned my addr 172.21.15.157:0/3475049080 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:28.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541affd640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f541c06c300 0x7f541c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541bfff640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c06d150 0x7f541c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 -- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c06d150 msgr2=0x7f541c159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c06d150 0x7f541c159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 -- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f541c06c300 msgr2=0x7f541c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f541c06c300 0x7f541c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 -- 172.21.15.157:0/3475049080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c1f2ef0 con 0x7f541c150190 2023-11-06T03:17:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541b7fe640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f540c001210 tx=0x7f540c00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c010e10 con 0x7f541c150190 2023-11-06T03:17:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c0143a0 con 0x7f541c150190 2023-11-06T03:17:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c1f31d0 con 0x7f541c150190 2023-11-06T03:17:28.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c014710 con 0x7f541c150190 2023-11-06T03:17:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c15c170 con 0x7f541c150190 2023-11-06T03:17:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c072750 con 0x7f541c150190 2023-11-06T03:17:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f540c014870 con 0x7f541c150190 2023-11-06T03:17:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 0x7f5408067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f540c085e20 con 0x7f541c150190 2023-11-06T03:17:28.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541affd640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 0x7f5408067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.090+0000 7f541affd640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 0x7f5408067f20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f541c0ae1f0 tx=0x7f5410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.094+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f540c0537c0 con 0x7f541c150190 2023-11-06T03:17:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f541c071730 con 0x7f541c150190 2023-11-06T03:17:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5403fff640 1 -- 172.21.15.157:0/3475049080 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v163) v1 ==== 100+0+11 (secure 0 0 0) 0x7f540c01b0a0 con 0x7f541c150190 2023-11-06T03:17:28.244 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:17:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 msgr2=0x7f5408067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 0x7f5408067f20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f541c0ae1f0 tx=0x7f5410009040 comp rx=0 tx=0).stop 2023-11-06T03:17:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 msgr2=0x7f541c06bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f540c001210 tx=0x7f540c00e4d0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 shutdown_connections 2023-11-06T03:17:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5408065a60 0x7f5408067f20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f541c06d150 0x7f541c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f541c06c300 0x7f541c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 --2- 172.21.15.157:0/3475049080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f541c150190 0x7f541c06bdc0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 >> 172.21.15.157:0/3475049080 conn(0x7f541c065930 msgr2=0x7f541c153260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 shutdown_connections 2023-11-06T03:17:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.246+0000 7f5422237640 1 -- 172.21.15.157:0/3475049080 wait complete. 2023-11-06T03:17:28.263 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:28.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 Processor -- start 2023-11-06T03:17:28.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 -- start start 2023-11-06T03:17:28.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f23b40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40d2920 con 0x7f23b40d00b0 2023-11-06T03:17:28.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40d18a0 con 0x7f23b415caa0 2023-11-06T03:17:28.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b960e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40d0520 con 0x7f23b415ce80 2023-11-06T03:17:28.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f23b40d00b0 0x7f23b40b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:41404/0 (socket says 172.21.15.157:41404) 2023-11-06T03:17:28.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b37fe640 1 -- 172.21.15.157:0/2661356898 learned_addr learned my addr 172.21.15.157:0/2661356898 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:28.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f23b40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673693809 0 0) 0x7f23b40d2920 con 0x7f23b40d00b0 2023-11-06T03:17:28.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c003530 con 0x7f23b40d00b0 2023-11-06T03:17:28.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1058019910 0 0) 0x7f239c003530 con 0x7f23b40d00b0 2023-11-06T03:17:28.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 msgr2=0x7f23b40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f23b40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 >> v1:172.21.15.195:6789/0 conn(0x7f23b415caa0 legacy=0x7f23b40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40d1b60 con 0x7f23b40d00b0 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f23ac001a00 con 0x7f23b40d00b0 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 >> v1:172.21.15.157:6789/0 conn(0x7f23b40d00b0 legacy=0x7f23b40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f239c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f239c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f239c004d10 0x7f239c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40d0520 con 0x7f239c003f30 2023-11-06T03:17:28.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00ba70 con 0x7f239c004d10 2023-11-06T03:17:28.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00bc30 con 0x7f23b415ce80 2023-11-06T03:17:28.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f239c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b2ffd640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f239c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b37fe640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f239c004d10 0x7f239c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 -- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 msgr2=0x7f239c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b415ce80 0x7f239c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 -- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f239c004d10 msgr2=0x7f239c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f239c004d10 0x7f239c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.382+0000 7f23b27fc640 1 -- 172.21.15.157:0/2661356898 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c003530 con 0x7f239c003f30 2023-11-06T03:17:28.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f239c0047d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ae0 tx=0x7f23a401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d77270cfdab5f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40a5530 con 0x7f239c003f30 2023-11-06T03:17:28.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40a5690 con 0x7f239c003f30 2023-11-06T03:17:28.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b1ffb640 1 -- 172.21.15.157:0/2661356898 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40a5a40 con 0x7f239c003f30 2023-11-06T03:17:28.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- 172.21.15.157:0/2661356898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 msgr2=0x7f239c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 --2- 172.21.15.157:0/2661356898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f239c0047d0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ae0 tx=0x7f23a401ca50 comp rx=0 tx=0).stop 2023-11-06T03:17:28.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b37fe640 1 -- 172.21.15.157:0/2661356898 reap_dead start 2023-11-06T03:17:28.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- 172.21.15.157:0/2661356898 shutdown_connections 2023-11-06T03:17:28.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- 172.21.15.157:0/2661356898 >> 172.21.15.157:0/2661356898 conn(0x7f23b4070790 msgr2=0x7f23b4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- 172.21.15.157:0/2661356898 shutdown_connections 2023-11-06T03:17:28.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- 172.21.15.157:0/2661356898 wait complete. 2023-11-06T03:17:28.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 Processor -- start 2023-11-06T03:17:28.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.386+0000 7f23b960e640 1 -- start start 2023-11-06T03:17:28.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b40c9cf0 0x7f23b41f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23b41f1960 0x7f23b41f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40cb460 con 0x7f239c003f30 2023-11-06T03:17:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40cb740 con 0x7f23b41f1960 2023-11-06T03:17:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40cba20 con 0x7f23b40c9cf0 2023-11-06T03:17:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38414/0 (socket says 172.21.15.157:38414) 2023-11-06T03:17:28.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 -- 172.21.15.157:0/2120871714 learned_addr learned my addr 172.21.15.157:0/2120871714 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:28.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b40c9cf0 0x7f23b41f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 -- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b40c9cf0 msgr2=0x7f23b41f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b40c9cf0 0x7f23b41f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 -- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23b41f1960 msgr2=0x7f23b41f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23b41f1960 0x7f23b41f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 -- 172.21.15.157:0/2120871714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40cbd00 con 0x7f239c003f30 2023-11-06T03:17:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b2ffd640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f23a800f7f0 tx=0x7f23a800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80050d0 con 0x7f239c003f30 2023-11-06T03:17:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8005230 con 0x7f239c003f30 2023-11-06T03:17:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b40cbfe0 con 0x7f239c003f30 2023-11-06T03:17:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800d730 con 0x7f239c003f30 2023-11-06T03:17:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b41f4b50 con 0x7f239c003f30 2023-11-06T03:17:28.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b40cb880 con 0x7f239c003f30 2023-11-06T03:17:28.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23a800d890 con 0x7f239c003f30 2023-11-06T03:17:28.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 0x7f23a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23a80832c0 con 0x7f239c003f30 2023-11-06T03:17:28.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.390+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 0x7f23a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.394+0000 7f23b27fc640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 0x7f23a0067f20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f23a40af000 tx=0x7f23a40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.394+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f23b40cb880 con 0x7f239c003f30 2023-11-06T03:17:28.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.578+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b40cb880 con 0x7f239c003f30 2023-11-06T03:17:28.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f239b7fe640 1 -- 172.21.15.157:0/2120871714 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f23b40cb880 con 0x7f239c003f30 2023-11-06T03:17:28.578 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:28.579 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:17:28.579 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:17:28.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 msgr2=0x7f23a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 0x7f23a0067f20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f23a40af000 tx=0x7f23a40046d0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 msgr2=0x7f23b40cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f23a800f7f0 tx=0x7f23a800fcb0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 shutdown_connections 2023-11-06T03:17:28.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f23a0065a60 0x7f23a0067f20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23b41f1960 0x7f23b41f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23b40c9cf0 0x7f23b41f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 --2- 172.21.15.157:0/2120871714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c003f30 0x7f23b40cabe0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 >> 172.21.15.157:0/2120871714 conn(0x7f23b4070790 msgr2=0x7f23b40b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 shutdown_connections 2023-11-06T03:17:28.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.582+0000 7f23b960e640 1 -- 172.21.15.157:0/2120871714 wait complete. 2023-11-06T03:17:28.596 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:28.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 Processor -- start 2023-11-06T03:17:28.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 -- start start 2023-11-06T03:17:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d801551e0 0x7f8d8006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 0x7f8d8015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8015f580 con 0x7f8d8015e5e0 2023-11-06T03:17:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8015efc0 con 0x7f8d801551e0 2023-11-06T03:17:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d87ae0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8015f2a0 con 0x7f8d800b3b20 2023-11-06T03:17:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d801551e0 0x7f8d8006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8d801551e0 0x7f8d8006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55768/0 (socket says 172.21.15.157:55768) 2023-11-06T03:17:28.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 -- 172.21.15.157:0/1993561538 learned_addr learned my addr 172.21.15.157:0/1993561538 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:28.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8607d640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 0x7f8d8015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 -- 172.21.15.157:0/1993561538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 msgr2=0x7f8d8015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 0x7f8d8015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 -- 172.21.15.157:0/1993561538 >> v1:172.21.15.195:6789/0 conn(0x7f8d8015e5e0 legacy=0x7f8d800b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8587c640 1 -- 172.21.15.157:0/1993561538 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8006f5a0 con 0x7f8d801551e0 2023-11-06T03:17:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.718+0000 7f8d8607d640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 0x7f8d8015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d8587c640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d801551e0 0x7f8d8006fda0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009e50 tx=0x7f8d7001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4278b49afded19 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d8487a640 1 -- 172.21.15.157:0/1993561538 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a57b0 con 0x7f8d801551e0 2023-11-06T03:17:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d8487a640 1 -- 172.21.15.157:0/1993561538 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700a5910 con 0x7f8d801551e0 2023-11-06T03:17:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d8487a640 1 -- 172.21.15.157:0/1993561538 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5ca0 con 0x7f8d801551e0 2023-11-06T03:17:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 -- 172.21.15.157:0/1993561538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d801551e0 msgr2=0x7f8d8006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d801551e0 0x7f8d8006fda0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009e50 tx=0x7f8d7001c910 comp rx=0 tx=0).stop 2023-11-06T03:17:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 -- 172.21.15.157:0/1993561538 shutdown_connections 2023-11-06T03:17:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d800b3b20 0x7f8d8015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/1993561538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d801551e0 0x7f8d8006fda0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 -- 172.21.15.157:0/1993561538 >> 172.21.15.157:0/1993561538 conn(0x7f8d80065930 msgr2=0x7f8d80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 -- 172.21.15.157:0/1993561538 shutdown_connections 2023-11-06T03:17:28.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 -- 172.21.15.157:0/1993561538 wait complete. 2023-11-06T03:17:28.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.722+0000 7f8d87ae0640 1 Processor -- start 2023-11-06T03:17:28.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- start start 2023-11-06T03:17:28.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 0x7f8d80141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d8015e5e0 0x7f8d8015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80141690 con 0x7f8d800b3b20 2023-11-06T03:17:28.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80141970 con 0x7f8d8015e5e0 2023-11-06T03:17:28.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80141c50 con 0x7f8d801551e0 2023-11-06T03:17:28.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55784/0 (socket says 172.21.15.157:55784) 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 -- 172.21.15.157:0/3559757956 learned_addr learned my addr 172.21.15.157:0/3559757956 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8507b640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 0x7f8d80141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 -- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 msgr2=0x7f8d80141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8607d640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d8015e5e0 0x7f8d8015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 0x7f8d80141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 -- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d8015e5e0 msgr2=0x7f8d8015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d8015e5e0 0x7f8d8015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 -- 172.21.15.157:0/3559757956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70009b30 con 0x7f8d800b3b20 2023-11-06T03:17:28.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8507b640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 0x7f8d80141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:28.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d8587c640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001afc0 tx=0x7f8d700196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b8880 con 0x7f8d800b3b20 2023-11-06T03:17:28.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700b89e0 con 0x7f8d800b3b20 2023-11-06T03:17:28.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8014ce70 con 0x7f8d800b3b20 2023-11-06T03:17:28.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b8d90 con 0x7f8d800b3b20 2023-11-06T03:17:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d80142470 con 0x7f8d800b3b20 2023-11-06T03:17:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80072750 con 0x7f8d800b3b20 2023-11-06T03:17:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d700b6440 con 0x7f8d800b3b20 2023-11-06T03:17:28.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 0x7f8d6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:28.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.726+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d7011e910 con 0x7f8d800b3b20 2023-11-06T03:17:28.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.730+0000 7f8d8507b640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 0x7f8d6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:28.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.730+0000 7f8d8507b640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 0x7f8d6c067f20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8d8014e9e0 tx=0x7f8d74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:28.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.734+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8d700ec230 con 0x7f8d800b3b20 2023-11-06T03:17:28.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.886+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d80071730 con 0x7f8d800b3b20 2023-11-06T03:17:28.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d7e7fc640 1 -- 172.21.15.157:0/3559757956 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17873 (secure 0 0 0) 0x7f8d700ef850 con 0x7f8d800b3b20 2023-11-06T03:17:28.888 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:28.888 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":163,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:44.441779+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:28.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 msgr2=0x7f8d6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 0x7f8d6c067f20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8d8014e9e0 tx=0x7f8d74008040 comp rx=0 tx=0).stop 2023-11-06T03:17:28.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 msgr2=0x7f8d80149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:28.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f8d7001afc0 tx=0x7f8d700196b0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 shutdown_connections 2023-11-06T03:17:28.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8d6c065a60 0x7f8d6c067f20 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8d8015e5e0 0x7f8d8015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8d801551e0 0x7f8d80141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 --2- 172.21.15.157:0/3559757956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d800b3b20 0x7f8d80149850 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:28.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 >> 172.21.15.157:0/3559757956 conn(0x7f8d80065930 msgr2=0x7f8d80151600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:28.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.890+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 shutdown_connections 2023-11-06T03:17:28.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:28.894+0000 7f8d87ae0640 1 -- 172.21.15.157:0/3559757956 wait complete. 2023-11-06T03:17:28.907 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:17:28.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:29.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 Processor -- start 2023-11-06T03:17:29.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 -- start start 2023-11-06T03:17:29.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f72b40b19b0 0x7f72b40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b40ce350 con 0x7f72b4159d70 2023-11-06T03:17:29.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b406bff0 con 0x7f72b4159990 2023-11-06T03:17:29.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72bba91640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b406c2d0 con 0x7f72b40b19b0 2023-11-06T03:17:29.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72ba02e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f72b4159d70 0x7f72b40702f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59908/0 (socket says 172.21.15.157:59908) 2023-11-06T03:17:29.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72ba02e640 1 -- 172.21.15.157:0/1404678970 learned_addr learned my addr 172.21.15.157:0/1404678970 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:29.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b902c640 1 --2- 172.21.15.157:0/1404678970 >> v2:172.21.15.195:3300/0 conn(0x7f72b40b19b0 0x7f72b40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031939652 0 0) 0x7f72b40ce350 con 0x7f72b4159d70 2023-11-06T03:17:29.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a0003530 con 0x7f72b4159d70 2023-11-06T03:17:29.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856710575 0 0) 0x7f72a0003530 con 0x7f72b4159d70 2023-11-06T03:17:29.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 >> v2:172.21.15.195:3300/0 conn(0x7f72b40b19b0 msgr2=0x7f72b40cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 --2- 172.21.15.157:0/1404678970 >> v2:172.21.15.195:3300/0 conn(0x7f72b40b19b0 0x7f72b40cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 >> v1:172.21.15.195:6789/0 conn(0x7f72b4159990 legacy=0x7f72b40b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b406c5b0 con 0x7f72b4159d70 2023-11-06T03:17:29.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f72ac001a00 con 0x7f72b4159d70 2023-11-06T03:17:29.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 >> v1:172.21.15.157:6789/0 conn(0x7f72b4159d70 legacy=0x7f72b40702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 0x7f72a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a00047d0 0x7f72a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a0005190 0x7f72a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b406c2d0 con 0x7f72a0003980 2023-11-06T03:17:29.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c130 con 0x7f72a0005190 2023-11-06T03:17:29.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c2f0 con 0x7f72a00047d0 2023-11-06T03:17:29.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 0x7f72a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b902c640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a00047d0 0x7f72a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 -- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a00047d0 msgr2=0x7f72a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a00047d0 0x7f72a0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72ba02e640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a0005190 0x7f72a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 -- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a0005190 msgr2=0x7f72a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72a0005190 0x7f72a000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72ba02e640 1 -- 172.21.15.157:0/1404678970 reap_dead start 2023-11-06T03:17:29.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.026+0000 7f72b982d640 1 -- 172.21.15.157:0/1404678970 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a0003530 con 0x7f72a0003980 2023-11-06T03:17:29.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72b982d640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 0x7f72a0003da0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001780 tx=0x7f72b001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec520528c85cfbbf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00052d0 con 0x7f72a0003980 2023-11-06T03:17:29.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b00a7050 con 0x7f72a0003980 2023-11-06T03:17:29.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72b882b640 1 -- 172.21.15.157:0/1404678970 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0003600 con 0x7f72a0003980 2023-11-06T03:17:29.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 msgr2=0x7f72a0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 0x7f72a0003da0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001780 tx=0x7f72b001c380 comp rx=0 tx=0).stop 2023-11-06T03:17:29.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- 172.21.15.157:0/1404678970 shutdown_connections 2023-11-06T03:17:29.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 --2- 172.21.15.157:0/1404678970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a0003980 0x7f72a0003da0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- 172.21.15.157:0/1404678970 >> 172.21.15.157:0/1404678970 conn(0x7f72b40cc2a0 msgr2=0x7f72b40cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- 172.21.15.157:0/1404678970 shutdown_connections 2023-11-06T03:17:29.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- 172.21.15.157:0/1404678970 wait complete. 2023-11-06T03:17:29.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 Processor -- start 2023-11-06T03:17:29.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.030+0000 7f72bba91640 1 -- start start 2023-11-06T03:17:29.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a0003980 0x7f72b4156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 0x7f72b4156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40d0620 con 0x7f72b40cfc60 2023-11-06T03:17:29.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40c46a0 con 0x7f72b41567d0 2023-11-06T03:17:29.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b3460 con 0x7f72a0003980 2023-11-06T03:17:29.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55796/0 (socket says 172.21.15.157:55796) 2023-11-06T03:17:29.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 -- 172.21.15.157:0/325095088 learned_addr learned my addr 172.21.15.157:0/325095088 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:29.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72b902c640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 0x7f72b4156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72b982d640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a0003980 0x7f72b4156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 -- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a0003980 msgr2=0x7f72b4156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a0003980 0x7f72b4156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 -- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 msgr2=0x7f72b4156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 0x7f72b4156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 -- 172.21.15.157:0/325095088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40b3180 con 0x7f72b40cfc60 2023-11-06T03:17:29.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72b902c640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 0x7f72b4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:29.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72ba02e640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001590 tx=0x7f72ac00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00b860 con 0x7f72b40cfc60 2023-11-06T03:17:29.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00ba00 con 0x7f72b40cfc60 2023-11-06T03:17:29.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b40c4940 con 0x7f72b40cfc60 2023-11-06T03:17:29.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00bdb0 con 0x7f72b40cfc60 2023-11-06T03:17:29.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b4156570 con 0x7f72b40cfc60 2023-11-06T03:17:29.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b40c4500 con 0x7f72b40cfc60 2023-11-06T03:17:29.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f72ac007070 con 0x7f72b40cfc60 2023-11-06T03:17:29.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 0x7f7298067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72ac0835f0 con 0x7f72b40cfc60 2023-11-06T03:17:29.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.034+0000 7f72b982d640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 0x7f7298067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.038+0000 7f72b982d640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 0x7f7298067f50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001750 tx=0x7f72b00011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.038+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f72b40c4500 con 0x7f72b40cfc60 2023-11-06T03:17:29.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.226+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72b40c4500 con 0x7f72b40cfc60 2023-11-06T03:17:29.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.226+0000 7f72aa7fc640 1 -- 172.21.15.157:0/325095088 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f72b40c4500 con 0x7f72b40cfc60 2023-11-06T03:17:29.226 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:29.226 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:17:29.226 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:17:29.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 msgr2=0x7f7298067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 0x7f7298067f50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001750 tx=0x7f72b00011a0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 msgr2=0x7f72b40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001590 tx=0x7f72ac00df60 comp rx=0 tx=0).stop 2023-11-06T03:17:29.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 shutdown_connections 2023-11-06T03:17:29.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7298065a90 0x7f7298067f50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cfc60 0x7f72b40d00e0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72b41567d0 0x7f72b4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 --2- 172.21.15.157:0/325095088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72a0003980 0x7f72b4156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 >> 172.21.15.157:0/325095088 conn(0x7f72b40cc2a0 msgr2=0x7f72b4067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 shutdown_connections 2023-11-06T03:17:29.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.230+0000 7f72bba91640 1 -- 172.21.15.157:0/325095088 wait complete. 2023-11-06T03:17:29.244 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:29.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.366+0000 7f1fa61af640 1 Processor -- start 2023-11-06T03:17:29.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.366+0000 7f1fa61af640 1 -- start start 2023-11-06T03:17:29.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1fa61af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1fa0160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1fa61af640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa01612f0 con 0x7f1fa0160490 2023-11-06T03:17:29.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1fa61af640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa0161780 con 0x7f1fa015e800 2023-11-06T03:17:29.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1fa61af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00cd860 con 0x7f1fa0160950 2023-11-06T03:17:29.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1fa0160490 0x7f1fa00cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59916/0 (socket says 172.21.15.157:59916) 2023-11-06T03:17:29.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9f7fe640 1 -- 172.21.15.157:0/2702263812 learned_addr learned my addr 172.21.15.157:0/2702263812 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:29.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1fa0160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127068748 0 0) 0x7f1fa01612f0 con 0x7f1fa0160490 2023-11-06T03:17:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f88003530 con 0x7f1fa0160490 2023-11-06T03:17:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472690170 0 0) 0x7f1fa0161780 con 0x7f1fa015e800 2023-11-06T03:17:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa01612f0 con 0x7f1fa015e800 2023-11-06T03:17:29.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166957020 0 0) 0x7f1f88003530 con 0x7f1fa0160490 2023-11-06T03:17:29.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 msgr2=0x7f1fa0160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1fa0160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 >> v1:172.21.15.195:6789/0 conn(0x7f1fa015e800 legacy=0x7f1fa015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00cdb40 con 0x7f1fa0160490 2023-11-06T03:17:29.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1f94001680 con 0x7f1fa0160490 2023-11-06T03:17:29.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 >> v1:172.21.15.157:6789/0 conn(0x7f1fa0160490 legacy=0x7f1fa00cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1f88003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f880040f0 0x7f1f880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f88004d10 0x7f1f88005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00cd860 con 0x7f1f88004d10 2023-11-06T03:17:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8800bad0 con 0x7f1f880040f0 2023-11-06T03:17:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8800bc90 con 0x7f1fa0160950 2023-11-06T03:17:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f88004d10 0x7f1f88005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f880040f0 0x7f1f880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9f7fe640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1f88003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 -- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 msgr2=0x7f1f88003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa0160950 0x7f1f88003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 -- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f880040f0 msgr2=0x7f1f880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f880040f0 0x7f1f880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.370+0000 7f1f9ffff640 1 -- 172.21.15.157:0/2702263812 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88003530 con 0x7f1f88004d10 2023-11-06T03:17:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9ffff640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f88004d10 0x7f1f88005190 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1f900018d0 tx=0x7f1f9001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b039ad09f43ae4d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900052d0 con 0x7f1f88004d10 2023-11-06T03:17:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f90005450 con 0x7f1f88004d10 2023-11-06T03:17:29.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9e7fc640 1 -- 172.21.15.157:0/2702263812 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90003840 con 0x7f1f88004d10 2023-11-06T03:17:29.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- 172.21.15.157:0/2702263812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f88004d10 msgr2=0x7f1f88005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2702263812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f88004d10 0x7f1f88005190 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1f900018d0 tx=0x7f1f9001c4a0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9ffff640 1 -- 172.21.15.157:0/2702263812 reap_dead start 2023-11-06T03:17:29.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- 172.21.15.157:0/2702263812 shutdown_connections 2023-11-06T03:17:29.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- 172.21.15.157:0/2702263812 >> 172.21.15.157:0/2702263812 conn(0x7f1fa0070790 msgr2=0x7f1fa0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- 172.21.15.157:0/2702263812 shutdown_connections 2023-11-06T03:17:29.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- 172.21.15.157:0/2702263812 wait complete. 2023-11-06T03:17:29.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 Processor -- start 2023-11-06T03:17:29.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- start start 2023-11-06T03:17:29.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 0x7f1fa00b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa00d2dc0 0x7f1fa00c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00ca6c0 con 0x7f1fa00c58b0 2023-11-06T03:17:29.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00c6c60 con 0x7f1fa00c6710 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1fa61af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00ca910 con 0x7f1fa00d2dc0 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55816/0 (socket says 172.21.15.157:55816) 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.374+0000 7f1f9f7fe640 1 -- 172.21.15.157:0/2122055002 learned_addr learned my addr 172.21.15.157:0/2122055002 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9ffff640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa00d2dc0 0x7f1fa00c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 0x7f1fa00b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 -- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa00d2dc0 msgr2=0x7f1fa00c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa00d2dc0 0x7f1fa00c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 -- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 msgr2=0x7f1fa00b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 0x7f1fa00b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 -- 172.21.15.157:0/2122055002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00cabf0 con 0x7f1fa00c58b0 2023-11-06T03:17:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 0x7f1fa00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9f7fe640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1f94011b80 tx=0x7f1f9400f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94011e70 con 0x7f1fa00c58b0 2023-11-06T03:17:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9400fe60 con 0x7f1fa00c58b0 2023-11-06T03:17:29.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa00caed0 con 0x7f1fa00c58b0 2023-11-06T03:17:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94015630 con 0x7f1fa00c58b0 2023-11-06T03:17:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa00cc990 con 0x7f1fa00c58b0 2023-11-06T03:17:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa00b6910 con 0x7f1fa00c58b0 2023-11-06T03:17:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f94015790 con 0x7f1fa00c58b0 2023-11-06T03:17:29.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 0x7f1f8c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f940867b0 con 0x7f1fa00c58b0 2023-11-06T03:17:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 0x7f1f8c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.378+0000 7f1f9effd640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 0x7f1f8c067f50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b7cb0 tx=0x7f1f9800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.382+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f94054010 con 0x7f1fa00c58b0 2023-11-06T03:17:29.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.570+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa00b3970 con 0x7f1fa00c58b0 2023-11-06T03:17:29.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.570+0000 7f1f87fff640 1 -- 172.21.15.157:0/2122055002 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f1fa00b3970 con 0x7f1fa00c58b0 2023-11-06T03:17:29.568 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:29.569 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":117,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:16:43.438847+0000","modified":"2023-11-06T03:16:45.454769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10392},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10392":{"gid":10392,"name":"d","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6837/3466838470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":3466838470},{"type":"v1","addr":"172.21.15.195:6837","nonce":3466838470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-11-06T03:17:29.569 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 117 2023-11-06T03:17:29.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 msgr2=0x7f1f8c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 0x7f1f8c067f50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b7cb0 tx=0x7f1f9800b040 comp rx=0 tx=0).stop 2023-11-06T03:17:29.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 msgr2=0x7f1fa00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1f94011b80 tx=0x7f1f9400f540 comp rx=0 tx=0).stop 2023-11-06T03:17:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 shutdown_connections 2023-11-06T03:17:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f8c065a90 0x7f1f8c067f50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa00d2dc0 0x7f1fa00c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa00c6710 0x7f1fa00b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 --2- 172.21.15.157:0/2122055002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa00c58b0 0x7f1fa00b2ef0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 >> 172.21.15.157:0/2122055002 conn(0x7f1fa0070790 msgr2=0x7f1fa00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 shutdown_connections 2023-11-06T03:17:29.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.574+0000 7f1fa61af640 1 -- 172.21.15.157:0/2122055002 wait complete. 2023-11-06T03:17:29.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:29.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.706+0000 7fcd57fbd640 1 Processor -- start 2023-11-06T03:17:29.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.706+0000 7fcd57fbd640 1 -- start start 2023-11-06T03:17:29.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcd5015caa0 0x7fcd500cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015ce80 0x7fcd500cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd500ce1a0 0x7fcd500d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500d2920 con 0x7fcd500ce1a0 2023-11-06T03:17:29.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500d18a0 con 0x7fcd5015caa0 2023-11-06T03:17:29.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd57fbd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500d1b60 con 0x7fcd5015ce80 2023-11-06T03:17:29.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd500ce1a0 0x7fcd500d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd500ce1a0 0x7fcd500d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55822/0 (socket says 172.21.15.157:55822) 2023-11-06T03:17:29.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 -- 172.21.15.157:0/1943568161 learned_addr learned my addr 172.21.15.157:0/1943568161 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:29.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd55558640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015ce80 0x7fcd500cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd55d59640 1 --2- 172.21.15.157:0/1943568161 >> v2:172.21.15.195:3300/0 conn(0x7fcd5015caa0 0x7fcd500cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 -- 172.21.15.157:0/1943568161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015ce80 msgr2=0x7fcd500cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015ce80 0x7fcd500cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 -- 172.21.15.157:0/1943568161 >> v2:172.21.15.195:3300/0 conn(0x7fcd5015caa0 msgr2=0x7fcd500cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:29.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 --2- 172.21.15.157:0/1943568161 >> v2:172.21.15.195:3300/0 conn(0x7fcd5015caa0 0x7fcd500cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 -- 172.21.15.157:0/1943568161 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500d1e40 con 0x7fcd500ce1a0 2023-11-06T03:17:29.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd5655a640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ce1a0 0x7fcd500d00b0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b4f80 tx=0x7fcd4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3d85c5cf406040 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd54d57640 1 -- 172.21.15.157:0/1943568161 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480a5820 con 0x7fcd500ce1a0 2023-11-06T03:17:29.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd54d57640 1 -- 172.21.15.157:0/1943568161 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd480a5980 con 0x7fcd500ce1a0 2023-11-06T03:17:29.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.710+0000 7fcd54d57640 1 -- 172.21.15.157:0/1943568161 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480a5d10 con 0x7fcd500ce1a0 2023-11-06T03:17:29.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/1943568161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ce1a0 msgr2=0x7fcd500d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ce1a0 0x7fcd500d00b0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fcd500b4f80 tx=0x7fcd4801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/1943568161 shutdown_connections 2023-11-06T03:17:29.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ce1a0 0x7fcd500d00b0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/1943568161 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015ce80 0x7fcd500cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/1943568161 >> v2:172.21.15.195:3300/0 conn(0x7fcd5015caa0 0x7fcd500cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/1943568161 >> 172.21.15.157:0/1943568161 conn(0x7fcd50070790 msgr2=0x7fcd50070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/1943568161 shutdown_connections 2023-11-06T03:17:29.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/1943568161 wait complete. 2023-11-06T03:17:29.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 Processor -- start 2023-11-06T03:17:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- start start 2023-11-06T03:17:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500ce1a0 0x7fcd501580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015caa0 0x7fcd5014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501585f0 con 0x7fcd5015ce80 2023-11-06T03:17:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015f960 con 0x7fcd5015caa0 2023-11-06T03:17:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c6e90 con 0x7fcd500ce1a0 2023-11-06T03:17:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55824/0 (socket says 172.21.15.157:55824) 2023-11-06T03:17:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 -- 172.21.15.157:0/4271780002 learned_addr learned my addr 172.21.15.157:0/4271780002 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd55d59640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500ce1a0 0x7fcd501580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd55558640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015caa0 0x7fcd5014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 -- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500ce1a0 msgr2=0x7fcd501580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500ce1a0 0x7fcd501580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 -- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015caa0 msgr2=0x7fcd5014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015caa0 0x7fcd5014d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 -- 172.21.15.157:0/4271780002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd48009b30 con 0x7fcd5015ce80 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd5655a640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af040 tx=0x7fcd48019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480b8670 con 0x7fcd5015ce80 2023-11-06T03:17:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd480b8810 con 0x7fcd5015ce80 2023-11-06T03:17:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5014dbe0 con 0x7fcd5015ce80 2023-11-06T03:17:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480b8bc0 con 0x7fcd5015ce80 2023-11-06T03:17:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.714+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500c6bb0 con 0x7fcd5015ce80 2023-11-06T03:17:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5015fb10 con 0x7fcd5015ce80 2023-11-06T03:17:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcd480b8d20 con 0x7fcd5015ce80 2023-11-06T03:17:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd467fc640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 0x7fcd34067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd4811e730 con 0x7fcd5015ce80 2023-11-06T03:17:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd55d59640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 0x7fcd34067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.718+0000 7fcd55d59640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 0x7fcd34067f50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcd500c6120 tx=0x7fcd40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:29.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.722+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcd5015fb10 con 0x7fcd5015ce80 2023-11-06T03:17:29.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.878+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd5015fb10 con 0x7fcd5015ce80 2023-11-06T03:17:29.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.878+0000 7fcd467fc640 1 -- 172.21.15.157:0/4271780002 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17873 (secure 0 0 0) 0x7fcd5015fb10 con 0x7fcd5015ce80 2023-11-06T03:17:29.877 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:29.877 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":163,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:16:44.441779+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:16:42.003695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-06T03:16:42.405981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 msgr2=0x7fcd34067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 0x7fcd34067f50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcd500c6120 tx=0x7fcd40008040 comp rx=0 tx=0).stop 2023-11-06T03:17:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 msgr2=0x7fcd50155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fcd480af040 tx=0x7fcd48019740 comp rx=0 tx=0).stop 2023-11-06T03:17:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 shutdown_connections 2023-11-06T03:17:29.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a90 0x7fcd34067f50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015ce80 0x7fcd50155610 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015caa0 0x7fcd5014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 --2- 172.21.15.157:0/4271780002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd500ce1a0 0x7fcd501580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:29.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 >> 172.21.15.157:0/4271780002 conn(0x7fcd50070790 msgr2=0x7fcd500cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:29.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 shutdown_connections 2023-11-06T03:17:29.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:29.882+0000 7fcd57fbd640 1 -- 172.21.15.157:0/4271780002 wait complete. 2023-11-06T03:17:29.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:17:30.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 Processor -- start 2023-11-06T03:17:30.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 -- start start 2023-11-06T03:17:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0adae0 0x7f91fc0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0adf40 0x7f91fc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc06c490 con 0x7f91fc156a70 2023-11-06T03:17:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc06b950 con 0x7f91fc0adf40 2023-11-06T03:17:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f920149a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc06fb80 con 0x7f91fc0adae0 2023-11-06T03:17:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f91fb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0adf40 0x7f91fc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f91fb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91fc0adf40 0x7f91fc0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55838/0 (socket says 172.21.15.157:55838) 2023-11-06T03:17:30.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f91fb7fe640 1 -- 172.21.15.157:0/3805405665 learned_addr learned my addr 172.21.15.157:0/3805405665 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.018+0000 7f91f27fc640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0adae0 0x7f91fc0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fb7fe640 1 -- 172.21.15.157:0/3805405665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0adae0 msgr2=0x7f91fc0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0adae0 0x7f91fc0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fb7fe640 1 -- 172.21.15.157:0/3805405665 >> v1:172.21.15.195:6789/0 conn(0x7f91fc156a70 legacy=0x7f91fc06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fb7fe640 1 -- 172.21.15.157:0/3805405665 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc06bb60 con 0x7f91fc0adf40 2023-11-06T03:17:30.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adf40 0x7f91fc0b2160 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d150 tx=0x7f91f401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb4555e49d066e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fa7fc640 1 -- 172.21.15.157:0/3805405665 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40a7c30 con 0x7f91fc0adf40 2023-11-06T03:17:30.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fa7fc640 1 -- 172.21.15.157:0/3805405665 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f40ab070 con 0x7f91fc0adf40 2023-11-06T03:17:30.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f91fa7fc640 1 -- 172.21.15.157:0/3805405665 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40b1400 con 0x7f91fc0adf40 2023-11-06T03:17:30.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 -- 172.21.15.157:0/3805405665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adf40 msgr2=0x7f91fc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adf40 0x7f91fc0b2160 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d150 tx=0x7f91f401ebe0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 -- 172.21.15.157:0/3805405665 shutdown_connections 2023-11-06T03:17:30.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adf40 0x7f91fc0b2160 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 --2- 172.21.15.157:0/3805405665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc0adae0 0x7f91fc0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 -- 172.21.15.157:0/3805405665 >> 172.21.15.157:0/3805405665 conn(0x7f91fc065930 msgr2=0x7f91fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:30.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 -- 172.21.15.157:0/3805405665 shutdown_connections 2023-11-06T03:17:30.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.022+0000 7f920149a640 1 -- 172.21.15.157:0/3805405665 wait complete. 2023-11-06T03:17:30.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 Processor -- start 2023-11-06T03:17:30.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- start start 2023-11-06T03:17:30.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 0x7f91fc160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc156a70 0x7f91fc1f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0af890 con 0x7f91fc0adae0 2023-11-06T03:17:30.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc1608b0 con 0x7f91fc0adf40 2023-11-06T03:17:30.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc1f8340 con 0x7f91fc156a70 2023-11-06T03:17:30.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55840/0 (socket says 172.21.15.157:55840) 2023-11-06T03:17:30.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 -- 172.21.15.157:0/683573492 learned_addr learned my addr 172.21.15.157:0/683573492 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:30.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91f27fc640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 0x7f91fc160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91fb7fe640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc156a70 0x7f91fc1f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 -- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc156a70 msgr2=0x7f91fc1f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc156a70 0x7f91fc1f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 -- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 msgr2=0x7f91fc160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 0x7f91fc160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 -- 172.21.15.157:0/683573492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f400bd40 con 0x7f91fc0adae0 2023-11-06T03:17:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91f27fc640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 0x7f91fc160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91faffd640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c3c0 tx=0x7f91ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0052d0 con 0x7f91fc0adae0 2023-11-06T03:17:30.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec005430 con 0x7f91fc0adae0 2023-11-06T03:17:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc1f85e0 con 0x7f91fc0adae0 2023-11-06T03:17:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec003840 con 0x7f91fc0adae0 2023-11-06T03:17:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc1f8ba0 con 0x7f91fc0adae0 2023-11-06T03:17:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.026+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc072750 con 0x7f91fc0adae0 2023-11-06T03:17:30.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f91ec0039a0 con 0x7f91fc0adae0 2023-11-06T03:17:30.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f3fff640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 0x7f91dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91ec082b70 con 0x7f91fc0adae0 2023-11-06T03:17:30.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f27fc640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 0x7f91dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f27fc640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 0x7f91dc067f20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f91fc15e180 tx=0x7f91e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.030+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f91ec050540 con 0x7f91fc0adae0 2023-11-06T03:17:30.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.222+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f91fc06a4c0 con 0x7f91fc0adae0 2023-11-06T03:17:30.222 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:17:30.226+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:17:30.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.242+0000 7f91f3fff640 1 -- 172.21.15.157:0/683573492 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) v1 ==== 161+0+0 (secure 0 0 0) 0x7f91ec053b60 con 0x7f91fc0adae0 2023-11-06T03:17:30.240 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:17:30.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.242+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 msgr2=0x7f91dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.242+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 0x7f91dc067f20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f91fc15e180 tx=0x7f91e4008040 comp rx=0 tx=0).stop 2023-11-06T03:17:30.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.242+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 msgr2=0x7f91fc0b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.242+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c3c0 tx=0x7f91ec00c880 comp rx=0 tx=0).stop 2023-11-06T03:17:30.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 shutdown_connections 2023-11-06T03:17:30.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f91dc065a60 0x7f91dc067f20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91fc156a70 0x7f91fc1f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91fc0adf40 0x7f91fc160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 --2- 172.21.15.157:0/683573492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91fc0adae0 0x7f91fc0b1080 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 >> 172.21.15.157:0/683573492 conn(0x7f91fc065930 msgr2=0x7f91fc15b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:30.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 shutdown_connections 2023-11-06T03:17:30.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.246+0000 7f920149a640 1 -- 172.21.15.157:0/683573492 wait complete. 2023-11-06T03:17:30.247 INFO:tasks.ceph.mds.d.smithi195.stderr: -5003> 2023-11-06T03:16:45.000+0000 7fc5ffb5a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:17:30.254 INFO:tasks.ceph.mds.d.smithi195.stderr: -3045> 2023-11-06T03:16:52.548+0000 7fc5fc353640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2023-11-06T03:16:51.191915+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10539=0-12582912@1} caps={10539=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@4},l=10539 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55a8f3c83b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(24)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-06T03:17:30.260 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:17:30.315 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:17:30.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.386+0000 7f39d7581640 1 Processor -- start 2023-11-06T03:17:30.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.386+0000 7f39d7581640 1 -- start start 2023-11-06T03:17:30.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d7581640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39d0154fc0 0x7f39d0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d7581640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b1090 0x7f39d00b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d7581640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d00b5450 con 0x7f39d00b5000 2023-11-06T03:17:30.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d7581640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00b0c70 con 0x7f39d0154fc0 2023-11-06T03:17:30.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d7581640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d015bfa0 con 0x7f39d00b1090 2023-11-06T03:17:30.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39d0154fc0 0x7f39d0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f39d0154fc0 0x7f39d0070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55856/0 (socket says 172.21.15.157:55856) 2023-11-06T03:17:30.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 -- 172.21.15.157:0/358951088 learned_addr learned my addr 172.21.15.157:0/358951088 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:30.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d5b1e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f39d00b5000 0x7f39d0065930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59922/0 (socket says 172.21.15.157:59922) 2023-11-06T03:17:30.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d4b1c640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b1090 0x7f39d00b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 -- 172.21.15.157:0/358951088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b1090 msgr2=0x7f39d00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:30.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b1090 0x7f39d00b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 -- 172.21.15.157:0/358951088 >> v1:172.21.15.157:6789/0 conn(0x7f39d00b5000 legacy=0x7f39d0065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 -- 172.21.15.157:0/358951088 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00b08d0 con 0x7f39d0154fc0 2023-11-06T03:17:30.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39d531d640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0070ee0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3cf301d7fb0585 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39c7fff640 1 -- 172.21.15.157:0/358951088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0ab680 con 0x7f39d0154fc0 2023-11-06T03:17:30.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39c7fff640 1 -- 172.21.15.157:0/358951088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc0ab7e0 con 0x7f39d0154fc0 2023-11-06T03:17:30.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.390+0000 7f39c7fff640 1 -- 172.21.15.157:0/358951088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0abb70 con 0x7f39d0154fc0 2023-11-06T03:17:30.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/358951088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 msgr2=0x7f39d0070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0070ee0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc01c540 comp rx=0 tx=0).stop 2023-11-06T03:17:30.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/358951088 shutdown_connections 2023-11-06T03:17:30.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b1090 0x7f39d00b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- 172.21.15.157:0/358951088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0070ee0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/358951088 >> 172.21.15.157:0/358951088 conn(0x7f39d0069d70 msgr2=0x7f39d006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:30.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/358951088 shutdown_connections 2023-11-06T03:17:30.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/358951088 wait complete. 2023-11-06T03:17:30.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 Processor -- start 2023-11-06T03:17:30.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- start start 2023-11-06T03:17:30.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d00b1090 0x7f39d014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 0x7f39d014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d0145370 con 0x7f39d0154fc0 2023-11-06T03:17:30.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01446f0 con 0x7f39d00b1090 2023-11-06T03:17:30.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01449d0 con 0x7f39d00b5000 2023-11-06T03:17:30.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55866/0 (socket says 172.21.15.157:55866) 2023-11-06T03:17:30.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 -- 172.21.15.157:0/3123124071 learned_addr learned my addr 172.21.15.157:0/3123124071 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:30.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d531d640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d00b1090 0x7f39d014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d4b1c640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 0x7f39d014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 -- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 msgr2=0x7f39d014fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 0x7f39d014fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 -- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d00b1090 msgr2=0x7f39d014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:30.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d00b1090 0x7f39d014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:30.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 -- 172.21.15.157:0/3123124071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0094d0 con 0x7f39d0154fc0 2023-11-06T03:17:30.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d4b1c640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 0x7f39d014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:30.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d5b1e640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c7a0 tx=0x7f39c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800a640 con 0x7f39d0154fc0 2023-11-06T03:17:30.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800a7a0 con 0x7f39d0154fc0 2023-11-06T03:17:30.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d0144cb0 con 0x7f39d0154fc0 2023-11-06T03:17:30.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ab10 con 0x7f39d0154fc0 2023-11-06T03:17:30.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.394+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d0143d30 con 0x7f39d0154fc0 2023-11-06T03:17:30.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d01454b0 con 0x7f39d0154fc0 2023-11-06T03:17:30.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f39c800ac70 con 0x7f39d0154fc0 2023-11-06T03:17:30.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39c5ffb640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 0x7f39b4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:30.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f39c8082960 con 0x7f39d0154fc0 2023-11-06T03:17:30.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39d531d640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 0x7f39b4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:30.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.398+0000 7f39d531d640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 0x7f39b4067f50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af000 tx=0x7f39cc0b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:30.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.402+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f39d01454b0 con 0x7f39d0154fc0 2023-11-06T03:17:30.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:30.586+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f39d00b5450 con 0x7f39d0154fc0 2023-11-06T03:17:31.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.246+0000 7f39c5ffb640 1 -- 172.21.15.157:0/3123124071 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v119) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39d00b5450 con 0x7f39d0154fc0 2023-11-06T03:17:31.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 msgr2=0x7f39b4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 0x7f39b4067f50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af000 tx=0x7f39cc0b6040 comp rx=0 tx=0).stop 2023-11-06T03:17:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 msgr2=0x7f39d0142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c7a0 tx=0x7f39c800cc60 comp rx=0 tx=0).stop 2023-11-06T03:17:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 shutdown_connections 2023-11-06T03:17:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f39b4065a90 0x7f39b4067f50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d0154fc0 0x7f39d0142fa0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f39d00b5000 0x7f39d014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 --2- 172.21.15.157:0/3123124071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f39d00b1090 0x7f39d014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 >> 172.21.15.157:0/3123124071 conn(0x7f39d0069d70 msgr2=0x7f39d0153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 shutdown_connections 2023-11-06T03:17:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.250+0000 7f39d7581640 1 -- 172.21.15.157:0/3123124071 wait complete. 2023-11-06T03:17:31.268 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:17:31.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 Processor -- start 2023-11-06T03:17:31.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 -- start start 2023-11-06T03:17:31.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f874006c550 0x7f874015b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400cb100 con 0x7f8740155310 2023-11-06T03:17:31.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400ce0a0 con 0x7f874006c170 2023-11-06T03:17:31.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8748273640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874015bef0 con 0x7f874006c550 2023-11-06T03:17:31.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874580e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f8740155310 0x7f874015b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59938/0 (socket says 172.21.15.157:59938) 2023-11-06T03:17:31.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874580e640 1 -- 172.21.15.157:0/3847911471 learned_addr learned my addr 172.21.15.157:0/3847911471 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:31.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252028941 0 0) 0x7f87400cb100 con 0x7f8740155310 2023-11-06T03:17:31.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8728003530 con 0x7f8740155310 2023-11-06T03:17:31.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8746810640 1 --2- 172.21.15.157:0/3847911471 >> v2:172.21.15.195:3300/0 conn(0x7f874006c550 0x7f874015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106276806 0 0) 0x7f8728003530 con 0x7f8740155310 2023-11-06T03:17:31.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 >> v2:172.21.15.195:3300/0 conn(0x7f874006c550 msgr2=0x7f874015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:31.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 --2- 172.21.15.157:0/3847911471 >> v2:172.21.15.195:3300/0 conn(0x7f874006c550 0x7f874015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 >> v1:172.21.15.195:6789/0 conn(0x7f874006c170 legacy=0x7f8740154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400cb350 con 0x7f8740155310 2023-11-06T03:17:31.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f8734001680 con 0x7f8740155310 2023-11-06T03:17:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 >> v1:172.21.15.157:6789/0 conn(0x7f8740155310 legacy=0x7f874015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728003980 0x7f8728003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87280047d0 0x7f8728004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 0x7f872800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874015bef0 con 0x7f8728005190 2023-11-06T03:17:31.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872800c0e0 con 0x7f87280047d0 2023-11-06T03:17:31.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872800c2a0 con 0x7f8728003980 2023-11-06T03:17:31.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f8746810640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 0x7f872800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874580e640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87280047d0 0x7f8728004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.398+0000 7f874600f640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728003980 0x7f8728003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 -- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728003980 msgr2=0x7f8728003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:31.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728003980 0x7f8728003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 -- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87280047d0 msgr2=0x7f8728004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:31.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87280047d0 0x7f8728004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 -- 172.21.15.157:0/3847911471 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728003530 con 0x7f8728005190 2023-11-06T03:17:31.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 -- 172.21.15.157:0/3847911471 reap_dead start 2023-11-06T03:17:31.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8746810640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 0x7f872800b990 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f8738007360 tx=0x7f873801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89490f9b921d6dd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:31.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380a5d60 con 0x7f8728005190 2023-11-06T03:17:31.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380039a0 con 0x7f8728005190 2023-11-06T03:17:31.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f874500d640 1 -- 172.21.15.157:0/3847911471 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738003d50 con 0x7f8728005190 2023-11-06T03:17:31.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 -- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 msgr2=0x7f872800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 0x7f872800b990 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f8738007360 tx=0x7f873801ce60 comp rx=0 tx=0).stop 2023-11-06T03:17:31.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 -- 172.21.15.157:0/3847911471 shutdown_connections 2023-11-06T03:17:31.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 --2- 172.21.15.157:0/3847911471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8728005190 0x7f872800b990 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 -- 172.21.15.157:0/3847911471 >> 172.21.15.157:0/3847911471 conn(0x7f87400b43b0 msgr2=0x7f8740065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 -- 172.21.15.157:0/3847911471 shutdown_connections 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.402+0000 7f8748273640 1 -- 172.21.15.157:0/3847911471 wait complete. 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 Processor -- start 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- start start 2023-11-06T03:17:31.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 0x7f87401492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8740152520 0x7f874014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874015dda0 con 0x7f874015d3b0 2023-11-06T03:17:31.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401529a0 con 0x7f8740152520 2023-11-06T03:17:31.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874014dc80 con 0x7f8728005190 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55874/0 (socket says 172.21.15.157:55874) 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 -- 172.21.15.157:0/3722091891 learned_addr learned my addr 172.21.15.157:0/3722091891 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f874580e640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8740152520 0x7f874014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f874600f640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 0x7f87401492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 -- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 msgr2=0x7f87401492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 0x7f87401492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 -- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8740152520 msgr2=0x7f874014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8740152520 0x7f874014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 -- 172.21.15.157:0/3722091891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87401f3e30 con 0x7f874015d3b0 2023-11-06T03:17:31.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f874600f640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 0x7f87401492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:31.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8746810640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f873801d700 tx=0x7f8738019c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:31.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380b5600 con 0x7f874015d3b0 2023-11-06T03:17:31.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380b5780 con 0x7f874015d3b0 2023-11-06T03:17:31.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380b5b30 con 0x7f874015d3b0 2023-11-06T03:17:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87401f4110 con 0x7f874015d3b0 2023-11-06T03:17:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401f46d0 con 0x7f874015d3b0 2023-11-06T03:17:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.406+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874004e050 con 0x7f874015d3b0 2023-11-06T03:17:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.410+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87380b5c90 con 0x7f874015d3b0 2023-11-06T03:17:31.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.410+0000 7f873effd640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 0x7f872c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.410+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f873811d9c0 con 0x7f874015d3b0 2023-11-06T03:17:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.410+0000 7f874600f640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 0x7f872c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.410+0000 7f874600f640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 0x7f872c067f50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8730007fd0 tx=0x7f8730009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:31.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.414+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f87380eb130 con 0x7f874015d3b0 2023-11-06T03:17:31.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:31.570+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87400ccbb0 con 0x7f874015d3b0 2023-11-06T03:17:32.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.258+0000 7f873effd640 1 -- 172.21.15.157:0/3722091891 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v165) v1 ==== 181+0+0 (secure 0 0 0) 0x7f87380ee750 con 0x7f874015d3b0 2023-11-06T03:17:32.257 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:17:32.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 msgr2=0x7f872c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 0x7f872c067f50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8730007fd0 tx=0x7f8730009040 comp rx=0 tx=0).stop 2023-11-06T03:17:32.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 msgr2=0x7f874015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f873801d700 tx=0x7f8738019c00 comp rx=0 tx=0).stop 2023-11-06T03:17:32.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 shutdown_connections 2023-11-06T03:17:32.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f872c065a90 0x7f872c067f50 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874015d3b0 0x7f874015d860 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8740152520 0x7f874014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 --2- 172.21.15.157:0/3722091891 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8728005190 0x7f87401492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 >> 172.21.15.157:0/3722091891 conn(0x7f87400b43b0 msgr2=0x7f8740065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:32.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 shutdown_connections 2023-11-06T03:17:32.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.262+0000 7f8748273640 1 -- 172.21.15.157:0/3722091891 wait complete. 2023-11-06T03:17:32.286 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:17:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 Processor -- start 2023-11-06T03:17:32.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 -- start start 2023-11-06T03:17:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf1815b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180cb100 con 0x7fdf1806c550 2023-11-06T03:17:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180ce0a0 con 0x7fdf1806c170 2023-11-06T03:17:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf1d352640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815bef0 con 0x7fdf18155310 2023-11-06T03:17:32.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf177fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdf1806c550 0x7fdf1815b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59954/0 (socket says 172.21.15.157:59954) 2023-11-06T03:17:32.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf177fe640 1 -- 172.21.15.157:0/3569935128 learned_addr learned my addr 172.21.15.157:0/3569935128 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:32.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf167fc640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf1815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166482543 0 0) 0x7fdf180cb100 con 0x7fdf1806c550 2023-11-06T03:17:32.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf00003530 con 0x7fdf1806c550 2023-11-06T03:17:32.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2946563056 0 0) 0x7fdf00003530 con 0x7fdf1806c550 2023-11-06T03:17:32.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 msgr2=0x7fdf1815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:32.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf1815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 >> v1:172.21.15.195:6789/0 conn(0x7fdf1806c170 legacy=0x7fdf18154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180cb350 con 0x7fdf1806c550 2023-11-06T03:17:32.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdf10001a00 con 0x7fdf1806c550 2023-11-06T03:17:32.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 >> v1:172.21.15.157:6789/0 conn(0x7fdf1806c550 legacy=0x7fdf1815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf000039b0 0x7fdf00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf00004d10 0x7fdf00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815bef0 con 0x7fdf000039b0 2023-11-06T03:17:32.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0000ba40 con 0x7fdf00004d10 2023-11-06T03:17:32.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0000bc00 con 0x7fdf18155310 2023-11-06T03:17:32.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf000039b0 0x7fdf00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf167fc640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf177fe640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf00004d10 0x7fdf00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 -- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 msgr2=0x7fdf000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf18155310 0x7fdf000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 -- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf00004d10 msgr2=0x7fdf00005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf00004d10 0x7fdf00005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf16ffd640 1 -- 172.21.15.157:0/3569935128 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf00003530 con 0x7fdf000039b0 2023-11-06T03:17:32.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.418+0000 7fdf177fe640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf00004d10 0x7fdf00005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf000039b0 0x7fdf00003dd0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fdf08001750 tx=0x7fdf0801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142091157ba84703 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080052d0 con 0x7fdf000039b0 2023-11-06T03:17:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf080a7050 con 0x7fdf000039b0 2023-11-06T03:17:32.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf15ffb640 1 -- 172.21.15.157:0/3569935128 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08003600 con 0x7fdf000039b0 2023-11-06T03:17:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- 172.21.15.157:0/3569935128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf000039b0 msgr2=0x7fdf00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 --2- 172.21.15.157:0/3569935128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf000039b0 0x7fdf00003dd0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fdf08001750 tx=0x7fdf0801c390 comp rx=0 tx=0).stop 2023-11-06T03:17:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf177fe640 1 -- 172.21.15.157:0/3569935128 reap_dead start 2023-11-06T03:17:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- 172.21.15.157:0/3569935128 shutdown_connections 2023-11-06T03:17:32.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- 172.21.15.157:0/3569935128 >> 172.21.15.157:0/3569935128 conn(0x7fdf180b43b0 msgr2=0x7fdf18065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- 172.21.15.157:0/3569935128 shutdown_connections 2023-11-06T03:17:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- 172.21.15.157:0/3569935128 wait complete. 2023-11-06T03:17:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 Processor -- start 2023-11-06T03:17:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.422+0000 7fdf1d352640 1 -- start start 2023-11-06T03:17:32.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf000039b0 0x7fdf180b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 0x7fdf18161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18162260 con 0x7fdf180b2c10 2023-11-06T03:17:32.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf180b1c60 con 0x7fdf181618a0 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1814fbd0 con 0x7fdf000039b0 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55892/0 (socket says 172.21.15.157:55892) 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 -- 172.21.15.157:0/1035896335 learned_addr learned my addr 172.21.15.157:0/1035896335 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf177fe640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 0x7fdf18161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf000039b0 0x7fdf180b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 -- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf000039b0 msgr2=0x7fdf180b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf000039b0 0x7fdf180b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 -- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 msgr2=0x7fdf18161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 0x7fdf18161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:32.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 -- 172.21.15.157:0/1035896335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180cbfb0 con 0x7fdf180b2c10 2023-11-06T03:17:32.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf177fe640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 0x7fdf18161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:32.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf167fc640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0115c0 tx=0x7fdf0c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:32.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c004220 con 0x7fdf180b2c10 2023-11-06T03:17:32.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c004380 con 0x7fdf180b2c10 2023-11-06T03:17:32.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00f8f0 con 0x7fdf180b2c10 2023-11-06T03:17:32.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf180b3090 con 0x7fdf180b2c10 2023-11-06T03:17:32.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf180b13c0 con 0x7fdf180b2c10 2023-11-06T03:17:32.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1804e050 con 0x7fdf180b2c10 2023-11-06T03:17:32.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdf0c00fa50 con 0x7fdf180b2c10 2023-11-06T03:17:32.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 0x7fdf04067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:32.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fdf0c083350 con 0x7fdf180b2c10 2023-11-06T03:17:32.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.426+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 0x7fdf04067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:32.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.430+0000 7fdf16ffd640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 0x7fdf04067f50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf000057b0 tx=0x7fdf080011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:32.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.430+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdf0c050c70 con 0x7fdf180b2c10 2023-11-06T03:17:32.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:32.586+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf1814fd10 con 0x7fdf180b2c10 2023-11-06T03:17:33.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.538+0000 7fdeff7fe640 1 -- 172.21.15.157:0/1035896335 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v166) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf0c054290 con 0x7fdf180b2c10 2023-11-06T03:17:33.536 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:17:33.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 msgr2=0x7fdf04067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 0x7fdf04067f50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf000057b0 tx=0x7fdf080011a0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 msgr2=0x7fdf1814e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0115c0 tx=0x7fdf0c011a80 comp rx=0 tx=0).stop 2023-11-06T03:17:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 shutdown_connections 2023-11-06T03:17:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdf04065a90 0x7fdf04067f50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf181618a0 0x7fdf18161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf180b2c10 0x7fdf1814e1b0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 --2- 172.21.15.157:0/1035896335 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdf000039b0 0x7fdf180b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 >> 172.21.15.157:0/1035896335 conn(0x7fdf180b43b0 msgr2=0x7fdf180ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 shutdown_connections 2023-11-06T03:17:33.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.542+0000 7fdf1d352640 1 -- 172.21.15.157:0/1035896335 wait complete. 2023-11-06T03:17:33.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:17:33.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 Processor -- start 2023-11-06T03:17:33.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 -- start start 2023-11-06T03:17:33.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 0x7f35ac06f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ac0adae0 0x7f35ac0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac06c490 con 0x7f35ac0adf40 2023-11-06T03:17:33.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06b950 con 0x7f35ac0adae0 2023-11-06T03:17:33.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b3e29640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06bb60 con 0x7f35ac156a70 2023-11-06T03:17:33.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ac0adae0 0x7f35ac0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:33.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b23c6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f35ac0adf40 0x7f35ac06c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59968/0 (socket says 172.21.15.157:59968) 2023-11-06T03:17:33.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b23c6640 1 -- 172.21.15.157:0/4013482259 learned_addr learned my addr 172.21.15.157:0/4013482259 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:33.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f35ac0adae0 0x7f35ac0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55904/0 (socket says 172.21.15.157:55904) 2023-11-06T03:17:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b1bc5640 1 --2- 172.21.15.157:0/4013482259 >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 0x7f35ac06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 -- 172.21.15.157:0/4013482259 >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 msgr2=0x7f35ac06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 --2- 172.21.15.157:0/4013482259 >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 0x7f35ac06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 -- 172.21.15.157:0/4013482259 >> v1:172.21.15.157:6789/0 conn(0x7f35ac0adf40 legacy=0x7f35ac06c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b13c4640 1 -- 172.21.15.157:0/4013482259 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac06be40 con 0x7f35ac0adae0 2023-11-06T03:17:33.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b1bc5640 1 --2- 172.21.15.157:0/4013482259 >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 0x7f35ac06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:33.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.690+0000 7f35b0bc3640 1 -- 172.21.15.157:0/4013482259 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081454374 0 0) 0x7f35ac06c490 con 0x7f35ac0adf40 2023-11-06T03:17:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b13c4640 1 --2- 172.21.15.157:0/4013482259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adae0 0x7f35ac0b1c20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0ce880 tx=0x7f359c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7ba860a0e91375 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b0bc3640 1 -- 172.21.15.157:0/4013482259 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0ab650 con 0x7f35ac0adae0 2023-11-06T03:17:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b0bc3640 1 -- 172.21.15.157:0/4013482259 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c0ab7b0 con 0x7f35ac0adae0 2023-11-06T03:17:33.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b0bc3640 1 -- 172.21.15.157:0/4013482259 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0abb40 con 0x7f35ac0adae0 2023-11-06T03:17:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- 172.21.15.157:0/4013482259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adae0 msgr2=0x7f35ac0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 --2- 172.21.15.157:0/4013482259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adae0 0x7f35ac0b1c20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0ce880 tx=0x7f359c01c430 comp rx=0 tx=0).stop 2023-11-06T03:17:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- 172.21.15.157:0/4013482259 shutdown_connections 2023-11-06T03:17:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 --2- 172.21.15.157:0/4013482259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adae0 0x7f35ac0b1c20 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 --2- 172.21.15.157:0/4013482259 >> v2:172.21.15.195:3300/0 conn(0x7f35ac156a70 0x7f35ac06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- 172.21.15.157:0/4013482259 >> 172.21.15.157:0/4013482259 conn(0x7f35ac065930 msgr2=0x7f35ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:33.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- 172.21.15.157:0/4013482259 shutdown_connections 2023-11-06T03:17:33.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- 172.21.15.157:0/4013482259 wait complete. 2023-11-06T03:17:33.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 Processor -- start 2023-11-06T03:17:33.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.694+0000 7f35b3e29640 1 -- start start 2023-11-06T03:17:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35ac0adae0 0x7f35ac1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35ac156a70 0x7f35ac0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac150b00 con 0x7f35ac0adf40 2023-11-06T03:17:33.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac151150 con 0x7f35ac0adae0 2023-11-06T03:17:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac151430 con 0x7f35ac156a70 2023-11-06T03:17:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55920/0 (socket says 172.21.15.157:55920) 2023-11-06T03:17:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 -- 172.21.15.157:0/3145307123 learned_addr learned my addr 172.21.15.157:0/3145307123 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:33.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 -- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35ac156a70 msgr2=0x7f35ac0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35ac156a70 0x7f35ac0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 -- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35ac0adae0 msgr2=0x7f35ac1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b1bc5640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35ac0adae0 0x7f35ac1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:33.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35ac0adae0 0x7f35ac1505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 -- 172.21.15.157:0/3145307123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c009420 con 0x7f35ac0adf40 2023-11-06T03:17:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b13c4640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f359c019040 tx=0x7f359c01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0b5810 con 0x7f35ac0adf40 2023-11-06T03:17:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c0b5990 con 0x7f35ac0adf40 2023-11-06T03:17:33.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac151710 con 0x7f35ac0adf40 2023-11-06T03:17:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c0b5d40 con 0x7f35ac0adf40 2023-11-06T03:17:33.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac1519f0 con 0x7f35ac0adf40 2023-11-06T03:17:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac151570 con 0x7f35ac0adf40 2023-11-06T03:17:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f359c0b4390 con 0x7f35ac0adf40 2023-11-06T03:17:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:33.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f359c11dea0 con 0x7f35ac0adf40 2023-11-06T03:17:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.698+0000 7f35b1bc5640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.702+0000 7f35b1bc5640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 0x7f3590067f20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006fd0 tx=0x7f35a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:33.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.702+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f35ac151570 con 0x7f35ac0adf40 2023-11-06T03:17:33.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.858+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f35ac151570 con 0x7f35ac0adf40 2023-11-06T03:17:33.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.866+0000 7f35a27fc640 1 -- 172.21.15.157:0/3145307123 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f35ac151570 con 0x7f35ac0adf40 2023-11-06T03:17:33.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 msgr2=0x7f3590067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 0x7f3590067f20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006fd0 tx=0x7f35a8008040 comp rx=0 tx=0).stop 2023-11-06T03:17:33.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 msgr2=0x7f35ac070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:33.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f359c019040 tx=0x7f359c01ac70 comp rx=0 tx=0).stop 2023-11-06T03:17:33.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 shutdown_connections 2023-11-06T03:17:33.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3590065a60 0x7f3590067f20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35ac156a70 0x7f35ac0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35ac0adf40 0x7f35ac070380 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 --2- 172.21.15.157:0/3145307123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35ac0adae0 0x7f35ac1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:33.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 >> 172.21.15.157:0/3145307123 conn(0x7f35ac065930 msgr2=0x7f35ac157bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:33.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 shutdown_connections 2023-11-06T03:17:33.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:33.870+0000 7f35b3e29640 1 -- 172.21.15.157:0/3145307123 wait complete. 2023-11-06T03:17:33.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2023-11-06T03:17:34.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 Processor -- start 2023-11-06T03:17:34.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 -- start start 2023-11-06T03:17:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab6c0adae0 0x7fab6c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 0x7fab6c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c06c490 con 0x7fab6c156a70 2023-11-06T03:17:34.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c06b950 con 0x7fab6c0adae0 2023-11-06T03:17:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab746cb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c06fb80 con 0x7fab6c0adf40 2023-11-06T03:17:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab6c0adae0 0x7fab6c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab6c0adae0 0x7fab6c0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55926/0 (socket says 172.21.15.157:55926) 2023-11-06T03:17:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 -- 172.21.15.157:0/696415847 learned_addr learned my addr 172.21.15.157:0/696415847 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:34.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab72c68640 1 --2- 172.21.15.157:0/696415847 >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 0x7fab6c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 -- 172.21.15.157:0/696415847 >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 msgr2=0x7fab6c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 --2- 172.21.15.157:0/696415847 >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 0x7fab6c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 -- 172.21.15.157:0/696415847 >> v1:172.21.15.195:6789/0 conn(0x7fab6c156a70 legacy=0x7fab6c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab71c66640 1 -- 172.21.15.157:0/696415847 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c06bb60 con 0x7fab6c0adae0 2023-11-06T03:17:34.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.006+0000 7fab72c68640 1 --2- 172.21.15.157:0/696415847 >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 0x7fab6c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab71c66640 1 --2- 172.21.15.157:0/696415847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b1c20 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0ce830 tx=0x7fab5c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e0afb122db054a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab71465640 1 -- 172.21.15.157:0/696415847 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0aa5f0 con 0x7fab6c0adae0 2023-11-06T03:17:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab71465640 1 -- 172.21.15.157:0/696415847 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c0aa750 con 0x7fab6c0adae0 2023-11-06T03:17:34.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab71465640 1 -- 172.21.15.157:0/696415847 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0aaae0 con 0x7fab6c0adae0 2023-11-06T03:17:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 -- 172.21.15.157:0/696415847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c0adae0 msgr2=0x7fab6c0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 --2- 172.21.15.157:0/696415847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b1c20 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0ce830 tx=0x7fab5c01c490 comp rx=0 tx=0).stop 2023-11-06T03:17:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 -- 172.21.15.157:0/696415847 shutdown_connections 2023-11-06T03:17:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 --2- 172.21.15.157:0/696415847 >> v2:172.21.15.195:3300/0 conn(0x7fab6c0adf40 0x7fab6c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 --2- 172.21.15.157:0/696415847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b1c20 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 -- 172.21.15.157:0/696415847 >> 172.21.15.157:0/696415847 conn(0x7fab6c065930 msgr2=0x7fab6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 -- 172.21.15.157:0/696415847 shutdown_connections 2023-11-06T03:17:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.010+0000 7fab746cb640 1 -- 172.21.15.157:0/696415847 wait complete. 2023-11-06T03:17:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 Processor -- start 2023-11-06T03:17:34.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- start start 2023-11-06T03:17:34.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6c0adf40 0x7fab6c149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c149120 con 0x7fab6c156a70 2023-11-06T03:17:34.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0ad420 con 0x7fab6c0adae0 2023-11-06T03:17:34.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0b1490 con 0x7fab6c0adf40 2023-11-06T03:17:34.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55938/0 (socket says 172.21.15.157:55938) 2023-11-06T03:17:34.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 -- 172.21.15.157:0/3551323917 learned_addr learned my addr 172.21.15.157:0/3551323917 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:34.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab71c66640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6c0adf40 0x7fab6c149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 -- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6c0adf40 msgr2=0x7fab6c149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6c0adf40 0x7fab6c149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 -- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab6c0adae0 msgr2=0x7fab6c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72467640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 -- 172.21.15.157:0/3551323917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0094d0 con 0x7fab6c156a70 2023-11-06T03:17:34.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72c68640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cef0 tx=0x7fab6400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400d840 con 0x7fab6c156a70 2023-11-06T03:17:34.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400d9c0 con 0x7fab6c156a70 2023-11-06T03:17:34.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1484a0 con 0x7fab6c156a70 2023-11-06T03:17:34.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400dd70 con 0x7fab6c156a70 2023-11-06T03:17:34.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c148a60 con 0x7fab6c156a70 2023-11-06T03:17:34.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c149260 con 0x7fab6c156a70 2023-11-06T03:17:34.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fab6401e3b0 con 0x7fab6c156a70 2023-11-06T03:17:34.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 0x7fab50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fab640839e0 con 0x7fab6c156a70 2023-11-06T03:17:34.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.014+0000 7fab72467640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 0x7fab50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.018+0000 7fab72467640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 0x7fab50067f20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fab6800a750 tx=0x7fab68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.018+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fab6c149260 con 0x7fab6c156a70 2023-11-06T03:17:34.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.238+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7fab6c149260 con 0x7fab6c156a70 2023-11-06T03:17:34.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.534+0000 7fab62ffd640 1 -- 172.21.15.157:0/3551323917 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v618) v1 ==== 133+0+0 (secure 0 0 0) 0x7fab6c149260 con 0x7fab6c156a70 2023-11-06T03:17:34.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 msgr2=0x7fab50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 0x7fab50067f20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fab6800a750 tx=0x7fab68008040 comp rx=0 tx=0).stop 2023-11-06T03:17:34.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 msgr2=0x7fab6c147930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fab6400cef0 tx=0x7fab6400a710 comp rx=0 tx=0).stop 2023-11-06T03:17:34.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 shutdown_connections 2023-11-06T03:17:34.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab50065a60 0x7fab50067f20 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab6c156a70 0x7fab6c147930 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab6c0adf40 0x7fab6c149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 --2- 172.21.15.157:0/3551323917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab6c0adae0 0x7fab6c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 >> 172.21.15.157:0/3551323917 conn(0x7fab6c065930 msgr2=0x7fab6c15d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:34.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 shutdown_connections 2023-11-06T03:17:34.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.538+0000 7fab746cb640 1 -- 172.21.15.157:0/3551323917 wait complete. 2023-11-06T03:17:34.556 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2023-11-06T03:17:34.557 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2023-11-06T03:17:34.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2023-11-06T03:17:34.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 Processor -- start 2023-11-06T03:17:34.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 -- start start 2023-11-06T03:17:34.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f55281599c0 0x7f55280ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552815e440 con 0x7f55280b2af0 2023-11-06T03:17:34.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552806c240 con 0x7f552806d000 2023-11-06T03:17:34.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f552d5f2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552806c520 con 0x7f55281599c0 2023-11-06T03:17:34.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55277fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f55280b2af0 0x7f5528159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59984/0 (socket says 172.21.15.157:59984) 2023-11-06T03:17:34.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55277fe640 1 -- 172.21.15.157:0/2398655662 learned_addr learned my addr 172.21.15.157:0/2398655662 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:34.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> v2:172.21.15.195:3300/0 conn(0x7f55281599c0 0x7f55280ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633126922 0 0) 0x7f552815e440 con 0x7f55280b2af0 2023-11-06T03:17:34.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5514003530 con 0x7f55280b2af0 2023-11-06T03:17:34.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464734031 0 0) 0x7f5514003530 con 0x7f55280b2af0 2023-11-06T03:17:34.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 >> v2:172.21.15.195:3300/0 conn(0x7f55281599c0 msgr2=0x7f55280ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:34.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 --2- 172.21.15.157:0/2398655662 >> v2:172.21.15.195:3300/0 conn(0x7f55281599c0 0x7f55280ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 >> v1:172.21.15.195:6789/0 conn(0x7f552806d000 legacy=0x7f55280b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552806c800 con 0x7f55280b2af0 2023-11-06T03:17:34.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5520001a00 con 0x7f55280b2af0 2023-11-06T03:17:34.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> v2:172.21.15.195:3300/0 conn(0x7f55281599c0 0x7f55280ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:34.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 >> v1:172.21.15.157:6789/0 conn(0x7f55280b2af0 legacy=0x7f5528159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 0x7f5514003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55140047d0 0x7f5514004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5514005190 0x7f551400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552806c520 con 0x7f5514003980 2023-11-06T03:17:34.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551400e2f0 con 0x7f5514005190 2023-11-06T03:17:34.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551400e4b0 con 0x7f55140047d0 2023-11-06T03:17:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 0x7f5514003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55277fe640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5514005190 0x7f551400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f551ffff640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55140047d0 0x7f5514004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 -- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55140047d0 msgr2=0x7f5514004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:34.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55140047d0 0x7f5514004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 -- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5514005190 msgr2=0x7f551400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:34.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5514005190 0x7f551400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f5526ffd640 1 -- 172.21.15.157:0/2398655662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514003530 con 0x7f5514003980 2023-11-06T03:17:34.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f55277fe640 1 -- 172.21.15.157:0/2398655662 reap_dead start 2023-11-06T03:17:34.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.686+0000 7f551ffff640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55140047d0 0x7f5514004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:34.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f5526ffd640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 0x7f5514003da0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f55180098d0 tx=0x7f551801c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6590410a2cfc9b50 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180aa3f0 con 0x7f5514003980 2023-11-06T03:17:34.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55180aa550 con 0x7f5514003980 2023-11-06T03:17:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55267fc640 1 -- 172.21.15.157:0/2398655662 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180aa8c0 con 0x7f5514003980 2023-11-06T03:17:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 msgr2=0x7f5514003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 0x7f5514003da0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f55180098d0 tx=0x7f551801c580 comp rx=0 tx=0).stop 2023-11-06T03:17:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- 172.21.15.157:0/2398655662 shutdown_connections 2023-11-06T03:17:34.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 --2- 172.21.15.157:0/2398655662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5514003980 0x7f5514003da0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- 172.21.15.157:0/2398655662 >> 172.21.15.157:0/2398655662 conn(0x7f5528065930 msgr2=0x7f5528065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:34.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- 172.21.15.157:0/2398655662 shutdown_connections 2023-11-06T03:17:34.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- 172.21.15.157:0/2398655662 wait complete. 2023-11-06T03:17:34.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 Processor -- start 2023-11-06T03:17:34.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- start start 2023-11-06T03:17:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 0x7f55281547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5528151a00 0x7f55281e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55281e5400 con 0x7f55281e4a40 2023-11-06T03:17:34.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5528151e80 con 0x7f5528151a00 2023-11-06T03:17:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f552d5f2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55281e47f0 con 0x7f5514003980 2023-11-06T03:17:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55277fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55952/0 (socket says 172.21.15.157:55952) 2023-11-06T03:17:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f55277fe640 1 -- 172.21.15.157:0/3789014778 learned_addr learned my addr 172.21.15.157:0/3789014778 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:34.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.690+0000 7f5526ffd640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 0x7f55281547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551ffff640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5528151a00 0x7f55281e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 -- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 msgr2=0x7f55281547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 0x7f55281547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 -- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5528151a00 msgr2=0x7f55281e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:34.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5528151a00 0x7f55281e3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 -- 172.21.15.157:0/3789014778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55281f30c0 con 0x7f55281e4a40 2023-11-06T03:17:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f5526ffd640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 0x7f55281547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f55277fe640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f5520007960 tx=0x7f552000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552000eb40 con 0x7f55281e4a40 2023-11-06T03:17:34.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552000eca0 con 0x7f55281e4a40 2023-11-06T03:17:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55281e4510 con 0x7f55281e4a40 2023-11-06T03:17:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520014430 con 0x7f55281e4a40 2023-11-06T03:17:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5528152480 con 0x7f55281e4a40 2023-11-06T03:17:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5528072750 con 0x7f55281e4a40 2023-11-06T03:17:34.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f55200145b0 con 0x7f55281e4a40 2023-11-06T03:17:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 0x7f5508067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5520082b60 con 0x7f55281e4a40 2023-11-06T03:17:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f5526ffd640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 0x7f5508067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.694+0000 7f5526ffd640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 0x7f5508067f50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f55180ae000 tx=0x7f55180046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:34.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.698+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f55200541e0 con 0x7f55281e4a40 2023-11-06T03:17:34.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:34.910+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f55280cb110 con 0x7f55281e4a40 2023-11-06T03:17:35.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f551f7fe640 1 -- 172.21.15.157:0/3789014778 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v619) v1 ==== 150+0+0 (secure 0 0 0) 0x7f5520053d70 con 0x7f55281e4a40 2023-11-06T03:17:35.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 msgr2=0x7f5508067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 0x7f5508067f50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f55180ae000 tx=0x7f55180046d0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 msgr2=0x7f55281e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f5520007960 tx=0x7f552000e530 comp rx=0 tx=0).stop 2023-11-06T03:17:35.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 shutdown_connections 2023-11-06T03:17:35.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5508065a90 0x7f5508067f50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55281e4a40 0x7f55281e4ec0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5528151a00 0x7f55281e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 --2- 172.21.15.157:0/3789014778 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5514003980 0x7f55281547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.546+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 >> 172.21.15.157:0/3789014778 conn(0x7f5528065930 msgr2=0x7f552815c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.550+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 shutdown_connections 2023-11-06T03:17:35.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.550+0000 7f552d5f2640 1 -- 172.21.15.157:0/3789014778 wait complete. 2023-11-06T03:17:35.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:17:35.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 Processor -- start 2023-11-06T03:17:35.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 -- start start 2023-11-06T03:17:35.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2d541551e0 0x7f2d5406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2d5415e5e0 0x7f2d540b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5415f580 con 0x7f2d5415ea90 2023-11-06T03:17:35.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5415efc0 con 0x7f2d541551e0 2023-11-06T03:17:35.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d59b4b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5415f2a0 con 0x7f2d5415e5e0 2023-11-06T03:17:35.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d53fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f2d5415ea90 0x7f2d5415c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:59996/0 (socket says 172.21.15.157:59996) 2023-11-06T03:17:35.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d53fff640 1 -- 172.21.15.157:0/1191829749 learned_addr learned my addr 172.21.15.157:0/1191829749 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:35.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d537fe640 1 --2- 172.21.15.157:0/1191829749 >> v2:172.21.15.157:3300/0 conn(0x7f2d541551e0 0x7f2d5406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d52ffd640 1 --2- 172.21.15.157:0/1191829749 >> v2:172.21.15.195:3300/0 conn(0x7f2d5415e5e0 0x7f2d540b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d527fc640 1 -- 172.21.15.157:0/1191829749 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614217087 0 0) 0x7f2d5415f580 con 0x7f2d5415ea90 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d527fc640 1 -- 172.21.15.157:0/1191829749 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d40003530 con 0x7f2d5415ea90 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d537fe640 1 -- 172.21.15.157:0/1191829749 >> v2:172.21.15.195:3300/0 conn(0x7f2d5415e5e0 msgr2=0x7f2d540b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d537fe640 1 --2- 172.21.15.157:0/1191829749 >> v2:172.21.15.195:3300/0 conn(0x7f2d5415e5e0 0x7f2d540b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d537fe640 1 -- 172.21.15.157:0/1191829749 >> v1:172.21.15.157:6789/0 conn(0x7f2d5415ea90 legacy=0x7f2d5415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.694+0000 7f2d537fe640 1 -- 172.21.15.157:0/1191829749 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5406f5a0 con 0x7f2d541551e0 2023-11-06T03:17:35.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d537fe640 1 --2- 172.21.15.157:0/1191829749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d541551e0 0x7f2d5406fda0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f2d54066880 tx=0x7f2d4401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a17dc6f40b32326 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:35.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d527fc640 1 -- 172.21.15.157:0/1191829749 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440a5820 con 0x7f2d541551e0 2023-11-06T03:17:35.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d527fc640 1 -- 172.21.15.157:0/1191829749 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d440a5980 con 0x7f2d541551e0 2023-11-06T03:17:35.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d527fc640 1 -- 172.21.15.157:0/1191829749 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d440a5d10 con 0x7f2d541551e0 2023-11-06T03:17:35.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 -- 172.21.15.157:0/1191829749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d541551e0 msgr2=0x7f2d5406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/1191829749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d541551e0 0x7f2d5406fda0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f2d54066880 tx=0x7f2d4401c850 comp rx=0 tx=0).stop 2023-11-06T03:17:35.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 -- 172.21.15.157:0/1191829749 shutdown_connections 2023-11-06T03:17:35.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/1191829749 >> v2:172.21.15.195:3300/0 conn(0x7f2d5415e5e0 0x7f2d540b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/1191829749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d541551e0 0x7f2d5406fda0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 -- 172.21.15.157:0/1191829749 >> 172.21.15.157:0/1191829749 conn(0x7f2d54065930 msgr2=0x7f2d54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:35.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 -- 172.21.15.157:0/1191829749 shutdown_connections 2023-11-06T03:17:35.700 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.698+0000 7f2d59b4b640 1 -- 172.21.15.157:0/1191829749 wait complete. 2023-11-06T03:17:35.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 Processor -- start 2023-11-06T03:17:35.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- start start 2023-11-06T03:17:35.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d541551e0 0x7f2d541563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 0x7f2d54153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5414d500 con 0x7f2d5415e5e0 2023-11-06T03:17:35.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5414d920 con 0x7f2d541551e0 2023-11-06T03:17:35.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5414c880 con 0x7f2d5415ea90 2023-11-06T03:17:35.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55968/0 (socket says 172.21.15.157:55968) 2023-11-06T03:17:35.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 -- 172.21.15.157:0/3248491385 learned_addr learned my addr 172.21.15.157:0/3248491385 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:35.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d537fe640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d541551e0 0x7f2d541563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d53fff640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 0x7f2d54153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 -- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 msgr2=0x7f2d54153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 0x7f2d54153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 -- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d541551e0 msgr2=0x7f2d541563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d541551e0 0x7f2d541563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 -- 172.21.15.157:0/3248491385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009a70 con 0x7f2d5415e5e0 2023-11-06T03:17:35.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d53fff640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 0x7f2d54153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:35.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d52ffd640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00d800 tx=0x7f2d3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:35.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00b6c0 con 0x7f2d5415e5e0 2023-11-06T03:17:35.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c00b820 con 0x7f2d5415e5e0 2023-11-06T03:17:35.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5414cb60 con 0x7f2d5415e5e0 2023-11-06T03:17:35.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00bb90 con 0x7f2d5415e5e0 2023-11-06T03:17:35.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5414d120 con 0x7f2d5415e5e0 2023-11-06T03:17:35.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.702+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5414d640 con 0x7f2d5415e5e0 2023-11-06T03:17:35.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.706+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d3c00bcf0 con 0x7f2d5415e5e0 2023-11-06T03:17:35.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.706+0000 7f2d4bfff640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 0x7f2d34067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:35.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.706+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2d3c083570 con 0x7f2d5415e5e0 2023-11-06T03:17:35.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.706+0000 7f2d537fe640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 0x7f2d34067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:35.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.706+0000 7f2d537fe640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 0x7f2d34067f20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2d440af000 tx=0x7f2d440a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:35.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.710+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f2d5414d640 con 0x7f2d5415e5e0 2023-11-06T03:17:35.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.866+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2d5414d640 con 0x7f2d5415e5e0 2023-11-06T03:17:35.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.870+0000 7f2d4bfff640 1 -- 172.21.15.157:0/3248491385 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d5414d640 con 0x7f2d5415e5e0 2023-11-06T03:17:35.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 msgr2=0x7f2d34067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 0x7f2d34067f20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2d440af000 tx=0x7f2d440a7070 comp rx=0 tx=0).stop 2023-11-06T03:17:35.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 msgr2=0x7f2d541532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:35.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00d800 tx=0x7f2d3c00dcc0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 shutdown_connections 2023-11-06T03:17:35.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2d34065a60 0x7f2d34067f20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2d5415ea90 0x7f2d54153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5415e5e0 0x7f2d541532c0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 --2- 172.21.15.157:0/3248491385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d541551e0 0x7f2d541563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:35.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 >> 172.21.15.157:0/3248491385 conn(0x7f2d54065930 msgr2=0x7f2d540b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:35.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 shutdown_connections 2023-11-06T03:17:35.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:35.874+0000 7f2d59b4b640 1 -- 172.21.15.157:0/3248491385 wait complete. 2023-11-06T03:17:35.889 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:17:35.890 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:35.935 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:35.975 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:17:35.976 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:35.976 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:35.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:17:35.982 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:35.983 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:36.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:36.023 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:36.023 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:36.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:17:36.031 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:36.031 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:36.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:36.071 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:17:36.072 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 Processor -- start 2023-11-06T03:17:36.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 -- start start 2023-11-06T03:17:36.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e1815ccc0 0x7f7e18065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e18065e70 0x7f7e180b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e180662d0 con 0x7f7e180b0d00 2023-11-06T03:17:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e18070d30 con 0x7f7e1815ccc0 2023-11-06T03:17:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e1d66f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b1190 con 0x7f7e18065e70 2023-11-06T03:17:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e1815ccc0 0x7f7e18065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e1815ccc0 0x7f7e18065930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55974/0 (socket says 172.21.15.157:55974) 2023-11-06T03:17:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e177fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7e180b0d00 0x7f7e1806c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60002/0 (socket says 172.21.15.157:60002) 2023-11-06T03:17:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 -- 172.21.15.157:0/1762831280 learned_addr learned my addr 172.21.15.157:0/1762831280 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 -- 172.21.15.157:0/1762831280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e18065e70 msgr2=0x7f7e180b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 --2- 172.21.15.157:0/1762831280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e18065e70 0x7f7e180b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 -- 172.21.15.157:0/1762831280 >> v1:172.21.15.157:6789/0 conn(0x7f7e180b0d00 legacy=0x7f7e1806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e16ffd640 1 -- 172.21.15.157:0/1762831280 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e181606c0 con 0x7f7e1815ccc0 2023-11-06T03:17:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.202+0000 7f7e15ffb640 1 -- 172.21.15.157:0/1762831280 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069407973 0 0) 0x7f7e180662d0 con 0x7f7e180b0d00 2023-11-06T03:17:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e16ffd640 1 --2- 172.21.15.157:0/1762831280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1815ccc0 0x7f7e18065930 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498697c7a094482f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e15ffb640 1 -- 172.21.15.157:0/1762831280 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080a9400 con 0x7f7e1815ccc0 2023-11-06T03:17:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e15ffb640 1 -- 172.21.15.157:0/1762831280 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e080a9560 con 0x7f7e1815ccc0 2023-11-06T03:17:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e15ffb640 1 -- 172.21.15.157:0/1762831280 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080a98f0 con 0x7f7e1815ccc0 2023-11-06T03:17:36.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- 172.21.15.157:0/1762831280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1815ccc0 msgr2=0x7f7e18065930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/1762831280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1815ccc0 0x7f7e18065930 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0801c540 comp rx=0 tx=0).stop 2023-11-06T03:17:36.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- 172.21.15.157:0/1762831280 shutdown_connections 2023-11-06T03:17:36.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/1762831280 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e18065e70 0x7f7e180b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/1762831280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e1815ccc0 0x7f7e18065930 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- 172.21.15.157:0/1762831280 >> 172.21.15.157:0/1762831280 conn(0x7f7e180732f0 msgr2=0x7f7e18069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:36.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- 172.21.15.157:0/1762831280 shutdown_connections 2023-11-06T03:17:36.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- 172.21.15.157:0/1762831280 wait complete. 2023-11-06T03:17:36.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 Processor -- start 2023-11-06T03:17:36.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- start start 2023-11-06T03:17:36.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18065e70 0x7f7e1814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 0x7f7e180c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1814d970 con 0x7f7e180b0d00 2023-11-06T03:17:36.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180b2e10 con 0x7f7e18065e70 2023-11-06T03:17:36.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e1d66f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c9ce0 con 0x7f7e1815ccc0 2023-11-06T03:17:36.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:55982/0 (socket says 172.21.15.157:55982) 2023-11-06T03:17:36.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 -- 172.21.15.157:0/2122195515 learned_addr learned my addr 172.21.15.157:0/2122195515 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e16ffd640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18065e70 0x7f7e1814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e177fe640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 0x7f7e180c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 -- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 msgr2=0x7f7e180c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 0x7f7e180c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 -- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18065e70 msgr2=0x7f7e1814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18065e70 0x7f7e1814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 -- 172.21.15.157:0/2122195515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080094d0 con 0x7f7e180b0d00 2023-11-06T03:17:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e177fe640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 0x7f7e180c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.206+0000 7f7e167fc640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000b550 tx=0x7f7e0000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00010870 con 0x7f7e180b0d00 2023-11-06T03:17:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e000109f0 con 0x7f7e180b0d00 2023-11-06T03:17:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e180c9fc0 con 0x7f7e180b0d00 2023-11-06T03:17:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00010da0 con 0x7f7e180b0d00 2023-11-06T03:17:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1814dbc0 con 0x7f7e180b0d00 2023-11-06T03:17:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e18072750 con 0x7f7e180b0d00 2023-11-06T03:17:36.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7e0000f3c0 con 0x7f7e180b0d00 2023-11-06T03:17:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 0x7f7df8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7e00082f40 con 0x7f7e180b0d00 2023-11-06T03:17:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e16ffd640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 0x7f7df8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.210+0000 7f7e16ffd640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 0x7f7df8067f20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7e080ae000 tx=0x7f7e080b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.214+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7e00050b30 con 0x7f7e180b0d00 2023-11-06T03:17:36.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.406+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e18067fa0 con 0x7f7e180b0d00 2023-11-06T03:17:36.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.406+0000 7f7e0f7fe640 1 -- 172.21.15.157:0/2122195515 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+3542 (secure 0 0 0) 0x7f7e0001edf0 con 0x7f7e180b0d00 2023-11-06T03:17:36.405 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:36.405 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":119,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/3214418557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3214418557},{"type":"v1","addr":"172.21.15.195:6836","nonce":3214418557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":7004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/183583040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":183583040},{"type":"v1","addr":"172.21.15.195:6837","nonce":183583040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/3738507547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":3738507547},{"type":"v1","addr":"172.21.15.157:6833","nonce":3738507547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114},{"gid":10377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/694600996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":694600996},{"type":"v1","addr":"172.21.15.157:6835","nonce":694600996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[]} 2023-11-06T03:17:36.406 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 119 2023-11-06T03:17:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 msgr2=0x7f7df8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 0x7f7df8067f20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7e080ae000 tx=0x7f7e080b3040 comp rx=0 tx=0).stop 2023-11-06T03:17:36.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 msgr2=0x7f7e180b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000b550 tx=0x7f7e0000ba10 comp rx=0 tx=0).stop 2023-11-06T03:17:36.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 shutdown_connections 2023-11-06T03:17:36.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7df8065a60 0x7f7df8067f20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e1815ccc0 0x7f7e180c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e180b0d00 0x7f7e180b28d0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 --2- 172.21.15.157:0/2122195515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e18065e70 0x7f7e1814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 >> 172.21.15.157:0/2122195515 conn(0x7f7e180732f0 msgr2=0x7f7e1815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:36.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 shutdown_connections 2023-11-06T03:17:36.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.410+0000 7f7e1d66f640 1 -- 172.21.15.157:0/2122195515 wait complete. 2023-11-06T03:17:36.425 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:17:36.425 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:17:36.425 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:17:36.426 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:17:36.426 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:17:36.426 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:17:36.427 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:17:36.427 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:17:36.427 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:17:36.427 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:17:36.428 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:17:36.428 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:17:36.428 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:17:36.429 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:17:36.429 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:17:36.429 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:17:36.434 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:17:36.435 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:17:36.436 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:17:36.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:17:36.442 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:17:36.442 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:17:36.443 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:17:36.443 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:17:36.479 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:17:36.479 DEBUG:teuthology.parallel:result is None 2023-11-06T03:17:36.480 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:17:36.481 DEBUG:teuthology.parallel:result is None 2023-11-06T03:17:36.487 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:17:36.487 DEBUG:teuthology.parallel:result is None 2023-11-06T03:17:36.488 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:17:36.488 DEBUG:teuthology.parallel:result is None 2023-11-06T03:17:36.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:17:36.543 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:17:36.543 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:17:36.559 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:17:36.559 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:17:36.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.610+0000 7f15c875e640 1 Processor -- start 2023-11-06T03:17:36.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.610+0000 7f15c875e640 1 -- start start 2023-11-06T03:17:36.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 0x7f15c006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15c00701b0 0x7f15c00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00733a0 con 0x7f15c0159ce0 2023-11-06T03:17:36.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00cc8d0 con 0x7f15c00701b0 2023-11-06T03:17:36.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00730c0 con 0x7f15c0071150 2023-11-06T03:17:36.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15c00701b0 0x7f15c00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f15c00701b0 0x7f15c00c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56042/0 (socket says 172.21.15.157:56042) 2023-11-06T03:17:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 -- 172.21.15.157:0/621065905 learned_addr learned my addr 172.21.15.157:0/621065905 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c5cf9640 1 --2- 172.21.15.157:0/621065905 >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 0x7f15c006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 -- 172.21.15.157:0/621065905 >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 msgr2=0x7f15c006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:36.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 --2- 172.21.15.157:0/621065905 >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 0x7f15c006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 -- 172.21.15.157:0/621065905 >> v1:172.21.15.195:6789/0 conn(0x7f15c0159ce0 legacy=0x7f15c0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 -- 172.21.15.157:0/621065905 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00cc5f0 con 0x7f15c00701b0 2023-11-06T03:17:36.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c5cf9640 1 --2- 172.21.15.157:0/621065905 >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 0x7f15c006fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:36.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c6cfb640 1 --2- 172.21.15.157:0/621065905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c00c9600 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f15c00cb560 tx=0x7f15b801eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df027eb6bda631e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c54f8640 1 -- 172.21.15.157:0/621065905 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b800b2d0 con 0x7f15c00701b0 2023-11-06T03:17:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c54f8640 1 -- 172.21.15.157:0/621065905 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b800b430 con 0x7f15c00701b0 2023-11-06T03:17:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c54f8640 1 -- 172.21.15.157:0/621065905 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80a7c10 con 0x7f15c00701b0 2023-11-06T03:17:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 -- 172.21.15.157:0/621065905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 msgr2=0x7f15c00c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.614+0000 7f15c875e640 1 --2- 172.21.15.157:0/621065905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c00c9600 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f15c00cb560 tx=0x7f15b801eb00 comp rx=0 tx=0).stop 2023-11-06T03:17:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/621065905 shutdown_connections 2023-11-06T03:17:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 --2- 172.21.15.157:0/621065905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c00c9600 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 --2- 172.21.15.157:0/621065905 >> v2:172.21.15.195:3300/0 conn(0x7f15c0071150 0x7f15c006fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/621065905 >> 172.21.15.157:0/621065905 conn(0x7f15c00cba30 msgr2=0x7f15c00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:36.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/621065905 shutdown_connections 2023-11-06T03:17:36.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/621065905 wait complete. 2023-11-06T03:17:36.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 Processor -- start 2023-11-06T03:17:36.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- start start 2023-11-06T03:17:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c0071150 0x7f15c014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 0x7f15c0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01f4e80 con 0x7f15c00701b0 2023-11-06T03:17:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c01f4590 con 0x7f15c0159ce0 2023-11-06T03:17:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c014d660 con 0x7f15c0071150 2023-11-06T03:17:36.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56046/0 (socket says 172.21.15.157:56046) 2023-11-06T03:17:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 -- 172.21.15.157:0/1590564374 learned_addr learned my addr 172.21.15.157:0/1590564374 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c6cfb640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 0x7f15c0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c5cf9640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c0071150 0x7f15c014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 -- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c0071150 msgr2=0x7f15c014cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c0071150 0x7f15c014cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 -- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 msgr2=0x7f15c0151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 0x7f15c0151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:36.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 -- 172.21.15.157:0/1590564374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b800bd40 con 0x7f15c00701b0 2023-11-06T03:17:36.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c6cfb640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 0x7f15c0151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:36.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c64fa640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c450 tx=0x7f15b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0012830 con 0x7f15c00701b0 2023-11-06T03:17:36.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b00129b0 con 0x7f15c00701b0 2023-11-06T03:17:36.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c01f4830 con 0x7f15c00701b0 2023-11-06T03:17:36.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0012d60 con 0x7f15c00701b0 2023-11-06T03:17:36.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c01494d0 con 0x7f15c00701b0 2023-11-06T03:17:36.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.618+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c0072750 con 0x7f15c00701b0 2023-11-06T03:17:36.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.622+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f15b001c460 con 0x7f15c00701b0 2023-11-06T03:17:36.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.622+0000 7f15b6ffd640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 0x7f15a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:36.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.622+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f15b0082e20 con 0x7f15c00701b0 2023-11-06T03:17:36.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.622+0000 7f15c5cf9640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 0x7f15a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:36.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.622+0000 7f15c5cf9640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 0x7f15a4067f20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:36.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.626+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f15b0050980 con 0x7f15c00701b0 2023-11-06T03:17:36.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:36.786+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f15c00b4f30 con 0x7f15c00701b0 2023-11-06T03:17:37.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.598+0000 7f15b6ffd640 1 -- 172.21.15.157:0/1590564374 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v171) v1 ==== 97+0+0 (secure 0 0 0) 0x7f15b0053fa0 con 0x7f15c00701b0 2023-11-06T03:17:37.598 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:17:37.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 msgr2=0x7f15a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 0x7f15a4067f20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fd0 tx=0x7f15bc008040 comp rx=0 tx=0).stop 2023-11-06T03:17:37.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 msgr2=0x7f15c0148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c450 tx=0x7f15b000c910 comp rx=0 tx=0).stop 2023-11-06T03:17:37.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 shutdown_connections 2023-11-06T03:17:37.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f15a4065a60 0x7f15a4067f20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0159ce0 0x7f15c0151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f15c0071150 0x7f15c014cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 --2- 172.21.15.157:0/1590564374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15c00701b0 0x7f15c0148f90 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 >> 172.21.15.157:0/1590564374 conn(0x7f15c00cba30 msgr2=0x7f15c00cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 shutdown_connections 2023-11-06T03:17:37.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.602+0000 7f15c875e640 1 -- 172.21.15.157:0/1590564374 wait complete. 2023-11-06T03:17:37.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:17:37.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.734+0000 7fd7503bf640 1 Processor -- start 2023-11-06T03:17:37.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.734+0000 7fd7503bf640 1 -- start start 2023-11-06T03:17:37.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd7503bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd7480ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd7503bf640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806c490 con 0x7fd74806f8d0 2023-11-06T03:17:37.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd7503bf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74806bb60 con 0x7fd74806f470 2023-11-06T03:17:37.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd7503bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806b950 con 0x7fd748156a70 2023-11-06T03:17:37.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e95c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd74806f8d0 0x7fd7480b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60028/0 (socket says 172.21.15.157:60028) 2023-11-06T03:17:37.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e95c640 1 -- 172.21.15.157:0/1891734932 learned_addr learned my addr 172.21.15.157:0/1891734932 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:37.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd7480ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812664677 0 0) 0x7fd74806c490 con 0x7fd74806f8d0 2023-11-06T03:17:37.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd734005740 con 0x7fd74806f8d0 2023-11-06T03:17:37.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303849798 0 0) 0x7fd734005740 con 0x7fd74806f8d0 2023-11-06T03:17:37.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 msgr2=0x7fd7480ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:37.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd7480ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 >> v1:172.21.15.195:6789/0 conn(0x7fd74806f470 legacy=0x7fd7480adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74806be40 con 0x7fd74806f8d0 2023-11-06T03:17:37.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fd740001a00 con 0x7fd74806f8d0 2023-11-06T03:17:37.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 >> v1:172.21.15.157:6789/0 conn(0x7fd74806f8d0 legacy=0x7fd7480b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd734005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd7340069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd734006f20 0x7fd7340073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74806b950 con 0x7fd734006140 2023-11-06T03:17:37.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73400dc80 con 0x7fd734006f20 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73400de40 con 0x7fd748156a70 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd7340069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd734005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e95c640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd734006f20 0x7fd7340073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 -- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 msgr2=0x7fd734005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:37.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd734005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 -- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd734006f20 msgr2=0x7fd7340073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:37.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd734006f20 0x7fd7340073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74d95a640 1 -- 172.21.15.157:0/1891734932 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd734005740 con 0x7fd734006140 2023-11-06T03:17:37.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748156a70 0x7fd734005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:37.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.738+0000 7fd74e95c640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd734006f20 0x7fd7340073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:37.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd7340069e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fd738001780 tx=0x7fd73801c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238921b0f1e1f7ea server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:37.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380052d0 con 0x7fd734006140 2023-11-06T03:17:37.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7380a7050 con 0x7fd734006140 2023-11-06T03:17:37.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74d159640 1 -- 172.21.15.157:0/1891734932 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738003600 con 0x7fd734006140 2023-11-06T03:17:37.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1891734932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 msgr2=0x7fd7340069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1891734932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd7340069e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fd738001780 tx=0x7fd73801c220 comp rx=0 tx=0).stop 2023-11-06T03:17:37.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e95c640 1 -- 172.21.15.157:0/1891734932 reap_dead start 2023-11-06T03:17:37.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1891734932 shutdown_connections 2023-11-06T03:17:37.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1891734932 >> 172.21.15.157:0/1891734932 conn(0x7fd748065930 msgr2=0x7fd748065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:37.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1891734932 shutdown_connections 2023-11-06T03:17:37.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1891734932 wait complete. 2023-11-06T03:17:37.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 Processor -- start 2023-11-06T03:17:37.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- start start 2023-11-06T03:17:37.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748150ad0 0x7fd74815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748150f50 0x7fd74815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74814b110 con 0x7fd734006140 2023-11-06T03:17:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74814f260 con 0x7fd748150f50 2023-11-06T03:17:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd748150080 con 0x7fd748150ad0 2023-11-06T03:17:37.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56050/0 (socket says 172.21.15.157:56050) 2023-11-06T03:17:37.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 -- 172.21.15.157:0/1051839057 learned_addr learned my addr 172.21.15.157:0/1051839057 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:37.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 -- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748150ad0 msgr2=0x7fd74815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748150ad0 0x7fd74815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748150ad0 0x7fd74815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 -- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748150f50 msgr2=0x7fd74815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748150f50 0x7fd74815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 -- 172.21.15.157:0/1051839057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74814b530 con 0x7fd734006140 2023-11-06T03:17:37.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd74e15b640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fd7440108d0 tx=0x7fd744010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:37.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744004220 con 0x7fd734006140 2023-11-06T03:17:37.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744004380 con 0x7fd734006140 2023-11-06T03:17:37.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74814f470 con 0x7fd734006140 2023-11-06T03:17:37.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400eb40 con 0x7fd734006140 2023-11-06T03:17:37.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.742+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74814fa30 con 0x7fd734006140 2023-11-06T03:17:37.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748072750 con 0x7fd734006140 2023-11-06T03:17:37.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd74400eca0 con 0x7fd734006140 2023-11-06T03:17:37.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd73effd640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 0x7fd72c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:37.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd744082f70 con 0x7fd734006140 2023-11-06T03:17:37.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 0x7fd72c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:37.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.746+0000 7fd74d95a640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 0x7fd72c067f20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd738001750 tx=0x7fd7380011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:37.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.750+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd7440b4070 con 0x7fd734006140 2023-11-06T03:17:37.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.942+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd7480b0420 con 0x7fd734006140 2023-11-06T03:17:37.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.942+0000 7fd73effd640 1 -- 172.21.15.157:0/1051839057 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd744050b20 con 0x7fd734006140 2023-11-06T03:17:37.941 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:37.941 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:37.941 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:17:37.942 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.942 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:17:37.942 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:17:37.942 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.943 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.943 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:17:37.943 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:17:37.943 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.943 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.944 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.944 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.944 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:17:37.944 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:17:37.944 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.945 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.945 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:17:37.945 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:17:37.945 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.945 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.946 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.946 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.946 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:17:37.946 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:17:37.946 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:17:37.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:17:37.947 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.947 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.948 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.948 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.948 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:17:37.948 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:17:37.948 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.949 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.949 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:17:37.949 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:17:37.949 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.949 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.950 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.950 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.950 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:17:37.950 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:17:37.950 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.951 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.951 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.951 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.951 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.951 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.952 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.953 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.953 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.953 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.953 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.953 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:17:37.954 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:17:37.954 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.954 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.954 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.954 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.955 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.955 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.955 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.955 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:17:37.955 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:17:37.956 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.956 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.956 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.956 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.956 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.957 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.957 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.957 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:17:37.957 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:17:37.957 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.958 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.958 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.958 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.958 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.958 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.959 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.959 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:17:37.959 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:17:37.959 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.959 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.960 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:17:37.961 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:17:37.961 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.961 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.961 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.961 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.962 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.962 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.962 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.962 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:17:37.962 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:17:37.963 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.963 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:17:37.963 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:17:37.963 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.963 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.964 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.964 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.964 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:17:37.964 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:17:37.964 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.965 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.966 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:17:37.966 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:17:37.966 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.966 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.966 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:17:37.967 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:17:37.967 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:17:37.967 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.967 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.967 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.968 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:17:37.968 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:17:37.968 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.968 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.968 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.969 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:17:37.970 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:17:37.970 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.970 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:17:37.970 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:17:37.970 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:17:37.971 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:17:37.971 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.971 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.971 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.971 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:17:37.972 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:17:37.972 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.972 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:17:37.972 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:17:37.972 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:17:37.973 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:17:37.974 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.974 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:17:37.974 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:17:37.974 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.974 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.975 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:37.975 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:37.975 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:17:37.975 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:17:37.975 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:17:37.976 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:17:37.976 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:17:37.976 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:17:37.976 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.976 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:37.977 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:17:37.977 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 msgr2=0x7fd72c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 0x7fd72c067f20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd738001750 tx=0x7fd7380011a0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 msgr2=0x7fd74814abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fd7440108d0 tx=0x7fd744010d90 comp rx=0 tx=0).stop 2023-11-06T03:17:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 shutdown_connections 2023-11-06T03:17:37.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd72c065a60 0x7fd72c067f20 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd748150f50 0x7fd74815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd748150ad0 0x7fd74815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 --2- 172.21.15.157:0/1051839057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd734006140 0x7fd74814abd0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:37.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 >> 172.21.15.157:0/1051839057 conn(0x7fd748065930 msgr2=0x7fd748157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:37.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 shutdown_connections 2023-11-06T03:17:37.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:37.946+0000 7fd7503bf640 1 -- 172.21.15.157:0/1051839057 wait complete. 2023-11-06T03:17:37.981 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:38.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 Processor -- start 2023-11-06T03:17:38.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 -- start start 2023-11-06T03:17:38.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f803c15caa0 0x7f803c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c0d0280 0x7f803c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0d2920 con 0x7f803c15ce80 2023-11-06T03:17:38.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803c0d18a0 con 0x7f803c15caa0 2023-11-06T03:17:38.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8042d0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0d1b60 con 0x7f803c0d0280 2023-11-06T03:17:38.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f803c15caa0 0x7f803c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803bfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f803c15ce80 0x7f803c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:60032/0 (socket says 172.21.15.157:60032) 2023-11-06T03:17:38.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f803c15caa0 0x7f803c0cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56052/0 (socket says 172.21.15.157:56052) 2023-11-06T03:17:38.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803bfff640 1 -- 172.21.15.157:0/905379217 learned_addr learned my addr 172.21.15.157:0/905379217 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:38.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 -- 172.21.15.157:0/905379217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c0d0280 msgr2=0x7f803c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 --2- 172.21.15.157:0/905379217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c0d0280 0x7f803c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 -- 172.21.15.157:0/905379217 >> v1:172.21.15.157:6789/0 conn(0x7f803c15ce80 legacy=0x7f803c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 -- 172.21.15.157:0/905379217 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0d1e40 con 0x7f803c15caa0 2023-11-06T03:17:38.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803b7fe640 1 -- 172.21.15.157:0/905379217 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821171306 0 0) 0x7f803c0d2920 con 0x7f803c15ce80 2023-11-06T03:17:38.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f8040aa7640 1 --2- 172.21.15.157:0/905379217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c15caa0 0x7f803c0cdc60 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9bea62243d70b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803b7fe640 1 -- 172.21.15.157:0/905379217 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300ab650 con 0x7f803c15caa0 2023-11-06T03:17:38.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803b7fe640 1 -- 172.21.15.157:0/905379217 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80300ab7b0 con 0x7f803c15caa0 2023-11-06T03:17:38.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.090+0000 7f803b7fe640 1 -- 172.21.15.157:0/905379217 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300abb40 con 0x7f803c15caa0 2023-11-06T03:17:38.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/905379217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c15caa0 msgr2=0x7f803c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- 172.21.15.157:0/905379217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c15caa0 0x7f803c0cdc60 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001c450 comp rx=0 tx=0).stop 2023-11-06T03:17:38.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/905379217 shutdown_connections 2023-11-06T03:17:38.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- 172.21.15.157:0/905379217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c0d0280 0x7f803c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- 172.21.15.157:0/905379217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c15caa0 0x7f803c0cdc60 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/905379217 >> 172.21.15.157:0/905379217 conn(0x7f803c070790 msgr2=0x7f803c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/905379217 shutdown_connections 2023-11-06T03:17:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/905379217 wait complete. 2023-11-06T03:17:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 Processor -- start 2023-11-06T03:17:38.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- start start 2023-11-06T03:17:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c15caa0 0x7f803c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c15ce80 0x7f803c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1551c0 con 0x7f803c0d0280 2023-11-06T03:17:38.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0c9340 con 0x7f803c15caa0 2023-11-06T03:17:38.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1566c0 con 0x7f803c15ce80 2023-11-06T03:17:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56060/0 (socket says 172.21.15.157:56060) 2023-11-06T03:17:38.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 -- 172.21.15.157:0/1413717178 learned_addr learned my addr 172.21.15.157:0/1413717178 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 -- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c15ce80 msgr2=0x7f803c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f80412a8640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c15ce80 0x7f803c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f803bfff640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c15caa0 0x7f803c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c15ce80 0x7f803c156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 -- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c15caa0 msgr2=0x7f803c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c15caa0 0x7f803c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 -- 172.21.15.157:0/1413717178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009420 con 0x7f803c0d0280 2023-11-06T03:17:38.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8040aa7640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004220 con 0x7f803c0d0280 2023-11-06T03:17:38.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80300b9070 con 0x7f803c0d0280 2023-11-06T03:17:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c1553c0 con 0x7f803c0d0280 2023-11-06T03:17:38.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300ab610 con 0x7f803c0d0280 2023-11-06T03:17:38.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.094+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c155980 con 0x7f803c0d0280 2023-11-06T03:17:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c072750 con 0x7f803c0d0280 2023-11-06T03:17:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8030003620 con 0x7f803c0d0280 2023-11-06T03:17:38.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f80397fa640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 0x7f8020067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f80300bb070 con 0x7f803c0d0280 2023-11-06T03:17:38.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f803bfff640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 0x7f8020067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.098+0000 7f803bfff640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 0x7f8020067f50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f803c157ed0 tx=0x7f8028009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.102+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80300eb7d0 con 0x7f803c0d0280 2023-11-06T03:17:38.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f803c0b5520 con 0x7f803c0d0280 2023-11-06T03:17:38.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f80397fa640 1 -- 172.21.15.157:0/1413717178 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+14229 (secure 0 0 0) 0x7f80300c4020 con 0x7f803c0d0280 2023-11-06T03:17:38.252 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:38.253 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":171,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:37.595433+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:38.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 msgr2=0x7f8020067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 0x7f8020067f50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f803c157ed0 tx=0x7f8028009040 comp rx=0 tx=0).stop 2023-11-06T03:17:38.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 msgr2=0x7f803c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.254+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001ac90 comp rx=0 tx=0).stop 2023-11-06T03:17:38.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 shutdown_connections 2023-11-06T03:17:38.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8020065a90 0x7f8020067f50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f803c15ce80 0x7f803c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f803c15caa0 0x7f803c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 --2- 172.21.15.157:0/1413717178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f803c0d0280 0x7f803c154c80 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 >> 172.21.15.157:0/1413717178 conn(0x7f803c070790 msgr2=0x7f803c0b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 shutdown_connections 2023-11-06T03:17:38.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.258+0000 7f8042d0b640 1 -- 172.21.15.157:0/1413717178 wait complete. 2023-11-06T03:17:38.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:17:38.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:17:38.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.382+0000 7fc5dd52c640 1 Processor -- start 2023-11-06T03:17:38.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.382+0000 7fc5dd52c640 1 -- start start 2023-11-06T03:17:38.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5dd52c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815e5e0 0x7fc5d80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5dd52c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc5d815ea60 0x7fc5d80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5dd52c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d815f580 con 0x7fc5d81551e0 2023-11-06T03:17:38.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5dd52c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80704b0 con 0x7fc5d815ea60 2023-11-06T03:17:38.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5dd52c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815efc0 con 0x7fc5d815e5e0 2023-11-06T03:17:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc5d815ea60 0x7fc5d80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815e5e0 0x7fc5d80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc5d815ea60 0x7fc5d80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:48170/0 (socket says 172.21.15.157:48170) 2023-11-06T03:17:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 -- 172.21.15.157:0/3875938916 learned_addr learned my addr 172.21.15.157:0/3875938916 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d6ffd640 1 --1- 172.21.15.157:0/3875938916 >> v1:172.21.15.195:6789/0 conn(0x7fc5d81551e0 0x7fc5d806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:45852/0 (socket says 172.21.15.157:45852) 2023-11-06T03:17:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 -- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815e5e0 msgr2=0x7fc5d80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 --2- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815e5e0 0x7fc5d80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 -- 172.21.15.157:0/3875938916 >> v1:172.21.15.195:6789/0 conn(0x7fc5d81551e0 legacy=0x7fc5d806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.386+0000 7fc5d77fe640 1 -- 172.21.15.157:0/3875938916 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d815f2a0 con 0x7fc5d815ea60 2023-11-06T03:17:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d77fe640 1 --2- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815ea60 0x7fc5d80b3950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009e50 tx=0x7fc5d001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3068647869976949 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d67fc640 1 -- 172.21.15.157:0/3875938916 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00a5810 con 0x7fc5d815ea60 2023-11-06T03:17:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d67fc640 1 -- 172.21.15.157:0/3875938916 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d00a5970 con 0x7fc5d815ea60 2023-11-06T03:17:38.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d67fc640 1 -- 172.21.15.157:0/3875938916 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00a5d00 con 0x7fc5d815ea60 2023-11-06T03:17:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815ea60 msgr2=0x7fc5d80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815ea60 0x7fc5d80b3950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009e50 tx=0x7fc5d001c950 comp rx=0 tx=0).stop 2023-11-06T03:17:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/3875938916 shutdown_connections 2023-11-06T03:17:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815ea60 0x7fc5d80b3950 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/3875938916 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815e5e0 0x7fc5d80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/3875938916 >> 172.21.15.157:0/3875938916 conn(0x7fc5d8065930 msgr2=0x7fc5d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/3875938916 shutdown_connections 2023-11-06T03:17:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/3875938916 wait complete. 2023-11-06T03:17:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 Processor -- start 2023-11-06T03:17:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- start start 2023-11-06T03:17:38.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815e5e0 0x7fc5d80afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815ea60 0x7fc5d814a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8135e00 con 0x7fc5d81551e0 2023-11-06T03:17:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81351f0 con 0x7fc5d815e5e0 2023-11-06T03:17:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81354d0 con 0x7fc5d815ea60 2023-11-06T03:17:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56062/0 (socket says 172.21.15.157:56062) 2023-11-06T03:17:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 -- 172.21.15.157:0/2406876663 learned_addr learned my addr 172.21.15.157:0/2406876663 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5cffff640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815e5e0 0x7fc5d80afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d77fe640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815ea60 0x7fc5d814a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 -- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815ea60 msgr2=0x7fc5d814a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815ea60 0x7fc5d814a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 -- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815e5e0 msgr2=0x7fc5d80afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815e5e0 0x7fc5d80afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 -- 172.21.15.157:0/2406876663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0009b30 con 0x7fc5d81551e0 2023-11-06T03:17:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5d6ffd640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c420 tx=0x7fc5c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80074e0 con 0x7fc5d81551e0 2023-11-06T03:17:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8007640 con 0x7fc5d81551e0 2023-11-06T03:17:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d81357b0 con 0x7fc5d81551e0 2023-11-06T03:17:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004a30 con 0x7fc5d81551e0 2023-11-06T03:17:38.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.390+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d8134e80 con 0x7fc5d81551e0 2023-11-06T03:17:38.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8072750 con 0x7fc5d81551e0 2023-11-06T03:17:38.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5c8004b90 con 0x7fc5d81551e0 2023-11-06T03:17:38.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5cf7fe640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 0x7fc5b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc5c8082a40 con 0x7fc5d81551e0 2023-11-06T03:17:38.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5cffff640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 0x7fc5b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.394+0000 7fc5cffff640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 0x7fc5b8067f20 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8132cc0 tx=0x7fc5c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.398+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc5c8084e60 con 0x7fc5d81551e0 2023-11-06T03:17:38.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.538+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5d8071730 con 0x7fc5d81551e0 2023-11-06T03:17:38.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5cf7fe640 1 -- 172.21.15.157:0/2406876663 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5c8054090 con 0x7fc5d81551e0 2023-11-06T03:17:38.538 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:17:38.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 msgr2=0x7fc5b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 0x7fc5b8067f20 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8132cc0 tx=0x7fc5c0008040 comp rx=0 tx=0).stop 2023-11-06T03:17:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 msgr2=0x7fc5d80af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c420 tx=0x7fc5c800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 shutdown_connections 2023-11-06T03:17:38.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc5b8065a60 0x7fc5b8067f20 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5d815ea60 0x7fc5d814a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5d815e5e0 0x7fc5d80afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 --2- 172.21.15.157:0/2406876663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d81551e0 0x7fc5d80af5d0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 >> 172.21.15.157:0/2406876663 conn(0x7fc5d8065930 msgr2=0x7fc5d80b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 shutdown_connections 2023-11-06T03:17:38.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.542+0000 7fc5dd52c640 1 -- 172.21.15.157:0/2406876663 wait complete. 2023-11-06T03:17:38.556 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:17:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.666+0000 7f998edab640 1 Processor -- start 2023-11-06T03:17:38.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.666+0000 7f998edab640 1 -- start start 2023-11-06T03:17:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.666+0000 7f998edab640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99880cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.666+0000 7f998edab640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880d2920 con 0x7f99880d0280 2023-11-06T03:17:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998edab640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880d18a0 con 0x7f998815ce80 2023-11-06T03:17:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998edab640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99880d1b60 con 0x7f998815caa0 2023-11-06T03:17:38.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998d348640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f99880d0280 0x7f99880b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51010/0 (socket says 172.21.15.157:51010) 2023-11-06T03:17:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998d348640 1 -- 172.21.15.157:0/2362502079 learned_addr learned my addr 172.21.15.157:0/2362502079 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764501553 0 0) 0x7f99880d2920 con 0x7f99880d0280 2023-11-06T03:17:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9978003530 con 0x7f99880d0280 2023-11-06T03:17:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99880cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361261896 0 0) 0x7f9978003530 con 0x7f99880d0280 2023-11-06T03:17:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 msgr2=0x7f99880cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99880cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 >> v1:172.21.15.195:6789/0 conn(0x7f998815ce80 legacy=0x7f99880cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880d1e40 con 0x7f99880d0280 2023-11-06T03:17:38.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9980001a00 con 0x7f99880d0280 2023-11-06T03:17:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 >> v1:172.21.15.157:6789/0 conn(0x7f99880d0280 legacy=0x7f99880b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9978004d10 0x7f9978005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99880d1b60 con 0x7f99780039b0 2023-11-06T03:17:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997800bb20 con 0x7f9978004d10 2023-11-06T03:17:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997800bcb0 con 0x7f998815caa0 2023-11-06T03:17:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 -- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 msgr2=0x7f99780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f9987fff640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998d348640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9978004d10 0x7f9978005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 -- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9978004d10 msgr2=0x7f9978005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9978004d10 0x7f9978005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 -- 172.21.15.157:0/2362502079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978003530 con 0x7f99780039b0 2023-11-06T03:17:38.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f9987fff640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f998815caa0 0x7f99780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:38.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f998cb47640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f997c00ba60 tx=0x7f997c01c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7291694d8a6940 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0ab620 con 0x7f99780039b0 2023-11-06T03:17:38.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c0ab780 con 0x7f99780039b0 2023-11-06T03:17:38.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.670+0000 7f99877fe640 1 -- 172.21.15.157:0/2362502079 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0abb30 con 0x7f99780039b0 2023-11-06T03:17:38.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/2362502079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780039b0 msgr2=0x7f9978003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 --2- 172.21.15.157:0/2362502079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99780039b0 0x7f9978003dd0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f997c00ba60 tx=0x7f997c01c630 comp rx=0 tx=0).stop 2023-11-06T03:17:38.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998d348640 1 -- 172.21.15.157:0/2362502079 reap_dead start 2023-11-06T03:17:38.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/2362502079 shutdown_connections 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/2362502079 >> 172.21.15.157:0/2362502079 conn(0x7f9988070790 msgr2=0x7f9988070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/2362502079 shutdown_connections 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/2362502079 wait complete. 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 Processor -- start 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- start start 2023-11-06T03:17:38.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 0x7f998815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 0x7f998815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9988157e40 con 0x7f99881f5040 2023-11-06T03:17:38.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998815e9c0 con 0x7f998815dc40 2023-11-06T03:17:38.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998815eca0 con 0x7f99881f4c60 2023-11-06T03:17:38.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51782/0 (socket says 172.21.15.157:51782) 2023-11-06T03:17:38.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 -- 172.21.15.157:0/119397755 learned_addr learned my addr 172.21.15.157:0/119397755 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998cb47640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 0x7f998815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998d348640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 0x7f998815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 -- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 msgr2=0x7f998815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 0x7f998815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 -- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 msgr2=0x7f998815e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 0x7f998815e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 -- 172.21.15.157:0/119397755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880c9cf0 con 0x7f99881f5040 2023-11-06T03:17:38.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998cb47640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 0x7f998815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:38.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998d348640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 0x7f998815e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:38.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f9987fff640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f997400d770 tx=0x7f997400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005270 con 0x7f99881f5040 2023-11-06T03:17:38.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99740053d0 con 0x7f99881f5040 2023-11-06T03:17:38.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99880c9fd0 con 0x7f99881f5040 2023-11-06T03:17:38.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400b8f0 con 0x7f99881f5040 2023-11-06T03:17:38.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.674+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998815e630 con 0x7f99881f5040 2023-11-06T03:17:38.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9988072750 con 0x7f99881f5040 2023-11-06T03:17:38.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f997400ba50 con 0x7f99881f5040 2023-11-06T03:17:38.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f99857fa640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 0x7f996c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9974082e50 con 0x7f99881f5040 2023-11-06T03:17:38.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f998cb47640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 0x7f996c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.678+0000 7f998cb47640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 0x7f996c067f50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f997c0af000 tx=0x7f997c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.682+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9974050a00 con 0x7f99881f5040 2023-11-06T03:17:38.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.822+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99880b88b0 con 0x7f99881f5040 2023-11-06T03:17:38.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f99857fa640 1 -- 172.21.15.157:0/119397755 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v171) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9974019310 con 0x7f99881f5040 2023-11-06T03:17:38.822 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:17:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 msgr2=0x7f996c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 0x7f996c067f50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f997c0af000 tx=0x7f997c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 msgr2=0x7f998815f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f997400d770 tx=0x7f997400dc30 comp rx=0 tx=0).stop 2023-11-06T03:17:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 shutdown_connections 2023-11-06T03:17:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f996c065a90 0x7f996c067f50 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f998815dc40 0x7f998815e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99881f5040 0x7f998815f6f0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 --2- 172.21.15.157:0/119397755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99881f4c60 0x7f998815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 >> 172.21.15.157:0/119397755 conn(0x7f9988070790 msgr2=0x7f99880d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 shutdown_connections 2023-11-06T03:17:38.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.826+0000 7f998edab640 1 -- 172.21.15.157:0/119397755 wait complete. 2023-11-06T03:17:38.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:17:38.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 Processor -- start 2023-11-06T03:17:38.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 -- start start 2023-11-06T03:17:38.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fefbc146370 0x7fefbc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefbc06bdb0 0x7fefbc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc0cf1a0 con 0x7fefbc0ce800 2023-11-06T03:17:38.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc06c3d0 con 0x7fefbc06bdb0 2023-11-06T03:17:38.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefc1015640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc06f5a0 con 0x7fefbc146370 2023-11-06T03:17:38.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefbc06bdb0 0x7fefbc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefbc06bdb0 0x7fefbc0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51790/0 (socket says 172.21.15.157:51790) 2023-11-06T03:17:38.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 -- 172.21.15.157:0/410478659 learned_addr learned my addr 172.21.15.157:0/410478659 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbad9d640 1 --2- 172.21.15.157:0/410478659 >> v2:172.21.15.195:3300/0 conn(0x7fefbc146370 0x7fefbc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 -- 172.21.15.157:0/410478659 >> v2:172.21.15.195:3300/0 conn(0x7fefbc146370 msgr2=0x7fefbc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 --2- 172.21.15.157:0/410478659 >> v2:172.21.15.195:3300/0 conn(0x7fefbc146370 0x7fefbc0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 -- 172.21.15.157:0/410478659 >> v1:172.21.15.195:6789/0 conn(0x7fefbc0ce800 legacy=0x7fefbc06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:38.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.954+0000 7fefbb59e640 1 -- 172.21.15.157:0/410478659 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc06f880 con 0x7fefbc06bdb0 2023-11-06T03:17:38.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefbb59e640 1 --2- 172.21.15.157:0/410478659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc06bdb0 0x7fefbc0cec60 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009e50 tx=0x7fefb401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13096eb554613625 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefb9d9b640 1 -- 172.21.15.157:0/410478659 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40a5790 con 0x7fefbc06bdb0 2023-11-06T03:17:38.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefb9d9b640 1 -- 172.21.15.157:0/410478659 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb40a58f0 con 0x7fefbc06bdb0 2023-11-06T03:17:38.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefb9d9b640 1 -- 172.21.15.157:0/410478659 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40a5c80 con 0x7fefbc06bdb0 2023-11-06T03:17:38.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- 172.21.15.157:0/410478659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc06bdb0 msgr2=0x7fefbc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 --2- 172.21.15.157:0/410478659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc06bdb0 0x7fefbc0cec60 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009e50 tx=0x7fefb401c8d0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- 172.21.15.157:0/410478659 shutdown_connections 2023-11-06T03:17:38.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 --2- 172.21.15.157:0/410478659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc06bdb0 0x7fefbc0cec60 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 --2- 172.21.15.157:0/410478659 >> v2:172.21.15.195:3300/0 conn(0x7fefbc146370 0x7fefbc0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- 172.21.15.157:0/410478659 >> 172.21.15.157:0/410478659 conn(0x7fefbc065930 msgr2=0x7fefbc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:38.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- 172.21.15.157:0/410478659 shutdown_connections 2023-11-06T03:17:38.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- 172.21.15.157:0/410478659 wait complete. 2023-11-06T03:17:38.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 Processor -- start 2023-11-06T03:17:38.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.958+0000 7fefc1015640 1 -- start start 2023-11-06T03:17:38.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc06bdb0 0x7fefbc15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc146370 0x7fefbc1449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc15cf90 con 0x7fefbc0ce800 2023-11-06T03:17:38.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc15e3c0 con 0x7fefbc06bdb0 2023-11-06T03:17:38.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc15d7f0 con 0x7fefbc146370 2023-11-06T03:17:38.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51798/0 (socket says 172.21.15.157:51798) 2023-11-06T03:17:38.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 -- 172.21.15.157:0/3457134219 learned_addr learned my addr 172.21.15.157:0/3457134219 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:38.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefbad9d640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc06bdb0 0x7fefbc15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 -- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc146370 msgr2=0x7fefbc1449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefbb59e640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc146370 0x7fefbc1449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc146370 0x7fefbc1449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 -- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc06bdb0 msgr2=0x7fefbc15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:38.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc06bdb0 0x7fefbc15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:38.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 -- 172.21.15.157:0/3457134219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4009b30 con 0x7fefbc0ce800 2023-11-06T03:17:38.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefba59c640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b440 tx=0x7fefb000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0011890 con 0x7fefbc0ce800 2023-11-06T03:17:38.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb00119f0 con 0x7fefbc0ce800 2023-11-06T03:17:38.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc15d230 con 0x7fefbc0ce800 2023-11-06T03:17:38.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0011da0 con 0x7fefbc0ce800 2023-11-06T03:17:38.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc15dbc0 con 0x7fefbc0ce800 2023-11-06T03:17:38.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc072750 con 0x7fefbc0ce800 2023-11-06T03:17:38.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefb001a400 con 0x7fefbc0ce800 2023-11-06T03:17:38.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 0x7fefa8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:38.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fefb0082fc0 con 0x7fefbc0ce800 2023-11-06T03:17:38.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.962+0000 7fefbad9d640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 0x7fefa8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:38.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.966+0000 7fefbad9d640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 0x7fefa8067f20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fefac00b9d0 tx=0x7fefac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:38.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:38.966+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fefb00b4070 con 0x7fefbc0ce800 2023-11-06T03:17:39.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.110+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fefbc065f20 con 0x7fefbc0ce800 2023-11-06T03:17:39.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.110+0000 7fefa37fe640 1 -- 172.21.15.157:0/3457134219 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v171) v1 ==== 84+0+126 (secure 0 0 0) 0x7fefb0015090 con 0x7fefbc0ce800 2023-11-06T03:17:39.110 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:39.111 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:17:39.111 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:39.111 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:17:39.111 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:17:39.112 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:39.112 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:17:39.112 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:17:39.112 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:17:39.112 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:39.113 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:17:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 msgr2=0x7fefa8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 0x7fefa8067f20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fefac00b9d0 tx=0x7fefac008040 comp rx=0 tx=0).stop 2023-11-06T03:17:39.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 msgr2=0x7fefbc13faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b440 tx=0x7fefb000b900 comp rx=0 tx=0).stop 2023-11-06T03:17:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 shutdown_connections 2023-11-06T03:17:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefa8065a60 0x7fefa8067f20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefbc146370 0x7fefbc1449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefbc0ce800 0x7fefbc13faa0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 --2- 172.21.15.157:0/3457134219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefbc06bdb0 0x7fefbc15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 >> 172.21.15.157:0/3457134219 conn(0x7fefbc065930 msgr2=0x7fefbc070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:39.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 shutdown_connections 2023-11-06T03:17:39.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.114+0000 7fefc1015640 1 -- 172.21.15.157:0/3457134219 wait complete. 2023-11-06T03:17:39.127 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:17:39.128 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:17:39.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 Processor -- start 2023-11-06T03:17:39.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 -- start start 2023-11-06T03:17:39.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f112806c170 0x7f1128154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f112815b860 0x7f1128155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11280cb100 con 0x7f112806c550 2023-11-06T03:17:39.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112815bcf0 con 0x7f112815b860 2023-11-06T03:17:39.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112ec4d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280ce0a0 con 0x7f112806c170 2023-11-06T03:17:39.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f1127fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f112806c550 0x7f112815b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51018/0 (socket says 172.21.15.157:51018) 2023-11-06T03:17:39.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112d1ea640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f112815b860 0x7f1128155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f1127fff640 1 -- 172.21.15.157:0/670655733 learned_addr learned my addr 172.21.15.157:0/670655733 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:39.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112c9e9640 1 --2- 172.21.15.157:0/670655733 >> v2:172.21.15.195:3300/0 conn(0x7f112806c170 0x7f1128154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112d1ea640 1 -- 172.21.15.157:0/670655733 >> v2:172.21.15.195:3300/0 conn(0x7f112806c170 msgr2=0x7f1128154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:39.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112d1ea640 1 --2- 172.21.15.157:0/670655733 >> v2:172.21.15.195:3300/0 conn(0x7f112806c170 0x7f1128154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112d1ea640 1 -- 172.21.15.157:0/670655733 >> v1:172.21.15.157:6789/0 conn(0x7f112806c550 legacy=0x7f112815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f11277fe640 1 -- 172.21.15.157:0/670655733 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483513849 0 0) 0x7f11280cb100 con 0x7f112806c550 2023-11-06T03:17:39.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.246+0000 7f112d1ea640 1 -- 172.21.15.157:0/670655733 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112815bef0 con 0x7f112815b860 2023-11-06T03:17:39.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112d1ea640 1 --2- 172.21.15.157:0/670655733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112815b860 0x7f1128155140 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f1128066460 tx=0x7f112001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4321f50ac8367ffe server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f11277fe640 1 -- 172.21.15.157:0/670655733 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200a59a0 con 0x7f112815b860 2023-11-06T03:17:39.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f11277fe640 1 -- 172.21.15.157:0/670655733 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11200a5b00 con 0x7f112815b860 2023-11-06T03:17:39.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f11277fe640 1 -- 172.21.15.157:0/670655733 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200a5e90 con 0x7f112815b860 2023-11-06T03:17:39.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- 172.21.15.157:0/670655733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112815b860 msgr2=0x7f1128155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- 172.21.15.157:0/670655733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112815b860 0x7f1128155140 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f1128066460 tx=0x7f112001c8e0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- 172.21.15.157:0/670655733 shutdown_connections 2023-11-06T03:17:39.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- 172.21.15.157:0/670655733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112815b860 0x7f1128155140 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- 172.21.15.157:0/670655733 >> v2:172.21.15.195:3300/0 conn(0x7f112806c170 0x7f1128154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- 172.21.15.157:0/670655733 >> 172.21.15.157:0/670655733 conn(0x7f11280b43b0 msgr2=0x7f1128065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:39.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- 172.21.15.157:0/670655733 shutdown_connections 2023-11-06T03:17:39.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- 172.21.15.157:0/670655733 wait complete. 2023-11-06T03:17:39.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 Processor -- start 2023-11-06T03:17:39.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- start start 2023-11-06T03:17:39.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112806c170 0x7f1128152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112815b860 0x7f11280acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112815d560 con 0x7f112806c550 2023-11-06T03:17:39.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112815cd50 con 0x7f112815b860 2023-11-06T03:17:39.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f112ec4d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112814e3f0 con 0x7f112806c170 2023-11-06T03:17:39.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f1127fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.250+0000 7f1127fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51808/0 (socket says 172.21.15.157:51808) 2023-11-06T03:17:39.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 -- 172.21.15.157:0/939276216 learned_addr learned my addr 172.21.15.157:0/939276216 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:39.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112d1ea640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112815b860 0x7f11280acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 -- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112806c170 msgr2=0x7f1128152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112806c170 0x7f1128152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 -- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112815b860 msgr2=0x7f11280acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112815b860 0x7f11280acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 -- 172.21.15.157:0/939276216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009a70 con 0x7f112806c550 2023-11-06T03:17:39.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f1127fff640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c520 tx=0x7f111c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0074e0 con 0x7f112806c550 2023-11-06T03:17:39.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c007640 con 0x7f112806c550 2023-11-06T03:17:39.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11280cbfb0 con 0x7f112806c550 2023-11-06T03:17:39.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0049f0 con 0x7f112806c550 2023-11-06T03:17:39.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112815cfa0 con 0x7f112806c550 2023-11-06T03:17:39.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112804e050 con 0x7f112806c550 2023-11-06T03:17:39.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f111c004b50 con 0x7f112806c550 2023-11-06T03:17:39.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 0x7f1114067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f111c0825f0 con 0x7f112806c550 2023-11-06T03:17:39.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112c9e9640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 0x7f1114067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.254+0000 7f112c9e9640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 0x7f1114067f20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f112815c2e0 tx=0x7f1118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.262+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f111c0501a0 con 0x7f112806c550 2023-11-06T03:17:39.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.402+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f11280ccbb0 con 0x7f112806c550 2023-11-06T03:17:39.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.574+0000 7f11257fa640 1 -- 172.21.15.157:0/939276216 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v172) v1 ==== 138+0+0 (secure 0 0 0) 0x7f111c018020 con 0x7f112806c550 2023-11-06T03:17:39.571 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:17:39.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 msgr2=0x7f1114067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 0x7f1114067f20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f112815c2e0 tx=0x7f1118008040 comp rx=0 tx=0).stop 2023-11-06T03:17:39.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 msgr2=0x7f11280ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c520 tx=0x7f111c00c9e0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 shutdown_connections 2023-11-06T03:17:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1114065a60 0x7f1114067f20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f112815b860 0x7f11280acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f112806c550 0x7f11280ac8b0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 --2- 172.21.15.157:0/939276216 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112806c170 0x7f1128152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 >> 172.21.15.157:0/939276216 conn(0x7f11280b43b0 msgr2=0x7f1128065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:39.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 shutdown_connections 2023-11-06T03:17:39.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.582+0000 7f112ec4d640 1 -- 172.21.15.157:0/939276216 wait complete. 2023-11-06T03:17:39.599 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:17:39.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 Processor -- start 2023-11-06T03:17:39.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 -- start start 2023-11-06T03:17:39.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 0x7efd300cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015ce80 0x7efd300cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd300ce1a0 0x7efd300d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300d2920 con 0x7efd300ce1a0 2023-11-06T03:17:39.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300d18a0 con 0x7efd3015caa0 2023-11-06T03:17:39.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd37a7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300d1b60 con 0x7efd3015ce80 2023-11-06T03:17:39.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd300ce1a0 0x7efd300d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7efd300ce1a0 0x7efd300d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51810/0 (socket says 172.21.15.157:51810) 2023-11-06T03:17:39.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 -- 172.21.15.157:0/3193707168 learned_addr learned my addr 172.21.15.157:0/3193707168 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:39.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd35817640 1 --2- 172.21.15.157:0/3193707168 >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 0x7efd300cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd35016640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015ce80 0x7efd300cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 -- 172.21.15.157:0/3193707168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015ce80 msgr2=0x7efd300cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:39.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015ce80 0x7efd300cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 -- 172.21.15.157:0/3193707168 >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 msgr2=0x7efd300cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:39.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 --2- 172.21.15.157:0/3193707168 >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 0x7efd300cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd36018640 1 -- 172.21.15.157:0/3193707168 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300d1e40 con 0x7efd300ce1a0 2023-11-06T03:17:39.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.730+0000 7efd35817640 1 --2- 172.21.15.157:0/3193707168 >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 0x7efd300cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:39.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd36018640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd300d00b0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7efd300b4f80 tx=0x7efd2801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819da712a4131358 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd34815640 1 -- 172.21.15.157:0/3193707168 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd280a5820 con 0x7efd300ce1a0 2023-11-06T03:17:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd34815640 1 -- 172.21.15.157:0/3193707168 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd280a5980 con 0x7efd300ce1a0 2023-11-06T03:17:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd34815640 1 -- 172.21.15.157:0/3193707168 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd280a5d10 con 0x7efd300ce1a0 2023-11-06T03:17:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- 172.21.15.157:0/3193707168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 msgr2=0x7efd300d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd300d00b0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7efd300b4f80 tx=0x7efd2801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- 172.21.15.157:0/3193707168 shutdown_connections 2023-11-06T03:17:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd300d00b0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 --2- 172.21.15.157:0/3193707168 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015ce80 0x7efd300cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 --2- 172.21.15.157:0/3193707168 >> v2:172.21.15.195:3300/0 conn(0x7efd3015caa0 0x7efd300cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- 172.21.15.157:0/3193707168 >> 172.21.15.157:0/3193707168 conn(0x7efd30070790 msgr2=0x7efd30070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- 172.21.15.157:0/3193707168 shutdown_connections 2023-11-06T03:17:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- 172.21.15.157:0/3193707168 wait complete. 2023-11-06T03:17:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 Processor -- start 2023-11-06T03:17:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.734+0000 7efd37a7b640 1 -- start start 2023-11-06T03:17:39.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015caa0 0x7efd300b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd3015ce80 0x7efd300b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015b930 con 0x7efd300ce1a0 2023-11-06T03:17:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015aa80 con 0x7efd3015ce80 2023-11-06T03:17:39.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015ad20 con 0x7efd3015caa0 2023-11-06T03:17:39.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51818/0 (socket says 172.21.15.157:51818) 2023-11-06T03:17:39.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 -- 172.21.15.157:0/2350963322 learned_addr learned my addr 172.21.15.157:0/2350963322 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:39.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd36018640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd3015ce80 0x7efd300b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35016640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015caa0 0x7efd300b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 -- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015caa0 msgr2=0x7efd300b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015caa0 0x7efd300b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 -- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd3015ce80 msgr2=0x7efd300b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:39.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd3015ce80 0x7efd300b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:39.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 -- 172.21.15.157:0/2350963322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009b30 con 0x7efd300ce1a0 2023-11-06T03:17:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd35817640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c7c0 tx=0x7efd2c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c007a30 con 0x7efd300ce1a0 2023-11-06T03:17:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c007b90 con 0x7efd300ce1a0 2023-11-06T03:17:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3015b000 con 0x7efd300ce1a0 2023-11-06T03:17:39.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00ce70 con 0x7efd300ce1a0 2023-11-06T03:17:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3015b5c0 con 0x7efd300ce1a0 2023-11-06T03:17:39.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30072750 con 0x7efd300ce1a0 2023-11-06T03:17:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd2c0113f0 con 0x7efd300ce1a0 2023-11-06T03:17:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 0x7efd14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.738+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5580+0+0 (secure 0 0 0) 0x7efd2c083320 con 0x7efd300ce1a0 2023-11-06T03:17:39.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.742+0000 7efd35016640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 0x7efd14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:39.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.742+0000 7efd35016640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 0x7efd14067f20 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7efd301f8300 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:39.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.746+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efd2c050cd0 con 0x7efd300ce1a0 2023-11-06T03:17:39.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:39.898+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efd30156590 con 0x7efd300ce1a0 2023-11-06T03:17:40.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.586+0000 7efd267fc640 1 -- 172.21.15.157:0/2350963322 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v173) v1 ==== 170+0+0 (secure 0 0 0) 0x7efd2c0542f0 con 0x7efd300ce1a0 2023-11-06T03:17:40.588 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:17:40.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 msgr2=0x7efd14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 0x7efd14067f20 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7efd301f8300 tx=0x7efd20008040 comp rx=0 tx=0).stop 2023-11-06T03:17:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 msgr2=0x7efd3015a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c7c0 tx=0x7efd2c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 shutdown_connections 2023-11-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7efd14065a60 0x7efd14067f20 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd3015ce80 0x7efd300b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efd3015caa0 0x7efd300b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 --2- 172.21.15.157:0/2350963322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd300ce1a0 0x7efd3015a540 secure :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00c7c0 tx=0x7efd2c00cc80 comp rx=0 tx=0).stop 2023-11-06T03:17:40.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.598+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 >> 172.21.15.157:0/2350963322 conn(0x7efd30070790 msgr2=0x7efd3015c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:40.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.602+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 shutdown_connections 2023-11-06T03:17:40.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.606+0000 7efd37a7b640 1 -- 172.21.15.157:0/2350963322 wait complete. 2023-11-06T03:17:40.630 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:17:40.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 Processor -- start 2023-11-06T03:17:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 -- start start 2023-11-06T03:17:40.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f80a80b3b20 0x7f80a815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a815f580 con 0x7f80a815e5e0 2023-11-06T03:17:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a815efc0 con 0x7f80a80b3b20 2023-11-06T03:17:40.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80ac8c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a815f2a0 con 0x7f80a81551e0 2023-11-06T03:17:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a6d9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f80a80b3b20 0x7f80a815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37888/0 (socket says 172.21.15.157:37888) 2023-11-06T03:17:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f809ffff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f80a815e5e0 0x7f80a80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:44070/0 (socket says 172.21.15.157:44070) 2023-11-06T03:17:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 -- 172.21.15.157:0/2428734504 learned_addr learned my addr 172.21.15.157:0/2428734504 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:40.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a5d9b640 1 -- 172.21.15.157:0/2428734504 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958823489 0 0) 0x7f80a815f580 con 0x7f80a815e5e0 2023-11-06T03:17:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a5d9b640 1 -- 172.21.15.157:0/2428734504 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8088003530 con 0x7f80a815e5e0 2023-11-06T03:17:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 -- 172.21.15.157:0/2428734504 >> v2:172.21.15.195:3300/0 conn(0x7f80a80b3b20 msgr2=0x7f80a815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 --2- 172.21.15.157:0/2428734504 >> v2:172.21.15.195:3300/0 conn(0x7f80a80b3b20 0x7f80a815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 -- 172.21.15.157:0/2428734504 >> v1:172.21.15.195:6789/0 conn(0x7f80a815e5e0 legacy=0x7f80a80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.754+0000 7f80a659c640 1 -- 172.21.15.157:0/2428734504 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a806f5a0 con 0x7f80a81551e0 2023-11-06T03:17:40.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80a659c640 1 --2- 172.21.15.157:0/2428734504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f809001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0dc1e2b4313bdae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80a5d9b640 1 -- 172.21.15.157:0/2428734504 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80900ad640 con 0x7f80a81551e0 2023-11-06T03:17:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80a5d9b640 1 -- 172.21.15.157:0/2428734504 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80900ad7a0 con 0x7f80a81551e0 2023-11-06T03:17:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80a5d9b640 1 -- 172.21.15.157:0/2428734504 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80900adb10 con 0x7f80a81551e0 2023-11-06T03:17:40.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 -- 172.21.15.157:0/2428734504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 msgr2=0x7f80a806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/2428734504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8090009870 tx=0x7f809001c480 comp rx=0 tx=0).stop 2023-11-06T03:17:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 -- 172.21.15.157:0/2428734504 shutdown_connections 2023-11-06T03:17:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/2428734504 >> v2:172.21.15.195:3300/0 conn(0x7f80a80b3b20 0x7f80a815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/2428734504 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a806fda0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 -- 172.21.15.157:0/2428734504 >> 172.21.15.157:0/2428734504 conn(0x7f80a8065930 msgr2=0x7f80a8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 -- 172.21.15.157:0/2428734504 shutdown_connections 2023-11-06T03:17:40.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 -- 172.21.15.157:0/2428734504 wait complete. 2023-11-06T03:17:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.758+0000 7f80ac8c4640 1 Processor -- start 2023-11-06T03:17:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- start start 2023-11-06T03:17:40.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80a80b3b20 0x7f80a80aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a80cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8155660 con 0x7f80a815e5e0 2023-11-06T03:17:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8155940 con 0x7f80a80b3b20 2023-11-06T03:17:40.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8155c20 con 0x7f80a81551e0 2023-11-06T03:17:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51824/0 (socket says 172.21.15.157:51824) 2023-11-06T03:17:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 -- 172.21.15.157:0/4053337352 learned_addr learned my addr 172.21.15.157:0/4053337352 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a659c640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80a80b3b20 0x7f80a80aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f809ffff640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a80cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 -- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 msgr2=0x7f80a80cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a80cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 -- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80a80b3b20 msgr2=0x7f80a80aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80a80b3b20 0x7f80a80aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 -- 172.21.15.157:0/4053337352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900094d0 con 0x7f80a815e5e0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80a6d9d640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f80a000efc0 tx=0x7f80a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0009ca0 con 0x7f80a815e5e0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000f040 con 0x7f80a815e5e0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a8155f00 con 0x7f80a815e5e0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00146c0 con 0x7f80a815e5e0 2023-11-06T03:17:40.765 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a80af5f0 con 0x7f80a815e5e0 2023-11-06T03:17:40.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a8072750 con 0x7f80a815e5e0 2023-11-06T03:17:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.762+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80a0014820 con 0x7f80a815e5e0 2023-11-06T03:17:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.766+0000 7f809effd640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 0x7f808c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.766+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f80a00846d0 con 0x7f80a815e5e0 2023-11-06T03:17:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.766+0000 7f80a659c640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 0x7f808c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:40.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.766+0000 7f80a659c640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 0x7f808c067f20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f80900af000 tx=0x7f8090009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:40.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.770+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f80a0052080 con 0x7f80a815e5e0 2023-11-06T03:17:40.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.958+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f80a8156480 con 0x7f80a815e5e0 2023-11-06T03:17:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.974+0000 7f809effd640 1 -- 172.21.15.157:0/4053337352 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v121) v1 ==== 174+0+0 (secure 0 0 0) 0x7f80a00556a0 con 0x7f80a815e5e0 2023-11-06T03:17:40.970 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 25 and data pool 26 2023-11-06T03:17:40.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 msgr2=0x7f808c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 0x7f808c067f20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f80900af000 tx=0x7f8090009040 comp rx=0 tx=0).stop 2023-11-06T03:17:40.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 msgr2=0x7f80a8158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:40.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f80a000efc0 tx=0x7f80a000c490 comp rx=0 tx=0).stop 2023-11-06T03:17:40.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 shutdown_connections 2023-11-06T03:17:40.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f808c065a60 0x7f808c067f20 secure :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f80900af000 tx=0x7f8090009040 comp rx=0 tx=0).stop 2023-11-06T03:17:40.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a815e5e0 0x7f80a8158540 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f80a81551e0 0x7f80a80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 --2- 172.21.15.157:0/4053337352 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f80a80b3b20 0x7f80a80aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:40.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.978+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 >> 172.21.15.157:0/4053337352 conn(0x7f80a8065930 msgr2=0x7f80a80b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:40.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.986+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 shutdown_connections 2023-11-06T03:17:40.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:40.990+0000 7f80ac8c4640 1 -- 172.21.15.157:0/4053337352 wait complete. 2023-11-06T03:17:41.013 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:41.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.130+0000 7ff0cb9b5640 1 Processor -- start 2023-11-06T03:17:41.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.130+0000 7ff0cb9b5640 1 -- start start 2023-11-06T03:17:41.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0c406d000 0x7ff0c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c415e440 con 0x7ff0c41599c0 2023-11-06T03:17:41.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c406c240 con 0x7ff0c40b2af0 2023-11-06T03:17:41.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c406c520 con 0x7ff0c406d000 2023-11-06T03:17:41.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9751640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff0c41599c0 0x7ff0c40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51030/0 (socket says 172.21.15.157:51030) 2023-11-06T03:17:41.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9751640 1 -- 172.21.15.157:0/1612668260 learned_addr learned my addr 172.21.15.157:0/1612668260 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:41.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9f52640 1 --2- 172.21.15.157:0/1612668260 >> v2:172.21.15.157:3300/0 conn(0x7ff0c406d000 0x7ff0c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c3fff640 1 -- 172.21.15.157:0/1612668260 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843893574 0 0) 0x7ff0c415e440 con 0x7ff0c41599c0 2023-11-06T03:17:41.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c3fff640 1 -- 172.21.15.157:0/1612668260 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac003530 con 0x7ff0c41599c0 2023-11-06T03:17:41.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9f52640 1 -- 172.21.15.157:0/1612668260 >> v1:172.21.15.195:6789/0 conn(0x7ff0c40b2af0 legacy=0x7ff0c4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9f52640 1 -- 172.21.15.157:0/1612668260 >> v1:172.21.15.157:6789/0 conn(0x7ff0c41599c0 legacy=0x7ff0c40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9f52640 1 -- 172.21.15.157:0/1612668260 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c406c800 con 0x7ff0c406d000 2023-11-06T03:17:41.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c9f52640 1 --2- 172.21.15.157:0/1612668260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c40ae000 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40097c0 tx=0x7ff0b401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5216f12d7845003 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c3fff640 1 -- 172.21.15.157:0/1612668260 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab670 con 0x7ff0c406d000 2023-11-06T03:17:41.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c3fff640 1 -- 172.21.15.157:0/1612668260 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40ab7d0 con 0x7ff0c406d000 2023-11-06T03:17:41.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0c3fff640 1 -- 172.21.15.157:0/1612668260 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40abb60 con 0x7ff0c406d000 2023-11-06T03:17:41.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/1612668260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 msgr2=0x7ff0c40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/1612668260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c40ae000 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40097c0 tx=0x7ff0b401c450 comp rx=0 tx=0).stop 2023-11-06T03:17:41.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/1612668260 shutdown_connections 2023-11-06T03:17:41.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/1612668260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c40ae000 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/1612668260 >> 172.21.15.157:0/1612668260 conn(0x7ff0c4065930 msgr2=0x7ff0c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/1612668260 shutdown_connections 2023-11-06T03:17:41.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.134+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/1612668260 wait complete. 2023-11-06T03:17:41.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 Processor -- start 2023-11-06T03:17:41.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- start start 2023-11-06T03:17:41.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0c40b2af0 0x7ff0c415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0c41599c0 0x7ff0c415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4155450 con 0x7ff0c406d000 2023-11-06T03:17:41.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4154ab0 con 0x7ff0c40b2af0 2023-11-06T03:17:41.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4154d90 con 0x7ff0c41599c0 2023-11-06T03:17:41.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51834/0 (socket says 172.21.15.157:51834) 2023-11-06T03:17:41.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 -- 172.21.15.157:0/3038297383 learned_addr learned my addr 172.21.15.157:0/3038297383 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:41.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 -- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0c41599c0 msgr2=0x7ff0c415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0c41599c0 0x7ff0c415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 -- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0c40b2af0 msgr2=0x7ff0c415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0c40b2af0 0x7ff0c415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 -- 172.21.15.157:0/3038297383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009420 con 0x7ff0c406d000 2023-11-06T03:17:41.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c9751640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00cbc0 tx=0x7ff0bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00ad40 con 0x7ff0c406d000 2023-11-06T03:17:41.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc012400 con 0x7ff0c406d000 2023-11-06T03:17:41.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c4155070 con 0x7ff0c406d000 2023-11-06T03:17:41.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00aea0 con 0x7ff0c406d000 2023-11-06T03:17:41.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c41f6160 con 0x7ff0c406d000 2023-11-06T03:17:41.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.138+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c4072750 con 0x7ff0c406d000 2023-11-06T03:17:41.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.142+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0bc007070 con 0x7ff0c406d000 2023-11-06T03:17:41.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.142+0000 7ff0c1ffb640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 0x7ff0b0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.142+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0bc083230 con 0x7ff0c406d000 2023-11-06T03:17:41.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.142+0000 7ff0c8f50640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 0x7ff0b0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.142+0000 7ff0c8f50640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 0x7ff0b0067f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8000f70 tx=0x7ff0b80079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.146+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0bc050b20 con 0x7ff0c406d000 2023-11-06T03:17:41.300 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.302+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0c40cb110 con 0x7ff0c406d000 2023-11-06T03:17:41.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0c1ffb640 1 -- 172.21.15.157:0/3038297383 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17890 (secure 0 0 0) 0x7ff0bc054140 con 0x7ff0c406d000 2023-11-06T03:17:41.304 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:41.304 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":174,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:40.965040+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:41.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 msgr2=0x7ff0b0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 0x7ff0b0067f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8000f70 tx=0x7ff0b80079c0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 msgr2=0x7ff0c41f42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00cbc0 tx=0x7ff0bc00a430 comp rx=0 tx=0).stop 2023-11-06T03:17:41.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 shutdown_connections 2023-11-06T03:17:41.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0b0065a60 0x7ff0b0067f20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0c41599c0 0x7ff0c415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0c40b2af0 0x7ff0c415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 --2- 172.21.15.157:0/3038297383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0c406d000 0x7ff0c41f42c0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 >> 172.21.15.157:0/3038297383 conn(0x7ff0c4065930 msgr2=0x7ff0c4159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 shutdown_connections 2023-11-06T03:17:41.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.306+0000 7ff0cb9b5640 1 -- 172.21.15.157:0/3038297383 wait complete. 2023-11-06T03:17:41.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.434+0000 7f35030b2640 1 Processor -- start 2023-11-06T03:17:41.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.434+0000 7f35030b2640 1 -- start start 2023-11-06T03:17:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.434+0000 7f35030b2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f34fc0ce800 0x7f34fc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 0x7f34fc06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc0cf1a0 con 0x7f34fc146370 2023-11-06T03:17:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0b2120 con 0x7f34fc0ce800 2023-11-06T03:17:41.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc06c3d0 con 0x7f34fc06b6a0 2023-11-06T03:17:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f34fc0ce800 0x7f34fc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f34fc0ce800 0x7f34fc0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51844/0 (socket says 172.21.15.157:51844) 2023-11-06T03:17:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f3500e4e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f34fc146370 0x7f34fc0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51046/0 (socket says 172.21.15.157:51046) 2023-11-06T03:17:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 -- 172.21.15.157:0/1512270409 learned_addr learned my addr 172.21.15.157:0/1512270409 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:41.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f350164f640 1 --2- 172.21.15.157:0/1512270409 >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 0x7f34fc06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 -- 172.21.15.157:0/1512270409 >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 msgr2=0x7f34fc06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 --2- 172.21.15.157:0/1512270409 >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 0x7f34fc06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 -- 172.21.15.157:0/1512270409 >> v1:172.21.15.157:6789/0 conn(0x7f34fc146370 legacy=0x7f34fc0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 -- 172.21.15.157:0/1512270409 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc06f5a0 con 0x7f34fc0ce800 2023-11-06T03:17:41.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f350164f640 1 --2- 172.21.15.157:0/1512270409 >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 0x7f34fc06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fbfff640 1 --2- 172.21.15.157:0/1512270409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc0ce800 0x7f34fc0cec60 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009870 tx=0x7f34f001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db044f9183f9e5b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fb7fe640 1 -- 172.21.15.157:0/1512270409 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00ab640 con 0x7f34fc0ce800 2023-11-06T03:17:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fb7fe640 1 -- 172.21.15.157:0/1512270409 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f00a7070 con 0x7f34fc0ce800 2023-11-06T03:17:41.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f34fb7fe640 1 -- 172.21.15.157:0/1512270409 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00ab9c0 con 0x7f34fc0ce800 2023-11-06T03:17:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- 172.21.15.157:0/1512270409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc0ce800 msgr2=0x7f34fc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 --2- 172.21.15.157:0/1512270409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc0ce800 0x7f34fc0cec60 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009870 tx=0x7f34f001c440 comp rx=0 tx=0).stop 2023-11-06T03:17:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- 172.21.15.157:0/1512270409 shutdown_connections 2023-11-06T03:17:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 --2- 172.21.15.157:0/1512270409 >> v2:172.21.15.195:3300/0 conn(0x7f34fc06b6a0 0x7f34fc06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 --2- 172.21.15.157:0/1512270409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc0ce800 0x7f34fc0cec60 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- 172.21.15.157:0/1512270409 >> 172.21.15.157:0/1512270409 conn(0x7f34fc065930 msgr2=0x7f34fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- 172.21.15.157:0/1512270409 shutdown_connections 2023-11-06T03:17:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.438+0000 7f35030b2640 1 -- 172.21.15.157:0/1512270409 wait complete. 2023-11-06T03:17:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 Processor -- start 2023-11-06T03:17:41.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- start start 2023-11-06T03:17:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 0x7f34fc15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34fc0ce800 0x7f34fc1398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1403f0 con 0x7f34fc146370 2023-11-06T03:17:41.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc140620 con 0x7f34fc06b6a0 2023-11-06T03:17:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc140900 con 0x7f34fc0ce800 2023-11-06T03:17:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51860/0 (socket says 172.21.15.157:51860) 2023-11-06T03:17:41.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 -- 172.21.15.157:0/225115865 learned_addr learned my addr 172.21.15.157:0/225115865 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f3500e4e640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 0x7f34fc15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f34fbfff640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34fc0ce800 0x7f34fc1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 -- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34fc0ce800 msgr2=0x7f34fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34fc0ce800 0x7f34fc1398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 -- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 msgr2=0x7f34fc15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 0x7f34fc15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 -- 172.21.15.157:0/225115865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00094d0 con 0x7f34fc146370 2023-11-06T03:17:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f3500e4e640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 0x7f34fc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f350164f640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c7f0 tx=0x7f34f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4004230 con 0x7f34fc146370 2023-11-06T03:17:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f40043b0 con 0x7f34fc146370 2023-11-06T03:17:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc140be0 con 0x7f34fc146370 2023-11-06T03:17:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4003860 con 0x7f34fc146370 2023-11-06T03:17:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1411a0 con 0x7f34fc146370 2023-11-06T03:17:41.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc1413c0 con 0x7f34fc146370 2023-11-06T03:17:41.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.442+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f34f4003bd0 con 0x7f34fc146370 2023-11-06T03:17:41.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.446+0000 7f34f97fa640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 0x7f34e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.446+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34f4083810 con 0x7f34fc146370 2023-11-06T03:17:41.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.446+0000 7f3500e4e640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 0x7f34e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.446+0000 7f3500e4e640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 0x7f34e8067f20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0012f0 tx=0x7f34ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.450+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f34f4051070 con 0x7f34fc146370 2023-11-06T03:17:41.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.606+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34fc072750 con 0x7f34fc146370 2023-11-06T03:17:41.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.606+0000 7f34f97fa640 1 -- 172.21.15.157:0/225115865 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17890 (secure 0 0 0) 0x7f34f4054690 con 0x7f34fc146370 2023-11-06T03:17:41.605 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:41.605 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":174,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:40.965040+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:41.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 msgr2=0x7f34e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 0x7f34e8067f20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0012f0 tx=0x7f34ec009040 comp rx=0 tx=0).stop 2023-11-06T03:17:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 msgr2=0x7f34fc139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c7f0 tx=0x7f34f400ccb0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 shutdown_connections 2023-11-06T03:17:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f34e8065a60 0x7f34e8067f20 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34fc146370 0x7f34fc139e30 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34fc0ce800 0x7f34fc1398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 --2- 172.21.15.157:0/225115865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34fc06b6a0 0x7f34fc15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 >> 172.21.15.157:0/225115865 conn(0x7f34fc065930 msgr2=0x7f34fc0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 shutdown_connections 2023-11-06T03:17:41.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.610+0000 7f35030b2640 1 -- 172.21.15.157:0/225115865 wait complete. 2023-11-06T03:17:41.624 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:17:41.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.738+0000 7f5e5a045640 1 Processor -- start 2023-11-06T03:17:41.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.738+0000 7f5e5a045640 1 -- start start 2023-11-06T03:17:41.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e5a045640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e540ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e5a045640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5415e440 con 0x7f5e5406d000 2023-11-06T03:17:41.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e5a045640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5406c240 con 0x7f5e541599c0 2023-11-06T03:17:41.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e5a045640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5406c520 con 0x7f5e540b2af0 2023-11-06T03:17:41.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5e5406d000 0x7f5e54159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51060/0 (socket says 172.21.15.157:51060) 2023-11-06T03:17:41.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 -- 172.21.15.157:0/2944753847 learned_addr learned my addr 172.21.15.157:0/2944753847 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:41.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e52ffd640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e540ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162155495 0 0) 0x7f5e5415e440 con 0x7f5e5406d000 2023-11-06T03:17:41.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e3c003530 con 0x7f5e5406d000 2023-11-06T03:17:41.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 537302573 0 0) 0x7f5e3c003530 con 0x7f5e5406d000 2023-11-06T03:17:41.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 msgr2=0x7f5e540ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e540ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 >> v1:172.21.15.195:6789/0 conn(0x7f5e541599c0 legacy=0x7f5e540b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5406c800 con 0x7f5e5406d000 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5e4c001a00 con 0x7f5e5406d000 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 >> v1:172.21.15.157:6789/0 conn(0x7f5e5406d000 legacy=0x7f5e54159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c003f30 0x7f5e3c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c004d10 0x7f5e3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5406c520 con 0x7f5e3c004d10 2023-11-06T03:17:41.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c00dc80 con 0x7f5e3c003f30 2023-11-06T03:17:41.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c00de40 con 0x7f5e540b2af0 2023-11-06T03:17:41.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c004d10 0x7f5e3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e52ffd640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c003f30 0x7f5e3c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 -- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 msgr2=0x7f5e3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:41.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e537fe640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540b2af0 0x7f5e3c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 -- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c003f30 msgr2=0x7f5e3c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:41.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c003f30 0x7f5e3c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e53fff640 1 -- 172.21.15.157:0/2944753847 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c003530 con 0x7f5e3c004d10 2023-11-06T03:17:41.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.742+0000 7f5e52ffd640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c003f30 0x7f5e3c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:41.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e53fff640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c004d10 0x7f5e3c005190 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f5e5406d430 tx=0x7f5e4c01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42230ed10ee3178 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0a5ea0 con 0x7f5e3c004d10 2023-11-06T03:17:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c0a8070 con 0x7f5e3c004d10 2023-11-06T03:17:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e527fc640 1 -- 172.21.15.157:0/2944753847 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c007530 con 0x7f5e3c004d10 2023-11-06T03:17:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2944753847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c004d10 msgr2=0x7f5e3c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2944753847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e3c004d10 0x7f5e3c005190 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f5e5406d430 tx=0x7f5e4c01d2c0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e53fff640 1 -- 172.21.15.157:0/2944753847 reap_dead start 2023-11-06T03:17:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2944753847 shutdown_connections 2023-11-06T03:17:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2944753847 >> 172.21.15.157:0/2944753847 conn(0x7f5e54065930 msgr2=0x7f5e54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2944753847 shutdown_connections 2023-11-06T03:17:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2944753847 wait complete. 2023-11-06T03:17:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 Processor -- start 2023-11-06T03:17:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- start start 2023-11-06T03:17:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540cba10 0x7f5e541ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e540cdd00 0x7f5e540ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540ce6c0 con 0x7f5e540acfe0 2023-11-06T03:17:41.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540cd110 con 0x7f5e540cdd00 2023-11-06T03:17:41.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e54153f70 con 0x7f5e540cba10 2023-11-06T03:17:41.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51876/0 (socket says 172.21.15.157:51876) 2023-11-06T03:17:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 -- 172.21.15.157:0/2682065960 learned_addr learned my addr 172.21.15.157:0/2682065960 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 -- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540cba10 msgr2=0x7f5e541ee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540cba10 0x7f5e541ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 -- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e540cdd00 msgr2=0x7f5e540ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e53fff640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e540cdd00 0x7f5e540ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e540cdd00 0x7f5e540ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 -- 172.21.15.157:0/2682065960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e541563f0 con 0x7f5e540acfe0 2023-11-06T03:17:41.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e537fe640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d7a0 tx=0x7f5e4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44005270 con 0x7f5e540acfe0 2023-11-06T03:17:41.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440053d0 con 0x7f5e540acfe0 2023-11-06T03:17:41.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e541566d0 con 0x7f5e540acfe0 2023-11-06T03:17:41.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400b850 con 0x7f5e540acfe0 2023-11-06T03:17:41.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.746+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e54156c90 con 0x7f5e540acfe0 2023-11-06T03:17:41.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e54156e50 con 0x7f5e540acfe0 2023-11-06T03:17:41.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e4400b9b0 con 0x7f5e540acfe0 2023-11-06T03:17:41.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e3bfff640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 0x7f5e40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:41.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e440830d0 con 0x7f5e540acfe0 2023-11-06T03:17:41.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e52ffd640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 0x7f5e40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:41.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.750+0000 7f5e52ffd640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 0x7f5e40067f20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5e54154270 tx=0x7f5e48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:41.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.754+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5e440509c0 con 0x7f5e540acfe0 2023-11-06T03:17:41.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.942+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e540adcd0 con 0x7f5e540acfe0 2023-11-06T03:17:41.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e3bfff640 1 -- 172.21.15.157:0/2682065960 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v122) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e44053fe0 con 0x7f5e540acfe0 2023-11-06T03:17:41.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 msgr2=0x7f5e40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 0x7f5e40067f20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5e54154270 tx=0x7f5e48008040 comp rx=0 tx=0).stop 2023-11-06T03:17:41.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 msgr2=0x7f5e540cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:41.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d7a0 tx=0x7f5e4400dc60 comp rx=0 tx=0).stop 2023-11-06T03:17:41.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 shutdown_connections 2023-11-06T03:17:41.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5e40065a60 0x7f5e40067f20 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e540cdd00 0x7f5e540ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e540cba10 0x7f5e541ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 --2- 172.21.15.157:0/2682065960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e540acfe0 0x7f5e540cc8e0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:41.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 >> 172.21.15.157:0/2682065960 conn(0x7f5e54065930 msgr2=0x7f5e5415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:41.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 shutdown_connections 2023-11-06T03:17:41.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:41.974+0000 7f5e5a045640 1 -- 172.21.15.157:0/2682065960 wait complete. 2023-11-06T03:17:41.992 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:17:41.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:17:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 Processor -- start 2023-11-06T03:17:42.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 -- start start 2023-11-06T03:17:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 0x7f737c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f737c0b3b20 0x7f737c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737c15f580 con 0x7f737c15e5e0 2023-11-06T03:17:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15efc0 con 0x7f737c0b3b20 2023-11-06T03:17:42.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f738305e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15f2a0 con 0x7f737c1551e0 2023-11-06T03:17:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f73815fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f737c0b3b20 0x7f737c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f73815fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f737c0b3b20 0x7f737c15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51880/0 (socket says 172.21.15.157:51880) 2023-11-06T03:17:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.118+0000 7f73815fb640 1 -- 172.21.15.157:0/58833602 learned_addr learned my addr 172.21.15.157:0/58833602 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f7380dfa640 1 --2- 172.21.15.157:0/58833602 >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 0x7f737c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f73815fb640 1 -- 172.21.15.157:0/58833602 >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 msgr2=0x7f737c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f73815fb640 1 --2- 172.21.15.157:0/58833602 >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 0x7f737c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f73815fb640 1 -- 172.21.15.157:0/58833602 >> v1:172.21.15.195:6789/0 conn(0x7f737c15e5e0 legacy=0x7f737c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f73815fb640 1 -- 172.21.15.157:0/58833602 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c06f5a0 con 0x7f737c0b3b20 2023-11-06T03:17:42.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f7380dfa640 1 --2- 172.21.15.157:0/58833602 >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 0x7f737c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:42.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f73815fb640 1 --2- 172.21.15.157:0/58833602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0b3b20 0x7f737c15c1b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f737c066830 tx=0x7f737401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffd94d7e27af446 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f737b7fe640 1 -- 172.21.15.157:0/58833602 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740a5720 con 0x7f737c0b3b20 2023-11-06T03:17:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f737b7fe640 1 -- 172.21.15.157:0/58833602 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73740a5880 con 0x7f737c0b3b20 2023-11-06T03:17:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f737b7fe640 1 -- 172.21.15.157:0/58833602 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740a5c10 con 0x7f737c0b3b20 2023-11-06T03:17:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 -- 172.21.15.157:0/58833602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0b3b20 msgr2=0x7f737c15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:42.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 --2- 172.21.15.157:0/58833602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0b3b20 0x7f737c15c1b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f737c066830 tx=0x7f737401c840 comp rx=0 tx=0).stop 2023-11-06T03:17:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 -- 172.21.15.157:0/58833602 shutdown_connections 2023-11-06T03:17:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 --2- 172.21.15.157:0/58833602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0b3b20 0x7f737c15c1b0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 --2- 172.21.15.157:0/58833602 >> v2:172.21.15.195:3300/0 conn(0x7f737c1551e0 0x7f737c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 -- 172.21.15.157:0/58833602 >> 172.21.15.157:0/58833602 conn(0x7f737c065930 msgr2=0x7f737c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:42.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 -- 172.21.15.157:0/58833602 shutdown_connections 2023-11-06T03:17:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.122+0000 7f738305e640 1 -- 172.21.15.157:0/58833602 wait complete. 2023-11-06T03:17:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 Processor -- start 2023-11-06T03:17:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- start start 2023-11-06T03:17:42.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0b3b20 0x7f737c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 0x7f737c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c152950 con 0x7f737c1551e0 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c152c30 con 0x7f737c0b3b20 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c152f10 con 0x7f737c15e5e0 2023-11-06T03:17:42.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51896/0 (socket says 172.21.15.157:51896) 2023-11-06T03:17:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 -- 172.21.15.157:0/465209053 learned_addr learned my addr 172.21.15.157:0/465209053 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f7380dfa640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0b3b20 0x7f737c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73815fb640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 0x7f737c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 -- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 msgr2=0x7f737c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 0x7f737c1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 -- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0b3b20 msgr2=0x7f737c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0b3b20 0x7f737c152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 -- 172.21.15.157:0/465209053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374009b30 con 0x7f737c1551e0 2023-11-06T03:17:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73815fb640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 0x7f737c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f737bfff640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f737000b410 tx=0x7f737000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370004020 con 0x7f737c1551e0 2023-11-06T03:17:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370004180 con 0x7f737c1551e0 2023-11-06T03:17:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c1531f0 con 0x7f737c1551e0 2023-11-06T03:17:42.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370011630 con 0x7f737c1551e0 2023-11-06T03:17:42.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c1568f0 con 0x7f737c1551e0 2023-11-06T03:17:42.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c153710 con 0x7f737c1551e0 2023-11-06T03:17:42.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.126+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7370011790 con 0x7f737c1551e0 2023-11-06T03:17:42.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.130+0000 7f73797fa640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:42.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.130+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73700839c0 con 0x7f737c1551e0 2023-11-06T03:17:42.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.130+0000 7f7380dfa640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:42.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.130+0000 7f7380dfa640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 0x7f7368067f20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f736c00a9a0 tx=0x7f736c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:42.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.130+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7370051200 con 0x7f737c1551e0 2023-11-06T03:17:42.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.322+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f737c072750 con 0x7f737c1551e0 2023-11-06T03:17:42.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:17:42.505 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:17:42.563 INFO:tasks.ceph.mds.c.smithi157.stderr:2023-11-06T03:17:42.566+0000 7f04f0d90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:17:42.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.982+0000 7f73797fa640 1 -- 172.21.15.157:0/465209053 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v123) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7370054820 con 0x7f737c1551e0 2023-11-06T03:17:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 msgr2=0x7f7368067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 0x7f7368067f20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f736c00a9a0 tx=0x7f736c008040 comp rx=0 tx=0).stop 2023-11-06T03:17:42.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 msgr2=0x7f737c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f737000b410 tx=0x7f737000b8d0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 shutdown_connections 2023-11-06T03:17:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c15e5e0 0x7f737c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c1551e0 0x7f737c1539f0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 --2- 172.21.15.157:0/465209053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0b3b20 0x7f737c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 >> 172.21.15.157:0/465209053 conn(0x7f737c065930 msgr2=0x7f737c0ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 shutdown_connections 2023-11-06T03:17:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:42.986+0000 7f738305e640 1 -- 172.21.15.157:0/465209053 wait complete. 2023-11-06T03:17:43.006 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:17:43.007 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.130+0000 7f9fc7eef640 1 Processor -- start 2023-11-06T03:17:43.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.130+0000 7f9fc7eef640 1 -- start start 2023-11-06T03:17:43.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc7eef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc7eef640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 0x7f9fc00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc7eef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc015f580 con 0x7f9fc01551e0 2023-11-06T03:17:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc7eef640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc00704b0 con 0x7f9fc015ea60 2023-11-06T03:17:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc7eef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015efc0 con 0x7f9fc015e5e0 2023-11-06T03:17:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc648c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 0x7f9fc00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37950/0 (socket says 172.21.15.157:37950) 2023-11-06T03:17:43.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc5c8b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f9fc01551e0 0x7f9fc006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:44104/0 (socket says 172.21.15.157:44104) 2023-11-06T03:17:43.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc648c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 0x7f9fc00b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:53240/0 (socket says 172.21.15.157:53240) 2023-11-06T03:17:43.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 -- 172.21.15.157:0/3680213925 learned_addr learned my addr 172.21.15.157:0/3680213925 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:43.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 -- 172.21.15.157:0/3680213925 >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 msgr2=0x7f9fc00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:43.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 --2- 172.21.15.157:0/3680213925 >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 0x7f9fc00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 -- 172.21.15.157:0/3680213925 >> v1:172.21.15.195:6789/0 conn(0x7f9fc01551e0 legacy=0x7f9fc006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.134+0000 7f9fc548a640 1 -- 172.21.15.157:0/3680213925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc015f2a0 con 0x7f9fc015e5e0 2023-11-06T03:17:43.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc548a640 1 --2- 172.21.15.157:0/3680213925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009870 tx=0x7f9fb001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba175341ba93074 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc4c89640 1 -- 172.21.15.157:0/3680213925 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00ab680 con 0x7f9fc015e5e0 2023-11-06T03:17:43.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc4c89640 1 -- 172.21.15.157:0/3680213925 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb00ab7e0 con 0x7f9fc015e5e0 2023-11-06T03:17:43.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc4c89640 1 -- 172.21.15.157:0/3680213925 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00abb70 con 0x7f9fc015e5e0 2023-11-06T03:17:43.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3680213925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 msgr2=0x7f9fc00b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3680213925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009870 tx=0x7f9fb001c480 comp rx=0 tx=0).stop 2023-11-06T03:17:43.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3680213925 shutdown_connections 2023-11-06T03:17:43.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3680213925 >> v2:172.21.15.195:3300/0 conn(0x7f9fc015ea60 0x7f9fc00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3680213925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc015e5e0 0x7f9fc00b3410 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3680213925 >> 172.21.15.157:0/3680213925 conn(0x7f9fc0065930 msgr2=0x7f9fc0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3680213925 shutdown_connections 2023-11-06T03:17:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3680213925 wait complete. 2023-11-06T03:17:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 Processor -- start 2023-11-06T03:17:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- start start 2023-11-06T03:17:43.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 0x7f9fc01439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 0x7f9fc0144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0143f10 con 0x7f9fc015ea60 2023-11-06T03:17:43.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0144a90 con 0x7f9fc015e5e0 2023-11-06T03:17:43.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0144d70 con 0x7f9fc01551e0 2023-11-06T03:17:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51910/0 (socket says 172.21.15.157:51910) 2023-11-06T03:17:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 -- 172.21.15.157:0/3127998267 learned_addr learned my addr 172.21.15.157:0/3127998267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc5c8b640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 0x7f9fc01439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc548a640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 0x7f9fc0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 -- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 msgr2=0x7f9fc01439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 0x7f9fc01439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 -- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 msgr2=0x7f9fc0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 0x7f9fc0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 -- 172.21.15.157:0/3127998267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00094d0 con 0x7f9fc015ea60 2023-11-06T03:17:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc548a640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 0x7f9fc0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc5c8b640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 0x7f9fc01439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc648c640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800efc0 tx=0x7f9fb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8009b20 con 0x7f9fc015ea60 2023-11-06T03:17:43.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc0145050 con 0x7f9fc015ea60 2023-11-06T03:17:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb800f040 con 0x7f9fc015ea60 2023-11-06T03:17:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80146c0 con 0x7f9fc015ea60 2023-11-06T03:17:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.138+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc00cca20 con 0x7f9fc015ea60 2023-11-06T03:17:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc0072750 con 0x7f9fc015ea60 2023-11-06T03:17:43.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9fb8014820 con 0x7f9fc015ea60 2023-11-06T03:17:43.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fb67fc640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 0x7f9fa4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fb8084120 con 0x7f9fc015ea60 2023-11-06T03:17:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fc5c8b640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 0x7f9fa4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.142+0000 7f9fc5c8b640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 0x7f9fa4067f20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc000f70 tx=0x7f9fbc007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.146+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9fb8051900 con 0x7f9fc015ea60 2023-11-06T03:17:43.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.334+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc0071730 con 0x7f9fc015ea60 2023-11-06T03:17:43.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.334+0000 7f9fb67fc640 1 -- 172.21.15.157:0/3127998267 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7f9fb8009e60 con 0x7f9fc015ea60 2023-11-06T03:17:43.333 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:43.333 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:17:43.333 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:17:43.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 msgr2=0x7f9fa4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 0x7f9fa4067f20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc000f70 tx=0x7f9fbc007d70 comp rx=0 tx=0).stop 2023-11-06T03:17:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 msgr2=0x7f9fc0149b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800efc0 tx=0x7f9fb800c490 comp rx=0 tx=0).stop 2023-11-06T03:17:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 shutdown_connections 2023-11-06T03:17:43.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9fa4065a60 0x7f9fa4067f20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9fc015ea60 0x7f9fc0149b20 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fc015e5e0 0x7f9fc0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 --2- 172.21.15.157:0/3127998267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9fc01551e0 0x7f9fc01439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 >> 172.21.15.157:0/3127998267 conn(0x7f9fc0065930 msgr2=0x7f9fc015d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 shutdown_connections 2023-11-06T03:17:43.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.338+0000 7f9fc7eef640 1 -- 172.21.15.157:0/3127998267 wait complete. 2023-11-06T03:17:43.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.470+0000 7f5beafe4640 1 Processor -- start 2023-11-06T03:17:43.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.470+0000 7f5beafe4640 1 -- start start 2023-11-06T03:17:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5beafe4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 0x7f5be40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5beafe4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5beafe4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be415e440 con 0x7f5be41599c0 2023-11-06T03:17:43.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5beafe4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be406c240 con 0x7f5be40b2af0 2023-11-06T03:17:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5beafe4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be406c520 con 0x7f5be406d000 2023-11-06T03:17:43.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be3fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 0x7f5be40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:37970/0 (socket says 172.21.15.157:37970) 2023-11-06T03:17:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 -- 172.21.15.157:0/2697062127 learned_addr learned my addr 172.21.15.157:0/2697062127 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:43.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be3fff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 0x7f5be40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:53250/0 (socket says 172.21.15.157:53250) 2023-11-06T03:17:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 -- 172.21.15.157:0/2697062127 >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 msgr2=0x7f5be40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 --2- 172.21.15.157:0/2697062127 >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 0x7f5be40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 -- 172.21.15.157:0/2697062127 >> v1:172.21.15.195:6789/0 conn(0x7f5be41599c0 legacy=0x7f5be40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.474+0000 7f5be9581640 1 -- 172.21.15.157:0/2697062127 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be406c800 con 0x7f5be406d000 2023-11-06T03:17:43.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5be9581640 1 --2- 172.21.15.157:0/2697062127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009e50 tx=0x7f5bdc01c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63fcab07328fb3d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5be37fe640 1 -- 172.21.15.157:0/2697062127 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0a58d0 con 0x7f5be406d000 2023-11-06T03:17:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5be37fe640 1 -- 172.21.15.157:0/2697062127 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc0a5a30 con 0x7f5be406d000 2023-11-06T03:17:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5be37fe640 1 -- 172.21.15.157:0/2697062127 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0a5dc0 con 0x7f5be406d000 2023-11-06T03:17:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- 172.21.15.157:0/2697062127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 msgr2=0x7f5be4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2697062127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009e50 tx=0x7f5bdc01c9e0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- 172.21.15.157:0/2697062127 shutdown_connections 2023-11-06T03:17:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2697062127 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be406d000 0x7f5be4159da0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2697062127 >> v2:172.21.15.195:3300/0 conn(0x7f5be40b2af0 0x7f5be40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- 172.21.15.157:0/2697062127 >> 172.21.15.157:0/2697062127 conn(0x7f5be4065930 msgr2=0x7f5be4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- 172.21.15.157:0/2697062127 shutdown_connections 2023-11-06T03:17:43.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- 172.21.15.157:0/2697062127 wait complete. 2023-11-06T03:17:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 Processor -- start 2023-11-06T03:17:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- start start 2023-11-06T03:17:43.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be40b2af0 0x7f5be4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be41599c0 0x7f5be40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41f9be0 con 0x7f5be406d000 2023-11-06T03:17:43.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be414e660 con 0x7f5be41599c0 2023-11-06T03:17:43.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.478+0000 7f5beafe4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be41f8fd0 con 0x7f5be40b2af0 2023-11-06T03:17:43.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be9581640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be41599c0 0x7f5be40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be40b2af0 0x7f5be4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51918/0 (socket says 172.21.15.157:51918) 2023-11-06T03:17:43.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 -- 172.21.15.157:0/2448556739 learned_addr learned my addr 172.21.15.157:0/2448556739 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 -- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be40b2af0 msgr2=0x7f5be4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be40b2af0 0x7f5be4148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 -- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be41599c0 msgr2=0x7f5be40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be41599c0 0x7f5be40ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.486 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 -- 172.21.15.157:0/2448556739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009b30 con 0x7f5be406d000 2023-11-06T03:17:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be8d80640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c500 tx=0x7f5bd800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8005220 con 0x7f5be406d000 2023-11-06T03:17:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8005380 con 0x7f5be406d000 2023-11-06T03:17:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be41f92b0 con 0x7f5be406d000 2023-11-06T03:17:43.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8007270 con 0x7f5be406d000 2023-11-06T03:17:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be41f9870 con 0x7f5be406d000 2023-11-06T03:17:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be4072750 con 0x7f5be406d000 2023-11-06T03:17:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5bd80073d0 con 0x7f5be406d000 2023-11-06T03:17:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 0x7f5bd0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bd8082a60 con 0x7f5be406d000 2023-11-06T03:17:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.482+0000 7f5be3fff640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 0x7f5bd0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.486+0000 7f5be3fff640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 0x7f5bd0067f50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5be4149b50 tx=0x7f5bd4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.486+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f5bd8050380 con 0x7f5be406d000 2023-11-06T03:17:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.642+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5be4065e10 con 0x7f5be406d000 2023-11-06T03:17:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5be17fa640 1 -- 172.21.15.157:0/2448556739 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+17877 (secure 0 0 0) 0x7f5bd80539a0 con 0x7f5be406d000 2023-11-06T03:17:43.643 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:43.643 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":175,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:41.970106+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 msgr2=0x7f5bd0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 0x7f5bd0067f50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5be4149b50 tx=0x7f5bd4009190 comp rx=0 tx=0).stop 2023-11-06T03:17:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 msgr2=0x7f5be4148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c500 tx=0x7f5bd800c9c0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 shutdown_connections 2023-11-06T03:17:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5bd0065a90 0x7f5bd0067f50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5be41599c0 0x7f5be40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5be40b2af0 0x7f5be4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 --2- 172.21.15.157:0/2448556739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be406d000 0x7f5be4148db0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 >> 172.21.15.157:0/2448556739 conn(0x7f5be4065930 msgr2=0x7f5be4158680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 shutdown_connections 2023-11-06T03:17:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.646+0000 7f5beafe4640 1 -- 172.21.15.157:0/2448556739 wait complete. 2023-11-06T03:17:43.661 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:17:43.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:17:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.782+0000 7f76cfc29640 1 Processor -- start 2023-11-06T03:17:43.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.782+0000 7f76cfc29640 1 -- start start 2023-11-06T03:17:43.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cfc29640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76c80cc3d0 0x7f76c8066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cfc29640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c815aff0 con 0x7f76c80b1e40 2023-11-06T03:17:43.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cfc29640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c806dd80 con 0x7f76c80cf090 2023-11-06T03:17:43.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cfc29640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c806e060 con 0x7f76c80cc3d0 2023-11-06T03:17:43.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f76c80b1e40 0x7f76c80cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51064/0 (socket says 172.21.15.157:51064) 2023-11-06T03:17:43.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 -- 172.21.15.157:0/2081110875 learned_addr learned my addr 172.21.15.157:0/2081110875 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:43.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cd1c4640 1 --2- 172.21.15.157:0/2081110875 >> v2:172.21.15.195:3300/0 conn(0x7f76c80cc3d0 0x7f76c8066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057708127 0 0) 0x7f76c815aff0 con 0x7f76c80b1e40 2023-11-06T03:17:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b0003530 con 0x7f76c80b1e40 2023-11-06T03:17:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094038192 0 0) 0x7f76b0003530 con 0x7f76c80b1e40 2023-11-06T03:17:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 >> v2:172.21.15.195:3300/0 conn(0x7f76c80cc3d0 msgr2=0x7f76c8066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:43.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 --2- 172.21.15.157:0/2081110875 >> v2:172.21.15.195:3300/0 conn(0x7f76c80cc3d0 0x7f76c8066340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 >> v1:172.21.15.195:6789/0 conn(0x7f76c80cf090 legacy=0x7f76c80b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c815b2b0 con 0x7f76c80b1e40 2023-11-06T03:17:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f76c0001a00 con 0x7f76c80b1e40 2023-11-06T03:17:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 >> v1:172.21.15.157:6789/0 conn(0x7f76c80b1e40 legacy=0x7f76c80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76b0003980 0x7f76b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76b00047d0 0x7f76b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c806e060 con 0x7f76b0005190 2023-11-06T03:17:43.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b000c030 con 0x7f76b0003980 2023-11-06T03:17:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b000c1f0 con 0x7f76b00047d0 2023-11-06T03:17:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cd9c5640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76b0003980 0x7f76b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76cd1c4640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76b00047d0 0x7f76b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 -- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76b00047d0 msgr2=0x7f76b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76b00047d0 0x7f76b0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 -- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76b0003980 msgr2=0x7f76b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76b0003980 0x7f76b0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.786+0000 7f76ce1c6640 1 -- 172.21.15.157:0/2081110875 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0003530 con 0x7f76b0005190 2023-11-06T03:17:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76ce1c6640 1 -- 172.21.15.157:0/2081110875 reap_dead start 2023-11-06T03:17:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76b000b990 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f76c000a830 tx=0x7f76c001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0394989413466cb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c00072d0 con 0x7f76b0005190 2023-11-06T03:17:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c00a8070 con 0x7f76b0005190 2023-11-06T03:17:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cc9c3640 1 -- 172.21.15.157:0/2081110875 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c00076b0 con 0x7f76b0005190 2023-11-06T03:17:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 msgr2=0x7f76b000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76b000b990 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f76c000a830 tx=0x7f76c001d340 comp rx=0 tx=0).stop 2023-11-06T03:17:43.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- 172.21.15.157:0/2081110875 shutdown_connections 2023-11-06T03:17:43.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 --2- 172.21.15.157:0/2081110875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76b000b990 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- 172.21.15.157:0/2081110875 >> 172.21.15.157:0/2081110875 conn(0x7f76c80732f0 msgr2=0x7f76c8069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- 172.21.15.157:0/2081110875 shutdown_connections 2023-11-06T03:17:43.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- 172.21.15.157:0/2081110875 wait complete. 2023-11-06T03:17:43.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 Processor -- start 2023-11-06T03:17:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- start start 2023-11-06T03:17:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76c80b0b10 0x7f76c8156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 0x7f76c81f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c8157250 con 0x7f76b0005190 2023-11-06T03:17:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80afd70 con 0x7f76c8156dd0 2023-11-06T03:17:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cfc29640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80b0030 con 0x7f76c80b0b10 2023-11-06T03:17:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51934/0 (socket says 172.21.15.157:51934) 2023-11-06T03:17:43.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 -- 172.21.15.157:0/1919598463 learned_addr learned my addr 172.21.15.157:0/1919598463 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 0x7f76c81f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd1c4640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76c80b0b10 0x7f76c8156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 -- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76c80b0b10 msgr2=0x7f76c8156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76c80b0b10 0x7f76c8156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 -- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 msgr2=0x7f76c81f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 0x7f76c81f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76cd9c5640 1 -- 172.21.15.157:0/1919598463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c80b0310 con 0x7f76b0005190 2023-11-06T03:17:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.790+0000 7f76ce1c6640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 0x7f76c81f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cd9c5640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e4d0 tx=0x7f76b800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b80052e0 con 0x7f76b0005190 2023-11-06T03:17:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8005440 con 0x7f76b0005190 2023-11-06T03:17:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c80b05f0 con 0x7f76b0005190 2023-11-06T03:17:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8003820 con 0x7f76b0005190 2023-11-06T03:17:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c81f5ef0 con 0x7f76b0005190 2023-11-06T03:17:43.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c8072750 con 0x7f76b0005190 2023-11-06T03:17:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76b8003980 con 0x7f76b0005190 2023-11-06T03:17:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 0x7f76b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76b8082fa0 con 0x7f76b0005190 2023-11-06T03:17:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cd1c4640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 0x7f76b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:43.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.794+0000 7f76cd1c4640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 0x7f76b4067f20 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f76c80ae3a0 tx=0x7f76bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:43.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.798+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f76b80b4070 con 0x7f76b0005190 2023-11-06T03:17:43.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.950+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76c815b590 con 0x7f76b4065a60 2023-11-06T03:17:43.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.970+0000 7f76c67fc640 1 -- 172.21.15.157:0/1919598463 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295725 (secure 0 0 0) 0x7f76bc0095e0 con 0x7f76b4065a60 2023-11-06T03:17:43.972 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:43.974 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:17:43.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 msgr2=0x7f76b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 0x7f76b4067f20 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f76c80ae3a0 tx=0x7f76bc008040 comp rx=0 tx=0).stop 2023-11-06T03:17:43.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 msgr2=0x7f76c814e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:43.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e4d0 tx=0x7f76b800e990 comp rx=0 tx=0).stop 2023-11-06T03:17:43.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 shutdown_connections 2023-11-06T03:17:43.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f76b4065a60 0x7f76b4067f20 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76c8156dd0 0x7f76c81f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76c80b0b10 0x7f76c8156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 --2- 172.21.15.157:0/1919598463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76b0005190 0x7f76c814e330 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:43.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 >> 172.21.15.157:0/1919598463 conn(0x7f76c80732f0 msgr2=0x7f76c80cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:43.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 shutdown_connections 2023-11-06T03:17:43.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:43.974+0000 7f76cfc29640 1 -- 172.21.15.157:0/1919598463 wait complete. 2023-11-06T03:17:43.991 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2023-11-06T03:17:43.391020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":643296,"kb_used_data":17888,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754331424,"statfs":{"total":773094113280,"available":772435378176,"internally_reserved":0,"allocated":18317312,"data_stored":9117671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001179"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475742+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:17:37.475742+0000","last_peered":"2023-11-06T03:17:37.475742+0000","last_clean":"2023-11-06T03:17:37.475742+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:17:37.475742+0000","last_undegraded":"2023-11-06T03:17:37.475742+0000","last_fullsized":"2023-11-06T03:17:37.475742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284344+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:17:37.284344+0000","last_peered":"2023-11-06T03:17:37.284344+0000","last_clean":"2023-11-06T03:17:37.284344+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:17:37.284344+0000","last_undegraded":"2023-11-06T03:17:37.284344+0000","last_fullsized":"2023-11-06T03:17:37.284344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475712+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:17:37.475712+0000","last_peered":"2023-11-06T03:17:37.475712+0000","last_clean":"2023-11-06T03:17:37.475712+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:17:37.475712+0000","last_undegraded":"2023-11-06T03:17:37.475712+0000","last_fullsized":"2023-11-06T03:17:37.475712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.312915+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:17:37.312915+0000","last_peered":"2023-11-06T03:17:37.312915+0000","last_clean":"2023-11-06T03:17:37.312915+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:17:37.312915+0000","last_undegraded":"2023-11-06T03:17:37.312915+0000","last_fullsized":"2023-11-06T03:17:37.312915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284366+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:17:37.284366+0000","last_peered":"2023-11-06T03:17:37.284366+0000","last_clean":"2023-11-06T03:17:37.284366+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:17:37.284366+0000","last_undegraded":"2023-11-06T03:17:37.284366+0000","last_fullsized":"2023-11-06T03:17:37.284366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596644+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:17:36.596644+0000","last_peered":"2023-11-06T03:17:36.596644+0000","last_clean":"2023-11-06T03:17:36.596644+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:17:36.596644+0000","last_undegraded":"2023-11-06T03:17:36.596644+0000","last_fullsized":"2023-11-06T03:17:36.596644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475721+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:17:37.475721+0000","last_peered":"2023-11-06T03:17:37.475721+0000","last_clean":"2023-11-06T03:17:37.475721+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:17:37.475721+0000","last_undegraded":"2023-11-06T03:17:37.475721+0000","last_fullsized":"2023-11-06T03:17:37.475721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596640+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:17:36.596640+0000","last_peered":"2023-11-06T03:17:36.596640+0000","last_clean":"2023-11-06T03:17:36.596640+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:17:36.596640+0000","last_undegraded":"2023-11-06T03:17:36.596640+0000","last_fullsized":"2023-11-06T03:17:36.596640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.285676+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:17:37.285676+0000","last_peered":"2023-11-06T03:17:37.285676+0000","last_clean":"2023-11-06T03:17:37.285676+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:17:37.285676+0000","last_undegraded":"2023-11-06T03:17:37.285676+0000","last_fullsized":"2023-11-06T03:17:37.285676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91904,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94279936,"statfs":{"total":96636764160,"available":96542654464,"internally_reserved":0,"allocated":2744320,"data_stored":1586292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":1.319}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":1.008},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":1.413},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":1.2749999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":1.056}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":1.165}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69164,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302676,"statfs":{"total":96636764160,"available":96565940224,"internally_reserved":0,"allocated":2134016,"data_stored":989425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":1.26}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":1.5169999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":1.321},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":1.716}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":1.351},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.2889999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.625}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47088,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324752,"statfs":{"total":96636764160,"available":96588546048,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":1.1419999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":1.1739999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.2},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":1.4470000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113468,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258372,"statfs":{"total":96636764160,"available":96520572928,"internally_reserved":0,"allocated":2150400,"data_stored":995924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.83899999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":1.012},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.75}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":1.127},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.0469999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":1.081}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":1.103},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83192,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288648,"statfs":{"total":96636764160,"available":96551575552,"internally_reserved":0,"allocated":2146304,"data_stored":994905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":0.55500000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":0.497},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":0.378},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":0.34599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":0.91500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73264,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298576,"statfs":{"total":96636764160,"available":96561741824,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.62},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":1.018},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":1.175}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":1.419},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69168,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302672,"statfs":{"total":96636764160,"available":96565936128,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.80400000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":0.624},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96048,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275792,"statfs":{"total":96636764160,"available":96538411008,"internally_reserved":0,"allocated":2727936,"data_stored":1579793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.95799999999999996,"5min":1.0149999999999999,"15min":2.6989999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61399999999999999,"15min":0.65000000000000002},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.008,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.20599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.1619999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":4.6920000000000002},"last":0.81599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.89100000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":0.98199999999999998,"5min":1.391,"15min":2.7999999999999998},"last":0.85899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.419,"5min":1.419,"15min":4.641},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.67500000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.4159999999999999,"15min":2.028},"last":0.73399999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.292,"5min":1.492,"15min":2.8559999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.224,"5min":1.496,"15min":4.7430000000000003},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91300000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.3220000000000001,"15min":2.931},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.69699999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.446,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":1.022}]},{"osd":7,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.95799999999999996,"15min":0.99099999999999999},"min":{"1min":0.73999999999999999,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.323,"5min":1.381,"15min":4.8339999999999996},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2569999999999999,"5min":1.335,"15min":1.97},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:17:43.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:17:44.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.154+0000 7fe59bc42640 1 Processor -- start 2023-11-06T03:17:44.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.154+0000 7fe59bc42640 1 -- start start 2023-11-06T03:17:44.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5941599c0 0x7fe5940ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 0x7fe5940b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe59415dea0 0x7fe5940b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59415e440 con 0x7fe59415dea0 2023-11-06T03:17:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59406c240 con 0x7fe5941599c0 2023-11-06T03:17:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59406c520 con 0x7fe59406d000 2023-11-06T03:17:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe59415dea0 0x7fe5940b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe59415dea0 0x7fe5940b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51950/0 (socket says 172.21.15.157:51950) 2023-11-06T03:17:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 -- 172.21.15.157:0/3744235505 learned_addr learned my addr 172.21.15.157:0/3744235505 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 -- 172.21.15.157:0/3744235505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 msgr2=0x7fe5940b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5999de640 1 --2- 172.21.15.157:0/3744235505 >> v2:172.21.15.195:3300/0 conn(0x7fe5941599c0 0x7fe5940ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5991dd640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 0x7fe5940b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 0x7fe5940b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 -- 172.21.15.157:0/3744235505 >> v2:172.21.15.195:3300/0 conn(0x7fe5941599c0 msgr2=0x7fe5940ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3744235505 >> v2:172.21.15.195:3300/0 conn(0x7fe5941599c0 0x7fe5940ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 -- 172.21.15.157:0/3744235505 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59406c800 con 0x7fe59415dea0 2023-11-06T03:17:44.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5991dd640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 0x7fe5940b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe5940b2920 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fe594066800 tx=0x7fe58c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c31e82eb069631 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5989dc640 1 -- 172.21.15.157:0/3744235505 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0a5840 con 0x7fe59415dea0 2023-11-06T03:17:44.161 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5989dc640 1 -- 172.21.15.157:0/3744235505 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c0a59a0 con 0x7fe59415dea0 2023-11-06T03:17:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe5989dc640 1 -- 172.21.15.157:0/3744235505 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0a5d30 con 0x7fe59415dea0 2023-11-06T03:17:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- 172.21.15.157:0/3744235505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 msgr2=0x7fe5940b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe5940b2920 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fe594066800 tx=0x7fe58c01c9a0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.162 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- 172.21.15.157:0/3744235505 shutdown_connections 2023-11-06T03:17:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe5940b2920 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3744235505 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe59406d000 0x7fe5940b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3744235505 >> v2:172.21.15.195:3300/0 conn(0x7fe5941599c0 0x7fe5940ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- 172.21.15.157:0/3744235505 >> 172.21.15.157:0/3744235505 conn(0x7fe594065930 msgr2=0x7fe594065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:44.163 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- 172.21.15.157:0/3744235505 shutdown_connections 2023-11-06T03:17:44.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.158+0000 7fe59bc42640 1 -- 172.21.15.157:0/3744235505 wait complete. 2023-11-06T03:17:44.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 Processor -- start 2023-11-06T03:17:44.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- start start 2023-11-06T03:17:44.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59406d000 0x7fe594140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.164 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5941599c0 0x7fe594151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5941f65b0 con 0x7fe59415dea0 2023-11-06T03:17:44.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594151120 con 0x7fe59406d000 2023-11-06T03:17:44.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe594151400 con 0x7fe5941599c0 2023-11-06T03:17:44.165 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51952/0 (socket says 172.21.15.157:51952) 2023-11-06T03:17:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 -- 172.21.15.157:0/3427332391 learned_addr learned my addr 172.21.15.157:0/3427332391 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe5991dd640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5941599c0 0x7fe594151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe5999de640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59406d000 0x7fe594140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.166 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 -- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5941599c0 msgr2=0x7fe594151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5941599c0 0x7fe594151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 -- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59406d000 msgr2=0x7fe594140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59406d000 0x7fe594140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 -- 172.21.15.157:0/3427332391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c009b30 con 0x7fe59415dea0 2023-11-06T03:17:44.167 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59a1df640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fe58c01b050 tx=0x7fe58c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0b8670 con 0x7fe59415dea0 2023-11-06T03:17:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c0b87d0 con 0x7fe59415dea0 2023-11-06T03:17:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5941516e0 con 0x7fe59415dea0 2023-11-06T03:17:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0b8b60 con 0x7fe59415dea0 2023-11-06T03:17:44.168 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5941519c0 con 0x7fe59415dea0 2023-11-06T03:17:44.169 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.162+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5941f66f0 con 0x7fe59415dea0 2023-11-06T03:17:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.166+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe58c0b8cc0 con 0x7fe59415dea0 2023-11-06T03:17:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.166+0000 7fe5867fc640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 0x7fe578067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.166+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe58c11e930 con 0x7fe59415dea0 2023-11-06T03:17:44.170 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.166+0000 7fe5999de640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 0x7fe578067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.166+0000 7fe5999de640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 0x7fe578067f50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe59000a8f0 tx=0x7fe590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.170+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe5941f66f0 con 0x7fe59415dea0 2023-11-06T03:17:44.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.310+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5941f66f0 con 0x7fe578065a90 2023-11-06T03:17:44.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.330+0000 7fe5867fc640 1 -- 172.21.15.157:0/3427332391 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295725 (secure 0 0 0) 0x7fe5941f66f0 con 0x7fe578065a90 2023-11-06T03:17:44.332 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:44.334 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:17:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 msgr2=0x7fe578067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 0x7fe578067f50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe59000a8f0 tx=0x7fe590008040 comp rx=0 tx=0).stop 2023-11-06T03:17:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 msgr2=0x7fe594147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fe58c01b050 tx=0x7fe58c019740 comp rx=0 tx=0).stop 2023-11-06T03:17:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 shutdown_connections 2023-11-06T03:17:44.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe578065a90 0x7fe578067f50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe59415dea0 0x7fe594147410 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5941599c0 0x7fe594151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 --2- 172.21.15.157:0/3427332391 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe59406d000 0x7fe594140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 >> 172.21.15.157:0/3427332391 conn(0x7fe594065930 msgr2=0x7fe594150460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 shutdown_connections 2023-11-06T03:17:44.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.334+0000 7fe59bc42640 1 -- 172.21.15.157:0/3427332391 wait complete. 2023-11-06T03:17:44.348 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2023-11-06T03:17:43.391020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":643296,"kb_used_data":17888,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754331424,"statfs":{"total":773094113280,"available":772435378176,"internally_reserved":0,"allocated":18317312,"data_stored":9117671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001179"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475742+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:17:37.475742+0000","last_peered":"2023-11-06T03:17:37.475742+0000","last_clean":"2023-11-06T03:17:37.475742+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:17:37.475742+0000","last_undegraded":"2023-11-06T03:17:37.475742+0000","last_fullsized":"2023-11-06T03:17:37.475742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284344+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:17:37.284344+0000","last_peered":"2023-11-06T03:17:37.284344+0000","last_clean":"2023-11-06T03:17:37.284344+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:17:37.284344+0000","last_undegraded":"2023-11-06T03:17:37.284344+0000","last_fullsized":"2023-11-06T03:17:37.284344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475712+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:17:37.475712+0000","last_peered":"2023-11-06T03:17:37.475712+0000","last_clean":"2023-11-06T03:17:37.475712+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:17:37.475712+0000","last_undegraded":"2023-11-06T03:17:37.475712+0000","last_fullsized":"2023-11-06T03:17:37.475712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.312915+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:17:37.312915+0000","last_peered":"2023-11-06T03:17:37.312915+0000","last_clean":"2023-11-06T03:17:37.312915+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:17:37.312915+0000","last_undegraded":"2023-11-06T03:17:37.312915+0000","last_fullsized":"2023-11-06T03:17:37.312915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284366+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:17:37.284366+0000","last_peered":"2023-11-06T03:17:37.284366+0000","last_clean":"2023-11-06T03:17:37.284366+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:17:37.284366+0000","last_undegraded":"2023-11-06T03:17:37.284366+0000","last_fullsized":"2023-11-06T03:17:37.284366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596644+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:17:36.596644+0000","last_peered":"2023-11-06T03:17:36.596644+0000","last_clean":"2023-11-06T03:17:36.596644+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:17:36.596644+0000","last_undegraded":"2023-11-06T03:17:36.596644+0000","last_fullsized":"2023-11-06T03:17:36.596644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475721+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:17:37.475721+0000","last_peered":"2023-11-06T03:17:37.475721+0000","last_clean":"2023-11-06T03:17:37.475721+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:17:37.475721+0000","last_undegraded":"2023-11-06T03:17:37.475721+0000","last_fullsized":"2023-11-06T03:17:37.475721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596640+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:17:36.596640+0000","last_peered":"2023-11-06T03:17:36.596640+0000","last_clean":"2023-11-06T03:17:36.596640+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:17:36.596640+0000","last_undegraded":"2023-11-06T03:17:36.596640+0000","last_fullsized":"2023-11-06T03:17:36.596640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.285676+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:17:37.285676+0000","last_peered":"2023-11-06T03:17:37.285676+0000","last_clean":"2023-11-06T03:17:37.285676+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:17:37.285676+0000","last_undegraded":"2023-11-06T03:17:37.285676+0000","last_fullsized":"2023-11-06T03:17:37.285676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91904,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94279936,"statfs":{"total":96636764160,"available":96542654464,"internally_reserved":0,"allocated":2744320,"data_stored":1586292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":1.319}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":1.008},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":1.413},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":1.2749999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":1.056}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":1.165}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69164,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302676,"statfs":{"total":96636764160,"available":96565940224,"internally_reserved":0,"allocated":2134016,"data_stored":989425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":1.26}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":1.5169999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":1.321},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":1.716}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":1.351},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.2889999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.625}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47088,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324752,"statfs":{"total":96636764160,"available":96588546048,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":1.1419999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":1.1739999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.2},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":1.4470000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113468,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258372,"statfs":{"total":96636764160,"available":96520572928,"internally_reserved":0,"allocated":2150400,"data_stored":995924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.83899999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":1.012},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.75}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":1.127},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.0469999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":1.081}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":1.103},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83192,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288648,"statfs":{"total":96636764160,"available":96551575552,"internally_reserved":0,"allocated":2146304,"data_stored":994905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":0.55500000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":0.497},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":0.378},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":0.34599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":0.91500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73264,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298576,"statfs":{"total":96636764160,"available":96561741824,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.62},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":1.018},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":1.175}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":1.419},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69168,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302672,"statfs":{"total":96636764160,"available":96565936128,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.80400000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":0.624},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96048,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275792,"statfs":{"total":96636764160,"available":96538411008,"internally_reserved":0,"allocated":2727936,"data_stored":1579793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.95799999999999996,"5min":1.0149999999999999,"15min":2.6989999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61399999999999999,"15min":0.65000000000000002},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.008,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.20599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.1619999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":4.6920000000000002},"last":0.81599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.89100000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":0.98199999999999998,"5min":1.391,"15min":2.7999999999999998},"last":0.85899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.419,"5min":1.419,"15min":4.641},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.67500000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.4159999999999999,"15min":2.028},"last":0.73399999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.292,"5min":1.492,"15min":2.8559999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.224,"5min":1.496,"15min":4.7430000000000003},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91300000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.3220000000000001,"15min":2.931},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.69699999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.446,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":1.022}]},{"osd":7,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.95799999999999996,"15min":0.99099999999999999},"min":{"1min":0.73999999999999999,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.323,"5min":1.381,"15min":4.8339999999999996},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2569999999999999,"5min":1.335,"15min":1.97},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:17:44.355 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:17:44.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 Processor -- start 2023-11-06T03:17:44.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 -- start start 2023-11-06T03:17:44.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20b80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b80d2920 con 0x7f20b80d00b0 2023-11-06T03:17:44.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b80d18a0 con 0x7f20b815caa0 2023-11-06T03:17:44.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20bcc04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80d0520 con 0x7f20b815ce80 2023-11-06T03:17:44.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b6d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f20b80d00b0 0x7f20b80b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51066/0 (socket says 172.21.15.157:51066) 2023-11-06T03:17:44.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b6d9d640 1 -- 172.21.15.157:0/1984105270 learned_addr learned my addr 172.21.15.157:0/1984105270 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:44.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20b80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180720406 0 0) 0x7f20b80d2920 con 0x7f20b80d00b0 2023-11-06T03:17:44.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a0003530 con 0x7f20b80d00b0 2023-11-06T03:17:44.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 309321861 0 0) 0x7f20a0003530 con 0x7f20b80d00b0 2023-11-06T03:17:44.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 msgr2=0x7f20b80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20b80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 >> v1:172.21.15.195:6789/0 conn(0x7f20b815caa0 legacy=0x7f20b80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80d1b60 con 0x7f20b80d00b0 2023-11-06T03:17:44.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f20b0001a00 con 0x7f20b80d00b0 2023-11-06T03:17:44.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 >> v1:172.21.15.157:6789/0 conn(0x7f20b80d00b0 legacy=0x7f20b80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20a00039b0 0x7f20a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00047d0 0x7f20a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80d0520 con 0x7f20a00039b0 2023-11-06T03:17:44.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a000ba70 con 0x7f20a00047d0 2023-11-06T03:17:44.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a000bc30 con 0x7f20b815ce80 2023-11-06T03:17:44.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20a00039b0 0x7f20a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00047d0 0x7f20a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b6d9d640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 -- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 msgr2=0x7f20a0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815ce80 0x7f20a0005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 -- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00047d0 msgr2=0x7f20a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00047d0 0x7f20a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b659c640 1 -- 172.21.15.157:0/1984105270 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0003530 con 0x7f20a00039b0 2023-11-06T03:17:44.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.510+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00047d0 0x7f20a0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:44.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b659c640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20a00039b0 0x7f20a0003dd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f20a8001750 tx=0x7f20a801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e1d90b0782fca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80052d0 con 0x7f20a00039b0 2023-11-06T03:17:44.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80a7050 con 0x7f20a00039b0 2023-11-06T03:17:44.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b559a640 1 -- 172.21.15.157:0/1984105270 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8003600 con 0x7f20a00039b0 2023-11-06T03:17:44.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- 172.21.15.157:0/1984105270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20a00039b0 msgr2=0x7f20a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 --2- 172.21.15.157:0/1984105270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20a00039b0 0x7f20a0003dd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f20a8001750 tx=0x7f20a801c370 comp rx=0 tx=0).stop 2023-11-06T03:17:44.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b6d9d640 1 -- 172.21.15.157:0/1984105270 reap_dead start 2023-11-06T03:17:44.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- 172.21.15.157:0/1984105270 shutdown_connections 2023-11-06T03:17:44.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- 172.21.15.157:0/1984105270 >> 172.21.15.157:0/1984105270 conn(0x7f20b8070790 msgr2=0x7f20b8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:44.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- 172.21.15.157:0/1984105270 shutdown_connections 2023-11-06T03:17:44.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- 172.21.15.157:0/1984105270 wait complete. 2023-11-06T03:17:44.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 Processor -- start 2023-11-06T03:17:44.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- start start 2023-11-06T03:17:44.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 0x7f20b80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815dc50 0x7f20b815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80ca230 con 0x7f20b815ee20 2023-11-06T03:17:44.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815d360 con 0x7f20a00039b0 2023-11-06T03:17:44.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20bcc04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815d640 con 0x7f20b815dc50 2023-11-06T03:17:44.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b5d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51970/0 (socket says 172.21.15.157:51970) 2023-11-06T03:17:44.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.514+0000 7f20b5d9b640 1 -- 172.21.15.157:0/2638696662 learned_addr learned my addr 172.21.15.157:0/2638696662 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:44.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b659c640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 0x7f20b80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 -- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815dc50 msgr2=0x7f20b815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815dc50 0x7f20b815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 -- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 msgr2=0x7f20b80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 0x7f20b80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 -- 172.21.15.157:0/2638696662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b815e920 con 0x7f20b815ee20 2023-11-06T03:17:44.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b659c640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 0x7f20b80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:44.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b5d9b640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f20ac010890 tx=0x7f20ac010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.527 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004220 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac004380 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b8156c80 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00eb80 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b8160020 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b8072750 con 0x7f20b815ee20 2023-11-06T03:17:44.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f20ac00ece0 con 0x7f20b815ee20 2023-11-06T03:17:44.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 0x7f2098067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20ac083470 con 0x7f20b815ee20 2023-11-06T03:17:44.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b659c640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 0x7f2098067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.518+0000 7f20b659c640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 0x7f2098067f20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f20a00057b0 tx=0x7f20a80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.522+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f20ac050cd0 con 0x7f20b815ee20 2023-11-06T03:17:44.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.666+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20b80b1f00 con 0x7f2098065a60 2023-11-06T03:17:44.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20a6ffd640 1 -- 172.21.15.157:0/2638696662 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295725 (secure 0 0 0) 0x7f20a8005410 con 0x7f2098065a60 2023-11-06T03:17:44.689 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:44.690 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:17:44.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 msgr2=0x7f2098067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 0x7f2098067f20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f20a00057b0 tx=0x7f20a80011a0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 msgr2=0x7f20b8156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f20ac010890 tx=0x7f20ac010d50 comp rx=0 tx=0).stop 2023-11-06T03:17:44.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 shutdown_connections 2023-11-06T03:17:44.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2098065a60 0x7f2098067f20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20b815dc50 0x7f20b815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20b815ee20 0x7f20b8156740 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 --2- 172.21.15.157:0/2638696662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20a00039b0 0x7f20b80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 >> 172.21.15.157:0/2638696662 conn(0x7f20b8070790 msgr2=0x7f20b80b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:44.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 shutdown_connections 2023-11-06T03:17:44.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.690+0000 7f20bcc04640 1 -- 172.21.15.157:0/2638696662 wait complete. 2023-11-06T03:17:44.705 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2023-11-06T03:17:43.391020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":643296,"kb_used_data":17888,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754331424,"statfs":{"total":773094113280,"available":772435378176,"internally_reserved":0,"allocated":18317312,"data_stored":9117671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001179"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475742+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:17:37.475742+0000","last_peered":"2023-11-06T03:17:37.475742+0000","last_clean":"2023-11-06T03:17:37.475742+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:17:37.475742+0000","last_undegraded":"2023-11-06T03:17:37.475742+0000","last_fullsized":"2023-11-06T03:17:37.475742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284344+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:17:37.284344+0000","last_peered":"2023-11-06T03:17:37.284344+0000","last_clean":"2023-11-06T03:17:37.284344+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:17:37.284344+0000","last_undegraded":"2023-11-06T03:17:37.284344+0000","last_fullsized":"2023-11-06T03:17:37.284344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475712+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:17:37.475712+0000","last_peered":"2023-11-06T03:17:37.475712+0000","last_clean":"2023-11-06T03:17:37.475712+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:17:37.475712+0000","last_undegraded":"2023-11-06T03:17:37.475712+0000","last_fullsized":"2023-11-06T03:17:37.475712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.312915+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:17:37.312915+0000","last_peered":"2023-11-06T03:17:37.312915+0000","last_clean":"2023-11-06T03:17:37.312915+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:17:37.312915+0000","last_undegraded":"2023-11-06T03:17:37.312915+0000","last_fullsized":"2023-11-06T03:17:37.312915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.284366+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:17:37.284366+0000","last_peered":"2023-11-06T03:17:37.284366+0000","last_clean":"2023-11-06T03:17:37.284366+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:17:37.284366+0000","last_undegraded":"2023-11-06T03:17:37.284366+0000","last_fullsized":"2023-11-06T03:17:37.284366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596644+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:17:36.596644+0000","last_peered":"2023-11-06T03:17:36.596644+0000","last_clean":"2023-11-06T03:17:36.596644+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:17:36.596644+0000","last_undegraded":"2023-11-06T03:17:36.596644+0000","last_fullsized":"2023-11-06T03:17:36.596644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.475721+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:17:37.475721+0000","last_peered":"2023-11-06T03:17:37.475721+0000","last_clean":"2023-11-06T03:17:37.475721+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:17:37.475721+0000","last_undegraded":"2023-11-06T03:17:37.475721+0000","last_fullsized":"2023-11-06T03:17:37.475721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:36.596640+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:17:36.596640+0000","last_peered":"2023-11-06T03:17:36.596640+0000","last_clean":"2023-11-06T03:17:36.596640+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:17:36.596640+0000","last_undegraded":"2023-11-06T03:17:36.596640+0000","last_fullsized":"2023-11-06T03:17:36.596640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:40.572684+0000","last_change":"2023-11-06T03:17:40.572684+0000","last_active":"2023-11-06T03:17:40.572684+0000","last_peered":"2023-11-06T03:17:40.572684+0000","last_clean":"2023-11-06T03:17:40.572684+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:40.572684+0000","last_undegraded":"2023-11-06T03:17:40.572684+0000","last_fullsized":"2023-11-06T03:17:40.572684+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-06T03:17:39.560744+0000","last_change":"2023-11-06T03:17:39.560744+0000","last_active":"2023-11-06T03:17:39.560744+0000","last_peered":"2023-11-06T03:17:39.560744+0000","last_clean":"2023-11-06T03:17:39.560744+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-06T03:17:39.560744+0000","last_undegraded":"2023-11-06T03:17:39.560744+0000","last_fullsized":"2023-11-06T03:17:39.560744+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-11-06T03:17:37.285676+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:17:37.285676+0000","last_peered":"2023-11-06T03:17:37.285676+0000","last_clean":"2023-11-06T03:17:37.285676+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:17:37.285676+0000","last_undegraded":"2023-11-06T03:17:37.285676+0000","last_fullsized":"2023-11-06T03:17:37.285676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91904,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94279936,"statfs":{"total":96636764160,"available":96542654464,"internally_reserved":0,"allocated":2744320,"data_stored":1586292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":1.319}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":1.008},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":1.413},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":1.2749999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":1.056}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":1.165}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69164,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302676,"statfs":{"total":96636764160,"available":96565940224,"internally_reserved":0,"allocated":2134016,"data_stored":989425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":1.26}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":1.5169999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":1.321},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":1.716}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":1.351},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.2889999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.625}]}]},{"osd":5,"up_from":13,"seq":55834575017,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47088,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324752,"statfs":{"total":96636764160,"available":96588546048,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":1.1419999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":1.1739999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.2},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.2609999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":1.4470000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113468,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258372,"statfs":{"total":96636764160,"available":96520572928,"internally_reserved":0,"allocated":2150400,"data_stored":995924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.83899999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":1.012},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.75}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":1.127},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.0469999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":1.081}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":1.103},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83192,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288648,"statfs":{"total":96636764160,"available":96551575552,"internally_reserved":0,"allocated":2146304,"data_stored":994905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":0.55500000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":0.497},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":0.378},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":0.34599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":0.91500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73264,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298576,"statfs":{"total":96636764160,"available":96561741824,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.62},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":1.018},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":1.175}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":1.419},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69168,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302672,"statfs":{"total":96636764160,"available":96565936128,"internally_reserved":0,"allocated":2138112,"data_stored":990444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.80400000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":0.624},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575015,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96048,"kb_used_data":2664,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275792,"statfs":{"total":96636764160,"available":96538411008,"internally_reserved":0,"allocated":2727936,"data_stored":1579793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.95799999999999996,"5min":1.0149999999999999,"15min":2.6989999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61399999999999999,"15min":0.65000000000000002},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.008,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.20599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.1619999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":4.6920000000000002},"last":0.81599999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.89100000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":0.98199999999999998,"5min":1.391,"15min":2.7999999999999998},"last":0.85899999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.419,"5min":1.419,"15min":4.641},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.67500000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.4159999999999999,"15min":2.028},"last":0.73399999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.292,"5min":1.492,"15min":2.8559999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.224,"5min":1.496,"15min":4.7430000000000003},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91300000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.3220000000000001,"15min":2.931},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.69699999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.446,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":1.022}]},{"osd":7,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.95799999999999996,"15min":0.99099999999999999},"min":{"1min":0.73999999999999999,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.323,"5min":1.381,"15min":4.8339999999999996},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2569999999999999,"5min":1.335,"15min":1.97},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:17:44.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3f is not active+clean 2023-11-06T03:17:44.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3e is not active+clean 2023-11-06T03:17:44.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3d is not active+clean 2023-11-06T03:17:44.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3c is not active+clean 2023-11-06T03:17:44.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3b is not active+clean 2023-11-06T03:17:44.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3a is not active+clean 2023-11-06T03:17:44.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.39 is not active+clean 2023-11-06T03:17:44.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.38 is not active+clean 2023-11-06T03:17:44.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.37 is not active+clean 2023-11-06T03:17:44.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.36 is not active+clean 2023-11-06T03:17:44.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.35 is not active+clean 2023-11-06T03:17:44.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.34 is not active+clean 2023-11-06T03:17:44.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.33 is not active+clean 2023-11-06T03:17:44.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.32 is not active+clean 2023-11-06T03:17:44.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.31 is not active+clean 2023-11-06T03:17:44.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.30 is not active+clean 2023-11-06T03:17:44.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2f is not active+clean 2023-11-06T03:17:44.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2e is not active+clean 2023-11-06T03:17:44.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2d is not active+clean 2023-11-06T03:17:44.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2c is not active+clean 2023-11-06T03:17:44.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2b is not active+clean 2023-11-06T03:17:44.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2a is not active+clean 2023-11-06T03:17:44.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.29 is not active+clean 2023-11-06T03:17:44.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.28 is not active+clean 2023-11-06T03:17:44.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.27 is not active+clean 2023-11-06T03:17:44.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.732 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.26 is not active+clean 2023-11-06T03:17:44.732 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.733 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.25 is not active+clean 2023-11-06T03:17:44.733 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.24 is not active+clean 2023-11-06T03:17:44.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.23 is not active+clean 2023-11-06T03:17:44.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.22 is not active+clean 2023-11-06T03:17:44.735 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.21 is not active+clean 2023-11-06T03:17:44.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.20 is not active+clean 2023-11-06T03:17:44.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.14 is not active+clean 2023-11-06T03:17:44.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.17 is not active+clean 2023-11-06T03:17:44.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.15 is not active+clean 2023-11-06T03:17:44.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.16 is not active+clean 2023-11-06T03:17:44.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.16 is not active+clean 2023-11-06T03:17:44.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.15 is not active+clean 2023-11-06T03:17:44.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.17 is not active+clean 2023-11-06T03:17:44.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.14 is not active+clean 2023-11-06T03:17:44.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.10 is not active+clean 2023-11-06T03:17:44.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.13 is not active+clean 2023-11-06T03:17:44.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.11 is not active+clean 2023-11-06T03:17:44.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.12 is not active+clean 2023-11-06T03:17:44.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.12 is not active+clean 2023-11-06T03:17:44.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.11 is not active+clean 2023-11-06T03:17:44.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.13 is not active+clean 2023-11-06T03:17:44.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.10 is not active+clean 2023-11-06T03:17:44.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.c is not active+clean 2023-11-06T03:17:44.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.f is not active+clean 2023-11-06T03:17:44.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.d is not active+clean 2023-11-06T03:17:44.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.e is not active+clean 2023-11-06T03:17:44.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.e is not active+clean 2023-11-06T03:17:44.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2023-11-06T03:17:44.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.f is not active+clean 2023-11-06T03:17:44.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.c is not active+clean 2023-11-06T03:17:44.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.8 is not active+clean 2023-11-06T03:17:44.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.b is not active+clean 2023-11-06T03:17:44.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.9 is not active+clean 2023-11-06T03:17:44.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.a is not active+clean 2023-11-06T03:17:44.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.a is not active+clean 2023-11-06T03:17:44.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.9 is not active+clean 2023-11-06T03:17:44.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.b is not active+clean 2023-11-06T03:17:44.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2023-11-06T03:17:44.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.4 is not active+clean 2023-11-06T03:17:44.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.7 is not active+clean 2023-11-06T03:17:44.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.5 is not active+clean 2023-11-06T03:17:44.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.6 is not active+clean 2023-11-06T03:17:44.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.6 is not active+clean 2023-11-06T03:17:44.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.5 is not active+clean 2023-11-06T03:17:44.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.7 is not active+clean 2023-11-06T03:17:44.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.4 is not active+clean 2023-11-06T03:17:44.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.0 is not active+clean 2023-11-06T03:17:44.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3 is not active+clean 2023-11-06T03:17:44.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1 is not active+clean 2023-11-06T03:17:44.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2 is not active+clean 2023-11-06T03:17:44.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.2 is not active+clean 2023-11-06T03:17:44.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1 is not active+clean 2023-11-06T03:17:44.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.3 is not active+clean 2023-11-06T03:17:44.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2023-11-06T03:17:44.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1b is not active+clean 2023-11-06T03:17:44.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.18 is not active+clean 2023-11-06T03:17:44.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.18 is not active+clean 2023-11-06T03:17:44.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1b is not active+clean 2023-11-06T03:17:44.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.19 is not active+clean 2023-11-06T03:17:44.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1a is not active+clean 2023-11-06T03:17:44.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1a is not active+clean 2023-11-06T03:17:44.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.19 is not active+clean 2023-11-06T03:17:44.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1c is not active+clean 2023-11-06T03:17:44.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1f is not active+clean 2023-11-06T03:17:44.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1d is not active+clean 2023-11-06T03:17:44.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1e is not active+clean 2023-11-06T03:17:44.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1e is not active+clean 2023-11-06T03:17:44.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1d is not active+clean 2023-11-06T03:17:44.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 26.1f is not active+clean 2023-11-06T03:17:44.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '26.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:40.572684+0000', 'last_change': '2023-11-06T03:17:40.572684+0000', 'last_active': '2023-11-06T03:17:40.572684+0000', 'last_peered': '2023-11-06T03:17:40.572684+0000', 'last_clean': '2023-11-06T03:17:40.572684+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:40.572684+0000', 'last_undegraded': '2023-11-06T03:17:40.572684+0000', 'last_fullsized': '2023-11-06T03:17:40.572684+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:40.572684+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1c is not active+clean 2023-11-06T03:17:44.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-06T03:17:39.560744+0000', 'last_change': '2023-11-06T03:17:39.560744+0000', 'last_active': '2023-11-06T03:17:39.560744+0000', 'last_peered': '2023-11-06T03:17:39.560744+0000', 'last_clean': '2023-11-06T03:17:39.560744+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-06T03:17:39.560744+0000', 'last_undegraded': '2023-11-06T03:17:39.560744+0000', 'last_fullsized': '2023-11-06T03:17:39.560744+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'last_clean_scrub_stamp': '2023-11-06T03:17:39.560744+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-06T03:17:44.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-06T03:17:44.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 Processor -- start 2023-11-06T03:17:44.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 -- start start 2023-11-06T03:17:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8500cde90 0x7fa8500cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cfed0 0x7fa8500d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa850160830 con 0x7fa85015cc70 2023-11-06T03:17:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8501614b0 con 0x7fa8500cde90 2023-11-06T03:17:44.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa8550ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500d0360 con 0x7fa8500cfed0 2023-11-06T03:17:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa84e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8500cde90 0x7fa8500cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa84e59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa8500cde90 0x7fa8500cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51974/0 (socket says 172.21.15.157:51974) 2023-11-06T03:17:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.910+0000 7fa84e59c640 1 -- 172.21.15.157:0/2534149205 learned_addr learned my addr 172.21.15.157:0/2534149205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:44.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84f59e640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cfed0 0x7fa8500d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84e59c640 1 -- 172.21.15.157:0/2534149205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cfed0 msgr2=0x7fa8500d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:44.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84e59c640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cfed0 0x7fa8500d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84e59c640 1 -- 172.21.15.157:0/2534149205 >> v1:172.21.15.195:6789/0 conn(0x7fa85015cc70 legacy=0x7fa8500d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84e59c640 1 -- 172.21.15.157:0/2534149205 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850160b10 con 0x7fa8500cde90 2023-11-06T03:17:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84e59c640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8500cde90 0x7fa8500cf990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa83801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb4ba8f47d0a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84dd9b640 1 -- 172.21.15.157:0/2534149205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380a8a30 con 0x7fa8500cde90 2023-11-06T03:17:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84dd9b640 1 -- 172.21.15.157:0/2534149205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8380a8b90 con 0x7fa8500cde90 2023-11-06T03:17:44.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa84dd9b640 1 -- 172.21.15.157:0/2534149205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380b0440 con 0x7fa8500cde90 2023-11-06T03:17:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 -- 172.21.15.157:0/2534149205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8500cde90 msgr2=0x7fa8500cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8500cde90 0x7fa8500cf990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa83801ed30 comp rx=0 tx=0).stop 2023-11-06T03:17:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 -- 172.21.15.157:0/2534149205 shutdown_connections 2023-11-06T03:17:44.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cfed0 0x7fa8500d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 --2- 172.21.15.157:0/2534149205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8500cde90 0x7fa8500cf990 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 -- 172.21.15.157:0/2534149205 >> 172.21.15.157:0/2534149205 conn(0x7fa850070560 msgr2=0x7fa850070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 -- 172.21.15.157:0/2534149205 shutdown_connections 2023-11-06T03:17:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.914+0000 7fa8550ae640 1 -- 172.21.15.157:0/2534149205 wait complete. 2023-11-06T03:17:44.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 Processor -- start 2023-11-06T03:17:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- start start 2023-11-06T03:17:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 0x7fa8501f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 0x7fa85015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850156370 con 0x7fa85015cc70 2023-11-06T03:17:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501f2900 con 0x7fa8500cfed0 2023-11-06T03:17:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500c9e00 con 0x7fa8500cde90 2023-11-06T03:17:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51980/0 (socket says 172.21.15.157:51980) 2023-11-06T03:17:44.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 -- 172.21.15.157:0/478145322 learned_addr learned my addr 172.21.15.157:0/478145322 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:44.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84e59c640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 0x7fa85015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84ed9d640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 0x7fa8501f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 -- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 msgr2=0x7fa8501f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 0x7fa8501f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 -- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 msgr2=0x7fa85015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:44.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 0x7fa85015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:44.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 -- 172.21.15.157:0/478145322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8380094d0 con 0x7fa85015cc70 2023-11-06T03:17:44.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84e59c640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 0x7fa85015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:44.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84ed9d640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 0x7fa8501f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:44.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa84f59e640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e930 tx=0x7fa84800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480096b0 con 0x7fa85015cc70 2023-11-06T03:17:44.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84800c9a0 con 0x7fa85015cc70 2023-11-06T03:17:44.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800cd30 con 0x7fa85015cc70 2023-11-06T03:17:44.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8500ca0e0 con 0x7fa85015cc70 2023-11-06T03:17:44.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8500ca6a0 con 0x7fa85015cc70 2023-11-06T03:17:44.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa850072750 con 0x7fa85015cc70 2023-11-06T03:17:44.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.918+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa848013470 con 0x7fa85015cc70 2023-11-06T03:17:44.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.922+0000 7fa8477fe640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 0x7fa830067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:44.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.922+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8480835d0 con 0x7fa85015cc70 2023-11-06T03:17:44.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.922+0000 7fa84ed9d640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 0x7fa830067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:44.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.922+0000 7fa84ed9d640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 0x7fa830067f20 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa840007fd0 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:44.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:44.926+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa848050e60 con 0x7fa85015cc70 2023-11-06T03:17:45.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.122+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa8500b51f0 con 0x7fa85015cc70 2023-11-06T03:17:45.122 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8477fe640 1 -- 172.21.15.157:0/478145322 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1577 (secure 0 0 0) 0x7fa848054480 con 0x7fa85015cc70 2023-11-06T03:17:45.122 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:45.123 INFO:teuthology.orchestra.run.smithi157.stdout:{"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":842,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":175,"num_osds":8,"num_up_osds":8,"osd_up_since":1699239833,"num_in_osds":8,"osd_in_since":1699239825,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":658735104,"bytes_avail":772435378176,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":123,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":10818}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":37,"modified":"2023-11-06T03:17:01.376559+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-06T03:17:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 msgr2=0x7fa830067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.124 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 0x7fa830067f20 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa840007fd0 tx=0x7fa840009040 comp rx=0 tx=0).stop 2023-11-06T03:17:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 msgr2=0x7fa8501f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e930 tx=0x7fa84800edf0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 shutdown_connections 2023-11-06T03:17:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa830065a60 0x7fa830067f20 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa85015cc70 0x7fa8501f5050 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8500cfed0 0x7fa85015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 --2- 172.21.15.157:0/478145322 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa8500cde90 0x7fa8501f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 >> 172.21.15.157:0/478145322 conn(0x7fa850070560 msgr2=0x7fa8500c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:45.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 shutdown_connections 2023-11-06T03:17:45.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.126+0000 7fa8550ae640 1 -- 172.21.15.157:0/478145322 wait complete. 2023-11-06T03:17:45.140 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-06T03:17:45.141 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:17:45.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.250+0000 7fb4f4898640 1 Processor -- start 2023-11-06T03:17:45.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.250+0000 7fb4f4898640 1 -- start start 2023-11-06T03:17:45.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4f4898640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb4f01551e0 0x7fb4f006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4f4898640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4f4898640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f015f580 con 0x7fb4f015e5e0 2023-11-06T03:17:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4f4898640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f015efc0 con 0x7fb4f01551e0 2023-11-06T03:17:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4f4898640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f015f2a0 con 0x7fb4f00b3b20 2023-11-06T03:17:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb4f01551e0 0x7fb4f006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4eed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4edd9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fb4f015e5e0 0x7fb4f00b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:44144/0 (socket says 172.21.15.157:44144) 2023-11-06T03:17:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb4f01551e0 0x7fb4f006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:53348/0 (socket says 172.21.15.157:53348) 2023-11-06T03:17:45.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4edd9b640 1 -- 172.21.15.157:0/4248109802 learned_addr learned my addr 172.21.15.157:0/4248109802 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:45.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 -- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 msgr2=0x7fb4f015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:45.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 -- 172.21.15.157:0/4248109802 >> v1:172.21.15.195:6789/0 conn(0x7fb4f015e5e0 legacy=0x7fb4f00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 -- 172.21.15.157:0/4248109802 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f006f5a0 con 0x7fb4f01551e0 2023-11-06T03:17:45.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4eed9d640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:17:45.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.254+0000 7fb4ee59c640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f006fda0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00097c0 tx=0x7fb4e001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f650d20e8eddf27c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:45.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4ed59a640 1 -- 172.21.15.157:0/4248109802 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00ac640 con 0x7fb4f01551e0 2023-11-06T03:17:45.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4ed59a640 1 -- 172.21.15.157:0/4248109802 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e00ac7a0 con 0x7fb4f01551e0 2023-11-06T03:17:45.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4ed59a640 1 -- 172.21.15.157:0/4248109802 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00acb30 con 0x7fb4f01551e0 2023-11-06T03:17:45.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 msgr2=0x7fb4f006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f006fda0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00097c0 tx=0x7fb4e001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/4248109802 shutdown_connections 2023-11-06T03:17:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- 172.21.15.157:0/4248109802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f006fda0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/4248109802 >> 172.21.15.157:0/4248109802 conn(0x7fb4f0065930 msgr2=0x7fb4f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/4248109802 shutdown_connections 2023-11-06T03:17:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/4248109802 wait complete. 2023-11-06T03:17:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 Processor -- start 2023-11-06T03:17:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- start start 2023-11-06T03:17:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f012f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f0151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00cdcc0 con 0x7fb4f015e5e0 2023-11-06T03:17:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0144790 con 0x7fb4f01551e0 2023-11-06T03:17:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f012f8d0 con 0x7fb4f00b3b20 2023-11-06T03:17:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:51988/0 (socket says 172.21.15.157:51988) 2023-11-06T03:17:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 -- 172.21.15.157:0/700098949 learned_addr learned my addr 172.21.15.157:0/700098949 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4ee59c640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f012f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4edd9b640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f0151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 -- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 msgr2=0x7fb4f012f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f012f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 -- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 msgr2=0x7fb4f0151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f0151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 -- 172.21.15.157:0/700098949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e8009a70 con 0x7fb4f015e5e0 2023-11-06T03:17:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4eed9d640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800efb0 tx=0x7fb4e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8014480 con 0x7fb4f015e5e0 2023-11-06T03:17:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e800f040 con 0x7fb4f015e5e0 2023-11-06T03:17:45.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8014800 con 0x7fb4f015e5e0 2023-11-06T03:17:45.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e0009420 con 0x7fb4f015e5e0 2023-11-06T03:17:45.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.258+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f01444b0 con 0x7fb4f015e5e0 2023-11-06T03:17:45.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f00cde00 con 0x7fb4f015e5e0 2023-11-06T03:17:45.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4e8014960 con 0x7fb4f015e5e0 2023-11-06T03:17:45.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4deffd640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 0x7fb4d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:45.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4e8084300 con 0x7fb4f015e5e0 2023-11-06T03:17:45.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4ee59c640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 0x7fb4d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:45.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.262+0000 7fb4ee59c640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 0x7fb4d0067f20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00ae000 tx=0x7fb4e00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:45.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.266+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb4f00cde00 con 0x7fb4f015e5e0 2023-11-06T03:17:45.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.402+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f015b1b0 con 0x7fb4d0065a60 2023-11-06T03:17:45.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.426+0000 7fb4deffd640 1 -- 172.21.15.157:0/700098949 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311015 (secure 0 0 0) 0x7fb4e00bc090 con 0x7fb4d0065a60 2023-11-06T03:17:45.428 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:45.430 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:17:45.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 msgr2=0x7fb4d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 0x7fb4d0067f20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00ae000 tx=0x7fb4e00a7000 comp rx=0 tx=0).stop 2023-11-06T03:17:45.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 msgr2=0x7fb4f01434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:45.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800efb0 tx=0x7fb4e800c490 comp rx=0 tx=0).stop 2023-11-06T03:17:45.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 shutdown_connections 2023-11-06T03:17:45.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb4d0065a60 0x7fb4d0067f20 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f015e5e0 0x7fb4f01434a0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4f01551e0 0x7fb4f0151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 --2- 172.21.15.157:0/700098949 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb4f00b3b20 0x7fb4f012f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:45.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 >> 172.21.15.157:0/700098949 conn(0x7fb4f0065930 msgr2=0x7fb4f00ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:45.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 shutdown_connections 2023-11-06T03:17:45.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:45.430+0000 7fb4f4898640 1 -- 172.21.15.157:0/700098949 wait complete. 2023-11-06T03:17:45.444 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2023-11-06T03:17:45.391690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":643804,"kb_used_data":18396,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754330916,"statfs":{"total":773094113280,"available":772434857984,"internally_reserved":0,"allocated":18837504,"data_stored":9362427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.815628"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167958+0000","last_change":"2023-11-06T03:17:41.018474+0000","last_active":"2023-11-06T03:17:42.167958+0000","last_peered":"2023-11-06T03:17:42.167958+0000","last_clean":"2023-11-06T03:17:42.167958+0000","last_became_active":"2023-11-06T03:17:41.018204+0000","last_became_peered":"2023-11-06T03:17:41.018204+0000","last_unstale":"2023-11-06T03:17:42.167958+0000","last_undegraded":"2023-11-06T03:17:42.167958+0000","last_fullsized":"2023-11-06T03:17:42.167958+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:40:18.156403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167900+0000","last_change":"2023-11-06T03:17:41.040598+0000","last_active":"2023-11-06T03:17:42.167900+0000","last_peered":"2023-11-06T03:17:42.167900+0000","last_clean":"2023-11-06T03:17:42.167900+0000","last_became_active":"2023-11-06T03:17:41.040274+0000","last_became_peered":"2023-11-06T03:17:41.040274+0000","last_unstale":"2023-11-06T03:17:42.167900+0000","last_undegraded":"2023-11-06T03:17:42.167900+0000","last_fullsized":"2023-11-06T03:17:42.167900+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:56.954833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169710+0000","last_change":"2023-11-06T03:17:41.010277+0000","last_active":"2023-11-06T03:17:42.169710+0000","last_peered":"2023-11-06T03:17:42.169710+0000","last_clean":"2023-11-06T03:17:42.169710+0000","last_became_active":"2023-11-06T03:17:41.009478+0000","last_became_peered":"2023-11-06T03:17:41.009478+0000","last_unstale":"2023-11-06T03:17:42.169710+0000","last_undegraded":"2023-11-06T03:17:42.169710+0000","last_fullsized":"2023-11-06T03:17:42.169710+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:09.838757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982812+0000","last_change":"2023-11-06T03:17:41.028802+0000","last_active":"2023-11-06T03:17:41.982812+0000","last_peered":"2023-11-06T03:17:41.982812+0000","last_clean":"2023-11-06T03:17:41.982812+0000","last_became_active":"2023-11-06T03:17:41.028557+0000","last_became_peered":"2023-11-06T03:17:41.028557+0000","last_unstale":"2023-11-06T03:17:41.982812+0000","last_undegraded":"2023-11-06T03:17:41.982812+0000","last_fullsized":"2023-11-06T03:17:41.982812+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:17.837799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166463+0000","last_change":"2023-11-06T03:17:41.024792+0000","last_active":"2023-11-06T03:17:42.166463+0000","last_peered":"2023-11-06T03:17:42.166463+0000","last_clean":"2023-11-06T03:17:42.166463+0000","last_became_active":"2023-11-06T03:17:41.024503+0000","last_became_peered":"2023-11-06T03:17:41.024503+0000","last_unstale":"2023-11-06T03:17:42.166463+0000","last_undegraded":"2023-11-06T03:17:42.166463+0000","last_fullsized":"2023-11-06T03:17:42.166463+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:24:19.734187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166238+0000","last_change":"2023-11-06T03:17:41.000262+0000","last_active":"2023-11-06T03:17:42.166238+0000","last_peered":"2023-11-06T03:17:42.166238+0000","last_clean":"2023-11-06T03:17:42.166238+0000","last_became_active":"2023-11-06T03:17:40.999827+0000","last_became_peered":"2023-11-06T03:17:40.999827+0000","last_unstale":"2023-11-06T03:17:42.166238+0000","last_undegraded":"2023-11-06T03:17:42.166238+0000","last_fullsized":"2023-11-06T03:17:42.166238+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:13.141458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.165987+0000","last_change":"2023-11-06T03:17:41.016307+0000","last_active":"2023-11-06T03:17:42.165987+0000","last_peered":"2023-11-06T03:17:42.165987+0000","last_clean":"2023-11-06T03:17:42.165987+0000","last_became_active":"2023-11-06T03:17:41.016038+0000","last_became_peered":"2023-11-06T03:17:41.016038+0000","last_unstale":"2023-11-06T03:17:42.165987+0000","last_undegraded":"2023-11-06T03:17:42.165987+0000","last_fullsized":"2023-11-06T03:17:42.165987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:59:49.993695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170230+0000","last_change":"2023-11-06T03:17:41.026233+0000","last_active":"2023-11-06T03:17:42.170230+0000","last_peered":"2023-11-06T03:17:42.170230+0000","last_clean":"2023-11-06T03:17:42.170230+0000","last_became_active":"2023-11-06T03:17:41.025664+0000","last_became_peered":"2023-11-06T03:17:41.025664+0000","last_unstale":"2023-11-06T03:17:42.170230+0000","last_undegraded":"2023-11-06T03:17:42.170230+0000","last_fullsized":"2023-11-06T03:17:42.170230+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:07.200307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168079+0000","last_change":"2023-11-06T03:17:41.009266+0000","last_active":"2023-11-06T03:17:42.168079+0000","last_peered":"2023-11-06T03:17:42.168079+0000","last_clean":"2023-11-06T03:17:42.168079+0000","last_became_active":"2023-11-06T03:17:41.008975+0000","last_became_peered":"2023-11-06T03:17:41.008975+0000","last_unstale":"2023-11-06T03:17:42.168079+0000","last_undegraded":"2023-11-06T03:17:42.168079+0000","last_fullsized":"2023-11-06T03:17:42.168079+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:10.769533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166423+0000","last_change":"2023-11-06T03:17:41.037272+0000","last_active":"2023-11-06T03:17:42.166423+0000","last_peered":"2023-11-06T03:17:42.166423+0000","last_clean":"2023-11-06T03:17:42.166423+0000","last_became_active":"2023-11-06T03:17:41.036991+0000","last_became_peered":"2023-11-06T03:17:41.036991+0000","last_unstale":"2023-11-06T03:17:42.166423+0000","last_undegraded":"2023-11-06T03:17:42.166423+0000","last_fullsized":"2023-11-06T03:17:42.166423+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:17.082356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172042+0000","last_change":"2023-11-06T03:17:41.024975+0000","last_active":"2023-11-06T03:17:42.172042+0000","last_peered":"2023-11-06T03:17:42.172042+0000","last_clean":"2023-11-06T03:17:42.172042+0000","last_became_active":"2023-11-06T03:17:41.024315+0000","last_became_peered":"2023-11-06T03:17:41.024315+0000","last_unstale":"2023-11-06T03:17:42.172042+0000","last_undegraded":"2023-11-06T03:17:42.172042+0000","last_fullsized":"2023-11-06T03:17:42.172042+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:54.286022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982880+0000","last_change":"2023-11-06T03:17:41.011960+0000","last_active":"2023-11-06T03:17:41.982880+0000","last_peered":"2023-11-06T03:17:41.982880+0000","last_clean":"2023-11-06T03:17:41.982880+0000","last_became_active":"2023-11-06T03:17:41.011710+0000","last_became_peered":"2023-11-06T03:17:41.011710+0000","last_unstale":"2023-11-06T03:17:41.982880+0000","last_undegraded":"2023-11-06T03:17:41.982880+0000","last_fullsized":"2023-11-06T03:17:41.982880+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:17.082356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988151+0000","last_change":"2023-11-06T03:17:41.026274+0000","last_active":"2023-11-06T03:17:41.988151+0000","last_peered":"2023-11-06T03:17:41.988151+0000","last_clean":"2023-11-06T03:17:41.988151+0000","last_became_active":"2023-11-06T03:17:41.025484+0000","last_became_peered":"2023-11-06T03:17:41.025484+0000","last_unstale":"2023-11-06T03:17:41.988151+0000","last_undegraded":"2023-11-06T03:17:41.988151+0000","last_fullsized":"2023-11-06T03:17:41.988151+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:26.138790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166485+0000","last_change":"2023-11-06T03:17:41.036938+0000","last_active":"2023-11-06T03:17:42.166485+0000","last_peered":"2023-11-06T03:17:42.166485+0000","last_clean":"2023-11-06T03:17:42.166485+0000","last_became_active":"2023-11-06T03:17:41.036312+0000","last_became_peered":"2023-11-06T03:17:41.036312+0000","last_unstale":"2023-11-06T03:17:42.166485+0000","last_undegraded":"2023-11-06T03:17:42.166485+0000","last_fullsized":"2023-11-06T03:17:42.166485+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:51.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167643+0000","last_change":"2023-11-06T03:17:41.036816+0000","last_active":"2023-11-06T03:17:42.167643+0000","last_peered":"2023-11-06T03:17:42.167643+0000","last_clean":"2023-11-06T03:17:42.167643+0000","last_became_active":"2023-11-06T03:17:41.036533+0000","last_became_peered":"2023-11-06T03:17:41.036533+0000","last_unstale":"2023-11-06T03:17:42.167643+0000","last_undegraded":"2023-11-06T03:17:42.167643+0000","last_fullsized":"2023-11-06T03:17:42.167643+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:57.056216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166068+0000","last_change":"2023-11-06T03:17:41.040016+0000","last_active":"2023-11-06T03:17:42.166068+0000","last_peered":"2023-11-06T03:17:42.166068+0000","last_clean":"2023-11-06T03:17:42.166068+0000","last_became_active":"2023-11-06T03:17:41.039696+0000","last_became_peered":"2023-11-06T03:17:41.039696+0000","last_unstale":"2023-11-06T03:17:42.166068+0000","last_undegraded":"2023-11-06T03:17:42.166068+0000","last_fullsized":"2023-11-06T03:17:42.166068+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:00.233658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166316+0000","last_change":"2023-11-06T03:17:41.011470+0000","last_active":"2023-11-06T03:17:42.166316+0000","last_peered":"2023-11-06T03:17:42.166316+0000","last_clean":"2023-11-06T03:17:42.166316+0000","last_became_active":"2023-11-06T03:17:41.011175+0000","last_became_peered":"2023-11-06T03:17:41.011175+0000","last_unstale":"2023-11-06T03:17:42.166316+0000","last_undegraded":"2023-11-06T03:17:42.166316+0000","last_fullsized":"2023-11-06T03:17:42.166316+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:55.431825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988728+0000","last_change":"2023-11-06T03:17:41.031251+0000","last_active":"2023-11-06T03:17:41.988728+0000","last_peered":"2023-11-06T03:17:41.988728+0000","last_clean":"2023-11-06T03:17:41.988728+0000","last_became_active":"2023-11-06T03:17:41.030557+0000","last_became_peered":"2023-11-06T03:17:41.030557+0000","last_unstale":"2023-11-06T03:17:41.988728+0000","last_undegraded":"2023-11-06T03:17:41.988728+0000","last_fullsized":"2023-11-06T03:17:41.988728+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:19.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988642+0000","last_change":"2023-11-06T03:17:41.024567+0000","last_active":"2023-11-06T03:17:41.988642+0000","last_peered":"2023-11-06T03:17:41.988642+0000","last_clean":"2023-11-06T03:17:41.988642+0000","last_became_active":"2023-11-06T03:17:41.023883+0000","last_became_peered":"2023-11-06T03:17:41.023883+0000","last_unstale":"2023-11-06T03:17:41.988642+0000","last_undegraded":"2023-11-06T03:17:41.988642+0000","last_fullsized":"2023-11-06T03:17:41.988642+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:01.801910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166110+0000","last_change":"2023-11-06T03:17:41.012112+0000","last_active":"2023-11-06T03:17:42.166110+0000","last_peered":"2023-11-06T03:17:42.166110+0000","last_clean":"2023-11-06T03:17:42.166110+0000","last_became_active":"2023-11-06T03:17:41.011771+0000","last_became_peered":"2023-11-06T03:17:41.011771+0000","last_unstale":"2023-11-06T03:17:42.166110+0000","last_undegraded":"2023-11-06T03:17:42.166110+0000","last_fullsized":"2023-11-06T03:17:42.166110+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:41.479552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167810+0000","last_change":"2023-11-06T03:17:41.009205+0000","last_active":"2023-11-06T03:17:42.167810+0000","last_peered":"2023-11-06T03:17:42.167810+0000","last_clean":"2023-11-06T03:17:42.167810+0000","last_became_active":"2023-11-06T03:17:41.008984+0000","last_became_peered":"2023-11-06T03:17:41.008984+0000","last_unstale":"2023-11-06T03:17:42.167810+0000","last_undegraded":"2023-11-06T03:17:42.167810+0000","last_fullsized":"2023-11-06T03:17:42.167810+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:12:46.556231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167911+0000","last_change":"2023-11-06T03:17:41.010065+0000","last_active":"2023-11-06T03:17:42.167911+0000","last_peered":"2023-11-06T03:17:42.167911+0000","last_clean":"2023-11-06T03:17:42.167911+0000","last_became_active":"2023-11-06T03:17:41.009671+0000","last_became_peered":"2023-11-06T03:17:41.009671+0000","last_unstale":"2023-11-06T03:17:42.167911+0000","last_undegraded":"2023-11-06T03:17:42.167911+0000","last_fullsized":"2023-11-06T03:17:42.167911+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:59.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170093+0000","last_change":"2023-11-06T03:17:41.016082+0000","last_active":"2023-11-06T03:17:42.170093+0000","last_peered":"2023-11-06T03:17:42.170093+0000","last_clean":"2023-11-06T03:17:42.170093+0000","last_became_active":"2023-11-06T03:17:41.015301+0000","last_became_peered":"2023-11-06T03:17:41.015301+0000","last_unstale":"2023-11-06T03:17:42.170093+0000","last_undegraded":"2023-11-06T03:17:42.170093+0000","last_fullsized":"2023-11-06T03:17:42.170093+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:38:44.882747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167610+0000","last_change":"2023-11-06T03:17:41.020101+0000","last_active":"2023-11-06T03:17:42.167610+0000","last_peered":"2023-11-06T03:17:42.167610+0000","last_clean":"2023-11-06T03:17:42.167610+0000","last_became_active":"2023-11-06T03:17:41.019822+0000","last_became_peered":"2023-11-06T03:17:41.019822+0000","last_unstale":"2023-11-06T03:17:42.167610+0000","last_undegraded":"2023-11-06T03:17:42.167610+0000","last_fullsized":"2023-11-06T03:17:42.167610+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:57:50.185204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167201+0000","last_change":"2023-11-06T03:17:40.994370+0000","last_active":"2023-11-06T03:17:42.167201+0000","last_peered":"2023-11-06T03:17:42.167201+0000","last_clean":"2023-11-06T03:17:42.167201+0000","last_became_active":"2023-11-06T03:17:40.994083+0000","last_became_peered":"2023-11-06T03:17:40.994083+0000","last_unstale":"2023-11-06T03:17:42.167201+0000","last_undegraded":"2023-11-06T03:17:42.167201+0000","last_fullsized":"2023-11-06T03:17:42.167201+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:15.895346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172093+0000","last_change":"2023-11-06T03:17:41.026258+0000","last_active":"2023-11-06T03:17:42.172093+0000","last_peered":"2023-11-06T03:17:42.172093+0000","last_clean":"2023-11-06T03:17:42.172093+0000","last_became_active":"2023-11-06T03:17:41.025181+0000","last_became_peered":"2023-11-06T03:17:41.025181+0000","last_unstale":"2023-11-06T03:17:42.172093+0000","last_undegraded":"2023-11-06T03:17:42.172093+0000","last_fullsized":"2023-11-06T03:17:42.172093+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:58.810371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170002+0000","last_change":"2023-11-06T03:17:41.013430+0000","last_active":"2023-11-06T03:17:42.170002+0000","last_peered":"2023-11-06T03:17:42.170002+0000","last_clean":"2023-11-06T03:17:42.170002+0000","last_became_active":"2023-11-06T03:17:41.012327+0000","last_became_peered":"2023-11-06T03:17:41.012327+0000","last_unstale":"2023-11-06T03:17:42.170002+0000","last_undegraded":"2023-11-06T03:17:42.170002+0000","last_fullsized":"2023-11-06T03:17:42.170002+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:38:30.244459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166246+0000","last_change":"2023-11-06T03:17:41.023744+0000","last_active":"2023-11-06T03:17:42.166246+0000","last_peered":"2023-11-06T03:17:42.166246+0000","last_clean":"2023-11-06T03:17:42.166246+0000","last_became_active":"2023-11-06T03:17:41.023472+0000","last_became_peered":"2023-11-06T03:17:41.023472+0000","last_unstale":"2023-11-06T03:17:42.166246+0000","last_undegraded":"2023-11-06T03:17:42.166246+0000","last_fullsized":"2023-11-06T03:17:42.166246+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:58.093338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171616+0000","last_change":"2023-11-06T03:17:41.016178+0000","last_active":"2023-11-06T03:17:42.171616+0000","last_peered":"2023-11-06T03:17:42.171616+0000","last_clean":"2023-11-06T03:17:42.171616+0000","last_became_active":"2023-11-06T03:17:41.015325+0000","last_became_peered":"2023-11-06T03:17:41.015325+0000","last_unstale":"2023-11-06T03:17:42.171616+0000","last_undegraded":"2023-11-06T03:17:42.171616+0000","last_fullsized":"2023-11-06T03:17:42.171616+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:22.155917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171095+0000","last_change":"2023-11-06T03:17:41.016417+0000","last_active":"2023-11-06T03:17:42.171095+0000","last_peered":"2023-11-06T03:17:42.171095+0000","last_clean":"2023-11-06T03:17:42.171095+0000","last_became_active":"2023-11-06T03:17:41.015601+0000","last_became_peered":"2023-11-06T03:17:41.015601+0000","last_unstale":"2023-11-06T03:17:42.171095+0000","last_undegraded":"2023-11-06T03:17:42.171095+0000","last_fullsized":"2023-11-06T03:17:42.171095+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:17.202855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166732+0000","last_change":"2023-11-06T03:17:40.999623+0000","last_active":"2023-11-06T03:17:42.166732+0000","last_peered":"2023-11-06T03:17:42.166732+0000","last_clean":"2023-11-06T03:17:42.166732+0000","last_became_active":"2023-11-06T03:17:40.999327+0000","last_became_peered":"2023-11-06T03:17:40.999327+0000","last_unstale":"2023-11-06T03:17:42.166732+0000","last_undegraded":"2023-11-06T03:17:42.166732+0000","last_fullsized":"2023-11-06T03:17:42.166732+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:30.549578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166561+0000","last_change":"2023-11-06T03:17:41.042233+0000","last_active":"2023-11-06T03:17:42.166561+0000","last_peered":"2023-11-06T03:17:42.166561+0000","last_clean":"2023-11-06T03:17:42.166561+0000","last_became_active":"2023-11-06T03:17:41.041984+0000","last_became_peered":"2023-11-06T03:17:41.041984+0000","last_unstale":"2023-11-06T03:17:42.166561+0000","last_undegraded":"2023-11-06T03:17:42.166561+0000","last_fullsized":"2023-11-06T03:17:42.166561+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:59:58.707564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170692+0000","last_change":"2023-11-06T03:17:41.039597+0000","last_active":"2023-11-06T03:17:42.170692+0000","last_peered":"2023-11-06T03:17:42.170692+0000","last_clean":"2023-11-06T03:17:42.170692+0000","last_became_active":"2023-11-06T03:17:41.038995+0000","last_became_peered":"2023-11-06T03:17:41.038995+0000","last_unstale":"2023-11-06T03:17:42.170692+0000","last_undegraded":"2023-11-06T03:17:42.170692+0000","last_fullsized":"2023-11-06T03:17:42.170692+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:01.801910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166392+0000","last_change":"2023-11-06T03:17:40.609942+0000","last_active":"2023-11-06T03:17:42.166392+0000","last_peered":"2023-11-06T03:17:42.166392+0000","last_clean":"2023-11-06T03:17:42.166392+0000","last_became_active":"2023-11-06T03:17:40.609458+0000","last_became_peered":"2023-11-06T03:17:40.609458+0000","last_unstale":"2023-11-06T03:17:42.166392+0000","last_undegraded":"2023-11-06T03:17:42.166392+0000","last_fullsized":"2023-11-06T03:17:42.166392+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:49.390773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166947+0000","last_change":"2023-11-06T03:17:40.992918+0000","last_active":"2023-11-06T03:17:42.166947+0000","last_peered":"2023-11-06T03:17:42.166947+0000","last_clean":"2023-11-06T03:17:42.166947+0000","last_became_active":"2023-11-06T03:17:40.992620+0000","last_became_peered":"2023-11-06T03:17:40.992620+0000","last_unstale":"2023-11-06T03:17:42.166947+0000","last_undegraded":"2023-11-06T03:17:42.166947+0000","last_fullsized":"2023-11-06T03:17:42.166947+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:57.056216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167982+0000","last_change":"2023-11-06T03:17:40.611372+0000","last_active":"2023-11-06T03:17:42.167982+0000","last_peered":"2023-11-06T03:17:42.167982+0000","last_clean":"2023-11-06T03:17:42.167982+0000","last_became_active":"2023-11-06T03:17:40.611003+0000","last_became_peered":"2023-11-06T03:17:40.611003+0000","last_unstale":"2023-11-06T03:17:42.167982+0000","last_undegraded":"2023-11-06T03:17:42.167982+0000","last_fullsized":"2023-11-06T03:17:42.167982+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:55:26.610548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988817+0000","last_change":"2023-11-06T03:17:41.027967+0000","last_active":"2023-11-06T03:17:41.988817+0000","last_peered":"2023-11-06T03:17:41.988817+0000","last_clean":"2023-11-06T03:17:41.988817+0000","last_became_active":"2023-11-06T03:17:41.027363+0000","last_became_peered":"2023-11-06T03:17:41.027363+0000","last_unstale":"2023-11-06T03:17:41.988817+0000","last_undegraded":"2023-11-06T03:17:41.988817+0000","last_fullsized":"2023-11-06T03:17:41.988817+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:58.282317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"174'3","reported_seq":14,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982898+0000","last_change":"2023-11-06T03:17:40.609910+0000","last_active":"2023-11-06T03:17:41.982898+0000","last_peered":"2023-11-06T03:17:41.982898+0000","last_clean":"2023-11-06T03:17:41.982898+0000","last_became_active":"2023-11-06T03:17:40.609545+0000","last_became_peered":"2023-11-06T03:17:40.609545+0000","last_unstale":"2023-11-06T03:17:41.982898+0000","last_undegraded":"2023-11-06T03:17:41.982898+0000","last_fullsized":"2023-11-06T03:17:41.982898+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:14.067635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168076+0000","last_change":"2023-11-06T03:17:41.039473+0000","last_active":"2023-11-06T03:17:42.168076+0000","last_peered":"2023-11-06T03:17:42.168076+0000","last_clean":"2023-11-06T03:17:42.168076+0000","last_became_active":"2023-11-06T03:17:41.039213+0000","last_became_peered":"2023-11-06T03:17:41.039213+0000","last_unstale":"2023-11-06T03:17:42.168076+0000","last_undegraded":"2023-11-06T03:17:42.168076+0000","last_fullsized":"2023-11-06T03:17:42.168076+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:12.988831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"174'6","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988484+0000","last_change":"2023-11-06T03:17:40.620815+0000","last_active":"2023-11-06T03:17:41.988484+0000","last_peered":"2023-11-06T03:17:41.988484+0000","last_clean":"2023-11-06T03:17:41.988484+0000","last_became_active":"2023-11-06T03:17:40.619326+0000","last_became_peered":"2023-11-06T03:17:40.619326+0000","last_unstale":"2023-11-06T03:17:41.988484+0000","last_undegraded":"2023-11-06T03:17:41.988484+0000","last_fullsized":"2023-11-06T03:17:41.988484+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:39.662107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.165971+0000","last_change":"2023-11-06T03:17:41.012387+0000","last_active":"2023-11-06T03:17:42.165971+0000","last_peered":"2023-11-06T03:17:42.165971+0000","last_clean":"2023-11-06T03:17:42.165971+0000","last_became_active":"2023-11-06T03:17:41.012051+0000","last_became_peered":"2023-11-06T03:17:41.012051+0000","last_unstale":"2023-11-06T03:17:42.165971+0000","last_undegraded":"2023-11-06T03:17:42.165971+0000","last_fullsized":"2023-11-06T03:17:42.165971+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:16.143274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988408+0000","last_change":"2023-11-06T03:17:40.627765+0000","last_active":"2023-11-06T03:17:41.988408+0000","last_peered":"2023-11-06T03:17:41.988408+0000","last_clean":"2023-11-06T03:17:41.988408+0000","last_became_active":"2023-11-06T03:17:40.624009+0000","last_became_peered":"2023-11-06T03:17:40.624009+0000","last_unstale":"2023-11-06T03:17:41.988408+0000","last_undegraded":"2023-11-06T03:17:41.988408+0000","last_fullsized":"2023-11-06T03:17:41.988408+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:21.143976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166125+0000","last_change":"2023-11-06T03:17:41.017369+0000","last_active":"2023-11-06T03:17:42.166125+0000","last_peered":"2023-11-06T03:17:42.166125+0000","last_clean":"2023-11-06T03:17:42.166125+0000","last_became_active":"2023-11-06T03:17:41.017082+0000","last_became_peered":"2023-11-06T03:17:41.017082+0000","last_unstale":"2023-11-06T03:17:42.166125+0000","last_undegraded":"2023-11-06T03:17:42.166125+0000","last_fullsized":"2023-11-06T03:17:42.166125+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:07.469836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168579+0000","last_change":"2023-11-06T03:17:40.614381+0000","last_active":"2023-11-06T03:17:42.168579+0000","last_peered":"2023-11-06T03:17:42.168579+0000","last_clean":"2023-11-06T03:17:42.168579+0000","last_became_active":"2023-11-06T03:17:40.614049+0000","last_became_peered":"2023-11-06T03:17:40.614049+0000","last_unstale":"2023-11-06T03:17:42.168579+0000","last_undegraded":"2023-11-06T03:17:42.168579+0000","last_fullsized":"2023-11-06T03:17:42.168579+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:08.596679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987828+0000","last_change":"2023-11-06T03:17:41.024674+0000","last_active":"2023-11-06T03:17:41.987828+0000","last_peered":"2023-11-06T03:17:41.987828+0000","last_clean":"2023-11-06T03:17:41.987828+0000","last_became_active":"2023-11-06T03:17:41.008571+0000","last_became_peered":"2023-11-06T03:17:41.008571+0000","last_unstale":"2023-11-06T03:17:41.987828+0000","last_undegraded":"2023-11-06T03:17:41.987828+0000","last_fullsized":"2023-11-06T03:17:41.987828+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:50.515130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982666+0000","last_change":"2023-11-06T03:17:40.621561+0000","last_active":"2023-11-06T03:17:41.982666+0000","last_peered":"2023-11-06T03:17:41.982666+0000","last_clean":"2023-11-06T03:17:41.982666+0000","last_became_active":"2023-11-06T03:17:40.621261+0000","last_became_peered":"2023-11-06T03:17:40.621261+0000","last_unstale":"2023-11-06T03:17:41.982666+0000","last_undegraded":"2023-11-06T03:17:41.982666+0000","last_fullsized":"2023-11-06T03:17:41.982666+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:54.419885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988152+0000","last_change":"2023-11-06T03:17:41.029051+0000","last_active":"2023-11-06T03:17:41.988152+0000","last_peered":"2023-11-06T03:17:41.988152+0000","last_clean":"2023-11-06T03:17:41.988152+0000","last_became_active":"2023-11-06T03:17:41.028368+0000","last_became_peered":"2023-11-06T03:17:41.028368+0000","last_unstale":"2023-11-06T03:17:41.988152+0000","last_undegraded":"2023-11-06T03:17:41.988152+0000","last_fullsized":"2023-11-06T03:17:41.988152+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:35.770871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"174'8","reported_seq":19,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171038+0000","last_change":"2023-11-06T03:17:40.615897+0000","last_active":"2023-11-06T03:17:42.171038+0000","last_peered":"2023-11-06T03:17:42.171038+0000","last_clean":"2023-11-06T03:17:42.171038+0000","last_became_active":"2023-11-06T03:17:40.615133+0000","last_became_peered":"2023-11-06T03:17:40.615133+0000","last_unstale":"2023-11-06T03:17:42.171038+0000","last_undegraded":"2023-11-06T03:17:42.171038+0000","last_fullsized":"2023-11-06T03:17:42.171038+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:57:07.911861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166838+0000","last_change":"2023-11-06T03:17:40.998864+0000","last_active":"2023-11-06T03:17:42.166838+0000","last_peered":"2023-11-06T03:17:42.166838+0000","last_clean":"2023-11-06T03:17:42.166838+0000","last_became_active":"2023-11-06T03:17:40.998558+0000","last_became_peered":"2023-11-06T03:17:40.998558+0000","last_unstale":"2023-11-06T03:17:42.166838+0000","last_undegraded":"2023-11-06T03:17:42.166838+0000","last_fullsized":"2023-11-06T03:17:42.166838+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:08.835610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172510+0000","last_change":"2023-11-06T03:17:40.624836+0000","last_active":"2023-11-06T03:17:42.172510+0000","last_peered":"2023-11-06T03:17:42.172510+0000","last_clean":"2023-11-06T03:17:42.172510+0000","last_became_active":"2023-11-06T03:17:40.624094+0000","last_became_peered":"2023-11-06T03:17:40.624094+0000","last_unstale":"2023-11-06T03:17:42.172510+0000","last_undegraded":"2023-11-06T03:17:42.172510+0000","last_fullsized":"2023-11-06T03:17:42.172510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:58:01.437634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166800+0000","last_change":"2023-11-06T03:17:41.042244+0000","last_active":"2023-11-06T03:17:42.166800+0000","last_peered":"2023-11-06T03:17:42.166800+0000","last_clean":"2023-11-06T03:17:42.166800+0000","last_became_active":"2023-11-06T03:17:41.042004+0000","last_became_peered":"2023-11-06T03:17:41.042004+0000","last_unstale":"2023-11-06T03:17:42.166800+0000","last_undegraded":"2023-11-06T03:17:42.166800+0000","last_fullsized":"2023-11-06T03:17:42.166800+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:20.426342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983199+0000","last_change":"2023-11-06T03:17:40.621557+0000","last_active":"2023-11-06T03:17:41.983199+0000","last_peered":"2023-11-06T03:17:41.983199+0000","last_clean":"2023-11-06T03:17:41.983199+0000","last_became_active":"2023-11-06T03:17:40.621236+0000","last_became_peered":"2023-11-06T03:17:40.621236+0000","last_unstale":"2023-11-06T03:17:41.983199+0000","last_undegraded":"2023-11-06T03:17:41.983199+0000","last_fullsized":"2023-11-06T03:17:41.983199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:57.081398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172420+0000","last_change":"2023-11-06T03:17:41.011334+0000","last_active":"2023-11-06T03:17:42.172420+0000","last_peered":"2023-11-06T03:17:42.172420+0000","last_clean":"2023-11-06T03:17:42.172420+0000","last_became_active":"2023-11-06T03:17:41.010527+0000","last_became_peered":"2023-11-06T03:17:41.010527+0000","last_unstale":"2023-11-06T03:17:42.172420+0000","last_undegraded":"2023-11-06T03:17:42.172420+0000","last_fullsized":"2023-11-06T03:17:42.172420+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:40.674047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169903+0000","last_change":"2023-11-06T03:17:40.630051+0000","last_active":"2023-11-06T03:17:42.169903+0000","last_peered":"2023-11-06T03:17:42.169903+0000","last_clean":"2023-11-06T03:17:42.169903+0000","last_became_active":"2023-11-06T03:17:40.629340+0000","last_became_peered":"2023-11-06T03:17:40.629340+0000","last_unstale":"2023-11-06T03:17:42.169903+0000","last_undegraded":"2023-11-06T03:17:42.169903+0000","last_fullsized":"2023-11-06T03:17:42.169903+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:04.388942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168140+0000","last_change":"2023-11-06T03:17:40.998928+0000","last_active":"2023-11-06T03:17:42.168140+0000","last_peered":"2023-11-06T03:17:42.168140+0000","last_clean":"2023-11-06T03:17:42.168140+0000","last_became_active":"2023-11-06T03:17:40.998679+0000","last_became_peered":"2023-11-06T03:17:40.998679+0000","last_unstale":"2023-11-06T03:17:42.168140+0000","last_undegraded":"2023-11-06T03:17:42.168140+0000","last_fullsized":"2023-11-06T03:17:42.168140+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:51.103931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171803+0000","last_change":"2023-11-06T03:17:40.622390+0000","last_active":"2023-11-06T03:17:42.171803+0000","last_peered":"2023-11-06T03:17:42.171803+0000","last_clean":"2023-11-06T03:17:42.171803+0000","last_became_active":"2023-11-06T03:17:40.620356+0000","last_became_peered":"2023-11-06T03:17:40.620356+0000","last_unstale":"2023-11-06T03:17:42.171803+0000","last_undegraded":"2023-11-06T03:17:42.171803+0000","last_fullsized":"2023-11-06T03:17:42.171803+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:11:30.703687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167561+0000","last_change":"2023-11-06T03:17:41.001437+0000","last_active":"2023-11-06T03:17:42.167561+0000","last_peered":"2023-11-06T03:17:42.167561+0000","last_clean":"2023-11-06T03:17:42.167561+0000","last_became_active":"2023-11-06T03:17:41.001164+0000","last_became_peered":"2023-11-06T03:17:41.001164+0000","last_unstale":"2023-11-06T03:17:42.167561+0000","last_undegraded":"2023-11-06T03:17:42.167561+0000","last_fullsized":"2023-11-06T03:17:42.167561+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:19.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988421+0000","last_change":"2023-11-06T03:17:40.624113+0000","last_active":"2023-11-06T03:17:41.988421+0000","last_peered":"2023-11-06T03:17:41.988421+0000","last_clean":"2023-11-06T03:17:41.988421+0000","last_became_active":"2023-11-06T03:17:40.623376+0000","last_became_peered":"2023-11-06T03:17:40.623376+0000","last_unstale":"2023-11-06T03:17:41.988421+0000","last_undegraded":"2023-11-06T03:17:41.988421+0000","last_fullsized":"2023-11-06T03:17:41.988421+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:18:30.101591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982648+0000","last_change":"2023-11-06T03:17:41.028875+0000","last_active":"2023-11-06T03:17:41.982648+0000","last_peered":"2023-11-06T03:17:41.982648+0000","last_clean":"2023-11-06T03:17:41.982648+0000","last_became_active":"2023-11-06T03:17:41.028545+0000","last_became_peered":"2023-11-06T03:17:41.028545+0000","last_unstale":"2023-11-06T03:17:41.982648+0000","last_undegraded":"2023-11-06T03:17:41.982648+0000","last_fullsized":"2023-11-06T03:17:41.982648+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:12.624747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982930+0000","last_change":"2023-11-06T03:17:40.610459+0000","last_active":"2023-11-06T03:17:41.982930+0000","last_peered":"2023-11-06T03:17:41.982930+0000","last_clean":"2023-11-06T03:17:41.982930+0000","last_became_active":"2023-11-06T03:17:40.610020+0000","last_became_peered":"2023-11-06T03:17:40.610020+0000","last_unstale":"2023-11-06T03:17:41.982930+0000","last_undegraded":"2023-11-06T03:17:41.982930+0000","last_fullsized":"2023-11-06T03:17:41.982930+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:08.779800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983000+0000","last_change":"2023-11-06T03:17:41.007861+0000","last_active":"2023-11-06T03:17:41.983000+0000","last_peered":"2023-11-06T03:17:41.983000+0000","last_clean":"2023-11-06T03:17:41.983000+0000","last_became_active":"2023-11-06T03:17:41.007611+0000","last_became_peered":"2023-11-06T03:17:41.007611+0000","last_unstale":"2023-11-06T03:17:41.983000+0000","last_undegraded":"2023-11-06T03:17:41.983000+0000","last_fullsized":"2023-11-06T03:17:41.983000+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:30.256371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166960+0000","last_change":"2023-11-06T03:17:40.614263+0000","last_active":"2023-11-06T03:17:42.166960+0000","last_peered":"2023-11-06T03:17:42.166960+0000","last_clean":"2023-11-06T03:17:42.166960+0000","last_became_active":"2023-11-06T03:17:40.613762+0000","last_became_peered":"2023-11-06T03:17:40.613762+0000","last_unstale":"2023-11-06T03:17:42.166960+0000","last_undegraded":"2023-11-06T03:17:42.166960+0000","last_fullsized":"2023-11-06T03:17:42.166960+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:59:16.545555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166554+0000","last_change":"2023-11-06T03:17:40.995296+0000","last_active":"2023-11-06T03:17:42.166554+0000","last_peered":"2023-11-06T03:17:42.166554+0000","last_clean":"2023-11-06T03:17:42.166554+0000","last_became_active":"2023-11-06T03:17:40.994520+0000","last_became_peered":"2023-11-06T03:17:40.994520+0000","last_unstale":"2023-11-06T03:17:42.166554+0000","last_undegraded":"2023-11-06T03:17:42.166554+0000","last_fullsized":"2023-11-06T03:17:42.166554+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:09.791741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169147+0000","last_change":"2023-11-06T03:17:40.630355+0000","last_active":"2023-11-06T03:17:42.169147+0000","last_peered":"2023-11-06T03:17:42.169147+0000","last_clean":"2023-11-06T03:17:42.169147+0000","last_became_active":"2023-11-06T03:17:40.629747+0000","last_became_peered":"2023-11-06T03:17:40.629747+0000","last_unstale":"2023-11-06T03:17:42.169147+0000","last_undegraded":"2023-11-06T03:17:42.169147+0000","last_fullsized":"2023-11-06T03:17:42.169147+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:50:33.501645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166939+0000","last_change":"2023-11-06T03:17:41.014314+0000","last_active":"2023-11-06T03:17:42.166939+0000","last_peered":"2023-11-06T03:17:42.166939+0000","last_clean":"2023-11-06T03:17:42.166939+0000","last_became_active":"2023-11-06T03:17:41.014000+0000","last_became_peered":"2023-11-06T03:17:41.014000+0000","last_unstale":"2023-11-06T03:17:42.166939+0000","last_undegraded":"2023-11-06T03:17:42.166939+0000","last_fullsized":"2023-11-06T03:17:42.166939+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:41.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983319+0000","last_change":"2023-11-06T03:17:40.616582+0000","last_active":"2023-11-06T03:17:41.983319+0000","last_peered":"2023-11-06T03:17:41.983319+0000","last_clean":"2023-11-06T03:17:41.983319+0000","last_became_active":"2023-11-06T03:17:40.616219+0000","last_became_peered":"2023-11-06T03:17:40.616219+0000","last_unstale":"2023-11-06T03:17:41.983319+0000","last_undegraded":"2023-11-06T03:17:41.983319+0000","last_fullsized":"2023-11-06T03:17:41.983319+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:35.852152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167791+0000","last_change":"2023-11-06T03:17:40.998769+0000","last_active":"2023-11-06T03:17:42.167791+0000","last_peered":"2023-11-06T03:17:42.167791+0000","last_clean":"2023-11-06T03:17:42.167791+0000","last_became_active":"2023-11-06T03:17:40.998436+0000","last_became_peered":"2023-11-06T03:17:40.998436+0000","last_unstale":"2023-11-06T03:17:42.167791+0000","last_undegraded":"2023-11-06T03:17:42.167791+0000","last_fullsized":"2023-11-06T03:17:42.167791+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:10:22.447983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166441+0000","last_change":"2023-11-06T03:17:40.614117+0000","last_active":"2023-11-06T03:17:42.166441+0000","last_peered":"2023-11-06T03:17:42.166441+0000","last_clean":"2023-11-06T03:17:42.166441+0000","last_became_active":"2023-11-06T03:17:40.613778+0000","last_became_peered":"2023-11-06T03:17:40.613778+0000","last_unstale":"2023-11-06T03:17:42.166441+0000","last_undegraded":"2023-11-06T03:17:42.166441+0000","last_fullsized":"2023-11-06T03:17:42.166441+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:49.398376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170514+0000","last_change":"2023-11-06T03:17:41.020719+0000","last_active":"2023-11-06T03:17:42.170514+0000","last_peered":"2023-11-06T03:17:42.170514+0000","last_clean":"2023-11-06T03:17:42.170514+0000","last_became_active":"2023-11-06T03:17:41.020035+0000","last_became_peered":"2023-11-06T03:17:41.020035+0000","last_unstale":"2023-11-06T03:17:42.170514+0000","last_undegraded":"2023-11-06T03:17:42.170514+0000","last_fullsized":"2023-11-06T03:17:42.170514+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:35.284829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167021+0000","last_change":"2023-11-06T03:17:40.616588+0000","last_active":"2023-11-06T03:17:42.167021+0000","last_peered":"2023-11-06T03:17:42.167021+0000","last_clean":"2023-11-06T03:17:42.167021+0000","last_became_active":"2023-11-06T03:17:40.615929+0000","last_became_peered":"2023-11-06T03:17:40.615929+0000","last_unstale":"2023-11-06T03:17:42.167021+0000","last_undegraded":"2023-11-06T03:17:42.167021+0000","last_fullsized":"2023-11-06T03:17:42.167021+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:48:30.860619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167135+0000","last_change":"2023-11-06T03:17:41.014552+0000","last_active":"2023-11-06T03:17:42.167135+0000","last_peered":"2023-11-06T03:17:42.167135+0000","last_clean":"2023-11-06T03:17:42.167135+0000","last_became_active":"2023-11-06T03:17:41.013994+0000","last_became_peered":"2023-11-06T03:17:41.013994+0000","last_unstale":"2023-11-06T03:17:42.167135+0000","last_undegraded":"2023-11-06T03:17:42.167135+0000","last_fullsized":"2023-11-06T03:17:42.167135+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:09.838757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166322+0000","last_change":"2023-11-06T03:17:40.612745+0000","last_active":"2023-11-06T03:17:42.166322+0000","last_peered":"2023-11-06T03:17:42.166322+0000","last_clean":"2023-11-06T03:17:42.166322+0000","last_became_active":"2023-11-06T03:17:40.611530+0000","last_became_peered":"2023-11-06T03:17:40.611530+0000","last_unstale":"2023-11-06T03:17:42.166322+0000","last_undegraded":"2023-11-06T03:17:42.166322+0000","last_fullsized":"2023-11-06T03:17:42.166322+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:53.477947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168764+0000","last_change":"2023-11-06T03:17:41.007952+0000","last_active":"2023-11-06T03:17:42.168764+0000","last_peered":"2023-11-06T03:17:42.168764+0000","last_clean":"2023-11-06T03:17:42.168764+0000","last_became_active":"2023-11-06T03:17:41.007214+0000","last_became_peered":"2023-11-06T03:17:41.007214+0000","last_unstale":"2023-11-06T03:17:42.168764+0000","last_undegraded":"2023-11-06T03:17:42.168764+0000","last_fullsized":"2023-11-06T03:17:42.168764+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:30.549578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170791+0000","last_change":"2023-11-06T03:17:40.629908+0000","last_active":"2023-11-06T03:17:42.170791+0000","last_peered":"2023-11-06T03:17:42.170791+0000","last_clean":"2023-11-06T03:17:42.170791+0000","last_became_active":"2023-11-06T03:17:40.629303+0000","last_became_peered":"2023-11-06T03:17:40.629303+0000","last_unstale":"2023-11-06T03:17:42.170791+0000","last_undegraded":"2023-11-06T03:17:42.170791+0000","last_fullsized":"2023-11-06T03:17:42.170791+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:11.745377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987615+0000","last_change":"2023-11-06T03:17:41.024605+0000","last_active":"2023-11-06T03:17:41.987615+0000","last_peered":"2023-11-06T03:17:41.987615+0000","last_clean":"2023-11-06T03:17:41.987615+0000","last_became_active":"2023-11-06T03:17:41.012804+0000","last_became_peered":"2023-11-06T03:17:41.012804+0000","last_unstale":"2023-11-06T03:17:41.987615+0000","last_undegraded":"2023-11-06T03:17:41.987615+0000","last_fullsized":"2023-11-06T03:17:41.987615+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:28.308334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170105+0000","last_change":"2023-11-06T03:17:40.620927+0000","last_active":"2023-11-06T03:17:42.170105+0000","last_peered":"2023-11-06T03:17:42.170105+0000","last_clean":"2023-11-06T03:17:42.170105+0000","last_became_active":"2023-11-06T03:17:40.620120+0000","last_became_peered":"2023-11-06T03:17:40.620120+0000","last_unstale":"2023-11-06T03:17:42.170105+0000","last_undegraded":"2023-11-06T03:17:42.170105+0000","last_fullsized":"2023-11-06T03:17:42.170105+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:13.389503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168029+0000","last_change":"2023-11-06T03:17:40.998827+0000","last_active":"2023-11-06T03:17:42.168029+0000","last_peered":"2023-11-06T03:17:42.168029+0000","last_clean":"2023-11-06T03:17:42.168029+0000","last_became_active":"2023-11-06T03:17:40.998571+0000","last_became_peered":"2023-11-06T03:17:40.998571+0000","last_unstale":"2023-11-06T03:17:42.168029+0000","last_undegraded":"2023-11-06T03:17:42.168029+0000","last_fullsized":"2023-11-06T03:17:42.168029+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:54.572064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166815+0000","last_change":"2023-11-06T03:17:40.628490+0000","last_active":"2023-11-06T03:17:42.166815+0000","last_peered":"2023-11-06T03:17:42.166815+0000","last_clean":"2023-11-06T03:17:42.166815+0000","last_became_active":"2023-11-06T03:17:40.628150+0000","last_became_peered":"2023-11-06T03:17:40.628150+0000","last_unstale":"2023-11-06T03:17:42.166815+0000","last_undegraded":"2023-11-06T03:17:42.166815+0000","last_fullsized":"2023-11-06T03:17:42.166815+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:24.072944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170593+0000","last_change":"2023-11-06T03:17:41.013651+0000","last_active":"2023-11-06T03:17:42.170593+0000","last_peered":"2023-11-06T03:17:42.170593+0000","last_clean":"2023-11-06T03:17:42.170593+0000","last_became_active":"2023-11-06T03:17:41.012857+0000","last_became_peered":"2023-11-06T03:17:41.012857+0000","last_unstale":"2023-11-06T03:17:42.170593+0000","last_undegraded":"2023-11-06T03:17:42.170593+0000","last_fullsized":"2023-11-06T03:17:42.170593+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:09.608619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169004+0000","last_change":"2023-11-06T03:17:40.629864+0000","last_active":"2023-11-06T03:17:42.169004+0000","last_peered":"2023-11-06T03:17:42.169004+0000","last_clean":"2023-11-06T03:17:42.169004+0000","last_became_active":"2023-11-06T03:17:40.629548+0000","last_became_peered":"2023-11-06T03:17:40.629548+0000","last_unstale":"2023-11-06T03:17:42.169004+0000","last_undegraded":"2023-11-06T03:17:42.169004+0000","last_fullsized":"2023-11-06T03:17:42.169004+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:30.160280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169786+0000","last_change":"2023-11-06T03:17:41.007656+0000","last_active":"2023-11-06T03:17:42.169786+0000","last_peered":"2023-11-06T03:17:42.169786+0000","last_clean":"2023-11-06T03:17:42.169786+0000","last_became_active":"2023-11-06T03:17:41.006897+0000","last_became_peered":"2023-11-06T03:17:41.006897+0000","last_unstale":"2023-11-06T03:17:42.169786+0000","last_undegraded":"2023-11-06T03:17:42.169786+0000","last_fullsized":"2023-11-06T03:17:42.169786+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:51.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982841+0000","last_change":"2023-11-06T03:17:40.629328+0000","last_active":"2023-11-06T03:17:41.982841+0000","last_peered":"2023-11-06T03:17:41.982841+0000","last_clean":"2023-11-06T03:17:41.982841+0000","last_became_active":"2023-11-06T03:17:40.628988+0000","last_became_peered":"2023-11-06T03:17:40.628988+0000","last_unstale":"2023-11-06T03:17:41.982841+0000","last_undegraded":"2023-11-06T03:17:41.982841+0000","last_fullsized":"2023-11-06T03:17:41.982841+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:24:18.722246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":292,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.995891+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:17:40.995891+0000","last_peered":"2023-11-06T03:17:40.995891+0000","last_clean":"2023-11-06T03:17:40.995891+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:17:40.995891+0000","last_undegraded":"2023-11-06T03:17:40.995891+0000","last_fullsized":"2023-11-06T03:17:40.995891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.010323+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:17:41.010323+0000","last_peered":"2023-11-06T03:17:41.010323+0000","last_clean":"2023-11-06T03:17:41.010323+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:17:41.010323+0000","last_undegraded":"2023-11-06T03:17:41.010323+0000","last_fullsized":"2023-11-06T03:17:41.010323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987424+0000","last_change":"2023-11-06T03:17:41.025178+0000","last_active":"2023-11-06T03:17:41.987424+0000","last_peered":"2023-11-06T03:17:41.987424+0000","last_clean":"2023-11-06T03:17:41.987424+0000","last_became_active":"2023-11-06T03:17:41.007458+0000","last_became_peered":"2023-11-06T03:17:41.007458+0000","last_unstale":"2023-11-06T03:17:41.987424+0000","last_undegraded":"2023-11-06T03:17:41.987424+0000","last_fullsized":"2023-11-06T03:17:41.987424+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:23.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166697+0000","last_change":"2023-11-06T03:17:40.622167+0000","last_active":"2023-11-06T03:17:42.166697+0000","last_peered":"2023-11-06T03:17:42.166697+0000","last_clean":"2023-11-06T03:17:42.166697+0000","last_became_active":"2023-11-06T03:17:40.621897+0000","last_became_peered":"2023-11-06T03:17:40.621897+0000","last_unstale":"2023-11-06T03:17:42.166697+0000","last_undegraded":"2023-11-06T03:17:42.166697+0000","last_fullsized":"2023-11-06T03:17:42.166697+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:40.316206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.995598+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:17:40.995598+0000","last_peered":"2023-11-06T03:17:40.995598+0000","last_clean":"2023-11-06T03:17:40.995598+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:17:40.995598+0000","last_undegraded":"2023-11-06T03:17:40.995598+0000","last_fullsized":"2023-11-06T03:17:40.995598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167773+0000","last_change":"2023-11-06T03:17:40.995606+0000","last_active":"2023-11-06T03:17:42.167773+0000","last_peered":"2023-11-06T03:17:42.167773+0000","last_clean":"2023-11-06T03:17:42.167773+0000","last_became_active":"2023-11-06T03:17:40.995268+0000","last_became_peered":"2023-11-06T03:17:40.995268+0000","last_unstale":"2023-11-06T03:17:42.167773+0000","last_undegraded":"2023-11-06T03:17:42.167773+0000","last_fullsized":"2023-11-06T03:17:42.167773+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:23.288564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983055+0000","last_change":"2023-11-06T03:17:40.604424+0000","last_active":"2023-11-06T03:17:41.983055+0000","last_peered":"2023-11-06T03:17:41.983055+0000","last_clean":"2023-11-06T03:17:41.983055+0000","last_became_active":"2023-11-06T03:17:40.604114+0000","last_became_peered":"2023-11-06T03:17:40.604114+0000","last_unstale":"2023-11-06T03:17:41.983055+0000","last_undegraded":"2023-11-06T03:17:41.983055+0000","last_fullsized":"2023-11-06T03:17:41.983055+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:32.594095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.981930+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:17:40.981930+0000","last_peered":"2023-11-06T03:17:40.981930+0000","last_clean":"2023-11-06T03:17:40.981930+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:17:40.981930+0000","last_undegraded":"2023-11-06T03:17:40.981930+0000","last_fullsized":"2023-11-06T03:17:40.981930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168647+0000","last_change":"2023-11-06T03:17:41.036912+0000","last_active":"2023-11-06T03:17:42.168647+0000","last_peered":"2023-11-06T03:17:42.168647+0000","last_clean":"2023-11-06T03:17:42.168647+0000","last_became_active":"2023-11-06T03:17:41.036622+0000","last_became_peered":"2023-11-06T03:17:41.036622+0000","last_unstale":"2023-11-06T03:17:42.168647+0000","last_undegraded":"2023-11-06T03:17:42.168647+0000","last_fullsized":"2023-11-06T03:17:42.168647+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:26:05.701338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987682+0000","last_change":"2023-11-06T03:17:40.625474+0000","last_active":"2023-11-06T03:17:41.987682+0000","last_peered":"2023-11-06T03:17:41.987682+0000","last_clean":"2023-11-06T03:17:41.987682+0000","last_became_active":"2023-11-06T03:17:40.624879+0000","last_became_peered":"2023-11-06T03:17:40.624879+0000","last_unstale":"2023-11-06T03:17:41.987682+0000","last_undegraded":"2023-11-06T03:17:41.987682+0000","last_fullsized":"2023-11-06T03:17:41.987682+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:16.190915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.993996+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:17:40.993996+0000","last_peered":"2023-11-06T03:17:40.993996+0000","last_clean":"2023-11-06T03:17:40.993996+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:17:40.993996+0000","last_undegraded":"2023-11-06T03:17:40.993996+0000","last_fullsized":"2023-11-06T03:17:40.993996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170424+0000","last_change":"2023-11-06T03:17:41.019389+0000","last_active":"2023-11-06T03:17:42.170424+0000","last_peered":"2023-11-06T03:17:42.170424+0000","last_clean":"2023-11-06T03:17:42.170424+0000","last_became_active":"2023-11-06T03:17:41.018688+0000","last_became_peered":"2023-11-06T03:17:41.018688+0000","last_unstale":"2023-11-06T03:17:42.170424+0000","last_undegraded":"2023-11-06T03:17:42.170424+0000","last_fullsized":"2023-11-06T03:17:42.170424+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:55:27.622489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"174'4","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983256+0000","last_change":"2023-11-06T03:17:40.612746+0000","last_active":"2023-11-06T03:17:41.983256+0000","last_peered":"2023-11-06T03:17:41.983256+0000","last_clean":"2023-11-06T03:17:41.983256+0000","last_became_active":"2023-11-06T03:17:40.612143+0000","last_became_peered":"2023-11-06T03:17:40.612143+0000","last_unstale":"2023-11-06T03:17:41.983256+0000","last_undegraded":"2023-11-06T03:17:41.983256+0000","last_fullsized":"2023-11-06T03:17:41.983256+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:59.221718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":145,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.996935+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:17:40.996935+0000","last_peered":"2023-11-06T03:17:40.996935+0000","last_clean":"2023-11-06T03:17:40.996935+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:17:40.996935+0000","last_undegraded":"2023-11-06T03:17:40.996935+0000","last_fullsized":"2023-11-06T03:17:40.996935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166738+0000","last_change":"2023-11-06T03:17:41.012654+0000","last_active":"2023-11-06T03:17:42.166738+0000","last_peered":"2023-11-06T03:17:42.166738+0000","last_clean":"2023-11-06T03:17:42.166738+0000","last_became_active":"2023-11-06T03:17:41.012215+0000","last_became_peered":"2023-11-06T03:17:41.012215+0000","last_unstale":"2023-11-06T03:17:42.166738+0000","last_undegraded":"2023-11-06T03:17:42.166738+0000","last_fullsized":"2023-11-06T03:17:42.166738+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:32.931597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166533+0000","last_change":"2023-11-06T03:17:40.617008+0000","last_active":"2023-11-06T03:17:42.166533+0000","last_peered":"2023-11-06T03:17:42.166533+0000","last_clean":"2023-11-06T03:17:42.166533+0000","last_became_active":"2023-11-06T03:17:40.616668+0000","last_became_peered":"2023-11-06T03:17:40.616668+0000","last_unstale":"2023-11-06T03:17:42.166533+0000","last_undegraded":"2023-11-06T03:17:42.166533+0000","last_fullsized":"2023-11-06T03:17:42.166533+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:50.091990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.994683+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:17:40.994683+0000","last_peered":"2023-11-06T03:17:40.994683+0000","last_clean":"2023-11-06T03:17:40.994683+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:17:40.994683+0000","last_undegraded":"2023-11-06T03:17:40.994683+0000","last_fullsized":"2023-11-06T03:17:40.994683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172180+0000","last_change":"2023-11-06T03:17:41.025856+0000","last_active":"2023-11-06T03:17:42.172180+0000","last_peered":"2023-11-06T03:17:42.172180+0000","last_clean":"2023-11-06T03:17:42.172180+0000","last_became_active":"2023-11-06T03:17:41.024533+0000","last_became_peered":"2023-11-06T03:17:41.024533+0000","last_unstale":"2023-11-06T03:17:42.172180+0000","last_undegraded":"2023-11-06T03:17:42.172180+0000","last_fullsized":"2023-11-06T03:17:42.172180+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:14:32.556871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171883+0000","last_change":"2023-11-06T03:17:40.627819+0000","last_active":"2023-11-06T03:17:42.171883+0000","last_peered":"2023-11-06T03:17:42.171883+0000","last_clean":"2023-11-06T03:17:42.171883+0000","last_became_active":"2023-11-06T03:17:40.627459+0000","last_became_peered":"2023-11-06T03:17:40.627459+0000","last_unstale":"2023-11-06T03:17:42.171883+0000","last_undegraded":"2023-11-06T03:17:42.171883+0000","last_fullsized":"2023-11-06T03:17:42.171883+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:48:10.443982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.997155+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:17:40.997155+0000","last_peered":"2023-11-06T03:17:40.997155+0000","last_clean":"2023-11-06T03:17:40.997155+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:17:40.997155+0000","last_undegraded":"2023-11-06T03:17:40.997155+0000","last_fullsized":"2023-11-06T03:17:40.997155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170955+0000","last_change":"2023-11-06T03:17:41.028805+0000","last_active":"2023-11-06T03:17:42.170955+0000","last_peered":"2023-11-06T03:17:42.170955+0000","last_clean":"2023-11-06T03:17:42.170955+0000","last_became_active":"2023-11-06T03:17:41.028223+0000","last_became_peered":"2023-11-06T03:17:41.028223+0000","last_unstale":"2023-11-06T03:17:42.170955+0000","last_undegraded":"2023-11-06T03:17:42.170955+0000","last_fullsized":"2023-11-06T03:17:42.170955+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:41.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988469+0000","last_change":"2023-11-06T03:17:40.621340+0000","last_active":"2023-11-06T03:17:41.988469+0000","last_peered":"2023-11-06T03:17:41.988469+0000","last_clean":"2023-11-06T03:17:41.988469+0000","last_became_active":"2023-11-06T03:17:40.620100+0000","last_became_peered":"2023-11-06T03:17:40.620100+0000","last_unstale":"2023-11-06T03:17:41.988469+0000","last_undegraded":"2023-11-06T03:17:41.988469+0000","last_fullsized":"2023-11-06T03:17:41.988469+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:49:30.814258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.993718+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:17:40.993718+0000","last_peered":"2023-11-06T03:17:40.993718+0000","last_clean":"2023-11-06T03:17:40.993718+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:17:40.993718+0000","last_undegraded":"2023-11-06T03:17:40.993718+0000","last_fullsized":"2023-11-06T03:17:40.993718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91952,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94279888,"statfs":{"total":96636764160,"available":96542605312,"internally_reserved":0,"allocated":2793472,"data_stored":1612871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":0.95499999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":0.84299999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.008},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":0.81799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":2199552,"data_stored":1021554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":1.129}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":1.1879999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.58799999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.274}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.093}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.23},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47164,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324676,"statfs":{"total":96636764160,"available":96588468224,"internally_reserved":0,"allocated":2215936,"data_stored":1024089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":0.745}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.0009999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.119},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113532,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258308,"statfs":{"total":96636764160,"available":96520507392,"internally_reserved":0,"allocated":2215936,"data_stored":1024101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.73899999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.97599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.86899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.82899999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.0529999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":1.0049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83248,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288592,"statfs":{"total":96636764160,"available":96551518208,"internally_reserved":0,"allocated":2203648,"data_stored":1021644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":5.7270000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":5.0670000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":3.9430000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":3.6739999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":3.855},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":3.645}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":5.109},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":14.170999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":3.9049999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":3.5819999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":3.617},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":3.9260000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":4.3460000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":4.9660000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73328,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298512,"statfs":{"total":96636764160,"available":96561676288,"internally_reserved":0,"allocated":2203648,"data_stored":1023085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":0.438},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":2199552,"data_stored":1022503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.84599999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":1.014}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":1.141},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":1.042},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.93000000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":1.111}]}]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96124,"kb_used_data":2740,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275716,"statfs":{"total":96636764160,"available":96538333184,"internally_reserved":0,"allocated":2805760,"data_stored":1612580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.95799999999999996,"5min":1.0149999999999999,"15min":2.6989999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61399999999999999,"15min":0.65000000000000002},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.008,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.46800000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.20599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.1619999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":4.6920000000000002},"last":0.374}]},{"osd":3,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.89100000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":0.98199999999999998,"5min":1.391,"15min":2.7999999999999998},"last":0.71099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.419,"5min":1.419,"15min":4.641},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.67500000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.4159999999999999,"15min":2.028},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.292,"5min":1.492,"15min":2.8559999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.224,"5min":1.496,"15min":4.7430000000000003},"last":0.90500000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91300000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.3220000000000001,"15min":2.931},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.69699999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.446,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.95799999999999996,"15min":0.99099999999999999},"min":{"1min":0.73999999999999999,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.323,"5min":1.381,"15min":4.8339999999999996},"last":0.751},{"interface":"front","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2569999999999999,"5min":1.335,"15min":1.97},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:17:48.453 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:17:48.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 Processor -- start 2023-11-06T03:17:48.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 -- start start 2023-11-06T03:17:48.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22141551e0 0x7f221406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 0x7f22140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221415f580 con 0x7f22141551e0 2023-11-06T03:17:48.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221415efc0 con 0x7f221415e5e0 2023-11-06T03:17:48.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221bd63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221415f2a0 con 0x7f221415ea90 2023-11-06T03:17:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22141551e0 0x7f221406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f22141551e0 0x7f221406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60176/0 (socket says 172.21.15.157:60176) 2023-11-06T03:17:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 -- 172.21.15.157:0/1252304390 learned_addr learned my addr 172.21.15.157:0/1252304390 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 -- 172.21.15.157:0/1252304390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 msgr2=0x7f22140b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:48.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221a300640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f22192fe640 1 --2- 172.21.15.157:0/1252304390 >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 0x7f22140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 -- 172.21.15.157:0/1252304390 >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 msgr2=0x7f22140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:48.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 --2- 172.21.15.157:0/1252304390 >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 0x7f22140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f2219aff640 1 -- 172.21.15.157:0/1252304390 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221406f5a0 con 0x7f22141551e0 2023-11-06T03:17:48.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f221a300640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:48.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.590+0000 7f22192fe640 1 --2- 172.21.15.157:0/1252304390 >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 0x7f22140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:48.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f2219aff640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22141551e0 0x7f221406fda0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f2214066880 tx=0x7f221001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7043bea11e2338f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f2218afd640 1 -- 172.21.15.157:0/1252304390 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100ab620 con 0x7f22141551e0 2023-11-06T03:17:48.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f2218afd640 1 -- 172.21.15.157:0/1252304390 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22100ab780 con 0x7f22141551e0 2023-11-06T03:17:48.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f2218afd640 1 -- 172.21.15.157:0/1252304390 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100abb10 con 0x7f22141551e0 2023-11-06T03:17:48.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- 172.21.15.157:0/1252304390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22141551e0 msgr2=0x7f221406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22141551e0 0x7f221406fda0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f2214066880 tx=0x7f221001c440 comp rx=0 tx=0).stop 2023-11-06T03:17:48.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- 172.21.15.157:0/1252304390 shutdown_connections 2023-11-06T03:17:48.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- 172.21.15.157:0/1252304390 >> v2:172.21.15.195:3300/0 conn(0x7f221415e5e0 0x7f22140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- 172.21.15.157:0/1252304390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22141551e0 0x7f221406fda0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- 172.21.15.157:0/1252304390 >> 172.21.15.157:0/1252304390 conn(0x7f2214065930 msgr2=0x7f2214065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:48.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- 172.21.15.157:0/1252304390 shutdown_connections 2023-11-06T03:17:48.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- 172.21.15.157:0/1252304390 wait complete. 2023-11-06T03:17:48.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 Processor -- start 2023-11-06T03:17:48.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- start start 2023-11-06T03:17:48.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22141551e0 0x7f221406b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221414dc50 con 0x7f221415e5e0 2023-11-06T03:17:48.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221414d900 con 0x7f22141551e0 2023-11-06T03:17:48.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221bd63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140ce1e0 con 0x7f221415ea90 2023-11-06T03:17:48.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60188/0 (socket says 172.21.15.157:60188) 2023-11-06T03:17:48.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 -- 172.21.15.157:0/2722844330 learned_addr learned my addr 172.21.15.157:0/2722844330 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:48.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f2219aff640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22141551e0 0x7f221406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f221a300640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 -- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 msgr2=0x7f22140b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 -- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22141551e0 msgr2=0x7f221406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.594+0000 7f22192fe640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22141551e0 0x7f221406b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f22192fe640 1 -- 172.21.15.157:0/2722844330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100094d0 con 0x7f221415e5e0 2023-11-06T03:17:48.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f22192fe640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f220400c7c0 tx=0x7f220400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204007a30 con 0x7f221415e5e0 2023-11-06T03:17:48.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204007b90 con 0x7f221415e5e0 2023-11-06T03:17:48.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22140ce4c0 con 0x7f221415e5e0 2023-11-06T03:17:48.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400ce70 con 0x7f221415e5e0 2023-11-06T03:17:48.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221406b940 con 0x7f221415e5e0 2023-11-06T03:17:48.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221414de00 con 0x7f221415e5e0 2023-11-06T03:17:48.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f22040113f0 con 0x7f221415e5e0 2023-11-06T03:17:48.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 0x7f21f8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2204083460 con 0x7f221415e5e0 2023-11-06T03:17:48.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f2219aff640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 0x7f21f8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.598+0000 7f2219aff640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 0x7f21f8067f50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f22100af000 tx=0x7f22100a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.602+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f221414de00 con 0x7f221415e5e0 2023-11-06T03:17:48.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.754+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f221415c9c0 con 0x7f21f8065a90 2023-11-06T03:17:48.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.774+0000 7f220a7fc640 1 -- 172.21.15.157:0/2722844330 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311015 (secure 0 0 0) 0x7f22100ab8f0 con 0x7f21f8065a90 2023-11-06T03:17:48.777 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:48.779 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:17:48.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 msgr2=0x7f21f8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 0x7f21f8067f50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f22100af000 tx=0x7f22100a7000 comp rx=0 tx=0).stop 2023-11-06T03:17:48.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 msgr2=0x7f221414e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f220400c7c0 tx=0x7f220400cc80 comp rx=0 tx=0).stop 2023-11-06T03:17:48.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 shutdown_connections 2023-11-06T03:17:48.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f21f8065a90 0x7f21f8067f50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f221415ea90 0x7f22140b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f221415e5e0 0x7f221414e0e0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 --2- 172.21.15.157:0/2722844330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f22141551e0 0x7f221406b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 >> 172.21.15.157:0/2722844330 conn(0x7f2214065930 msgr2=0x7f221415b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:48.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 shutdown_connections 2023-11-06T03:17:48.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.782+0000 7f221bd63640 1 -- 172.21.15.157:0/2722844330 wait complete. 2023-11-06T03:17:48.796 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2023-11-06T03:17:47.392577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":643804,"kb_used_data":18396,"kb_used_omap":0,"kb_used_meta":625344,"kb_avail":754330916,"statfs":{"total":773094113280,"available":772434857984,"internally_reserved":0,"allocated":18837504,"data_stored":9362427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":640352256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.802251"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167958+0000","last_change":"2023-11-06T03:17:41.018474+0000","last_active":"2023-11-06T03:17:42.167958+0000","last_peered":"2023-11-06T03:17:42.167958+0000","last_clean":"2023-11-06T03:17:42.167958+0000","last_became_active":"2023-11-06T03:17:41.018204+0000","last_became_peered":"2023-11-06T03:17:41.018204+0000","last_unstale":"2023-11-06T03:17:42.167958+0000","last_undegraded":"2023-11-06T03:17:42.167958+0000","last_fullsized":"2023-11-06T03:17:42.167958+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:40:18.156403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167900+0000","last_change":"2023-11-06T03:17:41.040598+0000","last_active":"2023-11-06T03:17:42.167900+0000","last_peered":"2023-11-06T03:17:42.167900+0000","last_clean":"2023-11-06T03:17:42.167900+0000","last_became_active":"2023-11-06T03:17:41.040274+0000","last_became_peered":"2023-11-06T03:17:41.040274+0000","last_unstale":"2023-11-06T03:17:42.167900+0000","last_undegraded":"2023-11-06T03:17:42.167900+0000","last_fullsized":"2023-11-06T03:17:42.167900+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:56.954833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169710+0000","last_change":"2023-11-06T03:17:41.010277+0000","last_active":"2023-11-06T03:17:42.169710+0000","last_peered":"2023-11-06T03:17:42.169710+0000","last_clean":"2023-11-06T03:17:42.169710+0000","last_became_active":"2023-11-06T03:17:41.009478+0000","last_became_peered":"2023-11-06T03:17:41.009478+0000","last_unstale":"2023-11-06T03:17:42.169710+0000","last_undegraded":"2023-11-06T03:17:42.169710+0000","last_fullsized":"2023-11-06T03:17:42.169710+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:09.838757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982812+0000","last_change":"2023-11-06T03:17:41.028802+0000","last_active":"2023-11-06T03:17:41.982812+0000","last_peered":"2023-11-06T03:17:41.982812+0000","last_clean":"2023-11-06T03:17:41.982812+0000","last_became_active":"2023-11-06T03:17:41.028557+0000","last_became_peered":"2023-11-06T03:17:41.028557+0000","last_unstale":"2023-11-06T03:17:41.982812+0000","last_undegraded":"2023-11-06T03:17:41.982812+0000","last_fullsized":"2023-11-06T03:17:41.982812+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:38:17.837799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166463+0000","last_change":"2023-11-06T03:17:41.024792+0000","last_active":"2023-11-06T03:17:42.166463+0000","last_peered":"2023-11-06T03:17:42.166463+0000","last_clean":"2023-11-06T03:17:42.166463+0000","last_became_active":"2023-11-06T03:17:41.024503+0000","last_became_peered":"2023-11-06T03:17:41.024503+0000","last_unstale":"2023-11-06T03:17:42.166463+0000","last_undegraded":"2023-11-06T03:17:42.166463+0000","last_fullsized":"2023-11-06T03:17:42.166463+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:24:19.734187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166238+0000","last_change":"2023-11-06T03:17:41.000262+0000","last_active":"2023-11-06T03:17:42.166238+0000","last_peered":"2023-11-06T03:17:42.166238+0000","last_clean":"2023-11-06T03:17:42.166238+0000","last_became_active":"2023-11-06T03:17:40.999827+0000","last_became_peered":"2023-11-06T03:17:40.999827+0000","last_unstale":"2023-11-06T03:17:42.166238+0000","last_undegraded":"2023-11-06T03:17:42.166238+0000","last_fullsized":"2023-11-06T03:17:42.166238+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:23:13.141458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.165987+0000","last_change":"2023-11-06T03:17:41.016307+0000","last_active":"2023-11-06T03:17:42.165987+0000","last_peered":"2023-11-06T03:17:42.165987+0000","last_clean":"2023-11-06T03:17:42.165987+0000","last_became_active":"2023-11-06T03:17:41.016038+0000","last_became_peered":"2023-11-06T03:17:41.016038+0000","last_unstale":"2023-11-06T03:17:42.165987+0000","last_undegraded":"2023-11-06T03:17:42.165987+0000","last_fullsized":"2023-11-06T03:17:42.165987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:59:49.993695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170230+0000","last_change":"2023-11-06T03:17:41.026233+0000","last_active":"2023-11-06T03:17:42.170230+0000","last_peered":"2023-11-06T03:17:42.170230+0000","last_clean":"2023-11-06T03:17:42.170230+0000","last_became_active":"2023-11-06T03:17:41.025664+0000","last_became_peered":"2023-11-06T03:17:41.025664+0000","last_unstale":"2023-11-06T03:17:42.170230+0000","last_undegraded":"2023-11-06T03:17:42.170230+0000","last_fullsized":"2023-11-06T03:17:42.170230+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:55:07.200307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168079+0000","last_change":"2023-11-06T03:17:41.009266+0000","last_active":"2023-11-06T03:17:42.168079+0000","last_peered":"2023-11-06T03:17:42.168079+0000","last_clean":"2023-11-06T03:17:42.168079+0000","last_became_active":"2023-11-06T03:17:41.008975+0000","last_became_peered":"2023-11-06T03:17:41.008975+0000","last_unstale":"2023-11-06T03:17:42.168079+0000","last_undegraded":"2023-11-06T03:17:42.168079+0000","last_fullsized":"2023-11-06T03:17:42.168079+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:10.769533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166423+0000","last_change":"2023-11-06T03:17:41.037272+0000","last_active":"2023-11-06T03:17:42.166423+0000","last_peered":"2023-11-06T03:17:42.166423+0000","last_clean":"2023-11-06T03:17:42.166423+0000","last_became_active":"2023-11-06T03:17:41.036991+0000","last_became_peered":"2023-11-06T03:17:41.036991+0000","last_unstale":"2023-11-06T03:17:42.166423+0000","last_undegraded":"2023-11-06T03:17:42.166423+0000","last_fullsized":"2023-11-06T03:17:42.166423+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:17.082356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172042+0000","last_change":"2023-11-06T03:17:41.024975+0000","last_active":"2023-11-06T03:17:42.172042+0000","last_peered":"2023-11-06T03:17:42.172042+0000","last_clean":"2023-11-06T03:17:42.172042+0000","last_became_active":"2023-11-06T03:17:41.024315+0000","last_became_peered":"2023-11-06T03:17:41.024315+0000","last_unstale":"2023-11-06T03:17:42.172042+0000","last_undegraded":"2023-11-06T03:17:42.172042+0000","last_fullsized":"2023-11-06T03:17:42.172042+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:22:54.286022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982880+0000","last_change":"2023-11-06T03:17:41.011960+0000","last_active":"2023-11-06T03:17:41.982880+0000","last_peered":"2023-11-06T03:17:41.982880+0000","last_clean":"2023-11-06T03:17:41.982880+0000","last_became_active":"2023-11-06T03:17:41.011710+0000","last_became_peered":"2023-11-06T03:17:41.011710+0000","last_unstale":"2023-11-06T03:17:41.982880+0000","last_undegraded":"2023-11-06T03:17:41.982880+0000","last_fullsized":"2023-11-06T03:17:41.982880+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:04:17.082356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988151+0000","last_change":"2023-11-06T03:17:41.026274+0000","last_active":"2023-11-06T03:17:41.988151+0000","last_peered":"2023-11-06T03:17:41.988151+0000","last_clean":"2023-11-06T03:17:41.988151+0000","last_became_active":"2023-11-06T03:17:41.025484+0000","last_became_peered":"2023-11-06T03:17:41.025484+0000","last_unstale":"2023-11-06T03:17:41.988151+0000","last_undegraded":"2023-11-06T03:17:41.988151+0000","last_fullsized":"2023-11-06T03:17:41.988151+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:54:26.138790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166485+0000","last_change":"2023-11-06T03:17:41.036938+0000","last_active":"2023-11-06T03:17:42.166485+0000","last_peered":"2023-11-06T03:17:42.166485+0000","last_clean":"2023-11-06T03:17:42.166485+0000","last_became_active":"2023-11-06T03:17:41.036312+0000","last_became_peered":"2023-11-06T03:17:41.036312+0000","last_unstale":"2023-11-06T03:17:42.166485+0000","last_undegraded":"2023-11-06T03:17:42.166485+0000","last_fullsized":"2023-11-06T03:17:42.166485+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:51.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167643+0000","last_change":"2023-11-06T03:17:41.036816+0000","last_active":"2023-11-06T03:17:42.167643+0000","last_peered":"2023-11-06T03:17:42.167643+0000","last_clean":"2023-11-06T03:17:42.167643+0000","last_became_active":"2023-11-06T03:17:41.036533+0000","last_became_peered":"2023-11-06T03:17:41.036533+0000","last_unstale":"2023-11-06T03:17:42.167643+0000","last_undegraded":"2023-11-06T03:17:42.167643+0000","last_fullsized":"2023-11-06T03:17:42.167643+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:57.056216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166068+0000","last_change":"2023-11-06T03:17:41.040016+0000","last_active":"2023-11-06T03:17:42.166068+0000","last_peered":"2023-11-06T03:17:42.166068+0000","last_clean":"2023-11-06T03:17:42.166068+0000","last_became_active":"2023-11-06T03:17:41.039696+0000","last_became_peered":"2023-11-06T03:17:41.039696+0000","last_unstale":"2023-11-06T03:17:42.166068+0000","last_undegraded":"2023-11-06T03:17:42.166068+0000","last_fullsized":"2023-11-06T03:17:42.166068+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:00.233658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166316+0000","last_change":"2023-11-06T03:17:41.011470+0000","last_active":"2023-11-06T03:17:42.166316+0000","last_peered":"2023-11-06T03:17:42.166316+0000","last_clean":"2023-11-06T03:17:42.166316+0000","last_became_active":"2023-11-06T03:17:41.011175+0000","last_became_peered":"2023-11-06T03:17:41.011175+0000","last_unstale":"2023-11-06T03:17:42.166316+0000","last_undegraded":"2023-11-06T03:17:42.166316+0000","last_fullsized":"2023-11-06T03:17:42.166316+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:55.431825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988728+0000","last_change":"2023-11-06T03:17:41.031251+0000","last_active":"2023-11-06T03:17:41.988728+0000","last_peered":"2023-11-06T03:17:41.988728+0000","last_clean":"2023-11-06T03:17:41.988728+0000","last_became_active":"2023-11-06T03:17:41.030557+0000","last_became_peered":"2023-11-06T03:17:41.030557+0000","last_unstale":"2023-11-06T03:17:41.988728+0000","last_undegraded":"2023-11-06T03:17:41.988728+0000","last_fullsized":"2023-11-06T03:17:41.988728+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:19.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988642+0000","last_change":"2023-11-06T03:17:41.024567+0000","last_active":"2023-11-06T03:17:41.988642+0000","last_peered":"2023-11-06T03:17:41.988642+0000","last_clean":"2023-11-06T03:17:41.988642+0000","last_became_active":"2023-11-06T03:17:41.023883+0000","last_became_peered":"2023-11-06T03:17:41.023883+0000","last_unstale":"2023-11-06T03:17:41.988642+0000","last_undegraded":"2023-11-06T03:17:41.988642+0000","last_fullsized":"2023-11-06T03:17:41.988642+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:01.801910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166110+0000","last_change":"2023-11-06T03:17:41.012112+0000","last_active":"2023-11-06T03:17:42.166110+0000","last_peered":"2023-11-06T03:17:42.166110+0000","last_clean":"2023-11-06T03:17:42.166110+0000","last_became_active":"2023-11-06T03:17:41.011771+0000","last_became_peered":"2023-11-06T03:17:41.011771+0000","last_unstale":"2023-11-06T03:17:42.166110+0000","last_undegraded":"2023-11-06T03:17:42.166110+0000","last_fullsized":"2023-11-06T03:17:42.166110+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:41.479552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167810+0000","last_change":"2023-11-06T03:17:41.009205+0000","last_active":"2023-11-06T03:17:42.167810+0000","last_peered":"2023-11-06T03:17:42.167810+0000","last_clean":"2023-11-06T03:17:42.167810+0000","last_became_active":"2023-11-06T03:17:41.008984+0000","last_became_peered":"2023-11-06T03:17:41.008984+0000","last_unstale":"2023-11-06T03:17:42.167810+0000","last_undegraded":"2023-11-06T03:17:42.167810+0000","last_fullsized":"2023-11-06T03:17:42.167810+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:12:46.556231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167911+0000","last_change":"2023-11-06T03:17:41.010065+0000","last_active":"2023-11-06T03:17:42.167911+0000","last_peered":"2023-11-06T03:17:42.167911+0000","last_clean":"2023-11-06T03:17:42.167911+0000","last_became_active":"2023-11-06T03:17:41.009671+0000","last_became_peered":"2023-11-06T03:17:41.009671+0000","last_unstale":"2023-11-06T03:17:42.167911+0000","last_undegraded":"2023-11-06T03:17:42.167911+0000","last_fullsized":"2023-11-06T03:17:42.167911+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:59.918715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170093+0000","last_change":"2023-11-06T03:17:41.016082+0000","last_active":"2023-11-06T03:17:42.170093+0000","last_peered":"2023-11-06T03:17:42.170093+0000","last_clean":"2023-11-06T03:17:42.170093+0000","last_became_active":"2023-11-06T03:17:41.015301+0000","last_became_peered":"2023-11-06T03:17:41.015301+0000","last_unstale":"2023-11-06T03:17:42.170093+0000","last_undegraded":"2023-11-06T03:17:42.170093+0000","last_fullsized":"2023-11-06T03:17:42.170093+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:38:44.882747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167610+0000","last_change":"2023-11-06T03:17:41.020101+0000","last_active":"2023-11-06T03:17:42.167610+0000","last_peered":"2023-11-06T03:17:42.167610+0000","last_clean":"2023-11-06T03:17:42.167610+0000","last_became_active":"2023-11-06T03:17:41.019822+0000","last_became_peered":"2023-11-06T03:17:41.019822+0000","last_unstale":"2023-11-06T03:17:42.167610+0000","last_undegraded":"2023-11-06T03:17:42.167610+0000","last_fullsized":"2023-11-06T03:17:42.167610+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:57:50.185204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167201+0000","last_change":"2023-11-06T03:17:40.994370+0000","last_active":"2023-11-06T03:17:42.167201+0000","last_peered":"2023-11-06T03:17:42.167201+0000","last_clean":"2023-11-06T03:17:42.167201+0000","last_became_active":"2023-11-06T03:17:40.994083+0000","last_became_peered":"2023-11-06T03:17:40.994083+0000","last_unstale":"2023-11-06T03:17:42.167201+0000","last_undegraded":"2023-11-06T03:17:42.167201+0000","last_fullsized":"2023-11-06T03:17:42.167201+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:03:15.895346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172093+0000","last_change":"2023-11-06T03:17:41.026258+0000","last_active":"2023-11-06T03:17:42.172093+0000","last_peered":"2023-11-06T03:17:42.172093+0000","last_clean":"2023-11-06T03:17:42.172093+0000","last_became_active":"2023-11-06T03:17:41.025181+0000","last_became_peered":"2023-11-06T03:17:41.025181+0000","last_unstale":"2023-11-06T03:17:42.172093+0000","last_undegraded":"2023-11-06T03:17:42.172093+0000","last_fullsized":"2023-11-06T03:17:42.172093+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:32:58.810371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170002+0000","last_change":"2023-11-06T03:17:41.013430+0000","last_active":"2023-11-06T03:17:42.170002+0000","last_peered":"2023-11-06T03:17:42.170002+0000","last_clean":"2023-11-06T03:17:42.170002+0000","last_became_active":"2023-11-06T03:17:41.012327+0000","last_became_peered":"2023-11-06T03:17:41.012327+0000","last_unstale":"2023-11-06T03:17:42.170002+0000","last_undegraded":"2023-11-06T03:17:42.170002+0000","last_fullsized":"2023-11-06T03:17:42.170002+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:38:30.244459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166246+0000","last_change":"2023-11-06T03:17:41.023744+0000","last_active":"2023-11-06T03:17:42.166246+0000","last_peered":"2023-11-06T03:17:42.166246+0000","last_clean":"2023-11-06T03:17:42.166246+0000","last_became_active":"2023-11-06T03:17:41.023472+0000","last_became_peered":"2023-11-06T03:17:41.023472+0000","last_unstale":"2023-11-06T03:17:42.166246+0000","last_undegraded":"2023-11-06T03:17:42.166246+0000","last_fullsized":"2023-11-06T03:17:42.166246+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:58.093338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171616+0000","last_change":"2023-11-06T03:17:41.016178+0000","last_active":"2023-11-06T03:17:42.171616+0000","last_peered":"2023-11-06T03:17:42.171616+0000","last_clean":"2023-11-06T03:17:42.171616+0000","last_became_active":"2023-11-06T03:17:41.015325+0000","last_became_peered":"2023-11-06T03:17:41.015325+0000","last_unstale":"2023-11-06T03:17:42.171616+0000","last_undegraded":"2023-11-06T03:17:42.171616+0000","last_fullsized":"2023-11-06T03:17:42.171616+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:22.155917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171095+0000","last_change":"2023-11-06T03:17:41.016417+0000","last_active":"2023-11-06T03:17:42.171095+0000","last_peered":"2023-11-06T03:17:42.171095+0000","last_clean":"2023-11-06T03:17:42.171095+0000","last_became_active":"2023-11-06T03:17:41.015601+0000","last_became_peered":"2023-11-06T03:17:41.015601+0000","last_unstale":"2023-11-06T03:17:42.171095+0000","last_undegraded":"2023-11-06T03:17:42.171095+0000","last_fullsized":"2023-11-06T03:17:42.171095+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:17.202855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166732+0000","last_change":"2023-11-06T03:17:40.999623+0000","last_active":"2023-11-06T03:17:42.166732+0000","last_peered":"2023-11-06T03:17:42.166732+0000","last_clean":"2023-11-06T03:17:42.166732+0000","last_became_active":"2023-11-06T03:17:40.999327+0000","last_became_peered":"2023-11-06T03:17:40.999327+0000","last_unstale":"2023-11-06T03:17:42.166732+0000","last_undegraded":"2023-11-06T03:17:42.166732+0000","last_fullsized":"2023-11-06T03:17:42.166732+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:30.549578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166561+0000","last_change":"2023-11-06T03:17:41.042233+0000","last_active":"2023-11-06T03:17:42.166561+0000","last_peered":"2023-11-06T03:17:42.166561+0000","last_clean":"2023-11-06T03:17:42.166561+0000","last_became_active":"2023-11-06T03:17:41.041984+0000","last_became_peered":"2023-11-06T03:17:41.041984+0000","last_unstale":"2023-11-06T03:17:42.166561+0000","last_undegraded":"2023-11-06T03:17:42.166561+0000","last_fullsized":"2023-11-06T03:17:42.166561+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:59:58.707564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170692+0000","last_change":"2023-11-06T03:17:41.039597+0000","last_active":"2023-11-06T03:17:42.170692+0000","last_peered":"2023-11-06T03:17:42.170692+0000","last_clean":"2023-11-06T03:17:42.170692+0000","last_became_active":"2023-11-06T03:17:41.038995+0000","last_became_peered":"2023-11-06T03:17:41.038995+0000","last_unstale":"2023-11-06T03:17:42.170692+0000","last_undegraded":"2023-11-06T03:17:42.170692+0000","last_fullsized":"2023-11-06T03:17:42.170692+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:40:01.801910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166392+0000","last_change":"2023-11-06T03:17:40.609942+0000","last_active":"2023-11-06T03:17:42.166392+0000","last_peered":"2023-11-06T03:17:42.166392+0000","last_clean":"2023-11-06T03:17:42.166392+0000","last_became_active":"2023-11-06T03:17:40.609458+0000","last_became_peered":"2023-11-06T03:17:40.609458+0000","last_unstale":"2023-11-06T03:17:42.166392+0000","last_undegraded":"2023-11-06T03:17:42.166392+0000","last_fullsized":"2023-11-06T03:17:42.166392+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:03:49.390773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166947+0000","last_change":"2023-11-06T03:17:40.992918+0000","last_active":"2023-11-06T03:17:42.166947+0000","last_peered":"2023-11-06T03:17:42.166947+0000","last_clean":"2023-11-06T03:17:42.166947+0000","last_became_active":"2023-11-06T03:17:40.992620+0000","last_became_peered":"2023-11-06T03:17:40.992620+0000","last_unstale":"2023-11-06T03:17:42.166947+0000","last_undegraded":"2023-11-06T03:17:42.166947+0000","last_fullsized":"2023-11-06T03:17:42.166947+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:28:57.056216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167982+0000","last_change":"2023-11-06T03:17:40.611372+0000","last_active":"2023-11-06T03:17:42.167982+0000","last_peered":"2023-11-06T03:17:42.167982+0000","last_clean":"2023-11-06T03:17:42.167982+0000","last_became_active":"2023-11-06T03:17:40.611003+0000","last_became_peered":"2023-11-06T03:17:40.611003+0000","last_unstale":"2023-11-06T03:17:42.167982+0000","last_undegraded":"2023-11-06T03:17:42.167982+0000","last_fullsized":"2023-11-06T03:17:42.167982+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:55:26.610548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988817+0000","last_change":"2023-11-06T03:17:41.027967+0000","last_active":"2023-11-06T03:17:41.988817+0000","last_peered":"2023-11-06T03:17:41.988817+0000","last_clean":"2023-11-06T03:17:41.988817+0000","last_became_active":"2023-11-06T03:17:41.027363+0000","last_became_peered":"2023-11-06T03:17:41.027363+0000","last_unstale":"2023-11-06T03:17:41.988817+0000","last_undegraded":"2023-11-06T03:17:41.988817+0000","last_fullsized":"2023-11-06T03:17:41.988817+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:58.282317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"174'3","reported_seq":14,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982898+0000","last_change":"2023-11-06T03:17:40.609910+0000","last_active":"2023-11-06T03:17:41.982898+0000","last_peered":"2023-11-06T03:17:41.982898+0000","last_clean":"2023-11-06T03:17:41.982898+0000","last_became_active":"2023-11-06T03:17:40.609545+0000","last_became_peered":"2023-11-06T03:17:40.609545+0000","last_unstale":"2023-11-06T03:17:41.982898+0000","last_undegraded":"2023-11-06T03:17:41.982898+0000","last_fullsized":"2023-11-06T03:17:41.982898+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:14.067635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168076+0000","last_change":"2023-11-06T03:17:41.039473+0000","last_active":"2023-11-06T03:17:42.168076+0000","last_peered":"2023-11-06T03:17:42.168076+0000","last_clean":"2023-11-06T03:17:42.168076+0000","last_became_active":"2023-11-06T03:17:41.039213+0000","last_became_peered":"2023-11-06T03:17:41.039213+0000","last_unstale":"2023-11-06T03:17:42.168076+0000","last_undegraded":"2023-11-06T03:17:42.168076+0000","last_fullsized":"2023-11-06T03:17:42.168076+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:11:12.988831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"174'6","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988484+0000","last_change":"2023-11-06T03:17:40.620815+0000","last_active":"2023-11-06T03:17:41.988484+0000","last_peered":"2023-11-06T03:17:41.988484+0000","last_clean":"2023-11-06T03:17:41.988484+0000","last_became_active":"2023-11-06T03:17:40.619326+0000","last_became_peered":"2023-11-06T03:17:40.619326+0000","last_unstale":"2023-11-06T03:17:41.988484+0000","last_undegraded":"2023-11-06T03:17:41.988484+0000","last_fullsized":"2023-11-06T03:17:41.988484+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:39.662107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.165971+0000","last_change":"2023-11-06T03:17:41.012387+0000","last_active":"2023-11-06T03:17:42.165971+0000","last_peered":"2023-11-06T03:17:42.165971+0000","last_clean":"2023-11-06T03:17:42.165971+0000","last_became_active":"2023-11-06T03:17:41.012051+0000","last_became_peered":"2023-11-06T03:17:41.012051+0000","last_unstale":"2023-11-06T03:17:42.165971+0000","last_undegraded":"2023-11-06T03:17:42.165971+0000","last_fullsized":"2023-11-06T03:17:42.165971+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:40:16.143274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988408+0000","last_change":"2023-11-06T03:17:40.627765+0000","last_active":"2023-11-06T03:17:41.988408+0000","last_peered":"2023-11-06T03:17:41.988408+0000","last_clean":"2023-11-06T03:17:41.988408+0000","last_became_active":"2023-11-06T03:17:40.624009+0000","last_became_peered":"2023-11-06T03:17:40.624009+0000","last_unstale":"2023-11-06T03:17:41.988408+0000","last_undegraded":"2023-11-06T03:17:41.988408+0000","last_fullsized":"2023-11-06T03:17:41.988408+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:22:21.143976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166125+0000","last_change":"2023-11-06T03:17:41.017369+0000","last_active":"2023-11-06T03:17:42.166125+0000","last_peered":"2023-11-06T03:17:42.166125+0000","last_clean":"2023-11-06T03:17:42.166125+0000","last_became_active":"2023-11-06T03:17:41.017082+0000","last_became_peered":"2023-11-06T03:17:41.017082+0000","last_unstale":"2023-11-06T03:17:42.166125+0000","last_undegraded":"2023-11-06T03:17:42.166125+0000","last_fullsized":"2023-11-06T03:17:42.166125+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:07.469836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168579+0000","last_change":"2023-11-06T03:17:40.614381+0000","last_active":"2023-11-06T03:17:42.168579+0000","last_peered":"2023-11-06T03:17:42.168579+0000","last_clean":"2023-11-06T03:17:42.168579+0000","last_became_active":"2023-11-06T03:17:40.614049+0000","last_became_peered":"2023-11-06T03:17:40.614049+0000","last_unstale":"2023-11-06T03:17:42.168579+0000","last_undegraded":"2023-11-06T03:17:42.168579+0000","last_fullsized":"2023-11-06T03:17:42.168579+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:08.596679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987828+0000","last_change":"2023-11-06T03:17:41.024674+0000","last_active":"2023-11-06T03:17:41.987828+0000","last_peered":"2023-11-06T03:17:41.987828+0000","last_clean":"2023-11-06T03:17:41.987828+0000","last_became_active":"2023-11-06T03:17:41.008571+0000","last_became_peered":"2023-11-06T03:17:41.008571+0000","last_unstale":"2023-11-06T03:17:41.987828+0000","last_undegraded":"2023-11-06T03:17:41.987828+0000","last_fullsized":"2023-11-06T03:17:41.987828+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:51:50.515130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982666+0000","last_change":"2023-11-06T03:17:40.621561+0000","last_active":"2023-11-06T03:17:41.982666+0000","last_peered":"2023-11-06T03:17:41.982666+0000","last_clean":"2023-11-06T03:17:41.982666+0000","last_became_active":"2023-11-06T03:17:40.621261+0000","last_became_peered":"2023-11-06T03:17:40.621261+0000","last_unstale":"2023-11-06T03:17:41.982666+0000","last_undegraded":"2023-11-06T03:17:41.982666+0000","last_fullsized":"2023-11-06T03:17:41.982666+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:27:54.419885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988152+0000","last_change":"2023-11-06T03:17:41.029051+0000","last_active":"2023-11-06T03:17:41.988152+0000","last_peered":"2023-11-06T03:17:41.988152+0000","last_clean":"2023-11-06T03:17:41.988152+0000","last_became_active":"2023-11-06T03:17:41.028368+0000","last_became_peered":"2023-11-06T03:17:41.028368+0000","last_unstale":"2023-11-06T03:17:41.988152+0000","last_undegraded":"2023-11-06T03:17:41.988152+0000","last_fullsized":"2023-11-06T03:17:41.988152+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:17:35.770871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"174'8","reported_seq":19,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171038+0000","last_change":"2023-11-06T03:17:40.615897+0000","last_active":"2023-11-06T03:17:42.171038+0000","last_peered":"2023-11-06T03:17:42.171038+0000","last_clean":"2023-11-06T03:17:42.171038+0000","last_became_active":"2023-11-06T03:17:40.615133+0000","last_became_peered":"2023-11-06T03:17:40.615133+0000","last_unstale":"2023-11-06T03:17:42.171038+0000","last_undegraded":"2023-11-06T03:17:42.171038+0000","last_fullsized":"2023-11-06T03:17:42.171038+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:57:07.911861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166838+0000","last_change":"2023-11-06T03:17:40.998864+0000","last_active":"2023-11-06T03:17:42.166838+0000","last_peered":"2023-11-06T03:17:42.166838+0000","last_clean":"2023-11-06T03:17:42.166838+0000","last_became_active":"2023-11-06T03:17:40.998558+0000","last_became_peered":"2023-11-06T03:17:40.998558+0000","last_unstale":"2023-11-06T03:17:42.166838+0000","last_undegraded":"2023-11-06T03:17:42.166838+0000","last_fullsized":"2023-11-06T03:17:42.166838+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:08.835610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172510+0000","last_change":"2023-11-06T03:17:40.624836+0000","last_active":"2023-11-06T03:17:42.172510+0000","last_peered":"2023-11-06T03:17:42.172510+0000","last_clean":"2023-11-06T03:17:42.172510+0000","last_became_active":"2023-11-06T03:17:40.624094+0000","last_became_peered":"2023-11-06T03:17:40.624094+0000","last_unstale":"2023-11-06T03:17:42.172510+0000","last_undegraded":"2023-11-06T03:17:42.172510+0000","last_fullsized":"2023-11-06T03:17:42.172510+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:58:01.437634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166800+0000","last_change":"2023-11-06T03:17:41.042244+0000","last_active":"2023-11-06T03:17:42.166800+0000","last_peered":"2023-11-06T03:17:42.166800+0000","last_clean":"2023-11-06T03:17:42.166800+0000","last_became_active":"2023-11-06T03:17:41.042004+0000","last_became_peered":"2023-11-06T03:17:41.042004+0000","last_unstale":"2023-11-06T03:17:42.166800+0000","last_undegraded":"2023-11-06T03:17:42.166800+0000","last_fullsized":"2023-11-06T03:17:42.166800+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:28:20.426342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983199+0000","last_change":"2023-11-06T03:17:40.621557+0000","last_active":"2023-11-06T03:17:41.983199+0000","last_peered":"2023-11-06T03:17:41.983199+0000","last_clean":"2023-11-06T03:17:41.983199+0000","last_became_active":"2023-11-06T03:17:40.621236+0000","last_became_peered":"2023-11-06T03:17:40.621236+0000","last_unstale":"2023-11-06T03:17:41.983199+0000","last_undegraded":"2023-11-06T03:17:41.983199+0000","last_fullsized":"2023-11-06T03:17:41.983199+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:41:57.081398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172420+0000","last_change":"2023-11-06T03:17:41.011334+0000","last_active":"2023-11-06T03:17:42.172420+0000","last_peered":"2023-11-06T03:17:42.172420+0000","last_clean":"2023-11-06T03:17:42.172420+0000","last_became_active":"2023-11-06T03:17:41.010527+0000","last_became_peered":"2023-11-06T03:17:41.010527+0000","last_unstale":"2023-11-06T03:17:42.172420+0000","last_undegraded":"2023-11-06T03:17:42.172420+0000","last_fullsized":"2023-11-06T03:17:42.172420+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:48:40.674047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169903+0000","last_change":"2023-11-06T03:17:40.630051+0000","last_active":"2023-11-06T03:17:42.169903+0000","last_peered":"2023-11-06T03:17:42.169903+0000","last_clean":"2023-11-06T03:17:42.169903+0000","last_became_active":"2023-11-06T03:17:40.629340+0000","last_became_peered":"2023-11-06T03:17:40.629340+0000","last_unstale":"2023-11-06T03:17:42.169903+0000","last_undegraded":"2023-11-06T03:17:42.169903+0000","last_fullsized":"2023-11-06T03:17:42.169903+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:23:04.388942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168140+0000","last_change":"2023-11-06T03:17:40.998928+0000","last_active":"2023-11-06T03:17:42.168140+0000","last_peered":"2023-11-06T03:17:42.168140+0000","last_clean":"2023-11-06T03:17:42.168140+0000","last_became_active":"2023-11-06T03:17:40.998679+0000","last_became_peered":"2023-11-06T03:17:40.998679+0000","last_unstale":"2023-11-06T03:17:42.168140+0000","last_undegraded":"2023-11-06T03:17:42.168140+0000","last_fullsized":"2023-11-06T03:17:42.168140+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:51.103931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171803+0000","last_change":"2023-11-06T03:17:40.622390+0000","last_active":"2023-11-06T03:17:42.171803+0000","last_peered":"2023-11-06T03:17:42.171803+0000","last_clean":"2023-11-06T03:17:42.171803+0000","last_became_active":"2023-11-06T03:17:40.620356+0000","last_became_peered":"2023-11-06T03:17:40.620356+0000","last_unstale":"2023-11-06T03:17:42.171803+0000","last_undegraded":"2023-11-06T03:17:42.171803+0000","last_fullsized":"2023-11-06T03:17:42.171803+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:11:30.703687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167561+0000","last_change":"2023-11-06T03:17:41.001437+0000","last_active":"2023-11-06T03:17:42.167561+0000","last_peered":"2023-11-06T03:17:42.167561+0000","last_clean":"2023-11-06T03:17:42.167561+0000","last_became_active":"2023-11-06T03:17:41.001164+0000","last_became_peered":"2023-11-06T03:17:41.001164+0000","last_unstale":"2023-11-06T03:17:42.167561+0000","last_undegraded":"2023-11-06T03:17:42.167561+0000","last_fullsized":"2023-11-06T03:17:42.167561+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:33:19.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988421+0000","last_change":"2023-11-06T03:17:40.624113+0000","last_active":"2023-11-06T03:17:41.988421+0000","last_peered":"2023-11-06T03:17:41.988421+0000","last_clean":"2023-11-06T03:17:41.988421+0000","last_became_active":"2023-11-06T03:17:40.623376+0000","last_became_peered":"2023-11-06T03:17:40.623376+0000","last_unstale":"2023-11-06T03:17:41.988421+0000","last_undegraded":"2023-11-06T03:17:41.988421+0000","last_fullsized":"2023-11-06T03:17:41.988421+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:18:30.101591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982648+0000","last_change":"2023-11-06T03:17:41.028875+0000","last_active":"2023-11-06T03:17:41.982648+0000","last_peered":"2023-11-06T03:17:41.982648+0000","last_clean":"2023-11-06T03:17:41.982648+0000","last_became_active":"2023-11-06T03:17:41.028545+0000","last_became_peered":"2023-11-06T03:17:41.028545+0000","last_unstale":"2023-11-06T03:17:41.982648+0000","last_undegraded":"2023-11-06T03:17:41.982648+0000","last_fullsized":"2023-11-06T03:17:41.982648+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:16:12.624747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982930+0000","last_change":"2023-11-06T03:17:40.610459+0000","last_active":"2023-11-06T03:17:41.982930+0000","last_peered":"2023-11-06T03:17:41.982930+0000","last_clean":"2023-11-06T03:17:41.982930+0000","last_became_active":"2023-11-06T03:17:40.610020+0000","last_became_peered":"2023-11-06T03:17:40.610020+0000","last_unstale":"2023-11-06T03:17:41.982930+0000","last_undegraded":"2023-11-06T03:17:41.982930+0000","last_fullsized":"2023-11-06T03:17:41.982930+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:08.779800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983000+0000","last_change":"2023-11-06T03:17:41.007861+0000","last_active":"2023-11-06T03:17:41.983000+0000","last_peered":"2023-11-06T03:17:41.983000+0000","last_clean":"2023-11-06T03:17:41.983000+0000","last_became_active":"2023-11-06T03:17:41.007611+0000","last_became_peered":"2023-11-06T03:17:41.007611+0000","last_unstale":"2023-11-06T03:17:41.983000+0000","last_undegraded":"2023-11-06T03:17:41.983000+0000","last_fullsized":"2023-11-06T03:17:41.983000+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:14:30.256371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166960+0000","last_change":"2023-11-06T03:17:40.614263+0000","last_active":"2023-11-06T03:17:42.166960+0000","last_peered":"2023-11-06T03:17:42.166960+0000","last_clean":"2023-11-06T03:17:42.166960+0000","last_became_active":"2023-11-06T03:17:40.613762+0000","last_became_peered":"2023-11-06T03:17:40.613762+0000","last_unstale":"2023-11-06T03:17:42.166960+0000","last_undegraded":"2023-11-06T03:17:42.166960+0000","last_fullsized":"2023-11-06T03:17:42.166960+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:59:16.545555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166554+0000","last_change":"2023-11-06T03:17:40.995296+0000","last_active":"2023-11-06T03:17:42.166554+0000","last_peered":"2023-11-06T03:17:42.166554+0000","last_clean":"2023-11-06T03:17:42.166554+0000","last_became_active":"2023-11-06T03:17:40.994520+0000","last_became_peered":"2023-11-06T03:17:40.994520+0000","last_unstale":"2023-11-06T03:17:42.166554+0000","last_undegraded":"2023-11-06T03:17:42.166554+0000","last_fullsized":"2023-11-06T03:17:42.166554+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:09.791741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169147+0000","last_change":"2023-11-06T03:17:40.630355+0000","last_active":"2023-11-06T03:17:42.169147+0000","last_peered":"2023-11-06T03:17:42.169147+0000","last_clean":"2023-11-06T03:17:42.169147+0000","last_became_active":"2023-11-06T03:17:40.629747+0000","last_became_peered":"2023-11-06T03:17:40.629747+0000","last_unstale":"2023-11-06T03:17:42.169147+0000","last_undegraded":"2023-11-06T03:17:42.169147+0000","last_fullsized":"2023-11-06T03:17:42.169147+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:50:33.501645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166939+0000","last_change":"2023-11-06T03:17:41.014314+0000","last_active":"2023-11-06T03:17:42.166939+0000","last_peered":"2023-11-06T03:17:42.166939+0000","last_clean":"2023-11-06T03:17:42.166939+0000","last_became_active":"2023-11-06T03:17:41.014000+0000","last_became_peered":"2023-11-06T03:17:41.014000+0000","last_unstale":"2023-11-06T03:17:42.166939+0000","last_undegraded":"2023-11-06T03:17:42.166939+0000","last_fullsized":"2023-11-06T03:17:42.166939+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:41.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983319+0000","last_change":"2023-11-06T03:17:40.616582+0000","last_active":"2023-11-06T03:17:41.983319+0000","last_peered":"2023-11-06T03:17:41.983319+0000","last_clean":"2023-11-06T03:17:41.983319+0000","last_became_active":"2023-11-06T03:17:40.616219+0000","last_became_peered":"2023-11-06T03:17:40.616219+0000","last_unstale":"2023-11-06T03:17:41.983319+0000","last_undegraded":"2023-11-06T03:17:41.983319+0000","last_fullsized":"2023-11-06T03:17:41.983319+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:19:35.852152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167791+0000","last_change":"2023-11-06T03:17:40.998769+0000","last_active":"2023-11-06T03:17:42.167791+0000","last_peered":"2023-11-06T03:17:42.167791+0000","last_clean":"2023-11-06T03:17:42.167791+0000","last_became_active":"2023-11-06T03:17:40.998436+0000","last_became_peered":"2023-11-06T03:17:40.998436+0000","last_unstale":"2023-11-06T03:17:42.167791+0000","last_undegraded":"2023-11-06T03:17:42.167791+0000","last_fullsized":"2023-11-06T03:17:42.167791+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:10:22.447983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166441+0000","last_change":"2023-11-06T03:17:40.614117+0000","last_active":"2023-11-06T03:17:42.166441+0000","last_peered":"2023-11-06T03:17:42.166441+0000","last_clean":"2023-11-06T03:17:42.166441+0000","last_became_active":"2023-11-06T03:17:40.613778+0000","last_became_peered":"2023-11-06T03:17:40.613778+0000","last_unstale":"2023-11-06T03:17:42.166441+0000","last_undegraded":"2023-11-06T03:17:42.166441+0000","last_fullsized":"2023-11-06T03:17:42.166441+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:16:49.398376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170514+0000","last_change":"2023-11-06T03:17:41.020719+0000","last_active":"2023-11-06T03:17:42.170514+0000","last_peered":"2023-11-06T03:17:42.170514+0000","last_clean":"2023-11-06T03:17:42.170514+0000","last_became_active":"2023-11-06T03:17:41.020035+0000","last_became_peered":"2023-11-06T03:17:41.020035+0000","last_unstale":"2023-11-06T03:17:42.170514+0000","last_undegraded":"2023-11-06T03:17:42.170514+0000","last_fullsized":"2023-11-06T03:17:42.170514+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:12:35.284829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167021+0000","last_change":"2023-11-06T03:17:40.616588+0000","last_active":"2023-11-06T03:17:42.167021+0000","last_peered":"2023-11-06T03:17:42.167021+0000","last_clean":"2023-11-06T03:17:42.167021+0000","last_became_active":"2023-11-06T03:17:40.615929+0000","last_became_peered":"2023-11-06T03:17:40.615929+0000","last_unstale":"2023-11-06T03:17:42.167021+0000","last_undegraded":"2023-11-06T03:17:42.167021+0000","last_fullsized":"2023-11-06T03:17:42.167021+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:48:30.860619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167135+0000","last_change":"2023-11-06T03:17:41.014552+0000","last_active":"2023-11-06T03:17:42.167135+0000","last_peered":"2023-11-06T03:17:42.167135+0000","last_clean":"2023-11-06T03:17:42.167135+0000","last_became_active":"2023-11-06T03:17:41.013994+0000","last_became_peered":"2023-11-06T03:17:41.013994+0000","last_unstale":"2023-11-06T03:17:42.167135+0000","last_undegraded":"2023-11-06T03:17:42.167135+0000","last_fullsized":"2023-11-06T03:17:42.167135+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:31:09.838757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166322+0000","last_change":"2023-11-06T03:17:40.612745+0000","last_active":"2023-11-06T03:17:42.166322+0000","last_peered":"2023-11-06T03:17:42.166322+0000","last_clean":"2023-11-06T03:17:42.166322+0000","last_became_active":"2023-11-06T03:17:40.611530+0000","last_became_peered":"2023-11-06T03:17:40.611530+0000","last_unstale":"2023-11-06T03:17:42.166322+0000","last_undegraded":"2023-11-06T03:17:42.166322+0000","last_fullsized":"2023-11-06T03:17:42.166322+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:33:53.477947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168764+0000","last_change":"2023-11-06T03:17:41.007952+0000","last_active":"2023-11-06T03:17:42.168764+0000","last_peered":"2023-11-06T03:17:42.168764+0000","last_clean":"2023-11-06T03:17:42.168764+0000","last_became_active":"2023-11-06T03:17:41.007214+0000","last_became_peered":"2023-11-06T03:17:41.007214+0000","last_unstale":"2023-11-06T03:17:42.168764+0000","last_undegraded":"2023-11-06T03:17:42.168764+0000","last_fullsized":"2023-11-06T03:17:42.168764+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:42:30.549578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170791+0000","last_change":"2023-11-06T03:17:40.629908+0000","last_active":"2023-11-06T03:17:42.170791+0000","last_peered":"2023-11-06T03:17:42.170791+0000","last_clean":"2023-11-06T03:17:42.170791+0000","last_became_active":"2023-11-06T03:17:40.629303+0000","last_became_peered":"2023-11-06T03:17:40.629303+0000","last_unstale":"2023-11-06T03:17:42.170791+0000","last_undegraded":"2023-11-06T03:17:42.170791+0000","last_fullsized":"2023-11-06T03:17:42.170791+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:21:11.745377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987615+0000","last_change":"2023-11-06T03:17:41.024605+0000","last_active":"2023-11-06T03:17:41.987615+0000","last_peered":"2023-11-06T03:17:41.987615+0000","last_clean":"2023-11-06T03:17:41.987615+0000","last_became_active":"2023-11-06T03:17:41.012804+0000","last_became_peered":"2023-11-06T03:17:41.012804+0000","last_unstale":"2023-11-06T03:17:41.987615+0000","last_undegraded":"2023-11-06T03:17:41.987615+0000","last_fullsized":"2023-11-06T03:17:41.987615+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:17:28.308334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170105+0000","last_change":"2023-11-06T03:17:40.620927+0000","last_active":"2023-11-06T03:17:42.170105+0000","last_peered":"2023-11-06T03:17:42.170105+0000","last_clean":"2023-11-06T03:17:42.170105+0000","last_became_active":"2023-11-06T03:17:40.620120+0000","last_became_peered":"2023-11-06T03:17:40.620120+0000","last_unstale":"2023-11-06T03:17:42.170105+0000","last_undegraded":"2023-11-06T03:17:42.170105+0000","last_fullsized":"2023-11-06T03:17:42.170105+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:55:13.389503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168029+0000","last_change":"2023-11-06T03:17:40.998827+0000","last_active":"2023-11-06T03:17:42.168029+0000","last_peered":"2023-11-06T03:17:42.168029+0000","last_clean":"2023-11-06T03:17:42.168029+0000","last_became_active":"2023-11-06T03:17:40.998571+0000","last_became_peered":"2023-11-06T03:17:40.998571+0000","last_unstale":"2023-11-06T03:17:42.168029+0000","last_undegraded":"2023-11-06T03:17:42.168029+0000","last_fullsized":"2023-11-06T03:17:42.168029+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:54.572064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166815+0000","last_change":"2023-11-06T03:17:40.628490+0000","last_active":"2023-11-06T03:17:42.166815+0000","last_peered":"2023-11-06T03:17:42.166815+0000","last_clean":"2023-11-06T03:17:42.166815+0000","last_became_active":"2023-11-06T03:17:40.628150+0000","last_became_peered":"2023-11-06T03:17:40.628150+0000","last_unstale":"2023-11-06T03:17:42.166815+0000","last_undegraded":"2023-11-06T03:17:42.166815+0000","last_fullsized":"2023-11-06T03:17:42.166815+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:36:24.072944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170593+0000","last_change":"2023-11-06T03:17:41.013651+0000","last_active":"2023-11-06T03:17:42.170593+0000","last_peered":"2023-11-06T03:17:42.170593+0000","last_clean":"2023-11-06T03:17:42.170593+0000","last_became_active":"2023-11-06T03:17:41.012857+0000","last_became_peered":"2023-11-06T03:17:41.012857+0000","last_unstale":"2023-11-06T03:17:42.170593+0000","last_undegraded":"2023-11-06T03:17:42.170593+0000","last_fullsized":"2023-11-06T03:17:42.170593+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:09.608619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169004+0000","last_change":"2023-11-06T03:17:40.629864+0000","last_active":"2023-11-06T03:17:42.169004+0000","last_peered":"2023-11-06T03:17:42.169004+0000","last_clean":"2023-11-06T03:17:42.169004+0000","last_became_active":"2023-11-06T03:17:40.629548+0000","last_became_peered":"2023-11-06T03:17:40.629548+0000","last_unstale":"2023-11-06T03:17:42.169004+0000","last_undegraded":"2023-11-06T03:17:42.169004+0000","last_fullsized":"2023-11-06T03:17:42.169004+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:24:30.160280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.169786+0000","last_change":"2023-11-06T03:17:41.007656+0000","last_active":"2023-11-06T03:17:42.169786+0000","last_peered":"2023-11-06T03:17:42.169786+0000","last_clean":"2023-11-06T03:17:42.169786+0000","last_became_active":"2023-11-06T03:17:41.006897+0000","last_became_peered":"2023-11-06T03:17:41.006897+0000","last_unstale":"2023-11-06T03:17:42.169786+0000","last_undegraded":"2023-11-06T03:17:42.169786+0000","last_fullsized":"2023-11-06T03:17:42.169786+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:51.159312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.982841+0000","last_change":"2023-11-06T03:17:40.629328+0000","last_active":"2023-11-06T03:17:41.982841+0000","last_peered":"2023-11-06T03:17:41.982841+0000","last_clean":"2023-11-06T03:17:41.982841+0000","last_became_active":"2023-11-06T03:17:40.628988+0000","last_became_peered":"2023-11-06T03:17:40.628988+0000","last_unstale":"2023-11-06T03:17:41.982841+0000","last_undegraded":"2023-11-06T03:17:41.982841+0000","last_fullsized":"2023-11-06T03:17:41.982841+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:24:18.722246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":292,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.995891+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:17:40.995891+0000","last_peered":"2023-11-06T03:17:40.995891+0000","last_clean":"2023-11-06T03:17:40.995891+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:17:40.995891+0000","last_undegraded":"2023-11-06T03:17:40.995891+0000","last_fullsized":"2023-11-06T03:17:40.995891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.010323+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:17:41.010323+0000","last_peered":"2023-11-06T03:17:41.010323+0000","last_clean":"2023-11-06T03:17:41.010323+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:17:41.010323+0000","last_undegraded":"2023-11-06T03:17:41.010323+0000","last_fullsized":"2023-11-06T03:17:41.010323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987424+0000","last_change":"2023-11-06T03:17:41.025178+0000","last_active":"2023-11-06T03:17:41.987424+0000","last_peered":"2023-11-06T03:17:41.987424+0000","last_clean":"2023-11-06T03:17:41.987424+0000","last_became_active":"2023-11-06T03:17:41.007458+0000","last_became_peered":"2023-11-06T03:17:41.007458+0000","last_unstale":"2023-11-06T03:17:41.987424+0000","last_undegraded":"2023-11-06T03:17:41.987424+0000","last_fullsized":"2023-11-06T03:17:41.987424+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:48:23.958898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166697+0000","last_change":"2023-11-06T03:17:40.622167+0000","last_active":"2023-11-06T03:17:42.166697+0000","last_peered":"2023-11-06T03:17:42.166697+0000","last_clean":"2023-11-06T03:17:42.166697+0000","last_became_active":"2023-11-06T03:17:40.621897+0000","last_became_peered":"2023-11-06T03:17:40.621897+0000","last_unstale":"2023-11-06T03:17:42.166697+0000","last_undegraded":"2023-11-06T03:17:42.166697+0000","last_fullsized":"2023-11-06T03:17:42.166697+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:51:40.316206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.995598+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:17:40.995598+0000","last_peered":"2023-11-06T03:17:40.995598+0000","last_clean":"2023-11-06T03:17:40.995598+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:17:40.995598+0000","last_undegraded":"2023-11-06T03:17:40.995598+0000","last_fullsized":"2023-11-06T03:17:40.995598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.167773+0000","last_change":"2023-11-06T03:17:40.995606+0000","last_active":"2023-11-06T03:17:42.167773+0000","last_peered":"2023-11-06T03:17:42.167773+0000","last_clean":"2023-11-06T03:17:42.167773+0000","last_became_active":"2023-11-06T03:17:40.995268+0000","last_became_peered":"2023-11-06T03:17:40.995268+0000","last_unstale":"2023-11-06T03:17:42.167773+0000","last_undegraded":"2023-11-06T03:17:42.167773+0000","last_fullsized":"2023-11-06T03:17:42.167773+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:18:23.288564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983055+0000","last_change":"2023-11-06T03:17:40.604424+0000","last_active":"2023-11-06T03:17:41.983055+0000","last_peered":"2023-11-06T03:17:41.983055+0000","last_clean":"2023-11-06T03:17:41.983055+0000","last_became_active":"2023-11-06T03:17:40.604114+0000","last_became_peered":"2023-11-06T03:17:40.604114+0000","last_unstale":"2023-11-06T03:17:41.983055+0000","last_undegraded":"2023-11-06T03:17:41.983055+0000","last_fullsized":"2023-11-06T03:17:41.983055+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:32.594095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.981930+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:17:40.981930+0000","last_peered":"2023-11-06T03:17:40.981930+0000","last_clean":"2023-11-06T03:17:40.981930+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:17:40.981930+0000","last_undegraded":"2023-11-06T03:17:40.981930+0000","last_fullsized":"2023-11-06T03:17:40.981930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.168647+0000","last_change":"2023-11-06T03:17:41.036912+0000","last_active":"2023-11-06T03:17:42.168647+0000","last_peered":"2023-11-06T03:17:42.168647+0000","last_clean":"2023-11-06T03:17:42.168647+0000","last_became_active":"2023-11-06T03:17:41.036622+0000","last_became_peered":"2023-11-06T03:17:41.036622+0000","last_unstale":"2023-11-06T03:17:42.168647+0000","last_undegraded":"2023-11-06T03:17:42.168647+0000","last_fullsized":"2023-11-06T03:17:42.168647+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:26:05.701338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":11,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.987682+0000","last_change":"2023-11-06T03:17:40.625474+0000","last_active":"2023-11-06T03:17:41.987682+0000","last_peered":"2023-11-06T03:17:41.987682+0000","last_clean":"2023-11-06T03:17:41.987682+0000","last_became_active":"2023-11-06T03:17:40.624879+0000","last_became_peered":"2023-11-06T03:17:40.624879+0000","last_unstale":"2023-11-06T03:17:41.987682+0000","last_undegraded":"2023-11-06T03:17:41.987682+0000","last_fullsized":"2023-11-06T03:17:41.987682+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:24:16.190915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.993996+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:17:40.993996+0000","last_peered":"2023-11-06T03:17:40.993996+0000","last_clean":"2023-11-06T03:17:40.993996+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:17:40.993996+0000","last_undegraded":"2023-11-06T03:17:40.993996+0000","last_fullsized":"2023-11-06T03:17:40.993996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170424+0000","last_change":"2023-11-06T03:17:41.019389+0000","last_active":"2023-11-06T03:17:42.170424+0000","last_peered":"2023-11-06T03:17:42.170424+0000","last_clean":"2023-11-06T03:17:42.170424+0000","last_became_active":"2023-11-06T03:17:41.018688+0000","last_became_peered":"2023-11-06T03:17:41.018688+0000","last_unstale":"2023-11-06T03:17:42.170424+0000","last_undegraded":"2023-11-06T03:17:42.170424+0000","last_fullsized":"2023-11-06T03:17:42.170424+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:55:27.622489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"174'4","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.983256+0000","last_change":"2023-11-06T03:17:40.612746+0000","last_active":"2023-11-06T03:17:41.983256+0000","last_peered":"2023-11-06T03:17:41.983256+0000","last_clean":"2023-11-06T03:17:41.983256+0000","last_became_active":"2023-11-06T03:17:40.612143+0000","last_became_peered":"2023-11-06T03:17:40.612143+0000","last_unstale":"2023-11-06T03:17:41.983256+0000","last_undegraded":"2023-11-06T03:17:41.983256+0000","last_fullsized":"2023-11-06T03:17:41.983256+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:27:59.221718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":145,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.996935+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:17:40.996935+0000","last_peered":"2023-11-06T03:17:40.996935+0000","last_clean":"2023-11-06T03:17:40.996935+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:17:40.996935+0000","last_undegraded":"2023-11-06T03:17:40.996935+0000","last_fullsized":"2023-11-06T03:17:40.996935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166738+0000","last_change":"2023-11-06T03:17:41.012654+0000","last_active":"2023-11-06T03:17:42.166738+0000","last_peered":"2023-11-06T03:17:42.166738+0000","last_clean":"2023-11-06T03:17:42.166738+0000","last_became_active":"2023-11-06T03:17:41.012215+0000","last_became_peered":"2023-11-06T03:17:41.012215+0000","last_unstale":"2023-11-06T03:17:42.166738+0000","last_undegraded":"2023-11-06T03:17:42.166738+0000","last_fullsized":"2023-11-06T03:17:42.166738+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:10:32.931597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"174'1","reported_seq":12,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.166533+0000","last_change":"2023-11-06T03:17:40.617008+0000","last_active":"2023-11-06T03:17:42.166533+0000","last_peered":"2023-11-06T03:17:42.166533+0000","last_clean":"2023-11-06T03:17:42.166533+0000","last_became_active":"2023-11-06T03:17:40.616668+0000","last_became_peered":"2023-11-06T03:17:40.616668+0000","last_unstale":"2023-11-06T03:17:42.166533+0000","last_undegraded":"2023-11-06T03:17:42.166533+0000","last_fullsized":"2023-11-06T03:17:42.166533+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:14:50.091990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.994683+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:17:40.994683+0000","last_peered":"2023-11-06T03:17:40.994683+0000","last_clean":"2023-11-06T03:17:40.994683+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:17:40.994683+0000","last_undegraded":"2023-11-06T03:17:40.994683+0000","last_fullsized":"2023-11-06T03:17:40.994683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.172180+0000","last_change":"2023-11-06T03:17:41.025856+0000","last_active":"2023-11-06T03:17:42.172180+0000","last_peered":"2023-11-06T03:17:42.172180+0000","last_clean":"2023-11-06T03:17:42.172180+0000","last_became_active":"2023-11-06T03:17:41.024533+0000","last_became_peered":"2023-11-06T03:17:41.024533+0000","last_unstale":"2023-11-06T03:17:42.172180+0000","last_undegraded":"2023-11-06T03:17:42.172180+0000","last_fullsized":"2023-11-06T03:17:42.172180+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:14:32.556871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.171883+0000","last_change":"2023-11-06T03:17:40.627819+0000","last_active":"2023-11-06T03:17:42.171883+0000","last_peered":"2023-11-06T03:17:42.171883+0000","last_clean":"2023-11-06T03:17:42.171883+0000","last_became_active":"2023-11-06T03:17:40.627459+0000","last_became_peered":"2023-11-06T03:17:40.627459+0000","last_unstale":"2023-11-06T03:17:42.171883+0000","last_undegraded":"2023-11-06T03:17:42.171883+0000","last_fullsized":"2023-11-06T03:17:42.171883+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:48:10.443982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.997155+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:17:40.997155+0000","last_peered":"2023-11-06T03:17:40.997155+0000","last_clean":"2023-11-06T03:17:40.997155+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:17:40.997155+0000","last_undegraded":"2023-11-06T03:17:40.997155+0000","last_fullsized":"2023-11-06T03:17:40.997155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":10,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:42.170955+0000","last_change":"2023-11-06T03:17:41.028805+0000","last_active":"2023-11-06T03:17:42.170955+0000","last_peered":"2023-11-06T03:17:42.170955+0000","last_clean":"2023-11-06T03:17:42.170955+0000","last_became_active":"2023-11-06T03:17:41.028223+0000","last_became_peered":"2023-11-06T03:17:41.028223+0000","last_unstale":"2023-11-06T03:17:42.170955+0000","last_undegraded":"2023-11-06T03:17:42.170955+0000","last_fullsized":"2023-11-06T03:17:42.170955+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:40.572684+0000","last_clean_scrub_stamp":"2023-11-06T03:17:40.572684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:41.067349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"174'2","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-06T03:17:41.988469+0000","last_change":"2023-11-06T03:17:40.621340+0000","last_active":"2023-11-06T03:17:41.988469+0000","last_peered":"2023-11-06T03:17:41.988469+0000","last_clean":"2023-11-06T03:17:41.988469+0000","last_became_active":"2023-11-06T03:17:40.620100+0000","last_became_peered":"2023-11-06T03:17:40.620100+0000","last_unstale":"2023-11-06T03:17:41.988469+0000","last_undegraded":"2023-11-06T03:17:41.988469+0000","last_fullsized":"2023-11-06T03:17:41.988469+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:17:39.560744+0000","last_clean_scrub_stamp":"2023-11-06T03:17:39.560744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:49:30.814258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-11-06T03:17:40.993718+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:17:40.993718+0000","last_peered":"2023-11-06T03:17:40.993718+0000","last_clean":"2023-11-06T03:17:40.993718+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:17:40.993718+0000","last_undegraded":"2023-11-06T03:17:40.993718+0000","last_fullsized":"2023-11-06T03:17:40.993718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91952,"kb_used_data":2728,"kb_used_omap":0,"kb_used_meta":89216,"kb_avail":94279888,"statfs":{"total":96636764160,"available":96542605312,"internally_reserved":0,"allocated":2793472,"data_stored":1612871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":91357184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":0.95499999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.88200000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":0.84299999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":1.008},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":0.81799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":2199552,"data_stored":1021554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":1.129}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":1.1879999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.58799999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.274}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.093}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.23},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47164,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324676,"statfs":{"total":96636764160,"available":96588468224,"internally_reserved":0,"allocated":2215936,"data_stored":1024089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":0.745}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":1.0009999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.119},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113532,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258308,"statfs":{"total":96636764160,"available":96520507392,"internally_reserved":0,"allocated":2215936,"data_stored":1024101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.73899999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.97599999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.86899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.82899999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.0529999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.89700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":1.0049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83248,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94288592,"statfs":{"total":96636764160,"available":96551518208,"internally_reserved":0,"allocated":2203648,"data_stored":1021644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":5.7270000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":5.0670000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":3.9430000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":3.6739999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":3.855},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":3.645}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":5.109},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":14.170999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":3.9049999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":3.5819999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":3.617},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":3.9260000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":4.3460000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":4.9660000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575017,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73328,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298512,"statfs":{"total":96636764160,"available":96561676288,"internally_reserved":0,"allocated":2203648,"data_stored":1023085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":0.438},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.77700000000000002}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69228,"kb_used_data":2148,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302612,"statfs":{"total":96636764160,"available":96565874688,"internally_reserved":0,"allocated":2199552,"data_stored":1022503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.84599999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":1.014}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":1.141},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":1.042},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.93000000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":1.111}]}]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96124,"kb_used_data":2740,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275716,"statfs":{"total":96636764160,"available":96538333184,"internally_reserved":0,"allocated":2805760,"data_stored":1612580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.95799999999999996,"5min":1.0149999999999999,"15min":2.6989999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61399999999999999,"15min":0.65000000000000002},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.008,"5min":1.1479999999999999,"15min":7.1230000000000002},"last":0.46800000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.36199999999999999,"5min":0.28999999999999998,"15min":0.20599999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.1619999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.311,"5min":0.25700000000000001,"15min":0.23899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":4.6920000000000002},"last":0.374}]},{"osd":3,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":0.89100000000000001,"5min":1.2070000000000001,"15min":2.1680000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.72099999999999997},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":0.98199999999999998,"5min":1.391,"15min":2.7999999999999998},"last":0.71099999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.419,"5min":1.419,"15min":4.641},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.84199999999999997,"15min":0.876},"min":{"1min":0.67500000000000004,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.4159999999999999,"15min":2.028},"last":0.78200000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.91300000000000003,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.292,"5min":1.492,"15min":2.8559999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.61599999999999999,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.224,"5min":1.496,"15min":4.7430000000000003},"last":0.90500000000000003}]},{"osd":6,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.91300000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.149,"5min":1.3220000000000001,"15min":2.931},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.995,"5min":0.95299999999999996,"15min":0.98999999999999999},"min":{"1min":0.69699999999999995,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.446,"5min":1.5900000000000001,"15min":2.7789999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.95799999999999996,"15min":0.99099999999999999},"min":{"1min":0.73999999999999999,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.323,"5min":1.381,"15min":4.8339999999999996},"last":0.751},{"interface":"front","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.58899999999999997,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2569999999999999,"5min":1.335,"15min":1.97},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:17:48.804 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:17:48.804 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:17:48.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 Processor -- start 2023-11-06T03:17:48.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 -- start start 2023-11-06T03:17:48.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 0x7f8ff406f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ff40adae0 0x7f8ff40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff406b950 con 0x7f8ff40adae0 2023-11-06T03:17:48.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff406c490 con 0x7f8ff4156a70 2023-11-06T03:17:48.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ffc489640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff406bb60 con 0x7f8ff40adf40 2023-11-06T03:17:48.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ff9a24640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ff40adae0 0x7f8ff40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ff9a24640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ff40adae0 0x7f8ff40b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60190/0 (socket says 172.21.15.157:60190) 2023-11-06T03:17:48.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.966+0000 7f8ff9a24640 1 -- 172.21.15.157:0/2546302177 learned_addr learned my addr 172.21.15.157:0/2546302177 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:48.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffa225640 1 --2- 172.21.15.157:0/2546302177 >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 0x7f8ff406f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 -- 172.21.15.157:0/2546302177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 msgr2=0x7f8ff40b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:48.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/2546302177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 -- 172.21.15.157:0/2546302177 >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 msgr2=0x7f8ff406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:48.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/2546302177 >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 0x7f8ff406f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 -- 172.21.15.157:0/2546302177 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff406be40 con 0x7f8ff40adae0 2023-11-06T03:17:48.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffa225640 1 --2- 172.21.15.157:0/2546302177 >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 0x7f8ff406f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:48.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/2546302177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff40b1c20 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40ce880 tx=0x7f8fe401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2988760cd5dc7b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9223640 1 -- 172.21.15.157:0/2546302177 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40a7070 con 0x7f8ff40adae0 2023-11-06T03:17:48.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9223640 1 -- 172.21.15.157:0/2546302177 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe40ab430 con 0x7f8ff40adae0 2023-11-06T03:17:48.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ff9223640 1 -- 172.21.15.157:0/2546302177 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40ab7c0 con 0x7f8ff40adae0 2023-11-06T03:17:48.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 -- 172.21.15.157:0/2546302177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 msgr2=0x7f8ff40b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 --2- 172.21.15.157:0/2546302177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff40b1c20 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40ce880 tx=0x7f8fe401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 -- 172.21.15.157:0/2546302177 shutdown_connections 2023-11-06T03:17:48.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 --2- 172.21.15.157:0/2546302177 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff40b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 --2- 172.21.15.157:0/2546302177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff40b1c20 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 --2- 172.21.15.157:0/2546302177 >> v2:172.21.15.195:3300/0 conn(0x7f8ff4156a70 0x7f8ff406f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 -- 172.21.15.157:0/2546302177 >> 172.21.15.157:0/2546302177 conn(0x7f8ff4065930 msgr2=0x7f8ff4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:48.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 -- 172.21.15.157:0/2546302177 shutdown_connections 2023-11-06T03:17:48.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.970+0000 7f8ffc489640 1 -- 172.21.15.157:0/2546302177 wait complete. 2023-11-06T03:17:48.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 Processor -- start 2023-11-06T03:17:48.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- start start 2023-11-06T03:17:48.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff4155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 0x7f8ff41556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40a8e40 con 0x7f8ff40adae0 2023-11-06T03:17:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff41508e0 con 0x7f8ff4156a70 2023-11-06T03:17:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40ad2d0 con 0x7f8ff40adf40 2023-11-06T03:17:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60198/0 (socket says 172.21.15.157:60198) 2023-11-06T03:17:48.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 -- 172.21.15.157:0/744723464 learned_addr learned my addr 172.21.15.157:0/744723464 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff4155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffaa26640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 0x7f8ff41556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 -- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 msgr2=0x7f8ff4155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff4155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 -- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 msgr2=0x7f8ff41556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 0x7f8ff41556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:48.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 -- 172.21.15.157:0/744723464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40094d0 con 0x7f8ff40adae0 2023-11-06T03:17:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffaa26640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 0x7f8ff41556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffa225640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c7c0 tx=0x7f8ff000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0007a30 con 0x7f8ff40adae0 2023-11-06T03:17:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0007b90 con 0x7f8ff40adae0 2023-11-06T03:17:48.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000ce70 con 0x7f8ff40adae0 2023-11-06T03:17:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff40a84a0 con 0x7f8ff40adae0 2023-11-06T03:17:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff40a8a60 con 0x7f8ff40adae0 2023-11-06T03:17:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.974+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff40a8f80 con 0x7f8ff40adae0 2023-11-06T03:17:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.978+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ff00113f0 con 0x7f8ff40adae0 2023-11-06T03:17:48.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.978+0000 7f8feaffd640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 0x7f8fd8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.978+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ff0083750 con 0x7f8ff40adae0 2023-11-06T03:17:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.978+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 0x7f8fd8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:48.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.978+0000 7f8ff9a24640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 0x7f8fd8067f50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40af000 tx=0x7f8fe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:48.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:48.982+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8ff40a8f80 con 0x7f8ff40adae0 2023-11-06T03:17:49.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.194+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8ff40a8f80 con 0x7f8ff40adae0 2023-11-06T03:17:49.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.202+0000 7f8feaffd640 1 -- 172.21.15.157:0/744723464 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8ff40a8f80 con 0x7f8ff40adae0 2023-11-06T03:17:49.201 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:17:49.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 msgr2=0x7f8fd8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 0x7f8fd8067f50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40af000 tx=0x7f8fe4008040 comp rx=0 tx=0).stop 2023-11-06T03:17:49.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 msgr2=0x7f8ff41503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c7c0 tx=0x7f8ff000cc80 comp rx=0 tx=0).stop 2023-11-06T03:17:49.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 shutdown_connections 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8fd8065a90 0x7f8fd8067f50 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ff4156a70 0x7f8ff41556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ff40adf40 0x7f8ff4155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 --2- 172.21.15.157:0/744723464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ff40adae0 0x7f8ff41503a0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 >> 172.21.15.157:0/744723464 conn(0x7f8ff4065930 msgr2=0x7f8ff4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:49.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 shutdown_connections 2023-11-06T03:17:49.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.206+0000 7f8ffc489640 1 -- 172.21.15.157:0/744723464 wait complete. 2023-11-06T03:17:49.220 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:49.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.338+0000 7fc5566e5640 1 Processor -- start 2023-11-06T03:17:49.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.338+0000 7fc5566e5640 1 -- start start 2023-11-06T03:17:49.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc5566e5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5501551e0 0x7fc55006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc5566e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015e5e0 0x7fc5500b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc5566e5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55015f580 con 0x7fc55015ea60 2023-11-06T03:17:49.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc5566e5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55015efc0 con 0x7fc5501551e0 2023-11-06T03:17:49.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc5566e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55015f2a0 con 0x7fc55015e5e0 2023-11-06T03:17:49.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5501551e0 0x7fc55006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:49.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5501551e0 0x7fc55006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60214/0 (socket says 172.21.15.157:60214) 2023-11-06T03:17:49.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 -- 172.21.15.157:0/2438696605 learned_addr learned my addr 172.21.15.157:0/2438696605 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:49.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 -- 172.21.15.157:0/2438696605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015e5e0 msgr2=0x7fc5500b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:49.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 --2- 172.21.15.157:0/2438696605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015e5e0 0x7fc5500b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 -- 172.21.15.157:0/2438696605 >> v1:172.21.15.157:6789/0 conn(0x7fc55015ea60 legacy=0x7fc55015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54effd640 1 -- 172.21.15.157:0/2438696605 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930599779 0 0) 0x7fc55015f580 con 0x7fc55015ea60 2023-11-06T03:17:49.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 -- 172.21.15.157:0/2438696605 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55006f5a0 con 0x7fc5501551e0 2023-11-06T03:17:49.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54ffff640 1 --2- 172.21.15.157:0/2438696605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5501551e0 0x7fc55006fda0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2002419672090 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:49.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54effd640 1 -- 172.21.15.157:0/2438696605 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400ad640 con 0x7fc5501551e0 2023-11-06T03:17:49.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54effd640 1 -- 172.21.15.157:0/2438696605 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5400ad7a0 con 0x7fc5501551e0 2023-11-06T03:17:49.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.342+0000 7fc54effd640 1 -- 172.21.15.157:0/2438696605 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400adb30 con 0x7fc5501551e0 2023-11-06T03:17:49.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/2438696605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5501551e0 msgr2=0x7fc55006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- 172.21.15.157:0/2438696605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5501551e0 0x7fc55006fda0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54001c480 comp rx=0 tx=0).stop 2023-11-06T03:17:49.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/2438696605 shutdown_connections 2023-11-06T03:17:49.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- 172.21.15.157:0/2438696605 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015e5e0 0x7fc5500b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- 172.21.15.157:0/2438696605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5501551e0 0x7fc55006fda0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/2438696605 >> 172.21.15.157:0/2438696605 conn(0x7fc550065930 msgr2=0x7fc550065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:49.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/2438696605 shutdown_connections 2023-11-06T03:17:49.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/2438696605 wait complete. 2023-11-06T03:17:49.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 Processor -- start 2023-11-06T03:17:49.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- start start 2023-11-06T03:17:49.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5501551e0 0x7fc550154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015ea60 0x7fc550156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc550157760 con 0x7fc55015e5e0 2023-11-06T03:17:49.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc550157a40 con 0x7fc5501551e0 2023-11-06T03:17:49.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc550153620 con 0x7fc55015ea60 2023-11-06T03:17:49.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:49.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60220/0 (socket says 172.21.15.157:60220) 2023-11-06T03:17:49.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 -- 172.21.15.157:0/3056570927 learned_addr learned my addr 172.21.15.157:0/3056570927 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:49.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 -- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015ea60 msgr2=0x7fc550156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015ea60 0x7fc550156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54ffff640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5501551e0 0x7fc550154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:49.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 -- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5501551e0 msgr2=0x7fc550154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5501551e0 0x7fc550154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 -- 172.21.15.157:0/3056570927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400094d0 con 0x7fc55015e5e0 2023-11-06T03:17:49.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54f7fe640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fc54400b550 tx=0x7fc54400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:49.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544010870 con 0x7fc55015e5e0 2023-11-06T03:17:49.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5440109f0 con 0x7fc55015e5e0 2023-11-06T03:17:49.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544010da0 con 0x7fc55015e5e0 2023-11-06T03:17:49.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc550153900 con 0x7fc55015e5e0 2023-11-06T03:17:49.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.346+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55014d670 con 0x7fc55015e5e0 2023-11-06T03:17:49.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc550153e20 con 0x7fc55015e5e0 2023-11-06T03:17:49.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc54400f3c0 con 0x7fc55015e5e0 2023-11-06T03:17:49.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc54cff9640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 0x7fc53c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:49.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc544083250 con 0x7fc55015e5e0 2023-11-06T03:17:49.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc54ffff640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 0x7fc53c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:49.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.350+0000 7fc54ffff640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 0x7fc53c067f20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc5400af000 tx=0x7fc5400a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:49.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.354+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc550153e20 con 0x7fc55015e5e0 2023-11-06T03:17:49.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.526+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc550153e20 con 0x7fc55015e5e0 2023-11-06T03:17:49.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.526+0000 7fc54cff9640 1 -- 172.21.15.157:0/3056570927 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7fc550153e20 con 0x7fc55015e5e0 2023-11-06T03:17:49.526 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:49.526 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:17:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:17:49.528 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 msgr2=0x7fc53c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 0x7fc53c067f20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc5400af000 tx=0x7fc5400a7070 comp rx=0 tx=0).stop 2023-11-06T03:17:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 msgr2=0x7fc55014cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fc54400b550 tx=0x7fc54400ba10 comp rx=0 tx=0).stop 2023-11-06T03:17:49.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 shutdown_connections 2023-11-06T03:17:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc53c065a60 0x7fc53c067f20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc55015ea60 0x7fc550156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc55015e5e0 0x7fc55014cbf0 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 --2- 172.21.15.157:0/3056570927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5501551e0 0x7fc550154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:49.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 >> 172.21.15.157:0/3056570927 conn(0x7fc550065930 msgr2=0x7fc55015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:49.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 shutdown_connections 2023-11-06T03:17:49.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:49.530+0000 7fc5566e5640 1 -- 172.21.15.157:0/3056570927 wait complete. 2023-11-06T03:17:49.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:17:40.965769+0000', 'modified': '2023-11-06T03:17:42.980401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10818}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10818': {'gid': 10818, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6834/4092537436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 4092537436}, {'type': 'v1', 'addr': '172.21.15.157:6834', 'nonce': 4092537436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:17:49.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:17:49.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-06T03:17:50.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 Processor -- start 2023-11-06T03:17:50.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 -- start start 2023-11-06T03:17:50.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f10241599c0 0x7f10240ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102415e440 con 0x7f10240b2af0 2023-11-06T03:17:50.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102406c240 con 0x7f102406d000 2023-11-06T03:17:50.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1029533640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102406c520 con 0x7f10241599c0 2023-11-06T03:17:50.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f10237fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f10240b2af0 0x7f1024159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57628/0 (socket says 172.21.15.157:57628) 2023-11-06T03:17:50.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f10237fe640 1 -- 172.21.15.157:0/3866777607 learned_addr learned my addr 172.21.15.157:0/3866777607 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:50.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1022ffd640 1 --2- 172.21.15.157:0/3866777607 >> v2:172.21.15.157:3300/0 conn(0x7f10241599c0 0x7f10240ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f10227fc640 1 -- 172.21.15.157:0/3866777607 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029570204 0 0) 0x7f102415e440 con 0x7f10240b2af0 2023-11-06T03:17:50.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f10227fc640 1 -- 172.21.15.157:0/3866777607 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1010003530 con 0x7f10240b2af0 2023-11-06T03:17:50.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1022ffd640 1 -- 172.21.15.157:0/3866777607 >> v1:172.21.15.195:6789/0 conn(0x7f102406d000 legacy=0x7f10240b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1022ffd640 1 -- 172.21.15.157:0/3866777607 >> v1:172.21.15.157:6789/0 conn(0x7f10240b2af0 legacy=0x7f1024159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.078+0000 7f1022ffd640 1 -- 172.21.15.157:0/3866777607 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102406c800 con 0x7f10241599c0 2023-11-06T03:17:50.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1022ffd640 1 --2- 172.21.15.157:0/3866777607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10241599c0 0x7f10240ae000 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f10140097c0 tx=0x7f101401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cf76e3abd18195 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f10227fc640 1 -- 172.21.15.157:0/3866777607 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140a7070 con 0x7f10241599c0 2023-11-06T03:17:50.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f10227fc640 1 -- 172.21.15.157:0/3866777607 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10140ab430 con 0x7f10241599c0 2023-11-06T03:17:50.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f10227fc640 1 -- 172.21.15.157:0/3866777607 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140ab7c0 con 0x7f10241599c0 2023-11-06T03:17:50.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- 172.21.15.157:0/3866777607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10241599c0 msgr2=0x7f10240ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 --2- 172.21.15.157:0/3866777607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10241599c0 0x7f10240ae000 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f10140097c0 tx=0x7f101401c410 comp rx=0 tx=0).stop 2023-11-06T03:17:50.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- 172.21.15.157:0/3866777607 shutdown_connections 2023-11-06T03:17:50.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 --2- 172.21.15.157:0/3866777607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10241599c0 0x7f10240ae000 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- 172.21.15.157:0/3866777607 >> 172.21.15.157:0/3866777607 conn(0x7f1024065930 msgr2=0x7f1024065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- 172.21.15.157:0/3866777607 shutdown_connections 2023-11-06T03:17:50.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- 172.21.15.157:0/3866777607 wait complete. 2023-11-06T03:17:50.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 Processor -- start 2023-11-06T03:17:50.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.082+0000 7f1029533640 1 -- start start 2023-11-06T03:17:50.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102406d000 0x7f1024150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10241599c0 0x7f10240cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102414da50 con 0x7f10240b2af0 2023-11-06T03:17:50.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024152fb0 con 0x7f102406d000 2023-11-06T03:17:50.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1024151410 con 0x7f10241599c0 2023-11-06T03:17:50.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60234/0 (socket says 172.21.15.157:60234) 2023-11-06T03:17:50.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 -- 172.21.15.157:0/1226971829 learned_addr learned my addr 172.21.15.157:0/1226971829 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102406d000 0x7f1024150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f10237fe640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10241599c0 0x7f10240cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 -- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10241599c0 msgr2=0x7f10240cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10241599c0 0x7f10240cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 -- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102406d000 msgr2=0x7f1024150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102406d000 0x7f1024150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 -- 172.21.15.157:0/1226971829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014009420 con 0x7f10240b2af0 2023-11-06T03:17:50.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c450 tx=0x7f100c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 -- 172.21.15.157:0/1226971829 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c012830 con 0x7f10240b2af0 2023-11-06T03:17:50.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 -- 172.21.15.157:0/1226971829 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c0129b0 con 0x7f10240b2af0 2023-11-06T03:17:50.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1024152430 con 0x7f10240b2af0 2023-11-06T03:17:50.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 -- 172.21.15.157:0/1226971829 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c012d60 con 0x7f10240b2af0 2023-11-06T03:17:50.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241529f0 con 0x7f10240b2af0 2023-11-06T03:17:50.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 -- 172.21.15.157:0/1226971829 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f100c01c460 con 0x7f10240b2af0 2023-11-06T03:17:50.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 0x7f1004067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f101bfff640 1 -- 172.21.15.157:0/1226971829 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f100c083120 con 0x7f10240b2af0 2023-11-06T03:17:50.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.086+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 0x7f1004067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 Processor -- start 2023-11-06T03:17:50.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- start start 2023-11-06T03:17:50.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 0x7f1004067f20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f10140af000 tx=0x7f10140a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa73550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9aa73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a93bfc0 con 0x55fd9aabd6e0 2023-11-06T03:17:50.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8ca1a0 con 0x55fd9aabe000 2023-11-06T03:17:50.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8c7770 con 0x55fd9aa73a90 2023-11-06T03:17:50.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60250/0 (socket says 172.21.15.157:60250) 2023-11-06T03:17:50.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 -- 172.21.15.157:0/2644265377 learned_addr learned my addr 172.21.15.157:0/2644265377 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f10237fe640 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9aa73e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f101a7fc640 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa73550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 -- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 msgr2=0x55fd9aa73e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9aa73e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 -- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 msgr2=0x55fd9aa73550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa73550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 -- 172.21.15.157:0/2644265377 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd9a5bf890 con 0x55fd9aabd6e0 2023-11-06T03:17:50.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1022ffd640 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1014019020 tx=0x7f1014004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85816667ced0e4e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1018ff9640 1 -- 172.21.15.157:0/2644265377 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140bf6d0 con 0x55fd9aabd6e0 2023-11-06T03:17:50.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1018ff9640 1 -- 172.21.15.157:0/2644265377 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10140c4070 con 0x55fd9aabd6e0 2023-11-06T03:17:50.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f1018ff9640 1 -- 172.21.15.157:0/2644265377 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140bfa90 con 0x55fd9aabd6e0 2023-11-06T03:17:50.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- 172.21.15.157:0/2644265377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 msgr2=0x55fd9aabdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1014019020 tx=0x7f1014004740 comp rx=0 tx=0).stop 2023-11-06T03:17:50.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- 172.21.15.157:0/2644265377 shutdown_connections 2023-11-06T03:17:50.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9aa73e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa73550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- 172.21.15.157:0/2644265377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabd6e0 0x55fd9aabdac0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- 172.21.15.157:0/2644265377 >> 172.21.15.157:0/2644265377 conn(0x55fd9aa79bf0 msgr2=0x55fd9aa71e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- 172.21.15.157:0/2644265377 shutdown_connections 2023-11-06T03:17:50.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- 172.21.15.157:0/2644265377 wait complete. 2023-11-06T03:17:50.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- start start 2023-11-06T03:17:50.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 Processor -- start 2023-11-06T03:17:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabd6e0 0x55fd9aa7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa7f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8c3060 con 0x55fd9aa73a90 2023-11-06T03:17:50.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a862290 con 0x55fd9aabe000 2023-11-06T03:17:50.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.090+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8806f0 con 0x55fd9aabd6e0 2023-11-06T03:17:50.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 10 client.0 _ll_register_callbacks cb 0x55fd9aa5f1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:50.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60260/0 (socket says 172.21.15.157:60260) 2023-11-06T03:17:50.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 -- 172.21.15.157:0/1291073148 learned_addr learned my addr 172.21.15.157:0/1291073148 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f10237fe640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa7f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabd6e0 0x55fd9aa7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 -- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabd6e0 msgr2=0x55fd9aa7efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabd6e0 0x55fd9aa7efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 -- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 msgr2=0x55fd9aa7f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa7f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 -- 172.21.15.157:0/1291073148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10140ba070 con 0x55fd9aa73a90 2023-11-06T03:17:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f10237fe640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa7f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1014000c10 tx=0x7f101401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760c963e1086e941 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:50.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014004280 con 0x55fd9aa73a90 2023-11-06T03:17:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10140043e0 con 0x55fd9aa73a90 2023-11-06T03:17:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd9a8c19b0 con 0x55fd9aa73a90 2023-11-06T03:17:50.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10140059a0 con 0x55fd9aa73a90 2023-11-06T03:17:50.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 10 client.10986 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x55fd9a8881e0 con 0x55fd9aa73a90 2023-11-06T03:17:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd9a8c19b0 con 0x55fd9aa73a90 2023-11-06T03:17:50.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1014005b00 con 0x55fd9aa73a90 2023-11-06T03:17:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 10 client.10986 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 10 client.10986 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:17:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 20 client.10986 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aa7fdb0 0x55fd9aacec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 4 client.10986 mds_command: new command op to 10818 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd9a8f24d0 con 0x55fd9aa7fdb0 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aa7fdb0 0x55fd9aacec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f101a7fc640 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aa7fdb0 0x55fd9aacec50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e994415464d10f server_cookie=74d4f2f2bde25354 in_seq=0 out_seq=0 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 10 client.10986 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 1 -- 172.21.15.157:0/1291073148 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd9a8f24d0 con 0x55fd9aa7fdb0 2023-11-06T03:17:50.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 10 client.10986 handle_command_reply: tid=0 2023-11-06T03:17:50.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 client.10986 shutdown 2023-11-06T03:17:50.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 msgr2=0x55fd9aa7ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1014000c10 tx=0x7f101401ac50 comp rx=0 tx=0).stop 2023-11-06T03:17:50.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 shutdown_connections 2023-11-06T03:17:50.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aa7fdb0 0x55fd9aacec50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabe000 0x55fd9aa7f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabd6e0 0x55fd9aa7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f1001ffb640 0 client.10986 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:50.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 --2- 172.21.15.157:0/1291073148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aa7ea90 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 >> 172.21.15.157:0/1291073148 conn(0x55fd9aa79bf0 msgr2=0x55fd9aa72530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 shutdown_connections 2023-11-06T03:17:50.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 1 -- 172.21.15.157:0/1291073148 wait complete. 2023-11-06T03:17:50.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.094+0000 7f102c65b000 20 client.10986 trim_cache size 0 max 16384 2023-11-06T03:17:50.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 Processor -- start 2023-11-06T03:17:50.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- start start 2023-11-06T03:17:50.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9aad08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabe000 0x55fd9aab6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a580940 con 0x55fd9aa73a90 2023-11-06T03:17:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8684f0 con 0x55fd9aabd6e0 2023-11-06T03:17:50.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8686f0 con 0x55fd9aabe000 2023-11-06T03:17:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60268/0 (socket says 172.21.15.157:60268) 2023-11-06T03:17:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 -- 172.21.15.157:0/1488542267 learned_addr learned my addr 172.21.15.157:0/1488542267 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 -- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabe000 msgr2=0x55fd9aab6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabe000 0x55fd9aab6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 -- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 msgr2=0x55fd9aad08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9aad08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 -- 172.21.15.157:0/1488542267 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd9a86e330 con 0x55fd9aa73a90 2023-11-06T03:17:50.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f1022ffd640 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f101401c8f0 tx=0x7f10140ce000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f24629c1efbfeee server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f10197fa640 1 -- 172.21.15.157:0/1488542267 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014004480 con 0x55fd9aa73a90 2023-11-06T03:17:50.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f10197fa640 1 -- 172.21.15.157:0/1488542267 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10140c4040 con 0x55fd9aa73a90 2023-11-06T03:17:50.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f10197fa640 1 -- 172.21.15.157:0/1488542267 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014005ba0 con 0x55fd9aa73a90 2023-11-06T03:17:50.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- 172.21.15.157:0/1488542267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 msgr2=0x55fd9aad0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f101401c8f0 tx=0x7f10140ce000 comp rx=0 tx=0).stop 2023-11-06T03:17:50.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- 172.21.15.157:0/1488542267 shutdown_connections 2023-11-06T03:17:50.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aabe000 0x55fd9aab6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9aad08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 --2- 172.21.15.157:0/1488542267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aa73a90 0x55fd9aad0370 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- 172.21.15.157:0/1488542267 >> 172.21.15.157:0/1488542267 conn(0x55fd9aa79bf0 msgr2=0x55fd9aacec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- 172.21.15.157:0/1488542267 shutdown_connections 2023-11-06T03:17:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.110+0000 7f102c65b000 1 -- 172.21.15.157:0/1488542267 wait complete. 2023-11-06T03:17:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- start start 2023-11-06T03:17:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 Processor -- start 2023-11-06T03:17:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9ab19bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9ab1a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8bfe60 con 0x55fd9aabe000 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a87c0f0 con 0x55fd9aabd6e0 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd9a8befd0 con 0x55fd9aa73a90 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 10 client.0 _ll_register_callbacks cb 0x55fd9aa3f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:50.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60274/0 (socket says 172.21.15.157:60274) 2023-11-06T03:17:50.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 -- 172.21.15.157:0/4084720620 learned_addr learned my addr 172.21.15.157:0/4084720620 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 -- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 msgr2=0x55fd9ab19bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9ab19bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 -- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 msgr2=0x55fd9ab1a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9ab1a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 -- 172.21.15.157:0/4084720620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd9a8881e0 con 0x55fd9aabe000 2023-11-06T03:17:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f10237fe640 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f101c00f7d0 tx=0x7f101c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7765eea5accdcc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:50.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c005de0 con 0x55fd9aabe000 2023-11-06T03:17:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f101c00c030 con 0x55fd9aabe000 2023-11-06T03:17:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd9a892b90 con 0x55fd9aabe000 2023-11-06T03:17:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101c00d790 con 0x55fd9aabe000 2023-11-06T03:17:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 10 client.10992 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:50.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x55fd9a5bfcc0 con 0x55fd9aabe000 2023-11-06T03:17:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd9a892b90 con 0x55fd9aabe000 2023-11-06T03:17:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f101c00d8f0 con 0x55fd9aabe000 2023-11-06T03:17:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 10 client.10992 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 10 client.10992 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-06T03:17:50.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 20 client.10992 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:50.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aab9650 0x55fd9aab9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 4 client.10992 mds_command: new command op to 10818 tid=0[{"prefix": "status"}] 2023-11-06T03:17:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd9a8bc0e0 con 0x55fd9aab9650 2023-11-06T03:17:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1022ffd640 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aab9650 0x55fd9aab9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 10 client.10992 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1022ffd640 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aab9650 0x55fd9aab9a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80002ab195e77aa6 server_cookie=30ce14178fde637 in_seq=0 out_seq=0 2023-11-06T03:17:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 1 -- 172.21.15.157:0/4084720620 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55fd9a8bc0e0 con 0x55fd9aab9650 2023-11-06T03:17:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 10 client.10992 handle_command_reply: tid=0 2023-11-06T03:17:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 client.10992 shutdown 2023-11-06T03:17:50.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 msgr2=0x55fd9aab9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f101c00f7d0 tx=0x7f101c00fc90 comp rx=0 tx=0).stop 2023-11-06T03:17:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 shutdown_connections 2023-11-06T03:17:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x55fd9aab9650 0x55fd9aab9a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fd9aabe000 0x55fd9aab9110 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fd9aabd6e0 0x55fd9ab1a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1001ffb640 0 client.10992 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 --2- 172.21.15.157:0/4084720620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55fd9aa73a90 0x55fd9ab19bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 >> 172.21.15.157:0/4084720620 conn(0x55fd9aa79bf0 msgr2=0x55fd9aacec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 shutdown_connections 2023-11-06T03:17:50.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 1 -- 172.21.15.157:0/4084720620 wait complete. 2023-11-06T03:17:50.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f102c65b000 20 client.10992 trim_cache size 0 max 16384 2023-11-06T03:17:50.143 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:50.143 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:17:50.144 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:17:50.144 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 10818, 2023-11-06T03:17:50.144 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:17:50.144 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:17:50.144 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:17:50.145 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 9.1504376569999994, 2023-11-06T03:17:50.145 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 123, 2023-11-06T03:17:50.145 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 175, 2023-11-06T03:17:50.146 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 175, 2023-11-06T03:17:50.146 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 13.568556430999999 2023-11-06T03:17:50.146 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 msgr2=0x7f1004067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.148 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 0x7f1004067f20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f10140af000 tx=0x7f10140a7000 comp rx=0 tx=0).stop 2023-11-06T03:17:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 msgr2=0x7f1024151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c450 tx=0x7f100c00c910 comp rx=0 tx=0).stop 2023-11-06T03:17:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 shutdown_connections 2023-11-06T03:17:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1004065a60 0x7f1004067f20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.149 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10241599c0 0x7f10240cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10240b2af0 0x7f1024151ac0 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 --2- 172.21.15.157:0/1226971829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f102406d000 0x7f1024150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.114+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 >> 172.21.15.157:0/1226971829 conn(0x7f1024065930 msgr2=0x7f102415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.118+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 shutdown_connections 2023-11-06T03:17:50.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.118+0000 7f1029533640 1 -- 172.21.15.157:0/1226971829 wait complete. 2023-11-06T03:17:50.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:17:50.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.266+0000 7fef9dca3640 1 Processor -- start 2023-11-06T03:17:50.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.266+0000 7fef9dca3640 1 -- start start 2023-11-06T03:17:50.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef9dca3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef9815ac80 0x7fef980b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef9dca3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980afb00 0x7fef980b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef9dca3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980cc2b0 con 0x7fef980cc720 2023-11-06T03:17:50.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef9dca3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980b01a0 con 0x7fef9815ac80 2023-11-06T03:17:50.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef9dca3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980cfb10 con 0x7fef980afb00 2023-11-06T03:17:50.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef977fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fef9815ac80 0x7fef980b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef96ffd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980afb00 0x7fef980b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef97fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fef980cc720 0x7fef980ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:34336/0 (socket says 172.21.15.157:34336) 2023-11-06T03:17:50.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef97fff640 1 -- 172.21.15.157:0/2706466310 learned_addr learned my addr 172.21.15.157:0/2706466310 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:50.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef977fe640 1 -- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980afb00 msgr2=0x7fef980b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:50.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef977fe640 1 --2- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980afb00 0x7fef980b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef977fe640 1 -- 172.21.15.157:0/2706466310 >> v1:172.21.15.195:6789/0 conn(0x7fef980cc720 legacy=0x7fef980ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.270+0000 7fef977fe640 1 -- 172.21.15.157:0/2706466310 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980cfdf0 con 0x7fef9815ac80 2023-11-06T03:17:50.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 --2- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef980b2400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fef880097c0 tx=0x7fef8801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a62e9295e1da9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef967fc640 1 -- 172.21.15.157:0/2706466310 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880ad640 con 0x7fef9815ac80 2023-11-06T03:17:50.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef967fc640 1 -- 172.21.15.157:0/2706466310 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef880ad7a0 con 0x7fef9815ac80 2023-11-06T03:17:50.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef967fc640 1 -- 172.21.15.157:0/2706466310 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880adb30 con 0x7fef9815ac80 2023-11-06T03:17:50.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 msgr2=0x7fef980b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef980b2400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fef880097c0 tx=0x7fef8801c480 comp rx=0 tx=0).stop 2023-11-06T03:17:50.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- 172.21.15.157:0/2706466310 shutdown_connections 2023-11-06T03:17:50.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980afb00 0x7fef980b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2706466310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef980b2400 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- 172.21.15.157:0/2706466310 >> 172.21.15.157:0/2706466310 conn(0x7fef9806d560 msgr2=0x7fef9806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- 172.21.15.157:0/2706466310 shutdown_connections 2023-11-06T03:17:50.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- 172.21.15.157:0/2706466310 wait complete. 2023-11-06T03:17:50.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 Processor -- start 2023-11-06T03:17:50.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- start start 2023-11-06T03:17:50.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980cc720 0x7fef98159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef98159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981547c0 con 0x7fef980afb00 2023-11-06T03:17:50.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981521e0 con 0x7fef9815ac80 2023-11-06T03:17:50.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef9dca3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef98155d00 con 0x7fef980cc720 2023-11-06T03:17:50.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60290/0 (socket says 172.21.15.157:60290) 2023-11-06T03:17:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 -- 172.21.15.157:0/2336177719 learned_addr learned my addr 172.21.15.157:0/2336177719 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef96ffd640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980cc720 0x7fef98159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef97fff640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef98159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 -- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980cc720 msgr2=0x7fef98159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980cc720 0x7fef98159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 -- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 msgr2=0x7fef98159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef98159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef977fe640 1 -- 172.21.15.157:0/2336177719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009420 con 0x7fef980afb00 2023-11-06T03:17:50.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.274+0000 7fef97fff640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef98159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:50.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef977fe640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fef880b0040 tx=0x7fef88019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880b6610 con 0x7fef980afb00 2023-11-06T03:17:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef880a6040 con 0x7fef980afb00 2023-11-06T03:17:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef98155fe0 con 0x7fef980afb00 2023-11-06T03:17:50.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880b69b0 con 0x7fef980afb00 2023-11-06T03:17:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981565a0 con 0x7fef980afb00 2023-11-06T03:17:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98156790 con 0x7fef980afb00 2023-11-06T03:17:50.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fef880b4040 con 0x7fef980afb00 2023-11-06T03:17:50.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 0x7fef84067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:50.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef8811d1c0 con 0x7fef980afb00 2023-11-06T03:17:50.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef96ffd640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 0x7fef84067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:50.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.278+0000 7fef96ffd640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 0x7fef84067f50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fef980b45c0 tx=0x7fef8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:50.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.282+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fef880eab40 con 0x7fef980afb00 2023-11-06T03:17:50.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.470+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef98072750 con 0x7fef980afb00 2023-11-06T03:17:50.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.470+0000 7fef7ffff640 1 -- 172.21.15.157:0/2336177719 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v123) v1 ==== 53+0+73 (secure 0 0 0) 0x7fef880b2070 con 0x7fef980afb00 2023-11-06T03:17:50.469 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:17:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 msgr2=0x7fef84067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 0x7fef84067f50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fef980b45c0 tx=0x7fef8c008040 comp rx=0 tx=0).stop 2023-11-06T03:17:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 msgr2=0x7fef98151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fef880b0040 tx=0x7fef88019220 comp rx=0 tx=0).stop 2023-11-06T03:17:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 shutdown_connections 2023-11-06T03:17:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fef84065a90 0x7fef84067f50 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef9815ac80 0x7fef98159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef980cc720 0x7fef98159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 --2- 172.21.15.157:0/2336177719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef980afb00 0x7fef98151680 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:50.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 >> 172.21.15.157:0/2336177719 conn(0x7fef9806d560 msgr2=0x7fef980d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:50.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 shutdown_connections 2023-11-06T03:17:50.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:50.474+0000 7fef9dca3640 1 -- 172.21.15.157:0/2336177719 wait complete. 2023-11-06T03:17:50.488 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:17:50.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:17:50.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:17:50.489 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:17:50.490 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:50.490 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:17:50.490 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:17:50.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:17:50.491 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:17:50.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.492 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:17:50.498 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:17:50.499 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:17:50.499 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:17:50.500 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.500 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:17:50.501 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.501 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:17:50.501 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.502 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:17:50.502 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:17:50.502 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.502 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:17:50.503 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:17:50.503 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.503 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:17:50.503 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.503 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:17:50.504 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41668sec preferred_lft 41668sec 2023-11-06T03:17:50.504 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:17:50.504 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.504 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:17:50.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:17:50.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.506 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:17:50.506 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:17:50.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:17:50.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.508 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:17:50.508 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.508 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:17:50.508 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:17:50.509 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:17:50.509 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.509 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:17:50.510 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:17:50.510 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:17:50.510 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:17:50.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.512 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:17:50.552 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:17:50.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.554 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:17:50.606 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:17:50.606 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:17:50.607 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:17:50.607 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:17:50.607 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:17:50.607 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:17:50.607 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:17:50.608 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:17:50.608 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:17:50.608 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:17:50.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.609 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:17:50.609 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:17:50.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:17:50.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:17:50.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:17:50.611 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:17:50.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.687 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:17:50.735 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:17:50.735 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:17:50.736 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:17:50.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:50.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:17:50.791 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:17:50.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:17:50.792 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:50.839 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:17:50.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:17:50.839 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:50.886 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:17:50.943 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:17:50.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:17:50.998 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:17:50.999 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:17:51.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:51.000 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:17:51.047 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:17:51.048 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:17:51.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:17:51.106 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:17:51.107 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:17:51.162 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:17:51.163 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:17:51.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:51.164 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:17:51.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:17:51.174+0000 7f3815c2e580 -1 init, newargv = 0x55c441437180 newargc=15 2023-11-06T03:17:51.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[97282]: starting ceph client 2023-11-06T03:17:51.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[97282]: starting fuse 2023-11-06T03:17:52.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:17:52.292 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:17:52.292 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:17:52.346 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:17:52.347 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:17:52.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:52.349 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:17:52.395 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:17:52.395 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:17:52.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:17:52.396 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:17:52.397 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:17:52.397 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:17:52.397 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:17:52.398 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.398 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:17:52.399 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:17:52.399 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:17:52.399 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:17:52.400 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.400 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:17:52.400 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:17:52.401 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:17:52.401 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:17:52.402 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.402 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:17:52.402 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:17:52.403 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:17:52.403 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:17:52.403 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:17:52.404 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:17:52.404 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:17:52.405 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:17:52.405 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.405 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:17:52.406 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:17:52.515 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.97282.asok 2023-11-06T03:17:52.697 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.97282.asok 2023-11-06T03:17:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:52.699 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.97282.asok status 2023-11-06T03:17:52.859 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:52.859 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:17:52.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:17:52.860 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:17:52.860 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:17:52.861 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:17:52.861 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:17:52.861 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "97282", 2023-11-06T03:17:52.862 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:17:52.862 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:52.862 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:17:52.863 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:17:52.863 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 11001, 2023-11-06T03:17:52.863 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:17:52.864 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:17:52.864 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:17:52.865 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 11001 2023-11-06T03:17:52.865 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:52.865 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:17:52.865 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:17:52.865 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:17:52.866 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1165877199 2023-11-06T03:17:52.866 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:17:52.866 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:52.866 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:17:52.866 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:17:52.867 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:17:52.867 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1165877199 2023-11-06T03:17:52.867 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:17:52.867 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.11001 192.168.0.1:0/1165877199", 2023-11-06T03:17:52.867 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1165877199", 2023-11-06T03:17:52.868 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:17:52.868 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 123, 2023-11-06T03:17:52.868 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 176, 2023-11-06T03:17:52.868 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:17:52.868 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:17:52.869 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:17:52.869 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:52.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:52.873 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:17:52.873 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:52.880 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:17:52.881 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:52.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:52.881 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:52.934 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:52.935 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:52.983 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:17:52.984 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:17:52.984 DEBUG:teuthology.orchestra.run.smithi157:> import uuid 2023-11-06T03:17:52.985 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.985 DEBUG:teuthology.orchestra.run.smithi157:> n = 1000 2023-11-06T03:17:52.985 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = False 2023-11-06T03:17:52.986 DEBUG:teuthology.orchestra.run.smithi157:> if 0 > 0: 2023-11-06T03:17:52.986 DEBUG:teuthology.orchestra.run.smithi157:> create_hard_links = True 2023-11-06T03:17:52.986 DEBUG:teuthology.orchestra.run.smithi157:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2023-11-06T03:17:52.987 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.987 DEBUG:teuthology.orchestra.run.smithi157:> dpath = os.path.dirname(path) 2023-11-06T03:17:52.988 DEBUG:teuthology.orchestra.run.smithi157:> fnameprefix = os.path.basename(path) 2023-11-06T03:17:52.988 DEBUG:teuthology.orchestra.run.smithi157:> os.makedirs(dpath, exist_ok=True) 2023-11-06T03:17:52.989 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.989 DEBUG:teuthology.orchestra.run.smithi157:> try: 2023-11-06T03:17:52.990 DEBUG:teuthology.orchestra.run.smithi157:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-11-06T03:17:52.990 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:17:52.991 DEBUG:teuthology.orchestra.run.smithi157:> for i in range(n): 2023-11-06T03:17:52.991 DEBUG:teuthology.orchestra.run.smithi157:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-11-06T03:17:52.991 DEBUG:teuthology.orchestra.run.smithi157:> with open(fpath, '"'"'w'"'"') as f: 2023-11-06T03:17:52.992 DEBUG:teuthology.orchestra.run.smithi157:> f.write(f"{i}") 2023-11-06T03:17:52.992 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:17:52.992 DEBUG:teuthology.orchestra.run.smithi157:> f.flush() 2023-11-06T03:17:52.993 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(f.fileno()) 2023-11-06T03:17:52.993 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:17:52.994 DEBUG:teuthology.orchestra.run.smithi157:> os.unlink(fpath) 2023-11-06T03:17:52.994 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:17:52.995 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:17:52.995 DEBUG:teuthology.orchestra.run.smithi157:> if create_hard_links: 2023-11-06T03:17:52.995 DEBUG:teuthology.orchestra.run.smithi157:> for j in range(0): 2023-11-06T03:17:52.996 DEBUG:teuthology.orchestra.run.smithi157:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-11-06T03:17:52.996 DEBUG:teuthology.orchestra.run.smithi157:> if False: 2023-11-06T03:17:52.997 DEBUG:teuthology.orchestra.run.smithi157:> os.fsync(dirfd) 2023-11-06T03:17:52.997 DEBUG:teuthology.orchestra.run.smithi157:> finally: 2023-11-06T03:17:52.998 DEBUG:teuthology.orchestra.run.smithi157:> os.close(dirfd) 2023-11-06T03:17:52.998 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:17:54.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.12:0 scrub status 2023-11-06T03:17:55.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.038+0000 7f47e5d5b640 1 Processor -- start 2023-11-06T03:17:55.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.038+0000 7f47e5d5b640 1 -- start start 2023-11-06T03:17:55.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47e5d5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47e00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47e5d5b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e015e440 con 0x7f47e00b2af0 2023-11-06T03:17:55.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47e5d5b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e006c240 con 0x7f47e006d000 2023-11-06T03:17:55.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47e5d5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006c520 con 0x7f47e01599c0 2023-11-06T03:17:55.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47e00b2af0 0x7f47e0159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57638/0 (socket says 172.21.15.157:57638) 2023-11-06T03:17:55.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 -- 172.21.15.157:0/1494148936 learned_addr learned my addr 172.21.15.157:0/1494148936 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:55.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10475841 0 0) 0x7f47e015e440 con 0x7f47e00b2af0 2023-11-06T03:17:55.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47df7fe640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47e00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47cc003530 con 0x7f47e00b2af0 2023-11-06T03:17:55.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165376790 0 0) 0x7f47cc003530 con 0x7f47e00b2af0 2023-11-06T03:17:55.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 msgr2=0x7f47e00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47e00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 >> v1:172.21.15.195:6789/0 conn(0x7f47e006d000 legacy=0x7f47e00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e006c800 con 0x7f47e00b2af0 2023-11-06T03:17:55.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f47d8001a00 con 0x7f47e00b2af0 2023-11-06T03:17:55.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 >> v1:172.21.15.157:6789/0 conn(0x7f47e00b2af0 legacy=0x7f47e0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47cc0039b0 0x7f47cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006c520 con 0x7f47cc004d10 2023-11-06T03:17:55.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00dd30 con 0x7f47cc0039b0 2023-11-06T03:17:55.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00dec0 con 0x7f47e01599c0 2023-11-06T03:17:55.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47df7fe640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47cc0039b0 0x7f47cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 -- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 msgr2=0x7f47cc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47cc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 -- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47cc0039b0 msgr2=0x7f47cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47cc0039b0 0x7f47cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47dffff640 1 -- 172.21.15.157:0/1494148936 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc003530 con 0x7f47cc004d10 2023-11-06T03:17:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.042+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e01599c0 0x7f47cc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dffff640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47cc005190 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f47d800a820 tx=0x7f47d801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78697816c72dff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80072d0 con 0x7f47cc004d10 2023-11-06T03:17:55.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d80a8070 con 0x7f47cc004d10 2023-11-06T03:17:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47deffd640 1 -- 172.21.15.157:0/1494148936 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80076b0 con 0x7f47cc004d10 2023-11-06T03:17:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1494148936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 msgr2=0x7f47cc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1494148936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47cc005190 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f47d800a820 tx=0x7f47d801d2e0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dffff640 1 -- 172.21.15.157:0/1494148936 reap_dead start 2023-11-06T03:17:55.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1494148936 shutdown_connections 2023-11-06T03:17:55.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1494148936 >> 172.21.15.157:0/1494148936 conn(0x7f47e0065930 msgr2=0x7f47e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1494148936 shutdown_connections 2023-11-06T03:17:55.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1494148936 wait complete. 2023-11-06T03:17:55.057 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 Processor -- start 2023-11-06T03:17:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- start start 2023-11-06T03:17:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e0156ae0 0x7f47e01523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.058 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e00ac8b0 0x7f47e00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00ad270 con 0x7f47cc004d10 2023-11-06T03:17:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0152920 con 0x7f47e00ac8b0 2023-11-06T03:17:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01589d0 con 0x7f47e0156ae0 2023-11-06T03:17:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60296/0 (socket says 172.21.15.157:60296) 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 -- 172.21.15.157:0/1894053818 learned_addr learned my addr 172.21.15.157:0/1894053818 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dffff640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e00ac8b0 0x7f47e00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e0156ae0 0x7f47e01523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 -- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e0156ae0 msgr2=0x7f47e01523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e0156ae0 0x7f47e01523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.060 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 -- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e00ac8b0 msgr2=0x7f47e00acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e00ac8b0 0x7f47e00acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 -- 172.21.15.157:0/1894053818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0158cb0 con 0x7f47cc004d10 2023-11-06T03:17:55.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47df7fe640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f47c800f580 tx=0x7f47c800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dcff9640 1 -- 172.21.15.157:0/1894053818 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8013810 con 0x7f47cc004d10 2023-11-06T03:17:55.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dcff9640 1 -- 172.21.15.157:0/1894053818 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8013970 con 0x7f47cc004d10 2023-11-06T03:17:55.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47dcff9640 1 -- 172.21.15.157:0/1894053818 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8013d20 con 0x7f47cc004d10 2023-11-06T03:17:55.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0158f90 con 0x7f47cc004d10 2023-11-06T03:17:55.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.046+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e01545c0 con 0x7f47cc004d10 2023-11-06T03:17:55.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dcff9640 1 -- 172.21.15.157:0/1894053818 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f47c801f460 con 0x7f47cc004d10 2023-11-06T03:17:55.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dcff9640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 0x7f47c0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dcff9640 1 -- 172.21.15.157:0/1894053818 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f47c80836b0 con 0x7f47cc004d10 2023-11-06T03:17:55.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 0x7f47c0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 Processor -- start 2023-11-06T03:17:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 -- start start 2023-11-06T03:17:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 0x7f47c0067f50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f47e0155ec0 tx=0x7f47d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x5609469f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609468b8c10 con 0x5609469f1260 2023-11-06T03:17:55.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609468473e0 con 0x560946a3b130 2023-11-06T03:17:55.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609468449b0 con 0x560946a3ba50 2023-11-06T03:17:55.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60306/0 (socket says 172.21.15.157:60306) 2023-11-06T03:17:55.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 -- 172.21.15.157:0/2312701011 learned_addr learned my addr 172.21.15.157:0/2312701011 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47df7fe640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47d7fff640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x5609469f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 -- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 msgr2=0x5609469f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x5609469f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 -- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 msgr2=0x560946a3b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 -- 172.21.15.157:0/2312701011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56094653ce20 con 0x5609469f1260 2023-11-06T03:17:55.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47df7fe640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47d7fff640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x5609469f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47dffff640 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f47d800a820 tx=0x7f47e006c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa6ed8a8008cab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.050+0000 7f47d57fa640 1 -- 172.21.15.157:0/2312701011 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80011d0 con 0x5609469f1260 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d57fa640 1 -- 172.21.15.157:0/2312701011 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d80017c0 con 0x5609469f1260 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d57fa640 1 -- 172.21.15.157:0/2312701011 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8004b70 con 0x5609469f1260 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/2312701011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 msgr2=0x5609469f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f47d800a820 tx=0x7f47e006c130 comp rx=0 tx=0).stop 2023-11-06T03:17:55.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/2312701011 shutdown_connections 2023-11-06T03:17:55.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x5609469f1640 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x5609469f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- 172.21.15.157:0/2312701011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/2312701011 >> 172.21.15.157:0/2312701011 conn(0x5609469f79f0 msgr2=0x5609469ef600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/2312701011 shutdown_connections 2023-11-06T03:17:55.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/2312701011 wait complete. 2023-11-06T03:17:55.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- start start 2023-11-06T03:17:55.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 Processor -- start 2023-11-06T03:17:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609468402a0 con 0x5609469f1260 2023-11-06T03:17:55.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609467df470 con 0x560946a3b130 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609467fd660 con 0x560946a3ba50 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 10 client.0 _ll_register_callbacks cb 0x5609469dcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60322/0 (socket says 172.21.15.157:60322) 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 -- 172.21.15.157:0/99264089 learned_addr learned my addr 172.21.15.157:0/99264089 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d7fff640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 -- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 msgr2=0x560946a3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 -- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 msgr2=0x560946a3c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 -- 172.21.15.157:0/99264089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d80b1040 con 0x5609469f1260 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d7fff640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47df7fe640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f47c805c630 tx=0x7f47c80553c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2babdc9b959cc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c805c8d0 con 0x5609469f1260 2023-11-06T03:17:55.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f47c8081e40 con 0x5609469f1260 2023-11-06T03:17:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609467fd0a0 con 0x5609469f1260 2023-11-06T03:17:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c8071120 con 0x5609469f1260 2023-11-06T03:17:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 10 client.11013 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x5609468051d0 con 0x5609469f1260 2023-11-06T03:17:55.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609467fd0a0 con 0x5609469f1260 2023-11-06T03:17:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f47c8055b90 con 0x5609469f1260 2023-11-06T03:17:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 10 client.11013 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 10 client.11013 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2023-11-06T03:17:55.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 20 client.11013 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:55.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a3d2c0 0x560946a4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 4 client.11013 mds_command: new command op to 10818 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:17:55.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56094683ebf0 con 0x560946a3d2c0 2023-11-06T03:17:55.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d7fff640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a3d2c0 0x560946a4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47d7fff640 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a3d2c0 0x560946a4cd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4135a62bcd8b948f server_cookie=ae10f6f17f47b60c in_seq=0 out_seq=0 2023-11-06T03:17:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.054+0000 7f47be7fc640 10 client.11013 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47be7fc640 1 -- 172.21.15.157:0/99264089 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56094683ebf0 con 0x560946a3d2c0 2023-11-06T03:17:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47be7fc640 10 client.11013 handle_command_reply: tid=0 2023-11-06T03:17:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 client.11013 shutdown 2023-11-06T03:17:55.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 msgr2=0x560946a3be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f47c805c630 tx=0x7f47c80553c0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 shutdown_connections 2023-11-06T03:17:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a3d2c0 0x560946a4cd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47be7fc640 0 client.11013 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3b130 0x560946a3c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 --2- 172.21.15.157:0/99264089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5609469f1260 0x560946a3be30 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 >> 172.21.15.157:0/99264089 conn(0x5609469f79f0 msgr2=0x5609469efd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 shutdown_connections 2023-11-06T03:17:55.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 1 -- 172.21.15.157:0/99264089 wait complete. 2023-11-06T03:17:55.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.058+0000 7f47e8e83000 20 client.11013 trim_cache size 0 max 16384 2023-11-06T03:17:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 Processor -- start 2023-11-06T03:17:55.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 -- start start 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5609469f1260 0x560946a8ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3ba50 0x560946a48370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609464fdaa0 con 0x560946a3b130 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609467fcaa0 con 0x560946a3ba50 2023-11-06T03:17:55.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609467f0d20 con 0x5609469f1260 2023-11-06T03:17:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60334/0 (socket says 172.21.15.157:60334) 2023-11-06T03:17:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 -- 172.21.15.157:0/1451399500 learned_addr learned my addr 172.21.15.157:0/1451399500 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 -- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5609469f1260 msgr2=0x560946a8ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5609469f1260 0x560946a8ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 -- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3ba50 msgr2=0x560946a48370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3ba50 0x560946a48370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 -- 172.21.15.157:0/1451399500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609467eb520 con 0x560946a3b130 2023-11-06T03:17:55.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d7fff640 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x5609467dd2e0 tx=0x7f47d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20ab558f1e7ba8b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d5ffb640 1 -- 172.21.15.157:0/1451399500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0029030 con 0x560946a3b130 2023-11-06T03:17:55.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d5ffb640 1 -- 172.21.15.157:0/1451399500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d000d050 con 0x560946a3b130 2023-11-06T03:17:55.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47d5ffb640 1 -- 172.21.15.157:0/1451399500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d000fdf0 con 0x560946a3b130 2023-11-06T03:17:55.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 -- 172.21.15.157:0/1451399500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 msgr2=0x560946a47e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.070+0000 7f47e8e83000 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x5609467dd2e0 tx=0x7f47d0012000 comp rx=0 tx=0).stop 2023-11-06T03:17:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/1451399500 shutdown_connections 2023-11-06T03:17:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560946a3ba50 0x560946a48370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a47e30 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/1451399500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5609469f1260 0x560946a8ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/1451399500 >> 172.21.15.157:0/1451399500 conn(0x5609469f79f0 msgr2=0x560946a47720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/1451399500 shutdown_connections 2023-11-06T03:17:55.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/1451399500 wait complete. 2023-11-06T03:17:55.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- start start 2023-11-06T03:17:55.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 Processor -- start 2023-11-06T03:17:55.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5609469f1260 0x560946a98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609469f4d40 con 0x560946a3b130 2023-11-06T03:17:55.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609469f4020 con 0x5609469f1260 2023-11-06T03:17:55.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094683f130 con 0x560946a3ba50 2023-11-06T03:17:55.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 10 client.0 _ll_register_callbacks cb 0x560946804cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:55.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60336/0 (socket says 172.21.15.157:60336) 2023-11-06T03:17:55.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 -- 172.21.15.157:0/3359510887 learned_addr learned my addr 172.21.15.157:0/3359510887 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 -- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 msgr2=0x560946a7c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 -- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5609469f1260 msgr2=0x560946a98e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5609469f1260 0x560946a98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 -- 172.21.15.157:0/3359510887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d0019040 con 0x560946a3b130 2023-11-06T03:17:55.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47d7fff640 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f47d0001430 tx=0x7f47d00011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f8c3212dc4a39b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:55.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0025130 con 0x560946a3b130 2023-11-06T03:17:55.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f47d002a070 con 0x560946a3b130 2023-11-06T03:17:55.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560946809620 con 0x560946a3b130 2023-11-06T03:17:55.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47d0025290 con 0x560946a3b130 2023-11-06T03:17:55.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 10 client.11019 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:55.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x5609467df090 con 0x560946a3b130 2023-11-06T03:17:55.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560946809620 con 0x560946a3b130 2023-11-06T03:17:55.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f47d0012070 con 0x560946a3b130 2023-11-06T03:17:55.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 10 client.11019 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:55.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 10 client.11019 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2023-11-06T03:17:55.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 20 client.11019 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:55.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a7c770 0x560946a7cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 4 client.11019 mds_command: new command op to 10818 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:17:55.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560946839320 con 0x560946a7c770 2023-11-06T03:17:55.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47df7fe640 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a7c770 0x560946a7cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 10 client.11019 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47df7fe640 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a7c770 0x560946a7cb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73394ae99fe6e26e server_cookie=4f53489f0c3db2ed in_seq=0 out_seq=0 2023-11-06T03:17:55.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 1 -- 172.21.15.157:0/3359510887 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560946839320 con 0x560946a7c770 2023-11-06T03:17:55.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 10 client.11019 handle_command_reply: tid=0 2023-11-06T03:17:55.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 client.11019 shutdown 2023-11-06T03:17:55.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 msgr2=0x560946a99390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f47d0001430 tx=0x7f47d00011a0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 shutdown_connections 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x560946a7c770 0x560946a7cb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560946a3ba50 0x560946a7c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560946a3b130 0x560946a99390 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 --2- 172.21.15.157:0/3359510887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5609469f1260 0x560946a98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47be7fc640 0 client.11019 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 >> 172.21.15.157:0/3359510887 conn(0x5609469f79f0 msgr2=0x560946a47fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 shutdown_connections 2023-11-06T03:17:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 1 -- 172.21.15.157:0/3359510887 wait complete. 2023-11-06T03:17:55.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e8e83000 20 client.11019 trim_cache size 0 max 16384 2023-11-06T03:17:55.107 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:17:55.107 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:17:55.107 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:17:55.108 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:17:55.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 msgr2=0x7f47c0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 0x7f47c0067f50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f47e0155ec0 tx=0x7f47d0009040 comp rx=0 tx=0).stop 2023-11-06T03:17:55.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 msgr2=0x7f47e0158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f47c800f580 tx=0x7f47c800fa40 comp rx=0 tx=0).stop 2023-11-06T03:17:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 shutdown_connections 2023-11-06T03:17:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47c0065a90 0x7f47c0067f50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47e00ac8b0 0x7f47e00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47e0156ae0 0x7f47e01523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 --2- 172.21.15.157:0/1894053818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47cc004d10 0x7f47e0158360 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 >> 172.21.15.157:0/1894053818 conn(0x7f47e0065930 msgr2=0x7f47e015b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.074+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 shutdown_connections 2023-11-06T03:17:55.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.078+0000 7f47e5d5b640 1 -- 172.21.15.157:0/1894053818 wait complete. 2023-11-06T03:17:55.112 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:17:55.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.12:0 damage ls 2023-11-06T03:17:55.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 Processor -- start 2023-11-06T03:17:55.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 -- start start 2023-11-06T03:17:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31fc0ccb70 0x7f31fc15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f31fc0cad20 0x7f31fc0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31fc15e100 con 0x7f31fc0adf70 2023-11-06T03:17:55.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc0cb690 con 0x7f31fc0ccb70 2023-11-06T03:17:55.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f3201f00640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31fc15cff0 con 0x7f31fc0cad20 2023-11-06T03:17:55.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31fc0ccb70 0x7f31fc15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f31fc0ccb70 0x7f31fc15dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60342/0 (socket says 172.21.15.157:60342) 2023-11-06T03:17:55.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 -- 172.21.15.157:0/3131613393 learned_addr learned my addr 172.21.15.157:0/3131613393 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31fbfff640 1 --2- 172.21.15.157:0/3131613393 >> v2:172.21.15.195:3300/0 conn(0x7f31fc0cad20 0x7f31fc0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 -- 172.21.15.157:0/3131613393 >> v2:172.21.15.195:3300/0 conn(0x7f31fc0cad20 msgr2=0x7f31fc0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 --2- 172.21.15.157:0/3131613393 >> v2:172.21.15.195:3300/0 conn(0x7f31fc0cad20 0x7f31fc0cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 -- 172.21.15.157:0/3131613393 >> v1:172.21.15.195:6789/0 conn(0x7f31fc0adf70 legacy=0x7f31fc161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.222+0000 7f31faffd640 1 -- 172.21.15.157:0/3131613393 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc0cd130 con 0x7f31fc0ccb70 2023-11-06T03:17:55.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f31faffd640 1 --2- 172.21.15.157:0/3131613393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc15dbc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f31fc066600 tx=0x7f31e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29a56531acd37fa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f31fa7fc640 1 -- 172.21.15.157:0/3131613393 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40a4ea0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f31fa7fc640 1 -- 172.21.15.157:0/3131613393 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e40a7050 con 0x7f31fc0ccb70 2023-11-06T03:17:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f31fa7fc640 1 -- 172.21.15.157:0/3131613393 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40ab820 con 0x7f31fc0ccb70 2023-11-06T03:17:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- 172.21.15.157:0/3131613393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 msgr2=0x7f31fc15dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 --2- 172.21.15.157:0/3131613393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc15dbc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f31fc066600 tx=0x7f31e401c410 comp rx=0 tx=0).stop 2023-11-06T03:17:55.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- 172.21.15.157:0/3131613393 shutdown_connections 2023-11-06T03:17:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 --2- 172.21.15.157:0/3131613393 >> v2:172.21.15.195:3300/0 conn(0x7f31fc0cad20 0x7f31fc0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 --2- 172.21.15.157:0/3131613393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc15dbc0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- 172.21.15.157:0/3131613393 >> 172.21.15.157:0/3131613393 conn(0x7f31fc0b0b50 msgr2=0x7f31fc0b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- 172.21.15.157:0/3131613393 shutdown_connections 2023-11-06T03:17:55.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- 172.21.15.157:0/3131613393 wait complete. 2023-11-06T03:17:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 Processor -- start 2023-11-06T03:17:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.226+0000 7f3201f00640 1 -- start start 2023-11-06T03:17:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 0x7f31fc153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 0x7f31fc1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc154cf0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc154f20 con 0x7f31fc0cad20 2023-11-06T03:17:55.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0c6d10 con 0x7f31fc0adf70 2023-11-06T03:17:55.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60350/0 (socket says 172.21.15.157:60350) 2023-11-06T03:17:55.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 -- 172.21.15.157:0/3213468050 learned_addr learned my addr 172.21.15.157:0/3213468050 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 0x7f31fc153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31faffd640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 0x7f31fc1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 -- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 msgr2=0x7f31fc153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 0x7f31fc153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 -- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 msgr2=0x7f31fc1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 0x7f31fc1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 -- 172.21.15.157:0/3213468050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40094d0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 0x7f31fc153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31faffd640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 0x7f31fc1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fbfff640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007e40 tx=0x7f31f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 -- 172.21.15.157:0/3213468050 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4014620 con 0x7f31fc0ccb70 2023-11-06T03:17:55.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 -- 172.21.15.157:0/3213468050 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f400ee90 con 0x7f31fc0ccb70 2023-11-06T03:17:55.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc0c6ff0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 -- 172.21.15.157:0/3213468050 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40149e0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0c69c0 con 0x7f31fc0ccb70 2023-11-06T03:17:55.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 -- 172.21.15.157:0/3213468050 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f31f4014b40 con 0x7f31fc0ccb70 2023-11-06T03:17:55.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 0x7f31dc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31f3fff640 1 -- 172.21.15.157:0/3213468050 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f31f4083c70 con 0x7f31fc0ccb70 2023-11-06T03:17:55.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.230+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 0x7f31dc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 Processor -- start 2023-11-06T03:17:55.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- start start 2023-11-06T03:17:55.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 0x7f31dc067f50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0c91a0 tx=0x7f31ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2c39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2befa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2ab7d60 con 0x5574b2beff90 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a463b0 con 0x5574b2c3a530 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a43980 con 0x5574b2c39c10 2023-11-06T03:17:55.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60360/0 (socket says 172.21.15.157:60360) 2023-11-06T03:17:55.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 -- 172.21.15.157:0/855396754 learned_addr learned my addr 172.21.15.157:0/855396754 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2c39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31faffd640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2befa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 -- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 msgr2=0x5574b2c39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2c39ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 -- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 msgr2=0x5574b2befa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2befa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 -- 172.21.15.157:0/855396754 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b29e0110 con 0x5574b2beff90 2023-11-06T03:17:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31faffd640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2befa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31fbfff640 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f31f405cc70 tx=0x7f31f40558c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b93e20c056ccf97 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31f17fa640 1 -- 172.21.15.157:0/855396754 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f406c070 con 0x5574b2beff90 2023-11-06T03:17:55.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31f17fa640 1 -- 172.21.15.157:0/855396754 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f4058ff0 con 0x5574b2beff90 2023-11-06T03:17:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f31f17fa640 1 -- 172.21.15.157:0/855396754 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f405d1a0 con 0x5574b2beff90 2023-11-06T03:17:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- 172.21.15.157:0/855396754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 msgr2=0x5574b2bf0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f31f405cc70 tx=0x7f31f40558c0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- 172.21.15.157:0/855396754 shutdown_connections 2023-11-06T03:17:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf0370 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2befa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- 172.21.15.157:0/855396754 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2c39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- 172.21.15.157:0/855396754 >> 172.21.15.157:0/855396754 conn(0x5574b2bebf40 msgr2=0x5574b2bee330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- 172.21.15.157:0/855396754 shutdown_connections 2023-11-06T03:17:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- 172.21.15.157:0/855396754 wait complete. 2023-11-06T03:17:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- start start 2023-11-06T03:17:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 Processor -- start 2023-11-06T03:17:55.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 0x5574b2bf5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c27ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a3f270 con 0x5574b2beff90 2023-11-06T03:17:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b29de5a0 con 0x5574b2c39c10 2023-11-06T03:17:55.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.234+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b29fc7c0 con 0x5574b2c3a530 2023-11-06T03:17:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 10 client.0 _ll_register_callbacks cb 0x5574b2a33130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60368/0 (socket says 172.21.15.157:60368) 2023-11-06T03:17:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 -- 172.21.15.157:0/2052999926 learned_addr learned my addr 172.21.15.157:0/2052999926 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fbfff640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c27ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31faffd640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 0x5574b2bf5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 -- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 msgr2=0x5574b2c27ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c27ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 -- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 msgr2=0x5574b2bf5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 0x5574b2bf5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 -- 172.21.15.157:0/2052999926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc0c69c0 con 0x5574b2beff90 2023-11-06T03:17:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31faffd640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 0x5574b2bf5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00dcf0 tx=0x7f31ec003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aac193fa6c6049 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:55.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007230 con 0x5574b2beff90 2023-11-06T03:17:55.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f31ec007390 con 0x5574b2beff90 2023-11-06T03:17:55.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574b2a03350 con 0x5574b2beff90 2023-11-06T03:17:55.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ec014e90 con 0x5574b2beff90 2023-11-06T03:17:55.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 10 client.11031 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:55.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x5574b273be20 con 0x5574b2beff90 2023-11-06T03:17:55.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574b2a03350 con 0x5574b2beff90 2023-11-06T03:17:55.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f31ec007550 con 0x5574b2beff90 2023-11-06T03:17:55.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 10 client.11031 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:55.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 10 client.11031 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2023-11-06T03:17:55.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 20 client.11031 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:55.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c288f0 0x5574b2c4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 4 client.11031 mds_command: new command op to 10818 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:17:55.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574b2a3dbc0 con 0x5574b2c288f0 2023-11-06T03:17:55.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31faffd640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c288f0 0x5574b2c4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31faffd640 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c288f0 0x5574b2c4bcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76a0802c6c3a9ca server_cookie=b96c071ce0e0b0c5 in_seq=0 out_seq=0 2023-11-06T03:17:55.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 10 client.11031 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 1 -- 172.21.15.157:0/2052999926 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5574b2a3dbc0 con 0x5574b2c288f0 2023-11-06T03:17:55.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 10 client.11031 handle_command_reply: tid=0 2023-11-06T03:17:55.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 client.11031 shutdown 2023-11-06T03:17:55.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 msgr2=0x5574b2bf4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00dcf0 tx=0x7f31ec003700 comp rx=0 tx=0).stop 2023-11-06T03:17:55.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 shutdown_connections 2023-11-06T03:17:55.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c288f0 0x5574b2c4bcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c27ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c39c10 0x5574b2bf5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 --2- 172.21.15.157:0/2052999926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bf4e10 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f31da7fc640 0 client.11031 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 >> 172.21.15.157:0/2052999926 conn(0x5574b2bebf40 msgr2=0x5574b2beebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 shutdown_connections 2023-11-06T03:17:55.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 1 -- 172.21.15.157:0/2052999926 wait complete. 2023-11-06T03:17:55.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.238+0000 7f3205028000 20 client.11031 trim_cache size 0 max 16384 2023-11-06T03:17:55.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 Processor -- start 2023-11-06T03:17:55.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- start start 2023-11-06T03:17:55.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2bef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2c4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b26fcaa0 con 0x5574b2beff90 2023-11-06T03:17:55.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a3c670 con 0x5574b2c3a530 2023-11-06T03:17:55.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a3abe0 con 0x5574b2c39c10 2023-11-06T03:17:55.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60376/0 (socket says 172.21.15.157:60376) 2023-11-06T03:17:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 -- 172.21.15.157:0/261924792 learned_addr learned my addr 172.21.15.157:0/261924792 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 -- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 msgr2=0x5574b2bef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2bef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 -- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 msgr2=0x5574b2c4bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2c4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 -- 172.21.15.157:0/261924792 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b2a041a0 con 0x5574b2beff90 2023-11-06T03:17:55.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f31ec003740 tx=0x7f31ec01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a645268572b9805 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31f1ffb640 1 -- 172.21.15.157:0/261924792 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec006c20 con 0x5574b2beff90 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31f1ffb640 1 -- 172.21.15.157:0/261924792 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec014e90 con 0x5574b2beff90 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f31f1ffb640 1 -- 172.21.15.157:0/261924792 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0071d0 con 0x5574b2beff90 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- 172.21.15.157:0/261924792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 msgr2=0x5574b2bef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f31ec003740 tx=0x7f31ec01d000 comp rx=0 tx=0).stop 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- 172.21.15.157:0/261924792 shutdown_connections 2023-11-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2c3a530 0x5574b2c4bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c39c10 0x5574b2bef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- 172.21.15.157:0/261924792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2beff90 0x5574b2bef340 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- 172.21.15.157:0/261924792 >> 172.21.15.157:0/261924792 conn(0x5574b2bebf40 msgr2=0x5574b2c47200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- 172.21.15.157:0/261924792 shutdown_connections 2023-11-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- 172.21.15.157:0/261924792 wait complete. 2023-11-06T03:17:55.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- start start 2023-11-06T03:17:55.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 Processor -- start 2023-11-06T03:17:55.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2beff90 0x5574b2c97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c9c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a3ca80 con 0x5574b2c39c10 2023-11-06T03:17:55.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a39610 con 0x5574b2beff90 2023-11-06T03:17:55.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b2a47640 con 0x5574b2c3a530 2023-11-06T03:17:55.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.250+0000 7f3205028000 10 client.0 _ll_register_callbacks cb 0x5574b29ed780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:17:55.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60384/0 (socket says 172.21.15.157:60384) 2023-11-06T03:17:55.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 -- 172.21.15.157:0/4118063948 learned_addr learned my addr 172.21.15.157:0/4118063948 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 -- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 msgr2=0x5574b2c9c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c9c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 -- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2beff90 msgr2=0x5574b2c97bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2beff90 0x5574b2c97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 -- 172.21.15.157:0/4118063948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574b273be20 con 0x5574b2c39c10 2023-11-06T03:17:55.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31faffd640 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006c90 tx=0x7f31e40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9338282dcacd89da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:17:55.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40ba430 con 0x5574b2c39c10 2023-11-06T03:17:55.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f31e40ab620 con 0x5574b2c39c10 2023-11-06T03:17:55.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574b29eec20 con 0x5574b2c39c10 2023-11-06T03:17:55.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31e40ba950 con 0x5574b2c39c10 2023-11-06T03:17:55.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 10 client.11037 fetch_fsmap learned FSMap version 123 2023-11-06T03:17:55.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=2+}) v3 -- 0x5574b29ea4f0 con 0x5574b2c39c10 2023-11-06T03:17:55.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574b29eec20 con 0x5574b2c39c10 2023-11-06T03:17:55.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 123) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f31e40baab0 con 0x5574b2c39c10 2023-11-06T03:17:55.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 10 client.11037 fetch_fsmap finished waiting for FSMap version 123 2023-11-06T03:17:55.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 10 client.11037 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.c 2023-11-06T03:17:55.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 20 client.11037 populate_metadata read hostname 'smithi157' 2023-11-06T03:17:55.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c9cb90 0x5574b2c9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 4 client.11037 mds_command: new command op to 10818 tid=0[{"prefix": "damage ls"}] 2023-11-06T03:17:55.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 --> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574b2a382f0 con 0x5574b2c9cb90 2023-11-06T03:17:55.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c9cb90 0x5574b2c9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31fb7fe640 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c9cb90 0x5574b2c9ef80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1cfe1f026e0da5 server_cookie=860e4f2412a4a175 in_seq=0 out_seq=0 2023-11-06T03:17:55.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 10 client.11037 ms_handle_connect on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 1 -- 172.21.15.157:0/4118063948 <== mds.0 v2:172.21.15.157:6832/4092537436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574b2a382f0 con 0x5574b2c9cb90 2023-11-06T03:17:55.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 10 client.11037 handle_command_reply: tid=0 2023-11-06T03:17:55.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 client.11037 shutdown 2023-11-06T03:17:55.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 msgr2=0x5574b2c9a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006c90 tx=0x7f31e40191b0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 shutdown_connections 2023-11-06T03:17:55.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:6832/4092537436,v1:172.21.15.157:6834/4092537436] conn(0x5574b2c9cb90 0x5574b2c9ef80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5574b2c3a530 0x5574b2c9c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f31da7fc640 0 client.11037 ms_handle_reset on v2:172.21.15.157:6832/4092537436 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5574b2c39c10 0x5574b2c9a100 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 --2- 172.21.15.157:0/4118063948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574b2beff90 0x5574b2c97bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 >> 172.21.15.157:0/4118063948 conn(0x5574b2bebf40 msgr2=0x5574b2c47200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 shutdown_connections 2023-11-06T03:17:55.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 1 -- 172.21.15.157:0/4118063948 wait complete. 2023-11-06T03:17:55.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3205028000 20 client.11037 trim_cache size 0 max 16384 2023-11-06T03:17:55.284 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2023-11-06T03:17:55.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 msgr2=0x7f31dc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 0x7f31dc067f50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0c91a0 tx=0x7f31ec009040 comp rx=0 tx=0).stop 2023-11-06T03:17:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 msgr2=0x7f31fc0c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f31f4007e40 tx=0x7f31f400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 shutdown_connections 2023-11-06T03:17:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f31dc065a90 0x7f31dc067f50 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.286 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31fc0ccb70 0x7f31fc0c2a90 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f31fc0cad20 0x7f31fc1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 --2- 172.21.15.157:0/3213468050 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f31fc0adf70 0x7f31fc153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 >> 172.21.15.157:0/3213468050 conn(0x7f31fc0b0b50 msgr2=0x7f31fc0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.287 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 shutdown_connections 2023-11-06T03:17:55.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.254+0000 7f3201f00640 1 -- 172.21.15.157:0/3213468050 wait complete. 2023-11-06T03:17:55.288 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:17:55.288 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:55.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:17:55.375 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:17:55.375 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:55.376 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:55.382 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.538+0000 7f4120220640 1 Processor -- start 2023-11-06T03:17:55.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.538+0000 7f4120220640 1 -- start start 2023-11-06T03:17:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f4120220640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f411815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f4120220640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4118160830 con 0x7f41180cde90 2023-11-06T03:17:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f4120220640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4118160b10 con 0x7f411815cc70 2023-11-06T03:17:55.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f4120220640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41181614b0 con 0x7f41180d00a0 2023-11-06T03:17:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411d7bb640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f41180cde90 0x7f41180cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57648/0 (socket says 172.21.15.157:57648) 2023-11-06T03:17:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411d7bb640 1 -- 172.21.15.157:0/1099345016 learned_addr learned my addr 172.21.15.157:0/1099345016 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f411815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405017247 0 0) 0x7f4118160830 con 0x7f41180cde90 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4100003530 con 0x7f41180cde90 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1200696656 0 0) 0x7f4100003530 con 0x7f41180cde90 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 msgr2=0x7f411815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f411815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 >> v1:172.21.15.195:6789/0 conn(0x7f411815cc70 legacy=0x7f41180d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118160df0 con 0x7f41180cde90 2023-11-06T03:17:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f410c001680 con 0x7f41180cde90 2023-11-06T03:17:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 >> v1:172.21.15.157:6789/0 conn(0x7f41180cde90 legacy=0x7f41180cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41000039b0 0x7f4100003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f41000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100004d10 0x7f4100005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181614b0 con 0x7f4100004d10 2023-11-06T03:17:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410000bb30 con 0x7f41000039b0 2023-11-06T03:17:55.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410000bcf0 con 0x7f41180d00a0 2023-11-06T03:17:55.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100004d10 0x7f4100005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411d7bb640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f41000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411dfbc640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41000039b0 0x7f4100003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 -- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 msgr2=0x7f41000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41180d00a0 0x7f41000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 -- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41000039b0 msgr2=0x7f4100003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41000039b0 0x7f4100003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.542+0000 7f411e7bd640 1 -- 172.21.15.157:0/1099345016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100003530 con 0x7f4100004d10 2023-11-06T03:17:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100004d10 0x7f4100005190 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f41180b5680 tx=0x7f410801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e0748c4487a84f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41080a5bd0 con 0x7f4100004d10 2023-11-06T03:17:55.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410800f040 con 0x7f4100004d10 2023-11-06T03:17:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411cfba640 1 -- 172.21.15.157:0/1099345016 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41080ad440 con 0x7f4100004d10 2023-11-06T03:17:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/1099345016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100004d10 msgr2=0x7f4100005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 --2- 172.21.15.157:0/1099345016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4100004d10 0x7f4100005190 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f41180b5680 tx=0x7f410801cbc0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 -- 172.21.15.157:0/1099345016 reap_dead start 2023-11-06T03:17:55.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/1099345016 shutdown_connections 2023-11-06T03:17:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/1099345016 >> 172.21.15.157:0/1099345016 conn(0x7f4118070560 msgr2=0x7f4118070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/1099345016 shutdown_connections 2023-11-06T03:17:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/1099345016 wait complete. 2023-11-06T03:17:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 Processor -- start 2023-11-06T03:17:55.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- start start 2023-11-06T03:17:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4100004d10 0x7f41180c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41180c7400 0x7f411815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c9e90 con 0x7f411815f060 2023-11-06T03:17:55.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411815fa50 con 0x7f41180c7400 2023-11-06T03:17:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c9350 con 0x7f4100004d10 2023-11-06T03:17:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60406/0 (socket says 172.21.15.157:60406) 2023-11-06T03:17:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 -- 172.21.15.157:0/3616586076 learned_addr learned my addr 172.21.15.157:0/3616586076 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411dfbc640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4100004d10 0x7f41180c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 -- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4100004d10 msgr2=0x7f41180c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4100004d10 0x7f41180c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 -- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41180c7400 msgr2=0x7f411815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411d7bb640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41180c7400 0x7f411815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41180c7400 0x7f411815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 -- 172.21.15.157:0/3616586076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180c8ab0 con 0x7f411815f060 2023-11-06T03:17:55.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f411e7bd640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f4108002240 tx=0x7f4108009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41080039a0 con 0x7f411815f060 2023-11-06T03:17:55.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108003b00 con 0x7f411815f060 2023-11-06T03:17:55.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108003e70 con 0x7f411815f060 2023-11-06T03:17:55.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41180c8d90 con 0x7f411815f060 2023-11-06T03:17:55.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.546+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180c87a0 con 0x7f411815f060 2023-11-06T03:17:55.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180c9fd0 con 0x7f411815f060 2023-11-06T03:17:55.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f41080b4040 con 0x7f411815f060 2023-11-06T03:17:55.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f41167fc640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 0x7f4104067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f410811ceb0 con 0x7f411815f060 2023-11-06T03:17:55.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f411dfbc640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 0x7f4104067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.550+0000 7f411dfbc640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 0x7f4104067f20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f41180b7220 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.554+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f41180c9fd0 con 0x7f411815f060 2023-11-06T03:17:55.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.698+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41180c9fd0 con 0x7f411815f060 2023-11-06T03:17:55.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f41167fc640 1 -- 172.21.15.157:0/3616586076 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18018 (secure 0 0 0) 0x7f41180c9fd0 con 0x7f411815f060 2023-11-06T03:17:55.698 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:55.698 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":177,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:51.542063+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.d","mappings":[{"from":3,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:55.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 msgr2=0x7f4104067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 0x7f4104067f20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f41180b7220 tx=0x7f4110009040 comp rx=0 tx=0).stop 2023-11-06T03:17:55.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 msgr2=0x7f411815f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f4108002240 tx=0x7f4108009750 comp rx=0 tx=0).stop 2023-11-06T03:17:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 shutdown_connections 2023-11-06T03:17:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4104065a60 0x7f4104067f20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f411815f060 0x7f411815f510 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41180c7400 0x7f411815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 --2- 172.21.15.157:0/3616586076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4100004d10 0x7f41180c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 >> 172.21.15.157:0/3616586076 conn(0x7f4118070560 msgr2=0x7f41180b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 shutdown_connections 2023-11-06T03:17:55.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.702+0000 7f4120220640 1 -- 172.21.15.157:0/3616586076 wait complete. 2023-11-06T03:17:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:17:39.446794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:17:39.943896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:17:55.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:17:55.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 Processor -- start 2023-11-06T03:17:55.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 -- start start 2023-11-06T03:17:55.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1460071cc0 0x7f146006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 0x7f146006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600cc960 con 0x7f14600ce570 2023-11-06T03:17:55.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146006e170 con 0x7f1460071cc0 2023-11-06T03:17:55.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f1466412640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14600cc680 con 0x7f14600b4910 2023-11-06T03:17:55.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1460071cc0 0x7f146006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1460071cc0 0x7f146006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60418/0 (socket says 172.21.15.157:60418) 2023-11-06T03:17:55.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 -- 172.21.15.157:0/2483747436 learned_addr learned my addr 172.21.15.157:0/2483747436 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f14649af640 1 --2- 172.21.15.157:0/2483747436 >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 0x7f146006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 -- 172.21.15.157:0/2483747436 >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 msgr2=0x7f146006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 --2- 172.21.15.157:0/2483747436 >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 0x7f146006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 -- 172.21.15.157:0/2483747436 >> v1:172.21.15.195:6789/0 conn(0x7f14600ce570 legacy=0x7f146006fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:55.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f145ffff640 1 -- 172.21.15.157:0/2483747436 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146006bde0 con 0x7f1460071cc0 2023-11-06T03:17:55.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.834+0000 7f14649af640 1 --2- 172.21.15.157:0/2483747436 >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 0x7f146006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:55.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- 172.21.15.157:0/2483747436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f146006d6f0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f14600cb9c0 tx=0x7f145001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd4726415f34146 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145effd640 1 -- 172.21.15.157:0/2483747436 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500ab600 con 0x7f1460071cc0 2023-11-06T03:17:55.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145effd640 1 -- 172.21.15.157:0/2483747436 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14500a7050 con 0x7f1460071cc0 2023-11-06T03:17:55.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145effd640 1 -- 172.21.15.157:0/2483747436 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500ab980 con 0x7f1460071cc0 2023-11-06T03:17:55.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- 172.21.15.157:0/2483747436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 msgr2=0x7f146006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- 172.21.15.157:0/2483747436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f146006d6f0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f14600cb9c0 tx=0x7f145001c370 comp rx=0 tx=0).stop 2023-11-06T03:17:55.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- 172.21.15.157:0/2483747436 shutdown_connections 2023-11-06T03:17:55.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- 172.21.15.157:0/2483747436 >> v2:172.21.15.195:3300/0 conn(0x7f14600b4910 0x7f146006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- 172.21.15.157:0/2483747436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f146006d6f0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- 172.21.15.157:0/2483747436 >> 172.21.15.157:0/2483747436 conn(0x7f14600cba30 msgr2=0x7f14600cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- 172.21.15.157:0/2483747436 shutdown_connections 2023-11-06T03:17:55.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- 172.21.15.157:0/2483747436 wait complete. 2023-11-06T03:17:55.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 Processor -- start 2023-11-06T03:17:55.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- start start 2023-11-06T03:17:55.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14600b4910 0x7f1460156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14600ce570 0x7f14601573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601f43f0 con 0x7f1460071cc0 2023-11-06T03:17:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14601f3eb0 con 0x7f14600b4910 2023-11-06T03:17:55.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f1466412640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14601f3610 con 0x7f14600ce570 2023-11-06T03:17:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60434/0 (socket says 172.21.15.157:60434) 2023-11-06T03:17:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 -- 172.21.15.157:0/2354148328 learned_addr learned my addr 172.21.15.157:0/2354148328 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 -- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14600ce570 msgr2=0x7f14601573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.842 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14600ce570 0x7f14601573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 -- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14600b4910 msgr2=0x7f1460156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14600b4910 0x7f1460156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 -- 172.21.15.157:0/2354148328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500094d0 con 0x7f1460071cc0 2023-11-06T03:17:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145ffff640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f14500ae040 tx=0x7f145001abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.843 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.838+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500b6620 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14500b67a0 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14601f38f0 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14500b6b50 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14601f3250 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1460072750 con 0x7f1460071cc0 2023-11-06T03:17:55.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f14500b2040 con 0x7f1460071cc0 2023-11-06T03:17:55.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145cff9640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 0x7f144c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:55.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f145011d860 con 0x7f1460071cc0 2023-11-06T03:17:55.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145f7fe640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 0x7f144c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:55.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.842+0000 7f145f7fe640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 0x7f144c067f20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1460152fc0 tx=0x7f1454006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:55.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.846+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f14500eb0f0 con 0x7f1460071cc0 2023-11-06T03:17:55.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1460072cf0 con 0x7f1460071cc0 2023-11-06T03:17:55.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f145cff9640 1 -- 172.21.15.157:0/2354148328 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14500ee710 con 0x7f1460071cc0 2023-11-06T03:17:55.987 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:17:55.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 msgr2=0x7f144c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.989 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 0x7f144c067f20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1460152fc0 tx=0x7f1454006bd0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 msgr2=0x7f14601f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f14500ae040 tx=0x7f145001abb0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 shutdown_connections 2023-11-06T03:17:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f144c065a60 0x7f144c067f20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14600ce570 0x7f14601573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14600b4910 0x7f1460156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 --2- 172.21.15.157:0/2354148328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1460071cc0 0x7f14601f2d10 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 >> 172.21.15.157:0/2354148328 conn(0x7f14600cba30 msgr2=0x7f14600b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 shutdown_connections 2023-11-06T03:17:55.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:55.990+0000 7f1466412640 1 -- 172.21.15.157:0/2354148328 wait complete. 2023-11-06T03:17:56.004 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:17:56.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.126+0000 7fc820c21640 1 Processor -- start 2023-11-06T03:17:56.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.126+0000 7fc820c21640 1 -- start start 2023-11-06T03:17:56.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc81c0b0ae0 0x7fc81c15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 0x7fc81c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c15e9f0 0x7fc81c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c15ee40 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c1553d0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc820c21640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15f020 con 0x7fc81c15e9f0 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc81c0b0ae0 0x7fc81c15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc81c0b0ae0 0x7fc81c15ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60436/0 (socket says 172.21.15.157:60436) 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 -- 172.21.15.157:0/309304202 learned_addr learned my addr 172.21.15.157:0/309304202 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc813fff640 1 --2- 172.21.15.157:0/309304202 >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 0x7fc81c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81ad9d640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c15e9f0 0x7fc81c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 -- 172.21.15.157:0/309304202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c15e9f0 msgr2=0x7fc81c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:56.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c15e9f0 0x7fc81c06bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 -- 172.21.15.157:0/309304202 >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 msgr2=0x7fc81c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:56.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 --2- 172.21.15.157:0/309304202 >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 0x7fc81c15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 -- 172.21.15.157:0/309304202 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c15f300 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc813fff640 1 --2- 172.21.15.157:0/309304202 >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 0x7fc81c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc81a59c640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ae0 0x7fc81c15ba10 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0cb610 tx=0x7fc80c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcc652b40daefa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc819d9b640 1 -- 172.21.15.157:0/309304202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c06ca90 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc819d9b640 1 -- 172.21.15.157:0/309304202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c0aa5f0 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.130+0000 7fc819d9b640 1 -- 172.21.15.157:0/309304202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0aa980 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/309304202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ae0 msgr2=0x7fc81c15ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ae0 0x7fc81c15ba10 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0cb610 tx=0x7fc80c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/309304202 shutdown_connections 2023-11-06T03:17:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c15e9f0 0x7fc81c06bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- 172.21.15.157:0/309304202 >> v2:172.21.15.195:3300/0 conn(0x7fc81c0b0ec0 0x7fc81c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- 172.21.15.157:0/309304202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ae0 0x7fc81c15ba10 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/309304202 >> 172.21.15.157:0/309304202 conn(0x7fc81c069d70 msgr2=0x7fc81c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/309304202 shutdown_connections 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/309304202 wait complete. 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 Processor -- start 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- start start 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c0b0ae0 0x7fc81c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 0x7fc81c14af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15c6b0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c14ba80 con 0x7fc81c15e9f0 2023-11-06T03:17:56.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c152d00 con 0x7fc81c0b0ae0 2023-11-06T03:17:56.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60444/0 (socket says 172.21.15.157:60444) 2023-11-06T03:17:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 -- 172.21.15.157:0/1193312347 learned_addr learned my addr 172.21.15.157:0/1193312347 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc81ad9d640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 0x7fc81c14af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc81a59c640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c0b0ae0 0x7fc81c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 -- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c0b0ae0 msgr2=0x7fc81c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c0b0ae0 0x7fc81c15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 -- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 msgr2=0x7fc81c14af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 0x7fc81c14af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 -- 172.21.15.157:0/1193312347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0094d0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc81ad9d640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 0x7fc81c14af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:56.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc813fff640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c7c0 tx=0x7fc80400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804007a30 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804007b90 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c15dea0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400ce70 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.134+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c152810 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c15c7f0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc8040113f0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc812ffd640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 0x7fc7fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fc804083860 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc81a59c640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 0x7fc7fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.138+0000 7fc81a59c640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 0x7fc7fc067f20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ae000 tx=0x7fc80c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.142+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc81c15c7f0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.282+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc81c15c7f0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.282+0000 7fc812ffd640 1 -- 172.21.15.157:0/1193312347 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v177) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc81c15c7f0 con 0x7fc81c0b0ec0 2023-11-06T03:17:56.281 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:17:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 msgr2=0x7fc7fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 0x7fc7fc067f20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ae000 tx=0x7fc80c008040 comp rx=0 tx=0).stop 2023-11-06T03:17:56.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 msgr2=0x7fc81c15cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c7c0 tx=0x7fc80400cc80 comp rx=0 tx=0).stop 2023-11-06T03:17:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 shutdown_connections 2023-11-06T03:17:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc7fc065a60 0x7fc7fc067f20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc81c15e9f0 0x7fc81c14af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0b0ec0 0x7fc81c15cad0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 --2- 172.21.15.157:0/1193312347 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc81c0b0ae0 0x7fc81c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 >> 172.21.15.157:0/1193312347 conn(0x7fc81c069d70 msgr2=0x7fc81c0ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 shutdown_connections 2023-11-06T03:17:56.285 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.286+0000 7fc820c21640 1 -- 172.21.15.157:0/1193312347 wait complete. 2023-11-06T03:17:56.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:17:56.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.410+0000 7fa18dc27640 1 Processor -- start 2023-11-06T03:17:56.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.410+0000 7fa18dc27640 1 -- start start 2023-11-06T03:17:56.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa18815e5e0 0x7fa1880b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18815f580 con 0x7fa1881551e0 2023-11-06T03:17:56.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18815efc0 con 0x7fa18815ea90 2023-11-06T03:17:56.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880704b0 con 0x7fa18815e5e0 2023-11-06T03:17:56.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa186ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa18815e5e0 0x7fa1880b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1877fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa1881551e0 0x7fa18806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57660/0 (socket says 172.21.15.157:57660) 2023-11-06T03:17:56.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1877fe640 1 -- 172.21.15.157:0/4145753219 learned_addr learned my addr 172.21.15.157:0/4145753219 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:56.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa186ffd640 1 -- 172.21.15.157:0/4145753219 >> v1:172.21.15.195:6789/0 conn(0x7fa18815ea90 legacy=0x7fa18815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:56.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa186ffd640 1 -- 172.21.15.157:0/4145753219 >> v1:172.21.15.157:6789/0 conn(0x7fa1881551e0 legacy=0x7fa18806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa186ffd640 1 -- 172.21.15.157:0/4145753219 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18815f2a0 con 0x7fa18815e5e0 2023-11-06T03:17:56.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1867fc640 1 -- 172.21.15.157:0/4145753219 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795146631 0 0) 0x7fa18815f580 con 0x7fa1881551e0 2023-11-06T03:17:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa186ffd640 1 --2- 172.21.15.157:0/4145753219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815e5e0 0x7fa1880b3410 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097c0 tx=0x7fa17c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130bb6e7fffbec40 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1867fc640 1 -- 172.21.15.157:0/4145753219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0ab600 con 0x7fa18815e5e0 2023-11-06T03:17:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1867fc640 1 -- 172.21.15.157:0/4145753219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c0a7050 con 0x7fa18815e5e0 2023-11-06T03:17:56.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa1867fc640 1 -- 172.21.15.157:0/4145753219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0ab980 con 0x7fa18815e5e0 2023-11-06T03:17:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- 172.21.15.157:0/4145753219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815e5e0 msgr2=0x7fa1880b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 --2- 172.21.15.157:0/4145753219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815e5e0 0x7fa1880b3410 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097c0 tx=0x7fa17c01c380 comp rx=0 tx=0).stop 2023-11-06T03:17:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- 172.21.15.157:0/4145753219 shutdown_connections 2023-11-06T03:17:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 --2- 172.21.15.157:0/4145753219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815e5e0 0x7fa1880b3410 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- 172.21.15.157:0/4145753219 >> 172.21.15.157:0/4145753219 conn(0x7fa188065930 msgr2=0x7fa188065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- 172.21.15.157:0/4145753219 shutdown_connections 2023-11-06T03:17:56.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.414+0000 7fa18dc27640 1 -- 172.21.15.157:0/4145753219 wait complete. 2023-11-06T03:17:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 Processor -- start 2023-11-06T03:17:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- start start 2023-11-06T03:17:56.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1881551e0 0x7fa188145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18815e5e0 0x7fa1880b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1881464f0 con 0x7fa18815ea90 2023-11-06T03:17:56.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18814ce30 con 0x7fa1881551e0 2023-11-06T03:17:56.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa188149a60 con 0x7fa18815e5e0 2023-11-06T03:17:56.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60460/0 (socket says 172.21.15.157:60460) 2023-11-06T03:17:56.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 -- 172.21.15.157:0/1490151777 learned_addr learned my addr 172.21.15.157:0/1490151777 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:56.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa1877fe640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1881551e0 0x7fa188145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 -- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18815e5e0 msgr2=0x7fa1880b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18815e5e0 0x7fa1880b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 -- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1881551e0 msgr2=0x7fa188145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1881551e0 0x7fa188145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 -- 172.21.15.157:0/1490151777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009420 con 0x7fa18815ea90 2023-11-06T03:17:56.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa187fff640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa18000b990 tx=0x7fa18000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18000cba0 con 0x7fa18815ea90 2023-11-06T03:17:56.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18000cd00 con 0x7fa18815ea90 2023-11-06T03:17:56.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18814cb50 con 0x7fa18815ea90 2023-11-06T03:17:56.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180014680 con 0x7fa18815ea90 2023-11-06T03:17:56.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1881496f0 con 0x7fa18815ea90 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1881477a0 con 0x7fa18815ea90 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa1800147e0 con 0x7fa18815ea90 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 0x7fa174067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.418+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fa180083260 con 0x7fa18815ea90 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.422+0000 7fa1877fe640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 0x7fa174067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.422+0000 7fa1877fe640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 0x7fa174067f20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa178000f70 tx=0x7fa178009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.422+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa1881477a0 con 0x7fa18815ea90 2023-11-06T03:17:56.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.574+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1881477a0 con 0x7fa18815ea90 2023-11-06T03:17:56.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.574+0000 7fa16ffff640 1 -- 172.21.15.157:0/1490151777 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v177) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1881477a0 con 0x7fa18815ea90 2023-11-06T03:17:56.573 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:17:56.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 msgr2=0x7fa174067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 0x7fa174067f20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa178000f70 tx=0x7fa178009040 comp rx=0 tx=0).stop 2023-11-06T03:17:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 msgr2=0x7fa18814ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa18000b990 tx=0x7fa18000be50 comp rx=0 tx=0).stop 2023-11-06T03:17:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 shutdown_connections 2023-11-06T03:17:56.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa174065a60 0x7fa174067f20 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa18815ea90 0x7fa18814ad20 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa18815e5e0 0x7fa1880b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 --2- 172.21.15.157:0/1490151777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa1881551e0 0x7fa188145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 >> 172.21.15.157:0/1490151777 conn(0x7fa188065930 msgr2=0x7fa1880afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 shutdown_connections 2023-11-06T03:17:56.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.578+0000 7fa18dc27640 1 -- 172.21.15.157:0/1490151777 wait complete. 2023-11-06T03:17:56.595 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:17:56.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.726+0000 7fcd58656640 1 Processor -- start 2023-11-06T03:17:56.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.726+0000 7fcd58656640 1 -- start start 2023-11-06T03:17:56.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd58656640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd5015a990 0x7fcd500cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd58656640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015eff0 0x7fcd500cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd58656640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500cee40 con 0x7fcd500ae710 2023-11-06T03:17:56.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd58656640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500b29f0 con 0x7fcd5015a990 2023-11-06T03:17:56.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd58656640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015f590 con 0x7fcd5015eff0 2023-11-06T03:17:56.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd5015a990 0x7fcd500cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd5015a990 0x7fcd500cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60470/0 (socket says 172.21.15.157:60470) 2023-11-06T03:17:56.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 -- 172.21.15.157:0/3691849640 learned_addr learned my addr 172.21.15.157:0/3691849640 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:56.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd55bf1640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015eff0 0x7fcd500cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 -- 172.21.15.157:0/3691849640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015eff0 msgr2=0x7fcd500cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:56.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015eff0 0x7fcd500cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 -- 172.21.15.157:0/3691849640 >> v1:172.21.15.195:6789/0 conn(0x7fcd500ae710 legacy=0x7fcd500cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 -- 172.21.15.157:0/3691849640 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500cf080 con 0x7fcd5015a990 2023-11-06T03:17:56.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.730+0000 7fcd563f2640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015a990 0x7fcd500cd940 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009860 tx=0x7fcd4001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2a2d860d80d33a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd553f0640 1 -- 172.21.15.157:0/3691849640 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400ab680 con 0x7fcd5015a990 2023-11-06T03:17:56.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd553f0640 1 -- 172.21.15.157:0/3691849640 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd400ab7e0 con 0x7fcd5015a990 2023-11-06T03:17:56.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd553f0640 1 -- 172.21.15.157:0/3691849640 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400abb70 con 0x7fcd5015a990 2023-11-06T03:17:56.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/3691849640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015a990 msgr2=0x7fcd500cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015a990 0x7fcd500cd940 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009860 tx=0x7fcd4001c460 comp rx=0 tx=0).stop 2023-11-06T03:17:56.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/3691849640 shutdown_connections 2023-11-06T03:17:56.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015eff0 0x7fcd500cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- 172.21.15.157:0/3691849640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd5015a990 0x7fcd500cd940 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/3691849640 >> 172.21.15.157:0/3691849640 conn(0x7fcd5006a4d0 msgr2=0x7fcd5006bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/3691849640 shutdown_connections 2023-11-06T03:17:56.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/3691849640 wait complete. 2023-11-06T03:17:56.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 Processor -- start 2023-11-06T03:17:56.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- start start 2023-11-06T03:17:56.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015a990 0x7fcd50163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015eff0 0x7fcd50164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5014a280 con 0x7fcd500ae710 2023-11-06T03:17:56.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50071170 con 0x7fcd5015eff0 2023-11-06T03:17:56.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50071450 con 0x7fcd5015a990 2023-11-06T03:17:56.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60478/0 (socket says 172.21.15.157:60478) 2023-11-06T03:17:56.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 -- 172.21.15.157:0/1514826687 learned_addr learned my addr 172.21.15.157:0/1514826687 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:56.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd55bf1640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015a990 0x7fcd50163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd56bf3640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015eff0 0x7fcd50164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 -- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015a990 msgr2=0x7fcd50163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015a990 0x7fcd50163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 -- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015eff0 msgr2=0x7fcd50164020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015eff0 0x7fcd50164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 -- 172.21.15.157:0/1514826687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40009470 con 0x7fcd500ae710 2023-11-06T03:17:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd563f2640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ad040 tx=0x7fcd4001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400b4800 con 0x7fcd500ae710 2023-11-06T03:17:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd400b4960 con 0x7fcd500ae710 2023-11-06T03:17:56.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400b4d10 con 0x7fcd500ae710 2023-11-06T03:17:56.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd50071730 con 0x7fcd500ae710 2023-11-06T03:17:56.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.734+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50157800 con 0x7fcd500ae710 2023-11-06T03:17:56.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd50157660 con 0x7fcd500ae710 2023-11-06T03:17:56.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcd400c83a0 con 0x7fcd500ae710 2023-11-06T03:17:56.743 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd46ffd640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 0x7fcd34067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:56.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fcd4011df70 con 0x7fcd500ae710 2023-11-06T03:17:56.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd55bf1640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 0x7fcd34067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:56.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.738+0000 7fcd55bf1640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 0x7fcd34067f20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcd5015e320 tx=0x7fcd4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:56.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.742+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fcd50157660 con 0x7fcd500ae710 2023-11-06T03:17:56.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd50157660 con 0x7fcd500ae710 2023-11-06T03:17:56.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd46ffd640 1 -- 172.21.15.157:0/1514826687 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v177) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd50157660 con 0x7fcd500ae710 2023-11-06T03:17:56.888 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:17:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 msgr2=0x7fcd34067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 0x7fcd34067f20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcd5015e320 tx=0x7fcd4c008040 comp rx=0 tx=0).stop 2023-11-06T03:17:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 msgr2=0x7fcd50148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:56.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ad040 tx=0x7fcd4001aca0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 shutdown_connections 2023-11-06T03:17:56.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fcd34065a60 0x7fcd34067f20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd5015eff0 0x7fcd50164020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd5015a990 0x7fcd50163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.890+0000 7fcd58656640 1 --2- 172.21.15.157:0/1514826687 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd500ae710 0x7fcd50148d70 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:56.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.894+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 >> 172.21.15.157:0/1514826687 conn(0x7fcd5006a4d0 msgr2=0x7fcd500b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:56.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.894+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 shutdown_connections 2023-11-06T03:17:56.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:56.894+0000 7fcd58656640 1 -- 172.21.15.157:0/1514826687 wait complete. 2023-11-06T03:17:56.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:17:57.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 Processor -- start 2023-11-06T03:17:57.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 -- start start 2023-11-06T03:17:57.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1ee80b2af0 0x7f1ee80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee815e440 con 0x7f1ee81599c0 2023-11-06T03:17:57.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee806c240 con 0x7f1ee806d000 2023-11-06T03:17:57.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1eed1fe640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee806c520 con 0x7f1ee80b2af0 2023-11-06T03:17:57.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1ee81599c0 0x7f1ee80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57672/0 (socket says 172.21.15.157:57672) 2023-11-06T03:17:57.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/1380498979 learned_addr learned my addr 172.21.15.157:0/1380498979 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:57.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1edffff640 1 --2- 172.21.15.157:0/1380498979 >> v2:172.21.15.195:3300/0 conn(0x7f1ee80b2af0 0x7f1ee80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354266077 0 0) 0x7f1ee815e440 con 0x7f1ee81599c0 2023-11-06T03:17:57.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed4003530 con 0x7f1ee81599c0 2023-11-06T03:17:57.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1269171246 0 0) 0x7f1ed4003530 con 0x7f1ee81599c0 2023-11-06T03:17:57.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 >> v2:172.21.15.195:3300/0 conn(0x7f1ee80b2af0 msgr2=0x7f1ee80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:57.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 --2- 172.21.15.157:0/1380498979 >> v2:172.21.15.195:3300/0 conn(0x7f1ee80b2af0 0x7f1ee80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 >> v1:172.21.15.195:6789/0 conn(0x7f1ee806d000 legacy=0x7f1ee8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee806c800 con 0x7f1ee81599c0 2023-11-06T03:17:57.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1ed8001680 con 0x7f1ee81599c0 2023-11-06T03:17:57.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 >> v1:172.21.15.157:6789/0 conn(0x7f1ee81599c0 legacy=0x7f1ee80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ed4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed40047d0 0x7f1ed4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ed4005190 0x7f1ed400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee806c520 con 0x7f1ed4003980 2023-11-06T03:17:57.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400e240 con 0x7f1ed4005190 2023-11-06T03:17:57.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400e400 con 0x7f1ed40047d0 2023-11-06T03:17:57.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ed4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee759e640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ed4005190 0x7f1ed400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed40047d0 msgr2=0x7f1ed4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:57.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ed40047d0 0x7f1ed4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ed4005190 msgr2=0x7f1ed400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:57.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ed4005190 0x7f1ed400dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee759e640 1 -- 172.21.15.157:0/1380498979 reap_dead start 2023-11-06T03:17:57.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.018+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/1380498979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4003530 con 0x7f1ed4003980 2023-11-06T03:17:57.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ed4003da0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001060 tx=0x7f1ed801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946c898a71c40553 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80058e0 con 0x7f1ed4003980 2023-11-06T03:17:57.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8005a40 con 0x7f1ed4003980 2023-11-06T03:17:57.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1ee659c640 1 -- 172.21.15.157:0/1380498979 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005c70 con 0x7f1ed4003980 2023-11-06T03:17:57.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 -- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 msgr2=0x7f1ed4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ed4003da0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001060 tx=0x7f1ed801cbd0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 -- 172.21.15.157:0/1380498979 shutdown_connections 2023-11-06T03:17:57.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/1380498979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ed4003da0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 -- 172.21.15.157:0/1380498979 >> 172.21.15.157:0/1380498979 conn(0x7f1ee8065930 msgr2=0x7f1ee8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:57.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 -- 172.21.15.157:0/1380498979 shutdown_connections 2023-11-06T03:17:57.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.022+0000 7f1eed1fe640 1 -- 172.21.15.157:0/1380498979 wait complete. 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 Processor -- start 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- start start 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 0x7f1ee8143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ee81547b0 0x7f1ee8144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee81e5890 con 0x7f1ed4003980 2023-11-06T03:17:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8144960 con 0x7f1ee81e5c90 2023-11-06T03:17:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee81e4c10 con 0x7f1ee81547b0 2023-11-06T03:17:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60492/0 (socket says 172.21.15.157:60492) 2023-11-06T03:17:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/2279777431 learned_addr learned my addr 172.21.15.157:0/2279777431 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee759e640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ee81547b0 0x7f1ee8144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edffff640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 0x7f1ee8143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ee81547b0 msgr2=0x7f1ee8144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ee81547b0 0x7f1ee8144420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 msgr2=0x7f1ee8143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 0x7f1ee8143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 -- 172.21.15.157:0/2279777431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee81e4ef0 con 0x7f1ed4003980 2023-11-06T03:17:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edffff640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 0x7f1ee8143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:57.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1ee6d9d640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801ca00 tx=0x7f1ed4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80047f0 con 0x7f1ed4003980 2023-11-06T03:17:57.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8004950 con 0x7f1ed4003980 2023-11-06T03:17:57.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee81e51d0 con 0x7f1ed4003980 2023-11-06T03:17:57.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8004cc0 con 0x7f1ed4003980 2023-11-06T03:17:57.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee81577b0 con 0x7f1ed4003980 2023-11-06T03:17:57.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee81579a0 con 0x7f1ed4003980 2023-11-06T03:17:57.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.026+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ed80a57c0 con 0x7f1ed4003980 2023-11-06T03:17:57.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.030+0000 7f1edf7fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 0x7f1ec8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.030+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1ed811ce40 con 0x7f1ed4003980 2023-11-06T03:17:57.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.030+0000 7f1edffff640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 0x7f1ec8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.030+0000 7f1edffff640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 0x7f1ec8067f20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00099e0 tx=0x7f1ed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.034+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1ee81579a0 con 0x7f1ed4003980 2023-11-06T03:17:57.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.186+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ee81579a0 con 0x7f1ed4003980 2023-11-06T03:17:57.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.186+0000 7f1edf7fe640 1 -- 172.21.15.157:0/2279777431 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v177) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1ee81579a0 con 0x7f1ed4003980 2023-11-06T03:17:57.185 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:17:57.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 msgr2=0x7f1ec8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 0x7f1ec8067f20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00099e0 tx=0x7f1ed0008040 comp rx=0 tx=0).stop 2023-11-06T03:17:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 msgr2=0x7f1ee8157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f1ed801ca00 tx=0x7f1ed4005800 comp rx=0 tx=0).stop 2023-11-06T03:17:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 shutdown_connections 2023-11-06T03:17:57.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1ec8065a60 0x7f1ec8067f20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1ee81547b0 0x7f1ee8144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81e5c90 0x7f1ee8143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 --2- 172.21.15.157:0/2279777431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ed4003980 0x7f1ee8157270 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 >> 172.21.15.157:0/2279777431 conn(0x7f1ee8065930 msgr2=0x7f1ee8154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 shutdown_connections 2023-11-06T03:17:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.190+0000 7f1eed1fe640 1 -- 172.21.15.157:0/2279777431 wait complete. 2023-11-06T03:17:57.201 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:17:57.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:57.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:17:57.211 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[97282]: fuse finished with error 0 and tester_r 0 2023-11-06T03:17:57.214 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:17:57.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:57.215 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:17:57.381 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:57.381 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:57.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:57.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:17:57.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:57.389 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:17:57.439 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:17:57.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:57.440 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:17:57.441 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:57.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:17:57.448 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:17:57.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:57.449 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:17:57.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:57.494 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:17:57.495 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:17:57.495 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:17:57.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:17:57.502 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:17:57.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:17:57.502 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:17:57.552 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:17:57.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:17:57.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:57.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 Processor -- start 2023-11-06T03:17:57.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 -- start start 2023-11-06T03:17:57.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f209c15cf10 0x7f209c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c06c880 con 0x7f209c15cb30 2023-11-06T03:17:57.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c06cb60 con 0x7f209c0660f0 2023-11-06T03:17:57.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f20a1fef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c06d500 con 0x7f209c15cf10 2023-11-06T03:17:57.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f209c15cf10 0x7f209c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209affd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f209c15cf10 0x7f209c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60500/0 (socket says 172.21.15.157:60500) 2023-11-06T03:17:57.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209affd640 1 -- 172.21.15.157:0/3217154936 learned_addr learned my addr 172.21.15.157:0/3217154936 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:57.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209affd640 1 -- 172.21.15.157:0/3217154936 >> v1:172.21.15.195:6789/0 conn(0x7f209c0660f0 legacy=0x7f209c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:57.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209affd640 1 -- 172.21.15.157:0/3217154936 >> v1:172.21.15.157:6789/0 conn(0x7f209c15cb30 legacy=0x7f209c0659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.674+0000 7f209a7fc640 1 -- 172.21.15.157:0/3217154936 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060203805 0 0) 0x7f209c06c880 con 0x7f209c15cb30 2023-11-06T03:17:57.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f209affd640 1 -- 172.21.15.157:0/3217154936 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c06ce40 con 0x7f209c15cf10 2023-11-06T03:17:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f209affd640 1 --2- 172.21.15.157:0/3217154936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c15cf10 0x7f209c0b28f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f2090009870 tx=0x7f209001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d932b897bd3aae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f209a7fc640 1 -- 172.21.15.157:0/3217154936 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900a89f0 con 0x7f209c15cf10 2023-11-06T03:17:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f209a7fc640 1 -- 172.21.15.157:0/3217154936 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900a8b50 con 0x7f209c15cf10 2023-11-06T03:17:57.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f209a7fc640 1 -- 172.21.15.157:0/3217154936 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900b0440 con 0x7f209c15cf10 2023-11-06T03:17:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 -- 172.21.15.157:0/3217154936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c15cf10 msgr2=0x7f209c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 --2- 172.21.15.157:0/3217154936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c15cf10 0x7f209c0b28f0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f2090009870 tx=0x7f209001ecf0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 -- 172.21.15.157:0/3217154936 shutdown_connections 2023-11-06T03:17:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 --2- 172.21.15.157:0/3217154936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c15cf10 0x7f209c0b28f0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 -- 172.21.15.157:0/3217154936 >> 172.21.15.157:0/3217154936 conn(0x7f209c06a170 msgr2=0x7f209c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 -- 172.21.15.157:0/3217154936 shutdown_connections 2023-11-06T03:17:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.678+0000 7f20a1fef640 1 -- 172.21.15.157:0/3217154936 wait complete. 2023-11-06T03:17:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 Processor -- start 2023-11-06T03:17:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- start start 2023-11-06T03:17:57.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f209c15cb30 0x7f209c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209c15cf10 0x7f209c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0c7f30 con 0x7f209c0660f0 2023-11-06T03:17:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f22e0 con 0x7f209c15cb30 2023-11-06T03:17:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c15d620 con 0x7f209c15cf10 2023-11-06T03:17:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60512/0 (socket says 172.21.15.157:60512) 2023-11-06T03:17:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 -- 172.21.15.157:0/1933503309 learned_addr learned my addr 172.21.15.157:0/1933503309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209affd640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f209c15cb30 0x7f209c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209bfff640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209c15cf10 0x7f209c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 -- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209c15cf10 msgr2=0x7f209c154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209c15cf10 0x7f209c154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 -- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f209c15cb30 msgr2=0x7f209c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f209c15cb30 0x7f209c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 -- 172.21.15.157:0/1933503309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900094d0 con 0x7f209c0660f0 2023-11-06T03:17:57.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f209b7fe640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f208c001570 tx=0x7f208c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c003db0 con 0x7f209c0660f0 2023-11-06T03:17:57.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c005070 con 0x7f209c0660f0 2023-11-06T03:17:57.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c005420 con 0x7f209c0660f0 2023-11-06T03:17:57.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c0c8180 con 0x7f209c0660f0 2023-11-06T03:17:57.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c0c8740 con 0x7f209c0660f0 2023-11-06T03:17:57.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c072750 con 0x7f209c0660f0 2023-11-06T03:17:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.682+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f208c005580 con 0x7f209c0660f0 2023-11-06T03:17:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.686+0000 7f2083fff640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 0x7f2088067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.686+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f208c082c20 con 0x7f209c0660f0 2023-11-06T03:17:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.686+0000 7f209affd640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 0x7f2088067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:57.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.686+0000 7f209affd640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 0x7f2088067f20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f20900b2000 tx=0x7f2090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:57.692 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.694+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f208c0504b0 con 0x7f209c0660f0 2023-11-06T03:17:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f209c0c9b80 con 0x7f209c0660f0 2023-11-06T03:17:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f2083fff640 1 -- 172.21.15.157:0/1933503309 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7f208c017020 con 0x7f209c0660f0 2023-11-06T03:17:57.888 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:57.888 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:17:57.888 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:17:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 msgr2=0x7f2088067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 0x7f2088067f20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f20900b2000 tx=0x7f2090008040 comp rx=0 tx=0).stop 2023-11-06T03:17:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 msgr2=0x7f209c1f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f208c001570 tx=0x7f208c00cd00 comp rx=0 tx=0).stop 2023-11-06T03:17:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 shutdown_connections 2023-11-06T03:17:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2088065a60 0x7f2088067f20 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209c15cf10 0x7f209c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f209c15cb30 0x7f209c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 --2- 172.21.15.157:0/1933503309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f209c0660f0 0x7f209c1f1da0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 >> 172.21.15.157:0/1933503309 conn(0x7f209c06a170 msgr2=0x7f209c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.890+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 shutdown_connections 2023-11-06T03:17:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:57.894+0000 7f20a1fef640 1 -- 172.21.15.157:0/1933503309 wait complete. 2023-11-06T03:17:57.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 Processor -- start 2023-11-06T03:17:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 -- start start 2023-11-06T03:17:58.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763815f580 con 0x7f763815ea90 2023-11-06T03:17:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763815efc0 con 0x7f76381551e0 2023-11-06T03:17:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763e50b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76380704b0 con 0x7f763815e5e0 2023-11-06T03:17:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f763815ea90 0x7f763815c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:57698/0 (socket says 172.21.15.157:57698) 2023-11-06T03:17:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 -- 172.21.15.157:0/2459931518 learned_addr learned my addr 172.21.15.157:0/2459931518 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f762f7fe640 1 --2- 172.21.15.157:0/2459931518 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405238068 0 0) 0x7f763815f580 con 0x7f763815ea90 2023-11-06T03:17:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7624003530 con 0x7f763815ea90 2023-11-06T03:17:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235720918 0 0) 0x7f7624003530 con 0x7f763815ea90 2023-11-06T03:17:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 msgr2=0x7f76380b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 --2- 172.21.15.157:0/2459931518 >> v2:172.21.15.195:3300/0 conn(0x7f763815e5e0 0x7f76380b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 >> v1:172.21.15.195:6789/0 conn(0x7f76381551e0 legacy=0x7f763806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763815f2a0 con 0x7f763815ea90 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7630001a00 con 0x7f763815ea90 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 >> v1:172.21.15.157:6789/0 conn(0x7f763815ea90 legacy=0x7f763815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7624003980 0x7f7624003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76240047d0 0x7f7624004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 0x7f762400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76380704b0 con 0x7f7624005190 2023-11-06T03:17:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762400c130 con 0x7f7624003980 2023-11-06T03:17:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762400c2f0 con 0x7f76240047d0 2023-11-06T03:17:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 0x7f762400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f762f7fe640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76240047d0 0x7f7624004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 -- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76240047d0 msgr2=0x7f7624004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76240047d0 0x7f7624004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 -- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7624003980 msgr2=0x7f7624003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f7637fff640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7624003980 0x7f7624003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7624003980 0x7f7624003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f763caa8640 1 -- 172.21.15.157:0/2459931518 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624003530 con 0x7f7624005190 2023-11-06T03:17:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.018+0000 7f762f7fe640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76240047d0 0x7f7624004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763caa8640 1 -- 172.21.15.157:0/2459931518 reap_dead start 2023-11-06T03:17:58.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763caa8640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 0x7f762400b990 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f763000a830 tx=0x7f763001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526ef1d199d17636 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76300aa6d0 con 0x7f7624005190 2023-11-06T03:17:58.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76300aa850 con 0x7f7624005190 2023-11-06T03:17:58.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f76377fe640 1 -- 172.21.15.157:0/2459931518 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76300aac00 con 0x7f7624005190 2023-11-06T03:17:58.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 msgr2=0x7f762400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 0x7f762400b990 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f763000a830 tx=0x7f763001faa0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- 172.21.15.157:0/2459931518 shutdown_connections 2023-11-06T03:17:58.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 --2- 172.21.15.157:0/2459931518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7624005190 0x7f762400b990 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- 172.21.15.157:0/2459931518 >> 172.21.15.157:0/2459931518 conn(0x7f7638065930 msgr2=0x7f7638065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- 172.21.15.157:0/2459931518 shutdown_connections 2023-11-06T03:17:58.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- 172.21.15.157:0/2459931518 wait complete. 2023-11-06T03:17:58.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 Processor -- start 2023-11-06T03:17:58.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- start start 2023-11-06T03:17:58.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7624005190 0x7f7638148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f763814f560 0x7f763815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763814fb30 con 0x7f763814f0e0 2023-11-06T03:17:58.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763814fe10 con 0x7f763814f560 2023-11-06T03:17:58.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763e50b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76381500f0 con 0x7f7624005190 2023-11-06T03:17:58.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60524/0 (socket says 172.21.15.157:60524) 2023-11-06T03:17:58.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 -- 172.21.15.157:0/1556703468 learned_addr learned my addr 172.21.15.157:0/1556703468 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:58.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f7637fff640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7624005190 0x7f7638148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f763caa8640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f763814f560 0x7f763815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 -- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7624005190 msgr2=0x7f7638148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7624005190 0x7f7638148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 -- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f763814f560 msgr2=0x7f763815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f763814f560 0x7f763815d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 -- 172.21.15.157:0/1556703468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76381503d0 con 0x7f763814f0e0 2023-11-06T03:17:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.022+0000 7f762f7fe640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7628010890 tx=0x7f7628010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628004220 con 0x7f763814f0e0 2023-11-06T03:17:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7628004380 con 0x7f763814f0e0 2023-11-06T03:17:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76381506b0 con 0x7f763814f0e0 2023-11-06T03:17:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762800eba0 con 0x7f763814f0e0 2023-11-06T03:17:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763814d830 con 0x7f763814f0e0 2023-11-06T03:17:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638072750 con 0x7f763814f0e0 2023-11-06T03:17:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f762800ed00 con 0x7f763814f0e0 2023-11-06T03:17:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 0x7f7618067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7628083090 con 0x7f763814f0e0 2023-11-06T03:17:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f7637fff640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 0x7f7618067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.026+0000 7f7637fff640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 0x7f7618067f50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7620000f70 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.030+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7628050920 con 0x7f763814f0e0 2023-11-06T03:17:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.182+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7638071730 con 0x7f763814f0e0 2023-11-06T03:17:58.183 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f76357fa640 1 -- 172.21.15.157:0/1556703468 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18018 (secure 0 0 0) 0x7f7628053f40 con 0x7f763814f0e0 2023-11-06T03:17:58.183 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:58.183 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":177,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:51.542063+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.d","mappings":[{"from":3,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:17:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 msgr2=0x7f7618067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 0x7f7618067f50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7620000f70 tx=0x7f7620009040 comp rx=0 tx=0).stop 2023-11-06T03:17:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 msgr2=0x7f763815cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f7628010890 tx=0x7f7628010d50 comp rx=0 tx=0).stop 2023-11-06T03:17:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 shutdown_connections 2023-11-06T03:17:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7618065a90 0x7f7618067f50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f763814f560 0x7f763815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f763814f0e0 0x7f763815cb90 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 --2- 172.21.15.157:0/1556703468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7624005190 0x7f7638148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 >> 172.21.15.157:0/1556703468 conn(0x7f7638065930 msgr2=0x7f76380b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 shutdown_connections 2023-11-06T03:17:58.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.186+0000 7f763e50b640 1 -- 172.21.15.157:0/1556703468 wait complete. 2023-11-06T03:17:58.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:17:39.446794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '174', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:17:39.943896+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:17:58.202 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:17:58.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 Processor -- start 2023-11-06T03:17:58.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 -- start start 2023-11-06T03:17:58.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a081599c0 0x7f0a080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a080b2af0 0x7f0a08159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0815e440 con 0x7f0a0806d000 2023-11-06T03:17:58.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0806c240 con 0x7f0a081599c0 2023-11-06T03:17:58.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0fa31640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0806c520 con 0x7f0a080b2af0 2023-11-06T03:17:58.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a081599c0 0x7f0a080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a081599c0 0x7f0a080ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60538/0 (socket says 172.21.15.157:60538) 2023-11-06T03:17:58.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 -- 172.21.15.157:0/1462520707 learned_addr learned my addr 172.21.15.157:0/1462520707 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:58.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0dfce640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a080b2af0 0x7f0a08159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 -- 172.21.15.157:0/1462520707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a080b2af0 msgr2=0x7f0a08159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a080b2af0 0x7f0a08159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 -- 172.21.15.157:0/1462520707 >> v1:172.21.15.195:6789/0 conn(0x7f0a0806d000 legacy=0x7f0a080b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.326+0000 7f0a0d7cd640 1 -- 172.21.15.157:0/1462520707 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0806c800 con 0x7f0a081599c0 2023-11-06T03:17:58.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0d7cd640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a081599c0 0x7f0a080ae000 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f0a08066800 tx=0x7f0a0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0d3f95f9618421 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f09fffff640 1 -- 172.21.15.157:0/1462520707 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040ab670 con 0x7f0a081599c0 2023-11-06T03:17:58.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f09fffff640 1 -- 172.21.15.157:0/1462520707 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040ab7d0 con 0x7f0a081599c0 2023-11-06T03:17:58.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f09fffff640 1 -- 172.21.15.157:0/1462520707 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040abb60 con 0x7f0a081599c0 2023-11-06T03:17:58.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- 172.21.15.157:0/1462520707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a081599c0 msgr2=0x7f0a080ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a081599c0 0x7f0a080ae000 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f0a08066800 tx=0x7f0a0401c490 comp rx=0 tx=0).stop 2023-11-06T03:17:58.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- 172.21.15.157:0/1462520707 shutdown_connections 2023-11-06T03:17:58.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a080b2af0 0x7f0a08159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/1462520707 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a081599c0 0x7f0a080ae000 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- 172.21.15.157:0/1462520707 >> 172.21.15.157:0/1462520707 conn(0x7f0a08065930 msgr2=0x7f0a08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- 172.21.15.157:0/1462520707 shutdown_connections 2023-11-06T03:17:58.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- 172.21.15.157:0/1462520707 wait complete. 2023-11-06T03:17:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 Processor -- start 2023-11-06T03:17:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.330+0000 7f0a0fa31640 1 -- start start 2023-11-06T03:17:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0806d000 0x7f0a080af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a081599c0 0x7f0a080ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080ab970 con 0x7f0a080b2af0 2023-11-06T03:17:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08151e90 con 0x7f0a081599c0 2023-11-06T03:17:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080abba0 con 0x7f0a0806d000 2023-11-06T03:17:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60548/0 (socket says 172.21.15.157:60548) 2023-11-06T03:17:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 -- 172.21.15.157:0/2000547333 learned_addr learned my addr 172.21.15.157:0/2000547333 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 -- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0806d000 msgr2=0x7f0a080af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0806d000 0x7f0a080af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 -- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a081599c0 msgr2=0x7f0a080ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a081599c0 0x7f0a080ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 -- 172.21.15.157:0/2000547333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040094d0 con 0x7f0a080b2af0 2023-11-06T03:17:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0cfcc640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c450 tx=0x7f09f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8012830 con 0x7f0a080b2af0 2023-11-06T03:17:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80129b0 con 0x7f0a080b2af0 2023-11-06T03:17:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a080abe40 con 0x7f0a080b2af0 2023-11-06T03:17:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8012d60 con 0x7f0a080b2af0 2023-11-06T03:17:58.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a080ac400 con 0x7f0a080b2af0 2023-11-06T03:17:58.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a080ac620 con 0x7f0a080b2af0 2023-11-06T03:17:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09f801c460 con 0x7f0a080b2af0 2023-11-06T03:17:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 0x7f09ec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f09f8083190 con 0x7f0a080b2af0 2023-11-06T03:17:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.334+0000 7f0a0d7cd640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 0x7f09ec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.338+0000 7f0a0d7cd640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 0x7f09ec067f50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0a040af000 tx=0x7f0a04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.338+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f09f8050990 con 0x7f0a080b2af0 2023-11-06T03:17:58.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a08072750 con 0x7f0a080b2af0 2023-11-06T03:17:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f09fdffb640 1 -- 172.21.15.157:0/2000547333 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09f8053fb0 con 0x7f0a080b2af0 2023-11-06T03:17:58.492 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 msgr2=0x7f09ec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 0x7f09ec067f50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0a040af000 tx=0x7f0a04008040 comp rx=0 tx=0).stop 2023-11-06T03:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 msgr2=0x7f0a080a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c450 tx=0x7f09f800c910 comp rx=0 tx=0).stop 2023-11-06T03:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 shutdown_connections 2023-11-06T03:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f09ec065a90 0x7f09ec067f50 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0a081599c0 0x7f0a080ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a080b2af0 0x7f0a080a84a0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 --2- 172.21.15.157:0/2000547333 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0a0806d000 0x7f0a080af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 >> 172.21.15.157:0/2000547333 conn(0x7f0a08065930 msgr2=0x7f0a080ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 shutdown_connections 2023-11-06T03:17:58.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.494+0000 7f0a0fa31640 1 -- 172.21.15.157:0/2000547333 wait complete. 2023-11-06T03:17:58.510 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:17:58.634 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 Processor -- start 2023-11-06T03:17:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 -- start start 2023-11-06T03:17:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6bfc1599c0 0x7f6bfc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 0x7f6bfc0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc15e440 con 0x7f6bfc15dea0 2023-11-06T03:17:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc15adb0 con 0x7f6bfc1599c0 2023-11-06T03:17:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c03a22640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc06c240 con 0x7f6bfc06d000 2023-11-06T03:17:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6bfc1599c0 0x7f6bfc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c00fbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 0x7f6bfc0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c01fbf640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6bfc15dea0 0x7f6bfc070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52528/0 (socket says 172.21.15.157:52528) 2023-11-06T03:17:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6bfc1599c0 0x7f6bfc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.157:60680/0 (socket says 172.21.15.157:60680) 2023-11-06T03:17:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c01fbf640 1 -- 172.21.15.157:0/3940855724 learned_addr learned my addr 172.21.15.157:0/3940855724 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c00fbd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 0x7f6bfc0b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:55894/0 (socket says 172.21.15.157:55894) 2023-11-06T03:17:58.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 -- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 msgr2=0x7f6bfc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 --2- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 0x7f6bfc0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 -- 172.21.15.157:0/3940855724 >> v1:172.21.15.195:6789/0 conn(0x7f6bfc15dea0 legacy=0x7f6bfc070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.638+0000 7f6c017be640 1 -- 172.21.15.157:0/3940855724 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc06c520 con 0x7f6bfc1599c0 2023-11-06T03:17:58.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c017be640 1 --2- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc1599c0 0x7f6bfc0ae000 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0097c0 tx=0x7f6bec01e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8821879382140251 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6bf3fff640 1 -- 172.21.15.157:0/3940855724 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0ab640 con 0x7f6bfc1599c0 2023-11-06T03:17:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6bf3fff640 1 -- 172.21.15.157:0/3940855724 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0ab7a0 con 0x7f6bfc1599c0 2023-11-06T03:17:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6bf3fff640 1 -- 172.21.15.157:0/3940855724 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0abb30 con 0x7f6bfc1599c0 2023-11-06T03:17:58.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc1599c0 msgr2=0x7f6bfc0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 --2- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc1599c0 0x7f6bfc0ae000 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0097c0 tx=0x7f6bec01e660 comp rx=0 tx=0).stop 2023-11-06T03:17:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- 172.21.15.157:0/3940855724 shutdown_connections 2023-11-06T03:17:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 --2- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc06d000 0x7f6bfc0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 --2- 172.21.15.157:0/3940855724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc1599c0 0x7f6bfc0ae000 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- 172.21.15.157:0/3940855724 >> 172.21.15.157:0/3940855724 conn(0x7f6bfc065930 msgr2=0x7f6bfc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- 172.21.15.157:0/3940855724 shutdown_connections 2023-11-06T03:17:58.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- 172.21.15.157:0/3940855724 wait complete. 2023-11-06T03:17:58.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 Processor -- start 2023-11-06T03:17:58.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.642+0000 7f6c03a22640 1 -- start start 2023-11-06T03:17:58.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc1599c0 0x7f6bfc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc15dea0 0x7f6bfc0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15ef70 con 0x7f6bfc06d000 2023-11-06T03:17:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0cc9b0 con 0x7f6bfc15dea0 2023-11-06T03:17:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0cd8e0 con 0x7f6bfc1599c0 2023-11-06T03:17:58.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35816/0 (socket says 172.21.15.157:35816) 2023-11-06T03:17:58.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 -- 172.21.15.157:0/1076120634 learned_addr learned my addr 172.21.15.157:0/1076120634 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:58.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 -- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc1599c0 msgr2=0x7f6bfc0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c00fbd640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc1599c0 0x7f6bfc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c01fbf640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc15dea0 0x7f6bfc0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc1599c0 0x7f6bfc0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 -- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc15dea0 msgr2=0x7f6bfc0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc15dea0 0x7f6bfc0cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 -- 172.21.15.157:0/1076120634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec009420 con 0x7f6bfc06d000 2023-11-06T03:17:58.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c017be640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f6bec01c290 tx=0x7f6bec01cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0be680 con 0x7f6bfc06d000 2023-11-06T03:17:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0b7070 con 0x7f6bfc06d000 2023-11-06T03:17:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc15e5d0 con 0x7f6bfc06d000 2023-11-06T03:17:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0be9e0 con 0x7f6bfc06d000 2023-11-06T03:17:58.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc15eb90 con 0x7f6bfc06d000 2023-11-06T03:17:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc072750 con 0x7f6bfc06d000 2023-11-06T03:17:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6bec0a9070 con 0x7f6bfc06d000 2023-11-06T03:17:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6bec0b9070 con 0x7f6bfc06d000 2023-11-06T03:17:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.646+0000 7f6c00fbd640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.650+0000 7f6c00fbd640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 0x7f6be0067f20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0cdbc0 tx=0x7f6bf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.650+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6bec0eb5b0 con 0x7f6bfc06d000 2023-11-06T03:17:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.802+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6bfc0cb110 con 0x7f6bfc06d000 2023-11-06T03:17:58.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6bf1ffb640 1 -- 172.21.15.157:0/1076120634 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v177) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6bec0eebd0 con 0x7f6bfc06d000 2023-11-06T03:17:58.803 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:17:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 msgr2=0x7f6be0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 0x7f6be0067f20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0cdbc0 tx=0x7f6bf8008040 comp rx=0 tx=0).stop 2023-11-06T03:17:58.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 msgr2=0x7f6bfc161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f6bec01c290 tx=0x7f6bec01cea0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 shutdown_connections 2023-11-06T03:17:58.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6be0065a60 0x7f6be0067f20 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6bfc15dea0 0x7f6bfc0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6bfc1599c0 0x7f6bfc0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 --2- 172.21.15.157:0/1076120634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bfc06d000 0x7f6bfc161670 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 >> 172.21.15.157:0/1076120634 conn(0x7f6bfc065930 msgr2=0x7f6bfc067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 shutdown_connections 2023-11-06T03:17:58.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.806+0000 7f6c03a22640 1 -- 172.21.15.157:0/1076120634 wait complete. 2023-11-06T03:17:58.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:17:58.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.934+0000 7f9136b34640 1 Processor -- start 2023-11-06T03:17:58.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.934+0000 7f9136b34640 1 -- start start 2023-11-06T03:17:58.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f9136b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91300ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f9136b34640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913015e440 con 0x7f91300b2af0 2023-11-06T03:17:58.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f9136b34640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913006c240 con 0x7f91301599c0 2023-11-06T03:17:58.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f9136b34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913006c520 con 0x7f913006d000 2023-11-06T03:17:58.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912ffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f91300b2af0 0x7f9130159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56382/0 (socket says 172.21.15.157:56382) 2023-11-06T03:17:58.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912ffff640 1 -- 172.21.15.157:0/572064925 learned_addr learned my addr 172.21.15.157:0/572064925 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91300ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859083741 0 0) 0x7f913015e440 con 0x7f91300b2af0 2023-11-06T03:17:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9118003530 con 0x7f91300b2af0 2023-11-06T03:17:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1852221280 0 0) 0x7f9118003530 con 0x7f91300b2af0 2023-11-06T03:17:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 msgr2=0x7f91300ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91300ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 >> v1:172.21.15.195:6789/0 conn(0x7f91301599c0 legacy=0x7f91300b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913006c800 con 0x7f91300b2af0 2023-11-06T03:17:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9124001680 con 0x7f91300b2af0 2023-11-06T03:17:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 >> v1:172.21.15.157:6789/0 conn(0x7f91300b2af0 legacy=0x7f9130159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9118003fe0 0x7f91180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9118004d10 0x7f9118005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913006c520 con 0x7f9118004d10 2023-11-06T03:17:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911800bb50 con 0x7f9118003fe0 2023-11-06T03:17:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911800bd10 con 0x7f913006d000 2023-11-06T03:17:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9118004d10 0x7f9118005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912ffff640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9118003fe0 0x7f91180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91348d0640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 -- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 msgr2=0x7f91180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f913006d000 0x7f91180039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 -- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9118003fe0 msgr2=0x7f91180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9118003fe0 0x7f91180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f91350d1640 1 -- 172.21.15.157:0/572064925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118003530 con 0x7f9118004d10 2023-11-06T03:17:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.938+0000 7f912ffff640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9118003fe0 0x7f91180047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:58.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f91350d1640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9118004d10 0x7f9118005190 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f9128007330 tx=0x7f912801ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2884e800b923fe81 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280a5d00 con 0x7f9118004d10 2023-11-06T03:17:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91280a5e60 con 0x7f9118004d10 2023-11-06T03:17:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912f7fe640 1 -- 172.21.15.157:0/572064925 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280ad660 con 0x7f9118004d10 2023-11-06T03:17:58.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/572064925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9118004d10 msgr2=0x7f9118005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 --2- 172.21.15.157:0/572064925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9118004d10 0x7f9118005190 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f9128007330 tx=0x7f912801ce00 comp rx=0 tx=0).stop 2023-11-06T03:17:58.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f91350d1640 1 -- 172.21.15.157:0/572064925 reap_dead start 2023-11-06T03:17:58.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/572064925 shutdown_connections 2023-11-06T03:17:58.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/572064925 >> 172.21.15.157:0/572064925 conn(0x7f9130065930 msgr2=0x7f9130065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:58.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/572064925 shutdown_connections 2023-11-06T03:17:58.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/572064925 wait complete. 2023-11-06T03:17:58.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 Processor -- start 2023-11-06T03:17:58.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- start start 2023-11-06T03:17:58.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91300acb10 0x7f91300c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 0x7f91301472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130147820 con 0x7f91300c3dc0 2023-11-06T03:17:58.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130147a60 con 0x7f91300acef0 2023-11-06T03:17:58.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91301f3200 con 0x7f91300acb10 2023-11-06T03:17:58.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35826/0 (socket says 172.21.15.157:35826) 2023-11-06T03:17:58.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 -- 172.21.15.157:0/1108201113 learned_addr learned my addr 172.21.15.157:0/1108201113 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:58.955 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f91348d0640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91300acb10 0x7f91300c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f91350d1640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 0x7f91301472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 -- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91300acb10 msgr2=0x7f91300c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91300acb10 0x7f91300c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 -- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 msgr2=0x7f91301472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:58.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 0x7f91301472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:58.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 -- 172.21.15.157:0/1108201113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300c42b0 con 0x7f91300c3dc0 2023-11-06T03:17:58.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f91350d1640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 0x7f91301472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:58.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912ffff640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f912400db40 tx=0x7f9124007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240074e0 con 0x7f91300c3dc0 2023-11-06T03:17:58.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9124007640 con 0x7f91300c3dc0 2023-11-06T03:17:58.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9130147cb0 con 0x7f91300c3dc0 2023-11-06T03:17:58.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240049f0 con 0x7f91300c3dc0 2023-11-06T03:17:58.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.942+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9130148270 con 0x7f91300c3dc0 2023-11-06T03:17:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9130072750 con 0x7f91300c3dc0 2023-11-06T03:17:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9124004b50 con 0x7f91300c3dc0 2023-11-06T03:17:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f912d7fa640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 0x7f911c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9124082dc0 con 0x7f91300c3dc0 2023-11-06T03:17:58.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f91348d0640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 0x7f911c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:58.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.946+0000 7f91348d0640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 0x7f911c067f20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f91200016e0 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:58.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:58.950+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f91240505c0 con 0x7f91300c3dc0 2023-11-06T03:17:59.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.090+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91300adcd0 con 0x7f91300c3dc0 2023-11-06T03:17:59.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.090+0000 7f912d7fa640 1 -- 172.21.15.157:0/1108201113 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v177) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9124019020 con 0x7f91300c3dc0 2023-11-06T03:17:59.089 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:17:59.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 msgr2=0x7f911c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 0x7f911c067f20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f91200016e0 tx=0x7f9120009040 comp rx=0 tx=0).stop 2023-11-06T03:17:59.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 msgr2=0x7f9130146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f912400db40 tx=0x7f9124007e00 comp rx=0 tx=0).stop 2023-11-06T03:17:59.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 shutdown_connections 2023-11-06T03:17:59.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f911c065a60 0x7f911c067f20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91300acef0 0x7f91301472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91300c3dc0 0x7f9130146da0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 --2- 172.21.15.157:0/1108201113 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f91300acb10 0x7f91300c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 >> 172.21.15.157:0/1108201113 conn(0x7f9130065930 msgr2=0x7f91300b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 shutdown_connections 2023-11-06T03:17:59.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.094+0000 7f9136b34640 1 -- 172.21.15.157:0/1108201113 wait complete. 2023-11-06T03:17:59.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:17:59.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.230+0000 7f6c583a5640 1 Processor -- start 2023-11-06T03:17:59.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.230+0000 7f6c583a5640 1 -- start start 2023-11-06T03:17:59.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 0x7f6c5006f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adae0 0x7f6c500b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c500adf40 0x7f6c500b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5006b950 con 0x7f6c500adf40 2023-11-06T03:17:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5006c490 con 0x7f6c50156a70 2023-11-06T03:17:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c583a5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5006bb60 con 0x7f6c500adae0 2023-11-06T03:17:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c500adf40 0x7f6c500b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6c500adf40 0x7f6c500b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35832/0 (socket says 172.21.15.157:35832) 2023-11-06T03:17:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 -- 172.21.15.157:0/3359275510 learned_addr learned my addr 172.21.15.157:0/3359275510 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56141640 1 --2- 172.21.15.157:0/3359275510 >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 0x7f6c5006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c55940640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adae0 0x7f6c500b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 -- 172.21.15.157:0/3359275510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adae0 msgr2=0x7f6c500b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adae0 0x7f6c500b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 -- 172.21.15.157:0/3359275510 >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 msgr2=0x7f6c5006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 --2- 172.21.15.157:0/3359275510 >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 0x7f6c5006f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 -- 172.21.15.157:0/3359275510 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5006be40 con 0x7f6c500adf40 2023-11-06T03:17:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56141640 1 --2- 172.21.15.157:0/3359275510 >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 0x7f6c5006f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:17:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c56942640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adf40 0x7f6c500b2160 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ce880 tx=0x7f6c4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d34eee2c78cff4b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c5513f640 1 -- 172.21.15.157:0/3359275510 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480a5840 con 0x7f6c500adf40 2023-11-06T03:17:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c5513f640 1 -- 172.21.15.157:0/3359275510 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c480a59a0 con 0x7f6c500adf40 2023-11-06T03:17:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.234+0000 7f6c5513f640 1 -- 172.21.15.157:0/3359275510 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480a5d30 con 0x7f6c500adf40 2023-11-06T03:17:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3359275510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adf40 msgr2=0x7f6c500b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adf40 0x7f6c500b2160 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ce880 tx=0x7f6c4801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3359275510 shutdown_connections 2023-11-06T03:17:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adf40 0x7f6c500b2160 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3359275510 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adae0 0x7f6c500b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3359275510 >> v2:172.21.15.195:3300/0 conn(0x7f6c50156a70 0x7f6c5006f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3359275510 >> 172.21.15.157:0/3359275510 conn(0x7f6c50065930 msgr2=0x7f6c50065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3359275510 shutdown_connections 2023-11-06T03:17:59.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3359275510 wait complete. 2023-11-06T03:17:59.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 Processor -- start 2023-11-06T03:17:59.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- start start 2023-11-06T03:17:59.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adf40 0x7f6c50160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 0x7f6c500b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5014b8d0 con 0x7f6c500adae0 2023-11-06T03:17:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50162080 con 0x7f6c50156a70 2023-11-06T03:17:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50161470 con 0x7f6c500adf40 2023-11-06T03:17:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35844/0 (socket says 172.21.15.157:35844) 2023-11-06T03:17:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 -- 172.21.15.157:0/3883978864 learned_addr learned my addr 172.21.15.157:0/3883978864 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56942640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 0x7f6c500b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c55940640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adf40 0x7f6c50160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 -- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adf40 msgr2=0x7f6c50160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adf40 0x7f6c50160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 -- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 msgr2=0x7f6c500b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 0x7f6c500b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 -- 172.21.15.157:0/3883978864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009b30 con 0x7f6c500adae0 2023-11-06T03:17:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56942640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 0x7f6c500b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c56141640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e840 tx=0x7f6c4c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00c8d0 con 0x7f6c500adae0 2023-11-06T03:17:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00ca30 con 0x7f6c500adae0 2023-11-06T03:17:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50161750 con 0x7f6c500adae0 2023-11-06T03:17:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00cde0 con 0x7f6c500adae0 2023-11-06T03:17:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c50161d10 con 0x7f6c500adae0 2023-11-06T03:17:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.238+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c50072750 con 0x7f6c500adae0 2023-11-06T03:17:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.242+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c4c013410 con 0x7f6c500adae0 2023-11-06T03:17:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.242+0000 7f6c46ffd640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 0x7f6c34067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.242+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f6c4c0830f0 con 0x7f6c500adae0 2023-11-06T03:17:59.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.242+0000 7f6c55940640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 0x7f6c34067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.242+0000 7f6c55940640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 0x7f6c34067f20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6c501589e0 tx=0x7f6c40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.246+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6c4c050980 con 0x7f6c500adae0 2023-11-06T03:17:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.398+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c50065f60 con 0x7f6c500adae0 2023-11-06T03:17:59.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.398+0000 7f6c46ffd640 1 -- 172.21.15.157:0/3883978864 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v177) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c4c053fa0 con 0x7f6c500adae0 2023-11-06T03:17:59.398 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:17:59.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 msgr2=0x7f6c34067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 0x7f6c34067f20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6c501589e0 tx=0x7f6c40008040 comp rx=0 tx=0).stop 2023-11-06T03:17:59.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 msgr2=0x7f6c5014b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e840 tx=0x7f6c4c00ed00 comp rx=0 tx=0).stop 2023-11-06T03:17:59.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 shutdown_connections 2023-11-06T03:17:59.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6c34065a60 0x7f6c34067f20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c50156a70 0x7f6c500b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6c500adf40 0x7f6c50160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 --2- 172.21.15.157:0/3883978864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c500adae0 0x7f6c5014b390 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 >> 172.21.15.157:0/3883978864 conn(0x7f6c50065930 msgr2=0x7f6c5015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 shutdown_connections 2023-11-06T03:17:59.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.402+0000 7f6c583a5640 1 -- 172.21.15.157:0/3883978864 wait complete. 2023-11-06T03:17:59.416 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:17:59.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 Processor -- start 2023-11-06T03:17:59.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 -- start start 2023-11-06T03:17:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 0x7fbae406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbae40701b0 0x7fbae40c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40733a0 con 0x7fbae4070a40 2023-11-06T03:17:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40cc8d0 con 0x7fbae40701b0 2023-11-06T03:17:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbaea167640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40730c0 con 0x7fbae4159ce0 2023-11-06T03:17:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbae40701b0 0x7fbae40c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fbae4070a40 0x7fbae406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56390/0 (socket says 172.21.15.157:56390) 2023-11-06T03:17:59.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae3fff640 1 -- 172.21.15.157:0/1122389532 learned_addr learned my addr 172.21.15.157:0/1122389532 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:59.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae37fe640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 0x7fbae406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae2ffd640 1 -- 172.21.15.157:0/1122389532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 msgr2=0x7fbae406fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae2ffd640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 0x7fbae406fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.538 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae2ffd640 1 -- 172.21.15.157:0/1122389532 >> v1:172.21.15.157:6789/0 conn(0x7fbae4070a40 legacy=0x7fbae406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae27fc640 1 -- 172.21.15.157:0/1122389532 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057169557 0 0) 0x7fbae40733a0 con 0x7fbae4070a40 2023-11-06T03:17:59.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae2ffd640 1 -- 172.21.15.157:0/1122389532 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40cc5f0 con 0x7fbae40701b0 2023-11-06T03:17:59.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.538+0000 7fbae37fe640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 0x7fbae406fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:59.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae40701b0 0x7fbae40c9600 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0097c0 tx=0x7fbacc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0c134c67608ac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae27fc640 1 -- 172.21.15.157:0/1122389532 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0ab600 con 0x7fbae40701b0 2023-11-06T03:17:59.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae27fc640 1 -- 172.21.15.157:0/1122389532 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc0a7050 con 0x7fbae40701b0 2023-11-06T03:17:59.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae27fc640 1 -- 172.21.15.157:0/1122389532 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0ab980 con 0x7fbae40701b0 2023-11-06T03:17:59.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/1122389532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae40701b0 msgr2=0x7fbae40c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae40701b0 0x7fbae40c9600 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0097c0 tx=0x7fbacc01c360 comp rx=0 tx=0).stop 2023-11-06T03:17:59.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/1122389532 shutdown_connections 2023-11-06T03:17:59.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae40701b0 0x7fbae40c9600 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- 172.21.15.157:0/1122389532 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae4159ce0 0x7fbae406fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/1122389532 >> 172.21.15.157:0/1122389532 conn(0x7fbae40cba30 msgr2=0x7fbae40cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/1122389532 shutdown_connections 2023-11-06T03:17:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/1122389532 wait complete. 2023-11-06T03:17:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 Processor -- start 2023-11-06T03:17:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- start start 2023-11-06T03:17:59.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae40701b0 0x7fbae415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbae4159ce0 0x7fbae4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40ce920 con 0x7fbae4070a40 2023-11-06T03:17:59.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae415d000 con 0x7fbae4159ce0 2023-11-06T03:17:59.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4151fe0 con 0x7fbae40701b0 2023-11-06T03:17:59.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35868/0 (socket says 172.21.15.157:35868) 2023-11-06T03:17:59.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 -- 172.21.15.157:0/808995848 learned_addr learned my addr 172.21.15.157:0/808995848 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:59.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 -- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae40701b0 msgr2=0x7fbae415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae37fe640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae40701b0 0x7fbae415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae3fff640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbae4159ce0 0x7fbae4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae40701b0 0x7fbae415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 -- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbae4159ce0 msgr2=0x7fbae4154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbae4159ce0 0x7fbae4154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 -- 172.21.15.157:0/808995848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009420 con 0x7fbae4070a40 2023-11-06T03:17:59.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbae2ffd640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0ae040 tx=0x7fbacc019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0b4630 con 0x7fbae4070a40 2023-11-06T03:17:59.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc0b47b0 con 0x7fbae4070a40 2023-11-06T03:17:59.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40cdca0 con 0x7fbae4070a40 2023-11-06T03:17:59.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.542+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0b4b60 con 0x7fbae4070a40 2023-11-06T03:17:59.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae40ce260 con 0x7fbae4070a40 2023-11-06T03:17:59.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae40cea60 con 0x7fbae4070a40 2023-11-06T03:17:59.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbacc0b2040 con 0x7fbae4070a40 2023-11-06T03:17:59.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbadbfff640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 0x7fbac4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fbacc11d130 con 0x7fbae4070a40 2023-11-06T03:17:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbae37fe640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 0x7fbac4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.546+0000 7fbae37fe640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 0x7fbac4067f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbadc006fd0 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.550+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbae40cea60 con 0x7fbae4070a40 2023-11-06T03:17:59.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.726+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbae40cea60 con 0x7fbae4070a40 2023-11-06T03:17:59.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbadbfff640 1 -- 172.21.15.157:0/808995848 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7fbae40cea60 con 0x7fbae4070a40 2023-11-06T03:17:59.727 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:17:59.727 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:17:59.729 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:17:59.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 msgr2=0x7fbac4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 0x7fbac4067f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbadc006fd0 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2023-11-06T03:17:59.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 msgr2=0x7fbae40cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0ae040 tx=0x7fbacc019100 comp rx=0 tx=0).stop 2023-11-06T03:17:59.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 shutdown_connections 2023-11-06T03:17:59.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbac4065a60 0x7fbac4067f20 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbae4159ce0 0x7fbae4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbae4070a40 0x7fbae40cd250 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 --2- 172.21.15.157:0/808995848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbae40701b0 0x7fbae415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 >> 172.21.15.157:0/808995848 conn(0x7fbae40cba30 msgr2=0x7fbae40ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 shutdown_connections 2023-11-06T03:17:59.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.730+0000 7fbaea167640 1 -- 172.21.15.157:0/808995848 wait complete. 2023-11-06T03:17:59.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:17:59.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.862+0000 7f7eaa1f9640 1 Processor -- start 2023-11-06T03:17:59.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 -- start start 2023-11-06T03:17:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7ea415caa0 0x7f7ea40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 0x7f7ea40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40d2920 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40d18a0 con 0x7f7ea415caa0 2023-11-06T03:17:59.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7eaa1f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40d1b60 con 0x7f7ea415ce80 2023-11-06T03:17:59.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7ea40ce1a0 0x7f7ea40d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56406/0 (socket says 172.21.15.157:56406) 2023-11-06T03:17:59.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea3fff640 1 -- 172.21.15.157:0/3251020984 learned_addr learned my addr 172.21.15.157:0/3251020984 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:17:59.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7e9affd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 0x7f7ea40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/3251020984 >> v2:172.21.15.195:3300/0 conn(0x7f7ea415caa0 0x7f7ea40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432629930 0 0) 0x7f7ea40d2920 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e84003530 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1599239558 0 0) 0x7f7e84003530 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 msgr2=0x7f7ea40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 0x7f7ea40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 >> v2:172.21.15.195:3300/0 conn(0x7f7ea415caa0 msgr2=0x7f7ea40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 --2- 172.21.15.157:0/3251020984 >> v2:172.21.15.195:3300/0 conn(0x7f7ea415caa0 0x7f7ea40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40d1e40 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7e9c001a00 con 0x7f7ea40ce1a0 2023-11-06T03:17:59.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 >> v1:172.21.15.157:6789/0 conn(0x7f7ea40ce1a0 legacy=0x7f7ea40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e84003980 0x7f7e84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e840047d0 0x7f7e84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 0x7f7e84005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40d18a0 con 0x7f7e84003980 2023-11-06T03:17:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40d1b60 con 0x7f7e840047d0 2023-11-06T03:17:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8400b990 con 0x7f7ea415ce80 2023-11-06T03:17:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e84003980 0x7f7e84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7e9affd640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e840047d0 0x7f7e84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 -- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 msgr2=0x7f7e84005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7ea415ce80 0x7f7e84005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 -- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e840047d0 msgr2=0x7f7e84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:17:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e840047d0 0x7f7e84004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.866+0000 7f7ea37fe640 1 -- 172.21.15.157:0/3251020984 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84003530 con 0x7f7e84003980 2023-11-06T03:17:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e84003980 0x7f7e84003da0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ae0 tx=0x7f7e8c01ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56e6076cd473ca2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0a6460 con 0x7f7e84003980 2023-11-06T03:17:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c0a65c0 con 0x7f7e84003980 2023-11-06T03:17:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea2ffd640 1 -- 172.21.15.157:0/3251020984 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0a6970 con 0x7f7e84003980 2023-11-06T03:17:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/3251020984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e84003980 msgr2=0x7f7e84003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/3251020984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e84003980 0x7f7e84003da0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ae0 tx=0x7f7e8c01ca60 comp rx=0 tx=0).stop 2023-11-06T03:17:59.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 -- 172.21.15.157:0/3251020984 reap_dead start 2023-11-06T03:17:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/3251020984 shutdown_connections 2023-11-06T03:17:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/3251020984 >> 172.21.15.157:0/3251020984 conn(0x7f7ea4070790 msgr2=0x7f7ea4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:17:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/3251020984 shutdown_connections 2023-11-06T03:17:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/3251020984 wait complete. 2023-11-06T03:17:59.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 Processor -- start 2023-11-06T03:17:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- start start 2023-11-06T03:17:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 0x7f7ea40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 0x7f7ea415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cbfd0 con 0x7f7ea40b7090 2023-11-06T03:17:59.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40b7a50 con 0x7f7ea41587a0 2023-11-06T03:17:59.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7eaa1f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41f51c0 con 0x7f7e84003980 2023-11-06T03:17:59.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35876/0 (socket says 172.21.15.157:35876) 2023-11-06T03:17:59.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 -- 172.21.15.157:0/2615782340 learned_addr learned my addr 172.21.15.157:0/2615782340 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:17:59.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 0x7f7ea40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7e9affd640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 0x7f7ea415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 -- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 msgr2=0x7f7ea40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 0x7f7ea40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 -- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 msgr2=0x7f7ea415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:17:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 0x7f7ea415d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:17:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 -- 172.21.15.157:0/2615782340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40cb450 con 0x7f7ea40b7090 2023-11-06T03:17:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 0x7f7ea40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:59.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7e9affd640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 0x7f7ea415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:17:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea3fff640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40b5760 tx=0x7f7e9c00ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.870+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00a5b0 con 0x7f7ea40b7090 2023-11-06T03:17:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00a730 con 0x7f7ea40b7090 2023-11-06T03:17:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40cb730 con 0x7f7ea40b7090 2023-11-06T03:17:59.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00aae0 con 0x7f7ea40b7090 2023-11-06T03:17:59.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea40cbcf0 con 0x7f7ea40b7090 2023-11-06T03:17:59.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea41f5390 con 0x7f7ea40b7090 2023-11-06T03:17:59.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7e9c00ac40 con 0x7f7ea40b7090 2023-11-06T03:17:59.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea0ff9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 0x7f7e88067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:17:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f7e9c082f10 con 0x7f7ea40b7090 2023-11-06T03:17:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 0x7f7e88067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:17:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.874+0000 7f7ea37fe640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 0x7f7e88067f50 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ad000 tx=0x7f7e8c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:17:59.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:17:59.878+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7ea41f5390 con 0x7f7ea40b7090 2023-11-06T03:18:00.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.078+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ea4157a20 con 0x7f7ea40b7090 2023-11-06T03:18:00.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.078+0000 7f7ea0ff9640 1 -- 172.21.15.157:0/2615782340 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18018 (secure 0 0 0) 0x7f7ea4157a20 con 0x7f7ea40b7090 2023-11-06T03:18:00.077 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:00.077 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":177,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:51.542063+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.d","mappings":[{"from":3,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 msgr2=0x7f7e88067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 0x7f7e88067f50 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ad000 tx=0x7f7e8c0046d0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 msgr2=0x7f7ea40b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40b5760 tx=0x7f7e9c00ce80 comp rx=0 tx=0).stop 2023-11-06T03:18:00.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 shutdown_connections 2023-11-06T03:18:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7e88065a90 0x7f7e88067f50 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ea40b7090 0x7f7ea40b7510 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ea41587a0 0x7f7ea415d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 --2- 172.21.15.157:0/2615782340 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7e84003980 0x7f7ea40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 >> 172.21.15.157:0/2615782340 conn(0x7f7ea4070790 msgr2=0x7f7ea41592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 shutdown_connections 2023-11-06T03:18:00.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.082+0000 7f7eaa1f9640 1 -- 172.21.15.157:0/2615782340 wait complete. 2023-11-06T03:18:00.096 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:18:00.096 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:00.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.222+0000 7fabf8cba640 1 Processor -- start 2023-11-06T03:18:00.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.222+0000 7fabf8cba640 1 -- start start 2023-11-06T03:18:00.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fabf415ce80 0x7fabf40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 0x7fabf40d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40d2920 con 0x7fabf415ce80 2023-11-06T03:18:00.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40d18a0 con 0x7fabf40ce1a0 2023-11-06T03:18:00.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf8cba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40d1b60 con 0x7fabf415caa0 2023-11-06T03:18:00.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fabf415ce80 0x7fabf40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fabf415ce80 0x7fabf40cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35880/0 (socket says 172.21.15.157:35880) 2023-11-06T03:18:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 -- 172.21.15.157:0/1916259654 learned_addr learned my addr 172.21.15.157:0/1916259654 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf2d9d640 1 --2- 172.21.15.157:0/1916259654 >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 0x7fabf40d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf259c640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 -- 172.21.15.157:0/1916259654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 msgr2=0x7fabf40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 -- 172.21.15.157:0/1916259654 >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 msgr2=0x7fabf40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 --2- 172.21.15.157:0/1916259654 >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 0x7fabf40d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf1d9b640 1 -- 172.21.15.157:0/1916259654 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40d1e40 con 0x7fabf415ce80 2023-11-06T03:18:00.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf2d9d640 1 --2- 172.21.15.157:0/1916259654 >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 0x7fabf40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:00.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.226+0000 7fabf259c640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf1d9b640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf40cfb70 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fabf40d0780 tx=0x7fabe801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d40683789f752b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf159a640 1 -- 172.21.15.157:0/1916259654 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80aa5f0 con 0x7fabf415ce80 2023-11-06T03:18:00.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf159a640 1 -- 172.21.15.157:0/1916259654 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe80aa750 con 0x7fabf415ce80 2023-11-06T03:18:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf159a640 1 -- 172.21.15.157:0/1916259654 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80aaae0 con 0x7fabf415ce80 2023-11-06T03:18:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- 172.21.15.157:0/1916259654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 msgr2=0x7fabf40cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf40cfb70 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fabf40d0780 tx=0x7fabe801c4d0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- 172.21.15.157:0/1916259654 shutdown_connections 2023-11-06T03:18:00.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 --2- 172.21.15.157:0/1916259654 >> v2:172.21.15.195:3300/0 conn(0x7fabf40ce1a0 0x7fabf40d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf40cfb70 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 --2- 172.21.15.157:0/1916259654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- 172.21.15.157:0/1916259654 >> 172.21.15.157:0/1916259654 conn(0x7fabf4070790 msgr2=0x7fabf4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- 172.21.15.157:0/1916259654 shutdown_connections 2023-11-06T03:18:00.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- 172.21.15.157:0/1916259654 wait complete. 2023-11-06T03:18:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 Processor -- start 2023-11-06T03:18:00.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 -- start start 2023-11-06T03:18:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.230+0000 7fabf8cba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 0x7fabf40ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40c6750 con 0x7fabf415ce80 2023-11-06T03:18:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40c72d0 con 0x7fabf40ce1a0 2023-11-06T03:18:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40c69a0 con 0x7fabf415caa0 2023-11-06T03:18:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35886/0 (socket says 172.21.15.157:35886) 2023-11-06T03:18:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 -- 172.21.15.157:0/2009900950 learned_addr learned my addr 172.21.15.157:0/2009900950 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf259c640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 0x7fabf40ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf1d9b640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 -- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 msgr2=0x7fabf40a9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40a9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 -- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 msgr2=0x7fabf40ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 0x7fabf40ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 -- 172.21.15.157:0/2009900950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80094d0 con 0x7fabf415ce80 2023-11-06T03:18:00.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf1d9b640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:00.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf259c640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 0x7fabf40ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:00.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf2d9d640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fabec007db0 tx=0x7fabec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0039a0 con 0x7fabf415ce80 2023-11-06T03:18:00.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec00f040 con 0x7fabf415ce80 2023-11-06T03:18:00.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf40c6c80 con 0x7fabf415ce80 2023-11-06T03:18:00.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec003d60 con 0x7fabf415ce80 2023-11-06T03:18:00.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf415a960 con 0x7fabf415ce80 2023-11-06T03:18:00.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4072750 con 0x7fabf415ce80 2023-11-06T03:18:00.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fabec004030 con 0x7fabf415ce80 2023-11-06T03:18:00.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 0x7fabd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7fabec083530 con 0x7fabf415ce80 2023-11-06T03:18:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf259c640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 0x7fabd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.234+0000 7fabf259c640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 0x7fabd4067f20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009980 tx=0x7fabe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.238+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fabec050dc0 con 0x7fabf415ce80 2023-11-06T03:18:00.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabf415a520 con 0x7fabf415ce80 2023-11-06T03:18:00.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabe2ffd640 1 -- 172.21.15.157:0/2009900950 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7fabec0543e0 con 0x7fabf415ce80 2023-11-06T03:18:00.412 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:00.412 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:18:00.412 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:18:00.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 msgr2=0x7fabd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 0x7fabd4067f20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009980 tx=0x7fabe4008040 comp rx=0 tx=0).stop 2023-11-06T03:18:00.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 msgr2=0x7fabf415b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fabec007db0 tx=0x7fabec00d4a0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 shutdown_connections 2023-11-06T03:18:00.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fabd4065a60 0x7fabd4067f20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabf415ce80 0x7fabf415b660 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fabf415caa0 0x7fabf40a9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 --2- 172.21.15.157:0/2009900950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fabf40ce1a0 0x7fabf40ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 >> 172.21.15.157:0/2009900950 conn(0x7fabf4070790 msgr2=0x7fabf415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 shutdown_connections 2023-11-06T03:18:00.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.414+0000 7fabf8cba640 1 -- 172.21.15.157:0/2009900950 wait complete. 2023-11-06T03:18:00.430 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:00.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 Processor -- start 2023-11-06T03:18:00.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 -- start start 2023-11-06T03:18:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1fa40cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1fa406ac30 0x7f1fa40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa406da10 con 0x7f1fa415ce30 2023-11-06T03:18:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa406dcf0 con 0x7f1fa406ac30 2023-11-06T03:18:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fabc8c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa406dfd0 con 0x7f1fa40d0ee0 2023-11-06T03:18:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9a28640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1fa415ce30 0x7f1fa40d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56408/0 (socket says 172.21.15.157:56408) 2023-11-06T03:18:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9a28640 1 -- 172.21.15.157:0/474230651 learned_addr learned my addr 172.21.15.157:0/474230651 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9227640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1fa40cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186612511 0 0) 0x7f1fa406da10 con 0x7f1fa415ce30 2023-11-06T03:18:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 --2- 172.21.15.157:0/474230651 >> v2:172.21.15.195:3300/0 conn(0x7f1fa406ac30 0x7f1fa40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f90003530 con 0x7f1fa415ce30 2023-11-06T03:18:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315989078 0 0) 0x7f1f90003530 con 0x7f1fa415ce30 2023-11-06T03:18:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 msgr2=0x7f1fa40cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1fa40cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 >> v2:172.21.15.195:3300/0 conn(0x7f1fa406ac30 msgr2=0x7f1fa40cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 --2- 172.21.15.157:0/474230651 >> v2:172.21.15.195:3300/0 conn(0x7f1fa406ac30 0x7f1fa40cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa406e2b0 con 0x7f1fa415ce30 2023-11-06T03:18:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1fa0001800 con 0x7f1fa415ce30 2023-11-06T03:18:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 >> v1:172.21.15.157:6789/0 conn(0x7f1fa415ce30 legacy=0x7f1fa40d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f90003980 0x7f1f90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1f900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f90004d10 0x7f1f90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa406dcf0 con 0x7f1f90004d10 2023-11-06T03:18:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa406dfd0 con 0x7f1f90003980 2023-11-06T03:18:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9000b990 con 0x7f1fa40d0ee0 2023-11-06T03:18:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f90004d10 0x7f1f90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9a28640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f90003980 0x7f1f90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9227640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1f900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 -- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 msgr2=0x7f1f900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1f900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 -- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f90003980 msgr2=0x7f1f90003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f90003980 0x7f1f90003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1faa229640 1 -- 172.21.15.157:0/474230651 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90003530 con 0x7f1f90004d10 2023-11-06T03:18:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.554+0000 7f1fa9227640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa40d0ee0 0x7f1f900047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:00.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1faa229640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f90004d10 0x7f1f90005190 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e170 tx=0x7f1f9c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c94804be094aaff server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0a5c90 con 0x7f1f90004d10 2023-11-06T03:18:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c00f040 con 0x7f1f90004d10 2023-11-06T03:18:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fa8a26640 1 -- 172.21.15.157:0/474230651 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0ad440 con 0x7f1f90004d10 2023-11-06T03:18:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- 172.21.15.157:0/474230651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f90004d10 msgr2=0x7f1f90005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/474230651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f90004d10 0x7f1f90005190 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e170 tx=0x7f1f9c01cc80 comp rx=0 tx=0).stop 2023-11-06T03:18:00.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1faa229640 1 -- 172.21.15.157:0/474230651 reap_dead start 2023-11-06T03:18:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- 172.21.15.157:0/474230651 shutdown_connections 2023-11-06T03:18:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- 172.21.15.157:0/474230651 >> 172.21.15.157:0/474230651 conn(0x7f1fa4071b90 msgr2=0x7f1fa4099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- 172.21.15.157:0/474230651 shutdown_connections 2023-11-06T03:18:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- 172.21.15.157:0/474230651 wait complete. 2023-11-06T03:18:00.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 Processor -- start 2023-11-06T03:18:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.558+0000 7f1fabc8c640 1 -- start start 2023-11-06T03:18:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa4154e50 0x7f1fa41f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa41e5de0 0x7f1fa41e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41e67a0 con 0x7f1fa41f2390 2023-11-06T03:18:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41f5ec0 con 0x7f1fa41e5de0 2023-11-06T03:18:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41f6890 con 0x7f1fa4154e50 2023-11-06T03:18:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35908/0 (socket says 172.21.15.157:35908) 2023-11-06T03:18:00.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 -- 172.21.15.157:0/1080173008 learned_addr learned my addr 172.21.15.157:0/1080173008 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9a28640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa4154e50 0x7f1fa41f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 -- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa4154e50 msgr2=0x7f1fa41f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1faa229640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa41e5de0 0x7f1fa41e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa4154e50 0x7f1fa41f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 -- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa41e5de0 msgr2=0x7f1fa41e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa41e5de0 0x7f1fa41e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 -- 172.21.15.157:0/1080173008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa41f2e30 con 0x7f1fa41f2390 2023-11-06T03:18:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9227640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f1f940108d0 tx=0x7f1f94010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94004220 con 0x7f1fa41f2390 2023-11-06T03:18:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94004380 con 0x7f1fa41f2390 2023-11-06T03:18:00.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa41f3110 con 0x7f1fa41f2390 2023-11-06T03:18:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400eba0 con 0x7f1fa41f2390 2023-11-06T03:18:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4155230 con 0x7f1fa41f2390 2023-11-06T03:18:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa41f3350 con 0x7f1fa41f2390 2023-11-06T03:18:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f9400ed00 con 0x7f1fa41f2390 2023-11-06T03:18:00.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 0x7f1f88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1f94083290 con 0x7f1fa41f2390 2023-11-06T03:18:00.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.562+0000 7f1fa9a28640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 0x7f1f88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.566+0000 7f1fa9a28640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 0x7f1f88067f20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000fe0 tx=0x7f1fa0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.566+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f94050b20 con 0x7f1fa41f2390 2023-11-06T03:18:00.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.754+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa4072750 con 0x7f1fa41f2390 2023-11-06T03:18:00.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.754+0000 7f1f9a7fc640 1 -- 172.21.15.157:0/1080173008 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4668 (secure 0 0 0) 0x7f1f94054140 con 0x7f1fa41f2390 2023-11-06T03:18:00.754 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:00.754 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":123,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:17:40.965769+0000","modified":"2023-11-06T03:17:42.980401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10818":{"gid":10818,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6834/4092537436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4092537436},{"type":"v1","addr":"172.21.15.157:6834","nonce":4092537436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-11-06T03:18:00.754 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 123 2023-11-06T03:18:00.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 msgr2=0x7f1f88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 0x7f1f88067f20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0000fe0 tx=0x7f1fa0009040 comp rx=0 tx=0).stop 2023-11-06T03:18:00.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 msgr2=0x7f1fa41f2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f1f940108d0 tx=0x7f1f94010d90 comp rx=0 tx=0).stop 2023-11-06T03:18:00.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 shutdown_connections 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f88065a60 0x7f1f88067f20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1fa41e5de0 0x7f1fa41e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa41f2390 0x7f1fa41f2810 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 --2- 172.21.15.157:0/1080173008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1fa4154e50 0x7f1fa41f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 >> 172.21.15.157:0/1080173008 conn(0x7f1fa4071b90 msgr2=0x7f1fa40cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 shutdown_connections 2023-11-06T03:18:00.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.758+0000 7f1fabc8c640 1 -- 172.21.15.157:0/1080173008 wait complete. 2023-11-06T03:18:00.773 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:00.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.894+0000 7f1b2722b640 1 Processor -- start 2023-11-06T03:18:00.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.894+0000 7f1b2722b640 1 -- start start 2023-11-06T03:18:00.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b2722b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b201599c0 0x7f1b200ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b2722b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b20159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b2722b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2015e440 con 0x7f1b2006d000 2023-11-06T03:18:00.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b2722b640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2006c240 con 0x7f1b201599c0 2023-11-06T03:18:00.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b2722b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2006c520 con 0x7f1b200b2af0 2023-11-06T03:18:00.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b201599c0 0x7f1b200ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1b201599c0 0x7f1b200ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35910/0 (socket says 172.21.15.157:35910) 2023-11-06T03:18:00.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 -- 172.21.15.157:0/2418295785 learned_addr learned my addr 172.21.15.157:0/2418295785 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:00.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b257c8640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b20159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 -- 172.21.15.157:0/2418295785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 msgr2=0x7f1b20159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b20159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 -- 172.21.15.157:0/2418295785 >> v1:172.21.15.195:6789/0 conn(0x7f1b2006d000 legacy=0x7f1b200b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:00.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 -- 172.21.15.157:0/2418295785 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2006c800 con 0x7f1b201599c0 2023-11-06T03:18:00.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b24fc7640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b201599c0 0x7f1b200ae000 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f1b20066800 tx=0x7f1b1c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5073202e2bac1d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b177fe640 1 -- 172.21.15.157:0/2418295785 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0a7c10 con 0x7f1b201599c0 2023-11-06T03:18:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b177fe640 1 -- 172.21.15.157:0/2418295785 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0a7d70 con 0x7f1b201599c0 2023-11-06T03:18:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.898+0000 7f1b177fe640 1 -- 172.21.15.157:0/2418295785 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0af660 con 0x7f1b201599c0 2023-11-06T03:18:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/2418295785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b201599c0 msgr2=0x7f1b200ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b201599c0 0x7f1b200ae000 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f1b20066800 tx=0x7f1b1c01eb60 comp rx=0 tx=0).stop 2023-11-06T03:18:00.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/2418295785 shutdown_connections 2023-11-06T03:18:00.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b20159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- 172.21.15.157:0/2418295785 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b201599c0 0x7f1b200ae000 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/2418295785 >> 172.21.15.157:0/2418295785 conn(0x7f1b20065930 msgr2=0x7f1b20065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:00.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/2418295785 shutdown_connections 2023-11-06T03:18:00.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/2418295785 wait complete. 2023-11-06T03:18:00.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 Processor -- start 2023-11-06T03:18:00.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- start start 2023-11-06T03:18:00.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b2014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 0x7f1b2014be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2014d5e0 con 0x7f1b2006d000 2023-11-06T03:18:00.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20150f30 con 0x7f1b201599c0 2023-11-06T03:18:00.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2014d880 con 0x7f1b200b2af0 2023-11-06T03:18:00.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35912/0 (socket says 172.21.15.157:35912) 2023-11-06T03:18:00.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 -- 172.21.15.157:0/1070689284 learned_addr learned my addr 172.21.15.157:0/1070689284 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 -- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 msgr2=0x7f1b2014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b2014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b257c8640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 0x7f1b2014be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 -- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 msgr2=0x7f1b2014be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:00.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 0x7f1b2014be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:00.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 -- 172.21.15.157:0/1070689284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c00b770 con 0x7f1b2006d000 2023-11-06T03:18:00.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b257c8640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 0x7f1b2014be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:00.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b24fc7640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00b2c0 tx=0x7f1b1c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0afdf0 con 0x7f1b2006d000 2023-11-06T03:18:00.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0ba480 con 0x7f1b2006d000 2023-11-06T03:18:00.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0ba830 con 0x7f1b2006d000 2023-11-06T03:18:00.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2014db60 con 0x7f1b2006d000 2023-11-06T03:18:00.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.902+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2014e120 con 0x7f1b2006d000 2023-11-06T03:18:00.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b20072750 con 0x7f1b2006d000 2023-11-06T03:18:00.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b1c0ba990 con 0x7f1b2006d000 2023-11-06T03:18:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b157fa640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 0x7f1b0c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f1b1c11e960 con 0x7f1b2006d000 2023-11-06T03:18:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b17fff640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 0x7f1b0c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.906+0000 7f1b17fff640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 0x7f1b0c067f50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006b940 tx=0x7f1b10006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:00.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:00.910+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1b1c0ec310 con 0x7f1b2006d000 2023-11-06T03:18:01.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.062+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b20072750 con 0x7f1b2006d000 2023-11-06T03:18:01.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b157fa640 1 -- 172.21.15.157:0/1070689284 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+18018 (secure 0 0 0) 0x7f1b1c0b4020 con 0x7f1b2006d000 2023-11-06T03:18:01.063 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:01.064 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":177,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:17:51.542063+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:17:39.446794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-06T03:17:39.943896+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.d","mappings":[{"from":3,"to":0}]},{"pgid":"26.17","mappings":[{"from":1,"to":0}]},{"pgid":"26.19","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:01.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 msgr2=0x7f1b0c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 0x7f1b0c067f50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006b940 tx=0x7f1b10006bd0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 msgr2=0x7f1b201511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00b2c0 tx=0x7f1b1c01d3a0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 shutdown_connections 2023-11-06T03:18:01.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1b0c065a90 0x7f1b0c067f50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1b201599c0 0x7f1b2014be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1b200b2af0 0x7f1b2014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 --2- 172.21.15.157:0/1070689284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2006d000 0x7f1b201511c0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 >> 172.21.15.157:0/1070689284 conn(0x7f1b20065930 msgr2=0x7f1b2006ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:01.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 shutdown_connections 2023-11-06T03:18:01.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.066+0000 7f1b2722b640 1 -- 172.21.15.157:0/1070689284 wait complete. 2023-11-06T03:18:01.081 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:18:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.194+0000 7f926c179640 1 Processor -- start 2023-11-06T03:18:01.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.194+0000 7f926c179640 1 -- start start 2023-11-06T03:18:01.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926c179640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f92640ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926c179640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926415e440 con 0x7f92640b2af0 2023-11-06T03:18:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926c179640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926406c240 con 0x7f92641599c0 2023-11-06T03:18:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926c179640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926406c520 con 0x7f926406d000 2023-11-06T03:18:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f92640b2af0 0x7f9264159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56420/0 (socket says 172.21.15.157:56420) 2023-11-06T03:18:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 -- 172.21.15.157:0/1981520286 learned_addr learned my addr 172.21.15.157:0/1981520286 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:01.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926a716640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f92640ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780139353 0 0) 0x7f926415e440 con 0x7f92640b2af0 2023-11-06T03:18:01.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9250003530 con 0x7f92640b2af0 2023-11-06T03:18:01.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717958037 0 0) 0x7f9250003530 con 0x7f92640b2af0 2023-11-06T03:18:01.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 msgr2=0x7f92640ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:01.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f92640ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 >> v1:172.21.15.195:6789/0 conn(0x7f92641599c0 legacy=0x7f92640b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926406c800 con 0x7f92640b2af0 2023-11-06T03:18:01.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f9254001680 con 0x7f92640b2af0 2023-11-06T03:18:01.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 >> v1:172.21.15.157:6789/0 conn(0x7f92640b2af0 legacy=0x7f9264159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92500039b0 0x7f9250003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9250004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f9250005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926406c520 con 0x7f92500047d0 2023-11-06T03:18:01.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925000b990 con 0x7f92500039b0 2023-11-06T03:18:01.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925000bb50 con 0x7f926406d000 2023-11-06T03:18:01.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9250004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f926a716640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f9250005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269f15640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92500039b0 0x7f9250003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 -- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 msgr2=0x7f9250005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:01.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926406d000 0x7f9250005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 -- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92500039b0 msgr2=0x7f9250003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:01.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f92500039b0 0x7f9250003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 -- 172.21.15.157:0/1981520286 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250003530 con 0x7f92500047d0 2023-11-06T03:18:01.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9269714640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9250004c50 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f925400c190 tx=0x7f925401f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a82848833b779f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254005ae0 con 0x7f92500047d0 2023-11-06T03:18:01.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92540ab070 con 0x7f92500047d0 2023-11-06T03:18:01.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.198+0000 7f9268f13640 1 -- 172.21.15.157:0/1981520286 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254007070 con 0x7f92500047d0 2023-11-06T03:18:01.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/1981520286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 msgr2=0x7f9250004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 --2- 172.21.15.157:0/1981520286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9250004c50 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f925400c190 tx=0x7f925401f330 comp rx=0 tx=0).stop 2023-11-06T03:18:01.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926a716640 1 -- 172.21.15.157:0/1981520286 reap_dead start 2023-11-06T03:18:01.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/1981520286 shutdown_connections 2023-11-06T03:18:01.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/1981520286 >> 172.21.15.157:0/1981520286 conn(0x7f9264065930 msgr2=0x7f9264065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:01.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/1981520286 shutdown_connections 2023-11-06T03:18:01.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/1981520286 wait complete. 2023-11-06T03:18:01.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 Processor -- start 2023-11-06T03:18:01.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- start start 2023-11-06T03:18:01.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926415b6d0 0x7f926415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9264158860 0x7f9264158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92641f1550 con 0x7f92500047d0 2023-11-06T03:18:01.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92641f23b0 con 0x7f9264158860 2023-11-06T03:18:01.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926415c090 con 0x7f926415b6d0 2023-11-06T03:18:01.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35932/0 (socket says 172.21.15.157:35932) 2023-11-06T03:18:01.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 -- 172.21.15.157:0/2753073083 learned_addr learned my addr 172.21.15.157:0/2753073083 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:01.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926a716640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9264158860 0x7f9264158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269714640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926415b6d0 0x7f926415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 -- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926415b6d0 msgr2=0x7f926415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926415b6d0 0x7f926415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 -- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9264158860 msgr2=0x7f9264158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9264158860 0x7f9264158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 -- 172.21.15.157:0/2753073083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9264159220 con 0x7f92500047d0 2023-11-06T03:18:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f9269f15640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f926000e4d0 tx=0x7f926000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600052e0 con 0x7f92500047d0 2023-11-06T03:18:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260005440 con 0x7f92500047d0 2023-11-06T03:18:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92641f1730 con 0x7f92500047d0 2023-11-06T03:18:01.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260003820 con 0x7f92500047d0 2023-11-06T03:18:01.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641f1cf0 con 0x7f92500047d0 2023-11-06T03:18:01.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.202+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264072750 con 0x7f92500047d0 2023-11-06T03:18:01.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.206+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9260003980 con 0x7f92500047d0 2023-11-06T03:18:01.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.206+0000 7f925a7fc640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 0x7f9248067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.206+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6149+0+0 (secure 0 0 0) 0x7f9260083010 con 0x7f92500047d0 2023-11-06T03:18:01.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.206+0000 7f9269714640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 0x7f9248067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.206+0000 7f9269714640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 0x7f9248067f20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f92540af000 tx=0x7f92540aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.210+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9260050810 con 0x7f92500047d0 2023-11-06T03:18:01.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.382+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92640ad2d0 con 0x7f92500047d0 2023-11-06T03:18:01.382 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:18:01.386+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:18:01.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.398+0000 7f925a7fc640 1 -- 172.21.15.157:0/2753073083 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v124) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9260053e30 con 0x7f92500047d0 2023-11-06T03:18:01.397 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:18:01.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 msgr2=0x7f9248067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 0x7f9248067f20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f92540af000 tx=0x7f92540aa000 comp rx=0 tx=0).stop 2023-11-06T03:18:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 msgr2=0x7f9264151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f926000e4d0 tx=0x7f926000e990 comp rx=0 tx=0).stop 2023-11-06T03:18:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 shutdown_connections 2023-11-06T03:18:01.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9248065a60 0x7f9248067f20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9264158860 0x7f9264158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f926415b6d0 0x7f926415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 --2- 172.21.15.157:0/2753073083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92500047d0 0x7f9264151560 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 >> 172.21.15.157:0/2753073083 conn(0x7f9264065930 msgr2=0x7f92640b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:01.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 shutdown_connections 2023-11-06T03:18:01.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.402+0000 7f926c179640 1 -- 172.21.15.157:0/2753073083 wait complete. 2023-11-06T03:18:01.416 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:18:01.434 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:18:01.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 Processor -- start 2023-11-06T03:18:01.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 -- start start 2023-11-06T03:18:01.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f773c0ce800 0x7f773c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f773c06b6a0 0x7f773c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c0cf1a0 con 0x7f773c146370 2023-11-06T03:18:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0b2120 con 0x7f773c0ce800 2023-11-06T03:18:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f774160c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c06c3d0 con 0x7f773c06b6a0 2023-11-06T03:18:01.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f773c0ce800 0x7f773c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f773c0ce800 0x7f773c0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35960/0 (socket says 172.21.15.157:35960) 2023-11-06T03:18:01.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 -- 172.21.15.157:0/2066439618 learned_addr learned my addr 172.21.15.157:0/2066439618 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:01.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773b7fe640 1 --2- 172.21.15.157:0/2066439618 >> v2:172.21.15.195:3300/0 conn(0x7f773c06b6a0 0x7f773c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 -- 172.21.15.157:0/2066439618 >> v2:172.21.15.195:3300/0 conn(0x7f773c06b6a0 msgr2=0x7f773c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:01.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 --2- 172.21.15.157:0/2066439618 >> v2:172.21.15.195:3300/0 conn(0x7f773c06b6a0 0x7f773c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 -- 172.21.15.157:0/2066439618 >> v1:172.21.15.195:6789/0 conn(0x7f773c146370 legacy=0x7f773c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:01.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.542+0000 7f773a7fc640 1 -- 172.21.15.157:0/2066439618 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c06f5a0 con 0x7f773c0ce800 2023-11-06T03:18:01.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f773a7fc640 1 --2- 172.21.15.157:0/2066439618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c0ce800 0x7f773c0cec60 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ebceaf38f0a2fd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f7739ffb640 1 -- 172.21.15.157:0/2066439618 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240ab600 con 0x7f773c0ce800 2023-11-06T03:18:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f7739ffb640 1 -- 172.21.15.157:0/2066439618 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77240ab760 con 0x7f773c0ce800 2023-11-06T03:18:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f7739ffb640 1 -- 172.21.15.157:0/2066439618 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240abaf0 con 0x7f773c0ce800 2023-11-06T03:18:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 -- 172.21.15.157:0/2066439618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c0ce800 msgr2=0x7f773c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 --2- 172.21.15.157:0/2066439618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c0ce800 0x7f773c0cec60 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772401c400 comp rx=0 tx=0).stop 2023-11-06T03:18:01.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 -- 172.21.15.157:0/2066439618 shutdown_connections 2023-11-06T03:18:01.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 --2- 172.21.15.157:0/2066439618 >> v2:172.21.15.195:3300/0 conn(0x7f773c06b6a0 0x7f773c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 --2- 172.21.15.157:0/2066439618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c0ce800 0x7f773c0cec60 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 -- 172.21.15.157:0/2066439618 >> 172.21.15.157:0/2066439618 conn(0x7f773c065930 msgr2=0x7f773c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:01.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 -- 172.21.15.157:0/2066439618 shutdown_connections 2023-11-06T03:18:01.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.546+0000 7f774160c640 1 -- 172.21.15.157:0/2066439618 wait complete. 2023-11-06T03:18:01.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 Processor -- start 2023-11-06T03:18:01.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- start start 2023-11-06T03:18:01.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f773c06b6a0 0x7f773c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 0x7f773c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0b0050 con 0x7f773c146370 2023-11-06T03:18:01.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c1403f0 con 0x7f773c0ce800 2023-11-06T03:18:01.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c1406d0 con 0x7f773c06b6a0 2023-11-06T03:18:01.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35962/0 (socket says 172.21.15.157:35962) 2023-11-06T03:18:01.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 -- 172.21.15.157:0/1428355979 learned_addr learned my addr 172.21.15.157:0/1428355979 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:01.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 -- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f773c06b6a0 msgr2=0x7f773c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773a7fc640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 0x7f773c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773affd640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f773c06b6a0 0x7f773c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f773c06b6a0 0x7f773c13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 -- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 msgr2=0x7f773c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:01.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 0x7f773c15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:01.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 -- 172.21.15.157:0/1428355979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240094d0 con 0x7f773c146370 2023-11-06T03:18:01.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773a7fc640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 0x7f773c15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:01.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f773b7fe640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f7734007e60 tx=0x7f773400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734014420 con 0x7f773c146370 2023-11-06T03:18:01.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77340145a0 con 0x7f773c146370 2023-11-06T03:18:01.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734014950 con 0x7f773c146370 2023-11-06T03:18:01.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c1409b0 con 0x7f773c146370 2023-11-06T03:18:01.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c140f70 con 0x7f773c146370 2023-11-06T03:18:01.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c072750 con 0x7f773c146370 2023-11-06T03:18:01.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7734014ab0 con 0x7f773c146370 2023-11-06T03:18:01.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.550+0000 7f77337fe640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 0x7f771c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:01.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.554+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f7734083cd0 con 0x7f773c146370 2023-11-06T03:18:01.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.554+0000 7f773affd640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 0x7f771c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:01.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.554+0000 7f773affd640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 0x7f771c067f20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f772c003d60 tx=0x7f772c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:01.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.558+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7734051480 con 0x7f773c146370 2023-11-06T03:18:01.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:01.742+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f773c0b1b90 con 0x7f773c146370 2023-11-06T03:18:02.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.410+0000 7f77337fe640 1 -- 172.21.15.157:0/1428355979 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v125) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7734054aa0 con 0x7f773c146370 2023-11-06T03:18:02.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 msgr2=0x7f771c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 0x7f771c067f20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f772c003d60 tx=0x7f772c009040 comp rx=0 tx=0).stop 2023-11-06T03:18:02.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 msgr2=0x7f773c13d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f7734007e60 tx=0x7f773400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 shutdown_connections 2023-11-06T03:18:02.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f771c065a60 0x7f771c067f20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f773c146370 0x7f773c13d660 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f773c0ce800 0x7f773c15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 --2- 172.21.15.157:0/1428355979 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f773c06b6a0 0x7f773c13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 >> 172.21.15.157:0/1428355979 conn(0x7f773c065930 msgr2=0x7f773c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:02.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 shutdown_connections 2023-11-06T03:18:02.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.414+0000 7f774160c640 1 -- 172.21.15.157:0/1428355979 wait complete. 2023-11-06T03:18:02.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:18:02.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.602+0000 7fedceba5640 1 Processor -- start 2023-11-06T03:18:02.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.602+0000 7fedceba5640 1 -- start start 2023-11-06T03:18:02.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.602+0000 7fedceba5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedc80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.602+0000 7fedceba5640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80cd180 con 0x7fedc806fa80 2023-11-06T03:18:02.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedceba5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80cdac0 con 0x7fedc81592f0 2023-11-06T03:18:02.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedceba5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80ce360 con 0x7fedc80ce6b0 2023-11-06T03:18:02.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcd142640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fedc806fa80 0x7fedc80b0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56442/0 (socket says 172.21.15.157:56442) 2023-11-06T03:18:02.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcd142640 1 -- 172.21.15.157:0/1135848496 learned_addr learned my addr 172.21.15.157:0/1135848496 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:02.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc7fff640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedc80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968108340 0 0) 0x7fedc80cd180 con 0x7fedc806fa80 2023-11-06T03:18:02.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb8003530 con 0x7fedc806fa80 2023-11-06T03:18:02.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4087526282 0 0) 0x7fedb8003530 con 0x7fedc806fa80 2023-11-06T03:18:02.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 msgr2=0x7fedc80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:02.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedc80cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 >> v1:172.21.15.195:6789/0 conn(0x7fedc81592f0 legacy=0x7fedc815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80cdda0 con 0x7fedc806fa80 2023-11-06T03:18:02.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fedc0001a00 con 0x7fedc806fa80 2023-11-06T03:18:02.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc7fff640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedc80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:02.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 >> v1:172.21.15.157:6789/0 conn(0x7fedc806fa80 legacy=0x7fedc80b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb80039b0 0x7fedb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedb80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedb8004d10 0x7fedb8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80ce360 con 0x7fedb80039b0 2023-11-06T03:18:02.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb800cbb0 con 0x7fedb8004d10 2023-11-06T03:18:02.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb800cd70 con 0x7fedc80ce6b0 2023-11-06T03:18:02.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb80039b0 0x7fedb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc7fff640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedb80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 -- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 msgr2=0x7fedb80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:02.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcd142640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedb8004d10 0x7fedb8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedc80ce6b0 0x7fedb80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 -- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedb8004d10 msgr2=0x7fedb8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:02.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedb8004d10 0x7fedb8005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 -- 172.21.15.157:0/1135848496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8003530 con 0x7fedb80039b0 2023-11-06T03:18:02.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedcc941640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb80039b0 0x7fedb8003dd0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fedbc001750 tx=0x7fedbc01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4e5c37f1dab345 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:02.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0052d0 con 0x7fedb80039b0 2023-11-06T03:18:02.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0a7050 con 0x7fedb80039b0 2023-11-06T03:18:02.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.606+0000 7fedc77fe640 1 -- 172.21.15.157:0/1135848496 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc003600 con 0x7fedb80039b0 2023-11-06T03:18:02.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/1135848496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb80039b0 msgr2=0x7fedb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 --2- 172.21.15.157:0/1135848496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedb80039b0 0x7fedb8003dd0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fedbc001750 tx=0x7fedbc01c2d0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 -- 172.21.15.157:0/1135848496 reap_dead start 2023-11-06T03:18:02.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/1135848496 shutdown_connections 2023-11-06T03:18:02.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/1135848496 >> 172.21.15.157:0/1135848496 conn(0x7fedc8069f40 msgr2=0x7fedc806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:02.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/1135848496 shutdown_connections 2023-11-06T03:18:02.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/1135848496 wait complete. 2023-11-06T03:18:02.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 Processor -- start 2023-11-06T03:18:02.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- start start 2023-11-06T03:18:02.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 0x7fedc8147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 0x7fedc8149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81587f0 con 0x7fedc814b350 2023-11-06T03:18:02.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc814bcd0 con 0x7fedc81494e0 2023-11-06T03:18:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8148440 con 0x7fedb80039b0 2023-11-06T03:18:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:35986/0 (socket says 172.21.15.157:35986) 2023-11-06T03:18:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 -- 172.21.15.157:0/3020893725 learned_addr learned my addr 172.21.15.157:0/3020893725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:02.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedc7fff640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 0x7fedc8149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcc941640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 0x7fedc8147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 -- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 msgr2=0x7fedc8147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 0x7fedc8147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 -- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 msgr2=0x7fedc8149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:02.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 0x7fedc8149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:02.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 -- 172.21.15.157:0/3020893725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8148720 con 0x7fedc814b350 2023-11-06T03:18:02.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedc7fff640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 0x7fedc8149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:02.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcc941640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 0x7fedc8147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:02.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedcd142640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fedc0001340 tx=0x7fedc000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:02.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc000bb20 con 0x7fedc814b350 2023-11-06T03:18:02.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc000bca0 con 0x7fedc814b350 2023-11-06T03:18:02.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc8148a00 con 0x7fedc814b350 2023-11-06T03:18:02.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0007070 con 0x7fedc814b350 2023-11-06T03:18:02.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.610+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc8148fc0 con 0x7fedc814b350 2023-11-06T03:18:02.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc81491b0 con 0x7fedc814b350 2023-11-06T03:18:02.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fedc00071d0 con 0x7fedc814b350 2023-11-06T03:18:02.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedc57fa640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 0x7fedac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:02.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fedc0083620 con 0x7fedc814b350 2023-11-06T03:18:02.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedcc941640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 0x7fedac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:02.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.614+0000 7fedcc941640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 0x7fedac067f20 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fedb8004750 tx=0x7fedbc0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:02.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.618+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fedc0050d40 con 0x7fedc814b350 2023-11-06T03:18:02.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:02.762+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fedc804e050 con 0x7fedc814b350 2023-11-06T03:18:03.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.618+0000 7fedc57fa640 1 -- 172.21.15.157:0/3020893725 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v179) v1 ==== 181+0+0 (secure 0 0 0) 0x7fedc0054360 con 0x7fedc814b350 2023-11-06T03:18:03.616 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:18:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 msgr2=0x7fedac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 0x7fedac067f20 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fedb8004750 tx=0x7fedbc0011a0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 msgr2=0x7fedc814b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fedc0001340 tx=0x7fedc000b4a0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 shutdown_connections 2023-11-06T03:18:03.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fedac065a60 0x7fedac067f20 secure :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fedb8004750 tx=0x7fedbc0011a0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc814b350 0x7fedc814b790 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fedc81494e0 0x7fedc8149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 --2- 172.21.15.157:0/3020893725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fedb80039b0 0x7fedc8147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 >> 172.21.15.157:0/3020893725 conn(0x7fedc8069f40 msgr2=0x7fedc80c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:03.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 shutdown_connections 2023-11-06T03:18:03.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.622+0000 7fedceba5640 1 -- 172.21.15.157:0/3020893725 wait complete. 2023-11-06T03:18:03.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:18:03.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 Processor -- start 2023-11-06T03:18:03.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 -- start start 2023-11-06T03:18:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77f0159c80 0x7f77f00b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f015dda0 0x7f77f015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f0070ee0 con 0x7f77f006b2f0 2023-11-06T03:18:03.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f015b060 con 0x7f77f0159c80 2023-11-06T03:18:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77f57e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f006b780 con 0x7f77f015dda0 2023-11-06T03:18:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77eeffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77f0159c80 0x7f77f00b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77ee7fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f77f006b2f0 0x7f77f015d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56450/0 (socket says 172.21.15.157:56450) 2023-11-06T03:18:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.770+0000 7f77ee7fc640 1 -- 172.21.15.157:0/1944993537 learned_addr learned my addr 172.21.15.157:0/1944993537 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:03.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77ef7fe640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f015dda0 0x7f77f015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77eeffd640 1 -- 172.21.15.157:0/1944993537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f015dda0 msgr2=0x7f77f015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77eeffd640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f015dda0 0x7f77f015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77eeffd640 1 -- 172.21.15.157:0/1944993537 >> v1:172.21.15.157:6789/0 conn(0x7f77f006b2f0 legacy=0x7f77f015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77eeffd640 1 -- 172.21.15.157:0/1944993537 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00b2a90 con 0x7f77f0159c80 2023-11-06T03:18:03.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77eeffd640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f0159c80 0x7f77f00b3570 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f77e00097c0 tx=0x7f77e001c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5d89e23a9adde0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77edffb640 1 -- 172.21.15.157:0/1944993537 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e00ab600 con 0x7f77f0159c80 2023-11-06T03:18:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77edffb640 1 -- 172.21.15.157:0/1944993537 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e00a7050 con 0x7f77f0159c80 2023-11-06T03:18:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77edffb640 1 -- 172.21.15.157:0/1944993537 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e00ab980 con 0x7f77f0159c80 2023-11-06T03:18:03.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 -- 172.21.15.157:0/1944993537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f0159c80 msgr2=0x7f77f00b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f0159c80 0x7f77f00b3570 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f77e00097c0 tx=0x7f77e001c250 comp rx=0 tx=0).stop 2023-11-06T03:18:03.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 -- 172.21.15.157:0/1944993537 shutdown_connections 2023-11-06T03:18:03.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f015dda0 0x7f77f015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 --2- 172.21.15.157:0/1944993537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f0159c80 0x7f77f00b3570 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 -- 172.21.15.157:0/1944993537 >> 172.21.15.157:0/1944993537 conn(0x7f77f00b43b0 msgr2=0x7f77f0065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:03.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 -- 172.21.15.157:0/1944993537 shutdown_connections 2023-11-06T03:18:03.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.774+0000 7f77f57e4640 1 -- 172.21.15.157:0/1944993537 wait complete. 2023-11-06T03:18:03.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 Processor -- start 2023-11-06T03:18:03.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- start start 2023-11-06T03:18:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f0159c80 0x7f77f014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77f00ae840 0x7f77f0151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00aed00 con 0x7f77f015dda0 2023-11-06T03:18:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00aefe0 con 0x7f77f00ae840 2023-11-06T03:18:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00af2c0 con 0x7f77f0159c80 2023-11-06T03:18:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36016/0 (socket says 172.21.15.157:36016) 2023-11-06T03:18:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 -- 172.21.15.157:0/3990158723 learned_addr learned my addr 172.21.15.157:0/3990158723 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 -- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f0159c80 msgr2=0x7f77f014c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f0159c80 0x7f77f014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 -- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77f00ae840 msgr2=0x7f77f0151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ef7fe640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77f00ae840 0x7f77f0151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77f00ae840 0x7f77f0151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 -- 172.21.15.157:0/3990158723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009420 con 0x7f77f015dda0 2023-11-06T03:18:03.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77ee7fc640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f77e400c780 tx=0x7f77e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40074e0 con 0x7f77f015dda0 2023-11-06T03:18:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4007640 con 0x7f77f015dda0 2023-11-06T03:18:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f00af5a0 con 0x7f77f015dda0 2023-11-06T03:18:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004a30 con 0x7f77f015dda0 2023-11-06T03:18:03.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f014d090 con 0x7f77f015dda0 2023-11-06T03:18:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f014d310 con 0x7f77f015dda0 2023-11-06T03:18:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f77e4004b90 con 0x7f77f015dda0 2023-11-06T03:18:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.778+0000 7f77d77fe640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 0x7f77dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.782+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f77e4082720 con 0x7f77f015dda0 2023-11-06T03:18:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.782+0000 7f77eeffd640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 0x7f77dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:03.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.782+0000 7f77eeffd640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 0x7f77dc067f20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f77e00ad000 tx=0x7f77e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:03.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.786+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f77e404fff0 con 0x7f77f015dda0 2023-11-06T03:18:03.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:03.954+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77f004e050 con 0x7f77f015dda0 2023-11-06T03:18:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.622+0000 7f77d77fe640 1 -- 172.21.15.157:0/3990158723 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v180) v1 ==== 169+0+0 (secure 0 0 0) 0x7f77e4018070 con 0x7f77f015dda0 2023-11-06T03:18:04.619 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:18:04.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 msgr2=0x7f77dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 0x7f77dc067f20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f77e00ad000 tx=0x7f77e0008040 comp rx=0 tx=0).stop 2023-11-06T03:18:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 msgr2=0x7f77f00ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f77e400c780 tx=0x7f77e400cc40 comp rx=0 tx=0).stop 2023-11-06T03:18:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 shutdown_connections 2023-11-06T03:18:04.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f77dc065a60 0x7f77dc067f20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77f00ae840 0x7f77f0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77f015dda0 0x7f77f00ae300 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 --2- 172.21.15.157:0/3990158723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77f0159c80 0x7f77f014c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 >> 172.21.15.157:0/3990158723 conn(0x7f77f00b43b0 msgr2=0x7f77f00b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:04.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 shutdown_connections 2023-11-06T03:18:04.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.626+0000 7f77f57e4640 1 -- 172.21.15.157:0/3990158723 wait complete. 2023-11-06T03:18:04.648 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:18:04.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.774+0000 7fe6c5e1c640 1 Processor -- start 2023-11-06T03:18:04.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.774+0000 7fe6c5e1c640 1 -- start start 2023-11-06T03:18:04.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6c00b2af0 0x7fe6c00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c006d000 0x7fe6c0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c015e440 con 0x7fe6c01599c0 2023-11-06T03:18:04.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c006c240 con 0x7fe6c00b2af0 2023-11-06T03:18:04.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c006c520 con 0x7fe6c006d000 2023-11-06T03:18:04.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6c00b2af0 0x7fe6c00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:04.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6c00b2af0 0x7fe6c00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36032/0 (socket says 172.21.15.157:36032) 2023-11-06T03:18:04.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 -- 172.21.15.157:0/75828805 learned_addr learned my addr 172.21.15.157:0/75828805 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:04.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6bf7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe6c01599c0 0x7fe6c00b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56464/0 (socket says 172.21.15.157:56464) 2023-11-06T03:18:04.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 -- 172.21.15.157:0/75828805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c006d000 msgr2=0x7fe6c0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:04.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6bffff640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c006d000 0x7fe6c0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:04.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c006d000 0x7fe6c0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 -- 172.21.15.157:0/75828805 >> v1:172.21.15.157:6789/0 conn(0x7fe6c01599c0 legacy=0x7fe6c00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6beffd640 1 -- 172.21.15.157:0/75828805 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433616687 0 0) 0x7fe6c015e440 con 0x7fe6c01599c0 2023-11-06T03:18:04.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 -- 172.21.15.157:0/75828805 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c006c800 con 0x7fe6c00b2af0 2023-11-06T03:18:04.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6b6ffd640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c00b2af0 0x7fe6c00ae000 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009870 tx=0x7fe6a801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39df4c8f7eba4ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:04.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6beffd640 1 -- 172.21.15.157:0/75828805 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80a95f0 con 0x7fe6c00b2af0 2023-11-06T03:18:04.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6beffd640 1 -- 172.21.15.157:0/75828805 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a80a9750 con 0x7fe6c00b2af0 2023-11-06T03:18:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6beffd640 1 -- 172.21.15.157:0/75828805 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80a9ae0 con 0x7fe6c00b2af0 2023-11-06T03:18:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/75828805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c00b2af0 msgr2=0x7fe6c00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c00b2af0 0x7fe6c00ae000 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009870 tx=0x7fe6a801c510 comp rx=0 tx=0).stop 2023-11-06T03:18:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/75828805 shutdown_connections 2023-11-06T03:18:04.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c006d000 0x7fe6c0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/75828805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c00b2af0 0x7fe6c00ae000 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/75828805 >> 172.21.15.157:0/75828805 conn(0x7fe6c0065930 msgr2=0x7fe6c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:04.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/75828805 shutdown_connections 2023-11-06T03:18:04.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.778+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/75828805 wait complete. 2023-11-06T03:18:04.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 Processor -- start 2023-11-06T03:18:04.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- start start 2023-11-06T03:18:04.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c00b2af0 0x7fe6c014f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c01599c0 0x7fe6c014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c014e230 con 0x7fe6c006d000 2023-11-06T03:18:04.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c014e4d0 con 0x7fe6c00b2af0 2023-11-06T03:18:04.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c014e7b0 con 0x7fe6c01599c0 2023-11-06T03:18:04.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:04.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36046/0 (socket says 172.21.15.157:36046) 2023-11-06T03:18:04.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 -- 172.21.15.157:0/2410504672 learned_addr learned my addr 172.21.15.157:0/2410504672 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:04.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 -- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c01599c0 msgr2=0x7fe6c014fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c01599c0 0x7fe6c014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 -- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c00b2af0 msgr2=0x7fe6c014f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6b6ffd640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c00b2af0 0x7fe6c014f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:04.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c00b2af0 0x7fe6c014f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 -- 172.21.15.157:0/2410504672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80094d0 con 0x7fe6c006d000 2023-11-06T03:18:04.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bf7fe640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00011f0 tx=0x7fe6b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:04.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00074e0 con 0x7fe6c006d000 2023-11-06T03:18:04.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0007640 con 0x7fe6c006d000 2023-11-06T03:18:04.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c014ea90 con 0x7fe6c006d000 2023-11-06T03:18:04.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0004a10 con 0x7fe6c006d000 2023-11-06T03:18:04.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c014f050 con 0x7fe6c006d000 2023-11-06T03:18:04.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.782+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c014f240 con 0x7fe6c006d000 2023-11-06T03:18:04.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.786+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe6b0004b70 con 0x7fe6c006d000 2023-11-06T03:18:04.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.786+0000 7fe6bcff9640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 0x7fe6a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:04.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.786+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe6b00825a0 con 0x7fe6c006d000 2023-11-06T03:18:04.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.786+0000 7fe6b6ffd640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 0x7fe6a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:04.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.786+0000 7fe6b6ffd640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 0x7fe6a0067f20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80ae000 tx=0x7fe6a80b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:04.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.790+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe6b0050100 con 0x7fe6c006d000 2023-11-06T03:18:04.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.946+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe6c0072750 con 0x7fe6c006d000 2023-11-06T03:18:04.954 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6bcff9640 1 -- 172.21.15.157:0/2410504672 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6b0053720 con 0x7fe6c006d000 2023-11-06T03:18:04.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 msgr2=0x7fe6a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 0x7fe6a0067f20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80ae000 tx=0x7fe6a80b3040 comp rx=0 tx=0).stop 2023-11-06T03:18:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 msgr2=0x7fe6c0132f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00011f0 tx=0x7fe6b000c9b0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 shutdown_connections 2023-11-06T03:18:04.957 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe6a0065a60 0x7fe6a0067f20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6c01599c0 0x7fe6c014fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6c00b2af0 0x7fe6c014f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 --2- 172.21.15.157:0/2410504672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6c006d000 0x7fe6c0132f60 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:04.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 >> 172.21.15.157:0/2410504672 conn(0x7fe6c0065930 msgr2=0x7fe6c00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:04.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 shutdown_connections 2023-11-06T03:18:04.959 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:04.958+0000 7fe6c5e1c640 1 -- 172.21.15.157:0/2410504672 wait complete. 2023-11-06T03:18:04.974 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2023-11-06T03:18:05.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 Processor -- start 2023-11-06T03:18:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 -- start start 2023-11-06T03:18:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f751415cc70 0x7f75140d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75140cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7514160830 con 0x7f75140cde90 2023-11-06T03:18:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75141614b0 con 0x7f751415cc70 2023-11-06T03:18:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f751b7ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7514160b10 con 0x7f75140d25a0 2023-11-06T03:18:05.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519556640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f751415cc70 0x7f75140d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519d57640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f75140cde90 0x7f75140d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56474/0 (socket says 172.21.15.157:56474) 2023-11-06T03:18:05.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519d57640 1 -- 172.21.15.157:0/2072554449 learned_addr learned my addr 172.21.15.157:0/2072554449 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:05.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519556640 1 -- 172.21.15.157:0/2072554449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 msgr2=0x7f75140cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:05.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519556640 1 --2- 172.21.15.157:0/2072554449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75140cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519556640 1 -- 172.21.15.157:0/2072554449 >> v1:172.21.15.157:6789/0 conn(0x7f75140cde90 legacy=0x7f75140d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f7519556640 1 -- 172.21.15.157:0/2072554449 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514160df0 con 0x7f751415cc70 2023-11-06T03:18:05.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.094+0000 7f750bfff640 1 -- 172.21.15.157:0/2072554449 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060212197 0 0) 0x7f7514160830 con 0x7f75140cde90 2023-11-06T03:18:05.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519556640 1 --2- 172.21.15.157:0/2072554449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f75140d2060 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f75100097c0 tx=0x7f751001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74273ca1a761f7d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f750bfff640 1 -- 172.21.15.157:0/2072554449 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75100a8970 con 0x7f751415cc70 2023-11-06T03:18:05.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f750bfff640 1 -- 172.21.15.157:0/2072554449 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75100a8ad0 con 0x7f751415cc70 2023-11-06T03:18:05.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f750bfff640 1 -- 172.21.15.157:0/2072554449 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75100a8e60 con 0x7f751415cc70 2023-11-06T03:18:05.097 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/2072554449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 msgr2=0x7f75140d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- 172.21.15.157:0/2072554449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f75140d2060 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f75100097c0 tx=0x7f751001ec70 comp rx=0 tx=0).stop 2023-11-06T03:18:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/2072554449 shutdown_connections 2023-11-06T03:18:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- 172.21.15.157:0/2072554449 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75140cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- 172.21.15.157:0/2072554449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f75140d2060 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/2072554449 >> 172.21.15.157:0/2072554449 conn(0x7f7514070560 msgr2=0x7f7514070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/2072554449 shutdown_connections 2023-11-06T03:18:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/2072554449 wait complete. 2023-11-06T03:18:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 Processor -- start 2023-11-06T03:18:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- start start 2023-11-06T03:18:05.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75140cde90 0x7f751414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.102 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75141f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751414b890 con 0x7f751415cc70 2023-11-06T03:18:05.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75141f6810 con 0x7f75140cde90 2023-11-06T03:18:05.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75141f5880 con 0x7f75140d25a0 2023-11-06T03:18:05.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36064/0 (socket says 172.21.15.157:36064) 2023-11-06T03:18:05.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 -- 172.21.15.157:0/4158770878 learned_addr learned my addr 172.21.15.157:0/4158770878 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:05.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519556640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75140cde90 0x7f751414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 -- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 msgr2=0x7f75141f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75141f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 -- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75140cde90 msgr2=0x7f751414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75140cde90 0x7f751414b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 -- 172.21.15.157:0/4158770878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510009420 con 0x7f751415cc70 2023-11-06T03:18:05.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7519d57640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f750c001570 tx=0x7f750c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00a5e0 con 0x7f751415cc70 2023-11-06T03:18:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c00a760 con 0x7f751415cc70 2023-11-06T03:18:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751414edc0 con 0x7f751415cc70 2023-11-06T03:18:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00ab10 con 0x7f751415cc70 2023-11-06T03:18:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.098+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75140c9340 con 0x7f751415cc70 2023-11-06T03:18:05.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514072750 con 0x7f751415cc70 2023-11-06T03:18:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f750c00ac70 con 0x7f751415cc70 2023-11-06T03:18:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f7509ffb640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 0x7f74f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f750c082510 con 0x7f751415cc70 2023-11-06T03:18:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f7519556640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 0x7f74f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.102+0000 7f7519556640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 0x7f74f8067f20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f75100b2000 tx=0x7f7510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.106+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f750c050100 con 0x7f751415cc70 2023-11-06T03:18:05.322 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.326+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f75140c6c30 con 0x7f751415cc70 2023-11-06T03:18:05.633 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.634+0000 7f7509ffb640 1 -- 172.21.15.157:0/4158770878 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v645) v1 ==== 147+0+0 (secure 0 0 0) 0x7f750c0180d0 con 0x7f751415cc70 2023-11-06T03:18:05.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 msgr2=0x7f74f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 0x7f74f8067f20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f75100b2000 tx=0x7f7510008040 comp rx=0 tx=0).stop 2023-11-06T03:18:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 msgr2=0x7f751414e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f750c001570 tx=0x7f750c00cca0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 shutdown_connections 2023-11-06T03:18:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f74f8065a60 0x7f74f8067f20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f751415cc70 0x7f751414e880 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75140d25a0 0x7f75141f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 --2- 172.21.15.157:0/4158770878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75140cde90 0x7f751414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 >> 172.21.15.157:0/4158770878 conn(0x7f7514070560 msgr2=0x7f75140b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 shutdown_connections 2023-11-06T03:18:05.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.638+0000 7f751b7ba640 1 -- 172.21.15.157:0/4158770878 wait complete. 2023-11-06T03:18:05.657 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2023-11-06T03:18:05.657 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2023-11-06T03:18:05.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2023-11-06T03:18:05.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 Processor -- start 2023-11-06T03:18:05.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 -- start start 2023-11-06T03:18:05.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f20d815ce80 0x7f20d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f20d80d00b0 0x7f20d80b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80d2920 con 0x7f20d815caa0 2023-11-06T03:18:05.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80d0520 con 0x7f20d815ce80 2023-11-06T03:18:05.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20ddbab640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80d18a0 con 0x7f20d80d00b0 2023-11-06T03:18:05.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f20d815caa0 0x7f20d80cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56476/0 (socket says 172.21.15.157:56476) 2023-11-06T03:18:05.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d77fe640 1 -- 172.21.15.157:0/4248703054 learned_addr learned my addr 172.21.15.157:0/4248703054 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:05.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/4248703054 >> v2:172.21.15.157:3300/0 conn(0x7f20d815ce80 0x7f20d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384293609 0 0) 0x7f20d80d2920 con 0x7f20d815caa0 2023-11-06T03:18:05.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c4003530 con 0x7f20d815caa0 2023-11-06T03:18:05.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d7fff640 1 --2- 172.21.15.157:0/4248703054 >> v2:172.21.15.195:3300/0 conn(0x7f20d80d00b0 0x7f20d80b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d6ffd640 1 -- 172.21.15.157:0/4248703054 >> v2:172.21.15.195:3300/0 conn(0x7f20d80d00b0 msgr2=0x7f20d80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/4248703054 >> v2:172.21.15.195:3300/0 conn(0x7f20d80d00b0 0x7f20d80b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624653674 0 0) 0x7f20c4003530 con 0x7f20d815caa0 2023-11-06T03:18:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d6ffd640 1 -- 172.21.15.157:0/4248703054 >> v1:172.21.15.157:6789/0 conn(0x7f20d815caa0 legacy=0x7f20d80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.782+0000 7f20d6ffd640 1 -- 172.21.15.157:0/4248703054 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80d1b60 con 0x7f20d815ce80 2023-11-06T03:18:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/4248703054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815ce80 0x7f20d80cfb70 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f20d80cf670 tx=0x7f20c001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9f9e10ea0a3827 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0001680 con 0x7f20d815ce80 2023-11-06T03:18:05.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c00a5640 con 0x7f20d815ce80 2023-11-06T03:18:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d67fc640 1 -- 172.21.15.157:0/4248703054 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00a59d0 con 0x7f20d815ce80 2023-11-06T03:18:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- 172.21.15.157:0/4248703054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815ce80 msgr2=0x7f20d80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- 172.21.15.157:0/4248703054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815ce80 0x7f20d80cfb70 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f20d80cf670 tx=0x7f20c001c800 comp rx=0 tx=0).stop 2023-11-06T03:18:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- 172.21.15.157:0/4248703054 shutdown_connections 2023-11-06T03:18:05.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- 172.21.15.157:0/4248703054 >> v2:172.21.15.195:3300/0 conn(0x7f20d80d00b0 0x7f20d80b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- 172.21.15.157:0/4248703054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815ce80 0x7f20d80cfb70 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- 172.21.15.157:0/4248703054 >> 172.21.15.157:0/4248703054 conn(0x7f20d8070790 msgr2=0x7f20d8070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- 172.21.15.157:0/4248703054 shutdown_connections 2023-11-06T03:18:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- 172.21.15.157:0/4248703054 wait complete. 2023-11-06T03:18:05.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 Processor -- start 2023-11-06T03:18:05.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- start start 2023-11-06T03:18:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20d80d00b0 0x7f20d8152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20d815ce80 0x7f20d815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81534c0 con 0x7f20d815caa0 2023-11-06T03:18:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8153650 con 0x7f20d815ce80 2023-11-06T03:18:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20ddbab640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8153c10 con 0x7f20d80d00b0 2023-11-06T03:18:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36092/0 (socket says 172.21.15.157:36092) 2023-11-06T03:18:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 -- 172.21.15.157:0/13465525 learned_addr learned my addr 172.21.15.157:0/13465525 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d77fe640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20d80d00b0 0x7f20d8152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d7fff640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20d815ce80 0x7f20d815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 -- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20d80d00b0 msgr2=0x7f20d8152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20d80d00b0 0x7f20d8152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 -- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20d815ce80 msgr2=0x7f20d815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:05.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20d815ce80 0x7f20d815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:05.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 -- 172.21.15.157:0/13465525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0009a70 con 0x7f20d815caa0 2023-11-06T03:18:05.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20d6ffd640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b0040 tx=0x7f20c00195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00ad850 con 0x7f20d815caa0 2023-11-06T03:18:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.786+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c00ad9b0 con 0x7f20d815caa0 2023-11-06T03:18:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d815deb0 con 0x7f20d815caa0 2023-11-06T03:18:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c00add60 con 0x7f20d815caa0 2023-11-06T03:18:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d815e470 con 0x7f20d815caa0 2023-11-06T03:18:05.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d80ce330 con 0x7f20d815caa0 2023-11-06T03:18:05.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f20c00b4050 con 0x7f20d815caa0 2023-11-06T03:18:05.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20cffff640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 0x7f20b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:05.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f20c011dd30 con 0x7f20d815caa0 2023-11-06T03:18:05.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20d77fe640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 0x7f20b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:05.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.790+0000 7f20d77fe640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 0x7f20b8067f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001ec0 tx=0x7f20d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:05.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:05.794+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f20c00eb9b0 con 0x7f20d815caa0 2023-11-06T03:18:05.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.002+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f20d8072750 con 0x7f20d815caa0 2023-11-06T03:18:06.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.638+0000 7f20cffff640 1 -- 172.21.15.157:0/13465525 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v646) v1 ==== 136+0+0 (secure 0 0 0) 0x7f20c00b20c0 con 0x7f20d815caa0 2023-11-06T03:18:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 msgr2=0x7f20b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 0x7f20b8067f20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f20d0001ec0 tx=0x7f20d0009040 comp rx=0 tx=0).stop 2023-11-06T03:18:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 msgr2=0x7f20d815fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f20c00b0040 tx=0x7f20c00195a0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 shutdown_connections 2023-11-06T03:18:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f20b8065a60 0x7f20b8067f20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20d815ce80 0x7f20d815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20d815caa0 0x7f20d815fb10 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 --2- 172.21.15.157:0/13465525 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20d80d00b0 0x7f20d8152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 >> 172.21.15.157:0/13465525 conn(0x7f20d8070790 msgr2=0x7f20d8155900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 shutdown_connections 2023-11-06T03:18:06.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.642+0000 7f20ddbab640 1 -- 172.21.15.157:0/13465525 wait complete. 2023-11-06T03:18:06.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-06T03:18:06.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.790+0000 7fbf51069640 1 Processor -- start 2023-11-06T03:18:06.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.790+0000 7fbf51069640 1 -- start start 2023-11-06T03:18:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbf4c156a70 0x7fbf4c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adf40 0x7fbf4c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c06b950 con 0x7fbf4c0adae0 2023-11-06T03:18:06.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c06c490 con 0x7fbf4c156a70 2023-11-06T03:18:06.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c06bb60 con 0x7fbf4c0adf40 2023-11-06T03:18:06.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36096/0 (socket says 172.21.15.157:36096) 2023-11-06T03:18:06.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 -- 172.21.15.157:0/3257443614 learned_addr learned my addr 172.21.15.157:0/3257443614 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:06.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4ad9d640 1 --2- 172.21.15.157:0/3257443614 >> v2:172.21.15.195:3300/0 conn(0x7fbf4c156a70 0x7fbf4c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4b59e640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adf40 0x7fbf4c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 -- 172.21.15.157:0/3257443614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adf40 msgr2=0x7fbf4c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:06.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adf40 0x7fbf4c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 -- 172.21.15.157:0/3257443614 >> v2:172.21.15.195:3300/0 conn(0x7fbf4c156a70 msgr2=0x7fbf4c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:06.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 --2- 172.21.15.157:0/3257443614 >> v2:172.21.15.195:3300/0 conn(0x7fbf4c156a70 0x7fbf4c06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 -- 172.21.15.157:0/3257443614 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c06be40 con 0x7fbf4c0adae0 2023-11-06T03:18:06.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf4a59c640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ce880 tx=0x7fbf4001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e973648b87f7a52 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf49d9b640 1 -- 172.21.15.157:0/3257443614 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400a7070 con 0x7fbf4c0adae0 2023-11-06T03:18:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf49d9b640 1 -- 172.21.15.157:0/3257443614 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf400ab430 con 0x7fbf4c0adae0 2023-11-06T03:18:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf49d9b640 1 -- 172.21.15.157:0/3257443614 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400ab7c0 con 0x7fbf4c0adae0 2023-11-06T03:18:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 -- 172.21.15.157:0/3257443614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0adae0 msgr2=0x7fbf4c0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.794+0000 7fbf51069640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ce880 tx=0x7fbf4001c4d0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/3257443614 shutdown_connections 2023-11-06T03:18:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adf40 0x7fbf4c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- 172.21.15.157:0/3257443614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c0adae0 0x7fbf4c0b1c20 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- 172.21.15.157:0/3257443614 >> v2:172.21.15.195:3300/0 conn(0x7fbf4c156a70 0x7fbf4c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/3257443614 >> 172.21.15.157:0/3257443614 conn(0x7fbf4c065930 msgr2=0x7fbf4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/3257443614 shutdown_connections 2023-11-06T03:18:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/3257443614 wait complete. 2023-11-06T03:18:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 Processor -- start 2023-11-06T03:18:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- start start 2023-11-06T03:18:06.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 0x7fbf4c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf4c0adf40 0x7fbf4c1463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1468f0 con 0x7fbf4c156a70 2023-11-06T03:18:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0aca80 con 0x7fbf4c0adf40 2023-11-06T03:18:06.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c15e730 con 0x7fbf4c0adae0 2023-11-06T03:18:06.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36102/0 (socket says 172.21.15.157:36102) 2023-11-06T03:18:06.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 -- 172.21.15.157:0/2575662251 learned_addr learned my addr 172.21.15.157:0/2575662251 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:06.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4a59c640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf4c0adf40 0x7fbf4c1463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4ad9d640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 0x7fbf4c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 -- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 msgr2=0x7fbf4c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 0x7fbf4c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 -- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf4c0adf40 msgr2=0x7fbf4c1463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf4c0adf40 0x7fbf4c1463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 -- 172.21.15.157:0/2575662251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf400094d0 con 0x7fbf4c156a70 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4ad9d640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 0x7fbf4c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf4b59e640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fbf44007e40 tx=0x7fbf4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44014620 con 0x7fbf4c156a70 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400ee90 con 0x7fbf4c156a70 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c15ea10 con 0x7fbf4c156a70 2023-11-06T03:18:06.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440149e0 con 0x7fbf4c156a70 2023-11-06T03:18:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c15efd0 con 0x7fbf4c156a70 2023-11-06T03:18:06.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.798+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c0ce330 con 0x7fbf4c156a70 2023-11-06T03:18:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.802+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbf44014b40 con 0x7fbf4c156a70 2023-11-06T03:18:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.802+0000 7fbf337fe640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 0x7fbf38067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:06.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.802+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbf44083910 con 0x7fbf4c156a70 2023-11-06T03:18:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.802+0000 7fbf4ad9d640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 0x7fbf38067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.802+0000 7fbf4ad9d640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 0x7fbf38067f50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0cdd90 tx=0x7fbf3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:06.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.806+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbf44051500 con 0x7fbf4c156a70 2023-11-06T03:18:06.956 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.958+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbf4c15f1c0 con 0x7fbf4c156a70 2023-11-06T03:18:06.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf337fe640 1 -- 172.21.15.157:0/2575662251 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbf44054b20 con 0x7fbf4c156a70 2023-11-06T03:18:06.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 msgr2=0x7fbf38067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 0x7fbf38067f50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0cdd90 tx=0x7fbf3c008040 comp rx=0 tx=0).stop 2023-11-06T03:18:06.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 msgr2=0x7fbf4c15f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:06.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fbf44007e40 tx=0x7fbf4400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 shutdown_connections 2023-11-06T03:18:06.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbf38065a90 0x7fbf38067f50 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf4c156a70 0x7fbf4c15f4f0 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbf4c0adf40 0x7fbf4c1463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 --2- 172.21.15.157:0/2575662251 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbf4c0adae0 0x7fbf4c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:06.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 >> 172.21.15.157:0/2575662251 conn(0x7fbf4c065930 msgr2=0x7fbf4c0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:06.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 shutdown_connections 2023-11-06T03:18:06.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:06.966+0000 7fbf51069640 1 -- 172.21.15.157:0/2575662251 wait complete. 2023-11-06T03:18:06.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:18:06.983 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:18:07.027 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:18:07.075 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:18:07.076 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:07.077 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:07.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:18:07.084 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:18:07.084 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:07.091 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:18:07.091 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:07.091 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:07.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:18:07.130 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:18:07.131 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:07.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:18:07.138 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-06T03:18:07.139 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:07.301 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.302+0000 7fec0a3fb640 1 Processor -- start 2023-11-06T03:18:07.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.302+0000 7fec0a3fb640 1 -- start start 2023-11-06T03:18:07.302 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec0a3fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec0a3fb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fec0406d000 0x7fec040b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec0a3fb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0415e440 con 0x7fec0415dea0 2023-11-06T03:18:07.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec0a3fb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0415adb0 con 0x7fec0406d000 2023-11-06T03:18:07.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec0a3fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0406c240 con 0x7fec041599c0 2023-11-06T03:18:07.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec03fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7febfbfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fec0406d000 0x7fec040b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.304 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec03fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:56190/0 (socket says 172.21.15.157:56190) 2023-11-06T03:18:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec03fff640 1 -- 172.21.15.157:0/3065594682 learned_addr learned my addr 172.21.15.157:0/3065594682 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7febfbfff640 1 -- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 msgr2=0x7fec040ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7febfbfff640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7febfbfff640 1 -- 172.21.15.157:0/3065594682 >> v1:172.21.15.195:6789/0 conn(0x7fec0415dea0 legacy=0x7fec04070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.305 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7febfbfff640 1 -- 172.21.15.157:0/3065594682 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0406c520 con 0x7fec0406d000 2023-11-06T03:18:07.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.306+0000 7fec03fff640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:18:07.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7febfbfff640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec0406d000 0x7fec040b23e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7febec0097c0 tx=0x7febec01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162df5186db4bafc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:07.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec037fe640 1 -- 172.21.15.157:0/3065594682 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0ad660 con 0x7fec0406d000 2023-11-06T03:18:07.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec037fe640 1 -- 172.21.15.157:0/3065594682 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec0ad7c0 con 0x7fec0406d000 2023-11-06T03:18:07.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec037fe640 1 -- 172.21.15.157:0/3065594682 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0adb50 con 0x7fec0406d000 2023-11-06T03:18:07.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec0406d000 msgr2=0x7fec040b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec0406d000 0x7fec040b23e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7febec0097c0 tx=0x7febec01c450 comp rx=0 tx=0).stop 2023-11-06T03:18:07.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- 172.21.15.157:0/3065594682 shutdown_connections 2023-11-06T03:18:07.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec0406d000 0x7fec040b23e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/3065594682 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec041599c0 0x7fec040ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- 172.21.15.157:0/3065594682 >> 172.21.15.157:0/3065594682 conn(0x7fec04065930 msgr2=0x7fec04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:07.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- 172.21.15.157:0/3065594682 shutdown_connections 2023-11-06T03:18:07.309 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- 172.21.15.157:0/3065594682 wait complete. 2023-11-06T03:18:07.311 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 Processor -- start 2023-11-06T03:18:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- start start 2023-11-06T03:18:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec0406d000 0x7fec0414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 0x7fec04151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.312 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0414ffb0 con 0x7fec0415dea0 2023-11-06T03:18:07.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec04150140 con 0x7fec041599c0 2023-11-06T03:18:07.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec0a3fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec041518c0 con 0x7fec0406d000 2023-11-06T03:18:07.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36104/0 (socket says 172.21.15.157:36104) 2023-11-06T03:18:07.313 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 -- 172.21.15.157:0/2887778312 learned_addr learned my addr 172.21.15.157:0/2887778312 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:07.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7febfbfff640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 0x7fec04151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec03fff640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec0406d000 0x7fec0414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 -- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec0406d000 msgr2=0x7fec0414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec0406d000 0x7fec0414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.314 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 -- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 msgr2=0x7fec04151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 0x7fec04151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7fec08998640 1 -- 172.21.15.157:0/2887778312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf400b770 con 0x7fec0415dea0 2023-11-06T03:18:07.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.310+0000 7febfbfff640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 0x7fec04151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:07.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec08998640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c730 tx=0x7febfc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:07.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc003b80 con 0x7fec0415dea0 2023-11-06T03:18:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc003ce0 con 0x7fec0415dea0 2023-11-06T03:18:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec009420 con 0x7fec0415dea0 2023-11-06T03:18:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc00aa30 con 0x7fec0415dea0 2023-11-06T03:18:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec041f9190 con 0x7fec0415dea0 2023-11-06T03:18:07.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec04072750 con 0x7fec0415dea0 2023-11-06T03:18:07.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7febfc00ab90 con 0x7fec0415dea0 2023-11-06T03:18:07.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 0x7febe4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:07.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5296+0+0 (secure 0 0 0) 0x7febfc0838d0 con 0x7fec0415dea0 2023-11-06T03:18:07.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec03fff640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 0x7febe4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:07.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.314+0000 7fec03fff640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 0x7febe4067f20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fec04155a10 tx=0x7febf400b450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:07.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.318+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7febfc051430 con 0x7fec0415dea0 2023-11-06T03:18:07.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.506+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec04150280 con 0x7fec0415dea0 2023-11-06T03:18:07.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.506+0000 7fec017fa640 1 -- 172.21.15.157:0/2887778312 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+3536 (secure 0 0 0) 0x7febfc054a50 con 0x7fec0415dea0 2023-11-06T03:18:07.505 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:07.505 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":125,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/582431629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":582431629},{"type":"v1","addr":"172.21.15.195:6836","nonce":582431629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/264446037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":264446037},{"type":"v1","addr":"172.21.15.195:6837","nonce":264446037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10815,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1102034615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6833","nonce":1102034615},{"type":"v1","addr":"172.21.15.157:6835","nonce":1102034615}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":11157,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/63869469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":63869469},{"type":"v1","addr":"172.21.15.157:6834","nonce":63869469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":125}],"filesystems":[]} 2023-11-06T03:18:07.506 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 125 2023-11-06T03:18:07.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 msgr2=0x7febe4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 0x7febe4067f20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fec04155a10 tx=0x7febf400b450 comp rx=0 tx=0).stop 2023-11-06T03:18:07.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 msgr2=0x7fec04154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:07.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c730 tx=0x7febfc00cbf0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 shutdown_connections 2023-11-06T03:18:07.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7febe4065a60 0x7febe4067f20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0415dea0 0x7fec04154e50 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec041599c0 0x7fec04151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 --2- 172.21.15.157:0/2887778312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec0406d000 0x7fec0414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:07.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 >> 172.21.15.157:0/2887778312 conn(0x7fec04065930 msgr2=0x7fec040afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:07.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 shutdown_connections 2023-11-06T03:18:07.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:07.510+0000 7fec0a3fb640 1 -- 172.21.15.157:0/2887778312 wait complete. 2023-11-06T03:18:07.524 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-06T03:18:07.524 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-06T03:18:07.525 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:18:07.525 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:18:07.525 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-06T03:18:07.526 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-06T03:18:07.526 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:18:07.526 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:18:07.527 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-06T03:18:07.527 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-06T03:18:07.527 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:18:07.527 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:18:07.528 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-06T03:18:07.528 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-06T03:18:07.528 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:18:07.528 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:18:07.536 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:18:07.536 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-06T03:18:07.540 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:18:07.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-06T03:18:07.542 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:18:07.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-06T03:18:07.581 INFO:tasks.ceph.mds.c:Started 2023-11-06T03:18:07.581 DEBUG:teuthology.parallel:result is None 2023-11-06T03:18:07.586 INFO:tasks.ceph.mds.b:Started 2023-11-06T03:18:07.587 DEBUG:teuthology.parallel:result is None 2023-11-06T03:18:07.587 INFO:tasks.ceph.mds.d:Started 2023-11-06T03:18:07.589 DEBUG:teuthology.parallel:result is None 2023-11-06T03:18:07.648 INFO:tasks.ceph.mds.c.smithi157.stdout:starting mds.c at 2023-11-06T03:18:07.658 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2023-11-06T03:18:07.659 INFO:tasks.ceph.mds.d.smithi195.stdout:starting mds.d at 2023-11-06T03:18:09.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-06T03:18:12.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:18:12.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:18:13.538 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:18:13.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-06T03:18:13.542 INFO:tasks.ceph.mds.a:Started 2023-11-06T03:18:13.543 DEBUG:teuthology.parallel:result is None 2023-11-06T03:18:13.544 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-06T03:18:13.616 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:18:13.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.666+0000 7ff8210dd640 1 Processor -- start 2023-11-06T03:18:13.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.666+0000 7ff8210dd640 1 -- start start 2023-11-06T03:18:13.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff8210dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff81c154fc0 0x7ff81c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff8210dd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff81c0b51d0 0x7ff81c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff8210dd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0b08d0 con 0x7ff81c0b1090 2023-11-06T03:18:13.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff8210dd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0b0c70 con 0x7ff81c154fc0 2023-11-06T03:18:13.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff8210dd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06c590 con 0x7ff81c0b51d0 2023-11-06T03:18:13.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff81c154fc0 0x7ff81c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:13.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff81c154fc0 0x7ff81c070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40976/0 (socket says 172.21.15.157:40976) 2023-11-06T03:18:13.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 -- 172.21.15.157:0/284783285 learned_addr learned my addr 172.21.15.157:0/284783285 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:13.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 -- 172.21.15.157:0/284783285 >> v2:172.21.15.195:3300/0 conn(0x7ff81c0b51d0 msgr2=0x7ff81c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:13.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 --2- 172.21.15.157:0/284783285 >> v2:172.21.15.195:3300/0 conn(0x7ff81c0b51d0 0x7ff81c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 -- 172.21.15.157:0/284783285 >> v1:172.21.15.195:6789/0 conn(0x7ff81c0b1090 legacy=0x7ff81c0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:13.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 -- 172.21.15.157:0/284783285 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c06cb30 con 0x7ff81c154fc0 2023-11-06T03:18:13.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81ad9d640 1 --2- 172.21.15.157:0/284783285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c154fc0 0x7ff81c070ee0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7ff81c15c160 tx=0x7ff80401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88a05dfd8e9cb2c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:13.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81a59c640 1 -- 172.21.15.157:0/284783285 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040aa640 con 0x7ff81c154fc0 2023-11-06T03:18:13.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81a59c640 1 -- 172.21.15.157:0/284783285 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040aa7a0 con 0x7ff81c154fc0 2023-11-06T03:18:13.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.670+0000 7ff81a59c640 1 -- 172.21.15.157:0/284783285 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040aab30 con 0x7ff81c154fc0 2023-11-06T03:18:13.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- 172.21.15.157:0/284783285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c154fc0 msgr2=0x7ff81c070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:13.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- 172.21.15.157:0/284783285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c154fc0 0x7ff81c070ee0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7ff81c15c160 tx=0x7ff80401c4c0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- 172.21.15.157:0/284783285 shutdown_connections 2023-11-06T03:18:13.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- 172.21.15.157:0/284783285 >> v2:172.21.15.195:3300/0 conn(0x7ff81c0b51d0 0x7ff81c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- 172.21.15.157:0/284783285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c154fc0 0x7ff81c070ee0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- 172.21.15.157:0/284783285 >> 172.21.15.157:0/284783285 conn(0x7ff81c069d70 msgr2=0x7ff81c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:13.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- 172.21.15.157:0/284783285 shutdown_connections 2023-11-06T03:18:13.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- 172.21.15.157:0/284783285 wait complete. 2023-11-06T03:18:13.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 Processor -- start 2023-11-06T03:18:13.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- start start 2023-11-06T03:18:13.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff81c0b51d0 0x7ff81c143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 0x7ff81c146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c144480 con 0x7ff81c0b1090 2023-11-06T03:18:13.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c147980 con 0x7ff81c0b51d0 2023-11-06T03:18:13.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff8210dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c1476a0 con 0x7ff81c154fc0 2023-11-06T03:18:13.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:13.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40986/0 (socket says 172.21.15.157:40986) 2023-11-06T03:18:13.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 -- 172.21.15.157:0/562923650 learned_addr learned my addr 172.21.15.157:0/562923650 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:13.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff813fff640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff81c0b51d0 0x7ff81c143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:13.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81b59e640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 0x7ff81c146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:13.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 -- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 msgr2=0x7ff81c146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:13.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 0x7ff81c146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 -- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff81c0b51d0 msgr2=0x7ff81c143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:13.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff81c0b51d0 0x7ff81c143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:13.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 -- 172.21.15.157:0/562923650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040094d0 con 0x7ff81c0b1090 2023-11-06T03:18:13.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81b59e640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 0x7ff81c146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:13.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.674+0000 7ff81ad9d640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ac040 tx=0x7ff80401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:13.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040bd5f0 con 0x7ff81c0b1090 2023-11-06T03:18:13.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040bd770 con 0x7ff81c0b1090 2023-11-06T03:18:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c155520 con 0x7ff81c0b1090 2023-11-06T03:18:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040bdb20 con 0x7ff81c0b1090 2023-11-06T03:18:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c152dd0 con 0x7ff81c0b1090 2023-11-06T03:18:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c072750 con 0x7ff81c0b1090 2023-11-06T03:18:13.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.678+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff8040a8050 con 0x7ff81c0b1090 2023-11-06T03:18:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.682+0000 7ff8137fe640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 0x7ff7fc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.682+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff80411d730 con 0x7ff81c0b1090 2023-11-06T03:18:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.682+0000 7ff813fff640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 0x7ff7fc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.682+0000 7ff813fff640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 0x7ff7fc067f50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a300 tx=0x7ff80c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:13.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.682+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff8040eb560 con 0x7ff81c0b1090 2023-11-06T03:18:13.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:13.850+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff81c0d0180 con 0x7ff81c0b1090 2023-11-06T03:18:14.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.686+0000 7ff8137fe640 1 -- 172.21.15.157:0/562923650 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v185) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff8040eeb80 con 0x7ff81c0b1090 2023-11-06T03:18:14.683 INFO:teuthology.orchestra.run.smithi157.stderr: removed all blocklist entries 2023-11-06T03:18:14.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 msgr2=0x7ff7fc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 0x7ff7fc067f50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff81c06a300 tx=0x7ff80c009040 comp rx=0 tx=0).stop 2023-11-06T03:18:14.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 msgr2=0x7ff81c14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7ff8040ac040 tx=0x7ff80401ad00 comp rx=0 tx=0).stop 2023-11-06T03:18:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 shutdown_connections 2023-11-06T03:18:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff7fc065a90 0x7ff7fc067f50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff81c154fc0 0x7ff81c146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff81c0b51d0 0x7ff81c143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 --2- 172.21.15.157:0/562923650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff81c0b1090 0x7ff81c14b150 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 >> 172.21.15.157:0/562923650 conn(0x7ff81c069d70 msgr2=0x7ff81c14db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:14.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 shutdown_connections 2023-11-06T03:18:14.689 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.690+0000 7ff8210dd640 1 -- 172.21.15.157:0/562923650 wait complete. 2023-11-06T03:18:14.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-06T03:18:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 Processor -- start 2023-11-06T03:18:14.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 -- start start 2023-11-06T03:18:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f19a415caa0 0x7f19a40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a40d2920 con 0x7f19a40d0280 2023-11-06T03:18:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a40d18a0 con 0x7f19a415ce80 2023-11-06T03:18:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a8a27640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a40d1b60 con 0x7f19a415caa0 2023-11-06T03:18:14.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a2d9d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f19a40d0280 0x7f19a40b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56934/0 (socket says 172.21.15.157:56934) 2023-11-06T03:18:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a2d9d640 1 -- 172.21.15.157:0/1867670010 learned_addr learned my addr 172.21.15.157:0/1867670010 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a259c640 1 --2- 172.21.15.157:0/1867670010 >> v2:172.21.15.195:3300/0 conn(0x7f19a415caa0 0x7f19a40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.834+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788874952 0 0) 0x7f19a40d2920 con 0x7f19a40d0280 2023-11-06T03:18:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198c003530 con 0x7f19a40d0280 2023-11-06T03:18:14.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693872084 0 0) 0x7f198c003530 con 0x7f19a40d0280 2023-11-06T03:18:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 >> v2:172.21.15.195:3300/0 conn(0x7f19a415caa0 msgr2=0x7f19a40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 --2- 172.21.15.157:0/1867670010 >> v2:172.21.15.195:3300/0 conn(0x7f19a415caa0 0x7f19a40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 >> v1:172.21.15.195:6789/0 conn(0x7f19a415ce80 legacy=0x7f19a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40d1e40 con 0x7f19a40d0280 2023-11-06T03:18:14.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f199c001a00 con 0x7f19a40d0280 2023-11-06T03:18:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 >> v1:172.21.15.157:6789/0 conn(0x7f19a40d0280 legacy=0x7f19a40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 0x7f198c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c0047d0 0x7f198c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c005190 0x7f198c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a40d1b60 con 0x7f198c003980 2023-11-06T03:18:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c030 con 0x7f198c005190 2023-11-06T03:18:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c1f0 con 0x7f198c0047d0 2023-11-06T03:18:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 0x7f198c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a1d9b640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c0047d0 0x7f198c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.839 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a2d9d640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c005190 0x7f198c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 -- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c0047d0 msgr2=0x7f198c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c0047d0 0x7f198c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 -- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c005190 msgr2=0x7f198c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c005190 0x7f198c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.840 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 -- 172.21.15.157:0/1867670010 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c003530 con 0x7f198c003980 2023-11-06T03:18:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a2d9d640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f198c005190 0x7f198c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:14.841 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a2d9d640 1 -- 172.21.15.157:0/1867670010 reap_dead start 2023-11-06T03:18:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a259c640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 0x7f198c003da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f1994009ae0 tx=0x7f199401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fa87bdebe2d8ca server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:14.844 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940a55f0 con 0x7f198c003980 2023-11-06T03:18:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19940a5750 con 0x7f198c003980 2023-11-06T03:18:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a159a640 1 -- 172.21.15.157:0/1867670010 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19940a5b00 con 0x7f198c003980 2023-11-06T03:18:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 -- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 msgr2=0x7f198c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 0x7f198c003da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f1994009ae0 tx=0x7f199401ca20 comp rx=0 tx=0).stop 2023-11-06T03:18:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 -- 172.21.15.157:0/1867670010 shutdown_connections 2023-11-06T03:18:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 --2- 172.21.15.157:0/1867670010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f198c003980 0x7f198c003da0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 -- 172.21.15.157:0/1867670010 >> 172.21.15.157:0/1867670010 conn(0x7f19a4070790 msgr2=0x7f19a4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 -- 172.21.15.157:0/1867670010 shutdown_connections 2023-11-06T03:18:14.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.838+0000 7f19a8a27640 1 -- 172.21.15.157:0/1867670010 wait complete. 2023-11-06T03:18:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 Processor -- start 2023-11-06T03:18:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- start start 2023-11-06T03:18:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 0x7f19a41f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19a41f5200 0x7f19a415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4157010 con 0x7f19a415fe40 2023-11-06T03:18:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a40c9cf0 con 0x7f19a41f5200 2023-11-06T03:18:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41f9830 con 0x7f198c003980 2023-11-06T03:18:14.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:40996/0 (socket says 172.21.15.157:40996) 2023-11-06T03:18:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 -- 172.21.15.157:0/844465653 learned_addr learned my addr 172.21.15.157:0/844465653 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a259c640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 0x7f19a41f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 -- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 msgr2=0x7f19a41f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a1d9b640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19a41f5200 0x7f19a415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 0x7f19a41f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 -- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19a41f5200 msgr2=0x7f19a415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19a41f5200 0x7f19a415a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 -- 172.21.15.157:0/844465653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40c9f40 con 0x7f19a415fe40 2023-11-06T03:18:14.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a259c640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 0x7f19a41f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a2d9d640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f19a40b49b0 tx=0x7f199c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c004d70 con 0x7f19a415fe40 2023-11-06T03:18:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c00bb80 con 0x7f19a415fe40 2023-11-06T03:18:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a40ca220 con 0x7f19a415fe40 2023-11-06T03:18:14.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c007110 con 0x7f19a415fe40 2023-11-06T03:18:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a41f9500 con 0x7f19a415fe40 2023-11-06T03:18:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a4072750 con 0x7f19a415fe40 2023-11-06T03:18:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.842+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f199c007270 con 0x7f19a415fe40 2023-11-06T03:18:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.846+0000 7f1992ffd640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 0x7f1984067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:14.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.846+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f199c083300 con 0x7f19a415fe40 2023-11-06T03:18:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.846+0000 7f19a259c640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 0x7f1984067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.846+0000 7f19a259c640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 0x7f1984067f20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f19940af000 tx=0x7f1994008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:14.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:14.850+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f199c050f40 con 0x7f19a415fe40 2023-11-06T03:18:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f19a40d0fe0 con 0x7f19a415fe40 2023-11-06T03:18:15.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f1992ffd640 1 -- 172.21.15.157:0/844465653 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f199c054560 con 0x7f19a415fe40 2023-11-06T03:18:15.051 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:15.051 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:15.051 INFO:teuthology.orchestra.run.smithi157.stdout: "auth_dump": [ 2023-11-06T03:18:15.051 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.052 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.a", 2023-11-06T03:18:15.052 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlxd8TJxAAb3XI0kFXnXwGW2T/33BYnw==", 2023-11-06T03:18:15.052 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.052 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.052 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:18:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:18:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.053 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.b", 2023-11-06T03:18:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlWY+lNRAAJV1Tq4R3OIdSU14/O8OUEA==", 2023-11-06T03:18:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.054 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:18:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:18:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.055 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.056 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.c", 2023-11-06T03:18:15.056 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl1Sp2LxAAczzvAlR5Wd36Af3dw7a5AA==", 2023-11-06T03:18:15.056 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.056 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.056 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:18:15.057 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:18:15.057 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.057 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.057 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.057 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.058 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mds.d", 2023-11-06T03:18:15.058 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hl0HNZOxAAaiXi8FW46fzju+ECe2r9Mg==", 2023-11-06T03:18:15.058 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.058 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.059 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:18:15.059 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:18:15.059 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.059 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.059 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.060 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.060 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.0", 2023-11-06T03:18:15.060 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBxV0hlpcXMFhAALV5yJHZFciLR1Bs6NlFfkg==", 2023-11-06T03:18:15.060 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.060 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.061 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.062 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.1", 2023-11-06T03:18:15.062 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB0V0hlxTTuHBAAtkhjgnr3lXv/Df44qF5EOA==", 2023-11-06T03:18:15.062 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.062 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.062 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.063 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.063 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.063 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.063 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.063 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.2", 2023-11-06T03:18:15.064 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB3V0hlo02LJRAA6csLIBZpsI7R52bHAOj2ow==", 2023-11-06T03:18:15.064 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.064 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.064 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.064 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.065 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.065 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.065 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.065 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.3", 2023-11-06T03:18:15.065 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB6V0hl6D8cLhAACljjIWELylmOScaxXDj6HQ==", 2023-11-06T03:18:15.066 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.066 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.066 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.066 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.067 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.067 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.067 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.067 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.4", 2023-11-06T03:18:15.067 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQB+V0hljBLLNBAA59iHWnbiWh19pqL6ulI69g==", 2023-11-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.5", 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCBV0hl6ql9OhAAViMaSsso33JSaBxdnTN9gg==", 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.071 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.6", 2023-11-06T03:18:15.071 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCFV0hlfMR7BxAAwA5QHd1AYhtHE0YmXIds+A==", 2023-11-06T03:18:15.071 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.071 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.071 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "osd.7", 2023-11-06T03:18:15.072 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQCIV0hl0PM5DxAAuhpHe1qwvomkL/8w0XCqfw==", 2023-11-06T03:18:15.073 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.073 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow profile osd", 2023-11-06T03:18:15.073 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile osd", 2023-11-06T03:18:15.073 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.073 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.074 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.074 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.074 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.0", 2023-11-06T03:18:15.074 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlovCrBRAAZXDm2zYdArVUc28Rl/ZlUA==", 2023-11-06T03:18:15.074 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow r", 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.075 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.076 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.1", 2023-11-06T03:18:15.076 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlx2SWDRAA+B6xX5MNtIFnBUorN5rwZg==", 2023-11-06T03:18:15.076 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.076 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.076 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:18:15.077 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:18:15.077 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:18:15.077 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.077 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.077 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.078 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.2", 2023-11-06T03:18:15.078 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hlWNzdChAAmioc0JwHEDD9R2EhaVjodg==", 2023-11-06T03:18:15.078 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.078 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.078 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:18:15.079 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:18:15.079 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:18:15.079 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.079 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.079 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.3", 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBwV0hl63KqEhAA5VidPmHZabRqv0klEq8F4w==", 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow", 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow r", 2023-11-06T03:18:15.080 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow rw", 2023-11-06T03:18:15.081 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow rwx" 2023-11-06T03:18:15.081 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.081 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.081 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.081 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "client.admin", 2023-11-06T03:18:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlIJ+SDBAAFn2kEY+Lh2/hqtmsMiIStw==", 2023-11-06T03:18:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:18:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: "mgr": "allow *", 2023-11-06T03:18:15.082 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow *", 2023-11-06T03:18:15.083 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.083 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.083 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.083 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.083 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.x", 2023-11-06T03:18:15.084 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlfJMJIxAAqiWFccCEGsbz0yB1WpJbTA==", 2023-11-06T03:18:15.084 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.084 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:18:15.084 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:18:15.084 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: "entity": "mgr.y", 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: "key": "AQBvV0hlAA8rIBAAmEvR+P8xBnunCeQWr39j/Q==", 2023-11-06T03:18:15.085 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": { 2023-11-06T03:18:15.086 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": "allow *", 2023-11-06T03:18:15.086 INFO:teuthology.orchestra.run.smithi157.stdout: "mon": "allow profile mgr", 2023-11-06T03:18:15.086 INFO:teuthology.orchestra.run.smithi157.stdout: "osd": "allow *" 2023-11-06T03:18:15.086 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.086 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:15.087 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-11-06T03:18:15.087 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:15.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 msgr2=0x7f1984067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 0x7f1984067f20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f19940af000 tx=0x7f1994008040 comp rx=0 tx=0).stop 2023-11-06T03:18:15.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 msgr2=0x7f19a415a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f19a40b49b0 tx=0x7f199c00b4a0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 shutdown_connections 2023-11-06T03:18:15.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1984065a60 0x7f1984067f20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19a415fe40 0x7f19a415a940 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19a41f5200 0x7f19a415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 --2- 172.21.15.157:0/844465653 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f198c003980 0x7f19a41f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 >> 172.21.15.157:0/844465653 conn(0x7f19a4070790 msgr2=0x7f19a40b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 shutdown_connections 2023-11-06T03:18:15.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.054+0000 7f19a8a27640 1 -- 172.21.15.157:0/844465653 wait complete. 2023-11-06T03:18:15.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:15.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 Processor -- start 2023-11-06T03:18:15.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 -- start start 2023-11-06T03:18:15.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa35c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0d2920 con 0x7fa35c15caa0 2023-11-06T03:18:15.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0d18a0 con 0x7fa35c0d00b0 2023-11-06T03:18:15.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa363716640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0d0520 con 0x7fa35c15ce80 2023-11-06T03:18:15.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa3614b2640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa35c15caa0 0x7fa35c0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56940/0 (socket says 172.21.15.157:56940) 2023-11-06T03:18:15.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa3614b2640 1 -- 172.21.15.157:0/3238685092 learned_addr learned my addr 172.21.15.157:0/3238685092 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:15.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa360cb1640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa35c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198265074 0 0) 0x7fa35c0d2920 con 0x7fa35c15caa0 2023-11-06T03:18:15.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa348003530 con 0x7fa35c15caa0 2023-11-06T03:18:15.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108568460 0 0) 0x7fa348003530 con 0x7fa35c15caa0 2023-11-06T03:18:15.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 msgr2=0x7fa35c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa35c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 >> v1:172.21.15.195:6789/0 conn(0x7fa35c0d00b0 legacy=0x7fa35c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0d1b60 con 0x7fa35c15caa0 2023-11-06T03:18:15.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fa358001680 con 0x7fa35c15caa0 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 >> v1:172.21.15.157:6789/0 conn(0x7fa35c15caa0 legacy=0x7fa35c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa3480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa348004d10 0x7fa348005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0d0520 con 0x7fa348004d10 2023-11-06T03:18:15.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800ba70 con 0x7fa3480039b0 2023-11-06T03:18:15.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800bc30 con 0x7fa35c15ce80 2023-11-06T03:18:15.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa348004d10 0x7fa348005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa360cb1640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa3480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa3614b2640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 -- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 msgr2=0x7fa3480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c15ce80 0x7fa3480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 -- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3480039b0 msgr2=0x7fa348003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.218+0000 7fa361cb3640 1 -- 172.21.15.157:0/3238685092 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348003530 con 0x7fa348004d10 2023-11-06T03:18:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa361cb3640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa348004d10 0x7fa348005190 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fa354001e40 tx=0x7fa35401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cc7fec9a95aad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354003b80 con 0x7fa348004d10 2023-11-06T03:18:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa354003ce0 con 0x7fa348004d10 2023-11-06T03:18:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa353fff640 1 -- 172.21.15.157:0/3238685092 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540052c0 con 0x7fa348004d10 2023-11-06T03:18:15.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- 172.21.15.157:0/3238685092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa348004d10 msgr2=0x7fa348005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 --2- 172.21.15.157:0/3238685092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa348004d10 0x7fa348005190 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fa354001e40 tx=0x7fa35401c740 comp rx=0 tx=0).stop 2023-11-06T03:18:15.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa361cb3640 1 -- 172.21.15.157:0/3238685092 reap_dead start 2023-11-06T03:18:15.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- 172.21.15.157:0/3238685092 shutdown_connections 2023-11-06T03:18:15.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- 172.21.15.157:0/3238685092 >> 172.21.15.157:0/3238685092 conn(0x7fa35c070790 msgr2=0x7fa35c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- 172.21.15.157:0/3238685092 shutdown_connections 2023-11-06T03:18:15.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- 172.21.15.157:0/3238685092 wait complete. 2023-11-06T03:18:15.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 Processor -- start 2023-11-06T03:18:15.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- start start 2023-11-06T03:18:15.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa348004d10 0x7fa35c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c0b31a0 0x7fa35c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0b3b60 con 0x7fa35c0a9f20 2023-11-06T03:18:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0ab810 con 0x7fa348004d10 2023-11-06T03:18:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa363716640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0aa770 con 0x7fa35c0b31a0 2023-11-06T03:18:15.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa360cb1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.222+0000 7fa360cb1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41026/0 (socket says 172.21.15.157:41026) 2023-11-06T03:18:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 -- 172.21.15.157:0/1826298090 learned_addr learned my addr 172.21.15.157:0/1826298090 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa3614b2640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa348004d10 0x7fa35c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa361cb3640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c0b31a0 0x7fa35c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 -- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c0b31a0 msgr2=0x7fa35c0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c0b31a0 0x7fa35c0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 -- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa348004d10 msgr2=0x7fa35c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa348004d10 0x7fa35c0cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 -- 172.21.15.157:0/1826298090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0aaa50 con 0x7fa35c0a9f20 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa360cb1640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fa34c011610 tx=0x7fa34c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00f8f0 con 0x7fa35c0a9f20 2023-11-06T03:18:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c00fa50 con 0x7fa35c0a9f20 2023-11-06T03:18:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c0aad30 con 0x7fa35c0a9f20 2023-11-06T03:18:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00fdc0 con 0x7fa35c0a9f20 2023-11-06T03:18:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c0ab2f0 con 0x7fa35c0a9f20 2023-11-06T03:18:15.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c072750 con 0x7fa35c0a9f20 2023-11-06T03:18:15.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa34c015460 con 0x7fa35c0a9f20 2023-11-06T03:18:15.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 0x7fa340067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa34c0837c0 con 0x7fa35c0a9f20 2023-11-06T03:18:15.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa3614b2640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 0x7fa340067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.226+0000 7fa3614b2640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 0x7fa340067f50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa358000f70 tx=0x7fa358009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.230+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa34c051450 con 0x7fa35c0a9f20 2023-11-06T03:18:15.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.382+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35c066120 con 0x7fa35c0a9f20 2023-11-06T03:18:15.381 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.382+0000 7fa351ffb640 1 -- 172.21.15.157:0/1826298090 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+14229 (secure 0 0 0) 0x7fa34c054a70 con 0x7fa35c0a9f20 2023-11-06T03:18:15.381 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:15.381 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":185,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:14.680448+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:15.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 msgr2=0x7fa340067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 0x7fa340067f50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa358000f70 tx=0x7fa358009040 comp rx=0 tx=0).stop 2023-11-06T03:18:15.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 msgr2=0x7fa35c15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fa34c011610 tx=0x7fa34c011ad0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 shutdown_connections 2023-11-06T03:18:15.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa340065a90 0x7fa340067f50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa35c0b31a0 0x7fa35c0b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c0a9f20 0x7fa35c15a0b0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 --2- 172.21.15.157:0/1826298090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa348004d10 0x7fa35c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 >> 172.21.15.157:0/1826298090 conn(0x7fa35c070790 msgr2=0x7fa35c0b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 shutdown_connections 2023-11-06T03:18:15.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.386+0000 7fa363716640 1 -- 172.21.15.157:0/1826298090 wait complete. 2023-11-06T03:18:15.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:18:15.401 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:18:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 Processor -- start 2023-11-06T03:18:15.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 -- start start 2023-11-06T03:18:15.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f175c1551e0 0x7f175c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c15f580 con 0x7f175c15e5e0 2023-11-06T03:18:15.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c15efc0 con 0x7f175c0b3b20 2023-11-06T03:18:15.529 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1761919640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175c15f2a0 con 0x7f175c1551e0 2023-11-06T03:18:15.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1753fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f175c15e5e0 0x7f175c0b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56946/0 (socket says 172.21.15.157:56946) 2023-11-06T03:18:15.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1753fff640 1 -- 172.21.15.157:0/683240245 learned_addr learned my addr 172.21.15.157:0/683240245 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:15.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175affd640 1 --2- 172.21.15.157:0/683240245 >> v2:172.21.15.195:3300/0 conn(0x7f175c1551e0 0x7f175c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.530 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694777530 0 0) 0x7f175c15f580 con 0x7f175c15e5e0 2023-11-06T03:18:15.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1748003530 con 0x7f175c15e5e0 2023-11-06T03:18:15.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192428421 0 0) 0x7f1748003530 con 0x7f175c15e5e0 2023-11-06T03:18:15.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 >> v2:172.21.15.195:3300/0 conn(0x7f175c1551e0 msgr2=0x7f175c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 --2- 172.21.15.157:0/683240245 >> v2:172.21.15.195:3300/0 conn(0x7f175c1551e0 0x7f175c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 >> v1:172.21.15.195:6789/0 conn(0x7f175c0b3b20 legacy=0x7f175c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c06f5a0 con 0x7f175c15e5e0 2023-11-06T03:18:15.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1744001680 con 0x7f175c15e5e0 2023-11-06T03:18:15.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 >> v1:172.21.15.157:6789/0 conn(0x7f175c15e5e0 legacy=0x7f175c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1748003980 0x7f1748003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17480047d0 0x7f1748004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f174800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c15f2a0 con 0x7f17480052b0 2023-11-06T03:18:15.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174800c180 con 0x7f1748003980 2023-11-06T03:18:15.533 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174800c340 con 0x7f17480047d0 2023-11-06T03:18:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f174800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175affd640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1748003980 0x7f1748003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f1753fff640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17480047d0 0x7f1748004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 -- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17480047d0 msgr2=0x7f1748004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.534 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17480047d0 0x7f1748004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 -- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1748003980 msgr2=0x7f1748003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1748003980 0x7f1748003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.530+0000 7f175b7fe640 1 -- 172.21.15.157:0/683240245 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748003530 con 0x7f17480052b0 2023-11-06T03:18:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175b7fe640 1 -- 172.21.15.157:0/683240245 reap_dead start 2023-11-06T03:18:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175b7fe640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f174800b990 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1754001e40 tx=0x7f175401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42273c29335b9de server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.539 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754003b80 con 0x7f17480052b0 2023-11-06T03:18:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1754003ce0 con 0x7f17480052b0 2023-11-06T03:18:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175a7fc640 1 -- 172.21.15.157:0/683240245 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754005260 con 0x7f17480052b0 2023-11-06T03:18:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 msgr2=0x7f174800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f174800b990 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1754001e40 tx=0x7f175401c710 comp rx=0 tx=0).stop 2023-11-06T03:18:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- 172.21.15.157:0/683240245 shutdown_connections 2023-11-06T03:18:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 --2- 172.21.15.157:0/683240245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f174800b990 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- 172.21.15.157:0/683240245 >> 172.21.15.157:0/683240245 conn(0x7f175c065930 msgr2=0x7f175c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- 172.21.15.157:0/683240245 shutdown_connections 2023-11-06T03:18:15.541 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- 172.21.15.157:0/683240245 wait complete. 2023-11-06T03:18:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 Processor -- start 2023-11-06T03:18:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- start start 2023-11-06T03:18:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f175c153ac0 0x7f175c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175c0b1ad0 0x7f175c0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c152810 con 0x7f17480052b0 2023-11-06T03:18:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0b2490 con 0x7f175c153ac0 2023-11-06T03:18:15.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f1761919640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c153630 con 0x7f175c0b1ad0 2023-11-06T03:18:15.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41040/0 (socket says 172.21.15.157:41040) 2023-11-06T03:18:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.534+0000 7f175affd640 1 -- 172.21.15.157:0/4088347634 learned_addr learned my addr 172.21.15.157:0/4088347634 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175b7fe640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175c0b1ad0 0x7f175c0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1753fff640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f175c153ac0 0x7f175c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 -- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175c0b1ad0 msgr2=0x7f175c0b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175c0b1ad0 0x7f175c0b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 -- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f175c153ac0 msgr2=0x7f175c1494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f175c153ac0 0x7f175c1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 -- 172.21.15.157:0/4088347634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c156820 con 0x7f17480052b0 2023-11-06T03:18:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f175affd640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f174c011560 tx=0x7f174c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c015930 con 0x7f17480052b0 2023-11-06T03:18:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c015a90 con 0x7f17480052b0 2023-11-06T03:18:15.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c152a20 con 0x7f17480052b0 2023-11-06T03:18:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c015e60 con 0x7f17480052b0 2023-11-06T03:18:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c152fe0 con 0x7f17480052b0 2023-11-06T03:18:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c072750 con 0x7f17480052b0 2023-11-06T03:18:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f174c01d400 con 0x7f17480052b0 2023-11-06T03:18:15.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 0x7f173c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f174c083730 con 0x7f17480052b0 2023-11-06T03:18:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1753fff640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 0x7f173c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.538+0000 7f1753fff640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 0x7f173c067f50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1744001ec0 tx=0x7f1744009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.542+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f174c0512e0 con 0x7f17480052b0 2023-11-06T03:18:15.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.682+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f175c071730 con 0x7f17480052b0 2023-11-06T03:18:15.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1758ff9640 1 -- 172.21.15.157:0/4088347634 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f174c054900 con 0x7f17480052b0 2023-11-06T03:18:15.683 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:18:15.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 msgr2=0x7f173c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 0x7f173c067f50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1744001ec0 tx=0x7f1744009040 comp rx=0 tx=0).stop 2023-11-06T03:18:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 msgr2=0x7f175c14c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f174c011560 tx=0x7f174c011a20 comp rx=0 tx=0).stop 2023-11-06T03:18:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 shutdown_connections 2023-11-06T03:18:15.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f173c065a90 0x7f173c067f50 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f175c0b1ad0 0x7f175c0b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f175c153ac0 0x7f175c1494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 --2- 172.21.15.157:0/4088347634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17480052b0 0x7f175c14c190 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 >> 172.21.15.157:0/4088347634 conn(0x7f175c065930 msgr2=0x7f175c0cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 shutdown_connections 2023-11-06T03:18:15.688 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.686+0000 7f1761919640 1 -- 172.21.15.157:0/4088347634 wait complete. 2023-11-06T03:18:15.701 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:18:15.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-06T03:18:15.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 Processor -- start 2023-11-06T03:18:15.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 -- start start 2023-11-06T03:18:15.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38f806d000 0x7f38f80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f815e440 con 0x7f38f81599c0 2023-11-06T03:18:15.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f806c240 con 0x7f38f80b2af0 2023-11-06T03:18:15.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38ff052640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f806c520 con 0x7f38f806d000 2023-11-06T03:18:15.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fcdee640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f38f81599c0 0x7f38f80b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56950/0 (socket says 172.21.15.157:56950) 2023-11-06T03:18:15.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fcdee640 1 -- 172.21.15.157:0/604519736 learned_addr learned my addr 172.21.15.157:0/604519736 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:15.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388417711 0 0) 0x7f38f815e440 con 0x7f38f81599c0 2023-11-06T03:18:15.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003530 con 0x7f38f81599c0 2023-11-06T03:18:15.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/604519736 >> v2:172.21.15.195:3300/0 conn(0x7f38f806d000 0x7f38f80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4067041387 0 0) 0x7f38e4003530 con 0x7f38f81599c0 2023-11-06T03:18:15.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 >> v2:172.21.15.195:3300/0 conn(0x7f38f806d000 msgr2=0x7f38f80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 --2- 172.21.15.157:0/604519736 >> v2:172.21.15.195:3300/0 conn(0x7f38f806d000 0x7f38f80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 >> v1:172.21.15.195:6789/0 conn(0x7f38f80b2af0 legacy=0x7f38f8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f806c800 con 0x7f38f81599c0 2023-11-06T03:18:15.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f38f0001680 con 0x7f38f81599c0 2023-11-06T03:18:15.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 >> v1:172.21.15.157:6789/0 conn(0x7f38f81599c0 legacy=0x7f38f80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38e40047d0 0x7f38e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f806c520 con 0x7f38e4005190 2023-11-06T03:18:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c0e0 con 0x7f38e4003980 2023-11-06T03:18:15.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c2a0 con 0x7f38e40047d0 2023-11-06T03:18:15.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fcdee640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 -- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38e40047d0 msgr2=0x7f38e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38e40047d0 0x7f38e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 -- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38e4003980 msgr2=0x7f38e4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:15.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fd5ef640 1 -- 172.21.15.157:0/604519736 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4003530 con 0x7f38e4005190 2023-11-06T03:18:15.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.822+0000 7f38fcdee640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:15.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38fd5ef640 1 -- 172.21.15.157:0/604519736 reap_dead start 2023-11-06T03:18:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38e400b990 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f38e8007360 tx=0x7f38e801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bcc7d2a4d52e77 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80a5d60 con 0x7f38e4005190 2023-11-06T03:18:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80039a0 con 0x7f38e4005190 2023-11-06T03:18:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38f77fe640 1 -- 172.21.15.157:0/604519736 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8003d50 con 0x7f38e4005190 2023-11-06T03:18:15.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 msgr2=0x7f38e400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38e400b990 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f38e8007360 tx=0x7f38e801ce60 comp rx=0 tx=0).stop 2023-11-06T03:18:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- 172.21.15.157:0/604519736 shutdown_connections 2023-11-06T03:18:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 --2- 172.21.15.157:0/604519736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- 172.21.15.157:0/604519736 >> 172.21.15.157:0/604519736 conn(0x7f38f8065930 msgr2=0x7f38f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- 172.21.15.157:0/604519736 shutdown_connections 2023-11-06T03:18:15.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- 172.21.15.157:0/604519736 wait complete. 2023-11-06T03:18:15.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 Processor -- start 2023-11-06T03:18:15.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- start start 2023-11-06T03:18:15.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38f80cd520 0x7f38f80ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38f80cc420 0x7f38f81559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80ccc00 con 0x7f38e4005190 2023-11-06T03:18:15.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80cd980 con 0x7f38f80cc420 2023-11-06T03:18:15.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38ff052640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f815ba90 con 0x7f38f80cd520 2023-11-06T03:18:15.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38fcdee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38fcdee640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41054/0 (socket says 172.21.15.157:41054) 2023-11-06T03:18:15.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.826+0000 7f38fcdee640 1 -- 172.21.15.157:0/2824073967 learned_addr learned my addr 172.21.15.157:0/2824073967 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:15.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f7fff640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38f80cd520 0x7f38f80ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fd5ef640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38f80cc420 0x7f38f81559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 -- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38f80cd520 msgr2=0x7f38f80ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38f80cd520 0x7f38f80ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 -- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38f80cc420 msgr2=0x7f38f81559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38f80cc420 0x7f38f81559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.835 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 -- 172.21.15.157:0/2824073967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f815bd70 con 0x7f38e4005190 2023-11-06T03:18:15.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38fcdee640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f38f0001000 tx=0x7f38f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00074e0 con 0x7f38e4005190 2023-11-06T03:18:15.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0007640 con 0x7f38e4005190 2023-11-06T03:18:15.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f815c050 con 0x7f38e4005190 2023-11-06T03:18:15.836 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00049f0 con 0x7f38e4005190 2023-11-06T03:18:15.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80cc8d0 con 0x7f38e4005190 2023-11-06T03:18:15.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f815bbd0 con 0x7f38e4005190 2023-11-06T03:18:15.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38f0004b50 con 0x7f38e4005190 2023-11-06T03:18:15.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 0x7f38d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:15.837 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f38f0082ce0 con 0x7f38e4005190 2023-11-06T03:18:15.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f7fff640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 0x7f38d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:15.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.830+0000 7f38f7fff640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 0x7f38d8067f20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f38f815b880 tx=0x7f38e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:15.838 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.834+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38f815bbd0 con 0x7f38e4005190 2023-11-06T03:18:15.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38f815bbd0 con 0x7f38e4005190 2023-11-06T03:18:15.991 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38f57fa640 1 -- 172.21.15.157:0/2824073967 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v185) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38f815bbd0 con 0x7f38e4005190 2023-11-06T03:18:15.992 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:18:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 msgr2=0x7f38d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 0x7f38d8067f20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f38f815b880 tx=0x7f38e0009040 comp rx=0 tx=0).stop 2023-11-06T03:18:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 msgr2=0x7f38f815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:15.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.994+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f38f0001000 tx=0x7f38f000dc90 comp rx=0 tx=0).stop 2023-11-06T03:18:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 shutdown_connections 2023-11-06T03:18:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f38d8065a60 0x7f38d8067f20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38f80cc420 0x7f38f81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38f80cd520 0x7f38f80ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 --2- 172.21.15.157:0/2824073967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38e4005190 0x7f38f815c850 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 >> 172.21.15.157:0/2824073967 conn(0x7f38f8065930 msgr2=0x7f38f806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 shutdown_connections 2023-11-06T03:18:15.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:15.998+0000 7f38ff052640 1 -- 172.21.15.157:0/2824073967 wait complete. 2023-11-06T03:18:16.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-06T03:18:16.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 Processor -- start 2023-11-06T03:18:16.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 -- start start 2023-11-06T03:18:16.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de40b5f60 0x7f7de40cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7de40cfac0 0x7f7de40d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de415d210 con 0x7f7de415ce30 2023-11-06T03:18:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40d17f0 con 0x7f7de40b5f60 2023-11-06T03:18:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de9d9f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40b6410 con 0x7f7de40cfac0 2023-11-06T03:18:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de40b5f60 0x7f7de40cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de40b5f60 0x7f7de40cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41066/0 (socket says 172.21.15.157:41066) 2023-11-06T03:18:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7de415ce30 0x7f7de40cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56962/0 (socket says 172.21.15.157:56962) 2023-11-06T03:18:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2564729902 learned_addr learned my addr 172.21.15.157:0/2564729902 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2564729902 >> v2:172.21.15.195:3300/0 conn(0x7f7de40cfac0 msgr2=0x7f7de40d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de3fff640 1 --2- 172.21.15.157:0/2564729902 >> v2:172.21.15.195:3300/0 conn(0x7f7de40cfac0 0x7f7de40d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 --2- 172.21.15.157:0/2564729902 >> v2:172.21.15.195:3300/0 conn(0x7f7de40cfac0 0x7f7de40d1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de27fc640 1 -- 172.21.15.157:0/2564729902 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648244421 0 0) 0x7f7de415d210 con 0x7f7de415ce30 2023-11-06T03:18:16.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2564729902 >> v1:172.21.15.157:6789/0 conn(0x7f7de415ce30 legacy=0x7f7de40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2564729902 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40ce090 con 0x7f7de40b5f60 2023-11-06T03:18:16.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de2ffd640 1 --2- 172.21.15.157:0/2564729902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40b5f60 0x7f7de40cf580 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f7de40d0ce0 tx=0x7f7dd801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df19d35ab4b2883 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de27fc640 1 -- 172.21.15.157:0/2564729902 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab600 con 0x7f7de40b5f60 2023-11-06T03:18:16.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de27fc640 1 -- 172.21.15.157:0/2564729902 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80a7050 con 0x7f7de40b5f60 2023-11-06T03:18:16.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.130+0000 7f7de27fc640 1 -- 172.21.15.157:0/2564729902 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab980 con 0x7f7de40b5f60 2023-11-06T03:18:16.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2564729902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40b5f60 msgr2=0x7f7de40cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.132 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2564729902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40b5f60 0x7f7de40cf580 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f7de40d0ce0 tx=0x7f7dd801c470 comp rx=0 tx=0).stop 2023-11-06T03:18:16.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2564729902 shutdown_connections 2023-11-06T03:18:16.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2564729902 >> v2:172.21.15.195:3300/0 conn(0x7f7de40cfac0 0x7f7de40d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.133 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2564729902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40b5f60 0x7f7de40cf580 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2564729902 >> 172.21.15.157:0/2564729902 conn(0x7f7de4070990 msgr2=0x7f7de4065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:16.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2564729902 shutdown_connections 2023-11-06T03:18:16.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2564729902 wait complete. 2023-11-06T03:18:16.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 Processor -- start 2023-11-06T03:18:16.134 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- start start 2023-11-06T03:18:16.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de40b5f60 0x7f7de4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.136 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de415ce30 0x7f7de40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40cc160 con 0x7f7de40cfac0 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40cade0 con 0x7f7de40b5f60 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40cb0c0 con 0x7f7de415ce30 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.137 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41080/0 (socket says 172.21.15.157:41080) 2023-11-06T03:18:16.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2980477038 learned_addr learned my addr 172.21.15.157:0/2980477038 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:16.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de415ce30 msgr2=0x7f7de40ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de415ce30 0x7f7de40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de40b5f60 msgr2=0x7f7de4156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de40b5f60 0x7f7de4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 -- 172.21.15.157:0/2980477038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd80094d0 con 0x7f7de40cfac0 2023-11-06T03:18:16.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de2ffd640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8019080 tx=0x7f7dd8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80b3890 con 0x7f7de40cfac0 2023-11-06T03:18:16.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80b39f0 con 0x7f7de40cfac0 2023-11-06T03:18:16.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40cb3a0 con 0x7f7de40cfac0 2023-11-06T03:18:16.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80b3da0 con 0x7f7de40cfac0 2023-11-06T03:18:16.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40cb960 con 0x7f7de40cfac0 2023-11-06T03:18:16.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.134+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de4160c10 con 0x7f7de40cfac0 2023-11-06T03:18:16.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.138+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7dd80b2070 con 0x7f7de40cfac0 2023-11-06T03:18:16.140 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.138+0000 7f7dcbfff640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 0x7f7dd0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.138+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7dd811d280 con 0x7f7de40cfac0 2023-11-06T03:18:16.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.138+0000 7f7de37fe640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 0x7f7dd0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.138+0000 7f7de37fe640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 0x7f7dd0067f20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40011f0 tx=0x7f7dd4007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.142+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7de4160c10 con 0x7f7de40cfac0 2023-11-06T03:18:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.286+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7de4160c10 con 0x7f7de40cfac0 2023-11-06T03:18:16.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.286+0000 7f7dcbfff640 1 -- 172.21.15.157:0/2980477038 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v185) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7de4160c10 con 0x7f7de40cfac0 2023-11-06T03:18:16.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:16.285 INFO:teuthology.orchestra.run.smithi157.stdout:[ 2023-11-06T03:18:16.285 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:16.285 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 1, 2023-11-06T03:18:16.286 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": ".mgr" 2023-11-06T03:18:16.286 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:16.286 INFO:teuthology.orchestra.run.smithi157.stdout: { 2023-11-06T03:18:16.286 INFO:teuthology.orchestra.run.smithi157.stdout: "poolnum": 2, 2023-11-06T03:18:16.286 INFO:teuthology.orchestra.run.smithi157.stdout: "poolname": "rbd" 2023-11-06T03:18:16.287 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:16.287 INFO:teuthology.orchestra.run.smithi157.stdout:] 2023-11-06T03:18:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 msgr2=0x7f7dd0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 0x7f7dd0067f20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40011f0 tx=0x7f7dd4007930 comp rx=0 tx=0).stop 2023-11-06T03:18:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 msgr2=0x7f7de40cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.288 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8019080 tx=0x7f7dd8019210 comp rx=0 tx=0).stop 2023-11-06T03:18:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 shutdown_connections 2023-11-06T03:18:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7dd0065a60 0x7f7dd0067f20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7de415ce30 0x7f7de40ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.289 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de40cfac0 0x7f7de40cc680 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 --2- 172.21.15.157:0/2980477038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7de40b5f60 0x7f7de4156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 >> 172.21.15.157:0/2980477038 conn(0x7f7de4070990 msgr2=0x7f7de415d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 shutdown_connections 2023-11-06T03:18:16.290 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.290+0000 7f7de9d9f640 1 -- 172.21.15.157:0/2980477038 wait complete. 2023-11-06T03:18:16.302 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-06T03:18:16.303 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-06T03:18:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 Processor -- start 2023-11-06T03:18:16.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 -- start start 2023-11-06T03:18:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f18380ad780 0x7f18380cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380adf90 con 0x7f18380cd4e0 2023-11-06T03:18:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183806d570 con 0x7f18380b2de0 2023-11-06T03:18:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f183df7c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380b1940 con 0x7f18380ad780 2023-11-06T03:18:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18377fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f18380cd4e0 0x7f183815de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56964/0 (socket says 172.21.15.157:56964) 2023-11-06T03:18:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18377fe640 1 -- 172.21.15.157:0/1176858500 learned_addr learned my addr 172.21.15.157:0/1176858500 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:16.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f1836ffd640 1 --2- 172.21.15.157:0/1176858500 >> v2:172.21.15.195:3300/0 conn(0x7f18380ad780 0x7f18380cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784316387 0 0) 0x7f1828001120 con 0x7f18380cd4e0 2023-11-06T03:18:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1824003530 con 0x7f18380cd4e0 2023-11-06T03:18:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611074653 0 0) 0x7f1824003530 con 0x7f18380cd4e0 2023-11-06T03:18:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 >> v2:172.21.15.195:3300/0 conn(0x7f18380ad780 msgr2=0x7f18380cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:16.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 --2- 172.21.15.157:0/1176858500 >> v2:172.21.15.195:3300/0 conn(0x7f18380ad780 0x7f18380cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 >> v1:172.21.15.195:6789/0 conn(0x7f18380b2de0 legacy=0x7f18380b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380adcb0 con 0x7f18380cd4e0 2023-11-06T03:18:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1828001810 con 0x7f18380cd4e0 2023-11-06T03:18:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 >> v1:172.21.15.157:6789/0 conn(0x7f18380cd4e0 legacy=0x7f183815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1824003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18240047d0 0x7f1824004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18240052b0 0x7f182400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380b1940 con 0x7f1824003980 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182400c180 con 0x7f18240047d0 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.430+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182400c340 con 0x7f18240052b0 2023-11-06T03:18:16.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1824003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f1836ffd640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18240047d0 0x7f1824004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f1837fff640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18240052b0 0x7f182400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 -- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18240052b0 msgr2=0x7f182400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18240052b0 0x7f182400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 -- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18240047d0 msgr2=0x7f1824004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18240047d0 0x7f1824004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 -- 172.21.15.157:0/1176858500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824003530 con 0x7f1824003980 2023-11-06T03:18:16.436 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f1837fff640 1 -- 172.21.15.157:0/1176858500 reap_dead start 2023-11-06T03:18:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18377fe640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1824003da0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f182800c210 tx=0x7f182801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3679b6e60d14f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828005ae0 con 0x7f1824003980 2023-11-06T03:18:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828005c60 con 0x7f1824003980 2023-11-06T03:18:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f18367fc640 1 -- 172.21.15.157:0/1176858500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280a5af0 con 0x7f1824003980 2023-11-06T03:18:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 -- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 msgr2=0x7f1824003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1824003da0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f182800c210 tx=0x7f182801cbd0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 -- 172.21.15.157:0/1176858500 shutdown_connections 2023-11-06T03:18:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 --2- 172.21.15.157:0/1176858500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1824003da0 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 -- 172.21.15.157:0/1176858500 >> 172.21.15.157:0/1176858500 conn(0x7f18380b43b0 msgr2=0x7f1838065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 -- 172.21.15.157:0/1176858500 shutdown_connections 2023-11-06T03:18:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.434+0000 7f183df7c640 1 -- 172.21.15.157:0/1176858500 wait complete. 2023-11-06T03:18:16.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 Processor -- start 2023-11-06T03:18:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- start start 2023-11-06T03:18:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18380b0320 0x7f18380b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 0x7f18381525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18380b0ce0 con 0x7f1824003980 2023-11-06T03:18:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1838151bb0 con 0x7f1838152150 2023-11-06T03:18:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1838152ee0 con 0x7f18380b0320 2023-11-06T03:18:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41106/0 (socket says 172.21.15.157:41106) 2023-11-06T03:18:16.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 -- 172.21.15.157:0/2727337763 learned_addr learned my addr 172.21.15.157:0/2727337763 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18380b0320 0x7f18380b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f1837fff640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 0x7f18381525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 -- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18380b0320 msgr2=0x7f18380b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18380b0320 0x7f18380b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 -- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 msgr2=0x7f18381525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 0x7f18381525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 -- 172.21.15.157:0/2727337763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18381531c0 con 0x7f1824003980 2023-11-06T03:18:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f1837fff640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 0x7f18381525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f18377fe640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f182801c640 tx=0x7f1824005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280b4070 con 0x7f1824003980 2023-11-06T03:18:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828001610 con 0x7f1824003980 2023-11-06T03:18:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18381534a0 con 0x7f1824003980 2023-11-06T03:18:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18280a5af0 con 0x7f1824003980 2023-11-06T03:18:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838159160 con 0x7f1824003980 2023-11-06T03:18:16.446 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183804e050 con 0x7f1824003980 2023-11-06T03:18:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.438+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f18280a5c50 con 0x7f1824003980 2023-11-06T03:18:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.442+0000 7f182ffff640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 0x7f1818067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.442+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f182811c580 con 0x7f1824003980 2023-11-06T03:18:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.442+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 0x7f1818067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.442+0000 7f1836ffd640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 0x7f1818067f20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f18381552b0 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.446+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f18280ea1c0 con 0x7f1824003980 2023-11-06T03:18:16.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.598+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18380ad050 con 0x7f1824003980 2023-11-06T03:18:16.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.710+0000 7f182ffff640 1 -- 172.21.15.157:0/2727337763 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v186) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18280ed7e0 con 0x7f1824003980 2023-11-06T03:18:16.708 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' created 2023-11-06T03:18:16.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 msgr2=0x7f1818067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 0x7f1818067f20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f18381552b0 tx=0x7f1820008040 comp rx=0 tx=0).stop 2023-11-06T03:18:16.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 msgr2=0x7f1838151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f182801c640 tx=0x7f1824005800 comp rx=0 tx=0).stop 2023-11-06T03:18:16.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 shutdown_connections 2023-11-06T03:18:16.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1818065a60 0x7f1818067f20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1838152150 0x7f18381525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f18380b0320 0x7f18380b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 --2- 172.21.15.157:0/2727337763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1824003980 0x7f1838151670 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 >> 172.21.15.157:0/2727337763 conn(0x7f18380b43b0 msgr2=0x7f183806bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:16.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 shutdown_connections 2023-11-06T03:18:16.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.718+0000 7f183df7c640 1 -- 172.21.15.157:0/2727337763 wait complete. 2023-11-06T03:18:16.737 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-06T03:18:16.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 Processor -- start 2023-11-06T03:18:16.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 -- start start 2023-11-06T03:18:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0d01551e0 0x7ff0d006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d015f580 con 0x7ff0d015ea60 2023-11-06T03:18:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d015efc0 con 0x7ff0d01551e0 2023-11-06T03:18:16.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d677f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d015f2a0 con 0x7ff0d015e5e0 2023-11-06T03:18:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0d01551e0 0x7ff0d006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cf7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cf7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d00b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:56448/0 (socket says 172.21.15.157:56448) 2023-11-06T03:18:16.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cf7fe640 1 -- 172.21.15.157:0/3140290659 learned_addr learned my addr 172.21.15.157:0/3140290659 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0d4d1c640 1 --1- 172.21.15.157:0/3140290659 >> v1:172.21.15.195:6789/0 conn(0x7ff0d015ea60 0x7ff0d015c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:56760/0 (socket says 172.21.15.157:56760) 2023-11-06T03:18:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cffff640 1 -- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 msgr2=0x7ff0d00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:16.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cffff640 1 --2- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cffff640 1 -- 172.21.15.157:0/3140290659 >> v1:172.21.15.195:6789/0 conn(0x7ff0d015ea60 legacy=0x7ff0d015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.870+0000 7ff0cffff640 1 -- 172.21.15.157:0/3140290659 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d006f5a0 con 0x7ff0d01551e0 2023-11-06T03:18:16.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0cffff640 1 --2- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d006fda0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17bc18461ccb303b server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0ceffd640 1 -- 172.21.15.157:0/3140290659 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40ab600 con 0x7ff0d01551e0 2023-11-06T03:18:16.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0ceffd640 1 -- 172.21.15.157:0/3140290659 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c40a7050 con 0x7ff0d01551e0 2023-11-06T03:18:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0ceffd640 1 -- 172.21.15.157:0/3140290659 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40ab980 con 0x7ff0d01551e0 2023-11-06T03:18:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 -- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 msgr2=0x7ff0d006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 --2- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d006fda0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c3c0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 -- 172.21.15.157:0/3140290659 shutdown_connections 2023-11-06T03:18:16.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 --2- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 --2- 172.21.15.157:0/3140290659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d006fda0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 -- 172.21.15.157:0/3140290659 >> 172.21.15.157:0/3140290659 conn(0x7ff0d0065930 msgr2=0x7ff0d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 -- 172.21.15.157:0/3140290659 shutdown_connections 2023-11-06T03:18:16.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.874+0000 7ff0d677f640 1 -- 172.21.15.157:0/3140290659 wait complete. 2023-11-06T03:18:16.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 Processor -- start 2023-11-06T03:18:16.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- start start 2023-11-06T03:18:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01530d0 con 0x7ff0d015ea60 2023-11-06T03:18:16.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01533b0 con 0x7ff0d01551e0 2023-11-06T03:18:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0153690 con 0x7ff0d015e5e0 2023-11-06T03:18:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41116/0 (socket says 172.21.15.157:41116) 2023-11-06T03:18:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 -- 172.21.15.157:0/4059841387 learned_addr learned my addr 172.21.15.157:0/4059841387 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:16.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0cf7fe640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0cffff640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 -- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 msgr2=0x7ff0d014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 -- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 msgr2=0x7ff0d014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:16.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 -- 172.21.15.157:0/4059841387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009420 con 0x7ff0d015ea60 2023-11-06T03:18:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d4d1c640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c6c0 tx=0x7ff0c800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004d70 con 0x7ff0d015ea60 2023-11-06T03:18:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c800a640 con 0x7ff0d015ea60 2023-11-06T03:18:16.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800a9f0 con 0x7ff0d015ea60 2023-11-06T03:18:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d006c540 con 0x7ff0d015ea60 2023-11-06T03:18:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d0152c60 con 0x7ff0d015ea60 2023-11-06T03:18:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0072750 con 0x7ff0d015ea60 2023-11-06T03:18:16.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.878+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0c800ab50 con 0x7ff0d015ea60 2023-11-06T03:18:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.882+0000 7ff0ccff9640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 0x7ff0bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.882+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5580+0+0 (secure 0 0 0) 0x7ff0c80828f0 con 0x7ff0d015ea60 2023-11-06T03:18:16.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.882+0000 7ff0cffff640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 0x7ff0bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.882+0000 7ff0cffff640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 0x7ff0bc067f20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ad000 tx=0x7ff0c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:16.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:16.886+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0c8050430 con 0x7ff0d015ea60 2023-11-06T03:18:17.038 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.038+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff0d006c9f0 con 0x7ff0d015ea60 2023-11-06T03:18:17.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.714+0000 7ff0ccff9640 1 -- 172.21.15.157:0/4059841387 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v187) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff0c8018020 con 0x7ff0d015ea60 2023-11-06T03:18:17.716 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' created 2023-11-06T03:18:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.726+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 msgr2=0x7ff0bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.726+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 0x7ff0bc067f20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ad000 tx=0x7ff0c4008040 comp rx=0 tx=0).stop 2023-11-06T03:18:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.726+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 msgr2=0x7ff0d006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.726+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c6c0 tx=0x7ff0c800cb80 comp rx=0 tx=0).stop 2023-11-06T03:18:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 shutdown_connections 2023-11-06T03:18:17.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0bc065a60 0x7ff0bc067f20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0d015ea60 0x7ff0d006cd40 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0d015e5e0 0x7ff0d014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 --2- 172.21.15.157:0/4059841387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0d01551e0 0x7ff0d014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 >> 172.21.15.157:0/4059841387 conn(0x7ff0d0065930 msgr2=0x7ff0d00b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:17.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 shutdown_connections 2023-11-06T03:18:17.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.734+0000 7ff0d677f640 1 -- 172.21.15.157:0/4059841387 wait complete. 2023-11-06T03:18:17.760 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-06T03:18:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.886+0000 7f3303336640 1 Processor -- start 2023-11-06T03:18:17.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f3303336640 1 -- start start 2023-11-06T03:18:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f3303336640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32fc167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f3303336640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0cecf0 con 0x7f32fc0ce860 2023-11-06T03:18:17.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f3303336640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0aca60 con 0x7f32fc0b0e90 2023-11-06T03:18:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f3303336640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0ad820 con 0x7f32fc0ae170 2023-11-06T03:18:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33008d1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f32fc0ce860 0x7f32fc0ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:56968/0 (socket says 172.21.15.157:56968) 2023-11-06T03:18:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33008d1640 1 -- 172.21.15.157:0/3950791826 learned_addr learned my addr 172.21.15.157:0/3950791826 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32fc167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439008066 0 0) 0x7f32f8001120 con 0x7f32fc0ce860 2023-11-06T03:18:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e8003530 con 0x7f32fc0ce860 2023-11-06T03:18:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318043833 0 0) 0x7f32e8003530 con 0x7f32fc0ce860 2023-11-06T03:18:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 msgr2=0x7f32fc167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32fc167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 >> v1:172.21.15.195:6789/0 conn(0x7f32fc0b0e90 legacy=0x7f32fc06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0acd40 con 0x7f32fc0ce860 2023-11-06T03:18:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f32f8001810 con 0x7f32fc0ce860 2023-11-06T03:18:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 >> v1:172.21.15.157:6789/0 conn(0x7f32fc0ce860 legacy=0x7f32fc0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32e8003f30 0x7f32e80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e8004d10 0x7f32e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0ad820 con 0x7f32e8004d10 2023-11-06T03:18:17.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e800bb20 con 0x7f32e8003f30 2023-11-06T03:18:17.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e800bce0 con 0x7f32fc0ae170 2023-11-06T03:18:17.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e8004d10 0x7f32e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33008d1640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32e8003f30 0x7f32e80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33010d2640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 -- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 msgr2=0x7f32e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32fc0ae170 0x7f32e80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 -- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32e8003f30 msgr2=0x7f32e80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:17.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32e8003f30 0x7f32e80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.890+0000 7f33018d3640 1 -- 172.21.15.157:0/3950791826 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8003530 con 0x7f32e8004d10 2023-11-06T03:18:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e8004d10 0x7f32e8005190 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ccf60 tx=0x7f32f401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78d21e0c725be1a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:17.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f40a5c10 con 0x7f32e8004d10 2023-11-06T03:18:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f400f040 con 0x7f32e8004d10 2023-11-06T03:18:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f32effff640 1 -- 172.21.15.157:0/3950791826 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f40ad440 con 0x7f32e8004d10 2023-11-06T03:18:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- 172.21.15.157:0/3950791826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e8004d10 msgr2=0x7f32e8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 --2- 172.21.15.157:0/3950791826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32e8004d10 0x7f32e8005190 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0ccf60 tx=0x7f32f401cc00 comp rx=0 tx=0).stop 2023-11-06T03:18:17.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 -- 172.21.15.157:0/3950791826 reap_dead start 2023-11-06T03:18:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- 172.21.15.157:0/3950791826 shutdown_connections 2023-11-06T03:18:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- 172.21.15.157:0/3950791826 >> 172.21.15.157:0/3950791826 conn(0x7f32fc0cb9b0 msgr2=0x7f32fc0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- 172.21.15.157:0/3950791826 shutdown_connections 2023-11-06T03:18:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- 172.21.15.157:0/3950791826 wait complete. 2023-11-06T03:18:17.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 Processor -- start 2023-11-06T03:18:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- start start 2023-11-06T03:18:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 0x7f32fc1f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 0x7f32fc0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc14ec70 con 0x7f32fc15f560 2023-11-06T03:18:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1f22a0 con 0x7f32fc15df70 2023-11-06T03:18:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f3303336640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc15e4c0 con 0x7f32e8004d10 2023-11-06T03:18:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41136/0 (socket says 172.21.15.157:41136) 2023-11-06T03:18:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 -- 172.21.15.157:0/2884967514 learned_addr learned my addr 172.21.15.157:0/2884967514 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33010d2640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 0x7f32fc1f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33008d1640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 0x7f32fc0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 -- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 msgr2=0x7f32fc1f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 0x7f32fc1f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 -- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 msgr2=0x7f32fc0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 0x7f32fc0b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.894+0000 7f33018d3640 1 -- 172.21.15.157:0/2884967514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc15e7a0 con 0x7f32fc15f560 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f33010d2640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 0x7f32fc1f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f33008d1640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 0x7f32fc0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:17.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f33018d3640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f32f4002240 tx=0x7f32f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f40039a0 con 0x7f32fc15f560 2023-11-06T03:18:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4003b00 con 0x7f32fc15f560 2023-11-06T03:18:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc15ea80 con 0x7f32fc15f560 2023-11-06T03:18:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4003e70 con 0x7f32fc15f560 2023-11-06T03:18:17.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc15f040 con 0x7f32fc15f560 2023-11-06T03:18:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc15f230 con 0x7f32fc15f560 2023-11-06T03:18:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f32f40b4040 con 0x7f32fc15f560 2023-11-06T03:18:17.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 0x7f32e0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:17.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f32f411d980 con 0x7f32fc15f560 2023-11-06T03:18:17.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f33010d2640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 0x7f32e0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.898+0000 7f33010d2640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 0x7f32e0067f50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000c10 tx=0x7f32f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:17.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:17.902+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f32f40eb4f0 con 0x7f32fc15f560 2023-11-06T03:18:18.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.090+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f32fc072670 con 0x7f32fc15f560 2023-11-06T03:18:18.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.106+0000 7f32edffb640 1 -- 172.21.15.157:0/2884967514 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v128) v1 ==== 174+0+0 (secure 0 0 0) 0x7f32f40eeb10 con 0x7f32fc15f560 2023-11-06T03:18:18.106 INFO:teuthology.orchestra.run.smithi157.stderr:new fs with metadata pool 27 and data pool 28 2023-11-06T03:18:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 msgr2=0x7f32e0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 0x7f32e0067f50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000c10 tx=0x7f32f0009040 comp rx=0 tx=0).stop 2023-11-06T03:18:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 msgr2=0x7f32fc0ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f32f4002240 tx=0x7f32f4009750 comp rx=0 tx=0).stop 2023-11-06T03:18:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 shutdown_connections 2023-11-06T03:18:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f32e0065a90 0x7f32e0067f50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f32fc15f560 0x7f32fc0ca820 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32fc15df70 0x7f32fc0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 --2- 172.21.15.157:0/2884967514 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f32e8004d10 0x7f32fc1f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.110+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 >> 172.21.15.157:0/2884967514 conn(0x7f32fc0cb9b0 msgr2=0x7f32fc0cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.114+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 shutdown_connections 2023-11-06T03:18:18.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.114+0000 7f3303336640 1 -- 172.21.15.157:0/2884967514 wait complete. 2023-11-06T03:18:18.138 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:18.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 Processor -- start 2023-11-06T03:18:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 -- start start 2023-11-06T03:18:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63841551e0 0x7f638406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 0x7f63840b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 0x7f63840b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638415f580 con 0x7f63841551e0 2023-11-06T03:18:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638415efc0 con 0x7f638415e5e0 2023-11-06T03:18:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.258+0000 7f6388b34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638415f2a0 con 0x7f638415ea90 2023-11-06T03:18:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63841551e0 0x7f638406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f63841551e0 0x7f638406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41146/0 (socket says 172.21.15.157:41146) 2023-11-06T03:18:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 -- 172.21.15.157:0/221820107 learned_addr learned my addr 172.21.15.157:0/221820107 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 -- 172.21.15.157:0/221820107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 msgr2=0x7f63840b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6382d9d640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 0x7f63840b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6381d9b640 1 --2- 172.21.15.157:0/221820107 >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 0x7f63840b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 0x7f63840b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 -- 172.21.15.157:0/221820107 >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 msgr2=0x7f63840b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 --2- 172.21.15.157:0/221820107 >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 0x7f63840b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 -- 172.21.15.157:0/221820107 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638406f5a0 con 0x7f63841551e0 2023-11-06T03:18:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6382d9d640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 0x7f63840b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:18.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6381d9b640 1 --2- 172.21.15.157:0/221820107 >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 0x7f63840b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638259c640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63841551e0 0x7f638406fda0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f6384066880 tx=0x7f637401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8593f75fd45db4aa server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638159a640 1 -- 172.21.15.157:0/221820107 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740ab620 con 0x7f63841551e0 2023-11-06T03:18:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638159a640 1 -- 172.21.15.157:0/221820107 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63740ab780 con 0x7f63841551e0 2023-11-06T03:18:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f638159a640 1 -- 172.21.15.157:0/221820107 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740abb10 con 0x7f63841551e0 2023-11-06T03:18:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 -- 172.21.15.157:0/221820107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63841551e0 msgr2=0x7f638406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63841551e0 0x7f638406fda0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f6384066880 tx=0x7f637401c440 comp rx=0 tx=0).stop 2023-11-06T03:18:18.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 -- 172.21.15.157:0/221820107 shutdown_connections 2023-11-06T03:18:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415ea90 0x7f63840b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 --2- 172.21.15.157:0/221820107 >> v2:172.21.15.195:3300/0 conn(0x7f638415e5e0 0x7f63840b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 --2- 172.21.15.157:0/221820107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63841551e0 0x7f638406fda0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 -- 172.21.15.157:0/221820107 >> 172.21.15.157:0/221820107 conn(0x7f6384065930 msgr2=0x7f6384065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 -- 172.21.15.157:0/221820107 shutdown_connections 2023-11-06T03:18:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.262+0000 7f6388b34640 1 -- 172.21.15.157:0/221820107 wait complete. 2023-11-06T03:18:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 Processor -- start 2023-11-06T03:18:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- start start 2023-11-06T03:18:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415e5e0 0x7f63840cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f638415ea90 0x7f6384142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840cd140 con 0x7f63840cdf20 2023-11-06T03:18:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840ccac0 con 0x7f638415ea90 2023-11-06T03:18:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63840ce9c0 con 0x7f638415e5e0 2023-11-06T03:18:18.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:41148/0 (socket says 172.21.15.157:41148) 2023-11-06T03:18:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 -- 172.21.15.157:0/4176910669 learned_addr learned my addr 172.21.15.157:0/4176910669 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6381d9b640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f638415ea90 0x7f6384142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f638259c640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415e5e0 0x7f63840cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 -- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415e5e0 msgr2=0x7f63840cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415e5e0 0x7f63840cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 -- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f638415ea90 msgr2=0x7f6384142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f638415ea90 0x7f6384142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 -- 172.21.15.157:0/4176910669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63740094d0 con 0x7f63840cdf20 2023-11-06T03:18:18.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6382d9d640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f637c00cd50 tx=0x7f637c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c00f680 con 0x7f63840cdf20 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c00f800 con 0x7f63840cdf20 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63840ccd60 con 0x7f63840cdf20 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c00fbb0 con 0x7f63840cdf20 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6384144850 con 0x7f63840cdf20 2023-11-06T03:18:18.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63840cd280 con 0x7f63840cdf20 2023-11-06T03:18:18.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.266+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f637c00b040 con 0x7f63840cdf20 2023-11-06T03:18:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.270+0000 7f6372ffd640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 0x7f6364067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.270+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f637c083c30 con 0x7f63840cdf20 2023-11-06T03:18:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.270+0000 7f638259c640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 0x7f6364067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.270+0000 7f638259c640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 0x7f6364067f20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f63740af000 tx=0x7f63740a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.274+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f637c051520 con 0x7f63840cdf20 2023-11-06T03:18:18.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.430+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6384072750 con 0x7f63840cdf20 2023-11-06T03:18:18.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.430+0000 7f6372ffd640 1 -- 172.21.15.157:0/4176910669 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f637c01d090 con 0x7f63840cdf20 2023-11-06T03:18:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:18.430 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":188,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:18.096733+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:18.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 msgr2=0x7f6364067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 0x7f6364067f20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f63740af000 tx=0x7f63740a7000 comp rx=0 tx=0).stop 2023-11-06T03:18:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 msgr2=0x7f63840ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f637c00cd50 tx=0x7f637c00a650 comp rx=0 tx=0).stop 2023-11-06T03:18:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 shutdown_connections 2023-11-06T03:18:18.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6364065a60 0x7f6364067f20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f63840cdf20 0x7f63840ce3d0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f638415ea90 0x7f6384142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 --2- 172.21.15.157:0/4176910669 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f638415e5e0 0x7f63840cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 >> 172.21.15.157:0/4176910669 conn(0x7f6384065930 msgr2=0x7f63840aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 shutdown_connections 2023-11-06T03:18:18.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.434+0000 7f6388b34640 1 -- 172.21.15.157:0/4176910669 wait complete. 2023-11-06T03:18:18.450 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.566+0000 7f619e0a6640 1 Processor -- start 2023-11-06T03:18:18.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.566+0000 7f619e0a6640 1 -- start start 2023-11-06T03:18:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f619e0a6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 0x7f61980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f619e0a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f619e0a6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619815f580 con 0x7f61981551e0 2023-11-06T03:18:18.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f619e0a6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61980704b0 con 0x7f619815e5e0 2023-11-06T03:18:18.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f619e0a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619815efc0 con 0x7f619815ea90 2023-11-06T03:18:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6196ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 0x7f61980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43502/0 (socket says 172.21.15.157:43502) 2023-11-06T03:18:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 -- 172.21.15.157:0/2082111712 learned_addr learned my addr 172.21.15.157:0/2082111712 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 -- 172.21.15.157:0/2082111712 >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 msgr2=0x7f61980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 --2- 172.21.15.157:0/2082111712 >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 0x7f61980b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 -- 172.21.15.157:0/2082111712 >> v1:172.21.15.195:6789/0 conn(0x7f61981551e0 legacy=0x7f619806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 -- 172.21.15.157:0/2082111712 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619815f2a0 con 0x7f619815ea90 2023-11-06T03:18:18.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6196ffd640 1 --2- 172.21.15.157:0/2082111712 >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 0x7f61980b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:18:18.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.570+0000 7f6197fff640 1 --2- 172.21.15.157:0/2082111712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5675ecbe563d49da server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f61967fc640 1 -- 172.21.15.157:0/2082111712 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900a5810 con 0x7f619815ea90 2023-11-06T03:18:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f61967fc640 1 -- 172.21.15.157:0/2082111712 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61900a5970 con 0x7f619815ea90 2023-11-06T03:18:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f61967fc640 1 -- 172.21.15.157:0/2082111712 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900a5d00 con 0x7f619815ea90 2023-11-06T03:18:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/2082111712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 msgr2=0x7f61980b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- 172.21.15.157:0/2082111712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619001c950 comp rx=0 tx=0).stop 2023-11-06T03:18:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/2082111712 shutdown_connections 2023-11-06T03:18:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- 172.21.15.157:0/2082111712 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f619815ea90 0x7f61980b3950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- 172.21.15.157:0/2082111712 >> v2:172.21.15.195:3300/0 conn(0x7f619815e5e0 0x7f61980b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/2082111712 >> 172.21.15.157:0/2082111712 conn(0x7f6198065930 msgr2=0x7f6198065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/2082111712 shutdown_connections 2023-11-06T03:18:18.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/2082111712 wait complete. 2023-11-06T03:18:18.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 Processor -- start 2023-11-06T03:18:18.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- start start 2023-11-06T03:18:18.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61981551e0 0x7f6198150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f619815ea90 0x7f619815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61981344e0 con 0x7f619815e5e0 2023-11-06T03:18:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6198136630 con 0x7f619815ea90 2023-11-06T03:18:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61981501b0 con 0x7f61981551e0 2023-11-06T03:18:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36270/0 (socket says 172.21.15.157:36270) 2023-11-06T03:18:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 -- 172.21.15.157:0/1412469573 learned_addr learned my addr 172.21.15.157:0/1412469573 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f61977fe640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61981551e0 0x7f6198150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6197fff640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f619815ea90 0x7f619815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 -- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61981551e0 msgr2=0x7f6198150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61981551e0 0x7f6198150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 -- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f619815ea90 msgr2=0x7f619815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f619815ea90 0x7f619815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 -- 172.21.15.157:0/1412469573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009420 con 0x7f619815e5e0 2023-11-06T03:18:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f6196ffd640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f618000ebe0 tx=0x7f618000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000eee0 con 0x7f619815e5e0 2023-11-06T03:18:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000cc30 con 0x7f619815e5e0 2023-11-06T03:18:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180013470 con 0x7f619815e5e0 2023-11-06T03:18:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6190009b30 con 0x7f619815e5e0 2023-11-06T03:18:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.574+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6198150770 con 0x7f619815e5e0 2023-11-06T03:18:18.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6198072750 con 0x7f619815e5e0 2023-11-06T03:18:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f61800135d0 con 0x7f619815e5e0 2023-11-06T03:18:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f618ffff640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 0x7f6178067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61800835a0 con 0x7f619815e5e0 2023-11-06T03:18:18.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f61977fe640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 0x7f6178067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.578+0000 7f61977fe640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 0x7f6178067f50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6188007fd0 tx=0x7f6188009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.582+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6180050e20 con 0x7f619815e5e0 2023-11-06T03:18:18.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.738+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6198071730 con 0x7f619815e5e0 2023-11-06T03:18:18.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.738+0000 7f618ffff640 1 -- 172.21.15.157:0/1412469573 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17954 (secure 0 0 0) 0x7f6180054440 con 0x7f619815e5e0 2023-11-06T03:18:18.738 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:18.738 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":188,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:18.096733+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:18.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 msgr2=0x7f6178067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 0x7f6178067f50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6188007fd0 tx=0x7f6188009040 comp rx=0 tx=0).stop 2023-11-06T03:18:18.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 msgr2=0x7f61981483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f618000ebe0 tx=0x7f618000c520 comp rx=0 tx=0).stop 2023-11-06T03:18:18.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 shutdown_connections 2023-11-06T03:18:18.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6178065a90 0x7f6178067f50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f619815ea90 0x7f619815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f619815e5e0 0x7f61981483b0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 --2- 172.21.15.157:0/1412469573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f61981551e0 0x7f6198150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 >> 172.21.15.157:0/1412469573 conn(0x7f6198065930 msgr2=0x7f61980b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 shutdown_connections 2023-11-06T03:18:18.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.742+0000 7f619e0a6640 1 -- 172.21.15.157:0/1412469573 wait complete. 2023-11-06T03:18:18.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-06T03:18:18.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 Processor -- start 2023-11-06T03:18:18.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 -- start start 2023-11-06T03:18:18.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb281599c0 0x7ffb280ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2815e440 con 0x7ffb280b2af0 2023-11-06T03:18:18.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2806c240 con 0x7ffb2806d000 2023-11-06T03:18:18.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2e8f1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2806c520 con 0x7ffb281599c0 2023-11-06T03:18:18.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb281599c0 0x7ffb280ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffb281599c0 0x7ffb280ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36276/0 (socket says 172.21.15.157:36276) 2023-11-06T03:18:18.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb2ce8e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffb280b2af0 0x7ffb28159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38538/0 (socket says 172.21.15.157:38538) 2023-11-06T03:18:18.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 -- 172.21.15.157:0/3883213935 learned_addr learned my addr 172.21.15.157:0/3883213935 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 -- 172.21.15.157:0/3883213935 >> v1:172.21.15.195:6789/0 conn(0x7ffb2806d000 legacy=0x7ffb280b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:18.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 -- 172.21.15.157:0/3883213935 >> v1:172.21.15.157:6789/0 conn(0x7ffb280b2af0 legacy=0x7ffb28159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 -- 172.21.15.157:0/3883213935 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2806c800 con 0x7ffb281599c0 2023-11-06T03:18:18.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb277fe640 1 -- 172.21.15.157:0/3883213935 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349117872 0 0) 0x7ffb2815e440 con 0x7ffb280b2af0 2023-11-06T03:18:18.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb27fff640 1 --2- 172.21.15.157:0/3883213935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb281599c0 0x7ffb280ae000 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009e50 tx=0x7ffb1801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdb847a0a427106 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb277fe640 1 -- 172.21.15.157:0/3883213935 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180a5880 con 0x7ffb281599c0 2023-11-06T03:18:18.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb277fe640 1 -- 172.21.15.157:0/3883213935 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180a59e0 con 0x7ffb281599c0 2023-11-06T03:18:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.882+0000 7ffb277fe640 1 -- 172.21.15.157:0/3883213935 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180a5d70 con 0x7ffb281599c0 2023-11-06T03:18:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/3883213935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb281599c0 msgr2=0x7ffb280ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/3883213935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb281599c0 0x7ffb280ae000 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009e50 tx=0x7ffb1801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/3883213935 shutdown_connections 2023-11-06T03:18:18.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/3883213935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb281599c0 0x7ffb280ae000 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/3883213935 >> 172.21.15.157:0/3883213935 conn(0x7ffb28065930 msgr2=0x7ffb28065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/3883213935 shutdown_connections 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/3883213935 wait complete. 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 Processor -- start 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- start start 2023-11-06T03:18:18.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2806d000 0x7ffb280acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 0x7ffb281f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280af220 con 0x7ffb280b2af0 2023-11-06T03:18:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280ae6a0 con 0x7ffb281599c0 2023-11-06T03:18:18.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280ae980 con 0x7ffb2806d000 2023-11-06T03:18:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36284/0 (socket says 172.21.15.157:36284) 2023-11-06T03:18:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/517762281 learned_addr learned my addr 172.21.15.157:0/517762281 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2ce8e640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 0x7ffb281f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb27fff640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2806d000 0x7ffb280acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2806d000 msgr2=0x7ffb280acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2806d000 0x7ffb280acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 msgr2=0x7ffb281f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 0x7ffb281f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:18.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 -- 172.21.15.157:0/517762281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009b30 con 0x7ffb280b2af0 2023-11-06T03:18:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2ce8e640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 0x7ffb281f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb1f7fe640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c450 tx=0x7ffb1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10012830 con 0x7ffb280b2af0 2023-11-06T03:18:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100129b0 con 0x7ffb280b2af0 2023-11-06T03:18:18.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb280aec60 con 0x7ffb280b2af0 2023-11-06T03:18:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10012d60 con 0x7ffb280b2af0 2023-11-06T03:18:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.886+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb280ad050 con 0x7ffb280b2af0 2023-11-06T03:18:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb28072750 con 0x7ffb280b2af0 2023-11-06T03:18:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffb1001c460 con 0x7ffb280b2af0 2023-11-06T03:18:18.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb257fa640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 0x7ffb08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb10083c30 con 0x7ffb280b2af0 2023-11-06T03:18:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb27fff640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 0x7ffb08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.890+0000 7ffb27fff640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 0x7ffb08067f20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ffb180ae000 tx=0x7ffb180a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:18.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:18.894+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ffb10051580 con 0x7ffb280b2af0 2023-11-06T03:18:19.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.082+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb280cb110 con 0x7ffb280b2af0 2023-11-06T03:18:19.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.102+0000 7ffb257fa640 1 -- 172.21.15.157:0/517762281 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v129) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb10054ba0 con 0x7ffb280b2af0 2023-11-06T03:18:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 msgr2=0x7ffb08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 0x7ffb08067f20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ffb180ae000 tx=0x7ffb180a7070 comp rx=0 tx=0).stop 2023-11-06T03:18:19.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 msgr2=0x7ffb280cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c450 tx=0x7ffb1000c910 comp rx=0 tx=0).stop 2023-11-06T03:18:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 shutdown_connections 2023-11-06T03:18:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ffb08065a60 0x7ffb08067f20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffb281599c0 0x7ffb281f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb280b2af0 0x7ffb280cea60 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 --2- 172.21.15.157:0/517762281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffb2806d000 0x7ffb280acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 >> 172.21.15.157:0/517762281 conn(0x7ffb28065930 msgr2=0x7ffb2806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 shutdown_connections 2023-11-06T03:18:19.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.106+0000 7ffb2e8f1640 1 -- 172.21.15.157:0/517762281 wait complete. 2023-11-06T03:18:19.120 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-06T03:18:19.121 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-06T03:18:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 Processor -- start 2023-11-06T03:18:19.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 -- start start 2023-11-06T03:18:19.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d801551e0 0x7f7d8006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 0x7f7d8015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8015f580 con 0x7f7d8015e5e0 2023-11-06T03:18:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8015efc0 con 0x7f7d801551e0 2023-11-06T03:18:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d86b0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8015f2a0 con 0x7f7d800b3b20 2023-11-06T03:18:19.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d801551e0 0x7f7d8006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d801551e0 0x7f7d8006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36300/0 (socket says 172.21.15.157:36300) 2023-11-06T03:18:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 -- 172.21.15.157:0/1374442309 learned_addr learned my addr 172.21.15.157:0/1374442309 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d850ab640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 0x7f7d8015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 -- 172.21.15.157:0/1374442309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 msgr2=0x7f7d8015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:19.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 0x7f7d8015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 -- 172.21.15.157:0/1374442309 >> v1:172.21.15.195:6789/0 conn(0x7f7d8015e5e0 legacy=0x7f7d800b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d848aa640 1 -- 172.21.15.157:0/1374442309 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8006f5a0 con 0x7f7d801551e0 2023-11-06T03:18:19.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.246+0000 7f7d850ab640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 0x7f7d8015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d848aa640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d801551e0 0x7f7d8006fda0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009e50 tx=0x7f7d7801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec455599c6e7db3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:19.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d7f7fe640 1 -- 172.21.15.157:0/1374442309 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a57b0 con 0x7f7d801551e0 2023-11-06T03:18:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d7f7fe640 1 -- 172.21.15.157:0/1374442309 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d780a5910 con 0x7f7d801551e0 2023-11-06T03:18:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d7f7fe640 1 -- 172.21.15.157:0/1374442309 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a5ca0 con 0x7f7d801551e0 2023-11-06T03:18:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- 172.21.15.157:0/1374442309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d801551e0 msgr2=0x7f7d8006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d801551e0 0x7f7d8006fda0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009e50 tx=0x7f7d7801c910 comp rx=0 tx=0).stop 2023-11-06T03:18:19.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- 172.21.15.157:0/1374442309 shutdown_connections 2023-11-06T03:18:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800b3b20 0x7f7d8015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/1374442309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d801551e0 0x7f7d8006fda0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- 172.21.15.157:0/1374442309 >> 172.21.15.157:0/1374442309 conn(0x7f7d80065930 msgr2=0x7f7d80065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- 172.21.15.157:0/1374442309 shutdown_connections 2023-11-06T03:18:19.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- 172.21.15.157:0/1374442309 wait complete. 2023-11-06T03:18:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 Processor -- start 2023-11-06T03:18:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- start start 2023-11-06T03:18:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 0x7f7d80149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d801551e0 0x7f7d80141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80141690 con 0x7f7d8015e5e0 2023-11-06T03:18:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80141970 con 0x7f7d800b3b20 2023-11-06T03:18:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d86b0e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80141c50 con 0x7f7d801551e0 2023-11-06T03:18:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d850ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d850ab640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36312/0 (socket says 172.21.15.157:36312) 2023-11-06T03:18:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.250+0000 7f7d850ab640 1 -- 172.21.15.157:0/3094938762 learned_addr learned my addr 172.21.15.157:0/3094938762 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d848aa640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 0x7f7d80149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7ffff640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d801551e0 0x7f7d80141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 -- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d801551e0 msgr2=0x7f7d80141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:19.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d801551e0 0x7f7d80141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 -- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 msgr2=0x7f7d80149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 0x7f7d80149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 -- 172.21.15.157:0/3094938762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009b30 con 0x7f7d8015e5e0 2023-11-06T03:18:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d848aa640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 0x7f7d80149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:19.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d850ab640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c7f0 tx=0x7f7d7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70007820 con 0x7f7d8015e5e0 2023-11-06T03:18:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d70007980 con 0x7f7d8015e5e0 2023-11-06T03:18:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8014ce70 con 0x7f7d8015e5e0 2023-11-06T03:18:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70007d70 con 0x7f7d8015e5e0 2023-11-06T03:18:19.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d80142470 con 0x7f7d8015e5e0 2023-11-06T03:18:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d80072750 con 0x7f7d8015e5e0 2023-11-06T03:18:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d700113f0 con 0x7f7d8015e5e0 2023-11-06T03:18:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d70083450 con 0x7f7d8015e5e0 2023-11-06T03:18:19.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d848aa640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.254+0000 7f7d848aa640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d780a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:19.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.258+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d70050cb0 con 0x7f7d8015e5e0 2023-11-06T03:18:19.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:19.450+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7d80071730 con 0x7f7d8015e5e0 2023-11-06T03:18:20.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.114+0000 7f7d7d7fa640 1 -- 172.21.15.157:0/3094938762 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v130) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d700542d0 con 0x7f7d8015e5e0 2023-11-06T03:18:20.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 msgr2=0x7f7d6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d780a7070 comp rx=0 tx=0).stop 2023-11-06T03:18:20.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 msgr2=0x7f7d8015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c7f0 tx=0x7f7d7000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 shutdown_connections 2023-11-06T03:18:20.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8015e5e0 0x7f7d8015d090 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d801551e0 0x7f7d80141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 --2- 172.21.15.157:0/3094938762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d800b3b20 0x7f7d80149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 >> 172.21.15.157:0/3094938762 conn(0x7f7d80065930 msgr2=0x7f7d80151600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 shutdown_connections 2023-11-06T03:18:20.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.118+0000 7f7d86b0e640 1 -- 172.21.15.157:0/3094938762 wait complete. 2023-11-06T03:18:20.135 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-06T03:18:20.136 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.262+0000 7ff66eb1f640 1 Processor -- start 2023-11-06T03:18:20.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.262+0000 7ff66eb1f640 1 -- start start 2023-11-06T03:18:20.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff6680cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff6680d0280 0x7ff6680b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680d2920 con 0x7ff66815ce80 2023-11-06T03:18:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6680d18a0 con 0x7ff6680d0280 2023-11-06T03:18:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680d1b60 con 0x7ff66815caa0 2023-11-06T03:18:20.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff667fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ff66815ce80 0x7ff6680cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38542/0 (socket says 172.21.15.157:38542) 2023-11-06T03:18:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff667fff640 1 -- 172.21.15.157:0/1937287594 learned_addr learned my addr 172.21.15.157:0/1937287594 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1937287594 >> v2:172.21.15.157:3300/0 conn(0x7ff6680d0280 0x7ff6680b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66c8bb640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff6680cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff6677fe640 1 -- 172.21.15.157:0/1937287594 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070290324 0 0) 0x7ff6680d2920 con 0x7ff66815ce80 2023-11-06T03:18:20.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1937287594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 msgr2=0x7ff6680cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff6680cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1937287594 >> v1:172.21.15.157:6789/0 conn(0x7ff66815ce80 legacy=0x7ff6680cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1937287594 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680d1e40 con 0x7ff6680d0280 2023-11-06T03:18:20.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6680d0280 0x7ff6680b5d50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ff660009d90 tx=0x7ff66001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8b9eac08909edf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff6677fe640 1 -- 172.21.15.157:0/1937287594 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600a59e0 con 0x7ff6680d0280 2023-11-06T03:18:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff6677fe640 1 -- 172.21.15.157:0/1937287594 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6600a5b40 con 0x7ff6680d0280 2023-11-06T03:18:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff6677fe640 1 -- 172.21.15.157:0/1937287594 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600ad440 con 0x7ff6680d0280 2023-11-06T03:18:20.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1937287594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6680d0280 msgr2=0x7ff6680b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6680d0280 0x7ff6680b5d50 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7ff660009d90 tx=0x7ff66001c920 comp rx=0 tx=0).stop 2023-11-06T03:18:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1937287594 shutdown_connections 2023-11-06T03:18:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6680d0280 0x7ff6680b5d50 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1937287594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff6680cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.266+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1937287594 >> 172.21.15.157:0/1937287594 conn(0x7ff668070790 msgr2=0x7ff668070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1937287594 shutdown_connections 2023-11-06T03:18:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1937287594 wait complete. 2023-11-06T03:18:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 Processor -- start 2023-11-06T03:18:20.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- start start 2023-11-06T03:18:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6680d0280 0x7ff6680b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff66814be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66815e3d0 con 0x7ff66815ce80 2023-11-06T03:18:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66815d7c0 con 0x7ff6680d0280 2023-11-06T03:18:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66815daa0 con 0x7ff66815caa0 2023-11-06T03:18:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36324/0 (socket says 172.21.15.157:36324) 2023-11-06T03:18:20.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1918143999 learned_addr learned my addr 172.21.15.157:0/1918143999 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff667fff640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff66814be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66c8bb640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6680d0280 0x7ff6680b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 msgr2=0x7ff66814be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff66814be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6680d0280 msgr2=0x7ff6680b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6680d0280 0x7ff6680b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 -- 172.21.15.157:0/1918143999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660009a70 con 0x7ff66815ce80 2023-11-06T03:18:20.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff667fff640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff66814be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:20.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66d0bc640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff66001a550 tx=0x7ff6600196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600a59e0 con 0x7ff66815ce80 2023-11-06T03:18:20.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660009450 con 0x7ff66815ce80 2023-11-06T03:18:20.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66815dd80 con 0x7ff66815ce80 2023-11-06T03:18:20.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660003ba0 con 0x7ff66815ce80 2023-11-06T03:18:20.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6680b3a80 con 0x7ff66815ce80 2023-11-06T03:18:20.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.270+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680b3cd0 con 0x7ff66815ce80 2023-11-06T03:18:20.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.274+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff6600b4050 con 0x7ff66815ce80 2023-11-06T03:18:20.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.274+0000 7ff6657fa640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 0x7ff654067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.274+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff66011d9e0 con 0x7ff66815ce80 2023-11-06T03:18:20.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.274+0000 7ff66c8bb640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 0x7ff654067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.274+0000 7ff66c8bb640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 0x7ff654067f50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff6681556e0 tx=0x7ff658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.278+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff6680b3cd0 con 0x7ff66815ce80 2023-11-06T03:18:20.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.462+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66815e580 con 0x7ff66815ce80 2023-11-06T03:18:20.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff6657fa640 1 -- 172.21.15.157:0/1918143999 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4669 (secure 0 0 0) 0x7ff66815e580 con 0x7ff66815ce80 2023-11-06T03:18:20.462 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:20.462 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:20.111160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11244":{"gid":11244,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/877366447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":877366447},{"type":"v1","addr":"172.21.15.157:6835","nonce":877366447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 130 2023-11-06T03:18:20.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 msgr2=0x7ff654067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 0x7ff654067f50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff6681556e0 tx=0x7ff658008040 comp rx=0 tx=0).stop 2023-11-06T03:18:20.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 msgr2=0x7ff66815a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7ff66001a550 tx=0x7ff6600196c0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 shutdown_connections 2023-11-06T03:18:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff654065a90 0x7ff654067f50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff66815ce80 0x7ff66815a250 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff66815caa0 0x7ff66814be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 --2- 172.21.15.157:0/1918143999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff6680d0280 0x7ff6680b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 >> 172.21.15.157:0/1918143999 conn(0x7ff668070790 msgr2=0x7ff668159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 shutdown_connections 2023-11-06T03:18:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.466+0000 7ff66eb1f640 1 -- 172.21.15.157:0/1918143999 wait complete. 2023-11-06T03:18:20.480 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:20.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 Processor -- start 2023-11-06T03:18:20.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 -- start start 2023-11-06T03:18:20.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbccc160db0 0x7fbccc161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 0x7fbccc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 0x7fbccc0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc06f470 con 0x7fbccc160db0 2023-11-06T03:18:20.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc0b1df0 con 0x7fbccc0701a0 2023-11-06T03:18:20.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbcd1fbc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc06f6c0 con 0x7fbccc0b1910 2023-11-06T03:18:20.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbccc160db0 0x7fbccc161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbccc160db0 0x7fbccc161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36328/0 (socket says 172.21.15.157:36328) 2023-11-06T03:18:20.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 -- 172.21.15.157:0/4200877576 learned_addr learned my addr 172.21.15.157:0/4200877576 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:20.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccaffd640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 0x7fbccc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccbfff640 1 --2- 172.21.15.157:0/4200877576 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 0x7fbccc0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 -- 172.21.15.157:0/4200877576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 msgr2=0x7fbccc06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:20.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 0x7fbccc06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 -- 172.21.15.157:0/4200877576 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 msgr2=0x7fbccc0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:20.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/4200877576 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 0x7fbccc0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccb7fe640 1 -- 172.21.15.157:0/4200877576 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc06f9a0 con 0x7fbccc160db0 2023-11-06T03:18:20.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccaffd640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 0x7fbccc06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:20.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.602+0000 7fbccbfff640 1 --2- 172.21.15.157:0/4200877576 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 0x7fbccc0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:20.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc161190 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0b3020 tx=0x7fbcc401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eb9e9310427253 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcca7fc640 1 -- 172.21.15.157:0/4200877576 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40a56a0 con 0x7fbccc160db0 2023-11-06T03:18:20.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcca7fc640 1 -- 172.21.15.157:0/4200877576 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc40a5800 con 0x7fbccc160db0 2023-11-06T03:18:20.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcca7fc640 1 -- 172.21.15.157:0/4200877576 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40a5b90 con 0x7fbccc160db0 2023-11-06T03:18:20.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/4200877576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 msgr2=0x7fbccc161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc161190 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0b3020 tx=0x7fbcc401c910 comp rx=0 tx=0).stop 2023-11-06T03:18:20.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/4200877576 shutdown_connections 2023-11-06T03:18:20.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/4200877576 >> v2:172.21.15.195:3300/0 conn(0x7fbccc0701a0 0x7fbccc0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0b1910 0x7fbccc06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/4200877576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc161190 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/4200877576 >> 172.21.15.157:0/4200877576 conn(0x7fbccc065930 msgr2=0x7fbccc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/4200877576 shutdown_connections 2023-11-06T03:18:20.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.606+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/4200877576 wait complete. 2023-11-06T03:18:20.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 Processor -- start 2023-11-06T03:18:20.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- start start 2023-11-06T03:18:20.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0701a0 0x7fbccc14aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 0x7fbccc14dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc161420 con 0x7fbccc160db0 2023-11-06T03:18:20.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc161fa0 con 0x7fbccc0b1910 2023-11-06T03:18:20.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc161670 con 0x7fbccc0701a0 2023-11-06T03:18:20.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36338/0 (socket says 172.21.15.157:36338) 2023-11-06T03:18:20.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 -- 172.21.15.157:0/747268807 learned_addr learned my addr 172.21.15.157:0/747268807 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:20.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0701a0 0x7fbccc14aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccaffd640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 0x7fbccc14dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 -- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0701a0 msgr2=0x7fbccc14aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0701a0 0x7fbccc14aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 -- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 msgr2=0x7fbccc14dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 0x7fbccc14dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 -- 172.21.15.157:0/747268807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4009b30 con 0x7fbccc160db0 2023-11-06T03:18:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccaffd640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 0x7fbccc14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbccbfff640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00d600 tx=0x7fbcbc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc004030 con 0x7fbccc160db0 2023-11-06T03:18:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc004190 con 0x7fbccc160db0 2023-11-06T03:18:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc003620 con 0x7fbccc160db0 2023-11-06T03:18:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc161950 con 0x7fbccc160db0 2023-11-06T03:18:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc14af60 con 0x7fbccc160db0 2023-11-06T03:18:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc072750 con 0x7fbccc160db0 2023-11-06T03:18:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbcbc003780 con 0x7fbccc160db0 2023-11-06T03:18:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 0x7fbcb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.610+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcbc083970 con 0x7fbccc160db0 2023-11-06T03:18:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.614+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 0x7fbcb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.614+0000 7fbccb7fe640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 0x7fbcb8067f20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40ae000 tx=0x7fbcc40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.614+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbcbc0511d0 con 0x7fbccc160db0 2023-11-06T03:18:20.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.762+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbccc1620e0 con 0x7fbccc160db0 2023-11-06T03:18:20.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.762+0000 7fbcb3fff640 1 -- 172.21.15.157:0/747268807 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+17941 (secure 0 0 0) 0x7fbcbc0547f0 con 0x7fbccc160db0 2023-11-06T03:18:20.762 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:20.762 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":189,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:19.101036+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:20.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 msgr2=0x7fbcb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 0x7fbcb8067f20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40ae000 tx=0x7fbcc40a7070 comp rx=0 tx=0).stop 2023-11-06T03:18:20.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 msgr2=0x7fbccc150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00d600 tx=0x7fbcbc00dac0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 shutdown_connections 2023-11-06T03:18:20.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbcb8065a60 0x7fbcb8067f20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbccc160db0 0x7fbccc150950 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbccc0b1910 0x7fbccc14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 --2- 172.21.15.157:0/747268807 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbccc0701a0 0x7fbccc14aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 >> 172.21.15.157:0/747268807 conn(0x7fbccc065930 msgr2=0x7fbccc156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 shutdown_connections 2023-11-06T03:18:20.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.766+0000 7fbcd1fbc640 1 -- 172.21.15.157:0/747268807 wait complete. 2023-11-06T03:18:20.779 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-06T03:18:20.780 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:18:20.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.890+0000 7fd698a10640 1 Processor -- start 2023-11-06T03:18:20.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.890+0000 7fd698a10640 1 -- start start 2023-11-06T03:18:20.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6941551e0 0x7fd69406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd69415e5e0 0x7fd6940b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69415f580 con 0x7fd69415ea60 2023-11-06T03:18:20.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69415efc0 con 0x7fd6941551e0 2023-11-06T03:18:20.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69415f2a0 con 0x7fd69415e5e0 2023-11-06T03:18:20.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6941551e0 0x7fd69406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd6941551e0 0x7fd69406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36350/0 (socket says 172.21.15.157:36350) 2023-11-06T03:18:20.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 -- 172.21.15.157:0/3410727090 learned_addr learned my addr 172.21.15.157:0/3410727090 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:20.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd691d9b640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd69415e5e0 0x7fd6940b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 -- 172.21.15.157:0/3410727090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd69415e5e0 msgr2=0x7fd6940b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:20.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd69415e5e0 0x7fd6940b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 -- 172.21.15.157:0/3410727090 >> v1:172.21.15.195:6789/0 conn(0x7fd69415ea60 legacy=0x7fd69415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:20.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 -- 172.21.15.157:0/3410727090 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69406f5a0 con 0x7fd6941551e0 2023-11-06T03:18:20.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69259c640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6941551e0 0x7fd69406fda0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fd684009870 tx=0x7fd68401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124d0c7537a60ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69159a640 1 -- 172.21.15.157:0/3410727090 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6840ac640 con 0x7fd6941551e0 2023-11-06T03:18:20.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69159a640 1 -- 172.21.15.157:0/3410727090 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6840ac7a0 con 0x7fd6941551e0 2023-11-06T03:18:20.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd69159a640 1 -- 172.21.15.157:0/3410727090 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6840acb30 con 0x7fd6941551e0 2023-11-06T03:18:20.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- 172.21.15.157:0/3410727090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6941551e0 msgr2=0x7fd69406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6941551e0 0x7fd69406fda0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fd684009870 tx=0x7fd68401c440 comp rx=0 tx=0).stop 2023-11-06T03:18:20.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- 172.21.15.157:0/3410727090 shutdown_connections 2023-11-06T03:18:20.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd69415e5e0 0x7fd6940b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 --2- 172.21.15.157:0/3410727090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6941551e0 0x7fd69406fda0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- 172.21.15.157:0/3410727090 >> 172.21.15.157:0/3410727090 conn(0x7fd694065930 msgr2=0x7fd694065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:20.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- 172.21.15.157:0/3410727090 shutdown_connections 2023-11-06T03:18:20.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.894+0000 7fd698a10640 1 -- 172.21.15.157:0/3410727090 wait complete. 2023-11-06T03:18:20.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 Processor -- start 2023-11-06T03:18:20.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- start start 2023-11-06T03:18:20.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6941551e0 0x7fd6940cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd69415ea60 0x7fd6941433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694152a20 con 0x7fd69415e5e0 2023-11-06T03:18:20.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6941526b0 con 0x7fd69415ea60 2023-11-06T03:18:20.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940cde10 con 0x7fd6941551e0 2023-11-06T03:18:20.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36364/0 (socket says 172.21.15.157:36364) 2023-11-06T03:18:20.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 -- 172.21.15.157:0/2679130698 learned_addr learned my addr 172.21.15.157:0/2679130698 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:20.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 -- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6941551e0 msgr2=0x7fd6940cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6941551e0 0x7fd6940cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 -- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd69415ea60 msgr2=0x7fd6941433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:20.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd69415ea60 0x7fd6941433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:20.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 -- 172.21.15.157:0/2679130698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840094d0 con 0x7fd69415e5e0 2023-11-06T03:18:20.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd691d9b640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c820 tx=0x7fd68800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688004220 con 0x7fd69415e5e0 2023-11-06T03:18:20.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6880043a0 con 0x7fd69415e5e0 2023-11-06T03:18:20.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688004510 con 0x7fd69415e5e0 2023-11-06T03:18:20.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940ce0f0 con 0x7fd69415e5e0 2023-11-06T03:18:20.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694144520 con 0x7fd69415e5e0 2023-11-06T03:18:20.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.898+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd694072750 con 0x7fd69415e5e0 2023-11-06T03:18:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.902+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd688007a70 con 0x7fd69415e5e0 2023-11-06T03:18:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.902+0000 7fd682ffd640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 0x7fd674067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:20.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.902+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd688083060 con 0x7fd69415e5e0 2023-11-06T03:18:20.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.902+0000 7fd69259c640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 0x7fd674067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:20.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.902+0000 7fd69259c640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 0x7fd674067f20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd6840ae000 tx=0x7fd684008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:20.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:20.906+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd688050950 con 0x7fd69415e5e0 2023-11-06T03:18:21.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.054+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6940b3d60 con 0x7fd674065a60 2023-11-06T03:18:21.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd682ffd640 1 -- 172.21.15.157:0/2679130698 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311057 (secure 0 0 0) 0x7fd6840ac910 con 0x7fd674065a60 2023-11-06T03:18:21.080 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:21.081 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:18:21.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 msgr2=0x7fd674067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 0x7fd674067f20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd6840ae000 tx=0x7fd684008040 comp rx=0 tx=0).stop 2023-11-06T03:18:21.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 msgr2=0x7fd6940aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c820 tx=0x7fd68800cce0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 shutdown_connections 2023-11-06T03:18:21.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd674065a60 0x7fd674067f20 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd69415ea60 0x7fd6941433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd69415e5e0 0x7fd6940aed40 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 --2- 172.21.15.157:0/2679130698 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6941551e0 0x7fd6940cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 >> 172.21.15.157:0/2679130698 conn(0x7fd694065930 msgr2=0x7fd6940b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:21.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 shutdown_connections 2023-11-06T03:18:21.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.082+0000 7fd698a10640 1 -- 172.21.15.157:0/2679130698 wait complete. 2023-11-06T03:18:21.099 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2023-11-06T03:18:19.403294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":679520,"kb_used_data":19744,"kb_used_omap":0,"kb_used_meta":659712,"kb_avail":754295200,"statfs":{"total":773094113280,"available":772398284800,"internally_reserved":0,"allocated":20217856,"data_stored":10015314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":675545088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001094"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120366+0000","last_change":"2023-11-06T03:18:18.168249+0000","last_active":"2023-11-06T03:18:19.120366+0000","last_peered":"2023-11-06T03:18:19.120366+0000","last_clean":"2023-11-06T03:18:19.120366+0000","last_became_active":"2023-11-06T03:18:18.167659+0000","last_became_peered":"2023-11-06T03:18:18.167659+0000","last_unstale":"2023-11-06T03:18:19.120366+0000","last_undegraded":"2023-11-06T03:18:19.120366+0000","last_fullsized":"2023-11-06T03:18:19.120366+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:36.666154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133610+0000","last_change":"2023-11-06T03:18:18.149372+0000","last_active":"2023-11-06T03:18:19.133610+0000","last_peered":"2023-11-06T03:18:19.133610+0000","last_clean":"2023-11-06T03:18:19.133610+0000","last_became_active":"2023-11-06T03:18:18.149110+0000","last_became_peered":"2023-11-06T03:18:18.149110+0000","last_unstale":"2023-11-06T03:18:19.133610+0000","last_undegraded":"2023-11-06T03:18:19.133610+0000","last_fullsized":"2023-11-06T03:18:19.133610+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:25.142841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115011+0000","last_change":"2023-11-06T03:18:18.142136+0000","last_active":"2023-11-06T03:18:19.115011+0000","last_peered":"2023-11-06T03:18:19.115011+0000","last_clean":"2023-11-06T03:18:19.115011+0000","last_became_active":"2023-11-06T03:18:18.141832+0000","last_became_peered":"2023-11-06T03:18:18.141832+0000","last_unstale":"2023-11-06T03:18:19.115011+0000","last_undegraded":"2023-11-06T03:18:19.115011+0000","last_fullsized":"2023-11-06T03:18:19.115011+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:28.554295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138244+0000","last_change":"2023-11-06T03:18:18.144675+0000","last_active":"2023-11-06T03:18:19.138244+0000","last_peered":"2023-11-06T03:18:19.138244+0000","last_clean":"2023-11-06T03:18:19.138244+0000","last_became_active":"2023-11-06T03:18:18.143937+0000","last_became_peered":"2023-11-06T03:18:18.143937+0000","last_unstale":"2023-11-06T03:18:19.138244+0000","last_undegraded":"2023-11-06T03:18:19.138244+0000","last_fullsized":"2023-11-06T03:18:19.138244+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135444+0000","last_change":"2023-11-06T03:18:18.144270+0000","last_active":"2023-11-06T03:18:19.135444+0000","last_peered":"2023-11-06T03:18:19.135444+0000","last_clean":"2023-11-06T03:18:19.135444+0000","last_became_active":"2023-11-06T03:18:18.143943+0000","last_became_peered":"2023-11-06T03:18:18.143943+0000","last_unstale":"2023-11-06T03:18:19.135444+0000","last_undegraded":"2023-11-06T03:18:19.135444+0000","last_fullsized":"2023-11-06T03:18:19.135444+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:10.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136969+0000","last_change":"2023-11-06T03:18:18.144879+0000","last_active":"2023-11-06T03:18:19.136969+0000","last_peered":"2023-11-06T03:18:19.136969+0000","last_clean":"2023-11-06T03:18:19.136969+0000","last_became_active":"2023-11-06T03:18:18.144119+0000","last_became_peered":"2023-11-06T03:18:18.144119+0000","last_unstale":"2023-11-06T03:18:19.136969+0000","last_undegraded":"2023-11-06T03:18:19.136969+0000","last_fullsized":"2023-11-06T03:18:19.136969+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:30.811652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135232+0000","last_change":"2023-11-06T03:18:18.143704+0000","last_active":"2023-11-06T03:18:19.135232+0000","last_peered":"2023-11-06T03:18:19.135232+0000","last_clean":"2023-11-06T03:18:19.135232+0000","last_became_active":"2023-11-06T03:18:18.143406+0000","last_became_peered":"2023-11-06T03:18:18.143406+0000","last_unstale":"2023-11-06T03:18:19.135232+0000","last_undegraded":"2023-11-06T03:18:19.135232+0000","last_fullsized":"2023-11-06T03:18:19.135232+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:16:31.706245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138113+0000","last_change":"2023-11-06T03:18:18.167330+0000","last_active":"2023-11-06T03:18:19.138113+0000","last_peered":"2023-11-06T03:18:19.138113+0000","last_clean":"2023-11-06T03:18:19.138113+0000","last_became_active":"2023-11-06T03:18:18.166668+0000","last_became_peered":"2023-11-06T03:18:18.166668+0000","last_unstale":"2023-11-06T03:18:19.138113+0000","last_undegraded":"2023-11-06T03:18:19.138113+0000","last_fullsized":"2023-11-06T03:18:19.138113+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:37.367839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135357+0000","last_change":"2023-11-06T03:18:18.142295+0000","last_active":"2023-11-06T03:18:19.135357+0000","last_peered":"2023-11-06T03:18:19.135357+0000","last_clean":"2023-11-06T03:18:19.135357+0000","last_became_active":"2023-11-06T03:18:18.142012+0000","last_became_peered":"2023-11-06T03:18:18.142012+0000","last_unstale":"2023-11-06T03:18:19.135357+0000","last_undegraded":"2023-11-06T03:18:19.135357+0000","last_fullsized":"2023-11-06T03:18:19.135357+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:34.379884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140699+0000","last_change":"2023-11-06T03:18:18.170704+0000","last_active":"2023-11-06T03:18:19.140699+0000","last_peered":"2023-11-06T03:18:19.140699+0000","last_clean":"2023-11-06T03:18:19.140699+0000","last_became_active":"2023-11-06T03:18:18.170251+0000","last_became_peered":"2023-11-06T03:18:18.170251+0000","last_unstale":"2023-11-06T03:18:19.140699+0000","last_undegraded":"2023-11-06T03:18:19.140699+0000","last_fullsized":"2023-11-06T03:18:19.140699+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:52.213757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115011+0000","last_change":"2023-11-06T03:18:18.173145+0000","last_active":"2023-11-06T03:18:19.115011+0000","last_peered":"2023-11-06T03:18:19.115011+0000","last_clean":"2023-11-06T03:18:19.115011+0000","last_became_active":"2023-11-06T03:18:18.172854+0000","last_became_peered":"2023-11-06T03:18:18.172854+0000","last_unstale":"2023-11-06T03:18:19.115011+0000","last_undegraded":"2023-11-06T03:18:19.115011+0000","last_fullsized":"2023-11-06T03:18:19.115011+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:47.039810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135358+0000","last_change":"2023-11-06T03:18:18.126511+0000","last_active":"2023-11-06T03:18:19.135358+0000","last_peered":"2023-11-06T03:18:19.135358+0000","last_clean":"2023-11-06T03:18:19.135358+0000","last_became_active":"2023-11-06T03:18:18.126213+0000","last_became_peered":"2023-11-06T03:18:18.126213+0000","last_unstale":"2023-11-06T03:18:19.135358+0000","last_undegraded":"2023-11-06T03:18:19.135358+0000","last_fullsized":"2023-11-06T03:18:19.135358+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:52.213757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137546+0000","last_change":"2023-11-06T03:18:18.145641+0000","last_active":"2023-11-06T03:18:19.137546+0000","last_peered":"2023-11-06T03:18:19.137546+0000","last_clean":"2023-11-06T03:18:19.137546+0000","last_became_active":"2023-11-06T03:18:18.144897+0000","last_became_peered":"2023-11-06T03:18:18.144897+0000","last_unstale":"2023-11-06T03:18:19.137546+0000","last_undegraded":"2023-11-06T03:18:19.137546+0000","last_fullsized":"2023-11-06T03:18:19.137546+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:09.769001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139642+0000","last_change":"2023-11-06T03:18:18.168993+0000","last_active":"2023-11-06T03:18:19.139642+0000","last_peered":"2023-11-06T03:18:19.139642+0000","last_clean":"2023-11-06T03:18:19.139642+0000","last_became_active":"2023-11-06T03:18:18.168073+0000","last_became_peered":"2023-11-06T03:18:18.168073+0000","last_unstale":"2023-11-06T03:18:19.139642+0000","last_undegraded":"2023-11-06T03:18:19.139642+0000","last_fullsized":"2023-11-06T03:18:19.139642+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:47.903714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137665+0000","last_change":"2023-11-06T03:18:18.168129+0000","last_active":"2023-11-06T03:18:19.137665+0000","last_peered":"2023-11-06T03:18:19.137665+0000","last_clean":"2023-11-06T03:18:19.137665+0000","last_became_active":"2023-11-06T03:18:18.167500+0000","last_became_peered":"2023-11-06T03:18:18.167500+0000","last_unstale":"2023-11-06T03:18:19.137665+0000","last_undegraded":"2023-11-06T03:18:19.137665+0000","last_fullsized":"2023-11-06T03:18:19.137665+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:30.811652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114663+0000","last_change":"2023-11-06T03:18:18.116147+0000","last_active":"2023-11-06T03:18:19.114663+0000","last_peered":"2023-11-06T03:18:19.114663+0000","last_clean":"2023-11-06T03:18:19.114663+0000","last_became_active":"2023-11-06T03:18:18.115862+0000","last_became_peered":"2023-11-06T03:18:18.115862+0000","last_unstale":"2023-11-06T03:18:19.114663+0000","last_undegraded":"2023-11-06T03:18:19.114663+0000","last_fullsized":"2023-11-06T03:18:19.114663+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:18.233201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135447+0000","last_change":"2023-11-06T03:18:18.166071+0000","last_active":"2023-11-06T03:18:19.135447+0000","last_peered":"2023-11-06T03:18:19.135447+0000","last_clean":"2023-11-06T03:18:19.135447+0000","last_became_active":"2023-11-06T03:18:18.165660+0000","last_became_peered":"2023-11-06T03:18:18.165660+0000","last_unstale":"2023-11-06T03:18:19.135447+0000","last_undegraded":"2023-11-06T03:18:19.135447+0000","last_fullsized":"2023-11-06T03:18:19.135447+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:54.297919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134876+0000","last_change":"2023-11-06T03:18:18.121145+0000","last_active":"2023-11-06T03:18:19.134876+0000","last_peered":"2023-11-06T03:18:19.134876+0000","last_clean":"2023-11-06T03:18:19.134876+0000","last_became_active":"2023-11-06T03:18:18.120922+0000","last_became_peered":"2023-11-06T03:18:18.120922+0000","last_unstale":"2023-11-06T03:18:19.134876+0000","last_undegraded":"2023-11-06T03:18:19.134876+0000","last_fullsized":"2023-11-06T03:18:19.134876+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:02.008853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140487+0000","last_change":"2023-11-06T03:18:18.164997+0000","last_active":"2023-11-06T03:18:19.140487+0000","last_peered":"2023-11-06T03:18:19.140487+0000","last_clean":"2023-11-06T03:18:19.140487+0000","last_became_active":"2023-11-06T03:18:18.164305+0000","last_became_peered":"2023-11-06T03:18:18.164305+0000","last_unstale":"2023-11-06T03:18:19.140487+0000","last_undegraded":"2023-11-06T03:18:19.140487+0000","last_fullsized":"2023-11-06T03:18:19.140487+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:10.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140289+0000","last_change":"2023-11-06T03:18:18.168423+0000","last_active":"2023-11-06T03:18:19.140289+0000","last_peered":"2023-11-06T03:18:19.140289+0000","last_clean":"2023-11-06T03:18:19.140289+0000","last_became_active":"2023-11-06T03:18:18.167817+0000","last_became_peered":"2023-11-06T03:18:18.167817+0000","last_unstale":"2023-11-06T03:18:19.140289+0000","last_undegraded":"2023-11-06T03:18:19.140289+0000","last_fullsized":"2023-11-06T03:18:19.140289+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:46.925922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139979+0000","last_change":"2023-11-06T03:18:18.171464+0000","last_active":"2023-11-06T03:18:19.139979+0000","last_peered":"2023-11-06T03:18:19.139979+0000","last_clean":"2023-11-06T03:18:19.139979+0000","last_became_active":"2023-11-06T03:18:18.165512+0000","last_became_peered":"2023-11-06T03:18:18.165512+0000","last_unstale":"2023-11-06T03:18:19.139979+0000","last_undegraded":"2023-11-06T03:18:19.139979+0000","last_fullsized":"2023-11-06T03:18:19.139979+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:25:37.052896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139571+0000","last_change":"2023-11-06T03:18:18.168613+0000","last_active":"2023-11-06T03:18:19.139571+0000","last_peered":"2023-11-06T03:18:19.139571+0000","last_clean":"2023-11-06T03:18:19.139571+0000","last_became_active":"2023-11-06T03:18:18.167584+0000","last_became_peered":"2023-11-06T03:18:18.167584+0000","last_unstale":"2023-11-06T03:18:19.139571+0000","last_undegraded":"2023-11-06T03:18:19.139571+0000","last_fullsized":"2023-11-06T03:18:19.139571+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:23.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133835+0000","last_change":"2023-11-06T03:18:18.142185+0000","last_active":"2023-11-06T03:18:19.133835+0000","last_peered":"2023-11-06T03:18:19.133835+0000","last_clean":"2023-11-06T03:18:19.133835+0000","last_became_active":"2023-11-06T03:18:18.141866+0000","last_became_peered":"2023-11-06T03:18:18.141866+0000","last_unstale":"2023-11-06T03:18:19.133835+0000","last_undegraded":"2023-11-06T03:18:19.133835+0000","last_fullsized":"2023-11-06T03:18:19.133835+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:09.439626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134294+0000","last_change":"2023-11-06T03:18:18.138324+0000","last_active":"2023-11-06T03:18:19.134294+0000","last_peered":"2023-11-06T03:18:19.134294+0000","last_clean":"2023-11-06T03:18:19.134294+0000","last_became_active":"2023-11-06T03:18:18.137983+0000","last_became_peered":"2023-11-06T03:18:18.137983+0000","last_unstale":"2023-11-06T03:18:19.134294+0000","last_undegraded":"2023-11-06T03:18:19.134294+0000","last_fullsized":"2023-11-06T03:18:19.134294+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:13.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135529+0000","last_change":"2023-11-06T03:18:18.124053+0000","last_active":"2023-11-06T03:18:19.135529+0000","last_peered":"2023-11-06T03:18:19.135529+0000","last_clean":"2023-11-06T03:18:19.135529+0000","last_became_active":"2023-11-06T03:18:18.123508+0000","last_became_peered":"2023-11-06T03:18:18.123508+0000","last_unstale":"2023-11-06T03:18:19.135529+0000","last_undegraded":"2023-11-06T03:18:19.135529+0000","last_fullsized":"2023-11-06T03:18:19.135529+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:03.885752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119942+0000","last_change":"2023-11-06T03:18:18.171722+0000","last_active":"2023-11-06T03:18:19.119942+0000","last_peered":"2023-11-06T03:18:19.119942+0000","last_clean":"2023-11-06T03:18:19.119942+0000","last_became_active":"2023-11-06T03:18:18.171263+0000","last_became_peered":"2023-11-06T03:18:18.171263+0000","last_unstale":"2023-11-06T03:18:19.119942+0000","last_undegraded":"2023-11-06T03:18:19.119942+0000","last_fullsized":"2023-11-06T03:18:19.119942+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:47.503452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115395+0000","last_change":"2023-11-06T03:18:18.172851+0000","last_active":"2023-11-06T03:18:19.115395+0000","last_peered":"2023-11-06T03:18:19.115395+0000","last_clean":"2023-11-06T03:18:19.115395+0000","last_became_active":"2023-11-06T03:18:18.172473+0000","last_became_peered":"2023-11-06T03:18:18.172473+0000","last_unstale":"2023-11-06T03:18:19.115395+0000","last_undegraded":"2023-11-06T03:18:19.115395+0000","last_fullsized":"2023-11-06T03:18:19.115395+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:48:12.235892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135885+0000","last_change":"2023-11-06T03:18:18.144877+0000","last_active":"2023-11-06T03:18:19.135885+0000","last_peered":"2023-11-06T03:18:19.135885+0000","last_clean":"2023-11-06T03:18:19.135885+0000","last_became_active":"2023-11-06T03:18:18.144604+0000","last_became_peered":"2023-11-06T03:18:18.144604+0000","last_unstale":"2023-11-06T03:18:19.135885+0000","last_undegraded":"2023-11-06T03:18:19.135885+0000","last_fullsized":"2023-11-06T03:18:19.135885+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:47.903714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134404+0000","last_change":"2023-11-06T03:18:18.121982+0000","last_active":"2023-11-06T03:18:19.134404+0000","last_peered":"2023-11-06T03:18:19.134404+0000","last_clean":"2023-11-06T03:18:19.134404+0000","last_became_active":"2023-11-06T03:18:18.121610+0000","last_became_peered":"2023-11-06T03:18:18.121610+0000","last_unstale":"2023-11-06T03:18:19.134404+0000","last_undegraded":"2023-11-06T03:18:19.134404+0000","last_fullsized":"2023-11-06T03:18:19.134404+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:32.125009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137892+0000","last_change":"2023-11-06T03:18:18.113711+0000","last_active":"2023-11-06T03:18:19.137892+0000","last_peered":"2023-11-06T03:18:19.137892+0000","last_clean":"2023-11-06T03:18:19.137892+0000","last_became_active":"2023-11-06T03:18:18.112940+0000","last_became_peered":"2023-11-06T03:18:18.112940+0000","last_unstale":"2023-11-06T03:18:19.137892+0000","last_undegraded":"2023-11-06T03:18:19.137892+0000","last_fullsized":"2023-11-06T03:18:19.137892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137182+0000","last_change":"2023-11-06T03:18:18.144724+0000","last_active":"2023-11-06T03:18:19.137182+0000","last_peered":"2023-11-06T03:18:19.137182+0000","last_clean":"2023-11-06T03:18:19.137182+0000","last_became_active":"2023-11-06T03:18:18.144024+0000","last_became_peered":"2023-11-06T03:18:18.144024+0000","last_unstale":"2023-11-06T03:18:19.137182+0000","last_undegraded":"2023-11-06T03:18:19.137182+0000","last_fullsized":"2023-11-06T03:18:19.137182+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:11:56.799818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136856+0000","last_change":"2023-11-06T03:18:18.155530+0000","last_active":"2023-11-06T03:18:19.136856+0000","last_peered":"2023-11-06T03:18:19.136856+0000","last_clean":"2023-11-06T03:18:19.136856+0000","last_became_active":"2023-11-06T03:18:18.154754+0000","last_became_peered":"2023-11-06T03:18:18.154754+0000","last_unstale":"2023-11-06T03:18:19.136856+0000","last_undegraded":"2023-11-06T03:18:19.136856+0000","last_fullsized":"2023-11-06T03:18:19.136856+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:02.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137463+0000","last_change":"2023-11-06T03:18:18.120085+0000","last_active":"2023-11-06T03:18:19.137463+0000","last_peered":"2023-11-06T03:18:19.137463+0000","last_clean":"2023-11-06T03:18:19.137463+0000","last_became_active":"2023-11-06T03:18:18.119375+0000","last_became_peered":"2023-11-06T03:18:18.119375+0000","last_unstale":"2023-11-06T03:18:19.137463+0000","last_undegraded":"2023-11-06T03:18:19.137463+0000","last_fullsized":"2023-11-06T03:18:19.137463+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:19.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136114+0000","last_change":"2023-11-06T03:18:17.768948+0000","last_active":"2023-11-06T03:18:19.136114+0000","last_peered":"2023-11-06T03:18:19.136114+0000","last_clean":"2023-11-06T03:18:19.136114+0000","last_became_active":"2023-11-06T03:18:17.768462+0000","last_became_peered":"2023-11-06T03:18:17.768462+0000","last_unstale":"2023-11-06T03:18:19.136114+0000","last_undegraded":"2023-11-06T03:18:19.136114+0000","last_fullsized":"2023-11-06T03:18:19.136114+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:10:58.578257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115294+0000","last_change":"2023-11-06T03:18:18.127656+0000","last_active":"2023-11-06T03:18:19.115294+0000","last_peered":"2023-11-06T03:18:19.115294+0000","last_clean":"2023-11-06T03:18:19.115294+0000","last_became_active":"2023-11-06T03:18:18.127382+0000","last_became_peered":"2023-11-06T03:18:18.127382+0000","last_unstale":"2023-11-06T03:18:19.115294+0000","last_undegraded":"2023-11-06T03:18:19.115294+0000","last_fullsized":"2023-11-06T03:18:19.115294+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:42:19.379590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138814+0000","last_change":"2023-11-06T03:18:17.763898+0000","last_active":"2023-11-06T03:18:19.138814+0000","last_peered":"2023-11-06T03:18:19.138814+0000","last_clean":"2023-11-06T03:18:19.138814+0000","last_became_active":"2023-11-06T03:18:17.763372+0000","last_became_peered":"2023-11-06T03:18:17.763372+0000","last_unstale":"2023-11-06T03:18:19.138814+0000","last_undegraded":"2023-11-06T03:18:19.138814+0000","last_fullsized":"2023-11-06T03:18:19.138814+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:44:38.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134405+0000","last_change":"2023-11-06T03:18:18.153919+0000","last_active":"2023-11-06T03:18:19.134405+0000","last_peered":"2023-11-06T03:18:19.134405+0000","last_clean":"2023-11-06T03:18:19.134405+0000","last_became_active":"2023-11-06T03:18:18.153636+0000","last_became_peered":"2023-11-06T03:18:18.153636+0000","last_unstale":"2023-11-06T03:18:19.134405+0000","last_undegraded":"2023-11-06T03:18:19.134405+0000","last_fullsized":"2023-11-06T03:18:19.134405+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.009161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"188'3","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134869+0000","last_change":"2023-11-06T03:18:17.742723+0000","last_active":"2023-11-06T03:18:19.134869+0000","last_peered":"2023-11-06T03:18:19.134869+0000","last_clean":"2023-11-06T03:18:19.134869+0000","last_became_active":"2023-11-06T03:18:17.742400+0000","last_became_peered":"2023-11-06T03:18:17.742400+0000","last_unstale":"2023-11-06T03:18:19.134869+0000","last_undegraded":"2023-11-06T03:18:19.134869+0000","last_fullsized":"2023-11-06T03:18:19.134869+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:24.339478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135577+0000","last_change":"2023-11-06T03:18:18.171521+0000","last_active":"2023-11-06T03:18:19.135577+0000","last_peered":"2023-11-06T03:18:19.135577+0000","last_clean":"2023-11-06T03:18:19.135577+0000","last_became_active":"2023-11-06T03:18:18.171217+0000","last_became_peered":"2023-11-06T03:18:18.171217+0000","last_unstale":"2023-11-06T03:18:19.135577+0000","last_undegraded":"2023-11-06T03:18:19.135577+0000","last_fullsized":"2023-11-06T03:18:19.135577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:37.367839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"188'6","reported_seq":17,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138516+0000","last_change":"2023-11-06T03:18:17.771139+0000","last_active":"2023-11-06T03:18:19.138516+0000","last_peered":"2023-11-06T03:18:19.138516+0000","last_clean":"2023-11-06T03:18:19.138516+0000","last_became_active":"2023-11-06T03:18:17.770775+0000","last_became_peered":"2023-11-06T03:18:17.770775+0000","last_unstale":"2023-11-06T03:18:19.138516+0000","last_undegraded":"2023-11-06T03:18:19.138516+0000","last_fullsized":"2023-11-06T03:18:19.138516+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:28.242405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133833+0000","last_change":"2023-11-06T03:18:18.127541+0000","last_active":"2023-11-06T03:18:19.133833+0000","last_peered":"2023-11-06T03:18:19.133833+0000","last_clean":"2023-11-06T03:18:19.133833+0000","last_became_active":"2023-11-06T03:18:18.127006+0000","last_became_peered":"2023-11-06T03:18:18.127006+0000","last_unstale":"2023-11-06T03:18:19.133833+0000","last_undegraded":"2023-11-06T03:18:19.133833+0000","last_fullsized":"2023-11-06T03:18:19.133833+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:04.360886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133835+0000","last_change":"2023-11-06T03:18:17.750251+0000","last_active":"2023-11-06T03:18:19.133835+0000","last_peered":"2023-11-06T03:18:19.133835+0000","last_clean":"2023-11-06T03:18:19.133835+0000","last_became_active":"2023-11-06T03:18:17.749673+0000","last_became_peered":"2023-11-06T03:18:17.749673+0000","last_unstale":"2023-11-06T03:18:19.133835+0000","last_undegraded":"2023-11-06T03:18:19.133835+0000","last_fullsized":"2023-11-06T03:18:19.133835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:46.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115382+0000","last_change":"2023-11-06T03:18:18.120600+0000","last_active":"2023-11-06T03:18:19.115382+0000","last_peered":"2023-11-06T03:18:19.115382+0000","last_clean":"2023-11-06T03:18:19.115382+0000","last_became_active":"2023-11-06T03:18:18.120335+0000","last_became_peered":"2023-11-06T03:18:18.120335+0000","last_unstale":"2023-11-06T03:18:19.115382+0000","last_undegraded":"2023-11-06T03:18:19.115382+0000","last_fullsized":"2023-11-06T03:18:19.115382+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:03.160587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114741+0000","last_change":"2023-11-06T03:18:17.734391+0000","last_active":"2023-11-06T03:18:19.114741+0000","last_peered":"2023-11-06T03:18:19.114741+0000","last_clean":"2023-11-06T03:18:19.114741+0000","last_became_active":"2023-11-06T03:18:17.734027+0000","last_became_peered":"2023-11-06T03:18:17.734027+0000","last_unstale":"2023-11-06T03:18:19.114741+0000","last_undegraded":"2023-11-06T03:18:19.114741+0000","last_fullsized":"2023-11-06T03:18:19.114741+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:49.437013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134956+0000","last_change":"2023-11-06T03:18:18.128385+0000","last_active":"2023-11-06T03:18:19.134956+0000","last_peered":"2023-11-06T03:18:19.134956+0000","last_clean":"2023-11-06T03:18:19.134956+0000","last_became_active":"2023-11-06T03:18:18.128126+0000","last_became_peered":"2023-11-06T03:18:18.128126+0000","last_unstale":"2023-11-06T03:18:19.134956+0000","last_undegraded":"2023-11-06T03:18:19.134956+0000","last_fullsized":"2023-11-06T03:18:19.134956+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:58:40.707472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134307+0000","last_change":"2023-11-06T03:18:17.760839+0000","last_active":"2023-11-06T03:18:19.134307+0000","last_peered":"2023-11-06T03:18:19.134307+0000","last_clean":"2023-11-06T03:18:19.134307+0000","last_became_active":"2023-11-06T03:18:17.760493+0000","last_became_peered":"2023-11-06T03:18:17.760493+0000","last_unstale":"2023-11-06T03:18:19.134307+0000","last_undegraded":"2023-11-06T03:18:19.134307+0000","last_fullsized":"2023-11-06T03:18:19.134307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:47.034347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138726+0000","last_change":"2023-11-06T03:18:18.149464+0000","last_active":"2023-11-06T03:18:19.138726+0000","last_peered":"2023-11-06T03:18:19.138726+0000","last_clean":"2023-11-06T03:18:19.138726+0000","last_became_active":"2023-11-06T03:18:18.145679+0000","last_became_peered":"2023-11-06T03:18:18.145679+0000","last_unstale":"2023-11-06T03:18:19.138726+0000","last_undegraded":"2023-11-06T03:18:19.138726+0000","last_fullsized":"2023-11-06T03:18:19.138726+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:11.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"188'8","reported_seq":19,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134933+0000","last_change":"2023-11-06T03:18:17.767978+0000","last_active":"2023-11-06T03:18:19.134933+0000","last_peered":"2023-11-06T03:18:19.134933+0000","last_clean":"2023-11-06T03:18:19.134933+0000","last_became_active":"2023-11-06T03:18:17.767592+0000","last_became_peered":"2023-11-06T03:18:17.767592+0000","last_unstale":"2023-11-06T03:18:19.134933+0000","last_undegraded":"2023-11-06T03:18:19.134933+0000","last_fullsized":"2023-11-06T03:18:19.134933+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:27.234205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138200+0000","last_change":"2023-11-06T03:18:18.166863+0000","last_active":"2023-11-06T03:18:19.138200+0000","last_peered":"2023-11-06T03:18:19.138200+0000","last_clean":"2023-11-06T03:18:19.138200+0000","last_became_active":"2023-11-06T03:18:18.165294+0000","last_became_peered":"2023-11-06T03:18:18.165294+0000","last_unstale":"2023-11-06T03:18:19.138200+0000","last_undegraded":"2023-11-06T03:18:19.138200+0000","last_fullsized":"2023-11-06T03:18:19.138200+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:13.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120396+0000","last_change":"2023-11-06T03:18:17.751838+0000","last_active":"2023-11-06T03:18:19.120396+0000","last_peered":"2023-11-06T03:18:19.120396+0000","last_clean":"2023-11-06T03:18:19.120396+0000","last_became_active":"2023-11-06T03:18:17.751022+0000","last_became_peered":"2023-11-06T03:18:17.751022+0000","last_unstale":"2023-11-06T03:18:19.120396+0000","last_undegraded":"2023-11-06T03:18:19.120396+0000","last_fullsized":"2023-11-06T03:18:19.120396+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:01.348159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119215+0000","last_change":"2023-11-06T03:18:18.170543+0000","last_active":"2023-11-06T03:18:19.119215+0000","last_peered":"2023-11-06T03:18:19.119215+0000","last_clean":"2023-11-06T03:18:19.119215+0000","last_became_active":"2023-11-06T03:18:18.169976+0000","last_became_peered":"2023-11-06T03:18:18.169976+0000","last_unstale":"2023-11-06T03:18:19.119215+0000","last_undegraded":"2023-11-06T03:18:19.119215+0000","last_fullsized":"2023-11-06T03:18:19.119215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120186+0000","last_change":"2023-11-06T03:18:17.760121+0000","last_active":"2023-11-06T03:18:19.120186+0000","last_peered":"2023-11-06T03:18:19.120186+0000","last_clean":"2023-11-06T03:18:19.120186+0000","last_became_active":"2023-11-06T03:18:17.758821+0000","last_became_peered":"2023-11-06T03:18:17.758821+0000","last_unstale":"2023-11-06T03:18:19.120186+0000","last_undegraded":"2023-11-06T03:18:19.120186+0000","last_fullsized":"2023-11-06T03:18:19.120186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:45:55.680814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119490+0000","last_change":"2023-11-06T03:18:18.142482+0000","last_active":"2023-11-06T03:18:19.119490+0000","last_peered":"2023-11-06T03:18:19.119490+0000","last_clean":"2023-11-06T03:18:19.119490+0000","last_became_active":"2023-11-06T03:18:18.141760+0000","last_became_peered":"2023-11-06T03:18:18.141760+0000","last_unstale":"2023-11-06T03:18:19.119490+0000","last_undegraded":"2023-11-06T03:18:19.119490+0000","last_fullsized":"2023-11-06T03:18:19.119490+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:30.607519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133914+0000","last_change":"2023-11-06T03:18:17.745717+0000","last_active":"2023-11-06T03:18:19.133914+0000","last_peered":"2023-11-06T03:18:19.133914+0000","last_clean":"2023-11-06T03:18:19.133914+0000","last_became_active":"2023-11-06T03:18:17.745430+0000","last_became_peered":"2023-11-06T03:18:17.745430+0000","last_unstale":"2023-11-06T03:18:19.133914+0000","last_undegraded":"2023-11-06T03:18:19.133914+0000","last_fullsized":"2023-11-06T03:18:19.133914+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:24.132619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140283+0000","last_change":"2023-11-06T03:18:18.166968+0000","last_active":"2023-11-06T03:18:19.140283+0000","last_peered":"2023-11-06T03:18:19.140283+0000","last_clean":"2023-11-06T03:18:19.140283+0000","last_became_active":"2023-11-06T03:18:18.164788+0000","last_became_peered":"2023-11-06T03:18:18.164788+0000","last_unstale":"2023-11-06T03:18:19.140283+0000","last_undegraded":"2023-11-06T03:18:19.140283+0000","last_fullsized":"2023-11-06T03:18:19.140283+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:24.326499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135601+0000","last_change":"2023-11-06T03:18:17.751638+0000","last_active":"2023-11-06T03:18:19.135601+0000","last_peered":"2023-11-06T03:18:19.135601+0000","last_clean":"2023-11-06T03:18:19.135601+0000","last_became_active":"2023-11-06T03:18:17.751360+0000","last_became_peered":"2023-11-06T03:18:17.751360+0000","last_unstale":"2023-11-06T03:18:19.135601+0000","last_undegraded":"2023-11-06T03:18:19.135601+0000","last_fullsized":"2023-11-06T03:18:19.135601+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:17.458421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133763+0000","last_change":"2023-11-06T03:18:18.142949+0000","last_active":"2023-11-06T03:18:19.133763+0000","last_peered":"2023-11-06T03:18:19.133763+0000","last_clean":"2023-11-06T03:18:19.133763+0000","last_became_active":"2023-11-06T03:18:18.142687+0000","last_became_peered":"2023-11-06T03:18:18.142687+0000","last_unstale":"2023-11-06T03:18:19.133763+0000","last_undegraded":"2023-11-06T03:18:19.133763+0000","last_fullsized":"2023-11-06T03:18:19.133763+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:55:31.054492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135668+0000","last_change":"2023-11-06T03:18:17.735782+0000","last_active":"2023-11-06T03:18:19.135668+0000","last_peered":"2023-11-06T03:18:19.135668+0000","last_clean":"2023-11-06T03:18:19.135668+0000","last_became_active":"2023-11-06T03:18:17.735371+0000","last_became_peered":"2023-11-06T03:18:17.735371+0000","last_unstale":"2023-11-06T03:18:19.135668+0000","last_undegraded":"2023-11-06T03:18:19.135668+0000","last_fullsized":"2023-11-06T03:18:19.135668+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:26.532987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138140+0000","last_change":"2023-11-06T03:18:18.149057+0000","last_active":"2023-11-06T03:18:19.138140+0000","last_peered":"2023-11-06T03:18:19.138140+0000","last_clean":"2023-11-06T03:18:19.138140+0000","last_became_active":"2023-11-06T03:18:18.148367+0000","last_became_peered":"2023-11-06T03:18:18.148367+0000","last_unstale":"2023-11-06T03:18:19.138140+0000","last_undegraded":"2023-11-06T03:18:19.138140+0000","last_fullsized":"2023-11-06T03:18:19.138140+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:50.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133923+0000","last_change":"2023-11-06T03:18:17.733090+0000","last_active":"2023-11-06T03:18:19.133923+0000","last_peered":"2023-11-06T03:18:19.133923+0000","last_clean":"2023-11-06T03:18:19.133923+0000","last_became_active":"2023-11-06T03:18:17.732466+0000","last_became_peered":"2023-11-06T03:18:17.732466+0000","last_unstale":"2023-11-06T03:18:19.133923+0000","last_undegraded":"2023-11-06T03:18:19.133923+0000","last_fullsized":"2023-11-06T03:18:19.133923+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:11:09.957215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135185+0000","last_change":"2023-11-06T03:18:18.142939+0000","last_active":"2023-11-06T03:18:19.135185+0000","last_peered":"2023-11-06T03:18:19.135185+0000","last_clean":"2023-11-06T03:18:19.135185+0000","last_became_active":"2023-11-06T03:18:18.142656+0000","last_became_peered":"2023-11-06T03:18:18.142656+0000","last_unstale":"2023-11-06T03:18:19.135185+0000","last_undegraded":"2023-11-06T03:18:19.135185+0000","last_fullsized":"2023-11-06T03:18:19.135185+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:59:20.459898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138067+0000","last_change":"2023-11-06T03:18:17.758130+0000","last_active":"2023-11-06T03:18:19.138067+0000","last_peered":"2023-11-06T03:18:19.138067+0000","last_clean":"2023-11-06T03:18:19.138067+0000","last_became_active":"2023-11-06T03:18:17.757349+0000","last_became_peered":"2023-11-06T03:18:17.757349+0000","last_unstale":"2023-11-06T03:18:19.138067+0000","last_undegraded":"2023-11-06T03:18:19.138067+0000","last_fullsized":"2023-11-06T03:18:19.138067+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:27:41.842899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.141013+0000","last_change":"2023-11-06T03:18:18.165210+0000","last_active":"2023-11-06T03:18:19.141013+0000","last_peered":"2023-11-06T03:18:19.141013+0000","last_clean":"2023-11-06T03:18:19.141013+0000","last_became_active":"2023-11-06T03:18:18.164338+0000","last_became_peered":"2023-11-06T03:18:18.164338+0000","last_unstale":"2023-11-06T03:18:19.141013+0000","last_undegraded":"2023-11-06T03:18:19.141013+0000","last_fullsized":"2023-11-06T03:18:19.141013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:40:55.290584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.118958+0000","last_change":"2023-11-06T03:18:17.745749+0000","last_active":"2023-11-06T03:18:19.118958+0000","last_peered":"2023-11-06T03:18:19.118958+0000","last_clean":"2023-11-06T03:18:19.118958+0000","last_became_active":"2023-11-06T03:18:17.741884+0000","last_became_peered":"2023-11-06T03:18:17.741884+0000","last_unstale":"2023-11-06T03:18:19.118958+0000","last_undegraded":"2023-11-06T03:18:19.118958+0000","last_fullsized":"2023-11-06T03:18:19.118958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:08.259498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138355+0000","last_change":"2023-11-06T03:18:18.164748+0000","last_active":"2023-11-06T03:18:19.138355+0000","last_peered":"2023-11-06T03:18:19.138355+0000","last_clean":"2023-11-06T03:18:19.138355+0000","last_became_active":"2023-11-06T03:18:18.164173+0000","last_became_peered":"2023-11-06T03:18:18.164173+0000","last_unstale":"2023-11-06T03:18:19.138355+0000","last_undegraded":"2023-11-06T03:18:19.138355+0000","last_fullsized":"2023-11-06T03:18:19.138355+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:34.089014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135697+0000","last_change":"2023-11-06T03:18:17.768585+0000","last_active":"2023-11-06T03:18:19.135697+0000","last_peered":"2023-11-06T03:18:19.135697+0000","last_clean":"2023-11-06T03:18:19.135697+0000","last_became_active":"2023-11-06T03:18:17.768193+0000","last_became_peered":"2023-11-06T03:18:17.768193+0000","last_unstale":"2023-11-06T03:18:19.135697+0000","last_undegraded":"2023-11-06T03:18:19.135697+0000","last_fullsized":"2023-11-06T03:18:19.135697+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:01:38.662396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114679+0000","last_change":"2023-11-06T03:18:18.166524+0000","last_active":"2023-11-06T03:18:19.114679+0000","last_peered":"2023-11-06T03:18:19.114679+0000","last_clean":"2023-11-06T03:18:19.114679+0000","last_became_active":"2023-11-06T03:18:18.166309+0000","last_became_peered":"2023-11-06T03:18:18.166309+0000","last_unstale":"2023-11-06T03:18:19.114679+0000","last_undegraded":"2023-11-06T03:18:19.114679+0000","last_fullsized":"2023-11-06T03:18:19.114679+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:28.300548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140790+0000","last_change":"2023-11-06T03:18:17.758544+0000","last_active":"2023-11-06T03:18:19.140790+0000","last_peered":"2023-11-06T03:18:19.140790+0000","last_clean":"2023-11-06T03:18:19.140790+0000","last_became_active":"2023-11-06T03:18:17.757834+0000","last_became_peered":"2023-11-06T03:18:17.757834+0000","last_unstale":"2023-11-06T03:18:19.140790+0000","last_undegraded":"2023-11-06T03:18:19.140790+0000","last_fullsized":"2023-11-06T03:18:19.140790+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:00.089172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134476+0000","last_change":"2023-11-06T03:18:18.127839+0000","last_active":"2023-11-06T03:18:19.134476+0000","last_peered":"2023-11-06T03:18:19.134476+0000","last_clean":"2023-11-06T03:18:19.134476+0000","last_became_active":"2023-11-06T03:18:18.127547+0000","last_became_peered":"2023-11-06T03:18:18.127547+0000","last_unstale":"2023-11-06T03:18:19.134476+0000","last_undegraded":"2023-11-06T03:18:19.134476+0000","last_fullsized":"2023-11-06T03:18:19.134476+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:12.037444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138965+0000","last_change":"2023-11-06T03:18:17.746341+0000","last_active":"2023-11-06T03:18:19.138965+0000","last_peered":"2023-11-06T03:18:19.138965+0000","last_clean":"2023-11-06T03:18:19.138965+0000","last_became_active":"2023-11-06T03:18:17.745581+0000","last_became_peered":"2023-11-06T03:18:17.745581+0000","last_unstale":"2023-11-06T03:18:19.138965+0000","last_undegraded":"2023-11-06T03:18:19.138965+0000","last_fullsized":"2023-11-06T03:18:19.138965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:29.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119895+0000","last_change":"2023-11-06T03:18:18.147737+0000","last_active":"2023-11-06T03:18:19.119895+0000","last_peered":"2023-11-06T03:18:19.119895+0000","last_clean":"2023-11-06T03:18:19.119895+0000","last_became_active":"2023-11-06T03:18:18.146988+0000","last_became_peered":"2023-11-06T03:18:18.146988+0000","last_unstale":"2023-11-06T03:18:19.119895+0000","last_undegraded":"2023-11-06T03:18:19.119895+0000","last_fullsized":"2023-11-06T03:18:19.119895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:29.368572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119757+0000","last_change":"2023-11-06T03:18:17.758659+0000","last_active":"2023-11-06T03:18:19.119757+0000","last_peered":"2023-11-06T03:18:19.119757+0000","last_clean":"2023-11-06T03:18:19.119757+0000","last_became_active":"2023-11-06T03:18:17.751005+0000","last_became_peered":"2023-11-06T03:18:17.751005+0000","last_unstale":"2023-11-06T03:18:19.119757+0000","last_undegraded":"2023-11-06T03:18:19.119757+0000","last_fullsized":"2023-11-06T03:18:19.119757+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:28.242405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140305+0000","last_change":"2023-11-06T03:18:18.170210+0000","last_active":"2023-11-06T03:18:19.140305+0000","last_peered":"2023-11-06T03:18:19.140305+0000","last_clean":"2023-11-06T03:18:19.140305+0000","last_became_active":"2023-11-06T03:18:18.165484+0000","last_became_peered":"2023-11-06T03:18:18.165484+0000","last_unstale":"2023-11-06T03:18:19.140305+0000","last_undegraded":"2023-11-06T03:18:19.140305+0000","last_fullsized":"2023-11-06T03:18:19.140305+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:16:31.706245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140240+0000","last_change":"2023-11-06T03:18:17.748639+0000","last_active":"2023-11-06T03:18:19.140240+0000","last_peered":"2023-11-06T03:18:19.140240+0000","last_clean":"2023-11-06T03:18:19.140240+0000","last_became_active":"2023-11-06T03:18:17.744033+0000","last_became_peered":"2023-11-06T03:18:17.744033+0000","last_unstale":"2023-11-06T03:18:19.140240+0000","last_undegraded":"2023-11-06T03:18:19.140240+0000","last_fullsized":"2023-11-06T03:18:19.140240+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:52:26.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138395+0000","last_change":"2023-11-06T03:18:18.149186+0000","last_active":"2023-11-06T03:18:19.138395+0000","last_peered":"2023-11-06T03:18:19.138395+0000","last_clean":"2023-11-06T03:18:19.138395+0000","last_became_active":"2023-11-06T03:18:18.148474+0000","last_became_peered":"2023-11-06T03:18:18.148474+0000","last_unstale":"2023-11-06T03:18:19.138395+0000","last_undegraded":"2023-11-06T03:18:19.138395+0000","last_fullsized":"2023-11-06T03:18:19.138395+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:19.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134509+0000","last_change":"2023-11-06T03:18:17.742453+0000","last_active":"2023-11-06T03:18:19.134509+0000","last_peered":"2023-11-06T03:18:19.134509+0000","last_clean":"2023-11-06T03:18:19.134509+0000","last_became_active":"2023-11-06T03:18:17.742088+0000","last_became_peered":"2023-11-06T03:18:17.742088+0000","last_unstale":"2023-11-06T03:18:19.134509+0000","last_undegraded":"2023-11-06T03:18:19.134509+0000","last_fullsized":"2023-11-06T03:18:19.134509+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:02.598746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138636+0000","last_change":"2023-11-06T03:18:18.154392+0000","last_active":"2023-11-06T03:18:19.138636+0000","last_peered":"2023-11-06T03:18:19.138636+0000","last_clean":"2023-11-06T03:18:19.138636+0000","last_became_active":"2023-11-06T03:18:18.151240+0000","last_became_peered":"2023-11-06T03:18:18.151240+0000","last_unstale":"2023-11-06T03:18:19.138636+0000","last_undegraded":"2023-11-06T03:18:19.138636+0000","last_fullsized":"2023-11-06T03:18:19.138636+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119255+0000","last_change":"2023-11-06T03:18:17.770837+0000","last_active":"2023-11-06T03:18:19.119255+0000","last_peered":"2023-11-06T03:18:19.119255+0000","last_clean":"2023-11-06T03:18:19.119255+0000","last_became_active":"2023-11-06T03:18:17.770040+0000","last_became_peered":"2023-11-06T03:18:17.770040+0000","last_unstale":"2023-11-06T03:18:19.119255+0000","last_undegraded":"2023-11-06T03:18:19.119255+0000","last_fullsized":"2023-11-06T03:18:19.119255+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:23:22.258878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114922+0000","last_change":"2023-11-06T03:18:18.166588+0000","last_active":"2023-11-06T03:18:19.114922+0000","last_peered":"2023-11-06T03:18:19.114922+0000","last_clean":"2023-11-06T03:18:19.114922+0000","last_became_active":"2023-11-06T03:18:18.166305+0000","last_became_peered":"2023-11-06T03:18:18.166305+0000","last_unstale":"2023-11-06T03:18:19.114922+0000","last_undegraded":"2023-11-06T03:18:19.114922+0000","last_fullsized":"2023-11-06T03:18:19.114922+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:22.806019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134952+0000","last_change":"2023-11-06T03:18:17.767954+0000","last_active":"2023-11-06T03:18:19.134952+0000","last_peered":"2023-11-06T03:18:19.134952+0000","last_clean":"2023-11-06T03:18:19.134952+0000","last_became_active":"2023-11-06T03:18:17.767597+0000","last_became_peered":"2023-11-06T03:18:17.767597+0000","last_unstale":"2023-11-06T03:18:19.134952+0000","last_undegraded":"2023-11-06T03:18:19.134952+0000","last_fullsized":"2023-11-06T03:18:19.134952+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:08.002833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119800+0000","last_change":"2023-11-06T03:18:18.170889+0000","last_active":"2023-11-06T03:18:19.119800+0000","last_peered":"2023-11-06T03:18:19.119800+0000","last_clean":"2023-11-06T03:18:19.119800+0000","last_became_active":"2023-11-06T03:18:18.169485+0000","last_became_peered":"2023-11-06T03:18:18.169485+0000","last_unstale":"2023-11-06T03:18:19.119800+0000","last_undegraded":"2023-11-06T03:18:19.119800+0000","last_fullsized":"2023-11-06T03:18:19.119800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134465+0000","last_change":"2023-11-06T03:18:17.743115+0000","last_active":"2023-11-06T03:18:19.134465+0000","last_peered":"2023-11-06T03:18:19.134465+0000","last_clean":"2023-11-06T03:18:19.134465+0000","last_became_active":"2023-11-06T03:18:17.742780+0000","last_became_peered":"2023-11-06T03:18:17.742780+0000","last_unstale":"2023-11-06T03:18:19.134465+0000","last_undegraded":"2023-11-06T03:18:19.134465+0000","last_fullsized":"2023-11-06T03:18:19.134465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:26:55.231407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.123656+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:18:18.123656+0000","last_peered":"2023-11-06T03:18:18.123656+0000","last_clean":"2023-11-06T03:18:18.123656+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:18:18.123656+0000","last_undegraded":"2023-11-06T03:18:18.123656+0000","last_fullsized":"2023-11-06T03:18:18.123656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.143716+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:18:18.143716+0000","last_peered":"2023-11-06T03:18:18.143716+0000","last_clean":"2023-11-06T03:18:18.143716+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:18:18.143716+0000","last_undegraded":"2023-11-06T03:18:18.143716+0000","last_fullsized":"2023-11-06T03:18:18.143716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138855+0000","last_change":"2023-11-06T03:18:18.129150+0000","last_active":"2023-11-06T03:18:19.138855+0000","last_peered":"2023-11-06T03:18:19.138855+0000","last_clean":"2023-11-06T03:18:19.138855+0000","last_became_active":"2023-11-06T03:18:18.128441+0000","last_became_peered":"2023-11-06T03:18:18.128441+0000","last_unstale":"2023-11-06T03:18:19.138855+0000","last_undegraded":"2023-11-06T03:18:19.138855+0000","last_fullsized":"2023-11-06T03:18:19.138855+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136919+0000","last_change":"2023-11-06T03:18:17.763803+0000","last_active":"2023-11-06T03:18:19.136919+0000","last_peered":"2023-11-06T03:18:19.136919+0000","last_clean":"2023-11-06T03:18:19.136919+0000","last_became_active":"2023-11-06T03:18:17.763044+0000","last_became_peered":"2023-11-06T03:18:17.763044+0000","last_unstale":"2023-11-06T03:18:19.136919+0000","last_undegraded":"2023-11-06T03:18:19.136919+0000","last_fullsized":"2023-11-06T03:18:19.136919+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:25.877643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.123408+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:18:18.123408+0000","last_peered":"2023-11-06T03:18:18.123408+0000","last_clean":"2023-11-06T03:18:18.123408+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:18:18.123408+0000","last_undegraded":"2023-11-06T03:18:18.123408+0000","last_fullsized":"2023-11-06T03:18:18.123408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135615+0000","last_change":"2023-11-06T03:18:18.144417+0000","last_active":"2023-11-06T03:18:19.135615+0000","last_peered":"2023-11-06T03:18:19.135615+0000","last_clean":"2023-11-06T03:18:19.135615+0000","last_became_active":"2023-11-06T03:18:18.144163+0000","last_became_peered":"2023-11-06T03:18:18.144163+0000","last_unstale":"2023-11-06T03:18:19.135615+0000","last_undegraded":"2023-11-06T03:18:19.135615+0000","last_fullsized":"2023-11-06T03:18:19.135615+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:59:40.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114663+0000","last_change":"2023-11-06T03:18:17.750355+0000","last_active":"2023-11-06T03:18:19.114663+0000","last_peered":"2023-11-06T03:18:19.114663+0000","last_clean":"2023-11-06T03:18:19.114663+0000","last_became_active":"2023-11-06T03:18:17.750041+0000","last_became_peered":"2023-11-06T03:18:17.750041+0000","last_unstale":"2023-11-06T03:18:19.114663+0000","last_undegraded":"2023-11-06T03:18:19.114663+0000","last_fullsized":"2023-11-06T03:18:19.114663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:10.791306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.113021+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:18:18.113021+0000","last_peered":"2023-11-06T03:18:18.113021+0000","last_clean":"2023-11-06T03:18:18.113021+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:18:18.113021+0000","last_undegraded":"2023-11-06T03:18:18.113021+0000","last_fullsized":"2023-11-06T03:18:18.113021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139327+0000","last_change":"2023-11-06T03:18:18.112781+0000","last_active":"2023-11-06T03:18:19.139327+0000","last_peered":"2023-11-06T03:18:19.139327+0000","last_clean":"2023-11-06T03:18:19.139327+0000","last_became_active":"2023-11-06T03:18:18.112123+0000","last_became_peered":"2023-11-06T03:18:18.112123+0000","last_unstale":"2023-11-06T03:18:19.139327+0000","last_undegraded":"2023-11-06T03:18:19.139327+0000","last_fullsized":"2023-11-06T03:18:19.139327+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:47.503452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137835+0000","last_change":"2023-11-06T03:18:17.758749+0000","last_active":"2023-11-06T03:18:19.137835+0000","last_peered":"2023-11-06T03:18:19.137835+0000","last_clean":"2023-11-06T03:18:19.137835+0000","last_became_active":"2023-11-06T03:18:17.757940+0000","last_became_peered":"2023-11-06T03:18:17.757940+0000","last_unstale":"2023-11-06T03:18:19.137835+0000","last_undegraded":"2023-11-06T03:18:19.137835+0000","last_fullsized":"2023-11-06T03:18:19.137835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:45:55.680814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.153361+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:18:18.153361+0000","last_peered":"2023-11-06T03:18:18.153361+0000","last_clean":"2023-11-06T03:18:18.153361+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:18:18.153361+0000","last_undegraded":"2023-11-06T03:18:18.153361+0000","last_fullsized":"2023-11-06T03:18:18.153361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139948+0000","last_change":"2023-11-06T03:18:18.167175+0000","last_active":"2023-11-06T03:18:19.139948+0000","last_peered":"2023-11-06T03:18:19.139948+0000","last_clean":"2023-11-06T03:18:19.139948+0000","last_became_active":"2023-11-06T03:18:18.164187+0000","last_became_peered":"2023-11-06T03:18:18.164187+0000","last_unstale":"2023-11-06T03:18:19.139948+0000","last_undegraded":"2023-11-06T03:18:19.139948+0000","last_fullsized":"2023-11-06T03:18:19.139948+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:34.190397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133679+0000","last_change":"2023-11-06T03:18:17.735600+0000","last_active":"2023-11-06T03:18:19.133679+0000","last_peered":"2023-11-06T03:18:19.133679+0000","last_clean":"2023-11-06T03:18:19.133679+0000","last_became_active":"2023-11-06T03:18:17.735310+0000","last_became_peered":"2023-11-06T03:18:17.735310+0000","last_unstale":"2023-11-06T03:18:19.133679+0000","last_undegraded":"2023-11-06T03:18:19.133679+0000","last_fullsized":"2023-11-06T03:18:19.133679+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:08:25.128836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.112153+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:18:18.112153+0000","last_peered":"2023-11-06T03:18:18.112153+0000","last_clean":"2023-11-06T03:18:18.112153+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:18:18.112153+0000","last_undegraded":"2023-11-06T03:18:18.112153+0000","last_fullsized":"2023-11-06T03:18:18.112153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135683+0000","last_change":"2023-11-06T03:18:18.127614+0000","last_active":"2023-11-06T03:18:19.135683+0000","last_peered":"2023-11-06T03:18:19.135683+0000","last_clean":"2023-11-06T03:18:19.135683+0000","last_became_active":"2023-11-06T03:18:18.126446+0000","last_became_peered":"2023-11-06T03:18:18.126446+0000","last_unstale":"2023-11-06T03:18:19.135683+0000","last_undegraded":"2023-11-06T03:18:19.135683+0000","last_fullsized":"2023-11-06T03:18:19.135683+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:24.326499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134692+0000","last_change":"2023-11-06T03:18:17.742458+0000","last_active":"2023-11-06T03:18:19.134692+0000","last_peered":"2023-11-06T03:18:19.134692+0000","last_clean":"2023-11-06T03:18:19.134692+0000","last_became_active":"2023-11-06T03:18:17.742102+0000","last_became_peered":"2023-11-06T03:18:17.742102+0000","last_unstale":"2023-11-06T03:18:19.134692+0000","last_undegraded":"2023-11-06T03:18:19.134692+0000","last_fullsized":"2023-11-06T03:18:19.134692+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:56.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.126832+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:18:18.126832+0000","last_peered":"2023-11-06T03:18:18.126832+0000","last_clean":"2023-11-06T03:18:18.126832+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:18:18.126832+0000","last_undegraded":"2023-11-06T03:18:18.126832+0000","last_fullsized":"2023-11-06T03:18:18.126832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133526+0000","last_change":"2023-11-06T03:18:18.142201+0000","last_active":"2023-11-06T03:18:19.133526+0000","last_peered":"2023-11-06T03:18:19.133526+0000","last_clean":"2023-11-06T03:18:19.133526+0000","last_became_active":"2023-11-06T03:18:18.141894+0000","last_became_peered":"2023-11-06T03:18:18.141894+0000","last_unstale":"2023-11-06T03:18:19.133526+0000","last_undegraded":"2023-11-06T03:18:19.133526+0000","last_fullsized":"2023-11-06T03:18:19.133526+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:31.626851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"188'4","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139467+0000","last_change":"2023-11-06T03:18:17.762622+0000","last_active":"2023-11-06T03:18:19.139467+0000","last_peered":"2023-11-06T03:18:19.139467+0000","last_clean":"2023-11-06T03:18:19.139467+0000","last_became_active":"2023-11-06T03:18:17.762132+0000","last_became_peered":"2023-11-06T03:18:17.762132+0000","last_unstale":"2023-11-06T03:18:19.139467+0000","last_undegraded":"2023-11-06T03:18:19.139467+0000","last_fullsized":"2023-11-06T03:18:19.139467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:40.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":158,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.112003+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:18:18.112003+0000","last_peered":"2023-11-06T03:18:18.112003+0000","last_clean":"2023-11-06T03:18:18.112003+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:18:18.112003+0000","last_undegraded":"2023-11-06T03:18:18.112003+0000","last_fullsized":"2023-11-06T03:18:18.112003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119035+0000","last_change":"2023-11-06T03:18:18.154046+0000","last_active":"2023-11-06T03:18:19.119035+0000","last_peered":"2023-11-06T03:18:19.119035+0000","last_clean":"2023-11-06T03:18:19.119035+0000","last_became_active":"2023-11-06T03:18:18.153294+0000","last_became_peered":"2023-11-06T03:18:18.153294+0000","last_unstale":"2023-11-06T03:18:19.119035+0000","last_undegraded":"2023-11-06T03:18:19.119035+0000","last_fullsized":"2023-11-06T03:18:19.119035+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:11:56.799818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120110+0000","last_change":"2023-11-06T03:18:17.751893+0000","last_active":"2023-11-06T03:18:19.120110+0000","last_peered":"2023-11-06T03:18:19.120110+0000","last_clean":"2023-11-06T03:18:19.120110+0000","last_became_active":"2023-11-06T03:18:17.751134+0000","last_became_peered":"2023-11-06T03:18:17.751134+0000","last_unstale":"2023-11-06T03:18:19.120110+0000","last_undegraded":"2023-11-06T03:18:19.120110+0000","last_fullsized":"2023-11-06T03:18:19.120110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:44:38.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.134021+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:18:18.134021+0000","last_peered":"2023-11-06T03:18:18.134021+0000","last_clean":"2023-11-06T03:18:18.134021+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:18:18.134021+0000","last_undegraded":"2023-11-06T03:18:18.134021+0000","last_fullsized":"2023-11-06T03:18:18.134021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96240,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94275600,"statfs":{"total":96636764160,"available":96538214400,"internally_reserved":0,"allocated":2990080,"data_stored":1700274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":0.76600000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":0.93100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69404,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302436,"statfs":{"total":96636764160,"available":96565694464,"internally_reserved":0,"allocated":2379776,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":0.79000000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.754}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.85499999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.25}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.181}]}]},{"osd":5,"up_from":13,"seq":55834575025,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69412,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302428,"statfs":{"total":96636764160,"available":96565686272,"internally_reserved":0,"allocated":2387968,"data_stored":1108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.751}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":1.004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.91400000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":0.93600000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.0309999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.86299999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":0.98399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113684,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258156,"statfs":{"total":96636764160,"available":96520351744,"internally_reserved":0,"allocated":2371584,"data_stored":1102201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.004}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87512,"kb_used_data":2320,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94284328,"statfs":{"total":96636764160,"available":96547151872,"internally_reserved":0,"allocated":2375680,"data_stored":1100726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":0.82599999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":1.042},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":0.84999999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":1.27},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":1.0900000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":1.232},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73504,"kb_used_data":2328,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298336,"statfs":{"total":96636764160,"available":96561496064,"internally_reserved":0,"allocated":2383872,"data_stored":1108362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.317}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.747},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73488,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298352,"statfs":{"total":96636764160,"available":96561512448,"internally_reserved":0,"allocated":2367488,"data_stored":1101619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.499},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.77100000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":1.006}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":1.508},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":1.2729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96276,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275564,"statfs":{"total":96636764160,"available":96538177536,"internally_reserved":0,"allocated":2961408,"data_stored":1691058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.313,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":2.6989999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.64900000000000002,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.45,"5min":1.45,"15min":7.1230000000000002},"last":0.45100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66100000000000003,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69099999999999995,"15min":0.66400000000000003},"min":{"1min":0.374,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.51300000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.35499999999999998,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.88900000000000001,"15min":0.91100000000000003},"min":{"1min":0.69399999999999995,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.3149999999999999,"5min":1.419,"15min":4.641},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.86399999999999999,"15min":0.872},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.2470000000000001,"5min":1.4159999999999999,"15min":2.028},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.72799999999999998,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.8559999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.96699999999999997,"15min":0.95299999999999996},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.623,"5min":1.623,"15min":4.7430000000000003},"last":0.93200000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.93999999999999995,"15min":0.91600000000000004},"min":{"1min":0.61699999999999999,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":2.931},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.996,"15min":0.997},"min":{"1min":0.68000000000000005,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.002,"15min":0.997},"min":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.495},"max":{"1min":1.544,"5min":1.544,"15min":4.8339999999999996},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.042,"5min":0.98599999999999999,"15min":0.97699999999999998},"min":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.97},"last":0.87}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:18:21.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:18:21.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 Processor -- start 2023-11-06T03:18:21.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 -- start start 2023-11-06T03:18:21.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a415ce80 0x7fe6a40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6a40d00b0 0x7fe6a40b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a40d2920 con 0x7fe6a415caa0 2023-11-06T03:18:21.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40d0520 con 0x7fe6a40d00b0 2023-11-06T03:18:21.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6ab088640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40d18a0 con 0x7fe6a415ce80 2023-11-06T03:18:21.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6a40d00b0 0x7fe6a40b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe6a40d00b0 0x7fe6a40b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36368/0 (socket says 172.21.15.157:36368) 2023-11-06T03:18:21.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 -- 172.21.15.157:0/1297672806 learned_addr learned my addr 172.21.15.157:0/1297672806 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:21.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a415ce80 0x7fe6a40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 -- 172.21.15.157:0/1297672806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a415ce80 msgr2=0x7fe6a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:21.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a415ce80 0x7fe6a40cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 -- 172.21.15.157:0/1297672806 >> v1:172.21.15.195:6789/0 conn(0x7fe6a415caa0 legacy=0x7fe6a40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:21.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 -- 172.21.15.157:0/1297672806 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40d1b60 con 0x7fe6a40d00b0 2023-11-06T03:18:21.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a9625640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a40d00b0 0x7fe6a40b5d50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009e50 tx=0x7fe69c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61901e0ff30c4b4c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1297672806 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0a5820 con 0x7fe6a40d00b0 2023-11-06T03:18:21.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1297672806 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0a5980 con 0x7fe6a40d00b0 2023-11-06T03:18:21.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.270+0000 7fe6a37fe640 1 -- 172.21.15.157:0/1297672806 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0a5d10 con 0x7fe6a40d00b0 2023-11-06T03:18:21.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/1297672806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a40d00b0 msgr2=0x7fe6a40b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a40d00b0 0x7fe6a40b5d50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009e50 tx=0x7fe69c01c9a0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/1297672806 shutdown_connections 2023-11-06T03:18:21.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a40d00b0 0x7fe6a40b5d50 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- 172.21.15.157:0/1297672806 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a415ce80 0x7fe6a40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/1297672806 >> 172.21.15.157:0/1297672806 conn(0x7fe6a4070790 msgr2=0x7fe6a4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:21.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/1297672806 shutdown_connections 2023-11-06T03:18:21.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/1297672806 wait complete. 2023-11-06T03:18:21.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 Processor -- start 2023-11-06T03:18:21.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- start start 2023-11-06T03:18:21.276 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 0x7fe6a4152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a415ce80 0x7fe6a40b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c9e20 con 0x7fe6a415caa0 2023-11-06T03:18:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c7760 con 0x7fe6a415ce80 2023-11-06T03:18:21.277 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40c7a40 con 0x7fe6a40d00b0 2023-11-06T03:18:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36370/0 (socket says 172.21.15.157:36370) 2023-11-06T03:18:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3991084755 learned_addr learned my addr 172.21.15.157:0/3991084755 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a8e24640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 0x7fe6a4152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a9625640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a415ce80 0x7fe6a40b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 msgr2=0x7fe6a4152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 0x7fe6a4152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a415ce80 msgr2=0x7fe6a40b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a415ce80 0x7fe6a40b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 -- 172.21.15.157:0/3991084755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c009b30 con 0x7fe6a415caa0 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a8e24640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 0x7fe6a4152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a3fff640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c7f0 tx=0x7fe69000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690004220 con 0x7fe6a415caa0 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6900043a0 con 0x7fe6a415caa0 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a40c7d20 con 0x7fe6a415caa0 2023-11-06T03:18:21.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690004510 con 0x7fe6a415caa0 2023-11-06T03:18:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.274+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a40c73c0 con 0x7fe6a415caa0 2023-11-06T03:18:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40c9f90 con 0x7fe6a415caa0 2023-11-06T03:18:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe690003a70 con 0x7fe6a415caa0 2023-11-06T03:18:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6a17fa640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 0x7fe688067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe690083290 con 0x7fe6a415caa0 2023-11-06T03:18:21.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6a8e24640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 0x7fe688067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.278+0000 7fe6a8e24640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 0x7fe688067f50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe698003e40 tx=0x7fe698009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.282+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe690050af0 con 0x7fe6a415caa0 2023-11-06T03:18:21.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.430+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6a40b1f00 con 0x7fe688065a90 2023-11-06T03:18:21.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6a17fa640 1 -- 172.21.15.157:0/3991084755 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311057 (secure 0 0 0) 0x7fe698005420 con 0x7fe688065a90 2023-11-06T03:18:21.454 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:21.456 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:18:21.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 msgr2=0x7fe688067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 0x7fe688067f50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe698003e40 tx=0x7fe698009040 comp rx=0 tx=0).stop 2023-11-06T03:18:21.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 msgr2=0x7fe6a415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c7f0 tx=0x7fe69000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 shutdown_connections 2023-11-06T03:18:21.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe688065a90 0x7fe688067f50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe6a415ce80 0x7fe6a40b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6a415caa0 0x7fe6a415c090 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 --2- 172.21.15.157:0/3991084755 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe6a40d00b0 0x7fe6a4152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 >> 172.21.15.157:0/3991084755 conn(0x7fe6a4070790 msgr2=0x7fe6a40cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:21.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 shutdown_connections 2023-11-06T03:18:21.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.458+0000 7fe6ab088640 1 -- 172.21.15.157:0/3991084755 wait complete. 2023-11-06T03:18:21.471 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2023-11-06T03:18:21.404127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":679520,"kb_used_data":19744,"kb_used_omap":0,"kb_used_meta":659712,"kb_avail":754295200,"statfs":{"total":773094113280,"available":772398284800,"internally_reserved":0,"allocated":20217856,"data_stored":10015314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":675545088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.693265"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120366+0000","last_change":"2023-11-06T03:18:18.168249+0000","last_active":"2023-11-06T03:18:19.120366+0000","last_peered":"2023-11-06T03:18:19.120366+0000","last_clean":"2023-11-06T03:18:19.120366+0000","last_became_active":"2023-11-06T03:18:18.167659+0000","last_became_peered":"2023-11-06T03:18:18.167659+0000","last_unstale":"2023-11-06T03:18:19.120366+0000","last_undegraded":"2023-11-06T03:18:19.120366+0000","last_fullsized":"2023-11-06T03:18:19.120366+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:36.666154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133610+0000","last_change":"2023-11-06T03:18:18.149372+0000","last_active":"2023-11-06T03:18:19.133610+0000","last_peered":"2023-11-06T03:18:19.133610+0000","last_clean":"2023-11-06T03:18:19.133610+0000","last_became_active":"2023-11-06T03:18:18.149110+0000","last_became_peered":"2023-11-06T03:18:18.149110+0000","last_unstale":"2023-11-06T03:18:19.133610+0000","last_undegraded":"2023-11-06T03:18:19.133610+0000","last_fullsized":"2023-11-06T03:18:19.133610+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:25.142841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115011+0000","last_change":"2023-11-06T03:18:18.142136+0000","last_active":"2023-11-06T03:18:19.115011+0000","last_peered":"2023-11-06T03:18:19.115011+0000","last_clean":"2023-11-06T03:18:19.115011+0000","last_became_active":"2023-11-06T03:18:18.141832+0000","last_became_peered":"2023-11-06T03:18:18.141832+0000","last_unstale":"2023-11-06T03:18:19.115011+0000","last_undegraded":"2023-11-06T03:18:19.115011+0000","last_fullsized":"2023-11-06T03:18:19.115011+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:26:28.554295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138244+0000","last_change":"2023-11-06T03:18:18.144675+0000","last_active":"2023-11-06T03:18:19.138244+0000","last_peered":"2023-11-06T03:18:19.138244+0000","last_clean":"2023-11-06T03:18:19.138244+0000","last_became_active":"2023-11-06T03:18:18.143937+0000","last_became_peered":"2023-11-06T03:18:18.143937+0000","last_unstale":"2023-11-06T03:18:19.138244+0000","last_undegraded":"2023-11-06T03:18:19.138244+0000","last_fullsized":"2023-11-06T03:18:19.138244+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135444+0000","last_change":"2023-11-06T03:18:18.144270+0000","last_active":"2023-11-06T03:18:19.135444+0000","last_peered":"2023-11-06T03:18:19.135444+0000","last_clean":"2023-11-06T03:18:19.135444+0000","last_became_active":"2023-11-06T03:18:18.143943+0000","last_became_peered":"2023-11-06T03:18:18.143943+0000","last_unstale":"2023-11-06T03:18:19.135444+0000","last_undegraded":"2023-11-06T03:18:19.135444+0000","last_fullsized":"2023-11-06T03:18:19.135444+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:10.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136969+0000","last_change":"2023-11-06T03:18:18.144879+0000","last_active":"2023-11-06T03:18:19.136969+0000","last_peered":"2023-11-06T03:18:19.136969+0000","last_clean":"2023-11-06T03:18:19.136969+0000","last_became_active":"2023-11-06T03:18:18.144119+0000","last_became_peered":"2023-11-06T03:18:18.144119+0000","last_unstale":"2023-11-06T03:18:19.136969+0000","last_undegraded":"2023-11-06T03:18:19.136969+0000","last_fullsized":"2023-11-06T03:18:19.136969+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:30.811652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135232+0000","last_change":"2023-11-06T03:18:18.143704+0000","last_active":"2023-11-06T03:18:19.135232+0000","last_peered":"2023-11-06T03:18:19.135232+0000","last_clean":"2023-11-06T03:18:19.135232+0000","last_became_active":"2023-11-06T03:18:18.143406+0000","last_became_peered":"2023-11-06T03:18:18.143406+0000","last_unstale":"2023-11-06T03:18:19.135232+0000","last_undegraded":"2023-11-06T03:18:19.135232+0000","last_fullsized":"2023-11-06T03:18:19.135232+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:16:31.706245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138113+0000","last_change":"2023-11-06T03:18:18.167330+0000","last_active":"2023-11-06T03:18:19.138113+0000","last_peered":"2023-11-06T03:18:19.138113+0000","last_clean":"2023-11-06T03:18:19.138113+0000","last_became_active":"2023-11-06T03:18:18.166668+0000","last_became_peered":"2023-11-06T03:18:18.166668+0000","last_unstale":"2023-11-06T03:18:19.138113+0000","last_undegraded":"2023-11-06T03:18:19.138113+0000","last_fullsized":"2023-11-06T03:18:19.138113+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:37.367839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135357+0000","last_change":"2023-11-06T03:18:18.142295+0000","last_active":"2023-11-06T03:18:19.135357+0000","last_peered":"2023-11-06T03:18:19.135357+0000","last_clean":"2023-11-06T03:18:19.135357+0000","last_became_active":"2023-11-06T03:18:18.142012+0000","last_became_peered":"2023-11-06T03:18:18.142012+0000","last_unstale":"2023-11-06T03:18:19.135357+0000","last_undegraded":"2023-11-06T03:18:19.135357+0000","last_fullsized":"2023-11-06T03:18:19.135357+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:25:34.379884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140699+0000","last_change":"2023-11-06T03:18:18.170704+0000","last_active":"2023-11-06T03:18:19.140699+0000","last_peered":"2023-11-06T03:18:19.140699+0000","last_clean":"2023-11-06T03:18:19.140699+0000","last_became_active":"2023-11-06T03:18:18.170251+0000","last_became_peered":"2023-11-06T03:18:18.170251+0000","last_unstale":"2023-11-06T03:18:19.140699+0000","last_undegraded":"2023-11-06T03:18:19.140699+0000","last_fullsized":"2023-11-06T03:18:19.140699+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:52.213757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115011+0000","last_change":"2023-11-06T03:18:18.173145+0000","last_active":"2023-11-06T03:18:19.115011+0000","last_peered":"2023-11-06T03:18:19.115011+0000","last_clean":"2023-11-06T03:18:19.115011+0000","last_became_active":"2023-11-06T03:18:18.172854+0000","last_became_peered":"2023-11-06T03:18:18.172854+0000","last_unstale":"2023-11-06T03:18:19.115011+0000","last_undegraded":"2023-11-06T03:18:19.115011+0000","last_fullsized":"2023-11-06T03:18:19.115011+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:47.039810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135358+0000","last_change":"2023-11-06T03:18:18.126511+0000","last_active":"2023-11-06T03:18:19.135358+0000","last_peered":"2023-11-06T03:18:19.135358+0000","last_clean":"2023-11-06T03:18:19.135358+0000","last_became_active":"2023-11-06T03:18:18.126213+0000","last_became_peered":"2023-11-06T03:18:18.126213+0000","last_unstale":"2023-11-06T03:18:19.135358+0000","last_undegraded":"2023-11-06T03:18:19.135358+0000","last_fullsized":"2023-11-06T03:18:19.135358+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:55:52.213757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137546+0000","last_change":"2023-11-06T03:18:18.145641+0000","last_active":"2023-11-06T03:18:19.137546+0000","last_peered":"2023-11-06T03:18:19.137546+0000","last_clean":"2023-11-06T03:18:19.137546+0000","last_became_active":"2023-11-06T03:18:18.144897+0000","last_became_peered":"2023-11-06T03:18:18.144897+0000","last_unstale":"2023-11-06T03:18:19.137546+0000","last_undegraded":"2023-11-06T03:18:19.137546+0000","last_fullsized":"2023-11-06T03:18:19.137546+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:19:09.769001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139642+0000","last_change":"2023-11-06T03:18:18.168993+0000","last_active":"2023-11-06T03:18:19.139642+0000","last_peered":"2023-11-06T03:18:19.139642+0000","last_clean":"2023-11-06T03:18:19.139642+0000","last_became_active":"2023-11-06T03:18:18.168073+0000","last_became_peered":"2023-11-06T03:18:18.168073+0000","last_unstale":"2023-11-06T03:18:19.139642+0000","last_undegraded":"2023-11-06T03:18:19.139642+0000","last_fullsized":"2023-11-06T03:18:19.139642+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:47.903714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137665+0000","last_change":"2023-11-06T03:18:18.168129+0000","last_active":"2023-11-06T03:18:19.137665+0000","last_peered":"2023-11-06T03:18:19.137665+0000","last_clean":"2023-11-06T03:18:19.137665+0000","last_became_active":"2023-11-06T03:18:18.167500+0000","last_became_peered":"2023-11-06T03:18:18.167500+0000","last_unstale":"2023-11-06T03:18:19.137665+0000","last_undegraded":"2023-11-06T03:18:19.137665+0000","last_fullsized":"2023-11-06T03:18:19.137665+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:30.811652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114663+0000","last_change":"2023-11-06T03:18:18.116147+0000","last_active":"2023-11-06T03:18:19.114663+0000","last_peered":"2023-11-06T03:18:19.114663+0000","last_clean":"2023-11-06T03:18:19.114663+0000","last_became_active":"2023-11-06T03:18:18.115862+0000","last_became_peered":"2023-11-06T03:18:18.115862+0000","last_unstale":"2023-11-06T03:18:19.114663+0000","last_undegraded":"2023-11-06T03:18:19.114663+0000","last_fullsized":"2023-11-06T03:18:19.114663+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:58:18.233201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135447+0000","last_change":"2023-11-06T03:18:18.166071+0000","last_active":"2023-11-06T03:18:19.135447+0000","last_peered":"2023-11-06T03:18:19.135447+0000","last_clean":"2023-11-06T03:18:19.135447+0000","last_became_active":"2023-11-06T03:18:18.165660+0000","last_became_peered":"2023-11-06T03:18:18.165660+0000","last_unstale":"2023-11-06T03:18:19.135447+0000","last_undegraded":"2023-11-06T03:18:19.135447+0000","last_fullsized":"2023-11-06T03:18:19.135447+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:36:54.297919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134876+0000","last_change":"2023-11-06T03:18:18.121145+0000","last_active":"2023-11-06T03:18:19.134876+0000","last_peered":"2023-11-06T03:18:19.134876+0000","last_clean":"2023-11-06T03:18:19.134876+0000","last_became_active":"2023-11-06T03:18:18.120922+0000","last_became_peered":"2023-11-06T03:18:18.120922+0000","last_unstale":"2023-11-06T03:18:19.134876+0000","last_undegraded":"2023-11-06T03:18:19.134876+0000","last_fullsized":"2023-11-06T03:18:19.134876+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:27:02.008853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140487+0000","last_change":"2023-11-06T03:18:18.164997+0000","last_active":"2023-11-06T03:18:19.140487+0000","last_peered":"2023-11-06T03:18:19.140487+0000","last_clean":"2023-11-06T03:18:19.140487+0000","last_became_active":"2023-11-06T03:18:18.164305+0000","last_became_peered":"2023-11-06T03:18:18.164305+0000","last_unstale":"2023-11-06T03:18:19.140487+0000","last_undegraded":"2023-11-06T03:18:19.140487+0000","last_fullsized":"2023-11-06T03:18:19.140487+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:51:10.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140289+0000","last_change":"2023-11-06T03:18:18.168423+0000","last_active":"2023-11-06T03:18:19.140289+0000","last_peered":"2023-11-06T03:18:19.140289+0000","last_clean":"2023-11-06T03:18:19.140289+0000","last_became_active":"2023-11-06T03:18:18.167817+0000","last_became_peered":"2023-11-06T03:18:18.167817+0000","last_unstale":"2023-11-06T03:18:19.140289+0000","last_undegraded":"2023-11-06T03:18:19.140289+0000","last_fullsized":"2023-11-06T03:18:19.140289+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:50:46.925922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139979+0000","last_change":"2023-11-06T03:18:18.171464+0000","last_active":"2023-11-06T03:18:19.139979+0000","last_peered":"2023-11-06T03:18:19.139979+0000","last_clean":"2023-11-06T03:18:19.139979+0000","last_became_active":"2023-11-06T03:18:18.165512+0000","last_became_peered":"2023-11-06T03:18:18.165512+0000","last_unstale":"2023-11-06T03:18:19.139979+0000","last_undegraded":"2023-11-06T03:18:19.139979+0000","last_fullsized":"2023-11-06T03:18:19.139979+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:25:37.052896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139571+0000","last_change":"2023-11-06T03:18:18.168613+0000","last_active":"2023-11-06T03:18:19.139571+0000","last_peered":"2023-11-06T03:18:19.139571+0000","last_clean":"2023-11-06T03:18:19.139571+0000","last_became_active":"2023-11-06T03:18:18.167584+0000","last_became_peered":"2023-11-06T03:18:18.167584+0000","last_unstale":"2023-11-06T03:18:19.139571+0000","last_undegraded":"2023-11-06T03:18:19.139571+0000","last_fullsized":"2023-11-06T03:18:19.139571+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:13:23.690412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133835+0000","last_change":"2023-11-06T03:18:18.142185+0000","last_active":"2023-11-06T03:18:19.133835+0000","last_peered":"2023-11-06T03:18:19.133835+0000","last_clean":"2023-11-06T03:18:19.133835+0000","last_became_active":"2023-11-06T03:18:18.141866+0000","last_became_peered":"2023-11-06T03:18:18.141866+0000","last_unstale":"2023-11-06T03:18:19.133835+0000","last_undegraded":"2023-11-06T03:18:19.133835+0000","last_fullsized":"2023-11-06T03:18:19.133835+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:29:09.439626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134294+0000","last_change":"2023-11-06T03:18:18.138324+0000","last_active":"2023-11-06T03:18:19.134294+0000","last_peered":"2023-11-06T03:18:19.134294+0000","last_clean":"2023-11-06T03:18:19.134294+0000","last_became_active":"2023-11-06T03:18:18.137983+0000","last_became_peered":"2023-11-06T03:18:18.137983+0000","last_unstale":"2023-11-06T03:18:19.134294+0000","last_undegraded":"2023-11-06T03:18:19.134294+0000","last_fullsized":"2023-11-06T03:18:19.134294+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:13.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135529+0000","last_change":"2023-11-06T03:18:18.124053+0000","last_active":"2023-11-06T03:18:19.135529+0000","last_peered":"2023-11-06T03:18:19.135529+0000","last_clean":"2023-11-06T03:18:19.135529+0000","last_became_active":"2023-11-06T03:18:18.123508+0000","last_became_peered":"2023-11-06T03:18:18.123508+0000","last_unstale":"2023-11-06T03:18:19.135529+0000","last_undegraded":"2023-11-06T03:18:19.135529+0000","last_fullsized":"2023-11-06T03:18:19.135529+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:37:03.885752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119942+0000","last_change":"2023-11-06T03:18:18.171722+0000","last_active":"2023-11-06T03:18:19.119942+0000","last_peered":"2023-11-06T03:18:19.119942+0000","last_clean":"2023-11-06T03:18:19.119942+0000","last_became_active":"2023-11-06T03:18:18.171263+0000","last_became_peered":"2023-11-06T03:18:18.171263+0000","last_unstale":"2023-11-06T03:18:19.119942+0000","last_undegraded":"2023-11-06T03:18:19.119942+0000","last_fullsized":"2023-11-06T03:18:19.119942+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:47.503452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115395+0000","last_change":"2023-11-06T03:18:18.172851+0000","last_active":"2023-11-06T03:18:19.115395+0000","last_peered":"2023-11-06T03:18:19.115395+0000","last_clean":"2023-11-06T03:18:19.115395+0000","last_became_active":"2023-11-06T03:18:18.172473+0000","last_became_peered":"2023-11-06T03:18:18.172473+0000","last_unstale":"2023-11-06T03:18:19.115395+0000","last_undegraded":"2023-11-06T03:18:19.115395+0000","last_fullsized":"2023-11-06T03:18:19.115395+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:48:12.235892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135885+0000","last_change":"2023-11-06T03:18:18.144877+0000","last_active":"2023-11-06T03:18:19.135885+0000","last_peered":"2023-11-06T03:18:19.135885+0000","last_clean":"2023-11-06T03:18:19.135885+0000","last_became_active":"2023-11-06T03:18:18.144604+0000","last_became_peered":"2023-11-06T03:18:18.144604+0000","last_unstale":"2023-11-06T03:18:19.135885+0000","last_undegraded":"2023-11-06T03:18:19.135885+0000","last_fullsized":"2023-11-06T03:18:19.135885+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:34:47.903714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134404+0000","last_change":"2023-11-06T03:18:18.121982+0000","last_active":"2023-11-06T03:18:19.134404+0000","last_peered":"2023-11-06T03:18:19.134404+0000","last_clean":"2023-11-06T03:18:19.134404+0000","last_became_active":"2023-11-06T03:18:18.121610+0000","last_became_peered":"2023-11-06T03:18:18.121610+0000","last_unstale":"2023-11-06T03:18:19.134404+0000","last_undegraded":"2023-11-06T03:18:19.134404+0000","last_fullsized":"2023-11-06T03:18:19.134404+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:10:32.125009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137892+0000","last_change":"2023-11-06T03:18:18.113711+0000","last_active":"2023-11-06T03:18:19.137892+0000","last_peered":"2023-11-06T03:18:19.137892+0000","last_clean":"2023-11-06T03:18:19.137892+0000","last_became_active":"2023-11-06T03:18:18.112940+0000","last_became_peered":"2023-11-06T03:18:18.112940+0000","last_unstale":"2023-11-06T03:18:19.137892+0000","last_undegraded":"2023-11-06T03:18:19.137892+0000","last_fullsized":"2023-11-06T03:18:19.137892+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137182+0000","last_change":"2023-11-06T03:18:18.144724+0000","last_active":"2023-11-06T03:18:19.137182+0000","last_peered":"2023-11-06T03:18:19.137182+0000","last_clean":"2023-11-06T03:18:19.137182+0000","last_became_active":"2023-11-06T03:18:18.144024+0000","last_became_peered":"2023-11-06T03:18:18.144024+0000","last_unstale":"2023-11-06T03:18:19.137182+0000","last_undegraded":"2023-11-06T03:18:19.137182+0000","last_fullsized":"2023-11-06T03:18:19.137182+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:11:56.799818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136856+0000","last_change":"2023-11-06T03:18:18.155530+0000","last_active":"2023-11-06T03:18:19.136856+0000","last_peered":"2023-11-06T03:18:19.136856+0000","last_clean":"2023-11-06T03:18:19.136856+0000","last_became_active":"2023-11-06T03:18:18.154754+0000","last_became_peered":"2023-11-06T03:18:18.154754+0000","last_unstale":"2023-11-06T03:18:19.136856+0000","last_undegraded":"2023-11-06T03:18:19.136856+0000","last_fullsized":"2023-11-06T03:18:19.136856+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:54:02.138784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137463+0000","last_change":"2023-11-06T03:18:18.120085+0000","last_active":"2023-11-06T03:18:19.137463+0000","last_peered":"2023-11-06T03:18:19.137463+0000","last_clean":"2023-11-06T03:18:19.137463+0000","last_became_active":"2023-11-06T03:18:18.119375+0000","last_became_peered":"2023-11-06T03:18:18.119375+0000","last_unstale":"2023-11-06T03:18:19.137463+0000","last_undegraded":"2023-11-06T03:18:19.137463+0000","last_fullsized":"2023-11-06T03:18:19.137463+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:19.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136114+0000","last_change":"2023-11-06T03:18:17.768948+0000","last_active":"2023-11-06T03:18:19.136114+0000","last_peered":"2023-11-06T03:18:19.136114+0000","last_clean":"2023-11-06T03:18:19.136114+0000","last_became_active":"2023-11-06T03:18:17.768462+0000","last_became_peered":"2023-11-06T03:18:17.768462+0000","last_unstale":"2023-11-06T03:18:19.136114+0000","last_undegraded":"2023-11-06T03:18:19.136114+0000","last_fullsized":"2023-11-06T03:18:19.136114+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:10:58.578257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115294+0000","last_change":"2023-11-06T03:18:18.127656+0000","last_active":"2023-11-06T03:18:19.115294+0000","last_peered":"2023-11-06T03:18:19.115294+0000","last_clean":"2023-11-06T03:18:19.115294+0000","last_became_active":"2023-11-06T03:18:18.127382+0000","last_became_peered":"2023-11-06T03:18:18.127382+0000","last_unstale":"2023-11-06T03:18:19.115294+0000","last_undegraded":"2023-11-06T03:18:19.115294+0000","last_fullsized":"2023-11-06T03:18:19.115294+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:42:19.379590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138814+0000","last_change":"2023-11-06T03:18:17.763898+0000","last_active":"2023-11-06T03:18:19.138814+0000","last_peered":"2023-11-06T03:18:19.138814+0000","last_clean":"2023-11-06T03:18:19.138814+0000","last_became_active":"2023-11-06T03:18:17.763372+0000","last_became_peered":"2023-11-06T03:18:17.763372+0000","last_unstale":"2023-11-06T03:18:19.138814+0000","last_undegraded":"2023-11-06T03:18:19.138814+0000","last_fullsized":"2023-11-06T03:18:19.138814+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:44:38.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134405+0000","last_change":"2023-11-06T03:18:18.153919+0000","last_active":"2023-11-06T03:18:19.134405+0000","last_peered":"2023-11-06T03:18:19.134405+0000","last_clean":"2023-11-06T03:18:19.134405+0000","last_became_active":"2023-11-06T03:18:18.153636+0000","last_became_peered":"2023-11-06T03:18:18.153636+0000","last_unstale":"2023-11-06T03:18:19.134405+0000","last_undegraded":"2023-11-06T03:18:19.134405+0000","last_fullsized":"2023-11-06T03:18:19.134405+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:41:22.009161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"188'3","reported_seq":14,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134869+0000","last_change":"2023-11-06T03:18:17.742723+0000","last_active":"2023-11-06T03:18:19.134869+0000","last_peered":"2023-11-06T03:18:19.134869+0000","last_clean":"2023-11-06T03:18:19.134869+0000","last_became_active":"2023-11-06T03:18:17.742400+0000","last_became_peered":"2023-11-06T03:18:17.742400+0000","last_unstale":"2023-11-06T03:18:19.134869+0000","last_undegraded":"2023-11-06T03:18:19.134869+0000","last_fullsized":"2023-11-06T03:18:19.134869+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:39:24.339478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135577+0000","last_change":"2023-11-06T03:18:18.171521+0000","last_active":"2023-11-06T03:18:19.135577+0000","last_peered":"2023-11-06T03:18:19.135577+0000","last_clean":"2023-11-06T03:18:19.135577+0000","last_became_active":"2023-11-06T03:18:18.171217+0000","last_became_peered":"2023-11-06T03:18:18.171217+0000","last_unstale":"2023-11-06T03:18:19.135577+0000","last_undegraded":"2023-11-06T03:18:19.135577+0000","last_fullsized":"2023-11-06T03:18:19.135577+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:28:37.367839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"188'6","reported_seq":17,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138516+0000","last_change":"2023-11-06T03:18:17.771139+0000","last_active":"2023-11-06T03:18:19.138516+0000","last_peered":"2023-11-06T03:18:19.138516+0000","last_clean":"2023-11-06T03:18:19.138516+0000","last_became_active":"2023-11-06T03:18:17.770775+0000","last_became_peered":"2023-11-06T03:18:17.770775+0000","last_unstale":"2023-11-06T03:18:19.138516+0000","last_undegraded":"2023-11-06T03:18:19.138516+0000","last_fullsized":"2023-11-06T03:18:19.138516+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:28.242405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133833+0000","last_change":"2023-11-06T03:18:18.127541+0000","last_active":"2023-11-06T03:18:19.133833+0000","last_peered":"2023-11-06T03:18:19.133833+0000","last_clean":"2023-11-06T03:18:19.133833+0000","last_became_active":"2023-11-06T03:18:18.127006+0000","last_became_peered":"2023-11-06T03:18:18.127006+0000","last_unstale":"2023-11-06T03:18:19.133833+0000","last_undegraded":"2023-11-06T03:18:19.133833+0000","last_fullsized":"2023-11-06T03:18:19.133833+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:04.360886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133835+0000","last_change":"2023-11-06T03:18:17.750251+0000","last_active":"2023-11-06T03:18:19.133835+0000","last_peered":"2023-11-06T03:18:19.133835+0000","last_clean":"2023-11-06T03:18:19.133835+0000","last_became_active":"2023-11-06T03:18:17.749673+0000","last_became_peered":"2023-11-06T03:18:17.749673+0000","last_unstale":"2023-11-06T03:18:19.133835+0000","last_undegraded":"2023-11-06T03:18:19.133835+0000","last_fullsized":"2023-11-06T03:18:19.133835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:14:46.035903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.115382+0000","last_change":"2023-11-06T03:18:18.120600+0000","last_active":"2023-11-06T03:18:19.115382+0000","last_peered":"2023-11-06T03:18:19.115382+0000","last_clean":"2023-11-06T03:18:19.115382+0000","last_became_active":"2023-11-06T03:18:18.120335+0000","last_became_peered":"2023-11-06T03:18:18.120335+0000","last_unstale":"2023-11-06T03:18:19.115382+0000","last_undegraded":"2023-11-06T03:18:19.115382+0000","last_fullsized":"2023-11-06T03:18:19.115382+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:28:03.160587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114741+0000","last_change":"2023-11-06T03:18:17.734391+0000","last_active":"2023-11-06T03:18:19.114741+0000","last_peered":"2023-11-06T03:18:19.114741+0000","last_clean":"2023-11-06T03:18:19.114741+0000","last_became_active":"2023-11-06T03:18:17.734027+0000","last_became_peered":"2023-11-06T03:18:17.734027+0000","last_unstale":"2023-11-06T03:18:19.114741+0000","last_undegraded":"2023-11-06T03:18:19.114741+0000","last_fullsized":"2023-11-06T03:18:19.114741+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:49.437013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134956+0000","last_change":"2023-11-06T03:18:18.128385+0000","last_active":"2023-11-06T03:18:19.134956+0000","last_peered":"2023-11-06T03:18:19.134956+0000","last_clean":"2023-11-06T03:18:19.134956+0000","last_became_active":"2023-11-06T03:18:18.128126+0000","last_became_peered":"2023-11-06T03:18:18.128126+0000","last_unstale":"2023-11-06T03:18:19.134956+0000","last_undegraded":"2023-11-06T03:18:19.134956+0000","last_fullsized":"2023-11-06T03:18:19.134956+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:58:40.707472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134307+0000","last_change":"2023-11-06T03:18:17.760839+0000","last_active":"2023-11-06T03:18:19.134307+0000","last_peered":"2023-11-06T03:18:19.134307+0000","last_clean":"2023-11-06T03:18:19.134307+0000","last_became_active":"2023-11-06T03:18:17.760493+0000","last_became_peered":"2023-11-06T03:18:17.760493+0000","last_unstale":"2023-11-06T03:18:19.134307+0000","last_undegraded":"2023-11-06T03:18:19.134307+0000","last_fullsized":"2023-11-06T03:18:19.134307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:35:47.034347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138726+0000","last_change":"2023-11-06T03:18:18.149464+0000","last_active":"2023-11-06T03:18:19.138726+0000","last_peered":"2023-11-06T03:18:19.138726+0000","last_clean":"2023-11-06T03:18:19.138726+0000","last_became_active":"2023-11-06T03:18:18.145679+0000","last_became_peered":"2023-11-06T03:18:18.145679+0000","last_unstale":"2023-11-06T03:18:19.138726+0000","last_undegraded":"2023-11-06T03:18:19.138726+0000","last_fullsized":"2023-11-06T03:18:19.138726+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:11.795214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"188'8","reported_seq":19,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134933+0000","last_change":"2023-11-06T03:18:17.767978+0000","last_active":"2023-11-06T03:18:19.134933+0000","last_peered":"2023-11-06T03:18:19.134933+0000","last_clean":"2023-11-06T03:18:19.134933+0000","last_became_active":"2023-11-06T03:18:17.767592+0000","last_became_peered":"2023-11-06T03:18:17.767592+0000","last_unstale":"2023-11-06T03:18:19.134933+0000","last_undegraded":"2023-11-06T03:18:19.134933+0000","last_fullsized":"2023-11-06T03:18:19.134933+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:15:27.234205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138200+0000","last_change":"2023-11-06T03:18:18.166863+0000","last_active":"2023-11-06T03:18:19.138200+0000","last_peered":"2023-11-06T03:18:19.138200+0000","last_clean":"2023-11-06T03:18:19.138200+0000","last_became_active":"2023-11-06T03:18:18.165294+0000","last_became_peered":"2023-11-06T03:18:18.165294+0000","last_unstale":"2023-11-06T03:18:19.138200+0000","last_undegraded":"2023-11-06T03:18:19.138200+0000","last_fullsized":"2023-11-06T03:18:19.138200+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:20:13.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120396+0000","last_change":"2023-11-06T03:18:17.751838+0000","last_active":"2023-11-06T03:18:19.120396+0000","last_peered":"2023-11-06T03:18:19.120396+0000","last_clean":"2023-11-06T03:18:19.120396+0000","last_became_active":"2023-11-06T03:18:17.751022+0000","last_became_peered":"2023-11-06T03:18:17.751022+0000","last_unstale":"2023-11-06T03:18:19.120396+0000","last_undegraded":"2023-11-06T03:18:19.120396+0000","last_fullsized":"2023-11-06T03:18:19.120396+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:22:01.348159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119215+0000","last_change":"2023-11-06T03:18:18.170543+0000","last_active":"2023-11-06T03:18:19.119215+0000","last_peered":"2023-11-06T03:18:19.119215+0000","last_clean":"2023-11-06T03:18:19.119215+0000","last_became_active":"2023-11-06T03:18:18.169976+0000","last_became_peered":"2023-11-06T03:18:18.169976+0000","last_unstale":"2023-11-06T03:18:19.119215+0000","last_undegraded":"2023-11-06T03:18:19.119215+0000","last_fullsized":"2023-11-06T03:18:19.119215+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120186+0000","last_change":"2023-11-06T03:18:17.760121+0000","last_active":"2023-11-06T03:18:19.120186+0000","last_peered":"2023-11-06T03:18:19.120186+0000","last_clean":"2023-11-06T03:18:19.120186+0000","last_became_active":"2023-11-06T03:18:17.758821+0000","last_became_peered":"2023-11-06T03:18:17.758821+0000","last_unstale":"2023-11-06T03:18:19.120186+0000","last_undegraded":"2023-11-06T03:18:19.120186+0000","last_fullsized":"2023-11-06T03:18:19.120186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:45:55.680814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119490+0000","last_change":"2023-11-06T03:18:18.142482+0000","last_active":"2023-11-06T03:18:19.119490+0000","last_peered":"2023-11-06T03:18:19.119490+0000","last_clean":"2023-11-06T03:18:19.119490+0000","last_became_active":"2023-11-06T03:18:18.141760+0000","last_became_peered":"2023-11-06T03:18:18.141760+0000","last_unstale":"2023-11-06T03:18:19.119490+0000","last_undegraded":"2023-11-06T03:18:19.119490+0000","last_fullsized":"2023-11-06T03:18:19.119490+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:30.607519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133914+0000","last_change":"2023-11-06T03:18:17.745717+0000","last_active":"2023-11-06T03:18:19.133914+0000","last_peered":"2023-11-06T03:18:19.133914+0000","last_clean":"2023-11-06T03:18:19.133914+0000","last_became_active":"2023-11-06T03:18:17.745430+0000","last_became_peered":"2023-11-06T03:18:17.745430+0000","last_unstale":"2023-11-06T03:18:19.133914+0000","last_undegraded":"2023-11-06T03:18:19.133914+0000","last_fullsized":"2023-11-06T03:18:19.133914+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:42:24.132619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140283+0000","last_change":"2023-11-06T03:18:18.166968+0000","last_active":"2023-11-06T03:18:19.140283+0000","last_peered":"2023-11-06T03:18:19.140283+0000","last_clean":"2023-11-06T03:18:19.140283+0000","last_became_active":"2023-11-06T03:18:18.164788+0000","last_became_peered":"2023-11-06T03:18:18.164788+0000","last_unstale":"2023-11-06T03:18:19.140283+0000","last_undegraded":"2023-11-06T03:18:19.140283+0000","last_fullsized":"2023-11-06T03:18:19.140283+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:24.326499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135601+0000","last_change":"2023-11-06T03:18:17.751638+0000","last_active":"2023-11-06T03:18:19.135601+0000","last_peered":"2023-11-06T03:18:19.135601+0000","last_clean":"2023-11-06T03:18:19.135601+0000","last_became_active":"2023-11-06T03:18:17.751360+0000","last_became_peered":"2023-11-06T03:18:17.751360+0000","last_unstale":"2023-11-06T03:18:19.135601+0000","last_undegraded":"2023-11-06T03:18:19.135601+0000","last_fullsized":"2023-11-06T03:18:19.135601+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:52:17.458421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133763+0000","last_change":"2023-11-06T03:18:18.142949+0000","last_active":"2023-11-06T03:18:19.133763+0000","last_peered":"2023-11-06T03:18:19.133763+0000","last_clean":"2023-11-06T03:18:19.133763+0000","last_became_active":"2023-11-06T03:18:18.142687+0000","last_became_peered":"2023-11-06T03:18:18.142687+0000","last_unstale":"2023-11-06T03:18:19.133763+0000","last_undegraded":"2023-11-06T03:18:19.133763+0000","last_fullsized":"2023-11-06T03:18:19.133763+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:55:31.054492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135668+0000","last_change":"2023-11-06T03:18:17.735782+0000","last_active":"2023-11-06T03:18:19.135668+0000","last_peered":"2023-11-06T03:18:19.135668+0000","last_clean":"2023-11-06T03:18:19.135668+0000","last_became_active":"2023-11-06T03:18:17.735371+0000","last_became_peered":"2023-11-06T03:18:17.735371+0000","last_unstale":"2023-11-06T03:18:19.135668+0000","last_undegraded":"2023-11-06T03:18:19.135668+0000","last_fullsized":"2023-11-06T03:18:19.135668+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:04:26.532987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138140+0000","last_change":"2023-11-06T03:18:18.149057+0000","last_active":"2023-11-06T03:18:19.138140+0000","last_peered":"2023-11-06T03:18:19.138140+0000","last_clean":"2023-11-06T03:18:19.138140+0000","last_became_active":"2023-11-06T03:18:18.148367+0000","last_became_peered":"2023-11-06T03:18:18.148367+0000","last_unstale":"2023-11-06T03:18:19.138140+0000","last_undegraded":"2023-11-06T03:18:19.138140+0000","last_fullsized":"2023-11-06T03:18:19.138140+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:32:50.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133923+0000","last_change":"2023-11-06T03:18:17.733090+0000","last_active":"2023-11-06T03:18:19.133923+0000","last_peered":"2023-11-06T03:18:19.133923+0000","last_clean":"2023-11-06T03:18:19.133923+0000","last_became_active":"2023-11-06T03:18:17.732466+0000","last_became_peered":"2023-11-06T03:18:17.732466+0000","last_unstale":"2023-11-06T03:18:19.133923+0000","last_undegraded":"2023-11-06T03:18:19.133923+0000","last_fullsized":"2023-11-06T03:18:19.133923+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:11:09.957215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135185+0000","last_change":"2023-11-06T03:18:18.142939+0000","last_active":"2023-11-06T03:18:19.135185+0000","last_peered":"2023-11-06T03:18:19.135185+0000","last_clean":"2023-11-06T03:18:19.135185+0000","last_became_active":"2023-11-06T03:18:18.142656+0000","last_became_peered":"2023-11-06T03:18:18.142656+0000","last_unstale":"2023-11-06T03:18:19.135185+0000","last_undegraded":"2023-11-06T03:18:19.135185+0000","last_fullsized":"2023-11-06T03:18:19.135185+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:59:20.459898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138067+0000","last_change":"2023-11-06T03:18:17.758130+0000","last_active":"2023-11-06T03:18:19.138067+0000","last_peered":"2023-11-06T03:18:19.138067+0000","last_clean":"2023-11-06T03:18:19.138067+0000","last_became_active":"2023-11-06T03:18:17.757349+0000","last_became_peered":"2023-11-06T03:18:17.757349+0000","last_unstale":"2023-11-06T03:18:19.138067+0000","last_undegraded":"2023-11-06T03:18:19.138067+0000","last_fullsized":"2023-11-06T03:18:19.138067+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:27:41.842899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.141013+0000","last_change":"2023-11-06T03:18:18.165210+0000","last_active":"2023-11-06T03:18:19.141013+0000","last_peered":"2023-11-06T03:18:19.141013+0000","last_clean":"2023-11-06T03:18:19.141013+0000","last_became_active":"2023-11-06T03:18:18.164338+0000","last_became_peered":"2023-11-06T03:18:18.164338+0000","last_unstale":"2023-11-06T03:18:19.141013+0000","last_undegraded":"2023-11-06T03:18:19.141013+0000","last_fullsized":"2023-11-06T03:18:19.141013+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:40:55.290584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.118958+0000","last_change":"2023-11-06T03:18:17.745749+0000","last_active":"2023-11-06T03:18:19.118958+0000","last_peered":"2023-11-06T03:18:19.118958+0000","last_clean":"2023-11-06T03:18:19.118958+0000","last_became_active":"2023-11-06T03:18:17.741884+0000","last_became_peered":"2023-11-06T03:18:17.741884+0000","last_unstale":"2023-11-06T03:18:19.118958+0000","last_undegraded":"2023-11-06T03:18:19.118958+0000","last_fullsized":"2023-11-06T03:18:19.118958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:25:08.259498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138355+0000","last_change":"2023-11-06T03:18:18.164748+0000","last_active":"2023-11-06T03:18:19.138355+0000","last_peered":"2023-11-06T03:18:19.138355+0000","last_clean":"2023-11-06T03:18:19.138355+0000","last_became_active":"2023-11-06T03:18:18.164173+0000","last_became_peered":"2023-11-06T03:18:18.164173+0000","last_unstale":"2023-11-06T03:18:19.138355+0000","last_undegraded":"2023-11-06T03:18:19.138355+0000","last_fullsized":"2023-11-06T03:18:19.138355+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:58:34.089014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135697+0000","last_change":"2023-11-06T03:18:17.768585+0000","last_active":"2023-11-06T03:18:19.135697+0000","last_peered":"2023-11-06T03:18:19.135697+0000","last_clean":"2023-11-06T03:18:19.135697+0000","last_became_active":"2023-11-06T03:18:17.768193+0000","last_became_peered":"2023-11-06T03:18:17.768193+0000","last_unstale":"2023-11-06T03:18:19.135697+0000","last_undegraded":"2023-11-06T03:18:19.135697+0000","last_fullsized":"2023-11-06T03:18:19.135697+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:01:38.662396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114679+0000","last_change":"2023-11-06T03:18:18.166524+0000","last_active":"2023-11-06T03:18:19.114679+0000","last_peered":"2023-11-06T03:18:19.114679+0000","last_clean":"2023-11-06T03:18:19.114679+0000","last_became_active":"2023-11-06T03:18:18.166309+0000","last_became_peered":"2023-11-06T03:18:18.166309+0000","last_unstale":"2023-11-06T03:18:19.114679+0000","last_undegraded":"2023-11-06T03:18:19.114679+0000","last_fullsized":"2023-11-06T03:18:19.114679+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:28.300548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140790+0000","last_change":"2023-11-06T03:18:17.758544+0000","last_active":"2023-11-06T03:18:19.140790+0000","last_peered":"2023-11-06T03:18:19.140790+0000","last_clean":"2023-11-06T03:18:19.140790+0000","last_became_active":"2023-11-06T03:18:17.757834+0000","last_became_peered":"2023-11-06T03:18:17.757834+0000","last_unstale":"2023-11-06T03:18:19.140790+0000","last_undegraded":"2023-11-06T03:18:19.140790+0000","last_fullsized":"2023-11-06T03:18:19.140790+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:49:00.089172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134476+0000","last_change":"2023-11-06T03:18:18.127839+0000","last_active":"2023-11-06T03:18:19.134476+0000","last_peered":"2023-11-06T03:18:19.134476+0000","last_clean":"2023-11-06T03:18:19.134476+0000","last_became_active":"2023-11-06T03:18:18.127547+0000","last_became_peered":"2023-11-06T03:18:18.127547+0000","last_unstale":"2023-11-06T03:18:19.134476+0000","last_undegraded":"2023-11-06T03:18:19.134476+0000","last_fullsized":"2023-11-06T03:18:19.134476+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:21:12.037444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138965+0000","last_change":"2023-11-06T03:18:17.746341+0000","last_active":"2023-11-06T03:18:19.138965+0000","last_peered":"2023-11-06T03:18:19.138965+0000","last_clean":"2023-11-06T03:18:19.138965+0000","last_became_active":"2023-11-06T03:18:17.745581+0000","last_became_peered":"2023-11-06T03:18:17.745581+0000","last_unstale":"2023-11-06T03:18:19.138965+0000","last_undegraded":"2023-11-06T03:18:19.138965+0000","last_fullsized":"2023-11-06T03:18:19.138965+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:12:29.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119895+0000","last_change":"2023-11-06T03:18:18.147737+0000","last_active":"2023-11-06T03:18:19.119895+0000","last_peered":"2023-11-06T03:18:19.119895+0000","last_clean":"2023-11-06T03:18:19.119895+0000","last_became_active":"2023-11-06T03:18:18.146988+0000","last_became_peered":"2023-11-06T03:18:18.146988+0000","last_unstale":"2023-11-06T03:18:19.119895+0000","last_undegraded":"2023-11-06T03:18:19.119895+0000","last_fullsized":"2023-11-06T03:18:19.119895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:17:29.368572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119757+0000","last_change":"2023-11-06T03:18:17.758659+0000","last_active":"2023-11-06T03:18:19.119757+0000","last_peered":"2023-11-06T03:18:19.119757+0000","last_clean":"2023-11-06T03:18:19.119757+0000","last_became_active":"2023-11-06T03:18:17.751005+0000","last_became_peered":"2023-11-06T03:18:17.751005+0000","last_unstale":"2023-11-06T03:18:19.119757+0000","last_undegraded":"2023-11-06T03:18:19.119757+0000","last_fullsized":"2023-11-06T03:18:19.119757+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:18:28.242405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140305+0000","last_change":"2023-11-06T03:18:18.170210+0000","last_active":"2023-11-06T03:18:19.140305+0000","last_peered":"2023-11-06T03:18:19.140305+0000","last_clean":"2023-11-06T03:18:19.140305+0000","last_became_active":"2023-11-06T03:18:18.165484+0000","last_became_peered":"2023-11-06T03:18:18.165484+0000","last_unstale":"2023-11-06T03:18:19.140305+0000","last_undegraded":"2023-11-06T03:18:19.140305+0000","last_fullsized":"2023-11-06T03:18:19.140305+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:16:31.706245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.140240+0000","last_change":"2023-11-06T03:18:17.748639+0000","last_active":"2023-11-06T03:18:19.140240+0000","last_peered":"2023-11-06T03:18:19.140240+0000","last_clean":"2023-11-06T03:18:19.140240+0000","last_became_active":"2023-11-06T03:18:17.744033+0000","last_became_peered":"2023-11-06T03:18:17.744033+0000","last_unstale":"2023-11-06T03:18:19.140240+0000","last_undegraded":"2023-11-06T03:18:19.140240+0000","last_fullsized":"2023-11-06T03:18:19.140240+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:52:26.645404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138395+0000","last_change":"2023-11-06T03:18:18.149186+0000","last_active":"2023-11-06T03:18:19.138395+0000","last_peered":"2023-11-06T03:18:19.138395+0000","last_clean":"2023-11-06T03:18:19.138395+0000","last_became_active":"2023-11-06T03:18:18.148474+0000","last_became_peered":"2023-11-06T03:18:18.148474+0000","last_unstale":"2023-11-06T03:18:19.138395+0000","last_undegraded":"2023-11-06T03:18:19.138395+0000","last_fullsized":"2023-11-06T03:18:19.138395+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:58:19.555494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134509+0000","last_change":"2023-11-06T03:18:17.742453+0000","last_active":"2023-11-06T03:18:19.134509+0000","last_peered":"2023-11-06T03:18:19.134509+0000","last_clean":"2023-11-06T03:18:19.134509+0000","last_became_active":"2023-11-06T03:18:17.742088+0000","last_became_peered":"2023-11-06T03:18:17.742088+0000","last_unstale":"2023-11-06T03:18:19.134509+0000","last_undegraded":"2023-11-06T03:18:19.134509+0000","last_fullsized":"2023-11-06T03:18:19.134509+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:37:02.598746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138636+0000","last_change":"2023-11-06T03:18:18.154392+0000","last_active":"2023-11-06T03:18:19.138636+0000","last_peered":"2023-11-06T03:18:19.138636+0000","last_clean":"2023-11-06T03:18:19.138636+0000","last_became_active":"2023-11-06T03:18:18.151240+0000","last_became_peered":"2023-11-06T03:18:18.151240+0000","last_unstale":"2023-11-06T03:18:19.138636+0000","last_undegraded":"2023-11-06T03:18:19.138636+0000","last_fullsized":"2023-11-06T03:18:19.138636+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119255+0000","last_change":"2023-11-06T03:18:17.770837+0000","last_active":"2023-11-06T03:18:19.119255+0000","last_peered":"2023-11-06T03:18:19.119255+0000","last_clean":"2023-11-06T03:18:19.119255+0000","last_became_active":"2023-11-06T03:18:17.770040+0000","last_became_peered":"2023-11-06T03:18:17.770040+0000","last_unstale":"2023-11-06T03:18:19.119255+0000","last_undegraded":"2023-11-06T03:18:19.119255+0000","last_fullsized":"2023-11-06T03:18:19.119255+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:23:22.258878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114922+0000","last_change":"2023-11-06T03:18:18.166588+0000","last_active":"2023-11-06T03:18:19.114922+0000","last_peered":"2023-11-06T03:18:19.114922+0000","last_clean":"2023-11-06T03:18:19.114922+0000","last_became_active":"2023-11-06T03:18:18.166305+0000","last_became_peered":"2023-11-06T03:18:18.166305+0000","last_unstale":"2023-11-06T03:18:19.114922+0000","last_undegraded":"2023-11-06T03:18:19.114922+0000","last_fullsized":"2023-11-06T03:18:19.114922+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:31:22.806019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134952+0000","last_change":"2023-11-06T03:18:17.767954+0000","last_active":"2023-11-06T03:18:19.134952+0000","last_peered":"2023-11-06T03:18:19.134952+0000","last_clean":"2023-11-06T03:18:19.134952+0000","last_became_active":"2023-11-06T03:18:17.767597+0000","last_became_peered":"2023-11-06T03:18:17.767597+0000","last_unstale":"2023-11-06T03:18:19.134952+0000","last_undegraded":"2023-11-06T03:18:19.134952+0000","last_fullsized":"2023-11-06T03:18:19.134952+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:49:08.002833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119800+0000","last_change":"2023-11-06T03:18:18.170889+0000","last_active":"2023-11-06T03:18:19.119800+0000","last_peered":"2023-11-06T03:18:19.119800+0000","last_clean":"2023-11-06T03:18:19.119800+0000","last_became_active":"2023-11-06T03:18:18.169485+0000","last_became_peered":"2023-11-06T03:18:18.169485+0000","last_unstale":"2023-11-06T03:18:19.119800+0000","last_undegraded":"2023-11-06T03:18:19.119800+0000","last_fullsized":"2023-11-06T03:18:19.119800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T08:57:41.127985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134465+0000","last_change":"2023-11-06T03:18:17.743115+0000","last_active":"2023-11-06T03:18:19.134465+0000","last_peered":"2023-11-06T03:18:19.134465+0000","last_clean":"2023-11-06T03:18:19.134465+0000","last_became_active":"2023-11-06T03:18:17.742780+0000","last_became_peered":"2023-11-06T03:18:17.742780+0000","last_unstale":"2023-11-06T03:18:19.134465+0000","last_undegraded":"2023-11-06T03:18:19.134465+0000","last_fullsized":"2023-11-06T03:18:19.134465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:26:55.231407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.123656+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:18:18.123656+0000","last_peered":"2023-11-06T03:18:18.123656+0000","last_clean":"2023-11-06T03:18:18.123656+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:18:18.123656+0000","last_undegraded":"2023-11-06T03:18:18.123656+0000","last_fullsized":"2023-11-06T03:18:18.123656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.143716+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:18:18.143716+0000","last_peered":"2023-11-06T03:18:18.143716+0000","last_clean":"2023-11-06T03:18:18.143716+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:18:18.143716+0000","last_undegraded":"2023-11-06T03:18:18.143716+0000","last_fullsized":"2023-11-06T03:18:18.143716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.138855+0000","last_change":"2023-11-06T03:18:18.129150+0000","last_active":"2023-11-06T03:18:19.138855+0000","last_peered":"2023-11-06T03:18:19.138855+0000","last_clean":"2023-11-06T03:18:19.138855+0000","last_became_active":"2023-11-06T03:18:18.128441+0000","last_became_peered":"2023-11-06T03:18:18.128441+0000","last_unstale":"2023-11-06T03:18:19.138855+0000","last_undegraded":"2023-11-06T03:18:19.138855+0000","last_fullsized":"2023-11-06T03:18:19.138855+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:13.175806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.136919+0000","last_change":"2023-11-06T03:18:17.763803+0000","last_active":"2023-11-06T03:18:19.136919+0000","last_peered":"2023-11-06T03:18:19.136919+0000","last_clean":"2023-11-06T03:18:19.136919+0000","last_became_active":"2023-11-06T03:18:17.763044+0000","last_became_peered":"2023-11-06T03:18:17.763044+0000","last_unstale":"2023-11-06T03:18:19.136919+0000","last_undegraded":"2023-11-06T03:18:19.136919+0000","last_fullsized":"2023-11-06T03:18:19.136919+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:02:25.877643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.123408+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:18:18.123408+0000","last_peered":"2023-11-06T03:18:18.123408+0000","last_clean":"2023-11-06T03:18:18.123408+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:18:18.123408+0000","last_undegraded":"2023-11-06T03:18:18.123408+0000","last_fullsized":"2023-11-06T03:18:18.123408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135615+0000","last_change":"2023-11-06T03:18:18.144417+0000","last_active":"2023-11-06T03:18:19.135615+0000","last_peered":"2023-11-06T03:18:19.135615+0000","last_clean":"2023-11-06T03:18:19.135615+0000","last_became_active":"2023-11-06T03:18:18.144163+0000","last_became_peered":"2023-11-06T03:18:18.144163+0000","last_unstale":"2023-11-06T03:18:19.135615+0000","last_undegraded":"2023-11-06T03:18:19.135615+0000","last_fullsized":"2023-11-06T03:18:19.135615+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T06:59:40.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.114663+0000","last_change":"2023-11-06T03:18:17.750355+0000","last_active":"2023-11-06T03:18:19.114663+0000","last_peered":"2023-11-06T03:18:19.114663+0000","last_clean":"2023-11-06T03:18:19.114663+0000","last_became_active":"2023-11-06T03:18:17.750041+0000","last_became_peered":"2023-11-06T03:18:17.750041+0000","last_unstale":"2023-11-06T03:18:19.114663+0000","last_undegraded":"2023-11-06T03:18:19.114663+0000","last_fullsized":"2023-11-06T03:18:19.114663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:22:10.791306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.113021+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:18:18.113021+0000","last_peered":"2023-11-06T03:18:18.113021+0000","last_clean":"2023-11-06T03:18:18.113021+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:18:18.113021+0000","last_undegraded":"2023-11-06T03:18:18.113021+0000","last_fullsized":"2023-11-06T03:18:18.113021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139327+0000","last_change":"2023-11-06T03:18:18.112781+0000","last_active":"2023-11-06T03:18:19.139327+0000","last_peered":"2023-11-06T03:18:19.139327+0000","last_clean":"2023-11-06T03:18:19.139327+0000","last_became_active":"2023-11-06T03:18:18.112123+0000","last_became_peered":"2023-11-06T03:18:18.112123+0000","last_unstale":"2023-11-06T03:18:19.139327+0000","last_undegraded":"2023-11-06T03:18:19.139327+0000","last_fullsized":"2023-11-06T03:18:19.139327+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:59:47.503452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.137835+0000","last_change":"2023-11-06T03:18:17.758749+0000","last_active":"2023-11-06T03:18:19.137835+0000","last_peered":"2023-11-06T03:18:19.137835+0000","last_clean":"2023-11-06T03:18:19.137835+0000","last_became_active":"2023-11-06T03:18:17.757940+0000","last_became_peered":"2023-11-06T03:18:17.757940+0000","last_unstale":"2023-11-06T03:18:19.137835+0000","last_undegraded":"2023-11-06T03:18:19.137835+0000","last_fullsized":"2023-11-06T03:18:19.137835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:45:55.680814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.153361+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:18:18.153361+0000","last_peered":"2023-11-06T03:18:18.153361+0000","last_clean":"2023-11-06T03:18:18.153361+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:18:18.153361+0000","last_undegraded":"2023-11-06T03:18:18.153361+0000","last_fullsized":"2023-11-06T03:18:18.153361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139948+0000","last_change":"2023-11-06T03:18:18.167175+0000","last_active":"2023-11-06T03:18:19.139948+0000","last_peered":"2023-11-06T03:18:19.139948+0000","last_clean":"2023-11-06T03:18:19.139948+0000","last_became_active":"2023-11-06T03:18:18.164187+0000","last_became_peered":"2023-11-06T03:18:18.164187+0000","last_unstale":"2023-11-06T03:18:19.139948+0000","last_undegraded":"2023-11-06T03:18:19.139948+0000","last_fullsized":"2023-11-06T03:18:19.139948+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:29:34.190397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133679+0000","last_change":"2023-11-06T03:18:17.735600+0000","last_active":"2023-11-06T03:18:19.133679+0000","last_peered":"2023-11-06T03:18:19.133679+0000","last_clean":"2023-11-06T03:18:19.133679+0000","last_became_active":"2023-11-06T03:18:17.735310+0000","last_became_peered":"2023-11-06T03:18:17.735310+0000","last_unstale":"2023-11-06T03:18:19.133679+0000","last_undegraded":"2023-11-06T03:18:19.133679+0000","last_fullsized":"2023-11-06T03:18:19.133679+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:08:25.128836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.112153+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:18:18.112153+0000","last_peered":"2023-11-06T03:18:18.112153+0000","last_clean":"2023-11-06T03:18:18.112153+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:18:18.112153+0000","last_undegraded":"2023-11-06T03:18:18.112153+0000","last_fullsized":"2023-11-06T03:18:18.112153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.135683+0000","last_change":"2023-11-06T03:18:18.127614+0000","last_active":"2023-11-06T03:18:19.135683+0000","last_peered":"2023-11-06T03:18:19.135683+0000","last_clean":"2023-11-06T03:18:19.135683+0000","last_became_active":"2023-11-06T03:18:18.126446+0000","last_became_peered":"2023-11-06T03:18:18.126446+0000","last_unstale":"2023-11-06T03:18:19.135683+0000","last_undegraded":"2023-11-06T03:18:19.135683+0000","last_fullsized":"2023-11-06T03:18:19.135683+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:59:24.326499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"188'2","reported_seq":13,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.134692+0000","last_change":"2023-11-06T03:18:17.742458+0000","last_active":"2023-11-06T03:18:19.134692+0000","last_peered":"2023-11-06T03:18:19.134692+0000","last_clean":"2023-11-06T03:18:19.134692+0000","last_became_active":"2023-11-06T03:18:17.742102+0000","last_became_peered":"2023-11-06T03:18:17.742102+0000","last_unstale":"2023-11-06T03:18:19.134692+0000","last_undegraded":"2023-11-06T03:18:19.134692+0000","last_fullsized":"2023-11-06T03:18:19.134692+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T12:12:56.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.126832+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:18:18.126832+0000","last_peered":"2023-11-06T03:18:18.126832+0000","last_clean":"2023-11-06T03:18:18.126832+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:18:18.126832+0000","last_undegraded":"2023-11-06T03:18:18.126832+0000","last_fullsized":"2023-11-06T03:18:18.126832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.133526+0000","last_change":"2023-11-06T03:18:18.142201+0000","last_active":"2023-11-06T03:18:19.133526+0000","last_peered":"2023-11-06T03:18:19.133526+0000","last_clean":"2023-11-06T03:18:19.133526+0000","last_became_active":"2023-11-06T03:18:18.141894+0000","last_became_peered":"2023-11-06T03:18:18.141894+0000","last_unstale":"2023-11-06T03:18:19.133526+0000","last_undegraded":"2023-11-06T03:18:19.133526+0000","last_fullsized":"2023-11-06T03:18:19.133526+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:44:31.626851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"188'4","reported_seq":15,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.139467+0000","last_change":"2023-11-06T03:18:17.762622+0000","last_active":"2023-11-06T03:18:19.139467+0000","last_peered":"2023-11-06T03:18:19.139467+0000","last_clean":"2023-11-06T03:18:19.139467+0000","last_became_active":"2023-11-06T03:18:17.762132+0000","last_became_peered":"2023-11-06T03:18:17.762132+0000","last_unstale":"2023-11-06T03:18:19.139467+0000","last_undegraded":"2023-11-06T03:18:19.139467+0000","last_fullsized":"2023-11-06T03:18:19.139467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:01:40.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":158,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.112003+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:18:18.112003+0000","last_peered":"2023-11-06T03:18:18.112003+0000","last_clean":"2023-11-06T03:18:18.112003+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:18:18.112003+0000","last_undegraded":"2023-11-06T03:18:18.112003+0000","last_fullsized":"2023-11-06T03:18:18.112003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":10,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.119035+0000","last_change":"2023-11-06T03:18:18.154046+0000","last_active":"2023-11-06T03:18:19.119035+0000","last_peered":"2023-11-06T03:18:19.119035+0000","last_clean":"2023-11-06T03:18:19.119035+0000","last_became_active":"2023-11-06T03:18:18.153294+0000","last_became_peered":"2023-11-06T03:18:18.153294+0000","last_unstale":"2023-11-06T03:18:19.119035+0000","last_undegraded":"2023-11-06T03:18:19.119035+0000","last_fullsized":"2023-11-06T03:18:19.119035+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":187,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:17.706865+0000","last_clean_scrub_stamp":"2023-11-06T03:18:17.706865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T15:11:56.799818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"188'1","reported_seq":12,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-11-06T03:18:19.120110+0000","last_change":"2023-11-06T03:18:17.751893+0000","last_active":"2023-11-06T03:18:19.120110+0000","last_peered":"2023-11-06T03:18:19.120110+0000","last_clean":"2023-11-06T03:18:19.120110+0000","last_became_active":"2023-11-06T03:18:17.751134+0000","last_became_peered":"2023-11-06T03:18:17.751134+0000","last_unstale":"2023-11-06T03:18:19.120110+0000","last_undegraded":"2023-11-06T03:18:19.120110+0000","last_fullsized":"2023-11-06T03:18:19.120110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:18:16.702958+0000","last_clean_scrub_stamp":"2023-11-06T03:18:16.702958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:44:38.931877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-11-06T03:18:18.134021+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:18:18.134021+0000","last_peered":"2023-11-06T03:18:18.134021+0000","last_clean":"2023-11-06T03:18:18.134021+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:18:18.134021+0000","last_undegraded":"2023-11-06T03:18:18.134021+0000","last_fullsized":"2023-11-06T03:18:18.134021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96240,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94275600,"statfs":{"total":96636764160,"available":96538214400,"internally_reserved":0,"allocated":2990080,"data_stored":1700274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82499999999999996,"15min":0.81100000000000005},"min":{"1min":0.54900000000000004,"5min":0.39400000000000002,"15min":0.34999999999999998},"max":{"1min":1.258,"5min":1.2649999999999999,"15min":1.4830000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77500000000000002,"15min":0.81999999999999995},"min":{"1min":0.52800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1000000000000001,"5min":1.1679999999999999,"15min":1.397},"last":0.59199999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.57599999999999996,"5min":0.46400000000000002,"15min":0.45400000000000001},"max":{"1min":1.0249999999999999,"5min":1.627,"15min":1.627},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.44500000000000001},"max":{"1min":1.171,"5min":1.1910000000000001,"15min":1.6899999999999999},"last":1.0209999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.871,"15min":0.88900000000000001},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.46500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.6080000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.88200000000000001,"15min":0.89600000000000002},"min":{"1min":0.64400000000000002,"5min":0.55800000000000005,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.2370000000000001,"15min":1.6259999999999999},"last":0.67800000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.438},"max":{"1min":1.1220000000000001,"5min":1.4099999999999999,"15min":2.512},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95499999999999996,"15min":1.03},"min":{"1min":0.71599999999999997,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.3500000000000001,"5min":1.431,"15min":10.596},"last":0.76600000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94399999999999995,"15min":0.95199999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3440000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.97199999999999998},"min":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1299999999999999,"5min":1.478,"15min":1.9399999999999999},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.0389999999999999,"15min":1.042},"min":{"1min":0.77700000000000002,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.3049999999999999,"5min":1.3720000000000001,"15min":1.9139999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0680000000000001},"min":{"1min":0.71099999999999997,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.2709999999999999,"5min":1.349,"15min":1.8839999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.1499999999999999,"15min":1.149},"min":{"1min":0.75,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.3959999999999999,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.032,"5min":1.028,"15min":1.081},"min":{"1min":0.79500000000000004,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.212,"5min":1.55,"15min":2.5430000000000001},"last":0.93100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69404,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302436,"statfs":{"total":96636764160,"available":96565694464,"internally_reserved":0,"allocated":2379776,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.85999999999999999},"min":{"1min":0.47499999999999998,"5min":0.442,"15min":0.40999999999999998},"max":{"1min":1.2090000000000001,"5min":2.2429999999999999,"15min":16.073},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.84099999999999997,"15min":0.97999999999999998},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":1.2030000000000001,"5min":2.1850000000000001,"15min":40.905000000000001},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85699999999999998,"15min":0.91400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.165,"5min":1.351,"15min":16.012},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85699999999999998,"15min":0.877},"min":{"1min":0.57399999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2989999999999999,"5min":1.3400000000000001,"15min":3.3050000000000002},"last":0.79000000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.85299999999999998,"15min":0.92700000000000005},"min":{"1min":0.58499999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3460000000000001,"5min":1.702,"15min":27.803999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.94199999999999995},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.429,"15min":2.843},"last":0.754}]},{"osd":3,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92100000000000004,"15min":1.171},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.389,"5min":1.389,"15min":63.737000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.90400000000000003,"15min":1.0589999999999999},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.117,"5min":1.7829999999999999,"15min":41.079999999999998},"last":0.85499999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98799999999999999,"15min":1.1379999999999999},"min":{"1min":0.58199999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1639999999999999,"5min":1.512,"15min":24.867999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0920000000000001,"15min":1.2170000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.276,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":1.139},"min":{"1min":0.64200000000000002,"5min":0.621,"15min":0.51800000000000002},"max":{"1min":1.325,"5min":1.8440000000000001,"15min":40.959000000000003},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.113},"min":{"1min":0.86099999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.3080000000000001,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.25}]},{"osd":7,"last update":"Mon Nov 6 03:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.117,"15min":1.2090000000000001},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4450000000000001,"5min":4.2389999999999999,"15min":23.748999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.1299999999999999,"15min":1.218},"min":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.56499999999999995},"max":{"1min":1.3819999999999999,"5min":5.2709999999999999,"15min":28.164000000000001},"last":1.181}]}]},{"osd":5,"up_from":13,"seq":55834575025,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69412,"kb_used_data":2332,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302428,"statfs":{"total":96636764160,"available":96565686272,"internally_reserved":0,"allocated":2387968,"data_stored":1108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81399999999999995,"15min":1.1299999999999999},"min":{"1min":0.60399999999999998,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":4.0510000000000002,"15min":125.256},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.139,"15min":1.266},"min":{"1min":0.69099999999999995,"5min":0.495,"15min":0.35499999999999998},"max":{"1min":1.119,"5min":26.518000000000001,"15min":126.98699999999999},"last":0.751}]},{"osd":1,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92700000000000005,"15min":0.84699999999999998},"min":{"1min":0.66600000000000004,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.163,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":1.004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.3240000000000001,"15min":1.002},"min":{"1min":0.58899999999999997,"5min":0.53600000000000003,"15min":0.376},"max":{"1min":1.3939999999999999,"5min":43.799999999999997,"15min":43.799999999999997},"last":0.91400000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.371,"15min":1.0249999999999999},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.33400000000000002},"max":{"1min":1.2549999999999999,"5min":43.741,"15min":43.741},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.872,"15min":0.85299999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.39200000000000002},"max":{"1min":1.071,"5min":1.385,"15min":2.9700000000000002},"last":0.93600000000000005}]},{"osd":3,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.95999999999999996,"15min":0.90700000000000003},"min":{"1min":0.64600000000000002,"5min":0.52400000000000002,"15min":0.40400000000000003},"max":{"1min":1.0800000000000001,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.454,"15min":1.119},"min":{"1min":0.84499999999999997,"5min":0.56699999999999995,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":43.841999999999999,"15min":43.841999999999999},"last":1.0309999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.93200000000000005,"15min":0.91500000000000004},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.29299999999999998},"max":{"1min":1.333,"5min":1.595,"15min":3.1349999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.1240000000000001,"15min":1.0309999999999999},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.393,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.86299999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0640000000000001,"15min":1.0289999999999999},"min":{"1min":0.63100000000000001,"5min":0.501,"15min":0.378},"max":{"1min":1.365,"5min":2,"15min":2.6909999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0940000000000001,"15min":1.048},"min":{"1min":0.69899999999999995,"5min":0.58099999999999996,"15min":0.35699999999999998},"max":{"1min":1.413,"5min":4.1230000000000002,"15min":5.6109999999999998},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.0860000000000001},"min":{"1min":0.79600000000000004,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.3129999999999999,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.498,"5min":3.738,"15min":3.738},"last":0.98399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113684,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258156,"statfs":{"total":96636764160,"available":96520351744,"internally_reserved":0,"allocated":2371584,"data_stored":1102201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78500000000000003,"15min":0.95299999999999996},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":1.1859999999999999,"5min":1.4319999999999999,"15min":42.853999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.79100000000000004,"15min":0.96299999999999997},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.308},"max":{"1min":1.196,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.50800000000000001,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":1.0149999999999999,"5min":1.5960000000000001,"15min":2.3679999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.90300000000000002,"15min":0.85999999999999999},"min":{"1min":0.60399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1539999999999999,"5min":1.639,"15min":1.6719999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.92800000000000005},"min":{"1min":0.65300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.159,"5min":1.3200000000000001,"15min":3.2879999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.2290000000000001,"5min":1.27,"15min":3.2639999999999998},"last":0.73899999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.75600000000000001,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":3.198},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90000000000000002,"15min":0.89900000000000002},"min":{"1min":0.64100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.056,"5min":1.4510000000000001,"15min":2.4500000000000002},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.024},"min":{"1min":0.61399999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2250000000000001,"5min":1.5429999999999999,"15min":10.192},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.73899999999999999,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":1.3839999999999999,"5min":1.494,"15min":5.3310000000000004},"last":1.004}]},{"osd":6,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.97099999999999997,"15min":0.97999999999999998},"min":{"1min":0.623,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":1.169,"5min":1.6140000000000001,"15min":2.5099999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.016,"5min":1.0720000000000001,"15min":1.0880000000000001},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.6180000000000001,"15min":2.6549999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0840000000000001,"15min":1.081},"min":{"1min":0.93300000000000005,"5min":0.66000000000000003,"15min":0.56599999999999995},"max":{"1min":1.3100000000000001,"5min":1.6000000000000001,"15min":3.3490000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.1020000000000001,"15min":1.1559999999999999},"min":{"1min":0.67600000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2430000000000001,"5min":1.6819999999999999,"15min":13.582000000000001},"last":0.84399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87512,"kb_used_data":2320,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94284328,"statfs":{"total":96636764160,"available":96547151872,"internally_reserved":0,"allocated":2375680,"data_stored":1100726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.217,"15min":0.17399999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.4199999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.33800000000000002,"5min":0.22900000000000001,"15min":0.19600000000000001},"max":{"1min":0.88800000000000001,"5min":0.90100000000000002,"15min":1.524},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59199999999999997,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.251,"15min":0.224},"max":{"1min":0.996,"5min":1.0780000000000001,"15min":1.371},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.29499999999999998,"15min":0.20999999999999999},"max":{"1min":1.0129999999999999,"5min":1.3029999999999999,"15min":1.4099999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.68799999999999994},"min":{"1min":0.38100000000000001,"5min":0.30499999999999999,"15min":0.23300000000000001},"max":{"1min":0.92000000000000004,"5min":1.421,"15min":1.532},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.68799999999999994,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.32900000000000001,"15min":0.255},"max":{"1min":1.123,"5min":1.206,"15min":1.284},"last":0.82599999999999996}]},{"osd":4,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.86299999999999999},"min":{"1min":0.61199999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1579999999999999,"5min":1.2529999999999999,"15min":1.7989999999999999},"last":1.042},{"interface":"front","average":{"1min":0.876,"5min":0.85399999999999998,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.49299999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.369,"15min":1.8300000000000001},"last":0.84999999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.91600000000000004,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2150000000000001,"5min":1.4890000000000001,"15min":1.6639999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.94599999999999995,"15min":0.97699999999999998},"min":{"1min":0.81799999999999995,"5min":0.504,"15min":0.35999999999999999},"max":{"1min":1.482,"5min":1.573,"15min":1.698},"last":1.1879999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.72299999999999998,"5min":0.44900000000000001,"15min":0.39300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.8620000000000001},"last":1.27},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89600000000000002,"15min":0.94699999999999995},"min":{"1min":0.63500000000000001,"5min":0.49399999999999999,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.3140000000000001,"15min":1.8740000000000001},"last":1.0900000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:17:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.96399999999999997,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.51500000000000001,"15min":0.51100000000000001},"max":{"1min":1.208,"5min":1.4410000000000001,"15min":1.835},"last":1.232},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.97899999999999998,"15min":1.0249999999999999},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.48299999999999998},"max":{"1min":1.2769999999999999,"5min":1.569,"15min":1.8979999999999999},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73504,"kb_used_data":2328,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298336,"statfs":{"total":96636764160,"available":96561496064,"internally_reserved":0,"allocated":2383872,"data_stored":1108362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.373,"5min":0.23200000000000001,"15min":0.17399999999999999},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":1.7270000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.56299999999999994,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26500000000000001,"15min":0.191},"max":{"1min":1.0609999999999999,"5min":1.0920000000000001,"15min":1.4590000000000001},"last":0.317}]},{"osd":1,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56699999999999995,"15min":0.60699999999999998},"min":{"1min":0.31900000000000001,"5min":0.29599999999999999,"15min":0.23000000000000001},"max":{"1min":0.82599999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.35399999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1579999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.45100000000000001,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.294},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.628,"15min":0.68100000000000005},"min":{"1min":0.39900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97299999999999998,"5min":1.1719999999999999,"15min":1.4199999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.73299999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":1.05,"5min":1.2010000000000001,"15min":1.526},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.879,"15min":0.89000000000000001},"min":{"1min":0.56899999999999995,"5min":0.52300000000000002,"15min":0.41399999999999998},"max":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.5229999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83199999999999996,"15min":0.84899999999999998},"min":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.44500000000000001},"max":{"1min":1.173,"5min":1.2729999999999999,"15min":1.6200000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86899999999999999,"15min":0.91300000000000003},"min":{"1min":0.75900000000000001,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.139,"5min":1.355,"15min":1.5780000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.68300000000000005,"5min":0.50600000000000001,"15min":0.46600000000000003},"max":{"1min":1.272,"5min":1.542,"15min":1.615},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.5},"max":{"1min":1.242,"5min":1.2689999999999999,"15min":1.3300000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:17:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.92400000000000004,"15min":0.95999999999999996},"min":{"1min":0.66400000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.5189999999999999,"15min":4.2809999999999997},"last":0.747},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.89400000000000002,"15min":0.91900000000000004},"min":{"1min":0.55700000000000005,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.1870000000000001,"5min":1.8680000000000001,"15min":3.4740000000000002},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73488,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298352,"statfs":{"total":96636764160,"available":96561512448,"internally_reserved":0,"allocated":2367488,"data_stored":1101619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":1.0229999999999999,"5min":1.0569999999999999,"15min":1.843},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.38500000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.85799999999999998,"5min":1.244,"15min":2.1150000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.69399999999999995},"min":{"1min":0.34999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.79200000000000004,"5min":1.143,"15min":30.861000000000001},"last":0.499},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61799999999999999,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.20899999999999999},"max":{"1min":0.90200000000000002,"5min":1.117,"15min":1.7330000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.28699999999999998,"15min":0.24199999999999999},"max":{"1min":1.0740000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.26400000000000001,"15min":0.20799999999999999},"max":{"1min":1.1419999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.77100000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84699999999999998,"15min":0.85599999999999998},"min":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1699999999999999,"5min":1.3029999999999999,"15min":2.258},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.83899999999999997,"15min":0.92000000000000004},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.41699999999999998},"max":{"1min":1.306,"5min":1.329,"15min":14.753},"last":1.006}]},{"osd":5,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88600000000000001,"15min":1.073},"min":{"1min":0.67800000000000005,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.2430000000000001,"5min":1.351,"15min":43.103999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":1.1120000000000001},"min":{"1min":0.67100000000000004,"5min":0.56100000000000005,"15min":0.39100000000000001},"max":{"1min":1.357,"5min":1.468,"15min":44.753999999999998},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91700000000000004,"15min":1.0169999999999999},"min":{"1min":0.59299999999999997,"5min":0.48399999999999999,"15min":0.45100000000000001},"max":{"1min":1.2789999999999999,"5min":1.3360000000000001,"15min":23.971},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.90500000000000003,"15min":1.028},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3600000000000001,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.98899999999999999,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.3100000000000001,"5min":1.581,"15min":2.2799999999999998},"last":1.508},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.97999999999999998,"15min":1.0069999999999999},"min":{"1min":0.70399999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.347,"5min":1.835,"15min":2.3559999999999999},"last":1.2729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96276,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94275564,"statfs":{"total":96636764160,"available":96538177536,"internally_reserved":0,"allocated":2961408,"data_stored":1691058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.313,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":1.214,"5min":1.214,"15min":2.6989999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.64900000000000002,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.45,"5min":1.45,"15min":7.1230000000000002},"last":0.45100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.66100000000000003,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69099999999999995,"15min":0.66400000000000003},"min":{"1min":0.374,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.51300000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64400000000000002,"15min":0.63500000000000001},"min":{"1min":0.35499999999999998,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.41599999999999998,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.88900000000000001,"15min":0.91100000000000003},"min":{"1min":0.69399999999999995,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":1.3149999999999999,"5min":1.419,"15min":4.641},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.86399999999999999,"15min":0.872},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.42899999999999999},"max":{"1min":1.2470000000000001,"5min":1.4159999999999999,"15min":2.028},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.97399999999999998,"15min":0.95599999999999996},"min":{"1min":0.72799999999999998,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.8559999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":0.96699999999999997,"15min":0.95299999999999996},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.432},"max":{"1min":1.623,"5min":1.623,"15min":4.7430000000000003},"last":0.93200000000000005}]},{"osd":6,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.93999999999999995,"15min":0.91600000000000004},"min":{"1min":0.61699999999999999,"5min":0.503,"15min":0.44800000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":2.931},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.996,"15min":0.997},"min":{"1min":0.68000000000000005,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.002,"15min":0.997},"min":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.495},"max":{"1min":1.544,"5min":1.544,"15min":4.8339999999999996},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.042,"5min":0.98599999999999999,"15min":0.97699999999999998},"min":{"1min":0.73799999999999999,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.97},"last":0.87}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:18:21.478 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-06T03:18:21.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-06T03:18:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 Processor -- start 2023-11-06T03:18:21.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 -- start start 2023-11-06T03:18:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 0x7fc48006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48015f580 con 0x7fc48015e5e0 2023-11-06T03:18:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48015efc0 con 0x7fc4800b3b20 2023-11-06T03:18:21.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc485ace640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015f2a0 con 0x7fc4801551e0 2023-11-06T03:18:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc47effd640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc48015e5e0 0x7fc4800b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38556/0 (socket says 172.21.15.157:38556) 2023-11-06T03:18:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc47effd640 1 -- 172.21.15.157:0/3811065305 learned_addr learned my addr 172.21.15.157:0/3811065305 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc47f7fe640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 0x7fc48006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440931129 0 0) 0x7fc48015f580 con 0x7fc48015e5e0 2023-11-06T03:18:21.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.646+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003530 con 0x7fc48015e5e0 2023-11-06T03:18:21.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508480693 0 0) 0x7fc468003530 con 0x7fc48015e5e0 2023-11-06T03:18:21.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 msgr2=0x7fc48006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:21.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 0x7fc48006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 >> v1:172.21.15.195:6789/0 conn(0x7fc4800b3b20 legacy=0x7fc48015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48006f5a0 con 0x7fc48015e5e0 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fc474001680 con 0x7fc48015e5e0 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 >> v1:172.21.15.157:6789/0 conn(0x7fc48015e5e0 legacy=0x7fc4800b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4680039b0 0x7fc468003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4680047d0 0x7fc468004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 0x7fc4680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015f2a0 con 0x7fc4680047d0 2023-11-06T03:18:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46800bac0 con 0x7fc4680039b0 2023-11-06T03:18:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46800bc80 con 0x7fc4801551e0 2023-11-06T03:18:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4680047d0 0x7fc468004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47f7fe640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4680039b0 0x7fc468003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 -- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 msgr2=0x7fc4680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4801551e0 0x7fc4680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 -- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4680039b0 msgr2=0x7fc468003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4680039b0 0x7fc468003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 -- 172.21.15.157:0/3811065305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468003530 con 0x7fc4680047d0 2023-11-06T03:18:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47effd640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4680047d0 0x7fc468004c50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c240 tx=0x7fc47401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6461503d503217d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740a5b10 con 0x7fc4680047d0 2023-11-06T03:18:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4740a5c70 con 0x7fc4680047d0 2023-11-06T03:18:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47e7fc640 1 -- 172.21.15.157:0/3811065305 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474005b20 con 0x7fc4680047d0 2023-11-06T03:18:21.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 -- 172.21.15.157:0/3811065305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4680047d0 msgr2=0x7fc468004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 --2- 172.21.15.157:0/3811065305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4680047d0 0x7fc468004c50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c240 tx=0x7fc47401cc10 comp rx=0 tx=0).stop 2023-11-06T03:18:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc47ffff640 1 -- 172.21.15.157:0/3811065305 reap_dead start 2023-11-06T03:18:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 -- 172.21.15.157:0/3811065305 shutdown_connections 2023-11-06T03:18:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 -- 172.21.15.157:0/3811065305 >> 172.21.15.157:0/3811065305 conn(0x7fc480065930 msgr2=0x7fc480065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:21.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 -- 172.21.15.157:0/3811065305 shutdown_connections 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.650+0000 7fc485ace640 1 -- 172.21.15.157:0/3811065305 wait complete. 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 Processor -- start 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- start start 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4680047d0 0x7fc48014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 0x7fc48014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800ce470 con 0x7fc4800cdab0 2023-11-06T03:18:21.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48014b120 con 0x7fc48014ef60 2023-11-06T03:18:21.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48014b400 con 0x7fc4680047d0 2023-11-06T03:18:21.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36384/0 (socket says 172.21.15.157:36384) 2023-11-06T03:18:21.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 -- 172.21.15.157:0/910579480 learned_addr learned my addr 172.21.15.157:0/910579480 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:21.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47f7fe640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4680047d0 0x7fc48014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47effd640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 0x7fc48014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 -- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4680047d0 msgr2=0x7fc48014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4680047d0 0x7fc48014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 -- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 msgr2=0x7fc48014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 0x7fc48014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 -- 172.21.15.157:0/910579480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48014b6e0 con 0x7fc4800cdab0 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47effd640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 0x7fc48014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc47ffff640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fc468004750 tx=0x7fc47800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478005d50 con 0x7fc4800cdab0 2023-11-06T03:18:21.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47800ced0 con 0x7fc4800cdab0 2023-11-06T03:18:21.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48014b9c0 con 0x7fc4800cdab0 2023-11-06T03:18:21.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47800a850 con 0x7fc4800cdab0 2023-11-06T03:18:21.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48014adb0 con 0x7fc4800cdab0 2023-11-06T03:18:21.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48014fbc0 con 0x7fc4800cdab0 2023-11-06T03:18:21.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.654+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc478003980 con 0x7fc4800cdab0 2023-11-06T03:18:21.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.658+0000 7fc467fff640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 0x7fc46c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:21.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.658+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc478082480 con 0x7fc4800cdab0 2023-11-06T03:18:21.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.658+0000 7fc47f7fe640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 0x7fc46c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:21.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.658+0000 7fc47f7fe640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 0x7fc46c067f50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc470009a50 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:21.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.662+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc48014fbc0 con 0x7fc4800cdab0 2023-11-06T03:18:21.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.870+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc48014fbc0 con 0x7fc4800cdab0 2023-11-06T03:18:21.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.878+0000 7fc467fff640 1 -- 172.21.15.157:0/910579480 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc48014fbc0 con 0x7fc4800cdab0 2023-11-06T03:18:21.877 INFO:teuthology.orchestra.run.smithi157.stderr:updated caps for client.0 2023-11-06T03:18:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 msgr2=0x7fc46c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 0x7fc46c067f50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc470009a50 tx=0x7fc470008040 comp rx=0 tx=0).stop 2023-11-06T03:18:21.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 msgr2=0x7fc4800cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fc468004750 tx=0x7fc47800cb90 comp rx=0 tx=0).stop 2023-11-06T03:18:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 shutdown_connections 2023-11-06T03:18:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc46c065a90 0x7fc46c067f50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4800cdab0 0x7fc4800cdf30 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc48014ef60 0x7fc48014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 --2- 172.21.15.157:0/910579480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4680047d0 0x7fc48014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 >> 172.21.15.157:0/910579480 conn(0x7fc480065930 msgr2=0x7fc480065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 shutdown_connections 2023-11-06T03:18:21.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:21.882+0000 7fc485ace640 1 -- 172.21.15.157:0/910579480 wait complete. 2023-11-06T03:18:21.895 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:22.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 Processor -- start 2023-11-06T03:18:22.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 -- start start 2023-11-06T03:18:22.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fecd40b4ac0 0x7fecd40cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd415b3d0 con 0x7fecd40b1ad0 2023-11-06T03:18:22.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd415b0c0 con 0x7fecd40b1eb0 2023-11-06T03:18:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecdc1d1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd4155610 con 0x7fecd40b4ac0 2023-11-06T03:18:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd9f6d640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fecd40b1ad0 0x7fecd40c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38566/0 (socket says 172.21.15.157:38566) 2023-11-06T03:18:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd9f6d640 1 -- 172.21.15.157:0/2983251262 learned_addr learned my addr 172.21.15.157:0/2983251262 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232302679 0 0) 0x7fecd415b3d0 con 0x7fecd40b1ad0 2023-11-06T03:18:22.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc0003530 con 0x7fecd40b1ad0 2023-11-06T03:18:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecda76e640 1 --2- 172.21.15.157:0/2983251262 >> v2:172.21.15.195:3300/0 conn(0x7fecd40b4ac0 0x7fecd40cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751697117 0 0) 0x7fecc0003530 con 0x7fecd40b1ad0 2023-11-06T03:18:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 >> v2:172.21.15.195:3300/0 conn(0x7fecd40b4ac0 msgr2=0x7fecd40cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 --2- 172.21.15.157:0/2983251262 >> v2:172.21.15.195:3300/0 conn(0x7fecd40b4ac0 0x7fecd40cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 >> v1:172.21.15.195:6789/0 conn(0x7fecd40b1eb0 legacy=0x7fecd415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40b50c0 con 0x7fecd40b1ad0 2023-11-06T03:18:22.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fecd0001680 con 0x7fecd40b1ad0 2023-11-06T03:18:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 >> v1:172.21.15.157:6789/0 conn(0x7fecd40b1ad0 legacy=0x7fecd40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecc0003980 0x7fecc0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 0x7fecc0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc0005190 0x7fecc000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4155610 con 0x7fecc00047d0 2023-11-06T03:18:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc000c0e0 con 0x7fecc0003980 2023-11-06T03:18:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc000c2a0 con 0x7fecc0005190 2023-11-06T03:18:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 0x7fecc0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd9f6d640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecc0003980 0x7fecc0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecda76e640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc0005190 0x7fecc000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 -- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc0005190 msgr2=0x7fecc000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc0005190 0x7fecc000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 -- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecc0003980 msgr2=0x7fecc0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecc0003980 0x7fecc0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecd976c640 1 -- 172.21.15.157:0/2983251262 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0003530 con 0x7fecc00047d0 2023-11-06T03:18:22.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.006+0000 7fecda76e640 1 -- 172.21.15.157:0/2983251262 reap_dead start 2023-11-06T03:18:22.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 0x7fecc0004c50 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001750 tx=0x7fecc401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676cddf99b1088f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40052d0 con 0x7fecc00047d0 2023-11-06T03:18:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40a7050 con 0x7fecc00047d0 2023-11-06T03:18:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd8f6b640 1 -- 172.21.15.157:0/2983251262 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4003600 con 0x7fecc00047d0 2023-11-06T03:18:22.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 msgr2=0x7fecc0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 0x7fecc0004c50 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001750 tx=0x7fecc401c3a0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/2983251262 shutdown_connections 2023-11-06T03:18:22.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/2983251262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecc00047d0 0x7fecc0004c50 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/2983251262 >> 172.21.15.157:0/2983251262 conn(0x7fecd406a170 msgr2=0x7fecd406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/2983251262 shutdown_connections 2023-11-06T03:18:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/2983251262 wait complete. 2023-11-06T03:18:22.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 Processor -- start 2023-11-06T03:18:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- start start 2023-11-06T03:18:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc00047d0 0x7fecd41487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecd4144cb0 0x7fecd41431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4148630 con 0x7fecd40c2cf0 2023-11-06T03:18:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4145130 con 0x7fecd4144cb0 2023-11-06T03:18:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4142d60 con 0x7fecc00047d0 2023-11-06T03:18:22.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36398/0 (socket says 172.21.15.157:36398) 2023-11-06T03:18:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 -- 172.21.15.157:0/1600806305 learned_addr learned my addr 172.21.15.157:0/1600806305 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 -- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc00047d0 msgr2=0x7fecd41487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc00047d0 0x7fecd41487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 -- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecd4144cb0 msgr2=0x7fecd41431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecd4144cb0 0x7fecd41431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 -- 172.21.15.157:0/1600806305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40c3170 con 0x7fecd40c2cf0 2023-11-06T03:18:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecd976c640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fecc401c1d0 tx=0x7fecc0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4003e40 con 0x7fecd40c2cf0 2023-11-06T03:18:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4007a40 con 0x7fecd40c2cf0 2023-11-06T03:18:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd415ce50 con 0x7fecd40c2cf0 2023-11-06T03:18:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007270 con 0x7fecd40c2cf0 2023-11-06T03:18:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.010+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd4142710 con 0x7fecd40c2cf0 2023-11-06T03:18:22.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd4072750 con 0x7fecd40c2cf0 2023-11-06T03:18:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fecc40073d0 con 0x7fecd40c2cf0 2023-11-06T03:18:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecca7fc640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 0x7fecb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fecc411e1c0 con 0x7fecd40c2cf0 2023-11-06T03:18:22.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecd9f6d640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 0x7fecb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.014+0000 7fecd9f6d640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 0x7fecb8067f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fecd0004880 tx=0x7fecd00047f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.018+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fecc40ef6e0 con 0x7fecd40c2cf0 2023-11-06T03:18:22.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.190+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecd4070f70 con 0x7fecd40c2cf0 2023-11-06T03:18:22.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecca7fc640 1 -- 172.21.15.157:0/1600806305 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4669 (secure 0 0 0) 0x7fecc40ef270 con 0x7fecd40c2cf0 2023-11-06T03:18:22.190 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:22.191 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:20.111160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11244":{"gid":11244,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/877366447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":877366447},{"type":"v1","addr":"172.21.15.157:6835","nonce":877366447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:22.191 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 130 2023-11-06T03:18:22.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 msgr2=0x7fecb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 0x7fecb8067f50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fecd0004880 tx=0x7fecd00047f0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 msgr2=0x7fecd4144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fecc401c1d0 tx=0x7fecc0005800 comp rx=0 tx=0).stop 2023-11-06T03:18:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 shutdown_connections 2023-11-06T03:18:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fecb8065a90 0x7fecb8067f50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fecd4144cb0 0x7fecd41431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecd40c2cf0 0x7fecd4144770 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 --2- 172.21.15.157:0/1600806305 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fecc00047d0 0x7fecd41487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 >> 172.21.15.157:0/1600806305 conn(0x7fecd406a170 msgr2=0x7fecd40cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 shutdown_connections 2023-11-06T03:18:22.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.194+0000 7fecdc1d1640 1 -- 172.21.15.157:0/1600806305 wait complete. 2023-11-06T03:18:22.612 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:18:18.097057+0000', 'modified': '2023-11-06T03:18:20.111160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11244': {'gid': 11244, 'name': 'a', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/877366447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 877366447}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 877366447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:18:22.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:18:22.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:18:22.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 Processor -- start 2023-11-06T03:18:22.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 -- start start 2023-11-06T03:18:22.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5f840b2af0 0x7f5f840ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8406d000 0x7f5f84159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8415e440 con 0x7f5f841599c0 2023-11-06T03:18:22.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8406c240 con 0x7f5f840b2af0 2023-11-06T03:18:22.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f88fc1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8406c520 con 0x7f5f8406d000 2023-11-06T03:18:22.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f8259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5f841599c0 0x7f5f840b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38574/0 (socket says 172.21.15.157:38574) 2023-11-06T03:18:22.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f8259c640 1 -- 172.21.15.157:0/3974645381 learned_addr learned my addr 172.21.15.157:0/3974645381 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:22.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/3974645381 >> v2:172.21.15.157:3300/0 conn(0x7f5f840b2af0 0x7f5f840ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.745 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8406d000 0x7f5f84159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f81d9b640 1 -- 172.21.15.157:0/3974645381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8406d000 msgr2=0x7f5f84159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8406d000 0x7f5f84159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f81d9b640 1 -- 172.21.15.157:0/3974645381 >> v1:172.21.15.157:6789/0 conn(0x7f5f841599c0 legacy=0x7f5f840b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f81d9b640 1 -- 172.21.15.157:0/3974645381 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8406c800 con 0x7f5f840b2af0 2023-11-06T03:18:22.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.746+0000 7f5f8159a640 1 -- 172.21.15.157:0/3974645381 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458758871 0 0) 0x7f5f8415e440 con 0x7f5f841599c0 2023-11-06T03:18:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f840b2af0 0x7f5f840ae000 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f5f780097c0 tx=0x7f5f7801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dcd49d14a480a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8159a640 1 -- 172.21.15.157:0/3974645381 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780ab670 con 0x7f5f840b2af0 2023-11-06T03:18:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8159a640 1 -- 172.21.15.157:0/3974645381 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780ab7d0 con 0x7f5f840b2af0 2023-11-06T03:18:22.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8159a640 1 -- 172.21.15.157:0/3974645381 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780abb60 con 0x7f5f840b2af0 2023-11-06T03:18:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- 172.21.15.157:0/3974645381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f840b2af0 msgr2=0x7f5f840ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f840b2af0 0x7f5f840ae000 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f5f780097c0 tx=0x7f5f7801c450 comp rx=0 tx=0).stop 2023-11-06T03:18:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- 172.21.15.157:0/3974645381 shutdown_connections 2023-11-06T03:18:22.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f8406d000 0x7f5f84159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/3974645381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f840b2af0 0x7f5f840ae000 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- 172.21.15.157:0/3974645381 >> 172.21.15.157:0/3974645381 conn(0x7f5f84065930 msgr2=0x7f5f84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- 172.21.15.157:0/3974645381 shutdown_connections 2023-11-06T03:18:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- 172.21.15.157:0/3974645381 wait complete. 2023-11-06T03:18:22.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 Processor -- start 2023-11-06T03:18:22.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- start start 2023-11-06T03:18:22.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f840b2af0 0x7f5f84153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f841599c0 0x7f5f8414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840adfc0 con 0x7f5f8406d000 2023-11-06T03:18:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840cebd0 con 0x7f5f840b2af0 2023-11-06T03:18:22.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f88fc1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840ceeb0 con 0x7f5f841599c0 2023-11-06T03:18:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36422/0 (socket says 172.21.15.157:36422) 2023-11-06T03:18:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 -- 172.21.15.157:0/976438550 learned_addr learned my addr 172.21.15.157:0/976438550 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f841599c0 0x7f5f8414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f840b2af0 0x7f5f84153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 -- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f841599c0 msgr2=0x7f5f8414d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f841599c0 0x7f5f8414d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 -- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f840b2af0 msgr2=0x7f5f84153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f840b2af0 0x7f5f84153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.750+0000 7f5f8259c640 1 -- 172.21.15.157:0/976438550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f78009420 con 0x7f5f8406d000 2023-11-06T03:18:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8259c640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f5f740011f0 tx=0x7f5f7400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 -- 172.21.15.157:0/976438550 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f740074e0 con 0x7f5f8406d000 2023-11-06T03:18:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 -- 172.21.15.157:0/976438550 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f74007640 con 0x7f5f8406d000 2023-11-06T03:18:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f840ae210 con 0x7f5f8406d000 2023-11-06T03:18:22.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 -- 172.21.15.157:0/976438550 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74004a10 con 0x7f5f8406d000 2023-11-06T03:18:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f84154360 con 0x7f5f8406d000 2023-11-06T03:18:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 -- 172.21.15.157:0/976438550 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5f74004b70 con 0x7f5f8406d000 2023-11-06T03:18:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 0x7f5f64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f72ffd640 1 -- 172.21.15.157:0/976438550 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f740827e0 con 0x7f5f8406d000 2023-11-06T03:18:22.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 0x7f5f64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 Processor -- start 2023-11-06T03:18:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 -- start start 2023-11-06T03:18:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 0x7f5f64067f20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5f780af000 tx=0x7f5f780a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c36d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c235fc0 con 0x558c3c36da90 2023-11-06T03:18:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c1c41a0 con 0x558c3c3b8000 2023-11-06T03:18:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.754+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c3c1c1770 con 0x558c3c3b76e0 2023-11-06T03:18:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36438/0 (socket says 172.21.15.157:36438) 2023-11-06T03:18:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/550855339 learned_addr learned my addr 172.21.15.157:0/550855339 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8259c640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c36d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 msgr2=0x558c3c3b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.764 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 msgr2=0x558c3c36d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c36d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/550855339 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c3beb9890 con 0x558c3c36da90 2023-11-06T03:18:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c36d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009f90 tx=0x7f5f7c0077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e1c84eaac5ce39 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f63fff640 1 -- 172.21.15.157:0/550855339 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0147e0 con 0x558c3c36da90 2023-11-06T03:18:22.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f63fff640 1 -- 172.21.15.157:0/550855339 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00f050 con 0x558c3c36da90 2023-11-06T03:18:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f63fff640 1 -- 172.21.15.157:0/550855339 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c014b60 con 0x558c3c36da90 2023-11-06T03:18:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/550855339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 msgr2=0x558c3c36de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009f90 tx=0x7f5f7c0077c0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/550855339 shutdown_connections 2023-11-06T03:18:22.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c36da90 0x558c3c36de70 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c36d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/550855339 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/550855339 >> 172.21.15.157:0/550855339 conn(0x558c3c373bf0 msgr2=0x558c3c36be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/550855339 shutdown_connections 2023-11-06T03:18:22.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/550855339 wait complete. 2023-11-06T03:18:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- start start 2023-11-06T03:18:22.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 Processor -- start 2023-11-06T03:18:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c378880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c378dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c1bd060 con 0x558c3c3b8000 2023-11-06T03:18:22.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c15c290 con 0x558c3c36da90 2023-11-06T03:18:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c3c17a6f0 con 0x558c3c3b76e0 2023-11-06T03:18:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 10 client.0 _ll_register_callbacks cb 0x558c3c3591d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36448/0 (socket says 172.21.15.157:36448) 2023-11-06T03:18:22.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/2474639753 learned_addr learned my addr 172.21.15.157:0/2474639753 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c378dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8259c640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c378880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 msgr2=0x558c3c378dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c378dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 msgr2=0x558c3c378880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c378880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 -- 172.21.15.157:0/2474639753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c009bf0 con 0x558c3c3b8000 2023-11-06T03:18:22.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8259c640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c378880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x558c3c23be40 tx=0x7f5f7c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632648b24045c766 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f617fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c02a680 con 0x558c3c3b8000 2023-11-06T03:18:22.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f7c025070 con 0x558c3c3b8000 2023-11-06T03:18:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.758+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c3c1bb9b0 con 0x558c3c3b8000 2023-11-06T03:18:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f7c014e70 con 0x558c3c3b8000 2023-11-06T03:18:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 10 client.11376 fetch_fsmap learned FSMap version 130 2023-11-06T03:18:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x558c3c1821e0 con 0x558c3c3b8000 2023-11-06T03:18:22.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c3c1bb9b0 con 0x558c3c3b8000 2023-11-06T03:18:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 130) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5f7c02a7e0 con 0x558c3c3b8000 2023-11-06T03:18:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 10 client.11376 fetch_fsmap finished waiting for FSMap version 130 2023-11-06T03:18:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 10 client.11376 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:18:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 20 client.11376 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:22.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3c6770 0x558c3c3c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 4 client.11376 mds_command: new command op to 11244 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 --> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c3c1ec4d0 con 0x558c3c3c6770 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8259c640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3c6770 0x558c3c3c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8259c640 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3c6770 0x558c3c3c8b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496ce75a73df8193 server_cookie=cff1fcbd704b6d11 in_seq=0 out_seq=0 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 10 client.11376 ms_handle_connect on v2:172.21.15.157:6834/877366447 2023-11-06T03:18:22.782 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 1 -- 172.21.15.157:0/2474639753 <== mds.0 v2:172.21.15.157:6834/877366447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558c3c1ec4d0 con 0x558c3c3c6770 2023-11-06T03:18:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 10 client.11376 handle_command_reply: tid=0 2023-11-06T03:18:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 client.11376 shutdown 2023-11-06T03:18:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 msgr2=0x558c3c379300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x558c3c23be40 tx=0x7f5f7c004ac0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 shutdown_connections 2023-11-06T03:18:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3c6770 0x558c3c3c8b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c379300 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c378dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/2474639753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c378880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 >> 172.21.15.157:0/2474639753 conn(0x558c3c373bf0 msgr2=0x558c3c36c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f617fa640 0 client.11376 ms_handle_reset on v2:172.21.15.157:6834/877366447 2023-11-06T03:18:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 shutdown_connections 2023-11-06T03:18:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/2474639753 wait complete. 2023-11-06T03:18:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.762+0000 7f5f8c0e9000 20 client.11376 trim_cache size 0 max 16384 2023-11-06T03:18:22.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 Processor -- start 2023-11-06T03:18:22.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- start start 2023-11-06T03:18:22.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c36da90 0x558c3c3c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c3b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3be7a940 con 0x558c3c3b76e0 2023-11-06T03:18:22.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c1bbef0 con 0x558c3c3b8000 2023-11-06T03:18:22.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c3c3cd1c0 con 0x558c3c36da90 2023-11-06T03:18:22.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36460/0 (socket says 172.21.15.157:36460) 2023-11-06T03:18:22.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 -- 172.21.15.157:0/143392079 learned_addr learned my addr 172.21.15.157:0/143392079 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8259c640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c36da90 0x558c3c3c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c3b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 -- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c36da90 msgr2=0x558c3c3c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c36da90 0x558c3c3c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 -- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 msgr2=0x558c3c3b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c3b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 -- 172.21.15.157:0/143392079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c3c168330 con 0x558c3c3b76e0 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c3b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5f7801ccf0 tx=0x7f5f78004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93296b613bee512e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f70ff9640 1 -- 172.21.15.157:0/143392079 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780abdc0 con 0x558c3c3b76e0 2023-11-06T03:18:22.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f70ff9640 1 -- 172.21.15.157:0/143392079 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78003620 con 0x558c3c3b76e0 2023-11-06T03:18:22.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f70ff9640 1 -- 172.21.15.157:0/143392079 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780039d0 con 0x558c3c3b76e0 2023-11-06T03:18:22.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/143392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 msgr2=0x558c3c3ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5f7801ccf0 tx=0x7f5f78004740 comp rx=0 tx=0).stop 2023-11-06T03:18:22.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/143392079 shutdown_connections 2023-11-06T03:18:22.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c3b8000 0x558c3c3b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b76e0 0x558c3c3ca3e0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/143392079 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c36da90 0x558c3c3c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/143392079 >> 172.21.15.157:0/143392079 conn(0x558c3c373bf0 msgr2=0x558c3c3c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/143392079 shutdown_connections 2023-11-06T03:18:22.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/143392079 wait complete. 2023-11-06T03:18:22.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 -- start start 2023-11-06T03:18:22.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.778+0000 7f5f8c0e9000 1 Processor -- start 2023-11-06T03:18:22.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c3c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.798 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c371dd0 con 0x558c3c3b8000 2023-11-06T03:18:22.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c3c18aa00 con 0x558c3c36da90 2023-11-06T03:18:22.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c3c1879f0 con 0x558c3c3b76e0 2023-11-06T03:18:22.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 10 client.0 _ll_register_callbacks cb 0x558c3c3399e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:22.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36476/0 (socket says 172.21.15.157:36476) 2023-11-06T03:18:22.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 -- 172.21.15.157:0/1115357319 learned_addr learned my addr 172.21.15.157:0/1115357319 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8259c640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c3c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f81d9b640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 -- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 msgr2=0x558c3c3fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 -- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 msgr2=0x558c3c3c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c3c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 -- 172.21.15.157:0/1115357319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f780bc070 con 0x558c3c3b8000 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8259c640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c3c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:22.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f82d9d640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c014b40 tx=0x7f5f7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c3312e40975cf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:22.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0157a0 con 0x558c3c3b8000 2023-11-06T03:18:22.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5f7c010070 con 0x558c3c3b8000 2023-11-06T03:18:22.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c3c18cb90 con 0x558c3c3b8000 2023-11-06T03:18:22.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f7c015c60 con 0x558c3c3b8000 2023-11-06T03:18:22.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 10 client.11382 fetch_fsmap learned FSMap version 130 2023-11-06T03:18:22.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x558c3beb9cc0 con 0x558c3c3b8000 2023-11-06T03:18:22.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c3c18cb90 con 0x558c3c3b8000 2023-11-06T03:18:22.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 130) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5f7c03e070 con 0x558c3c3b8000 2023-11-06T03:18:22.803 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 10 client.11382 fetch_fsmap finished waiting for FSMap version 130 2023-11-06T03:18:22.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 10 client.11382 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:18:22.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 20 client.11382 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:22.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3fc110 0x558c3c3fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 4 client.11382 mds_command: new command op to 11244 tid=0[{"prefix": "status"}] 2023-11-06T03:18:22.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 --> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558c3c1b60e0 con 0x558c3c3fc110 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8259c640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3fc110 0x558c3c3fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 10 client.11382 ms_handle_connect on v2:172.21.15.157:6834/877366447 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8259c640 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3fc110 0x558c3c3fc4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335ccdc65620c6dd server_cookie=961b5131de7473f8 in_seq=0 out_seq=0 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 1 -- 172.21.15.157:0/1115357319 <== mds.0 v2:172.21.15.157:6834/877366447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x558c3c1b60e0 con 0x558c3c3fc110 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 10 client.11382 handle_command_reply: tid=0 2023-11-06T03:18:22.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 client.11382 shutdown 2023-11-06T03:18:22.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 msgr2=0x558c3c3fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c014b40 tx=0x7f5f7c00b040 comp rx=0 tx=0).stop 2023-11-06T03:18:22.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 shutdown_connections 2023-11-06T03:18:22.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:6834/877366447,v1:172.21.15.157:6835/877366447] conn(0x558c3c3fc110 0x558c3c3fc4f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558c3c3b8000 0x558c3c3fb800 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x558c3c3b76e0 0x558c3c3fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f617fa640 0 client.11382 ms_handle_reset on v2:172.21.15.157:6834/877366447 2023-11-06T03:18:22.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 --2- 172.21.15.157:0/1115357319 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558c3c36da90 0x558c3c3c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 >> 172.21.15.157:0/1115357319 conn(0x558c3c373bf0 msgr2=0x558c3c3b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 shutdown_connections 2023-11-06T03:18:22.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 1 -- 172.21.15.157:0/1115357319 wait complete. 2023-11-06T03:18:22.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f8c0e9000 20 client.11382 trim_cache size 0 max 16384 2023-11-06T03:18:22.808 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:22.809 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:18:22.809 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:18:22.809 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 11244, 2023-11-06T03:18:22.809 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:18:22.809 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.6861294100000004, 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 130, 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 189, 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 189, 2023-11-06T03:18:22.810 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 9.1620551920000004 2023-11-06T03:18:22.811 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:22.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 msgr2=0x7f5f64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 0x7f5f64067f20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5f780af000 tx=0x7f5f780a7000 comp rx=0 tx=0).stop 2023-11-06T03:18:22.812 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 msgr2=0x7f5f841f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f5f740011f0 tx=0x7f5f7400c9b0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 shutdown_connections 2023-11-06T03:18:22.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5f64065a60 0x7f5f64067f20 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5f841599c0 0x7f5f8414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5f840b2af0 0x7f5f84153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 --2- 172.21.15.157:0/976438550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f8406d000 0x7f5f841f37a0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 >> 172.21.15.157:0/976438550 conn(0x7f5f84065930 msgr2=0x7f5f840b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 shutdown_connections 2023-11-06T03:18:22.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.782+0000 7f5f88fc1640 1 -- 172.21.15.157:0/976438550 wait complete. 2023-11-06T03:18:22.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-06T03:18:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.922+0000 7fc325621640 1 Processor -- start 2023-11-06T03:18:22.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.922+0000 7fc325621640 1 -- start start 2023-11-06T03:18:22.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc325621640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc320160db0 0x7fc320161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc325621640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc3200ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc325621640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32006f470 con 0x7fc3200b1910 2023-11-06T03:18:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc325621640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3200b1df0 con 0x7fc320160db0 2023-11-06T03:18:22.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc325621640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32006f6c0 con 0x7fc3200701a0 2023-11-06T03:18:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc320160db0 0x7fc320161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc320160db0 0x7fc320161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36488/0 (socket says 172.21.15.157:36488) 2023-11-06T03:18:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 -- 172.21.15.157:0/699065846 learned_addr learned my addr 172.21.15.157:0/699065846 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31f7fe640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc3200ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 -- 172.21.15.157:0/699065846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 msgr2=0x7fc3200ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc3200ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 -- 172.21.15.157:0/699065846 >> v1:172.21.15.195:6789/0 conn(0x7fc3200b1910 legacy=0x7fc32006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:22.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 -- 172.21.15.157:0/699065846 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32006f9a0 con 0x7fc320160db0 2023-11-06T03:18:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31effd640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc320160db0 0x7fc320161190 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098a0 tx=0x7fc31001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65acc4284c52dc19 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31dffb640 1 -- 172.21.15.157:0/699065846 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100ab600 con 0x7fc320160db0 2023-11-06T03:18:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31dffb640 1 -- 172.21.15.157:0/699065846 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3100a7050 con 0x7fc320160db0 2023-11-06T03:18:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.926+0000 7fc31dffb640 1 -- 172.21.15.157:0/699065846 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100ab980 con 0x7fc320160db0 2023-11-06T03:18:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/699065846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc320160db0 msgr2=0x7fc320161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc320160db0 0x7fc320161190 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098a0 tx=0x7fc31001c450 comp rx=0 tx=0).stop 2023-11-06T03:18:22.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/699065846 shutdown_connections 2023-11-06T03:18:22.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc3200ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- 172.21.15.157:0/699065846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc320160db0 0x7fc320161190 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/699065846 >> 172.21.15.157:0/699065846 conn(0x7fc320065930 msgr2=0x7fc320065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:22.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/699065846 shutdown_connections 2023-11-06T03:18:22.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/699065846 wait complete. 2023-11-06T03:18:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 Processor -- start 2023-11-06T03:18:22.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- start start 2023-11-06T03:18:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc32014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc320160db0 0x7fc32014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201508c0 con 0x7fc3200b1910 2023-11-06T03:18:22.932 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320150b10 con 0x7fc320160db0 2023-11-06T03:18:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc320150df0 con 0x7fc3200701a0 2023-11-06T03:18:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36492/0 (socket says 172.21.15.157:36492) 2023-11-06T03:18:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 -- 172.21.15.157:0/4250953103 learned_addr learned my addr 172.21.15.157:0/4250953103 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:22.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31effd640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc32014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 -- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 msgr2=0x7fc32014c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc32014c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 -- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc320160db0 msgr2=0x7fc32014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:22.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc320160db0 0x7fc32014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:22.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 -- 172.21.15.157:0/4250953103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100094d0 con 0x7fc3200b1910 2023-11-06T03:18:22.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc31e7fc640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c400 tx=0x7fc31400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314016070 con 0x7fc3200b1910 2023-11-06T03:18:22.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc314007020 con 0x7fc3200b1910 2023-11-06T03:18:22.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3201510d0 con 0x7fc3200b1910 2023-11-06T03:18:22.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314007410 con 0x7fc3200b1910 2023-11-06T03:18:22.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc320151690 con 0x7fc3200b1910 2023-11-06T03:18:22.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.930+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3201518b0 con 0x7fc3200b1910 2023-11-06T03:18:22.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.934+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc314007570 con 0x7fc3200b1910 2023-11-06T03:18:22.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.934+0000 7fc3077fe640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 0x7fc30c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:22.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.934+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc314082bc0 con 0x7fc3200b1910 2023-11-06T03:18:22.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.934+0000 7fc31effd640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 0x7fc30c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:22.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.934+0000 7fc31effd640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 0x7fc30c067f20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc3100ad000 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:22.937 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:22.938+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc314050420 con 0x7fc3200b1910 2023-11-06T03:18:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc320072750 con 0x7fc3200b1910 2023-11-06T03:18:23.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc3077fe640 1 -- 172.21.15.157:0/4250953103 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v130) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc314053a40 con 0x7fc3200b1910 2023-11-06T03:18:23.108 INFO:teuthology.orchestra.run.smithi157.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-06T03:18:23.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 msgr2=0x7fc30c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 0x7fc30c067f20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc3100ad000 tx=0x7fc310008040 comp rx=0 tx=0).stop 2023-11-06T03:18:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 msgr2=0x7fc32014b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c400 tx=0x7fc31400c8c0 comp rx=0 tx=0).stop 2023-11-06T03:18:23.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 shutdown_connections 2023-11-06T03:18:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc30c065a60 0x7fc30c067f20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc320160db0 0x7fc32014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3200b1910 0x7fc32014b2f0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.110+0000 7fc325621640 1 --2- 172.21.15.157:0/4250953103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3200701a0 0x7fc32014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:23.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.114+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 >> 172.21.15.157:0/4250953103 conn(0x7fc320065930 msgr2=0x7fc320164910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:23.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.114+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 shutdown_connections 2023-11-06T03:18:23.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:23.114+0000 7fc325621640 1 -- 172.21.15.157:0/4250953103 wait complete. 2023-11-06T03:18:23.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-06T03:18:23.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-06T03:18:23.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2023-11-06T03:18:23.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-06T03:18:23.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:23.127 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-06T03:18:23.127 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-06T03:18:23.127 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-06T03:18:23.127 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:18:23.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.128 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2023-11-06T03:18:23.134 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-06T03:18:23.134 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-06T03:18:23.134 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2023-11-06T03:18:23.135 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.135 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2023-11-06T03:18:23.135 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.135 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:18:23.136 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.136 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2023-11-06T03:18:23.136 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:18:23.136 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.136 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2023-11-06T03:18:23.137 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-06T03:18:23.137 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.137 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-06T03:18:23.137 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.137 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-06T03:18:23.138 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41636sec preferred_lft 41636sec 2023-11-06T03:18:23.138 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2023-11-06T03:18:23.138 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.138 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-06T03:18:23.139 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:5e:08:01:03 brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.139 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-06T03:18:23.139 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.139 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-06T03:18:23.139 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-06T03:18:23.140 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-06T03:18:23.140 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.140 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-06T03:18:23.140 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.140 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:18:23.141 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:18:23.141 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-06T03:18:23.141 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.141 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-06T03:18:23.141 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:18:23.142 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-06T03:18:23.142 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2023-11-06T03:18:23.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.143 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-06T03:18:23.187 INFO:teuthology.orchestra.run.smithi157.stdout:1 2023-11-06T03:18:23.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.189 DEBUG:teuthology.orchestra.run.smithi157:> route 2023-11-06T03:18:23.235 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2023-11-06T03:18:23.236 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-06T03:18:23.236 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-06T03:18:23.236 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-06T03:18:23.236 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-06T03:18:23.237 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-06T03:18:23.237 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:18:23.237 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:18:23.237 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-06T03:18:23.237 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-06T03:18:23.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.238 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec bash -c ' 2023-11-06T03:18:23.239 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2023-11-06T03:18:23.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-06T03:18:23.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-06T03:18:23.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-06T03:18:23.240 DEBUG:teuthology.orchestra.run.smithi157:> ') 2023-11-06T03:18:23.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.310 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:18:23.358 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:18:23.358 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:18:23.358 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:18:23.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2023-11-06T03:18:23.413 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-06T03:18:23.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:18:23.414 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:23.463 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-06T03:18:23.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-06T03:18:23.464 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:23.510 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:18:23.567 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:18:23.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:18:23.621 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:18:23.622 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:18:23.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.623 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:18:23.670 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-06T03:18:23.671 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-06T03:18:23.680 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:18:23.682+0000 7f4c294a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:18:23.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:18:23.695 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:18:23.696 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:18:23.749 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:18:23.750 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:18:23.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:23.752 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:18:23.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:2023-11-06T03:18:23.762+0000 7fab6889d580 -1 init, newargv = 0x5614b7b72050 newargc=15 2023-11-06T03:18:23.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[98817]: starting ceph client 2023-11-06T03:18:23.767 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[98817]: starting fuse 2023-11-06T03:18:24.867 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2023-11-06T03:18:24.884 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-06T03:18:24.884 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-06T03:18:24.938 INFO:teuthology.orchestra.run.smithi157.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-06T03:18:24.939 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-06T03:18:24.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:24.941 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2023-11-06T03:18:24.987 INFO:teuthology.orchestra.run.smithi157.stdout:45 2023-11-06T03:18:24.987 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-11-06T03:18:24.988 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-06T03:18:24.988 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2023-11-06T03:18:24.988 DEBUG:teuthology.orchestra.run.smithi157:> import re 2023-11-06T03:18:24.989 DEBUG:teuthology.orchestra.run.smithi157:> import os 2023-11-06T03:18:24.989 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2023-11-06T03:18:24.990 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:18:24.990 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2023-11-06T03:18:24.990 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-06T03:18:24.991 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2023-11-06T03:18:24.991 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-06T03:18:24.992 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:18:24.992 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2023-11-06T03:18:24.993 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2023-11-06T03:18:24.993 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2023-11-06T03:18:24.993 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2023-11-06T03:18:24.994 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:18:24.994 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2023-11-06T03:18:24.995 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-06T03:18:24.995 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-06T03:18:24.995 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-06T03:18:24.996 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2023-11-06T03:18:24.996 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2023-11-06T03:18:24.996 DEBUG:teuthology.orchestra.run.smithi157:> return f 2023-11-06T03:18:24.997 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-06T03:18:24.997 DEBUG:teuthology.orchestra.run.smithi157:> 2023-11-06T03:18:24.998 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.0")) 2023-11-06T03:18:24.998 DEBUG:teuthology.orchestra.run.smithi157:> ' 2023-11-06T03:18:25.105 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.0.98817.asok 2023-11-06T03:18:25.288 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98817.asok 2023-11-06T03:18:25.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.98817.asok status 2023-11-06T03:18:25.445 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:25.446 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2023-11-06T03:18:25.446 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "f7e9f9af51acf98e0ccc3037487018e4ae40caa5", 2023-11-06T03:18:25.447 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.2.0-1181-gf7e9f9af (f7e9f9af51acf98e0ccc3037487018e4ae40caa5) reef (stable)", 2023-11-06T03:18:25.447 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "0", 2023-11-06T03:18:25.447 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2023-11-06T03:18:25.448 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-06T03:18:25.448 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "98817", 2023-11-06T03:18:25.449 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2023-11-06T03:18:25.449 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:25.449 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2023-11-06T03:18:25.450 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2023-11-06T03:18:25.450 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 11394, 2023-11-06T03:18:25.450 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2023-11-06T03:18:25.451 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2023-11-06T03:18:25.451 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2023-11-06T03:18:25.451 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 11394 2023-11-06T03:18:25.452 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:25.452 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:18:25.453 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:18:25.453 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:18:25.453 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 4094825352 2023-11-06T03:18:25.454 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:25.454 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:25.454 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2023-11-06T03:18:25.454 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2023-11-06T03:18:25.455 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2023-11-06T03:18:25.455 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 4094825352 2023-11-06T03:18:25.455 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2023-11-06T03:18:25.456 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.11394 192.168.0.1:0/4094825352", 2023-11-06T03:18:25.456 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/4094825352", 2023-11-06T03:18:25.457 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2023-11-06T03:18:25.457 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 130, 2023-11-06T03:18:25.457 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 189, 2023-11-06T03:18:25.458 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2023-11-06T03:18:25.458 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2023-11-06T03:18:25.458 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2023-11-06T03:18:25.459 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:25.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.461 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2023-11-06T03:18:25.462 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:25.469 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2023-11-06T03:18:25.469 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:25.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.470 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:25.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.531 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2023-11-06T03:18:25.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.588 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2023-11-06T03:18:25.639 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:25.640 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:25.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.688 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T03:18:25.807 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records in 2023-11-06T03:18:25.808 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records out 2023-11-06T03:18:25.808 INFO:teuthology.orchestra.run.smithi157.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.070536 s, 89.2 MB/s 2023-11-06T03:18:25.809 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:25.809 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:25.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:25.816 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2023-11-06T03:18:25.943 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records in 2023-11-06T03:18:25.943 INFO:teuthology.orchestra.run.smithi157.stderr:6+0 records out 2023-11-06T03:18:25.944 INFO:teuthology.orchestra.run.smithi157.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0764692 s, 82.3 MB/s 2023-11-06T03:18:25.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:18:25.948 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:18:25.975 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-06T03:18:25.975 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-06T03:18:25.976 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:18:25.976 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-06T03:18:25.976 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.976 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-06T03:18:25.976 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.977 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-06T03:18:25.977 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.977 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-06T03:18:25.977 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-06T03:18:25.977 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.978 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-06T03:18:25.978 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-06T03:18:25.978 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:18:25.979 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-06T03:18:25.979 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.980 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:18:25.980 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:18:25.980 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.981 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-06T03:18:25.981 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.981 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-06T03:18:25.981 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-06T03:18:25.982 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:18:25.982 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:18:25.982 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-06T03:18:25.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:18:25.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-06T03:18:25.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:18:25.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:18:25.983 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-06T03:18:25.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.984 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:18:25.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.984 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-06T03:18:25.984 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-06T03:18:25.985 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:18:25.986 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:18:25.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.986 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-06T03:18:25.986 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-06T03:18:25.986 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.988 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-06T03:18:25.988 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-06T03:18:25.988 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-06T03:18:25.989 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-06T03:18:25.990 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-06T03:18:25.990 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.991 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-06T03:18:25.991 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-06T03:18:25.991 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.991 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.992 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:18:25.993 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-06T03:18:25.993 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.993 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-06T03:18:25.993 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-06T03:18:25.993 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-06T03:18:25.994 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-06T03:18:25.994 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-06T03:18:25.994 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-06T03:18:25.995 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-06T03:18:25.995 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-06T03:18:25.995 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-06T03:18:25.995 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:25.995 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-06T03:18:25.996 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-06T03:18:25.996 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:18:25.996 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-06T03:18:25.997 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:18:25.998 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:18:25.998 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-06T03:18:25.998 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:18:25.998 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:25.999 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-06T03:18:25.999 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:25.999 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-06T03:18:25.999 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:18:25.999 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:18:26.000 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.000 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.000 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-06T03:18:26.000 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.000 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-06T03:18:26.001 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.001 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:18:26.001 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-06T03:18:26.001 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:18:26.001 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.002 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.003 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:18:26.003 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:18:26.003 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:18:26.003 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.003 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:18:26.004 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.005 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:18:26.005 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-06T03:18:26.005 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.005 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-06T03:18:26.006 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-06T03:18:26.006 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-06T03:18:26.006 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-06T03:18:26.006 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:18:26.006 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.007 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-06T03:18:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-06T03:18:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-06T03:18:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:18:26.008 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-06T03:18:26.009 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.010 INFO:teuthology.orchestra.run.smithi157.stdout: mds cache max size = 1000 2023-11-06T03:18:26.010 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-06T03:18:26.010 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = False 2023-11-06T03:18:26.010 INFO:teuthology.orchestra.run.smithi157.stdout: mds log max segments = 1 2023-11-06T03:18:26.010 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-06T03:18:26.011 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = False 2023-11-06T03:18:26.011 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:18:26.011 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.011 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.013 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-06T03:18:26.013 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-06T03:18:26.013 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:18:26.014 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-06T03:18:26.014 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.014 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-06T03:18:26.014 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.014 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-06T03:18:26.015 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.015 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-06T03:18:26.015 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-06T03:18:26.015 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.015 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-06T03:18:26.016 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-06T03:18:26.016 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.016 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:18:26.016 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-06T03:18:26.016 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.017 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:18:26.017 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:18:26.017 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.017 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-06T03:18:26.017 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.018 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-06T03:18:26.018 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-06T03:18:26.018 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:18:26.018 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:18:26.018 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-06T03:18:26.019 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:18:26.019 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-06T03:18:26.019 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:18:26.019 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:18:26.019 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.020 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-06T03:18:26.021 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.021 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-06T03:18:26.021 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-06T03:18:26.021 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-06T03:18:26.021 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:18:26.022 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:18:26.022 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.022 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-06T03:18:26.022 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-06T03:18:26.022 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.023 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-06T03:18:26.023 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-06T03:18:26.023 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:18:26.023 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.023 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-06T03:18:26.024 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-06T03:18:26.024 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.024 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-06T03:18:26.024 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-06T03:18:26.024 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:18:26.025 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.026 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-06T03:18:26.026 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.026 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-06T03:18:26.026 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.026 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:18:26.027 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-06T03:18:26.027 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.027 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-06T03:18:26.027 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-06T03:18:26.029 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-06T03:18:26.029 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-06T03:18:26.029 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2023-11-06T03:18:26.029 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-06T03:18:26.030 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-06T03:18:26.030 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-06T03:18:26.030 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-06T03:18:26.030 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.030 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-06T03:18:26.031 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-06T03:18:26.031 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:18:26.031 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-06T03:18:26.031 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:18:26.031 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:18:26.032 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-06T03:18:26.032 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:18:26.032 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.032 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-06T03:18:26.032 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.033 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-06T03:18:26.034 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.034 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-06T03:18:26.034 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.034 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:18:26.034 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-06T03:18:26.035 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:18:26.035 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-06T03:18:26.035 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-06T03:18:26.035 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.035 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-06T03:18:26.036 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-06T03:18:26.036 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.036 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.036 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:18:26.036 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:18:26.037 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:18:26.037 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.037 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:18:26.037 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:18:26.037 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:18:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-06T03:18:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:18:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:18:26.039 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2023-11-06T03:18:26.039 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.039 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-06T03:18:26.039 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-06T03:18:26.039 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2023-11-06T03:18:26.040 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.041 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2023-11-06T03:18:26.041 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.041 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.041 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-06T03:18:26.041 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-06T03:18:26.042 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-06T03:18:26.042 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:18:26.042 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:18:26.042 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:18:26.042 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:18:26.043 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2023-11-06T03:18:26.043 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2023-11-06T03:18:26.043 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2023-11-06T03:18:26.043 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.043 INFO:teuthology.orchestra.run.smithi195.stdout: mds cache max size = 1000 2023-11-06T03:18:26.044 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2023-11-06T03:18:26.044 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = False 2023-11-06T03:18:26.044 INFO:teuthology.orchestra.run.smithi195.stdout: mds log max segments = 1 2023-11-06T03:18:26.044 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2023-11-06T03:18:26.044 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2023-11-06T03:18:26.045 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:18:26.045 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.045 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:18:26.059 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-06T03:18:26.086 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2023-11-06T03:18:26.087 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2023-11-06T03:18:26.087 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:18:26.087 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2023-11-06T03:18:26.087 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2023-11-06T03:18:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2023-11-06T03:18:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2023-11-06T03:18:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2023-11-06T03:18:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2023-11-06T03:18:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2023-11-06T03:18:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:18:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2023-11-06T03:18:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:18:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:18:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2023-11-06T03:18:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2023-11-06T03:18:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2023-11-06T03:18:26.092 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:18:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:18:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2023-11-06T03:18:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:18:26.093 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.094 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2023-11-06T03:18:26.094 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.094 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2023-11-06T03:18:26.094 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.094 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2023-11-06T03:18:26.095 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2023-11-06T03:18:26.095 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2023-11-06T03:18:26.095 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:18:26.095 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:18:26.095 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2023-11-06T03:18:26.096 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:18:26.097 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.097 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2023-11-06T03:18:26.097 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2023-11-06T03:18:26.097 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.097 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2023-11-06T03:18:26.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2023-11-06T03:18:26.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2023-11-06T03:18:26.098 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.098 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2023-11-06T03:18:26.099 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2023-11-06T03:18:26.099 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.100 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:18:26.100 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.100 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2023-11-06T03:18:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2023-11-06T03:18:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:18:26.101 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2023-11-06T03:18:26.102 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.102 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2023-11-06T03:18:26.102 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2023-11-06T03:18:26.102 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2023-11-06T03:18:26.102 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2023-11-06T03:18:26.103 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2023-11-06T03:18:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2023-11-06T03:18:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:18:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2023-11-06T03:18:26.104 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:18:26.105 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:18:26.105 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2023-11-06T03:18:26.105 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:18:26.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.105 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2023-11-06T03:18:26.106 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.106 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2023-11-06T03:18:26.106 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:18:26.106 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:18:26.106 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.107 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2023-11-06T03:18:26.108 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2023-11-06T03:18:26.108 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2023-11-06T03:18:26.108 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2023-11-06T03:18:26.108 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2023-11-06T03:18:26.108 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.109 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2023-11-06T03:18:26.109 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2023-11-06T03:18:26.109 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.109 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.109 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:18:26.110 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:18:26.110 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:18:26.110 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.110 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:18:26.110 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:18:26.112 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2023-11-06T03:18:26.112 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.112 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2023-11-06T03:18:26.112 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2023-11-06T03:18:26.112 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2023-11-06T03:18:26.113 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2023-11-06T03:18:26.113 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:18:26.113 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:18:26.113 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2023-11-06T03:18:26.113 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2023-11-06T03:18:26.114 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.114 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2023-11-06T03:18:26.114 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.114 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.114 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2023-11-06T03:18:26.115 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2023-11-06T03:18:26.116 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2023-11-06T03:18:26.116 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2023-11-06T03:18:26.116 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2023-11-06T03:18:26.116 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2023-11-06T03:18:26.116 INFO:teuthology.orchestra.run.smithi157.stdout: mds cache max size = 1000 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = False 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: mds log max segments = 1 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = False 2023-11-06T03:18:26.117 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2023-11-06T03:18:26.118 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.118 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.119 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-06T03:18:26.120 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-06T03:18:26.120 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-06T03:18:26.120 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-06T03:18:26.120 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-06T03:18:26.121 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-06T03:18:26.122 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.122 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-06T03:18:26.122 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-06T03:18:26.122 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.122 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-06T03:18:26.123 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-06T03:18:26.123 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.123 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-06T03:18:26.123 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-06T03:18:26.123 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.124 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-06T03:18:26.124 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.124 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-06T03:18:26.124 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-06T03:18:26.124 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-06T03:18:26.125 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-06T03:18:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-06T03:18:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-06T03:18:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.126 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-06T03:18:26.127 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.127 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-06T03:18:26.127 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.127 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-06T03:18:26.127 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-06T03:18:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-06T03:18:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.157,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2023-11-06T03:18:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = eca2e009-f15d-40e2-b739-48eaa1ef5309 2023-11-06T03:18:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.128 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-06T03:18:26.129 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.130 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-06T03:18:26.130 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-06T03:18:26.130 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.130 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-06T03:18:26.130 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.131 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-06T03:18:26.132 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.132 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-06T03:18:26.132 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.132 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-06T03:18:26.132 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-06T03:18:26.133 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-06T03:18:26.134 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-06T03:18:26.134 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2023-11-06T03:18:26.134 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-06T03:18:26.134 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-06T03:18:26.134 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-06T03:18:26.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-06T03:18:26.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-06T03:18:26.135 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-06T03:18:26.135 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-06T03:18:26.136 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-06T03:18:26.136 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-06T03:18:26.136 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-06T03:18:26.136 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-06T03:18:26.136 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:18:26.137 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:18:26.138 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.138 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.138 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-06T03:18:26.138 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.138 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-06T03:18:26.139 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-06T03:18:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-06T03:18:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-06T03:18:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-06T03:18:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-06T03:18:26.140 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.141 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-06T03:18:26.141 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-06T03:18:26.141 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-06T03:18:26.141 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.141 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-06T03:18:26.142 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-06T03:18:26.142 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-06T03:18:26.142 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.142 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-06T03:18:26.142 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-06T03:18:26.143 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-06T03:18:26.144 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-06T03:18:26.144 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-06T03:18:26.144 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-06T03:18:26.144 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-06T03:18:26.144 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.145 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-06T03:18:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-06T03:18:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-06T03:18:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-06T03:18:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.c] 2023-11-06T03:18:26.146 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2023-11-06T03:18:26.147 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.d] 2023-11-06T03:18:26.147 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2023-11-06T03:18:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2023-11-06T03:18:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2023-11-06T03:18:26.147 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds cache max size = 1000 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = False 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds log max segments = 1 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2023-11-06T03:18:26.148 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = False 2023-11-06T03:18:26.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2023-11-06T03:18:26.149 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2023-11-06T03:18:26.149 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2023-11-06T03:18:26.149 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:26.150 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:26.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.342+0000 7f47c4a5e640 1 Processor -- start 2023-11-06T03:18:26.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.342+0000 7f47c4a5e640 1 -- start start 2023-11-06T03:18:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47c4a5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47c00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47c4a5e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f47c015ea60 0x7f47c00b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47c4a5e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015f580 con 0x7f47c01551e0 2023-11-06T03:18:26.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47c4a5e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c00704b0 con 0x7f47c015ea60 2023-11-06T03:18:26.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47c4a5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015efc0 con 0x7f47c015e5e0 2023-11-06T03:18:26.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47be59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f47c01551e0 0x7f47c006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38586/0 (socket says 172.21.15.157:38586) 2023-11-06T03:18:26.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47be59c640 1 -- 172.21.15.157:0/501880788 learned_addr learned my addr 172.21.15.157:0/501880788 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:26.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782918264 0 0) 0x7f47c015f580 con 0x7f47c01551e0 2023-11-06T03:18:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a8003530 con 0x7f47c01551e0 2023-11-06T03:18:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bdd9b640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47c00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 --2- 172.21.15.157:0/501880788 >> v2:172.21.15.195:3300/0 conn(0x7f47c015ea60 0x7f47c00b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690838728 0 0) 0x7f47a8003530 con 0x7f47c01551e0 2023-11-06T03:18:26.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 msgr2=0x7f47c00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47c00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 >> v2:172.21.15.195:3300/0 conn(0x7f47c015ea60 msgr2=0x7f47c00b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 --2- 172.21.15.157:0/501880788 >> v2:172.21.15.195:3300/0 conn(0x7f47c015ea60 0x7f47c00b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c015f2a0 con 0x7f47c01551e0 2023-11-06T03:18:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f47b4001680 con 0x7f47c01551e0 2023-11-06T03:18:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 >> v1:172.21.15.157:6789/0 conn(0x7f47c01551e0 legacy=0x7f47c006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47a8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47a8003fb0 0x7f47a80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00704b0 con 0x7f47a8004d10 2023-11-06T03:18:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015efc0 con 0x7f47a8003fb0 2023-11-06T03:18:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800bb50 con 0x7f47c015e5e0 2023-11-06T03:18:26.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bdd9b640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47a8003fb0 0x7f47a80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47be59c640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47a8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 -- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 msgr2=0x7f47a8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c015e5e0 0x7f47a8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 -- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47a8003fb0 msgr2=0x7f47a80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47a8003fb0 0x7f47a80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 -- 172.21.15.157:0/501880788 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8003530 con 0x7f47a8004d10 2023-11-06T03:18:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.346+0000 7f47bed9d640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007330 tx=0x7f47b801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739ad271a474b56 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80a5c30 con 0x7f47a8004d10 2023-11-06T03:18:26.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80a5d90 con 0x7f47a8004d10 2023-11-06T03:18:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bd59a640 1 -- 172.21.15.157:0/501880788 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80ad680 con 0x7f47a8004d10 2023-11-06T03:18:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/501880788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a8004d10 msgr2=0x7f47a8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/501880788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a8004d10 0x7f47a8005190 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007330 tx=0x7f47b801ce40 comp rx=0 tx=0).stop 2023-11-06T03:18:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 -- 172.21.15.157:0/501880788 reap_dead start 2023-11-06T03:18:26.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/501880788 shutdown_connections 2023-11-06T03:18:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/501880788 >> 172.21.15.157:0/501880788 conn(0x7f47c0065930 msgr2=0x7f47c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/501880788 shutdown_connections 2023-11-06T03:18:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/501880788 wait complete. 2023-11-06T03:18:26.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 Processor -- start 2023-11-06T03:18:26.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- start start 2023-11-06T03:18:26.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c01599d0 0x7f47c00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c015dee0 0x7f47c0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00ae300 con 0x7f47c006b490 2023-11-06T03:18:26.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015a2f0 con 0x7f47c015dee0 2023-11-06T03:18:26.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00ae020 con 0x7f47c01599d0 2023-11-06T03:18:26.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36504/0 (socket says 172.21.15.157:36504) 2023-11-06T03:18:26.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 -- 172.21.15.157:0/1862873561 learned_addr learned my addr 172.21.15.157:0/1862873561 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:26.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47be59c640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c01599d0 0x7f47c00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bdd9b640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c015dee0 0x7f47c0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 -- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c01599d0 msgr2=0x7f47c00ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c01599d0 0x7f47c00ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 -- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c015dee0 msgr2=0x7f47c0159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c015dee0 0x7f47c0159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 -- 172.21.15.157:0/1862873561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0150db0 con 0x7f47c006b490 2023-11-06T03:18:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47bed9d640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f47b8002240 tx=0x7f47a8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80adbd0 con 0x7f47c006b490 2023-11-06T03:18:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80add30 con 0x7f47c006b490 2023-11-06T03:18:26.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0151090 con 0x7f47c006b490 2023-11-06T03:18:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80adea0 con 0x7f47c006b490 2023-11-06T03:18:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.350+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c015e360 con 0x7f47c006b490 2023-11-06T03:18:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c0150c10 con 0x7f47c006b490 2023-11-06T03:18:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f47b80b4040 con 0x7f47c006b490 2023-11-06T03:18:26.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47aeffd640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 0x7f47a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47b811d970 con 0x7f47c006b490 2023-11-06T03:18:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47be59c640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 0x7f47a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.354+0000 7f47be59c640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 0x7f47a0067f20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f47b4001060 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.358+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f47c0150c10 con 0x7f47c006b490 2023-11-06T03:18:26.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.514+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47c0150c10 con 0x7f47c006b490 2023-11-06T03:18:26.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47aeffd640 1 -- 172.21.15.157:0/1862873561 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f47c0150c10 con 0x7f47c006b490 2023-11-06T03:18:26.515 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:26.515 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":189,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:19.101036+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:26.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 msgr2=0x7f47a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 0x7f47a0067f20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f47b4001060 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2023-11-06T03:18:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 msgr2=0x7f47c006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f47b8002240 tx=0x7f47a8005800 comp rx=0 tx=0).stop 2023-11-06T03:18:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 shutdown_connections 2023-11-06T03:18:26.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f47a0065a60 0x7f47a0067f20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47c006b490 0x7f47c006b920 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f47c015dee0 0x7f47c0159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 --2- 172.21.15.157:0/1862873561 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f47c01599d0 0x7f47c00ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 >> 172.21.15.157:0/1862873561 conn(0x7f47c0065930 msgr2=0x7f47c015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:26.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 shutdown_connections 2023-11-06T03:18:26.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.518+0000 7f47c4a5e640 1 -- 172.21.15.157:0/1862873561 wait complete. 2023-11-06T03:18:26.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:18:16.644094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:18:17.084366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:18:26.536 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:18:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 Processor -- start 2023-11-06T03:18:26.658 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 -- start start 2023-11-06T03:18:26.659 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12201551e0 0x7f122006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200b3b20 0x7f122015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122015f580 con 0x7f122015e5e0 2023-11-06T03:18:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122015efc0 con 0x7f12201551e0 2023-11-06T03:18:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1226cc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122015f2a0 con 0x7f12200b3b20 2023-11-06T03:18:26.660 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f122525d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200b3b20 0x7f122015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1224a5c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12201551e0 0x7f122006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f121ffff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f122015e5e0 0x7f12200b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:47826/0 (socket says 172.21.15.157:47826) 2023-11-06T03:18:26.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f121ffff640 1 -- 172.21.15.157:0/2259369671 learned_addr learned my addr 172.21.15.157:0/2259369671 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1224a5c640 1 -- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200b3b20 msgr2=0x7f122015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1224a5c640 1 --2- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200b3b20 0x7f122015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1224a5c640 1 -- 172.21.15.157:0/2259369671 >> v1:172.21.15.195:6789/0 conn(0x7f122015e5e0 legacy=0x7f12200b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.662+0000 7f1224a5c640 1 -- 172.21.15.157:0/2259369671 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122006f5a0 con 0x7f12201551e0 2023-11-06T03:18:26.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1224a5c640 1 --2- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12201551e0 0x7f122006fda0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12100097c0 tx=0x7f121001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81887c35c9059860 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f121f7fe640 1 -- 172.21.15.157:0/2259369671 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100ac640 con 0x7f12201551e0 2023-11-06T03:18:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f121f7fe640 1 -- 172.21.15.157:0/2259369671 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12100ac7a0 con 0x7f12201551e0 2023-11-06T03:18:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f121f7fe640 1 -- 172.21.15.157:0/2259369671 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100acb30 con 0x7f12201551e0 2023-11-06T03:18:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12201551e0 msgr2=0x7f122006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12201551e0 0x7f122006fda0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12100097c0 tx=0x7f121001c3c0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- 172.21.15.157:0/2259369671 shutdown_connections 2023-11-06T03:18:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12200b3b20 0x7f122015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- 172.21.15.157:0/2259369671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12201551e0 0x7f122006fda0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- 172.21.15.157:0/2259369671 >> 172.21.15.157:0/2259369671 conn(0x7f1220065930 msgr2=0x7f1220065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:26.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- 172.21.15.157:0/2259369671 shutdown_connections 2023-11-06T03:18:26.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- 172.21.15.157:0/2259369671 wait complete. 2023-11-06T03:18:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 Processor -- start 2023-11-06T03:18:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- start start 2023-11-06T03:18:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12201551e0 0x7f12201418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f122015e5e0 0x7f1220140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201499f0 con 0x7f12200b3b20 2023-11-06T03:18:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1220141e20 con 0x7f122015e5e0 2023-11-06T03:18:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.666+0000 7f1226cc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12201420c0 con 0x7f12201551e0 2023-11-06T03:18:26.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36510/0 (socket says 172.21.15.157:36510) 2023-11-06T03:18:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 -- 172.21.15.157:0/3607792474 learned_addr learned my addr 172.21.15.157:0/3607792474 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121ffff640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12201551e0 0x7f12201418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f122525d640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f122015e5e0 0x7f1220140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 -- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12201551e0 msgr2=0x7f12201418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12201551e0 0x7f12201418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 -- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f122015e5e0 msgr2=0x7f1220140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f122015e5e0 0x7f1220140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 -- 172.21.15.157:0/3607792474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009420 con 0x7f12200b3b20 2023-11-06T03:18:26.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1224a5c640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f1210019ff0 tx=0x7f121001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100b8650 con 0x7f12200b3b20 2023-11-06T03:18:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12100b87b0 con 0x7f12200b3b20 2023-11-06T03:18:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1220149c40 con 0x7f12200b3b20 2023-11-06T03:18:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100b8b60 con 0x7f12200b3b20 2023-11-06T03:18:26.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122014a200 con 0x7f12200b3b20 2023-11-06T03:18:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1220072750 con 0x7f12200b3b20 2023-11-06T03:18:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f12100b8cc0 con 0x7f12200b3b20 2023-11-06T03:18:26.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 0x7f120c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f121011dec0 con 0x7f12200b3b20 2023-11-06T03:18:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121ffff640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 0x7f120c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.670+0000 7f121ffff640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 0x7f120c067f20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f12201428e0 tx=0x7f1214009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.674+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f12100eb680 con 0x7f12200b3b20 2023-11-06T03:18:26.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.826+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1220071730 con 0x7f12200b3b20 2023-11-06T03:18:26.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.826+0000 7f121d7fa640 1 -- 172.21.15.157:0/3607792474 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12100b10a0 con 0x7f12200b3b20 2023-11-06T03:18:26.826 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:18:26.827 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 msgr2=0x7f120c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 0x7f120c067f20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f12201428e0 tx=0x7f1214009040 comp rx=0 tx=0).stop 2023-11-06T03:18:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 msgr2=0x7f12201423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f1210019ff0 tx=0x7f121001ac00 comp rx=0 tx=0).stop 2023-11-06T03:18:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 shutdown_connections 2023-11-06T03:18:26.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f120c065a60 0x7f120c067f20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f122015e5e0 0x7f1220140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12201551e0 0x7f12201418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 --2- 172.21.15.157:0/3607792474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12200b3b20 0x7f12201423a0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 >> 172.21.15.157:0/3607792474 conn(0x7f1220065930 msgr2=0x7f1220151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:26.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 shutdown_connections 2023-11-06T03:18:26.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.830+0000 7f1226cc0640 1 -- 172.21.15.157:0/3607792474 wait complete. 2023-11-06T03:18:26.843 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:18:26.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 Processor -- start 2023-11-06T03:18:26.958 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 -- start start 2023-11-06T03:18:26.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdb040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fdb0406d000 0x7fdb040b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0415e440 con 0x7fdb0415dea0 2023-11-06T03:18:26.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0415adb0 con 0x7fdb0406d000 2023-11-06T03:18:26.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb0a4e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0406c240 con 0x7fdb041599c0 2023-11-06T03:18:26.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb08a81640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fdb0415dea0 0x7fdb04070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38590/0 (socket says 172.21.15.157:38590) 2023-11-06T03:18:26.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb08a81640 1 -- 172.21.15.157:0/4106994635 learned_addr learned my addr 172.21.15.157:0/4106994635 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:26.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdb040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb037fe640 1 --2- 172.21.15.157:0/4106994635 >> v2:172.21.15.195:3300/0 conn(0x7fdb0406d000 0x7fdb040b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169083827 0 0) 0x7fdb0415e440 con 0x7fdb0415dea0 2023-11-06T03:18:26.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaec003530 con 0x7fdb0415dea0 2023-11-06T03:18:26.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2081532144 0 0) 0x7fdaec003530 con 0x7fdb0415dea0 2023-11-06T03:18:26.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 msgr2=0x7fdb040ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdb040ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 >> v2:172.21.15.195:3300/0 conn(0x7fdb0406d000 msgr2=0x7fdb040b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 --2- 172.21.15.157:0/4106994635 >> v2:172.21.15.195:3300/0 conn(0x7fdb0406d000 0x7fdb040b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0406c520 con 0x7fdb0415dea0 2023-11-06T03:18:26.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fdafc001a00 con 0x7fdb0415dea0 2023-11-06T03:18:26.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 >> v1:172.21.15.157:6789/0 conn(0x7fdb0415dea0 legacy=0x7fdb04070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaec003980 0x7fdaec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec0047d0 0x7fdaec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdaec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0415adb0 con 0x7fdaec003980 2023-11-06T03:18:26.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0406c240 con 0x7fdaec0047d0 2023-11-06T03:18:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec00b990 con 0x7fdb041599c0 2023-11-06T03:18:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaec003980 0x7fdaec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb08a81640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdaec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb037fe640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec0047d0 0x7fdaec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 -- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 msgr2=0x7fdaec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb041599c0 0x7fdaec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 -- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec0047d0 msgr2=0x7fdaec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:26.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec0047d0 0x7fdaec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb03fff640 1 -- 172.21.15.157:0/4106994635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec003530 con 0x7fdaec003980 2023-11-06T03:18:26.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.962+0000 7fdb037fe640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec0047d0 0x7fdaec004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:26.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb03fff640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaec003980 0x7fdaec003da0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009ae0 tx=0x7fdaf401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223ca8fcf040823 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40a6430 con 0x7fdaec003980 2023-11-06T03:18:26.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf40a6590 con 0x7fdaec003980 2023-11-06T03:18:26.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb02ffd640 1 -- 172.21.15.157:0/4106994635 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40a6940 con 0x7fdaec003980 2023-11-06T03:18:26.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/4106994635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaec003980 msgr2=0x7fdaec003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/4106994635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdaec003980 0x7fdaec003da0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009ae0 tx=0x7fdaf401c9c0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb08a81640 1 -- 172.21.15.157:0/4106994635 reap_dead start 2023-11-06T03:18:26.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/4106994635 shutdown_connections 2023-11-06T03:18:26.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/4106994635 >> 172.21.15.157:0/4106994635 conn(0x7fdb04065930 msgr2=0x7fdb04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:26.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/4106994635 shutdown_connections 2023-11-06T03:18:26.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/4106994635 wait complete. 2023-11-06T03:18:26.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 Processor -- start 2023-11-06T03:18:26.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- start start 2023-11-06T03:18:26.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 0x7fdb04161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb04158d10 0x7fdb040acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04146910 con 0x7fdb04145f20 2023-11-06T03:18:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04161f50 con 0x7fdaec003980 2023-11-06T03:18:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb0a4e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04162230 con 0x7fdb04158d10 2023-11-06T03:18:26.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb08a81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb08a81640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36528/0 (socket says 172.21.15.157:36528) 2023-11-06T03:18:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb08a81640 1 -- 172.21.15.157:0/259393141 learned_addr learned my addr 172.21.15.157:0/259393141 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:26.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb037fe640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb04158d10 0x7fdb040acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.966+0000 7fdb03fff640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 0x7fdb04161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 -- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb04158d10 msgr2=0x7fdb040acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb04158d10 0x7fdb040acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 -- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 msgr2=0x7fdb04161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:26.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 0x7fdb04161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 -- 172.21.15.157:0/259393141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb04162510 con 0x7fdb04145f20 2023-11-06T03:18:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb03fff640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 0x7fdb04161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:26.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb08a81640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f90 tx=0x7fdafc007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc00eb60 con 0x7fdb04145f20 2023-11-06T03:18:26.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc00ece0 con 0x7fdb04145f20 2023-11-06T03:18:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb041627f0 con 0x7fdb04145f20 2023-11-06T03:18:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc013660 con 0x7fdb04145f20 2023-11-06T03:18:26.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb04162ff0 con 0x7fdb04145f20 2023-11-06T03:18:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb04072750 con 0x7fdb04145f20 2023-11-06T03:18:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdafc0137c0 con 0x7fdb04145f20 2023-11-06T03:18:26.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 0x7fdaf0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:26.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdafc0835e0 con 0x7fdb04145f20 2023-11-06T03:18:26.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb03fff640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 0x7fdaf0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:26.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.970+0000 7fdb03fff640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 0x7fdaf0067f50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40ad000 tx=0x7fdaf40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:26.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:26.974+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fdafc050e30 con 0x7fdb04145f20 2023-11-06T03:18:27.125 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.126+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdb04162ca0 con 0x7fdb04145f20 2023-11-06T03:18:27.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb00ff9640 1 -- 172.21.15.157:0/259393141 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v189) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdafc054450 con 0x7fdb04145f20 2023-11-06T03:18:27.127 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:18:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 msgr2=0x7fdaf0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.128 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 0x7fdaf0067f50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40ad000 tx=0x7fdaf40046d0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 msgr2=0x7fdb041463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f90 tx=0x7fdafc007bc0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 shutdown_connections 2023-11-06T03:18:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fdaf0065a90 0x7fdaf0067f50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb04145f20 0x7fdb041463d0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fdb04158d10 0x7fdb040acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 --2- 172.21.15.157:0/259393141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec003980 0x7fdb04161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 >> 172.21.15.157:0/259393141 conn(0x7fdb04065930 msgr2=0x7fdb04164060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 shutdown_connections 2023-11-06T03:18:27.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.130+0000 7fdb0a4e4640 1 -- 172.21.15.157:0/259393141 wait complete. 2023-11-06T03:18:27.145 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:18:27.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 Processor -- start 2023-11-06T03:18:27.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 -- start start 2023-11-06T03:18:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12e0159ce0 0x7f12e006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e0071150 0x7f12e00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00733a0 con 0x7f12e00701b0 2023-11-06T03:18:27.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00cc8d0 con 0x7f12e0071150 2023-11-06T03:18:27.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12e533f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00730c0 con 0x7f12e0159ce0 2023-11-06T03:18:27.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12df7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e0071150 0x7f12e00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12df7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12e0071150 0x7f12e00c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36540/0 (socket says 172.21.15.157:36540) 2023-11-06T03:18:27.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.270+0000 7f12df7fe640 1 -- 172.21.15.157:0/1186608939 learned_addr learned my addr 172.21.15.157:0/1186608939 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12deffd640 1 --2- 172.21.15.157:0/1186608939 >> v2:172.21.15.195:3300/0 conn(0x7f12e0159ce0 0x7f12e006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12df7fe640 1 -- 172.21.15.157:0/1186608939 >> v2:172.21.15.195:3300/0 conn(0x7f12e0159ce0 msgr2=0x7f12e006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:27.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12df7fe640 1 --2- 172.21.15.157:0/1186608939 >> v2:172.21.15.195:3300/0 conn(0x7f12e0159ce0 0x7f12e006fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12df7fe640 1 -- 172.21.15.157:0/1186608939 >> v1:172.21.15.157:6789/0 conn(0x7f12e00701b0 legacy=0x7f12e0070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12df7fe640 1 -- 172.21.15.157:0/1186608939 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00cc5f0 con 0x7f12e0071150 2023-11-06T03:18:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12df7fe640 1 --2- 172.21.15.157:0/1186608939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0071150 0x7f12e00c9600 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f12e00cfff0 tx=0x7f12d801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c956b287906e50ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12ddffb640 1 -- 172.21.15.157:0/1186608939 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a5820 con 0x7f12e0071150 2023-11-06T03:18:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12ddffb640 1 -- 172.21.15.157:0/1186608939 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80a5980 con 0x7f12e0071150 2023-11-06T03:18:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12ddffb640 1 -- 172.21.15.157:0/1186608939 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a5d10 con 0x7f12e0071150 2023-11-06T03:18:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 -- 172.21.15.157:0/1186608939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0071150 msgr2=0x7f12e00c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 --2- 172.21.15.157:0/1186608939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0071150 0x7f12e00c9600 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f12e00cfff0 tx=0x7f12d801c940 comp rx=0 tx=0).stop 2023-11-06T03:18:27.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 -- 172.21.15.157:0/1186608939 shutdown_connections 2023-11-06T03:18:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 --2- 172.21.15.157:0/1186608939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e0071150 0x7f12e00c9600 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 --2- 172.21.15.157:0/1186608939 >> v2:172.21.15.195:3300/0 conn(0x7f12e0159ce0 0x7f12e006fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 -- 172.21.15.157:0/1186608939 >> 172.21.15.157:0/1186608939 conn(0x7f12e00cba30 msgr2=0x7f12e00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 -- 172.21.15.157:0/1186608939 shutdown_connections 2023-11-06T03:18:27.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.274+0000 7f12e533f640 1 -- 172.21.15.157:0/1186608939 wait complete. 2023-11-06T03:18:27.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 Processor -- start 2023-11-06T03:18:27.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- start start 2023-11-06T03:18:27.278 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0071150 0x7f12e01514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e0159ce0 0x7f12e0151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0149ad0 con 0x7f12e00701b0 2023-11-06T03:18:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01491c0 con 0x7f12e0071150 2023-11-06T03:18:27.279 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e014cdf0 con 0x7f12e0159ce0 2023-11-06T03:18:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36552/0 (socket says 172.21.15.157:36552) 2023-11-06T03:18:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 -- 172.21.15.157:0/1549008973 learned_addr learned my addr 172.21.15.157:0/1549008973 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 -- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e0159ce0 msgr2=0x7f12e0151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.280 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12df7fe640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e0159ce0 0x7f12e0151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12de7fc640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0071150 0x7f12e01514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e0159ce0 0x7f12e0151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 -- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0071150 msgr2=0x7f12e01514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0071150 0x7f12e01514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.281 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 -- 172.21.15.157:0/1549008973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8009b30 con 0x7f12e00701b0 2023-11-06T03:18:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12deffd640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d800 tx=0x7f12d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000b6c0 con 0x7f12e00701b0 2023-11-06T03:18:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000b820 con 0x7f12e00701b0 2023-11-06T03:18:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e014d0b0 con 0x7f12e00701b0 2023-11-06T03:18:27.282 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000bb90 con 0x7f12e00701b0 2023-11-06T03:18:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01496f0 con 0x7f12e00701b0 2023-11-06T03:18:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0072750 con 0x7f12e00701b0 2023-11-06T03:18:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.278+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f12d000bcf0 con 0x7f12e00701b0 2023-11-06T03:18:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.282+0000 7f12c77fe640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 0x7f12cc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.283 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.282+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12d00842d0 con 0x7f12e00701b0 2023-11-06T03:18:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.282+0000 7f12de7fc640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 0x7f12cc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.282+0000 7f12de7fc640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 0x7f12cc067f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f12e0065d10 tx=0x7f12d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.284 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.282+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f12d0051c10 con 0x7f12e00701b0 2023-11-06T03:18:27.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.422+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12e00b4f30 con 0x7f12e00701b0 2023-11-06T03:18:27.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12c77fe640 1 -- 172.21.15.157:0/1549008973 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12d0055230 con 0x7f12e00701b0 2023-11-06T03:18:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:18:27.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 msgr2=0x7f12cc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 0x7f12cc067f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f12e0065d10 tx=0x7f12d4009040 comp rx=0 tx=0).stop 2023-11-06T03:18:27.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 msgr2=0x7f12e0147f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f12d000d800 tx=0x7f12d000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 shutdown_connections 2023-11-06T03:18:27.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f12cc065a60 0x7f12cc067f20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12e0159ce0 0x7f12e0151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12e0071150 0x7f12e01514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 --2- 172.21.15.157:0/1549008973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12e00701b0 0x7f12e0147f40 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 >> 172.21.15.157:0/1549008973 conn(0x7f12e00cba30 msgr2=0x7f12e00c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 shutdown_connections 2023-11-06T03:18:27.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.426+0000 7f12e533f640 1 -- 172.21.15.157:0/1549008973 wait complete. 2023-11-06T03:18:27.440 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:18:27.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.566+0000 7fd8fd1bd640 1 Processor -- start 2023-11-06T03:18:27.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.566+0000 7fd8fd1bd640 1 -- start start 2023-11-06T03:18:27.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8fd1bd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd8f81599c0 0x7fd8f80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8fd1bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8fd1bd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f815e440 con 0x7fd8f815dea0 2023-11-06T03:18:27.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8fd1bd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f815adb0 con 0x7fd8f81599c0 2023-11-06T03:18:27.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8fd1bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f806c240 con 0x7fd8f806d000 2023-11-06T03:18:27.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.157:43828/0 (socket says 172.21.15.157:43828) 2023-11-06T03:18:27.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 -- 172.21.15.157:0/1509168799 learned_addr learned my addr 172.21.15.157:0/1509168799 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f6d9d640 1 --2- 172.21.15.157:0/1509168799 >> v2:172.21.15.195:3300/0 conn(0x7fd8f81599c0 0x7fd8f80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 -- 172.21.15.157:0/1509168799 >> v2:172.21.15.195:3300/0 conn(0x7fd8f81599c0 msgr2=0x7fd8f80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:27.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 --2- 172.21.15.157:0/1509168799 >> v2:172.21.15.195:3300/0 conn(0x7fd8f81599c0 0x7fd8f80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 -- 172.21.15.157:0/1509168799 >> v1:172.21.15.195:6789/0 conn(0x7fd8f815dea0 legacy=0x7fd8f8070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 -- 172.21.15.157:0/1509168799 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f806c520 con 0x7fd8f806d000 2023-11-06T03:18:27.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.570+0000 7fd8f659c640 1 --2- 172.21.15.157:0/1509168799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009870 tx=0x7fd8e801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9363162c501dd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f5d9b640 1 -- 172.21.15.157:0/1509168799 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80ab620 con 0x7fd8f806d000 2023-11-06T03:18:27.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f5d9b640 1 -- 172.21.15.157:0/1509168799 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e80ab780 con 0x7fd8f806d000 2023-11-06T03:18:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f5d9b640 1 -- 172.21.15.157:0/1509168799 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80abb10 con 0x7fd8f806d000 2023-11-06T03:18:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/1509168799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 msgr2=0x7fd8f80b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/1509168799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009870 tx=0x7fd8e801c510 comp rx=0 tx=0).stop 2023-11-06T03:18:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/1509168799 shutdown_connections 2023-11-06T03:18:27.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/1509168799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f806d000 0x7fd8f80b23e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/1509168799 >> v2:172.21.15.195:3300/0 conn(0x7fd8f81599c0 0x7fd8f80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/1509168799 >> 172.21.15.157:0/1509168799 conn(0x7fd8f8065930 msgr2=0x7fd8f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/1509168799 shutdown_connections 2023-11-06T03:18:27.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/1509168799 wait complete. 2023-11-06T03:18:27.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 Processor -- start 2023-11-06T03:18:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- start start 2023-11-06T03:18:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f81599c0 0x7fd8f80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 0x7fd8f8140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80cc3e0 con 0x7fd8f806d000 2023-11-06T03:18:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80ccf60 con 0x7fd8f815dea0 2023-11-06T03:18:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8fd1bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8154700 con 0x7fd8f81599c0 2023-11-06T03:18:27.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36556/0 (socket says 172.21.15.157:36556) 2023-11-06T03:18:27.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.574+0000 7fd8f6d9d640 1 -- 172.21.15.157:0/3848757025 learned_addr learned my addr 172.21.15.157:0/3848757025 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f659c640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f81599c0 0x7fd8f80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 -- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f81599c0 msgr2=0x7fd8f80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f81599c0 0x7fd8f80cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 -- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 msgr2=0x7fd8f8140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f759e640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 0x7fd8f8140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 0x7fd8f8140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 -- 172.21.15.157:0/3848757025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80094d0 con 0x7fd8f806d000 2023-11-06T03:18:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f759e640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 0x7fd8f8140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f6d9d640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e930 tx=0x7fd8ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec009690 con 0x7fd8f806d000 2023-11-06T03:18:27.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00ca00 con 0x7fd8f806d000 2023-11-06T03:18:27.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f80cc630 con 0x7fd8f806d000 2023-11-06T03:18:27.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00cd70 con 0x7fd8f806d000 2023-11-06T03:18:27.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f80ccbf0 con 0x7fd8f806d000 2023-11-06T03:18:27.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f8072750 con 0x7fd8f806d000 2023-11-06T03:18:27.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8ec013470 con 0x7fd8f806d000 2023-11-06T03:18:27.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 0x7fd8e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8ec0834e0 con 0x7fd8f806d000 2023-11-06T03:18:27.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f659c640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 0x7fd8e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.578+0000 7fd8f659c640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 0x7fd8e4067f20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80af000 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.582+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd8ec050e90 con 0x7fd8f806d000 2023-11-06T03:18:27.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8f80adcd0 con 0x7fd8f806d000 2023-11-06T03:18:27.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8df7fe640 1 -- 172.21.15.157:0/3848757025 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8ec0544b0 con 0x7fd8f806d000 2023-11-06T03:18:27.735 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:18:27.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 msgr2=0x7fd8e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 0x7fd8e4067f20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80af000 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2023-11-06T03:18:27.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 msgr2=0x7fd8f81541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e930 tx=0x7fd8ec00edf0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 shutdown_connections 2023-11-06T03:18:27.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd8e4065a60 0x7fd8e4067f20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8f815dea0 0x7fd8f8140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8f81599c0 0x7fd8f80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 --2- 172.21.15.157:0/3848757025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8f806d000 0x7fd8f81541c0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 >> 172.21.15.157:0/3848757025 conn(0x7fd8f8065930 msgr2=0x7fd8f8070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 shutdown_connections 2023-11-06T03:18:27.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.738+0000 7fd8fd1bd640 1 -- 172.21.15.157:0/3848757025 wait complete. 2023-11-06T03:18:27.755 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:18:27.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 Processor -- start 2023-11-06T03:18:27.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 -- start start 2023-11-06T03:18:27.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07441551e0 0x7f074406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 0x7f07440b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f074415ea90 0x7f07440b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074415f580 con 0x7f07441551e0 2023-11-06T03:18:27.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074415efc0 con 0x7f074415e5e0 2023-11-06T03:18:27.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0749062640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074415f2a0 con 0x7f074415ea90 2023-11-06T03:18:27.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07441551e0 0x7f074406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f07441551e0 0x7f074406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36568/0 (socket says 172.21.15.157:36568) 2023-11-06T03:18:27.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 -- 172.21.15.157:0/3292750075 learned_addr learned my addr 172.21.15.157:0/3292750075 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f074359e640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f074415ea90 0x7f07440b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f074259c640 1 --2- 172.21.15.157:0/3292750075 >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 0x7f07440b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 -- 172.21.15.157:0/3292750075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f074415ea90 msgr2=0x7f07440b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:27.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f074415ea90 0x7f07440b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 -- 172.21.15.157:0/3292750075 >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 msgr2=0x7f07440b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:27.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3292750075 >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 0x7f07440b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f0742d9d640 1 -- 172.21.15.157:0/3292750075 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074406f5a0 con 0x7f07441551e0 2023-11-06T03:18:27.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.874+0000 7f074259c640 1 --2- 172.21.15.157:0/3292750075 >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 0x7f07440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:27.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07441551e0 0x7f074406fda0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f0744066880 tx=0x7f073401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1acab2977831e0f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0741d9b640 1 -- 172.21.15.157:0/3292750075 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340ab620 con 0x7f07441551e0 2023-11-06T03:18:27.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0741d9b640 1 -- 172.21.15.157:0/3292750075 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07340ab780 con 0x7f07441551e0 2023-11-06T03:18:27.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0741d9b640 1 -- 172.21.15.157:0/3292750075 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07340abb10 con 0x7f07441551e0 2023-11-06T03:18:27.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 -- 172.21.15.157:0/3292750075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07441551e0 msgr2=0x7f074406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07441551e0 0x7f074406fda0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f0744066880 tx=0x7f073401c440 comp rx=0 tx=0).stop 2023-11-06T03:18:27.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 -- 172.21.15.157:0/3292750075 shutdown_connections 2023-11-06T03:18:27.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f074415ea90 0x7f07440b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 --2- 172.21.15.157:0/3292750075 >> v2:172.21.15.195:3300/0 conn(0x7f074415e5e0 0x7f07440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 --2- 172.21.15.157:0/3292750075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07441551e0 0x7f074406fda0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 -- 172.21.15.157:0/3292750075 >> 172.21.15.157:0/3292750075 conn(0x7f0744065930 msgr2=0x7f0744065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:27.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 -- 172.21.15.157:0/3292750075 shutdown_connections 2023-11-06T03:18:27.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.878+0000 7f0749062640 1 -- 172.21.15.157:0/3292750075 wait complete. 2023-11-06T03:18:27.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 Processor -- start 2023-11-06T03:18:27.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- start start 2023-11-06T03:18:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07441551e0 0x7f0744145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f074415e5e0 0x7f0744144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074415a540 con 0x7f074415ea90 2023-11-06T03:18:27.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074415a820 con 0x7f074415e5e0 2023-11-06T03:18:27.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074415ab00 con 0x7f07441551e0 2023-11-06T03:18:27.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36580/0 (socket says 172.21.15.157:36580) 2023-11-06T03:18:27.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 -- 172.21.15.157:0/3901729074 learned_addr learned my addr 172.21.15.157:0/3901729074 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074259c640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f074415e5e0 0x7f0744144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07441551e0 0x7f0744145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 -- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07441551e0 msgr2=0x7f0744145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07441551e0 0x7f0744145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 -- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f074415e5e0 msgr2=0x7f0744144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f074415e5e0 0x7f0744144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 -- 172.21.15.157:0/3901729074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340094d0 con 0x7f074415ea90 2023-11-06T03:18:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f074359e640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f073c007e60 tx=0x7f073c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c014420 con 0x7f074415ea90 2023-11-06T03:18:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c0145a0 con 0x7f074415ea90 2023-11-06T03:18:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c014950 con 0x7f074415ea90 2023-11-06T03:18:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074406b530 con 0x7f074415ea90 2023-11-06T03:18:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074414b460 con 0x7f074415ea90 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0744072750 con 0x7f074415ea90 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.882+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f073c014ab0 con 0x7f074415ea90 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.886+0000 7f073b7fe640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 0x7f0724067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.886+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f073c083b90 con 0x7f074415ea90 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.886+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 0x7f0724067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:27.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.886+0000 7f0742d9d640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 0x7f0724067f20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f07340af000 tx=0x7f07340a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:27.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:27.890+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f073c051480 con 0x7f074415ea90 2023-11-06T03:18:28.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f074406b9e0 con 0x7f074415ea90 2023-11-06T03:18:28.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f073b7fe640 1 -- 172.21.15.157:0/3901729074 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v189) v1 ==== 80+0+0 (secure 0 0 0) 0x7f073c054aa0 con 0x7f074415ea90 2023-11-06T03:18:28.043 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:18:28.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 msgr2=0x7f0724067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 0x7f0724067f20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f07340af000 tx=0x7f07340a7000 comp rx=0 tx=0).stop 2023-11-06T03:18:28.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 msgr2=0x7f0744145400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f073c007e60 tx=0x7f073c00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 shutdown_connections 2023-11-06T03:18:28.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0724065a60 0x7f0724067f20 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f074415ea90 0x7f0744145400 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f074415e5e0 0x7f0744144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 --2- 172.21.15.157:0/3901729074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07441551e0 0x7f0744145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 >> 172.21.15.157:0/3901729074 conn(0x7f0744065930 msgr2=0x7f07440b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 shutdown_connections 2023-11-06T03:18:28.049 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.046+0000 7f0749062640 1 -- 172.21.15.157:0/3901729074 wait complete. 2023-11-06T03:18:28.060 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:18:28.060 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:18:28.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:18:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.230+0000 7f7095a36640 1 Processor -- start 2023-11-06T03:18:28.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.230+0000 7f7095a36640 1 -- start start 2023-11-06T03:18:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f7095a36640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f709015ce80 0x7f70900cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f7095a36640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70900d2920 con 0x7f70900d00b0 2023-11-06T03:18:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f7095a36640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70900d18a0 con 0x7f709015caa0 2023-11-06T03:18:28.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f7095a36640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70900d0520 con 0x7f709015ce80 2023-11-06T03:18:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708f7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f70900d00b0 0x7f70900b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:38602/0 (socket says 172.21.15.157:38602) 2023-11-06T03:18:28.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708f7fe640 1 -- 172.21.15.157:0/2572337385 learned_addr learned my addr 172.21.15.157:0/2572337385 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:28.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f70867fc640 1 --2- 172.21.15.157:0/2572337385 >> v2:172.21.15.195:3300/0 conn(0x7f709015ce80 0x7f70900cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938710885 0 0) 0x7f70900d2920 con 0x7f70900d00b0 2023-11-06T03:18:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707c003530 con 0x7f70900d00b0 2023-11-06T03:18:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31787379 0 0) 0x7f707c003530 con 0x7f70900d00b0 2023-11-06T03:18:28.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 >> v2:172.21.15.195:3300/0 conn(0x7f709015ce80 msgr2=0x7f70900cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 --2- 172.21.15.157:0/2572337385 >> v2:172.21.15.195:3300/0 conn(0x7f709015ce80 0x7f70900cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 >> v1:172.21.15.195:6789/0 conn(0x7f709015caa0 legacy=0x7f70900cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900d1b60 con 0x7f70900d00b0 2023-11-06T03:18:28.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7088001a00 con 0x7f70900d00b0 2023-11-06T03:18:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 >> v1:172.21.15.157:6789/0 conn(0x7f70900d00b0 legacy=0x7f70900b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f707c0047d0 0x7f707c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707c005190 0x7f707c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70900d0520 con 0x7f707c003980 2023-11-06T03:18:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c00c030 con 0x7f707c0047d0 2023-11-06T03:18:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c00c1f0 con 0x7f707c005190 2023-11-06T03:18:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f70867fc640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f707c0047d0 0x7f707c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708f7fe640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707c005190 0x7f707c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 -- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707c005190 msgr2=0x7f707c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f707c005190 0x7f707c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 -- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f707c0047d0 msgr2=0x7f707c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f707c0047d0 0x7f707c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708effd640 1 -- 172.21.15.157:0/2572337385 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c003530 con 0x7f707c003980 2023-11-06T03:18:28.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.234+0000 7f708f7fe640 1 -- 172.21.15.157:0/2572337385 reap_dead start 2023-11-06T03:18:28.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f707c003da0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f7080001780 tx=0x7f708001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cbd3c8444ce59f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800052d0 con 0x7f707c003980 2023-11-06T03:18:28.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080005450 con 0x7f707c003980 2023-11-06T03:18:28.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708e7fc640 1 -- 172.21.15.157:0/2572337385 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080003840 con 0x7f707c003980 2023-11-06T03:18:28.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 msgr2=0x7f707c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f707c003da0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f7080001780 tx=0x7f708001c3b0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- 172.21.15.157:0/2572337385 shutdown_connections 2023-11-06T03:18:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 --2- 172.21.15.157:0/2572337385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- 172.21.15.157:0/2572337385 >> 172.21.15.157:0/2572337385 conn(0x7f7090070790 msgr2=0x7f7090070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- 172.21.15.157:0/2572337385 shutdown_connections 2023-11-06T03:18:28.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- 172.21.15.157:0/2572337385 wait complete. 2023-11-06T03:18:28.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 Processor -- start 2023-11-06T03:18:28.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- start start 2023-11-06T03:18:28.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7090150e20 0x7f70901e9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7090152350 0x7f70900c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901512b0 con 0x7f707c003980 2023-11-06T03:18:28.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090151590 con 0x7f7090152350 2023-11-06T03:18:28.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f7095a36640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7090151870 con 0x7f7090150e20 2023-11-06T03:18:28.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:36600/0 (socket says 172.21.15.157:36600) 2023-11-06T03:18:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 -- 172.21.15.157:0/3773964661 learned_addr learned my addr 172.21.15.157:0/3773964661 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f70867fc640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7090150e20 0x7f70901e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708f7fe640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7090152350 0x7f70900c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 -- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7090150e20 msgr2=0x7f70901e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7090150e20 0x7f70901e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 -- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7090152350 msgr2=0x7f70900c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7090152350 0x7f70900c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 -- 172.21.15.157:0/3773964661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7090151b50 con 0x7f707c003980 2023-11-06T03:18:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.238+0000 7f708effd640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f708001c1e0 tx=0x7f707c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800b9070 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080007250 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7090151e30 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800073c0 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70900c6e60 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7090072750 con 0x7f707c003980 2023-11-06T03:18:28.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7080007520 con 0x7f707c003980 2023-11-06T03:18:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 0x7f7070067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f708011cae0 con 0x7f707c003980 2023-11-06T03:18:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f70867fc640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 0x7f7070067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.242+0000 7f70867fc640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 0x7f7070067f20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f70900b18b0 tx=0x7f7078008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.246+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f70800ea360 con 0x7f707c003980 2023-11-06T03:18:28.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.402+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7090072750 con 0x7f707c003980 2023-11-06T03:18:28.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.402+0000 7f708cff9640 1 -- 172.21.15.157:0/3773964661 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+17941 (secure 0 0 0) 0x7f70800ed980 con 0x7f707c003980 2023-11-06T03:18:28.401 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:28.401 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":189,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:19.101036+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:18:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 msgr2=0x7f7070067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 0x7f7070067f20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f70900b18b0 tx=0x7f7078008040 comp rx=0 tx=0).stop 2023-11-06T03:18:28.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 msgr2=0x7f70901e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f708001c1e0 tx=0x7f707c005800 comp rx=0 tx=0).stop 2023-11-06T03:18:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 shutdown_connections 2023-11-06T03:18:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7070065a60 0x7f7070067f20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7090152350 0x7f70900c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7090150e20 0x7f70901e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 --2- 172.21.15.157:0/3773964661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f707c003980 0x7f70901e8e30 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 >> 172.21.15.157:0/3773964661 conn(0x7f7090070790 msgr2=0x7f70900b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 shutdown_connections 2023-11-06T03:18:28.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.406+0000 7f7095a36640 1 -- 172.21.15.157:0/3773964661 wait complete. 2023-11-06T03:18:28.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:18:16.644094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:18:17.084366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:18:28.421 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:18:28.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.546+0000 7f0fcf76b640 1 Processor -- start 2023-11-06T03:18:28.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.546+0000 7f0fcf76b640 1 -- start start 2023-11-06T03:18:28.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcf76b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 0x7f0fc806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcf76b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc806c490 con 0x7f0fc80adae0 2023-11-06T03:18:28.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcf76b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc806bb60 con 0x7f0fc80b2330 2023-11-06T03:18:28.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcf76b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc806b950 con 0x7f0fc8156a70 2023-11-06T03:18:28.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fccd06640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0fc80adae0 0x7f0fc80b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33968/0 (socket says 172.21.15.157:33968) 2023-11-06T03:18:28.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fccd06640 1 -- 172.21.15.157:0/420889770 learned_addr learned my addr 172.21.15.157:0/420889770 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:28.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcd507640 1 --2- 172.21.15.157:0/420889770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 0x7f0fc806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcd507640 1 -- 172.21.15.157:0/420889770 >> v1:172.21.15.195:6789/0 conn(0x7f0fc80b2330 legacy=0x7f0fc806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcd507640 1 -- 172.21.15.157:0/420889770 >> v1:172.21.15.157:6789/0 conn(0x7f0fc80adae0 legacy=0x7f0fc80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcd507640 1 -- 172.21.15.157:0/420889770 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc806be40 con 0x7f0fc8156a70 2023-11-06T03:18:28.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.550+0000 7f0fcd507640 1 --2- 172.21.15.157:0/420889770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 0x7f0fc806f470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40097f0 tx=0x7f0fc401d5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de08830989877322 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fbbfff640 1 -- 172.21.15.157:0/420889770 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b1600 con 0x7f0fc8156a70 2023-11-06T03:18:28.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fbbfff640 1 -- 172.21.15.157:0/420889770 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40b1760 con 0x7f0fc8156a70 2023-11-06T03:18:28.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fbbfff640 1 -- 172.21.15.157:0/420889770 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b1af0 con 0x7f0fc8156a70 2023-11-06T03:18:28.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 -- 172.21.15.157:0/420889770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 msgr2=0x7f0fc806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/420889770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 0x7f0fc806f470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40097f0 tx=0x7f0fc401d5a0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 -- 172.21.15.157:0/420889770 shutdown_connections 2023-11-06T03:18:28.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/420889770 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc8156a70 0x7f0fc806f470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 -- 172.21.15.157:0/420889770 >> 172.21.15.157:0/420889770 conn(0x7f0fc8065930 msgr2=0x7f0fc8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 -- 172.21.15.157:0/420889770 shutdown_connections 2023-11-06T03:18:28.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.554+0000 7f0fcf76b640 1 -- 172.21.15.157:0/420889770 wait complete. 2023-11-06T03:18:28.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 Processor -- start 2023-11-06T03:18:28.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- start start 2023-11-06T03:18:28.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fc80adae0 0x7f0fc815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc80b2330 0x7f0fc81f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc8149eb0 con 0x7f0fc8156a70 2023-11-06T03:18:28.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81f9e20 con 0x7f0fc80adae0 2023-11-06T03:18:28.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81495a0 con 0x7f0fc80b2330 2023-11-06T03:18:28.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47312/0 (socket says 172.21.15.157:47312) 2023-11-06T03:18:28.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 -- 172.21.15.157:0/590239731 learned_addr learned my addr 172.21.15.157:0/590239731 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:28.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fccd06640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc80b2330 0x7f0fc81f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcd507640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fc80adae0 0x7f0fc815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 -- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc80b2330 msgr2=0x7f0fc81f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc80b2330 0x7f0fc81f98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 -- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fc80adae0 msgr2=0x7f0fc815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fc80adae0 0x7f0fc815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 -- 172.21.15.157:0/590239731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009420 con 0x7f0fc8156a70 2023-11-06T03:18:28.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcdd08640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c700 tx=0x7f0fc000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0003b80 con 0x7f0fc8156a70 2023-11-06T03:18:28.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0003ce0 con 0x7f0fc8156a70 2023-11-06T03:18:28.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc815c460 con 0x7f0fc8156a70 2023-11-06T03:18:28.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000aa90 con 0x7f0fc8156a70 2023-11-06T03:18:28.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.558+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc8149b40 con 0x7f0fc8156a70 2023-11-06T03:18:28.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc8072750 con 0x7f0fc8156a70 2023-11-06T03:18:28.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0fc000abf0 con 0x7f0fc8156a70 2023-11-06T03:18:28.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fb9ffb640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 0x7f0fac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fc0082fb0 con 0x7f0fc8156a70 2023-11-06T03:18:28.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fcd507640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 0x7f0fac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.562+0000 7f0fcd507640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 0x7f0fac067f20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40b0000 tx=0x7f0fc40b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.566+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0fc00b4070 con 0x7f0fc8156a70 2023-11-06T03:18:28.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.718+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fc8157ff0 con 0x7f0fc8156a70 2023-11-06T03:18:28.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.718+0000 7f0fb9ffb640 1 -- 172.21.15.157:0/590239731 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fc00508c0 con 0x7f0fc8156a70 2023-11-06T03:18:28.718 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:18:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 msgr2=0x7f0fac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 0x7f0fac067f20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40b0000 tx=0x7f0fc40b3040 comp rx=0 tx=0).stop 2023-11-06T03:18:28.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 msgr2=0x7f0fc8145730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c700 tx=0x7f0fc000cbc0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 shutdown_connections 2023-11-06T03:18:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0fac065a60 0x7f0fac067f20 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fc8156a70 0x7f0fc8145730 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0fc80b2330 0x7f0fc81f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 --2- 172.21.15.157:0/590239731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0fc80adae0 0x7f0fc815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 >> 172.21.15.157:0/590239731 conn(0x7f0fc8065930 msgr2=0x7f0fc81574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 shutdown_connections 2023-11-06T03:18:28.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.722+0000 7f0fcf76b640 1 -- 172.21.15.157:0/590239731 wait complete. 2023-11-06T03:18:28.736 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:18:28.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 Processor -- start 2023-11-06T03:18:28.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 -- start start 2023-11-06T03:18:28.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f24600cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600d2920 con 0x7f246015ce80 2023-11-06T03:18:28.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600d18a0 con 0x7f24600d0280 2023-11-06T03:18:28.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f2464b62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600d1b60 con 0x7f246015caa0 2023-11-06T03:18:28.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f246015ce80 0x7f24600cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33978/0 (socket says 172.21.15.157:33978) 2023-11-06T03:18:28.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 -- 172.21.15.157:0/4197990571 learned_addr learned my addr 172.21.15.157:0/4197990571 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:28.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245e59c640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f24600cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678392342 0 0) 0x7f24600d2920 con 0x7f246015ce80 2023-11-06T03:18:28.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003530 con 0x7f246015ce80 2023-11-06T03:18:28.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011705038 0 0) 0x7f244c003530 con 0x7f246015ce80 2023-11-06T03:18:28.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 msgr2=0x7f24600cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f24600cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 >> v1:172.21.15.195:6789/0 conn(0x7f24600d0280 legacy=0x7f24600b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600d1e40 con 0x7f246015ce80 2023-11-06T03:18:28.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f2448001680 con 0x7f246015ce80 2023-11-06T03:18:28.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 >> v1:172.21.15.157:6789/0 conn(0x7f246015ce80 legacy=0x7f24600cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f244c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c003f30 0x7f244c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f244c004d10 0x7f244c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600d1b60 con 0x7f244c003f30 2023-11-06T03:18:28.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00ba70 con 0x7f244c004d10 2023-11-06T03:18:28.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00bc30 con 0x7f246015caa0 2023-11-06T03:18:28.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c003f30 0x7f244c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245e59c640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f244c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245ed9d640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f244c004d10 0x7f244c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 -- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 msgr2=0x7f244c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f246015caa0 0x7f244c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 -- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f244c004d10 msgr2=0x7f244c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:28.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f244c004d10 0x7f244c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245dd9b640 1 -- 172.21.15.157:0/4197990571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c003530 con 0x7f244c003f30 2023-11-06T03:18:28.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.854+0000 7f245ed9d640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f244c004d10 0x7f244c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:28.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245dd9b640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c003f30 0x7f244c0047d0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f244800c240 tx=0x7f244801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91376785d3ffdf3c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448005ae0 con 0x7f244c003f30 2023-11-06T03:18:28.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24480ab070 con 0x7f244c003f30 2023-11-06T03:18:28.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245d59a640 1 -- 172.21.15.157:0/4197990571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448007070 con 0x7f244c003f30 2023-11-06T03:18:28.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- 172.21.15.157:0/4197990571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c003f30 msgr2=0x7f244c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 --2- 172.21.15.157:0/4197990571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f244c003f30 0x7f244c0047d0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f244800c240 tx=0x7f244801f3b0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245ed9d640 1 -- 172.21.15.157:0/4197990571 reap_dead start 2023-11-06T03:18:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- 172.21.15.157:0/4197990571 shutdown_connections 2023-11-06T03:18:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- 172.21.15.157:0/4197990571 >> 172.21.15.157:0/4197990571 conn(0x7f2460070790 msgr2=0x7f2460070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:28.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- 172.21.15.157:0/4197990571 shutdown_connections 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- 172.21.15.157:0/4197990571 wait complete. 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 Processor -- start 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- start start 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 0x7f24601595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 0x7f24601f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2460154050 con 0x7f24601f2920 2023-11-06T03:18:28.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601f5ad0 con 0x7f2460159b30 2023-11-06T03:18:28.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f2464b62640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2460157730 con 0x7f2460153c70 2023-11-06T03:18:28.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47334/0 (socket says 172.21.15.157:47334) 2023-11-06T03:18:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245ed9d640 1 -- 172.21.15.157:0/2582341078 learned_addr learned my addr 172.21.15.157:0/2582341078 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245e59c640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 0x7f24601595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245dd9b640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 0x7f24601f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.858+0000 7f245ed9d640 1 -- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 msgr2=0x7f24601595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245ed9d640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 0x7f24601595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245ed9d640 1 -- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 msgr2=0x7f24601f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245ed9d640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 0x7f24601f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245ed9d640 1 -- 172.21.15.157:0/2582341078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460157a10 con 0x7f24601f2920 2023-11-06T03:18:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245dd9b640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 0x7f24601f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:28.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245e59c640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 0x7f24601595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245ed9d640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f24600b5610 tx=0x7f245800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245800b070 con 0x7f24601f2920 2023-11-06T03:18:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2458005270 con 0x7f24601f2920 2023-11-06T03:18:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2460157cf0 con 0x7f24601f2920 2023-11-06T03:18:28.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458007030 con 0x7f24601f2920 2023-11-06T03:18:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600cb560 con 0x7f24601f2920 2023-11-06T03:18:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24600cb750 con 0x7f24601f2920 2023-11-06T03:18:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2458007190 con 0x7f24601f2920 2023-11-06T03:18:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 0x7f2440067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:28.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2458082bd0 con 0x7f24601f2920 2023-11-06T03:18:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245e59c640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 0x7f2440067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.862+0000 7f245e59c640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 0x7f2440067f50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f24500074d0 tx=0x7f245000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:28.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:28.866+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f24580504c0 con 0x7f24601f2920 2023-11-06T03:18:29.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.006+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f24601581a0 con 0x7f24601f2920 2023-11-06T03:18:29.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.006+0000 7f2456ffd640 1 -- 172.21.15.157:0/2582341078 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v189) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2458053ae0 con 0x7f24601f2920 2023-11-06T03:18:29.005 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:18:29.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 msgr2=0x7f2440067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 0x7f2440067f50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f24500074d0 tx=0x7f245000b040 comp rx=0 tx=0).stop 2023-11-06T03:18:29.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 msgr2=0x7f2460159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f24600b5610 tx=0x7f245800d7d0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 shutdown_connections 2023-11-06T03:18:29.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f2440065a90 0x7f2440067f50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24601f2920 0x7f2460159fb0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2460159b30 0x7f24601f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 --2- 172.21.15.157:0/2582341078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2460153c70 0x7f24601595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 >> 172.21.15.157:0/2582341078 conn(0x7f2460070790 msgr2=0x7f24600b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 shutdown_connections 2023-11-06T03:18:29.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.010+0000 7f2464b62640 1 -- 172.21.15.157:0/2582341078 wait complete. 2023-11-06T03:18:29.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:18:29.138 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 Processor -- start 2023-11-06T03:18:29.139 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 -- start start 2023-11-06T03:18:29.141 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f88640b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886406c490 con 0x7f88640adf40 2023-11-06T03:18:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886406fb80 con 0x7f8864156a70 2023-11-06T03:18:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88696e9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886406b950 con 0x7f88640adae0 2023-11-06T03:18:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88637fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f88640adf40 0x7f886406c920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33994/0 (socket says 172.21.15.157:33994) 2023-11-06T03:18:29.142 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88637fe640 1 -- 172.21.15.157:0/1419354923 learned_addr learned my addr 172.21.15.157:0/1419354923 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f88640b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694555005 0 0) 0x7f886406c490 con 0x7f88640adf40 2023-11-06T03:18:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003450 con 0x7f88640adf40 2023-11-06T03:18:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1859360451 0 0) 0x7f8850003450 con 0x7f88640adf40 2023-11-06T03:18:29.143 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 msgr2=0x7f88640b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f88640b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 >> v1:172.21.15.195:6789/0 conn(0x7f8864156a70 legacy=0x7f886406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886406bb60 con 0x7f88640adf40 2023-11-06T03:18:29.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f885c001a00 con 0x7f88640adf40 2023-11-06T03:18:29.144 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 >> v1:172.21.15.157:6789/0 conn(0x7f88640adf40 legacy=0x7f886406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f8850003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8850004020 0x7f885000a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885000afb0 0x7f885000d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886406b950 con 0x7f8850004020 2023-11-06T03:18:29.145 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885000dc80 con 0x7f885000afb0 2023-11-06T03:18:29.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885000de40 con 0x7f88640adae0 2023-11-06T03:18:29.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8850004020 0x7f885000a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f8862ffd640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f8850003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 -- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 msgr2=0x7f8850003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.146 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f88637fe640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885000afb0 0x7f885000d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88640adae0 0x7f8850003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 -- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885000afb0 msgr2=0x7f885000d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f885000afb0 0x7f885000d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.147 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.142+0000 7f885bfff640 1 -- 172.21.15.157:0/1419354923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850003450 con 0x7f8850004020 2023-11-06T03:18:29.150 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f885bfff640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8850004020 0x7f885000a870 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ae0 tx=0x7f884c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a7437b4fe5814a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0a5640 con 0x7f8850004020 2023-11-06T03:18:29.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0a57a0 con 0x7f8850004020 2023-11-06T03:18:29.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88627fc640 1 -- 172.21.15.157:0/1419354923 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0a5b50 con 0x7f8850004020 2023-11-06T03:18:29.151 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- 172.21.15.157:0/1419354923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8850004020 msgr2=0x7f885000a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 --2- 172.21.15.157:0/1419354923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8850004020 0x7f885000a870 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ae0 tx=0x7f884c01ca50 comp rx=0 tx=0).stop 2023-11-06T03:18:29.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88637fe640 1 -- 172.21.15.157:0/1419354923 reap_dead start 2023-11-06T03:18:29.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- 172.21.15.157:0/1419354923 shutdown_connections 2023-11-06T03:18:29.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- 172.21.15.157:0/1419354923 >> 172.21.15.157:0/1419354923 conn(0x7f8864065930 msgr2=0x7f8864065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.152 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- 172.21.15.157:0/1419354923 shutdown_connections 2023-11-06T03:18:29.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- 172.21.15.157:0/1419354923 wait complete. 2023-11-06T03:18:29.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 Processor -- start 2023-11-06T03:18:29.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- start start 2023-11-06T03:18:29.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8850004020 0x7f886414ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.153 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f886414cd90 0x7f886414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641e4430 con 0x7f88641e3a70 2023-11-06T03:18:29.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886414ef80 con 0x7f8850004020 2023-11-06T03:18:29.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f88696e9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88640aef70 con 0x7f886414cd90 2023-11-06T03:18:29.154 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f885bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f885bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47344/0 (socket says 172.21.15.157:47344) 2023-11-06T03:18:29.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.146+0000 7f885bfff640 1 -- 172.21.15.157:0/840333838 learned_addr learned my addr 172.21.15.157:0/840333838 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:29.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f88637fe640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f886414cd90 0x7f886414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f8862ffd640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8850004020 0x7f886414ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.155 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 -- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f886414cd90 msgr2=0x7f886414d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f886414cd90 0x7f886414d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 -- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8850004020 msgr2=0x7f886414ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8850004020 0x7f886414ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 -- 172.21.15.157:0/840333838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640af250 con 0x7f88641e3a70 2023-11-06T03:18:29.156 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885bfff640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ae0 tx=0x7f8850003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0be070 con 0x7f88641e3a70 2023-11-06T03:18:29.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c004020 con 0x7f88641e3a70 2023-11-06T03:18:29.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88640af530 con 0x7f88641e3a70 2023-11-06T03:18:29.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0a5640 con 0x7f88641e3a70 2023-11-06T03:18:29.157 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886414b3e0 con 0x7f88641e3a70 2023-11-06T03:18:29.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886414b5d0 con 0x7f88641e3a70 2023-11-06T03:18:29.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f884c0b4040 con 0x7f88641e3a70 2023-11-06T03:18:29.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 0x7f8844067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f884c11cc50 con 0x7f88641e3a70 2023-11-06T03:18:29.158 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f8862ffd640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 0x7f8844067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.150+0000 7f8862ffd640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 0x7f8844067f20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8854000c10 tx=0x7f8854009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.159 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.154+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f886414b5d0 con 0x7f88641e3a70 2023-11-06T03:18:29.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.306+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88640afd30 con 0x7f88641e3a70 2023-11-06T03:18:29.303 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.306+0000 7f885b7fe640 1 -- 172.21.15.157:0/840333838 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88640afd30 con 0x7f88641e3a70 2023-11-06T03:18:29.304 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:18:29.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 msgr2=0x7f8844067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 0x7f8844067f20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8854000c10 tx=0x7f8854009040 comp rx=0 tx=0).stop 2023-11-06T03:18:29.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 msgr2=0x7f88641e3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ae0 tx=0x7f8850003700 comp rx=0 tx=0).stop 2023-11-06T03:18:29.306 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 shutdown_connections 2023-11-06T03:18:29.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8844065a60 0x7f8844067f20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f886414cd90 0x7f886414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88641e3a70 0x7f88641e3ef0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 --2- 172.21.15.157:0/840333838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8850004020 0x7f886414ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.307 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 >> 172.21.15.157:0/840333838 conn(0x7f8864065930 msgr2=0x7f8864157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 shutdown_connections 2023-11-06T03:18:29.308 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.310+0000 7f88696e9640 1 -- 172.21.15.157:0/840333838 wait complete. 2023-11-06T03:18:29.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:18:29.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 Processor -- start 2023-11-06T03:18:29.435 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 -- start start 2023-11-06T03:18:29.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6d9415e5e0 0x7f6d940b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d940b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9415f580 con 0x7f6d941551e0 2023-11-06T03:18:29.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d940704b0 con 0x7f6d9415e5e0 2023-11-06T03:18:29.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9c0a0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9415efc0 con 0x7f6d9415ea90 2023-11-06T03:18:29.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f6d941551e0 0x7f6d9406fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:33998/0 (socket says 172.21.15.157:33998) 2023-11-06T03:18:29.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 -- 172.21.15.157:0/814394747 learned_addr learned my addr 172.21.15.157:0/814394747 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9a63d640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d940b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870534413 0 0) 0x7f6d9415f580 con 0x7f6d941551e0 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9963b640 1 --2- 172.21.15.157:0/814394747 >> v2:172.21.15.195:3300/0 conn(0x7f6d9415e5e0 0x7f6d940b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d80003530 con 0x7f6d941551e0 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2459179291 0 0) 0x7f6d80003530 con 0x7f6d941551e0 2023-11-06T03:18:29.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 msgr2=0x7f6d940b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d940b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 >> v2:172.21.15.195:3300/0 conn(0x7f6d9415e5e0 msgr2=0x7f6d940b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 --2- 172.21.15.157:0/814394747 >> v2:172.21.15.195:3300/0 conn(0x7f6d9415e5e0 0x7f6d940b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9415f2a0 con 0x7f6d941551e0 2023-11-06T03:18:29.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6d84001680 con 0x7f6d941551e0 2023-11-06T03:18:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9963b640 1 --2- 172.21.15.157:0/814394747 >> v2:172.21.15.195:3300/0 conn(0x7f6d9415e5e0 0x7f6d940b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 >> v1:172.21.15.157:6789/0 conn(0x7f6d941551e0 legacy=0x7f6d9406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d80003980 0x7f6d80003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d800047d0 0x7f6d80004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940704b0 con 0x7f6d80003980 2023-11-06T03:18:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9415efc0 con 0x7f6d800047d0 2023-11-06T03:18:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8000ba40 con 0x7f6d9415ea90 2023-11-06T03:18:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d80003980 0x7f6d80003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9963b640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d800047d0 0x7f6d80004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d9a63d640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 -- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 msgr2=0x7f6d80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d9415ea90 0x7f6d80005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 -- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d800047d0 msgr2=0x7f6d80004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d800047d0 0x7f6d80004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.438+0000 7f6d99e3c640 1 -- 172.21.15.157:0/814394747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80003530 con 0x7f6d80003980 2023-11-06T03:18:29.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d80003980 0x7f6d80003da0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c210 tx=0x7f6d8401cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738b196d1439df45 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840058e0 con 0x7f6d80003980 2023-11-06T03:18:29.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84005a40 con 0x7f6d80003980 2023-11-06T03:18:29.447 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d98e3a640 1 -- 172.21.15.157:0/814394747 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005e30 con 0x7f6d80003980 2023-11-06T03:18:29.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/814394747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d80003980 msgr2=0x7f6d80003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/814394747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d80003980 0x7f6d80003da0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c210 tx=0x7f6d8401cb30 comp rx=0 tx=0).stop 2023-11-06T03:18:29.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9a63d640 1 -- 172.21.15.157:0/814394747 reap_dead start 2023-11-06T03:18:29.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/814394747 shutdown_connections 2023-11-06T03:18:29.448 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/814394747 >> 172.21.15.157:0/814394747 conn(0x7f6d94065930 msgr2=0x7f6d94065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/814394747 shutdown_connections 2023-11-06T03:18:29.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/814394747 wait complete. 2023-11-06T03:18:29.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 Processor -- start 2023-11-06T03:18:29.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- start start 2023-11-06T03:18:29.449 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80003980 0x7f6d9414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d94159a00 0x7f6d94159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9415a3c0 con 0x7f6d9406cf00 2023-11-06T03:18:29.450 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9414c5c0 con 0x7f6d80003980 2023-11-06T03:18:29.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9c0a0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94152640 con 0x7f6d94159a00 2023-11-06T03:18:29.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9963b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9963b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47356/0 (socket says 172.21.15.157:47356) 2023-11-06T03:18:29.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.442+0000 7f6d9963b640 1 -- 172.21.15.157:0/2328914761 learned_addr learned my addr 172.21.15.157:0/2328914761 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:29.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80003980 0x7f6d9414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 -- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d94159a00 msgr2=0x7f6d94159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9a63d640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d94159a00 0x7f6d94159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d94159a00 0x7f6d94159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 -- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80003980 msgr2=0x7f6d9414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80003980 0x7f6d9414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 -- 172.21.15.157:0/2328914761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d94152920 con 0x7f6d9406cf00 2023-11-06T03:18:29.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9963b640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f6d90010890 tx=0x7f6d90010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000cb40 con 0x7f6d9406cf00 2023-11-06T03:18:29.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9000cca0 con 0x7f6d9406cf00 2023-11-06T03:18:29.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d94152c00 con 0x7f6d9406cf00 2023-11-06T03:18:29.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900156b0 con 0x7f6d9406cf00 2023-11-06T03:18:29.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9406d380 con 0x7f6d9406cf00 2023-11-06T03:18:29.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d94072750 con 0x7f6d9406cf00 2023-11-06T03:18:29.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6d90015810 con 0x7f6d9406cf00 2023-11-06T03:18:29.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 0x7f6d78067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d90083860 con 0x7f6d9406cf00 2023-11-06T03:18:29.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 0x7f6d78067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.446+0000 7f6d99e3c640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 0x7f6d78067f50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6d840aa000 tx=0x7f6d84001a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.450+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f6d900510c0 con 0x7f6d9406cf00 2023-11-06T03:18:29.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d94071730 con 0x7f6d9406cf00 2023-11-06T03:18:29.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d8a7fc640 1 -- 172.21.15.157:0/2328914761 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d9000ee60 con 0x7f6d9406cf00 2023-11-06T03:18:29.591 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:18:29.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 msgr2=0x7f6d78067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 0x7f6d78067f50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6d840aa000 tx=0x7f6d84001a80 comp rx=0 tx=0).stop 2023-11-06T03:18:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 msgr2=0x7f6d9413d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f6d90010890 tx=0x7f6d90010d50 comp rx=0 tx=0).stop 2023-11-06T03:18:29.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 shutdown_connections 2023-11-06T03:18:29.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f6d78065a90 0x7f6d78067f50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d94159a00 0x7f6d94159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9406cf00 0x7f6d9413d2b0 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 --2- 172.21.15.157:0/2328914761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d80003980 0x7f6d9414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 >> 172.21.15.157:0/2328914761 conn(0x7f6d94065930 msgr2=0x7f6d940b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 shutdown_connections 2023-11-06T03:18:29.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.594+0000 7f6d9c0a0640 1 -- 172.21.15.157:0/2328914761 wait complete. 2023-11-06T03:18:29.610 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-06T03:18:29.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 Processor -- start 2023-11-06T03:18:29.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 -- start start 2023-11-06T03:18:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ed80aff30 0x7f0ed80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 0x7f0ed806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed806bda0 con 0x7f0ed80b2490 2023-11-06T03:18:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80b0310 con 0x7f0ed80aff30 2023-11-06T03:18:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0edd6dd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed806b7e0 con 0x7f0ed8071db0 2023-11-06T03:18:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f0ed80b2490 0x7f0ed8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34006/0 (socket says 172.21.15.157:34006) 2023-11-06T03:18:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed77fe640 1 -- 172.21.15.157:0/2326185388 learned_addr learned my addr 172.21.15.157:0/2326185388 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:29.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ed80aff30 0x7f0ed80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed67fc640 1 --2- 172.21.15.157:0/2326185388 >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 0x7f0ed806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed6ffd640 1 -- 172.21.15.157:0/2326185388 >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 msgr2=0x7f0ed806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed6ffd640 1 --2- 172.21.15.157:0/2326185388 >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 0x7f0ed806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed6ffd640 1 -- 172.21.15.157:0/2326185388 >> v1:172.21.15.157:6789/0 conn(0x7f0ed80b2490 legacy=0x7f0ed8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed5ffb640 1 -- 172.21.15.157:0/2326185388 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921391128 0 0) 0x7f0ed806bda0 con 0x7f0ed80b2490 2023-11-06T03:18:29.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed6ffd640 1 -- 172.21.15.157:0/2326185388 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed806bac0 con 0x7f0ed80aff30 2023-11-06T03:18:29.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.726+0000 7f0ed67fc640 1 --2- 172.21.15.157:0/2326185388 >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 0x7f0ed806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed6ffd640 1 --2- 172.21.15.157:0/2326185388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80aff30 0x7f0ed80ce010 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80705f0 tx=0x7f0ec801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb9dc37bd6b13d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed5ffb640 1 -- 172.21.15.157:0/2326185388 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80ac600 con 0x7f0ed80aff30 2023-11-06T03:18:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed5ffb640 1 -- 172.21.15.157:0/2326185388 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80a8050 con 0x7f0ed80aff30 2023-11-06T03:18:29.730 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed5ffb640 1 -- 172.21.15.157:0/2326185388 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80ac980 con 0x7f0ed80aff30 2023-11-06T03:18:29.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/2326185388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80aff30 msgr2=0x7f0ed80ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/2326185388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80aff30 0x7f0ed80ce010 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80705f0 tx=0x7f0ec801c3a0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/2326185388 shutdown_connections 2023-11-06T03:18:29.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/2326185388 >> v2:172.21.15.195:3300/0 conn(0x7f0ed8071db0 0x7f0ed806af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/2326185388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80aff30 0x7f0ed80ce010 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/2326185388 >> 172.21.15.157:0/2326185388 conn(0x7f0ed80cd490 msgr2=0x7f0ed80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/2326185388 shutdown_connections 2023-11-06T03:18:29.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/2326185388 wait complete. 2023-11-06T03:18:29.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 Processor -- start 2023-11-06T03:18:29.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- start start 2023-11-06T03:18:29.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed8071db0 0x7f0ed815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed80aff30 0x7f0ed815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed815efd0 con 0x7f0ed80b2490 2023-11-06T03:18:29.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed81570e0 con 0x7f0ed8071db0 2023-11-06T03:18:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8156b20 con 0x7f0ed80aff30 2023-11-06T03:18:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47380/0 (socket says 172.21.15.157:47380) 2023-11-06T03:18:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 -- 172.21.15.157:0/590586496 learned_addr learned my addr 172.21.15.157:0/590586496 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:29.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed67fc640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed80aff30 0x7f0ed815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 -- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed80aff30 msgr2=0x7f0ed815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed80aff30 0x7f0ed815cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 -- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed8071db0 msgr2=0x7f0ed815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed8071db0 0x7f0ed815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 -- 172.21.15.157:0/590586496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8009420 con 0x7f0ed80b2490 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ed77fe640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c7a0 tx=0x7f0ed000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000a640 con 0x7f0ed80b2490 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed000a7a0 con 0x7f0ed80b2490 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.730+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed8156e00 con 0x7f0ed80b2490 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000ab90 con 0x7f0ed80b2490 2023-11-06T03:18:29.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed814e800 con 0x7f0ed80b2490 2023-11-06T03:18:29.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed814eae0 con 0x7f0ed80b2490 2023-11-06T03:18:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0ed000acf0 con 0x7f0ed80b2490 2023-11-06T03:18:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ebf7fe640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 0x7f0ec4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ed0082aa0 con 0x7f0ed80b2490 2023-11-06T03:18:29.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ed6ffd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 0x7f0ec4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.734+0000 7f0ed6ffd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 0x7f0ec4067f20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80ae000 tx=0x7f0ec8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:29.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.738+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0ed0050390 con 0x7f0ed80b2490 2023-11-06T03:18:29.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.890+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ed8072750 con 0x7f0ed80b2490 2023-11-06T03:18:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0ebf7fe640 1 -- 172.21.15.157:0/590586496 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v189) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ed0018080 con 0x7f0ed80b2490 2023-11-06T03:18:29.890 INFO:teuthology.orchestra.run.smithi157.stderr:listed 0 entries 2023-11-06T03:18:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 msgr2=0x7f0ec4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 0x7f0ec4067f20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80ae000 tx=0x7f0ec8008040 comp rx=0 tx=0).stop 2023-11-06T03:18:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 msgr2=0x7f0ed80b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c7a0 tx=0x7f0ed000cc60 comp rx=0 tx=0).stop 2023-11-06T03:18:29.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 shutdown_connections 2023-11-06T03:18:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0ec4065a60 0x7f0ec4067f20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80b2490 0x7f0ed80b3810 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0ed80aff30 0x7f0ed815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 --2- 172.21.15.157:0/590586496 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0ed8071db0 0x7f0ed815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:29.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 >> 172.21.15.157:0/590586496 conn(0x7f0ed80cd490 msgr2=0x7f0ed806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:29.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 shutdown_connections 2023-11-06T03:18:29.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:29.894+0000 7f0edd6dd640 1 -- 172.21.15.157:0/590586496 wait complete. 2023-11-06T03:18:29.907 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2023-11-06T03:18:29.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:29.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-06T03:18:29.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi157.stderr:ceph-fuse[98817]: fuse finished with error 0 and tester_r 0 2023-11-06T03:18:29.922 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:18:36.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:18:36.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:18:36.024 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:18:36.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:36.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.202+0000 7f7b0f097640 1 Processor -- start 2023-11-06T03:18:36.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.202+0000 7f7b0f097640 1 -- start start 2023-11-06T03:18:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0f097640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 0x7f7b0806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0f097640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b080b3b20 0x7f7b0815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0f097640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0815f580 con 0x7f7b0815e5e0 2023-11-06T03:18:36.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0f097640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0815efc0 con 0x7f7b080b3b20 2023-11-06T03:18:36.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0f097640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0815f2a0 con 0x7f7b081551e0 2023-11-06T03:18:36.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7b080b3b20 0x7f7b0815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b07fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f7b0815e5e0 0x7f7b080b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34010/0 (socket says 172.21.15.157:34010) 2023-11-06T03:18:36.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b07fff640 1 -- 172.21.15.157:0/4133732639 learned_addr learned my addr 172.21.15.157:0/4133732639 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:36.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0ce33640 1 --2- 172.21.15.157:0/4133732639 >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 0x7f7b0806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 -- 172.21.15.157:0/4133732639 >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 msgr2=0x7f7b0806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:36.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 --2- 172.21.15.157:0/4133732639 >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 0x7f7b0806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 -- 172.21.15.157:0/4133732639 >> v1:172.21.15.157:6789/0 conn(0x7f7b0815e5e0 legacy=0x7f7b080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 -- 172.21.15.157:0/4133732639 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0806f5a0 con 0x7f7b080b3b20 2023-11-06T03:18:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b077fe640 1 -- 172.21.15.157:0/4133732639 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494355988 0 0) 0x7f7b0815f580 con 0x7f7b0815e5e0 2023-11-06T03:18:36.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0ce33640 1 --2- 172.21.15.157:0/4133732639 >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 0x7f7b0806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b0d634640 1 --2- 172.21.15.157:0/4133732639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0815c1b0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f7b08066830 tx=0x7f7b0001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2501a9500c1f8a62 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b077fe640 1 -- 172.21.15.157:0/4133732639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000a58a0 con 0x7f7b080b3b20 2023-11-06T03:18:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b077fe640 1 -- 172.21.15.157:0/4133732639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000a5a00 con 0x7f7b080b3b20 2023-11-06T03:18:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.206+0000 7f7b077fe640 1 -- 172.21.15.157:0/4133732639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000a5d90 con 0x7f7b080b3b20 2023-11-06T03:18:36.212 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/4133732639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 msgr2=0x7f7b0815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- 172.21.15.157:0/4133732639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0815c1b0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f7b08066830 tx=0x7f7b0001c7c0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/4133732639 shutdown_connections 2023-11-06T03:18:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- 172.21.15.157:0/4133732639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0815c1b0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- 172.21.15.157:0/4133732639 >> v2:172.21.15.195:3300/0 conn(0x7f7b081551e0 0x7f7b0806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/4133732639 >> 172.21.15.157:0/4133732639 conn(0x7f7b08065930 msgr2=0x7f7b08065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/4133732639 shutdown_connections 2023-11-06T03:18:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/4133732639 wait complete. 2023-11-06T03:18:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 Processor -- start 2023-11-06T03:18:36.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- start start 2023-11-06T03:18:36.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 0x7f7b08154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 0x7f7b0814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080ce630 con 0x7f7b080b3b20 2023-11-06T03:18:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0806b630 con 0x7f7b081551e0 2023-11-06T03:18:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0806b910 con 0x7f7b0815e5e0 2023-11-06T03:18:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47402/0 (socket says 172.21.15.157:47402) 2023-11-06T03:18:36.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 -- 172.21.15.157:0/3612591689 learned_addr learned my addr 172.21.15.157:0/3612591689 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:36.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0d634640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 0x7f7b0814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b07fff640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 0x7f7b08154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 -- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 msgr2=0x7f7b0814e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 0x7f7b0814e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 -- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 msgr2=0x7f7b08154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 0x7f7b08154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 -- 172.21.15.157:0/3612591689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00009a70 con 0x7f7b080b3b20 2023-11-06T03:18:36.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0d634640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 0x7f7b0814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:36.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b07fff640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 0x7f7b08154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:36.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0ce33640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00c7c0 tx=0x7f7afc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc007a30 con 0x7f7b080b3b20 2023-11-06T03:18:36.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc007b90 con 0x7f7b080b3b20 2023-11-06T03:18:36.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0806bbf0 con 0x7f7b080b3b20 2023-11-06T03:18:36.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00ce70 con 0x7f7b080b3b20 2023-11-06T03:18:36.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.210+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b080cddc0 con 0x7f7b080b3b20 2023-11-06T03:18:36.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b080ce0a0 con 0x7f7b080b3b20 2023-11-06T03:18:36.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7afc0113f0 con 0x7f7b080b3b20 2023-11-06T03:18:36.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b057fa640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 0x7f7aec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7afc083810 con 0x7f7b080b3b20 2023-11-06T03:18:36.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b07fff640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 0x7f7aec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.214+0000 7f7b07fff640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 0x7f7aec067f20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.218+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7afc051070 con 0x7f7b080b3b20 2023-11-06T03:18:36.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b08072750 con 0x7f7b080b3b20 2023-11-06T03:18:36.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b057fa640 1 -- 172.21.15.157:0/3612591689 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4669 (secure 0 0 0) 0x7f7afc054690 con 0x7f7b080b3b20 2023-11-06T03:18:36.403 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:36.403 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:20.111160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11244":{"gid":11244,"name":"a","rank":0,"incarnation":128,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/877366447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":877366447},{"type":"v1","addr":"172.21.15.157:6835","nonce":877366447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:36.403 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 130 2023-11-06T03:18:36.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 msgr2=0x7f7aec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 0x7f7aec067f20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4009040 comp rx=0 tx=0).stop 2023-11-06T03:18:36.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 msgr2=0x7f7b0814d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00c7c0 tx=0x7f7afc00cc80 comp rx=0 tx=0).stop 2023-11-06T03:18:36.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 shutdown_connections 2023-11-06T03:18:36.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7aec065a60 0x7f7aec067f20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7b0815e5e0 0x7f7b0814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7b081551e0 0x7f7b08154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 --2- 172.21.15.157:0/3612591689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b080b3b20 0x7f7b0814d5b0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 >> 172.21.15.157:0/3612591689 conn(0x7f7b08065930 msgr2=0x7f7b080b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 shutdown_connections 2023-11-06T03:18:36.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.406+0000 7f7b0f097640 1 -- 172.21.15.157:0/3612591689 wait complete. 2023-11-06T03:18:36.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-06T03:18:36.577 INFO:teuthology.orchestra.run.smithi157.stdout:{"message":"","return_code":0} 2023-11-06T03:18:36.578 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-06T03:18:36.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:18:36.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 Processor -- start 2023-11-06T03:18:36.696 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 -- start start 2023-11-06T03:18:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa41c06d000 0x7fa41c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c15e440 con 0x7fa41c0b2af0 2023-11-06T03:18:36.697 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c06c240 con 0x7fa41c1599c0 2023-11-06T03:18:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa422849640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41c06c520 con 0x7fa41c06d000 2023-11-06T03:18:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa4137fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fa41c0b2af0 0x7fa41c159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34014/0 (socket says 172.21.15.157:34014) 2023-11-06T03:18:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa4137fe640 1 -- 172.21.15.157:0/3163698503 learned_addr learned my addr 172.21.15.157:0/3163698503 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:36.698 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa420de6640 1 --2- 172.21.15.157:0/3163698503 >> v2:172.21.15.157:3300/0 conn(0x7fa41c06d000 0x7fa41c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa41b7fe640 1 -- 172.21.15.157:0/3163698503 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981373000 0 0) 0x7fa41c15e440 con 0x7fa41c0b2af0 2023-11-06T03:18:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa41b7fe640 1 -- 172.21.15.157:0/3163698503 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa408003530 con 0x7fa41c0b2af0 2023-11-06T03:18:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa420de6640 1 -- 172.21.15.157:0/3163698503 >> v1:172.21.15.195:6789/0 conn(0x7fa41c1599c0 legacy=0x7fa41c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa420de6640 1 -- 172.21.15.157:0/3163698503 >> v1:172.21.15.157:6789/0 conn(0x7fa41c0b2af0 legacy=0x7fa41c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.698+0000 7fa420de6640 1 -- 172.21.15.157:0/3163698503 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c06c800 con 0x7fa41c06d000 2023-11-06T03:18:36.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa420de6640 1 --2- 172.21.15.157:0/3163698503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c0ae000 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0ce95f6128a9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa41b7fe640 1 -- 172.21.15.157:0/3163698503 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a59c0 con 0x7fa41c06d000 2023-11-06T03:18:36.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa41b7fe640 1 -- 172.21.15.157:0/3163698503 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140a5b20 con 0x7fa41c06d000 2023-11-06T03:18:36.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa41b7fe640 1 -- 172.21.15.157:0/3163698503 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a5e90 con 0x7fa41c06d000 2023-11-06T03:18:36.701 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 -- 172.21.15.157:0/3163698503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 msgr2=0x7fa41c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 --2- 172.21.15.157:0/3163698503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c0ae000 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41401c8e0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 -- 172.21.15.157:0/3163698503 shutdown_connections 2023-11-06T03:18:36.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 --2- 172.21.15.157:0/3163698503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c0ae000 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 -- 172.21.15.157:0/3163698503 >> 172.21.15.157:0/3163698503 conn(0x7fa41c065930 msgr2=0x7fa41c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 -- 172.21.15.157:0/3163698503 shutdown_connections 2023-11-06T03:18:36.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.702+0000 7fa422849640 1 -- 172.21.15.157:0/3163698503 wait complete. 2023-11-06T03:18:36.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 Processor -- start 2023-11-06T03:18:36.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- start start 2023-11-06T03:18:36.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41c0b2af0 0x7fa41c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa41c1599c0 0x7fa41c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0cede0 con 0x7fa41c06d000 2023-11-06T03:18:36.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0cbc20 con 0x7fa41c1599c0 2023-11-06T03:18:36.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0cbf00 con 0x7fa41c0b2af0 2023-11-06T03:18:36.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47416/0 (socket says 172.21.15.157:47416) 2023-11-06T03:18:36.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 -- 172.21.15.157:0/3002229479 learned_addr learned my addr 172.21.15.157:0/3002229479 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:36.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa4137fe640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41c0b2af0 0x7fa41c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa420de6640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa41c1599c0 0x7fa41c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 -- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41c0b2af0 msgr2=0x7fa41c0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41c0b2af0 0x7fa41c0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 -- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa41c1599c0 msgr2=0x7fa41c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa41c1599c0 0x7fa41c0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 -- 172.21.15.157:0/3002229479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa414009a70 con 0x7fa41c06d000 2023-11-06T03:18:36.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa41bfff640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b410 tx=0x7fa40c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c004020 con 0x7fa41c06d000 2023-11-06T03:18:36.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c004180 con 0x7fa41c06d000 2023-11-06T03:18:36.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c011630 con 0x7fa41c06d000 2023-11-06T03:18:36.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c0cc3b0 con 0x7fa41c06d000 2023-11-06T03:18:36.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c0cca80 con 0x7fa41c06d000 2023-11-06T03:18:36.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c072750 con 0x7fa41c06d000 2023-11-06T03:18:36.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.706+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa40c011790 con 0x7fa41c06d000 2023-11-06T03:18:36.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.710+0000 7fa4197fa640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 0x7fa3fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.710+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa40c083060 con 0x7fa41c06d000 2023-11-06T03:18:36.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.710+0000 7fa4137fe640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 0x7fa3fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.710+0000 7fa4137fe640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 0x7fa3fc067f20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa41c15b3d0 tx=0x7fa404009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.714+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa40c0508c0 con 0x7fa41c06d000 2023-11-06T03:18:36.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.898+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa41c165640 con 0x7fa41c06d000 2023-11-06T03:18:36.895 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:18:36.898+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:18:36.913 INFO:tasks.ceph.mds.a.smithi157.stderr: -4365> 2023-11-06T03:18:23.682+0000 7f4c294a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:18:36.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.914+0000 7fa4197fa640 1 -- 172.21.15.157:0/3002229479 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v131) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa40c053ee0 con 0x7fa41c06d000 2023-11-06T03:18:36.915 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:18:36.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 msgr2=0x7fa3fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 0x7fa3fc067f20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa41c15b3d0 tx=0x7fa404009040 comp rx=0 tx=0).stop 2023-11-06T03:18:36.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 msgr2=0x7fa41c14e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b410 tx=0x7fa40c00b8d0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 shutdown_connections 2023-11-06T03:18:36.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa3fc065a60 0x7fa3fc067f20 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa41c1599c0 0x7fa41c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa41c0b2af0 0x7fa41c0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 --2- 172.21.15.157:0/3002229479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa41c06d000 0x7fa41c14e1f0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 >> 172.21.15.157:0/3002229479 conn(0x7fa41c065930 msgr2=0x7fa41c065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 shutdown_connections 2023-11-06T03:18:36.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.918+0000 7fa422849640 1 -- 172.21.15.157:0/3002229479 wait complete. 2023-11-06T03:18:36.932 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2023-11-06T03:18:36.942 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:18:36.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 Processor -- start 2023-11-06T03:18:36.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- start start 2023-11-06T03:18:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370eddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370edef90 0x559370ee26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559370d4f450 con 0x559370ee2c20 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559370d4f150 con 0x7f09140030c0 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559370d45dc0 con 0x559370edef90 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47448/0 (socket says 172.21.15.157:47448) 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 -- 172.21.15.157:0/73509873 learned_addr learned my addr 172.21.15.157:0/73509873 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:36.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 -- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370edef90 msgr2=0x559370ee26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370edef90 0x559370ee26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 -- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 msgr2=0x559370eddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370eddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 -- 172.21.15.157:0/73509873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559370d5dff0 con 0x559370ee2c20 2023-11-06T03:18:36.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092b154640 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x559370e07920 tx=0x7f092400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc82c26e076701a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f0929150640 1 -- 172.21.15.157:0/73509873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924017070 con 0x559370ee2c20 2023-11-06T03:18:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f0929150640 1 -- 172.21.15.157:0/73509873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924012410 con 0x559370ee2c20 2023-11-06T03:18:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f0929150640 1 -- 172.21.15.157:0/73509873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400b070 con 0x559370ee2c20 2023-11-06T03:18:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- 172.21.15.157:0/73509873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 msgr2=0x559370edf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x559370e07920 tx=0x7f092400d8d0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- 172.21.15.157:0/73509873 shutdown_connections 2023-11-06T03:18:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370ee2c20 0x559370edf690 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370edef90 0x559370ee26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- 172.21.15.157:0/73509873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370eddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- 172.21.15.157:0/73509873 >> 172.21.15.157:0/73509873 conn(0x559370e33030 msgr2=0x559370edc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- 172.21.15.157:0/73509873 shutdown_connections 2023-11-06T03:18:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- 172.21.15.157:0/73509873 wait complete. 2023-11-06T03:18:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 Processor -- start 2023-11-06T03:18:36.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- start start 2023-11-06T03:18:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 0x559370e4f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370e4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370e49460 0x559370e4dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559370e49900 con 0x559370edef90 2023-11-06T03:18:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x559370e49ba0 con 0x7f09140030c0 2023-11-06T03:18:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.970+0000 7f092ce39080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x559370e49e40 con 0x559370e49460 2023-11-06T03:18:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 0x559370e4f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 0x559370e4f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47462/0 (socket says 172.21.15.157:47462) 2023-11-06T03:18:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 -- 172.21.15.157:0/3113447111 learned_addr learned my addr 172.21.15.157:0/3113447111 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092b154640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370e49460 0x559370e4dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a152640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370e4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 -- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370e49460 msgr2=0x559370e4dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370e49460 0x559370e4dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 -- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 msgr2=0x559370e4d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09140030c0 0x559370e4d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 -- 172.21.15.157:0/3113447111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092400d170 con 0x559370edef90 2023-11-06T03:18:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092b154640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x559370e49460 0x559370e4dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:36.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a953640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 0x559370e4f040 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f092000dc90 tx=0x7f092000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 -- 172.21.15.157:0/3113447111 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920017620 con 0x559370edef90 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 -- 172.21.15.157:0/3113447111 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920017780 con 0x559370edef90 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559370d51430 con 0x559370edef90 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 -- 172.21.15.157:0/3113447111 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920017b10 con 0x559370edef90 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559370d50e90 con 0x559370edef90 2023-11-06T03:18:36.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 -- 172.21.15.157:0/3113447111 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0920014020 con 0x559370edef90 2023-11-06T03:18:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0918064470 0x7f0918066930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f0913fff640 1 -- 172.21.15.157:0/3113447111 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0920082340 con 0x559370edef90 2023-11-06T03:18:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092ce39080 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x559370e4f580 0x559370f7a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- osd_op(unknown.0.0:1 27.10 27:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e190) v8 -- 0x559370edc540 con 0x559370e4f580 2023-11-06T03:18:36.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a152640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0918064470 0x7f0918066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092b154640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x559370e4f580 0x559370f7a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092b154640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x559370e4f580 0x559370f7a660 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.974+0000 7f092a152640 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0918064470 0x7f0918066930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f091c0ae000 tx=0x7f091c0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092b154640 1 -- 172.21.15.157:0/3113447111 <== osd.2 v2:172.21.15.157:6816/38222 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=527b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+527 (crc 0 0 0) 0x7f0924019070 con 0x559370e4f580 2023-11-06T03:18:36.983 INFO:teuthology.orchestra.run.smithi157.stderr:Writing to - 2023-11-06T03:18:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x559370e4f580 msgr2=0x559370f7a660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x559370e4f580 0x559370f7a660 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0918064470 msgr2=0x7f0918066930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0918064470 0x7f0918066930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f091c0ae000 tx=0x7f091c0b3040 comp rx=0 tx=0).stop 2023-11-06T03:18:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 msgr2=0x559370e4f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 --2- 172.21.15.157:0/3113447111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559370edef90 0x559370e4f040 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f092000dc90 tx=0x7f092000b6d0 comp rx=0 tx=0).stop 2023-11-06T03:18:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092b154640 1 -- 172.21.15.157:0/3113447111 reap_dead start 2023-11-06T03:18:36.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 shutdown_connections 2023-11-06T03:18:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 >> 172.21.15.157:0/3113447111 conn(0x559370e33030 msgr2=0x559370ee14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 shutdown_connections 2023-11-06T03:18:36.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:36.978+0000 7f092ce39080 1 -- 172.21.15.157:0/3113447111 wait complete. 2023-11-06T03:18:36.987 INFO:teuthology.orchestra.run.smithi157.stdout:i���[Heh/����@@`��������[Heh/[He9�V,(8`[Heh/8`[Heh/��������[He9�V,0������������ 2023-11-06T03:18:36.988 DEBUG:teuthology.orchestra.run.smithi157:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2023-11-06T03:18:37.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.062+0000 7fedbc0b3080 1 Processor -- start 2023-11-06T03:18:37.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- start start 2023-11-06T03:18:37.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4acd70 0x5560ac4a9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac4a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ac3f1450 con 0x5560ac4ac990 2023-11-06T03:18:37.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ac3f1150 con 0x5560ac4acd70 2023-11-06T03:18:37.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560ac3e7dc0 con 0x5560ac582900 2023-11-06T03:18:37.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47472/0 (socket says 172.21.15.157:47472) 2023-11-06T03:18:37.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 -- 172.21.15.157:0/2593155544 learned_addr learned my addr 172.21.15.157:0/2593155544 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:37.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedba3ce640 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac4a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 -- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 msgr2=0x5560ac4a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac4a9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 -- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4acd70 msgr2=0x5560ac4a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4acd70 0x5560ac4a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 -- 172.21.15.157:0/2593155544 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac005d30 con 0x5560ac4ac990 2023-11-06T03:18:37.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001d70 tx=0x7fedb000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7575eb594b86951a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedabfff640 1 -- 172.21.15.157:0/2593155544 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000be40 con 0x5560ac4ac990 2023-11-06T03:18:37.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedabfff640 1 -- 172.21.15.157:0/2593155544 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb000e690 con 0x5560ac4ac990 2023-11-06T03:18:37.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedabfff640 1 -- 172.21.15.157:0/2593155544 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000ea20 con 0x5560ac4ac990 2023-11-06T03:18:37.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2593155544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 msgr2=0x5560ac5804e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.068 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001d70 tx=0x7fedb000b6d0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2593155544 shutdown_connections 2023-11-06T03:18:37.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac4a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4acd70 0x5560ac4a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2593155544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4ac990 0x5560ac5804e0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.069 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2593155544 >> 172.21.15.157:0/2593155544 conn(0x5560ac4d7060 msgr2=0x5560ac55d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2593155544 shutdown_connections 2023-11-06T03:18:37.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2593155544 wait complete. 2023-11-06T03:18:37.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 Processor -- start 2023-11-06T03:18:37.070 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- start start 2023-11-06T03:18:37.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4ac990 0x5560ac4ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 0x5560ac4f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac57bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ac611e90 con 0x5560ac4acd70 2023-11-06T03:18:37.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560ac612130 con 0x5560ac4ac990 2023-11-06T03:18:37.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedbc0b3080 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560ac6123d0 con 0x5560ac582900 2023-11-06T03:18:37.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 0x5560ac4f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 0x5560ac4f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47478/0 (socket says 172.21.15.157:47478) 2023-11-06T03:18:37.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 -- 172.21.15.157:0/2773252552 learned_addr learned my addr 172.21.15.157:0/2773252552 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:37.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedba3ce640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac57bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4ac990 0x5560ac4ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 -- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 msgr2=0x5560ac57bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5560ac582900 0x5560ac57bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 -- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4ac990 msgr2=0x5560ac4ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5560ac4ac990 0x5560ac4ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.066+0000 7fedb93cc640 1 -- 172.21.15.157:0/2773252552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560ac3f30a0 con 0x5560ac4acd70 2023-11-06T03:18:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedb93cc640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 0x5560ac4f36f0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fedb40ae040 tx=0x7fedb40ab5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 -- 172.21.15.157:0/2773252552 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40abb30 con 0x5560ac4acd70 2023-11-06T03:18:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 -- 172.21.15.157:0/2773252552 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb40abc90 con 0x5560ac4acd70 2023-11-06T03:18:37.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb400a100 con 0x5560ac4acd70 2023-11-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 -- 172.21.15.157:0/2773252552 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40b4410 con 0x5560ac4acd70 2023-11-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560ac3ffff0 con 0x5560ac4acd70 2023-11-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 -- 172.21.15.157:0/2773252552 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fedb40b2040 con 0x5560ac4acd70 2023-11-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fed9c064440 0x7fed9c066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedaaffd640 1 -- 172.21.15.157:0/2773252552 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fedb411c550 con 0x5560ac4acd70 2023-11-06T03:18:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x5560ac611680 0x5560ac619fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- osd_op(unknown.0.0:1 27.10 27:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e190) v8 -- 0x5560ac61a510 con 0x5560ac611680 2023-11-06T03:18:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fed9c064440 0x7fed9c066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedba3ce640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x5560ac611680 0x5560ac619fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedba3ce640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x5560ac611680 0x5560ac619fd0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.070+0000 7fedb9bcd640 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fed9c064440 0x7fed9c066900 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001d70 tx=0x7fedb0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedba3ce640 1 -- 172.21.15.157:0/2773252552 <== osd.2 v2:172.21.15.157:6816/38222 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v190'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fedac009930 con 0x5560ac611680 2023-11-06T03:18:37.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x5560ac611680 msgr2=0x5560ac619fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x5560ac611680 0x5560ac619fd0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fed9c064440 msgr2=0x7fed9c066900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fed9c064440 0x7fed9c066900 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001d70 tx=0x7fedb0009000 comp rx=0 tx=0).stop 2023-11-06T03:18:37.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 msgr2=0x5560ac4f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 --2- 172.21.15.157:0/2773252552 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5560ac4acd70 0x5560ac4f36f0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fedb40ae040 tx=0x7fedb40ab5b0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedba3ce640 1 -- 172.21.15.157:0/2773252552 reap_dead start 2023-11-06T03:18:37.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 shutdown_connections 2023-11-06T03:18:37.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 >> 172.21.15.157:0/2773252552 conn(0x5560ac4d7060 msgr2=0x5560ac5807b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 shutdown_connections 2023-11-06T03:18:37.083 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.074+0000 7fedbc0b3080 1 -- 172.21.15.157:0/2773252552 wait complete. 2023-11-06T03:18:37.083 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2023-11-06T03:18:37.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 Processor -- start 2023-11-06T03:18:37.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 -- start start 2023-11-06T03:18:37.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb8a80b3ac0 0x7fb8a815a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a81543c0 con 0x7fb8a8159e90 2023-11-06T03:18:37.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a815a9a0 con 0x7fb8a80ce850 2023-11-06T03:18:37.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8ae4c1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a80b29a0 con 0x7fb8a80b3ac0 2023-11-06T03:18:37.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a77fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb8a8159e90 0x7fb8a80b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34032/0 (socket says 172.21.15.157:34032) 2023-11-06T03:18:37.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a77fe640 1 -- 172.21.15.157:0/2714416723 learned_addr learned my addr 172.21.15.157:0/2714416723 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:37.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2714416723 >> v2:172.21.15.195:3300/0 conn(0x7fb8a80b3ac0 0x7fb8a815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189803734 0 0) 0x7fb8a81543c0 con 0x7fb8a8159e90 2023-11-06T03:18:37.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb890005740 con 0x7fb8a8159e90 2023-11-06T03:18:37.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3102766967 0 0) 0x7fb890005740 con 0x7fb8a8159e90 2023-11-06T03:18:37.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 >> v2:172.21.15.195:3300/0 conn(0x7fb8a80b3ac0 msgr2=0x7fb8a815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 --2- 172.21.15.157:0/2714416723 >> v2:172.21.15.195:3300/0 conn(0x7fb8a80b3ac0 0x7fb8a815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 >> v1:172.21.15.195:6789/0 conn(0x7fb8a80ce850 legacy=0x7fb8a806ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a815ac80 con 0x7fb8a8159e90 2023-11-06T03:18:37.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb89c001680 con 0x7fb8a8159e90 2023-11-06T03:18:37.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 >> v1:172.21.15.157:6789/0 conn(0x7fb8a8159e90 legacy=0x7fb8a80b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb890005b90 0x7fb890005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900069e0 0x7fb890006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 0x7fb89000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80b29a0 con 0x7fb8900073a0 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000e240 con 0x7fb890005b90 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000e400 con 0x7fb8900069e0 2023-11-06T03:18:37.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 0x7fb89000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a7fff640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb890005b90 0x7fb890005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900069e0 msgr2=0x7fb890006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8a77fe640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900069e0 0x7fb890006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900069e0 0x7fb890006e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb890005b90 msgr2=0x7fb890005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb890005b90 0x7fb890005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.246+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2714416723 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890005740 con 0x7fb8900073a0 2023-11-06T03:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2714416723 reap_dead start 2023-11-06T03:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 0x7fb89000dba0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009fc0 tx=0x7fb8a001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd25e0fc2f175fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00a5cb0 con 0x7fb8900073a0 2023-11-06T03:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000f040 con 0x7fb8900073a0 2023-11-06T03:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8a6ffd640 1 -- 172.21.15.157:0/2714416723 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0003be0 con 0x7fb8900073a0 2023-11-06T03:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 msgr2=0x7fb89000dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 0x7fb89000dba0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009fc0 tx=0x7fb8a001cca0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2714416723 shutdown_connections 2023-11-06T03:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2714416723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8900073a0 0x7fb89000dba0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2714416723 >> 172.21.15.157:0/2714416723 conn(0x7fb8a8069d70 msgr2=0x7fb8a806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2714416723 shutdown_connections 2023-11-06T03:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2714416723 wait complete. 2023-11-06T03:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 Processor -- start 2023-11-06T03:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.250+0000 7fb8ae4c1640 1 -- start start 2023-11-06T03:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900073a0 0x7fb8a80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 0x7fb8a8140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8150430 con 0x7fb8a8140b80 2023-11-06T03:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8141000 con 0x7fb8a81f1d90 2023-11-06T03:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81f1900 con 0x7fb8900073a0 2023-11-06T03:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47500/0 (socket says 172.21.15.157:47500) 2023-11-06T03:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2239375804 learned_addr learned my addr 172.21.15.157:0/2239375804 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a7fff640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900073a0 0x7fb8a80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a77fe640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 0x7fb8a8140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900073a0 msgr2=0x7fb8a80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900073a0 0x7fb8a80cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 msgr2=0x7fb8a8140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 0x7fb8a8140640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 -- 172.21.15.157:0/2239375804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a81eeb30 con 0x7fb8a8140b80 2023-11-06T03:18:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a77fe640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 0x7fb8a8140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8aca5e640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00987d0 tx=0x7fb8a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00aecb0 con 0x7fb8a8140b80 2023-11-06T03:18:37.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a00aee30 con 0x7fb8a8140b80 2023-11-06T03:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a81ee850 con 0x7fb8a8140b80 2023-11-06T03:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0003be0 con 0x7fb8a8140b80 2023-11-06T03:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a81f1590 con 0x7fb8a8140b80 2023-11-06T03:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a81f1a90 con 0x7fb8a8140b80 2023-11-06T03:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb8a0003d40 con 0x7fb8a8140b80 2023-11-06T03:18:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 0x7fb894067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb8a00a6e40 con 0x7fb8a8140b80 2023-11-06T03:18:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.254+0000 7fb8a7fff640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 0x7fb894067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.258+0000 7fb8a7fff640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 0x7fb894067f20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb898000c10 tx=0x7fb898009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.258+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fb8a00ec650 con 0x7fb8a8140b80 2023-11-06T03:18:37.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.410+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7fb8a8072750 con 0x7fb8a8140b80 2023-11-06T03:18:37.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.422+0000 7fb8a4ff9640 1 -- 172.21.15.157:0/2239375804 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v33)=0 v33) v1 ==== 140+0+0 (secure 0 0 0) 0x7fb8a00efc70 con 0x7fb8a8140b80 2023-11-06T03:18:37.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 msgr2=0x7fb894067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 0x7fb894067f20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb898000c10 tx=0x7fb898009040 comp rx=0 tx=0).stop 2023-11-06T03:18:37.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 msgr2=0x7fb8a81edba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00987d0 tx=0x7fb8a0009750 comp rx=0 tx=0).stop 2023-11-06T03:18:37.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 shutdown_connections 2023-11-06T03:18:37.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb894065a60 0x7fb894067f20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8a8140b80 0x7fb8a81edba0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8a81f1d90 0x7fb8a8140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 --2- 172.21.15.157:0/2239375804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8900073a0 0x7fb8a80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 >> 172.21.15.157:0/2239375804 conn(0x7fb8a8069d70 msgr2=0x7fb8a8150720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 shutdown_connections 2023-11-06T03:18:37.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.426+0000 7fb8ae4c1640 1 -- 172.21.15.157:0/2239375804 wait complete. 2023-11-06T03:18:37.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-06T03:18:37.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 Processor -- start 2023-11-06T03:18:37.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 -- start start 2023-11-06T03:18:37.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa73015ac80 0x7fa7300b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7300b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300cc2b0 con 0x7fa7300cc720 2023-11-06T03:18:37.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300b01a0 con 0x7fa73015ac80 2023-11-06T03:18:37.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa7372ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300cfb10 con 0x7fa7300afb00 2023-11-06T03:18:37.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa73015ac80 0x7fa7300b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa73015ac80 0x7fa7300b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47504/0 (socket says 172.21.15.157:47504) 2023-11-06T03:18:37.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 -- 172.21.15.157:0/1219766288 learned_addr learned my addr 172.21.15.157:0/1219766288 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:37.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73584b640 1 --1- 172.21.15.157:0/1219766288 >> v1:172.21.15.157:6789/0 conn(0x7fa7300cc720 0x7fa7300ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34040/0 (socket says 172.21.15.157:34040) 2023-11-06T03:18:37.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 -- 172.21.15.157:0/1219766288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 msgr2=0x7fa7300b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:37.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa734849640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7300b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7300b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 -- 172.21.15.157:0/1219766288 >> v1:172.21.15.157:6789/0 conn(0x7fa7300cc720 legacy=0x7fa7300ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa727fff640 1 -- 172.21.15.157:0/1219766288 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923147172 0 0) 0x7fa7300cc2b0 con 0x7fa7300cc720 2023-11-06T03:18:37.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.566+0000 7fa73504a640 1 -- 172.21.15.157:0/1219766288 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300cfdf0 con 0x7fa73015ac80 2023-11-06T03:18:37.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa73504a640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73015ac80 0x7fa7300b2400 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009870 tx=0x7fa72c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263dd5dab24f4b2f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa727fff640 1 -- 172.21.15.157:0/1219766288 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0ab650 con 0x7fa73015ac80 2023-11-06T03:18:37.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa727fff640 1 -- 172.21.15.157:0/1219766288 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0ab7b0 con 0x7fa73015ac80 2023-11-06T03:18:37.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa727fff640 1 -- 172.21.15.157:0/1219766288 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0abb40 con 0x7fa73015ac80 2023-11-06T03:18:37.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/1219766288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73015ac80 msgr2=0x7fa7300b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73015ac80 0x7fa7300b2400 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009870 tx=0x7fa72c01c540 comp rx=0 tx=0).stop 2023-11-06T03:18:37.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/1219766288 shutdown_connections 2023-11-06T03:18:37.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7300b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- 172.21.15.157:0/1219766288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa73015ac80 0x7fa7300b2400 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/1219766288 >> 172.21.15.157:0/1219766288 conn(0x7fa73006d560 msgr2=0x7fa73006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/1219766288 shutdown_connections 2023-11-06T03:18:37.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/1219766288 wait complete. 2023-11-06T03:18:37.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 Processor -- start 2023-11-06T03:18:37.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- start start 2023-11-06T03:18:37.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7301595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa73015ac80 0x7fa730147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730159120 con 0x7fa7300cc720 2023-11-06T03:18:37.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730158510 con 0x7fa73015ac80 2023-11-06T03:18:37.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301587f0 con 0x7fa7300afb00 2023-11-06T03:18:37.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47516/0 (socket says 172.21.15.157:47516) 2023-11-06T03:18:37.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 -- 172.21.15.157:0/3153376115 learned_addr learned my addr 172.21.15.157:0/3153376115 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:37.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa73504a640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7301595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 -- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 msgr2=0x7fa7301595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7301595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 -- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa73015ac80 msgr2=0x7fa730147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa73015ac80 0x7fa730147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 -- 172.21.15.157:0/3153376115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0094d0 con 0x7fa7300cc720 2023-11-06T03:18:37.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa734849640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c7f0 tx=0x7fa72000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720007820 con 0x7fa7300cc720 2023-11-06T03:18:37.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720007980 con 0x7fa7300cc720 2023-11-06T03:18:37.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa730158ad0 con 0x7fa7300cc720 2023-11-06T03:18:37.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720007d30 con 0x7fa7300cc720 2023-11-06T03:18:37.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.570+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7301484e0 con 0x7fa7300cc720 2023-11-06T03:18:37.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa730072750 con 0x7fa7300cc720 2023-11-06T03:18:37.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7200113f0 con 0x7fa7300cc720 2023-11-06T03:18:37.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa725ffb640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 0x7fa714067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:37.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa720083610 con 0x7fa7300cc720 2023-11-06T03:18:37.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa73504a640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 0x7fa714067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:37.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.574+0000 7fa73504a640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 0x7fa714067f20 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0af000 tx=0x7fa72c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:37.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.578+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fa720050e10 con 0x7fa7300cc720 2023-11-06T03:18:37.763 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.766+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa7300d13d0 con 0x7fa7300cc720 2023-11-06T03:18:37.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.926+0000 7fa725ffb640 1 -- 172.21.15.157:0/3153376115 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v132) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa720054430 con 0x7fa7300cc720 2023-11-06T03:18:37.924 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-06T03:18:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 msgr2=0x7fa714067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 0x7fa714067f20 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0af000 tx=0x7fa72c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:18:37.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 msgr2=0x7fa7300ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:37.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c7f0 tx=0x7fa72000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 shutdown_connections 2023-11-06T03:18:37.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fa714065a60 0x7fa714067f20 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa73015ac80 0x7fa730147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7300cc720 0x7fa7300ca500 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 --2- 172.21.15.157:0/3153376115 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7300afb00 0x7fa7301595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:37.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 >> 172.21.15.157:0/3153376115 conn(0x7fa73006d560 msgr2=0x7fa7300c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:37.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 shutdown_connections 2023-11-06T03:18:37.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:37.930+0000 7fa7372ae640 1 -- 172.21.15.157:0/3153376115 wait complete. 2023-11-06T03:18:37.948 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:38.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 Processor -- start 2023-11-06T03:18:38.059 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 -- start start 2023-11-06T03:18:38.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2b41599c0 0x7ff2b40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff2b406d000 0x7ff2b40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b415e440 con 0x7ff2b415dea0 2023-11-06T03:18:38.061 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b415adb0 con 0x7ff2b41599c0 2023-11-06T03:18:38.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b8f19640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b406c240 con 0x7ff2b406d000 2023-11-06T03:18:38.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2b41599c0 0x7ff2b40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:38.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff2b41599c0 0x7ff2b40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47518/0 (socket says 172.21.15.157:47518) 2023-11-06T03:18:38.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 -- 172.21.15.157:0/1733207453 learned_addr learned my addr 172.21.15.157:0/1733207453 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:38.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b2d9d640 1 --1- 172.21.15.157:0/1733207453 >> v1:172.21.15.157:6789/0 conn(0x7ff2b415dea0 0x7ff2b4070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34044/0 (socket says 172.21.15.157:34044) 2023-11-06T03:18:38.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 -- 172.21.15.157:0/1733207453 >> v2:172.21.15.195:3300/0 conn(0x7ff2b406d000 msgr2=0x7ff2b40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:38.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 --2- 172.21.15.157:0/1733207453 >> v2:172.21.15.195:3300/0 conn(0x7ff2b406d000 0x7ff2b40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 -- 172.21.15.157:0/1733207453 >> v1:172.21.15.157:6789/0 conn(0x7ff2b415dea0 legacy=0x7ff2b4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.063 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.062+0000 7ff2b259c640 1 -- 172.21.15.157:0/1733207453 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b406c520 con 0x7ff2b41599c0 2023-11-06T03:18:38.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b259c640 1 --2- 172.21.15.157:0/1733207453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b41599c0 0x7ff2b40ae000 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40cfa20 tx=0x7ff2a401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff58997e262f05f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:38.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b159a640 1 -- 172.21.15.157:0/1733207453 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40a7b80 con 0x7ff2b41599c0 2023-11-06T03:18:38.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b159a640 1 -- 172.21.15.157:0/1733207453 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a40a7ce0 con 0x7ff2b41599c0 2023-11-06T03:18:38.065 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b159a640 1 -- 172.21.15.157:0/1733207453 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40af660 con 0x7ff2b41599c0 2023-11-06T03:18:38.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- 172.21.15.157:0/1733207453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b41599c0 msgr2=0x7ff2b40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/1733207453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b41599c0 0x7ff2b40ae000 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40cfa20 tx=0x7ff2a401eba0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- 172.21.15.157:0/1733207453 shutdown_connections 2023-11-06T03:18:38.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/1733207453 >> v2:172.21.15.195:3300/0 conn(0x7ff2b406d000 0x7ff2b40b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.066 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/1733207453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b41599c0 0x7ff2b40ae000 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- 172.21.15.157:0/1733207453 >> 172.21.15.157:0/1733207453 conn(0x7ff2b4065930 msgr2=0x7ff2b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:38.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- 172.21.15.157:0/1733207453 shutdown_connections 2023-11-06T03:18:38.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- 172.21.15.157:0/1733207453 wait complete. 2023-11-06T03:18:38.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 Processor -- start 2023-11-06T03:18:38.067 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 -- start start 2023-11-06T03:18:38.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 0x7ff2b4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 0x7ff2b414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.066+0000 7ff2b8f19640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.071 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4152880 con 0x7ff2b415dea0 2023-11-06T03:18:38.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40cbf40 con 0x7ff2b406d000 2023-11-06T03:18:38.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40cc220 con 0x7ff2b41599c0 2023-11-06T03:18:38.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:38.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47528/0 (socket says 172.21.15.157:47528) 2023-11-06T03:18:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 -- 172.21.15.157:0/2575461229 learned_addr learned my addr 172.21.15.157:0/2575461229 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b1d9b640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 0x7ff2b414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b259c640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 0x7ff2b4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 -- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 msgr2=0x7ff2b414f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.073 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 0x7ff2b414f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 -- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 msgr2=0x7ff2b4155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 0x7ff2b4155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 -- 172.21.15.157:0/2575461229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a400b770 con 0x7ff2b415dea0 2023-11-06T03:18:38.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b259c640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 0x7ff2b4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:38.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b1d9b640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 0x7ff2b414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:38.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b2d9d640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00c700 tx=0x7ff2ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:38.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00ce10 con 0x7ff2b415dea0 2023-11-06T03:18:38.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac00a640 con 0x7ff2b415dea0 2023-11-06T03:18:38.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b40cc500 con 0x7ff2b415dea0 2023-11-06T03:18:38.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00a9b0 con 0x7ff2b415dea0 2023-11-06T03:18:38.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b40cc7e0 con 0x7ff2b415dea0 2023-11-06T03:18:38.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4152a50 con 0x7ff2b415dea0 2023-11-06T03:18:38.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff2ac00ab10 con 0x7ff2b415dea0 2023-11-06T03:18:38.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 0x7ff294067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:38.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff2ac082940 con 0x7ff2b415dea0 2023-11-06T03:18:38.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b259c640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 0x7ff294067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:38.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.070+0000 7ff2b259c640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 0x7ff294067f50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40b1000 tx=0x7ff2a40a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:38.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.074+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff2b4152a50 con 0x7ff2b415dea0 2023-11-06T03:18:38.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.246+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b4072750 con 0x7ff2b415dea0 2023-11-06T03:18:38.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2a2ffd640 1 -- 172.21.15.157:0/2575461229 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4674 (secure 0 0 0) 0x7ff2ac017090 con 0x7ff2b415dea0 2023-11-06T03:18:38.247 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:38.247 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":133,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:37.931706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:38.247 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 133 2023-11-06T03:18:38.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 msgr2=0x7ff294067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 0x7ff294067f50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40b1000 tx=0x7ff2a40a9070 comp rx=0 tx=0).stop 2023-11-06T03:18:38.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 msgr2=0x7ff2b40acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:38.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00c700 tx=0x7ff2ac00cbc0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 shutdown_connections 2023-11-06T03:18:38.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff294065a90 0x7ff294067f50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2b415dea0 0x7ff2b40acb40 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2b41599c0 0x7ff2b414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 --2- 172.21.15.157:0/2575461229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2b406d000 0x7ff2b4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:38.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 >> 172.21.15.157:0/2575461229 conn(0x7ff2b4065930 msgr2=0x7ff2b40b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:38.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 shutdown_connections 2023-11-06T03:18:38.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:38.250+0000 7ff2b8f19640 1 -- 172.21.15.157:0/2575461229 wait complete. 2023-11-06T03:18:38.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 133, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:18:18.097057+0000', 'modified': '2023-11-06T03:18:37.931706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 190, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11487': {'gid': 11487, 'name': 'a', 'rank': 0, 'incarnation': 133, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/2374994036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2374994036}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2374994036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:18:38.667 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11487:up:replay 2023-11-06T03:18:39.667 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:39.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.802+0000 7f00069e0640 1 Processor -- start 2023-11-06T03:18:39.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.802+0000 7f00069e0640 1 -- start start 2023-11-06T03:18:39.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f00069e0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7f00000b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f00069e0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f000015ea60 0x7f00000b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f00069e0640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000015f580 con 0x7f00001551e0 2023-11-06T03:18:39.804 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f00069e0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00000704b0 con 0x7f000015ea60 2023-11-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f00069e0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000015efc0 con 0x7f000015e5e0 2023-11-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffffff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f00001551e0 0x7f000006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32890/0 (socket says 172.21.15.157:32890) 2023-11-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffffff640 1 -- 172.21.15.157:0/3678389088 learned_addr learned my addr 172.21.15.157:0/3678389088 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3678389088 >> v2:172.21.15.195:3300/0 conn(0x7f000015ea60 0x7f00000b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7effff7fe640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7f00000b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848829067 0 0) 0x7f000015f580 con 0x7f00001551e0 2023-11-06T03:18:39.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe8003530 con 0x7f00001551e0 2023-11-06T03:18:39.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575788977 0 0) 0x7effe8003530 con 0x7f00001551e0 2023-11-06T03:18:39.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 msgr2=0x7f00000b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:39.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7f00000b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 >> v2:172.21.15.195:3300/0 conn(0x7f000015ea60 msgr2=0x7f00000b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:39.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 --2- 172.21.15.157:0/3678389088 >> v2:172.21.15.195:3300/0 conn(0x7f000015ea60 0x7f00000b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000015f2a0 con 0x7f00001551e0 2023-11-06T03:18:39.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7efff0001a00 con 0x7f00001551e0 2023-11-06T03:18:39.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 >> v1:172.21.15.157:6789/0 conn(0x7f00001551e0 legacy=0x7f000006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7effe8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8003f00 0x7effe80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7effe8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00000704b0 con 0x7effe8004d10 2023-11-06T03:18:39.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000015efc0 con 0x7effe8003f00 2023-11-06T03:18:39.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe800b9e0 con 0x7f000015e5e0 2023-11-06T03:18:39.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7effe8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7efffffff640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7effe8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7effff7fe640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8003f00 0x7effe80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.809 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 -- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 msgr2=0x7effe8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:39.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f000015e5e0 0x7effe8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 -- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8003f00 msgr2=0x7effe80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:39.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8003f00 0x7effe80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.810 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.806+0000 7f0004f7d640 1 -- 172.21.15.157:0/3678389088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8003530 con 0x7effe8004d10 2023-11-06T03:18:39.813 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7effe8005190 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7efff8009ae0 tx=0x7efff801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcee536f51999fd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff80a5530 con 0x7effe8004d10 2023-11-06T03:18:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff80a5690 con 0x7effe8004d10 2023-11-06T03:18:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffeffd640 1 -- 172.21.15.157:0/3678389088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff80a5a40 con 0x7effe8004d10 2023-11-06T03:18:39.814 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- 172.21.15.157:0/3678389088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 msgr2=0x7effe8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 --2- 172.21.15.157:0/3678389088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7effe8005190 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7efff8009ae0 tx=0x7efff801ca50 comp rx=0 tx=0).stop 2023-11-06T03:18:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f0004f7d640 1 -- 172.21.15.157:0/3678389088 reap_dead start 2023-11-06T03:18:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- 172.21.15.157:0/3678389088 shutdown_connections 2023-11-06T03:18:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- 172.21.15.157:0/3678389088 >> 172.21.15.157:0/3678389088 conn(0x7f0000065930 msgr2=0x7f0000065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:39.815 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- 172.21.15.157:0/3678389088 shutdown_connections 2023-11-06T03:18:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- 172.21.15.157:0/3678389088 wait complete. 2023-11-06T03:18:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 Processor -- start 2023-11-06T03:18:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- start start 2023-11-06T03:18:39.816 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f000014a250 0x7f000014db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000ce760 0x7f000014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.817 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000014abb0 con 0x7effe8004d10 2023-11-06T03:18:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000014a970 con 0x7f000014a250 2023-11-06T03:18:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f00069e0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000014ae00 con 0x7f00000ce760 2023-11-06T03:18:39.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38480/0 (socket says 172.21.15.157:38480) 2023-11-06T03:18:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 -- 172.21.15.157:0/3046119489 learned_addr learned my addr 172.21.15.157:0/3046119489 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:39.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7f0004f7d640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000ce760 0x7f000014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7effff7fe640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f000014a250 0x7f000014db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 -- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000ce760 msgr2=0x7f000014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000ce760 0x7f000014f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 -- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f000014a250 msgr2=0x7f000014db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f000014a250 0x7f000014db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.810+0000 7efffffff640 1 -- 172.21.15.157:0/3046119489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000cdc80 con 0x7effe8004d10 2023-11-06T03:18:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffffff640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7efff0001b30 tx=0x7efff00115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0004d70 con 0x7effe8004d10 2023-11-06T03:18:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0006110 con 0x7effe8004d10 2023-11-06T03:18:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00000cdf60 con 0x7effe8004d10 2023-11-06T03:18:39.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff00064c0 con 0x7effe8004d10 2023-11-06T03:18:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000cebe0 con 0x7effe8004d10 2023-11-06T03:18:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0000072750 con 0x7effe8004d10 2023-11-06T03:18:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efff0006620 con 0x7effe8004d10 2023-11-06T03:18:39.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 0x7effec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7efff0086980 con 0x7effe8004d10 2023-11-06T03:18:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7effff7fe640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 0x7effec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:39.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.814+0000 7effff7fe640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 0x7effec067f20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0000154850 tx=0x7efff4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:39.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.818+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7efff0054180 con 0x7effe8004d10 2023-11-06T03:18:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0000071730 con 0x7effe8004d10 2023-11-06T03:18:39.990 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7efffcff9640 1 -- 172.21.15.157:0/3046119489 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+4674 (secure 0 0 0) 0x7efff001c020 con 0x7effe8004d10 2023-11-06T03:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:39.991 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":135,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:39.938192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:39.991 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 135 2023-11-06T03:18:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 msgr2=0x7effec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 0x7effec067f20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0000154850 tx=0x7efff4008040 comp rx=0 tx=0).stop 2023-11-06T03:18:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 msgr2=0x7f00001404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:39.993 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7efff0001b30 tx=0x7efff00115c0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 shutdown_connections 2023-11-06T03:18:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7effec065a60 0x7effec067f20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00000ce760 0x7f000014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f000014a250 0x7f000014db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.994 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 --2- 172.21.15.157:0/3046119489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7effe8004d10 0x7f00001404e0 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:39.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 >> 172.21.15.157:0/3046119489 conn(0x7f0000065930 msgr2=0x7f0000153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:39.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 shutdown_connections 2023-11-06T03:18:39.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:39.994+0000 7f00069e0640 1 -- 172.21.15.157:0/3046119489 wait complete. 2023-11-06T03:18:40.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 135, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:18:18.097057+0000', 'modified': '2023-11-06T03:18:39.938192+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 190, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11487': {'gid': 11487, 'name': 'a', 'rank': 0, 'incarnation': 133, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.157:6835/2374994036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2374994036}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2374994036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:18:40.412 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11487:up:rejoin 2023-11-06T03:18:41.414 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:18:41.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.546+0000 7f383389e640 1 Processor -- start 2023-11-06T03:18:41.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.546+0000 7f383389e640 1 -- start start 2023-11-06T03:18:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f383389e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f382c0cde90 0x7f382c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f383389e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382c0cfed0 0x7f382c0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f383389e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c0d0320 con 0x7f382c15cc70 2023-11-06T03:18:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f383389e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c1614b0 con 0x7f382c0cfed0 2023-11-06T03:18:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f383389e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c160830 con 0x7f382c0cde90 2023-11-06T03:18:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382c0cfed0 0x7f382c0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f382c0cfed0 0x7f382c0d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38488/0 (socket says 172.21.15.157:38488) 2023-11-06T03:18:41.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 -- 172.21.15.157:0/3227458976 learned_addr learned my addr 172.21.15.157:0/3227458976 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3830e39640 1 --2- 172.21.15.157:0/3227458976 >> v2:172.21.15.195:3300/0 conn(0x7f382c0cde90 0x7f382c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 -- 172.21.15.157:0/3227458976 >> v2:172.21.15.195:3300/0 conn(0x7f382c0cde90 msgr2=0x7f382c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 --2- 172.21.15.157:0/3227458976 >> v2:172.21.15.195:3300/0 conn(0x7f382c0cde90 0x7f382c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 -- 172.21.15.157:0/3227458976 >> v1:172.21.15.195:6789/0 conn(0x7f382c15cc70 legacy=0x7f382c0d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:41.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 -- 172.21.15.157:0/3227458976 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c160b10 con 0x7f382c0cfed0 2023-11-06T03:18:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3831e3b640 1 --2- 172.21.15.157:0/3227458976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c0cfed0 0x7f382c0d0bb0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f382c0d08f0 tx=0x7f382401f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220a8e41bdd949a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:41.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3823fff640 1 -- 172.21.15.157:0/3227458976 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240af400 con 0x7f382c0cfed0 2023-11-06T03:18:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3823fff640 1 -- 172.21.15.157:0/3227458976 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240ab050 con 0x7f382c0cfed0 2023-11-06T03:18:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.550+0000 7f3823fff640 1 -- 172.21.15.157:0/3227458976 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240af780 con 0x7f382c0cfed0 2023-11-06T03:18:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/3227458976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c0cfed0 msgr2=0x7f382c0d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- 172.21.15.157:0/3227458976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c0cfed0 0x7f382c0d0bb0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f382c0d08f0 tx=0x7f382401f180 comp rx=0 tx=0).stop 2023-11-06T03:18:41.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/3227458976 shutdown_connections 2023-11-06T03:18:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- 172.21.15.157:0/3227458976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c0cfed0 0x7f382c0d0bb0 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- 172.21.15.157:0/3227458976 >> v2:172.21.15.195:3300/0 conn(0x7f382c0cde90 0x7f382c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/3227458976 >> 172.21.15.157:0/3227458976 conn(0x7f382c070560 msgr2=0x7f382c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/3227458976 shutdown_connections 2023-11-06T03:18:41.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/3227458976 wait complete. 2023-11-06T03:18:41.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 Processor -- start 2023-11-06T03:18:41.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- start start 2023-11-06T03:18:41.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382c0cde90 0x7f382c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f382c0cfed0 0x7f382c1f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0cab50 con 0x7f382c15cc70 2023-11-06T03:18:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c9c60 con 0x7f382c0cde90 2023-11-06T03:18:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0c9f40 con 0x7f382c0cfed0 2023-11-06T03:18:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38490/0 (socket says 172.21.15.157:38490) 2023-11-06T03:18:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 -- 172.21.15.157:0/1976804594 learned_addr learned my addr 172.21.15.157:0/1976804594 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3830e39640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f382c0cfed0 0x7f382c1f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383163a640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382c0cde90 0x7f382c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 -- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f382c0cfed0 msgr2=0x7f382c1f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:41.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f382c0cfed0 0x7f382c1f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 -- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382c0cde90 msgr2=0x7f382c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382c0cde90 0x7f382c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 -- 172.21.15.157:0/1976804594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3824009b30 con 0x7f382c15cc70 2023-11-06T03:18:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3831e3b640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f382401d830 tx=0x7f382401bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:41.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824007000 con 0x7f382c15cc70 2023-11-06T03:18:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240afe40 con 0x7f382c15cc70 2023-11-06T03:18:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c0ca220 con 0x7f382c15cc70 2023-11-06T03:18:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240b9630 con 0x7f382c15cc70 2023-11-06T03:18:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.554+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c0ca7e0 con 0x7f382c15cc70 2023-11-06T03:18:41.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c0cacc0 con 0x7f382c15cc70 2023-11-06T03:18:41.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f38240b6070 con 0x7f382c15cc70 2023-11-06T03:18:41.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f3821ffb640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 0x7f3810067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:41.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f3824121d70 con 0x7f382c15cc70 2023-11-06T03:18:41.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f383163a640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 0x7f3810067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:41.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.558+0000 7f383163a640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 0x7f3810067f20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f382800b810 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:41.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.562+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f38240ef6e0 con 0x7f382c15cc70 2023-11-06T03:18:41.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.750+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382c0d0320 con 0x7f382c15cc70 2023-11-06T03:18:41.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f3821ffb640 1 -- 172.21.15.157:0/1976804594 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7f382c0d0320 con 0x7f382c15cc70 2023-11-06T03:18:41.751 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:18:41.751 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:18:41.751 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:18:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 msgr2=0x7f3810067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 0x7f3810067f20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f382800b810 tx=0x7f3828008040 comp rx=0 tx=0).stop 2023-11-06T03:18:41.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 msgr2=0x7f382c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f382401d830 tx=0x7f382401bf20 comp rx=0 tx=0).stop 2023-11-06T03:18:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 shutdown_connections 2023-11-06T03:18:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3810065a60 0x7f3810067f20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f382c15cc70 0x7f382c0c8cc0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f382c0cfed0 0x7f382c1f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 --2- 172.21.15.157:0/1976804594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f382c0cde90 0x7f382c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 >> 172.21.15.157:0/1976804594 conn(0x7f382c070560 msgr2=0x7f382c0b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 shutdown_connections 2023-11-06T03:18:41.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:41.754+0000 7f383389e640 1 -- 172.21.15.157:0/1976804594 wait complete. 2023-11-06T03:18:42.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:18:18.097057+0000', 'modified': '2023-11-06T03:18:40.949661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 190, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11487': {'gid': 11487, 'name': 'a', 'rank': 0, 'incarnation': 133, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6835/2374994036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2374994036}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2374994036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:18:42.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:18:42.174 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:18:42.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 Processor -- start 2023-11-06T03:18:42.315 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 -- start start 2023-11-06T03:18:42.316 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 0x7f04c4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04c40b4c80 0x7f04c406c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c406b8d0 con 0x7f04c40713c0 2023-11-06T03:18:42.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c406be90 con 0x7f04c40b4c80 2023-11-06T03:18:42.317 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c8cf2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c4071880 con 0x7f04c415adc0 2023-11-06T03:18:42.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04c40b4c80 0x7f04c406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c1d9b640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f04c40713c0 0x7f04c40b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32896/0 (socket says 172.21.15.157:32896) 2023-11-06T03:18:42.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f04c40b4c80 0x7f04c406c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38502/0 (socket says 172.21.15.157:38502) 2023-11-06T03:18:42.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c1d9b640 1 -- 172.21.15.157:0/371705266 learned_addr learned my addr 172.21.15.157:0/371705266 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:42.318 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c259c640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 0x7f04c4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 -- 172.21.15.157:0/371705266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 msgr2=0x7f04c4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 0x7f04c4070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 -- 172.21.15.157:0/371705266 >> v1:172.21.15.157:6789/0 conn(0x7f04c40713c0 legacy=0x7f04c40b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.319 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c159a640 1 -- 172.21.15.157:0/371705266 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443900296 0 0) 0x7f04c406b8d0 con 0x7f04c40713c0 2023-11-06T03:18:42.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c2d9d640 1 -- 172.21.15.157:0/371705266 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4071b60 con 0x7f04c40b4c80 2023-11-06T03:18:42.320 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.318+0000 7f04c259c640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 0x7f04c4070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:42.323 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c40b4c80 0x7f04c406c320 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097c0 tx=0x7f04b401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5d208be695c869 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c159a640 1 -- 172.21.15.157:0/371705266 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4004230 con 0x7f04c40b4c80 2023-11-06T03:18:42.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c159a640 1 -- 172.21.15.157:0/371705266 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b40a7050 con 0x7f04c40b4c80 2023-11-06T03:18:42.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c159a640 1 -- 172.21.15.157:0/371705266 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4003620 con 0x7f04c40b4c80 2023-11-06T03:18:42.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- 172.21.15.157:0/371705266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c40b4c80 msgr2=0x7f04c406c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.324 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c40b4c80 0x7f04c406c320 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097c0 tx=0x7f04b401c330 comp rx=0 tx=0).stop 2023-11-06T03:18:42.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- 172.21.15.157:0/371705266 shutdown_connections 2023-11-06T03:18:42.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c40b4c80 0x7f04c406c320 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/371705266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c415adc0 0x7f04c4070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- 172.21.15.157:0/371705266 >> 172.21.15.157:0/371705266 conn(0x7f04c40cb990 msgr2=0x7f04c40cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.325 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- 172.21.15.157:0/371705266 shutdown_connections 2023-11-06T03:18:42.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- 172.21.15.157:0/371705266 wait complete. 2023-11-06T03:18:42.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 Processor -- start 2023-11-06T03:18:42.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.322+0000 7f04c8cf2640 1 -- start start 2023-11-06T03:18:42.326 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 0x7f04c4150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c40b4c80 0x7f04c41475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40c73a0 con 0x7f04c415adc0 2023-11-06T03:18:42.328 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40c7680 con 0x7f04c40713c0 2023-11-06T03:18:42.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40c7960 con 0x7f04c40b4c80 2023-11-06T03:18:42.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38516/0 (socket says 172.21.15.157:38516) 2023-11-06T03:18:42.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 -- 172.21.15.157:0/1549339255 learned_addr learned my addr 172.21.15.157:0/1549339255 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c259c640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 0x7f04c4150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c40b4c80 0x7f04c41475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 -- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c40b4c80 msgr2=0x7f04c41475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c40b4c80 0x7f04c41475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 -- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 msgr2=0x7f04c4150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 0x7f04c4150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 -- 172.21.15.157:0/1549339255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009420 con 0x7f04c415adc0 2023-11-06T03:18:42.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c259c640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 0x7f04c4150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:42.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f04b4019f60 tx=0x7f04b40190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 -- 172.21.15.157:0/1549339255 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4003bb0 con 0x7f04c415adc0 2023-11-06T03:18:42.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 -- 172.21.15.157:0/1549339255 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4003d10 con 0x7f04c415adc0 2023-11-06T03:18:42.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 -- 172.21.15.157:0/1549339255 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40ab640 con 0x7f04c415adc0 2023-11-06T03:18:42.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c40c7c40 con 0x7f04c415adc0 2023-11-06T03:18:42.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c4150bb0 con 0x7f04c415adc0 2023-11-06T03:18:42.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 -- 172.21.15.157:0/1549339255 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f04b40b2050 con 0x7f04c415adc0 2023-11-06T03:18:42.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 0x7f04a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04b2ffd640 1 -- 172.21.15.157:0/1549339255 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f04b411d720 con 0x7f04c415adc0 2023-11-06T03:18:42.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.326+0000 7f04c259c640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 0x7f04a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 Processor -- start 2023-11-06T03:18:42.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- start start 2023-11-06T03:18:42.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c259c640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 0x7f04a4067f20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f04c4151700 tx=0x7f04bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56403663f6e0 0x56403663fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x5640365f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640364bdfc0 con 0x5640365f5a90 2023-11-06T03:18:42.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403644c1a0 con 0x564036640000 2023-11-06T03:18:42.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564036449770 con 0x56403663f6e0 2023-11-06T03:18:42.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38530/0 (socket says 172.21.15.157:38530) 2023-11-06T03:18:42.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 -- 172.21.15.157:0/931577023 learned_addr learned my addr 172.21.15.157:0/931577023 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c259c640 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56403663f6e0 0x56403663fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x5640365f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 -- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56403663f6e0 msgr2=0x56403663fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56403663f6e0 0x56403663fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 -- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 msgr2=0x5640365f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x5640365f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 -- 172.21.15.157:0/931577023 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564036141890 con 0x5640365f5a90 2023-11-06T03:18:42.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f04b40f6680 tx=0x7f04b40ee660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc433438cf30c34 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04a3fff640 1 -- 172.21.15.157:0/931577023 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40f6bb0 con 0x5640365f5a90 2023-11-06T03:18:42.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04a3fff640 1 -- 172.21.15.157:0/931577023 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b40ecf70 con 0x5640365f5a90 2023-11-06T03:18:42.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04a3fff640 1 -- 172.21.15.157:0/931577023 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40efbe0 con 0x5640365f5a90 2023-11-06T03:18:42.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- 172.21.15.157:0/931577023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 msgr2=0x5640365f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f04b40f6680 tx=0x7f04b40ee660 comp rx=0 tx=0).stop 2023-11-06T03:18:42.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- 172.21.15.157:0/931577023 shutdown_connections 2023-11-06T03:18:42.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5640365f5a90 0x5640365f5e70 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x5640365f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/931577023 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x56403663f6e0 0x56403663fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- 172.21.15.157:0/931577023 >> 172.21.15.157:0/931577023 conn(0x5640365fbbf0 msgr2=0x5640365f3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- 172.21.15.157:0/931577023 shutdown_connections 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- 172.21.15.157:0/931577023 wait complete. 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 -- start start 2023-11-06T03:18:42.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.330+0000 7f04cbe1a000 1 Processor -- start 2023-11-06T03:18:42.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640365f5a90 0x564036600880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564036640000 0x564036601300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036445060 con 0x56403663f6e0 2023-11-06T03:18:42.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640363e4290 con 0x5640365f5a90 2023-11-06T03:18:42.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640364026f0 con 0x564036640000 2023-11-06T03:18:42.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 10 client.0 _ll_register_callbacks cb 0x5640365e11d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38546/0 (socket says 172.21.15.157:38546) 2023-11-06T03:18:42.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 -- 172.21.15.157:0/647995323 learned_addr learned my addr 172.21.15.157:0/647995323 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564036640000 0x564036601300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 -- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564036640000 msgr2=0x564036601300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564036640000 0x564036601300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 -- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640365f5a90 msgr2=0x564036600880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640365f5a90 0x564036600880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 -- 172.21.15.157:0/647995323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c4150bb0 con 0x56403663f6e0 2023-11-06T03:18:42.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001720 tx=0x7f04b8007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b30a7ba21e5fd7d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011640 con 0x56403663f6e0 2023-11-06T03:18:42.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f04b8015cc0 con 0x56403663f6e0 2023-11-06T03:18:42.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640364439b0 con 0x56403663f6e0 2023-11-06T03:18:42.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b8015e80 con 0x56403663f6e0 2023-11-06T03:18:42.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 10 client.11553 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x56403640a1e0 con 0x56403663f6e0 2023-11-06T03:18:42.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640364439b0 con 0x56403663f6e0 2023-11-06T03:18:42.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f04b8012070 con 0x56403663f6e0 2023-11-06T03:18:42.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 10 client.11553 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 10 client.11553 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:18:42.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 20 client.11553 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x56403664e770 0x564036650b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.352 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 4 client.11553 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:18:42.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640364744d0 con 0x56403664e770 2023-11-06T03:18:42.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c259c640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x56403664e770 0x564036650b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 10 client.11553 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04c259c640 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x56403664e770 0x564036650b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c935489a6da7373 server_cookie=f9da5e4a77633375 in_seq=0 out_seq=0 2023-11-06T03:18:42.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 1 -- 172.21.15.157:0/647995323 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5640364744d0 con 0x56403664e770 2023-11-06T03:18:42.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 10 client.11553 handle_command_reply: tid=0 2023-11-06T03:18:42.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 client.11553 shutdown 2023-11-06T03:18:42.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 msgr2=0x564036600dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f04b8001720 tx=0x7f04b8007b30 comp rx=0 tx=0).stop 2023-11-06T03:18:42.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 shutdown_connections 2023-11-06T03:18:42.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x56403664e770 0x564036650b60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x564036640000 0x564036601300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04a17fa640 0 client.11553 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036600dc0 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/647995323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5640365f5a90 0x564036600880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 >> 172.21.15.157:0/647995323 conn(0x5640365fbbf0 msgr2=0x5640365f4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 shutdown_connections 2023-11-06T03:18:42.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 1 -- 172.21.15.157:0/647995323 wait complete. 2023-11-06T03:18:42.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.334+0000 7f04cbe1a000 20 client.11553 trim_cache size 0 max 16384 2023-11-06T03:18:42.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 Processor -- start 2023-11-06T03:18:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- start start 2023-11-06T03:18:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x564036637250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x564036637cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036102940 con 0x56403663f6e0 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403643f400 con 0x564036640000 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x564036442460 con 0x5640365f5a90 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38558/0 (socket says 172.21.15.157:38558) 2023-11-06T03:18:42.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 -- 172.21.15.157:0/1179985710 learned_addr learned my addr 172.21.15.157:0/1179985710 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 -- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 msgr2=0x564036637250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c259c640 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x564036637250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x564036637250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x564036637cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.364 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 -- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 msgr2=0x564036637cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x564036637cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 -- 172.21.15.157:0/1179985710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640363f0330 con 0x56403663f6e0 2023-11-06T03:18:42.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f04b8018040 tx=0x7f04b800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb7ce55d1bb7b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04b0ff9640 1 -- 172.21.15.157:0/1179985710 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80119d0 con 0x56403663f6e0 2023-11-06T03:18:42.365 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04b0ff9640 1 -- 172.21.15.157:0/1179985710 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800d030 con 0x56403663f6e0 2023-11-06T03:18:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04b0ff9640 1 -- 172.21.15.157:0/1179985710 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8011be0 con 0x56403663f6e0 2023-11-06T03:18:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- 172.21.15.157:0/1179985710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 msgr2=0x564036637790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f04b8018040 tx=0x7f04b800f000 comp rx=0 tx=0).stop 2023-11-06T03:18:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- 172.21.15.157:0/1179985710 shutdown_connections 2023-11-06T03:18:42.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564036640000 0x564036637cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56403663f6e0 0x564036637790 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/1179985710 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x564036637250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- 172.21.15.157:0/1179985710 >> 172.21.15.157:0/1179985710 conn(0x5640365fbbf0 msgr2=0x564036652330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- 172.21.15.157:0/1179985710 shutdown_connections 2023-11-06T03:18:42.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.350+0000 7f04cbe1a000 1 -- 172.21.15.157:0/1179985710 wait complete. 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- start start 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 Processor -- start 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x5640366a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56403663f6e0 0x5640366a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564036408ff0 con 0x564036640000 2023-11-06T03:18:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403640c7d0 con 0x56403663f6e0 2023-11-06T03:18:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640363fe0f0 con 0x5640365f5a90 2023-11-06T03:18:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 10 client.0 _ll_register_callbacks cb 0x5640365c19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38562/0 (socket says 172.21.15.157:38562) 2023-11-06T03:18:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 -- 172.21.15.157:0/24933429 learned_addr learned my addr 172.21.15.157:0/24933429 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c259c640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x5640366a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c1d9b640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56403663f6e0 0x5640366a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 -- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 msgr2=0x5640366a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x5640366a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 -- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56403663f6e0 msgr2=0x5640366a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56403663f6e0 0x5640366a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 -- 172.21.15.157:0/24933429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56403640a1e0 con 0x564036640000 2023-11-06T03:18:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c2d9d640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f04b40f1040 tx=0x7f04b4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4020a5b9be3f1c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.371 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4003e80 con 0x564036640000 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f04b40f6bb0 con 0x564036640000 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564036414b90 con 0x564036640000 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b40efbe0 con 0x564036640000 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 10 client.11559 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x564036141cc0 con 0x564036640000 2023-11-06T03:18:42.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564036414b90 con 0x564036640000 2023-11-06T03:18:42.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f04b40ee0c0 con 0x564036640000 2023-11-06T03:18:42.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 10 client.11559 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 10 client.11559 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:18:42.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 20 client.11559 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x564036688ff0 0x5640366893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 4 client.11559 mds_command: new command op to 11487 tid=0[{"prefix": "status"}] 2023-11-06T03:18:42.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56403643e0e0 con 0x564036688ff0 2023-11-06T03:18:42.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c259c640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x564036688ff0 0x5640366893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 10 client.11559 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04c259c640 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x564036688ff0 0x5640366893d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb811b9d447fde9 server_cookie=e972001e51072443 in_seq=0 out_seq=0 2023-11-06T03:18:42.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 1 -- 172.21.15.157:0/24933429 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56403643e0e0 con 0x564036688ff0 2023-11-06T03:18:42.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 10 client.11559 handle_command_reply: tid=0 2023-11-06T03:18:42.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 client.11559 shutdown 2023-11-06T03:18:42.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 msgr2=0x564036688ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f04b40f1040 tx=0x7f04b4009190 comp rx=0 tx=0).stop 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 shutdown_connections 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x564036688ff0 0x5640366893d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564036640000 0x564036688ab0 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56403663f6e0 0x5640366a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04a17fa640 0 client.11559 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 --2- 172.21.15.157:0/24933429 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5640365f5a90 0x5640366a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 >> 172.21.15.157:0/24933429 conn(0x5640365fbbf0 msgr2=0x564036651aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 shutdown_connections 2023-11-06T03:18:42.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 1 -- 172.21.15.157:0/24933429 wait complete. 2023-11-06T03:18:42.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.354+0000 7f04cbe1a000 20 client.11559 trim_cache size 0 max 16384 2023-11-06T03:18:42.378 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:42.378 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:18:42.378 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 11487, 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 4.4224163870000002, 2023-11-06T03:18:42.379 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 136, 2023-11-06T03:18:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 190, 2023-11-06T03:18:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 190, 2023-11-06T03:18:42.380 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 5.408957998 2023-11-06T03:18:42.380 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:42.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 msgr2=0x7f04a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 0x7f04a4067f20 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f04c4151700 tx=0x7f04bc008040 comp rx=0 tx=0).stop 2023-11-06T03:18:42.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 msgr2=0x7f04c40b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f04b4019f60 tx=0x7f04b40190d0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.382 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 shutdown_connections 2023-11-06T03:18:42.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f04a4065a60 0x7f04a4067f20 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04c415adc0 0x7f04c40b30c0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04c40b4c80 0x7f04c41475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 --2- 172.21.15.157:0/1549339255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04c40713c0 0x7f04c4150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.383 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 >> 172.21.15.157:0/1549339255 conn(0x7f04c40cb990 msgr2=0x7f04c4158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 shutdown_connections 2023-11-06T03:18:42.384 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.358+0000 7f04c8cf2640 1 -- 172.21.15.157:0/1549339255 wait complete. 2023-11-06T03:18:42.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub start / recursive,repair 2023-11-06T03:18:42.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 Processor -- start 2023-11-06T03:18:42.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 -- start start 2023-11-06T03:18:42.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feed00cbf90 0x7feed015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed00ce820 con 0x7feed015aa10 2023-11-06T03:18:42.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00ceae0 con 0x7feed00cbf90 2023-11-06T03:18:42.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feed70fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00cedc0 con 0x7feed00ae3f0 2023-11-06T03:18:42.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feecffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feed00cbf90 0x7feed015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feecffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feed00cbf90 0x7feed015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38564/0 (socket says 172.21.15.157:38564) 2023-11-06T03:18:42.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.550+0000 7feecffff640 1 -- 172.21.15.157:0/2606422734 learned_addr learned my addr 172.21.15.157:0/2606422734 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed5698640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecffff640 1 -- 172.21.15.157:0/2606422734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 msgr2=0x7feed006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecffff640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecffff640 1 -- 172.21.15.157:0/2606422734 >> v1:172.21.15.157:6789/0 conn(0x7feed015aa10 legacy=0x7feed00cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecffff640 1 -- 172.21.15.157:0/2606422734 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed006bec0 con 0x7feed00cbf90 2023-11-06T03:18:42.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed5698640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:42.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecffff640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed015dcc0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe505b9da2202cac server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecf7fe640 1 -- 172.21.15.157:0/2606422734 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40a9620 con 0x7feed00cbf90 2023-11-06T03:18:42.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecf7fe640 1 -- 172.21.15.157:0/2606422734 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40a9780 con 0x7feed00cbf90 2023-11-06T03:18:42.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feecf7fe640 1 -- 172.21.15.157:0/2606422734 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40a9b10 con 0x7feed00cbf90 2023-11-06T03:18:42.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 -- 172.21.15.157:0/2606422734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 msgr2=0x7feed015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed015dcc0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401cfa0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 -- 172.21.15.157:0/2606422734 shutdown_connections 2023-11-06T03:18:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed006c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 --2- 172.21.15.157:0/2606422734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed015dcc0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 -- 172.21.15.157:0/2606422734 >> 172.21.15.157:0/2606422734 conn(0x7feed00b0c30 msgr2=0x7feed00b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 -- 172.21.15.157:0/2606422734 shutdown_connections 2023-11-06T03:18:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.554+0000 7feed70fb640 1 -- 172.21.15.157:0/2606422734 wait complete. 2023-11-06T03:18:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 Processor -- start 2023-11-06T03:18:42.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- start start 2023-11-06T03:18:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed0142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed015aa10 0x7feed01552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0142fe0 con 0x7feed00cbf90 2023-11-06T03:18:42.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0154600 con 0x7feed015aa10 2023-11-06T03:18:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00aec20 con 0x7feed00ae3f0 2023-11-06T03:18:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38568/0 (socket says 172.21.15.157:38568) 2023-11-06T03:18:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 -- 172.21.15.157:0/1408076699 learned_addr learned my addr 172.21.15.157:0/1408076699 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed4e97640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed0142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed5698640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed015aa10 0x7feed01552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 -- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 msgr2=0x7feed0142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed0142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 -- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed015aa10 msgr2=0x7feed01552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed015aa10 0x7feed01552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 -- 172.21.15.157:0/1408076699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40094d0 con 0x7feed00cbf90 2023-11-06T03:18:42.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed4e97640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed0142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:42.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecffff640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7feec40ab040 tx=0x7feec401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecd7fa640 1 -- 172.21.15.157:0/1408076699 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40bc450 con 0x7feed00cbf90 2023-11-06T03:18:42.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecd7fa640 1 -- 172.21.15.157:0/1408076699 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40bc5d0 con 0x7feed00cbf90 2023-11-06T03:18:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed00aef00 con 0x7feed00cbf90 2023-11-06T03:18:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecd7fa640 1 -- 172.21.15.157:0/1408076699 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40bc980 con 0x7feed00cbf90 2023-11-06T03:18:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed00af4c0 con 0x7feed00cbf90 2023-11-06T03:18:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.558+0000 7feecd7fa640 1 -- 172.21.15.157:0/1408076699 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feec40bcae0 con 0x7feed00cbf90 2023-11-06T03:18:42.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecd7fa640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 0x7feebc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecd7fa640 1 -- 172.21.15.157:0/1408076699 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7feec411dcd0 con 0x7feed00cbf90 2023-11-06T03:18:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feed4e97640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 0x7feebc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 Processor -- start 2023-11-06T03:18:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- start start 2023-11-06T03:18:42.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feed4e97640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 0x7feebc067f20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feec0000f70 tx=0x7feec0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e86a0 0x55922e7e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e666a90 con 0x55922e7e8fc0 2023-11-06T03:18:42.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e5f4490 con 0x55922e7e86a0 2023-11-06T03:18:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e5f1a60 con 0x55922e7cd400 2023-11-06T03:18:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38570/0 (socket says 172.21.15.157:38570) 2023-11-06T03:18:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 -- 172.21.15.157:0/1801402294 learned_addr learned my addr 172.21.15.157:0/1801402294 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feed4e97640 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e86a0 0x55922e7e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feed5698640 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 -- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 msgr2=0x55922e7cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 -- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e86a0 msgr2=0x55922e7e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e86a0 0x55922e7e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 -- 172.21.15.157:0/1801402294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922e2ea070 con 0x55922e7e8fc0 2023-11-06T03:18:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feecffff640 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7feec40eefb0 tx=0x7feec4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833425221c1573f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeb67fc640 1 -- 172.21.15.157:0/1801402294 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40f7290 con 0x55922e7e8fc0 2023-11-06T03:18:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeb67fc640 1 -- 172.21.15.157:0/1801402294 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40bcd90 con 0x55922e7e8fc0 2023-11-06T03:18:42.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeb67fc640 1 -- 172.21.15.157:0/1801402294 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40f50b0 con 0x55922e7e8fc0 2023-11-06T03:18:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- 172.21.15.157:0/1801402294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 msgr2=0x55922e7ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7feec40eefb0 tx=0x7feec4009240 comp rx=0 tx=0).stop 2023-11-06T03:18:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- 172.21.15.157:0/1801402294 shutdown_connections 2023-11-06T03:18:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7ccec0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 --2- 172.21.15.157:0/1801402294 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e86a0 0x55922e7e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- 172.21.15.157:0/1801402294 >> 172.21.15.157:0/1801402294 conn(0x55922e795bc0 msgr2=0x55922e7a4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- 172.21.15.157:0/1801402294 shutdown_connections 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.562+0000 7feeda223000 1 -- 172.21.15.157:0/1801402294 wait complete. 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- start start 2023-11-06T03:18:42.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 Processor -- start 2023-11-06T03:18:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e5ed350 con 0x55922e7e86a0 2023-11-06T03:18:42.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e58c610 con 0x55922e7e8fc0 2023-11-06T03:18:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e5aa800 con 0x55922e7cd400 2023-11-06T03:18:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 10 client.0 _ll_register_callbacks cb 0x55922e5e2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38578/0 (socket says 172.21.15.157:38578) 2023-11-06T03:18:42.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 -- 172.21.15.157:0/1955284460 learned_addr learned my addr 172.21.15.157:0/1955284460 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feed4e97640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feed5698640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 -- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 msgr2=0x55922e7ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 -- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 msgr2=0x55922e7a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 -- 172.21.15.157:0/1955284460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed00af4c0 con 0x55922e7e86a0 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feecffff640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7feec401ca20 tx=0x7feec411bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa87b31fdbee6a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40f50b0 con 0x55922e7e86a0 2023-11-06T03:18:42.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55922e5bcc80 con 0x55922e7e86a0 2023-11-06T03:18:42.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7feec40f6e00 con 0x55922e7e86a0 2023-11-06T03:18:42.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec40f5210 con 0x55922e7e86a0 2023-11-06T03:18:42.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 10 client.11571 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55922e5b22d0 con 0x55922e7e86a0 2023-11-06T03:18:42.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55922e5bcc80 con 0x55922e7e86a0 2023-11-06T03:18:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7feec40eb0a0 con 0x55922e7e86a0 2023-11-06T03:18:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 10 client.11571 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 10 client.11571 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:18:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 20 client.11571 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7a2d40 0x55922e7a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 4 client.11571 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:18:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55922e5ebca0 con 0x55922e7a2d40 2023-11-06T03:18:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feed4e97640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7a2d40 0x55922e7a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feed4e97640 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7a2d40 0x55922e7a3150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5d036fbfa663b5 server_cookie=137ef46c19c640de in_seq=0 out_seq=0 2023-11-06T03:18:42.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 10 client.11571 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 1 -- 172.21.15.157:0/1955284460 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55922e5ebca0 con 0x55922e7a2d40 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 10 client.11571 handle_command_reply: tid=0 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 client.11571 shutdown 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 msgr2=0x55922e7a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7feec401ca20 tx=0x7feec411bf10 comp rx=0 tx=0).stop 2023-11-06T03:18:42.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 shutdown_connections 2023-11-06T03:18:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7a2d40 0x55922e7a3150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1df0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeaf7fe640 0 client.11571 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 --2- 172.21.15.157:0/1955284460 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 >> 172.21.15.157:0/1955284460 conn(0x55922e795bc0 msgr2=0x55922e7ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 shutdown_connections 2023-11-06T03:18:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 1 -- 172.21.15.157:0/1955284460 wait complete. 2023-11-06T03:18:42.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.566+0000 7feeda223000 20 client.11571 trim_cache size 0 max 16384 2023-11-06T03:18:42.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 Processor -- start 2023-11-06T03:18:42.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 -- start start 2023-11-06T03:18:42.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7f8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e2aacf0 con 0x55922e7e86a0 2023-11-06T03:18:42.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e5bc8e0 con 0x55922e7e8fc0 2023-11-06T03:18:42.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e7f6140 con 0x55922e7cd400 2023-11-06T03:18:42.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38588/0 (socket says 172.21.15.157:38588) 2023-11-06T03:18:42.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 -- 172.21.15.157:0/1820227811 learned_addr learned my addr 172.21.15.157:0/1820227811 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feed4e97640 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 -- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 msgr2=0x55922e7a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 -- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 msgr2=0x55922e7f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.593 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 -- 172.21.15.157:0/1820227811 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922e598620 con 0x55922e7e86a0 2023-11-06T03:18:42.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feecffff640 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7feec40ed600 tx=0x7feec40f8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b45fd21f2c451ed server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeb6ffd640 1 -- 172.21.15.157:0/1820227811 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40eedf0 con 0x55922e7e86a0 2023-11-06T03:18:42.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeb6ffd640 1 -- 172.21.15.157:0/1820227811 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40f7050 con 0x55922e7e86a0 2023-11-06T03:18:42.594 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.582+0000 7feeb6ffd640 1 -- 172.21.15.157:0/1820227811 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40eb0a0 con 0x55922e7e86a0 2023-11-06T03:18:42.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/1820227811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 msgr2=0x55922e7a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7feec40ed600 tx=0x7feec40f8000 comp rx=0 tx=0).stop 2023-11-06T03:18:42.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/1820227811 shutdown_connections 2023-11-06T03:18:42.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7e8fc0 0x55922e7f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.595 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e86a0 0x55922e7a1f00 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- 172.21.15.157:0/1820227811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7cd400 0x55922e7a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/1820227811 >> 172.21.15.157:0/1820227811 conn(0x55922e795bc0 msgr2=0x55922e7cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/1820227811 shutdown_connections 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/1820227811 wait complete. 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- start start 2023-11-06T03:18:42.596 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 Processor -- start 2023-11-06T03:18:42.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 0x55922e84cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7e86a0 0x55922e84d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e7e8120 con 0x55922e7e8fc0 2023-11-06T03:18:42.597 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e5bbfc0 con 0x55922e7cd400 2023-11-06T03:18:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e591340 con 0x55922e7e86a0 2023-11-06T03:18:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 10 client.0 _ll_register_callbacks cb 0x55922e5f30a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38600/0 (socket says 172.21.15.157:38600) 2023-11-06T03:18:42.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 -- 172.21.15.157:0/2336042171 learned_addr learned my addr 172.21.15.157:0/2336042171 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed4e97640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 0x55922e84cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feecffff640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7e86a0 0x55922e84d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 -- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7e86a0 msgr2=0x55922e84d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7e86a0 0x55922e84d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 -- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 msgr2=0x55922e84cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 0x55922e84cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 -- 172.21.15.157:0/2336042171 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55922e5b22d0 con 0x55922e7e8fc0 2023-11-06T03:18:42.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed4e97640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 0x55922e84cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:42.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed5698640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7feec800da50 tx=0x7feec800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bd50dd471b7607 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80039a0 con 0x55922e7e8fc0 2023-11-06T03:18:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7feec8003b00 con 0x55922e7e8fc0 2023-11-06T03:18:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55922e59cd50 con 0x55922e7e8fc0 2023-11-06T03:18:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec8003cc0 con 0x55922e7e8fc0 2023-11-06T03:18:42.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 10 client.11577 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55922e58c190 con 0x55922e7e8fc0 2023-11-06T03:18:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55922e59cd50 con 0x55922e7e8fc0 2023-11-06T03:18:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7feec8008040 con 0x55922e7e8fc0 2023-11-06T03:18:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 10 client.11577 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 10 client.11577 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 20 client.11577 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7fbd90 0x55922e7fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 4 client.11577 mds_command: new command op to 11487 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55922e5e63d0 con 0x55922e7fbd90 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed4e97640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7fbd90 0x55922e7fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feed4e97640 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7fbd90 0x55922e7fc1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac80e292513533fc server_cookie=4c5932fd36e1890b in_seq=0 out_seq=0 2023-11-06T03:18:42.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.586+0000 7feeaf7fe640 10 client.11577 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeaf7fe640 1 -- 172.21.15.157:0/2336042171 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55922e5e63d0 con 0x55922e7fbd90 2023-11-06T03:18:42.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeaf7fe640 10 client.11577 handle_command_reply: tid=0 2023-11-06T03:18:42.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 client.11577 shutdown 2023-11-06T03:18:42.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 msgr2=0x55922e7fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7feec800da50 tx=0x7feec800df10 comp rx=0 tx=0).stop 2023-11-06T03:18:42.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 shutdown_connections 2023-11-06T03:18:42.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55922e7fbd90 0x55922e7fc1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922e7e8fc0 0x55922e7fae70 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55922e7e86a0 0x55922e84d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeaf7fe640 0 client.11577 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 --2- 172.21.15.157:0/2336042171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55922e7cd400 0x55922e84cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 >> 172.21.15.157:0/2336042171 conn(0x55922e795bc0 msgr2=0x55922e7a09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 shutdown_connections 2023-11-06T03:18:42.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 1 -- 172.21.15.157:0/2336042171 wait complete. 2023-11-06T03:18:42.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feeda223000 20 client.11577 trim_cache size 0 max 16384 2023-11-06T03:18:42.607 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:42.607 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:18:42.608 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "7a0bf5b2-a91f-46c3-9ee2-c756fbb09e2b", 2023-11-06T03:18:42.608 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:18:42.608 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:42.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 msgr2=0x7feebc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 0x7feebc067f20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7feec0000f70 tx=0x7feec0009040 comp rx=0 tx=0).stop 2023-11-06T03:18:42.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 msgr2=0x7feed0154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7feec40ab040 tx=0x7feec401ad80 comp rx=0 tx=0).stop 2023-11-06T03:18:42.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 shutdown_connections 2023-11-06T03:18:42.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feebc065a60 0x7feebc067f20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feed015aa10 0x7feed01552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feed00cbf90 0x7feed0154d80 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 --2- 172.21.15.157:0/1408076699 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feed00ae3f0 0x7feed0142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 >> 172.21.15.157:0/1408076699 conn(0x7feed00b0c30 msgr2=0x7feed015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 shutdown_connections 2023-11-06T03:18:42.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.590+0000 7feed70fb640 1 -- 172.21.15.157:0/1408076699 wait complete. 2023-11-06T03:18:42.612 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub status 2023-11-06T03:18:42.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 Processor -- start 2023-11-06T03:18:42.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 -- start start 2023-11-06T03:18:42.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f67080b3b20 0x7f670815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670815f580 con 0x7f670815e5e0 2023-11-06T03:18:42.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670815efc0 con 0x7f67081551e0 2023-11-06T03:18:42.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f671071e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080704b0 con 0x7f67080b3b20 2023-11-06T03:18:42.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f670dcb9640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f670815e5e0 0x7f67080b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:32912/0 (socket says 172.21.15.157:32912) 2023-11-06T03:18:42.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f670dcb9640 1 -- 172.21.15.157:0/3946368780 learned_addr learned my addr 172.21.15.157:0/3946368780 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:18:42.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f670ecbb640 1 --2- 172.21.15.157:0/3946368780 >> v2:172.21.15.195:3300/0 conn(0x7f67080b3b20 0x7f670815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507656385 0 0) 0x7f670815f580 con 0x7f670815e5e0 2023-11-06T03:18:42.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.734+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f4003530 con 0x7f670815e5e0 2023-11-06T03:18:42.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1538559974 0 0) 0x7f66f4003530 con 0x7f670815e5e0 2023-11-06T03:18:42.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 >> v2:172.21.15.195:3300/0 conn(0x7f67080b3b20 msgr2=0x7f670815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 --2- 172.21.15.157:0/3946368780 >> v2:172.21.15.195:3300/0 conn(0x7f67080b3b20 0x7f670815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 >> v1:172.21.15.195:6789/0 conn(0x7f67081551e0 legacy=0x7f670806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670815f2a0 con 0x7f670815e5e0 2023-11-06T03:18:42.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f6704001680 con 0x7f670815e5e0 2023-11-06T03:18:42.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 >> v1:172.21.15.157:6789/0 conn(0x7f670815e5e0 legacy=0x7f67080b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f4003980 0x7f66f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 0x7f66f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40052b0 0x7f66f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080704b0 con 0x7f66f40047d0 2023-11-06T03:18:42.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f400c180 con 0x7f66f40052b0 2023-11-06T03:18:42.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f400c340 con 0x7f66f4003980 2023-11-06T03:18:42.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 0x7f66f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670ecbb640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40052b0 0x7f66f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670e4ba640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f4003980 0x7f66f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 -- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f4003980 msgr2=0x7f66f4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66f4003980 0x7f66f4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 -- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40052b0 msgr2=0x7f66f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40052b0 0x7f66f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670ecbb640 1 -- 172.21.15.157:0/3946368780 reap_dead start 2023-11-06T03:18:42.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 -- 172.21.15.157:0/3946368780 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4003530 con 0x7f66f40047d0 2023-11-06T03:18:42.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670dcb9640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 0x7f66f4004c50 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f6704001ec0 tx=0x7f670401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ac70f095a61e24 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.746 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040058e0 con 0x7f66f40047d0 2023-11-06T03:18:42.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704005a40 con 0x7f66f40047d0 2023-11-06T03:18:42.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f670d4b8640 1 -- 172.21.15.157:0/3946368780 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704005e70 con 0x7f66f40047d0 2023-11-06T03:18:42.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 -- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 msgr2=0x7f66f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 0x7f66f4004c50 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f6704001ec0 tx=0x7f670401cb70 comp rx=0 tx=0).stop 2023-11-06T03:18:42.747 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 -- 172.21.15.157:0/3946368780 shutdown_connections 2023-11-06T03:18:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 --2- 172.21.15.157:0/3946368780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66f40047d0 0x7f66f4004c50 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 -- 172.21.15.157:0/3946368780 >> 172.21.15.157:0/3946368780 conn(0x7f6708065930 msgr2=0x7f6708065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 -- 172.21.15.157:0/3946368780 shutdown_connections 2023-11-06T03:18:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.738+0000 7f671071e640 1 -- 172.21.15.157:0/3946368780 wait complete. 2023-11-06T03:18:42.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 Processor -- start 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- start start 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40047d0 0x7f670814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67080ccb60 0x7f67080ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cd520 con 0x7f670814a8b0 2023-11-06T03:18:42.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67081f50c0 con 0x7f66f40047d0 2023-11-06T03:18:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67081f4540 con 0x7f67080ccb60 2023-11-06T03:18:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38622/0 (socket says 172.21.15.157:38622) 2023-11-06T03:18:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 -- 172.21.15.157:0/282167413 learned_addr learned my addr 172.21.15.157:0/282167413 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670e4ba640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40047d0 0x7f670814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670ecbb640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67080ccb60 0x7f67080ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 -- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67080ccb60 msgr2=0x7f67080ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67080ccb60 0x7f67080ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 -- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40047d0 msgr2=0x7f670814ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.751 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40047d0 0x7f670814ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 -- 172.21.15.157:0/282167413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67081f4820 con 0x7f670814a8b0 2023-11-06T03:18:42.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f670dcb9640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f67040ab040 tx=0x7f66f4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f66feffd640 1 -- 172.21.15.157:0/282167413 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040047f0 con 0x7f670814a8b0 2023-11-06T03:18:42.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f66feffd640 1 -- 172.21.15.157:0/282167413 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704004950 con 0x7f670814a8b0 2023-11-06T03:18:42.752 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f66feffd640 1 -- 172.21.15.157:0/282167413 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704004cc0 con 0x7f670814a8b0 2023-11-06T03:18:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081f4b00 con 0x7f670814a8b0 2023-11-06T03:18:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.742+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6708151670 con 0x7f670814a8b0 2023-11-06T03:18:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66feffd640 1 -- 172.21.15.157:0/282167413 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f67040a5730 con 0x7f670814a8b0 2023-11-06T03:18:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66feffd640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 0x7f66ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.753 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66feffd640 1 -- 172.21.15.157:0/282167413 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f670411ccd0 con 0x7f670814a8b0 2023-11-06T03:18:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 0x7f66ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 Processor -- start 2023-11-06T03:18:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 -- start start 2023-11-06T03:18:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 0x7f66ec067f20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c10 tx=0x7f66f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.754 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd02fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd030640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcef7c10 con 0x55c7fd07a130 2023-11-06T03:18:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce863e0 con 0x55c7fd07aa50 2023-11-06T03:18:42.755 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce839b0 con 0x55c7fd030260 2023-11-06T03:18:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38636/0 (socket says 172.21.15.157:38636) 2023-11-06T03:18:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 -- 172.21.15.157:0/4055408632 learned_addr learned my addr 172.21.15.157:0/4055408632 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670dcb9640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd02fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.756 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 -- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 msgr2=0x55c7fd030640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670ecbb640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd030640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd030640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 -- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 msgr2=0x55c7fd02fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd02fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 -- 172.21.15.157:0/4055408632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fcb7be20 con 0x55c7fd07a130 2023-11-06T03:18:42.757 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670dcb9640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd02fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:42.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670ecbb640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd030640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:18:42.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f670e4ba640 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f66f8001460 tx=0x7f66f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7032ad845da991f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66ebfff640 1 -- 172.21.15.157:0/4055408632 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8014600 con 0x55c7fd07a130 2023-11-06T03:18:42.758 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66ebfff640 1 -- 172.21.15.157:0/4055408632 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8014760 con 0x55c7fd07a130 2023-11-06T03:18:42.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.746+0000 7f66ebfff640 1 -- 172.21.15.157:0/4055408632 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8014ad0 con 0x55c7fd07a130 2023-11-06T03:18:42.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/4055408632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 msgr2=0x55c7fd07a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f66f8001460 tx=0x7f66f8013000 comp rx=0 tx=0).stop 2023-11-06T03:18:42.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/4055408632 shutdown_connections 2023-11-06T03:18:42.759 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd030640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd02fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- 172.21.15.157:0/4055408632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd07a510 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/4055408632 >> 172.21.15.157:0/4055408632 conn(0x55c7fd0369f0 msgr2=0x55c7fd02e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/4055408632 shutdown_connections 2023-11-06T03:18:42.760 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/4055408632 wait complete. 2023-11-06T03:18:42.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- start start 2023-11-06T03:18:42.761 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 Processor -- start 2023-11-06T03:18:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd079620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07a130 0x55c7fd07b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce7f2a0 con 0x55c7fd07aa50 2023-11-06T03:18:42.766 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce1e470 con 0x55c7fd07a130 2023-11-06T03:18:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce3c660 con 0x55c7fd030260 2023-11-06T03:18:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 10 client.0 _ll_register_callbacks cb 0x55c7fd01bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38652/0 (socket says 172.21.15.157:38652) 2023-11-06T03:18:42.767 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 -- 172.21.15.157:0/1606453002 learned_addr learned my addr 172.21.15.157:0/1606453002 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd079620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670dcb9640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07a130 0x55c7fd07b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 -- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 msgr2=0x55c7fd079620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd079620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.768 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 -- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07a130 msgr2=0x55c7fd07b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07a130 0x55c7fd07b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 -- 172.21.15.157:0/1606453002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f800f040 con 0x55c7fd07aa50 2023-11-06T03:18:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd079620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:18:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670ecbb640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f670815e9c0 tx=0x7f6700004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d7bd26f865100 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.769 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000e040 con 0x55c7fd07aa50 2023-11-06T03:18:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6700007a70 con 0x55c7fd07aa50 2023-11-06T03:18:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7fce3c0a0 con 0x55c7fd07aa50 2023-11-06T03:18:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6700004890 con 0x55c7fd07aa50 2023-11-06T03:18:42.770 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 10 client.11589 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55c7fce441d0 con 0x55c7fd07aa50 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7fce3c0a0 con 0x55c7fd07aa50 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6700007c30 con 0x55c7fd07aa50 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 10 client.11589 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 10 client.11589 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:18:42.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 20 client.11589 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd07c300 0x55c7fd08c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 4 client.11589 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:18:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7fce7dbf0 con 0x55c7fd07c300 2023-11-06T03:18:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd07c300 0x55c7fd08c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd07c300 0x55c7fd08c560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94990c1345aae47 server_cookie=f3d902ff8947689e in_seq=0 out_seq=0 2023-11-06T03:18:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 10 client.11589 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 1 -- 172.21.15.157:0/1606453002 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c7fce7dbf0 con 0x55c7fd07c300 2023-11-06T03:18:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.750+0000 7f66e97fa640 10 client.11589 handle_command_reply: tid=0 2023-11-06T03:18:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 client.11589 shutdown 2023-11-06T03:18:42.773 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 msgr2=0x55c7fd07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f670815e9c0 tx=0x7f6700004640 comp rx=0 tx=0).stop 2023-11-06T03:18:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 shutdown_connections 2023-11-06T03:18:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd07c300 0x55c7fd08c560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07aa50 0x55c7fd07b580 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07a130 0x55c7fd07b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f66e97fa640 0 client.11589 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 --2- 172.21.15.157:0/1606453002 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd030260 0x55c7fd079620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 >> 172.21.15.157:0/1606453002 conn(0x55c7fd0369f0 msgr2=0x55c7fd02ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 shutdown_connections 2023-11-06T03:18:42.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 1 -- 172.21.15.157:0/1606453002 wait complete. 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.754+0000 7f6713846000 20 client.11589 trim_cache size 0 max 16384 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 Processor -- start 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 -- start start 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07a130 0x55c7fd035e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd08c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb3caa0 con 0x55c7fd030260 2023-11-06T03:18:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcec1cd0 con 0x55c7fd07aa50 2023-11-06T03:18:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fd079670 con 0x55c7fd07a130 2023-11-06T03:18:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38666/0 (socket says 172.21.15.157:38666) 2023-11-06T03:18:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 -- 172.21.15.157:0/1360981563 learned_addr learned my addr 172.21.15.157:0/1360981563 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 -- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07a130 msgr2=0x55c7fd035e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07a130 0x55c7fd035e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 -- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 msgr2=0x55c7fd08c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd08c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.766+0000 7f670e4ba640 1 -- 172.21.15.157:0/1360981563 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fce2a520 con 0x55c7fd030260 2023-11-06T03:18:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670e4ba640 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005630 tx=0x7f66f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60202ad43f394d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66fcff9640 1 -- 172.21.15.157:0/1360981563 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8021a20 con 0x55c7fd030260 2023-11-06T03:18:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66fcff9640 1 -- 172.21.15.157:0/1360981563 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8021b80 con 0x55c7fd030260 2023-11-06T03:18:42.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66fcff9640 1 -- 172.21.15.157:0/1360981563 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8007020 con 0x55c7fd030260 2023-11-06T03:18:42.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/1360981563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 msgr2=0x55c7fd035920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005630 tx=0x7f66f8014000 comp rx=0 tx=0).stop 2023-11-06T03:18:42.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/1360981563 shutdown_connections 2023-11-06T03:18:42.784 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd07aa50 0x55c7fd08c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07a130 0x55c7fd035e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- 172.21.15.157:0/1360981563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd030260 0x55c7fd035920 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/1360981563 >> 172.21.15.157:0/1360981563 conn(0x55c7fd0369f0 msgr2=0x55c7fd087890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/1360981563 shutdown_connections 2023-11-06T03:18:42.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/1360981563 wait complete. 2023-11-06T03:18:42.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- start start 2023-11-06T03:18:42.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 Processor -- start 2023-11-06T03:18:42.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd030260 0x55c7fd0ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07aa50 0x55c7fd0bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fd090d70 con 0x55c7fd07a130 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce7e130 con 0x55c7fd030260 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fce79640 con 0x55c7fd07aa50 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 10 client.0 _ll_register_callbacks cb 0x55c7fce43cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38676/0 (socket says 172.21.15.157:38676) 2023-11-06T03:18:42.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 -- 172.21.15.157:0/51665825 learned_addr learned my addr 172.21.15.157:0/51665825 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:18:42.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670e4ba640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd030260 0x55c7fd0ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670ecbb640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07aa50 0x55c7fd0bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 -- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07aa50 msgr2=0x55c7fd0bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07aa50 0x55c7fd0bb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 -- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd030260 msgr2=0x55c7fd0ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:18:42.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd030260 0x55c7fd0ba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 -- 172.21.15.157:0/51665825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7fcb7be20 con 0x55c7fd07a130 2023-11-06T03:18:42.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670dcb9640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f670401d050 tx=0x7f6704004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d496ba8b950ab960 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:18:42.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:18:42.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040f2050 con 0x55c7fd07a130 2023-11-06T03:18:42.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(190..190 src has 1..190) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f67040f6200 con 0x55c7fd07a130 2023-11-06T03:18:42.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7fce48620 con 0x55c7fd07a130 2023-11-06T03:18:42.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67040ec5c0 con 0x55c7fd07a130 2023-11-06T03:18:42.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 10 client.11595 fetch_fsmap learned FSMap version 136 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55c7fce1e090 con 0x55c7fd07a130 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7fce48620 con 0x55c7fd07a130 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f67040bf060 con 0x55c7fd07a130 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 10 client.11595 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 10 client.11595 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:18:42.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 20 client.11595 populate_metadata read hostname 'smithi157' 2023-11-06T03:18:42.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd08e6f0 0x55c7fd0bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:18:42.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 4 client.11595 mds_command: new command op to 11487 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:18:42.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7fce78320 con 0x55c7fd08e6f0 2023-11-06T03:18:42.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.770+0000 7f670e4ba640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd08e6f0 0x55c7fd0bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:18:42.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f66e97fa640 10 client.11595 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f670e4ba640 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd08e6f0 0x55c7fd0bb920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0117808d57d15fd server_cookie=c1ce97073d03944f in_seq=0 out_seq=0 2023-11-06T03:18:42.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f66e97fa640 1 -- 172.21.15.157:0/51665825 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c7fce78320 con 0x55c7fd08e6f0 2023-11-06T03:18:42.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f66e97fa640 10 client.11595 handle_command_reply: tid=0 2023-11-06T03:18:42.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 client.11595 shutdown 2023-11-06T03:18:42.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 msgr2=0x55c7fd0baea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f670401d050 tx=0x7f6704004e20 comp rx=0 tx=0).stop 2023-11-06T03:18:42.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 shutdown_connections 2023-11-06T03:18:42.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55c7fd08e6f0 0x55c7fd0bb920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55c7fd07aa50 0x55c7fd0bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c7fd07a130 0x55c7fd0baea0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 --2- 172.21.15.157:0/51665825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7fd030260 0x55c7fd0ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 >> 172.21.15.157:0/51665825 conn(0x55c7fd0369f0 msgr2=0x55c7fd0351a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f66e97fa640 0 client.11595 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:18:42.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 shutdown_connections 2023-11-06T03:18:42.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 1 -- 172.21.15.157:0/51665825 wait complete. 2023-11-06T03:18:42.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f6713846000 20 client.11595 trim_cache size 0 max 16384 2023-11-06T03:18:42.796 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:18:42.796 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:18:42.796 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:18:42.797 INFO:teuthology.orchestra.run.smithi157.stdout: "7a0bf5b2-a91f-46c3-9ee2-c756fbb09e2b": { 2023-11-06T03:18:42.797 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/", 2023-11-06T03:18:42.797 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "7a0bf5b2-a91f-46c3-9ee2-c756fbb09e2b", 2023-11-06T03:18:42.797 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive,repair" 2023-11-06T03:18:42.797 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:42.798 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:18:42.798 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:18:42.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 msgr2=0x7f66ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 0x7f66ec067f20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c10 tx=0x7f66f8009040 comp rx=0 tx=0).stop 2023-11-06T03:18:42.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 msgr2=0x7f67081f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:18:42.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f67040ab040 tx=0x7f66f4005800 comp rx=0 tx=0).stop 2023-11-06T03:18:42.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 shutdown_connections 2023-11-06T03:18:42.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66ec065a60 0x7f66ec067f20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f67080ccb60 0x7f67080ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f670814a8b0 0x7f67081f3f10 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 --2- 172.21.15.157:0/282167413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66f40047d0 0x7f670814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 >> 172.21.15.157:0/282167413 conn(0x7f6708065930 msgr2=0x7f67080b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 shutdown_connections 2023-11-06T03:18:42.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:18:42.774+0000 7f671071e640 1 -- 172.21.15.157:0/282167413 wait complete. 2023-11-06T03:18:42.802 INFO:tasks.cephfs.filesystem:scrub status for tag:7a0bf5b2-a91f-46c3-9ee2-c756fbb09e2b - {'path': '/', 'tag': '7a0bf5b2-a91f-46c3-9ee2-c756fbb09e2b', 'options': 'recursive,repair'} 2023-11-06T03:18:42.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:18:42.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:18:42.932 INFO:tasks.ceph.mds.a.smithi157.stderr:2023-11-06T03:18:42.934+0000 7f5a89db4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:19:12.803 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub status 2023-11-06T03:19:12.904 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:19:12.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:19:12.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 Processor -- start 2023-11-06T03:19:12.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 -- start start 2023-11-06T03:19:12.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f007c15caa0 0x7f007c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f007c0d0280 0x7f007c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c0d2920 con 0x7f007c15ce80 2023-11-06T03:19:12.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c0d18a0 con 0x7f007c15caa0 2023-11-06T03:19:12.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0083785640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c0d1b60 con 0x7f007c0d0280 2023-11-06T03:19:12.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081521640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f007c15caa0 0x7f007c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0080d20640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f007c15ce80 0x7f007c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47414/0 (socket says 172.21.15.157:47414) 2023-11-06T03:19:12.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0080d20640 1 -- 172.21.15.157:0/2063201550 learned_addr learned my addr 172.21.15.157:0/2063201550 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:12.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081d22640 1 --2- 172.21.15.157:0/2063201550 >> v2:172.21.15.195:3300/0 conn(0x7f007c0d0280 0x7f007c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0073fff640 1 -- 172.21.15.157:0/2063201550 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894561288 0 0) 0x7f007c0d2920 con 0x7f007c15ce80 2023-11-06T03:19:12.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0073fff640 1 -- 172.21.15.157:0/2063201550 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0068003530 con 0x7f007c15ce80 2023-11-06T03:19:12.949 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081521640 1 -- 172.21.15.157:0/2063201550 >> v2:172.21.15.195:3300/0 conn(0x7f007c0d0280 msgr2=0x7f007c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081521640 1 --2- 172.21.15.157:0/2063201550 >> v2:172.21.15.195:3300/0 conn(0x7f007c0d0280 0x7f007c0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081521640 1 -- 172.21.15.157:0/2063201550 >> v1:172.21.15.157:6789/0 conn(0x7f007c15ce80 legacy=0x7f007c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.950 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.949+0000 7f0081521640 1 -- 172.21.15.157:0/2063201550 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0d1e40 con 0x7f007c15caa0 2023-11-06T03:19:12.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0081521640 1 --2- 172.21.15.157:0/2063201550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15caa0 0x7f007c0cdc60 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f007c0cf670 tx=0x7f007801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cb9e4168d7ee87 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:12.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0073fff640 1 -- 172.21.15.157:0/2063201550 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a7070 con 0x7f007c15caa0 2023-11-06T03:19:12.951 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0073fff640 1 -- 172.21.15.157:0/2063201550 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780ab430 con 0x7f007c15caa0 2023-11-06T03:19:12.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0073fff640 1 -- 172.21.15.157:0/2063201550 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780ab7c0 con 0x7f007c15caa0 2023-11-06T03:19:12.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 -- 172.21.15.157:0/2063201550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15caa0 msgr2=0x7f007c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 --2- 172.21.15.157:0/2063201550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15caa0 0x7f007c0cdc60 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f007c0cf670 tx=0x7f007801c430 comp rx=0 tx=0).stop 2023-11-06T03:19:12.952 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 -- 172.21.15.157:0/2063201550 shutdown_connections 2023-11-06T03:19:12.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 --2- 172.21.15.157:0/2063201550 >> v2:172.21.15.195:3300/0 conn(0x7f007c0d0280 0x7f007c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 --2- 172.21.15.157:0/2063201550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15caa0 0x7f007c0cdc60 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 -- 172.21.15.157:0/2063201550 >> 172.21.15.157:0/2063201550 conn(0x7f007c070790 msgr2=0x7f007c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:12.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 -- 172.21.15.157:0/2063201550 shutdown_connections 2023-11-06T03:19:12.953 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.953+0000 7f0083785640 1 -- 172.21.15.157:0/2063201550 wait complete. 2023-11-06T03:19:12.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 Processor -- start 2023-11-06T03:19:12.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- start start 2023-11-06T03:19:12.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f007c0d0280 0x7f007c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f007c15caa0 0x7f007c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c1554c0 con 0x7f007c15ce80 2023-11-06T03:19:12.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c15be30 con 0x7f007c0d0280 2023-11-06T03:19:12.966 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0c9e20 con 0x7f007c15caa0 2023-11-06T03:19:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43536/0 (socket says 172.21.15.157:43536) 2023-11-06T03:19:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 -- 172.21.15.157:0/619838654 learned_addr learned my addr 172.21.15.157:0/619838654 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:12.967 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 -- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f007c15caa0 msgr2=0x7f007c0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0080d20640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f007c15caa0 0x7f007c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081521640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f007c0d0280 0x7f007c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f007c15caa0 0x7f007c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 -- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f007c0d0280 msgr2=0x7f007c154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f007c0d0280 0x7f007c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 -- 172.21.15.157:0/619838654 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078009420 con 0x7f007c15ce80 2023-11-06T03:19:12.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0081d22640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f007400efc0 tx=0x7f007400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:12.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0071ffb640 1 -- 172.21.15.157:0/619838654 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074009ba0 con 0x7f007c15ce80 2023-11-06T03:19:12.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0071ffb640 1 -- 172.21.15.157:0/619838654 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007400f040 con 0x7f007c15ce80 2023-11-06T03:19:12.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c0ca100 con 0x7f007c15ce80 2023-11-06T03:19:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0071ffb640 1 -- 172.21.15.157:0/619838654 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740146c0 con 0x7f007c15ce80 2023-11-06T03:19:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0ca6c0 con 0x7f007c15ce80 2023-11-06T03:19:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.957+0000 7f0071ffb640 1 -- 172.21.15.157:0/619838654 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0074014820 con 0x7f007c15ce80 2023-11-06T03:19:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0071ffb640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 0x7f0060067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0071ffb640 1 -- 172.21.15.157:0/619838654 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f00740840f0 con 0x7f007c15ce80 2023-11-06T03:19:12.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 0x7f0060067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 Processor -- start 2023-11-06T03:19:12.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 0x7f0060067f20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f00780af000 tx=0x7f0078008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:12.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- start start 2023-11-06T03:19:12.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7a50 0x5573874acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x5573874ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557387374c10 con 0x5573874f7130 2023-11-06T03:19:12.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573873033e0 con 0x5573874ad260 2023-11-06T03:19:12.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573873009b0 con 0x5573874f7a50 2023-11-06T03:19:12.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43550/0 (socket says 172.21.15.157:43550) 2023-11-06T03:19:12.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 -- 172.21.15.157:0/882587832 learned_addr learned my addr 172.21.15.157:0/882587832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:12.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081d22640 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x5573874ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0080d20640 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7a50 0x5573874acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 -- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7a50 msgr2=0x5573874acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7a50 0x5573874acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 -- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 msgr2=0x5573874ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x5573874ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 -- 172.21.15.157:0/882587832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557386ff8e20 con 0x5573874f7130 2023-11-06T03:19:12.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0081521640 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f0078019040 tx=0x7f00780a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e5e36bcbd4bddb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:12.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0066ffd640 1 -- 172.21.15.157:0/882587832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780b5ad0 con 0x5573874f7130 2023-11-06T03:19:12.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0066ffd640 1 -- 172.21.15.157:0/882587832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780bd040 con 0x5573874f7130 2023-11-06T03:19:12.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f0066ffd640 1 -- 172.21.15.157:0/882587832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780b5e30 con 0x5573874f7130 2023-11-06T03:19:12.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- 172.21.15.157:0/882587832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 msgr2=0x5573874f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f0078019040 tx=0x7f00780a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:12.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- 172.21.15.157:0/882587832 shutdown_connections 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x5573874ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7a50 0x5573874acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 --2- 172.21.15.157:0/882587832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874f7510 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- 172.21.15.157:0/882587832 >> 172.21.15.157:0/882587832 conn(0x5573874b39f0 msgr2=0x5573874ab600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- 172.21.15.157:0/882587832 shutdown_connections 2023-11-06T03:19:12.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.961+0000 7f00868ad000 1 -- 172.21.15.157:0/882587832 wait complete. 2023-11-06T03:19:12.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- start start 2023-11-06T03:19:12.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 Processor -- start 2023-11-06T03:19:12.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874ad260 0x5573874fe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573872fc2a0 con 0x5573874f7130 2023-11-06T03:19:12.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738729b470 con 0x5573874f7a50 2023-11-06T03:19:12.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573872b9660 con 0x5573874ad260 2023-11-06T03:19:12.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 10 client.0 _ll_register_callbacks cb 0x557387498c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:12.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43556/0 (socket says 172.21.15.157:43556) 2023-11-06T03:19:12.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 -- 172.21.15.157:0/4073690359 learned_addr learned my addr 172.21.15.157:0/4073690359 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:12.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0081d22640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0081521640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874ad260 0x5573874fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 -- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874ad260 msgr2=0x5573874fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874ad260 0x5573874fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 -- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 msgr2=0x5573874f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 -- 172.21.15.157:0/4073690359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0d1e40 con 0x5573874f7130 2023-11-06T03:19:12.981 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0080d20640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f006c001dc0 tx=0x7f006c00e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6bc8c172e473e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:12.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:12.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c005a60 con 0x5573874f7130 2023-11-06T03:19:12.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f006c005bc0 con 0x5573874f7130 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573872b90a0 con 0x5573874f7130 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c007db0 con 0x5573874f7130 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 10 client.11610 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x5573872c11d0 con 0x5573874f7130 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573872b90a0 con 0x5573874f7130 2023-11-06T03:19:12.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f006c0070d0 con 0x5573874f7130 2023-11-06T03:19:12.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 10 client.11610 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:12.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 10 client.11610 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:12.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 20 client.11610 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:12.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x5573874f8ba0 0x5573874f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.984 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 4 client.11610 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:19:12.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573872fabf0 con 0x5573874f8ba0 2023-11-06T03:19:12.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0081521640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x5573874f8ba0 0x5573874f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 10 client.11610 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:12.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f0081521640 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x5573874f8ba0 0x5573874f8fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d2a41644debe3e server_cookie=a19f2583283593d7 in_seq=0 out_seq=0 2023-11-06T03:19:12.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 1 -- 172.21.15.157:0/4073690359 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5573872fabf0 con 0x5573874f8ba0 2023-11-06T03:19:12.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 10 client.11610 handle_command_reply: tid=0 2023-11-06T03:19:12.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 client.11610 shutdown 2023-11-06T03:19:12.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 msgr2=0x5573874feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:12.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f006c001dc0 tx=0x7f006c00e590 comp rx=0 tx=0).stop 2023-11-06T03:19:12.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 shutdown_connections 2023-11-06T03:19:12.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x5573874f8ba0 0x5573874f8fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7130 0x5573874feb90 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f005bfff640 0 client.11610 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:12.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 --2- 172.21.15.157:0/4073690359 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874ad260 0x5573874fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.965+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 >> 172.21.15.157:0/4073690359 conn(0x5573874b39f0 msgr2=0x5573874abd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:12.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.969+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 shutdown_connections 2023-11-06T03:19:12.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.969+0000 7f00868ad000 1 -- 172.21.15.157:0/4073690359 wait complete. 2023-11-06T03:19:12.988 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.969+0000 7f00868ad000 20 client.11610 trim_cache size 0 max 16384 2023-11-06T03:19:12.995 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 Processor -- start 2023-11-06T03:19:12.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 -- start start 2023-11-06T03:19:12.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573874fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.996 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:12.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557386fb9aa0 con 0x5573874ad260 2023-11-06T03:19:12.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573872f7c10 con 0x5573874f7a50 2023-11-06T03:19:12.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573873002a0 con 0x5573874f7130 2023-11-06T03:19:12.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.997 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43570/0 (socket says 172.21.15.157:43570) 2023-11-06T03:19:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 -- 172.21.15.157:0/1934701827 learned_addr learned my addr 172.21.15.157:0/1934701827 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0080d20640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573874fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.998 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081d22640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 -- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 msgr2=0x5573874fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573874fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 -- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 msgr2=0x5573874fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:12.999 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 -- 172.21.15.157:0/1934701827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573872a7520 con 0x5573874ad260 2023-11-06T03:19:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081d22640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.981+0000 7f0081521640 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f00780c3040 tx=0x7f0078009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac20f1029cf4060 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00677fe640 1 -- 172.21.15.157:0/1934701827 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078004300 con 0x5573874ad260 2023-11-06T03:19:13.000 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00677fe640 1 -- 172.21.15.157:0/1934701827 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780bd040 con 0x5573874ad260 2023-11-06T03:19:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00677fe640 1 -- 172.21.15.157:0/1934701827 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780b58d0 con 0x5573874ad260 2023-11-06T03:19:13.001 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/1934701827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 msgr2=0x5573874fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f00780c3040 tx=0x7f0078009190 comp rx=0 tx=0).stop 2023-11-06T03:19:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/1934701827 shutdown_connections 2023-11-06T03:19:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874f7a50 0x5573874fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.002 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573874fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- 172.21.15.157:0/1934701827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874ad260 0x5573874fbfa0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/1934701827 >> 172.21.15.157:0/1934701827 conn(0x5573874b39f0 msgr2=0x5573874b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/1934701827 shutdown_connections 2023-11-06T03:19:13.003 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/1934701827 wait complete. 2023-11-06T03:19:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- start start 2023-11-06T03:19:13.004 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 Processor -- start 2023-11-06T03:19:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x557387553b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573875540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.005 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573872ba400 con 0x5573874f7a50 2023-11-06T03:19:13.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738737cc20 con 0x5573874ad260 2023-11-06T03:19:13.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573872838c0 con 0x5573874f7130 2023-11-06T03:19:13.006 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 10 client.0 _ll_register_callbacks cb 0x5573872c0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:13.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43572/0 (socket says 172.21.15.157:43572) 2023-11-06T03:19:13.007 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 -- 172.21.15.157:0/160714392 learned_addr learned my addr 172.21.15.157:0/160714392 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0080d20640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573875540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081521640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x557387553b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 -- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 msgr2=0x5573875540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.008 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573875540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 -- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 msgr2=0x557387553b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x557387553b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.009 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 -- 172.21.15.157:0/160714392 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557386ff8e20 con 0x5573874f7a50 2023-11-06T03:19:13.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081d22640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f00740552f0 tx=0x7f00740594e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad76ac1bde7e854 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:13.010 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074054ce0 con 0x5573874f7a50 2023-11-06T03:19:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f0074053a50 con 0x5573874f7a50 2023-11-06T03:19:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573872c5620 con 0x5573874f7a50 2023-11-06T03:19:13.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f007405d140 con 0x5573874f7a50 2023-11-06T03:19:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 10 client.11616 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55738729b090 con 0x5573874f7a50 2023-11-06T03:19:13.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573872c5620 con 0x5573874f7a50 2023-11-06T03:19:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0074024020 con 0x5573874f7a50 2023-11-06T03:19:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 10 client.11616 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 10 client.11616 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:13.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 20 client.11616 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x557387537f60 0x557387538370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 4 client.11616 mds_command: new command op to 11487 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:19:13.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573872f5320 con 0x557387537f60 2023-11-06T03:19:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081521640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x557387537f60 0x557387538370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f005bfff640 10 client.11616 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.985+0000 7f0081521640 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x557387537f60 0x557387538370 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278371972f1fc67a server_cookie=5abdcb59d9ed39b1 in_seq=0 out_seq=0 2023-11-06T03:19:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f005bfff640 1 -- 172.21.15.157:0/160714392 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5573872f5320 con 0x557387537f60 2023-11-06T03:19:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f005bfff640 10 client.11616 handle_command_reply: tid=0 2023-11-06T03:19:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 client.11616 shutdown 2023-11-06T03:19:13.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 msgr2=0x557387537a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f00740552f0 tx=0x7f00740594e0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 shutdown_connections 2023-11-06T03:19:13.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x557387537f60 0x557387538370 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5573874f7a50 0x557387537a20 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x5573874f7130 0x5573875540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 --2- 172.21.15.157:0/160714392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5573874ad260 0x557387553b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 >> 172.21.15.157:0/160714392 conn(0x5573874b39f0 msgr2=0x5573874faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f005bfff640 0 client.11616 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 shutdown_connections 2023-11-06T03:19:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 1 -- 172.21.15.157:0/160714392 wait complete. 2023-11-06T03:19:13.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f00868ad000 20 client.11616 trim_cache size 0 max 16384 2023-11-06T03:19:13.020 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:13.021 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:19:13.021 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:19:13.022 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 msgr2=0x7f0060067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 0x7f0060067f20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f00780af000 tx=0x7f0078008040 comp rx=0 tx=0).stop 2023-11-06T03:19:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 msgr2=0x7f007c15b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f007400efc0 tx=0x7f007400c490 comp rx=0 tx=0).stop 2023-11-06T03:19:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 shutdown_connections 2023-11-06T03:19:13.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0060065a60 0x7f0060067f20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f007c15ce80 0x7f007c15b3e0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f007c15caa0 0x7f007c0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 --2- 172.21.15.157:0/619838654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f007c0d0280 0x7f007c154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 >> 172.21.15.157:0/619838654 conn(0x7f007c070790 msgr2=0x7f007c0cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 shutdown_connections 2023-11-06T03:19:13.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:12.989+0000 7f0083785640 1 -- 172.21.15.157:0/619838654 wait complete. 2023-11-06T03:19:13.027 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:19:13.028 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub start / recursive,repair 2023-11-06T03:19:13.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 Processor -- start 2023-11-06T03:19:13.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 -- start start 2023-11-06T03:19:13.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c170 0x7fb9d8154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9d815b860 0x7fb9d8155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80cb100 con 0x7fb9d806c550 2023-11-06T03:19:13.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d815bcf0 con 0x7fb9d815b860 2023-11-06T03:19:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9df33f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80ce0a0 con 0x7fb9d806c170 2023-11-06T03:19:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9d815b860 0x7fb9d8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9d815b860 0x7fb9d8155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43576/0 (socket says 172.21.15.157:43576) 2023-11-06T03:19:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2037538877 learned_addr learned my addr 172.21.15.157:0/2037538877 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c170 0x7fb9d8154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2037538877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c170 msgr2=0x7fb9d8154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c170 0x7fb9d8154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2037538877 >> v1:172.21.15.195:6789/0 conn(0x7fb9d806c550 legacy=0x7fb9d815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.209+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2037538877 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d815bef0 con 0x7fb9d815b860 2023-11-06T03:19:13.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8155140 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009e50 tx=0x7fb9d001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5c8f965a3df1f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9cffff640 1 -- 172.21.15.157:0/2037538877 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00a58a0 con 0x7fb9d815b860 2023-11-06T03:19:13.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9cffff640 1 -- 172.21.15.157:0/2037538877 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00a5a00 con 0x7fb9d815b860 2023-11-06T03:19:13.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9cffff640 1 -- 172.21.15.157:0/2037538877 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00a5d90 con 0x7fb9d815b860 2023-11-06T03:19:13.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- 172.21.15.157:0/2037538877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 msgr2=0x7fb9d8155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8155140 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009e50 tx=0x7fb9d001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- 172.21.15.157:0/2037538877 shutdown_connections 2023-11-06T03:19:13.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8155140 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.210 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2037538877 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c170 0x7fb9d8154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- 172.21.15.157:0/2037538877 >> 172.21.15.157:0/2037538877 conn(0x7fb9d80b43b0 msgr2=0x7fb9d8065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- 172.21.15.157:0/2037538877 shutdown_connections 2023-11-06T03:19:13.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- 172.21.15.157:0/2037538877 wait complete. 2023-11-06T03:19:13.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 Processor -- start 2023-11-06T03:19:13.211 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.213+0000 7fb9df33f640 1 -- start start 2023-11-06T03:19:13.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9d806c170 0x7fb9d80aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c550 0x7fb9d80a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80a91d0 con 0x7fb9d815b860 2023-11-06T03:19:13.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8157910 con 0x7fb9d806c170 2023-11-06T03:19:13.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80aab20 con 0x7fb9d806c550 2023-11-06T03:19:13.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43582/0 (socket says 172.21.15.157:43582) 2023-11-06T03:19:13.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2805497007 learned_addr learned my addr 172.21.15.157:0/2805497007 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c550 msgr2=0x7fb9d80a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c550 0x7fb9d80a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9d806c170 msgr2=0x7fb9d80aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9d806c170 0x7fb9d80aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/2805497007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009b30 con 0x7fb9d815b860 2023-11-06T03:19:13.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fb9d001a610 tx=0x7fb9d0019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 -- 172.21.15.157:0/2805497007 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00b8680 con 0x7fb9d815b860 2023-11-06T03:19:13.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 -- 172.21.15.157:0/2805497007 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00b87e0 con 0x7fb9d815b860 2023-11-06T03:19:13.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80cbfb0 con 0x7fb9d815b860 2023-11-06T03:19:13.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 -- 172.21.15.157:0/2805497007 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00b8b90 con 0x7fb9d815b860 2023-11-06T03:19:13.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80a8fa0 con 0x7fb9d815b860 2023-11-06T03:19:13.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 -- 172.21.15.157:0/2805497007 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb9d00b8cf0 con 0x7fb9d815b860 2023-11-06T03:19:13.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 0x7fb9bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9cdffb640 1 -- 172.21.15.157:0/2805497007 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb9d011eaa0 con 0x7fb9d815b860 2023-11-06T03:19:13.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.217+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 0x7fb9bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 Processor -- start 2023-11-06T03:19:13.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- start start 2023-11-06T03:19:13.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 0x7fb9bc067f20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb819128f0 0x55cb81912cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 0x55cb818c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8178ca90 con 0x55cb818c8550 2023-11-06T03:19:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8171a490 con 0x55cb81913210 2023-11-06T03:19:13.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb81717a60 con 0x55cb819128f0 2023-11-06T03:19:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43590/0 (socket says 172.21.15.157:43590) 2023-11-06T03:19:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/288352798 learned_addr learned my addr 172.21.15.157:0/288352798 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb819128f0 0x55cb81912cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 0x55cb818c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb819128f0 msgr2=0x55cb81912cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb819128f0 0x55cb81912cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 msgr2=0x55cb818c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 0x55cb818c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/288352798 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb81410070 con 0x55cb818c8550 2023-11-06T03:19:13.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 0x55cb818c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00efc90 tx=0x7fb9d00a5990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cde9e34438d06e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9c2ffd640 1 -- 172.21.15.157:0/288352798 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00f7f70 con 0x55cb818c8550 2023-11-06T03:19:13.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9c2ffd640 1 -- 172.21.15.157:0/288352798 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00ee330 con 0x55cb818c8550 2023-11-06T03:19:13.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9c2ffd640 1 -- 172.21.15.157:0/288352798 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00f01f0 con 0x55cb818c8550 2023-11-06T03:19:13.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- 172.21.15.157:0/288352798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 msgr2=0x55cb818c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00efc90 tx=0x7fb9d00a5990 comp rx=0 tx=0).stop 2023-11-06T03:19:13.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- 172.21.15.157:0/288352798 shutdown_connections 2023-11-06T03:19:13.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb818c8550 0x55cb818c8930 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb81913210 0x55cb818c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- 172.21.15.157:0/288352798 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb819128f0 0x55cb81912cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- 172.21.15.157:0/288352798 >> 172.21.15.157:0/288352798 conn(0x55cb818bbbc0 msgr2=0x55cb818cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- 172.21.15.157:0/288352798 shutdown_connections 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- 172.21.15.157:0/288352798 wait complete. 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- start start 2023-11-06T03:19:13.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 Processor -- start 2023-11-06T03:19:13.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb818c8550 0x55cb818ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 0x55cb819001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb81713350 con 0x55cb81913210 2023-11-06T03:19:13.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb816b2610 con 0x55cb819128f0 2023-11-06T03:19:13.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb816d0800 con 0x55cb818c8550 2023-11-06T03:19:13.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9e2467000 10 client.0 _ll_register_callbacks cb 0x55cb81708960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:13.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.221+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43606/0 (socket says 172.21.15.157:43606) 2023-11-06T03:19:13.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/33844657 learned_addr learned my addr 172.21.15.157:0/33844657 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 0x55cb819001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb818c8550 0x55cb818ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb818c8550 msgr2=0x55cb818ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb818c8550 0x55cb818ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 msgr2=0x55cb819001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 0x55cb819001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 -- 172.21.15.157:0/33844657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80a8fa0 con 0x55cb81913210 2023-11-06T03:19:13.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 0x55cb819001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:19:13.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00f0150 tx=0x7fb9d001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee77261f66f6ea59 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:13.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d011c070 con 0x55cb81913210 2023-11-06T03:19:13.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb9d00ef680 con 0x55cb81913210 2023-11-06T03:19:13.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb816e2c80 con 0x55cb81913210 2023-11-06T03:19:13.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d00f01f0 con 0x55cb81913210 2023-11-06T03:19:13.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 10 client.11628 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:13.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55cb816d82d0 con 0x55cb81913210 2023-11-06T03:19:13.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb816e2c80 con 0x55cb81913210 2023-11-06T03:19:13.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb9d0005d20 con 0x55cb81913210 2023-11-06T03:19:13.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 10 client.11628 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:13.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 10 client.11628 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:13.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 20 client.11628 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:13.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81901140 0x55cb81923990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 4 client.11628 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:19:13.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb81711ca0 con 0x55cb81901140 2023-11-06T03:19:13.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81901140 0x55cb81923990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 10 client.11628 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81901140 0x55cb81923990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e1b8c75ed048f3 server_cookie=4db41b7e2022c2b9 in_seq=0 out_seq=0 2023-11-06T03:19:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 1 -- 172.21.15.157:0/33844657 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cb81711ca0 con 0x55cb81901140 2023-11-06T03:19:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 10 client.11628 handle_command_reply: tid=0 2023-11-06T03:19:13.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 client.11628 shutdown 2023-11-06T03:19:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 msgr2=0x55cb81900730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00f0150 tx=0x7fb9d001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 shutdown_connections 2023-11-06T03:19:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81901140 0x55cb81923990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb81913210 0x55cb81900730 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb819128f0 0x55cb819001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9b7fff640 0 client.11628 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 --2- 172.21.15.157:0/33844657 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb818c8550 0x55cb818ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 >> 172.21.15.157:0/33844657 conn(0x55cb818bbbc0 msgr2=0x55cb818c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 shutdown_connections 2023-11-06T03:19:13.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 1 -- 172.21.15.157:0/33844657 wait complete. 2023-11-06T03:19:13.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.225+0000 7fb9e2467000 20 client.11628 trim_cache size 0 max 16384 2023-11-06T03:19:13.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 Processor -- start 2023-11-06T03:19:13.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- start start 2023-11-06T03:19:13.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb818fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb818fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb813d0cf0 con 0x55cb819128f0 2023-11-06T03:19:13.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb816da440 con 0x55cb818c8550 2023-11-06T03:19:13.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb816e28e0 con 0x55cb81913210 2023-11-06T03:19:13.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43616/0 (socket says 172.21.15.157:43616) 2023-11-06T03:19:13.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 -- 172.21.15.157:0/384482848 learned_addr learned my addr 172.21.15.157:0/384482848 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb818fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb818fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 -- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 msgr2=0x55cb818fdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb818fdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 -- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 msgr2=0x55cb818fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb818fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 -- 172.21.15.157:0/384482848 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb816be620 con 0x55cb819128f0 2023-11-06T03:19:13.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb818fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001480 tx=0x7fb9d4001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04430b0d8a46ec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9c37fe640 1 -- 172.21.15.157:0/384482848 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40052e0 con 0x55cb819128f0 2023-11-06T03:19:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9c37fe640 1 -- 172.21.15.157:0/384482848 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d400d050 con 0x55cb819128f0 2023-11-06T03:19:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9c37fe640 1 -- 172.21.15.157:0/384482848 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4003600 con 0x55cb819128f0 2023-11-06T03:19:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- 172.21.15.157:0/384482848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 msgr2=0x55cb818fd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001480 tx=0x7fb9d4001940 comp rx=0 tx=0).stop 2023-11-06T03:19:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- 172.21.15.157:0/384482848 shutdown_connections 2023-11-06T03:19:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb818fdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb818fd9d0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 --2- 172.21.15.157:0/384482848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb818fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.256 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- 172.21.15.157:0/384482848 >> 172.21.15.157:0/384482848 conn(0x55cb818bbbc0 msgr2=0x55cb81911e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- 172.21.15.157:0/384482848 shutdown_connections 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- 172.21.15.157:0/384482848 wait complete. 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 -- start start 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.241+0000 7fb9e2467000 1 Processor -- start 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb8196f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb819240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb816c2490 con 0x55cb819128f0 2023-11-06T03:19:13.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb816e1fc0 con 0x55cb818c8550 2023-11-06T03:19:13.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb8170d6f0 con 0x55cb81913210 2023-11-06T03:19:13.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 10 client.0 _ll_register_callbacks cb 0x55cb818909d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:13.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43622/0 (socket says 172.21.15.157:43622) 2023-11-06T03:19:13.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 -- 172.21.15.157:0/3142793747 learned_addr learned my addr 172.21.15.157:0/3142793747 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dd8dc640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb819240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb8196f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 -- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 msgr2=0x55cb819240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb819240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 -- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 msgr2=0x55cb8196f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb8196f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 -- 172.21.15.157:0/3142793747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80cb100 con 0x55cb819128f0 2023-11-06T03:19:13.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb8196f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dc8da640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001980 tx=0x7fb9d4001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a440174e47ebbd server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:13.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4007230 con 0x55cb819128f0 2023-11-06T03:19:13.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb9d4007390 con 0x55cb819128f0 2023-11-06T03:19:13.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb818d0ed0 con 0x55cb819128f0 2023-11-06T03:19:13.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d4004e70 con 0x55cb819128f0 2023-11-06T03:19:13.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 10 client.11634 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55cb816b2190 con 0x55cb819128f0 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb818d0ed0 con 0x55cb819128f0 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb9d4012070 con 0x55cb819128f0 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 10 client.11634 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 10 client.11634 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:13.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 20 client.11634 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:13.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81924dc0 0x55cb819251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 4 client.11634 mds_command: new command op to 11487 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2023-11-06T03:19:13.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55cb816c2d50 con 0x55cb81924dc0 2023-11-06T03:19:13.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81924dc0 0x55cb819251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9dd0db640 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81924dc0 0x55cb819251d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612696c1d44e4454 server_cookie=220453f40c36c6b9 in_seq=0 out_seq=0 2023-11-06T03:19:13.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 10 client.11634 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 1 -- 172.21.15.157:0/3142793747 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cb816c2d50 con 0x55cb81924dc0 2023-11-06T03:19:13.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 10 client.11634 handle_command_reply: tid=0 2023-11-06T03:19:13.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 client.11634 shutdown 2023-11-06T03:19:13.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 msgr2=0x55cb8196fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001980 tx=0x7fb9d4001130 comp rx=0 tx=0).stop 2023-11-06T03:19:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 shutdown_connections 2023-11-06T03:19:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55cb81924dc0 0x55cb819251d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55cb81913210 0x55cb819240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cb819128f0 0x55cb8196fde0 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9b7fff640 0 client.11634 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 --2- 172.21.15.157:0/3142793747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cb818c8550 0x55cb8196f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 >> 172.21.15.157:0/3142793747 conn(0x55cb818bbbc0 msgr2=0x55cb818fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 shutdown_connections 2023-11-06T03:19:13.267 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 1 -- 172.21.15.157:0/3142793747 wait complete. 2023-11-06T03:19:13.268 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9e2467000 20 client.11634 trim_cache size 0 max 16384 2023-11-06T03:19:13.268 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:13.268 INFO:teuthology.orchestra.run.smithi157.stdout: "return_code": 0, 2023-11-06T03:19:13.269 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_tag": "b5aeba5f-d7b0-4b27-8d77-e04c69667dd1", 2023-11-06T03:19:13.269 INFO:teuthology.orchestra.run.smithi157.stdout: "mode": "asynchronous" 2023-11-06T03:19:13.269 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:13.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 msgr2=0x7fb9bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 0x7fb9bc067f20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8008040 comp rx=0 tx=0).stop 2023-11-06T03:19:13.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 msgr2=0x7fb9d8151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fb9d001a610 tx=0x7fb9d0019780 comp rx=0 tx=0).stop 2023-11-06T03:19:13.271 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 shutdown_connections 2023-11-06T03:19:13.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb9bc065a60 0x7fb9bc067f20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9d815b860 0x7fb9d8151250 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb9d806c550 0x7fb9d80a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 --2- 172.21.15.157:0/2805497007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9d806c170 0x7fb9d80aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.272 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.245+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 >> 172.21.15.157:0/2805497007 conn(0x7fb9d80b43b0 msgr2=0x7fb9d80ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.249+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 shutdown_connections 2023-11-06T03:19:13.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.249+0000 7fb9df33f640 1 -- 172.21.15.157:0/2805497007 wait complete. 2023-11-06T03:19:13.273 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub status 2023-11-06T03:19:13.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.397+0000 7fb25e4a9640 1 Processor -- start 2023-11-06T03:19:13.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.397+0000 7fb25e4a9640 1 -- start start 2023-11-06T03:19:13.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25e4a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb2580ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25e4a9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25815e440 con 0x7fb2580b2af0 2023-11-06T03:19:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25e4a9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25806c240 con 0x7fb2581599c0 2023-11-06T03:19:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25e4a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806c520 con 0x7fb25806d000 2023-11-06T03:19:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb2577fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb2580b2af0 0x7fb258159da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:47422/0 (socket says 172.21.15.157:47422) 2023-11-06T03:19:13.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb2577fe640 1 -- 172.21.15.157:0/3031085580 learned_addr learned my addr 172.21.15.157:0/3031085580 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:13.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25ca46640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb2580ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376242745 0 0) 0x7fb25815e440 con 0x7fb2580b2af0 2023-11-06T03:19:13.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb240003530 con 0x7fb2580b2af0 2023-11-06T03:19:13.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586440883 0 0) 0x7fb240003530 con 0x7fb2580b2af0 2023-11-06T03:19:13.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 msgr2=0x7fb2580ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb2580ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 >> v1:172.21.15.195:6789/0 conn(0x7fb2581599c0 legacy=0x7fb2580b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25806c800 con 0x7fb2580b2af0 2023-11-06T03:19:13.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fb24c001680 con 0x7fb2580b2af0 2023-11-06T03:19:13.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 >> v1:172.21.15.157:6789/0 conn(0x7fb2580b2af0 legacy=0x7fb258159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2400039b0 0x7fb240003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2400047d0 0x7fb240004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb240005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806c520 con 0x7fb2400039b0 2023-11-06T03:19:13.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24000ba40 con 0x7fb2400047d0 2023-11-06T03:19:13.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24000bc00 con 0x7fb25806d000 2023-11-06T03:19:13.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2400039b0 0x7fb240003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb2577fe640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2400047d0 0x7fb240004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb25ca46640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb240005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 -- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 msgr2=0x7fb240005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb25806d000 0x7fb240005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 -- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2400047d0 msgr2=0x7fb240004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2400047d0 0x7fb240004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 -- 172.21.15.157:0/3031085580 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb240003530 con 0x7fb2400039b0 2023-11-06T03:19:13.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb257fff640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2400039b0 0x7fb240003dd0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fb248001750 tx=0x7fb24801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3265454466a6e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480052d0 con 0x7fb2400039b0 2023-11-06T03:19:13.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2480a7050 con 0x7fb2400039b0 2023-11-06T03:19:13.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.401+0000 7fb256ffd640 1 -- 172.21.15.157:0/3031085580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248003600 con 0x7fb2400039b0 2023-11-06T03:19:13.414 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/3031085580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2400039b0 msgr2=0x7fb240003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/3031085580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2400039b0 0x7fb240003dd0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fb248001750 tx=0x7fb24801c370 comp rx=0 tx=0).stop 2023-11-06T03:19:13.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25ca46640 1 -- 172.21.15.157:0/3031085580 reap_dead start 2023-11-06T03:19:13.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/3031085580 shutdown_connections 2023-11-06T03:19:13.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/3031085580 >> 172.21.15.157:0/3031085580 conn(0x7fb258065930 msgr2=0x7fb258065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/3031085580 shutdown_connections 2023-11-06T03:19:13.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/3031085580 wait complete. 2023-11-06T03:19:13.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 Processor -- start 2023-11-06T03:19:13.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- start start 2023-11-06T03:19:13.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2400039b0 0x7fb258157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 0x7fb258144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258144960 con 0x7fb2580b0070 2023-11-06T03:19:13.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580cc410 con 0x7fb258143f70 2023-11-06T03:19:13.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580af5b0 con 0x7fb2400039b0 2023-11-06T03:19:13.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43652/0 (socket says 172.21.15.157:43652) 2023-11-06T03:19:13.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 -- 172.21.15.157:0/2597070725 learned_addr learned my addr 172.21.15.157:0/2597070725 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb257fff640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2400039b0 0x7fb258157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25ca46640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 0x7fb258144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 -- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2400039b0 msgr2=0x7fb258157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2400039b0 0x7fb258157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 -- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 msgr2=0x7fb258144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 0x7fb258144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 -- 172.21.15.157:0/2597070725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580cc6b0 con 0x7fb2580b0070 2023-11-06T03:19:13.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25ca46640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 0x7fb258144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb2577fe640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00e7b0 tx=0x7fb24c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb254ff9640 1 -- 172.21.15.157:0/2597070725 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00c6c0 con 0x7fb2580b0070 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb254ff9640 1 -- 172.21.15.157:0/2597070725 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c00c860 con 0x7fb2580b0070 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2580cc990 con 0x7fb2580b0070 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb254ff9640 1 -- 172.21.15.157:0/2597070725 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00cc10 con 0x7fb2580b0070 2023-11-06T03:19:13.421 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.405+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2580ccf50 con 0x7fb2580b0070 2023-11-06T03:19:13.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb254ff9640 1 -- 172.21.15.157:0/2597070725 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb24c01d460 con 0x7fb2580b0070 2023-11-06T03:19:13.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb254ff9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 0x7fb244067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb254ff9640 1 -- 172.21.15.157:0/2597070725 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb24c083270 con 0x7fb2580b0070 2023-11-06T03:19:13.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 0x7fb244067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 Processor -- start 2023-11-06T03:19:13.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 -- start start 2023-11-06T03:19:13.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 0x7fb244067f20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb2400057b0 tx=0x7fb2480011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0c65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0c66640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0b2dc10 con 0x562ee0cb0130 2023-11-06T03:19:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0abc3e0 con 0x562ee0c66260 2023-11-06T03:19:13.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ee0ab99b0 con 0x562ee0cb0a50 2023-11-06T03:19:13.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43666/0 (socket says 172.21.15.157:43666) 2023-11-06T03:19:13.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 -- 172.21.15.157:0/511706599 learned_addr learned my addr 172.21.15.157:0/511706599 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb2577fe640 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0c65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.425 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb25ca46640 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0c66640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 -- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 msgr2=0x562ee0c65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0c65d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.426 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 -- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 msgr2=0x562ee0c66640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0c66640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 -- 172.21.15.157:0/511706599 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee07b1e20 con 0x562ee0cb0130 2023-11-06T03:19:13.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.409+0000 7fb257fff640 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fb248005040 tx=0x7fb248001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fdee018234fbe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb23dffb640 1 -- 172.21.15.157:0/511706599 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480071c0 con 0x562ee0cb0130 2023-11-06T03:19:13.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb23dffb640 1 -- 172.21.15.157:0/511706599 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb248007320 con 0x562ee0cb0130 2023-11-06T03:19:13.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb23dffb640 1 -- 172.21.15.157:0/511706599 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248007490 con 0x562ee0cb0130 2023-11-06T03:19:13.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/511706599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 msgr2=0x562ee0cb0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fb248005040 tx=0x7fb248001130 comp rx=0 tx=0).stop 2023-11-06T03:19:13.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/511706599 shutdown_connections 2023-11-06T03:19:13.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0c66640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0c65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- 172.21.15.157:0/511706599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0cb0510 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/511706599 >> 172.21.15.157:0/511706599 conn(0x562ee0c6c9f0 msgr2=0x562ee0c64600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/511706599 shutdown_connections 2023-11-06T03:19:13.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/511706599 wait complete. 2023-11-06T03:19:13.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- start start 2023-11-06T03:19:13.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 Processor -- start 2023-11-06T03:19:13.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0ab52a0 con 0x562ee0c66260 2023-11-06T03:19:13.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0a54470 con 0x562ee0cb0130 2023-11-06T03:19:13.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ee0a72660 con 0x562ee0cb0a50 2023-11-06T03:19:13.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 10 client.0 _ll_register_callbacks cb 0x562ee0c51c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:13.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43670/0 (socket says 172.21.15.157:43670) 2023-11-06T03:19:13.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 -- 172.21.15.157:0/156954729 learned_addr learned my addr 172.21.15.157:0/156954729 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2577fe640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb25ca46640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 -- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 msgr2=0x562ee0cb0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 -- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 msgr2=0x562ee0cb7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 -- 172.21.15.157:0/156954729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb240003530 con 0x562ee0c66260 2023-11-06T03:19:13.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2577fe640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb25ca46640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:13.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb257fff640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb248005630 tx=0x7fb248019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdd72135f666627 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.434 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:13.437 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480076d0 con 0x562ee0c66260 2023-11-06T03:19:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb2480bc040 con 0x562ee0c66260 2023-11-06T03:19:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ee0a720a0 con 0x562ee0c66260 2023-11-06T03:19:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2480067e0 con 0x562ee0c66260 2023-11-06T03:19:13.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 10 client.11646 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:13.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x562ee0a7a1d0 con 0x562ee0c66260 2023-11-06T03:19:13.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ee0a720a0 con 0x562ee0c66260 2023-11-06T03:19:13.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb248006940 con 0x562ee0c66260 2023-11-06T03:19:13.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 10 client.11646 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:13.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 10 client.11646 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:13.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 20 client.11646 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:13.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cb1bd0 0x562ee0cb1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 4 client.11646 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:19:13.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ee0ab3bf0 con 0x562ee0cb1bd0 2023-11-06T03:19:13.440 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2577fe640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cb1bd0 0x562ee0cb1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb236ffd640 10 client.11646 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.413+0000 7fb2577fe640 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cb1bd0 0x562ee0cb1fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec32f215589ebf22 server_cookie=3bead5c2fd455420 in_seq=0 out_seq=0 2023-11-06T03:19:13.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb236ffd640 1 -- 172.21.15.157:0/156954729 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562ee0ab3bf0 con 0x562ee0cb1bd0 2023-11-06T03:19:13.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb236ffd640 10 client.11646 handle_command_reply: tid=0 2023-11-06T03:19:13.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 client.11646 shutdown 2023-11-06T03:19:13.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 msgr2=0x562ee0cb7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb248005630 tx=0x7fb248019110 comp rx=0 tx=0).stop 2023-11-06T03:19:13.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 shutdown_connections 2023-11-06T03:19:13.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cb1bd0 0x562ee0cb1fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb236ffd640 0 client.11646 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 --2- 172.21.15.157:0/156954729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb7670 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 >> 172.21.15.157:0/156954729 conn(0x562ee0c6c9f0 msgr2=0x562ee0c64d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.444 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 shutdown_connections 2023-11-06T03:19:13.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 1 -- 172.21.15.157:0/156954729 wait complete. 2023-11-06T03:19:13.445 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.417+0000 7fb2615d1000 20 client.11646 trim_cache size 0 max 16384 2023-11-06T03:19:13.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 Processor -- start 2023-11-06T03:19:13.451 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 -- start start 2023-11-06T03:19:13.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.452 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0772aa0 con 0x562ee0c66260 2023-11-06T03:19:13.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0ab7e00 con 0x562ee0cb0130 2023-11-06T03:19:13.453 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.429+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ee0c69530 con 0x562ee0cb0a50 2023-11-06T03:19:13.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43672/0 (socket says 172.21.15.157:43672) 2023-11-06T03:19:13.454 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 -- 172.21.15.157:0/2667173182 learned_addr learned my addr 172.21.15.157:0/2667173182 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb25ca46640 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 -- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 msgr2=0x562ee0cb5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.455 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 -- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 msgr2=0x562ee0cb5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 -- 172.21.15.157:0/2667173182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ee0a60520 con 0x562ee0c66260 2023-11-06T03:19:13.456 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb257fff640 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fb248001170 tx=0x7fb2480cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e13746d463b4f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb23e7fc640 1 -- 172.21.15.157:0/2667173182 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480074d0 con 0x562ee0c66260 2023-11-06T03:19:13.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb23e7fc640 1 -- 172.21.15.157:0/2667173182 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb248007630 con 0x562ee0c66260 2023-11-06T03:19:13.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb23e7fc640 1 -- 172.21.15.157:0/2667173182 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248006a20 con 0x562ee0c66260 2023-11-06T03:19:13.457 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/2667173182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 msgr2=0x562ee0cb4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fb248001170 tx=0x7fb2480cd000 comp rx=0 tx=0).stop 2023-11-06T03:19:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/2667173182 shutdown_connections 2023-11-06T03:19:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0cb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0cb0130 0x562ee0cb5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.458 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- 172.21.15.157:0/2667173182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0c66260 0x562ee0cb4fd0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/2667173182 >> 172.21.15.157:0/2667173182 conn(0x562ee0c6c9f0 msgr2=0x562ee0c6b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/2667173182 shutdown_connections 2023-11-06T03:19:13.459 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/2667173182 wait complete. 2023-11-06T03:19:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- start start 2023-11-06T03:19:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 Processor -- start 2023-11-06T03:19:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0d0c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.460 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0caac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0c5da70 con 0x562ee0cb0130 2023-11-06T03:19:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ee0ab2ab0 con 0x562ee0c66260 2023-11-06T03:19:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ee0ab0c10 con 0x562ee0cb0a50 2023-11-06T03:19:13.461 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 10 client.0 _ll_register_callbacks cb 0x562ee0a79cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:43688/0 (socket says 172.21.15.157:43688) 2023-11-06T03:19:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 -- 172.21.15.157:0/1004588850 learned_addr learned my addr 172.21.15.157:0/1004588850 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:13.462 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 -- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 msgr2=0x562ee0caac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0caac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 -- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 msgr2=0x562ee0d0c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:13.463 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0d0c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 -- 172.21.15.157:0/1004588850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ee0a7a1d0 con 0x562ee0cb0130 2023-11-06T03:19:13.464 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2577fe640 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fb24c050be0 tx=0x7fb24c083170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be51ee23f08e12d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:13.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:13.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00cd70 con 0x562ee0cb0130 2023-11-06T03:19:13.465 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ee0a7e620 con 0x562ee0cb0130 2023-11-06T03:19:13.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fb24c050df0 con 0x562ee0cb0130 2023-11-06T03:19:13.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb24c050fb0 con 0x562ee0cb0130 2023-11-06T03:19:13.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 10 client.11652 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:13.466 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x562ee0a54090 con 0x562ee0cb0130 2023-11-06T03:19:13.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ee0a7e620 con 0x562ee0cb0130 2023-11-06T03:19:13.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb24c054940 con 0x562ee0cb0130 2023-11-06T03:19:13.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 10 client.11652 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:13.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 10 client.11652 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:13.467 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 20 client.11652 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:13.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cab1b0 0x562ee0cab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:13.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 4 client.11652 mds_command: new command op to 11487 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:19:13.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.433+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ee0aae320 con 0x562ee0cab1b0 2023-11-06T03:19:13.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb257fff640 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cab1b0 0x562ee0cab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:13.468 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb236ffd640 10 client.11652 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb257fff640 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cab1b0 0x562ee0cab5c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50835c257e212c86 server_cookie=4e10fe1bc2f2dd1f in_seq=0 out_seq=0 2023-11-06T03:19:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb236ffd640 1 -- 172.21.15.157:0/1004588850 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x562ee0aae320 con 0x562ee0cab1b0 2023-11-06T03:19:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb236ffd640 10 client.11652 handle_command_reply: tid=0 2023-11-06T03:19:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 client.11652 shutdown 2023-11-06T03:19:13.469 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 msgr2=0x562ee0d0cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fb24c050be0 tx=0x7fb24c083170 comp rx=0 tx=0).stop 2023-11-06T03:19:13.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 shutdown_connections 2023-11-06T03:19:13.470 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x562ee0cab1b0 0x562ee0cab5c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x562ee0cb0a50 0x562ee0caac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562ee0cb0130 0x562ee0d0cc00 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 --2- 172.21.15.157:0/1004588850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ee0c66260 0x562ee0d0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.471 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb236ffd640 0 client.11652 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:13.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 >> 172.21.15.157:0/1004588850 conn(0x562ee0c6c9f0 msgr2=0x562ee0c6c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 shutdown_connections 2023-11-06T03:19:13.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 1 -- 172.21.15.157:0/1004588850 wait complete. 2023-11-06T03:19:13.472 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb2615d1000 20 client.11652 trim_cache size 0 max 16384 2023-11-06T03:19:13.473 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:13.473 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-06T03:19:13.474 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": { 2023-11-06T03:19:13.474 INFO:teuthology.orchestra.run.smithi157.stdout: "b5aeba5f-d7b0-4b27-8d77-e04c69667dd1": { 2023-11-06T03:19:13.474 INFO:teuthology.orchestra.run.smithi157.stdout: "path": "/", 2023-11-06T03:19:13.474 INFO:teuthology.orchestra.run.smithi157.stdout: "tag": "b5aeba5f-d7b0-4b27-8d77-e04c69667dd1", 2023-11-06T03:19:13.474 INFO:teuthology.orchestra.run.smithi157.stdout: "options": "recursive,repair" 2023-11-06T03:19:13.475 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:19:13.475 INFO:teuthology.orchestra.run.smithi157.stdout: } 2023-11-06T03:19:13.475 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:13.476 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 msgr2=0x7fb244067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 0x7fb244067f20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb2400057b0 tx=0x7fb2480011a0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 msgr2=0x7fb2580b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:13.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00e7b0 tx=0x7fb24c00ec70 comp rx=0 tx=0).stop 2023-11-06T03:19:13.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 shutdown_connections 2023-11-06T03:19:13.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fb244065a60 0x7fb244067f20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb258143f70 0x7fb258144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2580b0070 0x7fb2580b04f0 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 --2- 172.21.15.157:0/2597070725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2400039b0 0x7fb258157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:13.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 >> 172.21.15.157:0/2597070725 conn(0x7fb258065930 msgr2=0x7fb25806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:13.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 shutdown_connections 2023-11-06T03:19:13.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:13.437+0000 7fb25e4a9640 1 -- 172.21.15.157:0/2597070725 wait complete. 2023-11-06T03:19:13.480 INFO:tasks.cephfs.filesystem:scrub status for tag:b5aeba5f-d7b0-4b27-8d77-e04c69667dd1 - {'path': '/', 'tag': 'b5aeba5f-d7b0-4b27-8d77-e04c69667dd1', 'options': 'recursive,repair'} 2023-11-06T03:19:43.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:19:43.076 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:19:43.481 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 scrub status 2023-11-06T03:19:43.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.621+0000 7f8abdcfe640 1 Processor -- start 2023-11-06T03:19:43.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.621+0000 7f8abdcfe640 1 -- start start 2023-11-06T03:19:43.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab81551e0 0x7f8ab806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab815e5e0 0x7f8ab80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 0x7f8ab80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab815f580 con 0x7f8ab815e5e0 2023-11-06T03:19:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab815efc0 con 0x7f8ab815ea60 2023-11-06T03:19:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8abdcfe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab815f2a0 con 0x7f8ab81551e0 2023-11-06T03:19:43.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab815e5e0 0x7f8ab80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ab815e5e0 0x7f8ab80b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45944/0 (socket says 172.21.15.157:45944) 2023-11-06T03:19:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1818097824 learned_addr learned my addr 172.21.15.157:0/1818097824 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/1818097824 >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 0x7f8ab80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab81551e0 0x7f8ab806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1818097824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab81551e0 msgr2=0x7f8ab806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab81551e0 0x7f8ab806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1818097824 >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 msgr2=0x7f8ab80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1818097824 >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 0x7f8ab80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1818097824 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab806f5a0 con 0x7f8ab815e5e0 2023-11-06T03:19:43.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/1818097824 >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 0x7f8ab80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.625+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80b3410 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80cc5d0 tx=0x7f8aa801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a94507d9411e6e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8ab67fc640 1 -- 172.21.15.157:0/1818097824 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80ab620 con 0x7f8ab815e5e0 2023-11-06T03:19:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8ab67fc640 1 -- 172.21.15.157:0/1818097824 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80ab780 con 0x7f8ab815e5e0 2023-11-06T03:19:43.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8ab67fc640 1 -- 172.21.15.157:0/1818097824 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80abb10 con 0x7f8ab815e5e0 2023-11-06T03:19:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1818097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab815e5e0 msgr2=0x7f8ab80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80b3410 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80cc5d0 tx=0x7f8aa801c440 comp rx=0 tx=0).stop 2023-11-06T03:19:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1818097824 shutdown_connections 2023-11-06T03:19:43.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1818097824 >> v2:172.21.15.195:3300/0 conn(0x7f8ab815ea60 0x7f8ab80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80b3410 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1818097824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab81551e0 0x7f8ab806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1818097824 >> 172.21.15.157:0/1818097824 conn(0x7f8ab8065930 msgr2=0x7f8ab8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1818097824 shutdown_connections 2023-11-06T03:19:43.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1818097824 wait complete. 2023-11-06T03:19:43.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 Processor -- start 2023-11-06T03:19:43.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.629+0000 7f8abdcfe640 1 -- start start 2023-11-06T03:19:43.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 0x7f8ab80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8143a10 con 0x7f8ab81551e0 2023-11-06T03:19:43.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80cdef0 con 0x7f8ab815e5e0 2023-11-06T03:19:43.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80ce1d0 con 0x7f8ab815ea60 2023-11-06T03:19:43.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45950/0 (socket says 172.21.15.157:45950) 2023-11-06T03:19:43.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 -- 172.21.15.157:0/1084026157 learned_addr learned my addr 172.21.15.157:0/1084026157 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 0x7f8ab80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 -- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 msgr2=0x7f8ab80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 0x7f8ab80b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 -- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab815e5e0 msgr2=0x7f8ab80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.630 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 -- 172.21.15.157:0/1084026157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80094d0 con 0x7f8ab81551e0 2023-11-06T03:19:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 0x7f8ab80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7f0 tx=0x7f8aac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 -- 172.21.15.157:0/1084026157 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac004220 con 0x7f8ab81551e0 2023-11-06T03:19:43.631 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 -- 172.21.15.157:0/1084026157 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac004380 con 0x7f8ab81551e0 2023-11-06T03:19:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 -- 172.21.15.157:0/1084026157 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac003860 con 0x7f8ab81551e0 2023-11-06T03:19:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab80ce4b0 con 0x7f8ab81551e0 2023-11-06T03:19:43.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab814b3e0 con 0x7f8ab81551e0 2023-11-06T03:19:43.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 -- 172.21.15.157:0/1084026157 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8aac003bd0 con 0x7f8ab81551e0 2023-11-06T03:19:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 0x7f8aa4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8a9ffff640 1 -- 172.21.15.157:0/1084026157 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f8aac083390 con 0x7f8ab81551e0 2023-11-06T03:19:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.633+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 0x7f8aa4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.639 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 Processor -- start 2023-11-06T03:19:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 0x7f8aa4067f20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80af000 tx=0x7f8aa80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- start start 2023-11-06T03:19:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8cc82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8caffc10 con 0x560b8cc38260 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca8e3e0 con 0x560b8cc82a50 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca8b9b0 con 0x560b8cc82130 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45952/0 (socket says 172.21.15.157:45952) 2023-11-06T03:19:43.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 -- 172.21.15.157:0/685021350 learned_addr learned my addr 172.21.15.157:0/685021350 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8cc82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 -- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 msgr2=0x560b8cc82510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8cc82510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 -- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 msgr2=0x560b8cc37d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc37d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 -- 172.21.15.157:0/685021350 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8c783e20 con 0x560b8cc38260 2023-11-06T03:19:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007f60 tx=0x7f8ab0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab76d0815d34834 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8a9d7fa640 1 -- 172.21.15.157:0/685021350 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000f860 con 0x560b8cc38260 2023-11-06T03:19:43.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8a9d7fa640 1 -- 172.21.15.157:0/685021350 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab000f9c0 con 0x560b8cc38260 2023-11-06T03:19:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8a9d7fa640 1 -- 172.21.15.157:0/685021350 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000fd50 con 0x560b8cc38260 2023-11-06T03:19:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- 172.21.15.157:0/685021350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 msgr2=0x560b8cc38640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007f60 tx=0x7f8ab0007f90 comp rx=0 tx=0).stop 2023-11-06T03:19:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- 172.21.15.157:0/685021350 shutdown_connections 2023-11-06T03:19:43.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc38260 0x560b8cc38640 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc37d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/685021350 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8cc82510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- 172.21.15.157:0/685021350 >> 172.21.15.157:0/685021350 conn(0x560b8cc3e9f0 msgr2=0x560b8cc36600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- 172.21.15.157:0/685021350 shutdown_connections 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- 172.21.15.157:0/685021350 wait complete. 2023-11-06T03:19:43.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 -- start start 2023-11-06T03:19:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.637+0000 7f8ac0e26000 1 Processor -- start 2023-11-06T03:19:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8cc813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.646 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca872a0 con 0x560b8cc82130 2023-11-06T03:19:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca26470 con 0x560b8cc82a50 2023-11-06T03:19:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca44660 con 0x560b8cc38260 2023-11-06T03:19:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 10 client.0 _ll_register_callbacks cb 0x560b8cc23c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45958/0 (socket says 172.21.15.157:45958) 2023-11-06T03:19:43.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/2885347639 learned_addr learned my addr 172.21.15.157:0/2885347639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8cc813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 msgr2=0x560b8cc813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8cc813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 msgr2=0x560b8cc83580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc83580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/2885347639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab0007b90 con 0x560b8cc82130 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ad040 tx=0x7f8aa801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314b778cddfe6b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:43.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80c1820 con 0x560b8cc82130 2023-11-06T03:19:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f8aa80c1980 con 0x560b8cc82130 2023-11-06T03:19:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b8ca440a0 con 0x560b8cc82130 2023-11-06T03:19:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa80c1b40 con 0x560b8cc82130 2023-11-06T03:19:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 10 client.11664 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:43.650 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x560b8ca4c1d0 con 0x560b8cc82130 2023-11-06T03:19:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b8ca440a0 con 0x560b8cc82130 2023-11-06T03:19:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8aa80c1ca0 con 0x560b8cc82130 2023-11-06T03:19:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 10 client.11664 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 10 client.11664 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:43.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 20 client.11664 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc83f90 0x560b8cc843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 4 client.11664 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:19:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b8ca85bf0 con 0x560b8cc83f90 2023-11-06T03:19:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc83f90 0x560b8cc843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc83f90 0x560b8cc843a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0755e2b9146b49 server_cookie=c9367d480bdf6db2 in_seq=0 out_seq=0 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 10 client.11664 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 1 -- 172.21.15.157:0/2885347639 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560b8ca85bf0 con 0x560b8cc83f90 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 10 client.11664 handle_command_reply: tid=0 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 client.11664 shutdown 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 msgr2=0x560b8cc83040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ad040 tx=0x7f8aa801ac80 comp rx=0 tx=0).stop 2023-11-06T03:19:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 shutdown_connections 2023-11-06T03:19:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc83f90 0x560b8cc843a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8cc83580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8a967fc640 0 client.11664 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:43.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8cc83040 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2885347639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8cc813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 >> 172.21.15.157:0/2885347639 conn(0x560b8cc3e9f0 msgr2=0x560b8cc36d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 shutdown_connections 2023-11-06T03:19:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2885347639 wait complete. 2023-11-06T03:19:43.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.641+0000 7f8ac0e26000 20 client.11664 trim_cache size 0 max 16384 2023-11-06T03:19:43.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 Processor -- start 2023-11-06T03:19:43.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- start start 2023-11-06T03:19:43.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8ccd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.661 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8ccda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8c744aa0 con 0x560b8cc82130 2023-11-06T03:19:43.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca45400 con 0x560b8cc82a50 2023-11-06T03:19:43.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca84ab0 con 0x560b8cc38260 2023-11-06T03:19:43.662 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45960/0 (socket says 172.21.15.157:45960) 2023-11-06T03:19:43.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1727946706 learned_addr learned my addr 172.21.15.157:0/1727946706 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8ccda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 msgr2=0x560b8ccd9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.663 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8ccd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 msgr2=0x560b8ccda0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8ccda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 -- 172.21.15.157:0/1727946706 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8ca32520 con 0x560b8cc82130 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f8aa801c9b0 tx=0x7f8aa80c3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e378db249cbf46 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8a9dffb640 1 -- 172.21.15.157:0/1727946706 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80c1240 con 0x560b8cc82130 2023-11-06T03:19:43.664 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8a9dffb640 1 -- 172.21.15.157:0/1727946706 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80bf520 con 0x560b8cc82130 2023-11-06T03:19:43.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8a9dffb640 1 -- 172.21.15.157:0/1727946706 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80bfae0 con 0x560b8cc82130 2023-11-06T03:19:43.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- 172.21.15.157:0/1727946706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 msgr2=0x560b8ccd9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f8aa801c9b0 tx=0x7f8aa80c3040 comp rx=0 tx=0).stop 2023-11-06T03:19:43.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- 172.21.15.157:0/1727946706 shutdown_connections 2023-11-06T03:19:43.665 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc82a50 0x560b8ccda0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82130 0x560b8ccd9b60 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/1727946706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc38260 0x560b8ccd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- 172.21.15.157:0/1727946706 >> 172.21.15.157:0/1727946706 conn(0x560b8cc3e9f0 msgr2=0x560b8cc8fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- 172.21.15.157:0/1727946706 shutdown_connections 2023-11-06T03:19:43.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.657+0000 7f8ac0e26000 1 -- 172.21.15.157:0/1727946706 wait complete. 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- start start 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 Processor -- start 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 0x560b8ccdf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8ccdf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8cc98720 con 0x560b8cc82a50 2023-11-06T03:19:43.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ca86130 con 0x560b8cc38260 2023-11-06T03:19:43.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8cc3c4f0 con 0x560b8cc82130 2023-11-06T03:19:43.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 10 client.0 _ll_register_callbacks cb 0x560b8ca4bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:43.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45970/0 (socket says 172.21.15.157:45970) 2023-11-06T03:19:43.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 -- 172.21.15.157:0/2601285192 learned_addr learned my addr 172.21.15.157:0/2601285192 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:43.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 0x560b8ccdf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab6ffd640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8ccdf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 -- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 msgr2=0x560b8ccdf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8ccdf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 -- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 msgr2=0x560b8ccdf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:43.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 0x560b8ccdf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 -- 172.21.15.157:0/2601285192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8ca4c1d0 con 0x560b8cc82a50 2023-11-06T03:19:43.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 0x560b8ccdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:43.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab7fff640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x560b8ca89f10 tx=0x7f8ab0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5251d243aa40ad0c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:43.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:43.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000fba0 con 0x560b8cc82a50 2023-11-06T03:19:43.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f8ab0013070 con 0x560b8cc82a50 2023-11-06T03:19:43.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b8ca50620 con 0x560b8cc82a50 2023-11-06T03:19:43.671 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab000fd00 con 0x560b8cc82a50 2023-11-06T03:19:43.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 10 client.11670 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:43.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x560b8ca26090 con 0x560b8cc82a50 2023-11-06T03:19:43.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b8ca50620 con 0x560b8cc82a50 2023-11-06T03:19:43.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f8ab0018480 con 0x560b8cc82a50 2023-11-06T03:19:43.672 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 10 client.11670 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 10 client.11670 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 20 client.11670 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc7c840 0x560b8cc7cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 4 client.11670 mds_command: new command op to 11487 tid=0[{"prefix": "scrub status"}] 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b8ca80320 con 0x560b8cc7c840 2023-11-06T03:19:43.673 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc7c840 0x560b8cc7cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:43.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ab77fe640 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc7c840 0x560b8cc7cc50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cadde2062ecfc850 server_cookie=f5748bd61742bbc5 in_seq=0 out_seq=0 2023-11-06T03:19:43.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 10 client.11670 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:43.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 1 -- 172.21.15.157:0/2601285192 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560b8ca80320 con 0x560b8cc7c840 2023-11-06T03:19:43.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 10 client.11670 handle_command_reply: tid=0 2023-11-06T03:19:43.674 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 client.11670 shutdown 2023-11-06T03:19:43.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 msgr2=0x560b8cc7c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x560b8ca89f10 tx=0x7f8ab0010000 comp rx=0 tx=0).stop 2023-11-06T03:19:43.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 shutdown_connections 2023-11-06T03:19:43.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x560b8cc7c840 0x560b8cc7cc50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.675 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b8cc82a50 0x560b8cc7c300 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x560b8cc82130 0x560b8ccdf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8a967fc640 0 client.11670 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:43.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 --2- 172.21.15.157:0/2601285192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560b8cc38260 0x560b8ccdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 >> 172.21.15.157:0/2601285192 conn(0x560b8cc3e9f0 msgr2=0x560b8cc3d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.676 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 shutdown_connections 2023-11-06T03:19:43.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 1 -- 172.21.15.157:0/2601285192 wait complete. 2023-11-06T03:19:43.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.661+0000 7f8ac0e26000 20 client.11670 trim_cache size 0 max 16384 2023-11-06T03:19:43.677 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:43.677 INFO:teuthology.orchestra.run.smithi157.stdout: "status": "no active scrubs running", 2023-11-06T03:19:43.678 INFO:teuthology.orchestra.run.smithi157.stdout: "scrubs": {} 2023-11-06T03:19:43.678 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:43.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 msgr2=0x7f8aa4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 0x7f8aa4067f20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80af000 tx=0x7f8aa80a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 msgr2=0x7f8ab8145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7f0 tx=0x7f8aac00ccb0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 shutdown_connections 2023-11-06T03:19:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8aa4065a60 0x7f8aa4067f20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab815ea60 0x7f8ab80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab815e5e0 0x7f8ab80cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 --2- 172.21.15.157:0/1084026157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ab81551e0 0x7f8ab8145540 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:43.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 >> 172.21.15.157:0/1084026157 conn(0x7f8ab8065930 msgr2=0x7f8ab815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:43.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 shutdown_connections 2023-11-06T03:19:43.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:43.665+0000 7f8abdcfe640 1 -- 172.21.15.157:0/1084026157 wait complete. 2023-11-06T03:19:43.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-06T03:19:44.085 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:19:44.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 Processor -- start 2023-11-06T03:19:44.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 -- start start 2023-11-06T03:19:44.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe3380b2af0 0x7fe3380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33815e440 con 0x7fe3381599c0 2023-11-06T03:19:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33806c240 con 0x7fe33806d000 2023-11-06T03:19:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe33de27640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33806c520 con 0x7fe3380b2af0 2023-11-06T03:19:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe3377fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe3381599c0 0x7fe3380b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34442/0 (socket says 172.21.15.157:34442) 2023-11-06T03:19:44.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe3377fe640 1 -- 172.21.15.157:0/986438794 learned_addr learned my addr 172.21.15.157:0/986438794 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:44.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe32ffff640 1 --2- 172.21.15.157:0/986438794 >> v2:172.21.15.195:3300/0 conn(0x7fe3380b2af0 0x7fe3380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844854961 0 0) 0x7fe33815e440 con 0x7fe3381599c0 2023-11-06T03:19:44.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe324003530 con 0x7fe3381599c0 2023-11-06T03:19:44.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330561753 0 0) 0x7fe324003530 con 0x7fe3381599c0 2023-11-06T03:19:44.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 >> v2:172.21.15.195:3300/0 conn(0x7fe3380b2af0 msgr2=0x7fe3380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 --2- 172.21.15.157:0/986438794 >> v2:172.21.15.195:3300/0 conn(0x7fe3380b2af0 0x7fe3380ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 >> v1:172.21.15.195:6789/0 conn(0x7fe33806d000 legacy=0x7fe338159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33806c800 con 0x7fe3381599c0 2023-11-06T03:19:44.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fe328001680 con 0x7fe3381599c0 2023-11-06T03:19:44.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 >> v1:172.21.15.157:6789/0 conn(0x7fe3381599c0 legacy=0x7fe3380b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe324003980 0x7fe324003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3240047d0 0x7fe324004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe32400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33806c520 con 0x7fe324005190 2023-11-06T03:19:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32400c030 con 0x7fe324003980 2023-11-06T03:19:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32400c1f0 con 0x7fe3240047d0 2023-11-06T03:19:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe32400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe3377fe640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe324003980 0x7fe324003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe32ffff640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3240047d0 0x7fe324004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 -- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3240047d0 msgr2=0x7fe324004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3240047d0 0x7fe324004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 -- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe324003980 msgr2=0x7fe324003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe324003980 0x7fe324003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.225+0000 7fe337fff640 1 -- 172.21.15.157:0/986438794 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324003530 con 0x7fe324005190 2023-11-06T03:19:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe337fff640 1 -- 172.21.15.157:0/986438794 reap_dead start 2023-11-06T03:19:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe337fff640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe32400b990 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fe330001e40 tx=0x7fe33001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3acd5aca18c331 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330003b80 con 0x7fe324005190 2023-11-06T03:19:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330003ce0 con 0x7fe324005190 2023-11-06T03:19:44.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe336ffd640 1 -- 172.21.15.157:0/986438794 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330005260 con 0x7fe324005190 2023-11-06T03:19:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 msgr2=0x7fe32400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe32400b990 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fe330001e40 tx=0x7fe33001c740 comp rx=0 tx=0).stop 2023-11-06T03:19:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- 172.21.15.157:0/986438794 shutdown_connections 2023-11-06T03:19:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 --2- 172.21.15.157:0/986438794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe32400b990 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- 172.21.15.157:0/986438794 >> 172.21.15.157:0/986438794 conn(0x7fe338065930 msgr2=0x7fe338065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- 172.21.15.157:0/986438794 shutdown_connections 2023-11-06T03:19:44.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- 172.21.15.157:0/986438794 wait complete. 2023-11-06T03:19:44.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 Processor -- start 2023-11-06T03:19:44.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- start start 2023-11-06T03:19:44.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 0x7fe3380cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 0x7fe3381f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381f1e80 con 0x7fe324005190 2023-11-06T03:19:44.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33815cd90 con 0x7fe33815ba70 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.229+0000 7fe33de27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33815a850 con 0x7fe3381f14c0 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45984/0 (socket says 172.21.15.157:45984) 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 -- 172.21.15.157:0/402815326 learned_addr learned my addr 172.21.15.157:0/402815326 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe337fff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 0x7fe3381f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe32ffff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 0x7fe3380cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 -- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 msgr2=0x7fe3381f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 0x7fe3381f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 -- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 msgr2=0x7fe3380cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 0x7fe3380cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 -- 172.21.15.157:0/402815326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33815ab30 con 0x7fe324005190 2023-11-06T03:19:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe337fff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 0x7fe3381f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe32ffff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 0x7fe3380cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe3377fe640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fe328007290 tx=0x7fe328007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32800f9c0 con 0x7fe324005190 2023-11-06T03:19:44.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32800fb40 con 0x7fe324005190 2023-11-06T03:19:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33815ae10 con 0x7fe324005190 2023-11-06T03:19:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3280133f0 con 0x7fe324005190 2023-11-06T03:19:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33815bef0 con 0x7fe324005190 2023-11-06T03:19:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe338072750 con 0x7fe324005190 2023-11-06T03:19:44.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe328013550 con 0x7fe324005190 2023-11-06T03:19:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 0x7fe318067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fe3280835c0 con 0x7fe324005190 2023-11-06T03:19:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe32ffff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 0x7fe318067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.239 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.233+0000 7fe32ffff640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 0x7fe318067f20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cf310 tx=0x7fe320008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.240 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.237+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe328050d70 con 0x7fe324005190 2023-11-06T03:19:44.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.417+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3380adcd0 con 0x7fe324005190 2023-11-06T03:19:44.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.417+0000 7fe334ff9640 1 -- 172.21.15.157:0/402815326 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7fe328054390 con 0x7fe324005190 2023-11-06T03:19:44.413 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:44.413 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:19:44.413 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:19:44.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 msgr2=0x7fe318067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 0x7fe318067f20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cf310 tx=0x7fe320008040 comp rx=0 tx=0).stop 2023-11-06T03:19:44.415 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 msgr2=0x7fe33815c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fe328007290 tx=0x7fe328007750 comp rx=0 tx=0).stop 2023-11-06T03:19:44.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 shutdown_connections 2023-11-06T03:19:44.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe318065a60 0x7fe318067f20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe3381f14c0 0x7fe3381f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe33815ba70 0x7fe3380cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 --2- 172.21.15.157:0/402815326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe324005190 0x7fe33815c700 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 >> 172.21.15.157:0/402815326 conn(0x7fe338065930 msgr2=0x7fe3380b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 shutdown_connections 2023-11-06T03:19:44.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.421+0000 7fe33de27640 1 -- 172.21.15.157:0/402815326 wait complete. 2023-11-06T03:19:44.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-06T03:18:18.097057+0000', 'modified': '2023-11-06T03:18:40.949661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 190, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11487': {'gid': 11487, 'name': 'a', 'rank': 0, 'incarnation': 133, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6835/2374994036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 2374994036}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2374994036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-06T03:19:44.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-06T03:19:44.432 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-06T03:19:44.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 Processor -- start 2023-11-06T03:19:44.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 -- start start 2023-11-06T03:19:44.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 0x7f7d880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d880b2af0 0x7f7d88159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8815e440 con 0x7f7d8806d000 2023-11-06T03:19:44.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8806c240 con 0x7f7d880b2af0 2023-11-06T03:19:44.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8ff54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806c520 con 0x7f7d881599c0 2023-11-06T03:19:44.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d880b2af0 0x7f7d88159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7d880b2af0 0x7f7d88159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45992/0 (socket says 172.21.15.157:45992) 2023-11-06T03:19:44.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/419501610 learned_addr learned my addr 172.21.15.157:0/419501610 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 0x7f7d880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/419501610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 msgr2=0x7f7d880ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 0x7f7d880ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/419501610 >> v1:172.21.15.195:6789/0 conn(0x7f7d8806d000 legacy=0x7f7d880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/419501610 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8806c800 con 0x7f7d880b2af0 2023-11-06T03:19:44.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.561+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 0x7f7d880ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:19:44.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880b2af0 0x7f7d88159da0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009e50 tx=0x7f7d8001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2577cfc1a3f2bc2a server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ccee640 1 -- 172.21.15.157:0/419501610 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800a58d0 con 0x7f7d880b2af0 2023-11-06T03:19:44.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ccee640 1 -- 172.21.15.157:0/419501610 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800a5a30 con 0x7f7d880b2af0 2023-11-06T03:19:44.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ccee640 1 -- 172.21.15.157:0/419501610 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800a5dc0 con 0x7f7d880b2af0 2023-11-06T03:19:44.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- 172.21.15.157:0/419501610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880b2af0 msgr2=0x7f7d88159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880b2af0 0x7f7d88159da0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009e50 tx=0x7f7d8001c9e0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- 172.21.15.157:0/419501610 shutdown_connections 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d880b2af0 0x7f7d88159da0 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/419501610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d881599c0 0x7f7d880ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- 172.21.15.157:0/419501610 >> 172.21.15.157:0/419501610 conn(0x7f7d88065930 msgr2=0x7f7d88065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- 172.21.15.157:0/419501610 shutdown_connections 2023-11-06T03:19:44.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- 172.21.15.157:0/419501610 wait complete. 2023-11-06T03:19:44.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 Processor -- start 2023-11-06T03:19:44.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- start start 2023-11-06T03:19:44.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d880b2af0 0x7f7d880ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d881599c0 0x7f7d8806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8813bf10 con 0x7f7d8806d000 2023-11-06T03:19:44.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cdb50 con 0x7f7d881599c0 2023-11-06T03:19:44.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8ff54640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cde30 con 0x7f7d880b2af0 2023-11-06T03:19:44.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:45994/0 (socket says 172.21.15.157:45994) 2023-11-06T03:19:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/778497149 learned_addr learned my addr 172.21.15.157:0/778497149 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d880b2af0 msgr2=0x7f7d880ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d880b2af0 0x7f7d880ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d881599c0 msgr2=0x7f7d8806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d881599c0 0x7f7d8806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.565+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/778497149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009b30 con 0x7f7d8806d000 2023-11-06T03:19:44.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400d800 tx=0x7f7d8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 -- 172.21.15.157:0/778497149 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400b6c0 con 0x7f7d8806d000 2023-11-06T03:19:44.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 -- 172.21.15.157:0/778497149 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8400b820 con 0x7f7d8806d000 2023-11-06T03:19:44.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880ce110 con 0x7f7d8806d000 2023-11-06T03:19:44.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 -- 172.21.15.157:0/778497149 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400bbd0 con 0x7f7d8806d000 2023-11-06T03:19:44.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8814d360 con 0x7f7d8806d000 2023-11-06T03:19:44.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 -- 172.21.15.157:0/778497149 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d8400bd30 con 0x7f7d8806d000 2023-11-06T03:19:44.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d7e7fc640 1 -- 172.21.15.157:0/778497149 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f7d84083860 con 0x7f7d8806d000 2023-11-06T03:19:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 Processor -- start 2023-11-06T03:19:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 -- start start 2023-11-06T03:19:44.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000f70 tx=0x7f7d780079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de78f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de78fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de657fc0 con 0x55d6de7d96e0 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5e61a0 con 0x55d6de7da000 2023-11-06T03:19:44.573 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5e3770 con 0x55d6de78fa90 2023-11-06T03:19:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.569+0000 7f7d8dcf0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46006/0 (socket says 172.21.15.157:46006) 2023-11-06T03:19:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/6897344 learned_addr learned my addr 172.21.15.157:0/6897344 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.574 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de78fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 msgr2=0x55d6de78fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de78fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 msgr2=0x55d6de78f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de78f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de78f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 -- 172.21.15.157:0/6897344 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6de2db890 con 0x55d6de7d96e0 2023-11-06T03:19:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f7d8405c780 tx=0x7f7d84054760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc2b370721bd5c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d6b7fe640 1 -- 172.21.15.157:0/6897344 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8406c070 con 0x55d6de7d96e0 2023-11-06T03:19:44.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d6b7fe640 1 -- 172.21.15.157:0/6897344 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8405ccb0 con 0x55d6de7d96e0 2023-11-06T03:19:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d6b7fe640 1 -- 172.21.15.157:0/6897344 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8406c1d0 con 0x55d6de7d96e0 2023-11-06T03:19:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/6897344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 msgr2=0x55d6de7d9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f7d8405c780 tx=0x7f7d84054760 comp rx=0 tx=0).stop 2023-11-06T03:19:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/6897344 shutdown_connections 2023-11-06T03:19:44.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de78fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de78f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- 172.21.15.157:0/6897344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7d9ac0 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/6897344 >> 172.21.15.157:0/6897344 conn(0x55d6de795bf0 msgr2=0x55d6de78de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/6897344 shutdown_connections 2023-11-06T03:19:44.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/6897344 wait complete. 2023-11-06T03:19:44.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- start start 2023-11-06T03:19:44.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 Processor -- start 2023-11-06T03:19:44.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de79aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de79b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5df060 con 0x55d6de7d96e0 2023-11-06T03:19:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de57e290 con 0x55d6de7da000 2023-11-06T03:19:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6de59c6f0 con 0x55d6de78fa90 2023-11-06T03:19:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 10 client.0 _ll_register_callbacks cb 0x55d6de77b1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:44.583 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46014/0 (socket says 172.21.15.157:46014) 2023-11-06T03:19:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/171379205 learned_addr learned my addr 172.21.15.157:0/171379205 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 msgr2=0x55d6de79aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de79aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.584 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de79aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de79b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 msgr2=0x55d6de79b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de79b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/171379205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8814d360 con 0x55d6de7d96e0 2023-11-06T03:19:44.585 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f7d78014de0 tx=0x7f7d78001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a6785e1bbc905 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78005320 con 0x55d6de7d96e0 2023-11-06T03:19:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f7d78005480 con 0x55d6de7d96e0 2023-11-06T03:19:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6de5dd9b0 con 0x55d6de7d96e0 2023-11-06T03:19:44.586 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d78003600 con 0x55d6de7d96e0 2023-11-06T03:19:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 10 client.11688 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55d6de5a41e0 con 0x55d6de7d96e0 2023-11-06T03:19:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6de5dd9b0 con 0x55d6de7d96e0 2023-11-06T03:19:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7d78003760 con 0x55d6de7d96e0 2023-11-06T03:19:44.587 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 10 client.11688 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 10 client.11688 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:19:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 20 client.11688 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de79bde0 0x55d6de7eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 4 client.11688 mds_command: new command op to 11487 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-06T03:19:44.588 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.573+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6de60e4d0 con 0x55d6de79bde0 2023-11-06T03:19:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de79bde0 0x55d6de7eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de79bde0 0x55d6de7eae60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8bac3eed86c771 server_cookie=af1aec7564ce90c in_seq=0 out_seq=0 2023-11-06T03:19:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d68ff9640 10 client.11688 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d68ff9640 1 -- 172.21.15.157:0/171379205 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d6de60e4d0 con 0x55d6de79bde0 2023-11-06T03:19:44.589 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d68ff9640 10 client.11688 handle_command_reply: tid=0 2023-11-06T03:19:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 client.11688 shutdown 2023-11-06T03:19:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 msgr2=0x55d6de79afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f7d78014de0 tx=0x7f7d78001130 comp rx=0 tx=0).stop 2023-11-06T03:19:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 shutdown_connections 2023-11-06T03:19:44.590 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de79bde0 0x55d6de7eae60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de79b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de79afd0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 --2- 172.21.15.157:0/171379205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de79aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d68ff9640 0 client.11688 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 >> 172.21.15.157:0/171379205 conn(0x55d6de795bf0 msgr2=0x55d6de78e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.591 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 shutdown_connections 2023-11-06T03:19:44.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 1 -- 172.21.15.157:0/171379205 wait complete. 2023-11-06T03:19:44.592 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.577+0000 7f7d9307c000 20 client.11688 trim_cache size 0 max 16384 2023-11-06T03:19:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.589+0000 7f7d9307c000 1 Processor -- start 2023-11-06T03:19:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.589+0000 7f7d9307c000 1 -- start start 2023-11-06T03:19:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de834f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7d96e0 0x55d6de8354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.598 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de29c940 con 0x55d6de7da000 2023-11-06T03:19:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de7ef670 con 0x55d6de7d96e0 2023-11-06T03:19:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5a1960 con 0x55d6de78fa90 2023-11-06T03:19:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.599 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46022/0 (socket says 172.21.15.157:46022) 2023-11-06T03:19:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/1724082954 learned_addr learned my addr 172.21.15.157:0/1724082954 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de834f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7d96e0 0x55d6de8354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 msgr2=0x55d6de834f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.600 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de834f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7d96e0 msgr2=0x55d6de8354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7d96e0 0x55d6de8354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 -- 172.21.15.157:0/1724082954 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6de58a330 con 0x55d6de7da000 2023-11-06T03:19:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f7d8001a610 tx=0x7f7d80004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94928077ea53a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.601 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d6bfff640 1 -- 172.21.15.157:0/1724082954 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80009510 con 0x55d6de7da000 2023-11-06T03:19:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d6bfff640 1 -- 172.21.15.157:0/1724082954 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800a5ba0 con 0x55d6de7da000 2023-11-06T03:19:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d6bfff640 1 -- 172.21.15.157:0/1724082954 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800b7430 con 0x55d6de7da000 2023-11-06T03:19:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/1724082954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 msgr2=0x55d6de835a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.602 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f7d8001a610 tx=0x7f7d80004a50 comp rx=0 tx=0).stop 2023-11-06T03:19:44.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/1724082954 shutdown_connections 2023-11-06T03:19:44.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7da000 0x55d6de835a00 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7d96e0 0x55d6de8354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- 172.21.15.157:0/1724082954 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de834f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.603 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/1724082954 >> 172.21.15.157:0/1724082954 conn(0x55d6de795bf0 msgr2=0x55d6de794ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/1724082954 shutdown_connections 2023-11-06T03:19:44.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/1724082954 wait complete. 2023-11-06T03:19:44.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- start start 2023-11-06T03:19:44.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 Processor -- start 2023-11-06T03:19:44.604 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de7d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de7f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5a2ff0 con 0x55d6de7d96e0 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5a67d0 con 0x55d6de7da000 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6de5dafd0 con 0x55d6de78fa90 2023-11-06T03:19:44.605 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 10 client.0 _ll_register_callbacks cb 0x55d6de75b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-06T03:19:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46030/0 (socket says 172.21.15.157:46030) 2023-11-06T03:19:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/793839107 learned_addr learned my addr 172.21.15.157:0/793839107 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:44.606 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de7f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de7d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 msgr2=0x55d6de7d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de7d81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 msgr2=0x55d6de7f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:44.607 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de7f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 -- 172.21.15.157:0/793839107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8806c800 con 0x55d6de7d96e0 2023-11-06T03:19:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8e4f1640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de7f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d8d4ef640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f7d78003d20 tx=0x7f7d78013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d7e85cf858ea3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2023-11-06T03:19:44.608 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78003800 con 0x55d6de7d96e0 2023-11-06T03:19:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mon.0 v2:172.21.15.157:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f7d78034cb0 con 0x55d6de7d96e0 2023-11-06T03:19:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.593+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6de5aeb90 con 0x55d6de7d96e0 2023-11-06T03:19:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d78034e70 con 0x55d6de7d96e0 2023-11-06T03:19:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 10 client.11694 fetch_fsmap learned FSMap version 136 2023-11-06T03:19:44.609 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55d6de2dbcc0 con 0x55d6de7d96e0 2023-11-06T03:19:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mon.0 v2:172.21.15.157:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6de5aeb90 con 0x55d6de7d96e0 2023-11-06T03:19:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mon.0 v2:172.21.15.157:3300/0 5 ==== fsmap(e 136) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7d7803d470 con 0x55d6de7d96e0 2023-11-06T03:19:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 10 client.11694 fetch_fsmap finished waiting for FSMap version 136 2023-11-06T03:19:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 10 client.11694 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-06T03:19:44.610 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 20 client.11694 populate_metadata read hostname 'smithi157' 2023-11-06T03:19:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de7f59f0 0x55d6de842220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 4 client.11694 mds_command: new command op to 11487 tid=0[{"prefix": "status"}] 2023-11-06T03:19:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 --> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d6de5d80e0 con 0x55d6de7f59f0 2023-11-06T03:19:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de7f59f0 0x55d6de842220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:44.611 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 10 client.11694 ms_handle_connect on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:44.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8dcf0640 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de7f59f0 0x55d6de842220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76eba2d5f4725616 server_cookie=e1887ee60a6819f6 in_seq=0 out_seq=0 2023-11-06T03:19:44.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 1 -- 172.21.15.157:0/793839107 <== mds.0 v2:172.21.15.157:6834/2374994036 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55d6de5d80e0 con 0x55d6de7f59f0 2023-11-06T03:19:44.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 10 client.11694 handle_command_reply: tid=0 2023-11-06T03:19:44.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 client.11694 shutdown 2023-11-06T03:19:44.612 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 msgr2=0x55d6de7f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f7d78003d20 tx=0x7f7d78013000 comp rx=0 tx=0).stop 2023-11-06T03:19:44.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 shutdown_connections 2023-11-06T03:19:44.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:6834/2374994036,v1:172.21.15.157:6835/2374994036] conn(0x55d6de7f59f0 0x55d6de842220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6de7da000 0x55d6de7f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d6de7d96e0 0x55d6de7f4f70 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d68ff9640 0 client.11694 ms_handle_reset on v2:172.21.15.157:6834/2374994036 2023-11-06T03:19:44.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 --2- 172.21.15.157:0/793839107 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55d6de78fa90 0x55d6de7d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 >> 172.21.15.157:0/793839107 conn(0x55d6de795bf0 msgr2=0x55d6de7d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 shutdown_connections 2023-11-06T03:19:44.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 1 -- 172.21.15.157:0/793839107 wait complete. 2023-11-06T03:19:44.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d9307c000 20 client.11694 trim_cache size 0 max 16384 2023-11-06T03:19:44.615 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:44.615 INFO:teuthology.orchestra.run.smithi157.stdout: "cluster_fsid": "eca2e009-f15d-40e2-b739-48eaa1ef5309", 2023-11-06T03:19:44.616 INFO:teuthology.orchestra.run.smithi157.stdout: "whoami": 0, 2023-11-06T03:19:44.616 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 11487, 2023-11-06T03:19:44.616 INFO:teuthology.orchestra.run.smithi157.stdout: "want_state": "up:active", 2023-11-06T03:19:44.616 INFO:teuthology.orchestra.run.smithi157.stdout: "state": "up:active", 2023-11-06T03:19:44.617 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs", 2023-11-06T03:19:44.617 INFO:teuthology.orchestra.run.smithi157.stdout: "rank_uptime": 66.660012459000001, 2023-11-06T03:19:44.617 INFO:teuthology.orchestra.run.smithi157.stdout: "mdsmap_epoch": 136, 2023-11-06T03:19:44.617 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch": 192, 2023-11-06T03:19:44.617 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_epoch_barrier": 190, 2023-11-06T03:19:44.618 INFO:teuthology.orchestra.run.smithi157.stdout: "uptime": 67.646554346000002 2023-11-06T03:19:44.618 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:44.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 msgr2=0x7f7d6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000f70 tx=0x7f7d780079c0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 msgr2=0x7f7d8814ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400d800 tx=0x7f7d8400dcc0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 shutdown_connections 2023-11-06T03:19:44.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d6c065a60 0x7f7d6c067f20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d881599c0 0x7f7d8806f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d880b2af0 0x7f7d880ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 --2- 172.21.15.157:0/778497149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d8806d000 0x7f7d8814ce20 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:44.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 >> 172.21.15.157:0/778497149 conn(0x7f7d88065930 msgr2=0x7f7d88065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:44.621 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 shutdown_connections 2023-11-06T03:19:44.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:44.597+0000 7f7d8ff54640 1 -- 172.21.15.157:0/778497149 wait complete. 2023-11-06T03:19:44.622 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-06T03:19:44.623 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:19:44.671 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-06T03:19:44.722 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-06T03:19:44.723 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:19:44.723 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:44.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:19:44.730 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:19:44.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:44.731 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-06T03:19:44.777 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-06T03:19:44.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:19:44.779 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:19:44.779 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:44.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:19:44.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:19:44.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:44.787 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-06T03:19:44.833 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-06T03:19:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:19:44.834 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:19:44.835 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:44.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:19:44.841 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:19:44.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:44.842 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-06T03:19:44.890 INFO:teuthology.orchestra.run.smithi157.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-06T03:19:44.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:19:44.891 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:19:44.892 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:44.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:19:44.898 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:19:44.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:44.899 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-06T03:19:44.944 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-06T03:19:44.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:19:44.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:19:45.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 Processor -- start 2023-11-06T03:19:45.074 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 -- start start 2023-11-06T03:19:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb3815cdf0 0x7fbb380d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbb380cfdc0 0x7fbb380b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380d1c60 con 0x7fbb380d1720 2023-11-06T03:19:45.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb380d1f40 con 0x7fbb3815cdf0 2023-11-06T03:19:45.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb402d1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb380d2220 con 0x7fbb380cfdc0 2023-11-06T03:19:45.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb3815cdf0 0x7fbb380d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb3815cdf0 0x7fbb380d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46046/0 (socket says 172.21.15.157:46046) 2023-11-06T03:19:45.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 -- 172.21.15.157:0/133305329 learned_addr learned my addr 172.21.15.157:0/133305329 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:45.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3d86c640 1 --1- 172.21.15.157:0/133305329 >> v1:172.21.15.157:6789/0 conn(0x7fbb380d1720 0x7fbb380cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34450/0 (socket says 172.21.15.157:34450) 2023-11-06T03:19:45.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 -- 172.21.15.157:0/133305329 >> v2:172.21.15.195:3300/0 conn(0x7fbb380cfdc0 msgr2=0x7fbb380b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 --2- 172.21.15.157:0/133305329 >> v2:172.21.15.195:3300/0 conn(0x7fbb380cfdc0 0x7fbb380b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 -- 172.21.15.157:0/133305329 >> v1:172.21.15.157:6789/0 conn(0x7fbb380d1720 legacy=0x7fbb380cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.078 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.081+0000 7fbb3e06d640 1 -- 172.21.15.157:0/133305329 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380d2500 con 0x7fbb3815cdf0 2023-11-06T03:19:45.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb3e06d640 1 --2- 172.21.15.157:0/133305329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380d11e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009870 tx=0x7fbb3401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a7ae037e85606e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb3d06b640 1 -- 172.21.15.157:0/133305329 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340aa650 con 0x7fbb3815cdf0 2023-11-06T03:19:45.079 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb3d06b640 1 -- 172.21.15.157:0/133305329 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb340aa7b0 con 0x7fbb3815cdf0 2023-11-06T03:19:45.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb3d06b640 1 -- 172.21.15.157:0/133305329 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340aab40 con 0x7fbb3815cdf0 2023-11-06T03:19:45.080 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 -- 172.21.15.157:0/133305329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 msgr2=0x7fbb380d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 --2- 172.21.15.157:0/133305329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380d11e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009870 tx=0x7fbb3401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 -- 172.21.15.157:0/133305329 shutdown_connections 2023-11-06T03:19:45.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 --2- 172.21.15.157:0/133305329 >> v2:172.21.15.195:3300/0 conn(0x7fbb380cfdc0 0x7fbb380b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.081 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 --2- 172.21.15.157:0/133305329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380d11e0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 -- 172.21.15.157:0/133305329 >> 172.21.15.157:0/133305329 conn(0x7fbb38065930 msgr2=0x7fbb38065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 -- 172.21.15.157:0/133305329 shutdown_connections 2023-11-06T03:19:45.082 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.085+0000 7fbb402d1640 1 -- 172.21.15.157:0/133305329 wait complete. 2023-11-06T03:19:45.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 Processor -- start 2023-11-06T03:19:45.084 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- start start 2023-11-06T03:19:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb380cfdc0 0x7fbb380c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb380d1720 0x7fbb380c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c6b60 con 0x7fbb3815cdf0 2023-11-06T03:19:45.085 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb381518e0 con 0x7fbb380cfdc0 2023-11-06T03:19:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38151600 con 0x7fbb380d1720 2023-11-06T03:19:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46048/0 (socket says 172.21.15.157:46048) 2023-11-06T03:19:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 -- 172.21.15.157:0/1054168374 learned_addr learned my addr 172.21.15.157:0/1054168374 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:45.086 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3d86c640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb380d1720 0x7fbb380c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e06d640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb380cfdc0 0x7fbb380c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 -- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb380d1720 msgr2=0x7fbb380c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb380d1720 0x7fbb380c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.087 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 -- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb380cfdc0 msgr2=0x7fbb380c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb380cfdc0 0x7fbb380c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 -- 172.21.15.157:0/1054168374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340094d0 con 0x7fbb3815cdf0 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e86e640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000b9e0 tx=0x7fbb3000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000cb20 con 0x7fbb3815cdf0 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3000cca0 con 0x7fbb3815cdf0 2023-11-06T03:19:45.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3814bd40 con 0x7fbb3815cdf0 2023-11-06T03:19:45.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30014680 con 0x7fbb3815cdf0 2023-11-06T03:19:45.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3814c300 con 0x7fbb3815cdf0 2023-11-06T03:19:45.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3814c520 con 0x7fbb3815cdf0 2023-11-06T03:19:45.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbb3001d3b0 con 0x7fbb3815cdf0 2023-11-06T03:19:45.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 0x7fbb1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.090 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fbb30083200 con 0x7fbb3815cdf0 2023-11-06T03:19:45.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.089+0000 7fbb3e06d640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 0x7fbb1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.093+0000 7fbb3e06d640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 0x7fbb1c067f20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbb340ae000 tx=0x7fbb34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.093+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fbb30050a40 con 0x7fbb3815cdf0 2023-11-06T03:19:45.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.265+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb38072750 con 0x7fbb3815cdf0 2023-11-06T03:19:45.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb2effd640 1 -- 172.21.15.157:0/1054168374 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7fbb30054060 con 0x7fbb3815cdf0 2023-11-06T03:19:45.262 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:45.263 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:19:45.263 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:19:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 msgr2=0x7fbb1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.264 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 0x7fbb1c067f20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbb340ae000 tx=0x7fbb34008040 comp rx=0 tx=0).stop 2023-11-06T03:19:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 msgr2=0x7fbb380c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000b9e0 tx=0x7fbb3000bea0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 shutdown_connections 2023-11-06T03:19:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fbb1c065a60 0x7fbb1c067f20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.265 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb3815cdf0 0x7fbb380c5930 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb380d1720 0x7fbb380c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 --2- 172.21.15.157:0/1054168374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb380cfdc0 0x7fbb380c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 >> 172.21.15.157:0/1054168374 conn(0x7fbb38065930 msgr2=0x7fbb380b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 shutdown_connections 2023-11-06T03:19:45.266 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.269+0000 7fbb402d1640 1 -- 172.21.15.157:0/1054168374 wait complete. 2023-11-06T03:19:45.281 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:19:45.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.397+0000 7f53252c8640 1 Processor -- start 2023-11-06T03:19:45.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.397+0000 7f53252c8640 1 -- start start 2023-11-06T03:19:45.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53252c8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f53201551e0 0x7f532006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53252c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f532015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53252c8640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015f580 con 0x7f532015e5e0 2023-11-06T03:19:45.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53252c8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532015efc0 con 0x7f53201551e0 2023-11-06T03:19:45.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53252c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532015f2a0 con 0x7f53200b3b20 2023-11-06T03:19:45.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53167fc640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f532015e5e0 0x7f53200b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34458/0 (socket says 172.21.15.157:34458) 2023-11-06T03:19:45.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53167fc640 1 -- 172.21.15.157:0/1152165632 learned_addr learned my addr 172.21.15.157:0/1152165632 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531effd640 1 --2- 172.21.15.157:0/1152165632 >> v2:172.21.15.195:3300/0 conn(0x7f53201551e0 0x7f532006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f532015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192688256 0 0) 0x7f532015f580 con 0x7f532015e5e0 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c003530 con 0x7f532015e5e0 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141478621 0 0) 0x7f530c003530 con 0x7f532015e5e0 2023-11-06T03:19:45.397 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 msgr2=0x7f532015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f532015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 >> v2:172.21.15.195:3300/0 conn(0x7f53201551e0 msgr2=0x7f532006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 --2- 172.21.15.157:0/1152165632 >> v2:172.21.15.195:3300/0 conn(0x7f53201551e0 0x7f532006fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.398 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532006f5a0 con 0x7f532015e5e0 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f5308001680 con 0x7f532015e5e0 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 >> v1:172.21.15.157:6789/0 conn(0x7f532015e5e0 legacy=0x7f53200b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f530c003980 0x7f530c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f530c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f530c004d10 0x7f530c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.399 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532015efc0 con 0x7f530c004d10 2023-11-06T03:19:45.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532015f2a0 con 0x7f530c003980 2023-11-06T03:19:45.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00b9c0 con 0x7f53200b3b20 2023-11-06T03:19:45.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f530c004d10 0x7f530c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531effd640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f530c003980 0x7f530c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.400 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f53167fc640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f530c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 -- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 msgr2=0x7f530c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f53200b3b20 0x7f530c0047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 -- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f530c003980 msgr2=0x7f530c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f530c003980 0x7f530c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.401+0000 7f531f7fe640 1 -- 172.21.15.157:0/1152165632 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c003530 con 0x7f530c004d10 2023-11-06T03:19:45.404 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f530c004d10 0x7f530c005190 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f5318009f90 tx=0x7f531801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313db3314e88d69e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180a5c30 con 0x7f530c004d10 2023-11-06T03:19:45.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531800f040 con 0x7f530c004d10 2023-11-06T03:19:45.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531e7fc640 1 -- 172.21.15.157:0/1152165632 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180ad440 con 0x7f530c004d10 2023-11-06T03:19:45.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/1152165632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f530c004d10 msgr2=0x7f530c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 --2- 172.21.15.157:0/1152165632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f530c004d10 0x7f530c005190 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f5318009f90 tx=0x7f531801cc20 comp rx=0 tx=0).stop 2023-11-06T03:19:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 -- 172.21.15.157:0/1152165632 reap_dead start 2023-11-06T03:19:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/1152165632 shutdown_connections 2023-11-06T03:19:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/1152165632 >> 172.21.15.157:0/1152165632 conn(0x7f5320065930 msgr2=0x7f5320065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/1152165632 shutdown_connections 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/1152165632 wait complete. 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 Processor -- start 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- start start 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f530c004d10 0x7f532014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5320147810 0x7f532006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201f6a80 con 0x7f5320147c90 2023-11-06T03:19:45.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201f5f00 con 0x7f5320147810 2023-11-06T03:19:45.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53201f61e0 con 0x7f530c004d10 2023-11-06T03:19:45.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46066/0 (socket says 172.21.15.157:46066) 2023-11-06T03:19:45.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 -- 172.21.15.157:0/71620447 learned_addr learned my addr 172.21.15.157:0/71620447 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:45.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531effd640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f530c004d10 0x7f532014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53167fc640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5320147810 0x7f532006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 -- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f530c004d10 msgr2=0x7f532014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.409 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f530c004d10 0x7f532014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 -- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5320147810 msgr2=0x7f532006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5320147810 0x7f532006ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 -- 172.21.15.157:0/71620447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53201f64c0 con 0x7f5320147c90 2023-11-06T03:19:45.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f531f7fe640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f53180b0040 tx=0x7f5318009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180039a0 con 0x7f5320147c90 2023-11-06T03:19:45.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318003b00 con 0x7f5320147c90 2023-11-06T03:19:45.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53201f67a0 con 0x7f5320147c90 2023-11-06T03:19:45.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180ad650 con 0x7f5320147c90 2023-11-06T03:19:45.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.405+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532006d1f0 con 0x7f5320147c90 2023-11-06T03:19:45.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5320072750 con 0x7f5320147c90 2023-11-06T03:19:45.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53180b4040 con 0x7f5320147c90 2023-11-06T03:19:45.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f5317fff640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 0x7f5300067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f531811cef0 con 0x7f5320147c90 2023-11-06T03:19:45.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f531effd640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 0x7f5300067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.409+0000 7f531effd640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 0x7f5300067f20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5310009a30 tx=0x7f5310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.413+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f53180ea750 con 0x7f5320147c90 2023-11-06T03:19:45.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.569+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53200cca60 con 0x7f5320147c90 2023-11-06T03:19:45.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f5317fff640 1 -- 172.21.15.157:0/71620447 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+18156 (secure 0 0 0) 0x7f53180bc090 con 0x7f5320147c90 2023-11-06T03:19:45.566 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:45.567 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":192,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:51.597595+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.26","mappings":[{"from":6,"to":1}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/877366447":"2023-11-07T03:18:36.902417+0000","172.21.15.157:6834/877366447":"2023-11-07T03:18:36.902417+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:19:45.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 msgr2=0x7f5300067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 0x7f5300067f20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5310009a30 tx=0x7f5310008040 comp rx=0 tx=0).stop 2023-11-06T03:19:45.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 msgr2=0x7f53201599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f53180b0040 tx=0x7f5318009750 comp rx=0 tx=0).stop 2023-11-06T03:19:45.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 shutdown_connections 2023-11-06T03:19:45.569 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5300065a60 0x7f5300067f20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5320147c90 0x7f53201599d0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5320147810 0x7f532006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 --2- 172.21.15.157:0/71620447 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f530c004d10 0x7f532014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 >> 172.21.15.157:0/71620447 conn(0x7f5320065930 msgr2=0x7f53200b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 shutdown_connections 2023-11-06T03:19:45.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.573+0000 7f53252c8640 1 -- 172.21.15.157:0/71620447 wait complete. 2023-11-06T03:19:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-06T03:03:55.172089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-06T03:04:00.991075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-06T03:18:16.644094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-11-06T03:18:17.084366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-06T03:19:45.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-06T03:19:45.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 Processor -- start 2023-11-06T03:19:45.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 -- start start 2023-11-06T03:19:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c6c06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c066750 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c15e910 con 0x7f1c6c0b4ac0 2023-11-06T03:19:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c741e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c15ebf0 con 0x7f1c6c1444b0 2023-11-06T03:19:45.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f1c6c15c3b0 0x7f1c6c066040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34460/0 (socket says 172.21.15.157:34460) 2023-11-06T03:19:45.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 -- 172.21.15.157:0/4133477310 learned_addr learned my addr 172.21.15.157:0/4133477310 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:45.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948117619 0 0) 0x7f1c6c066750 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c58003530 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c71f82640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c6c06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.722 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2759181306 0 0) 0x7f1c58003530 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 msgr2=0x7f1c6c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c6c06c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 >> v1:172.21.15.195:6789/0 conn(0x7f1c6c0b4ac0 legacy=0x7f1c6c065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c15eed0 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f1c64001a00 con 0x7f1c6c15c3b0 2023-11-06T03:19:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c71f82640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c6c06c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:19:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 >> v1:172.21.15.157:6789/0 conn(0x7f1c6c15c3b0 legacy=0x7f1c6c066040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c58003fa0 0x7f1c580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c58004d10 0x7f1c58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c15ebf0 con 0x7f1c58004d10 2023-11-06T03:19:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5800bb80 con 0x7f1c58003fa0 2023-11-06T03:19:45.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5800bd40 con 0x7f1c6c1444b0 2023-11-06T03:19:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c58004d10 0x7f1c58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c71f82640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c71781640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c58003fa0 0x7f1c580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 -- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 msgr2=0x7f1c580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.726 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c1444b0 0x7f1c580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 -- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c58003fa0 msgr2=0x7f1c580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c58003fa0 0x7f1c580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.725+0000 7f1c72783640 1 -- 172.21.15.157:0/4133477310 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58003530 con 0x7f1c58004d10 2023-11-06T03:19:45.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c72783640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c58004d10 0x7f1c58005190 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400a830 tx=0x7f1c6401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dac7254725c02ab server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640aa6d0 con 0x7f1c58004d10 2023-11-06T03:19:45.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640aa830 con 0x7f1c58004d10 2023-11-06T03:19:45.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c70f80640 1 -- 172.21.15.157:0/4133477310 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640aac60 con 0x7f1c58004d10 2023-11-06T03:19:45.731 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- 172.21.15.157:0/4133477310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c58004d10 msgr2=0x7f1c58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 --2- 172.21.15.157:0/4133477310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c58004d10 0x7f1c58005190 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400a830 tx=0x7f1c6401faa0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c72783640 1 -- 172.21.15.157:0/4133477310 reap_dead start 2023-11-06T03:19:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- 172.21.15.157:0/4133477310 shutdown_connections 2023-11-06T03:19:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- 172.21.15.157:0/4133477310 >> 172.21.15.157:0/4133477310 conn(0x7f1c6c069d70 msgr2=0x7f1c6c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.732 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- 172.21.15.157:0/4133477310 shutdown_connections 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- 172.21.15.157:0/4133477310 wait complete. 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 Processor -- start 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- start start 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c15a630 0x7f1c6c139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.733 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c6c139b90 0x7f1c6c142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c1400a0 con 0x7f1c6c15aa10 2023-11-06T03:19:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c140360 con 0x7f1c6c139b90 2023-11-06T03:19:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c741e6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c140640 con 0x7f1c6c15a630 2023-11-06T03:19:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c71781640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.734 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c71781640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46078/0 (socket says 172.21.15.157:46078) 2023-11-06T03:19:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c71781640 1 -- 172.21.15.157:0/1258399547 learned_addr learned my addr 172.21.15.157:0/1258399547 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.729+0000 7f1c71f82640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c15a630 0x7f1c6c139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c72783640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c6c139b90 0x7f1c6c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 -- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c15a630 msgr2=0x7f1c6c139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c15a630 0x7f1c6c139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 -- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c6c139b90 msgr2=0x7f1c6c142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c6c139b90 0x7f1c6c142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 -- 172.21.15.157:0/1258399547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c140920 con 0x7f1c6c15aa10 2023-11-06T03:19:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71781640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e4b0 tx=0x7f1c5c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.736 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0052e0 con 0x7f1c6c15aa10 2023-11-06T03:19:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c005440 con 0x7f1c6c15aa10 2023-11-06T03:19:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c140c00 con 0x7f1c6c15aa10 2023-11-06T03:19:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c003820 con 0x7f1c6c15aa10 2023-11-06T03:19:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c137650 con 0x7f1c6c15aa10 2023-11-06T03:19:45.737 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c072750 con 0x7f1c6c15aa10 2023-11-06T03:19:45.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1c5c003980 con 0x7f1c6c15aa10 2023-11-06T03:19:45.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:45.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f1c5c083b40 con 0x7f1c6c15aa10 2023-11-06T03:19:45.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.733+0000 7f1c71f82640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:45.738 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.737+0000 7f1c71f82640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 0x7f1c50067f20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800e880 tx=0x7f1c68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:45.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.737+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1c5c0513d0 con 0x7f1c6c15aa10 2023-11-06T03:19:45.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.893+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c6c0d00f0 con 0x7f1c6c15aa10 2023-11-06T03:19:45.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.893+0000 7f1c627fc640 1 -- 172.21.15.157:0/1258399547 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c5c0549f0 con 0x7f1c6c15aa10 2023-11-06T03:19:45.888 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 1 2023-11-06T03:19:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 msgr2=0x7f1c50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 0x7f1c50067f20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800e880 tx=0x7f1c68008040 comp rx=0 tx=0).stop 2023-11-06T03:19:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 msgr2=0x7f1c6c1378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e4b0 tx=0x7f1c5c00e970 comp rx=0 tx=0).stop 2023-11-06T03:19:45.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 shutdown_connections 2023-11-06T03:19:45.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1c50065a60 0x7f1c50067f20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c6c139b90 0x7f1c6c142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c6c15aa10 0x7f1c6c1378d0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 --2- 172.21.15.157:0/1258399547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1c6c15a630 0x7f1c6c139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:45.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 >> 172.21.15.157:0/1258399547 conn(0x7f1c6c069d70 msgr2=0x7f1c6c0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:45.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 shutdown_connections 2023-11-06T03:19:45.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:45.897+0000 7f1c741e6640 1 -- 172.21.15.157:0/1258399547 wait complete. 2023-11-06T03:19:45.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-06T03:19:46.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 Processor -- start 2023-11-06T03:19:46.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 -- start start 2023-11-06T03:19:46.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4f940b6190 0x7f4f940cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 0x7f4f940d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9415bbd0 con 0x7f4f9415b6a0 2023-11-06T03:19:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f940b6570 con 0x7f4f940b6190 2023-11-06T03:19:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9b7e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9415beb0 con 0x7f4f940cdad0 2023-11-06T03:19:46.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f98d7e640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4f9415b6a0 0x7f4f940ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34476/0 (socket says 172.21.15.157:34476) 2023-11-06T03:19:46.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f98d7e640 1 -- 172.21.15.157:0/2081146966 learned_addr learned my addr 172.21.15.157:0/2081146966 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:46.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 --2- 172.21.15.157:0/2081146966 >> v2:172.21.15.195:3300/0 conn(0x7f4f940b6190 0x7f4f940cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f99d80640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 0x7f4f940d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807026270 0 0) 0x7f4f9415bbd0 con 0x7f4f9415b6a0 2023-11-06T03:19:46.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f80003530 con 0x7f4f9415b6a0 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2188373245 0 0) 0x7f4f80003530 con 0x7f4f9415b6a0 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 msgr2=0x7f4f940d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 0x7f4f940d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 >> v2:172.21.15.195:3300/0 conn(0x7f4f940b6190 msgr2=0x7f4f940cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 --2- 172.21.15.157:0/2081146966 >> v2:172.21.15.195:3300/0 conn(0x7f4f940b6190 0x7f4f940cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9415c190 con 0x7f4f9415b6a0 2023-11-06T03:19:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f4f84001680 con 0x7f4f9415b6a0 2023-11-06T03:19:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 >> v1:172.21.15.157:6789/0 conn(0x7f4f9415b6a0 legacy=0x7f4f940ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f80003980 0x7f4f80003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 0x7f4f800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f80004d10 0x7f4f80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940b6570 con 0x7f4f80003980 2023-11-06T03:19:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9415beb0 con 0x7f4f80004d10 2023-11-06T03:19:46.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8000b990 con 0x7f4f940cdad0 2023-11-06T03:19:46.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f80003980 0x7f4f80003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 -- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 msgr2=0x7f4f800047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940cdad0 0x7f4f800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f99d80640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f80004d10 0x7f4f80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.035 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 -- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f80004d10 msgr2=0x7f4f80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f80004d10 0x7f4f80005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.036 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.033+0000 7f4f9957f640 1 -- 172.21.15.157:0/2081146966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80003530 con 0x7f4f80003980 2023-11-06T03:19:46.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9957f640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f80003980 0x7f4f80003da0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009ae0 tx=0x7f4f9001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de70d20188172f3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.039 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900a55c0 con 0x7f4f80003980 2023-11-06T03:19:46.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f900a5720 con 0x7f4f80003980 2023-11-06T03:19:46.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f8bfff640 1 -- 172.21.15.157:0/2081146966 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900a5ad0 con 0x7f4f80003980 2023-11-06T03:19:46.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/2081146966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f80003980 msgr2=0x7f4f80003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/2081146966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f80003980 0x7f4f80003da0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009ae0 tx=0x7f4f9001ca60 comp rx=0 tx=0).stop 2023-11-06T03:19:46.040 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f99d80640 1 -- 172.21.15.157:0/2081146966 reap_dead start 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/2081146966 shutdown_connections 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/2081146966 >> 172.21.15.157:0/2081146966 conn(0x7f4f940b23e0 msgr2=0x7f4f940b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/2081146966 shutdown_connections 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/2081146966 wait complete. 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 Processor -- start 2023-11-06T03:19:46.041 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- start start 2023-11-06T03:19:46.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f94153950 0x7f4f941593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940b48e0 0x7f4f941f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940b4d70 con 0x7f4f941598f0 2023-11-06T03:19:46.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940c94f0 con 0x7f4f94153950 2023-11-06T03:19:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9b7e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94154630 con 0x7f4f940b48e0 2023-11-06T03:19:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46098/0 (socket says 172.21.15.157:46098) 2023-11-06T03:19:46.043 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 -- 172.21.15.157:0/3152478772 learned_addr learned my addr 172.21.15.157:0/3152478772 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f9957f640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f94153950 0x7f4f941593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 -- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940b48e0 msgr2=0x7f4f941f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f99d80640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940b48e0 0x7f4f941f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940b48e0 0x7f4f941f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 -- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f94153950 msgr2=0x7f4f941593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.044 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.037+0000 7f4f98d7e640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f94153950 0x7f4f941593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f98d7e640 1 -- 172.21.15.157:0/3152478772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94156050 con 0x7f4f941598f0 2023-11-06T03:19:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f98d7e640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c6b0 tx=0x7f4f8400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84004ab0 con 0x7f4f941598f0 2023-11-06T03:19:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84004c50 con 0x7f4f941598f0 2023-11-06T03:19:46.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f940c8c50 con 0x7f4f941598f0 2023-11-06T03:19:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f840070d0 con 0x7f4f941598f0 2023-11-06T03:19:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f940c9210 con 0x7f4f941598f0 2023-11-06T03:19:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f94072750 con 0x7f4f941598f0 2023-11-06T03:19:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4f84007230 con 0x7f4f941598f0 2023-11-06T03:19:46.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 0x7f4f78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f4f840828b0 con 0x7f4f941598f0 2023-11-06T03:19:46.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f9957f640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 0x7f4f78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.041+0000 7f4f9957f640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 0x7f4f78067f20 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.045+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f4f84050060 con 0x7f4f941598f0 2023-11-06T03:19:46.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4f9406c460 con 0x7f4f941598f0 2023-11-06T03:19:46.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f89ffb640 1 -- 172.21.15.157:0/3152478772 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v192) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f84018020 con 0x7f4f941598f0 2023-11-06T03:19:46.198 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 8 2023-11-06T03:19:46.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 msgr2=0x7f4f78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 0x7f4f78067f20 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4f900af000 tx=0x7f4f90008040 comp rx=0 tx=0).stop 2023-11-06T03:19:46.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 msgr2=0x7f4f940b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c6b0 tx=0x7f4f8400cb70 comp rx=0 tx=0).stop 2023-11-06T03:19:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 shutdown_connections 2023-11-06T03:19:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f4f78065a60 0x7f4f78067f20 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4f940b48e0 0x7f4f941f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f941598f0 0x7f4f940b43a0 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 --2- 172.21.15.157:0/3152478772 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4f94153950 0x7f4f941593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 >> 172.21.15.157:0/3152478772 conn(0x7f4f940b23e0 msgr2=0x7f4f940ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 shutdown_connections 2023-11-06T03:19:46.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.205+0000 7f4f9b7e3640 1 -- 172.21.15.157:0/3152478772 wait complete. 2023-11-06T03:19:46.216 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-06T03:19:46.329 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 Processor -- start 2023-11-06T03:19:46.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 -- start start 2023-11-06T03:19:46.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90cc0d0280 0x7f90cc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0d2920 con 0x7f90cc15ce80 2023-11-06T03:19:46.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0d18a0 con 0x7f90cc0d0280 2023-11-06T03:19:46.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90d0a9c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d1b60 con 0x7f90cc15caa0 2023-11-06T03:19:46.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90cad9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f90cc0d0280 0x7f90cc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90c9d9b640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f90cc15ce80 0x7f90cc0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:51162/0 (socket says 172.21.15.157:51162) 2023-11-06T03:19:46.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90c9d9b640 1 -- 172.21.15.157:0/4081795573 learned_addr learned my addr 172.21.15.157:0/4081795573 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:46.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90cad9d640 1 -- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 msgr2=0x7f90cc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90cad9d640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90cad9d640 1 -- 172.21.15.157:0/4081795573 >> v1:172.21.15.195:6789/0 conn(0x7f90cc15ce80 legacy=0x7f90cc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90cad9d640 1 -- 172.21.15.157:0/4081795573 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0d1e40 con 0x7f90cc0d0280 2023-11-06T03:19:46.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.337+0000 7f90ca59c640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:19:46.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90cad9d640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc0d0280 0x7f90cc0b5d50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009d90 tx=0x7f90c401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41740375499faf server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90c959a640 1 -- 172.21.15.157:0/4081795573 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40a59e0 con 0x7f90cc0d0280 2023-11-06T03:19:46.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90c959a640 1 -- 172.21.15.157:0/4081795573 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c40a5b40 con 0x7f90cc0d0280 2023-11-06T03:19:46.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90c959a640 1 -- 172.21.15.157:0/4081795573 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40ad440 con 0x7f90cc0d0280 2023-11-06T03:19:46.336 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc0d0280 msgr2=0x7f90cc0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc0d0280 0x7f90cc0b5d50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009d90 tx=0x7f90c401c920 comp rx=0 tx=0).stop 2023-11-06T03:19:46.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- 172.21.15.157:0/4081795573 shutdown_connections 2023-11-06T03:19:46.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc0d0280 0x7f90cc0b5d50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/4081795573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- 172.21.15.157:0/4081795573 >> 172.21.15.157:0/4081795573 conn(0x7f90cc070790 msgr2=0x7f90cc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- 172.21.15.157:0/4081795573 shutdown_connections 2023-11-06T03:19:46.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- 172.21.15.157:0/4081795573 wait complete. 2023-11-06T03:19:46.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 Processor -- start 2023-11-06T03:19:46.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.341+0000 7f90d0a9c640 1 -- start start 2023-11-06T03:19:46.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc15ce80 0x7f90cc14bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15c5d0 con 0x7f90cc0d0280 2023-11-06T03:19:46.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15d800 con 0x7f90cc15ce80 2023-11-06T03:19:46.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15c870 con 0x7f90cc15caa0 2023-11-06T03:19:46.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46110/0 (socket says 172.21.15.157:46110) 2023-11-06T03:19:46.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 -- 172.21.15.157:0/2251645158 learned_addr learned my addr 172.21.15.157:0/2251645158 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 -- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 msgr2=0x7f90cc15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90c9d9b640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90cad9d640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc15ce80 0x7f90cc14bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 -- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc15ce80 msgr2=0x7f90cc14bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc15ce80 0x7f90cc14bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 -- 172.21.15.157:0/2251645158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009420 con 0x7f90cc0d0280 2023-11-06T03:19:46.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90ca59c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00ebe0 tx=0x7f90bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00eee0 con 0x7f90cc0d0280 2023-11-06T03:19:46.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00cc30 con 0x7f90cc0d0280 2023-11-06T03:19:46.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4009a70 con 0x7f90cc0d0280 2023-11-06T03:19:46.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc013470 con 0x7f90cc0d0280 2023-11-06T03:19:46.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc0ca070 con 0x7f90cc0d0280 2023-11-06T03:19:46.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc072750 con 0x7f90cc0d0280 2023-11-06T03:19:46.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f90bc0135d0 con 0x7f90cc0d0280 2023-11-06T03:19:46.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 0x7f90ac067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f90bc0836c0 con 0x7f90cc0d0280 2023-11-06T03:19:46.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90c9d9b640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 0x7f90ac067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.345+0000 7f90c9d9b640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 0x7f90ac067f50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0b61c0 tx=0x7f90c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.349+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f90bc050e70 con 0x7f90cc0d0280 2023-11-06T03:19:46.484 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.489+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90cc155af0 con 0x7f90cc0d0280 2023-11-06T03:19:46.485 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.489+0000 7f90baffd640 1 -- 172.21.15.157:0/2251645158 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v192) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90bc054490 con 0x7f90cc0d0280 2023-11-06T03:19:46.485 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 32 2023-11-06T03:19:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 msgr2=0x7f90ac067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 0x7f90ac067f50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0b61c0 tx=0x7f90c0009040 comp rx=0 tx=0).stop 2023-11-06T03:19:46.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 msgr2=0x7f90cc15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00ebe0 tx=0x7f90bc00c520 comp rx=0 tx=0).stop 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 shutdown_connections 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f90ac065a90 0x7f90ac067f50 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90cc15ce80 0x7f90cc14bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f90cc15caa0 0x7f90cc15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 --2- 172.21.15.157:0/2251645158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90cc0d0280 0x7f90cc15c090 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 >> 172.21.15.157:0/2251645158 conn(0x7f90cc070790 msgr2=0x7f90cc14c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 shutdown_connections 2023-11-06T03:19:46.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.493+0000 7f90d0a9c640 1 -- 172.21.15.157:0/2251645158 wait complete. 2023-11-06T03:19:46.502 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-06T03:19:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.617+0000 7f0195aae640 1 Processor -- start 2023-11-06T03:19:46.613 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.617+0000 7f0195aae640 1 -- start start 2023-11-06T03:19:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0195aae640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01900b2af0 0x7f01900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.614 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0195aae640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f019006d000 0x7f0190159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0195aae640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019015e440 con 0x7f01901599c0 2023-11-06T03:19:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0195aae640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019006c240 con 0x7f01900b2af0 2023-11-06T03:19:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0195aae640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019006c520 con 0x7f019006d000 2023-11-06T03:19:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f01900b2af0 0x7f01900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.615 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018f7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f01901599c0 0x7f01900b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:51174/0 (socket says 172.21.15.157:51174) 2023-11-06T03:19:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018f7fe640 1 -- 172.21.15.157:0/3285735148 learned_addr learned my addr 172.21.15.157:0/3285735148 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018ffff640 1 --2- 172.21.15.157:0/3285735148 >> v2:172.21.15.195:3300/0 conn(0x7f019006d000 0x7f0190159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 -- 172.21.15.157:0/3285735148 >> v2:172.21.15.195:3300/0 conn(0x7f019006d000 msgr2=0x7f0190159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 --2- 172.21.15.157:0/3285735148 >> v2:172.21.15.195:3300/0 conn(0x7f019006d000 0x7f0190159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.616 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 -- 172.21.15.157:0/3285735148 >> v1:172.21.15.195:6789/0 conn(0x7f01901599c0 legacy=0x7f01900b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.617 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 -- 172.21.15.157:0/3285735148 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019006c800 con 0x7f01900b2af0 2023-11-06T03:19:46.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f0187fff640 1 --2- 172.21.15.157:0/3285735148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b2af0 0x7f01900ae000 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f019006d930 tx=0x7f017801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b0458c848ba805 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018effd640 1 -- 172.21.15.157:0/3285735148 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01780ab650 con 0x7f01900b2af0 2023-11-06T03:19:46.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018effd640 1 -- 172.21.15.157:0/3285735148 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01780ab7b0 con 0x7f01900b2af0 2023-11-06T03:19:46.618 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.621+0000 7f018effd640 1 -- 172.21.15.157:0/3285735148 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01780abb40 con 0x7f01900b2af0 2023-11-06T03:19:46.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3285735148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b2af0 msgr2=0x7f01900ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- 172.21.15.157:0/3285735148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b2af0 0x7f01900ae000 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f019006d930 tx=0x7f017801c430 comp rx=0 tx=0).stop 2023-11-06T03:19:46.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3285735148 shutdown_connections 2023-11-06T03:19:46.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- 172.21.15.157:0/3285735148 >> v2:172.21.15.195:3300/0 conn(0x7f019006d000 0x7f0190159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.619 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- 172.21.15.157:0/3285735148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01900b2af0 0x7f01900ae000 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3285735148 >> 172.21.15.157:0/3285735148 conn(0x7f0190065930 msgr2=0x7f0190065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3285735148 shutdown_connections 2023-11-06T03:19:46.620 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3285735148 wait complete. 2023-11-06T03:19:46.622 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 Processor -- start 2023-11-06T03:19:46.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- start start 2023-11-06T03:19:46.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 0x7f019014c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 0x7f019014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.623 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190156980 con 0x7f01901599c0 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190155d00 con 0x7f01900b2af0 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0190155fe0 con 0x7f019006d000 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46138/0 (socket says 172.21.15.157:46138) 2023-11-06T03:19:46.624 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 -- 172.21.15.157:0/3427452557 learned_addr learned my addr 172.21.15.157:0/3427452557 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:46.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018f7fe640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 0x7f019014c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0187fff640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 0x7f019014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 -- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 msgr2=0x7f019014c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 0x7f019014c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.625 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 -- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 msgr2=0x7f019014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 0x7f019014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 -- 172.21.15.157:0/3427452557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0178009420 con 0x7f01901599c0 2023-11-06T03:19:46.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018f7fe640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 0x7f019014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:46.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0187fff640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 0x7f019014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:46.626 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018ffff640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f018800efc0 tx=0x7f018800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0188009ca0 con 0x7f01901599c0 2023-11-06T03:19:46.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018800f040 con 0x7f01901599c0 2023-11-06T03:19:46.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880146c0 con 0x7f01901599c0 2023-11-06T03:19:46.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01901562c0 con 0x7f01901599c0 2023-11-06T03:19:46.627 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.625+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019014c700 con 0x7f01901599c0 2023-11-06T03:19:46.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190156ac0 con 0x7f01901599c0 2023-11-06T03:19:46.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0188014820 con 0x7f01901599c0 2023-11-06T03:19:46.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f018cff9640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 0x7f0170067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f0188084220 con 0x7f01901599c0 2023-11-06T03:19:46.628 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f018f7fe640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 0x7f0170067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.629+0000 7f018f7fe640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 0x7f0170067f50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0180005890 tx=0x7f0180009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.629 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.633+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f0190156ac0 con 0x7f01901599c0 2023-11-06T03:19:46.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.781+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0190156ac0 con 0x7f01901599c0 2023-11-06T03:19:46.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.781+0000 7f018cff9640 1 -- 172.21.15.157:0/3427452557 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v192) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0190156ac0 con 0x7f01901599c0 2023-11-06T03:19:46.777 INFO:teuthology.orchestra.run.smithi157.stdout:pg_num: 64 2023-11-06T03:19:46.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 msgr2=0x7f0170067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 0x7f0170067f50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0180005890 tx=0x7f0180009040 comp rx=0 tx=0).stop 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 msgr2=0x7f019014d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f018800efc0 tx=0x7f018800c490 comp rx=0 tx=0).stop 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 shutdown_connections 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f0170065a90 0x7f0170067f50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01901599c0 0x7f019014d820 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01900b2af0 0x7f019014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 --2- 172.21.15.157:0/3427452557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f019006d000 0x7f019014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 >> 172.21.15.157:0/3427452557 conn(0x7f0190065930 msgr2=0x7f019006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 shutdown_connections 2023-11-06T03:19:46.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.785+0000 7f0195aae640 1 -- 172.21.15.157:0/3427452557 wait complete. 2023-11-06T03:19:46.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:19:46.901 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 Processor -- start 2023-11-06T03:19:46.902 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 -- start start 2023-11-06T03:19:46.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76680adae0 0x7f76680b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76680adf40 0x7f76680b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766806c490 con 0x7f7668156a70 2023-11-06T03:19:46.903 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766806b950 con 0x7f76680adae0 2023-11-06T03:19:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f766cf4f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766806fb80 con 0x7f76680adf40 2023-11-06T03:19:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76680adae0 0x7f76680b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f76680adae0 0x7f76680b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46154/0 (socket says 172.21.15.157:46154) 2023-11-06T03:19:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 -- 172.21.15.157:0/2714964068 learned_addr learned my addr 172.21.15.157:0/2714964068 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:46.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 -- 172.21.15.157:0/2714964068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76680adf40 msgr2=0x7f76680b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 --2- 172.21.15.157:0/2714964068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76680adf40 0x7f76680b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 -- 172.21.15.157:0/2714964068 >> v1:172.21.15.195:6789/0 conn(0x7f7668156a70 legacy=0x7f766806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:46.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 -- 172.21.15.157:0/2714964068 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766806bb60 con 0x7f76680adae0 2023-11-06T03:19:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f765ffff640 1 --2- 172.21.15.157:0/2714964068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adae0 0x7f76680b1c20 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f765000aa10 tx=0x7f765001d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1274ad610aa8ae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.909+0000 7f7665d9b640 1 -- 172.21.15.157:0/2714964068 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500a6890 con 0x7f76680adae0 2023-11-06T03:19:46.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f7665d9b640 1 -- 172.21.15.157:0/2714964068 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76500a69f0 con 0x7f76680adae0 2023-11-06T03:19:46.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f7665d9b640 1 -- 172.21.15.157:0/2714964068 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500a6d80 con 0x7f76680adae0 2023-11-06T03:19:46.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- 172.21.15.157:0/2714964068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adae0 msgr2=0x7f76680b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- 172.21.15.157:0/2714964068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adae0 0x7f76680b1c20 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f765000aa10 tx=0x7f765001d8a0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- 172.21.15.157:0/2714964068 shutdown_connections 2023-11-06T03:19:46.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- 172.21.15.157:0/2714964068 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76680adf40 0x7f76680b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- 172.21.15.157:0/2714964068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adae0 0x7f76680b1c20 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- 172.21.15.157:0/2714964068 >> 172.21.15.157:0/2714964068 conn(0x7f7668065930 msgr2=0x7f7668065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:46.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- 172.21.15.157:0/2714964068 shutdown_connections 2023-11-06T03:19:46.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- 172.21.15.157:0/2714964068 wait complete. 2023-11-06T03:19:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 Processor -- start 2023-11-06T03:19:46.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- start start 2023-11-06T03:19:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76680adae0 0x7f7668146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7668156a70 0x7f7668145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668146b40 con 0x7f76680adf40 2023-11-06T03:19:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668149460 con 0x7f76680adae0 2023-11-06T03:19:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f766cf4f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7668149740 con 0x7f7668156a70 2023-11-06T03:19:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46162/0 (socket says 172.21.15.157:46162) 2023-11-06T03:19:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 -- 172.21.15.157:0/32208534 learned_addr learned my addr 172.21.15.157:0/32208534 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 -- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7668156a70 msgr2=0x7f7668145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7668156a70 0x7f7668145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 -- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76680adae0 msgr2=0x7f7668146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:46.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76680adae0 0x7f7668146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 -- 172.21.15.157:0/32208534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765000a670 con 0x7f76680adf40 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.913+0000 7f765ffff640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f765000a1c0 tx=0x7f765001a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500b79d0 con 0x7f76680adf40 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76500b7b50 con 0x7f76680adf40 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500ae4c0 con 0x7f76680adf40 2023-11-06T03:19:46.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668149a20 con 0x7f76680adf40 2023-11-06T03:19:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7668149fe0 con 0x7f76680adf40 2023-11-06T03:19:46.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766814a1d0 con 0x7f76680adf40 2023-11-06T03:19:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f76500b5040 con 0x7f76680adf40 2023-11-06T03:19:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 0x7f7648067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f765011f8e0 con 0x7f76680adf40 2023-11-06T03:19:46.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f766659c640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 0x7f7648067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.917+0000 7f766659c640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 0x7f7648067f20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7658009650 tx=0x7f7658006bb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:46.917 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:46.921+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f76500ed1c0 con 0x7f76680adf40 2023-11-06T03:19:47.088 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.093+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7668072750 con 0x7f76680adf40 2023-11-06T03:19:47.089 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f765effd640 1 -- 172.21.15.157:0/32208534 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7f76500f07e0 con 0x7f76680adf40 2023-11-06T03:19:47.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:47.090 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:19:47.090 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:19:47.091 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 msgr2=0x7f7648067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 0x7f7648067f20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7658009650 tx=0x7f7658006bb0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 msgr2=0x7f76680afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f765000a1c0 tx=0x7f765001a640 comp rx=0 tx=0).stop 2023-11-06T03:19:47.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 shutdown_connections 2023-11-06T03:19:47.092 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7648065a60 0x7f7648067f20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7668156a70 0x7f7668145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.093 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76680adf40 0x7f76680afdd0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 --2- 172.21.15.157:0/32208534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76680adae0 0x7f7668146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 >> 172.21.15.157:0/32208534 conn(0x7f7668065930 msgr2=0x7f7668065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 shutdown_connections 2023-11-06T03:19:47.094 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.097+0000 7f766cf4f640 1 -- 172.21.15.157:0/32208534 wait complete. 2023-11-06T03:19:47.107 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:19:47.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.217+0000 7ff400c09640 1 Processor -- start 2023-11-06T03:19:47.213 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.217+0000 7ff400c09640 1 -- start start 2023-11-06T03:19:47.214 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3fc15ce80 0x7ff3fc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc0d2920 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc0d18a0 con 0x7ff3fc15caa0 2023-11-06T03:19:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc0d1b60 con 0x7ff3fc15ce80 2023-11-06T03:19:47.215 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46176/0 (socket says 172.21.15.157:46176) 2023-11-06T03:19:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 -- 172.21.15.157:0/875127658 learned_addr learned my addr 172.21.15.157:0/875127658 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3f9d9b640 1 --2- 172.21.15.157:0/875127658 >> v2:172.21.15.195:3300/0 conn(0x7ff3fc15ce80 0x7ff3fc0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.216 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 -- 172.21.15.157:0/875127658 >> v2:172.21.15.195:3300/0 conn(0x7ff3fc15ce80 msgr2=0x7ff3fc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 --2- 172.21.15.157:0/875127658 >> v2:172.21.15.195:3300/0 conn(0x7ff3fc15ce80 0x7ff3fc0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 -- 172.21.15.157:0/875127658 >> v1:172.21.15.195:6789/0 conn(0x7ff3fc0ce1a0 legacy=0x7ff3fc0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:47.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 -- 172.21.15.157:0/875127658 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc0d1e40 con 0x7ff3fc15caa0 2023-11-06T03:19:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3fa59c640 1 --2- 172.21.15.157:0/875127658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0c6720 tx=0x7ff3ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ddbdc558385253 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3f959a640 1 -- 172.21.15.157:0/875127658 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0ac600 con 0x7ff3fc15caa0 2023-11-06T03:19:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3f959a640 1 -- 172.21.15.157:0/875127658 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0a8050 con 0x7ff3fc15caa0 2023-11-06T03:19:47.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff3f959a640 1 -- 172.21.15.157:0/875127658 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0ac980 con 0x7ff3fc15caa0 2023-11-06T03:19:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- 172.21.15.157:0/875127658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc15caa0 msgr2=0x7ff3fc0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 --2- 172.21.15.157:0/875127658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0c6720 tx=0x7ff3ec01c490 comp rx=0 tx=0).stop 2023-11-06T03:19:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- 172.21.15.157:0/875127658 shutdown_connections 2023-11-06T03:19:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 --2- 172.21.15.157:0/875127658 >> v2:172.21.15.195:3300/0 conn(0x7ff3fc15ce80 0x7ff3fc0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 --2- 172.21.15.157:0/875127658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc0cdc60 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- 172.21.15.157:0/875127658 >> 172.21.15.157:0/875127658 conn(0x7ff3fc070790 msgr2=0x7ff3fc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- 172.21.15.157:0/875127658 shutdown_connections 2023-11-06T03:19:47.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.221+0000 7ff400c09640 1 -- 172.21.15.157:0/875127658 wait complete. 2023-11-06T03:19:47.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 Processor -- start 2023-11-06T03:19:47.222 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- start start 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc13aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3fc15ce80 0x7ff3fc13e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc13aa20 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc13a6f0 con 0x7ff3fc15caa0 2023-11-06T03:19:47.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c9d90 con 0x7ff3fc15ce80 2023-11-06T03:19:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46188/0 (socket says 172.21.15.157:46188) 2023-11-06T03:19:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 -- 172.21.15.157:0/3020221735 learned_addr learned my addr 172.21.15.157:0/3020221735 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 -- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3fc15ce80 msgr2=0x7ff3fc13e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3fc15ce80 0x7ff3fc13e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 -- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3fc15caa0 msgr2=0x7ff3fc13aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc13aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 -- 172.21.15.157:0/3020221735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0094d0 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3fa59c640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec01a0c0 tx=0x7ff3ec01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0b7620 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0b7780 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc0ca070 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0b7af0 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc0b4060 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc13a860 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.225+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff3ec0b3040 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.229+0000 7ff3eaffd640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 0x7ff3dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.229+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7ff3ec11d930 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.229+0000 7ff3f9d9b640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 0x7ff3dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.229+0000 7ff3f9d9b640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 0x7ff3dc067f20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc1f89e0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.228 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.233+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff3fc13a860 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.373+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3fc13a860 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.370 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff3eaffd640 1 -- 172.21.15.157:0/3020221735 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+18156 (secure 0 0 0) 0x7ff3fc13a860 con 0x7ff3fc0ce1a0 2023-11-06T03:19:47.370 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:47.371 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":192,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:51.597595+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.26","mappings":[{"from":6,"to":1}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/877366447":"2023-11-07T03:18:36.902417+0000","172.21.15.157:6834/877366447":"2023-11-07T03:18:36.902417+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:19:47.372 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 msgr2=0x7ff3dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 0x7ff3dc067f20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc1f89e0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2023-11-06T03:19:47.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 msgr2=0x7ff3fc0b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec01a0c0 tx=0x7ff3ec01acd0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 shutdown_connections 2023-11-06T03:19:47.373 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff3dc065a60 0x7ff3dc067f20 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3fc15ce80 0x7ff3fc13e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff3fc15caa0 0x7ff3fc13aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 --2- 172.21.15.157:0/3020221735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3fc0ce1a0 0x7ff3fc0b3b20 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 >> 172.21.15.157:0/3020221735 conn(0x7ff3fc070790 msgr2=0x7ff3fc0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.374 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 shutdown_connections 2023-11-06T03:19:47.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.377+0000 7ff400c09640 1 -- 172.21.15.157:0/3020221735 wait complete. 2023-11-06T03:19:47.390 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:19:47.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:19:47.509 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.513+0000 7fefe240e640 1 Processor -- start 2023-11-06T03:19:47.510 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefe240e640 1 -- start start 2023-11-06T03:19:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefe240e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fefdc0cf6b0 0x7fefdc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefe240e640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0d0050 con 0x7fefdc15cdf0 2023-11-06T03:19:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefe240e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0d1c60 con 0x7fefdc0b6110 2023-11-06T03:19:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefe240e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0d1f40 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdbfff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fefdc15cdf0 0x7fefdc0d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34490/0 (socket says 172.21.15.157:34490) 2023-11-06T03:19:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdbfff640 1 -- 172.21.15.157:0/1248801855 learned_addr learned my addr 172.21.15.157:0/1248801855 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdb7fe640 1 --2- 172.21.15.157:0/1248801855 >> v2:172.21.15.157:3300/0 conn(0x7fefdc0cf6b0 0x7fefdc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdaffd640 1 -- 172.21.15.157:0/1248801855 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32249014 0 0) 0x7fefdc0d0050 con 0x7fefdc15cdf0 2023-11-06T03:19:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdaffd640 1 -- 172.21.15.157:0/1248801855 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefc4003530 con 0x7fefdc15cdf0 2023-11-06T03:19:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1248801855 >> v1:172.21.15.195:6789/0 conn(0x7fefdc0b6110 legacy=0x7fefdc0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1248801855 >> v1:172.21.15.157:6789/0 conn(0x7fefdc15cdf0 legacy=0x7fefdc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1248801855 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0d2220 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdb7fe640 1 --2- 172.21.15.157:0/1248801855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc0cfb10 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097c0 tx=0x7fefcc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4226ef850d34f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.517+0000 7fefdaffd640 1 -- 172.21.15.157:0/1248801855 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0ab610 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdaffd640 1 -- 172.21.15.157:0/1248801855 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc0ab770 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdaffd640 1 -- 172.21.15.157:0/1248801855 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0abb00 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1248801855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 msgr2=0x7fefdc0cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 --2- 172.21.15.157:0/1248801855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc0cfb10 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097c0 tx=0x7fefcc01c410 comp rx=0 tx=0).stop 2023-11-06T03:19:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1248801855 shutdown_connections 2023-11-06T03:19:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 --2- 172.21.15.157:0/1248801855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc0cfb10 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1248801855 >> 172.21.15.157:0/1248801855 conn(0x7fefdc065930 msgr2=0x7fefdc065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1248801855 shutdown_connections 2023-11-06T03:19:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1248801855 wait complete. 2023-11-06T03:19:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 Processor -- start 2023-11-06T03:19:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- start start 2023-11-06T03:19:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 0x7fefdc0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 0x7fefdc14b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1559c0 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1553b0 con 0x7fefdc0b6110 2023-11-06T03:19:47.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc150db0 con 0x7fefdc15cdf0 2023-11-06T03:19:47.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46212/0 (socket says 172.21.15.157:46212) 2023-11-06T03:19:47.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1440854874 learned_addr learned my addr 172.21.15.157:0/1440854874 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:47.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe09ab640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 0x7fefdc14b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.521 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 msgr2=0x7fefdc14b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 0x7fefdc14b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdbfff640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 0x7fefdc0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 msgr2=0x7fefdc0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.522 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 0x7fefdc0c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 -- 172.21.15.157:0/1440854874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009420 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe09ab640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 0x7fefdc14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdbfff640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 0x7fefdc0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefdb7fe640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0ad040 tx=0x7fefcc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.523 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0bf610 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc0bf7b0 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc155650 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0bfb60 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.521+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc150a10 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc155b30 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.524 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefcc0b5400 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefd8ff9640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 0x7fefc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fefcc11e030 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.525 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefdbfff640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 0x7fefc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.525+0000 7fefdbfff640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 0x7fefc8067f20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fefd00011f0 tx=0x7fefd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.526 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.529+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fefcc0eb900 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.709+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefdc0cd6a0 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefd8ff9640 1 -- 172.21.15.157:0/1440854874 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7fefcc0eef20 con 0x7fefdc0cf6b0 2023-11-06T03:19:47.706 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:47.706 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:19:47.706 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:19:47.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 msgr2=0x7fefc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 0x7fefc8067f20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fefd00011f0 tx=0x7fefd0009040 comp rx=0 tx=0).stop 2023-11-06T03:19:47.708 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 msgr2=0x7fefdc14af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0ad040 tx=0x7fefcc01ac50 comp rx=0 tx=0).stop 2023-11-06T03:19:47.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 shutdown_connections 2023-11-06T03:19:47.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fefc8065a60 0x7fefc8067f20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fefdc15cdf0 0x7fefdc14b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.709 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fefdc0cf6b0 0x7fefdc14af60 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 --2- 172.21.15.157:0/1440854874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefdc0b6110 0x7fefdc0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 >> 172.21.15.157:0/1440854874 conn(0x7fefdc065930 msgr2=0x7fefdc153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 shutdown_connections 2023-11-06T03:19:47.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.713+0000 7fefe240e640 1 -- 172.21.15.157:0/1440854874 wait complete. 2023-11-06T03:19:47.730 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:19:47.845 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.849+0000 7f3b82501640 1 Processor -- start 2023-11-06T03:19:47.846 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 -- start start 2023-11-06T03:19:47.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b7c1551e0 0x7f3b7c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.847 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c15f580 con 0x7f3b7c15ea60 2023-11-06T03:19:47.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c15efc0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b82501640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c15f2a0 con 0x7f3b7c15e5e0 2023-11-06T03:19:47.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b7c1551e0 0x7f3b7c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b7c1551e0 0x7f3b7c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46216/0 (socket says 172.21.15.157:46216) 2023-11-06T03:19:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 -- 172.21.15.157:0/2542551317 learned_addr learned my addr 172.21.15.157:0/2542551317 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b80a9e640 1 --1- 172.21.15.157:0/2542551317 >> v1:172.21.15.157:6789/0 conn(0x7f3b7c15ea60 0x7f3b7c15c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34492/0 (socket says 172.21.15.157:34492) 2023-11-06T03:19:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7b7fe640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 -- 172.21.15.157:0/2542551317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 msgr2=0x7f3b7c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:47.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 -- 172.21.15.157:0/2542551317 >> v1:172.21.15.157:6789/0 conn(0x7f3b7c15ea60 legacy=0x7f3b7c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7affd640 1 -- 172.21.15.157:0/2542551317 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114488617 0 0) 0x7f3b7c15f580 con 0x7f3b7c15ea60 2023-11-06T03:19:47.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 -- 172.21.15.157:0/2542551317 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c06f5a0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7b7fe640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:47.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7bfff640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c06fda0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c091f8745c7df44 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7affd640 1 -- 172.21.15.157:0/2542551317 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b740ad640 con 0x7f3b7c1551e0 2023-11-06T03:19:47.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7affd640 1 -- 172.21.15.157:0/2542551317 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b740ad7a0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.853+0000 7f3b7affd640 1 -- 172.21.15.157:0/2542551317 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b740adb30 con 0x7f3b7c1551e0 2023-11-06T03:19:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/2542551317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 msgr2=0x7f3b7c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c06fda0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009870 tx=0x7f3b7401c480 comp rx=0 tx=0).stop 2023-11-06T03:19:47.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/2542551317 shutdown_connections 2023-11-06T03:19:47.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- 172.21.15.157:0/2542551317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c06fda0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/2542551317 >> 172.21.15.157:0/2542551317 conn(0x7f3b7c065930 msgr2=0x7f3b7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:47.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/2542551317 shutdown_connections 2023-11-06T03:19:47.856 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/2542551317 wait complete. 2023-11-06T03:19:47.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 Processor -- start 2023-11-06T03:19:47.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- start start 2023-11-06T03:19:47.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.857 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 0x7f3b7c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c158c50 con 0x7f3b7c1551e0 2023-11-06T03:19:47.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c14bc40 con 0x7f3b7c15ea60 2023-11-06T03:19:47.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c14c840 con 0x7f3b7c15e5e0 2023-11-06T03:19:47.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46222/0 (socket says 172.21.15.157:46222) 2023-11-06T03:19:47.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 -- 172.21.15.157:0/1075280620 learned_addr learned my addr 172.21.15.157:0/1075280620 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:47.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b80a9e640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 0x7f3b7c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7b7fe640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 -- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 msgr2=0x7f3b7c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 -- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 msgr2=0x7f3b7c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:47.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 0x7f3b7c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:47.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 -- 172.21.15.157:0/1075280620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740094d0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b80a9e640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 0x7f3b7c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:47.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b7bfff640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b0040 tx=0x7f3b74019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b740b6830 con 0x7f3b7c1551e0 2023-11-06T03:19:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b740a6040 con 0x7f3b7c1551e0 2023-11-06T03:19:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c14cae0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b740b6bd0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.857+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c14d0a0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c072750 con 0x7f3b7c1551e0 2023-11-06T03:19:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b740b4070 con 0x7f3b7c1551e0 2023-11-06T03:19:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b78ff9640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 0x7f3b5c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:47.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f3b7411ddd0 con 0x7f3b7c1551e0 2023-11-06T03:19:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b7b7fe640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 0x7f3b5c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.861+0000 7f3b7b7fe640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 0x7f3b5c067f50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:47.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:47.865+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f3b740eb580 con 0x7f3b7c1551e0 2023-11-06T03:19:48.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.053+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b7c1589b0 con 0x7f3b7c1551e0 2023-11-06T03:19:48.048 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.053+0000 7f3b78ff9640 1 -- 172.21.15.157:0/1075280620 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4674 (secure 0 0 0) 0x7f3b740b2020 con 0x7f3b7c1551e0 2023-11-06T03:19:48.048 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:48.048 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":136,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-06T03:18:18.097057+0000","modified":"2023-11-06T03:18:40.949661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":190,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11487":{"gid":11487,"name":"a","rank":0,"incarnation":133,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6835/2374994036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2374994036},{"type":"v1","addr":"172.21.15.157:6835","nonce":2374994036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-11-06T03:19:48.049 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 136 2023-11-06T03:19:48.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 msgr2=0x7f3b5c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.050 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 0x7f3b5c067f50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64008040 comp rx=0 tx=0).stop 2023-11-06T03:19:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 msgr2=0x7f3b7c14b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f3b740b0040 tx=0x7f3b74019220 comp rx=0 tx=0).stop 2023-11-06T03:19:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 shutdown_connections 2023-11-06T03:19:48.051 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b5c065a90 0x7f3b5c067f50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b7c15ea60 0x7f3b7c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b7c15e5e0 0x7f3b7c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.052 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 --2- 172.21.15.157:0/1075280620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b7c1551e0 0x7f3b7c14b700 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 >> 172.21.15.157:0/1075280620 conn(0x7f3b7c065930 msgr2=0x7f3b7c0b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 shutdown_connections 2023-11-06T03:19:48.053 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.057+0000 7f3b82501640 1 -- 172.21.15.157:0/1075280620 wait complete. 2023-11-06T03:19:48.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:19:48.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 Processor -- start 2023-11-06T03:19:48.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 -- start start 2023-11-06T03:19:48.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae2006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2006c9b0 con 0x7fae20156b00 2023-11-06T03:19:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2006bda0 con 0x7fae200adcc0 2023-11-06T03:19:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae24eed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2006c080 con 0x7fae20156ee0 2023-11-06T03:19:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1e59c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fae20156b00 0x7fae2015a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34502/0 (socket says 172.21.15.157:34502) 2023-11-06T03:19:48.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1e59c640 1 -- 172.21.15.157:0/885723015 learned_addr learned my addr 172.21.15.157:0/885723015 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae2006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436332553 0 0) 0x7fae2006c9b0 con 0x7fae20156b00 2023-11-06T03:19:48.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae08003450 con 0x7fae20156b00 2023-11-06T03:19:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898702903 0 0) 0x7fae08003450 con 0x7fae20156b00 2023-11-06T03:19:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 msgr2=0x7fae2006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae2006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 >> v1:172.21.15.195:6789/0 conn(0x7fae200adcc0 legacy=0x7fae200b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2006c360 con 0x7fae20156b00 2023-11-06T03:19:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fae10001680 con 0x7fae20156b00 2023-11-06T03:19:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 >> v1:172.21.15.157:6789/0 conn(0x7fae20156b00 legacy=0x7fae2015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae08003aa0 0x7fae08003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08008860 0x7fae0800ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae0800d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2006c080 con 0x7fae08008860 2023-11-06T03:19:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0800dc50 con 0x7fae08003aa0 2023-11-06T03:19:48.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0800de10 con 0x7fae20156ee0 2023-11-06T03:19:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08008860 0x7fae0800ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1ed9d640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae0800d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 -- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 msgr2=0x7fae0800d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae20156ee0 0x7fae0800d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 -- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae08003aa0 msgr2=0x7fae08003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae08003aa0 0x7fae08003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.195 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.193+0000 7fae1dd9b640 1 -- 172.21.15.157:0/885723015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08003450 con 0x7fae08008860 2023-11-06T03:19:48.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08008860 0x7fae0800ac70 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fae14009ae0 tx=0x7fae1401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859ba968c6b1c64c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140a5640 con 0x7fae08008860 2023-11-06T03:19:48.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae140a57a0 con 0x7fae08008860 2023-11-06T03:19:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1d59a640 1 -- 172.21.15.157:0/885723015 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140a5b50 con 0x7fae08008860 2023-11-06T03:19:48.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- 172.21.15.157:0/885723015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08008860 msgr2=0x7fae0800ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 --2- 172.21.15.157:0/885723015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae08008860 0x7fae0800ac70 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fae14009ae0 tx=0x7fae1401ca50 comp rx=0 tx=0).stop 2023-11-06T03:19:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1ed9d640 1 -- 172.21.15.157:0/885723015 reap_dead start 2023-11-06T03:19:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- 172.21.15.157:0/885723015 shutdown_connections 2023-11-06T03:19:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- 172.21.15.157:0/885723015 >> 172.21.15.157:0/885723015 conn(0x7fae20065930 msgr2=0x7fae20065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- 172.21.15.157:0/885723015 shutdown_connections 2023-11-06T03:19:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- 172.21.15.157:0/885723015 wait complete. 2023-11-06T03:19:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 Processor -- start 2023-11-06T03:19:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- start start 2023-11-06T03:19:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae2015c0d0 0x7fae200cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae200cd6b0 0x7fae20140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20155fb0 con 0x7fae200cc810 2023-11-06T03:19:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2013ebf0 con 0x7fae2015c0d0 2023-11-06T03:19:48.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae24eed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2015c550 con 0x7fae200cd6b0 2023-11-06T03:19:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46242/0 (socket says 172.21.15.157:46242) 2023-11-06T03:19:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.197+0000 7fae1e59c640 1 -- 172.21.15.157:0/3320346325 learned_addr learned my addr 172.21.15.157:0/3320346325 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1ed9d640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae200cd6b0 0x7fae20140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.205 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae2015c0d0 0x7fae200cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 -- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae200cd6b0 msgr2=0x7fae20140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae200cd6b0 0x7fae20140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 -- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae2015c0d0 msgr2=0x7fae200cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae2015c0d0 0x7fae200cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.206 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 -- 172.21.15.157:0/3320346325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae201559f0 con 0x7fae200cc810 2023-11-06T03:19:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1e59c640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fae1000f4b0 tx=0x7fae1000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10007d00 con 0x7fae200cc810 2023-11-06T03:19:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae10007e80 con 0x7fae200cc810 2023-11-06T03:19:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae20155cd0 con 0x7fae200cc810 2023-11-06T03:19:48.207 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000a310 con 0x7fae200cc810 2023-11-06T03:19:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae200ccbf0 con 0x7fae200cc810 2023-11-06T03:19:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20072750 con 0x7fae200cc810 2023-11-06T03:19:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fae1000a470 con 0x7fae200cc810 2023-11-06T03:19:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 0x7fae00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.208 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7fae100857a0 con 0x7fae200cc810 2023-11-06T03:19:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 0x7fae00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.201+0000 7fae1dd9b640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 0x7fae00067f20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fae140af000 tx=0x7fae14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.209 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.205+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fae10052fe0 con 0x7fae200cc810 2023-11-06T03:19:48.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.361+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae2006b490 con 0x7fae200cc810 2023-11-06T03:19:48.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.361+0000 7fae0effd640 1 -- 172.21.15.157:0/3320346325 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+18156 (secure 0 0 0) 0x7fae10056600 con 0x7fae200cc810 2023-11-06T03:19:48.356 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:48.357 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":192,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:18:51.597595+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-11-06T03:18:16.644094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-11-06T03:18:17.084366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.26","mappings":[{"from":6,"to":1}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/877366447":"2023-11-07T03:18:36.902417+0000","172.21.15.157:6834/877366447":"2023-11-07T03:18:36.902417+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:19:48.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 msgr2=0x7fae00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 0x7fae00067f20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fae140af000 tx=0x7fae14008040 comp rx=0 tx=0).stop 2023-11-06T03:19:48.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 msgr2=0x7fae2013eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fae1000f4b0 tx=0x7fae1000f970 comp rx=0 tx=0).stop 2023-11-06T03:19:48.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 shutdown_connections 2023-11-06T03:19:48.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fae00065a60 0x7fae00067f20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae200cd6b0 0x7fae20140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae2015c0d0 0x7fae200cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 --2- 172.21.15.157:0/3320346325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fae200cc810 0x7fae2013eeb0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 >> 172.21.15.157:0/3320346325 conn(0x7fae20065930 msgr2=0x7fae2006fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 shutdown_connections 2023-11-06T03:19:48.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.365+0000 7fae24eed640 1 -- 172.21.15.157:0/3320346325 wait complete. 2023-11-06T03:19:48.375 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-06T03:19:48.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 Processor -- start 2023-11-06T03:19:48.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 -- start start 2023-11-06T03:19:48.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f58a40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40d2920 con 0x7f58a40d0280 2023-11-06T03:19:48.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40d18a0 con 0x7f58a415ce80 2023-11-06T03:19:48.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58ac119640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40d1b60 con 0x7f58a415caa0 2023-11-06T03:19:48.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58aa6b6640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f58a40d0280 0x7f58a40b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:34508/0 (socket says 172.21.15.157:34508) 2023-11-06T03:19:48.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2584980011 learned_addr learned my addr 172.21.15.157:0/2584980011 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:48.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a9eb5640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f58a40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088504297 0 0) 0x7f58a40d2920 con 0x7f58a40d0280 2023-11-06T03:19:48.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5890003530 con 0x7f58a40d0280 2023-11-06T03:19:48.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061775422 0 0) 0x7f5890003530 con 0x7f58a40d0280 2023-11-06T03:19:48.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 msgr2=0x7f58a40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f58a40cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 >> v1:172.21.15.195:6789/0 conn(0x7f58a415ce80 legacy=0x7f58a40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40d1e40 con 0x7f58a40d0280 2023-11-06T03:19:48.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f589c001a00 con 0x7f58a40d0280 2023-11-06T03:19:48.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 >> v1:172.21.15.157:6789/0 conn(0x7f58a40d0280 legacy=0x7f58a40b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900039b0 0x7f5890003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58900047d0 0x7f5890004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f5890005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40d1b60 con 0x7f58900047d0 2023-11-06T03:19:48.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589000ba40 con 0x7f58900039b0 2023-11-06T03:19:48.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589000bc00 con 0x7f58a415caa0 2023-11-06T03:19:48.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58900047d0 0x7f5890004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a9eb5640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900039b0 0x7f5890003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58aa6b6640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f5890005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 -- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 msgr2=0x7f5890005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415caa0 0x7f5890005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 -- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900039b0 msgr2=0x7f5890003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:48.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900039b0 0x7f5890003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.507 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.505+0000 7f58a96b4640 1 -- 172.21.15.157:0/2584980011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890003530 con 0x7f58900047d0 2023-11-06T03:19:48.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58a96b4640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58900047d0 0x7f5890004c50 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f5894001750 tx=0x7f589401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49875ceb85dc146 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58940052d0 con 0x7f58900047d0 2023-11-06T03:19:48.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58940a7050 con 0x7f58900047d0 2023-11-06T03:19:48.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58a8eb3640 1 -- 172.21.15.157:0/2584980011 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894003600 con 0x7f58900047d0 2023-11-06T03:19:48.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- 172.21.15.157:0/2584980011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58900047d0 msgr2=0x7f5890004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 --2- 172.21.15.157:0/2584980011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58900047d0 0x7f5890004c50 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f5894001750 tx=0x7f589401c3a0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2584980011 reap_dead start 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- 172.21.15.157:0/2584980011 shutdown_connections 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- 172.21.15.157:0/2584980011 >> 172.21.15.157:0/2584980011 conn(0x7f58a4070790 msgr2=0x7f58a4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- 172.21.15.157:0/2584980011 shutdown_connections 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- 172.21.15.157:0/2584980011 wait complete. 2023-11-06T03:19:48.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 Processor -- start 2023-11-06T03:19:48.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.509+0000 7f58ac119640 1 -- start start 2023-11-06T03:19:48.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900047d0 0x7f58a415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415ace0 0x7f58a415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.514 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40cc450 con 0x7f58a41f21e0 2023-11-06T03:19:48.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40cc730 con 0x7f58900047d0 2023-11-06T03:19:48.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41e3990 con 0x7f58a415ace0 2023-11-06T03:19:48.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46248/0 (socket says 172.21.15.157:46248) 2023-11-06T03:19:48.515 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2194991026 learned_addr learned my addr 172.21.15.157:0/2194991026 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:48.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58a9eb5640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900047d0 0x7f58a415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58a96b4640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415ace0 0x7f58a415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415ace0 msgr2=0x7f58a415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415ace0 0x7f58a415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.516 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900047d0 msgr2=0x7f58a415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900047d0 0x7f58a415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 -- 172.21.15.157:0/2194991026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a41e3c70 con 0x7f58a41f21e0 2023-11-06T03:19:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58aa6b6640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f589c002240 tx=0x7f589c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00f920 con 0x7f58a41f21e0 2023-11-06T03:19:48.517 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c00faa0 con 0x7f58a41f21e0 2023-11-06T03:19:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41e3f50 con 0x7f58a41f21e0 2023-11-06T03:19:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c00fe50 con 0x7f58a41f21e0 2023-11-06T03:19:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40b43b0 con 0x7f58a41f21e0 2023-11-06T03:19:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a41e4140 con 0x7f58a41f21e0 2023-11-06T03:19:48.518 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f589c01f460 con 0x7f58a41f21e0 2023-11-06T03:19:48.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 0x7f5888067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6235+0+0 (secure 0 0 0) 0x7f589c086380 con 0x7f58a41f21e0 2023-11-06T03:19:48.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.513+0000 7f58a9eb5640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 0x7f5888067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.519 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.517+0000 7f58a9eb5640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 0x7f5888067f50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b6110 tx=0x7f58a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.520 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.517+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f589c053b30 con 0x7f58a41f21e0 2023-11-06T03:19:48.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.697+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f58a40cc590 con 0x7f58a41f21e0 2023-11-06T03:19:48.691 INFO:tasks.ceph.mon.a.smithi157.stderr:2023-11-06T03:19:48.697+0000 7f09ed343640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-06T03:19:48.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f589a7fc640 1 -- 172.21.15.157:0/2194991026 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) v1 ==== 161+0+0 (secure 0 0 0) 0x7f58a40cc590 con 0x7f58a41f21e0 2023-11-06T03:19:48.711 INFO:teuthology.orchestra.run.smithi157.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-06T03:19:48.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 msgr2=0x7f5888067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 0x7f5888067f50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f58a40b6110 tx=0x7f58a0008040 comp rx=0 tx=0).stop 2023-11-06T03:19:48.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 msgr2=0x7f58a40b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f589c002240 tx=0x7f589c011c20 comp rx=0 tx=0).stop 2023-11-06T03:19:48.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 shutdown_connections 2023-11-06T03:19:48.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f5888065a90 0x7f5888067f50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58a41f21e0 0x7f58a40b6440 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58a415ace0 0x7f58a415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 --2- 172.21.15.157:0/2194991026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58900047d0 0x7f58a415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 >> 172.21.15.157:0/2194991026 conn(0x7f58a4070790 msgr2=0x7f58a40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 shutdown_connections 2023-11-06T03:19:48.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.717+0000 7f58ac119640 1 -- 172.21.15.157:0/2194991026 wait complete. 2023-11-06T03:19:48.717 INFO:tasks.ceph.mds.a.smithi157.stderr: -2159> 2023-11-06T03:18:42.934+0000 7f5a89db4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-06T03:19:48.730 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-06T03:19:48.755 INFO:tasks.ceph.mds.a.smithi157.stdout:starting mds.a at 2023-11-06T03:19:48.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 Processor -- start 2023-11-06T03:19:48.848 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 -- start start 2023-11-06T03:19:48.849 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f92881599c0 0x7f92880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928815e440 con 0x7f928806d000 2023-11-06T03:19:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928806c240 con 0x7f92880b2af0 2023-11-06T03:19:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.853+0000 7f928ee64640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928806c520 con 0x7f92881599c0 2023-11-06T03:19:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f9287fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f928806d000 0x7f92880b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36784/0 (socket says 172.21.15.157:36784) 2023-11-06T03:19:48.850 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f9287fff640 1 -- 172.21.15.157:0/3251080899 learned_addr learned my addr 172.21.15.157:0/3251080899 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928cc00640 1 --2- 172.21.15.157:0/3251080899 >> v2:172.21.15.157:3300/0 conn(0x7f92881599c0 0x7f92880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f92877fe640 1 -- 172.21.15.157:0/3251080899 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101745630 0 0) 0x7f928815e440 con 0x7f928806d000 2023-11-06T03:19:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f92877fe640 1 -- 172.21.15.157:0/3251080899 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9270003530 con 0x7f928806d000 2023-11-06T03:19:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928cc00640 1 -- 172.21.15.157:0/3251080899 >> v1:172.21.15.195:6789/0 conn(0x7f92880b2af0 legacy=0x7f9288159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.851 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928cc00640 1 -- 172.21.15.157:0/3251080899 >> v1:172.21.15.157:6789/0 conn(0x7f928806d000 legacy=0x7f92880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.852 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928cc00640 1 -- 172.21.15.157:0/3251080899 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928806c800 con 0x7f92881599c0 2023-11-06T03:19:48.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928cc00640 1 --2- 172.21.15.157:0/3251080899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880ae000 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f92780097c0 tx=0x7f927801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c6efbf6c320ef6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f92877fe640 1 -- 172.21.15.157:0/3251080899 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780ab670 con 0x7f92881599c0 2023-11-06T03:19:48.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f92877fe640 1 -- 172.21.15.157:0/3251080899 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780ab7d0 con 0x7f92881599c0 2023-11-06T03:19:48.853 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f92877fe640 1 -- 172.21.15.157:0/3251080899 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780abb60 con 0x7f92881599c0 2023-11-06T03:19:48.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 -- 172.21.15.157:0/3251080899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 msgr2=0x7f92880ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 --2- 172.21.15.157:0/3251080899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880ae000 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f92780097c0 tx=0x7f927801c450 comp rx=0 tx=0).stop 2023-11-06T03:19:48.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 -- 172.21.15.157:0/3251080899 shutdown_connections 2023-11-06T03:19:48.854 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 --2- 172.21.15.157:0/3251080899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880ae000 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 -- 172.21.15.157:0/3251080899 >> 172.21.15.157:0/3251080899 conn(0x7f9288065930 msgr2=0x7f9288065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:48.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 -- 172.21.15.157:0/3251080899 shutdown_connections 2023-11-06T03:19:48.855 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.857+0000 7f928ee64640 1 -- 172.21.15.157:0/3251080899 wait complete. 2023-11-06T03:19:48.858 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 Processor -- start 2023-11-06T03:19:48.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- start start 2023-11-06T03:19:48.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928806d000 0x7f928814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92880b2af0 0x7f92880a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.859 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880aab90 con 0x7f92881599c0 2023-11-06T03:19:48.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880cebb0 con 0x7f928806d000 2023-11-06T03:19:48.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880aae00 con 0x7f92880b2af0 2023-11-06T03:19:48.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57284/0 (socket says 172.21.15.157:57284) 2023-11-06T03:19:48.860 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 -- 172.21.15.157:0/1893599601 learned_addr learned my addr 172.21.15.157:0/1893599601 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:48.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f9287fff640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92880b2af0 0x7f92880a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928cc00640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928806d000 0x7f928814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 -- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92880b2af0 msgr2=0x7f92880a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92880b2af0 0x7f92880a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.861 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 -- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928806d000 msgr2=0x7f928814cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:48.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928806d000 0x7f928814cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:48.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 -- 172.21.15.157:0/1893599601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278009420 con 0x7f92881599c0 2023-11-06T03:19:48.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928d401640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f928000c7a0 tx=0x7f928000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280003b80 con 0x7f92881599c0 2023-11-06T03:19:48.862 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9280003ce0 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880ab0e0 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928000aa30 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928814d460 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.861+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288072750 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.865+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f928000ab90 con 0x7f92881599c0 2023-11-06T03:19:48.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.865+0000 7f92857fa640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 0x7f9274067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:48.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.865+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6321+0+0 (secure 0 0 0) 0x7f9280083170 con 0x7f92881599c0 2023-11-06T03:19:48.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.865+0000 7f928cc00640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 0x7f9274067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:48.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.865+0000 7f928cc00640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 0x7f9274067f50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f92780af000 tx=0x7f92780a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:48.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:48.869+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f9280050950 con 0x7f92881599c0 2023-11-06T03:19:49.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.053+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f92880ab590 con 0x7f92881599c0 2023-11-06T03:19:49.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.717+0000 7f92857fa640 1 -- 172.21.15.157:0/1893599601 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v138) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9280053f70 con 0x7f92881599c0 2023-11-06T03:19:49.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 msgr2=0x7f9274067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 0x7f9274067f50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f92780af000 tx=0x7f92780a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:49.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 msgr2=0x7f92880a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f928000c7a0 tx=0x7f928000cc60 comp rx=0 tx=0).stop 2023-11-06T03:19:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 shutdown_connections 2023-11-06T03:19:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f9274065a90 0x7f9274067f50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92881599c0 0x7f92880a9140 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f92880b2af0 0x7f92880a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 --2- 172.21.15.157:0/1893599601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f928806d000 0x7f928814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 >> 172.21.15.157:0/1893599601 conn(0x7f9288065930 msgr2=0x7f928815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 shutdown_connections 2023-11-06T03:19:49.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.721+0000 7f928ee64640 1 -- 172.21.15.157:0/1893599601 wait complete. 2023-11-06T03:19:49.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-06T03:19:49.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 Processor -- start 2023-11-06T03:19:49.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 -- start start 2023-11-06T03:19:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f28c015a9a0 0x7f28c006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c015e150 con 0x7f28c0065930 2023-11-06T03:19:49.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c00ca4a0 con 0x7f28c00c9f00 2023-11-06T03:19:49.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c780d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c006ed90 con 0x7f28c015a9a0 2023-11-06T03:19:49.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c5daa640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f28c0065930 0x7f28c0065d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36788/0 (socket says 172.21.15.157:36788) 2023-11-06T03:19:49.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c5daa640 1 -- 172.21.15.157:0/649113702 learned_addr learned my addr 172.21.15.157:0/649113702 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 --2- 172.21.15.157:0/649113702 >> v2:172.21.15.195:3300/0 conn(0x7f28c015a9a0 0x7f28c006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208685703 0 0) 0x7f28c015e150 con 0x7f28c0065930 2023-11-06T03:19:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003530 con 0x7f28c0065930 2023-11-06T03:19:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087042342 0 0) 0x7f28ac003530 con 0x7f28c0065930 2023-11-06T03:19:49.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 >> v2:172.21.15.195:3300/0 conn(0x7f28c015a9a0 msgr2=0x7f28c006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 --2- 172.21.15.157:0/649113702 >> v2:172.21.15.195:3300/0 conn(0x7f28c015a9a0 0x7f28c006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 >> v1:172.21.15.195:6789/0 conn(0x7f28c00c9f00 legacy=0x7f28c015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c006e7d0 con 0x7f28c0065930 2023-11-06T03:19:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f28b8001680 con 0x7f28c0065930 2023-11-06T03:19:49.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 >> v1:172.21.15.157:6789/0 conn(0x7f28c0065930 legacy=0x7f28c0065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 0x7f28ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28ac005190 0x7f28ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c006ed90 con 0x7f28ac003980 2023-11-06T03:19:49.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00c030 con 0x7f28ac0047d0 2023-11-06T03:19:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00c1f0 con 0x7f28ac005190 2023-11-06T03:19:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 0x7f28ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c4da8640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 -- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28ac005190 msgr2=0x7f28ac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28ac005190 0x7f28ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 -- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac0047d0 msgr2=0x7f28ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac0047d0 0x7f28ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.915 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c55a9640 1 -- 172.21.15.157:0/649113702 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac003530 con 0x7f28ac003980 2023-11-06T03:19:49.916 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.913+0000 7f28c5daa640 1 -- 172.21.15.157:0/649113702 reap_dead start 2023-11-06T03:19:49.919 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c55a9640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 0x7f28ac003da0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009ae0 tx=0x7f28b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58d7992a01e8763 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00a5550 con 0x7f28ac003980 2023-11-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b00a56b0 con 0x7f28ac003980 2023-11-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28b7fff640 1 -- 172.21.15.157:0/649113702 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00a5a60 con 0x7f28ac003980 2023-11-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 msgr2=0x7f28ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 0x7f28ac003da0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009ae0 tx=0x7f28b001ca70 comp rx=0 tx=0).stop 2023-11-06T03:19:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- 172.21.15.157:0/649113702 shutdown_connections 2023-11-06T03:19:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 --2- 172.21.15.157:0/649113702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28ac003980 0x7f28ac003da0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- 172.21.15.157:0/649113702 >> 172.21.15.157:0/649113702 conn(0x7f28c00cf8b0 msgr2=0x7f28c00cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- 172.21.15.157:0/649113702 shutdown_connections 2023-11-06T03:19:49.921 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- 172.21.15.157:0/649113702 wait complete. 2023-11-06T03:19:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 Processor -- start 2023-11-06T03:19:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.917+0000 7f28c780d640 1 -- start start 2023-11-06T03:19:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac003980 0x7f28c00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c00ac8c0 0x7f28c0154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.922 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00af210 con 0x7f28c015eb90 2023-11-06T03:19:49.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c00c5290 con 0x7f28ac003980 2023-11-06T03:19:49.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0154c50 con 0x7f28c00ac8c0 2023-11-06T03:19:49.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.923 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57306/0 (socket says 172.21.15.157:57306) 2023-11-06T03:19:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 -- 172.21.15.157:0/2497296231 learned_addr learned my addr 172.21.15.157:0/2497296231 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c4da8640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c00ac8c0 0x7f28c0154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c55a9640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac003980 0x7f28c00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 -- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c00ac8c0 msgr2=0x7f28c0154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.924 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c00ac8c0 0x7f28c0154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 -- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac003980 msgr2=0x7f28c00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac003980 0x7f28c00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 -- 172.21.15.157:0/2497296231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00ace10 con 0x7f28c015eb90 2023-11-06T03:19:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c5daa640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f28c0070320 tx=0x7f28b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:49.925 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004c00 con 0x7f28c015eb90 2023-11-06T03:19:49.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8004d80 con 0x7f28c015eb90 2023-11-06T03:19:49.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8007270 con 0x7f28c015eb90 2023-11-06T03:19:49.926 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c00ad0f0 con 0x7f28c015eb90 2023-11-06T03:19:49.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c00ad6b0 con 0x7f28c015eb90 2023-11-06T03:19:49.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c00af080 con 0x7f28c015eb90 2023-11-06T03:19:49.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f28b80073d0 con 0x7f28c015eb90 2023-11-06T03:19:49.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 0x7f28a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:49.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6321+0+0 (secure 0 0 0) 0x7f28b8082360 con 0x7f28c015eb90 2023-11-06T03:19:49.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.921+0000 7f28c55a9640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 0x7f28a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:49.928 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.925+0000 7f28c55a9640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 0x7f28a4067f20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f28b00af000 tx=0x7f28b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:49.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:49.925+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f28b804fab0 con 0x7f28c015eb90 2023-11-06T03:19:50.064 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.069+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28c004e050 con 0x7f28c015eb90 2023-11-06T03:19:50.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.729+0000 7f28b5ffb640 1 -- 172.21.15.157:0/2497296231 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v194) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28b8007680 con 0x7f28c015eb90 2023-11-06T03:19:50.724 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_metadata' removed 2023-11-06T03:19:50.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 msgr2=0x7f28a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:50.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 0x7f28a4067f20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f28b00af000 tx=0x7f28b0008040 comp rx=0 tx=0).stop 2023-11-06T03:19:50.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 msgr2=0x7f28c015f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:50.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f28c0070320 tx=0x7f28b800c8e0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.727 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 shutdown_connections 2023-11-06T03:19:50.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f28a4065a60 0x7f28a4067f20 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28c015eb90 0x7f28c015f010 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f28c00ac8c0 0x7f28c0154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 --2- 172.21.15.157:0/2497296231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28ac003980 0x7f28c00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.728 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 >> 172.21.15.157:0/2497296231 conn(0x7f28c00cf8b0 msgr2=0x7f28c015ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:50.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 shutdown_connections 2023-11-06T03:19:50.729 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.733+0000 7f28c780d640 1 -- 172.21.15.157:0/2497296231 wait complete. 2023-11-06T03:19:50.750 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-06T03:19:50.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 Processor -- start 2023-11-06T03:19:50.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 -- start start 2023-11-06T03:19:50.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0dc06c170 0x7ff0dc154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff0dc15b860 0x7ff0dc155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0cb100 con 0x7ff0dc06c550 2023-11-06T03:19:50.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc15bcf0 con 0x7ff0dc06c170 2023-11-06T03:19:50.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e40a5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc0ce0a0 con 0x7ff0dc15b860 2023-11-06T03:19:50.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0dc06c170 0x7ff0dc154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff0dc06c170 0x7ff0dc154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57322/0 (socket says 172.21.15.157:57322) 2023-11-06T03:19:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 -- 172.21.15.157:0/387750576 learned_addr learned my addr 172.21.15.157:0/387750576 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e2642640 1 --2- 172.21.15.157:0/387750576 >> v2:172.21.15.195:3300/0 conn(0x7ff0dc15b860 0x7ff0dc155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 -- 172.21.15.157:0/387750576 >> v2:172.21.15.195:3300/0 conn(0x7ff0dc15b860 msgr2=0x7ff0dc155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:50.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 --2- 172.21.15.157:0/387750576 >> v2:172.21.15.195:3300/0 conn(0x7ff0dc15b860 0x7ff0dc155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 -- 172.21.15.157:0/387750576 >> v1:172.21.15.195:6789/0 conn(0x7ff0dc06c550 legacy=0x7ff0dc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:50.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.885+0000 7ff0e1e41640 1 -- 172.21.15.157:0/387750576 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc15bef0 con 0x7ff0dc06c170 2023-11-06T03:19:50.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e1e41640 1 --2- 172.21.15.157:0/387750576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c170 0x7ff0dc154c00 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc06d610 tx=0x7ff0cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926b0ff71c23365 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:50.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e0e3f640 1 -- 172.21.15.157:0/387750576 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0aa5f0 con 0x7ff0dc06c170 2023-11-06T03:19:50.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e0e3f640 1 -- 172.21.15.157:0/387750576 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0aa750 con 0x7ff0dc06c170 2023-11-06T03:19:50.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e0e3f640 1 -- 172.21.15.157:0/387750576 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0aaae0 con 0x7ff0dc06c170 2023-11-06T03:19:50.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- 172.21.15.157:0/387750576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c170 msgr2=0x7ff0dc154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:50.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/387750576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c170 0x7ff0dc154c00 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc06d610 tx=0x7ff0cc01c490 comp rx=0 tx=0).stop 2023-11-06T03:19:50.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- 172.21.15.157:0/387750576 shutdown_connections 2023-11-06T03:19:50.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/387750576 >> v2:172.21.15.195:3300/0 conn(0x7ff0dc15b860 0x7ff0dc155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/387750576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c170 0x7ff0dc154c00 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- 172.21.15.157:0/387750576 >> 172.21.15.157:0/387750576 conn(0x7ff0dc0b43b0 msgr2=0x7ff0dc065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:50.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- 172.21.15.157:0/387750576 shutdown_connections 2023-11-06T03:19:50.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- 172.21.15.157:0/387750576 wait complete. 2023-11-06T03:19:50.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 Processor -- start 2023-11-06T03:19:50.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.889+0000 7ff0e40a5640 1 -- start start 2023-11-06T03:19:50.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0dc06c170 0x7ff0dc1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 0x7ff0dc148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1f4000 con 0x7ff0dc06c550 2023-11-06T03:19:50.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1f3570 con 0x7ff0dc06c170 2023-11-06T03:19:50.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc1f3760 con 0x7ff0dc15b860 2023-11-06T03:19:50.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57324/0 (socket says 172.21.15.157:57324) 2023-11-06T03:19:50.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 -- 172.21.15.157:0/1250255656 learned_addr learned my addr 172.21.15.157:0/1250255656 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:50.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e2642640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 0x7ff0dc148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 -- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 msgr2=0x7ff0dc148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:50.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 0x7ff0dc148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1e41640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0dc06c170 0x7ff0dc1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 -- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0dc06c170 msgr2=0x7ff0dc1513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:50.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0dc06c170 0x7ff0dc1513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:50.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 -- 172.21.15.157:0/1250255656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0094d0 con 0x7ff0dc06c550 2023-11-06T03:19:50.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e2642640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 0x7ff0dc148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:50.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1640640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c450 tx=0x7ff0d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:50.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8012830 con 0x7ff0dc06c550 2023-11-06T03:19:50.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d80129b0 con 0x7ff0dc06c550 2023-11-06T03:19:50.893 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc0cbfb0 con 0x7ff0dc06c550 2023-11-06T03:19:50.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8012d60 con 0x7ff0dc06c550 2023-11-06T03:19:50.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc1f3d20 con 0x7ff0dc06c550 2023-11-06T03:19:50.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc04e050 con 0x7ff0dc06c550 2023-11-06T03:19:50.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0d801c460 con 0x7ff0dc06c550 2023-11-06T03:19:50.894 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 0x7ff0c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:50.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ff0d8083020 con 0x7ff0dc06c550 2023-11-06T03:19:50.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.893+0000 7ff0e1e41640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 0x7ff0c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:50.895 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.897+0000 7ff0e1e41640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 0x7ff0c0067f20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0ae000 tx=0x7ff0cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:50.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:50.897+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff0d80509c0 con 0x7ff0dc06c550 2023-11-06T03:19:51.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.053+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0dc0ccbb0 con 0x7ff0dc06c550 2023-11-06T03:19:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.741+0000 7ff0d27fc640 1 -- 172.21.15.157:0/1250255656 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v195) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0d8053fe0 con 0x7ff0dc06c550 2023-11-06T03:19:51.735 INFO:teuthology.orchestra.run.smithi157.stderr:pool 'cephfs_data' removed 2023-11-06T03:19:51.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 msgr2=0x7ff0c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:51.739 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 0x7ff0c0067f20 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0ae000 tx=0x7ff0cc008040 comp rx=0 tx=0).stop 2023-11-06T03:19:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 msgr2=0x7ff0dc1f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c450 tx=0x7ff0d800c910 comp rx=0 tx=0).stop 2023-11-06T03:19:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 shutdown_connections 2023-11-06T03:19:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff0c0065a60 0x7ff0c0067f20 secure :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0ae000 tx=0x7ff0cc008040 comp rx=0 tx=0).stop 2023-11-06T03:19:51.740 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff0dc15b860 0x7ff0dc148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0dc06c550 0x7ff0dc1f3030 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 --2- 172.21.15.157:0/1250255656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0dc06c170 0x7ff0dc1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.741 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 >> 172.21.15.157:0/1250255656 conn(0x7ff0dc0b43b0 msgr2=0x7ff0dc0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 shutdown_connections 2023-11-06T03:19:51.742 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.745+0000 7ff0e40a5640 1 -- 172.21.15.157:0/1250255656 wait complete. 2023-11-06T03:19:51.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2023-11-06T03:19:51.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 Processor -- start 2023-11-06T03:19:51.896 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 -- start start 2023-11-06T03:19:51.897 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff3701599c0 0x7ff3700ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3700b2af0 0x7ff370159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37015e440 con 0x7ff37006d000 2023-11-06T03:19:51.898 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37006c240 con 0x7ff3700b2af0 2023-11-06T03:19:51.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff374bb8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37006c520 con 0x7ff3701599c0 2023-11-06T03:19:51.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3700b2af0 0x7ff370159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff3700b2af0 0x7ff370159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57326/0 (socket says 172.21.15.157:57326) 2023-11-06T03:19:51.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1157672595 learned_addr learned my addr 172.21.15.157:0/1157672595 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:51.899 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36e59c640 1 --2- 172.21.15.157:0/1157672595 >> v2:172.21.15.195:3300/0 conn(0x7ff3701599c0 0x7ff3700ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1157672595 >> v2:172.21.15.195:3300/0 conn(0x7ff3701599c0 msgr2=0x7ff3700ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:51.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 --2- 172.21.15.157:0/1157672595 >> v2:172.21.15.195:3300/0 conn(0x7ff3701599c0 0x7ff3700ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1157672595 >> v1:172.21.15.195:6789/0 conn(0x7ff37006d000 legacy=0x7ff3700b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:51.900 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.901+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1157672595 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37006c800 con 0x7ff3700b2af0 2023-11-06T03:19:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff36ed9d640 1 --2- 172.21.15.157:0/1157672595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700b2af0 0x7ff370159da0 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7ff370066800 tx=0x7ff36801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce978d414526cb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff36d59a640 1 -- 172.21.15.157:0/1157672595 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680a5840 con 0x7ff3700b2af0 2023-11-06T03:19:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff36d59a640 1 -- 172.21.15.157:0/1157672595 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3680a59a0 con 0x7ff3700b2af0 2023-11-06T03:19:51.904 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff36d59a640 1 -- 172.21.15.157:0/1157672595 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680a5d30 con 0x7ff3700b2af0 2023-11-06T03:19:51.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 -- 172.21.15.157:0/1157672595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700b2af0 msgr2=0x7ff370159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:51.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1157672595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700b2af0 0x7ff370159da0 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7ff370066800 tx=0x7ff36801c960 comp rx=0 tx=0).stop 2023-11-06T03:19:51.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 -- 172.21.15.157:0/1157672595 shutdown_connections 2023-11-06T03:19:51.905 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1157672595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3700b2af0 0x7ff370159da0 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1157672595 >> v2:172.21.15.195:3300/0 conn(0x7ff3701599c0 0x7ff3700ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 -- 172.21.15.157:0/1157672595 >> 172.21.15.157:0/1157672595 conn(0x7ff370065930 msgr2=0x7ff370065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 -- 172.21.15.157:0/1157672595 shutdown_connections 2023-11-06T03:19:51.906 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.905+0000 7ff374bb8640 1 -- 172.21.15.157:0/1157672595 wait complete. 2023-11-06T03:19:51.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 Processor -- start 2023-11-06T03:19:51.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- start start 2023-11-06T03:19:51.907 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff37006d000 0x7ff37015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3700b2af0 0x7ff37014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.908 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff370158770 con 0x7ff3701599c0 2023-11-06T03:19:51.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700ae160 con 0x7ff37006d000 2023-11-06T03:19:51.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff370157ed0 con 0x7ff3700b2af0 2023-11-06T03:19:51.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57336/0 (socket says 172.21.15.157:57336) 2023-11-06T03:19:51.909 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1733567741 learned_addr learned my addr 172.21.15.157:0/1733567741 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:51.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36e59c640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff37006d000 0x7ff37015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36dd9b640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3700b2af0 0x7ff37014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3700b2af0 msgr2=0x7ff37014c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:51.910 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3700b2af0 0x7ff37014c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff37006d000 msgr2=0x7ff37015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:51.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff37006d000 0x7ff37015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:51.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 -- 172.21.15.157:0/1733567741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009b30 con 0x7ff3701599c0 2023-11-06T03:19:51.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36ed9d640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7ff3680af040 tx=0x7ff368019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:51.911 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680b8670 con 0x7ff3701599c0 2023-11-06T03:19:51.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3680b87d0 con 0x7ff3701599c0 2023-11-06T03:19:51.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3701581b0 con 0x7ff3701599c0 2023-11-06T03:19:51.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3680b8b80 con 0x7ff3701599c0 2023-11-06T03:19:51.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37015cc30 con 0x7ff3701599c0 2023-11-06T03:19:51.912 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3700adfc0 con 0x7ff3701599c0 2023-11-06T03:19:51.913 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff3680b8ce0 con 0x7ff3701599c0 2023-11-06T03:19:51.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 0x7ff350067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:51.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff36811e600 con 0x7ff3701599c0 2023-11-06T03:19:51.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.909+0000 7ff36e59c640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 0x7ff350067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:51.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.913+0000 7ff36e59c640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 0x7ff350067f20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b930 tx=0x7ff360008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:51.914 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:51.913+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7ff3700adfc0 con 0x7ff3701599c0 2023-11-06T03:19:52.062 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.069+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7ff3700adfc0 con 0x7ff3701599c0 2023-11-06T03:19:52.072 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.077+0000 7ff35effd640 1 -- 172.21.15.157:0/1733567741 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3700adfc0 con 0x7ff3701599c0 2023-11-06T03:19:52.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 msgr2=0x7ff350067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 0x7ff350067f20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b930 tx=0x7ff360008040 comp rx=0 tx=0).stop 2023-11-06T03:19:52.075 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 msgr2=0x7ff37014ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7ff3680af040 tx=0x7ff368019700 comp rx=0 tx=0).stop 2023-11-06T03:19:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 shutdown_connections 2023-11-06T03:19:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7ff350065a60 0x7ff350067f20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3701599c0 0x7ff37014ca10 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.076 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff3700b2af0 0x7ff37014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 --2- 172.21.15.157:0/1733567741 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff37006d000 0x7ff37015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 >> 172.21.15.157:0/1733567741 conn(0x7ff370065930 msgr2=0x7ff37006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 shutdown_connections 2023-11-06T03:19:52.077 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.081+0000 7ff374bb8640 1 -- 172.21.15.157:0/1733567741 wait complete. 2023-11-06T03:19:52.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-06T03:19:52.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.221+0000 7f33db845640 1 Processor -- start 2023-11-06T03:19:52.217 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.221+0000 7f33db845640 1 -- start start 2023-11-06T03:19:52.218 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33db845640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33d406d000 0x7f33d40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33db845640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d415e440 con 0x7f33d41599c0 2023-11-06T03:19:52.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33db845640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d406c240 con 0x7f33d40b2af0 2023-11-06T03:19:52.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33db845640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d406c520 con 0x7f33d406d000 2023-11-06T03:19:52.219 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d95e1640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f33d41599c0 0x7f33d40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36802/0 (socket says 172.21.15.157:36802) 2023-11-06T03:19:52.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d95e1640 1 -- 172.21.15.157:0/2904387381 learned_addr learned my addr 172.21.15.157:0/2904387381 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:52.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d9de2640 1 --2- 172.21.15.157:0/2904387381 >> v2:172.21.15.157:3300/0 conn(0x7f33d406d000 0x7f33d40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d9de2640 1 -- 172.21.15.157:0/2904387381 >> v1:172.21.15.195:6789/0 conn(0x7f33d40b2af0 legacy=0x7f33d4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:52.220 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d9de2640 1 -- 172.21.15.157:0/2904387381 >> v1:172.21.15.157:6789/0 conn(0x7f33d41599c0 legacy=0x7f33d40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33cbfff640 1 -- 172.21.15.157:0/2904387381 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657842640 0 0) 0x7f33d415e440 con 0x7f33d41599c0 2023-11-06T03:19:52.221 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d9de2640 1 -- 172.21.15.157:0/2904387381 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d406c800 con 0x7f33d406d000 2023-11-06T03:19:52.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33d9de2640 1 --2- 172.21.15.157:0/2904387381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d406d000 0x7f33d40ae000 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009d90 tx=0x7f33cc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182e7a6e461649e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33cbfff640 1 -- 172.21.15.157:0/2904387381 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0a59c0 con 0x7f33d406d000 2023-11-06T03:19:52.223 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33cbfff640 1 -- 172.21.15.157:0/2904387381 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0a5b20 con 0x7f33d406d000 2023-11-06T03:19:52.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.225+0000 7f33cbfff640 1 -- 172.21.15.157:0/2904387381 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0ad440 con 0x7f33d406d000 2023-11-06T03:19:52.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/2904387381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d406d000 msgr2=0x7f33d40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 --2- 172.21.15.157:0/2904387381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d406d000 0x7f33d40ae000 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009d90 tx=0x7f33cc01c8e0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.224 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/2904387381 shutdown_connections 2023-11-06T03:19:52.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 --2- 172.21.15.157:0/2904387381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d406d000 0x7f33d40ae000 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/2904387381 >> 172.21.15.157:0/2904387381 conn(0x7f33d4065930 msgr2=0x7f33d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:52.225 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/2904387381 shutdown_connections 2023-11-06T03:19:52.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/2904387381 wait complete. 2023-11-06T03:19:52.226 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 Processor -- start 2023-11-06T03:19:52.227 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- start start 2023-11-06T03:19:52.229 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 0x7f33d40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 0x7f33d406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40ce1a0 con 0x7f33d41599c0 2023-11-06T03:19:52.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40ce480 con 0x7f33d406d000 2023-11-06T03:19:52.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40ce760 con 0x7f33d40b2af0 2023-11-06T03:19:52.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57364/0 (socket says 172.21.15.157:57364) 2023-11-06T03:19:52.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 -- 172.21.15.157:0/3243680573 learned_addr learned my addr 172.21.15.157:0/3243680573 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:52.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 -- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 msgr2=0x7f33d406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d95e1640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 0x7f33d40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d8de0640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 0x7f33d406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 0x7f33d406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 -- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 msgr2=0x7f33d40cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 0x7f33d40cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 -- 172.21.15.157:0/3243680573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc009a70 con 0x7f33d41599c0 2023-11-06T03:19:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d8de0640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 0x7f33d406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d95e1640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 0x7f33d40cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33d9de2640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009d90 tx=0x7f33cc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0adad0 con 0x7f33d41599c0 2023-11-06T03:19:52.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0adc30 con 0x7f33d41599c0 2023-11-06T03:19:52.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d40cea40 con 0x7f33d41599c0 2023-11-06T03:19:52.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0b8420 con 0x7f33d41599c0 2023-11-06T03:19:52.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.229+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d406fc10 con 0x7f33d41599c0 2023-11-06T03:19:52.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d406fda0 con 0x7f33d41599c0 2023-11-06T03:19:52.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33cc0b8580 con 0x7f33d41599c0 2023-11-06T03:19:52.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33c9ffb640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 0x7f33b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f33cc11e810 con 0x7f33d41599c0 2023-11-06T03:19:52.235 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33d95e1640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 0x7f33b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.233+0000 7f33d95e1640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 0x7f33b8067f20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f33d0007fd0 tx=0x7f33d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.237+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f33cc0ec4c0 con 0x7f33d41599c0 2023-11-06T03:19:52.401 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.409+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f33d4072750 con 0x7f33d41599c0 2023-11-06T03:19:52.408 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.413+0000 7f33c9ffb640 1 -- 172.21.15.157:0/3243680573 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33cc0efae0 con 0x7f33d41599c0 2023-11-06T03:19:52.410 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 msgr2=0x7f33b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 0x7f33b8067f20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f33d0007fd0 tx=0x7f33d0009040 comp rx=0 tx=0).stop 2023-11-06T03:19:52.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 msgr2=0x7f33d414c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009d90 tx=0x7f33cc019680 comp rx=0 tx=0).stop 2023-11-06T03:19:52.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 shutdown_connections 2023-11-06T03:19:52.411 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f33b8065a60 0x7f33b8067f20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33d41599c0 0x7f33d414c990 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33d40b2af0 0x7f33d406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 --2- 172.21.15.157:0/3243680573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33d406d000 0x7f33d40cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 >> 172.21.15.157:0/3243680573 conn(0x7f33d4065930 msgr2=0x7f33d4065d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:52.412 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 shutdown_connections 2023-11-06T03:19:52.413 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.417+0000 7f33db845640 1 -- 172.21.15.157:0/3243680573 wait complete. 2023-11-06T03:19:52.428 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2023-11-06T03:19:52.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 Processor -- start 2023-11-06T03:19:52.550 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 -- start start 2023-11-06T03:19:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9e00b2af0 0x7fe9e0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.551 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e015e440 con 0x7fe9e006d000 2023-11-06T03:19:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e006c240 con 0x7fe9e00b2af0 2023-11-06T03:19:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e8344640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e006c520 con 0x7fe9e01599c0 2023-11-06T03:19:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9e00b2af0 0x7fe9e0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9e00b2af0 0x7fe9e0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57372/0 (socket says 172.21.15.157:57372) 2023-11-06T03:19:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 -- 172.21.15.157:0/570422873 learned_addr learned my addr 172.21.15.157:0/570422873 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e60e0640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 -- 172.21.15.157:0/570422873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 msgr2=0x7fe9e00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 -- 172.21.15.157:0/570422873 >> v1:172.21.15.195:6789/0 conn(0x7fe9e006d000 legacy=0x7fe9e00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:52.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.557+0000 7fe9e68e1640 1 -- 172.21.15.157:0/570422873 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e006c800 con 0x7fe9e00b2af0 2023-11-06T03:19:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e68e1640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00b2af0 0x7fe9e0159da0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009870 tx=0x7fe9d801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d7d196b6a831e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e50de640 1 -- 172.21.15.157:0/570422873 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80a95f0 con 0x7fe9e00b2af0 2023-11-06T03:19:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e50de640 1 -- 172.21.15.157:0/570422873 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d80a9750 con 0x7fe9e00b2af0 2023-11-06T03:19:52.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e50de640 1 -- 172.21.15.157:0/570422873 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80a9ae0 con 0x7fe9e00b2af0 2023-11-06T03:19:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- 172.21.15.157:0/570422873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00b2af0 msgr2=0x7fe9e0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00b2af0 0x7fe9e0159da0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009870 tx=0x7fe9d801c510 comp rx=0 tx=0).stop 2023-11-06T03:19:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- 172.21.15.157:0/570422873 shutdown_connections 2023-11-06T03:19:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e00b2af0 0x7fe9e0159da0 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- 172.21.15.157:0/570422873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- 172.21.15.157:0/570422873 >> 172.21.15.157:0/570422873 conn(0x7fe9e0065930 msgr2=0x7fe9e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- 172.21.15.157:0/570422873 shutdown_connections 2023-11-06T03:19:52.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- 172.21.15.157:0/570422873 wait complete. 2023-11-06T03:19:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 Processor -- start 2023-11-06T03:19:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- start start 2023-11-06T03:19:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 0x7fe9e00af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.560 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e0147c80 con 0x7fe9e006d000 2023-11-06T03:19:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e01f5210 con 0x7fe9e00b2af0 2023-11-06T03:19:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e8344640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00aff80 con 0x7fe9e01599c0 2023-11-06T03:19:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e60e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.561 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e60e0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57378/0 (socket says 172.21.15.157:57378) 2023-11-06T03:19:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.561+0000 7fe9e60e0640 1 -- 172.21.15.157:0/2067070928 learned_addr learned my addr 172.21.15.157:0/2067070928 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e68e1640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e58df640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 0x7fe9e00af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 -- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 msgr2=0x7fe9e00af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.562 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 -- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 msgr2=0x7fe9e00af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 0x7fe9e00af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 -- 172.21.15.157:0/2067070928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80094d0 con 0x7fe9e006d000 2023-11-06T03:19:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e58df640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 0x7fe9e00af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:52.563 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e68e1640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e60e0640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc007e70 tx=0x7fe9dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc014620 con 0x7fe9e006d000 2023-11-06T03:19:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc014780 con 0x7fe9e006d000 2023-11-06T03:19:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e0157a60 con 0x7fe9e006d000 2023-11-06T03:19:52.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc014b30 con 0x7fe9e006d000 2023-11-06T03:19:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e01576f0 con 0x7fe9e006d000 2023-11-06T03:19:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e0157240 con 0x7fe9e006d000 2023-11-06T03:19:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9dc014c90 con 0x7fe9e006d000 2023-11-06T03:19:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 0x7fe9c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:52.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe9dc083950 con 0x7fe9e006d000 2023-11-06T03:19:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e58df640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 0x7fe9c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.565+0000 7fe9e58df640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 0x7fe9c4067f20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe9e014d790 tx=0x7fe9d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:52.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.569+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe9dc0514e0 con 0x7fe9e006d000 2023-11-06T03:19:52.783 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:52.789+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7fe9e0072750 con 0x7fe9e006d000 2023-11-06T03:19:53.744 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.749+0000 7fe9d6ffd640 1 -- 172.21.15.157:0/2067070928 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v719) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe9dc054b00 con 0x7fe9e006d000 2023-11-06T03:19:53.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 msgr2=0x7fe9c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:53.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 0x7fe9c4067f20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe9e014d790 tx=0x7fe9d0009040 comp rx=0 tx=0).stop 2023-11-06T03:19:53.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 msgr2=0x7fe9e01539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:53.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc007e70 tx=0x7fe9dc00e4b0 comp rx=0 tx=0).stop 2023-11-06T03:19:53.748 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 shutdown_connections 2023-11-06T03:19:53.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe9c4065a60 0x7fe9c4067f20 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:53.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe9e01599c0 0x7fe9e00af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:53.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9e00b2af0 0x7fe9e00af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:53.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 --2- 172.21.15.157:0/2067070928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9e006d000 0x7fe9e01539f0 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:53.749 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 >> 172.21.15.157:0/2067070928 conn(0x7fe9e0065930 msgr2=0x7fe9e00b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:53.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 shutdown_connections 2023-11-06T03:19:53.750 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:53.753+0000 7fe9e8344640 1 -- 172.21.15.157:0/2067070928 wait complete. 2023-11-06T03:19:53.767 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2023-11-06T03:19:53.768 INFO:tasks.cephfs_test_runner: 2023-11-06T03:19:53.768 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-06T03:19:53.768 INFO:tasks.cephfs_test_runner:Ran 12 tests in 910.282s 2023-11-06T03:19:53.769 INFO:tasks.cephfs_test_runner: 2023-11-06T03:19:53.769 INFO:tasks.cephfs_test_runner:OK 2023-11-06T03:19:53.769 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-06T03:19:53.778 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-06T03:19:53.815 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-06T03:19:53.816 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:19:53.816 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:53.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:19:53.823 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-11-06T03:19:53.823 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:53.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336548k,nr_inodes=4084137,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15808 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-06T03:19:53.871 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:19:53.871 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:53.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:19:53.878 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-06T03:19:53.879 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-06T03:19:53.925 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336500k,nr_inodes=4084125,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-06T03:19:53.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:53.926 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2023-11-06T03:19:53.932 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-06T03:19:53.932 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-06T03:19:53.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:53.933 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-06T03:19:53.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:53.987 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-06T03:19:54.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:54.047 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2023-11-06T03:19:54.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:54.231 DEBUG:teuthology.orchestra.run.smithi195:> ip netns list 2023-11-06T03:19:54.238 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-06T03:19:54.238 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-06T03:19:54.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:54.239 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-06T03:19:54.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:54.298 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-06T03:19:54.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-06T03:19:54.367 DEBUG:teuthology.orchestra.run.smithi195:> sudo ip link delete ceph-brx 2023-11-06T03:19:54.551 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-06T03:19:54.566 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-06T03:19:54.566 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:19:54.702 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 Processor -- start 2023-11-06T03:19:54.703 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 -- start start 2023-11-06T03:19:54.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09101551e0 0x7f091006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:54.704 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091015f580 con 0x7f09100b3b20 2023-11-06T03:19:54.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091015efc0 con 0x7f091015e5e0 2023-11-06T03:19:54.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0916d01640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091015f2a0 con 0x7f09101551e0 2023-11-06T03:19:54.705 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09101551e0 0x7f091006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:54.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09101551e0 0x7f091006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57388/0 (socket says 172.21.15.157:57388) 2023-11-06T03:19:54.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 -- 172.21.15.157:0/1366296197 learned_addr learned my addr 172.21.15.157:0/1366296197 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:54.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 -- 172.21.15.157:0/1366296197 >> v1:172.21.15.195:6789/0 conn(0x7f091015e5e0 legacy=0x7f09100b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.706 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 -- 172.21.15.157:0/1366296197 >> v1:172.21.15.157:6789/0 conn(0x7f09100b3b20 legacy=0x7f091015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f0914a9d640 1 -- 172.21.15.157:0/1366296197 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091006f5a0 con 0x7f09101551e0 2023-11-06T03:19:54.707 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.709+0000 7f090f7fe640 1 -- 172.21.15.157:0/1366296197 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230233078 0 0) 0x7f091015f580 con 0x7f09100b3b20 2023-11-06T03:19:54.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0914a9d640 1 --2- 172.21.15.157:0/1366296197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f091006fda0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a1794337baff3d server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:54.711 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090f7fe640 1 -- 172.21.15.157:0/1366296197 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000ab610 con 0x7f09101551e0 2023-11-06T03:19:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090f7fe640 1 -- 172.21.15.157:0/1366296197 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09000ab770 con 0x7f09101551e0 2023-11-06T03:19:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090f7fe640 1 -- 172.21.15.157:0/1366296197 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000abb00 con 0x7f09101551e0 2023-11-06T03:19:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- 172.21.15.157:0/1366296197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 msgr2=0x7f091006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.712 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 --2- 172.21.15.157:0/1366296197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f091006fda0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090001c3f0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- 172.21.15.157:0/1366296197 shutdown_connections 2023-11-06T03:19:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 --2- 172.21.15.157:0/1366296197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f091006fda0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- 172.21.15.157:0/1366296197 >> 172.21.15.157:0/1366296197 conn(0x7f0910065930 msgr2=0x7f0910065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- 172.21.15.157:0/1366296197 shutdown_connections 2023-11-06T03:19:54.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- 172.21.15.157:0/1366296197 wait complete. 2023-11-06T03:19:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 Processor -- start 2023-11-06T03:19:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- start start 2023-11-06T03:19:54.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09100b3b20 0x7f0910149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091015e5e0 0x7f09101532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910157090 con 0x7f09101551e0 2023-11-06T03:19:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910154940 con 0x7f091015e5e0 2023-11-06T03:19:54.716 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f0916d01640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0910152520 con 0x7f09100b3b20 2023-11-06T03:19:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57402/0 (socket says 172.21.15.157:57402) 2023-11-06T03:19:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.713+0000 7f090ffff640 1 -- 172.21.15.157:0/3359850500 learned_addr learned my addr 172.21.15.157:0/3359850500 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0914a9d640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09100b3b20 0x7f0910149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:54.717 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f091529e640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091015e5e0 0x7f09101532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 -- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09100b3b20 msgr2=0x7f0910149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09100b3b20 0x7f0910149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 -- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091015e5e0 msgr2=0x7f09101532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091015e5e0 0x7f09101532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.718 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 -- 172.21.15.157:0/3359850500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000094d0 con 0x7f09101551e0 2023-11-06T03:19:54.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090ffff640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f090400c3c0 tx=0x7f090400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:54.719 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040052d0 con 0x7f09101551e0 2023-11-06T03:19:54.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904005430 con 0x7f09101551e0 2023-11-06T03:19:54.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003840 con 0x7f09101551e0 2023-11-06T03:19:54.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0910152800 con 0x7f09101551e0 2023-11-06T03:19:54.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0910152dc0 con 0x7f09101551e0 2023-11-06T03:19:54.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0910152fe0 con 0x7f09101551e0 2023-11-06T03:19:54.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09040039a0 con 0x7f09101551e0 2023-11-06T03:19:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:54.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f09040825a0 con 0x7f09101551e0 2023-11-06T03:19:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0914a9d640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.717+0000 7f0914a9d640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 0x7f08fc067f20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f09000af000 tx=0x7f09000a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:54.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.721+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f09040500a0 con 0x7f09101551e0 2023-11-06T03:19:54.868 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.873+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0910152690 con 0x7f08fc065a60 2023-11-06T03:19:54.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.881+0000 7f090d7fa640 1 -- 172.21.15.157:0/3359850500 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71897 (secure 0 0 0) 0x7f0910152690 con 0x7f08fc065a60 2023-11-06T03:19:54.877 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:54.877 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:19:54.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 msgr2=0x7f08fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 0x7f08fc067f20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f09000af000 tx=0x7f09000a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:54.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 msgr2=0x7f0910149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:54.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f090400c3c0 tx=0x7f090400c880 comp rx=0 tx=0).stop 2023-11-06T03:19:54.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 shutdown_connections 2023-11-06T03:19:54.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f08fc065a60 0x7f08fc067f20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f091015e5e0 0x7f09101532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09101551e0 0x7f0910149a80 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 --2- 172.21.15.157:0/3359850500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09100b3b20 0x7f0910149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:54.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 >> 172.21.15.157:0/3359850500 conn(0x7f0910065930 msgr2=0x7f09100ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:54.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 shutdown_connections 2023-11-06T03:19:54.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:54.885+0000 7f0916d01640 1 -- 172.21.15.157:0/3359850500 wait complete. 2023-11-06T03:19:54.896 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":668,"stamp":"2023-11-06T03:19:53.438661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":704532,"kb_used_data":44756,"kb_used_omap":0,"kb_used_meta":659712,"kb_avail":754270188,"statfs":{"total":773094113280,"available":772372672512,"internally_reserved":0,"allocated":45830144,"data_stored":35450235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":675545088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618923,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-4194304,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002144"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":310,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.961075+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:18:51.961075+0000","last_peered":"2023-11-06T03:18:51.961075+0000","last_clean":"2023-11-06T03:18:51.961075+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:18:51.961075+0000","last_undegraded":"2023-11-06T03:18:51.961075+0000","last_fullsized":"2023-11-06T03:18:51.961075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.640327+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:18:50.640327+0000","last_peered":"2023-11-06T03:18:50.640327+0000","last_clean":"2023-11-06T03:18:50.640327+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:18:50.640327+0000","last_undegraded":"2023-11-06T03:18:50.640327+0000","last_fullsized":"2023-11-06T03:18:50.640327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.960958+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:18:51.960958+0000","last_peered":"2023-11-06T03:18:51.960958+0000","last_clean":"2023-11-06T03:18:51.960958+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:18:51.960958+0000","last_undegraded":"2023-11-06T03:18:51.960958+0000","last_fullsized":"2023-11-06T03:18:51.960958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.626474+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:18:50.626474+0000","last_peered":"2023-11-06T03:18:50.626474+0000","last_clean":"2023-11-06T03:18:50.626474+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:18:50.626474+0000","last_undegraded":"2023-11-06T03:18:50.626474+0000","last_fullsized":"2023-11-06T03:18:50.626474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.639705+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:18:50.639705+0000","last_peered":"2023-11-06T03:18:50.639705+0000","last_clean":"2023-11-06T03:18:50.639705+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:18:50.639705+0000","last_undegraded":"2023-11-06T03:18:50.639705+0000","last_fullsized":"2023-11-06T03:18:50.639705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.637086+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:18:50.637086+0000","last_peered":"2023-11-06T03:18:50.637086+0000","last_clean":"2023-11-06T03:18:50.637086+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:18:50.637086+0000","last_undegraded":"2023-11-06T03:18:50.637086+0000","last_fullsized":"2023-11-06T03:18:50.637086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.961657+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:18:51.961657+0000","last_peered":"2023-11-06T03:18:51.961657+0000","last_clean":"2023-11-06T03:18:51.961657+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:18:51.961657+0000","last_undegraded":"2023-11-06T03:18:51.961657+0000","last_fullsized":"2023-11-06T03:18:51.961657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.619482+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:18:51.619482+0000","last_peered":"2023-11-06T03:18:51.619482+0000","last_clean":"2023-11-06T03:18:51.619482+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:18:51.619482+0000","last_undegraded":"2023-11-06T03:18:51.619482+0000","last_fullsized":"2023-11-06T03:18:51.619482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.627096+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:18:51.627096+0000","last_peered":"2023-11-06T03:18:51.627096+0000","last_clean":"2023-11-06T03:18:51.627096+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:18:51.627096+0000","last_undegraded":"2023-11-06T03:18:51.627096+0000","last_fullsized":"2023-11-06T03:18:51.627096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98368,"kb_used_data":5048,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94273472,"statfs":{"total":96636764160,"available":96536035328,"internally_reserved":0,"allocated":5169152,"data_stored":3857081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89200000000000002,"15min":0.82499999999999996},"min":{"1min":0.78500000000000003,"5min":0.45800000000000002,"15min":0.34999999999999998},"max":{"1min":1.161,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.67400000000000004,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.61599999999999999,"5min":0.501,"15min":0.45400000000000001},"max":{"1min":1.0780000000000001,"5min":1.2310000000000001,"15min":1.627},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.61499999999999999,"5min":0.52200000000000002,"15min":0.44500000000000001},"max":{"1min":1.2470000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.1259999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.94699999999999995,"15min":0.90500000000000003},"min":{"1min":0.71799999999999997,"5min":0.625,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":2.246,"15min":2.246},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.48999999999999999},"max":{"1min":1.26,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.90100000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.66300000000000003,"5min":0.55900000000000005,"15min":0.438},"max":{"1min":1.143,"5min":1.3680000000000001,"15min":2.512},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.032},"min":{"1min":0.81999999999999995,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.1919999999999999,"5min":1.9770000000000001,"15min":10.596},"last":1.2629999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":0.95099999999999996},"min":{"1min":0.76700000000000002,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3500000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.96299999999999997},"min":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.411,"15min":1.9399999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.04},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.54000000000000004},"max":{"1min":1.325,"5min":2.129,"15min":2.129},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.82099999999999995,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.5780000000000001,"15min":1.8839999999999999},"last":1.302}]},{"osd":6,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.80200000000000005,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.5720000000000001,"5min":1.597,"15min":9.6159999999999997},"last":1.196},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0349999999999999,"15min":1.0669999999999999},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.173,"5min":1.53,"15min":2.5430000000000001},"last":1.2270000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71532,"kb_used_data":4452,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94300308,"statfs":{"total":96636764160,"available":96563515392,"internally_reserved":0,"allocated":4558848,"data_stored":3259599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.40999999999999998},"max":{"1min":1.0920000000000001,"5min":1.2310000000000001,"15min":16.073},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88700000000000001,"15min":0.99199999999999999},"min":{"1min":0.63300000000000001,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":1.1579999999999999,"5min":1.581,"15min":40.905000000000001},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.872,"15min":0.91800000000000004},"min":{"1min":0.627,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.159,"5min":1.2230000000000001,"15min":16.012},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.58999999999999997,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.306,"5min":1.5169999999999999,"15min":3.3050000000000002},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.93799999999999994},"min":{"1min":0.69799999999999995,"5min":0.49199999999999999,"15min":0.48799999999999999},"max":{"1min":1.2010000000000001,"5min":1.3460000000000001,"15min":27.803999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.93500000000000005},"min":{"1min":0.66400000000000003,"5min":0.55800000000000005,"15min":0.50700000000000001},"max":{"1min":1.1799999999999999,"5min":1.377,"15min":2.843},"last":0.98599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94299999999999995,"15min":1.1679999999999999},"min":{"1min":0.70399999999999996,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.2370000000000001,"5min":1.389,"15min":63.737000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90900000000000003,"15min":1.0640000000000001},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.171,"5min":1.716,"15min":41.079999999999998},"last":0.80700000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":1.119},"min":{"1min":0.63500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3,"5min":1.512,"15min":24.867999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1020000000000001,"15min":1.208},"min":{"1min":0.89800000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.0389999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96299999999999997,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.694,"15min":40.959000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.79500000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.375,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.0600000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0920000000000001,"15min":1.2090000000000001},"min":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.61299999999999999},"max":{"1min":1.409,"5min":1.5309999999999999,"15min":23.748999999999999},"last":1.236},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.2130000000000001},"min":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.56499999999999995},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":28.164000000000001},"last":1.012}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75592,"kb_used_data":8512,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94296248,"statfs":{"total":96636764160,"available":96559357952,"internally_reserved":0,"allocated":8716288,"data_stored":7419215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":1.1339999999999999},"min":{"1min":0.59099999999999997,"5min":0.46300000000000002,"15min":0.309},"max":{"1min":1.0720000000000001,"5min":1.1950000000000001,"15min":125.256},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.90700000000000003,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.35499999999999998},"max":{"1min":1.274,"5min":4.7050000000000001,"15min":126.98699999999999},"last":0.88500000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.78300000000000003,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.1970000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":1.004},"min":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.6459999999999999,"15min":43.799999999999997},"last":0.63100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90800000000000003,"15min":1.0349999999999999},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.3959999999999999,"15min":43.741},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.877,"15min":0.85699999999999998},"min":{"1min":0.65700000000000003,"5min":0.51200000000000001,"15min":0.39200000000000002},"max":{"1min":1.0880000000000001,"5min":1.2470000000000001,"15min":2.9700000000000002},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":0.92500000000000004},"min":{"1min":0.76000000000000001,"5min":0.61699999999999999,"15min":0.40400000000000003},"max":{"1min":1.216,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.012,"15min":1.1319999999999999},"min":{"1min":0.81299999999999994,"5min":0.57599999999999996,"15min":0.45300000000000001},"max":{"1min":1.179,"5min":1.796,"15min":43.841999999999999},"last":1.032}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.89300000000000002,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.29299999999999998},"max":{"1min":1.196,"5min":1.595,"15min":3.1349999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0289999999999999,"15min":1.036},"min":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.524,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.0780000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.038,"15min":1.034},"min":{"1min":0.64200000000000002,"5min":0.502,"15min":0.378},"max":{"1min":1.1930000000000001,"5min":1.72,"15min":2.6909999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.024,"15min":1.0469999999999999},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.413,"15min":5.6109999999999998},"last":1.1120000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.105},"min":{"1min":0.68500000000000005,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.135},{"interface":"front","average":{"1min":1.115,"5min":1.169,"15min":1.1359999999999999},"min":{"1min":0.82399999999999995,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.3069999999999999,"5min":3.738,"15min":3.738},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117828,"kb_used_data":6460,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94254012,"statfs":{"total":96636764160,"available":96516108288,"internally_reserved":0,"allocated":6615040,"data_stored":5322573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.93500000000000005,"15min":0.99299999999999999},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":42.853999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.437,"15min":0.308},"max":{"1min":1.143,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.95899999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":0.94399999999999995,"15min":0.83899999999999997},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":11.412000000000001,"5min":11.412000000000001,"15min":11.412000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.0169999999999999,"15min":0.91000000000000003},"min":{"1min":0.63700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.5530000000000008,"5min":9.5530000000000008,"15min":9.5530000000000008},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.042,"15min":0.94899999999999995},"min":{"1min":0.77900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.6280000000000001,"5min":9.6280000000000001,"15min":9.6280000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90800000000000003,"15min":0.88},"min":{"1min":0.78700000000000003,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":3.2639999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.56899999999999995,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":3.198},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.028,"15min":0.94099999999999995},"min":{"1min":0.72199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.507,"5min":11.507,"15min":11.507},"last":0.82999999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.4780000000000002,"5min":1.2949999999999999,"15min":1.121},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":24.943999999999999,"5min":24.943999999999999,"15min":24.943999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":2.3570000000000002,"5min":1.3080000000000001,"15min":1.1499999999999999},"min":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":21.824000000000002,"5min":21.824000000000002,"15min":21.824000000000002},"last":0.76800000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":3.7890000000000001,"5min":1.502,"15min":1.153},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":47.070999999999998,"5min":47.070999999999998,"15min":47.070999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.8940000000000001,"5min":1.4419999999999999,"15min":1.1950000000000001},"min":{"1min":0.70799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":31.748999999999999,"5min":31.748999999999999,"15min":31.748999999999999},"last":0.98199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.6230000000000002,"5min":1.395,"15min":1.1839999999999999},"min":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.56599999999999995},"max":{"1min":26.669,"5min":26.669,"15min":26.669},"last":1.01},{"interface":"front","average":{"1min":2.6890000000000001,"5min":1.403,"15min":1.252},"min":{"1min":0.63100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":27.440000000000001,"5min":27.440000000000001,"15min":27.440000000000001},"last":0.93899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93716,"kb_used_data":8524,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94278124,"statfs":{"total":96636764160,"available":96540798976,"internally_reserved":0,"allocated":8728576,"data_stored":7424873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.23000000000000001,"15min":0.17399999999999999},"max":{"1min":1.0029999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.61599999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":1.1120000000000001,"5min":5.0670000000000002,"15min":5.0670000000000002},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.224},"max":{"1min":1.0309999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.32100000000000001,"15min":0.20999999999999999},"max":{"1min":1.022,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.626}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.66300000000000003,"5min":0.34699999999999998,"15min":0.23300000000000001},"max":{"1min":1.7629999999999999,"5min":3.855,"15min":3.855},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.80100000000000005,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.318,"15min":0.255},"max":{"1min":1.4359999999999999,"5min":3.645,"15min":3.645},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.51000000000000001,"15min":0.40999999999999998},"max":{"1min":1.089,"5min":5.109,"15min":5.109},"last":1.046},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0369999999999999,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.45800000000000002,"15min":0.43099999999999999},"max":{"1min":1.1639999999999999,"5min":14.170999999999999,"15min":14.170999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98999999999999999,"15min":0.96199999999999997},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.35999999999999999},"max":{"1min":1.2110000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0229999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.69399999999999995,"5min":0.629,"15min":0.39300000000000002},"max":{"1min":1.3089999999999999,"5min":3.617,"15min":3.617},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.038,"5min":0.98899999999999999,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.5,"15min":0.47899999999999998},"max":{"1min":1.2589999999999999,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.86799999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0549999999999999,"15min":1.0089999999999999},"min":{"1min":0.71199999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"max":{"1min":1.3109999999999999,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0840000000000001,"15min":1.0409999999999999},"min":{"1min":0.872,"5min":0.56200000000000006,"15min":0.48299999999999998},"max":{"1min":1.5189999999999999,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":1.083}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73540,"kb_used_data":2364,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298300,"statfs":{"total":96636764160,"available":96561459200,"internally_reserved":0,"allocated":2420736,"data_stored":1127777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.17399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.7270000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.215,"15min":0.191},"max":{"1min":1.375,"5min":1.375,"15min":1.4590000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.23000000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64600000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72999999999999998,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.4199999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.86899999999999999,"15min":0.84699999999999998},"min":{"1min":0.76900000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.526},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.997,"5min":0.91600000000000004,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"max":{"1min":1.2889999999999999,"5min":1.343,"15min":1.5229999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.44500000000000001},"max":{"1min":1.1140000000000001,"5min":1.173,"15min":1.6200000000000001},"last":0.745},{"interface":"front","average":{"1min":1.046,"5min":0.92200000000000004,"15min":0.90900000000000003},"min":{"1min":0.71899999999999997,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5780000000000001},"last":1.071}]},{"osd":6,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.98099999999999998,"15min":0.96899999999999997},"min":{"1min":0.753,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.615},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.99099999999999999,"15min":0.96499999999999997},"min":{"1min":0.80300000000000005,"5min":0.51400000000000001,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":1.532,"15min":4.2809999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.91700000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":3.4740000000000002},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73536,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298304,"statfs":{"total":96636764160,"available":96561463296,"internally_reserved":0,"allocated":2416640,"data_stored":1127687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.81200000000000006,"5min":1.042,"15min":1.843},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.0669999999999999,"5min":1.244,"15min":2.1150000000000002},"last":0.78600000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":30.861000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.20899999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.7330000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.316,"15min":0.24199999999999999},"max":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.34699999999999998,"5min":0.30599999999999999,"15min":0.20799999999999999},"max":{"1min":1.0169999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.3029999999999999,"15min":2.258},"last":0.875},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89100000000000001,"15min":0.91800000000000004},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.41699999999999998},"max":{"1min":1.2,"5min":1.329,"15min":14.753},"last":1.0029999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":43.103999999999999},"last":1.073},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0109999999999999,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.39100000000000001},"max":{"1min":1.3420000000000001,"5min":1.468,"15min":44.753999999999998},"last":1.0920000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.94399999999999995,"15min":1.004},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.45100000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":23.971},"last":1.038},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96499999999999997,"15min":1.016},"min":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.42699999999999999},"max":{"1min":1.306,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.117}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.034,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.2609999999999999,"5min":1.55,"15min":2.2799999999999998},"last":1.234},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.016,"15min":0.999},"min":{"1min":0.69199999999999995,"5min":0.58099999999999996,"15min":0.52400000000000002},"max":{"1min":1.4750000000000001,"5min":1.835,"15min":2.3559999999999999},"last":1.2050000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100420,"kb_used_data":7036,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94271420,"statfs":{"total":96636764160,"available":96533934080,"internally_reserved":0,"allocated":7204864,"data_stored":5911430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.996,"5min":1.214,"15min":2.6989999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19800000000000001},"max":{"1min":0.88600000000000001,"5min":1.45,"15min":7.1230000000000002},"last":1.0720000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.63800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.23100000000000001},"max":{"1min":0.81399999999999995,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.054,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":1.153}]},{"osd":4,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.52000000000000002,"15min":0.41499999999999998},"max":{"1min":1.151,"5min":1.419,"15min":4.641},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42899999999999999},"max":{"1min":1.1120000000000001,"5min":1.4159999999999999,"15min":2.028},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98799999999999999,"15min":0.95499999999999996},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":1.399,"5min":2.145,"15min":2.8559999999999999},"last":1.208},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98899999999999999,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.432},"max":{"1min":1.2949999999999999,"5min":1.623,"15min":4.7430000000000003},"last":1.4419999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96799999999999997,"15min":0.92100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44800000000000001},"max":{"1min":1.214,"5min":1.4359999999999999,"15min":2.931},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.016,"15min":1},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.4590000000000001,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":1.377}]},{"osd":7,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0249999999999999,"15min":0.998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.495},"max":{"1min":1.24,"5min":1.544,"15min":4.8339999999999996},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":0.97899999999999998},"min":{"1min":0.66600000000000004,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2829999999999999,"5min":1.502,"15min":1.97},"last":1.667}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:19:54.898 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:19:55.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.021+0000 7f7779fa2640 1 Processor -- start 2023-11-06T03:19:55.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.021+0000 7f7779fa2640 1 -- start start 2023-11-06T03:19:55.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7779fa2640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77740cfac0 0x7f77740cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7779fa2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 0x7f77740d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7779fa2640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740d03e0 con 0x7f77740d2ac0 2023-11-06T03:19:55.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7779fa2640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777415d260 con 0x7f77740cfac0 2023-11-06T03:19:55.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7779fa2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740d1700 con 0x7f77740d0c40 2023-11-06T03:19:55.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f77740cfac0 0x7f77740cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7773fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f77740d2ac0 0x7f77740d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36812/0 (socket says 172.21.15.157:36812) 2023-11-06T03:19:55.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7773fff640 1 -- 172.21.15.157:0/1490124382 learned_addr learned my addr 172.21.15.157:0/1490124382 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:55.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f776affd640 1 --2- 172.21.15.157:0/1490124382 >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 0x7f77740d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 -- 172.21.15.157:0/1490124382 >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 msgr2=0x7f77740d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 --2- 172.21.15.157:0/1490124382 >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 0x7f77740d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 -- 172.21.15.157:0/1490124382 >> v1:172.21.15.157:6789/0 conn(0x7f77740d2ac0 legacy=0x7f77740d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 -- 172.21.15.157:0/1490124382 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740d19e0 con 0x7f77740cfac0 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7772ffd640 1 -- 172.21.15.157:0/1490124382 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395705195 0 0) 0x7f77740d03e0 con 0x7f77740d2ac0 2023-11-06T03:19:55.021 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f776affd640 1 --2- 172.21.15.157:0/1490124382 >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 0x7f77740d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:55.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f77737fe640 1 --2- 172.21.15.157:0/1490124382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740cfac0 0x7f77740cfea0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f77740cd110 tx=0x7f776401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fa5ff0b17c2b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7772ffd640 1 -- 172.21.15.157:0/1490124382 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640a8950 con 0x7f77740cfac0 2023-11-06T03:19:55.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7772ffd640 1 -- 172.21.15.157:0/1490124382 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640a8ab0 con 0x7f77740cfac0 2023-11-06T03:19:55.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.025+0000 7f7772ffd640 1 -- 172.21.15.157:0/1490124382 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640a8e40 con 0x7f77740cfac0 2023-11-06T03:19:55.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/1490124382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740cfac0 msgr2=0x7f77740cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- 172.21.15.157:0/1490124382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740cfac0 0x7f77740cfea0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f77740cd110 tx=0x7f776401ec50 comp rx=0 tx=0).stop 2023-11-06T03:19:55.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/1490124382 shutdown_connections 2023-11-06T03:19:55.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- 172.21.15.157:0/1490124382 >> v2:172.21.15.195:3300/0 conn(0x7f77740d0c40 0x7f77740d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- 172.21.15.157:0/1490124382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740cfac0 0x7f77740cfea0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/1490124382 >> 172.21.15.157:0/1490124382 conn(0x7f7774070790 msgr2=0x7f7774070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/1490124382 shutdown_connections 2023-11-06T03:19:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/1490124382 wait complete. 2023-11-06T03:19:55.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 Processor -- start 2023-11-06T03:19:55.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- start start 2023-11-06T03:19:55.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740cfac0 0x7f77740a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77740d0c40 0x7f777414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740a9b80 con 0x7f77740d2ac0 2023-11-06T03:19:55.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741e49f0 con 0x7f77740cfac0 2023-11-06T03:19:55.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77741e4cd0 con 0x7f77740d0c40 2023-11-06T03:19:55.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57424/0 (socket says 172.21.15.157:57424) 2023-11-06T03:19:55.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 -- 172.21.15.157:0/988113549 learned_addr learned my addr 172.21.15.157:0/988113549 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:55.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f77737fe640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740cfac0 0x7f77740a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f776affd640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77740d0c40 0x7f777414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 -- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77740d0c40 msgr2=0x7f777414bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.030 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77740d0c40 0x7f777414bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 -- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740cfac0 msgr2=0x7f77740a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740cfac0 0x7f77740a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 -- 172.21.15.157:0/988113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7764009420 con 0x7f77740d2ac0 2023-11-06T03:19:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7773fff640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c780 tx=0x7f776c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.031 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c00ce90 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c00a640 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741e4fb0 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c00a9d0 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.029+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777414c500 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77741e54d0 con 0x7f77740d2ac0 2023-11-06T03:19:55.032 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f776c00ab30 con 0x7f77740d2ac0 2023-11-06T03:19:55.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f7770ff9640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 0x7f7754067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f776c082780 con 0x7f77740d2ac0 2023-11-06T03:19:55.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f77737fe640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 0x7f7754067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.033 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.033+0000 7f77737fe640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 0x7f7754067f20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f77640b2000 tx=0x7f77640aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.037+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f77741e54d0 con 0x7f77740d2ac0 2023-11-06T03:19:55.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.189+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77741e4b60 con 0x7f7754065a60 2023-11-06T03:19:55.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7770ff9640 1 -- 172.21.15.157:0/988113549 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71897 (secure 0 0 0) 0x7f77741e4b60 con 0x7f7754065a60 2023-11-06T03:19:55.190 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:19:55.191 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 msgr2=0x7f7754067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 0x7f7754067f20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f77640b2000 tx=0x7f77640aa070 comp rx=0 tx=0).stop 2023-11-06T03:19:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 msgr2=0x7f77740ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c780 tx=0x7f776c00cc40 comp rx=0 tx=0).stop 2023-11-06T03:19:55.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 shutdown_connections 2023-11-06T03:19:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7754065a60 0x7f7754067f20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77740d2ac0 0x7f77740ca710 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f77740d0c40 0x7f777414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 --2- 172.21.15.157:0/988113549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77740cfac0 0x7f77740a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 >> 172.21.15.157:0/988113549 conn(0x7f7774070790 msgr2=0x7f777415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 shutdown_connections 2023-11-06T03:19:55.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.197+0000 7f7779fa2640 1 -- 172.21.15.157:0/988113549 wait complete. 2023-11-06T03:19:55.209 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":668,"stamp":"2023-11-06T03:19:53.438661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":704532,"kb_used_data":44756,"kb_used_omap":0,"kb_used_meta":659712,"kb_avail":754270188,"statfs":{"total":773094113280,"available":772372672512,"internally_reserved":0,"allocated":45830144,"data_stored":35450235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":675545088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618923,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-4194304,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002144"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":310,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.961075+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:18:51.961075+0000","last_peered":"2023-11-06T03:18:51.961075+0000","last_clean":"2023-11-06T03:18:51.961075+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:18:51.961075+0000","last_undegraded":"2023-11-06T03:18:51.961075+0000","last_fullsized":"2023-11-06T03:18:51.961075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.640327+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:18:50.640327+0000","last_peered":"2023-11-06T03:18:50.640327+0000","last_clean":"2023-11-06T03:18:50.640327+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:18:50.640327+0000","last_undegraded":"2023-11-06T03:18:50.640327+0000","last_fullsized":"2023-11-06T03:18:50.640327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.960958+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:18:51.960958+0000","last_peered":"2023-11-06T03:18:51.960958+0000","last_clean":"2023-11-06T03:18:51.960958+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:18:51.960958+0000","last_undegraded":"2023-11-06T03:18:51.960958+0000","last_fullsized":"2023-11-06T03:18:51.960958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.626474+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:18:50.626474+0000","last_peered":"2023-11-06T03:18:50.626474+0000","last_clean":"2023-11-06T03:18:50.626474+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:18:50.626474+0000","last_undegraded":"2023-11-06T03:18:50.626474+0000","last_fullsized":"2023-11-06T03:18:50.626474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.639705+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:18:50.639705+0000","last_peered":"2023-11-06T03:18:50.639705+0000","last_clean":"2023-11-06T03:18:50.639705+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:18:50.639705+0000","last_undegraded":"2023-11-06T03:18:50.639705+0000","last_fullsized":"2023-11-06T03:18:50.639705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-11-06T03:18:50.637086+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:18:50.637086+0000","last_peered":"2023-11-06T03:18:50.637086+0000","last_clean":"2023-11-06T03:18:50.637086+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:18:50.637086+0000","last_undegraded":"2023-11-06T03:18:50.637086+0000","last_fullsized":"2023-11-06T03:18:50.637086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":157,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.961657+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:18:51.961657+0000","last_peered":"2023-11-06T03:18:51.961657+0000","last_clean":"2023-11-06T03:18:51.961657+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:18:51.961657+0000","last_undegraded":"2023-11-06T03:18:51.961657+0000","last_fullsized":"2023-11-06T03:18:51.961657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.619482+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:18:51.619482+0000","last_peered":"2023-11-06T03:18:51.619482+0000","last_clean":"2023-11-06T03:18:51.619482+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:18:51.619482+0000","last_undegraded":"2023-11-06T03:18:51.619482+0000","last_fullsized":"2023-11-06T03:18:51.619482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-11-06T03:18:51.627096+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:18:51.627096+0000","last_peered":"2023-11-06T03:18:51.627096+0000","last_clean":"2023-11-06T03:18:51.627096+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:18:51.627096+0000","last_undegraded":"2023-11-06T03:18:51.627096+0000","last_fullsized":"2023-11-06T03:18:51.627096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98368,"kb_used_data":5048,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94273472,"statfs":{"total":96636764160,"available":96536035328,"internally_reserved":0,"allocated":5169152,"data_stored":3857081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89200000000000002,"15min":0.82499999999999996},"min":{"1min":0.78500000000000003,"5min":0.45800000000000002,"15min":0.34999999999999998},"max":{"1min":1.161,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.67400000000000004,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.61599999999999999,"5min":0.501,"15min":0.45400000000000001},"max":{"1min":1.0780000000000001,"5min":1.2310000000000001,"15min":1.627},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.61499999999999999,"5min":0.52200000000000002,"15min":0.44500000000000001},"max":{"1min":1.2470000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.1259999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.94699999999999995,"15min":0.90500000000000003},"min":{"1min":0.71799999999999997,"5min":0.625,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":2.246,"15min":2.246},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.48999999999999999},"max":{"1min":1.26,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.90100000000000002}]},{"osd":3,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.66300000000000003,"5min":0.55900000000000005,"15min":0.438},"max":{"1min":1.143,"5min":1.3680000000000001,"15min":2.512},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.032},"min":{"1min":0.81999999999999995,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.1919999999999999,"5min":1.9770000000000001,"15min":10.596},"last":1.2629999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":0.95099999999999996},"min":{"1min":0.76700000000000002,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3500000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.96299999999999997},"min":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.411,"15min":1.9399999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.04},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.54000000000000004},"max":{"1min":1.325,"5min":2.129,"15min":2.129},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.82099999999999995,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.5780000000000001,"15min":1.8839999999999999},"last":1.302}]},{"osd":6,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.80200000000000005,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.5720000000000001,"5min":1.597,"15min":9.6159999999999997},"last":1.196},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0349999999999999,"15min":1.0669999999999999},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.173,"5min":1.53,"15min":2.5430000000000001},"last":1.2270000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71532,"kb_used_data":4452,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94300308,"statfs":{"total":96636764160,"available":96563515392,"internally_reserved":0,"allocated":4558848,"data_stored":3259599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.40999999999999998},"max":{"1min":1.0920000000000001,"5min":1.2310000000000001,"15min":16.073},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88700000000000001,"15min":0.99199999999999999},"min":{"1min":0.63300000000000001,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":1.1579999999999999,"5min":1.581,"15min":40.905000000000001},"last":0.65200000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.872,"15min":0.91800000000000004},"min":{"1min":0.627,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.159,"5min":1.2230000000000001,"15min":16.012},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.58999999999999997,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.306,"5min":1.5169999999999999,"15min":3.3050000000000002},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.93799999999999994},"min":{"1min":0.69799999999999995,"5min":0.49199999999999999,"15min":0.48799999999999999},"max":{"1min":1.2010000000000001,"5min":1.3460000000000001,"15min":27.803999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.93500000000000005},"min":{"1min":0.66400000000000003,"5min":0.55800000000000005,"15min":0.50700000000000001},"max":{"1min":1.1799999999999999,"5min":1.377,"15min":2.843},"last":0.98599999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94299999999999995,"15min":1.1679999999999999},"min":{"1min":0.70399999999999996,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.2370000000000001,"5min":1.389,"15min":63.737000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90900000000000003,"15min":1.0640000000000001},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.171,"5min":1.716,"15min":41.079999999999998},"last":0.80700000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":1.119},"min":{"1min":0.63500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3,"5min":1.512,"15min":24.867999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1020000000000001,"15min":1.208},"min":{"1min":0.89800000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.0389999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96299999999999997,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.694,"15min":40.959000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.79500000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.375,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.0600000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0920000000000001,"15min":1.2090000000000001},"min":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.61299999999999999},"max":{"1min":1.409,"5min":1.5309999999999999,"15min":23.748999999999999},"last":1.236},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.2130000000000001},"min":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.56499999999999995},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":28.164000000000001},"last":1.012}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75592,"kb_used_data":8512,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94296248,"statfs":{"total":96636764160,"available":96559357952,"internally_reserved":0,"allocated":8716288,"data_stored":7419215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":1.1339999999999999},"min":{"1min":0.59099999999999997,"5min":0.46300000000000002,"15min":0.309},"max":{"1min":1.0720000000000001,"5min":1.1950000000000001,"15min":125.256},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.90700000000000003,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.35499999999999998},"max":{"1min":1.274,"5min":4.7050000000000001,"15min":126.98699999999999},"last":0.88500000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.78300000000000003,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.1970000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":1.004},"min":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.6459999999999999,"15min":43.799999999999997},"last":0.63100000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90800000000000003,"15min":1.0349999999999999},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.3959999999999999,"15min":43.741},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.877,"15min":0.85699999999999998},"min":{"1min":0.65700000000000003,"5min":0.51200000000000001,"15min":0.39200000000000002},"max":{"1min":1.0880000000000001,"5min":1.2470000000000001,"15min":2.9700000000000002},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":0.92500000000000004},"min":{"1min":0.76000000000000001,"5min":0.61699999999999999,"15min":0.40400000000000003},"max":{"1min":1.216,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.012,"15min":1.1319999999999999},"min":{"1min":0.81299999999999994,"5min":0.57599999999999996,"15min":0.45300000000000001},"max":{"1min":1.179,"5min":1.796,"15min":43.841999999999999},"last":1.032}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.89300000000000002,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.29299999999999998},"max":{"1min":1.196,"5min":1.595,"15min":3.1349999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0289999999999999,"15min":1.036},"min":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.524,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.0780000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.038,"15min":1.034},"min":{"1min":0.64200000000000002,"5min":0.502,"15min":0.378},"max":{"1min":1.1930000000000001,"5min":1.72,"15min":2.6909999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.024,"15min":1.0469999999999999},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.413,"15min":5.6109999999999998},"last":1.1120000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.105},"min":{"1min":0.68500000000000005,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.135},{"interface":"front","average":{"1min":1.115,"5min":1.169,"15min":1.1359999999999999},"min":{"1min":0.82399999999999995,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.3069999999999999,"5min":3.738,"15min":3.738},"last":1.161}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117828,"kb_used_data":6460,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94254012,"statfs":{"total":96636764160,"available":96516108288,"internally_reserved":0,"allocated":6615040,"data_stored":5322573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.93500000000000005,"15min":0.99299999999999999},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":42.853999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.437,"15min":0.308},"max":{"1min":1.143,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.95899999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":0.94399999999999995,"15min":0.83899999999999997},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":11.412000000000001,"5min":11.412000000000001,"15min":11.412000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.0169999999999999,"15min":0.91000000000000003},"min":{"1min":0.63700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.5530000000000008,"5min":9.5530000000000008,"15min":9.5530000000000008},"last":0.71699999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.042,"15min":0.94899999999999995},"min":{"1min":0.77900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.6280000000000001,"5min":9.6280000000000001,"15min":9.6280000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90800000000000003,"15min":0.88},"min":{"1min":0.78700000000000003,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":3.2639999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.56899999999999995,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":3.198},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.028,"15min":0.94099999999999995},"min":{"1min":0.72199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.507,"5min":11.507,"15min":11.507},"last":0.82999999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.4780000000000002,"5min":1.2949999999999999,"15min":1.121},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":24.943999999999999,"5min":24.943999999999999,"15min":24.943999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":2.3570000000000002,"5min":1.3080000000000001,"15min":1.1499999999999999},"min":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":21.824000000000002,"5min":21.824000000000002,"15min":21.824000000000002},"last":0.76800000000000002}]},{"osd":6,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":3.7890000000000001,"5min":1.502,"15min":1.153},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":47.070999999999998,"5min":47.070999999999998,"15min":47.070999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.8940000000000001,"5min":1.4419999999999999,"15min":1.1950000000000001},"min":{"1min":0.70799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":31.748999999999999,"5min":31.748999999999999,"15min":31.748999999999999},"last":0.98199999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.6230000000000002,"5min":1.395,"15min":1.1839999999999999},"min":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.56599999999999995},"max":{"1min":26.669,"5min":26.669,"15min":26.669},"last":1.01},{"interface":"front","average":{"1min":2.6890000000000001,"5min":1.403,"15min":1.252},"min":{"1min":0.63100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":27.440000000000001,"5min":27.440000000000001,"15min":27.440000000000001},"last":0.93899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93716,"kb_used_data":8524,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94278124,"statfs":{"total":96636764160,"available":96540798976,"internally_reserved":0,"allocated":8728576,"data_stored":7424873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.23000000000000001,"15min":0.17399999999999999},"max":{"1min":1.0029999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.61599999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":1.1120000000000001,"5min":5.0670000000000002,"15min":5.0670000000000002},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.224},"max":{"1min":1.0309999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.32100000000000001,"15min":0.20999999999999999},"max":{"1min":1.022,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.626}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.66300000000000003,"5min":0.34699999999999998,"15min":0.23300000000000001},"max":{"1min":1.7629999999999999,"5min":3.855,"15min":3.855},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.80100000000000005,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.318,"15min":0.255},"max":{"1min":1.4359999999999999,"5min":3.645,"15min":3.645},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.51000000000000001,"15min":0.40999999999999998},"max":{"1min":1.089,"5min":5.109,"15min":5.109},"last":1.046},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0369999999999999,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.45800000000000002,"15min":0.43099999999999999},"max":{"1min":1.1639999999999999,"5min":14.170999999999999,"15min":14.170999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98999999999999999,"15min":0.96199999999999997},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.35999999999999999},"max":{"1min":1.2110000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0229999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.69399999999999995,"5min":0.629,"15min":0.39300000000000002},"max":{"1min":1.3089999999999999,"5min":3.617,"15min":3.617},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.038,"5min":0.98899999999999999,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.5,"15min":0.47899999999999998},"max":{"1min":1.2589999999999999,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.86799999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0549999999999999,"15min":1.0089999999999999},"min":{"1min":0.71199999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"max":{"1min":1.3109999999999999,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0840000000000001,"15min":1.0409999999999999},"min":{"1min":0.872,"5min":0.56200000000000006,"15min":0.48299999999999998},"max":{"1min":1.5189999999999999,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":1.083}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73540,"kb_used_data":2364,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298300,"statfs":{"total":96636764160,"available":96561459200,"internally_reserved":0,"allocated":2420736,"data_stored":1127777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.17399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.7270000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.215,"15min":0.191},"max":{"1min":1.375,"5min":1.375,"15min":1.4590000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.23000000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64600000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72999999999999998,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.4199999999999999},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.86899999999999999,"15min":0.84699999999999998},"min":{"1min":0.76900000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.526},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.997,"5min":0.91600000000000004,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"max":{"1min":1.2889999999999999,"5min":1.343,"15min":1.5229999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.44500000000000001},"max":{"1min":1.1140000000000001,"5min":1.173,"15min":1.6200000000000001},"last":0.745},{"interface":"front","average":{"1min":1.046,"5min":0.92200000000000004,"15min":0.90900000000000003},"min":{"1min":0.71899999999999997,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5780000000000001},"last":1.071}]},{"osd":6,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.98099999999999998,"15min":0.96899999999999997},"min":{"1min":0.753,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.615},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.99099999999999999,"15min":0.96499999999999997},"min":{"1min":0.80300000000000005,"5min":0.51400000000000001,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":1.532,"15min":4.2809999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.91700000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":3.4740000000000002},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73536,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298304,"statfs":{"total":96636764160,"available":96561463296,"internally_reserved":0,"allocated":2416640,"data_stored":1127687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.81200000000000006,"5min":1.042,"15min":1.843},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.0669999999999999,"5min":1.244,"15min":2.1150000000000002},"last":0.78600000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":30.861000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.20899999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.7330000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.316,"15min":0.24199999999999999},"max":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.34699999999999998,"5min":0.30599999999999999,"15min":0.20799999999999999},"max":{"1min":1.0169999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.3029999999999999,"15min":2.258},"last":0.875},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89100000000000001,"15min":0.91800000000000004},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.41699999999999998},"max":{"1min":1.2,"5min":1.329,"15min":14.753},"last":1.0029999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":43.103999999999999},"last":1.073},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0109999999999999,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.39100000000000001},"max":{"1min":1.3420000000000001,"5min":1.468,"15min":44.753999999999998},"last":1.0920000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.94399999999999995,"15min":1.004},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.45100000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":23.971},"last":1.038},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96499999999999997,"15min":1.016},"min":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.42699999999999999},"max":{"1min":1.306,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.117}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.034,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.2609999999999999,"5min":1.55,"15min":2.2799999999999998},"last":1.234},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.016,"15min":0.999},"min":{"1min":0.69199999999999995,"5min":0.58099999999999996,"15min":0.52400000000000002},"max":{"1min":1.4750000000000001,"5min":1.835,"15min":2.3559999999999999},"last":1.2050000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100420,"kb_used_data":7036,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94271420,"statfs":{"total":96636764160,"available":96533934080,"internally_reserved":0,"allocated":7204864,"data_stored":5911430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.996,"5min":1.214,"15min":2.6989999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19800000000000001},"max":{"1min":0.88600000000000001,"5min":1.45,"15min":7.1230000000000002},"last":1.0720000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.63800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.84399999999999997}]},{"osd":3,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.23100000000000001},"max":{"1min":0.81399999999999995,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.054,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":1.153}]},{"osd":4,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.52000000000000002,"15min":0.41499999999999998},"max":{"1min":1.151,"5min":1.419,"15min":4.641},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42899999999999999},"max":{"1min":1.1120000000000001,"5min":1.4159999999999999,"15min":2.028},"last":1.1180000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98799999999999999,"15min":0.95499999999999996},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":1.399,"5min":2.145,"15min":2.8559999999999999},"last":1.208},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98899999999999999,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.432},"max":{"1min":1.2949999999999999,"5min":1.623,"15min":4.7430000000000003},"last":1.4419999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96799999999999997,"15min":0.92100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44800000000000001},"max":{"1min":1.214,"5min":1.4359999999999999,"15min":2.931},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.016,"15min":1},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.4590000000000001,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":1.377}]},{"osd":7,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0249999999999999,"15min":0.998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.495},"max":{"1min":1.24,"5min":1.544,"15min":4.8339999999999996},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":0.97899999999999998},"min":{"1min":0.66600000000000004,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2829999999999999,"5min":1.502,"15min":1.97},"last":1.667}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:19:55.211 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-06T03:19:55.212 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:19:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.341+0000 7fec42074640 1 Processor -- start 2023-11-06T03:19:55.337 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.341+0000 7fec42074640 1 -- start start 2023-11-06T03:19:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec42074640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec3c0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec42074640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c0cf1a0 con 0x7fec3c146370 2023-11-06T03:19:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec42074640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c06c3d0 con 0x7fec3c06b6a0 2023-11-06T03:19:55.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec42074640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b2120 con 0x7fec3c0ce800 2023-11-06T03:19:55.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3b7fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fec3c146370 0x7fec3c0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36824/0 (socket says 172.21.15.157:36824) 2023-11-06T03:19:55.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3b7fe640 1 -- 172.21.15.157:0/1482877165 learned_addr learned my addr 172.21.15.157:0/1482877165 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:55.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec33fff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec3c0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199750398 0 0) 0x7fec3c0cf1a0 con 0x7fec3c146370 2023-11-06T03:19:55.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec28003530 con 0x7fec3c146370 2023-11-06T03:19:55.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1692876220 0 0) 0x7fec28003530 con 0x7fec3c146370 2023-11-06T03:19:55.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 msgr2=0x7fec3c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:55.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec3c0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 >> v1:172.21.15.195:6789/0 conn(0x7fec3c06b6a0 legacy=0x7fec3c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c06f5a0 con 0x7fec3c146370 2023-11-06T03:19:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7fec2c001680 con 0x7fec3c146370 2023-11-06T03:19:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 >> v1:172.21.15.157:6789/0 conn(0x7fec3c146370 legacy=0x7fec3c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec280039b0 0x7fec28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec28005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b2120 con 0x7fec28004d10 2023-11-06T03:19:55.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800bac0 con 0x7fec280039b0 2023-11-06T03:19:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800bc80 con 0x7fec3c0ce800 2023-11-06T03:19:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec28005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3b7fe640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec280039b0 0x7fec28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.348 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec33fff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 -- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 msgr2=0x7fec280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c0ce800 0x7fec280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 -- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec280039b0 msgr2=0x7fec28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:55.349 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec280039b0 0x7fec28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.350 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.345+0000 7fec3bfff640 1 -- 172.21.15.157:0/1482877165 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec28003530 con 0x7fec28004d10 2023-11-06T03:19:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3bfff640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec28005190 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fec34001e70 tx=0x7fec3401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88c59008a21815 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34003b80 con 0x7fec28004d10 2023-11-06T03:19:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34003ce0 con 0x7fec28004d10 2023-11-06T03:19:55.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3affd640 1 -- 172.21.15.157:0/1482877165 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340052c0 con 0x7fec28004d10 2023-11-06T03:19:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/1482877165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 msgr2=0x7fec28005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 --2- 172.21.15.157:0/1482877165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec28005190 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fec34001e70 tx=0x7fec3401c760 comp rx=0 tx=0).stop 2023-11-06T03:19:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3bfff640 1 -- 172.21.15.157:0/1482877165 reap_dead start 2023-11-06T03:19:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/1482877165 shutdown_connections 2023-11-06T03:19:55.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/1482877165 >> 172.21.15.157:0/1482877165 conn(0x7fec3c065930 msgr2=0x7fec3c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/1482877165 shutdown_connections 2023-11-06T03:19:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/1482877165 wait complete. 2023-11-06T03:19:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 Processor -- start 2023-11-06T03:19:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- start start 2023-11-06T03:19:55.356 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c1f5720 0x7fec3c160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec3c070800 0x7fec3c070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0711c0 con 0x7fec28004d10 2023-11-06T03:19:55.357 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0ae530 con 0x7fec3c070800 2023-11-06T03:19:55.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1f1fa0 con 0x7fec3c1f5720 2023-11-06T03:19:55.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57432/0 (socket says 172.21.15.157:57432) 2023-11-06T03:19:55.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 -- 172.21.15.157:0/981417084 learned_addr learned my addr 172.21.15.157:0/981417084 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:55.358 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3bfff640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec3c070800 0x7fec3c070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec33fff640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c1f5720 0x7fec3c160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 -- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c1f5720 msgr2=0x7fec3c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c1f5720 0x7fec3c160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 -- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec3c070800 msgr2=0x7fec3c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.359 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec3c070800 0x7fec3c070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 -- 172.21.15.157:0/981417084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c139710 con 0x7fec28004d10 2023-11-06T03:19:55.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec3b7fe640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fec2c011810 tx=0x7fec2c011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00fb60 con 0x7fec28004d10 2023-11-06T03:19:55.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00fcc0 con 0x7fec28004d10 2023-11-06T03:19:55.360 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c1f5c50 con 0x7fec28004d10 2023-11-06T03:19:55.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.349+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c015630 con 0x7fec28004d10 2023-11-06T03:19:55.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c1f6210 con 0x7fec28004d10 2023-11-06T03:19:55.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c072750 con 0x7fec28004d10 2023-11-06T03:19:55.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fec2c015790 con 0x7fec28004d10 2023-11-06T03:19:55.361 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec38ff9640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 0x7fec1c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fec2c086370 con 0x7fec28004d10 2023-11-06T03:19:55.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec33fff640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 0x7fec1c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.353+0000 7fec33fff640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 0x7fec1c067f20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0b19e0 tx=0x7fec24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.357+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fec2c053e70 con 0x7fec28004d10 2023-11-06T03:19:55.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.505+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec3c06fb60 con 0x7fec1c065a60 2023-11-06T03:19:55.508 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.513+0000 7fec38ff9640 1 -- 172.21.15.157:0/981417084 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71863 (secure 0 0 0) 0x7fec24004030 con 0x7fec1c065a60 2023-11-06T03:19:55.509 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:55.510 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:19:55.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 msgr2=0x7fec1c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 0x7fec1c067f20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0b19e0 tx=0x7fec24008040 comp rx=0 tx=0).stop 2023-11-06T03:19:55.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 msgr2=0x7fec3c0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.511 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7fec2c011810 tx=0x7fec2c011cd0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 shutdown_connections 2023-11-06T03:19:55.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fec1c065a60 0x7fec1c067f20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec3c070800 0x7fec3c070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fec3c1f5720 0x7fec3c160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.512 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 --2- 172.21.15.157:0/981417084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec28004d10 0x7fec3c0b1010 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 >> 172.21.15.157:0/981417084 conn(0x7fec3c065930 msgr2=0x7fec3c0b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 shutdown_connections 2023-11-06T03:19:55.513 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.517+0000 7fec42074640 1 -- 172.21.15.157:0/981417084 wait complete. 2023-11-06T03:19:55.527 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":669,"stamp":"2023-11-06T03:19:55.439236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":684252,"kb_used_data":20380,"kb_used_omap":0,"kb_used_meta":663808,"kb_avail":754290468,"statfs":{"total":773094113280,"available":772393439232,"internally_reserved":0,"allocated":20869120,"data_stored":10352703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":679739392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618923,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-4194304,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.734105"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.111770+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:19:52.111770+0000","last_peered":"2023-11-06T03:19:52.111770+0000","last_clean":"2023-11-06T03:19:52.111770+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:19:52.111770+0000","last_undegraded":"2023-11-06T03:19:52.111770+0000","last_fullsized":"2023-11-06T03:19:52.111770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.734061+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:19:50.734061+0000","last_peered":"2023-11-06T03:19:50.734061+0000","last_clean":"2023-11-06T03:19:50.734061+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:19:50.734061+0000","last_undegraded":"2023-11-06T03:19:50.734061+0000","last_fullsized":"2023-11-06T03:19:50.734061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.111529+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:19:52.111529+0000","last_peered":"2023-11-06T03:19:52.111529+0000","last_clean":"2023-11-06T03:19:52.111529+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:19:52.111529+0000","last_undegraded":"2023-11-06T03:19:52.111529+0000","last_fullsized":"2023-11-06T03:19:52.111529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:51.834457+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:19:51.834457+0000","last_peered":"2023-11-06T03:19:51.834457+0000","last_clean":"2023-11-06T03:19:51.834457+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:19:51.834457+0000","last_undegraded":"2023-11-06T03:19:51.834457+0000","last_fullsized":"2023-11-06T03:19:51.834457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.733976+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:19:50.733976+0000","last_peered":"2023-11-06T03:19:50.733976+0000","last_clean":"2023-11-06T03:19:50.733976+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:19:50.733976+0000","last_undegraded":"2023-11-06T03:19:50.733976+0000","last_fullsized":"2023-11-06T03:19:50.733976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.729447+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:19:50.729447+0000","last_peered":"2023-11-06T03:19:50.729447+0000","last_clean":"2023-11-06T03:19:50.729447+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:19:50.729447+0000","last_undegraded":"2023-11-06T03:19:50.729447+0000","last_fullsized":"2023-11-06T03:19:50.729447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:39:40.346770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.114708+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:19:52.114708+0000","last_peered":"2023-11-06T03:19:52.114708+0000","last_clean":"2023-11-06T03:19:52.114708+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:19:52.114708+0000","last_undegraded":"2023-11-06T03:19:52.114708+0000","last_fullsized":"2023-11-06T03:19:52.114708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.730108+0000","last_change":"2023-11-06T03:04:02.275860+0000","last_active":"2023-11-06T03:19:50.730108+0000","last_peered":"2023-11-06T03:19:50.730108+0000","last_clean":"2023-11-06T03:19:50.730108+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:19:50.730108+0000","last_undegraded":"2023-11-06T03:19:50.730108+0000","last_fullsized":"2023-11-06T03:19:50.730108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:49:36.339151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.117900+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:19:52.117900+0000","last_peered":"2023-11-06T03:19:52.117900+0000","last_clean":"2023-11-06T03:19:52.117900+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:19:52.117900+0000","last_undegraded":"2023-11-06T03:19:52.117900+0000","last_fullsized":"2023-11-06T03:19:52.117900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96304,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94275536,"statfs":{"total":96636764160,"available":96538148864,"internally_reserved":0,"allocated":3055616,"data_stored":1737246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89200000000000002,"15min":0.82499999999999996},"min":{"1min":0.78500000000000003,"5min":0.45800000000000002,"15min":0.34999999999999998},"max":{"1min":1.161,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.405},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.67400000000000004,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.0169999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.61599999999999999,"5min":0.501,"15min":0.45400000000000001},"max":{"1min":1.0780000000000001,"5min":1.2310000000000001,"15min":1.627},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.61499999999999999,"5min":0.52200000000000002,"15min":0.44500000000000001},"max":{"1min":1.2470000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.0740000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.94699999999999995,"15min":0.90500000000000003},"min":{"1min":0.71799999999999997,"5min":0.625,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":2.246,"15min":2.246},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.48999999999999999},"max":{"1min":1.26,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":1.3380000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.66300000000000003,"5min":0.55900000000000005,"15min":0.438},"max":{"1min":1.143,"5min":1.3680000000000001,"15min":2.512},"last":1.123},{"interface":"front","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.032},"min":{"1min":0.81999999999999995,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.1919999999999999,"5min":1.9770000000000001,"15min":10.596},"last":1.4930000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":0.95099999999999996},"min":{"1min":0.76700000000000002,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3500000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.96299999999999997},"min":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.411,"15min":1.9399999999999999},"last":1.1930000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.04},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.54000000000000004},"max":{"1min":1.325,"5min":2.129,"15min":2.129},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.82099999999999995,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.5780000000000001,"15min":1.8839999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.80200000000000005,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.5720000000000001,"5min":1.597,"15min":9.6159999999999997},"last":1.151},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0349999999999999,"15min":1.0669999999999999},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.173,"5min":1.53,"15min":2.5430000000000001},"last":1.3680000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575043,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.40999999999999998},"max":{"1min":1.0920000000000001,"5min":1.2310000000000001,"15min":16.073},"last":1.181},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88700000000000001,"15min":0.99199999999999999},"min":{"1min":0.63300000000000001,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":1.1579999999999999,"5min":1.581,"15min":40.905000000000001},"last":1.768}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.872,"15min":0.91800000000000004},"min":{"1min":0.627,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.159,"5min":1.2230000000000001,"15min":16.012},"last":1.264},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.58999999999999997,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.306,"5min":1.5169999999999999,"15min":3.3050000000000002},"last":1.708}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.93799999999999994},"min":{"1min":0.69799999999999995,"5min":0.49199999999999999,"15min":0.48799999999999999},"max":{"1min":1.2010000000000001,"5min":1.3460000000000001,"15min":27.803999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.93500000000000005},"min":{"1min":0.66400000000000003,"5min":0.55800000000000005,"15min":0.50700000000000001},"max":{"1min":1.1799999999999999,"5min":1.377,"15min":2.843},"last":2.0329999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94299999999999995,"15min":1.1679999999999999},"min":{"1min":0.70399999999999996,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.2370000000000001,"5min":1.389,"15min":63.737000000000002},"last":1.8080000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90900000000000003,"15min":1.0640000000000001},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.171,"5min":1.716,"15min":41.079999999999998},"last":0.84599999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":1.119},"min":{"1min":0.63500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3,"5min":1.512,"15min":24.867999999999999},"last":2.1070000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1020000000000001,"15min":1.208},"min":{"1min":0.89800000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.8779999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96299999999999997,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.694,"15min":40.959000000000003},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.79500000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.375,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":2.1739999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0920000000000001,"15min":1.2090000000000001},"min":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.61299999999999999},"max":{"1min":1.409,"5min":1.5309999999999999,"15min":23.748999999999999},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.2130000000000001},"min":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.56499999999999995},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":28.164000000000001},"last":1.306}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":1.1339999999999999},"min":{"1min":0.59099999999999997,"5min":0.46300000000000002,"15min":0.309},"max":{"1min":1.0720000000000001,"5min":1.1950000000000001,"15min":125.256},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.90700000000000003,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.35499999999999998},"max":{"1min":1.274,"5min":4.7050000000000001,"15min":126.98699999999999},"last":0.95099999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.78300000000000003,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.1970000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":1.004},"min":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.6459999999999999,"15min":43.799999999999997},"last":0.83299999999999996}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90800000000000003,"15min":1.0349999999999999},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.3959999999999999,"15min":43.741},"last":1.292},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.877,"15min":0.85699999999999998},"min":{"1min":0.65700000000000003,"5min":0.51200000000000001,"15min":0.39200000000000002},"max":{"1min":1.0880000000000001,"5min":1.2470000000000001,"15min":2.9700000000000002},"last":0.874}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":0.92500000000000004},"min":{"1min":0.76000000000000001,"5min":0.61699999999999999,"15min":0.40400000000000003},"max":{"1min":1.216,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":1.254},{"interface":"front","average":{"1min":1.038,"5min":1.012,"15min":1.1319999999999999},"min":{"1min":0.81299999999999994,"5min":0.57599999999999996,"15min":0.45300000000000001},"max":{"1min":1.179,"5min":1.796,"15min":43.841999999999999},"last":1.121}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.89300000000000002,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.29299999999999998},"max":{"1min":1.196,"5min":1.595,"15min":3.1349999999999998},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0289999999999999,"15min":1.036},"min":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.524,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.038,"15min":1.034},"min":{"1min":0.64200000000000002,"5min":0.502,"15min":0.378},"max":{"1min":1.1930000000000001,"5min":1.72,"15min":2.6909999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.024,"15min":1.0469999999999999},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.413,"15min":5.6109999999999998},"last":1.3260000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.105},"min":{"1min":0.68500000000000005,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.169,"15min":1.1359999999999999},"min":{"1min":0.82399999999999995,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.3069999999999999,"5min":3.738,"15min":3.738},"last":1.5580000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113772,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258068,"statfs":{"total":96636764160,"available":96520261632,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.93500000000000005,"15min":0.99299999999999999},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":42.853999999999999},"last":1.008},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.437,"15min":0.308},"max":{"1min":1.143,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.85899999999999999}]},{"osd":1,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":0.94399999999999995,"15min":0.83899999999999997},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":11.412000000000001,"5min":11.412000000000001,"15min":11.412000000000001},"last":1.034},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.0169999999999999,"15min":0.91000000000000003},"min":{"1min":0.63700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.5530000000000008,"5min":9.5530000000000008,"15min":9.5530000000000008},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.042,"15min":0.94899999999999995},"min":{"1min":0.77900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.6280000000000001,"5min":9.6280000000000001,"15min":9.6280000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90800000000000003,"15min":0.88},"min":{"1min":0.78700000000000003,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":3.2639999999999998},"last":1.083}]},{"osd":3,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.56899999999999995,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":3.198},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.028,"15min":0.94099999999999995},"min":{"1min":0.72199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.507,"5min":11.507,"15min":11.507},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.4780000000000002,"5min":1.2949999999999999,"15min":1.121},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":24.943999999999999,"5min":24.943999999999999,"15min":24.943999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":2.3570000000000002,"5min":1.3080000000000001,"15min":1.1499999999999999},"min":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":21.824000000000002,"5min":21.824000000000002,"15min":21.824000000000002},"last":1.173}]},{"osd":6,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":3.7890000000000001,"5min":1.502,"15min":1.153},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":47.070999999999998,"5min":47.070999999999998,"15min":47.070999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":2.8940000000000001,"5min":1.4419999999999999,"15min":1.1950000000000001},"min":{"1min":0.70799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":31.748999999999999,"5min":31.748999999999999,"15min":31.748999999999999},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.6230000000000002,"5min":1.395,"15min":1.1839999999999999},"min":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.56599999999999995},"max":{"1min":26.669,"5min":26.669,"15min":26.669},"last":1.133},{"interface":"front","average":{"1min":2.6890000000000001,"5min":1.403,"15min":1.252},"min":{"1min":0.63100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":27.440000000000001,"5min":27.440000000000001,"15min":27.440000000000001},"last":1.1120000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87592,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94284248,"statfs":{"total":96636764160,"available":96547069952,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.23000000000000001,"15min":0.17399999999999999},"max":{"1min":1.0029999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.61599999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":1.1120000000000001,"5min":5.0670000000000002,"15min":5.0670000000000002},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.224},"max":{"1min":1.0309999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.32100000000000001,"15min":0.20999999999999999},"max":{"1min":1.022,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.79700000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.66300000000000003,"5min":0.34699999999999998,"15min":0.23300000000000001},"max":{"1min":1.7629999999999999,"5min":3.855,"15min":3.855},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.80100000000000005,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.318,"15min":0.255},"max":{"1min":1.4359999999999999,"5min":3.645,"15min":3.645},"last":0.92600000000000005}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.51000000000000001,"15min":0.40999999999999998},"max":{"1min":1.089,"5min":5.109,"15min":5.109},"last":1.048},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0369999999999999,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.45800000000000002,"15min":0.43099999999999999},"max":{"1min":1.1639999999999999,"5min":14.170999999999999,"15min":14.170999999999999},"last":0.98199999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98999999999999999,"15min":0.96199999999999997},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.35999999999999999},"max":{"1min":1.2110000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0649999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.69399999999999995,"5min":0.629,"15min":0.39300000000000002},"max":{"1min":1.3089999999999999,"5min":3.617,"15min":3.617},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.038,"5min":0.98899999999999999,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.5,"15min":0.47899999999999998},"max":{"1min":1.2589999999999999,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.95799999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0549999999999999,"15min":1.0089999999999999},"min":{"1min":0.71199999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"max":{"1min":1.3109999999999999,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0840000000000001,"15min":1.0409999999999999},"min":{"1min":0.872,"5min":0.56200000000000006,"15min":0.48299999999999998},"max":{"1min":1.5189999999999999,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":1.022}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.17399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.7270000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.215,"15min":0.191},"max":{"1min":1.375,"5min":1.375,"15min":1.4590000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.23000000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64600000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.443}]},{"osd":3,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72999999999999998,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.4199999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.86899999999999999,"15min":0.84699999999999998},"min":{"1min":0.76900000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.526},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.997,"5min":0.91600000000000004,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"max":{"1min":1.2889999999999999,"5min":1.343,"15min":1.5229999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.44500000000000001},"max":{"1min":1.1140000000000001,"5min":1.173,"15min":1.6200000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.046,"5min":0.92200000000000004,"15min":0.90900000000000003},"min":{"1min":0.71899999999999997,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5780000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.98099999999999998,"15min":0.96899999999999997},"min":{"1min":0.753,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.615},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.036,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84799999999999998}]},{"osd":7,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.99099999999999999,"15min":0.96499999999999997},"min":{"1min":0.80300000000000005,"5min":0.51400000000000001,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":1.532,"15min":4.2809999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.91700000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":3.4740000000000002},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.81200000000000006,"5min":1.042,"15min":1.843},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.0669999999999999,"5min":1.244,"15min":2.1150000000000002},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":30.861000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.20899999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.7330000000000001},"last":0.88}]},{"osd":3,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.316,"15min":0.24199999999999999},"max":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.34699999999999998,"5min":0.30599999999999999,"15min":0.20799999999999999},"max":{"1min":1.0169999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.3029999999999999,"15min":2.258},"last":0.755},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89100000000000001,"15min":0.91800000000000004},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.41699999999999998},"max":{"1min":1.2,"5min":1.329,"15min":14.753},"last":1.052}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":43.103999999999999},"last":1.117},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0109999999999999,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.39100000000000001},"max":{"1min":1.3420000000000001,"5min":1.468,"15min":44.753999999999998},"last":1.288}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.94399999999999995,"15min":1.004},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.45100000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":23.971},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96499999999999997,"15min":1.016},"min":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.42699999999999999},"max":{"1min":1.306,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.1870000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.034,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.2609999999999999,"5min":1.55,"15min":2.2799999999999998},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.016,"15min":0.999},"min":{"1min":0.69199999999999995,"5min":0.58099999999999996,"15min":0.52400000000000002},"max":{"1min":1.4750000000000001,"5min":1.835,"15min":2.3559999999999999},"last":1.0209999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575042,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100460,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":97472,"kb_avail":94271380,"statfs":{"total":96636764160,"available":96533893120,"internally_reserved":0,"allocated":3051520,"data_stored":1736227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":99811328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.996,"5min":1.214,"15min":2.6989999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19800000000000001},"max":{"1min":0.88600000000000001,"5min":1.45,"15min":7.1230000000000002},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.63800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.90800000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.23100000000000001},"max":{"1min":0.81399999999999995,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.879},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.054,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":0.98599999999999999}]},{"osd":4,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.52000000000000002,"15min":0.41499999999999998},"max":{"1min":1.151,"5min":1.419,"15min":4.641},"last":1.081},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42899999999999999},"max":{"1min":1.1120000000000001,"5min":1.4159999999999999,"15min":2.028},"last":0.77600000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98799999999999999,"15min":0.95499999999999996},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":1.399,"5min":2.145,"15min":2.8559999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98899999999999999,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.432},"max":{"1min":1.2949999999999999,"5min":1.623,"15min":4.7430000000000003},"last":1.161}]},{"osd":6,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96799999999999997,"15min":0.92100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44800000000000001},"max":{"1min":1.214,"5min":1.4359999999999999,"15min":2.931},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.016,"15min":1},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.4590000000000001,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":1.286}]},{"osd":7,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0249999999999999,"15min":0.998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.495},"max":{"1min":1.24,"5min":1.544,"15min":4.8339999999999996},"last":1.242},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":0.97899999999999998},"min":{"1min":0.66600000000000004,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2829999999999999,"5min":1.502,"15min":1.97},"last":1.0409999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:19:55.530 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-06T03:19:55.640 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.645+0000 7f1f79881640 1 Processor -- start 2023-11-06T03:19:55.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.645+0000 7f1f79881640 1 -- start start 2023-11-06T03:19:55.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f79881640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1f741584b0 0x7f1f7415e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.641 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f79881640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c9480 0x7f1f740cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f79881640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740b2b20 con 0x7f1f740c98e0 2023-11-06T03:19:55.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f79881640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7415af60 con 0x7f1f741584b0 2023-11-06T03:19:55.642 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f79881640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b2e00 con 0x7f1f740c9480 2023-11-06T03:19:55.643 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f737fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1f740c98e0 0x7f1f7415b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52082/0 (socket says 172.21.15.157:52082) 2023-11-06T03:19:55.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f737fe640 1 -- 172.21.15.157:0/3908197864 learned_addr learned my addr 172.21.15.157:0/3908197864 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:55.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/3908197864 >> v2:172.21.15.195:3300/0 conn(0x7f1f741584b0 0x7f1f7415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f727fc640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c9480 0x7f1f740cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f71ffb640 1 -- 172.21.15.157:0/3908197864 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823103371 0 0) 0x7f1f740b2b20 con 0x7f1f740c98e0 2023-11-06T03:19:55.644 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f71ffb640 1 -- 172.21.15.157:0/3908197864 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f5c003530 con 0x7f1f740c98e0 2023-11-06T03:19:55.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 -- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c9480 msgr2=0x7f1f740cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:55.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c9480 0x7f1f740cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 -- 172.21.15.157:0/3908197864 >> v1:172.21.15.195:6789/0 conn(0x7f1f740c98e0 legacy=0x7f1f7415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.645 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 -- 172.21.15.157:0/3908197864 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740b30e0 con 0x7f1f741584b0 2023-11-06T03:19:55.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.649+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f741584b0 0x7f1f7415e550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1f640097c0 tx=0x7f1f6401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448bcc7c5a0ddcc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f71ffb640 1 -- 172.21.15.157:0/3908197864 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640ad660 con 0x7f1f741584b0 2023-11-06T03:19:55.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f71ffb640 1 -- 172.21.15.157:0/3908197864 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f640ad7c0 con 0x7f1f741584b0 2023-11-06T03:19:55.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f71ffb640 1 -- 172.21.15.157:0/3908197864 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640adb30 con 0x7f1f741584b0 2023-11-06T03:19:55.647 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f741584b0 msgr2=0x7f1f7415e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f741584b0 0x7f1f7415e550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1f640097c0 tx=0x7f1f6401c450 comp rx=0 tx=0).stop 2023-11-06T03:19:55.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/3908197864 shutdown_connections 2023-11-06T03:19:55.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f740c9480 0x7f1f740cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- 172.21.15.157:0/3908197864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f741584b0 0x7f1f7415e550 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.648 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/3908197864 >> 172.21.15.157:0/3908197864 conn(0x7f1f7406a170 msgr2=0x7f1f7406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/3908197864 shutdown_connections 2023-11-06T03:19:55.649 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/3908197864 wait complete. 2023-11-06T03:19:55.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 Processor -- start 2023-11-06T03:19:55.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- start start 2023-11-06T03:19:55.651 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f740c98e0 0x7f1f740c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f741584b0 0x7f1f74156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c83c0 con 0x7f1f740c9480 2023-11-06T03:19:55.652 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c9ec0 con 0x7f1f740c98e0 2023-11-06T03:19:55.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c85f0 con 0x7f1f741584b0 2023-11-06T03:19:55.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57446/0 (socket says 172.21.15.157:57446) 2023-11-06T03:19:55.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 -- 172.21.15.157:0/1728035666 learned_addr learned my addr 172.21.15.157:0/1728035666 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:55.653 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f727fc640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f740c98e0 0x7f1f740c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f737fe640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f741584b0 0x7f1f74156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 -- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f741584b0 msgr2=0x7f1f74156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f741584b0 0x7f1f74156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 -- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f740c98e0 msgr2=0x7f1f740c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.654 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f740c98e0 0x7f1f740c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 -- 172.21.15.157:0/1728035666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009420 con 0x7f1f740c9480 2023-11-06T03:19:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f72ffd640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f1f6401a080 tx=0x7f1f640191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640adb30 con 0x7f1f740c9480 2023-11-06T03:19:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f640adc90 con 0x7f1f740c9480 2023-11-06T03:19:55.655 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740c8890 con 0x7f1f740c9480 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003620 con 0x7f1f740c9480 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.653+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740c8e50 con 0x7f1f740c9480 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7415e8c0 con 0x7f1f740c9480 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f640b4050 con 0x7f1f740c9480 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f5bfff640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 0x7f1f60067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:55.656 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1f6411d790 con 0x7f1f740c9480 2023-11-06T03:19:55.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f727fc640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 0x7f1f60067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:55.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.657+0000 7f1f727fc640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 0x7f1f60067f20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1f74158480 tx=0x7f1f68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:55.657 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.661+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f1f640eb2a0 con 0x7f1f740c9480 2023-11-06T03:19:55.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.805+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f74072750 con 0x7f1f740c9480 2023-11-06T03:19:55.802 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.809+0000 7f1f5bfff640 1 -- 172.21.15.157:0/1728035666 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+14490 (secure 0 0 0) 0x7f1f640ee8c0 con 0x7f1f740c9480 2023-11-06T03:19:55.803 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:19:55.803 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":195,"fsid":"eca2e009-f15d-40e2-b739-48eaa1ef5309","created":"2023-11-06T03:03:42.472609+0000","modified":"2023-11-06T03:19:51.725247+0000","last_up_change":"2023-11-06T03:03:53.744094+0000","last_in_change":"2023-11-06T03:03:45.158771+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-06T03:03:55.172089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-06T03:04:00.991075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"39ef8ea4-08cc-4be6-a7ec-19244f68d5c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":38220},{"type":"v1","addr":"172.21.15.157:6803","nonce":38220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":38220},{"type":"v1","addr":"172.21.15.157:6808","nonce":38220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":38220},{"type":"v1","addr":"172.21.15.157:6815","nonce":38220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":38220},{"type":"v1","addr":"172.21.15.157:6812","nonce":38220}]},"public_addr":"172.21.15.157:6803/38220","cluster_addr":"172.21.15.157:6808/38220","heartbeat_back_addr":"172.21.15.157:6815/38220","heartbeat_front_addr":"172.21.15.157:6812/38220","state":["exists","up"]},{"osd":1,"uuid":"720566bd-81fd-4db1-8eba-61ea738b0325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":38221},{"type":"v1","addr":"172.21.15.157:6802","nonce":38221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":38221},{"type":"v1","addr":"172.21.15.157:6805","nonce":38221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6811","nonce":38221},{"type":"v1","addr":"172.21.15.157:6813","nonce":38221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6807","nonce":38221},{"type":"v1","addr":"172.21.15.157:6809","nonce":38221}]},"public_addr":"172.21.15.157:6802/38221","cluster_addr":"172.21.15.157:6805/38221","heartbeat_back_addr":"172.21.15.157:6813/38221","heartbeat_front_addr":"172.21.15.157:6809/38221","state":["exists","up"]},{"osd":2,"uuid":"530cb9e2-3d91-43f6-87fe-af5789a292e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":38222},{"type":"v1","addr":"172.21.15.157:6817","nonce":38222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":38222},{"type":"v1","addr":"172.21.15.157:6819","nonce":38222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":38222},{"type":"v1","addr":"172.21.15.157:6823","nonce":38222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":38222},{"type":"v1","addr":"172.21.15.157:6821","nonce":38222}]},"public_addr":"172.21.15.157:6817/38222","cluster_addr":"172.21.15.157:6819/38222","heartbeat_back_addr":"172.21.15.157:6823/38222","heartbeat_front_addr":"172.21.15.157:6821/38222","state":["exists","up"]},{"osd":3,"uuid":"4c2c4aaf-be58-419f-ae34-a13d3aafdf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":38223},{"type":"v1","addr":"172.21.15.157:6825","nonce":38223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":38223},{"type":"v1","addr":"172.21.15.157:6827","nonce":38223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":38223},{"type":"v1","addr":"172.21.15.157:6831","nonce":38223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":38223},{"type":"v1","addr":"172.21.15.157:6829","nonce":38223}]},"public_addr":"172.21.15.157:6825/38223","cluster_addr":"172.21.15.157:6827/38223","heartbeat_back_addr":"172.21.15.157:6831/38223","heartbeat_front_addr":"172.21.15.157:6829/38223","state":["exists","up"]},{"osd":4,"uuid":"f4d19f9a-a162-4f54-aee2-b326475860e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":38425},{"type":"v1","addr":"172.21.15.195:6809","nonce":38425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":38425},{"type":"v1","addr":"172.21.15.195:6811","nonce":38425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":38425},{"type":"v1","addr":"172.21.15.195:6815","nonce":38425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":38425},{"type":"v1","addr":"172.21.15.195:6813","nonce":38425}]},"public_addr":"172.21.15.195:6809/38425","cluster_addr":"172.21.15.195:6811/38425","heartbeat_back_addr":"172.21.15.195:6815/38425","heartbeat_front_addr":"172.21.15.195:6813/38425","state":["exists","up"]},{"osd":5,"uuid":"2822a15c-dc53-4d42-9d13-2b2b0d914eef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":38428},{"type":"v1","addr":"172.21.15.195:6818","nonce":38428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":38428},{"type":"v1","addr":"172.21.15.195:6821","nonce":38428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":38428},{"type":"v1","addr":"172.21.15.195:6827","nonce":38428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":38428},{"type":"v1","addr":"172.21.15.195:6824","nonce":38428}]},"public_addr":"172.21.15.195:6818/38428","cluster_addr":"172.21.15.195:6821/38428","heartbeat_back_addr":"172.21.15.195:6827/38428","heartbeat_front_addr":"172.21.15.195:6824/38428","state":["exists","up"]},{"osd":6,"uuid":"41bae808-491c-4ce9-9f1a-22c974778030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6817","nonce":38426},{"type":"v1","addr":"172.21.15.195:6820","nonce":38426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6823","nonce":38426},{"type":"v1","addr":"172.21.15.195:6825","nonce":38426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":38426},{"type":"v1","addr":"172.21.15.195:6831","nonce":38426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":38426},{"type":"v1","addr":"172.21.15.195:6829","nonce":38426}]},"public_addr":"172.21.15.195:6820/38426","cluster_addr":"172.21.15.195:6825/38426","heartbeat_back_addr":"172.21.15.195:6831/38426","heartbeat_front_addr":"172.21.15.195:6829/38426","state":["exists","up"]},{"osd":7,"uuid":"3ceafe59-a8eb-4593-911c-f9e8338dafb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":38427},{"type":"v1","addr":"172.21.15.195:6801","nonce":38427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":38427},{"type":"v1","addr":"172.21.15.195:6803","nonce":38427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":38427},{"type":"v1","addr":"172.21.15.195:6807","nonce":38427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":38427},{"type":"v1","addr":"172.21.15.195:6805","nonce":38427}]},"public_addr":"172.21.15.195:6801/38427","cluster_addr":"172.21.15.195:6803/38427","heartbeat_back_addr":"172.21.15.195:6807/38427","heartbeat_front_addr":"172.21.15.195:6805/38427","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.815031+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.774811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.629010+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.696563+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.654295+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.665445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.676124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-06T03:03:50.666985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2374994036":"2023-11-07T03:19:48.699023+0000","172.21.15.157:6834/2374994036":"2023-11-07T03:19:48.699023+0000","172.21.15.157:6835/877366447":"2023-11-07T03:18:36.902417+0000","172.21.15.157:6834/877366447":"2023-11-07T03:18:36.902417+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-06T03:19:55.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 msgr2=0x7f1f60067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 0x7f1f60067f20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1f74158480 tx=0x7f1f68008040 comp rx=0 tx=0).stop 2023-11-06T03:19:55.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 msgr2=0x7f1f740c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:55.806 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f1f6401a080 tx=0x7f1f640191f0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 shutdown_connections 2023-11-06T03:19:55.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f1f60065a60 0x7f1f60067f20 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1f741584b0 0x7f1f74156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1f740c98e0 0x7f1f740c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.807 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 --2- 172.21.15.157:0/1728035666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f740c9480 0x7f1f740c3380 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:55.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 >> 172.21.15.157:0/1728035666 conn(0x7f1f7406a170 msgr2=0x7f1f7406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:55.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 shutdown_connections 2023-11-06T03:19:55.808 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:55.813+0000 7f1f79881640 1 -- 172.21.15.157:0/1728035666 wait complete. 2023-11-06T03:19:56.823 INFO:tasks.ceph:Scrubbing osd.0 2023-11-06T03:19:56.823 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:56.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 Processor -- start 2023-11-06T03:19:56.960 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 -- start start 2023-11-06T03:19:56.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b2c06c170 0x7f3b2c154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.961 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3b2c06c550 0x7f3b2c155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c15b150 0x7f3b2c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c0cb100 con 0x7f3b2c06c170 2023-11-06T03:19:56.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c15bef0 con 0x7f3b2c06c550 2023-11-06T03:19:56.962 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b32f10640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0cb350 con 0x7f3b2c15b150 2023-11-06T03:19:56.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b2c06c170 0x7f3b2c154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b2c06c170 0x7f3b2c154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57450/0 (socket says 172.21.15.157:57450) 2023-11-06T03:19:56.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 -- 172.21.15.157:0/212689392 learned_addr learned my addr 172.21.15.157:0/212689392 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:56.963 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b314ad640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c15b150 0x7f3b2c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b2bfff640 1 --2- 172.21.15.157:0/212689392 >> v2:172.21.15.195:3300/0 conn(0x7f3b2c06c550 0x7f3b2c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 -- 172.21.15.157:0/212689392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c15b150 msgr2=0x7f3b2c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:56.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c15b150 0x7f3b2c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 -- 172.21.15.157:0/212689392 >> v2:172.21.15.195:3300/0 conn(0x7f3b2c06c550 msgr2=0x7f3b2c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:56.964 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 --2- 172.21.15.157:0/212689392 >> v2:172.21.15.195:3300/0 conn(0x7f3b2c06c550 0x7f3b2c155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.965 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.965+0000 7f3b30cac640 1 -- 172.21.15.157:0/212689392 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0cb630 con 0x7f3b2c06c170 2023-11-06T03:19:56.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b30cac640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c170 0x7f3b2c154c00 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0cc2e0 tx=0x7f3b1c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a824d49976bc1c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:56.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b2b7fe640 1 -- 172.21.15.157:0/212689392 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0ac410 con 0x7f3b2c06c170 2023-11-06T03:19:56.968 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b2b7fe640 1 -- 172.21.15.157:0/212689392 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c0ac570 con 0x7f3b2c06c170 2023-11-06T03:19:56.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b2b7fe640 1 -- 172.21.15.157:0/212689392 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0ac900 con 0x7f3b2c06c170 2023-11-06T03:19:56.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 -- 172.21.15.157:0/212689392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c170 msgr2=0x7f3b2c154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c170 0x7f3b2c154c00 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0cc2e0 tx=0x7f3b1c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.969 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 -- 172.21.15.157:0/212689392 shutdown_connections 2023-11-06T03:19:56.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c15b150 0x7f3b2c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 --2- 172.21.15.157:0/212689392 >> v2:172.21.15.195:3300/0 conn(0x7f3b2c06c550 0x7f3b2c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 --2- 172.21.15.157:0/212689392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c170 0x7f3b2c154c00 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.970 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 -- 172.21.15.157:0/212689392 >> 172.21.15.157:0/212689392 conn(0x7f3b2c0b43b0 msgr2=0x7f3b2c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 -- 172.21.15.157:0/212689392 shutdown_connections 2023-11-06T03:19:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.969+0000 7f3b32f10640 1 -- 172.21.15.157:0/212689392 wait complete. 2023-11-06T03:19:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 Processor -- start 2023-11-06T03:19:56.971 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- start start 2023-11-06T03:19:56.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c06c170 0x7f3b2c0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 0x7f3b2c159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b2c15b150 0x7f3b2c159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c1f3040 con 0x7f3b2c06c550 2023-11-06T03:19:56.972 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0abd90 con 0x7f3b2c15b150 2023-11-06T03:19:56.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0abab0 con 0x7f3b2c06c170 2023-11-06T03:19:56.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 0x7f3b2c159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 0x7f3b2c159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57462/0 (socket says 172.21.15.157:57462) 2023-11-06T03:19:56.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 -- 172.21.15.157:0/77296088 learned_addr learned my addr 172.21.15.157:0/77296088 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:56.973 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b30cac640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c06c170 0x7f3b2c0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b314ad640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b2c15b150 0x7f3b2c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 -- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c06c170 msgr2=0x7f3b2c0ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2c06c170 0x7f3b2c0ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 -- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b2c15b150 msgr2=0x7f3b2c159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b2c15b150 0x7f3b2c159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0094d0 con 0x7f3b2c06c550 2023-11-06T03:19:56.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b2bfff640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 0x7f3b2c159230 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000d800 tx=0x7f3b2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:56.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000b6c0 con 0x7f3b2c06c550 2023-11-06T03:19:56.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2000b820 con 0x7f3b2c06c550 2023-11-06T03:19:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c1f32a0 con 0x7f3b2c06c550 2023-11-06T03:19:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000bb90 con 0x7f3b2c06c550 2023-11-06T03:19:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c1f3860 con 0x7f3b2c06c550 2023-11-06T03:19:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b2c0cf190 con 0x7f3b2c06c550 2023-11-06T03:19:56.976 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b2000bcf0 con 0x7f3b2c06c550 2023-11-06T03:19:56.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b18065a60 0x7f3b18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3b20083f80 con 0x7f3b2c06c550 2023-11-06T03:19:56.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f3b1806bf80 0x7f3b1806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:56.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b1806ea50 con 0x7f3b1806bf80 2023-11-06T03:19:56.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b200843d0 con 0x7f3b2c06c550 2023-11-06T03:19:56.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b314ad640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f3b1806bf80 0x7f3b1806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.973+0000 7f3b30cac640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b18065a60 0x7f3b18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:56.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.977+0000 7f3b314ad640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f3b1806bf80 0x7f3b1806e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:56.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.977+0000 7f3b30cac640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b18065a60 0x7f3b18067f20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0ae000 tx=0x7f3b1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:56.980 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.977+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== osd.0 v2:172.21.15.157:6801/38220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b1806ea50 con 0x7f3b1806bf80 2023-11-06T03:19:56.982 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 --> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b2c1f3a20 con 0x7f3b1806bf80 2023-11-06T03:19:56.983 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b297fa640 1 -- 172.21.15.157:0/77296088 <== osd.0 v2:172.21.15.157:6801/38220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b2c1f3a20 con 0x7f3b1806bf80 2023-11-06T03:19:56.983 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:56.983 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:56.984 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:56.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f3b1806bf80 msgr2=0x7f3b1806e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:6801/38220,v1:172.21.15.157:6803/38220] conn(0x7f3b1806bf80 0x7f3b1806e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b18065a60 msgr2=0x7f3b18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.985 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b18065a60 0x7f3b18067f20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0ae000 tx=0x7f3b1c008040 comp rx=0 tx=0).stop 2023-11-06T03:19:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 msgr2=0x7f3b2c159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 --2- 172.21.15.157:0/77296088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c06c550 0x7f3b2c159230 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000d800 tx=0x7f3b2000dcc0 comp rx=0 tx=0).stop 2023-11-06T03:19:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b314ad640 1 -- 172.21.15.157:0/77296088 reap_dead start 2023-11-06T03:19:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 shutdown_connections 2023-11-06T03:19:56.986 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 >> 172.21.15.157:0/77296088 conn(0x7f3b2c0b43b0 msgr2=0x7f3b2c0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:56.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 shutdown_connections 2023-11-06T03:19:56.987 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:56.989+0000 7f3b32f10640 1 -- 172.21.15.157:0/77296088 wait complete. 2023-11-06T03:19:56.999 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-06T03:19:57.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 Processor -- start 2023-11-06T03:19:57.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 -- start start 2023-11-06T03:19:57.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe738156a70 0x7fe73806f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7380b2330 0x7fe73806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73806c490 con 0x7fe7380adae0 2023-11-06T03:19:57.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73806b950 con 0x7fe7380b2330 2023-11-06T03:19:57.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73ff4a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73806bb60 con 0x7fe738156a70 2023-11-06T03:19:57.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7380b2330 0x7fe73806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe7380b2330 0x7fe73806c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57468/0 (socket says 172.21.15.157:57468) 2023-11-06T03:19:57.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 -- 172.21.15.157:0/1904209050 learned_addr learned my addr 172.21.15.157:0/1904209050 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 -- 172.21.15.157:0/1904209050 >> v2:172.21.15.195:3300/0 conn(0x7fe738156a70 msgr2=0x7fe73806f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:57.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 --2- 172.21.15.157:0/1904209050 >> v2:172.21.15.195:3300/0 conn(0x7fe738156a70 0x7fe73806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 -- 172.21.15.157:0/1904209050 >> v1:172.21.15.195:6789/0 conn(0x7fe7380adae0 legacy=0x7fe7380b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:57.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 -- 172.21.15.157:0/1904209050 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73806be40 con 0x7fe7380b2330 2023-11-06T03:19:57.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73e4e7640 1 --2- 172.21.15.157:0/1904209050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380b2330 0x7fe73806c920 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ce880 tx=0x7fe73001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe916401fc5f021 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73cce4640 1 -- 172.21.15.157:0/1904209050 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300a7c50 con 0x7fe7380b2330 2023-11-06T03:19:57.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73cce4640 1 -- 172.21.15.157:0/1904209050 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7300ab070 con 0x7fe7380b2330 2023-11-06T03:19:57.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.112+0000 7fe73cce4640 1 -- 172.21.15.157:0/1904209050 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300b1400 con 0x7fe7380b2330 2023-11-06T03:19:57.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/1904209050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380b2330 msgr2=0x7fe73806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/1904209050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380b2330 0x7fe73806c920 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7fe7380ce880 tx=0x7fe73001eb60 comp rx=0 tx=0).stop 2023-11-06T03:19:57.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/1904209050 shutdown_connections 2023-11-06T03:19:57.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/1904209050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380b2330 0x7fe73806c920 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/1904209050 >> v2:172.21.15.195:3300/0 conn(0x7fe738156a70 0x7fe73806f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/1904209050 >> 172.21.15.157:0/1904209050 conn(0x7fe738065930 msgr2=0x7fe738065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.112 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/1904209050 shutdown_connections 2023-11-06T03:19:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/1904209050 wait complete. 2023-11-06T03:19:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 Processor -- start 2023-11-06T03:19:57.113 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- start start 2023-11-06T03:19:57.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7380b2330 0x7fe7380b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe738156a70 0x7fe73814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73814fa70 con 0x7fe7380adae0 2023-11-06T03:19:57.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7381503d0 con 0x7fe7380b2330 2023-11-06T03:19:57.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73814edf0 con 0x7fe738156a70 2023-11-06T03:19:57.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57470/0 (socket says 172.21.15.157:57470) 2023-11-06T03:19:57.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 -- 172.21.15.157:0/516979832 learned_addr learned my addr 172.21.15.157:0/516979832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73e4e7640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe738156a70 0x7fe73814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 -- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe738156a70 msgr2=0x7fe73814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe738156a70 0x7fe73814e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 -- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7380b2330 msgr2=0x7fe7380b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7380b2330 0x7fe7380b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 -- 172.21.15.157:0/516979832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73000bd40 con 0x7fe7380adae0 2023-11-06T03:19:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73dce6640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fe73400b440 tx=0x7fe73400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011890 con 0x7fe7380adae0 2023-11-06T03:19:57.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe734011a10 con 0x7fe7380adae0 2023-11-06T03:19:57.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011dc0 con 0x7fe7380adae0 2023-11-06T03:19:57.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73814f0d0 con 0x7fe7380adae0 2023-11-06T03:19:57.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.116+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73814f690 con 0x7fe7380adae0 2023-11-06T03:19:57.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73814fbb0 con 0x7fe7380adae0 2023-11-06T03:19:57.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe73401a400 con 0x7fe7380adae0 2023-11-06T03:19:57.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe72e7fc640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 0x7fe71c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe734083360 con 0x7fe7380adae0 2023-11-06T03:19:57.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe73d4e5640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 0x7fe71c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.120+0000 7fe73d4e5640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 0x7fe71c067f20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe728007fd0 tx=0x7fe7280079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.124+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fe73814fbb0 con 0x7fe7380adae0 2023-11-06T03:19:57.269 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe73806d270 con 0x7fe71c065a60 2023-11-06T03:19:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe72e7fc640 1 -- 172.21.15.157:0/516979832 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe7280050d0 con 0x7fe71c065a60 2023-11-06T03:19:57.270 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 0 to deep-scrub 2023-11-06T03:19:57.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 msgr2=0x7fe71c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 0x7fe71c067f20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe728007fd0 tx=0x7fe7280079a0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.273 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 msgr2=0x7fe73814fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fe73400b440 tx=0x7fe73400b900 comp rx=0 tx=0).stop 2023-11-06T03:19:57.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 shutdown_connections 2023-11-06T03:19:57.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fe71c065a60 0x7fe71c067f20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe738156a70 0x7fe73814e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.274 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7380b2330 0x7fe7380b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 --2- 172.21.15.157:0/516979832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7380adae0 0x7fe73814fe90 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 >> 172.21.15.157:0/516979832 conn(0x7fe738065930 msgr2=0x7fe738163120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 shutdown_connections 2023-11-06T03:19:57.275 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.276+0000 7fe73ff4a640 1 -- 172.21.15.157:0/516979832 wait complete. 2023-11-06T03:19:57.289 INFO:tasks.ceph:Scrubbing osd.1 2023-11-06T03:19:57.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:57.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 Processor -- start 2023-11-06T03:19:57.416 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 -- start start 2023-11-06T03:19:57.417 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732c155310 0x7f732c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c06c550 0x7f732c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0cb100 con 0x7f732c06c170 2023-11-06T03:19:57.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0ce0a0 con 0x7f732c155310 2023-11-06T03:19:57.418 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f7333fa0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c15bef0 con 0x7f732c06c550 2023-11-06T03:19:57.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f733153b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732c155310 0x7f732c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f733153b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f732c155310 0x7f732c15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57480/0 (socket says 172.21.15.157:57480) 2023-11-06T03:19:57.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.420+0000 7f733153b640 1 -- 172.21.15.157:0/4065489968 learned_addr learned my addr 172.21.15.157:0/4065489968 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733253d640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c06c550 0x7f732c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.419 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733153b640 1 -- 172.21.15.157:0/4065489968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c06c550 msgr2=0x7f732c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:57.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733153b640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c06c550 0x7f732c15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733153b640 1 -- 172.21.15.157:0/4065489968 >> v1:172.21.15.195:6789/0 conn(0x7f732c06c170 legacy=0x7f732c154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.420 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733153b640 1 -- 172.21.15.157:0/4065489968 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0cb350 con 0x7f732c155310 2023-11-06T03:19:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f733153b640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c155310 0x7f732c15b150 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f731c009870 tx=0x7f731c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489e5c670710bbdc server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7330d3a640 1 -- 172.21.15.157:0/4065489968 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0aa670 con 0x7f732c155310 2023-11-06T03:19:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7330d3a640 1 -- 172.21.15.157:0/4065489968 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c0aa7d0 con 0x7f732c155310 2023-11-06T03:19:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7330d3a640 1 -- 172.21.15.157:0/4065489968 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0aab60 con 0x7f732c155310 2023-11-06T03:19:57.422 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 -- 172.21.15.157:0/4065489968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c155310 msgr2=0x7f732c15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c155310 0x7f732c15b150 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f731c009870 tx=0x7f731c01c510 comp rx=0 tx=0).stop 2023-11-06T03:19:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 -- 172.21.15.157:0/4065489968 shutdown_connections 2023-11-06T03:19:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c06c550 0x7f732c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 --2- 172.21.15.157:0/4065489968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c155310 0x7f732c15b150 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.423 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 -- 172.21.15.157:0/4065489968 >> 172.21.15.157:0/4065489968 conn(0x7f732c0b43b0 msgr2=0x7f732c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 -- 172.21.15.157:0/4065489968 shutdown_connections 2023-11-06T03:19:57.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.424+0000 7f7333fa0640 1 -- 172.21.15.157:0/4065489968 wait complete. 2023-11-06T03:19:57.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 Processor -- start 2023-11-06T03:19:57.424 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- start start 2023-11-06T03:19:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 0x7f732c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f732c06c550 0x7f732c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c155310 0x7f732c151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c154160 con 0x7f732c06c170 2023-11-06T03:19:57.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c154440 con 0x7f732c06c550 2023-11-06T03:19:57.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c154720 con 0x7f732c155310 2023-11-06T03:19:57.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 0x7f732c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 0x7f732c14a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57496/0 (socket says 172.21.15.157:57496) 2023-11-06T03:19:57.428 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 -- 172.21.15.157:0/1803149215 learned_addr learned my addr 172.21.15.157:0/1803149215 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f733253d640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c155310 0x7f732c151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f733153b640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f732c06c550 0x7f732c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 -- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c155310 msgr2=0x7f732c151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f732c155310 0x7f732c151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.429 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 -- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f732c06c550 msgr2=0x7f732c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f732c06c550 0x7f732c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0094d0 con 0x7f732c06c170 2023-11-06T03:19:57.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7331d3c640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 0x7f732c14a010 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f732800c500 tx=0x7f732800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328005220 con 0x7f732c06c170 2023-11-06T03:19:57.430 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7328005380 con 0x7f732c06c170 2023-11-06T03:19:57.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0cbfb0 con 0x7f732c06c170 2023-11-06T03:19:57.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328007270 con 0x7f732c06c170 2023-11-06T03:19:57.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c154ce0 con 0x7f732c06c170 2023-11-06T03:19:57.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f732c0cbe30 con 0x7f732c06c170 2023-11-06T03:19:57.431 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.428+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73280073d0 con 0x7f732c06c170 2023-11-06T03:19:57.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7310065a60 0x7f7310067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7328082970 con 0x7f732c06c170 2023-11-06T03:19:57.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f731006bf80 0x7f731006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f731006ea50 con 0x7f731006bf80 2023-11-06T03:19:57.432 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73280872c0 con 0x7f732c06c170 2023-11-06T03:19:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f733153b640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7310065a60 0x7f7310067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f733253d640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f731006bf80 0x7f731006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f733253d640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f731006bf80 0x7f731006e3a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f733153b640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7310065a60 0x7f7310067f20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f731c0ae000 tx=0x7f731c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.433 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.432+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== osd.1 v2:172.21.15.157:6800/38221 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f731006ea50 con 0x7f731006bf80 2023-11-06T03:19:57.438 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 --> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f732c04e050 con 0x7f731006bf80 2023-11-06T03:19:57.439 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f73227fc640 1 -- 172.21.15.157:0/1803149215 <== osd.1 v2:172.21.15.157:6800/38221 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f732c04e050 con 0x7f731006bf80 2023-11-06T03:19:57.439 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:57.439 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:57.440 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:57.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f731006bf80 msgr2=0x7f731006e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:6800/38221,v1:172.21.15.157:6802/38221] conn(0x7f731006bf80 0x7f731006e3a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7310065a60 msgr2=0x7f7310067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7310065a60 0x7f7310067f20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f731c0ae000 tx=0x7f731c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:57.441 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 msgr2=0x7f732c14a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 --2- 172.21.15.157:0/1803149215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c06c170 0x7f732c14a010 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f732800c500 tx=0x7f732800c9c0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f733253d640 1 -- 172.21.15.157:0/1803149215 reap_dead start 2023-11-06T03:19:57.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 shutdown_connections 2023-11-06T03:19:57.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 >> 172.21.15.157:0/1803149215 conn(0x7f732c0b43b0 msgr2=0x7f732c065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.442 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 shutdown_connections 2023-11-06T03:19:57.443 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.444+0000 7f7333fa0640 1 -- 172.21.15.157:0/1803149215 wait complete. 2023-11-06T03:19:57.454 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-06T03:19:57.564 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.568+0000 7fd391db3640 1 Processor -- start 2023-11-06T03:19:57.565 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.568+0000 7fd391db3640 1 -- start start 2023-11-06T03:19:57.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd38c159930 0x7fd38c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.566 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c15a860 con 0x7fd38c0b3020 2023-11-06T03:19:57.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0711b0 con 0x7fd38c0b5320 2023-11-06T03:19:57.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c159d10 con 0x7fd38c159930 2023-11-06T03:19:57.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd38c159930 0x7fd38c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.567 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd38c159930 0x7fd38c0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57510/0 (socket says 172.21.15.157:57510) 2023-11-06T03:19:57.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd383fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fd38c0b3020 0x7fd38c0b4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36832/0 (socket says 172.21.15.157:36832) 2023-11-06T03:19:57.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 -- 172.21.15.157:0/3863457138 learned_addr learned my addr 172.21.15.157:0/3863457138 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 -- 172.21.15.157:0/3863457138 >> v1:172.21.15.195:6789/0 conn(0x7fd38c0b5320 legacy=0x7fd38c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:57.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 -- 172.21.15.157:0/3863457138 >> v1:172.21.15.157:6789/0 conn(0x7fd38c0b3020 legacy=0x7fd38c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.568 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 -- 172.21.15.157:0/3863457138 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c159f30 con 0x7fd38c159930 2023-11-06T03:19:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38b7fe640 1 --2- 172.21.15.157:0/3863457138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c0b2ae0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009870 tx=0x7fd37c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddae32d03870877 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38affd640 1 -- 172.21.15.157:0/3863457138 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0ac640 con 0x7fd38c159930 2023-11-06T03:19:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38affd640 1 -- 172.21.15.157:0/3863457138 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0ac7a0 con 0x7fd38c159930 2023-11-06T03:19:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd38affd640 1 -- 172.21.15.157:0/3863457138 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0acb30 con 0x7fd38c159930 2023-11-06T03:19:57.570 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- 172.21.15.157:0/3863457138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 msgr2=0x7fd38c0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 --2- 172.21.15.157:0/3863457138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c0b2ae0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009870 tx=0x7fd37c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- 172.21.15.157:0/3863457138 shutdown_connections 2023-11-06T03:19:57.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 --2- 172.21.15.157:0/3863457138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c0b2ae0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- 172.21.15.157:0/3863457138 >> 172.21.15.157:0/3863457138 conn(0x7fd38c069d70 msgr2=0x7fd38c06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.571 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- 172.21.15.157:0/3863457138 shutdown_connections 2023-11-06T03:19:57.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.572+0000 7fd391db3640 1 -- 172.21.15.157:0/3863457138 wait complete. 2023-11-06T03:19:57.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 Processor -- start 2023-11-06T03:19:57.572 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- start start 2023-11-06T03:19:57.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0b3020 0x7fd38c14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b5320 0x7fd38c164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.575 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.576 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1654d0 con 0x7fd38c159930 2023-11-06T03:19:57.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1f9350 con 0x7fd38c0b3020 2023-11-06T03:19:57.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1f9630 con 0x7fd38c0b5320 2023-11-06T03:19:57.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.577 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57512/0 (socket says 172.21.15.157:57512) 2023-11-06T03:19:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 -- 172.21.15.157:0/2697140024 learned_addr learned my addr 172.21.15.157:0/2697140024 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 -- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b5320 msgr2=0x7fd38c164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b5320 0x7fd38c164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.578 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 -- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0b3020 msgr2=0x7fd38c14a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0b3020 0x7fd38c14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 -- 172.21.15.157:0/2697140024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0094d0 con 0x7fd38c159930 2023-11-06T03:19:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd38bfff640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c780 tx=0x7fd38400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384004b70 con 0x7fd38c159930 2023-11-06T03:19:57.579 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38400ce90 con 0x7fd38c159930 2023-11-06T03:19:57.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c1f9910 con 0x7fd38c159930 2023-11-06T03:19:57.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38400a850 con 0x7fd38c159930 2023-11-06T03:19:57.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1651c0 con 0x7fd38c159930 2023-11-06T03:19:57.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c15acd0 con 0x7fd38c159930 2023-11-06T03:19:57.580 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd38400a9b0 con 0x7fd38c159930 2023-11-06T03:19:57.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 0x7fd36c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.576+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd3840831e0 con 0x7fd38c159930 2023-11-06T03:19:57.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.580+0000 7fd38b7fe640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 0x7fd36c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.581 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.580+0000 7fd38b7fe640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 0x7fd36c067f20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0ae000 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.582 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.580+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd38c15acd0 con 0x7fd38c159930 2023-11-06T03:19:57.720 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.724+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd38c1f9e30 con 0x7fd36c065a60 2023-11-06T03:19:57.721 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.724+0000 7fd388ff9640 1 -- 172.21.15.157:0/2697140024 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd38c1f9e30 con 0x7fd36c065a60 2023-11-06T03:19:57.721 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 1 to deep-scrub 2023-11-06T03:19:57.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 msgr2=0x7fd36c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 0x7fd36c067f20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0ae000 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2023-11-06T03:19:57.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 msgr2=0x7fd38c14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.723 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c780 tx=0x7fd38400cc40 comp rx=0 tx=0).stop 2023-11-06T03:19:57.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 shutdown_connections 2023-11-06T03:19:57.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd36c065a60 0x7fd36c067f20 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c159930 0x7fd38c14c110 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd38c0b5320 0x7fd38c164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.724 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 --2- 172.21.15.157:0/2697140024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd38c0b3020 0x7fd38c14a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 >> 172.21.15.157:0/2697140024 conn(0x7fd38c069d70 msgr2=0x7fd38c0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 shutdown_connections 2023-11-06T03:19:57.725 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.728+0000 7fd391db3640 1 -- 172.21.15.157:0/2697140024 wait complete. 2023-11-06T03:19:57.738 INFO:tasks.ceph:Scrubbing osd.2 2023-11-06T03:19:57.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:57.863 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 Processor -- start 2023-11-06T03:19:57.864 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 -- start start 2023-11-06T03:19:57.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd47015dda0 0x7fd4700b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.865 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd470070ee0 con 0x7fd47006b2f0 2023-11-06T03:19:57.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47006b780 con 0x7fd470159c80 2023-11-06T03:19:57.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd475bd9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700b2a90 con 0x7fd47015dda0 2023-11-06T03:19:57.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd46ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd47015dda0 0x7fd4700b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd46ffff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd47015dda0 0x7fd4700b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57518/0 (socket says 172.21.15.157:57518) 2023-11-06T03:19:57.866 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.868+0000 7fd46ffff640 1 -- 172.21.15.157:0/2426317202 learned_addr learned my addr 172.21.15.157:0/2426317202 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46effd640 1 -- 172.21.15.157:0/2426317202 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554651565 0 0) 0x7fd470070ee0 con 0x7fd47006b2f0 2023-11-06T03:19:57.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46effd640 1 -- 172.21.15.157:0/2426317202 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd45c003530 con 0x7fd47006b2f0 2023-11-06T03:19:57.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46ffff640 1 -- 172.21.15.157:0/2426317202 >> v1:172.21.15.195:6789/0 conn(0x7fd470159c80 legacy=0x7fd47015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:57.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46ffff640 1 -- 172.21.15.157:0/2426317202 >> v1:172.21.15.157:6789/0 conn(0x7fd47006b2f0 legacy=0x7fd4700b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.867 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46ffff640 1 -- 172.21.15.157:0/2426317202 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700b2d70 con 0x7fd47015dda0 2023-11-06T03:19:57.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46ffff640 1 --2- 172.21.15.157:0/2426317202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47015dda0 0x7fd4700b3570 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fd468009e50 tx=0x7fd46801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c4e66e02d7ac69 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46effd640 1 -- 172.21.15.157:0/2426317202 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680a56f0 con 0x7fd47015dda0 2023-11-06T03:19:57.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46effd640 1 -- 172.21.15.157:0/2426317202 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4680a5850 con 0x7fd47015dda0 2023-11-06T03:19:57.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd46effd640 1 -- 172.21.15.157:0/2426317202 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680a5be0 con 0x7fd47015dda0 2023-11-06T03:19:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 -- 172.21.15.157:0/2426317202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47015dda0 msgr2=0x7fd4700b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 --2- 172.21.15.157:0/2426317202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47015dda0 0x7fd4700b3570 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fd468009e50 tx=0x7fd46801c9a0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 -- 172.21.15.157:0/2426317202 shutdown_connections 2023-11-06T03:19:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 --2- 172.21.15.157:0/2426317202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd47015dda0 0x7fd4700b3570 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 -- 172.21.15.157:0/2426317202 >> 172.21.15.157:0/2426317202 conn(0x7fd4700b43b0 msgr2=0x7fd470065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 -- 172.21.15.157:0/2426317202 shutdown_connections 2023-11-06T03:19:57.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.872+0000 7fd475bd9640 1 -- 172.21.15.157:0/2426317202 wait complete. 2023-11-06T03:19:57.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 Processor -- start 2023-11-06T03:19:57.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- start start 2023-11-06T03:19:57.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 0x7fd47014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd47015dda0 0x7fd4700aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4701519f0 0x7fd470137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470151f00 con 0x7fd470159c80 2023-11-06T03:19:57.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701521e0 con 0x7fd4701519f0 2023-11-06T03:19:57.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701524c0 con 0x7fd47015dda0 2023-11-06T03:19:57.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 0x7fd47014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 0x7fd47014fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57530/0 (socket says 172.21.15.157:57530) 2023-11-06T03:19:57.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 -- 172.21.15.157:0/1258612266 learned_addr learned my addr 172.21.15.157:0/1258612266 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:57.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd467fff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd47015dda0 0x7fd4700aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46ffff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4701519f0 0x7fd470137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 -- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd47015dda0 msgr2=0x7fd4700aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd47015dda0 0x7fd4700aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 -- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4701519f0 msgr2=0x7fd470137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4701519f0 0x7fd470137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009b30 con 0x7fd470159c80 2023-11-06T03:19:57.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46f7fe640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 0x7fd47014fc00 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c450 tx=0x7fd46000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460012830 con 0x7fd470159c80 2023-11-06T03:19:57.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4600129b0 con 0x7fd470159c80 2023-11-06T03:19:57.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4701527a0 con 0x7fd470159c80 2023-11-06T03:19:57.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460012d60 con 0x7fd470159c80 2023-11-06T03:19:57.879 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd470150140 con 0x7fd470159c80 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd47006ba60 con 0x7fd470159c80 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.876+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd46001c460 con 0x7fd470159c80 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a60 0x7fd450067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd460082ec0 con 0x7fd470159c80 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fd45006bf80 0x7fd45006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:57.880 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd45006ea50 con 0x7fd45006bf80 2023-11-06T03:19:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd460085580 con 0x7fd470159c80 2023-11-06T03:19:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd467fff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a60 0x7fd450067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46ffff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fd45006bf80 0x7fd45006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:57.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46ffff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fd45006bf80 0x7fd45006e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd467fff640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a60 0x7fd450067f20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd470067cc0 tx=0x7fd458009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:57.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.880+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== osd.2 v2:172.21.15.157:6816/38222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd45006ea50 con 0x7fd45006bf80 2023-11-06T03:19:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 --> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd47004e050 con 0x7fd45006bf80 2023-11-06T03:19:57.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd46cff9640 1 -- 172.21.15.157:0/1258612266 <== osd.2 v2:172.21.15.157:6816/38222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd47004e050 con 0x7fd45006bf80 2023-11-06T03:19:57.888 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:57.888 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:57.889 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:57.889 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fd45006bf80 msgr2=0x7fd45006e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:6816/38222,v1:172.21.15.157:6817/38222] conn(0x7fd45006bf80 0x7fd45006e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a60 msgr2=0x7fd450067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd450065a60 0x7fd450067f20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd470067cc0 tx=0x7fd458009040 comp rx=0 tx=0).stop 2023-11-06T03:19:57.890 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 msgr2=0x7fd47014fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 --2- 172.21.15.157:0/1258612266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd470159c80 0x7fd47014fc00 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c450 tx=0x7fd46000c910 comp rx=0 tx=0).stop 2023-11-06T03:19:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd46ffff640 1 -- 172.21.15.157:0/1258612266 reap_dead start 2023-11-06T03:19:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 shutdown_connections 2023-11-06T03:19:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 >> 172.21.15.157:0/1258612266 conn(0x7fd4700b43b0 msgr2=0x7fd470154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:57.891 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 shutdown_connections 2023-11-06T03:19:57.892 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:57.892+0000 7fd475bd9640 1 -- 172.21.15.157:0/1258612266 wait complete. 2023-11-06T03:19:57.904 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-06T03:19:58.011 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 Processor -- start 2023-11-06T03:19:58.012 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 -- start start 2023-11-06T03:19:58.013 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 0x7f7d2c0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 0x7f7d2c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0d2920 con 0x7f7d2c15ce80 2023-11-06T03:19:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0d18a0 con 0x7f7d2c0d0280 2023-11-06T03:19:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d32242640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0d1b60 con 0x7f7d2c15caa0 2023-11-06T03:19:58.014 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d2b7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 0x7f7d2c0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.016+0000 7f7d2bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 0x7f7d2c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2affd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f7d2c15ce80 0x7f7d2c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.157:52140/0 (socket says 172.21.15.157:52140) 2023-11-06T03:19:58.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2affd640 1 -- 172.21.15.157:0/1482904265 learned_addr learned my addr 172.21.15.157:0/1482904265 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:58.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2b7fe640 1 -- 172.21.15.157:0/1482904265 >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 msgr2=0x7f7d2c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.015 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2b7fe640 1 --2- 172.21.15.157:0/1482904265 >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 0x7f7d2c0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2b7fe640 1 -- 172.21.15.157:0/1482904265 >> v1:172.21.15.195:6789/0 conn(0x7f7d2c15ce80 legacy=0x7f7d2c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2b7fe640 1 -- 172.21.15.157:0/1482904265 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0d1e40 con 0x7f7d2c15caa0 2023-11-06T03:19:58.016 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2bfff640 1 --2- 172.21.15.157:0/1482904265 >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 0x7f7d2c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-06T03:19:58.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2b7fe640 1 --2- 172.21.15.157:0/1482904265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 0x7f7d2c0cdc60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0097c0 tx=0x7f7d1c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a25fd54bfb02a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.017 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2a7fc640 1 -- 172.21.15.157:0/1482904265 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0ab650 con 0x7f7d2c15caa0 2023-11-06T03:19:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2a7fc640 1 -- 172.21.15.157:0/1482904265 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0ab7b0 con 0x7f7d2c15caa0 2023-11-06T03:19:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d2a7fc640 1 -- 172.21.15.157:0/1482904265 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0abb40 con 0x7f7d2c15caa0 2023-11-06T03:19:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 -- 172.21.15.157:0/1482904265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 msgr2=0x7f7d2c0cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 --2- 172.21.15.157:0/1482904265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 0x7f7d2c0cdc60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0097c0 tx=0x7f7d1c01c450 comp rx=0 tx=0).stop 2023-11-06T03:19:58.018 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 -- 172.21.15.157:0/1482904265 shutdown_connections 2023-11-06T03:19:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 --2- 172.21.15.157:0/1482904265 >> v2:172.21.15.195:3300/0 conn(0x7f7d2c0d0280 0x7f7d2c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 --2- 172.21.15.157:0/1482904265 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c15caa0 0x7f7d2c0cdc60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 -- 172.21.15.157:0/1482904265 >> 172.21.15.157:0/1482904265 conn(0x7f7d2c070790 msgr2=0x7f7d2c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.019 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.020+0000 7f7d32242640 1 -- 172.21.15.157:0/1482904265 shutdown_connections 2023-11-06T03:19:58.020 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- 172.21.15.157:0/1482904265 wait complete. 2023-11-06T03:19:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 Processor -- start 2023-11-06T03:19:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- start start 2023-11-06T03:19:58.022 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c0d0280 0x7f7d2c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d2c15ce80 0x7f7d2c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0cb6c0 con 0x7f7d2c15caa0 2023-11-06T03:19:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1518f0 con 0x7f7d2c15ce80 2023-11-06T03:19:58.023 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c15e5f0 con 0x7f7d2c0d0280 2023-11-06T03:19:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57546/0 (socket says 172.21.15.157:57546) 2023-11-06T03:19:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 -- 172.21.15.157:0/775755757 learned_addr learned my addr 172.21.15.157:0/775755757 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.024 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2bfff640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d2c15ce80 0x7f7d2c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2b7fe640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c0d0280 0x7f7d2c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 -- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c0d0280 msgr2=0x7f7d2c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c0d0280 0x7f7d2c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 -- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d2c15ce80 msgr2=0x7f7d2c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.025 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d2c15ce80 0x7f7d2c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 -- 172.21.15.157:0/775755757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d24009a70 con 0x7f7d2c15caa0 2023-11-06T03:19:58.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d2affd640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c500 tx=0x7f7d2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20005cf0 con 0x7f7d2c15caa0 2023-11-06T03:19:58.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20005e50 con 0x7f7d2c15caa0 2023-11-06T03:19:58.026 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c009420 con 0x7f7d2c15caa0 2023-11-06T03:19:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200072f0 con 0x7f7d2c15caa0 2023-11-06T03:19:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c15eb70 con 0x7f7d2c15caa0 2023-11-06T03:19:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.024+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c072750 con 0x7f7d2c15caa0 2023-11-06T03:19:58.027 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.028+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7d20007450 con 0x7f7d2c15caa0 2023-11-06T03:19:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.028+0000 7f7d13fff640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 0x7f7d18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.028+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7d20082480 con 0x7f7d2c15caa0 2023-11-06T03:19:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.028+0000 7f7d2b7fe640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 0x7f7d18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.028+0000 7f7d2b7fe640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 0x7f7d18067f20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0af000 tx=0x7f7d1c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.029 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.032+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f7d200500c0 con 0x7f7d2c15caa0 2023-11-06T03:19:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.184+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d2c0b1ea0 con 0x7f7d18065a60 2023-11-06T03:19:58.181 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d13fff640 1 -- 172.21.15.157:0/775755757 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7d2c0b1ea0 con 0x7f7d18065a60 2023-11-06T03:19:58.182 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 2 to deep-scrub 2023-11-06T03:19:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 msgr2=0x7f7d18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 0x7f7d18067f20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0af000 tx=0x7f7d1c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 msgr2=0x7f7d2c0c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c500 tx=0x7f7d2000c9c0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 shutdown_connections 2023-11-06T03:19:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7d18065a60 0x7f7d18067f20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d2c15ce80 0x7f7d2c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d2c15caa0 0x7f7d2c0c6750 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 --2- 172.21.15.157:0/775755757 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d2c0d0280 0x7f7d2c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.185 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 >> 172.21.15.157:0/775755757 conn(0x7f7d2c070790 msgr2=0x7f7d2c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 shutdown_connections 2023-11-06T03:19:58.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.188+0000 7f7d32242640 1 -- 172.21.15.157:0/775755757 wait complete. 2023-11-06T03:19:58.200 INFO:tasks.ceph:Scrubbing osd.3 2023-11-06T03:19:58.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:58.327 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 Processor -- start 2023-11-06T03:19:58.330 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 -- start start 2023-11-06T03:19:58.331 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f158006b2f0 0x7f15800b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f158015dda0 0x7f158015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1580070ee0 con 0x7f1580159c80 2023-11-06T03:19:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158015b060 con 0x7f158015dda0 2023-11-06T03:19:58.332 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f15864da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158006b780 con 0x7f158006b2f0 2023-11-06T03:19:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f158015dda0 0x7f158015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f158015dda0 0x7f158015ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57552/0 (socket says 172.21.15.157:57552) 2023-11-06T03:19:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 -- 172.21.15.157:0/3149796406 learned_addr learned my addr 172.21.15.157:0/3149796406 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f157f7fe640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f158006b2f0 0x7f15800b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.333 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 -- 172.21.15.157:0/3149796406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f158006b2f0 msgr2=0x7f15800b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f158006b2f0 0x7f15800b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 -- 172.21.15.157:0/3149796406 >> v1:172.21.15.195:6789/0 conn(0x7f1580159c80 legacy=0x7f158015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.332+0000 7f1584a77640 1 -- 172.21.15.157:0/3149796406 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800b2a90 con 0x7f158015dda0 2023-11-06T03:19:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f1584a77640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158015dda0 0x7f158015ab20 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f1578009e50 tx=0x7f157801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4152d0c242c4e7ae server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.334 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f157effd640 1 -- 172.21.15.157:0/3149796406 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780a5790 con 0x7f158015dda0 2023-11-06T03:19:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f157effd640 1 -- 172.21.15.157:0/3149796406 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15780a58f0 con 0x7f158015dda0 2023-11-06T03:19:58.335 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f157effd640 1 -- 172.21.15.157:0/3149796406 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780a5c80 con 0x7f158015dda0 2023-11-06T03:19:58.338 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 -- 172.21.15.157:0/3149796406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158015dda0 msgr2=0x7f158015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158015dda0 0x7f158015ab20 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f1578009e50 tx=0x7f157801c910 comp rx=0 tx=0).stop 2023-11-06T03:19:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 -- 172.21.15.157:0/3149796406 shutdown_connections 2023-11-06T03:19:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f158015dda0 0x7f158015ab20 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 --2- 172.21.15.157:0/3149796406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f158006b2f0 0x7f15800b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.339 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 -- 172.21.15.157:0/3149796406 >> 172.21.15.157:0/3149796406 conn(0x7f15800b43b0 msgr2=0x7f1580065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 -- 172.21.15.157:0/3149796406 shutdown_connections 2023-11-06T03:19:58.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.336+0000 7f15864da640 1 -- 172.21.15.157:0/3149796406 wait complete. 2023-11-06T03:19:58.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 Processor -- start 2023-11-06T03:19:58.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- start start 2023-11-06T03:19:58.340 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1580159c80 0x7f1580153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158015dda0 0x7f1580153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 0x7f15800ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801647f0 con 0x7f1580153300 2023-11-06T03:19:58.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801596c0 con 0x7f158015dda0 2023-11-06T03:19:58.341 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158015c4b0 con 0x7f1580159c80 2023-11-06T03:19:58.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 0x7f15800ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 0x7f15800ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57564/0 (socket says 172.21.15.157:57564) 2023-11-06T03:19:58.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 -- 172.21.15.157:0/4160903594 learned_addr learned my addr 172.21.15.157:0/4160903594 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157ffff640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1580159c80 0x7f1580153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.342 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157f7fe640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158015dda0 0x7f1580153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 -- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1580159c80 msgr2=0x7f1580153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1580159c80 0x7f1580153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 -- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158015dda0 msgr2=0x7f1580153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f158015dda0 0x7f1580153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578009b30 con 0x7f1580153300 2023-11-06T03:19:58.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f1584a77640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 0x7f15800ac8b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f15780af040 tx=0x7f15780196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780b8670 con 0x7f1580153300 2023-11-06T03:19:58.344 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15780b87d0 con 0x7f1580153300 2023-11-06T03:19:58.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1580164230 con 0x7f1580153300 2023-11-06T03:19:58.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780b8b60 con 0x7f1580153300 2023-11-06T03:19:58.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158015c750 con 0x7f1580153300 2023-11-06T03:19:58.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f158006ba60 con 0x7f1580153300 2023-11-06T03:19:58.345 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.340+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f15780b8cc0 con 0x7f1580153300 2023-11-06T03:19:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f156c065a60 0x7f156c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f157811e980 con 0x7f1580153300 2023-11-06T03:19:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7f156c06bf80 0x7f156c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f156c06ea50 con 0x7f156c06bf80 2023-11-06T03:19:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157811edd0 con 0x7f1580153300 2023-11-06T03:19:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157ffff640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f156c065a60 0x7f156c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157f7fe640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7f156c06bf80 0x7f156c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157f7fe640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7f156c06bf80 0x7f156c06e3a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157ffff640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f156c065a60 0x7f156c067f20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1570007fd0 tx=0x7f1570007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.344+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== osd.3 v2:172.21.15.157:6824/38223 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f156c06ea50 con 0x7f156c06bf80 2023-11-06T03:19:58.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 --> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f158004e050 con 0x7f156c06bf80 2023-11-06T03:19:58.351 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f157cff9640 1 -- 172.21.15.157:0/4160903594 <== osd.3 v2:172.21.15.157:6824/38223 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f158004e050 con 0x7f156c06bf80 2023-11-06T03:19:58.352 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:58.352 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:58.352 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:58.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7f156c06bf80 msgr2=0x7f156c06e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:6824/38223,v1:172.21.15.157:6825/38223] conn(0x7f156c06bf80 0x7f156c06e3a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f156c065a60 msgr2=0x7f156c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f156c065a60 0x7f156c067f20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1570007fd0 tx=0x7f1570007e40 comp rx=0 tx=0).stop 2023-11-06T03:19:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 msgr2=0x7f15800ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 --2- 172.21.15.157:0/4160903594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1580153300 0x7f15800ac8b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f15780af040 tx=0x7f15780196b0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f1584a77640 1 -- 172.21.15.157:0/4160903594 reap_dead start 2023-11-06T03:19:58.354 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 shutdown_connections 2023-11-06T03:19:58.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 >> 172.21.15.157:0/4160903594 conn(0x7f15800b43b0 msgr2=0x7f158006fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 shutdown_connections 2023-11-06T03:19:58.355 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.356+0000 7f15864da640 1 -- 172.21.15.157:0/4160903594 wait complete. 2023-11-06T03:19:58.367 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-06T03:19:58.473 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.476+0000 7f7398f08640 1 Processor -- start 2023-11-06T03:19:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.476+0000 7f7398f08640 1 -- start start 2023-11-06T03:19:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7398f08640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f73940cced0 0x7f739406c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7398f08640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739406bfc0 con 0x7f739415d3a0 2023-11-06T03:19:58.474 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7398f08640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73940ce990 con 0x7f73940cca50 2023-11-06T03:19:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7398f08640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739406b970 con 0x7f73940cced0 2023-11-06T03:19:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739259c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f739415d3a0 0x7f739415d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:36836/0 (socket says 172.21.15.157:36836) 2023-11-06T03:19:58.475 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739259c640 1 -- 172.21.15.157:0/2059234667 learned_addr learned my addr 172.21.15.157:0/2059234667 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7392d9d640 1 --2- 172.21.15.157:0/2059234667 >> v2:172.21.15.195:3300/0 conn(0x7f73940cced0 0x7f739406c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.477 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120618821 0 0) 0x7f739406bfc0 con 0x7f739415d3a0 2023-11-06T03:19:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f737c003530 con 0x7f739415d3a0 2023-11-06T03:19:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165231082 0 0) 0x7f737c003530 con 0x7f739415d3a0 2023-11-06T03:19:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 >> v2:172.21.15.195:3300/0 conn(0x7f73940cced0 msgr2=0x7f739406c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 --2- 172.21.15.157:0/2059234667 >> v2:172.21.15.195:3300/0 conn(0x7f73940cced0 0x7f739406c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.478 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 >> v1:172.21.15.195:6789/0 conn(0x7f73940cca50 legacy=0x7f73940c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739406bc50 con 0x7f739415d3a0 2023-11-06T03:19:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f7384001680 con 0x7f739415d3a0 2023-11-06T03:19:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 >> v1:172.21.15.157:6789/0 conn(0x7f739415d3a0 legacy=0x7f739415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c003980 0x7f737c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.479 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 0x7f737c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c005190 0x7f737c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739406b970 con 0x7f737c0047d0 2023-11-06T03:19:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c030 con 0x7f737c003980 2023-11-06T03:19:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c1f0 con 0x7f737c005190 2023-11-06T03:19:58.480 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 0x7f737c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f739259c640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c003980 0x7f737c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7392d9d640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c005190 0x7f737c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 -- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c005190 msgr2=0x7f737c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f737c005190 0x7f737c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.481 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 -- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c003980 msgr2=0x7f737c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c003980 0x7f737c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 -- 172.21.15.157:0/2059234667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c003530 con 0x7f737c0047d0 2023-11-06T03:19:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7392d9d640 1 -- 172.21.15.157:0/2059234667 reap_dead start 2023-11-06T03:19:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.480+0000 7f7391d9b640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 0x7f737c004c50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7388001780 tx=0x7f738801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b878ec38b3d83de4 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.482 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880a4ea0 con 0x7f737c0047d0 2023-11-06T03:19:58.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73880a7050 con 0x7f737c0047d0 2023-11-06T03:19:58.483 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f739159a640 1 -- 172.21.15.157:0/2059234667 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388003600 con 0x7f737c0047d0 2023-11-06T03:19:58.487 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 msgr2=0x7f737c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 0x7f737c004c50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7388001780 tx=0x7f738801c3e0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/2059234667 shutdown_connections 2023-11-06T03:19:58.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 --2- 172.21.15.157:0/2059234667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f737c0047d0 0x7f737c004c50 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.488 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/2059234667 >> 172.21.15.157:0/2059234667 conn(0x7f73940cb990 msgr2=0x7f73940cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/2059234667 shutdown_connections 2023-11-06T03:19:58.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/2059234667 wait complete. 2023-11-06T03:19:58.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 Processor -- start 2023-11-06T03:19:58.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- start start 2023-11-06T03:19:58.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0047d0 0x7f739415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.490 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 0x7f73940709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73940c8f80 con 0x7f739415bf20 2023-11-06T03:19:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739415bd10 con 0x7f737c0047d0 2023-11-06T03:19:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73940c86e0 con 0x7f73940c81d0 2023-11-06T03:19:58.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:57576/0 (socket says 172.21.15.157:57576) 2023-11-06T03:19:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 -- 172.21.15.157:0/1646907349 learned_addr learned my addr 172.21.15.157:0/1646907349 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7391d9b640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 0x7f73940709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f739259c640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0047d0 0x7f739415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 -- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 msgr2=0x7f73940709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 0x7f73940709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 -- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0047d0 msgr2=0x7f739415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0047d0 0x7f739415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 -- 172.21.15.157:0/1646907349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73940c89c0 con 0x7f739415bf20 2023-11-06T03:19:58.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7391d9b640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 0x7f73940709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7392d9d640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f739406bf30 tx=0x7f738c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00e070 con 0x7f739415bf20 2023-11-06T03:19:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c011e10 con 0x7f739415bf20 2023-11-06T03:19:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73940c8ca0 con 0x7f739415bf20 2023-11-06T03:19:58.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c0039a0 con 0x7f739415bf20 2023-11-06T03:19:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.484+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73941e6b70 con 0x7f739415bf20 2023-11-06T03:19:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7394072750 con 0x7f739415bf20 2023-11-06T03:19:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f738c003b00 con 0x7f739415bf20 2023-11-06T03:19:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f7382ffd640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 0x7f7374067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f738c0836b0 con 0x7f739415bf20 2023-11-06T03:19:58.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f739259c640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 0x7f7374067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.488+0000 7f739259c640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 0x7f7374067f20 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7384001ec0 tx=0x7f7384009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.492+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f738c0512f0 con 0x7f739415bf20 2023-11-06T03:19:58.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.636+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f73940cec70 con 0x7f7374065a60 2023-11-06T03:19:58.632 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.636+0000 7f7382ffd640 1 -- 172.21.15.157:0/1646907349 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7384000ee0 con 0x7f7374065a60 2023-11-06T03:19:58.633 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 3 to deep-scrub 2023-11-06T03:19:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 msgr2=0x7f7374067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 0x7f7374067f20 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7384001ec0 tx=0x7f7384009040 comp rx=0 tx=0).stop 2023-11-06T03:19:58.635 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 msgr2=0x7f7394070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f739406bf30 tx=0x7f738c0115a0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 shutdown_connections 2023-11-06T03:19:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7374065a60 0x7f7374067f20 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.636 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f739415bf20 0x7f7394070f30 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f73940c81d0 0x7f73940709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 --2- 172.21.15.157:0/1646907349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f737c0047d0 0x7f739415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 >> 172.21.15.157:0/1646907349 conn(0x7f73940cb990 msgr2=0x7f73940ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.637 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 shutdown_connections 2023-11-06T03:19:58.638 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.640+0000 7f7398f08640 1 -- 172.21.15.157:0/1646907349 wait complete. 2023-11-06T03:19:58.650 INFO:tasks.ceph:Scrubbing osd.4 2023-11-06T03:19:58.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:58.771 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 Processor -- start 2023-11-06T03:19:58.772 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 -- start start 2023-11-06T03:19:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f66d815e760 0x7f66d815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 0x7f66d815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d0bd0 con 0x7f66d80afee0 2023-11-06T03:19:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80cd8f0 con 0x7f66d815e760 2023-11-06T03:19:58.774 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66de80b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d815f110 con 0x7f66d815a5d0 2023-11-06T03:19:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d7fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f66d80afee0 0x7f66d80cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51108/0 (socket says 172.21.15.157:51108) 2023-11-06T03:19:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d7fff640 1 -- 172.21.15.157:0/1472803805 learned_addr learned my addr 172.21.15.157:0/1472803805 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66dcda8640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 0x7f66d815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d77fe640 1 --2- 172.21.15.157:0/1472803805 >> v2:172.21.15.195:3300/0 conn(0x7f66d815e760 0x7f66d815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.775 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148665133 0 0) 0x7f66d80d0bd0 con 0x7f66d80afee0 2023-11-06T03:19:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c0003530 con 0x7f66d80afee0 2023-11-06T03:19:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927580612 0 0) 0x7f66c0003530 con 0x7f66d80afee0 2023-11-06T03:19:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 msgr2=0x7f66d815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 0x7f66d815a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.776 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 >> v2:172.21.15.195:3300/0 conn(0x7f66d815e760 msgr2=0x7f66d815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.776+0000 7f66d6ffd640 1 --2- 172.21.15.157:0/1472803805 >> v2:172.21.15.195:3300/0 conn(0x7f66d815e760 0x7f66d815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80ccc70 con 0x7f66d80afee0 2023-11-06T03:19:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4067347647 0 0) 0x7f66c8001680 con 0x7f66d80afee0 2023-11-06T03:19:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 >> v1:172.21.15.157:6789/0 conn(0x7f66d80afee0 legacy=0x7f66d80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.777 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c0003980 0x7f66c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 0x7f66c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0004d10 0x7f66c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80cd8f0 con 0x7f66c0003980 2023-11-06T03:19:58.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d815f110 con 0x7f66c0004d10 2023-11-06T03:19:58.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c000b990 con 0x7f66d815a5d0 2023-11-06T03:19:58.779 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c0003980 0x7f66c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 -- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 msgr2=0x7f66c00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d815a5d0 0x7f66c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 -- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0004d10 msgr2=0x7f66c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0004d10 0x7f66c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.781 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 -- 172.21.15.157:0/1472803805 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c0003530 con 0x7f66c0003980 2023-11-06T03:19:58.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d7fff640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c0003980 0x7f66c0003da0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f66c800c290 tx=0x7f66c801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2191d0f0d2121cc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c80058e0 con 0x7f66c0003980 2023-11-06T03:19:58.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c8005a40 con 0x7f66c0003980 2023-11-06T03:19:58.785 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66d6ffd640 1 -- 172.21.15.157:0/1472803805 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8005df0 con 0x7f66c0003980 2023-11-06T03:19:58.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 -- 172.21.15.157:0/1472803805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c0003980 msgr2=0x7f66c0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 --2- 172.21.15.157:0/1472803805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66c0003980 0x7f66c0003da0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f66c800c290 tx=0x7f66c801cce0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66dcda8640 1 -- 172.21.15.157:0/1472803805 reap_dead start 2023-11-06T03:19:58.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 -- 172.21.15.157:0/1472803805 shutdown_connections 2023-11-06T03:19:58.786 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 -- 172.21.15.157:0/1472803805 >> 172.21.15.157:0/1472803805 conn(0x7f66d8065d30 msgr2=0x7f66d8066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 -- 172.21.15.157:0/1472803805 shutdown_connections 2023-11-06T03:19:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.780+0000 7f66de80b640 1 -- 172.21.15.157:0/1472803805 wait complete. 2023-11-06T03:19:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 Processor -- start 2023-11-06T03:19:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- start start 2023-11-06T03:19:58.787 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0003980 0x7f66d8147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 0x7f66d80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d80c98f0 0x7f66d80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81f74e0 con 0x7f66d80b03e0 2023-11-06T03:19:58.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80b0da0 con 0x7f66c0003980 2023-11-06T03:19:58.788 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81482c0 con 0x7f66d80c98f0 2023-11-06T03:19:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 0x7f66d80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 0x7f66d80b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34840/0 (socket says 172.21.15.157:34840) 2023-11-06T03:19:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 -- 172.21.15.157:0/2781930517 learned_addr learned my addr 172.21.15.157:0/2781930517 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66dcda8640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d80c98f0 0x7f66d80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.789 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 -- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d80c98f0 msgr2=0x7f66d80c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d80c98f0 0x7f66d80c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d7fff640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0003980 0x7f66d8147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 -- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0003980 msgr2=0x7f66d8147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66c0003980 0x7f66d8147d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.790 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d81f6b40 con 0x7f66d80b03e0 2023-11-06T03:19:58.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66dcda8640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f66d80c98f0 0x7f66d80c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:58.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d77fe640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 0x7f66d80b0860 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e7c0 tx=0x7f66cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc00eed0 con 0x7f66d80b03e0 2023-11-06T03:19:58.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc00c940 con 0x7f66d80b03e0 2023-11-06T03:19:58.791 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d81f6e20 con 0x7f66d80b03e0 2023-11-06T03:19:58.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc00ccf0 con 0x7f66d80b03e0 2023-11-06T03:19:58.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80ca2b0 con 0x7f66d80b03e0 2023-11-06T03:19:58.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66d80cdd10 con 0x7f66d80b03e0 2023-11-06T03:19:58.792 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.784+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f66cc013470 con 0x7f66d80b03e0 2023-11-06T03:19:58.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66c4065a60 0x7f66c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f66cc0833e0 con 0x7f66d80b03e0 2023-11-06T03:19:58.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f66c406be10 0x7f66c406e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.793 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66c406e8e0 con 0x7f66c406be10 2023-11-06T03:19:58.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66cc083830 con 0x7f66d80b03e0 2023-11-06T03:19:58.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d7fff640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66c4065a60 0x7f66c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66dcda8640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f66c406be10 0x7f66c406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66dcda8640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f66c406be10 0x7f66c406e230 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.794 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d7fff640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66c4065a60 0x7f66c4067f20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f66c80aa000 tx=0x7f66c8000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.795 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.788+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== osd.4 v2:172.21.15.195:6808/38425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f66c406e8e0 con 0x7f66c406be10 2023-11-06T03:19:58.796 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.800+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 --> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f66d80cd510 con 0x7f66c406be10 2023-11-06T03:19:58.797 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66d4ff9640 1 -- 172.21.15.157:0/2781930517 <== osd.4 v2:172.21.15.195:6808/38425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f66d0011040 con 0x7f66c406be10 2023-11-06T03:19:58.798 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:58.798 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:58.798 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:58.799 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f66c406be10 msgr2=0x7f66c406e230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6808/38425,v1:172.21.15.195:6809/38425] conn(0x7f66c406be10 0x7f66c406e230 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66c4065a60 msgr2=0x7f66c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f66c4065a60 0x7f66c4067f20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f66c80aa000 tx=0x7f66c8000ff0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 msgr2=0x7f66d80b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.800 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 --2- 172.21.15.157:0/2781930517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66d80b03e0 0x7f66d80b0860 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e7c0 tx=0x7f66cc00ec80 comp rx=0 tx=0).stop 2023-11-06T03:19:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66dcda8640 1 -- 172.21.15.157:0/2781930517 reap_dead start 2023-11-06T03:19:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 shutdown_connections 2023-11-06T03:19:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 >> 172.21.15.157:0/2781930517 conn(0x7f66d8065d30 msgr2=0x7f66d80b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 shutdown_connections 2023-11-06T03:19:58.801 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.804+0000 7f66de80b640 1 -- 172.21.15.157:0/2781930517 wait complete. 2023-11-06T03:19:58.813 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-06T03:19:58.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 Processor -- start 2023-11-06T03:19:58.927 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 -- start start 2023-11-06T03:19:58.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ec41599c0 0x7f8ec40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec40b2af0 0x7f8ec4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec415e440 con 0x7f8ec406d000 2023-11-06T03:19:58.929 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec406c240 con 0x7f8ec41599c0 2023-11-06T03:19:58.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec9231640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec406c520 con 0x7f8ec40b2af0 2023-11-06T03:19:58.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ec41599c0 0x7f8ec40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ec41599c0 0x7f8ec40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34850/0 (socket says 172.21.15.157:34850) 2023-11-06T03:19:58.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 -- 172.21.15.157:0/2626960811 learned_addr learned my addr 172.21.15.157:0/2626960811 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.930 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec359e640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec40b2af0 0x7f8ec4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 -- 172.21.15.157:0/2626960811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec40b2af0 msgr2=0x7f8ec4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:58.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec40b2af0 0x7f8ec4159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 -- 172.21.15.157:0/2626960811 >> v1:172.21.15.157:6789/0 conn(0x7f8ec406d000 legacy=0x7f8ec40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec1d9b640 1 -- 172.21.15.157:0/2626960811 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804864023 0 0) 0x7f8ec415e440 con 0x7f8ec406d000 2023-11-06T03:19:58.931 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.932+0000 7f8ec2d9d640 1 -- 172.21.15.157:0/2626960811 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec406c800 con 0x7f8ec41599c0 2023-11-06T03:19:58.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec41599c0 0x7f8ec40ae000 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009870 tx=0x7f8eb401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d241d94c795e6be server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec1d9b640 1 -- 172.21.15.157:0/2626960811 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40a95f0 con 0x7f8ec41599c0 2023-11-06T03:19:58.933 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec1d9b640 1 -- 172.21.15.157:0/2626960811 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb40a9750 con 0x7f8ec41599c0 2023-11-06T03:19:58.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec1d9b640 1 -- 172.21.15.157:0/2626960811 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40a9ae0 con 0x7f8ec41599c0 2023-11-06T03:19:58.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- 172.21.15.157:0/2626960811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec41599c0 msgr2=0x7f8ec40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec41599c0 0x7f8ec40ae000 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009870 tx=0x7f8eb401c510 comp rx=0 tx=0).stop 2023-11-06T03:19:58.934 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- 172.21.15.157:0/2626960811 shutdown_connections 2023-11-06T03:19:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec40b2af0 0x7f8ec4159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 --2- 172.21.15.157:0/2626960811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec41599c0 0x7f8ec40ae000 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- 172.21.15.157:0/2626960811 >> 172.21.15.157:0/2626960811 conn(0x7f8ec4065930 msgr2=0x7f8ec4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- 172.21.15.157:0/2626960811 shutdown_connections 2023-11-06T03:19:58.935 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- 172.21.15.157:0/2626960811 wait complete. 2023-11-06T03:19:58.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 Processor -- start 2023-11-06T03:19:58.936 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.936+0000 7f8ec9231640 1 -- start start 2023-11-06T03:19:58.938 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 0x7f8ec41507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec41599c0 0x7f8ec40aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.939 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40ce230 con 0x7f8ec40b2af0 2023-11-06T03:19:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4152cc0 con 0x7f8ec41599c0 2023-11-06T03:19:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4152fa0 con 0x7f8ec406d000 2023-11-06T03:19:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34856/0 (socket says 172.21.15.157:34856) 2023-11-06T03:19:58.940 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 -- 172.21.15.157:0/1026862571 learned_addr learned my addr 172.21.15.157:0/1026862571 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec359e640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec41599c0 0x7f8ec40aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 0x7f8ec41507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 -- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 msgr2=0x7f8ec41507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.941 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 0x7f8ec41507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 -- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec41599c0 msgr2=0x7f8ec40aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec41599c0 0x7f8ec40aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 -- 172.21.15.157:0/1026862571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb40094d0 con 0x7f8ec40b2af0 2023-11-06T03:19:58.942 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 0x7f8ec41507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec259c640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00c520 tx=0x7f8eac00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0052d0 con 0x7f8ec40b2af0 2023-11-06T03:19:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac005430 con 0x7f8ec40b2af0 2023-11-06T03:19:58.943 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac012870 con 0x7f8ec40b2af0 2023-11-06T03:19:58.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec4153280 con 0x7f8ec40b2af0 2023-11-06T03:19:58.944 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec40a8f60 con 0x7f8ec40b2af0 2023-11-06T03:19:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec41537a0 con 0x7f8ec40b2af0 2023-11-06T03:19:58.945 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.940+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8eac003620 con 0x7f8ec40b2af0 2023-11-06T03:19:58.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.944+0000 7f8ebb7fe640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 0x7f8ea4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:58.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.944+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f8eac0825f0 con 0x7f8ec40b2af0 2023-11-06T03:19:58.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.944+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 0x7f8ea4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:58.946 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.944+0000 7f8ec2d9d640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 0x7f8ea4067f20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40ae000 tx=0x7f8eb40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:58.947 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:58.944+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f8eac053c10 con 0x7f8ec40b2af0 2023-11-06T03:19:59.095 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.100+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ec41537a0 con 0x7f8ea4065a60 2023-11-06T03:19:59.096 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.100+0000 7f8ebb7fe640 1 -- 172.21.15.157:0/1026862571 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8ec41537a0 con 0x7f8ea4065a60 2023-11-06T03:19:59.096 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 4 to deep-scrub 2023-11-06T03:19:59.098 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 msgr2=0x7f8ea4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 0x7f8ea4067f20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40ae000 tx=0x7f8eb40a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 msgr2=0x7f8ec40a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00c520 tx=0x7f8eac00c9e0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.099 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 shutdown_connections 2023-11-06T03:19:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f8ea4065a60 0x7f8ea4067f20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ec41599c0 0x7f8ec40aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ec40b2af0 0x7f8ec40a8850 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 --2- 172.21.15.157:0/1026862571 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ec406d000 0x7f8ec41507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.100 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 >> 172.21.15.157:0/1026862571 conn(0x7f8ec4065930 msgr2=0x7f8ec4157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 shutdown_connections 2023-11-06T03:19:59.101 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.104+0000 7f8ec9231640 1 -- 172.21.15.157:0/1026862571 wait complete. 2023-11-06T03:19:59.115 INFO:tasks.ceph:Scrubbing osd.5 2023-11-06T03:19:59.115 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:59.230 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 Processor -- start 2023-11-06T03:19:59.231 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 -- start start 2023-11-06T03:19:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7cc006c550 0x7f7cc0155140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc00cb100 con 0x7f7cc015b150 2023-11-06T03:19:59.232 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc015bcf0 con 0x7f7cc006c550 2023-11-06T03:19:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cc57e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00ce0a0 con 0x7f7cc006c170 2023-11-06T03:19:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7cc006c550 0x7f7cc0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7cc006c550 0x7f7cc0155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34868/0 (socket says 172.21.15.157:34868) 2023-11-06T03:19:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 -- 172.21.15.157:0/2273214520 learned_addr learned my addr 172.21.15.157:0/2273214520 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.233 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbeffd640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 -- 172.21.15.157:0/2273214520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 msgr2=0x7f7cc0154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc0154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 -- 172.21.15.157:0/2273214520 >> v1:172.21.15.195:6789/0 conn(0x7f7cc015b150 legacy=0x7f7cc015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.234 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.236+0000 7f7cbe7fc640 1 -- 172.21.15.157:0/2273214520 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc015bef0 con 0x7f7cc006c550 2023-11-06T03:19:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbe7fc640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc006c550 0x7f7cc0155140 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aaf1abaf14d20bb server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbdffb640 1 -- 172.21.15.157:0/2273214520 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40aa5f0 con 0x7f7cc006c550 2023-11-06T03:19:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbdffb640 1 -- 172.21.15.157:0/2273214520 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb40aa750 con 0x7f7cc006c550 2023-11-06T03:19:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbdffb640 1 -- 172.21.15.157:0/2273214520 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40aaae0 con 0x7f7cc006c550 2023-11-06T03:19:59.236 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- 172.21.15.157:0/2273214520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc006c550 msgr2=0x7f7cc0155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc006c550 0x7f7cc0155140 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb401c4d0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- 172.21.15.157:0/2273214520 shutdown_connections 2023-11-06T03:19:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc006c550 0x7f7cc0155140 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/2273214520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc0154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.237 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- 172.21.15.157:0/2273214520 >> 172.21.15.157:0/2273214520 conn(0x7f7cc00b43b0 msgr2=0x7f7cc0065930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- 172.21.15.157:0/2273214520 shutdown_connections 2023-11-06T03:19:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- 172.21.15.157:0/2273214520 wait complete. 2023-11-06T03:19:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 Processor -- start 2023-11-06T03:19:59.238 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- start start 2023-11-06T03:19:59.241 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cc006c550 0x7f7cc014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 0x7f7cc015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.242 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc014a940 con 0x7f7cc015b150 2023-11-06T03:19:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00b3730 con 0x7f7cc006c550 2023-11-06T03:19:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cc57e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc015dec0 con 0x7f7cc006c170 2023-11-06T03:19:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 0x7f7cc015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 0x7f7cc015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34882/0 (socket says 172.21.15.157:34882) 2023-11-06T03:19:59.243 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 -- 172.21.15.157:0/3298989810 learned_addr learned my addr 172.21.15.157:0/3298989810 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbe7fc640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cc006c550 0x7f7cc014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbeffd640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 -- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 msgr2=0x7f7cc014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc014f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.244 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 -- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cc006c550 msgr2=0x7f7cc014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7cc006c550 0x7f7cc014a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbf7fe640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40094d0 con 0x7f7cc015b150 2023-11-06T03:19:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.240+0000 7f7cbeffd640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7cc006c170 0x7f7cc014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cbf7fe640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 0x7f7cc015d980 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b970 tx=0x7f7cb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.245 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800cb40 con 0x7f7cc015b150 2023-11-06T03:19:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb800cca0 con 0x7f7cc015b150 2023-11-06T03:19:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc00cbfb0 con 0x7f7cc015b150 2023-11-06T03:19:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8014680 con 0x7f7cc015b150 2023-11-06T03:19:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc00b3450 con 0x7f7cc015b150 2023-11-06T03:19:59.246 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cc00cbe30 con 0x7f7cc015b150 2023-11-06T03:19:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7cb801d3b0 con 0x7f7cc015b150 2023-11-06T03:19:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cac065a60 0x7f7cac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7cb8083220 con 0x7f7cc015b150 2023-11-06T03:19:59.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f7cac06bf80 0x7f7cac06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cac06ea50 con 0x7f7cac06bf80 2023-11-06T03:19:59.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb8083670 con 0x7f7cc015b150 2023-11-06T03:19:59.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cbeffd640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cac065a60 0x7f7cac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cbe7fc640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f7cac06bf80 0x7f7cac06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cbe7fc640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f7cac06bf80 0x7f7cac06e3a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.244+0000 7f7cbeffd640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cac065a60 0x7f7cac067f20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.248+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== osd.5 v2:172.21.15.195:6816/38428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7cac06ea50 con 0x7f7cac06bf80 2023-11-06T03:19:59.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 --> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7cc00cb910 con 0x7f7cac06bf80 2023-11-06T03:19:59.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7ca77fe640 1 -- 172.21.15.157:0/3298989810 <== osd.5 v2:172.21.15.195:6816/38428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7cb40aa3f0 con 0x7f7cac06bf80 2023-11-06T03:19:59.255 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:59.256 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:59.256 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:59.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f7cac06bf80 msgr2=0x7f7cac06e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6816/38428,v1:172.21.15.195:6818/38428] conn(0x7f7cac06bf80 0x7f7cac06e3a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cac065a60 msgr2=0x7f7cac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f7cac065a60 0x7f7cac067f20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0008040 comp rx=0 tx=0).stop 2023-11-06T03:19:59.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 msgr2=0x7f7cc015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 --2- 172.21.15.157:0/3298989810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cc015b150 0x7f7cc015d980 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b970 tx=0x7f7cb800be30 comp rx=0 tx=0).stop 2023-11-06T03:19:59.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cbf7fe640 1 -- 172.21.15.157:0/3298989810 reap_dead start 2023-11-06T03:19:59.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 shutdown_connections 2023-11-06T03:19:59.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 >> 172.21.15.157:0/3298989810 conn(0x7f7cc00b43b0 msgr2=0x7f7cc00b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 shutdown_connections 2023-11-06T03:19:59.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.260+0000 7f7cc57e5640 1 -- 172.21.15.157:0/3298989810 wait complete. 2023-11-06T03:19:59.271 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-06T03:19:59.375 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 Processor -- start 2023-11-06T03:19:59.376 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 -- start start 2023-11-06T03:19:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc15815e5e0 0x7fc1580b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc15815ea90 0x7fc1580b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.377 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15815f580 con 0x7fc1581551e0 2023-11-06T03:19:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580704b0 con 0x7fc15815e5e0 2023-11-06T03:19:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc160770640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15815efc0 con 0x7fc15815ea90 2023-11-06T03:19:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc15dd0b640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc15815e5e0 0x7fc1580b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.378 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc15e50c640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc1581551e0 0x7fc15806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51138/0 (socket says 172.21.15.157:51138) 2023-11-06T03:19:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc15e50c640 1 -- 172.21.15.157:0/2589363323 learned_addr learned my addr 172.21.15.157:0/2589363323 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.380+0000 7fc15ed0d640 1 --2- 172.21.15.157:0/2589363323 >> v2:172.21.15.195:3300/0 conn(0x7fc15815ea90 0x7fc1580b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15dd0b640 1 -- 172.21.15.157:0/2589363323 >> v2:172.21.15.195:3300/0 conn(0x7fc15815ea90 msgr2=0x7fc1580b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15dd0b640 1 --2- 172.21.15.157:0/2589363323 >> v2:172.21.15.195:3300/0 conn(0x7fc15815ea90 0x7fc1580b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.379 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15dd0b640 1 -- 172.21.15.157:0/2589363323 >> v1:172.21.15.157:6789/0 conn(0x7fc1581551e0 legacy=0x7fc15806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15dd0b640 1 -- 172.21.15.157:0/2589363323 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15815f2a0 con 0x7fc15815e5e0 2023-11-06T03:19:59.380 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15d50a640 1 -- 172.21.15.157:0/2589363323 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758244175 0 0) 0x7fc15815f580 con 0x7fc1581551e0 2023-11-06T03:19:59.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15dd0b640 1 --2- 172.21.15.157:0/2589363323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15815e5e0 0x7fc1580b3410 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fc1580cc2f0 tx=0x7fc14801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443a56c29343e84e server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15d50a640 1 -- 172.21.15.157:0/2589363323 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480ac600 con 0x7fc15815e5e0 2023-11-06T03:19:59.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15d50a640 1 -- 172.21.15.157:0/2589363323 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1480a8050 con 0x7fc15815e5e0 2023-11-06T03:19:59.385 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc15d50a640 1 -- 172.21.15.157:0/2589363323 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480ac980 con 0x7fc15815e5e0 2023-11-06T03:19:59.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 -- 172.21.15.157:0/2589363323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15815e5e0 msgr2=0x7fc1580b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 --2- 172.21.15.157:0/2589363323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15815e5e0 0x7fc1580b3410 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fc1580cc2f0 tx=0x7fc14801c3a0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 -- 172.21.15.157:0/2589363323 shutdown_connections 2023-11-06T03:19:59.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 --2- 172.21.15.157:0/2589363323 >> v2:172.21.15.195:3300/0 conn(0x7fc15815ea90 0x7fc1580b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.386 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 --2- 172.21.15.157:0/2589363323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc15815e5e0 0x7fc1580b3410 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 -- 172.21.15.157:0/2589363323 >> 172.21.15.157:0/2589363323 conn(0x7fc158065930 msgr2=0x7fc158065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 -- 172.21.15.157:0/2589363323 shutdown_connections 2023-11-06T03:19:59.387 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.384+0000 7fc160770640 1 -- 172.21.15.157:0/2589363323 wait complete. 2023-11-06T03:19:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 Processor -- start 2023-11-06T03:19:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- start start 2023-11-06T03:19:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc15815e5e0 0x7fc158152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.388 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc15815ea90 0x7fc158152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158155760 con 0x7fc1581551e0 2023-11-06T03:19:59.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158155a40 con 0x7fc15815ea90 2023-11-06T03:19:59.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc158155d20 con 0x7fc15815e5e0 2023-11-06T03:19:59.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.389 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34890/0 (socket says 172.21.15.157:34890) 2023-11-06T03:19:59.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 -- 172.21.15.157:0/3481794827 learned_addr learned my addr 172.21.15.157:0/3481794827 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15dd0b640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc15815e5e0 0x7fc158152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.390 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15ed0d640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc15815ea90 0x7fc158152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 -- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc15815e5e0 msgr2=0x7fc158152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.391 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc15815e5e0 0x7fc158152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 -- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc15815ea90 msgr2=0x7fc158152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc15815ea90 0x7fc158152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 -- 172.21.15.157:0/3481794827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009420 con 0x7fc1581551e0 2023-11-06T03:19:59.392 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15e50c640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fc1540011f0 tx=0x7fc15400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540074e0 con 0x7fc1581551e0 2023-11-06T03:19:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc154007640 con 0x7fc1581551e0 2023-11-06T03:19:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154004a10 con 0x7fc1581551e0 2023-11-06T03:19:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc158156000 con 0x7fc1581551e0 2023-11-06T03:19:59.393 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc158156800 con 0x7fc1581551e0 2023-11-06T03:19:59.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158072750 con 0x7fc1581551e0 2023-11-06T03:19:59.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc154004b70 con 0x7fc1581551e0 2023-11-06T03:19:59.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.394 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc154082360 con 0x7fc1581551e0 2023-11-06T03:19:59.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.388+0000 7fc15dd0b640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.395 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.392+0000 7fc15dd0b640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 0x7fc13c067f20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ae000 tx=0x7fc148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.396 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.392+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc15404fe60 con 0x7fc1581551e0 2023-11-06T03:19:59.531 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.536+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fc158155bb0 con 0x7fc13c065a60 2023-11-06T03:19:59.532 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.536+0000 7fc14effd640 1 -- 172.21.15.157:0/3481794827 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc158155bb0 con 0x7fc13c065a60 2023-11-06T03:19:59.532 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 5 to deep-scrub 2023-11-06T03:19:59.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 msgr2=0x7fc13c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.535 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 0x7fc13c067f20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ae000 tx=0x7fc148008040 comp rx=0 tx=0).stop 2023-11-06T03:19:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 msgr2=0x7fc158157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fc1540011f0 tx=0x7fc15400c9b0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 shutdown_connections 2023-11-06T03:19:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc13c065a60 0x7fc13c067f20 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.536 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc15815ea90 0x7fc158152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc15815e5e0 0x7fc158152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 --2- 172.21.15.157:0/3481794827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1581551e0 0x7fc158157470 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 >> 172.21.15.157:0/3481794827 conn(0x7fc158065930 msgr2=0x7fc15815ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 shutdown_connections 2023-11-06T03:19:59.537 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.540+0000 7fc160770640 1 -- 172.21.15.157:0/3481794827 wait complete. 2023-11-06T03:19:59.550 INFO:tasks.ceph:Scrubbing osd.6 2023-11-06T03:19:59.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:19:59.666 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 Processor -- start 2023-11-06T03:19:59.667 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 -- start start 2023-11-06T03:19:59.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b140cd9c0 0x7f3b140d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3b140ce850 0x7f3b140cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.668 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b140cf1c0 con 0x7f3b1415afb0 2023-11-06T03:19:59.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140cbdc0 con 0x7f3b140cd9c0 2023-11-06T03:19:59.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b1ad4a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140ce250 con 0x7f3b140ce850 2023-11-06T03:19:59.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b140cd9c0 0x7f3b140d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3b140cd9c0 0x7f3b140d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34898/0 (socket says 172.21.15.157:34898) 2023-11-06T03:19:59.669 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 -- 172.21.15.157:0/3188157282 learned_addr learned my addr 172.21.15.157:0/3188157282 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 -- 172.21.15.157:0/3188157282 >> v2:172.21.15.195:3300/0 conn(0x7f3b140ce850 msgr2=0x7f3b140cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b192e7640 1 --2- 172.21.15.157:0/3188157282 >> v2:172.21.15.195:3300/0 conn(0x7f3b140ce850 0x7f3b140cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 --2- 172.21.15.157:0/3188157282 >> v2:172.21.15.195:3300/0 conn(0x7f3b140ce850 0x7f3b140cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 -- 172.21.15.157:0/3188157282 >> v1:172.21.15.195:6789/0 conn(0x7f3b1415afb0 legacy=0x7f3b140cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.670 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.672+0000 7f3b13fff640 1 -- 172.21.15.157:0/3188157282 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140cdf70 con 0x7f3b140cd9c0 2023-11-06T03:19:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b13fff640 1 --2- 172.21.15.157:0/3188157282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140cd9c0 0x7f3b140d1470 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f3b140722a0 tx=0x7f3b0801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9127924f357beb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b137fe640 1 -- 172.21.15.157:0/3188157282 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ab600 con 0x7f3b140cd9c0 2023-11-06T03:19:59.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b137fe640 1 -- 172.21.15.157:0/3188157282 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080ab760 con 0x7f3b140cd9c0 2023-11-06T03:19:59.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b137fe640 1 -- 172.21.15.157:0/3188157282 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080abaf0 con 0x7f3b140cd9c0 2023-11-06T03:19:59.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3188157282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140cd9c0 msgr2=0x7f3b140d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3188157282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140cd9c0 0x7f3b140d1470 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f3b140722a0 tx=0x7f3b0801c400 comp rx=0 tx=0).stop 2023-11-06T03:19:59.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3188157282 shutdown_connections 2023-11-06T03:19:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3188157282 >> v2:172.21.15.195:3300/0 conn(0x7f3b140ce850 0x7f3b140cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3188157282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140cd9c0 0x7f3b140d1470 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3188157282 >> 172.21.15.157:0/3188157282 conn(0x7f3b140d0950 msgr2=0x7f3b140d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3188157282 shutdown_connections 2023-11-06T03:19:59.679 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3188157282 wait complete. 2023-11-06T03:19:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 Processor -- start 2023-11-06T03:19:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- start start 2023-11-06T03:19:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b140cd9c0 0x7f3b141575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 0x7f3b140c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b1415afb0 0x7f3b14155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140ca690 con 0x7f3b140ce850 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14155e90 con 0x7f3b140cd9c0 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b1ad4a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140ca0d0 con 0x7f3b1415afb0 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 0x7f3b140c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 0x7f3b140c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34904/0 (socket says 172.21.15.157:34904) 2023-11-06T03:19:59.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b13fff640 1 -- 172.21.15.157:0/3664016253 learned_addr learned my addr 172.21.15.157:0/3664016253 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b18ae6640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b140cd9c0 0x7f3b141575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b13fff640 1 -- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b1415afb0 msgr2=0x7f3b14155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.676+0000 7f3b192e7640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b1415afb0 0x7f3b14155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b13fff640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b1415afb0 0x7f3b14155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b13fff640 1 -- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b140cd9c0 msgr2=0x7f3b141575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b13fff640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b140cd9c0 0x7f3b141575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b13fff640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080094d0 con 0x7f3b140ce850 2023-11-06T03:19:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b192e7640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b1415afb0 0x7f3b14155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b18ae6640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b140cd9c0 0x7f3b141575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:19:59.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b13fff640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 0x7f3b140c6980 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f3b0801c8e0 tx=0x7f3b0801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b4800 con 0x7f3b140ce850 2023-11-06T03:19:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080b4960 con 0x7f3b140ce850 2023-11-06T03:19:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b140ca3b0 con 0x7f3b140ce850 2023-11-06T03:19:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b4d10 con 0x7f3b140ce850 2023-11-06T03:19:59.684 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b140c6460 con 0x7f3b140ce850 2023-11-06T03:19:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b140cc8d0 con 0x7f3b140ce850 2023-11-06T03:19:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b080059a0 con 0x7f3b140ce850 2023-11-06T03:19:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b00065a60 0x7f3b00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3b0811de80 con 0x7f3b140ce850 2023-11-06T03:19:59.685 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3b0006bf80 0x7f3b0006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b0006ea50 con 0x7f3b0006bf80 2023-11-06T03:19:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b0811e2d0 con 0x7f3b140ce850 2023-11-06T03:19:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b18ae6640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b00065a60 0x7f3b00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b192e7640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3b0006bf80 0x7f3b0006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.686 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b18ae6640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b00065a60 0x7f3b00067f20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3b1415ded0 tx=0x7f3b04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.680+0000 7f3b192e7640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3b0006bf80 0x7f3b0006e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.687 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.684+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== osd.6 v2:172.21.15.195:6817/38426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b0006ea50 con 0x7f3b0006bf80 2023-11-06T03:19:59.690 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 --> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b1406da00 con 0x7f3b0006bf80 2023-11-06T03:19:59.691 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b117fa640 1 -- 172.21.15.157:0/3664016253 <== osd.6 v2:172.21.15.195:6817/38426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b0c00e3f0 con 0x7f3b0006bf80 2023-11-06T03:19:59.691 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:19:59.692 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:19:59.692 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:19:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3b0006bf80 msgr2=0x7f3b0006e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6817/38426,v1:172.21.15.195:6820/38426] conn(0x7f3b0006bf80 0x7f3b0006e3a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.693 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b00065a60 msgr2=0x7f3b00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3b00065a60 0x7f3b00067f20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3b1415ded0 tx=0x7f3b04009040 comp rx=0 tx=0).stop 2023-11-06T03:19:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 msgr2=0x7f3b140c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 --2- 172.21.15.157:0/3664016253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b140ce850 0x7f3b140c6980 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f3b0801c8e0 tx=0x7f3b0801ac40 comp rx=0 tx=0).stop 2023-11-06T03:19:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b192e7640 1 -- 172.21.15.157:0/3664016253 reap_dead start 2023-11-06T03:19:59.694 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 shutdown_connections 2023-11-06T03:19:59.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 >> 172.21.15.157:0/3664016253 conn(0x7f3b140d0950 msgr2=0x7f3b140ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 shutdown_connections 2023-11-06T03:19:59.695 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.696+0000 7f3b1ad4a640 1 -- 172.21.15.157:0/3664016253 wait complete. 2023-11-06T03:19:59.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-06T03:19:59.818 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 Processor -- start 2023-11-06T03:19:59.819 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 -- start start 2023-11-06T03:19:59.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65f415aa80 0x7f65f40b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f65f40b1d40 0x7f65f40cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f40c94f0 con 0x7f65f40b17f0 2023-11-06T03:19:59.820 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f415ae60 con 0x7f65f415aa80 2023-11-06T03:19:59.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f9af4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f40b3620 con 0x7f65f40b1d40 2023-11-06T03:19:59.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65f415aa80 0x7f65f40b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f3fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f65f40b17f0 0x7f65f415e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51152/0 (socket says 172.21.15.157:51152) 2023-11-06T03:19:59.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f65f415aa80 0x7f65f40b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34916/0 (socket says 172.21.15.157:34916) 2023-11-06T03:19:59.821 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f3fff640 1 -- 172.21.15.157:0/1982789741 learned_addr learned my addr 172.21.15.157:0/1982789741 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:19:59.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f2ffd640 1 --2- 172.21.15.157:0/1982789741 >> v2:172.21.15.195:3300/0 conn(0x7f65f40b1d40 0x7f65f40cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 -- 172.21.15.157:0/1982789741 >> v2:172.21.15.195:3300/0 conn(0x7f65f40b1d40 msgr2=0x7f65f40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:19:59.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 --2- 172.21.15.157:0/1982789741 >> v2:172.21.15.195:3300/0 conn(0x7f65f40b1d40 0x7f65f40cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 -- 172.21.15.157:0/1982789741 >> v1:172.21.15.157:6789/0 conn(0x7f65f40b17f0 legacy=0x7f65f415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.822 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f37fe640 1 -- 172.21.15.157:0/1982789741 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f40c97b0 con 0x7f65f415aa80 2023-11-06T03:19:59.823 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.824+0000 7f65f27fc640 1 -- 172.21.15.157:0/1982789741 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974897039 0 0) 0x7f65f40c94f0 con 0x7f65f40b17f0 2023-11-06T03:19:59.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f37fe640 1 --2- 172.21.15.157:0/1982789741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f415aa80 0x7f65f40b30e0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f65f4070480 tx=0x7f65dc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fc3ff211a3a714 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f27fc640 1 -- 172.21.15.157:0/1982789741 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0ab650 con 0x7f65f415aa80 2023-11-06T03:19:59.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f27fc640 1 -- 172.21.15.157:0/1982789741 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc0ab7b0 con 0x7f65f415aa80 2023-11-06T03:19:59.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f27fc640 1 -- 172.21.15.157:0/1982789741 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0abb40 con 0x7f65f415aa80 2023-11-06T03:19:59.824 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- 172.21.15.157:0/1982789741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f415aa80 msgr2=0x7f65f40b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 --2- 172.21.15.157:0/1982789741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f415aa80 0x7f65f40b30e0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f65f4070480 tx=0x7f65dc01c430 comp rx=0 tx=0).stop 2023-11-06T03:19:59.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- 172.21.15.157:0/1982789741 shutdown_connections 2023-11-06T03:19:59.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 --2- 172.21.15.157:0/1982789741 >> v2:172.21.15.195:3300/0 conn(0x7f65f40b1d40 0x7f65f40cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 --2- 172.21.15.157:0/1982789741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f415aa80 0x7f65f40b30e0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.825 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- 172.21.15.157:0/1982789741 >> 172.21.15.157:0/1982789741 conn(0x7f65f406a170 msgr2=0x7f65f406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- 172.21.15.157:0/1982789741 shutdown_connections 2023-11-06T03:19:59.826 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- 172.21.15.157:0/1982789741 wait complete. 2023-11-06T03:19:59.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 Processor -- start 2023-11-06T03:19:59.828 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 -- start start 2023-11-06T03:19:59.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f40b17f0 0x7f65f4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.828+0000 7f65f9af4640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f415aa80 0x7f65f41639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f41528c0 con 0x7f65f40b1d40 2023-11-06T03:19:59.829 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f4156e50 con 0x7f65f415aa80 2023-11-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f41535c0 con 0x7f65f40b17f0 2023-11-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34932/0 (socket says 172.21.15.157:34932) 2023-11-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 -- 172.21.15.157:0/3911256868 learned_addr learned my addr 172.21.15.157:0/3911256868 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f37fe640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f40b17f0 0x7f65f4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 -- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f40b17f0 msgr2=0x7f65f4156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f40b17f0 0x7f65f4156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 -- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f415aa80 msgr2=0x7f65f41639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f415aa80 0x7f65f41639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 -- 172.21.15.157:0/3911256868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc009420 con 0x7f65f40b1d40 2023-11-06T03:19:59.831 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f2ffd640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f65e4007ef0 tx=0x7f65e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4014620 con 0x7f65f40b1d40 2023-11-06T03:19:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4014780 con 0x7f65f40b1d40 2023-11-06T03:19:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f4156b70 con 0x7f65f40b1d40 2023-11-06T03:19:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4014b30 con 0x7f65f40b1d40 2023-11-06T03:19:59.832 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f4153b60 con 0x7f65f40b1d40 2023-11-06T03:19:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f4072750 con 0x7f65f40b1d40 2023-11-06T03:19:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f65e4014c90 con 0x7f65f40b1d40 2023-11-06T03:19:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 0x7f65d4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:19:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f65e4083950 con 0x7f65f40b1d40 2023-11-06T03:19:59.833 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f37fe640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 0x7f65d4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:19:59.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.832+0000 7f65f37fe640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 0x7f65d4067f50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0af000 tx=0x7f65dc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:19:59.834 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.836+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7f65e40514e0 con 0x7f65f40b1d40 2023-11-06T03:19:59.974 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f65f4153440 con 0x7f65d4065a90 2023-11-06T03:19:59.975 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65ebfff640 1 -- 172.21.15.157:0/3911256868 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65dc0ab4d0 con 0x7f65d4065a90 2023-11-06T03:19:59.975 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 6 to deep-scrub 2023-11-06T03:19:59.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 msgr2=0x7f65d4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 0x7f65d4067f50 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0af000 tx=0x7f65dc0a7000 comp rx=0 tx=0).stop 2023-11-06T03:19:59.977 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 msgr2=0x7f65f414e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:19:59.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f65e4007ef0 tx=0x7f65e400e4b0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 shutdown_connections 2023-11-06T03:19:59.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f65d4065a90 0x7f65d4067f50 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f65f415aa80 0x7f65f41639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.978 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65f40b1d40 0x7f65f414e930 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 --2- 172.21.15.157:0/3911256868 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f65f40b17f0 0x7f65f4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:19:59.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 >> 172.21.15.157:0/3911256868 conn(0x7f65f406a170 msgr2=0x7f65f415e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:19:59.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 shutdown_connections 2023-11-06T03:19:59.979 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:19:59.980+0000 7f65f9af4640 1 -- 172.21.15.157:0/3911256868 wait complete. 2023-11-06T03:19:59.992 INFO:tasks.ceph:Scrubbing osd.7 2023-11-06T03:19:59.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-06T03:20:00.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 Processor -- start 2023-11-06T03:20:00.103 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 -- start start 2023-11-06T03:20:00.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f343015a9a0 0x7f343006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.104 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 0x7f3430065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343015e150 con 0x7f34300c9f00 2023-11-06T03:20:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343006ed90 con 0x7f343015a9a0 2023-11-06T03:20:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3435c48640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300ca780 con 0x7f3430065930 2023-11-06T03:20:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f343015a9a0 0x7f343006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.105 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f343015a9a0 0x7f343006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34936/0 (socket says 172.21.15.157:34936) 2023-11-06T03:20:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 -- 172.21.15.157:0/1607246832 learned_addr learned my addr 172.21.15.157:0/1607246832 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f3427fff640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f34300c9f00 0x7f343015da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:51168/0 (socket says 172.21.15.157:51168) 2023-11-06T03:20:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342ffff640 1 --2- 172.21.15.157:0/1607246832 >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 0x7f3430065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 -- 172.21.15.157:0/1607246832 >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 msgr2=0x7f3430065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:00.106 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 --2- 172.21.15.157:0/1607246832 >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 0x7f3430065d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 -- 172.21.15.157:0/1607246832 >> v1:172.21.15.157:6789/0 conn(0x7f34300c9f00 legacy=0x7f343015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342f7fe640 1 -- 172.21.15.157:0/1607246832 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430066280 con 0x7f343015a9a0 2023-11-06T03:20:00.107 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.108+0000 7f342ffff640 1 --2- 172.21.15.157:0/1607246832 >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 0x7f3430065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:00.108 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342f7fe640 1 --2- 172.21.15.157:0/1607246832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343006c390 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f343006ef20 tx=0x7f342001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec0167e8f0ca0a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342effd640 1 -- 172.21.15.157:0/1607246832 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200ab5f0 con 0x7f343015a9a0 2023-11-06T03:20:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342effd640 1 -- 172.21.15.157:0/1607246832 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34200ab750 con 0x7f343015a9a0 2023-11-06T03:20:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342effd640 1 -- 172.21.15.157:0/1607246832 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34200abae0 con 0x7f343015a9a0 2023-11-06T03:20:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- 172.21.15.157:0/1607246832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 msgr2=0x7f343006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.109 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- 172.21.15.157:0/1607246832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343006c390 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f343006ef20 tx=0x7f342001c500 comp rx=0 tx=0).stop 2023-11-06T03:20:00.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- 172.21.15.157:0/1607246832 shutdown_connections 2023-11-06T03:20:00.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- 172.21.15.157:0/1607246832 >> v2:172.21.15.195:3300/0 conn(0x7f3430065930 0x7f3430065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- 172.21.15.157:0/1607246832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343006c390 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- 172.21.15.157:0/1607246832 >> 172.21.15.157:0/1607246832 conn(0x7f34300cf8b0 msgr2=0x7f34300cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.110 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- 172.21.15.157:0/1607246832 shutdown_connections 2023-11-06T03:20:00.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- 172.21.15.157:0/1607246832 wait complete. 2023-11-06T03:20:00.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 Processor -- start 2023-11-06T03:20:00.111 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- start start 2023-11-06T03:20:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3430065930 0x7f34301565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34300c9f00 0x7f34300acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343014a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.114 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34300adee0 con 0x7f343015a9a0 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343015e650 con 0x7f3430065930 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f3435c48640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34300ae130 con 0x7f34300c9f00 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343014a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343014a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34952/0 (socket says 172.21.15.157:34952) 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.112+0000 7f342ffff640 1 -- 172.21.15.157:0/3879803974 learned_addr learned my addr 172.21.15.157:0/3879803974 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.115 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342f7fe640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3430065930 0x7f34301565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 -- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34300c9f00 msgr2=0x7f34300acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3427fff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34300c9f00 0x7f34300acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34300c9f00 0x7f34300acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 -- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3430065930 msgr2=0x7f34301565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.116 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3430065930 0x7f34301565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200094d0 con 0x7f343015a9a0 2023-11-06T03:20:00.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3427fff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f34300c9f00 0x7f34300acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:00.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342f7fe640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3430065930 0x7f34301565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-06T03:20:00.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342ffff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343014a5c0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f342800cf20 tx=0x7f342800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.117 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800d840 con 0x7f343015a9a0 2023-11-06T03:20:00.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342800d9a0 con 0x7f343015a9a0 2023-11-06T03:20:00.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800dd50 con 0x7f343015a9a0 2023-11-06T03:20:00.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34300ae410 con 0x7f343015a9a0 2023-11-06T03:20:00.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343015e2e0 con 0x7f343015a9a0 2023-11-06T03:20:00.118 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f343006c8d0 con 0x7f343015a9a0 2023-11-06T03:20:00.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f342801e3b0 con 0x7f343015a9a0 2023-11-06T03:20:00.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3410065a60 0x7f3410067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f34280841a0 con 0x7f343015a9a0 2023-11-06T03:20:00.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7f341006bf80 0x7f341006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.119 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f341006ea50 con 0x7f341006bf80 2023-11-06T03:20:00.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34280845f0 con 0x7f343015a9a0 2023-11-06T03:20:00.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f342f7fe640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3410065a60 0x7f3410067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.116+0000 7f3427fff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7f341006bf80 0x7f341006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.120+0000 7f342f7fe640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3410065a60 0x7f3410067f20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f34200af000 tx=0x7f34200a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.120 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.120+0000 7f3427fff640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7f341006bf80 0x7f341006e3a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.121 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.120+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== osd.7 v2:172.21.15.195:6800/38427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f341006ea50 con 0x7f341006bf80 2023-11-06T03:20:00.126 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 --> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f343006eab0 con 0x7f341006bf80 2023-11-06T03:20:00.127 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f342cff9640 1 -- 172.21.15.157:0/3879803974 <== osd.7 v2:172.21.15.195:6800/38427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3418001850 con 0x7f341006bf80 2023-11-06T03:20:00.127 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-11-06T03:20:00.128 INFO:teuthology.orchestra.run.smithi157.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-06T03:20:00.128 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-11-06T03:20:00.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7f341006bf80 msgr2=0x7f341006e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6800/38427,v1:172.21.15.195:6801/38427] conn(0x7f341006bf80 0x7f341006e3a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3410065a60 msgr2=0x7f3410067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.129 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7f3410065a60 0x7f3410067f20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f34200af000 tx=0x7f34200a7000 comp rx=0 tx=0).stop 2023-11-06T03:20:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 msgr2=0x7f343014a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 --2- 172.21.15.157:0/3879803974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f343015a9a0 0x7f343014a5c0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f342800cf20 tx=0x7f342800a7d0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f342ffff640 1 -- 172.21.15.157:0/3879803974 reap_dead start 2023-11-06T03:20:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 shutdown_connections 2023-11-06T03:20:00.130 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 >> 172.21.15.157:0/3879803974 conn(0x7f34300cf8b0 msgr2=0x7f343015b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 shutdown_connections 2023-11-06T03:20:00.131 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.132+0000 7f3435c48640 1 -- 172.21.15.157:0/3879803974 wait complete. 2023-11-06T03:20:00.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-06T03:20:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 Processor -- start 2023-11-06T03:20:00.247 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 -- start start 2023-11-06T03:20:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fab40146370 0x7fab400b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab4006bdb0 0x7fab400cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab400cf1a0 con 0x7fab400ce800 2023-11-06T03:20:00.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4006c3d0 con 0x7fab4006bdb0 2023-11-06T03:20:00.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab465c9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4006f5a0 con 0x7fab40146370 2023-11-06T03:20:00.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab4006bdb0 0x7fab400cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.249 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab4006bdb0 0x7fab400cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34956/0 (socket says 172.21.15.157:34956) 2023-11-06T03:20:00.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 -- 172.21.15.157:0/2877993118 learned_addr learned my addr 172.21.15.157:0/2877993118 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 -- 172.21.15.157:0/2877993118 >> v2:172.21.15.195:3300/0 conn(0x7fab40146370 msgr2=0x7fab400b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:00.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab3ffff640 1 --2- 172.21.15.157:0/2877993118 >> v2:172.21.15.195:3300/0 conn(0x7fab40146370 0x7fab400b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.250 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 --2- 172.21.15.157:0/2877993118 >> v2:172.21.15.195:3300/0 conn(0x7fab40146370 0x7fab400b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 -- 172.21.15.157:0/2877993118 >> v1:172.21.15.195:6789/0 conn(0x7fab400ce800 legacy=0x7fab4006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:00.251 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.252+0000 7fab44b66640 1 -- 172.21.15.157:0/2877993118 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4006f880 con 0x7fab4006bdb0 2023-11-06T03:20:00.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab44b66640 1 --2- 172.21.15.157:0/2877993118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab400cec60 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e50 tx=0x7fab3801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ed2f8d09e8b634 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.252 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab3effd640 1 -- 172.21.15.157:0/2877993118 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380a5790 con 0x7fab4006bdb0 2023-11-06T03:20:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab3effd640 1 -- 172.21.15.157:0/2877993118 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380a58f0 con 0x7fab4006bdb0 2023-11-06T03:20:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab3effd640 1 -- 172.21.15.157:0/2877993118 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380a5c80 con 0x7fab4006bdb0 2023-11-06T03:20:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- 172.21.15.157:0/2877993118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 msgr2=0x7fab400cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- 172.21.15.157:0/2877993118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab400cec60 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fab38009e50 tx=0x7fab3801c8d0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.253 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- 172.21.15.157:0/2877993118 shutdown_connections 2023-11-06T03:20:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- 172.21.15.157:0/2877993118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab400cec60 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- 172.21.15.157:0/2877993118 >> v2:172.21.15.195:3300/0 conn(0x7fab40146370 0x7fab400b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- 172.21.15.157:0/2877993118 >> 172.21.15.157:0/2877993118 conn(0x7fab40065930 msgr2=0x7fab40065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.254 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- 172.21.15.157:0/2877993118 shutdown_connections 2023-11-06T03:20:00.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- 172.21.15.157:0/2877993118 wait complete. 2023-11-06T03:20:00.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 Processor -- start 2023-11-06T03:20:00.255 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- start start 2023-11-06T03:20:00.257 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab400ce800 0x7fab4013a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab40146370 0x7fab40143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40155a00 con 0x7fab4006bdb0 2023-11-06T03:20:00.258 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4013bd40 con 0x7fab400ce800 2023-11-06T03:20:00.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.256+0000 7fab465c9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4013c020 con 0x7fab40146370 2023-11-06T03:20:00.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34972/0 (socket says 172.21.15.157:34972) 2023-11-06T03:20:00.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 -- 172.21.15.157:0/4010207639 learned_addr learned my addr 172.21.15.157:0/4010207639 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.259 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab44b66640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab40146370 0x7fab40143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3f7fe640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab400ce800 0x7fab4013a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 -- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab40146370 msgr2=0x7fab40143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab40146370 0x7fab40143b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 -- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab400ce800 msgr2=0x7fab4013a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.260 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab400ce800 0x7fab4013a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 -- 172.21.15.157:0/4010207639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009b30 con 0x7fab4006bdb0 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3ffff640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c7f0 tx=0x7fab3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004220 con 0x7fab4006bdb0 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab300043a0 con 0x7fab4006bdb0 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4013c300 con 0x7fab4006bdb0 2023-11-06T03:20:00.261 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004510 con 0x7fab4006bdb0 2023-11-06T03:20:00.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4013c8b0 con 0x7fab4006bdb0 2023-11-06T03:20:00.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4013c540 con 0x7fab4006bdb0 2023-11-06T03:20:00.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fab30003a70 con 0x7fab4006bdb0 2023-11-06T03:20:00.262 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 0x7fab2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fab30083070 con 0x7fab4006bdb0 2023-11-06T03:20:00.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3f7fe640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 0x7fab2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.260+0000 7fab3f7fe640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 0x7fab2c067f20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fab4015a8e0 tx=0x7fab34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.263 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.264+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fab30050b70 con 0x7fab4006bdb0 2023-11-06T03:20:00.402 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fab40146750 con 0x7fab2c065a60 2023-11-06T03:20:00.403 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab3cff9640 1 -- 172.21.15.157:0/4010207639 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fab3400a380 con 0x7fab2c065a60 2023-11-06T03:20:00.403 INFO:teuthology.orchestra.run.smithi157.stderr:instructed osd(s) 7 to deep-scrub 2023-11-06T03:20:00.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 msgr2=0x7fab2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 0x7fab2c067f20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fab4015a8e0 tx=0x7fab34008040 comp rx=0 tx=0).stop 2023-11-06T03:20:00.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 msgr2=0x7fab40144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.405 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c7f0 tx=0x7fab3000ccb0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 shutdown_connections 2023-11-06T03:20:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fab2c065a60 0x7fab2c067f20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab40146370 0x7fab40143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab400ce800 0x7fab4013a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.406 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 --2- 172.21.15.157:0/4010207639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab4006bdb0 0x7fab40144430 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 >> 172.21.15.157:0/4010207639 conn(0x7fab40065930 msgr2=0x7fab40070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 shutdown_connections 2023-11-06T03:20:00.407 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.408+0000 7fab465c9640 1 -- 172.21.15.157:0/4010207639 wait complete. 2023-11-06T03:20:00.420 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:20:00.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.544+0000 7feb2592e640 1 Processor -- start 2023-11-06T03:20:00.540 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.544+0000 7feb2592e640 1 -- start start 2023-11-06T03:20:00.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.544+0000 7feb2592e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7feb2015ad30 0x7feb200b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb2592e640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb200b23b0 0x7feb200b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb2592e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb200cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.542 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb2592e640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200cef10 con 0x7feb200b23b0 2023-11-06T03:20:00.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb2592e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200cfb70 con 0x7feb2015ad30 2023-11-06T03:20:00.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb2592e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200b31e0 con 0x7feb200b2d50 2023-11-06T03:20:00.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb200b23b0 0x7feb200b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7feb200b23b0 0x7feb200b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34982/0 (socket says 172.21.15.157:34982) 2023-11-06T03:20:00.543 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 -- 172.21.15.157:0/3210196219 learned_addr learned my addr 172.21.15.157:0/3210196219 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1f7fe640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb200cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1effd640 1 --2- 172.21.15.157:0/3210196219 >> v2:172.21.15.195:3300/0 conn(0x7feb2015ad30 0x7feb200b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 -- 172.21.15.157:0/3210196219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 msgr2=0x7feb200cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:00.544 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb200cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 -- 172.21.15.157:0/3210196219 >> v2:172.21.15.195:3300/0 conn(0x7feb2015ad30 msgr2=0x7feb200b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:00.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 --2- 172.21.15.157:0/3210196219 >> v2:172.21.15.195:3300/0 conn(0x7feb2015ad30 0x7feb200b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.545 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.548+0000 7feb1e7fc640 1 -- 172.21.15.157:0/3210196219 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200cf160 con 0x7feb200b23b0 2023-11-06T03:20:00.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb1e7fc640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200b23b0 0x7feb200b2810 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7feb200cde70 tx=0x7feb1001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee28fc5350fb7111 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb1dffb640 1 -- 172.21.15.157:0/3210196219 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100ab600 con 0x7feb200b23b0 2023-11-06T03:20:00.546 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb1dffb640 1 -- 172.21.15.157:0/3210196219 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100ab760 con 0x7feb200b23b0 2023-11-06T03:20:00.547 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb1dffb640 1 -- 172.21.15.157:0/3210196219 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100abaf0 con 0x7feb200b23b0 2023-11-06T03:20:00.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- 172.21.15.157:0/3210196219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200b23b0 msgr2=0x7feb200b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200b23b0 0x7feb200b2810 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7feb200cde70 tx=0x7feb1001c440 comp rx=0 tx=0).stop 2023-11-06T03:20:00.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- 172.21.15.157:0/3210196219 shutdown_connections 2023-11-06T03:20:00.548 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb200cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 --2- 172.21.15.157:0/3210196219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb200b23b0 0x7feb200b2810 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 --2- 172.21.15.157:0/3210196219 >> v2:172.21.15.195:3300/0 conn(0x7feb2015ad30 0x7feb200b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- 172.21.15.157:0/3210196219 >> 172.21.15.157:0/3210196219 conn(0x7feb20065d30 msgr2=0x7feb20066140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- 172.21.15.157:0/3210196219 shutdown_connections 2023-11-06T03:20:00.549 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- 172.21.15.157:0/3210196219 wait complete. 2023-11-06T03:20:00.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 Processor -- start 2023-11-06T03:20:00.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.552+0000 7feb2592e640 1 -- start start 2023-11-06T03:20:00.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 0x7feb20142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb20158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.552 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20159b50 con 0x7feb2015ad30 2023-11-06T03:20:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20158f40 con 0x7feb200b23b0 2023-11-06T03:20:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb20159220 con 0x7feb200b2d50 2023-11-06T03:20:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.553 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:34998/0 (socket says 172.21.15.157:34998) 2023-11-06T03:20:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 -- 172.21.15.157:0/2742524077 learned_addr learned my addr 172.21.15.157:0/2742524077 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1e7fc640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb20158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1effd640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 0x7feb20142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 -- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 msgr2=0x7feb20158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.554 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb20158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 -- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 msgr2=0x7feb20142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 0x7feb20142ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 -- 172.21.15.157:0/2742524077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100094d0 con 0x7feb2015ad30 2023-11-06T03:20:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1effd640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 0x7feb20142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:00.555 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1e7fc640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb20158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb1f7fe640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7feb1800eed0 tx=0x7feb1800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18010040 con 0x7feb2015ad30 2023-11-06T03:20:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18014400 con 0x7feb2015ad30 2023-11-06T03:20:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18009ea0 con 0x7feb2015ad30 2023-11-06T03:20:00.556 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb20159500 con 0x7feb2015ad30 2023-11-06T03:20:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb201430e0 con 0x7feb2015ad30 2023-11-06T03:20:00.557 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb20159cc0 con 0x7feb2015ad30 2023-11-06T03:20:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb180147b0 con 0x7feb2015ad30 2023-11-06T03:20:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.556+0000 7feb17fff640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 0x7feb00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.560+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7feb18083f40 con 0x7feb2015ad30 2023-11-06T03:20:00.558 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.560+0000 7feb1effd640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 0x7feb00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.560+0000 7feb1effd640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 0x7feb00067f20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feb08009a40 tx=0x7feb08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:00.559 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.560+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7feb18051aa0 con 0x7feb2015ad30 2023-11-06T03:20:00.699 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.704+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb200d0000 con 0x7feb00065a60 2023-11-06T03:20:00.710 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb17fff640 1 -- 172.21.15.157:0/2742524077 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71805 (secure 0 0 0) 0x7feb08010930 con 0x7feb00065a60 2023-11-06T03:20:00.711 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:20:00.711 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:20:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 msgr2=0x7feb00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 0x7feb00067f20 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feb08009a40 tx=0x7feb08008040 comp rx=0 tx=0).stop 2023-11-06T03:20:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 msgr2=0x7feb200c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7feb1800eed0 tx=0x7feb1800b480 comp rx=0 tx=0).stop 2023-11-06T03:20:00.713 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 shutdown_connections 2023-11-06T03:20:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7feb00065a60 0x7feb00067f20 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb2015ad30 0x7feb200c7cc0 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.712+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7feb200b2d50 0x7feb20158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.716+0000 7feb2592e640 1 --2- 172.21.15.157:0/2742524077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb200b23b0 0x7feb20142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:00.714 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.716+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 >> 172.21.15.157:0/2742524077 conn(0x7feb20065d30 msgr2=0x7feb200cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:00.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.716+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 shutdown_connections 2023-11-06T03:20:00.715 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:00.716+0000 7feb2592e640 1 -- 172.21.15.157:0/2742524077 wait complete. 2023-11-06T03:20:00.725 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":671,"stamp":"2023-11-06T03:19:59.440361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":684252,"kb_used_data":20380,"kb_used_omap":0,"kb_used_meta":663808,"kb_avail":754290468,"statfs":{"total":773094113280,"available":772393439232,"internally_reserved":0,"allocated":20869120,"data_stored":10352703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":679739392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.711755"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.111770+0000","last_change":"2023-11-06T03:03:56.648502+0000","last_active":"2023-11-06T03:19:52.111770+0000","last_peered":"2023-11-06T03:19:52.111770+0000","last_clean":"2023-11-06T03:19:52.111770+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:19:52.111770+0000","last_undegraded":"2023-11-06T03:19:52.111770+0000","last_fullsized":"2023-11-06T03:19:52.111770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:03:55.173118+0000","last_clean_scrub_stamp":"2023-11-06T03:03:55.173118+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T05:56:19.973721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.734061+0000","last_change":"2023-11-06T03:04:02.279372+0000","last_active":"2023-11-06T03:19:50.734061+0000","last_peered":"2023-11-06T03:19:50.734061+0000","last_clean":"2023-11-06T03:19:50.734061+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:19:50.734061+0000","last_undegraded":"2023-11-06T03:19:50.734061+0000","last_fullsized":"2023-11-06T03:19:50.734061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.111529+0000","last_change":"2023-11-06T03:04:02.277944+0000","last_active":"2023-11-06T03:19:52.111529+0000","last_peered":"2023-11-06T03:19:52.111529+0000","last_clean":"2023-11-06T03:19:52.111529+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:19:52.111529+0000","last_undegraded":"2023-11-06T03:19:52.111529+0000","last_fullsized":"2023-11-06T03:19:52.111529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T09:58:42.869684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:19:51.836979+0000","last_change":"2023-11-06T03:04:02.651276+0000","last_active":"2023-11-06T03:19:51.836979+0000","last_peered":"2023-11-06T03:19:51.836979+0000","last_clean":"2023-11-06T03:19:51.836979+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:19:51.836979+0000","last_undegraded":"2023-11-06T03:19:51.836979+0000","last_fullsized":"2023-11-06T03:19:51.836979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:50.733976+0000","last_change":"2023-11-06T03:04:02.278596+0000","last_active":"2023-11-06T03:19:50.733976+0000","last_peered":"2023-11-06T03:19:50.733976+0000","last_clean":"2023-11-06T03:19:50.733976+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:19:50.733976+0000","last_undegraded":"2023-11-06T03:19:50.733976+0000","last_fullsized":"2023-11-06T03:19:50.733976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:46:44.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.363776+0000","last_change":"2023-11-06T03:04:02.621651+0000","last_active":"2023-11-06T03:19:52.363776+0000","last_peered":"2023-11-06T03:19:52.363776+0000","last_clean":"2023-11-06T03:19:52.363776+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:19:52.363776+0000","last_undegraded":"2023-11-06T03:19:52.363776+0000","last_fullsized":"2023-11-06T03:19:52.363776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.114708+0000","last_change":"2023-11-06T03:04:02.622102+0000","last_active":"2023-11-06T03:19:52.114708+0000","last_peered":"2023-11-06T03:19:52.114708+0000","last_clean":"2023-11-06T03:19:52.114708+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:19:52.114708+0000","last_undegraded":"2023-11-06T03:19:52.114708+0000","last_fullsized":"2023-11-06T03:19:52.114708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T03:56:29.434550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:19:58.591039+0000","last_change":"2023-11-06T03:19:58.591252+0000","last_active":"2023-11-06T03:19:58.591039+0000","last_peered":"2023-11-06T03:19:58.591039+0000","last_clean":"2023-11-06T03:19:58.591039+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:19:58.591039+0000","last_undegraded":"2023-11-06T03:19:58.591039+0000","last_fullsized":"2023-11-06T03:19:58.591039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:19:58.590992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:19:58.590992+0000","last_clean_scrub_stamp":"2023-11-06T03:19:58.590992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:29:36.799129+0000","scrub_duration":0.071211831000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-11-06T03:19:52.117900+0000","last_change":"2023-11-06T03:04:02.277833+0000","last_active":"2023-11-06T03:19:52.117900+0000","last_peered":"2023-11-06T03:19:52.117900+0000","last_clean":"2023-11-06T03:19:52.117900+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:19:52.117900+0000","last_undegraded":"2023-11-06T03:19:52.117900+0000","last_fullsized":"2023-11-06T03:19:52.117900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:04:01.248587+0000","last_clean_scrub_stamp":"2023-11-06T03:04:01.248587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.392461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96304,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94275536,"statfs":{"total":96636764160,"available":96538148864,"internally_reserved":0,"allocated":3055616,"data_stored":1737246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89200000000000002,"15min":0.82499999999999996},"min":{"1min":0.78500000000000003,"5min":0.45800000000000002,"15min":0.34999999999999998},"max":{"1min":1.161,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.67400000000000004,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.61599999999999999,"5min":0.501,"15min":0.45400000000000001},"max":{"1min":1.0780000000000001,"5min":1.2310000000000001,"15min":1.627},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.61499999999999999,"5min":0.52200000000000002,"15min":0.44500000000000001},"max":{"1min":1.2470000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.2210000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.94699999999999995,"15min":0.90500000000000003},"min":{"1min":0.71799999999999997,"5min":0.625,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":2.246,"15min":2.246},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.48999999999999999},"max":{"1min":1.26,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.66300000000000003,"5min":0.55900000000000005,"15min":0.438},"max":{"1min":1.143,"5min":1.3680000000000001,"15min":2.512},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.032},"min":{"1min":0.81999999999999995,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.1919999999999999,"5min":1.9770000000000001,"15min":10.596},"last":1.141}]},{"osd":4,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":0.95099999999999996},"min":{"1min":0.76700000000000002,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3500000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.069},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.96299999999999997},"min":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.411,"15min":1.9399999999999999},"last":0.97299999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.04},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.54000000000000004},"max":{"1min":1.325,"5min":2.129,"15min":2.129},"last":1.369},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.82099999999999995,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.5780000000000001,"15min":1.8839999999999999},"last":1.1739999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.80200000000000005,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.5720000000000001,"5min":1.597,"15min":9.6159999999999997},"last":1.244},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0349999999999999,"15min":1.0669999999999999},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.173,"5min":1.53,"15min":2.5430000000000001},"last":1.268}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.40999999999999998},"max":{"1min":1.0920000000000001,"5min":1.2310000000000001,"15min":16.073},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88700000000000001,"15min":0.99199999999999999},"min":{"1min":0.63300000000000001,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":1.1579999999999999,"5min":1.581,"15min":40.905000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.872,"15min":0.91800000000000004},"min":{"1min":0.627,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.159,"5min":1.2230000000000001,"15min":16.012},"last":1.022},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.58999999999999997,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.306,"5min":1.5169999999999999,"15min":3.3050000000000002},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.93799999999999994},"min":{"1min":0.69799999999999995,"5min":0.49199999999999999,"15min":0.48799999999999999},"max":{"1min":1.2010000000000001,"5min":1.3460000000000001,"15min":27.803999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.93500000000000005},"min":{"1min":0.66400000000000003,"5min":0.55800000000000005,"15min":0.50700000000000001},"max":{"1min":1.1799999999999999,"5min":1.377,"15min":2.843},"last":0.95099999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94299999999999995,"15min":1.1679999999999999},"min":{"1min":0.70399999999999996,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.2370000000000001,"5min":1.389,"15min":63.737000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90900000000000003,"15min":1.0640000000000001},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.171,"5min":1.716,"15min":41.079999999999998},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":1.119},"min":{"1min":0.63500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3,"5min":1.512,"15min":24.867999999999999},"last":1.077},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1020000000000001,"15min":1.208},"min":{"1min":0.89800000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.6140000000000001,"15min":23.789000000000001},"last":0.88500000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96299999999999997,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.694,"15min":40.959000000000003},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.79500000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.375,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.1259999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0920000000000001,"15min":1.2090000000000001},"min":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.61299999999999999},"max":{"1min":1.409,"5min":1.5309999999999999,"15min":23.748999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.2130000000000001},"min":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.56499999999999995},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":28.164000000000001},"last":0.98899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":1.1339999999999999},"min":{"1min":0.59099999999999997,"5min":0.46300000000000002,"15min":0.309},"max":{"1min":1.0720000000000001,"5min":1.1950000000000001,"15min":125.256},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.90700000000000003,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.35499999999999998},"max":{"1min":1.274,"5min":4.7050000000000001,"15min":126.98699999999999},"last":0.81200000000000006}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.78300000000000003,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.1970000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":1.004},"min":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.6459999999999999,"15min":43.799999999999997},"last":0.91700000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90800000000000003,"15min":1.0349999999999999},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.3959999999999999,"15min":43.741},"last":0.626},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.877,"15min":0.85699999999999998},"min":{"1min":0.65700000000000003,"5min":0.51200000000000001,"15min":0.39200000000000002},"max":{"1min":1.0880000000000001,"5min":1.2470000000000001,"15min":2.9700000000000002},"last":1.1240000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":0.92500000000000004},"min":{"1min":0.76000000000000001,"5min":0.61699999999999999,"15min":0.40400000000000003},"max":{"1min":1.216,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.038,"5min":1.012,"15min":1.1319999999999999},"min":{"1min":0.81299999999999994,"5min":0.57599999999999996,"15min":0.45300000000000001},"max":{"1min":1.179,"5min":1.796,"15min":43.841999999999999},"last":0.84799999999999998}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.89300000000000002,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.29299999999999998},"max":{"1min":1.196,"5min":1.595,"15min":3.1349999999999998},"last":1.038},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0289999999999999,"15min":1.036},"min":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.524,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.038,"15min":1.034},"min":{"1min":0.64200000000000002,"5min":0.502,"15min":0.378},"max":{"1min":1.1930000000000001,"5min":1.72,"15min":2.6909999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.024,"15min":1.0469999999999999},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.413,"15min":5.6109999999999998},"last":1.196}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.105},"min":{"1min":0.68500000000000005,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.169,"15min":1.1359999999999999},"min":{"1min":0.82399999999999995,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.3069999999999999,"5min":3.738,"15min":3.738},"last":0.998}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113772,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258068,"statfs":{"total":96636764160,"available":96520261632,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.93500000000000005,"15min":0.99299999999999999},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":42.853999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.437,"15min":0.308},"max":{"1min":1.143,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.93300000000000005}]},{"osd":1,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":0.94399999999999995,"15min":0.83899999999999997},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":11.412000000000001,"5min":11.412000000000001,"15min":11.412000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.0169999999999999,"15min":0.91000000000000003},"min":{"1min":0.63700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.5530000000000008,"5min":9.5530000000000008,"15min":9.5530000000000008},"last":0.871}]},{"osd":2,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.042,"15min":0.94899999999999995},"min":{"1min":0.77900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.6280000000000001,"5min":9.6280000000000001,"15min":9.6280000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90800000000000003,"15min":0.88},"min":{"1min":0.78700000000000003,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":3.2639999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.56899999999999995,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":3.198},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.028,"15min":0.94099999999999995},"min":{"1min":0.72199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.507,"5min":11.507,"15min":11.507},"last":0.89900000000000002}]},{"osd":5,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.4780000000000002,"5min":1.2949999999999999,"15min":1.121},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":24.943999999999999,"5min":24.943999999999999,"15min":24.943999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":2.3570000000000002,"5min":1.3080000000000001,"15min":1.1499999999999999},"min":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":21.824000000000002,"5min":21.824000000000002,"15min":21.824000000000002},"last":1.105}]},{"osd":6,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":3.7890000000000001,"5min":1.502,"15min":1.153},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":47.070999999999998,"5min":47.070999999999998,"15min":47.070999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":2.8940000000000001,"5min":1.4419999999999999,"15min":1.1950000000000001},"min":{"1min":0.70799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":31.748999999999999,"5min":31.748999999999999,"15min":31.748999999999999},"last":1.0760000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.6230000000000002,"5min":1.395,"15min":1.1839999999999999},"min":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.56599999999999995},"max":{"1min":26.669,"5min":26.669,"15min":26.669},"last":1.1539999999999999},{"interface":"front","average":{"1min":2.6890000000000001,"5min":1.403,"15min":1.252},"min":{"1min":0.63100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":27.440000000000001,"5min":27.440000000000001,"15min":27.440000000000001},"last":1.0089999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575046,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87592,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94284248,"statfs":{"total":96636764160,"available":96547069952,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.23000000000000001,"15min":0.17399999999999999},"max":{"1min":1.0029999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.61599999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":1.1120000000000001,"5min":5.0670000000000002,"15min":5.0670000000000002},"last":0.66200000000000003}]},{"osd":1,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.224},"max":{"1min":1.0309999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.32100000000000001,"15min":0.20999999999999999},"max":{"1min":1.022,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.66300000000000003,"5min":0.34699999999999998,"15min":0.23300000000000001},"max":{"1min":1.7629999999999999,"5min":3.855,"15min":3.855},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.80100000000000005,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.318,"15min":0.255},"max":{"1min":1.4359999999999999,"5min":3.645,"15min":3.645},"last":0.71199999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.51000000000000001,"15min":0.40999999999999998},"max":{"1min":1.089,"5min":5.109,"15min":5.109},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0369999999999999,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.45800000000000002,"15min":0.43099999999999999},"max":{"1min":1.1639999999999999,"5min":14.170999999999999,"15min":14.170999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98999999999999999,"15min":0.96199999999999997},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.35999999999999999},"max":{"1min":1.2110000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.117}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.69399999999999995,"5min":0.629,"15min":0.39300000000000002},"max":{"1min":1.3089999999999999,"5min":3.617,"15min":3.617},"last":1.198},{"interface":"front","average":{"1min":1.038,"5min":0.98899999999999999,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.5,"15min":0.47899999999999998},"max":{"1min":1.2589999999999999,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.91400000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0549999999999999,"15min":1.0089999999999999},"min":{"1min":0.71199999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"max":{"1min":1.3109999999999999,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0840000000000001,"15min":1.0409999999999999},"min":{"1min":0.872,"5min":0.56200000000000006,"15min":0.48299999999999998},"max":{"1min":1.5189999999999999,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":1.087}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.17399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.7270000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.215,"15min":0.191},"max":{"1min":1.375,"5min":1.375,"15min":1.4590000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.23000000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64600000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.47799999999999998}]},{"osd":3,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72999999999999998,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.5},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.4199999999999999},"last":0.39300000000000002}]},{"osd":4,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.86899999999999999,"15min":0.84699999999999998},"min":{"1min":0.76900000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.526},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.997,"5min":0.91600000000000004,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"max":{"1min":1.2889999999999999,"5min":1.343,"15min":1.5229999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.44500000000000001},"max":{"1min":1.1140000000000001,"5min":1.173,"15min":1.6200000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.046,"5min":0.92200000000000004,"15min":0.90900000000000003},"min":{"1min":0.71899999999999997,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5780000000000001},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.98099999999999998,"15min":0.96899999999999997},"min":{"1min":0.753,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.615},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.99099999999999999,"15min":0.96499999999999997},"min":{"1min":0.80300000000000005,"5min":0.51400000000000001,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":1.532,"15min":4.2809999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.91700000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":3.4740000000000002},"last":0.88}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.81200000000000006,"5min":1.042,"15min":1.843},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.0669999999999999,"5min":1.244,"15min":2.1150000000000002},"last":0.434}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":30.861000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.20899999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.7330000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.316,"15min":0.24199999999999999},"max":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.34699999999999998,"5min":0.30599999999999999,"15min":0.20799999999999999},"max":{"1min":1.0169999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.63100000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.3029999999999999,"15min":2.258},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89100000000000001,"15min":0.91800000000000004},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.41699999999999998},"max":{"1min":1.2,"5min":1.329,"15min":14.753},"last":0.98099999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":43.103999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0109999999999999,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.39100000000000001},"max":{"1min":1.3420000000000001,"5min":1.468,"15min":44.753999999999998},"last":1.0049999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.94399999999999995,"15min":1.004},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.45100000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":23.971},"last":1.032},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96499999999999997,"15min":1.016},"min":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.42699999999999999},"max":{"1min":1.306,"5min":1.3979999999999999,"15min":14.813000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.034,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.2609999999999999,"5min":1.55,"15min":2.2799999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.016,"15min":0.999},"min":{"1min":0.69199999999999995,"5min":0.58099999999999996,"15min":0.52400000000000002},"max":{"1min":1.4750000000000001,"5min":1.835,"15min":2.3559999999999999},"last":1.1080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100460,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":97472,"kb_avail":94271380,"statfs":{"total":96636764160,"available":96533893120,"internally_reserved":0,"allocated":3051520,"data_stored":1736227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":99811328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.996,"5min":1.214,"15min":2.6989999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19800000000000001},"max":{"1min":0.88600000000000001,"5min":1.45,"15min":7.1230000000000002},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.63800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.627},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.23100000000000001},"max":{"1min":0.81399999999999995,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.054,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.52000000000000002,"15min":0.41499999999999998},"max":{"1min":1.151,"5min":1.419,"15min":4.641},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42899999999999999},"max":{"1min":1.1120000000000001,"5min":1.4159999999999999,"15min":2.028},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98799999999999999,"15min":0.95499999999999996},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":1.399,"5min":2.145,"15min":2.8559999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98899999999999999,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.432},"max":{"1min":1.2949999999999999,"5min":1.623,"15min":4.7430000000000003},"last":0.79700000000000004}]},{"osd":6,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96799999999999997,"15min":0.92100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44800000000000001},"max":{"1min":1.214,"5min":1.4359999999999999,"15min":2.931},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.016,"15min":1},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.4590000000000001,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":0.76200000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0249999999999999,"15min":0.998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.495},"max":{"1min":1.24,"5min":1.544,"15min":4.8339999999999996},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":0.97899999999999998},"min":{"1min":0.66600000000000004,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2829999999999999,"5min":1.502,"15min":1.97},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:20:00.730 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-06T03:03:55.173118+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=3, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.730 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.730 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.731 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.731 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.731 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.731 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.732 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-06T03:04:01.248587+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=4, tm_sec=1, tm_wday=0, tm_yday=310, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=6, tm_hour=3, tm_min=19, tm_sec=55, tm_wday=0, tm_yday=310, tm_isdst=0) 2023-11-06T03:20:00.732 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-06T03:20:13.125 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:13.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:20.733 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-06T03:20:20.869 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.872+0000 7fc6c6f3d640 1 Processor -- start 2023-11-06T03:20:20.870 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c6f3d640 1 -- start start 2023-11-06T03:20:20.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c6f3d640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6c015caa0 0x7fc6c00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:20.871 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c6f3d640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00d2920 con 0x7fc6c00d0280 2023-11-06T03:20:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c6f3d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00d18a0 con 0x7fc6c015ce80 2023-11-06T03:20:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c6f3d640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00d1b60 con 0x7fc6c015caa0 2023-11-06T03:20:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c4cd9640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc6c015caa0 0x7fc6c00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:20.872 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c54da640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc6c00d0280 0x7fc6c00b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45510/0 (socket says 172.21.15.157:45510) 2023-11-06T03:20:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c54da640 1 -- 172.21.15.157:0/1112138103 learned_addr learned my addr 172.21.15.157:0/1112138103 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:20:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6bf7fe640 1 -- 172.21.15.157:0/1112138103 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775660857 0 0) 0x7fc6c00d2920 con 0x7fc6c00d0280 2023-11-06T03:20:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6bf7fe640 1 -- 172.21.15.157:0/1112138103 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b0003530 con 0x7fc6c00d0280 2023-11-06T03:20:20.873 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/1112138103 >> v1:172.21.15.195:6789/0 conn(0x7fc6c015ce80 legacy=0x7fc6c00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/1112138103 >> v1:172.21.15.157:6789/0 conn(0x7fc6c00d0280 legacy=0x7fc6c00b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:20.874 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.876+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/1112138103 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00d1e40 con 0x7fc6c015caa0 2023-11-06T03:20:20.875 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 --2- 172.21.15.157:0/1112138103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c015caa0 0x7fc6c00cdc60 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40097c0 tx=0x7fc6b401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbd5a23b1defa37 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6bf7fe640 1 -- 172.21.15.157:0/1112138103 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40ab650 con 0x7fc6c015caa0 2023-11-06T03:20:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6bf7fe640 1 -- 172.21.15.157:0/1112138103 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40ab7b0 con 0x7fc6c015caa0 2023-11-06T03:20:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6bf7fe640 1 -- 172.21.15.157:0/1112138103 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40abb40 con 0x7fc6c015caa0 2023-11-06T03:20:20.876 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/1112138103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c015caa0 msgr2=0x7fc6c00cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/1112138103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c015caa0 0x7fc6c00cdc60 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40097c0 tx=0x7fc6b401c450 comp rx=0 tx=0).stop 2023-11-06T03:20:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/1112138103 shutdown_connections 2023-11-06T03:20:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/1112138103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c015caa0 0x7fc6c00cdc60 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/1112138103 >> 172.21.15.157:0/1112138103 conn(0x7fc6c0070790 msgr2=0x7fc6c0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:20.877 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/1112138103 shutdown_connections 2023-11-06T03:20:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/1112138103 wait complete. 2023-11-06T03:20:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 Processor -- start 2023-11-06T03:20:20.878 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- start start 2023-11-06T03:20:20.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:20.881 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 0x7fc6c00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:20.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c015ce80 0x7fc6c00b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:20.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0159000 con 0x7fc6c00d0280 2023-11-06T03:20:20.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00ca230 con 0x7fc6c015ce80 2023-11-06T03:20:20.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c6f3d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015f730 con 0x7fc6c015caa0 2023-11-06T03:20:20.882 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:20.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47904/0 (socket says 172.21.15.157:47904) 2023-11-06T03:20:20.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/2954662804 learned_addr learned my addr 172.21.15.157:0/2954662804 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:20.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c54da640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c015ce80 0x7fc6c00b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:20.883 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6bffff640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 0x7fc6c00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 msgr2=0x7fc6c00c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 0x7fc6c00c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c015ce80 msgr2=0x7fc6c00b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c015ce80 0x7fc6c00b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6c4cd9640 1 -- 172.21.15.157:0/2954662804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4009420 con 0x7fc6c00d0280 2023-11-06T03:20:20.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.880+0000 7fc6bffff640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 0x7fc6c00c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:20.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6c4cd9640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40ad040 tx=0x7fc6b401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:20.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40bf610 con 0x7fc6c00d0280 2023-11-06T03:20:20.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40bf790 con 0x7fc6c00d0280 2023-11-06T03:20:20.885 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c015fa10 con 0x7fc6c00d0280 2023-11-06T03:20:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40bfb40 con 0x7fc6c00d0280 2023-11-06T03:20:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c015ffd0 con 0x7fc6c00d0280 2023-11-06T03:20:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0072750 con 0x7fc6c00d0280 2023-11-06T03:20:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc6b40b5400 con 0x7fc6c00d0280 2023-11-06T03:20:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 0x7fc6a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc6b411e0e0 con 0x7fc6c00d0280 2023-11-06T03:20:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bffff640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 0x7fc6a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.884+0000 7fc6bffff640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 0x7fc6a4067f20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc6c015ca70 tx=0x7fc6ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:20.888 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:20.888+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc6b40ebbe0 con 0x7fc6c00d0280 2023-11-06T03:20:21.034 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.040+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 --> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6c00b1f00 con 0x7fc6a4065a60 2023-11-06T03:20:21.042 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6bd7fa640 1 -- 172.21.15.157:0/2954662804 <== mgr.4105 v2:172.21.15.195:6832/38105 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71914 (secure 0 0 0) 0x7fc6ac001cc0 con 0x7fc6a4065a60 2023-11-06T03:20:21.043 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:20:21.044 INFO:teuthology.orchestra.run.smithi157.stderr:dumped all 2023-11-06T03:20:21.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 msgr2=0x7fc6a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 0x7fc6a4067f20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc6c015ca70 tx=0x7fc6ac009040 comp rx=0 tx=0).stop 2023-11-06T03:20:21.045 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 msgr2=0x7fc6c0156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40ad040 tx=0x7fc6b401ac90 comp rx=0 tx=0).stop 2023-11-06T03:20:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 shutdown_connections 2023-11-06T03:20:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc6a4065a60 0x7fc6a4067f20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6c015ce80 0x7fc6c00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.046 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc6c015caa0 0x7fc6c00c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 --2- 172.21.15.157:0/2954662804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6c00d0280 0x7fc6c0156150 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 >> 172.21.15.157:0/2954662804 conn(0x7fc6c0070790 msgr2=0x7fc6c00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 shutdown_connections 2023-11-06T03:20:21.047 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.048+0000 7fc6c6f3d640 1 -- 172.21.15.157:0/2954662804 wait complete. 2023-11-06T03:20:21.061 INFO:teuthology.orchestra.run.smithi157.stdout:{"pg_ready":true,"pg_map":{"version":681,"stamp":"2023-11-06T03:20:19.445774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":684252,"kb_used_data":20380,"kb_used_omap":0,"kb_used_meta":663808,"kb_avail":754290468,"statfs":{"total":773094113280,"available":772393439232,"internally_reserved":0,"allocated":20869120,"data_stored":10352703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":679739392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003218"},"pg_stats":[{"pgid":"1.0","version":"195'119","reported_seq":327,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:00.920151+0000","last_change":"2023-11-06T03:20:00.920151+0000","last_active":"2023-11-06T03:20:00.920151+0000","last_peered":"2023-11-06T03:20:00.920151+0000","last_clean":"2023-11-06T03:20:00.920151+0000","last_became_active":"2023-11-06T03:03:56.646630+0000","last_became_peered":"2023-11-06T03:03:56.646630+0000","last_unstale":"2023-11-06T03:20:00.920151+0000","last_undegraded":"2023-11-06T03:20:00.920151+0000","last_fullsized":"2023-11-06T03:20:00.920151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2023-11-06T03:20:00.919846+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2023-11-06T03:20:00.919846+0000","last_clean_scrub_stamp":"2023-11-06T03:20:00.919846+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T10:32:30.786890+0000","scrub_duration":0.061930616000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:03.126370+0000","last_change":"2023-11-06T03:20:03.126679+0000","last_active":"2023-11-06T03:20:03.126370+0000","last_peered":"2023-11-06T03:20:03.126370+0000","last_clean":"2023-11-06T03:20:03.126370+0000","last_became_active":"2023-11-06T03:04:02.276838+0000","last_became_peered":"2023-11-06T03:04:02.276838+0000","last_unstale":"2023-11-06T03:20:03.126370+0000","last_undegraded":"2023-11-06T03:20:03.126370+0000","last_fullsized":"2023-11-06T03:20:03.126370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:03.126323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:03.126323+0000","last_clean_scrub_stamp":"2023-11-06T03:20:03.126323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T11:53:55.165218+0000","scrub_duration":0.099519562000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:02.907646+0000","last_change":"2023-11-06T03:20:02.907646+0000","last_active":"2023-11-06T03:20:02.907646+0000","last_peered":"2023-11-06T03:20:02.907646+0000","last_clean":"2023-11-06T03:20:02.907646+0000","last_became_active":"2023-11-06T03:04:02.277111+0000","last_became_peered":"2023-11-06T03:04:02.277111+0000","last_unstale":"2023-11-06T03:20:02.907646+0000","last_undegraded":"2023-11-06T03:20:02.907646+0000","last_fullsized":"2023-11-06T03:20:02.907646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:02.907270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:02.907270+0000","last_clean_scrub_stamp":"2023-11-06T03:20:02.907270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:21.388084+0000","scrub_duration":0.067009883000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:00.170122+0000","last_change":"2023-11-06T03:20:00.170238+0000","last_active":"2023-11-06T03:20:00.170122+0000","last_peered":"2023-11-06T03:20:00.170122+0000","last_clean":"2023-11-06T03:20:00.170122+0000","last_became_active":"2023-11-06T03:04:02.650911+0000","last_became_peered":"2023-11-06T03:04:02.650911+0000","last_unstale":"2023-11-06T03:20:00.170122+0000","last_undegraded":"2023-11-06T03:20:00.170122+0000","last_fullsized":"2023-11-06T03:20:00.170122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:00.170105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:00.170105+0000","last_clean_scrub_stamp":"2023-11-06T03:20:00.170105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:22:39.684502+0000","scrub_duration":0.040516767000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:01.087355+0000","last_change":"2023-11-06T03:20:01.087509+0000","last_active":"2023-11-06T03:20:01.087355+0000","last_peered":"2023-11-06T03:20:01.087355+0000","last_clean":"2023-11-06T03:20:01.087355+0000","last_became_active":"2023-11-06T03:04:02.277271+0000","last_became_peered":"2023-11-06T03:04:02.277271+0000","last_unstale":"2023-11-06T03:20:01.087355+0000","last_undegraded":"2023-11-06T03:20:01.087355+0000","last_fullsized":"2023-11-06T03:20:01.087355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:01.087327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:01.087327+0000","last_clean_scrub_stamp":"2023-11-06T03:20:01.087327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T13:48:19.568141+0000","scrub_duration":0.100811447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":177,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:00.581781+0000","last_change":"2023-11-06T03:20:00.581959+0000","last_active":"2023-11-06T03:20:00.581781+0000","last_peered":"2023-11-06T03:20:00.581781+0000","last_clean":"2023-11-06T03:20:00.581781+0000","last_became_active":"2023-11-06T03:04:02.621210+0000","last_became_peered":"2023-11-06T03:04:02.621210+0000","last_unstale":"2023-11-06T03:20:00.581781+0000","last_undegraded":"2023-11-06T03:20:00.581781+0000","last_fullsized":"2023-11-06T03:20:00.581781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:00.581744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:00.581744+0000","last_clean_scrub_stamp":"2023-11-06T03:20:00.581744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T14:12:30.157142+0000","scrub_duration":0.052370381000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":171,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:05.940915+0000","last_change":"2023-11-06T03:20:05.941214+0000","last_active":"2023-11-06T03:20:05.940915+0000","last_peered":"2023-11-06T03:20:05.940915+0000","last_clean":"2023-11-06T03:20:05.940915+0000","last_became_active":"2023-11-06T03:04:02.621442+0000","last_became_peered":"2023-11-06T03:04:02.621442+0000","last_unstale":"2023-11-06T03:20:05.940915+0000","last_undegraded":"2023-11-06T03:20:05.940915+0000","last_fullsized":"2023-11-06T03:20:05.940915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:05.940861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:05.940861+0000","last_clean_scrub_stamp":"2023-11-06T03:20:05.940861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:01:16.518394+0000","scrub_duration":0.102103377,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:19:58.591039+0000","last_change":"2023-11-06T03:19:58.591252+0000","last_active":"2023-11-06T03:19:58.591039+0000","last_peered":"2023-11-06T03:19:58.591039+0000","last_clean":"2023-11-06T03:19:58.591039+0000","last_became_active":"2023-11-06T03:04:02.275475+0000","last_became_peered":"2023-11-06T03:04:02.275475+0000","last_unstale":"2023-11-06T03:19:58.591039+0000","last_undegraded":"2023-11-06T03:19:58.591039+0000","last_fullsized":"2023-11-06T03:19:58.591039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:19:58.590992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:19:58.590992+0000","last_clean_scrub_stamp":"2023-11-06T03:19:58.590992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T04:29:36.799129+0000","scrub_duration":0.071211831000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":168,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-06T03:20:03.545891+0000","last_change":"2023-11-06T03:20:03.546163+0000","last_active":"2023-11-06T03:20:03.545891+0000","last_peered":"2023-11-06T03:20:03.545891+0000","last_clean":"2023-11-06T03:20:03.545891+0000","last_became_active":"2023-11-06T03:04:02.277069+0000","last_became_peered":"2023-11-06T03:04:02.277069+0000","last_unstale":"2023-11-06T03:20:03.545891+0000","last_undegraded":"2023-11-06T03:20:03.545891+0000","last_fullsized":"2023-11-06T03:20:03.545891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-06T03:20:03.545844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-06T03:20:03.545844+0000","last_clean_scrub_stamp":"2023-11-06T03:20:03.545844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-07T07:22:43.060240+0000","scrub_duration":0.100686067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96304,"kb_used_data":2984,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94275536,"statfs":{"total":96636764160,"available":96538148864,"internally_reserved":0,"allocated":3055616,"data_stored":1737246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.89200000000000002,"15min":0.82499999999999996},"min":{"1min":0.78500000000000003,"5min":0.45800000000000002,"15min":0.34999999999999998},"max":{"1min":1.161,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.67400000000000004,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.879}]},{"osd":1,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.84999999999999998,"15min":0.82799999999999996},"min":{"1min":0.61599999999999999,"5min":0.501,"15min":0.45400000000000001},"max":{"1min":1.0780000000000001,"5min":1.2310000000000001,"15min":1.627},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.90000000000000002,"15min":0.878},"min":{"1min":0.61499999999999999,"5min":0.52200000000000002,"15min":0.44500000000000001},"max":{"1min":1.2470000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.69799999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.94699999999999995,"15min":0.90500000000000003},"min":{"1min":0.71799999999999997,"5min":0.625,"15min":0.46500000000000002},"max":{"1min":1.2,"5min":2.246,"15min":2.246},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.48999999999999999},"max":{"1min":1.26,"5min":1.4359999999999999,"15min":1.6259999999999999},"last":1.024}]},{"osd":3,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.66300000000000003,"5min":0.55900000000000005,"15min":0.438},"max":{"1min":1.143,"5min":1.3680000000000001,"15min":2.512},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.05,"5min":1.0089999999999999,"15min":1.032},"min":{"1min":0.81999999999999995,"5min":0.628,"15min":0.60099999999999998},"max":{"1min":1.1919999999999999,"5min":1.9770000000000001,"15min":10.596},"last":1.3460000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":0.95099999999999996},"min":{"1min":0.76700000000000002,"5min":0.58399999999999996,"15min":0.499},"max":{"1min":1.3500000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.96299999999999997},"min":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.411,"15min":1.9399999999999999},"last":0.99299999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.04},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.54000000000000004},"max":{"1min":1.325,"5min":2.129,"15min":2.129},"last":1.258},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.05,"15min":1.0620000000000001},"min":{"1min":0.82099999999999995,"5min":0.66000000000000003,"15min":0.55200000000000005},"max":{"1min":1.5009999999999999,"5min":1.5780000000000001,"15min":1.8839999999999999},"last":1.2090000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.80200000000000005,"5min":0.75,"15min":0.60799999999999998},"max":{"1min":1.5720000000000001,"5min":1.597,"15min":9.6159999999999997},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0349999999999999,"15min":1.0669999999999999},"min":{"1min":0.75600000000000001,"5min":0.67900000000000005,"15min":0.53300000000000003},"max":{"1min":1.173,"5min":1.53,"15min":2.5430000000000001},"last":1.0549999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575048,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69480,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302360,"statfs":{"total":96636764160,"available":96565616640,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.80400000000000005,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.441,"15min":0.40999999999999998},"max":{"1min":1.0920000000000001,"5min":1.2310000000000001,"15min":16.073},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88700000000000001,"15min":0.99199999999999999},"min":{"1min":0.63300000000000001,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":1.1579999999999999,"5min":1.581,"15min":40.905000000000001},"last":0.87}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.872,"15min":0.91800000000000004},"min":{"1min":0.627,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.159,"5min":1.2230000000000001,"15min":16.012},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.58999999999999997,"5min":0.47999999999999998,"15min":0.45600000000000002},"max":{"1min":1.306,"5min":1.5169999999999999,"15min":3.3050000000000002},"last":0.96199999999999997}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.93799999999999994},"min":{"1min":0.69799999999999995,"5min":0.49199999999999999,"15min":0.48799999999999999},"max":{"1min":1.2010000000000001,"5min":1.3460000000000001,"15min":27.803999999999998},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.93500000000000005},"min":{"1min":0.66400000000000003,"5min":0.55800000000000005,"15min":0.50700000000000001},"max":{"1min":1.1799999999999999,"5min":1.377,"15min":2.843},"last":1.0640000000000001}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.94299999999999995,"15min":1.1679999999999999},"min":{"1min":0.70399999999999996,"5min":0.59199999999999997,"15min":0.48699999999999999},"max":{"1min":1.2370000000000001,"5min":1.389,"15min":63.737000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90900000000000003,"15min":1.0640000000000001},"min":{"1min":0.55000000000000004,"5min":0.52000000000000002,"15min":0.46300000000000002},"max":{"1min":1.171,"5min":1.716,"15min":41.079999999999998},"last":1.004}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97899999999999998,"15min":1.119},"min":{"1min":0.63500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3,"5min":1.512,"15min":24.867999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1020000000000001,"15min":1.208},"min":{"1min":0.89800000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.3899999999999999,"5min":1.6140000000000001,"15min":23.789000000000001},"last":1.2110000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96299999999999997,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.61299999999999999,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.694,"15min":40.959000000000003},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.79500000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.375,"5min":1.8160000000000001,"15min":4.1159999999999997},"last":1.1479999999999999}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0920000000000001,"15min":1.2090000000000001},"min":{"1min":0.73199999999999998,"5min":0.71699999999999997,"15min":0.61299999999999999},"max":{"1min":1.409,"5min":1.5309999999999999,"15min":23.748999999999999},"last":1.099},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.095,"15min":1.2130000000000001},"min":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.56499999999999995},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":28.164000000000001},"last":1.288}]}]},{"osd":5,"up_from":13,"seq":55834575049,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":1.1339999999999999},"min":{"1min":0.59099999999999997,"5min":0.46300000000000002,"15min":0.309},"max":{"1min":1.0720000000000001,"5min":1.1950000000000001,"15min":125.256},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.90700000000000003,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.35499999999999998},"max":{"1min":1.274,"5min":4.7050000000000001,"15min":126.98699999999999},"last":1.0580000000000001}]},{"osd":1,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.878},"min":{"1min":0.78300000000000003,"5min":0.55200000000000005,"15min":0.28999999999999998},"max":{"1min":1.1970000000000001,"5min":4.8490000000000002,"15min":4.8490000000000002},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":1.004},"min":{"1min":0.54900000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.6459999999999999,"15min":43.799999999999997},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90800000000000003,"15min":1.0349999999999999},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.33400000000000002},"max":{"1min":1.1799999999999999,"5min":1.3959999999999999,"15min":43.741},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.877,"15min":0.85699999999999998},"min":{"1min":0.65700000000000003,"5min":0.51200000000000001,"15min":0.39200000000000002},"max":{"1min":1.0880000000000001,"5min":1.2470000000000001,"15min":2.9700000000000002},"last":0.81399999999999995}]},{"osd":3,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":0.92500000000000004},"min":{"1min":0.76000000000000001,"5min":0.61699999999999999,"15min":0.40400000000000003},"max":{"1min":1.216,"5min":4.7800000000000002,"15min":4.7800000000000002},"last":0.997},{"interface":"front","average":{"1min":1.038,"5min":1.012,"15min":1.1319999999999999},"min":{"1min":0.81299999999999994,"5min":0.57599999999999996,"15min":0.45300000000000001},"max":{"1min":1.179,"5min":1.796,"15min":43.841999999999999},"last":0.78400000000000003}]},{"osd":4,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.89300000000000002,"15min":0.90500000000000003},"min":{"1min":0.59899999999999998,"5min":0.57199999999999995,"15min":0.29299999999999998},"max":{"1min":1.196,"5min":1.595,"15min":3.1349999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0289999999999999,"15min":1.036},"min":{"1min":0.61199999999999999,"5min":0.48899999999999999,"15min":0.33300000000000002},"max":{"1min":1.524,"5min":4.6200000000000001,"15min":4.6200000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.038,"15min":1.034},"min":{"1min":0.64200000000000002,"5min":0.502,"15min":0.378},"max":{"1min":1.1930000000000001,"5min":1.72,"15min":2.6909999999999998},"last":0.876},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.024,"15min":1.0469999999999999},"min":{"1min":0.67100000000000004,"5min":0.65500000000000003,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.413,"15min":5.6109999999999998},"last":0.746}]},{"osd":7,"last update":"Mon Nov 6 03:19:27 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.105},"min":{"1min":0.68500000000000005,"5min":0.52500000000000002,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":4.9160000000000004,"15min":4.9160000000000004},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.169,"15min":1.1359999999999999},"min":{"1min":0.82399999999999995,"5min":0.65300000000000002,"15min":0.38700000000000001},"max":{"1min":1.3069999999999999,"5min":3.738,"15min":3.738},"last":1.0349999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575048,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":113772,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":111360,"kb_avail":94258068,"statfs":{"total":96636764160,"available":96520261632,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":114032640},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":0.93500000000000005,"15min":0.99299999999999999},"min":{"1min":0.66200000000000003,"5min":0.40200000000000002,"15min":0.372},"max":{"1min":9.3140000000000001,"5min":9.3140000000000001,"15min":42.853999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.437,"15min":0.308},"max":{"1min":1.143,"5min":1.4750000000000001,"15min":44.609000000000002},"last":0.91900000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":0.94399999999999995,"15min":0.83899999999999997},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.34799999999999998},"max":{"1min":11.412000000000001,"5min":11.412000000000001,"15min":11.412000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.0169999999999999,"15min":0.91000000000000003},"min":{"1min":0.63700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":9.5530000000000008,"5min":9.5530000000000008,"15min":9.5530000000000008},"last":0.81699999999999995}]},{"osd":2,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.042,"15min":0.94899999999999995},"min":{"1min":0.77900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":9.6280000000000001,"5min":9.6280000000000001,"15min":9.6280000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90800000000000003,"15min":0.88},"min":{"1min":0.78700000000000003,"5min":0.497,"15min":0.45900000000000002},"max":{"1min":1.1140000000000001,"5min":1.3660000000000001,"15min":3.2639999999999998},"last":1.137}]},{"osd":3,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.92900000000000005,"15min":0.91400000000000003},"min":{"1min":0.75800000000000001,"5min":0.56899999999999995,"15min":0.47599999999999998},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":3.198},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.028,"15min":0.94099999999999995},"min":{"1min":0.72199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":11.507,"5min":11.507,"15min":11.507},"last":0.88}]},{"osd":5,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.4780000000000002,"5min":1.2949999999999999,"15min":1.121},"min":{"1min":0.63900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":24.943999999999999,"5min":24.943999999999999,"15min":24.943999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":2.3570000000000002,"5min":1.3080000000000001,"15min":1.1499999999999999},"min":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.50700000000000001},"max":{"1min":21.824000000000002,"5min":21.824000000000002,"15min":21.824000000000002},"last":1.0660000000000001}]},{"osd":6,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":3.7890000000000001,"5min":1.502,"15min":1.153},"min":{"1min":0.68500000000000005,"5min":0.52400000000000002,"15min":0.40000000000000002},"max":{"1min":47.070999999999998,"5min":47.070999999999998,"15min":47.070999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":2.8940000000000001,"5min":1.4419999999999999,"15min":1.1950000000000001},"min":{"1min":0.70799999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":31.748999999999999,"5min":31.748999999999999,"15min":31.748999999999999},"last":1.101}]},{"osd":7,"last update":"Mon Nov 6 03:19:23 2023","interfaces":[{"interface":"back","average":{"1min":2.6230000000000002,"5min":1.395,"15min":1.1839999999999999},"min":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.56599999999999995},"max":{"1min":26.669,"5min":26.669,"15min":26.669},"last":0.995},{"interface":"front","average":{"1min":2.6890000000000001,"5min":1.403,"15min":1.252},"min":{"1min":0.63100000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":27.440000000000001,"5min":27.440000000000001,"15min":27.440000000000001},"last":1.022}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87592,"kb_used_data":2400,"kb_used_omap":0,"kb_used_meta":85184,"kb_avail":94284248,"statfs":{"total":96636764160,"available":96547069952,"internally_reserved":0,"allocated":2457600,"data_stored":1145859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":87228416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.60299999999999998},"min":{"1min":0.35299999999999998,"5min":0.23000000000000001,"15min":0.17399999999999999},"max":{"1min":1.0029999999999999,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.61599999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.19600000000000001},"max":{"1min":1.1120000000000001,"5min":5.0670000000000002,"15min":5.0670000000000002},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68799999999999994,"15min":0.64300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.224},"max":{"1min":1.0309999999999999,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72899999999999998,"15min":0.68400000000000005},"min":{"1min":0.59399999999999997,"5min":0.32100000000000001,"15min":0.20999999999999999},"max":{"1min":1.022,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.755}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.66300000000000003,"5min":0.34699999999999998,"15min":0.23300000000000001},"max":{"1min":1.7629999999999999,"5min":3.855,"15min":3.855},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.80100000000000005,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.318,"15min":0.255},"max":{"1min":1.4359999999999999,"5min":3.645,"15min":3.645},"last":0.88200000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.91300000000000003,"15min":0.872},"min":{"1min":0.64900000000000002,"5min":0.51000000000000001,"15min":0.40999999999999998},"max":{"1min":1.089,"5min":5.109,"15min":5.109},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.0369999999999999,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.45800000000000002,"15min":0.43099999999999999},"max":{"1min":1.1639999999999999,"5min":14.170999999999999,"15min":14.170999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.98999999999999999,"15min":0.96199999999999997},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.018,"15min":0.98399999999999999},"min":{"1min":0.79300000000000004,"5min":0.61499999999999999,"15min":0.35999999999999999},"max":{"1min":1.2110000000000001,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0389999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.69399999999999995,"5min":0.629,"15min":0.39300000000000002},"max":{"1min":1.3089999999999999,"5min":3.617,"15min":3.617},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.038,"5min":0.98899999999999999,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.5,"15min":0.47899999999999998},"max":{"1min":1.2589999999999999,"5min":3.9260000000000002,"15min":3.9260000000000002},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0549999999999999,"15min":1.0089999999999999},"min":{"1min":0.71199999999999997,"5min":0.53000000000000003,"15min":0.51100000000000001},"max":{"1min":1.3109999999999999,"5min":4.3460000000000001,"15min":4.3460000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0840000000000001,"15min":1.0409999999999999},"min":{"1min":0.872,"5min":0.56200000000000006,"15min":0.48299999999999998},"max":{"1min":1.5189999999999999,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":1.1020000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575048,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.56699999999999995,"15min":0.54200000000000004},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.17399999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.7270000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.215,"15min":0.191},"max":{"1min":1.375,"5min":1.375,"15min":1.4590000000000001},"last":1.056}]},{"osd":1,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.23000000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64600000000000002,"15min":0.61699999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22800000000000001},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72999999999999998,"15min":0.67500000000000004},"min":{"1min":0.42899999999999999,"5min":0.313,"15min":0.23200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.28000000000000003},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.4199999999999999},"last":1.0780000000000001}]},{"osd":4,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.86899999999999999,"15min":0.84699999999999998},"min":{"1min":0.76900000000000002,"5min":0.49099999999999999,"15min":0.40999999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.526},"last":0.871},{"interface":"front","average":{"1min":0.997,"5min":0.91600000000000004,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"max":{"1min":1.2889999999999999,"5min":1.343,"15min":1.5229999999999999},"last":1.147}]},{"osd":5,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.44500000000000001},"max":{"1min":1.1140000000000001,"5min":1.173,"15min":1.6200000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.046,"5min":0.92200000000000004,"15min":0.90900000000000003},"min":{"1min":0.71899999999999997,"5min":0.43099999999999999,"15min":0.35999999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5780000000000001},"last":0.96199999999999997}]},{"osd":6,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.98099999999999998,"15min":0.96899999999999997},"min":{"1min":0.753,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.615},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.036,"5min":0.94699999999999995,"15min":0.93400000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.034}]},{"osd":7,"last update":"Mon Nov 6 03:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.99099999999999999,"15min":0.96499999999999997},"min":{"1min":0.80300000000000005,"5min":0.51400000000000001,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":1.532,"15min":4.2809999999999997},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.91700000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.54200000000000004,"15min":0.47899999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":3.4740000000000002},"last":0.93200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73580,"kb_used_data":2404,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94298260,"statfs":{"total":96636764160,"available":96561418240,"internally_reserved":0,"allocated":2461696,"data_stored":1146878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.27500000000000002,"5min":0.23599999999999999,"15min":0.19},"max":{"1min":0.81200000000000006,"5min":1.042,"15min":1.843},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":1.0669999999999999,"5min":1.244,"15min":2.1150000000000002},"last":0.754}]},{"osd":2,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61099999999999999,"15min":0.68999999999999995},"min":{"1min":0.27800000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":30.861000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66400000000000003,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.20899999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.7330000000000001},"last":0.93700000000000006}]},{"osd":3,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.73999999999999999,"15min":0.69799999999999995},"min":{"1min":0.38700000000000001,"5min":0.316,"15min":0.24199999999999999},"max":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.9339999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70999999999999996,"15min":0.67700000000000005},"min":{"1min":0.34699999999999998,"5min":0.30599999999999999,"15min":0.20799999999999999},"max":{"1min":1.0169999999999999,"5min":1.3979999999999999,"15min":2.3079999999999998},"last":0.82299999999999995}]},{"osd":4,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.48099999999999998,"15min":0.434},"max":{"1min":1.1890000000000001,"5min":1.3029999999999999,"15min":2.258},"last":0.878},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89100000000000001,"15min":0.91800000000000004},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.41699999999999998},"max":{"1min":1.2,"5min":1.329,"15min":14.753},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":43.103999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0109999999999999,"15min":1.103},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.39100000000000001},"max":{"1min":1.3420000000000001,"5min":1.468,"15min":44.753999999999998},"last":1.0649999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.94399999999999995,"15min":1.004},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.45100000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":23.971},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96499999999999997,"15min":1.016},"min":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.42699999999999999},"max":{"1min":1.306,"5min":1.3979999999999999,"15min":14.813000000000001},"last":1.181}]},{"osd":7,"last update":"Mon Nov 6 03:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.034,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.52500000000000002,"15min":0.44500000000000001},"max":{"1min":1.2609999999999999,"5min":1.55,"15min":2.2799999999999998},"last":1.224},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.016,"15min":0.999},"min":{"1min":0.69199999999999995,"5min":0.58099999999999996,"15min":0.52400000000000002},"max":{"1min":1.4750000000000001,"5min":1.835,"15min":2.3559999999999999},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100460,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":97472,"kb_avail":94271380,"statfs":{"total":96636764160,"available":96533893120,"internally_reserved":0,"allocated":3051520,"data_stored":1736227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":99811328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.19600000000000001},"max":{"1min":0.996,"5min":1.214,"15min":2.6989999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.65700000000000003},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.19800000000000001},"max":{"1min":0.88600000000000001,"5min":1.45,"15min":7.1230000000000002},"last":0.76400000000000001}]},{"osd":2,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67200000000000004,"15min":0.63800000000000001},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.20599999999999999},"max":{"1min":0.89000000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.622},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.72499999999999998,"15min":0.67400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27600000000000002,"15min":0.23899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1919999999999999,"15min":4.6920000000000002},"last":0.873}]},{"osd":3,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.23100000000000001},"max":{"1min":0.81399999999999995,"5min":1.2829999999999999,"15min":2.1680000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.34399999999999997,"5min":0.32500000000000001,"15min":0.28699999999999998},"max":{"1min":1.054,"5min":1.5469999999999999,"15min":2.7999999999999998},"last":0.83799999999999997}]},{"osd":4,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.52000000000000002,"15min":0.41499999999999998},"max":{"1min":1.151,"5min":1.419,"15min":4.641},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42899999999999999},"max":{"1min":1.1120000000000001,"5min":1.4159999999999999,"15min":2.028},"last":1.0229999999999999}]},{"osd":5,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98799999999999999,"15min":0.95499999999999996},"min":{"1min":0.68500000000000005,"5min":0.57399999999999995,"15min":0.45300000000000001},"max":{"1min":1.399,"5min":2.145,"15min":2.8559999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98899999999999999,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.432},"max":{"1min":1.2949999999999999,"5min":1.623,"15min":4.7430000000000003},"last":1.3759999999999999}]},{"osd":6,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96799999999999997,"15min":0.92100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44800000000000001},"max":{"1min":1.214,"5min":1.4359999999999999,"15min":2.931},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.016,"15min":1},"min":{"1min":0.71399999999999997,"5min":0.56599999999999995,"15min":0.46500000000000002},"max":{"1min":1.4590000000000001,"5min":2.2389999999999999,"15min":2.7789999999999999},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Nov 6 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0249999999999999,"15min":0.998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.495},"max":{"1min":1.24,"5min":1.544,"15min":4.8339999999999996},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":0.97899999999999998},"min":{"1min":0.66600000000000004,"5min":0.52900000000000003,"15min":0.47799999999999998},"max":{"1min":1.2829999999999999,"5min":1.502,"15min":1.97},"last":1.0900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-06T03:20:21.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-06T03:20:21.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 Processor -- start 2023-11-06T03:20:21.186 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 -- start start 2023-11-06T03:20:21.187 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd76c1599c0 0x7fd76c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 0x7fd76c0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c15e440 con 0x7fd76c15dea0 2023-11-06T03:20:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c15adb0 con 0x7fd76c1599c0 2023-11-06T03:20:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd771d04640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd76c06c240 con 0x7fd76c06d000 2023-11-06T03:20:21.188 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd76c1599c0 0x7fd76c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd76c1599c0 0x7fd76c0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47908/0 (socket says 172.21.15.157:47908) 2023-11-06T03:20:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 -- 172.21.15.157:0/1754989603 learned_addr learned my addr 172.21.15.157:0/1754989603 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76bfff640 1 --1- 172.21.15.157:0/1754989603 >> v1:172.21.15.157:6789/0 conn(0x7fd76c15dea0 0x7fd76c070580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45512/0 (socket says 172.21.15.157:45512) 2023-11-06T03:20:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76affd640 1 --2- 172.21.15.157:0/1754989603 >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 0x7fd76c0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.189 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 -- 172.21.15.157:0/1754989603 >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 msgr2=0x7fd76c0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:21.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/1754989603 >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 0x7fd76c0b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 -- 172.21.15.157:0/1754989603 >> v1:172.21.15.157:6789/0 conn(0x7fd76c15dea0 legacy=0x7fd76c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76b7fe640 1 -- 172.21.15.157:0/1754989603 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c06c520 con 0x7fd76c1599c0 2023-11-06T03:20:21.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76a7fc640 1 -- 172.21.15.157:0/1754989603 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771740024 0 0) 0x7fd76c15e440 con 0x7fd76c15dea0 2023-11-06T03:20:21.190 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.192+0000 7fd76affd640 1 --2- 172.21.15.157:0/1754989603 >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 0x7fd76c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-06T03:20:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/1754989603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0ae000 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0cfa20 tx=0x7fd75c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f5847409c30c5c server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76a7fc640 1 -- 172.21.15.157:0/1754989603 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0ac640 con 0x7fd76c1599c0 2023-11-06T03:20:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76a7fc640 1 -- 172.21.15.157:0/1754989603 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c0ac7a0 con 0x7fd76c1599c0 2023-11-06T03:20:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76a7fc640 1 -- 172.21.15.157:0/1754989603 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0acb30 con 0x7fd76c1599c0 2023-11-06T03:20:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- 172.21.15.157:0/1754989603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 msgr2=0x7fd76c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- 172.21.15.157:0/1754989603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0ae000 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0cfa20 tx=0x7fd75c01c4d0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- 172.21.15.157:0/1754989603 shutdown_connections 2023-11-06T03:20:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- 172.21.15.157:0/1754989603 >> v2:172.21.15.195:3300/0 conn(0x7fd76c06d000 0x7fd76c0b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- 172.21.15.157:0/1754989603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0ae000 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.193 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- 172.21.15.157:0/1754989603 >> 172.21.15.157:0/1754989603 conn(0x7fd76c065930 msgr2=0x7fd76c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:21.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- 172.21.15.157:0/1754989603 shutdown_connections 2023-11-06T03:20:21.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- 172.21.15.157:0/1754989603 wait complete. 2023-11-06T03:20:21.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 Processor -- start 2023-11-06T03:20:21.194 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- start start 2023-11-06T03:20:21.197 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76c06d000 0x7fd76c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd76c15dea0 0x7fd76c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c156fa0 con 0x7fd76c1599c0 2023-11-06T03:20:21.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0cde10 con 0x7fd76c06d000 2023-11-06T03:20:21.198 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd771d04640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0ce0e0 con 0x7fd76c15dea0 2023-11-06T03:20:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47914/0 (socket says 172.21.15.157:47914) 2023-11-06T03:20:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 -- 172.21.15.157:0/288860601 learned_addr learned my addr 172.21.15.157:0/288860601 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76bfff640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd76c15dea0 0x7fd76c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.199 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76c06d000 0x7fd76c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 -- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd76c15dea0 msgr2=0x7fd76c156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd76c15dea0 0x7fd76c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 -- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76c06d000 msgr2=0x7fd76c151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76c06d000 0x7fd76c151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.200 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.196+0000 7fd76affd640 1 -- 172.21.15.157:0/288860601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0094d0 con 0x7fd76c1599c0 2023-11-06T03:20:21.201 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd76affd640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c7c0 tx=0x7fd76000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760007a30 con 0x7fd76c1599c0 2023-11-06T03:20:21.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd760007b90 con 0x7fd76c1599c0 2023-11-06T03:20:21.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0ce3c0 con 0x7fd76c1599c0 2023-11-06T03:20:21.202 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000ce70 con 0x7fd76c1599c0 2023-11-06T03:20:21.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c1571b0 con 0x7fd76c1599c0 2023-11-06T03:20:21.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c1573d0 con 0x7fd76c1599c0 2023-11-06T03:20:21.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7600113f0 con 0x7fd76c1599c0 2023-11-06T03:20:21.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 0x7fd758067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.203 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd7600834e0 con 0x7fd76c1599c0 2023-11-06T03:20:21.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 0x7fd758067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.200+0000 7fd76b7fe640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 0x7fd758067f20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0ae000 tx=0x7fd75c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.204 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.204+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fd760051070 con 0x7fd76c1599c0 2023-11-06T03:20:21.353 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.360+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd76c072750 con 0x7fd76c1599c0 2023-11-06T03:20:21.362 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.368+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd760051070 con 0x7fd76c1599c0 2023-11-06T03:20:21.363 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.368+0000 7fd753fff640 1 -- 172.21.15.157:0/288860601 <== mon.0 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v36)=0 v36) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd760054690 con 0x7fd76c1599c0 2023-11-06T03:20:21.366 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 msgr2=0x7fd758067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 0x7fd758067f20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd75c0ae000 tx=0x7fd75c0a7000 comp rx=0 tx=0).stop 2023-11-06T03:20:21.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 msgr2=0x7fd76c0cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c7c0 tx=0x7fd76000cc80 comp rx=0 tx=0).stop 2023-11-06T03:20:21.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 shutdown_connections 2023-11-06T03:20:21.367 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fd758065a60 0x7fd758067f20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd76c15dea0 0x7fd76c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd76c1599c0 0x7fd76c0cd8d0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 --2- 172.21.15.157:0/288860601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd76c06d000 0x7fd76c151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 >> 172.21.15.157:0/288860601 conn(0x7fd76c065930 msgr2=0x7fd76c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 shutdown_connections 2023-11-06T03:20:21.369 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.372+0000 7fd771d04640 1 -- 172.21.15.157:0/288860601 wait complete. 2023-11-06T03:20:21.382 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-06T03:20:21.383 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-06T03:20:21.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.492+0000 7fc4ea4ef640 1 Processor -- start 2023-11-06T03:20:21.489 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.492+0000 7fc4ea4ef640 1 -- start start 2023-11-06T03:20:21.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4ea4ef640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e41599c0 0x7fc4e40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4ea4ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e40b2af0 0x7fc4e4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4ea4ef640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e415e440 con 0x7fc4e406d000 2023-11-06T03:20:21.491 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4ea4ef640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e406c240 con 0x7fc4e41599c0 2023-11-06T03:20:21.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4ea4ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e406c520 con 0x7fc4e40b2af0 2023-11-06T03:20:21.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e41599c0 0x7fc4e40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e37fe640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fc4e406d000 0x7fc4e40b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:45518/0 (socket says 172.21.15.157:45518) 2023-11-06T03:20:21.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e41599c0 0x7fc4e40ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47916/0 (socket says 172.21.15.157:47916) 2023-11-06T03:20:21.492 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e37fe640 1 -- 172.21.15.157:0/3864847580 learned_addr learned my addr 172.21.15.157:0/3864847580 (peer_addr_for_me v1:172.21.15.157:0/0) 2023-11-06T03:20:21.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e8a8c640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e40b2af0 0x7fc4e4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 -- 172.21.15.157:0/3864847580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e40b2af0 msgr2=0x7fc4e4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-06T03:20:21.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e40b2af0 0x7fc4e4159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 -- 172.21.15.157:0/3864847580 >> v1:172.21.15.157:6789/0 conn(0x7fc4e406d000 legacy=0x7fc4e40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.493 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e2ffd640 1 -- 172.21.15.157:0/3864847580 <== mon.0 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845381125 0 0) 0x7fc4e415e440 con 0x7fc4e406d000 2023-11-06T03:20:21.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.496+0000 7fc4e3fff640 1 -- 172.21.15.157:0/3864847580 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e406c800 con 0x7fc4e41599c0 2023-11-06T03:20:21.494 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4e3fff640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e40ae000 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097c0 tx=0x7fc4dc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c428d66b9c5924 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4e2ffd640 1 -- 172.21.15.157:0/3864847580 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0ab670 con 0x7fc4e41599c0 2023-11-06T03:20:21.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4e2ffd640 1 -- 172.21.15.157:0/3864847580 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc4dc0ab7d0 con 0x7fc4e41599c0 2023-11-06T03:20:21.495 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4e2ffd640 1 -- 172.21.15.157:0/3864847580 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0abb80 con 0x7fc4e41599c0 2023-11-06T03:20:21.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3864847580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 msgr2=0x7fc4e40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.496 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e40ae000 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097c0 tx=0x7fc4dc01c450 comp rx=0 tx=0).stop 2023-11-06T03:20:21.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3864847580 shutdown_connections 2023-11-06T03:20:21.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e40b2af0 0x7fc4e4159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3864847580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e40ae000 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3864847580 >> 172.21.15.157:0/3864847580 conn(0x7fc4e4065930 msgr2=0x7fc4e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:21.497 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3864847580 shutdown_connections 2023-11-06T03:20:21.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3864847580 wait complete. 2023-11-06T03:20:21.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 Processor -- start 2023-11-06T03:20:21.498 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.500+0000 7fc4ea4ef640 1 -- start start 2023-11-06T03:20:21.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e406d000 0x7fc4e413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.500 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4e40b2af0 0x7fc4e414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e413c3f0 con 0x7fc4e41599c0 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e413c6d0 con 0x7fc4e40b2af0 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e413c9b0 con 0x7fc4e406d000 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.501 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:47924/0 (socket says 172.21.15.157:47924) 2023-11-06T03:20:21.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 -- 172.21.15.157:0/3551462276 learned_addr learned my addr 172.21.15.157:0/3551462276 (peer_addr_for_me v2:172.21.15.157:0/0) 2023-11-06T03:20:21.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e37fe640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4e40b2af0 0x7fc4e414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e3fff640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e406d000 0x7fc4e413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 -- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e406d000 msgr2=0x7fc4e413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.502 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e406d000 0x7fc4e413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 -- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4e40b2af0 msgr2=0x7fc4e414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4e40b2af0 0x7fc4e414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 -- 172.21.15.157:0/3551462276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009420 con 0x7fc4e41599c0 2023-11-06T03:20:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e8a8c640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800d5a0 tx=0x7fc4d800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.503 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80125f0 con 0x7fc4e41599c0 2023-11-06T03:20:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc4d8012770 con 0x7fc4e41599c0 2023-11-06T03:20:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8012b40 con 0x7fc4e41599c0 2023-11-06T03:20:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e413cc90 con 0x7fc4e41599c0 2023-11-06T03:20:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e413c010 con 0x7fc4e41599c0 2023-11-06T03:20:21.504 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e4072750 con 0x7fc4e41599c0 2023-11-06T03:20:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4d8012ca0 con 0x7fc4e41599c0 2023-11-06T03:20:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 0x7fc4d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-06T03:20:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc4d8083590 con 0x7fc4e41599c0 2023-11-06T03:20:21.505 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.504+0000 7fc4e3fff640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 0x7fc4d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-06T03:20:21.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.508+0000 7fc4e3fff640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 0x7fc4d0067f20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0af000 tx=0x7fc4dc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-06T03:20:21.506 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.508+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172122 (secure 0 0 0) 0x7fc4d8051000 con 0x7fc4e41599c0 2023-11-06T03:20:21.677 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.680+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4e413d140 con 0x7fc4e41599c0 2023-11-06T03:20:21.678 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4e0ff9640 1 -- 172.21.15.157:0/3551462276 <== mon.0 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+3546 (secure 0 0 0) 0x7fc4d8054620 con 0x7fc4e41599c0 2023-11-06T03:20:21.678 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-11-06T03:20:21.679 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":138,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6836/215517753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":215517753},{"type":"v1","addr":"172.21.15.195:6836","nonce":215517753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/2117812838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2117812838},{"type":"v1","addr":"172.21.15.157:6833","nonce":2117812838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/4186040628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6835","nonce":4186040628},{"type":"v1","addr":"172.21.15.195:6837","nonce":4186040628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":11781,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3751732785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3751732785},{"type":"v1","addr":"172.21.15.157:6835","nonce":3751732785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":138}],"filesystems":[]} 2023-11-06T03:20:21.679 INFO:teuthology.orchestra.run.smithi157.stderr:dumped fsmap epoch 138 2023-11-06T03:20:21.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 msgr2=0x7fc4d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.680 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 0x7fc4d0067f20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0af000 tx=0x7fc4dc0a7000 comp rx=0 tx=0).stop 2023-11-06T03:20:21.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 msgr2=0x7fc4e413d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-06T03:20:21.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800d5a0 tx=0x7fc4d800da60 comp rx=0 tx=0).stop 2023-11-06T03:20:21.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 shutdown_connections 2023-11-06T03:20:21.681 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:6832/38105,v1:172.21.15.195:6833/38105] conn(0x7fc4d0065a60 0x7fc4d0067f20 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e41599c0 0x7fc4e413d490 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc4e40b2af0 0x7fc4e414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 --2- 172.21.15.157:0/3551462276 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc4e406d000 0x7fc4e413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-06T03:20:21.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 >> 172.21.15.157:0/3551462276 conn(0x7fc4e4065930 msgr2=0x7fc4e415b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-06T03:20:21.682 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 shutdown_connections 2023-11-06T03:20:21.683 INFO:teuthology.orchestra.run.smithi157.stderr:2023-11-06T03:20:21.684+0000 7fc4ea4ef640 1 -- 172.21.15.157:0/3551462276 wait complete. 2023-11-06T03:20:21.696 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-06T03:20:21.696 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-06T03:20:21.697 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-06T03:20:21.697 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:21.706 INFO:tasks.ceph.mds.a:Stopped 2023-11-06T03:20:21.707 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-06T03:20:21.707 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:21.758 INFO:tasks.ceph.mds.c:Stopped 2023-11-06T03:20:21.759 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-06T03:20:21.759 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:21.772 INFO:tasks.ceph.mds.b:Stopped 2023-11-06T03:20:21.773 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-06T03:20:21.773 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:21.828 INFO:tasks.ceph.mds.d:Stopped 2023-11-06T03:20:21.829 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-06T03:20:21.829 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-06T03:20:21.829 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:21.916 INFO:tasks.ceph.osd.0:Stopped 2023-11-06T03:20:21.917 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-06T03:20:21.917 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:28.019 INFO:tasks.ceph.osd.1:Stopped 2023-11-06T03:20:28.020 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-06T03:20:28.020 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:28.109 INFO:tasks.ceph.osd.2:Stopped 2023-11-06T03:20:28.110 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-06T03:20:28.110 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:29.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-11-06T03:20:34.211 INFO:tasks.ceph.osd.3:Stopped 2023-11-06T03:20:34.212 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-06T03:20:34.213 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:34.294 INFO:tasks.ceph.osd.4:Stopped 2023-11-06T03:20:34.295 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-06T03:20:34.295 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:34.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-06T03:20:34.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-11-06T03:20:40.397 INFO:tasks.ceph.osd.5:Stopped 2023-11-06T03:20:40.398 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-06T03:20:40.398 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:40.490 INFO:tasks.ceph.osd.6:Stopped 2023-11-06T03:20:40.491 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-06T03:20:40.491 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:40.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-06T03:20:40.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-06T03:20:43.300 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:43.303 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:46.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-06T03:20:46.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-06T03:20:46.594 INFO:tasks.ceph.osd.7:Stopped 2023-11-06T03:20:46.594 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-06T03:20:46.596 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-06T03:20:46.596 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:46.624 INFO:tasks.ceph.mgr.y:Stopped 2023-11-06T03:20:46.625 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-06T03:20:46.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:46.655 INFO:tasks.ceph.mgr.x:Stopped 2023-11-06T03:20:46.656 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-06T03:20:46.656 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-06T03:20:46.657 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:46.681 INFO:tasks.ceph.mon.a:Stopped 2023-11-06T03:20:46.682 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-06T03:20:46.682 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:46.718 INFO:tasks.ceph.mon.b:Stopped 2023-11-06T03:20:46.718 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-06T03:20:46.719 INFO:teuthology.orchestra.run:waiting for 300 2023-11-06T03:20:46.774 INFO:tasks.ceph.mon.c:Stopped 2023-11-06T03:20:46.774 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-06T03:20:46.775 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-11-06T03:20:46.794 INFO:teuthology.orchestra.run.smithi157.stdout:2023-11-06T03:07:39.414197+0000 mon.a (mon.0) 485 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T03:20:46.795 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-06T03:20:46.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-11-06T03:20:46.852 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-11-06T03:20:46.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-11-06T03:20:46.965 INFO:teuthology.orchestra.run.smithi157.stdout:2023-11-06T03:07:39.414197+0000 mon.a (mon.0) 485 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-11-06T03:20:46.965 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:20:46.966 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-06T03:20:47.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:20:47.239 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-06T03:20:47.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:20:47.450 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-06T03:20:47.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:20:47.625 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-06T03:20:47.810 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:20:47.811 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-06T03:20:48.080 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:20:48.081 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-06T03:20:48.335 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:20:48.335 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-06T03:20:48.505 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:20:48.505 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-06T03:20:48.661 INFO:tasks.ceph:Archiving mon data... 2023-11-06T03:20:48.663 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/data/mon.a.tgz 2023-11-06T03:20:48.663 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-11-06T03:20:48.669 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.GO233qVPA5 2023-11-06T03:20:48.669 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.GO233qVPA5 2023-11-06T03:20:48.815 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.GO233qVPA5 2023-11-06T03:20:48.878 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.GO233qVPA5 is 428KB 2023-11-06T03:20:48.940 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.GO233qVPA5 2023-11-06T03:20:48.946 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/data/mon.b.tgz 2023-11-06T03:20:48.946 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2023-11-06T03:20:48.952 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.BTrsIhK14W 2023-11-06T03:20:48.953 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BTrsIhK14W 2023-11-06T03:20:49.093 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.BTrsIhK14W 2023-11-06T03:20:49.161 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.BTrsIhK14W is 429KB 2023-11-06T03:20:49.223 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.BTrsIhK14W 2023-11-06T03:20:49.230 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/data/mon.c.tgz 2023-11-06T03:20:49.230 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2023-11-06T03:20:49.276 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.vzn0ZwkhCC 2023-11-06T03:20:49.277 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vzn0ZwkhCC 2023-11-06T03:20:49.415 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.vzn0ZwkhCC 2023-11-06T03:20:49.480 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.vzn0ZwkhCC is 429KB 2023-11-06T03:20:49.539 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.vzn0ZwkhCC 2023-11-06T03:20:49.544 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-06T03:20:49.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T03:20:49.547 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-06T03:20:49.603 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-06T03:20:49.604 DEBUG:teuthology.orchestra.run.smithi157:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T03:20:49.606 DEBUG:teuthology.orchestra.run.smithi195:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-06T03:20:49.628 INFO:teuthology.orchestra.run.smithi157.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T03:20:49.679 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-06T03:20:49.684 INFO:tasks.ceph:Archiving crash dumps... 2023-11-06T03:20:49.686 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi157/crash 2023-11-06T03:20:49.687 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T03:20:49.699 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi195/crash 2023-11-06T03:20:49.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-06T03:20:49.742 INFO:tasks.ceph:Shutting down logrotate 2023-11-06T03:20:49.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:49.746 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:49.828 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:49.840 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-06T03:20:49.883 INFO:tasks.ceph:Compressing logs... 2023-11-06T03:20:49.883 DEBUG:teuthology.orchestra.run.smithi157:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T03:20:49.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T03:20:51.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-06T03:22:19.509 INFO:tasks.ceph:Archiving logs... 2023-11-06T03:22:19.509 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi157/log 2023-11-06T03:22:19.511 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T03:22:29.847 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi195/log 2023-11-06T03:22:29.848 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-06T03:22:31.236 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-06T03:22:31.245 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-06T03:22:31.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T03:22:31.249 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-06T03:22:31.318 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-06T03:22:31.327 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-06T03:22:31.327 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T03:22:31.330 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-06T03:22:31.342 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T03:22:31.343 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2023-11-06T03:22:31.343 INFO:teuthology.orchestra.run.smithi157.stdout:*hv01.front.sepi 44.4.53.2 2 u 7 64 377 0.100 -4.099 1.663 2023-11-06T03:22:31.343 INFO:teuthology.orchestra.run.smithi157.stdout:+hv02.front.sepi 63.231.80.2 3 u 3 64 377 0.091 -4.456 1.012 2023-11-06T03:22:31.343 INFO:teuthology.orchestra.run.smithi157.stdout:+hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.062 -3.643 1.313 2023-11-06T03:22:31.344 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-11-06T03:22:31.347 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2023-11-06T03:22:31.347 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2023-11-06T03:22:31.347 INFO:teuthology.orchestra.run.smithi195.stdout:*hv01.front.sepi 44.4.53.2 2 u 22 64 377 0.098 -5.407 2.332 2023-11-06T03:22:31.347 INFO:teuthology.orchestra.run.smithi195.stdout:+hv02.front.sepi 63.231.80.2 3 u 1 64 377 0.125 -4.888 1.609 2023-11-06T03:22:31.347 INFO:teuthology.orchestra.run.smithi195.stdout:+hv03.front.sepi 74.6.168.72 3 u 6 64 377 0.086 -4.531 2.497 2023-11-06T03:22:31.348 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2023-11-06T03:22:31.349 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-06T03:22:31.358 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-06T03:22:31.358 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-06T03:22:31.381 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-06T03:22:31.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-06T03:22:31.481 INFO:teuthology.task.internal:Duration was 1794.542470 seconds 2023-11-06T03:22:31.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-06T03:22:31.515 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-06T03:22:31.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T03:22:31.518 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-06T03:22:31.596 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-06T03:22:31.597 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2023-11-06T03:22:31.597 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T03:22:31.607 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2023-11-06T03:22:31.608 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-06T03:22:31.618 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-06T03:22:31.618 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T03:22:31.651 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-06T03:22:31.685 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-06T03:22:31.685 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T03:22:31.726 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-06T03:22:31.809 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-06T03:22:31.820 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-06T03:22:31.820 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T03:22:31.832 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-06T03:22:31.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-06T03:22:31.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T03:22:31.879 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-06T03:22:31.888 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2023-11-06T03:22:31.896 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2023-11-06T03:22:31.909 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T03:22:31.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:22:31.950 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-06T03:22:31.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-06T03:22:31.958 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-06T03:22:31.969 INFO:teuthology.task.internal:Transferring archived files... 2023-11-06T03:22:31.971 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi157 2023-11-06T03:22:31.972 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T03:22:32.013 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738/remote/smithi195 2023-11-06T03:22:32.013 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-06T03:22:32.033 INFO:teuthology.task.internal:Removing archive directory... 2023-11-06T03:22:32.034 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T03:22:32.050 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-06T03:22:32.083 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-06T03:22:32.094 INFO:teuthology.task.internal:Not uploading archives. 2023-11-06T03:22:32.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-06T03:22:32.116 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-06T03:22:32.116 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T03:22:32.118 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-06T03:22:32.121 INFO:teuthology.orchestra.run.smithi157.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 03:22 /home/ubuntu/cephtest 2023-11-06T03:22:32.131 INFO:teuthology.orchestra.run.smithi195.stdout: 655470 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 6 03:22 /home/ubuntu/cephtest 2023-11-06T03:22:32.133 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-06T03:22:32.149 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-06T03:22:32.216 INFO:teuthology.nuke:Checking targets against current locks 2023-11-06T03:22:32.240 DEBUG:teuthology.nuke:shortname: smithi157 2023-11-06T03:22:32.240 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T03:22:32.261 DEBUG:teuthology.nuke:shortname: smithi195 2023-11-06T03:22:32.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-06T03:22:32.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 02:42:54.301321', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgeOL5FtyrcN1GMmQwuIlZ0j86FC3xLAkooaUK1Flxytgx8S3U7T8Dnh0JRIiIgs6RG3WxlxL9T39UpfPzOnXQ='} 2023-11-06T03:22:32.291 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-05_23:44:49-fs-reef-release-distro-default-smithi/7448738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-06 02:42:54.303365', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAx7yMHLyHmaa5GecES0RDKKy6Zo72ZjhkX6pGRAgXz1f3EtApBvRPCC+Lteqqy4ps5/3/yIcm1XXP7qnsP8+jA='} 2023-11-06T03:22:32.317 INFO:teuthology.orchestra.console.smithi157:Power off 2023-11-06T03:22:32.318 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T03:22:32.343 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-11-06T03:22:32.344 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:32.361 INFO:teuthology.orchestra.console.smithi195:Power off 2023-11-06T03:22:32.362 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-06T03:22:32.372 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-11-06T03:22:32.384 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-11-06T03:22:32.385 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:32.409 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-11-06T03:22:36.373 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:36.410 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:36.484 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-11-06T03:22:36.521 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-11-06T03:22:40.489 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:40.522 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-06T03:22:40.600 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-11-06T03:22:40.636 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-06T03:22:40.701 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-11-06T03:22:40.737 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-11-06T03:22:40.842 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} duration: 1794.5424699783325 failure_reason: '"2023-11-06T03:07:39.414197+0000 mon.a (mon.0) 485 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-11-06T03:22:40.842 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-06T03:22:40.932 INFO:teuthology.run:FAIL